2024-09-13T17:37:00.630 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T17:37:00.655 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T17:37:00.755 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903931 branch: wip-rishabh-testing-20240910.100730-reef description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} email: null first_in_suite: false job_id: '7903931' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-rishabh-testing-20240910.100730-reef ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds osd ms inject socket failures: 2500 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon client directed command retry: 5 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds ms inject socket failures: 2500 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - overall HEALTH_ - \(MON_DOWN\) - SLOW_OPS - slow request sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240910.100730-reef sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 6439 sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 sleep_before_teardown: 0 subset: 964/1024 suite: fs suite_branch: wip-rishabh-testing-20240910.100730-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - mon_thrash: check_mds_failover: true revive_delay: 20 thrash_delay: 10 - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-13_14:42:16 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T17:37:00.756 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa; will attempt to use it 2024-09-13T17:37:00.756 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa/tasks 2024-09-13T17:37:00.757 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T17:37:00.758 INFO:teuthology.task.internal:Checking packages... 2024-09-13T17:37:00.780 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3' 2024-09-13T17:37:00.781 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T17:37:00.781 INFO:teuthology.packaging:ref: None 2024-09-13T17:37:00.781 INFO:teuthology.packaging:tag: None 2024-09-13T17:37:00.781 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240910.100730-reef 2024-09-13T17:37:00.781 INFO:teuthology.packaging:sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:37:00.781 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-rishabh-testing-20240910.100730-reef 2024-09-13T17:37:00.927 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-860-g6c6b083f-1focal 2024-09-13T17:37:00.928 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T17:37:00.935 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T17:37:00.935 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T17:37:01.011 INFO:teuthology.task.internal:Saving configuration 2024-09-13T17:37:01.029 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T17:37:01.089 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T17:37:01.123 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903931', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 17:31:10.702929', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-13T17:37:01.156 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903931', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 17:31:10.704890', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-13T17:37:01.157 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T17:37:01.163 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-09-13T17:37:01.163 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-09-13T17:37:01.163 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T17:37:01.243 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4dd915bbe0>, signals=[15]) 2024-09-13T17:37:01.243 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T17:37:01.249 INFO:teuthology.task.internal:Opening connections... 2024-09-13T17:37:01.249 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi040.front.sepia.ceph.com 2024-09-13T17:37:01.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:37:01.363 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2024-09-13T17:37:01.364 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:37:01.475 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T17:37:01.583 DEBUG:teuthology.orchestra.run.smithi040:> uname -m 2024-09-13T17:37:01.590 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2024-09-13T17:37:01.591 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2024-09-13T17:37:01.639 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="Ubuntu" 2024-09-13T17:37:01.640 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-13T17:37:01.640 INFO:teuthology.orchestra.run.smithi040.stdout:ID=ubuntu 2024-09-13T17:37:01.640 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE=debian 2024-09-13T17:37:01.640 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-13T17:37:01.640 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="20.04" 2024-09-13T17:37:01.640 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T17:37:01.640 INFO:teuthology.orchestra.run.smithi040.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T17:37:01.640 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T17:37:01.640 INFO:teuthology.orchestra.run.smithi040.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T17:37:01.640 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_CODENAME=focal 2024-09-13T17:37:01.641 INFO:teuthology.orchestra.run.smithi040.stdout:UBUNTU_CODENAME=focal 2024-09-13T17:37:01.641 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2024-09-13T17:37:01.672 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2024-09-13T17:37:01.679 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2024-09-13T17:37:01.679 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2024-09-13T17:37:01.730 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="Ubuntu" 2024-09-13T17:37:01.731 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-13T17:37:01.731 INFO:teuthology.orchestra.run.smithi043.stdout:ID=ubuntu 2024-09-13T17:37:01.731 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE=debian 2024-09-13T17:37:01.731 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-13T17:37:01.731 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="20.04" 2024-09-13T17:37:01.731 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T17:37:01.731 INFO:teuthology.orchestra.run.smithi043.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T17:37:01.731 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T17:37:01.731 INFO:teuthology.orchestra.run.smithi043.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T17:37:01.731 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_CODENAME=focal 2024-09-13T17:37:01.731 INFO:teuthology.orchestra.run.smithi043.stdout:UBUNTU_CODENAME=focal 2024-09-13T17:37:01.732 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2024-09-13T17:37:01.780 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T17:37:01.858 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T17:37:01.865 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T17:37:01.866 DEBUG:teuthology.orchestra.run.smithi040:> test '!' -e /home/ubuntu/cephtest 2024-09-13T17:37:01.869 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2024-09-13T17:37:01.876 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T17:37:01.881 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T17:37:01.881 DEBUG:teuthology.orchestra.run.smithi040:> test -z $(ls -A /var/lib/ceph) 2024-09-13T17:37:01.917 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2024-09-13T17:37:01.939 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T17:37:02.013 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T17:37:02.029 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T17:37:02.029 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T17:37:02.030 DEBUG:teuthology.orchestra.run.smithi040:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T17:37:02.030 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T17:37:02.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:37:02.036 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-09-13T17:37:02.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:37:02.037 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-09-13T17:37:02.043 INFO:teuthology.orchestra.run.smithi040.stdout:5.4.0-124-generic 2024-09-13T17:37:02.043 INFO:teuthology.task.kernel:Running kernel on smithi040: 5.4.0-124-generic 2024-09-13T17:37:02.043 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get clean 2024-09-13T17:37:02.044 INFO:teuthology.orchestra.run.smithi043.stdout:5.4.0-124-generic 2024-09-13T17:37:02.044 INFO:teuthology.task.kernel:Running kernel on smithi043: 5.4.0-124-generic 2024-09-13T17:37:02.044 DEBUG:teuthology.orchestra.run.smithi043:> sudo apt-get clean 2024-09-13T17:37:02.142 DEBUG:teuthology.orchestra.run.smithi043:> sudo apt-get update 2024-09-13T17:37:02.193 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get update 2024-09-13T17:37:02.321 INFO:teuthology.orchestra.run.smithi043.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-13T17:37:02.351 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T17:37:02.363 INFO:teuthology.orchestra.run.smithi040.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-13T17:37:02.421 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T17:37:02.427 INFO:teuthology.orchestra.run.smithi040.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-13T17:37:02.461 INFO:teuthology.orchestra.run.smithi040.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-13T17:37:02.503 INFO:teuthology.orchestra.run.smithi043.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-13T17:37:02.622 INFO:teuthology.orchestra.run.smithi043.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-13T17:37:02.715 INFO:teuthology.orchestra.run.smithi043.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-13T17:37:02.741 INFO:teuthology.orchestra.run.smithi040.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-13T17:37:02.758 INFO:teuthology.orchestra.run.smithi043.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-13T17:37:02.761 INFO:teuthology.orchestra.run.smithi043.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-13T17:37:02.761 INFO:teuthology.orchestra.run.smithi043.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-13T17:37:02.782 INFO:teuthology.orchestra.run.smithi043.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-13T17:37:02.782 INFO:teuthology.orchestra.run.smithi043.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-13T17:37:02.785 INFO:teuthology.orchestra.run.smithi043.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T17:37:02.785 INFO:teuthology.orchestra.run.smithi043.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-13T17:37:02.791 INFO:teuthology.orchestra.run.smithi043.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-13T17:37:02.798 INFO:teuthology.orchestra.run.smithi043.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-13T17:37:02.800 INFO:teuthology.orchestra.run.smithi043.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-13T17:37:02.800 INFO:teuthology.orchestra.run.smithi043.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-13T17:37:02.801 INFO:teuthology.orchestra.run.smithi043.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-13T17:37:02.811 INFO:teuthology.orchestra.run.smithi043.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-13T17:37:02.812 INFO:teuthology.orchestra.run.smithi043.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-13T17:37:02.857 INFO:teuthology.orchestra.run.smithi043.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-13T17:37:02.884 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-13T17:37:02.897 INFO:teuthology.orchestra.run.smithi040.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-13T17:37:02.903 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-13T17:37:02.903 INFO:teuthology.orchestra.run.smithi040.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-13T17:37:02.904 INFO:teuthology.orchestra.run.smithi040.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-13T17:37:02.927 INFO:teuthology.orchestra.run.smithi040.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-13T17:37:02.930 INFO:teuthology.orchestra.run.smithi040.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T17:37:02.930 INFO:teuthology.orchestra.run.smithi040.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,230 kB] 2024-09-13T17:37:02.938 INFO:teuthology.orchestra.run.smithi040.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-13T17:37:02.944 INFO:teuthology.orchestra.run.smithi040.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-13T17:37:02.946 INFO:teuthology.orchestra.run.smithi040.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-13T17:37:02.946 INFO:teuthology.orchestra.run.smithi040.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-13T17:37:02.947 INFO:teuthology.orchestra.run.smithi040.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-13T17:37:02.947 INFO:teuthology.orchestra.run.smithi040.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-13T17:37:02.950 INFO:teuthology.orchestra.run.smithi040.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-13T17:37:02.951 INFO:teuthology.orchestra.run.smithi040.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-13T17:37:02.951 INFO:teuthology.orchestra.run.smithi040.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-13T17:37:02.967 INFO:teuthology.orchestra.run.smithi040.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-13T17:37:02.968 INFO:teuthology.orchestra.run.smithi040.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-13T17:37:02.968 INFO:teuthology.orchestra.run.smithi040.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-13T17:37:02.969 INFO:teuthology.orchestra.run.smithi040.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-13T17:37:02.997 INFO:teuthology.orchestra.run.smithi043.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-13T17:37:03.080 INFO:teuthology.orchestra.run.smithi040.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-13T17:37:03.243 INFO:teuthology.orchestra.run.smithi043.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-13T17:37:03.418 INFO:teuthology.orchestra.run.smithi040.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-13T17:37:03.447 INFO:teuthology.orchestra.run.smithi043.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-13T17:37:03.464 INFO:teuthology.orchestra.run.smithi043.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-13T17:37:03.464 INFO:teuthology.orchestra.run.smithi043.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-13T17:37:03.465 INFO:teuthology.orchestra.run.smithi043.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-13T17:37:03.563 INFO:teuthology.orchestra.run.smithi043.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-13T17:37:03.576 INFO:teuthology.orchestra.run.smithi043.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T17:37:03.577 INFO:teuthology.orchestra.run.smithi043.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-13T17:37:03.594 INFO:teuthology.orchestra.run.smithi043.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,230 kB] 2024-09-13T17:37:03.610 INFO:teuthology.orchestra.run.smithi040.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-13T17:37:03.625 INFO:teuthology.orchestra.run.smithi040.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-13T17:37:03.625 INFO:teuthology.orchestra.run.smithi040.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-13T17:37:03.630 INFO:teuthology.orchestra.run.smithi043.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-13T17:37:03.640 INFO:teuthology.orchestra.run.smithi043.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-13T17:37:03.640 INFO:teuthology.orchestra.run.smithi043.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-13T17:37:03.641 INFO:teuthology.orchestra.run.smithi043.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-13T17:37:03.645 INFO:teuthology.orchestra.run.smithi043.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-13T17:37:03.698 INFO:teuthology.orchestra.run.smithi043.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-13T17:37:03.698 INFO:teuthology.orchestra.run.smithi043.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-13T17:37:03.699 INFO:teuthology.orchestra.run.smithi043.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-13T17:37:03.700 INFO:teuthology.orchestra.run.smithi043.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-13T17:37:03.714 INFO:teuthology.orchestra.run.smithi040.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-13T17:37:03.719 INFO:teuthology.orchestra.run.smithi040.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-13T17:37:03.729 INFO:teuthology.orchestra.run.smithi040.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T17:37:03.729 INFO:teuthology.orchestra.run.smithi040.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-13T17:37:03.749 INFO:teuthology.orchestra.run.smithi040.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-13T17:37:03.779 INFO:teuthology.orchestra.run.smithi040.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-13T17:37:03.787 INFO:teuthology.orchestra.run.smithi040.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-13T17:37:03.787 INFO:teuthology.orchestra.run.smithi040.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-13T17:37:03.787 INFO:teuthology.orchestra.run.smithi040.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-13T17:37:03.859 INFO:teuthology.orchestra.run.smithi040.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-13T17:37:03.859 INFO:teuthology.orchestra.run.smithi040.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-13T17:37:03.952 INFO:teuthology.orchestra.run.smithi043.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-13T17:37:03.952 INFO:teuthology.orchestra.run.smithi043.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-13T17:37:03.953 INFO:teuthology.orchestra.run.smithi043.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-13T17:37:07.292 INFO:teuthology.orchestra.run.smithi043.stdout:Fetched 21.8 MB in 3s (7,079 kB/s) 2024-09-13T17:37:07.515 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 21.8 MB in 3s (6,846 kB/s) 2024-09-13T17:37:09.079 INFO:teuthology.orchestra.run.smithi043.stdout:Reading package lists... 2024-09-13T17:37:09.107 DEBUG:teuthology.orchestra.run.smithi043:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T17:37:09.114 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-09-13T17:37:09.141 DEBUG:teuthology.orchestra.run.smithi040:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T17:37:09.195 INFO:teuthology.orchestra.run.smithi043.stdout:Reading package lists... 2024-09-13T17:37:09.214 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-09-13T17:37:09.421 INFO:teuthology.orchestra.run.smithi043.stdout:Building dependency tree... 2024-09-13T17:37:09.421 INFO:teuthology.orchestra.run.smithi043.stdout:Reading state information... 2024-09-13T17:37:09.437 INFO:teuthology.orchestra.run.smithi040.stdout:Building dependency tree... 2024-09-13T17:37:09.437 INFO:teuthology.orchestra.run.smithi040.stdout:Reading state information... 2024-09-13T17:37:09.650 INFO:teuthology.orchestra.run.smithi043.stdout:The following additional packages will be installed: 2024-09-13T17:37:09.650 INFO:teuthology.orchestra.run.smithi043.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T17:37:09.650 INFO:teuthology.orchestra.run.smithi043.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-13T17:37:09.650 INFO:teuthology.orchestra.run.smithi043.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-13T17:37:09.651 INFO:teuthology.orchestra.run.smithi043.stdout:Suggested packages: 2024-09-13T17:37:09.651 INFO:teuthology.orchestra.run.smithi043.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-13T17:37:09.651 INFO:teuthology.orchestra.run.smithi043.stdout:Recommended packages: 2024-09-13T17:37:09.651 INFO:teuthology.orchestra.run.smithi043.stdout: thermald 2024-09-13T17:37:09.652 INFO:teuthology.orchestra.run.smithi040.stdout:The following additional packages will be installed: 2024-09-13T17:37:09.652 INFO:teuthology.orchestra.run.smithi040.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T17:37:09.652 INFO:teuthology.orchestra.run.smithi040.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-13T17:37:09.653 INFO:teuthology.orchestra.run.smithi040.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-13T17:37:09.654 INFO:teuthology.orchestra.run.smithi040.stdout:Suggested packages: 2024-09-13T17:37:09.654 INFO:teuthology.orchestra.run.smithi040.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-13T17:37:09.654 INFO:teuthology.orchestra.run.smithi040.stdout:Recommended packages: 2024-09-13T17:37:09.654 INFO:teuthology.orchestra.run.smithi040.stdout: thermald 2024-09-13T17:37:09.710 INFO:teuthology.orchestra.run.smithi043.stdout:The following NEW packages will be installed: 2024-09-13T17:37:09.710 INFO:teuthology.orchestra.run.smithi043.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T17:37:09.710 INFO:teuthology.orchestra.run.smithi043.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-13T17:37:09.710 INFO:teuthology.orchestra.run.smithi043.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-13T17:37:09.711 INFO:teuthology.orchestra.run.smithi043.stdout:The following packages will be upgraded: 2024-09-13T17:37:09.712 INFO:teuthology.orchestra.run.smithi043.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T17:37:09.715 INFO:teuthology.orchestra.run.smithi040.stdout:The following NEW packages will be installed: 2024-09-13T17:37:09.715 INFO:teuthology.orchestra.run.smithi040.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T17:37:09.715 INFO:teuthology.orchestra.run.smithi040.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-13T17:37:09.715 INFO:teuthology.orchestra.run.smithi040.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-13T17:37:09.717 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages will be upgraded: 2024-09-13T17:37:09.717 INFO:teuthology.orchestra.run.smithi040.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T17:37:09.898 INFO:teuthology.orchestra.run.smithi043.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T17:37:09.898 INFO:teuthology.orchestra.run.smithi043.stdout:Need to get 77.2 MB of archives. 2024-09-13T17:37:09.898 INFO:teuthology.orchestra.run.smithi043.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-13T17:37:09.898 INFO:teuthology.orchestra.run.smithi043.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-13T17:37:09.901 INFO:teuthology.orchestra.run.smithi040.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T17:37:09.901 INFO:teuthology.orchestra.run.smithi040.stdout:Need to get 77.2 MB of archives. 2024-09-13T17:37:09.901 INFO:teuthology.orchestra.run.smithi040.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-13T17:37:09.901 INFO:teuthology.orchestra.run.smithi040.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-13T17:37:11.044 INFO:teuthology.orchestra.run.smithi043.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-13T17:37:11.074 INFO:teuthology.orchestra.run.smithi040.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-13T17:37:11.315 INFO:teuthology.orchestra.run.smithi043.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-13T17:37:11.395 INFO:teuthology.orchestra.run.smithi040.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-13T17:37:12.619 INFO:teuthology.orchestra.run.smithi043.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-13T17:37:12.620 INFO:teuthology.orchestra.run.smithi043.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-13T17:37:12.620 INFO:teuthology.orchestra.run.smithi043.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-13T17:37:12.705 INFO:teuthology.orchestra.run.smithi040.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-13T17:37:12.706 INFO:teuthology.orchestra.run.smithi040.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-13T17:37:12.707 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-13T17:37:12.901 INFO:teuthology.orchestra.run.smithi043.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-13T17:37:12.916 INFO:teuthology.orchestra.run.smithi043.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-13T17:37:12.982 INFO:teuthology.orchestra.run.smithi040.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-13T17:37:13.034 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-13T17:37:13.629 INFO:teuthology.orchestra.run.smithi043.stdout:Fetched 77.2 MB in 3s (24.1 MB/s) 2024-09-13T17:37:13.738 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 77.2 MB in 3s (23.3 MB/s) 2024-09-13T17:37:13.797 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-13T17:37:13.921 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-13T17:37:15.543 INFO:teuthology.orchestra.run.smithi043.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-13T17:37:15.548 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T17:37:15.660 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-13T17:37:15.666 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T17:37:15.694 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T17:37:15.827 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T17:37:17.636 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-13T17:37:17.653 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T17:37:17.784 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T17:37:17.911 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-13T17:37:17.925 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T17:37:18.026 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T17:37:18.930 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-13T17:37:18.944 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T17:37:18.988 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T17:37:19.189 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-13T17:37:19.202 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T17:37:19.239 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T17:37:23.908 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T17:37:24.052 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T17:37:24.401 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T17:37:24.546 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T17:37:24.549 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T17:37:24.695 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T17:37:24.892 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-13T17:37:24.907 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-13T17:37:24.949 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T17:37:25.056 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T17:37:25.174 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T17:37:25.527 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-13T17:37:25.541 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-13T17:37:25.585 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T17:37:31.226 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-13T17:37:31.230 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-13T17:37:31.248 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T17:37:31.253 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T17:37:31.296 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T17:37:31.309 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T17:37:34.157 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T17:37:34.302 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T17:37:34.356 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T17:37:34.478 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T17:37:34.637 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T17:37:34.854 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T17:37:36.095 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T17:37:36.760 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T17:37:37.279 INFO:teuthology.orchestra.run.smithi040.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-13T17:37:37.279 INFO:teuthology.orchestra.run.smithi040.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-13T17:37:37.438 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T17:37:38.010 INFO:teuthology.orchestra.run.smithi043.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-13T17:37:38.011 INFO:teuthology.orchestra.run.smithi043.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-13T17:37:38.153 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T17:37:39.413 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T17:37:39.539 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-13T17:37:39.621 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T17:37:39.688 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T17:37:39.772 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-13T17:37:39.841 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-13T17:37:39.882 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T17:37:39.992 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-13T17:37:40.041 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-13T17:37:40.119 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T17:37:40.184 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-13T17:37:40.193 INFO:teuthology.orchestra.run.smithi043.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T17:37:40.193 INFO:teuthology.orchestra.run.smithi043.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-13T17:37:40.319 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T17:37:40.416 INFO:teuthology.orchestra.run.smithi040.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T17:37:40.416 INFO:teuthology.orchestra.run.smithi040.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-13T17:38:02.696 INFO:teuthology.orchestra.run.smithi043.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T17:38:02.697 INFO:teuthology.orchestra.run.smithi043.stdout:Sourcing file `/etc/default/grub' 2024-09-13T17:38:02.709 INFO:teuthology.orchestra.run.smithi043.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T17:38:02.728 INFO:teuthology.orchestra.run.smithi043.stdout:Generating grub configuration file ... 2024-09-13T17:38:03.123 INFO:teuthology.orchestra.run.smithi040.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T17:38:03.123 INFO:teuthology.orchestra.run.smithi040.stdout:Sourcing file `/etc/default/grub' 2024-09-13T17:38:03.140 INFO:teuthology.orchestra.run.smithi040.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T17:38:03.162 INFO:teuthology.orchestra.run.smithi040.stdout:Generating grub configuration file ... 2024-09-13T17:38:03.177 INFO:teuthology.orchestra.run.smithi043.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T17:38:03.203 INFO:teuthology.orchestra.run.smithi043.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T17:38:03.589 INFO:teuthology.orchestra.run.smithi040.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T17:38:03.614 INFO:teuthology.orchestra.run.smithi040.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T17:38:03.711 INFO:teuthology.orchestra.run.smithi043.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T17:38:03.722 INFO:teuthology.orchestra.run.smithi043.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T17:38:04.122 INFO:teuthology.orchestra.run.smithi040.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T17:38:04.133 INFO:teuthology.orchestra.run.smithi040.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T17:38:05.746 INFO:teuthology.orchestra.run.smithi043.stdout:done 2024-09-13T17:38:06.227 DEBUG:teuthology.orchestra.run.smithi043:> dpkg -s linux-image-generic 2024-09-13T17:38:06.305 INFO:teuthology.orchestra.run.smithi043.stdout:Package: linux-image-generic 2024-09-13T17:38:06.305 INFO:teuthology.orchestra.run.smithi043.stdout:Status: install ok installed 2024-09-13T17:38:06.306 INFO:teuthology.orchestra.run.smithi043.stdout:Priority: optional 2024-09-13T17:38:06.306 INFO:teuthology.orchestra.run.smithi043.stdout:Section: kernel 2024-09-13T17:38:06.306 INFO:teuthology.orchestra.run.smithi043.stdout:Installed-Size: 20 2024-09-13T17:38:06.306 INFO:teuthology.orchestra.run.smithi043.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T17:38:06.306 INFO:teuthology.orchestra.run.smithi043.stdout:Architecture: amd64 2024-09-13T17:38:06.306 INFO:teuthology.orchestra.run.smithi043.stdout:Source: linux-meta 2024-09-13T17:38:06.306 INFO:teuthology.orchestra.run.smithi043.stdout:Version: 5.4.0.195.193 2024-09-13T17:38:06.306 INFO:teuthology.orchestra.run.smithi043.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T17:38:06.306 INFO:teuthology.orchestra.run.smithi043.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T17:38:06.306 INFO:teuthology.orchestra.run.smithi043.stdout:Recommends: thermald 2024-09-13T17:38:06.306 INFO:teuthology.orchestra.run.smithi043.stdout:Description: Generic Linux kernel image 2024-09-13T17:38:06.306 INFO:teuthology.orchestra.run.smithi043.stdout: This package will always depend on the latest generic kernel image 2024-09-13T17:38:06.306 INFO:teuthology.orchestra.run.smithi043.stdout: available. 2024-09-13T17:38:06.307 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-13T17:38:06.307 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T17:38:06.307 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T17:38:06.307 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-13T17:38:06.307 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi043.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T17:38:06.307 DEBUG:teuthology.orchestra.run.smithi043:> sudo apt-get clean 2024-09-13T17:38:06.397 DEBUG:teuthology.orchestra.run.smithi043:> sudo apt-get update 2024-09-13T17:38:06.544 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T17:38:06.544 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T17:38:06.551 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T17:38:06.569 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T17:38:06.852 INFO:teuthology.orchestra.run.smithi040.stdout:done 2024-09-13T17:38:07.327 DEBUG:teuthology.orchestra.run.smithi040:> dpkg -s linux-image-generic 2024-09-13T17:38:07.373 INFO:teuthology.orchestra.run.smithi040.stdout:Package: linux-image-generic 2024-09-13T17:38:07.373 INFO:teuthology.orchestra.run.smithi040.stdout:Status: install ok installed 2024-09-13T17:38:07.373 INFO:teuthology.orchestra.run.smithi040.stdout:Priority: optional 2024-09-13T17:38:07.373 INFO:teuthology.orchestra.run.smithi040.stdout:Section: kernel 2024-09-13T17:38:07.373 INFO:teuthology.orchestra.run.smithi040.stdout:Installed-Size: 20 2024-09-13T17:38:07.373 INFO:teuthology.orchestra.run.smithi040.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T17:38:07.373 INFO:teuthology.orchestra.run.smithi040.stdout:Architecture: amd64 2024-09-13T17:38:07.373 INFO:teuthology.orchestra.run.smithi040.stdout:Source: linux-meta 2024-09-13T17:38:07.373 INFO:teuthology.orchestra.run.smithi040.stdout:Version: 5.4.0.195.193 2024-09-13T17:38:07.373 INFO:teuthology.orchestra.run.smithi040.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T17:38:07.374 INFO:teuthology.orchestra.run.smithi040.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T17:38:07.374 INFO:teuthology.orchestra.run.smithi040.stdout:Recommends: thermald 2024-09-13T17:38:07.374 INFO:teuthology.orchestra.run.smithi040.stdout:Description: Generic Linux kernel image 2024-09-13T17:38:07.374 INFO:teuthology.orchestra.run.smithi040.stdout: This package will always depend on the latest generic kernel image 2024-09-13T17:38:07.374 INFO:teuthology.orchestra.run.smithi040.stdout: available. 2024-09-13T17:38:07.374 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-13T17:38:07.374 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T17:38:07.374 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T17:38:07.374 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T17:38:07.375 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi040.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T17:38:07.375 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get clean 2024-09-13T17:38:07.450 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get update 2024-09-13T17:38:07.720 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T17:38:07.720 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T17:38:07.788 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T17:38:07.870 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T17:38:08.454 INFO:teuthology.orchestra.run.smithi043.stdout:Reading package lists... 2024-09-13T17:38:08.482 DEBUG:teuthology.orchestra.run.smithi043:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T17:38:08.553 INFO:teuthology.orchestra.run.smithi043.stdout:Reading package lists... 2024-09-13T17:38:08.780 INFO:teuthology.orchestra.run.smithi043.stdout:Building dependency tree... 2024-09-13T17:38:08.780 INFO:teuthology.orchestra.run.smithi043.stdout:Reading state information... 2024-09-13T17:38:09.031 INFO:teuthology.orchestra.run.smithi043.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-13T17:38:09.031 INFO:teuthology.orchestra.run.smithi043.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T17:38:09.033 DEBUG:teuthology.orchestra.run.smithi043:> dpkg -s linux-image-generic 2024-09-13T17:38:09.054 INFO:teuthology.orchestra.run.smithi043.stdout:Package: linux-image-generic 2024-09-13T17:38:09.055 INFO:teuthology.orchestra.run.smithi043.stdout:Status: install ok installed 2024-09-13T17:38:09.055 INFO:teuthology.orchestra.run.smithi043.stdout:Priority: optional 2024-09-13T17:38:09.055 INFO:teuthology.orchestra.run.smithi043.stdout:Section: kernel 2024-09-13T17:38:09.055 INFO:teuthology.orchestra.run.smithi043.stdout:Installed-Size: 20 2024-09-13T17:38:09.055 INFO:teuthology.orchestra.run.smithi043.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T17:38:09.055 INFO:teuthology.orchestra.run.smithi043.stdout:Architecture: amd64 2024-09-13T17:38:09.055 INFO:teuthology.orchestra.run.smithi043.stdout:Source: linux-meta 2024-09-13T17:38:09.056 INFO:teuthology.orchestra.run.smithi043.stdout:Version: 5.4.0.195.193 2024-09-13T17:38:09.056 INFO:teuthology.orchestra.run.smithi043.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T17:38:09.056 INFO:teuthology.orchestra.run.smithi043.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T17:38:09.056 INFO:teuthology.orchestra.run.smithi043.stdout:Recommends: thermald 2024-09-13T17:38:09.056 INFO:teuthology.orchestra.run.smithi043.stdout:Description: Generic Linux kernel image 2024-09-13T17:38:09.056 INFO:teuthology.orchestra.run.smithi043.stdout: This package will always depend on the latest generic kernel image 2024-09-13T17:38:09.056 INFO:teuthology.orchestra.run.smithi043.stdout: available. 2024-09-13T17:38:09.056 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-09-13T17:38:09.061 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.4RV2iwz2qB 2024-09-13T17:38:09.061 DEBUG:teuthology.orchestra.run.smithi043:> sudo cp /boot/grub/grub.cfg /tmp/tmp.4RV2iwz2qB 2024-09-13T17:38:09.117 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.4RV2iwz2qB 2024-09-13T17:38:09.155 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.4RV2iwz2qB is 9KB 2024-09-13T17:38:09.169 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.4RV2iwz2qB 2024-09-13T17:38:09.175 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T17:38:09.230 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:38:09.230 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T17:38:09.230 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T17:38:09.259 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-13T17:38:09.259 DEBUG:teuthology.orchestra.run.smithi043:> sudo update-grub 2024-09-13T17:38:09.649 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-09-13T17:38:09.678 DEBUG:teuthology.orchestra.run.smithi040:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T17:38:09.747 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-09-13T17:38:09.970 INFO:teuthology.orchestra.run.smithi040.stdout:Building dependency tree... 2024-09-13T17:38:09.971 INFO:teuthology.orchestra.run.smithi040.stdout:Reading state information... 2024-09-13T17:38:10.220 INFO:teuthology.orchestra.run.smithi040.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-13T17:38:10.221 INFO:teuthology.orchestra.run.smithi040.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T17:38:10.222 DEBUG:teuthology.orchestra.run.smithi040:> dpkg -s linux-image-generic 2024-09-13T17:38:10.244 INFO:teuthology.orchestra.run.smithi040.stdout:Package: linux-image-generic 2024-09-13T17:38:10.244 INFO:teuthology.orchestra.run.smithi040.stdout:Status: install ok installed 2024-09-13T17:38:10.245 INFO:teuthology.orchestra.run.smithi040.stdout:Priority: optional 2024-09-13T17:38:10.245 INFO:teuthology.orchestra.run.smithi040.stdout:Section: kernel 2024-09-13T17:38:10.245 INFO:teuthology.orchestra.run.smithi040.stdout:Installed-Size: 20 2024-09-13T17:38:10.245 INFO:teuthology.orchestra.run.smithi040.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T17:38:10.245 INFO:teuthology.orchestra.run.smithi040.stdout:Architecture: amd64 2024-09-13T17:38:10.245 INFO:teuthology.orchestra.run.smithi040.stdout:Source: linux-meta 2024-09-13T17:38:10.245 INFO:teuthology.orchestra.run.smithi040.stdout:Version: 5.4.0.195.193 2024-09-13T17:38:10.245 INFO:teuthology.orchestra.run.smithi040.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T17:38:10.245 INFO:teuthology.orchestra.run.smithi040.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T17:38:10.245 INFO:teuthology.orchestra.run.smithi040.stdout:Recommends: thermald 2024-09-13T17:38:10.245 INFO:teuthology.orchestra.run.smithi040.stdout:Description: Generic Linux kernel image 2024-09-13T17:38:10.245 INFO:teuthology.orchestra.run.smithi040.stdout: This package will always depend on the latest generic kernel image 2024-09-13T17:38:10.245 INFO:teuthology.orchestra.run.smithi040.stdout: available. 2024-09-13T17:38:10.246 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2024-09-13T17:38:10.294 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.SiZVvyLYad 2024-09-13T17:38:10.295 DEBUG:teuthology.orchestra.run.smithi040:> sudo cp /boot/grub/grub.cfg /tmp/tmp.SiZVvyLYad 2024-09-13T17:38:10.311 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.SiZVvyLYad 2024-09-13T17:38:10.352 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.SiZVvyLYad is 9KB 2024-09-13T17:38:10.369 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.SiZVvyLYad 2024-09-13T17:38:10.376 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T17:38:10.431 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:38:10.431 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T17:38:10.431 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T17:38:10.466 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-13T17:38:10.466 DEBUG:teuthology.orchestra.run.smithi040:> sudo update-grub 2024-09-13T17:38:10.724 INFO:teuthology.orchestra.run.smithi043.stderr:Sourcing file `/etc/default/grub' 2024-09-13T17:38:10.730 INFO:teuthology.orchestra.run.smithi043.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T17:38:10.743 INFO:teuthology.orchestra.run.smithi043.stderr:Generating grub configuration file ... 2024-09-13T17:38:11.016 INFO:teuthology.orchestra.run.smithi043.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T17:38:11.028 INFO:teuthology.orchestra.run.smithi043.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T17:38:11.435 INFO:teuthology.orchestra.run.smithi043.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T17:38:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T17:38:11.949 INFO:teuthology.orchestra.run.smithi040.stderr:Sourcing file `/etc/default/grub' 2024-09-13T17:38:11.955 INFO:teuthology.orchestra.run.smithi040.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T17:38:11.969 INFO:teuthology.orchestra.run.smithi040.stderr:Generating grub configuration file ... 2024-09-13T17:38:12.339 INFO:teuthology.orchestra.run.smithi040.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T17:38:12.351 INFO:teuthology.orchestra.run.smithi040.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T17:38:12.507 INFO:teuthology.orchestra.run.smithi043.stderr:done 2024-09-13T17:38:12.509 DEBUG:teuthology.orchestra.run.smithi043:> sudo shutdown -r now 2024-09-13T17:38:12.777 INFO:teuthology.orchestra.run.smithi040.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T17:38:12.790 INFO:teuthology.orchestra.run.smithi040.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T17:38:14.491 INFO:teuthology.orchestra.run.smithi040.stderr:done 2024-09-13T17:38:14.493 DEBUG:teuthology.orchestra.run.smithi040:> sudo shutdown -r now 2024-09-13T17:38:42.541 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-13T17:38:42.542 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-13T17:38:42.543 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:38:44.497 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T17:38:44.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-09-13T17:38:44.497 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:39:02.849 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.40 2024-09-13T17:39:11.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-09-13T17:39:11.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:39:42.582 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T17:39:51.593 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-13T17:39:51.594 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:39:52.046 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-09-13T17:39:53.207 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-13T17:39:53.207 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-195-generic"... 2024-09-13T17:39:53.207 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-09-13T17:39:53.214 INFO:teuthology.orchestra.run.smithi043.stdout:5.4.0-195-generic 2024-09-13T17:39:53.214 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-13T17:39:53.214 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T17:39:53.214 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T17:39:54.215 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-13T17:39:54.215 DEBUG:teuthology.orchestra.run.smithi043:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T17:39:54.620 INFO:teuthology.orchestra.run.smithi043.stdout:ttyS1 2024-09-13T17:39:54.648 DEBUG:teuthology.parallel:result is None 2024-09-13T17:40:11.887 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T17:40:23.889 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-09-13T17:40:23.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:40:24.176 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-09-13T17:40:24.759 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-09-13T17:40:24.759 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-195-generic"... 2024-09-13T17:40:24.759 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-09-13T17:40:24.807 INFO:teuthology.orchestra.run.smithi040.stdout:5.4.0-195-generic 2024-09-13T17:40:24.808 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-13T17:40:24.808 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T17:40:24.808 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T17:40:25.809 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T17:40:25.809 DEBUG:teuthology.orchestra.run.smithi040:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T17:40:25.907 INFO:teuthology.orchestra.run.smithi040.stdout:ttyS1 2024-09-13T17:40:25.929 DEBUG:teuthology.parallel:result is None 2024-09-13T17:40:25.929 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T17:40:25.936 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T17:40:25.936 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T17:40:25.938 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T17:40:25.946 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T17:40:25.964 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T17:40:25.971 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T17:40:25.972 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T17:40:25.988 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T17:40:26.014 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T17:40:26.020 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T17:40:26.020 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T17:40:26.023 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T17:40:26.038 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T17:40:26.047 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T17:40:26.076 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T17:40:26.087 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T17:40:26.089 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T17:40:26.096 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T17:40:26.096 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T17:40:26.098 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T17:40:26.115 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T17:40:26.126 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T17:40:26.126 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T17:40:26.129 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T17:40:26.136 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T17:40:26.180 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T17:40:26.187 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:40:26.187 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T17:40:26.213 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T17:40:26.220 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T17:40:26.227 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:40:26.227 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T17:40:26.263 DEBUG:teuthology.orchestra.run.smithi040:> sudo service rsyslog restart 2024-09-13T17:40:26.265 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2024-09-13T17:40:26.357 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T17:40:26.364 INFO:teuthology.task.internal:Starting timer... 2024-09-13T17:40:26.364 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T17:40:26.378 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T17:40:26.387 DEBUG:teuthology.task.selinux:Excluding smithi040: OS 'ubuntu' does not support SELinux 2024-09-13T17:40:26.387 DEBUG:teuthology.task.selinux:Excluding smithi043: OS 'ubuntu' does not support SELinux 2024-09-13T17:40:26.387 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T17:40:26.388 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-13T17:40:26.388 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T17:40:26.388 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T17:40:26.397 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T17:40:26.411 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T17:40:26.412 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi040.front.sepia.ceph.com,smithi043.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T17:46:11.100 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi040.front.sepia.ceph.com'), Remote(name='ubuntu@smithi043.front.sepia.ceph.com')] 2024-09-13T17:46:11.102 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-09-13T17:46:11.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:46:11.227 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-09-13T17:46:11.354 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-09-13T17:46:11.355 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-13T17:46:11.355 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:46:11.452 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-09-13T17:46:11.579 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-13T17:46:11.579 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T17:46:11.589 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T17:46:11.589 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T17:46:11.590 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T17:46:11.592 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T17:46:11.592 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T17:46:11.624 INFO:teuthology.orchestra.run.smithi040.stdout:13 Sep 17:46:11 ntpd[13692]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-13T17:46:11.624 INFO:teuthology.orchestra.run.smithi040.stdout:13 Sep 17:46:11 ntpd[13692]: Command line: ntpd -gq 2024-09-13T17:46:11.625 INFO:teuthology.orchestra.run.smithi040.stdout:13 Sep 17:46:11 ntpd[13692]: proto: precision = 0.076 usec (-24) 2024-09-13T17:46:11.625 INFO:teuthology.orchestra.run.smithi040.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T17:46:11.626 INFO:teuthology.orchestra.run.smithi040.stdout:13 Sep 17:46:11 ntpd[13692]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T17:46:11.626 INFO:teuthology.orchestra.run.smithi040.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T17:46:11.627 INFO:teuthology.orchestra.run.smithi040.stdout:13 Sep 17:46:11 ntpd[13692]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T17:46:11.627 INFO:teuthology.orchestra.run.smithi040.stdout:13 Sep 17:46:11 ntpd[13692]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T17:46:11.627 INFO:teuthology.orchestra.run.smithi040.stdout:13 Sep 17:46:11 ntpd[13692]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T17:46:11.627 INFO:teuthology.orchestra.run.smithi040.stdout:13 Sep 17:46:11 ntpd[13692]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T17:46:11.627 INFO:teuthology.orchestra.run.smithi040.stdout:13 Sep 17:46:11 ntpd[13692]: Listen normally on 3 ens1f0 172.21.15.40:123 2024-09-13T17:46:11.627 INFO:teuthology.orchestra.run.smithi040.stdout:13 Sep 17:46:11 ntpd[13692]: Listen normally on 4 lo [::1]:123 2024-09-13T17:46:11.627 INFO:teuthology.orchestra.run.smithi040.stdout:13 Sep 17:46:11 ntpd[13692]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:12cc%3]:123 2024-09-13T17:46:11.628 INFO:teuthology.orchestra.run.smithi040.stdout:13 Sep 17:46:11 ntpd[13692]: Listening on routing socket on fd #22 for interface updates 2024-09-13T17:46:11.657 INFO:teuthology.orchestra.run.smithi043.stdout:13 Sep 17:46:11 ntpd[13600]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-13T17:46:11.658 INFO:teuthology.orchestra.run.smithi043.stdout:13 Sep 17:46:11 ntpd[13600]: Command line: ntpd -gq 2024-09-13T17:46:11.659 INFO:teuthology.orchestra.run.smithi043.stdout:13 Sep 17:46:11 ntpd[13600]: proto: precision = 0.090 usec (-23) 2024-09-13T17:46:11.659 INFO:teuthology.orchestra.run.smithi043.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T17:46:11.659 INFO:teuthology.orchestra.run.smithi043.stdout:13 Sep 17:46:11 ntpd[13600]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T17:46:11.660 INFO:teuthology.orchestra.run.smithi043.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T17:46:11.661 INFO:teuthology.orchestra.run.smithi043.stdout:13 Sep 17:46:11 ntpd[13600]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T17:46:11.661 INFO:teuthology.orchestra.run.smithi043.stdout:13 Sep 17:46:11 ntpd[13600]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T17:46:11.662 INFO:teuthology.orchestra.run.smithi043.stdout:13 Sep 17:46:11 ntpd[13600]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T17:46:11.662 INFO:teuthology.orchestra.run.smithi043.stdout:13 Sep 17:46:11 ntpd[13600]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T17:46:11.662 INFO:teuthology.orchestra.run.smithi043.stdout:13 Sep 17:46:11 ntpd[13600]: Listen normally on 3 ens1f0 172.21.15.43:123 2024-09-13T17:46:11.662 INFO:teuthology.orchestra.run.smithi043.stdout:13 Sep 17:46:11 ntpd[13600]: Listen normally on 4 lo [::1]:123 2024-09-13T17:46:11.662 INFO:teuthology.orchestra.run.smithi043.stdout:13 Sep 17:46:11 ntpd[13600]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1358%4]:123 2024-09-13T17:46:11.662 INFO:teuthology.orchestra.run.smithi043.stdout:13 Sep 17:46:11 ntpd[13600]: Listening on routing socket on fd #22 for interface updates 2024-09-13T17:46:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:12 ntpd[13600]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:12 ntpd[13600]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:12 ntpd[13600]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:12 ntpd[13600]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:12 ntpd[13600]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:46:12.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:12 ntpd[13600]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:46:14.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:14 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:14.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:14 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:14.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:14 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:14.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:14 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:14.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:14 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:14.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:14 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:14.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:14 ntpd[13692]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:46:14.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:14 ntpd[13692]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:46:14.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:14 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:14.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:14 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:14.627 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:14 ntpd[13692]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:46:14.627 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:14 ntpd[13692]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:46:14.658 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:14 ntpd[13600]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:14.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:14 ntpd[13600]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:14.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:14 ntpd[13600]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:14.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:14 ntpd[13600]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:14.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:14 ntpd[13600]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:46:14.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:14 ntpd[13600]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:46:15.625 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:15 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:15.625 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:15 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:15.625 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:15 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:15.625 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:15 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:15.625 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:15 ntpd[13692]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:46:15.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:15 ntpd[13692]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:46:15.658 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:15 ntpd[13600]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:15.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:15 ntpd[13600]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:15.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:15 ntpd[13600]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:15.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:15 ntpd[13600]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:15.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:15 ntpd[13600]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:46:15.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:15 ntpd[13600]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:46:16.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:16 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:16.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:16 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:16.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:16 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:16.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:16 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:16.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:16 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:16.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:16 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:16.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:16 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:16.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:16 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:16.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:16 ntpd[13692]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:46:16.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:16 ntpd[13692]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:46:16.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:16 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:16.627 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:16 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:16.627 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:16 ntpd[13692]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:46:16.627 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:16 ntpd[13692]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:46:16.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:16 ntpd[13600]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:16.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:16 ntpd[13600]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:16.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:16 ntpd[13600]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:16.660 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:16 ntpd[13600]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:16.660 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:16 ntpd[13600]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:16.660 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:16 ntpd[13600]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:16.660 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:16 ntpd[13600]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:16.660 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:16 ntpd[13600]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:16.660 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:16 ntpd[13600]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:46:16.660 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:16 ntpd[13600]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:46:16.660 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:16 ntpd[13600]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:16.660 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:16 ntpd[13600]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:16.660 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:16 ntpd[13600]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:46:16.660 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:16 ntpd[13600]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:46:17.625 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:17 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:17.625 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:17 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:17.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:17 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:17.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:17 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:17.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:17 ntpd[13692]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:46:17.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:17 ntpd[13692]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:46:17.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:17 ntpd[13600]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:17.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:17 ntpd[13600]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:17.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:17 ntpd[13600]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:17.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:17 ntpd[13600]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:17.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:17 ntpd[13600]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:46:17.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:17 ntpd[13600]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:46:18.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:18 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:18.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:18 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:18.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:18 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:18.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:18 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:18.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:18 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:18.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:18 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:18.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:18 ntpd[13692]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:46:18.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:18 ntpd[13692]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:46:18.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:18 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:18.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:18 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:18.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:18 ntpd[13692]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:46:18.627 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:18 ntpd[13692]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:46:18.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:18 ntpd[13600]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:18.659 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:18 ntpd[13600]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:18.660 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:18 ntpd[13600]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:18.660 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:18 ntpd[13600]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:18.660 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:18 ntpd[13600]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:18.660 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:18 ntpd[13600]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:18.660 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:18 ntpd[13600]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:46:18.660 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:18 ntpd[13600]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:46:18.660 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:18 ntpd[13600]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:18.660 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:18 ntpd[13600]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:18.661 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:18 ntpd[13600]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:46:18.661 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:18 ntpd[13600]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:46:18.661 INFO:teuthology.orchestra.run.smithi043.stdout:13 Sep 17:46:18 ntpd[13600]: ntpd: time slew +0.003893 s 2024-09-13T17:46:18.661 INFO:teuthology.orchestra.run.smithi043.stdout:ntpd: time slew +0.003893s 2024-09-13T17:46:18.662 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:18 ntpd[13600]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T17:46:18.662 INFO:teuthology.orchestra.run.smithi043.stderr:13 Sep 17:46:18 ntpd[13600]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T17:46:18.731 INFO:teuthology.orchestra.run.smithi043.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T17:46:18.731 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================== 2024-09-13T17:46:18.731 INFO:teuthology.orchestra.run.smithi043.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T17:46:18.731 INFO:teuthology.orchestra.run.smithi043.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T17:46:18.731 INFO:teuthology.orchestra.run.smithi043.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T17:46:18.731 INFO:teuthology.orchestra.run.smithi043.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T17:46:19.625 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:19 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:19.625 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:19 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:19.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:19 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:19.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:19 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:19.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:19 ntpd[13692]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:46:19.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:19 ntpd[13692]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:46:20.625 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:20 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:20.625 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:20 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:20.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:20 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:20.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:20 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:20.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:20 ntpd[13692]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:46:20.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:20 ntpd[13692]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:46:20.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:20 ntpd[13692]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:46:20.626 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:20 ntpd[13692]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:46:20.626 INFO:teuthology.orchestra.run.smithi040.stdout:13 Sep 17:46:20 ntpd[13692]: ntpd: time slew +0.001613 s 2024-09-13T17:46:20.626 INFO:teuthology.orchestra.run.smithi040.stdout:ntpd: time slew +0.001613s 2024-09-13T17:46:20.627 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:20 ntpd[13692]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T17:46:20.627 INFO:teuthology.orchestra.run.smithi040.stderr:13 Sep 17:46:20 ntpd[13692]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T17:46:20.692 INFO:teuthology.orchestra.run.smithi040.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T17:46:20.692 INFO:teuthology.orchestra.run.smithi040.stdout:============================================================================== 2024-09-13T17:46:20.692 INFO:teuthology.orchestra.run.smithi040.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T17:46:20.692 INFO:teuthology.orchestra.run.smithi040.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T17:46:20.692 INFO:teuthology.orchestra.run.smithi040.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T17:46:20.693 INFO:teuthology.orchestra.run.smithi040.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T17:46:20.693 INFO:teuthology.run_tasks:Running task install... 2024-09-13T17:46:20.703 DEBUG:teuthology.task.install:project ceph 2024-09-13T17:46:20.703 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}} 2024-09-13T17:46:20.703 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'} 2024-09-13T17:46:20.703 INFO:teuthology.task.install:Using flavor: default 2024-09-13T17:46:20.709 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T17:46:20.709 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-09-13T17:46:20.710 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-key list | grep Ceph 2024-09-13T17:46:20.710 DEBUG:teuthology.orchestra.run.smithi043:> sudo apt-key list | grep Ceph 2024-09-13T17:46:20.820 INFO:teuthology.orchestra.run.smithi043.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-13T17:46:20.838 INFO:teuthology.orchestra.run.smithi040.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-13T17:46:20.882 INFO:teuthology.orchestra.run.smithi043.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T17:46:20.882 INFO:teuthology.orchestra.run.smithi043.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T17:46:20.883 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-13T17:46:20.883 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-13T17:46:20.883 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:46:20.900 INFO:teuthology.orchestra.run.smithi040.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T17:46:20.901 INFO:teuthology.orchestra.run.smithi040.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T17:46:20.902 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-13T17:46:20.903 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-13T17:46:20.903 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:46:21.034 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default/ 2024-09-13T17:46:21.034 INFO:teuthology.task.install.deb:Package version is 18.2.4-860-g6c6b083f-1focal 2024-09-13T17:46:21.055 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default/ 2024-09-13T17:46:21.055 INFO:teuthology.task.install.deb:Package version is 18.2.4-860-g6c6b083f-1focal 2024-09-13T17:46:21.382 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:46:21.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T17:46:21.384 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:46:21.384 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T17:46:21.400 DEBUG:teuthology.orchestra.run.smithi043:> sudo apt-get update 2024-09-13T17:46:21.401 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get update 2024-09-13T17:46:21.565 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T17:46:21.592 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T17:46:21.599 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T17:46:21.617 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T17:46:21.663 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T17:46:21.709 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T17:46:21.744 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T17:46:21.823 INFO:teuthology.orchestra.run.smithi040.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal InRelease 2024-09-13T17:46:21.826 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T17:46:21.857 INFO:teuthology.orchestra.run.smithi043.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal InRelease 2024-09-13T17:46:21.901 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-13T17:46:21.934 INFO:teuthology.orchestra.run.smithi043.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-13T17:46:21.980 INFO:teuthology.orchestra.run.smithi040.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal Release.gpg 2024-09-13T17:46:22.014 INFO:teuthology.orchestra.run.smithi043.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal Release.gpg 2024-09-13T17:46:22.058 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main i386 Packages [3,433 B] 2024-09-13T17:46:22.090 INFO:teuthology.orchestra.run.smithi043.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main i386 Packages [3,433 B] 2024-09-13T17:46:22.139 INFO:teuthology.orchestra.run.smithi040.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-09-13T17:46:22.179 INFO:teuthology.orchestra.run.smithi043.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-09-13T17:46:22.244 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 28.7 kB in 1s (42.7 kB/s) 2024-09-13T17:46:22.279 INFO:teuthology.orchestra.run.smithi043.stdout:Fetched 28.7 kB in 1s (43.3 kB/s) 2024-09-13T17:46:23.434 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-09-13T17:46:23.463 DEBUG:teuthology.orchestra.run.smithi040:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-860-g6c6b083f-1focal cephadm=18.2.4-860-g6c6b083f-1focal ceph-mds=18.2.4-860-g6c6b083f-1focal ceph-mgr=18.2.4-860-g6c6b083f-1focal ceph-common=18.2.4-860-g6c6b083f-1focal ceph-fuse=18.2.4-860-g6c6b083f-1focal ceph-test=18.2.4-860-g6c6b083f-1focal ceph-volume=18.2.4-860-g6c6b083f-1focal radosgw=18.2.4-860-g6c6b083f-1focal python3-rados=18.2.4-860-g6c6b083f-1focal python3-rgw=18.2.4-860-g6c6b083f-1focal python3-cephfs=18.2.4-860-g6c6b083f-1focal python3-rbd=18.2.4-860-g6c6b083f-1focal libcephfs2=18.2.4-860-g6c6b083f-1focal libcephfs-dev=18.2.4-860-g6c6b083f-1focal librados2=18.2.4-860-g6c6b083f-1focal librbd1=18.2.4-860-g6c6b083f-1focal rbd-fuse=18.2.4-860-g6c6b083f-1focal python3-cephfs=18.2.4-860-g6c6b083f-1focal cephfs-shell=18.2.4-860-g6c6b083f-1focal cephfs-top=18.2.4-860-g6c6b083f-1focal cephfs-mirror=18.2.4-860-g6c6b083f-1focal 2024-09-13T17:46:23.483 INFO:teuthology.orchestra.run.smithi043.stdout:Reading package lists... 2024-09-13T17:46:23.512 DEBUG:teuthology.orchestra.run.smithi043:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-860-g6c6b083f-1focal cephadm=18.2.4-860-g6c6b083f-1focal ceph-mds=18.2.4-860-g6c6b083f-1focal ceph-mgr=18.2.4-860-g6c6b083f-1focal ceph-common=18.2.4-860-g6c6b083f-1focal ceph-fuse=18.2.4-860-g6c6b083f-1focal ceph-test=18.2.4-860-g6c6b083f-1focal ceph-volume=18.2.4-860-g6c6b083f-1focal radosgw=18.2.4-860-g6c6b083f-1focal python3-rados=18.2.4-860-g6c6b083f-1focal python3-rgw=18.2.4-860-g6c6b083f-1focal python3-cephfs=18.2.4-860-g6c6b083f-1focal python3-rbd=18.2.4-860-g6c6b083f-1focal libcephfs2=18.2.4-860-g6c6b083f-1focal libcephfs-dev=18.2.4-860-g6c6b083f-1focal librados2=18.2.4-860-g6c6b083f-1focal librbd1=18.2.4-860-g6c6b083f-1focal rbd-fuse=18.2.4-860-g6c6b083f-1focal python3-cephfs=18.2.4-860-g6c6b083f-1focal cephfs-shell=18.2.4-860-g6c6b083f-1focal cephfs-top=18.2.4-860-g6c6b083f-1focal cephfs-mirror=18.2.4-860-g6c6b083f-1focal 2024-09-13T17:46:23.538 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-09-13T17:46:23.583 INFO:teuthology.orchestra.run.smithi043.stdout:Reading package lists... 2024-09-13T17:46:23.746 INFO:teuthology.orchestra.run.smithi040.stdout:Building dependency tree... 2024-09-13T17:46:23.747 INFO:teuthology.orchestra.run.smithi040.stdout:Reading state information... 2024-09-13T17:46:23.807 INFO:teuthology.orchestra.run.smithi043.stdout:Building dependency tree... 2024-09-13T17:46:23.807 INFO:teuthology.orchestra.run.smithi043.stdout:Reading state information... 2024-09-13T17:46:23.948 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T17:46:23.949 INFO:teuthology.orchestra.run.smithi040.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T17:46:23.949 INFO:teuthology.orchestra.run.smithi040.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T17:46:23.950 INFO:teuthology.orchestra.run.smithi040.stdout:The following additional packages will be installed: 2024-09-13T17:46:23.950 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T17:46:23.950 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-13T17:46:23.950 INFO:teuthology.orchestra.run.smithi040.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-13T17:46:23.950 INFO:teuthology.orchestra.run.smithi040.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-13T17:46:23.950 INFO:teuthology.orchestra.run.smithi040.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-13T17:46:23.951 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-13T17:46:23.951 INFO:teuthology.orchestra.run.smithi040.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T17:46:23.951 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T17:46:23.951 INFO:teuthology.orchestra.run.smithi040.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T17:46:23.951 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-13T17:46:23.951 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-13T17:46:23.951 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-13T17:46:23.951 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-13T17:46:23.951 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-13T17:46:23.952 INFO:teuthology.orchestra.run.smithi040.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-13T17:46:23.952 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-13T17:46:23.952 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2024-09-13T17:46:23.952 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T17:46:23.952 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T17:46:23.952 INFO:teuthology.orchestra.run.smithi040.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T17:46:23.952 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T17:46:23.952 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T17:46:23.952 INFO:teuthology.orchestra.run.smithi040.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-13T17:46:23.952 INFO:teuthology.orchestra.run.smithi040.stdout:Suggested packages: 2024-09-13T17:46:23.953 INFO:teuthology.orchestra.run.smithi040.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-13T17:46:23.953 INFO:teuthology.orchestra.run.smithi040.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-13T17:46:23.953 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-13T17:46:23.953 INFO:teuthology.orchestra.run.smithi040.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-13T17:46:23.953 INFO:teuthology.orchestra.run.smithi040.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-13T17:46:23.953 INFO:teuthology.orchestra.run.smithi040.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-13T17:46:23.953 INFO:teuthology.orchestra.run.smithi040.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-13T17:46:23.953 INFO:teuthology.orchestra.run.smithi040.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-13T17:46:23.953 INFO:teuthology.orchestra.run.smithi040.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-13T17:46:23.953 INFO:teuthology.orchestra.run.smithi040.stdout:Recommended packages: 2024-09-13T17:46:23.953 INFO:teuthology.orchestra.run.smithi040.stdout: btrfs-tools 2024-09-13T17:46:24.021 INFO:teuthology.orchestra.run.smithi043.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T17:46:24.022 INFO:teuthology.orchestra.run.smithi043.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T17:46:24.022 INFO:teuthology.orchestra.run.smithi043.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T17:46:24.022 INFO:teuthology.orchestra.run.smithi043.stdout:The following additional packages will be installed: 2024-09-13T17:46:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T17:46:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-13T17:46:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-13T17:46:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-13T17:46:24.023 INFO:teuthology.orchestra.run.smithi043.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-13T17:46:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-13T17:46:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T17:46:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T17:46:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T17:46:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-13T17:46:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-13T17:46:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-13T17:46:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-13T17:46:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-13T17:46:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-13T17:46:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-13T17:46:24.024 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2024-09-13T17:46:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T17:46:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T17:46:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T17:46:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T17:46:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T17:46:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-13T17:46:24.025 INFO:teuthology.orchestra.run.smithi043.stdout:Suggested packages: 2024-09-13T17:46:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-13T17:46:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-13T17:46:24.025 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-13T17:46:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-13T17:46:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-13T17:46:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-13T17:46:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-13T17:46:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-13T17:46:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-13T17:46:24.026 INFO:teuthology.orchestra.run.smithi043.stdout:Recommended packages: 2024-09-13T17:46:24.026 INFO:teuthology.orchestra.run.smithi043.stdout: btrfs-tools 2024-09-13T17:46:24.084 INFO:teuthology.orchestra.run.smithi040.stdout:The following NEW packages will be installed: 2024-09-13T17:46:24.084 INFO:teuthology.orchestra.run.smithi040.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T17:46:24.084 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T17:46:24.084 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-13T17:46:24.084 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-09-13T17:46:24.084 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-09-13T17:46:24.084 INFO:teuthology.orchestra.run.smithi040.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-09-13T17:46:24.084 INFO:teuthology.orchestra.run.smithi040.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-09-13T17:46:24.085 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-09-13T17:46:24.085 INFO:teuthology.orchestra.run.smithi040.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T17:46:24.086 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T17:46:24.086 INFO:teuthology.orchestra.run.smithi040.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T17:46:24.086 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2024-09-13T17:46:24.086 INFO:teuthology.orchestra.run.smithi040.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-09-13T17:46:24.086 INFO:teuthology.orchestra.run.smithi040.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-09-13T17:46:24.086 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-13T17:46:24.086 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-13T17:46:24.086 INFO:teuthology.orchestra.run.smithi040.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-13T17:46:24.086 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-13T17:46:24.086 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-09-13T17:46:24.086 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-09-13T17:46:24.086 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-13T17:46:24.086 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T17:46:24.087 INFO:teuthology.orchestra.run.smithi040.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T17:46:24.087 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T17:46:24.087 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T17:46:24.087 INFO:teuthology.orchestra.run.smithi040.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-13T17:46:24.087 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages will be upgraded: 2024-09-13T17:46:24.087 INFO:teuthology.orchestra.run.smithi040.stdout: librados2 librbd1 2024-09-13T17:46:24.153 INFO:teuthology.orchestra.run.smithi043.stdout:The following NEW packages will be installed: 2024-09-13T17:46:24.154 INFO:teuthology.orchestra.run.smithi043.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T17:46:24.154 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T17:46:24.154 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-13T17:46:24.154 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-09-13T17:46:24.154 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-09-13T17:46:24.154 INFO:teuthology.orchestra.run.smithi043.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-09-13T17:46:24.154 INFO:teuthology.orchestra.run.smithi043.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-09-13T17:46:24.155 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-09-13T17:46:24.155 INFO:teuthology.orchestra.run.smithi043.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T17:46:24.155 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T17:46:24.155 INFO:teuthology.orchestra.run.smithi043.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T17:46:24.155 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2024-09-13T17:46:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-09-13T17:46:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-09-13T17:46:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-13T17:46:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-13T17:46:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-13T17:46:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-13T17:46:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-09-13T17:46:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-09-13T17:46:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-13T17:46:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T17:46:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T17:46:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T17:46:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T17:46:24.156 INFO:teuthology.orchestra.run.smithi043.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-13T17:46:24.157 INFO:teuthology.orchestra.run.smithi043.stdout:The following packages will be upgraded: 2024-09-13T17:46:24.157 INFO:teuthology.orchestra.run.smithi043.stdout: librados2 librbd1 2024-09-13T17:46:24.280 INFO:teuthology.orchestra.run.smithi040.stdout:2 upgraded, 116 newly installed, 0 to remove and 295 not upgraded. 2024-09-13T17:46:24.281 INFO:teuthology.orchestra.run.smithi040.stdout:Need to get 200 MB of archives. 2024-09-13T17:46:24.281 INFO:teuthology.orchestra.run.smithi040.stdout:After this operation, 979 MB of additional disk space will be used. 2024-09-13T17:46:24.281 INFO:teuthology.orchestra.run.smithi040.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-09-13T17:46:24.350 INFO:teuthology.orchestra.run.smithi043.stdout:2 upgraded, 116 newly installed, 0 to remove and 295 not upgraded. 2024-09-13T17:46:24.350 INFO:teuthology.orchestra.run.smithi043.stdout:Need to get 200 MB of archives. 2024-09-13T17:46:24.351 INFO:teuthology.orchestra.run.smithi043.stdout:After this operation, 979 MB of additional disk space will be used. 2024-09-13T17:46:24.351 INFO:teuthology.orchestra.run.smithi043.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-09-13T17:46:24.364 INFO:teuthology.orchestra.run.smithi040.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-13T17:46:24.398 INFO:teuthology.orchestra.run.smithi040.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-09-13T17:46:24.434 INFO:teuthology.orchestra.run.smithi043.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-13T17:46:24.468 INFO:teuthology.orchestra.run.smithi043.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-09-13T17:46:24.522 INFO:teuthology.orchestra.run.smithi040.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-13T17:46:24.523 INFO:teuthology.orchestra.run.smithi040.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.4-860-g6c6b083f-1focal [3,453 kB] 2024-09-13T17:46:24.589 INFO:teuthology.orchestra.run.smithi043.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.4-860-g6c6b083f-1focal [3,453 kB] 2024-09-13T17:46:24.593 INFO:teuthology.orchestra.run.smithi043.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-13T17:46:24.652 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-13T17:46:24.668 INFO:teuthology.orchestra.run.smithi040.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-13T17:46:24.722 INFO:teuthology.orchestra.run.smithi043.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-13T17:46:24.729 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-13T17:46:24.738 INFO:teuthology.orchestra.run.smithi043.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-13T17:46:24.800 INFO:teuthology.orchestra.run.smithi043.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-13T17:46:24.924 INFO:teuthology.orchestra.run.smithi040.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-13T17:46:24.933 INFO:teuthology.orchestra.run.smithi040.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-13T17:46:24.959 INFO:teuthology.orchestra.run.smithi040.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-13T17:46:24.967 INFO:teuthology.orchestra.run.smithi040.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-13T17:46:24.968 INFO:teuthology.orchestra.run.smithi040.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-13T17:46:24.973 INFO:teuthology.orchestra.run.smithi040.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-13T17:46:24.974 INFO:teuthology.orchestra.run.smithi040.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-13T17:46:24.986 INFO:teuthology.orchestra.run.smithi040.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-13T17:46:24.987 INFO:teuthology.orchestra.run.smithi040.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-13T17:46:24.995 INFO:teuthology.orchestra.run.smithi043.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-13T17:46:25.005 INFO:teuthology.orchestra.run.smithi043.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-13T17:46:25.007 INFO:teuthology.orchestra.run.smithi040.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-13T17:46:25.030 INFO:teuthology.orchestra.run.smithi043.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-13T17:46:25.038 INFO:teuthology.orchestra.run.smithi043.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-13T17:46:25.039 INFO:teuthology.orchestra.run.smithi043.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-13T17:46:25.044 INFO:teuthology.orchestra.run.smithi043.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-13T17:46:25.045 INFO:teuthology.orchestra.run.smithi043.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-13T17:46:25.057 INFO:teuthology.orchestra.run.smithi043.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-13T17:46:25.058 INFO:teuthology.orchestra.run.smithi043.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-13T17:46:25.073 INFO:teuthology.orchestra.run.smithi040.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-13T17:46:25.074 INFO:teuthology.orchestra.run.smithi040.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-13T17:46:25.075 INFO:teuthology.orchestra.run.smithi040.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-13T17:46:25.077 INFO:teuthology.orchestra.run.smithi040.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T17:46:25.078 INFO:teuthology.orchestra.run.smithi043.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-13T17:46:25.078 INFO:teuthology.orchestra.run.smithi040.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-13T17:46:25.078 INFO:teuthology.orchestra.run.smithi040.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-13T17:46:25.081 INFO:teuthology.orchestra.run.smithi040.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-13T17:46:25.082 INFO:teuthology.orchestra.run.smithi040.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-13T17:46:25.088 INFO:teuthology.orchestra.run.smithi040.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-13T17:46:25.148 INFO:teuthology.orchestra.run.smithi043.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-13T17:46:25.148 INFO:teuthology.orchestra.run.smithi043.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-13T17:46:25.149 INFO:teuthology.orchestra.run.smithi043.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-13T17:46:25.151 INFO:teuthology.orchestra.run.smithi043.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T17:46:25.152 INFO:teuthology.orchestra.run.smithi043.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-13T17:46:25.152 INFO:teuthology.orchestra.run.smithi043.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-13T17:46:25.155 INFO:teuthology.orchestra.run.smithi043.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-13T17:46:25.156 INFO:teuthology.orchestra.run.smithi043.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-13T17:46:25.160 INFO:teuthology.orchestra.run.smithi043.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-13T17:46:25.166 INFO:teuthology.orchestra.run.smithi040.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-13T17:46:25.167 INFO:teuthology.orchestra.run.smithi040.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-13T17:46:25.175 INFO:teuthology.orchestra.run.smithi040.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-13T17:46:25.176 INFO:teuthology.orchestra.run.smithi040.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-13T17:46:25.177 INFO:teuthology.orchestra.run.smithi040.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-13T17:46:25.178 INFO:teuthology.orchestra.run.smithi040.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-13T17:46:25.180 INFO:teuthology.orchestra.run.smithi040.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-13T17:46:25.182 INFO:teuthology.orchestra.run.smithi040.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-13T17:46:25.182 INFO:teuthology.orchestra.run.smithi040.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-13T17:46:25.241 INFO:teuthology.orchestra.run.smithi043.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-13T17:46:25.241 INFO:teuthology.orchestra.run.smithi043.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-13T17:46:25.249 INFO:teuthology.orchestra.run.smithi043.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-13T17:46:25.249 INFO:teuthology.orchestra.run.smithi043.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-13T17:46:25.249 INFO:teuthology.orchestra.run.smithi043.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-13T17:46:25.250 INFO:teuthology.orchestra.run.smithi043.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-13T17:46:25.250 INFO:teuthology.orchestra.run.smithi040.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-13T17:46:25.251 INFO:teuthology.orchestra.run.smithi043.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-13T17:46:25.252 INFO:teuthology.orchestra.run.smithi040.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-09-13T17:46:25.253 INFO:teuthology.orchestra.run.smithi043.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-13T17:46:25.259 INFO:teuthology.orchestra.run.smithi040.stdout:Get:39 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.4-860-g6c6b083f-1focal [4,041 kB] 2024-09-13T17:46:25.299 INFO:teuthology.orchestra.run.smithi043.stdout:Get:36 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.4-860-g6c6b083f-1focal [4,041 kB] 2024-09-13T17:46:25.322 INFO:teuthology.orchestra.run.smithi043.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-13T17:46:25.327 INFO:teuthology.orchestra.run.smithi043.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-13T17:46:25.332 INFO:teuthology.orchestra.run.smithi040.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-13T17:46:25.335 INFO:teuthology.orchestra.run.smithi040.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-13T17:46:25.336 INFO:teuthology.orchestra.run.smithi040.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-13T17:46:25.337 INFO:teuthology.orchestra.run.smithi040.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-13T17:46:25.338 INFO:teuthology.orchestra.run.smithi040.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-13T17:46:25.340 INFO:teuthology.orchestra.run.smithi040.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-13T17:46:25.341 INFO:teuthology.orchestra.run.smithi040.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-13T17:46:25.342 INFO:teuthology.orchestra.run.smithi040.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-13T17:46:25.403 INFO:teuthology.orchestra.run.smithi043.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-09-13T17:46:25.406 INFO:teuthology.orchestra.run.smithi043.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-13T17:46:25.409 INFO:teuthology.orchestra.run.smithi043.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-13T17:46:25.410 INFO:teuthology.orchestra.run.smithi043.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-13T17:46:25.412 INFO:teuthology.orchestra.run.smithi043.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-13T17:46:25.412 INFO:teuthology.orchestra.run.smithi043.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-13T17:46:25.413 INFO:teuthology.orchestra.run.smithi040.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T17:46:25.414 INFO:teuthology.orchestra.run.smithi040.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-13T17:46:25.414 INFO:teuthology.orchestra.run.smithi043.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-13T17:46:25.414 INFO:teuthology.orchestra.run.smithi043.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-13T17:46:25.485 INFO:teuthology.orchestra.run.smithi043.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-13T17:46:25.488 INFO:teuthology.orchestra.run.smithi043.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T17:46:25.495 INFO:teuthology.orchestra.run.smithi040.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-13T17:46:25.497 INFO:teuthology.orchestra.run.smithi040.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-13T17:46:25.499 INFO:teuthology.orchestra.run.smithi040.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-13T17:46:25.500 INFO:teuthology.orchestra.run.smithi040.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-13T17:46:25.501 INFO:teuthology.orchestra.run.smithi040.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-13T17:46:25.567 INFO:teuthology.orchestra.run.smithi043.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-13T17:46:25.569 INFO:teuthology.orchestra.run.smithi043.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-13T17:46:25.571 INFO:teuthology.orchestra.run.smithi043.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-13T17:46:25.573 INFO:teuthology.orchestra.run.smithi043.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-13T17:46:25.574 INFO:teuthology.orchestra.run.smithi043.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-13T17:46:25.575 INFO:teuthology.orchestra.run.smithi043.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-13T17:46:25.695 INFO:teuthology.orchestra.run.smithi040.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.4-860-g6c6b083f-1focal [747 kB] 2024-09-13T17:46:25.769 INFO:teuthology.orchestra.run.smithi040.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.4-860-g6c6b083f-1focal [388 kB] 2024-09-13T17:46:25.826 INFO:teuthology.orchestra.run.smithi040.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-13T17:46:25.836 INFO:teuthology.orchestra.run.smithi040.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.4-860-g6c6b083f-1focal [31.9 kB] 2024-09-13T17:46:25.837 INFO:teuthology.orchestra.run.smithi040.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.4-860-g6c6b083f-1focal [200 kB] 2024-09-13T17:46:25.840 INFO:teuthology.orchestra.run.smithi040.stdout:Get:60 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.4-860-g6c6b083f-1focal [62.5 kB] 2024-09-13T17:46:25.877 INFO:teuthology.orchestra.run.smithi043.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.4-860-g6c6b083f-1focal [747 kB] 2024-09-13T17:46:25.887 INFO:teuthology.orchestra.run.smithi040.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-13T17:46:25.887 INFO:teuthology.orchestra.run.smithi040.stdout:Get:62 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.4-860-g6c6b083f-1focal [368 kB] 2024-09-13T17:46:25.888 INFO:teuthology.orchestra.run.smithi040.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-13T17:46:25.899 INFO:teuthology.orchestra.run.smithi043.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-13T17:46:25.911 INFO:teuthology.orchestra.run.smithi040.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.4-860-g6c6b083f-1focal [6,634 kB] 2024-09-13T17:46:25.924 INFO:teuthology.orchestra.run.smithi040.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-13T17:46:25.924 INFO:teuthology.orchestra.run.smithi040.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-13T17:46:25.926 INFO:teuthology.orchestra.run.smithi040.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-13T17:46:25.926 INFO:teuthology.orchestra.run.smithi040.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-13T17:46:25.926 INFO:teuthology.orchestra.run.smithi040.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-13T17:46:25.926 INFO:teuthology.orchestra.run.smithi040.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-13T17:46:25.927 INFO:teuthology.orchestra.run.smithi043.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-13T17:46:25.927 INFO:teuthology.orchestra.run.smithi040.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-13T17:46:25.929 INFO:teuthology.orchestra.run.smithi043.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-13T17:46:25.955 INFO:teuthology.orchestra.run.smithi043.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.4-860-g6c6b083f-1focal [388 kB] 2024-09-13T17:46:25.990 INFO:teuthology.orchestra.run.smithi040.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-13T17:46:25.991 INFO:teuthology.orchestra.run.smithi040.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-13T17:46:26.000 INFO:teuthology.orchestra.run.smithi043.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-13T17:46:26.000 INFO:teuthology.orchestra.run.smithi043.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-13T17:46:26.001 INFO:teuthology.orchestra.run.smithi043.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-13T17:46:26.001 INFO:teuthology.orchestra.run.smithi043.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-13T17:46:26.001 INFO:teuthology.orchestra.run.smithi043.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-13T17:46:26.002 INFO:teuthology.orchestra.run.smithi043.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-13T17:46:26.002 INFO:teuthology.orchestra.run.smithi043.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-13T17:46:26.027 INFO:teuthology.orchestra.run.smithi043.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.4-860-g6c6b083f-1focal [31.9 kB] 2024-09-13T17:46:26.028 INFO:teuthology.orchestra.run.smithi043.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.4-860-g6c6b083f-1focal [200 kB] 2024-09-13T17:46:26.033 INFO:teuthology.orchestra.run.smithi043.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.4-860-g6c6b083f-1focal [62.5 kB] 2024-09-13T17:46:26.034 INFO:teuthology.orchestra.run.smithi043.stdout:Get:70 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.4-860-g6c6b083f-1focal [368 kB] 2024-09-13T17:46:26.066 INFO:teuthology.orchestra.run.smithi043.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-13T17:46:26.067 INFO:teuthology.orchestra.run.smithi043.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-13T17:46:26.071 INFO:teuthology.orchestra.run.smithi040.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-13T17:46:26.077 INFO:teuthology.orchestra.run.smithi040.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-13T17:46:26.081 INFO:teuthology.orchestra.run.smithi040.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-13T17:46:26.084 INFO:teuthology.orchestra.run.smithi040.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-13T17:46:26.085 INFO:teuthology.orchestra.run.smithi040.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-13T17:46:26.087 INFO:teuthology.orchestra.run.smithi040.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-13T17:46:26.090 INFO:teuthology.orchestra.run.smithi040.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-09-13T17:46:26.090 INFO:teuthology.orchestra.run.smithi040.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-13T17:46:26.104 INFO:teuthology.orchestra.run.smithi043.stdout:Get:73 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.4-860-g6c6b083f-1focal [6,634 kB] 2024-09-13T17:46:26.148 INFO:teuthology.orchestra.run.smithi043.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-13T17:46:26.152 INFO:teuthology.orchestra.run.smithi043.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-13T17:46:26.154 INFO:teuthology.orchestra.run.smithi043.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-13T17:46:26.157 INFO:teuthology.orchestra.run.smithi043.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-13T17:46:26.158 INFO:teuthology.orchestra.run.smithi043.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-13T17:46:26.163 INFO:teuthology.orchestra.run.smithi043.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-13T17:46:26.164 INFO:teuthology.orchestra.run.smithi040.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-13T17:46:26.165 INFO:teuthology.orchestra.run.smithi043.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-09-13T17:46:26.165 INFO:teuthology.orchestra.run.smithi043.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-13T17:46:26.167 INFO:teuthology.orchestra.run.smithi040.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-09-13T17:46:26.235 INFO:teuthology.orchestra.run.smithi040.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-13T17:46:26.239 INFO:teuthology.orchestra.run.smithi043.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-13T17:46:26.244 INFO:teuthology.orchestra.run.smithi043.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-09-13T17:46:26.311 INFO:teuthology.orchestra.run.smithi043.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-13T17:46:26.346 INFO:teuthology.orchestra.run.smithi040.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-13T17:46:26.401 INFO:teuthology.orchestra.run.smithi040.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-13T17:46:26.401 INFO:teuthology.orchestra.run.smithi040.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-13T17:46:26.403 INFO:teuthology.orchestra.run.smithi040.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-13T17:46:26.421 INFO:teuthology.orchestra.run.smithi043.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-13T17:46:26.421 INFO:teuthology.orchestra.run.smithi040.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T17:46:26.422 INFO:teuthology.orchestra.run.smithi040.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-13T17:46:26.423 INFO:teuthology.orchestra.run.smithi040.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-13T17:46:26.423 INFO:teuthology.orchestra.run.smithi040.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-13T17:46:26.424 INFO:teuthology.orchestra.run.smithi040.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-13T17:46:26.476 INFO:teuthology.orchestra.run.smithi043.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-13T17:46:26.476 INFO:teuthology.orchestra.run.smithi043.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-13T17:46:26.477 INFO:teuthology.orchestra.run.smithi043.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-13T17:46:26.494 INFO:teuthology.orchestra.run.smithi043.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T17:46:26.495 INFO:teuthology.orchestra.run.smithi043.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-13T17:46:26.496 INFO:teuthology.orchestra.run.smithi043.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-13T17:46:26.496 INFO:teuthology.orchestra.run.smithi043.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-13T17:46:26.497 INFO:teuthology.orchestra.run.smithi043.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-13T17:46:26.606 INFO:teuthology.orchestra.run.smithi040.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.4-860-g6c6b083f-1focal [113 kB] 2024-09-13T17:46:26.674 INFO:teuthology.orchestra.run.smithi040.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.4-860-g6c6b083f-1focal [478 kB] 2024-09-13T17:46:26.681 INFO:teuthology.orchestra.run.smithi040.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.4-860-g6c6b083f-1focal [25.2 MB] 2024-09-13T17:46:27.055 INFO:teuthology.orchestra.run.smithi043.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.4-860-g6c6b083f-1focal [113 kB] 2024-09-13T17:46:27.056 INFO:teuthology.orchestra.run.smithi043.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.4-860-g6c6b083f-1focal [478 kB] 2024-09-13T17:46:27.115 INFO:teuthology.orchestra.run.smithi043.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.4-860-g6c6b083f-1focal [25.2 MB] 2024-09-13T17:46:29.173 INFO:teuthology.orchestra.run.smithi040.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.4-860-g6c6b083f-1focal [9,609 kB] 2024-09-13T17:46:29.820 INFO:teuthology.orchestra.run.smithi040.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.4-860-g6c6b083f-1focal [238 kB] 2024-09-13T17:46:29.832 INFO:teuthology.orchestra.run.smithi040.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.4-860-g6c6b083f-1focal [147 kB] 2024-09-13T17:46:29.843 INFO:teuthology.orchestra.run.smithi040.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.4-860-g6c6b083f-1focal [1,576 kB] 2024-09-13T17:46:29.903 INFO:teuthology.orchestra.run.smithi040.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.4-860-g6c6b083f-1focal [6,641 kB] 2024-09-13T17:46:30.151 INFO:teuthology.orchestra.run.smithi040.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.4-860-g6c6b083f-1focal [24.5 MB] 2024-09-13T17:46:30.363 INFO:teuthology.orchestra.run.smithi043.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.4-860-g6c6b083f-1focal [9,609 kB] 2024-09-13T17:46:30.931 INFO:teuthology.orchestra.run.smithi043.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.4-860-g6c6b083f-1focal [238 kB] 2024-09-13T17:46:30.954 INFO:teuthology.orchestra.run.smithi043.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.4-860-g6c6b083f-1focal [147 kB] 2024-09-13T17:46:30.958 INFO:teuthology.orchestra.run.smithi043.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.4-860-g6c6b083f-1focal [1,576 kB] 2024-09-13T17:46:31.056 INFO:teuthology.orchestra.run.smithi043.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.4-860-g6c6b083f-1focal [6,641 kB] 2024-09-13T17:46:31.172 INFO:teuthology.orchestra.run.smithi040.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.4-860-g6c6b083f-1focal [13.7 kB] 2024-09-13T17:46:31.173 INFO:teuthology.orchestra.run.smithi040.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.4-860-g6c6b083f-1focal [857 kB] 2024-09-13T17:46:31.186 INFO:teuthology.orchestra.run.smithi040.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.4-860-g6c6b083f-1focal [2,296 kB] 2024-09-13T17:46:31.233 INFO:teuthology.orchestra.run.smithi040.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.4-860-g6c6b083f-1focal [239 kB] 2024-09-13T17:46:31.252 INFO:teuthology.orchestra.run.smithi040.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.4-860-g6c6b083f-1focal [143 kB] 2024-09-13T17:46:31.255 INFO:teuthology.orchestra.run.smithi040.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.4-860-g6c6b083f-1focal [7,015 kB] 2024-09-13T17:46:31.443 INFO:teuthology.orchestra.run.smithi043.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.4-860-g6c6b083f-1focal [24.5 MB] 2024-09-13T17:46:31.516 INFO:teuthology.orchestra.run.smithi040.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.4-860-g6c6b083f-1focal [7,894 kB] 2024-09-13T17:46:31.812 INFO:teuthology.orchestra.run.smithi040.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.4-860-g6c6b083f-1focal [13.9 kB] 2024-09-13T17:46:31.812 INFO:teuthology.orchestra.run.smithi040.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.4-860-g6c6b083f-1focal [47.1 MB] 2024-09-13T17:46:33.180 INFO:teuthology.orchestra.run.smithi043.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.4-860-g6c6b083f-1focal [13.7 kB] 2024-09-13T17:46:33.225 INFO:teuthology.orchestra.run.smithi043.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.4-860-g6c6b083f-1focal [857 kB] 2024-09-13T17:46:33.240 INFO:teuthology.orchestra.run.smithi043.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.4-860-g6c6b083f-1focal [2,296 kB] 2024-09-13T17:46:33.287 INFO:teuthology.orchestra.run.smithi043.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.4-860-g6c6b083f-1focal [239 kB] 2024-09-13T17:46:33.293 INFO:teuthology.orchestra.run.smithi043.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.4-860-g6c6b083f-1focal [143 kB] 2024-09-13T17:46:33.318 INFO:teuthology.orchestra.run.smithi043.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.4-860-g6c6b083f-1focal [7,015 kB] 2024-09-13T17:46:33.727 INFO:teuthology.orchestra.run.smithi043.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.4-860-g6c6b083f-1focal [7,894 kB] 2024-09-13T17:46:33.793 INFO:teuthology.orchestra.run.smithi040.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.4-860-g6c6b083f-1focal [120 kB] 2024-09-13T17:46:33.796 INFO:teuthology.orchestra.run.smithi040.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.2.4-860-g6c6b083f-1focal [271 kB] 2024-09-13T17:46:33.801 INFO:teuthology.orchestra.run.smithi040.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.2.4-860-g6c6b083f-1focal [33.0 kB] 2024-09-13T17:46:33.802 INFO:teuthology.orchestra.run.smithi040.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.2.4-860-g6c6b083f-1focal [22.8 kB] 2024-09-13T17:46:33.858 INFO:teuthology.orchestra.run.smithi040.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.4-860-g6c6b083f-1focal [37.4 kB] 2024-09-13T17:46:33.859 INFO:teuthology.orchestra.run.smithi040.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.4-860-g6c6b083f-1focal [13.5 MB] 2024-09-13T17:46:34.166 INFO:teuthology.orchestra.run.smithi043.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.4-860-g6c6b083f-1focal [13.9 kB] 2024-09-13T17:46:34.166 INFO:teuthology.orchestra.run.smithi043.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.4-860-g6c6b083f-1focal [47.1 MB] 2024-09-13T17:46:34.279 INFO:teuthology.orchestra.run.smithi040.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.4-860-g6c6b083f-1focal [108 kB] 2024-09-13T17:46:34.959 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 200 MB in 10s (19.6 MB/s) 2024-09-13T17:46:35.111 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-13T17:46:35.168 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-13T17:46:35.172 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-09-13T17:46:35.209 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-09-13T17:46:35.537 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T17:46:35.553 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-13T17:46:35.596 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T17:46:35.856 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cmd2. 2024-09-13T17:46:35.874 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-09-13T17:46:35.915 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-09-13T17:46:36.234 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-13T17:46:36.251 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-13T17:46:36.343 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T17:46:36.721 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T17:46:36.738 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-13T17:46:36.779 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T17:46:36.948 INFO:teuthology.orchestra.run.smithi043.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.4-860-g6c6b083f-1focal [120 kB] 2024-09-13T17:46:36.951 INFO:teuthology.orchestra.run.smithi043.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.2.4-860-g6c6b083f-1focal [271 kB] 2024-09-13T17:46:36.956 INFO:teuthology.orchestra.run.smithi043.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.2.4-860-g6c6b083f-1focal [33.0 kB] 2024-09-13T17:46:36.957 INFO:teuthology.orchestra.run.smithi043.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.2.4-860-g6c6b083f-1focal [22.8 kB] 2024-09-13T17:46:36.958 INFO:teuthology.orchestra.run.smithi043.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.4-860-g6c6b083f-1focal [37.4 kB] 2024-09-13T17:46:36.959 INFO:teuthology.orchestra.run.smithi043.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.4-860-g6c6b083f-1focal [13.5 MB] 2024-09-13T17:46:37.132 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T17:46:37.153 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-13T17:46:37.191 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T17:46:37.586 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T17:46:37.603 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T17:46:37.655 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T17:46:38.139 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T17:46:38.153 INFO:teuthology.orchestra.run.smithi043.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.4-860-g6c6b083f-1focal [108 kB] 2024-09-13T17:46:38.158 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T17:46:38.197 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T17:46:38.600 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T17:46:38.618 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T17:46:38.658 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T17:46:38.847 INFO:teuthology.orchestra.run.smithi043.stdout:Fetched 200 MB in 14s (14.2 MB/s) 2024-09-13T17:46:38.973 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-13T17:46:39.028 INFO:teuthology.orchestra.run.smithi043.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-13T17:46:39.032 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-09-13T17:46:39.036 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-13T17:46:39.054 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-13T17:46:39.088 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-09-13T17:46:39.095 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T17:46:39.366 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T17:46:39.383 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-13T17:46:39.424 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T17:46:39.615 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../010-librbd1_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:39.740 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librbd1 (18.2.4-860-g6c6b083f-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T17:46:40.077 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-cmd2. 2024-09-13T17:46:40.094 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-09-13T17:46:40.135 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-09-13T17:46:40.435 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../011-librados2_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:40.538 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-13T17:46:40.553 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librados2 (18.2.4-860-g6c6b083f-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T17:46:40.557 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-13T17:46:40.638 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T17:46:41.042 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T17:46:41.060 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-13T17:46:41.100 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T17:46:41.290 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T17:46:41.309 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../012-libcephfs2_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:41.348 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libcephfs2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:41.445 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T17:46:41.464 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-13T17:46:41.503 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T17:46:41.676 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-rados. 2024-09-13T17:46:41.694 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../013-python3-rados_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:41.759 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-rados (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:42.007 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T17:46:42.023 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T17:46:42.070 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T17:46:42.086 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T17:46:42.109 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T17:46:42.120 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-ceph-argparse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:42.364 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T17:46:42.382 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../015-python3-cephfs_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:42.422 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cephfs (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:42.668 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T17:46:42.685 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T17:46:42.726 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T17:46:42.783 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T17:46:42.800 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../016-python3-ceph-common_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T17:46:42.841 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:43.138 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T17:46:43.157 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T17:46:43.170 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T17:46:43.189 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2024-09-13T17:46:43.197 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T17:46:43.228 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-13T17:46:43.497 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T17:46:43.515 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../018-python3-rbd_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:43.555 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-rbd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:43.749 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-13T17:46:43.768 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-13T17:46:43.807 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T17:46:43.850 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-13T17:46:43.867 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-13T17:46:43.899 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T17:46:44.204 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../010-librbd1_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:44.295 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-13T17:46:44.311 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-13T17:46:44.352 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking librbd1 (18.2.4-860-g6c6b083f-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T17:46:44.353 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T17:46:44.695 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T17:46:44.713 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-13T17:46:44.755 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T17:46:45.090 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../011-librados2_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:45.092 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package librgw2. 2024-09-13T17:46:45.109 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../022-librgw2_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:45.141 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librgw2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:45.215 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking librados2 (18.2.4-860-g6c6b083f-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T17:46:45.986 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T17:46:46.002 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T17:46:46.004 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../012-libcephfs2_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:46.019 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../023-python3-rgw_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:46.044 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libcephfs2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:46.052 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-rgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:46.380 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package liboath0. 2024-09-13T17:46:46.396 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-13T17:46:46.438 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-13T17:46:46.513 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-rados. 2024-09-13T17:46:46.529 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../013-python3-rados_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:46.563 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-rados (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:46.766 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T17:46:46.783 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../025-libradosstriper1_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:46.824 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libradosstriper1 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:46.941 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T17:46:46.960 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T17:46:46.999 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-ceph-argparse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:47.269 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-common. 2024-09-13T17:46:47.285 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T17:46:47.285 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../026-ceph-common_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:47.301 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../015-python3-cephfs_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:47.319 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:47.342 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-cephfs (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:47.737 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T17:46:47.756 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../016-python3-ceph-common_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T17:46:47.795 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:48.132 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T17:46:48.152 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2024-09-13T17:46:48.190 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-13T17:46:48.443 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T17:46:48.461 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../018-python3-rbd_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:48.526 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-rbd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:48.887 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-13T17:46:48.903 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-13T17:46:48.945 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T17:46:49.373 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-13T17:46:49.390 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-13T17:46:49.431 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T17:46:49.784 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T17:46:49.801 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-13T17:46:49.804 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-base. 2024-09-13T17:46:49.822 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../027-ceph-base_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:49.859 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T17:46:49.876 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-base (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:50.237 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package librgw2. 2024-09-13T17:46:50.256 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../022-librgw2_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:50.295 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking librgw2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:51.121 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T17:46:51.137 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-13T17:46:51.179 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-13T17:46:51.215 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T17:46:51.233 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../023-python3-rgw_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:51.273 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-rgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:51.549 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T17:46:51.567 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2024-09-13T17:46:51.599 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-13T17:46:51.659 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package liboath0. 2024-09-13T17:46:51.677 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-13T17:46:51.734 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-13T17:46:51.886 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T17:46:51.903 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2024-09-13T17:46:51.944 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-13T17:46:52.053 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T17:46:52.072 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../025-libradosstriper1_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:52.112 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libradosstriper1 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:52.255 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T17:46:52.274 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-13T17:46:52.313 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-13T17:46:52.616 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-mako. 2024-09-13T17:46:52.631 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-common. 2024-09-13T17:46:52.632 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-13T17:46:52.647 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../026-ceph-common_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:52.674 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T17:46:52.689 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:53.027 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T17:46:53.046 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T17:46:53.085 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T17:46:53.363 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T17:46:53.380 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-13T17:46:53.422 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T17:46:53.716 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-webob. 2024-09-13T17:46:53.732 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-13T17:46:53.794 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T17:46:54.102 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-13T17:46:54.122 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-13T17:46:54.160 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T17:46:54.463 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-bs4. 2024-09-13T17:46:54.480 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2024-09-13T17:46:54.521 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-13T17:46:54.899 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T17:46:54.918 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-13T17:46:55.461 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T17:46:56.050 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T17:46:56.067 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2024-09-13T17:46:56.108 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-13T17:46:56.179 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-base. 2024-09-13T17:46:56.197 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../027-ceph-base_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:56.239 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-base (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:56.445 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-paste. 2024-09-13T17:46:56.463 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-13T17:46:56.504 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T17:46:56.815 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T17:46:56.831 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-13T17:46:56.873 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T17:46:57.201 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T17:46:57.217 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-13T17:46:57.259 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-13T17:46:57.529 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T17:46:57.549 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2024-09-13T17:46:57.575 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T17:46:57.587 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-13T17:46:57.591 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-13T17:46:57.633 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-13T17:46:57.907 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T17:46:57.926 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-13T17:46:57.965 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T17:46:58.028 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T17:46:58.045 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2024-09-13T17:46:58.086 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-13T17:46:58.312 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T17:46:58.328 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-13T17:46:58.370 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T17:46:58.381 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T17:46:58.398 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2024-09-13T17:46:58.431 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-13T17:46:58.631 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T17:46:58.648 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T17:46:58.723 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-modules-core (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:58.759 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T17:46:58.777 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-13T17:46:58.817 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-13T17:46:59.082 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T17:46:59.101 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:46:59.140 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:46:59.162 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-mako. 2024-09-13T17:46:59.178 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-13T17:46:59.220 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T17:46:59.459 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T17:46:59.475 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../048-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-13T17:46:59.534 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T17:46:59.590 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T17:46:59.608 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T17:46:59.647 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T17:46:59.844 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cryptography. 2024-09-13T17:46:59.860 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../049-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-13T17:46:59.902 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T17:46:59.942 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T17:46:59.960 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-13T17:46:59.992 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T17:47:00.315 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T17:47:00.334 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../050-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-13T17:47:00.337 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-webob. 2024-09-13T17:47:00.354 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-13T17:47:00.373 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-13T17:47:00.395 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T17:47:00.726 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T17:47:00.743 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../051-ceph-mgr_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:00.756 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-13T17:47:00.773 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-13T17:47:00.784 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:00.889 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T17:47:01.209 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-bs4. 2024-09-13T17:47:01.228 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2024-09-13T17:47:01.267 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-13T17:47:01.322 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T17:47:01.339 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../052-ceph-mon_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:01.379 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mon (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:01.685 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T17:47:01.702 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-13T17:47:01.746 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T17:47:02.134 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T17:47:02.153 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2024-09-13T17:47:02.198 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-13T17:47:02.258 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T17:47:02.275 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../053-ceph-osd_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:02.316 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-osd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:02.526 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-paste. 2024-09-13T17:47:02.542 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-13T17:47:02.584 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T17:47:02.962 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T17:47:02.978 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-13T17:47:03.020 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T17:47:03.357 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T17:47:03.374 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-13T17:47:03.415 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-13T17:47:03.776 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T17:47:03.796 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2024-09-13T17:47:03.834 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-13T17:47:04.171 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T17:47:04.189 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-13T17:47:04.229 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T17:47:04.586 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph. 2024-09-13T17:47:04.605 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../054-ceph_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:04.621 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T17:47:04.638 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-13T17:47:04.644 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:04.679 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T17:47:04.955 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T17:47:04.974 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../055-ceph-fuse_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:05.013 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:05.016 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T17:47:05.032 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T17:47:05.099 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mgr-modules-core (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:05.442 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T17:47:05.461 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../056-ceph-mds_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:05.486 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T17:47:05.499 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mds (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:05.503 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:05.544 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:05.922 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T17:47:05.980 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../048-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-13T17:47:05.980 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T17:47:06.053 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package cephadm. 2024-09-13T17:47:06.071 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../057-cephadm_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:06.111 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:06.308 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-cryptography. 2024-09-13T17:47:06.325 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../049-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-13T17:47:06.372 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T17:47:06.391 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../058-python3-asyncssh_1.12.2-1_all.deb ... 2024-09-13T17:47:06.391 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T17:47:06.430 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-09-13T17:47:06.750 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T17:47:06.770 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-13T17:47:06.804 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T17:47:06.821 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../050-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-13T17:47:06.823 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T17:47:06.862 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-13T17:47:07.161 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T17:47:07.182 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T17:47:07.219 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:07.240 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T17:47:07.260 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../051-ceph-mgr_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:07.290 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mgr (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:07.573 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-jwt. 2024-09-13T17:47:07.589 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-13T17:47:07.656 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T17:47:07.827 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T17:47:07.843 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../052-ceph-mon_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:07.885 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mon (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:07.942 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-attr. 2024-09-13T17:47:07.960 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../062-python3-attr_19.3.0-2_all.deb ... 2024-09-13T17:47:08.000 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-13T17:47:08.320 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T17:47:08.339 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../063-python3-pluggy_0.13.0-2_all.deb ... 2024-09-13T17:47:08.378 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-13T17:47:08.709 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-py. 2024-09-13T17:47:08.725 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../064-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-13T17:47:08.759 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T17:47:08.930 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T17:47:08.946 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../053-ceph-osd_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:08.988 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-osd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:09.037 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-13T17:47:09.057 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../065-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-13T17:47:09.095 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T17:47:09.398 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-packaging. 2024-09-13T17:47:09.418 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../066-python3-packaging_20.3-1_all.deb ... 2024-09-13T17:47:09.456 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-13T17:47:09.750 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T17:47:09.770 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../067-python3-pytest_4.6.9-1_all.deb ... 2024-09-13T17:47:09.808 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-13T17:47:10.162 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T17:47:10.178 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../068-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T17:47:10.220 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T17:47:10.456 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libjs-underscore. 2024-09-13T17:47:10.472 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../069-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-13T17:47:10.514 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T17:47:10.850 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-13T17:47:10.867 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../070-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-13T17:47:10.908 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T17:47:11.467 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph. 2024-09-13T17:47:11.485 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../054-ceph_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:11.661 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-routes. 2024-09-13T17:47:11.681 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../071-python3-routes_2.4.1-2_all.deb ... 2024-09-13T17:47:11.719 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-13T17:47:11.800 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:12.064 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T17:47:12.069 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T17:47:12.080 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T17:47:12.086 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../055-ceph-fuse_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:12.119 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:12.122 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-dashboard (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:12.606 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T17:47:12.622 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../056-ceph-mds_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:12.664 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mds (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:13.201 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package cephadm. 2024-09-13T17:47:13.220 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../057-cephadm_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:13.258 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:13.528 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-decorator. 2024-09-13T17:47:13.544 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../073-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-13T17:47:13.586 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T17:47:13.605 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T17:47:13.624 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../058-python3-asyncssh_1.12.2-1_all.deb ... 2024-09-13T17:47:13.661 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-09-13T17:47:13.872 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-13T17:47:13.888 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../074-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-13T17:47:13.930 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T17:47:14.015 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T17:47:14.033 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-13T17:47:14.073 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T17:47:14.283 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-scipy. 2024-09-13T17:47:14.300 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../075-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-13T17:47:14.341 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-13T17:47:14.442 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T17:47:14.461 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T17:47:14.500 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mgr-cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:14.905 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-jwt. 2024-09-13T17:47:14.925 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-13T17:47:14.971 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T17:47:15.308 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-attr. 2024-09-13T17:47:15.326 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../062-python3-attr_19.3.0-2_all.deb ... 2024-09-13T17:47:15.366 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-13T17:47:15.692 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-13T17:47:15.709 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../076-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-13T17:47:15.736 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T17:47:15.750 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T17:47:15.756 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../063-python3-pluggy_0.13.0-2_all.deb ... 2024-09-13T17:47:15.785 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-13T17:47:16.122 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-py. 2024-09-13T17:47:16.141 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../064-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-13T17:47:16.180 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T17:47:16.262 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T17:47:16.282 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../077-python3-joblib_0.14.0-3_all.deb ... 2024-09-13T17:47:16.320 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-13T17:47:16.558 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-13T17:47:16.575 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../065-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-13T17:47:16.616 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T17:47:16.749 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T17:47:16.769 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../078-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-13T17:47:16.799 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T17:47:16.944 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-packaging. 2024-09-13T17:47:16.960 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../066-python3-packaging_20.3-1_all.deb ... 2024-09-13T17:47:17.027 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-13T17:47:17.355 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T17:47:17.371 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../067-python3-pytest_4.6.9-1_all.deb ... 2024-09-13T17:47:17.413 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-13T17:47:17.433 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T17:47:17.454 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../079-ceph-mgr-diskprediction-local_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T17:47:17.491 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:17.800 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T17:47:17.820 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../068-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T17:47:17.858 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T17:47:18.119 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libjs-underscore. 2024-09-13T17:47:18.135 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../069-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-13T17:47:18.177 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T17:47:18.504 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-13T17:47:18.522 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../070-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-13T17:47:18.587 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T17:47:18.594 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T17:47:18.614 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../080-python3-cachetools_4.0.0-1_all.deb ... 2024-09-13T17:47:18.652 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-13T17:47:18.941 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-routes. 2024-09-13T17:47:18.958 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../071-python3-routes_2.4.1-2_all.deb ... 2024-09-13T17:47:18.998 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-13T17:47:19.069 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T17:47:19.089 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../081-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-13T17:47:19.127 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T17:47:19.393 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T17:47:19.410 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T17:47:19.449 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T17:47:19.451 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mgr-dashboard (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:19.465 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../082-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-13T17:47:19.507 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T17:47:19.852 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T17:47:19.869 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../083-python3-rsa_4.0-3_all.deb ... 2024-09-13T17:47:19.910 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-13T17:47:20.214 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T17:47:20.232 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../084-python3-google-auth_1.5.1-2_all.deb ... 2024-09-13T17:47:20.272 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-13T17:47:20.617 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-blinker. 2024-09-13T17:47:20.634 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../085-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-13T17:47:20.675 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T17:47:20.923 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-decorator. 2024-09-13T17:47:20.940 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../073-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-13T17:47:20.966 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-13T17:47:20.981 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T17:47:20.982 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../086-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-13T17:47:21.024 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T17:47:21.284 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-13T17:47:21.304 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../074-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-13T17:47:21.342 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T17:47:21.378 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T17:47:21.396 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-13T17:47:21.436 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T17:47:21.728 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-scipy. 2024-09-13T17:47:21.741 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T17:47:21.746 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../075-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-13T17:47:21.758 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../088-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-13T17:47:21.786 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-13T17:47:21.797 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T17:47:22.169 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T17:47:22.190 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../089-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-13T17:47:22.227 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T17:47:22.774 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T17:47:22.792 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T17:47:22.853 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:23.180 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-13T17:47:23.199 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../076-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-13T17:47:23.202 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T17:47:23.220 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../091-libonig5_6.9.4-1_amd64.deb ... 2024-09-13T17:47:23.237 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T17:47:23.260 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-13T17:47:23.613 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T17:47:23.631 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../092-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T17:47:23.688 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T17:47:23.775 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T17:47:23.794 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../077-python3-joblib_0.14.0-3_all.deb ... 2024-09-13T17:47:23.867 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-13T17:47:23.924 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package jq. 2024-09-13T17:47:23.942 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../093-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T17:47:23.990 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T17:47:24.243 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package socat. 2024-09-13T17:47:24.263 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../094-socat_1.7.3.3-2_amd64.deb ... 2024-09-13T17:47:24.270 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T17:47:24.287 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../078-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-13T17:47:24.301 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-13T17:47:24.349 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T17:47:24.647 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T17:47:24.663 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-13T17:47:24.704 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-13T17:47:25.051 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-test. 2024-09-13T17:47:25.068 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../096-ceph-test_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:25.091 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T17:47:25.108 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-test (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:25.110 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../079-ceph-mgr-diskprediction-local_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T17:47:25.174 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:26.435 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T17:47:26.451 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../080-python3-cachetools_4.0.0-1_all.deb ... 2024-09-13T17:47:26.493 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-13T17:47:26.940 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T17:47:26.958 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../081-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-13T17:47:26.998 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T17:47:27.378 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T17:47:27.397 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../082-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-13T17:47:27.436 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T17:47:27.757 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T17:47:27.777 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../083-python3-rsa_4.0-3_all.deb ... 2024-09-13T17:47:27.814 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-13T17:47:28.189 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T17:47:28.206 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../084-python3-google-auth_1.5.1-2_all.deb ... 2024-09-13T17:47:28.246 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-13T17:47:28.601 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-blinker. 2024-09-13T17:47:28.618 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../085-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-13T17:47:28.675 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T17:47:28.995 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-13T17:47:29.011 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../086-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-13T17:47:29.052 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T17:47:29.363 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T17:47:29.383 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T17:47:29.422 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-volume (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:29.475 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T17:47:29.492 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-13T17:47:29.533 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T17:47:29.787 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-13T17:47:29.806 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../098-cephfs-mirror_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:29.844 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking cephfs-mirror (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:29.923 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T17:47:29.941 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../088-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-13T17:47:29.986 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T17:47:30.206 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-colorama. 2024-09-13T17:47:30.224 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../099-python3-colorama_0.4.3-1build1_all.deb ... 2024-09-13T17:47:30.264 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-09-13T17:47:30.331 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T17:47:30.348 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../089-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-13T17:47:30.389 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T17:47:30.520 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package cephfs-shell. 2024-09-13T17:47:30.538 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../100-cephfs-shell_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T17:47:30.578 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking cephfs-shell (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:30.839 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package cephfs-top. 2024-09-13T17:47:30.859 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../101-cephfs-top_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T17:47:30.922 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking cephfs-top (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:31.053 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T17:47:31.073 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T17:47:31.111 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:31.292 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-13T17:47:31.309 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../102-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T17:47:31.350 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T17:47:31.464 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T17:47:31.485 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../091-libonig5_6.9.4-1_amd64.deb ... 2024-09-13T17:47:31.555 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-13T17:47:31.847 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T17:47:31.864 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../103-libcephfs-dev_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:31.905 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libcephfs-dev (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:31.933 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T17:47:31.951 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../092-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T17:47:31.991 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T17:47:32.300 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T17:47:32.317 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../104-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-13T17:47:32.319 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package jq. 2024-09-13T17:47:32.338 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../093-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T17:47:32.376 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T17:47:32.377 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T17:47:32.639 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package socat. 2024-09-13T17:47:32.657 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../094-socat_1.7.3.3-2_amd64.deb ... 2024-09-13T17:47:32.680 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T17:47:32.696 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-13T17:47:32.697 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../105-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-09-13T17:47:32.738 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-09-13T17:47:33.042 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T17:47:33.059 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-13T17:47:33.083 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-13T17:47:33.100 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../106-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T17:47:33.125 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-13T17:47:33.141 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T17:47:33.483 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-test. 2024-09-13T17:47:33.499 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../096-ceph-test_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:33.541 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-test (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:34.747 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-13T17:47:34.765 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../107-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-13T17:47:34.796 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-13T17:47:35.116 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-webencodings. 2024-09-13T17:47:35.135 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../108-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-13T17:47:35.174 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T17:47:35.479 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-html5lib. 2024-09-13T17:47:35.499 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../109-python3-html5lib_1.0.1-2_all.deb ... 2024-09-13T17:47:35.537 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-13T17:47:35.898 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-13T17:47:35.919 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../110-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-13T17:47:35.956 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T17:47:36.401 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T17:47:36.418 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../111-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T17:47:36.460 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T17:47:36.807 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T17:47:36.825 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../112-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-13T17:47:36.865 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T17:47:37.202 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T17:47:37.222 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../113-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-13T17:47:37.260 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T17:47:37.579 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T17:47:37.600 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-13T17:47:37.637 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T17:47:37.907 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T17:47:37.925 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-13T17:47:37.944 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T17:47:37.965 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T17:47:37.965 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T17:47:38.011 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-volume (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:38.435 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-13T17:47:38.454 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../098-cephfs-mirror_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:38.493 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking cephfs-mirror (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:38.654 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package radosgw. 2024-09-13T17:47:38.670 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../116-radosgw_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:38.711 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking radosgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:38.838 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-colorama. 2024-09-13T17:47:38.855 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../099-python3-colorama_0.4.3-1build1_all.deb ... 2024-09-13T17:47:38.937 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-09-13T17:47:39.209 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package cephfs-shell. 2024-09-13T17:47:39.226 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../100-cephfs-shell_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T17:47:39.267 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking cephfs-shell (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:39.538 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package cephfs-top. 2024-09-13T17:47:39.556 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../101-cephfs-top_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T17:47:39.596 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking cephfs-top (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:39.882 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-13T17:47:39.899 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../102-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T17:47:39.965 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T17:47:40.132 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T17:47:40.150 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../117-rbd-fuse_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:40.189 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking rbd-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:40.503 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T17:47:40.508 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-13T17:47:40.519 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../103-libcephfs-dev_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:40.552 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libcephfs-dev (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:40.836 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T17:47:40.962 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T17:47:41.014 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T17:47:41.031 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../104-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-13T17:47:41.088 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T17:47:41.089 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T17:47:41.290 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:41.427 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T17:47:41.445 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../105-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-09-13T17:47:41.510 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-09-13T17:47:41.866 INFO:teuthology.orchestra.run.smithi040.stdout:Adding system user cephadm....done 2024-09-13T17:47:41.971 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-13T17:47:41.988 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../106-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T17:47:42.054 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T17:47:42.059 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T17:47:42.606 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T17:47:42.991 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T17:47:43.267 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T17:47:43.368 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-13T17:47:43.485 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T17:47:43.812 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-13T17:47:44.167 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-13T17:47:44.189 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../107-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-13T17:47:44.206 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T17:47:44.258 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-13T17:47:44.499 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-13T17:47:44.620 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-webencodings. 2024-09-13T17:47:44.641 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../108-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-13T17:47:44.678 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T17:47:44.800 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-09-13T17:47:45.058 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-html5lib. 2024-09-13T17:47:45.059 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T17:47:45.076 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../109-python3-html5lib_1.0.1-2_all.deb ... 2024-09-13T17:47:45.116 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-13T17:47:45.311 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T17:47:45.522 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-13T17:47:45.542 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../110-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-13T17:47:45.586 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T17:47:45.612 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-ceph-argparse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:45.896 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T17:47:46.022 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-09-13T17:47:46.089 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T17:47:46.110 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../111-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T17:47:46.147 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T17:47:46.299 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-13T17:47:46.500 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T17:47:46.521 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../112-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-13T17:47:46.583 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T17:47:46.600 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T17:47:46.859 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T17:47:46.955 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T17:47:46.974 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../113-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-13T17:47:47.010 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T17:47:47.012 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T17:47:47.390 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T17:47:47.394 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-13T17:47:47.408 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-13T17:47:47.448 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T17:47:47.688 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T17:47:47.734 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T17:47:47.751 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-13T17:47:47.792 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T17:47:47.980 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T17:47:48.323 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T17:47:48.571 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package radosgw. 2024-09-13T17:47:48.589 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../116-radosgw_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:48.616 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T17:47:48.629 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking radosgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:48.942 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-13T17:47:49.218 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-13T17:47:49.503 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T17:47:49.637 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T17:47:49.763 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T17:47:50.031 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T17:47:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T17:47:50.227 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../117-rbd-fuse_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T17:47:50.265 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking rbd-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:50.433 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-09-13T17:47:50.589 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-13T17:47:50.700 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T17:47:50.912 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T17:47:51.063 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T17:47:51.069 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T17:47:51.189 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T17:47:51.399 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:51.437 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-13T17:47:51.563 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:51.890 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T17:47:52.016 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-13T17:47:52.125 INFO:teuthology.orchestra.run.smithi043.stdout:Adding system user cephadm....done 2024-09-13T17:47:52.322 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T17:47:52.367 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T17:47:52.448 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T17:47:52.477 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-13T17:47:52.619 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-13T17:47:52.820 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T17:47:52.913 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T17:47:53.134 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T17:47:53.222 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T17:47:53.260 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-13T17:47:53.373 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T17:47:53.419 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T17:47:53.499 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T17:47:53.801 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T17:47:53.896 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-13T17:47:54.086 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-13T17:47:54.315 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T17:47:54.412 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T17:47:54.538 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T17:47:54.749 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-13T17:47:54.823 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-13T17:47:55.034 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-09-13T17:47:55.124 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T17:47:55.327 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T17:47:55.425 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T17:47:55.611 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T17:47:55.821 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-13T17:47:55.912 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-ceph-argparse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:47:56.094 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-13T17:47:56.213 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T17:47:56.220 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T17:47:56.356 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-09-13T17:47:56.563 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-13T17:47:56.632 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-13T17:47:56.915 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T17:47:56.916 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T17:47:57.041 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T17:47:57.166 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T17:47:57.209 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T17:47:57.368 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T17:47:57.493 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T17:47:57.753 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-13T17:47:57.777 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-13T17:47:58.054 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T17:47:58.403 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T17:47:58.765 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T17:47:59.058 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T17:47:59.433 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T17:47:59.436 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-13T17:47:59.551 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-13T17:47:59.751 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-13T17:47:59.894 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-13T17:48:00.053 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T17:48:00.178 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T17:48:00.195 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-13T17:48:00.338 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T17:48:00.471 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T17:48:00.614 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-13T17:48:00.622 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T17:48:00.957 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T17:48:01.049 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-09-13T17:48:01.275 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-09-13T17:48:01.358 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T17:48:01.652 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T17:48:01.726 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T17:48:01.978 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T17:48:02.120 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-13T17:48:02.246 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:02.605 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T17:48:02.739 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-13T17:48:03.133 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T17:48:03.159 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T17:48:03.258 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-13T17:48:03.268 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-13T17:48:03.394 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T17:48:03.409 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-13T17:48:03.520 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-13T17:48:03.711 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T17:48:03.821 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T17:48:03.947 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-13T17:48:04.045 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T17:48:04.196 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T17:48:04.273 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T17:48:04.322 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T17:48:04.608 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T17:48:04.716 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T17:48:04.918 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T17:48:05.017 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-13T17:48:05.256 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librados2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:05.310 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T17:48:05.469 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T17:48:05.506 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-09-13T17:48:05.632 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librgw2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:05.745 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-13T17:48:05.758 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libsqlite3-mod-ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:05.884 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libcephfs2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:06.010 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-13T17:48:06.029 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T17:48:06.312 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libradosstriper1 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:06.339 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T17:48:06.438 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-13T17:48:06.699 INFO:teuthology.orchestra.run.smithi043.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-13T17:48:06.748 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librbd1 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:06.882 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-modules-core (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:07.034 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:07.041 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-13T17:48:07.167 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T17:48:07.257 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T17:48:07.257 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T17:48:07.485 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-13T17:48:07.718 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libcephfs-dev (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:07.861 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-rados (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:07.870 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T17:48:07.987 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up cephfs-top (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:07.988 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T17:48:08.113 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T17:48:08.147 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T17:48:08.498 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T17:48:08.856 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-13T17:48:09.109 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-rbd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:09.235 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up rbd-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:09.386 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-rgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:09.512 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cephfs (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:09.647 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up cephfs-shell (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:09.773 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:10.267 INFO:teuthology.orchestra.run.smithi040.stdout:Adding group ceph....done 2024-09-13T17:48:10.388 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T17:48:10.539 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-13T17:48:10.636 INFO:teuthology.orchestra.run.smithi040.stdout:Adding system user ceph....done 2024-09-13T17:48:10.803 INFO:teuthology.orchestra.run.smithi040.stdout:Setting system user ceph properties....done 2024-09-13T17:48:10.827 INFO:teuthology.orchestra.run.smithi040.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T17:48:10.957 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-13T17:48:10.991 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T17:48:11.283 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-13T17:48:11.318 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T17:48:11.601 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T17:48:11.769 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-13T17:48:11.783 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up cephfs-mirror (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:11.994 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T17:48:11.994 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T17:48:12.120 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T17:48:12.426 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-test (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:12.471 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-09-13T17:48:12.585 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up radosgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:12.914 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T17:48:13.067 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T17:48:13.068 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T17:48:13.316 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T17:48:13.612 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-base (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:14.002 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T17:48:14.396 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T17:48:14.448 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mds (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:14.522 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-13T17:48:14.681 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T17:48:14.681 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T17:48:14.682 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T17:48:14.807 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-13T17:48:15.124 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:15.150 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T17:48:15.251 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-13T17:48:15.357 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T17:48:15.357 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T17:48:15.561 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T17:48:15.792 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-osd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:15.920 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T17:48:16.255 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T17:48:16.289 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T17:48:16.289 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T17:48:16.623 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up librados2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:16.744 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-k8sevents (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:16.860 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-09-13T17:48:16.870 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:17.013 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mon (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:17.019 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up librgw2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:17.145 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libsqlite3-mod-ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:17.246 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T17:48:17.246 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T17:48:17.305 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libcephfs2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:17.455 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-13T17:48:17.706 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:17.774 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libradosstriper1 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:17.840 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:17.900 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-13T17:48:17.941 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-dashboard (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:18.059 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-volume (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:18.302 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up librbd1 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:18.444 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mgr-modules-core (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:18.462 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T17:48:18.622 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T17:48:19.004 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:19.008 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T17:48:19.234 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T17:48:19.235 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T17:48:19.739 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libcephfs-dev (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:19.865 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-rados (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:20.024 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up cephfs-top (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:20.125 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T17:48:21.115 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-rbd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:21.224 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up rbd-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:21.367 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-rgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:21.476 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-cephfs (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:21.644 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up cephfs-shell (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:21.770 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:21.927 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-13T17:48:22.423 INFO:teuthology.orchestra.run.smithi043.stdout:Adding group ceph....done 2024-09-13T17:48:22.542 INFO:teuthology.orchestra.run.smithi040.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T17:48:22.545 DEBUG:teuthology.orchestra.run.smithi040:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-13T17:48:22.618 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2024-09-13T17:48:22.832 INFO:teuthology.orchestra.run.smithi043.stdout:Adding system user ceph....done 2024-09-13T17:48:22.842 INFO:teuthology.orchestra.run.smithi040.stdout:Building dependency tree... 2024-09-13T17:48:22.843 INFO:teuthology.orchestra.run.smithi040.stdout:Reading state information... 2024-09-13T17:48:23.051 INFO:teuthology.orchestra.run.smithi043.stdout:Setting system user ceph properties....done 2024-09-13T17:48:23.073 INFO:teuthology.orchestra.run.smithi040.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-09-13T17:48:23.073 INFO:teuthology.orchestra.run.smithi040.stdout:flex is already the newest version (2.6.4-6.2). 2024-09-13T17:48:23.073 INFO:teuthology.orchestra.run.smithi040.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-09-13T17:48:23.073 INFO:teuthology.orchestra.run.smithi040.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-09-13T17:48:23.073 INFO:teuthology.orchestra.run.smithi040.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-09-13T17:48:23.073 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T17:48:23.074 INFO:teuthology.orchestra.run.smithi043.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T17:48:23.074 INFO:teuthology.orchestra.run.smithi040.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T17:48:23.074 INFO:teuthology.orchestra.run.smithi040.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T17:48:23.074 INFO:teuthology.orchestra.run.smithi040.stdout:The following additional packages will be installed: 2024-09-13T17:48:23.075 INFO:teuthology.orchestra.run.smithi040.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-09-13T17:48:23.075 INFO:teuthology.orchestra.run.smithi040.stdout: libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 2024-09-13T17:48:23.075 INFO:teuthology.orchestra.run.smithi040.stdout: libteamdctl0 libuuid1 modemmanager network-manager-pptp postgresql-12 2024-09-13T17:48:23.075 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-client-12 postgresql-client-common ppp pptp-linux usb-modeswitch 2024-09-13T17:48:23.075 INFO:teuthology.orchestra.run.smithi040.stdout: usb-modeswitch-data wpasupplicant 2024-09-13T17:48:23.077 INFO:teuthology.orchestra.run.smithi040.stdout:Suggested packages: 2024-09-13T17:48:23.077 INFO:teuthology.orchestra.run.smithi040.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2024-09-13T17:48:23.077 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-doc-12 libjson-perl comgt wvdial util-linux-locales wpagui 2024-09-13T17:48:23.077 INFO:teuthology.orchestra.run.smithi040.stdout: libengine-pkcs11-openssl 2024-09-13T17:48:23.141 INFO:teuthology.orchestra.run.smithi040.stdout:The following NEW packages will be installed: 2024-09-13T17:48:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-09-13T17:48:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2024-09-13T17:48:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-09-13T17:48:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2024-09-13T17:48:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2024-09-13T17:48:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-13T17:48:23.143 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages will be upgraded: 2024-09-13T17:48:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-09-13T17:48:23.232 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T17:48:23.330 INFO:teuthology.orchestra.run.smithi040.stdout:6 upgraded, 31 newly installed, 0 to remove and 289 not upgraded. 2024-09-13T17:48:23.330 INFO:teuthology.orchestra.run.smithi040.stdout:Need to get 25.4 MB of archives. 2024-09-13T17:48:23.330 INFO:teuthology.orchestra.run.smithi040.stdout:After this operation, 80.0 MB of additional disk space will be used. 2024-09-13T17:48:23.330 INFO:teuthology.orchestra.run.smithi040.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-09-13T17:48:23.564 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T17:48:23.955 INFO:teuthology.orchestra.run.smithi040.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-09-13T17:48:23.957 INFO:teuthology.orchestra.run.smithi040.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-09-13T17:48:23.958 INFO:teuthology.orchestra.run.smithi040.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-09-13T17:48:23.970 INFO:teuthology.orchestra.run.smithi040.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-09-13T17:48:23.973 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.23 [1,323 kB] 2024-09-13T17:48:24.030 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up cephfs-mirror (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:24.072 INFO:teuthology.orchestra.run.smithi040.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-09-13T17:48:24.076 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-09-13T17:48:24.082 INFO:teuthology.orchestra.run.smithi040.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-09-13T17:48:24.086 INFO:teuthology.orchestra.run.smithi040.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-09-13T17:48:24.087 INFO:teuthology.orchestra.run.smithi040.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.8 [60.6 kB] 2024-09-13T17:48:24.119 INFO:teuthology.orchestra.run.smithi040.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-09-13T17:48:24.121 INFO:teuthology.orchestra.run.smithi040.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-09-13T17:48:24.124 INFO:teuthology.orchestra.run.smithi040.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-09-13T17:48:24.125 INFO:teuthology.orchestra.run.smithi040.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-09-13T17:48:24.201 INFO:teuthology.orchestra.run.smithi040.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libndp0 amd64 1.7-0ubuntu1.1 [11.1 kB] 2024-09-13T17:48:24.202 INFO:teuthology.orchestra.run.smithi040.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-09-13T17:48:24.211 INFO:teuthology.orchestra.run.smithi040.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.20-0ubuntu0.20.04.1 [117 kB] 2024-09-13T17:48:24.215 INFO:teuthology.orchestra.run.smithi040.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-09-13T17:48:24.229 INFO:teuthology.orchestra.run.smithi040.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-09-13T17:48:24.230 INFO:teuthology.orchestra.run.smithi040.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.23 [1,586 kB] 2024-09-13T17:48:24.284 INFO:teuthology.orchestra.run.smithi040.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-09-13T17:48:24.284 INFO:teuthology.orchestra.run.smithi040.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-09-13T17:48:24.306 INFO:teuthology.orchestra.run.smithi040.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.4 [1,184 kB] 2024-09-13T17:48:24.334 INFO:teuthology.orchestra.run.smithi040.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-09-13T17:48:24.392 INFO:teuthology.orchestra.run.smithi040.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-09-13T17:48:24.399 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T17:48:24.399 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T17:48:24.399 INFO:teuthology.orchestra.run.smithi040.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-09-13T17:48:24.400 INFO:teuthology.orchestra.run.smithi040.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-09-13T17:48:24.404 INFO:teuthology.orchestra.run.smithi040.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2024-09-13T17:48:24.405 INFO:teuthology.orchestra.run.smithi040.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.20-0ubuntu0.20.04.1 [1,055 kB] 2024-09-13T17:48:24.428 INFO:teuthology.orchestra.run.smithi040.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2024-09-13T17:48:24.448 INFO:teuthology.orchestra.run.smithi040.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.20-0ubuntu0.20.04.1 [13.5 MB] 2024-09-13T17:48:24.831 INFO:teuthology.orchestra.run.smithi040.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2024-09-13T17:48:24.832 INFO:teuthology.orchestra.run.smithi040.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2024-09-13T17:48:24.849 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-test (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:24.900 INFO:teuthology.orchestra.run.smithi040.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2024-09-13T17:48:24.900 INFO:teuthology.orchestra.run.smithi040.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-09-13T17:48:24.900 INFO:teuthology.orchestra.run.smithi040.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-09-13T17:48:25.008 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up radosgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:25.330 INFO:teuthology.orchestra.run.smithi040.stdout:Preconfiguring packages ... 2024-09-13T17:48:25.481 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T17:48:25.481 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T17:48:25.530 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 25.4 MB in 2s (14.5 MB/s) 2024-09-13T17:48:25.762 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153099 files and directories currently installed.) 2024-09-13T17:48:25.767 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T17:48:25.914 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T17:48:25.918 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-base (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:26.352 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T17:48:26.845 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mds (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:27.069 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T17:48:27.069 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T17:48:27.088 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-09-13T17:48:27.513 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mgr (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:27.763 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T17:48:27.763 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T17:48:28.215 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-osd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:28.499 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153099 files and directories currently installed.) 2024-09-13T17:48:28.504 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T17:48:28.661 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T17:48:28.741 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T17:48:28.741 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T17:48:29.068 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T17:48:29.183 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mgr-k8sevents (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:29.309 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:29.552 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T17:48:29.552 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mon (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:29.818 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T17:48:29.818 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T17:48:30.254 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mgr-cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:30.332 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-09-13T17:48:30.416 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:30.548 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mgr-dashboard (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:30.699 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-volume (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T17:48:30.774 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153099 files and directories currently installed.) 2024-09-13T17:48:30.779 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T17:48:30.911 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-09-13T17:48:31.060 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T17:48:31.253 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T17:48:31.361 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T17:48:31.493 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-09-13T17:48:31.555 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T17:48:31.998 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-09-13T17:48:32.131 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.23) over (1.1.1f-1ubuntu2.16) ... 2024-09-13T17:48:32.612 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-13T17:48:32.632 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-09-13T17:48:32.668 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-09-13T17:48:32.988 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package dump. 2024-09-13T17:48:33.083 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-09-13T17:48:33.083 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking dump (0.4b46-6) ... 2024-09-13T17:48:33.350 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package indent. 2024-09-13T17:48:33.367 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T17:48:33.433 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-09-13T17:48:33.736 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-13T17:48:33.754 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-09-13T17:48:33.786 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-09-13T17:48:34.089 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-13T17:48:34.106 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.8_amd64.deb ... 2024-09-13T17:48:34.156 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-09-13T17:48:34.459 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-13T17:48:34.477 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T17:48:34.507 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-13T17:48:34.517 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T17:48:34.878 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-13T17:48:34.895 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T17:48:34.936 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T17:48:35.206 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-13T17:48:35.224 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T17:48:35.264 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T17:48:35.264 INFO:teuthology.orchestra.run.smithi043.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T17:48:35.267 DEBUG:teuthology.orchestra.run.smithi043:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-13T17:48:35.346 INFO:teuthology.orchestra.run.smithi043.stdout:Reading package lists... 2024-09-13T17:48:35.574 INFO:teuthology.orchestra.run.smithi043.stdout:Building dependency tree... 2024-09-13T17:48:35.575 INFO:teuthology.orchestra.run.smithi043.stdout:Reading state information... 2024-09-13T17:48:35.802 INFO:teuthology.orchestra.run.smithi043.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-09-13T17:48:35.802 INFO:teuthology.orchestra.run.smithi043.stdout:flex is already the newest version (2.6.4-6.2). 2024-09-13T17:48:35.802 INFO:teuthology.orchestra.run.smithi043.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-09-13T17:48:35.802 INFO:teuthology.orchestra.run.smithi043.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-09-13T17:48:35.802 INFO:teuthology.orchestra.run.smithi043.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-09-13T17:48:35.802 INFO:teuthology.orchestra.run.smithi043.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T17:48:35.803 INFO:teuthology.orchestra.run.smithi043.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T17:48:35.803 INFO:teuthology.orchestra.run.smithi043.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T17:48:35.803 INFO:teuthology.orchestra.run.smithi043.stdout:The following additional packages will be installed: 2024-09-13T17:48:35.804 INFO:teuthology.orchestra.run.smithi043.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-09-13T17:48:35.804 INFO:teuthology.orchestra.run.smithi043.stdout: libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 2024-09-13T17:48:35.804 INFO:teuthology.orchestra.run.smithi043.stdout: libteamdctl0 libuuid1 modemmanager network-manager-pptp postgresql-12 2024-09-13T17:48:35.804 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-client-12 postgresql-client-common ppp pptp-linux usb-modeswitch 2024-09-13T17:48:35.804 INFO:teuthology.orchestra.run.smithi043.stdout: usb-modeswitch-data wpasupplicant 2024-09-13T17:48:35.805 INFO:teuthology.orchestra.run.smithi043.stdout:Suggested packages: 2024-09-13T17:48:35.805 INFO:teuthology.orchestra.run.smithi043.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2024-09-13T17:48:35.805 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-doc-12 libjson-perl comgt wvdial util-linux-locales wpagui 2024-09-13T17:48:35.806 INFO:teuthology.orchestra.run.smithi043.stdout: libengine-pkcs11-openssl 2024-09-13T17:48:35.868 INFO:teuthology.orchestra.run.smithi043.stdout:The following NEW packages will be installed: 2024-09-13T17:48:35.868 INFO:teuthology.orchestra.run.smithi043.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-09-13T17:48:35.868 INFO:teuthology.orchestra.run.smithi043.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2024-09-13T17:48:35.869 INFO:teuthology.orchestra.run.smithi043.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-09-13T17:48:35.869 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2024-09-13T17:48:35.869 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2024-09-13T17:48:35.869 INFO:teuthology.orchestra.run.smithi043.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-13T17:48:35.870 INFO:teuthology.orchestra.run.smithi043.stdout:The following packages will be upgraded: 2024-09-13T17:48:35.871 INFO:teuthology.orchestra.run.smithi043.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-09-13T17:48:36.057 INFO:teuthology.orchestra.run.smithi043.stdout:6 upgraded, 31 newly installed, 0 to remove and 289 not upgraded. 2024-09-13T17:48:36.057 INFO:teuthology.orchestra.run.smithi043.stdout:Need to get 25.4 MB of archives. 2024-09-13T17:48:36.057 INFO:teuthology.orchestra.run.smithi043.stdout:After this operation, 80.0 MB of additional disk space will be used. 2024-09-13T17:48:36.057 INFO:teuthology.orchestra.run.smithi043.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-09-13T17:48:36.452 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-13T17:48:36.470 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T17:48:36.510 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T17:48:36.676 INFO:teuthology.orchestra.run.smithi043.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-09-13T17:48:36.678 INFO:teuthology.orchestra.run.smithi043.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-09-13T17:48:36.679 INFO:teuthology.orchestra.run.smithi043.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-09-13T17:48:36.692 INFO:teuthology.orchestra.run.smithi043.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-09-13T17:48:36.694 INFO:teuthology.orchestra.run.smithi043.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.23 [1,323 kB] 2024-09-13T17:48:36.792 INFO:teuthology.orchestra.run.smithi043.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-09-13T17:48:36.794 INFO:teuthology.orchestra.run.smithi043.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-09-13T17:48:36.801 INFO:teuthology.orchestra.run.smithi043.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-09-13T17:48:36.806 INFO:teuthology.orchestra.run.smithi043.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-09-13T17:48:36.806 INFO:teuthology.orchestra.run.smithi043.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.8 [60.6 kB] 2024-09-13T17:48:36.839 INFO:teuthology.orchestra.run.smithi043.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-09-13T17:48:36.841 INFO:teuthology.orchestra.run.smithi043.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-09-13T17:48:36.845 INFO:teuthology.orchestra.run.smithi043.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-09-13T17:48:36.845 INFO:teuthology.orchestra.run.smithi043.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-09-13T17:48:36.914 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-13T17:48:36.921 INFO:teuthology.orchestra.run.smithi043.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libndp0 amd64 1.7-0ubuntu1.1 [11.1 kB] 2024-09-13T17:48:36.922 INFO:teuthology.orchestra.run.smithi043.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-09-13T17:48:36.931 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1.1_amd64.deb ... 2024-09-13T17:48:36.931 INFO:teuthology.orchestra.run.smithi043.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.20-0ubuntu0.20.04.1 [117 kB] 2024-09-13T17:48:36.934 INFO:teuthology.orchestra.run.smithi043.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-09-13T17:48:36.950 INFO:teuthology.orchestra.run.smithi043.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-09-13T17:48:36.950 INFO:teuthology.orchestra.run.smithi043.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.23 [1,586 kB] 2024-09-13T17:48:36.971 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-09-13T17:48:37.003 INFO:teuthology.orchestra.run.smithi043.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-09-13T17:48:37.003 INFO:teuthology.orchestra.run.smithi043.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-09-13T17:48:37.026 INFO:teuthology.orchestra.run.smithi043.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.4 [1,184 kB] 2024-09-13T17:48:37.056 INFO:teuthology.orchestra.run.smithi043.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-09-13T17:48:37.114 INFO:teuthology.orchestra.run.smithi043.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-09-13T17:48:37.123 INFO:teuthology.orchestra.run.smithi043.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-09-13T17:48:37.124 INFO:teuthology.orchestra.run.smithi043.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-09-13T17:48:37.126 INFO:teuthology.orchestra.run.smithi043.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2024-09-13T17:48:37.127 INFO:teuthology.orchestra.run.smithi043.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.20-0ubuntu0.20.04.1 [1,055 kB] 2024-09-13T17:48:37.149 INFO:teuthology.orchestra.run.smithi043.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2024-09-13T17:48:37.167 INFO:teuthology.orchestra.run.smithi043.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.20-0ubuntu0.20.04.1 [13.5 MB] 2024-09-13T17:48:37.292 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-13T17:48:37.309 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-09-13T17:48:37.349 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-09-13T17:48:37.597 INFO:teuthology.orchestra.run.smithi043.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2024-09-13T17:48:37.597 INFO:teuthology.orchestra.run.smithi043.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2024-09-13T17:48:37.597 INFO:teuthology.orchestra.run.smithi043.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2024-09-13T17:48:37.597 INFO:teuthology.orchestra.run.smithi043.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-09-13T17:48:37.598 INFO:teuthology.orchestra.run.smithi043.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-09-13T17:48:37.761 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-13T17:48:37.778 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../14-libpq5_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T17:48:37.819 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libpq5:amd64 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T17:48:38.012 INFO:teuthology.orchestra.run.smithi043.stdout:Preconfiguring packages ... 2024-09-13T17:48:38.173 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-13T17:48:38.189 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../15-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T17:48:38.235 INFO:teuthology.orchestra.run.smithi043.stdout:Fetched 25.4 MB in 2s (14.8 MB/s) 2024-09-13T17:48:38.257 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T17:48:38.553 INFO:teuthology.orchestra.run.smithi043.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153099 files and directories currently installed.) 2024-09-13T17:48:38.557 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T17:48:38.600 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-13T17:48:38.617 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../16-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T17:48:38.658 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T17:48:38.752 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T17:48:38.902 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-13T17:48:38.921 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../17-libssl-dev_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-09-13T17:48:38.960 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T17:48:39.447 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-13T17:48:39.465 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../18-libteamdctl0_1.30-1_amd64.deb ... 2024-09-13T17:48:39.505 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-09-13T17:48:39.892 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package modemmanager. 2024-09-13T17:48:39.911 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../19-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T17:48:39.953 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T17:48:39.958 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-09-13T17:48:40.565 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package wpasupplicant. 2024-09-13T17:48:40.584 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../20-wpasupplicant_2%3a2.9-1ubuntu4.4_amd64.deb ... 2024-09-13T17:48:40.656 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-09-13T17:48:41.161 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package network-manager. 2024-09-13T17:48:41.181 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../21-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-09-13T17:48:41.244 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-09-13T17:48:42.335 INFO:teuthology.orchestra.run.smithi043.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153099 files and directories currently installed.) 2024-09-13T17:48:42.339 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T17:48:42.437 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ppp. 2024-09-13T17:48:42.456 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../22-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-09-13T17:48:42.501 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T17:48:42.722 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-09-13T17:48:42.909 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T17:48:43.069 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package pptp-linux. 2024-09-13T17:48:43.089 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../23-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-09-13T17:48:43.108 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T17:48:43.133 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-09-13T17:48:43.479 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-09-13T17:48:43.495 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-13T17:48:43.513 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../24-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-09-13T17:48:43.571 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-09-13T17:48:43.848 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-13T17:48:43.865 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../25-postgresql-client-common_214ubuntu0.1_all.deb ... 2024-09-13T17:48:43.915 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2024-09-13T17:48:43.970 INFO:teuthology.orchestra.run.smithi043.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153099 files and directories currently installed.) 2024-09-13T17:48:43.974 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T17:48:44.124 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-09-13T17:48:44.194 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-client-12. 2024-09-13T17:48:44.215 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../26-postgresql-client-12_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T17:48:44.252 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-client-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T17:48:44.664 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T17:48:44.824 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-09-13T17:48:44.907 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-common. 2024-09-13T17:48:44.925 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../27-postgresql-common_214ubuntu0.1_all.deb ... 2024-09-13T17:48:44.989 INFO:teuthology.orchestra.run.smithi040.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-13T17:48:45.017 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2024-09-13T17:48:45.387 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-09-13T17:48:45.514 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-12. 2024-09-13T17:48:45.517 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.23) over (1.1.1f-1ubuntu2.16) ... 2024-09-13T17:48:45.533 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../28-postgresql-12_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T17:48:45.575 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T17:48:46.047 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-13T17:48:46.064 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-09-13T17:48:46.163 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-09-13T17:48:47.129 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package dump. 2024-09-13T17:48:47.150 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-09-13T17:48:47.334 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking dump (0.4b46-6) ... 2024-09-13T17:48:47.655 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package indent. 2024-09-13T17:48:47.673 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T17:48:47.713 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-09-13T17:48:47.742 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql. 2024-09-13T17:48:47.759 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../29-postgresql_12+214ubuntu0.1_all.deb ... 2024-09-13T17:48:47.800 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2024-09-13T17:48:48.050 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-13T17:48:48.068 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-09-13T17:48:48.069 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-client. 2024-09-13T17:48:48.088 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../30-postgresql-client_12+214ubuntu0.1_all.deb ... 2024-09-13T17:48:48.124 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-09-13T17:48:48.127 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2024-09-13T17:48:48.371 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-13T17:48:48.388 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../31-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2024-09-13T17:48:48.429 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2024-09-13T17:48:48.478 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-13T17:48:48.496 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.8_amd64.deb ... 2024-09-13T17:48:48.544 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-09-13T17:48:48.698 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-13T17:48:48.718 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../32-usb-modeswitch-data_20191128-3_all.deb ... 2024-09-13T17:48:48.756 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-09-13T17:48:48.839 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-13T17:48:48.856 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T17:48:48.922 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T17:48:49.134 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-13T17:48:49.155 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../33-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-09-13T17:48:49.203 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-09-13T17:48:49.309 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-13T17:48:49.327 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T17:48:49.392 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T17:48:49.526 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2024-09-13T17:48:49.720 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-13T17:48:49.731 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T17:48:49.739 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T17:48:49.778 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T17:48:50.041 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libpq5:amd64 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T17:48:50.098 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-13T17:48:50.118 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T17:48:50.167 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-09-13T17:48:50.179 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T17:48:50.318 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-09-13T17:48:50.419 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-09-13T17:48:50.575 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-13T17:48:50.593 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1.1_amd64.deb ... 2024-09-13T17:48:50.633 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-09-13T17:48:50.769 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T17:48:50.769 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T17:48:51.020 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-13T17:48:51.040 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-09-13T17:48:51.078 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-09-13T17:48:51.239 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-client-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T17:48:51.515 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-13T17:48:51.535 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../14-libpq5_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T17:48:51.573 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libpq5:amd64 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T17:48:51.968 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-13T17:48:51.984 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../15-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T17:48:52.050 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T17:48:52.146 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-13T17:48:52.236 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-09-13T17:48:52.362 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-09-13T17:48:52.488 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2024-09-13T17:48:53.039 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-13T17:48:53.059 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../16-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T17:48:53.097 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T17:48:53.374 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-13T17:48:53.392 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../17-libssl-dev_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-09-13T17:48:53.455 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T17:48:54.011 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-13T17:48:54.013 INFO:teuthology.orchestra.run.smithi040.stdout:Adding user postgres to group ssl-cert 2024-09-13T17:48:54.028 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../18-libteamdctl0_1.30-1_amd64.deb ... 2024-09-13T17:48:54.078 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-09-13T17:48:54.275 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:48:54.275 INFO:teuthology.orchestra.run.smithi040.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-13T17:48:54.514 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package modemmanager. 2024-09-13T17:48:54.535 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../19-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T17:48:54.552 INFO:teuthology.orchestra.run.smithi040.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-13T17:48:54.552 INFO:teuthology.orchestra.run.smithi040.stdout:Removing obsolete dictionary files: 2024-09-13T17:48:54.575 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T17:48:54.902 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-13T17:48:55.212 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package wpasupplicant. 2024-09-13T17:48:55.231 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../20-wpasupplicant_2%3a2.9-1ubuntu4.4_amd64.deb ... 2024-09-13T17:48:55.295 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-09-13T17:48:55.674 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T17:48:55.801 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-09-13T17:48:55.867 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package network-manager. 2024-09-13T17:48:55.886 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../21-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-09-13T17:48:55.948 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-09-13T17:48:55.951 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-09-13T17:48:56.077 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-09-13T17:48:56.749 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ppp. 2024-09-13T17:48:56.767 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../22-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-09-13T17:48:57.033 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-09-13T17:48:57.077 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-09-13T17:48:57.341 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up dump (0.4b46-6) ... 2024-09-13T17:48:57.441 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package pptp-linux. 2024-09-13T17:48:57.441 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-13T17:48:57.456 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../23-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-09-13T17:48:57.496 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-09-13T17:48:57.526 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T17:48:57.652 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-09-13T17:48:57.778 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-09-13T17:48:57.883 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-13T17:48:57.903 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../24-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-09-13T17:48:57.904 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-09-13T17:48:57.958 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-09-13T17:48:58.080 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-09-13T17:48:58.206 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T17:48:58.486 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-13T17:48:58.504 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../25-postgresql-client-common_214ubuntu0.1_all.deb ... 2024-09-13T17:48:58.544 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2024-09-13T17:48:58.639 INFO:teuthology.orchestra.run.smithi040.stdout:Creating new PostgreSQL cluster 12/main ... 2024-09-13T17:48:58.653 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2024-09-13T17:48:58.700 INFO:teuthology.orchestra.run.smithi040.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-13T17:48:58.700 INFO:teuthology.orchestra.run.smithi040.stdout:This user must also own the server process. 2024-09-13T17:48:58.700 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:48:58.701 INFO:teuthology.orchestra.run.smithi040.stdout:The database cluster will be initialized with locale "en_US". 2024-09-13T17:48:58.701 INFO:teuthology.orchestra.run.smithi040.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-13T17:48:58.701 INFO:teuthology.orchestra.run.smithi040.stdout:The default text search configuration will be set to "english". 2024-09-13T17:48:58.701 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:48:58.701 INFO:teuthology.orchestra.run.smithi040.stdout:Data page checksums are disabled. 2024-09-13T17:48:58.701 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:48:58.701 INFO:teuthology.orchestra.run.smithi040.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2024-09-13T17:48:58.701 INFO:teuthology.orchestra.run.smithi040.stdout:creating subdirectories ... ok 2024-09-13T17:48:58.701 INFO:teuthology.orchestra.run.smithi040.stdout:selecting dynamic shared memory implementation ... posix 2024-09-13T17:48:58.722 INFO:teuthology.orchestra.run.smithi040.stdout:selecting default max_connections ... 100 2024-09-13T17:48:58.747 INFO:teuthology.orchestra.run.smithi040.stdout:selecting default shared_buffers ... 128MB 2024-09-13T17:48:58.755 INFO:teuthology.orchestra.run.smithi040.stdout:selecting default time zone ... Etc/UTC 2024-09-13T17:48:58.756 INFO:teuthology.orchestra.run.smithi040.stdout:creating configuration files ... ok 2024-09-13T17:48:58.862 INFO:teuthology.orchestra.run.smithi040.stdout:running bootstrap script ... ok 2024-09-13T17:48:58.865 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql-client-12. 2024-09-13T17:48:58.884 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../26-postgresql-client-12_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T17:48:58.949 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql-client-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T17:48:59.425 INFO:teuthology.orchestra.run.smithi040.stdout:performing post-bootstrap initialization ... ok 2024-09-13T17:48:59.595 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql-common. 2024-09-13T17:48:59.613 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../27-postgresql-common_214ubuntu0.1_all.deb ... 2024-09-13T17:48:59.715 INFO:teuthology.orchestra.run.smithi043.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-13T17:48:59.747 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2024-09-13T17:49:00.157 INFO:teuthology.orchestra.run.smithi040.stdout:syncing data to disk ... ok 2024-09-13T17:49:00.157 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:00.157 INFO:teuthology.orchestra.run.smithi040.stdout:Success. You can now start the database server using: 2024-09-13T17:49:00.157 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:00.157 INFO:teuthology.orchestra.run.smithi040.stdout: pg_ctlcluster 12 main start 2024-09-13T17:49:00.157 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:00.277 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql-12. 2024-09-13T17:49:00.294 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../28-postgresql-12_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T17:49:00.338 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T17:49:00.480 INFO:teuthology.orchestra.run.smithi040.stdout:Ver Cluster Port Status Owner Data directory Log file 2024-09-13T17:49:00.480 INFO:teuthology.orchestra.run.smithi040.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2024-09-13T17:49:00.561 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-13T17:49:02.130 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql. 2024-09-13T17:49:02.148 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../29-postgresql_12+214ubuntu0.1_all.deb ... 2024-09-13T17:49:02.188 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2024-09-13T17:49:02.474 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql-client. 2024-09-13T17:49:02.492 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../30-postgresql-client_12+214ubuntu0.1_all.deb ... 2024-09-13T17:49:02.557 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2024-09-13T17:49:02.826 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-13T17:49:02.846 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../31-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2024-09-13T17:49:02.884 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2024-09-13T17:49:03.153 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-13T17:49:03.173 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../32-usb-modeswitch-data_20191128-3_all.deb ... 2024-09-13T17:49:03.211 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-09-13T17:49:03.646 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T17:49:03.698 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-13T17:49:03.716 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../33-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-09-13T17:49:03.767 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-09-13T17:49:03.772 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T17:49:03.907 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-09-13T17:49:04.091 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T17:49:04.164 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2024-09-13T17:49:04.217 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2024-09-13T17:49:04.321 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T17:49:04.343 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-09-13T17:49:04.681 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libpq5:amd64 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T17:49:04.807 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-09-13T17:49:04.966 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-09-13T17:49:05.061 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:05.062 INFO:teuthology.orchestra.run.smithi040.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-13T17:49:05.062 INFO:teuthology.orchestra.run.smithi040.stdout:which means they are currently configured by ifupdown: 2024-09-13T17:49:05.062 INFO:teuthology.orchestra.run.smithi040.stdout:- ens1f0 2024-09-13T17:49:05.062 INFO:teuthology.orchestra.run.smithi040.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-13T17:49:05.062 INFO:teuthology.orchestra.run.smithi040.stdout:remove their configuration from /etc/network/interfaces. 2024-09-13T17:49:05.062 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:05.092 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-09-13T17:49:05.225 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-13T17:49:05.520 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T17:49:05.520 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T17:49:05.550 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-13T17:49:06.004 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql-client-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T17:49:06.072 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-13T17:49:06.555 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T17:49:06.689 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-09-13T17:49:06.824 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T17:49:06.902 INFO:teuthology.orchestra.run.smithi043.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-13T17:49:06.925 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2024-09-13T17:49:07.018 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-09-13T17:49:07.043 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2024-09-13T17:49:07.144 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-09-13T17:49:07.202 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T17:49:07.303 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2024-09-13T17:49:07.328 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T17:49:07.454 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T17:49:07.736 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T17:49:07.736 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T17:49:08.266 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-09-13T17:49:08.366 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-13T17:49:08.421 INFO:teuthology.orchestra.run.smithi043.stdout:Adding user postgres to group ssl-cert 2024-09-13T17:49:08.484 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T17:49:08.627 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T17:49:08.912 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T17:49:09.022 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:09.022 INFO:teuthology.orchestra.run.smithi043.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-13T17:49:09.313 INFO:teuthology.orchestra.run.smithi043.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-13T17:49:09.314 INFO:teuthology.orchestra.run.smithi043.stdout:Removing obsolete dictionary files: 2024-09-13T17:49:09.643 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-13T17:49:10.455 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T17:49:10.581 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-09-13T17:49:10.740 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-09-13T17:49:10.858 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-09-13T17:49:11.916 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-09-13T17:49:12.149 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up dump (0.4b46-6) ... 2024-09-13T17:49:12.280 INFO:teuthology.orchestra.run.smithi043.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-13T17:49:12.342 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T17:49:12.468 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-09-13T17:49:12.627 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-09-13T17:49:12.777 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-09-13T17:49:12.955 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-09-13T17:49:13.114 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T17:49:13.497 INFO:teuthology.orchestra.run.smithi043.stdout:Creating new PostgreSQL cluster 12/main ... 2024-09-13T17:49:13.517 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2024-09-13T17:49:13.567 INFO:teuthology.orchestra.run.smithi043.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-13T17:49:13.568 INFO:teuthology.orchestra.run.smithi043.stdout:This user must also own the server process. 2024-09-13T17:49:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:13.568 INFO:teuthology.orchestra.run.smithi043.stdout:The database cluster will be initialized with locale "en_US". 2024-09-13T17:49:13.568 INFO:teuthology.orchestra.run.smithi043.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-13T17:49:13.568 INFO:teuthology.orchestra.run.smithi043.stdout:The default text search configuration will be set to "english". 2024-09-13T17:49:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:13.568 INFO:teuthology.orchestra.run.smithi043.stdout:Data page checksums are disabled. 2024-09-13T17:49:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:13.568 INFO:teuthology.orchestra.run.smithi043.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2024-09-13T17:49:13.569 INFO:teuthology.orchestra.run.smithi043.stdout:creating subdirectories ... ok 2024-09-13T17:49:13.569 INFO:teuthology.orchestra.run.smithi043.stdout:selecting dynamic shared memory implementation ... posix 2024-09-13T17:49:13.585 INFO:teuthology.orchestra.run.smithi043.stdout:selecting default max_connections ... 100 2024-09-13T17:49:13.609 INFO:teuthology.orchestra.run.smithi043.stdout:selecting default shared_buffers ... 128MB 2024-09-13T17:49:13.615 INFO:teuthology.orchestra.run.smithi043.stdout:selecting default time zone ... Etc/UTC 2024-09-13T17:49:13.616 INFO:teuthology.orchestra.run.smithi043.stdout:creating configuration files ... ok 2024-09-13T17:49:13.724 INFO:teuthology.orchestra.run.smithi043.stdout:running bootstrap script ... ok 2024-09-13T17:49:14.308 INFO:teuthology.orchestra.run.smithi043.stdout:performing post-bootstrap initialization ... ok 2024-09-13T17:49:15.064 INFO:teuthology.orchestra.run.smithi043.stdout:syncing data to disk ... ok 2024-09-13T17:49:15.065 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:15.065 INFO:teuthology.orchestra.run.smithi043.stdout:Success. You can now start the database server using: 2024-09-13T17:49:15.065 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:15.065 INFO:teuthology.orchestra.run.smithi043.stdout: pg_ctlcluster 12 main start 2024-09-13T17:49:15.065 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:15.405 INFO:teuthology.orchestra.run.smithi043.stdout:Ver Cluster Port Status Owner Data directory Log file 2024-09-13T17:49:15.405 INFO:teuthology.orchestra.run.smithi043.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2024-09-13T17:49:15.493 INFO:teuthology.orchestra.run.smithi043.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-13T17:49:16.258 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-09-13T17:49:16.366 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-09-13T17:49:17.220 INFO:teuthology.orchestra.run.smithi040.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T17:49:17.226 DEBUG:teuthology.parallel:result is None 2024-09-13T17:49:18.588 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T17:49:18.722 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T17:49:18.865 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-09-13T17:49:19.033 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T17:49:19.192 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2024-09-13T17:49:19.345 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-09-13T17:49:20.107 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:20.107 INFO:teuthology.orchestra.run.smithi043.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-13T17:49:20.107 INFO:teuthology.orchestra.run.smithi043.stdout:which means they are currently configured by ifupdown: 2024-09-13T17:49:20.107 INFO:teuthology.orchestra.run.smithi043.stdout:- ens1f0 2024-09-13T17:49:20.107 INFO:teuthology.orchestra.run.smithi043.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-13T17:49:20.107 INFO:teuthology.orchestra.run.smithi043.stdout:remove their configuration from /etc/network/interfaces. 2024-09-13T17:49:20.107 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:20.266 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-13T17:49:20.585 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-13T17:49:21.128 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-13T17:49:21.621 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T17:49:21.789 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-09-13T17:49:21.965 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T17:49:22.091 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2024-09-13T17:49:22.242 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2024-09-13T17:49:22.368 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T17:49:22.528 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T17:49:22.637 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T17:49:22.934 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T17:49:22.934 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T17:49:23.456 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-09-13T17:49:23.591 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-13T17:49:23.708 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T17:49:23.893 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T17:49:24.203 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T17:49:31.457 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-09-13T17:49:31.591 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-09-13T17:49:32.420 INFO:teuthology.orchestra.run.smithi043.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T17:49:32.425 DEBUG:teuthology.parallel:result is None 2024-09-13T17:49:32.426 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:49:32.577 DEBUG:teuthology.orchestra.run.smithi040:> dpkg-query -W -f '${Version}' ceph 2024-09-13T17:49:32.602 INFO:teuthology.orchestra.run.smithi040.stdout:18.2.4-860-g6c6b083f-1focal 2024-09-13T17:49:32.603 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-860-g6c6b083f-1focal 2024-09-13T17:49:32.603 INFO:teuthology.task.install:The correct ceph version 18.2.4-860-g6c6b083f-1focal is installed. 2024-09-13T17:49:32.605 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:49:32.749 DEBUG:teuthology.orchestra.run.smithi043:> dpkg-query -W -f '${Version}' ceph 2024-09-13T17:49:32.773 INFO:teuthology.orchestra.run.smithi043.stdout:18.2.4-860-g6c6b083f-1focal 2024-09-13T17:49:32.774 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-860-g6c6b083f-1focal 2024-09-13T17:49:32.774 INFO:teuthology.task.install:The correct ceph version 18.2.4-860-g6c6b083f-1focal is installed. 2024-09-13T17:49:32.775 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T17:49:32.776 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:49:32.776 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T17:49:32.793 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:49:32.793 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T17:49:32.808 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T17:49:32.808 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:49:32.808 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T17:49:32.825 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T17:49:32.879 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:49:32.879 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T17:49:32.895 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T17:49:32.905 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T17:49:32.906 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:49:32.906 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T17:49:32.923 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T17:49:32.980 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:49:32.980 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T17:49:32.995 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T17:49:33.051 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T17:49:33.052 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:49:33.052 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T17:49:33.068 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T17:49:33.079 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:49:33.080 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T17:49:33.096 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T17:49:33.106 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T17:49:33.197 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T17:49:33.197 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 777 /var/log/ceph 2024-09-13T17:49:33.200 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /var/log/ceph 2024-09-13T17:49:33.215 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T17:49:33.215 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T17:49:33.217 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T17:49:33.226 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T17:49:33.226 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T17:49:33.271 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T17:49:33.292 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-13T17:49:33.292 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-13T17:49:33.292 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-13T17:49:33.293 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:49:33.293 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-13T17:49:33.299 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-13T17:49:33.339 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:49:33.339 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-13T17:49:33.346 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-13T17:49:33.389 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T17:49:33.389 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'overall HEALTH_', '\\(MON_DOWN\\)', 'SLOW_OPS', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T17:49:33.390 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903931', 'branch': 'wip-rishabh-testing-20240910.100730-reef', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}}', 'email': None, 'first_in_suite': False, 'job_id': '7903931', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240910.100730-reef'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'overall HEALTH_', '\\(MON_DOWN\\)', 'SLOW_OPS', 'slow request'], 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240910.100730-reef', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 6439, 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'sleep_before_teardown': 0, 'subset': '964/1024', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240910.100730-reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi040.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi043.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'overall HEALTH_', '\\(MON_DOWN\\)', 'SLOW_OPS', 'slow request'], 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'mon_thrash': {'check_mds_failover': True, 'revive_delay': 20, 'thrash_delay': 10}}, {'workunit': {'clients': {'all': ['suites/ffsb.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-13_14:42:16', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T17:49:33.390 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T17:49:33.393 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T17:49:33.399 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T17:49:33.402 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T17:49:33.417 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:49:33.417 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T17:49:33.422 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T17:49:33.422 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_1 2024-09-13T17:49:33.472 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T17:49:33.472 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:49:33.473 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 6h/6d Inode: 359 Links: 1 2024-09-13T17:49:33.473 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:49:33.473 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-09-13 17:44:41.336694923 +0000 2024-09-13T17:49:33.473 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-09-13 17:44:41.116699986 +0000 2024-09-13T17:49:33.473 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-09-13 17:44:41.116699986 +0000 2024-09-13T17:49:33.473 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2024-09-13T17:49:33.473 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T17:49:33.484 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-09-13T17:49:33.484 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-09-13T17:49:33.484 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000430094 s, 1.2 MB/s 2024-09-13T17:49:33.485 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T17:49:33.493 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_2 2024-09-13T17:49:33.544 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T17:49:33.544 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:49:33.544 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 6h/6d Inode: 367 Links: 1 2024-09-13T17:49:33.545 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:49:33.545 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-09-13 17:44:41.776684798 +0000 2024-09-13T17:49:33.545 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-09-13 17:44:41.560689768 +0000 2024-09-13T17:49:33.545 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-09-13 17:44:41.560689768 +0000 2024-09-13T17:49:33.545 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2024-09-13T17:49:33.545 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T17:49:33.558 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-09-13T17:49:33.558 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-09-13T17:49:33.558 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000395369 s, 1.3 MB/s 2024-09-13T17:49:33.559 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T17:49:33.567 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_3 2024-09-13T17:49:33.616 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T17:49:33.616 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:49:33.616 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 6h/6d Inode: 374 Links: 1 2024-09-13T17:49:33.617 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:49:33.617 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-09-13 17:44:42.212674764 +0000 2024-09-13T17:49:33.617 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-09-13 17:44:41.996679734 +0000 2024-09-13T17:49:33.617 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-09-13 17:44:41.996679734 +0000 2024-09-13T17:49:33.617 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2024-09-13T17:49:33.617 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T17:49:33.628 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-09-13T17:49:33.628 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-09-13T17:49:33.628 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000426828 s, 1.2 MB/s 2024-09-13T17:49:33.629 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T17:49:33.638 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_4 2024-09-13T17:49:33.689 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T17:49:33.689 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:49:33.689 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 6h/6d Inode: 379 Links: 1 2024-09-13T17:49:33.689 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:49:33.689 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-09-13 17:44:42.660664454 +0000 2024-09-13T17:49:33.689 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-09-13 17:44:42.460669057 +0000 2024-09-13T17:49:33.689 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-09-13 17:44:42.460669057 +0000 2024-09-13T17:49:33.689 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2024-09-13T17:49:33.689 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T17:49:33.701 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-09-13T17:49:33.701 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-09-13T17:49:33.701 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000369059 s, 1.4 MB/s 2024-09-13T17:49:33.703 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T17:49:33.711 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T17:49:33.711 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:49:33.711 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T17:49:33.718 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T17:49:33.718 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2024-09-13T17:49:33.726 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T17:49:33.726 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:49:33.726 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 6h/6d Inode: 360 Links: 1 2024-09-13T17:49:33.726 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:49:33.727 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-13 17:44:44.006647037 +0000 2024-09-13T17:49:33.727 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-13 17:44:43.802651738 +0000 2024-09-13T17:49:33.727 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-13 17:44:43.802651738 +0000 2024-09-13T17:49:33.727 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: - 2024-09-13T17:49:33.727 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T17:49:33.739 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-13T17:49:33.740 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-13T17:49:33.740 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000313264 s, 1.6 MB/s 2024-09-13T17:49:33.741 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T17:49:33.749 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2024-09-13T17:49:33.757 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T17:49:33.757 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:49:33.757 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 6h/6d Inode: 365 Links: 1 2024-09-13T17:49:33.758 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:49:33.758 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-13 17:44:44.454636711 +0000 2024-09-13T17:49:33.758 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-13 17:44:44.258641227 +0000 2024-09-13T17:49:33.758 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-13 17:44:44.258641227 +0000 2024-09-13T17:49:33.758 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: - 2024-09-13T17:49:33.759 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T17:49:33.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-13T17:49:33.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-13T17:49:33.814 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.00030011 s, 1.7 MB/s 2024-09-13T17:49:33.815 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T17:49:33.824 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2024-09-13T17:49:33.830 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T17:49:33.830 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:49:33.830 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 6h/6d Inode: 370 Links: 1 2024-09-13T17:49:33.830 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:49:33.830 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-13 17:44:44.886626753 +0000 2024-09-13T17:49:33.830 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-13 17:44:44.674631640 +0000 2024-09-13T17:49:33.831 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-13 17:44:44.674631640 +0000 2024-09-13T17:49:33.831 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: - 2024-09-13T17:49:33.831 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T17:49:33.887 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-13T17:49:33.887 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-13T17:49:33.887 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000295263 s, 1.7 MB/s 2024-09-13T17:49:33.889 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T17:49:33.897 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2024-09-13T17:49:33.904 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T17:49:33.904 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:49:33.904 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 6h/6d Inode: 376 Links: 1 2024-09-13T17:49:33.904 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:49:33.904 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-13 17:44:45.126621220 +0000 2024-09-13T17:49:33.904 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-13 17:44:45.122621313 +0000 2024-09-13T17:49:33.904 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-13 17:44:45.122621313 +0000 2024-09-13T17:49:33.904 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: - 2024-09-13T17:49:33.904 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T17:49:33.918 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-13T17:49:33.918 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-13T17:49:33.918 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000241943 s, 2.1 MB/s 2024-09-13T17:49:33.919 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T17:49:33.927 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T17:49:33.927 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi043.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T17:49:33.927 INFO:tasks.ceph:Generating config... 2024-09-13T17:49:33.929 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-13T17:49:33.929 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-09-13T17:49:33.929 INFO:tasks.ceph:[client] debug client = 20 2024-09-13T17:49:33.929 INFO:tasks.ceph:[client] debug ms = 1 2024-09-13T17:49:33.929 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-13T17:49:33.929 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-13T17:49:33.929 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-13T17:49:33.929 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-13T17:49:33.929 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-13T17:49:33.929 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-09-13T17:49:33.929 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-09-13T17:49:33.930 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-09-13T17:49:33.930 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-09-13T17:49:33.930 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-13T17:49:33.930 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-13T17:49:33.930 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-09-13T17:49:33.930 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-13T17:49:33.930 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-13T17:49:33.930 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2024-09-13T17:49:33.930 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-09-13T17:49:33.930 INFO:tasks.ceph:[mds] mon client directed command retry = 5 2024-09-13T17:49:33.930 INFO:tasks.ceph:[mds] ms inject delay max = 1 2024-09-13T17:49:33.930 INFO:tasks.ceph:[mds] ms inject delay probability = 0.005 2024-09-13T17:49:33.930 INFO:tasks.ceph:[mds] ms inject delay type = client mds osd 2024-09-13T17:49:33.930 INFO:tasks.ceph:[mds] ms inject socket failures = 2500 2024-09-13T17:49:33.930 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-13T17:49:33.931 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-13T17:49:33.931 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-13T17:49:33.931 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-13T17:49:33.931 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T17:49:33.931 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T17:49:33.931 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T17:49:33.931 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T17:49:33.931 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T17:49:33.931 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-13T17:49:33.931 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-13T17:49:33.931 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T17:49:33.931 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T17:49:33.931 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-13T17:49:33.931 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-13T17:49:33.931 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T17:49:33.932 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T17:49:33.932 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-13T17:49:33.932 INFO:tasks.ceph:[osd] filestore flush min = 0 2024-09-13T17:49:33.932 INFO:tasks.ceph:[osd] mon client directed command retry = 5 2024-09-13T17:49:33.932 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T17:49:33.932 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T17:49:33.932 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T17:49:33.932 INFO:tasks.ceph:[osd] ms inject delay max = 1 2024-09-13T17:49:33.932 INFO:tasks.ceph:[osd] ms inject delay probability = 0.005 2024-09-13T17:49:33.932 INFO:tasks.ceph:[osd] ms inject delay type = client mds 2024-09-13T17:49:33.932 INFO:tasks.ceph:[osd] ms inject socket failures = 2500 2024-09-13T17:49:33.932 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-13T17:49:33.932 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T17:49:33.933 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-13T17:49:33.933 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T17:49:33.933 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-13T17:49:33.933 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T17:49:33.933 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T17:49:33.973 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T17:49:33.986 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T17:49:34.026 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T17:49:34.036 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.40'), ('mon.b', '172.21.15.43'), ('mon.c', '[v2:172.21.15.43:3301,v1:172.21.15.43:6790]')] 2024-09-13T17:49:34.037 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.40,172.21.15.43,[v2:172.21.15.43:3301,v1:172.21.15.43:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-13T17:49:34.037 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:49:34.038 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T17:49:34.082 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.40 --add b 172.21.15.43 --addv c '[v2:172.21.15.43:3301,v1:172.21.15.43:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:49:34.127 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:49:34.128 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: generated fsid 841d6eb0-79fe-4108-906c-e8798e475996 2024-09-13T17:49:34.128 INFO:teuthology.orchestra.run.smithi040.stdout:setting min_mon_release = pacific 2024-09-13T17:49:34.128 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 0 2024-09-13T17:49:34.128 INFO:teuthology.orchestra.run.smithi040.stdout:fsid 841d6eb0-79fe-4108-906c-e8798e475996 2024-09-13T17:49:34.128 INFO:teuthology.orchestra.run.smithi040.stdout:last_changed 2024-09-13T17:49:34.123582+0000 2024-09-13T17:49:34.128 INFO:teuthology.orchestra.run.smithi040.stdout:created 2024-09-13T17:49:34.123582+0000 2024-09-13T17:49:34.128 INFO:teuthology.orchestra.run.smithi040.stdout:min_mon_release 16 (pacific) 2024-09-13T17:49:34.128 INFO:teuthology.orchestra.run.smithi040.stdout:election_strategy: 1 2024-09-13T17:49:34.128 INFO:teuthology.orchestra.run.smithi040.stdout:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2024-09-13T17:49:34.128 INFO:teuthology.orchestra.run.smithi040.stdout:1: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.b 2024-09-13T17:49:34.129 INFO:teuthology.orchestra.run.smithi040.stdout:2: [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] mon.c 2024-09-13T17:49:34.129 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-13T17:49:34.130 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T17:49:34.134 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 841d6eb0-79fe-4108-906c-e8798e475996... 2024-09-13T17:49:34.135 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T17:49:34.137 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T17:49:34.166 INFO:teuthology.orchestra.run.smithi040.stdout:[global] 2024-09-13T17:49:34.166 INFO:teuthology.orchestra.run.smithi040.stdout: chdir = "" 2024-09-13T17:49:34.166 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T17:49:34.166 INFO:teuthology.orchestra.run.smithi040.stdout: auth supported = cephx 2024-09-13T17:49:34.166 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.167 INFO:teuthology.orchestra.run.smithi040.stdout: filestore xattr use omap = true 2024-09-13T17:49:34.167 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.167 INFO:teuthology.orchestra.run.smithi040.stdout: mon clock drift allowed = 1.000 2024-09-13T17:49:34.167 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.167 INFO:teuthology.orchestra.run.smithi040.stdout: osd crush chooseleaf type = 0 2024-09-13T17:49:34.167 INFO:teuthology.orchestra.run.smithi040.stdout: auth debug = true 2024-09-13T17:49:34.167 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.167 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on old message = true 2024-09-13T17:49:34.167 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on bug = true 2024-09-13T17:49:34.167 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.167 INFO:teuthology.orchestra.run.smithi040.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T17:49:34.167 INFO:teuthology.orchestra.run.smithi040.stdout: mon pg warn max object skew = 0 2024-09-13T17:49:34.167 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T17:49:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T17:49:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default size = 2 2024-09-13T17:49:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow primary affinity = true 2024-09-13T17:49:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow pg remap = true 2024-09-13T17:49:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on legacy crush tunables = false 2024-09-13T17:49:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on crush straw calc version zero = false 2024-09-13T17:49:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on no sortbitwise = false 2024-09-13T17:49:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on osd down out interval zero = false 2024-09-13T17:49:34.168 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on too few osds = false 2024-09-13T17:49:34.169 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T17:49:34.169 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T17:49:34.169 INFO:teuthology.orchestra.run.smithi040.stdout: mon_allow_pool_size_one = true 2024-09-13T17:49:34.169 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.169 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T17:49:34.169 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.169 INFO:teuthology.orchestra.run.smithi040.stdout: osd default data pool replay window = 5 2024-09-13T17:49:34.169 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.169 INFO:teuthology.orchestra.run.smithi040.stdout: mon allow pool delete = true 2024-09-13T17:49:34.169 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.169 INFO:teuthology.orchestra.run.smithi040.stdout: mon cluster log file level = debug 2024-09-13T17:49:34.170 INFO:teuthology.orchestra.run.smithi040.stdout: debug asserts on shutdown = true 2024-09-13T17:49:34.170 INFO:teuthology.orchestra.run.smithi040.stdout: mon health detail to clog = false 2024-09-13T17:49:34.170 INFO:teuthology.orchestra.run.smithi040.stdout: mon host = "172.21.15.40,172.21.15.43,[v2:172.21.15.43:3301,v1:172.21.15.43:6790]" 2024-09-13T17:49:34.170 INFO:teuthology.orchestra.run.smithi040.stdout: fsid = 841d6eb0-79fe-4108-906c-e8798e475996 2024-09-13T17:49:34.170 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.170 INFO:teuthology.orchestra.run.smithi040.stdout:[osd] 2024-09-13T17:49:34.170 INFO:teuthology.orchestra.run.smithi040.stdout: osd journal size = 100 2024-09-13T17:49:34.170 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.170 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub load threshold = 5.0 2024-09-13T17:49:34.170 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub max interval = 600 2024-09-13T17:49:34.170 INFO:teuthology.orchestra.run.smithi040.stdout: osd mclock profile = high_recovery_ops 2024-09-13T17:49:34.170 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.171 INFO:teuthology.orchestra.run.smithi040.stdout: osd recover clone overlap = true 2024-09-13T17:49:34.171 INFO:teuthology.orchestra.run.smithi040.stdout: osd recovery max chunk = 1048576 2024-09-13T17:49:34.171 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.171 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug shutdown = true 2024-09-13T17:49:34.171 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug op order = true 2024-09-13T17:49:34.171 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug verify stray on activate = true 2024-09-13T17:49:34.171 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.171 INFO:teuthology.orchestra.run.smithi040.stdout: osd open classes on start = true 2024-09-13T17:49:34.171 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug pg log writeout = true 2024-09-13T17:49:34.171 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.171 INFO:teuthology.orchestra.run.smithi040.stdout: osd deep scrub update digest min age = 30 2024-09-13T17:49:34.171 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.171 INFO:teuthology.orchestra.run.smithi040.stdout: osd map max advance = 10 2024-09-13T17:49:34.172 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.172 INFO:teuthology.orchestra.run.smithi040.stdout: journal zero on create = true 2024-09-13T17:49:34.172 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.172 INFO:teuthology.orchestra.run.smithi040.stdout: filestore ondisk finisher threads = 3 2024-09-13T17:49:34.172 INFO:teuthology.orchestra.run.smithi040.stdout: filestore apply finisher threads = 3 2024-09-13T17:49:34.172 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.172 INFO:teuthology.orchestra.run.smithi040.stdout: bdev debug aio = true 2024-09-13T17:49:34.172 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug misdirected ops = true 2024-09-13T17:49:34.172 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore block size = 96636764160 2024-09-13T17:49:34.172 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore fsck on mount = True 2024-09-13T17:49:34.172 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluefs = 20 2024-09-13T17:49:34.172 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluestore = 20 2024-09-13T17:49:34.173 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-09-13T17:49:34.173 INFO:teuthology.orchestra.run.smithi040.stdout: debug osd = 20 2024-09-13T17:49:34.173 INFO:teuthology.orchestra.run.smithi040.stdout: debug rocksdb = 10 2024-09-13T17:49:34.173 INFO:teuthology.orchestra.run.smithi040.stdout: filestore flush min = 0 2024-09-13T17:49:34.173 INFO:teuthology.orchestra.run.smithi040.stdout: mon client directed command retry = 5 2024-09-13T17:49:34.173 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T17:49:34.173 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd full ratio = 0.9 2024-09-13T17:49:34.173 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd nearfull ratio = 0.8 2024-09-13T17:49:34.173 INFO:teuthology.orchestra.run.smithi040.stdout: ms inject delay max = 1 2024-09-13T17:49:34.173 INFO:teuthology.orchestra.run.smithi040.stdout: ms inject delay probability = 0.005 2024-09-13T17:49:34.173 INFO:teuthology.orchestra.run.smithi040.stdout: ms inject delay type = client mds 2024-09-13T17:49:34.174 INFO:teuthology.orchestra.run.smithi040.stdout: ms inject socket failures = 2500 2024-09-13T17:49:34.174 INFO:teuthology.orchestra.run.smithi040.stdout: osd blocked scrub grace period = 3600 2024-09-13T17:49:34.174 INFO:teuthology.orchestra.run.smithi040.stdout: osd failsafe full ratio = 0.95 2024-09-13T17:49:34.174 INFO:teuthology.orchestra.run.smithi040.stdout: osd max markdown count = 1000 2024-09-13T17:49:34.174 INFO:teuthology.orchestra.run.smithi040.stdout: osd objectstore = bluestore 2024-09-13T17:49:34.174 INFO:teuthology.orchestra.run.smithi040.stdout: osd op complaint time = 180 2024-09-13T17:49:34.174 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.174 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr] 2024-09-13T17:49:34.174 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-09-13T17:49:34.174 INFO:teuthology.orchestra.run.smithi040.stdout: debug mgr = 20 2024-09-13T17:49:34.174 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2024-09-13T17:49:34.174 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2024-09-13T17:49:34.175 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2024-09-13T17:49:34.175 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2024-09-13T17:49:34.175 INFO:teuthology.orchestra.run.smithi040.stdout: mgr/telemetry/nag = false 2024-09-13T17:49:34.175 INFO:teuthology.orchestra.run.smithi040.stdout: debug client = 20 2024-09-13T17:49:34.175 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.175 INFO:teuthology.orchestra.run.smithi040.stdout:[mon] 2024-09-13T17:49:34.175 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-09-13T17:49:34.175 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2024-09-13T17:49:34.175 INFO:teuthology.orchestra.run.smithi040.stdout: debug paxos = 20 2024-09-13T17:49:34.175 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2024-09-13T17:49:34.175 INFO:teuthology.orchestra.run.smithi040.stdout: mon data avail warn = 5 2024-09-13T17:49:34.175 INFO:teuthology.orchestra.run.smithi040.stdout: mon mgr mkfs grace = 240 2024-09-13T17:49:34.176 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2024-09-13T17:49:34.176 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd reporter subtree level = osd 2024-09-13T17:49:34.176 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd prime pg temp = true 2024-09-13T17:49:34.176 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2024-09-13T17:49:34.176 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.176 INFO:teuthology.orchestra.run.smithi040.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T17:49:34.176 INFO:teuthology.orchestra.run.smithi040.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T17:49:34.176 INFO:teuthology.orchestra.run.smithi040.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T17:49:34.176 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.176 INFO:teuthology.orchestra.run.smithi040.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T17:49:34.176 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T17:49:34.176 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T17:49:34.177 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.177 INFO:teuthology.orchestra.run.smithi040.stdout: # 1m isn't quite enough 2024-09-13T17:49:34.177 INFO:teuthology.orchestra.run.smithi040.stdout: mon_down_mkfs_grace = 2m 2024-09-13T17:49:34.177 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.177 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_filestore_osds = false 2024-09-13T17:49:34.177 INFO:teuthology.orchestra.run.smithi040.stdout: mon op complaint time = 120 2024-09-13T17:49:34.177 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on pool no app = False 2024-09-13T17:49:34.177 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:49:34.177 INFO:teuthology.orchestra.run.smithi040.stdout:[client] 2024-09-13T17:49:34.177 INFO:teuthology.orchestra.run.smithi040.stdout: rgw cache enabled = true 2024-09-13T17:49:34.177 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable ops log = true 2024-09-13T17:49:34.177 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable usage log = true 2024-09-13T17:49:34.177 INFO:teuthology.orchestra.run.smithi040.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T17:49:34.178 INFO:teuthology.orchestra.run.smithi040.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T17:49:34.178 INFO:teuthology.orchestra.run.smithi040.stdout: client mount timeout = 600 2024-09-13T17:49:34.178 INFO:teuthology.orchestra.run.smithi040.stdout: client_shutdown_timeout = 120 2024-09-13T17:49:34.178 INFO:teuthology.orchestra.run.smithi040.stdout: debug client = 20 2024-09-13T17:49:34.178 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-09-13T17:49:34.178 INFO:teuthology.orchestra.run.smithi040.stdout: rados mon op timeout = 900 2024-09-13T17:49:34.178 INFO:teuthology.orchestra.run.smithi040.stdout: rados osd op timeout = 900 2024-09-13T17:49:34.178 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.a] 2024-09-13T17:49:34.178 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.b] 2024-09-13T17:49:34.178 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.c] 2024-09-13T17:49:34.178 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a] 2024-09-13T17:49:34.178 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.c] 2024-09-13T17:49:34.179 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.e] 2024-09-13T17:49:34.179 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b] 2024-09-13T17:49:34.179 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.d] 2024-09-13T17:49:34.179 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.f] 2024-09-13T17:49:34.179 INFO:teuthology.orchestra.run.smithi040.stdout:[mds] 2024-09-13T17:49:34.179 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds = 20 2024-09-13T17:49:34.179 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds balancer = 20 2024-09-13T17:49:34.179 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2024-09-13T17:49:34.179 INFO:teuthology.orchestra.run.smithi040.stdout: mds bal fragment size max = 10000 2024-09-13T17:49:34.179 INFO:teuthology.orchestra.run.smithi040.stdout: mds bal merge size = 5 2024-09-13T17:49:34.179 INFO:teuthology.orchestra.run.smithi040.stdout: mds bal split bits = 3 2024-09-13T17:49:34.179 INFO:teuthology.orchestra.run.smithi040.stdout: mds bal split size = 100 2024-09-13T17:49:34.180 INFO:teuthology.orchestra.run.smithi040.stdout: mds debug frag = True 2024-09-13T17:49:34.180 INFO:teuthology.orchestra.run.smithi040.stdout: mds debug scatterstat = True 2024-09-13T17:49:34.180 INFO:teuthology.orchestra.run.smithi040.stdout: mds oft prefetch dirfrags = True 2024-09-13T17:49:34.180 INFO:teuthology.orchestra.run.smithi040.stdout: mds op complaint time = 180 2024-09-13T17:49:34.180 INFO:teuthology.orchestra.run.smithi040.stdout: mds verify scatter = True 2024-09-13T17:49:34.180 INFO:teuthology.orchestra.run.smithi040.stdout: mds_dir_prefetch = True 2024-09-13T17:49:34.180 INFO:teuthology.orchestra.run.smithi040.stdout: mds_sleep_rank_change = 5000000.0 2024-09-13T17:49:34.180 INFO:teuthology.orchestra.run.smithi040.stdout: mon client directed command retry = 5 2024-09-13T17:49:34.180 INFO:teuthology.orchestra.run.smithi040.stdout: ms inject delay max = 1 2024-09-13T17:49:34.180 INFO:teuthology.orchestra.run.smithi040.stdout: ms inject delay probability = 0.005 2024-09-13T17:49:34.180 INFO:teuthology.orchestra.run.smithi040.stdout: ms inject delay type = client mds osd 2024-09-13T17:49:34.180 INFO:teuthology.orchestra.run.smithi040.stdout: ms inject socket failures = 2500 2024-09-13T17:49:34.181 INFO:teuthology.orchestra.run.smithi040.stdout: osd op complaint time = 180 2024-09-13T17:49:34.181 INFO:teuthology.orchestra.run.smithi040.stdout: rados mon op timeout = 900 2024-09-13T17:49:34.181 INFO:teuthology.orchestra.run.smithi040.stdout: rados osd op timeout = 900 2024-09-13T17:49:34.182 INFO:teuthology.orchestra.run.smithi043.stdout:[global] 2024-09-13T17:49:34.182 INFO:teuthology.orchestra.run.smithi043.stdout: chdir = "" 2024-09-13T17:49:34.182 INFO:teuthology.orchestra.run.smithi043.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T17:49:34.183 INFO:teuthology.orchestra.run.smithi043.stdout: auth supported = cephx 2024-09-13T17:49:34.183 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.183 INFO:teuthology.orchestra.run.smithi043.stdout: filestore xattr use omap = true 2024-09-13T17:49:34.183 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.183 INFO:teuthology.orchestra.run.smithi043.stdout: mon clock drift allowed = 1.000 2024-09-13T17:49:34.183 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.183 INFO:teuthology.orchestra.run.smithi043.stdout: osd crush chooseleaf type = 0 2024-09-13T17:49:34.183 INFO:teuthology.orchestra.run.smithi043.stdout: auth debug = true 2024-09-13T17:49:34.183 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.183 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on old message = true 2024-09-13T17:49:34.183 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on bug = true 2024-09-13T17:49:34.183 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.184 INFO:teuthology.orchestra.run.smithi043.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T17:49:34.184 INFO:teuthology.orchestra.run.smithi043.stdout: mon pg warn max object skew = 0 2024-09-13T17:49:34.184 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.184 INFO:teuthology.orchestra.run.smithi043.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T17:49:34.184 INFO:teuthology.orchestra.run.smithi043.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T17:49:34.184 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.184 INFO:teuthology.orchestra.run.smithi043.stdout: osd pool default size = 2 2024-09-13T17:49:34.184 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.184 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd allow primary affinity = true 2024-09-13T17:49:34.184 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd allow pg remap = true 2024-09-13T17:49:34.184 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on legacy crush tunables = false 2024-09-13T17:49:34.185 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on crush straw calc version zero = false 2024-09-13T17:49:34.185 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on no sortbitwise = false 2024-09-13T17:49:34.185 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on osd down out interval zero = false 2024-09-13T17:49:34.185 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on too few osds = false 2024-09-13T17:49:34.185 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T17:49:34.185 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T17:49:34.185 INFO:teuthology.orchestra.run.smithi043.stdout: mon_allow_pool_size_one = true 2024-09-13T17:49:34.185 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.185 INFO:teuthology.orchestra.run.smithi043.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T17:49:34.185 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.185 INFO:teuthology.orchestra.run.smithi043.stdout: osd default data pool replay window = 5 2024-09-13T17:49:34.186 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.186 INFO:teuthology.orchestra.run.smithi043.stdout: mon allow pool delete = true 2024-09-13T17:49:34.186 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.186 INFO:teuthology.orchestra.run.smithi043.stdout: mon cluster log file level = debug 2024-09-13T17:49:34.186 INFO:teuthology.orchestra.run.smithi043.stdout: debug asserts on shutdown = true 2024-09-13T17:49:34.186 INFO:teuthology.orchestra.run.smithi043.stdout: mon health detail to clog = false 2024-09-13T17:49:34.186 INFO:teuthology.orchestra.run.smithi043.stdout: mon host = "172.21.15.40,172.21.15.43,[v2:172.21.15.43:3301,v1:172.21.15.43:6790]" 2024-09-13T17:49:34.186 INFO:teuthology.orchestra.run.smithi043.stdout: fsid = 841d6eb0-79fe-4108-906c-e8798e475996 2024-09-13T17:49:34.186 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.186 INFO:teuthology.orchestra.run.smithi043.stdout:[osd] 2024-09-13T17:49:34.186 INFO:teuthology.orchestra.run.smithi043.stdout: osd journal size = 100 2024-09-13T17:49:34.186 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.187 INFO:teuthology.orchestra.run.smithi043.stdout: osd scrub load threshold = 5.0 2024-09-13T17:49:34.187 INFO:teuthology.orchestra.run.smithi043.stdout: osd scrub max interval = 600 2024-09-13T17:49:34.187 INFO:teuthology.orchestra.run.smithi043.stdout: osd mclock profile = high_recovery_ops 2024-09-13T17:49:34.187 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.187 INFO:teuthology.orchestra.run.smithi043.stdout: osd recover clone overlap = true 2024-09-13T17:49:34.187 INFO:teuthology.orchestra.run.smithi043.stdout: osd recovery max chunk = 1048576 2024-09-13T17:49:34.187 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.187 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug shutdown = true 2024-09-13T17:49:34.187 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug op order = true 2024-09-13T17:49:34.187 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug verify stray on activate = true 2024-09-13T17:49:34.187 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.187 INFO:teuthology.orchestra.run.smithi043.stdout: osd open classes on start = true 2024-09-13T17:49:34.188 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug pg log writeout = true 2024-09-13T17:49:34.188 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.188 INFO:teuthology.orchestra.run.smithi043.stdout: osd deep scrub update digest min age = 30 2024-09-13T17:49:34.188 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.188 INFO:teuthology.orchestra.run.smithi043.stdout: osd map max advance = 10 2024-09-13T17:49:34.188 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.188 INFO:teuthology.orchestra.run.smithi043.stdout: journal zero on create = true 2024-09-13T17:49:34.188 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.188 INFO:teuthology.orchestra.run.smithi043.stdout: filestore ondisk finisher threads = 3 2024-09-13T17:49:34.188 INFO:teuthology.orchestra.run.smithi043.stdout: filestore apply finisher threads = 3 2024-09-13T17:49:34.188 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.189 INFO:teuthology.orchestra.run.smithi043.stdout: bdev debug aio = true 2024-09-13T17:49:34.189 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug misdirected ops = true 2024-09-13T17:49:34.189 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore block size = 96636764160 2024-09-13T17:49:34.189 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore fsck on mount = True 2024-09-13T17:49:34.189 INFO:teuthology.orchestra.run.smithi043.stdout: debug bluefs = 20 2024-09-13T17:49:34.189 INFO:teuthology.orchestra.run.smithi043.stdout: debug bluestore = 20 2024-09-13T17:49:34.189 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-09-13T17:49:34.189 INFO:teuthology.orchestra.run.smithi043.stdout: debug osd = 20 2024-09-13T17:49:34.189 INFO:teuthology.orchestra.run.smithi043.stdout: debug rocksdb = 10 2024-09-13T17:49:34.189 INFO:teuthology.orchestra.run.smithi043.stdout: filestore flush min = 0 2024-09-13T17:49:34.189 INFO:teuthology.orchestra.run.smithi043.stdout: mon client directed command retry = 5 2024-09-13T17:49:34.189 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T17:49:34.190 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd full ratio = 0.9 2024-09-13T17:49:34.190 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd nearfull ratio = 0.8 2024-09-13T17:49:34.190 INFO:teuthology.orchestra.run.smithi043.stdout: ms inject delay max = 1 2024-09-13T17:49:34.190 INFO:teuthology.orchestra.run.smithi043.stdout: ms inject delay probability = 0.005 2024-09-13T17:49:34.190 INFO:teuthology.orchestra.run.smithi043.stdout: ms inject delay type = client mds 2024-09-13T17:49:34.190 INFO:teuthology.orchestra.run.smithi043.stdout: ms inject socket failures = 2500 2024-09-13T17:49:34.190 INFO:teuthology.orchestra.run.smithi043.stdout: osd blocked scrub grace period = 3600 2024-09-13T17:49:34.190 INFO:teuthology.orchestra.run.smithi043.stdout: osd failsafe full ratio = 0.95 2024-09-13T17:49:34.190 INFO:teuthology.orchestra.run.smithi043.stdout: osd max markdown count = 1000 2024-09-13T17:49:34.190 INFO:teuthology.orchestra.run.smithi043.stdout: osd objectstore = bluestore 2024-09-13T17:49:34.190 INFO:teuthology.orchestra.run.smithi043.stdout: osd op complaint time = 180 2024-09-13T17:49:34.190 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.191 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr] 2024-09-13T17:49:34.191 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-09-13T17:49:34.191 INFO:teuthology.orchestra.run.smithi043.stdout: debug mgr = 20 2024-09-13T17:49:34.191 INFO:teuthology.orchestra.run.smithi043.stdout: debug mon = 20 2024-09-13T17:49:34.191 INFO:teuthology.orchestra.run.smithi043.stdout: debug auth = 20 2024-09-13T17:49:34.191 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min pgs per osd = 4 2024-09-13T17:49:34.191 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min bytes per osd = 10 2024-09-13T17:49:34.191 INFO:teuthology.orchestra.run.smithi043.stdout: mgr/telemetry/nag = false 2024-09-13T17:49:34.191 INFO:teuthology.orchestra.run.smithi043.stdout: debug client = 20 2024-09-13T17:49:34.191 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.191 INFO:teuthology.orchestra.run.smithi043.stdout:[mon] 2024-09-13T17:49:34.191 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-09-13T17:49:34.191 INFO:teuthology.orchestra.run.smithi043.stdout: debug mon = 20 2024-09-13T17:49:34.192 INFO:teuthology.orchestra.run.smithi043.stdout: debug paxos = 20 2024-09-13T17:49:34.192 INFO:teuthology.orchestra.run.smithi043.stdout: debug auth = 20 2024-09-13T17:49:34.192 INFO:teuthology.orchestra.run.smithi043.stdout: mon data avail warn = 5 2024-09-13T17:49:34.192 INFO:teuthology.orchestra.run.smithi043.stdout: mon mgr mkfs grace = 240 2024-09-13T17:49:34.192 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min pgs per osd = 4 2024-09-13T17:49:34.192 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd reporter subtree level = osd 2024-09-13T17:49:34.192 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd prime pg temp = true 2024-09-13T17:49:34.192 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min bytes per osd = 10 2024-09-13T17:49:34.192 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.192 INFO:teuthology.orchestra.run.smithi043.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T17:49:34.192 INFO:teuthology.orchestra.run.smithi043.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T17:49:34.192 INFO:teuthology.orchestra.run.smithi043.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T17:49:34.192 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.193 INFO:teuthology.orchestra.run.smithi043.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T17:49:34.193 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T17:49:34.193 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T17:49:34.193 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.193 INFO:teuthology.orchestra.run.smithi043.stdout: # 1m isn't quite enough 2024-09-13T17:49:34.193 INFO:teuthology.orchestra.run.smithi043.stdout: mon_down_mkfs_grace = 2m 2024-09-13T17:49:34.193 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.193 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_filestore_osds = false 2024-09-13T17:49:34.193 INFO:teuthology.orchestra.run.smithi043.stdout: mon op complaint time = 120 2024-09-13T17:49:34.193 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on pool no app = False 2024-09-13T17:49:34.193 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-13T17:49:34.193 INFO:teuthology.orchestra.run.smithi043.stdout:[client] 2024-09-13T17:49:34.193 INFO:teuthology.orchestra.run.smithi043.stdout: rgw cache enabled = true 2024-09-13T17:49:34.194 INFO:teuthology.orchestra.run.smithi043.stdout: rgw enable ops log = true 2024-09-13T17:49:34.194 INFO:teuthology.orchestra.run.smithi043.stdout: rgw enable usage log = true 2024-09-13T17:49:34.194 INFO:teuthology.orchestra.run.smithi043.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T17:49:34.194 INFO:teuthology.orchestra.run.smithi043.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T17:49:34.194 INFO:teuthology.orchestra.run.smithi043.stdout: client mount timeout = 600 2024-09-13T17:49:34.194 INFO:teuthology.orchestra.run.smithi043.stdout: client_shutdown_timeout = 120 2024-09-13T17:49:34.194 INFO:teuthology.orchestra.run.smithi043.stdout: debug client = 20 2024-09-13T17:49:34.194 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-09-13T17:49:34.194 INFO:teuthology.orchestra.run.smithi043.stdout: rados mon op timeout = 900 2024-09-13T17:49:34.194 INFO:teuthology.orchestra.run.smithi043.stdout: rados osd op timeout = 900 2024-09-13T17:49:34.194 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.a] 2024-09-13T17:49:34.194 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.b] 2024-09-13T17:49:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.c] 2024-09-13T17:49:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.a] 2024-09-13T17:49:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.c] 2024-09-13T17:49:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.e] 2024-09-13T17:49:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.b] 2024-09-13T17:49:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.d] 2024-09-13T17:49:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.f] 2024-09-13T17:49:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:[mds] 2024-09-13T17:49:34.195 INFO:teuthology.orchestra.run.smithi043.stdout: debug mds = 20 2024-09-13T17:49:34.195 INFO:teuthology.orchestra.run.smithi043.stdout: debug mds balancer = 20 2024-09-13T17:49:34.195 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-09-13T17:49:34.195 INFO:teuthology.orchestra.run.smithi043.stdout: mds bal fragment size max = 10000 2024-09-13T17:49:34.195 INFO:teuthology.orchestra.run.smithi043.stdout: mds bal merge size = 5 2024-09-13T17:49:34.196 INFO:teuthology.orchestra.run.smithi043.stdout: mds bal split bits = 3 2024-09-13T17:49:34.196 INFO:teuthology.orchestra.run.smithi043.stdout: mds bal split size = 100 2024-09-13T17:49:34.196 INFO:teuthology.orchestra.run.smithi043.stdout: mds debug frag = True 2024-09-13T17:49:34.196 INFO:teuthology.orchestra.run.smithi043.stdout: mds debug scatterstat = True 2024-09-13T17:49:34.196 INFO:teuthology.orchestra.run.smithi043.stdout: mds oft prefetch dirfrags = True 2024-09-13T17:49:34.196 INFO:teuthology.orchestra.run.smithi043.stdout: mds op complaint time = 180 2024-09-13T17:49:34.196 INFO:teuthology.orchestra.run.smithi043.stdout: mds verify scatter = True 2024-09-13T17:49:34.196 INFO:teuthology.orchestra.run.smithi043.stdout: mds_dir_prefetch = True 2024-09-13T17:49:34.196 INFO:teuthology.orchestra.run.smithi043.stdout: mds_sleep_rank_change = 5000000.0 2024-09-13T17:49:34.196 INFO:teuthology.orchestra.run.smithi043.stdout: mon client directed command retry = 5 2024-09-13T17:49:34.196 INFO:teuthology.orchestra.run.smithi043.stdout: ms inject delay max = 1 2024-09-13T17:49:34.197 INFO:teuthology.orchestra.run.smithi043.stdout: ms inject delay probability = 0.005 2024-09-13T17:49:34.197 INFO:teuthology.orchestra.run.smithi043.stdout: ms inject delay type = client mds osd 2024-09-13T17:49:34.197 INFO:teuthology.orchestra.run.smithi043.stdout: ms inject socket failures = 2500 2024-09-13T17:49:34.197 INFO:teuthology.orchestra.run.smithi043.stdout: osd op complaint time = 180 2024-09-13T17:49:34.197 INFO:teuthology.orchestra.run.smithi043.stdout: rados mon op timeout = 900 2024-09-13T17:49:34.197 INFO:teuthology.orchestra.run.smithi043.stdout: rados osd op timeout = 900 2024-09-13T17:49:34.197 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T17:49:34.198 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T17:49:34.330 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T17:49:34.330 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:49:34.330 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T17:49:34.337 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:49:34.337 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T17:49:34.344 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi040.front.sepia.ceph.com 2024-09-13T17:49:34.344 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:49:34.344 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T17:49:34.344 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T17:49:34.412 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:49:34.412 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:49:34.416 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi043.front.sepia.ceph.com 2024-09-13T17:49:34.417 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:49:34.417 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T17:49:34.417 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T17:49:34.445 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:49:34.445 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:49:34.452 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T17:49:34.452 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T17:49:34.452 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T17:49:34.502 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T17:49:34.505 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T17:49:34.557 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T17:49:34.568 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T17:49:34.568 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-13T17:49:34.618 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-13T17:49:34.621 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-13T17:49:34.638 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-13T17:49:34.686 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-13T17:49:34.692 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-13T17:49:34.707 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-09-13T17:49:34.754 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-09-13T17:49:34.757 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-09-13T17:49:34.769 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-13T17:49:34.821 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-13T17:49:34.824 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-13T17:49:34.841 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-13T17:49:34.888 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-13T17:49:34.891 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-13T17:49:34.904 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-09-13T17:49:34.947 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-09-13T17:49:34.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-09-13T17:49:34.966 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T17:49:34.966 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T17:49:35.009 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-13T17:49:35.023 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T17:49:35.023 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi043.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T17:49:35.023 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T17:49:35.035 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T17:49:35.035 INFO:tasks.ceph:role: osd.0 2024-09-13T17:49:35.035 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com 2024-09-13T17:49:35.035 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T17:49:35.548 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:49:35.549 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:49:35.549 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:49:35.549 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 2024-09-13T17:49:35.549 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:49:35.549 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:49:35.549 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:49:35.549 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T17:49:35.549 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:49:35.549 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:49:35.551 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-09-13T17:49:35.551 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T17:49:35.643 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T17:49:35.655 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T17:49:35.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:49:35.657 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T17:49:35.667 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T17:49:35.667 INFO:tasks.ceph:role: osd.1 2024-09-13T17:49:35.667 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com 2024-09-13T17:49:35.667 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T17:49:36.221 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:49:36.221 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:49:36.221 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:49:36.221 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 2024-09-13T17:49:36.221 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:49:36.221 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:49:36.221 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:49:36.222 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T17:49:36.222 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:49:36.222 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:49:36.223 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-09-13T17:49:36.223 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-13T17:49:36.256 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T17:49:36.265 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T17:49:36.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:49:36.266 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T17:49:36.275 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T17:49:36.275 INFO:tasks.ceph:role: osd.2 2024-09-13T17:49:36.275 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com 2024-09-13T17:49:36.276 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T17:49:36.737 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:49:36.737 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:49:36.737 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:49:36.737 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 2024-09-13T17:49:36.737 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:49:36.737 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:49:36.737 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:49:36.738 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T17:49:36.738 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:49:36.738 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:49:36.739 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-09-13T17:49:36.739 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-13T17:49:36.799 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T17:49:36.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:49:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T17:49:36.811 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-13T17:49:36.822 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T17:49:36.822 INFO:tasks.ceph:role: osd.3 2024-09-13T17:49:36.822 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com 2024-09-13T17:49:36.822 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T17:49:37.305 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:49:37.306 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:49:37.306 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:49:37.306 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 2024-09-13T17:49:37.306 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:49:37.306 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:49:37.306 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:49:37.306 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T17:49:37.306 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:49:37.306 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:49:37.307 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2024-09-13T17:49:37.308 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-13T17:49:37.360 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-13T17:49:37.370 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T17:49:37.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:49:37.372 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:49:37.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:37.439+0000 7f7af575d880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T17:49:37.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:37.439+0000 7f7af575d880 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T17:49:37.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:37.439+0000 7f7af575d880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T17:49:37.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:37.439+0000 7f7af575d880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T17:49:37.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:37.439+0000 7f7af575d880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T17:49:37.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:37.439+0000 7f7af575d880 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T17:49:40.442 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T17:49:40.459 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:49:40.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:40.539+0000 7f841132e880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T17:49:40.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:40.539+0000 7f841132e880 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T17:49:40.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:40.539+0000 7f841132e880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T17:49:40.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:40.539+0000 7f841132e880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T17:49:40.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:40.539+0000 7f841132e880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T17:49:40.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:40.539+0000 7f841132e880 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T17:49:43.575 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T17:49:43.591 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:49:43.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:43.631+0000 7f6a3c8f5880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T17:49:43.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:43.631+0000 7f6a3c8f5880 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T17:49:43.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:43.631+0000 7f6a3c8f5880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T17:49:43.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:43.631+0000 7f6a3c8f5880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T17:49:43.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:43.631+0000 7f6a3c8f5880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T17:49:43.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:43.631+0000 7f6a3c8f5880 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T17:49:46.625 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T17:49:46.641 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:49:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:46.723+0000 7f3e74f91880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-13T17:49:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:46.723+0000 7f3e74f91880 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-13T17:49:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:46.723+0000 7f3e74f91880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T17:49:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:46.723+0000 7f3e74f91880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T17:49:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:46.723+0000 7f3e74f91880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T17:49:46.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:49:46.723+0000 7f3e74f91880 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-13T17:49:49.727 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-13T17:49:49.744 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-13T17:49:49.759 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T17:49:49.759 INFO:tasks.ceph:role: osd.4 2024-09-13T17:49:49.759 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-13T17:49:49.759 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T17:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 2024-09-13T17:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:49:49.848 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T17:49:49.849 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:49:49.849 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:49:49.849 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-09-13T17:49:49.850 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-13T17:49:49.868 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-13T17:49:49.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:49:49.875 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T17:49:49.875 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-13T17:49:49.887 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T17:49:49.887 INFO:tasks.ceph:role: osd.5 2024-09-13T17:49:49.887 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-13T17:49:49.887 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T17:49:49.974 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:49:49.975 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:49:49.975 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:49:49.975 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 2024-09-13T17:49:49.975 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:49:49.975 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:49:49.975 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:49:49.975 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T17:49:49.975 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:49:49.975 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:49:49.976 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-09-13T17:49:49.976 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-13T17:49:49.993 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-13T17:49:50.001 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T17:49:50.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:49:50.002 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-13T17:49:50.015 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T17:49:50.015 INFO:tasks.ceph:role: osd.6 2024-09-13T17:49:50.015 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-13T17:49:50.015 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T17:49:50.054 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:49:50.054 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:49:50.055 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:49:50.055 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 2024-09-13T17:49:50.055 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:49:50.055 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:49:50.055 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:49:50.055 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T17:49:50.055 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:49:50.055 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:49:50.056 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-09-13T17:49:50.056 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-13T17:49:50.070 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-13T17:49:50.121 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T17:49:50.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:49:50.122 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-13T17:49:50.131 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T17:49:50.131 INFO:tasks.ceph:role: osd.7 2024-09-13T17:49:50.131 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-13T17:49:50.131 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T17:49:50.172 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:49:50.172 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:49:50.172 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:49:50.172 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 2024-09-13T17:49:50.172 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:49:50.172 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:49:50.173 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:49:50.173 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T17:49:50.173 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:49:50.173 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:49:50.174 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-09-13T17:49:50.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-13T17:49:50.237 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-13T17:49:50.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:49:50.248 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T17:49:50.248 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:49:50.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:50.328+0000 7f1f45b02880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-13T17:49:50.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:50.328+0000 7f1f45b02880 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-13T17:49:50.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:50.328+0000 7f1f45b02880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T17:49:50.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:50.328+0000 7f1f45b02880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T17:49:50.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:50.328+0000 7f1f45b02880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T17:49:50.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:50.328+0000 7f1f45b02880 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-13T17:49:53.331 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-13T17:49:53.345 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:49:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:53.428+0000 7f83d2700880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-13T17:49:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:53.428+0000 7f83d2700880 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-13T17:49:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:53.428+0000 7f83d2700880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T17:49:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:53.428+0000 7f83d2700880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T17:49:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:53.428+0000 7f83d2700880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T17:49:53.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:53.432+0000 7f83d2700880 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-13T17:49:56.430 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-13T17:49:56.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:49:56.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:56.484+0000 7faa70372880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-13T17:49:56.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:56.484+0000 7faa70372880 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-13T17:49:56.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:56.484+0000 7faa70372880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T17:49:56.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:56.484+0000 7faa70372880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T17:49:56.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:56.484+0000 7faa70372880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T17:49:56.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:56.484+0000 7faa70372880 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-13T17:49:59.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-13T17:49:59.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:49:59.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:59.580+0000 7f21470d9880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-13T17:49:59.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:59.580+0000 7f21470d9880 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-13T17:49:59.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:59.580+0000 7f21470d9880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T17:49:59.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:59.580+0000 7f21470d9880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T17:49:59.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:59.580+0000 7f21470d9880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T17:49:59.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:49:59.584+0000 7f21470d9880 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-13T17:50:02.598 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-13T17:50:02.616 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T17:50:02.616 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:50:02.616 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-13T17:50:02.633 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:50:02.633 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-13T17:50:02.690 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:50:02.690 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-13T17:50:02.700 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:50:02.700 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-09-13T17:50:02.711 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:50:02.711 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T17:50:02.722 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:50:02.722 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T17:50:02.779 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:50:02.779 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T17:50:02.790 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:50:02.790 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-13T17:50:02.802 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:50:02.802 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T17:50:02.818 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:50:02.818 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-13T17:50:02.829 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:50:02.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-13T17:50:02.845 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:50:02.845 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-09-13T17:50:02.860 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:50:02.860 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-13T17:50:02.875 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:50:02.875 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-13T17:50:02.886 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:50:02.886 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-13T17:50:02.943 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:50:02.943 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-13T17:50:02.954 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:50:02.954 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-13T17:50:02.961 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T17:50:02.961 DEBUG:teuthology.orchestra.run.smithi040:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T17:50:03.005 DEBUG:teuthology.orchestra.run.smithi043:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T17:50:03.017 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.y] 2024-09-13T17:50:03.017 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAue+RmqriwHRAA8pt04Nv00gCiZ0tlOArGAQ== 2024-09-13T17:50:03.017 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a] 2024-09-13T17:50:03.017 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAue+RmAMOdJBAAGZhzDOJftq1wq8QfeEIrvw== 2024-09-13T17:50:03.017 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.c] 2024-09-13T17:50:03.017 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAue+Rm95+kKBAA9J7859USeUakpl3mzfAV1g== 2024-09-13T17:50:03.017 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.e] 2024-09-13T17:50:03.018 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAue+Rmuym/LBAAK0gDNFc5y1nuo4S3p9Z2Eg== 2024-09-13T17:50:03.018 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.0] 2024-09-13T17:50:03.018 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAxe+RmyWV4GhAAVAQkESlRhxZTtAPXJ9/BUg== 2024-09-13T17:50:03.018 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.1] 2024-09-13T17:50:03.018 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQA0e+RmMm9mIBAA5rGjCdh/QXOA4KQm+6j6GA== 2024-09-13T17:50:03.018 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.2] 2024-09-13T17:50:03.018 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQA3e+RmYgTmJRAAeQEURYlSK/r4aaH0NAeqgA== 2024-09-13T17:50:03.018 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.3] 2024-09-13T17:50:03.018 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQA6e+Rm3XlnKxAAGQtJ4pOnY3mVKS6I8YhEvg== 2024-09-13T17:50:03.018 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.x] 2024-09-13T17:50:03.018 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAue+Rm2dIOIRAAum8EvVP/0L+vfVDceCIf/w== 2024-09-13T17:50:03.018 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b] 2024-09-13T17:50:03.018 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAue+Rm8tHQMBAAu2fSTeY5FquLo+O/FbVuWg== 2024-09-13T17:50:03.019 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.d] 2024-09-13T17:50:03.019 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAue+RmwEHLNBAANYIcIN68CaMJZy8sHx/ObQ== 2024-09-13T17:50:03.019 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.f] 2024-09-13T17:50:03.019 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAue+RmAMJPOBAAK0PC7RzhvbQlsYA8+9C61g== 2024-09-13T17:50:03.019 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.4] 2024-09-13T17:50:03.019 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQA+e+Rm7CarExAAYSopopT7h0YZ/Tp9A1yCxg== 2024-09-13T17:50:03.019 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.5] 2024-09-13T17:50:03.019 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBBe+RmdiDTGRAAgR1jH1M+7ADTZFzxf2afXQ== 2024-09-13T17:50:03.019 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.6] 2024-09-13T17:50:03.019 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBEe+RmBZ0MHRAAxu2v5fEXZYmQdKIHuPzdnA== 2024-09-13T17:50:03.019 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.7] 2024-09-13T17:50:03.019 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBHe+RmS+ziIhAAQhj5Ok/VUApA9Fh0uBfIiA== 2024-09-13T17:50:03.019 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2024-09-13T17:50:03.019 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAve+Rm2AFQABAA/xNiHy20Qa8nB4fJy/x9VQ== 2024-09-13T17:50:03.020 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr.y] 2024-09-13T17:50:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAue+RmqriwHRAA8pt04Nv00gCiZ0tlOArGAQ== 2024-09-13T17:50:03.020 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.a] 2024-09-13T17:50:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAue+RmAMOdJBAAGZhzDOJftq1wq8QfeEIrvw== 2024-09-13T17:50:03.020 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.c] 2024-09-13T17:50:03.020 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAue+Rm95+kKBAA9J7859USeUakpl3mzfAV1g== 2024-09-13T17:50:03.020 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.e] 2024-09-13T17:50:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAue+Rmuym/LBAAK0gDNFc5y1nuo4S3p9Z2Eg== 2024-09-13T17:50:03.021 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.0] 2024-09-13T17:50:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAxe+RmyWV4GhAAVAQkESlRhxZTtAPXJ9/BUg== 2024-09-13T17:50:03.021 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.1] 2024-09-13T17:50:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQA0e+RmMm9mIBAA5rGjCdh/QXOA4KQm+6j6GA== 2024-09-13T17:50:03.021 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.2] 2024-09-13T17:50:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQA3e+RmYgTmJRAAeQEURYlSK/r4aaH0NAeqgA== 2024-09-13T17:50:03.021 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.3] 2024-09-13T17:50:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQA6e+Rm3XlnKxAAGQtJ4pOnY3mVKS6I8YhEvg== 2024-09-13T17:50:03.021 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr.x] 2024-09-13T17:50:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAue+Rm2dIOIRAAum8EvVP/0L+vfVDceCIf/w== 2024-09-13T17:50:03.021 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.b] 2024-09-13T17:50:03.021 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAue+Rm8tHQMBAAu2fSTeY5FquLo+O/FbVuWg== 2024-09-13T17:50:03.021 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.d] 2024-09-13T17:50:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAue+RmwEHLNBAANYIcIN68CaMJZy8sHx/ObQ== 2024-09-13T17:50:03.022 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.f] 2024-09-13T17:50:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAue+RmAMJPOBAAK0PC7RzhvbQlsYA8+9C61g== 2024-09-13T17:50:03.022 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.4] 2024-09-13T17:50:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQA+e+Rm7CarExAAYSopopT7h0YZ/Tp9A1yCxg== 2024-09-13T17:50:03.022 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.5] 2024-09-13T17:50:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQBBe+RmdiDTGRAAgR1jH1M+7ADTZFzxf2afXQ== 2024-09-13T17:50:03.022 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.6] 2024-09-13T17:50:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQBEe+RmBZ0MHRAAxu2v5fEXZYmQdKIHuPzdnA== 2024-09-13T17:50:03.022 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.7] 2024-09-13T17:50:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQBHe+RmS+ziIhAAQhj5Ok/VUApA9Fh0uBfIiA== 2024-09-13T17:50:03.022 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-09-13T17:50:03.022 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAve+Rm2AFQABAA/xNiHy20Qa8nB4fJy/x9VQ== 2024-09-13T17:50:03.024 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T17:50:03.026 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T17:50:03.120 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:50:03.122 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:50:03.177 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:50:03.179 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:50:03.231 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:50:03.233 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:50:03.283 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:50:03.285 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:50:03.332 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:50:03.334 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:50:03.382 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:50:03.384 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:50:03.392 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:50:03.472 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:50:03.476 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:50:03.478 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:50:03.526 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T17:50:03.528 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T17:50:03.574 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:50:03.576 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:50:03.628 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:50:03.631 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:50:03.681 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:50:03.683 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:50:03.733 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:50:03.736 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:50:03.788 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:50:03.791 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:50:03.841 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:50:03.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:50:03.892 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:50:03.894 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:50:03.940 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T17:50:03.942 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T17:50:03.994 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T17:50:03.994 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T17:50:04.009 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T17:50:04.137 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T17:50:04.152 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-13T17:50:04.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T17:50:04.249 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-13T17:50:04.265 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-13T17:50:04.281 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T17:50:04.361 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-13T17:50:04.377 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:50:04.380 DEBUG:teuthology.orchestra.run.smithi043:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:50:04.385 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T17:50:04.385 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T17:50:04.385 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T17:50:04.387 INFO:tasks.ceph.mon.a:Started 2024-09-13T17:50:04.387 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T17:50:04.388 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T17:50:04.390 INFO:tasks.ceph.mon.b:Started 2024-09-13T17:50:04.390 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T17:50:04.390 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T17:50:04.392 INFO:tasks.ceph.mon.c:Started 2024-09-13T17:50:04.392 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T17:50:04.393 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-13T17:50:04.393 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-13T17:50:04.395 INFO:tasks.ceph.mgr.y:Started 2024-09-13T17:50:04.395 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T17:50:04.395 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T17:50:04.397 INFO:tasks.ceph.mgr.x:Started 2024-09-13T17:50:04.397 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T17:50:04.398 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T17:50:04.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:04.497+0000 7f1917c45700 1 Processor -- start 2024-09-13T17:50:04.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:04.497+0000 7f1917c45700 1 -- start start 2024-09-13T17:50:04.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:04.497+0000 7f1917c45700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 0x7f191015cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:04.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:04.497+0000 7f19151e0700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 0x7f191015cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:04.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:04.497+0000 7f1917c45700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19100729d0 con 0x7f191015d060 2024-09-13T17:50:04.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:04.497+0000 7f1917c45700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19100c6d00 con 0x7f1910153770 2024-09-13T17:50:04.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:04.497+0000 7f19159e1700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1910153770 0x7f191006a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:42386/0 (socket says 172.21.15.40:42386) 2024-09-13T17:50:04.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:04.497+0000 7f1917c45700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f191015a2c0 con 0x7f191006afb0 2024-09-13T17:50:04.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:04.497+0000 7f19159e1700 1 -- 172.21.15.40:0/2878708881 learned_addr learned my addr 172.21.15.40:0/2878708881 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:04.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:04.497+0000 7f19151e0700 1 -- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 msgr2=0x7f191015cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:50:04.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:04.497+0000 7f19151e0700 1 -- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 msgr2=0x7f191015cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:50:04.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:04.497+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 0x7f191015cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:50:04.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:04.497+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 0x7f191015cb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:50:04.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:04.701+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 0x7f191015cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:04.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:04.701+0000 7f19151e0700 1 -- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 msgr2=0x7f191015cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:50:04.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:04.701+0000 7f19151e0700 1 -- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 msgr2=0x7f191015cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:50:04.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:04.701+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 0x7f191015cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:50:04.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:04.701+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 0x7f191015cb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:50:05.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:05.101+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 0x7f191015cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:05.101+0000 7f19151e0700 1 -- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 msgr2=0x7f191015cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:50:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:05.101+0000 7f19151e0700 1 -- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 msgr2=0x7f191015cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:50:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:05.101+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 0x7f191015cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:50:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:05.101+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 0x7f191015cb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:50:05.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:05.901+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 0x7f191015cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:05.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:05.901+0000 7f19151e0700 1 -- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 msgr2=0x7f191015cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:50:05.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:05.901+0000 7f19151e0700 1 -- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 msgr2=0x7f191015cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:50:05.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:05.901+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 0x7f191015cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:50:05.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:05.901+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 0x7f191015cb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:50:07.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.497+0000 7f19077fe700 1 -- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 msgr2=0x7f191015cb20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:50:07.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.497+0000 7f19077fe700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 0x7f191015cb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:07.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.497+0000 7f19077fe700 1 -- 172.21.15.40:0/2878708881 >> v1:172.21.15.43:6789/0 conn(0x7f1910153770 legacy=0x7f191006a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:07.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.497+0000 7f19077fe700 1 -- 172.21.15.40:0/2878708881 >> v1:172.21.15.40:6789/0 conn(0x7f191015d060 legacy=0x7f1910163eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:07.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.497+0000 7f19077fe700 1 --2- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 0x7f18f4005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:07.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.497+0000 7f19077fe700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 0x7f18f4008780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:07.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.497+0000 7f19077fe700 1 -- 172.21.15.40:0/2878708881 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f191015a2c0 con 0x7f18f40062e0 2024-09-13T17:50:07.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.497+0000 7f19077fe700 1 -- 172.21.15.40:0/2878708881 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f4008e30 con 0x7f18f4003790 2024-09-13T17:50:07.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.497+0000 7f19159e1700 1 --2- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 0x7f18f4005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:07.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.497+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 0x7f18f4008780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:07.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.501+0000 7f19151e0700 1 -- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 msgr2=0x7f18f4008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:50:07.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.501+0000 7f19151e0700 1 -- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 msgr2=0x7f18f4008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:50:07.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.501+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 0x7f18f4008780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:50:07.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.501+0000 7f19159e1700 1 -- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 msgr2=0x7f18f4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:50:07.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.501+0000 7f19159e1700 1 -- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 msgr2=0x7f18f4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:50:07.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.501+0000 7f19159e1700 1 --2- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 0x7f18f4005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:50:07.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.501+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 0x7f18f4008780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:50:07.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.501+0000 7f19159e1700 1 --2- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 0x7f18f4005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:50:07.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.701+0000 7f19159e1700 1 --2- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 0x7f18f4005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:07.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.701+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 0x7f18f4008780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:07.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.701+0000 7f19159e1700 1 -- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 msgr2=0x7f18f4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:50:07.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.701+0000 7f19159e1700 1 -- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 msgr2=0x7f18f4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:50:07.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.701+0000 7f19159e1700 1 --2- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 0x7f18f4005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:50:07.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.701+0000 7f19151e0700 1 -- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 msgr2=0x7f18f4008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:50:07.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.701+0000 7f19151e0700 1 -- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 msgr2=0x7f18f4008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:50:07.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.701+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 0x7f18f4008780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:50:07.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.701+0000 7f19159e1700 1 --2- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 0x7f18f4005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:50:07.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:07.701+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 0x7f18f4008780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:50:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:08.101+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 0x7f18f4008780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:08.101+0000 7f19159e1700 1 --2- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 0x7f18f4005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:08.101+0000 7f19151e0700 1 -- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 msgr2=0x7f18f4008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:50:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:08.101+0000 7f19151e0700 1 -- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 msgr2=0x7f18f4008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:50:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:08.101+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 0x7f18f4008780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:50:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:08.101+0000 7f19159e1700 1 -- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 msgr2=0x7f18f4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:50:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:08.101+0000 7f19159e1700 1 -- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 msgr2=0x7f18f4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:50:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:08.101+0000 7f19159e1700 1 --2- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 0x7f18f4005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:50:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:08.101+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 0x7f18f4008780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:50:08.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:08.101+0000 7f19159e1700 1 --2- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 0x7f18f4005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:50:08.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:08.901+0000 7f19159e1700 1 --2- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 0x7f18f4005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:08.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:08.901+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 0x7f18f4008780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:08.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:08.905+0000 7f19159e1700 1 -- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 msgr2=0x7f18f4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:50:08.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:08.905+0000 7f19159e1700 1 -- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 msgr2=0x7f18f4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:50:08.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:08.905+0000 7f19159e1700 1 --2- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 0x7f18f4005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:50:08.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:08.905+0000 7f19151e0700 1 -- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 msgr2=0x7f18f4008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:50:08.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:08.905+0000 7f19151e0700 1 -- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 msgr2=0x7f18f4008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:50:08.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:08.905+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 0x7f18f4008780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:50:08.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:08.905+0000 7f19159e1700 1 --2- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 0x7f18f4005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:50:08.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:08.905+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 0x7f18f4008780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:50:09.792 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:09.788+0000 7fc023e452c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:50:09.823 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:09.821+0000 7f0ad16a92c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:50:10.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.497+0000 7f19077fe700 1 -- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 msgr2=0x7f18f4005bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:50:10.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.497+0000 7f19077fe700 1 --2- 172.21.15.40:0/2878708881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f4003790 0x7f18f4005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:10.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.497+0000 7f19077fe700 1 -- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 msgr2=0x7f18f4008780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:50:10.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.497+0000 7f19077fe700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 0x7f18f4008780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:10.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.497+0000 7f19077fe700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 0x7f18f400e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:10.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.497+0000 7f19161e2700 1 -- 172.21.15.40:0/2878708881 reap_dead start 2024-09-13T17:50:10.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.497+0000 7f19077fe700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 0x7f18f400ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:10.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.497+0000 7f19077fe700 1 -- 172.21.15.40:0/2878708881 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f191015d4d0 con 0x7f191015d060 2024-09-13T17:50:10.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.497+0000 7f19077fe700 1 -- 172.21.15.40:0/2878708881 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f191015a2c0 con 0x7f191006afb0 2024-09-13T17:50:10.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.497+0000 7f19077fe700 1 -- 172.21.15.40:0/2878708881 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f4008e30 con 0x7f18f40062e0 2024-09-13T17:50:10.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.497+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.40:3300/0 conn(0x7f191006afb0 0x7f18f400e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:10.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.501+0000 7f19161e2700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 0x7f18f400ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:10.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.501+0000 7f19149df700 1 -- 172.21.15.40:0/2878708881 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579770148 0 0) 0x7f191015d4d0 con 0x7f191015d060 2024-09-13T17:50:10.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.501+0000 7f19149df700 1 -- 172.21.15.40:0/2878708881 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18fc0035d0 con 0x7f191015d060 2024-09-13T17:50:10.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.501+0000 7f19151e0700 1 -- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 msgr2=0x7f18f400ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:10.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.501+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 0x7f18f400ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:10.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.501+0000 7f19151e0700 1 -- 172.21.15.40:0/2878708881 >> v1:172.21.15.40:6789/0 conn(0x7f191015d060 legacy=0x7f18f400b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:10.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.501+0000 7f19151e0700 1 -- 172.21.15.40:0/2878708881 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f40088b0 con 0x7f191006afb0 2024-09-13T17:50:10.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.501+0000 7f19151e0700 1 --2- 172.21.15.40:0/2878708881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f191006afb0 0x7f18f400e990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18f400ef00 tx=0x7f190c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580eb3507a0367b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:10.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.501+0000 7f19149df700 1 -- 172.21.15.40:0/2878708881 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c01ac30 con 0x7f191006afb0 2024-09-13T17:50:10.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.501+0000 7f19149df700 1 -- 172.21.15.40:0/2878708881 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190c01ad90 con 0x7f191006afb0 2024-09-13T17:50:10.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.501+0000 7f19149df700 1 -- 172.21.15.40:0/2878708881 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c018670 con 0x7f191006afb0 2024-09-13T17:50:10.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.501+0000 7f1917c45700 1 -- 172.21.15.40:0/2878708881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f191006afb0 msgr2=0x7f18f400e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:10.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.501+0000 7f1917c45700 1 --2- 172.21.15.40:0/2878708881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f191006afb0 0x7f18f400e990 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18f400ef00 tx=0x7f190c009f80 comp rx=0 tx=0).stop 2024-09-13T17:50:10.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.501+0000 7f1917c45700 1 -- 172.21.15.40:0/2878708881 shutdown_connections 2024-09-13T17:50:10.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.501+0000 7f1917c45700 1 --2- 172.21.15.40:0/2878708881 >> v2:172.21.15.43:3300/0 conn(0x7f18f40062e0 0x7f18f400ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:10.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.501+0000 7f1917c45700 1 --2- 172.21.15.40:0/2878708881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f191006afb0 0x7f18f400e990 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:10.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.501+0000 7f1917c45700 1 -- 172.21.15.40:0/2878708881 >> 172.21.15.40:0/2878708881 conn(0x7f191006d5b0 msgr2=0x7f191006d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:10.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.501+0000 7f1917c45700 1 -- 172.21.15.40:0/2878708881 shutdown_connections 2024-09-13T17:50:10.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.501+0000 7f1917c45700 1 -- 172.21.15.40:0/2878708881 wait complete. 2024-09-13T17:50:10.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.505+0000 7f1917c45700 1 Processor -- start 2024-09-13T17:50:10.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.505+0000 7f1917c45700 1 -- start start 2024-09-13T17:50:10.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f1917c45700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f40062e0 0x7f1910151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:10.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f1917c45700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f191006afb0 0x7f191015bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:10.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f1917c45700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f191015d060 0x7f191015c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:10.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f1917c45700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1910154640 con 0x7f18f40062e0 2024-09-13T17:50:10.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f1917c45700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1910154410 con 0x7f191006afb0 2024-09-13T17:50:10.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f1917c45700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19101ef550 con 0x7f191015d060 2024-09-13T17:50:10.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f19159e1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f40062e0 0x7f1910151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:10.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f19159e1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f40062e0 0x7f1910151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39642/0 (socket says 172.21.15.40:39642) 2024-09-13T17:50:10.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f19159e1700 1 -- 172.21.15.40:0/2195797570 learned_addr learned my addr 172.21.15.40:0/2195797570 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:10.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f19151e0700 1 --2- 172.21.15.40:0/2195797570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f191006afb0 0x7f191015bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:10.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f19161e2700 1 --2- 172.21.15.40:0/2195797570 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f191015d060 0x7f191015c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:10.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f19159e1700 1 -- 172.21.15.40:0/2195797570 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f191015d060 msgr2=0x7f191015c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:10.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f19159e1700 1 --2- 172.21.15.40:0/2195797570 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f191015d060 0x7f191015c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:10.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f19159e1700 1 -- 172.21.15.40:0/2195797570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f191006afb0 msgr2=0x7f191015bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:10.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f19159e1700 1 --2- 172.21.15.40:0/2195797570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f191006afb0 0x7f191015bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:10.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f19159e1700 1 -- 172.21.15.40:0/2195797570 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c009be0 con 0x7f18f40062e0 2024-09-13T17:50:10.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f19159e1700 1 --2- 172.21.15.40:0/2195797570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f40062e0 0x7f1910151fd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1900007ab0 tx=0x7f1900001540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:10.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f19067fc700 1 -- 172.21.15.40:0/2195797570 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1900001d60 con 0x7f18f40062e0 2024-09-13T17:50:10.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f19067fc700 1 -- 172.21.15.40:0/2195797570 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1900017460 con 0x7f18f40062e0 2024-09-13T17:50:10.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f1917c45700 1 -- 172.21.15.40:0/2195797570 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1910156f20 con 0x7f18f40062e0 2024-09-13T17:50:10.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f19067fc700 1 -- 172.21.15.40:0/2195797570 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19000177d0 con 0x7f18f40062e0 2024-09-13T17:50:10.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f1917c45700 1 -- 172.21.15.40:0/2195797570 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1910154b90 con 0x7f18f40062e0 2024-09-13T17:50:10.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f19067fc700 1 -- 172.21.15.40:0/2195797570 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f1900017930 con 0x7f18f40062e0 2024-09-13T17:50:10.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.509+0000 7f19067fc700 1 -- 172.21.15.40:0/2195797570 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1900020a80 con 0x7f18f40062e0 2024-09-13T17:50:10.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.513+0000 7f1917c45700 1 -- 172.21.15.40:0/2195797570 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1910067840 con 0x7f18f40062e0 2024-09-13T17:50:10.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.517+0000 7f19067fc700 1 -- 172.21.15.40:0/2195797570 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f190003fe30 con 0x7f18f40062e0 2024-09-13T17:50:10.566 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:10.561+0000 7f0ad16a92c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:50:10.576 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:10.572+0000 7fc023e452c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:50:10.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.593+0000 7f1917c45700 1 -- 172.21.15.40:0/2195797570 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f1910159c00 con 0x7f18f40062e0 2024-09-13T17:50:10.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.601+0000 7f19067fc700 1 -- 172.21.15.40:0/2195797570 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f190002e020 con 0x7f18f40062e0 2024-09-13T17:50:10.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.601+0000 7f1917c45700 1 -- 172.21.15.40:0/2195797570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f40062e0 msgr2=0x7f1910151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:10.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.601+0000 7f1917c45700 1 --2- 172.21.15.40:0/2195797570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f40062e0 0x7f1910151fd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1900007ab0 tx=0x7f1900001540 comp rx=0 tx=0).stop 2024-09-13T17:50:10.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.601+0000 7f1917c45700 1 -- 172.21.15.40:0/2195797570 shutdown_connections 2024-09-13T17:50:10.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.601+0000 7f1917c45700 1 --2- 172.21.15.40:0/2195797570 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f191015d060 0x7f191015c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:10.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.601+0000 7f1917c45700 1 --2- 172.21.15.40:0/2195797570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f191006afb0 0x7f191015bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:10.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.601+0000 7f1917c45700 1 --2- 172.21.15.40:0/2195797570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f40062e0 0x7f1910151fd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:10.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.601+0000 7f1917c45700 1 -- 172.21.15.40:0/2195797570 >> 172.21.15.40:0/2195797570 conn(0x7f191006d5b0 msgr2=0x7f191006d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:10.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.601+0000 7f1917c45700 1 -- 172.21.15.40:0/2195797570 shutdown_connections 2024-09-13T17:50:10.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:10.601+0000 7f1917c45700 1 -- 172.21.15.40:0/2195797570 wait complete. 2024-09-13T17:50:10.606 INFO:teuthology.orchestra.run.smithi040.stderr:adjusted tunables profile to default 2024-09-13T17:50:10.618 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T17:50:10.618 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T17:50:10.619 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:50:10.619 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T17:50:10.628 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:10.625+0000 7f0ad16a92c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:50:10.630 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:50:10.630 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T17:50:10.638 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:10.632+0000 7fc023e452c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:50:10.641 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:50:10.641 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T17:50:10.652 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:50:10.653 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-13T17:50:10.665 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:50:10.665 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-13T17:50:10.682 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:50:10.682 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-13T17:50:10.737 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:50:10.737 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-13T17:50:10.751 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-13T17:50:10.751 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-13T17:50:10.761 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new 89379989-479d-4bd7-b339-d984d636856c 0 2024-09-13T17:50:10.803 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:10.801+0000 7f0ad16a92c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:50:10.817 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:10.812+0000 7fc023e452c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:50:10.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.864+0000 7fded5bb8700 1 Processor -- start 2024-09-13T17:50:10.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.864+0000 7fded5bb8700 1 -- start start 2024-09-13T17:50:10.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.864+0000 7fded5bb8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fded014fdd0 0x7fded00c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:10.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.864+0000 7fded5bb8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fded00c0a10 0x7fded00c0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:10.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.864+0000 7fded5bb8700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded0159eb0 con 0x7fded00c72e0 2024-09-13T17:50:10.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.864+0000 7fded5bb8700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded00c76e0 con 0x7fded014fdd0 2024-09-13T17:50:10.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.864+0000 7fded5bb8700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded00c13a0 con 0x7fded00c0a10 2024-09-13T17:50:10.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.864+0000 7fdecffff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fded00c72e0 0x7fded00c59d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36222/0 (socket says 172.21.15.43:36222) 2024-09-13T17:50:10.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.864+0000 7fded53b7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fded00c0a10 0x7fded00c0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:10.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.864+0000 7fdecffff700 1 -- 172.21.15.43:0/38430541 learned_addr learned my addr 172.21.15.43:0/38430541 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-09-13T17:50:10.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.864+0000 7fded4bb6700 1 --2- 172.21.15.43:0/38430541 >> v2:172.21.15.40:3300/0 conn(0x7fded014fdd0 0x7fded00c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:10.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.864+0000 7fded53b7700 1 -- 172.21.15.43:0/38430541 >> v2:172.21.15.40:3300/0 conn(0x7fded014fdd0 msgr2=0x7fded00c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:10.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.864+0000 7fded53b7700 1 --2- 172.21.15.43:0/38430541 >> v2:172.21.15.40:3300/0 conn(0x7fded014fdd0 0x7fded00c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:10.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.864+0000 7fded53b7700 1 -- 172.21.15.43:0/38430541 >> v1:172.21.15.43:6789/0 conn(0x7fded00c72e0 legacy=0x7fded00c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:10.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.864+0000 7fdecf7fe700 1 -- 172.21.15.43:0/38430541 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 104243635 0 0) 0x7fded0159eb0 con 0x7fded00c72e0 2024-09-13T17:50:10.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.864+0000 7fded53b7700 1 -- 172.21.15.43:0/38430541 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded0158fc0 con 0x7fded00c0a10 2024-09-13T17:50:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded53b7700 1 --2- 172.21.15.43:0/38430541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fded00c0a10 0x7fded00c0e60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdec800a9f0 tx=0x7fdec800ad00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73c69d0899c67cf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fdecf7fe700 1 -- 172.21.15.43:0/38430541 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec800cb70 con 0x7fded00c0a10 2024-09-13T17:50:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fdecf7fe700 1 -- 172.21.15.43:0/38430541 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec800ae90 con 0x7fded00c0a10 2024-09-13T17:50:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fdecf7fe700 1 -- 172.21.15.43:0/38430541 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec8018400 con 0x7fded00c0a10 2024-09-13T17:50:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded5bb8700 1 -- 172.21.15.43:0/38430541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fded00c0a10 msgr2=0x7fded00c0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded5bb8700 1 --2- 172.21.15.43:0/38430541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fded00c0a10 0x7fded00c0e60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdec800a9f0 tx=0x7fdec800ad00 comp rx=0 tx=0).stop 2024-09-13T17:50:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded5bb8700 1 -- 172.21.15.43:0/38430541 shutdown_connections 2024-09-13T17:50:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded5bb8700 1 --2- 172.21.15.43:0/38430541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fded00c0a10 0x7fded00c0e60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded5bb8700 1 --2- 172.21.15.43:0/38430541 >> v2:172.21.15.40:3300/0 conn(0x7fded014fdd0 0x7fded00c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded5bb8700 1 -- 172.21.15.43:0/38430541 >> 172.21.15.43:0/38430541 conn(0x7fded0066090 msgr2=0x7fded00664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:10.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded5bb8700 1 -- 172.21.15.43:0/38430541 shutdown_connections 2024-09-13T17:50:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded5bb8700 1 -- 172.21.15.43:0/38430541 wait complete. 2024-09-13T17:50:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded5bb8700 1 Processor -- start 2024-09-13T17:50:10.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded5bb8700 1 -- start start 2024-09-13T17:50:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded5bb8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fded00c0a10 0x7fded00ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:10.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded5bb8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fded00c72e0 0x7fded00cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded5bb8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded014fdd0 0x7fded01ef310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded5bb8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded00aee60 con 0x7fded00c72e0 2024-09-13T17:50:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded5bb8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded01eee80 con 0x7fded014fdd0 2024-09-13T17:50:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded5bb8700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded00c8e30 con 0x7fded00c0a10 2024-09-13T17:50:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded4bb6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fded00c0a10 0x7fded00ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded4bb6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fded00c0a10 0x7fded00ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:55742/0 (socket says 172.21.15.43:55742) 2024-09-13T17:50:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded4bb6700 1 -- 172.21.15.43:0/1151927039 learned_addr learned my addr 172.21.15.43:0/1151927039 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T17:50:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.868+0000 7fded53b7700 1 --2- 172.21.15.43:0/1151927039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded014fdd0 0x7fded01ef310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.872+0000 7fdecffff700 1 --2- 172.21.15.43:0/1151927039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fded00c72e0 0x7fded00cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.872+0000 7fded4bb6700 1 -- 172.21.15.43:0/1151927039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded014fdd0 msgr2=0x7fded01ef310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.872+0000 7fded4bb6700 1 --2- 172.21.15.43:0/1151927039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded014fdd0 0x7fded01ef310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.872+0000 7fded4bb6700 1 -- 172.21.15.43:0/1151927039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fded00c72e0 msgr2=0x7fded00cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.872+0000 7fded4bb6700 1 --2- 172.21.15.43:0/1151927039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fded00c72e0 0x7fded00cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:10.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.872+0000 7fded4bb6700 1 -- 172.21.15.43:0/1151927039 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec800a650 con 0x7fded00c0a10 2024-09-13T17:50:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.872+0000 7fded4bb6700 1 --2- 172.21.15.43:0/1151927039 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fded00c0a10 0x7fded00ca220 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009d10 tx=0x7fdec000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:10.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.872+0000 7fdecd7fa700 1 -- 172.21.15.43:0/1151927039 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec000cdc0 con 0x7fded00c0a10 2024-09-13T17:50:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.872+0000 7fdecd7fa700 1 -- 172.21.15.43:0/1151927039 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec000f040 con 0x7fded00c0a10 2024-09-13T17:50:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.872+0000 7fded5bb8700 1 -- 172.21.15.43:0/1151927039 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded0149df0 con 0x7fded00c0a10 2024-09-13T17:50:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.872+0000 7fdecd7fa700 1 -- 172.21.15.43:0/1151927039 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec00146a0 con 0x7fded00c0a10 2024-09-13T17:50:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.872+0000 7fded5bb8700 1 -- 172.21.15.43:0/1151927039 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded00c93b0 con 0x7fded00c0a10 2024-09-13T17:50:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.872+0000 7fdecd7fa700 1 -- 172.21.15.43:0/1151927039 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fdec0014b00 con 0x7fded00c0a10 2024-09-13T17:50:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.872+0000 7fdecd7fa700 1 -- 172.21.15.43:0/1151927039 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdec001dac0 con 0x7fded00c0a10 2024-09-13T17:50:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.872+0000 7fded5bb8700 1 -- 172.21.15.43:0/1151927039 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded0070400 con 0x7fded00c0a10 2024-09-13T17:50:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.876+0000 7fdecd7fa700 1 -- 172.21.15.43:0/1151927039 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fdec0018020 con 0x7fded00c0a10 2024-09-13T17:50:10.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.952+0000 7fded5bb8700 1 -- 172.21.15.43:0/1151927039 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd new", "uuid": "89379989-479d-4bd7-b339-d984d636856c", "id": 0} v 0) v1 -- 0x7fded00c65a0 con 0x7fded00c0a10 2024-09-13T17:50:10.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.960+0000 7fdecd7fa700 1 -- 172.21.15.43:0/1151927039 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "89379989-479d-4bd7-b339-d984d636856c", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdec001d400 con 0x7fded00c0a10 2024-09-13T17:50:10.967 INFO:teuthology.orchestra.run.smithi043.stdout:0 2024-09-13T17:50:10.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.964+0000 7fded5bb8700 1 -- 172.21.15.43:0/1151927039 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fded00c0a10 msgr2=0x7fded00ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:10.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.964+0000 7fded5bb8700 1 --2- 172.21.15.43:0/1151927039 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fded00c0a10 0x7fded00ca220 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009d10 tx=0x7fdec000c3a0 comp rx=0 tx=0).stop 2024-09-13T17:50:10.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.964+0000 7fded5bb8700 1 -- 172.21.15.43:0/1151927039 shutdown_connections 2024-09-13T17:50:10.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.964+0000 7fded5bb8700 1 --2- 172.21.15.43:0/1151927039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded014fdd0 0x7fded01ef310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:10.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.964+0000 7fded5bb8700 1 --2- 172.21.15.43:0/1151927039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fded00c72e0 0x7fded00cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:10.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.964+0000 7fded5bb8700 1 --2- 172.21.15.43:0/1151927039 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fded00c0a10 0x7fded00ca220 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:10.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.964+0000 7fded5bb8700 1 -- 172.21.15.43:0/1151927039 >> 172.21.15.43:0/1151927039 conn(0x7fded0066090 msgr2=0x7fded00c67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:10.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.964+0000 7fded5bb8700 1 -- 172.21.15.43:0/1151927039 shutdown_connections 2024-09-13T17:50:10.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:10.964+0000 7fded5bb8700 1 -- 172.21.15.43:0/1151927039 wait complete. 2024-09-13T17:50:10.981 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new bb549a81-885b-4473-9158-943c07955975 1 2024-09-13T17:50:11.040 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:11.037+0000 7f0ad16a92c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:50:11.057 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:11.052+0000 7fc023e452c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:50:11.097 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:11.093+0000 7f0ad16a92c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:50:11.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.092+0000 7fd1b23e8700 1 Processor -- start 2024-09-13T17:50:11.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.092+0000 7fd1b23e8700 1 -- start start 2024-09-13T17:50:11.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1b23e8700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd1ac1514e0 0x7fd1ac0b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1b23e8700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ac0c8cc0 con 0x7fd1ac070460 2024-09-13T17:50:11.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1b23e8700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ac0c8fa0 con 0x7fd1ac1518c0 2024-09-13T17:50:11.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1b23e8700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ac157390 con 0x7fd1ac1514e0 2024-09-13T17:50:11.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1b13e6700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd1ac1514e0 0x7fd1ac0b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1b0be5700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd1ac1518c0 0x7fd1ac157aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36238/0 (socket says 172.21.15.43:36238) 2024-09-13T17:50:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1b0be5700 1 -- 172.21.15.43:0/736598917 learned_addr learned my addr 172.21.15.43:0/736598917 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-09-13T17:50:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1b13e6700 1 -- 172.21.15.43:0/736598917 >> v1:172.21.15.43:6789/0 conn(0x7fd1ac1518c0 legacy=0x7fd1ac157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1b13e6700 1 -- 172.21.15.43:0/736598917 >> v1:172.21.15.40:6789/0 conn(0x7fd1ac070460 legacy=0x7fd1ac0c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1b13e6700 1 -- 172.21.15.43:0/736598917 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac0c9280 con 0x7fd1ac1514e0 2024-09-13T17:50:11.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1b13e6700 1 --2- 172.21.15.43:0/736598917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac1514e0 0x7fd1ac0b2c70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009850 tx=0x7fd1a8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f50e42d32255177e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1a3fff700 1 -- 172.21.15.43:0/736598917 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8011830 con 0x7fd1ac1514e0 2024-09-13T17:50:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1a3fff700 1 -- 172.21.15.43:0/736598917 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a8011990 con 0x7fd1ac1514e0 2024-09-13T17:50:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1a3fff700 1 -- 172.21.15.43:0/736598917 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8011d20 con 0x7fd1ac1514e0 2024-09-13T17:50:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1b23e8700 1 -- 172.21.15.43:0/736598917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac1514e0 msgr2=0x7fd1ac0b2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1b23e8700 1 --2- 172.21.15.43:0/736598917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac1514e0 0x7fd1ac0b2c70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009850 tx=0x7fd1a8009b60 comp rx=0 tx=0).stop 2024-09-13T17:50:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1b23e8700 1 -- 172.21.15.43:0/736598917 shutdown_connections 2024-09-13T17:50:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1b23e8700 1 --2- 172.21.15.43:0/736598917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac1514e0 0x7fd1ac0b2c70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1b23e8700 1 -- 172.21.15.43:0/736598917 >> 172.21.15.43:0/736598917 conn(0x7fd1ac0ccef0 msgr2=0x7fd1ac0cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:11.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1b23e8700 1 -- 172.21.15.43:0/736598917 shutdown_connections 2024-09-13T17:50:11.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.096+0000 7fd1b23e8700 1 -- 172.21.15.43:0/736598917 wait complete. 2024-09-13T17:50:11.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b23e8700 1 Processor -- start 2024-09-13T17:50:11.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b23e8700 1 -- start start 2024-09-13T17:50:11.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b23e8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ac070460 0x7fd1ac140b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b23e8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1ac1514e0 0x7fd1ac1f2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b23e8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac1518c0 0x7fd1ac0cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b23e8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac0c49e0 con 0x7fd1ac070460 2024-09-13T17:50:11.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b23e8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac1ef9f0 con 0x7fd1ac1518c0 2024-09-13T17:50:11.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b23e8700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac1410d0 con 0x7fd1ac1514e0 2024-09-13T17:50:11.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b0be5700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1ac1514e0 0x7fd1ac1f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b1be7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac1518c0 0x7fd1ac0cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b0be5700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1ac1514e0 0x7fd1ac1f2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:55748/0 (socket says 172.21.15.43:55748) 2024-09-13T17:50:11.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b1be7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac1518c0 0x7fd1ac0cbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38118/0 (socket says 172.21.15.43:38118) 2024-09-13T17:50:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b0be5700 1 -- 172.21.15.43:0/3118670922 learned_addr learned my addr 172.21.15.43:0/3118670922 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T17:50:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b13e6700 1 --2- 172.21.15.43:0/3118670922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ac070460 0x7fd1ac140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b0be5700 1 -- 172.21.15.43:0/3118670922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac1518c0 msgr2=0x7fd1ac0cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b0be5700 1 --2- 172.21.15.43:0/3118670922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac1518c0 0x7fd1ac0cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b0be5700 1 -- 172.21.15.43:0/3118670922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ac070460 msgr2=0x7fd1ac140b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b0be5700 1 --2- 172.21.15.43:0/3118670922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ac070460 0x7fd1ac140b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b0be5700 1 -- 172.21.15.43:0/3118670922 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a80094b0 con 0x7fd1ac1514e0 2024-09-13T17:50:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b1be7700 1 --2- 172.21.15.43:0/3118670922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac1518c0 0x7fd1ac0cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:50:11.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b0be5700 1 --2- 172.21.15.43:0/3118670922 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1ac1514e0 0x7fd1ac1f2690 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00c4d0 tx=0x7fd19c00c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1a1ffb700 1 -- 172.21.15.43:0/3118670922 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c005d90 con 0x7fd1ac1514e0 2024-09-13T17:50:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1a1ffb700 1 -- 172.21.15.43:0/3118670922 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19c00ce40 con 0x7fd1ac1514e0 2024-09-13T17:50:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b23e8700 1 -- 172.21.15.43:0/3118670922 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ac1ef120 con 0x7fd1ac1514e0 2024-09-13T17:50:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1a1ffb700 1 -- 172.21.15.43:0/3118670922 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c0072d0 con 0x7fd1ac1514e0 2024-09-13T17:50:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1b23e8700 1 -- 172.21.15.43:0/3118670922 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ac06d300 con 0x7fd1ac1514e0 2024-09-13T17:50:11.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1a1ffb700 1 -- 172.21.15.43:0/3118670922 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fd19c01c440 con 0x7fd1ac1514e0 2024-09-13T17:50:11.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.100+0000 7fd1a1ffb700 1 -- 172.21.15.43:0/3118670922 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fd19c0127b0 con 0x7fd1ac1514e0 2024-09-13T17:50:11.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.104+0000 7fd1b23e8700 1 -- 172.21.15.43:0/3118670922 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ac067520 con 0x7fd1ac1514e0 2024-09-13T17:50:11.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.104+0000 7fd1a1ffb700 1 -- 172.21.15.43:0/3118670922 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fd19c021060 con 0x7fd1ac1514e0 2024-09-13T17:50:11.113 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:11.108+0000 7fc023e452c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:50:11.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.180+0000 7fd1b23e8700 1 -- 172.21.15.43:0/3118670922 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bb549a81-885b-4473-9158-943c07955975", "id": 1} v 0) v1 -- 0x7fd1ac0b2c70 con 0x7fd1ac1514e0 2024-09-13T17:50:11.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.192+0000 7fd1a1ffb700 1 -- 172.21.15.43:0/3118670922 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bb549a81-885b-4473-9158-943c07955975", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd19c017070 con 0x7fd1ac1514e0 2024-09-13T17:50:11.196 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-13T17:50:11.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.192+0000 7fd1b23e8700 1 -- 172.21.15.43:0/3118670922 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1ac1514e0 msgr2=0x7fd1ac1f2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.192+0000 7fd1b23e8700 1 --2- 172.21.15.43:0/3118670922 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1ac1514e0 0x7fd1ac1f2690 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00c4d0 tx=0x7fd19c00c7e0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.192+0000 7fd1b23e8700 1 -- 172.21.15.43:0/3118670922 shutdown_connections 2024-09-13T17:50:11.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.192+0000 7fd1b23e8700 1 --2- 172.21.15.43:0/3118670922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1ac1518c0 0x7fd1ac0cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.192+0000 7fd1b23e8700 1 --2- 172.21.15.43:0/3118670922 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1ac1514e0 0x7fd1ac1f2690 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.192+0000 7fd1b23e8700 1 --2- 172.21.15.43:0/3118670922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1ac070460 0x7fd1ac140b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.192+0000 7fd1b23e8700 1 -- 172.21.15.43:0/3118670922 >> 172.21.15.43:0/3118670922 conn(0x7fd1ac0ccef0 msgr2=0x7fd1ac0c7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:11.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.192+0000 7fd1b23e8700 1 -- 172.21.15.43:0/3118670922 shutdown_connections 2024-09-13T17:50:11.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.192+0000 7fd1b23e8700 1 -- 172.21.15.43:0/3118670922 wait complete. 2024-09-13T17:50:11.210 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268 2 2024-09-13T17:50:11.297 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:11.293+0000 7f0ad16a92c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:50:11.304 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:11.300+0000 7fc023e452c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:50:11.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.312+0000 7f6d28ac8700 1 Processor -- start 2024-09-13T17:50:11.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.312+0000 7f6d28ac8700 1 -- start start 2024-09-13T17:50:11.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d28ac8700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6d24157f40 0x7f6d2415e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d28ac8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d240c6870 0x7f6d24160f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d28ac8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6d2406c160 0x7f6d2406c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d28ac8700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d240c80e0 con 0x7f6d2406c160 2024-09-13T17:50:11.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d28ac8700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d2406c5f0 con 0x7f6d24157f40 2024-09-13T17:50:11.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d28ac8700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d240c8350 con 0x7f6d240c6870 2024-09-13T17:50:11.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d1bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d240c6870 0x7f6d24160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d2259c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6d24157f40 0x7f6d2415e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d1bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d240c6870 0x7f6d24160f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:55760/0 (socket says 172.21.15.43:55760) 2024-09-13T17:50:11.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d2259c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6d24157f40 0x7f6d2415e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38120/0 (socket says 172.21.15.43:38120) 2024-09-13T17:50:11.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d1bfff700 1 -- 172.21.15.43:0/224662581 learned_addr learned my addr 172.21.15.43:0/224662581 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T17:50:11.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d22d9d700 1 --2- 172.21.15.43:0/224662581 >> v2:172.21.15.40:3300/0 conn(0x7f6d2406c160 0x7f6d2406c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d1bfff700 1 -- 172.21.15.43:0/224662581 >> v2:172.21.15.43:3300/0 conn(0x7f6d24157f40 msgr2=0x7f6d2415e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:11.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d1bfff700 1 --2- 172.21.15.43:0/224662581 >> v2:172.21.15.43:3300/0 conn(0x7f6d24157f40 0x7f6d2415e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d1bfff700 1 -- 172.21.15.43:0/224662581 >> v2:172.21.15.40:3300/0 conn(0x7f6d2406c160 msgr2=0x7f6d2406c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:11.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d1bfff700 1 --2- 172.21.15.43:0/224662581 >> v2:172.21.15.40:3300/0 conn(0x7f6d2406c160 0x7f6d2406c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d1bfff700 1 -- 172.21.15.43:0/224662581 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2415dae0 con 0x7f6d240c6870 2024-09-13T17:50:11.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d2259c700 1 --2- 172.21.15.43:0/224662581 >> v2:172.21.15.43:3300/0 conn(0x7f6d24157f40 0x7f6d2415e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:50:11.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d1bfff700 1 --2- 172.21.15.43:0/224662581 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d240c6870 0x7f6d24160f40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6d240cce90 tx=0x7f6d0c009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80b8d6dccb6340d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d21d9b700 1 -- 172.21.15.43:0/224662581 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c011830 con 0x7f6d240c6870 2024-09-13T17:50:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d21d9b700 1 -- 172.21.15.43:0/224662581 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0c011990 con 0x7f6d240c6870 2024-09-13T17:50:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d21d9b700 1 -- 172.21.15.43:0/224662581 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c011d20 con 0x7f6d240c6870 2024-09-13T17:50:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d28ac8700 1 -- 172.21.15.43:0/224662581 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d240c6870 msgr2=0x7f6d24160f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d28ac8700 1 --2- 172.21.15.43:0/224662581 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d240c6870 0x7f6d24160f40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6d240cce90 tx=0x7f6d0c009bf0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d28ac8700 1 -- 172.21.15.43:0/224662581 shutdown_connections 2024-09-13T17:50:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d28ac8700 1 --2- 172.21.15.43:0/224662581 >> v2:172.21.15.40:3300/0 conn(0x7f6d2406c160 0x7f6d2406c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d28ac8700 1 --2- 172.21.15.43:0/224662581 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d240c6870 0x7f6d24160f40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d28ac8700 1 --2- 172.21.15.43:0/224662581 >> v2:172.21.15.43:3300/0 conn(0x7f6d24157f40 0x7f6d2415e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d28ac8700 1 -- 172.21.15.43:0/224662581 >> 172.21.15.43:0/224662581 conn(0x7f6d240c8970 msgr2=0x7f6d240c8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d28ac8700 1 -- 172.21.15.43:0/224662581 shutdown_connections 2024-09-13T17:50:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.316+0000 7f6d28ac8700 1 -- 172.21.15.43:0/224662581 wait complete. 2024-09-13T17:50:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d28ac8700 1 Processor -- start 2024-09-13T17:50:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d28ac8700 1 -- start start 2024-09-13T17:50:11.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d28ac8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d2406c160 0x7f6d240caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d28ac8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d240c6870 0x7f6d2415fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d28ac8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d24157f40 0x7f6d24160090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d28ac8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d241f33b0 con 0x7f6d2406c160 2024-09-13T17:50:11.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d28ac8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d241f3630 con 0x7f6d24157f40 2024-09-13T17:50:11.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d28ac8700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d241f3910 con 0x7f6d240c6870 2024-09-13T17:50:11.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d1bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d240c6870 0x7f6d2415fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d22d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d24157f40 0x7f6d24160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d1bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d240c6870 0x7f6d2415fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:55762/0 (socket says 172.21.15.43:55762) 2024-09-13T17:50:11.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d1bfff700 1 -- 172.21.15.43:0/313677424 learned_addr learned my addr 172.21.15.43:0/313677424 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T17:50:11.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d2259c700 1 --2- 172.21.15.43:0/313677424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d2406c160 0x7f6d240caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d22d9d700 1 -- 172.21.15.43:0/313677424 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d240c6870 msgr2=0x7f6d2415fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d22d9d700 1 --2- 172.21.15.43:0/313677424 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d240c6870 0x7f6d2415fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d22d9d700 1 -- 172.21.15.43:0/313677424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d2406c160 msgr2=0x7f6d240caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d22d9d700 1 --2- 172.21.15.43:0/313677424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d2406c160 0x7f6d240caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d22d9d700 1 -- 172.21.15.43:0/313677424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d140094b0 con 0x7f6d24157f40 2024-09-13T17:50:11.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d22d9d700 1 --2- 172.21.15.43:0/313677424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d24157f40 0x7f6d24160090 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00eed0 tx=0x7f6d1c00b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:11.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d1affd700 1 -- 172.21.15.43:0/313677424 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c010040 con 0x7f6d24157f40 2024-09-13T17:50:11.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d1affd700 1 -- 172.21.15.43:0/313677424 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1c00ba80 con 0x7f6d24157f40 2024-09-13T17:50:11.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d28ac8700 1 -- 172.21.15.43:0/313677424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d0c009570 con 0x7f6d24157f40 2024-09-13T17:50:11.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d1affd700 1 -- 172.21.15.43:0/313677424 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c00be30 con 0x7f6d24157f40 2024-09-13T17:50:11.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.320+0000 7f6d28ac8700 1 -- 172.21.15.43:0/313677424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d241f3ed0 con 0x7f6d24157f40 2024-09-13T17:50:11.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.324+0000 7f6d1affd700 1 -- 172.21.15.43:0/313677424 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f6d1c01d700 con 0x7f6d24157f40 2024-09-13T17:50:11.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.324+0000 7f6d1affd700 1 -- 172.21.15.43:0/313677424 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f6d1c014be0 con 0x7f6d24157f40 2024-09-13T17:50:11.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.324+0000 7f6d28ac8700 1 -- 172.21.15.43:0/313677424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d240c2e20 con 0x7f6d24157f40 2024-09-13T17:50:11.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.328+0000 7f6d1affd700 1 -- 172.21.15.43:0/313677424 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f6d1c03be30 con 0x7f6d24157f40 2024-09-13T17:50:11.366 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:11.361+0000 7f0ad16a92c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T17:50:11.373 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:11.368+0000 7fc023e452c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T17:50:11.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.400+0000 7f6d28ac8700 1 -- 172.21.15.43:0/313677424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268", "id": 2} v 0) v1 -- 0x7f6d240726f0 con 0x7f6d24157f40 2024-09-13T17:50:11.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.412+0000 7f6d1affd700 1 -- 172.21.15.43:0/313677424 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6d1c02a030 con 0x7f6d24157f40 2024-09-13T17:50:11.416 INFO:teuthology.orchestra.run.smithi043.stdout:2 2024-09-13T17:50:11.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.412+0000 7f6d28ac8700 1 -- 172.21.15.43:0/313677424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d24157f40 msgr2=0x7f6d24160090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.412+0000 7f6d28ac8700 1 --2- 172.21.15.43:0/313677424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d24157f40 0x7f6d24160090 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00eed0 tx=0x7f6d1c00b390 comp rx=0 tx=0).stop 2024-09-13T17:50:11.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.412+0000 7f6d28ac8700 1 -- 172.21.15.43:0/313677424 shutdown_connections 2024-09-13T17:50:11.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.412+0000 7f6d28ac8700 1 --2- 172.21.15.43:0/313677424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d24157f40 0x7f6d24160090 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.412+0000 7f6d28ac8700 1 --2- 172.21.15.43:0/313677424 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d240c6870 0x7f6d2415fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.412+0000 7f6d28ac8700 1 --2- 172.21.15.43:0/313677424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d2406c160 0x7f6d240caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.412+0000 7f6d28ac8700 1 -- 172.21.15.43:0/313677424 >> 172.21.15.43:0/313677424 conn(0x7f6d240c8970 msgr2=0x7f6d24157280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:11.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.412+0000 7f6d28ac8700 1 -- 172.21.15.43:0/313677424 shutdown_connections 2024-09-13T17:50:11.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.412+0000 7f6d28ac8700 1 -- 172.21.15.43:0/313677424 wait complete. 2024-09-13T17:50:11.433 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new df6ebccc-cd93-4154-9559-d2724c12fcae 3 2024-09-13T17:50:11.438 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:11.433+0000 7f0ad16a92c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:50:11.440 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:11.436+0000 7fc023e452c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:50:11.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62f0fd4700 1 Processor -- start 2024-09-13T17:50:11.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62f0fd4700 1 -- start start 2024-09-13T17:50:11.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62f0fd4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f62ec1503f0 0x7f62ec0c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62f0fd4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f62ec070a70 0x7f62ec071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62f0fd4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f62ec071970 0x7f62ec156710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62f0fd4700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ec0c6d20 con 0x7f62ec1503f0 2024-09-13T17:50:11.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62f0fd4700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ec159770 con 0x7f62ec071970 2024-09-13T17:50:11.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62f0fd4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec159a30 con 0x7f62ec070a70 2024-09-13T17:50:11.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62eaffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f62ec070a70 0x7f62ec071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62ebfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f62ec071970 0x7f62ec156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62eaffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f62ec070a70 0x7f62ec071430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:55772/0 (socket says 172.21.15.43:55772) 2024-09-13T17:50:11.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62ebfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f62ec071970 0x7f62ec156710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38134/0 (socket says 172.21.15.43:38134) 2024-09-13T17:50:11.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62eaffd700 1 -- 172.21.15.43:0/2325565899 learned_addr learned my addr 172.21.15.43:0/2325565899 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T17:50:11.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62eb7fe700 1 --2- 172.21.15.43:0/2325565899 >> v2:172.21.15.40:3300/0 conn(0x7f62ec1503f0 0x7f62ec0c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62ebfff700 1 -- 172.21.15.43:0/2325565899 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f62ec070a70 msgr2=0x7f62ec071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:11.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62ebfff700 1 --2- 172.21.15.43:0/2325565899 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f62ec070a70 0x7f62ec071430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62ebfff700 1 -- 172.21.15.43:0/2325565899 >> v2:172.21.15.40:3300/0 conn(0x7f62ec1503f0 msgr2=0x7f62ec0c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:11.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62ebfff700 1 --2- 172.21.15.43:0/2325565899 >> v2:172.21.15.40:3300/0 conn(0x7f62ec1503f0 0x7f62ec0c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62ebfff700 1 -- 172.21.15.43:0/2325565899 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec159d10 con 0x7f62ec071970 2024-09-13T17:50:11.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62eb7fe700 1 --2- 172.21.15.43:0/2325565899 >> v2:172.21.15.40:3300/0 conn(0x7f62ec1503f0 0x7f62ec0c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:50:11.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62ebfff700 1 --2- 172.21.15.43:0/2325565899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62ec071970 0x7f62ec156710 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f62ec153d70 tx=0x7f62e400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2261be07f4bbaea server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:11.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62ea7fc700 1 -- 172.21.15.43:0/2325565899 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e400bd20 con 0x7f62ec071970 2024-09-13T17:50:11.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62ea7fc700 1 -- 172.21.15.43:0/2325565899 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e400be80 con 0x7f62ec071970 2024-09-13T17:50:11.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.536+0000 7f62ea7fc700 1 -- 172.21.15.43:0/2325565899 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e40116a0 con 0x7f62ec071970 2024-09-13T17:50:11.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.540+0000 7f62f0fd4700 1 -- 172.21.15.43:0/2325565899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62ec071970 msgr2=0x7f62ec156710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.540+0000 7f62f0fd4700 1 --2- 172.21.15.43:0/2325565899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62ec071970 0x7f62ec156710 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f62ec153d70 tx=0x7f62e400b330 comp rx=0 tx=0).stop 2024-09-13T17:50:11.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.540+0000 7f62f0fd4700 1 -- 172.21.15.43:0/2325565899 shutdown_connections 2024-09-13T17:50:11.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.540+0000 7f62f0fd4700 1 --2- 172.21.15.43:0/2325565899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62ec071970 0x7f62ec156710 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.540+0000 7f62f0fd4700 1 --2- 172.21.15.43:0/2325565899 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f62ec070a70 0x7f62ec071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.540+0000 7f62f0fd4700 1 --2- 172.21.15.43:0/2325565899 >> v2:172.21.15.40:3300/0 conn(0x7f62ec1503f0 0x7f62ec0c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.540+0000 7f62f0fd4700 1 -- 172.21.15.43:0/2325565899 >> 172.21.15.43:0/2325565899 conn(0x7f62ec0cc7f0 msgr2=0x7f62ec0c88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:11.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.540+0000 7f62f0fd4700 1 -- 172.21.15.43:0/2325565899 shutdown_connections 2024-09-13T17:50:11.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.540+0000 7f62f0fd4700 1 -- 172.21.15.43:0/2325565899 wait complete. 2024-09-13T17:50:11.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.540+0000 7f62f0fd4700 1 Processor -- start 2024-09-13T17:50:11.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.540+0000 7f62f0fd4700 1 -- start start 2024-09-13T17:50:11.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.540+0000 7f62f0fd4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62ec070a70 0x7f62ec153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.540+0000 7f62f0fd4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f62ec071970 0x7f62ec143900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.540+0000 7f62f0fd4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62ec1503f0 0x7f62ec0ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.540+0000 7f62f0fd4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec1595e0 con 0x7f62ec070a70 2024-09-13T17:50:11.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.540+0000 7f62f0fd4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0c9df0 con 0x7f62ec1503f0 2024-09-13T17:50:11.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.540+0000 7f62f0fd4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec158730 con 0x7f62ec071970 2024-09-13T17:50:11.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.544+0000 7f62eaffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f62ec071970 0x7f62ec143900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.544+0000 7f62ebfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62ec1503f0 0x7f62ec0ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.544+0000 7f62eaffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f62ec071970 0x7f62ec143900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:55780/0 (socket says 172.21.15.43:55780) 2024-09-13T17:50:11.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.544+0000 7f62eaffd700 1 -- 172.21.15.43:0/1763519138 learned_addr learned my addr 172.21.15.43:0/1763519138 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T17:50:11.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.544+0000 7f62eb7fe700 1 --2- 172.21.15.43:0/1763519138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62ec070a70 0x7f62ec153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.544+0000 7f62eaffd700 1 -- 172.21.15.43:0/1763519138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62ec1503f0 msgr2=0x7f62ec0ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.544+0000 7f62eaffd700 1 --2- 172.21.15.43:0/1763519138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62ec1503f0 0x7f62ec0ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.544+0000 7f62eaffd700 1 -- 172.21.15.43:0/1763519138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62ec070a70 msgr2=0x7f62ec153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.544+0000 7f62eaffd700 1 --2- 172.21.15.43:0/1763519138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62ec070a70 0x7f62ec153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.544+0000 7f62eaffd700 1 -- 172.21.15.43:0/1763519138 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e4009a70 con 0x7f62ec071970 2024-09-13T17:50:11.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.544+0000 7f62ebfff700 1 --2- 172.21.15.43:0/1763519138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62ec1503f0 0x7f62ec0ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:50:11.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.544+0000 7f62eaffd700 1 --2- 172.21.15.43:0/1763519138 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f62ec071970 0x7f62ec143900 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f62e000e8e0 tx=0x7f62e000ebf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:11.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.544+0000 7f62d3fff700 1 -- 172.21.15.43:0/1763519138 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e0013680 con 0x7f62ec071970 2024-09-13T17:50:11.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.544+0000 7f62d3fff700 1 -- 172.21.15.43:0/1763519138 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e00137e0 con 0x7f62ec071970 2024-09-13T17:50:11.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.544+0000 7f62f0fd4700 1 -- 172.21.15.43:0/1763519138 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ec1574d0 con 0x7f62ec071970 2024-09-13T17:50:11.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.544+0000 7f62d3fff700 1 -- 172.21.15.43:0/1763519138 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e0013b50 con 0x7f62ec071970 2024-09-13T17:50:11.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.544+0000 7f62f0fd4700 1 -- 172.21.15.43:0/1763519138 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ec154390 con 0x7f62ec071970 2024-09-13T17:50:11.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.544+0000 7f62d3fff700 1 -- 172.21.15.43:0/1763519138 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f62e001c450 con 0x7f62ec071970 2024-09-13T17:50:11.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.544+0000 7f62d3fff700 1 -- 172.21.15.43:0/1763519138 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f62e00129f0 con 0x7f62ec071970 2024-09-13T17:50:11.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.544+0000 7f62f0fd4700 1 -- 172.21.15.43:0/1763519138 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ec06c510 con 0x7f62ec071970 2024-09-13T17:50:11.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.548+0000 7f62d3fff700 1 -- 172.21.15.43:0/1763519138 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f62e0021060 con 0x7f62ec071970 2024-09-13T17:50:11.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.632+0000 7f62f0fd4700 1 -- 172.21.15.43:0/1763519138 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd new", "uuid": "df6ebccc-cd93-4154-9559-d2724c12fcae", "id": 3} v 0) v1 -- 0x7f62ec0c7740 con 0x7f62ec071970 2024-09-13T17:50:11.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.640+0000 7f62d3fff700 1 -- 172.21.15.43:0/1763519138 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "df6ebccc-cd93-4154-9559-d2724c12fcae", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f62e0017070 con 0x7f62ec071970 2024-09-13T17:50:11.646 INFO:teuthology.orchestra.run.smithi043.stdout:3 2024-09-13T17:50:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.644+0000 7f62f0fd4700 1 -- 172.21.15.43:0/1763519138 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f62ec071970 msgr2=0x7f62ec143900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.644+0000 7f62f0fd4700 1 --2- 172.21.15.43:0/1763519138 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f62ec071970 0x7f62ec143900 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f62e000e8e0 tx=0x7f62e000ebf0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.644+0000 7f62f0fd4700 1 -- 172.21.15.43:0/1763519138 shutdown_connections 2024-09-13T17:50:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.644+0000 7f62f0fd4700 1 --2- 172.21.15.43:0/1763519138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62ec1503f0 0x7f62ec0ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.644+0000 7f62f0fd4700 1 --2- 172.21.15.43:0/1763519138 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f62ec071970 0x7f62ec143900 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.644+0000 7f62f0fd4700 1 --2- 172.21.15.43:0/1763519138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f62ec070a70 0x7f62ec153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.644+0000 7f62f0fd4700 1 -- 172.21.15.43:0/1763519138 >> 172.21.15.43:0/1763519138 conn(0x7f62ec0cc7f0 msgr2=0x7f62ec06a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.644+0000 7f62f0fd4700 1 -- 172.21.15.43:0/1763519138 shutdown_connections 2024-09-13T17:50:11.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.644+0000 7f62f0fd4700 1 -- 172.21.15.43:0/1763519138 wait complete. 2024-09-13T17:50:11.660 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new 7780b8ba-bfbc-4d79-a70a-36373693a8de 4 2024-09-13T17:50:11.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.764+0000 7ff822ea0700 1 Processor -- start 2024-09-13T17:50:11.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.764+0000 7ff822ea0700 1 -- start start 2024-09-13T17:50:11.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.764+0000 7ff822ea0700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff81c15a260 0x7ff81c0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.764+0000 7ff822ea0700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c0cbdc0 con 0x7ff81c0cf580 2024-09-13T17:50:11.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.764+0000 7ff822ea0700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c153f80 con 0x7ff81c0cf1a0 2024-09-13T17:50:11.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.764+0000 7ff822ea0700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c0caed0 con 0x7ff81c15a260 2024-09-13T17:50:11.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.768+0000 7ff821e9e700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff81c0cf1a0 0x7ff81c0c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36244/0 (socket says 172.21.15.43:36244) 2024-09-13T17:50:11.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.768+0000 7ff821e9e700 1 -- 172.21.15.43:0/1536252361 learned_addr learned my addr 172.21.15.43:0/1536252361 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-09-13T17:50:11.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.768+0000 7ff82269f700 1 --2- 172.21.15.43:0/1536252361 >> v2:172.21.15.40:3300/0 conn(0x7ff81c15a260 0x7ff81c0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.768+0000 7ff820e9c700 1 -- 172.21.15.43:0/1536252361 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3562159210 0 0) 0x7ff81c153f80 con 0x7ff81c0cf1a0 2024-09-13T17:50:11.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.768+0000 7ff820e9c700 1 -- 172.21.15.43:0/1536252361 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8080035d0 con 0x7ff81c0cf1a0 2024-09-13T17:50:11.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.768+0000 7ff82269f700 1 -- 172.21.15.43:0/1536252361 >> v1:172.21.15.43:6789/0 conn(0x7ff81c0cf1a0 legacy=0x7ff81c0c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.768+0000 7ff82269f700 1 -- 172.21.15.43:0/1536252361 >> v1:172.21.15.40:6789/0 conn(0x7ff81c0cf580 legacy=0x7ff81c0cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.768+0000 7ff82269f700 1 -- 172.21.15.43:0/1536252361 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c0cb1b0 con 0x7ff81c15a260 2024-09-13T17:50:11.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.768+0000 7ff82269f700 1 --2- 172.21.15.43:0/1536252361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff81c15a260 0x7ff81c0c8aa0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff814009d90 tx=0x7ff81400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339c3aaa5d2eed06 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:11.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.768+0000 7ff820e9c700 1 -- 172.21.15.43:0/1536252361 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400e070 con 0x7ff81c15a260 2024-09-13T17:50:11.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.768+0000 7ff820e9c700 1 -- 172.21.15.43:0/1536252361 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81400bb20 con 0x7ff81c15a260 2024-09-13T17:50:11.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.768+0000 7ff820e9c700 1 -- 172.21.15.43:0/1536252361 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814012430 con 0x7ff81c15a260 2024-09-13T17:50:11.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.768+0000 7ff822ea0700 1 -- 172.21.15.43:0/1536252361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff81c15a260 msgr2=0x7ff81c0c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.768+0000 7ff822ea0700 1 --2- 172.21.15.43:0/1536252361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff81c15a260 0x7ff81c0c8aa0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff814009d90 tx=0x7ff81400b330 comp rx=0 tx=0).stop 2024-09-13T17:50:11.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.768+0000 7ff822ea0700 1 -- 172.21.15.43:0/1536252361 shutdown_connections 2024-09-13T17:50:11.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.768+0000 7ff822ea0700 1 --2- 172.21.15.43:0/1536252361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff81c15a260 0x7ff81c0c8aa0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.768+0000 7ff822ea0700 1 -- 172.21.15.43:0/1536252361 >> 172.21.15.43:0/1536252361 conn(0x7ff81c066090 msgr2=0x7ff81c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:11.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.768+0000 7ff822ea0700 1 -- 172.21.15.43:0/1536252361 shutdown_connections 2024-09-13T17:50:11.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.768+0000 7ff822ea0700 1 -- 172.21.15.43:0/1536252361 wait complete. 2024-09-13T17:50:11.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.772+0000 7ff822ea0700 1 Processor -- start 2024-09-13T17:50:11.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.772+0000 7ff822ea0700 1 -- start start 2024-09-13T17:50:11.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.772+0000 7ff822ea0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff81c0cf1a0 0x7ff81c159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.772+0000 7ff822ea0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c0cf580 0x7ff81c158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.772+0000 7ff822ea0700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff81c15a260 0x7ff81c14e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.772+0000 7ff822ea0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c14fd70 con 0x7ff81c0cf1a0 2024-09-13T17:50:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.772+0000 7ff822ea0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c1500b0 con 0x7ff81c0cf580 2024-09-13T17:50:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.772+0000 7ff822ea0700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c159f40 con 0x7ff81c15a260 2024-09-13T17:50:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.772+0000 7ff82169d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c0cf580 0x7ff81c158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.772+0000 7ff82269f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff81c15a260 0x7ff81c14e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.776+0000 7ff82169d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c0cf580 0x7ff81c158c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38152/0 (socket says 172.21.15.43:38152) 2024-09-13T17:50:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.776+0000 7ff82169d700 1 -- 172.21.15.43:0/4116829762 learned_addr learned my addr 172.21.15.43:0/4116829762 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T17:50:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.776+0000 7ff821e9e700 1 --2- 172.21.15.43:0/4116829762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff81c0cf1a0 0x7ff81c159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.776+0000 7ff82169d700 1 -- 172.21.15.43:0/4116829762 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff81c15a260 msgr2=0x7ff81c14e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.776+0000 7ff82169d700 1 --2- 172.21.15.43:0/4116829762 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff81c15a260 0x7ff81c14e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.776+0000 7ff82169d700 1 -- 172.21.15.43:0/4116829762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff81c0cf1a0 msgr2=0x7ff81c159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.776+0000 7ff82169d700 1 --2- 172.21.15.43:0/4116829762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff81c0cf1a0 0x7ff81c159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.776+0000 7ff82169d700 1 -- 172.21.15.43:0/4116829762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff814009a70 con 0x7ff81c0cf580 2024-09-13T17:50:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.776+0000 7ff82169d700 1 --2- 172.21.15.43:0/4116829762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c0cf580 0x7ff81c158c10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff80c00c4d0 tx=0x7ff80c00c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.776+0000 7ff8127fc700 1 -- 172.21.15.43:0/4116829762 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c0052d0 con 0x7ff81c0cf580 2024-09-13T17:50:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.776+0000 7ff8127fc700 1 -- 172.21.15.43:0/4116829762 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80c005430 con 0x7ff81c0cf580 2024-09-13T17:50:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.776+0000 7ff822ea0700 1 -- 172.21.15.43:0/4116829762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81c14eea0 con 0x7ff81c0cf580 2024-09-13T17:50:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.776+0000 7ff8127fc700 1 -- 172.21.15.43:0/4116829762 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c0055a0 con 0x7ff81c0cf580 2024-09-13T17:50:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.776+0000 7ff822ea0700 1 -- 172.21.15.43:0/4116829762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81c14f3d0 con 0x7ff81c0cf580 2024-09-13T17:50:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.776+0000 7ff8127fc700 1 -- 172.21.15.43:0/4116829762 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff80c003bc0 con 0x7ff81c0cf580 2024-09-13T17:50:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.776+0000 7ff8127fc700 1 -- 172.21.15.43:0/4116829762 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7ff80c011d60 con 0x7ff81c0cf580 2024-09-13T17:50:11.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.776+0000 7ff822ea0700 1 -- 172.21.15.43:0/4116829762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81c067a40 con 0x7ff81c0cf580 2024-09-13T17:50:11.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.780+0000 7ff8127fc700 1 -- 172.21.15.43:0/4116829762 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7ff80c017070 con 0x7ff81c0cf580 2024-09-13T17:50:11.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.852+0000 7ff822ea0700 1 -- 172.21.15.43:0/4116829762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7780b8ba-bfbc-4d79-a70a-36373693a8de", "id": 4} v 0) v1 -- 0x7ff81c15ac50 con 0x7ff81c0cf580 2024-09-13T17:50:11.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.864+0000 7ff8127fc700 1 -- 172.21.15.43:0/4116829762 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7780b8ba-bfbc-4d79-a70a-36373693a8de", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff80c011590 con 0x7ff81c0cf580 2024-09-13T17:50:11.868 INFO:teuthology.orchestra.run.smithi043.stdout:4 2024-09-13T17:50:11.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.864+0000 7ff822ea0700 1 -- 172.21.15.43:0/4116829762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c0cf580 msgr2=0x7ff81c158c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.864+0000 7ff822ea0700 1 --2- 172.21.15.43:0/4116829762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c0cf580 0x7ff81c158c10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff80c00c4d0 tx=0x7ff80c00c7e0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.864+0000 7ff822ea0700 1 -- 172.21.15.43:0/4116829762 shutdown_connections 2024-09-13T17:50:11.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.864+0000 7ff822ea0700 1 --2- 172.21.15.43:0/4116829762 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff81c15a260 0x7ff81c14e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.864+0000 7ff822ea0700 1 --2- 172.21.15.43:0/4116829762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c0cf580 0x7ff81c158c10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.864+0000 7ff822ea0700 1 --2- 172.21.15.43:0/4116829762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff81c0cf1a0 0x7ff81c159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.864+0000 7ff822ea0700 1 -- 172.21.15.43:0/4116829762 >> 172.21.15.43:0/4116829762 conn(0x7ff81c066090 msgr2=0x7ff81c0ce230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:11.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.864+0000 7ff822ea0700 1 -- 172.21.15.43:0/4116829762 shutdown_connections 2024-09-13T17:50:11.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.864+0000 7ff822ea0700 1 -- 172.21.15.43:0/4116829762 wait complete. 2024-09-13T17:50:11.881 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:11.877+0000 7f0ad16a92c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:50:11.882 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e 5 2024-09-13T17:50:11.883 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:11.876+0000 7fc023e452c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:50:11.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.984+0000 7f131d995700 1 Processor -- start 2024-09-13T17:50:11.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.984+0000 7f131d995700 1 -- start start 2024-09-13T17:50:11.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f131d995700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f13181504f0 0x7f13181599d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f131d995700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13180703b0 con 0x7f13180b0fa0 2024-09-13T17:50:11.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f131d995700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1318072960 con 0x7f13181508d0 2024-09-13T17:50:11.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f131d995700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13180c0cf0 con 0x7f13181504f0 2024-09-13T17:50:11.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f130ffff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f13181508d0 0x7f13180c6970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36254/0 (socket says 172.21.15.43:36254) 2024-09-13T17:50:11.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f130ffff700 1 -- 172.21.15.43:0/3601064396 learned_addr learned my addr 172.21.15.43:0/3601064396 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-09-13T17:50:11.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f1316ffd700 1 --2- 172.21.15.43:0/3601064396 >> v2:172.21.15.40:3300/0 conn(0x7f13181504f0 0x7f13181599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:11.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f13167fc700 1 -- 172.21.15.43:0/3601064396 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800341793 0 0) 0x7f1318072960 con 0x7f13181508d0 2024-09-13T17:50:11.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f13167fc700 1 -- 172.21.15.43:0/3601064396 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13040035d0 con 0x7f13181508d0 2024-09-13T17:50:11.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f13167fc700 1 -- 172.21.15.43:0/3601064396 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3203840264 0 0) 0x7f13180703b0 con 0x7f13180b0fa0 2024-09-13T17:50:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f1316ffd700 1 -- 172.21.15.43:0/3601064396 >> v1:172.21.15.43:6789/0 conn(0x7f13181508d0 legacy=0x7f13180c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f1316ffd700 1 -- 172.21.15.43:0/3601064396 >> v1:172.21.15.40:6789/0 conn(0x7f13180b0fa0 legacy=0x7f1318160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f1316ffd700 1 -- 172.21.15.43:0/3601064396 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13180c0730 con 0x7f13181504f0 2024-09-13T17:50:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f1316ffd700 1 --2- 172.21.15.43:0/3601064396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13181504f0 0x7f13181599d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f13080097d0 tx=0x7f1308009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5738d5f74c2799a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f13167fc700 1 -- 172.21.15.43:0/3601064396 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13000015f0 con 0x7f13181504f0 2024-09-13T17:50:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f13167fc700 1 -- 172.21.15.43:0/3601064396 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1308010590 con 0x7f13181504f0 2024-09-13T17:50:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f13167fc700 1 -- 172.21.15.43:0/3601064396 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308010940 con 0x7f13181504f0 2024-09-13T17:50:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f131d995700 1 -- 172.21.15.43:0/3601064396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13181504f0 msgr2=0x7f13181599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f131d995700 1 --2- 172.21.15.43:0/3601064396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13181504f0 0x7f13181599d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f13080097d0 tx=0x7f1308009ae0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f131d995700 1 -- 172.21.15.43:0/3601064396 shutdown_connections 2024-09-13T17:50:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f131d995700 1 --2- 172.21.15.43:0/3601064396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13181504f0 0x7f13181599d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f131d995700 1 -- 172.21.15.43:0/3601064396 >> 172.21.15.43:0/3601064396 conn(0x7f131806d5b0 msgr2=0x7f131806d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:11.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f131d995700 1 -- 172.21.15.43:0/3601064396 shutdown_connections 2024-09-13T17:50:11.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.988+0000 7f131d995700 1 -- 172.21.15.43:0/3601064396 wait complete. 2024-09-13T17:50:11.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.992+0000 7f131d995700 1 Processor -- start 2024-09-13T17:50:11.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.992+0000 7f131d995700 1 -- start start 2024-09-13T17:50:11.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.992+0000 7f131d995700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f13180b0fa0 0x7f13180c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:11.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.992+0000 7f131d995700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13181504f0 0x7f13181ece60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.992+0000 7f131d995700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13181508d0 0x7f13181ed3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.992+0000 7f131d995700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1318159b30 con 0x7f13181508d0 2024-09-13T17:50:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.992+0000 7f131d995700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131815a850 con 0x7f13181504f0 2024-09-13T17:50:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.992+0000 7f131d995700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131814aae0 con 0x7f13180b0fa0 2024-09-13T17:50:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.992+0000 7f130ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13181504f0 0x7f13181ece60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.992+0000 7f1316ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f13180b0fa0 0x7f13180c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.992+0000 7f130ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13181504f0 0x7f13181ece60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38154/0 (socket says 172.21.15.43:38154) 2024-09-13T17:50:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.992+0000 7f130ffff700 1 -- 172.21.15.43:0/1922868601 learned_addr learned my addr 172.21.15.43:0/1922868601 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T17:50:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.992+0000 7f13177fe700 1 --2- 172.21.15.43:0/1922868601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13181508d0 0x7f13181ed3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.992+0000 7f130ffff700 1 -- 172.21.15.43:0/1922868601 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f13180b0fa0 msgr2=0x7f13180c46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.992+0000 7f130ffff700 1 --2- 172.21.15.43:0/1922868601 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f13180b0fa0 0x7f13180c46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.992+0000 7f130ffff700 1 -- 172.21.15.43:0/1922868601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13181508d0 msgr2=0x7f13181ed3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.992+0000 7f130ffff700 1 --2- 172.21.15.43:0/1922868601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13181508d0 0x7f13181ed3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.992+0000 7f130ffff700 1 -- 172.21.15.43:0/1922868601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13080094b0 con 0x7f13181504f0 2024-09-13T17:50:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.992+0000 7f130ffff700 1 --2- 172.21.15.43:0/1922868601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13181504f0 0x7f13181ece60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f130000c770 tx=0x7f130000ca80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.996+0000 7f1314ff9700 1 -- 172.21.15.43:0/1922868601 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300007560 con 0x7f13181504f0 2024-09-13T17:50:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.996+0000 7f1314ff9700 1 -- 172.21.15.43:0/1922868601 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13000076c0 con 0x7f13181504f0 2024-09-13T17:50:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.996+0000 7f131d995700 1 -- 172.21.15.43:0/1922868601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131814a460 con 0x7f13181504f0 2024-09-13T17:50:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.996+0000 7f1314ff9700 1 -- 172.21.15.43:0/1922868601 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300004ab0 con 0x7f13181504f0 2024-09-13T17:50:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.996+0000 7f131d995700 1 -- 172.21.15.43:0/1922868601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f131814ad80 con 0x7f13181504f0 2024-09-13T17:50:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.996+0000 7f1314ff9700 1 -- 172.21.15.43:0/1922868601 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f130001d3f0 con 0x7f13181504f0 2024-09-13T17:50:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.996+0000 7f1314ff9700 1 -- 172.21.15.43:0/1922868601 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f1300013940 con 0x7f13181504f0 2024-09-13T17:50:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:11.996+0000 7f131d995700 1 -- 172.21.15.43:0/1922868601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1318070820 con 0x7f13181504f0 2024-09-13T17:50:12.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.000+0000 7f1314ff9700 1 -- 172.21.15.43:0/1922868601 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f1318070820 con 0x7f13181504f0 2024-09-13T17:50:12.011 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:12.004+0000 7fc023e452c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:50:12.012 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:12.009+0000 7f0ad16a92c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:50:12.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.068+0000 7f131d995700 1 -- 172.21.15.43:0/1922868601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e", "id": 5} v 0) v1 -- 0x7f1318153e00 con 0x7f13181504f0 2024-09-13T17:50:12.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.080+0000 7f1314ff9700 1 -- 172.21.15.43:0/1922868601 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1300013dc0 con 0x7f13181504f0 2024-09-13T17:50:12.085 INFO:teuthology.orchestra.run.smithi043.stdout:5 2024-09-13T17:50:12.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.080+0000 7f131d995700 1 -- 172.21.15.43:0/1922868601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13181504f0 msgr2=0x7f13181ece60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:12.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.080+0000 7f131d995700 1 --2- 172.21.15.43:0/1922868601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13181504f0 0x7f13181ece60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f130000c770 tx=0x7f130000ca80 comp rx=0 tx=0).stop 2024-09-13T17:50:12.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.080+0000 7f131d995700 1 -- 172.21.15.43:0/1922868601 shutdown_connections 2024-09-13T17:50:12.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.080+0000 7f131d995700 1 --2- 172.21.15.43:0/1922868601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f13181508d0 0x7f13181ed3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.080+0000 7f131d995700 1 --2- 172.21.15.43:0/1922868601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13181504f0 0x7f13181ece60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.080+0000 7f131d995700 1 --2- 172.21.15.43:0/1922868601 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f13180b0fa0 0x7f13180c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.080+0000 7f131d995700 1 -- 172.21.15.43:0/1922868601 >> 172.21.15.43:0/1922868601 conn(0x7f131806d5b0 msgr2=0x7f131806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:12.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.080+0000 7f131d995700 1 -- 172.21.15.43:0/1922868601 shutdown_connections 2024-09-13T17:50:12.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.080+0000 7f131d995700 1 -- 172.21.15.43:0/1922868601 wait complete. 2024-09-13T17:50:12.099 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new a1c0a15b-f29b-477c-a0fa-63832c91da48 6 2024-09-13T17:50:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f291a1e9700 1 Processor -- start 2024-09-13T17:50:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f291a1e9700 1 -- start start 2024-09-13T17:50:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f291a1e9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29141504a0 0x7f29140ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f291a1e9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f29140b1c60 0x7f29141561f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f291a1e9700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29140ca5f0 con 0x7f2914156730 2024-09-13T17:50:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f291a1e9700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29140c2970 con 0x7f29140b1c60 2024-09-13T17:50:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f291a1e9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29140cab00 con 0x7f29141504a0 2024-09-13T17:50:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f29191e7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29141504a0 0x7f29140ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f29199e8700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2914156730 0x7f29140c8630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36256/0 (socket says 172.21.15.43:36256) 2024-09-13T17:50:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f29199e8700 1 -- 172.21.15.43:0/932442865 learned_addr learned my addr 172.21.15.43:0/932442865 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-09-13T17:50:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f29189e6700 1 --2- 172.21.15.43:0/932442865 >> v2:172.21.15.40:3300/0 conn(0x7f29140b1c60 0x7f29141561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f290bfff700 1 -- 172.21.15.43:0/932442865 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3972192791 0 0) 0x7f29140ca5f0 con 0x7f2914156730 2024-09-13T17:50:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f29191e7700 1 -- 172.21.15.43:0/932442865 >> v2:172.21.15.40:3300/0 conn(0x7f29140b1c60 msgr2=0x7f29141561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f29191e7700 1 --2- 172.21.15.43:0/932442865 >> v2:172.21.15.40:3300/0 conn(0x7f29140b1c60 0x7f29141561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f29191e7700 1 -- 172.21.15.43:0/932442865 >> v1:172.21.15.43:6789/0 conn(0x7f2914156730 legacy=0x7f29140c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:12.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f29191e7700 1 -- 172.21.15.43:0/932442865 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29140caea0 con 0x7f29141504a0 2024-09-13T17:50:12.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f29191e7700 1 --2- 172.21.15.43:0/932442865 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29141504a0 0x7f29140ca0b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2910009880 tx=0x7f2910009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3d9918cf2629324 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:12.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f290bfff700 1 -- 172.21.15.43:0/932442865 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910011830 con 0x7f29141504a0 2024-09-13T17:50:12.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f290bfff700 1 -- 172.21.15.43:0/932442865 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2910011990 con 0x7f29141504a0 2024-09-13T17:50:12.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f290bfff700 1 -- 172.21.15.43:0/932442865 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910011d20 con 0x7f29141504a0 2024-09-13T17:50:12.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f291a1e9700 1 -- 172.21.15.43:0/932442865 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29141504a0 msgr2=0x7f29140ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:12.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f291a1e9700 1 --2- 172.21.15.43:0/932442865 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29141504a0 0x7f29140ca0b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2910009880 tx=0x7f2910009b90 comp rx=0 tx=0).stop 2024-09-13T17:50:12.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f291a1e9700 1 -- 172.21.15.43:0/932442865 shutdown_connections 2024-09-13T17:50:12.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f291a1e9700 1 --2- 172.21.15.43:0/932442865 >> v2:172.21.15.40:3300/0 conn(0x7f29140b1c60 0x7f29141561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f291a1e9700 1 --2- 172.21.15.43:0/932442865 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29141504a0 0x7f29140ca0b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f291a1e9700 1 -- 172.21.15.43:0/932442865 >> 172.21.15.43:0/932442865 conn(0x7f291406e710 msgr2=0x7f291406eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:12.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f291a1e9700 1 -- 172.21.15.43:0/932442865 shutdown_connections 2024-09-13T17:50:12.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.208+0000 7f291a1e9700 1 -- 172.21.15.43:0/932442865 wait complete. 2024-09-13T17:50:12.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.212+0000 7f291a1e9700 1 Processor -- start 2024-09-13T17:50:12.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.212+0000 7f291a1e9700 1 -- start start 2024-09-13T17:50:12.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.212+0000 7f291a1e9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29140b1c60 0x7f291414ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:12.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.212+0000 7f291a1e9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29141504a0 0x7f29141edd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:12.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.212+0000 7f291a1e9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2914156730 0x7f2914152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:12.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.212+0000 7f291a1e9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29141ed8d0 con 0x7f2914156730 2024-09-13T17:50:12.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.212+0000 7f291a1e9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2914147250 con 0x7f29140b1c60 2024-09-13T17:50:12.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.212+0000 7f291a1e9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29141474f0 con 0x7f29141504a0 2024-09-13T17:50:12.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.212+0000 7f29191e7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29140b1c60 0x7f291414ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:12.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.212+0000 7f29189e6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29141504a0 0x7f29141edd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:12.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.212+0000 7f29191e7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29140b1c60 0x7f291414ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38162/0 (socket says 172.21.15.43:38162) 2024-09-13T17:50:12.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.212+0000 7f29191e7700 1 -- 172.21.15.43:0/3805334434 learned_addr learned my addr 172.21.15.43:0/3805334434 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T17:50:12.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.212+0000 7f29199e8700 1 --2- 172.21.15.43:0/3805334434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2914156730 0x7f2914152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:12.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.212+0000 7f29191e7700 1 -- 172.21.15.43:0/3805334434 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29141504a0 msgr2=0x7f29141edd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:12.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.212+0000 7f29191e7700 1 --2- 172.21.15.43:0/3805334434 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29141504a0 0x7f29141edd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.212+0000 7f29191e7700 1 -- 172.21.15.43:0/3805334434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2914156730 msgr2=0x7f2914152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:12.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.212+0000 7f29191e7700 1 --2- 172.21.15.43:0/3805334434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2914156730 0x7f2914152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.212+0000 7f29191e7700 1 -- 172.21.15.43:0/3805334434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29100094b0 con 0x7f29140b1c60 2024-09-13T17:50:12.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.212+0000 7f29189e6700 1 --2- 172.21.15.43:0/3805334434 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29141504a0 0x7f29141edd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:50:12.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.216+0000 7f29191e7700 1 --2- 172.21.15.43:0/3805334434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29140b1c60 0x7f291414ba30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f29100090b0 tx=0x7f2910004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:12.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.216+0000 7f2909ffb700 1 -- 172.21.15.43:0/3805334434 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910006ea0 con 0x7f29140b1c60 2024-09-13T17:50:12.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.216+0000 7f2909ffb700 1 -- 172.21.15.43:0/3805334434 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291001f070 con 0x7f29140b1c60 2024-09-13T17:50:12.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.216+0000 7f2909ffb700 1 -- 172.21.15.43:0/3805334434 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291001ac50 con 0x7f29140b1c60 2024-09-13T17:50:12.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.216+0000 7f291a1e9700 1 -- 172.21.15.43:0/3805334434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29141ed2a0 con 0x7f29140b1c60 2024-09-13T17:50:12.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.216+0000 7f291a1e9700 1 -- 172.21.15.43:0/3805334434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29141f15a0 con 0x7f29140b1c60 2024-09-13T17:50:12.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.216+0000 7f2909ffb700 1 -- 172.21.15.43:0/3805334434 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f291001b730 con 0x7f29140b1c60 2024-09-13T17:50:12.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.216+0000 7f2909ffb700 1 -- 172.21.15.43:0/3805334434 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f2910025c10 con 0x7f29140b1c60 2024-09-13T17:50:12.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.216+0000 7f291a1e9700 1 -- 172.21.15.43:0/3805334434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29141f1760 con 0x7f29140b1c60 2024-09-13T17:50:12.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.220+0000 7f2909ffb700 1 -- 172.21.15.43:0/3805334434 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f29141f1760 con 0x7f29140b1c60 2024-09-13T17:50:12.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.292+0000 7f291a1e9700 1 -- 172.21.15.43:0/3805334434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a1c0a15b-f29b-477c-a0fa-63832c91da48", "id": 6} v 0) v1 -- 0x7f291414e970 con 0x7f29140b1c60 2024-09-13T17:50:12.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.300+0000 7f2909ffb700 1 -- 172.21.15.43:0/3805334434 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a1c0a15b-f29b-477c-a0fa-63832c91da48", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2910033060 con 0x7f29140b1c60 2024-09-13T17:50:12.306 INFO:teuthology.orchestra.run.smithi043.stdout:6 2024-09-13T17:50:12.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.304+0000 7f291a1e9700 1 -- 172.21.15.43:0/3805334434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29140b1c60 msgr2=0x7f291414ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.304+0000 7f291a1e9700 1 --2- 172.21.15.43:0/3805334434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29140b1c60 0x7f291414ba30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f29100090b0 tx=0x7f2910004770 comp rx=0 tx=0).stop 2024-09-13T17:50:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.304+0000 7f291a1e9700 1 -- 172.21.15.43:0/3805334434 shutdown_connections 2024-09-13T17:50:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.304+0000 7f291a1e9700 1 --2- 172.21.15.43:0/3805334434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2914156730 0x7f2914152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.304+0000 7f291a1e9700 1 --2- 172.21.15.43:0/3805334434 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29141504a0 0x7f29141edd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.304+0000 7f291a1e9700 1 --2- 172.21.15.43:0/3805334434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29140b1c60 0x7f291414ba30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.304+0000 7f291a1e9700 1 -- 172.21.15.43:0/3805334434 >> 172.21.15.43:0/3805334434 conn(0x7f291406e710 msgr2=0x7f291414dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.304+0000 7f291a1e9700 1 -- 172.21.15.43:0/3805334434 shutdown_connections 2024-09-13T17:50:12.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.304+0000 7f291a1e9700 1 -- 172.21.15.43:0/3805334434 wait complete. 2024-09-13T17:50:12.326 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd new 708c7296-d725-48c1-b96e-ecf8975d02fd 7 2024-09-13T17:50:12.373 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:12.368+0000 7fc023e452c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:50:12.391 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:12.385+0000 7f0ad16a92c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:50:12.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.428+0000 7fb5b189a700 1 Processor -- start 2024-09-13T17:50:12.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.428+0000 7fb5b189a700 1 -- start start 2024-09-13T17:50:12.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5b189a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb5ac153720 0x7fb5ac0ca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:12.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5b189a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5ac161d40 0x7fb5ac164130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:12.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5b189a700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5ac159e50 con 0x7fb5ac0cf9b0 2024-09-13T17:50:12.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5b189a700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5ac0c5ea0 con 0x7fb5ac153720 2024-09-13T17:50:12.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5b189a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac15d390 con 0x7fb5ac161d40 2024-09-13T17:50:12.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5abfff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb5ac0cf9b0 0x7fb5ac06c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36266/0 (socket says 172.21.15.43:36266) 2024-09-13T17:50:12.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5abfff700 1 -- 172.21.15.43:0/3281048862 learned_addr learned my addr 172.21.15.43:0/3281048862 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-09-13T17:50:12.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5b0898700 1 --2- 172.21.15.43:0/3281048862 >> v2:172.21.15.43:3300/0 conn(0x7fb5ac153720 0x7fb5ac0ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:12.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5b1099700 1 --2- 172.21.15.43:0/3281048862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5ac161d40 0x7fb5ac164130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5ab7fe700 1 -- 172.21.15.43:0/3281048862 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1543676704 0 0) 0x7fb5ac159e50 con 0x7fb5ac0cf9b0 2024-09-13T17:50:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5ab7fe700 1 -- 172.21.15.43:0/3281048862 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb59c0035d0 con 0x7fb5ac0cf9b0 2024-09-13T17:50:12.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5b0898700 1 -- 172.21.15.43:0/3281048862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5ac161d40 msgr2=0x7fb5ac164130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:12.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5b0898700 1 --2- 172.21.15.43:0/3281048862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5ac161d40 0x7fb5ac164130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5b0898700 1 -- 172.21.15.43:0/3281048862 >> v1:172.21.15.43:6789/0 conn(0x7fb5ac0cf9b0 legacy=0x7fb5ac06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:12.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5b0898700 1 -- 172.21.15.43:0/3281048862 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ac15a080 con 0x7fb5ac153720 2024-09-13T17:50:12.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5b0898700 1 --2- 172.21.15.43:0/3281048862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5ac153720 0x7fb5ac0ca510 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0009850 tx=0x7fb5a0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e896d7546bce3ca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:12.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5ab7fe700 1 -- 172.21.15.43:0/3281048862 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0011830 con 0x7fb5ac153720 2024-09-13T17:50:12.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5ab7fe700 1 -- 172.21.15.43:0/3281048862 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a0011990 con 0x7fb5ac153720 2024-09-13T17:50:12.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5ab7fe700 1 -- 172.21.15.43:0/3281048862 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0011d00 con 0x7fb5ac153720 2024-09-13T17:50:12.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5b189a700 1 -- 172.21.15.43:0/3281048862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5ac153720 msgr2=0x7fb5ac0ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:12.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5b189a700 1 --2- 172.21.15.43:0/3281048862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5ac153720 0x7fb5ac0ca510 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0009850 tx=0x7fb5a0009b60 comp rx=0 tx=0).stop 2024-09-13T17:50:12.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5b189a700 1 -- 172.21.15.43:0/3281048862 shutdown_connections 2024-09-13T17:50:12.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5b189a700 1 --2- 172.21.15.43:0/3281048862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5ac161d40 0x7fb5ac164130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5b189a700 1 --2- 172.21.15.43:0/3281048862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5ac153720 0x7fb5ac0ca510 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5b189a700 1 -- 172.21.15.43:0/3281048862 >> 172.21.15.43:0/3281048862 conn(0x7fb5ac0666c0 msgr2=0x7fb5ac0cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:12.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5b189a700 1 -- 172.21.15.43:0/3281048862 shutdown_connections 2024-09-13T17:50:12.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.432+0000 7fb5b189a700 1 -- 172.21.15.43:0/3281048862 wait complete. 2024-09-13T17:50:12.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5b189a700 1 Processor -- start 2024-09-13T17:50:12.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5b189a700 1 -- start start 2024-09-13T17:50:12.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5b189a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5ac0cf9b0 0x7fb5ac1eff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:12.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5b189a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5ac153720 0x7fb5ac158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:12.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5b189a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5ac161d40 0x7fb5ac0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:12.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5b189a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac0cd550 con 0x7fb5ac0cf9b0 2024-09-13T17:50:12.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5b189a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac0ce830 con 0x7fb5ac153720 2024-09-13T17:50:12.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5b189a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac0ccf00 con 0x7fb5ac161d40 2024-09-13T17:50:12.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5abfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5ac153720 0x7fb5ac158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:12.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5abfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5ac153720 0x7fb5ac158420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38182/0 (socket says 172.21.15.43:38182) 2024-09-13T17:50:12.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5abfff700 1 -- 172.21.15.43:0/375975608 learned_addr learned my addr 172.21.15.43:0/375975608 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-13T17:50:12.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5b1099700 1 --2- 172.21.15.43:0/375975608 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5ac161d40 0x7fb5ac0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:12.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5b0898700 1 --2- 172.21.15.43:0/375975608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5ac0cf9b0 0x7fb5ac1eff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:12.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5abfff700 1 -- 172.21.15.43:0/375975608 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5ac161d40 msgr2=0x7fb5ac0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:12.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5abfff700 1 --2- 172.21.15.43:0/375975608 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5ac161d40 0x7fb5ac0cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5abfff700 1 -- 172.21.15.43:0/375975608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5ac0cf9b0 msgr2=0x7fb5ac1eff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:12.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5abfff700 1 --2- 172.21.15.43:0/375975608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5ac0cf9b0 0x7fb5ac1eff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5abfff700 1 -- 172.21.15.43:0/375975608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a00094b0 con 0x7fb5ac153720 2024-09-13T17:50:12.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5abfff700 1 --2- 172.21.15.43:0/375975608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5ac153720 0x7fb5ac158420 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb59800c4f0 tx=0x7fb59800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:12.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5a97fa700 1 -- 172.21.15.43:0/375975608 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb598005d90 con 0x7fb5ac153720 2024-09-13T17:50:12.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5a97fa700 1 -- 172.21.15.43:0/375975608 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5980070d0 con 0x7fb5ac153720 2024-09-13T17:50:12.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5b189a700 1 -- 172.21.15.43:0/375975608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ac0cd1e0 con 0x7fb5ac153720 2024-09-13T17:50:12.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5a97fa700 1 -- 172.21.15.43:0/375975608 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb598007440 con 0x7fb5ac153720 2024-09-13T17:50:12.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5b189a700 1 -- 172.21.15.43:0/375975608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ac0ce4e0 con 0x7fb5ac153720 2024-09-13T17:50:12.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5a97fa700 1 -- 172.21.15.43:0/375975608 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fb59801c440 con 0x7fb5ac153720 2024-09-13T17:50:12.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5a97fa700 1 -- 172.21.15.43:0/375975608 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fb59801cd50 con 0x7fb5ac153720 2024-09-13T17:50:12.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.436+0000 7fb5b189a700 1 -- 172.21.15.43:0/375975608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5ac067a40 con 0x7fb5ac153720 2024-09-13T17:50:12.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.440+0000 7fb5a97fa700 1 -- 172.21.15.43:0/375975608 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fb598021060 con 0x7fb5ac153720 2024-09-13T17:50:12.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.512+0000 7fb5b189a700 1 -- 172.21.15.43:0/375975608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd new", "uuid": "708c7296-d725-48c1-b96e-ecf8975d02fd", "id": 7} v 0) v1 -- 0x7fb5ac0b2180 con 0x7fb5ac153720 2024-09-13T17:50:12.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.524+0000 7fb5a97fa700 1 -- 172.21.15.43:0/375975608 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "708c7296-d725-48c1-b96e-ecf8975d02fd", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb598017070 con 0x7fb5ac153720 2024-09-13T17:50:12.528 INFO:teuthology.orchestra.run.smithi043.stdout:7 2024-09-13T17:50:12.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.524+0000 7fb5b189a700 1 -- 172.21.15.43:0/375975608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5ac153720 msgr2=0x7fb5ac158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:12.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.524+0000 7fb5b189a700 1 --2- 172.21.15.43:0/375975608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5ac153720 0x7fb5ac158420 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb59800c4f0 tx=0x7fb59800c800 comp rx=0 tx=0).stop 2024-09-13T17:50:12.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.524+0000 7fb5b189a700 1 -- 172.21.15.43:0/375975608 shutdown_connections 2024-09-13T17:50:12.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.524+0000 7fb5b189a700 1 --2- 172.21.15.43:0/375975608 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5ac161d40 0x7fb5ac0cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.524+0000 7fb5b189a700 1 --2- 172.21.15.43:0/375975608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5ac153720 0x7fb5ac158420 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.524+0000 7fb5b189a700 1 --2- 172.21.15.43:0/375975608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5ac0cf9b0 0x7fb5ac1eff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.524+0000 7fb5b189a700 1 -- 172.21.15.43:0/375975608 >> 172.21.15.43:0/375975608 conn(0x7fb5ac0666c0 msgr2=0x7fb5ac1507a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:12.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.524+0000 7fb5b189a700 1 -- 172.21.15.43:0/375975608 shutdown_connections 2024-09-13T17:50:12.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-13T17:50:12.524+0000 7fb5b189a700 1 -- 172.21.15.43:0/375975608 wait complete. 2024-09-13T17:50:12.540 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T17:50:12.540 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T17:50:12.543 INFO:tasks.ceph.osd.0:Started 2024-09-13T17:50:12.543 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T17:50:12.543 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T17:50:12.545 INFO:tasks.ceph.osd.1:Started 2024-09-13T17:50:12.545 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T17:50:12.545 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T17:50:12.547 INFO:tasks.ceph.osd.2:Started 2024-09-13T17:50:12.547 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T17:50:12.548 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T17:50:12.550 INFO:tasks.ceph.osd.3:Started 2024-09-13T17:50:12.550 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-13T17:50:12.550 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-13T17:50:12.552 INFO:tasks.ceph.osd.4:Started 2024-09-13T17:50:12.552 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-13T17:50:12.552 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-13T17:50:12.554 INFO:tasks.ceph.osd.5:Started 2024-09-13T17:50:12.555 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-13T17:50:12.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-13T17:50:12.557 INFO:tasks.ceph.osd.6:Started 2024-09-13T17:50:12.557 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-13T17:50:12.557 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-13T17:50:12.559 INFO:tasks.ceph.osd.7:Started 2024-09-13T17:50:12.560 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:50:12.583 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:12.576+0000 7fc023e452c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:50:12.613 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:12.609+0000 7f0ad16a92c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:50:12.668 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:12.664+0000 7fc023e452c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:50:12.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.681+0000 7f9b4ace9700 1 Processor -- start 2024-09-13T17:50:12.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.681+0000 7f9b4ace9700 1 -- start start 2024-09-13T17:50:12.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.681+0000 7f9b4ace9700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9b44150690 0x7f9b440c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:12.686 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:12.681+0000 7f0ad16a92c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:50:12.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.681+0000 7f9b4ace9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9b441567e0 0x7f9b44156c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:12.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.681+0000 7f9b4ace9700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b44159800 con 0x7f9b44070bc0 2024-09-13T17:50:12.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.681+0000 7f9b4ace9700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b440c8350 con 0x7f9b44150690 2024-09-13T17:50:12.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.681+0000 7f9b4ace9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440c6d20 con 0x7f9b441567e0 2024-09-13T17:50:12.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.681+0000 7f9b49ce7700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9b44150690 0x7f9b440c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:12.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.681+0000 7f9b4a4e8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9b441567e0 0x7f9b44156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:12.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.681+0000 7f9b494e6700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9b44070bc0 0x7f9b4406b4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:36926/0 (socket says 172.21.15.40:36926) 2024-09-13T17:50:12.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.681+0000 7f9b4a4e8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9b441567e0 0x7f9b44156c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:57316/0 (socket says 172.21.15.40:57316) 2024-09-13T17:50:12.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.681+0000 7f9b49ce7700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9b44150690 0x7f9b440c67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:52628/0 (socket says 172.21.15.40:52628) 2024-09-13T17:50:12.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.681+0000 7f9b494e6700 1 -- 172.21.15.40:0/2205328367 learned_addr learned my addr 172.21.15.40:0/2205328367 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:12.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.681+0000 7f9b49ce7700 1 -- 172.21.15.40:0/2205328367 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9b441567e0 msgr2=0x7f9b44156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:12.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.681+0000 7f9b49ce7700 1 --2- 172.21.15.40:0/2205328367 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9b441567e0 0x7f9b44156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.681+0000 7f9b49ce7700 1 -- 172.21.15.40:0/2205328367 >> v1:172.21.15.43:6789/0 conn(0x7f9b44070bc0 legacy=0x7f9b4406b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:12.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.681+0000 7f9b49ce7700 1 -- 172.21.15.40:0/2205328367 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b441571b0 con 0x7f9b44150690 2024-09-13T17:50:12.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.685+0000 7f9b49ce7700 1 --2- 172.21.15.40:0/2205328367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b44150690 0x7f9b440c67e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009850 tx=0x7f9b40009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd597cd246c13fe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:12.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.685+0000 7f9b48ce5700 1 -- 172.21.15.40:0/2205328367 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40004230 con 0x7f9b44150690 2024-09-13T17:50:12.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.685+0000 7f9b48ce5700 1 -- 172.21.15.40:0/2205328367 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b40004390 con 0x7f9b44150690 2024-09-13T17:50:12.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.685+0000 7f9b48ce5700 1 -- 172.21.15.40:0/2205328367 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b400036c0 con 0x7f9b44150690 2024-09-13T17:50:12.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.685+0000 7f9b4ace9700 1 -- 172.21.15.40:0/2205328367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b44150690 msgr2=0x7f9b440c67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:12.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.685+0000 7f9b4ace9700 1 --2- 172.21.15.40:0/2205328367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b44150690 0x7f9b440c67e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009850 tx=0x7f9b40009b60 comp rx=0 tx=0).stop 2024-09-13T17:50:12.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.685+0000 7f9b4ace9700 1 -- 172.21.15.40:0/2205328367 shutdown_connections 2024-09-13T17:50:12.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.685+0000 7f9b4ace9700 1 --2- 172.21.15.40:0/2205328367 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9b441567e0 0x7f9b44156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.685+0000 7f9b4ace9700 1 --2- 172.21.15.40:0/2205328367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b44150690 0x7f9b440c67e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.685+0000 7f9b4ace9700 1 -- 172.21.15.40:0/2205328367 >> 172.21.15.40:0/2205328367 conn(0x7f9b440c88e0 msgr2=0x7f9b440c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:12.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.685+0000 7f9b4ace9700 1 -- 172.21.15.40:0/2205328367 shutdown_connections 2024-09-13T17:50:12.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.685+0000 7f9b4ace9700 1 -- 172.21.15.40:0/2205328367 wait complete. 2024-09-13T17:50:12.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.685+0000 7f9b4ace9700 1 Processor -- start 2024-09-13T17:50:12.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.685+0000 7f9b4ace9700 1 -- start start 2024-09-13T17:50:12.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b4ace9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9b44070bc0 0x7f9b4406a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:12.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b4ace9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b44150690 0x7f9b441ed6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:12.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b4ace9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b441567e0 0x7f9b441edc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:12.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b4ace9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b441f1480 con 0x7f9b441567e0 2024-09-13T17:50:12.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b4ace9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b441f0900 con 0x7f9b44150690 2024-09-13T17:50:12.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b4ace9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b441f0be0 con 0x7f9b44070bc0 2024-09-13T17:50:12.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b494e6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b44150690 0x7f9b441ed6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:12.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b494e6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b44150690 0x7f9b441ed6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:52634/0 (socket says 172.21.15.40:52634) 2024-09-13T17:50:12.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b4a4e8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b441567e0 0x7f9b441edc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:12.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b494e6700 1 -- 172.21.15.40:0/2027030851 learned_addr learned my addr 172.21.15.40:0/2027030851 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:12.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b49ce7700 1 --2- 172.21.15.40:0/2027030851 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9b44070bc0 0x7f9b4406a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:12.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b494e6700 1 -- 172.21.15.40:0/2027030851 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9b44070bc0 msgr2=0x7f9b4406a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:12.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b494e6700 1 --2- 172.21.15.40:0/2027030851 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9b44070bc0 0x7f9b4406a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b494e6700 1 -- 172.21.15.40:0/2027030851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b441567e0 msgr2=0x7f9b441edc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:12.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b494e6700 1 --2- 172.21.15.40:0/2027030851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b441567e0 0x7f9b441edc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b494e6700 1 -- 172.21.15.40:0/2027030851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b400094b0 con 0x7f9b44150690 2024-09-13T17:50:12.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b494e6700 1 --2- 172.21.15.40:0/2027030851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b44150690 0x7f9b441ed6f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400c780 tx=0x7f9b3400ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:12.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b3a7fc700 1 -- 172.21.15.40:0/2027030851 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b34004ca0 con 0x7f9b44150690 2024-09-13T17:50:12.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b3a7fc700 1 -- 172.21.15.40:0/2027030851 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b34004e00 con 0x7f9b44150690 2024-09-13T17:50:12.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b4ace9700 1 -- 172.21.15.40:0/2027030851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b441f0ec0 con 0x7f9b44150690 2024-09-13T17:50:12.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b3a7fc700 1 -- 172.21.15.40:0/2027030851 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b34007310 con 0x7f9b44150690 2024-09-13T17:50:12.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b4ace9700 1 -- 172.21.15.40:0/2027030851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b441ed1d0 con 0x7f9b44150690 2024-09-13T17:50:12.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b3a7fc700 1 -- 172.21.15.40:0/2027030851 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f9b3401d3f0 con 0x7f9b44150690 2024-09-13T17:50:12.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b3a7fc700 1 -- 172.21.15.40:0/2027030851 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f9b34013a90 con 0x7f9b44150690 2024-09-13T17:50:12.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.689+0000 7f9b4ace9700 1 -- 172.21.15.40:0/2027030851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b441ed030 con 0x7f9b44150690 2024-09-13T17:50:12.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.697+0000 7f9b3a7fc700 1 -- 172.21.15.40:0/2027030851 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f9b441ed030 con 0x7f9b44150690 2024-09-13T17:50:12.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.765+0000 7f9b4ace9700 1 -- 172.21.15.40:0/2027030851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b4415a020 con 0x7f9b44150690 2024-09-13T17:50:12.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.765+0000 7f9b3a7fc700 1 -- 172.21.15.40:0/2027030851 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f9b34013e20 con 0x7f9b44150690 2024-09-13T17:50:12.771 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:50:12.771 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":10,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:50:12.518018+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:50:12.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.769+0000 7f9b4ace9700 1 -- 172.21.15.40:0/2027030851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b44150690 msgr2=0x7f9b441ed6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:12.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.769+0000 7f9b4ace9700 1 --2- 172.21.15.40:0/2027030851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b44150690 0x7f9b441ed6f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400c780 tx=0x7f9b3400ca90 comp rx=0 tx=0).stop 2024-09-13T17:50:12.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.769+0000 7f9b4ace9700 1 -- 172.21.15.40:0/2027030851 shutdown_connections 2024-09-13T17:50:12.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.769+0000 7f9b4ace9700 1 --2- 172.21.15.40:0/2027030851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9b441567e0 0x7f9b441edc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.769+0000 7f9b4ace9700 1 --2- 172.21.15.40:0/2027030851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b44150690 0x7f9b441ed6f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.769+0000 7f9b4ace9700 1 --2- 172.21.15.40:0/2027030851 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9b44070bc0 0x7f9b4406a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:12.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.769+0000 7f9b4ace9700 1 -- 172.21.15.40:0/2027030851 >> 172.21.15.40:0/2027030851 conn(0x7f9b440c88e0 msgr2=0x7f9b440c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:12.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.769+0000 7f9b4ace9700 1 -- 172.21.15.40:0/2027030851 shutdown_connections 2024-09-13T17:50:12.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:12.769+0000 7f9b4ace9700 1 -- 172.21.15.40:0/2027030851 wait complete. 2024-09-13T17:50:12.784 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T17:50:12.784 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T17:50:12.832 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:12.824+0000 7fc023e452c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:50:12.853 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:12.849+0000 7f0ad16a92c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:50:12.913 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-13T17:50:12.908+0000 7fc2588c8880 -1 Falling back to public interface 2024-09-13T17:50:12.917 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-13T17:50:12.913+0000 7f5e7729b880 -1 Falling back to public interface 2024-09-13T17:50:12.925 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-13T17:50:12.920+0000 7f59fd24d880 -1 Falling back to public interface 2024-09-13T17:50:12.935 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-13T17:50:12.933+0000 7f4b2507f880 -1 Falling back to public interface 2024-09-13T17:50:12.938 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-13T17:50:12.932+0000 7fd08c970880 -1 Falling back to public interface 2024-09-13T17:50:12.944 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-13T17:50:12.941+0000 7f2514005880 -1 Falling back to public interface 2024-09-13T17:50:12.972 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-13T17:50:12.969+0000 7fad55976880 -1 Falling back to public interface 2024-09-13T17:50:13.177 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-13T17:50:13.172+0000 7fdb42918880 -1 Falling back to public interface 2024-09-13T17:50:13.548 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:13.545+0000 7f0ad16a92c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:50:13.589 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T17:50:13.610 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:13.604+0000 7fc023e452c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:50:13.619 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:13.613+0000 7f0ad16a92c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:50:13.683 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:13.676+0000 7fc023e452c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:50:13.689 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.685+0000 7fe829226700 1 Processor -- start 2024-09-13T17:50:13.690 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.685+0000 7fe829226700 1 -- start start 2024-09-13T17:50:13.691 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.685+0000 7fe829226700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe824153420 0x7fe8240b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:13.691 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.685+0000 7fe829226700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe8240c2de0 0x7fe824153800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:13.692 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.685+0000 7fe829226700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe824071f60 con 0x7fe824159470 2024-09-13T17:50:13.692 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.685+0000 7fe829226700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8240c7dc0 con 0x7fe824153420 2024-09-13T17:50:13.692 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.685+0000 7fe829226700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8240c3200 con 0x7fe8240c2de0 2024-09-13T17:50:13.692 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe823fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe824153420 0x7fe8240b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:13.692 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe823fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe824153420 0x7fe8240b1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36378/0 (socket says 172.21.15.40:36378) 2024-09-13T17:50:13.692 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe823fff700 1 -- 172.21.15.40:0/2117473838 learned_addr learned my addr 172.21.15.40:0/2117473838 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:13.692 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe8237fe700 1 --2- 172.21.15.40:0/2117473838 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe8240c2de0 0x7fe824153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:13.692 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe823fff700 1 -- 172.21.15.40:0/2117473838 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe8240c2de0 msgr2=0x7fe824153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:13.692 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe823fff700 1 --2- 172.21.15.40:0/2117473838 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe8240c2de0 0x7fe824153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:13.692 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe823fff700 1 -- 172.21.15.40:0/2117473838 >> v1:172.21.15.43:6789/0 conn(0x7fe824159470 legacy=0x7fe8240cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:13.692 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe823fff700 1 -- 172.21.15.40:0/2117473838 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8240c69c0 con 0x7fe824153420 2024-09-13T17:50:13.692 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe823fff700 1 --2- 172.21.15.40:0/2117473838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe824153420 0x7fe8240b1220 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe814009940 tx=0x7fe814009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835a503faf2e0322 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:13.694 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe822ffd700 1 -- 172.21.15.40:0/2117473838 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81400ba20 con 0x7fe824153420 2024-09-13T17:50:13.694 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe822ffd700 1 -- 172.21.15.40:0/2117473838 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe814009e90 con 0x7fe824153420 2024-09-13T17:50:13.694 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe822ffd700 1 -- 172.21.15.40:0/2117473838 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81400bda0 con 0x7fe824153420 2024-09-13T17:50:13.694 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe829226700 1 -- 172.21.15.40:0/2117473838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe824153420 msgr2=0x7fe8240b1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:13.694 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe829226700 1 --2- 172.21.15.40:0/2117473838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe824153420 0x7fe8240b1220 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe814009940 tx=0x7fe814009c50 comp rx=0 tx=0).stop 2024-09-13T17:50:13.694 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe829226700 1 -- 172.21.15.40:0/2117473838 shutdown_connections 2024-09-13T17:50:13.694 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe829226700 1 --2- 172.21.15.40:0/2117473838 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe8240c2de0 0x7fe824153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:13.694 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe829226700 1 --2- 172.21.15.40:0/2117473838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe824153420 0x7fe8240b1220 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:13.694 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe829226700 1 -- 172.21.15.40:0/2117473838 >> 172.21.15.40:0/2117473838 conn(0x7fe82406cbc0 msgr2=0x7fe82406cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:13.694 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe829226700 1 -- 172.21.15.40:0/2117473838 shutdown_connections 2024-09-13T17:50:13.694 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe829226700 1 -- 172.21.15.40:0/2117473838 wait complete. 2024-09-13T17:50:13.695 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe829226700 1 Processor -- start 2024-09-13T17:50:13.695 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe829226700 1 -- start start 2024-09-13T17:50:13.697 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe829226700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8240c2de0 0x7fe8240c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:13.697 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe829226700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe824153420 0x7fe8240c5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:13.697 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe829226700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe824159470 0x7fe8240c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:13.697 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe829226700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8241f0040 con 0x7fe8240c2de0 2024-09-13T17:50:13.697 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe829226700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8241ef150 con 0x7fe824159470 2024-09-13T17:50:13.697 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe829226700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8241ef430 con 0x7fe824153420 2024-09-13T17:50:13.697 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe823fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8240c2de0 0x7fe8240c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:13.697 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe823fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8240c2de0 0x7fe8240c4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36394/0 (socket says 172.21.15.40:36394) 2024-09-13T17:50:13.698 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.689+0000 7fe823fff700 1 -- 172.21.15.40:0/2818156408 learned_addr learned my addr 172.21.15.40:0/2818156408 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:13.698 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.693+0000 7fe8237fe700 1 --2- 172.21.15.40:0/2818156408 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe824153420 0x7fe8240c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:13.698 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.693+0000 7fe828a25700 1 --2- 172.21.15.40:0/2818156408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe824159470 0x7fe8240c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:13.698 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.693+0000 7fe823fff700 1 -- 172.21.15.40:0/2818156408 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe824153420 msgr2=0x7fe8240c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:13.698 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.693+0000 7fe823fff700 1 --2- 172.21.15.40:0/2818156408 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe824153420 0x7fe8240c5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:13.698 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.693+0000 7fe823fff700 1 -- 172.21.15.40:0/2818156408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe824159470 msgr2=0x7fe8240c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:13.698 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.693+0000 7fe823fff700 1 --2- 172.21.15.40:0/2818156408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe824159470 0x7fe8240c3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:13.698 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.693+0000 7fe823fff700 1 -- 172.21.15.40:0/2818156408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe814009570 con 0x7fe8240c2de0 2024-09-13T17:50:13.698 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.693+0000 7fe828a25700 1 --2- 172.21.15.40:0/2818156408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe824159470 0x7fe8240c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:50:13.698 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.693+0000 7fe823fff700 1 --2- 172.21.15.40:0/2818156408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8240c2de0 0x7fe8240c4e10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe814000c00 tx=0x7fe814004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:13.698 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.693+0000 7fe820ff9700 1 -- 172.21.15.40:0/2818156408 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814004550 con 0x7fe8240c2de0 2024-09-13T17:50:13.698 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.693+0000 7fe820ff9700 1 -- 172.21.15.40:0/2818156408 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81401f070 con 0x7fe8240c2de0 2024-09-13T17:50:13.698 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.693+0000 7fe829226700 1 -- 172.21.15.40:0/2818156408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8241ef710 con 0x7fe8240c2de0 2024-09-13T17:50:13.698 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.693+0000 7fe820ff9700 1 -- 172.21.15.40:0/2818156408 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814003900 con 0x7fe8240c2de0 2024-09-13T17:50:13.699 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.693+0000 7fe829226700 1 -- 172.21.15.40:0/2818156408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8241efcd0 con 0x7fe8240c2de0 2024-09-13T17:50:13.699 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.693+0000 7fe820ff9700 1 -- 172.21.15.40:0/2818156408 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fe81401a430 con 0x7fe8240c2de0 2024-09-13T17:50:13.699 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.693+0000 7fe820ff9700 1 -- 172.21.15.40:0/2818156408 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fe814021070 con 0x7fe8240c2de0 2024-09-13T17:50:13.699 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.693+0000 7fe829226700 1 -- 172.21.15.40:0/2818156408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8240c15a0 con 0x7fe8240c2de0 2024-09-13T17:50:13.701 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.697+0000 7fe820ff9700 1 -- 172.21.15.40:0/2818156408 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fe814015020 con 0x7fe8240c2de0 2024-09-13T17:50:13.764 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:13.761+0000 7f0ad16a92c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:50:13.767 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.765+0000 7fe829226700 1 -- 172.21.15.40:0/2818156408 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe824158240 con 0x7fe8240c2de0 2024-09-13T17:50:13.768 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.765+0000 7fe820ff9700 1 -- 172.21.15.40:0/2818156408 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fe81401a820 con 0x7fe8240c2de0 2024-09-13T17:50:13.768 INFO:teuthology.misc.health.smithi040.stdout: 2024-09-13T17:50:13.768 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":10,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:50:12.518018+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:50:13.769 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.765+0000 7fe829226700 1 -- 172.21.15.40:0/2818156408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8240c2de0 msgr2=0x7fe8240c4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:13.769 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.765+0000 7fe829226700 1 --2- 172.21.15.40:0/2818156408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8240c2de0 0x7fe8240c4e10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe814000c00 tx=0x7fe814004770 comp rx=0 tx=0).stop 2024-09-13T17:50:13.770 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.765+0000 7fe829226700 1 -- 172.21.15.40:0/2818156408 shutdown_connections 2024-09-13T17:50:13.770 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.765+0000 7fe829226700 1 --2- 172.21.15.40:0/2818156408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe824159470 0x7fe8240c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:13.770 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.765+0000 7fe829226700 1 --2- 172.21.15.40:0/2818156408 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe824153420 0x7fe8240c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:13.770 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.765+0000 7fe829226700 1 --2- 172.21.15.40:0/2818156408 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8240c2de0 0x7fe8240c4e10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:13.770 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.765+0000 7fe829226700 1 -- 172.21.15.40:0/2818156408 >> 172.21.15.40:0/2818156408 conn(0x7fe82406cbc0 msgr2=0x7fe8241575f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:13.770 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.765+0000 7fe829226700 1 -- 172.21.15.40:0/2818156408 shutdown_connections 2024-09-13T17:50:13.770 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:13.765+0000 7fe829226700 1 -- 172.21.15.40:0/2818156408 wait complete. 2024-09-13T17:50:13.780 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T17:50:13.829 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:13.825+0000 7f0ad16a92c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:50:13.846 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:13.840+0000 7fc023e452c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:50:13.897 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:13.893+0000 7f0ad16a92c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:50:13.915 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:13.908+0000 7fc023e452c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:50:13.992 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:13.988+0000 7fc023e452c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:50:14.198 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:14.193+0000 7f0ad16a92c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:50:14.271 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:14.269+0000 7f0ad16a92c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:50:14.321 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:14.316+0000 7fc023e452c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:50:14.400 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:14.396+0000 7fc023e452c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:50:14.960 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:14.957+0000 7f0ad16a92c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:50:15.115 INFO:tasks.ceph.mgr.x.smithi043.stderr:2024-09-13T17:50:15.108+0000 7fc023e452c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:50:16.004 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:16.001+0000 7f0ab821f700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T17:50:16.004 INFO:tasks.ceph.mgr.y.smithi040.stderr:2024-09-13T17:50:16.001+0000 7f0ab821f700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T17:50:16.314 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-13T17:50:16.308+0000 7fd08c970880 -1 osd.7 0 log_to_monitors true 2024-09-13T17:50:16.394 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-13T17:50:16.388+0000 7f59fd24d880 -1 osd.6 0 log_to_monitors true 2024-09-13T17:50:16.542 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-13T17:50:16.537+0000 7f5e7729b880 -1 osd.2 0 log_to_monitors true 2024-09-13T17:50:16.561 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-13T17:50:16.557+0000 7f2514005880 -1 osd.0 0 log_to_monitors true 2024-09-13T17:50:16.784 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-13T17:50:16.780+0000 7fc2588c8880 -1 osd.5 0 log_to_monitors true 2024-09-13T17:50:16.791 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-13T17:50:16.789+0000 7f4b2507f880 -1 osd.1 0 log_to_monitors true 2024-09-13T17:50:16.886 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-13T17:50:16.881+0000 7fad55976880 -1 osd.3 0 log_to_monitors true 2024-09-13T17:50:17.081 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-13T17:50:17.076+0000 7fdb42918880 -1 osd.4 0 log_to_monitors true 2024-09-13T17:50:19.267 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-13T17:50:19.260+0000 7f59f908c700 -1 osd.6 0 waiting for initial osdmap 2024-09-13T17:50:19.273 INFO:tasks.ceph.osd.6.smithi043.stderr:2024-09-13T17:50:19.268+0000 7f59f0b3b700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:50:19.277 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-13T17:50:19.272+0000 7fd0887af700 -1 osd.7 0 waiting for initial osdmap 2024-09-13T17:50:19.284 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-13T17:50:19.276+0000 7fc254707700 -1 osd.5 0 waiting for initial osdmap 2024-09-13T17:50:19.284 INFO:tasks.ceph.osd.7.smithi043.stderr:2024-09-13T17:50:19.280+0000 7fd08025e700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:50:19.287 INFO:tasks.ceph.osd.5.smithi043.stderr:2024-09-13T17:50:19.284+0000 7fc24c1b6700 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:50:19.947 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-13T17:50:19.944+0000 7fdb3e757700 -1 osd.4 0 waiting for initial osdmap 2024-09-13T17:50:19.955 INFO:tasks.ceph.osd.4.smithi043.stderr:2024-09-13T17:50:19.952+0000 7fdb36206700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:50:20.120 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-13T17:50:20.117+0000 7f4b20ebe700 -1 osd.1 0 waiting for initial osdmap 2024-09-13T17:50:20.134 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-13T17:50:20.129+0000 7f4b1896d700 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:50:20.207 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-13T17:50:20.201+0000 7fad517b5700 -1 osd.3 0 waiting for initial osdmap 2024-09-13T17:50:20.211 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-13T17:50:20.205+0000 7f5e730da700 -1 osd.2 0 waiting for initial osdmap 2024-09-13T17:50:20.212 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-13T17:50:20.209+0000 7fad49264700 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:50:20.215 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-13T17:50:20.213+0000 7f5e6ab89700 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:50:20.230 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-13T17:50:20.225+0000 7f250fe44700 -1 osd.0 0 waiting for initial osdmap 2024-09-13T17:50:20.236 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-13T17:50:20.233+0000 7f25078f3700 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:50:20.587 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T17:50:20.699 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8f59e700 1 Processor -- start 2024-09-13T17:50:20.699 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8f59e700 1 -- start start 2024-09-13T17:50:20.700 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8f59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac9006ac90 0x7fac9006a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:20.701 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8f59e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fac90160df0 0x7fac90161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:20.701 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8f59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9006a8d0 con 0x7fac90157a00 2024-09-13T17:50:20.701 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8f59e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac90157de0 con 0x7fac90160df0 2024-09-13T17:50:20.701 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8f59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac90161740 con 0x7fac9006ac90 2024-09-13T17:50:20.701 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8e59c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fac90157a00 0x7fac9006e390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48588/0 (socket says 172.21.15.40:48588) 2024-09-13T17:50:20.701 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8e59c700 1 -- 172.21.15.40:0/3829717326 learned_addr learned my addr 172.21.15.40:0/3829717326 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:20.701 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8dd9b700 1 --2- 172.21.15.40:0/3829717326 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac9006ac90 0x7fac9006a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:20.701 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8ed9d700 1 --2- 172.21.15.40:0/3829717326 >> v2:172.21.15.43:3300/0 conn(0x7fac90160df0 0x7fac90161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:20.701 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8d59a700 1 -- 172.21.15.40:0/3829717326 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334764975 0 0) 0x7fac78001110 con 0x7fac90157a00 2024-09-13T17:50:20.701 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8d59a700 1 -- 172.21.15.40:0/3829717326 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac7c0035d0 con 0x7fac90157a00 2024-09-13T17:50:20.705 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8dd9b700 1 -- 172.21.15.40:0/3829717326 >> v2:172.21.15.43:3300/0 conn(0x7fac90160df0 msgr2=0x7fac90161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:20.705 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8dd9b700 1 --2- 172.21.15.40:0/3829717326 >> v2:172.21.15.43:3300/0 conn(0x7fac90160df0 0x7fac90161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:20.705 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8dd9b700 1 -- 172.21.15.40:0/3829717326 >> v1:172.21.15.40:6789/0 conn(0x7fac90157a00 legacy=0x7fac9006e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:20.705 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8d59a700 1 -- 172.21.15.40:0/3829717326 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1486136811 0 0) 0x7fac7c0035d0 con 0x7fac90157a00 2024-09-13T17:50:20.705 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8dd9b700 1 -- 172.21.15.40:0/3829717326 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9006be80 con 0x7fac9006ac90 2024-09-13T17:50:20.705 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8dd9b700 1 --2- 172.21.15.40:0/3829717326 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac9006ac90 0x7fac9006a390 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fac80009850 tx=0x7fac80009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b617e4b3dcf30001 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:20.705 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8d59a700 1 -- 172.21.15.40:0/3829717326 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac80011830 con 0x7fac9006ac90 2024-09-13T17:50:20.705 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8d59a700 1 -- 172.21.15.40:0/3829717326 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac80011990 con 0x7fac9006ac90 2024-09-13T17:50:20.705 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8d59a700 1 -- 172.21.15.40:0/3829717326 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac80011d00 con 0x7fac9006ac90 2024-09-13T17:50:20.705 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8f59e700 1 -- 172.21.15.40:0/3829717326 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac9006ac90 msgr2=0x7fac9006a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:20.705 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8f59e700 1 --2- 172.21.15.40:0/3829717326 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac9006ac90 0x7fac9006a390 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fac80009850 tx=0x7fac80009b60 comp rx=0 tx=0).stop 2024-09-13T17:50:20.706 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8f59e700 1 -- 172.21.15.40:0/3829717326 shutdown_connections 2024-09-13T17:50:20.706 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8f59e700 1 --2- 172.21.15.40:0/3829717326 >> v2:172.21.15.43:3300/0 conn(0x7fac90160df0 0x7fac90161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:20.706 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8f59e700 1 --2- 172.21.15.40:0/3829717326 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac9006ac90 0x7fac9006a390 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:20.706 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.697+0000 7fac8f59e700 1 -- 172.21.15.40:0/3829717326 >> 172.21.15.40:0/3829717326 conn(0x7fac9006d870 msgr2=0x7fac9006dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:20.706 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8f59e700 1 -- 172.21.15.40:0/3829717326 shutdown_connections 2024-09-13T17:50:20.706 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8f59e700 1 -- 172.21.15.40:0/3829717326 wait complete. 2024-09-13T17:50:20.706 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8f59e700 1 Processor -- start 2024-09-13T17:50:20.706 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8f59e700 1 -- start start 2024-09-13T17:50:20.708 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8f59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac9006ac90 0x7fac90150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:20.708 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8f59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac90157a00 0x7fac90160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:20.708 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8f59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac90160df0 0x7fac901606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:20.708 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8f59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac900c49e0 con 0x7fac90157a00 2024-09-13T17:50:20.708 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8f59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac900c4c30 con 0x7fac90160df0 2024-09-13T17:50:20.708 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8f59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9015f760 con 0x7fac9006ac90 2024-09-13T17:50:20.708 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac90157a00 0x7fac90160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:20.708 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac90157a00 0x7fac90160160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36518/0 (socket says 172.21.15.40:36518) 2024-09-13T17:50:20.708 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8dd9b700 1 -- 172.21.15.40:0/2063446685 learned_addr learned my addr 172.21.15.40:0/2063446685 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:20.708 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8e59c700 1 --2- 172.21.15.40:0/2063446685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac9006ac90 0x7fac90150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:20.708 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8ed9d700 1 --2- 172.21.15.40:0/2063446685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac90160df0 0x7fac901606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:20.709 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8dd9b700 1 -- 172.21.15.40:0/2063446685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac9006ac90 msgr2=0x7fac90150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:20.709 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8dd9b700 1 --2- 172.21.15.40:0/2063446685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac9006ac90 0x7fac90150f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:20.709 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8dd9b700 1 -- 172.21.15.40:0/2063446685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac90160df0 msgr2=0x7fac901606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:20.709 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8dd9b700 1 --2- 172.21.15.40:0/2063446685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac90160df0 0x7fac901606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:20.709 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8dd9b700 1 -- 172.21.15.40:0/2063446685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac800094b0 con 0x7fac90157a00 2024-09-13T17:50:20.709 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8ed9d700 1 --2- 172.21.15.40:0/2063446685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac90160df0 0x7fac901606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:50:20.709 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.701+0000 7fac8dd9b700 1 --2- 172.21.15.40:0/2063446685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac90157a00 0x7fac90160160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fac800090b0 tx=0x7fac80004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:20.709 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.705+0000 7fac86ffd700 1 -- 172.21.15.40:0/2063446685 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac80004480 con 0x7fac90157a00 2024-09-13T17:50:20.709 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.705+0000 7fac8f59e700 1 -- 172.21.15.40:0/2063446685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9015fa40 con 0x7fac90157a00 2024-09-13T17:50:20.709 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.705+0000 7fac8f59e700 1 -- 172.21.15.40:0/2063446685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac900727e0 con 0x7fac90157a00 2024-09-13T17:50:20.709 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.705+0000 7fac86ffd700 1 -- 172.21.15.40:0/2063446685 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8001f070 con 0x7fac90157a00 2024-09-13T17:50:20.709 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.705+0000 7fac86ffd700 1 -- 172.21.15.40:0/2063446685 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac800038c0 con 0x7fac90157a00 2024-09-13T17:50:20.709 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.705+0000 7fac8f59e700 1 -- 172.21.15.40:0/2063446685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac900c2f50 con 0x7fac90157a00 2024-09-13T17:50:20.710 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.705+0000 7fac86ffd700 1 -- 172.21.15.40:0/2063446685 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7fac80021030 con 0x7fac90157a00 2024-09-13T17:50:20.710 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.705+0000 7fac86ffd700 1 --2- 172.21.15.40:0/2063446685 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fac700666a0 0x7fac70068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:20.710 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.705+0000 7fac86ffd700 1 -- 172.21.15.40:0/2063446685 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3351+0+0 (secure 0 0 0) 0x7fac80084c50 con 0x7fac90157a00 2024-09-13T17:50:20.710 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.705+0000 7fac8e59c700 1 --2- 172.21.15.40:0/2063446685 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fac700666a0 0x7fac70068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:20.711 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.709+0000 7fac8e59c700 1 --2- 172.21.15.40:0/2063446685 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fac700666a0 0x7fac70068b60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fac78001390 tx=0x7fac78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:20.715 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.713+0000 7fac86ffd700 1 -- 172.21.15.40:0/2063446685 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fac80052af0 con 0x7fac90157a00 2024-09-13T17:50:20.885 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.881+0000 7fac8f59e700 1 -- 172.21.15.40:0/2063446685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac9006ff40 con 0x7fac90157a00 2024-09-13T17:50:20.887 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.885+0000 7fac86ffd700 1 -- 172.21.15.40:0/2063446685 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+8952 (secure 0 0 0) 0x7fac80015070 con 0x7fac90157a00 2024-09-13T17:50:20.887 INFO:teuthology.misc.health.smithi040.stdout: 2024-09-13T17:50:20.887 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":14,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:50:20.038025+0000","last_up_change":"2024-09-13T17:50:20.038025+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:50:20.889 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.885+0000 7fac8f59e700 1 -- 172.21.15.40:0/2063446685 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fac700666a0 msgr2=0x7fac70068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:20.889 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.885+0000 7fac8f59e700 1 --2- 172.21.15.40:0/2063446685 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fac700666a0 0x7fac70068b60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fac78001390 tx=0x7fac78009040 comp rx=0 tx=0).stop 2024-09-13T17:50:20.889 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.885+0000 7fac8f59e700 1 -- 172.21.15.40:0/2063446685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac90157a00 msgr2=0x7fac90160160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:20.890 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.885+0000 7fac8f59e700 1 --2- 172.21.15.40:0/2063446685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac90157a00 0x7fac90160160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fac800090b0 tx=0x7fac80004770 comp rx=0 tx=0).stop 2024-09-13T17:50:20.890 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.885+0000 7fac8f59e700 1 -- 172.21.15.40:0/2063446685 shutdown_connections 2024-09-13T17:50:20.890 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.885+0000 7fac8f59e700 1 --2- 172.21.15.40:0/2063446685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac90160df0 0x7fac901606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:20.890 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.885+0000 7fac8f59e700 1 --2- 172.21.15.40:0/2063446685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac90157a00 0x7fac90160160 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:20.890 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.885+0000 7fac8f59e700 1 --2- 172.21.15.40:0/2063446685 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fac700666a0 0x7fac70068b60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:20.890 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.885+0000 7fac8f59e700 1 --2- 172.21.15.40:0/2063446685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac9006ac90 0x7fac90150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:20.890 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.885+0000 7fac8f59e700 1 -- 172.21.15.40:0/2063446685 >> 172.21.15.40:0/2063446685 conn(0x7fac9006d870 msgr2=0x7fac900ca380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:20.890 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.885+0000 7fac8f59e700 1 -- 172.21.15.40:0/2063446685 shutdown_connections 2024-09-13T17:50:20.890 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:20.885+0000 7fac8f59e700 1 -- 172.21.15.40:0/2063446685 wait complete. 2024-09-13T17:50:20.900 DEBUG:teuthology.misc:4 of 8 OSDs are up 2024-09-13T17:50:27.708 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T17:50:27.823 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f649c1ce700 1 Processor -- start 2024-09-13T17:50:27.824 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f649c1ce700 1 -- start start 2024-09-13T17:50:27.825 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f649c1ce700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6494157f50 0x7f64940c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:27.825 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f649c1ce700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f649406b740 0x7f64940cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:27.825 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f649c1ce700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64940b4f60 con 0x7f64940ca3b0 2024-09-13T17:50:27.825 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f649c1ce700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64940c4370 con 0x7f6494157f50 2024-09-13T17:50:27.825 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f649c1ce700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649415db60 con 0x7f649406b740 2024-09-13T17:50:27.825 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f6499769700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f64940ca3b0 0x7f649406b030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48304/0 (socket says 172.21.15.40:48304) 2024-09-13T17:50:27.825 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f6499769700 1 -- 172.21.15.40:0/2004542142 learned_addr learned my addr 172.21.15.40:0/2004542142 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:27.825 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f6499f6a700 1 --2- 172.21.15.40:0/2004542142 >> v2:172.21.15.40:3300/0 conn(0x7f6494157f50 0x7f64940c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:27.825 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f649a76b700 1 --2- 172.21.15.40:0/2004542142 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f649406b740 0x7f64940cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:27.825 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f6498f68700 1 -- 172.21.15.40:0/2004542142 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 618633323 0 0) 0x7f64940b4f60 con 0x7f64940ca3b0 2024-09-13T17:50:27.825 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f6498f68700 1 -- 172.21.15.40:0/2004542142 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f647c0035d0 con 0x7f64940ca3b0 2024-09-13T17:50:27.826 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f6499f6a700 1 -- 172.21.15.40:0/2004542142 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f649406b740 msgr2=0x7f64940cabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:27.826 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f6498f68700 1 -- 172.21.15.40:0/2004542142 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 733448778 0 0) 0x7f647c0035d0 con 0x7f64940ca3b0 2024-09-13T17:50:27.826 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f6499f6a700 1 --2- 172.21.15.40:0/2004542142 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f649406b740 0x7f64940cabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:27.827 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f6499f6a700 1 -- 172.21.15.40:0/2004542142 >> v1:172.21.15.40:6789/0 conn(0x7f64940ca3b0 legacy=0x7f649406b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:27.827 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f6499f6a700 1 -- 172.21.15.40:0/2004542142 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649415de40 con 0x7f6494157f50 2024-09-13T17:50:27.827 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f6499f6a700 1 --2- 172.21.15.40:0/2004542142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6494157f50 0x7f64940c9e70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6488009850 tx=0x7f6488009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d8bbb626b6b451 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:27.827 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f6498f68700 1 -- 172.21.15.40:0/2004542142 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c001a10 con 0x7f6494157f50 2024-09-13T17:50:27.827 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f6498f68700 1 -- 172.21.15.40:0/2004542142 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6488004030 con 0x7f6494157f50 2024-09-13T17:50:27.827 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.821+0000 7f6498f68700 1 -- 172.21.15.40:0/2004542142 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64880043e0 con 0x7f6494157f50 2024-09-13T17:50:27.828 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.825+0000 7f649c1ce700 1 -- 172.21.15.40:0/2004542142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6494157f50 msgr2=0x7f64940c9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:27.828 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.825+0000 7f649c1ce700 1 --2- 172.21.15.40:0/2004542142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6494157f50 0x7f64940c9e70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6488009850 tx=0x7f6488009b60 comp rx=0 tx=0).stop 2024-09-13T17:50:27.828 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.825+0000 7f649c1ce700 1 -- 172.21.15.40:0/2004542142 shutdown_connections 2024-09-13T17:50:27.828 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.825+0000 7f649c1ce700 1 --2- 172.21.15.40:0/2004542142 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f649406b740 0x7f64940cabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:27.828 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.825+0000 7f649c1ce700 1 --2- 172.21.15.40:0/2004542142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6494157f50 0x7f64940c9e70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:27.828 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.825+0000 7f649c1ce700 1 -- 172.21.15.40:0/2004542142 >> 172.21.15.40:0/2004542142 conn(0x7f6494066620 msgr2=0x7f6494066a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:27.828 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.825+0000 7f649c1ce700 1 -- 172.21.15.40:0/2004542142 shutdown_connections 2024-09-13T17:50:27.828 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.825+0000 7f649c1ce700 1 -- 172.21.15.40:0/2004542142 wait complete. 2024-09-13T17:50:27.830 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.825+0000 7f649c1ce700 1 Processor -- start 2024-09-13T17:50:27.830 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.825+0000 7f649c1ce700 1 -- start start 2024-09-13T17:50:27.833 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f649c1ce700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f649406b740 0x7f649414efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:27.833 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f649c1ce700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f64940ca3b0 0x7f64940ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:27.833 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f649c1ce700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6494157f50 0x7f649414f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:27.833 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f649c1ce700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64941f6f40 con 0x7f649406b740 2024-09-13T17:50:27.833 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f649c1ce700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64940c8b50 con 0x7f6494157f50 2024-09-13T17:50:27.833 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f649c1ce700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64941f7150 con 0x7f64940ca3b0 2024-09-13T17:50:27.834 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f6499f6a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f649406b740 0x7f649414efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:27.834 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f6499f6a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f649406b740 0x7f649414efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35242/0 (socket says 172.21.15.40:35242) 2024-09-13T17:50:27.834 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f6499f6a700 1 -- 172.21.15.40:0/3801464379 learned_addr learned my addr 172.21.15.40:0/3801464379 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:27.834 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f649a76b700 1 --2- 172.21.15.40:0/3801464379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6494157f50 0x7f649414f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:27.834 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f6499769700 1 --2- 172.21.15.40:0/3801464379 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f64940ca3b0 0x7f64940ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:27.834 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f6499f6a700 1 -- 172.21.15.40:0/3801464379 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f64940ca3b0 msgr2=0x7f64940ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:27.834 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f6499f6a700 1 --2- 172.21.15.40:0/3801464379 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f64940ca3b0 0x7f64940ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:27.834 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f6499f6a700 1 -- 172.21.15.40:0/3801464379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6494157f50 msgr2=0x7f649414f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:27.834 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f6499f6a700 1 --2- 172.21.15.40:0/3801464379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6494157f50 0x7f649414f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:27.834 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f6499f6a700 1 -- 172.21.15.40:0/3801464379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64880094b0 con 0x7f649406b740 2024-09-13T17:50:27.835 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f6499f6a700 1 --2- 172.21.15.40:0/3801464379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f649406b740 0x7f649414efc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6488000c00 tx=0x7f6488004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:27.835 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f64927fc700 1 -- 172.21.15.40:0/3801464379 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6488003b00 con 0x7f649406b740 2024-09-13T17:50:27.835 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f64927fc700 1 -- 172.21.15.40:0/3801464379 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648801f070 con 0x7f649406b740 2024-09-13T17:50:27.835 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f64927fc700 1 -- 172.21.15.40:0/3801464379 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6488011630 con 0x7f649406b740 2024-09-13T17:50:27.835 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f649c1ce700 1 -- 172.21.15.40:0/3801464379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64940c8df0 con 0x7f649406b740 2024-09-13T17:50:27.835 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.829+0000 7f649c1ce700 1 -- 172.21.15.40:0/3801464379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64941f7410 con 0x7f649406b740 2024-09-13T17:50:27.837 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.833+0000 7f64927fc700 1 -- 172.21.15.40:0/3801464379 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6488021030 con 0x7f649406b740 2024-09-13T17:50:27.837 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.833+0000 7f649c1ce700 1 -- 172.21.15.40:0/3801464379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f649415d5f0 con 0x7f649406b740 2024-09-13T17:50:27.838 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.833+0000 7f64927fc700 1 --2- 172.21.15.40:0/3801464379 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6480066730 0x7f6480068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:27.838 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.833+0000 7f64927fc700 1 -- 172.21.15.40:0/3801464379 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f64880850b0 con 0x7f649406b740 2024-09-13T17:50:27.838 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.833+0000 7f6499769700 1 --2- 172.21.15.40:0/3801464379 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6480066730 0x7f6480068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:27.839 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.837+0000 7f6499769700 1 --2- 172.21.15.40:0/3801464379 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6480066730 0x7f6480068bf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f64940c5ff0 tx=0x7f648c0015d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:27.845 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:27.841+0000 7f64927fc700 1 -- 172.21.15.40:0/3801464379 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6488052800 con 0x7f649406b740 2024-09-13T17:50:28.017 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:28.009+0000 7f649c1ce700 1 -- 172.21.15.40:0/3801464379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f649415b750 con 0x7f649406b740 2024-09-13T17:50:28.019 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:28.013+0000 7f64927fc700 1 -- 172.21.15.40:0/3801464379 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12766 (secure 0 0 0) 0x7f6488055e20 con 0x7f649406b740 2024-09-13T17:50:28.019 INFO:teuthology.misc.health.smithi040.stdout: 2024-09-13T17:50:28.019 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":17,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:50:23.070493+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:50:28.022 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:28.013+0000 7f649c1ce700 1 -- 172.21.15.40:0/3801464379 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6480066730 msgr2=0x7f6480068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:28.022 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:28.013+0000 7f649c1ce700 1 --2- 172.21.15.40:0/3801464379 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6480066730 0x7f6480068bf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f64940c5ff0 tx=0x7f648c0015d0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.022 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:28.013+0000 7f649c1ce700 1 -- 172.21.15.40:0/3801464379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f649406b740 msgr2=0x7f649414efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:28.022 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:28.013+0000 7f649c1ce700 1 --2- 172.21.15.40:0/3801464379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f649406b740 0x7f649414efc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6488000c00 tx=0x7f6488004770 comp rx=0 tx=0).stop 2024-09-13T17:50:28.022 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:28.013+0000 7f649c1ce700 1 -- 172.21.15.40:0/3801464379 shutdown_connections 2024-09-13T17:50:28.022 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:28.013+0000 7f649c1ce700 1 --2- 172.21.15.40:0/3801464379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6494157f50 0x7f649414f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.022 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:28.013+0000 7f649c1ce700 1 --2- 172.21.15.40:0/3801464379 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6480066730 0x7f6480068bf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.022 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:28.013+0000 7f649c1ce700 1 --2- 172.21.15.40:0/3801464379 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f64940ca3b0 0x7f64940ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.022 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:28.013+0000 7f649c1ce700 1 --2- 172.21.15.40:0/3801464379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f649406b740 0x7f649414efc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.022 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:28.013+0000 7f649c1ce700 1 -- 172.21.15.40:0/3801464379 >> 172.21.15.40:0/3801464379 conn(0x7f6494066620 msgr2=0x7f649415c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:28.022 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:28.013+0000 7f649c1ce700 1 -- 172.21.15.40:0/3801464379 shutdown_connections 2024-09-13T17:50:28.022 INFO:teuthology.misc.health.smithi040.stderr:2024-09-13T17:50:28.013+0000 7f649c1ce700 1 -- 172.21.15.40:0/3801464379 wait complete. 2024-09-13T17:50:28.033 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-13T17:50:28.033 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T17:50:28.033 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T17:50:28.033 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T17:50:28.036 INFO:tasks.ceph.mds.a:Started 2024-09-13T17:50:28.036 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T17:50:28.036 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T17:50:28.038 INFO:tasks.ceph.mds.c:Started 2024-09-13T17:50:28.039 INFO:tasks.ceph.mds.e:Restarting daemon 2024-09-13T17:50:28.039 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-09-13T17:50:28.041 INFO:tasks.ceph.mds.e:Started 2024-09-13T17:50:28.041 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T17:50:28.042 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T17:50:28.044 INFO:tasks.ceph.mds.b:Started 2024-09-13T17:50:28.044 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T17:50:28.044 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T17:50:28.046 INFO:tasks.ceph.mds.d:Started 2024-09-13T17:50:28.046 INFO:tasks.ceph.mds.f:Restarting daemon 2024-09-13T17:50:28.046 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-09-13T17:50:28.048 INFO:tasks.ceph.mds.f:Started 2024-09-13T17:50:28.048 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-13T17:50:28.049 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:50:28.109 INFO:tasks.ceph.mds.e.smithi040.stdout:starting mds.e at 2024-09-13T17:50:28.114 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2024-09-13T17:50:28.129 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2024-09-13T17:50:28.132 INFO:tasks.ceph.mds.f.smithi043.stdout:starting mds.f at 2024-09-13T17:50:28.134 INFO:tasks.ceph.mds.d.smithi043.stdout:starting mds.d at 2024-09-13T17:50:28.138 INFO:tasks.ceph.mds.b.smithi043.stdout:starting mds.b at 2024-09-13T17:50:28.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.157+0000 7f454f915700 1 Processor -- start 2024-09-13T17:50:28.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.157+0000 7f454f915700 1 -- start start 2024-09-13T17:50:28.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.161+0000 7f454f915700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f45480c6d80 0x7f4548155a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:28.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.161+0000 7f454f915700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f45480cb520 0x7f4548155f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:28.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.161+0000 7f454f915700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4548156650 con 0x7f45480c69a0 2024-09-13T17:50:28.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.161+0000 7f454f915700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4548157650 con 0x7f45480c6d80 2024-09-13T17:50:28.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.161+0000 7f454f915700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45481590e0 con 0x7f45480cb520 2024-09-13T17:50:28.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.161+0000 7f454d6b1700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f45480c69a0 0x7f4548156f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48330/0 (socket says 172.21.15.40:48330) 2024-09-13T17:50:28.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.161+0000 7f454d6b1700 1 -- 172.21.15.40:0/2735916674 learned_addr learned my addr 172.21.15.40:0/2735916674 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:28.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.161+0000 7f454ceb0700 1 --2- 172.21.15.40:0/2735916674 >> v2:172.21.15.43:3300/0 conn(0x7f45480c6d80 0x7f4548155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.161+0000 7f454deb2700 1 --2- 172.21.15.40:0/2735916674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f45480cb520 0x7f4548155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.161+0000 7f4547fff700 1 -- 172.21.15.40:0/2735916674 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1096805252 0 0) 0x7f4548156650 con 0x7f45480c69a0 2024-09-13T17:50:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.161+0000 7f4547fff700 1 -- 172.21.15.40:0/2735916674 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45300035d0 con 0x7f45480c69a0 2024-09-13T17:50:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.161+0000 7f454ceb0700 1 -- 172.21.15.40:0/2735916674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f45480cb520 msgr2=0x7f4548155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.161+0000 7f454ceb0700 1 --2- 172.21.15.40:0/2735916674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f45480cb520 0x7f4548155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.161+0000 7f454ceb0700 1 -- 172.21.15.40:0/2735916674 >> v1:172.21.15.40:6789/0 conn(0x7f45480c69a0 legacy=0x7f4548156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.161+0000 7f454ceb0700 1 -- 172.21.15.40:0/2735916674 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45481593c0 con 0x7f45480c6d80 2024-09-13T17:50:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.161+0000 7f454deb2700 1 --2- 172.21.15.40:0/2735916674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f45480cb520 0x7f4548155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:50:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.161+0000 7f454ceb0700 1 --2- 172.21.15.40:0/2735916674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45480c6d80 0x7f4548155a50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4538009e10 tx=0x7f453800d540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2222873c892fb34d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:28.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.161+0000 7f4547fff700 1 -- 172.21.15.40:0/2735916674 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538011460 con 0x7f45480c6d80 2024-09-13T17:50:28.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.161+0000 7f4547fff700 1 -- 172.21.15.40:0/2735916674 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45380115c0 con 0x7f45480c6d80 2024-09-13T17:50:28.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.161+0000 7f4547fff700 1 -- 172.21.15.40:0/2735916674 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538011970 con 0x7f45480c6d80 2024-09-13T17:50:28.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.165+0000 7f454f915700 1 -- 172.21.15.40:0/2735916674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45480c6d80 msgr2=0x7f4548155a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:28.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.165+0000 7f454f915700 1 --2- 172.21.15.40:0/2735916674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45480c6d80 0x7f4548155a50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4538009e10 tx=0x7f453800d540 comp rx=0 tx=0).stop 2024-09-13T17:50:28.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.165+0000 7f454f915700 1 -- 172.21.15.40:0/2735916674 shutdown_connections 2024-09-13T17:50:28.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.165+0000 7f454f915700 1 --2- 172.21.15.40:0/2735916674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f45480cb520 0x7f4548155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.165+0000 7f454f915700 1 --2- 172.21.15.40:0/2735916674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45480c6d80 0x7f4548155a50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.165+0000 7f454f915700 1 -- 172.21.15.40:0/2735916674 >> 172.21.15.40:0/2735916674 conn(0x7f454806a390 msgr2=0x7f454806a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:28.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.165+0000 7f454f915700 1 -- 172.21.15.40:0/2735916674 shutdown_connections 2024-09-13T17:50:28.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.165+0000 7f454f915700 1 -- 172.21.15.40:0/2735916674 wait complete. 2024-09-13T17:50:28.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.165+0000 7f454f915700 1 Processor -- start 2024-09-13T17:50:28.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454f915700 1 -- start start 2024-09-13T17:50:28.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454f915700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45480c69a0 0x7f45481e1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:28.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454f915700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45480c6d80 0x7f45480c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:28.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454f915700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f45480cb520 0x7f45481e2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:28.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454f915700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45480703e0 con 0x7f45480c69a0 2024-09-13T17:50:28.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454f915700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45480c3b70 con 0x7f45480c6d80 2024-09-13T17:50:28.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454f915700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45480c3e50 con 0x7f45480cb520 2024-09-13T17:50:28.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454d6b1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45480c69a0 0x7f45481e1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454d6b1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45480c69a0 0x7f45481e1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35292/0 (socket says 172.21.15.40:35292) 2024-09-13T17:50:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454d6b1700 1 -- 172.21.15.40:0/568049640 learned_addr learned my addr 172.21.15.40:0/568049640 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454ceb0700 1 --2- 172.21.15.40:0/568049640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45480c6d80 0x7f45480c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454deb2700 1 --2- 172.21.15.40:0/568049640 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f45480cb520 0x7f45481e2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454d6b1700 1 -- 172.21.15.40:0/568049640 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f45480cb520 msgr2=0x7f45481e2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454d6b1700 1 --2- 172.21.15.40:0/568049640 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f45480cb520 0x7f45481e2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454d6b1700 1 -- 172.21.15.40:0/568049640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45480c6d80 msgr2=0x7f45480c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454d6b1700 1 --2- 172.21.15.40:0/568049640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45480c6d80 0x7f45480c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454d6b1700 1 -- 172.21.15.40:0/568049640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454000a560 con 0x7f45480c69a0 2024-09-13T17:50:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454ceb0700 1 --2- 172.21.15.40:0/568049640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45480c6d80 0x7f45480c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:50:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454d6b1700 1 --2- 172.21.15.40:0/568049640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45480c69a0 0x7f45481e1ac0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f453c00c700 tx=0x7f453c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f4545ffb700 1 -- 172.21.15.40:0/568049640 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c005d90 con 0x7f45480c69a0 2024-09-13T17:50:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f4545ffb700 1 -- 172.21.15.40:0/568049640 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453c0070d0 con 0x7f45480c69a0 2024-09-13T17:50:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454f915700 1 -- 172.21.15.40:0/568049640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4538009a70 con 0x7f45480c69a0 2024-09-13T17:50:28.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f4545ffb700 1 -- 172.21.15.40:0/568049640 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c007480 con 0x7f45480c69a0 2024-09-13T17:50:28.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.169+0000 7f454f915700 1 -- 172.21.15.40:0/568049640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45480c4410 con 0x7f45480c69a0 2024-09-13T17:50:28.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.173+0000 7f454f915700 1 -- 172.21.15.40:0/568049640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4548159940 con 0x7f45480c69a0 2024-09-13T17:50:28.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.173+0000 7f4545ffb700 1 -- 172.21.15.40:0/568049640 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f453c0075e0 con 0x7f45480c69a0 2024-09-13T17:50:28.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.173+0000 7f4545ffb700 1 --2- 172.21.15.40:0/568049640 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4534066730 0x7f4534068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:28.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.173+0000 7f454ceb0700 1 --2- 172.21.15.40:0/568049640 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4534066730 0x7f4534068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:28.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.173+0000 7f4545ffb700 1 -- 172.21.15.40:0/568049640 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f453c083360 con 0x7f45480c69a0 2024-09-13T17:50:28.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.173+0000 7f454ceb0700 1 --2- 172.21.15.40:0/568049640 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4534066730 0x7f4534068bf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f45480c7190 tx=0x7f453801b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:28.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.177+0000 7f4545ffb700 1 -- 172.21.15.40:0/568049640 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f453c050a20 con 0x7f45480c69a0 2024-09-13T17:50:28.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.345+0000 7f454f915700 1 -- 172.21.15.40:0/568049640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4548156f40 con 0x7f45480c69a0 2024-09-13T17:50:28.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.345+0000 7f4545ffb700 1 -- 172.21.15.40:0/568049640 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12766 (secure 0 0 0) 0x7f453c054040 con 0x7f45480c69a0 2024-09-13T17:50:28.353 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:50:28.353 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:50:23.070493+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:50:28.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.345+0000 7f454f915700 1 -- 172.21.15.40:0/568049640 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4534066730 msgr2=0x7f4534068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:28.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.345+0000 7f454f915700 1 --2- 172.21.15.40:0/568049640 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4534066730 0x7f4534068bf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f45480c7190 tx=0x7f453801b040 comp rx=0 tx=0).stop 2024-09-13T17:50:28.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.349+0000 7f454f915700 1 -- 172.21.15.40:0/568049640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45480c69a0 msgr2=0x7f45481e1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:28.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.349+0000 7f454f915700 1 --2- 172.21.15.40:0/568049640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45480c69a0 0x7f45481e1ac0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f453c00c700 tx=0x7f453c00ca10 comp rx=0 tx=0).stop 2024-09-13T17:50:28.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.349+0000 7f454f915700 1 -- 172.21.15.40:0/568049640 shutdown_connections 2024-09-13T17:50:28.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.349+0000 7f454f915700 1 --2- 172.21.15.40:0/568049640 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4534066730 0x7f4534068bf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.349+0000 7f454f915700 1 --2- 172.21.15.40:0/568049640 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f45480cb520 0x7f45481e2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.349+0000 7f454f915700 1 --2- 172.21.15.40:0/568049640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45480c6d80 0x7f45480c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.349+0000 7f454f915700 1 --2- 172.21.15.40:0/568049640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45480c69a0 0x7f45481e1ac0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.349+0000 7f454f915700 1 -- 172.21.15.40:0/568049640 >> 172.21.15.40:0/568049640 conn(0x7f454806a390 msgr2=0x7f4548145eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:28.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.349+0000 7f454f915700 1 -- 172.21.15.40:0/568049640 shutdown_connections 2024-09-13T17:50:28.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.349+0000 7f454f915700 1 -- 172.21.15.40:0/568049640 wait complete. 2024-09-13T17:50:28.367 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:50:21.057171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:50:28.367 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:50:28.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.481+0000 7f0f7b59e700 1 Processor -- start 2024-09-13T17:50:28.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.481+0000 7f0f7b59e700 1 -- start start 2024-09-13T17:50:28.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.481+0000 7f0f7b59e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0f7c0cf1a0 0x7f0f7c15a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:28.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.481+0000 7f0f7b59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f7c0cf580 0x7f0f7c0c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:28.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.481+0000 7f0f7b59e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f7c15a7a0 con 0x7f0f7c0cc250 2024-09-13T17:50:28.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.481+0000 7f0f7b59e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f7c0cbdc0 con 0x7f0f7c0cf1a0 2024-09-13T17:50:28.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.481+0000 7f0f7b59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c153f80 con 0x7f0f7c0cf580 2024-09-13T17:50:28.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.481+0000 7f0f7a59c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0f7c0cf1a0 0x7f0f7c15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:28.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.481+0000 7f0f79d9b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f7c0cf580 0x7f0f7c0c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:28.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.481+0000 7f0f7a59c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0f7c0cf1a0 0x7f0f7c15a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:46972/0 (socket says 172.21.15.40:46972) 2024-09-13T17:50:28.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.481+0000 7f0f7a59c700 1 -- 172.21.15.40:0/1532346321 learned_addr learned my addr 172.21.15.40:0/1532346321 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:28.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.481+0000 7f0f7ad9d700 1 --1- 172.21.15.40:0/1532346321 >> v1:172.21.15.43:6789/0 conn(0x7f0f7c0cc250 0x7f0f7c0c8aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:38290/0 (socket says 172.21.15.40:38290) 2024-09-13T17:50:28.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.485+0000 7f0f79d9b700 1 -- 172.21.15.40:0/1532346321 >> v2:172.21.15.43:3300/0 conn(0x7f0f7c0cf1a0 msgr2=0x7f0f7c15a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:28.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.485+0000 7f0f79d9b700 1 --2- 172.21.15.40:0/1532346321 >> v2:172.21.15.43:3300/0 conn(0x7f0f7c0cf1a0 0x7f0f7c15a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.485+0000 7f0f79d9b700 1 -- 172.21.15.40:0/1532346321 >> v1:172.21.15.43:6789/0 conn(0x7f0f7c0cc250 legacy=0x7f0f7c0c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:28.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.485+0000 7f0f79d9b700 1 -- 172.21.15.40:0/1532346321 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c0caed0 con 0x7f0f7c0cf580 2024-09-13T17:50:28.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.485+0000 7f0f7a59c700 1 --2- 172.21.15.40:0/1532346321 >> v2:172.21.15.43:3300/0 conn(0x7f0f7c0cf1a0 0x7f0f7c15a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:50:28.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.485+0000 7f0f79d9b700 1 --2- 172.21.15.40:0/1532346321 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f7c0cf580 0x7f0f7c0c9fd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009850 tx=0x7f0f6c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37180050351a0495 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:28.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.485+0000 7f0f7959a700 1 -- 172.21.15.40:0/1532346321 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c011830 con 0x7f0f7c0cf580 2024-09-13T17:50:28.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.485+0000 7f0f7959a700 1 -- 172.21.15.40:0/1532346321 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6c011990 con 0x7f0f7c0cf580 2024-09-13T17:50:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.485+0000 7f0f7959a700 1 -- 172.21.15.40:0/1532346321 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c011d40 con 0x7f0f7c0cf580 2024-09-13T17:50:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.485+0000 7f0f7b59e700 1 -- 172.21.15.40:0/1532346321 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f7c0cf580 msgr2=0x7f0f7c0c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.485+0000 7f0f7b59e700 1 --2- 172.21.15.40:0/1532346321 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f7c0cf580 0x7f0f7c0c9fd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009850 tx=0x7f0f6c009b60 comp rx=0 tx=0).stop 2024-09-13T17:50:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.485+0000 7f0f7b59e700 1 -- 172.21.15.40:0/1532346321 shutdown_connections 2024-09-13T17:50:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.485+0000 7f0f7b59e700 1 --2- 172.21.15.40:0/1532346321 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f7c0cf580 0x7f0f7c0c9fd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.485+0000 7f0f7b59e700 1 --2- 172.21.15.40:0/1532346321 >> v2:172.21.15.43:3300/0 conn(0x7f0f7c0cf1a0 0x7f0f7c15a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.485+0000 7f0f7b59e700 1 -- 172.21.15.40:0/1532346321 >> 172.21.15.40:0/1532346321 conn(0x7f0f7c066090 msgr2=0x7f0f7c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:28.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.485+0000 7f0f7b59e700 1 -- 172.21.15.40:0/1532346321 shutdown_connections 2024-09-13T17:50:28.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.485+0000 7f0f7b59e700 1 -- 172.21.15.40:0/1532346321 wait complete. 2024-09-13T17:50:28.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.489+0000 7f0f7b59e700 1 Processor -- start 2024-09-13T17:50:28.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.489+0000 7f0f7b59e700 1 -- start start 2024-09-13T17:50:28.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.489+0000 7f0f7b59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f7c0cc250 0x7f0f7c0c5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:28.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.489+0000 7f0f7b59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f7c0cf1a0 0x7f0f7c159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:28.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.489+0000 7f0f7b59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f7c0cf580 0x7f0f7c1f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:28.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.489+0000 7f0f7b59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c1e4870 con 0x7f0f7c0cf1a0 2024-09-13T17:50:28.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.489+0000 7f0f7b59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c15c540 con 0x7f0f7c0cf580 2024-09-13T17:50:28.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.489+0000 7f0f7b59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c15c2d0 con 0x7f0f7c0cc250 2024-09-13T17:50:28.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.489+0000 7f0f79d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f7c0cf1a0 0x7f0f7c159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:28.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.489+0000 7f0f79d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f7c0cf1a0 0x7f0f7c159240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35308/0 (socket says 172.21.15.40:35308) 2024-09-13T17:50:28.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.489+0000 7f0f79d9b700 1 -- 172.21.15.40:0/653568523 learned_addr learned my addr 172.21.15.40:0/653568523 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:28.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.489+0000 7f0f7ad9d700 1 --2- 172.21.15.40:0/653568523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f7c0cf580 0x7f0f7c1f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:28.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.489+0000 7f0f7a59c700 1 --2- 172.21.15.40:0/653568523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f7c0cc250 0x7f0f7c0c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:28.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.489+0000 7f0f79d9b700 1 -- 172.21.15.40:0/653568523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f7c0cc250 msgr2=0x7f0f7c0c5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:28.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.489+0000 7f0f79d9b700 1 --2- 172.21.15.40:0/653568523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f7c0cc250 0x7f0f7c0c5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.489+0000 7f0f79d9b700 1 -- 172.21.15.40:0/653568523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f7c0cf580 msgr2=0x7f0f7c1f1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:28.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.489+0000 7f0f79d9b700 1 --2- 172.21.15.40:0/653568523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f7c0cf580 0x7f0f7c1f1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.489+0000 7f0f79d9b700 1 -- 172.21.15.40:0/653568523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f70009570 con 0x7f0f7c0cf1a0 2024-09-13T17:50:28.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.489+0000 7f0f79d9b700 1 --2- 172.21.15.40:0/653568523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f7c0cf1a0 0x7f0f7c159240 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c0090b0 tx=0x7f0f6c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:28.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.489+0000 7f0f6affd700 1 -- 172.21.15.40:0/653568523 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c004610 con 0x7f0f7c0cf1a0 2024-09-13T17:50:28.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.493+0000 7f0f6affd700 1 -- 172.21.15.40:0/653568523 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6c01e070 con 0x7f0f7c0cf1a0 2024-09-13T17:50:28.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.493+0000 7f0f6affd700 1 -- 172.21.15.40:0/653568523 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c003920 con 0x7f0f7c0cf1a0 2024-09-13T17:50:28.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.493+0000 7f0f7b59e700 1 -- 172.21.15.40:0/653568523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f6c0094b0 con 0x7f0f7c0cf1a0 2024-09-13T17:50:28.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.493+0000 7f0f7b59e700 1 -- 172.21.15.40:0/653568523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f7c1e4500 con 0x7f0f7c0cf1a0 2024-09-13T17:50:28.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.493+0000 7f0f6affd700 1 -- 172.21.15.40:0/653568523 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0f6c003a80 con 0x7f0f7c0cf1a0 2024-09-13T17:50:28.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.493+0000 7f0f7b59e700 1 -- 172.21.15.40:0/653568523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f7c067a40 con 0x7f0f7c0cf1a0 2024-09-13T17:50:28.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.493+0000 7f0f6affd700 1 --2- 172.21.15.40:0/653568523 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0f5c066760 0x7f0f5c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:28.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.497+0000 7f0f6affd700 1 -- 172.21.15.40:0/653568523 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0f6c020070 con 0x7f0f7c0cf1a0 2024-09-13T17:50:28.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.497+0000 7f0f7a59c700 1 --2- 172.21.15.40:0/653568523 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0f5c066760 0x7f0f5c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:28.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.497+0000 7f0f7a59c700 1 --2- 172.21.15.40:0/653568523 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0f5c066760 0x7f0f5c068c20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0f70006d50 tx=0x7f0f700092e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:28.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.505+0000 7f0f6affd700 1 -- 172.21.15.40:0/653568523 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0f6c052580 con 0x7f0f7c0cf1a0 2024-09-13T17:50:28.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.677+0000 7f0f7b59e700 1 -- 172.21.15.40:0/653568523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0f7c1e4330 con 0x7f0f7c0cf1a0 2024-09-13T17:50:28.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.677+0000 7f0f6affd700 1 -- 172.21.15.40:0/653568523 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0f6c055ba0 con 0x7f0f7c0cf1a0 2024-09-13T17:50:28.685 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-09-13T17:50:28.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.681+0000 7f0f7b59e700 1 -- 172.21.15.40:0/653568523 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0f5c066760 msgr2=0x7f0f5c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:28.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.681+0000 7f0f7b59e700 1 --2- 172.21.15.40:0/653568523 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0f5c066760 0x7f0f5c068c20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0f70006d50 tx=0x7f0f700092e0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.681+0000 7f0f7b59e700 1 -- 172.21.15.40:0/653568523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f7c0cf1a0 msgr2=0x7f0f7c159240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:28.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.681+0000 7f0f7b59e700 1 --2- 172.21.15.40:0/653568523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f7c0cf1a0 0x7f0f7c159240 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c0090b0 tx=0x7f0f6c004770 comp rx=0 tx=0).stop 2024-09-13T17:50:28.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.681+0000 7f0f7b59e700 1 -- 172.21.15.40:0/653568523 shutdown_connections 2024-09-13T17:50:28.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.681+0000 7f0f7b59e700 1 --2- 172.21.15.40:0/653568523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f7c0cf580 0x7f0f7c1f1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.681+0000 7f0f7b59e700 1 --2- 172.21.15.40:0/653568523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f7c0cf1a0 0x7f0f7c159240 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.681+0000 7f0f7b59e700 1 --2- 172.21.15.40:0/653568523 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0f5c066760 0x7f0f5c068c20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.681+0000 7f0f7b59e700 1 --2- 172.21.15.40:0/653568523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f7c0cc250 0x7f0f7c0c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.681+0000 7f0f7b59e700 1 -- 172.21.15.40:0/653568523 >> 172.21.15.40:0/653568523 conn(0x7f0f7c066090 msgr2=0x7f0f7c0b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:28.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.681+0000 7f0f7b59e700 1 -- 172.21.15.40:0/653568523 shutdown_connections 2024-09-13T17:50:28.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.681+0000 7f0f7b59e700 1 -- 172.21.15.40:0/653568523 wait complete. 2024-09-13T17:50:28.700 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:50:28.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66dce06700 1 Processor -- start 2024-09-13T17:50:28.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66dce06700 1 -- start start 2024-09-13T17:50:28.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66dce06700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f66d8070500 0x7f66d8071b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:28.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66dce06700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66d8072080 0x7f66d81608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:28.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66dce06700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d80cba20 con 0x7f66d81500c0 2024-09-13T17:50:28.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66dce06700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66d80724f0 con 0x7f66d8070500 2024-09-13T17:50:28.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66dce06700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d81560e0 con 0x7f66d8072080 2024-09-13T17:50:28.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66d6ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f66d8070500 0x7f66d8071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:28.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66d6ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f66d8070500 0x7f66d8071b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:46988/0 (socket says 172.21.15.40:46988) 2024-09-13T17:50:28.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66d6ffd700 1 -- 172.21.15.40:0/3592843354 learned_addr learned my addr 172.21.15.40:0/3592843354 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:28.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66d7fff700 1 --2- 172.21.15.40:0/3592843354 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66d8072080 0x7f66d81608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:28.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66d77fe700 1 --1- 172.21.15.40:0/3592843354 >> v1:172.21.15.43:6789/0 conn(0x7f66d81500c0 0x7f66d80c66d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:38292/0 (socket says 172.21.15.40:38292) 2024-09-13T17:50:28.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66d6ffd700 1 -- 172.21.15.40:0/3592843354 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66d8072080 msgr2=0x7f66d81608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:28.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66d6ffd700 1 --2- 172.21.15.40:0/3592843354 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66d8072080 0x7f66d81608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66d6ffd700 1 -- 172.21.15.40:0/3592843354 >> v1:172.21.15.43:6789/0 conn(0x7f66d81500c0 legacy=0x7f66d80c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:28.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66d6ffd700 1 -- 172.21.15.40:0/3592843354 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d81563c0 con 0x7f66d8070500 2024-09-13T17:50:28.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66d6ffd700 1 --2- 172.21.15.40:0/3592843354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66d8070500 0x7f66d8071b40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f66c8009e50 tx=0x7f66c800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd0f0e7adac5efac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:28.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66d67fc700 1 -- 172.21.15.40:0/3592843354 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c800e070 con 0x7f66d8070500 2024-09-13T17:50:28.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66d67fc700 1 -- 172.21.15.40:0/3592843354 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c800b9c0 con 0x7f66d8070500 2024-09-13T17:50:28.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66d67fc700 1 -- 172.21.15.40:0/3592843354 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c800bd70 con 0x7f66d8070500 2024-09-13T17:50:28.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66dce06700 1 -- 172.21.15.40:0/3592843354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66d8070500 msgr2=0x7f66d8071b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:28.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66dce06700 1 --2- 172.21.15.40:0/3592843354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66d8070500 0x7f66d8071b40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f66c8009e50 tx=0x7f66c800b330 comp rx=0 tx=0).stop 2024-09-13T17:50:28.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66dce06700 1 -- 172.21.15.40:0/3592843354 shutdown_connections 2024-09-13T17:50:28.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66dce06700 1 --2- 172.21.15.40:0/3592843354 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66d8072080 0x7f66d81608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66dce06700 1 --2- 172.21.15.40:0/3592843354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66d8070500 0x7f66d8071b40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66dce06700 1 -- 172.21.15.40:0/3592843354 >> 172.21.15.40:0/3592843354 conn(0x7f66d80cabc0 msgr2=0x7f66d80cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:28.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66dce06700 1 -- 172.21.15.40:0/3592843354 shutdown_connections 2024-09-13T17:50:28.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.805+0000 7f66dce06700 1 -- 172.21.15.40:0/3592843354 wait complete. 2024-09-13T17:50:28.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66dce06700 1 Processor -- start 2024-09-13T17:50:28.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66dce06700 1 -- start start 2024-09-13T17:50:28.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66dce06700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66d8070500 0x7f66d81492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:28.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66dce06700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66d8072080 0x7f66d814db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:28.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66dce06700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d81500c0 0x7f66d814e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:28.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66dce06700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d814aa60 con 0x7f66d81500c0 2024-09-13T17:50:28.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66dce06700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d814b8f0 con 0x7f66d8070500 2024-09-13T17:50:28.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66dce06700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d8162840 con 0x7f66d8072080 2024-09-13T17:50:28.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66d7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d81500c0 0x7f66d814e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:28.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66d7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d81500c0 0x7f66d814e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35316/0 (socket says 172.21.15.40:35316) 2024-09-13T17:50:28.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66d7fff700 1 -- 172.21.15.40:0/3926969898 learned_addr learned my addr 172.21.15.40:0/3926969898 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:28.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66d77fe700 1 --2- 172.21.15.40:0/3926969898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66d8070500 0x7f66d81492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:28.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66d6ffd700 1 --2- 172.21.15.40:0/3926969898 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66d8072080 0x7f66d814db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:28.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66d7fff700 1 -- 172.21.15.40:0/3926969898 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66d8072080 msgr2=0x7f66d814db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:28.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66d7fff700 1 --2- 172.21.15.40:0/3926969898 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66d8072080 0x7f66d814db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66d7fff700 1 -- 172.21.15.40:0/3926969898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66d8070500 msgr2=0x7f66d81492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:28.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66d7fff700 1 --2- 172.21.15.40:0/3926969898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66d8070500 0x7f66d81492f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:28.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66d7fff700 1 -- 172.21.15.40:0/3926969898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c8009b30 con 0x7f66d81500c0 2024-09-13T17:50:28.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66d7fff700 1 --2- 172.21.15.40:0/3926969898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d81500c0 0x7f66d814e050 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f66d000e810 tx=0x7f66d000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:28.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66cffff700 1 -- 172.21.15.40:0/3926969898 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d000c070 con 0x7f66d81500c0 2024-09-13T17:50:28.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66cffff700 1 -- 172.21.15.40:0/3926969898 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d0013610 con 0x7f66d81500c0 2024-09-13T17:50:28.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66dce06700 1 -- 172.21.15.40:0/3926969898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d81633c0 con 0x7f66d81500c0 2024-09-13T17:50:28.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66cffff700 1 -- 172.21.15.40:0/3926969898 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d00139c0 con 0x7f66d81500c0 2024-09-13T17:50:28.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66dce06700 1 -- 172.21.15.40:0/3926969898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d8163850 con 0x7f66d81500c0 2024-09-13T17:50:28.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66dce06700 1 -- 172.21.15.40:0/3926969898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d814fc00 con 0x7f66d81500c0 2024-09-13T17:50:28.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66cffff700 1 -- 172.21.15.40:0/3926969898 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f66d0013b20 con 0x7f66d81500c0 2024-09-13T17:50:28.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66cffff700 1 --2- 172.21.15.40:0/3926969898 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f66b8066760 0x7f66b8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:28.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66cffff700 1 -- 172.21.15.40:0/3926969898 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f66d00838f0 con 0x7f66d81500c0 2024-09-13T17:50:28.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66d77fe700 1 --2- 172.21.15.40:0/3926969898 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f66b8066760 0x7f66b8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:28.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.813+0000 7f66d77fe700 1 --2- 172.21.15.40:0/3926969898 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f66b8066760 0x7f66b8068c20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f66c0000f60 tx=0x7f66c0007ef0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:28.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:28.825+0000 7f66cffff700 1 -- 172.21.15.40:0/3926969898 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f66d0051040 con 0x7f66d81500c0 2024-09-13T17:50:29.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:29.029+0000 7f66dce06700 1 -- 172.21.15.40:0/3926969898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66d8163550 con 0x7f66d81500c0 2024-09-13T17:50:29.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:29.033+0000 7f66cffff700 1 -- 172.21.15.40:0/3926969898 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1253 (secure 0 0 0) 0x7f66d0017090 con 0x7f66d81500c0 2024-09-13T17:50:29.039 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:50:29.039 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/486611242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":486611242},{"type":"v1","addr":"172.21.15.40:6837","nonce":486611242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-09-13T17:50:29.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:29.033+0000 7f66dce06700 1 -- 172.21.15.40:0/3926969898 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f66b8066760 msgr2=0x7f66b8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:29.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:29.033+0000 7f66dce06700 1 --2- 172.21.15.40:0/3926969898 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f66b8066760 0x7f66b8068c20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f66c0000f60 tx=0x7f66c0007ef0 comp rx=0 tx=0).stop 2024-09-13T17:50:29.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:29.033+0000 7f66dce06700 1 -- 172.21.15.40:0/3926969898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d81500c0 msgr2=0x7f66d814e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:29.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:29.033+0000 7f66dce06700 1 --2- 172.21.15.40:0/3926969898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d81500c0 0x7f66d814e050 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f66d000e810 tx=0x7f66d000eb20 comp rx=0 tx=0).stop 2024-09-13T17:50:29.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:29.033+0000 7f66dce06700 1 -- 172.21.15.40:0/3926969898 shutdown_connections 2024-09-13T17:50:29.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:29.033+0000 7f66dce06700 1 --2- 172.21.15.40:0/3926969898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66d81500c0 0x7f66d814e050 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:29.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:29.033+0000 7f66dce06700 1 --2- 172.21.15.40:0/3926969898 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f66b8066760 0x7f66b8068c20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:29.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:29.033+0000 7f66dce06700 1 --2- 172.21.15.40:0/3926969898 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66d8072080 0x7f66d814db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:29.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:29.033+0000 7f66dce06700 1 --2- 172.21.15.40:0/3926969898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66d8070500 0x7f66d81492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:29.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:29.033+0000 7f66dce06700 1 -- 172.21.15.40:0/3926969898 >> 172.21.15.40:0/3926969898 conn(0x7f66d80cabc0 msgr2=0x7f66d80c61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:29.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:29.033+0000 7f66dce06700 1 -- 172.21.15.40:0/3926969898 shutdown_connections 2024-09-13T17:50:29.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:29.033+0000 7f66dce06700 1 -- 172.21.15.40:0/3926969898 wait complete. 2024-09-13T17:50:29.043 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 2 2024-09-13T17:50:31.055 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:50:31.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82f5c4b700 1 Processor -- start 2024-09-13T17:50:31.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82f5c4b700 1 -- start start 2024-09-13T17:50:31.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82f5c4b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f82f006faf0 0x7f82f0159f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:31.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82f5c4b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f82f00709a0 0x7f82f006ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:31.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82f5c4b700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f0070df0 con 0x7f82f01535f0 2024-09-13T17:50:31.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82f5c4b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f0159b50 con 0x7f82f00709a0 2024-09-13T17:50:31.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82f5c4b700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f006ff30 con 0x7f82f006faf0 2024-09-13T17:50:31.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82f4c49700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f82f01535f0 0x7f82f00c6960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48332/0 (socket says 172.21.15.40:48332) 2024-09-13T17:50:31.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82f4c49700 1 -- 172.21.15.40:0/3935610356 learned_addr learned my addr 172.21.15.40:0/3935610356 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:31.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82f544a700 1 --2- 172.21.15.40:0/3935610356 >> v2:172.21.15.40:3300/0 conn(0x7f82f00709a0 0x7f82f006ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:31.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82effff700 1 --2- 172.21.15.40:0/3935610356 >> v2:172.21.15.43:3300/0 conn(0x7f82f006faf0 0x7f82f0159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82ef7fe700 1 -- 172.21.15.40:0/3935610356 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2231557352 0 0) 0x7f82e4001110 con 0x7f82f01535f0 2024-09-13T17:50:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82ef7fe700 1 -- 172.21.15.40:0/3935610356 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82e00035d0 con 0x7f82f01535f0 2024-09-13T17:50:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82f544a700 1 -- 172.21.15.40:0/3935610356 >> v2:172.21.15.43:3300/0 conn(0x7f82f006faf0 msgr2=0x7f82f0159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82f544a700 1 --2- 172.21.15.40:0/3935610356 >> v2:172.21.15.43:3300/0 conn(0x7f82f006faf0 0x7f82f0159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82f544a700 1 -- 172.21.15.40:0/3935610356 >> v1:172.21.15.40:6789/0 conn(0x7f82f01535f0 legacy=0x7f82f00c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82f544a700 1 -- 172.21.15.40:0/3935610356 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f00c1db0 con 0x7f82f00709a0 2024-09-13T17:50:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82effff700 1 --2- 172.21.15.40:0/3935610356 >> v2:172.21.15.43:3300/0 conn(0x7f82f006faf0 0x7f82f0159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:50:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82f544a700 1 --2- 172.21.15.40:0/3935610356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f00709a0 0x7f82f006ad50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f82f006f500 tx=0x7f82e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157124d7188f448 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:31.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82ef7fe700 1 -- 172.21.15.40:0/3935610356 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e800e070 con 0x7f82f00709a0 2024-09-13T17:50:31.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82ef7fe700 1 -- 172.21.15.40:0/3935610356 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e800bb20 con 0x7f82f00709a0 2024-09-13T17:50:31.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82ef7fe700 1 -- 172.21.15.40:0/3935610356 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8012430 con 0x7f82f00709a0 2024-09-13T17:50:31.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82f5c4b700 1 -- 172.21.15.40:0/3935610356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f00709a0 msgr2=0x7f82f006ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:31.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.173+0000 7f82f5c4b700 1 --2- 172.21.15.40:0/3935610356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f00709a0 0x7f82f006ad50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f82f006f500 tx=0x7f82e800b330 comp rx=0 tx=0).stop 2024-09-13T17:50:31.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.177+0000 7f82f5c4b700 1 -- 172.21.15.40:0/3935610356 shutdown_connections 2024-09-13T17:50:31.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.177+0000 7f82f5c4b700 1 --2- 172.21.15.40:0/3935610356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f00709a0 0x7f82f006ad50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.177+0000 7f82f5c4b700 1 --2- 172.21.15.40:0/3935610356 >> v2:172.21.15.43:3300/0 conn(0x7f82f006faf0 0x7f82f0159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.177+0000 7f82f5c4b700 1 -- 172.21.15.40:0/3935610356 >> 172.21.15.40:0/3935610356 conn(0x7f82f00cabc0 msgr2=0x7f82f00cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:31.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.177+0000 7f82f5c4b700 1 -- 172.21.15.40:0/3935610356 shutdown_connections 2024-09-13T17:50:31.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.177+0000 7f82f5c4b700 1 -- 172.21.15.40:0/3935610356 wait complete. 2024-09-13T17:50:31.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.177+0000 7f82f5c4b700 1 Processor -- start 2024-09-13T17:50:31.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.177+0000 7f82f5c4b700 1 -- start start 2024-09-13T17:50:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.177+0000 7f82f5c4b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82f006faf0 0x7f82f01efe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.177+0000 7f82f5c4b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f00709a0 0x7f82f014c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.177+0000 7f82f5c4b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82f01535f0 0x7f82f015c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.177+0000 7f82f5c4b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f0151c30 con 0x7f82f00709a0 2024-09-13T17:50:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.177+0000 7f82f5c4b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f00c64b0 con 0x7f82f01535f0 2024-09-13T17:50:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.177+0000 7f82f5c4b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f01f03a0 con 0x7f82f006faf0 2024-09-13T17:50:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.177+0000 7f82effff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f00709a0 0x7f82f014c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.177+0000 7f82effff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f00709a0 0x7f82f014c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35340/0 (socket says 172.21.15.40:35340) 2024-09-13T17:50:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.177+0000 7f82effff700 1 -- 172.21.15.40:0/3844537265 learned_addr learned my addr 172.21.15.40:0/3844537265 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.177+0000 7f82f4c49700 1 --2- 172.21.15.40:0/3844537265 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82f006faf0 0x7f82f01efe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:31.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.177+0000 7f82f544a700 1 --2- 172.21.15.40:0/3844537265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82f01535f0 0x7f82f015c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:31.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.181+0000 7f82effff700 1 -- 172.21.15.40:0/3844537265 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82f006faf0 msgr2=0x7f82f01efe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:31.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.181+0000 7f82effff700 1 --2- 172.21.15.40:0/3844537265 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82f006faf0 0x7f82f01efe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.181+0000 7f82effff700 1 -- 172.21.15.40:0/3844537265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82f01535f0 msgr2=0x7f82f015c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:31.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.181+0000 7f82effff700 1 --2- 172.21.15.40:0/3844537265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82f01535f0 0x7f82f015c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.181+0000 7f82effff700 1 -- 172.21.15.40:0/3844537265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e8009a70 con 0x7f82f00709a0 2024-09-13T17:50:31.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.181+0000 7f82effff700 1 --2- 172.21.15.40:0/3844537265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f00709a0 0x7f82f014c740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00e790 tx=0x7f82dc00eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:31.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.181+0000 7f82ed7fa700 1 -- 172.21.15.40:0/3844537265 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc00c070 con 0x7f82f00709a0 2024-09-13T17:50:31.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.181+0000 7f82ed7fa700 1 -- 172.21.15.40:0/3844537265 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82dc004230 con 0x7f82f00709a0 2024-09-13T17:50:31.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.181+0000 7f82f5c4b700 1 -- 172.21.15.40:0/3844537265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f00c66e0 con 0x7f82f00709a0 2024-09-13T17:50:31.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.181+0000 7f82ed7fa700 1 -- 172.21.15.40:0/3844537265 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0045c0 con 0x7f82f00709a0 2024-09-13T17:50:31.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.181+0000 7f82f5c4b700 1 -- 172.21.15.40:0/3844537265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f00c6ca0 con 0x7f82f00709a0 2024-09-13T17:50:31.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.181+0000 7f82f5c4b700 1 -- 172.21.15.40:0/3844537265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f0159770 con 0x7f82f00709a0 2024-09-13T17:50:31.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.181+0000 7f82ed7fa700 1 -- 172.21.15.40:0/3844537265 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f82dc0036c0 con 0x7f82f00709a0 2024-09-13T17:50:31.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.181+0000 7f82ed7fa700 1 --2- 172.21.15.40:0/3844537265 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f82d4066730 0x7f82d4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:31.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.181+0000 7f82ed7fa700 1 -- 172.21.15.40:0/3844537265 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f82dc083ba0 con 0x7f82f00709a0 2024-09-13T17:50:31.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.185+0000 7f82f4c49700 1 --2- 172.21.15.40:0/3844537265 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f82d4066730 0x7f82d4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:31.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.185+0000 7f82f4c49700 1 --2- 172.21.15.40:0/3844537265 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f82d4066730 0x7f82d4068bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f82e4007fd0 tx=0x7f82e4007c60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:31.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.189+0000 7f82ed7fa700 1 -- 172.21.15.40:0/3844537265 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f82dc051150 con 0x7f82f00709a0 2024-09-13T17:50:31.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.401+0000 7f82f5c4b700 1 -- 172.21.15.40:0/3844537265 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82f014f870 con 0x7f82f00709a0 2024-09-13T17:50:31.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.401+0000 7f82ed7fa700 1 -- 172.21.15.40:0/3844537265 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+5027 (secure 0 0 0) 0x7f82dc0094e0 con 0x7f82f00709a0 2024-09-13T17:50:31.410 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:50:31.410 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6839/1861594206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1861594206},{"type":"v1","addr":"172.21.15.40:6839","nonce":1861594206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1548394691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1548394691},{"type":"v1","addr":"172.21.15.40:6835","nonce":1548394691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/486611242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":486611242},{"type":"v1","addr":"172.21.15.40:6837","nonce":486611242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4279,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/526975022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":526975022},{"type":"v1","addr":"172.21.15.43:6833","nonce":526975022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2586910944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2586910944},{"type":"v1","addr":"172.21.15.43:6835","nonce":2586910944}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/334160033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":334160033},{"type":"v1","addr":"172.21.15.43:6837","nonce":334160033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-09-13T17:50:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.405+0000 7f82f5c4b700 1 -- 172.21.15.40:0/3844537265 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f82d4066730 msgr2=0x7f82d4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.405+0000 7f82f5c4b700 1 --2- 172.21.15.40:0/3844537265 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f82d4066730 0x7f82d4068bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f82e4007fd0 tx=0x7f82e4007c60 comp rx=0 tx=0).stop 2024-09-13T17:50:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.405+0000 7f82f5c4b700 1 -- 172.21.15.40:0/3844537265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f00709a0 msgr2=0x7f82f014c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.405+0000 7f82f5c4b700 1 --2- 172.21.15.40:0/3844537265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f00709a0 0x7f82f014c740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00e790 tx=0x7f82dc00eaa0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.405+0000 7f82f5c4b700 1 -- 172.21.15.40:0/3844537265 shutdown_connections 2024-09-13T17:50:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.405+0000 7f82f5c4b700 1 --2- 172.21.15.40:0/3844537265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82f01535f0 0x7f82f015c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.405+0000 7f82f5c4b700 1 --2- 172.21.15.40:0/3844537265 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82f00709a0 0x7f82f014c740 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.405+0000 7f82f5c4b700 1 --2- 172.21.15.40:0/3844537265 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f82d4066730 0x7f82d4068bf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.405+0000 7f82f5c4b700 1 --2- 172.21.15.40:0/3844537265 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82f006faf0 0x7f82f01efe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.405+0000 7f82f5c4b700 1 -- 172.21.15.40:0/3844537265 >> 172.21.15.40:0/3844537265 conn(0x7f82f00cabc0 msgr2=0x7f82f0151ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.405+0000 7f82f5c4b700 1 -- 172.21.15.40:0/3844537265 shutdown_connections 2024-09-13T17:50:31.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.405+0000 7f82f5c4b700 1 -- 172.21.15.40:0/3844537265 wait complete. 2024-09-13T17:50:31.414 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 3 2024-09-13T17:50:31.429 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:50:31.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.533+0000 7f0854933700 1 Processor -- start 2024-09-13T17:50:31.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.533+0000 7f0854933700 1 -- start start 2024-09-13T17:50:31.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.533+0000 7f0854933700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0850153800 0x7f085015c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:31.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.533+0000 7f0854933700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f08500c9d90 0x7f08500cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:31.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.533+0000 7f0854933700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0850153be0 con 0x7f08500ca190 2024-09-13T17:50:31.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f0854933700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f085015cf00 con 0x7f08500c9d90 2024-09-13T17:50:31.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f0854933700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08500ca770 con 0x7f0850153800 2024-09-13T17:50:31.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f084effd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f08500c9d90 0x7f08500cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:31.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f084effd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f08500c9d90 0x7f08500cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:55314/0 (socket says 172.21.15.40:55314) 2024-09-13T17:50:31.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f084effd700 1 -- 172.21.15.40:0/2288759828 learned_addr learned my addr 172.21.15.40:0/2288759828 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:31.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f084f7fe700 1 --2- 172.21.15.40:0/2288759828 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0850153800 0x7f085015c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:31.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f084e7fc700 1 -- 172.21.15.40:0/2288759828 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 300690238 0 0) 0x7f0850153be0 con 0x7f08500ca190 2024-09-13T17:50:31.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f084e7fc700 1 -- 172.21.15.40:0/2288759828 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08380035d0 con 0x7f08500ca190 2024-09-13T17:50:31.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f084effd700 1 -- 172.21.15.40:0/2288759828 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0850153800 msgr2=0x7f085015c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:31.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f084effd700 1 --2- 172.21.15.40:0/2288759828 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0850153800 0x7f085015c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f084effd700 1 -- 172.21.15.40:0/2288759828 >> v1:172.21.15.43:6789/0 conn(0x7f08500ca190 legacy=0x7f08500c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:31.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f084effd700 1 -- 172.21.15.40:0/2288759828 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085015d100 con 0x7f08500c9d90 2024-09-13T17:50:31.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f084effd700 1 --2- 172.21.15.40:0/2288759828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08500c9d90 0x7f08500cee80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0840009910 tx=0x7f0840009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71e9134bb240f369 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:31.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f084e7fc700 1 -- 172.21.15.40:0/2288759828 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0840011830 con 0x7f08500c9d90 2024-09-13T17:50:31.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f084e7fc700 1 -- 172.21.15.40:0/2288759828 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0840011990 con 0x7f08500c9d90 2024-09-13T17:50:31.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f084e7fc700 1 -- 172.21.15.40:0/2288759828 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0840011d40 con 0x7f08500c9d90 2024-09-13T17:50:31.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f0854933700 1 -- 172.21.15.40:0/2288759828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08500c9d90 msgr2=0x7f08500cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:31.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f0854933700 1 --2- 172.21.15.40:0/2288759828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08500c9d90 0x7f08500cee80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0840009910 tx=0x7f0840009c20 comp rx=0 tx=0).stop 2024-09-13T17:50:31.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f0854933700 1 -- 172.21.15.40:0/2288759828 shutdown_connections 2024-09-13T17:50:31.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f0854933700 1 --2- 172.21.15.40:0/2288759828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08500c9d90 0x7f08500cee80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f0854933700 1 --2- 172.21.15.40:0/2288759828 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0850153800 0x7f085015c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f0854933700 1 -- 172.21.15.40:0/2288759828 >> 172.21.15.40:0/2288759828 conn(0x7f0850066090 msgr2=0x7f08500664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:31.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f0854933700 1 -- 172.21.15.40:0/2288759828 shutdown_connections 2024-09-13T17:50:31.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.537+0000 7f0854933700 1 -- 172.21.15.40:0/2288759828 wait complete. 2024-09-13T17:50:31.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f0854933700 1 Processor -- start 2024-09-13T17:50:31.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f0854933700 1 -- start start 2024-09-13T17:50:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f0854933700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08500c9d90 0x7f08501f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f0854933700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f08500ca190 0x7f08501ef690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f0854933700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0850153800 0x7f0850155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f0854933700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085015bcc0 con 0x7f0850153800 2024-09-13T17:50:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f0854933700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085015a5b0 con 0x7f08500c9d90 2024-09-13T17:50:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f0854933700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08501eebd0 con 0x7f08500ca190 2024-09-13T17:50:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f084ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0850153800 0x7f0850155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f084ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0850153800 0x7f0850155eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38238/0 (socket says 172.21.15.40:38238) 2024-09-13T17:50:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f084ffff700 1 -- 172.21.15.40:0/2006052370 learned_addr learned my addr 172.21.15.40:0/2006052370 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:31.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f084effd700 1 --2- 172.21.15.40:0/2006052370 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f08500ca190 0x7f08501ef690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:31.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f084f7fe700 1 --2- 172.21.15.40:0/2006052370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08500c9d90 0x7f08501f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:31.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f084ffff700 1 -- 172.21.15.40:0/2006052370 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f08500ca190 msgr2=0x7f08501ef690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:31.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f084ffff700 1 --2- 172.21.15.40:0/2006052370 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f08500ca190 0x7f08501ef690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f084ffff700 1 -- 172.21.15.40:0/2006052370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08500c9d90 msgr2=0x7f08501f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:31.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f084ffff700 1 --2- 172.21.15.40:0/2006052370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08500c9d90 0x7f08501f2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f084ffff700 1 -- 172.21.15.40:0/2006052370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0840009570 con 0x7f0850153800 2024-09-13T17:50:31.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f084f7fe700 1 --2- 172.21.15.40:0/2006052370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08500c9d90 0x7f08501f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:50:31.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f084ffff700 1 --2- 172.21.15.40:0/2006052370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0850153800 0x7f0850155eb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f084800cc50 tx=0x7f084800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:31.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f0837fff700 1 -- 172.21.15.40:0/2006052370 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0848003c20 con 0x7f0850153800 2024-09-13T17:50:31.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f0837fff700 1 -- 172.21.15.40:0/2006052370 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0848003d80 con 0x7f0850153800 2024-09-13T17:50:31.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f0854933700 1 -- 172.21.15.40:0/2006052370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f085015a800 con 0x7f0850153800 2024-09-13T17:50:31.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f0837fff700 1 -- 172.21.15.40:0/2006052370 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084800ab60 con 0x7f0850153800 2024-09-13T17:50:31.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.541+0000 7f0854933700 1 -- 172.21.15.40:0/2006052370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08501eee90 con 0x7f0850153800 2024-09-13T17:50:31.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.545+0000 7f0854933700 1 -- 172.21.15.40:0/2006052370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f085015c9c0 con 0x7f0850153800 2024-09-13T17:50:31.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.545+0000 7f0837fff700 1 -- 172.21.15.40:0/2006052370 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f084800acc0 con 0x7f0850153800 2024-09-13T17:50:31.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.545+0000 7f0837fff700 1 --2- 172.21.15.40:0/2006052370 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f083c066760 0x7f083c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:31.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.545+0000 7f084f7fe700 1 --2- 172.21.15.40:0/2006052370 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f083c066760 0x7f083c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:31.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.545+0000 7f0837fff700 1 -- 172.21.15.40:0/2006052370 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0848083f20 con 0x7f0850153800 2024-09-13T17:50:31.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.549+0000 7f084f7fe700 1 --2- 172.21.15.40:0/2006052370 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f083c066760 0x7f083c068c20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0844009c40 tx=0x7f0844008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:31.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.553+0000 7f0837fff700 1 -- 172.21.15.40:0/2006052370 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0848051550 con 0x7f0850153800 2024-09-13T17:50:31.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.729+0000 7f0854933700 1 -- 172.21.15.40:0/2006052370 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08500c5ef0 con 0x7f0850153800 2024-09-13T17:50:31.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.729+0000 7f0837fff700 1 -- 172.21.15.40:0/2006052370 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12766 (secure 0 0 0) 0x7f0848054b70 con 0x7f0850153800 2024-09-13T17:50:31.738 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:50:31.738 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":17,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:50:23.070493+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:50:31.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.733+0000 7f0854933700 1 -- 172.21.15.40:0/2006052370 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f083c066760 msgr2=0x7f083c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:31.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.733+0000 7f0854933700 1 --2- 172.21.15.40:0/2006052370 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f083c066760 0x7f083c068c20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0844009c40 tx=0x7f0844008040 comp rx=0 tx=0).stop 2024-09-13T17:50:31.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.733+0000 7f0854933700 1 -- 172.21.15.40:0/2006052370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0850153800 msgr2=0x7f0850155eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:31.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.733+0000 7f0854933700 1 --2- 172.21.15.40:0/2006052370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0850153800 0x7f0850155eb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f084800cc50 tx=0x7f084800cf60 comp rx=0 tx=0).stop 2024-09-13T17:50:31.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.733+0000 7f0854933700 1 -- 172.21.15.40:0/2006052370 shutdown_connections 2024-09-13T17:50:31.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.733+0000 7f0854933700 1 --2- 172.21.15.40:0/2006052370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0850153800 0x7f0850155eb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.733+0000 7f0854933700 1 --2- 172.21.15.40:0/2006052370 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f083c066760 0x7f083c068c20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.733+0000 7f0854933700 1 --2- 172.21.15.40:0/2006052370 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f08500ca190 0x7f08501ef690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.733+0000 7f0854933700 1 --2- 172.21.15.40:0/2006052370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08500c9d90 0x7f08501f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.733+0000 7f0854933700 1 -- 172.21.15.40:0/2006052370 >> 172.21.15.40:0/2006052370 conn(0x7f0850066090 msgr2=0x7f0850066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:31.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.733+0000 7f0854933700 1 -- 172.21.15.40:0/2006052370 shutdown_connections 2024-09-13T17:50:31.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.733+0000 7f0854933700 1 -- 172.21.15.40:0/2006052370 wait complete. 2024-09-13T17:50:31.756 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:50:21.057171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:50:31.756 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:50:31.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.873+0000 7ff8a5671700 1 Processor -- start 2024-09-13T17:50:31.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.873+0000 7ff8a5671700 1 -- start start 2024-09-13T17:50:31.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.873+0000 7ff8a5671700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff8a0151050 0x7ff8a0066220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:31.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.873+0000 7ff8a5671700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff8a00cb2a0 0x7ff8a00cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:31.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.873+0000 7ff8a5671700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a01578b0 con 0x7ff8a0157d40 2024-09-13T17:50:31.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.873+0000 7ff8a5671700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8a0157090 con 0x7ff8a0151050 2024-09-13T17:50:31.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.873+0000 7ff8a5671700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a00c6630 con 0x7ff8a00cb2a0 2024-09-13T17:50:31.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.873+0000 7ff89f7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff8a0157d40 0x7ff8a00c2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35732/0 (socket says 172.21.15.40:35732) 2024-09-13T17:50:31.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.873+0000 7ff89f7fe700 1 -- 172.21.15.40:0/1051604757 learned_addr learned my addr 172.21.15.40:0/1051604757 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:31.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.873+0000 7ff89ffff700 1 --2- 172.21.15.40:0/1051604757 >> v2:172.21.15.43:3300/0 conn(0x7ff8a0151050 0x7ff8a0066220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:31.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.873+0000 7ff8a4e70700 1 --2- 172.21.15.40:0/1051604757 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff8a00cb2a0 0x7ff8a00cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:31.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.873+0000 7ff89effd700 1 -- 172.21.15.40:0/1051604757 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323975031 0 0) 0x7ff8a01578b0 con 0x7ff8a0157d40 2024-09-13T17:50:31.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.873+0000 7ff89effd700 1 -- 172.21.15.40:0/1051604757 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8880035d0 con 0x7ff8a0157d40 2024-09-13T17:50:31.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.873+0000 7ff8a4e70700 1 -- 172.21.15.40:0/1051604757 >> v2:172.21.15.43:3300/0 conn(0x7ff8a0151050 msgr2=0x7ff8a0066220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:31.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.873+0000 7ff8a4e70700 1 --2- 172.21.15.40:0/1051604757 >> v2:172.21.15.43:3300/0 conn(0x7ff8a0151050 0x7ff8a0066220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.873+0000 7ff8a4e70700 1 -- 172.21.15.40:0/1051604757 >> v1:172.21.15.40:6789/0 conn(0x7ff8a0157d40 legacy=0x7ff8a00c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:31.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.873+0000 7ff8a4e70700 1 -- 172.21.15.40:0/1051604757 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a006b290 con 0x7ff8a00cb2a0 2024-09-13T17:50:31.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.873+0000 7ff89ffff700 1 --2- 172.21.15.40:0/1051604757 >> v2:172.21.15.43:3300/0 conn(0x7ff8a0151050 0x7ff8a0066220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:50:31.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.873+0000 7ff8a4e70700 1 --2- 172.21.15.40:0/1051604757 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff8a00cb2a0 0x7ff8a00cb6d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff898009d90 tx=0x7ff89800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=173874f41a2b0400 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:31.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.877+0000 7ff89effd700 1 -- 172.21.15.40:0/1051604757 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89800e070 con 0x7ff8a00cb2a0 2024-09-13T17:50:31.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.877+0000 7ff89effd700 1 -- 172.21.15.40:0/1051604757 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89800bb20 con 0x7ff8a00cb2a0 2024-09-13T17:50:31.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.877+0000 7ff89effd700 1 -- 172.21.15.40:0/1051604757 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89800be90 con 0x7ff8a00cb2a0 2024-09-13T17:50:31.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.877+0000 7ff8a5671700 1 -- 172.21.15.40:0/1051604757 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff8a00cb2a0 msgr2=0x7ff8a00cb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:31.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.877+0000 7ff8a5671700 1 --2- 172.21.15.40:0/1051604757 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff8a00cb2a0 0x7ff8a00cb6d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff898009d90 tx=0x7ff89800b330 comp rx=0 tx=0).stop 2024-09-13T17:50:31.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.877+0000 7ff8a5671700 1 -- 172.21.15.40:0/1051604757 shutdown_connections 2024-09-13T17:50:31.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.877+0000 7ff8a5671700 1 --2- 172.21.15.40:0/1051604757 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff8a00cb2a0 0x7ff8a00cb6d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.877+0000 7ff8a5671700 1 --2- 172.21.15.40:0/1051604757 >> v2:172.21.15.43:3300/0 conn(0x7ff8a0151050 0x7ff8a0066220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.877+0000 7ff8a5671700 1 -- 172.21.15.40:0/1051604757 >> 172.21.15.40:0/1051604757 conn(0x7ff8a006a390 msgr2=0x7ff8a006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:31.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.877+0000 7ff8a5671700 1 -- 172.21.15.40:0/1051604757 shutdown_connections 2024-09-13T17:50:31.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.877+0000 7ff8a5671700 1 -- 172.21.15.40:0/1051604757 wait complete. 2024-09-13T17:50:31.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.877+0000 7ff8a5671700 1 Processor -- start 2024-09-13T17:50:31.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.877+0000 7ff8a5671700 1 -- start start 2024-09-13T17:50:31.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff8a5671700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8a00cb2a0 0x7ff8a01e2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:31.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff8a5671700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff8a0151050 0x7ff8a014de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:31.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff8a5671700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8a0157d40 0x7ff8a014e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:31.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff8a5671700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a014f9d0 con 0x7ff8a00cb2a0 2024-09-13T17:50:31.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff8a5671700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a01eec20 con 0x7ff8a0157d40 2024-09-13T17:50:31.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff8a5671700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a01f2330 con 0x7ff8a0151050 2024-09-13T17:50:31.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff89ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8a00cb2a0 0x7ff8a01e2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:31.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff89ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8a00cb2a0 0x7ff8a01e2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38244/0 (socket says 172.21.15.40:38244) 2024-09-13T17:50:31.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff89ffff700 1 -- 172.21.15.40:0/3701411366 learned_addr learned my addr 172.21.15.40:0/3701411366 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:31.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff8a4e70700 1 --2- 172.21.15.40:0/3701411366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8a0157d40 0x7ff8a014e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:31.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff89f7fe700 1 --2- 172.21.15.40:0/3701411366 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff8a0151050 0x7ff8a014de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:31.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff89ffff700 1 -- 172.21.15.40:0/3701411366 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff8a0151050 msgr2=0x7ff8a014de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:31.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff89ffff700 1 --2- 172.21.15.40:0/3701411366 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff8a0151050 0x7ff8a014de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff89ffff700 1 -- 172.21.15.40:0/3701411366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8a0157d40 msgr2=0x7ff8a014e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:31.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff89ffff700 1 --2- 172.21.15.40:0/3701411366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8a0157d40 0x7ff8a014e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:31.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff89ffff700 1 -- 172.21.15.40:0/3701411366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8900094b0 con 0x7ff8a00cb2a0 2024-09-13T17:50:31.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff8a4e70700 1 --2- 172.21.15.40:0/3701411366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8a0157d40 0x7ff8a014e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:50:31.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff89ffff700 1 --2- 172.21.15.40:0/3701411366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8a00cb2a0 0x7ff8a01e2690 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff89000ebe0 tx=0x7ff890009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:31.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff89cff9700 1 -- 172.21.15.40:0/3701411366 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89000c910 con 0x7ff8a00cb2a0 2024-09-13T17:50:31.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff89cff9700 1 -- 172.21.15.40:0/3701411366 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89000ca70 con 0x7ff8a00cb2a0 2024-09-13T17:50:31.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff8a5671700 1 -- 172.21.15.40:0/3701411366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff898009a70 con 0x7ff8a00cb2a0 2024-09-13T17:50:31.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff89cff9700 1 -- 172.21.15.40:0/3701411366 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89000ce00 con 0x7ff8a00cb2a0 2024-09-13T17:50:31.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.881+0000 7ff8a5671700 1 -- 172.21.15.40:0/3701411366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a01f28f0 con 0x7ff8a00cb2a0 2024-09-13T17:50:31.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.885+0000 7ff8a5671700 1 -- 172.21.15.40:0/3701411366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a00c4a90 con 0x7ff8a00cb2a0 2024-09-13T17:50:31.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.885+0000 7ff89cff9700 1 -- 172.21.15.40:0/3701411366 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff890013470 con 0x7ff8a00cb2a0 2024-09-13T17:50:31.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.885+0000 7ff89cff9700 1 --2- 172.21.15.40:0/3701411366 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff88c066730 0x7ff88c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:31.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.885+0000 7ff89f7fe700 1 --2- 172.21.15.40:0/3701411366 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff88c066730 0x7ff88c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:31.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.885+0000 7ff89cff9700 1 -- 172.21.15.40:0/3701411366 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff890084110 con 0x7ff8a00cb2a0 2024-09-13T17:50:31.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.885+0000 7ff89f7fe700 1 --2- 172.21.15.40:0/3701411366 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff88c066730 0x7ff88c068bf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0066700 tx=0x7ff894009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:31.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:31.893+0000 7ff89cff9700 1 -- 172.21.15.40:0/3701411366 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff890051730 con 0x7ff8a00cb2a0 2024-09-13T17:50:32.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.057+0000 7ff8a5671700 1 -- 172.21.15.40:0/3701411366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff8a00c5d50 con 0x7ff8a00cb2a0 2024-09-13T17:50:32.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.057+0000 7ff89cff9700 1 -- 172.21.15.40:0/3701411366 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff890054d50 con 0x7ff8a00cb2a0 2024-09-13T17:50:32.065 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-09-13T17:50:32.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.061+0000 7ff8a5671700 1 -- 172.21.15.40:0/3701411366 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff88c066730 msgr2=0x7ff88c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:32.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.061+0000 7ff8a5671700 1 --2- 172.21.15.40:0/3701411366 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff88c066730 0x7ff88c068bf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0066700 tx=0x7ff894009040 comp rx=0 tx=0).stop 2024-09-13T17:50:32.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.061+0000 7ff8a5671700 1 -- 172.21.15.40:0/3701411366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8a00cb2a0 msgr2=0x7ff8a01e2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:32.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.061+0000 7ff8a5671700 1 --2- 172.21.15.40:0/3701411366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8a00cb2a0 0x7ff8a01e2690 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff89000ebe0 tx=0x7ff890009ed0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.061+0000 7ff8a5671700 1 -- 172.21.15.40:0/3701411366 shutdown_connections 2024-09-13T17:50:32.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.061+0000 7ff8a5671700 1 --2- 172.21.15.40:0/3701411366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8a0157d40 0x7ff8a014e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.061+0000 7ff8a5671700 1 --2- 172.21.15.40:0/3701411366 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff88c066730 0x7ff88c068bf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.061+0000 7ff8a5671700 1 --2- 172.21.15.40:0/3701411366 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff8a0151050 0x7ff8a014de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.061+0000 7ff8a5671700 1 --2- 172.21.15.40:0/3701411366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8a00cb2a0 0x7ff8a01e2690 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.061+0000 7ff8a5671700 1 -- 172.21.15.40:0/3701411366 >> 172.21.15.40:0/3701411366 conn(0x7ff8a006a390 msgr2=0x7ff8a014ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:32.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.061+0000 7ff8a5671700 1 -- 172.21.15.40:0/3701411366 shutdown_connections 2024-09-13T17:50:32.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.061+0000 7ff8a5671700 1 -- 172.21.15.40:0/3701411366 wait complete. 2024-09-13T17:50:32.079 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T17:50:32.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.185+0000 7fc0566f0700 1 Processor -- start 2024-09-13T17:50:32.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.185+0000 7fc0566f0700 1 -- start start 2024-09-13T17:50:32.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.185+0000 7fc0566f0700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0500c8070 0x7fc0500c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:32.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.185+0000 7fc0566f0700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0501595b0 0x7fc0500c2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:32.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.185+0000 7fc0566f0700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0500b2270 con 0x7fc0500c0e90 2024-09-13T17:50:32.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.185+0000 7fc0566f0700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0500c2400 con 0x7fc0501595b0 2024-09-13T17:50:32.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.185+0000 7fc0566f0700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0500c62a0 con 0x7fc0500c8070 2024-09-13T17:50:32.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.185+0000 7fc054eed700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0501595b0 0x7fc0500c2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:32.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.185+0000 7fc054eed700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0501595b0 0x7fc0500c2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38256/0 (socket says 172.21.15.40:38256) 2024-09-13T17:50:32.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.185+0000 7fc054eed700 1 -- 172.21.15.40:0/4177148292 learned_addr learned my addr 172.21.15.40:0/4177148292 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:32.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.185+0000 7fc0556ee700 1 --2- 172.21.15.40:0/4177148292 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0500c8070 0x7fc0500c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.185+0000 7fc054eed700 1 -- 172.21.15.40:0/4177148292 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0500c8070 msgr2=0x7fc0500c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.185+0000 7fc054eed700 1 --2- 172.21.15.40:0/4177148292 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0500c8070 0x7fc0500c32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.185+0000 7fc054eed700 1 -- 172.21.15.40:0/4177148292 >> v1:172.21.15.43:6789/0 conn(0x7fc0500c0e90 legacy=0x7fc050071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.185+0000 7fc054eed700 1 -- 172.21.15.40:0/4177148292 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0500c3820 con 0x7fc0501595b0 2024-09-13T17:50:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.185+0000 7fc054eed700 1 --2- 172.21.15.40:0/4177148292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0501595b0 0x7fc0500c2890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc040009910 tx=0x7fc040009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c664eb3b012bc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.185+0000 7fc04ffff700 1 -- 172.21.15.40:0/4177148292 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc040011830 con 0x7fc0501595b0 2024-09-13T17:50:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.185+0000 7fc04ffff700 1 -- 172.21.15.40:0/4177148292 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc040011990 con 0x7fc0501595b0 2024-09-13T17:50:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.185+0000 7fc04ffff700 1 -- 172.21.15.40:0/4177148292 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc040011d40 con 0x7fc0501595b0 2024-09-13T17:50:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0566f0700 1 -- 172.21.15.40:0/4177148292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0501595b0 msgr2=0x7fc0500c2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0566f0700 1 --2- 172.21.15.40:0/4177148292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0501595b0 0x7fc0500c2890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc040009910 tx=0x7fc040009c20 comp rx=0 tx=0).stop 2024-09-13T17:50:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0566f0700 1 -- 172.21.15.40:0/4177148292 shutdown_connections 2024-09-13T17:50:32.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0566f0700 1 --2- 172.21.15.40:0/4177148292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0501595b0 0x7fc0500c2890 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0566f0700 1 --2- 172.21.15.40:0/4177148292 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0500c8070 0x7fc0500c32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0566f0700 1 -- 172.21.15.40:0/4177148292 >> 172.21.15.40:0/4177148292 conn(0x7fc05006e4f0 msgr2=0x7fc05006e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:32.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0566f0700 1 -- 172.21.15.40:0/4177148292 shutdown_connections 2024-09-13T17:50:32.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0566f0700 1 -- 172.21.15.40:0/4177148292 wait complete. 2024-09-13T17:50:32.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0566f0700 1 Processor -- start 2024-09-13T17:50:32.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0566f0700 1 -- start start 2024-09-13T17:50:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0566f0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0500c0e90 0x7fc05015be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0566f0700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0500c8070 0x7fc05015c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0566f0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0501595b0 0x7fc05015a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0566f0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0501e6a60 con 0x7fc0500c0e90 2024-09-13T17:50:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0566f0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0501e67a0 con 0x7fc0501595b0 2024-09-13T17:50:32.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0566f0700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05006c670 con 0x7fc0500c8070 2024-09-13T17:50:32.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0556ee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0500c0e90 0x7fc05015be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:32.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0556ee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0500c0e90 0x7fc05015be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38272/0 (socket says 172.21.15.40:38272) 2024-09-13T17:50:32.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0556ee700 1 -- 172.21.15.40:0/851679807 learned_addr learned my addr 172.21.15.40:0/851679807 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:32.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc054eed700 1 --2- 172.21.15.40:0/851679807 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0500c8070 0x7fc05015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:32.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0556ee700 1 -- 172.21.15.40:0/851679807 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0500c8070 msgr2=0x7fc05015c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:32.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc055eef700 1 --2- 172.21.15.40:0/851679807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0501595b0 0x7fc05015a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:32.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0556ee700 1 --2- 172.21.15.40:0/851679807 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0500c8070 0x7fc05015c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0556ee700 1 -- 172.21.15.40:0/851679807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0501595b0 msgr2=0x7fc05015a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:32.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0556ee700 1 --2- 172.21.15.40:0/851679807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0501595b0 0x7fc05015a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0556ee700 1 -- 172.21.15.40:0/851679807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc040009570 con 0x7fc0500c0e90 2024-09-13T17:50:32.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc055eef700 1 --2- 172.21.15.40:0/851679807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0501595b0 0x7fc05015a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:50:32.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.189+0000 7fc0556ee700 1 --2- 172.21.15.40:0/851679807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0500c0e90 0x7fc05015be00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc04400efc0 tx=0x7fc04400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:32.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.193+0000 7fc04dffb700 1 -- 172.21.15.40:0/851679807 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04400cdc0 con 0x7fc0500c0e90 2024-09-13T17:50:32.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.193+0000 7fc0566f0700 1 -- 172.21.15.40:0/851679807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05006c950 con 0x7fc0500c0e90 2024-09-13T17:50:32.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.193+0000 7fc04dffb700 1 -- 172.21.15.40:0/851679807 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04400f040 con 0x7fc0500c0e90 2024-09-13T17:50:32.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.193+0000 7fc04dffb700 1 -- 172.21.15.40:0/851679807 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0440146a0 con 0x7fc0500c0e90 2024-09-13T17:50:32.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.193+0000 7fc0566f0700 1 -- 172.21.15.40:0/851679807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05006cf10 con 0x7fc0500c0e90 2024-09-13T17:50:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.193+0000 7fc0566f0700 1 -- 172.21.15.40:0/851679807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0500c1b40 con 0x7fc0500c0e90 2024-09-13T17:50:32.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.193+0000 7fc04dffb700 1 -- 172.21.15.40:0/851679807 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc044014800 con 0x7fc0500c0e90 2024-09-13T17:50:32.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.193+0000 7fc04dffb700 1 --2- 172.21.15.40:0/851679807 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc03c066730 0x7fc03c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:32.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.193+0000 7fc04dffb700 1 -- 172.21.15.40:0/851679807 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc044084880 con 0x7fc0500c0e90 2024-09-13T17:50:32.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.193+0000 7fc054eed700 1 --2- 172.21.15.40:0/851679807 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc03c066730 0x7fc03c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:32.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.197+0000 7fc054eed700 1 --2- 172.21.15.40:0/851679807 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc03c066730 0x7fc03c068bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc040009910 tx=0x7fc040008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:32.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.201+0000 7fc04dffb700 1 -- 172.21.15.40:0/851679807 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc044051ec0 con 0x7fc0500c0e90 2024-09-13T17:50:32.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.373+0000 7fc0566f0700 1 -- 172.21.15.40:0/851679807 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc05006f5a0 con 0x7fc0500c0e90 2024-09-13T17:50:32.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.373+0000 7fc04dffb700 1 -- 172.21.15.40:0/851679807 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc0440554e0 con 0x7fc0500c0e90 2024-09-13T17:50:32.381 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:50:32.381 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2024-09-13T17:50:32.381 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:50:32.381 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2024-09-13T17:50:32.381 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2024-09-13T17:50:32.381 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:50:32.382 INFO:teuthology.orchestra.run.smithi040.stdout:] 2024-09-13T17:50:32.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.377+0000 7fc0566f0700 1 -- 172.21.15.40:0/851679807 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc03c066730 msgr2=0x7fc03c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:32.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.377+0000 7fc0566f0700 1 --2- 172.21.15.40:0/851679807 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc03c066730 0x7fc03c068bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc040009910 tx=0x7fc040008030 comp rx=0 tx=0).stop 2024-09-13T17:50:32.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.377+0000 7fc0566f0700 1 -- 172.21.15.40:0/851679807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0500c0e90 msgr2=0x7fc05015be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:32.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.377+0000 7fc0566f0700 1 --2- 172.21.15.40:0/851679807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0500c0e90 0x7fc05015be00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc04400efc0 tx=0x7fc04400c3a0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.377+0000 7fc0566f0700 1 -- 172.21.15.40:0/851679807 shutdown_connections 2024-09-13T17:50:32.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.377+0000 7fc0566f0700 1 --2- 172.21.15.40:0/851679807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0501595b0 0x7fc05015a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.377+0000 7fc0566f0700 1 --2- 172.21.15.40:0/851679807 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc03c066730 0x7fc03c068bf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.377+0000 7fc0566f0700 1 --2- 172.21.15.40:0/851679807 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0500c8070 0x7fc05015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.377+0000 7fc0566f0700 1 --2- 172.21.15.40:0/851679807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0500c0e90 0x7fc05015be00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.377+0000 7fc0566f0700 1 -- 172.21.15.40:0/851679807 >> 172.21.15.40:0/851679807 conn(0x7fc05006e4f0 msgr2=0x7fc050158900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:32.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.377+0000 7fc0566f0700 1 -- 172.21.15.40:0/851679807 shutdown_connections 2024-09-13T17:50:32.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.377+0000 7fc0566f0700 1 -- 172.21.15.40:0/851679807 wait complete. 2024-09-13T17:50:32.395 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T17:50:32.395 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T17:50:32.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbd50da700 1 Processor -- start 2024-09-13T17:50:32.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbd50da700 1 -- start start 2024-09-13T17:50:32.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbd50da700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdbd0153900 0x7fdbd00c9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:32.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbd50da700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdbd00c2980 0x7fdbd015d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:32.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbd50da700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd00cad20 con 0x7fdbd015d720 2024-09-13T17:50:32.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbd50da700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd0070a30 con 0x7fdbd00c2980 2024-09-13T17:50:32.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbd50da700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd0071e50 con 0x7fdbd0153900 2024-09-13T17:50:32.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbcf7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdbd00c2980 0x7fdbd015d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:32.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbcf7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdbd00c2980 0x7fdbd015d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38286/0 (socket says 172.21.15.40:38286) 2024-09-13T17:50:32.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbd48d9700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdbd015d720 0x7fdbd00cbb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35744/0 (socket says 172.21.15.40:35744) 2024-09-13T17:50:32.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbcf7fe700 1 -- 172.21.15.40:0/2128524552 learned_addr learned my addr 172.21.15.40:0/2128524552 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:32.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbcffff700 1 --2- 172.21.15.40:0/2128524552 >> v2:172.21.15.43:3300/0 conn(0x7fdbd0153900 0x7fdbd00c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:32.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbcf7fe700 1 -- 172.21.15.40:0/2128524552 >> v2:172.21.15.43:3300/0 conn(0x7fdbd0153900 msgr2=0x7fdbd00c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:32.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbcf7fe700 1 --2- 172.21.15.40:0/2128524552 >> v2:172.21.15.43:3300/0 conn(0x7fdbd0153900 0x7fdbd00c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbcf7fe700 1 -- 172.21.15.40:0/2128524552 >> v1:172.21.15.40:6789/0 conn(0x7fdbd015d720 legacy=0x7fdbd00cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:32.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbceffd700 1 -- 172.21.15.40:0/2128524552 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 410805947 0 0) 0x7fdbd00cad20 con 0x7fdbd015d720 2024-09-13T17:50:32.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbcf7fe700 1 -- 172.21.15.40:0/2128524552 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd00b2250 con 0x7fdbd00c2980 2024-09-13T17:50:32.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbcf7fe700 1 --2- 172.21.15.40:0/2128524552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd00c2980 0x7fdbd015d1e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00cc770 tx=0x7fdbc4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27d6f9a48611670 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:32.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbceffd700 1 -- 172.21.15.40:0/2128524552 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4011830 con 0x7fdbd00c2980 2024-09-13T17:50:32.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbceffd700 1 -- 172.21.15.40:0/2128524552 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc4011990 con 0x7fdbd00c2980 2024-09-13T17:50:32.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.509+0000 7fdbceffd700 1 -- 172.21.15.40:0/2128524552 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4011d20 con 0x7fdbd00c2980 2024-09-13T17:50:32.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.513+0000 7fdbd50da700 1 -- 172.21.15.40:0/2128524552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd00c2980 msgr2=0x7fdbd015d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:32.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.513+0000 7fdbd50da700 1 --2- 172.21.15.40:0/2128524552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd00c2980 0x7fdbd015d1e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00cc770 tx=0x7fdbc4009bf0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.513+0000 7fdbd50da700 1 -- 172.21.15.40:0/2128524552 shutdown_connections 2024-09-13T17:50:32.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.513+0000 7fdbd50da700 1 --2- 172.21.15.40:0/2128524552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd00c2980 0x7fdbd015d1e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.513+0000 7fdbd50da700 1 --2- 172.21.15.40:0/2128524552 >> v2:172.21.15.43:3300/0 conn(0x7fdbd0153900 0x7fdbd00c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.513+0000 7fdbd50da700 1 -- 172.21.15.40:0/2128524552 >> 172.21.15.40:0/2128524552 conn(0x7fdbd006e620 msgr2=0x7fdbd006ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:32.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.513+0000 7fdbd50da700 1 -- 172.21.15.40:0/2128524552 shutdown_connections 2024-09-13T17:50:32.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.513+0000 7fdbd50da700 1 -- 172.21.15.40:0/2128524552 wait complete. 2024-09-13T17:50:32.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.513+0000 7fdbd50da700 1 Processor -- start 2024-09-13T17:50:32.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.513+0000 7fdbd50da700 1 -- start start 2024-09-13T17:50:32.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbd50da700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbd00c2980 0x7fdbd0150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:32.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbd50da700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbd0153900 0x7fdbd015bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:32.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbd50da700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd015d720 0x7fdbd015c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:32.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbd50da700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd0154ef0 con 0x7fdbd015d720 2024-09-13T17:50:32.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbd50da700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd0157c20 con 0x7fdbd00c2980 2024-09-13T17:50:32.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbd50da700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd0158a60 con 0x7fdbd0153900 2024-09-13T17:50:32.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbd48d9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd015d720 0x7fdbd015c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:32.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbd48d9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd015d720 0x7fdbd015c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38296/0 (socket says 172.21.15.40:38296) 2024-09-13T17:50:32.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbd48d9700 1 -- 172.21.15.40:0/2880863434 learned_addr learned my addr 172.21.15.40:0/2880863434 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:32.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbcffff700 1 --2- 172.21.15.40:0/2880863434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbd00c2980 0x7fdbd0150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:32.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbcf7fe700 1 --2- 172.21.15.40:0/2880863434 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbd0153900 0x7fdbd015bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:32.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbd48d9700 1 -- 172.21.15.40:0/2880863434 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbd0153900 msgr2=0x7fdbd015bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:32.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbd48d9700 1 --2- 172.21.15.40:0/2880863434 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbd0153900 0x7fdbd015bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbd48d9700 1 -- 172.21.15.40:0/2880863434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbd00c2980 msgr2=0x7fdbd0150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:32.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbd48d9700 1 --2- 172.21.15.40:0/2880863434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbd00c2980 0x7fdbd0150230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:32.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbd48d9700 1 -- 172.21.15.40:0/2880863434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc4009570 con 0x7fdbd015d720 2024-09-13T17:50:32.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbd48d9700 1 --2- 172.21.15.40:0/2880863434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd015d720 0x7fdbd015c240 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800cba0 tx=0x7fdbc800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:32.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbccff9700 1 -- 172.21.15.40:0/2880863434 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc800aaa0 con 0x7fdbd015d720 2024-09-13T17:50:32.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbd50da700 1 -- 172.21.15.40:0/2880863434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd0157e70 con 0x7fdbd015d720 2024-09-13T17:50:32.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbd50da700 1 -- 172.21.15.40:0/2880863434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd01551b0 con 0x7fdbd015d720 2024-09-13T17:50:32.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbccff9700 1 -- 172.21.15.40:0/2880863434 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc800ac20 con 0x7fdbd015d720 2024-09-13T17:50:32.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbccff9700 1 -- 172.21.15.40:0/2880863434 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc80070d0 con 0x7fdbd015d720 2024-09-13T17:50:32.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.517+0000 7fdbd50da700 1 -- 172.21.15.40:0/2880863434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd006c6d0 con 0x7fdbd015d720 2024-09-13T17:50:32.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.521+0000 7fdbccff9700 1 -- 172.21.15.40:0/2880863434 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdbc8007230 con 0x7fdbd015d720 2024-09-13T17:50:32.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.521+0000 7fdbccff9700 1 --2- 172.21.15.40:0/2880863434 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdbbc066730 0x7fdbbc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:32.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.521+0000 7fdbccff9700 1 -- 172.21.15.40:0/2880863434 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdbc8083860 con 0x7fdbd015d720 2024-09-13T17:50:32.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.521+0000 7fdbcffff700 1 --2- 172.21.15.40:0/2880863434 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdbbc066730 0x7fdbbc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:32.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.521+0000 7fdbcffff700 1 --2- 172.21.15.40:0/2880863434 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdbbc066730 0x7fdbbc068bf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdbd006be20 tx=0x7fdbc0009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:32.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.525+0000 7fdbccff9700 1 -- 172.21.15.40:0/2880863434 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdbc8050f30 con 0x7fdbd015d720 2024-09-13T17:50:32.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:32.693+0000 7fdbd50da700 1 -- 172.21.15.40:0/2880863434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fdbd015ccc0 con 0x7fdbd015d720 2024-09-13T17:50:33.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.173+0000 7fdbccff9700 1 -- 172.21.15.40:0/2880863434 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdbc8018020 con 0x7fdbd015d720 2024-09-13T17:50:33.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.181+0000 7fdbd50da700 1 -- 172.21.15.40:0/2880863434 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdbbc066730 msgr2=0x7fdbbc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:33.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.181+0000 7fdbd50da700 1 --2- 172.21.15.40:0/2880863434 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdbbc066730 0x7fdbbc068bf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdbd006be20 tx=0x7fdbc0009240 comp rx=0 tx=0).stop 2024-09-13T17:50:33.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.181+0000 7fdbd50da700 1 -- 172.21.15.40:0/2880863434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd015d720 msgr2=0x7fdbd015c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:33.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.181+0000 7fdbd50da700 1 --2- 172.21.15.40:0/2880863434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd015d720 0x7fdbd015c240 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800cba0 tx=0x7fdbc800ceb0 comp rx=0 tx=0).stop 2024-09-13T17:50:33.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.181+0000 7fdbd50da700 1 -- 172.21.15.40:0/2880863434 shutdown_connections 2024-09-13T17:50:33.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.181+0000 7fdbd50da700 1 --2- 172.21.15.40:0/2880863434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbd015d720 0x7fdbd015c240 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:33.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.181+0000 7fdbd50da700 1 --2- 172.21.15.40:0/2880863434 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdbbc066730 0x7fdbbc068bf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:33.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.181+0000 7fdbd50da700 1 --2- 172.21.15.40:0/2880863434 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbd0153900 0x7fdbd015bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:33.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.181+0000 7fdbd50da700 1 --2- 172.21.15.40:0/2880863434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbd00c2980 0x7fdbd0150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:33.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.181+0000 7fdbd50da700 1 -- 172.21.15.40:0/2880863434 >> 172.21.15.40:0/2880863434 conn(0x7fdbd006e620 msgr2=0x7fdbd00c2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:33.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.181+0000 7fdbd50da700 1 -- 172.21.15.40:0/2880863434 shutdown_connections 2024-09-13T17:50:33.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.181+0000 7fdbd50da700 1 -- 172.21.15.40:0/2880863434 wait complete. 2024-09-13T17:50:33.191 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2024-09-13T17:50:33.203 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-13T17:50:33.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.317+0000 7fc668f65700 1 Processor -- start 2024-09-13T17:50:33.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.317+0000 7fc668f65700 1 -- start start 2024-09-13T17:50:33.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.317+0000 7fc668f65700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc6640cf1a0 0x7fc66415a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:33.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.317+0000 7fc668f65700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc6640cc250 0x7fc6640c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:33.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.317+0000 7fc668f65700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66415a7a0 con 0x7fc6640cf580 2024-09-13T17:50:33.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.317+0000 7fc668f65700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6640cbdc0 con 0x7fc6640cc250 2024-09-13T17:50:33.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.317+0000 7fc668f65700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc664153f80 con 0x7fc6640cf1a0 2024-09-13T17:50:33.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.317+0000 7fc663fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc6640cc250 0x7fc6640c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:33.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.321+0000 7fc663fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc6640cc250 0x7fc6640c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38304/0 (socket says 172.21.15.40:38304) 2024-09-13T17:50:33.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.321+0000 7fc663fff700 1 -- 172.21.15.40:0/1936808394 learned_addr learned my addr 172.21.15.40:0/1936808394 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:33.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.321+0000 7fc6637fe700 1 --2- 172.21.15.40:0/1936808394 >> v2:172.21.15.43:3300/0 conn(0x7fc6640cf1a0 0x7fc66415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:33.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.321+0000 7fc663fff700 1 -- 172.21.15.40:0/1936808394 >> v2:172.21.15.43:3300/0 conn(0x7fc6640cf1a0 msgr2=0x7fc66415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:33.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.321+0000 7fc663fff700 1 --2- 172.21.15.40:0/1936808394 >> v2:172.21.15.43:3300/0 conn(0x7fc6640cf1a0 0x7fc66415a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:33.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.321+0000 7fc663fff700 1 -- 172.21.15.40:0/1936808394 >> v1:172.21.15.43:6789/0 conn(0x7fc6640cf580 legacy=0x7fc6640c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:33.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.321+0000 7fc663fff700 1 -- 172.21.15.40:0/1936808394 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6640caed0 con 0x7fc6640cc250 2024-09-13T17:50:33.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.321+0000 7fc663fff700 1 --2- 172.21.15.40:0/1936808394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6640cc250 0x7fc6640c8aa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc664067020 tx=0x7fc65c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8d3caaf0e1c9a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:33.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.321+0000 7fc662ffd700 1 -- 172.21.15.40:0/1936808394 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c00e070 con 0x7fc6640cc250 2024-09-13T17:50:33.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.321+0000 7fc662ffd700 1 -- 172.21.15.40:0/1936808394 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65c00b910 con 0x7fc6640cc250 2024-09-13T17:50:33.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.321+0000 7fc662ffd700 1 -- 172.21.15.40:0/1936808394 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c00bcc0 con 0x7fc6640cc250 2024-09-13T17:50:33.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.321+0000 7fc668f65700 1 -- 172.21.15.40:0/1936808394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6640cc250 msgr2=0x7fc6640c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:33.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.321+0000 7fc668f65700 1 --2- 172.21.15.40:0/1936808394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6640cc250 0x7fc6640c8aa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc664067020 tx=0x7fc65c00b330 comp rx=0 tx=0).stop 2024-09-13T17:50:33.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.321+0000 7fc668f65700 1 -- 172.21.15.40:0/1936808394 shutdown_connections 2024-09-13T17:50:33.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.321+0000 7fc668f65700 1 --2- 172.21.15.40:0/1936808394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6640cc250 0x7fc6640c8aa0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:33.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.321+0000 7fc668f65700 1 --2- 172.21.15.40:0/1936808394 >> v2:172.21.15.43:3300/0 conn(0x7fc6640cf1a0 0x7fc66415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:33.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.321+0000 7fc668f65700 1 -- 172.21.15.40:0/1936808394 >> 172.21.15.40:0/1936808394 conn(0x7fc664066090 msgr2=0x7fc664066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:33.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.321+0000 7fc668f65700 1 -- 172.21.15.40:0/1936808394 shutdown_connections 2024-09-13T17:50:33.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.321+0000 7fc668f65700 1 -- 172.21.15.40:0/1936808394 wait complete. 2024-09-13T17:50:33.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.321+0000 7fc668f65700 1 Processor -- start 2024-09-13T17:50:33.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc668f65700 1 -- start start 2024-09-13T17:50:33.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc668f65700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6640cc250 0x7fc664155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:33.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc668f65700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6640cf1a0 0x7fc6641f0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:33.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc668f65700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6640cf580 0x7fc6640c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:33.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc668f65700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc664154ea0 con 0x7fc6640cc250 2024-09-13T17:50:33.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc668f65700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6641545b0 con 0x7fc6640cf580 2024-09-13T17:50:33.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc668f65700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6641efb20 con 0x7fc6640cf1a0 2024-09-13T17:50:33.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc6637fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6640cc250 0x7fc664155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:33.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc6637fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6640cc250 0x7fc664155330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38318/0 (socket says 172.21.15.40:38318) 2024-09-13T17:50:33.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc6637fe700 1 -- 172.21.15.40:0/96170603 learned_addr learned my addr 172.21.15.40:0/96170603 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:33.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc663fff700 1 --2- 172.21.15.40:0/96170603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6640cf580 0x7fc6640c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:33.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc65affd700 1 --2- 172.21.15.40:0/96170603 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6640cf1a0 0x7fc6641f0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:33.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc6637fe700 1 -- 172.21.15.40:0/96170603 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6640cf1a0 msgr2=0x7fc6641f0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:33.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc6637fe700 1 --2- 172.21.15.40:0/96170603 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6640cf1a0 0x7fc6641f0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:33.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc6637fe700 1 -- 172.21.15.40:0/96170603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6640cf580 msgr2=0x7fc6640c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:33.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc6637fe700 1 --2- 172.21.15.40:0/96170603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6640cf580 0x7fc6640c4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:33.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc6637fe700 1 -- 172.21.15.40:0/96170603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65c009b30 con 0x7fc6640cc250 2024-09-13T17:50:33.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc6637fe700 1 --2- 172.21.15.40:0/96170603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6640cc250 0x7fc664155330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc65400e9d0 tx=0x7fc65400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:33.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc660ff9700 1 -- 172.21.15.40:0/96170603 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc654009720 con 0x7fc6640cc250 2024-09-13T17:50:33.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc660ff9700 1 -- 172.21.15.40:0/96170603 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65400ee70 con 0x7fc6640cc250 2024-09-13T17:50:33.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc668f65700 1 -- 172.21.15.40:0/96170603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc664154850 con 0x7fc6640cc250 2024-09-13T17:50:33.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc660ff9700 1 -- 172.21.15.40:0/96170603 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65400ca50 con 0x7fc6640cc250 2024-09-13T17:50:33.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.325+0000 7fc668f65700 1 -- 172.21.15.40:0/96170603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6641efe00 con 0x7fc6640cc250 2024-09-13T17:50:33.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.329+0000 7fc668f65700 1 -- 172.21.15.40:0/96170603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc664067a40 con 0x7fc6640cc250 2024-09-13T17:50:33.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.329+0000 7fc660ff9700 1 -- 172.21.15.40:0/96170603 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc65400cbb0 con 0x7fc6640cc250 2024-09-13T17:50:33.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.329+0000 7fc660ff9700 1 --2- 172.21.15.40:0/96170603 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc644066730 0x7fc644068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:33.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.329+0000 7fc660ff9700 1 -- 172.21.15.40:0/96170603 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fc654084380 con 0x7fc6640cc250 2024-09-13T17:50:33.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.329+0000 7fc65affd700 1 --2- 172.21.15.40:0/96170603 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc644066730 0x7fc644068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:33.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.329+0000 7fc65affd700 1 --2- 172.21.15.40:0/96170603 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc644066730 0x7fc644068bf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc6640cd280 tx=0x7fc64c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:33.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.337+0000 7fc660ff9700 1 -- 172.21.15.40:0/96170603 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc654051830 con 0x7fc6640cc250 2024-09-13T17:50:33.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:33.509+0000 7fc668f65700 1 -- 172.21.15.40:0/96170603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fc66415ac50 con 0x7fc6640cc250 2024-09-13T17:50:33.552 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:50:33.554 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:50:34.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.181+0000 7fc660ff9700 1 -- 172.21.15.40:0/96170603 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7fc65400ce60 con 0x7fc6640cc250 2024-09-13T17:50:34.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.201+0000 7fc668f65700 1 -- 172.21.15.40:0/96170603 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc644066730 msgr2=0x7fc644068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:34.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.201+0000 7fc668f65700 1 --2- 172.21.15.40:0/96170603 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc644066730 0x7fc644068bf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc6640cd280 tx=0x7fc64c009040 comp rx=0 tx=0).stop 2024-09-13T17:50:34.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.201+0000 7fc668f65700 1 -- 172.21.15.40:0/96170603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6640cc250 msgr2=0x7fc664155330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:34.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.201+0000 7fc668f65700 1 --2- 172.21.15.40:0/96170603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6640cc250 0x7fc664155330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc65400e9d0 tx=0x7fc65400ece0 comp rx=0 tx=0).stop 2024-09-13T17:50:34.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.201+0000 7fc668f65700 1 -- 172.21.15.40:0/96170603 shutdown_connections 2024-09-13T17:50:34.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.201+0000 7fc668f65700 1 --2- 172.21.15.40:0/96170603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6640cf580 0x7fc6640c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:34.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.201+0000 7fc668f65700 1 --2- 172.21.15.40:0/96170603 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc644066730 0x7fc644068bf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:34.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.201+0000 7fc668f65700 1 --2- 172.21.15.40:0/96170603 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6640cf1a0 0x7fc6641f0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:34.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.201+0000 7fc668f65700 1 --2- 172.21.15.40:0/96170603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6640cc250 0x7fc664155330 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:34.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.201+0000 7fc668f65700 1 -- 172.21.15.40:0/96170603 >> 172.21.15.40:0/96170603 conn(0x7fc664066090 msgr2=0x7fc6640ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:34.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.201+0000 7fc668f65700 1 -- 172.21.15.40:0/96170603 shutdown_connections 2024-09-13T17:50:34.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.201+0000 7fc668f65700 1 -- 172.21.15.40:0/96170603 wait complete. 2024-09-13T17:50:34.209 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2024-09-13T17:50:34.224 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T17:50:34.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.329+0000 7fe6bdda5700 1 Processor -- start 2024-09-13T17:50:34.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.329+0000 7fe6bdda5700 1 -- start start 2024-09-13T17:50:34.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.329+0000 7fe6bdda5700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe6b80c8340 0x7fe6b80c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:34.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.329+0000 7fe6bdda5700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe6b80c7250 0x7fe6b81577f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:34.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.329+0000 7fe6bdda5700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b806b400 con 0x7fe6b80c7f60 2024-09-13T17:50:34.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.329+0000 7fe6bdda5700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b815e920 con 0x7fe6b80c8340 2024-09-13T17:50:34.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.329+0000 7fe6bdda5700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b815dda0 con 0x7fe6b80c7250 2024-09-13T17:50:34.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.329+0000 7fe6bcda3700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe6b80c7f60 0x7fe6b806d920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35758/0 (socket says 172.21.15.40:35758) 2024-09-13T17:50:34.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.329+0000 7fe6bcda3700 1 -- 172.21.15.40:0/2146305100 learned_addr learned my addr 172.21.15.40:0/2146305100 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6bd5a4700 1 --2- 172.21.15.40:0/2146305100 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe6b80c7250 0x7fe6b81577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b7fff700 1 --2- 172.21.15.40:0/2146305100 >> v2:172.21.15.43:3300/0 conn(0x7fe6b80c8340 0x7fe6b80c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b77fe700 1 -- 172.21.15.40:0/2146305100 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 463371098 0 0) 0x7fe6b806b400 con 0x7fe6b80c7f60 2024-09-13T17:50:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b77fe700 1 -- 172.21.15.40:0/2146305100 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6a40035d0 con 0x7fe6b80c7f60 2024-09-13T17:50:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b77fe700 1 -- 172.21.15.40:0/2146305100 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3904858894 0 0) 0x7fe6a40035d0 con 0x7fe6b80c7f60 2024-09-13T17:50:34.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b77fe700 1 -- 172.21.15.40:0/2146305100 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe6b80c7250 msgr2=0x7fe6b81577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b77fe700 1 --2- 172.21.15.40:0/2146305100 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe6b80c7250 0x7fe6b81577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b77fe700 1 -- 172.21.15.40:0/2146305100 >> v2:172.21.15.43:3300/0 conn(0x7fe6b80c8340 msgr2=0x7fe6b80c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b77fe700 1 --2- 172.21.15.40:0/2146305100 >> v2:172.21.15.43:3300/0 conn(0x7fe6b80c8340 0x7fe6b80c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b77fe700 1 -- 172.21.15.40:0/2146305100 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b815e080 con 0x7fe6b80c7f60 2024-09-13T17:50:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b77fe700 1 -- 172.21.15.40:0/2146305100 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fe6b00019f0 con 0x7fe6b80c7f60 2024-09-13T17:50:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b7fff700 1 --2- 172.21.15.40:0/2146305100 >> v2:172.21.15.43:3300/0 conn(0x7fe6b80c8340 0x7fe6b80c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:50:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b77fe700 1 -- 172.21.15.40:0/2146305100 >> v1:172.21.15.40:6789/0 conn(0x7fe6b80c7f60 legacy=0x7fe6b806d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b77fe700 1 --2- 172.21.15.40:0/2146305100 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe6b80c7250 0x7fe6a4003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b77fe700 1 --2- 172.21.15.40:0/2146305100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6a4003f60 0x7fe6a4004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b77fe700 1 --2- 172.21.15.40:0/2146305100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a4004db0 0x7fe6a4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b77fe700 1 -- 172.21.15.40:0/2146305100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b815e920 con 0x7fe6a4003f60 2024-09-13T17:50:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b77fe700 1 -- 172.21.15.40:0/2146305100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b815dda0 con 0x7fe6a4004db0 2024-09-13T17:50:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b77fe700 1 -- 172.21.15.40:0/2146305100 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a400ba30 con 0x7fe6b80c7250 2024-09-13T17:50:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b7fff700 1 --2- 172.21.15.40:0/2146305100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6a4003f60 0x7fe6a4004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6bcda3700 1 --2- 172.21.15.40:0/2146305100 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe6b80c7250 0x7fe6a4003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6bd5a4700 1 --2- 172.21.15.40:0/2146305100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a4004db0 0x7fe6a4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b7fff700 1 -- 172.21.15.40:0/2146305100 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe6b80c7250 msgr2=0x7fe6a4003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b7fff700 1 --2- 172.21.15.40:0/2146305100 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe6b80c7250 0x7fe6a4003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b7fff700 1 -- 172.21.15.40:0/2146305100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a4004db0 msgr2=0x7fe6a4005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b7fff700 1 --2- 172.21.15.40:0/2146305100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a4004db0 0x7fe6a4005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b7fff700 1 -- 172.21.15.40:0/2146305100 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a40035d0 con 0x7fe6a4003f60 2024-09-13T17:50:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b7fff700 1 --2- 172.21.15.40:0/2146305100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6a4003f60 0x7fe6a4004870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009d10 tx=0x7fe6a40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2ed8583442f257 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b77fe700 1 -- 172.21.15.40:0/2146305100 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0009e60 con 0x7fe6a4003f60 2024-09-13T17:50:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b77fe700 1 -- 172.21.15.40:0/2146305100 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a000e9a0 con 0x7fe6a4003f60 2024-09-13T17:50:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6b77fe700 1 -- 172.21.15.40:0/2146305100 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a000ed10 con 0x7fe6a4003f60 2024-09-13T17:50:34.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6bdda5700 1 -- 172.21.15.40:0/2146305100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6a4003f60 msgr2=0x7fe6a4004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:34.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6bdda5700 1 --2- 172.21.15.40:0/2146305100 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6a4003f60 0x7fe6a4004870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009d10 tx=0x7fe6a40058a0 comp rx=0 tx=0).stop 2024-09-13T17:50:34.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6bd5a4700 1 -- 172.21.15.40:0/2146305100 reap_dead start 2024-09-13T17:50:34.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6bdda5700 1 -- 172.21.15.40:0/2146305100 shutdown_connections 2024-09-13T17:50:34.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6bdda5700 1 -- 172.21.15.40:0/2146305100 >> 172.21.15.40:0/2146305100 conn(0x7fe6b8066090 msgr2=0x7fe6b80664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:34.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6bdda5700 1 -- 172.21.15.40:0/2146305100 shutdown_connections 2024-09-13T17:50:34.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.333+0000 7fe6bdda5700 1 -- 172.21.15.40:0/2146305100 wait complete. 2024-09-13T17:50:34.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6bdda5700 1 Processor -- start 2024-09-13T17:50:34.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6bdda5700 1 -- start start 2024-09-13T17:50:34.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6bdda5700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe6a4003f60 0x7fe6b8152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:34.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6bdda5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b8153070 0x7fe6b8147910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:34.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6bdda5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6b8147e50 0x7fe6b8151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:34.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6bdda5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b8153570 con 0x7fe6b8147e50 2024-09-13T17:50:34.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6bdda5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b81582f0 con 0x7fe6b8153070 2024-09-13T17:50:34.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6bdda5700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b814c0e0 con 0x7fe6a4003f60 2024-09-13T17:50:34.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6bd5a4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6b8147e50 0x7fe6b8151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:34.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6bd5a4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6b8147e50 0x7fe6b8151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38328/0 (socket says 172.21.15.40:38328) 2024-09-13T17:50:34.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6bd5a4700 1 -- 172.21.15.40:0/2368958726 learned_addr learned my addr 172.21.15.40:0/2368958726 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:34.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6bcda3700 1 --2- 172.21.15.40:0/2368958726 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe6a4003f60 0x7fe6b8152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:34.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6b7fff700 1 --2- 172.21.15.40:0/2368958726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b8153070 0x7fe6b8147910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:34.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6bd5a4700 1 -- 172.21.15.40:0/2368958726 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe6a4003f60 msgr2=0x7fe6b8152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:34.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6bd5a4700 1 --2- 172.21.15.40:0/2368958726 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe6a4003f60 0x7fe6b8152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:34.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6bd5a4700 1 -- 172.21.15.40:0/2368958726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b8153070 msgr2=0x7fe6b8147910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:34.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6bd5a4700 1 --2- 172.21.15.40:0/2368958726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b8153070 0x7fe6b8147910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:34.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6bd5a4700 1 -- 172.21.15.40:0/2368958726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b8070da0 con 0x7fe6b8147e50 2024-09-13T17:50:34.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6bd5a4700 1 --2- 172.21.15.40:0/2368958726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6b8147e50 0x7fe6b8151db0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe6b806ab00 tx=0x7fe6a8011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:34.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6b57fa700 1 -- 172.21.15.40:0/2368958726 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8004240 con 0x7fe6b8147e50 2024-09-13T17:50:34.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6b57fa700 1 -- 172.21.15.40:0/2368958726 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a80043a0 con 0x7fe6b8147e50 2024-09-13T17:50:34.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6bdda5700 1 -- 172.21.15.40:0/2368958726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b8071080 con 0x7fe6b8147e50 2024-09-13T17:50:34.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6b57fa700 1 -- 172.21.15.40:0/2368958726 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8003900 con 0x7fe6b8147e50 2024-09-13T17:50:34.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.337+0000 7fe6bdda5700 1 -- 172.21.15.40:0/2368958726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b8072580 con 0x7fe6b8147e50 2024-09-13T17:50:34.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.341+0000 7fe6b57fa700 1 -- 172.21.15.40:0/2368958726 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe6a8003a60 con 0x7fe6b8147e50 2024-09-13T17:50:34.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.341+0000 7fe6bdda5700 1 -- 172.21.15.40:0/2368958726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b8072a10 con 0x7fe6b8147e50 2024-09-13T17:50:34.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.341+0000 7fe6b57fa700 1 --2- 172.21.15.40:0/2368958726 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe698066730 0x7fe698068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:34.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.341+0000 7fe6b57fa700 1 -- 172.21.15.40:0/2368958726 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fe6a8084240 con 0x7fe6b8147e50 2024-09-13T17:50:34.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.341+0000 7fe6bcda3700 1 --2- 172.21.15.40:0/2368958726 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe698066730 0x7fe698068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:34.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.341+0000 7fe6bcda3700 1 --2- 172.21.15.40:0/2368958726 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe698066730 0x7fe698068bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0005150 tx=0x7fe6b0005330 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:34.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.349+0000 7fe6b57fa700 1 -- 172.21.15.40:0/2368958726 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe6a8051600 con 0x7fe6b8147e50 2024-09-13T17:50:34.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.557+0000 7fe6bdda5700 1 -- 172.21.15.40:0/2368958726 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe6b8157290 con 0x7fe6b8147e50 2024-09-13T17:50:34.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.565+0000 7fe6b57fa700 1 -- 172.21.15.40:0/2368958726 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe6a8000e60 con 0x7fe6b8147e50 2024-09-13T17:50:34.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.569+0000 7fe6bdda5700 1 -- 172.21.15.40:0/2368958726 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe698066730 msgr2=0x7fe698068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:34.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.569+0000 7fe6bdda5700 1 --2- 172.21.15.40:0/2368958726 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe698066730 0x7fe698068bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0005150 tx=0x7fe6b0005330 comp rx=0 tx=0).stop 2024-09-13T17:50:34.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.569+0000 7fe6bdda5700 1 -- 172.21.15.40:0/2368958726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6b8147e50 msgr2=0x7fe6b8151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:34.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.569+0000 7fe6bdda5700 1 --2- 172.21.15.40:0/2368958726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6b8147e50 0x7fe6b8151db0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe6b806ab00 tx=0x7fe6a8011a90 comp rx=0 tx=0).stop 2024-09-13T17:50:34.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.573+0000 7fe6bdda5700 1 -- 172.21.15.40:0/2368958726 shutdown_connections 2024-09-13T17:50:34.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.573+0000 7fe6bdda5700 1 --2- 172.21.15.40:0/2368958726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6b8147e50 0x7fe6b8151db0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:34.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.573+0000 7fe6bdda5700 1 --2- 172.21.15.40:0/2368958726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b8153070 0x7fe6b8147910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:34.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.573+0000 7fe6bdda5700 1 --2- 172.21.15.40:0/2368958726 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe698066730 0x7fe698068bf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:34.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.573+0000 7fe6bdda5700 1 --2- 172.21.15.40:0/2368958726 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe6a4003f60 0x7fe6b8152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:34.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.573+0000 7fe6bdda5700 1 -- 172.21.15.40:0/2368958726 >> 172.21.15.40:0/2368958726 conn(0x7fe6b8066090 msgr2=0x7fe6b806d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:34.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.573+0000 7fe6bdda5700 1 -- 172.21.15.40:0/2368958726 shutdown_connections 2024-09-13T17:50:34.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.577+0000 7fe6bdda5700 1 -- 172.21.15.40:0/2368958726 wait complete. 2024-09-13T17:50:34.589 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 2 and data pool 3 2024-09-13T17:50:34.601 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-13T17:50:34.602 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-13T17:50:34.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.709+0000 7fb6aa7e6700 1 Processor -- start 2024-09-13T17:50:34.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.709+0000 7fb6aa7e6700 1 -- start start 2024-09-13T17:50:34.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.709+0000 7fb6aa7e6700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb6a406b600 0x7fb6a406ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:34.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.709+0000 7fb6aa7e6700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a4070730 con 0x7fb6a4153650 2024-09-13T17:50:34.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.709+0000 7fb6aa7e6700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a406bfa0 con 0x7fb6a406dbc0 2024-09-13T17:50:34.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.709+0000 7fb6aa7e6700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a406cb50 con 0x7fb6a406b600 2024-09-13T17:50:34.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.709+0000 7fb6a8fe3700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb6a406b600 0x7fb6a406ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:34.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.709+0000 7fb6a97e4700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb6a4153650 0x7fb6a4159dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35772/0 (socket says 172.21.15.40:35772) 2024-09-13T17:50:34.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.709+0000 7fb6a8fe3700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb6a406b600 0x7fb6a406ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38336/0 (socket says 172.21.15.40:38336) 2024-09-13T17:50:34.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.709+0000 7fb6a97e4700 1 -- 172.21.15.40:0/3794035685 learned_addr learned my addr 172.21.15.40:0/3794035685 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:34.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.709+0000 7fb6a8fe3700 1 -- 172.21.15.40:0/3794035685 >> v1:172.21.15.43:6789/0 conn(0x7fb6a406dbc0 legacy=0x7fb6a406e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:34.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.709+0000 7fb6a8fe3700 1 -- 172.21.15.40:0/3794035685 >> v1:172.21.15.40:6789/0 conn(0x7fb6a4153650 legacy=0x7fb6a4159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:34.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.709+0000 7fb69bfff700 1 -- 172.21.15.40:0/3794035685 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3483597057 0 0) 0x7fb6a4070730 con 0x7fb6a4153650 2024-09-13T17:50:34.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.709+0000 7fb6a8fe3700 1 -- 172.21.15.40:0/3794035685 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a406f840 con 0x7fb6a406b600 2024-09-13T17:50:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.709+0000 7fb6a8fe3700 1 --2- 172.21.15.40:0/3794035685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6a406b600 0x7fb6a406ba60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009850 tx=0x7fb6a0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cddca2cf69371a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.709+0000 7fb69bfff700 1 -- 172.21.15.40:0/3794035685 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0011830 con 0x7fb6a406b600 2024-09-13T17:50:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.709+0000 7fb69bfff700 1 -- 172.21.15.40:0/3794035685 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a0011990 con 0x7fb6a406b600 2024-09-13T17:50:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.709+0000 7fb69bfff700 1 -- 172.21.15.40:0/3794035685 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0011d20 con 0x7fb6a406b600 2024-09-13T17:50:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.709+0000 7fb6aa7e6700 1 -- 172.21.15.40:0/3794035685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6a406b600 msgr2=0x7fb6a406ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.709+0000 7fb6aa7e6700 1 --2- 172.21.15.40:0/3794035685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6a406b600 0x7fb6a406ba60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009850 tx=0x7fb6a0009b60 comp rx=0 tx=0).stop 2024-09-13T17:50:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.713+0000 7fb6aa7e6700 1 -- 172.21.15.40:0/3794035685 shutdown_connections 2024-09-13T17:50:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.713+0000 7fb6aa7e6700 1 --2- 172.21.15.40:0/3794035685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6a406b600 0x7fb6a406ba60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.713+0000 7fb6aa7e6700 1 -- 172.21.15.40:0/3794035685 >> 172.21.15.40:0/3794035685 conn(0x7fb6a40ccef0 msgr2=0x7fb6a40cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.713+0000 7fb6aa7e6700 1 -- 172.21.15.40:0/3794035685 shutdown_connections 2024-09-13T17:50:34.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.713+0000 7fb6aa7e6700 1 -- 172.21.15.40:0/3794035685 wait complete. 2024-09-13T17:50:34.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.713+0000 7fb6aa7e6700 1 Processor -- start 2024-09-13T17:50:34.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.713+0000 7fb6aa7e6700 1 -- start start 2024-09-13T17:50:34.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.713+0000 7fb6aa7e6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6a406b600 0x7fb6a41efd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:34.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.713+0000 7fb6aa7e6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6a406dbc0 0x7fb6a40c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:34.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.713+0000 7fb6aa7e6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb6a4153650 0x7fb6a414fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.713+0000 7fb6aa7e6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a41e2e60 con 0x7fb6a406b600 2024-09-13T17:50:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.713+0000 7fb6aa7e6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a40b2e60 con 0x7fb6a406dbc0 2024-09-13T17:50:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.713+0000 7fb6aa7e6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a41f0280 con 0x7fb6a4153650 2024-09-13T17:50:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.713+0000 7fb6a97e4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6a406b600 0x7fb6a41efd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.713+0000 7fb6a97e4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6a406b600 0x7fb6a41efd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38352/0 (socket says 172.21.15.40:38352) 2024-09-13T17:50:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.713+0000 7fb6a97e4700 1 -- 172.21.15.40:0/4020561947 learned_addr learned my addr 172.21.15.40:0/4020561947 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.713+0000 7fb6a8fe3700 1 --2- 172.21.15.40:0/4020561947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6a406dbc0 0x7fb6a40c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.717+0000 7fb6a9fe5700 1 --2- 172.21.15.40:0/4020561947 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb6a4153650 0x7fb6a414fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.717+0000 7fb6a97e4700 1 -- 172.21.15.40:0/4020561947 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb6a4153650 msgr2=0x7fb6a414fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.717+0000 7fb6a97e4700 1 --2- 172.21.15.40:0/4020561947 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb6a4153650 0x7fb6a414fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.717+0000 7fb6a97e4700 1 -- 172.21.15.40:0/4020561947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6a406dbc0 msgr2=0x7fb6a40c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:34.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.717+0000 7fb6a97e4700 1 --2- 172.21.15.40:0/4020561947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6a406dbc0 0x7fb6a40c6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:34.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.717+0000 7fb6a97e4700 1 -- 172.21.15.40:0/4020561947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a00094b0 con 0x7fb6a406b600 2024-09-13T17:50:34.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.717+0000 7fb6a97e4700 1 --2- 172.21.15.40:0/4020561947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6a406b600 0x7fb6a41efd40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb69400c4a0 tx=0x7fb69400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:34.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.717+0000 7fb699ffb700 1 -- 172.21.15.40:0/4020561947 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb694004ca0 con 0x7fb6a406b600 2024-09-13T17:50:34.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.717+0000 7fb699ffb700 1 -- 172.21.15.40:0/4020561947 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb694004e00 con 0x7fb6a406b600 2024-09-13T17:50:34.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.717+0000 7fb6aa7e6700 1 -- 172.21.15.40:0/4020561947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a40c5340 con 0x7fb6a406b600 2024-09-13T17:50:34.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.717+0000 7fb699ffb700 1 -- 172.21.15.40:0/4020561947 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb694007310 con 0x7fb6a406b600 2024-09-13T17:50:34.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.717+0000 7fb6aa7e6700 1 -- 172.21.15.40:0/4020561947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a40c61a0 con 0x7fb6a406b600 2024-09-13T17:50:34.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.717+0000 7fb6aa7e6700 1 -- 172.21.15.40:0/4020561947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a4153a30 con 0x7fb6a406b600 2024-09-13T17:50:34.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.717+0000 7fb699ffb700 1 -- 172.21.15.40:0/4020561947 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb694007470 con 0x7fb6a406b600 2024-09-13T17:50:34.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.721+0000 7fb699ffb700 1 --2- 172.21.15.40:0/4020561947 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb688066730 0x7fb688068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:34.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.721+0000 7fb699ffb700 1 -- 172.21.15.40:0/4020561947 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb694083d20 con 0x7fb6a406b600 2024-09-13T17:50:34.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.721+0000 7fb6a8fe3700 1 --2- 172.21.15.40:0/4020561947 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb688066730 0x7fb688068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:34.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.721+0000 7fb6a8fe3700 1 --2- 172.21.15.40:0/4020561947 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb688066730 0x7fb688068bf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0014040 tx=0x7fb6a0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:34.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.725+0000 7fb699ffb700 1 -- 172.21.15.40:0/4020561947 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb6940510a0 con 0x7fb6a406b600 2024-09-13T17:50:34.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:34.897+0000 7fb6aa7e6700 1 -- 172.21.15.40:0/4020561947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fb6a406c3d0 con 0x7fb6a406b600 2024-09-13T17:50:35.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.573+0000 7fb699ffb700 1 -- 172.21.15.40:0/4020561947 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7fb694007720 con 0x7fb6a406b600 2024-09-13T17:50:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.577+0000 7fb6aa7e6700 1 -- 172.21.15.40:0/4020561947 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb688066730 msgr2=0x7fb688068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.577+0000 7fb6aa7e6700 1 --2- 172.21.15.40:0/4020561947 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb688066730 0x7fb688068bf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0014040 tx=0x7fb6a0008030 comp rx=0 tx=0).stop 2024-09-13T17:50:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.577+0000 7fb6aa7e6700 1 -- 172.21.15.40:0/4020561947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6a406b600 msgr2=0x7fb6a41efd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.577+0000 7fb6aa7e6700 1 --2- 172.21.15.40:0/4020561947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6a406b600 0x7fb6a41efd40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb69400c4a0 tx=0x7fb69400c7b0 comp rx=0 tx=0).stop 2024-09-13T17:50:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.577+0000 7fb6aa7e6700 1 -- 172.21.15.40:0/4020561947 shutdown_connections 2024-09-13T17:50:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.577+0000 7fb6aa7e6700 1 --2- 172.21.15.40:0/4020561947 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb688066730 0x7fb688068bf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:35.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.577+0000 7fb6aa7e6700 1 --2- 172.21.15.40:0/4020561947 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb6a4153650 0x7fb6a414fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.577+0000 7fb6aa7e6700 1 --2- 172.21.15.40:0/4020561947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6a406dbc0 0x7fb6a40c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.577+0000 7fb6aa7e6700 1 --2- 172.21.15.40:0/4020561947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6a406b600 0x7fb6a41efd40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.577+0000 7fb6aa7e6700 1 -- 172.21.15.40:0/4020561947 >> 172.21.15.40:0/4020561947 conn(0x7fb6a40ccef0 msgr2=0x7fb6a41517e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.577+0000 7fb6aa7e6700 1 -- 172.21.15.40:0/4020561947 shutdown_connections 2024-09-13T17:50:35.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.577+0000 7fb6aa7e6700 1 -- 172.21.15.40:0/4020561947 wait complete. 2024-09-13T17:50:35.602 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T17:50:35.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.729+0000 7f8af4a3f700 1 Processor -- start 2024-09-13T17:50:35.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.733+0000 7f8af4a3f700 1 -- start start 2024-09-13T17:50:35.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.733+0000 7f8af4a3f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8af00c6ee0 0x7f8af015a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:35.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.733+0000 7f8af4a3f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af015a640 con 0x7f8af0156b60 2024-09-13T17:50:35.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.733+0000 7f8af4a3f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af00c7740 con 0x7f8af00c7930 2024-09-13T17:50:35.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.733+0000 7f8af4a3f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af00ccd30 con 0x7f8af00c6ee0 2024-09-13T17:50:35.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.733+0000 7f8aef7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8af0156b60 0x7f8af00c7e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35784/0 (socket says 172.21.15.40:35784) 2024-09-13T17:50:35.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.733+0000 7f8aef7fe700 1 -- 172.21.15.40:0/1392123698 learned_addr learned my addr 172.21.15.40:0/1392123698 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:35.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.733+0000 7f8aeffff700 1 --2- 172.21.15.40:0/1392123698 >> v2:172.21.15.43:3300/0 conn(0x7f8af00c6ee0 0x7f8af015a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:35.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.733+0000 7f8aeeffd700 1 -- 172.21.15.40:0/1392123698 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4123405537 0 0) 0x7f8ae0001110 con 0x7f8af0156b60 2024-09-13T17:50:35.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.733+0000 7f8aeeffd700 1 -- 172.21.15.40:0/1392123698 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8adc0035d0 con 0x7f8af0156b60 2024-09-13T17:50:35.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.733+0000 7f8aeffff700 1 -- 172.21.15.40:0/1392123698 >> v1:172.21.15.43:6789/0 conn(0x7f8af00c7930 legacy=0x7f8af0161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:35.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.733+0000 7f8aeffff700 1 -- 172.21.15.40:0/1392123698 >> v1:172.21.15.40:6789/0 conn(0x7f8af0156b60 legacy=0x7f8af00c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:35.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.733+0000 7f8aeffff700 1 -- 172.21.15.40:0/1392123698 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af00c69a0 con 0x7f8af00c6ee0 2024-09-13T17:50:35.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.733+0000 7f8aeffff700 1 --2- 172.21.15.40:0/1392123698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8af00c6ee0 0x7f8af015a100 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009d90 tx=0x7f8ae800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=797f2fc834392e54 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:35.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.737+0000 7f8aeeffd700 1 -- 172.21.15.40:0/1392123698 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae800e070 con 0x7f8af00c6ee0 2024-09-13T17:50:35.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.737+0000 7f8aeeffd700 1 -- 172.21.15.40:0/1392123698 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae800bb20 con 0x7f8af00c6ee0 2024-09-13T17:50:35.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.737+0000 7f8aeeffd700 1 -- 172.21.15.40:0/1392123698 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae800be90 con 0x7f8af00c6ee0 2024-09-13T17:50:35.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.737+0000 7f8af4a3f700 1 -- 172.21.15.40:0/1392123698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8af00c6ee0 msgr2=0x7f8af015a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:35.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.737+0000 7f8af4a3f700 1 --2- 172.21.15.40:0/1392123698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8af00c6ee0 0x7f8af015a100 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009d90 tx=0x7f8ae800b330 comp rx=0 tx=0).stop 2024-09-13T17:50:35.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.737+0000 7f8af4a3f700 1 -- 172.21.15.40:0/1392123698 shutdown_connections 2024-09-13T17:50:35.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.737+0000 7f8af4a3f700 1 --2- 172.21.15.40:0/1392123698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8af00c6ee0 0x7f8af015a100 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:35.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.737+0000 7f8af4a3f700 1 -- 172.21.15.40:0/1392123698 >> 172.21.15.40:0/1392123698 conn(0x7f8af006e580 msgr2=0x7f8af006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:35.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.737+0000 7f8af4a3f700 1 -- 172.21.15.40:0/1392123698 shutdown_connections 2024-09-13T17:50:35.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.737+0000 7f8af4a3f700 1 -- 172.21.15.40:0/1392123698 wait complete. 2024-09-13T17:50:35.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.737+0000 7f8af4a3f700 1 Processor -- start 2024-09-13T17:50:35.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.737+0000 7f8af4a3f700 1 -- start start 2024-09-13T17:50:35.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8af4a3f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8af00c6ee0 0x7f8af01ef7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:35.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8af4a3f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8af00c7930 0x7f8af0148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:35.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8af4a3f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8af0156b60 0x7f8af01f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:35.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8af4a3f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af0154f30 con 0x7f8af00c7930 2024-09-13T17:50:35.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8af4a3f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af01f2310 con 0x7f8af00c6ee0 2024-09-13T17:50:35.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8af4a3f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af01efd00 con 0x7f8af0156b60 2024-09-13T17:50:35.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8ae6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8af00c7930 0x7f8af0148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:35.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8ae6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8af00c7930 0x7f8af0148ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38362/0 (socket says 172.21.15.40:38362) 2024-09-13T17:50:35.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8ae6ffd700 1 -- 172.21.15.40:0/1683963526 learned_addr learned my addr 172.21.15.40:0/1683963526 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:35.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8aef7fe700 1 --2- 172.21.15.40:0/1683963526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8af00c6ee0 0x7f8af01ef7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:35.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8aeffff700 1 --2- 172.21.15.40:0/1683963526 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8af0156b60 0x7f8af01f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:35.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8ae6ffd700 1 -- 172.21.15.40:0/1683963526 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8af0156b60 msgr2=0x7f8af01f2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:35.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8ae6ffd700 1 --2- 172.21.15.40:0/1683963526 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8af0156b60 0x7f8af01f2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:35.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8ae6ffd700 1 -- 172.21.15.40:0/1683963526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8af00c6ee0 msgr2=0x7f8af01ef7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:35.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8ae6ffd700 1 --2- 172.21.15.40:0/1683963526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8af00c6ee0 0x7f8af01ef7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:35.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8ae6ffd700 1 -- 172.21.15.40:0/1683963526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae8009a70 con 0x7f8af00c7930 2024-09-13T17:50:35.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8aef7fe700 1 --2- 172.21.15.40:0/1683963526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8af00c6ee0 0x7f8af01ef7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:50:35.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8ae6ffd700 1 --2- 172.21.15.40:0/1683963526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8af00c7930 0x7f8af0148ae0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800c410 tx=0x7f8ad800c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:35.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8aecff9700 1 -- 172.21.15.40:0/1683963526 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad80052d0 con 0x7f8af00c7930 2024-09-13T17:50:35.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8aecff9700 1 -- 172.21.15.40:0/1683963526 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad8005430 con 0x7f8af00c7930 2024-09-13T17:50:35.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8aecff9700 1 -- 172.21.15.40:0/1683963526 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad80055a0 con 0x7f8af00c7930 2024-09-13T17:50:35.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8af4a3f700 1 -- 172.21.15.40:0/1683963526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af006af60 con 0x7f8af00c7930 2024-09-13T17:50:35.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.741+0000 7f8af4a3f700 1 -- 172.21.15.40:0/1683963526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af006a530 con 0x7f8af00c7930 2024-09-13T17:50:35.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.745+0000 7f8af4a3f700 1 -- 172.21.15.40:0/1683963526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af00b19f0 con 0x7f8af00c7930 2024-09-13T17:50:35.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.745+0000 7f8aecff9700 1 -- 172.21.15.40:0/1683963526 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8ad80036a0 con 0x7f8af00c7930 2024-09-13T17:50:35.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.745+0000 7f8aecff9700 1 --2- 172.21.15.40:0/1683963526 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ad0066760 0x7f8ad0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:35.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.745+0000 7f8aef7fe700 1 --2- 172.21.15.40:0/1683963526 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ad0066760 0x7f8ad0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:35.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.745+0000 7f8aecff9700 1 -- 172.21.15.40:0/1683963526 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5935+0+0 (secure 0 0 0) 0x7f8ad8083410 con 0x7f8af00c7930 2024-09-13T17:50:35.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.745+0000 7f8aef7fe700 1 --2- 172.21.15.40:0/1683963526 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ad0066760 0x7f8ad0068c20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0007fd0 tx=0x7f8ae0007c60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:35.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.757+0000 7f8aecff9700 1 -- 172.21.15.40:0/1683963526 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8ad8050600 con 0x7f8af00c7930 2024-09-13T17:50:35.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:35.925+0000 7f8af4a3f700 1 -- 172.21.15.40:0/1683963526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8af00b1340 con 0x7f8af00c7930 2024-09-13T17:50:37.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.581+0000 7f8aecff9700 1 -- 172.21.15.40:0/1683963526 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7f8ad8053c20 con 0x7f8af00c7930 2024-09-13T17:50:37.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.589+0000 7f8af4a3f700 1 -- 172.21.15.40:0/1683963526 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ad0066760 msgr2=0x7f8ad0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:37.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.589+0000 7f8af4a3f700 1 --2- 172.21.15.40:0/1683963526 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ad0066760 0x7f8ad0068c20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0007fd0 tx=0x7f8ae0007c60 comp rx=0 tx=0).stop 2024-09-13T17:50:37.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.589+0000 7f8af4a3f700 1 -- 172.21.15.40:0/1683963526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8af00c7930 msgr2=0x7f8af0148ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:37.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.589+0000 7f8af4a3f700 1 --2- 172.21.15.40:0/1683963526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8af00c7930 0x7f8af0148ae0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800c410 tx=0x7f8ad800c720 comp rx=0 tx=0).stop 2024-09-13T17:50:37.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.593+0000 7f8af4a3f700 1 -- 172.21.15.40:0/1683963526 shutdown_connections 2024-09-13T17:50:37.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.593+0000 7f8af4a3f700 1 --2- 172.21.15.40:0/1683963526 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ad0066760 0x7f8ad0068c20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:37.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.593+0000 7f8af4a3f700 1 --2- 172.21.15.40:0/1683963526 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8af0156b60 0x7f8af01f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:37.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.593+0000 7f8af4a3f700 1 --2- 172.21.15.40:0/1683963526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8af00c7930 0x7f8af0148ae0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:37.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.593+0000 7f8af4a3f700 1 --2- 172.21.15.40:0/1683963526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8af00c6ee0 0x7f8af01ef7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:37.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.593+0000 7f8af4a3f700 1 -- 172.21.15.40:0/1683963526 >> 172.21.15.40:0/1683963526 conn(0x7f8af006e580 msgr2=0x7f8af00c0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:37.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.593+0000 7f8af4a3f700 1 -- 172.21.15.40:0/1683963526 shutdown_connections 2024-09-13T17:50:37.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.593+0000 7f8af4a3f700 1 -- 172.21.15.40:0/1683963526 wait complete. 2024-09-13T17:50:37.603 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data_ec' created 2024-09-13T17:50:37.616 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-13T17:50:37.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.729+0000 7fe5928dc700 1 Processor -- start 2024-09-13T17:50:37.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.729+0000 7fe5928dc700 1 -- start start 2024-09-13T17:50:37.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5928dc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe58c0c3e90 0x7fe58c0c4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:37.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5928dc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe58c0c9e00 0x7fe58c0cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:37.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5928dc700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe58c15a2c0 con 0x7fe58c0cae70 2024-09-13T17:50:37.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5928dc700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe58c0cfd90 con 0x7fe58c0c9e00 2024-09-13T17:50:37.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5928dc700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe58c0c47b0 con 0x7fe58c0c3e90 2024-09-13T17:50:37.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5910d9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe58c0c9e00 0x7fe58c0cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:37.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5920db700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe58c0cae70 0x7fe58c15c970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35800/0 (socket says 172.21.15.40:35800) 2024-09-13T17:50:37.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5920db700 1 -- 172.21.15.40:0/435993729 learned_addr learned my addr 172.21.15.40:0/435993729 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:37.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5918da700 1 --2- 172.21.15.40:0/435993729 >> v2:172.21.15.43:3300/0 conn(0x7fe58c0c3e90 0x7fe58c0c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:37.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5910d9700 1 -- 172.21.15.40:0/435993729 >> v2:172.21.15.43:3300/0 conn(0x7fe58c0c3e90 msgr2=0x7fe58c0c4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:37.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5910d9700 1 --2- 172.21.15.40:0/435993729 >> v2:172.21.15.43:3300/0 conn(0x7fe58c0c3e90 0x7fe58c0c4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:37.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5910d9700 1 -- 172.21.15.40:0/435993729 >> v1:172.21.15.40:6789/0 conn(0x7fe58c0cae70 legacy=0x7fe58c15c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:37.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5908d8700 1 -- 172.21.15.40:0/435993729 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965287135 0 0) 0x7fe58c15a2c0 con 0x7fe58c0cae70 2024-09-13T17:50:37.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5910d9700 1 -- 172.21.15.40:0/435993729 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58c0ca310 con 0x7fe58c0c9e00 2024-09-13T17:50:37.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5918da700 1 --2- 172.21.15.40:0/435993729 >> v2:172.21.15.43:3300/0 conn(0x7fe58c0c3e90 0x7fe58c0c4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:50:37.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5910d9700 1 --2- 172.21.15.40:0/435993729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58c0c9e00 0x7fe58c0cb8a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe58c08c380 tx=0x7fe57c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eafec0a04a512ac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:37.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5908d8700 1 -- 172.21.15.40:0/435993729 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c011830 con 0x7fe58c0c9e00 2024-09-13T17:50:37.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5908d8700 1 -- 172.21.15.40:0/435993729 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c011990 con 0x7fe58c0c9e00 2024-09-13T17:50:37.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5908d8700 1 -- 172.21.15.40:0/435993729 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c011d40 con 0x7fe58c0c9e00 2024-09-13T17:50:37.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5928dc700 1 -- 172.21.15.40:0/435993729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58c0c9e00 msgr2=0x7fe58c0cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:37.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5928dc700 1 --2- 172.21.15.40:0/435993729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58c0c9e00 0x7fe58c0cb8a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe58c08c380 tx=0x7fe57c009b30 comp rx=0 tx=0).stop 2024-09-13T17:50:37.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5928dc700 1 -- 172.21.15.40:0/435993729 shutdown_connections 2024-09-13T17:50:37.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5928dc700 1 --2- 172.21.15.40:0/435993729 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58c0c9e00 0x7fe58c0cb8a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:37.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5928dc700 1 --2- 172.21.15.40:0/435993729 >> v2:172.21.15.43:3300/0 conn(0x7fe58c0c3e90 0x7fe58c0c4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:37.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5928dc700 1 -- 172.21.15.40:0/435993729 >> 172.21.15.40:0/435993729 conn(0x7fe58c066270 msgr2=0x7fe58c066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:37.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5928dc700 1 -- 172.21.15.40:0/435993729 shutdown_connections 2024-09-13T17:50:37.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.733+0000 7fe5928dc700 1 -- 172.21.15.40:0/435993729 wait complete. 2024-09-13T17:50:37.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.737+0000 7fe5928dc700 1 Processor -- start 2024-09-13T17:50:37.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.737+0000 7fe5928dc700 1 -- start start 2024-09-13T17:50:37.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.737+0000 7fe5928dc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe58c0c3e90 0x7fe58c140bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:37.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5928dc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe58c0c9e00 0x7fe58c0cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:37.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5928dc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58c0cae70 0x7fe58c147260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:37.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5928dc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c0cd130 con 0x7fe58c0cae70 2024-09-13T17:50:37.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5928dc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c0ce6e0 con 0x7fe58c0c3e90 2024-09-13T17:50:37.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5928dc700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58c0b3d00 con 0x7fe58c0c9e00 2024-09-13T17:50:37.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5920db700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58c0cae70 0x7fe58c147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:37.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5918da700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe58c0c3e90 0x7fe58c140bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:37.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5920db700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58c0cae70 0x7fe58c147260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38370/0 (socket says 172.21.15.40:38370) 2024-09-13T17:50:37.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5910d9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe58c0c9e00 0x7fe58c0cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:37.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5920db700 1 -- 172.21.15.40:0/1737836200 learned_addr learned my addr 172.21.15.40:0/1737836200 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:37.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5918da700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe58c0c3e90 0x7fe58c140bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:55456/0 (socket says 172.21.15.40:55456) 2024-09-13T17:50:37.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5920db700 1 -- 172.21.15.40:0/1737836200 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe58c0c9e00 msgr2=0x7fe58c0cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:37.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5920db700 1 --2- 172.21.15.40:0/1737836200 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe58c0c9e00 0x7fe58c0cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:37.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5920db700 1 -- 172.21.15.40:0/1737836200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe58c0c3e90 msgr2=0x7fe58c140bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:37.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5920db700 1 --2- 172.21.15.40:0/1737836200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe58c0c3e90 0x7fe58c140bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:37.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5920db700 1 -- 172.21.15.40:0/1737836200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c0094b0 con 0x7fe58c0cae70 2024-09-13T17:50:37.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5920db700 1 --2- 172.21.15.40:0/1737836200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58c0cae70 0x7fe58c147260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe58400cb60 tx=0x7fe58400ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:37.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5827fc700 1 -- 172.21.15.40:0/1737836200 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58400a9e0 con 0x7fe58c0cae70 2024-09-13T17:50:37.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5827fc700 1 -- 172.21.15.40:0/1737836200 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58400ab40 con 0x7fe58c0cae70 2024-09-13T17:50:37.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5928dc700 1 -- 172.21.15.40:0/1737836200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58c0b4ba0 con 0x7fe58c0cae70 2024-09-13T17:50:37.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5827fc700 1 -- 172.21.15.40:0/1737836200 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5840070d0 con 0x7fe58c0cae70 2024-09-13T17:50:37.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5928dc700 1 -- 172.21.15.40:0/1737836200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58c146a10 con 0x7fe58c0cae70 2024-09-13T17:50:37.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.741+0000 7fe5928dc700 1 -- 172.21.15.40:0/1737836200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe58c0ce820 con 0x7fe58c0cae70 2024-09-13T17:50:37.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.745+0000 7fe5827fc700 1 -- 172.21.15.40:0/1737836200 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe584007230 con 0x7fe58c0cae70 2024-09-13T17:50:37.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.745+0000 7fe5827fc700 1 --2- 172.21.15.40:0/1737836200 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe570066730 0x7fe570068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:37.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.745+0000 7fe5827fc700 1 -- 172.21.15.40:0/1737836200 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7fe584083e60 con 0x7fe58c0cae70 2024-09-13T17:50:37.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.745+0000 7fe5918da700 1 --2- 172.21.15.40:0/1737836200 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe570066730 0x7fe570068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:37.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.745+0000 7fe5918da700 1 --2- 172.21.15.40:0/1737836200 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe570066730 0x7fe570068bf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe5880099f0 tx=0x7fe588008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:37.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.753+0000 7fe5827fc700 1 -- 172.21.15.40:0/1737836200 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe58c0ce820 con 0x7fe58c0cae70 2024-09-13T17:50:37.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:37.917+0000 7fe5928dc700 1 -- 172.21.15.40:0/1737836200 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fe58c0b4ba0 con 0x7fe58c0cae70 2024-09-13T17:50:38.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.585+0000 7fe5827fc700 1 -- 172.21.15.40:0/1737836200 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe58c0b4ba0 con 0x7fe58c0cae70 2024-09-13T17:50:38.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.597+0000 7fe5928dc700 1 -- 172.21.15.40:0/1737836200 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe570066730 msgr2=0x7fe570068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:38.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.597+0000 7fe5928dc700 1 --2- 172.21.15.40:0/1737836200 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe570066730 0x7fe570068bf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe5880099f0 tx=0x7fe588008040 comp rx=0 tx=0).stop 2024-09-13T17:50:38.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.597+0000 7fe5928dc700 1 -- 172.21.15.40:0/1737836200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58c0cae70 msgr2=0x7fe58c147260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:38.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.597+0000 7fe5928dc700 1 --2- 172.21.15.40:0/1737836200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58c0cae70 0x7fe58c147260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe58400cb60 tx=0x7fe58400ce70 comp rx=0 tx=0).stop 2024-09-13T17:50:38.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.597+0000 7fe5928dc700 1 -- 172.21.15.40:0/1737836200 shutdown_connections 2024-09-13T17:50:38.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.597+0000 7fe5928dc700 1 --2- 172.21.15.40:0/1737836200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58c0cae70 0x7fe58c147260 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:38.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.597+0000 7fe5928dc700 1 --2- 172.21.15.40:0/1737836200 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe570066730 0x7fe570068bf0 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe5880099f0 tx=0x7fe588008040 comp rx=0 tx=0).stop 2024-09-13T17:50:38.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.597+0000 7fe5928dc700 1 --2- 172.21.15.40:0/1737836200 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe58c0c9e00 0x7fe58c0cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:38.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.597+0000 7fe5928dc700 1 --2- 172.21.15.40:0/1737836200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe58c0c3e90 0x7fe58c140bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:38.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.597+0000 7fe5928dc700 1 -- 172.21.15.40:0/1737836200 >> 172.21.15.40:0/1737836200 conn(0x7fe58c066270 msgr2=0x7fe58c1e4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:38.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.601+0000 7fe5928dc700 1 -- 172.21.15.40:0/1737836200 shutdown_connections 2024-09-13T17:50:38.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.601+0000 7fe5928dc700 1 -- 172.21.15.40:0/1737836200 wait complete. 2024-09-13T17:50:38.620 INFO:teuthology.orchestra.run.smithi040.stderr:set pool 4 allow_ec_overwrites to true 2024-09-13T17:50:38.633 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-13T17:50:38.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.737+0000 7f1b54b51700 1 Processor -- start 2024-09-13T17:50:38.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.737+0000 7f1b54b51700 1 -- start start 2024-09-13T17:50:38.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b54b51700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1b480d6670 0x7f1b480e0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:38.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b54b51700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1b48046df0 0x7f1b4804de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:38.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b54b51700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b480dd3c0 con 0x7f1b4804d0f0 2024-09-13T17:50:38.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b54b51700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b480e0180 con 0x7f1b48046df0 2024-09-13T17:50:38.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b54b51700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b480dcb80 con 0x7f1b480d6670 2024-09-13T17:50:38.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b4effd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1b4804d0f0 0x7f1b480dc470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35812/0 (socket says 172.21.15.40:35812) 2024-09-13T17:50:38.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b4effd700 1 -- 172.21.15.40:0/1350676653 learned_addr learned my addr 172.21.15.40:0/1350676653 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:38.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b4ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1b48046df0 0x7f1b4804de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:38.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b4f7fe700 1 --2- 172.21.15.40:0/1350676653 >> v2:172.21.15.43:3300/0 conn(0x7f1b480d6670 0x7f1b480e0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:38.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b4e7fc700 1 -- 172.21.15.40:0/1350676653 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1734035903 0 0) 0x7f1b480dd3c0 con 0x7f1b4804d0f0 2024-09-13T17:50:38.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b4ffff700 1 -- 172.21.15.40:0/1350676653 >> v2:172.21.15.43:3300/0 conn(0x7f1b480d6670 msgr2=0x7f1b480e0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:38.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b4ffff700 1 --2- 172.21.15.40:0/1350676653 >> v2:172.21.15.43:3300/0 conn(0x7f1b480d6670 0x7f1b480e0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:38.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b4ffff700 1 -- 172.21.15.40:0/1350676653 >> v1:172.21.15.40:6789/0 conn(0x7f1b4804d0f0 legacy=0x7f1b480dc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:38.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b4ffff700 1 -- 172.21.15.40:0/1350676653 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b48047220 con 0x7f1b48046df0 2024-09-13T17:50:38.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b4ffff700 1 --2- 172.21.15.40:0/1350676653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b48046df0 0x7f1b4804de20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1b48048f90 tx=0x7f1b44009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6596589ccccf632f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:38.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b4e7fc700 1 -- 172.21.15.40:0/1350676653 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44011830 con 0x7f1b48046df0 2024-09-13T17:50:38.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b4e7fc700 1 -- 172.21.15.40:0/1350676653 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b44011990 con 0x7f1b48046df0 2024-09-13T17:50:38.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b4e7fc700 1 -- 172.21.15.40:0/1350676653 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44011d20 con 0x7f1b48046df0 2024-09-13T17:50:38.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b54b51700 1 -- 172.21.15.40:0/1350676653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b48046df0 msgr2=0x7f1b4804de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:38.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b54b51700 1 --2- 172.21.15.40:0/1350676653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b48046df0 0x7f1b4804de20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1b48048f90 tx=0x7f1b44009b30 comp rx=0 tx=0).stop 2024-09-13T17:50:38.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b54b51700 1 -- 172.21.15.40:0/1350676653 shutdown_connections 2024-09-13T17:50:38.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b54b51700 1 --2- 172.21.15.40:0/1350676653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b48046df0 0x7f1b4804de20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:38.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b54b51700 1 --2- 172.21.15.40:0/1350676653 >> v2:172.21.15.43:3300/0 conn(0x7f1b480d6670 0x7f1b480e0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:38.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b54b51700 1 -- 172.21.15.40:0/1350676653 >> 172.21.15.40:0/1350676653 conn(0x7f1b4800dc40 msgr2=0x7f1b4800e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:38.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b54b51700 1 -- 172.21.15.40:0/1350676653 shutdown_connections 2024-09-13T17:50:38.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.741+0000 7f1b54b51700 1 -- 172.21.15.40:0/1350676653 wait complete. 2024-09-13T17:50:38.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.745+0000 7f1b54b51700 1 Processor -- start 2024-09-13T17:50:38.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.745+0000 7f1b54b51700 1 -- start start 2024-09-13T17:50:38.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.745+0000 7f1b54b51700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b48046df0 0x7f1b48009200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:38.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.745+0000 7f1b54b51700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b4804d0f0 0x7f1b480cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:38.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.745+0000 7f1b54b51700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b480d6670 0x7f1b480c27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:38.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.745+0000 7f1b54b51700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480dead0 con 0x7f1b480d6670 2024-09-13T17:50:38.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.745+0000 7f1b54b51700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480dedb0 con 0x7f1b4804d0f0 2024-09-13T17:50:38.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.745+0000 7f1b54b51700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480df090 con 0x7f1b48046df0 2024-09-13T17:50:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.745+0000 7f1b4ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b480d6670 0x7f1b480c27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.745+0000 7f1b4ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b480d6670 0x7f1b480c27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38388/0 (socket says 172.21.15.40:38388) 2024-09-13T17:50:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.745+0000 7f1b4ffff700 1 -- 172.21.15.40:0/1607402051 learned_addr learned my addr 172.21.15.40:0/1607402051 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.745+0000 7f1b4f7fe700 1 --2- 172.21.15.40:0/1607402051 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b48046df0 0x7f1b48009200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.745+0000 7f1b4effd700 1 --2- 172.21.15.40:0/1607402051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b4804d0f0 0x7f1b480cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.745+0000 7f1b4ffff700 1 -- 172.21.15.40:0/1607402051 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b48046df0 msgr2=0x7f1b48009200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.745+0000 7f1b4ffff700 1 --2- 172.21.15.40:0/1607402051 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b48046df0 0x7f1b48009200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.745+0000 7f1b4ffff700 1 -- 172.21.15.40:0/1607402051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b4804d0f0 msgr2=0x7f1b480cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.745+0000 7f1b4ffff700 1 --2- 172.21.15.40:0/1607402051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b4804d0f0 0x7f1b480cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.745+0000 7f1b4ffff700 1 -- 172.21.15.40:0/1607402051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b440094b0 con 0x7f1b480d6670 2024-09-13T17:50:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.745+0000 7f1b4ffff700 1 --2- 172.21.15.40:0/1607402051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b480d6670 0x7f1b480c27b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009b70 tx=0x7f1b44004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:38.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.749+0000 7f1b37fff700 1 -- 172.21.15.40:0/1607402051 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44004380 con 0x7f1b480d6670 2024-09-13T17:50:38.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.749+0000 7f1b37fff700 1 -- 172.21.15.40:0/1607402051 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4401f070 con 0x7f1b480d6670 2024-09-13T17:50:38.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.749+0000 7f1b54b51700 1 -- 172.21.15.40:0/1607402051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b480df370 con 0x7f1b480d6670 2024-09-13T17:50:38.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.749+0000 7f1b37fff700 1 -- 172.21.15.40:0/1607402051 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440036c0 con 0x7f1b480d6670 2024-09-13T17:50:38.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.749+0000 7f1b54b51700 1 -- 172.21.15.40:0/1607402051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b48037ac0 con 0x7f1b480d6670 2024-09-13T17:50:38.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.749+0000 7f1b54b51700 1 -- 172.21.15.40:0/1607402051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b480077b0 con 0x7f1b480d6670 2024-09-13T17:50:38.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.749+0000 7f1b37fff700 1 -- 172.21.15.40:0/1607402051 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1b44021030 con 0x7f1b480d6670 2024-09-13T17:50:38.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.749+0000 7f1b37fff700 1 --2- 172.21.15.40:0/1607402051 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b3c066730 0x7f1b3c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:38.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.749+0000 7f1b4f7fe700 1 --2- 172.21.15.40:0/1607402051 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b3c066730 0x7f1b3c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:38.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.749+0000 7f1b37fff700 1 -- 172.21.15.40:0/1607402051 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f1b44085840 con 0x7f1b480d6670 2024-09-13T17:50:38.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.753+0000 7f1b4f7fe700 1 --2- 172.21.15.40:0/1607402051 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b3c066730 0x7f1b3c068bf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009b50 tx=0x7f1b40009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:38.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.757+0000 7f1b37fff700 1 -- 172.21.15.40:0/1607402051 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1b44052a90 con 0x7f1b480d6670 2024-09-13T17:50:38.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.965+0000 7f1b54b51700 1 -- 172.21.15.40:0/1607402051 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f1b4804ce50 con 0x7f1b480d6670 2024-09-13T17:50:38.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.977+0000 7f1b37fff700 1 -- 172.21.15.40:0/1607402051 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v5) v1 ==== 137+0+0 (secure 0 0 0) 0x7f1b440560b0 con 0x7f1b480d6670 2024-09-13T17:50:38.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.977+0000 7f1b54b51700 1 -- 172.21.15.40:0/1607402051 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b3c066730 msgr2=0x7f1b3c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:38.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.977+0000 7f1b54b51700 1 --2- 172.21.15.40:0/1607402051 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b3c066730 0x7f1b3c068bf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009b50 tx=0x7f1b40009240 comp rx=0 tx=0).stop 2024-09-13T17:50:38.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.977+0000 7f1b54b51700 1 -- 172.21.15.40:0/1607402051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b480d6670 msgr2=0x7f1b480c27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:38.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.977+0000 7f1b54b51700 1 --2- 172.21.15.40:0/1607402051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b480d6670 0x7f1b480c27b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009b70 tx=0x7f1b44004770 comp rx=0 tx=0).stop 2024-09-13T17:50:38.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.977+0000 7f1b54b51700 1 -- 172.21.15.40:0/1607402051 shutdown_connections 2024-09-13T17:50:38.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.977+0000 7f1b54b51700 1 --2- 172.21.15.40:0/1607402051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b480d6670 0x7f1b480c27b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:38.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.977+0000 7f1b54b51700 1 --2- 172.21.15.40:0/1607402051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b4804d0f0 0x7f1b480cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:38.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.977+0000 7f1b54b51700 1 --2- 172.21.15.40:0/1607402051 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b3c066730 0x7f1b3c068bf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:38.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.977+0000 7f1b54b51700 1 --2- 172.21.15.40:0/1607402051 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b48046df0 0x7f1b48009200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:38.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.977+0000 7f1b54b51700 1 -- 172.21.15.40:0/1607402051 >> 172.21.15.40:0/1607402051 conn(0x7f1b4800dc40 msgr2=0x7f1b480d55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:38.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.977+0000 7f1b54b51700 1 -- 172.21.15.40:0/1607402051 shutdown_connections 2024-09-13T17:50:38.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:38.977+0000 7f1b54b51700 1 -- 172.21.15.40:0/1607402051 wait complete. 2024-09-13T17:50:38.987 INFO:teuthology.orchestra.run.smithi040.stderr:added data pool 4 to fsmap 2024-09-13T17:50:39.002 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:50:39.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.101+0000 7f9caa478700 1 Processor -- start 2024-09-13T17:50:39.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.101+0000 7f9caa478700 1 -- start start 2024-09-13T17:50:39.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.101+0000 7f9caa478700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9ca40cc4f0 0x7f9ca40c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:39.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.101+0000 7f9caa478700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9ca40c9e20 0x7f9ca40cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:39.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.101+0000 7f9caa478700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca40cc060 con 0x7f9ca4153840 2024-09-13T17:50:39.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.101+0000 7f9caa478700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca40b2840 con 0x7f9ca40c9e20 2024-09-13T17:50:39.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.101+0000 7f9caa478700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca40ca870 con 0x7f9ca40cc4f0 2024-09-13T17:50:39.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.101+0000 7f9ca9c77700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9ca40c9e20 0x7f9ca40cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:39.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.101+0000 7f9ca9c77700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9ca40c9e20 0x7f9ca40cf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38398/0 (socket says 172.21.15.40:38398) 2024-09-13T17:50:39.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.101+0000 7f9ca9476700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9ca4153840 0x7f9ca415d410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35818/0 (socket says 172.21.15.40:35818) 2024-09-13T17:50:39.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.101+0000 7f9ca9c77700 1 -- 172.21.15.40:0/2388903599 learned_addr learned my addr 172.21.15.40:0/2388903599 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:39.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.101+0000 7f9ca8c75700 1 --2- 172.21.15.40:0/2388903599 >> v2:172.21.15.43:3300/0 conn(0x7f9ca40cc4f0 0x7f9ca40c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:39.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.101+0000 7f9ca9c77700 1 -- 172.21.15.40:0/2388903599 >> v2:172.21.15.43:3300/0 conn(0x7f9ca40cc4f0 msgr2=0x7f9ca40c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:39.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.101+0000 7f9ca9c77700 1 --2- 172.21.15.40:0/2388903599 >> v2:172.21.15.43:3300/0 conn(0x7f9ca40cc4f0 0x7f9ca40c3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.101+0000 7f9ca9c77700 1 -- 172.21.15.40:0/2388903599 >> v1:172.21.15.40:6789/0 conn(0x7f9ca4153840 legacy=0x7f9ca415d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:39.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.101+0000 7f9ca9c77700 1 -- 172.21.15.40:0/2388903599 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca4153c20 con 0x7f9ca40c9e20 2024-09-13T17:50:39.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.101+0000 7f9c97fff700 1 -- 172.21.15.40:0/2388903599 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 906286315 0 0) 0x7f9ca40cc060 con 0x7f9ca4153840 2024-09-13T17:50:39.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.101+0000 7f9ca9c77700 1 --2- 172.21.15.40:0/2388903599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca40c9e20 0x7f9ca40cf260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40c9550 tx=0x7f9c9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a1b138ec6f7351 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:39.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.105+0000 7f9c97fff700 1 -- 172.21.15.40:0/2388903599 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c00e070 con 0x7f9ca40c9e20 2024-09-13T17:50:39.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.105+0000 7f9c97fff700 1 -- 172.21.15.40:0/2388903599 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9c00b910 con 0x7f9ca40c9e20 2024-09-13T17:50:39.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.105+0000 7f9c97fff700 1 -- 172.21.15.40:0/2388903599 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c00bcc0 con 0x7f9ca40c9e20 2024-09-13T17:50:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.105+0000 7f9caa478700 1 -- 172.21.15.40:0/2388903599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca40c9e20 msgr2=0x7f9ca40cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.105+0000 7f9caa478700 1 --2- 172.21.15.40:0/2388903599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca40c9e20 0x7f9ca40cf260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40c9550 tx=0x7f9c9c00b330 comp rx=0 tx=0).stop 2024-09-13T17:50:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.105+0000 7f9caa478700 1 -- 172.21.15.40:0/2388903599 shutdown_connections 2024-09-13T17:50:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.105+0000 7f9caa478700 1 --2- 172.21.15.40:0/2388903599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca40c9e20 0x7f9ca40cf260 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.105+0000 7f9caa478700 1 --2- 172.21.15.40:0/2388903599 >> v2:172.21.15.43:3300/0 conn(0x7f9ca40cc4f0 0x7f9ca40c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.105+0000 7f9caa478700 1 -- 172.21.15.40:0/2388903599 >> 172.21.15.40:0/2388903599 conn(0x7f9ca406c800 msgr2=0x7f9ca406cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.105+0000 7f9caa478700 1 -- 172.21.15.40:0/2388903599 shutdown_connections 2024-09-13T17:50:39.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.105+0000 7f9caa478700 1 -- 172.21.15.40:0/2388903599 wait complete. 2024-09-13T17:50:39.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9caa478700 1 Processor -- start 2024-09-13T17:50:39.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9caa478700 1 -- start start 2024-09-13T17:50:39.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9caa478700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca40c9e20 0x7f9ca414e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:39.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9caa478700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ca40cc4f0 0x7f9ca41e3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:39.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9caa478700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ca4153840 0x7f9ca41e3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:39.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9caa478700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca4150ac0 con 0x7f9ca40c9e20 2024-09-13T17:50:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9caa478700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca4146400 con 0x7f9ca4153840 2024-09-13T17:50:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9caa478700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca41e4ab0 con 0x7f9ca40cc4f0 2024-09-13T17:50:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9ca9476700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca40c9e20 0x7f9ca414e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9ca9476700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca40c9e20 0x7f9ca414e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38414/0 (socket says 172.21.15.40:38414) 2024-09-13T17:50:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9ca9476700 1 -- 172.21.15.40:0/2799483131 learned_addr learned my addr 172.21.15.40:0/2799483131 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9ca9c77700 1 --2- 172.21.15.40:0/2799483131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ca4153840 0x7f9ca41e3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9ca8c75700 1 --2- 172.21.15.40:0/2799483131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ca40cc4f0 0x7f9ca41e3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9ca9476700 1 -- 172.21.15.40:0/2799483131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ca40cc4f0 msgr2=0x7f9ca41e3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9ca9476700 1 --2- 172.21.15.40:0/2799483131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ca40cc4f0 0x7f9ca41e3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9ca9476700 1 -- 172.21.15.40:0/2799483131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ca4153840 msgr2=0x7f9ca41e3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9ca9476700 1 --2- 172.21.15.40:0/2799483131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ca4153840 0x7f9ca41e3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9ca9476700 1 -- 172.21.15.40:0/2799483131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9c009b30 con 0x7f9ca40c9e20 2024-09-13T17:50:39.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9ca8c75700 1 --2- 172.21.15.40:0/2799483131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ca40cc4f0 0x7f9ca41e3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:50:39.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9ca9476700 1 --2- 172.21.15.40:0/2799483131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca40c9e20 0x7f9ca414e910 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800c4e0 tx=0x7f9c9800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:39.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9c95ffb700 1 -- 172.21.15.40:0/2799483131 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c98004ca0 con 0x7f9ca40c9e20 2024-09-13T17:50:39.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9c95ffb700 1 -- 172.21.15.40:0/2799483131 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c98004e20 con 0x7f9ca40c9e20 2024-09-13T17:50:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9c95ffb700 1 -- 172.21.15.40:0/2799483131 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c98007310 con 0x7f9ca40c9e20 2024-09-13T17:50:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9caa478700 1 -- 172.21.15.40:0/2799483131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca41e4d50 con 0x7f9ca40c9e20 2024-09-13T17:50:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.109+0000 7f9caa478700 1 -- 172.21.15.40:0/2799483131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca4145db0 con 0x7f9ca40c9e20 2024-09-13T17:50:39.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.113+0000 7f9caa478700 1 -- 172.21.15.40:0/2799483131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca4067e20 con 0x7f9ca40c9e20 2024-09-13T17:50:39.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.113+0000 7f9c95ffb700 1 -- 172.21.15.40:0/2799483131 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9c98007470 con 0x7f9ca40c9e20 2024-09-13T17:50:39.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.113+0000 7f9c95ffb700 1 --2- 172.21.15.40:0/2799483131 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9c88066730 0x7f9c88068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:39.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.113+0000 7f9c95ffb700 1 -- 172.21.15.40:0/2799483131 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9c98084150 con 0x7f9ca40c9e20 2024-09-13T17:50:39.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.113+0000 7f9ca8c75700 1 --2- 172.21.15.40:0/2799483131 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9c88066730 0x7f9c88068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:39.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.117+0000 7f9ca8c75700 1 --2- 172.21.15.40:0/2799483131 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9c88066730 0x7f9c88068bf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9ca406d030 tx=0x7f9ca0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:39.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.125+0000 7f9c95ffb700 1 -- 172.21.15.40:0/2799483131 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9c98051140 con 0x7f9ca40c9e20 2024-09-13T17:50:39.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.333+0000 7f9caa478700 1 -- 172.21.15.40:0/2799483131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ca40cbcf0 con 0x7f9ca40c9e20 2024-09-13T17:50:39.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.333+0000 7f9c95ffb700 1 -- 172.21.15.40:0/2799483131 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+6235 (secure 0 0 0) 0x7f9c98007720 con 0x7f9ca40c9e20 2024-09-13T17:50:39.340 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:50:39.340 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6839/1861594206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1861594206},{"type":"v1","addr":"172.21.15.40:6839","nonce":1861594206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1548394691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1548394691},{"type":"v1","addr":"172.21.15.40:6835","nonce":1548394691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/486611242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":486611242},{"type":"v1","addr":"172.21.15.40:6837","nonce":486611242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4279,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/526975022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":526975022},{"type":"v1","addr":"172.21.15.43:6833","nonce":526975022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2586910944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2586910944},{"type":"v1","addr":"172.21.15.43:6835","nonce":2586910944}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:50:34.562623+0000","modified":"2024-09-13T17:50:38.974320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.43:6837/334160033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":334160033},{"type":"v1","addr":"172.21.15.43:6837","nonce":334160033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:50:39.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.333+0000 7f9caa478700 1 -- 172.21.15.40:0/2799483131 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9c88066730 msgr2=0x7f9c88068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:39.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.333+0000 7f9caa478700 1 --2- 172.21.15.40:0/2799483131 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9c88066730 0x7f9c88068bf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9ca406d030 tx=0x7f9ca0008040 comp rx=0 tx=0).stop 2024-09-13T17:50:39.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.333+0000 7f9caa478700 1 -- 172.21.15.40:0/2799483131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca40c9e20 msgr2=0x7f9ca414e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:39.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.333+0000 7f9caa478700 1 --2- 172.21.15.40:0/2799483131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca40c9e20 0x7f9ca414e910 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800c4e0 tx=0x7f9c9800c7f0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.333+0000 7f9caa478700 1 -- 172.21.15.40:0/2799483131 shutdown_connections 2024-09-13T17:50:39.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.333+0000 7f9caa478700 1 --2- 172.21.15.40:0/2799483131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ca4153840 0x7f9ca41e3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.337+0000 7f9caa478700 1 --2- 172.21.15.40:0/2799483131 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9c88066730 0x7f9c88068bf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.337+0000 7f9caa478700 1 --2- 172.21.15.40:0/2799483131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ca40cc4f0 0x7f9ca41e3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.337+0000 7f9caa478700 1 --2- 172.21.15.40:0/2799483131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ca40c9e20 0x7f9ca414e910 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.337+0000 7f9caa478700 1 -- 172.21.15.40:0/2799483131 >> 172.21.15.40:0/2799483131 conn(0x7f9ca406c800 msgr2=0x7f9ca4146be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:39.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.337+0000 7f9caa478700 1 -- 172.21.15.40:0/2799483131 shutdown_connections 2024-09-13T17:50:39.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.337+0000 7f9caa478700 1 -- 172.21.15.40:0/2799483131 wait complete. 2024-09-13T17:50:39.344 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 5 2024-09-13T17:50:39.355 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:50:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.457+0000 7f9245513700 1 Processor -- start 2024-09-13T17:50:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.457+0000 7f9245513700 1 -- start start 2024-09-13T17:50:39.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.457+0000 7f9245513700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f92400cb2f0 0x7f92400cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:39.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.457+0000 7f9245513700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f92400cf3c0 0x7f924015cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:39.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.457+0000 7f9245513700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9240159770 con 0x7f9240153880 2024-09-13T17:50:39.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.457+0000 7f9245513700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9240159a50 con 0x7f92400cb2f0 2024-09-13T17:50:39.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.457+0000 7f9245513700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9240159d30 con 0x7f92400cf3c0 2024-09-13T17:50:39.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.457+0000 7f923f7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f92400cb2f0 0x7f92400cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:39.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.457+0000 7f923f7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f92400cb2f0 0x7f92400cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38430/0 (socket says 172.21.15.40:38430) 2024-09-13T17:50:39.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.457+0000 7f923f7fe700 1 -- 172.21.15.40:0/3768041287 learned_addr learned my addr 172.21.15.40:0/3768041287 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:39.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.457+0000 7f9244d12700 1 --2- 172.21.15.40:0/3768041287 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f92400cf3c0 0x7f924015cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:39.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.457+0000 7f923f7fe700 1 -- 172.21.15.40:0/3768041287 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f92400cf3c0 msgr2=0x7f924015cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:39.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.457+0000 7f923f7fe700 1 --2- 172.21.15.40:0/3768041287 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f92400cf3c0 0x7f924015cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.457+0000 7f923f7fe700 1 -- 172.21.15.40:0/3768041287 >> v1:172.21.15.40:6789/0 conn(0x7f9240153880 legacy=0x7f92400c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:39.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.457+0000 7f923f7fe700 1 -- 172.21.15.40:0/3768041287 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924015a010 con 0x7f92400cb2f0 2024-09-13T17:50:39.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.457+0000 7f923f7fe700 1 --2- 172.21.15.40:0/3768041287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92400cb2f0 0x7f92400cee80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9234009910 tx=0x7f9234009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ed225c83240e17 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:39.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.457+0000 7f923effd700 1 -- 172.21.15.40:0/3768041287 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234011830 con 0x7f92400cb2f0 2024-09-13T17:50:39.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.457+0000 7f923effd700 1 -- 172.21.15.40:0/3768041287 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9234011990 con 0x7f92400cb2f0 2024-09-13T17:50:39.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.457+0000 7f923effd700 1 -- 172.21.15.40:0/3768041287 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234011d40 con 0x7f92400cb2f0 2024-09-13T17:50:39.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.461+0000 7f9245513700 1 -- 172.21.15.40:0/3768041287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92400cb2f0 msgr2=0x7f92400cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:39.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.461+0000 7f9245513700 1 --2- 172.21.15.40:0/3768041287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92400cb2f0 0x7f92400cee80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9234009910 tx=0x7f9234009c20 comp rx=0 tx=0).stop 2024-09-13T17:50:39.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.461+0000 7f9245513700 1 -- 172.21.15.40:0/3768041287 shutdown_connections 2024-09-13T17:50:39.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.461+0000 7f9245513700 1 --2- 172.21.15.40:0/3768041287 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f92400cf3c0 0x7f924015cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.461+0000 7f9245513700 1 --2- 172.21.15.40:0/3768041287 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92400cb2f0 0x7f92400cee80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.461+0000 7f9245513700 1 -- 172.21.15.40:0/3768041287 >> 172.21.15.40:0/3768041287 conn(0x7f924006c5b0 msgr2=0x7f924006c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:39.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.461+0000 7f9245513700 1 -- 172.21.15.40:0/3768041287 shutdown_connections 2024-09-13T17:50:39.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.461+0000 7f9245513700 1 -- 172.21.15.40:0/3768041287 wait complete. 2024-09-13T17:50:39.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f9245513700 1 Processor -- start 2024-09-13T17:50:39.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f9245513700 1 -- start start 2024-09-13T17:50:39.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f9245513700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92400cb2f0 0x7f92400c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:39.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f9245513700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f92400cf3c0 0x7f924014b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:39.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f9245513700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9240153880 0x7f92401498e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f9245513700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92400c6cd0 con 0x7f92400cb2f0 2024-09-13T17:50:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f9245513700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92400c74d0 con 0x7f9240153880 2024-09-13T17:50:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f9245513700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924014a9e0 con 0x7f92400cf3c0 2024-09-13T17:50:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f923ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92400cb2f0 0x7f92400c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f923ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92400cb2f0 0x7f92400c6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38440/0 (socket says 172.21.15.40:38440) 2024-09-13T17:50:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f923ffff700 1 -- 172.21.15.40:0/802606625 learned_addr learned my addr 172.21.15.40:0/802606625 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f923f7fe700 1 --2- 172.21.15.40:0/802606625 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f92400cf3c0 0x7f924014b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f9244d12700 1 --2- 172.21.15.40:0/802606625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9240153880 0x7f92401498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f923ffff700 1 -- 172.21.15.40:0/802606625 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f92400cf3c0 msgr2=0x7f924014b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f923ffff700 1 --2- 172.21.15.40:0/802606625 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f92400cf3c0 0x7f924014b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f923ffff700 1 -- 172.21.15.40:0/802606625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9240153880 msgr2=0x7f92401498e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f923ffff700 1 --2- 172.21.15.40:0/802606625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9240153880 0x7f92401498e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f923ffff700 1 -- 172.21.15.40:0/802606625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9234009570 con 0x7f92400cb2f0 2024-09-13T17:50:39.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f923ffff700 1 --2- 172.21.15.40:0/802606625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92400cb2f0 0x7f92400c6f90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f923000c500 tx=0x7f923000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:39.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f923cff9700 1 -- 172.21.15.40:0/802606625 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92300052d0 con 0x7f92400cb2f0 2024-09-13T17:50:39.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f923cff9700 1 -- 172.21.15.40:0/802606625 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9230005430 con 0x7f92400cb2f0 2024-09-13T17:50:39.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f923cff9700 1 -- 172.21.15.40:0/802606625 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92300055a0 con 0x7f92400cb2f0 2024-09-13T17:50:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.465+0000 7f9245513700 1 -- 172.21.15.40:0/802606625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924014acc0 con 0x7f92400cb2f0 2024-09-13T17:50:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.469+0000 7f9245513700 1 -- 172.21.15.40:0/802606625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f924014c120 con 0x7f92400cb2f0 2024-09-13T17:50:39.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.469+0000 7f9245513700 1 -- 172.21.15.40:0/802606625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92401513d0 con 0x7f92400cb2f0 2024-09-13T17:50:39.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.469+0000 7f923cff9700 1 -- 172.21.15.40:0/802606625 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f92300038e0 con 0x7f92400cb2f0 2024-09-13T17:50:39.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.469+0000 7f923cff9700 1 --2- 172.21.15.40:0/802606625 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f922c066730 0x7f922c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:39.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.469+0000 7f923cff9700 1 -- 172.21.15.40:0/802606625 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9230083810 con 0x7f92400cb2f0 2024-09-13T17:50:39.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.469+0000 7f923f7fe700 1 --2- 172.21.15.40:0/802606625 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f922c066730 0x7f922c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:39.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.473+0000 7f923f7fe700 1 --2- 172.21.15.40:0/802606625 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f922c066730 0x7f922c068bf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9234000c00 tx=0x7f9234008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:39.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.477+0000 7f923cff9700 1 -- 172.21.15.40:0/802606625 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9230050810 con 0x7f92400cb2f0 2024-09-13T17:50:39.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.665+0000 7f9245513700 1 -- 172.21.15.40:0/802606625 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f924015d860 con 0x7f92400cb2f0 2024-09-13T17:50:39.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.665+0000 7f923cff9700 1 -- 172.21.15.40:0/802606625 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18262 (secure 0 0 0) 0x7f9230053e30 con 0x7f92400cb2f0 2024-09-13T17:50:39.674 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:50:39.674 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:50:38.974071+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:50:32.740336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:50:33.552914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:50:36.623639+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:50:39.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.669+0000 7f9245513700 1 -- 172.21.15.40:0/802606625 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f922c066730 msgr2=0x7f922c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:39.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.669+0000 7f9245513700 1 --2- 172.21.15.40:0/802606625 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f922c066730 0x7f922c068bf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9234000c00 tx=0x7f9234008030 comp rx=0 tx=0).stop 2024-09-13T17:50:39.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.669+0000 7f9245513700 1 -- 172.21.15.40:0/802606625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92400cb2f0 msgr2=0x7f92400c6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:39.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.669+0000 7f9245513700 1 --2- 172.21.15.40:0/802606625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92400cb2f0 0x7f92400c6f90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f923000c500 tx=0x7f923000c810 comp rx=0 tx=0).stop 2024-09-13T17:50:39.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.669+0000 7f9245513700 1 -- 172.21.15.40:0/802606625 shutdown_connections 2024-09-13T17:50:39.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.669+0000 7f9245513700 1 --2- 172.21.15.40:0/802606625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9240153880 0x7f92401498e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.669+0000 7f9245513700 1 --2- 172.21.15.40:0/802606625 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f922c066730 0x7f922c068bf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.669+0000 7f9245513700 1 --2- 172.21.15.40:0/802606625 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f92400cf3c0 0x7f924014b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.669+0000 7f9245513700 1 --2- 172.21.15.40:0/802606625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92400cb2f0 0x7f92400c6f90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.669+0000 7f9245513700 1 -- 172.21.15.40:0/802606625 >> 172.21.15.40:0/802606625 conn(0x7f924006c5b0 msgr2=0x7f9240152650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:39.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.669+0000 7f9245513700 1 -- 172.21.15.40:0/802606625 shutdown_connections 2024-09-13T17:50:39.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.669+0000 7f9245513700 1 -- 172.21.15.40:0/802606625 wait complete. 2024-09-13T17:50:39.689 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:50:39.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1523052700 1 Processor -- start 2024-09-13T17:50:39.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1523052700 1 -- start start 2024-09-13T17:50:39.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1523052700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f151c157c60 0x7f151c0c0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:39.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1523052700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f151c15e700 0x7f151c0c0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:39.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1523052700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f151c15db60 con 0x7f151c0c57d0 2024-09-13T17:50:39.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1523052700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f151c160fb0 con 0x7f151c157c60 2024-09-13T17:50:39.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1523052700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f151c161250 con 0x7f151c15e700 2024-09-13T17:50:39.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1520dee700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f151c157c60 0x7f151c0c0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:39.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1520dee700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f151c157c60 0x7f151c0c0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38450/0 (socket says 172.21.15.40:38450) 2024-09-13T17:50:39.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1520dee700 1 -- 172.21.15.40:0/1088099795 learned_addr learned my addr 172.21.15.40:0/1088099795 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:39.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f15215ef700 1 --2- 172.21.15.40:0/1088099795 >> v2:172.21.15.43:3300/0 conn(0x7f151c15e700 0x7f151c0c0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1520dee700 1 -- 172.21.15.40:0/1088099795 >> v2:172.21.15.43:3300/0 conn(0x7f151c15e700 msgr2=0x7f151c0c0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1520dee700 1 --2- 172.21.15.40:0/1088099795 >> v2:172.21.15.43:3300/0 conn(0x7f151c15e700 0x7f151c0c0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1520dee700 1 -- 172.21.15.40:0/1088099795 >> v1:172.21.15.43:6789/0 conn(0x7f151c0c57d0 legacy=0x7f151c15dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1520dee700 1 -- 172.21.15.40:0/1088099795 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151c161510 con 0x7f151c157c60 2024-09-13T17:50:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f15215ef700 1 --2- 172.21.15.40:0/1088099795 >> v2:172.21.15.43:3300/0 conn(0x7f151c15e700 0x7f151c0c0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:50:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1520dee700 1 --2- 172.21.15.40:0/1088099795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f151c157c60 0x7f151c0c0840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f151c06b6a0 tx=0x7f1504009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a338bfcab88ec4a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f151b7fe700 1 -- 172.21.15.40:0/1088099795 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150400b8a0 con 0x7f151c157c60 2024-09-13T17:50:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f151b7fe700 1 -- 172.21.15.40:0/1088099795 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150400ba00 con 0x7f151c157c60 2024-09-13T17:50:39.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f151b7fe700 1 -- 172.21.15.40:0/1088099795 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150400bd90 con 0x7f151c157c60 2024-09-13T17:50:39.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1523052700 1 -- 172.21.15.40:0/1088099795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f151c157c60 msgr2=0x7f151c0c0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:39.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1523052700 1 --2- 172.21.15.40:0/1088099795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f151c157c60 0x7f151c0c0840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f151c06b6a0 tx=0x7f1504009bf0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1523052700 1 -- 172.21.15.40:0/1088099795 shutdown_connections 2024-09-13T17:50:39.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1523052700 1 --2- 172.21.15.40:0/1088099795 >> v2:172.21.15.43:3300/0 conn(0x7f151c15e700 0x7f151c0c0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1523052700 1 --2- 172.21.15.40:0/1088099795 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f151c157c60 0x7f151c0c0840 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1523052700 1 -- 172.21.15.40:0/1088099795 >> 172.21.15.40:0/1088099795 conn(0x7f151c0cadd0 msgr2=0x7f151c0cb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:39.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1523052700 1 -- 172.21.15.40:0/1088099795 shutdown_connections 2024-09-13T17:50:39.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.801+0000 7f1523052700 1 -- 172.21.15.40:0/1088099795 wait complete. 2024-09-13T17:50:39.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.805+0000 7f1523052700 1 Processor -- start 2024-09-13T17:50:39.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.805+0000 7f1523052700 1 -- start start 2024-09-13T17:50:39.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.805+0000 7f1523052700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f151c0c57d0 0x7f151c1f3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:39.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.805+0000 7f1523052700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f151c157c60 0x7f151c0ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:39.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.805+0000 7f1523052700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c15e700 0x7f151c0ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:39.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.805+0000 7f1523052700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c15f7f0 con 0x7f151c157c60 2024-09-13T17:50:39.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.805+0000 7f1523052700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c15ff90 con 0x7f151c15e700 2024-09-13T17:50:39.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.805+0000 7f1523052700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151c1f7080 con 0x7f151c0c57d0 2024-09-13T17:50:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.805+0000 7f151bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f151c157c60 0x7f151c0ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.805+0000 7f151bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f151c157c60 0x7f151c0ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38454/0 (socket says 172.21.15.40:38454) 2024-09-13T17:50:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.805+0000 7f151bfff700 1 -- 172.21.15.40:0/756249974 learned_addr learned my addr 172.21.15.40:0/756249974 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.805+0000 7f1520dee700 1 --2- 172.21.15.40:0/756249974 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f151c0c57d0 0x7f151c1f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.805+0000 7f15215ef700 1 --2- 172.21.15.40:0/756249974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c15e700 0x7f151c0ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.805+0000 7f151bfff700 1 -- 172.21.15.40:0/756249974 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f151c0c57d0 msgr2=0x7f151c1f3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.805+0000 7f151bfff700 1 --2- 172.21.15.40:0/756249974 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f151c0c57d0 0x7f151c1f3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.805+0000 7f151bfff700 1 -- 172.21.15.40:0/756249974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c15e700 msgr2=0x7f151c0ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.805+0000 7f151bfff700 1 --2- 172.21.15.40:0/756249974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c15e700 0x7f151c0ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.805+0000 7f151bfff700 1 -- 172.21.15.40:0/756249974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1504009570 con 0x7f151c157c60 2024-09-13T17:50:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.805+0000 7f15215ef700 1 --2- 172.21.15.40:0/756249974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c15e700 0x7f151c0ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:50:39.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.809+0000 7f151bfff700 1 --2- 172.21.15.40:0/756249974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f151c157c60 0x7f151c0ca2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f151400cc50 tx=0x7f151400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:39.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.809+0000 7f15197fa700 1 -- 172.21.15.40:0/756249974 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514003c20 con 0x7f151c157c60 2024-09-13T17:50:39.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.809+0000 7f15197fa700 1 -- 172.21.15.40:0/756249974 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1514003d80 con 0x7f151c157c60 2024-09-13T17:50:39.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.809+0000 7f1523052700 1 -- 172.21.15.40:0/756249974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151c160d50 con 0x7f151c157c60 2024-09-13T17:50:39.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.809+0000 7f15197fa700 1 -- 172.21.15.40:0/756249974 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151400ab20 con 0x7f151c157c60 2024-09-13T17:50:39.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.809+0000 7f1523052700 1 -- 172.21.15.40:0/756249974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151c15fcb0 con 0x7f151c157c60 2024-09-13T17:50:39.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.809+0000 7f1523052700 1 -- 172.21.15.40:0/756249974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f151c06e330 con 0x7f151c157c60 2024-09-13T17:50:39.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.809+0000 7f15197fa700 1 -- 172.21.15.40:0/756249974 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f151400ac80 con 0x7f151c157c60 2024-09-13T17:50:39.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.809+0000 7f15197fa700 1 --2- 172.21.15.40:0/756249974 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f14fc066730 0x7f14fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:39.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.809+0000 7f1520dee700 1 --2- 172.21.15.40:0/756249974 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f14fc066730 0x7f14fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:39.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.813+0000 7f15197fa700 1 -- 172.21.15.40:0/756249974 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1514084590 con 0x7f151c157c60 2024-09-13T17:50:39.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.813+0000 7f1520dee700 1 --2- 172.21.15.40:0/756249974 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f14fc066730 0x7f14fc068bf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1504000c00 tx=0x7f1504008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:39.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.821+0000 7f15197fa700 1 -- 172.21.15.40:0/756249974 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f15140514e0 con 0x7f151c157c60 2024-09-13T17:50:40.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:39.997+0000 7f1523052700 1 -- 172.21.15.40:0/756249974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f151c1f71c0 con 0x7f151c157c60 2024-09-13T17:50:40.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.001+0000 7f15197fa700 1 -- 172.21.15.40:0/756249974 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18262 (secure 0 0 0) 0x7f1514054b00 con 0x7f151c157c60 2024-09-13T17:50:40.009 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:50:40.010 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:50:38.974071+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:50:32.740336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:50:33.552914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:50:36.623639+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:50:40.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.005+0000 7f1523052700 1 -- 172.21.15.40:0/756249974 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f14fc066730 msgr2=0x7f14fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:40.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.005+0000 7f1523052700 1 --2- 172.21.15.40:0/756249974 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f14fc066730 0x7f14fc068bf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1504000c00 tx=0x7f1504008030 comp rx=0 tx=0).stop 2024-09-13T17:50:40.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.005+0000 7f1523052700 1 -- 172.21.15.40:0/756249974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f151c157c60 msgr2=0x7f151c0ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:40.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.005+0000 7f1523052700 1 --2- 172.21.15.40:0/756249974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f151c157c60 0x7f151c0ca2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f151400cc50 tx=0x7f151400cf60 comp rx=0 tx=0).stop 2024-09-13T17:50:40.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.005+0000 7f1523052700 1 -- 172.21.15.40:0/756249974 shutdown_connections 2024-09-13T17:50:40.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.005+0000 7f1523052700 1 --2- 172.21.15.40:0/756249974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c15e700 0x7f151c0ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.005+0000 7f1523052700 1 --2- 172.21.15.40:0/756249974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f151c157c60 0x7f151c0ca2f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.005+0000 7f1523052700 1 --2- 172.21.15.40:0/756249974 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f14fc066730 0x7f14fc068bf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.005+0000 7f1523052700 1 --2- 172.21.15.40:0/756249974 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f151c0c57d0 0x7f151c1f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.005+0000 7f1523052700 1 -- 172.21.15.40:0/756249974 >> 172.21.15.40:0/756249974 conn(0x7f151c0cadd0 msgr2=0x7f151c0c7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:40.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.005+0000 7f1523052700 1 -- 172.21.15.40:0/756249974 shutdown_connections 2024-09-13T17:50:40.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.005+0000 7f1523052700 1 -- 172.21.15.40:0/756249974 wait complete. 2024-09-13T17:50:40.044 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:50:40.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.149+0000 7fae70fbb700 1 Processor -- start 2024-09-13T17:50:40.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.149+0000 7fae70fbb700 1 -- start start 2024-09-13T17:50:40.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.149+0000 7fae70fbb700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae6c153880 0x7fae6c0c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:40.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.149+0000 7fae70fbb700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fae6c0cb2f0 0x7fae6c0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:40.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.149+0000 7fae70fbb700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fae6c0cf3c0 0x7fae6c15cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:40.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.149+0000 7fae70fbb700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae6c159770 con 0x7fae6c0cb2f0 2024-09-13T17:50:40.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.149+0000 7fae70fbb700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae6c159a50 con 0x7fae6c0cf3c0 2024-09-13T17:50:40.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.149+0000 7fae70fbb700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c159d30 con 0x7fae6c153880 2024-09-13T17:50:40.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae63fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fae6c0cb2f0 0x7fae6c0cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:40.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae63fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fae6c0cb2f0 0x7fae6c0cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38470/0 (socket says 172.21.15.40:38470) 2024-09-13T17:50:40.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae63fff700 1 -- 172.21.15.40:0/582684128 learned_addr learned my addr 172.21.15.40:0/582684128 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:40.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae6bfff700 1 --2- 172.21.15.40:0/582684128 >> v2:172.21.15.43:3300/0 conn(0x7fae6c0cf3c0 0x7fae6c15cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:40.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae6b7fe700 1 --2- 172.21.15.40:0/582684128 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae6c153880 0x7fae6c0c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:40.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae63fff700 1 -- 172.21.15.40:0/582684128 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae6c153880 msgr2=0x7fae6c0c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:40.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae63fff700 1 --2- 172.21.15.40:0/582684128 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae6c153880 0x7fae6c0c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae63fff700 1 -- 172.21.15.40:0/582684128 >> v2:172.21.15.43:3300/0 conn(0x7fae6c0cf3c0 msgr2=0x7fae6c15cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:40.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae63fff700 1 --2- 172.21.15.40:0/582684128 >> v2:172.21.15.43:3300/0 conn(0x7fae6c0cf3c0 0x7fae6c15cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae63fff700 1 -- 172.21.15.40:0/582684128 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6c15a010 con 0x7fae6c0cb2f0 2024-09-13T17:50:40.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae63fff700 1 --2- 172.21.15.40:0/582684128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0cb2f0 0x7fae6c0cee80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0cffc0 tx=0x7fae54009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cdeee5aeca2c49 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:40.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae6affd700 1 -- 172.21.15.40:0/582684128 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5400b8a0 con 0x7fae6c0cb2f0 2024-09-13T17:50:40.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae6affd700 1 -- 172.21.15.40:0/582684128 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5400ba00 con 0x7fae6c0cb2f0 2024-09-13T17:50:40.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae6affd700 1 -- 172.21.15.40:0/582684128 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5400bd90 con 0x7fae6c0cb2f0 2024-09-13T17:50:40.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae70fbb700 1 -- 172.21.15.40:0/582684128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0cb2f0 msgr2=0x7fae6c0cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:40.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae70fbb700 1 --2- 172.21.15.40:0/582684128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0cb2f0 0x7fae6c0cee80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0cffc0 tx=0x7fae54009bf0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae70fbb700 1 -- 172.21.15.40:0/582684128 shutdown_connections 2024-09-13T17:50:40.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae70fbb700 1 --2- 172.21.15.40:0/582684128 >> v2:172.21.15.43:3300/0 conn(0x7fae6c0cf3c0 0x7fae6c15cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae70fbb700 1 --2- 172.21.15.40:0/582684128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c0cb2f0 0x7fae6c0cee80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae70fbb700 1 --2- 172.21.15.40:0/582684128 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae6c153880 0x7fae6c0c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae70fbb700 1 -- 172.21.15.40:0/582684128 >> 172.21.15.40:0/582684128 conn(0x7fae6c06c5b0 msgr2=0x7fae6c06c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:40.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae70fbb700 1 -- 172.21.15.40:0/582684128 shutdown_connections 2024-09-13T17:50:40.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae70fbb700 1 -- 172.21.15.40:0/582684128 wait complete. 2024-09-13T17:50:40.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.153+0000 7fae70fbb700 1 Processor -- start 2024-09-13T17:50:40.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae70fbb700 1 -- start start 2024-09-13T17:50:40.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae70fbb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae6c0cb2f0 0x7fae6c14b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:40.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae70fbb700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae6c0cf3c0 0x7fae6c151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:40.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae70fbb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c153880 0x7fae6c0c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:40.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae70fbb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c0c85d0 con 0x7fae6c153880 2024-09-13T17:50:40.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae70fbb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c0c6f10 con 0x7fae6c0cb2f0 2024-09-13T17:50:40.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae70fbb700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae6c0c90e0 con 0x7fae6c0cf3c0 2024-09-13T17:50:40.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae6bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c153880 0x7fae6c0c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:40.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae6bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c153880 0x7fae6c0c73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38486/0 (socket says 172.21.15.40:38486) 2024-09-13T17:50:40.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae6bfff700 1 -- 172.21.15.40:0/1496575929 learned_addr learned my addr 172.21.15.40:0/1496575929 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:40.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae63fff700 1 --2- 172.21.15.40:0/1496575929 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae6c0cf3c0 0x7fae6c151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:40.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae6b7fe700 1 --2- 172.21.15.40:0/1496575929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae6c0cb2f0 0x7fae6c14b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:40.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae6bfff700 1 -- 172.21.15.40:0/1496575929 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae6c0cf3c0 msgr2=0x7fae6c151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:40.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae6bfff700 1 --2- 172.21.15.40:0/1496575929 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae6c0cf3c0 0x7fae6c151310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae6bfff700 1 -- 172.21.15.40:0/1496575929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae6c0cb2f0 msgr2=0x7fae6c14b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:40.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae6bfff700 1 --2- 172.21.15.40:0/1496575929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae6c0cb2f0 0x7fae6c14b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae6bfff700 1 -- 172.21.15.40:0/1496575929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae54009570 con 0x7fae6c153880 2024-09-13T17:50:40.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae6bfff700 1 --2- 172.21.15.40:0/1496575929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c153880 0x7fae6c0c73a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fae64007e60 tx=0x7fae6400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:40.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae68ff9700 1 -- 172.21.15.40:0/1496575929 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6400ede0 con 0x7fae6c153880 2024-09-13T17:50:40.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae68ff9700 1 -- 172.21.15.40:0/1496575929 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae64014420 con 0x7fae6c153880 2024-09-13T17:50:40.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae70fbb700 1 -- 172.21.15.40:0/1496575929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae6c0c87b0 con 0x7fae6c153880 2024-09-13T17:50:40.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae68ff9700 1 -- 172.21.15.40:0/1496575929 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae640147d0 con 0x7fae6c153880 2024-09-13T17:50:40.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae70fbb700 1 -- 172.21.15.40:0/1496575929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae6c0c8d70 con 0x7fae6c153880 2024-09-13T17:50:40.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.157+0000 7fae70fbb700 1 -- 172.21.15.40:0/1496575929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae6c0cee80 con 0x7fae6c153880 2024-09-13T17:50:40.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.161+0000 7fae68ff9700 1 -- 172.21.15.40:0/1496575929 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fae64014930 con 0x7fae6c153880 2024-09-13T17:50:40.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.161+0000 7fae68ff9700 1 --2- 172.21.15.40:0/1496575929 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fae4c066730 0x7fae4c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:40.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.161+0000 7fae6b7fe700 1 --2- 172.21.15.40:0/1496575929 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fae4c066730 0x7fae4c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:40.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.161+0000 7fae68ff9700 1 -- 172.21.15.40:0/1496575929 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fae64084c10 con 0x7fae6c153880 2024-09-13T17:50:40.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.161+0000 7fae6b7fe700 1 --2- 172.21.15.40:0/1496575929 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fae4c066730 0x7fae4c068bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00b9d0 tx=0x7fae5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:40.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.169+0000 7fae68ff9700 1 -- 172.21.15.40:0/1496575929 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fae64051c80 con 0x7fae6c153880 2024-09-13T17:50:40.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.377+0000 7fae70fbb700 1 -- 172.21.15.40:0/1496575929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae6c0cf910 con 0x7fae6c153880 2024-09-13T17:50:40.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.377+0000 7fae68ff9700 1 -- 172.21.15.40:0/1496575929 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+6233 (secure 0 0 0) 0x7fae640552a0 con 0x7fae6c153880 2024-09-13T17:50:40.386 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:50:40.386 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6839/1861594206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1861594206},{"type":"v1","addr":"172.21.15.40:6839","nonce":1861594206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1548394691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1548394691},{"type":"v1","addr":"172.21.15.40:6835","nonce":1548394691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/486611242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":486611242},{"type":"v1","addr":"172.21.15.40:6837","nonce":486611242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4279,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/526975022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":526975022},{"type":"v1","addr":"172.21.15.43:6833","nonce":526975022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2586910944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2586910944},{"type":"v1","addr":"172.21.15.43:6835","nonce":2586910944}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:50:34.562623+0000","modified":"2024-09-13T17:50:39.976517+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/334160033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":334160033},{"type":"v1","addr":"172.21.15.43:6837","nonce":334160033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-09-13T17:50:40.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.381+0000 7fae70fbb700 1 -- 172.21.15.40:0/1496575929 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fae4c066730 msgr2=0x7fae4c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:40.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.381+0000 7fae70fbb700 1 --2- 172.21.15.40:0/1496575929 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fae4c066730 0x7fae4c068bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00b9d0 tx=0x7fae5c008040 comp rx=0 tx=0).stop 2024-09-13T17:50:40.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.381+0000 7fae70fbb700 1 -- 172.21.15.40:0/1496575929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c153880 msgr2=0x7fae6c0c73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:40.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.381+0000 7fae70fbb700 1 --2- 172.21.15.40:0/1496575929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c153880 0x7fae6c0c73a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fae64007e60 tx=0x7fae6400e3c0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.381+0000 7fae70fbb700 1 -- 172.21.15.40:0/1496575929 shutdown_connections 2024-09-13T17:50:40.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.381+0000 7fae70fbb700 1 --2- 172.21.15.40:0/1496575929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae6c153880 0x7fae6c0c73a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.381+0000 7fae70fbb700 1 --2- 172.21.15.40:0/1496575929 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fae4c066730 0x7fae4c068bf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.381+0000 7fae70fbb700 1 --2- 172.21.15.40:0/1496575929 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae6c0cf3c0 0x7fae6c151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.381+0000 7fae70fbb700 1 --2- 172.21.15.40:0/1496575929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae6c0cb2f0 0x7fae6c14b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.381+0000 7fae70fbb700 1 -- 172.21.15.40:0/1496575929 >> 172.21.15.40:0/1496575929 conn(0x7fae6c06c5b0 msgr2=0x7fae6c06c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:40.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.381+0000 7fae70fbb700 1 -- 172.21.15.40:0/1496575929 shutdown_connections 2024-09-13T17:50:40.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.381+0000 7fae70fbb700 1 -- 172.21.15.40:0/1496575929 wait complete. 2024-09-13T17:50:40.390 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 7 2024-09-13T17:50:40.401 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:50:40.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.513+0000 7f9eadf5d700 1 Processor -- start 2024-09-13T17:50:40.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.513+0000 7f9eadf5d700 1 -- start start 2024-09-13T17:50:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.513+0000 7f9eadf5d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9ea80cc4f0 0x7f9ea80c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.513+0000 7f9eadf5d700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea80cc060 con 0x7f9ea80c9e20 2024-09-13T17:50:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.513+0000 7f9eadf5d700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea8153c20 con 0x7f9ea8153840 2024-09-13T17:50:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.513+0000 7f9eadf5d700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea80b2840 con 0x7f9ea80cc4f0 2024-09-13T17:50:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.513+0000 7f9ea7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9ea80cc4f0 0x7f9ea80c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.513+0000 7f9ea7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9ea80cc4f0 0x7f9ea80c3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38500/0 (socket says 172.21.15.40:38500) 2024-09-13T17:50:40.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.513+0000 7f9ea7fff700 1 -- 172.21.15.40:0/3521239281 learned_addr learned my addr 172.21.15.40:0/3521239281 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:40.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.513+0000 7f9ead75c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9ea80c9e20 0x7f9ea80cf260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35840/0 (socket says 172.21.15.40:35840) 2024-09-13T17:50:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.513+0000 7f9ea7fff700 1 -- 172.21.15.40:0/3521239281 >> v1:172.21.15.43:6789/0 conn(0x7f9ea8153840 legacy=0x7f9ea815d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.513+0000 7f9ea7fff700 1 -- 172.21.15.40:0/3521239281 >> v1:172.21.15.40:6789/0 conn(0x7f9ea80c9e20 legacy=0x7f9ea80cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.513+0000 7f9ea7fff700 1 -- 172.21.15.40:0/3521239281 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea80cb1e0 con 0x7f9ea80cc4f0 2024-09-13T17:50:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.513+0000 7f9ea7fff700 1 --2- 172.21.15.40:0/3521239281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea80cc4f0 0x7f9ea80c3eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c009e50 tx=0x7f9e9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b568cca511065df7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.513+0000 7f9ea77fe700 1 -- 172.21.15.40:0/3521239281 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00e070 con 0x7f9ea80cc4f0 2024-09-13T17:50:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.513+0000 7f9ea77fe700 1 -- 172.21.15.40:0/3521239281 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c00b9c0 con 0x7f9ea80cc4f0 2024-09-13T17:50:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.513+0000 7f9ea77fe700 1 -- 172.21.15.40:0/3521239281 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00bd70 con 0x7f9ea80cc4f0 2024-09-13T17:50:40.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eadf5d700 1 -- 172.21.15.40:0/3521239281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea80cc4f0 msgr2=0x7f9ea80c3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eadf5d700 1 --2- 172.21.15.40:0/3521239281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea80cc4f0 0x7f9ea80c3eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c009e50 tx=0x7f9e9c00b330 comp rx=0 tx=0).stop 2024-09-13T17:50:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eadf5d700 1 -- 172.21.15.40:0/3521239281 shutdown_connections 2024-09-13T17:50:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eadf5d700 1 --2- 172.21.15.40:0/3521239281 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea80cc4f0 0x7f9ea80c3eb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eadf5d700 1 -- 172.21.15.40:0/3521239281 >> 172.21.15.40:0/3521239281 conn(0x7f9ea806c800 msgr2=0x7f9ea806cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eadf5d700 1 -- 172.21.15.40:0/3521239281 shutdown_connections 2024-09-13T17:50:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eadf5d700 1 -- 172.21.15.40:0/3521239281 wait complete. 2024-09-13T17:50:40.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eadf5d700 1 Processor -- start 2024-09-13T17:50:40.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eadf5d700 1 -- start start 2024-09-13T17:50:40.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eadf5d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea80c9e20 0x7f9ea81dfa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:40.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eadf5d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ea80cc4f0 0x7f9ea81e4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:40.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eadf5d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ea8153840 0x7f9ea81f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:40.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eadf5d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea81555b0 con 0x7f9ea80c9e20 2024-09-13T17:50:40.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eadf5d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea8150800 con 0x7f9ea80cc4f0 2024-09-13T17:50:40.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eadf5d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea8153de0 con 0x7f9ea8153840 2024-09-13T17:50:40.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eacf5b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea80c9e20 0x7f9ea81dfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:40.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eacf5b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea80c9e20 0x7f9ea81dfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38504/0 (socket says 172.21.15.40:38504) 2024-09-13T17:50:40.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eacf5b700 1 -- 172.21.15.40:0/764776889 learned_addr learned my addr 172.21.15.40:0/764776889 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:40.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9ead75c700 1 --2- 172.21.15.40:0/764776889 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ea8153840 0x7f9ea81f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:40.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9ea7fff700 1 --2- 172.21.15.40:0/764776889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ea80cc4f0 0x7f9ea81e4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:40.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eacf5b700 1 -- 172.21.15.40:0/764776889 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ea8153840 msgr2=0x7f9ea81f28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:40.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eacf5b700 1 --2- 172.21.15.40:0/764776889 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ea8153840 0x7f9ea81f28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eacf5b700 1 -- 172.21.15.40:0/764776889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ea80cc4f0 msgr2=0x7f9ea81e4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:40.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eacf5b700 1 --2- 172.21.15.40:0/764776889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ea80cc4f0 0x7f9ea81e4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eacf5b700 1 -- 172.21.15.40:0/764776889 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c009b30 con 0x7f9ea80c9e20 2024-09-13T17:50:40.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eacf5b700 1 --2- 172.21.15.40:0/764776889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea80c9e20 0x7f9ea81dfa80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800c500 tx=0x7f9e9800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:40.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9ea57fa700 1 -- 172.21.15.40:0/764776889 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e980052d0 con 0x7f9ea80c9e20 2024-09-13T17:50:40.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9ea57fa700 1 -- 172.21.15.40:0/764776889 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e98005430 con 0x7f9ea80c9e20 2024-09-13T17:50:40.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9ea57fa700 1 -- 172.21.15.40:0/764776889 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e980055a0 con 0x7f9ea80c9e20 2024-09-13T17:50:40.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.517+0000 7f9eadf5d700 1 -- 172.21.15.40:0/764776889 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea81dffc0 con 0x7f9ea80c9e20 2024-09-13T17:50:40.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.521+0000 7f9eadf5d700 1 -- 172.21.15.40:0/764776889 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea81e04f0 con 0x7f9ea80c9e20 2024-09-13T17:50:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.521+0000 7f9ea57fa700 1 -- 172.21.15.40:0/764776889 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9e980038e0 con 0x7f9ea80c9e20 2024-09-13T17:50:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.521+0000 7f9eadf5d700 1 -- 172.21.15.40:0/764776889 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea80cb730 con 0x7f9ea80c9e20 2024-09-13T17:50:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.521+0000 7f9ea57fa700 1 --2- 172.21.15.40:0/764776889 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e94066760 0x7f9e94068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.521+0000 7f9ea57fa700 1 -- 172.21.15.40:0/764776889 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9e98083680 con 0x7f9ea80c9e20 2024-09-13T17:50:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.521+0000 7f9ea7fff700 1 --2- 172.21.15.40:0/764776889 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e94066760 0x7f9e94068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:40.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.521+0000 7f9ea7fff700 1 --2- 172.21.15.40:0/764776889 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e94066760 0x7f9e94068c20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00b370 tx=0x7f9e9c009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:40.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.529+0000 7f9ea57fa700 1 -- 172.21.15.40:0/764776889 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9e98050790 con 0x7f9ea80c9e20 2024-09-13T17:50:40.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.705+0000 7f9eadf5d700 1 -- 172.21.15.40:0/764776889 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ea80ca250 con 0x7f9ea80c9e20 2024-09-13T17:50:40.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.705+0000 7f9ea57fa700 1 -- 172.21.15.40:0/764776889 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18262 (secure 0 0 0) 0x7f9e98053db0 con 0x7f9ea80c9e20 2024-09-13T17:50:40.714 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:50:40.715 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:50:38.974071+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:50:32.740336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:50:33.552914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:50:36.623639+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:50:40.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.709+0000 7f9eadf5d700 1 -- 172.21.15.40:0/764776889 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e94066760 msgr2=0x7f9e94068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:40.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.709+0000 7f9eadf5d700 1 --2- 172.21.15.40:0/764776889 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e94066760 0x7f9e94068c20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00b370 tx=0x7f9e9c009000 comp rx=0 tx=0).stop 2024-09-13T17:50:40.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.709+0000 7f9eadf5d700 1 -- 172.21.15.40:0/764776889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea80c9e20 msgr2=0x7f9ea81dfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:40.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.709+0000 7f9eadf5d700 1 --2- 172.21.15.40:0/764776889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea80c9e20 0x7f9ea81dfa80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9e9800c500 tx=0x7f9e9800c810 comp rx=0 tx=0).stop 2024-09-13T17:50:40.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.709+0000 7f9eadf5d700 1 -- 172.21.15.40:0/764776889 shutdown_connections 2024-09-13T17:50:40.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.709+0000 7f9eadf5d700 1 --2- 172.21.15.40:0/764776889 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e94066760 0x7f9e94068c20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.709+0000 7f9eadf5d700 1 --2- 172.21.15.40:0/764776889 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ea8153840 0x7f9ea81f28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.709+0000 7f9eadf5d700 1 --2- 172.21.15.40:0/764776889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ea80cc4f0 0x7f9ea81e4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.709+0000 7f9eadf5d700 1 --2- 172.21.15.40:0/764776889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea80c9e20 0x7f9ea81dfa80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:40.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.709+0000 7f9eadf5d700 1 -- 172.21.15.40:0/764776889 >> 172.21.15.40:0/764776889 conn(0x7f9ea806c800 msgr2=0x7f9ea8152980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:40.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.709+0000 7f9eadf5d700 1 -- 172.21.15.40:0/764776889 shutdown_connections 2024-09-13T17:50:40.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:40.709+0000 7f9eadf5d700 1 -- 172.21.15.40:0/764776889 wait complete. 2024-09-13T17:50:41.340 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T17:50:34.562623+0000', 'modified': '2024-09-13T17:50:39.976517+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.43:6837/334160033', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6836', 'nonce': 334160033}, {'type': 'v1', 'addr': '172.21.15.43:6837', 'nonce': 334160033}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-09-13T17:50:41.340 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T17:50:41.340 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-13T17:50:41.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.461+0000 7f01d89fb700 1 Processor -- start 2024-09-13T17:50:41.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.461+0000 7f01d89fb700 1 -- start start 2024-09-13T17:50:41.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.461+0000 7f01d89fb700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f01d40c7f60 0x7f01d4072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:41.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.461+0000 7f01d89fb700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f01d40c7320 0x7f01d415ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:41.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.461+0000 7f01d89fb700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d40c5290 con 0x7f01d40c8340 2024-09-13T17:50:41.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.461+0000 7f01d89fb700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d4071390 con 0x7f01d40c7320 2024-09-13T17:50:41.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.461+0000 7f01d89fb700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d40727f0 con 0x7f01d40c7f60 2024-09-13T17:50:41.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.461+0000 7f01d259c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f01d40c7320 0x7f01d415ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:41.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.461+0000 7f01d359e700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f01d40c8340 0x7f01d4151190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32856/0 (socket says 172.21.15.40:32856) 2024-09-13T17:50:41.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.461+0000 7f01d359e700 1 -- 172.21.15.40:0/1310713692 learned_addr learned my addr 172.21.15.40:0/1310713692 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:41.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.461+0000 7f01d2d9d700 1 --2- 172.21.15.40:0/1310713692 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f01d40c7f60 0x7f01d4072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:41.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.461+0000 7f01d259c700 1 -- 172.21.15.40:0/1310713692 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f01d40c7f60 msgr2=0x7f01d4072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:41.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.461+0000 7f01d259c700 1 --2- 172.21.15.40:0/1310713692 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f01d40c7f60 0x7f01d4072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:41.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.461+0000 7f01d259c700 1 -- 172.21.15.40:0/1310713692 >> v1:172.21.15.40:6789/0 conn(0x7f01d40c8340 legacy=0x7f01d4151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:41.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.461+0000 7f01d259c700 1 -- 172.21.15.40:0/1310713692 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d40c5870 con 0x7f01d40c7320 2024-09-13T17:50:41.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.461+0000 7f01d2d9d700 1 --2- 172.21.15.40:0/1310713692 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f01d40c7f60 0x7f01d4072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:50:41.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.465+0000 7f01d259c700 1 --2- 172.21.15.40:0/1310713692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d40c7320 0x7f01d415ab50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009d90 tx=0x7f01c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa27d982b3f74d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:41.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.465+0000 7f01d1d9b700 1 -- 172.21.15.40:0/1310713692 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c400e070 con 0x7f01d40c7320 2024-09-13T17:50:41.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.465+0000 7f01d1d9b700 1 -- 172.21.15.40:0/1310713692 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c400bad0 con 0x7f01d40c7320 2024-09-13T17:50:41.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.465+0000 7f01d1d9b700 1 -- 172.21.15.40:0/1310713692 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c400be80 con 0x7f01d40c7320 2024-09-13T17:50:41.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.465+0000 7f01d89fb700 1 -- 172.21.15.40:0/1310713692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d40c7320 msgr2=0x7f01d415ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:41.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.465+0000 7f01d89fb700 1 --2- 172.21.15.40:0/1310713692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d40c7320 0x7f01d415ab50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009d90 tx=0x7f01c400b330 comp rx=0 tx=0).stop 2024-09-13T17:50:41.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.465+0000 7f01d89fb700 1 -- 172.21.15.40:0/1310713692 shutdown_connections 2024-09-13T17:50:41.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.465+0000 7f01d89fb700 1 --2- 172.21.15.40:0/1310713692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d40c7320 0x7f01d415ab50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:41.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.465+0000 7f01d89fb700 1 --2- 172.21.15.40:0/1310713692 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f01d40c7f60 0x7f01d4072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:41.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.465+0000 7f01d89fb700 1 -- 172.21.15.40:0/1310713692 >> 172.21.15.40:0/1310713692 conn(0x7f01d4066090 msgr2=0x7f01d40664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:41.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.465+0000 7f01d89fb700 1 -- 172.21.15.40:0/1310713692 shutdown_connections 2024-09-13T17:50:41.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.465+0000 7f01d89fb700 1 -- 172.21.15.40:0/1310713692 wait complete. 2024-09-13T17:50:41.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.465+0000 7f01d89fb700 1 Processor -- start 2024-09-13T17:50:41.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.465+0000 7f01d89fb700 1 -- start start 2024-09-13T17:50:41.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.465+0000 7f01d89fb700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f01d40c7320 0x7f01d414c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:41.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.465+0000 7f01d89fb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01d40c7f60 0x7f01d40a9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.465+0000 7f01d89fb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d40c8340 0x7f01d40a9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.465+0000 7f01d89fb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d40c40a0 con 0x7f01d40c8340 2024-09-13T17:50:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.465+0000 7f01d89fb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d40c8bd0 con 0x7f01d40c7f60 2024-09-13T17:50:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.465+0000 7f01d89fb700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d40c8e90 con 0x7f01d40c7320 2024-09-13T17:50:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.469+0000 7f01d359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d40c8340 0x7f01d40a9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.469+0000 7f01d359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d40c8340 0x7f01d40a9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52508/0 (socket says 172.21.15.40:52508) 2024-09-13T17:50:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.469+0000 7f01d359e700 1 -- 172.21.15.40:0/33220753 learned_addr learned my addr 172.21.15.40:0/33220753 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.469+0000 7f01d259c700 1 --2- 172.21.15.40:0/33220753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01d40c7f60 0x7f01d40a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:41.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.469+0000 7f01d2d9d700 1 --2- 172.21.15.40:0/33220753 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f01d40c7320 0x7f01d414c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:41.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.469+0000 7f01d359e700 1 -- 172.21.15.40:0/33220753 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f01d40c7320 msgr2=0x7f01d414c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:41.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.469+0000 7f01d359e700 1 --2- 172.21.15.40:0/33220753 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f01d40c7320 0x7f01d414c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:41.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.469+0000 7f01d359e700 1 -- 172.21.15.40:0/33220753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01d40c7f60 msgr2=0x7f01d40a9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:41.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.469+0000 7f01d359e700 1 --2- 172.21.15.40:0/33220753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01d40c7f60 0x7f01d40a9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:41.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.469+0000 7f01d359e700 1 -- 172.21.15.40:0/33220753 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c4009a70 con 0x7f01d40c8340 2024-09-13T17:50:41.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.469+0000 7f01d359e700 1 --2- 172.21.15.40:0/33220753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d40c8340 0x7f01d40a9eb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00c520 tx=0x7f01cc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:41.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.469+0000 7f01bb7fe700 1 -- 172.21.15.40:0/33220753 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc004ca0 con 0x7f01d40c8340 2024-09-13T17:50:41.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.469+0000 7f01bb7fe700 1 -- 172.21.15.40:0/33220753 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01cc004e00 con 0x7f01d40c8340 2024-09-13T17:50:41.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.469+0000 7f01d89fb700 1 -- 172.21.15.40:0/33220753 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d41e3910 con 0x7f01d40c8340 2024-09-13T17:50:41.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.469+0000 7f01bb7fe700 1 -- 172.21.15.40:0/33220753 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc0072f0 con 0x7f01d40c8340 2024-09-13T17:50:41.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.469+0000 7f01d89fb700 1 -- 172.21.15.40:0/33220753 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d41f12a0 con 0x7f01d40c8340 2024-09-13T17:50:41.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.473+0000 7f01bb7fe700 1 -- 172.21.15.40:0/33220753 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f01cc007450 con 0x7f01d40c8340 2024-09-13T17:50:41.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.473+0000 7f01bb7fe700 1 --2- 172.21.15.40:0/33220753 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f01c0066730 0x7f01c0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:41.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.473+0000 7f01bb7fe700 1 -- 172.21.15.40:0/33220753 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f01cc0840d0 con 0x7f01d40c8340 2024-09-13T17:50:41.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.473+0000 7f01d2d9d700 1 --2- 172.21.15.40:0/33220753 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f01c0066730 0x7f01c0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:41.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.473+0000 7f01d2d9d700 1 --2- 172.21.15.40:0/33220753 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f01c0066730 0x7f01c0068bf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009930 tx=0x7f01c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:41.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.473+0000 7f01dc4ad740 1 Processor -- start 2024-09-13T17:50:41.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.473+0000 7f01dc4ad740 1 -- start start 2024-09-13T17:50:41.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01dc4ad740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe3ddb0 0xe299d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:41.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01dc4ad740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe29f10 0xe2a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:41.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01dc4ad740 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe2a830 0xe2ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:41.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01dc4ad740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0xd74d80 con 0xe3ddb0 2024-09-13T17:50:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01dc4ad740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0xd72510 con 0xe29f10 2024-09-13T17:50:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01dc4ad740 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0xd6de00 con 0xe2a830 2024-09-13T17:50:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01d2d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe3ddb0 0xe299d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01d2d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe3ddb0 0xe299d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52514/0 (socket says 172.21.15.40:52514) 2024-09-13T17:50:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01d2d9d700 1 -- 172.21.15.40:0/2955733817 learned_addr learned my addr 172.21.15.40:0/2955733817 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01d359e700 1 --2- 172.21.15.40:0/2955733817 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe2a830 0xe2ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01d259c700 1 --2- 172.21.15.40:0/2955733817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe29f10 0xe2a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01d2d9d700 1 -- 172.21.15.40:0/2955733817 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe2a830 msgr2=0xe2ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01d2d9d700 1 --2- 172.21.15.40:0/2955733817 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe2a830 0xe2ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01d2d9d700 1 -- 172.21.15.40:0/2955733817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe29f10 msgr2=0xe2a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:41.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01d2d9d700 1 --2- 172.21.15.40:0/2955733817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe29f10 0xe2a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:41.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01d2d9d700 1 -- 172.21.15.40:0/2955733817 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xbe0960 con 0xe3ddb0 2024-09-13T17:50:41.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01d259c700 1 --2- 172.21.15.40:0/2955733817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe29f10 0xe2a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:50:41.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01d2d9d700 1 --2- 172.21.15.40:0/2955733817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe3ddb0 0xe299d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f01c8010870 tx=0x7f01c8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77d9a63b0524441 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:41.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01b8ff9700 1 -- 172.21.15.40:0/2955733817 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8015900 con 0xe3ddb0 2024-09-13T17:50:41.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01b8ff9700 1 -- 172.21.15.40:0/2955733817 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c801a070 con 0xe3ddb0 2024-09-13T17:50:41.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01b8ff9700 1 -- 172.21.15.40:0/2955733817 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8015c80 con 0xe3ddb0 2024-09-13T17:50:41.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01dc4ad740 1 -- 172.21.15.40:0/2955733817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe3ddb0 msgr2=0xe299d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:41.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01dc4ad740 1 --2- 172.21.15.40:0/2955733817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe3ddb0 0xe299d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f01c8010870 tx=0x7f01c8013000 comp rx=0 tx=0).stop 2024-09-13T17:50:41.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01dc4ad740 1 -- 172.21.15.40:0/2955733817 shutdown_connections 2024-09-13T17:50:41.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01dc4ad740 1 --2- 172.21.15.40:0/2955733817 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe2a830 0xe2ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:41.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01dc4ad740 1 --2- 172.21.15.40:0/2955733817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe29f10 0xe2a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:41.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01dc4ad740 1 --2- 172.21.15.40:0/2955733817 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe3ddb0 0xe299d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:41.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01dc4ad740 1 -- 172.21.15.40:0/2955733817 >> 172.21.15.40:0/2955733817 conn(0xb954d0 msgr2=0xe40ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:41.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01dc4ad740 1 -- 172.21.15.40:0/2955733817 shutdown_connections 2024-09-13T17:50:41.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01dc4ad740 1 -- 172.21.15.40:0/2955733817 wait complete. 2024-09-13T17:50:41.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.477+0000 7f01dc4ad740 1 -- start start 2024-09-13T17:50:41.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01dc4ad740 1 Processor -- start 2024-09-13T17:50:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01dc4ad740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe29f10 0xe3d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01dc4ad740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe2a830 0xe04c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01dc4ad740 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe3ddb0 0xe05180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01dc4ad740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0xd2b730 con 0xe29f10 2024-09-13T17:50:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01dc4ad740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0xd315b0 con 0xe2a830 2024-09-13T17:50:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01dc4ad740 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0xc19aa0 con 0xe3ddb0 2024-09-13T17:50:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01dc4ad740 10 client.0 _ll_register_callbacks cb 0xd67610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:50:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01d2d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe29f10 0xe3d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01d2d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe29f10 0xe3d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52526/0 (socket says 172.21.15.40:52526) 2024-09-13T17:50:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01d2d9d700 1 -- 172.21.15.40:0/1301009458 learned_addr learned my addr 172.21.15.40:0/1301009458 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01d359e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe3ddb0 0xe05180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:41.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01d259c700 1 --2- 172.21.15.40:0/1301009458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe2a830 0xe04c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01d2d9d700 1 -- 172.21.15.40:0/1301009458 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe3ddb0 msgr2=0xe05180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01d2d9d700 1 --2- 172.21.15.40:0/1301009458 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe3ddb0 0xe05180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01d2d9d700 1 -- 172.21.15.40:0/1301009458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe2a830 msgr2=0xe04c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01d2d9d700 1 --2- 172.21.15.40:0/1301009458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe2a830 0xe04c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:41.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01d2d9d700 1 -- 172.21.15.40:0/1301009458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c800e050 con 0xe29f10 2024-09-13T17:50:41.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01d2d9d700 1 --2- 172.21.15.40:0/1301009458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe29f10 0xe3d650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0xd717f0 tx=0x7f01c80047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8c7335cb91c2bf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:41.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-09-13T17:50:41.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01b1ffb700 1 -- 172.21.15.40:0/1301009458 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8035670 con 0xe29f10 2024-09-13T17:50:41.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01b1ffb700 1 -- 172.21.15.40:0/1301009458 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f01c80357d0 con 0xe29f10 2024-09-13T17:50:41.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01dc4ad740 1 -- 172.21.15.40:0/1301009458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xba21e0 con 0xe29f10 2024-09-13T17:50:41.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01b1ffb700 1 -- 172.21.15.40:0/1301009458 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01c80359d0 con 0xe29f10 2024-09-13T17:50:41.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01dc4ad740 10 client.4443 fetch_fsmap learned FSMap version 7 2024-09-13T17:50:41.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01dc4ad740 1 -- 172.21.15.40:0/1301009458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0xd34220 con 0xe29f10 2024-09-13T17:50:41.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01b1ffb700 1 -- 172.21.15.40:0/1301009458 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xba21e0 con 0xe29f10 2024-09-13T17:50:41.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.481+0000 7f01b1ffb700 1 -- 172.21.15.40:0/1301009458 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 7) v1 ==== 4212+0+0 (secure 0 0 0) 0x7f01c8035b30 con 0xe29f10 2024-09-13T17:50:41.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.485+0000 7f01dc4ad740 10 client.4443 fetch_fsmap finished waiting for FSMap version 7 2024-09-13T17:50:41.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.485+0000 7f01dc4ad740 10 client.4443 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T17:50:41.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.485+0000 7f01dc4ad740 20 client.4443 populate_metadata read hostname 'smithi040' 2024-09-13T17:50:41.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.485+0000 7f01dc4ad740 1 --2- 172.21.15.40:0/1301009458 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0xe27380 0xe27760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:41.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.485+0000 7f01dc4ad740 4 client.4443 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:50:41.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.485+0000 7f01dc4ad740 1 -- 172.21.15.40:0/1301009458 --> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xbec460 con 0xe27380 2024-09-13T17:50:41.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.485+0000 7f01d259c700 1 --2- 172.21.15.40:0/1301009458 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0xe27380 0xe27760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:41.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.485+0000 7f01b1ffb700 10 client.4443 ms_handle_connect on v2:172.21.15.43:6836/334160033 2024-09-13T17:50:41.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:41.485+0000 7f01d259c700 1 --2- 172.21.15.40:0/1301009458 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0xe27380 0xe27760 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9017792da9cd2bc server_cookie=94b3afe8eef7786d in_seq=0 out_seq=0 2024-09-13T17:50:44.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:44.981+0000 7f01b1ffb700 1 -- 172.21.15.40:0/1301009458 <== mds.0 v2:172.21.15.43:6836/334160033 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0xbec460 con 0xe27380 2024-09-13T17:50:44.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:44.981+0000 7f01b1ffb700 10 client.4443 handle_command_reply: tid=0 2024-09-13T17:50:44.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:44.981+0000 7f01dc4ad740 1 client.4443 shutdown 2024-09-13T17:50:44.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:44.981+0000 7f01dc4ad740 1 -- 172.21.15.40:0/1301009458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe29f10 msgr2=0xe3d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:44.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:44.981+0000 7f01dc4ad740 1 --2- 172.21.15.40:0/1301009458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe29f10 0xe3d650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0xd717f0 tx=0x7f01c80047e0 comp rx=0 tx=0).stop 2024-09-13T17:50:44.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:44.981+0000 7f01dc4ad740 1 -- 172.21.15.40:0/1301009458 shutdown_connections 2024-09-13T17:50:44.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:44.981+0000 7f01dc4ad740 1 --2- 172.21.15.40:0/1301009458 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0xe27380 0xe27760 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:44.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:44.981+0000 7f01dc4ad740 1 --2- 172.21.15.40:0/1301009458 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe3ddb0 0xe05180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:44.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:44.981+0000 7f01dc4ad740 1 --2- 172.21.15.40:0/1301009458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe2a830 0xe04c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:44.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:44.981+0000 7f01b1ffb700 0 client.4443 ms_handle_reset on v2:172.21.15.43:6836/334160033 2024-09-13T17:50:44.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:44.981+0000 7f01dc4ad740 1 --2- 172.21.15.40:0/1301009458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe29f10 0xe3d650 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:44.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:44.981+0000 7f01dc4ad740 1 -- 172.21.15.40:0/1301009458 >> 172.21.15.40:0/1301009458 conn(0xb954d0 msgr2=0xe3e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:44.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:44.981+0000 7f01dc4ad740 1 -- 172.21.15.40:0/1301009458 shutdown_connections 2024-09-13T17:50:44.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:44.981+0000 7f01dc4ad740 1 -- 172.21.15.40:0/1301009458 wait complete. 2024-09-13T17:50:44.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:44.981+0000 7f01dc4ad740 20 client.4443 trim_cache size 0 max 16384 2024-09-13T17:50:45.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 Processor -- start 2024-09-13T17:50:45.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 -- start start 2024-09-13T17:50:45.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe29f10 0xe59210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:45.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2a830 0xe27760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:45.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe3ddb0 0xe27ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:45.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0xd0c470 con 0xe2a830 2024-09-13T17:50:45.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0xd354c0 con 0xe3ddb0 2024-09-13T17:50:45.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0xd14950 con 0xe29f10 2024-09-13T17:50:45.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2a830 0xe27760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:45.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2a830 0xe27760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52536/0 (socket says 172.21.15.40:52536) 2024-09-13T17:50:45.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d259c700 1 -- 172.21.15.40:0/2437012393 learned_addr learned my addr 172.21.15.40:0/2437012393 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:45.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d2d9d700 1 --2- 172.21.15.40:0/2437012393 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe29f10 0xe59210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:45.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d359e700 1 --2- 172.21.15.40:0/2437012393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe3ddb0 0xe27ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:45.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d259c700 1 -- 172.21.15.40:0/2437012393 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe29f10 msgr2=0xe59210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:45.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d259c700 1 --2- 172.21.15.40:0/2437012393 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe29f10 0xe59210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d259c700 1 -- 172.21.15.40:0/2437012393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe3ddb0 msgr2=0xe27ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:45.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d259c700 1 --2- 172.21.15.40:0/2437012393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe3ddb0 0xe27ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d259c700 1 -- 172.21.15.40:0/2437012393 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xd1a450 con 0xe2a830 2024-09-13T17:50:45.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d359e700 1 --2- 172.21.15.40:0/2437012393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe3ddb0 0xe27ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:50:45.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d2d9d700 1 --2- 172.21.15.40:0/2437012393 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe29f10 0xe59210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:50:45.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d259c700 1 --2- 172.21.15.40:0/2437012393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2a830 0xe27760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f01c400b730 tx=0x7f01c4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db44e87da4e71fd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:45.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01b97fa700 1 -- 172.21.15.40:0/2437012393 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4023030 con 0xe2a830 2024-09-13T17:50:45.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01b97fa700 1 -- 172.21.15.40:0/2437012393 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c4024040 con 0xe2a830 2024-09-13T17:50:45.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01b97fa700 1 -- 172.21.15.40:0/2437012393 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4027a90 con 0xe2a830 2024-09-13T17:50:45.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 -- 172.21.15.40:0/2437012393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2a830 msgr2=0xe27760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:45.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 --2- 172.21.15.40:0/2437012393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2a830 0xe27760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f01c400b730 tx=0x7f01c4010000 comp rx=0 tx=0).stop 2024-09-13T17:50:45.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 -- 172.21.15.40:0/2437012393 shutdown_connections 2024-09-13T17:50:45.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 --2- 172.21.15.40:0/2437012393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe3ddb0 0xe27ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 --2- 172.21.15.40:0/2437012393 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2a830 0xe27760 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 --2- 172.21.15.40:0/2437012393 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe29f10 0xe59210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 -- 172.21.15.40:0/2437012393 >> 172.21.15.40:0/2437012393 conn(0xb954d0 msgr2=0xe07450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:45.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 -- 172.21.15.40:0/2437012393 shutdown_connections 2024-09-13T17:50:45.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 -- 172.21.15.40:0/2437012393 wait complete. 2024-09-13T17:50:45.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 -- start start 2024-09-13T17:50:45.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 Processor -- start 2024-09-13T17:50:45.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe29f10 0xeada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:45.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2a830 0xe580b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:45.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe3ddb0 0xea3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:45.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0xe29bb0 con 0xe2a830 2024-09-13T17:50:45.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0xe6c4c0 con 0xe29f10 2024-09-13T17:50:45.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0xe3d140 con 0xe3ddb0 2024-09-13T17:50:45.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01dc4ad740 10 client.0 _ll_register_callbacks cb 0xd1fce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:50:45.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2a830 0xe580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:45.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2a830 0xe580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52544/0 (socket says 172.21.15.40:52544) 2024-09-13T17:50:45.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d259c700 1 -- 172.21.15.40:0/2446494916 learned_addr learned my addr 172.21.15.40:0/2446494916 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d2d9d700 1 --2- 172.21.15.40:0/2446494916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe29f10 0xeada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d359e700 1 --2- 172.21.15.40:0/2446494916 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe3ddb0 0xea3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d259c700 1 -- 172.21.15.40:0/2446494916 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe3ddb0 msgr2=0xea3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d259c700 1 --2- 172.21.15.40:0/2446494916 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe3ddb0 0xea3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d259c700 1 -- 172.21.15.40:0/2446494916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe29f10 msgr2=0xeada40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d259c700 1 --2- 172.21.15.40:0/2446494916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe29f10 0xeada40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d259c700 1 -- 172.21.15.40:0/2446494916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d40c5870 con 0xe2a830 2024-09-13T17:50:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01d259c700 1 --2- 172.21.15.40:0/2446494916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2a830 0xe580b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f01c400d010 tx=0x7f01c400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ea053278511ccf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.005+0000 7f01b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-09-13T17:50:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.009+0000 7f01b1ffb700 1 -- 172.21.15.40:0/2446494916 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4023050 con 0xe2a830 2024-09-13T17:50:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.009+0000 7f01b1ffb700 1 -- 172.21.15.40:0/2446494916 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f01c4004030 con 0xe2a830 2024-09-13T17:50:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.009+0000 7f01dc4ad740 1 -- 172.21.15.40:0/2446494916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xe2b1a0 con 0xe2a830 2024-09-13T17:50:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.009+0000 7f01b1ffb700 1 -- 172.21.15.40:0/2446494916 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01c40041f0 con 0xe2a830 2024-09-13T17:50:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.009+0000 7f01dc4ad740 10 client.4449 fetch_fsmap learned FSMap version 7 2024-09-13T17:50:45.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.009+0000 7f01dc4ad740 1 -- 172.21.15.40:0/2446494916 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0xd0e720 con 0xe2a830 2024-09-13T17:50:45.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.009+0000 7f01b1ffb700 1 -- 172.21.15.40:0/2446494916 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xe2b1a0 con 0xe2a830 2024-09-13T17:50:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.009+0000 7f01b1ffb700 1 -- 172.21.15.40:0/2446494916 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 7) v1 ==== 4212+0+0 (secure 0 0 0) 0x7f01c400b050 con 0xe2a830 2024-09-13T17:50:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.009+0000 7f01dc4ad740 10 client.4449 fetch_fsmap finished waiting for FSMap version 7 2024-09-13T17:50:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.009+0000 7f01dc4ad740 10 client.4449 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T17:50:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.009+0000 7f01dc4ad740 20 client.4449 populate_metadata read hostname 'smithi040' 2024-09-13T17:50:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.009+0000 7f01dc4ad740 1 --2- 172.21.15.40:0/2446494916 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0xe6b420 0xe6b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.009+0000 7f01dc4ad740 4 client.4449 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T17:50:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.009+0000 7f01dc4ad740 1 -- 172.21.15.40:0/2446494916 --> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f01c801a070 con 0xe6b420 2024-09-13T17:50:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.009+0000 7f01d2d9d700 1 --2- 172.21.15.40:0/2446494916 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0xe6b420 0xe6b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.009+0000 7f01d2d9d700 1 -- 172.21.15.40:0/2446494916 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0xe6b420 msgr2=0xe6b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 17 2024-09-13T17:50:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.009+0000 7f01d2d9d700 1 -- 172.21.15.40:0/2446494916 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0xe6b420 msgr2=0xe6b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:50:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.009+0000 7f01d2d9d700 1 --2- 172.21.15.40:0/2446494916 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0xe6b420 0xe6b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:50:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.009+0000 7f01d2d9d700 1 --2- 172.21.15.40:0/2446494916 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0xe6b420 0xe6b800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:50:45.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01d2d9d700 1 --2- 172.21.15.40:0/2446494916 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0xe6b420 0xe6b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01d2d9d700 1 --2- 172.21.15.40:0/2446494916 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0xe6b420 0xe6b800 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41bd5856e4f0d4f server_cookie=a8eeab18768db90e in_seq=0 out_seq=0 2024-09-13T17:50:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01b1ffb700 10 client.4449 ms_handle_connect on v2:172.21.15.43:6836/334160033 2024-09-13T17:50:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01b1ffb700 1 -- 172.21.15.40:0/2446494916 <== mds.0 v2:172.21.15.43:6836/334160033 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x7f01c801a070 con 0xe6b420 2024-09-13T17:50:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01b1ffb700 10 client.4449 handle_command_reply: tid=0 2024-09-13T17:50:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01dc4ad740 1 client.4449 shutdown 2024-09-13T17:50:45.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01dc4ad740 1 -- 172.21.15.40:0/2446494916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2a830 msgr2=0xe580b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01dc4ad740 1 --2- 172.21.15.40:0/2446494916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2a830 0xe580b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f01c400d010 tx=0x7f01c400e000 comp rx=0 tx=0).stop 2024-09-13T17:50:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01dc4ad740 1 -- 172.21.15.40:0/2446494916 shutdown_connections 2024-09-13T17:50:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01dc4ad740 1 --2- 172.21.15.40:0/2446494916 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0xe6b420 0xe6b800 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01dc4ad740 1 --2- 172.21.15.40:0/2446494916 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe3ddb0 0xea3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01b1ffb700 0 client.4449 ms_handle_reset on v2:172.21.15.43:6836/334160033 2024-09-13T17:50:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01dc4ad740 1 --2- 172.21.15.40:0/2446494916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe2a830 0xe580b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01dc4ad740 1 --2- 172.21.15.40:0/2446494916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe29f10 0xeada40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01dc4ad740 1 -- 172.21.15.40:0/2446494916 >> 172.21.15.40:0/2446494916 conn(0xb954d0 msgr2=0xe081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01dc4ad740 1 -- 172.21.15.40:0/2446494916 shutdown_connections 2024-09-13T17:50:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01dc4ad740 1 -- 172.21.15.40:0/2446494916 wait complete. 2024-09-13T17:50:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01dc4ad740 20 client.4449 trim_cache size 0 max 16384 2024-09-13T17:50:45.221 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:50:45.221 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:50:45.221 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-09-13T17:50:45.221 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4288, 2024-09-13T17:50:45.221 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-09-13T17:50:45.221 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-09-13T17:50:45.221 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-09-13T17:50:45.222 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 10.650168455999999, 2024-09-13T17:50:45.222 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 6, 2024-09-13T17:50:45.222 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 25, 2024-09-13T17:50:45.222 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 25, 2024-09-13T17:50:45.222 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 17.077556230999999 2024-09-13T17:50:45.222 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:50:45.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01d89fb700 1 -- 172.21.15.40:0/33220753 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f01c0066730 msgr2=0x7f01c0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:45.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01d89fb700 1 --2- 172.21.15.40:0/33220753 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f01c0066730 0x7f01c0068bf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009930 tx=0x7f01c8008040 comp rx=0 tx=0).stop 2024-09-13T17:50:45.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01d89fb700 1 -- 172.21.15.40:0/33220753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d40c8340 msgr2=0x7f01d40a9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:45.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01d89fb700 1 --2- 172.21.15.40:0/33220753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d40c8340 0x7f01d40a9eb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00c520 tx=0x7f01cc00c830 comp rx=0 tx=0).stop 2024-09-13T17:50:45.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01d89fb700 1 -- 172.21.15.40:0/33220753 shutdown_connections 2024-09-13T17:50:45.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01d89fb700 1 --2- 172.21.15.40:0/33220753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01d40c8340 0x7f01d40a9eb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01d89fb700 1 --2- 172.21.15.40:0/33220753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01d40c7f60 0x7f01d40a9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01d89fb700 1 --2- 172.21.15.40:0/33220753 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f01c0066730 0x7f01c0068bf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01d89fb700 1 --2- 172.21.15.40:0/33220753 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f01d40c7320 0x7f01d414c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01d89fb700 1 -- 172.21.15.40:0/33220753 >> 172.21.15.40:0/33220753 conn(0x7f01d4066090 msgr2=0x7f01d414f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:45.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01d89fb700 1 -- 172.21.15.40:0/33220753 shutdown_connections 2024-09-13T17:50:45.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.213+0000 7f01d89fb700 1 -- 172.21.15.40:0/33220753 wait complete. 2024-09-13T17:50:45.240 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T17:50:45.240 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:50:45.240 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:50:45.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.353+0000 7f5e8dd7f700 1 Processor -- start 2024-09-13T17:50:45.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.353+0000 7f5e8dd7f700 1 -- start start 2024-09-13T17:50:45.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e8dd7f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5e88072240 0x7f5e880c0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:45.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e8dd7f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5e880c28f0 0x7f5e881565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:45.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e8dd7f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e880b2210 con 0x7f5e88071e60 2024-09-13T17:50:45.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e8dd7f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e880c1780 con 0x7f5e88072240 2024-09-13T17:50:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e8dd7f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e880c1a60 con 0x7f5e880c28f0 2024-09-13T17:50:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e87fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5e88072240 0x7f5e880c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e8cd7d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5e88071e60 0x7f5e880c32e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32860/0 (socket says 172.21.15.40:32860) 2024-09-13T17:50:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e8cd7d700 1 -- 172.21.15.40:0/2663069842 learned_addr learned my addr 172.21.15.40:0/2663069842 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e8d57e700 1 --2- 172.21.15.40:0/2663069842 >> v2:172.21.15.43:3300/0 conn(0x7f5e880c28f0 0x7f5e881565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e877fe700 1 -- 172.21.15.40:0/2663069842 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713773510 0 0) 0x7f5e880b2210 con 0x7f5e88071e60 2024-09-13T17:50:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e877fe700 1 -- 172.21.15.40:0/2663069842 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e740035d0 con 0x7f5e88071e60 2024-09-13T17:50:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e87fff700 1 -- 172.21.15.40:0/2663069842 >> v2:172.21.15.43:3300/0 conn(0x7f5e880c28f0 msgr2=0x7f5e881565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e87fff700 1 --2- 172.21.15.40:0/2663069842 >> v2:172.21.15.43:3300/0 conn(0x7f5e880c28f0 0x7f5e881565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e87fff700 1 -- 172.21.15.40:0/2663069842 >> v1:172.21.15.40:6789/0 conn(0x7f5e88071e60 legacy=0x7f5e880c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e87fff700 1 -- 172.21.15.40:0/2663069842 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e880c5d70 con 0x7f5e88072240 2024-09-13T17:50:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e87fff700 1 --2- 172.21.15.40:0/2663069842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e88072240 0x7f5e880c0e90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5e880c9c70 tx=0x7f5e70009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1235844d262bf996 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e877fe700 1 -- 172.21.15.40:0/2663069842 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70011830 con 0x7f5e88072240 2024-09-13T17:50:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e877fe700 1 -- 172.21.15.40:0/2663069842 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e70011990 con 0x7f5e88072240 2024-09-13T17:50:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e877fe700 1 -- 172.21.15.40:0/2663069842 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70011d20 con 0x7f5e88072240 2024-09-13T17:50:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e8dd7f700 1 -- 172.21.15.40:0/2663069842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e88072240 msgr2=0x7f5e880c0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e8dd7f700 1 --2- 172.21.15.40:0/2663069842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e88072240 0x7f5e880c0e90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5e880c9c70 tx=0x7f5e70009b30 comp rx=0 tx=0).stop 2024-09-13T17:50:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e8dd7f700 1 -- 172.21.15.40:0/2663069842 shutdown_connections 2024-09-13T17:50:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e8dd7f700 1 --2- 172.21.15.40:0/2663069842 >> v2:172.21.15.43:3300/0 conn(0x7f5e880c28f0 0x7f5e881565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e8dd7f700 1 --2- 172.21.15.40:0/2663069842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e88072240 0x7f5e880c0e90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e8dd7f700 1 -- 172.21.15.40:0/2663069842 >> 172.21.15.40:0/2663069842 conn(0x7f5e8806e4f0 msgr2=0x7f5e8806e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e8dd7f700 1 -- 172.21.15.40:0/2663069842 shutdown_connections 2024-09-13T17:50:45.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.357+0000 7f5e8dd7f700 1 -- 172.21.15.40:0/2663069842 wait complete. 2024-09-13T17:50:45.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.361+0000 7f5e8dd7f700 1 Processor -- start 2024-09-13T17:50:45.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.361+0000 7f5e8dd7f700 1 -- start start 2024-09-13T17:50:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.361+0000 7f5e8dd7f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e88071e60 0x7f5e881f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.361+0000 7f5e8dd7f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e88072240 0x7f5e881f2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.361+0000 7f5e8dd7f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5e880c28f0 0x7f5e881f06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.361+0000 7f5e8dd7f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e881f0250 con 0x7f5e88072240 2024-09-13T17:50:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.361+0000 7f5e8dd7f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e881f2d50 con 0x7f5e88071e60 2024-09-13T17:50:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.361+0000 7f5e8dd7f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e881eff20 con 0x7f5e880c28f0 2024-09-13T17:50:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.361+0000 7f5e87fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e88072240 0x7f5e881f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.361+0000 7f5e87fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e88072240 0x7f5e881f2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52568/0 (socket says 172.21.15.40:52568) 2024-09-13T17:50:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.361+0000 7f5e87fff700 1 -- 172.21.15.40:0/3609120486 learned_addr learned my addr 172.21.15.40:0/3609120486 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.361+0000 7f5e8cd7d700 1 --2- 172.21.15.40:0/3609120486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e88071e60 0x7f5e881f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.361+0000 7f5e8d57e700 1 --2- 172.21.15.40:0/3609120486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5e880c28f0 0x7f5e881f06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.361+0000 7f5e87fff700 1 -- 172.21.15.40:0/3609120486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5e880c28f0 msgr2=0x7f5e881f06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.361+0000 7f5e87fff700 1 --2- 172.21.15.40:0/3609120486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5e880c28f0 0x7f5e881f06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.361+0000 7f5e87fff700 1 -- 172.21.15.40:0/3609120486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e88071e60 msgr2=0x7f5e881f21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.361+0000 7f5e87fff700 1 --2- 172.21.15.40:0/3609120486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e88071e60 0x7f5e881f21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.361+0000 7f5e87fff700 1 -- 172.21.15.40:0/3609120486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e700094b0 con 0x7f5e88072240 2024-09-13T17:50:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.361+0000 7f5e8d57e700 1 --2- 172.21.15.40:0/3609120486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5e880c28f0 0x7f5e881f06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:50:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.365+0000 7f5e87fff700 1 --2- 172.21.15.40:0/3609120486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e88072240 0x7f5e881f2730 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5e700090b0 tx=0x7f5e70004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.365+0000 7f5e857fa700 1 -- 172.21.15.40:0/3609120486 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70004400 con 0x7f5e88072240 2024-09-13T17:50:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.365+0000 7f5e857fa700 1 -- 172.21.15.40:0/3609120486 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7001f070 con 0x7f5e88072240 2024-09-13T17:50:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.365+0000 7f5e8dd7f700 1 -- 172.21.15.40:0/3609120486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e881542b0 con 0x7f5e88072240 2024-09-13T17:50:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.365+0000 7f5e857fa700 1 -- 172.21.15.40:0/3609120486 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e700036c0 con 0x7f5e88072240 2024-09-13T17:50:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.365+0000 7f5e8dd7f700 1 -- 172.21.15.40:0/3609120486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e881551a0 con 0x7f5e88072240 2024-09-13T17:50:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.365+0000 7f5e8dd7f700 1 -- 172.21.15.40:0/3609120486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e880c1ec0 con 0x7f5e88072240 2024-09-13T17:50:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.365+0000 7f5e857fa700 1 -- 172.21.15.40:0/3609120486 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5e70021030 con 0x7f5e88072240 2024-09-13T17:50:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.365+0000 7f5e857fa700 1 --2- 172.21.15.40:0/3609120486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5e68066760 0x7f5e68068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.365+0000 7f5e857fa700 1 -- 172.21.15.40:0/3609120486 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5e700855e0 con 0x7f5e88072240 2024-09-13T17:50:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.365+0000 7f5e8cd7d700 1 --2- 172.21.15.40:0/3609120486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5e68066760 0x7f5e68068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.369+0000 7f5e8cd7d700 1 --2- 172.21.15.40:0/3609120486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5e68066760 0x7f5e68068c20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5e78007fd0 tx=0x7f5e78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:45.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.373+0000 7f5e857fa700 1 -- 172.21.15.40:0/3609120486 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5e700526f0 con 0x7f5e88072240 2024-09-13T17:50:45.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.585+0000 7f5e8dd7f700 1 -- 172.21.15.40:0/3609120486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5e880c0e90 con 0x7f5e88072240 2024-09-13T17:50:45.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.585+0000 7f5e857fa700 1 -- 172.21.15.40:0/3609120486 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f5e70055d10 con 0x7f5e88072240 2024-09-13T17:50:45.593 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-13T17:50:45.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.589+0000 7f5e8dd7f700 1 -- 172.21.15.40:0/3609120486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5e68066760 msgr2=0x7f5e68068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:45.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.589+0000 7f5e8dd7f700 1 --2- 172.21.15.40:0/3609120486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5e68066760 0x7f5e68068c20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5e78007fd0 tx=0x7f5e78009040 comp rx=0 tx=0).stop 2024-09-13T17:50:45.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.589+0000 7f5e8dd7f700 1 -- 172.21.15.40:0/3609120486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e88072240 msgr2=0x7f5e881f2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:45.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.589+0000 7f5e8dd7f700 1 --2- 172.21.15.40:0/3609120486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e88072240 0x7f5e881f2730 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5e700090b0 tx=0x7f5e70004770 comp rx=0 tx=0).stop 2024-09-13T17:50:45.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.589+0000 7f5e8dd7f700 1 -- 172.21.15.40:0/3609120486 shutdown_connections 2024-09-13T17:50:45.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.589+0000 7f5e8dd7f700 1 --2- 172.21.15.40:0/3609120486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5e68066760 0x7f5e68068c20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.589+0000 7f5e8dd7f700 1 --2- 172.21.15.40:0/3609120486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5e880c28f0 0x7f5e881f06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.589+0000 7f5e8dd7f700 1 --2- 172.21.15.40:0/3609120486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5e88072240 0x7f5e881f2730 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.589+0000 7f5e8dd7f700 1 --2- 172.21.15.40:0/3609120486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e88071e60 0x7f5e881f21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:45.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.589+0000 7f5e8dd7f700 1 -- 172.21.15.40:0/3609120486 >> 172.21.15.40:0/3609120486 conn(0x7f5e8806e4f0 msgr2=0x7f5e88158990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:45.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.589+0000 7f5e8dd7f700 1 -- 172.21.15.40:0/3609120486 shutdown_connections 2024-09-13T17:50:45.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:45.589+0000 7f5e8dd7f700 1 -- 172.21.15.40:0/3609120486 wait complete. 2024-09-13T17:50:45.608 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-13T17:50:45.608 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:50:45.608 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-09-13T17:50:45.608 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-13T17:50:45.608 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:50:45.608 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-13T17:50:45.608 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:50:45.608 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:50:45.608 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-13T17:50:45.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:50:45.609 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-09-13T17:50:45.616 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:50:45.616 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:50:45.616 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:50:45.616 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:50:45.616 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-09-13T17:50:45.617 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:50:45.617 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:50:45.617 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:50:45.617 INFO:teuthology.orchestra.run.smithi040.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:50:45.617 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-09-13T17:50:45.617 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T17:50:45.617 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42549sec preferred_lft 42549sec 2024-09-13T17:50:45.617 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link 2024-09-13T17:50:45.617 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:50:45.617 INFO:teuthology.orchestra.run.smithi040.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:50:45.617 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:50:45.617 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:50:45.617 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-09-13T17:50:45.618 INFO:teuthology.orchestra.run.smithi040.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T17:50:45.618 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 02:42:8a:21:55:c6 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:50:45.618 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T17:50:45.618 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:50:45.618 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T17:50:45.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:50:45.619 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-09-13T17:50:45.619 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-09-13T17:50:45.619 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add name ceph-brx type bridge 2024-09-13T17:50:45.619 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr flush dev ceph-brx 2024-09-13T17:50:45.619 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set ceph-brx up 2024-09-13T17:50:45.619 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T17:50:45.619 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-09-13T17:50:45.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:50:45.659 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:50:45.669 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-09-13T17:50:45.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:50:45.671 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-09-13T17:50:45.697 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-09-13T17:50:45.698 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:50:45.698 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T17:50:45.698 INFO:teuthology.orchestra.run.smithi040.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T17:50:45.698 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T17:50:45.698 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:50:45.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:50:45.698 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-09-13T17:50:45.699 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-09-13T17:50:45.699 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T17:50:45.699 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T17:50:45.699 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T17:50:45.699 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-09-13T17:50:45.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:50:45.740 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-09-13T17:50:45.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:50:45.790 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2024-09-13T17:50:45.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:50:45.796 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-09-13T17:50:45.796 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-09-13T17:50:45.796 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-13T17:50:45.796 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-13T17:50:45.796 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-09-13T17:50:45.827 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-13T17:50:45.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:50:45.828 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-09-13T17:50:45.828 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-09-13T17:50:45.828 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-13T17:50:45.828 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T17:50:45.828 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-13T17:50:45.828 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-13T17:50:45.828 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-13T17:50:45.828 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-09-13T17:50:45.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:50:45.895 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-09-13T17:50:45.895 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-09-13T17:50:45.895 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-09-13T17:50:45.895 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T17:50:45.895 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-09-13T17:50:45.921 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-13T17:50:45.921 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:50:45.922 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:50:45.969 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-13T17:50:45.969 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:50:45.969 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:50:45.976 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2024-09-13T17:50:45.989 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:50:45.989 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:50:46.046 INFO:teuthology.orchestra.run.smithi040.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:50:46.047 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:50:46.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:50:46.048 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2024-09-13T17:50:46.055 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T17:50:46.055 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-13T17:50:46.057 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2024-09-13T17:50:46.074 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:50:46.074 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:50:46.134 INFO:teuthology.orchestra.run.smithi040.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:50:46.135 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:50:46.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:50:46.136 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2024-09-13T17:50:46.146 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi040.stderr:2024-09-13T17:50:46.137+0000 7f0642cd5640 -1 init, newargv = 0x563ab77b9640 newargc=15 2024-09-13T17:50:46.146 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi040.stderr:ceph-fuse[33014]: starting ceph client 2024-09-13T17:50:46.156 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi040.stderr:ceph-fuse[33014]: starting fuse 2024-09-13T17:50:47.242 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2024-09-13T17:50:47.259 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:50:47.259 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:50:47.318 INFO:teuthology.orchestra.run.smithi040.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:50:47.318 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:50:47.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:50:47.319 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2024-09-13T17:50:47.326 INFO:teuthology.orchestra.run.smithi040.stdout:52 2024-09-13T17:50:47.326 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-09-13T17:50:47.326 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:50:47.327 DEBUG:teuthology.orchestra.run.smithi040:> import glob 2024-09-13T17:50:47.327 DEBUG:teuthology.orchestra.run.smithi040:> import re 2024-09-13T17:50:47.327 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-09-13T17:50:47.327 DEBUG:teuthology.orchestra.run.smithi040:> import subprocess 2024-09-13T17:50:47.327 DEBUG:teuthology.orchestra.run.smithi040:> 2024-09-13T17:50:47.327 DEBUG:teuthology.orchestra.run.smithi040:> def _find_admin_socket(client_name): 2024-09-13T17:50:47.327 DEBUG:teuthology.orchestra.run.smithi040:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-13T17:50:47.327 DEBUG:teuthology.orchestra.run.smithi040:> files = glob.glob(asok_path) 2024-09-13T17:50:47.327 DEBUG:teuthology.orchestra.run.smithi040:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-13T17:50:47.327 DEBUG:teuthology.orchestra.run.smithi040:> 2024-09-13T17:50:47.327 DEBUG:teuthology.orchestra.run.smithi040:> # Given a non-glob path, it better be there 2024-09-13T17:50:47.327 DEBUG:teuthology.orchestra.run.smithi040:> if "*" not in asok_path: 2024-09-13T17:50:47.327 DEBUG:teuthology.orchestra.run.smithi040:> assert(len(files) == 1) 2024-09-13T17:50:47.327 DEBUG:teuthology.orchestra.run.smithi040:> return files[0] 2024-09-13T17:50:47.328 DEBUG:teuthology.orchestra.run.smithi040:> 2024-09-13T17:50:47.328 DEBUG:teuthology.orchestra.run.smithi040:> for f in files: 2024-09-13T17:50:47.328 DEBUG:teuthology.orchestra.run.smithi040:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:50:47.328 DEBUG:teuthology.orchestra.run.smithi040:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:50:47.328 DEBUG:teuthology.orchestra.run.smithi040:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:50:47.328 DEBUG:teuthology.orchestra.run.smithi040:> contents = proc_f.read() 2024-09-13T17:50:47.328 DEBUG:teuthology.orchestra.run.smithi040:> if mountpoint in contents: 2024-09-13T17:50:47.328 DEBUG:teuthology.orchestra.run.smithi040:> return f 2024-09-13T17:50:47.328 DEBUG:teuthology.orchestra.run.smithi040:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:50:47.328 DEBUG:teuthology.orchestra.run.smithi040:> 2024-09-13T17:50:47.328 DEBUG:teuthology.orchestra.run.smithi040:> print(_find_admin_socket("client.admin")) 2024-09-13T17:50:47.328 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-09-13T17:50:47.408 INFO:teuthology.orchestra.run.smithi040.stdout:/var/run/ceph/ceph-client.admin.33014.asok 2024-09-13T17:50:47.593 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.33014.asok 2024-09-13T17:50:47.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:50:47.594 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.33014.asok status 2024-09-13T17:50:47.737 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:50:47.737 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": { 2024-09-13T17:50:47.737 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:50:47.737 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:50:47.737 INFO:teuthology.orchestra.run.smithi040.stdout: "entity_id": "admin", 2024-09-13T17:50:47.738 INFO:teuthology.orchestra.run.smithi040.stdout: "hostname": "smithi040", 2024-09-13T17:50:47.738 INFO:teuthology.orchestra.run.smithi040.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-13T17:50:47.738 INFO:teuthology.orchestra.run.smithi040.stdout: "pid": "33014", 2024-09-13T17:50:47.738 INFO:teuthology.orchestra.run.smithi040.stdout: "root": "/" 2024-09-13T17:50:47.738 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:50:47.738 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_count": 0, 2024-09-13T17:50:47.738 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_pinned_count": 0, 2024-09-13T17:50:47.738 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4464, 2024-09-13T17:50:47.738 INFO:teuthology.orchestra.run.smithi040.stdout: "inst": { 2024-09-13T17:50:47.738 INFO:teuthology.orchestra.run.smithi040.stdout: "name": { 2024-09-13T17:50:47.738 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-09-13T17:50:47.738 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4464 2024-09-13T17:50:47.738 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:50:47.739 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2024-09-13T17:50:47.739 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:50:47.739 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:50:47.739 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1162054516 2024-09-13T17:50:47.739 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:50:47.739 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:50:47.739 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2024-09-13T17:50:47.739 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:50:47.739 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:50:47.739 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1162054516 2024-09-13T17:50:47.739 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:50:47.739 INFO:teuthology.orchestra.run.smithi040.stdout: "inst_str": "client.4464 192.168.0.1:0/1162054516", 2024-09-13T17:50:47.740 INFO:teuthology.orchestra.run.smithi040.stdout: "addr_str": "192.168.0.1:0/1162054516", 2024-09-13T17:50:47.740 INFO:teuthology.orchestra.run.smithi040.stdout: "inode_count": 1, 2024-09-13T17:50:47.740 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_epoch": 6, 2024-09-13T17:50:47.740 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch": 25, 2024-09-13T17:50:47.740 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:50:47.740 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklisted": false, 2024-09-13T17:50:47.740 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs" 2024-09-13T17:50:47.740 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:50:47.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:50:47.747 DEBUG:teuthology.orchestra.run.smithi040:> stat --file-system '--printf=%T 2024-09-13T17:50:47.747 DEBUG:teuthology.orchestra.run.smithi040:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:50:47.752 INFO:teuthology.orchestra.run.smithi040.stdout:fuseblk 2024-09-13T17:50:47.752 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:50:47.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:50:47.753 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:50:47.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:50:47.769 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-13T17:50:47.817 INFO:teuthology.orchestra.run.smithi040.stdout:# file: . 2024-09-13T17:50:47.817 INFO:teuthology.orchestra.run.smithi040.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-13T17:50:47.817 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:50:47.818 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:50:47.818 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:50:47.825 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:50:47.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.977+0000 7f0d7eca9700 1 Processor -- start 2024-09-13T17:50:47.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.977+0000 7f0d7eca9700 1 -- start start 2024-09-13T17:50:47.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7eca9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0d78071f70 0x7f0d780c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:47.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7eca9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0d780c7200 0x7f0d78161ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:47.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7eca9700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d780c55c0 con 0x7f0d78072350 2024-09-13T17:50:47.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7eca9700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d7806a390 con 0x7f0d78071f70 2024-09-13T17:50:47.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7eca9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d78157710 con 0x7f0d780c7200 2024-09-13T17:50:47.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7d4a6700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0d78072350 0x7f0d78150d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32872/0 (socket says 172.21.15.40:32872) 2024-09-13T17:50:47.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7d4a6700 1 -- 172.21.15.40:0/6402975 learned_addr learned my addr 172.21.15.40:0/6402975 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:47.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7dca7700 1 --2- 172.21.15.40:0/6402975 >> v2:172.21.15.40:3300/0 conn(0x7f0d78071f70 0x7f0d780c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:47.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7e4a8700 1 --2- 172.21.15.40:0/6402975 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0d780c7200 0x7f0d78161ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:47.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7cca5700 1 -- 172.21.15.40:0/6402975 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3976962101 0 0) 0x7f0d780c55c0 con 0x7f0d78072350 2024-09-13T17:50:47.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7cca5700 1 -- 172.21.15.40:0/6402975 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d640035d0 con 0x7f0d78072350 2024-09-13T17:50:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7dca7700 1 -- 172.21.15.40:0/6402975 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0d780c7200 msgr2=0x7f0d78161ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7dca7700 1 --2- 172.21.15.40:0/6402975 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0d780c7200 0x7f0d78161ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7dca7700 1 -- 172.21.15.40:0/6402975 >> v1:172.21.15.40:6789/0 conn(0x7f0d78072350 legacy=0x7f0d78150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7dca7700 1 -- 172.21.15.40:0/6402975 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d781579d0 con 0x7f0d78071f70 2024-09-13T17:50:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7cca5700 1 -- 172.21.15.40:0/6402975 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1812648525 0 0) 0x7f0d640035d0 con 0x7f0d78072350 2024-09-13T17:50:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7dca7700 1 --2- 172.21.15.40:0/6402975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d78071f70 0x7f0d780c5d30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c009850 tx=0x7f0d6c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed2541cc9d9751 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7cca5700 1 -- 172.21.15.40:0/6402975 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d74001670 con 0x7f0d78071f70 2024-09-13T17:50:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7cca5700 1 -- 172.21.15.40:0/6402975 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6c004030 con 0x7f0d78071f70 2024-09-13T17:50:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7cca5700 1 -- 172.21.15.40:0/6402975 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c0043e0 con 0x7f0d78071f70 2024-09-13T17:50:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7eca9700 1 -- 172.21.15.40:0/6402975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d78071f70 msgr2=0x7f0d780c5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7eca9700 1 --2- 172.21.15.40:0/6402975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d78071f70 0x7f0d780c5d30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c009850 tx=0x7f0d6c009b60 comp rx=0 tx=0).stop 2024-09-13T17:50:47.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7eca9700 1 -- 172.21.15.40:0/6402975 shutdown_connections 2024-09-13T17:50:47.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7eca9700 1 --2- 172.21.15.40:0/6402975 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0d780c7200 0x7f0d78161ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:47.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7eca9700 1 --2- 172.21.15.40:0/6402975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d78071f70 0x7f0d780c5d30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:47.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7eca9700 1 -- 172.21.15.40:0/6402975 >> 172.21.15.40:0/6402975 conn(0x7f0d780cabc0 msgr2=0x7f0d780cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:47.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7eca9700 1 -- 172.21.15.40:0/6402975 shutdown_connections 2024-09-13T17:50:47.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.981+0000 7f0d7eca9700 1 -- 172.21.15.40:0/6402975 wait complete. 2024-09-13T17:50:47.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7eca9700 1 Processor -- start 2024-09-13T17:50:47.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7eca9700 1 -- start start 2024-09-13T17:50:47.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7eca9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0d78071f70 0x7f0d78159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:47.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7eca9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d78072350 0x7f0d78156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:47.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7eca9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d780c7200 0x7f0d78070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:47.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7eca9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d78158a80 con 0x7f0d78072350 2024-09-13T17:50:47.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7eca9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d78159e00 con 0x7f0d780c7200 2024-09-13T17:50:47.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7eca9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7815af20 con 0x7f0d78071f70 2024-09-13T17:50:47.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7d4a6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d78072350 0x7f0d78156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:47.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7d4a6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d78072350 0x7f0d78156e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52582/0 (socket says 172.21.15.40:52582) 2024-09-13T17:50:47.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7d4a6700 1 -- 172.21.15.40:0/3026678994 learned_addr learned my addr 172.21.15.40:0/3026678994 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:47.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7dca7700 1 --2- 172.21.15.40:0/3026678994 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0d78071f70 0x7f0d78159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:47.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7e4a8700 1 --2- 172.21.15.40:0/3026678994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d780c7200 0x7f0d78070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:47.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7d4a6700 1 -- 172.21.15.40:0/3026678994 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0d78071f70 msgr2=0x7f0d78159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:47.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7d4a6700 1 --2- 172.21.15.40:0/3026678994 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0d78071f70 0x7f0d78159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:47.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7d4a6700 1 -- 172.21.15.40:0/3026678994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d780c7200 msgr2=0x7f0d78070cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:47.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7d4a6700 1 --2- 172.21.15.40:0/3026678994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d780c7200 0x7f0d78070cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:47.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7d4a6700 1 -- 172.21.15.40:0/3026678994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6c0094b0 con 0x7f0d78072350 2024-09-13T17:50:47.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7d4a6700 1 --2- 172.21.15.40:0/3026678994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d78072350 0x7f0d78156e10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0d740011e0 tx=0x7f0d7400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:47.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d6a7fc700 1 -- 172.21.15.40:0/3026678994 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d74004ca0 con 0x7f0d78072350 2024-09-13T17:50:47.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d6a7fc700 1 -- 172.21.15.40:0/3026678994 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d74004e20 con 0x7f0d78072350 2024-09-13T17:50:47.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7eca9700 1 -- 172.21.15.40:0/3026678994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7815b170 con 0x7f0d78072350 2024-09-13T17:50:47.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d6a7fc700 1 -- 172.21.15.40:0/3026678994 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d74007310 con 0x7f0d78072350 2024-09-13T17:50:47.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.985+0000 7f0d7eca9700 1 -- 172.21.15.40:0/3026678994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d781586a0 con 0x7f0d78072350 2024-09-13T17:50:47.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.989+0000 7f0d7eca9700 1 -- 172.21.15.40:0/3026678994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d78157f70 con 0x7f0d78072350 2024-09-13T17:50:47.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.989+0000 7f0d6a7fc700 1 -- 172.21.15.40:0/3026678994 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0d74007470 con 0x7f0d78072350 2024-09-13T17:50:47.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.989+0000 7f0d6a7fc700 1 --2- 172.21.15.40:0/3026678994 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0d5c066730 0x7f0d5c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:47.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.989+0000 7f0d7dca7700 1 --2- 172.21.15.40:0/3026678994 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0d5c066730 0x7f0d5c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:47.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.989+0000 7f0d6a7fc700 1 -- 172.21.15.40:0/3026678994 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0d740840b0 con 0x7f0d78072350 2024-09-13T17:50:47.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.989+0000 7f0d7dca7700 1 --2- 172.21.15.40:0/3026678994 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0d5c066730 0x7f0d5c068bf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0090b0 tx=0x7f0d6c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:48.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:47.997+0000 7f0d6a7fc700 1 -- 172.21.15.40:0/3026678994 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0d78157f70 con 0x7f0d78072350 2024-09-13T17:50:48.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.173+0000 7f0d7eca9700 1 -- 172.21.15.40:0/3026678994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d78157f70 con 0x7f0d78072350 2024-09-13T17:50:48.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.177+0000 7f0d6a7fc700 1 -- 172.21.15.40:0/3026678994 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18262 (secure 0 0 0) 0x7f0d78157f70 con 0x7f0d78072350 2024-09-13T17:50:48.183 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:50:48.183 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:50:38.974071+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:50:32.740336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:50:33.552914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:50:36.623639+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:50:48.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.177+0000 7f0d7eca9700 1 -- 172.21.15.40:0/3026678994 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0d5c066730 msgr2=0x7f0d5c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:48.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.177+0000 7f0d7eca9700 1 --2- 172.21.15.40:0/3026678994 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0d5c066730 0x7f0d5c068bf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0090b0 tx=0x7f0d6c008030 comp rx=0 tx=0).stop 2024-09-13T17:50:48.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.177+0000 7f0d7eca9700 1 -- 172.21.15.40:0/3026678994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d78072350 msgr2=0x7f0d78156e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:48.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.177+0000 7f0d7eca9700 1 --2- 172.21.15.40:0/3026678994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d78072350 0x7f0d78156e10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0d740011e0 tx=0x7f0d7400c9e0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.177+0000 7f0d7eca9700 1 -- 172.21.15.40:0/3026678994 shutdown_connections 2024-09-13T17:50:48.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.177+0000 7f0d7eca9700 1 --2- 172.21.15.40:0/3026678994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d780c7200 0x7f0d78070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.177+0000 7f0d7eca9700 1 --2- 172.21.15.40:0/3026678994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d78072350 0x7f0d78156e10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.177+0000 7f0d7eca9700 1 --2- 172.21.15.40:0/3026678994 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0d5c066730 0x7f0d5c068bf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.177+0000 7f0d7eca9700 1 --2- 172.21.15.40:0/3026678994 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0d78071f70 0x7f0d78159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.177+0000 7f0d7eca9700 1 -- 172.21.15.40:0/3026678994 >> 172.21.15.40:0/3026678994 conn(0x7f0d780cabc0 msgr2=0x7f0d78150b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:48.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.177+0000 7f0d7eca9700 1 -- 172.21.15.40:0/3026678994 shutdown_connections 2024-09-13T17:50:48.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.177+0000 7f0d7eca9700 1 -- 172.21.15.40:0/3026678994 wait complete. 2024-09-13T17:50:48.198 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:50:21.057171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:50:32.740336+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:50:33.552914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:50:36.623639+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:50:48.198 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:50:48.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.313+0000 7ff49a0cd700 1 Processor -- start 2024-09-13T17:50:48.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.313+0000 7ff49a0cd700 1 -- start start 2024-09-13T17:50:48.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff49a0cd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4940c8430 0x7ff4941562e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:48.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff49a0cd700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff494159f40 con 0x7ff494070a60 2024-09-13T17:50:48.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff49a0cd700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4940c6ee0 con 0x7ff4940c8050 2024-09-13T17:50:48.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff49a0cd700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff494156820 con 0x7ff4940c8430 2024-09-13T17:50:48.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff4998cc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff494070a60 0x7ff494150160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32878/0 (socket says 172.21.15.40:32878) 2024-09-13T17:50:48.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff4998cc700 1 -- 172.21.15.40:0/1872366152 learned_addr learned my addr 172.21.15.40:0/1872366152 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:48.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff4988ca700 1 --2- 172.21.15.40:0/1872366152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4940c8430 0x7ff4941562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:48.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff493fff700 1 -- 172.21.15.40:0/1872366152 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 278762811 0 0) 0x7ff494159f40 con 0x7ff494070a60 2024-09-13T17:50:48.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff493fff700 1 -- 172.21.15.40:0/1872366152 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4800035d0 con 0x7ff494070a60 2024-09-13T17:50:48.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff493fff700 1 -- 172.21.15.40:0/1872366152 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1016871791 0 0) 0x7ff4800035d0 con 0x7ff494070a60 2024-09-13T17:50:48.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff493fff700 1 -- 172.21.15.40:0/1872366152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4940c8430 msgr2=0x7ff4941562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:48.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff493fff700 1 --2- 172.21.15.40:0/1872366152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4940c8430 0x7ff4941562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff493fff700 1 -- 172.21.15.40:0/1872366152 >> v1:172.21.15.43:6789/0 conn(0x7ff4940c8050 legacy=0x7ff4940c66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:48.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff493fff700 1 -- 172.21.15.40:0/1872366152 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4940c71c0 con 0x7ff494070a60 2024-09-13T17:50:48.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff493fff700 1 -- 172.21.15.40:0/1872366152 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7ff48c0019f0 con 0x7ff494070a60 2024-09-13T17:50:48.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff493fff700 1 -- 172.21.15.40:0/1872366152 >> v1:172.21.15.40:6789/0 conn(0x7ff494070a60 legacy=0x7ff494150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:48.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff493fff700 1 --2- 172.21.15.40:0/1872366152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff480003a50 0x7ff480003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:48.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff493fff700 1 --2- 172.21.15.40:0/1872366152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff480004870 0x7ff480004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:48.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff493fff700 1 --2- 172.21.15.40:0/1872366152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4940c8430 0x7ff480005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:48.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff493fff700 1 -- 172.21.15.40:0/1872366152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff494156820 con 0x7ff480004870 2024-09-13T17:50:48.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff493fff700 1 -- 172.21.15.40:0/1872366152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48000bae0 con 0x7ff480003a50 2024-09-13T17:50:48.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff493fff700 1 -- 172.21.15.40:0/1872366152 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48000bca0 con 0x7ff4940c8430 2024-09-13T17:50:48.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff4988ca700 1 --2- 172.21.15.40:0/1872366152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff480004870 0x7ff480004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:48.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff4990cb700 1 --2- 172.21.15.40:0/1872366152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff480003a50 0x7ff480003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:48.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff4988ca700 1 -- 172.21.15.40:0/1872366152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4940c8430 msgr2=0x7ff480005230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T17:50:48.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff4988ca700 1 --2- 172.21.15.40:0/1872366152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4940c8430 0x7ff480005230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff4988ca700 1 -- 172.21.15.40:0/1872366152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff480003a50 msgr2=0x7ff480003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:48.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff4988ca700 1 --2- 172.21.15.40:0/1872366152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff480003a50 0x7ff480003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff4988ca700 1 -- 172.21.15.40:0/1872366152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4800035d0 con 0x7ff480004870 2024-09-13T17:50:48.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff4990cb700 1 --2- 172.21.15.40:0/1872366152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff480003a50 0x7ff480003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:50:48.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff4988ca700 1 --2- 172.21.15.40:0/1872366152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff480004870 0x7ff480004cf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff4940c6240 tx=0x7ff4800058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc6ab437911c051 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:48.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff493fff700 1 -- 172.21.15.40:0/1872366152 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48400e6c0 con 0x7ff480004870 2024-09-13T17:50:48.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff493fff700 1 -- 172.21.15.40:0/1872366152 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48400e820 con 0x7ff480004870 2024-09-13T17:50:48.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff493fff700 1 -- 172.21.15.40:0/1872366152 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48400eb90 con 0x7ff480004870 2024-09-13T17:50:48.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff49a0cd700 1 -- 172.21.15.40:0/1872366152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff480004870 msgr2=0x7ff480004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:48.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff49a0cd700 1 --2- 172.21.15.40:0/1872366152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff480004870 0x7ff480004cf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff4940c6240 tx=0x7ff4800058a0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff4998cc700 1 -- 172.21.15.40:0/1872366152 reap_dead start 2024-09-13T17:50:48.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff49a0cd700 1 -- 172.21.15.40:0/1872366152 shutdown_connections 2024-09-13T17:50:48.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff49a0cd700 1 -- 172.21.15.40:0/1872366152 >> 172.21.15.40:0/1872366152 conn(0x7ff494066090 msgr2=0x7ff4940664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:48.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff49a0cd700 1 -- 172.21.15.40:0/1872366152 shutdown_connections 2024-09-13T17:50:48.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.317+0000 7ff49a0cd700 1 -- 172.21.15.40:0/1872366152 wait complete. 2024-09-13T17:50:48.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff49a0cd700 1 Processor -- start 2024-09-13T17:50:48.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff49a0cd700 1 -- start start 2024-09-13T17:50:48.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff49a0cd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4941ee5c0 0x7ff4941ee9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:48.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff49a0cd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff49414c2a0 0x7ff4941e1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:48.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff49a0cd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff494145910 0x7ff494145dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:48.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff49a0cd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49414bdc0 con 0x7ff4941ee5c0 2024-09-13T17:50:48.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff49a0cd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4941eeee0 con 0x7ff49414c2a0 2024-09-13T17:50:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff49a0cd700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4941ef0c0 con 0x7ff494145910 2024-09-13T17:50:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff4990cb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4941ee5c0 0x7ff4941ee9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff4990cb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4941ee5c0 0x7ff4941ee9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52606/0 (socket says 172.21.15.40:52606) 2024-09-13T17:50:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff4990cb700 1 -- 172.21.15.40:0/1205291379 learned_addr learned my addr 172.21.15.40:0/1205291379 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff4988ca700 1 --2- 172.21.15.40:0/1205291379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff49414c2a0 0x7ff4941e1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff4998cc700 1 --2- 172.21.15.40:0/1205291379 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff494145910 0x7ff494145dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff4990cb700 1 -- 172.21.15.40:0/1205291379 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff494145910 msgr2=0x7ff494145dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff4990cb700 1 --2- 172.21.15.40:0/1205291379 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff494145910 0x7ff494145dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff4990cb700 1 -- 172.21.15.40:0/1205291379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff49414c2a0 msgr2=0x7ff4941e1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff4990cb700 1 --2- 172.21.15.40:0/1205291379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff49414c2a0 0x7ff4941e1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff4990cb700 1 -- 172.21.15.40:0/1205291379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49414b520 con 0x7ff4941ee5c0 2024-09-13T17:50:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff4990cb700 1 --2- 172.21.15.40:0/1205291379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4941ee5c0 0x7ff4941ee9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0018b0 tx=0x7ff47c001bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:48.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff491ffb700 1 -- 172.21.15.40:0/1205291379 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c0054e0 con 0x7ff4941ee5c0 2024-09-13T17:50:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff491ffb700 1 -- 172.21.15.40:0/1205291379 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c001d50 con 0x7ff4941ee5c0 2024-09-13T17:50:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff49a0cd700 1 -- 172.21.15.40:0/1205291379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49414b800 con 0x7ff4941ee5c0 2024-09-13T17:50:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff491ffb700 1 -- 172.21.15.40:0/1205291379 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c0038c0 con 0x7ff4941ee5c0 2024-09-13T17:50:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.321+0000 7ff49a0cd700 1 -- 172.21.15.40:0/1205291379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4941e1a60 con 0x7ff4941ee5c0 2024-09-13T17:50:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.325+0000 7ff49a0cd700 1 -- 172.21.15.40:0/1205291379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff494072580 con 0x7ff4941ee5c0 2024-09-13T17:50:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.325+0000 7ff491ffb700 1 -- 172.21.15.40:0/1205291379 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff47c003a20 con 0x7ff4941ee5c0 2024-09-13T17:50:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.325+0000 7ff491ffb700 1 --2- 172.21.15.40:0/1205291379 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff474066730 0x7ff474068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.325+0000 7ff491ffb700 1 -- 172.21.15.40:0/1205291379 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff47c087c30 con 0x7ff4941ee5c0 2024-09-13T17:50:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.325+0000 7ff4988ca700 1 --2- 172.21.15.40:0/1205291379 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff474066730 0x7ff474068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:48.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.325+0000 7ff4988ca700 1 --2- 172.21.15.40:0/1205291379 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff474066730 0x7ff474068bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff4840090b0 tx=0x7ff48401f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:48.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.333+0000 7ff491ffb700 1 -- 172.21.15.40:0/1205291379 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff47c054cd0 con 0x7ff4941ee5c0 2024-09-13T17:50:48.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.505+0000 7ff49a0cd700 1 -- 172.21.15.40:0/1205291379 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff4941df8f0 con 0x7ff4941ee5c0 2024-09-13T17:50:48.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.509+0000 7ff491ffb700 1 -- 172.21.15.40:0/1205291379 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff47c01c070 con 0x7ff4941ee5c0 2024-09-13T17:50:48.514 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-09-13T17:50:48.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.509+0000 7ff49a0cd700 1 -- 172.21.15.40:0/1205291379 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff474066730 msgr2=0x7ff474068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:48.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.509+0000 7ff49a0cd700 1 --2- 172.21.15.40:0/1205291379 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff474066730 0x7ff474068bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff4840090b0 tx=0x7ff48401f040 comp rx=0 tx=0).stop 2024-09-13T17:50:48.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.509+0000 7ff49a0cd700 1 -- 172.21.15.40:0/1205291379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4941ee5c0 msgr2=0x7ff4941ee9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:48.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.509+0000 7ff49a0cd700 1 --2- 172.21.15.40:0/1205291379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4941ee5c0 0x7ff4941ee9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0018b0 tx=0x7ff47c001bc0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.509+0000 7ff49a0cd700 1 -- 172.21.15.40:0/1205291379 shutdown_connections 2024-09-13T17:50:48.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.509+0000 7ff49a0cd700 1 --2- 172.21.15.40:0/1205291379 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff474066730 0x7ff474068bf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.509+0000 7ff49a0cd700 1 --2- 172.21.15.40:0/1205291379 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff494145910 0x7ff494145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.509+0000 7ff49a0cd700 1 --2- 172.21.15.40:0/1205291379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff49414c2a0 0x7ff4941e1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.509+0000 7ff49a0cd700 1 --2- 172.21.15.40:0/1205291379 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4941ee5c0 0x7ff4941ee9a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.509+0000 7ff49a0cd700 1 -- 172.21.15.40:0/1205291379 >> 172.21.15.40:0/1205291379 conn(0x7ff494066090 msgr2=0x7ff494156990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:48.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.509+0000 7ff49a0cd700 1 -- 172.21.15.40:0/1205291379 shutdown_connections 2024-09-13T17:50:48.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.509+0000 7ff49a0cd700 1 -- 172.21.15.40:0/1205291379 wait complete. 2024-09-13T17:50:48.529 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:50:48.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.645+0000 7fa4fe0ab700 1 Processor -- start 2024-09-13T17:50:48.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.645+0000 7fa4fe0ab700 1 -- start start 2024-09-13T17:50:48.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.645+0000 7fa4fe0ab700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4f80c7fc0 0x7fa4f80c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:48.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.645+0000 7fa4fe0ab700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4f80c6d20 0x7fa4f8156880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:48.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.645+0000 7fa4fe0ab700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f815a6d0 con 0x7fa4f80c83a0 2024-09-13T17:50:48.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.645+0000 7fa4fe0ab700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f80c2570 con 0x7fa4f80c6d20 2024-09-13T17:50:48.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.645+0000 7fa4fe0ab700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f80c12f0 con 0x7fa4f80c7fc0 2024-09-13T17:50:48.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.645+0000 7fa4fd0a9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4f80c7fc0 0x7fa4f80c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:48.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.645+0000 7fa4fc8a8700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4f80c6d20 0x7fa4f8156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:48.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.649+0000 7fa4fd0a9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4f80c7fc0 0x7fa4f80c67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:53508/0 (socket says 172.21.15.40:53508) 2024-09-13T17:50:48.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.649+0000 7fa4fd8aa700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa4f80c83a0 0x7fa4f815cce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:53138/0 (socket says 172.21.15.40:53138) 2024-09-13T17:50:48.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.649+0000 7fa4fd0a9700 1 -- 172.21.15.40:0/3469970418 learned_addr learned my addr 172.21.15.40:0/3469970418 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:48.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.649+0000 7fa4fc8a8700 1 -- 172.21.15.40:0/3469970418 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4f80c7fc0 msgr2=0x7fa4f80c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:48.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.649+0000 7fa4fc8a8700 1 --2- 172.21.15.40:0/3469970418 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4f80c7fc0 0x7fa4f80c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.649+0000 7fa4fc8a8700 1 -- 172.21.15.40:0/3469970418 >> v1:172.21.15.43:6789/0 conn(0x7fa4f80c83a0 legacy=0x7fa4f815cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:48.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.649+0000 7fa4fc8a8700 1 -- 172.21.15.40:0/3469970418 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f8159910 con 0x7fa4f80c6d20 2024-09-13T17:50:48.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.649+0000 7fa4fc8a8700 1 --2- 172.21.15.40:0/3469970418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f80c6d20 0x7fa4f8156880 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009850 tx=0x7fa4f4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=906970e5b3427283 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:48.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.649+0000 7fa4effff700 1 -- 172.21.15.40:0/3469970418 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4011830 con 0x7fa4f80c6d20 2024-09-13T17:50:48.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.649+0000 7fa4effff700 1 -- 172.21.15.40:0/3469970418 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f4011990 con 0x7fa4f80c6d20 2024-09-13T17:50:48.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.649+0000 7fa4effff700 1 -- 172.21.15.40:0/3469970418 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4011d40 con 0x7fa4f80c6d20 2024-09-13T17:50:48.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.649+0000 7fa4fe0ab700 1 -- 172.21.15.40:0/3469970418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f80c6d20 msgr2=0x7fa4f8156880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:48.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.649+0000 7fa4fe0ab700 1 --2- 172.21.15.40:0/3469970418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f80c6d20 0x7fa4f8156880 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4009850 tx=0x7fa4f4009b60 comp rx=0 tx=0).stop 2024-09-13T17:50:48.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.649+0000 7fa4fe0ab700 1 -- 172.21.15.40:0/3469970418 shutdown_connections 2024-09-13T17:50:48.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.649+0000 7fa4fe0ab700 1 --2- 172.21.15.40:0/3469970418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f80c6d20 0x7fa4f8156880 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.649+0000 7fa4fe0ab700 1 --2- 172.21.15.40:0/3469970418 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4f80c7fc0 0x7fa4f80c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.649+0000 7fa4fe0ab700 1 -- 172.21.15.40:0/3469970418 >> 172.21.15.40:0/3469970418 conn(0x7fa4f80cc7a0 msgr2=0x7fa4f80c88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:48.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.649+0000 7fa4fe0ab700 1 -- 172.21.15.40:0/3469970418 shutdown_connections 2024-09-13T17:50:48.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.649+0000 7fa4fe0ab700 1 -- 172.21.15.40:0/3469970418 wait complete. 2024-09-13T17:50:48.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.653+0000 7fa4fe0ab700 1 Processor -- start 2024-09-13T17:50:48.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.653+0000 7fa4fe0ab700 1 -- start start 2024-09-13T17:50:48.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.653+0000 7fa4fe0ab700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4f80c6d20 0x7fa4f81f3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:48.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.653+0000 7fa4fe0ab700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f80c7fc0 0x7fa4f8158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:48.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.653+0000 7fa4fe0ab700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4f80c83a0 0x7fa4f8151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:48.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.653+0000 7fa4fe0ab700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f81f3960 con 0x7fa4f80c6d20 2024-09-13T17:50:48.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.653+0000 7fa4fe0ab700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f8153a20 con 0x7fa4f80c7fc0 2024-09-13T17:50:48.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.653+0000 7fa4fe0ab700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f81f0420 con 0x7fa4f80c83a0 2024-09-13T17:50:48.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.653+0000 7fa4fd0a9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4f80c6d20 0x7fa4f81f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:48.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.653+0000 7fa4fd0a9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4f80c6d20 0x7fa4f81f3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52616/0 (socket says 172.21.15.40:52616) 2024-09-13T17:50:48.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.653+0000 7fa4fd0a9700 1 -- 172.21.15.40:0/834223492 learned_addr learned my addr 172.21.15.40:0/834223492 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:48.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.653+0000 7fa4fc8a8700 1 --2- 172.21.15.40:0/834223492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f80c7fc0 0x7fa4f8158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:48.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.653+0000 7fa4fd8aa700 1 --2- 172.21.15.40:0/834223492 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4f80c83a0 0x7fa4f8151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:48.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.657+0000 7fa4fc8a8700 1 -- 172.21.15.40:0/834223492 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4f80c83a0 msgr2=0x7fa4f8151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:48.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.657+0000 7fa4fc8a8700 1 --2- 172.21.15.40:0/834223492 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4f80c83a0 0x7fa4f8151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.657+0000 7fa4fc8a8700 1 -- 172.21.15.40:0/834223492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4f80c6d20 msgr2=0x7fa4f81f3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:48.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.657+0000 7fa4fc8a8700 1 --2- 172.21.15.40:0/834223492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4f80c6d20 0x7fa4f81f3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.657+0000 7fa4fc8a8700 1 -- 172.21.15.40:0/834223492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f40094b0 con 0x7fa4f80c7fc0 2024-09-13T17:50:48.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.657+0000 7fa4fc8a8700 1 --2- 172.21.15.40:0/834223492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f80c7fc0 0x7fa4f8158640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4016000 tx=0x7fa4f4004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:48.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.657+0000 7fa4edffb700 1 -- 172.21.15.40:0/834223492 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f40036c0 con 0x7fa4f80c7fc0 2024-09-13T17:50:48.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.657+0000 7fa4edffb700 1 -- 172.21.15.40:0/834223492 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f401f070 con 0x7fa4f80c7fc0 2024-09-13T17:50:48.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.657+0000 7fa4fe0ab700 1 -- 172.21.15.40:0/834223492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f81ef8a0 con 0x7fa4f80c7fc0 2024-09-13T17:50:48.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.657+0000 7fa4edffb700 1 -- 172.21.15.40:0/834223492 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4003a60 con 0x7fa4f80c7fc0 2024-09-13T17:50:48.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.657+0000 7fa4fe0ab700 1 -- 172.21.15.40:0/834223492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f81efe60 con 0x7fa4f80c7fc0 2024-09-13T17:50:48.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.657+0000 7fa4fe0ab700 1 -- 172.21.15.40:0/834223492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f80b1320 con 0x7fa4f80c7fc0 2024-09-13T17:50:48.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.661+0000 7fa4edffb700 1 -- 172.21.15.40:0/834223492 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa4f4003bc0 con 0x7fa4f80c7fc0 2024-09-13T17:50:48.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.661+0000 7fa4edffb700 1 --2- 172.21.15.40:0/834223492 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4dc066730 0x7fa4dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:48.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.661+0000 7fa4edffb700 1 -- 172.21.15.40:0/834223492 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa4f4021070 con 0x7fa4f80c7fc0 2024-09-13T17:50:48.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.661+0000 7fa4fd0a9700 1 --2- 172.21.15.40:0/834223492 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4dc066730 0x7fa4dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:48.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.661+0000 7fa4fd0a9700 1 --2- 172.21.15.40:0/834223492 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4dc066730 0x7fa4dc068bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8006fd0 tx=0x7fa4e80092e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:48.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.673+0000 7fa4edffb700 1 -- 172.21.15.40:0/834223492 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa4f4052930 con 0x7fa4f80c7fc0 2024-09-13T17:50:48.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.845+0000 7fa4fe0ab700 1 -- 172.21.15.40:0/834223492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa4f806c540 con 0x7fa4f80c7fc0 2024-09-13T17:50:48.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.845+0000 7fa4edffb700 1 -- 172.21.15.40:0/834223492 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa4f4055f50 con 0x7fa4f80c7fc0 2024-09-13T17:50:48.852 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-09-13T17:50:48.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.849+0000 7fa4fe0ab700 1 -- 172.21.15.40:0/834223492 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4dc066730 msgr2=0x7fa4dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:48.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.849+0000 7fa4fe0ab700 1 --2- 172.21.15.40:0/834223492 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4dc066730 0x7fa4dc068bf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8006fd0 tx=0x7fa4e80092e0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.849+0000 7fa4fe0ab700 1 -- 172.21.15.40:0/834223492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f80c7fc0 msgr2=0x7fa4f8158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:48.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.849+0000 7fa4fe0ab700 1 --2- 172.21.15.40:0/834223492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f80c7fc0 0x7fa4f8158640 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4016000 tx=0x7fa4f4004770 comp rx=0 tx=0).stop 2024-09-13T17:50:48.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.849+0000 7fa4fe0ab700 1 -- 172.21.15.40:0/834223492 shutdown_connections 2024-09-13T17:50:48.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.849+0000 7fa4fe0ab700 1 --2- 172.21.15.40:0/834223492 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4dc066730 0x7fa4dc068bf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.849+0000 7fa4fe0ab700 1 --2- 172.21.15.40:0/834223492 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4f80c83a0 0x7fa4f8151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.849+0000 7fa4fe0ab700 1 --2- 172.21.15.40:0/834223492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f80c7fc0 0x7fa4f8158640 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.849+0000 7fa4fe0ab700 1 --2- 172.21.15.40:0/834223492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4f80c6d20 0x7fa4f81f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.849+0000 7fa4fe0ab700 1 -- 172.21.15.40:0/834223492 >> 172.21.15.40:0/834223492 conn(0x7fa4f80cc7a0 msgr2=0x7fa4f80c8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:48.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.849+0000 7fa4fe0ab700 1 -- 172.21.15.40:0/834223492 shutdown_connections 2024-09-13T17:50:48.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.849+0000 7fa4fe0ab700 1 -- 172.21.15.40:0/834223492 wait complete. 2024-09-13T17:50:48.870 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:50:48.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.981+0000 7fd25f385700 1 Processor -- start 2024-09-13T17:50:48.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.981+0000 7fd25f385700 1 -- start start 2024-09-13T17:50:48.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25f385700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd2580cb460 0x7fd25815d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:48.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25f385700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd2580cefd0 0x7fd2580cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:48.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25f385700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd25815a310 con 0x7fd258153a30 2024-09-13T17:50:48.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25f385700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2580b2720 con 0x7fd2580cefd0 2024-09-13T17:50:48.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25f385700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd25815dc20 con 0x7fd2580cb460 2024-09-13T17:50:48.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25eb84700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd2580cefd0 0x7fd2580cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:48.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25e383700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd258153a30 0x7fd2580c3f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32886/0 (socket says 172.21.15.40:32886) 2024-09-13T17:50:48.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25e383700 1 -- 172.21.15.40:0/2345116069 learned_addr learned my addr 172.21.15.40:0/2345116069 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:48.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25db82700 1 --2- 172.21.15.40:0/2345116069 >> v2:172.21.15.43:3300/0 conn(0x7fd2580cb460 0x7fd25815d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:48.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25db82700 1 -- 172.21.15.40:0/2345116069 >> v2:172.21.15.40:3300/0 conn(0x7fd2580cefd0 msgr2=0x7fd2580cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:48.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25db82700 1 --2- 172.21.15.40:0/2345116069 >> v2:172.21.15.40:3300/0 conn(0x7fd2580cefd0 0x7fd2580cf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25db82700 1 -- 172.21.15.40:0/2345116069 >> v1:172.21.15.40:6789/0 conn(0x7fd258153a30 legacy=0x7fd2580c3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:48.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25db82700 1 -- 172.21.15.40:0/2345116069 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd258159760 con 0x7fd2580cb460 2024-09-13T17:50:48.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25db82700 1 --2- 172.21.15.40:0/2345116069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2580cb460 0x7fd25815d6e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd2580cfaf0 tx=0x7fd248009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=992fe7c3cb74b355 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:48.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25d381700 1 -- 172.21.15.40:0/2345116069 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd248011830 con 0x7fd2580cb460 2024-09-13T17:50:48.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25d381700 1 -- 172.21.15.40:0/2345116069 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd248011990 con 0x7fd2580cb460 2024-09-13T17:50:48.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25d381700 1 -- 172.21.15.40:0/2345116069 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd248011d20 con 0x7fd2580cb460 2024-09-13T17:50:48.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25f385700 1 -- 172.21.15.40:0/2345116069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2580cb460 msgr2=0x7fd25815d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:48.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25f385700 1 --2- 172.21.15.40:0/2345116069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2580cb460 0x7fd25815d6e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd2580cfaf0 tx=0x7fd248009b30 comp rx=0 tx=0).stop 2024-09-13T17:50:48.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25f385700 1 -- 172.21.15.40:0/2345116069 shutdown_connections 2024-09-13T17:50:48.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25f385700 1 --2- 172.21.15.40:0/2345116069 >> v2:172.21.15.40:3300/0 conn(0x7fd2580cefd0 0x7fd2580cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25f385700 1 --2- 172.21.15.40:0/2345116069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2580cb460 0x7fd25815d6e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25f385700 1 -- 172.21.15.40:0/2345116069 >> 172.21.15.40:0/2345116069 conn(0x7fd25806c500 msgr2=0x7fd25806c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:48.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25f385700 1 -- 172.21.15.40:0/2345116069 shutdown_connections 2024-09-13T17:50:48.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.985+0000 7fd25f385700 1 -- 172.21.15.40:0/2345116069 wait complete. 2024-09-13T17:50:48.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.989+0000 7fd25f385700 1 Processor -- start 2024-09-13T17:50:48.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.989+0000 7fd25f385700 1 -- start start 2024-09-13T17:50:48.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.989+0000 7fd25f385700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2580cb460 0x7fd2581437e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:48.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.989+0000 7fd25f385700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2580cefd0 0x7fd258144630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:48.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.989+0000 7fd25f385700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd258153a30 0x7fd258152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:48.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.989+0000 7fd25f385700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580b5010 con 0x7fd2580cb460 2024-09-13T17:50:48.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.989+0000 7fd25f385700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580c9380 con 0x7fd258153a30 2024-09-13T17:50:48.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.989+0000 7fd25f385700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580c8ef0 con 0x7fd2580cefd0 2024-09-13T17:50:48.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.989+0000 7fd25e383700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2580cb460 0x7fd2581437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:48.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.989+0000 7fd25e383700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2580cb460 0x7fd2581437e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52640/0 (socket says 172.21.15.40:52640) 2024-09-13T17:50:48.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.989+0000 7fd25e383700 1 -- 172.21.15.40:0/2937463091 learned_addr learned my addr 172.21.15.40:0/2937463091 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:48.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.989+0000 7fd25db82700 1 --2- 172.21.15.40:0/2937463091 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2580cefd0 0x7fd258144630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:48.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.989+0000 7fd25eb84700 1 --2- 172.21.15.40:0/2937463091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd258153a30 0x7fd258152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:48.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.989+0000 7fd25e383700 1 -- 172.21.15.40:0/2937463091 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2580cefd0 msgr2=0x7fd258144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:48.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.989+0000 7fd25e383700 1 --2- 172.21.15.40:0/2937463091 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2580cefd0 0x7fd258144630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.989+0000 7fd25e383700 1 -- 172.21.15.40:0/2937463091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd258153a30 msgr2=0x7fd258152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:48.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.989+0000 7fd25e383700 1 --2- 172.21.15.40:0/2937463091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd258153a30 0x7fd258152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:48.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.989+0000 7fd25e383700 1 -- 172.21.15.40:0/2937463091 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2480094b0 con 0x7fd2580cb460 2024-09-13T17:50:48.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.989+0000 7fd25e383700 1 --2- 172.21.15.40:0/2937463091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2580cb460 0x7fd2581437e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd25400c440 tx=0x7fd25400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:49.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.993+0000 7fd24effd700 1 -- 172.21.15.40:0/2937463091 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254005d90 con 0x7fd2580cb460 2024-09-13T17:50:49.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.993+0000 7fd24effd700 1 -- 172.21.15.40:0/2937463091 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2540070d0 con 0x7fd2580cb460 2024-09-13T17:50:49.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.993+0000 7fd25f385700 1 -- 172.21.15.40:0/2937463091 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2580c8600 con 0x7fd2580cb460 2024-09-13T17:50:49.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.993+0000 7fd24effd700 1 -- 172.21.15.40:0/2937463091 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254007460 con 0x7fd2580cb460 2024-09-13T17:50:49.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.993+0000 7fd25f385700 1 -- 172.21.15.40:0/2937463091 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2580c8b80 con 0x7fd2580cb460 2024-09-13T17:50:49.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.993+0000 7fd25f385700 1 -- 172.21.15.40:0/2937463091 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd258067a40 con 0x7fd2580cb460 2024-09-13T17:50:49.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.993+0000 7fd24effd700 1 -- 172.21.15.40:0/2937463091 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd2540075c0 con 0x7fd2580cb460 2024-09-13T17:50:49.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.993+0000 7fd24effd700 1 --2- 172.21.15.40:0/2937463091 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd23c066760 0x7fd23c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:49.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.993+0000 7fd25db82700 1 --2- 172.21.15.40:0/2937463091 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd23c066760 0x7fd23c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:49.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.997+0000 7fd24effd700 1 -- 172.21.15.40:0/2937463091 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd2540838e0 con 0x7fd2580cb460 2024-09-13T17:50:49.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:48.997+0000 7fd25db82700 1 --2- 172.21.15.40:0/2937463091 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd23c066760 0x7fd23c068c20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd248000c00 tx=0x7fd248008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:49.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.005+0000 7fd24effd700 1 -- 172.21.15.40:0/2937463091 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd2540508c0 con 0x7fd2580cb460 2024-09-13T17:50:49.149 INFO:tasks.ceph.mds.b.smithi043.stderr:2024-09-13T17:50:49.144+0000 7fb6d9c92700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:50:49.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.177+0000 7fd25f385700 1 -- 172.21.15.40:0/2937463091 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd2580b2b00 con 0x7fd2580cb460 2024-09-13T17:50:49.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.177+0000 7fd24effd700 1 -- 172.21.15.40:0/2937463091 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd254053ee0 con 0x7fd2580cb460 2024-09-13T17:50:49.183 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-09-13T17:50:49.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.177+0000 7fd25f385700 1 -- 172.21.15.40:0/2937463091 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd23c066760 msgr2=0x7fd23c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:49.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.177+0000 7fd25f385700 1 --2- 172.21.15.40:0/2937463091 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd23c066760 0x7fd23c068c20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd248000c00 tx=0x7fd248008030 comp rx=0 tx=0).stop 2024-09-13T17:50:49.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.177+0000 7fd25f385700 1 -- 172.21.15.40:0/2937463091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2580cb460 msgr2=0x7fd2581437e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:49.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.177+0000 7fd25f385700 1 --2- 172.21.15.40:0/2937463091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2580cb460 0x7fd2581437e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd25400c440 tx=0x7fd25400c800 comp rx=0 tx=0).stop 2024-09-13T17:50:49.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.177+0000 7fd25f385700 1 -- 172.21.15.40:0/2937463091 shutdown_connections 2024-09-13T17:50:49.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.177+0000 7fd25f385700 1 --2- 172.21.15.40:0/2937463091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd258153a30 0x7fd258152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.177+0000 7fd25f385700 1 --2- 172.21.15.40:0/2937463091 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd23c066760 0x7fd23c068c20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.177+0000 7fd25f385700 1 --2- 172.21.15.40:0/2937463091 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2580cefd0 0x7fd258144630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.177+0000 7fd25f385700 1 --2- 172.21.15.40:0/2937463091 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2580cb460 0x7fd2581437e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.177+0000 7fd25f385700 1 -- 172.21.15.40:0/2937463091 >> 172.21.15.40:0/2937463091 conn(0x7fd25806c500 msgr2=0x7fd2580c5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:49.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.181+0000 7fd25f385700 1 -- 172.21.15.40:0/2937463091 shutdown_connections 2024-09-13T17:50:49.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.181+0000 7fd25f385700 1 -- 172.21.15.40:0/2937463091 wait complete. 2024-09-13T17:50:49.205 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:50:49.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.321+0000 7f0715092700 1 Processor -- start 2024-09-13T17:50:49.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.321+0000 7f0715092700 1 -- start start 2024-09-13T17:50:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f0715092700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0710156e10 0x7f071006c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f0715092700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07100c16f0 con 0x7f0710161f90 2024-09-13T17:50:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f0715092700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07100c2d80 con 0x7f071006cda0 2024-09-13T17:50:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f0715092700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0710159700 con 0x7f0710156e10 2024-09-13T17:50:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f0714891700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0710161f90 0x7f0710164380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32890/0 (socket says 172.21.15.40:32890) 2024-09-13T17:50:49.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f0714891700 1 -- 172.21.15.40:0/2288987697 learned_addr learned my addr 172.21.15.40:0/2288987697 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:49.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070ffff700 1 --2- 172.21.15.40:0/2288987697 >> v2:172.21.15.43:3300/0 conn(0x7f0710156e10 0x7f071006c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:49.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070f7fe700 1 -- 172.21.15.40:0/2288987697 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675185090 0 0) 0x7f07100c16f0 con 0x7f0710161f90 2024-09-13T17:50:49.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070f7fe700 1 -- 172.21.15.40:0/2288987697 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06f00035d0 con 0x7f0710161f90 2024-09-13T17:50:49.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070f7fe700 1 -- 172.21.15.40:0/2288987697 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 634161865 0 0) 0x7f07100c2d80 con 0x7f071006cda0 2024-09-13T17:50:49.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070f7fe700 1 -- 172.21.15.40:0/2288987697 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07100c16f0 con 0x7f071006cda0 2024-09-13T17:50:49.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070f7fe700 1 -- 172.21.15.40:0/2288987697 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 694811659 0 0) 0x7f06f00035d0 con 0x7f0710161f90 2024-09-13T17:50:49.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070f7fe700 1 -- 172.21.15.40:0/2288987697 >> v2:172.21.15.43:3300/0 conn(0x7f0710156e10 msgr2=0x7f071006c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:49.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070f7fe700 1 --2- 172.21.15.40:0/2288987697 >> v2:172.21.15.43:3300/0 conn(0x7f0710156e10 0x7f071006c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070f7fe700 1 -- 172.21.15.40:0/2288987697 >> v1:172.21.15.43:6789/0 conn(0x7f071006cda0 legacy=0x7f0710071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070f7fe700 1 -- 172.21.15.40:0/2288987697 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07100c1030 con 0x7f0710161f90 2024-09-13T17:50:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070f7fe700 1 -- 172.21.15.40:0/2288987697 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f07080019f0 con 0x7f0710161f90 2024-09-13T17:50:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070f7fe700 1 -- 172.21.15.40:0/2288987697 >> v1:172.21.15.40:6789/0 conn(0x7f0710161f90 legacy=0x7f0710164380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070f7fe700 1 --2- 172.21.15.40:0/2288987697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06f0003c00 0x7f06f0004000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070f7fe700 1 --2- 172.21.15.40:0/2288987697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f0004870 0x7f06f0004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070f7fe700 1 --2- 172.21.15.40:0/2288987697 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06f0005230 0x7f06f000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070f7fe700 1 -- 172.21.15.40:0/2288987697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f000c100 con 0x7f06f0003c00 2024-09-13T17:50:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070f7fe700 1 -- 172.21.15.40:0/2288987697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f000c240 con 0x7f06f0004870 2024-09-13T17:50:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070f7fe700 1 -- 172.21.15.40:0/2288987697 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f000c400 con 0x7f06f0005230 2024-09-13T17:50:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070ffff700 1 --2- 172.21.15.40:0/2288987697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06f0003c00 0x7f06f0004000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f07077fe700 1 --2- 172.21.15.40:0/2288987697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f0004870 0x7f06f0004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:49.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f0714891700 1 --2- 172.21.15.40:0/2288987697 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06f0005230 0x7f06f000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070ffff700 1 -- 172.21.15.40:0/2288987697 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06f0005230 msgr2=0x7f06f000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070ffff700 1 --2- 172.21.15.40:0/2288987697 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06f0005230 0x7f06f000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070ffff700 1 -- 172.21.15.40:0/2288987697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f0004870 msgr2=0x7f06f0004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070ffff700 1 --2- 172.21.15.40:0/2288987697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f0004870 0x7f06f0004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070ffff700 1 -- 172.21.15.40:0/2288987697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f00035d0 con 0x7f06f0003c00 2024-09-13T17:50:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f0714891700 1 --2- 172.21.15.40:0/2288987697 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06f0005230 0x7f06f000ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:50:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f0714891700 1 -- 172.21.15.40:0/2288987697 reap_dead start 2024-09-13T17:50:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070ffff700 1 --2- 172.21.15.40:0/2288987697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06f0003c00 0x7f06f0004000 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0700009b70 tx=0x7f06f00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1695a9229ccfe44a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070f7fe700 1 -- 172.21.15.40:0/2288987697 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070000e6c0 con 0x7f06f0003c00 2024-09-13T17:50:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070f7fe700 1 -- 172.21.15.40:0/2288987697 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070000e820 con 0x7f06f0003c00 2024-09-13T17:50:49.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.325+0000 7f070f7fe700 1 -- 172.21.15.40:0/2288987697 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070000eb90 con 0x7f06f0003c00 2024-09-13T17:50:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.329+0000 7f0715092700 1 -- 172.21.15.40:0/2288987697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06f0003c00 msgr2=0x7f06f0004000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.329+0000 7f0715092700 1 --2- 172.21.15.40:0/2288987697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06f0003c00 0x7f06f0004000 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0700009b70 tx=0x7f06f00058a0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.329+0000 7f0715092700 1 -- 172.21.15.40:0/2288987697 shutdown_connections 2024-09-13T17:50:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.329+0000 7f0715092700 1 --2- 172.21.15.40:0/2288987697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06f0003c00 0x7f06f0004000 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.329+0000 7f0715092700 1 -- 172.21.15.40:0/2288987697 >> 172.21.15.40:0/2288987697 conn(0x7f07100cc7f0 msgr2=0x7f07100c88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:49.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.329+0000 7f0715092700 1 -- 172.21.15.40:0/2288987697 shutdown_connections 2024-09-13T17:50:49.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.329+0000 7f0715092700 1 -- 172.21.15.40:0/2288987697 wait complete. 2024-09-13T17:50:49.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.329+0000 7f0715092700 1 Processor -- start 2024-09-13T17:50:49.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.329+0000 7f0715092700 1 -- start start 2024-09-13T17:50:49.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f0715092700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06f0003c00 0x7f071006ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:49.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f0715092700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f071014c370 0x7f07101f2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:49.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f0715092700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0710149460 0x7f0710148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:49.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f0715092700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071014b680 con 0x7f071014c370 2024-09-13T17:50:49.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f0715092700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07101f20e0 con 0x7f0710149460 2024-09-13T17:50:49.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f0715092700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07101effe0 con 0x7f06f0003c00 2024-09-13T17:50:49.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f07077fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f071014c370 0x7f07101f2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:49.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f07077fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f071014c370 0x7f07101f2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52660/0 (socket says 172.21.15.40:52660) 2024-09-13T17:50:49.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f07077fe700 1 -- 172.21.15.40:0/2020124667 learned_addr learned my addr 172.21.15.40:0/2020124667 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:49.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f070ffff700 1 --2- 172.21.15.40:0/2020124667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06f0003c00 0x7f071006ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:49.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f0714891700 1 --2- 172.21.15.40:0/2020124667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0710149460 0x7f0710148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:49.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f07077fe700 1 -- 172.21.15.40:0/2020124667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06f0003c00 msgr2=0x7f071006ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:49.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f07077fe700 1 --2- 172.21.15.40:0/2020124667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06f0003c00 0x7f071006ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f07077fe700 1 -- 172.21.15.40:0/2020124667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0710149460 msgr2=0x7f0710148670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:49.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f07077fe700 1 --2- 172.21.15.40:0/2020124667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0710149460 0x7f0710148670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f07077fe700 1 -- 172.21.15.40:0/2020124667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0710149930 con 0x7f071014c370 2024-09-13T17:50:49.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f07077fe700 1 --2- 172.21.15.40:0/2020124667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f071014c370 0x7f07101f2850 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f06f800f7c0 tx=0x7f06f800fad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:49.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f070d7fa700 1 -- 172.21.15.40:0/2020124667 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f8005320 con 0x7f071014c370 2024-09-13T17:50:49.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f070d7fa700 1 -- 172.21.15.40:0/2020124667 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f80054a0 con 0x7f071014c370 2024-09-13T17:50:49.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f0715092700 1 -- 172.21.15.40:0/2020124667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0710149c10 con 0x7f071014c370 2024-09-13T17:50:49.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f070d7fa700 1 -- 172.21.15.40:0/2020124667 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f8006ac0 con 0x7f071014c370 2024-09-13T17:50:49.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f0715092700 1 -- 172.21.15.40:0/2020124667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071014a1d0 con 0x7f071014c370 2024-09-13T17:50:49.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.333+0000 7f0715092700 1 -- 172.21.15.40:0/2020124667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071014b2a0 con 0x7f071014c370 2024-09-13T17:50:49.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.337+0000 7f070d7fa700 1 -- 172.21.15.40:0/2020124667 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f06f8006c20 con 0x7f071014c370 2024-09-13T17:50:49.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.337+0000 7f070d7fa700 1 --2- 172.21.15.40:0/2020124667 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f06f4066760 0x7f06f4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:49.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.337+0000 7f070d7fa700 1 -- 172.21.15.40:0/2020124667 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f06f8083c10 con 0x7f071014c370 2024-09-13T17:50:49.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.337+0000 7f070ffff700 1 --2- 172.21.15.40:0/2020124667 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f06f4066760 0x7f06f4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:49.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.337+0000 7f070ffff700 1 --2- 172.21.15.40:0/2020124667 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f06f4066760 0x7f06f4068c20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f07000090b0 tx=0x7f070001e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:49.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.345+0000 7f070d7fa700 1 -- 172.21.15.40:0/2020124667 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f071014b2a0 con 0x7f071014c370 2024-09-13T17:50:49.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.517+0000 7f0715092700 1 -- 172.21.15.40:0/2020124667 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f071014b2a0 con 0x7f071014c370 2024-09-13T17:50:49.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.517+0000 7f070d7fa700 1 -- 172.21.15.40:0/2020124667 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f071014b2a0 con 0x7f071014c370 2024-09-13T17:50:49.525 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-09-13T17:50:49.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.521+0000 7f0715092700 1 -- 172.21.15.40:0/2020124667 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f06f4066760 msgr2=0x7f06f4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:49.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.521+0000 7f0715092700 1 --2- 172.21.15.40:0/2020124667 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f06f4066760 0x7f06f4068c20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f07000090b0 tx=0x7f070001e040 comp rx=0 tx=0).stop 2024-09-13T17:50:49.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.521+0000 7f0715092700 1 -- 172.21.15.40:0/2020124667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f071014c370 msgr2=0x7f07101f2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:49.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.521+0000 7f0715092700 1 --2- 172.21.15.40:0/2020124667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f071014c370 0x7f07101f2850 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f06f800f7c0 tx=0x7f06f800fad0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.521+0000 7f0715092700 1 -- 172.21.15.40:0/2020124667 shutdown_connections 2024-09-13T17:50:49.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.521+0000 7f0715092700 1 --2- 172.21.15.40:0/2020124667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0710149460 0x7f0710148670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.521+0000 7f0715092700 1 --2- 172.21.15.40:0/2020124667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f071014c370 0x7f07101f2850 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.521+0000 7f0715092700 1 --2- 172.21.15.40:0/2020124667 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f06f4066760 0x7f06f4068c20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.521+0000 7f0715092700 1 --2- 172.21.15.40:0/2020124667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06f0003c00 0x7f071006ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.521+0000 7f0715092700 1 -- 172.21.15.40:0/2020124667 >> 172.21.15.40:0/2020124667 conn(0x7f07100cc7f0 msgr2=0x7f0710153d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:49.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.521+0000 7f0715092700 1 -- 172.21.15.40:0/2020124667 shutdown_connections 2024-09-13T17:50:49.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.521+0000 7f0715092700 1 -- 172.21.15.40:0/2020124667 wait complete. 2024-09-13T17:50:49.540 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:50:49.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.653+0000 7f801f166700 1 Processor -- start 2024-09-13T17:50:49.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.653+0000 7f801f166700 1 -- start start 2024-09-13T17:50:49.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.653+0000 7f801f166700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f801815a260 0x7f80180c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:49.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.653+0000 7f801f166700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80180cbdc0 con 0x7f80180cf1a0 2024-09-13T17:50:49.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.653+0000 7f801f166700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8018153f80 con 0x7f80180cf580 2024-09-13T17:50:49.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.653+0000 7f801f166700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80180caed0 con 0x7f801815a260 2024-09-13T17:50:49.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.653+0000 7f801e164700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f80180cf1a0 0x7f80180c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32906/0 (socket says 172.21.15.40:32906) 2024-09-13T17:50:49.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.653+0000 7f801e164700 1 -- 172.21.15.40:0/771077995 learned_addr learned my addr 172.21.15.40:0/771077995 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:49.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.653+0000 7f801e965700 1 --2- 172.21.15.40:0/771077995 >> v2:172.21.15.43:3300/0 conn(0x7f801815a260 0x7f80180c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:49.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.657+0000 7f801e965700 1 -- 172.21.15.40:0/771077995 >> v1:172.21.15.43:6789/0 conn(0x7f80180cf580 legacy=0x7f80180cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:49.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.657+0000 7f801e965700 1 -- 172.21.15.40:0/771077995 >> v1:172.21.15.40:6789/0 conn(0x7f80180cf1a0 legacy=0x7f80180c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:49.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.657+0000 7f801e965700 1 -- 172.21.15.40:0/771077995 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80180cb1b0 con 0x7f801815a260 2024-09-13T17:50:49.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.657+0000 7f801d162700 1 -- 172.21.15.40:0/771077995 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4092900942 0 0) 0x7f80180cbdc0 con 0x7f80180cf1a0 2024-09-13T17:50:49.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.657+0000 7f801e965700 1 --2- 172.21.15.40:0/771077995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f801815a260 0x7f80180c8aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8010009d90 tx=0x7f801000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=927e1fd0e2bf8739 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:49.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.657+0000 7f801d162700 1 -- 172.21.15.40:0/771077995 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801000e070 con 0x7f801815a260 2024-09-13T17:50:49.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.657+0000 7f801d162700 1 -- 172.21.15.40:0/771077995 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801000bbd0 con 0x7f801815a260 2024-09-13T17:50:49.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.657+0000 7f801d162700 1 -- 172.21.15.40:0/771077995 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8010012430 con 0x7f801815a260 2024-09-13T17:50:49.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.657+0000 7f801f166700 1 -- 172.21.15.40:0/771077995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f801815a260 msgr2=0x7f80180c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:49.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.657+0000 7f801f166700 1 --2- 172.21.15.40:0/771077995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f801815a260 0x7f80180c8aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8010009d90 tx=0x7f801000b330 comp rx=0 tx=0).stop 2024-09-13T17:50:49.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.657+0000 7f801f166700 1 -- 172.21.15.40:0/771077995 shutdown_connections 2024-09-13T17:50:49.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.657+0000 7f801f166700 1 --2- 172.21.15.40:0/771077995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f801815a260 0x7f80180c8aa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.657+0000 7f801f166700 1 -- 172.21.15.40:0/771077995 >> 172.21.15.40:0/771077995 conn(0x7f8018066090 msgr2=0x7f8018066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:49.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.657+0000 7f801f166700 1 -- 172.21.15.40:0/771077995 shutdown_connections 2024-09-13T17:50:49.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.657+0000 7f801f166700 1 -- 172.21.15.40:0/771077995 wait complete. 2024-09-13T17:50:49.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.657+0000 7f801f166700 1 Processor -- start 2024-09-13T17:50:49.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.657+0000 7f801f166700 1 -- start start 2024-09-13T17:50:49.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f801f166700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80180cf1a0 0x7f80180b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:49.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f801f166700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80180cf580 0x7f801815ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:49.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f801f166700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f801815a260 0x7f801815b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:49.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f801f166700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80180ad7f0 con 0x7f80180cf580 2024-09-13T17:50:49.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f801f166700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80180ae3b0 con 0x7f80180cf1a0 2024-09-13T17:50:49.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f801f166700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8018155ce0 con 0x7f801815a260 2024-09-13T17:50:49.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f801d963700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80180cf580 0x7f801815ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:49.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f801d963700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80180cf580 0x7f801815ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52668/0 (socket says 172.21.15.40:52668) 2024-09-13T17:50:49.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f801d963700 1 -- 172.21.15.40:0/2490930616 learned_addr learned my addr 172.21.15.40:0/2490930616 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:49.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f801e965700 1 --2- 172.21.15.40:0/2490930616 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f801815a260 0x7f801815b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:49.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f801e164700 1 --2- 172.21.15.40:0/2490930616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80180cf1a0 0x7f80180b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:49.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f801d963700 1 -- 172.21.15.40:0/2490930616 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f801815a260 msgr2=0x7f801815b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:49.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f801d963700 1 --2- 172.21.15.40:0/2490930616 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f801815a260 0x7f801815b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f801d963700 1 -- 172.21.15.40:0/2490930616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80180cf1a0 msgr2=0x7f80180b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:49.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f801d963700 1 --2- 172.21.15.40:0/2490930616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80180cf1a0 0x7f80180b0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f801d963700 1 -- 172.21.15.40:0/2490930616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8010009a70 con 0x7f80180cf580 2024-09-13T17:50:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f801d963700 1 --2- 172.21.15.40:0/2490930616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80180cf580 0x7f801815ae60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f800c00c480 tx=0x7f800c00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f800affd700 1 -- 172.21.15.40:0/2490930616 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c0052d0 con 0x7f80180cf580 2024-09-13T17:50:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f800affd700 1 -- 172.21.15.40:0/2490930616 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800c00ce90 con 0x7f80180cf580 2024-09-13T17:50:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f801f166700 1 -- 172.21.15.40:0/2490930616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8018155fc0 con 0x7f80180cf580 2024-09-13T17:50:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f800affd700 1 -- 172.21.15.40:0/2490930616 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c006880 con 0x7f80180cf580 2024-09-13T17:50:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f801f166700 1 -- 172.21.15.40:0/2490930616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80181eff90 con 0x7f80180cf580 2024-09-13T17:50:49.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.661+0000 7f801f166700 1 -- 172.21.15.40:0/2490930616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8018067a40 con 0x7f80180cf580 2024-09-13T17:50:49.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.665+0000 7f800affd700 1 -- 172.21.15.40:0/2490930616 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f800c0036a0 con 0x7f80180cf580 2024-09-13T17:50:49.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.665+0000 7f800affd700 1 --2- 172.21.15.40:0/2490930616 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7ffc066730 0x7f7ffc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:49.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.665+0000 7f800affd700 1 -- 172.21.15.40:0/2490930616 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f800c083700 con 0x7f80180cf580 2024-09-13T17:50:49.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.665+0000 7f801e164700 1 --2- 172.21.15.40:0/2490930616 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7ffc066730 0x7f7ffc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:49.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.665+0000 7f801e164700 1 --2- 172.21.15.40:0/2490930616 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7ffc066730 0x7f7ffc068bf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8014000f60 tx=0x7f8014007e00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:49.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.673+0000 7f800affd700 1 -- 172.21.15.40:0/2490930616 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f800c050660 con 0x7f80180cf580 2024-09-13T17:50:49.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.857+0000 7f801f166700 1 -- 172.21.15.40:0/2490930616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f801815ac50 con 0x7f80180cf580 2024-09-13T17:50:49.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.857+0000 7f800affd700 1 -- 172.21.15.40:0/2490930616 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f800c053c80 con 0x7f80180cf580 2024-09-13T17:50:49.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.861+0000 7f801f166700 1 -- 172.21.15.40:0/2490930616 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7ffc066730 msgr2=0x7f7ffc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:49.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.861+0000 7f801f166700 1 --2- 172.21.15.40:0/2490930616 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7ffc066730 0x7f7ffc068bf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8014000f60 tx=0x7f8014007e00 comp rx=0 tx=0).stop 2024-09-13T17:50:49.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.861+0000 7f801f166700 1 -- 172.21.15.40:0/2490930616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80180cf580 msgr2=0x7f801815ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:49.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.861+0000 7f801f166700 1 --2- 172.21.15.40:0/2490930616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80180cf580 0x7f801815ae60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f800c00c480 tx=0x7f800c00c790 comp rx=0 tx=0).stop 2024-09-13T17:50:49.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.861+0000 7f801f166700 1 -- 172.21.15.40:0/2490930616 shutdown_connections 2024-09-13T17:50:49.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.861+0000 7f801f166700 1 --2- 172.21.15.40:0/2490930616 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7ffc066730 0x7f7ffc068bf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.861+0000 7f801f166700 1 --2- 172.21.15.40:0/2490930616 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f801815a260 0x7f801815b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.861+0000 7f801f166700 1 --2- 172.21.15.40:0/2490930616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f80180cf580 0x7f801815ae60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.861+0000 7f801f166700 1 --2- 172.21.15.40:0/2490930616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80180cf1a0 0x7f80180b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:49.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.861+0000 7f801f166700 1 -- 172.21.15.40:0/2490930616 >> 172.21.15.40:0/2490930616 conn(0x7f8018066090 msgr2=0x7f8018066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:49.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.861+0000 7f801f166700 1 -- 172.21.15.40:0/2490930616 shutdown_connections 2024-09-13T17:50:49.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:49.861+0000 7f801f166700 1 -- 172.21.15.40:0/2490930616 wait complete. 2024-09-13T17:50:49.869 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-09-13T17:50:49.880 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:50:49.881 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:50:49.889 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:50:50.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.045+0000 7f2ad605e700 1 Processor -- start 2024-09-13T17:50:50.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.045+0000 7f2ad605e700 1 -- start start 2024-09-13T17:50:50.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad605e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2ad01502f0 0x7f2ad0072560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:50.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad605e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ad006b4a0 0x7f2ad006b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:50.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad605e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad006d930 con 0x7f2ad0070bc0 2024-09-13T17:50:50.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad605e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ad00712f0 con 0x7f2ad01502f0 2024-09-13T17:50:50.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad605e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad006ca60 con 0x7f2ad006b4a0 2024-09-13T17:50:50.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad505c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2ad01502f0 0x7f2ad0072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:50.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad505c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2ad01502f0 0x7f2ad0072560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52682/0 (socket says 172.21.15.40:52682) 2024-09-13T17:50:50.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad505c700 1 -- 172.21.15.40:0/1617928629 learned_addr learned my addr 172.21.15.40:0/1617928629 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:50.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad585d700 1 --2- 172.21.15.40:0/1617928629 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ad006b4a0 0x7f2ad006b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:50.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad505c700 1 -- 172.21.15.40:0/1617928629 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ad006b4a0 msgr2=0x7f2ad006b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:50.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad505c700 1 --2- 172.21.15.40:0/1617928629 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ad006b4a0 0x7f2ad006b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad505c700 1 -- 172.21.15.40:0/1617928629 >> v1:172.21.15.43:6789/0 conn(0x7f2ad0070bc0 legacy=0x7f2ad006d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:50.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad505c700 1 -- 172.21.15.40:0/1617928629 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad0156e40 con 0x7f2ad01502f0 2024-09-13T17:50:50.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad585d700 1 --2- 172.21.15.40:0/1617928629 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ad006b4a0 0x7f2ad006b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:50:50.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad505c700 1 --2- 172.21.15.40:0/1617928629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ad01502f0 0x7f2ad0072560 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00bbb0 tx=0x7f2acc00bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ec92204f2c5c41 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:50.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ac7fff700 1 -- 172.21.15.40:0/1617928629 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc00dd90 con 0x7f2ad01502f0 2024-09-13T17:50:50.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ac7fff700 1 -- 172.21.15.40:0/1617928629 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2acc010470 con 0x7f2ad01502f0 2024-09-13T17:50:50.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ac7fff700 1 -- 172.21.15.40:0/1617928629 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc010820 con 0x7f2ad01502f0 2024-09-13T17:50:50.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad605e700 1 -- 172.21.15.40:0/1617928629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ad01502f0 msgr2=0x7f2ad0072560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:50.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad605e700 1 --2- 172.21.15.40:0/1617928629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ad01502f0 0x7f2ad0072560 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00bbb0 tx=0x7f2acc00bec0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad605e700 1 -- 172.21.15.40:0/1617928629 shutdown_connections 2024-09-13T17:50:50.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad605e700 1 --2- 172.21.15.40:0/1617928629 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ad006b4a0 0x7f2ad006b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad605e700 1 --2- 172.21.15.40:0/1617928629 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ad01502f0 0x7f2ad0072560 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad605e700 1 -- 172.21.15.40:0/1617928629 >> 172.21.15.40:0/1617928629 conn(0x7f2ad00c88e0 msgr2=0x7f2ad00c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad605e700 1 -- 172.21.15.40:0/1617928629 shutdown_connections 2024-09-13T17:50:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.049+0000 7f2ad605e700 1 -- 172.21.15.40:0/1617928629 wait complete. 2024-09-13T17:50:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad605e700 1 Processor -- start 2024-09-13T17:50:50.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad605e700 1 -- start start 2024-09-13T17:50:50.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad605e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ad006b4a0 0x7f2ad01dfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:50.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad605e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ad0070bc0 0x7f2ad00c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:50.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad605e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ad01502f0 0x7f2ad01f2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:50.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad605e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00c3840 con 0x7f2ad006b4a0 2024-09-13T17:50:50.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad605e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00c3b20 con 0x7f2ad0070bc0 2024-09-13T17:50:50.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad605e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad00c3e00 con 0x7f2ad01502f0 2024-09-13T17:50:50.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad505c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ad006b4a0 0x7f2ad01dfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:50.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad505c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ad006b4a0 0x7f2ad01dfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52690/0 (socket says 172.21.15.40:52690) 2024-09-13T17:50:50.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad505c700 1 -- 172.21.15.40:0/1010351227 learned_addr learned my addr 172.21.15.40:0/1010351227 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:50.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad485b700 1 --2- 172.21.15.40:0/1010351227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ad0070bc0 0x7f2ad00c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:50.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad585d700 1 --2- 172.21.15.40:0/1010351227 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ad01502f0 0x7f2ad01f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:50.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad505c700 1 -- 172.21.15.40:0/1010351227 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ad01502f0 msgr2=0x7f2ad01f2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:50.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad505c700 1 --2- 172.21.15.40:0/1010351227 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ad01502f0 0x7f2ad01f2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad505c700 1 -- 172.21.15.40:0/1010351227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ad0070bc0 msgr2=0x7f2ad00c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:50.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad505c700 1 --2- 172.21.15.40:0/1010351227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ad0070bc0 0x7f2ad00c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad505c700 1 -- 172.21.15.40:0/1010351227 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2acc00b810 con 0x7f2ad006b4a0 2024-09-13T17:50:50.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad505c700 1 --2- 172.21.15.40:0/1010351227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ad006b4a0 0x7f2ad01dfbb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00bf00 tx=0x7f2acc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:50.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ac5ffb700 1 -- 172.21.15.40:0/1010351227 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc003b90 con 0x7f2ad006b4a0 2024-09-13T17:50:50.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ac5ffb700 1 -- 172.21.15.40:0/1010351227 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2acc003cf0 con 0x7f2ad006b4a0 2024-09-13T17:50:50.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad605e700 1 -- 172.21.15.40:0/1010351227 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad00c40e0 con 0x7f2ad006b4a0 2024-09-13T17:50:50.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ac5ffb700 1 -- 172.21.15.40:0/1010351227 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2acc010800 con 0x7f2ad006b4a0 2024-09-13T17:50:50.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.053+0000 7f2ad605e700 1 -- 172.21.15.40:0/1010351227 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad00c46a0 con 0x7f2ad006b4a0 2024-09-13T17:50:50.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.057+0000 7f2ad605e700 1 -- 172.21.15.40:0/1010351227 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad00c8680 con 0x7f2ad006b4a0 2024-09-13T17:50:50.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.057+0000 7f2ac5ffb700 1 -- 172.21.15.40:0/1010351227 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2acc019400 con 0x7f2ad006b4a0 2024-09-13T17:50:50.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.057+0000 7f2ac5ffb700 1 --2- 172.21.15.40:0/1010351227 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2ab4066760 0x7f2ab4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:50.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.057+0000 7f2ac5ffb700 1 -- 172.21.15.40:0/1010351227 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2acc085830 con 0x7f2ad006b4a0 2024-09-13T17:50:50.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.057+0000 7f2ad485b700 1 --2- 172.21.15.40:0/1010351227 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2ab4066760 0x7f2ab4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:50.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.057+0000 7f2ad485b700 1 --2- 172.21.15.40:0/1010351227 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2ab4066760 0x7f2ab4068c20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2ad01e0400 tx=0x7f2ac0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:50.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.065+0000 7f2ac5ffb700 1 -- 172.21.15.40:0/1010351227 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2acc052940 con 0x7f2ad006b4a0 2024-09-13T17:50:50.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.245+0000 7f2ad605e700 1 -- 172.21.15.40:0/1010351227 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ad006c4d0 con 0x7f2ad006b4a0 2024-09-13T17:50:50.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.249+0000 7f2ac5ffb700 1 -- 172.21.15.40:0/1010351227 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18262 (secure 0 0 0) 0x7f2acc003e60 con 0x7f2ad006b4a0 2024-09-13T17:50:50.254 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:50:50.255 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:50:38.974071+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:50:32.740336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:50:33.552914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:50:36.623639+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:50:50.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.249+0000 7f2ad605e700 1 -- 172.21.15.40:0/1010351227 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2ab4066760 msgr2=0x7f2ab4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:50.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.249+0000 7f2ad605e700 1 --2- 172.21.15.40:0/1010351227 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2ab4066760 0x7f2ab4068c20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2ad01e0400 tx=0x7f2ac0009040 comp rx=0 tx=0).stop 2024-09-13T17:50:50.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.249+0000 7f2ad605e700 1 -- 172.21.15.40:0/1010351227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ad006b4a0 msgr2=0x7f2ad01dfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:50.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.249+0000 7f2ad605e700 1 --2- 172.21.15.40:0/1010351227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ad006b4a0 0x7f2ad01dfbb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2acc00bf00 tx=0x7f2acc004770 comp rx=0 tx=0).stop 2024-09-13T17:50:50.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.249+0000 7f2ad605e700 1 -- 172.21.15.40:0/1010351227 shutdown_connections 2024-09-13T17:50:50.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.249+0000 7f2ad605e700 1 --2- 172.21.15.40:0/1010351227 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2ab4066760 0x7f2ab4068c20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.249+0000 7f2ad605e700 1 --2- 172.21.15.40:0/1010351227 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ad01502f0 0x7f2ad01f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.249+0000 7f2ad605e700 1 --2- 172.21.15.40:0/1010351227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ad0070bc0 0x7f2ad00c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.249+0000 7f2ad605e700 1 --2- 172.21.15.40:0/1010351227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ad006b4a0 0x7f2ad01dfbb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.249+0000 7f2ad605e700 1 -- 172.21.15.40:0/1010351227 >> 172.21.15.40:0/1010351227 conn(0x7f2ad00c88e0 msgr2=0x7f2ad00c64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:50.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.249+0000 7f2ad605e700 1 -- 172.21.15.40:0/1010351227 shutdown_connections 2024-09-13T17:50:50.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.249+0000 7f2ad605e700 1 -- 172.21.15.40:0/1010351227 wait complete. 2024-09-13T17:50:50.270 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:50:21.057171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:50:32.740336+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:50:33.552914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:50:36.623639+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:50:50.270 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:50:50.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.385+0000 7f0ebffff700 1 Processor -- start 2024-09-13T17:50:50.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.385+0000 7f0ebffff700 1 -- start start 2024-09-13T17:50:50.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.385+0000 7f0ebffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0ec0156a90 0x7f0ec006e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:50.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.385+0000 7f0ebffff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec0066460 con 0x7f0ec0150520 2024-09-13T17:50:50.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.385+0000 7f0ebffff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec0066130 con 0x7f0ec0150900 2024-09-13T17:50:50.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.385+0000 7f0ebffff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec00c3830 con 0x7f0ec0156a90 2024-09-13T17:50:50.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.385+0000 7f0ebf7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0ec0156a90 0x7f0ec006e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:50.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.385+0000 7f0ebeffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0ec0150520 0x7f0ec00c4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32914/0 (socket says 172.21.15.40:32914) 2024-09-13T17:50:50.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.385+0000 7f0ebeffd700 1 -- 172.21.15.40:0/1716409131 learned_addr learned my addr 172.21.15.40:0/1716409131 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:50.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.385+0000 7f0ebf7fe700 1 -- 172.21.15.40:0/1716409131 >> v1:172.21.15.43:6789/0 conn(0x7f0ec0150900 legacy=0x7f0ec0156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:50.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.385+0000 7f0ebe7fc700 1 -- 172.21.15.40:0/1716409131 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2315519980 0 0) 0x7f0ec0066460 con 0x7f0ec0150520 2024-09-13T17:50:50.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.385+0000 7f0ebf7fe700 1 -- 172.21.15.40:0/1716409131 >> v1:172.21.15.40:6789/0 conn(0x7f0ec0150520 legacy=0x7f0ec00c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:50.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.385+0000 7f0ebf7fe700 1 -- 172.21.15.40:0/1716409131 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec0070e80 con 0x7f0ec0156a90 2024-09-13T17:50:50.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.389+0000 7f0ebf7fe700 1 --2- 172.21.15.40:0/1716409131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ec0156a90 0x7f0ec006e860 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009d90 tx=0x7f0eb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d693a7dd15980979 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:50.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.389+0000 7f0ebe7fc700 1 -- 172.21.15.40:0/1716409131 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb800e070 con 0x7f0ec0156a90 2024-09-13T17:50:50.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.389+0000 7f0ebe7fc700 1 -- 172.21.15.40:0/1716409131 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb800bbd0 con 0x7f0ec0156a90 2024-09-13T17:50:50.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.389+0000 7f0ebe7fc700 1 -- 172.21.15.40:0/1716409131 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8012430 con 0x7f0ec0156a90 2024-09-13T17:50:50.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.389+0000 7f0ebffff700 1 -- 172.21.15.40:0/1716409131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ec0156a90 msgr2=0x7f0ec006e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:50.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.389+0000 7f0ebffff700 1 --2- 172.21.15.40:0/1716409131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ec0156a90 0x7f0ec006e860 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009d90 tx=0x7f0eb800b330 comp rx=0 tx=0).stop 2024-09-13T17:50:50.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.389+0000 7f0ebffff700 1 -- 172.21.15.40:0/1716409131 shutdown_connections 2024-09-13T17:50:50.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.389+0000 7f0ebffff700 1 --2- 172.21.15.40:0/1716409131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ec0156a90 0x7f0ec006e860 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.389+0000 7f0ebffff700 1 -- 172.21.15.40:0/1716409131 >> 172.21.15.40:0/1716409131 conn(0x7f0ec00cb450 msgr2=0x7f0ec00cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:50.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.389+0000 7f0ebffff700 1 -- 172.21.15.40:0/1716409131 shutdown_connections 2024-09-13T17:50:50.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.389+0000 7f0ebffff700 1 -- 172.21.15.40:0/1716409131 wait complete. 2024-09-13T17:50:50.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.389+0000 7f0ebffff700 1 Processor -- start 2024-09-13T17:50:50.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.389+0000 7f0ebffff700 1 -- start start 2024-09-13T17:50:50.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0ebffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ec0150520 0x7f0ec014c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:50.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0ebffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ec0156a90 0x7f0ec01eec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:50.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0ebffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ec01438f0 0x7f0ec0142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:50.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0ebffff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec01ef140 con 0x7f0ec0150520 2024-09-13T17:50:50.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0ebffff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec0156360 con 0x7f0ec0156a90 2024-09-13T17:50:50.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0ebffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec0155ae0 con 0x7f0ec01438f0 2024-09-13T17:50:50.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0ebeffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ec0150520 0x7f0ec014c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:50.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0ebeffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ec0150520 0x7f0ec014c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52716/0 (socket says 172.21.15.40:52716) 2024-09-13T17:50:50.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0ebeffd700 1 -- 172.21.15.40:0/2487891459 learned_addr learned my addr 172.21.15.40:0/2487891459 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:50.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0ebf7fe700 1 --2- 172.21.15.40:0/2487891459 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ec01438f0 0x7f0ec0142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:50.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0eb7fff700 1 --2- 172.21.15.40:0/2487891459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ec0156a90 0x7f0ec01eec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:50.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0ebeffd700 1 -- 172.21.15.40:0/2487891459 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ec01438f0 msgr2=0x7f0ec0142fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:50.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0ebeffd700 1 --2- 172.21.15.40:0/2487891459 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ec01438f0 0x7f0ec0142fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0ebeffd700 1 -- 172.21.15.40:0/2487891459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ec0156a90 msgr2=0x7f0ec01eec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:50.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0ebeffd700 1 --2- 172.21.15.40:0/2487891459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ec0156a90 0x7f0ec01eec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0ebeffd700 1 -- 172.21.15.40:0/2487891459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb8009a70 con 0x7f0ec0150520 2024-09-13T17:50:50.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0eb7fff700 1 --2- 172.21.15.40:0/2487891459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ec0156a90 0x7f0ec01eec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:50:50.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0ebf7fe700 1 --2- 172.21.15.40:0/2487891459 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ec01438f0 0x7f0ec0142fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:50:50.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0ebeffd700 1 --2- 172.21.15.40:0/2487891459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ec0150520 0x7f0ec014c430 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000c550 tx=0x7f0eb000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:50.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0eb77fe700 1 -- 172.21.15.40:0/2487891459 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0004ca0 con 0x7f0ec0150520 2024-09-13T17:50:50.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0eb77fe700 1 -- 172.21.15.40:0/2487891459 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb0004e00 con 0x7f0ec0150520 2024-09-13T17:50:50.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0ebffff700 1 -- 172.21.15.40:0/2487891459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec0155da0 con 0x7f0ec0150520 2024-09-13T17:50:50.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0eb77fe700 1 -- 172.21.15.40:0/2487891459 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0007310 con 0x7f0ec0150520 2024-09-13T17:50:50.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0ebffff700 1 -- 172.21.15.40:0/2487891459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec014bc30 con 0x7f0ec0150520 2024-09-13T17:50:50.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0ebffff700 1 -- 172.21.15.40:0/2487891459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec014ba90 con 0x7f0ec0150520 2024-09-13T17:50:50.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0eb77fe700 1 -- 172.21.15.40:0/2487891459 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0eb0007470 con 0x7f0ec0150520 2024-09-13T17:50:50.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0eb77fe700 1 --2- 172.21.15.40:0/2487891459 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0ea0066730 0x7f0ea0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:50.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0eb77fe700 1 -- 172.21.15.40:0/2487891459 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0eb0084130 con 0x7f0ec0150520 2024-09-13T17:50:50.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.393+0000 7f0eb7fff700 1 --2- 172.21.15.40:0/2487891459 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0ea0066730 0x7f0ea0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:50.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.397+0000 7f0eb7fff700 1 --2- 172.21.15.40:0/2487891459 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0ea0066730 0x7f0ea0068bf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0072a30 tx=0x7f0ea8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:50.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.401+0000 7f0eb77fe700 1 -- 172.21.15.40:0/2487891459 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0ec014ba90 con 0x7f0ec0150520 2024-09-13T17:50:50.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.565+0000 7f0ebffff700 1 -- 172.21.15.40:0/2487891459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0ec014ba90 con 0x7f0ec0150520 2024-09-13T17:50:50.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.565+0000 7f0eb77fe700 1 -- 172.21.15.40:0/2487891459 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ec014ba90 con 0x7f0ec0150520 2024-09-13T17:50:50.573 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-09-13T17:50:50.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.569+0000 7f0ebffff700 1 -- 172.21.15.40:0/2487891459 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0ea0066730 msgr2=0x7f0ea0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:50.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.569+0000 7f0ebffff700 1 --2- 172.21.15.40:0/2487891459 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0ea0066730 0x7f0ea0068bf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0072a30 tx=0x7f0ea8009040 comp rx=0 tx=0).stop 2024-09-13T17:50:50.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.569+0000 7f0ebffff700 1 -- 172.21.15.40:0/2487891459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ec0150520 msgr2=0x7f0ec014c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:50.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.569+0000 7f0ebffff700 1 --2- 172.21.15.40:0/2487891459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ec0150520 0x7f0ec014c430 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000c550 tx=0x7f0eb000c910 comp rx=0 tx=0).stop 2024-09-13T17:50:50.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.569+0000 7f0ebffff700 1 -- 172.21.15.40:0/2487891459 shutdown_connections 2024-09-13T17:50:50.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.569+0000 7f0ebffff700 1 --2- 172.21.15.40:0/2487891459 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0ea0066730 0x7f0ea0068bf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.569+0000 7f0ebffff700 1 --2- 172.21.15.40:0/2487891459 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ec01438f0 0x7f0ec0142fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.569+0000 7f0ebffff700 1 --2- 172.21.15.40:0/2487891459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ec0156a90 0x7f0ec01eec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.569+0000 7f0ebffff700 1 --2- 172.21.15.40:0/2487891459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ec0150520 0x7f0ec014c430 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.569+0000 7f0ebffff700 1 -- 172.21.15.40:0/2487891459 >> 172.21.15.40:0/2487891459 conn(0x7f0ec00cb450 msgr2=0x7f0ec00c65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:50.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.569+0000 7f0ebffff700 1 -- 172.21.15.40:0/2487891459 shutdown_connections 2024-09-13T17:50:50.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.569+0000 7f0ebffff700 1 -- 172.21.15.40:0/2487891459 wait complete. 2024-09-13T17:50:50.587 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:50:50.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.697+0000 7f6ba759e700 1 Processor -- start 2024-09-13T17:50:50.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.697+0000 7f6ba759e700 1 -- start start 2024-09-13T17:50:50.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.697+0000 7f6ba759e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6ba8151590 0x7f6ba8157270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.697+0000 7f6ba759e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6ba80b2d90 0x7f6ba815a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.697+0000 7f6ba759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ba806e7e0 0x7f6ba80c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.697+0000 7f6ba759e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba815b7b0 con 0x7f6ba80b2d90 2024-09-13T17:50:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.697+0000 7f6ba759e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba806e350 con 0x7f6ba8151590 2024-09-13T17:50:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.697+0000 7f6ba759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba815af10 con 0x7f6ba806e7e0 2024-09-13T17:50:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.697+0000 7f6b9ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6ba80b2d90 0x7f6ba815a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.697+0000 7f6ba6d9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ba806e7e0 0x7f6ba80c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.697+0000 7f6b9ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6ba80b2d90 0x7f6ba815a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52724/0 (socket says 172.21.15.40:52724) 2024-09-13T17:50:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.697+0000 7f6ba659c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6ba8151590 0x7f6ba8157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.697+0000 7f6b9ffff700 1 -- 172.21.15.40:0/917440985 learned_addr learned my addr 172.21.15.40:0/917440985 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.701+0000 7f6b9ffff700 1 -- 172.21.15.40:0/917440985 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ba806e7e0 msgr2=0x7f6ba80c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.701+0000 7f6b9ffff700 1 --2- 172.21.15.40:0/917440985 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ba806e7e0 0x7f6ba80c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.701+0000 7f6b9ffff700 1 -- 172.21.15.40:0/917440985 >> v2:172.21.15.43:3300/0 conn(0x7f6ba8151590 msgr2=0x7f6ba8157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.701+0000 7f6b9ffff700 1 --2- 172.21.15.40:0/917440985 >> v2:172.21.15.43:3300/0 conn(0x7f6ba8151590 0x7f6ba8157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.701+0000 7f6b9ffff700 1 -- 172.21.15.40:0/917440985 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba815b1f0 con 0x7f6ba80b2d90 2024-09-13T17:50:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.701+0000 7f6ba6d9d700 1 --2- 172.21.15.40:0/917440985 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ba806e7e0 0x7f6ba80c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:50:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.701+0000 7f6b9ffff700 1 --2- 172.21.15.40:0/917440985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba80b2d90 0x7f6ba815a860 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80675b0 tx=0x7f6b90009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85281a080316427 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.701+0000 7f6ba5d9b700 1 -- 172.21.15.40:0/917440985 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b90011830 con 0x7f6ba80b2d90 2024-09-13T17:50:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.701+0000 7f6ba5d9b700 1 -- 172.21.15.40:0/917440985 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b90011990 con 0x7f6ba80b2d90 2024-09-13T17:50:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.701+0000 7f6ba5d9b700 1 -- 172.21.15.40:0/917440985 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b90011d20 con 0x7f6ba80b2d90 2024-09-13T17:50:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.701+0000 7f6ba759e700 1 -- 172.21.15.40:0/917440985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba80b2d90 msgr2=0x7f6ba815a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:50.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.701+0000 7f6ba759e700 1 --2- 172.21.15.40:0/917440985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba80b2d90 0x7f6ba815a860 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80675b0 tx=0x7f6b90009be0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.701+0000 7f6ba759e700 1 -- 172.21.15.40:0/917440985 shutdown_connections 2024-09-13T17:50:50.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.701+0000 7f6ba759e700 1 --2- 172.21.15.40:0/917440985 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ba806e7e0 0x7f6ba80c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.701+0000 7f6ba759e700 1 --2- 172.21.15.40:0/917440985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba80b2d90 0x7f6ba815a860 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.701+0000 7f6ba759e700 1 --2- 172.21.15.40:0/917440985 >> v2:172.21.15.43:3300/0 conn(0x7f6ba8151590 0x7f6ba8157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.701+0000 7f6ba759e700 1 -- 172.21.15.40:0/917440985 >> 172.21.15.40:0/917440985 conn(0x7f6ba80ccef0 msgr2=0x7f6ba80cd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:50.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.701+0000 7f6ba759e700 1 -- 172.21.15.40:0/917440985 shutdown_connections 2024-09-13T17:50:50.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.701+0000 7f6ba759e700 1 -- 172.21.15.40:0/917440985 wait complete. 2024-09-13T17:50:50.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba759e700 1 Processor -- start 2024-09-13T17:50:50.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba759e700 1 -- start start 2024-09-13T17:50:50.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba806e7e0 0x7f6ba81eefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:50.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ba80b2d90 0x7f6ba814cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:50.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba8151590 0x7f6ba806b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:50.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba8154080 con 0x7f6ba8151590 2024-09-13T17:50:50.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba815a8e0 con 0x7f6ba806e7e0 2024-09-13T17:50:50.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba8159f10 con 0x7f6ba80b2d90 2024-09-13T17:50:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba8151590 0x7f6ba806b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba8151590 0x7f6ba806b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52728/0 (socket says 172.21.15.40:52728) 2024-09-13T17:50:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba6d9d700 1 -- 172.21.15.40:0/666253208 learned_addr learned my addr 172.21.15.40:0/666253208 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6b9ffff700 1 --2- 172.21.15.40:0/666253208 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ba80b2d90 0x7f6ba814cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba659c700 1 --2- 172.21.15.40:0/666253208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba806e7e0 0x7f6ba81eefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba6d9d700 1 -- 172.21.15.40:0/666253208 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ba80b2d90 msgr2=0x7f6ba814cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba6d9d700 1 --2- 172.21.15.40:0/666253208 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ba80b2d90 0x7f6ba814cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba6d9d700 1 -- 172.21.15.40:0/666253208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba806e7e0 msgr2=0x7f6ba81eefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba6d9d700 1 --2- 172.21.15.40:0/666253208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba806e7e0 0x7f6ba81eefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba6d9d700 1 -- 172.21.15.40:0/666253208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba0009a70 con 0x7f6ba8151590 2024-09-13T17:50:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba6d9d700 1 --2- 172.21.15.40:0/666253208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba8151590 0x7f6ba806b4a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000eed0 tx=0x7f6ba000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6b9effd700 1 -- 172.21.15.40:0/666253208 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0010040 con 0x7f6ba8151590 2024-09-13T17:50:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6b9effd700 1 -- 172.21.15.40:0/666253208 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba000ba80 con 0x7f6ba8151590 2024-09-13T17:50:50.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba759e700 1 -- 172.21.15.40:0/666253208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b90009510 con 0x7f6ba8151590 2024-09-13T17:50:50.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6b9effd700 1 -- 172.21.15.40:0/666253208 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba000be10 con 0x7f6ba8151590 2024-09-13T17:50:50.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba759e700 1 -- 172.21.15.40:0/666253208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba8153d10 con 0x7f6ba8151590 2024-09-13T17:50:50.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.705+0000 7f6ba759e700 1 -- 172.21.15.40:0/666253208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba8154230 con 0x7f6ba8151590 2024-09-13T17:50:50.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.709+0000 7f6b9effd700 1 -- 172.21.15.40:0/666253208 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6ba0003a40 con 0x7f6ba8151590 2024-09-13T17:50:50.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.709+0000 7f6b9effd700 1 --2- 172.21.15.40:0/666253208 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6b88066760 0x7f6b88068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:50.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.709+0000 7f6b9effd700 1 -- 172.21.15.40:0/666253208 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6ba0084e40 con 0x7f6ba8151590 2024-09-13T17:50:50.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.709+0000 7f6ba659c700 1 --2- 172.21.15.40:0/666253208 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6b88066760 0x7f6b88068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:50.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.709+0000 7f6ba659c700 1 --2- 172.21.15.40:0/666253208 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6b88066760 0x7f6b88068c20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009bf0 tx=0x7f6b98009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:50.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.713+0000 7f6b9effd700 1 -- 172.21.15.40:0/666253208 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6ba0051ea0 con 0x7f6ba8151590 2024-09-13T17:50:50.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.881+0000 7f6ba759e700 1 -- 172.21.15.40:0/666253208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ba8154230 con 0x7f6ba8151590 2024-09-13T17:50:50.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.881+0000 7f6b9effd700 1 -- 172.21.15.40:0/666253208 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6ba8154230 con 0x7f6ba8151590 2024-09-13T17:50:50.888 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-09-13T17:50:50.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.881+0000 7f6ba759e700 1 -- 172.21.15.40:0/666253208 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6b88066760 msgr2=0x7f6b88068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:50.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.881+0000 7f6ba759e700 1 --2- 172.21.15.40:0/666253208 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6b88066760 0x7f6b88068c20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009bf0 tx=0x7f6b98009240 comp rx=0 tx=0).stop 2024-09-13T17:50:50.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.881+0000 7f6ba759e700 1 -- 172.21.15.40:0/666253208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba8151590 msgr2=0x7f6ba806b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:50.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.881+0000 7f6ba759e700 1 --2- 172.21.15.40:0/666253208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba8151590 0x7f6ba806b4a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000eed0 tx=0x7f6ba000b390 comp rx=0 tx=0).stop 2024-09-13T17:50:50.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.881+0000 7f6ba759e700 1 -- 172.21.15.40:0/666253208 shutdown_connections 2024-09-13T17:50:50.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.881+0000 7f6ba759e700 1 --2- 172.21.15.40:0/666253208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ba8151590 0x7f6ba806b4a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.881+0000 7f6ba759e700 1 --2- 172.21.15.40:0/666253208 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6b88066760 0x7f6b88068c20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.881+0000 7f6ba759e700 1 --2- 172.21.15.40:0/666253208 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ba80b2d90 0x7f6ba814cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.881+0000 7f6ba759e700 1 --2- 172.21.15.40:0/666253208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba806e7e0 0x7f6ba81eefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:50.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.881+0000 7f6ba759e700 1 -- 172.21.15.40:0/666253208 >> 172.21.15.40:0/666253208 conn(0x7f6ba80ccef0 msgr2=0x7f6ba80c4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:50.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.885+0000 7f6ba759e700 1 -- 172.21.15.40:0/666253208 shutdown_connections 2024-09-13T17:50:50.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:50.885+0000 7f6ba759e700 1 -- 172.21.15.40:0/666253208 wait complete. 2024-09-13T17:50:50.902 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:50:51.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.013+0000 7f5dca427700 1 Processor -- start 2024-09-13T17:50:51.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.013+0000 7f5dca427700 1 -- start start 2024-09-13T17:50:51.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dca427700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5dc40cf580 0x7f5dc415a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:51.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dca427700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5dc40cc250 0x7f5dc40c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:51.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dca427700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc415a7a0 con 0x7f5dc40cf1a0 2024-09-13T17:50:51.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dca427700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc40cbdc0 con 0x7f5dc40cf580 2024-09-13T17:50:51.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dca427700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc4153f80 con 0x7f5dc40cc250 2024-09-13T17:50:51.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dc8c24700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5dc40cf580 0x7f5dc415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dc8c24700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5dc40cf580 0x7f5dc415a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52744/0 (socket says 172.21.15.40:52744) 2024-09-13T17:50:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dc8c24700 1 -- 172.21.15.40:0/1318109043 learned_addr learned my addr 172.21.15.40:0/1318109043 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dc9c26700 1 --2- 172.21.15.40:0/1318109043 >> v2:172.21.15.43:3300/0 conn(0x7f5dc40cc250 0x7f5dc40c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dc8c24700 1 -- 172.21.15.40:0/1318109043 >> v2:172.21.15.43:3300/0 conn(0x7f5dc40cc250 msgr2=0x7f5dc40c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dc8c24700 1 --2- 172.21.15.40:0/1318109043 >> v2:172.21.15.43:3300/0 conn(0x7f5dc40cc250 0x7f5dc40c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dc8c24700 1 -- 172.21.15.40:0/1318109043 >> v1:172.21.15.43:6789/0 conn(0x7f5dc40cf1a0 legacy=0x7f5dc40c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dc8c24700 1 -- 172.21.15.40:0/1318109043 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc40caed0 con 0x7f5dc40cf580 2024-09-13T17:50:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dc8c24700 1 --2- 172.21.15.40:0/1318109043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dc40cf580 0x7f5dc415a260 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4074130 tx=0x7f5db8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda5a867f713abf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5db7fff700 1 -- 172.21.15.40:0/1318109043 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db800b8a0 con 0x7f5dc40cf580 2024-09-13T17:50:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5db7fff700 1 -- 172.21.15.40:0/1318109043 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db800ba00 con 0x7f5dc40cf580 2024-09-13T17:50:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5db7fff700 1 -- 172.21.15.40:0/1318109043 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db800bd90 con 0x7f5dc40cf580 2024-09-13T17:50:51.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dca427700 1 -- 172.21.15.40:0/1318109043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dc40cf580 msgr2=0x7f5dc415a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:51.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dca427700 1 --2- 172.21.15.40:0/1318109043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dc40cf580 0x7f5dc415a260 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4074130 tx=0x7f5db8009bf0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dca427700 1 -- 172.21.15.40:0/1318109043 shutdown_connections 2024-09-13T17:50:51.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dca427700 1 --2- 172.21.15.40:0/1318109043 >> v2:172.21.15.43:3300/0 conn(0x7f5dc40cc250 0x7f5dc40c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dca427700 1 --2- 172.21.15.40:0/1318109043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dc40cf580 0x7f5dc415a260 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dca427700 1 -- 172.21.15.40:0/1318109043 >> 172.21.15.40:0/1318109043 conn(0x7f5dc4066090 msgr2=0x7f5dc4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:51.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dca427700 1 -- 172.21.15.40:0/1318109043 shutdown_connections 2024-09-13T17:50:51.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.017+0000 7f5dca427700 1 -- 172.21.15.40:0/1318109043 wait complete. 2024-09-13T17:50:51.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dca427700 1 Processor -- start 2024-09-13T17:50:51.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dca427700 1 -- start start 2024-09-13T17:50:51.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dca427700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5dc40cc250 0x7f5dc40c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dca427700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dc40cf1a0 0x7f5dc414e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dca427700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5dc40cf580 0x7f5dc414e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dca427700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40b3520 con 0x7f5dc40cf1a0 2024-09-13T17:50:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dca427700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc41f2ac0 con 0x7f5dc40cf580 2024-09-13T17:50:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dca427700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc41f1ef0 con 0x7f5dc40cc250 2024-09-13T17:50:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dc8c24700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dc40cf1a0 0x7f5dc414e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dc8c24700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dc40cf1a0 0x7f5dc414e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52752/0 (socket says 172.21.15.40:52752) 2024-09-13T17:50:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dc8c24700 1 -- 172.21.15.40:0/3940556584 learned_addr learned my addr 172.21.15.40:0/3940556584 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dc9425700 1 --2- 172.21.15.40:0/3940556584 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5dc40cc250 0x7f5dc40c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dc9c26700 1 --2- 172.21.15.40:0/3940556584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5dc40cf580 0x7f5dc414e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dc8c24700 1 -- 172.21.15.40:0/3940556584 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5dc40cc250 msgr2=0x7f5dc40c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dc8c24700 1 --2- 172.21.15.40:0/3940556584 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5dc40cc250 0x7f5dc40c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dc8c24700 1 -- 172.21.15.40:0/3940556584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5dc40cf580 msgr2=0x7f5dc414e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:51.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dc8c24700 1 --2- 172.21.15.40:0/3940556584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5dc40cf580 0x7f5dc414e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dc8c24700 1 -- 172.21.15.40:0/3940556584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db8009570 con 0x7f5dc40cf1a0 2024-09-13T17:50:51.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dc8c24700 1 --2- 172.21.15.40:0/3940556584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dc40cf1a0 0x7f5dc414e2e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5db8000c00 tx=0x7f5db8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:51.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5db5ffb700 1 -- 172.21.15.40:0/3940556584 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db80038c0 con 0x7f5dc40cf1a0 2024-09-13T17:50:51.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5db5ffb700 1 -- 172.21.15.40:0/3940556584 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db801e070 con 0x7f5dc40cf1a0 2024-09-13T17:50:51.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dca427700 1 -- 172.21.15.40:0/3940556584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc4150520 con 0x7f5dc40cf1a0 2024-09-13T17:50:51.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5db5ffb700 1 -- 172.21.15.40:0/3940556584 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8003c80 con 0x7f5dc40cf1a0 2024-09-13T17:50:51.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.021+0000 7f5dca427700 1 -- 172.21.15.40:0/3940556584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc41f2470 con 0x7f5dc40cf1a0 2024-09-13T17:50:51.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.025+0000 7f5db5ffb700 1 -- 172.21.15.40:0/3940556584 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5db8011420 con 0x7f5dc40cf1a0 2024-09-13T17:50:51.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.025+0000 7f5dca427700 1 -- 172.21.15.40:0/3940556584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc4067a40 con 0x7f5dc40cf1a0 2024-09-13T17:50:51.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.025+0000 7f5db5ffb700 1 --2- 172.21.15.40:0/3940556584 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5da8066730 0x7f5da8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:51.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.025+0000 7f5db5ffb700 1 -- 172.21.15.40:0/3940556584 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5db8020070 con 0x7f5dc40cf1a0 2024-09-13T17:50:51.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.025+0000 7f5dc9425700 1 --2- 172.21.15.40:0/3940556584 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5da8066730 0x7f5da8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:51.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.025+0000 7f5dc9425700 1 --2- 172.21.15.40:0/3940556584 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5da8066730 0x7f5da8068bf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00018a0 tx=0x7f5dc0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:51.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.033+0000 7f5db5ffb700 1 -- 172.21.15.40:0/3940556584 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5db8052ae0 con 0x7f5dc40cf1a0 2024-09-13T17:50:51.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.205+0000 7f5dca427700 1 -- 172.21.15.40:0/3940556584 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5dc41f2c30 con 0x7f5dc40cf1a0 2024-09-13T17:50:51.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.205+0000 7f5db5ffb700 1 -- 172.21.15.40:0/3940556584 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5db80150a0 con 0x7f5dc40cf1a0 2024-09-13T17:50:51.213 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-09-13T17:50:51.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.209+0000 7f5dca427700 1 -- 172.21.15.40:0/3940556584 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5da8066730 msgr2=0x7f5da8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:51.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.209+0000 7f5dca427700 1 --2- 172.21.15.40:0/3940556584 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5da8066730 0x7f5da8068bf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00018a0 tx=0x7f5dc0009040 comp rx=0 tx=0).stop 2024-09-13T17:50:51.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.209+0000 7f5dca427700 1 -- 172.21.15.40:0/3940556584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dc40cf1a0 msgr2=0x7f5dc414e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:51.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.209+0000 7f5dca427700 1 --2- 172.21.15.40:0/3940556584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dc40cf1a0 0x7f5dc414e2e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5db8000c00 tx=0x7f5db8004770 comp rx=0 tx=0).stop 2024-09-13T17:50:51.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.209+0000 7f5dca427700 1 -- 172.21.15.40:0/3940556584 shutdown_connections 2024-09-13T17:50:51.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.209+0000 7f5dca427700 1 --2- 172.21.15.40:0/3940556584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5dc40cf580 0x7f5dc414e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.209+0000 7f5dca427700 1 --2- 172.21.15.40:0/3940556584 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dc40cf1a0 0x7f5dc414e2e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.209+0000 7f5dca427700 1 --2- 172.21.15.40:0/3940556584 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5da8066730 0x7f5da8068bf0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.209+0000 7f5dca427700 1 --2- 172.21.15.40:0/3940556584 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5dc40cc250 0x7f5dc40c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.209+0000 7f5dca427700 1 -- 172.21.15.40:0/3940556584 >> 172.21.15.40:0/3940556584 conn(0x7f5dc4066090 msgr2=0x7f5dc40ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:51.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.209+0000 7f5dca427700 1 -- 172.21.15.40:0/3940556584 shutdown_connections 2024-09-13T17:50:51.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.209+0000 7f5dca427700 1 -- 172.21.15.40:0/3940556584 wait complete. 2024-09-13T17:50:51.228 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:50:51.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.337+0000 7fb17eb90700 1 Processor -- start 2024-09-13T17:50:51.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.337+0000 7fb17eb90700 1 -- start start 2024-09-13T17:50:51.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.341+0000 7fb17eb90700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb178066090 0x7fb1780c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:51.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.341+0000 7fb17eb90700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1780c2d80 con 0x7fb1780c2950 2024-09-13T17:50:51.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.341+0000 7fb17eb90700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1780c51b0 con 0x7fb1781513a0 2024-09-13T17:50:51.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.341+0000 7fb17eb90700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1780c6690 con 0x7fb178066090 2024-09-13T17:50:51.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.341+0000 7fb17e38f700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb1780c2950 0x7fb178161bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56128/0 (socket says 172.21.15.40:56128) 2024-09-13T17:50:51.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.341+0000 7fb17e38f700 1 -- 172.21.15.40:0/2000504007 learned_addr learned my addr 172.21.15.40:0/2000504007 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:51.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.341+0000 7fb17d38d700 1 --2- 172.21.15.40:0/2000504007 >> v2:172.21.15.43:3300/0 conn(0x7fb178066090 0x7fb1780c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:51.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.341+0000 7fb17cb8c700 1 -- 172.21.15.40:0/2000504007 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2024356020 0 0) 0x7fb1780c2d80 con 0x7fb1780c2950 2024-09-13T17:50:51.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.341+0000 7fb17cb8c700 1 -- 172.21.15.40:0/2000504007 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1640035d0 con 0x7fb1780c2950 2024-09-13T17:50:51.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.341+0000 7fb17d38d700 1 -- 172.21.15.40:0/2000504007 >> v1:172.21.15.43:6789/0 conn(0x7fb1781513a0 legacy=0x7fb1780c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:51.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.341+0000 7fb17d38d700 1 -- 172.21.15.40:0/2000504007 >> v1:172.21.15.40:6789/0 conn(0x7fb1780c2950 legacy=0x7fb178161bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:51.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.341+0000 7fb17d38d700 1 -- 172.21.15.40:0/2000504007 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1780c5470 con 0x7fb178066090 2024-09-13T17:50:51.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.341+0000 7fb17d38d700 1 --2- 172.21.15.40:0/2000504007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb178066090 0x7fb1780c6ab0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb168009850 tx=0x7fb168009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=627bfe16eadea344 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:51.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.341+0000 7fb17cb8c700 1 -- 172.21.15.40:0/2000504007 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168011830 con 0x7fb178066090 2024-09-13T17:50:51.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.341+0000 7fb17cb8c700 1 -- 172.21.15.40:0/2000504007 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb168011990 con 0x7fb178066090 2024-09-13T17:50:51.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.341+0000 7fb17cb8c700 1 -- 172.21.15.40:0/2000504007 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168011d00 con 0x7fb178066090 2024-09-13T17:50:51.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.345+0000 7fb17eb90700 1 -- 172.21.15.40:0/2000504007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb178066090 msgr2=0x7fb1780c6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:51.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.345+0000 7fb17eb90700 1 --2- 172.21.15.40:0/2000504007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb178066090 0x7fb1780c6ab0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb168009850 tx=0x7fb168009b60 comp rx=0 tx=0).stop 2024-09-13T17:50:51.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.345+0000 7fb17eb90700 1 -- 172.21.15.40:0/2000504007 shutdown_connections 2024-09-13T17:50:51.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.345+0000 7fb17eb90700 1 --2- 172.21.15.40:0/2000504007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb178066090 0x7fb1780c6ab0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.345+0000 7fb17eb90700 1 -- 172.21.15.40:0/2000504007 >> 172.21.15.40:0/2000504007 conn(0x7fb1780cb450 msgr2=0x7fb1780cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:51.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.345+0000 7fb17eb90700 1 -- 172.21.15.40:0/2000504007 shutdown_connections 2024-09-13T17:50:51.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.345+0000 7fb17eb90700 1 -- 172.21.15.40:0/2000504007 wait complete. 2024-09-13T17:50:51.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.345+0000 7fb17eb90700 1 Processor -- start 2024-09-13T17:50:51.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.345+0000 7fb17eb90700 1 -- start start 2024-09-13T17:50:51.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb17eb90700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb178066090 0x7fb178151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:51.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb17eb90700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb1780c2950 0x7fb1780712e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:51.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb17eb90700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1781513a0 0x7fb1781e1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:51.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb17eb90700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1781e3000 con 0x7fb1781513a0 2024-09-13T17:50:51.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb17eb90700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1781ef4f0 con 0x7fb178066090 2024-09-13T17:50:51.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb17eb90700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb178071820 con 0x7fb1780c2950 2024-09-13T17:50:51.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb17e38f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1781513a0 0x7fb1781e1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:51.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb17e38f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1781513a0 0x7fb1781e1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49532/0 (socket says 172.21.15.40:49532) 2024-09-13T17:50:51.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb17e38f700 1 -- 172.21.15.40:0/2685383995 learned_addr learned my addr 172.21.15.40:0/2685383995 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:51.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb17d38d700 1 --2- 172.21.15.40:0/2685383995 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb1780c2950 0x7fb1780712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:51.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb17db8e700 1 --2- 172.21.15.40:0/2685383995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb178066090 0x7fb178151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:51.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb17e38f700 1 -- 172.21.15.40:0/2685383995 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb1780c2950 msgr2=0x7fb1780712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:51.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb17e38f700 1 --2- 172.21.15.40:0/2685383995 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb1780c2950 0x7fb1780712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb17e38f700 1 -- 172.21.15.40:0/2685383995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb178066090 msgr2=0x7fb178151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:51.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb17e38f700 1 --2- 172.21.15.40:0/2685383995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb178066090 0x7fb178151b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb17e38f700 1 -- 172.21.15.40:0/2685383995 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1680094b0 con 0x7fb1781513a0 2024-09-13T17:50:51.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb17e38f700 1 --2- 172.21.15.40:0/2685383995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1781513a0 0x7fb1781e1a40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb17000cba0 tx=0x7fb17000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:51.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb16e7fc700 1 -- 172.21.15.40:0/2685383995 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb170007480 con 0x7fb1781513a0 2024-09-13T17:50:51.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb17eb90700 1 -- 172.21.15.40:0/2685383995 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1781e2a40 con 0x7fb1781513a0 2024-09-13T17:50:51.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb16e7fc700 1 -- 172.21.15.40:0/2685383995 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1700075e0 con 0x7fb1781513a0 2024-09-13T17:50:51.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb16e7fc700 1 -- 172.21.15.40:0/2685383995 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb170007970 con 0x7fb1781513a0 2024-09-13T17:50:51.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.349+0000 7fb17eb90700 1 -- 172.21.15.40:0/2685383995 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1781eeea0 con 0x7fb1781513a0 2024-09-13T17:50:51.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.353+0000 7fb16e7fc700 1 -- 172.21.15.40:0/2685383995 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb17001b3b0 con 0x7fb1781513a0 2024-09-13T17:50:51.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.353+0000 7fb17eb90700 1 -- 172.21.15.40:0/2685383995 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1781eed00 con 0x7fb1781513a0 2024-09-13T17:50:51.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.353+0000 7fb16e7fc700 1 --2- 172.21.15.40:0/2685383995 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb15c066730 0x7fb15c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:51.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.353+0000 7fb17db8e700 1 --2- 172.21.15.40:0/2685383995 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb15c066730 0x7fb15c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:51.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.353+0000 7fb16e7fc700 1 -- 172.21.15.40:0/2685383995 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb170083fb0 con 0x7fb1781513a0 2024-09-13T17:50:51.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.353+0000 7fb17db8e700 1 --2- 172.21.15.40:0/2685383995 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb15c066730 0x7fb15c068bf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb174000f60 tx=0x7fb174009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:51.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.361+0000 7fb16e7fc700 1 -- 172.21.15.40:0/2685383995 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb1781eed00 con 0x7fb1781513a0 2024-09-13T17:50:51.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.529+0000 7fb17eb90700 1 -- 172.21.15.40:0/2685383995 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb1781eed00 con 0x7fb1781513a0 2024-09-13T17:50:51.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.533+0000 7fb16e7fc700 1 -- 172.21.15.40:0/2685383995 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb1781eed00 con 0x7fb1781513a0 2024-09-13T17:50:51.538 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-09-13T17:50:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.533+0000 7fb17eb90700 1 -- 172.21.15.40:0/2685383995 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb15c066730 msgr2=0x7fb15c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.533+0000 7fb17eb90700 1 --2- 172.21.15.40:0/2685383995 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb15c066730 0x7fb15c068bf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb174000f60 tx=0x7fb174009040 comp rx=0 tx=0).stop 2024-09-13T17:50:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.533+0000 7fb17eb90700 1 -- 172.21.15.40:0/2685383995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1781513a0 msgr2=0x7fb1781e1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.533+0000 7fb17eb90700 1 --2- 172.21.15.40:0/2685383995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1781513a0 0x7fb1781e1a40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb17000cba0 tx=0x7fb17000cbd0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.533+0000 7fb17eb90700 1 -- 172.21.15.40:0/2685383995 shutdown_connections 2024-09-13T17:50:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.533+0000 7fb17eb90700 1 --2- 172.21.15.40:0/2685383995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1781513a0 0x7fb1781e1a40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.533+0000 7fb17eb90700 1 --2- 172.21.15.40:0/2685383995 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb15c066730 0x7fb15c068bf0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.533+0000 7fb17eb90700 1 --2- 172.21.15.40:0/2685383995 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb1780c2950 0x7fb1780712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.533+0000 7fb17eb90700 1 --2- 172.21.15.40:0/2685383995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb178066090 0x7fb178151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.533+0000 7fb17eb90700 1 -- 172.21.15.40:0/2685383995 >> 172.21.15.40:0/2685383995 conn(0x7fb1780cb450 msgr2=0x7fb178145c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:51.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.533+0000 7fb17eb90700 1 -- 172.21.15.40:0/2685383995 shutdown_connections 2024-09-13T17:50:51.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.533+0000 7fb17eb90700 1 -- 172.21.15.40:0/2685383995 wait complete. 2024-09-13T17:50:51.553 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:50:51.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.665+0000 7f871b59e700 1 Processor -- start 2024-09-13T17:50:51.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.665+0000 7f871b59e700 1 -- start start 2024-09-13T17:50:51.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.665+0000 7f871b59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f871c150240 0x7f871c071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:51.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.665+0000 7f871b59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f871c0c54e0 0x7f871c15a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:51.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.665+0000 7f871b59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871c0706f0 con 0x7f871c0c4f90 2024-09-13T17:50:51.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.665+0000 7f871b59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f871c0c28c0 con 0x7f871c150240 2024-09-13T17:50:51.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.665+0000 7f871b59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871c070450 con 0x7f871c0c54e0 2024-09-13T17:50:51.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.665+0000 7f871a59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f871c150240 0x7f871c071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:51.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f871ad9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f871c0c4f90 0x7f871c070ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56136/0 (socket says 172.21.15.40:56136) 2024-09-13T17:50:51.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f871ad9d700 1 -- 172.21.15.40:0/4287441793 learned_addr learned my addr 172.21.15.40:0/4287441793 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:51.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f8719d9b700 1 --2- 172.21.15.40:0/4287441793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f871c0c54e0 0x7f871c15a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:51.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f871959a700 1 -- 172.21.15.40:0/4287441793 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1621480266 0 0) 0x7f871c0706f0 con 0x7f871c0c4f90 2024-09-13T17:50:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f871959a700 1 -- 172.21.15.40:0/4287441793 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87040035d0 con 0x7f871c0c4f90 2024-09-13T17:50:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f8719d9b700 1 -- 172.21.15.40:0/4287441793 >> v2:172.21.15.40:3300/0 conn(0x7f871c150240 msgr2=0x7f871c071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f8719d9b700 1 --2- 172.21.15.40:0/4287441793 >> v2:172.21.15.40:3300/0 conn(0x7f871c150240 0x7f871c071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f8719d9b700 1 -- 172.21.15.40:0/4287441793 >> v1:172.21.15.40:6789/0 conn(0x7f871c0c4f90 legacy=0x7f871c070ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f8719d9b700 1 -- 172.21.15.40:0/4287441793 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c156c00 con 0x7f871c0c54e0 2024-09-13T17:50:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f871a59c700 1 --2- 172.21.15.40:0/4287441793 >> v2:172.21.15.40:3300/0 conn(0x7f871c150240 0x7f871c071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:50:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f8719d9b700 1 --2- 172.21.15.40:0/4287441793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f871c0c54e0 0x7f871c15a0e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8710009850 tx=0x7f8710009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6f046c1b5ad3774 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f871959a700 1 -- 172.21.15.40:0/4287441793 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710011830 con 0x7f871c0c54e0 2024-09-13T17:50:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f871959a700 1 -- 172.21.15.40:0/4287441793 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8710011990 con 0x7f871c0c54e0 2024-09-13T17:50:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f871959a700 1 -- 172.21.15.40:0/4287441793 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710011d40 con 0x7f871c0c54e0 2024-09-13T17:50:51.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f871b59e700 1 -- 172.21.15.40:0/4287441793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f871c0c54e0 msgr2=0x7f871c15a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:51.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f871b59e700 1 --2- 172.21.15.40:0/4287441793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f871c0c54e0 0x7f871c15a0e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8710009850 tx=0x7f8710009b60 comp rx=0 tx=0).stop 2024-09-13T17:50:51.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f871b59e700 1 -- 172.21.15.40:0/4287441793 shutdown_connections 2024-09-13T17:50:51.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f871b59e700 1 --2- 172.21.15.40:0/4287441793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f871c0c54e0 0x7f871c15a0e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f871b59e700 1 --2- 172.21.15.40:0/4287441793 >> v2:172.21.15.40:3300/0 conn(0x7f871c150240 0x7f871c071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f871b59e700 1 -- 172.21.15.40:0/4287441793 >> 172.21.15.40:0/4287441793 conn(0x7f871c06a390 msgr2=0x7f871c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:51.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f871b59e700 1 -- 172.21.15.40:0/4287441793 shutdown_connections 2024-09-13T17:50:51.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f871b59e700 1 -- 172.21.15.40:0/4287441793 wait complete. 2024-09-13T17:50:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f871b59e700 1 Processor -- start 2024-09-13T17:50:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.669+0000 7f871b59e700 1 -- start start 2024-09-13T17:50:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f871b59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f871c0c4f90 0x7f871c1e88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:51.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f871b59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f871c0c54e0 0x7f871c1604c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f871b59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f871c150240 0x7f871c160a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f871b59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c1ef1e0 con 0x7f871c0c4f90 2024-09-13T17:50:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f871b59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c1efa40 con 0x7f871c0c54e0 2024-09-13T17:50:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f871b59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871c15ef80 con 0x7f871c150240 2024-09-13T17:50:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f871a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f871c0c4f90 0x7f871c1e88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f871a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f871c0c4f90 0x7f871c1e88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49560/0 (socket says 172.21.15.40:49560) 2024-09-13T17:50:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f871a59c700 1 -- 172.21.15.40:0/51079556 learned_addr learned my addr 172.21.15.40:0/51079556 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f8719d9b700 1 --2- 172.21.15.40:0/51079556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f871c0c54e0 0x7f871c1604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:51.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f871ad9d700 1 --2- 172.21.15.40:0/51079556 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f871c150240 0x7f871c160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:51.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f871a59c700 1 -- 172.21.15.40:0/51079556 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f871c150240 msgr2=0x7f871c160a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:51.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f871a59c700 1 --2- 172.21.15.40:0/51079556 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f871c150240 0x7f871c160a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f871a59c700 1 -- 172.21.15.40:0/51079556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f871c0c54e0 msgr2=0x7f871c1604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:51.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f871a59c700 1 --2- 172.21.15.40:0/51079556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f871c0c54e0 0x7f871c1604c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f871a59c700 1 -- 172.21.15.40:0/51079556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870c0094b0 con 0x7f871c0c4f90 2024-09-13T17:50:51.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f871ad9d700 1 --2- 172.21.15.40:0/51079556 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f871c150240 0x7f871c160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:50:51.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f871a59c700 1 --2- 172.21.15.40:0/51079556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f871c0c4f90 0x7f871c1e88c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f870c00ec10 tx=0x7f870c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:51.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f870affd700 1 -- 172.21.15.40:0/51079556 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c00c890 con 0x7f871c0c4f90 2024-09-13T17:50:51.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f870affd700 1 -- 172.21.15.40:0/51079556 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870c00c9f0 con 0x7f871c0c4f90 2024-09-13T17:50:51.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f871b59e700 1 -- 172.21.15.40:0/51079556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87100094b0 con 0x7f871c0c4f90 2024-09-13T17:50:51.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f870affd700 1 -- 172.21.15.40:0/51079556 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c00cd60 con 0x7f871c0c4f90 2024-09-13T17:50:51.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.673+0000 7f871b59e700 1 -- 172.21.15.40:0/51079556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871c15eca0 con 0x7f871c0c4f90 2024-09-13T17:50:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.677+0000 7f871b59e700 1 -- 172.21.15.40:0/51079556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871c0c24e0 con 0x7f871c0c4f90 2024-09-13T17:50:51.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.677+0000 7f870affd700 1 -- 172.21.15.40:0/51079556 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f870c013470 con 0x7f871c0c4f90 2024-09-13T17:50:51.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.677+0000 7f870affd700 1 --2- 172.21.15.40:0/51079556 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86fc066730 0x7f86fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:51.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.677+0000 7f870affd700 1 -- 172.21.15.40:0/51079556 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f870c0849e0 con 0x7f871c0c4f90 2024-09-13T17:50:51.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.677+0000 7f8719d9b700 1 --2- 172.21.15.40:0/51079556 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86fc066730 0x7f86fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:51.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.677+0000 7f8719d9b700 1 --2- 172.21.15.40:0/51079556 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86fc066730 0x7f86fc068bf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f87100090b0 tx=0x7f871001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:51.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.685+0000 7f870affd700 1 -- 172.21.15.40:0/51079556 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f870c0519c0 con 0x7f871c0c4f90 2024-09-13T17:50:51.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.873+0000 7f871b59e700 1 -- 172.21.15.40:0/51079556 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f871c150620 con 0x7f871c0c4f90 2024-09-13T17:50:51.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.873+0000 7f870affd700 1 -- 172.21.15.40:0/51079556 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f870c017080 con 0x7f871c0c4f90 2024-09-13T17:50:51.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.877+0000 7f871b59e700 1 -- 172.21.15.40:0/51079556 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86fc066730 msgr2=0x7f86fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:51.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.877+0000 7f871b59e700 1 --2- 172.21.15.40:0/51079556 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86fc066730 0x7f86fc068bf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f87100090b0 tx=0x7f871001c040 comp rx=0 tx=0).stop 2024-09-13T17:50:51.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.877+0000 7f871b59e700 1 -- 172.21.15.40:0/51079556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f871c0c4f90 msgr2=0x7f871c1e88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:51.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.877+0000 7f871b59e700 1 --2- 172.21.15.40:0/51079556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f871c0c4f90 0x7f871c1e88c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f870c00ec10 tx=0x7f870c00ef20 comp rx=0 tx=0).stop 2024-09-13T17:50:51.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.877+0000 7f871b59e700 1 -- 172.21.15.40:0/51079556 shutdown_connections 2024-09-13T17:50:51.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.877+0000 7f871b59e700 1 --2- 172.21.15.40:0/51079556 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86fc066730 0x7f86fc068bf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.877+0000 7f871b59e700 1 --2- 172.21.15.40:0/51079556 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f871c150240 0x7f871c160a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.877+0000 7f871b59e700 1 --2- 172.21.15.40:0/51079556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f871c0c54e0 0x7f871c1604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.877+0000 7f871b59e700 1 --2- 172.21.15.40:0/51079556 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f871c0c4f90 0x7f871c1e88c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:51.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.877+0000 7f871b59e700 1 -- 172.21.15.40:0/51079556 >> 172.21.15.40:0/51079556 conn(0x7f871c06a390 msgr2=0x7f871c14ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:51.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.877+0000 7f871b59e700 1 -- 172.21.15.40:0/51079556 shutdown_connections 2024-09-13T17:50:51.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:51.877+0000 7f871b59e700 1 -- 172.21.15.40:0/51079556 wait complete. 2024-09-13T17:50:51.886 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-09-13T17:50:51.896 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi040.front.sepia.ceph.com... 2024-09-13T17:50:51.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:50:51.897 DEBUG:teuthology.orchestra.run.smithi040:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-13T17:50:51.923 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi040.stderr:ceph-fuse[33014]: fuse finished with error 0 and tester_r 0 2024-09-13T17:50:51.925 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:50:58.027 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-09-13T17:50:58.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:50:58.028 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-13T17:50:58.046 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:50:58.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.206+0000 7fd1bffff700 1 Processor -- start 2024-09-13T17:50:58.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.206+0000 7fd1bffff700 1 -- start start 2024-09-13T17:50:58.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.206+0000 7fd1bffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1c01500c0 0x7fd1c00c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:58.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.206+0000 7fd1bffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd1c0072250 0x7fd1c00c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:58.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.206+0000 7fd1bffff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c00cba20 con 0x7fd1c0070500 2024-09-13T17:50:58.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.206+0000 7fd1bffff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c01560e0 con 0x7fd1c0072250 2024-09-13T17:50:58.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.206+0000 7fd1bffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c01563c0 con 0x7fd1c01500c0 2024-09-13T17:50:58.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.206+0000 7fd1b7fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd1c0070500 0x7fd1c0071b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56152/0 (socket says 172.21.15.40:56152) 2024-09-13T17:50:58.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.206+0000 7fd1b7fff700 1 -- 172.21.15.40:0/258080826 learned_addr learned my addr 172.21.15.40:0/258080826 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:58.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.206+0000 7fd1bf7fe700 1 --2- 172.21.15.40:0/258080826 >> v2:172.21.15.40:3300/0 conn(0x7fd1c0072250 0x7fd1c00c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:58.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.206+0000 7fd1beffd700 1 --2- 172.21.15.40:0/258080826 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1c01500c0 0x7fd1c00c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:58.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.206+0000 7fd1beffd700 1 -- 172.21.15.40:0/258080826 >> v2:172.21.15.40:3300/0 conn(0x7fd1c0072250 msgr2=0x7fd1c00c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:58.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.206+0000 7fd1beffd700 1 --2- 172.21.15.40:0/258080826 >> v2:172.21.15.40:3300/0 conn(0x7fd1c0072250 0x7fd1c00c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.206+0000 7fd1beffd700 1 -- 172.21.15.40:0/258080826 >> v1:172.21.15.40:6789/0 conn(0x7fd1c0070500 legacy=0x7fd1c0071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:58.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.206+0000 7fd1beffd700 1 -- 172.21.15.40:0/258080826 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c01566a0 con 0x7fd1c01500c0 2024-09-13T17:50:58.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.206+0000 7fd1beffd700 1 --2- 172.21.15.40:0/258080826 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1c01500c0 0x7fd1c00c66d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009850 tx=0x7fd1b0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f444fd25697f2a09 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:58.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.206+0000 7fd1be7fc700 1 -- 172.21.15.40:0/258080826 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0011830 con 0x7fd1c01500c0 2024-09-13T17:50:58.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.210+0000 7fd1be7fc700 1 -- 172.21.15.40:0/258080826 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b0011990 con 0x7fd1c01500c0 2024-09-13T17:50:58.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.210+0000 7fd1be7fc700 1 -- 172.21.15.40:0/258080826 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0011d40 con 0x7fd1c01500c0 2024-09-13T17:50:58.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.210+0000 7fd1bffff700 1 -- 172.21.15.40:0/258080826 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1c01500c0 msgr2=0x7fd1c00c66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:58.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.210+0000 7fd1bffff700 1 --2- 172.21.15.40:0/258080826 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1c01500c0 0x7fd1c00c66d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009850 tx=0x7fd1b0009b60 comp rx=0 tx=0).stop 2024-09-13T17:50:58.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.210+0000 7fd1bffff700 1 -- 172.21.15.40:0/258080826 shutdown_connections 2024-09-13T17:50:58.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.210+0000 7fd1bffff700 1 --2- 172.21.15.40:0/258080826 >> v2:172.21.15.40:3300/0 conn(0x7fd1c0072250 0x7fd1c00c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.210+0000 7fd1bffff700 1 --2- 172.21.15.40:0/258080826 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1c01500c0 0x7fd1c00c66d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.210+0000 7fd1bffff700 1 -- 172.21.15.40:0/258080826 >> 172.21.15.40:0/258080826 conn(0x7fd1c00cabc0 msgr2=0x7fd1c00cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:58.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.210+0000 7fd1bffff700 1 -- 172.21.15.40:0/258080826 shutdown_connections 2024-09-13T17:50:58.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.210+0000 7fd1bffff700 1 -- 172.21.15.40:0/258080826 wait complete. 2024-09-13T17:50:58.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.210+0000 7fd1bffff700 1 Processor -- start 2024-09-13T17:50:58.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.210+0000 7fd1bffff700 1 -- start start 2024-09-13T17:50:58.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1bffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1c0070500 0x7fd1c00c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:58.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1bffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1c0072250 0x7fd1c01ed270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:58.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1bffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1c01500c0 0x7fd1c0142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:58.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1bffff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c01ece50 con 0x7fd1c0072250 2024-09-13T17:50:58.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1bffff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c0147420 con 0x7fd1c0070500 2024-09-13T17:50:58.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1bffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c014c430 con 0x7fd1c01500c0 2024-09-13T17:50:58.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1b7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1c0072250 0x7fd1c01ed270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:58.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1beffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1c0070500 0x7fd1c00c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:58.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1b7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1c0072250 0x7fd1c01ed270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49578/0 (socket says 172.21.15.40:49578) 2024-09-13T17:50:58.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1bf7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1c01500c0 0x7fd1c0142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:58.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1b7fff700 1 -- 172.21.15.40:0/1941462016 learned_addr learned my addr 172.21.15.40:0/1941462016 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:58.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1b7fff700 1 -- 172.21.15.40:0/1941462016 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1c01500c0 msgr2=0x7fd1c0142470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:58.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1b7fff700 1 --2- 172.21.15.40:0/1941462016 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1c01500c0 0x7fd1c0142470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1b7fff700 1 -- 172.21.15.40:0/1941462016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1c0070500 msgr2=0x7fd1c00c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:58.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1b7fff700 1 --2- 172.21.15.40:0/1941462016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1c0070500 0x7fd1c00c77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1b7fff700 1 -- 172.21.15.40:0/1941462016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b00094b0 con 0x7fd1c0072250 2024-09-13T17:50:58.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1b7fff700 1 --2- 172.21.15.40:0/1941462016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1c0072250 0x7fd1c01ed270 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800c500 tx=0x7fd1a800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:58.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1b77fe700 1 -- 172.21.15.40:0/1941462016 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8005d90 con 0x7fd1c0072250 2024-09-13T17:50:58.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1b77fe700 1 -- 172.21.15.40:0/1941462016 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a800ce50 con 0x7fd1c0072250 2024-09-13T17:50:58.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1bffff700 1 -- 172.21.15.40:0/1941462016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c00c48a0 con 0x7fd1c0072250 2024-09-13T17:50:58.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1b77fe700 1 -- 172.21.15.40:0/1941462016 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80072f0 con 0x7fd1c0072250 2024-09-13T17:50:58.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.214+0000 7fd1bffff700 1 -- 172.21.15.40:0/1941462016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c014bb90 con 0x7fd1c0072250 2024-09-13T17:50:58.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.218+0000 7fd1bffff700 1 -- 172.21.15.40:0/1941462016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c00c7360 con 0x7fd1c0072250 2024-09-13T17:50:58.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.218+0000 7fd1b77fe700 1 -- 172.21.15.40:0/1941462016 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd1a8007450 con 0x7fd1c0072250 2024-09-13T17:50:58.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.218+0000 7fd1b77fe700 1 --2- 172.21.15.40:0/1941462016 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd1a00667b0 0x7fd1a0068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:58.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.218+0000 7fd1b77fe700 1 -- 172.21.15.40:0/1941462016 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd1a8083600 con 0x7fd1c0072250 2024-09-13T17:50:58.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.218+0000 7fd1beffd700 1 --2- 172.21.15.40:0/1941462016 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd1a00667b0 0x7fd1a0068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:58.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.218+0000 7fd1beffd700 1 --2- 172.21.15.40:0/1941462016 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd1a00667b0 0x7fd1a0068c70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd1b00090b0 tx=0x7fd1b001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:58.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.226+0000 7fd1b77fe700 1 -- 172.21.15.40:0/1941462016 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd1a80507f0 con 0x7fd1c0072250 2024-09-13T17:50:58.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.398+0000 7fd1bffff700 1 -- 172.21.15.40:0/1941462016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1c00c3180 con 0x7fd1c0072250 2024-09-13T17:50:58.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.402+0000 7fd1b77fe700 1 -- 172.21.15.40:0/1941462016 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18262 (secure 0 0 0) 0x7fd1a8053e10 con 0x7fd1c0072250 2024-09-13T17:50:58.408 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:50:58.409 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:50:38.974071+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:50:32.740336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:50:33.552914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:50:36.623639+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:50:58.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.406+0000 7fd1bffff700 1 -- 172.21.15.40:0/1941462016 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd1a00667b0 msgr2=0x7fd1a0068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:58.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.406+0000 7fd1bffff700 1 --2- 172.21.15.40:0/1941462016 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd1a00667b0 0x7fd1a0068c70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd1b00090b0 tx=0x7fd1b001c040 comp rx=0 tx=0).stop 2024-09-13T17:50:58.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.406+0000 7fd1bffff700 1 -- 172.21.15.40:0/1941462016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1c0072250 msgr2=0x7fd1c01ed270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:58.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.406+0000 7fd1bffff700 1 --2- 172.21.15.40:0/1941462016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1c0072250 0x7fd1c01ed270 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800c500 tx=0x7fd1a800c810 comp rx=0 tx=0).stop 2024-09-13T17:50:58.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.406+0000 7fd1bffff700 1 -- 172.21.15.40:0/1941462016 shutdown_connections 2024-09-13T17:50:58.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.406+0000 7fd1bffff700 1 --2- 172.21.15.40:0/1941462016 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd1a00667b0 0x7fd1a0068c70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.406+0000 7fd1bffff700 1 --2- 172.21.15.40:0/1941462016 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1c01500c0 0x7fd1c0142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.406+0000 7fd1bffff700 1 --2- 172.21.15.40:0/1941462016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1c0072250 0x7fd1c01ed270 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.406+0000 7fd1bffff700 1 --2- 172.21.15.40:0/1941462016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1c0070500 0x7fd1c00c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.406+0000 7fd1bffff700 1 -- 172.21.15.40:0/1941462016 >> 172.21.15.40:0/1941462016 conn(0x7fd1c00cabc0 msgr2=0x7fd1c014ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:58.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.406+0000 7fd1bffff700 1 -- 172.21.15.40:0/1941462016 shutdown_connections 2024-09-13T17:50:58.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.406+0000 7fd1bffff700 1 -- 172.21.15.40:0/1941462016 wait complete. 2024-09-13T17:50:58.424 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:50:58.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc30ceaf700 1 Processor -- start 2024-09-13T17:50:58.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc30ceaf700 1 -- start start 2024-09-13T17:50:58.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc30ceaf700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc3081515a0 0x7fc30806fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:58.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc30ceaf700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc3080c57d0 0x7fc3080704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:58.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc30ceaf700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc30806d300 0x7fc30806d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:58.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc30ceaf700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3080718a0 con 0x7fc3080c57d0 2024-09-13T17:50:58.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc30ceaf700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc30806e4f0 con 0x7fc3081515a0 2024-09-13T17:50:58.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc30ceaf700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3080c6840 con 0x7fc30806d300 2024-09-13T17:50:58.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc2fdd9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc3080c57d0 0x7fc3080704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:58.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc2fdd9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc3080c57d0 0x7fc3080704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49588/0 (socket says 172.21.15.40:49588) 2024-09-13T17:50:58.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc2fdd9b700 1 -- 172.21.15.40:0/1584820192 learned_addr learned my addr 172.21.15.40:0/1584820192 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:58.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc30659c700 1 --2- 172.21.15.40:0/1584820192 >> v2:172.21.15.43:3300/0 conn(0x7fc3081515a0 0x7fc30806fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:58.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc306d9d700 1 --2- 172.21.15.40:0/1584820192 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc30806d300 0x7fc30806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:58.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc2fdd9b700 1 -- 172.21.15.40:0/1584820192 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc30806d300 msgr2=0x7fc30806d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:58.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc2fdd9b700 1 --2- 172.21.15.40:0/1584820192 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc30806d300 0x7fc30806d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc2fdd9b700 1 -- 172.21.15.40:0/1584820192 >> v2:172.21.15.43:3300/0 conn(0x7fc3081515a0 msgr2=0x7fc30806fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:50:58.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc2fdd9b700 1 --2- 172.21.15.40:0/1584820192 >> v2:172.21.15.43:3300/0 conn(0x7fc3081515a0 0x7fc30806fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc2fdd9b700 1 -- 172.21.15.40:0/1584820192 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3080c5c20 con 0x7fc3080c57d0 2024-09-13T17:50:58.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc30659c700 1 --2- 172.21.15.40:0/1584820192 >> v2:172.21.15.43:3300/0 conn(0x7fc3081515a0 0x7fc30806fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:50:58.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc2fdd9b700 1 --2- 172.21.15.40:0/1584820192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3080c57d0 0x7fc3080704a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc3080c8920 tx=0x7fc2f0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf1273ee4cd53cc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:58.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc305d9b700 1 -- 172.21.15.40:0/1584820192 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000b8a0 con 0x7fc3080c57d0 2024-09-13T17:50:58.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc305d9b700 1 -- 172.21.15.40:0/1584820192 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f000ba00 con 0x7fc3080c57d0 2024-09-13T17:50:58.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc305d9b700 1 -- 172.21.15.40:0/1584820192 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000bd90 con 0x7fc3080c57d0 2024-09-13T17:50:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc30ceaf700 1 -- 172.21.15.40:0/1584820192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3080c57d0 msgr2=0x7fc3080704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc30ceaf700 1 --2- 172.21.15.40:0/1584820192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3080c57d0 0x7fc3080704a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc3080c8920 tx=0x7fc2f0009bf0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc30ceaf700 1 -- 172.21.15.40:0/1584820192 shutdown_connections 2024-09-13T17:50:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc30ceaf700 1 --2- 172.21.15.40:0/1584820192 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc30806d300 0x7fc30806d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc30ceaf700 1 --2- 172.21.15.40:0/1584820192 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3080c57d0 0x7fc3080704a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc30ceaf700 1 --2- 172.21.15.40:0/1584820192 >> v2:172.21.15.43:3300/0 conn(0x7fc3081515a0 0x7fc30806fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc30ceaf700 1 -- 172.21.15.40:0/1584820192 >> 172.21.15.40:0/1584820192 conn(0x7fc3080cadd0 msgr2=0x7fc3080cb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc30ceaf700 1 -- 172.21.15.40:0/1584820192 shutdown_connections 2024-09-13T17:50:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.538+0000 7fc30ceaf700 1 -- 172.21.15.40:0/1584820192 wait complete. 2024-09-13T17:50:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc30ceaf700 1 Processor -- start 2024-09-13T17:50:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc30ceaf700 1 -- start start 2024-09-13T17:50:58.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc30ceaf700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc30806d300 0x7fc30814f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:58.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc30ceaf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3080c57d0 0x7fc3081eec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:58.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc30ceaf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3081515a0 0x7fc3080c2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:58.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc30ceaf700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc308149130 con 0x7fc3081515a0 2024-09-13T17:50:58.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc30ceaf700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30814a0d0 con 0x7fc3080c57d0 2024-09-13T17:50:58.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc30ceaf700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30814aba0 con 0x7fc30806d300 2024-09-13T17:50:58.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc306d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3081515a0 0x7fc3080c2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:58.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc2fdd9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3080c57d0 0x7fc3081eec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:58.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc30659c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc30806d300 0x7fc30814f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:58.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc306d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3081515a0 0x7fc3080c2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49602/0 (socket says 172.21.15.40:49602) 2024-09-13T17:50:58.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc30659c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc30806d300 0x7fc30814f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:59248/0 (socket says 172.21.15.40:59248) 2024-09-13T17:50:58.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc2fdd9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3080c57d0 0x7fc3081eec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:43838/0 (socket says 172.21.15.40:43838) 2024-09-13T17:50:58.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc30659c700 1 -- 172.21.15.40:0/2617524688 learned_addr learned my addr 172.21.15.40:0/2617524688 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:58.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc2fdd9b700 1 -- 172.21.15.40:0/2617524688 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc30806d300 msgr2=0x7fc30814f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:58.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc2fdd9b700 1 --2- 172.21.15.40:0/2617524688 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc30806d300 0x7fc30814f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc2fdd9b700 1 -- 172.21.15.40:0/2617524688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3081515a0 msgr2=0x7fc3080c2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:58.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc2fdd9b700 1 --2- 172.21.15.40:0/2617524688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3081515a0 0x7fc3080c2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc2fdd9b700 1 -- 172.21.15.40:0/2617524688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f0009570 con 0x7fc3080c57d0 2024-09-13T17:50:58.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc306d9d700 1 --2- 172.21.15.40:0/2617524688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3081515a0 0x7fc3080c2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:50:58.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc30659c700 1 --2- 172.21.15.40:0/2617524688 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc30806d300 0x7fc30814f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:50:58.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc2fdd9b700 1 --2- 172.21.15.40:0/2617524688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3080c57d0 0x7fc3081eec00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0016000 tx=0x7fc2f0004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:58.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.542+0000 7fc2ff7fe700 1 -- 172.21.15.40:0/2617524688 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f00045a0 con 0x7fc3080c57d0 2024-09-13T17:50:58.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.546+0000 7fc2ff7fe700 1 -- 172.21.15.40:0/2617524688 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f001f070 con 0x7fc3080c57d0 2024-09-13T17:50:58.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.546+0000 7fc2ff7fe700 1 -- 172.21.15.40:0/2617524688 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0003900 con 0x7fc3080c57d0 2024-09-13T17:50:58.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.546+0000 7fc30ceaf700 1 -- 172.21.15.40:0/2617524688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc308149310 con 0x7fc3080c57d0 2024-09-13T17:50:58.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.546+0000 7fc30ceaf700 1 -- 172.21.15.40:0/2617524688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3081498d0 con 0x7fc3080c57d0 2024-09-13T17:50:58.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.546+0000 7fc30ceaf700 1 -- 172.21.15.40:0/2617524688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3080c6cd0 con 0x7fc3080c57d0 2024-09-13T17:50:58.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.546+0000 7fc2ff7fe700 1 -- 172.21.15.40:0/2617524688 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc2f0021030 con 0x7fc3080c57d0 2024-09-13T17:50:58.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.546+0000 7fc2ff7fe700 1 --2- 172.21.15.40:0/2617524688 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc2e8066760 0x7fc2e8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:58.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.550+0000 7fc30659c700 1 --2- 172.21.15.40:0/2617524688 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc2e8066760 0x7fc2e8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:58.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.550+0000 7fc2ff7fe700 1 -- 172.21.15.40:0/2617524688 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc2f00858d0 con 0x7fc3080c57d0 2024-09-13T17:50:58.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.550+0000 7fc30659c700 1 --2- 172.21.15.40:0/2617524688 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc2e8066760 0x7fc2e8068c20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800ce10 tx=0x7fc2f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:58.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.558+0000 7fc2ff7fe700 1 -- 172.21.15.40:0/2617524688 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc2f0052940 con 0x7fc3080c57d0 2024-09-13T17:50:58.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.738+0000 7fc30ceaf700 1 -- 172.21.15.40:0/2617524688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3080c5e20 con 0x7fc3080c57d0 2024-09-13T17:50:58.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.742+0000 7fc2ff7fe700 1 -- 172.21.15.40:0/2617524688 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18262 (secure 0 0 0) 0x7fc2f0055f60 con 0x7fc3080c57d0 2024-09-13T17:50:58.747 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:50:58.747 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:50:38.974071+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:50:32.740336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:50:33.552914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:50:36.623639+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:50:58.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.742+0000 7fc30ceaf700 1 -- 172.21.15.40:0/2617524688 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc2e8066760 msgr2=0x7fc2e8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.742+0000 7fc30ceaf700 1 --2- 172.21.15.40:0/2617524688 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc2e8066760 0x7fc2e8068c20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800ce10 tx=0x7fc2f8008040 comp rx=0 tx=0).stop 2024-09-13T17:50:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.742+0000 7fc30ceaf700 1 -- 172.21.15.40:0/2617524688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3080c57d0 msgr2=0x7fc3081eec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.742+0000 7fc30ceaf700 1 --2- 172.21.15.40:0/2617524688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3080c57d0 0x7fc3081eec00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0016000 tx=0x7fc2f0004770 comp rx=0 tx=0).stop 2024-09-13T17:50:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.742+0000 7fc30ceaf700 1 -- 172.21.15.40:0/2617524688 shutdown_connections 2024-09-13T17:50:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.742+0000 7fc30ceaf700 1 --2- 172.21.15.40:0/2617524688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3081515a0 0x7fc3080c2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.742+0000 7fc30ceaf700 1 --2- 172.21.15.40:0/2617524688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3080c57d0 0x7fc3081eec00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.742+0000 7fc30ceaf700 1 --2- 172.21.15.40:0/2617524688 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc2e8066760 0x7fc2e8068c20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.742+0000 7fc30ceaf700 1 --2- 172.21.15.40:0/2617524688 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc30806d300 0x7fc30814f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.742+0000 7fc30ceaf700 1 -- 172.21.15.40:0/2617524688 >> 172.21.15.40:0/2617524688 conn(0x7fc3080cadd0 msgr2=0x7fc3080c8060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.742+0000 7fc30ceaf700 1 -- 172.21.15.40:0/2617524688 shutdown_connections 2024-09-13T17:50:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.742+0000 7fc30ceaf700 1 -- 172.21.15.40:0/2617524688 wait complete. 2024-09-13T17:50:58.765 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T17:50:58.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.882+0000 7face1de3700 1 Processor -- start 2024-09-13T17:50:58.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.882+0000 7face1de3700 1 -- start start 2024-09-13T17:50:58.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.882+0000 7face1de3700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7facdc157c30 0x7facdc070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:58.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.882+0000 7face1de3700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facdc15e7d0 con 0x7facdc15f720 2024-09-13T17:50:58.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.882+0000 7face1de3700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facdc1611b0 con 0x7facdc0c5580 2024-09-13T17:50:58.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.882+0000 7face1de3700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facdc072850 con 0x7facdc157c30 2024-09-13T17:50:58.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.882+0000 7face0de1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7facdc157c30 0x7facdc070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:58.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.882+0000 7face15e2700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7facdc15f720 0x7facdc168260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56154/0 (socket says 172.21.15.40:56154) 2024-09-13T17:50:58.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.882+0000 7face15e2700 1 -- 172.21.15.40:0/461276567 learned_addr learned my addr 172.21.15.40:0/461276567 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:50:58.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.882+0000 7facdb7fe700 1 -- 172.21.15.40:0/461276567 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1532240934 0 0) 0x7facdc15e7d0 con 0x7facdc15f720 2024-09-13T17:50:58.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.882+0000 7facdb7fe700 1 -- 172.21.15.40:0/461276567 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facc40035d0 con 0x7facdc15f720 2024-09-13T17:50:58.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.882+0000 7face0de1700 1 -- 172.21.15.40:0/461276567 >> v1:172.21.15.43:6789/0 conn(0x7facdc0c5580 legacy=0x7facdc15f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:58.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.882+0000 7facdb7fe700 1 -- 172.21.15.40:0/461276567 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 16527934 0 0) 0x7facdc1611b0 con 0x7facdc0c5580 2024-09-13T17:50:58.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.882+0000 7face0de1700 1 -- 172.21.15.40:0/461276567 >> v1:172.21.15.40:6789/0 conn(0x7facdc15f720 legacy=0x7facdc168260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:58.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.882+0000 7face0de1700 1 -- 172.21.15.40:0/461276567 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc161490 con 0x7facdc157c30 2024-09-13T17:50:58.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.882+0000 7face0de1700 1 --2- 172.21.15.40:0/461276567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc157c30 0x7facdc070c60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7facd0009850 tx=0x7facd0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b07621457539746 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:58.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.886+0000 7facdb7fe700 1 -- 172.21.15.40:0/461276567 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd0011830 con 0x7facdc157c30 2024-09-13T17:50:58.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.886+0000 7facdb7fe700 1 -- 172.21.15.40:0/461276567 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd0011990 con 0x7facdc157c30 2024-09-13T17:50:58.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.886+0000 7facdb7fe700 1 -- 172.21.15.40:0/461276567 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd0011d00 con 0x7facdc157c30 2024-09-13T17:50:58.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.886+0000 7face1de3700 1 -- 172.21.15.40:0/461276567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc157c30 msgr2=0x7facdc070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:58.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.886+0000 7face1de3700 1 --2- 172.21.15.40:0/461276567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc157c30 0x7facdc070c60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7facd0009850 tx=0x7facd0009b60 comp rx=0 tx=0).stop 2024-09-13T17:50:58.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.886+0000 7face1de3700 1 -- 172.21.15.40:0/461276567 shutdown_connections 2024-09-13T17:50:58.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.886+0000 7face1de3700 1 --2- 172.21.15.40:0/461276567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc157c30 0x7facdc070c60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.886+0000 7face1de3700 1 -- 172.21.15.40:0/461276567 >> 172.21.15.40:0/461276567 conn(0x7facdc06a390 msgr2=0x7facdc06a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:50:58.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.886+0000 7face1de3700 1 -- 172.21.15.40:0/461276567 shutdown_connections 2024-09-13T17:50:58.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.886+0000 7face1de3700 1 -- 172.21.15.40:0/461276567 wait complete. 2024-09-13T17:50:58.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.886+0000 7face1de3700 1 Processor -- start 2024-09-13T17:50:58.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.886+0000 7face1de3700 1 -- start start 2024-09-13T17:50:58.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7face1de3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7facdc0c5580 0x7facdc0c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:58.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7face1de3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc157c30 0x7facdc0c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:58.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7face1de3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facdc15f720 0x7facdc0c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:58.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7face1de3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc0cadd0 con 0x7facdc157c30 2024-09-13T17:50:58.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7face1de3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc153310 con 0x7facdc15f720 2024-09-13T17:50:58.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7face1de3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facdc1535f0 con 0x7facdc0c5580 2024-09-13T17:50:58.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7facdbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc157c30 0x7facdc0c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:58.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7facdbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc157c30 0x7facdc0c91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49624/0 (socket says 172.21.15.40:49624) 2024-09-13T17:50:58.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7face0de1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7facdc0c5580 0x7facdc0c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:58.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7facdbfff700 1 -- 172.21.15.40:0/587226216 learned_addr learned my addr 172.21.15.40:0/587226216 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:50:58.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7face15e2700 1 --2- 172.21.15.40:0/587226216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facdc15f720 0x7facdc0c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:58.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7facdbfff700 1 -- 172.21.15.40:0/587226216 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7facdc0c5580 msgr2=0x7facdc0c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:58.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7facdbfff700 1 --2- 172.21.15.40:0/587226216 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7facdc0c5580 0x7facdc0c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7facdbfff700 1 -- 172.21.15.40:0/587226216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facdc15f720 msgr2=0x7facdc0c96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:50:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7facdbfff700 1 --2- 172.21.15.40:0/587226216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facdc15f720 0x7facdc0c96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:50:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7facdbfff700 1 -- 172.21.15.40:0/587226216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd00094b0 con 0x7facdc157c30 2024-09-13T17:50:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7facdbfff700 1 --2- 172.21.15.40:0/587226216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc157c30 0x7facdc0c91b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faccc00c450 tx=0x7faccc00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7facd97fa700 1 -- 172.21.15.40:0/587226216 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc0054e0 con 0x7facdc157c30 2024-09-13T17:50:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7facd97fa700 1 -- 172.21.15.40:0/587226216 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faccc00ce60 con 0x7facdc157c30 2024-09-13T17:50:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7face1de3700 1 -- 172.21.15.40:0/587226216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facdc1538d0 con 0x7facdc157c30 2024-09-13T17:50:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7facd97fa700 1 -- 172.21.15.40:0/587226216 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc0038e0 con 0x7facdc157c30 2024-09-13T17:50:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.890+0000 7face1de3700 1 -- 172.21.15.40:0/587226216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facdc153e90 con 0x7facdc157c30 2024-09-13T17:50:58.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.894+0000 7face1de3700 1 -- 172.21.15.40:0/587226216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facdc1543b0 con 0x7facdc157c30 2024-09-13T17:50:58.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.894+0000 7facd97fa700 1 -- 172.21.15.40:0/587226216 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faccc003a40 con 0x7facdc157c30 2024-09-13T17:50:58.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.894+0000 7facd97fa700 1 --2- 172.21.15.40:0/587226216 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7facc8066730 0x7facc8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:50:58.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.894+0000 7facd97fa700 1 -- 172.21.15.40:0/587226216 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7faccc083710 con 0x7facdc157c30 2024-09-13T17:50:58.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.894+0000 7face0de1700 1 --2- 172.21.15.40:0/587226216 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7facc8066730 0x7facc8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:50:58.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.894+0000 7face0de1700 1 --2- 172.21.15.40:0/587226216 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7facc8066730 0x7facc8068bf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7facd00090b0 tx=0x7facd0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:50:58.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:58.902+0000 7facd97fa700 1 -- 172.21.15.40:0/587226216 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7facdc1543b0 con 0x7facdc157c30 2024-09-13T17:50:59.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:50:59.114+0000 7face1de3700 1 -- 172.21.15.40:0/587226216 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7facdc1543b0 con 0x7facdc157c30 2024-09-13T17:51:00.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.110+0000 7facd97fa700 1 -- 172.21.15.40:0/587226216 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7facdc1543b0 con 0x7facdc157c30 2024-09-13T17:51:00.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.114+0000 7face1de3700 1 -- 172.21.15.40:0/587226216 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7facc8066730 msgr2=0x7facc8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:00.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.114+0000 7face1de3700 1 --2- 172.21.15.40:0/587226216 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7facc8066730 0x7facc8068bf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7facd00090b0 tx=0x7facd0008030 comp rx=0 tx=0).stop 2024-09-13T17:51:00.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.114+0000 7face1de3700 1 -- 172.21.15.40:0/587226216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc157c30 msgr2=0x7facdc0c91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:00.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.114+0000 7face1de3700 1 --2- 172.21.15.40:0/587226216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc157c30 0x7facdc0c91b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faccc00c450 tx=0x7faccc00c760 comp rx=0 tx=0).stop 2024-09-13T17:51:00.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.118+0000 7face1de3700 1 -- 172.21.15.40:0/587226216 shutdown_connections 2024-09-13T17:51:00.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.118+0000 7face1de3700 1 --2- 172.21.15.40:0/587226216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facdc15f720 0x7facdc0c96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:00.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.118+0000 7face1de3700 1 --2- 172.21.15.40:0/587226216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facdc157c30 0x7facdc0c91b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:00.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.118+0000 7face1de3700 1 --2- 172.21.15.40:0/587226216 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7facc8066730 0x7facc8068bf0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:00.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.118+0000 7face1de3700 1 --2- 172.21.15.40:0/587226216 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7facdc0c5580 0x7facdc0c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:00.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.118+0000 7face1de3700 1 -- 172.21.15.40:0/587226216 >> 172.21.15.40:0/587226216 conn(0x7facdc06a390 msgr2=0x7facdc06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:00.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.118+0000 7face1de3700 1 -- 172.21.15.40:0/587226216 shutdown_connections 2024-09-13T17:51:00.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.118+0000 7face1de3700 1 -- 172.21.15.40:0/587226216 wait complete. 2024-09-13T17:51:00.140 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300} 2024-09-13T17:51:00.140 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-13T17:51:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.262+0000 7f007cb3b700 1 Processor -- start 2024-09-13T17:51:00.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.262+0000 7f007cb3b700 1 -- start start 2024-09-13T17:51:00.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.262+0000 7f007cb3b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f007806e1e0 0x7f007806e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:00.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.262+0000 7f007cb3b700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f007815dd60 con 0x7f00780c8d10 2024-09-13T17:51:00.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.262+0000 7f007cb3b700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0078154010 con 0x7f0078153c30 2024-09-13T17:51:00.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.262+0000 7f007cb3b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007806ebb0 con 0x7f007806e1e0 2024-09-13T17:51:00.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.262+0000 7f0075d9b700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f00780c8d10 0x7f00780c9110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56166/0 (socket says 172.21.15.40:56166) 2024-09-13T17:51:00.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.262+0000 7f0075d9b700 1 -- 172.21.15.40:0/1447053205 learned_addr learned my addr 172.21.15.40:0/1447053205 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:00.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.262+0000 7f0076d9d700 1 --2- 172.21.15.40:0/1447053205 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f007806e1e0 0x7f007806e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:00.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.262+0000 7f007559a700 1 -- 172.21.15.40:0/1447053205 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216744949 0 0) 0x7f007815dd60 con 0x7f00780c8d10 2024-09-13T17:51:00.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.262+0000 7f007559a700 1 -- 172.21.15.40:0/1447053205 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00600035d0 con 0x7f00780c8d10 2024-09-13T17:51:00.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.266+0000 7f007559a700 1 -- 172.21.15.40:0/1447053205 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1905456774 0 0) 0x7f006c001110 con 0x7f0078153c30 2024-09-13T17:51:00.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.266+0000 7f007559a700 1 -- 172.21.15.40:0/1447053205 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f007815dd60 con 0x7f0078153c30 2024-09-13T17:51:00.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.266+0000 7f007559a700 1 -- 172.21.15.40:0/1447053205 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1321852845 0 0) 0x7f00600035d0 con 0x7f00780c8d10 2024-09-13T17:51:00.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.266+0000 7f0076d9d700 1 -- 172.21.15.40:0/1447053205 >> v1:172.21.15.43:6789/0 conn(0x7f0078153c30 legacy=0x7f007806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:00.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.266+0000 7f0076d9d700 1 -- 172.21.15.40:0/1447053205 >> v1:172.21.15.40:6789/0 conn(0x7f00780c8d10 legacy=0x7f00780c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:00.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.266+0000 7f0076d9d700 1 -- 172.21.15.40:0/1447053205 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007815ce70 con 0x7f007806e1e0 2024-09-13T17:51:00.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.266+0000 7f0076d9d700 1 --2- 172.21.15.40:0/1447053205 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f007806e1e0 0x7f007806e670 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0070009dc0 tx=0x7f007000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c01534225fd1cd58 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:00.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.266+0000 7f007559a700 1 -- 172.21.15.40:0/1447053205 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007000e070 con 0x7f007806e1e0 2024-09-13T17:51:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.266+0000 7f007559a700 1 -- 172.21.15.40:0/1447053205 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0070003a40 con 0x7f007806e1e0 2024-09-13T17:51:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.266+0000 7f007559a700 1 -- 172.21.15.40:0/1447053205 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068001670 con 0x7f007806e1e0 2024-09-13T17:51:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.266+0000 7f007cb3b700 1 -- 172.21.15.40:0/1447053205 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f007806e1e0 msgr2=0x7f007806e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:00.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.266+0000 7f007cb3b700 1 --2- 172.21.15.40:0/1447053205 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f007806e1e0 0x7f007806e670 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0070009dc0 tx=0x7f007000b330 comp rx=0 tx=0).stop 2024-09-13T17:51:00.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.266+0000 7f007cb3b700 1 -- 172.21.15.40:0/1447053205 shutdown_connections 2024-09-13T17:51:00.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.266+0000 7f007cb3b700 1 --2- 172.21.15.40:0/1447053205 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f007806e1e0 0x7f007806e670 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:00.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.266+0000 7f007cb3b700 1 -- 172.21.15.40:0/1447053205 >> 172.21.15.40:0/1447053205 conn(0x7f00780ccef0 msgr2=0x7f00780cd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:00.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.266+0000 7f007cb3b700 1 -- 172.21.15.40:0/1447053205 shutdown_connections 2024-09-13T17:51:00.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.266+0000 7f007cb3b700 1 -- 172.21.15.40:0/1447053205 wait complete. 2024-09-13T17:51:00.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f007cb3b700 1 Processor -- start 2024-09-13T17:51:00.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f007cb3b700 1 -- start start 2024-09-13T17:51:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f007cb3b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f007806e1e0 0x7f00780ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f007cb3b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00780c8d10 0x7f00781f2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f007cb3b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0078153c30 0x7f00781f0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f007cb3b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00781f33c0 con 0x7f00780c8d10 2024-09-13T17:51:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f007cb3b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00780c4360 con 0x7f007806e1e0 2024-09-13T17:51:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f007cb3b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00780c4640 con 0x7f0078153c30 2024-09-13T17:51:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f0075d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00780c8d10 0x7f00781f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f0075d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00780c8d10 0x7f00781f2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49628/0 (socket says 172.21.15.40:49628) 2024-09-13T17:51:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f0075d9b700 1 -- 172.21.15.40:0/3200397653 learned_addr learned my addr 172.21.15.40:0/3200397653 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f007659c700 1 --2- 172.21.15.40:0/3200397653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f007806e1e0 0x7f00780ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:00.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f0076d9d700 1 --2- 172.21.15.40:0/3200397653 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0078153c30 0x7f00781f0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:00.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f0075d9b700 1 -- 172.21.15.40:0/3200397653 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0078153c30 msgr2=0x7f00781f0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:00.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f0075d9b700 1 --2- 172.21.15.40:0/3200397653 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0078153c30 0x7f00781f0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:00.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f0075d9b700 1 -- 172.21.15.40:0/3200397653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f007806e1e0 msgr2=0x7f00780ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:00.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f0075d9b700 1 --2- 172.21.15.40:0/3200397653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f007806e1e0 0x7f00780ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:00.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f0075d9b700 1 -- 172.21.15.40:0/3200397653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0070009a70 con 0x7f00780c8d10 2024-09-13T17:51:00.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f0075d9b700 1 --2- 172.21.15.40:0/3200397653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00780c8d10 0x7f00781f2950 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0068001eb0 tx=0x7f006800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:00.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f0066ffd700 1 -- 172.21.15.40:0/3200397653 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068004ca0 con 0x7f00780c8d10 2024-09-13T17:51:00.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f0066ffd700 1 -- 172.21.15.40:0/3200397653 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0068004e20 con 0x7f00780c8d10 2024-09-13T17:51:00.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f007cb3b700 1 -- 172.21.15.40:0/3200397653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00780ae610 con 0x7f00780c8d10 2024-09-13T17:51:00.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f0066ffd700 1 -- 172.21.15.40:0/3200397653 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068007310 con 0x7f00780c8d10 2024-09-13T17:51:00.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.270+0000 7f007cb3b700 1 -- 172.21.15.40:0/3200397653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007806c7b0 con 0x7f00780c8d10 2024-09-13T17:51:00.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.274+0000 7f007cb3b700 1 -- 172.21.15.40:0/3200397653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00780c41c0 con 0x7f00780c8d10 2024-09-13T17:51:00.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.274+0000 7f0066ffd700 1 -- 172.21.15.40:0/3200397653 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0068007470 con 0x7f00780c8d10 2024-09-13T17:51:00.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.274+0000 7f0066ffd700 1 --2- 172.21.15.40:0/3200397653 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0058066730 0x7f0058068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:00.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.274+0000 7f0066ffd700 1 -- 172.21.15.40:0/3200397653 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0068083fc0 con 0x7f00780c8d10 2024-09-13T17:51:00.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.274+0000 7f007659c700 1 --2- 172.21.15.40:0/3200397653 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0058066730 0x7f0058068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:00.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.278+0000 7f007659c700 1 --2- 172.21.15.40:0/3200397653 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0058066730 0x7f0058068bf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f006c001370 tx=0x7f006c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:00.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.282+0000 7f0066ffd700 1 -- 172.21.15.40:0/3200397653 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f00680b5db0 con 0x7f00780c8d10 2024-09-13T17:51:00.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:00.490+0000 7f007cb3b700 1 -- 172.21.15.40:0/3200397653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f00780c86b0 con 0x7f00780c8d10 2024-09-13T17:51:01.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.122+0000 7f0066ffd700 1 -- 172.21.15.40:0/3200397653 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v9) v1 ==== 105+0+0 (secure 0 0 0) 0x7f00780c41c0 con 0x7f00780c8d10 2024-09-13T17:51:01.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.126+0000 7f007cb3b700 1 -- 172.21.15.40:0/3200397653 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0058066730 msgr2=0x7f0058068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:01.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.126+0000 7f007cb3b700 1 --2- 172.21.15.40:0/3200397653 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0058066730 0x7f0058068bf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f006c001370 tx=0x7f006c009040 comp rx=0 tx=0).stop 2024-09-13T17:51:01.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.126+0000 7f007cb3b700 1 -- 172.21.15.40:0/3200397653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00780c8d10 msgr2=0x7f00781f2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:01.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.126+0000 7f007cb3b700 1 --2- 172.21.15.40:0/3200397653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00780c8d10 0x7f00781f2950 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0068001eb0 tx=0x7f006800c800 comp rx=0 tx=0).stop 2024-09-13T17:51:01.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.126+0000 7f007cb3b700 1 -- 172.21.15.40:0/3200397653 shutdown_connections 2024-09-13T17:51:01.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.126+0000 7f007cb3b700 1 --2- 172.21.15.40:0/3200397653 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0058066730 0x7f0058068bf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:01.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.126+0000 7f007cb3b700 1 --2- 172.21.15.40:0/3200397653 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0078153c30 0x7f00781f0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:01.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.126+0000 7f007cb3b700 1 --2- 172.21.15.40:0/3200397653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00780c8d10 0x7f00781f2950 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:01.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.126+0000 7f007cb3b700 1 --2- 172.21.15.40:0/3200397653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f007806e1e0 0x7f00780ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:01.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.126+0000 7f007cb3b700 1 -- 172.21.15.40:0/3200397653 >> 172.21.15.40:0/3200397653 conn(0x7f00780ccef0 msgr2=0x7f00781520a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:01.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.126+0000 7f007cb3b700 1 -- 172.21.15.40:0/3200397653 shutdown_connections 2024-09-13T17:51:01.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.126+0000 7f007cb3b700 1 -- 172.21.15.40:0/3200397653 wait complete. 2024-09-13T17:51:01.149 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T17:51:01.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.274+0000 7f311a334700 1 Processor -- start 2024-09-13T17:51:01.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.274+0000 7f311a334700 1 -- start start 2024-09-13T17:51:01.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f311a334700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3114150480 0x7f3114070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:01.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f311a334700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3114071170 0x7f31140c5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:01.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f311a334700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31140c4400 con 0x7f3114150860 2024-09-13T17:51:01.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f311a334700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31140709b0 con 0x7f3114150480 2024-09-13T17:51:01.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f311a334700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31140c46e0 con 0x7f3114071170 2024-09-13T17:51:01.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f3113fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3114150480 0x7f3114070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:01.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f3113fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3114150480 0x7f3114070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49644/0 (socket says 172.21.15.40:49644) 2024-09-13T17:51:01.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f3113fff700 1 -- 172.21.15.40:0/3715410696 learned_addr learned my addr 172.21.15.40:0/3715410696 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:01.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f31188d1700 1 --2- 172.21.15.40:0/3715410696 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3114071170 0x7f31140c5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:01.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f3113fff700 1 -- 172.21.15.40:0/3715410696 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3114071170 msgr2=0x7f31140c5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:01.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f3113fff700 1 --2- 172.21.15.40:0/3715410696 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3114071170 0x7f31140c5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:01.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f31137fe700 1 -- 172.21.15.40:0/3715410696 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3829886834 0 0) 0x7f31140c4400 con 0x7f3114150860 2024-09-13T17:51:01.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f3113fff700 1 -- 172.21.15.40:0/3715410696 >> v1:172.21.15.43:6789/0 conn(0x7f3114150860 legacy=0x7f31140c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:01.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f3113fff700 1 -- 172.21.15.40:0/3715410696 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31140c49c0 con 0x7f3114150480 2024-09-13T17:51:01.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f3113fff700 1 --2- 172.21.15.40:0/3715410696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3114150480 0x7f3114070370 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3104009910 tx=0x7f3104009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6efaa82858ada0f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:01.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f31137fe700 1 -- 172.21.15.40:0/3715410696 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104011830 con 0x7f3114150480 2024-09-13T17:51:01.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f31137fe700 1 -- 172.21.15.40:0/3715410696 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3104011990 con 0x7f3114150480 2024-09-13T17:51:01.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f31137fe700 1 -- 172.21.15.40:0/3715410696 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104011d40 con 0x7f3114150480 2024-09-13T17:51:01.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f311a334700 1 -- 172.21.15.40:0/3715410696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3114150480 msgr2=0x7f3114070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:01.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f311a334700 1 --2- 172.21.15.40:0/3715410696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3114150480 0x7f3114070370 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3104009910 tx=0x7f3104009c20 comp rx=0 tx=0).stop 2024-09-13T17:51:01.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f311a334700 1 -- 172.21.15.40:0/3715410696 shutdown_connections 2024-09-13T17:51:01.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f311a334700 1 --2- 172.21.15.40:0/3715410696 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3114071170 0x7f31140c5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:01.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f311a334700 1 --2- 172.21.15.40:0/3715410696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3114150480 0x7f3114070370 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:01.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f311a334700 1 -- 172.21.15.40:0/3715410696 >> 172.21.15.40:0/3715410696 conn(0x7f311406a390 msgr2=0x7f311406a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:01.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f311a334700 1 -- 172.21.15.40:0/3715410696 shutdown_connections 2024-09-13T17:51:01.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.278+0000 7f311a334700 1 -- 172.21.15.40:0/3715410696 wait complete. 2024-09-13T17:51:01.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.282+0000 7f311a334700 1 Processor -- start 2024-09-13T17:51:01.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.282+0000 7f311a334700 1 -- start start 2024-09-13T17:51:01.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.282+0000 7f311a334700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3114071170 0x7f31140bbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:01.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.282+0000 7f311a334700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3114150480 0x7f3114152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:01.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.282+0000 7f311a334700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3114150860 0x7f311414ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:01.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.282+0000 7f311a334700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31140bc150 con 0x7f3114071170 2024-09-13T17:51:01.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.282+0000 7f311a334700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311414fb90 con 0x7f3114150860 2024-09-13T17:51:01.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.282+0000 7f311a334700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31140b7ef0 con 0x7f3114150480 2024-09-13T17:51:01.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.282+0000 7f3113fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3114071170 0x7f31140bbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:01.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.282+0000 7f3113fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3114071170 0x7f31140bbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49656/0 (socket says 172.21.15.40:49656) 2024-09-13T17:51:01.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.282+0000 7f3113fff700 1 -- 172.21.15.40:0/1634757271 learned_addr learned my addr 172.21.15.40:0/1634757271 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:01.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.282+0000 7f31188d1700 1 --2- 172.21.15.40:0/1634757271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3114150860 0x7f311414ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:01.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.282+0000 7f310bfff700 1 --2- 172.21.15.40:0/1634757271 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3114150480 0x7f3114152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:01.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.286+0000 7f3113fff700 1 -- 172.21.15.40:0/1634757271 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3114150480 msgr2=0x7f3114152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:01.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.286+0000 7f3113fff700 1 --2- 172.21.15.40:0/1634757271 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3114150480 0x7f3114152ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:01.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.286+0000 7f3113fff700 1 -- 172.21.15.40:0/1634757271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3114150860 msgr2=0x7f311414ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:01.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.286+0000 7f3113fff700 1 --2- 172.21.15.40:0/1634757271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3114150860 0x7f311414ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:01.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.286+0000 7f3113fff700 1 -- 172.21.15.40:0/1634757271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3104009570 con 0x7f3114071170 2024-09-13T17:51:01.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.286+0000 7f3113fff700 1 --2- 172.21.15.40:0/1634757271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3114071170 0x7f31140bbc10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3104009f70 tx=0x7f3104004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:01.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.286+0000 7f31117fa700 1 -- 172.21.15.40:0/1634757271 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310401a900 con 0x7f3114071170 2024-09-13T17:51:01.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.286+0000 7f31117fa700 1 -- 172.21.15.40:0/1634757271 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310401f070 con 0x7f3114071170 2024-09-13T17:51:01.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.286+0000 7f311a334700 1 -- 172.21.15.40:0/1634757271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31140b81d0 con 0x7f3114071170 2024-09-13T17:51:01.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.286+0000 7f31117fa700 1 -- 172.21.15.40:0/1634757271 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310401acc0 con 0x7f3114071170 2024-09-13T17:51:01.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.286+0000 7f311a334700 1 -- 172.21.15.40:0/1634757271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31141529d0 con 0x7f3114071170 2024-09-13T17:51:01.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.286+0000 7f311a334700 1 -- 172.21.15.40:0/1634757271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3114070370 con 0x7f3114071170 2024-09-13T17:51:01.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.286+0000 7f31117fa700 1 -- 172.21.15.40:0/1634757271 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f310401b430 con 0x7f3114071170 2024-09-13T17:51:01.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.290+0000 7f31117fa700 1 --2- 172.21.15.40:0/1634757271 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f30f4066760 0x7f30f4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:01.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.290+0000 7f31117fa700 1 -- 172.21.15.40:0/1634757271 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3104021070 con 0x7f3114071170 2024-09-13T17:51:01.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.290+0000 7f310bfff700 1 --2- 172.21.15.40:0/1634757271 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f30f4066760 0x7f30f4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:01.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.290+0000 7f310bfff700 1 --2- 172.21.15.40:0/1634757271 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f30f4066760 0x7f30f4068c20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f31140c1dc0 tx=0x7f30fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:01.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.294+0000 7f31117fa700 1 -- 172.21.15.40:0/1634757271 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f31040526d0 con 0x7f3114071170 2024-09-13T17:51:01.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:01.506+0000 7f311a334700 1 -- 172.21.15.40:0/1634757271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f31140725e0 con 0x7f3114071170 2024-09-13T17:51:02.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.130+0000 7f31117fa700 1 -- 172.21.15.40:0/1634757271 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v11) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3104006da0 con 0x7f3114071170 2024-09-13T17:51:02.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.138+0000 7f311a334700 1 -- 172.21.15.40:0/1634757271 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f30f4066760 msgr2=0x7f30f4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:02.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.138+0000 7f311a334700 1 --2- 172.21.15.40:0/1634757271 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f30f4066760 0x7f30f4068c20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f31140c1dc0 tx=0x7f30fc009040 comp rx=0 tx=0).stop 2024-09-13T17:51:02.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.138+0000 7f311a334700 1 -- 172.21.15.40:0/1634757271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3114071170 msgr2=0x7f31140bbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:02.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.138+0000 7f311a334700 1 --2- 172.21.15.40:0/1634757271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3114071170 0x7f31140bbc10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3104009f70 tx=0x7f3104004770 comp rx=0 tx=0).stop 2024-09-13T17:51:02.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.138+0000 7f311a334700 1 -- 172.21.15.40:0/1634757271 shutdown_connections 2024-09-13T17:51:02.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.138+0000 7f311a334700 1 --2- 172.21.15.40:0/1634757271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3114150860 0x7f311414ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:02.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.138+0000 7f311a334700 1 --2- 172.21.15.40:0/1634757271 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f30f4066760 0x7f30f4068c20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:02.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.138+0000 7f311a334700 1 --2- 172.21.15.40:0/1634757271 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3114150480 0x7f3114152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:02.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.138+0000 7f311a334700 1 --2- 172.21.15.40:0/1634757271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3114071170 0x7f31140bbc10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:02.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.138+0000 7f311a334700 1 -- 172.21.15.40:0/1634757271 >> 172.21.15.40:0/1634757271 conn(0x7f311406a390 msgr2=0x7f3114154b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:02.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.138+0000 7f311a334700 1 -- 172.21.15.40:0/1634757271 shutdown_connections 2024-09-13T17:51:02.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.138+0000 7f311a334700 1 -- 172.21.15.40:0/1634757271 wait complete. 2024-09-13T17:51:02.159 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-09-13T17:51:02.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c61cd1700 1 Processor -- start 2024-09-13T17:51:02.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c61cd1700 1 -- start start 2024-09-13T17:51:02.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c61cd1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2c5c0c7f60 0x7f2c5c072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:02.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c61cd1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2c5c0c8340 0x7f2c5c151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:02.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c61cd1700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c5c0c5290 con 0x7f2c5c0c7320 2024-09-13T17:51:02.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c61cd1700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c5c071390 con 0x7f2c5c0c7f60 2024-09-13T17:51:02.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c61cd1700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c0727f0 con 0x7f2c5c0c8340 2024-09-13T17:51:02.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c60ccf700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2c5c0c7f60 0x7f2c5c072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:02.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c60ccf700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2c5c0c7f60 0x7f2c5c072200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57756/0 (socket says 172.21.15.40:57756) 2024-09-13T17:51:02.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c60ccf700 1 -- 172.21.15.40:0/3405674492 learned_addr learned my addr 172.21.15.40:0/3405674492 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:02.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c614d0700 1 --2- 172.21.15.40:0/3405674492 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2c5c0c8340 0x7f2c5c151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:02.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c60ccf700 1 -- 172.21.15.40:0/3405674492 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2c5c0c8340 msgr2=0x7f2c5c151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:02.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c60ccf700 1 --2- 172.21.15.40:0/3405674492 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2c5c0c8340 0x7f2c5c151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:02.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c60ccf700 1 -- 172.21.15.40:0/3405674492 >> v1:172.21.15.43:6789/0 conn(0x7f2c5c0c7320 legacy=0x7f2c5c15ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:02.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c60ccf700 1 -- 172.21.15.40:0/3405674492 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c0c5870 con 0x7f2c5c0c7f60 2024-09-13T17:51:02.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c60ccf700 1 --2- 172.21.15.40:0/3405674492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c5c0c7f60 0x7f2c5c072200 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009910 tx=0x7f2c50009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b49b110d8e9a207 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:02.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c5b7fe700 1 -- 172.21.15.40:0/3405674492 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50011830 con 0x7f2c5c0c7f60 2024-09-13T17:51:02.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c5b7fe700 1 -- 172.21.15.40:0/3405674492 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c50011990 con 0x7f2c5c0c7f60 2024-09-13T17:51:02.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c5b7fe700 1 -- 172.21.15.40:0/3405674492 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50011d40 con 0x7f2c5c0c7f60 2024-09-13T17:51:02.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c61cd1700 1 -- 172.21.15.40:0/3405674492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c5c0c7f60 msgr2=0x7f2c5c072200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:02.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.286+0000 7f2c61cd1700 1 --2- 172.21.15.40:0/3405674492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c5c0c7f60 0x7f2c5c072200 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009910 tx=0x7f2c50009c20 comp rx=0 tx=0).stop 2024-09-13T17:51:02.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.290+0000 7f2c61cd1700 1 -- 172.21.15.40:0/3405674492 shutdown_connections 2024-09-13T17:51:02.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.290+0000 7f2c61cd1700 1 --2- 172.21.15.40:0/3405674492 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2c5c0c8340 0x7f2c5c151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:02.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.290+0000 7f2c61cd1700 1 --2- 172.21.15.40:0/3405674492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c5c0c7f60 0x7f2c5c072200 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:02.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.290+0000 7f2c61cd1700 1 -- 172.21.15.40:0/3405674492 >> 172.21.15.40:0/3405674492 conn(0x7f2c5c066090 msgr2=0x7f2c5c0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:02.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.290+0000 7f2c61cd1700 1 -- 172.21.15.40:0/3405674492 shutdown_connections 2024-09-13T17:51:02.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.290+0000 7f2c61cd1700 1 -- 172.21.15.40:0/3405674492 wait complete. 2024-09-13T17:51:02.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.290+0000 7f2c61cd1700 1 Processor -- start 2024-09-13T17:51:02.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.290+0000 7f2c61cd1700 1 -- start start 2024-09-13T17:51:02.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.290+0000 7f2c61cd1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c5c0c7320 0x7f2c5c1f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:02.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.290+0000 7f2c61cd1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c5c0c7f60 0x7f2c5c14ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:02.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.290+0000 7f2c61cd1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2c5c0c8340 0x7f2c5c14a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:02.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.290+0000 7f2c61cd1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c1f19e0 con 0x7f2c5c0c7320 2024-09-13T17:51:02.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.290+0000 7f2c61cd1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c0c4140 con 0x7f2c5c0c7f60 2024-09-13T17:51:02.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.290+0000 7f2c61cd1700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c0c4420 con 0x7f2c5c0c8340 2024-09-13T17:51:02.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.290+0000 7f2c60ccf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c5c0c7320 0x7f2c5c1f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:02.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.290+0000 7f2c60ccf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c5c0c7320 0x7f2c5c1f14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57766/0 (socket says 172.21.15.40:57766) 2024-09-13T17:51:02.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.290+0000 7f2c60ccf700 1 -- 172.21.15.40:0/1614483028 learned_addr learned my addr 172.21.15.40:0/1614483028 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:02.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.290+0000 7f2c614d0700 1 --2- 172.21.15.40:0/1614483028 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2c5c0c8340 0x7f2c5c14a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:02.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.290+0000 7f2c5bfff700 1 --2- 172.21.15.40:0/1614483028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c5c0c7f60 0x7f2c5c14ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:02.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.294+0000 7f2c60ccf700 1 -- 172.21.15.40:0/1614483028 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2c5c0c8340 msgr2=0x7f2c5c14a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:02.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.294+0000 7f2c60ccf700 1 --2- 172.21.15.40:0/1614483028 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2c5c0c8340 0x7f2c5c14a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:02.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.294+0000 7f2c60ccf700 1 -- 172.21.15.40:0/1614483028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c5c0c7f60 msgr2=0x7f2c5c14ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:02.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.294+0000 7f2c60ccf700 1 --2- 172.21.15.40:0/1614483028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c5c0c7f60 0x7f2c5c14ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:02.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.294+0000 7f2c60ccf700 1 -- 172.21.15.40:0/1614483028 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c50009570 con 0x7f2c5c0c7320 2024-09-13T17:51:02.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.294+0000 7f2c60ccf700 1 --2- 172.21.15.40:0/1614483028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c5c0c7320 0x7f2c5c1f14a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009c60 tx=0x7f2c50004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:02.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.294+0000 7f2c597fa700 1 -- 172.21.15.40:0/1614483028 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50004300 con 0x7f2c5c0c7320 2024-09-13T17:51:02.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.294+0000 7f2c597fa700 1 -- 172.21.15.40:0/1614483028 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5001f070 con 0x7f2c5c0c7320 2024-09-13T17:51:02.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.294+0000 7f2c61cd1700 1 -- 172.21.15.40:0/1614483028 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5c1f0ca0 con 0x7f2c5c0c7320 2024-09-13T17:51:02.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.294+0000 7f2c597fa700 1 -- 172.21.15.40:0/1614483028 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c500036c0 con 0x7f2c5c0c7320 2024-09-13T17:51:02.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.294+0000 7f2c61cd1700 1 -- 172.21.15.40:0/1614483028 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5c0c3290 con 0x7f2c5c0c7320 2024-09-13T17:51:02.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.298+0000 7f2c597fa700 1 -- 172.21.15.40:0/1614483028 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2c50021030 con 0x7f2c5c0c7320 2024-09-13T17:51:02.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.298+0000 7f2c61cd1700 1 -- 172.21.15.40:0/1614483028 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5c1f0b00 con 0x7f2c5c0c7320 2024-09-13T17:51:02.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.298+0000 7f2c597fa700 1 --2- 172.21.15.40:0/1614483028 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2c40066760 0x7f2c40068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:02.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.298+0000 7f2c597fa700 1 -- 172.21.15.40:0/1614483028 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2c500857d0 con 0x7f2c5c0c7320 2024-09-13T17:51:02.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.298+0000 7f2c5bfff700 1 --2- 172.21.15.40:0/1614483028 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2c40066760 0x7f2c40068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:02.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.298+0000 7f2c5bfff700 1 --2- 172.21.15.40:0/1614483028 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2c40066760 0x7f2c40068c20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c070af0 tx=0x7f2c48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:02.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.306+0000 7f2c597fa700 1 -- 172.21.15.40:0/1614483028 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2c5c1f0b00 con 0x7f2c5c0c7320 2024-09-13T17:51:02.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:02.510+0000 7f2c61cd1700 1 -- 172.21.15.40:0/1614483028 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f2c5c1f11c0 con 0x7f2c5c0c7320 2024-09-13T17:51:03.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.134+0000 7f2c597fa700 1 -- 172.21.15.40:0/1614483028 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v12) v1 ==== 115+0+0 (secure 0 0 0) 0x7f2c5c1f11c0 con 0x7f2c5c0c7320 2024-09-13T17:51:03.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.138+0000 7f2c61cd1700 1 -- 172.21.15.40:0/1614483028 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2c40066760 msgr2=0x7f2c40068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:03.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.138+0000 7f2c61cd1700 1 --2- 172.21.15.40:0/1614483028 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2c40066760 0x7f2c40068c20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c070af0 tx=0x7f2c48009040 comp rx=0 tx=0).stop 2024-09-13T17:51:03.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.138+0000 7f2c61cd1700 1 -- 172.21.15.40:0/1614483028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c5c0c7320 msgr2=0x7f2c5c1f14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:03.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.138+0000 7f2c61cd1700 1 --2- 172.21.15.40:0/1614483028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c5c0c7320 0x7f2c5c1f14a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009c60 tx=0x7f2c50004770 comp rx=0 tx=0).stop 2024-09-13T17:51:03.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.142+0000 7f2c61cd1700 1 -- 172.21.15.40:0/1614483028 shutdown_connections 2024-09-13T17:51:03.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.142+0000 7f2c61cd1700 1 --2- 172.21.15.40:0/1614483028 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2c40066760 0x7f2c40068c20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.142+0000 7f2c61cd1700 1 --2- 172.21.15.40:0/1614483028 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2c5c0c8340 0x7f2c5c14a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.142+0000 7f2c61cd1700 1 --2- 172.21.15.40:0/1614483028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c5c0c7f60 0x7f2c5c14ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.142+0000 7f2c61cd1700 1 --2- 172.21.15.40:0/1614483028 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2c5c0c7320 0x7f2c5c1f14a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.142+0000 7f2c61cd1700 1 -- 172.21.15.40:0/1614483028 >> 172.21.15.40:0/1614483028 conn(0x7f2c5c066090 msgr2=0x7f2c5c0701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:03.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.142+0000 7f2c61cd1700 1 -- 172.21.15.40:0/1614483028 shutdown_connections 2024-09-13T17:51:03.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.142+0000 7f2c61cd1700 1 -- 172.21.15.40:0/1614483028 wait complete. 2024-09-13T17:51:03.163 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T17:51:03.163 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:51:03.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.270+0000 7f085759e700 1 Processor -- start 2024-09-13T17:51:03.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.270+0000 7f085759e700 1 -- start start 2024-09-13T17:51:03.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.274+0000 7f085759e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0858156310 0x7f08581566f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:03.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.274+0000 7f085759e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0858150580 0x7f0858156c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:03.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.274+0000 7f085759e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08580c3fd0 con 0x7f0858070b00 2024-09-13T17:51:03.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.274+0000 7f085759e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0858071c00 con 0x7f0858150580 2024-09-13T17:51:03.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.274+0000 7f085759e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0858071e70 con 0x7f0858156310 2024-09-13T17:51:03.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.274+0000 7f0856d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0858150580 0x7f0858156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:03.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.274+0000 7f0856d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0858150580 0x7f0858156c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57774/0 (socket says 172.21.15.40:57774) 2024-09-13T17:51:03.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.274+0000 7f0856d9d700 1 -- 172.21.15.40:0/2200750145 learned_addr learned my addr 172.21.15.40:0/2200750145 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:03.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.274+0000 7f085659c700 1 --2- 172.21.15.40:0/2200750145 >> v2:172.21.15.43:3300/0 conn(0x7f0858156310 0x7f08581566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:03.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.274+0000 7f0856d9d700 1 -- 172.21.15.40:0/2200750145 >> v2:172.21.15.43:3300/0 conn(0x7f0858156310 msgr2=0x7f08581566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:03.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.274+0000 7f0856d9d700 1 --2- 172.21.15.40:0/2200750145 >> v2:172.21.15.43:3300/0 conn(0x7f0858156310 0x7f08581566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.274+0000 7f0856d9d700 1 -- 172.21.15.40:0/2200750145 >> v1:172.21.15.43:6789/0 conn(0x7f0858070b00 legacy=0x7f085814fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:03.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.274+0000 7f0856d9d700 1 -- 172.21.15.40:0/2200750145 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0858072150 con 0x7f0858150580 2024-09-13T17:51:03.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.274+0000 7f085659c700 1 --2- 172.21.15.40:0/2200750145 >> v2:172.21.15.43:3300/0 conn(0x7f0858156310 0x7f08581566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:03.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.274+0000 7f0856d9d700 1 --2- 172.21.15.40:0/2200750145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0858150580 0x7f0858156c30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f08580cbdd0 tx=0x7f085000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ffc081fd70d2c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:03.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.274+0000 7f085559a700 1 -- 172.21.15.40:0/2200750145 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085000e070 con 0x7f0858150580 2024-09-13T17:51:03.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.274+0000 7f085559a700 1 -- 172.21.15.40:0/2200750145 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085000b9c0 con 0x7f0858150580 2024-09-13T17:51:03.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.274+0000 7f085559a700 1 -- 172.21.15.40:0/2200750145 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085000bd70 con 0x7f0858150580 2024-09-13T17:51:03.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.278+0000 7f085759e700 1 -- 172.21.15.40:0/2200750145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0858150580 msgr2=0x7f0858156c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:03.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.278+0000 7f085759e700 1 --2- 172.21.15.40:0/2200750145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0858150580 0x7f0858156c30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f08580cbdd0 tx=0x7f085000b330 comp rx=0 tx=0).stop 2024-09-13T17:51:03.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.278+0000 7f085759e700 1 -- 172.21.15.40:0/2200750145 shutdown_connections 2024-09-13T17:51:03.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.278+0000 7f085759e700 1 --2- 172.21.15.40:0/2200750145 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0858150580 0x7f0858156c30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.278+0000 7f085759e700 1 --2- 172.21.15.40:0/2200750145 >> v2:172.21.15.43:3300/0 conn(0x7f0858156310 0x7f08581566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.278+0000 7f085759e700 1 -- 172.21.15.40:0/2200750145 >> 172.21.15.40:0/2200750145 conn(0x7f08580cb2e0 msgr2=0x7f08580cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:03.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.278+0000 7f085759e700 1 -- 172.21.15.40:0/2200750145 shutdown_connections 2024-09-13T17:51:03.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.278+0000 7f085759e700 1 -- 172.21.15.40:0/2200750145 wait complete. 2024-09-13T17:51:03.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.278+0000 7f085759e700 1 Processor -- start 2024-09-13T17:51:03.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.278+0000 7f085759e700 1 -- start start 2024-09-13T17:51:03.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.278+0000 7f085759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0858070b00 0x7f08581ed6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:03.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.278+0000 7f085759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0858150580 0x7f08580c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:03.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.278+0000 7f085759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0858156310 0x7f08580c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:03.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.278+0000 7f085759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08580c8c10 con 0x7f0858156310 2024-09-13T17:51:03.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.278+0000 7f085759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085814bac0 con 0x7f0858070b00 2024-09-13T17:51:03.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.278+0000 7f085759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08581edbe0 con 0x7f0858150580 2024-09-13T17:51:03.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.282+0000 7f0856d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0858156310 0x7f08580c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:03.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.282+0000 7f0855d9b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0858150580 0x7f08580c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:03.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.282+0000 7f0856d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0858156310 0x7f08580c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57776/0 (socket says 172.21.15.40:57776) 2024-09-13T17:51:03.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.282+0000 7f0855d9b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0858150580 0x7f08580c65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:48730/0 (socket says 172.21.15.40:48730) 2024-09-13T17:51:03.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.282+0000 7f0856d9d700 1 -- 172.21.15.40:0/3972542302 learned_addr learned my addr 172.21.15.40:0/3972542302 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:03.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.282+0000 7f085659c700 1 --2- 172.21.15.40:0/3972542302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0858070b00 0x7f08581ed6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:03.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.282+0000 7f0856d9d700 1 -- 172.21.15.40:0/3972542302 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0858150580 msgr2=0x7f08580c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.282+0000 7f0856d9d700 1 --2- 172.21.15.40:0/3972542302 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0858150580 0x7f08580c65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.282+0000 7f0856d9d700 1 -- 172.21.15.40:0/3972542302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0858070b00 msgr2=0x7f08581ed6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.282+0000 7f0856d9d700 1 --2- 172.21.15.40:0/3972542302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0858070b00 0x7f08581ed6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.282+0000 7f0856d9d700 1 -- 172.21.15.40:0/3972542302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0850009b30 con 0x7f0858156310 2024-09-13T17:51:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.282+0000 7f0856d9d700 1 --2- 172.21.15.40:0/3972542302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0858156310 0x7f08580c6b00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f085000b730 tx=0x7f0850004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.282+0000 7f0846ffd700 1 -- 172.21.15.40:0/3972542302 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0850004870 con 0x7f0858156310 2024-09-13T17:51:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.282+0000 7f0846ffd700 1 -- 172.21.15.40:0/3972542302 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08500092e0 con 0x7f0858156310 2024-09-13T17:51:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.282+0000 7f085759e700 1 -- 172.21.15.40:0/3972542302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08580c8690 con 0x7f0858156310 2024-09-13T17:51:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.282+0000 7f0846ffd700 1 -- 172.21.15.40:0/3972542302 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085001c660 con 0x7f0858156310 2024-09-13T17:51:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.282+0000 7f085759e700 1 -- 172.21.15.40:0/3972542302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085814b470 con 0x7f0858156310 2024-09-13T17:51:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.282+0000 7f085759e700 1 -- 172.21.15.40:0/3972542302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f085814b2d0 con 0x7f0858156310 2024-09-13T17:51:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.286+0000 7f0846ffd700 1 -- 172.21.15.40:0/3972542302 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f085001c7c0 con 0x7f0858156310 2024-09-13T17:51:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.286+0000 7f0846ffd700 1 --2- 172.21.15.40:0/3972542302 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0838066730 0x7f0838068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.286+0000 7f0846ffd700 1 -- 172.21.15.40:0/3972542302 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f08500863f0 con 0x7f0858156310 2024-09-13T17:51:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.286+0000 7f085659c700 1 --2- 172.21.15.40:0/3972542302 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0838066730 0x7f0838068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:03.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.286+0000 7f085659c700 1 --2- 172.21.15.40:0/3972542302 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0838066730 0x7f0838068bf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f08480044b0 tx=0x7f0848008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:03.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.294+0000 7f0846ffd700 1 -- 172.21.15.40:0/3972542302 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0850053700 con 0x7f0858156310 2024-09-13T17:51:03.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.502+0000 7f085759e700 1 -- 172.21.15.40:0/3972542302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f085814b2d0 con 0x7f0858156310 2024-09-13T17:51:03.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.502+0000 7f0846ffd700 1 -- 172.21.15.40:0/3972542302 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6253 (secure 0 0 0) 0x7f085814b2d0 con 0x7f0858156310 2024-09-13T17:51:03.510 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:03.510 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6839/1861594206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1861594206},{"type":"v1","addr":"172.21.15.40:6839","nonce":1861594206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1548394691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1548394691},{"type":"v1","addr":"172.21.15.40:6835","nonce":1548394691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/486611242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":486611242},{"type":"v1","addr":"172.21.15.40:6837","nonce":486611242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4279,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/526975022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":526975022},{"type":"v1","addr":"172.21.15.43:6833","nonce":526975022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:50:34.562623+0000","modified":"2024-09-13T17:51:03.132787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":4288,"mds_1":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":1,"incarnation":10,"state":"up:creating","state_seq":1,"addr":"172.21.15.43:6835/2586910944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2586910944},{"type":"v1","addr":"172.21.15.43:6835","nonce":2586910944}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/334160033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":334160033},{"type":"v1","addr":"172.21.15.43:6837","nonce":334160033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:51:03.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.506+0000 7f085759e700 1 -- 172.21.15.40:0/3972542302 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0838066730 msgr2=0x7f0838068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:03.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.506+0000 7f085759e700 1 --2- 172.21.15.40:0/3972542302 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0838066730 0x7f0838068bf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f08480044b0 tx=0x7f0848008040 comp rx=0 tx=0).stop 2024-09-13T17:51:03.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.506+0000 7f085759e700 1 -- 172.21.15.40:0/3972542302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0858156310 msgr2=0x7f08580c6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:03.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.506+0000 7f085759e700 1 --2- 172.21.15.40:0/3972542302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0858156310 0x7f08580c6b00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f085000b730 tx=0x7f0850004af0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.506+0000 7f085759e700 1 -- 172.21.15.40:0/3972542302 shutdown_connections 2024-09-13T17:51:03.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.506+0000 7f085759e700 1 --2- 172.21.15.40:0/3972542302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0858156310 0x7f08580c6b00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.506+0000 7f085759e700 1 --2- 172.21.15.40:0/3972542302 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0838066730 0x7f0838068bf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.506+0000 7f085759e700 1 --2- 172.21.15.40:0/3972542302 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0858150580 0x7f08580c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.506+0000 7f085759e700 1 --2- 172.21.15.40:0/3972542302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0858070b00 0x7f08581ed6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.506+0000 7f085759e700 1 -- 172.21.15.40:0/3972542302 >> 172.21.15.40:0/3972542302 conn(0x7f08580cb2e0 msgr2=0x7f08580c4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:03.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.506+0000 7f085759e700 1 -- 172.21.15.40:0/3972542302 shutdown_connections 2024-09-13T17:51:03.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.506+0000 7f085759e700 1 -- 172.21.15.40:0/3972542302 wait complete. 2024-09-13T17:51:03.514 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 12 2024-09-13T17:51:03.529 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:51:03.600 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:51:03.603 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:51:03.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.642+0000 7fa02afd0700 1 Processor -- start 2024-09-13T17:51:03.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.642+0000 7fa02afd0700 1 -- start start 2024-09-13T17:51:03.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.642+0000 7fa02afd0700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa0240c6130 0x7fa024070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:03.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.642+0000 7fa02afd0700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa0240c6510 0x7fa02406e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:03.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.642+0000 7fa02afd0700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa02406df10 con 0x7fa0240c7280 2024-09-13T17:51:03.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.642+0000 7fa02afd0700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa02415a3e0 con 0x7fa0240c6130 2024-09-13T17:51:03.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.642+0000 7fa02afd0700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240c6e40 con 0x7fa0240c6510 2024-09-13T17:51:03.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.642+0000 7fa029fce700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa0240c6130 0x7fa024070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:03.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.642+0000 7fa029fce700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa0240c6130 0x7fa024070af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:36806/0 (socket says 172.21.15.40:36806) 2024-09-13T17:51:03.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.642+0000 7fa029fce700 1 -- 172.21.15.40:0/25634664 learned_addr learned my addr 172.21.15.40:0/25634664 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:03.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.642+0000 7fa0297cd700 1 --2- 172.21.15.40:0/25634664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa0240c6510 0x7fa02406e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:03.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.646+0000 7fa029fce700 1 -- 172.21.15.40:0/25634664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa0240c6510 msgr2=0x7fa02406e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:03.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.646+0000 7fa029fce700 1 --2- 172.21.15.40:0/25634664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa0240c6510 0x7fa02406e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.646+0000 7fa029fce700 1 -- 172.21.15.40:0/25634664 >> v1:172.21.15.43:6789/0 conn(0x7fa0240c7280 legacy=0x7fa024159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:03.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.646+0000 7fa029fce700 1 -- 172.21.15.40:0/25634664 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0240c76b0 con 0x7fa0240c6130 2024-09-13T17:51:03.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.646+0000 7fa029fce700 1 --2- 172.21.15.40:0/25634664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0240c6130 0x7fa024070af0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa020009e50 tx=0x7fa02000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a9d9d61391a7ac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:03.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.646+0000 7fa028fcc700 1 -- 172.21.15.40:0/25634664 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02000e070 con 0x7fa0240c6130 2024-09-13T17:51:03.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.646+0000 7fa028fcc700 1 -- 172.21.15.40:0/25634664 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02000b910 con 0x7fa0240c6130 2024-09-13T17:51:03.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.646+0000 7fa028fcc700 1 -- 172.21.15.40:0/25634664 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02000bcc0 con 0x7fa0240c6130 2024-09-13T17:51:03.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.646+0000 7fa02afd0700 1 -- 172.21.15.40:0/25634664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0240c6130 msgr2=0x7fa024070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:03.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.646+0000 7fa02afd0700 1 --2- 172.21.15.40:0/25634664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0240c6130 0x7fa024070af0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa020009e50 tx=0x7fa02000b330 comp rx=0 tx=0).stop 2024-09-13T17:51:03.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.646+0000 7fa02afd0700 1 -- 172.21.15.40:0/25634664 shutdown_connections 2024-09-13T17:51:03.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.646+0000 7fa02afd0700 1 --2- 172.21.15.40:0/25634664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa0240c6510 0x7fa02406e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.646+0000 7fa02afd0700 1 --2- 172.21.15.40:0/25634664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0240c6130 0x7fa024070af0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.646+0000 7fa02afd0700 1 -- 172.21.15.40:0/25634664 >> 172.21.15.40:0/25634664 conn(0x7fa0240cb4c0 msgr2=0x7fa0240cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:03.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.646+0000 7fa02afd0700 1 -- 172.21.15.40:0/25634664 shutdown_connections 2024-09-13T17:51:03.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.646+0000 7fa02afd0700 1 -- 172.21.15.40:0/25634664 wait complete. 2024-09-13T17:51:03.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.646+0000 7fa02afd0700 1 Processor -- start 2024-09-13T17:51:03.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa02afd0700 1 -- start start 2024-09-13T17:51:03.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa02afd0700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa0240c6130 0x7fa024157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:03.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa02afd0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0240c6510 0x7fa0240c38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:03.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa02afd0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0240c7280 0x7fa0240c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:03.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa02afd0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa024154510 con 0x7fa0240c6510 2024-09-13T17:51:03.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa02afd0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0241547f0 con 0x7fa0240c7280 2024-09-13T17:51:03.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa02afd0700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa024154ad0 con 0x7fa0240c6130 2024-09-13T17:51:03.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa0297cd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0240c6510 0x7fa0240c38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:03.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa029fce700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa0240c6130 0x7fa024157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:03.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa02a7cf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0240c7280 0x7fa0240c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:03.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa0297cd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0240c6510 0x7fa0240c38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57782/0 (socket says 172.21.15.40:57782) 2024-09-13T17:51:03.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa0297cd700 1 -- 172.21.15.40:0/2233105498 learned_addr learned my addr 172.21.15.40:0/2233105498 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:03.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa0297cd700 1 -- 172.21.15.40:0/2233105498 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa0240c6130 msgr2=0x7fa024157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:03.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa0297cd700 1 --2- 172.21.15.40:0/2233105498 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa0240c6130 0x7fa024157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa0297cd700 1 -- 172.21.15.40:0/2233105498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0240c7280 msgr2=0x7fa0240c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:03.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa0297cd700 1 --2- 172.21.15.40:0/2233105498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0240c7280 0x7fa0240c3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa0297cd700 1 -- 172.21.15.40:0/2233105498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa020009b30 con 0x7fa0240c6510 2024-09-13T17:51:03.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa0297cd700 1 --2- 172.21.15.40:0/2233105498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0240c6510 0x7fa0240c38b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa01400e840 tx=0x7fa01400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:03.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa01a7fc700 1 -- 172.21.15.40:0/2233105498 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0140095a0 con 0x7fa0240c6510 2024-09-13T17:51:03.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa01a7fc700 1 -- 172.21.15.40:0/2233105498 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa014013680 con 0x7fa0240c6510 2024-09-13T17:51:03.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa02afd0700 1 -- 172.21.15.40:0/2233105498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0240c2dd0 con 0x7fa0240c6510 2024-09-13T17:51:03.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa01a7fc700 1 -- 172.21.15.40:0/2233105498 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0140139f0 con 0x7fa0240c6510 2024-09-13T17:51:03.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.650+0000 7fa02afd0700 1 -- 172.21.15.40:0/2233105498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0240c3390 con 0x7fa0240c6510 2024-09-13T17:51:03.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.654+0000 7fa02afd0700 1 -- 172.21.15.40:0/2233105498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0240c3550 con 0x7fa0240c6510 2024-09-13T17:51:03.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.654+0000 7fa01a7fc700 1 -- 172.21.15.40:0/2233105498 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa014013b50 con 0x7fa0240c6510 2024-09-13T17:51:03.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.654+0000 7fa01a7fc700 1 --2- 172.21.15.40:0/2233105498 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa008066730 0x7fa008068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:03.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.654+0000 7fa01a7fc700 1 -- 172.21.15.40:0/2233105498 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa014084320 con 0x7fa0240c6510 2024-09-13T17:51:03.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.654+0000 7fa029fce700 1 --2- 172.21.15.40:0/2233105498 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa008066730 0x7fa008068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:03.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.654+0000 7fa029fce700 1 --2- 172.21.15.40:0/2233105498 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa008066730 0x7fa008068bf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa02000b370 tx=0x7fa020009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:03.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.662+0000 7fa01a7fc700 1 -- 172.21.15.40:0/2233105498 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa014051310 con 0x7fa0240c6510 2024-09-13T17:51:03.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.834+0000 7fa02afd0700 1 -- 172.21.15.40:0/2233105498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa024157970 con 0x7fa0240c6510 2024-09-13T17:51:03.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.838+0000 7fa01a7fc700 1 -- 172.21.15.40:0/2233105498 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18262 (secure 0 0 0) 0x7fa024157970 con 0x7fa0240c6510 2024-09-13T17:51:03.844 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:03.844 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:50:38.974071+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:50:32.740336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:50:33.552914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:50:36.623639+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:51:03.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.842+0000 7fa02afd0700 1 -- 172.21.15.40:0/2233105498 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa008066730 msgr2=0x7fa008068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.842+0000 7fa02afd0700 1 --2- 172.21.15.40:0/2233105498 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa008066730 0x7fa008068bf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa02000b370 tx=0x7fa020009000 comp rx=0 tx=0).stop 2024-09-13T17:51:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.842+0000 7fa02afd0700 1 -- 172.21.15.40:0/2233105498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0240c6510 msgr2=0x7fa0240c38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.842+0000 7fa02afd0700 1 --2- 172.21.15.40:0/2233105498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0240c6510 0x7fa0240c38b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa01400e840 tx=0x7fa01400eb50 comp rx=0 tx=0).stop 2024-09-13T17:51:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.842+0000 7fa02afd0700 1 -- 172.21.15.40:0/2233105498 shutdown_connections 2024-09-13T17:51:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.842+0000 7fa02afd0700 1 --2- 172.21.15.40:0/2233105498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0240c7280 0x7fa0240c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.842+0000 7fa02afd0700 1 --2- 172.21.15.40:0/2233105498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa0240c6510 0x7fa0240c38b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.842+0000 7fa02afd0700 1 --2- 172.21.15.40:0/2233105498 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa008066730 0x7fa008068bf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.842+0000 7fa02afd0700 1 --2- 172.21.15.40:0/2233105498 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa0240c6130 0x7fa024157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.842+0000 7fa02afd0700 1 -- 172.21.15.40:0/2233105498 >> 172.21.15.40:0/2233105498 conn(0x7fa0240cb4c0 msgr2=0x7fa0240707c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.842+0000 7fa02afd0700 1 -- 172.21.15.40:0/2233105498 shutdown_connections 2024-09-13T17:51:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:03.842+0000 7fa02afd0700 1 -- 172.21.15.40:0/2233105498 wait complete. 2024-09-13T17:51:03.859 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T17:51:03.859 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:51:04.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6ee493700 1 Processor -- start 2024-09-13T17:51:04.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6ee493700 1 -- start start 2024-09-13T17:51:04.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6ee493700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd6e8072350 0x7fd6e8150d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:04.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6ee493700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd6e80c7200 0x7fd6e8151280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:04.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6ee493700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e80c55c0 con 0x7fd6e8071f70 2024-09-13T17:51:04.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6ee493700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e806a390 con 0x7fd6e80c7200 2024-09-13T17:51:04.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6ee493700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e8157710 con 0x7fd6e8072350 2024-09-13T17:51:04.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6edc92700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd6e80c7200 0x7fd6e8151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:04.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6ed491700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd6e8071f70 0x7fd6e80c5d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46054/0 (socket says 172.21.15.40:46054) 2024-09-13T17:51:04.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6edc92700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd6e80c7200 0x7fd6e8151280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57784/0 (socket says 172.21.15.40:57784) 2024-09-13T17:51:04.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6ed491700 1 -- 172.21.15.40:0/204863963 learned_addr learned my addr 172.21.15.40:0/204863963 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:04.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6ecc90700 1 --2- 172.21.15.40:0/204863963 >> v2:172.21.15.43:3300/0 conn(0x7fd6e8072350 0x7fd6e8150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:04.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6edc92700 1 -- 172.21.15.40:0/204863963 >> v2:172.21.15.43:3300/0 conn(0x7fd6e8072350 msgr2=0x7fd6e8150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:04.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6edc92700 1 --2- 172.21.15.40:0/204863963 >> v2:172.21.15.43:3300/0 conn(0x7fd6e8072350 0x7fd6e8150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6edc92700 1 -- 172.21.15.40:0/204863963 >> v1:172.21.15.40:6789/0 conn(0x7fd6e8071f70 legacy=0x7fd6e80c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6edc92700 1 -- 172.21.15.40:0/204863963 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e81579d0 con 0x7fd6e80c7200 2024-09-13T17:51:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6dffff700 1 -- 172.21.15.40:0/204863963 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528560670 0 0) 0x7fd6e80c55c0 con 0x7fd6e8071f70 2024-09-13T17:51:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6edc92700 1 --2- 172.21.15.40:0/204863963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6e80c7200 0x7fd6e8151280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd6e806f500 tx=0x7fd6e0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeeaa27da0fcf7fb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6dffff700 1 -- 172.21.15.40:0/204863963 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0011830 con 0x7fd6e80c7200 2024-09-13T17:51:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6dffff700 1 -- 172.21.15.40:0/204863963 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e0011990 con 0x7fd6e80c7200 2024-09-13T17:51:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.014+0000 7fd6dffff700 1 -- 172.21.15.40:0/204863963 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0011d20 con 0x7fd6e80c7200 2024-09-13T17:51:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ee493700 1 -- 172.21.15.40:0/204863963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6e80c7200 msgr2=0x7fd6e8151280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ee493700 1 --2- 172.21.15.40:0/204863963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6e80c7200 0x7fd6e8151280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd6e806f500 tx=0x7fd6e0009b30 comp rx=0 tx=0).stop 2024-09-13T17:51:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ee493700 1 -- 172.21.15.40:0/204863963 shutdown_connections 2024-09-13T17:51:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ee493700 1 --2- 172.21.15.40:0/204863963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6e80c7200 0x7fd6e8151280 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ee493700 1 --2- 172.21.15.40:0/204863963 >> v2:172.21.15.43:3300/0 conn(0x7fd6e8072350 0x7fd6e8150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ee493700 1 -- 172.21.15.40:0/204863963 >> 172.21.15.40:0/204863963 conn(0x7fd6e80cabc0 msgr2=0x7fd6e80cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:04.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ee493700 1 -- 172.21.15.40:0/204863963 shutdown_connections 2024-09-13T17:51:04.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ee493700 1 -- 172.21.15.40:0/204863963 wait complete. 2024-09-13T17:51:04.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ee493700 1 Processor -- start 2024-09-13T17:51:04.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ee493700 1 -- start start 2024-09-13T17:51:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ee493700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd6e8071f70 0x7fd6e81540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ee493700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6e8072350 0x7fd6e8156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ee493700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e80c7200 0x7fd6e814c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ee493700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e8154600 con 0x7fd6e8072350 2024-09-13T17:51:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ee493700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e8156aa0 con 0x7fd6e80c7200 2024-09-13T17:51:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ee493700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e8153740 con 0x7fd6e8071f70 2024-09-13T17:51:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ecc90700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6e8072350 0x7fd6e8156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ecc90700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6e8072350 0x7fd6e8156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57798/0 (socket says 172.21.15.40:57798) 2024-09-13T17:51:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ecc90700 1 -- 172.21.15.40:0/553096714 learned_addr learned my addr 172.21.15.40:0/553096714 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6edc92700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e80c7200 0x7fd6e814c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ed491700 1 --2- 172.21.15.40:0/553096714 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd6e8071f70 0x7fd6e81540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ecc90700 1 -- 172.21.15.40:0/553096714 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd6e8071f70 msgr2=0x7fd6e81540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ecc90700 1 --2- 172.21.15.40:0/553096714 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd6e8071f70 0x7fd6e81540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ecc90700 1 -- 172.21.15.40:0/553096714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e80c7200 msgr2=0x7fd6e814c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:04.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ecc90700 1 --2- 172.21.15.40:0/553096714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e80c7200 0x7fd6e814c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ecc90700 1 -- 172.21.15.40:0/553096714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e00094b0 con 0x7fd6e8072350 2024-09-13T17:51:04.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ecc90700 1 --2- 172.21.15.40:0/553096714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6e8072350 0x7fd6e8156f30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009dd0 tx=0x7fd6e400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:04.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ddffb700 1 -- 172.21.15.40:0/553096714 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4009c40 con 0x7fd6e8072350 2024-09-13T17:51:04.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ddffb700 1 -- 172.21.15.40:0/553096714 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e400f040 con 0x7fd6e8072350 2024-09-13T17:51:04.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ddffb700 1 -- 172.21.15.40:0/553096714 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4014480 con 0x7fd6e8072350 2024-09-13T17:51:04.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.018+0000 7fd6ee493700 1 -- 172.21.15.40:0/553096714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e8154930 con 0x7fd6e8072350 2024-09-13T17:51:04.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.022+0000 7fd6ee493700 1 -- 172.21.15.40:0/553096714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e81574d0 con 0x7fd6e8072350 2024-09-13T17:51:04.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.022+0000 7fd6ee493700 1 -- 172.21.15.40:0/553096714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e80cb6e0 con 0x7fd6e8072350 2024-09-13T17:51:04.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.022+0000 7fd6ddffb700 1 -- 172.21.15.40:0/553096714 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd6e40145e0 con 0x7fd6e8072350 2024-09-13T17:51:04.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.022+0000 7fd6ddffb700 1 --2- 172.21.15.40:0/553096714 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd6cc066760 0x7fd6cc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:04.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.022+0000 7fd6ddffb700 1 -- 172.21.15.40:0/553096714 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd6e4084b40 con 0x7fd6e8072350 2024-09-13T17:51:04.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.022+0000 7fd6ed491700 1 --2- 172.21.15.40:0/553096714 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd6cc066760 0x7fd6cc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:04.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.022+0000 7fd6ed491700 1 --2- 172.21.15.40:0/553096714 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd6cc066760 0x7fd6cc068c20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8000f60 tx=0x7fd6d8007e00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:04.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.030+0000 7fd6ddffb700 1 -- 172.21.15.40:0/553096714 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd6e4051d00 con 0x7fd6e8072350 2024-09-13T17:51:04.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.198+0000 7fd6ee493700 1 -- 172.21.15.40:0/553096714 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6e80c5810 con 0x7fd6cc066760 2024-09-13T17:51:04.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.218+0000 7fd6ddffb700 1 -- 172.21.15.40:0/553096714 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345205 (secure 0 0 0) 0x7fd6d8001370 con 0x7fd6cc066760 2024-09-13T17:51:04.228 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:04.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.222+0000 7fd6ee493700 1 -- 172.21.15.40:0/553096714 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd6cc066760 msgr2=0x7fd6cc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:04.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.222+0000 7fd6ee493700 1 --2- 172.21.15.40:0/553096714 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd6cc066760 0x7fd6cc068c20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8000f60 tx=0x7fd6d8007e00 comp rx=0 tx=0).stop 2024-09-13T17:51:04.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.222+0000 7fd6ee493700 1 -- 172.21.15.40:0/553096714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6e8072350 msgr2=0x7fd6e8156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:04.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.222+0000 7fd6ee493700 1 --2- 172.21.15.40:0/553096714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6e8072350 0x7fd6e8156f30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009dd0 tx=0x7fd6e400c3a0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.222+0000 7fd6ee493700 1 -- 172.21.15.40:0/553096714 shutdown_connections 2024-09-13T17:51:04.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.222+0000 7fd6ee493700 1 --2- 172.21.15.40:0/553096714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e80c7200 0x7fd6e814c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.222+0000 7fd6ee493700 1 --2- 172.21.15.40:0/553096714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6e8072350 0x7fd6e8156f30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.222+0000 7fd6ee493700 1 --2- 172.21.15.40:0/553096714 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd6cc066760 0x7fd6cc068c20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.222+0000 7fd6ee493700 1 --2- 172.21.15.40:0/553096714 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd6e8071f70 0x7fd6e81540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.222+0000 7fd6ee493700 1 -- 172.21.15.40:0/553096714 >> 172.21.15.40:0/553096714 conn(0x7fd6e80cabc0 msgr2=0x7fd6e814f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:04.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.222+0000 7fd6ee493700 1 -- 172.21.15.40:0/553096714 shutdown_connections 2024-09-13T17:51:04.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.222+0000 7fd6ee493700 1 -- 172.21.15.40:0/553096714 wait complete. 2024-09-13T17:51:04.232 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-13T17:51:04.243 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-09-13T17:51:03.005575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235816,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738904,"statfs":{"total":773094113280,"available":772852637696,"internally_reserved":0,"allocated":3317760,"data_stored":2005562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2231,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004830"},"pg_stats":[{"pgid":"2.1f","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.584829+0000","last_change":"2024-09-13T17:50:34.212158+0000","last_active":"2024-09-13T17:50:39.584829+0000","last_peered":"2024-09-13T17:50:39.584829+0000","last_clean":"2024-09-13T17:50:39.584829+0000","last_became_active":"2024-09-13T17:50:34.211867+0000","last_became_peered":"2024-09-13T17:50:34.211867+0000","last_unstale":"2024-09-13T17:50:39.584829+0000","last_undegraded":"2024-09-13T17:50:39.584829+0000","last_fullsized":"2024-09-13T17:50:39.584829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:14.671095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589172+0000","last_change":"2024-09-13T17:50:34.587396+0000","last_active":"2024-09-13T17:50:38.589172+0000","last_peered":"2024-09-13T17:50:38.589172+0000","last_clean":"2024-09-13T17:50:38.589172+0000","last_became_active":"2024-09-13T17:50:34.587108+0000","last_became_peered":"2024-09-13T17:50:34.587108+0000","last_unstale":"2024-09-13T17:50:38.589172+0000","last_undegraded":"2024-09-13T17:50:38.589172+0000","last_fullsized":"2024-09-13T17:50:38.589172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:39:16.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.995312+0000","last_change":"2024-09-13T17:50:38.615059+0000","last_active":"2024-09-13T17:50:38.995312+0000","last_peered":"2024-09-13T17:50:38.995312+0000","last_clean":"2024-09-13T17:50:38.995312+0000","last_became_active":"2024-09-13T17:50:38.614723+0000","last_became_peered":"2024-09-13T17:50:38.614723+0000","last_unstale":"2024-09-13T17:50:38.995312+0000","last_undegraded":"2024-09-13T17:50:38.995312+0000","last_fullsized":"2024-09-13T17:50:38.995312+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:23:01.902181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"25'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.996807+0000","last_change":"2024-09-13T17:50:34.204760+0000","last_active":"2024-09-13T17:50:38.996807+0000","last_peered":"2024-09-13T17:50:38.996807+0000","last_clean":"2024-09-13T17:50:38.996807+0000","last_became_active":"2024-09-13T17:50:34.202930+0000","last_became_peered":"2024-09-13T17:50:34.202930+0000","last_unstale":"2024-09-13T17:50:38.996807+0000","last_undegraded":"2024-09-13T17:50:38.996807+0000","last_fullsized":"2024-09-13T17:50:38.996807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:59.784701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.598529+0000","last_change":"2024-09-13T17:50:34.583174+0000","last_active":"2024-09-13T17:50:38.598529+0000","last_peered":"2024-09-13T17:50:38.598529+0000","last_clean":"2024-09-13T17:50:38.598529+0000","last_became_active":"2024-09-13T17:50:34.582884+0000","last_became_peered":"2024-09-13T17:50:34.582884+0000","last_unstale":"2024-09-13T17:50:38.598529+0000","last_undegraded":"2024-09-13T17:50:38.598529+0000","last_fullsized":"2024-09-13T17:50:38.598529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:22:27.539574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501034+0000","last_change":"2024-09-13T17:50:38.618751+0000","last_active":"2024-09-13T17:50:39.501034+0000","last_peered":"2024-09-13T17:50:39.501034+0000","last_clean":"2024-09-13T17:50:39.501034+0000","last_became_active":"2024-09-13T17:50:38.618423+0000","last_became_peered":"2024-09-13T17:50:38.618423+0000","last_unstale":"2024-09-13T17:50:39.501034+0000","last_undegraded":"2024-09-13T17:50:39.501034+0000","last_fullsized":"2024-09-13T17:50:39.501034+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:50:02.418785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.582559+0000","last_change":"2024-09-13T17:50:34.214972+0000","last_active":"2024-09-13T17:50:39.582559+0000","last_peered":"2024-09-13T17:50:39.582559+0000","last_clean":"2024-09-13T17:50:39.582559+0000","last_became_active":"2024-09-13T17:50:34.214652+0000","last_became_peered":"2024-09-13T17:50:34.214652+0000","last_unstale":"2024-09-13T17:50:39.582559+0000","last_undegraded":"2024-09-13T17:50:39.582559+0000","last_fullsized":"2024-09-13T17:50:39.582559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:57.176854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590539+0000","last_change":"2024-09-13T17:50:34.594493+0000","last_active":"2024-09-13T17:50:38.590539+0000","last_peered":"2024-09-13T17:50:38.590539+0000","last_clean":"2024-09-13T17:50:38.590539+0000","last_became_active":"2024-09-13T17:50:34.594182+0000","last_became_peered":"2024-09-13T17:50:34.594182+0000","last_unstale":"2024-09-13T17:50:38.590539+0000","last_undegraded":"2024-09-13T17:50:38.590539+0000","last_fullsized":"2024-09-13T17:50:38.590539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:18.059925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501399+0000","last_change":"2024-09-13T17:50:38.616219+0000","last_active":"2024-09-13T17:50:39.501399+0000","last_peered":"2024-09-13T17:50:39.501399+0000","last_clean":"2024-09-13T17:50:39.501399+0000","last_became_active":"2024-09-13T17:50:38.614329+0000","last_became_peered":"2024-09-13T17:50:38.614329+0000","last_unstale":"2024-09-13T17:50:39.501399+0000","last_undegraded":"2024-09-13T17:50:39.501399+0000","last_fullsized":"2024-09-13T17:50:39.501399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:25.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583345+0000","last_change":"2024-09-13T17:50:34.216123+0000","last_active":"2024-09-13T17:50:39.583345+0000","last_peered":"2024-09-13T17:50:39.583345+0000","last_clean":"2024-09-13T17:50:39.583345+0000","last_became_active":"2024-09-13T17:50:34.215661+0000","last_became_peered":"2024-09-13T17:50:34.215661+0000","last_unstale":"2024-09-13T17:50:39.583345+0000","last_undegraded":"2024-09-13T17:50:39.583345+0000","last_fullsized":"2024-09-13T17:50:39.583345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:20.895037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591506+0000","last_change":"2024-09-13T17:50:34.599673+0000","last_active":"2024-09-13T17:50:38.591506+0000","last_peered":"2024-09-13T17:50:38.591506+0000","last_clean":"2024-09-13T17:50:38.591506+0000","last_became_active":"2024-09-13T17:50:34.599409+0000","last_became_peered":"2024-09-13T17:50:34.599409+0000","last_unstale":"2024-09-13T17:50:38.591506+0000","last_undegraded":"2024-09-13T17:50:38.591506+0000","last_fullsized":"2024-09-13T17:50:38.591506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:03:31.394385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501275+0000","last_change":"2024-09-13T17:50:38.616317+0000","last_active":"2024-09-13T17:50:39.501275+0000","last_peered":"2024-09-13T17:50:39.501275+0000","last_clean":"2024-09-13T17:50:39.501275+0000","last_became_active":"2024-09-13T17:50:38.614311+0000","last_became_peered":"2024-09-13T17:50:38.614311+0000","last_unstale":"2024-09-13T17:50:39.501275+0000","last_undegraded":"2024-09-13T17:50:39.501275+0000","last_fullsized":"2024-09-13T17:50:39.501275+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:29.221522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.604632+0000","last_change":"2024-09-13T17:50:34.599490+0000","last_active":"2024-09-13T17:50:38.604632+0000","last_peered":"2024-09-13T17:50:38.604632+0000","last_clean":"2024-09-13T17:50:38.604632+0000","last_became_active":"2024-09-13T17:50:34.599207+0000","last_became_peered":"2024-09-13T17:50:34.599207+0000","last_unstale":"2024-09-13T17:50:38.604632+0000","last_undegraded":"2024-09-13T17:50:38.604632+0000","last_fullsized":"2024-09-13T17:50:38.604632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:11.178161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583429+0000","last_change":"2024-09-13T17:50:34.215697+0000","last_active":"2024-09-13T17:50:39.583429+0000","last_peered":"2024-09-13T17:50:39.583429+0000","last_clean":"2024-09-13T17:50:39.583429+0000","last_became_active":"2024-09-13T17:50:34.215409+0000","last_became_peered":"2024-09-13T17:50:34.215409+0000","last_unstale":"2024-09-13T17:50:39.583429+0000","last_undegraded":"2024-09-13T17:50:39.583429+0000","last_fullsized":"2024-09-13T17:50:39.583429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:21.609567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.502325+0000","last_change":"2024-09-13T17:50:38.611817+0000","last_active":"2024-09-13T17:50:39.502325+0000","last_peered":"2024-09-13T17:50:39.502325+0000","last_clean":"2024-09-13T17:50:39.502325+0000","last_became_active":"2024-09-13T17:50:38.611263+0000","last_became_peered":"2024-09-13T17:50:38.611263+0000","last_unstale":"2024-09-13T17:50:39.502325+0000","last_undegraded":"2024-09-13T17:50:39.502325+0000","last_fullsized":"2024-09-13T17:50:39.502325+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:56:47.466672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.588598+0000","last_change":"2024-09-13T17:50:34.586307+0000","last_active":"2024-09-13T17:50:38.588598+0000","last_peered":"2024-09-13T17:50:38.588598+0000","last_clean":"2024-09-13T17:50:38.588598+0000","last_became_active":"2024-09-13T17:50:34.585831+0000","last_became_peered":"2024-09-13T17:50:34.585831+0000","last_unstale":"2024-09-13T17:50:38.588598+0000","last_undegraded":"2024-09-13T17:50:38.588598+0000","last_fullsized":"2024-09-13T17:50:38.588598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:40.912352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599468+0000","last_change":"2024-09-13T17:50:34.211850+0000","last_active":"2024-09-13T17:50:38.599468+0000","last_peered":"2024-09-13T17:50:38.599468+0000","last_clean":"2024-09-13T17:50:38.599468+0000","last_became_active":"2024-09-13T17:50:34.211594+0000","last_became_peered":"2024-09-13T17:50:34.211594+0000","last_unstale":"2024-09-13T17:50:38.599468+0000","last_undegraded":"2024-09-13T17:50:38.599468+0000","last_fullsized":"2024-09-13T17:50:38.599468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:35:02.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.502004+0000","last_change":"2024-09-13T17:50:38.613576+0000","last_active":"2024-09-13T17:50:39.502004+0000","last_peered":"2024-09-13T17:50:39.502004+0000","last_clean":"2024-09-13T17:50:39.502004+0000","last_became_active":"2024-09-13T17:50:38.612577+0000","last_became_peered":"2024-09-13T17:50:38.612577+0000","last_unstale":"2024-09-13T17:50:39.502004+0000","last_undegraded":"2024-09-13T17:50:39.502004+0000","last_fullsized":"2024-09-13T17:50:39.502004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:25.551060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590056+0000","last_change":"2024-09-13T17:50:34.587273+0000","last_active":"2024-09-13T17:50:38.590056+0000","last_peered":"2024-09-13T17:50:38.590056+0000","last_clean":"2024-09-13T17:50:38.590056+0000","last_became_active":"2024-09-13T17:50:34.587027+0000","last_became_peered":"2024-09-13T17:50:34.587027+0000","last_unstale":"2024-09-13T17:50:38.590056+0000","last_undegraded":"2024-09-13T17:50:38.590056+0000","last_fullsized":"2024-09-13T17:50:38.590056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:58.826737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599621+0000","last_change":"2024-09-13T17:50:34.211845+0000","last_active":"2024-09-13T17:50:38.599621+0000","last_peered":"2024-09-13T17:50:38.599621+0000","last_clean":"2024-09-13T17:50:38.599621+0000","last_became_active":"2024-09-13T17:50:34.211568+0000","last_became_peered":"2024-09-13T17:50:34.211568+0000","last_unstale":"2024-09-13T17:50:38.599621+0000","last_undegraded":"2024-09-13T17:50:38.599621+0000","last_fullsized":"2024-09-13T17:50:38.599621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:56.348984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000608+0000","last_change":"2024-09-13T17:50:38.610292+0000","last_active":"2024-09-13T17:50:39.000608+0000","last_peered":"2024-09-13T17:50:39.000608+0000","last_clean":"2024-09-13T17:50:39.000608+0000","last_became_active":"2024-09-13T17:50:38.609832+0000","last_became_peered":"2024-09-13T17:50:38.609832+0000","last_unstale":"2024-09-13T17:50:39.000608+0000","last_undegraded":"2024-09-13T17:50:39.000608+0000","last_fullsized":"2024-09-13T17:50:39.000608+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:13.783068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.603557+0000","last_change":"2024-09-13T17:50:34.593566+0000","last_active":"2024-09-13T17:50:38.603557+0000","last_peered":"2024-09-13T17:50:38.603557+0000","last_clean":"2024-09-13T17:50:38.603557+0000","last_became_active":"2024-09-13T17:50:34.588794+0000","last_became_peered":"2024-09-13T17:50:34.588794+0000","last_unstale":"2024-09-13T17:50:38.603557+0000","last_undegraded":"2024-09-13T17:50:38.603557+0000","last_fullsized":"2024-09-13T17:50:38.603557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:13.192453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589828+0000","last_change":"2024-09-13T17:50:34.214638+0000","last_active":"2024-09-13T17:50:38.589828+0000","last_peered":"2024-09-13T17:50:38.589828+0000","last_clean":"2024-09-13T17:50:38.589828+0000","last_became_active":"2024-09-13T17:50:34.214292+0000","last_became_peered":"2024-09-13T17:50:34.214292+0000","last_unstale":"2024-09-13T17:50:38.589828+0000","last_undegraded":"2024-09-13T17:50:38.589828+0000","last_fullsized":"2024-09-13T17:50:38.589828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:57.377104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501996+0000","last_change":"2024-09-13T17:50:38.613368+0000","last_active":"2024-09-13T17:50:39.501996+0000","last_peered":"2024-09-13T17:50:39.501996+0000","last_clean":"2024-09-13T17:50:39.501996+0000","last_became_active":"2024-09-13T17:50:38.613010+0000","last_became_peered":"2024-09-13T17:50:38.613010+0000","last_unstale":"2024-09-13T17:50:39.501996+0000","last_undegraded":"2024-09-13T17:50:39.501996+0000","last_fullsized":"2024-09-13T17:50:39.501996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:42.706625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.604457+0000","last_change":"2024-09-13T17:50:34.582327+0000","last_active":"2024-09-13T17:50:38.604457+0000","last_peered":"2024-09-13T17:50:38.604457+0000","last_clean":"2024-09-13T17:50:38.604457+0000","last_became_active":"2024-09-13T17:50:34.582050+0000","last_became_peered":"2024-09-13T17:50:34.582050+0000","last_unstale":"2024-09-13T17:50:38.604457+0000","last_undegraded":"2024-09-13T17:50:38.604457+0000","last_fullsized":"2024-09-13T17:50:38.604457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:52.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583664+0000","last_change":"2024-09-13T17:50:34.211512+0000","last_active":"2024-09-13T17:50:39.583664+0000","last_peered":"2024-09-13T17:50:39.583664+0000","last_clean":"2024-09-13T17:50:39.583664+0000","last_became_active":"2024-09-13T17:50:34.211230+0000","last_became_peered":"2024-09-13T17:50:34.211230+0000","last_unstale":"2024-09-13T17:50:39.583664+0000","last_undegraded":"2024-09-13T17:50:39.583664+0000","last_fullsized":"2024-09-13T17:50:39.583664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:40.707777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501244+0000","last_change":"2024-09-13T17:50:38.608255+0000","last_active":"2024-09-13T17:50:39.501244+0000","last_peered":"2024-09-13T17:50:39.501244+0000","last_clean":"2024-09-13T17:50:39.501244+0000","last_became_active":"2024-09-13T17:50:38.606697+0000","last_became_peered":"2024-09-13T17:50:38.606697+0000","last_unstale":"2024-09-13T17:50:39.501244+0000","last_undegraded":"2024-09-13T17:50:39.501244+0000","last_fullsized":"2024-09-13T17:50:39.501244+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:47.265007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591328+0000","last_change":"2024-09-13T17:50:34.594476+0000","last_active":"2024-09-13T17:50:38.591328+0000","last_peered":"2024-09-13T17:50:38.591328+0000","last_clean":"2024-09-13T17:50:38.591328+0000","last_became_active":"2024-09-13T17:50:34.594215+0000","last_became_peered":"2024-09-13T17:50:34.594215+0000","last_unstale":"2024-09-13T17:50:38.591328+0000","last_undegraded":"2024-09-13T17:50:38.591328+0000","last_fullsized":"2024-09-13T17:50:38.591328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:02:27.528019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000656+0000","last_change":"2024-09-13T17:50:34.204018+0000","last_active":"2024-09-13T17:50:39.000656+0000","last_peered":"2024-09-13T17:50:39.000656+0000","last_clean":"2024-09-13T17:50:39.000656+0000","last_became_active":"2024-09-13T17:50:34.203702+0000","last_became_peered":"2024-09-13T17:50:34.203702+0000","last_unstale":"2024-09-13T17:50:39.000656+0000","last_undegraded":"2024-09-13T17:50:39.000656+0000","last_fullsized":"2024-09-13T17:50:39.000656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:44.431226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.995065+0000","last_change":"2024-09-13T17:50:38.615677+0000","last_active":"2024-09-13T17:50:38.995065+0000","last_peered":"2024-09-13T17:50:38.995065+0000","last_clean":"2024-09-13T17:50:38.995065+0000","last_became_active":"2024-09-13T17:50:38.615270+0000","last_became_peered":"2024-09-13T17:50:38.615270+0000","last_unstale":"2024-09-13T17:50:38.995065+0000","last_undegraded":"2024-09-13T17:50:38.995065+0000","last_fullsized":"2024-09-13T17:50:38.995065+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:55.768090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599593+0000","last_change":"2024-09-13T17:50:34.586802+0000","last_active":"2024-09-13T17:50:38.599593+0000","last_peered":"2024-09-13T17:50:38.599593+0000","last_clean":"2024-09-13T17:50:38.599593+0000","last_became_active":"2024-09-13T17:50:34.586552+0000","last_became_peered":"2024-09-13T17:50:34.586552+0000","last_unstale":"2024-09-13T17:50:38.599593+0000","last_undegraded":"2024-09-13T17:50:38.599593+0000","last_fullsized":"2024-09-13T17:50:38.599593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:43.383975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.592964+0000","last_change":"2024-09-13T17:50:34.217083+0000","last_active":"2024-09-13T17:50:38.592964+0000","last_peered":"2024-09-13T17:50:38.592964+0000","last_clean":"2024-09-13T17:50:38.592964+0000","last_became_active":"2024-09-13T17:50:34.216758+0000","last_became_peered":"2024-09-13T17:50:34.216758+0000","last_unstale":"2024-09-13T17:50:38.592964+0000","last_undegraded":"2024-09-13T17:50:38.592964+0000","last_fullsized":"2024-09-13T17:50:38.592964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:36.473942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.500902+0000","last_change":"2024-09-13T17:50:38.614667+0000","last_active":"2024-09-13T17:50:39.500902+0000","last_peered":"2024-09-13T17:50:39.500902+0000","last_clean":"2024-09-13T17:50:39.500902+0000","last_became_active":"2024-09-13T17:50:38.614284+0000","last_became_peered":"2024-09-13T17:50:38.614284+0000","last_unstale":"2024-09-13T17:50:39.500902+0000","last_undegraded":"2024-09-13T17:50:39.500902+0000","last_fullsized":"2024-09-13T17:50:39.500902+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:29.616794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591705+0000","last_change":"2024-09-13T17:50:34.588603+0000","last_active":"2024-09-13T17:50:38.591705+0000","last_peered":"2024-09-13T17:50:38.591705+0000","last_clean":"2024-09-13T17:50:38.591705+0000","last_became_active":"2024-09-13T17:50:34.588317+0000","last_became_peered":"2024-09-13T17:50:34.588317+0000","last_unstale":"2024-09-13T17:50:38.591705+0000","last_undegraded":"2024-09-13T17:50:38.591705+0000","last_fullsized":"2024-09-13T17:50:38.591705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:03.743130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599679+0000","last_change":"2024-09-13T17:50:34.204335+0000","last_active":"2024-09-13T17:50:38.599679+0000","last_peered":"2024-09-13T17:50:38.599679+0000","last_clean":"2024-09-13T17:50:38.599679+0000","last_became_active":"2024-09-13T17:50:34.204060+0000","last_became_peered":"2024-09-13T17:50:34.204060+0000","last_unstale":"2024-09-13T17:50:38.599679+0000","last_undegraded":"2024-09-13T17:50:38.599679+0000","last_fullsized":"2024-09-13T17:50:38.599679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:54:41.831344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.999822+0000","last_change":"2024-09-13T17:50:38.619590+0000","last_active":"2024-09-13T17:50:38.999822+0000","last_peered":"2024-09-13T17:50:38.999822+0000","last_clean":"2024-09-13T17:50:38.999822+0000","last_became_active":"2024-09-13T17:50:38.617257+0000","last_became_peered":"2024-09-13T17:50:38.617257+0000","last_unstale":"2024-09-13T17:50:38.999822+0000","last_undegraded":"2024-09-13T17:50:38.999822+0000","last_fullsized":"2024-09-13T17:50:38.999822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:34.870291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.602091+0000","last_change":"2024-09-13T17:50:34.588466+0000","last_active":"2024-09-13T17:50:38.602091+0000","last_peered":"2024-09-13T17:50:38.602091+0000","last_clean":"2024-09-13T17:50:38.602091+0000","last_became_active":"2024-09-13T17:50:34.588063+0000","last_became_peered":"2024-09-13T17:50:34.588063+0000","last_unstale":"2024-09-13T17:50:38.602091+0000","last_undegraded":"2024-09-13T17:50:38.602091+0000","last_fullsized":"2024-09-13T17:50:38.602091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:43:00.669203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589993+0000","last_change":"2024-09-13T17:50:34.209239+0000","last_active":"2024-09-13T17:50:38.589993+0000","last_peered":"2024-09-13T17:50:38.589993+0000","last_clean":"2024-09-13T17:50:38.589993+0000","last_became_active":"2024-09-13T17:50:34.208196+0000","last_became_peered":"2024-09-13T17:50:34.208196+0000","last_unstale":"2024-09-13T17:50:38.589993+0000","last_undegraded":"2024-09-13T17:50:38.589993+0000","last_fullsized":"2024-09-13T17:50:38.589993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:39.253917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000394+0000","last_change":"2024-09-13T17:50:38.611021+0000","last_active":"2024-09-13T17:50:39.000394+0000","last_peered":"2024-09-13T17:50:39.000394+0000","last_clean":"2024-09-13T17:50:39.000394+0000","last_became_active":"2024-09-13T17:50:38.609794+0000","last_became_peered":"2024-09-13T17:50:38.609794+0000","last_unstale":"2024-09-13T17:50:39.000394+0000","last_undegraded":"2024-09-13T17:50:39.000394+0000","last_fullsized":"2024-09-13T17:50:39.000394+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:59.980236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.600038+0000","last_change":"2024-09-13T17:50:34.584290+0000","last_active":"2024-09-13T17:50:38.600038+0000","last_peered":"2024-09-13T17:50:38.600038+0000","last_clean":"2024-09-13T17:50:38.600038+0000","last_became_active":"2024-09-13T17:50:34.584023+0000","last_became_peered":"2024-09-13T17:50:34.584023+0000","last_unstale":"2024-09-13T17:50:38.600038+0000","last_undegraded":"2024-09-13T17:50:38.600038+0000","last_fullsized":"2024-09-13T17:50:38.600038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:11.304986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.597167+0000","last_change":"2024-09-13T17:50:34.198932+0000","last_active":"2024-09-13T17:50:38.597167+0000","last_peered":"2024-09-13T17:50:38.597167+0000","last_clean":"2024-09-13T17:50:38.597167+0000","last_became_active":"2024-09-13T17:50:34.198603+0000","last_became_peered":"2024-09-13T17:50:34.198603+0000","last_unstale":"2024-09-13T17:50:38.597167+0000","last_undegraded":"2024-09-13T17:50:38.597167+0000","last_fullsized":"2024-09-13T17:50:38.597167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:00.917126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000707+0000","last_change":"2024-09-13T17:50:38.611326+0000","last_active":"2024-09-13T17:50:39.000707+0000","last_peered":"2024-09-13T17:50:39.000707+0000","last_clean":"2024-09-13T17:50:39.000707+0000","last_became_active":"2024-09-13T17:50:38.610900+0000","last_became_peered":"2024-09-13T17:50:38.610900+0000","last_unstale":"2024-09-13T17:50:39.000707+0000","last_undegraded":"2024-09-13T17:50:39.000707+0000","last_fullsized":"2024-09-13T17:50:39.000707+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:17:09.994736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.605384+0000","last_change":"2024-09-13T17:50:34.594482+0000","last_active":"2024-09-13T17:50:38.605384+0000","last_peered":"2024-09-13T17:50:38.605384+0000","last_clean":"2024-09-13T17:50:38.605384+0000","last_became_active":"2024-09-13T17:50:34.594201+0000","last_became_peered":"2024-09-13T17:50:34.594201+0000","last_unstale":"2024-09-13T17:50:38.605384+0000","last_undegraded":"2024-09-13T17:50:38.605384+0000","last_fullsized":"2024-09-13T17:50:38.605384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:06.697507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590063+0000","last_change":"2024-09-13T17:50:34.218376+0000","last_active":"2024-09-13T17:50:38.590063+0000","last_peered":"2024-09-13T17:50:38.590063+0000","last_clean":"2024-09-13T17:50:38.590063+0000","last_became_active":"2024-09-13T17:50:34.218069+0000","last_became_peered":"2024-09-13T17:50:34.218069+0000","last_unstale":"2024-09-13T17:50:38.590063+0000","last_undegraded":"2024-09-13T17:50:38.590063+0000","last_fullsized":"2024-09-13T17:50:38.590063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:53:54.288969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501171+0000","last_change":"2024-09-13T17:50:38.613206+0000","last_active":"2024-09-13T17:50:39.501171+0000","last_peered":"2024-09-13T17:50:39.501171+0000","last_clean":"2024-09-13T17:50:39.501171+0000","last_became_active":"2024-09-13T17:50:38.612856+0000","last_became_peered":"2024-09-13T17:50:38.612856+0000","last_unstale":"2024-09-13T17:50:39.501171+0000","last_undegraded":"2024-09-13T17:50:39.501171+0000","last_fullsized":"2024-09-13T17:50:39.501171+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:31:20.190996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.588765+0000","last_change":"2024-09-13T17:50:34.582931+0000","last_active":"2024-09-13T17:50:38.588765+0000","last_peered":"2024-09-13T17:50:38.588765+0000","last_clean":"2024-09-13T17:50:38.588765+0000","last_became_active":"2024-09-13T17:50:34.582446+0000","last_became_peered":"2024-09-13T17:50:34.582446+0000","last_unstale":"2024-09-13T17:50:38.588765+0000","last_undegraded":"2024-09-13T17:50:38.588765+0000","last_fullsized":"2024-09-13T17:50:38.588765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:30.554785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.588978+0000","last_change":"2024-09-13T17:50:34.201237+0000","last_active":"2024-09-13T17:50:38.588978+0000","last_peered":"2024-09-13T17:50:38.588978+0000","last_clean":"2024-09-13T17:50:38.588978+0000","last_became_active":"2024-09-13T17:50:34.200873+0000","last_became_peered":"2024-09-13T17:50:34.200873+0000","last_unstale":"2024-09-13T17:50:38.588978+0000","last_undegraded":"2024-09-13T17:50:38.588978+0000","last_fullsized":"2024-09-13T17:50:38.588978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:02.711342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590078+0000","last_change":"2024-09-13T17:50:22.095367+0000","last_active":"2024-09-13T17:50:38.590078+0000","last_peered":"2024-09-13T17:50:38.590078+0000","last_clean":"2024-09-13T17:50:38.590078+0000","last_became_active":"2024-09-13T17:50:22.093734+0000","last_became_peered":"2024-09-13T17:50:22.093734+0000","last_unstale":"2024-09-13T17:50:38.590078+0000","last_undegraded":"2024-09-13T17:50:38.590078+0000","last_fullsized":"2024-09-13T17:50:38.590078+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_clean_scrub_stamp":"2024-09-13T17:50:21.057812+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:23.531613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.982380+0000","last_change":"2024-09-13T17:50:38.618880+0000","last_active":"2024-09-13T17:50:38.982380+0000","last_peered":"2024-09-13T17:50:38.982380+0000","last_clean":"2024-09-13T17:50:38.982380+0000","last_became_active":"2024-09-13T17:50:38.618488+0000","last_became_peered":"2024-09-13T17:50:38.618488+0000","last_unstale":"2024-09-13T17:50:38.982380+0000","last_undegraded":"2024-09-13T17:50:38.982380+0000","last_fullsized":"2024-09-13T17:50:38.982380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:03.431876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.592679+0000","last_change":"2024-09-13T17:50:34.585881+0000","last_active":"2024-09-13T17:50:38.592679+0000","last_peered":"2024-09-13T17:50:38.592679+0000","last_clean":"2024-09-13T17:50:38.592679+0000","last_became_active":"2024-09-13T17:50:34.585166+0000","last_became_peered":"2024-09-13T17:50:34.585166+0000","last_unstale":"2024-09-13T17:50:38.592679+0000","last_undegraded":"2024-09-13T17:50:38.592679+0000","last_fullsized":"2024-09-13T17:50:38.592679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:06.654658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.602557+0000","last_change":"2024-09-13T17:50:34.210934+0000","last_active":"2024-09-13T17:50:38.602557+0000","last_peered":"2024-09-13T17:50:38.602557+0000","last_clean":"2024-09-13T17:50:38.602557+0000","last_became_active":"2024-09-13T17:50:34.210589+0000","last_became_peered":"2024-09-13T17:50:34.210589+0000","last_unstale":"2024-09-13T17:50:38.602557+0000","last_undegraded":"2024-09-13T17:50:38.602557+0000","last_fullsized":"2024-09-13T17:50:38.602557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:26.213295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.982569+0000","last_change":"2024-09-13T17:50:38.615811+0000","last_active":"2024-09-13T17:50:38.982569+0000","last_peered":"2024-09-13T17:50:38.982569+0000","last_clean":"2024-09-13T17:50:38.982569+0000","last_became_active":"2024-09-13T17:50:38.615358+0000","last_became_peered":"2024-09-13T17:50:38.615358+0000","last_unstale":"2024-09-13T17:50:38.982569+0000","last_undegraded":"2024-09-13T17:50:38.982569+0000","last_fullsized":"2024-09-13T17:50:38.982569+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:05.667393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590751+0000","last_change":"2024-09-13T17:50:34.590169+0000","last_active":"2024-09-13T17:50:38.590751+0000","last_peered":"2024-09-13T17:50:38.590751+0000","last_clean":"2024-09-13T17:50:38.590751+0000","last_became_active":"2024-09-13T17:50:34.589434+0000","last_became_peered":"2024-09-13T17:50:34.589434+0000","last_unstale":"2024-09-13T17:50:38.590751+0000","last_undegraded":"2024-09-13T17:50:38.590751+0000","last_fullsized":"2024-09-13T17:50:38.590751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:12.545412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000847+0000","last_change":"2024-09-13T17:50:34.204269+0000","last_active":"2024-09-13T17:50:39.000847+0000","last_peered":"2024-09-13T17:50:39.000847+0000","last_clean":"2024-09-13T17:50:39.000847+0000","last_became_active":"2024-09-13T17:50:34.203874+0000","last_became_peered":"2024-09-13T17:50:34.203874+0000","last_unstale":"2024-09-13T17:50:39.000847+0000","last_undegraded":"2024-09-13T17:50:39.000847+0000","last_fullsized":"2024-09-13T17:50:39.000847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:10.410267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501092+0000","last_change":"2024-09-13T17:50:38.615989+0000","last_active":"2024-09-13T17:50:39.501092+0000","last_peered":"2024-09-13T17:50:39.501092+0000","last_clean":"2024-09-13T17:50:39.501092+0000","last_became_active":"2024-09-13T17:50:38.615530+0000","last_became_peered":"2024-09-13T17:50:38.615530+0000","last_unstale":"2024-09-13T17:50:39.501092+0000","last_undegraded":"2024-09-13T17:50:39.501092+0000","last_fullsized":"2024-09-13T17:50:39.501092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:41.064092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.593217+0000","last_change":"2024-09-13T17:50:34.595120+0000","last_active":"2024-09-13T17:50:38.593217+0000","last_peered":"2024-09-13T17:50:38.593217+0000","last_clean":"2024-09-13T17:50:38.593217+0000","last_became_active":"2024-09-13T17:50:34.594842+0000","last_became_peered":"2024-09-13T17:50:34.594842+0000","last_unstale":"2024-09-13T17:50:38.593217+0000","last_undegraded":"2024-09-13T17:50:38.593217+0000","last_fullsized":"2024-09-13T17:50:38.593217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:15:38.982524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.587423+0000","last_change":"2024-09-13T17:50:34.199957+0000","last_active":"2024-09-13T17:50:39.587423+0000","last_peered":"2024-09-13T17:50:39.587423+0000","last_clean":"2024-09-13T17:50:39.587423+0000","last_became_active":"2024-09-13T17:50:34.199650+0000","last_became_peered":"2024-09-13T17:50:34.199650+0000","last_unstale":"2024-09-13T17:50:39.587423+0000","last_undegraded":"2024-09-13T17:50:39.587423+0000","last_fullsized":"2024-09-13T17:50:39.587423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:26.929549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.193989+0000","last_change":"2024-09-13T17:50:38.609731+0000","last_active":"2024-09-13T17:50:39.193989+0000","last_peered":"2024-09-13T17:50:39.193989+0000","last_clean":"2024-09-13T17:50:39.193989+0000","last_became_active":"2024-09-13T17:50:38.609354+0000","last_became_peered":"2024-09-13T17:50:38.609354+0000","last_unstale":"2024-09-13T17:50:39.193989+0000","last_undegraded":"2024-09-13T17:50:39.193989+0000","last_fullsized":"2024-09-13T17:50:39.193989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:14.142369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.593339+0000","last_change":"2024-09-13T17:50:34.595146+0000","last_active":"2024-09-13T17:50:38.593339+0000","last_peered":"2024-09-13T17:50:38.593339+0000","last_clean":"2024-09-13T17:50:38.593339+0000","last_became_active":"2024-09-13T17:50:34.594876+0000","last_became_peered":"2024-09-13T17:50:34.594876+0000","last_unstale":"2024-09-13T17:50:38.593339+0000","last_undegraded":"2024-09-13T17:50:38.593339+0000","last_fullsized":"2024-09-13T17:50:38.593339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:26:10.638438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.502430+0000","last_change":"2024-09-13T17:50:34.200640+0000","last_active":"2024-09-13T17:50:39.502430+0000","last_peered":"2024-09-13T17:50:39.502430+0000","last_clean":"2024-09-13T17:50:39.502430+0000","last_became_active":"2024-09-13T17:50:34.200197+0000","last_became_peered":"2024-09-13T17:50:34.200197+0000","last_unstale":"2024-09-13T17:50:39.502430+0000","last_undegraded":"2024-09-13T17:50:39.502430+0000","last_fullsized":"2024-09-13T17:50:39.502430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:37:53.039720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.500911+0000","last_change":"2024-09-13T17:50:38.619880+0000","last_active":"2024-09-13T17:50:39.500911+0000","last_peered":"2024-09-13T17:50:39.500911+0000","last_clean":"2024-09-13T17:50:39.500911+0000","last_became_active":"2024-09-13T17:50:38.618491+0000","last_became_peered":"2024-09-13T17:50:38.618491+0000","last_unstale":"2024-09-13T17:50:39.500911+0000","last_undegraded":"2024-09-13T17:50:39.500911+0000","last_fullsized":"2024-09-13T17:50:39.500911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:10:14.136054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590688+0000","last_change":"2024-09-13T17:50:34.585741+0000","last_active":"2024-09-13T17:50:38.590688+0000","last_peered":"2024-09-13T17:50:38.590688+0000","last_clean":"2024-09-13T17:50:38.590688+0000","last_became_active":"2024-09-13T17:50:34.585464+0000","last_became_peered":"2024-09-13T17:50:34.585464+0000","last_unstale":"2024-09-13T17:50:38.590688+0000","last_undegraded":"2024-09-13T17:50:38.590688+0000","last_fullsized":"2024-09-13T17:50:38.590688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:59:22.031459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"25'8","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.585286+0000","last_change":"2024-09-13T17:50:34.201272+0000","last_active":"2024-09-13T17:50:39.585286+0000","last_peered":"2024-09-13T17:50:39.585286+0000","last_clean":"2024-09-13T17:50:39.585286+0000","last_became_active":"2024-09-13T17:50:34.200936+0000","last_became_peered":"2024-09-13T17:50:34.200936+0000","last_unstale":"2024-09-13T17:50:39.585286+0000","last_undegraded":"2024-09-13T17:50:39.585286+0000","last_fullsized":"2024-09-13T17:50:39.585286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:52.094879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.500719+0000","last_change":"2024-09-13T17:50:38.615229+0000","last_active":"2024-09-13T17:50:39.500719+0000","last_peered":"2024-09-13T17:50:39.500719+0000","last_clean":"2024-09-13T17:50:39.500719+0000","last_became_active":"2024-09-13T17:50:38.614739+0000","last_became_peered":"2024-09-13T17:50:38.614739+0000","last_unstale":"2024-09-13T17:50:39.500719+0000","last_undegraded":"2024-09-13T17:50:39.500719+0000","last_fullsized":"2024-09-13T17:50:39.500719+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:18.619529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.604486+0000","last_change":"2024-09-13T17:50:34.588848+0000","last_active":"2024-09-13T17:50:38.604486+0000","last_peered":"2024-09-13T17:50:38.604486+0000","last_clean":"2024-09-13T17:50:38.604486+0000","last_became_active":"2024-09-13T17:50:34.588565+0000","last_became_peered":"2024-09-13T17:50:34.588565+0000","last_unstale":"2024-09-13T17:50:38.604486+0000","last_undegraded":"2024-09-13T17:50:38.604486+0000","last_fullsized":"2024-09-13T17:50:38.604486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:47.927865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.604169+0000","last_change":"2024-09-13T17:50:34.199008+0000","last_active":"2024-09-13T17:50:38.604169+0000","last_peered":"2024-09-13T17:50:38.604169+0000","last_clean":"2024-09-13T17:50:38.604169+0000","last_became_active":"2024-09-13T17:50:34.197839+0000","last_became_peered":"2024-09-13T17:50:34.197839+0000","last_unstale":"2024-09-13T17:50:38.604169+0000","last_undegraded":"2024-09-13T17:50:38.604169+0000","last_fullsized":"2024-09-13T17:50:38.604169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:51.430710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.996853+0000","last_change":"2024-09-13T17:50:38.611080+0000","last_active":"2024-09-13T17:50:38.996853+0000","last_peered":"2024-09-13T17:50:38.996853+0000","last_clean":"2024-09-13T17:50:38.996853+0000","last_became_active":"2024-09-13T17:50:38.610688+0000","last_became_peered":"2024-09-13T17:50:38.610688+0000","last_unstale":"2024-09-13T17:50:38.996853+0000","last_undegraded":"2024-09-13T17:50:38.996853+0000","last_fullsized":"2024-09-13T17:50:38.996853+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:28.527617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591242+0000","last_change":"2024-09-13T17:50:34.594763+0000","last_active":"2024-09-13T17:50:38.591242+0000","last_peered":"2024-09-13T17:50:38.591242+0000","last_clean":"2024-09-13T17:50:38.591242+0000","last_became_active":"2024-09-13T17:50:34.594471+0000","last_became_peered":"2024-09-13T17:50:34.594471+0000","last_unstale":"2024-09-13T17:50:38.591242+0000","last_undegraded":"2024-09-13T17:50:38.591242+0000","last_fullsized":"2024-09-13T17:50:38.591242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:20:58.718271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589771+0000","last_change":"2024-09-13T17:50:34.209432+0000","last_active":"2024-09-13T17:50:38.589771+0000","last_peered":"2024-09-13T17:50:38.589771+0000","last_clean":"2024-09-13T17:50:38.589771+0000","last_became_active":"2024-09-13T17:50:34.208817+0000","last_became_peered":"2024-09-13T17:50:34.208817+0000","last_unstale":"2024-09-13T17:50:38.589771+0000","last_undegraded":"2024-09-13T17:50:38.589771+0000","last_fullsized":"2024-09-13T17:50:38.589771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:49.587528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.995813+0000","last_change":"2024-09-13T17:50:38.609461+0000","last_active":"2024-09-13T17:50:38.995813+0000","last_peered":"2024-09-13T17:50:38.995813+0000","last_clean":"2024-09-13T17:50:38.995813+0000","last_became_active":"2024-09-13T17:50:38.609128+0000","last_became_peered":"2024-09-13T17:50:38.609128+0000","last_unstale":"2024-09-13T17:50:38.995813+0000","last_undegraded":"2024-09-13T17:50:38.995813+0000","last_fullsized":"2024-09-13T17:50:38.995813+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:02.286825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.604049+0000","last_change":"2024-09-13T17:50:34.592970+0000","last_active":"2024-09-13T17:50:38.604049+0000","last_peered":"2024-09-13T17:50:38.604049+0000","last_clean":"2024-09-13T17:50:38.604049+0000","last_became_active":"2024-09-13T17:50:34.592669+0000","last_became_peered":"2024-09-13T17:50:34.592669+0000","last_unstale":"2024-09-13T17:50:38.604049+0000","last_undegraded":"2024-09-13T17:50:38.604049+0000","last_fullsized":"2024-09-13T17:50:38.604049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:52.031261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583358+0000","last_change":"2024-09-13T17:50:34.200798+0000","last_active":"2024-09-13T17:50:39.583358+0000","last_peered":"2024-09-13T17:50:39.583358+0000","last_clean":"2024-09-13T17:50:39.583358+0000","last_became_active":"2024-09-13T17:50:34.200529+0000","last_became_peered":"2024-09-13T17:50:34.200529+0000","last_unstale":"2024-09-13T17:50:39.583358+0000","last_undegraded":"2024-09-13T17:50:39.583358+0000","last_fullsized":"2024-09-13T17:50:39.583358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:26.334296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.195285+0000","last_change":"2024-09-13T17:50:38.606287+0000","last_active":"2024-09-13T17:50:39.195285+0000","last_peered":"2024-09-13T17:50:39.195285+0000","last_clean":"2024-09-13T17:50:39.195285+0000","last_became_active":"2024-09-13T17:50:38.605826+0000","last_became_peered":"2024-09-13T17:50:38.605826+0000","last_unstale":"2024-09-13T17:50:39.195285+0000","last_undegraded":"2024-09-13T17:50:39.195285+0000","last_fullsized":"2024-09-13T17:50:39.195285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:18.709454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.592530+0000","last_change":"2024-09-13T17:50:34.591579+0000","last_active":"2024-09-13T17:50:38.592530+0000","last_peered":"2024-09-13T17:50:38.592530+0000","last_clean":"2024-09-13T17:50:38.592530+0000","last_became_active":"2024-09-13T17:50:34.586165+0000","last_became_peered":"2024-09-13T17:50:34.586165+0000","last_unstale":"2024-09-13T17:50:38.592530+0000","last_undegraded":"2024-09-13T17:50:38.592530+0000","last_fullsized":"2024-09-13T17:50:38.592530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:56:19.205176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:47.801705+0000","last_change":"2024-09-13T17:50:34.203218+0000","last_active":"2024-09-13T17:50:47.801705+0000","last_peered":"2024-09-13T17:50:47.801705+0000","last_clean":"2024-09-13T17:50:47.801705+0000","last_became_active":"2024-09-13T17:50:34.202948+0000","last_became_peered":"2024-09-13T17:50:34.202948+0000","last_unstale":"2024-09-13T17:50:47.801705+0000","last_undegraded":"2024-09-13T17:50:47.801705+0000","last_fullsized":"2024-09-13T17:50:47.801705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:46.535569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000203+0000","last_change":"2024-09-13T17:50:38.610597+0000","last_active":"2024-09-13T17:50:39.000203+0000","last_peered":"2024-09-13T17:50:39.000203+0000","last_clean":"2024-09-13T17:50:39.000203+0000","last_became_active":"2024-09-13T17:50:38.610055+0000","last_became_peered":"2024-09-13T17:50:38.610055+0000","last_unstale":"2024-09-13T17:50:39.000203+0000","last_undegraded":"2024-09-13T17:50:39.000203+0000","last_fullsized":"2024-09-13T17:50:39.000203+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:55.754936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.602441+0000","last_change":"2024-09-13T17:50:34.593963+0000","last_active":"2024-09-13T17:50:38.602441+0000","last_peered":"2024-09-13T17:50:38.602441+0000","last_clean":"2024-09-13T17:50:38.602441+0000","last_became_active":"2024-09-13T17:50:34.593454+0000","last_became_peered":"2024-09-13T17:50:34.593454+0000","last_unstale":"2024-09-13T17:50:38.602441+0000","last_undegraded":"2024-09-13T17:50:38.602441+0000","last_fullsized":"2024-09-13T17:50:38.602441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:18.883521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583588+0000","last_change":"2024-09-13T17:50:34.207936+0000","last_active":"2024-09-13T17:50:39.583588+0000","last_peered":"2024-09-13T17:50:39.583588+0000","last_clean":"2024-09-13T17:50:39.583588+0000","last_became_active":"2024-09-13T17:50:34.207605+0000","last_became_peered":"2024-09-13T17:50:34.207605+0000","last_unstale":"2024-09-13T17:50:39.583588+0000","last_undegraded":"2024-09-13T17:50:39.583588+0000","last_fullsized":"2024-09-13T17:50:39.583588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:05.494406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501828+0000","last_change":"2024-09-13T17:50:38.616297+0000","last_active":"2024-09-13T17:50:39.501828+0000","last_peered":"2024-09-13T17:50:39.501828+0000","last_clean":"2024-09-13T17:50:39.501828+0000","last_became_active":"2024-09-13T17:50:38.614165+0000","last_became_peered":"2024-09-13T17:50:38.614165+0000","last_unstale":"2024-09-13T17:50:39.501828+0000","last_undegraded":"2024-09-13T17:50:39.501828+0000","last_fullsized":"2024-09-13T17:50:39.501828+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:11.357026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.598628+0000","last_change":"2024-09-13T17:50:34.583001+0000","last_active":"2024-09-13T17:50:38.598628+0000","last_peered":"2024-09-13T17:50:38.598628+0000","last_clean":"2024-09-13T17:50:38.598628+0000","last_became_active":"2024-09-13T17:50:34.582529+0000","last_became_peered":"2024-09-13T17:50:34.582529+0000","last_unstale":"2024-09-13T17:50:38.598628+0000","last_undegraded":"2024-09-13T17:50:38.598628+0000","last_fullsized":"2024-09-13T17:50:38.598628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:05.287181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591968+0000","last_change":"2024-09-13T17:50:34.205138+0000","last_active":"2024-09-13T17:50:38.591968+0000","last_peered":"2024-09-13T17:50:38.591968+0000","last_clean":"2024-09-13T17:50:38.591968+0000","last_became_active":"2024-09-13T17:50:34.203360+0000","last_became_peered":"2024-09-13T17:50:34.203360+0000","last_unstale":"2024-09-13T17:50:38.591968+0000","last_undegraded":"2024-09-13T17:50:38.591968+0000","last_fullsized":"2024-09-13T17:50:38.591968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:35.756963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.995168+0000","last_change":"2024-09-13T17:50:38.611036+0000","last_active":"2024-09-13T17:50:38.995168+0000","last_peered":"2024-09-13T17:50:38.995168+0000","last_clean":"2024-09-13T17:50:38.995168+0000","last_became_active":"2024-09-13T17:50:38.610692+0000","last_became_peered":"2024-09-13T17:50:38.610692+0000","last_unstale":"2024-09-13T17:50:38.995168+0000","last_undegraded":"2024-09-13T17:50:38.995168+0000","last_fullsized":"2024-09-13T17:50:38.995168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:10.147998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591770+0000","last_change":"2024-09-13T17:50:34.596882+0000","last_active":"2024-09-13T17:50:38.591770+0000","last_peered":"2024-09-13T17:50:38.591770+0000","last_clean":"2024-09-13T17:50:38.591770+0000","last_became_active":"2024-09-13T17:50:34.596495+0000","last_became_peered":"2024-09-13T17:50:34.596495+0000","last_unstale":"2024-09-13T17:50:38.591770+0000","last_undegraded":"2024-09-13T17:50:38.591770+0000","last_fullsized":"2024-09-13T17:50:38.591770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:21.441114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.609505+0000","last_change":"2024-09-13T17:50:34.204376+0000","last_active":"2024-09-13T17:50:38.609505+0000","last_peered":"2024-09-13T17:50:38.609505+0000","last_clean":"2024-09-13T17:50:38.609505+0000","last_became_active":"2024-09-13T17:50:34.204003+0000","last_became_peered":"2024-09-13T17:50:34.204003+0000","last_unstale":"2024-09-13T17:50:38.609505+0000","last_undegraded":"2024-09-13T17:50:38.609505+0000","last_fullsized":"2024-09-13T17:50:38.609505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:14.023990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.995437+0000","last_change":"2024-09-13T17:50:38.606144+0000","last_active":"2024-09-13T17:50:38.995437+0000","last_peered":"2024-09-13T17:50:38.995437+0000","last_clean":"2024-09-13T17:50:38.995437+0000","last_became_active":"2024-09-13T17:50:38.605646+0000","last_became_peered":"2024-09-13T17:50:38.605646+0000","last_unstale":"2024-09-13T17:50:38.995437+0000","last_undegraded":"2024-09-13T17:50:38.995437+0000","last_fullsized":"2024-09-13T17:50:38.995437+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:07.120075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.500554+0000","last_change":"2024-09-13T17:50:38.618855+0000","last_active":"2024-09-13T17:50:39.500554+0000","last_peered":"2024-09-13T17:50:39.500554+0000","last_clean":"2024-09-13T17:50:39.500554+0000","last_became_active":"2024-09-13T17:50:38.618449+0000","last_became_peered":"2024-09-13T17:50:38.618449+0000","last_unstale":"2024-09-13T17:50:39.500554+0000","last_undegraded":"2024-09-13T17:50:39.500554+0000","last_fullsized":"2024-09-13T17:50:39.500554+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:29:01.114087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.600195+0000","last_change":"2024-09-13T17:50:34.590079+0000","last_active":"2024-09-13T17:50:38.600195+0000","last_peered":"2024-09-13T17:50:38.600195+0000","last_clean":"2024-09-13T17:50:38.600195+0000","last_became_active":"2024-09-13T17:50:34.589396+0000","last_became_peered":"2024-09-13T17:50:34.589396+0000","last_unstale":"2024-09-13T17:50:38.600195+0000","last_undegraded":"2024-09-13T17:50:38.600195+0000","last_fullsized":"2024-09-13T17:50:38.600195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:03.308341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589925+0000","last_change":"2024-09-13T17:50:34.206575+0000","last_active":"2024-09-13T17:50:38.589925+0000","last_peered":"2024-09-13T17:50:38.589925+0000","last_clean":"2024-09-13T17:50:38.589925+0000","last_became_active":"2024-09-13T17:50:34.206308+0000","last_became_peered":"2024-09-13T17:50:34.206308+0000","last_unstale":"2024-09-13T17:50:38.589925+0000","last_undegraded":"2024-09-13T17:50:38.589925+0000","last_fullsized":"2024-09-13T17:50:38.589925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:34.766538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.982710+0000","last_change":"2024-09-13T17:50:38.618131+0000","last_active":"2024-09-13T17:50:38.982710+0000","last_peered":"2024-09-13T17:50:38.982710+0000","last_clean":"2024-09-13T17:50:38.982710+0000","last_became_active":"2024-09-13T17:50:38.617611+0000","last_became_peered":"2024-09-13T17:50:38.617611+0000","last_unstale":"2024-09-13T17:50:38.982710+0000","last_undegraded":"2024-09-13T17:50:38.982710+0000","last_fullsized":"2024-09-13T17:50:38.982710+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:30.746445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589814+0000","last_change":"2024-09-13T17:50:34.582153+0000","last_active":"2024-09-13T17:50:38.589814+0000","last_peered":"2024-09-13T17:50:38.589814+0000","last_clean":"2024-09-13T17:50:38.589814+0000","last_became_active":"2024-09-13T17:50:34.581708+0000","last_became_peered":"2024-09-13T17:50:34.581708+0000","last_unstale":"2024-09-13T17:50:38.589814+0000","last_undegraded":"2024-09-13T17:50:38.589814+0000","last_fullsized":"2024-09-13T17:50:38.589814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:22.238810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590175+0000","last_change":"2024-09-13T17:50:34.205489+0000","last_active":"2024-09-13T17:50:38.590175+0000","last_peered":"2024-09-13T17:50:38.590175+0000","last_clean":"2024-09-13T17:50:38.590175+0000","last_became_active":"2024-09-13T17:50:34.205202+0000","last_became_peered":"2024-09-13T17:50:34.205202+0000","last_unstale":"2024-09-13T17:50:38.590175+0000","last_undegraded":"2024-09-13T17:50:38.590175+0000","last_fullsized":"2024-09-13T17:50:38.590175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:54.450276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.983249+0000","last_change":"2024-09-13T17:50:34.202470+0000","last_active":"2024-09-13T17:50:38.983249+0000","last_peered":"2024-09-13T17:50:38.983249+0000","last_clean":"2024-09-13T17:50:38.983249+0000","last_became_active":"2024-09-13T17:50:34.202163+0000","last_became_peered":"2024-09-13T17:50:34.202163+0000","last_unstale":"2024-09-13T17:50:38.983249+0000","last_undegraded":"2024-09-13T17:50:38.983249+0000","last_fullsized":"2024-09-13T17:50:38.983249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:03.472932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.603393+0000","last_change":"2024-09-13T17:50:34.590947+0000","last_active":"2024-09-13T17:50:38.603393+0000","last_peered":"2024-09-13T17:50:38.603393+0000","last_clean":"2024-09-13T17:50:38.603393+0000","last_became_active":"2024-09-13T17:50:34.590497+0000","last_became_peered":"2024-09-13T17:50:34.590497+0000","last_unstale":"2024-09-13T17:50:38.603393+0000","last_undegraded":"2024-09-13T17:50:38.603393+0000","last_fullsized":"2024-09-13T17:50:38.603393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:56:27.848092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.980209+0000","last_change":"2024-09-13T17:50:38.618518+0000","last_active":"2024-09-13T17:50:38.980209+0000","last_peered":"2024-09-13T17:50:38.980209+0000","last_clean":"2024-09-13T17:50:38.980209+0000","last_became_active":"2024-09-13T17:50:38.618108+0000","last_became_peered":"2024-09-13T17:50:38.618108+0000","last_unstale":"2024-09-13T17:50:38.980209+0000","last_undegraded":"2024-09-13T17:50:38.980209+0000","last_fullsized":"2024-09-13T17:50:38.980209+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:42.260237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"25'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.996384+0000","last_change":"2024-09-13T17:50:34.218459+0000","last_active":"2024-09-13T17:50:38.996384+0000","last_peered":"2024-09-13T17:50:38.996384+0000","last_clean":"2024-09-13T17:50:38.996384+0000","last_became_active":"2024-09-13T17:50:34.218087+0000","last_became_peered":"2024-09-13T17:50:34.218087+0000","last_unstale":"2024-09-13T17:50:38.996384+0000","last_undegraded":"2024-09-13T17:50:38.996384+0000","last_fullsized":"2024-09-13T17:50:38.996384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:56.914544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.603583+0000","last_change":"2024-09-13T17:50:34.583845+0000","last_active":"2024-09-13T17:50:38.603583+0000","last_peered":"2024-09-13T17:50:38.603583+0000","last_clean":"2024-09-13T17:50:38.603583+0000","last_became_active":"2024-09-13T17:50:34.583283+0000","last_became_peered":"2024-09-13T17:50:34.583283+0000","last_unstale":"2024-09-13T17:50:38.603583+0000","last_undegraded":"2024-09-13T17:50:38.603583+0000","last_fullsized":"2024-09-13T17:50:38.603583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:55.630285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.996396+0000","last_change":"2024-09-13T17:50:38.611987+0000","last_active":"2024-09-13T17:50:38.996396+0000","last_peered":"2024-09-13T17:50:38.996396+0000","last_clean":"2024-09-13T17:50:38.996396+0000","last_became_active":"2024-09-13T17:50:38.611636+0000","last_became_peered":"2024-09-13T17:50:38.611636+0000","last_unstale":"2024-09-13T17:50:38.996396+0000","last_undegraded":"2024-09-13T17:50:38.996396+0000","last_fullsized":"2024-09-13T17:50:38.996396+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:41.111562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.602963+0000","last_change":"2024-09-13T17:50:34.588927+0000","last_active":"2024-09-13T17:50:38.602963+0000","last_peered":"2024-09-13T17:50:38.602963+0000","last_clean":"2024-09-13T17:50:38.602963+0000","last_became_active":"2024-09-13T17:50:34.588651+0000","last_became_peered":"2024-09-13T17:50:34.588651+0000","last_unstale":"2024-09-13T17:50:38.602963+0000","last_undegraded":"2024-09-13T17:50:38.602963+0000","last_fullsized":"2024-09-13T17:50:38.602963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:49.429670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.597005+0000","last_change":"2024-09-13T17:50:34.583795+0000","last_active":"2024-09-13T17:50:38.597005+0000","last_peered":"2024-09-13T17:50:38.597005+0000","last_clean":"2024-09-13T17:50:38.597005+0000","last_became_active":"2024-09-13T17:50:34.583493+0000","last_became_peered":"2024-09-13T17:50:34.583493+0000","last_unstale":"2024-09-13T17:50:38.597005+0000","last_undegraded":"2024-09-13T17:50:38.597005+0000","last_fullsized":"2024-09-13T17:50:38.597005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:30.463932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589528+0000","last_change":"2024-09-13T17:50:34.587624+0000","last_active":"2024-09-13T17:50:38.589528+0000","last_peered":"2024-09-13T17:50:38.589528+0000","last_clean":"2024-09-13T17:50:38.589528+0000","last_became_active":"2024-09-13T17:50:34.587119+0000","last_became_peered":"2024-09-13T17:50:34.587119+0000","last_unstale":"2024-09-13T17:50:38.589528+0000","last_undegraded":"2024-09-13T17:50:38.589528+0000","last_fullsized":"2024-09-13T17:50:38.589528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:16.783899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599861+0000","last_change":"2024-09-13T17:50:34.581923+0000","last_active":"2024-09-13T17:50:38.599861+0000","last_peered":"2024-09-13T17:50:38.599861+0000","last_clean":"2024-09-13T17:50:38.599861+0000","last_became_active":"2024-09-13T17:50:34.581628+0000","last_became_peered":"2024-09-13T17:50:34.581628+0000","last_unstale":"2024-09-13T17:50:38.599861+0000","last_undegraded":"2024-09-13T17:50:38.599861+0000","last_fullsized":"2024-09-13T17:50:38.599861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:28:55.936864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589257+0000","last_change":"2024-09-13T17:50:34.595037+0000","last_active":"2024-09-13T17:50:38.589257+0000","last_peered":"2024-09-13T17:50:38.589257+0000","last_clean":"2024-09-13T17:50:38.589257+0000","last_became_active":"2024-09-13T17:50:34.594857+0000","last_became_peered":"2024-09-13T17:50:34.594857+0000","last_unstale":"2024-09-13T17:50:38.589257+0000","last_undegraded":"2024-09-13T17:50:38.589257+0000","last_fullsized":"2024-09-13T17:50:38.589257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:11.420413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.597995+0000","last_change":"2024-09-13T17:50:34.589651+0000","last_active":"2024-09-13T17:50:38.597995+0000","last_peered":"2024-09-13T17:50:38.597995+0000","last_clean":"2024-09-13T17:50:38.597995+0000","last_became_active":"2024-09-13T17:50:34.589354+0000","last_became_peered":"2024-09-13T17:50:34.589354+0000","last_unstale":"2024-09-13T17:50:38.597995+0000","last_undegraded":"2024-09-13T17:50:38.597995+0000","last_fullsized":"2024-09-13T17:50:38.597995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:31.294048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.609987+0000","last_change":"2024-09-13T17:50:34.597949+0000","last_active":"2024-09-13T17:50:38.609987+0000","last_peered":"2024-09-13T17:50:38.609987+0000","last_clean":"2024-09-13T17:50:38.609987+0000","last_became_active":"2024-09-13T17:50:34.597621+0000","last_became_peered":"2024-09-13T17:50:34.597621+0000","last_unstale":"2024-09-13T17:50:38.609987+0000","last_undegraded":"2024-09-13T17:50:38.609987+0000","last_fullsized":"2024-09-13T17:50:38.609987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:23.133462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591133+0000","last_change":"2024-09-13T17:50:34.589893+0000","last_active":"2024-09-13T17:50:38.591133+0000","last_peered":"2024-09-13T17:50:38.591133+0000","last_clean":"2024-09-13T17:50:38.591133+0000","last_became_active":"2024-09-13T17:50:34.589626+0000","last_became_peered":"2024-09-13T17:50:34.589626+0000","last_unstale":"2024-09-13T17:50:38.591133+0000","last_undegraded":"2024-09-13T17:50:38.591133+0000","last_fullsized":"2024-09-13T17:50:38.591133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:41.321025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589874+0000","last_change":"2024-09-13T17:50:34.590046+0000","last_active":"2024-09-13T17:50:38.589874+0000","last_peered":"2024-09-13T17:50:38.589874+0000","last_clean":"2024-09-13T17:50:38.589874+0000","last_became_active":"2024-09-13T17:50:34.589757+0000","last_became_peered":"2024-09-13T17:50:34.589757+0000","last_unstale":"2024-09-13T17:50:38.589874+0000","last_undegraded":"2024-09-13T17:50:38.589874+0000","last_fullsized":"2024-09-13T17:50:38.589874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:28.239550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589800+0000","last_change":"2024-09-13T17:50:34.595518+0000","last_active":"2024-09-13T17:50:38.589800+0000","last_peered":"2024-09-13T17:50:38.589800+0000","last_clean":"2024-09-13T17:50:38.589800+0000","last_became_active":"2024-09-13T17:50:34.595242+0000","last_became_peered":"2024-09-13T17:50:34.595242+0000","last_unstale":"2024-09-13T17:50:38.589800+0000","last_undegraded":"2024-09-13T17:50:38.589800+0000","last_fullsized":"2024-09-13T17:50:38.589800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:26.149384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589330+0000","last_change":"2024-09-13T17:50:34.593183+0000","last_active":"2024-09-13T17:50:38.589330+0000","last_peered":"2024-09-13T17:50:38.589330+0000","last_clean":"2024-09-13T17:50:38.589330+0000","last_became_active":"2024-09-13T17:50:34.592893+0000","last_became_peered":"2024-09-13T17:50:34.592893+0000","last_unstale":"2024-09-13T17:50:38.589330+0000","last_undegraded":"2024-09-13T17:50:38.589330+0000","last_fullsized":"2024-09-13T17:50:38.589330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:19.459505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599698+0000","last_change":"2024-09-13T17:50:34.594679+0000","last_active":"2024-09-13T17:50:38.599698+0000","last_peered":"2024-09-13T17:50:38.599698+0000","last_clean":"2024-09-13T17:50:38.599698+0000","last_became_active":"2024-09-13T17:50:34.593750+0000","last_became_peered":"2024-09-13T17:50:34.593750+0000","last_unstale":"2024-09-13T17:50:38.599698+0000","last_undegraded":"2024-09-13T17:50:38.599698+0000","last_fullsized":"2024-09-13T17:50:38.599698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:02.135577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.609111+0000","last_change":"2024-09-13T17:50:34.598736+0000","last_active":"2024-09-13T17:50:38.609111+0000","last_peered":"2024-09-13T17:50:38.609111+0000","last_clean":"2024-09-13T17:50:38.609111+0000","last_became_active":"2024-09-13T17:50:34.598433+0000","last_became_peered":"2024-09-13T17:50:34.598433+0000","last_unstale":"2024-09-13T17:50:38.609111+0000","last_undegraded":"2024-09-13T17:50:38.609111+0000","last_fullsized":"2024-09-13T17:50:38.609111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:08.975139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.592906+0000","last_change":"2024-09-13T17:50:34.591795+0000","last_active":"2024-09-13T17:50:38.592906+0000","last_peered":"2024-09-13T17:50:38.592906+0000","last_clean":"2024-09-13T17:50:38.592906+0000","last_became_active":"2024-09-13T17:50:34.591286+0000","last_became_peered":"2024-09-13T17:50:34.591286+0000","last_unstale":"2024-09-13T17:50:38.592906+0000","last_undegraded":"2024-09-13T17:50:38.592906+0000","last_fullsized":"2024-09-13T17:50:38.592906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:31.580247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.595844+0000","last_change":"2024-09-13T17:50:34.587774+0000","last_active":"2024-09-13T17:50:38.595844+0000","last_peered":"2024-09-13T17:50:38.595844+0000","last_clean":"2024-09-13T17:50:38.595844+0000","last_became_active":"2024-09-13T17:50:34.587506+0000","last_became_peered":"2024-09-13T17:50:34.587506+0000","last_unstale":"2024-09-13T17:50:38.595844+0000","last_undegraded":"2024-09-13T17:50:38.595844+0000","last_fullsized":"2024-09-13T17:50:38.595844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:46.275528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591171+0000","last_change":"2024-09-13T17:50:34.596100+0000","last_active":"2024-09-13T17:50:38.591171+0000","last_peered":"2024-09-13T17:50:38.591171+0000","last_clean":"2024-09-13T17:50:38.591171+0000","last_became_active":"2024-09-13T17:50:34.595798+0000","last_became_peered":"2024-09-13T17:50:34.595798+0000","last_unstale":"2024-09-13T17:50:38.591171+0000","last_undegraded":"2024-09-13T17:50:38.591171+0000","last_fullsized":"2024-09-13T17:50:38.591171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:49.543845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589759+0000","last_change":"2024-09-13T17:50:34.584818+0000","last_active":"2024-09-13T17:50:38.589759+0000","last_peered":"2024-09-13T17:50:38.589759+0000","last_clean":"2024-09-13T17:50:38.589759+0000","last_became_active":"2024-09-13T17:50:34.584516+0000","last_became_peered":"2024-09-13T17:50:34.584516+0000","last_unstale":"2024-09-13T17:50:38.589759+0000","last_undegraded":"2024-09-13T17:50:38.589759+0000","last_fullsized":"2024-09-13T17:50:38.589759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:52.402024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599531+0000","last_change":"2024-09-13T17:50:34.598886+0000","last_active":"2024-09-13T17:50:38.599531+0000","last_peered":"2024-09-13T17:50:38.599531+0000","last_clean":"2024-09-13T17:50:38.599531+0000","last_became_active":"2024-09-13T17:50:34.598648+0000","last_became_peered":"2024-09-13T17:50:34.598648+0000","last_unstale":"2024-09-13T17:50:38.599531+0000","last_undegraded":"2024-09-13T17:50:38.599531+0000","last_fullsized":"2024-09-13T17:50:38.599531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:00:43.063223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.603939+0000","last_change":"2024-09-13T17:50:34.582649+0000","last_active":"2024-09-13T17:50:38.603939+0000","last_peered":"2024-09-13T17:50:38.603939+0000","last_clean":"2024-09-13T17:50:38.603939+0000","last_became_active":"2024-09-13T17:50:34.581699+0000","last_became_peered":"2024-09-13T17:50:34.581699+0000","last_unstale":"2024-09-13T17:50:38.603939+0000","last_undegraded":"2024-09-13T17:50:38.603939+0000","last_fullsized":"2024-09-13T17:50:38.603939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:18.959303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.588487+0000","last_change":"2024-09-13T17:50:34.587484+0000","last_active":"2024-09-13T17:50:38.588487+0000","last_peered":"2024-09-13T17:50:38.588487+0000","last_clean":"2024-09-13T17:50:38.588487+0000","last_became_active":"2024-09-13T17:50:34.587217+0000","last_became_peered":"2024-09-13T17:50:34.587217+0000","last_unstale":"2024-09-13T17:50:38.588487+0000","last_undegraded":"2024-09-13T17:50:38.588487+0000","last_fullsized":"2024-09-13T17:50:38.588487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:24:19.172607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.602349+0000","last_change":"2024-09-13T17:50:34.592871+0000","last_active":"2024-09-13T17:50:38.602349+0000","last_peered":"2024-09-13T17:50:38.602349+0000","last_clean":"2024-09-13T17:50:38.602349+0000","last_became_active":"2024-09-13T17:50:34.588513+0000","last_became_peered":"2024-09-13T17:50:34.588513+0000","last_unstale":"2024-09-13T17:50:38.602349+0000","last_undegraded":"2024-09-13T17:50:38.602349+0000","last_fullsized":"2024-09-13T17:50:38.602349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:33.131420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.608986+0000","last_change":"2024-09-13T17:50:34.585713+0000","last_active":"2024-09-13T17:50:38.608986+0000","last_peered":"2024-09-13T17:50:38.608986+0000","last_clean":"2024-09-13T17:50:38.608986+0000","last_became_active":"2024-09-13T17:50:34.585438+0000","last_became_peered":"2024-09-13T17:50:34.585438+0000","last_unstale":"2024-09-13T17:50:38.608986+0000","last_undegraded":"2024-09-13T17:50:38.608986+0000","last_fullsized":"2024-09-13T17:50:38.608986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:56:49.905393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.609903+0000","last_change":"2024-09-13T17:50:34.589628+0000","last_active":"2024-09-13T17:50:38.609903+0000","last_peered":"2024-09-13T17:50:38.609903+0000","last_clean":"2024-09-13T17:50:38.609903+0000","last_became_active":"2024-09-13T17:50:34.587665+0000","last_became_peered":"2024-09-13T17:50:34.587665+0000","last_unstale":"2024-09-13T17:50:38.609903+0000","last_undegraded":"2024-09-13T17:50:38.609903+0000","last_fullsized":"2024-09-13T17:50:38.609903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:15.518264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.593116+0000","last_change":"2024-09-13T17:50:34.591776+0000","last_active":"2024-09-13T17:50:38.593116+0000","last_peered":"2024-09-13T17:50:38.593116+0000","last_clean":"2024-09-13T17:50:38.593116+0000","last_became_active":"2024-09-13T17:50:34.591496+0000","last_became_peered":"2024-09-13T17:50:34.591496+0000","last_unstale":"2024-09-13T17:50:38.593116+0000","last_undegraded":"2024-09-13T17:50:38.593116+0000","last_fullsized":"2024-09-13T17:50:38.593116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:48.651999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.588146+0000","last_change":"2024-09-13T17:50:34.582661+0000","last_active":"2024-09-13T17:50:38.588146+0000","last_peered":"2024-09-13T17:50:38.588146+0000","last_clean":"2024-09-13T17:50:38.588146+0000","last_became_active":"2024-09-13T17:50:34.582400+0000","last_became_peered":"2024-09-13T17:50:34.582400+0000","last_unstale":"2024-09-13T17:50:38.588146+0000","last_undegraded":"2024-09-13T17:50:38.588146+0000","last_fullsized":"2024-09-13T17:50:38.588146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:16:59.648638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589018+0000","last_change":"2024-09-13T17:50:34.585531+0000","last_active":"2024-09-13T17:50:38.589018+0000","last_peered":"2024-09-13T17:50:38.589018+0000","last_clean":"2024-09-13T17:50:38.589018+0000","last_became_active":"2024-09-13T17:50:34.585226+0000","last_became_peered":"2024-09-13T17:50:34.585226+0000","last_unstale":"2024-09-13T17:50:38.589018+0000","last_undegraded":"2024-09-13T17:50:38.589018+0000","last_fullsized":"2024-09-13T17:50:38.589018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:17.105028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589126+0000","last_change":"2024-09-13T17:50:34.586750+0000","last_active":"2024-09-13T17:50:38.589126+0000","last_peered":"2024-09-13T17:50:38.589126+0000","last_clean":"2024-09-13T17:50:38.589126+0000","last_became_active":"2024-09-13T17:50:34.584564+0000","last_became_peered":"2024-09-13T17:50:34.584564+0000","last_unstale":"2024-09-13T17:50:38.589126+0000","last_undegraded":"2024-09-13T17:50:38.589126+0000","last_fullsized":"2024-09-13T17:50:38.589126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:54.033257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591038+0000","last_change":"2024-09-13T17:50:34.585938+0000","last_active":"2024-09-13T17:50:38.591038+0000","last_peered":"2024-09-13T17:50:38.591038+0000","last_clean":"2024-09-13T17:50:38.591038+0000","last_became_active":"2024-09-13T17:50:34.585584+0000","last_became_peered":"2024-09-13T17:50:34.585584+0000","last_unstale":"2024-09-13T17:50:38.591038+0000","last_undegraded":"2024-09-13T17:50:38.591038+0000","last_fullsized":"2024-09-13T17:50:38.591038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:32.623468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590323+0000","last_change":"2024-09-13T17:50:34.588762+0000","last_active":"2024-09-13T17:50:38.590323+0000","last_peered":"2024-09-13T17:50:38.590323+0000","last_clean":"2024-09-13T17:50:38.590323+0000","last_became_active":"2024-09-13T17:50:34.588489+0000","last_became_peered":"2024-09-13T17:50:34.588489+0000","last_unstale":"2024-09-13T17:50:38.590323+0000","last_undegraded":"2024-09-13T17:50:38.590323+0000","last_fullsized":"2024-09-13T17:50:38.590323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:38.266297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.598178+0000","last_change":"2024-09-13T17:50:34.583889+0000","last_active":"2024-09-13T17:50:38.598178+0000","last_peered":"2024-09-13T17:50:38.598178+0000","last_clean":"2024-09-13T17:50:38.598178+0000","last_became_active":"2024-09-13T17:50:34.583551+0000","last_became_peered":"2024-09-13T17:50:34.583551+0000","last_unstale":"2024-09-13T17:50:38.598178+0000","last_undegraded":"2024-09-13T17:50:38.598178+0000","last_fullsized":"2024-09-13T17:50:38.598178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:19.888626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599385+0000","last_change":"2024-09-13T17:50:34.585617+0000","last_active":"2024-09-13T17:50:38.599385+0000","last_peered":"2024-09-13T17:50:38.599385+0000","last_clean":"2024-09-13T17:50:38.599385+0000","last_became_active":"2024-09-13T17:50:34.585340+0000","last_became_peered":"2024-09-13T17:50:34.585340+0000","last_unstale":"2024-09-13T17:50:38.599385+0000","last_undegraded":"2024-09-13T17:50:38.599385+0000","last_fullsized":"2024-09-13T17:50:38.599385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:18.331045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.593186+0000","last_change":"2024-09-13T17:50:34.592365+0000","last_active":"2024-09-13T17:50:38.593186+0000","last_peered":"2024-09-13T17:50:38.593186+0000","last_clean":"2024-09-13T17:50:38.593186+0000","last_became_active":"2024-09-13T17:50:34.592046+0000","last_became_peered":"2024-09-13T17:50:34.592046+0000","last_unstale":"2024-09-13T17:50:38.593186+0000","last_undegraded":"2024-09-13T17:50:38.593186+0000","last_fullsized":"2024-09-13T17:50:38.593186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:54.283683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":105750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":258048,"data_stored":101955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":102115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509451,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":864256,"data_stored":692961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]}]},{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":856064,"data_stored":692905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":270336,"data_stored":105750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":262144,"data_stored":102045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.373}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]}]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":270336,"data_stored":102081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:51:04.251 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:51:04.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.362+0000 7f1f66b3c700 1 Processor -- start 2024-09-13T17:51:04.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.362+0000 7f1f66b3c700 1 -- start start 2024-09-13T17:51:04.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f66b3c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f600cbda0 0x7f1f6015d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:04.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f66b3c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f600cc180 0x7f1f600c6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:04.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f66b3c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f60070b00 0x7f1f600c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:04.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f66b3c700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f600b22c0 con 0x7f1f60070b00 2024-09-13T17:51:04.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f66b3c700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f600c8520 con 0x7f1f600cbda0 2024-09-13T17:51:04.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f66b3c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f600c7fa0 con 0x7f1f600cc180 2024-09-13T17:51:04.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f6633b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f60070b00 0x7f1f600c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:04.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f65339700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f600cc180 0x7f1f600c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:04.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f6633b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f60070b00 0x7f1f600c6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57802/0 (socket says 172.21.15.40:57802) 2024-09-13T17:51:04.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f6633b700 1 -- 172.21.15.40:0/110315778 learned_addr learned my addr 172.21.15.40:0/110315778 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:04.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f65b3a700 1 --2- 172.21.15.40:0/110315778 >> v2:172.21.15.43:3300/0 conn(0x7f1f600cbda0 0x7f1f6015d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:04.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f6633b700 1 -- 172.21.15.40:0/110315778 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f600cc180 msgr2=0x7f1f600c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:04.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f6633b700 1 --2- 172.21.15.40:0/110315778 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f600cc180 0x7f1f600c6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f6633b700 1 -- 172.21.15.40:0/110315778 >> v2:172.21.15.43:3300/0 conn(0x7f1f600cbda0 msgr2=0x7f1f6015d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:04.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f6633b700 1 --2- 172.21.15.40:0/110315778 >> v2:172.21.15.43:3300/0 conn(0x7f1f600cbda0 0x7f1f6015d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f6633b700 1 -- 172.21.15.40:0/110315778 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f60071e80 con 0x7f1f60070b00 2024-09-13T17:51:04.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f65339700 1 --2- 172.21.15.40:0/110315778 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f600cc180 0x7f1f600c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:04.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f6633b700 1 --2- 172.21.15.40:0/110315778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f60070b00 0x7f1f600c6dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1f6006f620 tx=0x7f1f5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20204d75d13251c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:04.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f64b38700 1 -- 172.21.15.40:0/110315778 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800e070 con 0x7f1f60070b00 2024-09-13T17:51:04.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f64b38700 1 -- 172.21.15.40:0/110315778 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5800b910 con 0x7f1f60070b00 2024-09-13T17:51:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.366+0000 7f1f64b38700 1 -- 172.21.15.40:0/110315778 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800bcc0 con 0x7f1f60070b00 2024-09-13T17:51:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.370+0000 7f1f66b3c700 1 -- 172.21.15.40:0/110315778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f60070b00 msgr2=0x7f1f600c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.370+0000 7f1f66b3c700 1 --2- 172.21.15.40:0/110315778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f60070b00 0x7f1f600c6dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1f6006f620 tx=0x7f1f5800b330 comp rx=0 tx=0).stop 2024-09-13T17:51:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.370+0000 7f1f66b3c700 1 -- 172.21.15.40:0/110315778 shutdown_connections 2024-09-13T17:51:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.370+0000 7f1f66b3c700 1 --2- 172.21.15.40:0/110315778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f60070b00 0x7f1f600c6dc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.370+0000 7f1f66b3c700 1 --2- 172.21.15.40:0/110315778 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f600cc180 0x7f1f600c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.370+0000 7f1f66b3c700 1 --2- 172.21.15.40:0/110315778 >> v2:172.21.15.43:3300/0 conn(0x7f1f600cbda0 0x7f1f6015d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.370+0000 7f1f66b3c700 1 -- 172.21.15.40:0/110315778 >> 172.21.15.40:0/110315778 conn(0x7f1f6006e6d0 msgr2=0x7f1f6006eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.370+0000 7f1f66b3c700 1 -- 172.21.15.40:0/110315778 shutdown_connections 2024-09-13T17:51:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.370+0000 7f1f66b3c700 1 -- 172.21.15.40:0/110315778 wait complete. 2024-09-13T17:51:04.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f66b3c700 1 Processor -- start 2024-09-13T17:51:04.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f66b3c700 1 -- start start 2024-09-13T17:51:04.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f66b3c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f60070b00 0x7f1f60154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:04.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f66b3c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f600cbda0 0x7f1f6014e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f66b3c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f600cc180 0x7f1f600c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f66b3c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6014e7e0 con 0x7f1f600cbda0 2024-09-13T17:51:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f66b3c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6014de10 con 0x7f1f60070b00 2024-09-13T17:51:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f66b3c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f600b1c80 con 0x7f1f600cc180 2024-09-13T17:51:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f65339700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f600cbda0 0x7f1f6014e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f65339700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f600cbda0 0x7f1f6014e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57808/0 (socket says 172.21.15.40:57808) 2024-09-13T17:51:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f65339700 1 -- 172.21.15.40:0/479491087 learned_addr learned my addr 172.21.15.40:0/479491087 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f65b3a700 1 --2- 172.21.15.40:0/479491087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f60070b00 0x7f1f60154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f6633b700 1 --2- 172.21.15.40:0/479491087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f600cc180 0x7f1f600c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f65339700 1 -- 172.21.15.40:0/479491087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f600cc180 msgr2=0x7f1f600c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f65339700 1 --2- 172.21.15.40:0/479491087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f600cc180 0x7f1f600c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f65339700 1 -- 172.21.15.40:0/479491087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f60070b00 msgr2=0x7f1f60154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:04.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f65339700 1 --2- 172.21.15.40:0/479491087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f60070b00 0x7f1f60154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f65339700 1 -- 172.21.15.40:0/479491087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f500094b0 con 0x7f1f600cbda0 2024-09-13T17:51:04.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f65339700 1 --2- 172.21.15.40:0/479491087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f600cbda0 0x7f1f6014e2a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000ebe0 tx=0x7f1f50009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:04.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f567fc700 1 -- 172.21.15.40:0/479491087 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5000c910 con 0x7f1f600cbda0 2024-09-13T17:51:04.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f567fc700 1 -- 172.21.15.40:0/479491087 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5000ca70 con 0x7f1f600cbda0 2024-09-13T17:51:04.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f66b3c700 1 -- 172.21.15.40:0/479491087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f58009b30 con 0x7f1f600cbda0 2024-09-13T17:51:04.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f567fc700 1 -- 172.21.15.40:0/479491087 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5000ce00 con 0x7f1f600cbda0 2024-09-13T17:51:04.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.374+0000 7f1f66b3c700 1 -- 172.21.15.40:0/479491087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6014c780 con 0x7f1f600cbda0 2024-09-13T17:51:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.378+0000 7f1f66b3c700 1 -- 172.21.15.40:0/479491087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f600c2b80 con 0x7f1f600cbda0 2024-09-13T17:51:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.378+0000 7f1f567fc700 1 -- 172.21.15.40:0/479491087 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1f50013470 con 0x7f1f600cbda0 2024-09-13T17:51:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.378+0000 7f1f567fc700 1 --2- 172.21.15.40:0/479491087 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f44066730 0x7f1f44068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.378+0000 7f1f567fc700 1 -- 172.21.15.40:0/479491087 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1f50084a80 con 0x7f1f600cbda0 2024-09-13T17:51:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.378+0000 7f1f65b3a700 1 --2- 172.21.15.40:0/479491087 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f44066730 0x7f1f44068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.378+0000 7f1f65b3a700 1 --2- 172.21.15.40:0/479491087 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f44066730 0x7f1f44068bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009b00 tx=0x7f1f5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:04.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.382+0000 7f1f567fc700 1 -- 172.21.15.40:0/479491087 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1f50051a70 con 0x7f1f600cbda0 2024-09-13T17:51:04.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.550+0000 7f1f66b3c700 1 -- 172.21.15.40:0/479491087 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1f600c8240 con 0x7f1f44066730 2024-09-13T17:51:04.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.566+0000 7f1f567fc700 1 -- 172.21.15.40:0/479491087 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345205 (secure 0 0 0) 0x7f1f5c011750 con 0x7f1f44066730 2024-09-13T17:51:04.578 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.574+0000 7f1f66b3c700 1 -- 172.21.15.40:0/479491087 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f44066730 msgr2=0x7f1f44068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.574+0000 7f1f66b3c700 1 --2- 172.21.15.40:0/479491087 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f44066730 0x7f1f44068bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009b00 tx=0x7f1f5c008040 comp rx=0 tx=0).stop 2024-09-13T17:51:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.574+0000 7f1f66b3c700 1 -- 172.21.15.40:0/479491087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f600cbda0 msgr2=0x7f1f6014e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.574+0000 7f1f66b3c700 1 --2- 172.21.15.40:0/479491087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f600cbda0 0x7f1f6014e2a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000ebe0 tx=0x7f1f50009ed0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.574+0000 7f1f66b3c700 1 -- 172.21.15.40:0/479491087 shutdown_connections 2024-09-13T17:51:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.574+0000 7f1f66b3c700 1 --2- 172.21.15.40:0/479491087 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f44066730 0x7f1f44068bf0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.574+0000 7f1f66b3c700 1 --2- 172.21.15.40:0/479491087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f600cc180 0x7f1f600c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.574+0000 7f1f66b3c700 1 --2- 172.21.15.40:0/479491087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f600cbda0 0x7f1f6014e2a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.574+0000 7f1f66b3c700 1 --2- 172.21.15.40:0/479491087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f60070b00 0x7f1f60154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.574+0000 7f1f66b3c700 1 -- 172.21.15.40:0/479491087 >> 172.21.15.40:0/479491087 conn(0x7f1f6006e6d0 msgr2=0x7f1f6006eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.574+0000 7f1f66b3c700 1 -- 172.21.15.40:0/479491087 shutdown_connections 2024-09-13T17:51:04.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.574+0000 7f1f66b3c700 1 -- 172.21.15.40:0/479491087 wait complete. 2024-09-13T17:51:04.582 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-13T17:51:04.594 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-09-13T17:51:03.005575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235816,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738904,"statfs":{"total":773094113280,"available":772852637696,"internally_reserved":0,"allocated":3317760,"data_stored":2005562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2231,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004830"},"pg_stats":[{"pgid":"2.1f","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.584829+0000","last_change":"2024-09-13T17:50:34.212158+0000","last_active":"2024-09-13T17:50:39.584829+0000","last_peered":"2024-09-13T17:50:39.584829+0000","last_clean":"2024-09-13T17:50:39.584829+0000","last_became_active":"2024-09-13T17:50:34.211867+0000","last_became_peered":"2024-09-13T17:50:34.211867+0000","last_unstale":"2024-09-13T17:50:39.584829+0000","last_undegraded":"2024-09-13T17:50:39.584829+0000","last_fullsized":"2024-09-13T17:50:39.584829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:14.671095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589172+0000","last_change":"2024-09-13T17:50:34.587396+0000","last_active":"2024-09-13T17:50:38.589172+0000","last_peered":"2024-09-13T17:50:38.589172+0000","last_clean":"2024-09-13T17:50:38.589172+0000","last_became_active":"2024-09-13T17:50:34.587108+0000","last_became_peered":"2024-09-13T17:50:34.587108+0000","last_unstale":"2024-09-13T17:50:38.589172+0000","last_undegraded":"2024-09-13T17:50:38.589172+0000","last_fullsized":"2024-09-13T17:50:38.589172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:39:16.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.995312+0000","last_change":"2024-09-13T17:50:38.615059+0000","last_active":"2024-09-13T17:50:38.995312+0000","last_peered":"2024-09-13T17:50:38.995312+0000","last_clean":"2024-09-13T17:50:38.995312+0000","last_became_active":"2024-09-13T17:50:38.614723+0000","last_became_peered":"2024-09-13T17:50:38.614723+0000","last_unstale":"2024-09-13T17:50:38.995312+0000","last_undegraded":"2024-09-13T17:50:38.995312+0000","last_fullsized":"2024-09-13T17:50:38.995312+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:23:01.902181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"25'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.996807+0000","last_change":"2024-09-13T17:50:34.204760+0000","last_active":"2024-09-13T17:50:38.996807+0000","last_peered":"2024-09-13T17:50:38.996807+0000","last_clean":"2024-09-13T17:50:38.996807+0000","last_became_active":"2024-09-13T17:50:34.202930+0000","last_became_peered":"2024-09-13T17:50:34.202930+0000","last_unstale":"2024-09-13T17:50:38.996807+0000","last_undegraded":"2024-09-13T17:50:38.996807+0000","last_fullsized":"2024-09-13T17:50:38.996807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:59.784701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.598529+0000","last_change":"2024-09-13T17:50:34.583174+0000","last_active":"2024-09-13T17:50:38.598529+0000","last_peered":"2024-09-13T17:50:38.598529+0000","last_clean":"2024-09-13T17:50:38.598529+0000","last_became_active":"2024-09-13T17:50:34.582884+0000","last_became_peered":"2024-09-13T17:50:34.582884+0000","last_unstale":"2024-09-13T17:50:38.598529+0000","last_undegraded":"2024-09-13T17:50:38.598529+0000","last_fullsized":"2024-09-13T17:50:38.598529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:22:27.539574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501034+0000","last_change":"2024-09-13T17:50:38.618751+0000","last_active":"2024-09-13T17:50:39.501034+0000","last_peered":"2024-09-13T17:50:39.501034+0000","last_clean":"2024-09-13T17:50:39.501034+0000","last_became_active":"2024-09-13T17:50:38.618423+0000","last_became_peered":"2024-09-13T17:50:38.618423+0000","last_unstale":"2024-09-13T17:50:39.501034+0000","last_undegraded":"2024-09-13T17:50:39.501034+0000","last_fullsized":"2024-09-13T17:50:39.501034+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:50:02.418785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.582559+0000","last_change":"2024-09-13T17:50:34.214972+0000","last_active":"2024-09-13T17:50:39.582559+0000","last_peered":"2024-09-13T17:50:39.582559+0000","last_clean":"2024-09-13T17:50:39.582559+0000","last_became_active":"2024-09-13T17:50:34.214652+0000","last_became_peered":"2024-09-13T17:50:34.214652+0000","last_unstale":"2024-09-13T17:50:39.582559+0000","last_undegraded":"2024-09-13T17:50:39.582559+0000","last_fullsized":"2024-09-13T17:50:39.582559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:57.176854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590539+0000","last_change":"2024-09-13T17:50:34.594493+0000","last_active":"2024-09-13T17:50:38.590539+0000","last_peered":"2024-09-13T17:50:38.590539+0000","last_clean":"2024-09-13T17:50:38.590539+0000","last_became_active":"2024-09-13T17:50:34.594182+0000","last_became_peered":"2024-09-13T17:50:34.594182+0000","last_unstale":"2024-09-13T17:50:38.590539+0000","last_undegraded":"2024-09-13T17:50:38.590539+0000","last_fullsized":"2024-09-13T17:50:38.590539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:18.059925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501399+0000","last_change":"2024-09-13T17:50:38.616219+0000","last_active":"2024-09-13T17:50:39.501399+0000","last_peered":"2024-09-13T17:50:39.501399+0000","last_clean":"2024-09-13T17:50:39.501399+0000","last_became_active":"2024-09-13T17:50:38.614329+0000","last_became_peered":"2024-09-13T17:50:38.614329+0000","last_unstale":"2024-09-13T17:50:39.501399+0000","last_undegraded":"2024-09-13T17:50:39.501399+0000","last_fullsized":"2024-09-13T17:50:39.501399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:25.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583345+0000","last_change":"2024-09-13T17:50:34.216123+0000","last_active":"2024-09-13T17:50:39.583345+0000","last_peered":"2024-09-13T17:50:39.583345+0000","last_clean":"2024-09-13T17:50:39.583345+0000","last_became_active":"2024-09-13T17:50:34.215661+0000","last_became_peered":"2024-09-13T17:50:34.215661+0000","last_unstale":"2024-09-13T17:50:39.583345+0000","last_undegraded":"2024-09-13T17:50:39.583345+0000","last_fullsized":"2024-09-13T17:50:39.583345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:20.895037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591506+0000","last_change":"2024-09-13T17:50:34.599673+0000","last_active":"2024-09-13T17:50:38.591506+0000","last_peered":"2024-09-13T17:50:38.591506+0000","last_clean":"2024-09-13T17:50:38.591506+0000","last_became_active":"2024-09-13T17:50:34.599409+0000","last_became_peered":"2024-09-13T17:50:34.599409+0000","last_unstale":"2024-09-13T17:50:38.591506+0000","last_undegraded":"2024-09-13T17:50:38.591506+0000","last_fullsized":"2024-09-13T17:50:38.591506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:03:31.394385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501275+0000","last_change":"2024-09-13T17:50:38.616317+0000","last_active":"2024-09-13T17:50:39.501275+0000","last_peered":"2024-09-13T17:50:39.501275+0000","last_clean":"2024-09-13T17:50:39.501275+0000","last_became_active":"2024-09-13T17:50:38.614311+0000","last_became_peered":"2024-09-13T17:50:38.614311+0000","last_unstale":"2024-09-13T17:50:39.501275+0000","last_undegraded":"2024-09-13T17:50:39.501275+0000","last_fullsized":"2024-09-13T17:50:39.501275+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:29.221522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.604632+0000","last_change":"2024-09-13T17:50:34.599490+0000","last_active":"2024-09-13T17:50:38.604632+0000","last_peered":"2024-09-13T17:50:38.604632+0000","last_clean":"2024-09-13T17:50:38.604632+0000","last_became_active":"2024-09-13T17:50:34.599207+0000","last_became_peered":"2024-09-13T17:50:34.599207+0000","last_unstale":"2024-09-13T17:50:38.604632+0000","last_undegraded":"2024-09-13T17:50:38.604632+0000","last_fullsized":"2024-09-13T17:50:38.604632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:11.178161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583429+0000","last_change":"2024-09-13T17:50:34.215697+0000","last_active":"2024-09-13T17:50:39.583429+0000","last_peered":"2024-09-13T17:50:39.583429+0000","last_clean":"2024-09-13T17:50:39.583429+0000","last_became_active":"2024-09-13T17:50:34.215409+0000","last_became_peered":"2024-09-13T17:50:34.215409+0000","last_unstale":"2024-09-13T17:50:39.583429+0000","last_undegraded":"2024-09-13T17:50:39.583429+0000","last_fullsized":"2024-09-13T17:50:39.583429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:21.609567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.502325+0000","last_change":"2024-09-13T17:50:38.611817+0000","last_active":"2024-09-13T17:50:39.502325+0000","last_peered":"2024-09-13T17:50:39.502325+0000","last_clean":"2024-09-13T17:50:39.502325+0000","last_became_active":"2024-09-13T17:50:38.611263+0000","last_became_peered":"2024-09-13T17:50:38.611263+0000","last_unstale":"2024-09-13T17:50:39.502325+0000","last_undegraded":"2024-09-13T17:50:39.502325+0000","last_fullsized":"2024-09-13T17:50:39.502325+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:56:47.466672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.588598+0000","last_change":"2024-09-13T17:50:34.586307+0000","last_active":"2024-09-13T17:50:38.588598+0000","last_peered":"2024-09-13T17:50:38.588598+0000","last_clean":"2024-09-13T17:50:38.588598+0000","last_became_active":"2024-09-13T17:50:34.585831+0000","last_became_peered":"2024-09-13T17:50:34.585831+0000","last_unstale":"2024-09-13T17:50:38.588598+0000","last_undegraded":"2024-09-13T17:50:38.588598+0000","last_fullsized":"2024-09-13T17:50:38.588598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:40.912352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599468+0000","last_change":"2024-09-13T17:50:34.211850+0000","last_active":"2024-09-13T17:50:38.599468+0000","last_peered":"2024-09-13T17:50:38.599468+0000","last_clean":"2024-09-13T17:50:38.599468+0000","last_became_active":"2024-09-13T17:50:34.211594+0000","last_became_peered":"2024-09-13T17:50:34.211594+0000","last_unstale":"2024-09-13T17:50:38.599468+0000","last_undegraded":"2024-09-13T17:50:38.599468+0000","last_fullsized":"2024-09-13T17:50:38.599468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:35:02.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.502004+0000","last_change":"2024-09-13T17:50:38.613576+0000","last_active":"2024-09-13T17:50:39.502004+0000","last_peered":"2024-09-13T17:50:39.502004+0000","last_clean":"2024-09-13T17:50:39.502004+0000","last_became_active":"2024-09-13T17:50:38.612577+0000","last_became_peered":"2024-09-13T17:50:38.612577+0000","last_unstale":"2024-09-13T17:50:39.502004+0000","last_undegraded":"2024-09-13T17:50:39.502004+0000","last_fullsized":"2024-09-13T17:50:39.502004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:25.551060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590056+0000","last_change":"2024-09-13T17:50:34.587273+0000","last_active":"2024-09-13T17:50:38.590056+0000","last_peered":"2024-09-13T17:50:38.590056+0000","last_clean":"2024-09-13T17:50:38.590056+0000","last_became_active":"2024-09-13T17:50:34.587027+0000","last_became_peered":"2024-09-13T17:50:34.587027+0000","last_unstale":"2024-09-13T17:50:38.590056+0000","last_undegraded":"2024-09-13T17:50:38.590056+0000","last_fullsized":"2024-09-13T17:50:38.590056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:58.826737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599621+0000","last_change":"2024-09-13T17:50:34.211845+0000","last_active":"2024-09-13T17:50:38.599621+0000","last_peered":"2024-09-13T17:50:38.599621+0000","last_clean":"2024-09-13T17:50:38.599621+0000","last_became_active":"2024-09-13T17:50:34.211568+0000","last_became_peered":"2024-09-13T17:50:34.211568+0000","last_unstale":"2024-09-13T17:50:38.599621+0000","last_undegraded":"2024-09-13T17:50:38.599621+0000","last_fullsized":"2024-09-13T17:50:38.599621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:56.348984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000608+0000","last_change":"2024-09-13T17:50:38.610292+0000","last_active":"2024-09-13T17:50:39.000608+0000","last_peered":"2024-09-13T17:50:39.000608+0000","last_clean":"2024-09-13T17:50:39.000608+0000","last_became_active":"2024-09-13T17:50:38.609832+0000","last_became_peered":"2024-09-13T17:50:38.609832+0000","last_unstale":"2024-09-13T17:50:39.000608+0000","last_undegraded":"2024-09-13T17:50:39.000608+0000","last_fullsized":"2024-09-13T17:50:39.000608+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:13.783068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.603557+0000","last_change":"2024-09-13T17:50:34.593566+0000","last_active":"2024-09-13T17:50:38.603557+0000","last_peered":"2024-09-13T17:50:38.603557+0000","last_clean":"2024-09-13T17:50:38.603557+0000","last_became_active":"2024-09-13T17:50:34.588794+0000","last_became_peered":"2024-09-13T17:50:34.588794+0000","last_unstale":"2024-09-13T17:50:38.603557+0000","last_undegraded":"2024-09-13T17:50:38.603557+0000","last_fullsized":"2024-09-13T17:50:38.603557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:13.192453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589828+0000","last_change":"2024-09-13T17:50:34.214638+0000","last_active":"2024-09-13T17:50:38.589828+0000","last_peered":"2024-09-13T17:50:38.589828+0000","last_clean":"2024-09-13T17:50:38.589828+0000","last_became_active":"2024-09-13T17:50:34.214292+0000","last_became_peered":"2024-09-13T17:50:34.214292+0000","last_unstale":"2024-09-13T17:50:38.589828+0000","last_undegraded":"2024-09-13T17:50:38.589828+0000","last_fullsized":"2024-09-13T17:50:38.589828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:57.377104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501996+0000","last_change":"2024-09-13T17:50:38.613368+0000","last_active":"2024-09-13T17:50:39.501996+0000","last_peered":"2024-09-13T17:50:39.501996+0000","last_clean":"2024-09-13T17:50:39.501996+0000","last_became_active":"2024-09-13T17:50:38.613010+0000","last_became_peered":"2024-09-13T17:50:38.613010+0000","last_unstale":"2024-09-13T17:50:39.501996+0000","last_undegraded":"2024-09-13T17:50:39.501996+0000","last_fullsized":"2024-09-13T17:50:39.501996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:42.706625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.604457+0000","last_change":"2024-09-13T17:50:34.582327+0000","last_active":"2024-09-13T17:50:38.604457+0000","last_peered":"2024-09-13T17:50:38.604457+0000","last_clean":"2024-09-13T17:50:38.604457+0000","last_became_active":"2024-09-13T17:50:34.582050+0000","last_became_peered":"2024-09-13T17:50:34.582050+0000","last_unstale":"2024-09-13T17:50:38.604457+0000","last_undegraded":"2024-09-13T17:50:38.604457+0000","last_fullsized":"2024-09-13T17:50:38.604457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:52.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583664+0000","last_change":"2024-09-13T17:50:34.211512+0000","last_active":"2024-09-13T17:50:39.583664+0000","last_peered":"2024-09-13T17:50:39.583664+0000","last_clean":"2024-09-13T17:50:39.583664+0000","last_became_active":"2024-09-13T17:50:34.211230+0000","last_became_peered":"2024-09-13T17:50:34.211230+0000","last_unstale":"2024-09-13T17:50:39.583664+0000","last_undegraded":"2024-09-13T17:50:39.583664+0000","last_fullsized":"2024-09-13T17:50:39.583664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:40.707777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501244+0000","last_change":"2024-09-13T17:50:38.608255+0000","last_active":"2024-09-13T17:50:39.501244+0000","last_peered":"2024-09-13T17:50:39.501244+0000","last_clean":"2024-09-13T17:50:39.501244+0000","last_became_active":"2024-09-13T17:50:38.606697+0000","last_became_peered":"2024-09-13T17:50:38.606697+0000","last_unstale":"2024-09-13T17:50:39.501244+0000","last_undegraded":"2024-09-13T17:50:39.501244+0000","last_fullsized":"2024-09-13T17:50:39.501244+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:47.265007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591328+0000","last_change":"2024-09-13T17:50:34.594476+0000","last_active":"2024-09-13T17:50:38.591328+0000","last_peered":"2024-09-13T17:50:38.591328+0000","last_clean":"2024-09-13T17:50:38.591328+0000","last_became_active":"2024-09-13T17:50:34.594215+0000","last_became_peered":"2024-09-13T17:50:34.594215+0000","last_unstale":"2024-09-13T17:50:38.591328+0000","last_undegraded":"2024-09-13T17:50:38.591328+0000","last_fullsized":"2024-09-13T17:50:38.591328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:02:27.528019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000656+0000","last_change":"2024-09-13T17:50:34.204018+0000","last_active":"2024-09-13T17:50:39.000656+0000","last_peered":"2024-09-13T17:50:39.000656+0000","last_clean":"2024-09-13T17:50:39.000656+0000","last_became_active":"2024-09-13T17:50:34.203702+0000","last_became_peered":"2024-09-13T17:50:34.203702+0000","last_unstale":"2024-09-13T17:50:39.000656+0000","last_undegraded":"2024-09-13T17:50:39.000656+0000","last_fullsized":"2024-09-13T17:50:39.000656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:44.431226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.995065+0000","last_change":"2024-09-13T17:50:38.615677+0000","last_active":"2024-09-13T17:50:38.995065+0000","last_peered":"2024-09-13T17:50:38.995065+0000","last_clean":"2024-09-13T17:50:38.995065+0000","last_became_active":"2024-09-13T17:50:38.615270+0000","last_became_peered":"2024-09-13T17:50:38.615270+0000","last_unstale":"2024-09-13T17:50:38.995065+0000","last_undegraded":"2024-09-13T17:50:38.995065+0000","last_fullsized":"2024-09-13T17:50:38.995065+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:55.768090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599593+0000","last_change":"2024-09-13T17:50:34.586802+0000","last_active":"2024-09-13T17:50:38.599593+0000","last_peered":"2024-09-13T17:50:38.599593+0000","last_clean":"2024-09-13T17:50:38.599593+0000","last_became_active":"2024-09-13T17:50:34.586552+0000","last_became_peered":"2024-09-13T17:50:34.586552+0000","last_unstale":"2024-09-13T17:50:38.599593+0000","last_undegraded":"2024-09-13T17:50:38.599593+0000","last_fullsized":"2024-09-13T17:50:38.599593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:43.383975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.592964+0000","last_change":"2024-09-13T17:50:34.217083+0000","last_active":"2024-09-13T17:50:38.592964+0000","last_peered":"2024-09-13T17:50:38.592964+0000","last_clean":"2024-09-13T17:50:38.592964+0000","last_became_active":"2024-09-13T17:50:34.216758+0000","last_became_peered":"2024-09-13T17:50:34.216758+0000","last_unstale":"2024-09-13T17:50:38.592964+0000","last_undegraded":"2024-09-13T17:50:38.592964+0000","last_fullsized":"2024-09-13T17:50:38.592964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:36.473942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.500902+0000","last_change":"2024-09-13T17:50:38.614667+0000","last_active":"2024-09-13T17:50:39.500902+0000","last_peered":"2024-09-13T17:50:39.500902+0000","last_clean":"2024-09-13T17:50:39.500902+0000","last_became_active":"2024-09-13T17:50:38.614284+0000","last_became_peered":"2024-09-13T17:50:38.614284+0000","last_unstale":"2024-09-13T17:50:39.500902+0000","last_undegraded":"2024-09-13T17:50:39.500902+0000","last_fullsized":"2024-09-13T17:50:39.500902+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:29.616794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591705+0000","last_change":"2024-09-13T17:50:34.588603+0000","last_active":"2024-09-13T17:50:38.591705+0000","last_peered":"2024-09-13T17:50:38.591705+0000","last_clean":"2024-09-13T17:50:38.591705+0000","last_became_active":"2024-09-13T17:50:34.588317+0000","last_became_peered":"2024-09-13T17:50:34.588317+0000","last_unstale":"2024-09-13T17:50:38.591705+0000","last_undegraded":"2024-09-13T17:50:38.591705+0000","last_fullsized":"2024-09-13T17:50:38.591705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:03.743130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599679+0000","last_change":"2024-09-13T17:50:34.204335+0000","last_active":"2024-09-13T17:50:38.599679+0000","last_peered":"2024-09-13T17:50:38.599679+0000","last_clean":"2024-09-13T17:50:38.599679+0000","last_became_active":"2024-09-13T17:50:34.204060+0000","last_became_peered":"2024-09-13T17:50:34.204060+0000","last_unstale":"2024-09-13T17:50:38.599679+0000","last_undegraded":"2024-09-13T17:50:38.599679+0000","last_fullsized":"2024-09-13T17:50:38.599679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:54:41.831344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.999822+0000","last_change":"2024-09-13T17:50:38.619590+0000","last_active":"2024-09-13T17:50:38.999822+0000","last_peered":"2024-09-13T17:50:38.999822+0000","last_clean":"2024-09-13T17:50:38.999822+0000","last_became_active":"2024-09-13T17:50:38.617257+0000","last_became_peered":"2024-09-13T17:50:38.617257+0000","last_unstale":"2024-09-13T17:50:38.999822+0000","last_undegraded":"2024-09-13T17:50:38.999822+0000","last_fullsized":"2024-09-13T17:50:38.999822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:34.870291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.602091+0000","last_change":"2024-09-13T17:50:34.588466+0000","last_active":"2024-09-13T17:50:38.602091+0000","last_peered":"2024-09-13T17:50:38.602091+0000","last_clean":"2024-09-13T17:50:38.602091+0000","last_became_active":"2024-09-13T17:50:34.588063+0000","last_became_peered":"2024-09-13T17:50:34.588063+0000","last_unstale":"2024-09-13T17:50:38.602091+0000","last_undegraded":"2024-09-13T17:50:38.602091+0000","last_fullsized":"2024-09-13T17:50:38.602091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:43:00.669203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589993+0000","last_change":"2024-09-13T17:50:34.209239+0000","last_active":"2024-09-13T17:50:38.589993+0000","last_peered":"2024-09-13T17:50:38.589993+0000","last_clean":"2024-09-13T17:50:38.589993+0000","last_became_active":"2024-09-13T17:50:34.208196+0000","last_became_peered":"2024-09-13T17:50:34.208196+0000","last_unstale":"2024-09-13T17:50:38.589993+0000","last_undegraded":"2024-09-13T17:50:38.589993+0000","last_fullsized":"2024-09-13T17:50:38.589993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:39.253917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000394+0000","last_change":"2024-09-13T17:50:38.611021+0000","last_active":"2024-09-13T17:50:39.000394+0000","last_peered":"2024-09-13T17:50:39.000394+0000","last_clean":"2024-09-13T17:50:39.000394+0000","last_became_active":"2024-09-13T17:50:38.609794+0000","last_became_peered":"2024-09-13T17:50:38.609794+0000","last_unstale":"2024-09-13T17:50:39.000394+0000","last_undegraded":"2024-09-13T17:50:39.000394+0000","last_fullsized":"2024-09-13T17:50:39.000394+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:59.980236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.600038+0000","last_change":"2024-09-13T17:50:34.584290+0000","last_active":"2024-09-13T17:50:38.600038+0000","last_peered":"2024-09-13T17:50:38.600038+0000","last_clean":"2024-09-13T17:50:38.600038+0000","last_became_active":"2024-09-13T17:50:34.584023+0000","last_became_peered":"2024-09-13T17:50:34.584023+0000","last_unstale":"2024-09-13T17:50:38.600038+0000","last_undegraded":"2024-09-13T17:50:38.600038+0000","last_fullsized":"2024-09-13T17:50:38.600038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:11.304986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.597167+0000","last_change":"2024-09-13T17:50:34.198932+0000","last_active":"2024-09-13T17:50:38.597167+0000","last_peered":"2024-09-13T17:50:38.597167+0000","last_clean":"2024-09-13T17:50:38.597167+0000","last_became_active":"2024-09-13T17:50:34.198603+0000","last_became_peered":"2024-09-13T17:50:34.198603+0000","last_unstale":"2024-09-13T17:50:38.597167+0000","last_undegraded":"2024-09-13T17:50:38.597167+0000","last_fullsized":"2024-09-13T17:50:38.597167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:00.917126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000707+0000","last_change":"2024-09-13T17:50:38.611326+0000","last_active":"2024-09-13T17:50:39.000707+0000","last_peered":"2024-09-13T17:50:39.000707+0000","last_clean":"2024-09-13T17:50:39.000707+0000","last_became_active":"2024-09-13T17:50:38.610900+0000","last_became_peered":"2024-09-13T17:50:38.610900+0000","last_unstale":"2024-09-13T17:50:39.000707+0000","last_undegraded":"2024-09-13T17:50:39.000707+0000","last_fullsized":"2024-09-13T17:50:39.000707+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:17:09.994736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.605384+0000","last_change":"2024-09-13T17:50:34.594482+0000","last_active":"2024-09-13T17:50:38.605384+0000","last_peered":"2024-09-13T17:50:38.605384+0000","last_clean":"2024-09-13T17:50:38.605384+0000","last_became_active":"2024-09-13T17:50:34.594201+0000","last_became_peered":"2024-09-13T17:50:34.594201+0000","last_unstale":"2024-09-13T17:50:38.605384+0000","last_undegraded":"2024-09-13T17:50:38.605384+0000","last_fullsized":"2024-09-13T17:50:38.605384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:06.697507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590063+0000","last_change":"2024-09-13T17:50:34.218376+0000","last_active":"2024-09-13T17:50:38.590063+0000","last_peered":"2024-09-13T17:50:38.590063+0000","last_clean":"2024-09-13T17:50:38.590063+0000","last_became_active":"2024-09-13T17:50:34.218069+0000","last_became_peered":"2024-09-13T17:50:34.218069+0000","last_unstale":"2024-09-13T17:50:38.590063+0000","last_undegraded":"2024-09-13T17:50:38.590063+0000","last_fullsized":"2024-09-13T17:50:38.590063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:53:54.288969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501171+0000","last_change":"2024-09-13T17:50:38.613206+0000","last_active":"2024-09-13T17:50:39.501171+0000","last_peered":"2024-09-13T17:50:39.501171+0000","last_clean":"2024-09-13T17:50:39.501171+0000","last_became_active":"2024-09-13T17:50:38.612856+0000","last_became_peered":"2024-09-13T17:50:38.612856+0000","last_unstale":"2024-09-13T17:50:39.501171+0000","last_undegraded":"2024-09-13T17:50:39.501171+0000","last_fullsized":"2024-09-13T17:50:39.501171+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:31:20.190996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.588765+0000","last_change":"2024-09-13T17:50:34.582931+0000","last_active":"2024-09-13T17:50:38.588765+0000","last_peered":"2024-09-13T17:50:38.588765+0000","last_clean":"2024-09-13T17:50:38.588765+0000","last_became_active":"2024-09-13T17:50:34.582446+0000","last_became_peered":"2024-09-13T17:50:34.582446+0000","last_unstale":"2024-09-13T17:50:38.588765+0000","last_undegraded":"2024-09-13T17:50:38.588765+0000","last_fullsized":"2024-09-13T17:50:38.588765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:30.554785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.588978+0000","last_change":"2024-09-13T17:50:34.201237+0000","last_active":"2024-09-13T17:50:38.588978+0000","last_peered":"2024-09-13T17:50:38.588978+0000","last_clean":"2024-09-13T17:50:38.588978+0000","last_became_active":"2024-09-13T17:50:34.200873+0000","last_became_peered":"2024-09-13T17:50:34.200873+0000","last_unstale":"2024-09-13T17:50:38.588978+0000","last_undegraded":"2024-09-13T17:50:38.588978+0000","last_fullsized":"2024-09-13T17:50:38.588978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:02.711342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590078+0000","last_change":"2024-09-13T17:50:22.095367+0000","last_active":"2024-09-13T17:50:38.590078+0000","last_peered":"2024-09-13T17:50:38.590078+0000","last_clean":"2024-09-13T17:50:38.590078+0000","last_became_active":"2024-09-13T17:50:22.093734+0000","last_became_peered":"2024-09-13T17:50:22.093734+0000","last_unstale":"2024-09-13T17:50:38.590078+0000","last_undegraded":"2024-09-13T17:50:38.590078+0000","last_fullsized":"2024-09-13T17:50:38.590078+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_clean_scrub_stamp":"2024-09-13T17:50:21.057812+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:23.531613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.982380+0000","last_change":"2024-09-13T17:50:38.618880+0000","last_active":"2024-09-13T17:50:38.982380+0000","last_peered":"2024-09-13T17:50:38.982380+0000","last_clean":"2024-09-13T17:50:38.982380+0000","last_became_active":"2024-09-13T17:50:38.618488+0000","last_became_peered":"2024-09-13T17:50:38.618488+0000","last_unstale":"2024-09-13T17:50:38.982380+0000","last_undegraded":"2024-09-13T17:50:38.982380+0000","last_fullsized":"2024-09-13T17:50:38.982380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:03.431876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.592679+0000","last_change":"2024-09-13T17:50:34.585881+0000","last_active":"2024-09-13T17:50:38.592679+0000","last_peered":"2024-09-13T17:50:38.592679+0000","last_clean":"2024-09-13T17:50:38.592679+0000","last_became_active":"2024-09-13T17:50:34.585166+0000","last_became_peered":"2024-09-13T17:50:34.585166+0000","last_unstale":"2024-09-13T17:50:38.592679+0000","last_undegraded":"2024-09-13T17:50:38.592679+0000","last_fullsized":"2024-09-13T17:50:38.592679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:06.654658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.602557+0000","last_change":"2024-09-13T17:50:34.210934+0000","last_active":"2024-09-13T17:50:38.602557+0000","last_peered":"2024-09-13T17:50:38.602557+0000","last_clean":"2024-09-13T17:50:38.602557+0000","last_became_active":"2024-09-13T17:50:34.210589+0000","last_became_peered":"2024-09-13T17:50:34.210589+0000","last_unstale":"2024-09-13T17:50:38.602557+0000","last_undegraded":"2024-09-13T17:50:38.602557+0000","last_fullsized":"2024-09-13T17:50:38.602557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:26.213295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.982569+0000","last_change":"2024-09-13T17:50:38.615811+0000","last_active":"2024-09-13T17:50:38.982569+0000","last_peered":"2024-09-13T17:50:38.982569+0000","last_clean":"2024-09-13T17:50:38.982569+0000","last_became_active":"2024-09-13T17:50:38.615358+0000","last_became_peered":"2024-09-13T17:50:38.615358+0000","last_unstale":"2024-09-13T17:50:38.982569+0000","last_undegraded":"2024-09-13T17:50:38.982569+0000","last_fullsized":"2024-09-13T17:50:38.982569+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:05.667393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590751+0000","last_change":"2024-09-13T17:50:34.590169+0000","last_active":"2024-09-13T17:50:38.590751+0000","last_peered":"2024-09-13T17:50:38.590751+0000","last_clean":"2024-09-13T17:50:38.590751+0000","last_became_active":"2024-09-13T17:50:34.589434+0000","last_became_peered":"2024-09-13T17:50:34.589434+0000","last_unstale":"2024-09-13T17:50:38.590751+0000","last_undegraded":"2024-09-13T17:50:38.590751+0000","last_fullsized":"2024-09-13T17:50:38.590751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:12.545412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000847+0000","last_change":"2024-09-13T17:50:34.204269+0000","last_active":"2024-09-13T17:50:39.000847+0000","last_peered":"2024-09-13T17:50:39.000847+0000","last_clean":"2024-09-13T17:50:39.000847+0000","last_became_active":"2024-09-13T17:50:34.203874+0000","last_became_peered":"2024-09-13T17:50:34.203874+0000","last_unstale":"2024-09-13T17:50:39.000847+0000","last_undegraded":"2024-09-13T17:50:39.000847+0000","last_fullsized":"2024-09-13T17:50:39.000847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:10.410267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501092+0000","last_change":"2024-09-13T17:50:38.615989+0000","last_active":"2024-09-13T17:50:39.501092+0000","last_peered":"2024-09-13T17:50:39.501092+0000","last_clean":"2024-09-13T17:50:39.501092+0000","last_became_active":"2024-09-13T17:50:38.615530+0000","last_became_peered":"2024-09-13T17:50:38.615530+0000","last_unstale":"2024-09-13T17:50:39.501092+0000","last_undegraded":"2024-09-13T17:50:39.501092+0000","last_fullsized":"2024-09-13T17:50:39.501092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:41.064092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.593217+0000","last_change":"2024-09-13T17:50:34.595120+0000","last_active":"2024-09-13T17:50:38.593217+0000","last_peered":"2024-09-13T17:50:38.593217+0000","last_clean":"2024-09-13T17:50:38.593217+0000","last_became_active":"2024-09-13T17:50:34.594842+0000","last_became_peered":"2024-09-13T17:50:34.594842+0000","last_unstale":"2024-09-13T17:50:38.593217+0000","last_undegraded":"2024-09-13T17:50:38.593217+0000","last_fullsized":"2024-09-13T17:50:38.593217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:15:38.982524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.587423+0000","last_change":"2024-09-13T17:50:34.199957+0000","last_active":"2024-09-13T17:50:39.587423+0000","last_peered":"2024-09-13T17:50:39.587423+0000","last_clean":"2024-09-13T17:50:39.587423+0000","last_became_active":"2024-09-13T17:50:34.199650+0000","last_became_peered":"2024-09-13T17:50:34.199650+0000","last_unstale":"2024-09-13T17:50:39.587423+0000","last_undegraded":"2024-09-13T17:50:39.587423+0000","last_fullsized":"2024-09-13T17:50:39.587423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:26.929549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.193989+0000","last_change":"2024-09-13T17:50:38.609731+0000","last_active":"2024-09-13T17:50:39.193989+0000","last_peered":"2024-09-13T17:50:39.193989+0000","last_clean":"2024-09-13T17:50:39.193989+0000","last_became_active":"2024-09-13T17:50:38.609354+0000","last_became_peered":"2024-09-13T17:50:38.609354+0000","last_unstale":"2024-09-13T17:50:39.193989+0000","last_undegraded":"2024-09-13T17:50:39.193989+0000","last_fullsized":"2024-09-13T17:50:39.193989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:14.142369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.593339+0000","last_change":"2024-09-13T17:50:34.595146+0000","last_active":"2024-09-13T17:50:38.593339+0000","last_peered":"2024-09-13T17:50:38.593339+0000","last_clean":"2024-09-13T17:50:38.593339+0000","last_became_active":"2024-09-13T17:50:34.594876+0000","last_became_peered":"2024-09-13T17:50:34.594876+0000","last_unstale":"2024-09-13T17:50:38.593339+0000","last_undegraded":"2024-09-13T17:50:38.593339+0000","last_fullsized":"2024-09-13T17:50:38.593339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:26:10.638438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.502430+0000","last_change":"2024-09-13T17:50:34.200640+0000","last_active":"2024-09-13T17:50:39.502430+0000","last_peered":"2024-09-13T17:50:39.502430+0000","last_clean":"2024-09-13T17:50:39.502430+0000","last_became_active":"2024-09-13T17:50:34.200197+0000","last_became_peered":"2024-09-13T17:50:34.200197+0000","last_unstale":"2024-09-13T17:50:39.502430+0000","last_undegraded":"2024-09-13T17:50:39.502430+0000","last_fullsized":"2024-09-13T17:50:39.502430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:37:53.039720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.500911+0000","last_change":"2024-09-13T17:50:38.619880+0000","last_active":"2024-09-13T17:50:39.500911+0000","last_peered":"2024-09-13T17:50:39.500911+0000","last_clean":"2024-09-13T17:50:39.500911+0000","last_became_active":"2024-09-13T17:50:38.618491+0000","last_became_peered":"2024-09-13T17:50:38.618491+0000","last_unstale":"2024-09-13T17:50:39.500911+0000","last_undegraded":"2024-09-13T17:50:39.500911+0000","last_fullsized":"2024-09-13T17:50:39.500911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:10:14.136054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590688+0000","last_change":"2024-09-13T17:50:34.585741+0000","last_active":"2024-09-13T17:50:38.590688+0000","last_peered":"2024-09-13T17:50:38.590688+0000","last_clean":"2024-09-13T17:50:38.590688+0000","last_became_active":"2024-09-13T17:50:34.585464+0000","last_became_peered":"2024-09-13T17:50:34.585464+0000","last_unstale":"2024-09-13T17:50:38.590688+0000","last_undegraded":"2024-09-13T17:50:38.590688+0000","last_fullsized":"2024-09-13T17:50:38.590688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:59:22.031459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"25'8","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.585286+0000","last_change":"2024-09-13T17:50:34.201272+0000","last_active":"2024-09-13T17:50:39.585286+0000","last_peered":"2024-09-13T17:50:39.585286+0000","last_clean":"2024-09-13T17:50:39.585286+0000","last_became_active":"2024-09-13T17:50:34.200936+0000","last_became_peered":"2024-09-13T17:50:34.200936+0000","last_unstale":"2024-09-13T17:50:39.585286+0000","last_undegraded":"2024-09-13T17:50:39.585286+0000","last_fullsized":"2024-09-13T17:50:39.585286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:52.094879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.500719+0000","last_change":"2024-09-13T17:50:38.615229+0000","last_active":"2024-09-13T17:50:39.500719+0000","last_peered":"2024-09-13T17:50:39.500719+0000","last_clean":"2024-09-13T17:50:39.500719+0000","last_became_active":"2024-09-13T17:50:38.614739+0000","last_became_peered":"2024-09-13T17:50:38.614739+0000","last_unstale":"2024-09-13T17:50:39.500719+0000","last_undegraded":"2024-09-13T17:50:39.500719+0000","last_fullsized":"2024-09-13T17:50:39.500719+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:18.619529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.604486+0000","last_change":"2024-09-13T17:50:34.588848+0000","last_active":"2024-09-13T17:50:38.604486+0000","last_peered":"2024-09-13T17:50:38.604486+0000","last_clean":"2024-09-13T17:50:38.604486+0000","last_became_active":"2024-09-13T17:50:34.588565+0000","last_became_peered":"2024-09-13T17:50:34.588565+0000","last_unstale":"2024-09-13T17:50:38.604486+0000","last_undegraded":"2024-09-13T17:50:38.604486+0000","last_fullsized":"2024-09-13T17:50:38.604486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:47.927865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.604169+0000","last_change":"2024-09-13T17:50:34.199008+0000","last_active":"2024-09-13T17:50:38.604169+0000","last_peered":"2024-09-13T17:50:38.604169+0000","last_clean":"2024-09-13T17:50:38.604169+0000","last_became_active":"2024-09-13T17:50:34.197839+0000","last_became_peered":"2024-09-13T17:50:34.197839+0000","last_unstale":"2024-09-13T17:50:38.604169+0000","last_undegraded":"2024-09-13T17:50:38.604169+0000","last_fullsized":"2024-09-13T17:50:38.604169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:51.430710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.996853+0000","last_change":"2024-09-13T17:50:38.611080+0000","last_active":"2024-09-13T17:50:38.996853+0000","last_peered":"2024-09-13T17:50:38.996853+0000","last_clean":"2024-09-13T17:50:38.996853+0000","last_became_active":"2024-09-13T17:50:38.610688+0000","last_became_peered":"2024-09-13T17:50:38.610688+0000","last_unstale":"2024-09-13T17:50:38.996853+0000","last_undegraded":"2024-09-13T17:50:38.996853+0000","last_fullsized":"2024-09-13T17:50:38.996853+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:28.527617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591242+0000","last_change":"2024-09-13T17:50:34.594763+0000","last_active":"2024-09-13T17:50:38.591242+0000","last_peered":"2024-09-13T17:50:38.591242+0000","last_clean":"2024-09-13T17:50:38.591242+0000","last_became_active":"2024-09-13T17:50:34.594471+0000","last_became_peered":"2024-09-13T17:50:34.594471+0000","last_unstale":"2024-09-13T17:50:38.591242+0000","last_undegraded":"2024-09-13T17:50:38.591242+0000","last_fullsized":"2024-09-13T17:50:38.591242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:20:58.718271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589771+0000","last_change":"2024-09-13T17:50:34.209432+0000","last_active":"2024-09-13T17:50:38.589771+0000","last_peered":"2024-09-13T17:50:38.589771+0000","last_clean":"2024-09-13T17:50:38.589771+0000","last_became_active":"2024-09-13T17:50:34.208817+0000","last_became_peered":"2024-09-13T17:50:34.208817+0000","last_unstale":"2024-09-13T17:50:38.589771+0000","last_undegraded":"2024-09-13T17:50:38.589771+0000","last_fullsized":"2024-09-13T17:50:38.589771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:49.587528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.995813+0000","last_change":"2024-09-13T17:50:38.609461+0000","last_active":"2024-09-13T17:50:38.995813+0000","last_peered":"2024-09-13T17:50:38.995813+0000","last_clean":"2024-09-13T17:50:38.995813+0000","last_became_active":"2024-09-13T17:50:38.609128+0000","last_became_peered":"2024-09-13T17:50:38.609128+0000","last_unstale":"2024-09-13T17:50:38.995813+0000","last_undegraded":"2024-09-13T17:50:38.995813+0000","last_fullsized":"2024-09-13T17:50:38.995813+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:02.286825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.604049+0000","last_change":"2024-09-13T17:50:34.592970+0000","last_active":"2024-09-13T17:50:38.604049+0000","last_peered":"2024-09-13T17:50:38.604049+0000","last_clean":"2024-09-13T17:50:38.604049+0000","last_became_active":"2024-09-13T17:50:34.592669+0000","last_became_peered":"2024-09-13T17:50:34.592669+0000","last_unstale":"2024-09-13T17:50:38.604049+0000","last_undegraded":"2024-09-13T17:50:38.604049+0000","last_fullsized":"2024-09-13T17:50:38.604049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:52.031261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583358+0000","last_change":"2024-09-13T17:50:34.200798+0000","last_active":"2024-09-13T17:50:39.583358+0000","last_peered":"2024-09-13T17:50:39.583358+0000","last_clean":"2024-09-13T17:50:39.583358+0000","last_became_active":"2024-09-13T17:50:34.200529+0000","last_became_peered":"2024-09-13T17:50:34.200529+0000","last_unstale":"2024-09-13T17:50:39.583358+0000","last_undegraded":"2024-09-13T17:50:39.583358+0000","last_fullsized":"2024-09-13T17:50:39.583358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:26.334296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.195285+0000","last_change":"2024-09-13T17:50:38.606287+0000","last_active":"2024-09-13T17:50:39.195285+0000","last_peered":"2024-09-13T17:50:39.195285+0000","last_clean":"2024-09-13T17:50:39.195285+0000","last_became_active":"2024-09-13T17:50:38.605826+0000","last_became_peered":"2024-09-13T17:50:38.605826+0000","last_unstale":"2024-09-13T17:50:39.195285+0000","last_undegraded":"2024-09-13T17:50:39.195285+0000","last_fullsized":"2024-09-13T17:50:39.195285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:18.709454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.592530+0000","last_change":"2024-09-13T17:50:34.591579+0000","last_active":"2024-09-13T17:50:38.592530+0000","last_peered":"2024-09-13T17:50:38.592530+0000","last_clean":"2024-09-13T17:50:38.592530+0000","last_became_active":"2024-09-13T17:50:34.586165+0000","last_became_peered":"2024-09-13T17:50:34.586165+0000","last_unstale":"2024-09-13T17:50:38.592530+0000","last_undegraded":"2024-09-13T17:50:38.592530+0000","last_fullsized":"2024-09-13T17:50:38.592530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:56:19.205176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:47.801705+0000","last_change":"2024-09-13T17:50:34.203218+0000","last_active":"2024-09-13T17:50:47.801705+0000","last_peered":"2024-09-13T17:50:47.801705+0000","last_clean":"2024-09-13T17:50:47.801705+0000","last_became_active":"2024-09-13T17:50:34.202948+0000","last_became_peered":"2024-09-13T17:50:34.202948+0000","last_unstale":"2024-09-13T17:50:47.801705+0000","last_undegraded":"2024-09-13T17:50:47.801705+0000","last_fullsized":"2024-09-13T17:50:47.801705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:46.535569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000203+0000","last_change":"2024-09-13T17:50:38.610597+0000","last_active":"2024-09-13T17:50:39.000203+0000","last_peered":"2024-09-13T17:50:39.000203+0000","last_clean":"2024-09-13T17:50:39.000203+0000","last_became_active":"2024-09-13T17:50:38.610055+0000","last_became_peered":"2024-09-13T17:50:38.610055+0000","last_unstale":"2024-09-13T17:50:39.000203+0000","last_undegraded":"2024-09-13T17:50:39.000203+0000","last_fullsized":"2024-09-13T17:50:39.000203+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:55.754936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.602441+0000","last_change":"2024-09-13T17:50:34.593963+0000","last_active":"2024-09-13T17:50:38.602441+0000","last_peered":"2024-09-13T17:50:38.602441+0000","last_clean":"2024-09-13T17:50:38.602441+0000","last_became_active":"2024-09-13T17:50:34.593454+0000","last_became_peered":"2024-09-13T17:50:34.593454+0000","last_unstale":"2024-09-13T17:50:38.602441+0000","last_undegraded":"2024-09-13T17:50:38.602441+0000","last_fullsized":"2024-09-13T17:50:38.602441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:18.883521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583588+0000","last_change":"2024-09-13T17:50:34.207936+0000","last_active":"2024-09-13T17:50:39.583588+0000","last_peered":"2024-09-13T17:50:39.583588+0000","last_clean":"2024-09-13T17:50:39.583588+0000","last_became_active":"2024-09-13T17:50:34.207605+0000","last_became_peered":"2024-09-13T17:50:34.207605+0000","last_unstale":"2024-09-13T17:50:39.583588+0000","last_undegraded":"2024-09-13T17:50:39.583588+0000","last_fullsized":"2024-09-13T17:50:39.583588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:05.494406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501828+0000","last_change":"2024-09-13T17:50:38.616297+0000","last_active":"2024-09-13T17:50:39.501828+0000","last_peered":"2024-09-13T17:50:39.501828+0000","last_clean":"2024-09-13T17:50:39.501828+0000","last_became_active":"2024-09-13T17:50:38.614165+0000","last_became_peered":"2024-09-13T17:50:38.614165+0000","last_unstale":"2024-09-13T17:50:39.501828+0000","last_undegraded":"2024-09-13T17:50:39.501828+0000","last_fullsized":"2024-09-13T17:50:39.501828+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:11.357026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.598628+0000","last_change":"2024-09-13T17:50:34.583001+0000","last_active":"2024-09-13T17:50:38.598628+0000","last_peered":"2024-09-13T17:50:38.598628+0000","last_clean":"2024-09-13T17:50:38.598628+0000","last_became_active":"2024-09-13T17:50:34.582529+0000","last_became_peered":"2024-09-13T17:50:34.582529+0000","last_unstale":"2024-09-13T17:50:38.598628+0000","last_undegraded":"2024-09-13T17:50:38.598628+0000","last_fullsized":"2024-09-13T17:50:38.598628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:05.287181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591968+0000","last_change":"2024-09-13T17:50:34.205138+0000","last_active":"2024-09-13T17:50:38.591968+0000","last_peered":"2024-09-13T17:50:38.591968+0000","last_clean":"2024-09-13T17:50:38.591968+0000","last_became_active":"2024-09-13T17:50:34.203360+0000","last_became_peered":"2024-09-13T17:50:34.203360+0000","last_unstale":"2024-09-13T17:50:38.591968+0000","last_undegraded":"2024-09-13T17:50:38.591968+0000","last_fullsized":"2024-09-13T17:50:38.591968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:35.756963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.995168+0000","last_change":"2024-09-13T17:50:38.611036+0000","last_active":"2024-09-13T17:50:38.995168+0000","last_peered":"2024-09-13T17:50:38.995168+0000","last_clean":"2024-09-13T17:50:38.995168+0000","last_became_active":"2024-09-13T17:50:38.610692+0000","last_became_peered":"2024-09-13T17:50:38.610692+0000","last_unstale":"2024-09-13T17:50:38.995168+0000","last_undegraded":"2024-09-13T17:50:38.995168+0000","last_fullsized":"2024-09-13T17:50:38.995168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:10.147998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591770+0000","last_change":"2024-09-13T17:50:34.596882+0000","last_active":"2024-09-13T17:50:38.591770+0000","last_peered":"2024-09-13T17:50:38.591770+0000","last_clean":"2024-09-13T17:50:38.591770+0000","last_became_active":"2024-09-13T17:50:34.596495+0000","last_became_peered":"2024-09-13T17:50:34.596495+0000","last_unstale":"2024-09-13T17:50:38.591770+0000","last_undegraded":"2024-09-13T17:50:38.591770+0000","last_fullsized":"2024-09-13T17:50:38.591770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:21.441114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.609505+0000","last_change":"2024-09-13T17:50:34.204376+0000","last_active":"2024-09-13T17:50:38.609505+0000","last_peered":"2024-09-13T17:50:38.609505+0000","last_clean":"2024-09-13T17:50:38.609505+0000","last_became_active":"2024-09-13T17:50:34.204003+0000","last_became_peered":"2024-09-13T17:50:34.204003+0000","last_unstale":"2024-09-13T17:50:38.609505+0000","last_undegraded":"2024-09-13T17:50:38.609505+0000","last_fullsized":"2024-09-13T17:50:38.609505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:14.023990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.995437+0000","last_change":"2024-09-13T17:50:38.606144+0000","last_active":"2024-09-13T17:50:38.995437+0000","last_peered":"2024-09-13T17:50:38.995437+0000","last_clean":"2024-09-13T17:50:38.995437+0000","last_became_active":"2024-09-13T17:50:38.605646+0000","last_became_peered":"2024-09-13T17:50:38.605646+0000","last_unstale":"2024-09-13T17:50:38.995437+0000","last_undegraded":"2024-09-13T17:50:38.995437+0000","last_fullsized":"2024-09-13T17:50:38.995437+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:07.120075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.500554+0000","last_change":"2024-09-13T17:50:38.618855+0000","last_active":"2024-09-13T17:50:39.500554+0000","last_peered":"2024-09-13T17:50:39.500554+0000","last_clean":"2024-09-13T17:50:39.500554+0000","last_became_active":"2024-09-13T17:50:38.618449+0000","last_became_peered":"2024-09-13T17:50:38.618449+0000","last_unstale":"2024-09-13T17:50:39.500554+0000","last_undegraded":"2024-09-13T17:50:39.500554+0000","last_fullsized":"2024-09-13T17:50:39.500554+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:29:01.114087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.600195+0000","last_change":"2024-09-13T17:50:34.590079+0000","last_active":"2024-09-13T17:50:38.600195+0000","last_peered":"2024-09-13T17:50:38.600195+0000","last_clean":"2024-09-13T17:50:38.600195+0000","last_became_active":"2024-09-13T17:50:34.589396+0000","last_became_peered":"2024-09-13T17:50:34.589396+0000","last_unstale":"2024-09-13T17:50:38.600195+0000","last_undegraded":"2024-09-13T17:50:38.600195+0000","last_fullsized":"2024-09-13T17:50:38.600195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:03.308341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589925+0000","last_change":"2024-09-13T17:50:34.206575+0000","last_active":"2024-09-13T17:50:38.589925+0000","last_peered":"2024-09-13T17:50:38.589925+0000","last_clean":"2024-09-13T17:50:38.589925+0000","last_became_active":"2024-09-13T17:50:34.206308+0000","last_became_peered":"2024-09-13T17:50:34.206308+0000","last_unstale":"2024-09-13T17:50:38.589925+0000","last_undegraded":"2024-09-13T17:50:38.589925+0000","last_fullsized":"2024-09-13T17:50:38.589925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:34.766538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.982710+0000","last_change":"2024-09-13T17:50:38.618131+0000","last_active":"2024-09-13T17:50:38.982710+0000","last_peered":"2024-09-13T17:50:38.982710+0000","last_clean":"2024-09-13T17:50:38.982710+0000","last_became_active":"2024-09-13T17:50:38.617611+0000","last_became_peered":"2024-09-13T17:50:38.617611+0000","last_unstale":"2024-09-13T17:50:38.982710+0000","last_undegraded":"2024-09-13T17:50:38.982710+0000","last_fullsized":"2024-09-13T17:50:38.982710+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:30.746445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589814+0000","last_change":"2024-09-13T17:50:34.582153+0000","last_active":"2024-09-13T17:50:38.589814+0000","last_peered":"2024-09-13T17:50:38.589814+0000","last_clean":"2024-09-13T17:50:38.589814+0000","last_became_active":"2024-09-13T17:50:34.581708+0000","last_became_peered":"2024-09-13T17:50:34.581708+0000","last_unstale":"2024-09-13T17:50:38.589814+0000","last_undegraded":"2024-09-13T17:50:38.589814+0000","last_fullsized":"2024-09-13T17:50:38.589814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:22.238810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590175+0000","last_change":"2024-09-13T17:50:34.205489+0000","last_active":"2024-09-13T17:50:38.590175+0000","last_peered":"2024-09-13T17:50:38.590175+0000","last_clean":"2024-09-13T17:50:38.590175+0000","last_became_active":"2024-09-13T17:50:34.205202+0000","last_became_peered":"2024-09-13T17:50:34.205202+0000","last_unstale":"2024-09-13T17:50:38.590175+0000","last_undegraded":"2024-09-13T17:50:38.590175+0000","last_fullsized":"2024-09-13T17:50:38.590175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:54.450276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.983249+0000","last_change":"2024-09-13T17:50:34.202470+0000","last_active":"2024-09-13T17:50:38.983249+0000","last_peered":"2024-09-13T17:50:38.983249+0000","last_clean":"2024-09-13T17:50:38.983249+0000","last_became_active":"2024-09-13T17:50:34.202163+0000","last_became_peered":"2024-09-13T17:50:34.202163+0000","last_unstale":"2024-09-13T17:50:38.983249+0000","last_undegraded":"2024-09-13T17:50:38.983249+0000","last_fullsized":"2024-09-13T17:50:38.983249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:03.472932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.603393+0000","last_change":"2024-09-13T17:50:34.590947+0000","last_active":"2024-09-13T17:50:38.603393+0000","last_peered":"2024-09-13T17:50:38.603393+0000","last_clean":"2024-09-13T17:50:38.603393+0000","last_became_active":"2024-09-13T17:50:34.590497+0000","last_became_peered":"2024-09-13T17:50:34.590497+0000","last_unstale":"2024-09-13T17:50:38.603393+0000","last_undegraded":"2024-09-13T17:50:38.603393+0000","last_fullsized":"2024-09-13T17:50:38.603393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:56:27.848092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.980209+0000","last_change":"2024-09-13T17:50:38.618518+0000","last_active":"2024-09-13T17:50:38.980209+0000","last_peered":"2024-09-13T17:50:38.980209+0000","last_clean":"2024-09-13T17:50:38.980209+0000","last_became_active":"2024-09-13T17:50:38.618108+0000","last_became_peered":"2024-09-13T17:50:38.618108+0000","last_unstale":"2024-09-13T17:50:38.980209+0000","last_undegraded":"2024-09-13T17:50:38.980209+0000","last_fullsized":"2024-09-13T17:50:38.980209+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:42.260237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"25'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.996384+0000","last_change":"2024-09-13T17:50:34.218459+0000","last_active":"2024-09-13T17:50:38.996384+0000","last_peered":"2024-09-13T17:50:38.996384+0000","last_clean":"2024-09-13T17:50:38.996384+0000","last_became_active":"2024-09-13T17:50:34.218087+0000","last_became_peered":"2024-09-13T17:50:34.218087+0000","last_unstale":"2024-09-13T17:50:38.996384+0000","last_undegraded":"2024-09-13T17:50:38.996384+0000","last_fullsized":"2024-09-13T17:50:38.996384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:56.914544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.603583+0000","last_change":"2024-09-13T17:50:34.583845+0000","last_active":"2024-09-13T17:50:38.603583+0000","last_peered":"2024-09-13T17:50:38.603583+0000","last_clean":"2024-09-13T17:50:38.603583+0000","last_became_active":"2024-09-13T17:50:34.583283+0000","last_became_peered":"2024-09-13T17:50:34.583283+0000","last_unstale":"2024-09-13T17:50:38.603583+0000","last_undegraded":"2024-09-13T17:50:38.603583+0000","last_fullsized":"2024-09-13T17:50:38.603583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:55.630285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.996396+0000","last_change":"2024-09-13T17:50:38.611987+0000","last_active":"2024-09-13T17:50:38.996396+0000","last_peered":"2024-09-13T17:50:38.996396+0000","last_clean":"2024-09-13T17:50:38.996396+0000","last_became_active":"2024-09-13T17:50:38.611636+0000","last_became_peered":"2024-09-13T17:50:38.611636+0000","last_unstale":"2024-09-13T17:50:38.996396+0000","last_undegraded":"2024-09-13T17:50:38.996396+0000","last_fullsized":"2024-09-13T17:50:38.996396+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:41.111562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.602963+0000","last_change":"2024-09-13T17:50:34.588927+0000","last_active":"2024-09-13T17:50:38.602963+0000","last_peered":"2024-09-13T17:50:38.602963+0000","last_clean":"2024-09-13T17:50:38.602963+0000","last_became_active":"2024-09-13T17:50:34.588651+0000","last_became_peered":"2024-09-13T17:50:34.588651+0000","last_unstale":"2024-09-13T17:50:38.602963+0000","last_undegraded":"2024-09-13T17:50:38.602963+0000","last_fullsized":"2024-09-13T17:50:38.602963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:49.429670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.597005+0000","last_change":"2024-09-13T17:50:34.583795+0000","last_active":"2024-09-13T17:50:38.597005+0000","last_peered":"2024-09-13T17:50:38.597005+0000","last_clean":"2024-09-13T17:50:38.597005+0000","last_became_active":"2024-09-13T17:50:34.583493+0000","last_became_peered":"2024-09-13T17:50:34.583493+0000","last_unstale":"2024-09-13T17:50:38.597005+0000","last_undegraded":"2024-09-13T17:50:38.597005+0000","last_fullsized":"2024-09-13T17:50:38.597005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:30.463932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589528+0000","last_change":"2024-09-13T17:50:34.587624+0000","last_active":"2024-09-13T17:50:38.589528+0000","last_peered":"2024-09-13T17:50:38.589528+0000","last_clean":"2024-09-13T17:50:38.589528+0000","last_became_active":"2024-09-13T17:50:34.587119+0000","last_became_peered":"2024-09-13T17:50:34.587119+0000","last_unstale":"2024-09-13T17:50:38.589528+0000","last_undegraded":"2024-09-13T17:50:38.589528+0000","last_fullsized":"2024-09-13T17:50:38.589528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:16.783899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599861+0000","last_change":"2024-09-13T17:50:34.581923+0000","last_active":"2024-09-13T17:50:38.599861+0000","last_peered":"2024-09-13T17:50:38.599861+0000","last_clean":"2024-09-13T17:50:38.599861+0000","last_became_active":"2024-09-13T17:50:34.581628+0000","last_became_peered":"2024-09-13T17:50:34.581628+0000","last_unstale":"2024-09-13T17:50:38.599861+0000","last_undegraded":"2024-09-13T17:50:38.599861+0000","last_fullsized":"2024-09-13T17:50:38.599861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:28:55.936864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589257+0000","last_change":"2024-09-13T17:50:34.595037+0000","last_active":"2024-09-13T17:50:38.589257+0000","last_peered":"2024-09-13T17:50:38.589257+0000","last_clean":"2024-09-13T17:50:38.589257+0000","last_became_active":"2024-09-13T17:50:34.594857+0000","last_became_peered":"2024-09-13T17:50:34.594857+0000","last_unstale":"2024-09-13T17:50:38.589257+0000","last_undegraded":"2024-09-13T17:50:38.589257+0000","last_fullsized":"2024-09-13T17:50:38.589257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:11.420413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.597995+0000","last_change":"2024-09-13T17:50:34.589651+0000","last_active":"2024-09-13T17:50:38.597995+0000","last_peered":"2024-09-13T17:50:38.597995+0000","last_clean":"2024-09-13T17:50:38.597995+0000","last_became_active":"2024-09-13T17:50:34.589354+0000","last_became_peered":"2024-09-13T17:50:34.589354+0000","last_unstale":"2024-09-13T17:50:38.597995+0000","last_undegraded":"2024-09-13T17:50:38.597995+0000","last_fullsized":"2024-09-13T17:50:38.597995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:31.294048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.609987+0000","last_change":"2024-09-13T17:50:34.597949+0000","last_active":"2024-09-13T17:50:38.609987+0000","last_peered":"2024-09-13T17:50:38.609987+0000","last_clean":"2024-09-13T17:50:38.609987+0000","last_became_active":"2024-09-13T17:50:34.597621+0000","last_became_peered":"2024-09-13T17:50:34.597621+0000","last_unstale":"2024-09-13T17:50:38.609987+0000","last_undegraded":"2024-09-13T17:50:38.609987+0000","last_fullsized":"2024-09-13T17:50:38.609987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:23.133462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591133+0000","last_change":"2024-09-13T17:50:34.589893+0000","last_active":"2024-09-13T17:50:38.591133+0000","last_peered":"2024-09-13T17:50:38.591133+0000","last_clean":"2024-09-13T17:50:38.591133+0000","last_became_active":"2024-09-13T17:50:34.589626+0000","last_became_peered":"2024-09-13T17:50:34.589626+0000","last_unstale":"2024-09-13T17:50:38.591133+0000","last_undegraded":"2024-09-13T17:50:38.591133+0000","last_fullsized":"2024-09-13T17:50:38.591133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:41.321025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589874+0000","last_change":"2024-09-13T17:50:34.590046+0000","last_active":"2024-09-13T17:50:38.589874+0000","last_peered":"2024-09-13T17:50:38.589874+0000","last_clean":"2024-09-13T17:50:38.589874+0000","last_became_active":"2024-09-13T17:50:34.589757+0000","last_became_peered":"2024-09-13T17:50:34.589757+0000","last_unstale":"2024-09-13T17:50:38.589874+0000","last_undegraded":"2024-09-13T17:50:38.589874+0000","last_fullsized":"2024-09-13T17:50:38.589874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:28.239550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589800+0000","last_change":"2024-09-13T17:50:34.595518+0000","last_active":"2024-09-13T17:50:38.589800+0000","last_peered":"2024-09-13T17:50:38.589800+0000","last_clean":"2024-09-13T17:50:38.589800+0000","last_became_active":"2024-09-13T17:50:34.595242+0000","last_became_peered":"2024-09-13T17:50:34.595242+0000","last_unstale":"2024-09-13T17:50:38.589800+0000","last_undegraded":"2024-09-13T17:50:38.589800+0000","last_fullsized":"2024-09-13T17:50:38.589800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:26.149384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589330+0000","last_change":"2024-09-13T17:50:34.593183+0000","last_active":"2024-09-13T17:50:38.589330+0000","last_peered":"2024-09-13T17:50:38.589330+0000","last_clean":"2024-09-13T17:50:38.589330+0000","last_became_active":"2024-09-13T17:50:34.592893+0000","last_became_peered":"2024-09-13T17:50:34.592893+0000","last_unstale":"2024-09-13T17:50:38.589330+0000","last_undegraded":"2024-09-13T17:50:38.589330+0000","last_fullsized":"2024-09-13T17:50:38.589330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:19.459505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599698+0000","last_change":"2024-09-13T17:50:34.594679+0000","last_active":"2024-09-13T17:50:38.599698+0000","last_peered":"2024-09-13T17:50:38.599698+0000","last_clean":"2024-09-13T17:50:38.599698+0000","last_became_active":"2024-09-13T17:50:34.593750+0000","last_became_peered":"2024-09-13T17:50:34.593750+0000","last_unstale":"2024-09-13T17:50:38.599698+0000","last_undegraded":"2024-09-13T17:50:38.599698+0000","last_fullsized":"2024-09-13T17:50:38.599698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:02.135577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.609111+0000","last_change":"2024-09-13T17:50:34.598736+0000","last_active":"2024-09-13T17:50:38.609111+0000","last_peered":"2024-09-13T17:50:38.609111+0000","last_clean":"2024-09-13T17:50:38.609111+0000","last_became_active":"2024-09-13T17:50:34.598433+0000","last_became_peered":"2024-09-13T17:50:34.598433+0000","last_unstale":"2024-09-13T17:50:38.609111+0000","last_undegraded":"2024-09-13T17:50:38.609111+0000","last_fullsized":"2024-09-13T17:50:38.609111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:08.975139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.592906+0000","last_change":"2024-09-13T17:50:34.591795+0000","last_active":"2024-09-13T17:50:38.592906+0000","last_peered":"2024-09-13T17:50:38.592906+0000","last_clean":"2024-09-13T17:50:38.592906+0000","last_became_active":"2024-09-13T17:50:34.591286+0000","last_became_peered":"2024-09-13T17:50:34.591286+0000","last_unstale":"2024-09-13T17:50:38.592906+0000","last_undegraded":"2024-09-13T17:50:38.592906+0000","last_fullsized":"2024-09-13T17:50:38.592906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:31.580247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.595844+0000","last_change":"2024-09-13T17:50:34.587774+0000","last_active":"2024-09-13T17:50:38.595844+0000","last_peered":"2024-09-13T17:50:38.595844+0000","last_clean":"2024-09-13T17:50:38.595844+0000","last_became_active":"2024-09-13T17:50:34.587506+0000","last_became_peered":"2024-09-13T17:50:34.587506+0000","last_unstale":"2024-09-13T17:50:38.595844+0000","last_undegraded":"2024-09-13T17:50:38.595844+0000","last_fullsized":"2024-09-13T17:50:38.595844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:46.275528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591171+0000","last_change":"2024-09-13T17:50:34.596100+0000","last_active":"2024-09-13T17:50:38.591171+0000","last_peered":"2024-09-13T17:50:38.591171+0000","last_clean":"2024-09-13T17:50:38.591171+0000","last_became_active":"2024-09-13T17:50:34.595798+0000","last_became_peered":"2024-09-13T17:50:34.595798+0000","last_unstale":"2024-09-13T17:50:38.591171+0000","last_undegraded":"2024-09-13T17:50:38.591171+0000","last_fullsized":"2024-09-13T17:50:38.591171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:49.543845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589759+0000","last_change":"2024-09-13T17:50:34.584818+0000","last_active":"2024-09-13T17:50:38.589759+0000","last_peered":"2024-09-13T17:50:38.589759+0000","last_clean":"2024-09-13T17:50:38.589759+0000","last_became_active":"2024-09-13T17:50:34.584516+0000","last_became_peered":"2024-09-13T17:50:34.584516+0000","last_unstale":"2024-09-13T17:50:38.589759+0000","last_undegraded":"2024-09-13T17:50:38.589759+0000","last_fullsized":"2024-09-13T17:50:38.589759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:52.402024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599531+0000","last_change":"2024-09-13T17:50:34.598886+0000","last_active":"2024-09-13T17:50:38.599531+0000","last_peered":"2024-09-13T17:50:38.599531+0000","last_clean":"2024-09-13T17:50:38.599531+0000","last_became_active":"2024-09-13T17:50:34.598648+0000","last_became_peered":"2024-09-13T17:50:34.598648+0000","last_unstale":"2024-09-13T17:50:38.599531+0000","last_undegraded":"2024-09-13T17:50:38.599531+0000","last_fullsized":"2024-09-13T17:50:38.599531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:00:43.063223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.603939+0000","last_change":"2024-09-13T17:50:34.582649+0000","last_active":"2024-09-13T17:50:38.603939+0000","last_peered":"2024-09-13T17:50:38.603939+0000","last_clean":"2024-09-13T17:50:38.603939+0000","last_became_active":"2024-09-13T17:50:34.581699+0000","last_became_peered":"2024-09-13T17:50:34.581699+0000","last_unstale":"2024-09-13T17:50:38.603939+0000","last_undegraded":"2024-09-13T17:50:38.603939+0000","last_fullsized":"2024-09-13T17:50:38.603939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:18.959303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.588487+0000","last_change":"2024-09-13T17:50:34.587484+0000","last_active":"2024-09-13T17:50:38.588487+0000","last_peered":"2024-09-13T17:50:38.588487+0000","last_clean":"2024-09-13T17:50:38.588487+0000","last_became_active":"2024-09-13T17:50:34.587217+0000","last_became_peered":"2024-09-13T17:50:34.587217+0000","last_unstale":"2024-09-13T17:50:38.588487+0000","last_undegraded":"2024-09-13T17:50:38.588487+0000","last_fullsized":"2024-09-13T17:50:38.588487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:24:19.172607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.602349+0000","last_change":"2024-09-13T17:50:34.592871+0000","last_active":"2024-09-13T17:50:38.602349+0000","last_peered":"2024-09-13T17:50:38.602349+0000","last_clean":"2024-09-13T17:50:38.602349+0000","last_became_active":"2024-09-13T17:50:34.588513+0000","last_became_peered":"2024-09-13T17:50:34.588513+0000","last_unstale":"2024-09-13T17:50:38.602349+0000","last_undegraded":"2024-09-13T17:50:38.602349+0000","last_fullsized":"2024-09-13T17:50:38.602349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:33.131420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.608986+0000","last_change":"2024-09-13T17:50:34.585713+0000","last_active":"2024-09-13T17:50:38.608986+0000","last_peered":"2024-09-13T17:50:38.608986+0000","last_clean":"2024-09-13T17:50:38.608986+0000","last_became_active":"2024-09-13T17:50:34.585438+0000","last_became_peered":"2024-09-13T17:50:34.585438+0000","last_unstale":"2024-09-13T17:50:38.608986+0000","last_undegraded":"2024-09-13T17:50:38.608986+0000","last_fullsized":"2024-09-13T17:50:38.608986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:56:49.905393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.609903+0000","last_change":"2024-09-13T17:50:34.589628+0000","last_active":"2024-09-13T17:50:38.609903+0000","last_peered":"2024-09-13T17:50:38.609903+0000","last_clean":"2024-09-13T17:50:38.609903+0000","last_became_active":"2024-09-13T17:50:34.587665+0000","last_became_peered":"2024-09-13T17:50:34.587665+0000","last_unstale":"2024-09-13T17:50:38.609903+0000","last_undegraded":"2024-09-13T17:50:38.609903+0000","last_fullsized":"2024-09-13T17:50:38.609903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:15.518264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.593116+0000","last_change":"2024-09-13T17:50:34.591776+0000","last_active":"2024-09-13T17:50:38.593116+0000","last_peered":"2024-09-13T17:50:38.593116+0000","last_clean":"2024-09-13T17:50:38.593116+0000","last_became_active":"2024-09-13T17:50:34.591496+0000","last_became_peered":"2024-09-13T17:50:34.591496+0000","last_unstale":"2024-09-13T17:50:38.593116+0000","last_undegraded":"2024-09-13T17:50:38.593116+0000","last_fullsized":"2024-09-13T17:50:38.593116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:48.651999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.588146+0000","last_change":"2024-09-13T17:50:34.582661+0000","last_active":"2024-09-13T17:50:38.588146+0000","last_peered":"2024-09-13T17:50:38.588146+0000","last_clean":"2024-09-13T17:50:38.588146+0000","last_became_active":"2024-09-13T17:50:34.582400+0000","last_became_peered":"2024-09-13T17:50:34.582400+0000","last_unstale":"2024-09-13T17:50:38.588146+0000","last_undegraded":"2024-09-13T17:50:38.588146+0000","last_fullsized":"2024-09-13T17:50:38.588146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:16:59.648638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589018+0000","last_change":"2024-09-13T17:50:34.585531+0000","last_active":"2024-09-13T17:50:38.589018+0000","last_peered":"2024-09-13T17:50:38.589018+0000","last_clean":"2024-09-13T17:50:38.589018+0000","last_became_active":"2024-09-13T17:50:34.585226+0000","last_became_peered":"2024-09-13T17:50:34.585226+0000","last_unstale":"2024-09-13T17:50:38.589018+0000","last_undegraded":"2024-09-13T17:50:38.589018+0000","last_fullsized":"2024-09-13T17:50:38.589018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:17.105028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589126+0000","last_change":"2024-09-13T17:50:34.586750+0000","last_active":"2024-09-13T17:50:38.589126+0000","last_peered":"2024-09-13T17:50:38.589126+0000","last_clean":"2024-09-13T17:50:38.589126+0000","last_became_active":"2024-09-13T17:50:34.584564+0000","last_became_peered":"2024-09-13T17:50:34.584564+0000","last_unstale":"2024-09-13T17:50:38.589126+0000","last_undegraded":"2024-09-13T17:50:38.589126+0000","last_fullsized":"2024-09-13T17:50:38.589126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:54.033257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591038+0000","last_change":"2024-09-13T17:50:34.585938+0000","last_active":"2024-09-13T17:50:38.591038+0000","last_peered":"2024-09-13T17:50:38.591038+0000","last_clean":"2024-09-13T17:50:38.591038+0000","last_became_active":"2024-09-13T17:50:34.585584+0000","last_became_peered":"2024-09-13T17:50:34.585584+0000","last_unstale":"2024-09-13T17:50:38.591038+0000","last_undegraded":"2024-09-13T17:50:38.591038+0000","last_fullsized":"2024-09-13T17:50:38.591038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:32.623468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590323+0000","last_change":"2024-09-13T17:50:34.588762+0000","last_active":"2024-09-13T17:50:38.590323+0000","last_peered":"2024-09-13T17:50:38.590323+0000","last_clean":"2024-09-13T17:50:38.590323+0000","last_became_active":"2024-09-13T17:50:34.588489+0000","last_became_peered":"2024-09-13T17:50:34.588489+0000","last_unstale":"2024-09-13T17:50:38.590323+0000","last_undegraded":"2024-09-13T17:50:38.590323+0000","last_fullsized":"2024-09-13T17:50:38.590323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:38.266297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.598178+0000","last_change":"2024-09-13T17:50:34.583889+0000","last_active":"2024-09-13T17:50:38.598178+0000","last_peered":"2024-09-13T17:50:38.598178+0000","last_clean":"2024-09-13T17:50:38.598178+0000","last_became_active":"2024-09-13T17:50:34.583551+0000","last_became_peered":"2024-09-13T17:50:34.583551+0000","last_unstale":"2024-09-13T17:50:38.598178+0000","last_undegraded":"2024-09-13T17:50:38.598178+0000","last_fullsized":"2024-09-13T17:50:38.598178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:19.888626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599385+0000","last_change":"2024-09-13T17:50:34.585617+0000","last_active":"2024-09-13T17:50:38.599385+0000","last_peered":"2024-09-13T17:50:38.599385+0000","last_clean":"2024-09-13T17:50:38.599385+0000","last_became_active":"2024-09-13T17:50:34.585340+0000","last_became_peered":"2024-09-13T17:50:34.585340+0000","last_unstale":"2024-09-13T17:50:38.599385+0000","last_undegraded":"2024-09-13T17:50:38.599385+0000","last_fullsized":"2024-09-13T17:50:38.599385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:18.331045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.593186+0000","last_change":"2024-09-13T17:50:34.592365+0000","last_active":"2024-09-13T17:50:38.593186+0000","last_peered":"2024-09-13T17:50:38.593186+0000","last_clean":"2024-09-13T17:50:38.593186+0000","last_became_active":"2024-09-13T17:50:34.592046+0000","last_became_peered":"2024-09-13T17:50:34.592046+0000","last_unstale":"2024-09-13T17:50:38.593186+0000","last_undegraded":"2024-09-13T17:50:38.593186+0000","last_fullsized":"2024-09-13T17:50:38.593186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:54.283683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":105750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":258048,"data_stored":101955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":102115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509451,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":864256,"data_stored":692961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]}]},{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":856064,"data_stored":692905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":270336,"data_stored":105750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":262144,"data_stored":102045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.373}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]}]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":270336,"data_stored":102081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:51:04.599 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T17:51:04.599 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T17:51:04.600 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T17:51:04.600 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T17:51:04.600 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T17:51:04.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.706+0000 7f710b4e9700 1 Processor -- start 2024-09-13T17:51:04.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.706+0000 7f710b4e9700 1 -- start start 2024-09-13T17:51:04.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710b4e9700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7104072350 0x7f7104150d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:04.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710b4e9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71040c7200 0x7f7104151280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:04.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710b4e9700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71040c55c0 con 0x7f7104071f70 2024-09-13T17:51:04.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710b4e9700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f710406a390 con 0x7f71040c7200 2024-09-13T17:51:04.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710b4e9700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7104157710 con 0x7f7104072350 2024-09-13T17:51:04.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710ace8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71040c7200 0x7f7104151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:04.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710ace8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71040c7200 0x7f7104151280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57810/0 (socket says 172.21.15.40:57810) 2024-09-13T17:51:04.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710ace8700 1 -- 172.21.15.40:0/3584303031 learned_addr learned my addr 172.21.15.40:0/3584303031 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:04.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f7109ce6700 1 --2- 172.21.15.40:0/3584303031 >> v2:172.21.15.43:3300/0 conn(0x7f7104072350 0x7f7104150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:04.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710ace8700 1 -- 172.21.15.40:0/3584303031 >> v2:172.21.15.43:3300/0 conn(0x7f7104072350 msgr2=0x7f7104150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:04.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710ace8700 1 --2- 172.21.15.40:0/3584303031 >> v2:172.21.15.43:3300/0 conn(0x7f7104072350 0x7f7104150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710ace8700 1 -- 172.21.15.40:0/3584303031 >> v1:172.21.15.43:6789/0 conn(0x7f7104071f70 legacy=0x7f71040c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:04.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710ace8700 1 -- 172.21.15.40:0/3584303031 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71041579d0 con 0x7f71040c7200 2024-09-13T17:51:04.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710ace8700 1 --2- 172.21.15.40:0/3584303031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71040c7200 0x7f7104151280 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f710406f500 tx=0x7f70fc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef935c2ea8c838d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:04.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f71094e5700 1 -- 172.21.15.40:0/3584303031 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc011830 con 0x7f71040c7200 2024-09-13T17:51:04.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f71094e5700 1 -- 172.21.15.40:0/3584303031 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70fc011990 con 0x7f71040c7200 2024-09-13T17:51:04.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f71094e5700 1 -- 172.21.15.40:0/3584303031 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc011d20 con 0x7f71040c7200 2024-09-13T17:51:04.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710b4e9700 1 -- 172.21.15.40:0/3584303031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71040c7200 msgr2=0x7f7104151280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:04.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710b4e9700 1 --2- 172.21.15.40:0/3584303031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71040c7200 0x7f7104151280 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f710406f500 tx=0x7f70fc009bf0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710b4e9700 1 -- 172.21.15.40:0/3584303031 shutdown_connections 2024-09-13T17:51:04.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710b4e9700 1 --2- 172.21.15.40:0/3584303031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71040c7200 0x7f7104151280 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710b4e9700 1 --2- 172.21.15.40:0/3584303031 >> v2:172.21.15.43:3300/0 conn(0x7f7104072350 0x7f7104150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710b4e9700 1 -- 172.21.15.40:0/3584303031 >> 172.21.15.40:0/3584303031 conn(0x7f71040cabc0 msgr2=0x7f71040cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:04.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710b4e9700 1 -- 172.21.15.40:0/3584303031 shutdown_connections 2024-09-13T17:51:04.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710b4e9700 1 -- 172.21.15.40:0/3584303031 wait complete. 2024-09-13T17:51:04.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710b4e9700 1 Processor -- start 2024-09-13T17:51:04.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.710+0000 7f710b4e9700 1 -- start start 2024-09-13T17:51:04.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f710b4e9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7104071f70 0x7f7104143790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:04.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f710b4e9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7104072350 0x7f710415a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:04.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f710b4e9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71040c7200 0x7f7104155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:04.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f710b4e9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7104071a90 con 0x7f7104072350 2024-09-13T17:51:04.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f710b4e9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710406fed0 con 0x7f71040c7200 2024-09-13T17:51:04.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f710b4e9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71040717b0 con 0x7f7104071f70 2024-09-13T17:51:04.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f7109ce6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7104072350 0x7f710415a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:04.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f7109ce6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7104072350 0x7f710415a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57818/0 (socket says 172.21.15.40:57818) 2024-09-13T17:51:04.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f710ace8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71040c7200 0x7f7104155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:04.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f7109ce6700 1 -- 172.21.15.40:0/2779170084 learned_addr learned my addr 172.21.15.40:0/2779170084 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:04.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f710a4e7700 1 --2- 172.21.15.40:0/2779170084 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7104071f70 0x7f7104143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:04.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f710ace8700 1 -- 172.21.15.40:0/2779170084 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7104071f70 msgr2=0x7f7104143790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:04.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f710ace8700 1 --2- 172.21.15.40:0/2779170084 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7104071f70 0x7f7104143790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f710ace8700 1 -- 172.21.15.40:0/2779170084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7104072350 msgr2=0x7f710415a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:04.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f710ace8700 1 --2- 172.21.15.40:0/2779170084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7104072350 0x7f710415a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f710ace8700 1 -- 172.21.15.40:0/2779170084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc009570 con 0x7f71040c7200 2024-09-13T17:51:04.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f7109ce6700 1 --2- 172.21.15.40:0/2779170084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7104072350 0x7f710415a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:04.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f710ace8700 1 --2- 172.21.15.40:0/2779170084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71040c7200 0x7f7104155e40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f71040672a0 tx=0x7f70fc004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:04.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f70faffd700 1 -- 172.21.15.40:0/2779170084 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc004450 con 0x7f71040c7200 2024-09-13T17:51:04.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f70faffd700 1 -- 172.21.15.40:0/2779170084 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70fc01e070 con 0x7f71040c7200 2024-09-13T17:51:04.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f710b4e9700 1 -- 172.21.15.40:0/2779170084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710415af50 con 0x7f71040c7200 2024-09-13T17:51:04.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f70faffd700 1 -- 172.21.15.40:0/2779170084 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc003900 con 0x7f71040c7200 2024-09-13T17:51:04.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.714+0000 7f710b4e9700 1 -- 172.21.15.40:0/2779170084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f710415a930 con 0x7f71040c7200 2024-09-13T17:51:04.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.718+0000 7f710b4e9700 1 -- 172.21.15.40:0/2779170084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f710415ad90 con 0x7f71040c7200 2024-09-13T17:51:04.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.718+0000 7f70faffd700 1 -- 172.21.15.40:0/2779170084 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f70fc003a60 con 0x7f71040c7200 2024-09-13T17:51:04.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.718+0000 7f70faffd700 1 --2- 172.21.15.40:0/2779170084 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70e8066730 0x7f70e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:04.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.718+0000 7f70faffd700 1 -- 172.21.15.40:0/2779170084 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f70fc020070 con 0x7f71040c7200 2024-09-13T17:51:04.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.718+0000 7f710a4e7700 1 --2- 172.21.15.40:0/2779170084 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70e8066730 0x7f70e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:04.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.718+0000 7f710a4e7700 1 --2- 172.21.15.40:0/2779170084 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70e8066730 0x7f70e8068bf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f70f4000f60 tx=0x7f70f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:04.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.726+0000 7f70faffd700 1 -- 172.21.15.40:0/2779170084 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f70fc0527f0 con 0x7f71040c7200 2024-09-13T17:51:04.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.942+0000 7f710b4e9700 1 -- 172.21.15.40:0/2779170084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f71040cb6e0 con 0x7f71040c7200 2024-09-13T17:51:04.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.950+0000 7f70faffd700 1 -- 172.21.15.40:0/2779170084 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164900 (secure 0 0 0) 0x7f70fc0045d0 con 0x7f71040c7200 2024-09-13T17:51:04.956 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:04.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.954+0000 7f710b4e9700 1 -- 172.21.15.40:0/2779170084 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70e8066730 msgr2=0x7f70e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:04.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.954+0000 7f710b4e9700 1 --2- 172.21.15.40:0/2779170084 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70e8066730 0x7f70e8068bf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f70f4000f60 tx=0x7f70f4009040 comp rx=0 tx=0).stop 2024-09-13T17:51:04.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.954+0000 7f710b4e9700 1 -- 172.21.15.40:0/2779170084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71040c7200 msgr2=0x7f7104155e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:04.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.954+0000 7f710b4e9700 1 --2- 172.21.15.40:0/2779170084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71040c7200 0x7f7104155e40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f71040672a0 tx=0x7f70fc004770 comp rx=0 tx=0).stop 2024-09-13T17:51:04.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.954+0000 7f710b4e9700 1 -- 172.21.15.40:0/2779170084 shutdown_connections 2024-09-13T17:51:04.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.954+0000 7f710b4e9700 1 --2- 172.21.15.40:0/2779170084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71040c7200 0x7f7104155e40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.954+0000 7f710b4e9700 1 --2- 172.21.15.40:0/2779170084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7104072350 0x7f710415a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.954+0000 7f710b4e9700 1 --2- 172.21.15.40:0/2779170084 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70e8066730 0x7f70e8068bf0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.954+0000 7f710b4e9700 1 --2- 172.21.15.40:0/2779170084 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7104071f70 0x7f7104143790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:04.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.954+0000 7f710b4e9700 1 -- 172.21.15.40:0/2779170084 >> 172.21.15.40:0/2779170084 conn(0x7f71040cabc0 msgr2=0x7f71040c3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:04.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.954+0000 7f710b4e9700 1 -- 172.21.15.40:0/2779170084 shutdown_connections 2024-09-13T17:51:04.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:04.954+0000 7f710b4e9700 1 -- 172.21.15.40:0/2779170084 wait complete. 2024-09-13T17:51:04.972 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":3394537609},{"type":"v1","addr":"172.21.15.40:6833","nonce":3394537609}]},"active_addr":"172.21.15.40:6833/3394537609","active_change":"2024-09-13T17:50:14.980022+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":3209422443}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":2512263266}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":2484122676}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.40:0","nonce":1894778110}]}]} 2024-09-13T17:51:04.975 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T17:51:04.975 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T17:51:04.975 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:51:05.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.086+0000 7f582dc4a700 1 Processor -- start 2024-09-13T17:51:05.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.086+0000 7f582dc4a700 1 -- start start 2024-09-13T17:51:05.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.086+0000 7f582dc4a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5828070500 0x7f5828071b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.086+0000 7f582dc4a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5828072080 0x7f58281608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.086+0000 7f582dc4a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58280cba20 con 0x7f58281500c0 2024-09-13T17:51:05.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.086+0000 7f582dc4a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58280724f0 con 0x7f5828070500 2024-09-13T17:51:05.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.086+0000 7f582dc4a700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58281560e0 con 0x7f5828072080 2024-09-13T17:51:05.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.086+0000 7f5827fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5828070500 0x7f5828071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.086+0000 7f582cc48700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f58281500c0 0x7f58280c66d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46058/0 (socket says 172.21.15.40:46058) 2024-09-13T17:51:05.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.086+0000 7f5827fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5828070500 0x7f5828071b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57830/0 (socket says 172.21.15.40:57830) 2024-09-13T17:51:05.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.086+0000 7f582cc48700 1 -- 172.21.15.40:0/442635319 learned_addr learned my addr 172.21.15.40:0/442635319 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:05.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.090+0000 7f582d449700 1 --2- 172.21.15.40:0/442635319 >> v2:172.21.15.43:3300/0 conn(0x7f5828072080 0x7f58281608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.090+0000 7f5827fff700 1 -- 172.21.15.40:0/442635319 >> v2:172.21.15.43:3300/0 conn(0x7f5828072080 msgr2=0x7f58281608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:05.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.090+0000 7f5827fff700 1 --2- 172.21.15.40:0/442635319 >> v2:172.21.15.43:3300/0 conn(0x7f5828072080 0x7f58281608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.090+0000 7f5827fff700 1 -- 172.21.15.40:0/442635319 >> v1:172.21.15.40:6789/0 conn(0x7f58281500c0 legacy=0x7f58280c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.090+0000 7f5827fff700 1 -- 172.21.15.40:0/442635319 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58281563c0 con 0x7f5828070500 2024-09-13T17:51:05.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.090+0000 7f5827fff700 1 --2- 172.21.15.40:0/442635319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5828070500 0x7f5828071b40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f58280c5280 tx=0x7f5814009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3065e60a25eae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.090+0000 7f58277fe700 1 -- 172.21.15.40:0/442635319 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5814011830 con 0x7f5828070500 2024-09-13T17:51:05.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.090+0000 7f58277fe700 1 -- 172.21.15.40:0/442635319 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5814011990 con 0x7f5828070500 2024-09-13T17:51:05.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.090+0000 7f58277fe700 1 -- 172.21.15.40:0/442635319 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5814011d40 con 0x7f5828070500 2024-09-13T17:51:05.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.090+0000 7f582dc4a700 1 -- 172.21.15.40:0/442635319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5828070500 msgr2=0x7f5828071b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.090+0000 7f582dc4a700 1 --2- 172.21.15.40:0/442635319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5828070500 0x7f5828071b40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f58280c5280 tx=0x7f5814009b30 comp rx=0 tx=0).stop 2024-09-13T17:51:05.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.090+0000 7f582dc4a700 1 -- 172.21.15.40:0/442635319 shutdown_connections 2024-09-13T17:51:05.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.090+0000 7f582dc4a700 1 --2- 172.21.15.40:0/442635319 >> v2:172.21.15.43:3300/0 conn(0x7f5828072080 0x7f58281608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.090+0000 7f582dc4a700 1 --2- 172.21.15.40:0/442635319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5828070500 0x7f5828071b40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.090+0000 7f582dc4a700 1 -- 172.21.15.40:0/442635319 >> 172.21.15.40:0/442635319 conn(0x7f58280cabc0 msgr2=0x7f58280cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:05.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.090+0000 7f582dc4a700 1 -- 172.21.15.40:0/442635319 shutdown_connections 2024-09-13T17:51:05.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.090+0000 7f582dc4a700 1 -- 172.21.15.40:0/442635319 wait complete. 2024-09-13T17:51:05.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.090+0000 7f582dc4a700 1 Processor -- start 2024-09-13T17:51:05.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582dc4a700 1 -- start start 2024-09-13T17:51:05.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582dc4a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5828070500 0x7f5828145f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582dc4a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5828072080 0x7f58281413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582dc4a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58281500c0 0x7f58281ee010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582dc4a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58281475c0 con 0x7f58281500c0 2024-09-13T17:51:05.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582dc4a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582814f5b0 con 0x7f5828072080 2024-09-13T17:51:05.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582dc4a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5828147810 con 0x7f5828070500 2024-09-13T17:51:05.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582d449700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58281500c0 0x7f58281ee010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582d449700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58281500c0 0x7f58281ee010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57844/0 (socket says 172.21.15.40:57844) 2024-09-13T17:51:05.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582d449700 1 -- 172.21.15.40:0/10795681 learned_addr learned my addr 172.21.15.40:0/10795681 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:05.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582cc48700 1 --2- 172.21.15.40:0/10795681 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5828070500 0x7f5828145f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f5827fff700 1 --2- 172.21.15.40:0/10795681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5828072080 0x7f58281413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582d449700 1 -- 172.21.15.40:0/10795681 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5828070500 msgr2=0x7f5828145f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582d449700 1 --2- 172.21.15.40:0/10795681 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5828070500 0x7f5828145f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582d449700 1 -- 172.21.15.40:0/10795681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5828072080 msgr2=0x7f58281413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582d449700 1 --2- 172.21.15.40:0/10795681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5828072080 0x7f58281413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582d449700 1 -- 172.21.15.40:0/10795681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58140094b0 con 0x7f58281500c0 2024-09-13T17:51:05.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582cc48700 1 --2- 172.21.15.40:0/10795681 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5828070500 0x7f5828145f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:05.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f5827fff700 1 --2- 172.21.15.40:0/10795681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5828072080 0x7f58281413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:05.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582d449700 1 --2- 172.21.15.40:0/10795681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58281500c0 0x7f58281ee010 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5820007d90 tx=0x7f582000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f58257fa700 1 -- 172.21.15.40:0/10795681 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582000ede0 con 0x7f58281500c0 2024-09-13T17:51:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f58257fa700 1 -- 172.21.15.40:0/10795681 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5820014420 con 0x7f58281500c0 2024-09-13T17:51:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582dc4a700 1 -- 172.21.15.40:0/10795681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5828147af0 con 0x7f58281500c0 2024-09-13T17:51:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f58257fa700 1 -- 172.21.15.40:0/10795681 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58200147d0 con 0x7f58281500c0 2024-09-13T17:51:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582dc4a700 1 -- 172.21.15.40:0/10795681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582813f620 con 0x7f58281500c0 2024-09-13T17:51:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.094+0000 7f582dc4a700 1 -- 172.21.15.40:0/10795681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582814d410 con 0x7f58281500c0 2024-09-13T17:51:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.098+0000 7f58257fa700 1 -- 172.21.15.40:0/10795681 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5820014930 con 0x7f58281500c0 2024-09-13T17:51:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.098+0000 7f58257fa700 1 --2- 172.21.15.40:0/10795681 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f580c066730 0x7f580c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.098+0000 7f58257fa700 1 -- 172.21.15.40:0/10795681 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5820085a30 con 0x7f58281500c0 2024-09-13T17:51:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.098+0000 7f582cc48700 1 --2- 172.21.15.40:0/10795681 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f580c066730 0x7f580c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.098+0000 7f582cc48700 1 --2- 172.21.15.40:0/10795681 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f580c066730 0x7f580c068bf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f581c007fd0 tx=0x7f581c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.106+0000 7f58257fa700 1 -- 172.21.15.40:0/10795681 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5820052b90 con 0x7f58281500c0 2024-09-13T17:51:05.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.282+0000 7f582dc4a700 1 -- 172.21.15.40:0/10795681 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58280cb6e0 con 0x7f58281500c0 2024-09-13T17:51:05.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.282+0000 7f58257fa700 1 -- 172.21.15.40:0/10795681 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18262 (secure 0 0 0) 0x7f5820083e60 con 0x7f58281500c0 2024-09-13T17:51:05.290 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:05.290 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:50:38.974071+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:50:32.740336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:50:33.552914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:50:36.623639+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:51:05.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.286+0000 7f582dc4a700 1 -- 172.21.15.40:0/10795681 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f580c066730 msgr2=0x7f580c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.286+0000 7f582dc4a700 1 --2- 172.21.15.40:0/10795681 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f580c066730 0x7f580c068bf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f581c007fd0 tx=0x7f581c009040 comp rx=0 tx=0).stop 2024-09-13T17:51:05.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.286+0000 7f582dc4a700 1 -- 172.21.15.40:0/10795681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58281500c0 msgr2=0x7f58281ee010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.286+0000 7f582dc4a700 1 --2- 172.21.15.40:0/10795681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58281500c0 0x7f58281ee010 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5820007d90 tx=0x7f582000e3c0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.286+0000 7f582dc4a700 1 -- 172.21.15.40:0/10795681 shutdown_connections 2024-09-13T17:51:05.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.286+0000 7f582dc4a700 1 --2- 172.21.15.40:0/10795681 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58281500c0 0x7f58281ee010 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.286+0000 7f582dc4a700 1 --2- 172.21.15.40:0/10795681 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5828072080 0x7f58281413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.286+0000 7f582dc4a700 1 --2- 172.21.15.40:0/10795681 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f580c066730 0x7f580c068bf0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.286+0000 7f582dc4a700 1 --2- 172.21.15.40:0/10795681 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5828070500 0x7f5828145f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.286+0000 7f582dc4a700 1 -- 172.21.15.40:0/10795681 >> 172.21.15.40:0/10795681 conn(0x7f58280cabc0 msgr2=0x7f5828148590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:05.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.286+0000 7f582dc4a700 1 -- 172.21.15.40:0/10795681 shutdown_connections 2024-09-13T17:51:05.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.286+0000 7f582dc4a700 1 -- 172.21.15.40:0/10795681 wait complete. 2024-09-13T17:51:05.306 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T17:51:05.307 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:51:05.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.418+0000 7fa7169e7700 1 Processor -- start 2024-09-13T17:51:05.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.418+0000 7fa7169e7700 1 -- start start 2024-09-13T17:51:05.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7169e7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa710154240 0x7fa71015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7169e7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa71015f0a0 0x7fa71006aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7169e7700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa71015f510 con 0x7fa71015ec60 2024-09-13T17:51:05.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7169e7700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7100721e0 con 0x7fa71015f0a0 2024-09-13T17:51:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7169e7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71015e6e0 con 0x7fa710154240 2024-09-13T17:51:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7161e6700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa71015f0a0 0x7fa71006aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7151e4700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa71015ec60 0x7fa71006a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46060/0 (socket says 172.21.15.40:46060) 2024-09-13T17:51:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7151e4700 1 -- 172.21.15.40:0/2937183200 learned_addr learned my addr 172.21.15.40:0/2937183200 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:05.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7161e6700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa71015f0a0 0x7fa71006aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57856/0 (socket says 172.21.15.40:57856) 2024-09-13T17:51:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7161e6700 1 -- 172.21.15.40:0/2937183200 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa710154240 msgr2=0x7fa71015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7161e6700 1 --2- 172.21.15.40:0/2937183200 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa710154240 0x7fa71015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7161e6700 1 -- 172.21.15.40:0/2937183200 >> v1:172.21.15.40:6789/0 conn(0x7fa71015ec60 legacy=0x7fa71006a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7161e6700 1 -- 172.21.15.40:0/2937183200 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa710071690 con 0x7fa71015f0a0 2024-09-13T17:51:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7161e6700 1 --2- 172.21.15.40:0/2937183200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa71015f0a0 0x7fa71006aaa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa708009850 tx=0x7fa708009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830263281772c26 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7149e3700 1 -- 172.21.15.40:0/2937183200 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa708011830 con 0x7fa71015f0a0 2024-09-13T17:51:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7149e3700 1 -- 172.21.15.40:0/2937183200 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa708011990 con 0x7fa71015f0a0 2024-09-13T17:51:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7149e3700 1 -- 172.21.15.40:0/2937183200 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa708011d20 con 0x7fa71015f0a0 2024-09-13T17:51:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7169e7700 1 -- 172.21.15.40:0/2937183200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa71015f0a0 msgr2=0x7fa71006aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7169e7700 1 --2- 172.21.15.40:0/2937183200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa71015f0a0 0x7fa71006aaa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa708009850 tx=0x7fa708009b60 comp rx=0 tx=0).stop 2024-09-13T17:51:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7169e7700 1 -- 172.21.15.40:0/2937183200 shutdown_connections 2024-09-13T17:51:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7169e7700 1 --2- 172.21.15.40:0/2937183200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa71015f0a0 0x7fa71006aaa0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7169e7700 1 --2- 172.21.15.40:0/2937183200 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa710154240 0x7fa71015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7169e7700 1 -- 172.21.15.40:0/2937183200 >> 172.21.15.40:0/2937183200 conn(0x7fa7100cabc0 msgr2=0x7fa7100cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7169e7700 1 -- 172.21.15.40:0/2937183200 shutdown_connections 2024-09-13T17:51:05.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.422+0000 7fa7169e7700 1 -- 172.21.15.40:0/2937183200 wait complete. 2024-09-13T17:51:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7169e7700 1 Processor -- start 2024-09-13T17:51:05.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7169e7700 1 -- start start 2024-09-13T17:51:05.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7169e7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa710154240 0x7fa7101f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7169e7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa71015ec60 0x7fa71014e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7169e7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa71015f0a0 0x7fa710157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7169e7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7101f3880 con 0x7fa71015ec60 2024-09-13T17:51:05.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7169e7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7100c5430 con 0x7fa710154240 2024-09-13T17:51:05.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7169e7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7100c56d0 con 0x7fa71015f0a0 2024-09-13T17:51:05.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7151e4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa71015ec60 0x7fa71014e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7151e4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa71015ec60 0x7fa71014e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57858/0 (socket says 172.21.15.40:57858) 2024-09-13T17:51:05.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7151e4700 1 -- 172.21.15.40:0/2694023394 learned_addr learned my addr 172.21.15.40:0/2694023394 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:05.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7159e5700 1 --2- 172.21.15.40:0/2694023394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa710154240 0x7fa7101f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7161e6700 1 --2- 172.21.15.40:0/2694023394 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa71015f0a0 0x7fa710157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7151e4700 1 -- 172.21.15.40:0/2694023394 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa71015f0a0 msgr2=0x7fa710157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7151e4700 1 --2- 172.21.15.40:0/2694023394 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa71015f0a0 0x7fa710157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7151e4700 1 -- 172.21.15.40:0/2694023394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa710154240 msgr2=0x7fa7101f3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7151e4700 1 --2- 172.21.15.40:0/2694023394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa710154240 0x7fa7101f3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7151e4700 1 -- 172.21.15.40:0/2694023394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7080094b0 con 0x7fa71015ec60 2024-09-13T17:51:05.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7151e4700 1 --2- 172.21.15.40:0/2694023394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa71015ec60 0x7fa71014e5d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa70400cba0 tx=0x7fa70400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7027fc700 1 -- 172.21.15.40:0/2694023394 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70400a770 con 0x7fa71015ec60 2024-09-13T17:51:05.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7027fc700 1 -- 172.21.15.40:0/2694023394 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70400a8d0 con 0x7fa71015ec60 2024-09-13T17:51:05.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7169e7700 1 -- 172.21.15.40:0/2694023394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7100c59b0 con 0x7fa71015ec60 2024-09-13T17:51:05.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7027fc700 1 -- 172.21.15.40:0/2694023394 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70400ac40 con 0x7fa71015ec60 2024-09-13T17:51:05.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.426+0000 7fa7169e7700 1 -- 172.21.15.40:0/2694023394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7101f31c0 con 0x7fa71015ec60 2024-09-13T17:51:05.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.430+0000 7fa7169e7700 1 -- 172.21.15.40:0/2694023394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa710070d60 con 0x7fa71015ec60 2024-09-13T17:51:05.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.430+0000 7fa7027fc700 1 -- 172.21.15.40:0/2694023394 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa70401b3b0 con 0x7fa71015ec60 2024-09-13T17:51:05.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.430+0000 7fa7027fc700 1 --2- 172.21.15.40:0/2694023394 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa6f4066730 0x7fa6f4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.430+0000 7fa7027fc700 1 -- 172.21.15.40:0/2694023394 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa704083fd0 con 0x7fa71015ec60 2024-09-13T17:51:05.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.430+0000 7fa7159e5700 1 --2- 172.21.15.40:0/2694023394 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa6f4066730 0x7fa6f4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.430+0000 7fa7159e5700 1 --2- 172.21.15.40:0/2694023394 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa6f4066730 0x7fa6f4068bf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009ac0 tx=0x7fa70c0091c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.442+0000 7fa7027fc700 1 -- 172.21.15.40:0/2694023394 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa704051150 con 0x7fa71015ec60 2024-09-13T17:51:05.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.614+0000 7fa7169e7700 1 -- 172.21.15.40:0/2694023394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa71015f510 con 0x7fa71015ec60 2024-09-13T17:51:05.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.618+0000 7fa7027fc700 1 -- 172.21.15.40:0/2694023394 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18262 (secure 0 0 0) 0x7fa71015f510 con 0x7fa71015ec60 2024-09-13T17:51:05.622 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:05.622 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:50:38.974071+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:50:32.740336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:50:33.552914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:50:36.623639+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:51:05.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.618+0000 7fa7169e7700 1 -- 172.21.15.40:0/2694023394 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa6f4066730 msgr2=0x7fa6f4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.618+0000 7fa7169e7700 1 --2- 172.21.15.40:0/2694023394 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa6f4066730 0x7fa6f4068bf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009ac0 tx=0x7fa70c0091c0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.618+0000 7fa7169e7700 1 -- 172.21.15.40:0/2694023394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa71015ec60 msgr2=0x7fa71014e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.618+0000 7fa7169e7700 1 --2- 172.21.15.40:0/2694023394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa71015ec60 0x7fa71014e5d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa70400cba0 tx=0x7fa70400cbd0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.618+0000 7fa7169e7700 1 -- 172.21.15.40:0/2694023394 shutdown_connections 2024-09-13T17:51:05.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.618+0000 7fa7169e7700 1 --2- 172.21.15.40:0/2694023394 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa6f4066730 0x7fa6f4068bf0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.618+0000 7fa7169e7700 1 --2- 172.21.15.40:0/2694023394 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa71015f0a0 0x7fa710157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.618+0000 7fa7169e7700 1 --2- 172.21.15.40:0/2694023394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa71015ec60 0x7fa71014e5d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.618+0000 7fa7169e7700 1 --2- 172.21.15.40:0/2694023394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa710154240 0x7fa7101f3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.618+0000 7fa7169e7700 1 -- 172.21.15.40:0/2694023394 >> 172.21.15.40:0/2694023394 conn(0x7fa7100cabc0 msgr2=0x7fa71015de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:05.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.618+0000 7fa7169e7700 1 -- 172.21.15.40:0/2694023394 shutdown_connections 2024-09-13T17:51:05.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.618+0000 7fa7169e7700 1 -- 172.21.15.40:0/2694023394 wait complete. 2024-09-13T17:51:05.638 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T17:51:05.638 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T17:51:05.638 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T17:51:05.639 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T17:51:05.639 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T17:51:05.640 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T17:51:05.640 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T17:51:05.640 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T17:51:05.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c509e1700 1 Processor -- start 2024-09-13T17:51:05.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c509e1700 1 -- start start 2024-09-13T17:51:05.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7fe89e281700 1 Processor -- start 2024-09-13T17:51:05.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c509e1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9c4c14f8c0 0x7f9c4c0c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c509e1700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9c4c0724e0 0x7f9c4c06d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c509e1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c4c158bc0 0x7f9c4c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c509e1700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4c0c7700 con 0x7f9c4c14f8c0 2024-09-13T17:51:05.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c509e1700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4c0c6890 con 0x7f9c4c0724e0 2024-09-13T17:51:05.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c509e1700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c0c5f30 con 0x7f9c4c158bc0 2024-09-13T17:51:05.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c4ad9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9c4c14f8c0 0x7f9c4c0c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c4ad9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9c4c14f8c0 0x7f9c4c0c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57860/0 (socket says 172.21.15.40:57860) 2024-09-13T17:51:05.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c4ad9d700 1 -- 172.21.15.40:0/4142385680 learned_addr learned my addr 172.21.15.40:0/4142385680 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:05.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c4a59c700 1 --2- 172.21.15.40:0/4142385680 >> v2:172.21.15.43:3300/0 conn(0x7f9c4c0724e0 0x7f9c4c06d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c4ad9d700 1 -- 172.21.15.40:0/4142385680 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c4c158bc0 msgr2=0x7f9c4c06d810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T17:51:05.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c4ad9d700 1 --2- 172.21.15.40:0/4142385680 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c4c158bc0 0x7f9c4c06d810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c4ad9d700 1 -- 172.21.15.40:0/4142385680 >> v2:172.21.15.43:3300/0 conn(0x7f9c4c0724e0 msgr2=0x7f9c4c06d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:05.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c4ad9d700 1 --2- 172.21.15.40:0/4142385680 >> v2:172.21.15.43:3300/0 conn(0x7f9c4c0724e0 0x7f9c4c06d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c4ad9d700 1 -- 172.21.15.40:0/4142385680 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c0c1320 con 0x7f9c4c14f8c0 2024-09-13T17:51:05.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c4ad9d700 1 --2- 172.21.15.40:0/4142385680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c4c14f8c0 0x7f9c4c0c71c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0c0580 tx=0x7f9c3c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8bb06d99aab267 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c49d9b700 1 -- 172.21.15.40:0/4142385680 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c00b8a0 con 0x7f9c4c14f8c0 2024-09-13T17:51:05.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c49d9b700 1 -- 172.21.15.40:0/4142385680 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3c00ba00 con 0x7f9c4c14f8c0 2024-09-13T17:51:05.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7f9c49d9b700 1 -- 172.21.15.40:0/4142385680 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c00bd90 con 0x7f9c4c14f8c0 2024-09-13T17:51:05.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7fe89e281700 1 -- start start 2024-09-13T17:51:05.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7fe89e281700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe8980724e0 0x7fe8980c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7fe89e281700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe898158bc0 0x7fe89815ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7fe89e281700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8980c6890 con 0x7fe89814f8c0 2024-09-13T17:51:05.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7fe89e281700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8980c7700 con 0x7fe8980724e0 2024-09-13T17:51:05.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7fe89e281700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8980c5f30 con 0x7fe898158bc0 2024-09-13T17:51:05.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7ff6dbfff700 1 Processor -- start 2024-09-13T17:51:05.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7fe89ca7e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe8980724e0 0x7fe8980c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7fe89ca7e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe8980724e0 0x7fe8980c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57874/0 (socket says 172.21.15.40:57874) 2024-09-13T17:51:05.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7fe89ca7e700 1 -- 172.21.15.40:0/3845431604 learned_addr learned my addr 172.21.15.40:0/3845431604 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:05.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7fe89da80700 1 --2- 172.21.15.40:0/3845431604 >> v2:172.21.15.43:3300/0 conn(0x7fe898158bc0 0x7fe89815ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7fe89ca7e700 1 -- 172.21.15.40:0/3845431604 >> v2:172.21.15.43:3300/0 conn(0x7fe898158bc0 msgr2=0x7fe89815ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:05.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7fe89ca7e700 1 --2- 172.21.15.40:0/3845431604 >> v2:172.21.15.43:3300/0 conn(0x7fe898158bc0 0x7fe89815ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7fe89ca7e700 1 -- 172.21.15.40:0/3845431604 >> v1:172.21.15.40:6789/0 conn(0x7fe89814f8c0 legacy=0x7fe89806d2d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:51:05.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fe89ca7e700 1 -- 172.21.15.40:0/3845431604 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8980c1320 con 0x7fe8980724e0 2024-09-13T17:51:05.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fe89ca7e700 1 --2- 172.21.15.40:0/3845431604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8980724e0 0x7fe8980c71c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe89806b5a0 tx=0x7fe89000c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56016b1b963aaf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.790+0000 7ff6dbfff700 1 -- start start 2024-09-13T17:51:05.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6dbfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff6dc0724e0 0x7ff6dc0c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6dbfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6dc158bc0 0x7ff6dc15ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6dbfff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6dc0c6890 con 0x7ff6dc14f8c0 2024-09-13T17:51:05.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6dbfff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6dc0c7700 con 0x7ff6dc0724e0 2024-09-13T17:51:05.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6dbfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc0c5f30 con 0x7ff6dc158bc0 2024-09-13T17:51:05.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fe88ffff700 1 -- 172.21.15.40:0/3845431604 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe890003c40 con 0x7fe8980724e0 2024-09-13T17:51:05.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fe88ffff700 1 -- 172.21.15.40:0/3845431604 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe890003da0 con 0x7fe8980724e0 2024-09-13T17:51:05.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fe88ffff700 1 -- 172.21.15.40:0/3845431604 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe890004270 con 0x7fe8980724e0 2024-09-13T17:51:05.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f9c509e1700 1 -- 172.21.15.40:0/4142385680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c4c14f8c0 msgr2=0x7f9c4c0c71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f9c509e1700 1 --2- 172.21.15.40:0/4142385680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c4c14f8c0 0x7f9c4c0c71c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0c0580 tx=0x7f9c3c009bf0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6daffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff6dc14f8c0 0x7ff6dc06d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46086/0 (socket says 172.21.15.40:46086) 2024-09-13T17:51:05.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6daffd700 1 -- 172.21.15.40:0/2146244466 learned_addr learned my addr 172.21.15.40:0/2146244466 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:05.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6db7fe700 1 --2- 172.21.15.40:0/2146244466 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6dc158bc0 0x7ff6dc15ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f9c509e1700 1 -- 172.21.15.40:0/4142385680 shutdown_connections 2024-09-13T17:51:05.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f9c509e1700 1 --2- 172.21.15.40:0/4142385680 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c4c158bc0 0x7f9c4c06d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f9c509e1700 1 --2- 172.21.15.40:0/4142385680 >> v2:172.21.15.43:3300/0 conn(0x7f9c4c0724e0 0x7f9c4c06d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f9c509e1700 1 --2- 172.21.15.40:0/4142385680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c4c14f8c0 0x7f9c4c0c71c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f9c509e1700 1 -- 172.21.15.40:0/4142385680 >> 172.21.15.40:0/4142385680 conn(0x7f9c4c066090 msgr2=0x7f9c4c0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:05.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f9c509e1700 1 -- 172.21.15.40:0/4142385680 shutdown_connections 2024-09-13T17:51:05.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f9c509e1700 1 -- 172.21.15.40:0/4142385680 wait complete. 2024-09-13T17:51:05.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6db7fe700 1 -- 172.21.15.40:0/2146244466 >> v2:172.21.15.43:3300/0 conn(0x7ff6dc0724e0 msgr2=0x7ff6dc0c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:05.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6db7fe700 1 --2- 172.21.15.40:0/2146244466 >> v2:172.21.15.43:3300/0 conn(0x7ff6dc0724e0 0x7ff6dc0c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6db7fe700 1 -- 172.21.15.40:0/2146244466 >> v1:172.21.15.40:6789/0 conn(0x7ff6dc14f8c0 legacy=0x7ff6dc06d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6db7fe700 1 -- 172.21.15.40:0/2146244466 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6dc0c1320 con 0x7ff6dc158bc0 2024-09-13T17:51:05.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fe89e281700 1 -- 172.21.15.40:0/3845431604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8980724e0 msgr2=0x7fe8980c71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fe89e281700 1 --2- 172.21.15.40:0/3845431604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8980724e0 0x7fe8980c71c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe89806b5a0 tx=0x7fe89000c770 comp rx=0 tx=0).stop 2024-09-13T17:51:05.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6db7fe700 1 --2- 172.21.15.40:0/2146244466 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6dc158bc0 0x7ff6dc15ff50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00a9f0 tx=0x7ff6cc00ad00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4185e1a6732c75ca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fe89e281700 1 -- 172.21.15.40:0/3845431604 shutdown_connections 2024-09-13T17:51:05.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fe89e281700 1 --2- 172.21.15.40:0/3845431604 >> v2:172.21.15.43:3300/0 conn(0x7fe898158bc0 0x7fe89815ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fe89e281700 1 --2- 172.21.15.40:0/3845431604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8980724e0 0x7fe8980c71c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fe89e281700 1 -- 172.21.15.40:0/3845431604 >> 172.21.15.40:0/3845431604 conn(0x7fe898066090 msgr2=0x7fe8980664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:05.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fe89e281700 1 -- 172.21.15.40:0/3845431604 shutdown_connections 2024-09-13T17:51:05.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6d9ffb700 1 -- 172.21.15.40:0/2146244466 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc004230 con 0x7ff6dc158bc0 2024-09-13T17:51:05.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6d9ffb700 1 -- 172.21.15.40:0/2146244466 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6cc004390 con 0x7ff6dc158bc0 2024-09-13T17:51:05.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6d9ffb700 1 -- 172.21.15.40:0/2146244466 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc00c870 con 0x7ff6dc158bc0 2024-09-13T17:51:05.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fe89e281700 1 -- 172.21.15.40:0/3845431604 wait complete. 2024-09-13T17:51:05.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f2166f68700 1 Processor -- start 2024-09-13T17:51:05.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f2166f68700 1 -- start start 2024-09-13T17:51:05.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f2166f68700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2160157e90 0x7f21600c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f2166f68700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21600c57d0 0x7f216015e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f2166f68700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21601619c0 con 0x7f216015e6b0 2024-09-13T17:51:05.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f2166f68700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2160158270 con 0x7f21600c57d0 2024-09-13T17:51:05.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f2166f68700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216015dd50 con 0x7f2160157e90 2024-09-13T17:51:05.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fbfc6a9a700 1 Processor -- start 2024-09-13T17:51:05.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fbfc6a9a700 1 -- start start 2024-09-13T17:51:05.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fbfc6a9a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbfc00c7f60 0x7fbfc00c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fbfc6a9a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbfc00c8340 0x7fbfc00c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fbfc6a9a700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc006b400 con 0x7fbfc006d920 2024-09-13T17:51:05.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fbfc6a9a700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc00c7790 con 0x7fbfc00c8340 2024-09-13T17:51:05.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fbfc6a9a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc015e920 con 0x7fbfc00c7f60 2024-09-13T17:51:05.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f2166767700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f216015e6b0 0x7f216006d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46088/0 (socket says 172.21.15.40:46088) 2024-09-13T17:51:05.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f2166767700 1 -- 172.21.15.40:0/20220677 learned_addr learned my addr 172.21.15.40:0/20220677 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:05.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fbfc5a98700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbfc00c7f60 0x7fbfc00c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fbfc5a98700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbfc00c7f60 0x7fbfc00c61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:48842/0 (socket says 172.21.15.40:48842) 2024-09-13T17:51:05.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fbfc5a98700 1 -- 172.21.15.40:0/92921959 learned_addr learned my addr 172.21.15.40:0/92921959 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:05.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6dbfff700 1 -- 172.21.15.40:0/2146244466 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6dc158bc0 msgr2=0x7ff6dc15ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6dbfff700 1 --2- 172.21.15.40:0/2146244466 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6dc158bc0 0x7ff6dc15ff50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc00a9f0 tx=0x7ff6cc00ad00 comp rx=0 tx=0).stop 2024-09-13T17:51:05.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6dbfff700 1 -- 172.21.15.40:0/2146244466 shutdown_connections 2024-09-13T17:51:05.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6dbfff700 1 --2- 172.21.15.40:0/2146244466 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6dc158bc0 0x7ff6dc15ff50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6dbfff700 1 --2- 172.21.15.40:0/2146244466 >> v2:172.21.15.43:3300/0 conn(0x7ff6dc0724e0 0x7ff6dc0c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7ff6dbfff700 1 -- 172.21.15.40:0/2146244466 >> 172.21.15.40:0/2146244466 conn(0x7ff6dc066090 msgr2=0x7ff6dc0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:05.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fbfc5297700 1 --2- 172.21.15.40:0/92921959 >> v2:172.21.15.43:3300/0 conn(0x7fbfc00c8340 0x7fbfc00c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe89e281700 1 Processor -- start 2024-09-13T17:51:05.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f9c509e1700 1 Processor -- start 2024-09-13T17:51:05.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f9c509e1700 1 -- start start 2024-09-13T17:51:05.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7ff6dbfff700 1 -- 172.21.15.40:0/2146244466 shutdown_connections 2024-09-13T17:51:05.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7ff6dbfff700 1 -- 172.21.15.40:0/2146244466 wait complete. 2024-09-13T17:51:05.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f2164f64700 1 -- 172.21.15.40:0/20220677 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2302615165 0 0) 0x7f21601619c0 con 0x7f216015e6b0 2024-09-13T17:51:05.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f2165765700 1 --2- 172.21.15.40:0/20220677 >> v2:172.21.15.40:3300/0 conn(0x7f21600c57d0 0x7f216015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7f2164f64700 1 -- 172.21.15.40:0/20220677 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f214c0035d0 con 0x7f216015e6b0 2024-09-13T17:51:05.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fbfc5a98700 1 -- 172.21.15.40:0/92921959 >> v2:172.21.15.43:3300/0 conn(0x7fbfc00c8340 msgr2=0x7fbfc00c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:05.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fbfc5a98700 1 --2- 172.21.15.40:0/92921959 >> v2:172.21.15.43:3300/0 conn(0x7fbfc00c8340 0x7fbfc00c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fbfc5a98700 1 -- 172.21.15.40:0/92921959 >> v1:172.21.15.43:6789/0 conn(0x7fbfc006d920 legacy=0x7fbfc01577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:05.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.794+0000 7fbfc5a98700 1 -- 172.21.15.40:0/92921959 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc015dda0 con 0x7fbfc00c7f60 2024-09-13T17:51:05.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f2165f66700 1 --2- 172.21.15.40:0/20220677 >> v2:172.21.15.43:3300/0 conn(0x7f2160157e90 0x7f21600c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f2165765700 1 -- 172.21.15.40:0/20220677 >> v2:172.21.15.43:3300/0 conn(0x7f2160157e90 msgr2=0x7f21600c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:05.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f2165765700 1 --2- 172.21.15.40:0/20220677 >> v2:172.21.15.43:3300/0 conn(0x7f2160157e90 0x7f21600c71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f2165765700 1 -- 172.21.15.40:0/20220677 >> v1:172.21.15.40:6789/0 conn(0x7f216015e6b0 legacy=0x7f216006d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f2164f64700 1 -- 172.21.15.40:0/20220677 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2001497904 0 0) 0x7f214c0035d0 con 0x7f216015e6b0 2024-09-13T17:51:05.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f2165765700 1 -- 172.21.15.40:0/20220677 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2160161ed0 con 0x7f21600c57d0 2024-09-13T17:51:05.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f2165f66700 1 --2- 172.21.15.40:0/20220677 >> v2:172.21.15.43:3300/0 conn(0x7f2160157e90 0x7f21600c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:51:05.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fbfc5a98700 1 --2- 172.21.15.40:0/92921959 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbfc00c7f60 0x7fbfc00c61e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009910 tx=0x7fbfbc009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c24b108f932f8fc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe89e281700 1 -- start start 2024-09-13T17:51:05.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f2165765700 1 --2- 172.21.15.40:0/20220677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21600c57d0 0x7f216015e170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f21600cbe90 tx=0x7f215c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21c1cc63edd5d4d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fbfc4a96700 1 -- 172.21.15.40:0/92921959 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc004230 con 0x7fbfc00c7f60 2024-09-13T17:51:05.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe3a3fff700 1 Processor -- start 2024-09-13T17:51:05.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe3a3fff700 1 -- start start 2024-09-13T17:51:05.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f2164f64700 1 -- 172.21.15.40:0/20220677 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21580019f0 con 0x7f21600c57d0 2024-09-13T17:51:05.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fbfc4a96700 1 -- 172.21.15.40:0/92921959 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc004390 con 0x7fbfc00c7f60 2024-09-13T17:51:05.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fbfc4a96700 1 -- 172.21.15.40:0/92921959 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc009e60 con 0x7fbfc00c7f60 2024-09-13T17:51:05.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f2164f64700 1 -- 172.21.15.40:0/20220677 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c004030 con 0x7f21600c57d0 2024-09-13T17:51:05.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f2164f64700 1 -- 172.21.15.40:0/20220677 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c0043c0 con 0x7f21600c57d0 2024-09-13T17:51:05.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c509e1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c4c0724e0 0x7f9c4c14d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c509e1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c4c14f8c0 0x7f9c4c162660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c509e1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c4c158bc0 0x7f9c4c1455b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fd2a7725700 1 Processor -- start 2024-09-13T17:51:05.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fd2a7725700 1 -- start start 2024-09-13T17:51:05.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c509e1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c14d190 con 0x7f9c4c14f8c0 2024-09-13T17:51:05.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c509e1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c151460 con 0x7f9c4c158bc0 2024-09-13T17:51:05.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c4a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c4c14f8c0 0x7f9c4c162660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c509e1700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c14daf0 con 0x7f9c4c0724e0 2024-09-13T17:51:05.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c4a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c4c14f8c0 0x7f9c4c162660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57882/0 (socket says 172.21.15.40:57882) 2024-09-13T17:51:05.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c4a59c700 1 -- 172.21.15.40:0/650272344 learned_addr learned my addr 172.21.15.40:0/650272344 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:05.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f6f48fef700 1 Processor -- start 2024-09-13T17:51:05.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c4b59e700 1 --2- 172.21.15.40:0/650272344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c4c158bc0 0x7f9c4c1455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f6f48fef700 1 -- start start 2024-09-13T17:51:05.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe89e281700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8980724e0 0x7fe8980c0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c4ad9d700 1 --2- 172.21.15.40:0/650272344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c4c0724e0 0x7f9c4c14d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c4a59c700 1 -- 172.21.15.40:0/650272344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c4c0724e0 msgr2=0x7f9c4c14d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c4a59c700 1 --2- 172.21.15.40:0/650272344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c4c0724e0 0x7f9c4c14d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c4a59c700 1 -- 172.21.15.40:0/650272344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c4c158bc0 msgr2=0x7f9c4c1455b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c4a59c700 1 --2- 172.21.15.40:0/650272344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c4c158bc0 0x7f9c4c1455b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c4a59c700 1 -- 172.21.15.40:0/650272344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c009570 con 0x7f9c4c14f8c0 2024-09-13T17:51:05.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe89d27f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8980724e0 0x7fe8980c0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe89d27f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8980724e0 0x7fe8980c0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57894/0 (socket says 172.21.15.40:57894) 2024-09-13T17:51:05.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f2166f68700 1 -- 172.21.15.40:0/20220677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21600c57d0 msgr2=0x7f216015e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f2166f68700 1 --2- 172.21.15.40:0/20220677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21600c57d0 0x7f216015e170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f21600cbe90 tx=0x7f215c009b30 comp rx=0 tx=0).stop 2024-09-13T17:51:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe89d27f700 1 -- 172.21.15.40:0/1686426634 learned_addr learned my addr 172.21.15.40:0/1686426634 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe89e281700 1 --2- 172.21.15.40:0/1686426634 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe89814f8c0 0x7fe8981466c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe89e281700 1 --2- 172.21.15.40:0/1686426634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe898158bc0 0x7fe898146c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe89e281700 1 -- 172.21.15.40:0/1686426634 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8981f41c0 con 0x7fe8980724e0 2024-09-13T17:51:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe89e281700 1 -- 172.21.15.40:0/1686426634 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8981f3540 con 0x7fe898158bc0 2024-09-13T17:51:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe89e281700 1 -- 172.21.15.40:0/1686426634 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8981f3820 con 0x7fe89814f8c0 2024-09-13T17:51:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c4a59c700 1 --2- 172.21.15.40:0/650272344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c4c14f8c0 0x7f9c4c162660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000e760 tx=0x7f9c4000ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c337fe700 1 -- 172.21.15.40:0/650272344 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c40013610 con 0x7f9c4c14f8c0 2024-09-13T17:51:05.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe89ca7e700 1 --2- 172.21.15.40:0/1686426634 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe89814f8c0 0x7fe8981466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7ff6dbfff700 1 Processor -- start 2024-09-13T17:51:05.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7ff6dbfff700 1 -- start start 2024-09-13T17:51:05.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fd2a7725700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2a006faf0 0x7fd2a00c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fd2a7725700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a006d400 con 0x7fd2a00c6710 2024-09-13T17:51:05.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fd2a7725700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a014fc40 con 0x7fd2a00c57d0 2024-09-13T17:51:05.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fd2a7725700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a006b540 con 0x7fd2a006faf0 2024-09-13T17:51:05.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7ff6dbfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6dc0724e0 0x7ff6dc149730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7ff6dbfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6dc14f8c0 0x7ff6dc149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7ff6dbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6dc158bc0 0x7ff6dc147910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7ff6dbfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc151cb0 con 0x7ff6dc158bc0 2024-09-13T17:51:05.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7ff6dbfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc157610 con 0x7ff6dc14f8c0 2024-09-13T17:51:05.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7ff6dbfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc0c2260 con 0x7ff6dc0724e0 2024-09-13T17:51:05.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe89d27f700 1 -- 172.21.15.40:0/1686426634 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe89814f8c0 msgr2=0x7fe8981466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe89d27f700 1 --2- 172.21.15.40:0/1686426634 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe89814f8c0 0x7fe8981466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe89d27f700 1 -- 172.21.15.40:0/1686426634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe898158bc0 msgr2=0x7fe898146c00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T17:51:05.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe89d27f700 1 --2- 172.21.15.40:0/1686426634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe898158bc0 0x7fe898146c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe89d27f700 1 -- 172.21.15.40:0/1686426634 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89000c3d0 con 0x7fe8980724e0 2024-09-13T17:51:05.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c337fe700 1 -- 172.21.15.40:0/650272344 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c40013770 con 0x7f9c4c14f8c0 2024-09-13T17:51:05.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c509e1700 1 -- 172.21.15.40:0/650272344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c4c14b850 con 0x7f9c4c14f8c0 2024-09-13T17:51:05.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c337fe700 1 -- 172.21.15.40:0/650272344 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c40013ae0 con 0x7f9c4c14f8c0 2024-09-13T17:51:05.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f9c509e1700 1 -- 172.21.15.40:0/650272344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c4c14acf0 con 0x7f9c4c14f8c0 2024-09-13T17:51:05.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe89d27f700 1 --2- 172.21.15.40:0/1686426634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8980724e0 0x7fe8980c0860 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe88800c4e0 tx=0x7fe88800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe3a3fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe3a4150520 0x7fe3a41597d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7ff6daffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6dc0724e0 0x7ff6dc149730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7ff6daffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6dc0724e0 0x7ff6dc149730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:48866/0 (socket says 172.21.15.40:48866) 2024-09-13T17:51:05.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7ff6daffd700 1 -- 172.21.15.40:0/3618904688 learned_addr learned my addr 172.21.15.40:0/3618904688 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:05.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f2166f68700 1 -- 172.21.15.40:0/20220677 shutdown_connections 2024-09-13T17:51:05.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f2166f68700 1 --2- 172.21.15.40:0/20220677 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21600c57d0 0x7f216015e170 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f2166f68700 1 --2- 172.21.15.40:0/20220677 >> v2:172.21.15.43:3300/0 conn(0x7f2160157e90 0x7f21600c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f2166f68700 1 -- 172.21.15.40:0/20220677 >> 172.21.15.40:0/20220677 conn(0x7f2160066090 msgr2=0x7f21600664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:05.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fbfc6a9a700 1 -- 172.21.15.40:0/92921959 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbfc00c7f60 msgr2=0x7fbfc00c61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fbfc6a9a700 1 --2- 172.21.15.40:0/92921959 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbfc00c7f60 0x7fbfc00c61e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009910 tx=0x7fbfbc009c20 comp rx=0 tx=0).stop 2024-09-13T17:51:05.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f6f48fef700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f440c8500 0x7f6f440c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f6f48fef700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6f440c71c0 0x7f6f44159630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f6f48fef700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f440c8f80 con 0x7f6f441503c0 2024-09-13T17:51:05.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f6f48fef700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f440c2a80 con 0x7f6f440c71c0 2024-09-13T17:51:05.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f6f48fef700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f440c80d0 con 0x7f6f440c8500 2024-09-13T17:51:05.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f6f43fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6f440c71c0 0x7f6f44159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f6f43fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6f440c71c0 0x7f6f44159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57902/0 (socket says 172.21.15.40:57902) 2024-09-13T17:51:05.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7f6f43fff700 1 -- 172.21.15.40:0/1251654458 learned_addr learned my addr 172.21.15.40:0/1251654458 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:05.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe3a3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3a4072410 0x7fe3a4160b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe3a3fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a4150900 con 0x7fe3a4159d10 2024-09-13T17:51:05.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe3a2ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe3a4150520 0x7fe3a41597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fe3a2ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe3a4150520 0x7fe3a41597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:36942/0 (socket says 172.21.15.40:36942) 2024-09-13T17:51:05.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fe3a2ffd700 1 -- 172.21.15.40:0/1211108590 learned_addr learned my addr 172.21.15.40:0/1211108590 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:05.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fe88dffb700 1 -- 172.21.15.40:0/1686426634 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888011790 con 0x7fe8980724e0 2024-09-13T17:51:05.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fe88dffb700 1 -- 172.21.15.40:0/1686426634 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8880118f0 con 0x7fe8980724e0 2024-09-13T17:51:05.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fe88dffb700 1 -- 172.21.15.40:0/1686426634 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888011c80 con 0x7fe8980724e0 2024-09-13T17:51:05.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fe89e281700 1 -- 172.21.15.40:0/1686426634 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8981f3b00 con 0x7fe8980724e0 2024-09-13T17:51:05.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fe89e281700 1 -- 172.21.15.40:0/1686426634 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8980c0da0 con 0x7fe8980724e0 2024-09-13T17:51:05.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fe3a3fff700 1 -- 172.21.15.40:0/1211108590 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a4070aa0 con 0x7fe3a4150520 2024-09-13T17:51:05.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fe3a3fff700 1 -- 172.21.15.40:0/1211108590 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a4070d80 con 0x7fe3a4072410 2024-09-13T17:51:05.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fe3a2ffd700 1 -- 172.21.15.40:0/1211108590 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3a4072410 msgr2=0x7fe3a4160b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:05.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fe3a2ffd700 1 --2- 172.21.15.40:0/1211108590 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3a4072410 0x7fe3a4160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fe3a2ffd700 1 -- 172.21.15.40:0/1211108590 >> v1:172.21.15.43:6789/0 conn(0x7fe3a4159d10 legacy=0x7fe3a40c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fe3a2ffd700 1 -- 172.21.15.40:0/1211108590 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a40728c0 con 0x7fe3a4150520 2024-09-13T17:51:05.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7ff6da7fc700 1 --2- 172.21.15.40:0/3618904688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6dc14f8c0 0x7ff6dc149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7ff6db7fe700 1 --2- 172.21.15.40:0/3618904688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6dc158bc0 0x7ff6dc147910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7ff6daffd700 1 -- 172.21.15.40:0/3618904688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6dc14f8c0 msgr2=0x7ff6dc149c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7ff6daffd700 1 --2- 172.21.15.40:0/3618904688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6dc14f8c0 0x7ff6dc149c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7ff6daffd700 1 -- 172.21.15.40:0/3618904688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6dc158bc0 msgr2=0x7ff6dc147910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7ff6daffd700 1 --2- 172.21.15.40:0/3618904688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6dc158bc0 0x7ff6dc147910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7ff6daffd700 1 -- 172.21.15.40:0/3618904688 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc00a650 con 0x7ff6dc0724e0 2024-09-13T17:51:05.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fd2a5cc2700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd2a00c6710 0x7fd2a006d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46094/0 (socket says 172.21.15.40:46094) 2024-09-13T17:51:05.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.798+0000 7fd2a5cc2700 1 -- 172.21.15.40:0/1201912460 learned_addr learned my addr 172.21.15.40:0/1201912460 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:05.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd297fff700 1 -- 172.21.15.40:0/1201912460 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3728282396 0 0) 0x7fd2a006d400 con 0x7fd2a00c6710 2024-09-13T17:51:05.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd297fff700 1 -- 172.21.15.40:0/1201912460 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd28c0035d0 con 0x7fd2a00c6710 2024-09-13T17:51:05.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd297fff700 1 -- 172.21.15.40:0/1201912460 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2383057552 0 0) 0x7fd28c0035d0 con 0x7fd2a00c6710 2024-09-13T17:51:05.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd297fff700 1 -- 172.21.15.40:0/1201912460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2a006faf0 msgr2=0x7fd2a00c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:05.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd297fff700 1 --2- 172.21.15.40:0/1201912460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2a006faf0 0x7fd2a00c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd297fff700 1 -- 172.21.15.40:0/1201912460 >> v1:172.21.15.43:6789/0 conn(0x7fd2a00c57d0 legacy=0x7fd2a014f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd297fff700 1 -- 172.21.15.40:0/1201912460 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a0156190 con 0x7fd2a00c6710 2024-09-13T17:51:05.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd297fff700 1 -- 172.21.15.40:0/1201912460 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fd29c001670 con 0x7fd2a00c6710 2024-09-13T17:51:05.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd297fff700 1 -- 172.21.15.40:0/1201912460 >> v1:172.21.15.40:6789/0 conn(0x7fd2a00c6710 legacy=0x7fd2a006d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd297fff700 1 --2- 172.21.15.40:0/1201912460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd28c003a50 0x7fd28c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd297fff700 1 --2- 172.21.15.40:0/1201912460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2a006faf0 0x7fd28c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd297fff700 1 --2- 172.21.15.40:0/1201912460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd28c004e80 0x7fd28c005300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd297fff700 1 -- 172.21.15.40:0/1201912460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a006b540 con 0x7fd28c003a50 2024-09-13T17:51:05.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd297fff700 1 -- 172.21.15.40:0/1201912460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c00bcc0 con 0x7fd28c004e80 2024-09-13T17:51:05.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd297fff700 1 -- 172.21.15.40:0/1201912460 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c00be80 con 0x7fd2a006faf0 2024-09-13T17:51:05.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd2a4cc0700 1 --2- 172.21.15.40:0/1201912460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2a006faf0 0x7fd28c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f2166f68700 1 -- 172.21.15.40:0/20220677 shutdown_connections 2024-09-13T17:51:05.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f2166f68700 1 -- 172.21.15.40:0/20220677 wait complete. 2024-09-13T17:51:05.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f2166f68700 1 Processor -- start 2024-09-13T17:51:05.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fbfc6a9a700 1 -- 172.21.15.40:0/92921959 shutdown_connections 2024-09-13T17:51:05.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fbfc6a9a700 1 --2- 172.21.15.40:0/92921959 >> v2:172.21.15.43:3300/0 conn(0x7fbfc00c8340 0x7fbfc00c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fbfc6a9a700 1 --2- 172.21.15.40:0/92921959 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbfc00c7f60 0x7fbfc00c61e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fbfc6a9a700 1 -- 172.21.15.40:0/92921959 >> 172.21.15.40:0/92921959 conn(0x7fbfc0066090 msgr2=0x7fbfc00664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:05.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fbfc6a9a700 1 -- 172.21.15.40:0/92921959 shutdown_connections 2024-09-13T17:51:05.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fbfc6a9a700 1 -- 172.21.15.40:0/92921959 wait complete. 2024-09-13T17:51:05.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f6f42ffd700 1 --2- 172.21.15.40:0/1251654458 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f440c8500 0x7f6f440c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f6f43fff700 1 -- 172.21.15.40:0/1251654458 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f440c8500 msgr2=0x7f6f440c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:05.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f6f43fff700 1 --2- 172.21.15.40:0/1251654458 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f440c8500 0x7f6f440c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f6f43fff700 1 -- 172.21.15.40:0/1251654458 >> v1:172.21.15.43:6789/0 conn(0x7f6f441503c0 legacy=0x7f6f440c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:05.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f6f43fff700 1 -- 172.21.15.40:0/1251654458 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f441507a0 con 0x7f6f440c71c0 2024-09-13T17:51:05.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f6f43fff700 1 --2- 172.21.15.40:0/1251654458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f440c71c0 0x7f6f44159630 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00d150 tx=0x7f6f3c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c644d8f58aefb354 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fe89e281700 1 -- 172.21.15.40:0/1686426634 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe89814fca0 con 0x7fe8980724e0 2024-09-13T17:51:05.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f9c337fe700 1 -- 172.21.15.40:0/650272344 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9c40013c40 con 0x7f9c4c14f8c0 2024-09-13T17:51:05.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7ff6daffd700 1 --2- 172.21.15.40:0/3618904688 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6dc0724e0 0x7ff6dc149730 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4001be0 tx=0x7ff6d4001ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7ff6c37fe700 1 -- 172.21.15.40:0/3618904688 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4006570 con 0x7ff6dc0724e0 2024-09-13T17:51:05.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7ff6c37fe700 1 -- 172.21.15.40:0/3618904688 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d40066d0 con 0x7ff6dc0724e0 2024-09-13T17:51:05.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7ff6dbfff700 1 -- 172.21.15.40:0/3618904688 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6dc1f4c90 con 0x7ff6dc0724e0 2024-09-13T17:51:05.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7ff6dbfff700 1 -- 172.21.15.40:0/3618904688 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6dc1f7640 con 0x7ff6dc0724e0 2024-09-13T17:51:05.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7ff6c37fe700 1 -- 172.21.15.40:0/3618904688 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4006a80 con 0x7ff6dc0724e0 2024-09-13T17:51:05.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fbfc6a9a700 1 Processor -- start 2024-09-13T17:51:05.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fe3a2ffd700 1 --2- 172.21.15.40:0/1211108590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3a4150520 0x7fe3a41597d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe394009940 tx=0x7fe394009c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=131aa402a7a10578 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f6f427fc700 1 -- 172.21.15.40:0/1251654458 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c010070 con 0x7f6f440c71c0 2024-09-13T17:51:05.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f2166f68700 1 -- start start 2024-09-13T17:51:05.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd2a5cc2700 1 --2- 172.21.15.40:0/1201912460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd28c004e80 0x7fd28c005300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd2a54c1700 1 --2- 172.21.15.40:0/1201912460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd28c003a50 0x7fd28c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd2a5cc2700 1 -- 172.21.15.40:0/1201912460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2a006faf0 msgr2=0x7fd28c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:05.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd2a5cc2700 1 --2- 172.21.15.40:0/1201912460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2a006faf0 0x7fd28c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd2a5cc2700 1 -- 172.21.15.40:0/1201912460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd28c003a50 msgr2=0x7fd28c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:05.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd2a5cc2700 1 --2- 172.21.15.40:0/1201912460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd28c003a50 0x7fd28c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fd2a5cc2700 1 -- 172.21.15.40:0/1201912460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c0035d0 con 0x7fd28c004e80 2024-09-13T17:51:05.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f6f427fc700 1 -- 172.21.15.40:0/1251654458 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3c00dd80 con 0x7f6f440c71c0 2024-09-13T17:51:05.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f2166f68700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21600c57d0 0x7f216014ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f2166f68700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2160157e90 0x7f216014f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f2166f68700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f216015e6b0 0x7f21600c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f2166f68700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2160151f70 con 0x7f21600c57d0 2024-09-13T17:51:05.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f2166f68700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21600c9850 con 0x7f2160157e90 2024-09-13T17:51:05.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f2166f68700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21600c3720 con 0x7f216015e6b0 2024-09-13T17:51:05.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f2165765700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2160157e90 0x7f216014f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f2165765700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2160157e90 0x7f216014f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:36956/0 (socket says 172.21.15.40:36956) 2024-09-13T17:51:05.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f2165765700 1 -- 172.21.15.40:0/819394052 learned_addr learned my addr 172.21.15.40:0/819394052 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:05.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f2166767700 1 --2- 172.21.15.40:0/819394052 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f216015e6b0 0x7f21600c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fd2a5cc2700 1 --2- 172.21.15.40:0/1201912460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd28c004e80 0x7fd28c005300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd29c007c70 tx=0x7fd28c0058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f731e607fd4987e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fbfc6a9a700 1 -- start start 2024-09-13T17:51:05.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfc6a9a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfc006d920 0x7fbfc00c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfc6a9a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfc00c7f60 0x7fbfc00c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfc6a9a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbfc00c8340 0x7fbfc00cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfc6a9a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00cadf0 con 0x7fbfc00c7f60 2024-09-13T17:51:05.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfc6a9a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00c2360 con 0x7fbfc006d920 2024-09-13T17:51:05.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfc6a9a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00c2590 con 0x7fbfc00c8340 2024-09-13T17:51:05.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7ff6dbfff700 1 -- 172.21.15.40:0/3618904688 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff6bc000fc0 con 0x7ff6dc0724e0 2024-09-13T17:51:05.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fd297fff700 1 -- 172.21.15.40:0/1201912460 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c00cb00 con 0x7fd28c004e80 2024-09-13T17:51:05.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f6f427fc700 1 -- 172.21.15.40:0/1251654458 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c00b310 con 0x7f6f440c71c0 2024-09-13T17:51:05.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f9c509e1700 1 -- 172.21.15.40:0/650272344 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9c4c0c8490 con 0x7f9c4c14f8c0 2024-09-13T17:51:05.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f9c337fe700 1 --2- 172.21.15.40:0/650272344 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9c38066780 0x7f9c38068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f9c337fe700 1 -- 172.21.15.40:0/650272344 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9c40083eb0 con 0x7f9c4c14f8c0 2024-09-13T17:51:05.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f9c337fe700 1 --2- 172.21.15.40:0/650272344 >> [v2:172.21.15.40:6808/2576246869,v1:172.21.15.40:6809/2576246869] conn(0x7f9c3806fd40 0x7f9c38072180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f9c337fe700 1 -- 172.21.15.40:0/650272344 --> [v2:172.21.15.40:6808/2576246869,v1:172.21.15.40:6809/2576246869] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9c38072850 con 0x7f9c3806fd40 2024-09-13T17:51:05.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f9c337fe700 1 -- 172.21.15.40:0/650272344 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c400510a0 con 0x7f9c4c14f8c0 2024-09-13T17:51:05.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7f9c4b59e700 1 --2- 172.21.15.40:0/650272344 >> [v2:172.21.15.40:6808/2576246869,v1:172.21.15.40:6809/2576246869] conn(0x7f9c3806fd40 0x7f9c38072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f9c4ad9d700 1 --2- 172.21.15.40:0/650272344 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9c38066780 0x7f9c38068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.802+0000 7fe88dffb700 1 -- 172.21.15.40:0/1686426634 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe88801c460 con 0x7fe8980724e0 2024-09-13T17:51:05.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fe88dffb700 1 --2- 172.21.15.40:0/1686426634 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe87c066730 0x7fe87c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fe88dffb700 1 -- 172.21.15.40:0/1686426634 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe8880842a0 con 0x7fe8980724e0 2024-09-13T17:51:05.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fe88dffb700 1 --2- 172.21.15.40:0/1686426634 >> [v2:172.21.15.40:6816/3493711342,v1:172.21.15.40:6817/3493711342] conn(0x7fe87c06fcf0 0x7fe87c072130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fe88dffb700 1 -- 172.21.15.40:0/1686426634 --> [v2:172.21.15.40:6816/3493711342,v1:172.21.15.40:6817/3493711342] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe87c072800 con 0x7fe87c06fcf0 2024-09-13T17:51:05.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fe89da80700 1 --2- 172.21.15.40:0/1686426634 >> [v2:172.21.15.40:6816/3493711342,v1:172.21.15.40:6817/3493711342] conn(0x7fe87c06fcf0 0x7fe87c072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fe89ca7e700 1 --2- 172.21.15.40:0/1686426634 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe87c066730 0x7fe87c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f2165765700 1 -- 172.21.15.40:0/819394052 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f216015e6b0 msgr2=0x7f21600c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f2165765700 1 --2- 172.21.15.40:0/819394052 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f216015e6b0 0x7f21600c4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f2165765700 1 -- 172.21.15.40:0/819394052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21600c57d0 msgr2=0x7f216014ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f2165765700 1 --2- 172.21.15.40:0/819394052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21600c57d0 0x7f216014ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f2165765700 1 -- 172.21.15.40:0/819394052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215c0094b0 con 0x7f2160157e90 2024-09-13T17:51:05.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfc6299700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbfc00c8340 0x7fbfc00cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfc6299700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbfc00c8340 0x7fbfc00cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:48910/0 (socket says 172.21.15.40:48910) 2024-09-13T17:51:05.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfc6299700 1 -- 172.21.15.40:0/2701487249 learned_addr learned my addr 172.21.15.40:0/2701487249 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:05.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfc5297700 1 --2- 172.21.15.40:0/2701487249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfc00c7f60 0x7fbfc00c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fe88dffb700 1 -- 172.21.15.40:0/1686426634 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe888051280 con 0x7fe8980724e0 2024-09-13T17:51:05.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f9c4ad9d700 1 --2- 172.21.15.40:0/650272344 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9c38066780 0x7f9c38068c40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c000c00 tx=0x7f9c3c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fd297fff700 1 -- 172.21.15.40:0/1201912460 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29c00cc60 con 0x7fd28c004e80 2024-09-13T17:51:05.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f2165765700 1 --2- 172.21.15.40:0/819394052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2160157e90 0x7f216014f360 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f215c0090b0 tx=0x7f215c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f6f48fef700 1 -- 172.21.15.40:0/1251654458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f440c71c0 msgr2=0x7f6f44159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f6f48fef700 1 --2- 172.21.15.40:0/1251654458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f440c71c0 0x7f6f44159630 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00d150 tx=0x7f6f3c00d460 comp rx=0 tx=0).stop 2024-09-13T17:51:05.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f6f48fef700 1 -- 172.21.15.40:0/1251654458 shutdown_connections 2024-09-13T17:51:05.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f6f48fef700 1 --2- 172.21.15.40:0/1251654458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f440c71c0 0x7f6f44159630 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f6f48fef700 1 --2- 172.21.15.40:0/1251654458 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f440c8500 0x7f6f440c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f6f48fef700 1 -- 172.21.15.40:0/1251654458 >> 172.21.15.40:0/1251654458 conn(0x7f6f4406e6d0 msgr2=0x7f6f4406eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:05.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f9c4b59e700 1 --2- 172.21.15.40:0/650272344 >> [v2:172.21.15.40:6808/2576246869,v1:172.21.15.40:6809/2576246869] conn(0x7f9c3806fd40 0x7f9c38072180 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f6f48fef700 1 -- 172.21.15.40:0/1251654458 shutdown_connections 2024-09-13T17:51:05.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfc6299700 1 -- 172.21.15.40:0/2701487249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfc006d920 msgr2=0x7fbfc00c2ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T17:51:05.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfc6299700 1 --2- 172.21.15.40:0/2701487249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfc006d920 0x7fbfc00c2ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfc6299700 1 -- 172.21.15.40:0/2701487249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfc00c7f60 msgr2=0x7fbfc00c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfc6299700 1 --2- 172.21.15.40:0/2701487249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbfc00c7f60 0x7fbfc00c8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfc6299700 1 -- 172.21.15.40:0/2701487249 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc009570 con 0x7fbfc00c8340 2024-09-13T17:51:05.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7ff6c37fe700 1 -- 172.21.15.40:0/3618904688 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff6d4006be0 con 0x7ff6dc0724e0 2024-09-13T17:51:05.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fe89ca7e700 1 --2- 172.21.15.40:0/1686426634 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe87c066730 0x7fe87c068bf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe89000c7b0 tx=0x7fe89001b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f21567fc700 1 -- 172.21.15.40:0/819394052 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c003d70 con 0x7f2160157e90 2024-09-13T17:51:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f21567fc700 1 -- 172.21.15.40:0/819394052 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c01f070 con 0x7f2160157e90 2024-09-13T17:51:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f21567fc700 1 -- 172.21.15.40:0/819394052 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c011870 con 0x7f2160157e90 2024-09-13T17:51:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f2166f68700 1 -- 172.21.15.40:0/819394052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2160151730 con 0x7f2160157e90 2024-09-13T17:51:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f2166f68700 1 -- 172.21.15.40:0/819394052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2160151c90 con 0x7f2160157e90 2024-09-13T17:51:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fe89da80700 1 --2- 172.21.15.40:0/1686426634 >> [v2:172.21.15.40:6816/3493711342,v1:172.21.15.40:6817/3493711342] conn(0x7fe87c06fcf0 0x7fe87c072130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fd2a7725700 1 -- 172.21.15.40:0/1201912460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd28c004e80 msgr2=0x7fd28c005300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fd2a7725700 1 --2- 172.21.15.40:0/1201912460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd28c004e80 0x7fd28c005300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd29c007c70 tx=0x7fd28c0058a0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fd297fff700 1 -- 172.21.15.40:0/1201912460 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c005980 con 0x7fd28c004e80 2024-09-13T17:51:05.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fd2a5cc2700 1 -- 172.21.15.40:0/1201912460 reap_dead start 2024-09-13T17:51:05.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfc6299700 1 --2- 172.21.15.40:0/2701487249 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbfc00c8340 0x7fbfc00cb240 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800b970 tx=0x7fbfb800bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7ff6c37fe700 1 --2- 172.21.15.40:0/3618904688 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff6c8066730 0x7ff6c8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfb67fc700 1 -- 172.21.15.40:0/2701487249 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb800c930 con 0x7fbfc00c8340 2024-09-13T17:51:05.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7ff6da7fc700 1 --2- 172.21.15.40:0/3618904688 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff6c8066730 0x7ff6c8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7ff6c37fe700 1 -- 172.21.15.40:0/3618904688 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff6d4083d40 con 0x7ff6dc0724e0 2024-09-13T17:51:05.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfc6a9a700 1 -- 172.21.15.40:0/2701487249 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc00c2830 con 0x7fbfc00c8340 2024-09-13T17:51:05.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7ff6c37fe700 1 --2- 172.21.15.40:0/3618904688 >> [v2:172.21.15.40:6824/1926907414,v1:172.21.15.40:6825/1926907414] conn(0x7ff6c806fc70 0x7ff6c80720b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfc6a9a700 1 -- 172.21.15.40:0/2701487249 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc00c4dc0 con 0x7fbfc00c8340 2024-09-13T17:51:05.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7ff6c37fe700 1 -- 172.21.15.40:0/3618904688 --> [v2:172.21.15.40:6824/1926907414,v1:172.21.15.40:6825/1926907414] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff6c8072780 con 0x7ff6c806fc70 2024-09-13T17:51:05.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f9c337fe700 1 -- 172.21.15.40:0/650272344 <== osd.1 v2:172.21.15.40:6808/2576246869 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f9c38072850 con 0x7f9c3806fd40 2024-09-13T17:51:05.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7ff6c37fe700 1 -- 172.21.15.40:0/3618904688 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6d40086e0 con 0x7ff6dc0724e0 2024-09-13T17:51:05.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fe3a1ffb700 1 -- 172.21.15.40:0/1211108590 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39400ba20 con 0x7fe3a4150520 2024-09-13T17:51:05.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fe3a1ffb700 1 -- 172.21.15.40:0/1211108590 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe394009e90 con 0x7fe3a4150520 2024-09-13T17:51:05.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fe3a1ffb700 1 -- 172.21.15.40:0/1211108590 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39400bda0 con 0x7fe3a4150520 2024-09-13T17:51:05.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7ff6da7fc700 1 --2- 172.21.15.40:0/3618904688 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff6c8066730 0x7ff6c8068bf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc14a8c0 tx=0x7ff6d00091c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f6f48fef700 1 -- 172.21.15.40:0/1251654458 wait complete. 2024-09-13T17:51:05.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fe3a3fff700 1 -- 172.21.15.40:0/1211108590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3a4150520 msgr2=0x7fe3a41597d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fe3a3fff700 1 --2- 172.21.15.40:0/1211108590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3a4150520 0x7fe3a41597d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe394009940 tx=0x7fe394009c50 comp rx=0 tx=0).stop 2024-09-13T17:51:05.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfb67fc700 1 -- 172.21.15.40:0/2701487249 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb800cad0 con 0x7fbfc00c8340 2024-09-13T17:51:05.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfb67fc700 1 -- 172.21.15.40:0/2701487249 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb800ce80 con 0x7fbfc00c8340 2024-09-13T17:51:05.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f6f48fef700 1 Processor -- start 2024-09-13T17:51:05.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7ff6db7fe700 1 --2- 172.21.15.40:0/3618904688 >> [v2:172.21.15.40:6824/1926907414,v1:172.21.15.40:6825/1926907414] conn(0x7ff6c806fc70 0x7ff6c80720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fe88dffb700 1 -- 172.21.15.40:0/1686426634 <== osd.0 v2:172.21.15.40:6816/3493711342 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fe87c072800 con 0x7fe87c06fcf0 2024-09-13T17:51:05.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7fbfc6a9a700 1 -- 172.21.15.40:0/2701487249 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbfa8000fc0 con 0x7fbfc00c8340 2024-09-13T17:51:05.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7ff6db7fe700 1 --2- 172.21.15.40:0/3618904688 >> [v2:172.21.15.40:6824/1926907414,v1:172.21.15.40:6825/1926907414] conn(0x7ff6c806fc70 0x7ff6c80720b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7ff6c37fe700 1 -- 172.21.15.40:0/3618904688 <== osd.3 v2:172.21.15.40:6824/1926907414 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff6c8072780 con 0x7ff6c806fc70 2024-09-13T17:51:05.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f2166f68700 1 -- 172.21.15.40:0/819394052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f21600c62f0 con 0x7f2160157e90 2024-09-13T17:51:05.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f21567fc700 1 -- 172.21.15.40:0/819394052 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f215c0119d0 con 0x7f2160157e90 2024-09-13T17:51:05.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f21567fc700 1 --2- 172.21.15.40:0/819394052 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2144066760 0x7f2144068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f21567fc700 1 -- 172.21.15.40:0/819394052 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f215c021070 con 0x7f2160157e90 2024-09-13T17:51:05.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f2165f66700 1 --2- 172.21.15.40:0/819394052 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2144066760 0x7f2144068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f21567fc700 1 --2- 172.21.15.40:0/819394052 >> [v2:172.21.15.43:6800/858063952,v1:172.21.15.43:6801/858063952] conn(0x7f214406fd20 0x7f2144072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f21567fc700 1 -- 172.21.15.40:0/819394052 --> [v2:172.21.15.43:6800/858063952,v1:172.21.15.43:6801/858063952] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2144072830 con 0x7f214406fd20 2024-09-13T17:51:05.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f21567fc700 1 -- 172.21.15.40:0/819394052 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f215c087c10 con 0x7f2160157e90 2024-09-13T17:51:05.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7fbfb67fc700 1 -- 172.21.15.40:0/2701487249 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbfb8014440 con 0x7fbfc00c8340 2024-09-13T17:51:05.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.806+0000 7f6f48fef700 1 -- start start 2024-09-13T17:51:05.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f48fef700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f440c71c0 0x7f6f441e3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f48fef700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f440c8500 0x7f6f441e4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f48fef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f441503c0 0x7f6f441e4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f48fef700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f440c3ab0 con 0x7f6f441503c0 2024-09-13T17:51:05.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f48fef700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f440cb9e0 con 0x7f6f440c71c0 2024-09-13T17:51:05.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f48fef700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f440cb4f0 con 0x7f6f440c8500 2024-09-13T17:51:05.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f43fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f441503c0 0x7f6f441e4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f437fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f440c71c0 0x7f6f441e3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f43fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f441503c0 0x7f6f441e4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57956/0 (socket says 172.21.15.40:57956) 2024-09-13T17:51:05.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f437fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f440c71c0 0x7f6f441e3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:36980/0 (socket says 172.21.15.40:36980) 2024-09-13T17:51:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f43fff700 1 -- 172.21.15.40:0/2654103124 learned_addr learned my addr 172.21.15.40:0/2654103124 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f42ffd700 1 --2- 172.21.15.40:0/2654103124 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f440c8500 0x7f6f441e4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f43fff700 1 -- 172.21.15.40:0/2654103124 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f440c8500 msgr2=0x7f6f441e4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f43fff700 1 --2- 172.21.15.40:0/2654103124 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f440c8500 0x7f6f441e4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f43fff700 1 -- 172.21.15.40:0/2654103124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f440c71c0 msgr2=0x7f6f441e3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f43fff700 1 --2- 172.21.15.40:0/2654103124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f440c71c0 0x7f6f441e3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f43fff700 1 -- 172.21.15.40:0/2654103124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c00bd40 con 0x7f6f441503c0 2024-09-13T17:51:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7fbfb67fc700 1 --2- 172.21.15.40:0/2701487249 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbfa4066730 0x7fbfa4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f2165f66700 1 --2- 172.21.15.40:0/819394052 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2144066760 0x7f2144068c20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2150007940 tx=0x7f2150008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f437fe700 1 --2- 172.21.15.40:0/2654103124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f440c71c0 0x7f6f441e3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:05.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7fbfb67fc700 1 -- 172.21.15.40:0/2701487249 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbfb8084b20 con 0x7fbfc00c8340 2024-09-13T17:51:05.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7fbfc5a98700 1 --2- 172.21.15.40:0/2701487249 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbfa4066730 0x7fbfa4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7fbfb67fc700 1 --2- 172.21.15.40:0/2701487249 >> [v2:172.21.15.43:6808/480073165,v1:172.21.15.43:6809/480073165] conn(0x7fbfa406fcf0 0x7fbfa4072130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f43fff700 1 --2- 172.21.15.40:0/2654103124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f441503c0 0x7f6f441e4b90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6f44066a90 tx=0x7f6f3c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7fbfb67fc700 1 -- 172.21.15.40:0/2701487249 --> [v2:172.21.15.43:6808/480073165,v1:172.21.15.43:6809/480073165] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfa4072800 con 0x7fbfa406fcf0 2024-09-13T17:51:05.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7fbfb67fc700 1 -- 172.21.15.40:0/2701487249 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfb8084f70 con 0x7fbfc00c8340 2024-09-13T17:51:05.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f2bfff700 1 -- 172.21.15.40:0/2654103124 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c009800 con 0x7f6f441503c0 2024-09-13T17:51:05.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f2bfff700 1 -- 172.21.15.40:0/2654103124 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3c009960 con 0x7f6f441503c0 2024-09-13T17:51:05.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f48fef700 1 -- 172.21.15.40:0/2654103124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f441e41c0 con 0x7f6f441503c0 2024-09-13T17:51:05.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f2bfff700 1 -- 172.21.15.40:0/2654103124 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c009d10 con 0x7f6f441503c0 2024-09-13T17:51:05.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7fbfc5a98700 1 --2- 172.21.15.40:0/2701487249 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbfa4066730 0x7fbfa4068bf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009910 tx=0x7fbfbc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f48fef700 1 -- 172.21.15.40:0/2654103124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f440cb700 con 0x7f6f441503c0 2024-09-13T17:51:05.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.810+0000 7f6f48fef700 1 -- 172.21.15.40:0/2654103124 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6f440c8ce0 con 0x7f6f441503c0 2024-09-13T17:51:05.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7fbfc5297700 1 --2- 172.21.15.40:0/2701487249 >> [v2:172.21.15.43:6808/480073165,v1:172.21.15.43:6809/480073165] conn(0x7fbfa406fcf0 0x7fbfa4072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7fd2a7725700 1 -- 172.21.15.40:0/1201912460 shutdown_connections 2024-09-13T17:51:05.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7fd2a7725700 1 -- 172.21.15.40:0/1201912460 >> 172.21.15.40:0/1201912460 conn(0x7fd2a0066090 msgr2=0x7fd2a00664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:05.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7fe3a3fff700 1 -- 172.21.15.40:0/1211108590 shutdown_connections 2024-09-13T17:51:05.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7fe3a3fff700 1 --2- 172.21.15.40:0/1211108590 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3a4072410 0x7fe3a4160b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7fe3a3fff700 1 --2- 172.21.15.40:0/1211108590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3a4150520 0x7fe3a41597d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7fe3a3fff700 1 -- 172.21.15.40:0/1211108590 >> 172.21.15.40:0/1211108590 conn(0x7fe3a4066090 msgr2=0x7fe3a40664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:05.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7fe3a3fff700 1 -- 172.21.15.40:0/1211108590 shutdown_connections 2024-09-13T17:51:05.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7f2166767700 1 --2- 172.21.15.40:0/819394052 >> [v2:172.21.15.43:6800/858063952,v1:172.21.15.43:6801/858063952] conn(0x7f214406fd20 0x7f2144072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7fe3a3fff700 1 -- 172.21.15.40:0/1211108590 wait complete. 2024-09-13T17:51:05.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7f2166767700 1 --2- 172.21.15.40:0/819394052 >> [v2:172.21.15.43:6800/858063952,v1:172.21.15.43:6801/858063952] conn(0x7f214406fd20 0x7f2144072160 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7f6f2bfff700 1 -- 172.21.15.40:0/2654103124 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6f3c009050 con 0x7f6f441503c0 2024-09-13T17:51:05.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7f6f2bfff700 1 --2- 172.21.15.40:0/2654103124 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6f30066730 0x7f6f30068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7f6f2bfff700 1 -- 172.21.15.40:0/2654103124 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6f3c085600 con 0x7f6f441503c0 2024-09-13T17:51:05.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7f6f437fe700 1 --2- 172.21.15.40:0/2654103124 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6f30066730 0x7f6f30068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7f6f2bfff700 1 --2- 172.21.15.40:0/2654103124 >> [v2:172.21.15.43:6816/2080568799,v1:172.21.15.43:6817/2080568799] conn(0x7f6f3006fc70 0x7f6f300720b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7f6f2bfff700 1 -- 172.21.15.40:0/2654103124 --> [v2:172.21.15.43:6816/2080568799,v1:172.21.15.43:6817/2080568799] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f30072780 con 0x7f6f3006fc70 2024-09-13T17:51:05.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7f6f2bfff700 1 -- 172.21.15.40:0/2654103124 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f3c087cc0 con 0x7f6f441503c0 2024-09-13T17:51:05.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7f21567fc700 1 -- 172.21.15.40:0/819394052 <== osd.5 v2:172.21.15.43:6800/858063952 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f2144072830 con 0x7f214406fd20 2024-09-13T17:51:05.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7f6f42ffd700 1 --2- 172.21.15.40:0/2654103124 >> [v2:172.21.15.43:6816/2080568799,v1:172.21.15.43:6817/2080568799] conn(0x7f6f3006fc70 0x7f6f300720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7fe3a3fff700 1 Processor -- start 2024-09-13T17:51:05.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7fbfc5297700 1 --2- 172.21.15.40:0/2701487249 >> [v2:172.21.15.43:6808/480073165,v1:172.21.15.43:6809/480073165] conn(0x7fbfa406fcf0 0x7fbfa4072130 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7fbfb67fc700 1 -- 172.21.15.40:0/2701487249 <== osd.6 v2:172.21.15.43:6808/480073165 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fbfa4072800 con 0x7fbfa406fcf0 2024-09-13T17:51:05.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7fe3a3fff700 1 -- start start 2024-09-13T17:51:05.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7fe3a3fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3a4072410 0x7fe3a40caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7fe3a3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3a4150520 0x7fe3a41eeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7fe3a3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3a4159d10 0x7fe3a414dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7fe3a3fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a40cbaf0 con 0x7fe3a4159d10 2024-09-13T17:51:05.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7fe3a3fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a40a9e70 con 0x7fe3a4072410 2024-09-13T17:51:05.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7fe3a3fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a40cb570 con 0x7fe3a4150520 2024-09-13T17:51:05.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7f6f437fe700 1 --2- 172.21.15.40:0/2654103124 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6f30066730 0x7f6f30068bf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009d50 tx=0x7f6f380090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.814+0000 7f6f42ffd700 1 --2- 172.21.15.40:0/2654103124 >> [v2:172.21.15.43:6816/2080568799,v1:172.21.15.43:6817/2080568799] conn(0x7f6f3006fc70 0x7f6f300720b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe3a2ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3a4072410 0x7fe3a40caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe3a27fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3a4150520 0x7fe3a41eeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe3a37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3a4159d10 0x7fe3a414dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe3a2ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3a4072410 0x7fe3a40caee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:36994/0 (socket says 172.21.15.40:36994) 2024-09-13T17:51:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe3a2ffd700 1 -- 172.21.15.40:0/3445674181 learned_addr learned my addr 172.21.15.40:0/3445674181 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe3a2ffd700 1 -- 172.21.15.40:0/3445674181 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3a4150520 msgr2=0x7fe3a41eeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe3a2ffd700 1 --2- 172.21.15.40:0/3445674181 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3a4150520 0x7fe3a41eeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe3a2ffd700 1 -- 172.21.15.40:0/3445674181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3a4159d10 msgr2=0x7fe3a414dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe3a2ffd700 1 --2- 172.21.15.40:0/3445674181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3a4159d10 0x7fe3a414dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe3a2ffd700 1 -- 172.21.15.40:0/3445674181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe394009570 con 0x7fe3a4072410 2024-09-13T17:51:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe3a27fc700 1 --2- 172.21.15.40:0/3445674181 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3a4150520 0x7fe3a41eeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe3a2ffd700 1 --2- 172.21.15.40:0/3445674181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3a4072410 0x7fe3a40caee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe394009890 tx=0x7fe394004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe38b7fe700 1 -- 172.21.15.40:0/3445674181 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39401d040 con 0x7fe3a4072410 2024-09-13T17:51:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe38b7fe700 1 -- 172.21.15.40:0/3445674181 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe394003af0 con 0x7fe3a4072410 2024-09-13T17:51:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe38b7fe700 1 -- 172.21.15.40:0/3445674181 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe394003ea0 con 0x7fe3a4072410 2024-09-13T17:51:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe3a3fff700 1 -- 172.21.15.40:0/3445674181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a40c17a0 con 0x7fe3a4072410 2024-09-13T17:51:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe3a3fff700 1 -- 172.21.15.40:0/3445674181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a41ec1e0 con 0x7fe3a4072410 2024-09-13T17:51:05.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe3a3fff700 1 -- 172.21.15.40:0/3445674181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe3a40c6630 con 0x7fe3a4072410 2024-09-13T17:51:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe38b7fe700 1 -- 172.21.15.40:0/3445674181 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe394019400 con 0x7fe3a4072410 2024-09-13T17:51:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe38b7fe700 1 --2- 172.21.15.40:0/3445674181 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe38c06acf0 0x7fe38c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe3a27fc700 1 --2- 172.21.15.40:0/3445674181 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe38c06acf0 0x7fe38c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe38b7fe700 1 -- 172.21.15.40:0/3445674181 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe394085930 con 0x7fe3a4072410 2024-09-13T17:51:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe38b7fe700 1 --2- 172.21.15.40:0/3445674181 >> [v2:172.21.15.40:6800/44856195,v1:172.21.15.40:6801/44856195] conn(0x7fe38c0742b0 0x7fe38c0766f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe38b7fe700 1 -- 172.21.15.40:0/3445674181 --> [v2:172.21.15.40:6800/44856195,v1:172.21.15.40:6801/44856195] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe38c076dc0 con 0x7fe38c0742b0 2024-09-13T17:51:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7fe38b7fe700 1 -- 172.21.15.40:0/3445674181 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe394085d80 con 0x7fe3a4072410 2024-09-13T17:51:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.822+0000 7fe3a37fe700 1 --2- 172.21.15.40:0/3445674181 >> [v2:172.21.15.40:6800/44856195,v1:172.21.15.40:6801/44856195] conn(0x7fe38c0742b0 0x7fe38c0766f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.822+0000 7fe3a27fc700 1 --2- 172.21.15.40:0/3445674181 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe38c06acf0 0x7fe38c06d1b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe3a40c1040 tx=0x7fe398009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.822+0000 7fe3a37fe700 1 --2- 172.21.15.40:0/3445674181 >> [v2:172.21.15.40:6800/44856195,v1:172.21.15.40:6801/44856195] conn(0x7fe38c0742b0 0x7fe38c0766f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.822+0000 7fe38b7fe700 1 -- 172.21.15.40:0/3445674181 <== osd.2 v2:172.21.15.40:6800/44856195 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fe38c076dc0 con 0x7fe38c0742b0 2024-09-13T17:51:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.818+0000 7f6f2bfff700 1 -- 172.21.15.40:0/2654103124 <== osd.7 v2:172.21.15.43:6816/2080568799 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f6f30072780 con 0x7f6f3006fc70 2024-09-13T17:51:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.826+0000 7fd2a7725700 1 -- 172.21.15.40:0/1201912460 shutdown_connections 2024-09-13T17:51:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.826+0000 7f9c509e1700 1 -- 172.21.15.40:0/650272344 --> [v2:172.21.15.40:6808/2576246869,v1:172.21.15.40:6809/2576246869] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9c4c14fca0 con 0x7f9c3806fd40 2024-09-13T17:51:05.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.826+0000 7fd2a7725700 1 -- 172.21.15.40:0/1201912460 wait complete. 2024-09-13T17:51:05.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7f9c337fe700 1 -- 172.21.15.40:0/650272344 <== osd.1 v2:172.21.15.40:6808/2576246869 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9c44009bf0 con 0x7f9c3806fd40 2024-09-13T17:51:05.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.826+0000 7ff6dbfff700 1 -- 172.21.15.40:0/3618904688 --> [v2:172.21.15.40:6824/1926907414,v1:172.21.15.40:6825/1926907414] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff6bc002d60 con 0x7ff6c806fc70 2024-09-13T17:51:05.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7ff6c37fe700 1 -- 172.21.15.40:0/3618904688 <== osd.3 v2:172.21.15.40:6824/1926907414 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff6bc002d60 con 0x7ff6c806fc70 2024-09-13T17:51:05.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a7725700 1 Processor -- start 2024-09-13T17:51:05.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a7725700 1 -- start start 2024-09-13T17:51:05.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a7725700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd28c004e80 0x7fd2a0143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a7725700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2a0153f80 0x7fd2a0071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a7725700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2a0154580 0x7fd2a0154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a7725700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a01447b0 con 0x7fd2a0154580 2024-09-13T17:51:05.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a7725700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a01503e0 con 0x7fd28c004e80 2024-09-13T17:51:05.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a7725700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a01eb140 con 0x7fd2a0153f80 2024-09-13T17:51:05.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a5cc2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2a0154580 0x7fd2a0154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a5cc2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2a0154580 0x7fd2a0154a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57974/0 (socket says 172.21.15.40:57974) 2024-09-13T17:51:05.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a5cc2700 1 -- 172.21.15.40:0/166031289 learned_addr learned my addr 172.21.15.40:0/166031289 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:05.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a54c1700 1 --2- 172.21.15.40:0/166031289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd28c004e80 0x7fd2a0143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a4cc0700 1 --2- 172.21.15.40:0/166031289 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2a0153f80 0x7fd2a0071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a5cc2700 1 -- 172.21.15.40:0/166031289 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2a0153f80 msgr2=0x7fd2a0071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a5cc2700 1 --2- 172.21.15.40:0/166031289 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2a0153f80 0x7fd2a0071ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a5cc2700 1 -- 172.21.15.40:0/166031289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd28c004e80 msgr2=0x7fd2a0143c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a5cc2700 1 --2- 172.21.15.40:0/166031289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd28c004e80 0x7fd2a0143c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a5cc2700 1 -- 172.21.15.40:0/166031289 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a01ec110 con 0x7fd2a0154580 2024-09-13T17:51:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a5cc2700 1 --2- 172.21.15.40:0/166031289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2a0154580 0x7fd2a0154a00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00be50 tx=0x7fd2a0072950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd295ffb700 1 -- 172.21.15.40:0/166031289 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c007a90 con 0x7fd2a0154580 2024-09-13T17:51:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a7725700 1 -- 172.21.15.40:0/166031289 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a01444d0 con 0x7fd2a0154580 2024-09-13T17:51:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd2a7725700 1 -- 172.21.15.40:0/166031289 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a01eb630 con 0x7fd2a0154580 2024-09-13T17:51:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd295ffb700 1 -- 172.21.15.40:0/166031289 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29c004d30 con 0x7fd2a0154580 2024-09-13T17:51:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.830+0000 7fd295ffb700 1 -- 172.21.15.40:0/166031289 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c004ea0 con 0x7fd2a0154580 2024-09-13T17:51:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.834+0000 7fd295ffb700 1 -- 172.21.15.40:0/166031289 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd29c0072d0 con 0x7fd2a0154580 2024-09-13T17:51:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.834+0000 7fd295ffb700 1 --2- 172.21.15.40:0/166031289 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd284066760 0x7fd284068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.834+0000 7fd295ffb700 1 -- 172.21.15.40:0/166031289 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd29c0957b0 con 0x7fd2a0154580 2024-09-13T17:51:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.834+0000 7fd2a54c1700 1 --2- 172.21.15.40:0/166031289 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd284066760 0x7fd284068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.834+0000 7fd2a54c1700 1 --2- 172.21.15.40:0/166031289 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd284066760 0x7fd284068c20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd298012fd0 tx=0x7fd298014040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7fe89e281700 1 -- 172.21.15.40:0/1686426634 --> [v2:172.21.15.40:6816/3493711342,v1:172.21.15.40:6817/3493711342] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe8980c5c50 con 0x7fe87c06fcf0 2024-09-13T17:51:05.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7f9c509e1700 1 -- 172.21.15.40:0/650272344 >> [v2:172.21.15.40:6808/2576246869,v1:172.21.15.40:6809/2576246869] conn(0x7f9c3806fd40 msgr2=0x7f9c38072180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7f9c509e1700 1 --2- 172.21.15.40:0/650272344 >> [v2:172.21.15.40:6808/2576246869,v1:172.21.15.40:6809/2576246869] conn(0x7f9c3806fd40 0x7f9c38072180 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7fd2a7725700 1 --2- 172.21.15.40:0/166031289 >> [v2:172.21.15.43:6824/3105831381,v1:172.21.15.43:6825/3105831381] conn(0x7fd2800015e0 0x7fd280003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:05.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7fd2a7725700 1 -- 172.21.15.40:0/166031289 --> [v2:172.21.15.43:6824/3105831381,v1:172.21.15.43:6825/3105831381] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd280006bf0 con 0x7fd2800015e0 2024-09-13T17:51:05.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7f9c509e1700 1 -- 172.21.15.40:0/650272344 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9c38066780 msgr2=0x7f9c38068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7f9c509e1700 1 --2- 172.21.15.40:0/650272344 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9c38066780 0x7f9c38068c40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c000c00 tx=0x7f9c3c008030 comp rx=0 tx=0).stop 2024-09-13T17:51:05.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7f9c509e1700 1 -- 172.21.15.40:0/650272344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c4c14f8c0 msgr2=0x7f9c4c162660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7f9c509e1700 1 --2- 172.21.15.40:0/650272344 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c4c14f8c0 0x7f9c4c162660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000e760 tx=0x7f9c4000ea70 comp rx=0 tx=0).stop 2024-09-13T17:51:05.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7ff6c17fa700 1 -- 172.21.15.40:0/3618904688 >> [v2:172.21.15.40:6824/1926907414,v1:172.21.15.40:6825/1926907414] conn(0x7ff6c806fc70 msgr2=0x7ff6c80720b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7ff6c17fa700 1 --2- 172.21.15.40:0/3618904688 >> [v2:172.21.15.40:6824/1926907414,v1:172.21.15.40:6825/1926907414] conn(0x7ff6c806fc70 0x7ff6c80720b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7f9c4b59e700 1 -- 172.21.15.40:0/650272344 reap_dead start 2024-09-13T17:51:05.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7f9c509e1700 1 -- 172.21.15.40:0/650272344 shutdown_connections 2024-09-13T17:51:05.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7f9c509e1700 1 -- 172.21.15.40:0/650272344 >> 172.21.15.40:0/650272344 conn(0x7f9c4c066090 msgr2=0x7f9c4c148c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:05.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7ff6c17fa700 1 -- 172.21.15.40:0/3618904688 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff6c8066730 msgr2=0x7ff6c8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7ff6c17fa700 1 --2- 172.21.15.40:0/3618904688 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff6c8066730 0x7ff6c8068bf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc14a8c0 tx=0x7ff6d00091c0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7ff6c17fa700 1 -- 172.21.15.40:0/3618904688 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6dc0724e0 msgr2=0x7ff6dc149730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7ff6c17fa700 1 --2- 172.21.15.40:0/3618904688 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6dc0724e0 0x7ff6dc149730 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4001be0 tx=0x7ff6d4001ef0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7ff6db7fe700 1 -- 172.21.15.40:0/3618904688 reap_dead start 2024-09-13T17:51:05.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7fe88dffb700 1 -- 172.21.15.40:0/1686426634 <== osd.0 v2:172.21.15.40:6816/3493711342 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe894009880 con 0x7fe87c06fcf0 2024-09-13T17:51:05.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7f9c509e1700 1 -- 172.21.15.40:0/650272344 shutdown_connections 2024-09-13T17:51:05.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7f9c509e1700 1 -- 172.21.15.40:0/650272344 wait complete. 2024-09-13T17:51:05.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7fd2a4cc0700 1 --2- 172.21.15.40:0/166031289 >> [v2:172.21.15.43:6824/3105831381,v1:172.21.15.43:6825/3105831381] conn(0x7fd2800015e0 0x7fd280003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:05.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7fd2a4cc0700 1 --2- 172.21.15.40:0/166031289 >> [v2:172.21.15.43:6824/3105831381,v1:172.21.15.43:6825/3105831381] conn(0x7fd2800015e0 0x7fd280003aa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:05.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.838+0000 7fd295ffb700 1 -- 172.21.15.40:0/166031289 <== osd.4 v2:172.21.15.43:6824/3105831381 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fd280006bf0 con 0x7fd2800015e0 2024-09-13T17:51:05.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.842+0000 7ff6c17fa700 1 -- 172.21.15.40:0/3618904688 shutdown_connections 2024-09-13T17:51:05.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.842+0000 7ff6c17fa700 1 -- 172.21.15.40:0/3618904688 >> 172.21.15.40:0/3618904688 conn(0x7ff6dc066090 msgr2=0x7ff6dc071340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:05.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.842+0000 7f2166f68700 1 -- 172.21.15.40:0/819394052 --> [v2:172.21.15.43:6800/858063952,v1:172.21.15.43:6801/858063952] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f21600c6010 con 0x7f214406fd20 2024-09-13T17:51:05.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.842+0000 7f21567fc700 1 -- 172.21.15.40:0/819394052 <== osd.5 v2:172.21.15.43:6800/858063952 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2158001f80 con 0x7f214406fd20 2024-09-13T17:51:05.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.842+0000 7f2166f68700 1 -- 172.21.15.40:0/819394052 >> [v2:172.21.15.43:6800/858063952,v1:172.21.15.43:6801/858063952] conn(0x7f214406fd20 msgr2=0x7f2144072160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.842+0000 7f2166f68700 1 --2- 172.21.15.40:0/819394052 >> [v2:172.21.15.43:6800/858063952,v1:172.21.15.43:6801/858063952] conn(0x7f214406fd20 0x7f2144072160 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.842+0000 7f2166f68700 1 -- 172.21.15.40:0/819394052 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2144066760 msgr2=0x7f2144068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.842+0000 7f2166f68700 1 --2- 172.21.15.40:0/819394052 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2144066760 0x7f2144068c20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2150007940 tx=0x7f2150008040 comp rx=0 tx=0).stop 2024-09-13T17:51:05.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.842+0000 7f2166f68700 1 -- 172.21.15.40:0/819394052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2160157e90 msgr2=0x7f216014f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.842+0000 7f2166f68700 1 --2- 172.21.15.40:0/819394052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2160157e90 0x7f216014f360 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f215c0090b0 tx=0x7f215c004770 comp rx=0 tx=0).stop 2024-09-13T17:51:05.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.842+0000 7f2166767700 1 -- 172.21.15.40:0/819394052 reap_dead start 2024-09-13T17:51:05.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.842+0000 7f2166f68700 1 -- 172.21.15.40:0/819394052 shutdown_connections 2024-09-13T17:51:05.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.842+0000 7f2166f68700 1 -- 172.21.15.40:0/819394052 >> 172.21.15.40:0/819394052 conn(0x7f2160066090 msgr2=0x7f216006d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:05.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.842+0000 7f2166f68700 1 -- 172.21.15.40:0/819394052 shutdown_connections 2024-09-13T17:51:05.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.842+0000 7f2166f68700 1 -- 172.21.15.40:0/819394052 wait complete. 2024-09-13T17:51:05.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.846+0000 7fe89e281700 1 -- 172.21.15.40:0/1686426634 >> [v2:172.21.15.40:6816/3493711342,v1:172.21.15.40:6817/3493711342] conn(0x7fe87c06fcf0 msgr2=0x7fe87c072130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.846+0000 7fe89e281700 1 --2- 172.21.15.40:0/1686426634 >> [v2:172.21.15.40:6816/3493711342,v1:172.21.15.40:6817/3493711342] conn(0x7fe87c06fcf0 0x7fe87c072130 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.846+0000 7fe89e281700 1 -- 172.21.15.40:0/1686426634 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe87c066730 msgr2=0x7fe87c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.846+0000 7fe89e281700 1 --2- 172.21.15.40:0/1686426634 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe87c066730 0x7fe87c068bf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe89000c7b0 tx=0x7fe89001b040 comp rx=0 tx=0).stop 2024-09-13T17:51:05.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.846+0000 7fe89e281700 1 -- 172.21.15.40:0/1686426634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8980724e0 msgr2=0x7fe8980c0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.846+0000 7fe89e281700 1 --2- 172.21.15.40:0/1686426634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8980724e0 0x7fe8980c0860 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe88800c4e0 tx=0x7fe88800c7f0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.846+0000 7fe89da80700 1 -- 172.21.15.40:0/1686426634 reap_dead start 2024-09-13T17:51:05.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.846+0000 7fe89e281700 1 -- 172.21.15.40:0/1686426634 shutdown_connections 2024-09-13T17:51:05.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.846+0000 7fe89e281700 1 -- 172.21.15.40:0/1686426634 >> 172.21.15.40:0/1686426634 conn(0x7fe898066090 msgr2=0x7fe8980710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:05.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.846+0000 7fe89e281700 1 -- 172.21.15.40:0/1686426634 shutdown_connections 2024-09-13T17:51:05.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.846+0000 7fe89e281700 1 -- 172.21.15.40:0/1686426634 wait complete. 2024-09-13T17:51:05.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.846+0000 7ff6c17fa700 1 -- 172.21.15.40:0/3618904688 shutdown_connections 2024-09-13T17:51:05.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.846+0000 7ff6c17fa700 1 -- 172.21.15.40:0/3618904688 wait complete. 2024-09-13T17:51:05.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.854+0000 7f6f48fef700 1 -- 172.21.15.40:0/2654103124 --> [v2:172.21.15.43:6816/2080568799,v1:172.21.15.43:6817/2080568799] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6f440c8a40 con 0x7f6f3006fc70 2024-09-13T17:51:05.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.854+0000 7fbfc6a9a700 1 -- 172.21.15.40:0/2701487249 --> [v2:172.21.15.43:6808/480073165,v1:172.21.15.43:6809/480073165] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbfa8002d60 con 0x7fbfa406fcf0 2024-09-13T17:51:05.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.854+0000 7fbfb67fc700 1 -- 172.21.15.40:0/2701487249 <== osd.6 v2:172.21.15.43:6808/480073165 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbfa8002d60 con 0x7fbfa406fcf0 2024-09-13T17:51:05.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.854+0000 7f6f2bfff700 1 -- 172.21.15.40:0/2654103124 <== osd.7 v2:172.21.15.43:6816/2080568799 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6f34006de0 con 0x7f6f3006fc70 2024-09-13T17:51:05.888 INFO:teuthology.orchestra.run.smithi040.stdout:64424509451 2024-09-13T17:51:05.888 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T17:51:05.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.858+0000 7fe3a3fff700 1 -- 172.21.15.40:0/3445674181 --> [v2:172.21.15.40:6800/44856195,v1:172.21.15.40:6801/44856195] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe384000f80 con 0x7fe38c0742b0 2024-09-13T17:51:05.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.858+0000 7fbfc6a9a700 1 -- 172.21.15.40:0/2701487249 >> [v2:172.21.15.43:6808/480073165,v1:172.21.15.43:6809/480073165] conn(0x7fbfa406fcf0 msgr2=0x7fbfa4072130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.858+0000 7fbfc6a9a700 1 --2- 172.21.15.40:0/2701487249 >> [v2:172.21.15.43:6808/480073165,v1:172.21.15.43:6809/480073165] conn(0x7fbfa406fcf0 0x7fbfa4072130 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.858+0000 7fbfc6a9a700 1 -- 172.21.15.40:0/2701487249 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbfa4066730 msgr2=0x7fbfa4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.858+0000 7fbfc6a9a700 1 --2- 172.21.15.40:0/2701487249 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbfa4066730 0x7fbfa4068bf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009910 tx=0x7fbfbc008030 comp rx=0 tx=0).stop 2024-09-13T17:51:05.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.858+0000 7fbfc6a9a700 1 -- 172.21.15.40:0/2701487249 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbfc00c8340 msgr2=0x7fbfc00cb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.858+0000 7fbfc6a9a700 1 --2- 172.21.15.40:0/2701487249 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbfc00c8340 0x7fbfc00cb240 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800b970 tx=0x7fbfb800bc80 comp rx=0 tx=0).stop 2024-09-13T17:51:05.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.858+0000 7fbfc6299700 1 -- 172.21.15.40:0/2701487249 reap_dead start 2024-09-13T17:51:05.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.858+0000 7fbfc6a9a700 1 -- 172.21.15.40:0/2701487249 shutdown_connections 2024-09-13T17:51:05.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.858+0000 7fbfc6a9a700 1 -- 172.21.15.40:0/2701487249 >> 172.21.15.40:0/2701487249 conn(0x7fbfc0066090 msgr2=0x7fbfc015cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:05.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.858+0000 7fe38b7fe700 1 -- 172.21.15.40:0/3445674181 <== osd.2 v2:172.21.15.40:6800/44856195 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe384000f80 con 0x7fe38c0742b0 2024-09-13T17:51:05.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.858+0000 7fbfc6a9a700 1 -- 172.21.15.40:0/2701487249 shutdown_connections 2024-09-13T17:51:05.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.858+0000 7fbfc6a9a700 1 -- 172.21.15.40:0/2701487249 wait complete. 2024-09-13T17:51:05.891 INFO:teuthology.orchestra.run.smithi040.stdout:60129542154 2024-09-13T17:51:05.892 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T17:51:05.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.858+0000 7f6f48fef700 1 -- 172.21.15.40:0/2654103124 >> [v2:172.21.15.43:6816/2080568799,v1:172.21.15.43:6817/2080568799] conn(0x7f6f3006fc70 msgr2=0x7f6f300720b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.858+0000 7f6f48fef700 1 --2- 172.21.15.40:0/2654103124 >> [v2:172.21.15.43:6816/2080568799,v1:172.21.15.43:6817/2080568799] conn(0x7f6f3006fc70 0x7f6f300720b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.858+0000 7f6f48fef700 1 -- 172.21.15.40:0/2654103124 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6f30066730 msgr2=0x7f6f30068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.858+0000 7f6f48fef700 1 --2- 172.21.15.40:0/2654103124 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6f30066730 0x7f6f30068bf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009d50 tx=0x7f6f380090e0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.858+0000 7f6f48fef700 1 -- 172.21.15.40:0/2654103124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f441503c0 msgr2=0x7f6f441e4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.858+0000 7f6f48fef700 1 --2- 172.21.15.40:0/2654103124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f441503c0 0x7f6f441e4b90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6f44066a90 tx=0x7f6f3c004af0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.858+0000 7f6f43fff700 1 -- 172.21.15.40:0/2654103124 reap_dead start 2024-09-13T17:51:05.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.862+0000 7f6f48fef700 1 -- 172.21.15.40:0/2654103124 shutdown_connections 2024-09-13T17:51:05.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.862+0000 7f6f48fef700 1 -- 172.21.15.40:0/2654103124 >> 172.21.15.40:0/2654103124 conn(0x7f6f4406e6d0 msgr2=0x7f6f4406eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:05.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.862+0000 7f6f48fef700 1 -- 172.21.15.40:0/2654103124 shutdown_connections 2024-09-13T17:51:05.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.862+0000 7f6f48fef700 1 -- 172.21.15.40:0/2654103124 wait complete. 2024-09-13T17:51:05.894 INFO:teuthology.orchestra.run.smithi040.stdout:64424509452 2024-09-13T17:51:05.894 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T17:51:05.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.862+0000 7fe3a3fff700 1 -- 172.21.15.40:0/3445674181 >> [v2:172.21.15.40:6800/44856195,v1:172.21.15.40:6801/44856195] conn(0x7fe38c0742b0 msgr2=0x7fe38c0766f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.862+0000 7fe3a3fff700 1 --2- 172.21.15.40:0/3445674181 >> [v2:172.21.15.40:6800/44856195,v1:172.21.15.40:6801/44856195] conn(0x7fe38c0742b0 0x7fe38c0766f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.862+0000 7fe3a3fff700 1 -- 172.21.15.40:0/3445674181 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe38c06acf0 msgr2=0x7fe38c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.862+0000 7fe3a3fff700 1 --2- 172.21.15.40:0/3445674181 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe38c06acf0 0x7fe38c06d1b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe3a40c1040 tx=0x7fe398009040 comp rx=0 tx=0).stop 2024-09-13T17:51:05.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.862+0000 7fe3a3fff700 1 -- 172.21.15.40:0/3445674181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3a4072410 msgr2=0x7fe3a40caee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.862+0000 7fe3a3fff700 1 --2- 172.21.15.40:0/3445674181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3a4072410 0x7fe3a40caee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe394009890 tx=0x7fe394004770 comp rx=0 tx=0).stop 2024-09-13T17:51:05.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.862+0000 7fe3a37fe700 1 -- 172.21.15.40:0/3445674181 reap_dead start 2024-09-13T17:51:05.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.862+0000 7fe3a3fff700 1 -- 172.21.15.40:0/3445674181 shutdown_connections 2024-09-13T17:51:05.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.862+0000 7fe3a3fff700 1 -- 172.21.15.40:0/3445674181 >> 172.21.15.40:0/3445674181 conn(0x7fe3a4066090 msgr2=0x7fe3a4149de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:05.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.862+0000 7fe3a3fff700 1 -- 172.21.15.40:0/3445674181 shutdown_connections 2024-09-13T17:51:05.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.862+0000 7fe3a3fff700 1 -- 172.21.15.40:0/3445674181 wait complete. 2024-09-13T17:51:05.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.862+0000 7fd2a7725700 1 -- 172.21.15.40:0/166031289 --> [v2:172.21.15.43:6824/3105831381,v1:172.21.15.43:6825/3105831381] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd280005cd0 con 0x7fd2800015e0 2024-09-13T17:51:05.897 INFO:teuthology.orchestra.run.smithi040.stdout:64424509451 2024-09-13T17:51:05.897 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T17:51:05.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.862+0000 7fd295ffb700 1 -- 172.21.15.40:0/166031289 <== osd.4 v2:172.21.15.43:6824/3105831381 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd280005cd0 con 0x7fd2800015e0 2024-09-13T17:51:05.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.862+0000 7fd2a7725700 1 -- 172.21.15.40:0/166031289 >> [v2:172.21.15.43:6824/3105831381,v1:172.21.15.43:6825/3105831381] conn(0x7fd2800015e0 msgr2=0x7fd280003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.862+0000 7fd2a7725700 1 --2- 172.21.15.40:0/166031289 >> [v2:172.21.15.43:6824/3105831381,v1:172.21.15.43:6825/3105831381] conn(0x7fd2800015e0 0x7fd280003aa0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:05.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.866+0000 7fd2a7725700 1 -- 172.21.15.40:0/166031289 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd284066760 msgr2=0x7fd284068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.866+0000 7fd2a7725700 1 --2- 172.21.15.40:0/166031289 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd284066760 0x7fd284068c20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd298012fd0 tx=0x7fd298014040 comp rx=0 tx=0).stop 2024-09-13T17:51:05.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.866+0000 7fd2a7725700 1 -- 172.21.15.40:0/166031289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2a0154580 msgr2=0x7fd2a0154a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:05.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.866+0000 7fd2a7725700 1 --2- 172.21.15.40:0/166031289 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd2a0154580 0x7fd2a0154a00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00be50 tx=0x7fd2a0072950 comp rx=0 tx=0).stop 2024-09-13T17:51:05.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.866+0000 7fd2a5cc2700 1 -- 172.21.15.40:0/166031289 reap_dead start 2024-09-13T17:51:05.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.866+0000 7fd2a7725700 1 -- 172.21.15.40:0/166031289 shutdown_connections 2024-09-13T17:51:05.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.866+0000 7fd2a7725700 1 -- 172.21.15.40:0/166031289 >> 172.21.15.40:0/166031289 conn(0x7fd2a0066090 msgr2=0x7fd2a00c0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:05.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.866+0000 7fd2a7725700 1 -- 172.21.15.40:0/166031289 shutdown_connections 2024-09-13T17:51:05.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:05.866+0000 7fd2a7725700 1 -- 172.21.15.40:0/166031289 wait complete. 2024-09-13T17:51:05.900 INFO:teuthology.orchestra.run.smithi040.stdout:60129542154 2024-09-13T17:51:05.900 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T17:51:05.901 INFO:teuthology.orchestra.run.smithi040.stdout:60129542154 2024-09-13T17:51:05.901 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T17:51:05.902 INFO:teuthology.orchestra.run.smithi040.stdout:64424509451 2024-09-13T17:51:05.902 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T17:51:05.903 INFO:teuthology.orchestra.run.smithi040.stdout:60129542154 2024-09-13T17:51:05.904 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T17:51:06.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.090+0000 7f0bb13fd700 1 Processor -- start 2024-09-13T17:51:06.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.090+0000 7f86c759e700 1 Processor -- start 2024-09-13T17:51:06.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.090+0000 7f0bb13fd700 1 -- start start 2024-09-13T17:51:06.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.090+0000 7f86c759e700 1 -- start start 2024-09-13T17:51:06.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.090+0000 7f86c759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86c80724e0 0x7f86c80c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.090+0000 7f86c759e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86c8158bc0 0x7f86c815ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.090+0000 7f86c759e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c80c6890 con 0x7f86c814f8c0 2024-09-13T17:51:06.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.090+0000 7f86c759e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c80c7700 con 0x7f86c8158bc0 2024-09-13T17:51:06.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.090+0000 7f86c759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c80c5f30 con 0x7f86c80724e0 2024-09-13T17:51:06.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.090+0000 7f86c6d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86c8158bc0 0x7f86c815ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.090+0000 7f86c6d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86c8158bc0 0x7f86c815ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57982/0 (socket says 172.21.15.40:57982) 2024-09-13T17:51:06.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.090+0000 7f86c6d9d700 1 -- 172.21.15.40:0/447366837 learned_addr learned my addr 172.21.15.40:0/447366837 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:06.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.090+0000 7f86c5d9b700 1 --2- 172.21.15.40:0/447366837 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86c80724e0 0x7f86c80c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.090+0000 7f86c6d9d700 1 -- 172.21.15.40:0/447366837 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86c80724e0 msgr2=0x7f86c80c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:06.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.090+0000 7f86c6d9d700 1 --2- 172.21.15.40:0/447366837 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86c80724e0 0x7f86c80c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.090+0000 7f86c6d9d700 1 -- 172.21.15.40:0/447366837 >> v1:172.21.15.43:6789/0 conn(0x7f86c814f8c0 legacy=0x7f86c806d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.090+0000 7f86c6d9d700 1 -- 172.21.15.40:0/447366837 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c80c1320 con 0x7f86c8158bc0 2024-09-13T17:51:06.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.090+0000 7f0bb13fd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0bac158bc0 0x7f0bac0c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0bb13fd700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bac0c6890 con 0x7f0bac14f8c0 2024-09-13T17:51:06.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0bb13fd700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bac0c5f30 con 0x7f0bac0724e0 2024-09-13T17:51:06.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0bb13fd700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bac0c7700 con 0x7f0bac158bc0 2024-09-13T17:51:06.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0babfff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0bac14f8c0 0x7f0bac06d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46110/0 (socket says 172.21.15.40:46110) 2024-09-13T17:51:06.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0babfff700 1 -- 172.21.15.40:0/686056885 learned_addr learned my addr 172.21.15.40:0/686056885 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:06.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f86c6d9d700 1 --2- 172.21.15.40:0/447366837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c8158bc0 0x7f86c815ff50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009e50 tx=0x7f86c000bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62a112baef6495c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0bb0bfc700 1 --2- 172.21.15.40:0/686056885 >> v2:172.21.15.40:3300/0 conn(0x7f0bac158bc0 0x7f0bac0c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f86c559a700 1 -- 172.21.15.40:0/447366837 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c00115f0 con 0x7f86c8158bc0 2024-09-13T17:51:06.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f86c559a700 1 -- 172.21.15.40:0/447366837 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c0011750 con 0x7f86c8158bc0 2024-09-13T17:51:06.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f86c559a700 1 -- 172.21.15.40:0/447366837 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0011b00 con 0x7f86c8158bc0 2024-09-13T17:51:06.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0bb0bfc700 1 -- 172.21.15.40:0/686056885 >> v1:172.21.15.43:6789/0 conn(0x7f0bac0724e0 legacy=0x7f0bac15df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0bb0bfc700 1 -- 172.21.15.40:0/686056885 >> v1:172.21.15.40:6789/0 conn(0x7f0bac14f8c0 legacy=0x7f0bac06d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0baaffd700 1 -- 172.21.15.40:0/686056885 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3297326639 0 0) 0x7f0bac0c6890 con 0x7f0bac14f8c0 2024-09-13T17:51:06.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0bb0bfc700 1 -- 172.21.15.40:0/686056885 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bac0c1320 con 0x7f0bac158bc0 2024-09-13T17:51:06.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0bb0bfc700 1 --2- 172.21.15.40:0/686056885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bac158bc0 0x7f0bac0c71c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009d90 tx=0x7f0ba400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4160a4df5a4f0f1e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0baaffd700 1 -- 172.21.15.40:0/686056885 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba400e070 con 0x7f0bac158bc0 2024-09-13T17:51:06.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0baaffd700 1 -- 172.21.15.40:0/686056885 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba400bbd0 con 0x7f0bac158bc0 2024-09-13T17:51:06.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0baaffd700 1 -- 172.21.15.40:0/686056885 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4012430 con 0x7f0bac158bc0 2024-09-13T17:51:06.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f86c759e700 1 -- 172.21.15.40:0/447366837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c8158bc0 msgr2=0x7f86c815ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f86c759e700 1 --2- 172.21.15.40:0/447366837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c8158bc0 0x7f86c815ff50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009e50 tx=0x7f86c000bbc0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f86c759e700 1 -- 172.21.15.40:0/447366837 shutdown_connections 2024-09-13T17:51:06.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f86c759e700 1 --2- 172.21.15.40:0/447366837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c8158bc0 0x7f86c815ff50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f86c759e700 1 --2- 172.21.15.40:0/447366837 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86c80724e0 0x7f86c80c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f86c759e700 1 -- 172.21.15.40:0/447366837 >> 172.21.15.40:0/447366837 conn(0x7f86c8066090 msgr2=0x7f86c80664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:06.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f86c759e700 1 -- 172.21.15.40:0/447366837 shutdown_connections 2024-09-13T17:51:06.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f86c759e700 1 -- 172.21.15.40:0/447366837 wait complete. 2024-09-13T17:51:06.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f49bee2d700 1 Processor -- start 2024-09-13T17:51:06.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f49bee2d700 1 -- start start 2024-09-13T17:51:06.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0bb13fd700 1 -- 172.21.15.40:0/686056885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bac158bc0 msgr2=0x7f0bac0c71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0bb13fd700 1 --2- 172.21.15.40:0/686056885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bac158bc0 0x7f0bac0c71c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009d90 tx=0x7f0ba400b330 comp rx=0 tx=0).stop 2024-09-13T17:51:06.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0bb13fd700 1 -- 172.21.15.40:0/686056885 shutdown_connections 2024-09-13T17:51:06.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f9ae311c700 1 Processor -- start 2024-09-13T17:51:06.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0bb13fd700 1 --2- 172.21.15.40:0/686056885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bac158bc0 0x7f0bac0c71c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0da9d7e700 1 Processor -- start 2024-09-13T17:51:06.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0bb13fd700 1 -- 172.21.15.40:0/686056885 >> 172.21.15.40:0/686056885 conn(0x7f0bac066090 msgr2=0x7f0bac0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:06.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0da9d7e700 1 -- start start 2024-09-13T17:51:06.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f9ae311c700 1 -- start start 2024-09-13T17:51:06.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f49bee2d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49b80cf1a0 0x7f49b815a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f86c759e700 1 Processor -- start 2024-09-13T17:51:06.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f9ae311c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9adc06e090 0x7f9adc0c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f9ae311c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9adc0b00b0 0x7f9adc0c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f9ae311c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9adc159970 con 0x7f9adc06dcb0 2024-09-13T17:51:06.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f9ae311c700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9adc06d2d0 con 0x7f9adc06e090 2024-09-13T17:51:06.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f9ae311c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc159620 con 0x7f9adc0b00b0 2024-09-13T17:51:06.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f49bee2d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f49b80cf580 0x7f49b80c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f49bee2d700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49b815a7a0 con 0x7f49b80cc250 2024-09-13T17:51:06.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f49bee2d700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b80cbdc0 con 0x7f49b80cf580 2024-09-13T17:51:06.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f49bee2d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b8153f80 con 0x7f49b80cf1a0 2024-09-13T17:51:06.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0bb13fd700 1 -- 172.21.15.40:0/686056885 shutdown_connections 2024-09-13T17:51:06.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f9ae211a700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9adc06dcb0 0x7f9adc159e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46130/0 (socket says 172.21.15.40:46130) 2024-09-13T17:51:06.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f9ae211a700 1 -- 172.21.15.40:0/363507392 learned_addr learned my addr 172.21.15.40:0/363507392 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:06.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f9ae291b700 1 --2- 172.21.15.40:0/363507392 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9adc0b00b0 0x7f9adc0c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f0bb13fd700 1 -- 172.21.15.40:0/686056885 wait complete. 2024-09-13T17:51:06.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f49b7fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f49b80cf580 0x7f49b80c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f9ae1919700 1 --2- 172.21.15.40:0/363507392 >> v2:172.21.15.43:3300/0 conn(0x7f9adc06e090 0x7f9adc0c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f49bd3ca700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f49b80cc250 0x7f49b80c8aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46120/0 (socket says 172.21.15.40:46120) 2024-09-13T17:51:06.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f49bd3ca700 1 -- 172.21.15.40:0/2793606027 learned_addr learned my addr 172.21.15.40:0/2793606027 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:06.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0da9d7e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0da4071110 0x7f0da40714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f9ae1118700 1 -- 172.21.15.40:0/363507392 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318005123 0 0) 0x7f9adc159970 con 0x7f9adc06dcb0 2024-09-13T17:51:06.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f9ae291b700 1 -- 172.21.15.40:0/363507392 >> v2:172.21.15.43:3300/0 conn(0x7f9adc06e090 msgr2=0x7f9adc0c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:06.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0da9d7e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0da4156c10 0x7f0da4160b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f9ae291b700 1 --2- 172.21.15.40:0/363507392 >> v2:172.21.15.43:3300/0 conn(0x7f9adc06e090 0x7f9adc0c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f9ae291b700 1 -- 172.21.15.40:0/363507392 >> v1:172.21.15.40:6789/0 conn(0x7f9adc06dcb0 legacy=0x7f9adc159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0da3fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0da40c7010 0x7f0da4156500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46140/0 (socket says 172.21.15.40:46140) 2024-09-13T17:51:06.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f9ae291b700 1 -- 172.21.15.40:0/363507392 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9adc0c6c90 con 0x7f9adc0b00b0 2024-09-13T17:51:06.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f49bcbc9700 1 --2- 172.21.15.40:0/2793606027 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49b80cf1a0 0x7f49b815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.094+0000 7f86c759e700 1 -- start start 2024-09-13T17:51:06.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f86c759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80724e0 0x7f86c80c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f86c759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86c814f8c0 0x7f86c8150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f86c759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86c8158bc0 0x7f86c80cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f86c759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c814ff80 con 0x7f86c80724e0 2024-09-13T17:51:06.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f86c759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c81513b0 con 0x7f86c814f8c0 2024-09-13T17:51:06.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f86c759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c8152b80 con 0x7f86c8158bc0 2024-09-13T17:51:06.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f86c659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80724e0 0x7f86c80c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f86c659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80724e0 0x7f86c80c47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57988/0 (socket says 172.21.15.40:57988) 2024-09-13T17:51:06.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f86c659c700 1 -- 172.21.15.40:0/1072769599 learned_addr learned my addr 172.21.15.40:0/1072769599 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:06.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f86c5d9b700 1 --2- 172.21.15.40:0/1072769599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86c814f8c0 0x7f86c8150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f9ae291b700 1 --2- 172.21.15.40:0/363507392 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9adc0b00b0 0x7f9adc0c8230 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009d90 tx=0x7f9ad400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=740e9a950554ad5c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f49b77fe700 1 -- 172.21.15.40:0/2793606027 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3255099163 0 0) 0x7f49b0001490 con 0x7f49b80cc250 2024-09-13T17:51:06.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0da957d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0da4156c10 0x7f0da4160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0da957d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0da4156c10 0x7f0da4160b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57992/0 (socket says 172.21.15.40:57992) 2024-09-13T17:51:06.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f86c6d9d700 1 --2- 172.21.15.40:0/1072769599 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86c8158bc0 0x7f86c80cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f49b77fe700 1 -- 172.21.15.40:0/2793606027 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49a00035d0 con 0x7f49b80cc250 2024-09-13T17:51:06.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0bb13fd700 1 Processor -- start 2024-09-13T17:51:06.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0da3fff700 1 -- 172.21.15.40:0/1540131869 learned_addr learned my addr 172.21.15.40:0/1540131869 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:06.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0da8d7c700 1 --2- 172.21.15.40:0/1540131869 >> v2:172.21.15.43:3300/0 conn(0x7f0da4071110 0x7f0da40714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f86c659c700 1 -- 172.21.15.40:0/1072769599 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86c8158bc0 msgr2=0x7f86c80cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f86c659c700 1 --2- 172.21.15.40:0/1072769599 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86c8158bc0 0x7f86c80cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f86c659c700 1 -- 172.21.15.40:0/1072769599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86c814f8c0 msgr2=0x7f86c8150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f86c659c700 1 --2- 172.21.15.40:0/1072769599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86c814f8c0 0x7f86c8150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f86c659c700 1 -- 172.21.15.40:0/1072769599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c0009b30 con 0x7f86c80724e0 2024-09-13T17:51:06.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0bb13fd700 1 -- start start 2024-09-13T17:51:06.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f49b7fff700 1 -- 172.21.15.40:0/2793606027 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49b80cf1a0 msgr2=0x7f49b815a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:06.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f49b7fff700 1 --2- 172.21.15.40:0/2793606027 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49b80cf1a0 0x7f49b815a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f49b7fff700 1 -- 172.21.15.40:0/2793606027 >> v1:172.21.15.40:6789/0 conn(0x7f49b80cc250 legacy=0x7f49b80c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f49b7fff700 1 -- 172.21.15.40:0/2793606027 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b80caed0 con 0x7f49b80cf580 2024-09-13T17:51:06.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f44b083a700 1 Processor -- start 2024-09-13T17:51:06.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f9ae1118700 1 -- 172.21.15.40:0/363507392 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad400e070 con 0x7f9adc0b00b0 2024-09-13T17:51:06.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f9ae1118700 1 -- 172.21.15.40:0/363507392 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad400bbd0 con 0x7f9adc0b00b0 2024-09-13T17:51:06.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f9ae1118700 1 -- 172.21.15.40:0/363507392 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4012430 con 0x7f9adc0b00b0 2024-09-13T17:51:06.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0da9d7e700 1 -- 172.21.15.40:0/1540131869 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da40c7570 con 0x7f0da40c7010 2024-09-13T17:51:06.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f86c659c700 1 --2- 172.21.15.40:0/1072769599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80724e0 0x7f86c80c47f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00c500 tx=0x7f86bc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f49b7fff700 1 --2- 172.21.15.40:0/2793606027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b80cf580 0x7f49b80c9fd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f49ac009850 tx=0x7f49ac009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32ddf9e54b0b6451 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0da9d7e700 1 -- 172.21.15.40:0/1540131869 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da4071a30 con 0x7f0da4156c10 2024-09-13T17:51:06.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0da9d7e700 1 -- 172.21.15.40:0/1540131869 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da406dde0 con 0x7f0da4071110 2024-09-13T17:51:06.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f49b77fe700 1 -- 172.21.15.40:0/2793606027 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac011830 con 0x7f49b80cf580 2024-09-13T17:51:06.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f49b77fe700 1 -- 172.21.15.40:0/2793606027 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ac011990 con 0x7f49b80cf580 2024-09-13T17:51:06.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f49b77fe700 1 -- 172.21.15.40:0/2793606027 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac011d40 con 0x7f49b80cf580 2024-09-13T17:51:06.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f9ae311c700 1 -- 172.21.15.40:0/363507392 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9adc0b00b0 msgr2=0x7f9adc0c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f9ae311c700 1 --2- 172.21.15.40:0/363507392 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9adc0b00b0 0x7f9adc0c8230 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009d90 tx=0x7f9ad400b330 comp rx=0 tx=0).stop 2024-09-13T17:51:06.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0da37fe700 1 -- 172.21.15.40:0/1540131869 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3729849206 0 0) 0x7f0da40c7570 con 0x7f0da40c7010 2024-09-13T17:51:06.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0da37fe700 1 -- 172.21.15.40:0/1540131869 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d8c0057e0 con 0x7f0da40c7010 2024-09-13T17:51:06.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0bb13fd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0bac0724e0 0x7f0bac155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0bb13fd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bac14f8c0 0x7f0bac14dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0bb13fd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bac158bc0 0x7f0bac1ea540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0bb13fd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac0cbc30 con 0x7f0bac158bc0 2024-09-13T17:51:06.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0bb13fd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac0cad10 con 0x7f0bac14f8c0 2024-09-13T17:51:06.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0bb13fd700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac1eea10 con 0x7f0bac0724e0 2024-09-13T17:51:06.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0bb0bfc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bac158bc0 0x7f0bac1ea540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0bb0bfc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bac158bc0 0x7f0bac1ea540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58008/0 (socket says 172.21.15.40:58008) 2024-09-13T17:51:06.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0bb0bfc700 1 -- 172.21.15.40:0/2937177590 learned_addr learned my addr 172.21.15.40:0/2937177590 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:06.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0e1b54e700 1 Processor -- start 2024-09-13T17:51:06.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f44b083a700 1 -- start start 2024-09-13T17:51:06.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f44b083a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44ac0c7f60 0x7f44ac0c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0bab7fe700 1 --2- 172.21.15.40:0/2937177590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bac14f8c0 0x7f0bac14dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0da8d7c700 1 -- 172.21.15.40:0/1540131869 >> v2:172.21.15.40:3300/0 conn(0x7f0da4156c10 msgr2=0x7f0da4160b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:06.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0da8d7c700 1 --2- 172.21.15.40:0/1540131869 >> v2:172.21.15.40:3300/0 conn(0x7f0da4156c10 0x7f0da4160b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0da8d7c700 1 -- 172.21.15.40:0/1540131869 >> v1:172.21.15.40:6789/0 conn(0x7f0da40c7010 legacy=0x7f0da4156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0da8d7c700 1 -- 172.21.15.40:0/1540131869 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da40bff90 con 0x7f0da4071110 2024-09-13T17:51:06.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f44b083a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f44ac0c7250 0x7f44ac1577f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f44b083a700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44ac06b400 con 0x7f44ac0c8340 2024-09-13T17:51:06.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f44b083a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44ac15e920 con 0x7f44ac0c7250 2024-09-13T17:51:06.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f44b083a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ac0c76c0 con 0x7f44ac0c7f60 2024-09-13T17:51:06.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f44abfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f44ac0c7250 0x7f44ac1577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0da8d7c700 1 --2- 172.21.15.40:0/1540131869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0da4071110 0x7f0da40714f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0da4071c80 tx=0x7f0d98009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b510b4d4e3600b7d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0e1b54e700 1 -- start start 2024-09-13T17:51:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f44aaffd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f44ac0c8340 0x7f44ac06d920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:35396/0 (socket says 172.21.15.40:35396) 2024-09-13T17:51:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f44abfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f44ac0c7250 0x7f44ac1577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58020/0 (socket says 172.21.15.40:58020) 2024-09-13T17:51:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f44aaffd700 1 -- 172.21.15.40:0/61488912 learned_addr learned my addr 172.21.15.40:0/61488912 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f44ab7fe700 1 --2- 172.21.15.40:0/61488912 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44ac0c7f60 0x7f44ac0c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f9ae311c700 1 -- 172.21.15.40:0/363507392 shutdown_connections 2024-09-13T17:51:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f9ae311c700 1 --2- 172.21.15.40:0/363507392 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9adc0b00b0 0x7f9adc0c8230 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f9ae311c700 1 --2- 172.21.15.40:0/363507392 >> v2:172.21.15.43:3300/0 conn(0x7f9adc06e090 0x7f9adc0c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f9ae311c700 1 -- 172.21.15.40:0/363507392 >> 172.21.15.40:0/363507392 conn(0x7f9adc066090 msgr2=0x7f9adc0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:06.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0babfff700 1 --2- 172.21.15.40:0/2937177590 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0bac0724e0 0x7f0bac155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0bb0bfc700 1 -- 172.21.15.40:0/2937177590 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0bac0724e0 msgr2=0x7f0bac155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0bb0bfc700 1 --2- 172.21.15.40:0/2937177590 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0bac0724e0 0x7f0bac155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0bb0bfc700 1 -- 172.21.15.40:0/2937177590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bac14f8c0 msgr2=0x7f0bac14dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0bb0bfc700 1 --2- 172.21.15.40:0/2937177590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bac14f8c0 0x7f0bac14dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0bb0bfc700 1 -- 172.21.15.40:0/2937177590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba4009a70 con 0x7f0bac158bc0 2024-09-13T17:51:06.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.098+0000 7f0bb0bfc700 1 --2- 172.21.15.40:0/2937177590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bac158bc0 0x7f0bac1ea540 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0bac06b3c0 tx=0x7f0ba4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0ba8ff9700 1 -- 172.21.15.40:0/2937177590 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba401ba90 con 0x7f0bac158bc0 2024-09-13T17:51:06.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0da37fe700 1 -- 172.21.15.40:0/1540131869 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98011830 con 0x7f0da4071110 2024-09-13T17:51:06.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f86b6ffd700 1 -- 172.21.15.40:0/1072769599 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0052d0 con 0x7f86c80724e0 2024-09-13T17:51:06.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0bb13fd700 1 -- 172.21.15.40:0/2937177590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bac1ee580 con 0x7f0bac158bc0 2024-09-13T17:51:06.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0da37fe700 1 -- 172.21.15.40:0/1540131869 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d98011990 con 0x7f0da4071110 2024-09-13T17:51:06.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0da37fe700 1 -- 172.21.15.40:0/1540131869 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98011d20 con 0x7f0da4071110 2024-09-13T17:51:06.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0bb13fd700 1 -- 172.21.15.40:0/2937177590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bac1edc50 con 0x7f0bac158bc0 2024-09-13T17:51:06.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f86c759e700 1 -- 172.21.15.40:0/1072769599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c81f3ba0 con 0x7f86c80724e0 2024-09-13T17:51:06.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f86c759e700 1 -- 172.21.15.40:0/1072769599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c81f3e80 con 0x7f86c80724e0 2024-09-13T17:51:06.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f49bee2d700 1 -- 172.21.15.40:0/2793606027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b80cf580 msgr2=0x7f49b80c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f49bee2d700 1 --2- 172.21.15.40:0/2793606027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b80cf580 0x7f49b80c9fd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f49ac009850 tx=0x7f49ac009b60 comp rx=0 tx=0).stop 2024-09-13T17:51:06.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f9ae311c700 1 -- 172.21.15.40:0/363507392 shutdown_connections 2024-09-13T17:51:06.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0e1b54e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0e1406d920 0x7f0e140c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0e1b54e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0e141566b0 0x7f0e1414f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0e1b54e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e140c6110 con 0x7f0e1406d540 2024-09-13T17:51:06.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0e1b54e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e141559c0 con 0x7f0e141566b0 2024-09-13T17:51:06.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0e1b54e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e140c0f10 con 0x7f0e1406d920 2024-09-13T17:51:06.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0da9d7e700 1 -- 172.21.15.40:0/1540131869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0da4071110 msgr2=0x7f0da40714f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0da9d7e700 1 --2- 172.21.15.40:0/1540131869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0da4071110 0x7f0da40714f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0da4071c80 tx=0x7f0d98009b30 comp rx=0 tx=0).stop 2024-09-13T17:51:06.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0da9d7e700 1 -- 172.21.15.40:0/1540131869 shutdown_connections 2024-09-13T17:51:06.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0da9d7e700 1 --2- 172.21.15.40:0/1540131869 >> v2:172.21.15.40:3300/0 conn(0x7f0da4156c10 0x7f0da4160b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0da9d7e700 1 --2- 172.21.15.40:0/1540131869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0da4071110 0x7f0da40714f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0da9d7e700 1 -- 172.21.15.40:0/1540131869 >> 172.21.15.40:0/1540131869 conn(0x7f0da4066090 msgr2=0x7f0da40664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:06.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0da9d7e700 1 -- 172.21.15.40:0/1540131869 shutdown_connections 2024-09-13T17:51:06.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0da9d7e700 1 -- 172.21.15.40:0/1540131869 wait complete. 2024-09-13T17:51:06.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f44abfff700 1 -- 172.21.15.40:0/61488912 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44ac0c7f60 msgr2=0x7f44ac0c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:06.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f44abfff700 1 --2- 172.21.15.40:0/61488912 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44ac0c7f60 0x7f44ac0c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f44aa7fc700 1 -- 172.21.15.40:0/61488912 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977165167 0 0) 0x7f44ac06b400 con 0x7f44ac0c8340 2024-09-13T17:51:06.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f44abfff700 1 -- 172.21.15.40:0/61488912 >> v1:172.21.15.43:6789/0 conn(0x7f44ac0c8340 legacy=0x7f44ac06d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f44abfff700 1 -- 172.21.15.40:0/61488912 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ac15dda0 con 0x7f44ac0c7250 2024-09-13T17:51:06.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f44ab7fe700 1 --2- 172.21.15.40:0/61488912 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44ac0c7f60 0x7f44ac0c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:06.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f86b6ffd700 1 -- 172.21.15.40:0/1072769599 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86bc005430 con 0x7f86c80724e0 2024-09-13T17:51:06.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f86b6ffd700 1 -- 172.21.15.40:0/1072769599 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0055a0 con 0x7f86c80724e0 2024-09-13T17:51:06.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f86c759e700 1 -- 172.21.15.40:0/1072769599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86ac005310 con 0x7f86c80724e0 2024-09-13T17:51:06.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f9ae311c700 1 -- 172.21.15.40:0/363507392 wait complete. 2024-09-13T17:51:06.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0e19d4b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0e1406d920 0x7f0e140c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0e19d4b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0e1406d920 0x7f0e140c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:37070/0 (socket says 172.21.15.40:37070) 2024-09-13T17:51:06.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0e19d4b700 1 -- 172.21.15.40:0/1864178771 learned_addr learned my addr 172.21.15.40:0/1864178771 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:06.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0e19d4b700 1 -- 172.21.15.40:0/1864178771 >> v2:172.21.15.40:3300/0 conn(0x7f0e141566b0 msgr2=0x7f0e1414f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:06.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0e19d4b700 1 --2- 172.21.15.40:0/1864178771 >> v2:172.21.15.40:3300/0 conn(0x7f0e141566b0 0x7f0e1414f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0e19d4b700 1 -- 172.21.15.40:0/1864178771 >> v1:172.21.15.43:6789/0 conn(0x7f0e1406d540 legacy=0x7f0e140c0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0ba8ff9700 1 -- 172.21.15.40:0/2937177590 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba4028070 con 0x7f0bac158bc0 2024-09-13T17:51:06.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0ba8ff9700 1 -- 172.21.15.40:0/2937177590 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba401bbf0 con 0x7f0bac158bc0 2024-09-13T17:51:06.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0e19d4b700 1 -- 172.21.15.40:0/1864178771 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e140c11d0 con 0x7f0e1406d920 2024-09-13T17:51:06.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0e19d4b700 1 --2- 172.21.15.40:0/1864178771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e1406d920 0x7f0e140c5bd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0e140c5b60 tx=0x7f0e0c00d430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8ac7f5067066d83 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f49bee2d700 1 -- 172.21.15.40:0/2793606027 shutdown_connections 2024-09-13T17:51:06.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f49bee2d700 1 --2- 172.21.15.40:0/2793606027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b80cf580 0x7f49b80c9fd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f49bee2d700 1 --2- 172.21.15.40:0/2793606027 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49b80cf1a0 0x7f49b815a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f49bee2d700 1 -- 172.21.15.40:0/2793606027 >> 172.21.15.40:0/2793606027 conn(0x7f49b8066090 msgr2=0x7f49b8066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:06.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f9ae311c700 1 Processor -- start 2024-09-13T17:51:06.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f9ae311c700 1 -- start start 2024-09-13T17:51:06.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0ba8ff9700 1 -- 172.21.15.40:0/2937177590 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0ba402e400 con 0x7f0bac158bc0 2024-09-13T17:51:06.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0e1954a700 1 -- 172.21.15.40:0/1864178771 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c010070 con 0x7f0e1406d920 2024-09-13T17:51:06.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f44abfff700 1 --2- 172.21.15.40:0/61488912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44ac0c7250 0x7f44ac1577f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f44a400b050 tx=0x7f44a400b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb38622538bd0f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0e1954a700 1 -- 172.21.15.40:0/1864178771 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0c00db80 con 0x7f0e1406d920 2024-09-13T17:51:06.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f86b6ffd700 1 -- 172.21.15.40:0/1072769599 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f86bc0038e0 con 0x7f86c80724e0 2024-09-13T17:51:06.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0bb13fd700 1 -- 172.21.15.40:0/2937177590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b8c005310 con 0x7f0bac158bc0 2024-09-13T17:51:06.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0ba8ff9700 1 --2- 172.21.15.40:0/2937177590 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0b98066730 0x7f0b98068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0ba8ff9700 1 -- 172.21.15.40:0/2937177590 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0ba408ec00 con 0x7f0bac158bc0 2024-09-13T17:51:06.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f86b6ffd700 1 --2- 172.21.15.40:0/1072769599 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86a8066800 0x7f86a8068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0da9d7e700 1 Processor -- start 2024-09-13T17:51:06.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f0da9d7e700 1 -- start start 2024-09-13T17:51:06.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.102+0000 7f49bee2d700 1 -- 172.21.15.40:0/2793606027 shutdown_connections 2024-09-13T17:51:06.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f86b6ffd700 1 -- 172.21.15.40:0/1072769599 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f86bc083a10 con 0x7f86c80724e0 2024-09-13T17:51:06.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e1b54e700 1 -- 172.21.15.40:0/1864178771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e1406d920 msgr2=0x7f0e140c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e1b54e700 1 --2- 172.21.15.40:0/1864178771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e1406d920 0x7f0e140c5bd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0e140c5b60 tx=0x7f0e0c00d430 comp rx=0 tx=0).stop 2024-09-13T17:51:06.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0babfff700 1 --2- 172.21.15.40:0/2937177590 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0b98066730 0x7f0b98068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ae311c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9adc06dcb0 0x7f9adc1f2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f86c5d9b700 1 --2- 172.21.15.40:0/1072769599 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86a8066800 0x7f86a8068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ae311c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9adc06e090 0x7f9adc071400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ae311c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9adc0b00b0 0x7f9adc0c33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ae311c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc1ef390 con 0x7f9adc06e090 2024-09-13T17:51:06.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ae311c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc1ef8c0 con 0x7f9adc06dcb0 2024-09-13T17:51:06.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e1b54e700 1 -- 172.21.15.40:0/1864178771 shutdown_connections 2024-09-13T17:51:06.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e1b54e700 1 --2- 172.21.15.40:0/1864178771 >> v2:172.21.15.40:3300/0 conn(0x7f0e141566b0 0x7f0e1414f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e1b54e700 1 --2- 172.21.15.40:0/1864178771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e1406d920 0x7f0e140c5bd0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0e140c5b60 tx=0x7f0e0c00d430 comp rx=0 tx=0).stop 2024-09-13T17:51:06.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e1b54e700 1 -- 172.21.15.40:0/1864178771 >> 172.21.15.40:0/1864178771 conn(0x7f0e14066090 msgr2=0x7f0e140664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:06.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e1b54e700 1 -- 172.21.15.40:0/1864178771 shutdown_connections 2024-09-13T17:51:06.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e1b54e700 1 -- 172.21.15.40:0/1864178771 wait complete. 2024-09-13T17:51:06.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ae211a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9adc06dcb0 0x7f9adc1f2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ae211a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9adc06dcb0 0x7f9adc1f2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:37074/0 (socket says 172.21.15.40:37074) 2024-09-13T17:51:06.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ae211a700 1 -- 172.21.15.40:0/2619580716 learned_addr learned my addr 172.21.15.40:0/2619580716 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:06.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ae311c700 1 -- 172.21.15.40:0/2619580716 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc1f2030 con 0x7f9adc0b00b0 2024-09-13T17:51:06.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f86c5d9b700 1 --2- 172.21.15.40:0/1072769599 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86a8066800 0x7f86a8068cc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f86c814a4d0 tx=0x7f86b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ae1919700 1 --2- 172.21.15.40:0/2619580716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9adc06e090 0x7f9adc071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0da9d7e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0da4071110 0x7f0da413f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0da9d7e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0da40c7010 0x7f0da4146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0da9d7e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0da4156c10 0x7f0da41e2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0da9d7e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da414fc80 con 0x7f0da4071110 2024-09-13T17:51:06.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0da9d7e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da41e1f60 con 0x7f0da40c7010 2024-09-13T17:51:06.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0da9d7e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da413fda0 con 0x7f0da4156c10 2024-09-13T17:51:06.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ae211a700 1 -- 172.21.15.40:0/2619580716 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9adc0b00b0 msgr2=0x7f9adc0c33f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:51:06.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ae211a700 1 --2- 172.21.15.40:0/2619580716 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9adc0b00b0 0x7f9adc0c33f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ae211a700 1 -- 172.21.15.40:0/2619580716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9adc06e090 msgr2=0x7f9adc071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ae211a700 1 --2- 172.21.15.40:0/2619580716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9adc06e090 0x7f9adc071400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ae211a700 1 -- 172.21.15.40:0/2619580716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad4009a70 con 0x7f9adc06dcb0 2024-09-13T17:51:06.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0babfff700 1 --2- 172.21.15.40:0/2937177590 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0b98066730 0x7f0b98068bf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c000f60 tx=0x7f0b9c007c40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0da957d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0da4156c10 0x7f0da41e2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0da957d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0da4156c10 0x7f0da41e2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:48996/0 (socket says 172.21.15.40:48996) 2024-09-13T17:51:06.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0da957d700 1 -- 172.21.15.40:0/688391132 learned_addr learned my addr 172.21.15.40:0/688391132 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:06.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0da8d7c700 1 --2- 172.21.15.40:0/688391132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0da4071110 0x7f0da413f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f49bee2d700 1 -- 172.21.15.40:0/2793606027 wait complete. 2024-09-13T17:51:06.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ae1919700 1 --2- 172.21.15.40:0/2619580716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9adc06e090 0x7f9adc071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:06.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ae211a700 1 --2- 172.21.15.40:0/2619580716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9adc06dcb0 0x7f9adc1f2ad0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800c580 tx=0x7f9ad800c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f49bee2d700 1 Processor -- start 2024-09-13T17:51:06.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0da957d700 1 -- 172.21.15.40:0/688391132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0da40c7010 msgr2=0x7f0da4146520 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T17:51:06.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0da957d700 1 --2- 172.21.15.40:0/688391132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0da40c7010 0x7f0da4146520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0da957d700 1 -- 172.21.15.40:0/688391132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0da4071110 msgr2=0x7f0da413f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0da957d700 1 --2- 172.21.15.40:0/688391132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0da4071110 0x7f0da413f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0da957d700 1 -- 172.21.15.40:0/688391132 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d980094b0 con 0x7f0da4156c10 2024-09-13T17:51:06.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e1b54e700 1 Processor -- start 2024-09-13T17:51:06.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f49bee2d700 1 -- start start 2024-09-13T17:51:06.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ad2ffd700 1 -- 172.21.15.40:0/2619580716 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8004ca0 con 0x7f9adc06dcb0 2024-09-13T17:51:06.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e1b54e700 1 -- start start 2024-09-13T17:51:06.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ad2ffd700 1 -- 172.21.15.40:0/2619580716 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad8004e00 con 0x7f9adc06dcb0 2024-09-13T17:51:06.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ad2ffd700 1 -- 172.21.15.40:0/2619580716 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8007310 con 0x7f9adc06dcb0 2024-09-13T17:51:06.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ae311c700 1 -- 172.21.15.40:0/2619580716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9adc1ef5e0 con 0x7f9adc06dcb0 2024-09-13T17:51:06.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f9ae311c700 1 -- 172.21.15.40:0/2619580716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9adc1f25b0 con 0x7f9adc06dcb0 2024-09-13T17:51:06.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0da957d700 1 --2- 172.21.15.40:0/688391132 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0da4156c10 0x7f0da41e2690 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400e7f0 tx=0x7f0d9400eb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e1b54e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e1406d540 0x7f0e140aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e1b54e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e141566b0 0x7f0e140ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e1b54e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e140cacb0 0x7f0e140c35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e1b54e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e140c2e70 con 0x7f0e141566b0 2024-09-13T17:51:06.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f49bee2d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b80cc250 0x7f49b80c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e1b54e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e140c9d80 con 0x7f0e140cacb0 2024-09-13T17:51:06.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e1b54e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e140c4cc0 con 0x7f0e1406d540 2024-09-13T17:51:06.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f49bee2d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49b80cf1a0 0x7f49b8156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e19d4b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e141566b0 0x7f0e140ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f49bee2d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49b80cf580 0x7f49b81ef140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f49bee2d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b81ff300 con 0x7f49b80cf580 2024-09-13T17:51:06.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f49bee2d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b815a5e0 con 0x7f49b80cc250 2024-09-13T17:51:06.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f49bee2d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b81fcc10 con 0x7f49b80cf1a0 2024-09-13T17:51:06.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e1a54c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e1406d540 0x7f0e140aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e1a54c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e1406d540 0x7f0e140aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:49010/0 (socket says 172.21.15.40:49010) 2024-09-13T17:51:06.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e1a54c700 1 -- 172.21.15.40:0/2914058291 learned_addr learned my addr 172.21.15.40:0/2914058291 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:06.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e19d4b700 1 --2- 172.21.15.40:0/2914058291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e141566b0 0x7f0e140ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58062/0 (socket says 172.21.15.40:58062) 2024-09-13T17:51:06.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f49bd3ca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49b80cf580 0x7f49b81ef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0e1ad4d700 1 --2- 172.21.15.40:0/2914058291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e140cacb0 0x7f0e140c35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f49b7fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49b80cf1a0 0x7f49b8156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f49bcbc9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b80cc250 0x7f49b80c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f49bd3ca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49b80cf580 0x7f49b81ef140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58066/0 (socket says 172.21.15.40:58066) 2024-09-13T17:51:06.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f49bd3ca700 1 -- 172.21.15.40:0/1730148452 learned_addr learned my addr 172.21.15.40:0/1730148452 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:06.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0da17fa700 1 -- 172.21.15.40:0/688391132 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d940095c0 con 0x7f0da4156c10 2024-09-13T17:51:06.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0da17fa700 1 -- 172.21.15.40:0/688391132 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d94013680 con 0x7f0da4156c10 2024-09-13T17:51:06.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.106+0000 7f0da9d7e700 1 -- 172.21.15.40:0/688391132 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da414d9a0 con 0x7f0da4156c10 2024-09-13T17:51:06.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f0da17fa700 1 -- 172.21.15.40:0/688391132 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d94013a10 con 0x7f0da4156c10 2024-09-13T17:51:06.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f0da9d7e700 1 -- 172.21.15.40:0/688391132 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da41e2170 con 0x7f0da4156c10 2024-09-13T17:51:06.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f49b7fff700 1 -- 172.21.15.40:0/1730148452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b80cc250 msgr2=0x7f49b80c46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f49b7fff700 1 --2- 172.21.15.40:0/1730148452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b80cc250 0x7f49b80c46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f49b7fff700 1 -- 172.21.15.40:0/1730148452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49b80cf580 msgr2=0x7f49b81ef140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f49b7fff700 1 --2- 172.21.15.40:0/1730148452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49b80cf580 0x7f49b81ef140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f49b7fff700 1 -- 172.21.15.40:0/1730148452 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac0094b0 con 0x7f49b80cf1a0 2024-09-13T17:51:06.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f9ae311c700 1 -- 172.21.15.40:0/2619580716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9adc1f27d0 con 0x7f9adc06dcb0 2024-09-13T17:51:06.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f49b7fff700 1 --2- 172.21.15.40:0/1730148452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49b80cf1a0 0x7f49b8156790 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f49ac016000 tx=0x7f49ac004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f0e19d4b700 1 -- 172.21.15.40:0/2914058291 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e1406d540 msgr2=0x7f0e140aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f0e19d4b700 1 --2- 172.21.15.40:0/2914058291 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e1406d540 0x7f0e140aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f0e19d4b700 1 -- 172.21.15.40:0/2914058291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e140cacb0 msgr2=0x7f0e140c35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f0e19d4b700 1 --2- 172.21.15.40:0/2914058291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e140cacb0 0x7f0e140c35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f0e19d4b700 1 -- 172.21.15.40:0/2914058291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c00bd40 con 0x7f0e141566b0 2024-09-13T17:51:06.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f0e1a54c700 1 --2- 172.21.15.40:0/2914058291 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e1406d540 0x7f0e140aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:06.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f0e19d4b700 1 --2- 172.21.15.40:0/2914058291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e141566b0 0x7f0e140ca770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c019000 tx=0x7f0e0c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f49b57fa700 1 -- 172.21.15.40:0/1730148452 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac004450 con 0x7f49b80cf1a0 2024-09-13T17:51:06.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f0da9d7e700 1 -- 172.21.15.40:0/688391132 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da41e2390 con 0x7f0da4156c10 2024-09-13T17:51:06.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f9ad2ffd700 1 -- 172.21.15.40:0/2619580716 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9ad8007470 con 0x7f9adc06dcb0 2024-09-13T17:51:06.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f49bee2d700 1 -- 172.21.15.40:0/1730148452 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b81fc8e0 con 0x7f49b80cf1a0 2024-09-13T17:51:06.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f49b57fa700 1 -- 172.21.15.40:0/1730148452 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ac01e070 con 0x7f49b80cf1a0 2024-09-13T17:51:06.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f49bee2d700 1 -- 172.21.15.40:0/1730148452 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b81ff4f0 con 0x7f49b80cf1a0 2024-09-13T17:51:06.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f49b57fa700 1 -- 172.21.15.40:0/1730148452 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac0038c0 con 0x7f49b80cf1a0 2024-09-13T17:51:06.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f9ad2ffd700 1 --2- 172.21.15.40:0/2619580716 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9ac0066780 0x7f9ac0068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f1ae759e700 1 Processor -- start 2024-09-13T17:51:06.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f9ad2ffd700 1 -- 172.21.15.40:0/2619580716 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9ad80848d0 con 0x7f9adc06dcb0 2024-09-13T17:51:06.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f9ae1919700 1 --2- 172.21.15.40:0/2619580716 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9ac0066780 0x7f9ac0068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f1ae759e700 1 -- start start 2024-09-13T17:51:06.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f49bee2d700 1 -- 172.21.15.40:0/1730148452 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b8067a40 con 0x7f49b80cf1a0 2024-09-13T17:51:06.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f0da17fa700 1 -- 172.21.15.40:0/688391132 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0d94013b70 con 0x7f0da4156c10 2024-09-13T17:51:06.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f0da17fa700 1 --2- 172.21.15.40:0/688391132 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0d90066780 0x7f0d90068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f0e0affd700 1 -- 172.21.15.40:0/2914058291 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c022070 con 0x7f0e141566b0 2024-09-13T17:51:06.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f1ae759e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1ae80c78d0 0x7f1ae814f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f0da17fa700 1 -- 172.21.15.40:0/688391132 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0d940849f0 con 0x7f0da4156c10 2024-09-13T17:51:06.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f0da8d7c700 1 --2- 172.21.15.40:0/688391132 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0d90066780 0x7f0d90068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f86b6ffd700 1 -- 172.21.15.40:0/1072769599 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f86bc050a70 con 0x7f86c80724e0 2024-09-13T17:51:06.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f0e1b54e700 1 -- 172.21.15.40:0/2914058291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e140c4fa0 con 0x7f0e141566b0 2024-09-13T17:51:06.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f1ae759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ae8158bc0 0x7f1ae80c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f1ae759e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ae80c6890 con 0x7f1ae80724e0 2024-09-13T17:51:06.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f1ae759e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae80c7700 con 0x7f1ae80c78d0 2024-09-13T17:51:06.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f1ae759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae80c5f30 con 0x7f1ae8158bc0 2024-09-13T17:51:06.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f44aa7fc700 1 -- 172.21.15.40:0/61488912 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a400e070 con 0x7f44ac0c7250 2024-09-13T17:51:06.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f9ae1919700 1 --2- 172.21.15.40:0/2619580716 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9ac0066780 0x7f9ac0068c40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9adc157a50 tx=0x7f9acc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f0e1b54e700 1 -- 172.21.15.40:0/2914058291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e140c2440 con 0x7f0e141566b0 2024-09-13T17:51:06.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f1ae659c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1ae80c78d0 0x7f1ae814f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f1ae659c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1ae80c78d0 0x7f1ae814f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:37104/0 (socket says 172.21.15.40:37104) 2024-09-13T17:51:06.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f1ae5d9b700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1ae80724e0 0x7f1ae806d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:35412/0 (socket says 172.21.15.40:35412) 2024-09-13T17:51:06.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f1ae659c700 1 -- 172.21.15.40:0/437975094 learned_addr learned my addr 172.21.15.40:0/437975094 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:06.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.110+0000 7f1ae6d9d700 1 --2- 172.21.15.40:0/437975094 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ae8158bc0 0x7f1ae80c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.114+0000 7f0da8d7c700 1 --2- 172.21.15.40:0/688391132 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0d90066780 0x7f0d90068c40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0d98000c00 tx=0x7f0d98008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.114+0000 7f1ae659c700 1 -- 172.21.15.40:0/437975094 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ae8158bc0 msgr2=0x7f1ae80c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:06.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.114+0000 7f1ae659c700 1 --2- 172.21.15.40:0/437975094 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ae8158bc0 0x7f1ae80c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.114+0000 7f1ae659c700 1 -- 172.21.15.40:0/437975094 >> v1:172.21.15.43:6789/0 conn(0x7f1ae80724e0 legacy=0x7f1ae806d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.114+0000 7f1ae659c700 1 -- 172.21.15.40:0/437975094 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae80c1320 con 0x7f1ae80c78d0 2024-09-13T17:51:06.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.114+0000 7f1ae659c700 1 --2- 172.21.15.40:0/437975094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ae80c78d0 0x7f1ae814f8c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009910 tx=0x7f1ad8009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c48493e22602bf65 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f0e1b54e700 1 -- 172.21.15.40:0/2914058291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e140c3300 con 0x7f0e141566b0 2024-09-13T17:51:06.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f44aa7fc700 1 -- 172.21.15.40:0/61488912 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a400bb10 con 0x7f44ac0c7250 2024-09-13T17:51:06.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f44aa7fc700 1 -- 172.21.15.40:0/61488912 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a4012430 con 0x7f44ac0c7250 2024-09-13T17:51:06.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f0e0affd700 1 -- 172.21.15.40:0/2914058291 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0c00dc80 con 0x7f0e141566b0 2024-09-13T17:51:06.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f0e0affd700 1 -- 172.21.15.40:0/2914058291 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c00ddf0 con 0x7f0e141566b0 2024-09-13T17:51:06.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f0ba8ff9700 1 -- 172.21.15.40:0/2937177590 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0ba402a020 con 0x7f0bac158bc0 2024-09-13T17:51:06.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f44b083a700 1 -- 172.21.15.40:0/61488912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44ac0c7250 msgr2=0x7f44ac1577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f44b083a700 1 --2- 172.21.15.40:0/61488912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44ac0c7250 0x7f44ac1577f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f44a400b050 tx=0x7f44a400b360 comp rx=0 tx=0).stop 2024-09-13T17:51:06.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f44b083a700 1 -- 172.21.15.40:0/61488912 shutdown_connections 2024-09-13T17:51:06.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f44b083a700 1 --2- 172.21.15.40:0/61488912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44ac0c7250 0x7f44ac1577f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f44b083a700 1 --2- 172.21.15.40:0/61488912 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44ac0c7f60 0x7f44ac0c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f44b083a700 1 -- 172.21.15.40:0/61488912 >> 172.21.15.40:0/61488912 conn(0x7f44ac066090 msgr2=0x7f44ac0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:06.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f44b083a700 1 -- 172.21.15.40:0/61488912 shutdown_connections 2024-09-13T17:51:06.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f44b083a700 1 -- 172.21.15.40:0/61488912 wait complete. 2024-09-13T17:51:06.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f0e0affd700 1 -- 172.21.15.40:0/2914058291 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0e0c004040 con 0x7f0e141566b0 2024-09-13T17:51:06.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f1ae559a700 1 -- 172.21.15.40:0/437975094 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8011830 con 0x7f1ae80c78d0 2024-09-13T17:51:06.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f1ae559a700 1 -- 172.21.15.40:0/437975094 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad8011990 con 0x7f1ae80c78d0 2024-09-13T17:51:06.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f1ae559a700 1 -- 172.21.15.40:0/437975094 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8011d40 con 0x7f1ae80c78d0 2024-09-13T17:51:06.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f0e0affd700 1 --2- 172.21.15.40:0/2914058291 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0df8066730 0x7f0df8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f0e0affd700 1 -- 172.21.15.40:0/2914058291 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0e0c0855b0 con 0x7f0e141566b0 2024-09-13T17:51:06.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f0e1a54c700 1 --2- 172.21.15.40:0/2914058291 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0df8066730 0x7f0df8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f0e1a54c700 1 --2- 172.21.15.40:0/2914058291 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0df8066730 0x7f0df8068bf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0e10001d30 tx=0x7f0e10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f1ae759e700 1 -- 172.21.15.40:0/437975094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ae80c78d0 msgr2=0x7f1ae814f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.118+0000 7f1ae759e700 1 --2- 172.21.15.40:0/437975094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ae80c78d0 0x7f1ae814f8c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009910 tx=0x7f1ad8009c20 comp rx=0 tx=0).stop 2024-09-13T17:51:06.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f9ad2ffd700 1 -- 172.21.15.40:0/2619580716 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9ad8051930 con 0x7f9adc06dcb0 2024-09-13T17:51:06.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f1ae759e700 1 -- 172.21.15.40:0/437975094 shutdown_connections 2024-09-13T17:51:06.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f1ae759e700 1 --2- 172.21.15.40:0/437975094 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ae8158bc0 0x7f1ae80c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f1ae759e700 1 --2- 172.21.15.40:0/437975094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ae80c78d0 0x7f1ae814f8c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f1ae759e700 1 -- 172.21.15.40:0/437975094 >> 172.21.15.40:0/437975094 conn(0x7f1ae8066090 msgr2=0x7f1ae80664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f44b083a700 1 Processor -- start 2024-09-13T17:51:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f44b083a700 1 -- start start 2024-09-13T17:51:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f44b083a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44ac0c7250 0x7f44ac1530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f44b083a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44ac0c7f60 0x7f44ac1f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f44b083a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44ac0c8340 0x7f44ac071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f44b083a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ac14d510 con 0x7f44ac0c8340 2024-09-13T17:51:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f44b083a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ac14d8c0 con 0x7f44ac0c7f60 2024-09-13T17:51:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f44b083a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ac0ca7e0 con 0x7f44ac0c7250 2024-09-13T17:51:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f44aaffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44ac0c7f60 0x7f44ac1f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f44aaffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44ac0c7f60 0x7f44ac1f67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:37112/0 (socket says 172.21.15.40:37112) 2024-09-13T17:51:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f44aaffd700 1 -- 172.21.15.40:0/526032025 learned_addr learned my addr 172.21.15.40:0/526032025 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f44ab7fe700 1 --2- 172.21.15.40:0/526032025 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44ac0c7250 0x7f44ac1530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f1ae759e700 1 -- 172.21.15.40:0/437975094 shutdown_connections 2024-09-13T17:51:06.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f1ae759e700 1 -- 172.21.15.40:0/437975094 wait complete. 2024-09-13T17:51:06.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f44aaffd700 1 -- 172.21.15.40:0/526032025 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44ac0c7250 msgr2=0x7f44ac1530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f44aaffd700 1 --2- 172.21.15.40:0/526032025 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44ac0c7250 0x7f44ac1530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f44aaffd700 1 -- 172.21.15.40:0/526032025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44ac0c8340 msgr2=0x7f44ac071030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:51:06.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f44aaffd700 1 --2- 172.21.15.40:0/526032025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44ac0c8340 0x7f44ac071030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f44aaffd700 1 -- 172.21.15.40:0/526032025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a4009a70 con 0x7f44ac0c7f60 2024-09-13T17:51:06.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f44aaffd700 1 --2- 172.21.15.40:0/526032025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44ac0c7f60 0x7f44ac1f67e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f44a000c750 tx=0x7f44a000cb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f0da17fa700 1 -- 172.21.15.40:0/688391132 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0d94051b50 con 0x7f0da4156c10 2024-09-13T17:51:06.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f0e0affd700 1 -- 172.21.15.40:0/2914058291 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0e0c0526c0 con 0x7f0e141566b0 2024-09-13T17:51:06.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f4493fff700 1 -- 172.21.15.40:0/526032025 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0004ca0 con 0x7f44ac0c7f60 2024-09-13T17:51:06.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f44b083a700 1 -- 172.21.15.40:0/526032025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44ac0c97b0 con 0x7f44ac0c7f60 2024-09-13T17:51:06.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f44b083a700 1 -- 172.21.15.40:0/526032025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ac0ca470 con 0x7f44ac0c7f60 2024-09-13T17:51:06.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f4493fff700 1 -- 172.21.15.40:0/526032025 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a0004e00 con 0x7f44ac0c7f60 2024-09-13T17:51:06.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f49b57fa700 1 -- 172.21.15.40:0/1730148452 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f49ac003a20 con 0x7f49b80cf1a0 2024-09-13T17:51:06.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.126+0000 7f49b57fa700 1 --2- 172.21.15.40:0/1730148452 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f49a4066730 0x7f49a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.126+0000 7f49b57fa700 1 -- 172.21.15.40:0/1730148452 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f49ac085770 con 0x7f49b80cf1a0 2024-09-13T17:51:06.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.126+0000 7f4493fff700 1 -- 172.21.15.40:0/526032025 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a0007310 con 0x7f44ac0c7f60 2024-09-13T17:51:06.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.126+0000 7f44b083a700 1 -- 172.21.15.40:0/526032025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448c005310 con 0x7f44ac0c7f60 2024-09-13T17:51:06.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.126+0000 7f49bcbc9700 1 --2- 172.21.15.40:0/1730148452 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f49a4066730 0x7f49a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.126+0000 7f49bcbc9700 1 --2- 172.21.15.40:0/1730148452 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f49a4066730 0x7f49a4068bf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009bf0 tx=0x7f49a8009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.126+0000 7f4493fff700 1 -- 172.21.15.40:0/526032025 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f44a0007470 con 0x7f44ac0c7f60 2024-09-13T17:51:06.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.126+0000 7f4493fff700 1 --2- 172.21.15.40:0/526032025 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4498066780 0x7f4498068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.126+0000 7f44ab7fe700 1 --2- 172.21.15.40:0/526032025 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4498066780 0x7f4498068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.126+0000 7f4493fff700 1 -- 172.21.15.40:0/526032025 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f44a00847d0 con 0x7f44ac0c7f60 2024-09-13T17:51:06.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.126+0000 7f44ab7fe700 1 --2- 172.21.15.40:0/526032025 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4498066780 0x7f4498068c40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f449c009950 tx=0x7f449c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.126+0000 7f49b57fa700 1 -- 172.21.15.40:0/1730148452 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f49ac052880 con 0x7f49b80cf1a0 2024-09-13T17:51:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f1ae759e700 1 Processor -- start 2024-09-13T17:51:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.122+0000 7f1ae759e700 1 -- start start 2024-09-13T17:51:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.130+0000 7f4493fff700 1 -- 172.21.15.40:0/526032025 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f44a0051930 con 0x7f44ac0c7f60 2024-09-13T17:51:06.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.130+0000 7f1ae759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ae80724e0 0x7f1ae80cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.130+0000 7f1ae759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ae80c78d0 0x7f1ae80706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.130+0000 7f1ae759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ae8158bc0 0x7f1ae80c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.130+0000 7f1ae759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae81410e0 con 0x7f1ae80724e0 2024-09-13T17:51:06.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.130+0000 7f1ae759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae8152530 con 0x7f1ae80c78d0 2024-09-13T17:51:06.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.130+0000 7f1ae759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae8152250 con 0x7f1ae8158bc0 2024-09-13T17:51:06.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.130+0000 7f1ae6d9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ae8158bc0 0x7f1ae80c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.130+0000 7f1ae6d9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ae8158bc0 0x7f1ae80c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:49054/0 (socket says 172.21.15.40:49054) 2024-09-13T17:51:06.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.130+0000 7f1ae6d9d700 1 -- 172.21.15.40:0/2656959874 learned_addr learned my addr 172.21.15.40:0/2656959874 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:06.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.134+0000 7f1ae6d9d700 1 -- 172.21.15.40:0/2656959874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ae80c78d0 msgr2=0x7f1ae80706e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:51:06.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.134+0000 7f1ae6d9d700 1 --2- 172.21.15.40:0/2656959874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ae80c78d0 0x7f1ae80706e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.134+0000 7f1ae6d9d700 1 -- 172.21.15.40:0/2656959874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ae80724e0 msgr2=0x7f1ae80cb650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:51:06.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.134+0000 7f1ae6d9d700 1 --2- 172.21.15.40:0/2656959874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ae80724e0 0x7f1ae80cb650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.134+0000 7f1ae6d9d700 1 -- 172.21.15.40:0/2656959874 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad8009570 con 0x7f1ae8158bc0 2024-09-13T17:51:06.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.134+0000 7f1ae6d9d700 1 --2- 172.21.15.40:0/2656959874 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ae8158bc0 0x7f1ae80c49d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000efc0 tx=0x7f1ae000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.138+0000 7f1ad6ffd700 1 -- 172.21.15.40:0/2656959874 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae000cd40 con 0x7f1ae8158bc0 2024-09-13T17:51:06.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.138+0000 7f1ad6ffd700 1 -- 172.21.15.40:0/2656959874 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae000f040 con 0x7f1ae8158bc0 2024-09-13T17:51:06.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.138+0000 7f1ae759e700 1 -- 172.21.15.40:0/2656959874 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae814ed10 con 0x7f1ae8158bc0 2024-09-13T17:51:06.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.138+0000 7f1ae759e700 1 -- 172.21.15.40:0/2656959874 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae80cb130 con 0x7f1ae8158bc0 2024-09-13T17:51:06.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.138+0000 7f1ad6ffd700 1 -- 172.21.15.40:0/2656959874 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae00146c0 con 0x7f1ae8158bc0 2024-09-13T17:51:06.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.142+0000 7f1ad6ffd700 1 -- 172.21.15.40:0/2656959874 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1ae0014820 con 0x7f1ae8158bc0 2024-09-13T17:51:06.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.142+0000 7f1ad6ffd700 1 --2- 172.21.15.40:0/2656959874 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1ac80667b0 0x7f1ac8068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:06.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.142+0000 7f1ad6ffd700 1 -- 172.21.15.40:0/2656959874 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1ae00842d0 con 0x7f1ae8158bc0 2024-09-13T17:51:06.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.142+0000 7f1ae659c700 1 --2- 172.21.15.40:0/2656959874 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1ac80667b0 0x7f1ac8068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:06.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.142+0000 7f1ae659c700 1 --2- 172.21.15.40:0/2656959874 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1ac80667b0 0x7f1ac8068c70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009c60 tx=0x7f1ad8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:06.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.150+0000 7f1ad4ff9700 1 -- 172.21.15.40:0/2656959874 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae8072a10 con 0x7f1ae8158bc0 2024-09-13T17:51:06.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.158+0000 7f1ad6ffd700 1 -- 172.21.15.40:0/2656959874 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1ae0089060 con 0x7f1ae8158bc0 2024-09-13T17:51:06.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.446+0000 7f86c759e700 1 -- 172.21.15.40:0/1072769599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f86ac0055c0 con 0x7f86c80724e0 2024-09-13T17:51:06.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.446+0000 7f86b6ffd700 1 -- 172.21.15.40:0/1072769599 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f86bc054090 con 0x7f86c80724e0 2024-09-13T17:51:06.455 INFO:teuthology.orchestra.run.smithi040.stdout:60129542153 2024-09-13T17:51:06.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.454+0000 7f86b4ff9700 1 -- 172.21.15.40:0/1072769599 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86a8066800 msgr2=0x7f86a8068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.454+0000 7f86b4ff9700 1 --2- 172.21.15.40:0/1072769599 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86a8066800 0x7f86a8068cc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f86c814a4d0 tx=0x7f86b8008040 comp rx=0 tx=0).stop 2024-09-13T17:51:06.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.454+0000 7f86b4ff9700 1 -- 172.21.15.40:0/1072769599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80724e0 msgr2=0x7f86c80c47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.454+0000 7f86b4ff9700 1 --2- 172.21.15.40:0/1072769599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80724e0 0x7f86c80c47f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00c500 tx=0x7f86bc00c8c0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.454+0000 7f86b4ff9700 1 -- 172.21.15.40:0/1072769599 shutdown_connections 2024-09-13T17:51:06.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.454+0000 7f86b4ff9700 1 --2- 172.21.15.40:0/1072769599 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86a8066800 0x7f86a8068cc0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.454+0000 7f86b4ff9700 1 --2- 172.21.15.40:0/1072769599 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86c8158bc0 0x7f86c80cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.454+0000 7f86b4ff9700 1 --2- 172.21.15.40:0/1072769599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86c814f8c0 0x7f86c8150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.454+0000 7f86b4ff9700 1 --2- 172.21.15.40:0/1072769599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c80724e0 0x7f86c80c47f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.454+0000 7f86b4ff9700 1 -- 172.21.15.40:0/1072769599 >> 172.21.15.40:0/1072769599 conn(0x7f86c8066090 msgr2=0x7f86c80c05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:06.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.454+0000 7f86b4ff9700 1 -- 172.21.15.40:0/1072769599 shutdown_connections 2024-09-13T17:51:06.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.454+0000 7f86b4ff9700 1 -- 172.21.15.40:0/1072769599 wait complete. 2024-09-13T17:51:06.469 INFO:teuthology.orchestra.run.smithi040.stdout:60129542153 2024-09-13T17:51:06.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.454+0000 7f9ae311c700 1 -- 172.21.15.40:0/2619580716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f9adc0c57d0 con 0x7f9adc06dcb0 2024-09-13T17:51:06.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.454+0000 7f9ad2ffd700 1 -- 172.21.15.40:0/2619580716 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9ad8007720 con 0x7f9adc06dcb0 2024-09-13T17:51:06.470 INFO:teuthology.orchestra.run.smithi040.stdout:64424509450 2024-09-13T17:51:06.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.462+0000 7f0bb13fd700 1 -- 172.21.15.40:0/2937177590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0b8c005890 con 0x7f0bac158bc0 2024-09-13T17:51:06.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.462+0000 7f0ba8ff9700 1 -- 172.21.15.40:0/2937177590 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0ba405bcc0 con 0x7f0bac158bc0 2024-09-13T17:51:06.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.466+0000 7f0bb13fd700 1 -- 172.21.15.40:0/2937177590 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0b98066730 msgr2=0x7f0b98068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.466+0000 7f0bb13fd700 1 --2- 172.21.15.40:0/2937177590 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0b98066730 0x7f0b98068bf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c000f60 tx=0x7f0b9c007c40 comp rx=0 tx=0).stop 2024-09-13T17:51:06.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.466+0000 7f0bb13fd700 1 -- 172.21.15.40:0/2937177590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bac158bc0 msgr2=0x7f0bac1ea540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.466+0000 7f0bb13fd700 1 --2- 172.21.15.40:0/2937177590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bac158bc0 0x7f0bac1ea540 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0bac06b3c0 tx=0x7f0ba4004af0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.466+0000 7f0bb13fd700 1 -- 172.21.15.40:0/2937177590 shutdown_connections 2024-09-13T17:51:06.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.466+0000 7f0bb13fd700 1 --2- 172.21.15.40:0/2937177590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bac158bc0 0x7f0bac1ea540 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.466+0000 7f0bb13fd700 1 --2- 172.21.15.40:0/2937177590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bac14f8c0 0x7f0bac14dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.466+0000 7f0bb13fd700 1 --2- 172.21.15.40:0/2937177590 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0b98066730 0x7f0b98068bf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.466+0000 7f0bb13fd700 1 --2- 172.21.15.40:0/2937177590 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0bac0724e0 0x7f0bac155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.466+0000 7f0bb13fd700 1 -- 172.21.15.40:0/2937177590 >> 172.21.15.40:0/2937177590 conn(0x7f0bac066090 msgr2=0x7f0bac070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:06.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.470+0000 7f9ad0ff9700 1 -- 172.21.15.40:0/2619580716 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9ac0066780 msgr2=0x7f9ac0068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.470+0000 7f9ad0ff9700 1 --2- 172.21.15.40:0/2619580716 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9ac0066780 0x7f9ac0068c40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9adc157a50 tx=0x7f9acc008040 comp rx=0 tx=0).stop 2024-09-13T17:51:06.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.470+0000 7f9ad0ff9700 1 -- 172.21.15.40:0/2619580716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9adc06dcb0 msgr2=0x7f9adc1f2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.470+0000 7f9ad0ff9700 1 --2- 172.21.15.40:0/2619580716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9adc06dcb0 0x7f9adc1f2ad0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800c580 tx=0x7f9ad800c890 comp rx=0 tx=0).stop 2024-09-13T17:51:06.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.470+0000 7f9ad0ff9700 1 -- 172.21.15.40:0/2619580716 shutdown_connections 2024-09-13T17:51:06.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.470+0000 7f9ad0ff9700 1 --2- 172.21.15.40:0/2619580716 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9ac0066780 0x7f9ac0068c40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.470+0000 7f9ad0ff9700 1 --2- 172.21.15.40:0/2619580716 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9adc0b00b0 0x7f9adc0c33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.470+0000 7f9ad0ff9700 1 --2- 172.21.15.40:0/2619580716 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9adc06e090 0x7f9adc071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.466+0000 7f0bb13fd700 1 -- 172.21.15.40:0/2937177590 shutdown_connections 2024-09-13T17:51:06.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.466+0000 7f0bb13fd700 1 -- 172.21.15.40:0/2937177590 wait complete. 2024-09-13T17:51:06.475 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.5 2024-09-13T17:51:06.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.470+0000 7f9ad0ff9700 1 --2- 172.21.15.40:0/2619580716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9adc06dcb0 0x7f9adc1f2ad0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.470+0000 7f9ad0ff9700 1 -- 172.21.15.40:0/2619580716 >> 172.21.15.40:0/2619580716 conn(0x7f9adc066090 msgr2=0x7f9adc157fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:06.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.470+0000 7f9ad0ff9700 1 -- 172.21.15.40:0/2619580716 shutdown_connections 2024-09-13T17:51:06.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.470+0000 7f9ad0ff9700 1 -- 172.21.15.40:0/2619580716 wait complete. 2024-09-13T17:51:06.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.470+0000 7f0e1b54e700 1 -- 172.21.15.40:0/2914058291 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f0e140c3300 con 0x7f0e141566b0 2024-09-13T17:51:06.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.470+0000 7f0e0affd700 1 -- 172.21.15.40:0/2914058291 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0e140c3300 con 0x7f0e141566b0 2024-09-13T17:51:06.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.470+0000 7f0da9d7e700 1 -- 172.21.15.40:0/688391132 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0da41508b0 con 0x7f0da4156c10 2024-09-13T17:51:06.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.470+0000 7f44b083a700 1 -- 172.21.15.40:0/526032025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f448c005890 con 0x7f44ac0c7f60 2024-09-13T17:51:06.478 INFO:teuthology.orchestra.run.smithi040.stdout:60129542153 2024-09-13T17:51:06.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.474+0000 7f0da17fa700 1 -- 172.21.15.40:0/688391132 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0d94017020 con 0x7f0da4156c10 2024-09-13T17:51:06.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.474+0000 7f4493fff700 1 -- 172.21.15.40:0/526032025 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f44a0007720 con 0x7f44ac0c7f60 2024-09-13T17:51:06.479 INFO:teuthology.orchestra.run.smithi040.stdout:64424509450 2024-09-13T17:51:06.479 INFO:teuthology.orchestra.run.smithi040.stdout:64424509450 2024-09-13T17:51:06.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.474+0000 7f0e1b54e700 1 -- 172.21.15.40:0/2914058291 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0df8066730 msgr2=0x7f0df8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.474+0000 7f0e1b54e700 1 --2- 172.21.15.40:0/2914058291 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0df8066730 0x7f0df8068bf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0e10001d30 tx=0x7f0e10009040 comp rx=0 tx=0).stop 2024-09-13T17:51:06.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.474+0000 7f0e1b54e700 1 -- 172.21.15.40:0/2914058291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e141566b0 msgr2=0x7f0e140ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.474+0000 7f0e1b54e700 1 --2- 172.21.15.40:0/2914058291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e141566b0 0x7f0e140ca770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c019000 tx=0x7f0e0c004af0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.474+0000 7f0e1b54e700 1 -- 172.21.15.40:0/2914058291 shutdown_connections 2024-09-13T17:51:06.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.474+0000 7f0e1b54e700 1 --2- 172.21.15.40:0/2914058291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e140cacb0 0x7f0e140c35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.474+0000 7f0e1b54e700 1 --2- 172.21.15.40:0/2914058291 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e141566b0 0x7f0e140ca770 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.474+0000 7f0e1b54e700 1 --2- 172.21.15.40:0/2914058291 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0df8066730 0x7f0df8068bf0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.474+0000 7f0e1b54e700 1 --2- 172.21.15.40:0/2914058291 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e1406d540 0x7f0e140aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.474+0000 7f0e1b54e700 1 -- 172.21.15.40:0/2914058291 >> 172.21.15.40:0/2914058291 conn(0x7f0e14066090 msgr2=0x7f0e140c6450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:06.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.474+0000 7f0e1b54e700 1 -- 172.21.15.40:0/2914058291 shutdown_connections 2024-09-13T17:51:06.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.474+0000 7f0e1b54e700 1 -- 172.21.15.40:0/2914058291 wait complete. 2024-09-13T17:51:06.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f0da9d7e700 1 -- 172.21.15.40:0/688391132 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0d90066780 msgr2=0x7f0d90068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f44b083a700 1 -- 172.21.15.40:0/526032025 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4498066780 msgr2=0x7f4498068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f44b083a700 1 --2- 172.21.15.40:0/526032025 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4498066780 0x7f4498068c40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f449c009950 tx=0x7f449c008040 comp rx=0 tx=0).stop 2024-09-13T17:51:06.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f44b083a700 1 -- 172.21.15.40:0/526032025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44ac0c7f60 msgr2=0x7f44ac1f67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f44b083a700 1 --2- 172.21.15.40:0/526032025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44ac0c7f60 0x7f44ac1f67e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f44a000c750 tx=0x7f44a000cb10 comp rx=0 tx=0).stop 2024-09-13T17:51:06.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f0da9d7e700 1 --2- 172.21.15.40:0/688391132 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0d90066780 0x7f0d90068c40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0d98000c00 tx=0x7f0d98008030 comp rx=0 tx=0).stop 2024-09-13T17:51:06.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f0da9d7e700 1 -- 172.21.15.40:0/688391132 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0da4156c10 msgr2=0x7f0da41e2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f0da9d7e700 1 --2- 172.21.15.40:0/688391132 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0da4156c10 0x7f0da41e2690 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400e7f0 tx=0x7f0d9400eb00 comp rx=0 tx=0).stop 2024-09-13T17:51:06.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f0da9d7e700 1 -- 172.21.15.40:0/688391132 shutdown_connections 2024-09-13T17:51:06.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f0da9d7e700 1 --2- 172.21.15.40:0/688391132 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0d90066780 0x7f0d90068c40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f0da9d7e700 1 --2- 172.21.15.40:0/688391132 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0da4156c10 0x7f0da41e2690 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f44b083a700 1 -- 172.21.15.40:0/526032025 shutdown_connections 2024-09-13T17:51:06.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f44b083a700 1 --2- 172.21.15.40:0/526032025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44ac0c8340 0x7f44ac071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f44b083a700 1 --2- 172.21.15.40:0/526032025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44ac0c7f60 0x7f44ac1f67e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f0da9d7e700 1 --2- 172.21.15.40:0/688391132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0da40c7010 0x7f0da4146520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f44b083a700 1 --2- 172.21.15.40:0/526032025 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4498066780 0x7f4498068c40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f0da9d7e700 1 --2- 172.21.15.40:0/688391132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0da4071110 0x7f0da413f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f44b083a700 1 --2- 172.21.15.40:0/526032025 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44ac0c7250 0x7f44ac1530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f0da9d7e700 1 -- 172.21.15.40:0/688391132 >> 172.21.15.40:0/688391132 conn(0x7f0da4066090 msgr2=0x7f0da40c0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:06.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f44b083a700 1 -- 172.21.15.40:0/526032025 >> 172.21.15.40:0/526032025 conn(0x7f44ac066090 msgr2=0x7f44ac15c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:06.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f44b083a700 1 -- 172.21.15.40:0/526032025 shutdown_connections 2024-09-13T17:51:06.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f44b083a700 1 -- 172.21.15.40:0/526032025 wait complete. 2024-09-13T17:51:06.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f0da9d7e700 1 -- 172.21.15.40:0/688391132 shutdown_connections 2024-09-13T17:51:06.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f0da9d7e700 1 -- 172.21.15.40:0/688391132 wait complete. 2024-09-13T17:51:06.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f49bee2d700 1 -- 172.21.15.40:0/1730148452 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f49b815bdd0 con 0x7f49b80cf1a0 2024-09-13T17:51:06.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.478+0000 7f49b57fa700 1 -- 172.21.15.40:0/1730148452 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f49ac0045d0 con 0x7f49b80cf1a0 2024-09-13T17:51:06.487 INFO:teuthology.orchestra.run.smithi040.stdout:64424509451 2024-09-13T17:51:06.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.482+0000 7f49bee2d700 1 -- 172.21.15.40:0/1730148452 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f49a4066730 msgr2=0x7f49a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.482+0000 7f49bee2d700 1 --2- 172.21.15.40:0/1730148452 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f49a4066730 0x7f49a4068bf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009bf0 tx=0x7f49a8009240 comp rx=0 tx=0).stop 2024-09-13T17:51:06.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.482+0000 7f49bee2d700 1 -- 172.21.15.40:0/1730148452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49b80cf1a0 msgr2=0x7f49b8156790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.482+0000 7f49bee2d700 1 --2- 172.21.15.40:0/1730148452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49b80cf1a0 0x7f49b8156790 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f49ac016000 tx=0x7f49ac004770 comp rx=0 tx=0).stop 2024-09-13T17:51:06.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.482+0000 7f49bee2d700 1 -- 172.21.15.40:0/1730148452 shutdown_connections 2024-09-13T17:51:06.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.482+0000 7f49bee2d700 1 --2- 172.21.15.40:0/1730148452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49b80cf580 0x7f49b81ef140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.482+0000 7f49bee2d700 1 --2- 172.21.15.40:0/1730148452 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f49a4066730 0x7f49a4068bf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.482+0000 7f49bee2d700 1 --2- 172.21.15.40:0/1730148452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49b80cf1a0 0x7f49b8156790 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.482+0000 7f49bee2d700 1 --2- 172.21.15.40:0/1730148452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49b80cc250 0x7f49b80c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.482+0000 7f49bee2d700 1 -- 172.21.15.40:0/1730148452 >> 172.21.15.40:0/1730148452 conn(0x7f49b8066090 msgr2=0x7f49b80c7770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:06.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.482+0000 7f49bee2d700 1 -- 172.21.15.40:0/1730148452 shutdown_connections 2024-09-13T17:51:06.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.482+0000 7f49bee2d700 1 -- 172.21.15.40:0/1730148452 wait complete. 2024-09-13T17:51:06.490 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.2 2024-09-13T17:51:06.490 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.6 2024-09-13T17:51:06.497 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.1 2024-09-13T17:51:06.499 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.3 2024-09-13T17:51:06.500 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.7 2024-09-13T17:51:06.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.494+0000 7f1ad4ff9700 1 -- 172.21.15.40:0/2656959874 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f1ae80cb350 con 0x7f1ae8158bc0 2024-09-13T17:51:06.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.494+0000 7f1ad6ffd700 1 -- 172.21.15.40:0/2656959874 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1ae0018020 con 0x7f1ae8158bc0 2024-09-13T17:51:06.501 INFO:teuthology.orchestra.run.smithi040.stdout:60129542153 2024-09-13T17:51:06.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.498+0000 7f1ad4ff9700 1 -- 172.21.15.40:0/2656959874 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1ac80667b0 msgr2=0x7f1ac8068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.498+0000 7f1ad4ff9700 1 --2- 172.21.15.40:0/2656959874 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1ac80667b0 0x7f1ac8068c70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009c60 tx=0x7f1ad8008030 comp rx=0 tx=0).stop 2024-09-13T17:51:06.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.498+0000 7f1ad4ff9700 1 -- 172.21.15.40:0/2656959874 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ae8158bc0 msgr2=0x7f1ae80c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:06.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.498+0000 7f1ad4ff9700 1 --2- 172.21.15.40:0/2656959874 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ae8158bc0 0x7f1ae80c49d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000efc0 tx=0x7f1ae000c3a0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.498+0000 7f1ad4ff9700 1 -- 172.21.15.40:0/2656959874 shutdown_connections 2024-09-13T17:51:06.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.498+0000 7f1ad4ff9700 1 --2- 172.21.15.40:0/2656959874 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1ac80667b0 0x7f1ac8068c70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.498+0000 7f1ad4ff9700 1 --2- 172.21.15.40:0/2656959874 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ae8158bc0 0x7f1ae80c49d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.498+0000 7f1ad4ff9700 1 --2- 172.21.15.40:0/2656959874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ae80c78d0 0x7f1ae80706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.498+0000 7f1ad4ff9700 1 --2- 172.21.15.40:0/2656959874 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ae80724e0 0x7f1ae80cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:06.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.498+0000 7f1ad4ff9700 1 -- 172.21.15.40:0/2656959874 >> 172.21.15.40:0/2656959874 conn(0x7f1ae8066090 msgr2=0x7f1ae8159080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:06.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.498+0000 7f1ad4ff9700 1 -- 172.21.15.40:0/2656959874 shutdown_connections 2024-09-13T17:51:06.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:06.498+0000 7f1ad4ff9700 1 -- 172.21.15.40:0/2656959874 wait complete. 2024-09-13T17:51:06.505 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509451 for osd.0 2024-09-13T17:51:06.513 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.4 2024-09-13T17:51:07.477 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T17:51:07.490 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T17:51:07.491 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T17:51:07.497 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T17:51:07.500 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T17:51:07.500 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T17:51:07.506 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T17:51:07.513 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T17:51:07.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.630+0000 7fbf44317700 1 Processor -- start 2024-09-13T17:51:07.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.630+0000 7fbf44317700 1 -- start start 2024-09-13T17:51:07.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.630+0000 7fbf44317700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbf3c14f8c0 0x7fbf3c0c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.630+0000 7fbf44317700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbf3c0724e0 0x7fbf3c06d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.630+0000 7fbf44317700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf3c0c6890 con 0x7fbf3c158bc0 2024-09-13T17:51:07.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.630+0000 7fbf44317700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3c0c7700 con 0x7fbf3c14f8c0 2024-09-13T17:51:07.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.630+0000 7fbf44317700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3c0c5f30 con 0x7fbf3c0724e0 2024-09-13T17:51:07.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.630+0000 7fbf428b4700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbf3c158bc0 0x7fbf3c06a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46144/0 (socket says 172.21.15.40:46144) 2024-09-13T17:51:07.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.630+0000 7fbf428b4700 1 -- 172.21.15.40:0/3215330309 learned_addr learned my addr 172.21.15.40:0/3215330309 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:07.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.630+0000 7fbf420b3700 1 --2- 172.21.15.40:0/3215330309 >> v2:172.21.15.40:3300/0 conn(0x7fbf3c14f8c0 0x7fbf3c0c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.630+0000 7fbf420b3700 1 -- 172.21.15.40:0/3215330309 >> v2:172.21.15.43:3300/0 conn(0x7fbf3c0724e0 msgr2=0x7fbf3c06d2d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T17:51:07.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.630+0000 7fbf420b3700 1 --2- 172.21.15.40:0/3215330309 >> v2:172.21.15.43:3300/0 conn(0x7fbf3c0724e0 0x7fbf3c06d2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.630+0000 7fbf420b3700 1 -- 172.21.15.40:0/3215330309 >> v1:172.21.15.40:6789/0 conn(0x7fbf3c158bc0 legacy=0x7fbf3c06a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.630+0000 7fbf420b3700 1 -- 172.21.15.40:0/3215330309 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c0c1320 con 0x7fbf3c14f8c0 2024-09-13T17:51:07.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.630+0000 7fbf420b3700 1 --2- 172.21.15.40:0/3215330309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c14f8c0 0x7fbf3c0c71c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0c0580 tx=0x7fbf38009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f272a94541c7496b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.630+0000 7fbf410b1700 1 -- 172.21.15.40:0/3215330309 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3800c900 con 0x7fbf3c14f8c0 2024-09-13T17:51:07.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.630+0000 7fbf410b1700 1 -- 172.21.15.40:0/3215330309 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3800ca60 con 0x7fbf3c14f8c0 2024-09-13T17:51:07.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.630+0000 7fbf410b1700 1 -- 172.21.15.40:0/3215330309 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3800cdf0 con 0x7fbf3c14f8c0 2024-09-13T17:51:07.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.634+0000 7fbf44317700 1 -- 172.21.15.40:0/3215330309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c14f8c0 msgr2=0x7fbf3c0c71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.634+0000 7fbf44317700 1 --2- 172.21.15.40:0/3215330309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c14f8c0 0x7fbf3c0c71c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0c0580 tx=0x7fbf38009b30 comp rx=0 tx=0).stop 2024-09-13T17:51:07.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.634+0000 7fbf44317700 1 -- 172.21.15.40:0/3215330309 shutdown_connections 2024-09-13T17:51:07.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.634+0000 7fbf44317700 1 --2- 172.21.15.40:0/3215330309 >> v2:172.21.15.43:3300/0 conn(0x7fbf3c0724e0 0x7fbf3c06d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.634+0000 7fbf44317700 1 --2- 172.21.15.40:0/3215330309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c14f8c0 0x7fbf3c0c71c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.634+0000 7fbf44317700 1 -- 172.21.15.40:0/3215330309 >> 172.21.15.40:0/3215330309 conn(0x7fbf3c066090 msgr2=0x7fbf3c0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:07.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.634+0000 7fbf44317700 1 -- 172.21.15.40:0/3215330309 shutdown_connections 2024-09-13T17:51:07.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.634+0000 7fbf44317700 1 -- 172.21.15.40:0/3215330309 wait complete. 2024-09-13T17:51:07.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.634+0000 7fbf44317700 1 Processor -- start 2024-09-13T17:51:07.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.634+0000 7fbf44317700 1 -- start start 2024-09-13T17:51:07.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf44317700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf3c0724e0 0x7fbf3c1fd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf44317700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c14f8c0 0x7fbf3c151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf44317700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf3c158bc0 0x7fbf3c0c4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf44317700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0cafa0 con 0x7fbf3c14f8c0 2024-09-13T17:51:07.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf44317700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0cb280 con 0x7fbf3c158bc0 2024-09-13T17:51:07.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf44317700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c150d80 con 0x7fbf3c0724e0 2024-09-13T17:51:07.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf418b2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c14f8c0 0x7fbf3c151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf418b2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c14f8c0 0x7fbf3c151b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58090/0 (socket says 172.21.15.40:58090) 2024-09-13T17:51:07.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf418b2700 1 -- 172.21.15.40:0/1503967791 learned_addr learned my addr 172.21.15.40:0/1503967791 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:07.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf420b3700 1 --2- 172.21.15.40:0/1503967791 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf3c0724e0 0x7fbf3c1fd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf428b4700 1 --2- 172.21.15.40:0/1503967791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf3c158bc0 0x7fbf3c0c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf418b2700 1 -- 172.21.15.40:0/1503967791 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf3c0724e0 msgr2=0x7fbf3c1fd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf418b2700 1 --2- 172.21.15.40:0/1503967791 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf3c0724e0 0x7fbf3c1fd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf418b2700 1 -- 172.21.15.40:0/1503967791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf3c158bc0 msgr2=0x7fbf3c0c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf418b2700 1 --2- 172.21.15.40:0/1503967791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf3c158bc0 0x7fbf3c0c4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf418b2700 1 -- 172.21.15.40:0/1503967791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf380094b0 con 0x7fbf3c14f8c0 2024-09-13T17:51:07.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf428b4700 1 --2- 172.21.15.40:0/1503967791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf3c158bc0 0x7fbf3c0c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:07.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf418b2700 1 --2- 172.21.15.40:0/1503967791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c14f8c0 0x7fbf3c151b40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c00b520 tx=0x7fbf2c00b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf32ffd700 1 -- 172.21.15.40:0/1503967791 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c011830 con 0x7fbf3c14f8c0 2024-09-13T17:51:07.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf32ffd700 1 -- 172.21.15.40:0/1503967791 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2c0119b0 con 0x7fbf3c14f8c0 2024-09-13T17:51:07.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf44317700 1 -- 172.21.15.40:0/1503967791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3c151060 con 0x7fbf3c14f8c0 2024-09-13T17:51:07.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf44317700 1 -- 172.21.15.40:0/1503967791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3c151620 con 0x7fbf3c14f8c0 2024-09-13T17:51:07.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.638+0000 7fbf32ffd700 1 -- 172.21.15.40:0/1503967791 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c011d60 con 0x7fbf3c14f8c0 2024-09-13T17:51:07.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.642+0000 7fbf32ffd700 1 -- 172.21.15.40:0/1503967791 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbf2c01a400 con 0x7fbf3c14f8c0 2024-09-13T17:51:07.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.642+0000 7fbf44317700 1 -- 172.21.15.40:0/1503967791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3c072a10 con 0x7fbf3c14f8c0 2024-09-13T17:51:07.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.650+0000 7fbf32ffd700 1 --2- 172.21.15.40:0/1503967791 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbf20066780 0x7fbf20068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.650+0000 7fbf32ffd700 1 -- 172.21.15.40:0/1503967791 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbf2c0843b0 con 0x7fbf3c14f8c0 2024-09-13T17:51:07.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.650+0000 7fbf420b3700 1 --2- 172.21.15.40:0/1503967791 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbf20066780 0x7fbf20068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.650+0000 7fbf32ffd700 1 -- 172.21.15.40:0/1503967791 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbf2c086a30 con 0x7fbf3c14f8c0 2024-09-13T17:51:07.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.650+0000 7fbf420b3700 1 --2- 172.21.15.40:0/1503967791 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbf20066780 0x7fbf20068c40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbf380090b0 tx=0x7fbf3801b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7fafa8829700 1 Processor -- start 2024-09-13T17:51:07.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7fafa8829700 1 -- start start 2024-09-13T17:51:07.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7fafa8829700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fafa414f8c0 0x7fafa40c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7fafa8829700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fafa40c78d0 0x7fafa41607e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7fafa8829700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa40c6890 con 0x7fafa40724e0 2024-09-13T17:51:07.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7fafa8829700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafa40c7700 con 0x7fafa40c78d0 2024-09-13T17:51:07.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7fafa8829700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa40c5f30 con 0x7fafa414f8c0 2024-09-13T17:51:07.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7fafa359e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fafa40c78d0 0x7fafa41607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7fafa359e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fafa40c78d0 0x7fafa41607e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58100/0 (socket says 172.21.15.40:58100) 2024-09-13T17:51:07.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7fafa359e700 1 -- 172.21.15.40:0/3280887781 learned_addr learned my addr 172.21.15.40:0/3280887781 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:07.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7fafa2d9d700 1 --2- 172.21.15.40:0/3280887781 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fafa414f8c0 0x7fafa40c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7fafa359e700 1 -- 172.21.15.40:0/3280887781 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fafa414f8c0 msgr2=0x7fafa40c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:07.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7fafa359e700 1 --2- 172.21.15.40:0/3280887781 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fafa414f8c0 0x7fafa40c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7fafa359e700 1 -- 172.21.15.40:0/3280887781 >> v1:172.21.15.43:6789/0 conn(0x7fafa40724e0 legacy=0x7fafa406d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7fafa359e700 1 -- 172.21.15.40:0/3280887781 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa40c1320 con 0x7fafa40c78d0 2024-09-13T17:51:07.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7f6872400700 1 Processor -- start 2024-09-13T17:51:07.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7fce058eb700 1 Processor -- start 2024-09-13T17:51:07.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7f6872400700 1 -- start start 2024-09-13T17:51:07.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7fce058eb700 1 -- start start 2024-09-13T17:51:07.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7f59b8f9c700 1 Processor -- start 2024-09-13T17:51:07.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7f59b8f9c700 1 -- start start 2024-09-13T17:51:07.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f65030fd700 1 Processor -- start 2024-09-13T17:51:07.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.670+0000 7fafa359e700 1 --2- 172.21.15.40:0/3280887781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafa40c78d0 0x7fafa41607e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faf98009910 tx=0x7faf98009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7fced40fcaf10 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f65030fd700 1 -- start start 2024-09-13T17:51:07.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fafa1d9b700 1 -- 172.21.15.40:0/3280887781 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9800ba60 con 0x7fafa40c78d0 2024-09-13T17:51:07.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fafa1d9b700 1 -- 172.21.15.40:0/3280887781 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9800bbc0 con 0x7fafa40c78d0 2024-09-13T17:51:07.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fafa1d9b700 1 -- 172.21.15.40:0/3280887781 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf98009e60 con 0x7fafa40c78d0 2024-09-13T17:51:07.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f6872400700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f686c14fdd0 0x7f686c0c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f6872400700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f686c0c71a0 0x7f686c0c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f6872400700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f686c159eb0 con 0x7f686c0c6da0 2024-09-13T17:51:07.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f6872400700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f686c158fc0 con 0x7f686c14fdd0 2024-09-13T17:51:07.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f59b8f9c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f59b4163a20 0x7f59b4166d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f59b8f9c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f59b40c5b20 0x7f59b40c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f59b8f9c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b406ad40 con 0x7f59b40b0de0 2024-09-13T17:51:07.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f59b8f9c700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b406fd30 con 0x7f59b40c5b20 2024-09-13T17:51:07.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f59b8f9c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b4163e00 con 0x7f59b4163a20 2024-09-13T17:51:07.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f6872400700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f686c1592a0 con 0x7f686c0c71a0 2024-09-13T17:51:07.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f59b3fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f59b40c5b20 0x7f59b40c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f59b3fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f59b40c5b20 0x7f59b40c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58110/0 (socket says 172.21.15.40:58110) 2024-09-13T17:51:07.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f59b3fff700 1 -- 172.21.15.40:0/2493672133 learned_addr learned my addr 172.21.15.40:0/2493672133 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:07.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f68713fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f686c14fdd0 0x7f686c0c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f68713fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f686c14fdd0 0x7f686c0c7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58122/0 (socket says 172.21.15.40:58122) 2024-09-13T17:51:07.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f68713fe700 1 -- 172.21.15.40:0/1327780634 learned_addr learned my addr 172.21.15.40:0/1327780634 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:07.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fcdfeffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcdf80dcdc0 0x7fcdf804b890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46156/0 (socket says 172.21.15.40:46156) 2024-09-13T17:51:07.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f6871bff700 1 --2- 172.21.15.40:0/1327780634 >> v2:172.21.15.43:3300/0 conn(0x7f686c0c71a0 0x7f686c0c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f65030fd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f64fc072270 0x7f64fc159690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f65030fd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f64fc0c6390 0x7f64fc163f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f65030fd700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64fc072650 con 0x7f64fc0c7320 2024-09-13T17:51:07.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f65030fd700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64fc0c5f00 con 0x7f64fc0c6390 2024-09-13T17:51:07.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f65030fd700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc0c7130 con 0x7f64fc072270 2024-09-13T17:51:07.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f65028fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f64fc0c6390 0x7f64fc163f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fafa8829700 1 -- 172.21.15.40:0/3280887781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafa40c78d0 msgr2=0x7fafa41607e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fafa8829700 1 --2- 172.21.15.40:0/3280887781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafa40c78d0 0x7fafa41607e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faf98009910 tx=0x7faf98009c20 comp rx=0 tx=0).stop 2024-09-13T17:51:07.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f65020fb700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f64fc072270 0x7f64fc159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f65028fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f64fc0c6390 0x7f64fc163f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58136/0 (socket says 172.21.15.40:58136) 2024-09-13T17:51:07.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f68713fe700 1 -- 172.21.15.40:0/1327780634 >> v2:172.21.15.43:3300/0 conn(0x7f686c0c71a0 msgr2=0x7f686c0c0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:07.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f68713fe700 1 --2- 172.21.15.40:0/1327780634 >> v2:172.21.15.43:3300/0 conn(0x7f686c0c71a0 0x7f686c0c0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f68713fe700 1 -- 172.21.15.40:0/1327780634 >> v1:172.21.15.40:6789/0 conn(0x7f686c0c6da0 legacy=0x7f686c0c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:07.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f68713fe700 1 -- 172.21.15.40:0/1327780634 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686c159580 con 0x7f686c14fdd0 2024-09-13T17:51:07.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f59b3fff700 1 -- 172.21.15.40:0/2493672133 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f59b4163a20 msgr2=0x7f59b4166d00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T17:51:07.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f59b3fff700 1 --2- 172.21.15.40:0/2493672133 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f59b4163a20 0x7f59b4166d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f59b3fff700 1 -- 172.21.15.40:0/2493672133 >> v1:172.21.15.43:6789/0 conn(0x7f59b40b0de0 legacy=0x7f59b40c68a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T17:51:07.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f59b3fff700 1 -- 172.21.15.40:0/2493672133 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b406a4a0 con 0x7f59b40c5b20 2024-09-13T17:51:07.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f65020fb700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f64fc072270 0x7f64fc159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:49070/0 (socket says 172.21.15.40:49070) 2024-09-13T17:51:07.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f65020fb700 1 -- 172.21.15.40:0/3507149278 learned_addr learned my addr 172.21.15.40:0/3507149278 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:07.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fafa8829700 1 -- 172.21.15.40:0/3280887781 shutdown_connections 2024-09-13T17:51:07.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fafa8829700 1 --2- 172.21.15.40:0/3280887781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafa40c78d0 0x7fafa41607e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fafa8829700 1 --2- 172.21.15.40:0/3280887781 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fafa414f8c0 0x7fafa40c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fafa8829700 1 -- 172.21.15.40:0/3280887781 >> 172.21.15.40:0/3280887781 conn(0x7fafa4066090 msgr2=0x7fafa40664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:07.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f68713fe700 1 --2- 172.21.15.40:0/1327780634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c14fdd0 0x7f686c0c7cf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f686c073e90 tx=0x7f6868009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ccf8fd58a9a336b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f59b3fff700 1 --2- 172.21.15.40:0/2493672133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b40c5b20 0x7f59b40c5fd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f59a8008910 tx=0x7f59a800d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d15484aec3e968 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fafa8829700 1 -- 172.21.15.40:0/3280887781 shutdown_connections 2024-09-13T17:51:07.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fafa8829700 1 -- 172.21.15.40:0/3280887781 wait complete. 2024-09-13T17:51:07.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fce058eb700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcdf80071b0 0x7fcdf80ee440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fce058eb700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf804b4a0 con 0x7fcdf80dcdc0 2024-09-13T17:51:07.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fce058eb700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf80e6c50 con 0x7fcdf804ab50 2024-09-13T17:51:07.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fce058eb700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf8006680 con 0x7fcdf80071b0 2024-09-13T17:51:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f65020fb700 1 -- 172.21.15.40:0/3507149278 >> v2:172.21.15.40:3300/0 conn(0x7f64fc0c6390 msgr2=0x7f64fc163f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f65020fb700 1 --2- 172.21.15.40:0/3507149278 >> v2:172.21.15.40:3300/0 conn(0x7f64fc0c6390 0x7f64fc163f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f65020fb700 1 -- 172.21.15.40:0/3507149278 >> v1:172.21.15.43:6789/0 conn(0x7f64fc0c7320 legacy=0x7f64fc06d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f65020fb700 1 -- 172.21.15.40:0/3507149278 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc0c8450 con 0x7f64fc072270 2024-09-13T17:51:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fcdfeffd700 1 -- 172.21.15.40:0/2098941497 learned_addr learned my addr 172.21.15.40:0/2098941497 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:07.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f65028fc700 1 --2- 172.21.15.40:0/3507149278 >> v2:172.21.15.40:3300/0 conn(0x7f64fc0c6390 0x7f64fc163f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:07.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fcdff7fe700 1 --2- 172.21.15.40:0/2098941497 >> v2:172.21.15.43:3300/0 conn(0x7fcdf80071b0 0x7fcdf80ee440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f65020fb700 1 --2- 172.21.15.40:0/3507149278 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f64fc072270 0x7f64fc159690 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009910 tx=0x7f64ec009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5ca18ddd7273fc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f6863fff700 1 -- 172.21.15.40:0/1327780634 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868004230 con 0x7f686c14fdd0 2024-09-13T17:51:07.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fcdfdffb700 1 -- 172.21.15.40:0/2098941497 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512013016 0 0) 0x7fcdf804b4a0 con 0x7fcdf80dcdc0 2024-09-13T17:51:07.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fcdfdffb700 1 -- 172.21.15.40:0/2098941497 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcde80035d0 con 0x7fcdf80dcdc0 2024-09-13T17:51:07.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f6863fff700 1 -- 172.21.15.40:0/1327780634 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6868004390 con 0x7f686c14fdd0 2024-09-13T17:51:07.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f6863fff700 1 -- 172.21.15.40:0/1327780634 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68680036c0 con 0x7f686c14fdd0 2024-09-13T17:51:07.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fcdff7fe700 1 -- 172.21.15.40:0/2098941497 >> v1:172.21.15.43:6789/0 conn(0x7fcdf804ab50 legacy=0x7fcdf8006aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fcdff7fe700 1 -- 172.21.15.40:0/2098941497 >> v1:172.21.15.40:6789/0 conn(0x7fcdf80dcdc0 legacy=0x7fcdf804b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fcdff7fe700 1 -- 172.21.15.40:0/2098941497 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf80e6f30 con 0x7fcdf80071b0 2024-09-13T17:51:07.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fcdff7fe700 1 --2- 172.21.15.40:0/2098941497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf80071b0 0x7fcdf80ee440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009850 tx=0x7fcdf0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=576458de1c7caa69 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f59b27fc700 1 -- 172.21.15.40:0/2493672133 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a8010070 con 0x7f59b40c5b20 2024-09-13T17:51:07.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fafa8829700 1 Processor -- start 2024-09-13T17:51:07.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7fafa8829700 1 -- start start 2024-09-13T17:51:07.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f59b27fc700 1 -- 172.21.15.40:0/2493672133 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a800da40 con 0x7f59b40c5b20 2024-09-13T17:51:07.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.674+0000 7f59b27fc700 1 -- 172.21.15.40:0/2493672133 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a800ddd0 con 0x7f59b40c5b20 2024-09-13T17:51:07.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f65010f9700 1 -- 172.21.15.40:0/3507149278 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec011830 con 0x7f64fc072270 2024-09-13T17:51:07.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f65010f9700 1 -- 172.21.15.40:0/3507149278 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ec011990 con 0x7f64fc072270 2024-09-13T17:51:07.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f65010f9700 1 -- 172.21.15.40:0/3507149278 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec011d40 con 0x7f64fc072270 2024-09-13T17:51:07.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f6872400700 1 -- 172.21.15.40:0/1327780634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c14fdd0 msgr2=0x7f686c0c7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f6872400700 1 --2- 172.21.15.40:0/1327780634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c14fdd0 0x7f686c0c7cf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f686c073e90 tx=0x7f6868009bf0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f6872400700 1 -- 172.21.15.40:0/1327780634 shutdown_connections 2024-09-13T17:51:07.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f6872400700 1 --2- 172.21.15.40:0/1327780634 >> v2:172.21.15.43:3300/0 conn(0x7f686c0c71a0 0x7f686c0c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f6872400700 1 --2- 172.21.15.40:0/1327780634 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c14fdd0 0x7f686c0c7cf0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f6872400700 1 -- 172.21.15.40:0/1327780634 >> 172.21.15.40:0/1327780634 conn(0x7f686c066090 msgr2=0x7f686c0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:07.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f6872400700 1 -- 172.21.15.40:0/1327780634 shutdown_connections 2024-09-13T17:51:07.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f6872400700 1 -- 172.21.15.40:0/1327780634 wait complete. 2024-09-13T17:51:07.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f59b8f9c700 1 -- 172.21.15.40:0/2493672133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b40c5b20 msgr2=0x7f59b40c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f59b8f9c700 1 --2- 172.21.15.40:0/2493672133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b40c5b20 0x7f59b40c5fd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f59a8008910 tx=0x7f59a800d330 comp rx=0 tx=0).stop 2024-09-13T17:51:07.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fcdfdffb700 1 -- 172.21.15.40:0/2098941497 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0011830 con 0x7fcdf80071b0 2024-09-13T17:51:07.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fcdfdffb700 1 -- 172.21.15.40:0/2098941497 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf0011990 con 0x7fcdf80071b0 2024-09-13T17:51:07.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fcdfdffb700 1 -- 172.21.15.40:0/2098941497 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0011d20 con 0x7fcdf80071b0 2024-09-13T17:51:07.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fafa8829700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fafa40724e0 0x7fafa41ee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fafa8829700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafa40c78d0 0x7fafa4150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fafa8829700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafa414f8c0 0x7fafa4150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fafa8829700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa41526b0 con 0x7fafa40c78d0 2024-09-13T17:51:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fafa8829700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa414ff70 con 0x7fafa414f8c0 2024-09-13T17:51:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fafa8829700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa41eeb80 con 0x7fafa40724e0 2024-09-13T17:51:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f59b8f9c700 1 -- 172.21.15.40:0/2493672133 shutdown_connections 2024-09-13T17:51:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f59b8f9c700 1 --2- 172.21.15.40:0/2493672133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b40c5b20 0x7f59b40c5fd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f59b8f9c700 1 --2- 172.21.15.40:0/2493672133 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f59b4163a20 0x7f59b4166d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f59b8f9c700 1 -- 172.21.15.40:0/2493672133 >> 172.21.15.40:0/2493672133 conn(0x7f59b40cad00 msgr2=0x7f59b40cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fce058eb700 1 -- 172.21.15.40:0/2098941497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf80071b0 msgr2=0x7fcdf80ee440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fce058eb700 1 --2- 172.21.15.40:0/2098941497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf80071b0 0x7fcdf80ee440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009850 tx=0x7fcdf0009b60 comp rx=0 tx=0).stop 2024-09-13T17:51:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fafa359e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafa414f8c0 0x7fafa4150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f59b8f9c700 1 -- 172.21.15.40:0/2493672133 shutdown_connections 2024-09-13T17:51:07.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fafa359e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafa414f8c0 0x7fafa4150940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:37158/0 (socket says 172.21.15.40:37158) 2024-09-13T17:51:07.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fafa359e700 1 -- 172.21.15.40:0/1317140617 learned_addr learned my addr 172.21.15.40:0/1317140617 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:07.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fce058eb700 1 -- 172.21.15.40:0/2098941497 shutdown_connections 2024-09-13T17:51:07.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fce058eb700 1 --2- 172.21.15.40:0/2098941497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf80071b0 0x7fcdf80ee440 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fce058eb700 1 -- 172.21.15.40:0/2098941497 >> 172.21.15.40:0/2098941497 conn(0x7fcdf8050120 msgr2=0x7fcdf8050530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:07.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fafa259c700 1 --2- 172.21.15.40:0/1317140617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafa40c78d0 0x7fafa4150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f65030fd700 1 -- 172.21.15.40:0/3507149278 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f64fc072270 msgr2=0x7f64fc159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f65030fd700 1 --2- 172.21.15.40:0/3507149278 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f64fc072270 0x7f64fc159690 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009910 tx=0x7f64ec009c20 comp rx=0 tx=0).stop 2024-09-13T17:51:07.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fafa359e700 1 -- 172.21.15.40:0/1317140617 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fafa40724e0 msgr2=0x7fafa41ee2c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:51:07.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fafa359e700 1 --2- 172.21.15.40:0/1317140617 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fafa40724e0 0x7fafa41ee2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fafa359e700 1 -- 172.21.15.40:0/1317140617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafa40c78d0 msgr2=0x7fafa4150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fafa359e700 1 --2- 172.21.15.40:0/1317140617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafa40c78d0 0x7fafa4150400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fafa359e700 1 -- 172.21.15.40:0/1317140617 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf98009570 con 0x7fafa414f8c0 2024-09-13T17:51:07.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f6872400700 1 Processor -- start 2024-09-13T17:51:07.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fce058eb700 1 -- 172.21.15.40:0/2098941497 shutdown_connections 2024-09-13T17:51:07.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f65030fd700 1 -- 172.21.15.40:0/3507149278 shutdown_connections 2024-09-13T17:51:07.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f65030fd700 1 --2- 172.21.15.40:0/3507149278 >> v2:172.21.15.40:3300/0 conn(0x7f64fc0c6390 0x7f64fc163f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f65030fd700 1 --2- 172.21.15.40:0/3507149278 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f64fc072270 0x7f64fc159690 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f65030fd700 1 -- 172.21.15.40:0/3507149278 >> 172.21.15.40:0/3507149278 conn(0x7f64fc066090 msgr2=0x7f64fc0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:07.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fce058eb700 1 -- 172.21.15.40:0/2098941497 wait complete. 2024-09-13T17:51:07.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f6872400700 1 -- start start 2024-09-13T17:51:07.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f59b8f9c700 1 -- 172.21.15.40:0/2493672133 wait complete. 2024-09-13T17:51:07.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f59b8f9c700 1 Processor -- start 2024-09-13T17:51:07.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f59b8f9c700 1 -- start start 2024-09-13T17:51:07.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f6872400700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c0c6da0 0x7f686c072100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f6872400700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f686c0c71a0 0x7f686c1526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f6872400700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f686c14fdd0 0x7f686c152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f68713fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c0c6da0 0x7f686c072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f68713fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c0c6da0 0x7f686c072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58158/0 (socket says 172.21.15.40:58158) 2024-09-13T17:51:07.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f68713fe700 1 -- 172.21.15.40:0/433452979 learned_addr learned my addr 172.21.15.40:0/433452979 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:07.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f6872400700 1 -- 172.21.15.40:0/433452979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c151a20 con 0x7f686c0c6da0 2024-09-13T17:51:07.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f6872400700 1 -- 172.21.15.40:0/433452979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c150600 con 0x7f686c14fdd0 2024-09-13T17:51:07.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f6872400700 1 -- 172.21.15.40:0/433452979 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686c152210 con 0x7f686c0c71a0 2024-09-13T17:51:07.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f6870bfd700 1 --2- 172.21.15.40:0/433452979 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f686c0c71a0 0x7f686c1526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f6871bff700 1 --2- 172.21.15.40:0/433452979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f686c14fdd0 0x7f686c152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fce058eb700 1 Processor -- start 2024-09-13T17:51:07.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fafa359e700 1 --2- 172.21.15.40:0/1317140617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafa414f8c0 0x7fafa4150940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fafa406b3c0 tx=0x7faf98004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7faf8b7fe700 1 -- 172.21.15.40:0/1317140617 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf980036c0 con 0x7fafa414f8c0 2024-09-13T17:51:07.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fafa8829700 1 -- 172.21.15.40:0/1317140617 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa41eedf0 con 0x7fafa414f8c0 2024-09-13T17:51:07.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7faf8b7fe700 1 -- 172.21.15.40:0/1317140617 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf98003820 con 0x7fafa414f8c0 2024-09-13T17:51:07.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7faf8b7fe700 1 -- 172.21.15.40:0/1317140617 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf98003bd0 con 0x7fafa414f8c0 2024-09-13T17:51:07.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fafa8829700 1 -- 172.21.15.40:0/1317140617 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa4152be0 con 0x7fafa414f8c0 2024-09-13T17:51:07.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f59b8f9c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59b40b0de0 0x7f59b420cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f59b8f9c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b40c5b20 0x7f59b40c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f59b8f9c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f59b4163a20 0x7f59b41653e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f59b8f9c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b4164fc0 con 0x7f59b40c5b20 2024-09-13T17:51:07.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f59b8f9c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b40ca8e0 con 0x7f59b40b0de0 2024-09-13T17:51:07.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f59b8f9c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b4164080 con 0x7f59b4163a20 2024-09-13T17:51:07.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7fce058eb700 1 -- start start 2024-09-13T17:51:07.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f42a2246700 1 Processor -- start 2024-09-13T17:51:07.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f65030fd700 1 -- 172.21.15.40:0/3507149278 shutdown_connections 2024-09-13T17:51:07.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f68713fe700 1 -- 172.21.15.40:0/433452979 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f686c0c71a0 msgr2=0x7f686c1526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f68713fe700 1 --2- 172.21.15.40:0/433452979 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f686c0c71a0 0x7f686c1526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f68713fe700 1 -- 172.21.15.40:0/433452979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f686c14fdd0 msgr2=0x7f686c152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f68713fe700 1 --2- 172.21.15.40:0/433452979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f686c14fdd0 0x7f686c152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.678+0000 7f68713fe700 1 -- 172.21.15.40:0/433452979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6868009570 con 0x7f686c0c6da0 2024-09-13T17:51:07.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f42a2246700 1 -- start start 2024-09-13T17:51:07.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f59b2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b40c5b20 0x7f59b40c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f59b2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b40c5b20 0x7f59b40c9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58174/0 (socket says 172.21.15.40:58174) 2024-09-13T17:51:07.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f59b2ffd700 1 -- 172.21.15.40:0/509865854 learned_addr learned my addr 172.21.15.40:0/509865854 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:07.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f59b3fff700 1 --2- 172.21.15.40:0/509865854 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f59b4163a20 0x7f59b41653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7fafa8829700 1 -- 172.21.15.40:0/1317140617 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa4072a10 con 0x7fafa414f8c0 2024-09-13T17:51:07.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f68713fe700 1 --2- 172.21.15.40:0/433452979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c0c6da0 0x7f686c072100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6868004000 tx=0x7f6868004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f6861ffb700 1 -- 172.21.15.40:0/433452979 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868011630 con 0x7f686c0c6da0 2024-09-13T17:51:07.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f6861ffb700 1 -- 172.21.15.40:0/433452979 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68680117b0 con 0x7f686c0c6da0 2024-09-13T17:51:07.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f6861ffb700 1 -- 172.21.15.40:0/433452979 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868011b60 con 0x7f686c0c6da0 2024-09-13T17:51:07.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f59b2ffd700 1 -- 172.21.15.40:0/509865854 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f59b4163a20 msgr2=0x7f59b41653e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f59b2ffd700 1 --2- 172.21.15.40:0/509865854 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f59b4163a20 0x7f59b41653e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f59b2ffd700 1 -- 172.21.15.40:0/509865854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59b40b0de0 msgr2=0x7f59b420cfc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T17:51:07.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f59b2ffd700 1 --2- 172.21.15.40:0/509865854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59b40b0de0 0x7f59b420cfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f59b2ffd700 1 -- 172.21.15.40:0/509865854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a80085f0 con 0x7f59b40c5b20 2024-09-13T17:51:07.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f6872400700 1 -- 172.21.15.40:0/433452979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686c0c9c80 con 0x7f686c0c6da0 2024-09-13T17:51:07.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f6872400700 1 -- 172.21.15.40:0/433452979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686c151ea0 con 0x7f686c0c6da0 2024-09-13T17:51:07.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7fce058eb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf80071b0 0x7fcdf80dae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7fce058eb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf804ab50 0x7fcdf817c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7fce058eb700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcdf80dcdc0 0x7fcdf817c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7fce058eb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf80d6d50 con 0x7fcdf804ab50 2024-09-13T17:51:07.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7fce058eb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf816e1b0 con 0x7fcdf80071b0 2024-09-13T17:51:07.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7fce058eb700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf80d3a30 con 0x7fcdf80dcdc0 2024-09-13T17:51:07.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7fcdfe7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf804ab50 0x7fcdf817c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7fcdfe7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf804ab50 0x7fcdf817c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58176/0 (socket says 172.21.15.40:58176) 2024-09-13T17:51:07.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7fcdfe7fc700 1 -- 172.21.15.40:0/3383528931 learned_addr learned my addr 172.21.15.40:0/3383528931 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:07.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f59b2ffd700 1 --2- 172.21.15.40:0/509865854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b40c5b20 0x7f59b40c9c50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00b9e0 tx=0x7f59ac00bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f599bfff700 1 -- 172.21.15.40:0/509865854 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac00c9b0 con 0x7f59b40c5b20 2024-09-13T17:51:07.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f599bfff700 1 -- 172.21.15.40:0/509865854 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ac00cb10 con 0x7f59b40c5b20 2024-09-13T17:51:07.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f599bfff700 1 -- 172.21.15.40:0/509865854 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac014440 con 0x7f59b40c5b20 2024-09-13T17:51:07.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7fcdfeffd700 1 --2- 172.21.15.40:0/3383528931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf80071b0 0x7fcdf80dae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f42a2246700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f429c0caf30 0x7f429c15dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f42a2246700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429c0c42f0 0x7f429c15e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f42a2246700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f429c161130 con 0x7f429c157e40 2024-09-13T17:51:07.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f42a2246700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f429c158220 con 0x7f429c0caf30 2024-09-13T17:51:07.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f42a2246700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f429c0cb490 con 0x7f429c0c42f0 2024-09-13T17:51:07.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f59b8f9c700 1 -- 172.21.15.40:0/509865854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b4164340 con 0x7f59b40c5b20 2024-09-13T17:51:07.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f59b8f9c700 1 -- 172.21.15.40:0/509865854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b4164900 con 0x7f59b40c5b20 2024-09-13T17:51:07.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7faf8b7fe700 1 -- 172.21.15.40:0/1317140617 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faf98003d30 con 0x7fafa414f8c0 2024-09-13T17:51:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7fcdfe7fc700 1 -- 172.21.15.40:0/3383528931 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcdf80dcdc0 msgr2=0x7fcdf817c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7fcdfe7fc700 1 --2- 172.21.15.40:0/3383528931 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcdf80dcdc0 0x7fcdf817c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7fcdfe7fc700 1 -- 172.21.15.40:0/3383528931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf80071b0 msgr2=0x7fcdf80dae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7fcdfe7fc700 1 --2- 172.21.15.40:0/3383528931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf80071b0 0x7fcdf80dae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7fcdfe7fc700 1 -- 172.21.15.40:0/3383528931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf00094b0 con 0x7fcdf804ab50 2024-09-13T17:51:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7faf8b7fe700 1 --2- 172.21.15.40:0/1317140617 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faf90066760 0x7faf90068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7faf8b7fe700 1 -- 172.21.15.40:0/1317140617 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7faf980865f0 con 0x7fafa414f8c0 2024-09-13T17:51:07.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7fcdfe7fc700 1 --2- 172.21.15.40:0/3383528931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf804ab50 0x7fcdf817c1c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fce00068370 tx=0x7fce000680b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f42a1a45700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429c0c42f0 0x7f429c15e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f42a1a45700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429c0c42f0 0x7f429c15e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:49130/0 (socket says 172.21.15.40:49130) 2024-09-13T17:51:07.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f42a1a45700 1 -- 172.21.15.40:0/1213672167 learned_addr learned my addr 172.21.15.40:0/1213672167 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:07.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f42a1a45700 1 -- 172.21.15.40:0/1213672167 >> v2:172.21.15.43:3300/0 conn(0x7f429c0caf30 msgr2=0x7f429c15dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:07.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f42a1a45700 1 --2- 172.21.15.40:0/1213672167 >> v2:172.21.15.43:3300/0 conn(0x7f429c0caf30 0x7f429c15dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f42a1a45700 1 -- 172.21.15.40:0/1213672167 >> v1:172.21.15.40:6789/0 conn(0x7f429c157e40 legacy=0x7f429c0cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:07.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f42a1a45700 1 -- 172.21.15.40:0/1213672167 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429c161410 con 0x7f429c0c42f0 2024-09-13T17:51:07.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f42a1a45700 1 --2- 172.21.15.40:0/1213672167 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429c0c42f0 0x7f429c15e180 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4298009910 tx=0x7f4298009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74d644c36b21ab1a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7feaaefb6700 1 Processor -- start 2024-09-13T17:51:07.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7feaaefb6700 1 -- start start 2024-09-13T17:51:07.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7fafa2d9d700 1 --2- 172.21.15.40:0/1317140617 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faf90066760 0x7faf90068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7fafa2d9d700 1 --2- 172.21.15.40:0/1317140617 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faf90066760 0x7faf90068c20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009dd0 tx=0x7faf9c009790 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7fcde7fff700 1 -- 172.21.15.40:0/3383528931 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0008f030 con 0x7fcdf804ab50 2024-09-13T17:51:07.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f6861ffb700 1 -- 172.21.15.40:0/433452979 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6868021070 con 0x7f686c0c6da0 2024-09-13T17:51:07.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f6872400700 1 -- 172.21.15.40:0/433452979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f686c0c0340 con 0x7f686c0c6da0 2024-09-13T17:51:07.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f65030fd700 1 -- 172.21.15.40:0/3507149278 wait complete. 2024-09-13T17:51:07.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.682+0000 7f65030fd700 1 Processor -- start 2024-09-13T17:51:07.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f65030fd700 1 -- start start 2024-09-13T17:51:07.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7feaaefb6700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feaa80c4f90 0x7feaa8071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7feaaefb6700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaa80706f0 con 0x7feaa8150240 2024-09-13T17:51:07.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7feaaefb6700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaa80c28c0 con 0x7feaa80c54e0 2024-09-13T17:51:07.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7feaaefb6700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa8070450 con 0x7feaa80c4f90 2024-09-13T17:51:07.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f4293fff700 1 -- 172.21.15.40:0/1213672167 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4298004230 con 0x7f429c0c42f0 2024-09-13T17:51:07.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7feaad7b3700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7feaa80c54e0 0x7feaa815e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:35438/0 (socket says 172.21.15.40:35438) 2024-09-13T17:51:07.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7feaad7b3700 1 -- 172.21.15.40:0/528482417 learned_addr learned my addr 172.21.15.40:0/528482417 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:07.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f4293fff700 1 -- 172.21.15.40:0/1213672167 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4298004390 con 0x7f429c0c42f0 2024-09-13T17:51:07.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f65030fd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64fc072270 0x7f64fc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f65030fd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f64fc0c6390 0x7f64fc124100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f65030fd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64fc0c7320 0x7f64fc15a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f65030fd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc0c22d0 con 0x7f64fc072270 2024-09-13T17:51:07.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f65030fd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1ef0d0 con 0x7f64fc0c7320 2024-09-13T17:51:07.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f65030fd700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1efef0 con 0x7f64fc0c6390 2024-09-13T17:51:07.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f4293fff700 1 -- 172.21.15.40:0/1213672167 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4298009e60 con 0x7f429c0c42f0 2024-09-13T17:51:07.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f599bfff700 1 -- 172.21.15.40:0/509865854 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f59ac0145a0 con 0x7f59b40c5b20 2024-09-13T17:51:07.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7fcde7fff700 1 -- 172.21.15.40:0/3383528931 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce00067930 con 0x7fcdf804ab50 2024-09-13T17:51:07.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7fcde7fff700 1 -- 172.21.15.40:0/3383528931 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce00072350 con 0x7fcdf804ab50 2024-09-13T17:51:07.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7fce058eb700 1 -- 172.21.15.40:0/3383528931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf816dbf0 con 0x7fcdf804ab50 2024-09-13T17:51:07.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7fce058eb700 1 -- 172.21.15.40:0/3383528931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf80d6270 con 0x7fcdf804ab50 2024-09-13T17:51:07.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f65018fa700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f64fc0c6390 0x7f64fc124100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f65028fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64fc0c7320 0x7f64fc15a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f65018fa700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f64fc0c6390 0x7f64fc124100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:49142/0 (socket says 172.21.15.40:49142) 2024-09-13T17:51:07.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f65020fb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64fc072270 0x7f64fc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f65018fa700 1 -- 172.21.15.40:0/4032135952 learned_addr learned my addr 172.21.15.40:0/4032135952 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:07.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f599bfff700 1 --2- 172.21.15.40:0/509865854 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f59a0066760 0x7f59a0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f65020fb700 1 -- 172.21.15.40:0/4032135952 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f64fc0c6390 msgr2=0x7f64fc124100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f65020fb700 1 --2- 172.21.15.40:0/4032135952 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f64fc0c6390 0x7f64fc124100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f65020fb700 1 -- 172.21.15.40:0/4032135952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64fc0c7320 msgr2=0x7f64fc15a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f65020fb700 1 --2- 172.21.15.40:0/4032135952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64fc0c7320 0x7f64fc15a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f599bfff700 1 -- 172.21.15.40:0/509865854 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f59ac083e00 con 0x7f59b40c5b20 2024-09-13T17:51:07.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f65020fb700 1 -- 172.21.15.40:0/4032135952 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f80094b0 con 0x7f64fc072270 2024-09-13T17:51:07.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f59b37fe700 1 --2- 172.21.15.40:0/509865854 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f59a0066760 0x7f59a0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f65020fb700 1 --2- 172.21.15.40:0/4032135952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64fc072270 0x7f64fc06d540 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f64ec016000 tx=0x7f64ec004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f59b37fe700 1 --2- 172.21.15.40:0/509865854 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f59a0066760 0x7f59a0068c20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009690 tx=0x7f59a4006be0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f42a2246700 1 -- 172.21.15.40:0/1213672167 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429c0c42f0 msgr2=0x7f429c15e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f42a2246700 1 --2- 172.21.15.40:0/1213672167 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429c0c42f0 0x7f429c15e180 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4298009910 tx=0x7f4298009c20 comp rx=0 tx=0).stop 2024-09-13T17:51:07.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f42a2246700 1 -- 172.21.15.40:0/1213672167 shutdown_connections 2024-09-13T17:51:07.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f42a2246700 1 --2- 172.21.15.40:0/1213672167 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429c0c42f0 0x7f429c15e180 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f42a2246700 1 --2- 172.21.15.40:0/1213672167 >> v2:172.21.15.43:3300/0 conn(0x7f429c0caf30 0x7f429c15dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f42a2246700 1 -- 172.21.15.40:0/1213672167 >> 172.21.15.40:0/1213672167 conn(0x7f429c06c800 msgr2=0x7f429c06cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:07.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7fce058eb700 1 -- 172.21.15.40:0/3383528931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf80d60d0 con 0x7fcdf804ab50 2024-09-13T17:51:07.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f42a2246700 1 -- 172.21.15.40:0/1213672167 shutdown_connections 2024-09-13T17:51:07.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f42a2246700 1 -- 172.21.15.40:0/1213672167 wait complete. 2024-09-13T17:51:07.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f6861ffb700 1 --2- 172.21.15.40:0/433452979 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6850066730 0x7f6850068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f6861ffb700 1 -- 172.21.15.40:0/433452979 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f68680859c0 con 0x7f686c0c6da0 2024-09-13T17:51:07.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f6870bfd700 1 --2- 172.21.15.40:0/433452979 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6850066730 0x7f6850068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.686+0000 7f6870bfd700 1 --2- 172.21.15.40:0/433452979 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6850066730 0x7f6850068bf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f686c156b20 tx=0x7f685c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7f42a2246700 1 Processor -- start 2024-09-13T17:51:07.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7f42a2246700 1 -- start start 2024-09-13T17:51:07.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7f42a2246700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429c0c42f0 0x7f429c1e8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7f42a2246700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429c0caf30 0x7f429c1515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7f42a2246700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f429c157e40 0x7f429c151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7f42a2246700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c156ed0 con 0x7f429c0c42f0 2024-09-13T17:51:07.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7f42a2246700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c0cd910 con 0x7f429c157e40 2024-09-13T17:51:07.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7f42a2246700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f429c06d150 con 0x7f429c0caf30 2024-09-13T17:51:07.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7f42a1244700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429c0c42f0 0x7f429c1e8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7feaae7b5700 1 --2- 172.21.15.40:0/528482417 >> v2:172.21.15.43:3300/0 conn(0x7feaa80c4f90 0x7feaa8071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7f42a1244700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429c0c42f0 0x7f429c1e8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58198/0 (socket says 172.21.15.40:58198) 2024-09-13T17:51:07.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7f42a1244700 1 -- 172.21.15.40:0/2687677243 learned_addr learned my addr 172.21.15.40:0/2687677243 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:07.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7f42a0a43700 1 --2- 172.21.15.40:0/2687677243 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429c0caf30 0x7f429c1515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7f42a1a45700 1 --2- 172.21.15.40:0/2687677243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f429c157e40 0x7f429c151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7f42a1244700 1 -- 172.21.15.40:0/2687677243 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429c0caf30 msgr2=0x7f429c1515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7f42a1244700 1 --2- 172.21.15.40:0/2687677243 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429c0caf30 0x7f429c1515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7f42a1244700 1 -- 172.21.15.40:0/2687677243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f429c157e40 msgr2=0x7f429c151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7f42a1244700 1 --2- 172.21.15.40:0/2687677243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f429c157e40 0x7f429c151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7f42a1244700 1 -- 172.21.15.40:0/2687677243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4298009570 con 0x7f429c0c42f0 2024-09-13T17:51:07.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7f42a1244700 1 --2- 172.21.15.40:0/2687677243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429c0c42f0 0x7f429c1e8240 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f428c00b500 tx=0x7f428c00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7feaae7b5700 1 -- 172.21.15.40:0/528482417 >> v1:172.21.15.43:6789/0 conn(0x7feaa80c54e0 legacy=0x7feaa815e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7feaae7b5700 1 -- 172.21.15.40:0/528482417 >> v1:172.21.15.40:6789/0 conn(0x7feaa8150240 legacy=0x7feaa8070ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7feaae7b5700 1 -- 172.21.15.40:0/528482417 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa8156c00 con 0x7feaa80c4f90 2024-09-13T17:51:07.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7feaae7b5700 1 --2- 172.21.15.40:0/528482417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaa80c4f90 0x7feaa8071660 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fea98009850 tx=0x7fea98009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a060a59f50b2eea8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7feaacfb2700 1 -- 172.21.15.40:0/528482417 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea98011830 con 0x7feaa80c4f90 2024-09-13T17:51:07.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7feaacfb2700 1 -- 172.21.15.40:0/528482417 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea98011990 con 0x7feaa80c4f90 2024-09-13T17:51:07.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7feaacfb2700 1 -- 172.21.15.40:0/528482417 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea98011d20 con 0x7feaa80c4f90 2024-09-13T17:51:07.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7faf8b7fe700 1 -- 172.21.15.40:0/1317140617 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7faf980535d0 con 0x7fafa414f8c0 2024-09-13T17:51:07.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.690+0000 7feaaefb6700 1 -- 172.21.15.40:0/528482417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaa80c4f90 msgr2=0x7feaa8071660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaaefb6700 1 --2- 172.21.15.40:0/528482417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaa80c4f90 0x7feaa8071660 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fea98009850 tx=0x7fea98009b60 comp rx=0 tx=0).stop 2024-09-13T17:51:07.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaaefb6700 1 -- 172.21.15.40:0/528482417 shutdown_connections 2024-09-13T17:51:07.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaaefb6700 1 --2- 172.21.15.40:0/528482417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaa80c4f90 0x7feaa8071660 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaaefb6700 1 -- 172.21.15.40:0/528482417 >> 172.21.15.40:0/528482417 conn(0x7feaa806a390 msgr2=0x7feaa806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:07.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaaefb6700 1 -- 172.21.15.40:0/528482417 shutdown_connections 2024-09-13T17:51:07.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7f59b8f9c700 1 -- 172.21.15.40:0/509865854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b40c5f00 con 0x7f59b40c5b20 2024-09-13T17:51:07.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaaefb6700 1 -- 172.21.15.40:0/528482417 wait complete. 2024-09-13T17:51:07.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7f6861ffb700 1 -- 172.21.15.40:0/433452979 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6868052bd0 con 0x7f686c0c6da0 2024-09-13T17:51:07.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaaefb6700 1 Processor -- start 2024-09-13T17:51:07.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaaefb6700 1 -- start start 2024-09-13T17:51:07.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7fcde7fff700 1 -- 172.21.15.40:0/3383528931 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fce00092070 con 0x7fcdf804ab50 2024-09-13T17:51:07.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaaefb6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaa80c4f90 0x7feaa813ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaaefb6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaa80c54e0 0x7feaa81509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7f64f2ffd700 1 -- 172.21.15.40:0/4032135952 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec004450 con 0x7f64fc072270 2024-09-13T17:51:07.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7f65030fd700 1 -- 172.21.15.40:0/4032135952 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ec009570 con 0x7f64fc072270 2024-09-13T17:51:07.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaaefb6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feaa8150240 0x7feaa81434e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaaefb6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa813efa0 con 0x7feaa80c4f90 2024-09-13T17:51:07.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaaefb6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa813e5d0 con 0x7feaa80c54e0 2024-09-13T17:51:07.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaaefb6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa80c8490 con 0x7feaa8150240 2024-09-13T17:51:07.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaadfb4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaa80c4f90 0x7feaa813ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaadfb4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaa80c4f90 0x7feaa813ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58210/0 (socket says 172.21.15.40:58210) 2024-09-13T17:51:07.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaadfb4700 1 -- 172.21.15.40:0/2984529369 learned_addr learned my addr 172.21.15.40:0/2984529369 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:07.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7f65030fd700 1 -- 172.21.15.40:0/4032135952 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64fc1ef8a0 con 0x7f64fc072270 2024-09-13T17:51:07.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaad7b3700 1 --2- 172.21.15.40:0/2984529369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaa80c54e0 0x7feaa81509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaae7b5700 1 --2- 172.21.15.40:0/2984529369 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feaa8150240 0x7feaa81434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7fcde7fff700 1 --2- 172.21.15.40:0/3383528931 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcdec066730 0x7fcdec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7fcdfeffd700 1 --2- 172.21.15.40:0/3383528931 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcdec066730 0x7fcdec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaad7b3700 1 -- 172.21.15.40:0/2984529369 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feaa8150240 msgr2=0x7feaa81434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaad7b3700 1 --2- 172.21.15.40:0/2984529369 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feaa8150240 0x7feaa81434e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaad7b3700 1 -- 172.21.15.40:0/2984529369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaa80c4f90 msgr2=0x7feaa813ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaad7b3700 1 --2- 172.21.15.40:0/2984529369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaa80c4f90 0x7feaa813ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaad7b3700 1 -- 172.21.15.40:0/2984529369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea980094b0 con 0x7feaa80c54e0 2024-09-13T17:51:07.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.694+0000 7feaad7b3700 1 --2- 172.21.15.40:0/2984529369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaa80c54e0 0x7feaa81509e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7feaa000cb10 tx=0x7feaa000cb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.698+0000 7f4291ffb700 1 -- 172.21.15.40:0/2687677243 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c011830 con 0x7f429c0c42f0 2024-09-13T17:51:07.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.698+0000 7fcdfeffd700 1 --2- 172.21.15.40:0/3383528931 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcdec066730 0x7fcdec068bf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4007fd0 tx=0x7fcdf4007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.698+0000 7f42a2246700 1 -- 172.21.15.40:0/2687677243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f429c1e8780 con 0x7f429c0c42f0 2024-09-13T17:51:07.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.698+0000 7f42a2246700 1 -- 172.21.15.40:0/2687677243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429c0cde20 con 0x7f429c0c42f0 2024-09-13T17:51:07.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.698+0000 7f65030fd700 1 -- 172.21.15.40:0/4032135952 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64fc0c6dd0 con 0x7f64fc072270 2024-09-13T17:51:07.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.702+0000 7fea9e7fc700 1 -- 172.21.15.40:0/2984529369 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa000a770 con 0x7feaa80c54e0 2024-09-13T17:51:07.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.702+0000 7fea9e7fc700 1 -- 172.21.15.40:0/2984529369 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa000a8d0 con 0x7feaa80c54e0 2024-09-13T17:51:07.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.702+0000 7fea9e7fc700 1 -- 172.21.15.40:0/2984529369 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa000ac40 con 0x7feaa80c54e0 2024-09-13T17:51:07.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.702+0000 7f42a2246700 1 -- 172.21.15.40:0/2687677243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429c0c3f10 con 0x7f429c0c42f0 2024-09-13T17:51:07.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.702+0000 7feaaefb6700 1 -- 172.21.15.40:0/2984529369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa813d9c0 con 0x7feaa80c54e0 2024-09-13T17:51:07.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.702+0000 7feaaefb6700 1 -- 172.21.15.40:0/2984529369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa813df80 con 0x7feaa80c54e0 2024-09-13T17:51:07.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.702+0000 7fcde7fff700 1 -- 172.21.15.40:0/3383528931 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fce00090070 con 0x7fcdf804ab50 2024-09-13T17:51:07.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.702+0000 7f4291ffb700 1 -- 172.21.15.40:0/2687677243 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428c011990 con 0x7f429c0c42f0 2024-09-13T17:51:07.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.702+0000 7f4291ffb700 1 -- 172.21.15.40:0/2687677243 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c011d20 con 0x7f429c0c42f0 2024-09-13T17:51:07.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.702+0000 7f599bfff700 1 -- 172.21.15.40:0/509865854 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f59ac050e80 con 0x7f59b40c5b20 2024-09-13T17:51:07.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.706+0000 7f64f2ffd700 1 -- 172.21.15.40:0/4032135952 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ec01f070 con 0x7f64fc072270 2024-09-13T17:51:07.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.706+0000 7f64f2ffd700 1 -- 172.21.15.40:0/4032135952 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec0038e0 con 0x7f64fc072270 2024-09-13T17:51:07.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.706+0000 7feaaefb6700 1 -- 172.21.15.40:0/2984529369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa813e740 con 0x7feaa80c54e0 2024-09-13T17:51:07.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.710+0000 7fea9e7fc700 1 -- 172.21.15.40:0/2984529369 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feaa001b3b0 con 0x7feaa80c54e0 2024-09-13T17:51:07.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.710+0000 7fea9e7fc700 1 --2- 172.21.15.40:0/2984529369 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fea8c066780 0x7fea8c068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.710+0000 7fcde7fff700 1 -- 172.21.15.40:0/3383528931 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fce000cbfb0 con 0x7fcdf804ab50 2024-09-13T17:51:07.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.710+0000 7fea9e7fc700 1 -- 172.21.15.40:0/2984529369 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7feaa0083be0 con 0x7feaa80c54e0 2024-09-13T17:51:07.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.710+0000 7feaadfb4700 1 --2- 172.21.15.40:0/2984529369 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fea8c066780 0x7fea8c068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.710+0000 7feaadfb4700 1 --2- 172.21.15.40:0/2984529369 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fea8c066780 0x7fea8c068c40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feaa4001800 tx=0x7feaa4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.714+0000 7f4291ffb700 1 -- 172.21.15.40:0/2687677243 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f428c01a400 con 0x7f429c0c42f0 2024-09-13T17:51:07.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.714+0000 7f4291ffb700 1 --2- 172.21.15.40:0/2687677243 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4280066730 0x7f4280068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.714+0000 7f4291ffb700 1 -- 172.21.15.40:0/2687677243 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f428c084c00 con 0x7f429c0c42f0 2024-09-13T17:51:07.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.714+0000 7f64f2ffd700 1 -- 172.21.15.40:0/4032135952 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f64ec003a40 con 0x7f64fc072270 2024-09-13T17:51:07.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.714+0000 7f64f2ffd700 1 --2- 172.21.15.40:0/4032135952 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f64e0066730 0x7f64e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:07.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.714+0000 7f64f2ffd700 1 -- 172.21.15.40:0/4032135952 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f64ec021070 con 0x7f64fc072270 2024-09-13T17:51:07.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.714+0000 7f65018fa700 1 --2- 172.21.15.40:0/4032135952 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f64e0066730 0x7f64e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.714+0000 7f42a0a43700 1 --2- 172.21.15.40:0/2687677243 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4280066730 0x7f4280068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:07.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.714+0000 7f65018fa700 1 --2- 172.21.15.40:0/4032135952 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f64e0066730 0x7f64e0068bf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0cbbc0 tx=0x7f64f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.714+0000 7f42a0a43700 1 --2- 172.21.15.40:0/2687677243 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4280066730 0x7f4280068bf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f429c0cce90 tx=0x7f42940094f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:07.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.718+0000 7f64f2ffd700 1 -- 172.21.15.40:0/4032135952 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f64ec052c20 con 0x7f64fc072270 2024-09-13T17:51:07.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.718+0000 7fea9e7fc700 1 -- 172.21.15.40:0/2984529369 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7feaa0050df0 con 0x7feaa80c54e0 2024-09-13T17:51:07.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.726+0000 7f4291ffb700 1 -- 172.21.15.40:0/2687677243 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f428c051cf0 con 0x7f429c0c42f0 2024-09-13T17:51:07.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.982+0000 7fbf44317700 1 -- 172.21.15.40:0/1503967791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fbf3c151830 con 0x7fbf3c14f8c0 2024-09-13T17:51:07.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.982+0000 7fbf32ffd700 1 -- 172.21.15.40:0/1503967791 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbf2c015090 con 0x7fbf3c14f8c0 2024-09-13T17:51:07.992 INFO:teuthology.orchestra.run.smithi040.stdout:60129542155 2024-09-13T17:51:07.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.990+0000 7fbf30ff9700 1 -- 172.21.15.40:0/1503967791 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbf20066780 msgr2=0x7fbf20068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.990+0000 7fbf30ff9700 1 --2- 172.21.15.40:0/1503967791 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbf20066780 0x7fbf20068c40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbf380090b0 tx=0x7fbf3801b040 comp rx=0 tx=0).stop 2024-09-13T17:51:07.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.990+0000 7fbf30ff9700 1 -- 172.21.15.40:0/1503967791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c14f8c0 msgr2=0x7fbf3c151b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:07.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.990+0000 7fbf30ff9700 1 --2- 172.21.15.40:0/1503967791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c14f8c0 0x7fbf3c151b40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c00b520 tx=0x7fbf2c00b830 comp rx=0 tx=0).stop 2024-09-13T17:51:07.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.990+0000 7fbf30ff9700 1 -- 172.21.15.40:0/1503967791 shutdown_connections 2024-09-13T17:51:07.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.990+0000 7fbf30ff9700 1 --2- 172.21.15.40:0/1503967791 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf3c158bc0 0x7fbf3c0c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.990+0000 7fbf30ff9700 1 --2- 172.21.15.40:0/1503967791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf3c14f8c0 0x7fbf3c151b40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.990+0000 7fbf30ff9700 1 --2- 172.21.15.40:0/1503967791 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbf20066780 0x7fbf20068c40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.990+0000 7fbf30ff9700 1 --2- 172.21.15.40:0/1503967791 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf3c0724e0 0x7fbf3c1fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:07.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.990+0000 7fbf30ff9700 1 -- 172.21.15.40:0/1503967791 >> 172.21.15.40:0/1503967791 conn(0x7fbf3c066090 msgr2=0x7fbf3c0c05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:07.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.990+0000 7fbf30ff9700 1 -- 172.21.15.40:0/1503967791 shutdown_connections 2024-09-13T17:51:07.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:07.990+0000 7fbf30ff9700 1 -- 172.21.15.40:0/1503967791 wait complete. 2024-09-13T17:51:08.010 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542155 for osd.5 2024-09-13T17:51:08.010 DEBUG:teuthology.parallel:result is None 2024-09-13T17:51:08.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.014+0000 7fafa8829700 1 -- 172.21.15.40:0/1317140617 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fafa40c6aa0 con 0x7fafa414f8c0 2024-09-13T17:51:08.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.018+0000 7faf8b7fe700 1 -- 172.21.15.40:0/1317140617 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faf98056bf0 con 0x7fafa414f8c0 2024-09-13T17:51:08.022 INFO:teuthology.orchestra.run.smithi040.stdout:64424509451 2024-09-13T17:51:08.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.022+0000 7fafa8829700 1 -- 172.21.15.40:0/1317140617 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faf90066760 msgr2=0x7faf90068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.022+0000 7fafa8829700 1 --2- 172.21.15.40:0/1317140617 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faf90066760 0x7faf90068c20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009dd0 tx=0x7faf9c009790 comp rx=0 tx=0).stop 2024-09-13T17:51:08.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.022+0000 7fafa8829700 1 -- 172.21.15.40:0/1317140617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafa414f8c0 msgr2=0x7fafa4150940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.022+0000 7fafa8829700 1 --2- 172.21.15.40:0/1317140617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafa414f8c0 0x7fafa4150940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fafa406b3c0 tx=0x7faf98004770 comp rx=0 tx=0).stop 2024-09-13T17:51:08.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.022+0000 7fafa8829700 1 -- 172.21.15.40:0/1317140617 shutdown_connections 2024-09-13T17:51:08.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.022+0000 7fafa8829700 1 --2- 172.21.15.40:0/1317140617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fafa414f8c0 0x7fafa4150940 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.022+0000 7fafa8829700 1 --2- 172.21.15.40:0/1317140617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fafa40c78d0 0x7fafa4150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.022+0000 7fafa8829700 1 --2- 172.21.15.40:0/1317140617 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faf90066760 0x7faf90068c20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.022+0000 7fafa8829700 1 --2- 172.21.15.40:0/1317140617 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fafa40724e0 0x7fafa41ee2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.022+0000 7fafa8829700 1 -- 172.21.15.40:0/1317140617 >> 172.21.15.40:0/1317140617 conn(0x7fafa4066090 msgr2=0x7fafa4158c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:08.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.022+0000 7fafa8829700 1 -- 172.21.15.40:0/1317140617 shutdown_connections 2024-09-13T17:51:08.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.022+0000 7fafa8829700 1 -- 172.21.15.40:0/1317140617 wait complete. 2024-09-13T17:51:08.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.026+0000 7feaaefb6700 1 -- 172.21.15.40:0/2984529369 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7feaa8150620 con 0x7feaa80c54e0 2024-09-13T17:51:08.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.026+0000 7fea9e7fc700 1 -- 172.21.15.40:0/2984529369 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feaa0017090 con 0x7feaa80c54e0 2024-09-13T17:51:08.032 INFO:teuthology.orchestra.run.smithi040.stdout:60129542155 2024-09-13T17:51:08.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.030+0000 7f59b8f9c700 1 -- 172.21.15.40:0/509865854 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f59b40703c0 con 0x7f59b40c5b20 2024-09-13T17:51:08.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.030+0000 7f599bfff700 1 -- 172.21.15.40:0/509865854 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f59ac019020 con 0x7f59b40c5b20 2024-09-13T17:51:08.035 INFO:teuthology.orchestra.run.smithi040.stdout:64424509451 2024-09-13T17:51:08.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.030+0000 7feaaefb6700 1 -- 172.21.15.40:0/2984529369 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fea8c066780 msgr2=0x7fea8c068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.030+0000 7feaaefb6700 1 --2- 172.21.15.40:0/2984529369 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fea8c066780 0x7fea8c068c40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feaa4001800 tx=0x7feaa4009040 comp rx=0 tx=0).stop 2024-09-13T17:51:08.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.030+0000 7f6872400700 1 -- 172.21.15.40:0/433452979 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f686c06d3b0 con 0x7f686c0c6da0 2024-09-13T17:51:08.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.030+0000 7feaaefb6700 1 -- 172.21.15.40:0/2984529369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaa80c54e0 msgr2=0x7feaa81509e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.030+0000 7feaaefb6700 1 --2- 172.21.15.40:0/2984529369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaa80c54e0 0x7feaa81509e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7feaa000cb10 tx=0x7feaa000cb40 comp rx=0 tx=0).stop 2024-09-13T17:51:08.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.030+0000 7f6861ffb700 1 -- 172.21.15.40:0/433452979 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f68680561f0 con 0x7f686c0c6da0 2024-09-13T17:51:08.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.030+0000 7feaaefb6700 1 -- 172.21.15.40:0/2984529369 shutdown_connections 2024-09-13T17:51:08.037 INFO:teuthology.orchestra.run.smithi040.stdout:60129542155 2024-09-13T17:51:08.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.030+0000 7feaaefb6700 1 --2- 172.21.15.40:0/2984529369 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fea8c066780 0x7fea8c068c40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.030+0000 7feaaefb6700 1 --2- 172.21.15.40:0/2984529369 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feaa8150240 0x7feaa81434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.030+0000 7feaaefb6700 1 --2- 172.21.15.40:0/2984529369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaa80c54e0 0x7feaa81509e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.030+0000 7feaaefb6700 1 --2- 172.21.15.40:0/2984529369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaa80c4f90 0x7feaa813ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.030+0000 7feaaefb6700 1 -- 172.21.15.40:0/2984529369 >> 172.21.15.40:0/2984529369 conn(0x7feaa806a390 msgr2=0x7feaa8158090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:08.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.030+0000 7feaaefb6700 1 -- 172.21.15.40:0/2984529369 shutdown_connections 2024-09-13T17:51:08.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.030+0000 7feaaefb6700 1 -- 172.21.15.40:0/2984529369 wait complete. 2024-09-13T17:51:08.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f6872400700 1 -- 172.21.15.40:0/433452979 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6850066730 msgr2=0x7f6850068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f6872400700 1 --2- 172.21.15.40:0/433452979 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6850066730 0x7f6850068bf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f686c156b20 tx=0x7f685c009040 comp rx=0 tx=0).stop 2024-09-13T17:51:08.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f6872400700 1 -- 172.21.15.40:0/433452979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c0c6da0 msgr2=0x7f686c072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f6872400700 1 --2- 172.21.15.40:0/433452979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c0c6da0 0x7f686c072100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6868004000 tx=0x7f6868004770 comp rx=0 tx=0).stop 2024-09-13T17:51:08.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f6872400700 1 -- 172.21.15.40:0/433452979 shutdown_connections 2024-09-13T17:51:08.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f6872400700 1 --2- 172.21.15.40:0/433452979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f686c14fdd0 0x7f686c152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f6872400700 1 --2- 172.21.15.40:0/433452979 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6850066730 0x7f6850068bf0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f6872400700 1 --2- 172.21.15.40:0/433452979 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f686c0c71a0 0x7f686c1526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f6872400700 1 --2- 172.21.15.40:0/433452979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f686c0c6da0 0x7f686c072100 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f6872400700 1 -- 172.21.15.40:0/433452979 >> 172.21.15.40:0/433452979 conn(0x7f686c066090 msgr2=0x7f686c06d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:08.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f6872400700 1 -- 172.21.15.40:0/433452979 shutdown_connections 2024-09-13T17:51:08.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f6872400700 1 -- 172.21.15.40:0/433452979 wait complete. 2024-09-13T17:51:08.041 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.1 2024-09-13T17:51:08.042 DEBUG:teuthology.parallel:result is None 2024-09-13T17:51:08.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f59b8f9c700 1 -- 172.21.15.40:0/509865854 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f59a0066760 msgr2=0x7f59a0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f59b8f9c700 1 --2- 172.21.15.40:0/509865854 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f59a0066760 0x7f59a0068c20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009690 tx=0x7f59a4006be0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f59b8f9c700 1 -- 172.21.15.40:0/509865854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b40c5b20 msgr2=0x7f59b40c9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f59b8f9c700 1 --2- 172.21.15.40:0/509865854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b40c5b20 0x7f59b40c9c50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f59ac00b9e0 tx=0x7f59ac00bcf0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f59b8f9c700 1 -- 172.21.15.40:0/509865854 shutdown_connections 2024-09-13T17:51:08.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f59b8f9c700 1 --2- 172.21.15.40:0/509865854 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f59a0066760 0x7f59a0068c20 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009690 tx=0x7f59a4006be0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f59b8f9c700 1 --2- 172.21.15.40:0/509865854 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f59b4163a20 0x7f59b41653e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f59b8f9c700 1 --2- 172.21.15.40:0/509865854 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59b40c5b20 0x7f59b40c9c50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f59b8f9c700 1 --2- 172.21.15.40:0/509865854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59b40b0de0 0x7f59b420cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.034+0000 7f59b8f9c700 1 -- 172.21.15.40:0/509865854 >> 172.21.15.40:0/509865854 conn(0x7f59b40cad00 msgr2=0x7f59b40cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:08.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.038+0000 7f59b8f9c700 1 -- 172.21.15.40:0/509865854 shutdown_connections 2024-09-13T17:51:08.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.038+0000 7f59b8f9c700 1 -- 172.21.15.40:0/509865854 wait complete. 2024-09-13T17:51:08.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.038+0000 7fce058eb700 1 -- 172.21.15.40:0/3383528931 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fcdf80d60d0 con 0x7fcdf804ab50 2024-09-13T17:51:08.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.038+0000 7fcde7fff700 1 -- 172.21.15.40:0/3383528931 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcdf80d60d0 con 0x7fcdf804ab50 2024-09-13T17:51:08.044 INFO:teuthology.orchestra.run.smithi040.stdout:60129542155 2024-09-13T17:51:08.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.042+0000 7fce058eb700 1 -- 172.21.15.40:0/3383528931 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcdec066730 msgr2=0x7fcdec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.042+0000 7fce058eb700 1 --2- 172.21.15.40:0/3383528931 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcdec066730 0x7fcdec068bf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4007fd0 tx=0x7fcdf4007de0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.042+0000 7fce058eb700 1 -- 172.21.15.40:0/3383528931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf804ab50 msgr2=0x7fcdf817c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.042+0000 7fce058eb700 1 --2- 172.21.15.40:0/3383528931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf804ab50 0x7fcdf817c1c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fce00068370 tx=0x7fce000680b0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.042+0000 7fce058eb700 1 -- 172.21.15.40:0/3383528931 shutdown_connections 2024-09-13T17:51:08.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.042+0000 7fce058eb700 1 --2- 172.21.15.40:0/3383528931 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcdec066730 0x7fcdec068bf0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.042+0000 7fce058eb700 1 --2- 172.21.15.40:0/3383528931 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcdf80dcdc0 0x7fcdf817c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.042+0000 7fce058eb700 1 --2- 172.21.15.40:0/3383528931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdf804ab50 0x7fcdf817c1c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.042+0000 7fce058eb700 1 --2- 172.21.15.40:0/3383528931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf80071b0 0x7fcdf80dae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.042+0000 7fce058eb700 1 -- 172.21.15.40:0/3383528931 >> 172.21.15.40:0/3383528931 conn(0x7fcdf8050120 msgr2=0x7fcdf8004f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:08.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.042+0000 7fce058eb700 1 -- 172.21.15.40:0/3383528931 shutdown_connections 2024-09-13T17:51:08.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.042+0000 7fce058eb700 1 -- 172.21.15.40:0/3383528931 wait complete. 2024-09-13T17:51:08.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.046+0000 7f65030fd700 1 -- 172.21.15.40:0/4032135952 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f64fc0c10c0 con 0x7f64fc072270 2024-09-13T17:51:08.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.046+0000 7f64f2ffd700 1 -- 172.21.15.40:0/4032135952 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f64ec056240 con 0x7f64fc072270 2024-09-13T17:51:08.050 INFO:teuthology.orchestra.run.smithi040.stdout:64424509451 2024-09-13T17:51:08.054 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542155 for osd.4 2024-09-13T17:51:08.054 DEBUG:teuthology.parallel:result is None 2024-09-13T17:51:08.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.046+0000 7f65030fd700 1 -- 172.21.15.40:0/4032135952 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f64e0066730 msgr2=0x7f64e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.046+0000 7f65030fd700 1 --2- 172.21.15.40:0/4032135952 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f64e0066730 0x7f64e0068bf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f64fc0cbbc0 tx=0x7f64f4009040 comp rx=0 tx=0).stop 2024-09-13T17:51:08.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.046+0000 7f65030fd700 1 -- 172.21.15.40:0/4032135952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64fc072270 msgr2=0x7f64fc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.046+0000 7f65030fd700 1 --2- 172.21.15.40:0/4032135952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64fc072270 0x7f64fc06d540 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f64ec016000 tx=0x7f64ec004770 comp rx=0 tx=0).stop 2024-09-13T17:51:08.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.050+0000 7f65030fd700 1 -- 172.21.15.40:0/4032135952 shutdown_connections 2024-09-13T17:51:08.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.050+0000 7f65030fd700 1 --2- 172.21.15.40:0/4032135952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64fc0c7320 0x7f64fc15a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.050+0000 7f65030fd700 1 --2- 172.21.15.40:0/4032135952 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f64e0066730 0x7f64e0068bf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.050+0000 7f65030fd700 1 --2- 172.21.15.40:0/4032135952 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f64fc0c6390 0x7f64fc124100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.050+0000 7f65030fd700 1 --2- 172.21.15.40:0/4032135952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64fc072270 0x7f64fc06d540 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.050+0000 7f65030fd700 1 -- 172.21.15.40:0/4032135952 >> 172.21.15.40:0/4032135952 conn(0x7f64fc066090 msgr2=0x7f64fc06ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:08.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.050+0000 7f65030fd700 1 -- 172.21.15.40:0/4032135952 shutdown_connections 2024-09-13T17:51:08.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.050+0000 7f65030fd700 1 -- 172.21.15.40:0/4032135952 wait complete. 2024-09-13T17:51:08.056 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.3 2024-09-13T17:51:08.057 DEBUG:teuthology.parallel:result is None 2024-09-13T17:51:08.057 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542155 for osd.6 2024-09-13T17:51:08.057 DEBUG:teuthology.parallel:result is None 2024-09-13T17:51:08.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.054+0000 7f42a2246700 1 -- 172.21.15.40:0/2687677243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f429c154050 con 0x7f429c0c42f0 2024-09-13T17:51:08.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.054+0000 7f4291ffb700 1 -- 172.21.15.40:0/2687677243 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f428c015020 con 0x7f429c0c42f0 2024-09-13T17:51:08.058 INFO:teuthology.orchestra.run.smithi040.stdout:64424509452 2024-09-13T17:51:08.059 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542155 for osd.7 2024-09-13T17:51:08.059 DEBUG:teuthology.parallel:result is None 2024-09-13T17:51:08.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.054+0000 7f42a2246700 1 -- 172.21.15.40:0/2687677243 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4280066730 msgr2=0x7f4280068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.054+0000 7f42a2246700 1 --2- 172.21.15.40:0/2687677243 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4280066730 0x7f4280068bf0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f429c0cce90 tx=0x7f42940094f0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.054+0000 7f42a2246700 1 -- 172.21.15.40:0/2687677243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429c0c42f0 msgr2=0x7f429c1e8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.054+0000 7f42a2246700 1 --2- 172.21.15.40:0/2687677243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429c0c42f0 0x7f429c1e8240 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f428c00b500 tx=0x7f428c00b8c0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.054+0000 7f42a2246700 1 -- 172.21.15.40:0/2687677243 shutdown_connections 2024-09-13T17:51:08.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.054+0000 7f42a2246700 1 --2- 172.21.15.40:0/2687677243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f429c157e40 0x7f429c151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.054+0000 7f42a2246700 1 --2- 172.21.15.40:0/2687677243 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4280066730 0x7f4280068bf0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.054+0000 7f42a2246700 1 --2- 172.21.15.40:0/2687677243 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429c0caf30 0x7f429c1515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.054+0000 7f42a2246700 1 --2- 172.21.15.40:0/2687677243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429c0c42f0 0x7f429c1e8240 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.054+0000 7f42a2246700 1 -- 172.21.15.40:0/2687677243 >> 172.21.15.40:0/2687677243 conn(0x7f429c06c800 msgr2=0x7f429c0c4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:08.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.054+0000 7f42a2246700 1 -- 172.21.15.40:0/2687677243 shutdown_connections 2024-09-13T17:51:08.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.054+0000 7f42a2246700 1 -- 172.21.15.40:0/2687677243 wait complete. 2024-09-13T17:51:08.065 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.2 2024-09-13T17:51:08.065 DEBUG:teuthology.parallel:result is None 2024-09-13T17:51:08.070 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509452 for osd.0 2024-09-13T17:51:08.070 DEBUG:teuthology.parallel:result is None 2024-09-13T17:51:08.071 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T17:51:08.071 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:51:08.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.218+0000 7f1b596e5700 1 Processor -- start 2024-09-13T17:51:08.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.218+0000 7f1b596e5700 1 -- start start 2024-09-13T17:51:08.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b596e5700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b4c0d4ba0 0x7f1b4c04a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:08.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b596e5700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1b4c0dea90 0x7f1b4c0e5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:08.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b596e5700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b4c046730 con 0x7f1b4c04abc0 2024-09-13T17:51:08.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b596e5700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b4c007020 con 0x7f1b4c0dea90 2024-09-13T17:51:08.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b596e5700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c046bc0 con 0x7f1b4c0d4ba0 2024-09-13T17:51:08.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b58ee4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1b4c0dea90 0x7f1b4c0e5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:08.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b537fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1b4c04abc0 0x7f1b4c006390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:35454/0 (socket says 172.21.15.40:35454) 2024-09-13T17:51:08.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b537fe700 1 -- 172.21.15.40:0/2588463525 learned_addr learned my addr 172.21.15.40:0/2588463525 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:08.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b53fff700 1 --2- 172.21.15.40:0/2588463525 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b4c0d4ba0 0x7f1b4c04a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:08.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b58ee4700 1 -- 172.21.15.40:0/2588463525 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b4c0d4ba0 msgr2=0x7f1b4c04a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:08.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b58ee4700 1 --2- 172.21.15.40:0/2588463525 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b4c0d4ba0 0x7f1b4c04a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b58ee4700 1 -- 172.21.15.40:0/2588463525 >> v1:172.21.15.43:6789/0 conn(0x7f1b4c04abc0 legacy=0x7f1b4c006390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b58ee4700 1 -- 172.21.15.40:0/2588463525 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c0defd0 con 0x7f1b4c0dea90 2024-09-13T17:51:08.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b58ee4700 1 --2- 172.21.15.40:0/2588463525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b4c0dea90 0x7f1b4c0e5410 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1b54069620 tx=0x7f1b54069e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62b30731a470164b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:08.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b52ffd700 1 -- 172.21.15.40:0/2588463525 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b54067dd0 con 0x7f1b4c0dea90 2024-09-13T17:51:08.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b52ffd700 1 -- 172.21.15.40:0/2588463525 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b54067f30 con 0x7f1b4c0dea90 2024-09-13T17:51:08.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b52ffd700 1 -- 172.21.15.40:0/2588463525 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b54074720 con 0x7f1b4c0dea90 2024-09-13T17:51:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b596e5700 1 -- 172.21.15.40:0/2588463525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b4c0dea90 msgr2=0x7f1b4c0e5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b596e5700 1 --2- 172.21.15.40:0/2588463525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b4c0dea90 0x7f1b4c0e5410 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1b54069620 tx=0x7f1b54069e70 comp rx=0 tx=0).stop 2024-09-13T17:51:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b596e5700 1 -- 172.21.15.40:0/2588463525 shutdown_connections 2024-09-13T17:51:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b596e5700 1 --2- 172.21.15.40:0/2588463525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b4c0dea90 0x7f1b4c0e5410 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b596e5700 1 --2- 172.21.15.40:0/2588463525 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b4c0d4ba0 0x7f1b4c04a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b596e5700 1 -- 172.21.15.40:0/2588463525 >> 172.21.15.40:0/2588463525 conn(0x7f1b4c04fd00 msgr2=0x7f1b4c050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b596e5700 1 -- 172.21.15.40:0/2588463525 shutdown_connections 2024-09-13T17:51:08.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.222+0000 7f1b596e5700 1 -- 172.21.15.40:0/2588463525 wait complete. 2024-09-13T17:51:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b596e5700 1 Processor -- start 2024-09-13T17:51:08.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b596e5700 1 -- start start 2024-09-13T17:51:08.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b596e5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b4c04abc0 0x7f1b4c049300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:08.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b596e5700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b4c0d4ba0 0x7f1b4c173970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:08.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b596e5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b4c0dea90 0x7f1b4c0dc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:08.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b596e5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c049f00 con 0x7f1b4c04abc0 2024-09-13T17:51:08.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b596e5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c0d5250 con 0x7f1b4c0dea90 2024-09-13T17:51:08.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b596e5700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c0d5460 con 0x7f1b4c0d4ba0 2024-09-13T17:51:08.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b53fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b4c04abc0 0x7f1b4c049300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:08.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b53fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b4c04abc0 0x7f1b4c049300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58214/0 (socket says 172.21.15.40:58214) 2024-09-13T17:51:08.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b53fff700 1 -- 172.21.15.40:0/2714600824 learned_addr learned my addr 172.21.15.40:0/2714600824 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:08.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b537fe700 1 --2- 172.21.15.40:0/2714600824 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b4c0d4ba0 0x7f1b4c173970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:08.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b58ee4700 1 --2- 172.21.15.40:0/2714600824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b4c0dea90 0x7f1b4c0dc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:08.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b53fff700 1 -- 172.21.15.40:0/2714600824 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b4c0d4ba0 msgr2=0x7f1b4c173970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b53fff700 1 --2- 172.21.15.40:0/2714600824 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b4c0d4ba0 0x7f1b4c173970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b53fff700 1 -- 172.21.15.40:0/2714600824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b4c0dea90 msgr2=0x7f1b4c0dc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b53fff700 1 --2- 172.21.15.40:0/2714600824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b4c0dea90 0x7f1b4c0dc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b53fff700 1 -- 172.21.15.40:0/2714600824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5404f5c0 con 0x7f1b4c04abc0 2024-09-13T17:51:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b58ee4700 1 --2- 172.21.15.40:0/2714600824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b4c0dea90 0x7f1b4c0dc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b537fe700 1 --2- 172.21.15.40:0/2714600824 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b4c0d4ba0 0x7f1b4c173970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b53fff700 1 --2- 172.21.15.40:0/2714600824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b4c04abc0 0x7f1b4c049300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800e9d0 tx=0x7f1b4800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b50ff9700 1 -- 172.21.15.40:0/2714600824 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b48009720 con 0x7f1b4c04abc0 2024-09-13T17:51:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b50ff9700 1 -- 172.21.15.40:0/2714600824 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4800c850 con 0x7f1b4c04abc0 2024-09-13T17:51:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.226+0000 7f1b596e5700 1 -- 172.21.15.40:0/2714600824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4c174330 con 0x7f1b4c04abc0 2024-09-13T17:51:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.230+0000 7f1b50ff9700 1 -- 172.21.15.40:0/2714600824 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4800cbc0 con 0x7f1b4c04abc0 2024-09-13T17:51:08.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.230+0000 7f1b596e5700 1 -- 172.21.15.40:0/2714600824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4c048b00 con 0x7f1b4c04abc0 2024-09-13T17:51:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.230+0000 7f1b596e5700 1 -- 172.21.15.40:0/2714600824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b4c0460c0 con 0x7f1b4c04abc0 2024-09-13T17:51:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.230+0000 7f1b50ff9700 1 -- 172.21.15.40:0/2714600824 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1b4800cd20 con 0x7f1b4c04abc0 2024-09-13T17:51:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.230+0000 7f1b50ff9700 1 --2- 172.21.15.40:0/2714600824 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b40066730 0x7f1b40068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.230+0000 7f1b50ff9700 1 -- 172.21.15.40:0/2714600824 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1b48084680 con 0x7f1b4c04abc0 2024-09-13T17:51:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.230+0000 7f1b537fe700 1 --2- 172.21.15.40:0/2714600824 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b40066730 0x7f1b40068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:08.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.230+0000 7f1b537fe700 1 --2- 172.21.15.40:0/2714600824 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b40066730 0x7f1b40068bf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1b44007fd0 tx=0x7f1b44009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:08.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.242+0000 7f1b50ff9700 1 -- 172.21.15.40:0/2714600824 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1b4c0460c0 con 0x7f1b4c04abc0 2024-09-13T17:51:08.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.402+0000 7f1b596e5700 1 -- 172.21.15.40:0/2714600824 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b4c045e00 con 0x7f1b40066730 2024-09-13T17:51:08.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.418+0000 7f1b50ff9700 1 -- 172.21.15.40:0/2714600824 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345217 (secure 0 0 0) 0x7f1b44001ce0 con 0x7f1b40066730 2024-09-13T17:51:08.427 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:08.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.422+0000 7f1b596e5700 1 -- 172.21.15.40:0/2714600824 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b40066730 msgr2=0x7f1b40068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.422+0000 7f1b596e5700 1 --2- 172.21.15.40:0/2714600824 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b40066730 0x7f1b40068bf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1b44007fd0 tx=0x7f1b44009040 comp rx=0 tx=0).stop 2024-09-13T17:51:08.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.422+0000 7f1b596e5700 1 -- 172.21.15.40:0/2714600824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b4c04abc0 msgr2=0x7f1b4c049300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.422+0000 7f1b596e5700 1 --2- 172.21.15.40:0/2714600824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b4c04abc0 0x7f1b4c049300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800e9d0 tx=0x7f1b4800ed90 comp rx=0 tx=0).stop 2024-09-13T17:51:08.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.422+0000 7f1b596e5700 1 -- 172.21.15.40:0/2714600824 shutdown_connections 2024-09-13T17:51:08.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.422+0000 7f1b596e5700 1 --2- 172.21.15.40:0/2714600824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b4c0dea90 0x7f1b4c0dc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.422+0000 7f1b596e5700 1 --2- 172.21.15.40:0/2714600824 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b40066730 0x7f1b40068bf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.422+0000 7f1b596e5700 1 --2- 172.21.15.40:0/2714600824 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b4c0d4ba0 0x7f1b4c173970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.422+0000 7f1b596e5700 1 --2- 172.21.15.40:0/2714600824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b4c04abc0 0x7f1b4c049300 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.422+0000 7f1b596e5700 1 -- 172.21.15.40:0/2714600824 >> 172.21.15.40:0/2714600824 conn(0x7f1b4c04fd00 msgr2=0x7f1b4c04aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:08.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.422+0000 7f1b596e5700 1 -- 172.21.15.40:0/2714600824 shutdown_connections 2024-09-13T17:51:08.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.422+0000 7f1b596e5700 1 -- 172.21.15.40:0/2714600824 wait complete. 2024-09-13T17:51:08.431 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-13T17:51:08.440 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-09-13T17:51:07.007430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235816,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738904,"statfs":{"total":773094113280,"available":772852637696,"internally_reserved":0,"allocated":3317760,"data_stored":2005562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004879"},"pg_stats":[{"pgid":"2.1f","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.584829+0000","last_change":"2024-09-13T17:50:34.212158+0000","last_active":"2024-09-13T17:50:39.584829+0000","last_peered":"2024-09-13T17:50:39.584829+0000","last_clean":"2024-09-13T17:50:39.584829+0000","last_became_active":"2024-09-13T17:50:34.211867+0000","last_became_peered":"2024-09-13T17:50:34.211867+0000","last_unstale":"2024-09-13T17:50:39.584829+0000","last_undegraded":"2024-09-13T17:50:39.584829+0000","last_fullsized":"2024-09-13T17:50:39.584829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:14.671095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589172+0000","last_change":"2024-09-13T17:50:34.587396+0000","last_active":"2024-09-13T17:50:38.589172+0000","last_peered":"2024-09-13T17:50:38.589172+0000","last_clean":"2024-09-13T17:50:38.589172+0000","last_became_active":"2024-09-13T17:50:34.587108+0000","last_became_peered":"2024-09-13T17:50:34.587108+0000","last_unstale":"2024-09-13T17:50:38.589172+0000","last_undegraded":"2024-09-13T17:50:38.589172+0000","last_fullsized":"2024-09-13T17:50:38.589172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:39:16.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.995312+0000","last_change":"2024-09-13T17:50:38.615059+0000","last_active":"2024-09-13T17:50:38.995312+0000","last_peered":"2024-09-13T17:50:38.995312+0000","last_clean":"2024-09-13T17:50:38.995312+0000","last_became_active":"2024-09-13T17:50:38.614723+0000","last_became_peered":"2024-09-13T17:50:38.614723+0000","last_unstale":"2024-09-13T17:50:38.995312+0000","last_undegraded":"2024-09-13T17:50:38.995312+0000","last_fullsized":"2024-09-13T17:50:38.995312+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:23:01.902181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"25'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.996807+0000","last_change":"2024-09-13T17:50:34.204760+0000","last_active":"2024-09-13T17:50:38.996807+0000","last_peered":"2024-09-13T17:50:38.996807+0000","last_clean":"2024-09-13T17:50:38.996807+0000","last_became_active":"2024-09-13T17:50:34.202930+0000","last_became_peered":"2024-09-13T17:50:34.202930+0000","last_unstale":"2024-09-13T17:50:38.996807+0000","last_undegraded":"2024-09-13T17:50:38.996807+0000","last_fullsized":"2024-09-13T17:50:38.996807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:59.784701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.598529+0000","last_change":"2024-09-13T17:50:34.583174+0000","last_active":"2024-09-13T17:50:38.598529+0000","last_peered":"2024-09-13T17:50:38.598529+0000","last_clean":"2024-09-13T17:50:38.598529+0000","last_became_active":"2024-09-13T17:50:34.582884+0000","last_became_peered":"2024-09-13T17:50:34.582884+0000","last_unstale":"2024-09-13T17:50:38.598529+0000","last_undegraded":"2024-09-13T17:50:38.598529+0000","last_fullsized":"2024-09-13T17:50:38.598529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:22:27.539574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501034+0000","last_change":"2024-09-13T17:50:38.618751+0000","last_active":"2024-09-13T17:50:39.501034+0000","last_peered":"2024-09-13T17:50:39.501034+0000","last_clean":"2024-09-13T17:50:39.501034+0000","last_became_active":"2024-09-13T17:50:38.618423+0000","last_became_peered":"2024-09-13T17:50:38.618423+0000","last_unstale":"2024-09-13T17:50:39.501034+0000","last_undegraded":"2024-09-13T17:50:39.501034+0000","last_fullsized":"2024-09-13T17:50:39.501034+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:50:02.418785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.582559+0000","last_change":"2024-09-13T17:50:34.214972+0000","last_active":"2024-09-13T17:50:39.582559+0000","last_peered":"2024-09-13T17:50:39.582559+0000","last_clean":"2024-09-13T17:50:39.582559+0000","last_became_active":"2024-09-13T17:50:34.214652+0000","last_became_peered":"2024-09-13T17:50:34.214652+0000","last_unstale":"2024-09-13T17:50:39.582559+0000","last_undegraded":"2024-09-13T17:50:39.582559+0000","last_fullsized":"2024-09-13T17:50:39.582559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:57.176854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590539+0000","last_change":"2024-09-13T17:50:34.594493+0000","last_active":"2024-09-13T17:50:38.590539+0000","last_peered":"2024-09-13T17:50:38.590539+0000","last_clean":"2024-09-13T17:50:38.590539+0000","last_became_active":"2024-09-13T17:50:34.594182+0000","last_became_peered":"2024-09-13T17:50:34.594182+0000","last_unstale":"2024-09-13T17:50:38.590539+0000","last_undegraded":"2024-09-13T17:50:38.590539+0000","last_fullsized":"2024-09-13T17:50:38.590539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:18.059925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501399+0000","last_change":"2024-09-13T17:50:38.616219+0000","last_active":"2024-09-13T17:50:39.501399+0000","last_peered":"2024-09-13T17:50:39.501399+0000","last_clean":"2024-09-13T17:50:39.501399+0000","last_became_active":"2024-09-13T17:50:38.614329+0000","last_became_peered":"2024-09-13T17:50:38.614329+0000","last_unstale":"2024-09-13T17:50:39.501399+0000","last_undegraded":"2024-09-13T17:50:39.501399+0000","last_fullsized":"2024-09-13T17:50:39.501399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:25.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583345+0000","last_change":"2024-09-13T17:50:34.216123+0000","last_active":"2024-09-13T17:50:39.583345+0000","last_peered":"2024-09-13T17:50:39.583345+0000","last_clean":"2024-09-13T17:50:39.583345+0000","last_became_active":"2024-09-13T17:50:34.215661+0000","last_became_peered":"2024-09-13T17:50:34.215661+0000","last_unstale":"2024-09-13T17:50:39.583345+0000","last_undegraded":"2024-09-13T17:50:39.583345+0000","last_fullsized":"2024-09-13T17:50:39.583345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:20.895037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591506+0000","last_change":"2024-09-13T17:50:34.599673+0000","last_active":"2024-09-13T17:50:38.591506+0000","last_peered":"2024-09-13T17:50:38.591506+0000","last_clean":"2024-09-13T17:50:38.591506+0000","last_became_active":"2024-09-13T17:50:34.599409+0000","last_became_peered":"2024-09-13T17:50:34.599409+0000","last_unstale":"2024-09-13T17:50:38.591506+0000","last_undegraded":"2024-09-13T17:50:38.591506+0000","last_fullsized":"2024-09-13T17:50:38.591506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:03:31.394385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501275+0000","last_change":"2024-09-13T17:50:38.616317+0000","last_active":"2024-09-13T17:50:39.501275+0000","last_peered":"2024-09-13T17:50:39.501275+0000","last_clean":"2024-09-13T17:50:39.501275+0000","last_became_active":"2024-09-13T17:50:38.614311+0000","last_became_peered":"2024-09-13T17:50:38.614311+0000","last_unstale":"2024-09-13T17:50:39.501275+0000","last_undegraded":"2024-09-13T17:50:39.501275+0000","last_fullsized":"2024-09-13T17:50:39.501275+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:29.221522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.604632+0000","last_change":"2024-09-13T17:50:34.599490+0000","last_active":"2024-09-13T17:50:38.604632+0000","last_peered":"2024-09-13T17:50:38.604632+0000","last_clean":"2024-09-13T17:50:38.604632+0000","last_became_active":"2024-09-13T17:50:34.599207+0000","last_became_peered":"2024-09-13T17:50:34.599207+0000","last_unstale":"2024-09-13T17:50:38.604632+0000","last_undegraded":"2024-09-13T17:50:38.604632+0000","last_fullsized":"2024-09-13T17:50:38.604632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:11.178161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583429+0000","last_change":"2024-09-13T17:50:34.215697+0000","last_active":"2024-09-13T17:50:39.583429+0000","last_peered":"2024-09-13T17:50:39.583429+0000","last_clean":"2024-09-13T17:50:39.583429+0000","last_became_active":"2024-09-13T17:50:34.215409+0000","last_became_peered":"2024-09-13T17:50:34.215409+0000","last_unstale":"2024-09-13T17:50:39.583429+0000","last_undegraded":"2024-09-13T17:50:39.583429+0000","last_fullsized":"2024-09-13T17:50:39.583429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:21.609567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.502325+0000","last_change":"2024-09-13T17:50:38.611817+0000","last_active":"2024-09-13T17:50:39.502325+0000","last_peered":"2024-09-13T17:50:39.502325+0000","last_clean":"2024-09-13T17:50:39.502325+0000","last_became_active":"2024-09-13T17:50:38.611263+0000","last_became_peered":"2024-09-13T17:50:38.611263+0000","last_unstale":"2024-09-13T17:50:39.502325+0000","last_undegraded":"2024-09-13T17:50:39.502325+0000","last_fullsized":"2024-09-13T17:50:39.502325+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:56:47.466672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.588598+0000","last_change":"2024-09-13T17:50:34.586307+0000","last_active":"2024-09-13T17:50:38.588598+0000","last_peered":"2024-09-13T17:50:38.588598+0000","last_clean":"2024-09-13T17:50:38.588598+0000","last_became_active":"2024-09-13T17:50:34.585831+0000","last_became_peered":"2024-09-13T17:50:34.585831+0000","last_unstale":"2024-09-13T17:50:38.588598+0000","last_undegraded":"2024-09-13T17:50:38.588598+0000","last_fullsized":"2024-09-13T17:50:38.588598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:40.912352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599468+0000","last_change":"2024-09-13T17:50:34.211850+0000","last_active":"2024-09-13T17:50:38.599468+0000","last_peered":"2024-09-13T17:50:38.599468+0000","last_clean":"2024-09-13T17:50:38.599468+0000","last_became_active":"2024-09-13T17:50:34.211594+0000","last_became_peered":"2024-09-13T17:50:34.211594+0000","last_unstale":"2024-09-13T17:50:38.599468+0000","last_undegraded":"2024-09-13T17:50:38.599468+0000","last_fullsized":"2024-09-13T17:50:38.599468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:35:02.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.502004+0000","last_change":"2024-09-13T17:50:38.613576+0000","last_active":"2024-09-13T17:50:39.502004+0000","last_peered":"2024-09-13T17:50:39.502004+0000","last_clean":"2024-09-13T17:50:39.502004+0000","last_became_active":"2024-09-13T17:50:38.612577+0000","last_became_peered":"2024-09-13T17:50:38.612577+0000","last_unstale":"2024-09-13T17:50:39.502004+0000","last_undegraded":"2024-09-13T17:50:39.502004+0000","last_fullsized":"2024-09-13T17:50:39.502004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:25.551060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590056+0000","last_change":"2024-09-13T17:50:34.587273+0000","last_active":"2024-09-13T17:50:38.590056+0000","last_peered":"2024-09-13T17:50:38.590056+0000","last_clean":"2024-09-13T17:50:38.590056+0000","last_became_active":"2024-09-13T17:50:34.587027+0000","last_became_peered":"2024-09-13T17:50:34.587027+0000","last_unstale":"2024-09-13T17:50:38.590056+0000","last_undegraded":"2024-09-13T17:50:38.590056+0000","last_fullsized":"2024-09-13T17:50:38.590056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:58.826737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599621+0000","last_change":"2024-09-13T17:50:34.211845+0000","last_active":"2024-09-13T17:50:38.599621+0000","last_peered":"2024-09-13T17:50:38.599621+0000","last_clean":"2024-09-13T17:50:38.599621+0000","last_became_active":"2024-09-13T17:50:34.211568+0000","last_became_peered":"2024-09-13T17:50:34.211568+0000","last_unstale":"2024-09-13T17:50:38.599621+0000","last_undegraded":"2024-09-13T17:50:38.599621+0000","last_fullsized":"2024-09-13T17:50:38.599621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:56.348984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000608+0000","last_change":"2024-09-13T17:50:38.610292+0000","last_active":"2024-09-13T17:50:39.000608+0000","last_peered":"2024-09-13T17:50:39.000608+0000","last_clean":"2024-09-13T17:50:39.000608+0000","last_became_active":"2024-09-13T17:50:38.609832+0000","last_became_peered":"2024-09-13T17:50:38.609832+0000","last_unstale":"2024-09-13T17:50:39.000608+0000","last_undegraded":"2024-09-13T17:50:39.000608+0000","last_fullsized":"2024-09-13T17:50:39.000608+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:13.783068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.603557+0000","last_change":"2024-09-13T17:50:34.593566+0000","last_active":"2024-09-13T17:50:38.603557+0000","last_peered":"2024-09-13T17:50:38.603557+0000","last_clean":"2024-09-13T17:50:38.603557+0000","last_became_active":"2024-09-13T17:50:34.588794+0000","last_became_peered":"2024-09-13T17:50:34.588794+0000","last_unstale":"2024-09-13T17:50:38.603557+0000","last_undegraded":"2024-09-13T17:50:38.603557+0000","last_fullsized":"2024-09-13T17:50:38.603557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:13.192453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589828+0000","last_change":"2024-09-13T17:50:34.214638+0000","last_active":"2024-09-13T17:50:38.589828+0000","last_peered":"2024-09-13T17:50:38.589828+0000","last_clean":"2024-09-13T17:50:38.589828+0000","last_became_active":"2024-09-13T17:50:34.214292+0000","last_became_peered":"2024-09-13T17:50:34.214292+0000","last_unstale":"2024-09-13T17:50:38.589828+0000","last_undegraded":"2024-09-13T17:50:38.589828+0000","last_fullsized":"2024-09-13T17:50:38.589828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:57.377104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501996+0000","last_change":"2024-09-13T17:50:38.613368+0000","last_active":"2024-09-13T17:50:39.501996+0000","last_peered":"2024-09-13T17:50:39.501996+0000","last_clean":"2024-09-13T17:50:39.501996+0000","last_became_active":"2024-09-13T17:50:38.613010+0000","last_became_peered":"2024-09-13T17:50:38.613010+0000","last_unstale":"2024-09-13T17:50:39.501996+0000","last_undegraded":"2024-09-13T17:50:39.501996+0000","last_fullsized":"2024-09-13T17:50:39.501996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:42.706625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.604457+0000","last_change":"2024-09-13T17:50:34.582327+0000","last_active":"2024-09-13T17:50:38.604457+0000","last_peered":"2024-09-13T17:50:38.604457+0000","last_clean":"2024-09-13T17:50:38.604457+0000","last_became_active":"2024-09-13T17:50:34.582050+0000","last_became_peered":"2024-09-13T17:50:34.582050+0000","last_unstale":"2024-09-13T17:50:38.604457+0000","last_undegraded":"2024-09-13T17:50:38.604457+0000","last_fullsized":"2024-09-13T17:50:38.604457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:52.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583664+0000","last_change":"2024-09-13T17:50:34.211512+0000","last_active":"2024-09-13T17:50:39.583664+0000","last_peered":"2024-09-13T17:50:39.583664+0000","last_clean":"2024-09-13T17:50:39.583664+0000","last_became_active":"2024-09-13T17:50:34.211230+0000","last_became_peered":"2024-09-13T17:50:34.211230+0000","last_unstale":"2024-09-13T17:50:39.583664+0000","last_undegraded":"2024-09-13T17:50:39.583664+0000","last_fullsized":"2024-09-13T17:50:39.583664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:40.707777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501244+0000","last_change":"2024-09-13T17:50:38.608255+0000","last_active":"2024-09-13T17:50:39.501244+0000","last_peered":"2024-09-13T17:50:39.501244+0000","last_clean":"2024-09-13T17:50:39.501244+0000","last_became_active":"2024-09-13T17:50:38.606697+0000","last_became_peered":"2024-09-13T17:50:38.606697+0000","last_unstale":"2024-09-13T17:50:39.501244+0000","last_undegraded":"2024-09-13T17:50:39.501244+0000","last_fullsized":"2024-09-13T17:50:39.501244+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:47.265007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591328+0000","last_change":"2024-09-13T17:50:34.594476+0000","last_active":"2024-09-13T17:50:38.591328+0000","last_peered":"2024-09-13T17:50:38.591328+0000","last_clean":"2024-09-13T17:50:38.591328+0000","last_became_active":"2024-09-13T17:50:34.594215+0000","last_became_peered":"2024-09-13T17:50:34.594215+0000","last_unstale":"2024-09-13T17:50:38.591328+0000","last_undegraded":"2024-09-13T17:50:38.591328+0000","last_fullsized":"2024-09-13T17:50:38.591328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:02:27.528019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000656+0000","last_change":"2024-09-13T17:50:34.204018+0000","last_active":"2024-09-13T17:50:39.000656+0000","last_peered":"2024-09-13T17:50:39.000656+0000","last_clean":"2024-09-13T17:50:39.000656+0000","last_became_active":"2024-09-13T17:50:34.203702+0000","last_became_peered":"2024-09-13T17:50:34.203702+0000","last_unstale":"2024-09-13T17:50:39.000656+0000","last_undegraded":"2024-09-13T17:50:39.000656+0000","last_fullsized":"2024-09-13T17:50:39.000656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:44.431226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.995065+0000","last_change":"2024-09-13T17:50:38.615677+0000","last_active":"2024-09-13T17:50:38.995065+0000","last_peered":"2024-09-13T17:50:38.995065+0000","last_clean":"2024-09-13T17:50:38.995065+0000","last_became_active":"2024-09-13T17:50:38.615270+0000","last_became_peered":"2024-09-13T17:50:38.615270+0000","last_unstale":"2024-09-13T17:50:38.995065+0000","last_undegraded":"2024-09-13T17:50:38.995065+0000","last_fullsized":"2024-09-13T17:50:38.995065+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:55.768090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599593+0000","last_change":"2024-09-13T17:50:34.586802+0000","last_active":"2024-09-13T17:50:38.599593+0000","last_peered":"2024-09-13T17:50:38.599593+0000","last_clean":"2024-09-13T17:50:38.599593+0000","last_became_active":"2024-09-13T17:50:34.586552+0000","last_became_peered":"2024-09-13T17:50:34.586552+0000","last_unstale":"2024-09-13T17:50:38.599593+0000","last_undegraded":"2024-09-13T17:50:38.599593+0000","last_fullsized":"2024-09-13T17:50:38.599593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:43.383975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.592964+0000","last_change":"2024-09-13T17:50:34.217083+0000","last_active":"2024-09-13T17:50:38.592964+0000","last_peered":"2024-09-13T17:50:38.592964+0000","last_clean":"2024-09-13T17:50:38.592964+0000","last_became_active":"2024-09-13T17:50:34.216758+0000","last_became_peered":"2024-09-13T17:50:34.216758+0000","last_unstale":"2024-09-13T17:50:38.592964+0000","last_undegraded":"2024-09-13T17:50:38.592964+0000","last_fullsized":"2024-09-13T17:50:38.592964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:36.473942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.500902+0000","last_change":"2024-09-13T17:50:38.614667+0000","last_active":"2024-09-13T17:50:39.500902+0000","last_peered":"2024-09-13T17:50:39.500902+0000","last_clean":"2024-09-13T17:50:39.500902+0000","last_became_active":"2024-09-13T17:50:38.614284+0000","last_became_peered":"2024-09-13T17:50:38.614284+0000","last_unstale":"2024-09-13T17:50:39.500902+0000","last_undegraded":"2024-09-13T17:50:39.500902+0000","last_fullsized":"2024-09-13T17:50:39.500902+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:29.616794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591705+0000","last_change":"2024-09-13T17:50:34.588603+0000","last_active":"2024-09-13T17:50:38.591705+0000","last_peered":"2024-09-13T17:50:38.591705+0000","last_clean":"2024-09-13T17:50:38.591705+0000","last_became_active":"2024-09-13T17:50:34.588317+0000","last_became_peered":"2024-09-13T17:50:34.588317+0000","last_unstale":"2024-09-13T17:50:38.591705+0000","last_undegraded":"2024-09-13T17:50:38.591705+0000","last_fullsized":"2024-09-13T17:50:38.591705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:03.743130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599679+0000","last_change":"2024-09-13T17:50:34.204335+0000","last_active":"2024-09-13T17:50:38.599679+0000","last_peered":"2024-09-13T17:50:38.599679+0000","last_clean":"2024-09-13T17:50:38.599679+0000","last_became_active":"2024-09-13T17:50:34.204060+0000","last_became_peered":"2024-09-13T17:50:34.204060+0000","last_unstale":"2024-09-13T17:50:38.599679+0000","last_undegraded":"2024-09-13T17:50:38.599679+0000","last_fullsized":"2024-09-13T17:50:38.599679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:54:41.831344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.999822+0000","last_change":"2024-09-13T17:50:38.619590+0000","last_active":"2024-09-13T17:50:38.999822+0000","last_peered":"2024-09-13T17:50:38.999822+0000","last_clean":"2024-09-13T17:50:38.999822+0000","last_became_active":"2024-09-13T17:50:38.617257+0000","last_became_peered":"2024-09-13T17:50:38.617257+0000","last_unstale":"2024-09-13T17:50:38.999822+0000","last_undegraded":"2024-09-13T17:50:38.999822+0000","last_fullsized":"2024-09-13T17:50:38.999822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:34.870291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.602091+0000","last_change":"2024-09-13T17:50:34.588466+0000","last_active":"2024-09-13T17:50:38.602091+0000","last_peered":"2024-09-13T17:50:38.602091+0000","last_clean":"2024-09-13T17:50:38.602091+0000","last_became_active":"2024-09-13T17:50:34.588063+0000","last_became_peered":"2024-09-13T17:50:34.588063+0000","last_unstale":"2024-09-13T17:50:38.602091+0000","last_undegraded":"2024-09-13T17:50:38.602091+0000","last_fullsized":"2024-09-13T17:50:38.602091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:43:00.669203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589993+0000","last_change":"2024-09-13T17:50:34.209239+0000","last_active":"2024-09-13T17:50:38.589993+0000","last_peered":"2024-09-13T17:50:38.589993+0000","last_clean":"2024-09-13T17:50:38.589993+0000","last_became_active":"2024-09-13T17:50:34.208196+0000","last_became_peered":"2024-09-13T17:50:34.208196+0000","last_unstale":"2024-09-13T17:50:38.589993+0000","last_undegraded":"2024-09-13T17:50:38.589993+0000","last_fullsized":"2024-09-13T17:50:38.589993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:39.253917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000394+0000","last_change":"2024-09-13T17:50:38.611021+0000","last_active":"2024-09-13T17:50:39.000394+0000","last_peered":"2024-09-13T17:50:39.000394+0000","last_clean":"2024-09-13T17:50:39.000394+0000","last_became_active":"2024-09-13T17:50:38.609794+0000","last_became_peered":"2024-09-13T17:50:38.609794+0000","last_unstale":"2024-09-13T17:50:39.000394+0000","last_undegraded":"2024-09-13T17:50:39.000394+0000","last_fullsized":"2024-09-13T17:50:39.000394+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:59.980236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.600038+0000","last_change":"2024-09-13T17:50:34.584290+0000","last_active":"2024-09-13T17:50:38.600038+0000","last_peered":"2024-09-13T17:50:38.600038+0000","last_clean":"2024-09-13T17:50:38.600038+0000","last_became_active":"2024-09-13T17:50:34.584023+0000","last_became_peered":"2024-09-13T17:50:34.584023+0000","last_unstale":"2024-09-13T17:50:38.600038+0000","last_undegraded":"2024-09-13T17:50:38.600038+0000","last_fullsized":"2024-09-13T17:50:38.600038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:11.304986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.597167+0000","last_change":"2024-09-13T17:50:34.198932+0000","last_active":"2024-09-13T17:50:38.597167+0000","last_peered":"2024-09-13T17:50:38.597167+0000","last_clean":"2024-09-13T17:50:38.597167+0000","last_became_active":"2024-09-13T17:50:34.198603+0000","last_became_peered":"2024-09-13T17:50:34.198603+0000","last_unstale":"2024-09-13T17:50:38.597167+0000","last_undegraded":"2024-09-13T17:50:38.597167+0000","last_fullsized":"2024-09-13T17:50:38.597167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:00.917126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000707+0000","last_change":"2024-09-13T17:50:38.611326+0000","last_active":"2024-09-13T17:50:39.000707+0000","last_peered":"2024-09-13T17:50:39.000707+0000","last_clean":"2024-09-13T17:50:39.000707+0000","last_became_active":"2024-09-13T17:50:38.610900+0000","last_became_peered":"2024-09-13T17:50:38.610900+0000","last_unstale":"2024-09-13T17:50:39.000707+0000","last_undegraded":"2024-09-13T17:50:39.000707+0000","last_fullsized":"2024-09-13T17:50:39.000707+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:17:09.994736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.605384+0000","last_change":"2024-09-13T17:50:34.594482+0000","last_active":"2024-09-13T17:50:38.605384+0000","last_peered":"2024-09-13T17:50:38.605384+0000","last_clean":"2024-09-13T17:50:38.605384+0000","last_became_active":"2024-09-13T17:50:34.594201+0000","last_became_peered":"2024-09-13T17:50:34.594201+0000","last_unstale":"2024-09-13T17:50:38.605384+0000","last_undegraded":"2024-09-13T17:50:38.605384+0000","last_fullsized":"2024-09-13T17:50:38.605384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:06.697507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590063+0000","last_change":"2024-09-13T17:50:34.218376+0000","last_active":"2024-09-13T17:50:38.590063+0000","last_peered":"2024-09-13T17:50:38.590063+0000","last_clean":"2024-09-13T17:50:38.590063+0000","last_became_active":"2024-09-13T17:50:34.218069+0000","last_became_peered":"2024-09-13T17:50:34.218069+0000","last_unstale":"2024-09-13T17:50:38.590063+0000","last_undegraded":"2024-09-13T17:50:38.590063+0000","last_fullsized":"2024-09-13T17:50:38.590063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:53:54.288969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501171+0000","last_change":"2024-09-13T17:50:38.613206+0000","last_active":"2024-09-13T17:50:39.501171+0000","last_peered":"2024-09-13T17:50:39.501171+0000","last_clean":"2024-09-13T17:50:39.501171+0000","last_became_active":"2024-09-13T17:50:38.612856+0000","last_became_peered":"2024-09-13T17:50:38.612856+0000","last_unstale":"2024-09-13T17:50:39.501171+0000","last_undegraded":"2024-09-13T17:50:39.501171+0000","last_fullsized":"2024-09-13T17:50:39.501171+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:31:20.190996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.588765+0000","last_change":"2024-09-13T17:50:34.582931+0000","last_active":"2024-09-13T17:50:38.588765+0000","last_peered":"2024-09-13T17:50:38.588765+0000","last_clean":"2024-09-13T17:50:38.588765+0000","last_became_active":"2024-09-13T17:50:34.582446+0000","last_became_peered":"2024-09-13T17:50:34.582446+0000","last_unstale":"2024-09-13T17:50:38.588765+0000","last_undegraded":"2024-09-13T17:50:38.588765+0000","last_fullsized":"2024-09-13T17:50:38.588765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:30.554785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.588978+0000","last_change":"2024-09-13T17:50:34.201237+0000","last_active":"2024-09-13T17:50:38.588978+0000","last_peered":"2024-09-13T17:50:38.588978+0000","last_clean":"2024-09-13T17:50:38.588978+0000","last_became_active":"2024-09-13T17:50:34.200873+0000","last_became_peered":"2024-09-13T17:50:34.200873+0000","last_unstale":"2024-09-13T17:50:38.588978+0000","last_undegraded":"2024-09-13T17:50:38.588978+0000","last_fullsized":"2024-09-13T17:50:38.588978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:02.711342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590078+0000","last_change":"2024-09-13T17:50:22.095367+0000","last_active":"2024-09-13T17:50:38.590078+0000","last_peered":"2024-09-13T17:50:38.590078+0000","last_clean":"2024-09-13T17:50:38.590078+0000","last_became_active":"2024-09-13T17:50:22.093734+0000","last_became_peered":"2024-09-13T17:50:22.093734+0000","last_unstale":"2024-09-13T17:50:38.590078+0000","last_undegraded":"2024-09-13T17:50:38.590078+0000","last_fullsized":"2024-09-13T17:50:38.590078+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_clean_scrub_stamp":"2024-09-13T17:50:21.057812+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:23.531613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.982380+0000","last_change":"2024-09-13T17:50:38.618880+0000","last_active":"2024-09-13T17:50:38.982380+0000","last_peered":"2024-09-13T17:50:38.982380+0000","last_clean":"2024-09-13T17:50:38.982380+0000","last_became_active":"2024-09-13T17:50:38.618488+0000","last_became_peered":"2024-09-13T17:50:38.618488+0000","last_unstale":"2024-09-13T17:50:38.982380+0000","last_undegraded":"2024-09-13T17:50:38.982380+0000","last_fullsized":"2024-09-13T17:50:38.982380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:03.431876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.592679+0000","last_change":"2024-09-13T17:50:34.585881+0000","last_active":"2024-09-13T17:50:38.592679+0000","last_peered":"2024-09-13T17:50:38.592679+0000","last_clean":"2024-09-13T17:50:38.592679+0000","last_became_active":"2024-09-13T17:50:34.585166+0000","last_became_peered":"2024-09-13T17:50:34.585166+0000","last_unstale":"2024-09-13T17:50:38.592679+0000","last_undegraded":"2024-09-13T17:50:38.592679+0000","last_fullsized":"2024-09-13T17:50:38.592679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:06.654658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.602557+0000","last_change":"2024-09-13T17:50:34.210934+0000","last_active":"2024-09-13T17:50:38.602557+0000","last_peered":"2024-09-13T17:50:38.602557+0000","last_clean":"2024-09-13T17:50:38.602557+0000","last_became_active":"2024-09-13T17:50:34.210589+0000","last_became_peered":"2024-09-13T17:50:34.210589+0000","last_unstale":"2024-09-13T17:50:38.602557+0000","last_undegraded":"2024-09-13T17:50:38.602557+0000","last_fullsized":"2024-09-13T17:50:38.602557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:26.213295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.982569+0000","last_change":"2024-09-13T17:50:38.615811+0000","last_active":"2024-09-13T17:50:38.982569+0000","last_peered":"2024-09-13T17:50:38.982569+0000","last_clean":"2024-09-13T17:50:38.982569+0000","last_became_active":"2024-09-13T17:50:38.615358+0000","last_became_peered":"2024-09-13T17:50:38.615358+0000","last_unstale":"2024-09-13T17:50:38.982569+0000","last_undegraded":"2024-09-13T17:50:38.982569+0000","last_fullsized":"2024-09-13T17:50:38.982569+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:05.667393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590751+0000","last_change":"2024-09-13T17:50:34.590169+0000","last_active":"2024-09-13T17:50:38.590751+0000","last_peered":"2024-09-13T17:50:38.590751+0000","last_clean":"2024-09-13T17:50:38.590751+0000","last_became_active":"2024-09-13T17:50:34.589434+0000","last_became_peered":"2024-09-13T17:50:34.589434+0000","last_unstale":"2024-09-13T17:50:38.590751+0000","last_undegraded":"2024-09-13T17:50:38.590751+0000","last_fullsized":"2024-09-13T17:50:38.590751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:12.545412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000847+0000","last_change":"2024-09-13T17:50:34.204269+0000","last_active":"2024-09-13T17:50:39.000847+0000","last_peered":"2024-09-13T17:50:39.000847+0000","last_clean":"2024-09-13T17:50:39.000847+0000","last_became_active":"2024-09-13T17:50:34.203874+0000","last_became_peered":"2024-09-13T17:50:34.203874+0000","last_unstale":"2024-09-13T17:50:39.000847+0000","last_undegraded":"2024-09-13T17:50:39.000847+0000","last_fullsized":"2024-09-13T17:50:39.000847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:10.410267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501092+0000","last_change":"2024-09-13T17:50:38.615989+0000","last_active":"2024-09-13T17:50:39.501092+0000","last_peered":"2024-09-13T17:50:39.501092+0000","last_clean":"2024-09-13T17:50:39.501092+0000","last_became_active":"2024-09-13T17:50:38.615530+0000","last_became_peered":"2024-09-13T17:50:38.615530+0000","last_unstale":"2024-09-13T17:50:39.501092+0000","last_undegraded":"2024-09-13T17:50:39.501092+0000","last_fullsized":"2024-09-13T17:50:39.501092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:41.064092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.593217+0000","last_change":"2024-09-13T17:50:34.595120+0000","last_active":"2024-09-13T17:50:38.593217+0000","last_peered":"2024-09-13T17:50:38.593217+0000","last_clean":"2024-09-13T17:50:38.593217+0000","last_became_active":"2024-09-13T17:50:34.594842+0000","last_became_peered":"2024-09-13T17:50:34.594842+0000","last_unstale":"2024-09-13T17:50:38.593217+0000","last_undegraded":"2024-09-13T17:50:38.593217+0000","last_fullsized":"2024-09-13T17:50:38.593217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:15:38.982524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.587423+0000","last_change":"2024-09-13T17:50:34.199957+0000","last_active":"2024-09-13T17:50:39.587423+0000","last_peered":"2024-09-13T17:50:39.587423+0000","last_clean":"2024-09-13T17:50:39.587423+0000","last_became_active":"2024-09-13T17:50:34.199650+0000","last_became_peered":"2024-09-13T17:50:34.199650+0000","last_unstale":"2024-09-13T17:50:39.587423+0000","last_undegraded":"2024-09-13T17:50:39.587423+0000","last_fullsized":"2024-09-13T17:50:39.587423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:26.929549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.193989+0000","last_change":"2024-09-13T17:50:38.609731+0000","last_active":"2024-09-13T17:50:39.193989+0000","last_peered":"2024-09-13T17:50:39.193989+0000","last_clean":"2024-09-13T17:50:39.193989+0000","last_became_active":"2024-09-13T17:50:38.609354+0000","last_became_peered":"2024-09-13T17:50:38.609354+0000","last_unstale":"2024-09-13T17:50:39.193989+0000","last_undegraded":"2024-09-13T17:50:39.193989+0000","last_fullsized":"2024-09-13T17:50:39.193989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:14.142369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.593339+0000","last_change":"2024-09-13T17:50:34.595146+0000","last_active":"2024-09-13T17:50:38.593339+0000","last_peered":"2024-09-13T17:50:38.593339+0000","last_clean":"2024-09-13T17:50:38.593339+0000","last_became_active":"2024-09-13T17:50:34.594876+0000","last_became_peered":"2024-09-13T17:50:34.594876+0000","last_unstale":"2024-09-13T17:50:38.593339+0000","last_undegraded":"2024-09-13T17:50:38.593339+0000","last_fullsized":"2024-09-13T17:50:38.593339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:26:10.638438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.502430+0000","last_change":"2024-09-13T17:50:34.200640+0000","last_active":"2024-09-13T17:50:39.502430+0000","last_peered":"2024-09-13T17:50:39.502430+0000","last_clean":"2024-09-13T17:50:39.502430+0000","last_became_active":"2024-09-13T17:50:34.200197+0000","last_became_peered":"2024-09-13T17:50:34.200197+0000","last_unstale":"2024-09-13T17:50:39.502430+0000","last_undegraded":"2024-09-13T17:50:39.502430+0000","last_fullsized":"2024-09-13T17:50:39.502430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:37:53.039720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.500911+0000","last_change":"2024-09-13T17:50:38.619880+0000","last_active":"2024-09-13T17:50:39.500911+0000","last_peered":"2024-09-13T17:50:39.500911+0000","last_clean":"2024-09-13T17:50:39.500911+0000","last_became_active":"2024-09-13T17:50:38.618491+0000","last_became_peered":"2024-09-13T17:50:38.618491+0000","last_unstale":"2024-09-13T17:50:39.500911+0000","last_undegraded":"2024-09-13T17:50:39.500911+0000","last_fullsized":"2024-09-13T17:50:39.500911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:10:14.136054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590688+0000","last_change":"2024-09-13T17:50:34.585741+0000","last_active":"2024-09-13T17:50:38.590688+0000","last_peered":"2024-09-13T17:50:38.590688+0000","last_clean":"2024-09-13T17:50:38.590688+0000","last_became_active":"2024-09-13T17:50:34.585464+0000","last_became_peered":"2024-09-13T17:50:34.585464+0000","last_unstale":"2024-09-13T17:50:38.590688+0000","last_undegraded":"2024-09-13T17:50:38.590688+0000","last_fullsized":"2024-09-13T17:50:38.590688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:59:22.031459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"25'8","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.585286+0000","last_change":"2024-09-13T17:50:34.201272+0000","last_active":"2024-09-13T17:50:39.585286+0000","last_peered":"2024-09-13T17:50:39.585286+0000","last_clean":"2024-09-13T17:50:39.585286+0000","last_became_active":"2024-09-13T17:50:34.200936+0000","last_became_peered":"2024-09-13T17:50:34.200936+0000","last_unstale":"2024-09-13T17:50:39.585286+0000","last_undegraded":"2024-09-13T17:50:39.585286+0000","last_fullsized":"2024-09-13T17:50:39.585286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:52.094879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.500719+0000","last_change":"2024-09-13T17:50:38.615229+0000","last_active":"2024-09-13T17:50:39.500719+0000","last_peered":"2024-09-13T17:50:39.500719+0000","last_clean":"2024-09-13T17:50:39.500719+0000","last_became_active":"2024-09-13T17:50:38.614739+0000","last_became_peered":"2024-09-13T17:50:38.614739+0000","last_unstale":"2024-09-13T17:50:39.500719+0000","last_undegraded":"2024-09-13T17:50:39.500719+0000","last_fullsized":"2024-09-13T17:50:39.500719+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:18.619529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.604486+0000","last_change":"2024-09-13T17:50:34.588848+0000","last_active":"2024-09-13T17:50:38.604486+0000","last_peered":"2024-09-13T17:50:38.604486+0000","last_clean":"2024-09-13T17:50:38.604486+0000","last_became_active":"2024-09-13T17:50:34.588565+0000","last_became_peered":"2024-09-13T17:50:34.588565+0000","last_unstale":"2024-09-13T17:50:38.604486+0000","last_undegraded":"2024-09-13T17:50:38.604486+0000","last_fullsized":"2024-09-13T17:50:38.604486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:47.927865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.604169+0000","last_change":"2024-09-13T17:50:34.199008+0000","last_active":"2024-09-13T17:50:38.604169+0000","last_peered":"2024-09-13T17:50:38.604169+0000","last_clean":"2024-09-13T17:50:38.604169+0000","last_became_active":"2024-09-13T17:50:34.197839+0000","last_became_peered":"2024-09-13T17:50:34.197839+0000","last_unstale":"2024-09-13T17:50:38.604169+0000","last_undegraded":"2024-09-13T17:50:38.604169+0000","last_fullsized":"2024-09-13T17:50:38.604169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:51.430710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.996853+0000","last_change":"2024-09-13T17:50:38.611080+0000","last_active":"2024-09-13T17:50:38.996853+0000","last_peered":"2024-09-13T17:50:38.996853+0000","last_clean":"2024-09-13T17:50:38.996853+0000","last_became_active":"2024-09-13T17:50:38.610688+0000","last_became_peered":"2024-09-13T17:50:38.610688+0000","last_unstale":"2024-09-13T17:50:38.996853+0000","last_undegraded":"2024-09-13T17:50:38.996853+0000","last_fullsized":"2024-09-13T17:50:38.996853+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:28.527617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591242+0000","last_change":"2024-09-13T17:50:34.594763+0000","last_active":"2024-09-13T17:50:38.591242+0000","last_peered":"2024-09-13T17:50:38.591242+0000","last_clean":"2024-09-13T17:50:38.591242+0000","last_became_active":"2024-09-13T17:50:34.594471+0000","last_became_peered":"2024-09-13T17:50:34.594471+0000","last_unstale":"2024-09-13T17:50:38.591242+0000","last_undegraded":"2024-09-13T17:50:38.591242+0000","last_fullsized":"2024-09-13T17:50:38.591242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:20:58.718271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589771+0000","last_change":"2024-09-13T17:50:34.209432+0000","last_active":"2024-09-13T17:50:38.589771+0000","last_peered":"2024-09-13T17:50:38.589771+0000","last_clean":"2024-09-13T17:50:38.589771+0000","last_became_active":"2024-09-13T17:50:34.208817+0000","last_became_peered":"2024-09-13T17:50:34.208817+0000","last_unstale":"2024-09-13T17:50:38.589771+0000","last_undegraded":"2024-09-13T17:50:38.589771+0000","last_fullsized":"2024-09-13T17:50:38.589771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:49.587528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.995813+0000","last_change":"2024-09-13T17:50:38.609461+0000","last_active":"2024-09-13T17:50:38.995813+0000","last_peered":"2024-09-13T17:50:38.995813+0000","last_clean":"2024-09-13T17:50:38.995813+0000","last_became_active":"2024-09-13T17:50:38.609128+0000","last_became_peered":"2024-09-13T17:50:38.609128+0000","last_unstale":"2024-09-13T17:50:38.995813+0000","last_undegraded":"2024-09-13T17:50:38.995813+0000","last_fullsized":"2024-09-13T17:50:38.995813+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:02.286825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.604049+0000","last_change":"2024-09-13T17:50:34.592970+0000","last_active":"2024-09-13T17:50:38.604049+0000","last_peered":"2024-09-13T17:50:38.604049+0000","last_clean":"2024-09-13T17:50:38.604049+0000","last_became_active":"2024-09-13T17:50:34.592669+0000","last_became_peered":"2024-09-13T17:50:34.592669+0000","last_unstale":"2024-09-13T17:50:38.604049+0000","last_undegraded":"2024-09-13T17:50:38.604049+0000","last_fullsized":"2024-09-13T17:50:38.604049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:52.031261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583358+0000","last_change":"2024-09-13T17:50:34.200798+0000","last_active":"2024-09-13T17:50:39.583358+0000","last_peered":"2024-09-13T17:50:39.583358+0000","last_clean":"2024-09-13T17:50:39.583358+0000","last_became_active":"2024-09-13T17:50:34.200529+0000","last_became_peered":"2024-09-13T17:50:34.200529+0000","last_unstale":"2024-09-13T17:50:39.583358+0000","last_undegraded":"2024-09-13T17:50:39.583358+0000","last_fullsized":"2024-09-13T17:50:39.583358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:26.334296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.195285+0000","last_change":"2024-09-13T17:50:38.606287+0000","last_active":"2024-09-13T17:50:39.195285+0000","last_peered":"2024-09-13T17:50:39.195285+0000","last_clean":"2024-09-13T17:50:39.195285+0000","last_became_active":"2024-09-13T17:50:38.605826+0000","last_became_peered":"2024-09-13T17:50:38.605826+0000","last_unstale":"2024-09-13T17:50:39.195285+0000","last_undegraded":"2024-09-13T17:50:39.195285+0000","last_fullsized":"2024-09-13T17:50:39.195285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:18.709454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.592530+0000","last_change":"2024-09-13T17:50:34.591579+0000","last_active":"2024-09-13T17:50:38.592530+0000","last_peered":"2024-09-13T17:50:38.592530+0000","last_clean":"2024-09-13T17:50:38.592530+0000","last_became_active":"2024-09-13T17:50:34.586165+0000","last_became_peered":"2024-09-13T17:50:34.586165+0000","last_unstale":"2024-09-13T17:50:38.592530+0000","last_undegraded":"2024-09-13T17:50:38.592530+0000","last_fullsized":"2024-09-13T17:50:38.592530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:56:19.205176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:47.801705+0000","last_change":"2024-09-13T17:50:34.203218+0000","last_active":"2024-09-13T17:50:47.801705+0000","last_peered":"2024-09-13T17:50:47.801705+0000","last_clean":"2024-09-13T17:50:47.801705+0000","last_became_active":"2024-09-13T17:50:34.202948+0000","last_became_peered":"2024-09-13T17:50:34.202948+0000","last_unstale":"2024-09-13T17:50:47.801705+0000","last_undegraded":"2024-09-13T17:50:47.801705+0000","last_fullsized":"2024-09-13T17:50:47.801705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:46.535569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000203+0000","last_change":"2024-09-13T17:50:38.610597+0000","last_active":"2024-09-13T17:50:39.000203+0000","last_peered":"2024-09-13T17:50:39.000203+0000","last_clean":"2024-09-13T17:50:39.000203+0000","last_became_active":"2024-09-13T17:50:38.610055+0000","last_became_peered":"2024-09-13T17:50:38.610055+0000","last_unstale":"2024-09-13T17:50:39.000203+0000","last_undegraded":"2024-09-13T17:50:39.000203+0000","last_fullsized":"2024-09-13T17:50:39.000203+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:55.754936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.602441+0000","last_change":"2024-09-13T17:50:34.593963+0000","last_active":"2024-09-13T17:50:38.602441+0000","last_peered":"2024-09-13T17:50:38.602441+0000","last_clean":"2024-09-13T17:50:38.602441+0000","last_became_active":"2024-09-13T17:50:34.593454+0000","last_became_peered":"2024-09-13T17:50:34.593454+0000","last_unstale":"2024-09-13T17:50:38.602441+0000","last_undegraded":"2024-09-13T17:50:38.602441+0000","last_fullsized":"2024-09-13T17:50:38.602441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:18.883521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583588+0000","last_change":"2024-09-13T17:50:34.207936+0000","last_active":"2024-09-13T17:50:39.583588+0000","last_peered":"2024-09-13T17:50:39.583588+0000","last_clean":"2024-09-13T17:50:39.583588+0000","last_became_active":"2024-09-13T17:50:34.207605+0000","last_became_peered":"2024-09-13T17:50:34.207605+0000","last_unstale":"2024-09-13T17:50:39.583588+0000","last_undegraded":"2024-09-13T17:50:39.583588+0000","last_fullsized":"2024-09-13T17:50:39.583588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:05.494406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501828+0000","last_change":"2024-09-13T17:50:38.616297+0000","last_active":"2024-09-13T17:50:39.501828+0000","last_peered":"2024-09-13T17:50:39.501828+0000","last_clean":"2024-09-13T17:50:39.501828+0000","last_became_active":"2024-09-13T17:50:38.614165+0000","last_became_peered":"2024-09-13T17:50:38.614165+0000","last_unstale":"2024-09-13T17:50:39.501828+0000","last_undegraded":"2024-09-13T17:50:39.501828+0000","last_fullsized":"2024-09-13T17:50:39.501828+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:11.357026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.598628+0000","last_change":"2024-09-13T17:50:34.583001+0000","last_active":"2024-09-13T17:50:38.598628+0000","last_peered":"2024-09-13T17:50:38.598628+0000","last_clean":"2024-09-13T17:50:38.598628+0000","last_became_active":"2024-09-13T17:50:34.582529+0000","last_became_peered":"2024-09-13T17:50:34.582529+0000","last_unstale":"2024-09-13T17:50:38.598628+0000","last_undegraded":"2024-09-13T17:50:38.598628+0000","last_fullsized":"2024-09-13T17:50:38.598628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:05.287181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591968+0000","last_change":"2024-09-13T17:50:34.205138+0000","last_active":"2024-09-13T17:50:38.591968+0000","last_peered":"2024-09-13T17:50:38.591968+0000","last_clean":"2024-09-13T17:50:38.591968+0000","last_became_active":"2024-09-13T17:50:34.203360+0000","last_became_peered":"2024-09-13T17:50:34.203360+0000","last_unstale":"2024-09-13T17:50:38.591968+0000","last_undegraded":"2024-09-13T17:50:38.591968+0000","last_fullsized":"2024-09-13T17:50:38.591968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:35.756963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.995168+0000","last_change":"2024-09-13T17:50:38.611036+0000","last_active":"2024-09-13T17:50:38.995168+0000","last_peered":"2024-09-13T17:50:38.995168+0000","last_clean":"2024-09-13T17:50:38.995168+0000","last_became_active":"2024-09-13T17:50:38.610692+0000","last_became_peered":"2024-09-13T17:50:38.610692+0000","last_unstale":"2024-09-13T17:50:38.995168+0000","last_undegraded":"2024-09-13T17:50:38.995168+0000","last_fullsized":"2024-09-13T17:50:38.995168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:10.147998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591770+0000","last_change":"2024-09-13T17:50:34.596882+0000","last_active":"2024-09-13T17:50:38.591770+0000","last_peered":"2024-09-13T17:50:38.591770+0000","last_clean":"2024-09-13T17:50:38.591770+0000","last_became_active":"2024-09-13T17:50:34.596495+0000","last_became_peered":"2024-09-13T17:50:34.596495+0000","last_unstale":"2024-09-13T17:50:38.591770+0000","last_undegraded":"2024-09-13T17:50:38.591770+0000","last_fullsized":"2024-09-13T17:50:38.591770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:21.441114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.609505+0000","last_change":"2024-09-13T17:50:34.204376+0000","last_active":"2024-09-13T17:50:38.609505+0000","last_peered":"2024-09-13T17:50:38.609505+0000","last_clean":"2024-09-13T17:50:38.609505+0000","last_became_active":"2024-09-13T17:50:34.204003+0000","last_became_peered":"2024-09-13T17:50:34.204003+0000","last_unstale":"2024-09-13T17:50:38.609505+0000","last_undegraded":"2024-09-13T17:50:38.609505+0000","last_fullsized":"2024-09-13T17:50:38.609505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:14.023990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.995437+0000","last_change":"2024-09-13T17:50:38.606144+0000","last_active":"2024-09-13T17:50:38.995437+0000","last_peered":"2024-09-13T17:50:38.995437+0000","last_clean":"2024-09-13T17:50:38.995437+0000","last_became_active":"2024-09-13T17:50:38.605646+0000","last_became_peered":"2024-09-13T17:50:38.605646+0000","last_unstale":"2024-09-13T17:50:38.995437+0000","last_undegraded":"2024-09-13T17:50:38.995437+0000","last_fullsized":"2024-09-13T17:50:38.995437+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:07.120075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.500554+0000","last_change":"2024-09-13T17:50:38.618855+0000","last_active":"2024-09-13T17:50:39.500554+0000","last_peered":"2024-09-13T17:50:39.500554+0000","last_clean":"2024-09-13T17:50:39.500554+0000","last_became_active":"2024-09-13T17:50:38.618449+0000","last_became_peered":"2024-09-13T17:50:38.618449+0000","last_unstale":"2024-09-13T17:50:39.500554+0000","last_undegraded":"2024-09-13T17:50:39.500554+0000","last_fullsized":"2024-09-13T17:50:39.500554+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:29:01.114087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.600195+0000","last_change":"2024-09-13T17:50:34.590079+0000","last_active":"2024-09-13T17:50:38.600195+0000","last_peered":"2024-09-13T17:50:38.600195+0000","last_clean":"2024-09-13T17:50:38.600195+0000","last_became_active":"2024-09-13T17:50:34.589396+0000","last_became_peered":"2024-09-13T17:50:34.589396+0000","last_unstale":"2024-09-13T17:50:38.600195+0000","last_undegraded":"2024-09-13T17:50:38.600195+0000","last_fullsized":"2024-09-13T17:50:38.600195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:03.308341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589925+0000","last_change":"2024-09-13T17:50:34.206575+0000","last_active":"2024-09-13T17:50:38.589925+0000","last_peered":"2024-09-13T17:50:38.589925+0000","last_clean":"2024-09-13T17:50:38.589925+0000","last_became_active":"2024-09-13T17:50:34.206308+0000","last_became_peered":"2024-09-13T17:50:34.206308+0000","last_unstale":"2024-09-13T17:50:38.589925+0000","last_undegraded":"2024-09-13T17:50:38.589925+0000","last_fullsized":"2024-09-13T17:50:38.589925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:34.766538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.982710+0000","last_change":"2024-09-13T17:50:38.618131+0000","last_active":"2024-09-13T17:50:38.982710+0000","last_peered":"2024-09-13T17:50:38.982710+0000","last_clean":"2024-09-13T17:50:38.982710+0000","last_became_active":"2024-09-13T17:50:38.617611+0000","last_became_peered":"2024-09-13T17:50:38.617611+0000","last_unstale":"2024-09-13T17:50:38.982710+0000","last_undegraded":"2024-09-13T17:50:38.982710+0000","last_fullsized":"2024-09-13T17:50:38.982710+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:30.746445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589814+0000","last_change":"2024-09-13T17:50:34.582153+0000","last_active":"2024-09-13T17:50:38.589814+0000","last_peered":"2024-09-13T17:50:38.589814+0000","last_clean":"2024-09-13T17:50:38.589814+0000","last_became_active":"2024-09-13T17:50:34.581708+0000","last_became_peered":"2024-09-13T17:50:34.581708+0000","last_unstale":"2024-09-13T17:50:38.589814+0000","last_undegraded":"2024-09-13T17:50:38.589814+0000","last_fullsized":"2024-09-13T17:50:38.589814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:22.238810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590175+0000","last_change":"2024-09-13T17:50:34.205489+0000","last_active":"2024-09-13T17:50:38.590175+0000","last_peered":"2024-09-13T17:50:38.590175+0000","last_clean":"2024-09-13T17:50:38.590175+0000","last_became_active":"2024-09-13T17:50:34.205202+0000","last_became_peered":"2024-09-13T17:50:34.205202+0000","last_unstale":"2024-09-13T17:50:38.590175+0000","last_undegraded":"2024-09-13T17:50:38.590175+0000","last_fullsized":"2024-09-13T17:50:38.590175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:54.450276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.983249+0000","last_change":"2024-09-13T17:50:34.202470+0000","last_active":"2024-09-13T17:50:38.983249+0000","last_peered":"2024-09-13T17:50:38.983249+0000","last_clean":"2024-09-13T17:50:38.983249+0000","last_became_active":"2024-09-13T17:50:34.202163+0000","last_became_peered":"2024-09-13T17:50:34.202163+0000","last_unstale":"2024-09-13T17:50:38.983249+0000","last_undegraded":"2024-09-13T17:50:38.983249+0000","last_fullsized":"2024-09-13T17:50:38.983249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:03.472932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.603393+0000","last_change":"2024-09-13T17:50:34.590947+0000","last_active":"2024-09-13T17:50:38.603393+0000","last_peered":"2024-09-13T17:50:38.603393+0000","last_clean":"2024-09-13T17:50:38.603393+0000","last_became_active":"2024-09-13T17:50:34.590497+0000","last_became_peered":"2024-09-13T17:50:34.590497+0000","last_unstale":"2024-09-13T17:50:38.603393+0000","last_undegraded":"2024-09-13T17:50:38.603393+0000","last_fullsized":"2024-09-13T17:50:38.603393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:56:27.848092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.980209+0000","last_change":"2024-09-13T17:50:38.618518+0000","last_active":"2024-09-13T17:50:38.980209+0000","last_peered":"2024-09-13T17:50:38.980209+0000","last_clean":"2024-09-13T17:50:38.980209+0000","last_became_active":"2024-09-13T17:50:38.618108+0000","last_became_peered":"2024-09-13T17:50:38.618108+0000","last_unstale":"2024-09-13T17:50:38.980209+0000","last_undegraded":"2024-09-13T17:50:38.980209+0000","last_fullsized":"2024-09-13T17:50:38.980209+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:42.260237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"25'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.996384+0000","last_change":"2024-09-13T17:50:34.218459+0000","last_active":"2024-09-13T17:50:38.996384+0000","last_peered":"2024-09-13T17:50:38.996384+0000","last_clean":"2024-09-13T17:50:38.996384+0000","last_became_active":"2024-09-13T17:50:34.218087+0000","last_became_peered":"2024-09-13T17:50:34.218087+0000","last_unstale":"2024-09-13T17:50:38.996384+0000","last_undegraded":"2024-09-13T17:50:38.996384+0000","last_fullsized":"2024-09-13T17:50:38.996384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:56.914544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.603583+0000","last_change":"2024-09-13T17:50:34.583845+0000","last_active":"2024-09-13T17:50:38.603583+0000","last_peered":"2024-09-13T17:50:38.603583+0000","last_clean":"2024-09-13T17:50:38.603583+0000","last_became_active":"2024-09-13T17:50:34.583283+0000","last_became_peered":"2024-09-13T17:50:34.583283+0000","last_unstale":"2024-09-13T17:50:38.603583+0000","last_undegraded":"2024-09-13T17:50:38.603583+0000","last_fullsized":"2024-09-13T17:50:38.603583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:55.630285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.996396+0000","last_change":"2024-09-13T17:50:38.611987+0000","last_active":"2024-09-13T17:50:38.996396+0000","last_peered":"2024-09-13T17:50:38.996396+0000","last_clean":"2024-09-13T17:50:38.996396+0000","last_became_active":"2024-09-13T17:50:38.611636+0000","last_became_peered":"2024-09-13T17:50:38.611636+0000","last_unstale":"2024-09-13T17:50:38.996396+0000","last_undegraded":"2024-09-13T17:50:38.996396+0000","last_fullsized":"2024-09-13T17:50:38.996396+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:41.111562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.602963+0000","last_change":"2024-09-13T17:50:34.588927+0000","last_active":"2024-09-13T17:50:38.602963+0000","last_peered":"2024-09-13T17:50:38.602963+0000","last_clean":"2024-09-13T17:50:38.602963+0000","last_became_active":"2024-09-13T17:50:34.588651+0000","last_became_peered":"2024-09-13T17:50:34.588651+0000","last_unstale":"2024-09-13T17:50:38.602963+0000","last_undegraded":"2024-09-13T17:50:38.602963+0000","last_fullsized":"2024-09-13T17:50:38.602963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:49.429670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.597005+0000","last_change":"2024-09-13T17:50:34.583795+0000","last_active":"2024-09-13T17:50:38.597005+0000","last_peered":"2024-09-13T17:50:38.597005+0000","last_clean":"2024-09-13T17:50:38.597005+0000","last_became_active":"2024-09-13T17:50:34.583493+0000","last_became_peered":"2024-09-13T17:50:34.583493+0000","last_unstale":"2024-09-13T17:50:38.597005+0000","last_undegraded":"2024-09-13T17:50:38.597005+0000","last_fullsized":"2024-09-13T17:50:38.597005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:30.463932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589528+0000","last_change":"2024-09-13T17:50:34.587624+0000","last_active":"2024-09-13T17:50:38.589528+0000","last_peered":"2024-09-13T17:50:38.589528+0000","last_clean":"2024-09-13T17:50:38.589528+0000","last_became_active":"2024-09-13T17:50:34.587119+0000","last_became_peered":"2024-09-13T17:50:34.587119+0000","last_unstale":"2024-09-13T17:50:38.589528+0000","last_undegraded":"2024-09-13T17:50:38.589528+0000","last_fullsized":"2024-09-13T17:50:38.589528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:16.783899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599861+0000","last_change":"2024-09-13T17:50:34.581923+0000","last_active":"2024-09-13T17:50:38.599861+0000","last_peered":"2024-09-13T17:50:38.599861+0000","last_clean":"2024-09-13T17:50:38.599861+0000","last_became_active":"2024-09-13T17:50:34.581628+0000","last_became_peered":"2024-09-13T17:50:34.581628+0000","last_unstale":"2024-09-13T17:50:38.599861+0000","last_undegraded":"2024-09-13T17:50:38.599861+0000","last_fullsized":"2024-09-13T17:50:38.599861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:28:55.936864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589257+0000","last_change":"2024-09-13T17:50:34.595037+0000","last_active":"2024-09-13T17:50:38.589257+0000","last_peered":"2024-09-13T17:50:38.589257+0000","last_clean":"2024-09-13T17:50:38.589257+0000","last_became_active":"2024-09-13T17:50:34.594857+0000","last_became_peered":"2024-09-13T17:50:34.594857+0000","last_unstale":"2024-09-13T17:50:38.589257+0000","last_undegraded":"2024-09-13T17:50:38.589257+0000","last_fullsized":"2024-09-13T17:50:38.589257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:11.420413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.597995+0000","last_change":"2024-09-13T17:50:34.589651+0000","last_active":"2024-09-13T17:50:38.597995+0000","last_peered":"2024-09-13T17:50:38.597995+0000","last_clean":"2024-09-13T17:50:38.597995+0000","last_became_active":"2024-09-13T17:50:34.589354+0000","last_became_peered":"2024-09-13T17:50:34.589354+0000","last_unstale":"2024-09-13T17:50:38.597995+0000","last_undegraded":"2024-09-13T17:50:38.597995+0000","last_fullsized":"2024-09-13T17:50:38.597995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:31.294048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.609987+0000","last_change":"2024-09-13T17:50:34.597949+0000","last_active":"2024-09-13T17:50:38.609987+0000","last_peered":"2024-09-13T17:50:38.609987+0000","last_clean":"2024-09-13T17:50:38.609987+0000","last_became_active":"2024-09-13T17:50:34.597621+0000","last_became_peered":"2024-09-13T17:50:34.597621+0000","last_unstale":"2024-09-13T17:50:38.609987+0000","last_undegraded":"2024-09-13T17:50:38.609987+0000","last_fullsized":"2024-09-13T17:50:38.609987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:23.133462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591133+0000","last_change":"2024-09-13T17:50:34.589893+0000","last_active":"2024-09-13T17:50:38.591133+0000","last_peered":"2024-09-13T17:50:38.591133+0000","last_clean":"2024-09-13T17:50:38.591133+0000","last_became_active":"2024-09-13T17:50:34.589626+0000","last_became_peered":"2024-09-13T17:50:34.589626+0000","last_unstale":"2024-09-13T17:50:38.591133+0000","last_undegraded":"2024-09-13T17:50:38.591133+0000","last_fullsized":"2024-09-13T17:50:38.591133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:41.321025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589874+0000","last_change":"2024-09-13T17:50:34.590046+0000","last_active":"2024-09-13T17:50:38.589874+0000","last_peered":"2024-09-13T17:50:38.589874+0000","last_clean":"2024-09-13T17:50:38.589874+0000","last_became_active":"2024-09-13T17:50:34.589757+0000","last_became_peered":"2024-09-13T17:50:34.589757+0000","last_unstale":"2024-09-13T17:50:38.589874+0000","last_undegraded":"2024-09-13T17:50:38.589874+0000","last_fullsized":"2024-09-13T17:50:38.589874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:28.239550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589800+0000","last_change":"2024-09-13T17:50:34.595518+0000","last_active":"2024-09-13T17:50:38.589800+0000","last_peered":"2024-09-13T17:50:38.589800+0000","last_clean":"2024-09-13T17:50:38.589800+0000","last_became_active":"2024-09-13T17:50:34.595242+0000","last_became_peered":"2024-09-13T17:50:34.595242+0000","last_unstale":"2024-09-13T17:50:38.589800+0000","last_undegraded":"2024-09-13T17:50:38.589800+0000","last_fullsized":"2024-09-13T17:50:38.589800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:26.149384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589330+0000","last_change":"2024-09-13T17:50:34.593183+0000","last_active":"2024-09-13T17:50:38.589330+0000","last_peered":"2024-09-13T17:50:38.589330+0000","last_clean":"2024-09-13T17:50:38.589330+0000","last_became_active":"2024-09-13T17:50:34.592893+0000","last_became_peered":"2024-09-13T17:50:34.592893+0000","last_unstale":"2024-09-13T17:50:38.589330+0000","last_undegraded":"2024-09-13T17:50:38.589330+0000","last_fullsized":"2024-09-13T17:50:38.589330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:19.459505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599698+0000","last_change":"2024-09-13T17:50:34.594679+0000","last_active":"2024-09-13T17:50:38.599698+0000","last_peered":"2024-09-13T17:50:38.599698+0000","last_clean":"2024-09-13T17:50:38.599698+0000","last_became_active":"2024-09-13T17:50:34.593750+0000","last_became_peered":"2024-09-13T17:50:34.593750+0000","last_unstale":"2024-09-13T17:50:38.599698+0000","last_undegraded":"2024-09-13T17:50:38.599698+0000","last_fullsized":"2024-09-13T17:50:38.599698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:02.135577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.609111+0000","last_change":"2024-09-13T17:50:34.598736+0000","last_active":"2024-09-13T17:50:38.609111+0000","last_peered":"2024-09-13T17:50:38.609111+0000","last_clean":"2024-09-13T17:50:38.609111+0000","last_became_active":"2024-09-13T17:50:34.598433+0000","last_became_peered":"2024-09-13T17:50:34.598433+0000","last_unstale":"2024-09-13T17:50:38.609111+0000","last_undegraded":"2024-09-13T17:50:38.609111+0000","last_fullsized":"2024-09-13T17:50:38.609111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:08.975139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.592906+0000","last_change":"2024-09-13T17:50:34.591795+0000","last_active":"2024-09-13T17:50:38.592906+0000","last_peered":"2024-09-13T17:50:38.592906+0000","last_clean":"2024-09-13T17:50:38.592906+0000","last_became_active":"2024-09-13T17:50:34.591286+0000","last_became_peered":"2024-09-13T17:50:34.591286+0000","last_unstale":"2024-09-13T17:50:38.592906+0000","last_undegraded":"2024-09-13T17:50:38.592906+0000","last_fullsized":"2024-09-13T17:50:38.592906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:31.580247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.595844+0000","last_change":"2024-09-13T17:50:34.587774+0000","last_active":"2024-09-13T17:50:38.595844+0000","last_peered":"2024-09-13T17:50:38.595844+0000","last_clean":"2024-09-13T17:50:38.595844+0000","last_became_active":"2024-09-13T17:50:34.587506+0000","last_became_peered":"2024-09-13T17:50:34.587506+0000","last_unstale":"2024-09-13T17:50:38.595844+0000","last_undegraded":"2024-09-13T17:50:38.595844+0000","last_fullsized":"2024-09-13T17:50:38.595844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:46.275528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591171+0000","last_change":"2024-09-13T17:50:34.596100+0000","last_active":"2024-09-13T17:50:38.591171+0000","last_peered":"2024-09-13T17:50:38.591171+0000","last_clean":"2024-09-13T17:50:38.591171+0000","last_became_active":"2024-09-13T17:50:34.595798+0000","last_became_peered":"2024-09-13T17:50:34.595798+0000","last_unstale":"2024-09-13T17:50:38.591171+0000","last_undegraded":"2024-09-13T17:50:38.591171+0000","last_fullsized":"2024-09-13T17:50:38.591171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:49.543845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589759+0000","last_change":"2024-09-13T17:50:34.584818+0000","last_active":"2024-09-13T17:50:38.589759+0000","last_peered":"2024-09-13T17:50:38.589759+0000","last_clean":"2024-09-13T17:50:38.589759+0000","last_became_active":"2024-09-13T17:50:34.584516+0000","last_became_peered":"2024-09-13T17:50:34.584516+0000","last_unstale":"2024-09-13T17:50:38.589759+0000","last_undegraded":"2024-09-13T17:50:38.589759+0000","last_fullsized":"2024-09-13T17:50:38.589759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:52.402024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599531+0000","last_change":"2024-09-13T17:50:34.598886+0000","last_active":"2024-09-13T17:50:38.599531+0000","last_peered":"2024-09-13T17:50:38.599531+0000","last_clean":"2024-09-13T17:50:38.599531+0000","last_became_active":"2024-09-13T17:50:34.598648+0000","last_became_peered":"2024-09-13T17:50:34.598648+0000","last_unstale":"2024-09-13T17:50:38.599531+0000","last_undegraded":"2024-09-13T17:50:38.599531+0000","last_fullsized":"2024-09-13T17:50:38.599531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:00:43.063223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.603939+0000","last_change":"2024-09-13T17:50:34.582649+0000","last_active":"2024-09-13T17:50:38.603939+0000","last_peered":"2024-09-13T17:50:38.603939+0000","last_clean":"2024-09-13T17:50:38.603939+0000","last_became_active":"2024-09-13T17:50:34.581699+0000","last_became_peered":"2024-09-13T17:50:34.581699+0000","last_unstale":"2024-09-13T17:50:38.603939+0000","last_undegraded":"2024-09-13T17:50:38.603939+0000","last_fullsized":"2024-09-13T17:50:38.603939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:18.959303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.588487+0000","last_change":"2024-09-13T17:50:34.587484+0000","last_active":"2024-09-13T17:50:38.588487+0000","last_peered":"2024-09-13T17:50:38.588487+0000","last_clean":"2024-09-13T17:50:38.588487+0000","last_became_active":"2024-09-13T17:50:34.587217+0000","last_became_peered":"2024-09-13T17:50:34.587217+0000","last_unstale":"2024-09-13T17:50:38.588487+0000","last_undegraded":"2024-09-13T17:50:38.588487+0000","last_fullsized":"2024-09-13T17:50:38.588487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:24:19.172607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.602349+0000","last_change":"2024-09-13T17:50:34.592871+0000","last_active":"2024-09-13T17:50:38.602349+0000","last_peered":"2024-09-13T17:50:38.602349+0000","last_clean":"2024-09-13T17:50:38.602349+0000","last_became_active":"2024-09-13T17:50:34.588513+0000","last_became_peered":"2024-09-13T17:50:34.588513+0000","last_unstale":"2024-09-13T17:50:38.602349+0000","last_undegraded":"2024-09-13T17:50:38.602349+0000","last_fullsized":"2024-09-13T17:50:38.602349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:33.131420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.608986+0000","last_change":"2024-09-13T17:50:34.585713+0000","last_active":"2024-09-13T17:50:38.608986+0000","last_peered":"2024-09-13T17:50:38.608986+0000","last_clean":"2024-09-13T17:50:38.608986+0000","last_became_active":"2024-09-13T17:50:34.585438+0000","last_became_peered":"2024-09-13T17:50:34.585438+0000","last_unstale":"2024-09-13T17:50:38.608986+0000","last_undegraded":"2024-09-13T17:50:38.608986+0000","last_fullsized":"2024-09-13T17:50:38.608986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:56:49.905393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.609903+0000","last_change":"2024-09-13T17:50:34.589628+0000","last_active":"2024-09-13T17:50:38.609903+0000","last_peered":"2024-09-13T17:50:38.609903+0000","last_clean":"2024-09-13T17:50:38.609903+0000","last_became_active":"2024-09-13T17:50:34.587665+0000","last_became_peered":"2024-09-13T17:50:34.587665+0000","last_unstale":"2024-09-13T17:50:38.609903+0000","last_undegraded":"2024-09-13T17:50:38.609903+0000","last_fullsized":"2024-09-13T17:50:38.609903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:15.518264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.593116+0000","last_change":"2024-09-13T17:50:34.591776+0000","last_active":"2024-09-13T17:50:38.593116+0000","last_peered":"2024-09-13T17:50:38.593116+0000","last_clean":"2024-09-13T17:50:38.593116+0000","last_became_active":"2024-09-13T17:50:34.591496+0000","last_became_peered":"2024-09-13T17:50:34.591496+0000","last_unstale":"2024-09-13T17:50:38.593116+0000","last_undegraded":"2024-09-13T17:50:38.593116+0000","last_fullsized":"2024-09-13T17:50:38.593116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:48.651999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.588146+0000","last_change":"2024-09-13T17:50:34.582661+0000","last_active":"2024-09-13T17:50:38.588146+0000","last_peered":"2024-09-13T17:50:38.588146+0000","last_clean":"2024-09-13T17:50:38.588146+0000","last_became_active":"2024-09-13T17:50:34.582400+0000","last_became_peered":"2024-09-13T17:50:34.582400+0000","last_unstale":"2024-09-13T17:50:38.588146+0000","last_undegraded":"2024-09-13T17:50:38.588146+0000","last_fullsized":"2024-09-13T17:50:38.588146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:16:59.648638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589018+0000","last_change":"2024-09-13T17:50:34.585531+0000","last_active":"2024-09-13T17:50:38.589018+0000","last_peered":"2024-09-13T17:50:38.589018+0000","last_clean":"2024-09-13T17:50:38.589018+0000","last_became_active":"2024-09-13T17:50:34.585226+0000","last_became_peered":"2024-09-13T17:50:34.585226+0000","last_unstale":"2024-09-13T17:50:38.589018+0000","last_undegraded":"2024-09-13T17:50:38.589018+0000","last_fullsized":"2024-09-13T17:50:38.589018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:17.105028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589126+0000","last_change":"2024-09-13T17:50:34.586750+0000","last_active":"2024-09-13T17:50:38.589126+0000","last_peered":"2024-09-13T17:50:38.589126+0000","last_clean":"2024-09-13T17:50:38.589126+0000","last_became_active":"2024-09-13T17:50:34.584564+0000","last_became_peered":"2024-09-13T17:50:34.584564+0000","last_unstale":"2024-09-13T17:50:38.589126+0000","last_undegraded":"2024-09-13T17:50:38.589126+0000","last_fullsized":"2024-09-13T17:50:38.589126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:54.033257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591038+0000","last_change":"2024-09-13T17:50:34.585938+0000","last_active":"2024-09-13T17:50:38.591038+0000","last_peered":"2024-09-13T17:50:38.591038+0000","last_clean":"2024-09-13T17:50:38.591038+0000","last_became_active":"2024-09-13T17:50:34.585584+0000","last_became_peered":"2024-09-13T17:50:34.585584+0000","last_unstale":"2024-09-13T17:50:38.591038+0000","last_undegraded":"2024-09-13T17:50:38.591038+0000","last_fullsized":"2024-09-13T17:50:38.591038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:32.623468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590323+0000","last_change":"2024-09-13T17:50:34.588762+0000","last_active":"2024-09-13T17:50:38.590323+0000","last_peered":"2024-09-13T17:50:38.590323+0000","last_clean":"2024-09-13T17:50:38.590323+0000","last_became_active":"2024-09-13T17:50:34.588489+0000","last_became_peered":"2024-09-13T17:50:34.588489+0000","last_unstale":"2024-09-13T17:50:38.590323+0000","last_undegraded":"2024-09-13T17:50:38.590323+0000","last_fullsized":"2024-09-13T17:50:38.590323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:38.266297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.598178+0000","last_change":"2024-09-13T17:50:34.583889+0000","last_active":"2024-09-13T17:50:38.598178+0000","last_peered":"2024-09-13T17:50:38.598178+0000","last_clean":"2024-09-13T17:50:38.598178+0000","last_became_active":"2024-09-13T17:50:34.583551+0000","last_became_peered":"2024-09-13T17:50:34.583551+0000","last_unstale":"2024-09-13T17:50:38.598178+0000","last_undegraded":"2024-09-13T17:50:38.598178+0000","last_fullsized":"2024-09-13T17:50:38.598178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:19.888626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599385+0000","last_change":"2024-09-13T17:50:34.585617+0000","last_active":"2024-09-13T17:50:38.599385+0000","last_peered":"2024-09-13T17:50:38.599385+0000","last_clean":"2024-09-13T17:50:38.599385+0000","last_became_active":"2024-09-13T17:50:34.585340+0000","last_became_peered":"2024-09-13T17:50:34.585340+0000","last_unstale":"2024-09-13T17:50:38.599385+0000","last_undegraded":"2024-09-13T17:50:38.599385+0000","last_fullsized":"2024-09-13T17:50:38.599385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:18.331045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.593186+0000","last_change":"2024-09-13T17:50:34.592365+0000","last_active":"2024-09-13T17:50:38.593186+0000","last_peered":"2024-09-13T17:50:38.593186+0000","last_clean":"2024-09-13T17:50:38.593186+0000","last_became_active":"2024-09-13T17:50:34.592046+0000","last_became_peered":"2024-09-13T17:50:34.592046+0000","last_unstale":"2024-09-13T17:50:38.593186+0000","last_undegraded":"2024-09-13T17:50:38.593186+0000","last_fullsized":"2024-09-13T17:50:38.593186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:54.283683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509451,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":105750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":258048,"data_stored":101955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":102115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509452,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":864256,"data_stored":692961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542155,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":856064,"data_stored":692905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]}]},{"osd":6,"up_from":14,"seq":60129542155,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":270336,"data_stored":105750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5429999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3039999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6679999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7490000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542155,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":262144,"data_stored":102045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3360000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.357}]}]},{"osd":4,"up_from":14,"seq":60129542155,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":270336,"data_stored":102081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:51:08.446 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:51:08.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.550+0000 7f4f233dd700 1 Processor -- start 2024-09-13T17:51:08.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.550+0000 7f4f233dd700 1 -- start start 2024-09-13T17:51:08.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.550+0000 7f4f233dd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4f1c1500c0 0x7f4f1c0c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:08.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.550+0000 7f4f233dd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4f1c070500 0x7f4f1c0c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:08.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.550+0000 7f4f233dd700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1c0cba20 con 0x7f4f1c071b40 2024-09-13T17:51:08.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.550+0000 7f4f233dd700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f1c1560e0 con 0x7f4f1c070500 2024-09-13T17:51:08.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.550+0000 7f4f233dd700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f1c1563c0 con 0x7f4f1c1500c0 2024-09-13T17:51:08.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.550+0000 7f4f21bda700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4f1c070500 0x7f4f1c0c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:08.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.550+0000 7f4f21bda700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4f1c070500 0x7f4f1c0c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58220/0 (socket says 172.21.15.40:58220) 2024-09-13T17:51:08.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.550+0000 7f4f21bda700 1 -- 172.21.15.40:0/4173799600 learned_addr learned my addr 172.21.15.40:0/4173799600 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:08.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.550+0000 7f4f223db700 1 --2- 172.21.15.40:0/4173799600 >> v2:172.21.15.43:3300/0 conn(0x7f4f1c1500c0 0x7f4f1c0c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:08.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.550+0000 7f4f21bda700 1 -- 172.21.15.40:0/4173799600 >> v2:172.21.15.43:3300/0 conn(0x7f4f1c1500c0 msgr2=0x7f4f1c0c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:08.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.550+0000 7f4f21bda700 1 --2- 172.21.15.40:0/4173799600 >> v2:172.21.15.43:3300/0 conn(0x7f4f1c1500c0 0x7f4f1c0c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.550+0000 7f4f21bda700 1 -- 172.21.15.40:0/4173799600 >> v1:172.21.15.43:6789/0 conn(0x7f4f1c071b40 legacy=0x7f4f1c071f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.550+0000 7f4f21bda700 1 -- 172.21.15.40:0/4173799600 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c1566a0 con 0x7f4f1c070500 2024-09-13T17:51:08.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.554+0000 7f4f21bda700 1 --2- 172.21.15.40:0/4173799600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f1c070500 0x7f4f1c0c6c10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0c1f90 tx=0x7f4f1000be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48a575c1ab4aeaf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:08.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.554+0000 7f4f213d9700 1 -- 172.21.15.40:0/4173799600 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1000dd30 con 0x7f4f1c070500 2024-09-13T17:51:08.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.554+0000 7f4f213d9700 1 -- 172.21.15.40:0/4173799600 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1000de90 con 0x7f4f1c070500 2024-09-13T17:51:08.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.554+0000 7f4f213d9700 1 -- 172.21.15.40:0/4173799600 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10010690 con 0x7f4f1c070500 2024-09-13T17:51:08.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.554+0000 7f4f233dd700 1 -- 172.21.15.40:0/4173799600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f1c070500 msgr2=0x7f4f1c0c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.554+0000 7f4f233dd700 1 --2- 172.21.15.40:0/4173799600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f1c070500 0x7f4f1c0c6c10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0c1f90 tx=0x7f4f1000be90 comp rx=0 tx=0).stop 2024-09-13T17:51:08.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.554+0000 7f4f233dd700 1 -- 172.21.15.40:0/4173799600 shutdown_connections 2024-09-13T17:51:08.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.554+0000 7f4f233dd700 1 --2- 172.21.15.40:0/4173799600 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f1c070500 0x7f4f1c0c6c10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.554+0000 7f4f233dd700 1 --2- 172.21.15.40:0/4173799600 >> v2:172.21.15.43:3300/0 conn(0x7f4f1c1500c0 0x7f4f1c0c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.554+0000 7f4f233dd700 1 -- 172.21.15.40:0/4173799600 >> 172.21.15.40:0/4173799600 conn(0x7f4f1c0cabc0 msgr2=0x7f4f1c0cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:08.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.554+0000 7f4f233dd700 1 -- 172.21.15.40:0/4173799600 shutdown_connections 2024-09-13T17:51:08.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.554+0000 7f4f233dd700 1 -- 172.21.15.40:0/4173799600 wait complete. 2024-09-13T17:51:08.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.554+0000 7f4f233dd700 1 Processor -- start 2024-09-13T17:51:08.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.554+0000 7f4f233dd700 1 -- start start 2024-09-13T17:51:08.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f233dd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c070500 0x7f4f1c151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:08.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f233dd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f1c071b40 0x7f4f1c0c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:08.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f233dd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f1c1500c0 0x7f4f1c1553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:08.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f233dd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c0c1e00 con 0x7f4f1c1500c0 2024-09-13T17:51:08.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f233dd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c1540f0 con 0x7f4f1c070500 2024-09-13T17:51:08.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f233dd700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c153810 con 0x7f4f1c071b40 2024-09-13T17:51:08.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f22bdc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f1c1500c0 0x7f4f1c1553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:08.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f22bdc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f1c1500c0 0x7f4f1c1553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58224/0 (socket says 172.21.15.40:58224) 2024-09-13T17:51:08.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f22bdc700 1 -- 172.21.15.40:0/432548250 learned_addr learned my addr 172.21.15.40:0/432548250 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:08.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f223db700 1 --2- 172.21.15.40:0/432548250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c070500 0x7f4f1c151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:08.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f21bda700 1 --2- 172.21.15.40:0/432548250 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f1c071b40 0x7f4f1c0c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:08.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f22bdc700 1 -- 172.21.15.40:0/432548250 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f1c071b40 msgr2=0x7f4f1c0c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f22bdc700 1 --2- 172.21.15.40:0/432548250 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f1c071b40 0x7f4f1c0c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f22bdc700 1 -- 172.21.15.40:0/432548250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c070500 msgr2=0x7f4f1c151710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f22bdc700 1 --2- 172.21.15.40:0/432548250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c070500 0x7f4f1c151710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f22bdc700 1 -- 172.21.15.40:0/432548250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1000b810 con 0x7f4f1c1500c0 2024-09-13T17:51:08.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f22bdc700 1 --2- 172.21.15.40:0/432548250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f1c1500c0 0x7f4f1c1553e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4f1400caf0 tx=0x7f4f1400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:08.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f0effd700 1 -- 172.21.15.40:0/432548250 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f14003c20 con 0x7f4f1c1500c0 2024-09-13T17:51:08.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f0effd700 1 -- 172.21.15.40:0/432548250 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f14003d80 con 0x7f4f1c1500c0 2024-09-13T17:51:08.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f233dd700 1 -- 172.21.15.40:0/432548250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1c151c50 con 0x7f4f1c1500c0 2024-09-13T17:51:08.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f0effd700 1 -- 172.21.15.40:0/432548250 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1400ab10 con 0x7f4f1c1500c0 2024-09-13T17:51:08.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f233dd700 1 -- 172.21.15.40:0/432548250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1c150c80 con 0x7f4f1c1500c0 2024-09-13T17:51:08.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f233dd700 1 -- 172.21.15.40:0/432548250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1c154260 con 0x7f4f1c1500c0 2024-09-13T17:51:08.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.558+0000 7f4f0effd700 1 -- 172.21.15.40:0/432548250 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4f1400ac70 con 0x7f4f1c1500c0 2024-09-13T17:51:08.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.562+0000 7f4f0effd700 1 --2- 172.21.15.40:0/432548250 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4f00066730 0x7f4f00068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:08.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.562+0000 7f4f0effd700 1 -- 172.21.15.40:0/432548250 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4f140844d0 con 0x7f4f1c1500c0 2024-09-13T17:51:08.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.562+0000 7f4f223db700 1 --2- 172.21.15.40:0/432548250 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4f00066730 0x7f4f00068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:08.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.562+0000 7f4f223db700 1 --2- 172.21.15.40:0/432548250 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4f00066730 0x7f4f00068bf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4f18009ae0 tx=0x7f4f18008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:08.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.566+0000 7f4f0effd700 1 -- 172.21.15.40:0/432548250 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4f14051560 con 0x7f4f1c1500c0 2024-09-13T17:51:08.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.726+0000 7f4f233dd700 1 -- 172.21.15.40:0/432548250 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4f1c070bc0 con 0x7f4f00066730 2024-09-13T17:51:08.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.742+0000 7f4f0effd700 1 -- 172.21.15.40:0/432548250 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345217 (secure 0 0 0) 0x7f4f18011750 con 0x7f4f00066730 2024-09-13T17:51:08.753 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:08.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.750+0000 7f4f233dd700 1 -- 172.21.15.40:0/432548250 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4f00066730 msgr2=0x7f4f00068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.750+0000 7f4f233dd700 1 --2- 172.21.15.40:0/432548250 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4f00066730 0x7f4f00068bf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4f18009ae0 tx=0x7f4f18008040 comp rx=0 tx=0).stop 2024-09-13T17:51:08.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.750+0000 7f4f233dd700 1 -- 172.21.15.40:0/432548250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f1c1500c0 msgr2=0x7f4f1c1553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.750+0000 7f4f233dd700 1 --2- 172.21.15.40:0/432548250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f1c1500c0 0x7f4f1c1553e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4f1400caf0 tx=0x7f4f1400ceb0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.750+0000 7f4f233dd700 1 -- 172.21.15.40:0/432548250 shutdown_connections 2024-09-13T17:51:08.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.750+0000 7f4f233dd700 1 --2- 172.21.15.40:0/432548250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f1c1500c0 0x7f4f1c1553e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.750+0000 7f4f233dd700 1 --2- 172.21.15.40:0/432548250 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4f00066730 0x7f4f00068bf0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.750+0000 7f4f233dd700 1 --2- 172.21.15.40:0/432548250 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f1c071b40 0x7f4f1c0c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.750+0000 7f4f233dd700 1 --2- 172.21.15.40:0/432548250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1c070500 0x7f4f1c151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.750+0000 7f4f233dd700 1 -- 172.21.15.40:0/432548250 >> 172.21.15.40:0/432548250 conn(0x7f4f1c0cabc0 msgr2=0x7f4f1c0c52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:08.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.750+0000 7f4f233dd700 1 -- 172.21.15.40:0/432548250 shutdown_connections 2024-09-13T17:51:08.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.750+0000 7f4f233dd700 1 -- 172.21.15.40:0/432548250 wait complete. 2024-09-13T17:51:08.757 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-13T17:51:08.768 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-09-13T17:51:07.007430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594961,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235816,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738904,"statfs":{"total":773094113280,"available":772852637696,"internally_reserved":0,"allocated":3317760,"data_stored":2005562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004879"},"pg_stats":[{"pgid":"2.1f","version":"25'4","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.584829+0000","last_change":"2024-09-13T17:50:34.212158+0000","last_active":"2024-09-13T17:50:39.584829+0000","last_peered":"2024-09-13T17:50:39.584829+0000","last_clean":"2024-09-13T17:50:39.584829+0000","last_became_active":"2024-09-13T17:50:34.211867+0000","last_became_peered":"2024-09-13T17:50:34.211867+0000","last_unstale":"2024-09-13T17:50:39.584829+0000","last_undegraded":"2024-09-13T17:50:39.584829+0000","last_fullsized":"2024-09-13T17:50:39.584829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:14.671095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589172+0000","last_change":"2024-09-13T17:50:34.587396+0000","last_active":"2024-09-13T17:50:38.589172+0000","last_peered":"2024-09-13T17:50:38.589172+0000","last_clean":"2024-09-13T17:50:38.589172+0000","last_became_active":"2024-09-13T17:50:34.587108+0000","last_became_peered":"2024-09-13T17:50:34.587108+0000","last_unstale":"2024-09-13T17:50:38.589172+0000","last_undegraded":"2024-09-13T17:50:38.589172+0000","last_fullsized":"2024-09-13T17:50:38.589172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:39:16.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.995312+0000","last_change":"2024-09-13T17:50:38.615059+0000","last_active":"2024-09-13T17:50:38.995312+0000","last_peered":"2024-09-13T17:50:38.995312+0000","last_clean":"2024-09-13T17:50:38.995312+0000","last_became_active":"2024-09-13T17:50:38.614723+0000","last_became_peered":"2024-09-13T17:50:38.614723+0000","last_unstale":"2024-09-13T17:50:38.995312+0000","last_undegraded":"2024-09-13T17:50:38.995312+0000","last_fullsized":"2024-09-13T17:50:38.995312+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:23:01.902181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"25'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.996807+0000","last_change":"2024-09-13T17:50:34.204760+0000","last_active":"2024-09-13T17:50:38.996807+0000","last_peered":"2024-09-13T17:50:38.996807+0000","last_clean":"2024-09-13T17:50:38.996807+0000","last_became_active":"2024-09-13T17:50:34.202930+0000","last_became_peered":"2024-09-13T17:50:34.202930+0000","last_unstale":"2024-09-13T17:50:38.996807+0000","last_undegraded":"2024-09-13T17:50:38.996807+0000","last_fullsized":"2024-09-13T17:50:38.996807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:59.784701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.598529+0000","last_change":"2024-09-13T17:50:34.583174+0000","last_active":"2024-09-13T17:50:38.598529+0000","last_peered":"2024-09-13T17:50:38.598529+0000","last_clean":"2024-09-13T17:50:38.598529+0000","last_became_active":"2024-09-13T17:50:34.582884+0000","last_became_peered":"2024-09-13T17:50:34.582884+0000","last_unstale":"2024-09-13T17:50:38.598529+0000","last_undegraded":"2024-09-13T17:50:38.598529+0000","last_fullsized":"2024-09-13T17:50:38.598529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:22:27.539574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501034+0000","last_change":"2024-09-13T17:50:38.618751+0000","last_active":"2024-09-13T17:50:39.501034+0000","last_peered":"2024-09-13T17:50:39.501034+0000","last_clean":"2024-09-13T17:50:39.501034+0000","last_became_active":"2024-09-13T17:50:38.618423+0000","last_became_peered":"2024-09-13T17:50:38.618423+0000","last_unstale":"2024-09-13T17:50:39.501034+0000","last_undegraded":"2024-09-13T17:50:39.501034+0000","last_fullsized":"2024-09-13T17:50:39.501034+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:50:02.418785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.582559+0000","last_change":"2024-09-13T17:50:34.214972+0000","last_active":"2024-09-13T17:50:39.582559+0000","last_peered":"2024-09-13T17:50:39.582559+0000","last_clean":"2024-09-13T17:50:39.582559+0000","last_became_active":"2024-09-13T17:50:34.214652+0000","last_became_peered":"2024-09-13T17:50:34.214652+0000","last_unstale":"2024-09-13T17:50:39.582559+0000","last_undegraded":"2024-09-13T17:50:39.582559+0000","last_fullsized":"2024-09-13T17:50:39.582559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:57.176854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590539+0000","last_change":"2024-09-13T17:50:34.594493+0000","last_active":"2024-09-13T17:50:38.590539+0000","last_peered":"2024-09-13T17:50:38.590539+0000","last_clean":"2024-09-13T17:50:38.590539+0000","last_became_active":"2024-09-13T17:50:34.594182+0000","last_became_peered":"2024-09-13T17:50:34.594182+0000","last_unstale":"2024-09-13T17:50:38.590539+0000","last_undegraded":"2024-09-13T17:50:38.590539+0000","last_fullsized":"2024-09-13T17:50:38.590539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:18.059925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501399+0000","last_change":"2024-09-13T17:50:38.616219+0000","last_active":"2024-09-13T17:50:39.501399+0000","last_peered":"2024-09-13T17:50:39.501399+0000","last_clean":"2024-09-13T17:50:39.501399+0000","last_became_active":"2024-09-13T17:50:38.614329+0000","last_became_peered":"2024-09-13T17:50:38.614329+0000","last_unstale":"2024-09-13T17:50:39.501399+0000","last_undegraded":"2024-09-13T17:50:39.501399+0000","last_fullsized":"2024-09-13T17:50:39.501399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:25.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583345+0000","last_change":"2024-09-13T17:50:34.216123+0000","last_active":"2024-09-13T17:50:39.583345+0000","last_peered":"2024-09-13T17:50:39.583345+0000","last_clean":"2024-09-13T17:50:39.583345+0000","last_became_active":"2024-09-13T17:50:34.215661+0000","last_became_peered":"2024-09-13T17:50:34.215661+0000","last_unstale":"2024-09-13T17:50:39.583345+0000","last_undegraded":"2024-09-13T17:50:39.583345+0000","last_fullsized":"2024-09-13T17:50:39.583345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:20.895037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591506+0000","last_change":"2024-09-13T17:50:34.599673+0000","last_active":"2024-09-13T17:50:38.591506+0000","last_peered":"2024-09-13T17:50:38.591506+0000","last_clean":"2024-09-13T17:50:38.591506+0000","last_became_active":"2024-09-13T17:50:34.599409+0000","last_became_peered":"2024-09-13T17:50:34.599409+0000","last_unstale":"2024-09-13T17:50:38.591506+0000","last_undegraded":"2024-09-13T17:50:38.591506+0000","last_fullsized":"2024-09-13T17:50:38.591506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:03:31.394385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501275+0000","last_change":"2024-09-13T17:50:38.616317+0000","last_active":"2024-09-13T17:50:39.501275+0000","last_peered":"2024-09-13T17:50:39.501275+0000","last_clean":"2024-09-13T17:50:39.501275+0000","last_became_active":"2024-09-13T17:50:38.614311+0000","last_became_peered":"2024-09-13T17:50:38.614311+0000","last_unstale":"2024-09-13T17:50:39.501275+0000","last_undegraded":"2024-09-13T17:50:39.501275+0000","last_fullsized":"2024-09-13T17:50:39.501275+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:29.221522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.604632+0000","last_change":"2024-09-13T17:50:34.599490+0000","last_active":"2024-09-13T17:50:38.604632+0000","last_peered":"2024-09-13T17:50:38.604632+0000","last_clean":"2024-09-13T17:50:38.604632+0000","last_became_active":"2024-09-13T17:50:34.599207+0000","last_became_peered":"2024-09-13T17:50:34.599207+0000","last_unstale":"2024-09-13T17:50:38.604632+0000","last_undegraded":"2024-09-13T17:50:38.604632+0000","last_fullsized":"2024-09-13T17:50:38.604632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:11.178161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583429+0000","last_change":"2024-09-13T17:50:34.215697+0000","last_active":"2024-09-13T17:50:39.583429+0000","last_peered":"2024-09-13T17:50:39.583429+0000","last_clean":"2024-09-13T17:50:39.583429+0000","last_became_active":"2024-09-13T17:50:34.215409+0000","last_became_peered":"2024-09-13T17:50:34.215409+0000","last_unstale":"2024-09-13T17:50:39.583429+0000","last_undegraded":"2024-09-13T17:50:39.583429+0000","last_fullsized":"2024-09-13T17:50:39.583429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:21.609567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.502325+0000","last_change":"2024-09-13T17:50:38.611817+0000","last_active":"2024-09-13T17:50:39.502325+0000","last_peered":"2024-09-13T17:50:39.502325+0000","last_clean":"2024-09-13T17:50:39.502325+0000","last_became_active":"2024-09-13T17:50:38.611263+0000","last_became_peered":"2024-09-13T17:50:38.611263+0000","last_unstale":"2024-09-13T17:50:39.502325+0000","last_undegraded":"2024-09-13T17:50:39.502325+0000","last_fullsized":"2024-09-13T17:50:39.502325+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:56:47.466672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.588598+0000","last_change":"2024-09-13T17:50:34.586307+0000","last_active":"2024-09-13T17:50:38.588598+0000","last_peered":"2024-09-13T17:50:38.588598+0000","last_clean":"2024-09-13T17:50:38.588598+0000","last_became_active":"2024-09-13T17:50:34.585831+0000","last_became_peered":"2024-09-13T17:50:34.585831+0000","last_unstale":"2024-09-13T17:50:38.588598+0000","last_undegraded":"2024-09-13T17:50:38.588598+0000","last_fullsized":"2024-09-13T17:50:38.588598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:40.912352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599468+0000","last_change":"2024-09-13T17:50:34.211850+0000","last_active":"2024-09-13T17:50:38.599468+0000","last_peered":"2024-09-13T17:50:38.599468+0000","last_clean":"2024-09-13T17:50:38.599468+0000","last_became_active":"2024-09-13T17:50:34.211594+0000","last_became_peered":"2024-09-13T17:50:34.211594+0000","last_unstale":"2024-09-13T17:50:38.599468+0000","last_undegraded":"2024-09-13T17:50:38.599468+0000","last_fullsized":"2024-09-13T17:50:38.599468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:35:02.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.502004+0000","last_change":"2024-09-13T17:50:38.613576+0000","last_active":"2024-09-13T17:50:39.502004+0000","last_peered":"2024-09-13T17:50:39.502004+0000","last_clean":"2024-09-13T17:50:39.502004+0000","last_became_active":"2024-09-13T17:50:38.612577+0000","last_became_peered":"2024-09-13T17:50:38.612577+0000","last_unstale":"2024-09-13T17:50:39.502004+0000","last_undegraded":"2024-09-13T17:50:39.502004+0000","last_fullsized":"2024-09-13T17:50:39.502004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:25.551060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590056+0000","last_change":"2024-09-13T17:50:34.587273+0000","last_active":"2024-09-13T17:50:38.590056+0000","last_peered":"2024-09-13T17:50:38.590056+0000","last_clean":"2024-09-13T17:50:38.590056+0000","last_became_active":"2024-09-13T17:50:34.587027+0000","last_became_peered":"2024-09-13T17:50:34.587027+0000","last_unstale":"2024-09-13T17:50:38.590056+0000","last_undegraded":"2024-09-13T17:50:38.590056+0000","last_fullsized":"2024-09-13T17:50:38.590056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:58.826737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599621+0000","last_change":"2024-09-13T17:50:34.211845+0000","last_active":"2024-09-13T17:50:38.599621+0000","last_peered":"2024-09-13T17:50:38.599621+0000","last_clean":"2024-09-13T17:50:38.599621+0000","last_became_active":"2024-09-13T17:50:34.211568+0000","last_became_peered":"2024-09-13T17:50:34.211568+0000","last_unstale":"2024-09-13T17:50:38.599621+0000","last_undegraded":"2024-09-13T17:50:38.599621+0000","last_fullsized":"2024-09-13T17:50:38.599621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:56.348984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000608+0000","last_change":"2024-09-13T17:50:38.610292+0000","last_active":"2024-09-13T17:50:39.000608+0000","last_peered":"2024-09-13T17:50:39.000608+0000","last_clean":"2024-09-13T17:50:39.000608+0000","last_became_active":"2024-09-13T17:50:38.609832+0000","last_became_peered":"2024-09-13T17:50:38.609832+0000","last_unstale":"2024-09-13T17:50:39.000608+0000","last_undegraded":"2024-09-13T17:50:39.000608+0000","last_fullsized":"2024-09-13T17:50:39.000608+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:13.783068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.603557+0000","last_change":"2024-09-13T17:50:34.593566+0000","last_active":"2024-09-13T17:50:38.603557+0000","last_peered":"2024-09-13T17:50:38.603557+0000","last_clean":"2024-09-13T17:50:38.603557+0000","last_became_active":"2024-09-13T17:50:34.588794+0000","last_became_peered":"2024-09-13T17:50:34.588794+0000","last_unstale":"2024-09-13T17:50:38.603557+0000","last_undegraded":"2024-09-13T17:50:38.603557+0000","last_fullsized":"2024-09-13T17:50:38.603557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:13.192453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589828+0000","last_change":"2024-09-13T17:50:34.214638+0000","last_active":"2024-09-13T17:50:38.589828+0000","last_peered":"2024-09-13T17:50:38.589828+0000","last_clean":"2024-09-13T17:50:38.589828+0000","last_became_active":"2024-09-13T17:50:34.214292+0000","last_became_peered":"2024-09-13T17:50:34.214292+0000","last_unstale":"2024-09-13T17:50:38.589828+0000","last_undegraded":"2024-09-13T17:50:38.589828+0000","last_fullsized":"2024-09-13T17:50:38.589828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:57.377104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501996+0000","last_change":"2024-09-13T17:50:38.613368+0000","last_active":"2024-09-13T17:50:39.501996+0000","last_peered":"2024-09-13T17:50:39.501996+0000","last_clean":"2024-09-13T17:50:39.501996+0000","last_became_active":"2024-09-13T17:50:38.613010+0000","last_became_peered":"2024-09-13T17:50:38.613010+0000","last_unstale":"2024-09-13T17:50:39.501996+0000","last_undegraded":"2024-09-13T17:50:39.501996+0000","last_fullsized":"2024-09-13T17:50:39.501996+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:42.706625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.604457+0000","last_change":"2024-09-13T17:50:34.582327+0000","last_active":"2024-09-13T17:50:38.604457+0000","last_peered":"2024-09-13T17:50:38.604457+0000","last_clean":"2024-09-13T17:50:38.604457+0000","last_became_active":"2024-09-13T17:50:34.582050+0000","last_became_peered":"2024-09-13T17:50:34.582050+0000","last_unstale":"2024-09-13T17:50:38.604457+0000","last_undegraded":"2024-09-13T17:50:38.604457+0000","last_fullsized":"2024-09-13T17:50:38.604457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:52.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583664+0000","last_change":"2024-09-13T17:50:34.211512+0000","last_active":"2024-09-13T17:50:39.583664+0000","last_peered":"2024-09-13T17:50:39.583664+0000","last_clean":"2024-09-13T17:50:39.583664+0000","last_became_active":"2024-09-13T17:50:34.211230+0000","last_became_peered":"2024-09-13T17:50:34.211230+0000","last_unstale":"2024-09-13T17:50:39.583664+0000","last_undegraded":"2024-09-13T17:50:39.583664+0000","last_fullsized":"2024-09-13T17:50:39.583664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:40.707777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501244+0000","last_change":"2024-09-13T17:50:38.608255+0000","last_active":"2024-09-13T17:50:39.501244+0000","last_peered":"2024-09-13T17:50:39.501244+0000","last_clean":"2024-09-13T17:50:39.501244+0000","last_became_active":"2024-09-13T17:50:38.606697+0000","last_became_peered":"2024-09-13T17:50:38.606697+0000","last_unstale":"2024-09-13T17:50:39.501244+0000","last_undegraded":"2024-09-13T17:50:39.501244+0000","last_fullsized":"2024-09-13T17:50:39.501244+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:47.265007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591328+0000","last_change":"2024-09-13T17:50:34.594476+0000","last_active":"2024-09-13T17:50:38.591328+0000","last_peered":"2024-09-13T17:50:38.591328+0000","last_clean":"2024-09-13T17:50:38.591328+0000","last_became_active":"2024-09-13T17:50:34.594215+0000","last_became_peered":"2024-09-13T17:50:34.594215+0000","last_unstale":"2024-09-13T17:50:38.591328+0000","last_undegraded":"2024-09-13T17:50:38.591328+0000","last_fullsized":"2024-09-13T17:50:38.591328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:02:27.528019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000656+0000","last_change":"2024-09-13T17:50:34.204018+0000","last_active":"2024-09-13T17:50:39.000656+0000","last_peered":"2024-09-13T17:50:39.000656+0000","last_clean":"2024-09-13T17:50:39.000656+0000","last_became_active":"2024-09-13T17:50:34.203702+0000","last_became_peered":"2024-09-13T17:50:34.203702+0000","last_unstale":"2024-09-13T17:50:39.000656+0000","last_undegraded":"2024-09-13T17:50:39.000656+0000","last_fullsized":"2024-09-13T17:50:39.000656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:44.431226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.995065+0000","last_change":"2024-09-13T17:50:38.615677+0000","last_active":"2024-09-13T17:50:38.995065+0000","last_peered":"2024-09-13T17:50:38.995065+0000","last_clean":"2024-09-13T17:50:38.995065+0000","last_became_active":"2024-09-13T17:50:38.615270+0000","last_became_peered":"2024-09-13T17:50:38.615270+0000","last_unstale":"2024-09-13T17:50:38.995065+0000","last_undegraded":"2024-09-13T17:50:38.995065+0000","last_fullsized":"2024-09-13T17:50:38.995065+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:55.768090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599593+0000","last_change":"2024-09-13T17:50:34.586802+0000","last_active":"2024-09-13T17:50:38.599593+0000","last_peered":"2024-09-13T17:50:38.599593+0000","last_clean":"2024-09-13T17:50:38.599593+0000","last_became_active":"2024-09-13T17:50:34.586552+0000","last_became_peered":"2024-09-13T17:50:34.586552+0000","last_unstale":"2024-09-13T17:50:38.599593+0000","last_undegraded":"2024-09-13T17:50:38.599593+0000","last_fullsized":"2024-09-13T17:50:38.599593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:43.383975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.592964+0000","last_change":"2024-09-13T17:50:34.217083+0000","last_active":"2024-09-13T17:50:38.592964+0000","last_peered":"2024-09-13T17:50:38.592964+0000","last_clean":"2024-09-13T17:50:38.592964+0000","last_became_active":"2024-09-13T17:50:34.216758+0000","last_became_peered":"2024-09-13T17:50:34.216758+0000","last_unstale":"2024-09-13T17:50:38.592964+0000","last_undegraded":"2024-09-13T17:50:38.592964+0000","last_fullsized":"2024-09-13T17:50:38.592964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:36.473942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.500902+0000","last_change":"2024-09-13T17:50:38.614667+0000","last_active":"2024-09-13T17:50:39.500902+0000","last_peered":"2024-09-13T17:50:39.500902+0000","last_clean":"2024-09-13T17:50:39.500902+0000","last_became_active":"2024-09-13T17:50:38.614284+0000","last_became_peered":"2024-09-13T17:50:38.614284+0000","last_unstale":"2024-09-13T17:50:39.500902+0000","last_undegraded":"2024-09-13T17:50:39.500902+0000","last_fullsized":"2024-09-13T17:50:39.500902+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:29.616794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591705+0000","last_change":"2024-09-13T17:50:34.588603+0000","last_active":"2024-09-13T17:50:38.591705+0000","last_peered":"2024-09-13T17:50:38.591705+0000","last_clean":"2024-09-13T17:50:38.591705+0000","last_became_active":"2024-09-13T17:50:34.588317+0000","last_became_peered":"2024-09-13T17:50:34.588317+0000","last_unstale":"2024-09-13T17:50:38.591705+0000","last_undegraded":"2024-09-13T17:50:38.591705+0000","last_fullsized":"2024-09-13T17:50:38.591705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:03.743130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599679+0000","last_change":"2024-09-13T17:50:34.204335+0000","last_active":"2024-09-13T17:50:38.599679+0000","last_peered":"2024-09-13T17:50:38.599679+0000","last_clean":"2024-09-13T17:50:38.599679+0000","last_became_active":"2024-09-13T17:50:34.204060+0000","last_became_peered":"2024-09-13T17:50:34.204060+0000","last_unstale":"2024-09-13T17:50:38.599679+0000","last_undegraded":"2024-09-13T17:50:38.599679+0000","last_fullsized":"2024-09-13T17:50:38.599679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:54:41.831344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.999822+0000","last_change":"2024-09-13T17:50:38.619590+0000","last_active":"2024-09-13T17:50:38.999822+0000","last_peered":"2024-09-13T17:50:38.999822+0000","last_clean":"2024-09-13T17:50:38.999822+0000","last_became_active":"2024-09-13T17:50:38.617257+0000","last_became_peered":"2024-09-13T17:50:38.617257+0000","last_unstale":"2024-09-13T17:50:38.999822+0000","last_undegraded":"2024-09-13T17:50:38.999822+0000","last_fullsized":"2024-09-13T17:50:38.999822+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:34.870291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.602091+0000","last_change":"2024-09-13T17:50:34.588466+0000","last_active":"2024-09-13T17:50:38.602091+0000","last_peered":"2024-09-13T17:50:38.602091+0000","last_clean":"2024-09-13T17:50:38.602091+0000","last_became_active":"2024-09-13T17:50:34.588063+0000","last_became_peered":"2024-09-13T17:50:34.588063+0000","last_unstale":"2024-09-13T17:50:38.602091+0000","last_undegraded":"2024-09-13T17:50:38.602091+0000","last_fullsized":"2024-09-13T17:50:38.602091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:43:00.669203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589993+0000","last_change":"2024-09-13T17:50:34.209239+0000","last_active":"2024-09-13T17:50:38.589993+0000","last_peered":"2024-09-13T17:50:38.589993+0000","last_clean":"2024-09-13T17:50:38.589993+0000","last_became_active":"2024-09-13T17:50:34.208196+0000","last_became_peered":"2024-09-13T17:50:34.208196+0000","last_unstale":"2024-09-13T17:50:38.589993+0000","last_undegraded":"2024-09-13T17:50:38.589993+0000","last_fullsized":"2024-09-13T17:50:38.589993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:39.253917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000394+0000","last_change":"2024-09-13T17:50:38.611021+0000","last_active":"2024-09-13T17:50:39.000394+0000","last_peered":"2024-09-13T17:50:39.000394+0000","last_clean":"2024-09-13T17:50:39.000394+0000","last_became_active":"2024-09-13T17:50:38.609794+0000","last_became_peered":"2024-09-13T17:50:38.609794+0000","last_unstale":"2024-09-13T17:50:39.000394+0000","last_undegraded":"2024-09-13T17:50:39.000394+0000","last_fullsized":"2024-09-13T17:50:39.000394+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:59.980236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.600038+0000","last_change":"2024-09-13T17:50:34.584290+0000","last_active":"2024-09-13T17:50:38.600038+0000","last_peered":"2024-09-13T17:50:38.600038+0000","last_clean":"2024-09-13T17:50:38.600038+0000","last_became_active":"2024-09-13T17:50:34.584023+0000","last_became_peered":"2024-09-13T17:50:34.584023+0000","last_unstale":"2024-09-13T17:50:38.600038+0000","last_undegraded":"2024-09-13T17:50:38.600038+0000","last_fullsized":"2024-09-13T17:50:38.600038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:11.304986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.597167+0000","last_change":"2024-09-13T17:50:34.198932+0000","last_active":"2024-09-13T17:50:38.597167+0000","last_peered":"2024-09-13T17:50:38.597167+0000","last_clean":"2024-09-13T17:50:38.597167+0000","last_became_active":"2024-09-13T17:50:34.198603+0000","last_became_peered":"2024-09-13T17:50:34.198603+0000","last_unstale":"2024-09-13T17:50:38.597167+0000","last_undegraded":"2024-09-13T17:50:38.597167+0000","last_fullsized":"2024-09-13T17:50:38.597167+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:00.917126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000707+0000","last_change":"2024-09-13T17:50:38.611326+0000","last_active":"2024-09-13T17:50:39.000707+0000","last_peered":"2024-09-13T17:50:39.000707+0000","last_clean":"2024-09-13T17:50:39.000707+0000","last_became_active":"2024-09-13T17:50:38.610900+0000","last_became_peered":"2024-09-13T17:50:38.610900+0000","last_unstale":"2024-09-13T17:50:39.000707+0000","last_undegraded":"2024-09-13T17:50:39.000707+0000","last_fullsized":"2024-09-13T17:50:39.000707+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:17:09.994736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.605384+0000","last_change":"2024-09-13T17:50:34.594482+0000","last_active":"2024-09-13T17:50:38.605384+0000","last_peered":"2024-09-13T17:50:38.605384+0000","last_clean":"2024-09-13T17:50:38.605384+0000","last_became_active":"2024-09-13T17:50:34.594201+0000","last_became_peered":"2024-09-13T17:50:34.594201+0000","last_unstale":"2024-09-13T17:50:38.605384+0000","last_undegraded":"2024-09-13T17:50:38.605384+0000","last_fullsized":"2024-09-13T17:50:38.605384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:06.697507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590063+0000","last_change":"2024-09-13T17:50:34.218376+0000","last_active":"2024-09-13T17:50:38.590063+0000","last_peered":"2024-09-13T17:50:38.590063+0000","last_clean":"2024-09-13T17:50:38.590063+0000","last_became_active":"2024-09-13T17:50:34.218069+0000","last_became_peered":"2024-09-13T17:50:34.218069+0000","last_unstale":"2024-09-13T17:50:38.590063+0000","last_undegraded":"2024-09-13T17:50:38.590063+0000","last_fullsized":"2024-09-13T17:50:38.590063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:53:54.288969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501171+0000","last_change":"2024-09-13T17:50:38.613206+0000","last_active":"2024-09-13T17:50:39.501171+0000","last_peered":"2024-09-13T17:50:39.501171+0000","last_clean":"2024-09-13T17:50:39.501171+0000","last_became_active":"2024-09-13T17:50:38.612856+0000","last_became_peered":"2024-09-13T17:50:38.612856+0000","last_unstale":"2024-09-13T17:50:39.501171+0000","last_undegraded":"2024-09-13T17:50:39.501171+0000","last_fullsized":"2024-09-13T17:50:39.501171+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:31:20.190996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.588765+0000","last_change":"2024-09-13T17:50:34.582931+0000","last_active":"2024-09-13T17:50:38.588765+0000","last_peered":"2024-09-13T17:50:38.588765+0000","last_clean":"2024-09-13T17:50:38.588765+0000","last_became_active":"2024-09-13T17:50:34.582446+0000","last_became_peered":"2024-09-13T17:50:34.582446+0000","last_unstale":"2024-09-13T17:50:38.588765+0000","last_undegraded":"2024-09-13T17:50:38.588765+0000","last_fullsized":"2024-09-13T17:50:38.588765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:30.554785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.588978+0000","last_change":"2024-09-13T17:50:34.201237+0000","last_active":"2024-09-13T17:50:38.588978+0000","last_peered":"2024-09-13T17:50:38.588978+0000","last_clean":"2024-09-13T17:50:38.588978+0000","last_became_active":"2024-09-13T17:50:34.200873+0000","last_became_peered":"2024-09-13T17:50:34.200873+0000","last_unstale":"2024-09-13T17:50:38.588978+0000","last_undegraded":"2024-09-13T17:50:38.588978+0000","last_fullsized":"2024-09-13T17:50:38.588978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:02.711342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590078+0000","last_change":"2024-09-13T17:50:22.095367+0000","last_active":"2024-09-13T17:50:38.590078+0000","last_peered":"2024-09-13T17:50:38.590078+0000","last_clean":"2024-09-13T17:50:38.590078+0000","last_became_active":"2024-09-13T17:50:22.093734+0000","last_became_peered":"2024-09-13T17:50:22.093734+0000","last_unstale":"2024-09-13T17:50:38.590078+0000","last_undegraded":"2024-09-13T17:50:38.590078+0000","last_fullsized":"2024-09-13T17:50:38.590078+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_clean_scrub_stamp":"2024-09-13T17:50:21.057812+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:23.531613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.982380+0000","last_change":"2024-09-13T17:50:38.618880+0000","last_active":"2024-09-13T17:50:38.982380+0000","last_peered":"2024-09-13T17:50:38.982380+0000","last_clean":"2024-09-13T17:50:38.982380+0000","last_became_active":"2024-09-13T17:50:38.618488+0000","last_became_peered":"2024-09-13T17:50:38.618488+0000","last_unstale":"2024-09-13T17:50:38.982380+0000","last_undegraded":"2024-09-13T17:50:38.982380+0000","last_fullsized":"2024-09-13T17:50:38.982380+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:03.431876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.592679+0000","last_change":"2024-09-13T17:50:34.585881+0000","last_active":"2024-09-13T17:50:38.592679+0000","last_peered":"2024-09-13T17:50:38.592679+0000","last_clean":"2024-09-13T17:50:38.592679+0000","last_became_active":"2024-09-13T17:50:34.585166+0000","last_became_peered":"2024-09-13T17:50:34.585166+0000","last_unstale":"2024-09-13T17:50:38.592679+0000","last_undegraded":"2024-09-13T17:50:38.592679+0000","last_fullsized":"2024-09-13T17:50:38.592679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:06.654658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.602557+0000","last_change":"2024-09-13T17:50:34.210934+0000","last_active":"2024-09-13T17:50:38.602557+0000","last_peered":"2024-09-13T17:50:38.602557+0000","last_clean":"2024-09-13T17:50:38.602557+0000","last_became_active":"2024-09-13T17:50:34.210589+0000","last_became_peered":"2024-09-13T17:50:34.210589+0000","last_unstale":"2024-09-13T17:50:38.602557+0000","last_undegraded":"2024-09-13T17:50:38.602557+0000","last_fullsized":"2024-09-13T17:50:38.602557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:26.213295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.982569+0000","last_change":"2024-09-13T17:50:38.615811+0000","last_active":"2024-09-13T17:50:38.982569+0000","last_peered":"2024-09-13T17:50:38.982569+0000","last_clean":"2024-09-13T17:50:38.982569+0000","last_became_active":"2024-09-13T17:50:38.615358+0000","last_became_peered":"2024-09-13T17:50:38.615358+0000","last_unstale":"2024-09-13T17:50:38.982569+0000","last_undegraded":"2024-09-13T17:50:38.982569+0000","last_fullsized":"2024-09-13T17:50:38.982569+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:05.667393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590751+0000","last_change":"2024-09-13T17:50:34.590169+0000","last_active":"2024-09-13T17:50:38.590751+0000","last_peered":"2024-09-13T17:50:38.590751+0000","last_clean":"2024-09-13T17:50:38.590751+0000","last_became_active":"2024-09-13T17:50:34.589434+0000","last_became_peered":"2024-09-13T17:50:34.589434+0000","last_unstale":"2024-09-13T17:50:38.590751+0000","last_undegraded":"2024-09-13T17:50:38.590751+0000","last_fullsized":"2024-09-13T17:50:38.590751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:12.545412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000847+0000","last_change":"2024-09-13T17:50:34.204269+0000","last_active":"2024-09-13T17:50:39.000847+0000","last_peered":"2024-09-13T17:50:39.000847+0000","last_clean":"2024-09-13T17:50:39.000847+0000","last_became_active":"2024-09-13T17:50:34.203874+0000","last_became_peered":"2024-09-13T17:50:34.203874+0000","last_unstale":"2024-09-13T17:50:39.000847+0000","last_undegraded":"2024-09-13T17:50:39.000847+0000","last_fullsized":"2024-09-13T17:50:39.000847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:10.410267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501092+0000","last_change":"2024-09-13T17:50:38.615989+0000","last_active":"2024-09-13T17:50:39.501092+0000","last_peered":"2024-09-13T17:50:39.501092+0000","last_clean":"2024-09-13T17:50:39.501092+0000","last_became_active":"2024-09-13T17:50:38.615530+0000","last_became_peered":"2024-09-13T17:50:38.615530+0000","last_unstale":"2024-09-13T17:50:39.501092+0000","last_undegraded":"2024-09-13T17:50:39.501092+0000","last_fullsized":"2024-09-13T17:50:39.501092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:41.064092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.593217+0000","last_change":"2024-09-13T17:50:34.595120+0000","last_active":"2024-09-13T17:50:38.593217+0000","last_peered":"2024-09-13T17:50:38.593217+0000","last_clean":"2024-09-13T17:50:38.593217+0000","last_became_active":"2024-09-13T17:50:34.594842+0000","last_became_peered":"2024-09-13T17:50:34.594842+0000","last_unstale":"2024-09-13T17:50:38.593217+0000","last_undegraded":"2024-09-13T17:50:38.593217+0000","last_fullsized":"2024-09-13T17:50:38.593217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:15:38.982524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.587423+0000","last_change":"2024-09-13T17:50:34.199957+0000","last_active":"2024-09-13T17:50:39.587423+0000","last_peered":"2024-09-13T17:50:39.587423+0000","last_clean":"2024-09-13T17:50:39.587423+0000","last_became_active":"2024-09-13T17:50:34.199650+0000","last_became_peered":"2024-09-13T17:50:34.199650+0000","last_unstale":"2024-09-13T17:50:39.587423+0000","last_undegraded":"2024-09-13T17:50:39.587423+0000","last_fullsized":"2024-09-13T17:50:39.587423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:26.929549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.193989+0000","last_change":"2024-09-13T17:50:38.609731+0000","last_active":"2024-09-13T17:50:39.193989+0000","last_peered":"2024-09-13T17:50:39.193989+0000","last_clean":"2024-09-13T17:50:39.193989+0000","last_became_active":"2024-09-13T17:50:38.609354+0000","last_became_peered":"2024-09-13T17:50:38.609354+0000","last_unstale":"2024-09-13T17:50:39.193989+0000","last_undegraded":"2024-09-13T17:50:39.193989+0000","last_fullsized":"2024-09-13T17:50:39.193989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:14.142369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.593339+0000","last_change":"2024-09-13T17:50:34.595146+0000","last_active":"2024-09-13T17:50:38.593339+0000","last_peered":"2024-09-13T17:50:38.593339+0000","last_clean":"2024-09-13T17:50:38.593339+0000","last_became_active":"2024-09-13T17:50:34.594876+0000","last_became_peered":"2024-09-13T17:50:34.594876+0000","last_unstale":"2024-09-13T17:50:38.593339+0000","last_undegraded":"2024-09-13T17:50:38.593339+0000","last_fullsized":"2024-09-13T17:50:38.593339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:26:10.638438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.502430+0000","last_change":"2024-09-13T17:50:34.200640+0000","last_active":"2024-09-13T17:50:39.502430+0000","last_peered":"2024-09-13T17:50:39.502430+0000","last_clean":"2024-09-13T17:50:39.502430+0000","last_became_active":"2024-09-13T17:50:34.200197+0000","last_became_peered":"2024-09-13T17:50:34.200197+0000","last_unstale":"2024-09-13T17:50:39.502430+0000","last_undegraded":"2024-09-13T17:50:39.502430+0000","last_fullsized":"2024-09-13T17:50:39.502430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:37:53.039720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.500911+0000","last_change":"2024-09-13T17:50:38.619880+0000","last_active":"2024-09-13T17:50:39.500911+0000","last_peered":"2024-09-13T17:50:39.500911+0000","last_clean":"2024-09-13T17:50:39.500911+0000","last_became_active":"2024-09-13T17:50:38.618491+0000","last_became_peered":"2024-09-13T17:50:38.618491+0000","last_unstale":"2024-09-13T17:50:39.500911+0000","last_undegraded":"2024-09-13T17:50:39.500911+0000","last_fullsized":"2024-09-13T17:50:39.500911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:10:14.136054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590688+0000","last_change":"2024-09-13T17:50:34.585741+0000","last_active":"2024-09-13T17:50:38.590688+0000","last_peered":"2024-09-13T17:50:38.590688+0000","last_clean":"2024-09-13T17:50:38.590688+0000","last_became_active":"2024-09-13T17:50:34.585464+0000","last_became_peered":"2024-09-13T17:50:34.585464+0000","last_unstale":"2024-09-13T17:50:38.590688+0000","last_undegraded":"2024-09-13T17:50:38.590688+0000","last_fullsized":"2024-09-13T17:50:38.590688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:59:22.031459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"25'8","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.585286+0000","last_change":"2024-09-13T17:50:34.201272+0000","last_active":"2024-09-13T17:50:39.585286+0000","last_peered":"2024-09-13T17:50:39.585286+0000","last_clean":"2024-09-13T17:50:39.585286+0000","last_became_active":"2024-09-13T17:50:34.200936+0000","last_became_peered":"2024-09-13T17:50:34.200936+0000","last_unstale":"2024-09-13T17:50:39.585286+0000","last_undegraded":"2024-09-13T17:50:39.585286+0000","last_fullsized":"2024-09-13T17:50:39.585286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:52.094879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.500719+0000","last_change":"2024-09-13T17:50:38.615229+0000","last_active":"2024-09-13T17:50:39.500719+0000","last_peered":"2024-09-13T17:50:39.500719+0000","last_clean":"2024-09-13T17:50:39.500719+0000","last_became_active":"2024-09-13T17:50:38.614739+0000","last_became_peered":"2024-09-13T17:50:38.614739+0000","last_unstale":"2024-09-13T17:50:39.500719+0000","last_undegraded":"2024-09-13T17:50:39.500719+0000","last_fullsized":"2024-09-13T17:50:39.500719+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:18.619529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.604486+0000","last_change":"2024-09-13T17:50:34.588848+0000","last_active":"2024-09-13T17:50:38.604486+0000","last_peered":"2024-09-13T17:50:38.604486+0000","last_clean":"2024-09-13T17:50:38.604486+0000","last_became_active":"2024-09-13T17:50:34.588565+0000","last_became_peered":"2024-09-13T17:50:34.588565+0000","last_unstale":"2024-09-13T17:50:38.604486+0000","last_undegraded":"2024-09-13T17:50:38.604486+0000","last_fullsized":"2024-09-13T17:50:38.604486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:47.927865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.604169+0000","last_change":"2024-09-13T17:50:34.199008+0000","last_active":"2024-09-13T17:50:38.604169+0000","last_peered":"2024-09-13T17:50:38.604169+0000","last_clean":"2024-09-13T17:50:38.604169+0000","last_became_active":"2024-09-13T17:50:34.197839+0000","last_became_peered":"2024-09-13T17:50:34.197839+0000","last_unstale":"2024-09-13T17:50:38.604169+0000","last_undegraded":"2024-09-13T17:50:38.604169+0000","last_fullsized":"2024-09-13T17:50:38.604169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:51.430710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.996853+0000","last_change":"2024-09-13T17:50:38.611080+0000","last_active":"2024-09-13T17:50:38.996853+0000","last_peered":"2024-09-13T17:50:38.996853+0000","last_clean":"2024-09-13T17:50:38.996853+0000","last_became_active":"2024-09-13T17:50:38.610688+0000","last_became_peered":"2024-09-13T17:50:38.610688+0000","last_unstale":"2024-09-13T17:50:38.996853+0000","last_undegraded":"2024-09-13T17:50:38.996853+0000","last_fullsized":"2024-09-13T17:50:38.996853+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:28.527617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591242+0000","last_change":"2024-09-13T17:50:34.594763+0000","last_active":"2024-09-13T17:50:38.591242+0000","last_peered":"2024-09-13T17:50:38.591242+0000","last_clean":"2024-09-13T17:50:38.591242+0000","last_became_active":"2024-09-13T17:50:34.594471+0000","last_became_peered":"2024-09-13T17:50:34.594471+0000","last_unstale":"2024-09-13T17:50:38.591242+0000","last_undegraded":"2024-09-13T17:50:38.591242+0000","last_fullsized":"2024-09-13T17:50:38.591242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:20:58.718271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589771+0000","last_change":"2024-09-13T17:50:34.209432+0000","last_active":"2024-09-13T17:50:38.589771+0000","last_peered":"2024-09-13T17:50:38.589771+0000","last_clean":"2024-09-13T17:50:38.589771+0000","last_became_active":"2024-09-13T17:50:34.208817+0000","last_became_peered":"2024-09-13T17:50:34.208817+0000","last_unstale":"2024-09-13T17:50:38.589771+0000","last_undegraded":"2024-09-13T17:50:38.589771+0000","last_fullsized":"2024-09-13T17:50:38.589771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:49.587528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.995813+0000","last_change":"2024-09-13T17:50:38.609461+0000","last_active":"2024-09-13T17:50:38.995813+0000","last_peered":"2024-09-13T17:50:38.995813+0000","last_clean":"2024-09-13T17:50:38.995813+0000","last_became_active":"2024-09-13T17:50:38.609128+0000","last_became_peered":"2024-09-13T17:50:38.609128+0000","last_unstale":"2024-09-13T17:50:38.995813+0000","last_undegraded":"2024-09-13T17:50:38.995813+0000","last_fullsized":"2024-09-13T17:50:38.995813+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:02.286825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.604049+0000","last_change":"2024-09-13T17:50:34.592970+0000","last_active":"2024-09-13T17:50:38.604049+0000","last_peered":"2024-09-13T17:50:38.604049+0000","last_clean":"2024-09-13T17:50:38.604049+0000","last_became_active":"2024-09-13T17:50:34.592669+0000","last_became_peered":"2024-09-13T17:50:34.592669+0000","last_unstale":"2024-09-13T17:50:38.604049+0000","last_undegraded":"2024-09-13T17:50:38.604049+0000","last_fullsized":"2024-09-13T17:50:38.604049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:52.031261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583358+0000","last_change":"2024-09-13T17:50:34.200798+0000","last_active":"2024-09-13T17:50:39.583358+0000","last_peered":"2024-09-13T17:50:39.583358+0000","last_clean":"2024-09-13T17:50:39.583358+0000","last_became_active":"2024-09-13T17:50:34.200529+0000","last_became_peered":"2024-09-13T17:50:34.200529+0000","last_unstale":"2024-09-13T17:50:39.583358+0000","last_undegraded":"2024-09-13T17:50:39.583358+0000","last_fullsized":"2024-09-13T17:50:39.583358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:26.334296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.195285+0000","last_change":"2024-09-13T17:50:38.606287+0000","last_active":"2024-09-13T17:50:39.195285+0000","last_peered":"2024-09-13T17:50:39.195285+0000","last_clean":"2024-09-13T17:50:39.195285+0000","last_became_active":"2024-09-13T17:50:38.605826+0000","last_became_peered":"2024-09-13T17:50:38.605826+0000","last_unstale":"2024-09-13T17:50:39.195285+0000","last_undegraded":"2024-09-13T17:50:39.195285+0000","last_fullsized":"2024-09-13T17:50:39.195285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:18.709454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.592530+0000","last_change":"2024-09-13T17:50:34.591579+0000","last_active":"2024-09-13T17:50:38.592530+0000","last_peered":"2024-09-13T17:50:38.592530+0000","last_clean":"2024-09-13T17:50:38.592530+0000","last_became_active":"2024-09-13T17:50:34.586165+0000","last_became_peered":"2024-09-13T17:50:34.586165+0000","last_unstale":"2024-09-13T17:50:38.592530+0000","last_undegraded":"2024-09-13T17:50:38.592530+0000","last_fullsized":"2024-09-13T17:50:38.592530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:56:19.205176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:47.801705+0000","last_change":"2024-09-13T17:50:34.203218+0000","last_active":"2024-09-13T17:50:47.801705+0000","last_peered":"2024-09-13T17:50:47.801705+0000","last_clean":"2024-09-13T17:50:47.801705+0000","last_became_active":"2024-09-13T17:50:34.202948+0000","last_became_peered":"2024-09-13T17:50:34.202948+0000","last_unstale":"2024-09-13T17:50:47.801705+0000","last_undegraded":"2024-09-13T17:50:47.801705+0000","last_fullsized":"2024-09-13T17:50:47.801705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:46.535569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.000203+0000","last_change":"2024-09-13T17:50:38.610597+0000","last_active":"2024-09-13T17:50:39.000203+0000","last_peered":"2024-09-13T17:50:39.000203+0000","last_clean":"2024-09-13T17:50:39.000203+0000","last_became_active":"2024-09-13T17:50:38.610055+0000","last_became_peered":"2024-09-13T17:50:38.610055+0000","last_unstale":"2024-09-13T17:50:39.000203+0000","last_undegraded":"2024-09-13T17:50:39.000203+0000","last_fullsized":"2024-09-13T17:50:39.000203+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:55.754936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.602441+0000","last_change":"2024-09-13T17:50:34.593963+0000","last_active":"2024-09-13T17:50:38.602441+0000","last_peered":"2024-09-13T17:50:38.602441+0000","last_clean":"2024-09-13T17:50:38.602441+0000","last_became_active":"2024-09-13T17:50:34.593454+0000","last_became_peered":"2024-09-13T17:50:34.593454+0000","last_unstale":"2024-09-13T17:50:38.602441+0000","last_undegraded":"2024-09-13T17:50:38.602441+0000","last_fullsized":"2024-09-13T17:50:38.602441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:18.883521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583588+0000","last_change":"2024-09-13T17:50:34.207936+0000","last_active":"2024-09-13T17:50:39.583588+0000","last_peered":"2024-09-13T17:50:39.583588+0000","last_clean":"2024-09-13T17:50:39.583588+0000","last_became_active":"2024-09-13T17:50:34.207605+0000","last_became_peered":"2024-09-13T17:50:34.207605+0000","last_unstale":"2024-09-13T17:50:39.583588+0000","last_undegraded":"2024-09-13T17:50:39.583588+0000","last_fullsized":"2024-09-13T17:50:39.583588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:05.494406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501828+0000","last_change":"2024-09-13T17:50:38.616297+0000","last_active":"2024-09-13T17:50:39.501828+0000","last_peered":"2024-09-13T17:50:39.501828+0000","last_clean":"2024-09-13T17:50:39.501828+0000","last_became_active":"2024-09-13T17:50:38.614165+0000","last_became_peered":"2024-09-13T17:50:38.614165+0000","last_unstale":"2024-09-13T17:50:39.501828+0000","last_undegraded":"2024-09-13T17:50:39.501828+0000","last_fullsized":"2024-09-13T17:50:39.501828+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:11.357026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.598628+0000","last_change":"2024-09-13T17:50:34.583001+0000","last_active":"2024-09-13T17:50:38.598628+0000","last_peered":"2024-09-13T17:50:38.598628+0000","last_clean":"2024-09-13T17:50:38.598628+0000","last_became_active":"2024-09-13T17:50:34.582529+0000","last_became_peered":"2024-09-13T17:50:34.582529+0000","last_unstale":"2024-09-13T17:50:38.598628+0000","last_undegraded":"2024-09-13T17:50:38.598628+0000","last_fullsized":"2024-09-13T17:50:38.598628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:05.287181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591968+0000","last_change":"2024-09-13T17:50:34.205138+0000","last_active":"2024-09-13T17:50:38.591968+0000","last_peered":"2024-09-13T17:50:38.591968+0000","last_clean":"2024-09-13T17:50:38.591968+0000","last_became_active":"2024-09-13T17:50:34.203360+0000","last_became_peered":"2024-09-13T17:50:34.203360+0000","last_unstale":"2024-09-13T17:50:38.591968+0000","last_undegraded":"2024-09-13T17:50:38.591968+0000","last_fullsized":"2024-09-13T17:50:38.591968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:35.756963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.995168+0000","last_change":"2024-09-13T17:50:38.611036+0000","last_active":"2024-09-13T17:50:38.995168+0000","last_peered":"2024-09-13T17:50:38.995168+0000","last_clean":"2024-09-13T17:50:38.995168+0000","last_became_active":"2024-09-13T17:50:38.610692+0000","last_became_peered":"2024-09-13T17:50:38.610692+0000","last_unstale":"2024-09-13T17:50:38.995168+0000","last_undegraded":"2024-09-13T17:50:38.995168+0000","last_fullsized":"2024-09-13T17:50:38.995168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:10.147998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591770+0000","last_change":"2024-09-13T17:50:34.596882+0000","last_active":"2024-09-13T17:50:38.591770+0000","last_peered":"2024-09-13T17:50:38.591770+0000","last_clean":"2024-09-13T17:50:38.591770+0000","last_became_active":"2024-09-13T17:50:34.596495+0000","last_became_peered":"2024-09-13T17:50:34.596495+0000","last_unstale":"2024-09-13T17:50:38.591770+0000","last_undegraded":"2024-09-13T17:50:38.591770+0000","last_fullsized":"2024-09-13T17:50:38.591770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:21.441114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.609505+0000","last_change":"2024-09-13T17:50:34.204376+0000","last_active":"2024-09-13T17:50:38.609505+0000","last_peered":"2024-09-13T17:50:38.609505+0000","last_clean":"2024-09-13T17:50:38.609505+0000","last_became_active":"2024-09-13T17:50:34.204003+0000","last_became_peered":"2024-09-13T17:50:34.204003+0000","last_unstale":"2024-09-13T17:50:38.609505+0000","last_undegraded":"2024-09-13T17:50:38.609505+0000","last_fullsized":"2024-09-13T17:50:38.609505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:14.023990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.995437+0000","last_change":"2024-09-13T17:50:38.606144+0000","last_active":"2024-09-13T17:50:38.995437+0000","last_peered":"2024-09-13T17:50:38.995437+0000","last_clean":"2024-09-13T17:50:38.995437+0000","last_became_active":"2024-09-13T17:50:38.605646+0000","last_became_peered":"2024-09-13T17:50:38.605646+0000","last_unstale":"2024-09-13T17:50:38.995437+0000","last_undegraded":"2024-09-13T17:50:38.995437+0000","last_fullsized":"2024-09-13T17:50:38.995437+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:07.120075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.500554+0000","last_change":"2024-09-13T17:50:38.618855+0000","last_active":"2024-09-13T17:50:39.500554+0000","last_peered":"2024-09-13T17:50:39.500554+0000","last_clean":"2024-09-13T17:50:39.500554+0000","last_became_active":"2024-09-13T17:50:38.618449+0000","last_became_peered":"2024-09-13T17:50:38.618449+0000","last_unstale":"2024-09-13T17:50:39.500554+0000","last_undegraded":"2024-09-13T17:50:39.500554+0000","last_fullsized":"2024-09-13T17:50:39.500554+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:29:01.114087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.600195+0000","last_change":"2024-09-13T17:50:34.590079+0000","last_active":"2024-09-13T17:50:38.600195+0000","last_peered":"2024-09-13T17:50:38.600195+0000","last_clean":"2024-09-13T17:50:38.600195+0000","last_became_active":"2024-09-13T17:50:34.589396+0000","last_became_peered":"2024-09-13T17:50:34.589396+0000","last_unstale":"2024-09-13T17:50:38.600195+0000","last_undegraded":"2024-09-13T17:50:38.600195+0000","last_fullsized":"2024-09-13T17:50:38.600195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:03.308341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589925+0000","last_change":"2024-09-13T17:50:34.206575+0000","last_active":"2024-09-13T17:50:38.589925+0000","last_peered":"2024-09-13T17:50:38.589925+0000","last_clean":"2024-09-13T17:50:38.589925+0000","last_became_active":"2024-09-13T17:50:34.206308+0000","last_became_peered":"2024-09-13T17:50:34.206308+0000","last_unstale":"2024-09-13T17:50:38.589925+0000","last_undegraded":"2024-09-13T17:50:38.589925+0000","last_fullsized":"2024-09-13T17:50:38.589925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:34.766538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.982710+0000","last_change":"2024-09-13T17:50:38.618131+0000","last_active":"2024-09-13T17:50:38.982710+0000","last_peered":"2024-09-13T17:50:38.982710+0000","last_clean":"2024-09-13T17:50:38.982710+0000","last_became_active":"2024-09-13T17:50:38.617611+0000","last_became_peered":"2024-09-13T17:50:38.617611+0000","last_unstale":"2024-09-13T17:50:38.982710+0000","last_undegraded":"2024-09-13T17:50:38.982710+0000","last_fullsized":"2024-09-13T17:50:38.982710+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:30.746445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589814+0000","last_change":"2024-09-13T17:50:34.582153+0000","last_active":"2024-09-13T17:50:38.589814+0000","last_peered":"2024-09-13T17:50:38.589814+0000","last_clean":"2024-09-13T17:50:38.589814+0000","last_became_active":"2024-09-13T17:50:34.581708+0000","last_became_peered":"2024-09-13T17:50:34.581708+0000","last_unstale":"2024-09-13T17:50:38.589814+0000","last_undegraded":"2024-09-13T17:50:38.589814+0000","last_fullsized":"2024-09-13T17:50:38.589814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:22.238810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590175+0000","last_change":"2024-09-13T17:50:34.205489+0000","last_active":"2024-09-13T17:50:38.590175+0000","last_peered":"2024-09-13T17:50:38.590175+0000","last_clean":"2024-09-13T17:50:38.590175+0000","last_became_active":"2024-09-13T17:50:34.205202+0000","last_became_peered":"2024-09-13T17:50:34.205202+0000","last_unstale":"2024-09-13T17:50:38.590175+0000","last_undegraded":"2024-09-13T17:50:38.590175+0000","last_fullsized":"2024-09-13T17:50:38.590175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:54.450276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.983249+0000","last_change":"2024-09-13T17:50:34.202470+0000","last_active":"2024-09-13T17:50:38.983249+0000","last_peered":"2024-09-13T17:50:38.983249+0000","last_clean":"2024-09-13T17:50:38.983249+0000","last_became_active":"2024-09-13T17:50:34.202163+0000","last_became_peered":"2024-09-13T17:50:34.202163+0000","last_unstale":"2024-09-13T17:50:38.983249+0000","last_undegraded":"2024-09-13T17:50:38.983249+0000","last_fullsized":"2024-09-13T17:50:38.983249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:03.472932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.603393+0000","last_change":"2024-09-13T17:50:34.590947+0000","last_active":"2024-09-13T17:50:38.603393+0000","last_peered":"2024-09-13T17:50:38.603393+0000","last_clean":"2024-09-13T17:50:38.603393+0000","last_became_active":"2024-09-13T17:50:34.590497+0000","last_became_peered":"2024-09-13T17:50:34.590497+0000","last_unstale":"2024-09-13T17:50:38.603393+0000","last_undegraded":"2024-09-13T17:50:38.603393+0000","last_fullsized":"2024-09-13T17:50:38.603393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:56:27.848092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.980209+0000","last_change":"2024-09-13T17:50:38.618518+0000","last_active":"2024-09-13T17:50:38.980209+0000","last_peered":"2024-09-13T17:50:38.980209+0000","last_clean":"2024-09-13T17:50:38.980209+0000","last_became_active":"2024-09-13T17:50:38.618108+0000","last_became_peered":"2024-09-13T17:50:38.618108+0000","last_unstale":"2024-09-13T17:50:38.980209+0000","last_undegraded":"2024-09-13T17:50:38.980209+0000","last_fullsized":"2024-09-13T17:50:38.980209+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:42.260237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"25'1","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.996384+0000","last_change":"2024-09-13T17:50:34.218459+0000","last_active":"2024-09-13T17:50:38.996384+0000","last_peered":"2024-09-13T17:50:38.996384+0000","last_clean":"2024-09-13T17:50:38.996384+0000","last_became_active":"2024-09-13T17:50:34.218087+0000","last_became_peered":"2024-09-13T17:50:34.218087+0000","last_unstale":"2024-09-13T17:50:38.996384+0000","last_undegraded":"2024-09-13T17:50:38.996384+0000","last_fullsized":"2024-09-13T17:50:38.996384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:56.914544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.603583+0000","last_change":"2024-09-13T17:50:34.583845+0000","last_active":"2024-09-13T17:50:38.603583+0000","last_peered":"2024-09-13T17:50:38.603583+0000","last_clean":"2024-09-13T17:50:38.603583+0000","last_became_active":"2024-09-13T17:50:34.583283+0000","last_became_peered":"2024-09-13T17:50:34.583283+0000","last_unstale":"2024-09-13T17:50:38.603583+0000","last_undegraded":"2024-09-13T17:50:38.603583+0000","last_fullsized":"2024-09-13T17:50:38.603583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:55.630285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.996396+0000","last_change":"2024-09-13T17:50:38.611987+0000","last_active":"2024-09-13T17:50:38.996396+0000","last_peered":"2024-09-13T17:50:38.996396+0000","last_clean":"2024-09-13T17:50:38.996396+0000","last_became_active":"2024-09-13T17:50:38.611636+0000","last_became_peered":"2024-09-13T17:50:38.611636+0000","last_unstale":"2024-09-13T17:50:38.996396+0000","last_undegraded":"2024-09-13T17:50:38.996396+0000","last_fullsized":"2024-09-13T17:50:38.996396+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:41.111562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.602963+0000","last_change":"2024-09-13T17:50:34.588927+0000","last_active":"2024-09-13T17:50:38.602963+0000","last_peered":"2024-09-13T17:50:38.602963+0000","last_clean":"2024-09-13T17:50:38.602963+0000","last_became_active":"2024-09-13T17:50:34.588651+0000","last_became_peered":"2024-09-13T17:50:34.588651+0000","last_unstale":"2024-09-13T17:50:38.602963+0000","last_undegraded":"2024-09-13T17:50:38.602963+0000","last_fullsized":"2024-09-13T17:50:38.602963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:49.429670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.597005+0000","last_change":"2024-09-13T17:50:34.583795+0000","last_active":"2024-09-13T17:50:38.597005+0000","last_peered":"2024-09-13T17:50:38.597005+0000","last_clean":"2024-09-13T17:50:38.597005+0000","last_became_active":"2024-09-13T17:50:34.583493+0000","last_became_peered":"2024-09-13T17:50:34.583493+0000","last_unstale":"2024-09-13T17:50:38.597005+0000","last_undegraded":"2024-09-13T17:50:38.597005+0000","last_fullsized":"2024-09-13T17:50:38.597005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:30.463932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589528+0000","last_change":"2024-09-13T17:50:34.587624+0000","last_active":"2024-09-13T17:50:38.589528+0000","last_peered":"2024-09-13T17:50:38.589528+0000","last_clean":"2024-09-13T17:50:38.589528+0000","last_became_active":"2024-09-13T17:50:34.587119+0000","last_became_peered":"2024-09-13T17:50:34.587119+0000","last_unstale":"2024-09-13T17:50:38.589528+0000","last_undegraded":"2024-09-13T17:50:38.589528+0000","last_fullsized":"2024-09-13T17:50:38.589528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:16.783899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599861+0000","last_change":"2024-09-13T17:50:34.581923+0000","last_active":"2024-09-13T17:50:38.599861+0000","last_peered":"2024-09-13T17:50:38.599861+0000","last_clean":"2024-09-13T17:50:38.599861+0000","last_became_active":"2024-09-13T17:50:34.581628+0000","last_became_peered":"2024-09-13T17:50:34.581628+0000","last_unstale":"2024-09-13T17:50:38.599861+0000","last_undegraded":"2024-09-13T17:50:38.599861+0000","last_fullsized":"2024-09-13T17:50:38.599861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:28:55.936864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589257+0000","last_change":"2024-09-13T17:50:34.595037+0000","last_active":"2024-09-13T17:50:38.589257+0000","last_peered":"2024-09-13T17:50:38.589257+0000","last_clean":"2024-09-13T17:50:38.589257+0000","last_became_active":"2024-09-13T17:50:34.594857+0000","last_became_peered":"2024-09-13T17:50:34.594857+0000","last_unstale":"2024-09-13T17:50:38.589257+0000","last_undegraded":"2024-09-13T17:50:38.589257+0000","last_fullsized":"2024-09-13T17:50:38.589257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:11.420413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.597995+0000","last_change":"2024-09-13T17:50:34.589651+0000","last_active":"2024-09-13T17:50:38.597995+0000","last_peered":"2024-09-13T17:50:38.597995+0000","last_clean":"2024-09-13T17:50:38.597995+0000","last_became_active":"2024-09-13T17:50:34.589354+0000","last_became_peered":"2024-09-13T17:50:34.589354+0000","last_unstale":"2024-09-13T17:50:38.597995+0000","last_undegraded":"2024-09-13T17:50:38.597995+0000","last_fullsized":"2024-09-13T17:50:38.597995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:31.294048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.609987+0000","last_change":"2024-09-13T17:50:34.597949+0000","last_active":"2024-09-13T17:50:38.609987+0000","last_peered":"2024-09-13T17:50:38.609987+0000","last_clean":"2024-09-13T17:50:38.609987+0000","last_became_active":"2024-09-13T17:50:34.597621+0000","last_became_peered":"2024-09-13T17:50:34.597621+0000","last_unstale":"2024-09-13T17:50:38.609987+0000","last_undegraded":"2024-09-13T17:50:38.609987+0000","last_fullsized":"2024-09-13T17:50:38.609987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:23.133462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591133+0000","last_change":"2024-09-13T17:50:34.589893+0000","last_active":"2024-09-13T17:50:38.591133+0000","last_peered":"2024-09-13T17:50:38.591133+0000","last_clean":"2024-09-13T17:50:38.591133+0000","last_became_active":"2024-09-13T17:50:34.589626+0000","last_became_peered":"2024-09-13T17:50:34.589626+0000","last_unstale":"2024-09-13T17:50:38.591133+0000","last_undegraded":"2024-09-13T17:50:38.591133+0000","last_fullsized":"2024-09-13T17:50:38.591133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:41.321025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589874+0000","last_change":"2024-09-13T17:50:34.590046+0000","last_active":"2024-09-13T17:50:38.589874+0000","last_peered":"2024-09-13T17:50:38.589874+0000","last_clean":"2024-09-13T17:50:38.589874+0000","last_became_active":"2024-09-13T17:50:34.589757+0000","last_became_peered":"2024-09-13T17:50:34.589757+0000","last_unstale":"2024-09-13T17:50:38.589874+0000","last_undegraded":"2024-09-13T17:50:38.589874+0000","last_fullsized":"2024-09-13T17:50:38.589874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:28.239550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589800+0000","last_change":"2024-09-13T17:50:34.595518+0000","last_active":"2024-09-13T17:50:38.589800+0000","last_peered":"2024-09-13T17:50:38.589800+0000","last_clean":"2024-09-13T17:50:38.589800+0000","last_became_active":"2024-09-13T17:50:34.595242+0000","last_became_peered":"2024-09-13T17:50:34.595242+0000","last_unstale":"2024-09-13T17:50:38.589800+0000","last_undegraded":"2024-09-13T17:50:38.589800+0000","last_fullsized":"2024-09-13T17:50:38.589800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:26.149384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589330+0000","last_change":"2024-09-13T17:50:34.593183+0000","last_active":"2024-09-13T17:50:38.589330+0000","last_peered":"2024-09-13T17:50:38.589330+0000","last_clean":"2024-09-13T17:50:38.589330+0000","last_became_active":"2024-09-13T17:50:34.592893+0000","last_became_peered":"2024-09-13T17:50:34.592893+0000","last_unstale":"2024-09-13T17:50:38.589330+0000","last_undegraded":"2024-09-13T17:50:38.589330+0000","last_fullsized":"2024-09-13T17:50:38.589330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:19.459505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599698+0000","last_change":"2024-09-13T17:50:34.594679+0000","last_active":"2024-09-13T17:50:38.599698+0000","last_peered":"2024-09-13T17:50:38.599698+0000","last_clean":"2024-09-13T17:50:38.599698+0000","last_became_active":"2024-09-13T17:50:34.593750+0000","last_became_peered":"2024-09-13T17:50:34.593750+0000","last_unstale":"2024-09-13T17:50:38.599698+0000","last_undegraded":"2024-09-13T17:50:38.599698+0000","last_fullsized":"2024-09-13T17:50:38.599698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:02.135577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.609111+0000","last_change":"2024-09-13T17:50:34.598736+0000","last_active":"2024-09-13T17:50:38.609111+0000","last_peered":"2024-09-13T17:50:38.609111+0000","last_clean":"2024-09-13T17:50:38.609111+0000","last_became_active":"2024-09-13T17:50:34.598433+0000","last_became_peered":"2024-09-13T17:50:34.598433+0000","last_unstale":"2024-09-13T17:50:38.609111+0000","last_undegraded":"2024-09-13T17:50:38.609111+0000","last_fullsized":"2024-09-13T17:50:38.609111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:08.975139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.592906+0000","last_change":"2024-09-13T17:50:34.591795+0000","last_active":"2024-09-13T17:50:38.592906+0000","last_peered":"2024-09-13T17:50:38.592906+0000","last_clean":"2024-09-13T17:50:38.592906+0000","last_became_active":"2024-09-13T17:50:34.591286+0000","last_became_peered":"2024-09-13T17:50:34.591286+0000","last_unstale":"2024-09-13T17:50:38.592906+0000","last_undegraded":"2024-09-13T17:50:38.592906+0000","last_fullsized":"2024-09-13T17:50:38.592906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:31.580247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.595844+0000","last_change":"2024-09-13T17:50:34.587774+0000","last_active":"2024-09-13T17:50:38.595844+0000","last_peered":"2024-09-13T17:50:38.595844+0000","last_clean":"2024-09-13T17:50:38.595844+0000","last_became_active":"2024-09-13T17:50:34.587506+0000","last_became_peered":"2024-09-13T17:50:34.587506+0000","last_unstale":"2024-09-13T17:50:38.595844+0000","last_undegraded":"2024-09-13T17:50:38.595844+0000","last_fullsized":"2024-09-13T17:50:38.595844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:46.275528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591171+0000","last_change":"2024-09-13T17:50:34.596100+0000","last_active":"2024-09-13T17:50:38.591171+0000","last_peered":"2024-09-13T17:50:38.591171+0000","last_clean":"2024-09-13T17:50:38.591171+0000","last_became_active":"2024-09-13T17:50:34.595798+0000","last_became_peered":"2024-09-13T17:50:34.595798+0000","last_unstale":"2024-09-13T17:50:38.591171+0000","last_undegraded":"2024-09-13T17:50:38.591171+0000","last_fullsized":"2024-09-13T17:50:38.591171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:49.543845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589759+0000","last_change":"2024-09-13T17:50:34.584818+0000","last_active":"2024-09-13T17:50:38.589759+0000","last_peered":"2024-09-13T17:50:38.589759+0000","last_clean":"2024-09-13T17:50:38.589759+0000","last_became_active":"2024-09-13T17:50:34.584516+0000","last_became_peered":"2024-09-13T17:50:34.584516+0000","last_unstale":"2024-09-13T17:50:38.589759+0000","last_undegraded":"2024-09-13T17:50:38.589759+0000","last_fullsized":"2024-09-13T17:50:38.589759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:52.402024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599531+0000","last_change":"2024-09-13T17:50:34.598886+0000","last_active":"2024-09-13T17:50:38.599531+0000","last_peered":"2024-09-13T17:50:38.599531+0000","last_clean":"2024-09-13T17:50:38.599531+0000","last_became_active":"2024-09-13T17:50:34.598648+0000","last_became_peered":"2024-09-13T17:50:34.598648+0000","last_unstale":"2024-09-13T17:50:38.599531+0000","last_undegraded":"2024-09-13T17:50:38.599531+0000","last_fullsized":"2024-09-13T17:50:38.599531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:00:43.063223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.603939+0000","last_change":"2024-09-13T17:50:34.582649+0000","last_active":"2024-09-13T17:50:38.603939+0000","last_peered":"2024-09-13T17:50:38.603939+0000","last_clean":"2024-09-13T17:50:38.603939+0000","last_became_active":"2024-09-13T17:50:34.581699+0000","last_became_peered":"2024-09-13T17:50:34.581699+0000","last_unstale":"2024-09-13T17:50:38.603939+0000","last_undegraded":"2024-09-13T17:50:38.603939+0000","last_fullsized":"2024-09-13T17:50:38.603939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:18.959303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.588487+0000","last_change":"2024-09-13T17:50:34.587484+0000","last_active":"2024-09-13T17:50:38.588487+0000","last_peered":"2024-09-13T17:50:38.588487+0000","last_clean":"2024-09-13T17:50:38.588487+0000","last_became_active":"2024-09-13T17:50:34.587217+0000","last_became_peered":"2024-09-13T17:50:34.587217+0000","last_unstale":"2024-09-13T17:50:38.588487+0000","last_undegraded":"2024-09-13T17:50:38.588487+0000","last_fullsized":"2024-09-13T17:50:38.588487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:24:19.172607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.602349+0000","last_change":"2024-09-13T17:50:34.592871+0000","last_active":"2024-09-13T17:50:38.602349+0000","last_peered":"2024-09-13T17:50:38.602349+0000","last_clean":"2024-09-13T17:50:38.602349+0000","last_became_active":"2024-09-13T17:50:34.588513+0000","last_became_peered":"2024-09-13T17:50:34.588513+0000","last_unstale":"2024-09-13T17:50:38.602349+0000","last_undegraded":"2024-09-13T17:50:38.602349+0000","last_fullsized":"2024-09-13T17:50:38.602349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:33.131420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.608986+0000","last_change":"2024-09-13T17:50:34.585713+0000","last_active":"2024-09-13T17:50:38.608986+0000","last_peered":"2024-09-13T17:50:38.608986+0000","last_clean":"2024-09-13T17:50:38.608986+0000","last_became_active":"2024-09-13T17:50:34.585438+0000","last_became_peered":"2024-09-13T17:50:34.585438+0000","last_unstale":"2024-09-13T17:50:38.608986+0000","last_undegraded":"2024-09-13T17:50:38.608986+0000","last_fullsized":"2024-09-13T17:50:38.608986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:56:49.905393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.609903+0000","last_change":"2024-09-13T17:50:34.589628+0000","last_active":"2024-09-13T17:50:38.609903+0000","last_peered":"2024-09-13T17:50:38.609903+0000","last_clean":"2024-09-13T17:50:38.609903+0000","last_became_active":"2024-09-13T17:50:34.587665+0000","last_became_peered":"2024-09-13T17:50:34.587665+0000","last_unstale":"2024-09-13T17:50:38.609903+0000","last_undegraded":"2024-09-13T17:50:38.609903+0000","last_fullsized":"2024-09-13T17:50:38.609903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:15.518264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.593116+0000","last_change":"2024-09-13T17:50:34.591776+0000","last_active":"2024-09-13T17:50:38.593116+0000","last_peered":"2024-09-13T17:50:38.593116+0000","last_clean":"2024-09-13T17:50:38.593116+0000","last_became_active":"2024-09-13T17:50:34.591496+0000","last_became_peered":"2024-09-13T17:50:34.591496+0000","last_unstale":"2024-09-13T17:50:38.593116+0000","last_undegraded":"2024-09-13T17:50:38.593116+0000","last_fullsized":"2024-09-13T17:50:38.593116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:48.651999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.588146+0000","last_change":"2024-09-13T17:50:34.582661+0000","last_active":"2024-09-13T17:50:38.588146+0000","last_peered":"2024-09-13T17:50:38.588146+0000","last_clean":"2024-09-13T17:50:38.588146+0000","last_became_active":"2024-09-13T17:50:34.582400+0000","last_became_peered":"2024-09-13T17:50:34.582400+0000","last_unstale":"2024-09-13T17:50:38.588146+0000","last_undegraded":"2024-09-13T17:50:38.588146+0000","last_fullsized":"2024-09-13T17:50:38.588146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:16:59.648638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589018+0000","last_change":"2024-09-13T17:50:34.585531+0000","last_active":"2024-09-13T17:50:38.589018+0000","last_peered":"2024-09-13T17:50:38.589018+0000","last_clean":"2024-09-13T17:50:38.589018+0000","last_became_active":"2024-09-13T17:50:34.585226+0000","last_became_peered":"2024-09-13T17:50:34.585226+0000","last_unstale":"2024-09-13T17:50:38.589018+0000","last_undegraded":"2024-09-13T17:50:38.589018+0000","last_fullsized":"2024-09-13T17:50:38.589018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:17.105028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.589126+0000","last_change":"2024-09-13T17:50:34.586750+0000","last_active":"2024-09-13T17:50:38.589126+0000","last_peered":"2024-09-13T17:50:38.589126+0000","last_clean":"2024-09-13T17:50:38.589126+0000","last_became_active":"2024-09-13T17:50:34.584564+0000","last_became_peered":"2024-09-13T17:50:34.584564+0000","last_unstale":"2024-09-13T17:50:38.589126+0000","last_undegraded":"2024-09-13T17:50:38.589126+0000","last_fullsized":"2024-09-13T17:50:38.589126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:54.033257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.591038+0000","last_change":"2024-09-13T17:50:34.585938+0000","last_active":"2024-09-13T17:50:38.591038+0000","last_peered":"2024-09-13T17:50:38.591038+0000","last_clean":"2024-09-13T17:50:38.591038+0000","last_became_active":"2024-09-13T17:50:34.585584+0000","last_became_peered":"2024-09-13T17:50:34.585584+0000","last_unstale":"2024-09-13T17:50:38.591038+0000","last_undegraded":"2024-09-13T17:50:38.591038+0000","last_fullsized":"2024-09-13T17:50:38.591038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:32.623468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.590323+0000","last_change":"2024-09-13T17:50:34.588762+0000","last_active":"2024-09-13T17:50:38.590323+0000","last_peered":"2024-09-13T17:50:38.590323+0000","last_clean":"2024-09-13T17:50:38.590323+0000","last_became_active":"2024-09-13T17:50:34.588489+0000","last_became_peered":"2024-09-13T17:50:34.588489+0000","last_unstale":"2024-09-13T17:50:38.590323+0000","last_undegraded":"2024-09-13T17:50:38.590323+0000","last_fullsized":"2024-09-13T17:50:38.590323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:38.266297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.598178+0000","last_change":"2024-09-13T17:50:34.583889+0000","last_active":"2024-09-13T17:50:38.598178+0000","last_peered":"2024-09-13T17:50:38.598178+0000","last_clean":"2024-09-13T17:50:38.598178+0000","last_became_active":"2024-09-13T17:50:34.583551+0000","last_became_peered":"2024-09-13T17:50:34.583551+0000","last_unstale":"2024-09-13T17:50:38.598178+0000","last_undegraded":"2024-09-13T17:50:38.598178+0000","last_fullsized":"2024-09-13T17:50:38.598178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:19.888626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.599385+0000","last_change":"2024-09-13T17:50:34.585617+0000","last_active":"2024-09-13T17:50:38.599385+0000","last_peered":"2024-09-13T17:50:38.599385+0000","last_clean":"2024-09-13T17:50:38.599385+0000","last_became_active":"2024-09-13T17:50:34.585340+0000","last_became_peered":"2024-09-13T17:50:34.585340+0000","last_unstale":"2024-09-13T17:50:38.599385+0000","last_undegraded":"2024-09-13T17:50:38.599385+0000","last_fullsized":"2024-09-13T17:50:38.599385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:18.331045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.593186+0000","last_change":"2024-09-13T17:50:34.592365+0000","last_active":"2024-09-13T17:50:38.593186+0000","last_peered":"2024-09-13T17:50:38.593186+0000","last_clean":"2024-09-13T17:50:38.593186+0000","last_became_active":"2024-09-13T17:50:34.592046+0000","last_became_peered":"2024-09-13T17:50:34.592046+0000","last_unstale":"2024-09-13T17:50:38.593186+0000","last_undegraded":"2024-09-13T17:50:38.593186+0000","last_fullsized":"2024-09-13T17:50:38.593186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:54.283683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4593,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509451,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":105750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":258048,"data_stored":101955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":102115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509452,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":864256,"data_stored":692961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542155,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":856064,"data_stored":692905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]}]},{"osd":6,"up_from":14,"seq":60129542155,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":270336,"data_stored":105750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5429999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3039999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6679999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7490000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542155,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":262144,"data_stored":102045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3360000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.357}]}]},{"osd":4,"up_from":14,"seq":60129542155,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":270336,"data_stored":102081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:51:08.774 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T17:51:08.774 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T17:51:08.774 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T17:51:08.774 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T17:51:08.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.870+0000 7f246c871700 1 Processor -- start 2024-09-13T17:51:08.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.870+0000 7f246c871700 1 -- start start 2024-09-13T17:51:08.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.870+0000 7f246c871700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f246815cf60 0x7f246815d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:08.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.870+0000 7f246c871700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f246815d910 con 0x7f24680cb2f0 2024-09-13T17:51:08.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.870+0000 7f246c871700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2468159770 con 0x7f2468153880 2024-09-13T17:51:08.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.870+0000 7f246c871700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2468159a50 con 0x7f246815cf60 2024-09-13T17:51:08.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.870+0000 7f246659c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f24680cb2f0 0x7f24680cee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46200/0 (socket says 172.21.15.40:46200) 2024-09-13T17:51:08.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.870+0000 7f246659c700 1 -- 172.21.15.40:0/452222244 learned_addr learned my addr 172.21.15.40:0/452222244 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:08.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.870+0000 7f246759e700 1 --2- 172.21.15.40:0/452222244 >> v2:172.21.15.40:3300/0 conn(0x7f246815cf60 0x7f246815d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:08.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.870+0000 7f2465d9b700 1 -- 172.21.15.40:0/452222244 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446053836 0 0) 0x7f246815d910 con 0x7f24680cb2f0 2024-09-13T17:51:08.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.870+0000 7f2465d9b700 1 -- 172.21.15.40:0/452222244 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24500035d0 con 0x7f24680cb2f0 2024-09-13T17:51:08.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.874+0000 7f246759e700 1 -- 172.21.15.40:0/452222244 >> v1:172.21.15.43:6789/0 conn(0x7f2468153880 legacy=0x7f24680c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.874+0000 7f2465d9b700 1 -- 172.21.15.40:0/452222244 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 430997845 0 0) 0x7f2468159770 con 0x7f2468153880 2024-09-13T17:51:08.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.874+0000 7f246759e700 1 -- 172.21.15.40:0/452222244 >> v1:172.21.15.40:6789/0 conn(0x7f24680cb2f0 legacy=0x7f24680cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.874+0000 7f246759e700 1 -- 172.21.15.40:0/452222244 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2468159d30 con 0x7f246815cf60 2024-09-13T17:51:08.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.874+0000 7f246759e700 1 --2- 172.21.15.40:0/452222244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f246815cf60 0x7f246815d3d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2460009d90 tx=0x7f246000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845fca034813063 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:08.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.874+0000 7f2465d9b700 1 -- 172.21.15.40:0/452222244 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246000e070 con 0x7f246815cf60 2024-09-13T17:51:08.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.874+0000 7f2465d9b700 1 -- 172.21.15.40:0/452222244 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246000bb20 con 0x7f246815cf60 2024-09-13T17:51:08.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.874+0000 7f2465d9b700 1 -- 172.21.15.40:0/452222244 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246000be90 con 0x7f246815cf60 2024-09-13T17:51:08.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.874+0000 7f246c871700 1 -- 172.21.15.40:0/452222244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f246815cf60 msgr2=0x7f246815d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.874+0000 7f246c871700 1 --2- 172.21.15.40:0/452222244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f246815cf60 0x7f246815d3d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2460009d90 tx=0x7f246000b330 comp rx=0 tx=0).stop 2024-09-13T17:51:08.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.874+0000 7f246c871700 1 -- 172.21.15.40:0/452222244 shutdown_connections 2024-09-13T17:51:08.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.874+0000 7f246c871700 1 --2- 172.21.15.40:0/452222244 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f246815cf60 0x7f246815d3d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.874+0000 7f246c871700 1 -- 172.21.15.40:0/452222244 >> 172.21.15.40:0/452222244 conn(0x7f246806c5b0 msgr2=0x7f246806c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:08.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.874+0000 7f246c871700 1 -- 172.21.15.40:0/452222244 shutdown_connections 2024-09-13T17:51:08.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.874+0000 7f246c871700 1 -- 172.21.15.40:0/452222244 wait complete. 2024-09-13T17:51:08.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.874+0000 7f246c871700 1 Processor -- start 2024-09-13T17:51:08.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.874+0000 7f246c871700 1 -- start start 2024-09-13T17:51:08.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f246c871700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24680cb2f0 0x7f2468158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:08.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f246c871700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2468153880 0x7f24680ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:08.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f246c871700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f246815cf60 0x7f24681542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:08.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f246c871700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24680cd4b0 con 0x7f24680cb2f0 2024-09-13T17:51:08.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f246c871700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246815b2e0 con 0x7f2468153880 2024-09-13T17:51:08.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f246c871700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246815b970 con 0x7f246815cf60 2024-09-13T17:51:08.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f2466d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24680cb2f0 0x7f2468158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:08.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f2466d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24680cb2f0 0x7f2468158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58240/0 (socket says 172.21.15.40:58240) 2024-09-13T17:51:08.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f2466d9d700 1 -- 172.21.15.40:0/1516634196 learned_addr learned my addr 172.21.15.40:0/1516634196 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:08.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f246659c700 1 --2- 172.21.15.40:0/1516634196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2468153880 0x7f24680ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:08.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f246759e700 1 --2- 172.21.15.40:0/1516634196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f246815cf60 0x7f24681542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:08.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f2466d9d700 1 -- 172.21.15.40:0/1516634196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f246815cf60 msgr2=0x7f24681542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f2466d9d700 1 --2- 172.21.15.40:0/1516634196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f246815cf60 0x7f24681542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f2466d9d700 1 -- 172.21.15.40:0/1516634196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2468153880 msgr2=0x7f24680ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:08.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f2466d9d700 1 --2- 172.21.15.40:0/1516634196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2468153880 0x7f24680ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:08.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f2466d9d700 1 -- 172.21.15.40:0/1516634196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2460009a70 con 0x7f24680cb2f0 2024-09-13T17:51:08.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f246659c700 1 --2- 172.21.15.40:0/1516634196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2468153880 0x7f24680ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:08.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f2466d9d700 1 --2- 172.21.15.40:0/1516634196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24680cb2f0 0x7f2468158620 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f24580011e0 tx=0x7f245800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:08.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f244f7fe700 1 -- 172.21.15.40:0/1516634196 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24580054e0 con 0x7f24680cb2f0 2024-09-13T17:51:08.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f244f7fe700 1 -- 172.21.15.40:0/1516634196 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24580036a0 con 0x7f24680cb2f0 2024-09-13T17:51:08.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f246c871700 1 -- 172.21.15.40:0/1516634196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f246815bc50 con 0x7f24680cb2f0 2024-09-13T17:51:08.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f244f7fe700 1 -- 172.21.15.40:0/1516634196 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2458003a50 con 0x7f24680cb2f0 2024-09-13T17:51:08.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f246c871700 1 -- 172.21.15.40:0/1516634196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24680cd1c0 con 0x7f24680cb2f0 2024-09-13T17:51:08.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f246c871700 1 -- 172.21.15.40:0/1516634196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f246815c170 con 0x7f24680cb2f0 2024-09-13T17:51:08.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.878+0000 7f244f7fe700 1 -- 172.21.15.40:0/1516634196 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2458003bb0 con 0x7f24680cb2f0 2024-09-13T17:51:08.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.882+0000 7f244f7fe700 1 --2- 172.21.15.40:0/1516634196 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2454066730 0x7f2454068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:08.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.882+0000 7f244f7fe700 1 -- 172.21.15.40:0/1516634196 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f24580838d0 con 0x7f24680cb2f0 2024-09-13T17:51:08.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.882+0000 7f246659c700 1 --2- 172.21.15.40:0/1516634196 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2454066730 0x7f2454068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:08.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.882+0000 7f246659c700 1 --2- 172.21.15.40:0/1516634196 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2454066730 0x7f2454068bf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f24681f3280 tx=0x7f245c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:08.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:08.890+0000 7f244f7fe700 1 -- 172.21.15.40:0/1516634196 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f246815c170 con 0x7f24680cb2f0 2024-09-13T17:51:09.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.090+0000 7f246c871700 1 -- 172.21.15.40:0/1516634196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f246815d3d0 con 0x7f24680cb2f0 2024-09-13T17:51:09.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.090+0000 7f244f7fe700 1 -- 172.21.15.40:0/1516634196 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f2458017080 con 0x7f24680cb2f0 2024-09-13T17:51:09.095 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:09.095 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T17:51:09.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.090+0000 7f246c871700 1 -- 172.21.15.40:0/1516634196 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2454066730 msgr2=0x7f2454068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:09.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.090+0000 7f246c871700 1 --2- 172.21.15.40:0/1516634196 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2454066730 0x7f2454068bf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f24681f3280 tx=0x7f245c009040 comp rx=0 tx=0).stop 2024-09-13T17:51:09.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.090+0000 7f246c871700 1 -- 172.21.15.40:0/1516634196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24680cb2f0 msgr2=0x7f2468158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:09.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.090+0000 7f246c871700 1 --2- 172.21.15.40:0/1516634196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24680cb2f0 0x7f2468158620 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f24580011e0 tx=0x7f245800cae0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.090+0000 7f246c871700 1 -- 172.21.15.40:0/1516634196 shutdown_connections 2024-09-13T17:51:09.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.090+0000 7f246c871700 1 --2- 172.21.15.40:0/1516634196 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2454066730 0x7f2454068bf0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.090+0000 7f246c871700 1 --2- 172.21.15.40:0/1516634196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f246815cf60 0x7f24681542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.090+0000 7f246c871700 1 --2- 172.21.15.40:0/1516634196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2468153880 0x7f24680ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.090+0000 7f246c871700 1 --2- 172.21.15.40:0/1516634196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24680cb2f0 0x7f2468158620 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.090+0000 7f246c871700 1 -- 172.21.15.40:0/1516634196 >> 172.21.15.40:0/1516634196 conn(0x7f246806c5b0 msgr2=0x7f24680c5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:09.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.090+0000 7f246c871700 1 -- 172.21.15.40:0/1516634196 shutdown_connections 2024-09-13T17:51:09.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.090+0000 7f246c871700 1 -- 172.21.15.40:0/1516634196 wait complete. 2024-09-13T17:51:09.108 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T17:51:09.109 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:51:09.150 INFO:tasks.ceph.mds.b.smithi043.stderr:2024-09-13T17:51:09.144+0000 7fb6d9c92700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-13T17:51:09.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.210+0000 7f1625c2a700 1 Processor -- start 2024-09-13T17:51:09.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.210+0000 7f1625c2a700 1 -- start start 2024-09-13T17:51:09.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.210+0000 7f1625c2a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1620071f70 0x7f16200c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:09.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.210+0000 7f1625c2a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1620072350 0x7f1620150d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:09.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.210+0000 7f1625c2a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16200c55c0 con 0x7f16200c7200 2024-09-13T17:51:09.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.210+0000 7f1625c2a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f162006a390 con 0x7f1620072350 2024-09-13T17:51:09.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.210+0000 7f1625c2a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1620157710 con 0x7f1620071f70 2024-09-13T17:51:09.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.210+0000 7f161ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1620072350 0x7f1620150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:09.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.210+0000 7f161ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1620072350 0x7f1620150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58252/0 (socket says 172.21.15.40:58252) 2024-09-13T17:51:09.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.210+0000 7f161ffff700 1 -- 172.21.15.40:0/372744819 learned_addr learned my addr 172.21.15.40:0/372744819 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:09.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.210+0000 7f1624c28700 1 --2- 172.21.15.40:0/372744819 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1620071f70 0x7f16200c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:09.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.210+0000 7f161ffff700 1 -- 172.21.15.40:0/372744819 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1620071f70 msgr2=0x7f16200c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:09.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.210+0000 7f161ffff700 1 --2- 172.21.15.40:0/372744819 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1620071f70 0x7f16200c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.210+0000 7f161ffff700 1 -- 172.21.15.40:0/372744819 >> v1:172.21.15.40:6789/0 conn(0x7f16200c7200 legacy=0x7f162015a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:09.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.210+0000 7f161ffff700 1 -- 172.21.15.40:0/372744819 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16201579d0 con 0x7f1620072350 2024-09-13T17:51:09.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.210+0000 7f1624c28700 1 --2- 172.21.15.40:0/372744819 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1620071f70 0x7f16200c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:51:09.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.214+0000 7f161ffff700 1 --2- 172.21.15.40:0/372744819 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1620072350 0x7f1620150d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1608009910 tx=0x7f1608009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc8ac93a0526382 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:09.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.214+0000 7f161f7fe700 1 -- 172.21.15.40:0/372744819 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160800ba60 con 0x7f1620072350 2024-09-13T17:51:09.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.214+0000 7f161f7fe700 1 -- 172.21.15.40:0/372744819 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160800bbc0 con 0x7f1620072350 2024-09-13T17:51:09.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.214+0000 7f161f7fe700 1 -- 172.21.15.40:0/372744819 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1608009e60 con 0x7f1620072350 2024-09-13T17:51:09.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.214+0000 7f1625c2a700 1 -- 172.21.15.40:0/372744819 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1620072350 msgr2=0x7f1620150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:09.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.214+0000 7f1625c2a700 1 --2- 172.21.15.40:0/372744819 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1620072350 0x7f1620150d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1608009910 tx=0x7f1608009c20 comp rx=0 tx=0).stop 2024-09-13T17:51:09.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.214+0000 7f1625c2a700 1 -- 172.21.15.40:0/372744819 shutdown_connections 2024-09-13T17:51:09.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.214+0000 7f1625c2a700 1 --2- 172.21.15.40:0/372744819 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1620072350 0x7f1620150d40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.214+0000 7f1625c2a700 1 --2- 172.21.15.40:0/372744819 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1620071f70 0x7f16200c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.214+0000 7f1625c2a700 1 -- 172.21.15.40:0/372744819 >> 172.21.15.40:0/372744819 conn(0x7f16200cabc0 msgr2=0x7f16200cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:09.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.214+0000 7f1625c2a700 1 -- 172.21.15.40:0/372744819 shutdown_connections 2024-09-13T17:51:09.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.214+0000 7f1625c2a700 1 -- 172.21.15.40:0/372744819 wait complete. 2024-09-13T17:51:09.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.214+0000 7f1625c2a700 1 Processor -- start 2024-09-13T17:51:09.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.214+0000 7f1625c2a700 1 -- start start 2024-09-13T17:51:09.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.214+0000 7f1625c2a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1620071f70 0x7f16201411a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:09.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.214+0000 7f1625c2a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1620072350 0x7f1620142f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:09.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.214+0000 7f1625c2a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16200c7200 0x7f1620143490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:09.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.214+0000 7f1625c2a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1620140df0 con 0x7f16200c7200 2024-09-13T17:51:09.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.214+0000 7f1625c2a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162013c050 con 0x7f1620071f70 2024-09-13T17:51:09.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.214+0000 7f1625c2a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16201eede0 con 0x7f1620072350 2024-09-13T17:51:09.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.218+0000 7f1625429700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16200c7200 0x7f1620143490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:09.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.218+0000 7f1625429700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16200c7200 0x7f1620143490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58268/0 (socket says 172.21.15.40:58268) 2024-09-13T17:51:09.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.218+0000 7f1625429700 1 -- 172.21.15.40:0/2698477365 learned_addr learned my addr 172.21.15.40:0/2698477365 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:09.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.218+0000 7f161ffff700 1 --2- 172.21.15.40:0/2698477365 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1620072350 0x7f1620142f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:09.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.218+0000 7f1624c28700 1 --2- 172.21.15.40:0/2698477365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1620071f70 0x7f16201411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:09.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.218+0000 7f1625429700 1 -- 172.21.15.40:0/2698477365 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1620072350 msgr2=0x7f1620142f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:09.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.218+0000 7f1625429700 1 --2- 172.21.15.40:0/2698477365 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1620072350 0x7f1620142f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.218+0000 7f1625429700 1 -- 172.21.15.40:0/2698477365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1620071f70 msgr2=0x7f16201411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:09.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.218+0000 7f1625429700 1 --2- 172.21.15.40:0/2698477365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1620071f70 0x7f16201411a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.218+0000 7f1625429700 1 -- 172.21.15.40:0/2698477365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1608009570 con 0x7f16200c7200 2024-09-13T17:51:09.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.218+0000 7f1625429700 1 --2- 172.21.15.40:0/2698477365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16200c7200 0x7f1620143490 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f161800ca70 tx=0x7f161800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:09.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.218+0000 7f161d7fa700 1 -- 172.21.15.40:0/2698477365 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1618003c20 con 0x7f16200c7200 2024-09-13T17:51:09.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.218+0000 7f161d7fa700 1 -- 172.21.15.40:0/2698477365 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1618003d80 con 0x7f16200c7200 2024-09-13T17:51:09.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.218+0000 7f1625c2a700 1 -- 172.21.15.40:0/2698477365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f162014ab00 con 0x7f16200c7200 2024-09-13T17:51:09.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.218+0000 7f161d7fa700 1 -- 172.21.15.40:0/2698477365 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161800aa20 con 0x7f16200c7200 2024-09-13T17:51:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.218+0000 7f1625c2a700 1 -- 172.21.15.40:0/2698477365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f162013faa0 con 0x7f16200c7200 2024-09-13T17:51:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.218+0000 7f1625c2a700 1 -- 172.21.15.40:0/2698477365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16201eef50 con 0x7f16200c7200 2024-09-13T17:51:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.218+0000 7f161d7fa700 1 -- 172.21.15.40:0/2698477365 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f161800ab80 con 0x7f16200c7200 2024-09-13T17:51:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.222+0000 7f161d7fa700 1 --2- 172.21.15.40:0/2698477365 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1604066730 0x7f1604068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.222+0000 7f161d7fa700 1 -- 172.21.15.40:0/2698477365 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1618084260 con 0x7f16200c7200 2024-09-13T17:51:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.222+0000 7f1624c28700 1 --2- 172.21.15.40:0/2698477365 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1604066730 0x7f1604068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.222+0000 7f1624c28700 1 --2- 172.21.15.40:0/2698477365 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1604066730 0x7f1604068bf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1610006fd0 tx=0x7f1610008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:09.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.226+0000 7f161d7fa700 1 -- 172.21.15.40:0/2698477365 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f16201eef50 con 0x7f16200c7200 2024-09-13T17:51:09.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.390+0000 7f1625c2a700 1 -- 172.21.15.40:0/2698477365 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16201eef50 con 0x7f16200c7200 2024-09-13T17:51:09.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.390+0000 7f161d7fa700 1 -- 172.21.15.40:0/2698477365 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18262 (secure 0 0 0) 0x7f16201eef50 con 0x7f16200c7200 2024-09-13T17:51:09.397 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:09.397 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:50:38.974071+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:50:32.740336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:50:33.552914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:50:36.623639+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:51:09.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.394+0000 7f1625c2a700 1 -- 172.21.15.40:0/2698477365 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1604066730 msgr2=0x7f1604068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:09.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.394+0000 7f1625c2a700 1 --2- 172.21.15.40:0/2698477365 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1604066730 0x7f1604068bf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1610006fd0 tx=0x7f1610008040 comp rx=0 tx=0).stop 2024-09-13T17:51:09.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.394+0000 7f1625c2a700 1 -- 172.21.15.40:0/2698477365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16200c7200 msgr2=0x7f1620143490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:09.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.394+0000 7f1625c2a700 1 --2- 172.21.15.40:0/2698477365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16200c7200 0x7f1620143490 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f161800ca70 tx=0x7f161800cd80 comp rx=0 tx=0).stop 2024-09-13T17:51:09.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.394+0000 7f1625c2a700 1 -- 172.21.15.40:0/2698477365 shutdown_connections 2024-09-13T17:51:09.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.394+0000 7f1625c2a700 1 --2- 172.21.15.40:0/2698477365 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16200c7200 0x7f1620143490 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.394+0000 7f1625c2a700 1 --2- 172.21.15.40:0/2698477365 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1604066730 0x7f1604068bf0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.394+0000 7f1625c2a700 1 --2- 172.21.15.40:0/2698477365 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1620072350 0x7f1620142f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.394+0000 7f1625c2a700 1 --2- 172.21.15.40:0/2698477365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1620071f70 0x7f16201411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.394+0000 7f1625c2a700 1 -- 172.21.15.40:0/2698477365 >> 172.21.15.40:0/2698477365 conn(0x7f16200cabc0 msgr2=0x7f16200c5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:09.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.394+0000 7f1625c2a700 1 -- 172.21.15.40:0/2698477365 shutdown_connections 2024-09-13T17:51:09.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.394+0000 7f1625c2a700 1 -- 172.21.15.40:0/2698477365 wait complete. 2024-09-13T17:51:09.411 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:50:21.057171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:50:32.740336+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:50:33.552914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:50:36.623639+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:51:09.411 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:51:09.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.530+0000 7f9e2ff14700 1 Processor -- start 2024-09-13T17:51:09.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.530+0000 7f9e2ff14700 1 -- start start 2024-09-13T17:51:09.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.530+0000 7f9e2ff14700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9e280cc620 0x7f9e28153520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:09.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.530+0000 7f9e2ff14700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e28159780 con 0x7f9e2815d620 2024-09-13T17:51:09.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.530+0000 7f9e2ff14700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e28159a40 con 0x7f9e280cc240 2024-09-13T17:51:09.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.530+0000 7f9e2ff14700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e28159d20 con 0x7f9e280cc620 2024-09-13T17:51:09.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.530+0000 7f9e2e4b1700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9e2815d620 0x7f9e280c3fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46212/0 (socket says 172.21.15.40:46212) 2024-09-13T17:51:09.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.530+0000 7f9e2e4b1700 1 -- 172.21.15.40:0/434813953 learned_addr learned my addr 172.21.15.40:0/434813953 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:09.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.530+0000 7f9e2d4af700 1 --2- 172.21.15.40:0/434813953 >> v2:172.21.15.43:3300/0 conn(0x7f9e280cc620 0x7f9e28153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:09.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.530+0000 7f9e2ccae700 1 -- 172.21.15.40:0/434813953 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 95725950 0 0) 0x7f9e28159780 con 0x7f9e2815d620 2024-09-13T17:51:09.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.530+0000 7f9e2ccae700 1 -- 172.21.15.40:0/434813953 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e140035d0 con 0x7f9e2815d620 2024-09-13T17:51:09.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.530+0000 7f9e2d4af700 1 -- 172.21.15.40:0/434813953 >> v1:172.21.15.43:6789/0 conn(0x7f9e280cc240 legacy=0x7f9e2815cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:09.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.530+0000 7f9e2d4af700 1 -- 172.21.15.40:0/434813953 >> v1:172.21.15.40:6789/0 conn(0x7f9e2815d620 legacy=0x7f9e280c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:09.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.530+0000 7f9e2d4af700 1 -- 172.21.15.40:0/434813953 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2815a000 con 0x7f9e280cc620 2024-09-13T17:51:09.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.530+0000 7f9e2d4af700 1 --2- 172.21.15.40:0/434813953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e280cc620 0x7f9e28153520 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009850 tx=0x7f9e1c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78c8aeda9b7e1ecc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:09.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.534+0000 7f9e2ccae700 1 -- 172.21.15.40:0/434813953 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c011830 con 0x7f9e280cc620 2024-09-13T17:51:09.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.534+0000 7f9e2ccae700 1 -- 172.21.15.40:0/434813953 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1c011990 con 0x7f9e280cc620 2024-09-13T17:51:09.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.534+0000 7f9e2ccae700 1 -- 172.21.15.40:0/434813953 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c011d40 con 0x7f9e280cc620 2024-09-13T17:51:09.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.534+0000 7f9e2ff14700 1 -- 172.21.15.40:0/434813953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e280cc620 msgr2=0x7f9e28153520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:09.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.534+0000 7f9e2ff14700 1 --2- 172.21.15.40:0/434813953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e280cc620 0x7f9e28153520 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009850 tx=0x7f9e1c009b60 comp rx=0 tx=0).stop 2024-09-13T17:51:09.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.534+0000 7f9e2ff14700 1 -- 172.21.15.40:0/434813953 shutdown_connections 2024-09-13T17:51:09.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.534+0000 7f9e2ff14700 1 --2- 172.21.15.40:0/434813953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e280cc620 0x7f9e28153520 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.534+0000 7f9e2ff14700 1 -- 172.21.15.40:0/434813953 >> 172.21.15.40:0/434813953 conn(0x7f9e28066480 msgr2=0x7f9e28066890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:09.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.534+0000 7f9e2ff14700 1 -- 172.21.15.40:0/434813953 shutdown_connections 2024-09-13T17:51:09.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.534+0000 7f9e2ff14700 1 -- 172.21.15.40:0/434813953 wait complete. 2024-09-13T17:51:09.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.534+0000 7f9e2ff14700 1 Processor -- start 2024-09-13T17:51:09.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.534+0000 7f9e2ff14700 1 -- start start 2024-09-13T17:51:09.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2ff14700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e280cc240 0x7f9e280b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:09.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2ff14700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9e280cc620 0x7f9e2815aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:09.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2ff14700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e2815d620 0x7f9e2815af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:09.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2ff14700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e280cfe70 con 0x7f9e280cc240 2024-09-13T17:51:09.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2ff14700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2815c250 con 0x7f9e2815d620 2024-09-13T17:51:09.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2ff14700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e28157720 con 0x7f9e280cc620 2024-09-13T17:51:09.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2dcb0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e280cc240 0x7f9e280b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:09.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2dcb0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e280cc240 0x7f9e280b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58280/0 (socket says 172.21.15.40:58280) 2024-09-13T17:51:09.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2dcb0700 1 -- 172.21.15.40:0/587195964 learned_addr learned my addr 172.21.15.40:0/587195964 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:09.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2e4b1700 1 --2- 172.21.15.40:0/587195964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e2815d620 0x7f9e2815af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:09.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2d4af700 1 --2- 172.21.15.40:0/587195964 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9e280cc620 0x7f9e2815aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:09.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2dcb0700 1 -- 172.21.15.40:0/587195964 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9e280cc620 msgr2=0x7f9e2815aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:09.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2dcb0700 1 --2- 172.21.15.40:0/587195964 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9e280cc620 0x7f9e2815aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2dcb0700 1 -- 172.21.15.40:0/587195964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e2815d620 msgr2=0x7f9e2815af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:09.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2dcb0700 1 --2- 172.21.15.40:0/587195964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e2815d620 0x7f9e2815af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2dcb0700 1 -- 172.21.15.40:0/587195964 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1c0094b0 con 0x7f9e280cc240 2024-09-13T17:51:09.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2d4af700 1 --2- 172.21.15.40:0/587195964 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9e280cc620 0x7f9e2815aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:09.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2dcb0700 1 --2- 172.21.15.40:0/587195964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e280cc240 0x7f9e280b4b50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400c500 tx=0x7f9e2400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:09.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e1a7fc700 1 -- 172.21.15.40:0/587195964 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e240052d0 con 0x7f9e280cc240 2024-09-13T17:51:09.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e1a7fc700 1 -- 172.21.15.40:0/587195964 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e24005430 con 0x7f9e280cc240 2024-09-13T17:51:09.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2ff14700 1 -- 172.21.15.40:0/587195964 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e281579c0 con 0x7f9e280cc240 2024-09-13T17:51:09.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e1a7fc700 1 -- 172.21.15.40:0/587195964 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e240055a0 con 0x7f9e280cc240 2024-09-13T17:51:09.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2ff14700 1 -- 172.21.15.40:0/587195964 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e281efb40 con 0x7f9e280cc240 2024-09-13T17:51:09.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.538+0000 7f9e2ff14700 1 -- 172.21.15.40:0/587195964 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e280c4a70 con 0x7f9e280cc240 2024-09-13T17:51:09.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.542+0000 7f9e1a7fc700 1 -- 172.21.15.40:0/587195964 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9e240038e0 con 0x7f9e280cc240 2024-09-13T17:51:09.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.542+0000 7f9e1a7fc700 1 --2- 172.21.15.40:0/587195964 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e0c066730 0x7f9e0c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:09.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.542+0000 7f9e2d4af700 1 --2- 172.21.15.40:0/587195964 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e0c066730 0x7f9e0c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:09.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.542+0000 7f9e1a7fc700 1 -- 172.21.15.40:0/587195964 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9e24083d30 con 0x7f9e280cc240 2024-09-13T17:51:09.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.542+0000 7f9e2d4af700 1 --2- 172.21.15.40:0/587195964 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e0c066730 0x7f9e0c068bf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0090b0 tx=0x7f9e1c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:09.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.546+0000 7f9e1a7fc700 1 -- 172.21.15.40:0/587195964 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9e24050e20 con 0x7f9e280cc240 2024-09-13T17:51:09.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.714+0000 7f9e2ff14700 1 -- 172.21.15.40:0/587195964 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9e2814fcc0 con 0x7f9e280cc240 2024-09-13T17:51:09.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.714+0000 7f9e1a7fc700 1 -- 172.21.15.40:0/587195964 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9e240170a0 con 0x7f9e280cc240 2024-09-13T17:51:09.719 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-09-13T17:51:09.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.714+0000 7f9e2ff14700 1 -- 172.21.15.40:0/587195964 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e0c066730 msgr2=0x7f9e0c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:09.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.714+0000 7f9e2ff14700 1 --2- 172.21.15.40:0/587195964 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e0c066730 0x7f9e0c068bf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0090b0 tx=0x7f9e1c008030 comp rx=0 tx=0).stop 2024-09-13T17:51:09.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.714+0000 7f9e2ff14700 1 -- 172.21.15.40:0/587195964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e280cc240 msgr2=0x7f9e280b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:09.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.714+0000 7f9e2ff14700 1 --2- 172.21.15.40:0/587195964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e280cc240 0x7f9e280b4b50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400c500 tx=0x7f9e2400c8c0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.714+0000 7f9e2ff14700 1 -- 172.21.15.40:0/587195964 shutdown_connections 2024-09-13T17:51:09.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.714+0000 7f9e2ff14700 1 --2- 172.21.15.40:0/587195964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e2815d620 0x7f9e2815af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.714+0000 7f9e2ff14700 1 --2- 172.21.15.40:0/587195964 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e0c066730 0x7f9e0c068bf0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.714+0000 7f9e2ff14700 1 --2- 172.21.15.40:0/587195964 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9e280cc620 0x7f9e2815aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.714+0000 7f9e2ff14700 1 --2- 172.21.15.40:0/587195964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e280cc240 0x7f9e280b4b50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.714+0000 7f9e2ff14700 1 -- 172.21.15.40:0/587195964 >> 172.21.15.40:0/587195964 conn(0x7f9e28066480 msgr2=0x7f9e281501e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:09.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.718+0000 7f9e2ff14700 1 -- 172.21.15.40:0/587195964 shutdown_connections 2024-09-13T17:51:09.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.718+0000 7f9e2ff14700 1 -- 172.21.15.40:0/587195964 wait complete. 2024-09-13T17:51:09.733 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:51:09.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.846+0000 7f63b3fff700 1 Processor -- start 2024-09-13T17:51:09.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.846+0000 7f63b3fff700 1 -- start start 2024-09-13T17:51:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.846+0000 7f63b3fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63b40cf1a0 0x7f63b415a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.846+0000 7f63b3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63b40cf580 0x7f63b40c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.846+0000 7f63b3fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b415a7a0 con 0x7f63b40cc250 2024-09-13T17:51:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.846+0000 7f63b3fff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63b40cbdc0 con 0x7f63b40cf1a0 2024-09-13T17:51:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.846+0000 7f63b3fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b4153f80 con 0x7f63b40cf580 2024-09-13T17:51:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.846+0000 7f63b37fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f63b40cc250 0x7f63b40c8aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46216/0 (socket says 172.21.15.40:46216) 2024-09-13T17:51:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.846+0000 7f63b37fe700 1 -- 172.21.15.40:0/829973989 learned_addr learned my addr 172.21.15.40:0/829973989 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.846+0000 7f63b2ffd700 1 --2- 172.21.15.40:0/829973989 >> v2:172.21.15.40:3300/0 conn(0x7f63b40cf1a0 0x7f63b415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.846+0000 7f63b27fc700 1 --2- 172.21.15.40:0/829973989 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63b40cf580 0x7f63b40c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.850+0000 7f63b1ffb700 1 -- 172.21.15.40:0/829973989 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 665599723 0 0) 0x7f63ac001490 con 0x7f63b40cc250 2024-09-13T17:51:09.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.850+0000 7f63b1ffb700 1 -- 172.21.15.40:0/829973989 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f639c0035d0 con 0x7f63b40cc250 2024-09-13T17:51:09.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.850+0000 7f63b2ffd700 1 -- 172.21.15.40:0/829973989 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63b40cf580 msgr2=0x7f63b40c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:09.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.850+0000 7f63b2ffd700 1 --2- 172.21.15.40:0/829973989 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63b40cf580 0x7f63b40c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.850+0000 7f63b2ffd700 1 -- 172.21.15.40:0/829973989 >> v1:172.21.15.40:6789/0 conn(0x7f63b40cc250 legacy=0x7f63b40c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:09.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.850+0000 7f63b2ffd700 1 -- 172.21.15.40:0/829973989 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b40caed0 con 0x7f63b40cf1a0 2024-09-13T17:51:09.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.850+0000 7f63b2ffd700 1 --2- 172.21.15.40:0/829973989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b40cf1a0 0x7f63b415a260 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009850 tx=0x7f63a8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b27e89e45d357 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:09.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.850+0000 7f63b1ffb700 1 -- 172.21.15.40:0/829973989 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8011830 con 0x7f63b40cf1a0 2024-09-13T17:51:09.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.850+0000 7f63b1ffb700 1 -- 172.21.15.40:0/829973989 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a8011990 con 0x7f63b40cf1a0 2024-09-13T17:51:09.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.850+0000 7f63b1ffb700 1 -- 172.21.15.40:0/829973989 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8011d00 con 0x7f63b40cf1a0 2024-09-13T17:51:09.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.850+0000 7f63b3fff700 1 -- 172.21.15.40:0/829973989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b40cf1a0 msgr2=0x7f63b415a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:09.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.850+0000 7f63b3fff700 1 --2- 172.21.15.40:0/829973989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b40cf1a0 0x7f63b415a260 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009850 tx=0x7f63a8009b60 comp rx=0 tx=0).stop 2024-09-13T17:51:09.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.850+0000 7f63b3fff700 1 -- 172.21.15.40:0/829973989 shutdown_connections 2024-09-13T17:51:09.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.850+0000 7f63b3fff700 1 --2- 172.21.15.40:0/829973989 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63b40cf580 0x7f63b40c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.850+0000 7f63b3fff700 1 --2- 172.21.15.40:0/829973989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b40cf1a0 0x7f63b415a260 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.850+0000 7f63b3fff700 1 -- 172.21.15.40:0/829973989 >> 172.21.15.40:0/829973989 conn(0x7f63b4066090 msgr2=0x7f63b4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:09.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.850+0000 7f63b3fff700 1 -- 172.21.15.40:0/829973989 shutdown_connections 2024-09-13T17:51:09.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.850+0000 7f63b3fff700 1 -- 172.21.15.40:0/829973989 wait complete. 2024-09-13T17:51:09.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.850+0000 7f63b3fff700 1 Processor -- start 2024-09-13T17:51:09.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.850+0000 7f63b3fff700 1 -- start start 2024-09-13T17:51:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b40cc250 0x7f63b40abb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b3fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b40cf1a0 0x7f63b4157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63b40cf580 0x7f63b40aa590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b3fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b41578e0 con 0x7f63b40cc250 2024-09-13T17:51:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b3fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b41582b0 con 0x7f63b40cf1a0 2024-09-13T17:51:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b3fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b4158b00 con 0x7f63b40cf580 2024-09-13T17:51:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b40cc250 0x7f63b40abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b40cc250 0x7f63b40abb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58288/0 (socket says 172.21.15.40:58288) 2024-09-13T17:51:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b2ffd700 1 -- 172.21.15.40:0/3947525983 learned_addr learned my addr 172.21.15.40:0/3947525983 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b37fe700 1 --2- 172.21.15.40:0/3947525983 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63b40cf580 0x7f63b40aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b27fc700 1 --2- 172.21.15.40:0/3947525983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b40cf1a0 0x7f63b4157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b2ffd700 1 -- 172.21.15.40:0/3947525983 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63b40cf580 msgr2=0x7f63b40aa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:09.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b2ffd700 1 --2- 172.21.15.40:0/3947525983 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63b40cf580 0x7f63b40aa590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b2ffd700 1 -- 172.21.15.40:0/3947525983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b40cf1a0 msgr2=0x7f63b4157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:09.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b2ffd700 1 --2- 172.21.15.40:0/3947525983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b40cf1a0 0x7f63b4157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:09.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b2ffd700 1 -- 172.21.15.40:0/3947525983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a80094b0 con 0x7f63b40cc250 2024-09-13T17:51:09.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b37fe700 1 --2- 172.21.15.40:0/3947525983 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63b40cf580 0x7f63b40aa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:51:09.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b27fc700 1 --2- 172.21.15.40:0/3947525983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b40cf1a0 0x7f63b4157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:51:09.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b2ffd700 1 --2- 172.21.15.40:0/3947525983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b40cc250 0x7f63b40abb30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f63a80090b0 tx=0x7f63a8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:09.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f639b7fe700 1 -- 172.21.15.40:0/3947525983 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a80045d0 con 0x7f63b40cc250 2024-09-13T17:51:09.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f639b7fe700 1 -- 172.21.15.40:0/3947525983 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a801f070 con 0x7f63b40cc250 2024-09-13T17:51:09.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b3fff700 1 -- 172.21.15.40:0/3947525983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b4158de0 con 0x7f63b40cc250 2024-09-13T17:51:09.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f639b7fe700 1 -- 172.21.15.40:0/3947525983 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a80038e0 con 0x7f63b40cc250 2024-09-13T17:51:09.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.854+0000 7f63b3fff700 1 -- 172.21.15.40:0/3947525983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b41593a0 con 0x7f63b40cc250 2024-09-13T17:51:09.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.858+0000 7f63b3fff700 1 -- 172.21.15.40:0/3947525983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b4067a40 con 0x7f63b40cc250 2024-09-13T17:51:09.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.858+0000 7f639b7fe700 1 -- 172.21.15.40:0/3947525983 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f63a8003a40 con 0x7f63b40cc250 2024-09-13T17:51:09.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.858+0000 7f639b7fe700 1 --2- 172.21.15.40:0/3947525983 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f63a0066730 0x7f63a0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:09.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.858+0000 7f639b7fe700 1 -- 172.21.15.40:0/3947525983 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f63a8021070 con 0x7f63b40cc250 2024-09-13T17:51:09.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.858+0000 7f63b27fc700 1 --2- 172.21.15.40:0/3947525983 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f63a0066730 0x7f63a0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:09.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.858+0000 7f63b27fc700 1 --2- 172.21.15.40:0/3947525983 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f63a0066730 0x7f63a0068bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f63b40c82a0 tx=0x7f63a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:09.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:09.866+0000 7f639b7fe700 1 -- 172.21.15.40:0/3947525983 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f63a8052e10 con 0x7f63b40cc250 2024-09-13T17:51:10.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.034+0000 7f63b3fff700 1 -- 172.21.15.40:0/3947525983 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f63b41576b0 con 0x7f63b40cc250 2024-09-13T17:51:10.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.038+0000 7f639b7fe700 1 -- 172.21.15.40:0/3947525983 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f63a8015020 con 0x7f63b40cc250 2024-09-13T17:51:10.042 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-09-13T17:51:10.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.038+0000 7f63b3fff700 1 -- 172.21.15.40:0/3947525983 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f63a0066730 msgr2=0x7f63a0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:10.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.038+0000 7f63b3fff700 1 --2- 172.21.15.40:0/3947525983 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f63a0066730 0x7f63a0068bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f63b40c82a0 tx=0x7f63a4008040 comp rx=0 tx=0).stop 2024-09-13T17:51:10.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.038+0000 7f63b3fff700 1 -- 172.21.15.40:0/3947525983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b40cc250 msgr2=0x7f63b40abb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:10.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.038+0000 7f63b3fff700 1 --2- 172.21.15.40:0/3947525983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b40cc250 0x7f63b40abb30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f63a80090b0 tx=0x7f63a8004770 comp rx=0 tx=0).stop 2024-09-13T17:51:10.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.038+0000 7f63b3fff700 1 -- 172.21.15.40:0/3947525983 shutdown_connections 2024-09-13T17:51:10.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.038+0000 7f63b3fff700 1 --2- 172.21.15.40:0/3947525983 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f63a0066730 0x7f63a0068bf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.038+0000 7f63b3fff700 1 --2- 172.21.15.40:0/3947525983 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63b40cf580 0x7f63b40aa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.038+0000 7f63b3fff700 1 --2- 172.21.15.40:0/3947525983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63b40cf1a0 0x7f63b4157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.038+0000 7f63b3fff700 1 --2- 172.21.15.40:0/3947525983 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63b40cc250 0x7f63b40abb30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.038+0000 7f63b3fff700 1 -- 172.21.15.40:0/3947525983 >> 172.21.15.40:0/3947525983 conn(0x7f63b4066090 msgr2=0x7f63b40cd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:10.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.038+0000 7f63b3fff700 1 -- 172.21.15.40:0/3947525983 shutdown_connections 2024-09-13T17:51:10.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.038+0000 7f63b3fff700 1 -- 172.21.15.40:0/3947525983 wait complete. 2024-09-13T17:51:10.058 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:51:10.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.174+0000 7fda8cdad700 1 Processor -- start 2024-09-13T17:51:10.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.174+0000 7fda8cdad700 1 -- start start 2024-09-13T17:51:10.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.174+0000 7fda8cdad700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fda88072510 0x7fda88070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:10.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.174+0000 7fda8cdad700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda880c9fd0 0x7fda880ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:10.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.174+0000 7fda8cdad700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda88071880 con 0x7fda8814ff00 2024-09-13T17:51:10.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.174+0000 7fda8cdad700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda880667e0 con 0x7fda88072510 2024-09-13T17:51:10.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.174+0000 7fda8cdad700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda880cbfe0 con 0x7fda880c9fd0 2024-09-13T17:51:10.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.174+0000 7fda877fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fda8814ff00 0x7fda881594d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46224/0 (socket says 172.21.15.40:46224) 2024-09-13T17:51:10.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.174+0000 7fda86ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fda88072510 0x7fda88070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:10.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.174+0000 7fda877fe700 1 -- 172.21.15.40:0/1484680147 learned_addr learned my addr 172.21.15.40:0/1484680147 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:10.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.174+0000 7fda87fff700 1 --2- 172.21.15.40:0/1484680147 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda880c9fd0 0x7fda880ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:10.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.178+0000 7fda86ffd700 1 -- 172.21.15.40:0/1484680147 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda880c9fd0 msgr2=0x7fda880ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:10.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.178+0000 7fda86ffd700 1 --2- 172.21.15.40:0/1484680147 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda880c9fd0 0x7fda880ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.178+0000 7fda86ffd700 1 -- 172.21.15.40:0/1484680147 >> v1:172.21.15.40:6789/0 conn(0x7fda8814ff00 legacy=0x7fda881594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:10.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.178+0000 7fda86ffd700 1 -- 172.21.15.40:0/1484680147 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda880712c0 con 0x7fda88072510 2024-09-13T17:51:10.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.178+0000 7fda867fc700 1 -- 172.21.15.40:0/1484680147 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759279675 0 0) 0x7fda88071880 con 0x7fda8814ff00 2024-09-13T17:51:10.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.178+0000 7fda86ffd700 1 --2- 172.21.15.40:0/1484680147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda88072510 0x7fda88070c10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fda70009850 tx=0x7fda70009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9567604d1667e57 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:10.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.178+0000 7fda867fc700 1 -- 172.21.15.40:0/1484680147 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70011830 con 0x7fda88072510 2024-09-13T17:51:10.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.178+0000 7fda867fc700 1 -- 172.21.15.40:0/1484680147 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda70011990 con 0x7fda88072510 2024-09-13T17:51:10.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.178+0000 7fda867fc700 1 -- 172.21.15.40:0/1484680147 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70011d40 con 0x7fda88072510 2024-09-13T17:51:10.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.178+0000 7fda8cdad700 1 -- 172.21.15.40:0/1484680147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda88072510 msgr2=0x7fda88070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:10.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.178+0000 7fda8cdad700 1 --2- 172.21.15.40:0/1484680147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda88072510 0x7fda88070c10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fda70009850 tx=0x7fda70009b60 comp rx=0 tx=0).stop 2024-09-13T17:51:10.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.178+0000 7fda8cdad700 1 -- 172.21.15.40:0/1484680147 shutdown_connections 2024-09-13T17:51:10.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.178+0000 7fda8cdad700 1 --2- 172.21.15.40:0/1484680147 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda880c9fd0 0x7fda880ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.178+0000 7fda8cdad700 1 --2- 172.21.15.40:0/1484680147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda88072510 0x7fda88070c10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.178+0000 7fda8cdad700 1 -- 172.21.15.40:0/1484680147 >> 172.21.15.40:0/1484680147 conn(0x7fda8806a390 msgr2=0x7fda8806a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:10.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.178+0000 7fda8cdad700 1 -- 172.21.15.40:0/1484680147 shutdown_connections 2024-09-13T17:51:10.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.178+0000 7fda8cdad700 1 -- 172.21.15.40:0/1484680147 wait complete. 2024-09-13T17:51:10.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.178+0000 7fda8cdad700 1 Processor -- start 2024-09-13T17:51:10.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.178+0000 7fda8cdad700 1 -- start start 2024-09-13T17:51:10.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda8cdad700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda88072510 0x7fda88150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda8cdad700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda880c9fd0 0x7fda8815cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda8cdad700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda8814ff00 0x7fda881e2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda8cdad700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda88151440 con 0x7fda880c9fd0 2024-09-13T17:51:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda8cdad700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda881f2ce0 con 0x7fda88072510 2024-09-13T17:51:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda8cdad700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda881e4e80 con 0x7fda8814ff00 2024-09-13T17:51:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda86ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda880c9fd0 0x7fda8815cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda86ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda880c9fd0 0x7fda8815cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58296/0 (socket says 172.21.15.40:58296) 2024-09-13T17:51:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda86ffd700 1 -- 172.21.15.40:0/1066063418 learned_addr learned my addr 172.21.15.40:0/1066063418 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda87fff700 1 --2- 172.21.15.40:0/1066063418 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda8814ff00 0x7fda881e2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda877fe700 1 --2- 172.21.15.40:0/1066063418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda88072510 0x7fda88150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda87fff700 1 -- 172.21.15.40:0/1066063418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda88072510 msgr2=0x7fda88150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda87fff700 1 --2- 172.21.15.40:0/1066063418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda88072510 0x7fda88150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda87fff700 1 -- 172.21.15.40:0/1066063418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda880c9fd0 msgr2=0x7fda8815cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:10.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda87fff700 1 --2- 172.21.15.40:0/1066063418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda880c9fd0 0x7fda8815cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda87fff700 1 -- 172.21.15.40:0/1066063418 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda700094b0 con 0x7fda8814ff00 2024-09-13T17:51:10.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda86ffd700 1 --2- 172.21.15.40:0/1066063418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda880c9fd0 0x7fda8815cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:10.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda87fff700 1 --2- 172.21.15.40:0/1066063418 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda8814ff00 0x7fda881e2890 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fda7800e860 tx=0x7fda7800eb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:10.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda7ffff700 1 -- 172.21.15.40:0/1066063418 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda78013680 con 0x7fda8814ff00 2024-09-13T17:51:10.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda8cdad700 1 -- 172.21.15.40:0/1066063418 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda880caed0 con 0x7fda8814ff00 2024-09-13T17:51:10.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda8cdad700 1 -- 172.21.15.40:0/1066063418 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda881f20d0 con 0x7fda8814ff00 2024-09-13T17:51:10.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda7ffff700 1 -- 172.21.15.40:0/1066063418 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda780137e0 con 0x7fda8814ff00 2024-09-13T17:51:10.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.182+0000 7fda7ffff700 1 -- 172.21.15.40:0/1066063418 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda78013b50 con 0x7fda8814ff00 2024-09-13T17:51:10.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.186+0000 7fda8cdad700 1 -- 172.21.15.40:0/1066063418 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda881f2e90 con 0x7fda8814ff00 2024-09-13T17:51:10.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.186+0000 7fda7ffff700 1 -- 172.21.15.40:0/1066063418 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fda78013cb0 con 0x7fda8814ff00 2024-09-13T17:51:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.186+0000 7fda7ffff700 1 --2- 172.21.15.40:0/1066063418 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fda6c066760 0x7fda6c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.186+0000 7fda7ffff700 1 -- 172.21.15.40:0/1066063418 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fda78084d10 con 0x7fda8814ff00 2024-09-13T17:51:10.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.186+0000 7fda877fe700 1 --2- 172.21.15.40:0/1066063418 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fda6c066760 0x7fda6c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:10.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.190+0000 7fda877fe700 1 --2- 172.21.15.40:0/1066063418 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fda6c066760 0x7fda6c068c20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fda800012e0 tx=0x7fda80001b70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:10.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.194+0000 7fda7ffff700 1 -- 172.21.15.40:0/1066063418 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fda78055700 con 0x7fda8814ff00 2024-09-13T17:51:10.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.366+0000 7fda8cdad700 1 -- 172.21.15.40:0/1066063418 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fda881502e0 con 0x7fda8814ff00 2024-09-13T17:51:10.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.366+0000 7fda7ffff700 1 -- 172.21.15.40:0/1066063418 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fda78055290 con 0x7fda8814ff00 2024-09-13T17:51:10.372 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-09-13T17:51:10.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.370+0000 7fda8cdad700 1 -- 172.21.15.40:0/1066063418 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fda6c066760 msgr2=0x7fda6c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:10.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.370+0000 7fda8cdad700 1 --2- 172.21.15.40:0/1066063418 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fda6c066760 0x7fda6c068c20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fda800012e0 tx=0x7fda80001b70 comp rx=0 tx=0).stop 2024-09-13T17:51:10.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.370+0000 7fda8cdad700 1 -- 172.21.15.40:0/1066063418 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda8814ff00 msgr2=0x7fda881e2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:10.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.370+0000 7fda8cdad700 1 --2- 172.21.15.40:0/1066063418 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda8814ff00 0x7fda881e2890 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fda7800e860 tx=0x7fda7800eb70 comp rx=0 tx=0).stop 2024-09-13T17:51:10.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.370+0000 7fda8cdad700 1 -- 172.21.15.40:0/1066063418 shutdown_connections 2024-09-13T17:51:10.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.370+0000 7fda8cdad700 1 --2- 172.21.15.40:0/1066063418 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fda6c066760 0x7fda6c068c20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.370+0000 7fda8cdad700 1 --2- 172.21.15.40:0/1066063418 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda8814ff00 0x7fda881e2890 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.370+0000 7fda8cdad700 1 --2- 172.21.15.40:0/1066063418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda880c9fd0 0x7fda8815cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.370+0000 7fda8cdad700 1 --2- 172.21.15.40:0/1066063418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda88072510 0x7fda88150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.370+0000 7fda8cdad700 1 -- 172.21.15.40:0/1066063418 >> 172.21.15.40:0/1066063418 conn(0x7fda8806a390 msgr2=0x7fda880c7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:10.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.370+0000 7fda8cdad700 1 -- 172.21.15.40:0/1066063418 shutdown_connections 2024-09-13T17:51:10.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.370+0000 7fda8cdad700 1 -- 172.21.15.40:0/1066063418 wait complete. 2024-09-13T17:51:10.387 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:51:10.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.502+0000 7ff610e1b700 1 Processor -- start 2024-09-13T17:51:10.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.502+0000 7ff610e1b700 1 -- start start 2024-09-13T17:51:10.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff610e1b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff60c0cf580 0x7ff60c15a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:10.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff610e1b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff60c0cc250 0x7ff60c0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:10.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff610e1b700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60c15a7a0 con 0x7ff60c0cf1a0 2024-09-13T17:51:10.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff610e1b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff60c0cbdc0 con 0x7ff60c0cf580 2024-09-13T17:51:10.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff610e1b700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff60c153f80 con 0x7ff60c0cc250 2024-09-13T17:51:10.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff602ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff60c0cf580 0x7ff60c15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:10.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff60b7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff60c0cf1a0 0x7ff60c0c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46238/0 (socket says 172.21.15.40:46238) 2024-09-13T17:51:10.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff60b7fe700 1 -- 172.21.15.40:0/2712817324 learned_addr learned my addr 172.21.15.40:0/2712817324 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:10.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff60bfff700 1 --2- 172.21.15.40:0/2712817324 >> v2:172.21.15.43:3300/0 conn(0x7ff60c0cc250 0x7ff60c0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:10.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff602ffd700 1 -- 172.21.15.40:0/2712817324 >> v2:172.21.15.43:3300/0 conn(0x7ff60c0cc250 msgr2=0x7ff60c0c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:10.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff602ffd700 1 --2- 172.21.15.40:0/2712817324 >> v2:172.21.15.43:3300/0 conn(0x7ff60c0cc250 0x7ff60c0c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff602ffd700 1 -- 172.21.15.40:0/2712817324 >> v1:172.21.15.40:6789/0 conn(0x7ff60c0cf1a0 legacy=0x7ff60c0c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:10.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff602ffd700 1 -- 172.21.15.40:0/2712817324 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c0caed0 con 0x7ff60c0cf580 2024-09-13T17:51:10.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff60affd700 1 -- 172.21.15.40:0/2712817324 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605035452 0 0) 0x7ff5fc001110 con 0x7ff60c0cf1a0 2024-09-13T17:51:10.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff602ffd700 1 --2- 172.21.15.40:0/2712817324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c0cf580 0x7ff60c15a260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff60c074130 tx=0x7ff5f4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6a6633b50f9ac9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:10.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff60affd700 1 -- 172.21.15.40:0/2712817324 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4011830 con 0x7ff60c0cf580 2024-09-13T17:51:10.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff60affd700 1 -- 172.21.15.40:0/2712817324 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f4011990 con 0x7ff60c0cf580 2024-09-13T17:51:10.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff60affd700 1 -- 172.21.15.40:0/2712817324 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4011d20 con 0x7ff60c0cf580 2024-09-13T17:51:10.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff610e1b700 1 -- 172.21.15.40:0/2712817324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c0cf580 msgr2=0x7ff60c15a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:10.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff610e1b700 1 --2- 172.21.15.40:0/2712817324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c0cf580 0x7ff60c15a260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff60c074130 tx=0x7ff5f4009b30 comp rx=0 tx=0).stop 2024-09-13T17:51:10.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff610e1b700 1 -- 172.21.15.40:0/2712817324 shutdown_connections 2024-09-13T17:51:10.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff610e1b700 1 --2- 172.21.15.40:0/2712817324 >> v2:172.21.15.43:3300/0 conn(0x7ff60c0cc250 0x7ff60c0c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff610e1b700 1 --2- 172.21.15.40:0/2712817324 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c0cf580 0x7ff60c15a260 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff610e1b700 1 -- 172.21.15.40:0/2712817324 >> 172.21.15.40:0/2712817324 conn(0x7ff60c066090 msgr2=0x7ff60c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:10.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff610e1b700 1 -- 172.21.15.40:0/2712817324 shutdown_connections 2024-09-13T17:51:10.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.506+0000 7ff610e1b700 1 -- 172.21.15.40:0/2712817324 wait complete. 2024-09-13T17:51:10.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.510+0000 7ff610e1b700 1 Processor -- start 2024-09-13T17:51:10.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.510+0000 7ff610e1b700 1 -- start start 2024-09-13T17:51:10.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.510+0000 7ff610e1b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff60c0cc250 0x7ff60c1f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:10.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.510+0000 7ff610e1b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff60c0cf1a0 0x7ff60c157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:10.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.510+0000 7ff610e1b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c0cf580 0x7ff60c0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:10.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.510+0000 7ff610e1b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c0c4310 con 0x7ff60c0cf580 2024-09-13T17:51:10.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.510+0000 7ff610e1b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c157440 con 0x7ff60c0cc250 2024-09-13T17:51:10.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.510+0000 7ff610e1b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c1f2ff0 con 0x7ff60c0cf1a0 2024-09-13T17:51:10.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.510+0000 7ff60bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c0cf580 0x7ff60c0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:10.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.510+0000 7ff60bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c0cf580 0x7ff60c0b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58310/0 (socket says 172.21.15.40:58310) 2024-09-13T17:51:10.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.510+0000 7ff60bfff700 1 -- 172.21.15.40:0/686861971 learned_addr learned my addr 172.21.15.40:0/686861971 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:10.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.510+0000 7ff60b7fe700 1 --2- 172.21.15.40:0/686861971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff60c0cc250 0x7ff60c1f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:10.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.510+0000 7ff602ffd700 1 --2- 172.21.15.40:0/686861971 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff60c0cf1a0 0x7ff60c157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:10.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.514+0000 7ff60bfff700 1 -- 172.21.15.40:0/686861971 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff60c0cf1a0 msgr2=0x7ff60c157e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:10.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.514+0000 7ff60bfff700 1 --2- 172.21.15.40:0/686861971 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff60c0cf1a0 0x7ff60c157e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.514+0000 7ff60bfff700 1 -- 172.21.15.40:0/686861971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff60c0cc250 msgr2=0x7ff60c1f2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:10.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.514+0000 7ff60bfff700 1 --2- 172.21.15.40:0/686861971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff60c0cc250 0x7ff60c1f2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.514+0000 7ff60bfff700 1 -- 172.21.15.40:0/686861971 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f40094b0 con 0x7ff60c0cf580 2024-09-13T17:51:10.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.514+0000 7ff602ffd700 1 --2- 172.21.15.40:0/686861971 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff60c0cf1a0 0x7ff60c157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:10.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.514+0000 7ff60bfff700 1 --2- 172.21.15.40:0/686861971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c0cf580 0x7ff60c0b1d50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff60400efc0 tx=0x7ff60400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:10.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.514+0000 7ff608ff9700 1 -- 172.21.15.40:0/686861971 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604009ea0 con 0x7ff60c0cf580 2024-09-13T17:51:10.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.514+0000 7ff608ff9700 1 -- 172.21.15.40:0/686861971 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60400f040 con 0x7ff60c0cf580 2024-09-13T17:51:10.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.514+0000 7ff610e1b700 1 -- 172.21.15.40:0/686861971 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60c0c5470 con 0x7ff60c0cf580 2024-09-13T17:51:10.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.514+0000 7ff608ff9700 1 -- 172.21.15.40:0/686861971 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60400ced0 con 0x7ff60c0cf580 2024-09-13T17:51:10.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.514+0000 7ff610e1b700 1 -- 172.21.15.40:0/686861971 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60c0c5a10 con 0x7ff60c0cf580 2024-09-13T17:51:10.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.514+0000 7ff610e1b700 1 -- 172.21.15.40:0/686861971 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff60c067a40 con 0x7ff60c0cf580 2024-09-13T17:51:10.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.514+0000 7ff608ff9700 1 -- 172.21.15.40:0/686861971 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff604014480 con 0x7ff60c0cf580 2024-09-13T17:51:10.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.518+0000 7ff608ff9700 1 --2- 172.21.15.40:0/686861971 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff5ec066730 0x7ff5ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:10.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.518+0000 7ff608ff9700 1 -- 172.21.15.40:0/686861971 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff604084f10 con 0x7ff60c0cf580 2024-09-13T17:51:10.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.518+0000 7ff60b7fe700 1 --2- 172.21.15.40:0/686861971 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff5ec066730 0x7ff5ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:10.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.518+0000 7ff60b7fe700 1 --2- 172.21.15.40:0/686861971 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff5ec066730 0x7ff5ec068bf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc000f60 tx=0x7ff5fc007c40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:10.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.526+0000 7ff608ff9700 1 -- 172.21.15.40:0/686861971 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff604052040 con 0x7ff60c0cf580 2024-09-13T17:51:10.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.698+0000 7ff610e1b700 1 -- 172.21.15.40:0/686861971 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff60c15ac50 con 0x7ff60c0cf580 2024-09-13T17:51:10.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.698+0000 7ff608ff9700 1 -- 172.21.15.40:0/686861971 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff604018070 con 0x7ff60c0cf580 2024-09-13T17:51:10.704 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-09-13T17:51:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.702+0000 7ff610e1b700 1 -- 172.21.15.40:0/686861971 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff5ec066730 msgr2=0x7ff5ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.702+0000 7ff610e1b700 1 --2- 172.21.15.40:0/686861971 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff5ec066730 0x7ff5ec068bf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc000f60 tx=0x7ff5fc007c40 comp rx=0 tx=0).stop 2024-09-13T17:51:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.702+0000 7ff610e1b700 1 -- 172.21.15.40:0/686861971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c0cf580 msgr2=0x7ff60c0b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.702+0000 7ff610e1b700 1 --2- 172.21.15.40:0/686861971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c0cf580 0x7ff60c0b1d50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff60400efc0 tx=0x7ff60400c3a0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.702+0000 7ff610e1b700 1 -- 172.21.15.40:0/686861971 shutdown_connections 2024-09-13T17:51:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.702+0000 7ff610e1b700 1 --2- 172.21.15.40:0/686861971 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c0cf580 0x7ff60c0b1d50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.702+0000 7ff610e1b700 1 --2- 172.21.15.40:0/686861971 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff5ec066730 0x7ff5ec068bf0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.702+0000 7ff610e1b700 1 --2- 172.21.15.40:0/686861971 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff60c0cf1a0 0x7ff60c157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.702+0000 7ff610e1b700 1 --2- 172.21.15.40:0/686861971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff60c0cc250 0x7ff60c1f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.702+0000 7ff610e1b700 1 -- 172.21.15.40:0/686861971 >> 172.21.15.40:0/686861971 conn(0x7ff60c066090 msgr2=0x7ff60c0c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:10.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.702+0000 7ff610e1b700 1 -- 172.21.15.40:0/686861971 shutdown_connections 2024-09-13T17:51:10.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.702+0000 7ff610e1b700 1 -- 172.21.15.40:0/686861971 wait complete. 2024-09-13T17:51:10.717 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:51:10.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.818+0000 7ffa7adaa700 1 Processor -- start 2024-09-13T17:51:10.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa7adaa700 1 -- start start 2024-09-13T17:51:10.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa7adaa700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa74071f70 0x7ffa740c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:10.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa7adaa700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffa74072350 0x7ffa74150d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:10.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa7adaa700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa740c55c0 con 0x7ffa740c7200 2024-09-13T17:51:10.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa7adaa700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7406a390 con 0x7ffa74072350 2024-09-13T17:51:10.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa7adaa700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa74157710 con 0x7ffa74071f70 2024-09-13T17:51:10.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa795a7700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffa74072350 0x7ffa74150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:10.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa79da8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa74071f70 0x7ffa740c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:10.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa7a5a9700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ffa740c7200 0x7ffa7415a700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:35498/0 (socket says 172.21.15.40:35498) 2024-09-13T17:51:10.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa7a5a9700 1 -- 172.21.15.40:0/435033807 learned_addr learned my addr 172.21.15.40:0/435033807 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:10.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa795a7700 1 -- 172.21.15.40:0/435033807 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa74071f70 msgr2=0x7ffa740c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:10.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa795a7700 1 --2- 172.21.15.40:0/435033807 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa74071f70 0x7ffa740c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa795a7700 1 -- 172.21.15.40:0/435033807 >> v1:172.21.15.43:6789/0 conn(0x7ffa740c7200 legacy=0x7ffa7415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:10.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa795a7700 1 -- 172.21.15.40:0/435033807 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa741579d0 con 0x7ffa74072350 2024-09-13T17:51:10.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa795a7700 1 --2- 172.21.15.40:0/435033807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa74072350 0x7ffa74150d40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009850 tx=0x7ffa64009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18ea56bd1e20667d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:10.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa78da6700 1 -- 172.21.15.40:0/435033807 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64011830 con 0x7ffa74072350 2024-09-13T17:51:10.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa78da6700 1 -- 172.21.15.40:0/435033807 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa64011990 con 0x7ffa74072350 2024-09-13T17:51:10.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa78da6700 1 -- 172.21.15.40:0/435033807 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64011d40 con 0x7ffa74072350 2024-09-13T17:51:10.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa7adaa700 1 -- 172.21.15.40:0/435033807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa74072350 msgr2=0x7ffa74150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:10.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa7adaa700 1 --2- 172.21.15.40:0/435033807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa74072350 0x7ffa74150d40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009850 tx=0x7ffa64009b60 comp rx=0 tx=0).stop 2024-09-13T17:51:10.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa7adaa700 1 -- 172.21.15.40:0/435033807 shutdown_connections 2024-09-13T17:51:10.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa7adaa700 1 --2- 172.21.15.40:0/435033807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa74072350 0x7ffa74150d40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa7adaa700 1 --2- 172.21.15.40:0/435033807 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa74071f70 0x7ffa740c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa7adaa700 1 -- 172.21.15.40:0/435033807 >> 172.21.15.40:0/435033807 conn(0x7ffa740cabc0 msgr2=0x7ffa740cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:10.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa7adaa700 1 -- 172.21.15.40:0/435033807 shutdown_connections 2024-09-13T17:51:10.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.822+0000 7ffa7adaa700 1 -- 172.21.15.40:0/435033807 wait complete. 2024-09-13T17:51:10.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa7adaa700 1 Processor -- start 2024-09-13T17:51:10.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa7adaa700 1 -- start start 2024-09-13T17:51:10.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa7adaa700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa74071f70 0x7ffa741e2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:10.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa7adaa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa74072350 0x7ffa740c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:10.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa7adaa700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa740c7200 0x7ffa740c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:10.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa7adaa700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa741e0a10 con 0x7ffa74072350 2024-09-13T17:51:10.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa7adaa700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa740c8ac0 con 0x7ffa74071f70 2024-09-13T17:51:10.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa7adaa700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa741e0cd0 con 0x7ffa740c7200 2024-09-13T17:51:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa795a7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa74072350 0x7ffa740c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa795a7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa74072350 0x7ffa740c8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58314/0 (socket says 172.21.15.40:58314) 2024-09-13T17:51:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa795a7700 1 -- 172.21.15.40:0/843437653 learned_addr learned my addr 172.21.15.40:0/843437653 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa79da8700 1 --2- 172.21.15.40:0/843437653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa74071f70 0x7ffa741e2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa7a5a9700 1 --2- 172.21.15.40:0/843437653 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa740c7200 0x7ffa740c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa795a7700 1 -- 172.21.15.40:0/843437653 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa740c7200 msgr2=0x7ffa740c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa795a7700 1 --2- 172.21.15.40:0/843437653 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa740c7200 0x7ffa740c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa795a7700 1 -- 172.21.15.40:0/843437653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa74071f70 msgr2=0x7ffa741e2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa795a7700 1 --2- 172.21.15.40:0/843437653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa74071f70 0x7ffa741e2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa795a7700 1 -- 172.21.15.40:0/843437653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa640094b0 con 0x7ffa74072350 2024-09-13T17:51:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa795a7700 1 --2- 172.21.15.40:0/843437653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa74072350 0x7ffa740c8fc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffa640090b0 tx=0x7ffa64004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa6a7fc700 1 -- 172.21.15.40:0/843437653 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64004480 con 0x7ffa74072350 2024-09-13T17:51:10.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa6a7fc700 1 -- 172.21.15.40:0/843437653 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6401f070 con 0x7ffa74072350 2024-09-13T17:51:10.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa7adaa700 1 -- 172.21.15.40:0/843437653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa740ca300 con 0x7ffa74072350 2024-09-13T17:51:10.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa6a7fc700 1 -- 172.21.15.40:0/843437653 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64003900 con 0x7ffa74072350 2024-09-13T17:51:10.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.826+0000 7ffa7adaa700 1 -- 172.21.15.40:0/843437653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa740ca8c0 con 0x7ffa74072350 2024-09-13T17:51:10.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.830+0000 7ffa7adaa700 1 -- 172.21.15.40:0/843437653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa740cb6e0 con 0x7ffa74072350 2024-09-13T17:51:10.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.830+0000 7ffa6a7fc700 1 -- 172.21.15.40:0/843437653 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffa64021030 con 0x7ffa74072350 2024-09-13T17:51:10.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.830+0000 7ffa6a7fc700 1 --2- 172.21.15.40:0/843437653 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffa58066760 0x7ffa58068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:10.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.830+0000 7ffa6a7fc700 1 -- 172.21.15.40:0/843437653 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ffa64085310 con 0x7ffa74072350 2024-09-13T17:51:10.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.830+0000 7ffa79da8700 1 --2- 172.21.15.40:0/843437653 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffa58066760 0x7ffa58068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:10.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.830+0000 7ffa79da8700 1 --2- 172.21.15.40:0/843437653 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffa58066760 0x7ffa58068c20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009bf0 tx=0x7ffa70009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:10.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:10.838+0000 7ffa6a7fc700 1 -- 172.21.15.40:0/843437653 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ffa64052310 con 0x7ffa74072350 2024-09-13T17:51:11.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.042+0000 7ffa7adaa700 1 -- 172.21.15.40:0/843437653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa741e31c0 con 0x7ffa74072350 2024-09-13T17:51:11.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.042+0000 7ffa6a7fc700 1 -- 172.21.15.40:0/843437653 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+6269 (secure 0 0 0) 0x7ffa64055930 con 0x7ffa74072350 2024-09-13T17:51:11.049 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:11.049 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6839/1861594206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1861594206},{"type":"v1","addr":"172.21.15.40:6839","nonce":1861594206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1548394691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1548394691},{"type":"v1","addr":"172.21.15.40:6835","nonce":1548394691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/486611242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":486611242},{"type":"v1","addr":"172.21.15.40:6837","nonce":486611242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:50:34.562623+0000","modified":"2024-09-13T17:51:06.207762+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4288,"mds_1":4285,"mds_2":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"f","rank":2,"incarnation":14,"state":"up:creating","state_seq":1,"addr":"172.21.15.43:6833/526975022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":526975022},{"type":"v1","addr":"172.21.15.43:6833","nonce":526975022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4285":{"gid":4285,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.43:6835/2586910944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2586910944},{"type":"v1","addr":"172.21.15.43:6835","nonce":2586910944}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/334160033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":334160033},{"type":"v1","addr":"172.21.15.43:6837","nonce":334160033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:51:11.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.046+0000 7ffa7adaa700 1 -- 172.21.15.40:0/843437653 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffa58066760 msgr2=0x7ffa58068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:11.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.046+0000 7ffa7adaa700 1 --2- 172.21.15.40:0/843437653 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffa58066760 0x7ffa58068c20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffa70009bf0 tx=0x7ffa70009240 comp rx=0 tx=0).stop 2024-09-13T17:51:11.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.046+0000 7ffa7adaa700 1 -- 172.21.15.40:0/843437653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa74072350 msgr2=0x7ffa740c8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:11.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.046+0000 7ffa7adaa700 1 --2- 172.21.15.40:0/843437653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa74072350 0x7ffa740c8fc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffa640090b0 tx=0x7ffa64004770 comp rx=0 tx=0).stop 2024-09-13T17:51:11.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.046+0000 7ffa7adaa700 1 -- 172.21.15.40:0/843437653 shutdown_connections 2024-09-13T17:51:11.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.046+0000 7ffa7adaa700 1 --2- 172.21.15.40:0/843437653 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffa58066760 0x7ffa58068c20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.046+0000 7ffa7adaa700 1 --2- 172.21.15.40:0/843437653 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa740c7200 0x7ffa740c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.046+0000 7ffa7adaa700 1 --2- 172.21.15.40:0/843437653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa74072350 0x7ffa740c8fc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.046+0000 7ffa7adaa700 1 --2- 172.21.15.40:0/843437653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa74071f70 0x7ffa741e2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.046+0000 7ffa7adaa700 1 -- 172.21.15.40:0/843437653 >> 172.21.15.40:0/843437653 conn(0x7ffa740cabc0 msgr2=0x7ffa740c76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:11.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.046+0000 7ffa7adaa700 1 -- 172.21.15.40:0/843437653 shutdown_connections 2024-09-13T17:51:11.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.046+0000 7ffa7adaa700 1 -- 172.21.15.40:0/843437653 wait complete. 2024-09-13T17:51:11.053 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 14 2024-09-13T17:51:11.065 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:51:11.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd3320700 1 Processor -- start 2024-09-13T17:51:11.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd3320700 1 -- start start 2024-09-13T17:51:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd3320700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ccc15e1a0 0x7f4ccc06a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd3320700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4ccc06a8d0 0x7f4ccc06ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd3320700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ccc071690 con 0x7f4ccc154240 2024-09-13T17:51:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd3320700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ccc0721e0 con 0x7f4ccc15e1a0 2024-09-13T17:51:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd3320700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc154620 con 0x7f4ccc06a8d0 2024-09-13T17:51:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd1b1d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ccc15e1a0 0x7f4ccc06a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:11.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd1b1d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4ccc15e1a0 0x7f4ccc06a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58322/0 (socket says 172.21.15.40:58322) 2024-09-13T17:51:11.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd1b1d700 1 -- 172.21.15.40:0/2064836535 learned_addr learned my addr 172.21.15.40:0/2064836535 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:11.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd2b1f700 1 --2- 172.21.15.40:0/2064836535 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4ccc06a8d0 0x7f4ccc06ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:11.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd1b1d700 1 -- 172.21.15.40:0/2064836535 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4ccc06a8d0 msgr2=0x7f4ccc06ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:11.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd1b1d700 1 --2- 172.21.15.40:0/2064836535 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4ccc06a8d0 0x7f4ccc06ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd1b1d700 1 -- 172.21.15.40:0/2064836535 >> v1:172.21.15.43:6789/0 conn(0x7f4ccc154240 legacy=0x7f4ccc15ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:11.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd1b1d700 1 -- 172.21.15.40:0/2064836535 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ccc071b20 con 0x7f4ccc15e1a0 2024-09-13T17:51:11.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd1b1d700 1 --2- 172.21.15.40:0/2064836535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc15e1a0 0x7f4ccc06a390 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009e50 tx=0x7f4cc400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123c7b1b387afccf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:11.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd131c700 1 -- 172.21.15.40:0/2064836535 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc400e070 con 0x7f4ccc15e1a0 2024-09-13T17:51:11.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd131c700 1 -- 172.21.15.40:0/2064836535 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc400b9c0 con 0x7f4ccc15e1a0 2024-09-13T17:51:11.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd131c700 1 -- 172.21.15.40:0/2064836535 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc400bd70 con 0x7f4ccc15e1a0 2024-09-13T17:51:11.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd3320700 1 -- 172.21.15.40:0/2064836535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc15e1a0 msgr2=0x7f4ccc06a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:11.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd3320700 1 --2- 172.21.15.40:0/2064836535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc15e1a0 0x7f4ccc06a390 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009e50 tx=0x7f4cc400b330 comp rx=0 tx=0).stop 2024-09-13T17:51:11.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd3320700 1 -- 172.21.15.40:0/2064836535 shutdown_connections 2024-09-13T17:51:11.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd3320700 1 --2- 172.21.15.40:0/2064836535 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4ccc06a8d0 0x7f4ccc06ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd3320700 1 --2- 172.21.15.40:0/2064836535 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc15e1a0 0x7f4ccc06a390 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd3320700 1 -- 172.21.15.40:0/2064836535 >> 172.21.15.40:0/2064836535 conn(0x7f4ccc0cabc0 msgr2=0x7f4ccc0cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:11.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd3320700 1 -- 172.21.15.40:0/2064836535 shutdown_connections 2024-09-13T17:51:11.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.174+0000 7f4cd3320700 1 -- 172.21.15.40:0/2064836535 wait complete. 2024-09-13T17:51:11.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cd3320700 1 Processor -- start 2024-09-13T17:51:11.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cd3320700 1 -- start start 2024-09-13T17:51:11.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cd3320700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4ccc06a8d0 0x7f4ccc150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:11.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cd3320700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ccc154240 0x7f4ccc1550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:11.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cd3320700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc15e1a0 0x7f4ccc1f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:11.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cd3320700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc1555f0 con 0x7f4ccc15e1a0 2024-09-13T17:51:11.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cd3320700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc1f6a60 con 0x7f4ccc154240 2024-09-13T17:51:11.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cd3320700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc0c6800 con 0x7f4ccc06a8d0 2024-09-13T17:51:11.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cd2b1f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc15e1a0 0x7f4ccc1f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:11.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cd2b1f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc15e1a0 0x7f4ccc1f71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58334/0 (socket says 172.21.15.40:58334) 2024-09-13T17:51:11.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cd2b1f700 1 -- 172.21.15.40:0/41905467 learned_addr learned my addr 172.21.15.40:0/41905467 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:11.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cd1b1d700 1 --2- 172.21.15.40:0/41905467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ccc154240 0x7f4ccc1550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:11.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cd231e700 1 --2- 172.21.15.40:0/41905467 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4ccc06a8d0 0x7f4ccc150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:11.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cd2b1f700 1 -- 172.21.15.40:0/41905467 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4ccc06a8d0 msgr2=0x7f4ccc150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:11.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cd2b1f700 1 --2- 172.21.15.40:0/41905467 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4ccc06a8d0 0x7f4ccc150ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cd2b1f700 1 -- 172.21.15.40:0/41905467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ccc154240 msgr2=0x7f4ccc1550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:11.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cd2b1f700 1 --2- 172.21.15.40:0/41905467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ccc154240 0x7f4ccc1550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cd2b1f700 1 -- 172.21.15.40:0/41905467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc4009b30 con 0x7f4ccc15e1a0 2024-09-13T17:51:11.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cd2b1f700 1 --2- 172.21.15.40:0/41905467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc15e1a0 0x7f4ccc1f71d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00e810 tx=0x7f4cbc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:11.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cc2ffd700 1 -- 172.21.15.40:0/41905467 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc00c070 con 0x7f4ccc15e1a0 2024-09-13T17:51:11.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cc2ffd700 1 -- 172.21.15.40:0/41905467 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cbc013630 con 0x7f4ccc15e1a0 2024-09-13T17:51:11.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cd3320700 1 -- 172.21.15.40:0/41905467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ccc1f43b0 con 0x7f4ccc15e1a0 2024-09-13T17:51:11.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.178+0000 7f4cc2ffd700 1 -- 172.21.15.40:0/41905467 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc0139e0 con 0x7f4ccc15e1a0 2024-09-13T17:51:11.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.182+0000 7f4cd3320700 1 -- 172.21.15.40:0/41905467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ccc0c5630 con 0x7f4ccc15e1a0 2024-09-13T17:51:11.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.182+0000 7f4cd3320700 1 -- 172.21.15.40:0/41905467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ccc071320 con 0x7f4ccc15e1a0 2024-09-13T17:51:11.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.182+0000 7f4cc2ffd700 1 -- 172.21.15.40:0/41905467 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4cbc013b40 con 0x7f4ccc15e1a0 2024-09-13T17:51:11.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.182+0000 7f4cc2ffd700 1 --2- 172.21.15.40:0/41905467 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4cb8066730 0x7f4cb8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:11.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.182+0000 7f4cc2ffd700 1 -- 172.21.15.40:0/41905467 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4cbc0840c0 con 0x7f4ccc15e1a0 2024-09-13T17:51:11.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.182+0000 7f4cd231e700 1 --2- 172.21.15.40:0/41905467 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4cb8066730 0x7f4cb8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:11.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.186+0000 7f4cd231e700 1 --2- 172.21.15.40:0/41905467 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4cb8066730 0x7f4cb8068bf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8000f60 tx=0x7f4cc8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:11.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.190+0000 7f4cc2ffd700 1 -- 172.21.15.40:0/41905467 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4cbc051140 con 0x7f4ccc15e1a0 2024-09-13T17:51:11.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.370+0000 7f4cd3320700 1 -- 172.21.15.40:0/41905467 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ccc15a960 con 0x7f4ccc15e1a0 2024-09-13T17:51:11.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.374+0000 7f4cc2ffd700 1 -- 172.21.15.40:0/41905467 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18262 (secure 0 0 0) 0x7f4cbc054760 con 0x7f4ccc15e1a0 2024-09-13T17:51:11.380 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:11.381 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:50:38.974071+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:50:32.740336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:50:33.552914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:50:36.623639+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:51:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.378+0000 7f4cd3320700 1 -- 172.21.15.40:0/41905467 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4cb8066730 msgr2=0x7f4cb8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.378+0000 7f4cd3320700 1 --2- 172.21.15.40:0/41905467 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4cb8066730 0x7f4cb8068bf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8000f60 tx=0x7f4cc8009040 comp rx=0 tx=0).stop 2024-09-13T17:51:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.378+0000 7f4cd3320700 1 -- 172.21.15.40:0/41905467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc15e1a0 msgr2=0x7f4ccc1f71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.378+0000 7f4cd3320700 1 --2- 172.21.15.40:0/41905467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc15e1a0 0x7f4ccc1f71d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00e810 tx=0x7f4cbc00eb20 comp rx=0 tx=0).stop 2024-09-13T17:51:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.378+0000 7f4cd3320700 1 -- 172.21.15.40:0/41905467 shutdown_connections 2024-09-13T17:51:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.378+0000 7f4cd3320700 1 --2- 172.21.15.40:0/41905467 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ccc15e1a0 0x7f4ccc1f71d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.378+0000 7f4cd3320700 1 --2- 172.21.15.40:0/41905467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ccc154240 0x7f4ccc1550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.378+0000 7f4cd3320700 1 --2- 172.21.15.40:0/41905467 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4cb8066730 0x7f4cb8068bf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.378+0000 7f4cd3320700 1 --2- 172.21.15.40:0/41905467 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4ccc06a8d0 0x7f4ccc150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.378+0000 7f4cd3320700 1 -- 172.21.15.40:0/41905467 >> 172.21.15.40:0/41905467 conn(0x7f4ccc0cabc0 msgr2=0x7f4ccc0c8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.378+0000 7f4cd3320700 1 -- 172.21.15.40:0/41905467 shutdown_connections 2024-09-13T17:51:11.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.378+0000 7f4cd3320700 1 -- 172.21.15.40:0/41905467 wait complete. 2024-09-13T17:51:11.396 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:50:21.057171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:50:32.740336+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:50:33.552914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:50:36.623639+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:51:11.396 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:51:11.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.506+0000 7f05ca06a700 1 Processor -- start 2024-09-13T17:51:11.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.506+0000 7f05ca06a700 1 -- start start 2024-09-13T17:51:11.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.506+0000 7f05ca06a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05c40cf1a0 0x7f05c415a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:11.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.506+0000 7f05ca06a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f05c40cf580 0x7f05c40c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:11.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.506+0000 7f05ca06a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c415a7a0 con 0x7f05c40cc250 2024-09-13T17:51:11.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.506+0000 7f05ca06a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c40cbdc0 con 0x7f05c40cf580 2024-09-13T17:51:11.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.506+0000 7f05ca06a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c4153f80 con 0x7f05c40cf1a0 2024-09-13T17:51:11.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.506+0000 7f05c8867700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f05c40cf580 0x7f05c40c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:11.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.506+0000 7f05c9869700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f05c40cc250 0x7f05c40c8aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59858/0 (socket says 172.21.15.40:59858) 2024-09-13T17:51:11.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.506+0000 7f05c9869700 1 -- 172.21.15.40:0/3203029540 learned_addr learned my addr 172.21.15.40:0/3203029540 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:11.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.506+0000 7f05c9068700 1 --2- 172.21.15.40:0/3203029540 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05c40cf1a0 0x7f05c415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:11.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.510+0000 7f05b7fff700 1 -- 172.21.15.40:0/3203029540 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099076048 0 0) 0x7f05bc001490 con 0x7f05c40cc250 2024-09-13T17:51:11.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.510+0000 7f05c8867700 1 -- 172.21.15.40:0/3203029540 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05c40cf1a0 msgr2=0x7f05c415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:11.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.510+0000 7f05c8867700 1 --2- 172.21.15.40:0/3203029540 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05c40cf1a0 0x7f05c415a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.510+0000 7f05c8867700 1 -- 172.21.15.40:0/3203029540 >> v1:172.21.15.40:6789/0 conn(0x7f05c40cc250 legacy=0x7f05c40c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:11.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.510+0000 7f05c8867700 1 -- 172.21.15.40:0/3203029540 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c40caed0 con 0x7f05c40cf580 2024-09-13T17:51:11.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.510+0000 7f05c8867700 1 --2- 172.21.15.40:0/3203029540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c40cf580 0x7f05c40c9fd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009850 tx=0x7f05c0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b7387360dac64e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:11.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.510+0000 7f05b7fff700 1 -- 172.21.15.40:0/3203029540 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c0011830 con 0x7f05c40cf580 2024-09-13T17:51:11.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.510+0000 7f05b7fff700 1 -- 172.21.15.40:0/3203029540 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c0011990 con 0x7f05c40cf580 2024-09-13T17:51:11.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.510+0000 7f05b7fff700 1 -- 172.21.15.40:0/3203029540 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c0011d40 con 0x7f05c40cf580 2024-09-13T17:51:11.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.510+0000 7f05ca06a700 1 -- 172.21.15.40:0/3203029540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c40cf580 msgr2=0x7f05c40c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:11.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.510+0000 7f05ca06a700 1 --2- 172.21.15.40:0/3203029540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c40cf580 0x7f05c40c9fd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f05c0009850 tx=0x7f05c0009b60 comp rx=0 tx=0).stop 2024-09-13T17:51:11.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.510+0000 7f05ca06a700 1 -- 172.21.15.40:0/3203029540 shutdown_connections 2024-09-13T17:51:11.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.510+0000 7f05ca06a700 1 --2- 172.21.15.40:0/3203029540 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c40cf580 0x7f05c40c9fd0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.510+0000 7f05ca06a700 1 --2- 172.21.15.40:0/3203029540 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05c40cf1a0 0x7f05c415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.510+0000 7f05ca06a700 1 -- 172.21.15.40:0/3203029540 >> 172.21.15.40:0/3203029540 conn(0x7f05c4066090 msgr2=0x7f05c4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:11.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.510+0000 7f05ca06a700 1 -- 172.21.15.40:0/3203029540 shutdown_connections 2024-09-13T17:51:11.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.510+0000 7f05ca06a700 1 -- 172.21.15.40:0/3203029540 wait complete. 2024-09-13T17:51:11.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.510+0000 7f05ca06a700 1 Processor -- start 2024-09-13T17:51:11.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05ca06a700 1 -- start start 2024-09-13T17:51:11.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05ca06a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05c40cc250 0x7f05c4159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:11.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05ca06a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c40cf1a0 0x7f05c414dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:11.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05ca06a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05c40cf580 0x7f05c415b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:11.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05ca06a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c4157ca0 con 0x7f05c40cf1a0 2024-09-13T17:51:11.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05ca06a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c4159f70 con 0x7f05c40cf580 2024-09-13T17:51:11.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05ca06a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c4158840 con 0x7f05c40cc250 2024-09-13T17:51:11.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05c8867700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c40cf1a0 0x7f05c414dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:11.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05c8867700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c40cf1a0 0x7f05c414dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51084/0 (socket says 172.21.15.40:51084) 2024-09-13T17:51:11.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05c8867700 1 -- 172.21.15.40:0/3327499674 learned_addr learned my addr 172.21.15.40:0/3327499674 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:11.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05c9068700 1 --2- 172.21.15.40:0/3327499674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05c40cc250 0x7f05c4159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:11.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05c9869700 1 --2- 172.21.15.40:0/3327499674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05c40cf580 0x7f05c415b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:11.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05c8867700 1 -- 172.21.15.40:0/3327499674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05c40cc250 msgr2=0x7f05c4159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:11.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05c8867700 1 --2- 172.21.15.40:0/3327499674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05c40cc250 0x7f05c4159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05c8867700 1 -- 172.21.15.40:0/3327499674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05c40cf580 msgr2=0x7f05c415b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:11.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05c8867700 1 --2- 172.21.15.40:0/3327499674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05c40cf580 0x7f05c415b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05c8867700 1 -- 172.21.15.40:0/3327499674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c00094b0 con 0x7f05c40cf1a0 2024-09-13T17:51:11.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05c8867700 1 --2- 172.21.15.40:0/3327499674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c40cf1a0 0x7f05c414dd90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f05c0000c00 tx=0x7f05c0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:11.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05b5ffb700 1 -- 172.21.15.40:0/3327499674 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c0019ab0 con 0x7f05c40cf1a0 2024-09-13T17:51:11.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05b5ffb700 1 -- 172.21.15.40:0/3327499674 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c001e070 con 0x7f05c40cf1a0 2024-09-13T17:51:11.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05b5ffb700 1 -- 172.21.15.40:0/3327499674 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c0019e50 con 0x7f05c40cf1a0 2024-09-13T17:51:11.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05ca06a700 1 -- 172.21.15.40:0/3327499674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c4158b00 con 0x7f05c40cf1a0 2024-09-13T17:51:11.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05ca06a700 1 -- 172.21.15.40:0/3327499674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c41579c0 con 0x7f05c40cf1a0 2024-09-13T17:51:11.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.514+0000 7f05ca06a700 1 -- 172.21.15.40:0/3327499674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c4067a40 con 0x7f05c40cf1a0 2024-09-13T17:51:11.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.518+0000 7f05b5ffb700 1 -- 172.21.15.40:0/3327499674 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f05c002d450 con 0x7f05c40cf1a0 2024-09-13T17:51:11.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.518+0000 7f05b5ffb700 1 --2- 172.21.15.40:0/3327499674 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f05a8066760 0x7f05a8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:11.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.518+0000 7f05b5ffb700 1 -- 172.21.15.40:0/3327499674 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f05c00855f0 con 0x7f05c40cf1a0 2024-09-13T17:51:11.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.518+0000 7f05c9068700 1 --2- 172.21.15.40:0/3327499674 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f05a8066760 0x7f05a8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:11.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.518+0000 7f05c9068700 1 --2- 172.21.15.40:0/3327499674 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f05a8066760 0x7f05a8068c20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f05c415c150 tx=0x7f05b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:11.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.530+0000 7f05b5ffb700 1 -- 172.21.15.40:0/3327499674 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f05c00525e0 con 0x7f05c40cf1a0 2024-09-13T17:51:11.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.702+0000 7f05ca06a700 1 -- 172.21.15.40:0/3327499674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f05c40cfb00 con 0x7f05c40cf1a0 2024-09-13T17:51:11.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.702+0000 7f05b5ffb700 1 -- 172.21.15.40:0/3327499674 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f05c00200a0 con 0x7f05c40cf1a0 2024-09-13T17:51:11.708 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-09-13T17:51:11.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.706+0000 7f05ca06a700 1 -- 172.21.15.40:0/3327499674 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f05a8066760 msgr2=0x7f05a8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:11.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.706+0000 7f05ca06a700 1 --2- 172.21.15.40:0/3327499674 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f05a8066760 0x7f05a8068c20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f05c415c150 tx=0x7f05b8008040 comp rx=0 tx=0).stop 2024-09-13T17:51:11.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.706+0000 7f05ca06a700 1 -- 172.21.15.40:0/3327499674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c40cf1a0 msgr2=0x7f05c414dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:11.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.706+0000 7f05ca06a700 1 --2- 172.21.15.40:0/3327499674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c40cf1a0 0x7f05c414dd90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f05c0000c00 tx=0x7f05c0004770 comp rx=0 tx=0).stop 2024-09-13T17:51:11.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.706+0000 7f05ca06a700 1 -- 172.21.15.40:0/3327499674 shutdown_connections 2024-09-13T17:51:11.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.706+0000 7f05ca06a700 1 --2- 172.21.15.40:0/3327499674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05c40cf580 0x7f05c415b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.706+0000 7f05ca06a700 1 --2- 172.21.15.40:0/3327499674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05c40cf1a0 0x7f05c414dd90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.706+0000 7f05ca06a700 1 --2- 172.21.15.40:0/3327499674 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f05a8066760 0x7f05a8068c20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.706+0000 7f05ca06a700 1 --2- 172.21.15.40:0/3327499674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05c40cc250 0x7f05c4159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.706+0000 7f05ca06a700 1 -- 172.21.15.40:0/3327499674 >> 172.21.15.40:0/3327499674 conn(0x7f05c4066090 msgr2=0x7f05c40c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:11.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.706+0000 7f05ca06a700 1 -- 172.21.15.40:0/3327499674 shutdown_connections 2024-09-13T17:51:11.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.706+0000 7f05ca06a700 1 -- 172.21.15.40:0/3327499674 wait complete. 2024-09-13T17:51:11.727 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:51:11.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.846+0000 7f63de4dd700 1 Processor -- start 2024-09-13T17:51:11.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.846+0000 7f63de4dd700 1 -- start start 2024-09-13T17:51:11.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.846+0000 7f63de4dd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63d8156b40 0x7f63d80c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:11.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.846+0000 7f63de4dd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63d8072420 0x7f63d80c5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:11.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.846+0000 7f63de4dd700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d80c3940 con 0x7f63d80b1fc0 2024-09-13T17:51:11.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.846+0000 7f63de4dd700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d80c5c10 con 0x7f63d8072420 2024-09-13T17:51:11.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.846+0000 7f63de4dd700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d80c4580 con 0x7f63d8156b40 2024-09-13T17:51:11.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.846+0000 7f63ddcdc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63d8072420 0x7f63d80c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:11.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.846+0000 7f63ddcdc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63d8072420 0x7f63d80c5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51086/0 (socket says 172.21.15.40:51086) 2024-09-13T17:51:11.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.846+0000 7f63ddcdc700 1 -- 172.21.15.40:0/3232183493 learned_addr learned my addr 172.21.15.40:0/3232183493 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:11.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.846+0000 7f63dd4db700 1 --2- 172.21.15.40:0/3232183493 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63d8156b40 0x7f63d80c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:11.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.846+0000 7f63dccda700 1 --1- 172.21.15.40:0/3232183493 >> v1:172.21.15.43:6789/0 conn(0x7f63d80b1fc0 0x7f63d8071d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:51030/0 (socket says 172.21.15.40:51030) 2024-09-13T17:51:11.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.846+0000 7f63ddcdc700 1 -- 172.21.15.40:0/3232183493 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63d8156b40 msgr2=0x7f63d80c4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:11.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.846+0000 7f63ddcdc700 1 --2- 172.21.15.40:0/3232183493 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63d8156b40 0x7f63d80c4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.846+0000 7f63ddcdc700 1 -- 172.21.15.40:0/3232183493 >> v1:172.21.15.43:6789/0 conn(0x7f63d80b1fc0 legacy=0x7f63d8071d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:11.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.846+0000 7f63ddcdc700 1 -- 172.21.15.40:0/3232183493 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d80c12b0 con 0x7f63d8072420 2024-09-13T17:51:11.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63ddcdc700 1 --2- 172.21.15.40:0/3232183493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d8072420 0x7f63d80c5380 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009e50 tx=0x7f63d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1f29f4bd7b93e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:11.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63cffff700 1 -- 172.21.15.40:0/3232183493 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d000e070 con 0x7f63d8072420 2024-09-13T17:51:11.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63cffff700 1 -- 172.21.15.40:0/3232183493 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d000b910 con 0x7f63d8072420 2024-09-13T17:51:11.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63cffff700 1 -- 172.21.15.40:0/3232183493 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d000bcc0 con 0x7f63d8072420 2024-09-13T17:51:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63de4dd700 1 -- 172.21.15.40:0/3232183493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d8072420 msgr2=0x7f63d80c5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63de4dd700 1 --2- 172.21.15.40:0/3232183493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d8072420 0x7f63d80c5380 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f63d0009e50 tx=0x7f63d000b330 comp rx=0 tx=0).stop 2024-09-13T17:51:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63de4dd700 1 -- 172.21.15.40:0/3232183493 shutdown_connections 2024-09-13T17:51:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63de4dd700 1 --2- 172.21.15.40:0/3232183493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d8072420 0x7f63d80c5380 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63de4dd700 1 --2- 172.21.15.40:0/3232183493 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63d8156b40 0x7f63d80c4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63de4dd700 1 -- 172.21.15.40:0/3232183493 >> 172.21.15.40:0/3232183493 conn(0x7f63d806e4f0 msgr2=0x7f63d806e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63de4dd700 1 -- 172.21.15.40:0/3232183493 shutdown_connections 2024-09-13T17:51:11.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63de4dd700 1 -- 172.21.15.40:0/3232183493 wait complete. 2024-09-13T17:51:11.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63de4dd700 1 Processor -- start 2024-09-13T17:51:11.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63de4dd700 1 -- start start 2024-09-13T17:51:11.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63de4dd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63d8072420 0x7f63d81511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:11.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63de4dd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63d80b1fc0 0x7f63d8153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:11.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63de4dd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d8156b40 0x7f63d8158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:11.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63de4dd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d806bdb0 con 0x7f63d8156b40 2024-09-13T17:51:11.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63de4dd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d806a6f0 con 0x7f63d80b1fc0 2024-09-13T17:51:11.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63de4dd700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d8152350 con 0x7f63d8072420 2024-09-13T17:51:11.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63ddcdc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d8156b40 0x7f63d8158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:11.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63ddcdc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d8156b40 0x7f63d8158c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51090/0 (socket says 172.21.15.40:51090) 2024-09-13T17:51:11.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63ddcdc700 1 -- 172.21.15.40:0/3434495597 learned_addr learned my addr 172.21.15.40:0/3434495597 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:11.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63dd4db700 1 --2- 172.21.15.40:0/3434495597 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63d8072420 0x7f63d81511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:11.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.850+0000 7f63dccda700 1 --2- 172.21.15.40:0/3434495597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63d80b1fc0 0x7f63d8153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:11.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.854+0000 7f63ddcdc700 1 -- 172.21.15.40:0/3434495597 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63d8072420 msgr2=0x7f63d81511a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:11.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.854+0000 7f63ddcdc700 1 --2- 172.21.15.40:0/3434495597 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63d8072420 0x7f63d81511a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.854+0000 7f63ddcdc700 1 -- 172.21.15.40:0/3434495597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63d80b1fc0 msgr2=0x7f63d8153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:11.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.854+0000 7f63ddcdc700 1 --2- 172.21.15.40:0/3434495597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63d80b1fc0 0x7f63d8153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:11.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.854+0000 7f63ddcdc700 1 -- 172.21.15.40:0/3434495597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d0009b30 con 0x7f63d8156b40 2024-09-13T17:51:11.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.854+0000 7f63ddcdc700 1 --2- 172.21.15.40:0/3434495597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d8156b40 0x7f63d8158c50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f63d0012000 tx=0x7f63d0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:11.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.854+0000 7f63cdffb700 1 -- 172.21.15.40:0/3434495597 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d0004800 con 0x7f63d8156b40 2024-09-13T17:51:11.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.854+0000 7f63cdffb700 1 -- 172.21.15.40:0/3434495597 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d00099b0 con 0x7f63d8156b40 2024-09-13T17:51:11.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.854+0000 7f63cdffb700 1 -- 172.21.15.40:0/3434495597 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d001c680 con 0x7f63d8156b40 2024-09-13T17:51:11.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.854+0000 7f63de4dd700 1 -- 172.21.15.40:0/3434495597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d8152630 con 0x7f63d8156b40 2024-09-13T17:51:11.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.854+0000 7f63de4dd700 1 -- 172.21.15.40:0/3434495597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d8152bf0 con 0x7f63d8156b40 2024-09-13T17:51:11.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.854+0000 7f63de4dd700 1 -- 172.21.15.40:0/3434495597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d8153650 con 0x7f63d8156b40 2024-09-13T17:51:11.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.854+0000 7f63cdffb700 1 -- 172.21.15.40:0/3434495597 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f63d001c7e0 con 0x7f63d8156b40 2024-09-13T17:51:11.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.858+0000 7f63cdffb700 1 --2- 172.21.15.40:0/3434495597 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f63bc066760 0x7f63bc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:11.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.858+0000 7f63dd4db700 1 --2- 172.21.15.40:0/3434495597 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f63bc066760 0x7f63bc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:11.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.858+0000 7f63cdffb700 1 -- 172.21.15.40:0/3434495597 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f63d0085c40 con 0x7f63d8156b40 2024-09-13T17:51:11.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.858+0000 7f63dd4db700 1 --2- 172.21.15.40:0/3434495597 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f63bc066760 0x7f63bc068c20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009b50 tx=0x7f63d4009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:11.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:11.862+0000 7f63cdffb700 1 -- 172.21.15.40:0/3434495597 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f63d0052cc0 con 0x7f63d8156b40 2024-09-13T17:51:12.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.030+0000 7f63de4dd700 1 -- 172.21.15.40:0/3434495597 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f63d8067670 con 0x7f63d8156b40 2024-09-13T17:51:12.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.030+0000 7f63cdffb700 1 -- 172.21.15.40:0/3434495597 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f63d00562e0 con 0x7f63d8156b40 2024-09-13T17:51:12.035 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-09-13T17:51:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.034+0000 7f63de4dd700 1 -- 172.21.15.40:0/3434495597 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f63bc066760 msgr2=0x7f63bc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.034+0000 7f63de4dd700 1 --2- 172.21.15.40:0/3434495597 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f63bc066760 0x7f63bc068c20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009b50 tx=0x7f63d4009240 comp rx=0 tx=0).stop 2024-09-13T17:51:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.034+0000 7f63de4dd700 1 -- 172.21.15.40:0/3434495597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d8156b40 msgr2=0x7f63d8158c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.034+0000 7f63de4dd700 1 --2- 172.21.15.40:0/3434495597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d8156b40 0x7f63d8158c50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f63d0012000 tx=0x7f63d0004af0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.034+0000 7f63de4dd700 1 -- 172.21.15.40:0/3434495597 shutdown_connections 2024-09-13T17:51:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.034+0000 7f63de4dd700 1 --2- 172.21.15.40:0/3434495597 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d8156b40 0x7f63d8158c50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.034+0000 7f63de4dd700 1 --2- 172.21.15.40:0/3434495597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63d80b1fc0 0x7f63d8153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.034+0000 7f63de4dd700 1 --2- 172.21.15.40:0/3434495597 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f63bc066760 0x7f63bc068c20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.034+0000 7f63de4dd700 1 --2- 172.21.15.40:0/3434495597 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63d8072420 0x7f63d81511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.034+0000 7f63de4dd700 1 -- 172.21.15.40:0/3434495597 >> 172.21.15.40:0/3434495597 conn(0x7f63d806e4f0 msgr2=0x7f63d80c19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:12.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.034+0000 7f63de4dd700 1 -- 172.21.15.40:0/3434495597 shutdown_connections 2024-09-13T17:51:12.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.034+0000 7f63de4dd700 1 -- 172.21.15.40:0/3434495597 wait complete. 2024-09-13T17:51:12.050 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:51:12.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.158+0000 7f94c4f75700 1 Processor -- start 2024-09-13T17:51:12.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.158+0000 7f94c4f75700 1 -- start start 2024-09-13T17:51:12.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.158+0000 7f94c4f75700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f94c0151750 0x7f94c0157f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:12.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.158+0000 7f94c4f75700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f94c006a950 0x7f94c006ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:12.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.158+0000 7f94c4f75700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c0158440 con 0x7f94c00b0d70 2024-09-13T17:51:12.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.158+0000 7f94c4f75700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c006a4c0 con 0x7f94c006a950 2024-09-13T17:51:12.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.158+0000 7f94c4f75700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c006b320 con 0x7f94c0151750 2024-09-13T17:51:12.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.158+0000 7f94bdd9b700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f94c00b0d70 0x7f94c006fbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59866/0 (socket says 172.21.15.40:59866) 2024-09-13T17:51:12.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.158+0000 7f94bdd9b700 1 -- 172.21.15.40:0/2109679676 learned_addr learned my addr 172.21.15.40:0/2109679676 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:12.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.158+0000 7f94bed9d700 1 --2- 172.21.15.40:0/2109679676 >> v2:172.21.15.40:3300/0 conn(0x7f94c006a950 0x7f94c006ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:12.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.158+0000 7f94be59c700 1 --2- 172.21.15.40:0/2109679676 >> v2:172.21.15.43:3300/0 conn(0x7f94c0151750 0x7f94c0157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:12.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.158+0000 7f94bd59a700 1 -- 172.21.15.40:0/2109679676 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474562605 0 0) 0x7f94a8001490 con 0x7f94c00b0d70 2024-09-13T17:51:12.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.158+0000 7f94bd59a700 1 -- 172.21.15.40:0/2109679676 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94a00035d0 con 0x7f94c00b0d70 2024-09-13T17:51:12.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.158+0000 7f94be59c700 1 -- 172.21.15.40:0/2109679676 >> v2:172.21.15.40:3300/0 conn(0x7f94c006a950 msgr2=0x7f94c006ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:12.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.158+0000 7f94be59c700 1 --2- 172.21.15.40:0/2109679676 >> v2:172.21.15.40:3300/0 conn(0x7f94c006a950 0x7f94c006ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.158+0000 7f94be59c700 1 -- 172.21.15.40:0/2109679676 >> v1:172.21.15.40:6789/0 conn(0x7f94c00b0d70 legacy=0x7f94c006fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:12.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.158+0000 7f94be59c700 1 -- 172.21.15.40:0/2109679676 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c006cf60 con 0x7f94c0151750 2024-09-13T17:51:12.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.158+0000 7f94bed9d700 1 --2- 172.21.15.40:0/2109679676 >> v2:172.21.15.40:3300/0 conn(0x7f94c006a950 0x7f94c006ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.162+0000 7f94be59c700 1 --2- 172.21.15.40:0/2109679676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94c0151750 0x7f94c0157f00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f94c006f7c0 tx=0x7f94b8009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dbf4c9e7d83aeab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.162+0000 7f94bd59a700 1 -- 172.21.15.40:0/2109679676 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b8011830 con 0x7f94c0151750 2024-09-13T17:51:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.162+0000 7f94bd59a700 1 -- 172.21.15.40:0/2109679676 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b8011990 con 0x7f94c0151750 2024-09-13T17:51:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.162+0000 7f94bd59a700 1 -- 172.21.15.40:0/2109679676 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b8011d20 con 0x7f94c0151750 2024-09-13T17:51:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.162+0000 7f94c4f75700 1 -- 172.21.15.40:0/2109679676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94c0151750 msgr2=0x7f94c0157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.162+0000 7f94c4f75700 1 --2- 172.21.15.40:0/2109679676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94c0151750 0x7f94c0157f00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f94c006f7c0 tx=0x7f94b8009b30 comp rx=0 tx=0).stop 2024-09-13T17:51:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.162+0000 7f94c4f75700 1 -- 172.21.15.40:0/2109679676 shutdown_connections 2024-09-13T17:51:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.162+0000 7f94c4f75700 1 --2- 172.21.15.40:0/2109679676 >> v2:172.21.15.40:3300/0 conn(0x7f94c006a950 0x7f94c006ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.162+0000 7f94c4f75700 1 --2- 172.21.15.40:0/2109679676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94c0151750 0x7f94c0157f00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.162+0000 7f94c4f75700 1 -- 172.21.15.40:0/2109679676 >> 172.21.15.40:0/2109679676 conn(0x7f94c006d5b0 msgr2=0x7f94c006d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.162+0000 7f94c4f75700 1 -- 172.21.15.40:0/2109679676 shutdown_connections 2024-09-13T17:51:12.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.162+0000 7f94c4f75700 1 -- 172.21.15.40:0/2109679676 wait complete. 2024-09-13T17:51:12.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.162+0000 7f94c4f75700 1 Processor -- start 2024-09-13T17:51:12.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.162+0000 7f94c4f75700 1 -- start start 2024-09-13T17:51:12.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94c4f75700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94c006a950 0x7f94c00ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:12.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94c4f75700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f94c00b0d70 0x7f94c01e62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:12.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94c4f75700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94c0151750 0x7f94c0149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:12.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94c4f75700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c0152b80 con 0x7f94c0151750 2024-09-13T17:51:12.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94c4f75700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c00c9e10 con 0x7f94c006a950 2024-09-13T17:51:12.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94c4f75700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c01536f0 con 0x7f94c00b0d70 2024-09-13T17:51:12.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94bed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94c0151750 0x7f94c0149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:12.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94bed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94c0151750 0x7f94c0149840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51098/0 (socket says 172.21.15.40:51098) 2024-09-13T17:51:12.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94bed9d700 1 -- 172.21.15.40:0/2245145500 learned_addr learned my addr 172.21.15.40:0/2245145500 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:12.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94be59c700 1 --2- 172.21.15.40:0/2245145500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94c006a950 0x7f94c00ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:12.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94bdd9b700 1 --2- 172.21.15.40:0/2245145500 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f94c00b0d70 0x7f94c01e62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:12.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94bed9d700 1 -- 172.21.15.40:0/2245145500 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f94c00b0d70 msgr2=0x7f94c01e62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:12.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94bed9d700 1 --2- 172.21.15.40:0/2245145500 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f94c00b0d70 0x7f94c01e62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94bed9d700 1 -- 172.21.15.40:0/2245145500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94c006a950 msgr2=0x7f94c00ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:12.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94bed9d700 1 --2- 172.21.15.40:0/2245145500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94c006a950 0x7f94c00ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94bed9d700 1 -- 172.21.15.40:0/2245145500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b00094b0 con 0x7f94c0151750 2024-09-13T17:51:12.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94bed9d700 1 --2- 172.21.15.40:0/2245145500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94c0151750 0x7f94c0149840 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f94b000e960 tx=0x7f94b000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:12.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94b6ffd700 1 -- 172.21.15.40:0/2245145500 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0009710 con 0x7f94c0151750 2024-09-13T17:51:12.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94c4f75700 1 -- 172.21.15.40:0/2245145500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b80094b0 con 0x7f94c0151750 2024-09-13T17:51:12.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94b6ffd700 1 -- 172.21.15.40:0/2245145500 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b0004020 con 0x7f94c0151750 2024-09-13T17:51:12.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94b6ffd700 1 -- 172.21.15.40:0/2245145500 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b00043b0 con 0x7f94c0151750 2024-09-13T17:51:12.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.166+0000 7f94c4f75700 1 -- 172.21.15.40:0/2245145500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c01521e0 con 0x7f94c0151750 2024-09-13T17:51:12.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.170+0000 7f94c4f75700 1 -- 172.21.15.40:0/2245145500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c0071800 con 0x7f94c0151750 2024-09-13T17:51:12.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.170+0000 7f94b6ffd700 1 -- 172.21.15.40:0/2245145500 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f94b00036c0 con 0x7f94c0151750 2024-09-13T17:51:12.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.170+0000 7f94b6ffd700 1 --2- 172.21.15.40:0/2245145500 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f94a4066730 0x7f94a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:12.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.170+0000 7f94be59c700 1 --2- 172.21.15.40:0/2245145500 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f94a4066730 0x7f94a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:12.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.170+0000 7f94b6ffd700 1 -- 172.21.15.40:0/2245145500 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f94b00841b0 con 0x7f94c0151750 2024-09-13T17:51:12.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.170+0000 7f94be59c700 1 --2- 172.21.15.40:0/2245145500 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f94a4066730 0x7f94a4068bf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f94b8000c00 tx=0x7f94b8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:12.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.178+0000 7f94b6ffd700 1 -- 172.21.15.40:0/2245145500 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f94b0051120 con 0x7f94c0151750 2024-09-13T17:51:12.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.350+0000 7f94c4f75700 1 -- 172.21.15.40:0/2245145500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f94c00cac40 con 0x7f94c0151750 2024-09-13T17:51:12.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.354+0000 7f94b6ffd700 1 -- 172.21.15.40:0/2245145500 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f94b0054740 con 0x7f94c0151750 2024-09-13T17:51:12.358 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-09-13T17:51:12.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.354+0000 7f94c4f75700 1 -- 172.21.15.40:0/2245145500 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f94a4066730 msgr2=0x7f94a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:12.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.354+0000 7f94c4f75700 1 --2- 172.21.15.40:0/2245145500 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f94a4066730 0x7f94a4068bf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f94b8000c00 tx=0x7f94b8008030 comp rx=0 tx=0).stop 2024-09-13T17:51:12.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.354+0000 7f94c4f75700 1 -- 172.21.15.40:0/2245145500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94c0151750 msgr2=0x7f94c0149840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:12.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.354+0000 7f94c4f75700 1 --2- 172.21.15.40:0/2245145500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94c0151750 0x7f94c0149840 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f94b000e960 tx=0x7f94b000ec70 comp rx=0 tx=0).stop 2024-09-13T17:51:12.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.354+0000 7f94c4f75700 1 -- 172.21.15.40:0/2245145500 shutdown_connections 2024-09-13T17:51:12.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.354+0000 7f94c4f75700 1 --2- 172.21.15.40:0/2245145500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94c0151750 0x7f94c0149840 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.354+0000 7f94c4f75700 1 --2- 172.21.15.40:0/2245145500 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f94a4066730 0x7f94a4068bf0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.354+0000 7f94c4f75700 1 --2- 172.21.15.40:0/2245145500 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f94c00b0d70 0x7f94c01e62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.354+0000 7f94c4f75700 1 --2- 172.21.15.40:0/2245145500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94c006a950 0x7f94c00ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.354+0000 7f94c4f75700 1 -- 172.21.15.40:0/2245145500 >> 172.21.15.40:0/2245145500 conn(0x7f94c006d5b0 msgr2=0x7f94c0071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:12.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.354+0000 7f94c4f75700 1 -- 172.21.15.40:0/2245145500 shutdown_connections 2024-09-13T17:51:12.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.354+0000 7f94c4f75700 1 -- 172.21.15.40:0/2245145500 wait complete. 2024-09-13T17:51:12.373 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:51:12.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.490+0000 7fb680883700 1 Processor -- start 2024-09-13T17:51:12.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.490+0000 7fb680883700 1 -- start start 2024-09-13T17:51:12.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.490+0000 7fb680883700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb67c0c8340 0x7fb67c0c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:12.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.490+0000 7fb680883700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb67c0c7250 0x7fb67c1577f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:12.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.490+0000 7fb680883700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67c06b400 con 0x7fb67c0c7f60 2024-09-13T17:51:12.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.490+0000 7fb680883700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb67c15e920 con 0x7fb67c0c7250 2024-09-13T17:51:12.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.490+0000 7fb680883700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0c76a0 con 0x7fb67c0c8340 2024-09-13T17:51:12.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.490+0000 7fb67bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb67c0c7250 0x7fb67c1577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:12.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.490+0000 7fb67bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb67c0c7250 0x7fb67c1577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51102/0 (socket says 172.21.15.40:51102) 2024-09-13T17:51:12.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.490+0000 7fb67bfff700 1 -- 172.21.15.40:0/388402743 learned_addr learned my addr 172.21.15.40:0/388402743 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:12.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.490+0000 7fb67affd700 1 --2- 172.21.15.40:0/388402743 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb67c0c8340 0x7fb67c0c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:12.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.490+0000 7fb67bfff700 1 -- 172.21.15.40:0/388402743 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb67c0c8340 msgr2=0x7fb67c0c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:12.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.490+0000 7fb67bfff700 1 --2- 172.21.15.40:0/388402743 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb67c0c8340 0x7fb67c0c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.490+0000 7fb67bfff700 1 -- 172.21.15.40:0/388402743 >> v1:172.21.15.43:6789/0 conn(0x7fb67c0c7f60 legacy=0x7fb67c06d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:12.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.490+0000 7fb67bfff700 1 -- 172.21.15.40:0/388402743 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c15dda0 con 0x7fb67c0c7250 2024-09-13T17:51:12.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.490+0000 7fb67affd700 1 --2- 172.21.15.40:0/388402743 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb67c0c8340 0x7fb67c0c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:12.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.490+0000 7fb67bfff700 1 --2- 172.21.15.40:0/388402743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c0c7250 0x7fb67c1577f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb674009e50 tx=0x7fb67400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d980f9775ef3b2a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:12.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.490+0000 7fb67a7fc700 1 -- 172.21.15.40:0/388402743 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67400e070 con 0x7fb67c0c7250 2024-09-13T17:51:12.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.494+0000 7fb67a7fc700 1 -- 172.21.15.40:0/388402743 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67400b9c0 con 0x7fb67c0c7250 2024-09-13T17:51:12.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.494+0000 7fb67a7fc700 1 -- 172.21.15.40:0/388402743 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67400bd70 con 0x7fb67c0c7250 2024-09-13T17:51:12.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.494+0000 7fb680883700 1 -- 172.21.15.40:0/388402743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c0c7250 msgr2=0x7fb67c1577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:12.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.494+0000 7fb680883700 1 --2- 172.21.15.40:0/388402743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c0c7250 0x7fb67c1577f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb674009e50 tx=0x7fb67400b330 comp rx=0 tx=0).stop 2024-09-13T17:51:12.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.494+0000 7fb680883700 1 -- 172.21.15.40:0/388402743 shutdown_connections 2024-09-13T17:51:12.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.494+0000 7fb680883700 1 --2- 172.21.15.40:0/388402743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c0c7250 0x7fb67c1577f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.494+0000 7fb680883700 1 --2- 172.21.15.40:0/388402743 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb67c0c8340 0x7fb67c0c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.494+0000 7fb680883700 1 -- 172.21.15.40:0/388402743 >> 172.21.15.40:0/388402743 conn(0x7fb67c066090 msgr2=0x7fb67c0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:12.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.494+0000 7fb680883700 1 -- 172.21.15.40:0/388402743 shutdown_connections 2024-09-13T17:51:12.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.494+0000 7fb680883700 1 -- 172.21.15.40:0/388402743 wait complete. 2024-09-13T17:51:12.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.494+0000 7fb680883700 1 Processor -- start 2024-09-13T17:51:12.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.494+0000 7fb680883700 1 -- start start 2024-09-13T17:51:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.494+0000 7fb680883700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c0c7250 0x7fb67c0c2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.494+0000 7fb680883700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67c0c7f60 0x7fb67c157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.494+0000 7fb680883700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb67c0c8340 0x7fb67c0ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.494+0000 7fb680883700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c1f45a0 con 0x7fb67c0c7250 2024-09-13T17:51:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.494+0000 7fb680883700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0ca110 con 0x7fb67c0c7f60 2024-09-13T17:51:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.494+0000 7fb680883700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0c9da0 con 0x7fb67c0c8340 2024-09-13T17:51:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.498+0000 7fb67b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c0c7250 0x7fb67c0c2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.498+0000 7fb67b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c0c7250 0x7fb67c0c2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51110/0 (socket says 172.21.15.40:51110) 2024-09-13T17:51:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.498+0000 7fb67b7fe700 1 -- 172.21.15.40:0/547047929 learned_addr learned my addr 172.21.15.40:0/547047929 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.498+0000 7fb67bfff700 1 --2- 172.21.15.40:0/547047929 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb67c0c8340 0x7fb67c0ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.498+0000 7fb67affd700 1 --2- 172.21.15.40:0/547047929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67c0c7f60 0x7fb67c157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.498+0000 7fb67b7fe700 1 -- 172.21.15.40:0/547047929 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb67c0c8340 msgr2=0x7fb67c0ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.498+0000 7fb67b7fe700 1 --2- 172.21.15.40:0/547047929 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb67c0c8340 0x7fb67c0ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.498+0000 7fb67b7fe700 1 -- 172.21.15.40:0/547047929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67c0c7f60 msgr2=0x7fb67c157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.498+0000 7fb67b7fe700 1 --2- 172.21.15.40:0/547047929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67c0c7f60 0x7fb67c157830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.498+0000 7fb67b7fe700 1 -- 172.21.15.40:0/547047929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb674009b30 con 0x7fb67c0c7250 2024-09-13T17:51:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.498+0000 7fb67b7fe700 1 --2- 172.21.15.40:0/547047929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c0c7250 0x7fb67c0c2510 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb66400c560 tx=0x7fb66400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:12.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.498+0000 7fb673fff700 1 -- 172.21.15.40:0/547047929 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6640052d0 con 0x7fb67c0c7250 2024-09-13T17:51:12.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.498+0000 7fb673fff700 1 -- 172.21.15.40:0/547047929 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb664005430 con 0x7fb67c0c7250 2024-09-13T17:51:12.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.498+0000 7fb680883700 1 -- 172.21.15.40:0/547047929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67c159830 con 0x7fb67c0c7250 2024-09-13T17:51:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.498+0000 7fb673fff700 1 -- 172.21.15.40:0/547047929 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6640055a0 con 0x7fb67c0c7250 2024-09-13T17:51:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.498+0000 7fb680883700 1 -- 172.21.15.40:0/547047929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67c0c21d0 con 0x7fb67c0c7250 2024-09-13T17:51:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.498+0000 7fb680883700 1 -- 172.21.15.40:0/547047929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67c072a10 con 0x7fb67c0c7250 2024-09-13T17:51:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.498+0000 7fb673fff700 1 -- 172.21.15.40:0/547047929 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb6640038e0 con 0x7fb67c0c7250 2024-09-13T17:51:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.498+0000 7fb673fff700 1 --2- 172.21.15.40:0/547047929 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb65c066760 0x7fb65c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:12.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.502+0000 7fb673fff700 1 -- 172.21.15.40:0/547047929 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb6640837f0 con 0x7fb67c0c7250 2024-09-13T17:51:12.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.502+0000 7fb67affd700 1 --2- 172.21.15.40:0/547047929 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb65c066760 0x7fb65c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:12.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.502+0000 7fb67affd700 1 --2- 172.21.15.40:0/547047929 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb65c066760 0x7fb65c068c20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb67c0c5360 tx=0x7fb66c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:12.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.506+0000 7fb673fff700 1 -- 172.21.15.40:0/547047929 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb664050870 con 0x7fb67c0c7250 2024-09-13T17:51:12.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.674+0000 7fb680883700 1 -- 172.21.15.40:0/547047929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb67c15d9d0 con 0x7fb67c0c7250 2024-09-13T17:51:12.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.674+0000 7fb673fff700 1 -- 172.21.15.40:0/547047929 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb664053e90 con 0x7fb67c0c7250 2024-09-13T17:51:12.679 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-09-13T17:51:12.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.678+0000 7fb680883700 1 -- 172.21.15.40:0/547047929 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb65c066760 msgr2=0x7fb65c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:12.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.678+0000 7fb680883700 1 --2- 172.21.15.40:0/547047929 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb65c066760 0x7fb65c068c20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb67c0c5360 tx=0x7fb66c008040 comp rx=0 tx=0).stop 2024-09-13T17:51:12.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.678+0000 7fb680883700 1 -- 172.21.15.40:0/547047929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c0c7250 msgr2=0x7fb67c0c2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:12.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.678+0000 7fb680883700 1 --2- 172.21.15.40:0/547047929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c0c7250 0x7fb67c0c2510 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb66400c560 tx=0x7fb66400c870 comp rx=0 tx=0).stop 2024-09-13T17:51:12.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.678+0000 7fb680883700 1 -- 172.21.15.40:0/547047929 shutdown_connections 2024-09-13T17:51:12.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.678+0000 7fb680883700 1 --2- 172.21.15.40:0/547047929 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb65c066760 0x7fb65c068c20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.678+0000 7fb680883700 1 --2- 172.21.15.40:0/547047929 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb67c0c8340 0x7fb67c0ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.678+0000 7fb680883700 1 --2- 172.21.15.40:0/547047929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67c0c7f60 0x7fb67c157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.678+0000 7fb680883700 1 --2- 172.21.15.40:0/547047929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c0c7250 0x7fb67c0c2510 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.678+0000 7fb680883700 1 -- 172.21.15.40:0/547047929 >> 172.21.15.40:0/547047929 conn(0x7fb67c066090 msgr2=0x7fb67c071ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:12.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.678+0000 7fb680883700 1 -- 172.21.15.40:0/547047929 shutdown_connections 2024-09-13T17:51:12.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.678+0000 7fb680883700 1 -- 172.21.15.40:0/547047929 wait complete. 2024-09-13T17:51:12.699 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:51:12.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.806+0000 7fb1ad7bd700 1 Processor -- start 2024-09-13T17:51:12.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.806+0000 7fb1ad7bd700 1 -- start start 2024-09-13T17:51:12.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.806+0000 7fb1ad7bd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb1a8154240 0x7fb1a815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:12.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.806+0000 7fb1ad7bd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb1a815ec60 0x7fb1a815f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:12.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.806+0000 7fb1ad7bd700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a8071690 con 0x7fb1a806a390 2024-09-13T17:51:12.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.806+0000 7fb1ad7bd700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a80721e0 con 0x7fb1a8154240 2024-09-13T17:51:12.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.806+0000 7fb1ad7bd700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a815e6e0 con 0x7fb1a815ec60 2024-09-13T17:51:12.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.806+0000 7fb1a77fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb1a815ec60 0x7fb1a815f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:12.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.806+0000 7fb1a7fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb1a8154240 0x7fb1a815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:12.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.806+0000 7fb1a77fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb1a815ec60 0x7fb1a815f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:36854/0 (socket says 172.21.15.40:36854) 2024-09-13T17:51:12.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.806+0000 7fb1a77fe700 1 -- 172.21.15.40:0/822096329 learned_addr learned my addr 172.21.15.40:0/822096329 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:12.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.810+0000 7fb1a77fe700 1 -- 172.21.15.40:0/822096329 >> v2:172.21.15.43:3300/0 conn(0x7fb1a8154240 msgr2=0x7fb1a815e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:12.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.810+0000 7fb1a77fe700 1 --2- 172.21.15.40:0/822096329 >> v2:172.21.15.43:3300/0 conn(0x7fb1a8154240 0x7fb1a815e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.810+0000 7fb1a77fe700 1 -- 172.21.15.40:0/822096329 >> v1:172.21.15.43:6789/0 conn(0x7fb1a806a390 legacy=0x7fb1a806a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:12.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.810+0000 7fb1a77fe700 1 -- 172.21.15.40:0/822096329 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a8154620 con 0x7fb1a815ec60 2024-09-13T17:51:12.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.810+0000 7fb1a77fe700 1 --2- 172.21.15.40:0/822096329 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb1a815ec60 0x7fb1a815f0a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb198009910 tx=0x7fb198009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ac7738aedca854 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:12.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.810+0000 7fb1a6ffd700 1 -- 172.21.15.40:0/822096329 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb198011830 con 0x7fb1a815ec60 2024-09-13T17:51:12.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.810+0000 7fb1a6ffd700 1 -- 172.21.15.40:0/822096329 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb198011990 con 0x7fb1a815ec60 2024-09-13T17:51:12.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.810+0000 7fb1a6ffd700 1 -- 172.21.15.40:0/822096329 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb198011d40 con 0x7fb1a815ec60 2024-09-13T17:51:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.810+0000 7fb1ad7bd700 1 -- 172.21.15.40:0/822096329 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb1a815ec60 msgr2=0x7fb1a815f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.810+0000 7fb1ad7bd700 1 --2- 172.21.15.40:0/822096329 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb1a815ec60 0x7fb1a815f0a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb198009910 tx=0x7fb198009c20 comp rx=0 tx=0).stop 2024-09-13T17:51:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.810+0000 7fb1ad7bd700 1 -- 172.21.15.40:0/822096329 shutdown_connections 2024-09-13T17:51:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.810+0000 7fb1ad7bd700 1 --2- 172.21.15.40:0/822096329 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb1a815ec60 0x7fb1a815f0a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.810+0000 7fb1ad7bd700 1 --2- 172.21.15.40:0/822096329 >> v2:172.21.15.43:3300/0 conn(0x7fb1a8154240 0x7fb1a815e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.810+0000 7fb1ad7bd700 1 -- 172.21.15.40:0/822096329 >> 172.21.15.40:0/822096329 conn(0x7fb1a80cabc0 msgr2=0x7fb1a80cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.810+0000 7fb1ad7bd700 1 -- 172.21.15.40:0/822096329 shutdown_connections 2024-09-13T17:51:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.810+0000 7fb1ad7bd700 1 -- 172.21.15.40:0/822096329 wait complete. 2024-09-13T17:51:12.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.810+0000 7fb1ad7bd700 1 Processor -- start 2024-09-13T17:51:12.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.810+0000 7fb1ad7bd700 1 -- start start 2024-09-13T17:51:12.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1ad7bd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1a806a390 0x7fb1a80c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:12.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1ad7bd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb1a8154240 0x7fb1a80c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:12.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1ad7bd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1a815ec60 0x7fb1a81f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:12.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1ad7bd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a81f3690 con 0x7fb1a815ec60 2024-09-13T17:51:12.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1ad7bd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a814e500 con 0x7fb1a806a390 2024-09-13T17:51:12.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1ad7bd700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a814e750 con 0x7fb1a8154240 2024-09-13T17:51:12.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1acfbc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1a815ec60 0x7fb1a81f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:12.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1a77fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb1a8154240 0x7fb1a80c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:12.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1a7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1a806a390 0x7fb1a80c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:12.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1a77fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb1a8154240 0x7fb1a80c9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:36870/0 (socket says 172.21.15.40:36870) 2024-09-13T17:51:12.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1acfbc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1a815ec60 0x7fb1a81f3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51118/0 (socket says 172.21.15.40:51118) 2024-09-13T17:51:12.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1a77fe700 1 -- 172.21.15.40:0/1608360325 learned_addr learned my addr 172.21.15.40:0/1608360325 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:12.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1a77fe700 1 -- 172.21.15.40:0/1608360325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1a806a390 msgr2=0x7fb1a80c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:12.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1a77fe700 1 --2- 172.21.15.40:0/1608360325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1a806a390 0x7fb1a80c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1a77fe700 1 -- 172.21.15.40:0/1608360325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1a815ec60 msgr2=0x7fb1a81f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1a77fe700 1 --2- 172.21.15.40:0/1608360325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1a815ec60 0x7fb1a81f3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1a77fe700 1 -- 172.21.15.40:0/1608360325 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb198009570 con 0x7fb1a8154240 2024-09-13T17:51:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1acfbc700 1 --2- 172.21.15.40:0/1608360325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1a815ec60 0x7fb1a81f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1a77fe700 1 --2- 172.21.15.40:0/1608360325 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb1a8154240 0x7fb1a80c9ba0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb1a815e820 tx=0x7fb198004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1a4ff9700 1 -- 172.21.15.40:0/1608360325 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19801a8f0 con 0x7fb1a8154240 2024-09-13T17:51:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1a4ff9700 1 -- 172.21.15.40:0/1608360325 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19801e070 con 0x7fb1a8154240 2024-09-13T17:51:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1ad7bd700 1 -- 172.21.15.40:0/1608360325 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a814ea30 con 0x7fb1a8154240 2024-09-13T17:51:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1a4ff9700 1 -- 172.21.15.40:0/1608360325 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19801ac70 con 0x7fb1a8154240 2024-09-13T17:51:12.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.814+0000 7fb1ad7bd700 1 -- 172.21.15.40:0/1608360325 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a814eff0 con 0x7fb1a8154240 2024-09-13T17:51:12.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.818+0000 7fb1ad7bd700 1 -- 172.21.15.40:0/1608360325 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a8071040 con 0x7fb1a8154240 2024-09-13T17:51:12.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.818+0000 7fb1a4ff9700 1 -- 172.21.15.40:0/1608360325 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb19802d450 con 0x7fb1a8154240 2024-09-13T17:51:12.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.818+0000 7fb1a4ff9700 1 --2- 172.21.15.40:0/1608360325 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb188066730 0x7fb188068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:12.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.818+0000 7fb1a4ff9700 1 -- 172.21.15.40:0/1608360325 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb1980852f0 con 0x7fb1a8154240 2024-09-13T17:51:12.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.818+0000 7fb1a7fff700 1 --2- 172.21.15.40:0/1608360325 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb188066730 0x7fb188068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:12.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.818+0000 7fb1a7fff700 1 --2- 172.21.15.40:0/1608360325 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb188066730 0x7fb188068bf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb190009bf0 tx=0x7fb190009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:12.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:12.826+0000 7fb1a4ff9700 1 -- 172.21.15.40:0/1608360325 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb198052400 con 0x7fb1a8154240 2024-09-13T17:51:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.034+0000 7fb1ad7bd700 1 -- 172.21.15.40:0/1608360325 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1a806b370 con 0x7fb1a8154240 2024-09-13T17:51:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.038+0000 7fb1a4ff9700 1 -- 172.21.15.40:0/1608360325 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6268 (secure 0 0 0) 0x7fb198055a20 con 0x7fb1a8154240 2024-09-13T17:51:13.042 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:13.042 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6839/1861594206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1861594206},{"type":"v1","addr":"172.21.15.40:6839","nonce":1861594206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1548394691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1548394691},{"type":"v1","addr":"172.21.15.40:6835","nonce":1548394691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/486611242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":486611242},{"type":"v1","addr":"172.21.15.40:6837","nonce":486611242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:50:34.562623+0000","modified":"2024-09-13T17:51:12.236489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4288,"mds_1":4285,"mds_2":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"f","rank":2,"incarnation":14,"state":"up:active","state_seq":12,"addr":"172.21.15.43:6833/526975022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":526975022},{"type":"v1","addr":"172.21.15.43:6833","nonce":526975022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4285":{"gid":4285,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.43:6835/2586910944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2586910944},{"type":"v1","addr":"172.21.15.43:6835","nonce":2586910944}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/334160033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":334160033},{"type":"v1","addr":"172.21.15.43:6837","nonce":334160033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:51:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.042+0000 7fb1ad7bd700 1 -- 172.21.15.40:0/1608360325 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb188066730 msgr2=0x7fb188068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.042+0000 7fb1ad7bd700 1 --2- 172.21.15.40:0/1608360325 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb188066730 0x7fb188068bf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb190009bf0 tx=0x7fb190009240 comp rx=0 tx=0).stop 2024-09-13T17:51:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.042+0000 7fb1ad7bd700 1 -- 172.21.15.40:0/1608360325 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb1a8154240 msgr2=0x7fb1a80c9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.042+0000 7fb1ad7bd700 1 --2- 172.21.15.40:0/1608360325 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb1a8154240 0x7fb1a80c9ba0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb1a815e820 tx=0x7fb198004770 comp rx=0 tx=0).stop 2024-09-13T17:51:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.042+0000 7fb1ad7bd700 1 -- 172.21.15.40:0/1608360325 shutdown_connections 2024-09-13T17:51:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.042+0000 7fb1ad7bd700 1 --2- 172.21.15.40:0/1608360325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1a815ec60 0x7fb1a81f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.042+0000 7fb1ad7bd700 1 --2- 172.21.15.40:0/1608360325 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb188066730 0x7fb188068bf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.042+0000 7fb1ad7bd700 1 --2- 172.21.15.40:0/1608360325 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb1a8154240 0x7fb1a80c9ba0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.042+0000 7fb1ad7bd700 1 --2- 172.21.15.40:0/1608360325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1a806a390 0x7fb1a80c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.042+0000 7fb1ad7bd700 1 -- 172.21.15.40:0/1608360325 >> 172.21.15.40:0/1608360325 conn(0x7fb1a80cabc0 msgr2=0x7fb1a80c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.042+0000 7fb1ad7bd700 1 -- 172.21.15.40:0/1608360325 shutdown_connections 2024-09-13T17:51:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.042+0000 7fb1ad7bd700 1 -- 172.21.15.40:0/1608360325 wait complete. 2024-09-13T17:51:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 15 2024-09-13T17:51:13.062 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:51:13.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.170+0000 7f7e6759e700 1 Processor -- start 2024-09-13T17:51:13.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.170+0000 7f7e6759e700 1 -- start start 2024-09-13T17:51:13.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6759e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7e600e6320 0x7f7e6004bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:13.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6759e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7e60045ce0 0x7f7e600460c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:13.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6759e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e60005b00 con 0x7f7e600e6700 2024-09-13T17:51:13.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6759e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e60035710 con 0x7f7e600e6320 2024-09-13T17:51:13.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6759e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e60046600 con 0x7f7e60045ce0 2024-09-13T17:51:13.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6659c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7e600e6320 0x7f7e6004bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:13.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6659c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7e600e6320 0x7f7e6004bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51128/0 (socket says 172.21.15.40:51128) 2024-09-13T17:51:13.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6659c700 1 -- 172.21.15.40:0/2479647483 learned_addr learned my addr 172.21.15.40:0/2479647483 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:13.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e65d9b700 1 --2- 172.21.15.40:0/2479647483 >> v2:172.21.15.43:3300/0 conn(0x7f7e60045ce0 0x7f7e600460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:13.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6659c700 1 -- 172.21.15.40:0/2479647483 >> v2:172.21.15.43:3300/0 conn(0x7f7e60045ce0 msgr2=0x7f7e600460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:13.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6659c700 1 --2- 172.21.15.40:0/2479647483 >> v2:172.21.15.43:3300/0 conn(0x7f7e60045ce0 0x7f7e600460c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6659c700 1 -- 172.21.15.40:0/2479647483 >> v1:172.21.15.43:6789/0 conn(0x7f7e600e6700 legacy=0x7f7e600dc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:13.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6659c700 1 -- 172.21.15.40:0/2479647483 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e600e3560 con 0x7f7e600e6320 2024-09-13T17:51:13.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6659c700 1 --2- 172.21.15.40:0/2479647483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e600e6320 0x7f7e6004bfd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000e260 tx=0x7f7e58009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84edbd11147aadcf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:13.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6559a700 1 -- 172.21.15.40:0/2479647483 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58011830 con 0x7f7e600e6320 2024-09-13T17:51:13.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6559a700 1 -- 172.21.15.40:0/2479647483 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e58011990 con 0x7f7e600e6320 2024-09-13T17:51:13.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6559a700 1 -- 172.21.15.40:0/2479647483 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58011d20 con 0x7f7e600e6320 2024-09-13T17:51:13.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6759e700 1 -- 172.21.15.40:0/2479647483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e600e6320 msgr2=0x7f7e6004bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:13.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6759e700 1 --2- 172.21.15.40:0/2479647483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e600e6320 0x7f7e6004bfd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000e260 tx=0x7f7e58009bf0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6759e700 1 -- 172.21.15.40:0/2479647483 shutdown_connections 2024-09-13T17:51:13.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6759e700 1 --2- 172.21.15.40:0/2479647483 >> v2:172.21.15.43:3300/0 conn(0x7f7e60045ce0 0x7f7e600460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6759e700 1 --2- 172.21.15.40:0/2479647483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e600e6320 0x7f7e6004bfd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6759e700 1 -- 172.21.15.40:0/2479647483 >> 172.21.15.40:0/2479647483 conn(0x7f7e6004fd00 msgr2=0x7f7e60050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:13.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6759e700 1 -- 172.21.15.40:0/2479647483 shutdown_connections 2024-09-13T17:51:13.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.174+0000 7f7e6759e700 1 -- 172.21.15.40:0/2479647483 wait complete. 2024-09-13T17:51:13.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e6759e700 1 Processor -- start 2024-09-13T17:51:13.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e6759e700 1 -- start start 2024-09-13T17:51:13.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e6759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e60045ce0 0x7f7e60047a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:13.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e6759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e600e6320 0x7f7e6017bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:13.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e6759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7e600e6700 0x7f7e600e2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:13.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e6759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e600d89f0 con 0x7f7e600e6320 2024-09-13T17:51:13.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e6759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e60032d90 con 0x7f7e60045ce0 2024-09-13T17:51:13.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e6759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e60048a90 con 0x7f7e600e6700 2024-09-13T17:51:13.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e65d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e600e6320 0x7f7e6017bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:13.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e65d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e600e6320 0x7f7e6017bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51134/0 (socket says 172.21.15.40:51134) 2024-09-13T17:51:13.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e65d9b700 1 -- 172.21.15.40:0/1332333468 learned_addr learned my addr 172.21.15.40:0/1332333468 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:13.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e6659c700 1 --2- 172.21.15.40:0/1332333468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e60045ce0 0x7f7e60047a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:13.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e66d9d700 1 --2- 172.21.15.40:0/1332333468 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7e600e6700 0x7f7e600e2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e65d9b700 1 -- 172.21.15.40:0/1332333468 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7e600e6700 msgr2=0x7f7e600e2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e65d9b700 1 --2- 172.21.15.40:0/1332333468 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7e600e6700 0x7f7e600e2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e65d9b700 1 -- 172.21.15.40:0/1332333468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e60045ce0 msgr2=0x7f7e60047a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e65d9b700 1 --2- 172.21.15.40:0/1332333468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e60045ce0 0x7f7e60047a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e65d9b700 1 -- 172.21.15.40:0/1332333468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e58009570 con 0x7f7e600e6320 2024-09-13T17:51:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e66d9d700 1 --2- 172.21.15.40:0/1332333468 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7e600e6700 0x7f7e600e2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:13.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e65d9b700 1 --2- 172.21.15.40:0/1332333468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e600e6320 0x7f7e6017bad0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000ea00 tx=0x7f7e5000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:13.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e5effd700 1 -- 172.21.15.40:0/1332333468 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e50009720 con 0x7f7e600e6320 2024-09-13T17:51:13.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e5effd700 1 -- 172.21.15.40:0/1332333468 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5000c7f0 con 0x7f7e600e6320 2024-09-13T17:51:13.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.178+0000 7f7e6759e700 1 -- 172.21.15.40:0/1332333468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e60047fb0 con 0x7f7e600e6320 2024-09-13T17:51:13.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.182+0000 7f7e5effd700 1 -- 172.21.15.40:0/1332333468 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000cb60 con 0x7f7e600e6320 2024-09-13T17:51:13.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.182+0000 7f7e6759e700 1 -- 172.21.15.40:0/1332333468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e60048520 con 0x7f7e600e6320 2024-09-13T17:51:13.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.182+0000 7f7e6759e700 1 -- 172.21.15.40:0/1332333468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e60045200 con 0x7f7e600e6320 2024-09-13T17:51:13.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.182+0000 7f7e5effd700 1 -- 172.21.15.40:0/1332333468 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7e5000ccc0 con 0x7f7e600e6320 2024-09-13T17:51:13.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.182+0000 7f7e5effd700 1 --2- 172.21.15.40:0/1332333468 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7e48066760 0x7f7e48068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:13.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.182+0000 7f7e6659c700 1 --2- 172.21.15.40:0/1332333468 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7e48066760 0x7f7e48068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:13.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.182+0000 7f7e5effd700 1 -- 172.21.15.40:0/1332333468 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7e50084710 con 0x7f7e600e6320 2024-09-13T17:51:13.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.186+0000 7f7e6659c700 1 --2- 172.21.15.40:0/1332333468 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7e48066760 0x7f7e48068c20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7e58000c00 tx=0x7f7e58008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:13.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.190+0000 7f7e5effd700 1 -- 172.21.15.40:0/1332333468 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7e50051820 con 0x7f7e600e6320 2024-09-13T17:51:13.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.366+0000 7f7e6759e700 1 -- 172.21.15.40:0/1332333468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e6004aea0 con 0x7f7e600e6320 2024-09-13T17:51:13.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.366+0000 7f7e5effd700 1 -- 172.21.15.40:0/1332333468 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18262 (secure 0 0 0) 0x7f7e50054e40 con 0x7f7e600e6320 2024-09-13T17:51:13.371 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:13.371 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":25,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:50:38.974071+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:50:32.740336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:50:33.552914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:50:36.623639+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:51:13.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.370+0000 7f7e6759e700 1 -- 172.21.15.40:0/1332333468 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7e48066760 msgr2=0x7f7e48068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:13.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.370+0000 7f7e6759e700 1 --2- 172.21.15.40:0/1332333468 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7e48066760 0x7f7e48068c20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7e58000c00 tx=0x7f7e58008030 comp rx=0 tx=0).stop 2024-09-13T17:51:13.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.370+0000 7f7e6759e700 1 -- 172.21.15.40:0/1332333468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e600e6320 msgr2=0x7f7e6017bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:13.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.370+0000 7f7e6759e700 1 --2- 172.21.15.40:0/1332333468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e600e6320 0x7f7e6017bad0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000ea00 tx=0x7f7e5000ed10 comp rx=0 tx=0).stop 2024-09-13T17:51:13.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.370+0000 7f7e6759e700 1 -- 172.21.15.40:0/1332333468 shutdown_connections 2024-09-13T17:51:13.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.370+0000 7f7e6759e700 1 --2- 172.21.15.40:0/1332333468 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7e48066760 0x7f7e48068c20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.370+0000 7f7e6759e700 1 --2- 172.21.15.40:0/1332333468 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7e600e6700 0x7f7e600e2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.370+0000 7f7e6759e700 1 --2- 172.21.15.40:0/1332333468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7e600e6320 0x7f7e6017bad0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.370+0000 7f7e6759e700 1 --2- 172.21.15.40:0/1332333468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e60045ce0 0x7f7e60047a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.370+0000 7f7e6759e700 1 -- 172.21.15.40:0/1332333468 >> 172.21.15.40:0/1332333468 conn(0x7f7e6004fd00 msgr2=0x7f7e6004d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:13.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.370+0000 7f7e6759e700 1 -- 172.21.15.40:0/1332333468 shutdown_connections 2024-09-13T17:51:13.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.370+0000 7f7e6759e700 1 -- 172.21.15.40:0/1332333468 wait complete. 2024-09-13T17:51:13.387 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:51:13.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.494+0000 7f6b5ceff700 1 Processor -- start 2024-09-13T17:51:13.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.494+0000 7f6b5ceff700 1 -- start start 2024-09-13T17:51:13.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.494+0000 7f6b5ceff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6b58071f70 0x7f6b580c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:13.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.494+0000 7f6b5ceff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6b58072350 0x7f6b58150d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:13.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.494+0000 7f6b5ceff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b580c55c0 con 0x7f6b580c7200 2024-09-13T17:51:13.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.494+0000 7f6b5ceff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b5806a390 con 0x7f6b58072350 2024-09-13T17:51:13.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.494+0000 7f6b5ceff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b58157710 con 0x7f6b58071f70 2024-09-13T17:51:13.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.494+0000 7f6b57fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6b580c7200 0x7f6b5815a700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59872/0 (socket says 172.21.15.40:59872) 2024-09-13T17:51:13.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.494+0000 7f6b57fff700 1 -- 172.21.15.40:0/1595465435 learned_addr learned my addr 172.21.15.40:0/1595465435 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:13.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.494+0000 7f6b56ffd700 1 --2- 172.21.15.40:0/1595465435 >> v2:172.21.15.43:3300/0 conn(0x7f6b58072350 0x7f6b58150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:13.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.494+0000 7f6b577fe700 1 --2- 172.21.15.40:0/1595465435 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6b58071f70 0x7f6b580c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:13.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.498+0000 7f6b567fc700 1 -- 172.21.15.40:0/1595465435 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912166612 0 0) 0x7f6b580c55c0 con 0x7f6b580c7200 2024-09-13T17:51:13.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.498+0000 7f6b567fc700 1 -- 172.21.15.40:0/1595465435 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b400035d0 con 0x7f6b580c7200 2024-09-13T17:51:13.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.498+0000 7f6b56ffd700 1 -- 172.21.15.40:0/1595465435 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6b58071f70 msgr2=0x7f6b580c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:13.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.498+0000 7f6b56ffd700 1 --2- 172.21.15.40:0/1595465435 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6b58071f70 0x7f6b580c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.498+0000 7f6b56ffd700 1 -- 172.21.15.40:0/1595465435 >> v1:172.21.15.40:6789/0 conn(0x7f6b580c7200 legacy=0x7f6b5815a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:13.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.498+0000 7f6b56ffd700 1 -- 172.21.15.40:0/1595465435 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b581579d0 con 0x7f6b58072350 2024-09-13T17:51:13.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.498+0000 7f6b577fe700 1 --2- 172.21.15.40:0/1595465435 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6b58071f70 0x7f6b580c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:13.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.498+0000 7f6b56ffd700 1 --2- 172.21.15.40:0/1595465435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b58072350 0x7f6b58150d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6b48009850 tx=0x7f6b48009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96c49e3db1d1de1e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:13.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.498+0000 7f6b567fc700 1 -- 172.21.15.40:0/1595465435 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b48011830 con 0x7f6b58072350 2024-09-13T17:51:13.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.498+0000 7f6b567fc700 1 -- 172.21.15.40:0/1595465435 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b48011990 con 0x7f6b58072350 2024-09-13T17:51:13.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.498+0000 7f6b567fc700 1 -- 172.21.15.40:0/1595465435 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b48011d00 con 0x7f6b58072350 2024-09-13T17:51:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.498+0000 7f6b5ceff700 1 -- 172.21.15.40:0/1595465435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b58072350 msgr2=0x7f6b58150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.498+0000 7f6b5ceff700 1 --2- 172.21.15.40:0/1595465435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b58072350 0x7f6b58150d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6b48009850 tx=0x7f6b48009b60 comp rx=0 tx=0).stop 2024-09-13T17:51:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.498+0000 7f6b5ceff700 1 -- 172.21.15.40:0/1595465435 shutdown_connections 2024-09-13T17:51:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.498+0000 7f6b5ceff700 1 --2- 172.21.15.40:0/1595465435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b58072350 0x7f6b58150d40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.498+0000 7f6b5ceff700 1 --2- 172.21.15.40:0/1595465435 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6b58071f70 0x7f6b580c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.498+0000 7f6b5ceff700 1 -- 172.21.15.40:0/1595465435 >> 172.21.15.40:0/1595465435 conn(0x7f6b580cabc0 msgr2=0x7f6b580cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.498+0000 7f6b5ceff700 1 -- 172.21.15.40:0/1595465435 shutdown_connections 2024-09-13T17:51:13.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.498+0000 7f6b5ceff700 1 -- 172.21.15.40:0/1595465435 wait complete. 2024-09-13T17:51:13.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b5ceff700 1 Processor -- start 2024-09-13T17:51:13.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b5ceff700 1 -- start start 2024-09-13T17:51:13.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b5ceff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6b58071f70 0x7f6b581ef050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:13.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b5ceff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b58072350 0x7f6b580c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:13.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b5ceff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b580c7200 0x7f6b581e18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:13.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b5ceff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b580c9e70 con 0x7f6b58072350 2024-09-13T17:51:13.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b5ceff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b581ef590 con 0x7f6b580c7200 2024-09-13T17:51:13.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b5ceff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b580ca5f0 con 0x7f6b58071f70 2024-09-13T17:51:13.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b56ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b58072350 0x7f6b580c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:13.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b56ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b58072350 0x7f6b580c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51150/0 (socket says 172.21.15.40:51150) 2024-09-13T17:51:13.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b577fe700 1 --2- 172.21.15.40:0/2665767356 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6b58071f70 0x7f6b581ef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:13.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b56ffd700 1 -- 172.21.15.40:0/2665767356 learned_addr learned my addr 172.21.15.40:0/2665767356 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:13.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b57fff700 1 --2- 172.21.15.40:0/2665767356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b580c7200 0x7f6b581e18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:13.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b56ffd700 1 -- 172.21.15.40:0/2665767356 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6b58071f70 msgr2=0x7f6b581ef050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:13.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b56ffd700 1 --2- 172.21.15.40:0/2665767356 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6b58071f70 0x7f6b581ef050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b56ffd700 1 -- 172.21.15.40:0/2665767356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b580c7200 msgr2=0x7f6b581e18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:13.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b56ffd700 1 --2- 172.21.15.40:0/2665767356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b580c7200 0x7f6b581e18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b56ffd700 1 -- 172.21.15.40:0/2665767356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c0094b0 con 0x7f6b58072350 2024-09-13T17:51:13.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b57fff700 1 --2- 172.21.15.40:0/2665767356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b580c7200 0x7f6b581e18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:13.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b56ffd700 1 --2- 172.21.15.40:0/2665767356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b58072350 0x7f6b580c8be0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6b5806f390 tx=0x7f6b48004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:13.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b3ffff700 1 -- 172.21.15.40:0/2665767356 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b480044d0 con 0x7f6b58072350 2024-09-13T17:51:13.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b3ffff700 1 -- 172.21.15.40:0/2665767356 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4801f070 con 0x7f6b58072350 2024-09-13T17:51:13.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b3ffff700 1 -- 172.21.15.40:0/2665767356 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b48003900 con 0x7f6b58072350 2024-09-13T17:51:13.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b5ceff700 1 -- 172.21.15.40:0/2665767356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b480094b0 con 0x7f6b58072350 2024-09-13T17:51:13.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.502+0000 7f6b5ceff700 1 -- 172.21.15.40:0/2665767356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b580ca310 con 0x7f6b58072350 2024-09-13T17:51:13.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.506+0000 7f6b5ceff700 1 -- 172.21.15.40:0/2665767356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b580cb6e0 con 0x7f6b58072350 2024-09-13T17:51:13.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.506+0000 7f6b3ffff700 1 -- 172.21.15.40:0/2665767356 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6b48003a60 con 0x7f6b58072350 2024-09-13T17:51:13.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.506+0000 7f6b3ffff700 1 --2- 172.21.15.40:0/2665767356 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6b44066730 0x7f6b44068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:13.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.506+0000 7f6b3ffff700 1 -- 172.21.15.40:0/2665767356 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6b48021070 con 0x7f6b58072350 2024-09-13T17:51:13.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.506+0000 7f6b577fe700 1 --2- 172.21.15.40:0/2665767356 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6b44066730 0x7f6b44068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:13.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.506+0000 7f6b577fe700 1 --2- 172.21.15.40:0/2665767356 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6b44066730 0x7f6b44068bf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c009d80 tx=0x7f6b4c009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:13.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.518+0000 7f6b3ffff700 1 -- 172.21.15.40:0/2665767356 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6b48052c60 con 0x7f6b58072350 2024-09-13T17:51:13.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.726+0000 7f6b5ceff700 1 -- 172.21.15.40:0/2665767356 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b580cb320 con 0x7f6b58072350 2024-09-13T17:51:13.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.726+0000 7f6b3ffff700 1 -- 172.21.15.40:0/2665767356 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6268 (secure 0 0 0) 0x7f6b4802b090 con 0x7f6b58072350 2024-09-13T17:51:13.731 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:13.731 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6839/1861594206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1861594206},{"type":"v1","addr":"172.21.15.40:6839","nonce":1861594206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1548394691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1548394691},{"type":"v1","addr":"172.21.15.40:6835","nonce":1548394691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/486611242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":486611242},{"type":"v1","addr":"172.21.15.40:6837","nonce":486611242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:50:34.562623+0000","modified":"2024-09-13T17:51:12.236489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4288,"mds_1":4285,"mds_2":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"f","rank":2,"incarnation":14,"state":"up:active","state_seq":12,"addr":"172.21.15.43:6833/526975022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":526975022},{"type":"v1","addr":"172.21.15.43:6833","nonce":526975022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4285":{"gid":4285,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.43:6835/2586910944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2586910944},{"type":"v1","addr":"172.21.15.43:6835","nonce":2586910944}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/334160033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":334160033},{"type":"v1","addr":"172.21.15.43:6837","nonce":334160033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:51:13.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.726+0000 7f6b5ceff700 1 -- 172.21.15.40:0/2665767356 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6b44066730 msgr2=0x7f6b44068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:13.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.726+0000 7f6b5ceff700 1 --2- 172.21.15.40:0/2665767356 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6b44066730 0x7f6b44068bf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c009d80 tx=0x7f6b4c009240 comp rx=0 tx=0).stop 2024-09-13T17:51:13.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.726+0000 7f6b5ceff700 1 -- 172.21.15.40:0/2665767356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b58072350 msgr2=0x7f6b580c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:13.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.726+0000 7f6b5ceff700 1 --2- 172.21.15.40:0/2665767356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b58072350 0x7f6b580c8be0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6b5806f390 tx=0x7f6b48004770 comp rx=0 tx=0).stop 2024-09-13T17:51:13.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.730+0000 7f6b5ceff700 1 -- 172.21.15.40:0/2665767356 shutdown_connections 2024-09-13T17:51:13.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.730+0000 7f6b5ceff700 1 --2- 172.21.15.40:0/2665767356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b580c7200 0x7f6b581e18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.730+0000 7f6b5ceff700 1 --2- 172.21.15.40:0/2665767356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b58072350 0x7f6b580c8be0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.730+0000 7f6b5ceff700 1 --2- 172.21.15.40:0/2665767356 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6b44066730 0x7f6b44068bf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.730+0000 7f6b5ceff700 1 --2- 172.21.15.40:0/2665767356 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6b58071f70 0x7f6b581ef050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:13.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.730+0000 7f6b5ceff700 1 -- 172.21.15.40:0/2665767356 >> 172.21.15.40:0/2665767356 conn(0x7f6b580cabc0 msgr2=0x7f6b580c5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:13.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.730+0000 7f6b5ceff700 1 -- 172.21.15.40:0/2665767356 shutdown_connections 2024-09-13T17:51:13.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:13.730+0000 7f6b5ceff700 1 -- 172.21.15.40:0/2665767356 wait complete. 2024-09-13T17:51:13.735 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 15 2024-09-13T17:51:14.150 INFO:tasks.ceph.mds.b.smithi043.stderr:2024-09-13T17:51:14.144+0000 7fb6d9c92700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-13T17:51:14.358 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T17:50:34.562623+0000', 'modified': '2024-09-13T17:51:12.236489+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4288, 'mds_1': 4285, 'mds_2': 4279}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4279': {'gid': 4279, 'name': 'f', 'rank': 2, 'incarnation': 14, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.43:6833/526975022', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6832', 'nonce': 526975022}, {'type': 'v1', 'addr': '172.21.15.43:6833', 'nonce': 526975022}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_4285': {'gid': 4285, 'name': 'd', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.43:6835/2586910944', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6834', 'nonce': 2586910944}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 2586910944}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_4288': {'gid': 4288, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.43:6837/334160033', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6836', 'nonce': 334160033}, {'type': 'v1', 'addr': '172.21.15.43:6837', 'nonce': 334160033}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T17:51:14.358 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-13T17:51:14.358 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-09-13T17:51:14.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.478+0000 7eff85639700 1 Processor -- start 2024-09-13T17:51:14.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.478+0000 7eff85639700 1 -- start start 2024-09-13T17:51:14.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.478+0000 7eff85639700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7eff800b0d70 0x7eff80157f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:14.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.478+0000 7eff85639700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff80158440 con 0x7eff8006a950 2024-09-13T17:51:14.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.478+0000 7eff85639700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff8006a4c0 con 0x7eff80151750 2024-09-13T17:51:14.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.478+0000 7eff85639700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff8006cf60 con 0x7eff800b0d70 2024-09-13T17:51:14.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.478+0000 7eff7f7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7eff8006a950 0x7eff8006ade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59874/0 (socket says 172.21.15.40:59874) 2024-09-13T17:51:14.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.478+0000 7eff7f7fe700 1 -- 172.21.15.40:0/3699399049 learned_addr learned my addr 172.21.15.40:0/3699399049 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:14.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.478+0000 7eff7e7fc700 1 --2- 172.21.15.40:0/3699399049 >> v2:172.21.15.43:3300/0 conn(0x7eff800b0d70 0x7eff80157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:14.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.478+0000 7eff7dffb700 1 -- 172.21.15.40:0/3699399049 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3555046416 0 0) 0x7eff78001110 con 0x7eff8006a950 2024-09-13T17:51:14.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.478+0000 7eff7dffb700 1 -- 172.21.15.40:0/3699399049 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff600035d0 con 0x7eff8006a950 2024-09-13T17:51:14.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.478+0000 7eff7e7fc700 1 -- 172.21.15.40:0/3699399049 >> v1:172.21.15.43:6789/0 conn(0x7eff80151750 legacy=0x7eff8006fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:14.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.478+0000 7eff7e7fc700 1 -- 172.21.15.40:0/3699399049 >> v1:172.21.15.40:6789/0 conn(0x7eff8006a950 legacy=0x7eff8006ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:14.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.482+0000 7eff7e7fc700 1 -- 172.21.15.40:0/3699399049 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff80070c80 con 0x7eff800b0d70 2024-09-13T17:51:14.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.482+0000 7eff7e7fc700 1 --2- 172.21.15.40:0/3699399049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff800b0d70 0x7eff80157f00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7eff70009d90 tx=0x7eff7000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db93588e5eebf669 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:14.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.482+0000 7eff7dffb700 1 -- 172.21.15.40:0/3699399049 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7000e070 con 0x7eff800b0d70 2024-09-13T17:51:14.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.482+0000 7eff7dffb700 1 -- 172.21.15.40:0/3699399049 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7000bb20 con 0x7eff800b0d70 2024-09-13T17:51:14.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.482+0000 7eff7dffb700 1 -- 172.21.15.40:0/3699399049 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff70012430 con 0x7eff800b0d70 2024-09-13T17:51:14.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.482+0000 7eff85639700 1 -- 172.21.15.40:0/3699399049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff800b0d70 msgr2=0x7eff80157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:14.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.482+0000 7eff85639700 1 --2- 172.21.15.40:0/3699399049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff800b0d70 0x7eff80157f00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7eff70009d90 tx=0x7eff7000b330 comp rx=0 tx=0).stop 2024-09-13T17:51:14.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.482+0000 7eff85639700 1 -- 172.21.15.40:0/3699399049 shutdown_connections 2024-09-13T17:51:14.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.482+0000 7eff85639700 1 --2- 172.21.15.40:0/3699399049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff800b0d70 0x7eff80157f00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:14.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.482+0000 7eff85639700 1 -- 172.21.15.40:0/3699399049 >> 172.21.15.40:0/3699399049 conn(0x7eff8006d5b0 msgr2=0x7eff8006d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:14.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.482+0000 7eff85639700 1 -- 172.21.15.40:0/3699399049 shutdown_connections 2024-09-13T17:51:14.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.482+0000 7eff85639700 1 -- 172.21.15.40:0/3699399049 wait complete. 2024-09-13T17:51:14.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.482+0000 7eff85639700 1 Processor -- start 2024-09-13T17:51:14.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.482+0000 7eff85639700 1 -- start start 2024-09-13T17:51:14.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff85639700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff8006a950 0x7eff800c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:14.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff85639700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7eff800b0d70 0x7eff801f05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:14.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff85639700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff80151750 0x7eff801408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:14.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff85639700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8014f160 con 0x7eff8006a950 2024-09-13T17:51:14.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff85639700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff80145e60 con 0x7eff80151750 2024-09-13T17:51:14.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff85639700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8014fb40 con 0x7eff800b0d70 2024-09-13T17:51:14.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff7effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff8006a950 0x7eff800c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:14.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff7effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff8006a950 0x7eff800c3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51154/0 (socket says 172.21.15.40:51154) 2024-09-13T17:51:14.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff7f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff80151750 0x7eff801408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:14.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff7effd700 1 -- 172.21.15.40:0/29129938 learned_addr learned my addr 172.21.15.40:0/29129938 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:14.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff7e7fc700 1 --2- 172.21.15.40:0/29129938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7eff800b0d70 0x7eff801f05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:14.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff7effd700 1 -- 172.21.15.40:0/29129938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7eff800b0d70 msgr2=0x7eff801f05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:14.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff7effd700 1 --2- 172.21.15.40:0/29129938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7eff800b0d70 0x7eff801f05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:14.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff7effd700 1 -- 172.21.15.40:0/29129938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff80151750 msgr2=0x7eff801408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:14.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff7effd700 1 --2- 172.21.15.40:0/29129938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff80151750 0x7eff801408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:14.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff7effd700 1 -- 172.21.15.40:0/29129938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff70009a70 con 0x7eff8006a950 2024-09-13T17:51:14.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff7effd700 1 --2- 172.21.15.40:0/29129938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff8006a950 0x7eff800c3b60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7eff6800c500 tx=0x7eff6800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:14.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff777fe700 1 -- 172.21.15.40:0/29129938 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff680052d0 con 0x7eff8006a950 2024-09-13T17:51:14.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff777fe700 1 -- 172.21.15.40:0/29129938 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff68005430 con 0x7eff8006a950 2024-09-13T17:51:14.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff777fe700 1 -- 172.21.15.40:0/29129938 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff680055a0 con 0x7eff8006a950 2024-09-13T17:51:14.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff85639700 1 -- 172.21.15.40:0/29129938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff801550e0 con 0x7eff8006a950 2024-09-13T17:51:14.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.486+0000 7eff85639700 1 -- 172.21.15.40:0/29129938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff801553a0 con 0x7eff8006a950 2024-09-13T17:51:14.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.490+0000 7eff777fe700 1 -- 172.21.15.40:0/29129938 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7eff680038e0 con 0x7eff8006a950 2024-09-13T17:51:14.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.490+0000 7eff777fe700 1 --2- 172.21.15.40:0/29129938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7eff64066760 0x7eff64068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:14.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.490+0000 7eff777fe700 1 -- 172.21.15.40:0/29129938 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7eff680838c0 con 0x7eff8006a950 2024-09-13T17:51:14.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.490+0000 7eff7e7fc700 1 --2- 172.21.15.40:0/29129938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7eff64066760 0x7eff64068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:14.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.490+0000 7eff7e7fc700 1 --2- 172.21.15.40:0/29129938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7eff64066760 0x7eff64068c20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7eff70014040 tx=0x7eff70011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:14.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff888ea740 1 Processor -- start 2024-09-13T17:51:14.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff888ea740 1 -- start start 2024-09-13T17:51:14.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff888ea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe74ba0 0xe607c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:14.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff888ea740 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe60d00 0xe610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:14.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff888ea740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe61620 0xe61a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:14.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff888ea740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0xdabd80 con 0xe61620 2024-09-13T17:51:14.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff888ea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0xda9510 con 0xe74ba0 2024-09-13T17:51:14.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff888ea740 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0xda4e00 con 0xe60d00 2024-09-13T17:51:14.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff7f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe61620 0xe61a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:14.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff7f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe61620 0xe61a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51164/0 (socket says 172.21.15.40:51164) 2024-09-13T17:51:14.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff7f7fe700 1 -- 172.21.15.40:0/334822167 learned_addr learned my addr 172.21.15.40:0/334822167 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:14.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff7effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe74ba0 0xe607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:14.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff7e7fc700 1 --2- 172.21.15.40:0/334822167 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe60d00 0xe610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff7f7fe700 1 -- 172.21.15.40:0/334822167 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe60d00 msgr2=0xe610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff7f7fe700 1 --2- 172.21.15.40:0/334822167 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe60d00 0xe610e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff7f7fe700 1 -- 172.21.15.40:0/334822167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe74ba0 msgr2=0xe607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff7f7fe700 1 --2- 172.21.15.40:0/334822167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe74ba0 0xe607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff7f7fe700 1 -- 172.21.15.40:0/334822167 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xc17960 con 0xe61620 2024-09-13T17:51:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff7f7fe700 1 --2- 172.21.15.40:0/334822167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe61620 0xe61a00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7eff78007fd0 tx=0x7eff78007320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6b4ef8746fc740 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff74ff9700 1 -- 172.21.15.40:0/334822167 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff78017070 con 0xe61620 2024-09-13T17:51:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff74ff9700 1 -- 172.21.15.40:0/334822167 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff78005900 con 0xe61620 2024-09-13T17:51:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.494+0000 7eff74ff9700 1 -- 172.21.15.40:0/334822167 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff78005a90 con 0xe61620 2024-09-13T17:51:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff888ea740 1 -- 172.21.15.40:0/334822167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe61620 msgr2=0xe61a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff888ea740 1 --2- 172.21.15.40:0/334822167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe61620 0xe61a00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7eff78007fd0 tx=0x7eff78007320 comp rx=0 tx=0).stop 2024-09-13T17:51:14.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff888ea740 1 -- 172.21.15.40:0/334822167 shutdown_connections 2024-09-13T17:51:14.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff888ea740 1 --2- 172.21.15.40:0/334822167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe61620 0xe61a00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:14.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff888ea740 1 --2- 172.21.15.40:0/334822167 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe60d00 0xe610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:14.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff888ea740 1 --2- 172.21.15.40:0/334822167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe74ba0 0xe607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:14.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff888ea740 1 -- 172.21.15.40:0/334822167 >> 172.21.15.40:0/334822167 conn(0xbcc4d0 msgr2=0xe77990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:14.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff888ea740 1 -- 172.21.15.40:0/334822167 shutdown_connections 2024-09-13T17:51:14.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff888ea740 1 -- 172.21.15.40:0/334822167 wait complete. 2024-09-13T17:51:14.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff888ea740 1 -- start start 2024-09-13T17:51:14.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff888ea740 1 Processor -- start 2024-09-13T17:51:14.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff888ea740 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe60d00 0xe85f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:14.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff888ea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe61620 0xe86470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:14.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff888ea740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe74ba0 0xe3db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:14.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff888ea740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0xd62730 con 0xe74ba0 2024-09-13T17:51:14.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff888ea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0xd685b0 con 0xe61620 2024-09-13T17:51:14.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff888ea740 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0xc50aa0 con 0xe60d00 2024-09-13T17:51:14.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff888ea740 10 client.0 _ll_register_callbacks cb 0xd9e610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:51:14.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff7f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe74ba0 0xe3db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:14.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff7f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe74ba0 0xe3db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51172/0 (socket says 172.21.15.40:51172) 2024-09-13T17:51:14.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.498+0000 7eff7f7fe700 1 -- 172.21.15.40:0/3475635502 learned_addr learned my addr 172.21.15.40:0/3475635502 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:14.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff7effd700 1 --2- 172.21.15.40:0/3475635502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe60d00 0xe85f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:14.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff7e7fc700 1 --2- 172.21.15.40:0/3475635502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe61620 0xe86470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:14.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff7f7fe700 1 -- 172.21.15.40:0/3475635502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe60d00 msgr2=0xe85f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:14.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff7f7fe700 1 --2- 172.21.15.40:0/3475635502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe60d00 0xe85f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:14.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff7f7fe700 1 -- 172.21.15.40:0/3475635502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe61620 msgr2=0xe86470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:14.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff7f7fe700 1 --2- 172.21.15.40:0/3475635502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe61620 0xe86470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:14.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff7f7fe700 1 -- 172.21.15.40:0/3475635502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff600035d0 con 0xe74ba0 2024-09-13T17:51:14.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff7f7fe700 1 --2- 172.21.15.40:0/3475635502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe74ba0 0xe3db70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7eff78015040 tx=0x7eff78000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82987ef0c90c1485 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:14.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-09-13T17:51:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff6dffb700 1 -- 172.21.15.40:0/3475635502 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff78017030 con 0xe74ba0 2024-09-13T17:51:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff6dffb700 1 -- 172.21.15.40:0/3475635502 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7eff78018070 con 0xe74ba0 2024-09-13T17:51:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff888ea740 1 -- 172.21.15.40:0/3475635502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xbd91e0 con 0xe74ba0 2024-09-13T17:51:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff6dffb700 1 -- 172.21.15.40:0/3475635502 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff78004d60 con 0xe74ba0 2024-09-13T17:51:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff888ea740 10 client.4890 fetch_fsmap learned FSMap version 15 2024-09-13T17:51:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff888ea740 1 -- 172.21.15.40:0/3475635502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0xd6b220 con 0xe74ba0 2024-09-13T17:51:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff6dffb700 1 -- 172.21.15.40:0/3475635502 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xbd91e0 con 0xe74ba0 2024-09-13T17:51:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff6dffb700 1 -- 172.21.15.40:0/3475635502 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 15) v1 ==== 4236+0+0 (secure 0 0 0) 0x7eff78012070 con 0xe74ba0 2024-09-13T17:51:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff888ea740 10 client.4890 fetch_fsmap finished waiting for FSMap version 15 2024-09-13T17:51:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff888ea740 10 client.4890 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-13T17:51:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff888ea740 20 client.4890 populate_metadata read hostname 'smithi040' 2024-09-13T17:51:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff888ea740 1 --2- 172.21.15.40:0/3475635502 >> [v2:172.21.15.43:6832/526975022,v1:172.21.15.43:6833/526975022] conn(0xe3bb20 0xe3bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:14.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff888ea740 4 client.4890 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:51:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff888ea740 1 -- 172.21.15.40:0/3475635502 --> [v2:172.21.15.43:6832/526975022,v1:172.21.15.43:6833/526975022] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xc23460 con 0xe3bb20 2024-09-13T17:51:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff7effd700 1 --2- 172.21.15.40:0/3475635502 >> [v2:172.21.15.43:6832/526975022,v1:172.21.15.43:6833/526975022] conn(0xe3bb20 0xe3bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff6dffb700 10 client.4890 ms_handle_connect on v2:172.21.15.43:6832/526975022 2024-09-13T17:51:14.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:14.502+0000 7eff7effd700 1 --2- 172.21.15.40:0/3475635502 >> [v2:172.21.15.43:6832/526975022,v1:172.21.15.43:6833/526975022] conn(0xe3bb20 0xe3bf00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de7e13ca5495ccbf server_cookie=f517bb71fdcb943 in_seq=0 out_seq=0 2024-09-13T17:51:17.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.242+0000 7eff6dffb700 1 -- 172.21.15.40:0/3475635502 <== mds.2 v2:172.21.15.43:6832/526975022 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0xc23460 con 0xe3bb20 2024-09-13T17:51:17.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.242+0000 7eff6dffb700 10 client.4890 handle_command_reply: tid=0 2024-09-13T17:51:17.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.242+0000 7eff888ea740 1 client.4890 shutdown 2024-09-13T17:51:17.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.242+0000 7eff888ea740 1 -- 172.21.15.40:0/3475635502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe74ba0 msgr2=0xe3db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.242+0000 7eff888ea740 1 --2- 172.21.15.40:0/3475635502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe74ba0 0xe3db70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7eff78015040 tx=0x7eff78000ed0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.242+0000 7eff888ea740 1 -- 172.21.15.40:0/3475635502 shutdown_connections 2024-09-13T17:51:17.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.242+0000 7eff888ea740 1 --2- 172.21.15.40:0/3475635502 >> [v2:172.21.15.43:6832/526975022,v1:172.21.15.43:6833/526975022] conn(0xe3bb20 0xe3bf00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.242+0000 7eff888ea740 1 --2- 172.21.15.40:0/3475635502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe74ba0 0xe3db70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.242+0000 7eff6dffb700 0 client.4890 ms_handle_reset on v2:172.21.15.43:6832/526975022 2024-09-13T17:51:17.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.242+0000 7eff888ea740 1 --2- 172.21.15.40:0/3475635502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe61620 0xe86470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.242+0000 7eff888ea740 1 --2- 172.21.15.40:0/3475635502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe60d00 0xe85f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.242+0000 7eff888ea740 1 -- 172.21.15.40:0/3475635502 >> 172.21.15.40:0/3475635502 conn(0xbcc4d0 msgr2=0xe74f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:17.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.242+0000 7eff888ea740 1 -- 172.21.15.40:0/3475635502 shutdown_connections 2024-09-13T17:51:17.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.242+0000 7eff888ea740 1 -- 172.21.15.40:0/3475635502 wait complete. 2024-09-13T17:51:17.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.242+0000 7eff888ea740 20 client.4890 trim_cache size 0 max 16384 2024-09-13T17:51:17.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.262+0000 7eff888ea740 1 Processor -- start 2024-09-13T17:51:17.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff888ea740 1 -- start start 2024-09-13T17:51:17.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff888ea740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe3bb20 0xe3a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff888ea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe60d00 0xe3ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff888ea740 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe61620 0xe3b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff888ea740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0xdd3ec0 con 0xe3bb20 2024-09-13T17:51:17.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff888ea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0xd6f4f0 con 0xe60d00 2024-09-13T17:51:17.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff888ea740 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0xe98ba0 con 0xe61620 2024-09-13T17:51:17.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff7effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe3bb20 0xe3a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff7effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe3bb20 0xe3a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51182/0 (socket says 172.21.15.40:51182) 2024-09-13T17:51:17.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff7effd700 1 -- 172.21.15.40:0/2216136127 learned_addr learned my addr 172.21.15.40:0/2216136127 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:17.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff7e7fc700 1 --2- 172.21.15.40:0/2216136127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe60d00 0xe3ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff7f7fe700 1 --2- 172.21.15.40:0/2216136127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe61620 0xe3b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff7effd700 1 -- 172.21.15.40:0/2216136127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe61620 msgr2=0xe3b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff7effd700 1 --2- 172.21.15.40:0/2216136127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe61620 0xe3b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff7effd700 1 -- 172.21.15.40:0/2216136127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe60d00 msgr2=0xe3ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff7effd700 1 --2- 172.21.15.40:0/2216136127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe60d00 0xe3ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff7effd700 1 -- 172.21.15.40:0/2216136127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xd51450 con 0xe3bb20 2024-09-13T17:51:17.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff7effd700 1 --2- 172.21.15.40:0/2216136127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe3bb20 0xe3a710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7eff68050950 tx=0x7eff68083040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77b28dd6f85e183 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:17.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff757fa700 1 -- 172.21.15.40:0/2216136127 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68054450 con 0xe3bb20 2024-09-13T17:51:17.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff757fa700 1 -- 172.21.15.40:0/2216136127 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff68058280 con 0xe3bb20 2024-09-13T17:51:17.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff757fa700 1 -- 172.21.15.40:0/2216136127 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6805c8d0 con 0xe3bb20 2024-09-13T17:51:17.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff888ea740 1 -- 172.21.15.40:0/2216136127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe3bb20 msgr2=0xe3a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff888ea740 1 --2- 172.21.15.40:0/2216136127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe3bb20 0xe3a710 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7eff68050950 tx=0x7eff68083040 comp rx=0 tx=0).stop 2024-09-13T17:51:17.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff888ea740 1 -- 172.21.15.40:0/2216136127 shutdown_connections 2024-09-13T17:51:17.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff888ea740 1 --2- 172.21.15.40:0/2216136127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe61620 0xe3b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff888ea740 1 --2- 172.21.15.40:0/2216136127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe60d00 0xe3ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff888ea740 1 --2- 172.21.15.40:0/2216136127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe3bb20 0xe3a710 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff888ea740 1 -- 172.21.15.40:0/2216136127 >> 172.21.15.40:0/2216136127 conn(0xbcc4d0 msgr2=0xe85c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:17.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.266+0000 7eff888ea740 1 -- 172.21.15.40:0/2216136127 shutdown_connections 2024-09-13T17:51:17.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff888ea740 1 -- 172.21.15.40:0/2216136127 wait complete. 2024-09-13T17:51:17.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff888ea740 1 -- start start 2024-09-13T17:51:17.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff888ea740 1 Processor -- start 2024-09-13T17:51:17.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff888ea740 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe3bb20 0xee5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff888ea740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe60d00 0xea6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff888ea740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe61620 0xea66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff888ea740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0xe83460 con 0xe60d00 2024-09-13T17:51:17.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff888ea740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0xe60b80 con 0xe61620 2024-09-13T17:51:17.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff888ea740 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0xe88e50 con 0xe3bb20 2024-09-13T17:51:17.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff888ea740 10 client.0 _ll_register_callbacks cb 0x7eff78006950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:51:17.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff7e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe60d00 0xea6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff7e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe60d00 0xea6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51198/0 (socket says 172.21.15.40:51198) 2024-09-13T17:51:17.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff7e7fc700 1 -- 172.21.15.40:0/2057782258 learned_addr learned my addr 172.21.15.40:0/2057782258 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:17.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff7effd700 1 --2- 172.21.15.40:0/2057782258 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe3bb20 0xee5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff7f7fe700 1 --2- 172.21.15.40:0/2057782258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe61620 0xea66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff7e7fc700 1 -- 172.21.15.40:0/2057782258 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe3bb20 msgr2=0xee5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff7e7fc700 1 --2- 172.21.15.40:0/2057782258 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe3bb20 0xee5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff7e7fc700 1 -- 172.21.15.40:0/2057782258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe61620 msgr2=0xea66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff7e7fc700 1 --2- 172.21.15.40:0/2057782258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe61620 0xea66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff7e7fc700 1 -- 172.21.15.40:0/2057782258 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff801553a0 con 0xe60d00 2024-09-13T17:51:17.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff7e7fc700 1 --2- 172.21.15.40:0/2057782258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe60d00 0xea6170 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7eff8006f5d0 tx=0x7eff70004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f599e7cfc0204030 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:17.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-09-13T17:51:17.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff6dffb700 1 -- 172.21.15.40:0/2057782258 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7002e040 con 0xe60d00 2024-09-13T17:51:17.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff6dffb700 1 -- 172.21.15.40:0/2057782258 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7eff70035650 con 0xe60d00 2024-09-13T17:51:17.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff888ea740 1 -- 172.21.15.40:0/2057782258 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff78005900 con 0xe60d00 2024-09-13T17:51:17.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff6dffb700 1 -- 172.21.15.40:0/2057782258 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff70035810 con 0xe60d00 2024-09-13T17:51:17.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff888ea740 10 client.4896 fetch_fsmap learned FSMap version 15 2024-09-13T17:51:17.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff888ea740 1 -- 172.21.15.40:0/2057782258 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0xd45720 con 0xe60d00 2024-09-13T17:51:17.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.270+0000 7eff6dffb700 1 -- 172.21.15.40:0/2057782258 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff78005900 con 0xe60d00 2024-09-13T17:51:17.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff6dffb700 1 -- 172.21.15.40:0/2057782258 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 15) v1 ==== 4236+0+0 (secure 0 0 0) 0x7eff70035970 con 0xe60d00 2024-09-13T17:51:17.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff888ea740 10 client.4896 fetch_fsmap finished waiting for FSMap version 15 2024-09-13T17:51:17.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff888ea740 10 client.4896 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-13T17:51:17.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff888ea740 20 client.4896 populate_metadata read hostname 'smithi040' 2024-09-13T17:51:17.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff888ea740 1 --2- 172.21.15.40:0/2057782258 >> [v2:172.21.15.43:6832/526975022,v1:172.21.15.43:6833/526975022] conn(0xe74ba0 0xea54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff888ea740 4 client.4896 mds_command: new command op to 4279 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T17:51:17.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff888ea740 1 -- 172.21.15.40:0/2057782258 --> [v2:172.21.15.43:6832/526975022,v1:172.21.15.43:6833/526975022] -- command(tid 0: {"prefix": "status"}) v1 -- 0xc481e0 con 0xe74ba0 2024-09-13T17:51:17.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff7effd700 1 --2- 172.21.15.40:0/2057782258 >> [v2:172.21.15.43:6832/526975022,v1:172.21.15.43:6833/526975022] conn(0xe74ba0 0xea54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff6dffb700 10 client.4896 ms_handle_connect on v2:172.21.15.43:6832/526975022 2024-09-13T17:51:17.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff7effd700 1 --2- 172.21.15.40:0/2057782258 >> [v2:172.21.15.43:6832/526975022,v1:172.21.15.43:6833/526975022] conn(0xe74ba0 0xea54a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fcfce01f87ccbd3e server_cookie=6f0c083341af7a3 in_seq=0 out_seq=0 2024-09-13T17:51:17.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff6dffb700 1 -- 172.21.15.40:0/2057782258 <== mds.2 v2:172.21.15.43:6832/526975022 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0xc481e0 con 0xe74ba0 2024-09-13T17:51:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff6dffb700 10 client.4896 handle_command_reply: tid=0 2024-09-13T17:51:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff888ea740 1 client.4896 shutdown 2024-09-13T17:51:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff888ea740 1 -- 172.21.15.40:0/2057782258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe60d00 msgr2=0xea6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff888ea740 1 --2- 172.21.15.40:0/2057782258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe60d00 0xea6170 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7eff8006f5d0 tx=0x7eff70004b60 comp rx=0 tx=0).stop 2024-09-13T17:51:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff888ea740 1 -- 172.21.15.40:0/2057782258 shutdown_connections 2024-09-13T17:51:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff888ea740 1 --2- 172.21.15.40:0/2057782258 >> [v2:172.21.15.43:6832/526975022,v1:172.21.15.43:6833/526975022] conn(0xe74ba0 0xea54a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff888ea740 1 --2- 172.21.15.40:0/2057782258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0xe61620 0xea66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff6dffb700 0 client.4896 ms_handle_reset on v2:172.21.15.43:6832/526975022 2024-09-13T17:51:17.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff888ea740 1 --2- 172.21.15.40:0/2057782258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0xe60d00 0xea6170 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff888ea740 1 --2- 172.21.15.40:0/2057782258 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0xe3bb20 0xee5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff888ea740 1 -- 172.21.15.40:0/2057782258 >> 172.21.15.40:0/2057782258 conn(0xbcc4d0 msgr2=0xe865b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff888ea740 1 -- 172.21.15.40:0/2057782258 shutdown_connections 2024-09-13T17:51:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff888ea740 1 -- 172.21.15.40:0/2057782258 wait complete. 2024-09-13T17:51:17.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff888ea740 20 client.4896 trim_cache size 0 max 16384 2024-09-13T17:51:17.284 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:51:17.284 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:51:17.284 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 2, 2024-09-13T17:51:17.284 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4279, 2024-09-13T17:51:17.285 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-09-13T17:51:17.285 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-09-13T17:51:17.285 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-09-13T17:51:17.285 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 11.060286122000001, 2024-09-13T17:51:17.285 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 15, 2024-09-13T17:51:17.285 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 27, 2024-09-13T17:51:17.285 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 25, 2024-09-13T17:51:17.285 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 49.144566588000004 2024-09-13T17:51:17.285 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:51:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff85639700 1 -- 172.21.15.40:0/29129938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7eff64066760 msgr2=0x7eff64068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff85639700 1 --2- 172.21.15.40:0/29129938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7eff64066760 0x7eff64068c20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7eff70014040 tx=0x7eff70011000 comp rx=0 tx=0).stop 2024-09-13T17:51:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff85639700 1 -- 172.21.15.40:0/29129938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff8006a950 msgr2=0x7eff800c3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff85639700 1 --2- 172.21.15.40:0/29129938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff8006a950 0x7eff800c3b60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7eff6800c500 tx=0x7eff6800c810 comp rx=0 tx=0).stop 2024-09-13T17:51:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff85639700 1 -- 172.21.15.40:0/29129938 shutdown_connections 2024-09-13T17:51:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff85639700 1 --2- 172.21.15.40:0/29129938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff80151750 0x7eff801408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff85639700 1 --2- 172.21.15.40:0/29129938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7eff64066760 0x7eff64068c20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff85639700 1 --2- 172.21.15.40:0/29129938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7eff800b0d70 0x7eff801f05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff85639700 1 --2- 172.21.15.40:0/29129938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7eff8006a950 0x7eff800c3b60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff85639700 1 -- 172.21.15.40:0/29129938 >> 172.21.15.40:0/29129938 conn(0x7eff8006d5b0 msgr2=0x7eff8006d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:17.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff85639700 1 -- 172.21.15.40:0/29129938 shutdown_connections 2024-09-13T17:51:17.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.274+0000 7eff85639700 1 -- 172.21.15.40:0/29129938 wait complete. 2024-09-13T17:51:17.300 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-13T17:51:17.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.418+0000 7f0c45c74700 1 Processor -- start 2024-09-13T17:51:17.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.418+0000 7f0c45c74700 1 -- start start 2024-09-13T17:51:17.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.418+0000 7f0c45c74700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0c401515a0 0x7f0c4006e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.418+0000 7f0c45c74700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0c4006dde0 0x7f0c4015a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.418+0000 7f0c45c74700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c4006e580 con 0x7f0c400c3de0 2024-09-13T17:51:17.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.418+0000 7f0c45c74700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c40151980 con 0x7f0c4006dde0 2024-09-13T17:51:17.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.418+0000 7f0c45c74700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c400c3930 con 0x7f0c401515a0 2024-09-13T17:51:17.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.418+0000 7f0c3ffff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0c4006dde0 0x7f0c4015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.418+0000 7f0c45473700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0c400c3de0 0x7f0c400cd3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:51056/0 (socket says 172.21.15.40:51056) 2024-09-13T17:51:17.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.418+0000 7f0c44c72700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0c401515a0 0x7f0c4006e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.418+0000 7f0c45473700 1 -- 172.21.15.40:0/541632805 learned_addr learned my addr 172.21.15.40:0/541632805 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:17.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.418+0000 7f0c3ffff700 1 -- 172.21.15.40:0/541632805 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0c401515a0 msgr2=0x7f0c4006e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.418+0000 7f0c3ffff700 1 --2- 172.21.15.40:0/541632805 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0c401515a0 0x7f0c4006e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.418+0000 7f0c3ffff700 1 -- 172.21.15.40:0/541632805 >> v1:172.21.15.43:6789/0 conn(0x7f0c400c3de0 legacy=0x7f0c400cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.418+0000 7f0c3ffff700 1 -- 172.21.15.40:0/541632805 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4015ad50 con 0x7f0c4006dde0 2024-09-13T17:51:17.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.418+0000 7f0c3ffff700 1 --2- 172.21.15.40:0/541632805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c4006dde0 0x7f0c4015a810 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009850 tx=0x7f0c30009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28598cc97cee7b6f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:17.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.418+0000 7f0c3f7fe700 1 -- 172.21.15.40:0/541632805 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30011830 con 0x7f0c4006dde0 2024-09-13T17:51:17.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.418+0000 7f0c3f7fe700 1 -- 172.21.15.40:0/541632805 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c30011990 con 0x7f0c4006dde0 2024-09-13T17:51:17.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.418+0000 7f0c3f7fe700 1 -- 172.21.15.40:0/541632805 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30011d40 con 0x7f0c4006dde0 2024-09-13T17:51:17.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.422+0000 7f0c45c74700 1 -- 172.21.15.40:0/541632805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c4006dde0 msgr2=0x7f0c4015a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.422+0000 7f0c45c74700 1 --2- 172.21.15.40:0/541632805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c4006dde0 0x7f0c4015a810 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009850 tx=0x7f0c30009b60 comp rx=0 tx=0).stop 2024-09-13T17:51:17.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.422+0000 7f0c45c74700 1 -- 172.21.15.40:0/541632805 shutdown_connections 2024-09-13T17:51:17.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.422+0000 7f0c45c74700 1 --2- 172.21.15.40:0/541632805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c4006dde0 0x7f0c4015a810 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.422+0000 7f0c45c74700 1 --2- 172.21.15.40:0/541632805 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0c401515a0 0x7f0c4006e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.422+0000 7f0c45c74700 1 -- 172.21.15.40:0/541632805 >> 172.21.15.40:0/541632805 conn(0x7f0c40072070 msgr2=0x7f0c40072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:17.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.422+0000 7f0c45c74700 1 -- 172.21.15.40:0/541632805 shutdown_connections 2024-09-13T17:51:17.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.422+0000 7f0c45c74700 1 -- 172.21.15.40:0/541632805 wait complete. 2024-09-13T17:51:17.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.422+0000 7f0c45c74700 1 Processor -- start 2024-09-13T17:51:17.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.422+0000 7f0c45c74700 1 -- start start 2024-09-13T17:51:17.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.422+0000 7f0c45c74700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c4006dde0 0x7f0c40156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.422+0000 7f0c45c74700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0c400c3de0 0x7f0c4014e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.422+0000 7f0c45c74700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c401515a0 0x7f0c401f2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.422+0000 7f0c45c74700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c401ef4d0 con 0x7f0c401515a0 2024-09-13T17:51:17.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.422+0000 7f0c45c74700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c401ef7b0 con 0x7f0c4006dde0 2024-09-13T17:51:17.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.422+0000 7f0c45c74700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c401efa90 con 0x7f0c400c3de0 2024-09-13T17:51:17.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.422+0000 7f0c45473700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c401515a0 0x7f0c401f2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.426+0000 7f0c45473700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c401515a0 0x7f0c401f2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51214/0 (socket says 172.21.15.40:51214) 2024-09-13T17:51:17.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.426+0000 7f0c45473700 1 -- 172.21.15.40:0/1898336432 learned_addr learned my addr 172.21.15.40:0/1898336432 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:17.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.426+0000 7f0c3ffff700 1 --2- 172.21.15.40:0/1898336432 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0c400c3de0 0x7f0c4014e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.426+0000 7f0c44c72700 1 --2- 172.21.15.40:0/1898336432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c4006dde0 0x7f0c40156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.426+0000 7f0c45473700 1 -- 172.21.15.40:0/1898336432 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0c400c3de0 msgr2=0x7f0c4014e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.426+0000 7f0c45473700 1 --2- 172.21.15.40:0/1898336432 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0c400c3de0 0x7f0c4014e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.426+0000 7f0c45473700 1 -- 172.21.15.40:0/1898336432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c4006dde0 msgr2=0x7f0c40156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.426+0000 7f0c45473700 1 --2- 172.21.15.40:0/1898336432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c4006dde0 0x7f0c40156a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.426+0000 7f0c45473700 1 -- 172.21.15.40:0/1898336432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c300094b0 con 0x7f0c401515a0 2024-09-13T17:51:17.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.426+0000 7f0c44c72700 1 --2- 172.21.15.40:0/1898336432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c4006dde0 0x7f0c40156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:17.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.426+0000 7f0c45473700 1 --2- 172.21.15.40:0/1898336432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c401515a0 0x7f0c401f2ca0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0c3800caf0 tx=0x7f0c3800cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:17.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.426+0000 7f0c3d7fa700 1 -- 172.21.15.40:0/1898336432 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3800a770 con 0x7f0c401515a0 2024-09-13T17:51:17.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.426+0000 7f0c3d7fa700 1 -- 172.21.15.40:0/1898336432 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3800a8d0 con 0x7f0c401515a0 2024-09-13T17:51:17.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.426+0000 7f0c3d7fa700 1 -- 172.21.15.40:0/1898336432 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3800ac40 con 0x7f0c401515a0 2024-09-13T17:51:17.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.426+0000 7f0c45c74700 1 -- 172.21.15.40:0/1898336432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c401efd70 con 0x7f0c401515a0 2024-09-13T17:51:17.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.426+0000 7f0c45c74700 1 -- 172.21.15.40:0/1898336432 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c40156f50 con 0x7f0c401515a0 2024-09-13T17:51:17.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.426+0000 7f0c3d7fa700 1 -- 172.21.15.40:0/1898336432 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0c3801b3b0 con 0x7f0c401515a0 2024-09-13T17:51:17.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.430+0000 7f0c3d7fa700 1 --2- 172.21.15.40:0/1898336432 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0c2c066730 0x7f0c2c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.430+0000 7f0c3d7fa700 1 -- 172.21.15.40:0/1898336432 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0c38083da0 con 0x7f0c401515a0 2024-09-13T17:51:17.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.430+0000 7f0c44c72700 1 --2- 172.21.15.40:0/1898336432 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0c2c066730 0x7f0c2c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.430+0000 7f0c44c72700 1 --2- 172.21.15.40:0/1898336432 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0c2c066730 0x7f0c2c068bf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009c40 tx=0x7f0c34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:17.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.430+0000 7f0c4a187740 1 Processor -- start 2024-09-13T17:51:17.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.430+0000 7f0c4a187740 1 -- start start 2024-09-13T17:51:17.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c4a187740 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x1b34db0 0x1b209d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c4a187740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 0x1b212f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c4a187740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 0x1b21c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c4a187740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a6bd80 con 0x1b20f10 2024-09-13T17:51:17.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c4a187740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a69510 con 0x1b21830 2024-09-13T17:51:17.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c4a187740 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a64e00 con 0x1b34db0 2024-09-13T17:51:17.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c3ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 0x1b212f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c3ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 0x1b212f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51224/0 (socket says 172.21.15.40:51224) 2024-09-13T17:51:17.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c3ffff700 1 -- 172.21.15.40:0/964660604 learned_addr learned my addr 172.21.15.40:0/964660604 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:17.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c45473700 1 --2- 172.21.15.40:0/964660604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 0x1b21c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c44c72700 1 --2- 172.21.15.40:0/964660604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x1b34db0 0x1b209d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c3ffff700 1 -- 172.21.15.40:0/964660604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x1b34db0 msgr2=0x1b209d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c3ffff700 1 --2- 172.21.15.40:0/964660604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x1b34db0 0x1b209d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c3ffff700 1 -- 172.21.15.40:0/964660604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 msgr2=0x1b21c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c3ffff700 1 --2- 172.21.15.40:0/964660604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 0x1b21c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c3ffff700 1 -- 172.21.15.40:0/964660604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x18d7960 con 0x1b20f10 2024-09-13T17:51:17.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c3ffff700 1 --2- 172.21.15.40:0/964660604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 0x1b212f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0c30014040 tx=0x7f0c30008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e959fe688187cf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:17.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c267fc700 1 -- 172.21.15.40:0/964660604 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30011d00 con 0x1b20f10 2024-09-13T17:51:17.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c267fc700 1 -- 172.21.15.40:0/964660604 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c300117b0 con 0x1b20f10 2024-09-13T17:51:17.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c267fc700 1 -- 172.21.15.40:0/964660604 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30004260 con 0x1b20f10 2024-09-13T17:51:17.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c4a187740 1 -- 172.21.15.40:0/964660604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 msgr2=0x1b212f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c4a187740 1 --2- 172.21.15.40:0/964660604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 0x1b212f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0c30014040 tx=0x7f0c30008030 comp rx=0 tx=0).stop 2024-09-13T17:51:17.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c4a187740 1 -- 172.21.15.40:0/964660604 shutdown_connections 2024-09-13T17:51:17.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c4a187740 1 --2- 172.21.15.40:0/964660604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 0x1b21c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c4a187740 1 --2- 172.21.15.40:0/964660604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 0x1b212f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c4a187740 1 --2- 172.21.15.40:0/964660604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x1b34db0 0x1b209d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c4a187740 1 -- 172.21.15.40:0/964660604 >> 172.21.15.40:0/964660604 conn(0x188c4d0 msgr2=0x1b37ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:17.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c4a187740 1 -- 172.21.15.40:0/964660604 shutdown_connections 2024-09-13T17:51:17.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.434+0000 7f0c4a187740 1 -- 172.21.15.40:0/964660604 wait complete. 2024-09-13T17:51:17.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c4a187740 1 -- start start 2024-09-13T17:51:17.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c4a187740 1 Processor -- start 2024-09-13T17:51:17.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c4a187740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 0x1b37b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c4a187740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 0x1b46370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c4a187740 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x1b34db0 0x1b468b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c4a187740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a22730 con 0x1b20f10 2024-09-13T17:51:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c4a187740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a285b0 con 0x1b21830 2024-09-13T17:51:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c4a187740 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x1910aa0 con 0x1b34db0 2024-09-13T17:51:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c4a187740 10 client.0 _ll_register_callbacks cb 0x1a5e610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:51:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c44c72700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 0x1b37b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c44c72700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 0x1b37b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51240/0 (socket says 172.21.15.40:51240) 2024-09-13T17:51:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c44c72700 1 -- 172.21.15.40:0/3944564828 learned_addr learned my addr 172.21.15.40:0/3944564828 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c45473700 1 --2- 172.21.15.40:0/3944564828 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x1b34db0 0x1b468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c3ffff700 1 --2- 172.21.15.40:0/3944564828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 0x1b46370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c44c72700 1 -- 172.21.15.40:0/3944564828 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x1b34db0 msgr2=0x1b468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c44c72700 1 --2- 172.21.15.40:0/3944564828 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x1b34db0 0x1b468b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c44c72700 1 -- 172.21.15.40:0/3944564828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 msgr2=0x1b46370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c44c72700 1 --2- 172.21.15.40:0/3944564828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 0x1b46370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c44c72700 1 -- 172.21.15.40:0/3944564828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4015ad50 con 0x1b20f10 2024-09-13T17:51:17.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c44c72700 1 --2- 172.21.15.40:0/3944564828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 0x1b37b80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0c34011c80 tx=0x7f0c34004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a7b818adb17d68 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:17.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-09-13T17:51:17.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c1f7fe700 1 -- 172.21.15.40:0/3944564828 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c340041f0 con 0x1b20f10 2024-09-13T17:51:17.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c1f7fe700 1 -- 172.21.15.40:0/3944564828 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0c3401e040 con 0x1b20f10 2024-09-13T17:51:17.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c4a187740 1 -- 172.21.15.40:0/3944564828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x18991e0 con 0x1b20f10 2024-09-13T17:51:17.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c1f7fe700 1 -- 172.21.15.40:0/3944564828 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c34003d60 con 0x1b20f10 2024-09-13T17:51:17.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c4a187740 10 client.4905 fetch_fsmap learned FSMap version 15 2024-09-13T17:51:17.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.438+0000 7f0c4a187740 1 -- 172.21.15.40:0/3944564828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x1a2b220 con 0x1b20f10 2024-09-13T17:51:17.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.442+0000 7f0c1f7fe700 1 -- 172.21.15.40:0/3944564828 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x18991e0 con 0x1b20f10 2024-09-13T17:51:17.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.442+0000 7f0c1f7fe700 1 -- 172.21.15.40:0/3944564828 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 15) v1 ==== 4236+0+0 (secure 0 0 0) 0x7f0c3402c3f0 con 0x1b20f10 2024-09-13T17:51:17.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.442+0000 7f0c4a187740 10 client.4905 fetch_fsmap finished waiting for FSMap version 15 2024-09-13T17:51:17.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.442+0000 7f0c4a187740 10 client.4905 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-13T17:51:17.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.442+0000 7f0c4a187740 20 client.4905 populate_metadata read hostname 'smithi040' 2024-09-13T17:51:17.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.442+0000 7f0c4a187740 1 --2- 172.21.15.40:0/3944564828 >> [v2:172.21.15.43:6834/2586910944,v1:172.21.15.43:6835/2586910944] conn(0x1b1e4c0 0x1b1e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.442+0000 7f0c4a187740 4 client.4905 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:51:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.442+0000 7f0c4a187740 1 -- 172.21.15.40:0/3944564828 --> [v2:172.21.15.43:6834/2586910944,v1:172.21.15.43:6835/2586910944] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x18e3460 con 0x1b1e4c0 2024-09-13T17:51:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.442+0000 7f0c3ffff700 1 --2- 172.21.15.40:0/3944564828 >> [v2:172.21.15.43:6834/2586910944,v1:172.21.15.43:6835/2586910944] conn(0x1b1e4c0 0x1b1e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.442+0000 7f0c3ffff700 1 -- 172.21.15.40:0/3944564828 >> [v2:172.21.15.43:6834/2586910944,v1:172.21.15.43:6835/2586910944] conn(0x1b1e4c0 msgr2=0x1b1e8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2024-09-13T17:51:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.442+0000 7f0c3ffff700 1 -- 172.21.15.40:0/3944564828 >> [v2:172.21.15.43:6834/2586910944,v1:172.21.15.43:6835/2586910944] conn(0x1b1e4c0 msgr2=0x1b1e8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:51:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.442+0000 7f0c3ffff700 1 --2- 172.21.15.40:0/3944564828 >> [v2:172.21.15.43:6834/2586910944,v1:172.21.15.43:6835/2586910944] conn(0x1b1e4c0 0x1b1e8a0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:51:17.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.442+0000 7f0c3ffff700 1 --2- 172.21.15.40:0/3944564828 >> [v2:172.21.15.43:6834/2586910944,v1:172.21.15.43:6835/2586910944] conn(0x1b1e4c0 0x1b1e8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:51:17.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.646+0000 7f0c3ffff700 1 --2- 172.21.15.40:0/3944564828 >> [v2:172.21.15.43:6834/2586910944,v1:172.21.15.43:6835/2586910944] conn(0x1b1e4c0 0x1b1e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.646+0000 7f0c3ffff700 1 --2- 172.21.15.40:0/3944564828 >> [v2:172.21.15.43:6834/2586910944,v1:172.21.15.43:6835/2586910944] conn(0x1b1e4c0 0x1b1e8a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e6edfa7087fbdca server_cookie=d3c5a37659ca4d78 in_seq=0 out_seq=0 2024-09-13T17:51:17.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.646+0000 7f0c1f7fe700 10 client.4905 ms_handle_connect on v2:172.21.15.43:6834/2586910944 2024-09-13T17:51:17.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.646+0000 7f0c1f7fe700 1 -- 172.21.15.40:0/3944564828 <== mds.1 v2:172.21.15.43:6834/2586910944 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x18e3460 con 0x1b1e4c0 2024-09-13T17:51:17.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.646+0000 7f0c1f7fe700 10 client.4905 handle_command_reply: tid=0 2024-09-13T17:51:17.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.646+0000 7f0c4a187740 1 client.4905 shutdown 2024-09-13T17:51:17.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.646+0000 7f0c4a187740 1 -- 172.21.15.40:0/3944564828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 msgr2=0x1b37b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.646+0000 7f0c4a187740 1 --2- 172.21.15.40:0/3944564828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 0x1b37b80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0c34011c80 tx=0x7f0c34004770 comp rx=0 tx=0).stop 2024-09-13T17:51:17.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.646+0000 7f0c4a187740 1 -- 172.21.15.40:0/3944564828 shutdown_connections 2024-09-13T17:51:17.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.646+0000 7f0c4a187740 1 --2- 172.21.15.40:0/3944564828 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x1b34db0 0x1b468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.646+0000 7f0c4a187740 1 --2- 172.21.15.40:0/3944564828 >> [v2:172.21.15.43:6834/2586910944,v1:172.21.15.43:6835/2586910944] conn(0x1b1e4c0 0x1b1e8a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.646+0000 7f0c4a187740 1 --2- 172.21.15.40:0/3944564828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 0x1b46370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.646+0000 7f0c4a187740 1 --2- 172.21.15.40:0/3944564828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 0x1b37b80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.646+0000 7f0c1f7fe700 0 client.4905 ms_handle_reset on v2:172.21.15.43:6834/2586910944 2024-09-13T17:51:17.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.646+0000 7f0c4a187740 1 -- 172.21.15.40:0/3944564828 >> 172.21.15.40:0/3944564828 conn(0x188c4d0 msgr2=0x1b35190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:17.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.646+0000 7f0c4a187740 1 -- 172.21.15.40:0/3944564828 shutdown_connections 2024-09-13T17:51:17.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.646+0000 7f0c4a187740 1 -- 172.21.15.40:0/3944564828 wait complete. 2024-09-13T17:51:17.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.646+0000 7f0c4a187740 20 client.4905 trim_cache size 0 max 16384 2024-09-13T17:51:17.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.670+0000 7f0c4a187740 1 Processor -- start 2024-09-13T17:51:17.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.670+0000 7f0c4a187740 1 -- start start 2024-09-13T17:51:17.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c4a187740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 0x1b438d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c4a187740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 0x1b4c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c4a187740 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x1b34db0 0x1b4ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c4a187740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a03470 con 0x1b20f10 2024-09-13T17:51:17.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c4a187740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ac60f0 con 0x1b21830 2024-09-13T17:51:17.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c4a187740 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a93ec0 con 0x1b34db0 2024-09-13T17:51:17.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c44c72700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 0x1b438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c44c72700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 0x1b438d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51246/0 (socket says 172.21.15.40:51246) 2024-09-13T17:51:17.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c44c72700 1 -- 172.21.15.40:0/1579030229 learned_addr learned my addr 172.21.15.40:0/1579030229 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:17.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c45473700 1 --2- 172.21.15.40:0/1579030229 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x1b34db0 0x1b4ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c3ffff700 1 --2- 172.21.15.40:0/1579030229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 0x1b4c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c44c72700 1 -- 172.21.15.40:0/1579030229 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x1b34db0 msgr2=0x1b4ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c44c72700 1 --2- 172.21.15.40:0/1579030229 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x1b34db0 0x1b4ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c44c72700 1 -- 172.21.15.40:0/1579030229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 msgr2=0x1b4c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c44c72700 1 --2- 172.21.15.40:0/1579030229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 0x1b4c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c44c72700 1 -- 172.21.15.40:0/1579030229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1a11450 con 0x1b20f10 2024-09-13T17:51:17.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c3ffff700 1 --2- 172.21.15.40:0/1579030229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 0x1b4c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:17.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c44c72700 1 --2- 172.21.15.40:0/1579030229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 0x1b438d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009820 tx=0x7f0c34023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b53997c6db5d3a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:17.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c26ffd700 1 -- 172.21.15.40:0/1579030229 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3402cc60 con 0x1b20f10 2024-09-13T17:51:17.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c26ffd700 1 -- 172.21.15.40:0/1579030229 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c34003d60 con 0x1b20f10 2024-09-13T17:51:17.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.674+0000 7f0c26ffd700 1 -- 172.21.15.40:0/1579030229 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3402c8b0 con 0x1b20f10 2024-09-13T17:51:17.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c4a187740 1 -- 172.21.15.40:0/1579030229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 msgr2=0x1b438d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c4a187740 1 --2- 172.21.15.40:0/1579030229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 0x1b438d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009820 tx=0x7f0c34023000 comp rx=0 tx=0).stop 2024-09-13T17:51:17.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c4a187740 1 -- 172.21.15.40:0/1579030229 shutdown_connections 2024-09-13T17:51:17.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c4a187740 1 --2- 172.21.15.40:0/1579030229 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x1b34db0 0x1b4ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c4a187740 1 --2- 172.21.15.40:0/1579030229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 0x1b4c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c4a187740 1 --2- 172.21.15.40:0/1579030229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b20f10 0x1b438d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c4a187740 1 -- 172.21.15.40:0/1579030229 >> 172.21.15.40:0/1579030229 conn(0x188c4d0 msgr2=0x1afe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:17.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c4a187740 1 -- 172.21.15.40:0/1579030229 shutdown_connections 2024-09-13T17:51:17.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c4a187740 1 -- 172.21.15.40:0/1579030229 wait complete. 2024-09-13T17:51:17.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c4a187740 1 -- start start 2024-09-13T17:51:17.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c4a187740 1 Processor -- start 2024-09-13T17:51:17.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c4a187740 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x1b20f10 0x1afd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c4a187740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 0x1afb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c4a187740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b34db0 0x1b631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c4a187740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x1b498a0 con 0x1b34db0 2024-09-13T17:51:17.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c4a187740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x1a2f4f0 con 0x1b21830 2024-09-13T17:51:17.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c4a187740 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ba1b00 con 0x1b20f10 2024-09-13T17:51:17.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c4a187740 10 client.0 _ll_register_callbacks cb 0x7f0c340090e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:51:17.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c45473700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b34db0 0x1b631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c45473700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b34db0 0x1b631c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51250/0 (socket says 172.21.15.40:51250) 2024-09-13T17:51:17.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c45473700 1 -- 172.21.15.40:0/3018524848 learned_addr learned my addr 172.21.15.40:0/3018524848 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:17.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c3ffff700 1 --2- 172.21.15.40:0/3018524848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 0x1afb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.678+0000 7f0c44c72700 1 --2- 172.21.15.40:0/3018524848 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x1b20f10 0x1afd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c45473700 1 -- 172.21.15.40:0/3018524848 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x1b20f10 msgr2=0x1afd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c45473700 1 --2- 172.21.15.40:0/3018524848 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x1b20f10 0x1afd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c45473700 1 -- 172.21.15.40:0/3018524848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 msgr2=0x1afb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c45473700 1 --2- 172.21.15.40:0/3018524848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 0x1afb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c45473700 1 -- 172.21.15.40:0/3018524848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x18d7960 con 0x1b34db0 2024-09-13T17:51:17.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c45473700 1 --2- 172.21.15.40:0/3018524848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b34db0 0x1b631c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0c380546f0 tx=0x7f0c38054f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d4d7f7ee0e15a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:17.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-09-13T17:51:17.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c1f7fe700 1 -- 172.21.15.40:0/3018524848 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3806c1f0 con 0x1b34db0 2024-09-13T17:51:17.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c1f7fe700 1 -- 172.21.15.40:0/3018524848 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0c3806c350 con 0x1b34db0 2024-09-13T17:51:17.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c4a187740 1 -- 172.21.15.40:0/3018524848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c300117b0 con 0x1b34db0 2024-09-13T17:51:17.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c1f7fe700 1 -- 172.21.15.40:0/3018524848 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c380711f0 con 0x1b34db0 2024-09-13T17:51:17.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c4a187740 10 client.4911 fetch_fsmap learned FSMap version 15 2024-09-13T17:51:17.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c4a187740 1 -- 172.21.15.40:0/3018524848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x1a2b220 con 0x1b34db0 2024-09-13T17:51:17.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c1f7fe700 1 -- 172.21.15.40:0/3018524848 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c300117b0 con 0x1b34db0 2024-09-13T17:51:17.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c1f7fe700 1 -- 172.21.15.40:0/3018524848 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 15) v1 ==== 4236+0+0 (secure 0 0 0) 0x7f0c3805c8f0 con 0x1b34db0 2024-09-13T17:51:17.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c4a187740 10 client.4911 fetch_fsmap finished waiting for FSMap version 15 2024-09-13T17:51:17.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c4a187740 10 client.4911 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-13T17:51:17.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c4a187740 20 client.4911 populate_metadata read hostname 'smithi040' 2024-09-13T17:51:17.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c4a187740 1 --2- 172.21.15.40:0/3018524848 >> [v2:172.21.15.43:6834/2586910944,v1:172.21.15.43:6835/2586910944] conn(0x1ba5230 0x1b9a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c4a187740 4 client.4911 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T17:51:17.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c4a187740 1 -- 172.21.15.40:0/3018524848 --> [v2:172.21.15.43:6834/2586910944,v1:172.21.15.43:6835/2586910944] -- command(tid 0: {"prefix": "status"}) v1 -- 0x19081e0 con 0x1ba5230 2024-09-13T17:51:17.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c44c72700 1 --2- 172.21.15.40:0/3018524848 >> [v2:172.21.15.43:6834/2586910944,v1:172.21.15.43:6835/2586910944] conn(0x1ba5230 0x1b9a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c44c72700 1 --2- 172.21.15.40:0/3018524848 >> [v2:172.21.15.43:6834/2586910944,v1:172.21.15.43:6835/2586910944] conn(0x1ba5230 0x1b9a9f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88e4de89300ac690 server_cookie=99ab2ad10265bc95 in_seq=0 out_seq=0 2024-09-13T17:51:17.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c1f7fe700 10 client.4911 ms_handle_connect on v2:172.21.15.43:6834/2586910944 2024-09-13T17:51:17.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.682+0000 7f0c1f7fe700 1 -- 172.21.15.40:0/3018524848 <== mds.1 v2:172.21.15.43:6834/2586910944 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x19081e0 con 0x1ba5230 2024-09-13T17:51:17.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c1f7fe700 10 client.4911 handle_command_reply: tid=0 2024-09-13T17:51:17.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c4a187740 1 client.4911 shutdown 2024-09-13T17:51:17.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c4a187740 1 -- 172.21.15.40:0/3018524848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b34db0 msgr2=0x1b631c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c4a187740 1 --2- 172.21.15.40:0/3018524848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b34db0 0x1b631c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0c380546f0 tx=0x7f0c38054f80 comp rx=0 tx=0).stop 2024-09-13T17:51:17.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c4a187740 1 -- 172.21.15.40:0/3018524848 shutdown_connections 2024-09-13T17:51:17.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c4a187740 1 --2- 172.21.15.40:0/3018524848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x1b34db0 0x1b631c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c4a187740 1 --2- 172.21.15.40:0/3018524848 >> [v2:172.21.15.43:6834/2586910944,v1:172.21.15.43:6835/2586910944] conn(0x1ba5230 0x1b9a9f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c4a187740 1 --2- 172.21.15.40:0/3018524848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x1b21830 0x1afb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c4a187740 1 --2- 172.21.15.40:0/3018524848 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x1b20f10 0x1afd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c4a187740 1 -- 172.21.15.40:0/3018524848 >> 172.21.15.40:0/3018524848 conn(0x188c4d0 msgr2=0x1afa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:17.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c1f7fe700 0 client.4911 ms_handle_reset on v2:172.21.15.43:6834/2586910944 2024-09-13T17:51:17.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c4a187740 1 -- 172.21.15.40:0/3018524848 shutdown_connections 2024-09-13T17:51:17.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c4a187740 1 -- 172.21.15.40:0/3018524848 wait complete. 2024-09-13T17:51:17.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c4a187740 20 client.4911 trim_cache size 0 max 16384 2024-09-13T17:51:17.692 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:51:17.692 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:51:17.693 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 1, 2024-09-13T17:51:17.693 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4285, 2024-09-13T17:51:17.693 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-09-13T17:51:17.693 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-09-13T17:51:17.693 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-09-13T17:51:17.693 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 16.558821744999999, 2024-09-13T17:51:17.693 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 15, 2024-09-13T17:51:17.693 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 27, 2024-09-13T17:51:17.693 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 25, 2024-09-13T17:51:17.693 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 49.551532952000002 2024-09-13T17:51:17.693 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:51:17.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c45c74700 1 -- 172.21.15.40:0/1898336432 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0c2c066730 msgr2=0x7f0c2c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c45c74700 1 --2- 172.21.15.40:0/1898336432 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0c2c066730 0x7f0c2c068bf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009c40 tx=0x7f0c34008040 comp rx=0 tx=0).stop 2024-09-13T17:51:17.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c45c74700 1 -- 172.21.15.40:0/1898336432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c401515a0 msgr2=0x7f0c401f2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c45c74700 1 --2- 172.21.15.40:0/1898336432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c401515a0 0x7f0c401f2ca0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0c3800caf0 tx=0x7f0c3800cb20 comp rx=0 tx=0).stop 2024-09-13T17:51:17.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c45c74700 1 -- 172.21.15.40:0/1898336432 shutdown_connections 2024-09-13T17:51:17.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c45c74700 1 --2- 172.21.15.40:0/1898336432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0c401515a0 0x7f0c401f2ca0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c45c74700 1 --2- 172.21.15.40:0/1898336432 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0c2c066730 0x7f0c2c068bf0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c45c74700 1 --2- 172.21.15.40:0/1898336432 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0c400c3de0 0x7f0c4014e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c45c74700 1 --2- 172.21.15.40:0/1898336432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c4006dde0 0x7f0c40156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c45c74700 1 -- 172.21.15.40:0/1898336432 >> 172.21.15.40:0/1898336432 conn(0x7f0c40072070 msgr2=0x7f0c4014edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:17.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c45c74700 1 -- 172.21.15.40:0/1898336432 shutdown_connections 2024-09-13T17:51:17.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.686+0000 7f0c45c74700 1 -- 172.21.15.40:0/1898336432 wait complete. 2024-09-13T17:51:17.708 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-13T17:51:17.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.822+0000 7f49fad79700 1 Processor -- start 2024-09-13T17:51:17.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.822+0000 7f49fad79700 1 -- start start 2024-09-13T17:51:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49fad79700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49f4151050 0x7f49f4066220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49fad79700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f49f40cb2a0 0x7f49f40cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49fad79700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f41578b0 con 0x7f49f4157d40 2024-09-13T17:51:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49fad79700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49f4157090 con 0x7f49f40cb2a0 2024-09-13T17:51:17.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49fad79700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f40c6630 con 0x7f49f4151050 2024-09-13T17:51:17.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49fa578700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f49f40cb2a0 0x7f49f40cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49f9d77700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49f4151050 0x7f49f4066220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49f9576700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f49f4157d40 0x7f49f40c2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:51068/0 (socket says 172.21.15.40:51068) 2024-09-13T17:51:17.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49f9576700 1 -- 172.21.15.40:0/3957714157 learned_addr learned my addr 172.21.15.40:0/3957714157 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:17.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49f9d77700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49f4151050 0x7f49f4066220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:37010/0 (socket says 172.21.15.40:37010) 2024-09-13T17:51:17.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49f8d75700 1 -- 172.21.15.40:0/3957714157 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792063881 0 0) 0x7f49f41578b0 con 0x7f49f4157d40 2024-09-13T17:51:17.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49f8d75700 1 -- 172.21.15.40:0/3957714157 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49e00035d0 con 0x7f49f4157d40 2024-09-13T17:51:17.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49f9d77700 1 -- 172.21.15.40:0/3957714157 >> v2:172.21.15.43:3300/0 conn(0x7f49f40cb2a0 msgr2=0x7f49f40cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49f9d77700 1 --2- 172.21.15.40:0/3957714157 >> v2:172.21.15.43:3300/0 conn(0x7f49f40cb2a0 0x7f49f40cb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49f9d77700 1 -- 172.21.15.40:0/3957714157 >> v1:172.21.15.43:6789/0 conn(0x7f49f4157d40 legacy=0x7f49f40c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49f9d77700 1 -- 172.21.15.40:0/3957714157 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f406b290 con 0x7f49f4151050 2024-09-13T17:51:17.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49fa578700 1 --2- 172.21.15.40:0/3957714157 >> v2:172.21.15.43:3300/0 conn(0x7f49f40cb2a0 0x7f49f40cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:17.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49f9d77700 1 --2- 172.21.15.40:0/3957714157 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49f4151050 0x7f49f4066220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009d90 tx=0x7f49f000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a4cf6063c25307b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:17.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49f8d75700 1 -- 172.21.15.40:0/3957714157 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f000e070 con 0x7f49f4151050 2024-09-13T17:51:17.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49f8d75700 1 -- 172.21.15.40:0/3957714157 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f000bb20 con 0x7f49f4151050 2024-09-13T17:51:17.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49f8d75700 1 -- 172.21.15.40:0/3957714157 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0012430 con 0x7f49f4151050 2024-09-13T17:51:17.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49fad79700 1 -- 172.21.15.40:0/3957714157 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49f4151050 msgr2=0x7f49f4066220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49fad79700 1 --2- 172.21.15.40:0/3957714157 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49f4151050 0x7f49f4066220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009d90 tx=0x7f49f000b330 comp rx=0 tx=0).stop 2024-09-13T17:51:17.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49fad79700 1 -- 172.21.15.40:0/3957714157 shutdown_connections 2024-09-13T17:51:17.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49fad79700 1 --2- 172.21.15.40:0/3957714157 >> v2:172.21.15.43:3300/0 conn(0x7f49f40cb2a0 0x7f49f40cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49fad79700 1 --2- 172.21.15.40:0/3957714157 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49f4151050 0x7f49f4066220 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49fad79700 1 -- 172.21.15.40:0/3957714157 >> 172.21.15.40:0/3957714157 conn(0x7f49f406a390 msgr2=0x7f49f406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:17.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49fad79700 1 -- 172.21.15.40:0/3957714157 shutdown_connections 2024-09-13T17:51:17.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.826+0000 7f49fad79700 1 -- 172.21.15.40:0/3957714157 wait complete. 2024-09-13T17:51:17.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.830+0000 7f49fad79700 1 Processor -- start 2024-09-13T17:51:17.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.830+0000 7f49fad79700 1 -- start start 2024-09-13T17:51:17.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.830+0000 7f49fad79700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f40cb2a0 0x7f49f4154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.830+0000 7f49fad79700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49f4151050 0x7f49f40c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.830+0000 7f49fad79700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49f4157d40 0x7f49f4149520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.830+0000 7f49fad79700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f414cfb0 con 0x7f49f4151050 2024-09-13T17:51:17.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.830+0000 7f49fad79700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f414ba70 con 0x7f49f40cb2a0 2024-09-13T17:51:17.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.830+0000 7f49fad79700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f4150890 con 0x7f49f4157d40 2024-09-13T17:51:17.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.830+0000 7f49f9576700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49f4151050 0x7f49f40c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.830+0000 7f49f9576700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49f4151050 0x7f49f40c99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51266/0 (socket says 172.21.15.40:51266) 2024-09-13T17:51:17.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.830+0000 7f49f9576700 1 -- 172.21.15.40:0/2229449013 learned_addr learned my addr 172.21.15.40:0/2229449013 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:17.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.830+0000 7f49f9d77700 1 --2- 172.21.15.40:0/2229449013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f40cb2a0 0x7f49f4154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.830+0000 7f49fa578700 1 --2- 172.21.15.40:0/2229449013 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49f4157d40 0x7f49f4149520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.834+0000 7f49f9576700 1 -- 172.21.15.40:0/2229449013 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49f4157d40 msgr2=0x7f49f4149520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.834+0000 7f49f9576700 1 --2- 172.21.15.40:0/2229449013 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49f4157d40 0x7f49f4149520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.834+0000 7f49f9576700 1 -- 172.21.15.40:0/2229449013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f40cb2a0 msgr2=0x7f49f4154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.834+0000 7f49f9576700 1 --2- 172.21.15.40:0/2229449013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f40cb2a0 0x7f49f4154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.834+0000 7f49f9576700 1 -- 172.21.15.40:0/2229449013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec0094b0 con 0x7f49f4151050 2024-09-13T17:51:17.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.834+0000 7f49f9576700 1 --2- 172.21.15.40:0/2229449013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49f4151050 0x7f49f40c99d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f49e800c4d0 tx=0x7f49e800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:17.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.834+0000 7f49e67fc700 1 -- 172.21.15.40:0/2229449013 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8005d90 con 0x7f49f4151050 2024-09-13T17:51:17.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.834+0000 7f49fad79700 1 -- 172.21.15.40:0/2229449013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f0009a70 con 0x7f49f4151050 2024-09-13T17:51:17.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.834+0000 7f49e67fc700 1 -- 172.21.15.40:0/2229449013 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e80070d0 con 0x7f49f4151050 2024-09-13T17:51:17.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.834+0000 7f49fad79700 1 -- 172.21.15.40:0/2229449013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f414b140 con 0x7f49f4151050 2024-09-13T17:51:17.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.834+0000 7f49e67fc700 1 -- 172.21.15.40:0/2229449013 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8007440 con 0x7f49f4151050 2024-09-13T17:51:17.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.838+0000 7f49e67fc700 1 -- 172.21.15.40:0/2229449013 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f49e80075a0 con 0x7f49f4151050 2024-09-13T17:51:17.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.838+0000 7f49e67fc700 1 --2- 172.21.15.40:0/2229449013 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f49d8066760 0x7f49d8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.838+0000 7f49e67fc700 1 -- 172.21.15.40:0/2229449013 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f49e80838f0 con 0x7f49f4151050 2024-09-13T17:51:17.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.838+0000 7f49f9d77700 1 --2- 172.21.15.40:0/2229449013 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f49d8066760 0x7f49d8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.838+0000 7f49f9d77700 1 --2- 172.21.15.40:0/2229449013 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f49d8066760 0x7f49d8068c20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f49f0014040 tx=0x7f49f0011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:17.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.838+0000 7f49ff28c740 1 Processor -- start 2024-09-13T17:51:17.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.838+0000 7f49ff28c740 1 -- start start 2024-09-13T17:51:17.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49ff28c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14d9ba0 0x14c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49ff28c740 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x14c5d00 0x14c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49ff28c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14c6620 0x14c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49ff28c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x1410d80 con 0x14c6620 2024-09-13T17:51:17.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49ff28c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x140e510 con 0x14d9ba0 2024-09-13T17:51:17.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49ff28c740 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x1409e00 con 0x14c5d00 2024-09-13T17:51:17.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49fa578700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14c6620 0x14c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49f9d77700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14d9ba0 0x14c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49fa578700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14c6620 0x14c6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51276/0 (socket says 172.21.15.40:51276) 2024-09-13T17:51:17.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49f9d77700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14d9ba0 0x14c57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:37984/0 (socket says 172.21.15.40:37984) 2024-09-13T17:51:17.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49f9576700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x14c5d00 0x14c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49fa578700 1 -- 172.21.15.40:0/4085139882 learned_addr learned my addr 172.21.15.40:0/4085139882 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:17.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49fa578700 1 -- 172.21.15.40:0/4085139882 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x14c5d00 msgr2=0x14c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49fa578700 1 --2- 172.21.15.40:0/4085139882 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x14c5d00 0x14c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49fa578700 1 -- 172.21.15.40:0/4085139882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14d9ba0 msgr2=0x14c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49fa578700 1 --2- 172.21.15.40:0/4085139882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14d9ba0 0x14c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49fa578700 1 -- 172.21.15.40:0/4085139882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x127c960 con 0x14c6620 2024-09-13T17:51:17.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49f9d77700 1 --2- 172.21.15.40:0/4085139882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14d9ba0 0x14c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:17.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49fa578700 1 --2- 172.21.15.40:0/4085139882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14c6620 0x14c6a00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009e40 tx=0x7f49ec0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7460829b06f57dc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:17.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49d77fe700 1 -- 172.21.15.40:0/4085139882 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec011d10 con 0x14c6620 2024-09-13T17:51:17.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49d77fe700 1 -- 172.21.15.40:0/4085139882 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec011e70 con 0x14c6620 2024-09-13T17:51:17.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49d77fe700 1 -- 172.21.15.40:0/4085139882 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec004220 con 0x14c6620 2024-09-13T17:51:17.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49ff28c740 1 -- 172.21.15.40:0/4085139882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14c6620 msgr2=0x14c6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.842+0000 7f49ff28c740 1 --2- 172.21.15.40:0/4085139882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14c6620 0x14c6a00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009e40 tx=0x7f49ec0115a0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49ff28c740 1 -- 172.21.15.40:0/4085139882 shutdown_connections 2024-09-13T17:51:17.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49ff28c740 1 --2- 172.21.15.40:0/4085139882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14c6620 0x14c6a00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49ff28c740 1 --2- 172.21.15.40:0/4085139882 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x14c5d00 0x14c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49ff28c740 1 --2- 172.21.15.40:0/4085139882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14d9ba0 0x14c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49ff28c740 1 -- 172.21.15.40:0/4085139882 >> 172.21.15.40:0/4085139882 conn(0x12314d0 msgr2=0x14dc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:17.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49ff28c740 1 -- 172.21.15.40:0/4085139882 shutdown_connections 2024-09-13T17:51:17.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49ff28c740 1 -- 172.21.15.40:0/4085139882 wait complete. 2024-09-13T17:51:17.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49ff28c740 1 -- start start 2024-09-13T17:51:17.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49ff28c740 1 Processor -- start 2024-09-13T17:51:17.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49ff28c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14c5d00 0x14ee740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49ff28c740 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x14c6620 0x14eb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49ff28c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14d9ba0 0x14c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49ff28c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x13c7730 con 0x14d9ba0 2024-09-13T17:51:17.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49ff28c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x13cd5b0 con 0x14c5d00 2024-09-13T17:51:17.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49ff28c740 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x12b5aa0 con 0x14c6620 2024-09-13T17:51:17.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49ff28c740 10 client.0 _ll_register_callbacks cb 0x1403610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:51:17.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49fa578700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14d9ba0 0x14c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49f9d77700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14c5d00 0x14ee740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49fa578700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14d9ba0 0x14c4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51280/0 (socket says 172.21.15.40:51280) 2024-09-13T17:51:17.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49fa578700 1 -- 172.21.15.40:0/3566336545 learned_addr learned my addr 172.21.15.40:0/3566336545 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:17.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.846+0000 7f49f9576700 1 --2- 172.21.15.40:0/3566336545 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x14c6620 0x14eb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49fa578700 1 -- 172.21.15.40:0/3566336545 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x14c6620 msgr2=0x14eb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49fa578700 1 --2- 172.21.15.40:0/3566336545 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x14c6620 0x14eb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49fa578700 1 -- 172.21.15.40:0/3566336545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14c5d00 msgr2=0x14ee740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49fa578700 1 --2- 172.21.15.40:0/3566336545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14c5d00 0x14ee740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49fa578700 1 -- 172.21.15.40:0/3566336545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f0022070 con 0x14d9ba0 2024-09-13T17:51:17.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49fa578700 1 --2- 172.21.15.40:0/3566336545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14d9ba0 0x14c4ea0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x1405b60 tx=0x7f49ec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d1a87a01d1cc62 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:17.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-09-13T17:51:17.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49d4ff9700 1 -- 172.21.15.40:0/3566336545 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec022040 con 0x14d9ba0 2024-09-13T17:51:17.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49d4ff9700 1 -- 172.21.15.40:0/3566336545 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f49ec003ad0 con 0x14d9ba0 2024-09-13T17:51:17.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49ff28c740 1 -- 172.21.15.40:0/3566336545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x123e1e0 con 0x14d9ba0 2024-09-13T17:51:17.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49d4ff9700 1 -- 172.21.15.40:0/3566336545 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49ec003c90 con 0x14d9ba0 2024-09-13T17:51:17.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49ff28c740 10 client.4920 fetch_fsmap learned FSMap version 15 2024-09-13T17:51:17.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49ff28c740 1 -- 172.21.15.40:0/3566336545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x7f49ec009af0 con 0x14d9ba0 2024-09-13T17:51:17.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49d4ff9700 1 -- 172.21.15.40:0/3566336545 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x123e1e0 con 0x14d9ba0 2024-09-13T17:51:17.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49d4ff9700 1 -- 172.21.15.40:0/3566336545 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 15) v1 ==== 4236+0+0 (secure 0 0 0) 0x7f49ec01b030 con 0x14d9ba0 2024-09-13T17:51:17.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49ff28c740 10 client.4920 fetch_fsmap finished waiting for FSMap version 15 2024-09-13T17:51:17.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49ff28c740 10 client.4920 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T17:51:17.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49ff28c740 20 client.4920 populate_metadata read hostname 'smithi040' 2024-09-13T17:51:17.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49ff28c740 1 --2- 172.21.15.40:0/3566336545 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0x14a0ce0 0x14a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49ff28c740 4 client.4920 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:51:17.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49ff28c740 1 -- 172.21.15.40:0/3566336545 --> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1288460 con 0x14a0ce0 2024-09-13T17:51:17.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49f9d77700 1 --2- 172.21.15.40:0/3566336545 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0x14a0ce0 0x14a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49f9d77700 1 --2- 172.21.15.40:0/3566336545 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0x14a0ce0 0x14a10c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d9719c9bfe0e6ee server_cookie=671c601d53645164 in_seq=0 out_seq=0 2024-09-13T17:51:17.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.850+0000 7f49d4ff9700 10 client.4920 ms_handle_connect on v2:172.21.15.43:6836/334160033 2024-09-13T17:51:17.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.854+0000 7f49d4ff9700 1 -- 172.21.15.40:0/3566336545 <== mds.0 v2:172.21.15.43:6836/334160033 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x1288460 con 0x14a0ce0 2024-09-13T17:51:17.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.854+0000 7f49d4ff9700 10 client.4920 handle_command_reply: tid=0 2024-09-13T17:51:17.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.854+0000 7f49ff28c740 1 client.4920 shutdown 2024-09-13T17:51:17.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.854+0000 7f49ff28c740 1 -- 172.21.15.40:0/3566336545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14d9ba0 msgr2=0x14c4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.854+0000 7f49ff28c740 1 --2- 172.21.15.40:0/3566336545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14d9ba0 0x14c4ea0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x1405b60 tx=0x7f49ec014000 comp rx=0 tx=0).stop 2024-09-13T17:51:17.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.854+0000 7f49ff28c740 1 -- 172.21.15.40:0/3566336545 shutdown_connections 2024-09-13T17:51:17.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.854+0000 7f49ff28c740 1 --2- 172.21.15.40:0/3566336545 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0x14a0ce0 0x14a10c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.854+0000 7f49ff28c740 1 --2- 172.21.15.40:0/3566336545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14d9ba0 0x14c4ea0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.854+0000 7f49ff28c740 1 --2- 172.21.15.40:0/3566336545 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x14c6620 0x14eb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.854+0000 7f49d4ff9700 0 client.4920 ms_handle_reset on v2:172.21.15.43:6836/334160033 2024-09-13T17:51:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.854+0000 7f49ff28c740 1 --2- 172.21.15.40:0/3566336545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14c5d00 0x14ee740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.854+0000 7f49ff28c740 1 -- 172.21.15.40:0/3566336545 >> 172.21.15.40:0/3566336545 conn(0x12314d0 msgr2=0x14d9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.854+0000 7f49ff28c740 1 -- 172.21.15.40:0/3566336545 shutdown_connections 2024-09-13T17:51:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.854+0000 7f49ff28c740 1 -- 172.21.15.40:0/3566336545 wait complete. 2024-09-13T17:51:17.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.854+0000 7f49ff28c740 20 client.4920 trim_cache size 0 max 16384 2024-09-13T17:51:17.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.874+0000 7f49ff28c740 1 Processor -- start 2024-09-13T17:51:17.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.874+0000 7f49ff28c740 1 -- start start 2024-09-13T17:51:17.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.874+0000 7f49ff28c740 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x12314d0 0x149f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.874+0000 7f49ff28c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14c5d00 0x149fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.874+0000 7f49ff28c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14c6620 0x14a0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.874+0000 7f49ff28c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x13a8470 con 0x14c5d00 2024-09-13T17:51:17.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.874+0000 7f49ff28c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x14dcd80 con 0x14c6620 2024-09-13T17:51:17.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.874+0000 7f49ff28c740 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x13d44f0 con 0x12314d0 2024-09-13T17:51:17.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.874+0000 7f49f9576700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14c5d00 0x149fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.874+0000 7f49f9576700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14c5d00 0x149fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51286/0 (socket says 172.21.15.40:51286) 2024-09-13T17:51:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.874+0000 7f49f9576700 1 -- 172.21.15.40:0/2353221259 learned_addr learned my addr 172.21.15.40:0/2353221259 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.874+0000 7f49f9d77700 1 --2- 172.21.15.40:0/2353221259 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x12314d0 0x149f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.874+0000 7f49fa578700 1 --2- 172.21.15.40:0/2353221259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14c6620 0x14a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.874+0000 7f49f9576700 1 -- 172.21.15.40:0/2353221259 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x12314d0 msgr2=0x149f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.874+0000 7f49f9576700 1 --2- 172.21.15.40:0/2353221259 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x12314d0 0x149f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.874+0000 7f49f9576700 1 -- 172.21.15.40:0/2353221259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14c6620 msgr2=0x14a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.874+0000 7f49f9576700 1 --2- 172.21.15.40:0/2353221259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14c6620 0x14a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.874+0000 7f49f9576700 1 -- 172.21.15.40:0/2353221259 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x13d0220 con 0x14c5d00 2024-09-13T17:51:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.874+0000 7f49f9d77700 1 --2- 172.21.15.40:0/2353221259 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x12314d0 0x149f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.874+0000 7f49fa578700 1 --2- 172.21.15.40:0/2353221259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14c6620 0x14a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49f9576700 1 --2- 172.21.15.40:0/2353221259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14c5d00 0x149fc50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f49e8054200 tx=0x7f49e8081930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9bb3c9d204b22a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49d7fff700 1 -- 172.21.15.40:0/2353221259 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8054560 con 0x14c5d00 2024-09-13T17:51:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49d7fff700 1 -- 172.21.15.40:0/2353221259 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e8058280 con 0x14c5d00 2024-09-13T17:51:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49d7fff700 1 -- 172.21.15.40:0/2353221259 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e805c8d0 con 0x14c5d00 2024-09-13T17:51:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49ff28c740 1 -- 172.21.15.40:0/2353221259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14c5d00 msgr2=0x149fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49ff28c740 1 --2- 172.21.15.40:0/2353221259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14c5d00 0x149fc50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f49e8054200 tx=0x7f49e8081930 comp rx=0 tx=0).stop 2024-09-13T17:51:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49ff28c740 1 -- 172.21.15.40:0/2353221259 shutdown_connections 2024-09-13T17:51:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49ff28c740 1 --2- 172.21.15.40:0/2353221259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14c6620 0x14a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49ff28c740 1 --2- 172.21.15.40:0/2353221259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x14c5d00 0x149fc50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49ff28c740 1 --2- 172.21.15.40:0/2353221259 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x12314d0 0x149f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49ff28c740 1 -- 172.21.15.40:0/2353221259 >> 172.21.15.40:0/2353221259 conn(0x14c8060 msgr2=0x14c45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49ff28c740 1 -- 172.21.15.40:0/2353221259 shutdown_connections 2024-09-13T17:51:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49ff28c740 1 -- 172.21.15.40:0/2353221259 wait complete. 2024-09-13T17:51:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49ff28c740 1 -- start start 2024-09-13T17:51:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49ff28c740 1 Processor -- start 2024-09-13T17:51:17.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49ff28c740 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x12314d0 0x14a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49ff28c740 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x14c5d00 0x15477d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49ff28c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14c6620 0x14f4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49ff28c740 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x1438ec0 con 0x12314d0 2024-09-13T17:51:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49ff28c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x14e45f0 con 0x14c6620 2024-09-13T17:51:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49ff28c740 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x14edf80 con 0x14c5d00 2024-09-13T17:51:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49ff28c740 10 client.0 _ll_register_callbacks cb 0x7f49ec009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:51:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49f9d77700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x12314d0 0x14a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49f9d77700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x12314d0 0x14a3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51294/0 (socket says 172.21.15.40:51294) 2024-09-13T17:51:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49f9d77700 1 -- 172.21.15.40:0/2821062263 learned_addr learned my addr 172.21.15.40:0/2821062263 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49fa578700 1 --2- 172.21.15.40:0/2821062263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14c6620 0x14f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49f9576700 1 --2- 172.21.15.40:0/2821062263 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x14c5d00 0x15477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49f9d77700 1 -- 172.21.15.40:0/2821062263 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x14c5d00 msgr2=0x15477d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49f9d77700 1 --2- 172.21.15.40:0/2821062263 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x14c5d00 0x15477d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49f9d77700 1 -- 172.21.15.40:0/2821062263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14c6620 msgr2=0x14f4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.878+0000 7f49f9d77700 1 --2- 172.21.15.40:0/2821062263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14c6620 0x14f4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49f9d77700 1 -- 172.21.15.40:0/2821062263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f406b290 con 0x12314d0 2024-09-13T17:51:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49f9d77700 1 --2- 172.21.15.40:0/2821062263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x12314d0 0x14a3d80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f49f000d010 tx=0x7f49f0004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da278b91cbdd7e77 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49d4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2024-09-13T17:51:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49d4ff9700 1 -- 172.21.15.40:0/2821062263 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f002c040 con 0x12314d0 2024-09-13T17:51:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49d4ff9700 1 -- 172.21.15.40:0/2821062263 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f49f0036070 con 0x12314d0 2024-09-13T17:51:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49ff28c740 1 -- 172.21.15.40:0/2821062263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49ec011e70 con 0x12314d0 2024-09-13T17:51:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49d4ff9700 1 -- 172.21.15.40:0/2821062263 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49f0036230 con 0x12314d0 2024-09-13T17:51:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49ff28c740 10 client.4926 fetch_fsmap learned FSMap version 15 2024-09-13T17:51:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49ff28c740 1 -- 172.21.15.40:0/2821062263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x7f49ec009af0 con 0x12314d0 2024-09-13T17:51:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49d4ff9700 1 -- 172.21.15.40:0/2821062263 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec011e70 con 0x12314d0 2024-09-13T17:51:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49d4ff9700 1 -- 172.21.15.40:0/2821062263 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 15) v1 ==== 4236+0+0 (secure 0 0 0) 0x7f49f003d020 con 0x12314d0 2024-09-13T17:51:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49ff28c740 10 client.4926 fetch_fsmap finished waiting for FSMap version 15 2024-09-13T17:51:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49ff28c740 10 client.4926 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T17:51:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49ff28c740 20 client.4926 populate_metadata read hostname 'smithi040' 2024-09-13T17:51:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49ff28c740 1 --2- 172.21.15.40:0/2821062263 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0x14d9ba0 0x14f3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49ff28c740 4 client.4926 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T17:51:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49ff28c740 1 -- 172.21.15.40:0/2821062263 --> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] -- command(tid 0: {"prefix": "status"}) v1 -- 0x12ad1e0 con 0x14d9ba0 2024-09-13T17:51:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49f9576700 1 --2- 172.21.15.40:0/2821062263 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0x14d9ba0 0x14f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49d4ff9700 10 client.4926 ms_handle_connect on v2:172.21.15.43:6836/334160033 2024-09-13T17:51:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49f9576700 1 --2- 172.21.15.40:0/2821062263 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0x14d9ba0 0x14f3550 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb1655b80254bc48 server_cookie=8c1062955f315aee in_seq=0 out_seq=0 2024-09-13T17:51:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49d4ff9700 1 -- 172.21.15.40:0/2821062263 <== mds.0 v2:172.21.15.43:6836/334160033 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x12ad1e0 con 0x14d9ba0 2024-09-13T17:51:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49d4ff9700 10 client.4926 handle_command_reply: tid=0 2024-09-13T17:51:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.882+0000 7f49ff28c740 1 client.4926 shutdown 2024-09-13T17:51:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49ff28c740 1 -- 172.21.15.40:0/2821062263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x12314d0 msgr2=0x14a3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49ff28c740 1 --2- 172.21.15.40:0/2821062263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x12314d0 0x14a3d80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f49f000d010 tx=0x7f49f0004b60 comp rx=0 tx=0).stop 2024-09-13T17:51:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49ff28c740 1 -- 172.21.15.40:0/2821062263 shutdown_connections 2024-09-13T17:51:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49ff28c740 1 --2- 172.21.15.40:0/2821062263 >> [v2:172.21.15.43:6836/334160033,v1:172.21.15.43:6837/334160033] conn(0x14d9ba0 0x14f3550 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49ff28c740 1 --2- 172.21.15.40:0/2821062263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x14c6620 0x14f4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49ff28c740 1 --2- 172.21.15.40:0/2821062263 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x14c5d00 0x15477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49ff28c740 1 --2- 172.21.15.40:0/2821062263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x12314d0 0x14a3d80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49d4ff9700 0 client.4926 ms_handle_reset on v2:172.21.15.43:6836/334160033 2024-09-13T17:51:17.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49ff28c740 1 -- 172.21.15.40:0/2821062263 >> 172.21.15.40:0/2821062263 conn(0x14c8060 msgr2=0x14a30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:17.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49ff28c740 1 -- 172.21.15.40:0/2821062263 shutdown_connections 2024-09-13T17:51:17.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49ff28c740 1 -- 172.21.15.40:0/2821062263 wait complete. 2024-09-13T17:51:17.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49ff28c740 20 client.4926 trim_cache size 0 max 16384 2024-09-13T17:51:17.892 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:51:17.892 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:51:17.892 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2024-09-13T17:51:17.893 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4288, 2024-09-13T17:51:17.893 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2024-09-13T17:51:17.893 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2024-09-13T17:51:17.893 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2024-09-13T17:51:17.893 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 43.321237826999997, 2024-09-13T17:51:17.893 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 15, 2024-09-13T17:51:17.893 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 27, 2024-09-13T17:51:17.893 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 25, 2024-09-13T17:51:17.893 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 49.748625445000002 2024-09-13T17:51:17.894 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:51:17.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49fad79700 1 -- 172.21.15.40:0/2229449013 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f49d8066760 msgr2=0x7f49d8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49fad79700 1 --2- 172.21.15.40:0/2229449013 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f49d8066760 0x7f49d8068c20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f49f0014040 tx=0x7f49f0011000 comp rx=0 tx=0).stop 2024-09-13T17:51:17.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49fad79700 1 -- 172.21.15.40:0/2229449013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49f4151050 msgr2=0x7f49f40c99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:17.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49fad79700 1 --2- 172.21.15.40:0/2229449013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49f4151050 0x7f49f40c99d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f49e800c4d0 tx=0x7f49e800c7e0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49fad79700 1 -- 172.21.15.40:0/2229449013 shutdown_connections 2024-09-13T17:51:17.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49fad79700 1 --2- 172.21.15.40:0/2229449013 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f49d8066760 0x7f49d8068c20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49fad79700 1 --2- 172.21.15.40:0/2229449013 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49f4157d40 0x7f49f4149520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49fad79700 1 --2- 172.21.15.40:0/2229449013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49f4151050 0x7f49f40c99d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49fad79700 1 --2- 172.21.15.40:0/2229449013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f40cb2a0 0x7f49f4154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:17.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49fad79700 1 -- 172.21.15.40:0/2229449013 >> 172.21.15.40:0/2229449013 conn(0x7f49f406a390 msgr2=0x7f49f406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:17.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49fad79700 1 -- 172.21.15.40:0/2229449013 shutdown_connections 2024-09-13T17:51:17.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:17.886+0000 7f49fad79700 1 -- 172.21.15.40:0/2229449013 wait complete. 2024-09-13T17:51:17.907 INFO:teuthology.run_tasks:Running task sequential... 2024-09-13T17:51:17.917 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-13T17:51:17.917 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-13T17:51:17.918 INFO:teuthology.task.print:Enabling mgr modules 2024-09-13T17:51:17.918 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-13T17:51:17.960 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-13T17:51:17.960 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-13T17:51:17.960 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-13T17:51:17.960 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T17:51:17.961 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T17:51:17.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:51:17.961 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-09-13T17:51:17.969 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-13T17:51:17.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:51:17.969 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-13T17:51:17.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:51:17.985 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-09-13T17:51:18.119 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-13T17:51:18.119 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-13T17:51:18.120 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:51:18.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.238+0000 7f5ba70c4700 1 Processor -- start 2024-09-13T17:51:18.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.238+0000 7f5ba70c4700 1 -- start start 2024-09-13T17:51:18.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.238+0000 7f5ba70c4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5ba015cbc0 0x7f5ba006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:18.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.238+0000 7f5ba70c4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5ba015cfa0 0x7f5ba015d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:18.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.238+0000 7f5ba70c4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5ba0153120 0x7f5ba0153590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:18.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.238+0000 7f5ba70c4700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba006eb10 con 0x7f5ba015cfa0 2024-09-13T17:51:18.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.238+0000 7f5ba70c4700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba0153ad0 con 0x7f5ba0153120 2024-09-13T17:51:18.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.238+0000 7f5ba70c4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba00c7bb0 con 0x7f5ba015cbc0 2024-09-13T17:51:18.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.238+0000 7f5ba58c1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5ba015cfa0 0x7f5ba015d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:18.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.238+0000 7f5ba58c1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5ba015cfa0 0x7f5ba015d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51306/0 (socket says 172.21.15.40:51306) 2024-09-13T17:51:18.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.238+0000 7f5ba58c1700 1 -- 172.21.15.40:0/1598213977 learned_addr learned my addr 172.21.15.40:0/1598213977 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:18.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.238+0000 7f5ba60c2700 1 --2- 172.21.15.40:0/1598213977 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5ba015cbc0 0x7f5ba006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:18.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.238+0000 7f5ba68c3700 1 --2- 172.21.15.40:0/1598213977 >> v2:172.21.15.43:3300/0 conn(0x7f5ba0153120 0x7f5ba0153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:18.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.242+0000 7f5ba58c1700 1 -- 172.21.15.40:0/1598213977 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5ba015cbc0 msgr2=0x7f5ba006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:18.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.242+0000 7f5ba58c1700 1 --2- 172.21.15.40:0/1598213977 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5ba015cbc0 0x7f5ba006e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:18.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.242+0000 7f5ba58c1700 1 -- 172.21.15.40:0/1598213977 >> v2:172.21.15.43:3300/0 conn(0x7f5ba0153120 msgr2=0x7f5ba0153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:18.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.242+0000 7f5ba58c1700 1 --2- 172.21.15.40:0/1598213977 >> v2:172.21.15.43:3300/0 conn(0x7f5ba0153120 0x7f5ba0153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:18.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.242+0000 7f5ba58c1700 1 -- 172.21.15.40:0/1598213977 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba006b4c0 con 0x7f5ba015cfa0 2024-09-13T17:51:18.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.242+0000 7f5ba68c3700 1 --2- 172.21.15.40:0/1598213977 >> v2:172.21.15.43:3300/0 conn(0x7f5ba0153120 0x7f5ba0153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:18.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.242+0000 7f5ba58c1700 1 --2- 172.21.15.40:0/1598213977 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ba015cfa0 0x7f5ba015d3a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5ba00b3030 tx=0x7f5b90009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40f66316dc2ab4a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:18.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.242+0000 7f5ba50c0700 1 -- 172.21.15.40:0/1598213977 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90011830 con 0x7f5ba015cfa0 2024-09-13T17:51:18.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.242+0000 7f5ba50c0700 1 -- 172.21.15.40:0/1598213977 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b90011990 con 0x7f5ba015cfa0 2024-09-13T17:51:18.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.242+0000 7f5ba50c0700 1 -- 172.21.15.40:0/1598213977 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90011d20 con 0x7f5ba015cfa0 2024-09-13T17:51:18.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.242+0000 7f5ba70c4700 1 -- 172.21.15.40:0/1598213977 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ba015cfa0 msgr2=0x7f5ba015d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:18.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.242+0000 7f5ba70c4700 1 --2- 172.21.15.40:0/1598213977 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ba015cfa0 0x7f5ba015d3a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5ba00b3030 tx=0x7f5b90009bf0 comp rx=0 tx=0).stop 2024-09-13T17:51:18.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.242+0000 7f5ba70c4700 1 -- 172.21.15.40:0/1598213977 shutdown_connections 2024-09-13T17:51:18.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.242+0000 7f5ba70c4700 1 --2- 172.21.15.40:0/1598213977 >> v2:172.21.15.43:3300/0 conn(0x7f5ba0153120 0x7f5ba0153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:18.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.242+0000 7f5ba70c4700 1 --2- 172.21.15.40:0/1598213977 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ba015cfa0 0x7f5ba015d3a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:18.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.242+0000 7f5ba70c4700 1 --2- 172.21.15.40:0/1598213977 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5ba015cbc0 0x7f5ba006e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:18.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.242+0000 7f5ba70c4700 1 -- 172.21.15.40:0/1598213977 >> 172.21.15.40:0/1598213977 conn(0x7f5ba00ccef0 msgr2=0x7f5ba00cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:18.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.242+0000 7f5ba70c4700 1 -- 172.21.15.40:0/1598213977 shutdown_connections 2024-09-13T17:51:18.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.242+0000 7f5ba70c4700 1 -- 172.21.15.40:0/1598213977 wait complete. 2024-09-13T17:51:18.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.242+0000 7f5ba70c4700 1 Processor -- start 2024-09-13T17:51:18.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba70c4700 1 -- start start 2024-09-13T17:51:18.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba70c4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ba0153120 0x7f5ba0151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:18.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba70c4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ba015cbc0 0x7f5ba0157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:18.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba70c4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5ba015cfa0 0x7f5ba00cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:18.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba70c4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba01eef10 con 0x7f5ba015cbc0 2024-09-13T17:51:18.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba70c4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba01f2f20 con 0x7f5ba0153120 2024-09-13T17:51:18.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba70c4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba01ef1b0 con 0x7f5ba015cfa0 2024-09-13T17:51:18.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba58c1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ba015cbc0 0x7f5ba0157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:18.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba60c2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ba0153120 0x7f5ba0151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:18.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba58c1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ba015cbc0 0x7f5ba0157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51310/0 (socket says 172.21.15.40:51310) 2024-09-13T17:51:18.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba58c1700 1 -- 172.21.15.40:0/1109131882 learned_addr learned my addr 172.21.15.40:0/1109131882 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:18.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba68c3700 1 --2- 172.21.15.40:0/1109131882 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5ba015cfa0 0x7f5ba00cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:18.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba58c1700 1 -- 172.21.15.40:0/1109131882 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5ba015cfa0 msgr2=0x7f5ba00cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:18.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba58c1700 1 --2- 172.21.15.40:0/1109131882 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5ba015cfa0 0x7f5ba00cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:18.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba58c1700 1 -- 172.21.15.40:0/1109131882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ba0153120 msgr2=0x7f5ba0151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:18.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba58c1700 1 --2- 172.21.15.40:0/1109131882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ba0153120 0x7f5ba0151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:18.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba58c1700 1 -- 172.21.15.40:0/1109131882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b90009570 con 0x7f5ba015cbc0 2024-09-13T17:51:18.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba68c3700 1 --2- 172.21.15.40:0/1109131882 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5ba015cfa0 0x7f5ba00cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:18.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba58c1700 1 --2- 172.21.15.40:0/1109131882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ba015cbc0 0x7f5ba0157f70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5b90000c00 tx=0x7f5b90004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:18.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5b96ffd700 1 -- 172.21.15.40:0/1109131882 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90004520 con 0x7f5ba015cbc0 2024-09-13T17:51:18.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5b96ffd700 1 -- 172.21.15.40:0/1109131882 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9001e070 con 0x7f5ba015cbc0 2024-09-13T17:51:18.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba70c4700 1 -- 172.21.15.40:0/1109131882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba01ef490 con 0x7f5ba015cbc0 2024-09-13T17:51:18.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5b96ffd700 1 -- 172.21.15.40:0/1109131882 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90003900 con 0x7f5ba015cbc0 2024-09-13T17:51:18.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.246+0000 7f5ba70c4700 1 -- 172.21.15.40:0/1109131882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba01f3200 con 0x7f5ba015cbc0 2024-09-13T17:51:18.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.250+0000 7f5ba70c4700 1 -- 172.21.15.40:0/1109131882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba006c3b0 con 0x7f5ba015cbc0 2024-09-13T17:51:18.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.250+0000 7f5b96ffd700 1 -- 172.21.15.40:0/1109131882 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5b90003a60 con 0x7f5ba015cbc0 2024-09-13T17:51:18.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.250+0000 7f5b96ffd700 1 --2- 172.21.15.40:0/1109131882 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5b84066730 0x7f5b84068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:18.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.250+0000 7f5b96ffd700 1 -- 172.21.15.40:0/1109131882 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5b90020070 con 0x7f5ba015cbc0 2024-09-13T17:51:18.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.250+0000 7f5ba60c2700 1 --2- 172.21.15.40:0/1109131882 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5b84066730 0x7f5b84068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:18.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.250+0000 7f5ba60c2700 1 --2- 172.21.15.40:0/1109131882 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5b84066730 0x7f5b84068bf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009bf0 tx=0x7f5b9c009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:18.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.258+0000 7f5b96ffd700 1 -- 172.21.15.40:0/1109131882 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5b90052940 con 0x7f5ba015cbc0 2024-09-13T17:51:18.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.466+0000 7f5ba70c4700 1 -- 172.21.15.40:0/1109131882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5ba014b2d0 con 0x7f5ba015cbc0 2024-09-13T17:51:18.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.466+0000 7f5b96ffd700 1 -- 172.21.15.40:0/1109131882 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+88 (secure 0 0 0) 0x7f5b90055f60 con 0x7f5ba015cbc0 2024-09-13T17:51:18.473 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-13T17:51:18.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.474+0000 7f5ba70c4700 1 -- 172.21.15.40:0/1109131882 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5b84066730 msgr2=0x7f5b84068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:18.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.474+0000 7f5ba70c4700 1 --2- 172.21.15.40:0/1109131882 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5b84066730 0x7f5b84068bf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009bf0 tx=0x7f5b9c009240 comp rx=0 tx=0).stop 2024-09-13T17:51:18.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.474+0000 7f5ba70c4700 1 -- 172.21.15.40:0/1109131882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ba015cbc0 msgr2=0x7f5ba0157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:18.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.474+0000 7f5ba70c4700 1 --2- 172.21.15.40:0/1109131882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ba015cbc0 0x7f5ba0157f70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5b90000c00 tx=0x7f5b90004770 comp rx=0 tx=0).stop 2024-09-13T17:51:18.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.474+0000 7f5ba70c4700 1 -- 172.21.15.40:0/1109131882 shutdown_connections 2024-09-13T17:51:18.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.474+0000 7f5ba70c4700 1 --2- 172.21.15.40:0/1109131882 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5b84066730 0x7f5b84068bf0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:18.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.474+0000 7f5ba70c4700 1 --2- 172.21.15.40:0/1109131882 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5ba015cfa0 0x7f5ba00cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:18.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.474+0000 7f5ba70c4700 1 --2- 172.21.15.40:0/1109131882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ba015cbc0 0x7f5ba0157f70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:18.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.474+0000 7f5ba70c4700 1 --2- 172.21.15.40:0/1109131882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ba0153120 0x7f5ba0151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:18.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.474+0000 7f5ba70c4700 1 -- 172.21.15.40:0/1109131882 >> 172.21.15.40:0/1109131882 conn(0x7f5ba00ccef0 msgr2=0x7f5ba006f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:18.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.474+0000 7f5ba70c4700 1 -- 172.21.15.40:0/1109131882 shutdown_connections 2024-09-13T17:51:18.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:18.474+0000 7f5ba70c4700 1 -- 172.21.15.40:0/1109131882 wait complete. 2024-09-13T17:51:18.493 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T17:51:18.494 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:51:18.494 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-09-13T17:51:18.494 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T17:51:18.494 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T17:51:18.494 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T17:51:18.494 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:51:18.494 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:51:18.494 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:51:18.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:51:18.494 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-09-13T17:51:18.502 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:51:18.502 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:51:18.502 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:51:18.503 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:51:18.503 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-09-13T17:51:18.503 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:51:18.503 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:51:18.503 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:51:18.503 INFO:teuthology.orchestra.run.smithi040.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:51:18.503 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-09-13T17:51:18.503 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T17:51:18.503 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42516sec preferred_lft 42516sec 2024-09-13T17:51:18.503 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link 2024-09-13T17:51:18.503 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:51:18.503 INFO:teuthology.orchestra.run.smithi040.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:51:18.504 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:51:18.504 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:51:18.504 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-09-13T17:51:18.504 INFO:teuthology.orchestra.run.smithi040.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T17:51:18.504 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 02:42:8a:21:55:c6 brd ff:ff:ff:ff:ff:ff 2024-09-13T17:51:18.504 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T17:51:18.504 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:51:18.504 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T17:51:18.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:51:18.505 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-09-13T17:51:18.505 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-09-13T17:51:18.505 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add name ceph-brx type bridge 2024-09-13T17:51:18.505 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr flush dev ceph-brx 2024-09-13T17:51:18.505 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set ceph-brx up 2024-09-13T17:51:18.505 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T17:51:18.505 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-09-13T17:51:18.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:51:18.591 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:51:18.602 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-09-13T17:51:18.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:51:18.604 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-09-13T17:51:18.611 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-09-13T17:51:18.611 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:51:18.611 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T17:51:18.611 INFO:teuthology.orchestra.run.smithi040.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T17:51:18.612 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T17:51:18.612 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:51:18.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:51:18.612 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-09-13T17:51:18.612 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-09-13T17:51:18.612 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T17:51:18.612 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T17:51:18.613 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T17:51:18.613 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-09-13T17:51:18.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:51:18.693 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-09-13T17:51:18.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:51:18.699 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2024-09-13T17:51:18.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:51:18.705 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-09-13T17:51:18.705 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-09-13T17:51:18.706 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:51:18.706 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-13T17:51:18.706 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-09-13T17:51:18.738 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-13T17:51:18.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:51:18.739 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-09-13T17:51:18.739 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-09-13T17:51:18.739 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-13T17:51:18.739 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T17:51:18.739 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-13T17:51:18.739 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-13T17:51:18.739 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-13T17:51:18.739 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-09-13T17:51:18.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:51:18.841 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2024-09-13T17:51:18.841 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-09-13T17:51:18.841 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-09-13T17:51:18.841 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T17:51:18.841 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-09-13T17:51:18.867 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-13T17:51:18.868 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:51:18.868 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T17:51:18.875 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:51:18.875 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:51:18.875 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:51:18.880 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2024-09-13T17:51:18.894 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:51:18.894 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:51:18.955 INFO:teuthology.orchestra.run.smithi040.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:51:18.956 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:51:18.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:51:18.957 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2024-09-13T17:51:18.962 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T17:51:18.962 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-13T17:51:18.964 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2024-09-13T17:51:18.980 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:51:18.980 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:51:19.036 INFO:teuthology.orchestra.run.smithi040.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:51:19.037 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:51:19.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:51:19.038 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2024-09-13T17:51:19.053 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:2024-09-13T17:51:19.046+0000 7f92490c4640 -1 init, newargv = 0x55f14cd10440 newargc=15 2024-09-13T17:51:19.053 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:ceph-fuse[35133]: starting ceph client 2024-09-13T17:51:19.064 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:ceph-fuse[35133]: starting fuse 2024-09-13T17:51:20.145 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2024-09-13T17:51:20.164 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:51:20.164 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:51:20.180 INFO:teuthology.orchestra.run.smithi040.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:51:20.181 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:51:20.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:51:20.181 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2024-09-13T17:51:20.188 INFO:teuthology.orchestra.run.smithi040.stdout:52 2024-09-13T17:51:20.189 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-09-13T17:51:20.189 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:51:20.189 DEBUG:teuthology.orchestra.run.smithi040:> import glob 2024-09-13T17:51:20.189 DEBUG:teuthology.orchestra.run.smithi040:> import re 2024-09-13T17:51:20.189 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-09-13T17:51:20.189 DEBUG:teuthology.orchestra.run.smithi040:> import subprocess 2024-09-13T17:51:20.189 DEBUG:teuthology.orchestra.run.smithi040:> 2024-09-13T17:51:20.189 DEBUG:teuthology.orchestra.run.smithi040:> def _find_admin_socket(client_name): 2024-09-13T17:51:20.189 DEBUG:teuthology.orchestra.run.smithi040:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T17:51:20.190 DEBUG:teuthology.orchestra.run.smithi040:> files = glob.glob(asok_path) 2024-09-13T17:51:20.190 DEBUG:teuthology.orchestra.run.smithi040:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T17:51:20.190 DEBUG:teuthology.orchestra.run.smithi040:> 2024-09-13T17:51:20.190 DEBUG:teuthology.orchestra.run.smithi040:> # Given a non-glob path, it better be there 2024-09-13T17:51:20.190 DEBUG:teuthology.orchestra.run.smithi040:> if "*" not in asok_path: 2024-09-13T17:51:20.190 DEBUG:teuthology.orchestra.run.smithi040:> assert(len(files) == 1) 2024-09-13T17:51:20.190 DEBUG:teuthology.orchestra.run.smithi040:> return files[0] 2024-09-13T17:51:20.190 DEBUG:teuthology.orchestra.run.smithi040:> 2024-09-13T17:51:20.190 DEBUG:teuthology.orchestra.run.smithi040:> for f in files: 2024-09-13T17:51:20.190 DEBUG:teuthology.orchestra.run.smithi040:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:51:20.191 DEBUG:teuthology.orchestra.run.smithi040:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:51:20.191 DEBUG:teuthology.orchestra.run.smithi040:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:51:20.191 DEBUG:teuthology.orchestra.run.smithi040:> contents = proc_f.read() 2024-09-13T17:51:20.191 DEBUG:teuthology.orchestra.run.smithi040:> if mountpoint in contents: 2024-09-13T17:51:20.191 DEBUG:teuthology.orchestra.run.smithi040:> return f 2024-09-13T17:51:20.191 DEBUG:teuthology.orchestra.run.smithi040:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:51:20.191 DEBUG:teuthology.orchestra.run.smithi040:> 2024-09-13T17:51:20.191 DEBUG:teuthology.orchestra.run.smithi040:> print(_find_admin_socket("client.0")) 2024-09-13T17:51:20.191 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-09-13T17:51:20.317 INFO:teuthology.orchestra.run.smithi040.stdout:/var/run/ceph/ceph-client.0.35133.asok 2024-09-13T17:51:20.501 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35133.asok 2024-09-13T17:51:20.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:51:20.502 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35133.asok status 2024-09-13T17:51:20.607 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:51:20.607 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": { 2024-09-13T17:51:20.607 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:51:20.607 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:51:20.607 INFO:teuthology.orchestra.run.smithi040.stdout: "entity_id": "0", 2024-09-13T17:51:20.607 INFO:teuthology.orchestra.run.smithi040.stdout: "hostname": "smithi040", 2024-09-13T17:51:20.607 INFO:teuthology.orchestra.run.smithi040.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-13T17:51:20.607 INFO:teuthology.orchestra.run.smithi040.stdout: "pid": "35133", 2024-09-13T17:51:20.607 INFO:teuthology.orchestra.run.smithi040.stdout: "root": "/" 2024-09-13T17:51:20.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:20.608 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_count": 0, 2024-09-13T17:51:20.608 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_pinned_count": 0, 2024-09-13T17:51:20.608 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4938, 2024-09-13T17:51:20.608 INFO:teuthology.orchestra.run.smithi040.stdout: "inst": { 2024-09-13T17:51:20.608 INFO:teuthology.orchestra.run.smithi040.stdout: "name": { 2024-09-13T17:51:20.608 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-09-13T17:51:20.608 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4938 2024-09-13T17:51:20.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:20.608 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2024-09-13T17:51:20.608 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:51:20.608 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:51:20.608 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1985053690 2024-09-13T17:51:20.608 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:20.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2024-09-13T17:51:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:51:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:51:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 1985053690 2024-09-13T17:51:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: "inst_str": "client.4938 192.168.0.1:0/1985053690", 2024-09-13T17:51:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: "addr_str": "192.168.0.1:0/1985053690", 2024-09-13T17:51:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: "inode_count": 1, 2024-09-13T17:51:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_epoch": 15, 2024-09-13T17:51:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch": 27, 2024-09-13T17:51:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:51:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklisted": false, 2024-09-13T17:51:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs" 2024-09-13T17:51:20.610 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:51:20.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:51:20.618 DEBUG:teuthology.orchestra.run.smithi040:> stat --file-system '--printf=%T 2024-09-13T17:51:20.618 DEBUG:teuthology.orchestra.run.smithi040:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-13T17:51:20.624 INFO:teuthology.orchestra.run.smithi040.stdout:fuseblk 2024-09-13T17:51:20.624 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-13T17:51:20.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:51:20.624 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:51:20.638 INFO:teuthology.run_tasks:Running task mon_thrash... 2024-09-13T17:51:20.653 INFO:tasks.mon_thrash:Beginning mon_thrash... 2024-09-13T17:51:20.653 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:51:20.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.798+0000 7f0811daf700 1 Processor -- start 2024-09-13T17:51:20.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.798+0000 7f0811daf700 1 -- start start 2024-09-13T17:51:20.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.798+0000 7f0811daf700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f080c0c7390 0x7f080c0c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:20.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.798+0000 7f0811daf700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f080c0c5560 0x7f080c0c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:20.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.798+0000 7f0811daf700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f080c0c50d0 con 0x7f080c1512e0 2024-09-13T17:51:20.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.798+0000 7f0811daf700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f080c066760 con 0x7f080c0c7390 2024-09-13T17:51:20.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.798+0000 7f0811daf700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f080c066220 con 0x7f080c0c5560 2024-09-13T17:51:20.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.798+0000 7f080bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f080c0c7390 0x7f080c0c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:20.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.798+0000 7f0810dad700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f080c1512e0 0x7f080c1570c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:51074/0 (socket says 172.21.15.40:51074) 2024-09-13T17:51:20.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.798+0000 7f0810dad700 1 -- 172.21.15.40:0/2852026798 learned_addr learned my addr 172.21.15.40:0/2852026798 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:20.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.798+0000 7f08115ae700 1 --2- 172.21.15.40:0/2852026798 >> v2:172.21.15.43:3300/0 conn(0x7f080c0c5560 0x7f080c0c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:20.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.802+0000 7f080bfff700 1 -- 172.21.15.40:0/2852026798 >> v2:172.21.15.43:3300/0 conn(0x7f080c0c5560 msgr2=0x7f080c0c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:20.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.802+0000 7f080bfff700 1 --2- 172.21.15.40:0/2852026798 >> v2:172.21.15.43:3300/0 conn(0x7f080c0c5560 0x7f080c0c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:20.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.802+0000 7f080bfff700 1 -- 172.21.15.40:0/2852026798 >> v1:172.21.15.43:6789/0 conn(0x7f080c1512e0 legacy=0x7f080c1570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:20.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.802+0000 7f080b7fe700 1 -- 172.21.15.40:0/2852026798 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 642036778 0 0) 0x7f080c0c50d0 con 0x7f080c1512e0 2024-09-13T17:51:20.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.802+0000 7f080bfff700 1 -- 172.21.15.40:0/2852026798 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080c066480 con 0x7f080c0c7390 2024-09-13T17:51:20.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.802+0000 7f080bfff700 1 --2- 172.21.15.40:0/2852026798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f080c0c7390 0x7f080c0c7a70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f080c0c7770 tx=0x7f0800009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383b59129fbd3579 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:20.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.802+0000 7f080b7fe700 1 -- 172.21.15.40:0/2852026798 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800011830 con 0x7f080c0c7390 2024-09-13T17:51:20.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.802+0000 7f080b7fe700 1 -- 172.21.15.40:0/2852026798 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0800011990 con 0x7f080c0c7390 2024-09-13T17:51:20.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.802+0000 7f080b7fe700 1 -- 172.21.15.40:0/2852026798 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800011d20 con 0x7f080c0c7390 2024-09-13T17:51:20.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.802+0000 7f0811daf700 1 -- 172.21.15.40:0/2852026798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f080c0c7390 msgr2=0x7f080c0c7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:20.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.802+0000 7f0811daf700 1 --2- 172.21.15.40:0/2852026798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f080c0c7390 0x7f080c0c7a70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f080c0c7770 tx=0x7f0800009b30 comp rx=0 tx=0).stop 2024-09-13T17:51:20.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.802+0000 7f0811daf700 1 -- 172.21.15.40:0/2852026798 shutdown_connections 2024-09-13T17:51:20.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.802+0000 7f0811daf700 1 --2- 172.21.15.40:0/2852026798 >> v2:172.21.15.43:3300/0 conn(0x7f080c0c5560 0x7f080c0c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:20.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.802+0000 7f0811daf700 1 --2- 172.21.15.40:0/2852026798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f080c0c7390 0x7f080c0c7a70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:20.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.802+0000 7f0811daf700 1 -- 172.21.15.40:0/2852026798 >> 172.21.15.40:0/2852026798 conn(0x7f080c06a390 msgr2=0x7f080c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:20.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.802+0000 7f0811daf700 1 -- 172.21.15.40:0/2852026798 shutdown_connections 2024-09-13T17:51:20.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.802+0000 7f0811daf700 1 -- 172.21.15.40:0/2852026798 wait complete. 2024-09-13T17:51:20.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.806+0000 7f0811daf700 1 Processor -- start 2024-09-13T17:51:20.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.806+0000 7f0811daf700 1 -- start start 2024-09-13T17:51:20.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.806+0000 7f0811daf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f080c0c5560 0x7f080c0ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:20.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.806+0000 7f0811daf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f080c0c7390 0x7f080c0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:20.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.806+0000 7f0811daf700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f080c1512e0 0x7f080c14dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:20.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.806+0000 7f0811daf700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c0cb7c0 con 0x7f080c0c7390 2024-09-13T17:51:20.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.806+0000 7f0811daf700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c0c9de0 con 0x7f080c0c5560 2024-09-13T17:51:20.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.806+0000 7f0811daf700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080c1f0c00 con 0x7f080c1512e0 2024-09-13T17:51:20.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.810+0000 7f080bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f080c0c7390 0x7f080c0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:20.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.810+0000 7f0810dad700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f080c0c5560 0x7f080c0ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:20.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.810+0000 7f080bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f080c0c7390 0x7f080c0ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51336/0 (socket says 172.21.15.40:51336) 2024-09-13T17:51:20.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.810+0000 7f080bfff700 1 -- 172.21.15.40:0/2888028314 learned_addr learned my addr 172.21.15.40:0/2888028314 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:20.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.810+0000 7f08115ae700 1 --2- 172.21.15.40:0/2888028314 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f080c1512e0 0x7f080c14dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:20.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.810+0000 7f080bfff700 1 -- 172.21.15.40:0/2888028314 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f080c1512e0 msgr2=0x7f080c14dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:20.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.810+0000 7f080bfff700 1 --2- 172.21.15.40:0/2888028314 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f080c1512e0 0x7f080c14dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:20.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.810+0000 7f080bfff700 1 -- 172.21.15.40:0/2888028314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f080c0c5560 msgr2=0x7f080c0ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:20.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.810+0000 7f080bfff700 1 --2- 172.21.15.40:0/2888028314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f080c0c5560 0x7f080c0ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:20.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.810+0000 7f080bfff700 1 -- 172.21.15.40:0/2888028314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08000094b0 con 0x7f080c0c7390 2024-09-13T17:51:20.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.810+0000 7f080bfff700 1 --2- 172.21.15.40:0/2888028314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f080c0c7390 0x7f080c0ca7b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0800009b70 tx=0x7f0800004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:20.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.810+0000 7f08097fa700 1 -- 172.21.15.40:0/2888028314 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800004550 con 0x7f080c0c7390 2024-09-13T17:51:20.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.810+0000 7f08097fa700 1 -- 172.21.15.40:0/2888028314 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080001f070 con 0x7f080c0c7390 2024-09-13T17:51:20.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.810+0000 7f0811daf700 1 -- 172.21.15.40:0/2888028314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f080c1e4320 con 0x7f080c0c7390 2024-09-13T17:51:20.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.810+0000 7f08097fa700 1 -- 172.21.15.40:0/2888028314 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08000038e0 con 0x7f080c0c7390 2024-09-13T17:51:20.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.810+0000 7f0811daf700 1 -- 172.21.15.40:0/2888028314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080c1e48e0 con 0x7f080c0c7390 2024-09-13T17:51:20.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.810+0000 7f0811daf700 1 -- 172.21.15.40:0/2888028314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f080c0c4610 con 0x7f080c0c7390 2024-09-13T17:51:20.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.810+0000 7f08097fa700 1 -- 172.21.15.40:0/2888028314 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0800003a40 con 0x7f080c0c7390 2024-09-13T17:51:20.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.810+0000 7f08097fa700 1 --2- 172.21.15.40:0/2888028314 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f07f8066760 0x7f07f8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:20.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.814+0000 7f08097fa700 1 -- 172.21.15.40:0/2888028314 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0800021070 con 0x7f080c0c7390 2024-09-13T17:51:20.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.814+0000 7f0810dad700 1 --2- 172.21.15.40:0/2888028314 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f07f8066760 0x7f07f8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:20.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.814+0000 7f0810dad700 1 --2- 172.21.15.40:0/2888028314 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f07f8066760 0x7f07f8068c20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f07fc007b30 tx=0x7f07fc007a80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:20.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.822+0000 7f08097fa700 1 -- 172.21.15.40:0/2888028314 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0800052cb0 con 0x7f080c0c7390 2024-09-13T17:51:20.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.994+0000 7f0811daf700 1 -- 172.21.15.40:0/2888028314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f080c0c9f70 con 0x7f080c0c7390 2024-09-13T17:51:21.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.994+0000 7f08097fa700 1 -- 172.21.15.40:0/2888028314 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18650 (secure 0 0 0) 0x7f0800015090 con 0x7f080c0c7390 2024-09-13T17:51:21.002 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:21.002 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":27,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:51:16.266176+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:50:32.740336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:50:33.552914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:50:36.623639+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:51:21.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.998+0000 7f0811daf700 1 -- 172.21.15.40:0/2888028314 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f07f8066760 msgr2=0x7f07f8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:21.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.998+0000 7f0811daf700 1 --2- 172.21.15.40:0/2888028314 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f07f8066760 0x7f07f8068c20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f07fc007b30 tx=0x7f07fc007a80 comp rx=0 tx=0).stop 2024-09-13T17:51:21.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.998+0000 7f0811daf700 1 -- 172.21.15.40:0/2888028314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f080c0c7390 msgr2=0x7f080c0ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:21.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.998+0000 7f0811daf700 1 --2- 172.21.15.40:0/2888028314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f080c0c7390 0x7f080c0ca7b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0800009b70 tx=0x7f0800004770 comp rx=0 tx=0).stop 2024-09-13T17:51:21.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.998+0000 7f0811daf700 1 -- 172.21.15.40:0/2888028314 shutdown_connections 2024-09-13T17:51:21.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.998+0000 7f0811daf700 1 --2- 172.21.15.40:0/2888028314 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f07f8066760 0x7f07f8068c20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.998+0000 7f0811daf700 1 --2- 172.21.15.40:0/2888028314 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f080c1512e0 0x7f080c14dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.998+0000 7f0811daf700 1 --2- 172.21.15.40:0/2888028314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f080c0c7390 0x7f080c0ca7b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.998+0000 7f0811daf700 1 --2- 172.21.15.40:0/2888028314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f080c0c5560 0x7f080c0ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.998+0000 7f0811daf700 1 -- 172.21.15.40:0/2888028314 >> 172.21.15.40:0/2888028314 conn(0x7f080c06a390 msgr2=0x7f080c154d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:21.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.998+0000 7f0811daf700 1 -- 172.21.15.40:0/2888028314 shutdown_connections 2024-09-13T17:51:21.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:20.998+0000 7f0811daf700 1 -- 172.21.15.40:0/2888028314 wait complete. 2024-09-13T17:51:21.023 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:50:21.057171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:50:32.740336+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:50:33.552914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:50:36.623639+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:51:21.023 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:51:21.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.142+0000 7f388bfff700 1 Processor -- start 2024-09-13T17:51:21.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.142+0000 7f388bfff700 1 -- start start 2024-09-13T17:51:21.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.142+0000 7f388bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f388c0cf580 0x7f388c15a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:21.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.142+0000 7f388bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f388c0cc250 0x7f388c0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:21.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.142+0000 7f388bfff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f388c15a7a0 con 0x7f388c0cf1a0 2024-09-13T17:51:21.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.142+0000 7f388bfff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388c0cbdc0 con 0x7f388c0cc250 2024-09-13T17:51:21.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.142+0000 7f388bfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388c153f80 con 0x7f388c0cf580 2024-09-13T17:51:21.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.146+0000 7f388b7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f388c0cc250 0x7f388c0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:21.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.146+0000 7f388a7fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f388c0cf580 0x7f388c15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:21.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.146+0000 7f388b7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f388c0cc250 0x7f388c0c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51346/0 (socket says 172.21.15.40:51346) 2024-09-13T17:51:21.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.146+0000 7f388b7fe700 1 -- 172.21.15.40:0/1158622110 learned_addr learned my addr 172.21.15.40:0/1158622110 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:21.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.146+0000 7f388b7fe700 1 -- 172.21.15.40:0/1158622110 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f388c0cf580 msgr2=0x7f388c15a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:21.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.146+0000 7f388b7fe700 1 --2- 172.21.15.40:0/1158622110 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f388c0cf580 0x7f388c15a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.146+0000 7f388b7fe700 1 -- 172.21.15.40:0/1158622110 >> v1:172.21.15.43:6789/0 conn(0x7f388c0cf1a0 legacy=0x7f388c0c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:21.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.146+0000 7f388b7fe700 1 -- 172.21.15.40:0/1158622110 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c0caed0 con 0x7f388c0cc250 2024-09-13T17:51:21.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.146+0000 7f388b7fe700 1 --2- 172.21.15.40:0/1158622110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f388c0cc250 0x7f388c0c8aa0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3884009e50 tx=0x7f388400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880340ae7b0c5500 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:21.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.146+0000 7f3889ffb700 1 -- 172.21.15.40:0/1158622110 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388400e070 con 0x7f388c0cc250 2024-09-13T17:51:21.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.146+0000 7f3889ffb700 1 -- 172.21.15.40:0/1158622110 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388400b910 con 0x7f388c0cc250 2024-09-13T17:51:21.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.146+0000 7f3889ffb700 1 -- 172.21.15.40:0/1158622110 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388400bcc0 con 0x7f388c0cc250 2024-09-13T17:51:21.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.146+0000 7f388bfff700 1 -- 172.21.15.40:0/1158622110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f388c0cc250 msgr2=0x7f388c0c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:21.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.146+0000 7f388bfff700 1 --2- 172.21.15.40:0/1158622110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f388c0cc250 0x7f388c0c8aa0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3884009e50 tx=0x7f388400b330 comp rx=0 tx=0).stop 2024-09-13T17:51:21.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.146+0000 7f388bfff700 1 -- 172.21.15.40:0/1158622110 shutdown_connections 2024-09-13T17:51:21.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.146+0000 7f388bfff700 1 --2- 172.21.15.40:0/1158622110 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f388c0cc250 0x7f388c0c8aa0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.146+0000 7f388bfff700 1 --2- 172.21.15.40:0/1158622110 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f388c0cf580 0x7f388c15a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.146+0000 7f388bfff700 1 -- 172.21.15.40:0/1158622110 >> 172.21.15.40:0/1158622110 conn(0x7f388c066090 msgr2=0x7f388c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:21.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.146+0000 7f388bfff700 1 -- 172.21.15.40:0/1158622110 shutdown_connections 2024-09-13T17:51:21.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.146+0000 7f388bfff700 1 -- 172.21.15.40:0/1158622110 wait complete. 2024-09-13T17:51:21.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.150+0000 7f388bfff700 1 Processor -- start 2024-09-13T17:51:21.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.150+0000 7f388bfff700 1 -- start start 2024-09-13T17:51:21.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.150+0000 7f388bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f388c0cc250 0x7f388c154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:21.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.150+0000 7f388bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f388c0cf1a0 0x7f388c14fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:21.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.150+0000 7f388bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f388c0cf580 0x7f388c1f2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:21.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.150+0000 7f388bfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c155be0 con 0x7f388c0cf580 2024-09-13T17:51:21.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.150+0000 7f388bfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c0c9a40 con 0x7f388c0cf1a0 2024-09-13T17:51:21.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.150+0000 7f388bfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388c154790 con 0x7f388c0cc250 2024-09-13T17:51:21.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.150+0000 7f388b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f388c0cf580 0x7f388c1f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:21.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.150+0000 7f388b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f388c0cf580 0x7f388c1f2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51362/0 (socket says 172.21.15.40:51362) 2024-09-13T17:51:21.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.150+0000 7f388b7fe700 1 -- 172.21.15.40:0/419204477 learned_addr learned my addr 172.21.15.40:0/419204477 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:21.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.150+0000 7f388affd700 1 --2- 172.21.15.40:0/419204477 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f388c0cc250 0x7f388c154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:21.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.150+0000 7f388a7fc700 1 --2- 172.21.15.40:0/419204477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f388c0cf1a0 0x7f388c14fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:21.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.154+0000 7f388b7fe700 1 -- 172.21.15.40:0/419204477 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f388c0cc250 msgr2=0x7f388c154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:21.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.154+0000 7f388b7fe700 1 --2- 172.21.15.40:0/419204477 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f388c0cc250 0x7f388c154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.154+0000 7f388b7fe700 1 -- 172.21.15.40:0/419204477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f388c0cf1a0 msgr2=0x7f388c14fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:21.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.154+0000 7f388b7fe700 1 --2- 172.21.15.40:0/419204477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f388c0cf1a0 0x7f388c14fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.154+0000 7f388b7fe700 1 -- 172.21.15.40:0/419204477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3884009b30 con 0x7f388c0cf580 2024-09-13T17:51:21.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.154+0000 7f388b7fe700 1 --2- 172.21.15.40:0/419204477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f388c0cf580 0x7f388c1f2510 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f388c08c270 tx=0x7f3884004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:21.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.154+0000 7f38737fe700 1 -- 172.21.15.40:0/419204477 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3884004800 con 0x7f388c0cf580 2024-09-13T17:51:21.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.154+0000 7f38737fe700 1 -- 172.21.15.40:0/419204477 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38840099b0 con 0x7f388c0cf580 2024-09-13T17:51:21.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.154+0000 7f388bfff700 1 -- 172.21.15.40:0/419204477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f388c14cbd0 con 0x7f388c0cf580 2024-09-13T17:51:21.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.154+0000 7f38737fe700 1 -- 172.21.15.40:0/419204477 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388401c680 con 0x7f388c0cf580 2024-09-13T17:51:21.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.154+0000 7f388bfff700 1 -- 172.21.15.40:0/419204477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388c14d190 con 0x7f388c0cf580 2024-09-13T17:51:21.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.154+0000 7f388bfff700 1 -- 172.21.15.40:0/419204477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f388c067a40 con 0x7f388c0cf580 2024-09-13T17:51:21.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.154+0000 7f38737fe700 1 -- 172.21.15.40:0/419204477 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f388401c7e0 con 0x7f388c0cf580 2024-09-13T17:51:21.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.154+0000 7f38737fe700 1 --2- 172.21.15.40:0/419204477 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3878066730 0x7f3878068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:21.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.154+0000 7f38737fe700 1 -- 172.21.15.40:0/419204477 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f38840856f0 con 0x7f388c0cf580 2024-09-13T17:51:21.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.154+0000 7f388affd700 1 --2- 172.21.15.40:0/419204477 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3878066730 0x7f3878068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:21.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.158+0000 7f388affd700 1 --2- 172.21.15.40:0/419204477 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3878066730 0x7f3878068bf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3880000f60 tx=0x7f3880009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:21.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.162+0000 7f38737fe700 1 -- 172.21.15.40:0/419204477 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3884052590 con 0x7f388c0cf580 2024-09-13T17:51:21.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.330+0000 7f388bfff700 1 -- 172.21.15.40:0/419204477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f388c15bdd0 con 0x7f388c0cf580 2024-09-13T17:51:21.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.330+0000 7f38737fe700 1 -- 172.21.15.40:0/419204477 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3884055bb0 con 0x7f388c0cf580 2024-09-13T17:51:21.337 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-09-13T17:51:21.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.334+0000 7f388bfff700 1 -- 172.21.15.40:0/419204477 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3878066730 msgr2=0x7f3878068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:21.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.334+0000 7f388bfff700 1 --2- 172.21.15.40:0/419204477 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3878066730 0x7f3878068bf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3880000f60 tx=0x7f3880009040 comp rx=0 tx=0).stop 2024-09-13T17:51:21.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.334+0000 7f388bfff700 1 -- 172.21.15.40:0/419204477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f388c0cf580 msgr2=0x7f388c1f2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:21.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.334+0000 7f388bfff700 1 --2- 172.21.15.40:0/419204477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f388c0cf580 0x7f388c1f2510 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f388c08c270 tx=0x7f3884004af0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.334+0000 7f388bfff700 1 -- 172.21.15.40:0/419204477 shutdown_connections 2024-09-13T17:51:21.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.334+0000 7f388bfff700 1 --2- 172.21.15.40:0/419204477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f388c0cf580 0x7f388c1f2510 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.334+0000 7f388bfff700 1 --2- 172.21.15.40:0/419204477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f388c0cf1a0 0x7f388c14fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.334+0000 7f388bfff700 1 --2- 172.21.15.40:0/419204477 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3878066730 0x7f3878068bf0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.334+0000 7f388bfff700 1 --2- 172.21.15.40:0/419204477 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f388c0cc250 0x7f388c154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.334+0000 7f388bfff700 1 -- 172.21.15.40:0/419204477 >> 172.21.15.40:0/419204477 conn(0x7f388c066090 msgr2=0x7f388c0cd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:21.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.334+0000 7f388bfff700 1 -- 172.21.15.40:0/419204477 shutdown_connections 2024-09-13T17:51:21.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.334+0000 7f388bfff700 1 -- 172.21.15.40:0/419204477 wait complete. 2024-09-13T17:51:21.351 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:51:21.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1fd08700 1 Processor -- start 2024-09-13T17:51:21.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1fd08700 1 -- start start 2024-09-13T17:51:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1fd08700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fed18157d10 0x7fed180b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1fd08700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed18161c60 0x7fed180c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1fd08700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed180b1000 con 0x7fed1806afb0 2024-09-13T17:51:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1fd08700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed180729e0 con 0x7fed18161c60 2024-09-13T17:51:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1fd08700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed180c6c20 con 0x7fed18157d10 2024-09-13T17:51:21.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1d2a3700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed18161c60 0x7fed180c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:21.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1d2a3700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fed18161c60 0x7fed180c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55056/0 (socket says 172.21.15.40:55056) 2024-09-13T17:51:21.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1d2a3700 1 -- 172.21.15.40:0/2554640995 learned_addr learned my addr 172.21.15.40:0/2554640995 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:21.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1daa4700 1 --2- 172.21.15.40:0/2554640995 >> v2:172.21.15.43:3300/0 conn(0x7fed18157d10 0x7fed180b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:21.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1d2a3700 1 -- 172.21.15.40:0/2554640995 >> v2:172.21.15.43:3300/0 conn(0x7fed18157d10 msgr2=0x7fed180b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:21.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1d2a3700 1 --2- 172.21.15.40:0/2554640995 >> v2:172.21.15.43:3300/0 conn(0x7fed18157d10 0x7fed180b0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1d2a3700 1 -- 172.21.15.40:0/2554640995 >> v1:172.21.15.43:6789/0 conn(0x7fed1806afb0 legacy=0x7fed1806a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:21.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1d2a3700 1 -- 172.21.15.40:0/2554640995 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed181617d0 con 0x7fed18161c60 2024-09-13T17:51:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1d2a3700 1 --2- 172.21.15.40:0/2554640995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed18161c60 0x7fed180c5860 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fed1804ed50 tx=0x7fed0c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8e9e661a4aac57 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1caa2700 1 -- 172.21.15.40:0/2554640995 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c00b8a0 con 0x7fed18161c60 2024-09-13T17:51:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1caa2700 1 -- 172.21.15.40:0/2554640995 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0c00ba00 con 0x7fed18161c60 2024-09-13T17:51:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1caa2700 1 -- 172.21.15.40:0/2554640995 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c00bd90 con 0x7fed18161c60 2024-09-13T17:51:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1fd08700 1 -- 172.21.15.40:0/2554640995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed18161c60 msgr2=0x7fed180c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1fd08700 1 --2- 172.21.15.40:0/2554640995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed18161c60 0x7fed180c5860 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fed1804ed50 tx=0x7fed0c009bf0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1fd08700 1 -- 172.21.15.40:0/2554640995 shutdown_connections 2024-09-13T17:51:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1fd08700 1 --2- 172.21.15.40:0/2554640995 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed18161c60 0x7fed180c5860 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1fd08700 1 --2- 172.21.15.40:0/2554640995 >> v2:172.21.15.43:3300/0 conn(0x7fed18157d10 0x7fed180b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1fd08700 1 -- 172.21.15.40:0/2554640995 >> 172.21.15.40:0/2554640995 conn(0x7fed1806d5b0 msgr2=0x7fed1806d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:21.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1fd08700 1 -- 172.21.15.40:0/2554640995 shutdown_connections 2024-09-13T17:51:21.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.462+0000 7fed1fd08700 1 -- 172.21.15.40:0/2554640995 wait complete. 2024-09-13T17:51:21.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.466+0000 7fed1fd08700 1 Processor -- start 2024-09-13T17:51:21.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.466+0000 7fed1fd08700 1 -- start start 2024-09-13T17:51:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.466+0000 7fed1fd08700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed1806afb0 0x7fed181f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.466+0000 7fed1fd08700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed18157d10 0x7fed181f4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.466+0000 7fed1fd08700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed18161c60 0x7fed180c34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.466+0000 7fed1fd08700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed181f6f20 con 0x7fed1806afb0 2024-09-13T17:51:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.466+0000 7fed1fd08700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180c39f0 con 0x7fed18157d10 2024-09-13T17:51:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.466+0000 7fed1fd08700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed180aa0a0 con 0x7fed18161c60 2024-09-13T17:51:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.466+0000 7fed1daa4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed1806afb0 0x7fed181f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.466+0000 7fed1daa4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed1806afb0 0x7fed181f7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55062/0 (socket says 172.21.15.40:55062) 2024-09-13T17:51:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.466+0000 7fed1daa4700 1 -- 172.21.15.40:0/3834408162 learned_addr learned my addr 172.21.15.40:0/3834408162 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.466+0000 7fed1d2a3700 1 --2- 172.21.15.40:0/3834408162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed18157d10 0x7fed181f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.466+0000 7fed1e2a5700 1 --2- 172.21.15.40:0/3834408162 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed18161c60 0x7fed180c34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.466+0000 7fed1daa4700 1 -- 172.21.15.40:0/3834408162 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed18161c60 msgr2=0x7fed180c34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.466+0000 7fed1daa4700 1 --2- 172.21.15.40:0/3834408162 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed18161c60 0x7fed180c34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.466+0000 7fed1daa4700 1 -- 172.21.15.40:0/3834408162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed18157d10 msgr2=0x7fed181f4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.466+0000 7fed1daa4700 1 --2- 172.21.15.40:0/3834408162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed18157d10 0x7fed181f4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.466+0000 7fed1daa4700 1 -- 172.21.15.40:0/3834408162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed0c009570 con 0x7fed1806afb0 2024-09-13T17:51:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.466+0000 7fed1e2a5700 1 --2- 172.21.15.40:0/3834408162 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed18161c60 0x7fed180c34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.470+0000 7fed1daa4700 1 --2- 172.21.15.40:0/3834408162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed1806afb0 0x7fed181f7340 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fed1400e760 tx=0x7fed1400ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:21.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.470+0000 7fed0a7fc700 1 -- 172.21.15.40:0/3834408162 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14013610 con 0x7fed1806afb0 2024-09-13T17:51:21.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.470+0000 7fed0a7fc700 1 -- 172.21.15.40:0/3834408162 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed14013770 con 0x7fed1806afb0 2024-09-13T17:51:21.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.470+0000 7fed1fd08700 1 -- 172.21.15.40:0/3834408162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed180c2ef0 con 0x7fed1806afb0 2024-09-13T17:51:21.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.470+0000 7fed0a7fc700 1 -- 172.21.15.40:0/3834408162 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14013ae0 con 0x7fed1806afb0 2024-09-13T17:51:21.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.470+0000 7fed1fd08700 1 -- 172.21.15.40:0/3834408162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed181f6860 con 0x7fed1806afb0 2024-09-13T17:51:21.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.470+0000 7fed1fd08700 1 -- 172.21.15.40:0/3834408162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed18161180 con 0x7fed1806afb0 2024-09-13T17:51:21.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.470+0000 7fed0a7fc700 1 -- 172.21.15.40:0/3834408162 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fed14013c40 con 0x7fed1806afb0 2024-09-13T17:51:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.470+0000 7fed0a7fc700 1 --2- 172.21.15.40:0/3834408162 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fecfc066730 0x7fecfc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.470+0000 7fed0a7fc700 1 -- 172.21.15.40:0/3834408162 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fed14084080 con 0x7fed1806afb0 2024-09-13T17:51:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.470+0000 7fed1d2a3700 1 --2- 172.21.15.40:0/3834408162 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fecfc066730 0x7fecfc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:21.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.474+0000 7fed1d2a3700 1 --2- 172.21.15.40:0/3834408162 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fecfc066730 0x7fecfc068bf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fed0c000c00 tx=0x7fed0c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:21.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.478+0000 7fed0a7fc700 1 -- 172.21.15.40:0/3834408162 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fed140510c0 con 0x7fed1806afb0 2024-09-13T17:51:21.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.650+0000 7fed1fd08700 1 -- 172.21.15.40:0/3834408162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fed180c9350 con 0x7fed1806afb0 2024-09-13T17:51:21.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.654+0000 7fed0a7fc700 1 -- 172.21.15.40:0/3834408162 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fed14021020 con 0x7fed1806afb0 2024-09-13T17:51:21.658 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-09-13T17:51:21.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.654+0000 7fed1fd08700 1 -- 172.21.15.40:0/3834408162 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fecfc066730 msgr2=0x7fecfc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:21.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.654+0000 7fed1fd08700 1 --2- 172.21.15.40:0/3834408162 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fecfc066730 0x7fecfc068bf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fed0c000c00 tx=0x7fed0c008030 comp rx=0 tx=0).stop 2024-09-13T17:51:21.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.654+0000 7fed1fd08700 1 -- 172.21.15.40:0/3834408162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed1806afb0 msgr2=0x7fed181f7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:21.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.654+0000 7fed1fd08700 1 --2- 172.21.15.40:0/3834408162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed1806afb0 0x7fed181f7340 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fed1400e760 tx=0x7fed1400ea70 comp rx=0 tx=0).stop 2024-09-13T17:51:21.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.654+0000 7fed1fd08700 1 -- 172.21.15.40:0/3834408162 shutdown_connections 2024-09-13T17:51:21.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.654+0000 7fed1fd08700 1 --2- 172.21.15.40:0/3834408162 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fecfc066730 0x7fecfc068bf0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.654+0000 7fed1fd08700 1 --2- 172.21.15.40:0/3834408162 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed18161c60 0x7fed180c34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.654+0000 7fed1fd08700 1 --2- 172.21.15.40:0/3834408162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed18157d10 0x7fed181f4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.654+0000 7fed1fd08700 1 --2- 172.21.15.40:0/3834408162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed1806afb0 0x7fed181f7340 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.654+0000 7fed1fd08700 1 -- 172.21.15.40:0/3834408162 >> 172.21.15.40:0/3834408162 conn(0x7fed1806d5b0 msgr2=0x7fed18073700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:21.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.654+0000 7fed1fd08700 1 -- 172.21.15.40:0/3834408162 shutdown_connections 2024-09-13T17:51:21.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.654+0000 7fed1fd08700 1 -- 172.21.15.40:0/3834408162 wait complete. 2024-09-13T17:51:21.674 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:51:21.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.786+0000 7f48cce51700 1 Processor -- start 2024-09-13T17:51:21.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.786+0000 7f48cce51700 1 -- start start 2024-09-13T17:51:21.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48cce51700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f48c80cc240 0x7f48c815cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:21.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48cce51700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f48c80cc620 0x7f48c815d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:21.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48cce51700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c8159780 con 0x7f48c8153520 2024-09-13T17:51:21.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48cce51700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c8159a40 con 0x7f48c80cc240 2024-09-13T17:51:21.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48cce51700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c8159d20 con 0x7f48c80cc620 2024-09-13T17:51:21.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48c659c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f48c80cc240 0x7f48c815cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:21.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48c659c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f48c80cc240 0x7f48c815cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55070/0 (socket says 172.21.15.40:55070) 2024-09-13T17:51:21.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48c659c700 1 -- 172.21.15.40:0/3218360796 learned_addr learned my addr 172.21.15.40:0/3218360796 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:21.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48c5d9b700 1 --2- 172.21.15.40:0/3218360796 >> v2:172.21.15.43:3300/0 conn(0x7f48c80cc620 0x7f48c815d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:21.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48c659c700 1 -- 172.21.15.40:0/3218360796 >> v2:172.21.15.43:3300/0 conn(0x7f48c80cc620 msgr2=0x7f48c815d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:21.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48c659c700 1 --2- 172.21.15.40:0/3218360796 >> v2:172.21.15.43:3300/0 conn(0x7f48c80cc620 0x7f48c815d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48c659c700 1 -- 172.21.15.40:0/3218360796 >> v1:172.21.15.43:6789/0 conn(0x7f48c8153520 legacy=0x7f48c8153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:21.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48c659c700 1 -- 172.21.15.40:0/3218360796 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c815a000 con 0x7f48c80cc240 2024-09-13T17:51:21.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48c659c700 1 --2- 172.21.15.40:0/3218360796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48c80cc240 0x7f48c815cf10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f48c8074470 tx=0x7f48bc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c064ad7f21fe17d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:21.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48c559a700 1 -- 172.21.15.40:0/3218360796 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc011830 con 0x7f48c80cc240 2024-09-13T17:51:21.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48c559a700 1 -- 172.21.15.40:0/3218360796 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48bc011990 con 0x7f48c80cc240 2024-09-13T17:51:21.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48c559a700 1 -- 172.21.15.40:0/3218360796 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc011d20 con 0x7f48c80cc240 2024-09-13T17:51:21.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48cce51700 1 -- 172.21.15.40:0/3218360796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48c80cc240 msgr2=0x7f48c815cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:21.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48cce51700 1 --2- 172.21.15.40:0/3218360796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48c80cc240 0x7f48c815cf10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f48c8074470 tx=0x7f48bc009bf0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48cce51700 1 -- 172.21.15.40:0/3218360796 shutdown_connections 2024-09-13T17:51:21.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48cce51700 1 --2- 172.21.15.40:0/3218360796 >> v2:172.21.15.43:3300/0 conn(0x7f48c80cc620 0x7f48c815d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48cce51700 1 --2- 172.21.15.40:0/3218360796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48c80cc240 0x7f48c815cf10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48cce51700 1 -- 172.21.15.40:0/3218360796 >> 172.21.15.40:0/3218360796 conn(0x7f48c8066480 msgr2=0x7f48c8066890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:21.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48cce51700 1 -- 172.21.15.40:0/3218360796 shutdown_connections 2024-09-13T17:51:21.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.790+0000 7f48cce51700 1 -- 172.21.15.40:0/3218360796 wait complete. 2024-09-13T17:51:21.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48cce51700 1 Processor -- start 2024-09-13T17:51:21.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48cce51700 1 -- start start 2024-09-13T17:51:21.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48cce51700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48c80cc240 0x7f48c81f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:21.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48cce51700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f48c80cc620 0x7f48c80b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:21.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48cce51700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48c8153520 0x7f48c80b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:21.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48cce51700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c815c390 con 0x7f48c80cc240 2024-09-13T17:51:21.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48cce51700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c815c940 con 0x7f48c8153520 2024-09-13T17:51:21.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48cce51700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c815b220 con 0x7f48c80cc620 2024-09-13T17:51:21.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48c659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48c80cc240 0x7f48c81f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:21.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48c659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48c80cc240 0x7f48c81f2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55074/0 (socket says 172.21.15.40:55074) 2024-09-13T17:51:21.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48c659c700 1 -- 172.21.15.40:0/239999444 learned_addr learned my addr 172.21.15.40:0/239999444 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:21.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48c5d9b700 1 --2- 172.21.15.40:0/239999444 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f48c80cc620 0x7f48c80b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:21.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48c6d9d700 1 --2- 172.21.15.40:0/239999444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48c8153520 0x7f48c80b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:21.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48c659c700 1 -- 172.21.15.40:0/239999444 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f48c80cc620 msgr2=0x7f48c80b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:21.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48c659c700 1 --2- 172.21.15.40:0/239999444 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f48c80cc620 0x7f48c80b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48c659c700 1 -- 172.21.15.40:0/239999444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48c8153520 msgr2=0x7f48c80b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:21.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48c659c700 1 --2- 172.21.15.40:0/239999444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48c8153520 0x7f48c80b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48c659c700 1 -- 172.21.15.40:0/239999444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48bc009570 con 0x7f48c80cc240 2024-09-13T17:51:21.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48c6d9d700 1 --2- 172.21.15.40:0/239999444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48c8153520 0x7f48c80b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:51:21.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48c659c700 1 --2- 172.21.15.40:0/239999444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48c80cc240 0x7f48c81f2580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f48c808c630 tx=0x7f48bc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:21.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48b6ffd700 1 -- 172.21.15.40:0/239999444 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc004480 con 0x7f48c80cc240 2024-09-13T17:51:21.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48b6ffd700 1 -- 172.21.15.40:0/239999444 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48bc01f070 con 0x7f48c80cc240 2024-09-13T17:51:21.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48cce51700 1 -- 172.21.15.40:0/239999444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c81f2fb0 con 0x7f48c80cc240 2024-09-13T17:51:21.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48b6ffd700 1 -- 172.21.15.40:0/239999444 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc0038e0 con 0x7f48c80cc240 2024-09-13T17:51:21.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48cce51700 1 -- 172.21.15.40:0/239999444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c80b50f0 con 0x7f48c80cc240 2024-09-13T17:51:21.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48cce51700 1 -- 172.21.15.40:0/239999444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c8159490 con 0x7f48c80cc240 2024-09-13T17:51:21.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.794+0000 7f48b6ffd700 1 -- 172.21.15.40:0/239999444 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f48bc021030 con 0x7f48c80cc240 2024-09-13T17:51:21.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.798+0000 7f48b6ffd700 1 --2- 172.21.15.40:0/239999444 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f48a8066730 0x7f48a8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:21.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.798+0000 7f48b6ffd700 1 -- 172.21.15.40:0/239999444 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f48bc085810 con 0x7f48c80cc240 2024-09-13T17:51:21.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.798+0000 7f48c5d9b700 1 --2- 172.21.15.40:0/239999444 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f48a8066730 0x7f48a8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:21.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.798+0000 7f48c5d9b700 1 --2- 172.21.15.40:0/239999444 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f48a8066730 0x7f48a8068bf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f48c815b720 tx=0x7f48b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:21.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.802+0000 7f48b6ffd700 1 -- 172.21.15.40:0/239999444 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f48bc052740 con 0x7f48c80cc240 2024-09-13T17:51:21.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.974+0000 7f48cce51700 1 -- 172.21.15.40:0/239999444 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f48c8074db0 con 0x7f48c80cc240 2024-09-13T17:51:21.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.974+0000 7f48b6ffd700 1 -- 172.21.15.40:0/239999444 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f48bc055d60 con 0x7f48c80cc240 2024-09-13T17:51:21.980 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-09-13T17:51:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.978+0000 7f48cce51700 1 -- 172.21.15.40:0/239999444 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f48a8066730 msgr2=0x7f48a8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.978+0000 7f48cce51700 1 --2- 172.21.15.40:0/239999444 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f48a8066730 0x7f48a8068bf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f48c815b720 tx=0x7f48b8008040 comp rx=0 tx=0).stop 2024-09-13T17:51:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.978+0000 7f48cce51700 1 -- 172.21.15.40:0/239999444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48c80cc240 msgr2=0x7f48c81f2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.978+0000 7f48cce51700 1 --2- 172.21.15.40:0/239999444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48c80cc240 0x7f48c81f2580 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f48c808c630 tx=0x7f48bc004770 comp rx=0 tx=0).stop 2024-09-13T17:51:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.978+0000 7f48cce51700 1 -- 172.21.15.40:0/239999444 shutdown_connections 2024-09-13T17:51:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.978+0000 7f48cce51700 1 --2- 172.21.15.40:0/239999444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48c8153520 0x7f48c80b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.978+0000 7f48cce51700 1 --2- 172.21.15.40:0/239999444 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f48a8066730 0x7f48a8068bf0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.978+0000 7f48cce51700 1 --2- 172.21.15.40:0/239999444 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f48c80cc620 0x7f48c80b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.978+0000 7f48cce51700 1 --2- 172.21.15.40:0/239999444 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48c80cc240 0x7f48c81f2580 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.978+0000 7f48cce51700 1 -- 172.21.15.40:0/239999444 >> 172.21.15.40:0/239999444 conn(0x7f48c8066480 msgr2=0x7f48c80c4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.978+0000 7f48cce51700 1 -- 172.21.15.40:0/239999444 shutdown_connections 2024-09-13T17:51:21.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:21.978+0000 7f48cce51700 1 -- 172.21.15.40:0/239999444 wait complete. 2024-09-13T17:51:21.995 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:51:22.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.106+0000 7f5f83fff700 1 Processor -- start 2024-09-13T17:51:22.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.106+0000 7f5f83fff700 1 -- start start 2024-09-13T17:51:22.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f83fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5f840c8050 0x7f5f841562e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:22.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f83fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f840c8430 0x7f5f840c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:22.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f83fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f84159f40 con 0x7f5f84070a60 2024-09-13T17:51:22.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f83fff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f84156820 con 0x7f5f840c8050 2024-09-13T17:51:22.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f83fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f840c6ee0 con 0x7f5f840c8430 2024-09-13T17:51:22.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f82ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5f840c8050 0x7f5f841562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:22.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f837fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5f84070a60 0x7f5f84150160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:37096/0 (socket says 172.21.15.40:37096) 2024-09-13T17:51:22.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f827fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f840c8430 0x7f5f840c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:22.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f837fe700 1 -- 172.21.15.40:0/3811790888 learned_addr learned my addr 172.21.15.40:0/3811790888 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:22.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f82ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5f840c8050 0x7f5f841562e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55076/0 (socket says 172.21.15.40:55076) 2024-09-13T17:51:22.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f81ffb700 1 -- 172.21.15.40:0/3811790888 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645650879 0 0) 0x7f5f84159f40 con 0x7f5f84070a60 2024-09-13T17:51:22.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f82ffd700 1 -- 172.21.15.40:0/3811790888 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f840c8430 msgr2=0x7f5f840c66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:22.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f82ffd700 1 --2- 172.21.15.40:0/3811790888 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f840c8430 0x7f5f840c66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f82ffd700 1 -- 172.21.15.40:0/3811790888 >> v1:172.21.15.43:6789/0 conn(0x7f5f84070a60 legacy=0x7f5f84150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:22.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f82ffd700 1 -- 172.21.15.40:0/3811790888 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f840c71c0 con 0x7f5f840c8050 2024-09-13T17:51:22.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f82ffd700 1 --2- 172.21.15.40:0/3811790888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f840c8050 0x7f5f841562e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009850 tx=0x7f5f6c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb4575c88d8fac1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:22.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f81ffb700 1 -- 172.21.15.40:0/3811790888 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c011830 con 0x7f5f840c8050 2024-09-13T17:51:22.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f81ffb700 1 -- 172.21.15.40:0/3811790888 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c011990 con 0x7f5f840c8050 2024-09-13T17:51:22.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f81ffb700 1 -- 172.21.15.40:0/3811790888 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c011d40 con 0x7f5f840c8050 2024-09-13T17:51:22.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f83fff700 1 -- 172.21.15.40:0/3811790888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f840c8050 msgr2=0x7f5f841562e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:22.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f83fff700 1 --2- 172.21.15.40:0/3811790888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f840c8050 0x7f5f841562e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009850 tx=0x7f5f6c009b60 comp rx=0 tx=0).stop 2024-09-13T17:51:22.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f83fff700 1 -- 172.21.15.40:0/3811790888 shutdown_connections 2024-09-13T17:51:22.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f83fff700 1 --2- 172.21.15.40:0/3811790888 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f840c8430 0x7f5f840c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f83fff700 1 --2- 172.21.15.40:0/3811790888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f840c8050 0x7f5f841562e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f83fff700 1 -- 172.21.15.40:0/3811790888 >> 172.21.15.40:0/3811790888 conn(0x7f5f84066090 msgr2=0x7f5f840664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:22.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f83fff700 1 -- 172.21.15.40:0/3811790888 shutdown_connections 2024-09-13T17:51:22.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.110+0000 7f5f83fff700 1 -- 172.21.15.40:0/3811790888 wait complete. 2024-09-13T17:51:22.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f83fff700 1 Processor -- start 2024-09-13T17:51:22.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f83fff700 1 -- start start 2024-09-13T17:51:22.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f83fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f84070a60 0x7f5f84149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:22.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f83fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f840c8050 0x7f5f841f0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:22.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f83fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f840c8430 0x7f5f841558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:22.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f83fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f84153180 con 0x7f5f840c8430 2024-09-13T17:51:22.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f83fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f841f09f0 con 0x7f5f84070a60 2024-09-13T17:51:22.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f83fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f841581e0 con 0x7f5f840c8050 2024-09-13T17:51:22.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f837fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f840c8430 0x7f5f841558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:22.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f827fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f840c8050 0x7f5f841f0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:22.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f827fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f840c8050 0x7f5f841f0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:59676/0 (socket says 172.21.15.40:59676) 2024-09-13T17:51:22.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f837fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f840c8430 0x7f5f841558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55078/0 (socket says 172.21.15.40:55078) 2024-09-13T17:51:22.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f827fc700 1 -- 172.21.15.40:0/48295282 learned_addr learned my addr 172.21.15.40:0/48295282 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:22.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f82ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f84070a60 0x7f5f84149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:22.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f837fe700 1 -- 172.21.15.40:0/48295282 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f840c8050 msgr2=0x7f5f841f0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:22.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f837fe700 1 --2- 172.21.15.40:0/48295282 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f840c8050 0x7f5f841f0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f837fe700 1 -- 172.21.15.40:0/48295282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f84070a60 msgr2=0x7f5f84149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:22.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f837fe700 1 --2- 172.21.15.40:0/48295282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f84070a60 0x7f5f84149cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f837fe700 1 -- 172.21.15.40:0/48295282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c0094b0 con 0x7f5f840c8430 2024-09-13T17:51:22.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f837fe700 1 --2- 172.21.15.40:0/48295282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f840c8430 0x7f5f841558d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c00c450 tx=0x7f5f7c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:22.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f7b7fe700 1 -- 172.21.15.40:0/48295282 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c004ca0 con 0x7f5f840c8430 2024-09-13T17:51:22.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f7b7fe700 1 -- 172.21.15.40:0/48295282 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7c004e20 con 0x7f5f840c8430 2024-09-13T17:51:22.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.114+0000 7f5f83fff700 1 -- 172.21.15.40:0/48295282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f84158480 con 0x7f5f840c8430 2024-09-13T17:51:22.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.118+0000 7f5f7b7fe700 1 -- 172.21.15.40:0/48295282 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c007310 con 0x7f5f840c8430 2024-09-13T17:51:22.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.118+0000 7f5f83fff700 1 -- 172.21.15.40:0/48295282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f841f0680 con 0x7f5f840c8430 2024-09-13T17:51:22.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.118+0000 7f5f83fff700 1 -- 172.21.15.40:0/48295282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f8414df90 con 0x7f5f840c8430 2024-09-13T17:51:22.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.118+0000 7f5f7b7fe700 1 -- 172.21.15.40:0/48295282 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5f7c007470 con 0x7f5f840c8430 2024-09-13T17:51:22.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.118+0000 7f5f7b7fe700 1 --2- 172.21.15.40:0/48295282 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5f64066730 0x7f5f64068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:22.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.118+0000 7f5f7b7fe700 1 -- 172.21.15.40:0/48295282 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5f7c084100 con 0x7f5f840c8430 2024-09-13T17:51:22.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.118+0000 7f5f82ffd700 1 --2- 172.21.15.40:0/48295282 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5f64066730 0x7f5f64068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:22.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.118+0000 7f5f82ffd700 1 --2- 172.21.15.40:0/48295282 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5f64066730 0x7f5f64068bf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c000c00 tx=0x7f5f6c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:22.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.126+0000 7f5f7b7fe700 1 -- 172.21.15.40:0/48295282 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5f7c051010 con 0x7f5f840c8430 2024-09-13T17:51:22.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.298+0000 7f5f83fff700 1 -- 172.21.15.40:0/48295282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5f841f0b60 con 0x7f5f840c8430 2024-09-13T17:51:22.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.298+0000 7f5f7b7fe700 1 -- 172.21.15.40:0/48295282 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5f7c00ce60 con 0x7f5f840c8430 2024-09-13T17:51:22.304 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-09-13T17:51:22.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.302+0000 7f5f83fff700 1 -- 172.21.15.40:0/48295282 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5f64066730 msgr2=0x7f5f64068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:22.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.302+0000 7f5f83fff700 1 --2- 172.21.15.40:0/48295282 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5f64066730 0x7f5f64068bf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c000c00 tx=0x7f5f6c008030 comp rx=0 tx=0).stop 2024-09-13T17:51:22.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.302+0000 7f5f83fff700 1 -- 172.21.15.40:0/48295282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f840c8430 msgr2=0x7f5f841558d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:22.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.302+0000 7f5f83fff700 1 --2- 172.21.15.40:0/48295282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f840c8430 0x7f5f841558d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c00c450 tx=0x7f5f7c00c760 comp rx=0 tx=0).stop 2024-09-13T17:51:22.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.302+0000 7f5f83fff700 1 -- 172.21.15.40:0/48295282 shutdown_connections 2024-09-13T17:51:22.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.302+0000 7f5f83fff700 1 --2- 172.21.15.40:0/48295282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f840c8430 0x7f5f841558d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.302+0000 7f5f83fff700 1 --2- 172.21.15.40:0/48295282 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5f64066730 0x7f5f64068bf0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.302+0000 7f5f83fff700 1 --2- 172.21.15.40:0/48295282 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f840c8050 0x7f5f841f0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.302+0000 7f5f83fff700 1 --2- 172.21.15.40:0/48295282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f84070a60 0x7f5f84149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.302+0000 7f5f83fff700 1 -- 172.21.15.40:0/48295282 >> 172.21.15.40:0/48295282 conn(0x7f5f84066090 msgr2=0x7f5f8406d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:22.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.302+0000 7f5f83fff700 1 -- 172.21.15.40:0/48295282 shutdown_connections 2024-09-13T17:51:22.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.302+0000 7f5f83fff700 1 -- 172.21.15.40:0/48295282 wait complete. 2024-09-13T17:51:22.325 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2024-09-13T17:51:22.325 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:51:22.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b3225700 1 Processor -- start 2024-09-13T17:51:22.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b3225700 1 -- start start 2024-09-13T17:51:22.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b3225700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc8ac157c60 0x7fc8ac0c0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:22.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b3225700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8ac0c57d0 0x7fc8ac0c0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:22.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b3225700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8ac15db60 con 0x7fc8ac06d300 2024-09-13T17:51:22.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b3225700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8ac160fb0 con 0x7fc8ac157c60 2024-09-13T17:51:22.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b3225700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac161250 con 0x7fc8ac0c57d0 2024-09-13T17:51:22.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b0fc1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc8ac157c60 0x7fc8ac0c0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:22.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b17c2700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc8ac06d300 0x7fc8ac15dff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44512/0 (socket says 172.21.15.40:44512) 2024-09-13T17:51:22.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b17c2700 1 -- 172.21.15.40:0/799885603 learned_addr learned my addr 172.21.15.40:0/799885603 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:22.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8abfff700 1 --2- 172.21.15.40:0/799885603 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8ac0c57d0 0x7fc8ac0c0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:22.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8ab7fe700 1 -- 172.21.15.40:0/799885603 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1666093922 0 0) 0x7fc8ac15db60 con 0x7fc8ac06d300 2024-09-13T17:51:22.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8ab7fe700 1 -- 172.21.15.40:0/799885603 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8980035d0 con 0x7fc8ac06d300 2024-09-13T17:51:22.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b0fc1700 1 -- 172.21.15.40:0/799885603 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8ac0c57d0 msgr2=0x7fc8ac0c0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:22.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b0fc1700 1 --2- 172.21.15.40:0/799885603 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8ac0c57d0 0x7fc8ac0c0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b0fc1700 1 -- 172.21.15.40:0/799885603 >> v1:172.21.15.40:6789/0 conn(0x7fc8ac06d300 legacy=0x7fc8ac15dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:22.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b0fc1700 1 -- 172.21.15.40:0/799885603 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ac161510 con 0x7fc8ac157c60 2024-09-13T17:51:22.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b0fc1700 1 --2- 172.21.15.40:0/799885603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8ac157c60 0x7fc8ac0c0840 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc894009850 tx=0x7fc894009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ddb5d52623dcd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:22.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8ab7fe700 1 -- 172.21.15.40:0/799885603 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894011830 con 0x7fc8ac157c60 2024-09-13T17:51:22.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8ab7fe700 1 -- 172.21.15.40:0/799885603 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc894011990 con 0x7fc8ac157c60 2024-09-13T17:51:22.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8ab7fe700 1 -- 172.21.15.40:0/799885603 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894011d40 con 0x7fc8ac157c60 2024-09-13T17:51:22.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b3225700 1 -- 172.21.15.40:0/799885603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8ac157c60 msgr2=0x7fc8ac0c0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b3225700 1 --2- 172.21.15.40:0/799885603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8ac157c60 0x7fc8ac0c0840 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc894009850 tx=0x7fc894009b60 comp rx=0 tx=0).stop 2024-09-13T17:51:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b3225700 1 -- 172.21.15.40:0/799885603 shutdown_connections 2024-09-13T17:51:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b3225700 1 --2- 172.21.15.40:0/799885603 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8ac0c57d0 0x7fc8ac0c0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b3225700 1 --2- 172.21.15.40:0/799885603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8ac157c60 0x7fc8ac0c0840 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b3225700 1 -- 172.21.15.40:0/799885603 >> 172.21.15.40:0/799885603 conn(0x7fc8ac0cadd0 msgr2=0x7fc8ac0cb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b3225700 1 -- 172.21.15.40:0/799885603 shutdown_connections 2024-09-13T17:51:22.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.434+0000 7fc8b3225700 1 -- 172.21.15.40:0/799885603 wait complete. 2024-09-13T17:51:22.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8b3225700 1 Processor -- start 2024-09-13T17:51:22.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8b3225700 1 -- start start 2024-09-13T17:51:22.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8b3225700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8ac06d300 0x7fc8ac071410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:22.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8b3225700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8ac0c57d0 0x7fc8ac159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:22.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8b3225700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8ac157c60 0x7fc8ac15a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:22.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8b3225700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac15ff40 con 0x7fc8ac0c57d0 2024-09-13T17:51:22.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8b3225700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac159760 con 0x7fc8ac06d300 2024-09-13T17:51:22.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8b3225700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac160560 con 0x7fc8ac157c60 2024-09-13T17:51:22.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8abfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8ac0c57d0 0x7fc8ac159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:22.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8abfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8ac0c57d0 0x7fc8ac159bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55090/0 (socket says 172.21.15.40:55090) 2024-09-13T17:51:22.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8abfff700 1 -- 172.21.15.40:0/2042502304 learned_addr learned my addr 172.21.15.40:0/2042502304 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:22.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8b17c2700 1 --2- 172.21.15.40:0/2042502304 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8ac157c60 0x7fc8ac15a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:22.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8b0fc1700 1 --2- 172.21.15.40:0/2042502304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8ac06d300 0x7fc8ac071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:22.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8abfff700 1 -- 172.21.15.40:0/2042502304 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8ac157c60 msgr2=0x7fc8ac15a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:22.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8abfff700 1 --2- 172.21.15.40:0/2042502304 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8ac157c60 0x7fc8ac15a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8abfff700 1 -- 172.21.15.40:0/2042502304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8ac06d300 msgr2=0x7fc8ac071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:22.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8abfff700 1 --2- 172.21.15.40:0/2042502304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8ac06d300 0x7fc8ac071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8abfff700 1 -- 172.21.15.40:0/2042502304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8940094b0 con 0x7fc8ac0c57d0 2024-09-13T17:51:22.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8b17c2700 1 --2- 172.21.15.40:0/2042502304 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8ac157c60 0x7fc8ac15a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:22.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8b0fc1700 1 --2- 172.21.15.40:0/2042502304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8ac06d300 0x7fc8ac071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:22.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8abfff700 1 --2- 172.21.15.40:0/2042502304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8ac0c57d0 0x7fc8ac159bf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009d70 tx=0x7fc8a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8a97fa700 1 -- 172.21.15.40:0/2042502304 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a400cdc0 con 0x7fc8ac0c57d0 2024-09-13T17:51:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8a97fa700 1 -- 172.21.15.40:0/2042502304 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a400f040 con 0x7fc8ac0c57d0 2024-09-13T17:51:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8b3225700 1 -- 172.21.15.40:0/2042502304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ac15f6a0 con 0x7fc8ac0c57d0 2024-09-13T17:51:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.438+0000 7fc8a97fa700 1 -- 172.21.15.40:0/2042502304 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a40146a0 con 0x7fc8ac0c57d0 2024-09-13T17:51:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.442+0000 7fc8b3225700 1 -- 172.21.15.40:0/2042502304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ac15fc60 con 0x7fc8ac0c57d0 2024-09-13T17:51:22.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.442+0000 7fc8b3225700 1 -- 172.21.15.40:0/2042502304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ac06e330 con 0x7fc8ac0c57d0 2024-09-13T17:51:22.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.446+0000 7fc8a97fa700 1 -- 172.21.15.40:0/2042502304 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc8a4014800 con 0x7fc8ac0c57d0 2024-09-13T17:51:22.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.446+0000 7fc8a97fa700 1 --2- 172.21.15.40:0/2042502304 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc88c066730 0x7fc88c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:22.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.446+0000 7fc8a97fa700 1 -- 172.21.15.40:0/2042502304 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc8a4085770 con 0x7fc8ac0c57d0 2024-09-13T17:51:22.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.446+0000 7fc8b0fc1700 1 --2- 172.21.15.40:0/2042502304 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc88c066730 0x7fc88c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:22.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.446+0000 7fc8b0fc1700 1 --2- 172.21.15.40:0/2042502304 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc88c066730 0x7fc88c068bf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc8940090b0 tx=0x7fc894008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:22.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.450+0000 7fc8a97fa700 1 -- 172.21.15.40:0/2042502304 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc8a4052710 con 0x7fc8ac0c57d0 2024-09-13T17:51:22.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.622+0000 7fc8b3225700 1 -- 172.21.15.40:0/2042502304 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc8ac1559d0 con 0x7fc88c066730 2024-09-13T17:51:22.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.638+0000 7fc8a97fa700 1 -- 172.21.15.40:0/2042502304 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345231 (secure 0 0 0) 0x7fc8ac1559d0 con 0x7fc88c066730 2024-09-13T17:51:22.648 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:22.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.646+0000 7fc8b3225700 1 -- 172.21.15.40:0/2042502304 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc88c066730 msgr2=0x7fc88c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:22.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.646+0000 7fc8b3225700 1 --2- 172.21.15.40:0/2042502304 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc88c066730 0x7fc88c068bf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc8940090b0 tx=0x7fc894008030 comp rx=0 tx=0).stop 2024-09-13T17:51:22.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.646+0000 7fc8b3225700 1 -- 172.21.15.40:0/2042502304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8ac0c57d0 msgr2=0x7fc8ac159bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:22.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.646+0000 7fc8b3225700 1 --2- 172.21.15.40:0/2042502304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8ac0c57d0 0x7fc8ac159bf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009d70 tx=0x7fc8a400c3a0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.646+0000 7fc8b3225700 1 -- 172.21.15.40:0/2042502304 shutdown_connections 2024-09-13T17:51:22.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.646+0000 7fc8b3225700 1 --2- 172.21.15.40:0/2042502304 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc88c066730 0x7fc88c068bf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.646+0000 7fc8b3225700 1 --2- 172.21.15.40:0/2042502304 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8ac157c60 0x7fc8ac15a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.646+0000 7fc8b3225700 1 --2- 172.21.15.40:0/2042502304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8ac0c57d0 0x7fc8ac159bf0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.646+0000 7fc8b3225700 1 --2- 172.21.15.40:0/2042502304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8ac06d300 0x7fc8ac071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.646+0000 7fc8b3225700 1 -- 172.21.15.40:0/2042502304 >> 172.21.15.40:0/2042502304 conn(0x7fc8ac0cadd0 msgr2=0x7fc8ac153380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:22.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.646+0000 7fc8b3225700 1 -- 172.21.15.40:0/2042502304 shutdown_connections 2024-09-13T17:51:22.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.646+0000 7fc8b3225700 1 -- 172.21.15.40:0/2042502304 wait complete. 2024-09-13T17:51:22.652 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-13T17:51:22.664 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2024-09-13T17:51:21.013440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597677,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":286,"num_write_kb":2458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":327,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235968,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738752,"statfs":{"total":773094113280,"available":772852482048,"internally_reserved":0,"allocated":3514368,"data_stored":2067196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":574,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003462"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282601+0000","last_change":"2024-09-13T17:50:34.212158+0000","last_active":"2024-09-13T17:51:15.282601+0000","last_peered":"2024-09-13T17:51:15.282601+0000","last_clean":"2024-09-13T17:51:15.282601+0000","last_became_active":"2024-09-13T17:50:34.211867+0000","last_became_peered":"2024-09-13T17:50:34.211867+0000","last_unstale":"2024-09-13T17:51:15.282601+0000","last_undegraded":"2024-09-13T17:51:15.282601+0000","last_fullsized":"2024-09-13T17:51:15.282601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:14.671095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271433+0000","last_change":"2024-09-13T17:50:34.587396+0000","last_active":"2024-09-13T17:51:15.271433+0000","last_peered":"2024-09-13T17:51:15.271433+0000","last_clean":"2024-09-13T17:51:15.271433+0000","last_became_active":"2024-09-13T17:50:34.587108+0000","last_became_peered":"2024-09-13T17:50:34.587108+0000","last_unstale":"2024-09-13T17:51:15.271433+0000","last_undegraded":"2024-09-13T17:51:15.271433+0000","last_fullsized":"2024-09-13T17:51:15.271433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:39:16.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270087+0000","last_change":"2024-09-13T17:50:38.615059+0000","last_active":"2024-09-13T17:51:15.270087+0000","last_peered":"2024-09-13T17:51:15.270087+0000","last_clean":"2024-09-13T17:51:15.270087+0000","last_became_active":"2024-09-13T17:50:38.614723+0000","last_became_peered":"2024-09-13T17:50:38.614723+0000","last_unstale":"2024-09-13T17:51:15.270087+0000","last_undegraded":"2024-09-13T17:51:15.270087+0000","last_fullsized":"2024-09-13T17:51:15.270087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:23:01.902181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271688+0000","last_change":"2024-09-13T17:50:34.204760+0000","last_active":"2024-09-13T17:51:15.271688+0000","last_peered":"2024-09-13T17:51:15.271688+0000","last_clean":"2024-09-13T17:51:15.271688+0000","last_became_active":"2024-09-13T17:50:34.202930+0000","last_became_peered":"2024-09-13T17:50:34.202930+0000","last_unstale":"2024-09-13T17:51:15.271688+0000","last_undegraded":"2024-09-13T17:51:15.271688+0000","last_fullsized":"2024-09-13T17:51:15.271688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:59.784701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.366352+0000","last_change":"2024-09-13T17:51:16.366352+0000","last_active":"2024-09-13T17:51:16.366352+0000","last_peered":"2024-09-13T17:51:16.366352+0000","last_clean":"2024-09-13T17:51:16.366352+0000","last_became_active":"2024-09-13T17:51:16.365492+0000","last_became_peered":"2024-09-13T17:51:16.365492+0000","last_unstale":"2024-09-13T17:51:16.366352+0000","last_undegraded":"2024-09-13T17:51:16.366352+0000","last_fullsized":"2024-09-13T17:51:16.366352+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:42.192182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.029188+0000","last_change":"2024-09-13T17:50:38.618751+0000","last_active":"2024-09-13T17:50:41.029188+0000","last_peered":"2024-09-13T17:50:41.029188+0000","last_clean":"2024-09-13T17:50:41.029188+0000","last_became_active":"2024-09-13T17:50:38.618423+0000","last_became_peered":"2024-09-13T17:50:38.618423+0000","last_unstale":"2024-09-13T17:50:41.029188+0000","last_undegraded":"2024-09-13T17:50:41.029188+0000","last_fullsized":"2024-09-13T17:50:41.029188+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:50:02.418785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"25'4","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:11.230982+0000","last_change":"2024-09-13T17:50:34.214972+0000","last_active":"2024-09-13T17:51:11.230982+0000","last_peered":"2024-09-13T17:51:11.230982+0000","last_clean":"2024-09-13T17:51:11.230982+0000","last_became_active":"2024-09-13T17:50:34.214652+0000","last_became_peered":"2024-09-13T17:50:34.214652+0000","last_unstale":"2024-09-13T17:51:11.230982+0000","last_undegraded":"2024-09-13T17:51:11.230982+0000","last_fullsized":"2024-09-13T17:51:11.230982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:57.176854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.282672+0000","last_change":"2024-09-13T17:51:16.281831+0000","last_active":"2024-09-13T17:51:16.282672+0000","last_peered":"2024-09-13T17:51:16.282672+0000","last_clean":"2024-09-13T17:51:16.282672+0000","last_became_active":"2024-09-13T17:51:16.281549+0000","last_became_peered":"2024-09-13T17:51:16.281549+0000","last_unstale":"2024-09-13T17:51:16.282672+0000","last_undegraded":"2024-09-13T17:51:16.282672+0000","last_fullsized":"2024-09-13T17:51:16.282672+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:43.894437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.029352+0000","last_change":"2024-09-13T17:50:38.616219+0000","last_active":"2024-09-13T17:50:41.029352+0000","last_peered":"2024-09-13T17:50:41.029352+0000","last_clean":"2024-09-13T17:50:41.029352+0000","last_became_active":"2024-09-13T17:50:38.614329+0000","last_became_peered":"2024-09-13T17:50:38.614329+0000","last_unstale":"2024-09-13T17:50:41.029352+0000","last_undegraded":"2024-09-13T17:50:41.029352+0000","last_fullsized":"2024-09-13T17:50:41.029352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:25.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"25'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583439+0000","last_change":"2024-09-13T17:50:34.216123+0000","last_active":"2024-09-13T17:50:39.583439+0000","last_peered":"2024-09-13T17:50:39.583439+0000","last_clean":"2024-09-13T17:50:39.583439+0000","last_became_active":"2024-09-13T17:50:34.215661+0000","last_became_peered":"2024-09-13T17:50:34.215661+0000","last_unstale":"2024-09-13T17:50:39.583439+0000","last_undegraded":"2024-09-13T17:50:39.583439+0000","last_fullsized":"2024-09-13T17:50:39.583439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:20.895037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.195197+0000","last_change":"2024-09-13T17:50:34.599673+0000","last_active":"2024-09-13T17:50:39.195197+0000","last_peered":"2024-09-13T17:50:39.195197+0000","last_clean":"2024-09-13T17:50:39.195197+0000","last_became_active":"2024-09-13T17:50:34.599409+0000","last_became_peered":"2024-09-13T17:50:34.599409+0000","last_unstale":"2024-09-13T17:50:39.195197+0000","last_undegraded":"2024-09-13T17:50:39.195197+0000","last_fullsized":"2024-09-13T17:50:39.195197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:03:31.394385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.029311+0000","last_change":"2024-09-13T17:50:38.616317+0000","last_active":"2024-09-13T17:50:41.029311+0000","last_peered":"2024-09-13T17:50:41.029311+0000","last_clean":"2024-09-13T17:50:41.029311+0000","last_became_active":"2024-09-13T17:50:38.614311+0000","last_became_peered":"2024-09-13T17:50:38.614311+0000","last_unstale":"2024-09-13T17:50:41.029311+0000","last_undegraded":"2024-09-13T17:50:41.029311+0000","last_fullsized":"2024-09-13T17:50:41.029311+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:29.221522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.982805+0000","last_change":"2024-09-13T17:50:34.599490+0000","last_active":"2024-09-13T17:50:38.982805+0000","last_peered":"2024-09-13T17:50:38.982805+0000","last_clean":"2024-09-13T17:50:38.982805+0000","last_became_active":"2024-09-13T17:50:34.599207+0000","last_became_peered":"2024-09-13T17:50:34.599207+0000","last_unstale":"2024-09-13T17:50:38.982805+0000","last_undegraded":"2024-09-13T17:50:38.982805+0000","last_fullsized":"2024-09-13T17:50:38.982805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:11.178161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:06.142751+0000","last_change":"2024-09-13T17:50:34.215697+0000","last_active":"2024-09-13T17:51:06.142751+0000","last_peered":"2024-09-13T17:51:06.142751+0000","last_clean":"2024-09-13T17:51:06.142751+0000","last_became_active":"2024-09-13T17:50:34.215409+0000","last_became_peered":"2024-09-13T17:50:34.215409+0000","last_unstale":"2024-09-13T17:51:06.142751+0000","last_undegraded":"2024-09-13T17:51:06.142751+0000","last_fullsized":"2024-09-13T17:51:06.142751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:21.609567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.029485+0000","last_change":"2024-09-13T17:50:38.611817+0000","last_active":"2024-09-13T17:50:41.029485+0000","last_peered":"2024-09-13T17:50:41.029485+0000","last_clean":"2024-09-13T17:50:41.029485+0000","last_became_active":"2024-09-13T17:50:38.611263+0000","last_became_peered":"2024-09-13T17:50:38.611263+0000","last_unstale":"2024-09-13T17:50:41.029485+0000","last_undegraded":"2024-09-13T17:50:41.029485+0000","last_fullsized":"2024-09-13T17:50:41.029485+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:56:47.466672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271037+0000","last_change":"2024-09-13T17:50:34.586307+0000","last_active":"2024-09-13T17:51:15.271037+0000","last_peered":"2024-09-13T17:51:15.271037+0000","last_clean":"2024-09-13T17:51:15.271037+0000","last_became_active":"2024-09-13T17:50:34.585831+0000","last_became_peered":"2024-09-13T17:50:34.585831+0000","last_unstale":"2024-09-13T17:51:15.271037+0000","last_undegraded":"2024-09-13T17:51:15.271037+0000","last_fullsized":"2024-09-13T17:51:15.271037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:40.912352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.280577+0000","last_change":"2024-09-13T17:50:34.211850+0000","last_active":"2024-09-13T17:51:15.280577+0000","last_peered":"2024-09-13T17:51:15.280577+0000","last_clean":"2024-09-13T17:51:15.280577+0000","last_became_active":"2024-09-13T17:50:34.211594+0000","last_became_peered":"2024-09-13T17:50:34.211594+0000","last_unstale":"2024-09-13T17:51:15.280577+0000","last_undegraded":"2024-09-13T17:51:15.280577+0000","last_fullsized":"2024-09-13T17:51:15.280577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:35:02.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.029420+0000","last_change":"2024-09-13T17:50:38.613576+0000","last_active":"2024-09-13T17:50:41.029420+0000","last_peered":"2024-09-13T17:50:41.029420+0000","last_clean":"2024-09-13T17:50:41.029420+0000","last_became_active":"2024-09-13T17:50:38.612577+0000","last_became_peered":"2024-09-13T17:50:38.612577+0000","last_unstale":"2024-09-13T17:50:41.029420+0000","last_undegraded":"2024-09-13T17:50:41.029420+0000","last_fullsized":"2024-09-13T17:50:41.029420+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:25.551060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272679+0000","last_change":"2024-09-13T17:50:34.587273+0000","last_active":"2024-09-13T17:51:15.272679+0000","last_peered":"2024-09-13T17:51:15.272679+0000","last_clean":"2024-09-13T17:51:15.272679+0000","last_became_active":"2024-09-13T17:50:34.587027+0000","last_became_peered":"2024-09-13T17:50:34.587027+0000","last_unstale":"2024-09-13T17:51:15.272679+0000","last_undegraded":"2024-09-13T17:51:15.272679+0000","last_fullsized":"2024-09-13T17:51:15.272679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:58.826737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.280692+0000","last_change":"2024-09-13T17:50:34.211845+0000","last_active":"2024-09-13T17:51:15.280692+0000","last_peered":"2024-09-13T17:51:15.280692+0000","last_clean":"2024-09-13T17:51:15.280692+0000","last_became_active":"2024-09-13T17:50:34.211568+0000","last_became_peered":"2024-09-13T17:50:34.211568+0000","last_unstale":"2024-09-13T17:51:15.280692+0000","last_undegraded":"2024-09-13T17:51:15.280692+0000","last_fullsized":"2024-09-13T17:51:15.280692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:56.348984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281699+0000","last_change":"2024-09-13T17:50:38.610292+0000","last_active":"2024-09-13T17:51:15.281699+0000","last_peered":"2024-09-13T17:51:15.281699+0000","last_clean":"2024-09-13T17:51:15.281699+0000","last_became_active":"2024-09-13T17:50:38.609832+0000","last_became_peered":"2024-09-13T17:50:38.609832+0000","last_unstale":"2024-09-13T17:51:15.281699+0000","last_undegraded":"2024-09-13T17:51:15.281699+0000","last_fullsized":"2024-09-13T17:51:15.281699+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:13.783068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501657+0000","last_change":"2024-09-13T17:50:34.593566+0000","last_active":"2024-09-13T17:50:39.501657+0000","last_peered":"2024-09-13T17:50:39.501657+0000","last_clean":"2024-09-13T17:50:39.501657+0000","last_became_active":"2024-09-13T17:50:34.588794+0000","last_became_peered":"2024-09-13T17:50:34.588794+0000","last_unstale":"2024-09-13T17:50:39.501657+0000","last_undegraded":"2024-09-13T17:50:39.501657+0000","last_fullsized":"2024-09-13T17:50:39.501657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:13.192453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:06.142903+0000","last_change":"2024-09-13T17:50:34.214638+0000","last_active":"2024-09-13T17:51:06.142903+0000","last_peered":"2024-09-13T17:51:06.142903+0000","last_clean":"2024-09-13T17:51:06.142903+0000","last_became_active":"2024-09-13T17:50:34.214292+0000","last_became_peered":"2024-09-13T17:50:34.214292+0000","last_unstale":"2024-09-13T17:51:06.142903+0000","last_undegraded":"2024-09-13T17:51:06.142903+0000","last_fullsized":"2024-09-13T17:51:06.142903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:57.377104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.029513+0000","last_change":"2024-09-13T17:50:38.613368+0000","last_active":"2024-09-13T17:50:41.029513+0000","last_peered":"2024-09-13T17:50:41.029513+0000","last_clean":"2024-09-13T17:50:41.029513+0000","last_became_active":"2024-09-13T17:50:38.613010+0000","last_became_peered":"2024-09-13T17:50:38.613010+0000","last_unstale":"2024-09-13T17:50:41.029513+0000","last_undegraded":"2024-09-13T17:50:41.029513+0000","last_fullsized":"2024-09-13T17:50:41.029513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:42.706625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282530+0000","last_change":"2024-09-13T17:50:34.582327+0000","last_active":"2024-09-13T17:51:15.282530+0000","last_peered":"2024-09-13T17:51:15.282530+0000","last_clean":"2024-09-13T17:51:15.282530+0000","last_became_active":"2024-09-13T17:50:34.582050+0000","last_became_peered":"2024-09-13T17:50:34.582050+0000","last_unstale":"2024-09-13T17:51:15.282530+0000","last_undegraded":"2024-09-13T17:51:15.282530+0000","last_fullsized":"2024-09-13T17:51:15.282530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:52.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:06.145008+0000","last_change":"2024-09-13T17:50:34.211512+0000","last_active":"2024-09-13T17:51:06.145008+0000","last_peered":"2024-09-13T17:51:06.145008+0000","last_clean":"2024-09-13T17:51:06.145008+0000","last_became_active":"2024-09-13T17:50:34.211230+0000","last_became_peered":"2024-09-13T17:50:34.211230+0000","last_unstale":"2024-09-13T17:51:06.145008+0000","last_undegraded":"2024-09-13T17:51:06.145008+0000","last_fullsized":"2024-09-13T17:51:06.145008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:40.707777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.029264+0000","last_change":"2024-09-13T17:50:38.608255+0000","last_active":"2024-09-13T17:50:41.029264+0000","last_peered":"2024-09-13T17:50:41.029264+0000","last_clean":"2024-09-13T17:50:41.029264+0000","last_became_active":"2024-09-13T17:50:38.606697+0000","last_became_peered":"2024-09-13T17:50:38.606697+0000","last_unstale":"2024-09-13T17:50:41.029264+0000","last_undegraded":"2024-09-13T17:50:41.029264+0000","last_fullsized":"2024-09-13T17:50:41.029264+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:47.265007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272916+0000","last_change":"2024-09-13T17:50:34.594476+0000","last_active":"2024-09-13T17:51:15.272916+0000","last_peered":"2024-09-13T17:51:15.272916+0000","last_clean":"2024-09-13T17:51:15.272916+0000","last_became_active":"2024-09-13T17:50:34.594215+0000","last_became_peered":"2024-09-13T17:50:34.594215+0000","last_unstale":"2024-09-13T17:51:15.272916+0000","last_undegraded":"2024-09-13T17:51:15.272916+0000","last_fullsized":"2024-09-13T17:51:15.272916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:02:27.528019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281479+0000","last_change":"2024-09-13T17:50:34.204018+0000","last_active":"2024-09-13T17:51:15.281479+0000","last_peered":"2024-09-13T17:51:15.281479+0000","last_clean":"2024-09-13T17:51:15.281479+0000","last_became_active":"2024-09-13T17:50:34.203702+0000","last_became_peered":"2024-09-13T17:50:34.203702+0000","last_unstale":"2024-09-13T17:51:15.281479+0000","last_undegraded":"2024-09-13T17:51:15.281479+0000","last_fullsized":"2024-09-13T17:51:15.281479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:44.431226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.365782+0000","last_change":"2024-09-13T17:51:16.365782+0000","last_active":"2024-09-13T17:51:16.365782+0000","last_peered":"2024-09-13T17:51:16.365782+0000","last_clean":"2024-09-13T17:51:16.365782+0000","last_became_active":"2024-09-13T17:51:16.364540+0000","last_became_peered":"2024-09-13T17:51:16.364540+0000","last_unstale":"2024-09-13T17:51:16.365782+0000","last_undegraded":"2024-09-13T17:51:16.365782+0000","last_fullsized":"2024-09-13T17:51:16.365782+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:18.753557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281218+0000","last_change":"2024-09-13T17:50:34.586802+0000","last_active":"2024-09-13T17:51:15.281218+0000","last_peered":"2024-09-13T17:51:15.281218+0000","last_clean":"2024-09-13T17:51:15.281218+0000","last_became_active":"2024-09-13T17:50:34.586552+0000","last_became_peered":"2024-09-13T17:50:34.586552+0000","last_unstale":"2024-09-13T17:51:15.281218+0000","last_undegraded":"2024-09-13T17:51:15.281218+0000","last_fullsized":"2024-09-13T17:51:15.281218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:43.383975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:06.145771+0000","last_change":"2024-09-13T17:50:34.217083+0000","last_active":"2024-09-13T17:51:06.145771+0000","last_peered":"2024-09-13T17:51:06.145771+0000","last_clean":"2024-09-13T17:51:06.145771+0000","last_became_active":"2024-09-13T17:50:34.216758+0000","last_became_peered":"2024-09-13T17:50:34.216758+0000","last_unstale":"2024-09-13T17:51:06.145771+0000","last_undegraded":"2024-09-13T17:51:06.145771+0000","last_fullsized":"2024-09-13T17:51:06.145771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:36.473942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277926+0000","last_change":"2024-09-13T17:50:38.614667+0000","last_active":"2024-09-13T17:51:15.277926+0000","last_peered":"2024-09-13T17:51:15.277926+0000","last_clean":"2024-09-13T17:51:15.277926+0000","last_became_active":"2024-09-13T17:50:38.614284+0000","last_became_peered":"2024-09-13T17:50:38.614284+0000","last_unstale":"2024-09-13T17:51:15.277926+0000","last_undegraded":"2024-09-13T17:51:15.277926+0000","last_fullsized":"2024-09-13T17:51:15.277926+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:29.616794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.195375+0000","last_change":"2024-09-13T17:50:34.588603+0000","last_active":"2024-09-13T17:50:39.195375+0000","last_peered":"2024-09-13T17:50:39.195375+0000","last_clean":"2024-09-13T17:50:39.195375+0000","last_became_active":"2024-09-13T17:50:34.588317+0000","last_became_peered":"2024-09-13T17:50:34.588317+0000","last_unstale":"2024-09-13T17:50:39.195375+0000","last_undegraded":"2024-09-13T17:50:39.195375+0000","last_fullsized":"2024-09-13T17:50:39.195375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:03.743130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281302+0000","last_change":"2024-09-13T17:50:34.204335+0000","last_active":"2024-09-13T17:51:15.281302+0000","last_peered":"2024-09-13T17:51:15.281302+0000","last_clean":"2024-09-13T17:51:15.281302+0000","last_became_active":"2024-09-13T17:50:34.204060+0000","last_became_peered":"2024-09-13T17:50:34.204060+0000","last_unstale":"2024-09-13T17:51:15.281302+0000","last_undegraded":"2024-09-13T17:51:15.281302+0000","last_fullsized":"2024-09-13T17:51:15.281302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:54:41.831344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279638+0000","last_change":"2024-09-13T17:50:38.619590+0000","last_active":"2024-09-13T17:51:15.279638+0000","last_peered":"2024-09-13T17:51:15.279638+0000","last_clean":"2024-09-13T17:51:15.279638+0000","last_became_active":"2024-09-13T17:50:38.617257+0000","last_became_peered":"2024-09-13T17:50:38.617257+0000","last_unstale":"2024-09-13T17:51:15.279638+0000","last_undegraded":"2024-09-13T17:51:15.279638+0000","last_fullsized":"2024-09-13T17:51:15.279638+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:34.870291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.502147+0000","last_change":"2024-09-13T17:50:34.588466+0000","last_active":"2024-09-13T17:50:39.502147+0000","last_peered":"2024-09-13T17:50:39.502147+0000","last_clean":"2024-09-13T17:50:39.502147+0000","last_became_active":"2024-09-13T17:50:34.588063+0000","last_became_peered":"2024-09-13T17:50:34.588063+0000","last_unstale":"2024-09-13T17:50:39.502147+0000","last_undegraded":"2024-09-13T17:50:39.502147+0000","last_fullsized":"2024-09-13T17:50:39.502147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:43:00.669203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:11.233511+0000","last_change":"2024-09-13T17:50:34.209239+0000","last_active":"2024-09-13T17:51:11.233511+0000","last_peered":"2024-09-13T17:51:11.233511+0000","last_clean":"2024-09-13T17:51:11.233511+0000","last_became_active":"2024-09-13T17:50:34.208196+0000","last_became_peered":"2024-09-13T17:50:34.208196+0000","last_unstale":"2024-09-13T17:51:11.233511+0000","last_undegraded":"2024-09-13T17:51:11.233511+0000","last_fullsized":"2024-09-13T17:51:11.233511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:39.253917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281566+0000","last_change":"2024-09-13T17:50:38.611021+0000","last_active":"2024-09-13T17:51:15.281566+0000","last_peered":"2024-09-13T17:51:15.281566+0000","last_clean":"2024-09-13T17:51:15.281566+0000","last_became_active":"2024-09-13T17:50:38.609794+0000","last_became_peered":"2024-09-13T17:50:38.609794+0000","last_unstale":"2024-09-13T17:51:15.281566+0000","last_undegraded":"2024-09-13T17:51:15.281566+0000","last_fullsized":"2024-09-13T17:51:15.281566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:59.980236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281553+0000","last_change":"2024-09-13T17:50:34.584290+0000","last_active":"2024-09-13T17:51:15.281553+0000","last_peered":"2024-09-13T17:51:15.281553+0000","last_clean":"2024-09-13T17:51:15.281553+0000","last_became_active":"2024-09-13T17:50:34.584023+0000","last_became_peered":"2024-09-13T17:50:34.584023+0000","last_unstale":"2024-09-13T17:51:15.281553+0000","last_undegraded":"2024-09-13T17:51:15.281553+0000","last_fullsized":"2024-09-13T17:51:15.281553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:11.304986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"25'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278819+0000","last_change":"2024-09-13T17:50:34.198932+0000","last_active":"2024-09-13T17:51:15.278819+0000","last_peered":"2024-09-13T17:51:15.278819+0000","last_clean":"2024-09-13T17:51:15.278819+0000","last_became_active":"2024-09-13T17:50:34.198603+0000","last_became_peered":"2024-09-13T17:50:34.198603+0000","last_unstale":"2024-09-13T17:51:15.278819+0000","last_undegraded":"2024-09-13T17:51:15.278819+0000","last_fullsized":"2024-09-13T17:51:15.278819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:00.917126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281714+0000","last_change":"2024-09-13T17:50:38.611326+0000","last_active":"2024-09-13T17:51:15.281714+0000","last_peered":"2024-09-13T17:51:15.281714+0000","last_clean":"2024-09-13T17:51:15.281714+0000","last_became_active":"2024-09-13T17:50:38.610900+0000","last_became_peered":"2024-09-13T17:50:38.610900+0000","last_unstale":"2024-09-13T17:51:15.281714+0000","last_undegraded":"2024-09-13T17:51:15.281714+0000","last_fullsized":"2024-09-13T17:51:15.281714+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:17:09.994736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276990+0000","last_change":"2024-09-13T17:50:34.594482+0000","last_active":"2024-09-13T17:51:15.276990+0000","last_peered":"2024-09-13T17:51:15.276990+0000","last_clean":"2024-09-13T17:51:15.276990+0000","last_became_active":"2024-09-13T17:50:34.594201+0000","last_became_peered":"2024-09-13T17:50:34.594201+0000","last_unstale":"2024-09-13T17:51:15.276990+0000","last_undegraded":"2024-09-13T17:51:15.276990+0000","last_fullsized":"2024-09-13T17:51:15.276990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:06.697507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:11.233385+0000","last_change":"2024-09-13T17:50:34.218376+0000","last_active":"2024-09-13T17:51:11.233385+0000","last_peered":"2024-09-13T17:51:11.233385+0000","last_clean":"2024-09-13T17:51:11.233385+0000","last_became_active":"2024-09-13T17:50:34.218069+0000","last_became_peered":"2024-09-13T17:50:34.218069+0000","last_unstale":"2024-09-13T17:51:11.233385+0000","last_undegraded":"2024-09-13T17:51:11.233385+0000","last_fullsized":"2024-09-13T17:51:11.233385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:53:54.288969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277801+0000","last_change":"2024-09-13T17:50:38.613206+0000","last_active":"2024-09-13T17:51:15.277801+0000","last_peered":"2024-09-13T17:51:15.277801+0000","last_clean":"2024-09-13T17:51:15.277801+0000","last_became_active":"2024-09-13T17:50:38.612856+0000","last_became_peered":"2024-09-13T17:50:38.612856+0000","last_unstale":"2024-09-13T17:51:15.277801+0000","last_undegraded":"2024-09-13T17:51:15.277801+0000","last_fullsized":"2024-09-13T17:51:15.277801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:31:20.190996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270767+0000","last_change":"2024-09-13T17:50:34.582931+0000","last_active":"2024-09-13T17:51:15.270767+0000","last_peered":"2024-09-13T17:51:15.270767+0000","last_clean":"2024-09-13T17:51:15.270767+0000","last_became_active":"2024-09-13T17:50:34.582446+0000","last_became_peered":"2024-09-13T17:50:34.582446+0000","last_unstale":"2024-09-13T17:51:15.270767+0000","last_undegraded":"2024-09-13T17:51:15.270767+0000","last_fullsized":"2024-09-13T17:51:15.270767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:30.554785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:11.234327+0000","last_change":"2024-09-13T17:50:34.201237+0000","last_active":"2024-09-13T17:51:11.234327+0000","last_peered":"2024-09-13T17:51:11.234327+0000","last_clean":"2024-09-13T17:51:11.234327+0000","last_became_active":"2024-09-13T17:50:34.200873+0000","last_became_peered":"2024-09-13T17:50:34.200873+0000","last_unstale":"2024-09-13T17:51:11.234327+0000","last_undegraded":"2024-09-13T17:51:11.234327+0000","last_fullsized":"2024-09-13T17:51:11.234327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:02.711342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":169,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.984391+0000","last_change":"2024-09-13T17:50:22.095367+0000","last_active":"2024-09-13T17:50:38.984391+0000","last_peered":"2024-09-13T17:50:38.984391+0000","last_clean":"2024-09-13T17:50:38.984391+0000","last_became_active":"2024-09-13T17:50:22.093734+0000","last_became_peered":"2024-09-13T17:50:22.093734+0000","last_unstale":"2024-09-13T17:50:38.984391+0000","last_undegraded":"2024-09-13T17:50:38.984391+0000","last_fullsized":"2024-09-13T17:50:38.984391+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_clean_scrub_stamp":"2024-09-13T17:50:21.057812+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:23.531613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.014187+0000","last_change":"2024-09-13T17:50:38.618880+0000","last_active":"2024-09-13T17:50:41.014187+0000","last_peered":"2024-09-13T17:50:41.014187+0000","last_clean":"2024-09-13T17:50:41.014187+0000","last_became_active":"2024-09-13T17:50:38.618488+0000","last_became_peered":"2024-09-13T17:50:38.618488+0000","last_unstale":"2024-09-13T17:50:41.014187+0000","last_undegraded":"2024-09-13T17:50:41.014187+0000","last_fullsized":"2024-09-13T17:50:41.014187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:03.431876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.985276+0000","last_change":"2024-09-13T17:50:34.585881+0000","last_active":"2024-09-13T17:50:38.985276+0000","last_peered":"2024-09-13T17:50:38.985276+0000","last_clean":"2024-09-13T17:50:38.985276+0000","last_became_active":"2024-09-13T17:50:34.585166+0000","last_became_peered":"2024-09-13T17:50:34.585166+0000","last_unstale":"2024-09-13T17:50:38.985276+0000","last_undegraded":"2024-09-13T17:50:38.985276+0000","last_fullsized":"2024-09-13T17:50:38.985276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:06.654658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279549+0000","last_change":"2024-09-13T17:50:34.210934+0000","last_active":"2024-09-13T17:51:15.279549+0000","last_peered":"2024-09-13T17:51:15.279549+0000","last_clean":"2024-09-13T17:51:15.279549+0000","last_became_active":"2024-09-13T17:50:34.210589+0000","last_became_peered":"2024-09-13T17:50:34.210589+0000","last_unstale":"2024-09-13T17:51:15.279549+0000","last_undegraded":"2024-09-13T17:51:15.279549+0000","last_fullsized":"2024-09-13T17:51:15.279549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:26.213295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"peering","last_fresh":"2024-09-13T17:51:15.275839+0000","last_change":"2024-09-13T17:51:15.275971+0000","last_active":"2024-09-13T17:51:15.275839+0000","last_peered":"2024-09-13T17:50:41.014101+0000","last_clean":"2024-09-13T17:50:41.014101+0000","last_became_active":"2024-09-13T17:50:38.615358+0000","last_became_peered":"2024-09-13T17:50:38.615358+0000","last_unstale":"2024-09-13T17:51:15.275839+0000","last_undegraded":"2024-09-13T17:51:15.275839+0000","last_fullsized":"2024-09-13T17:51:15.275839+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:31.449312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1,2,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.195149+0000","last_change":"2024-09-13T17:50:34.590169+0000","last_active":"2024-09-13T17:50:39.195149+0000","last_peered":"2024-09-13T17:50:39.195149+0000","last_clean":"2024-09-13T17:50:39.195149+0000","last_became_active":"2024-09-13T17:50:34.589434+0000","last_became_peered":"2024-09-13T17:50:34.589434+0000","last_unstale":"2024-09-13T17:50:39.195149+0000","last_undegraded":"2024-09-13T17:50:39.195149+0000","last_fullsized":"2024-09-13T17:50:39.195149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:12.545412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"25'1","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281689+0000","last_change":"2024-09-13T17:50:34.204269+0000","last_active":"2024-09-13T17:51:15.281689+0000","last_peered":"2024-09-13T17:51:15.281689+0000","last_clean":"2024-09-13T17:51:15.281689+0000","last_became_active":"2024-09-13T17:50:34.203874+0000","last_became_peered":"2024-09-13T17:50:34.203874+0000","last_unstale":"2024-09-13T17:51:15.281689+0000","last_undegraded":"2024-09-13T17:51:15.281689+0000","last_fullsized":"2024-09-13T17:51:15.281689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:10.410267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278070+0000","last_change":"2024-09-13T17:50:38.615989+0000","last_active":"2024-09-13T17:51:15.278070+0000","last_peered":"2024-09-13T17:51:15.278070+0000","last_clean":"2024-09-13T17:51:15.278070+0000","last_became_active":"2024-09-13T17:50:38.615530+0000","last_became_peered":"2024-09-13T17:50:38.615530+0000","last_unstale":"2024-09-13T17:51:15.278070+0000","last_undegraded":"2024-09-13T17:51:15.278070+0000","last_fullsized":"2024-09-13T17:51:15.278070+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:41.064092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.985714+0000","last_change":"2024-09-13T17:50:34.595120+0000","last_active":"2024-09-13T17:50:38.985714+0000","last_peered":"2024-09-13T17:50:38.985714+0000","last_clean":"2024-09-13T17:50:38.985714+0000","last_became_active":"2024-09-13T17:50:34.594842+0000","last_became_peered":"2024-09-13T17:50:34.594842+0000","last_unstale":"2024-09-13T17:50:38.985714+0000","last_undegraded":"2024-09-13T17:50:38.985714+0000","last_fullsized":"2024-09-13T17:50:38.985714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:15:38.982524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"25'2","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279806+0000","last_change":"2024-09-13T17:50:34.199957+0000","last_active":"2024-09-13T17:51:15.279806+0000","last_peered":"2024-09-13T17:51:15.279806+0000","last_clean":"2024-09-13T17:51:15.279806+0000","last_became_active":"2024-09-13T17:50:34.199650+0000","last_became_peered":"2024-09-13T17:50:34.199650+0000","last_unstale":"2024-09-13T17:51:15.279806+0000","last_undegraded":"2024-09-13T17:51:15.279806+0000","last_fullsized":"2024-09-13T17:51:15.279806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:26.929549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.013644+0000","last_change":"2024-09-13T17:50:38.609731+0000","last_active":"2024-09-13T17:50:41.013644+0000","last_peered":"2024-09-13T17:50:41.013644+0000","last_clean":"2024-09-13T17:50:41.013644+0000","last_became_active":"2024-09-13T17:50:38.609354+0000","last_became_peered":"2024-09-13T17:50:38.609354+0000","last_unstale":"2024-09-13T17:50:41.013644+0000","last_undegraded":"2024-09-13T17:50:41.013644+0000","last_fullsized":"2024-09-13T17:50:41.013644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:14.142369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.985412+0000","last_change":"2024-09-13T17:50:34.595146+0000","last_active":"2024-09-13T17:50:38.985412+0000","last_peered":"2024-09-13T17:50:38.985412+0000","last_clean":"2024-09-13T17:50:38.985412+0000","last_became_active":"2024-09-13T17:50:34.594876+0000","last_became_peered":"2024-09-13T17:50:34.594876+0000","last_unstale":"2024-09-13T17:50:38.985412+0000","last_undegraded":"2024-09-13T17:50:38.985412+0000","last_fullsized":"2024-09-13T17:50:38.985412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:26:10.638438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"25'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.582895+0000","last_change":"2024-09-13T17:50:34.200640+0000","last_active":"2024-09-13T17:50:39.582895+0000","last_peered":"2024-09-13T17:50:39.582895+0000","last_clean":"2024-09-13T17:50:39.582895+0000","last_became_active":"2024-09-13T17:50:34.200197+0000","last_became_peered":"2024-09-13T17:50:34.200197+0000","last_unstale":"2024-09-13T17:50:39.582895+0000","last_undegraded":"2024-09-13T17:50:39.582895+0000","last_fullsized":"2024-09-13T17:50:39.582895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:37:53.039720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"peering","last_fresh":"2024-09-13T17:51:15.270506+0000","last_change":"2024-09-13T17:51:15.270770+0000","last_active":"2024-09-13T17:51:15.270506+0000","last_peered":"2024-09-13T17:50:41.029222+0000","last_clean":"2024-09-13T17:50:41.029222+0000","last_became_active":"2024-09-13T17:50:38.618491+0000","last_became_peered":"2024-09-13T17:50:38.618491+0000","last_unstale":"2024-09-13T17:51:15.270506+0000","last_undegraded":"2024-09-13T17:51:15.270506+0000","last_fullsized":"2024-09-13T17:51:15.270506+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:30:49.103522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3,6],"acting":[4,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,3,6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.984764+0000","last_change":"2024-09-13T17:50:34.585741+0000","last_active":"2024-09-13T17:50:38.984764+0000","last_peered":"2024-09-13T17:50:38.984764+0000","last_clean":"2024-09-13T17:50:38.984764+0000","last_became_active":"2024-09-13T17:50:34.585464+0000","last_became_peered":"2024-09-13T17:50:34.585464+0000","last_unstale":"2024-09-13T17:50:38.984764+0000","last_undegraded":"2024-09-13T17:50:38.984764+0000","last_fullsized":"2024-09-13T17:50:38.984764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:59:22.031459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"25'15","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278697+0000","last_change":"2024-09-13T17:50:34.201272+0000","last_active":"2024-09-13T17:51:15.278697+0000","last_peered":"2024-09-13T17:51:15.278697+0000","last_clean":"2024-09-13T17:51:15.278697+0000","last_became_active":"2024-09-13T17:50:34.200936+0000","last_became_peered":"2024-09-13T17:50:34.200936+0000","last_unstale":"2024-09-13T17:51:15.278697+0000","last_undegraded":"2024-09-13T17:51:15.278697+0000","last_fullsized":"2024-09-13T17:51:15.278697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:52.094879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277648+0000","last_change":"2024-09-13T17:50:38.615229+0000","last_active":"2024-09-13T17:51:15.277648+0000","last_peered":"2024-09-13T17:51:15.277648+0000","last_clean":"2024-09-13T17:51:15.277648+0000","last_became_active":"2024-09-13T17:50:38.614739+0000","last_became_peered":"2024-09-13T17:50:38.614739+0000","last_unstale":"2024-09-13T17:51:15.277648+0000","last_undegraded":"2024-09-13T17:51:15.277648+0000","last_fullsized":"2024-09-13T17:51:15.277648+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:18.619529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.982245+0000","last_change":"2024-09-13T17:50:34.588848+0000","last_active":"2024-09-13T17:50:38.982245+0000","last_peered":"2024-09-13T17:50:38.982245+0000","last_clean":"2024-09-13T17:50:38.982245+0000","last_became_active":"2024-09-13T17:50:34.588565+0000","last_became_peered":"2024-09-13T17:50:34.588565+0000","last_unstale":"2024-09-13T17:50:38.982245+0000","last_undegraded":"2024-09-13T17:50:38.982245+0000","last_fullsized":"2024-09-13T17:50:38.982245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:47.927865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:11.235969+0000","last_change":"2024-09-13T17:50:34.199008+0000","last_active":"2024-09-13T17:51:11.235969+0000","last_peered":"2024-09-13T17:51:11.235969+0000","last_clean":"2024-09-13T17:51:11.235969+0000","last_became_active":"2024-09-13T17:50:34.197839+0000","last_became_peered":"2024-09-13T17:50:34.197839+0000","last_unstale":"2024-09-13T17:51:11.235969+0000","last_undegraded":"2024-09-13T17:51:11.235969+0000","last_fullsized":"2024-09-13T17:51:11.235969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:51.430710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272409+0000","last_change":"2024-09-13T17:50:38.611080+0000","last_active":"2024-09-13T17:51:15.272409+0000","last_peered":"2024-09-13T17:51:15.272409+0000","last_clean":"2024-09-13T17:51:15.272409+0000","last_became_active":"2024-09-13T17:50:38.610688+0000","last_became_peered":"2024-09-13T17:50:38.610688+0000","last_unstale":"2024-09-13T17:51:15.272409+0000","last_undegraded":"2024-09-13T17:51:15.272409+0000","last_fullsized":"2024-09-13T17:51:15.272409+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:28.527617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.984724+0000","last_change":"2024-09-13T17:50:34.594763+0000","last_active":"2024-09-13T17:50:38.984724+0000","last_peered":"2024-09-13T17:50:38.984724+0000","last_clean":"2024-09-13T17:50:38.984724+0000","last_became_active":"2024-09-13T17:50:34.594471+0000","last_became_peered":"2024-09-13T17:50:34.594471+0000","last_unstale":"2024-09-13T17:50:38.984724+0000","last_undegraded":"2024-09-13T17:50:38.984724+0000","last_fullsized":"2024-09-13T17:50:38.984724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:20:58.718271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:06.145238+0000","last_change":"2024-09-13T17:50:34.209432+0000","last_active":"2024-09-13T17:51:06.145238+0000","last_peered":"2024-09-13T17:51:06.145238+0000","last_clean":"2024-09-13T17:51:06.145238+0000","last_became_active":"2024-09-13T17:50:34.208817+0000","last_became_peered":"2024-09-13T17:50:34.208817+0000","last_unstale":"2024-09-13T17:51:06.145238+0000","last_undegraded":"2024-09-13T17:51:06.145238+0000","last_fullsized":"2024-09-13T17:51:06.145238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:49.587528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272159+0000","last_change":"2024-09-13T17:50:38.609461+0000","last_active":"2024-09-13T17:51:15.272159+0000","last_peered":"2024-09-13T17:51:15.272159+0000","last_clean":"2024-09-13T17:51:15.272159+0000","last_became_active":"2024-09-13T17:50:38.609128+0000","last_became_peered":"2024-09-13T17:50:38.609128+0000","last_unstale":"2024-09-13T17:51:15.272159+0000","last_undegraded":"2024-09-13T17:51:15.272159+0000","last_fullsized":"2024-09-13T17:51:15.272159+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:02.286825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.283064+0000","last_change":"2024-09-13T17:50:34.592970+0000","last_active":"2024-09-13T17:51:15.283064+0000","last_peered":"2024-09-13T17:51:15.283064+0000","last_clean":"2024-09-13T17:51:15.283064+0000","last_became_active":"2024-09-13T17:50:34.592669+0000","last_became_peered":"2024-09-13T17:50:34.592669+0000","last_unstale":"2024-09-13T17:51:15.283064+0000","last_undegraded":"2024-09-13T17:51:15.283064+0000","last_fullsized":"2024-09-13T17:51:15.283064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:52.031261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282599+0000","last_change":"2024-09-13T17:50:34.200798+0000","last_active":"2024-09-13T17:51:15.282599+0000","last_peered":"2024-09-13T17:51:15.282599+0000","last_clean":"2024-09-13T17:51:15.282599+0000","last_became_active":"2024-09-13T17:50:34.200529+0000","last_became_peered":"2024-09-13T17:50:34.200529+0000","last_unstale":"2024-09-13T17:51:15.282599+0000","last_undegraded":"2024-09-13T17:51:15.282599+0000","last_fullsized":"2024-09-13T17:51:15.282599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:26.334296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.013911+0000","last_change":"2024-09-13T17:50:38.606287+0000","last_active":"2024-09-13T17:50:41.013911+0000","last_peered":"2024-09-13T17:50:41.013911+0000","last_clean":"2024-09-13T17:50:41.013911+0000","last_became_active":"2024-09-13T17:50:38.605826+0000","last_became_peered":"2024-09-13T17:50:38.605826+0000","last_unstale":"2024-09-13T17:50:41.013911+0000","last_undegraded":"2024-09-13T17:50:41.013911+0000","last_fullsized":"2024-09-13T17:50:41.013911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:18.709454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.985113+0000","last_change":"2024-09-13T17:50:34.591579+0000","last_active":"2024-09-13T17:50:38.985113+0000","last_peered":"2024-09-13T17:50:38.985113+0000","last_clean":"2024-09-13T17:50:38.985113+0000","last_became_active":"2024-09-13T17:50:34.586165+0000","last_became_peered":"2024-09-13T17:50:34.586165+0000","last_unstale":"2024-09-13T17:50:38.985113+0000","last_undegraded":"2024-09-13T17:50:38.985113+0000","last_fullsized":"2024-09-13T17:50:38.985113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:56:19.205176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'11","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:06.146224+0000","last_change":"2024-09-13T17:50:34.203218+0000","last_active":"2024-09-13T17:51:06.146224+0000","last_peered":"2024-09-13T17:51:06.146224+0000","last_clean":"2024-09-13T17:51:06.146224+0000","last_became_active":"2024-09-13T17:50:34.202948+0000","last_became_peered":"2024-09-13T17:50:34.202948+0000","last_unstale":"2024-09-13T17:51:06.146224+0000","last_undegraded":"2024-09-13T17:51:06.146224+0000","last_fullsized":"2024-09-13T17:51:06.146224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:46.535569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.280382+0000","last_change":"2024-09-13T17:50:38.610597+0000","last_active":"2024-09-13T17:51:15.280382+0000","last_peered":"2024-09-13T17:51:15.280382+0000","last_clean":"2024-09-13T17:51:15.280382+0000","last_became_active":"2024-09-13T17:50:38.610055+0000","last_became_peered":"2024-09-13T17:50:38.610055+0000","last_unstale":"2024-09-13T17:51:15.280382+0000","last_undegraded":"2024-09-13T17:51:15.280382+0000","last_fullsized":"2024-09-13T17:51:15.280382+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:55.754936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.502265+0000","last_change":"2024-09-13T17:50:34.593963+0000","last_active":"2024-09-13T17:50:39.502265+0000","last_peered":"2024-09-13T17:50:39.502265+0000","last_clean":"2024-09-13T17:50:39.502265+0000","last_became_active":"2024-09-13T17:50:34.593454+0000","last_became_peered":"2024-09-13T17:50:34.593454+0000","last_unstale":"2024-09-13T17:50:39.502265+0000","last_undegraded":"2024-09-13T17:50:39.502265+0000","last_fullsized":"2024-09-13T17:50:39.502265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:18.883521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281506+0000","last_change":"2024-09-13T17:50:34.207936+0000","last_active":"2024-09-13T17:51:15.281506+0000","last_peered":"2024-09-13T17:51:15.281506+0000","last_clean":"2024-09-13T17:51:15.281506+0000","last_became_active":"2024-09-13T17:50:34.207605+0000","last_became_peered":"2024-09-13T17:50:34.207605+0000","last_unstale":"2024-09-13T17:51:15.281506+0000","last_undegraded":"2024-09-13T17:51:15.281506+0000","last_fullsized":"2024-09-13T17:51:15.281506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:05.494406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.029376+0000","last_change":"2024-09-13T17:50:38.616297+0000","last_active":"2024-09-13T17:50:41.029376+0000","last_peered":"2024-09-13T17:50:41.029376+0000","last_clean":"2024-09-13T17:50:41.029376+0000","last_became_active":"2024-09-13T17:50:38.614165+0000","last_became_peered":"2024-09-13T17:50:38.614165+0000","last_unstale":"2024-09-13T17:50:41.029376+0000","last_undegraded":"2024-09-13T17:50:41.029376+0000","last_fullsized":"2024-09-13T17:50:41.029376+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:11.357026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278023+0000","last_change":"2024-09-13T17:50:34.583001+0000","last_active":"2024-09-13T17:51:15.278023+0000","last_peered":"2024-09-13T17:51:15.278023+0000","last_clean":"2024-09-13T17:51:15.278023+0000","last_became_active":"2024-09-13T17:50:34.582529+0000","last_became_peered":"2024-09-13T17:50:34.582529+0000","last_unstale":"2024-09-13T17:51:15.278023+0000","last_undegraded":"2024-09-13T17:51:15.278023+0000","last_fullsized":"2024-09-13T17:51:15.278023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:05.287181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:06.143515+0000","last_change":"2024-09-13T17:50:34.205138+0000","last_active":"2024-09-13T17:51:06.143515+0000","last_peered":"2024-09-13T17:51:06.143515+0000","last_clean":"2024-09-13T17:51:06.143515+0000","last_became_active":"2024-09-13T17:50:34.203360+0000","last_became_peered":"2024-09-13T17:50:34.203360+0000","last_unstale":"2024-09-13T17:51:06.143515+0000","last_undegraded":"2024-09-13T17:51:06.143515+0000","last_fullsized":"2024-09-13T17:51:06.143515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:35.756963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272372+0000","last_change":"2024-09-13T17:50:38.611036+0000","last_active":"2024-09-13T17:51:15.272372+0000","last_peered":"2024-09-13T17:51:15.272372+0000","last_clean":"2024-09-13T17:51:15.272372+0000","last_became_active":"2024-09-13T17:50:38.610692+0000","last_became_peered":"2024-09-13T17:50:38.610692+0000","last_unstale":"2024-09-13T17:51:15.272372+0000","last_undegraded":"2024-09-13T17:51:15.272372+0000","last_fullsized":"2024-09-13T17:51:15.272372+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:10.147998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.368023+0000","last_change":"2024-09-13T17:51:16.365295+0000","last_active":"2024-09-13T17:51:16.368023+0000","last_peered":"2024-09-13T17:51:16.368023+0000","last_clean":"2024-09-13T17:51:16.368023+0000","last_became_active":"2024-09-13T17:51:16.364437+0000","last_became_peered":"2024-09-13T17:51:16.364437+0000","last_unstale":"2024-09-13T17:51:16.368023+0000","last_undegraded":"2024-09-13T17:51:16.368023+0000","last_fullsized":"2024-09-13T17:51:16.368023+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:23.208899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:06.146374+0000","last_change":"2024-09-13T17:50:34.204376+0000","last_active":"2024-09-13T17:51:06.146374+0000","last_peered":"2024-09-13T17:51:06.146374+0000","last_clean":"2024-09-13T17:51:06.146374+0000","last_became_active":"2024-09-13T17:50:34.204003+0000","last_became_peered":"2024-09-13T17:50:34.204003+0000","last_unstale":"2024-09-13T17:51:06.146374+0000","last_undegraded":"2024-09-13T17:51:06.146374+0000","last_fullsized":"2024-09-13T17:51:06.146374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:14.023990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270270+0000","last_change":"2024-09-13T17:50:38.606144+0000","last_active":"2024-09-13T17:51:15.270270+0000","last_peered":"2024-09-13T17:51:15.270270+0000","last_clean":"2024-09-13T17:51:15.270270+0000","last_became_active":"2024-09-13T17:50:38.605646+0000","last_became_peered":"2024-09-13T17:50:38.605646+0000","last_unstale":"2024-09-13T17:51:15.270270+0000","last_undegraded":"2024-09-13T17:51:15.270270+0000","last_fullsized":"2024-09-13T17:51:15.270270+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:07.120075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277435+0000","last_change":"2024-09-13T17:50:38.618855+0000","last_active":"2024-09-13T17:51:15.277435+0000","last_peered":"2024-09-13T17:51:15.277435+0000","last_clean":"2024-09-13T17:51:15.277435+0000","last_became_active":"2024-09-13T17:50:38.618449+0000","last_became_peered":"2024-09-13T17:50:38.618449+0000","last_unstale":"2024-09-13T17:51:15.277435+0000","last_undegraded":"2024-09-13T17:51:15.277435+0000","last_fullsized":"2024-09-13T17:51:15.277435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:29:01.114087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.280511+0000","last_change":"2024-09-13T17:50:34.590079+0000","last_active":"2024-09-13T17:51:15.280511+0000","last_peered":"2024-09-13T17:51:15.280511+0000","last_clean":"2024-09-13T17:51:15.280511+0000","last_became_active":"2024-09-13T17:50:34.589396+0000","last_became_peered":"2024-09-13T17:50:34.589396+0000","last_unstale":"2024-09-13T17:51:15.280511+0000","last_undegraded":"2024-09-13T17:51:15.280511+0000","last_fullsized":"2024-09-13T17:51:15.280511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:03.308341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:06.143408+0000","last_change":"2024-09-13T17:50:34.206575+0000","last_active":"2024-09-13T17:51:06.143408+0000","last_peered":"2024-09-13T17:51:06.143408+0000","last_clean":"2024-09-13T17:51:06.143408+0000","last_became_active":"2024-09-13T17:50:34.206308+0000","last_became_peered":"2024-09-13T17:50:34.206308+0000","last_unstale":"2024-09-13T17:51:06.143408+0000","last_undegraded":"2024-09-13T17:51:06.143408+0000","last_fullsized":"2024-09-13T17:51:06.143408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:34.766538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.014256+0000","last_change":"2024-09-13T17:50:38.618131+0000","last_active":"2024-09-13T17:50:41.014256+0000","last_peered":"2024-09-13T17:50:41.014256+0000","last_clean":"2024-09-13T17:50:41.014256+0000","last_became_active":"2024-09-13T17:50:38.617611+0000","last_became_peered":"2024-09-13T17:50:38.617611+0000","last_unstale":"2024-09-13T17:50:41.014256+0000","last_undegraded":"2024-09-13T17:50:41.014256+0000","last_fullsized":"2024-09-13T17:50:41.014256+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:30.746445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272535+0000","last_change":"2024-09-13T17:50:34.582153+0000","last_active":"2024-09-13T17:51:15.272535+0000","last_peered":"2024-09-13T17:51:15.272535+0000","last_clean":"2024-09-13T17:51:15.272535+0000","last_became_active":"2024-09-13T17:50:34.581708+0000","last_became_peered":"2024-09-13T17:50:34.581708+0000","last_unstale":"2024-09-13T17:51:15.272535+0000","last_undegraded":"2024-09-13T17:51:15.272535+0000","last_fullsized":"2024-09-13T17:51:15.272535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:22.238810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270828+0000","last_change":"2024-09-13T17:50:34.205489+0000","last_active":"2024-09-13T17:51:15.270828+0000","last_peered":"2024-09-13T17:51:15.270828+0000","last_clean":"2024-09-13T17:51:15.270828+0000","last_became_active":"2024-09-13T17:50:34.205202+0000","last_became_peered":"2024-09-13T17:50:34.205202+0000","last_unstale":"2024-09-13T17:51:15.270828+0000","last_undegraded":"2024-09-13T17:51:15.270828+0000","last_fullsized":"2024-09-13T17:51:15.270828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:54.450276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:11.234223+0000","last_change":"2024-09-13T17:50:34.202470+0000","last_active":"2024-09-13T17:51:11.234223+0000","last_peered":"2024-09-13T17:51:11.234223+0000","last_clean":"2024-09-13T17:51:11.234223+0000","last_became_active":"2024-09-13T17:50:34.202163+0000","last_became_peered":"2024-09-13T17:50:34.202163+0000","last_unstale":"2024-09-13T17:51:11.234223+0000","last_undegraded":"2024-09-13T17:51:11.234223+0000","last_fullsized":"2024-09-13T17:51:11.234223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:03.472932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282749+0000","last_change":"2024-09-13T17:50:34.590947+0000","last_active":"2024-09-13T17:51:15.282749+0000","last_peered":"2024-09-13T17:51:15.282749+0000","last_clean":"2024-09-13T17:51:15.282749+0000","last_became_active":"2024-09-13T17:50:34.590497+0000","last_became_peered":"2024-09-13T17:50:34.590497+0000","last_unstale":"2024-09-13T17:51:15.282749+0000","last_undegraded":"2024-09-13T17:51:15.282749+0000","last_fullsized":"2024-09-13T17:51:15.282749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:56:27.848092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279328+0000","last_change":"2024-09-13T17:50:38.618518+0000","last_active":"2024-09-13T17:51:15.279328+0000","last_peered":"2024-09-13T17:51:15.279328+0000","last_clean":"2024-09-13T17:51:15.279328+0000","last_became_active":"2024-09-13T17:50:38.618108+0000","last_became_peered":"2024-09-13T17:50:38.618108+0000","last_unstale":"2024-09-13T17:51:15.279328+0000","last_undegraded":"2024-09-13T17:51:15.279328+0000","last_fullsized":"2024-09-13T17:51:15.279328+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:42.260237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271153+0000","last_change":"2024-09-13T17:50:34.218459+0000","last_active":"2024-09-13T17:51:15.271153+0000","last_peered":"2024-09-13T17:51:15.271153+0000","last_clean":"2024-09-13T17:51:15.271153+0000","last_became_active":"2024-09-13T17:50:34.218087+0000","last_became_peered":"2024-09-13T17:50:34.218087+0000","last_unstale":"2024-09-13T17:51:15.271153+0000","last_undegraded":"2024-09-13T17:51:15.271153+0000","last_fullsized":"2024-09-13T17:51:15.271153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:56.914544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501733+0000","last_change":"2024-09-13T17:50:34.583845+0000","last_active":"2024-09-13T17:50:39.501733+0000","last_peered":"2024-09-13T17:50:39.501733+0000","last_clean":"2024-09-13T17:50:39.501733+0000","last_became_active":"2024-09-13T17:50:34.583283+0000","last_became_peered":"2024-09-13T17:50:34.583283+0000","last_unstale":"2024-09-13T17:50:39.501733+0000","last_undegraded":"2024-09-13T17:50:39.501733+0000","last_fullsized":"2024-09-13T17:50:39.501733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:55.630285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271562+0000","last_change":"2024-09-13T17:50:38.611987+0000","last_active":"2024-09-13T17:51:15.271562+0000","last_peered":"2024-09-13T17:51:15.271562+0000","last_clean":"2024-09-13T17:51:15.271562+0000","last_became_active":"2024-09-13T17:50:38.611636+0000","last_became_peered":"2024-09-13T17:50:38.611636+0000","last_unstale":"2024-09-13T17:51:15.271562+0000","last_undegraded":"2024-09-13T17:51:15.271562+0000","last_fullsized":"2024-09-13T17:51:15.271562+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:41.111562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.981989+0000","last_change":"2024-09-13T17:50:34.588927+0000","last_active":"2024-09-13T17:50:38.981989+0000","last_peered":"2024-09-13T17:50:38.981989+0000","last_clean":"2024-09-13T17:50:38.981989+0000","last_became_active":"2024-09-13T17:50:34.588651+0000","last_became_peered":"2024-09-13T17:50:34.588651+0000","last_unstale":"2024-09-13T17:50:38.981989+0000","last_undegraded":"2024-09-13T17:50:38.981989+0000","last_fullsized":"2024-09-13T17:50:38.981989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:49.429670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278614+0000","last_change":"2024-09-13T17:50:34.583795+0000","last_active":"2024-09-13T17:51:15.278614+0000","last_peered":"2024-09-13T17:51:15.278614+0000","last_clean":"2024-09-13T17:51:15.278614+0000","last_became_active":"2024-09-13T17:50:34.583493+0000","last_became_peered":"2024-09-13T17:50:34.583493+0000","last_unstale":"2024-09-13T17:51:15.278614+0000","last_undegraded":"2024-09-13T17:51:15.278614+0000","last_fullsized":"2024-09-13T17:51:15.278614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:30.463932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.194789+0000","last_change":"2024-09-13T17:50:34.587624+0000","last_active":"2024-09-13T17:50:39.194789+0000","last_peered":"2024-09-13T17:50:39.194789+0000","last_clean":"2024-09-13T17:50:39.194789+0000","last_became_active":"2024-09-13T17:50:34.587119+0000","last_became_peered":"2024-09-13T17:50:34.587119+0000","last_unstale":"2024-09-13T17:50:39.194789+0000","last_undegraded":"2024-09-13T17:50:39.194789+0000","last_fullsized":"2024-09-13T17:50:39.194789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:16.783899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278177+0000","last_change":"2024-09-13T17:50:34.581923+0000","last_active":"2024-09-13T17:51:15.278177+0000","last_peered":"2024-09-13T17:51:15.278177+0000","last_clean":"2024-09-13T17:51:15.278177+0000","last_became_active":"2024-09-13T17:50:34.581628+0000","last_became_peered":"2024-09-13T17:50:34.581628+0000","last_unstale":"2024-09-13T17:51:15.278177+0000","last_undegraded":"2024-09-13T17:51:15.278177+0000","last_fullsized":"2024-09-13T17:51:15.278177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:28:55.936864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272460+0000","last_change":"2024-09-13T17:50:34.595037+0000","last_active":"2024-09-13T17:51:15.272460+0000","last_peered":"2024-09-13T17:51:15.272460+0000","last_clean":"2024-09-13T17:51:15.272460+0000","last_became_active":"2024-09-13T17:50:34.594857+0000","last_became_peered":"2024-09-13T17:50:34.594857+0000","last_unstale":"2024-09-13T17:51:15.272460+0000","last_undegraded":"2024-09-13T17:51:15.272460+0000","last_fullsized":"2024-09-13T17:51:15.272460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:11.420413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281006+0000","last_change":"2024-09-13T17:50:34.589651+0000","last_active":"2024-09-13T17:51:15.281006+0000","last_peered":"2024-09-13T17:51:15.281006+0000","last_clean":"2024-09-13T17:51:15.281006+0000","last_became_active":"2024-09-13T17:50:34.589354+0000","last_became_peered":"2024-09-13T17:50:34.589354+0000","last_unstale":"2024-09-13T17:51:15.281006+0000","last_undegraded":"2024-09-13T17:51:15.281006+0000","last_fullsized":"2024-09-13T17:51:15.281006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:31.294048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.983352+0000","last_change":"2024-09-13T17:50:34.597949+0000","last_active":"2024-09-13T17:50:38.983352+0000","last_peered":"2024-09-13T17:50:38.983352+0000","last_clean":"2024-09-13T17:50:38.983352+0000","last_became_active":"2024-09-13T17:50:34.597621+0000","last_became_peered":"2024-09-13T17:50:34.597621+0000","last_unstale":"2024-09-13T17:50:38.983352+0000","last_undegraded":"2024-09-13T17:50:38.983352+0000","last_fullsized":"2024-09-13T17:50:38.983352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:23.133462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272676+0000","last_change":"2024-09-13T17:50:34.589893+0000","last_active":"2024-09-13T17:51:15.272676+0000","last_peered":"2024-09-13T17:51:15.272676+0000","last_clean":"2024-09-13T17:51:15.272676+0000","last_became_active":"2024-09-13T17:50:34.589626+0000","last_became_peered":"2024-09-13T17:50:34.589626+0000","last_unstale":"2024-09-13T17:51:15.272676+0000","last_undegraded":"2024-09-13T17:51:15.272676+0000","last_fullsized":"2024-09-13T17:51:15.272676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:41.321025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.194245+0000","last_change":"2024-09-13T17:50:34.590046+0000","last_active":"2024-09-13T17:50:39.194245+0000","last_peered":"2024-09-13T17:50:39.194245+0000","last_clean":"2024-09-13T17:50:39.194245+0000","last_became_active":"2024-09-13T17:50:34.589757+0000","last_became_peered":"2024-09-13T17:50:34.589757+0000","last_unstale":"2024-09-13T17:50:39.194245+0000","last_undegraded":"2024-09-13T17:50:39.194245+0000","last_fullsized":"2024-09-13T17:50:39.194245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:28.239550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270567+0000","last_change":"2024-09-13T17:50:34.595518+0000","last_active":"2024-09-13T17:51:15.270567+0000","last_peered":"2024-09-13T17:51:15.270567+0000","last_clean":"2024-09-13T17:51:15.270567+0000","last_became_active":"2024-09-13T17:50:34.595242+0000","last_became_peered":"2024-09-13T17:50:34.595242+0000","last_unstale":"2024-09-13T17:51:15.270567+0000","last_undegraded":"2024-09-13T17:51:15.270567+0000","last_fullsized":"2024-09-13T17:51:15.270567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:26.149384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.194664+0000","last_change":"2024-09-13T17:50:34.593183+0000","last_active":"2024-09-13T17:50:39.194664+0000","last_peered":"2024-09-13T17:50:39.194664+0000","last_clean":"2024-09-13T17:50:39.194664+0000","last_became_active":"2024-09-13T17:50:34.592893+0000","last_became_peered":"2024-09-13T17:50:34.592893+0000","last_unstale":"2024-09-13T17:50:39.194664+0000","last_undegraded":"2024-09-13T17:50:39.194664+0000","last_fullsized":"2024-09-13T17:50:39.194664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:19.459505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278000+0000","last_change":"2024-09-13T17:50:34.594679+0000","last_active":"2024-09-13T17:51:15.278000+0000","last_peered":"2024-09-13T17:51:15.278000+0000","last_clean":"2024-09-13T17:51:15.278000+0000","last_became_active":"2024-09-13T17:50:34.593750+0000","last_became_peered":"2024-09-13T17:50:34.593750+0000","last_unstale":"2024-09-13T17:51:15.278000+0000","last_undegraded":"2024-09-13T17:51:15.278000+0000","last_fullsized":"2024-09-13T17:51:15.278000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:02.135577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.983701+0000","last_change":"2024-09-13T17:50:34.598736+0000","last_active":"2024-09-13T17:50:38.983701+0000","last_peered":"2024-09-13T17:50:38.983701+0000","last_clean":"2024-09-13T17:50:38.983701+0000","last_became_active":"2024-09-13T17:50:34.598433+0000","last_became_peered":"2024-09-13T17:50:34.598433+0000","last_unstale":"2024-09-13T17:50:38.983701+0000","last_undegraded":"2024-09-13T17:50:38.983701+0000","last_fullsized":"2024-09-13T17:50:38.983701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:08.975139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.985564+0000","last_change":"2024-09-13T17:50:34.591795+0000","last_active":"2024-09-13T17:50:38.985564+0000","last_peered":"2024-09-13T17:50:38.985564+0000","last_clean":"2024-09-13T17:50:38.985564+0000","last_became_active":"2024-09-13T17:50:34.591286+0000","last_became_peered":"2024-09-13T17:50:34.591286+0000","last_unstale":"2024-09-13T17:50:38.985564+0000","last_undegraded":"2024-09-13T17:50:38.985564+0000","last_fullsized":"2024-09-13T17:50:38.985564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:31.580247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279687+0000","last_change":"2024-09-13T17:50:34.587774+0000","last_active":"2024-09-13T17:51:15.279687+0000","last_peered":"2024-09-13T17:51:15.279687+0000","last_clean":"2024-09-13T17:51:15.279687+0000","last_became_active":"2024-09-13T17:50:34.587506+0000","last_became_peered":"2024-09-13T17:50:34.587506+0000","last_unstale":"2024-09-13T17:51:15.279687+0000","last_undegraded":"2024-09-13T17:51:15.279687+0000","last_fullsized":"2024-09-13T17:51:15.279687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:46.275528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.195748+0000","last_change":"2024-09-13T17:50:34.596100+0000","last_active":"2024-09-13T17:50:39.195748+0000","last_peered":"2024-09-13T17:50:39.195748+0000","last_clean":"2024-09-13T17:50:39.195748+0000","last_became_active":"2024-09-13T17:50:34.595798+0000","last_became_peered":"2024-09-13T17:50:34.595798+0000","last_unstale":"2024-09-13T17:50:39.195748+0000","last_undegraded":"2024-09-13T17:50:39.195748+0000","last_fullsized":"2024-09-13T17:50:39.195748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:49.543845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.194125+0000","last_change":"2024-09-13T17:50:34.584818+0000","last_active":"2024-09-13T17:50:39.194125+0000","last_peered":"2024-09-13T17:50:39.194125+0000","last_clean":"2024-09-13T17:50:39.194125+0000","last_became_active":"2024-09-13T17:50:34.584516+0000","last_became_peered":"2024-09-13T17:50:34.584516+0000","last_unstale":"2024-09-13T17:50:39.194125+0000","last_undegraded":"2024-09-13T17:50:39.194125+0000","last_fullsized":"2024-09-13T17:50:39.194125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:52.402024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281649+0000","last_change":"2024-09-13T17:50:34.598886+0000","last_active":"2024-09-13T17:51:15.281649+0000","last_peered":"2024-09-13T17:51:15.281649+0000","last_clean":"2024-09-13T17:51:15.281649+0000","last_became_active":"2024-09-13T17:50:34.598648+0000","last_became_peered":"2024-09-13T17:50:34.598648+0000","last_unstale":"2024-09-13T17:51:15.281649+0000","last_undegraded":"2024-09-13T17:51:15.281649+0000","last_fullsized":"2024-09-13T17:51:15.281649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:00:43.063223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282938+0000","last_change":"2024-09-13T17:50:34.582649+0000","last_active":"2024-09-13T17:51:15.282938+0000","last_peered":"2024-09-13T17:51:15.282938+0000","last_clean":"2024-09-13T17:51:15.282938+0000","last_became_active":"2024-09-13T17:50:34.581699+0000","last_became_peered":"2024-09-13T17:50:34.581699+0000","last_unstale":"2024-09-13T17:51:15.282938+0000","last_undegraded":"2024-09-13T17:51:15.282938+0000","last_fullsized":"2024-09-13T17:51:15.282938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:18.959303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270646+0000","last_change":"2024-09-13T17:50:34.587484+0000","last_active":"2024-09-13T17:51:15.270646+0000","last_peered":"2024-09-13T17:51:15.270646+0000","last_clean":"2024-09-13T17:51:15.270646+0000","last_became_active":"2024-09-13T17:50:34.587217+0000","last_became_peered":"2024-09-13T17:50:34.587217+0000","last_unstale":"2024-09-13T17:51:15.270646+0000","last_undegraded":"2024-09-13T17:51:15.270646+0000","last_fullsized":"2024-09-13T17:51:15.270646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:24:19.172607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.502178+0000","last_change":"2024-09-13T17:50:34.592871+0000","last_active":"2024-09-13T17:50:39.502178+0000","last_peered":"2024-09-13T17:50:39.502178+0000","last_clean":"2024-09-13T17:50:39.502178+0000","last_became_active":"2024-09-13T17:50:34.588513+0000","last_became_peered":"2024-09-13T17:50:34.588513+0000","last_unstale":"2024-09-13T17:50:39.502178+0000","last_undegraded":"2024-09-13T17:50:39.502178+0000","last_fullsized":"2024-09-13T17:50:39.502178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:33.131420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.983592+0000","last_change":"2024-09-13T17:50:34.585713+0000","last_active":"2024-09-13T17:50:38.983592+0000","last_peered":"2024-09-13T17:50:38.983592+0000","last_clean":"2024-09-13T17:50:38.983592+0000","last_became_active":"2024-09-13T17:50:34.585438+0000","last_became_peered":"2024-09-13T17:50:34.585438+0000","last_unstale":"2024-09-13T17:50:38.983592+0000","last_undegraded":"2024-09-13T17:50:38.983592+0000","last_fullsized":"2024-09-13T17:50:38.983592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:56:49.905393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.983306+0000","last_change":"2024-09-13T17:50:34.589628+0000","last_active":"2024-09-13T17:50:38.983306+0000","last_peered":"2024-09-13T17:50:38.983306+0000","last_clean":"2024-09-13T17:50:38.983306+0000","last_became_active":"2024-09-13T17:50:34.587665+0000","last_became_peered":"2024-09-13T17:50:34.587665+0000","last_unstale":"2024-09-13T17:50:38.983306+0000","last_undegraded":"2024-09-13T17:50:38.983306+0000","last_fullsized":"2024-09-13T17:50:38.983306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:15.518264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.985649+0000","last_change":"2024-09-13T17:50:34.591776+0000","last_active":"2024-09-13T17:50:38.985649+0000","last_peered":"2024-09-13T17:50:38.985649+0000","last_clean":"2024-09-13T17:50:38.985649+0000","last_became_active":"2024-09-13T17:50:34.591496+0000","last_became_peered":"2024-09-13T17:50:34.591496+0000","last_unstale":"2024-09-13T17:50:38.985649+0000","last_undegraded":"2024-09-13T17:50:38.985649+0000","last_fullsized":"2024-09-13T17:50:38.985649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:48.651999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.283047+0000","last_change":"2024-09-13T17:51:16.282372+0000","last_active":"2024-09-13T17:51:16.283047+0000","last_peered":"2024-09-13T17:51:16.283047+0000","last_clean":"2024-09-13T17:51:16.283047+0000","last_became_active":"2024-09-13T17:51:16.282065+0000","last_became_peered":"2024-09-13T17:51:16.282065+0000","last_unstale":"2024-09-13T17:51:16.283047+0000","last_undegraded":"2024-09-13T17:51:16.283047+0000","last_fullsized":"2024-09-13T17:51:16.283047+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:49.291224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.984603+0000","last_change":"2024-09-13T17:50:34.585531+0000","last_active":"2024-09-13T17:50:38.984603+0000","last_peered":"2024-09-13T17:50:38.984603+0000","last_clean":"2024-09-13T17:50:38.984603+0000","last_became_active":"2024-09-13T17:50:34.585226+0000","last_became_peered":"2024-09-13T17:50:34.585226+0000","last_unstale":"2024-09-13T17:50:38.984603+0000","last_undegraded":"2024-09-13T17:50:38.984603+0000","last_fullsized":"2024-09-13T17:50:38.984603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:17.105028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.194517+0000","last_change":"2024-09-13T17:50:34.586750+0000","last_active":"2024-09-13T17:50:39.194517+0000","last_peered":"2024-09-13T17:50:39.194517+0000","last_clean":"2024-09-13T17:50:39.194517+0000","last_became_active":"2024-09-13T17:50:34.584564+0000","last_became_peered":"2024-09-13T17:50:34.584564+0000","last_unstale":"2024-09-13T17:50:39.194517+0000","last_undegraded":"2024-09-13T17:50:39.194517+0000","last_fullsized":"2024-09-13T17:50:39.194517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:54.033257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.984499+0000","last_change":"2024-09-13T17:50:34.585938+0000","last_active":"2024-09-13T17:50:38.984499+0000","last_peered":"2024-09-13T17:50:38.984499+0000","last_clean":"2024-09-13T17:50:38.984499+0000","last_became_active":"2024-09-13T17:50:34.585584+0000","last_became_peered":"2024-09-13T17:50:34.585584+0000","last_unstale":"2024-09-13T17:50:38.984499+0000","last_undegraded":"2024-09-13T17:50:38.984499+0000","last_fullsized":"2024-09-13T17:50:38.984499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:32.623468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.194917+0000","last_change":"2024-09-13T17:50:34.588762+0000","last_active":"2024-09-13T17:50:39.194917+0000","last_peered":"2024-09-13T17:50:39.194917+0000","last_clean":"2024-09-13T17:50:39.194917+0000","last_became_active":"2024-09-13T17:50:34.588489+0000","last_became_peered":"2024-09-13T17:50:34.588489+0000","last_unstale":"2024-09-13T17:50:39.194917+0000","last_undegraded":"2024-09-13T17:50:39.194917+0000","last_fullsized":"2024-09-13T17:50:39.194917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:38.266297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277059+0000","last_change":"2024-09-13T17:50:34.583889+0000","last_active":"2024-09-13T17:51:15.277059+0000","last_peered":"2024-09-13T17:51:15.277059+0000","last_clean":"2024-09-13T17:51:15.277059+0000","last_became_active":"2024-09-13T17:50:34.583551+0000","last_became_peered":"2024-09-13T17:50:34.583551+0000","last_unstale":"2024-09-13T17:51:15.277059+0000","last_undegraded":"2024-09-13T17:51:15.277059+0000","last_fullsized":"2024-09-13T17:51:15.277059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:19.888626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281343+0000","last_change":"2024-09-13T17:50:34.585617+0000","last_active":"2024-09-13T17:51:15.281343+0000","last_peered":"2024-09-13T17:51:15.281343+0000","last_clean":"2024-09-13T17:51:15.281343+0000","last_became_active":"2024-09-13T17:50:34.585340+0000","last_became_peered":"2024-09-13T17:50:34.585340+0000","last_unstale":"2024-09-13T17:51:15.281343+0000","last_undegraded":"2024-09-13T17:51:15.281343+0000","last_fullsized":"2024-09-13T17:51:15.281343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:18.331045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.985639+0000","last_change":"2024-09-13T17:50:34.592365+0000","last_active":"2024-09-13T17:50:38.985639+0000","last_peered":"2024-09-13T17:50:38.985639+0000","last_clean":"2024-09-13T17:50:38.985639+0000","last_became_active":"2024-09-13T17:50:34.592046+0000","last_became_peered":"2024-09-13T17:50:34.592046+0000","last_unstale":"2024-09-13T17:50:38.985639+0000","last_undegraded":"2024-09-13T17:50:38.985639+0000","last_fullsized":"2024-09-13T17:50:38.985639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:54.283683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7309,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":14618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509454,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":113480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4870000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]}]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":303104,"data_stored":116742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]}]},{"osd":1,"up_from":15,"seq":64424509454,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":109965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]}]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":892928,"data_stored":706952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5790000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4660000000000002}]}]},{"osd":7,"up_from":14,"seq":60129542157,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":872448,"data_stored":693591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]}]},{"osd":6,"up_from":14,"seq":60129542157,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":278528,"data_stored":106380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]}]},{"osd":5,"up_from":14,"seq":60129542157,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":290816,"data_stored":110399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542157,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":286720,"data_stored":109687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:51:22.670 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:51:22.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a873e9700 1 Processor -- start 2024-09-13T17:51:22.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a873e9700 1 -- start start 2024-09-13T17:51:22.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a873e9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7a801512c0 0x7f7a80157b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:22.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a873e9700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7a800b2b70 0x7f7a8006f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:22.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a873e9700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a80070390 con 0x7f7a8006fee0 2024-09-13T17:51:22.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a873e9700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a8006e780 con 0x7f7a800b2b70 2024-09-13T17:51:22.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a873e9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a800c3a80 con 0x7f7a801512c0 2024-09-13T17:51:22.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a86be8700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7a8006fee0 0x7f7a800c1f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44520/0 (socket says 172.21.15.40:44520) 2024-09-13T17:51:22.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a86be8700 1 -- 172.21.15.40:0/2207875269 learned_addr learned my addr 172.21.15.40:0/2207875269 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:22.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a85be6700 1 --2- 172.21.15.40:0/2207875269 >> v2:172.21.15.43:3300/0 conn(0x7f7a800b2b70 0x7f7a8006f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:22.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a863e7700 1 --2- 172.21.15.40:0/2207875269 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7a801512c0 0x7f7a80157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:22.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a853e5700 1 -- 172.21.15.40:0/2207875269 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3847438714 0 0) 0x7f7a80070390 con 0x7f7a8006fee0 2024-09-13T17:51:22.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a853e5700 1 -- 172.21.15.40:0/2207875269 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a680035d0 con 0x7f7a8006fee0 2024-09-13T17:51:22.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a863e7700 1 -- 172.21.15.40:0/2207875269 >> v2:172.21.15.43:3300/0 conn(0x7f7a800b2b70 msgr2=0x7f7a8006f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:22.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a863e7700 1 --2- 172.21.15.40:0/2207875269 >> v2:172.21.15.43:3300/0 conn(0x7f7a800b2b70 0x7f7a8006f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a863e7700 1 -- 172.21.15.40:0/2207875269 >> v1:172.21.15.40:6789/0 conn(0x7f7a8006fee0 legacy=0x7f7a800c1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:22.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a863e7700 1 -- 172.21.15.40:0/2207875269 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a80158070 con 0x7f7a801512c0 2024-09-13T17:51:22.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a863e7700 1 --2- 172.21.15.40:0/2207875269 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7a801512c0 0x7f7a80157b30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009850 tx=0x7f7a74009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30a70fe40d6cf91c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:22.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a853e5700 1 -- 172.21.15.40:0/2207875269 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74011830 con 0x7f7a801512c0 2024-09-13T17:51:22.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a853e5700 1 -- 172.21.15.40:0/2207875269 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a74011990 con 0x7f7a801512c0 2024-09-13T17:51:22.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.778+0000 7f7a853e5700 1 -- 172.21.15.40:0/2207875269 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a74011d00 con 0x7f7a801512c0 2024-09-13T17:51:22.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.782+0000 7f7a873e9700 1 -- 172.21.15.40:0/2207875269 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7a801512c0 msgr2=0x7f7a80157b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:22.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.782+0000 7f7a873e9700 1 --2- 172.21.15.40:0/2207875269 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7a801512c0 0x7f7a80157b30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7a74009850 tx=0x7f7a74009b60 comp rx=0 tx=0).stop 2024-09-13T17:51:22.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.782+0000 7f7a873e9700 1 -- 172.21.15.40:0/2207875269 shutdown_connections 2024-09-13T17:51:22.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.782+0000 7f7a873e9700 1 --2- 172.21.15.40:0/2207875269 >> v2:172.21.15.43:3300/0 conn(0x7f7a800b2b70 0x7f7a8006f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.782+0000 7f7a873e9700 1 --2- 172.21.15.40:0/2207875269 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7a801512c0 0x7f7a80157b30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.782+0000 7f7a873e9700 1 -- 172.21.15.40:0/2207875269 >> 172.21.15.40:0/2207875269 conn(0x7f7a80066090 msgr2=0x7f7a80066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:22.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.782+0000 7f7a873e9700 1 -- 172.21.15.40:0/2207875269 shutdown_connections 2024-09-13T17:51:22.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.782+0000 7f7a873e9700 1 -- 172.21.15.40:0/2207875269 wait complete. 2024-09-13T17:51:22.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.782+0000 7f7a873e9700 1 Processor -- start 2024-09-13T17:51:22.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.782+0000 7f7a873e9700 1 -- start start 2024-09-13T17:51:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a873e9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a8006fee0 0x7f7a80154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a873e9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800b2b70 0x7f7a80151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a873e9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7a801512c0 0x7f7a801eda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a873e9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a800c6ae0 con 0x7f7a800b2b70 2024-09-13T17:51:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a873e9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a801534b0 con 0x7f7a8006fee0 2024-09-13T17:51:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a873e9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a800c6d30 con 0x7f7a801512c0 2024-09-13T17:51:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a85be6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800b2b70 0x7f7a80151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a85be6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800b2b70 0x7f7a80151a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55096/0 (socket says 172.21.15.40:55096) 2024-09-13T17:51:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a85be6700 1 -- 172.21.15.40:0/3099114049 learned_addr learned my addr 172.21.15.40:0/3099114049 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a863e7700 1 --2- 172.21.15.40:0/3099114049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a8006fee0 0x7f7a80154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a86be8700 1 --2- 172.21.15.40:0/3099114049 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7a801512c0 0x7f7a801eda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a85be6700 1 -- 172.21.15.40:0/3099114049 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7a801512c0 msgr2=0x7f7a801eda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a85be6700 1 --2- 172.21.15.40:0/3099114049 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7a801512c0 0x7f7a801eda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a85be6700 1 -- 172.21.15.40:0/3099114049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a8006fee0 msgr2=0x7f7a80154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a85be6700 1 --2- 172.21.15.40:0/3099114049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a8006fee0 0x7f7a80154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a85be6700 1 -- 172.21.15.40:0/3099114049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a740094b0 con 0x7f7a800b2b70 2024-09-13T17:51:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a85be6700 1 --2- 172.21.15.40:0/3099114049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800b2b70 0x7f7a80151a90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800ede0 tx=0x7f7a7800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a7effd700 1 -- 172.21.15.40:0/3099114049 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7800cb00 con 0x7f7a800b2b70 2024-09-13T17:51:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a7effd700 1 -- 172.21.15.40:0/3099114049 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7800f040 con 0x7f7a800b2b70 2024-09-13T17:51:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a873e9700 1 -- 172.21.15.40:0/3099114049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a800c7010 con 0x7f7a800b2b70 2024-09-13T17:51:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a7effd700 1 -- 172.21.15.40:0/3099114049 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7800ce80 con 0x7f7a800b2b70 2024-09-13T17:51:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.786+0000 7f7a873e9700 1 -- 172.21.15.40:0/3099114049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a80153140 con 0x7f7a800b2b70 2024-09-13T17:51:22.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.790+0000 7f7a873e9700 1 -- 172.21.15.40:0/3099114049 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a80152c90 con 0x7f7a800b2b70 2024-09-13T17:51:22.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.790+0000 7f7a7effd700 1 -- 172.21.15.40:0/3099114049 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7a78014480 con 0x7f7a800b2b70 2024-09-13T17:51:22.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.790+0000 7f7a7effd700 1 --2- 172.21.15.40:0/3099114049 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7a6c066760 0x7f7a6c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:22.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.790+0000 7f7a7effd700 1 -- 172.21.15.40:0/3099114049 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7a78084ff0 con 0x7f7a800b2b70 2024-09-13T17:51:22.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.790+0000 7f7a863e7700 1 --2- 172.21.15.40:0/3099114049 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7a6c066760 0x7f7a6c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:22.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.790+0000 7f7a863e7700 1 --2- 172.21.15.40:0/3099114049 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7a6c066760 0x7f7a6c068c20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7a740090b0 tx=0x7f7a74008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:22.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.798+0000 7f7a7effd700 1 -- 172.21.15.40:0/3099114049 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7a780b6db0 con 0x7f7a800b2b70 2024-09-13T17:51:22.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.966+0000 7f7a873e9700 1 -- 172.21.15.40:0/3099114049 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7a8006e4a0 con 0x7f7a6c066760 2024-09-13T17:51:22.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.986+0000 7f7a7effd700 1 -- 172.21.15.40:0/3099114049 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345231 (secure 0 0 0) 0x7f7a74011630 con 0x7f7a6c066760 2024-09-13T17:51:22.994 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:22.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.990+0000 7f7a873e9700 1 -- 172.21.15.40:0/3099114049 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7a6c066760 msgr2=0x7f7a6c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:22.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.990+0000 7f7a873e9700 1 --2- 172.21.15.40:0/3099114049 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7a6c066760 0x7f7a6c068c20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7a740090b0 tx=0x7f7a74008030 comp rx=0 tx=0).stop 2024-09-13T17:51:22.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.990+0000 7f7a873e9700 1 -- 172.21.15.40:0/3099114049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800b2b70 msgr2=0x7f7a80151a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:22.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.990+0000 7f7a873e9700 1 --2- 172.21.15.40:0/3099114049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800b2b70 0x7f7a80151a90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800ede0 tx=0x7f7a7800c3a0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.990+0000 7f7a873e9700 1 -- 172.21.15.40:0/3099114049 shutdown_connections 2024-09-13T17:51:22.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.990+0000 7f7a873e9700 1 --2- 172.21.15.40:0/3099114049 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7a6c066760 0x7f7a6c068c20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.990+0000 7f7a873e9700 1 --2- 172.21.15.40:0/3099114049 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7a801512c0 0x7f7a801eda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.990+0000 7f7a873e9700 1 --2- 172.21.15.40:0/3099114049 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a800b2b70 0x7f7a80151a90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.990+0000 7f7a873e9700 1 --2- 172.21.15.40:0/3099114049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a8006fee0 0x7f7a80154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:22.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.990+0000 7f7a873e9700 1 -- 172.21.15.40:0/3099114049 >> 172.21.15.40:0/3099114049 conn(0x7f7a80066090 msgr2=0x7f7a8014ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:22.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.990+0000 7f7a873e9700 1 -- 172.21.15.40:0/3099114049 shutdown_connections 2024-09-13T17:51:22.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:22.990+0000 7f7a873e9700 1 -- 172.21.15.40:0/3099114049 wait complete. 2024-09-13T17:51:22.998 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-13T17:51:23.012 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2024-09-13T17:51:21.013440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597677,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":286,"num_write_kb":2458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":327,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235968,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738752,"statfs":{"total":773094113280,"available":772852482048,"internally_reserved":0,"allocated":3514368,"data_stored":2067196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":574,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003462"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282601+0000","last_change":"2024-09-13T17:50:34.212158+0000","last_active":"2024-09-13T17:51:15.282601+0000","last_peered":"2024-09-13T17:51:15.282601+0000","last_clean":"2024-09-13T17:51:15.282601+0000","last_became_active":"2024-09-13T17:50:34.211867+0000","last_became_peered":"2024-09-13T17:50:34.211867+0000","last_unstale":"2024-09-13T17:51:15.282601+0000","last_undegraded":"2024-09-13T17:51:15.282601+0000","last_fullsized":"2024-09-13T17:51:15.282601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:14.671095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271433+0000","last_change":"2024-09-13T17:50:34.587396+0000","last_active":"2024-09-13T17:51:15.271433+0000","last_peered":"2024-09-13T17:51:15.271433+0000","last_clean":"2024-09-13T17:51:15.271433+0000","last_became_active":"2024-09-13T17:50:34.587108+0000","last_became_peered":"2024-09-13T17:50:34.587108+0000","last_unstale":"2024-09-13T17:51:15.271433+0000","last_undegraded":"2024-09-13T17:51:15.271433+0000","last_fullsized":"2024-09-13T17:51:15.271433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:39:16.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270087+0000","last_change":"2024-09-13T17:50:38.615059+0000","last_active":"2024-09-13T17:51:15.270087+0000","last_peered":"2024-09-13T17:51:15.270087+0000","last_clean":"2024-09-13T17:51:15.270087+0000","last_became_active":"2024-09-13T17:50:38.614723+0000","last_became_peered":"2024-09-13T17:50:38.614723+0000","last_unstale":"2024-09-13T17:51:15.270087+0000","last_undegraded":"2024-09-13T17:51:15.270087+0000","last_fullsized":"2024-09-13T17:51:15.270087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:23:01.902181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271688+0000","last_change":"2024-09-13T17:50:34.204760+0000","last_active":"2024-09-13T17:51:15.271688+0000","last_peered":"2024-09-13T17:51:15.271688+0000","last_clean":"2024-09-13T17:51:15.271688+0000","last_became_active":"2024-09-13T17:50:34.202930+0000","last_became_peered":"2024-09-13T17:50:34.202930+0000","last_unstale":"2024-09-13T17:51:15.271688+0000","last_undegraded":"2024-09-13T17:51:15.271688+0000","last_fullsized":"2024-09-13T17:51:15.271688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:59.784701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.366352+0000","last_change":"2024-09-13T17:51:16.366352+0000","last_active":"2024-09-13T17:51:16.366352+0000","last_peered":"2024-09-13T17:51:16.366352+0000","last_clean":"2024-09-13T17:51:16.366352+0000","last_became_active":"2024-09-13T17:51:16.365492+0000","last_became_peered":"2024-09-13T17:51:16.365492+0000","last_unstale":"2024-09-13T17:51:16.366352+0000","last_undegraded":"2024-09-13T17:51:16.366352+0000","last_fullsized":"2024-09-13T17:51:16.366352+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:42.192182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.029188+0000","last_change":"2024-09-13T17:50:38.618751+0000","last_active":"2024-09-13T17:50:41.029188+0000","last_peered":"2024-09-13T17:50:41.029188+0000","last_clean":"2024-09-13T17:50:41.029188+0000","last_became_active":"2024-09-13T17:50:38.618423+0000","last_became_peered":"2024-09-13T17:50:38.618423+0000","last_unstale":"2024-09-13T17:50:41.029188+0000","last_undegraded":"2024-09-13T17:50:41.029188+0000","last_fullsized":"2024-09-13T17:50:41.029188+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:50:02.418785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"25'4","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:11.230982+0000","last_change":"2024-09-13T17:50:34.214972+0000","last_active":"2024-09-13T17:51:11.230982+0000","last_peered":"2024-09-13T17:51:11.230982+0000","last_clean":"2024-09-13T17:51:11.230982+0000","last_became_active":"2024-09-13T17:50:34.214652+0000","last_became_peered":"2024-09-13T17:50:34.214652+0000","last_unstale":"2024-09-13T17:51:11.230982+0000","last_undegraded":"2024-09-13T17:51:11.230982+0000","last_fullsized":"2024-09-13T17:51:11.230982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:57.176854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.282672+0000","last_change":"2024-09-13T17:51:16.281831+0000","last_active":"2024-09-13T17:51:16.282672+0000","last_peered":"2024-09-13T17:51:16.282672+0000","last_clean":"2024-09-13T17:51:16.282672+0000","last_became_active":"2024-09-13T17:51:16.281549+0000","last_became_peered":"2024-09-13T17:51:16.281549+0000","last_unstale":"2024-09-13T17:51:16.282672+0000","last_undegraded":"2024-09-13T17:51:16.282672+0000","last_fullsized":"2024-09-13T17:51:16.282672+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:43.894437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.029352+0000","last_change":"2024-09-13T17:50:38.616219+0000","last_active":"2024-09-13T17:50:41.029352+0000","last_peered":"2024-09-13T17:50:41.029352+0000","last_clean":"2024-09-13T17:50:41.029352+0000","last_became_active":"2024-09-13T17:50:38.614329+0000","last_became_peered":"2024-09-13T17:50:38.614329+0000","last_unstale":"2024-09-13T17:50:41.029352+0000","last_undegraded":"2024-09-13T17:50:41.029352+0000","last_fullsized":"2024-09-13T17:50:41.029352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:25.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"25'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.583439+0000","last_change":"2024-09-13T17:50:34.216123+0000","last_active":"2024-09-13T17:50:39.583439+0000","last_peered":"2024-09-13T17:50:39.583439+0000","last_clean":"2024-09-13T17:50:39.583439+0000","last_became_active":"2024-09-13T17:50:34.215661+0000","last_became_peered":"2024-09-13T17:50:34.215661+0000","last_unstale":"2024-09-13T17:50:39.583439+0000","last_undegraded":"2024-09-13T17:50:39.583439+0000","last_fullsized":"2024-09-13T17:50:39.583439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:20.895037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.195197+0000","last_change":"2024-09-13T17:50:34.599673+0000","last_active":"2024-09-13T17:50:39.195197+0000","last_peered":"2024-09-13T17:50:39.195197+0000","last_clean":"2024-09-13T17:50:39.195197+0000","last_became_active":"2024-09-13T17:50:34.599409+0000","last_became_peered":"2024-09-13T17:50:34.599409+0000","last_unstale":"2024-09-13T17:50:39.195197+0000","last_undegraded":"2024-09-13T17:50:39.195197+0000","last_fullsized":"2024-09-13T17:50:39.195197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:03:31.394385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.029311+0000","last_change":"2024-09-13T17:50:38.616317+0000","last_active":"2024-09-13T17:50:41.029311+0000","last_peered":"2024-09-13T17:50:41.029311+0000","last_clean":"2024-09-13T17:50:41.029311+0000","last_became_active":"2024-09-13T17:50:38.614311+0000","last_became_peered":"2024-09-13T17:50:38.614311+0000","last_unstale":"2024-09-13T17:50:41.029311+0000","last_undegraded":"2024-09-13T17:50:41.029311+0000","last_fullsized":"2024-09-13T17:50:41.029311+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:29.221522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.982805+0000","last_change":"2024-09-13T17:50:34.599490+0000","last_active":"2024-09-13T17:50:38.982805+0000","last_peered":"2024-09-13T17:50:38.982805+0000","last_clean":"2024-09-13T17:50:38.982805+0000","last_became_active":"2024-09-13T17:50:34.599207+0000","last_became_peered":"2024-09-13T17:50:34.599207+0000","last_unstale":"2024-09-13T17:50:38.982805+0000","last_undegraded":"2024-09-13T17:50:38.982805+0000","last_fullsized":"2024-09-13T17:50:38.982805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:11.178161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:06.142751+0000","last_change":"2024-09-13T17:50:34.215697+0000","last_active":"2024-09-13T17:51:06.142751+0000","last_peered":"2024-09-13T17:51:06.142751+0000","last_clean":"2024-09-13T17:51:06.142751+0000","last_became_active":"2024-09-13T17:50:34.215409+0000","last_became_peered":"2024-09-13T17:50:34.215409+0000","last_unstale":"2024-09-13T17:51:06.142751+0000","last_undegraded":"2024-09-13T17:51:06.142751+0000","last_fullsized":"2024-09-13T17:51:06.142751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:21.609567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.029485+0000","last_change":"2024-09-13T17:50:38.611817+0000","last_active":"2024-09-13T17:50:41.029485+0000","last_peered":"2024-09-13T17:50:41.029485+0000","last_clean":"2024-09-13T17:50:41.029485+0000","last_became_active":"2024-09-13T17:50:38.611263+0000","last_became_peered":"2024-09-13T17:50:38.611263+0000","last_unstale":"2024-09-13T17:50:41.029485+0000","last_undegraded":"2024-09-13T17:50:41.029485+0000","last_fullsized":"2024-09-13T17:50:41.029485+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:56:47.466672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271037+0000","last_change":"2024-09-13T17:50:34.586307+0000","last_active":"2024-09-13T17:51:15.271037+0000","last_peered":"2024-09-13T17:51:15.271037+0000","last_clean":"2024-09-13T17:51:15.271037+0000","last_became_active":"2024-09-13T17:50:34.585831+0000","last_became_peered":"2024-09-13T17:50:34.585831+0000","last_unstale":"2024-09-13T17:51:15.271037+0000","last_undegraded":"2024-09-13T17:51:15.271037+0000","last_fullsized":"2024-09-13T17:51:15.271037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:40.912352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.280577+0000","last_change":"2024-09-13T17:50:34.211850+0000","last_active":"2024-09-13T17:51:15.280577+0000","last_peered":"2024-09-13T17:51:15.280577+0000","last_clean":"2024-09-13T17:51:15.280577+0000","last_became_active":"2024-09-13T17:50:34.211594+0000","last_became_peered":"2024-09-13T17:50:34.211594+0000","last_unstale":"2024-09-13T17:51:15.280577+0000","last_undegraded":"2024-09-13T17:51:15.280577+0000","last_fullsized":"2024-09-13T17:51:15.280577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:35:02.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.029420+0000","last_change":"2024-09-13T17:50:38.613576+0000","last_active":"2024-09-13T17:50:41.029420+0000","last_peered":"2024-09-13T17:50:41.029420+0000","last_clean":"2024-09-13T17:50:41.029420+0000","last_became_active":"2024-09-13T17:50:38.612577+0000","last_became_peered":"2024-09-13T17:50:38.612577+0000","last_unstale":"2024-09-13T17:50:41.029420+0000","last_undegraded":"2024-09-13T17:50:41.029420+0000","last_fullsized":"2024-09-13T17:50:41.029420+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:25.551060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272679+0000","last_change":"2024-09-13T17:50:34.587273+0000","last_active":"2024-09-13T17:51:15.272679+0000","last_peered":"2024-09-13T17:51:15.272679+0000","last_clean":"2024-09-13T17:51:15.272679+0000","last_became_active":"2024-09-13T17:50:34.587027+0000","last_became_peered":"2024-09-13T17:50:34.587027+0000","last_unstale":"2024-09-13T17:51:15.272679+0000","last_undegraded":"2024-09-13T17:51:15.272679+0000","last_fullsized":"2024-09-13T17:51:15.272679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:58.826737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.280692+0000","last_change":"2024-09-13T17:50:34.211845+0000","last_active":"2024-09-13T17:51:15.280692+0000","last_peered":"2024-09-13T17:51:15.280692+0000","last_clean":"2024-09-13T17:51:15.280692+0000","last_became_active":"2024-09-13T17:50:34.211568+0000","last_became_peered":"2024-09-13T17:50:34.211568+0000","last_unstale":"2024-09-13T17:51:15.280692+0000","last_undegraded":"2024-09-13T17:51:15.280692+0000","last_fullsized":"2024-09-13T17:51:15.280692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:56.348984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281699+0000","last_change":"2024-09-13T17:50:38.610292+0000","last_active":"2024-09-13T17:51:15.281699+0000","last_peered":"2024-09-13T17:51:15.281699+0000","last_clean":"2024-09-13T17:51:15.281699+0000","last_became_active":"2024-09-13T17:50:38.609832+0000","last_became_peered":"2024-09-13T17:50:38.609832+0000","last_unstale":"2024-09-13T17:51:15.281699+0000","last_undegraded":"2024-09-13T17:51:15.281699+0000","last_fullsized":"2024-09-13T17:51:15.281699+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:13.783068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501657+0000","last_change":"2024-09-13T17:50:34.593566+0000","last_active":"2024-09-13T17:50:39.501657+0000","last_peered":"2024-09-13T17:50:39.501657+0000","last_clean":"2024-09-13T17:50:39.501657+0000","last_became_active":"2024-09-13T17:50:34.588794+0000","last_became_peered":"2024-09-13T17:50:34.588794+0000","last_unstale":"2024-09-13T17:50:39.501657+0000","last_undegraded":"2024-09-13T17:50:39.501657+0000","last_fullsized":"2024-09-13T17:50:39.501657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:13.192453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:06.142903+0000","last_change":"2024-09-13T17:50:34.214638+0000","last_active":"2024-09-13T17:51:06.142903+0000","last_peered":"2024-09-13T17:51:06.142903+0000","last_clean":"2024-09-13T17:51:06.142903+0000","last_became_active":"2024-09-13T17:50:34.214292+0000","last_became_peered":"2024-09-13T17:50:34.214292+0000","last_unstale":"2024-09-13T17:51:06.142903+0000","last_undegraded":"2024-09-13T17:51:06.142903+0000","last_fullsized":"2024-09-13T17:51:06.142903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:57.377104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.029513+0000","last_change":"2024-09-13T17:50:38.613368+0000","last_active":"2024-09-13T17:50:41.029513+0000","last_peered":"2024-09-13T17:50:41.029513+0000","last_clean":"2024-09-13T17:50:41.029513+0000","last_became_active":"2024-09-13T17:50:38.613010+0000","last_became_peered":"2024-09-13T17:50:38.613010+0000","last_unstale":"2024-09-13T17:50:41.029513+0000","last_undegraded":"2024-09-13T17:50:41.029513+0000","last_fullsized":"2024-09-13T17:50:41.029513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:42.706625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282530+0000","last_change":"2024-09-13T17:50:34.582327+0000","last_active":"2024-09-13T17:51:15.282530+0000","last_peered":"2024-09-13T17:51:15.282530+0000","last_clean":"2024-09-13T17:51:15.282530+0000","last_became_active":"2024-09-13T17:50:34.582050+0000","last_became_peered":"2024-09-13T17:50:34.582050+0000","last_unstale":"2024-09-13T17:51:15.282530+0000","last_undegraded":"2024-09-13T17:51:15.282530+0000","last_fullsized":"2024-09-13T17:51:15.282530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:52.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:06.145008+0000","last_change":"2024-09-13T17:50:34.211512+0000","last_active":"2024-09-13T17:51:06.145008+0000","last_peered":"2024-09-13T17:51:06.145008+0000","last_clean":"2024-09-13T17:51:06.145008+0000","last_became_active":"2024-09-13T17:50:34.211230+0000","last_became_peered":"2024-09-13T17:50:34.211230+0000","last_unstale":"2024-09-13T17:51:06.145008+0000","last_undegraded":"2024-09-13T17:51:06.145008+0000","last_fullsized":"2024-09-13T17:51:06.145008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:40.707777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.029264+0000","last_change":"2024-09-13T17:50:38.608255+0000","last_active":"2024-09-13T17:50:41.029264+0000","last_peered":"2024-09-13T17:50:41.029264+0000","last_clean":"2024-09-13T17:50:41.029264+0000","last_became_active":"2024-09-13T17:50:38.606697+0000","last_became_peered":"2024-09-13T17:50:38.606697+0000","last_unstale":"2024-09-13T17:50:41.029264+0000","last_undegraded":"2024-09-13T17:50:41.029264+0000","last_fullsized":"2024-09-13T17:50:41.029264+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:47.265007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272916+0000","last_change":"2024-09-13T17:50:34.594476+0000","last_active":"2024-09-13T17:51:15.272916+0000","last_peered":"2024-09-13T17:51:15.272916+0000","last_clean":"2024-09-13T17:51:15.272916+0000","last_became_active":"2024-09-13T17:50:34.594215+0000","last_became_peered":"2024-09-13T17:50:34.594215+0000","last_unstale":"2024-09-13T17:51:15.272916+0000","last_undegraded":"2024-09-13T17:51:15.272916+0000","last_fullsized":"2024-09-13T17:51:15.272916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:02:27.528019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281479+0000","last_change":"2024-09-13T17:50:34.204018+0000","last_active":"2024-09-13T17:51:15.281479+0000","last_peered":"2024-09-13T17:51:15.281479+0000","last_clean":"2024-09-13T17:51:15.281479+0000","last_became_active":"2024-09-13T17:50:34.203702+0000","last_became_peered":"2024-09-13T17:50:34.203702+0000","last_unstale":"2024-09-13T17:51:15.281479+0000","last_undegraded":"2024-09-13T17:51:15.281479+0000","last_fullsized":"2024-09-13T17:51:15.281479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:44.431226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.365782+0000","last_change":"2024-09-13T17:51:16.365782+0000","last_active":"2024-09-13T17:51:16.365782+0000","last_peered":"2024-09-13T17:51:16.365782+0000","last_clean":"2024-09-13T17:51:16.365782+0000","last_became_active":"2024-09-13T17:51:16.364540+0000","last_became_peered":"2024-09-13T17:51:16.364540+0000","last_unstale":"2024-09-13T17:51:16.365782+0000","last_undegraded":"2024-09-13T17:51:16.365782+0000","last_fullsized":"2024-09-13T17:51:16.365782+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:18.753557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281218+0000","last_change":"2024-09-13T17:50:34.586802+0000","last_active":"2024-09-13T17:51:15.281218+0000","last_peered":"2024-09-13T17:51:15.281218+0000","last_clean":"2024-09-13T17:51:15.281218+0000","last_became_active":"2024-09-13T17:50:34.586552+0000","last_became_peered":"2024-09-13T17:50:34.586552+0000","last_unstale":"2024-09-13T17:51:15.281218+0000","last_undegraded":"2024-09-13T17:51:15.281218+0000","last_fullsized":"2024-09-13T17:51:15.281218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:43.383975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:06.145771+0000","last_change":"2024-09-13T17:50:34.217083+0000","last_active":"2024-09-13T17:51:06.145771+0000","last_peered":"2024-09-13T17:51:06.145771+0000","last_clean":"2024-09-13T17:51:06.145771+0000","last_became_active":"2024-09-13T17:50:34.216758+0000","last_became_peered":"2024-09-13T17:50:34.216758+0000","last_unstale":"2024-09-13T17:51:06.145771+0000","last_undegraded":"2024-09-13T17:51:06.145771+0000","last_fullsized":"2024-09-13T17:51:06.145771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:36.473942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277926+0000","last_change":"2024-09-13T17:50:38.614667+0000","last_active":"2024-09-13T17:51:15.277926+0000","last_peered":"2024-09-13T17:51:15.277926+0000","last_clean":"2024-09-13T17:51:15.277926+0000","last_became_active":"2024-09-13T17:50:38.614284+0000","last_became_peered":"2024-09-13T17:50:38.614284+0000","last_unstale":"2024-09-13T17:51:15.277926+0000","last_undegraded":"2024-09-13T17:51:15.277926+0000","last_fullsized":"2024-09-13T17:51:15.277926+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:29.616794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.195375+0000","last_change":"2024-09-13T17:50:34.588603+0000","last_active":"2024-09-13T17:50:39.195375+0000","last_peered":"2024-09-13T17:50:39.195375+0000","last_clean":"2024-09-13T17:50:39.195375+0000","last_became_active":"2024-09-13T17:50:34.588317+0000","last_became_peered":"2024-09-13T17:50:34.588317+0000","last_unstale":"2024-09-13T17:50:39.195375+0000","last_undegraded":"2024-09-13T17:50:39.195375+0000","last_fullsized":"2024-09-13T17:50:39.195375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:03.743130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281302+0000","last_change":"2024-09-13T17:50:34.204335+0000","last_active":"2024-09-13T17:51:15.281302+0000","last_peered":"2024-09-13T17:51:15.281302+0000","last_clean":"2024-09-13T17:51:15.281302+0000","last_became_active":"2024-09-13T17:50:34.204060+0000","last_became_peered":"2024-09-13T17:50:34.204060+0000","last_unstale":"2024-09-13T17:51:15.281302+0000","last_undegraded":"2024-09-13T17:51:15.281302+0000","last_fullsized":"2024-09-13T17:51:15.281302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:54:41.831344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279638+0000","last_change":"2024-09-13T17:50:38.619590+0000","last_active":"2024-09-13T17:51:15.279638+0000","last_peered":"2024-09-13T17:51:15.279638+0000","last_clean":"2024-09-13T17:51:15.279638+0000","last_became_active":"2024-09-13T17:50:38.617257+0000","last_became_peered":"2024-09-13T17:50:38.617257+0000","last_unstale":"2024-09-13T17:51:15.279638+0000","last_undegraded":"2024-09-13T17:51:15.279638+0000","last_fullsized":"2024-09-13T17:51:15.279638+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:34.870291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.502147+0000","last_change":"2024-09-13T17:50:34.588466+0000","last_active":"2024-09-13T17:50:39.502147+0000","last_peered":"2024-09-13T17:50:39.502147+0000","last_clean":"2024-09-13T17:50:39.502147+0000","last_became_active":"2024-09-13T17:50:34.588063+0000","last_became_peered":"2024-09-13T17:50:34.588063+0000","last_unstale":"2024-09-13T17:50:39.502147+0000","last_undegraded":"2024-09-13T17:50:39.502147+0000","last_fullsized":"2024-09-13T17:50:39.502147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:43:00.669203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:11.233511+0000","last_change":"2024-09-13T17:50:34.209239+0000","last_active":"2024-09-13T17:51:11.233511+0000","last_peered":"2024-09-13T17:51:11.233511+0000","last_clean":"2024-09-13T17:51:11.233511+0000","last_became_active":"2024-09-13T17:50:34.208196+0000","last_became_peered":"2024-09-13T17:50:34.208196+0000","last_unstale":"2024-09-13T17:51:11.233511+0000","last_undegraded":"2024-09-13T17:51:11.233511+0000","last_fullsized":"2024-09-13T17:51:11.233511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:39.253917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281566+0000","last_change":"2024-09-13T17:50:38.611021+0000","last_active":"2024-09-13T17:51:15.281566+0000","last_peered":"2024-09-13T17:51:15.281566+0000","last_clean":"2024-09-13T17:51:15.281566+0000","last_became_active":"2024-09-13T17:50:38.609794+0000","last_became_peered":"2024-09-13T17:50:38.609794+0000","last_unstale":"2024-09-13T17:51:15.281566+0000","last_undegraded":"2024-09-13T17:51:15.281566+0000","last_fullsized":"2024-09-13T17:51:15.281566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:59.980236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281553+0000","last_change":"2024-09-13T17:50:34.584290+0000","last_active":"2024-09-13T17:51:15.281553+0000","last_peered":"2024-09-13T17:51:15.281553+0000","last_clean":"2024-09-13T17:51:15.281553+0000","last_became_active":"2024-09-13T17:50:34.584023+0000","last_became_peered":"2024-09-13T17:50:34.584023+0000","last_unstale":"2024-09-13T17:51:15.281553+0000","last_undegraded":"2024-09-13T17:51:15.281553+0000","last_fullsized":"2024-09-13T17:51:15.281553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:11.304986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"25'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278819+0000","last_change":"2024-09-13T17:50:34.198932+0000","last_active":"2024-09-13T17:51:15.278819+0000","last_peered":"2024-09-13T17:51:15.278819+0000","last_clean":"2024-09-13T17:51:15.278819+0000","last_became_active":"2024-09-13T17:50:34.198603+0000","last_became_peered":"2024-09-13T17:50:34.198603+0000","last_unstale":"2024-09-13T17:51:15.278819+0000","last_undegraded":"2024-09-13T17:51:15.278819+0000","last_fullsized":"2024-09-13T17:51:15.278819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:00.917126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281714+0000","last_change":"2024-09-13T17:50:38.611326+0000","last_active":"2024-09-13T17:51:15.281714+0000","last_peered":"2024-09-13T17:51:15.281714+0000","last_clean":"2024-09-13T17:51:15.281714+0000","last_became_active":"2024-09-13T17:50:38.610900+0000","last_became_peered":"2024-09-13T17:50:38.610900+0000","last_unstale":"2024-09-13T17:51:15.281714+0000","last_undegraded":"2024-09-13T17:51:15.281714+0000","last_fullsized":"2024-09-13T17:51:15.281714+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:17:09.994736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276990+0000","last_change":"2024-09-13T17:50:34.594482+0000","last_active":"2024-09-13T17:51:15.276990+0000","last_peered":"2024-09-13T17:51:15.276990+0000","last_clean":"2024-09-13T17:51:15.276990+0000","last_became_active":"2024-09-13T17:50:34.594201+0000","last_became_peered":"2024-09-13T17:50:34.594201+0000","last_unstale":"2024-09-13T17:51:15.276990+0000","last_undegraded":"2024-09-13T17:51:15.276990+0000","last_fullsized":"2024-09-13T17:51:15.276990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:06.697507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:11.233385+0000","last_change":"2024-09-13T17:50:34.218376+0000","last_active":"2024-09-13T17:51:11.233385+0000","last_peered":"2024-09-13T17:51:11.233385+0000","last_clean":"2024-09-13T17:51:11.233385+0000","last_became_active":"2024-09-13T17:50:34.218069+0000","last_became_peered":"2024-09-13T17:50:34.218069+0000","last_unstale":"2024-09-13T17:51:11.233385+0000","last_undegraded":"2024-09-13T17:51:11.233385+0000","last_fullsized":"2024-09-13T17:51:11.233385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:53:54.288969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277801+0000","last_change":"2024-09-13T17:50:38.613206+0000","last_active":"2024-09-13T17:51:15.277801+0000","last_peered":"2024-09-13T17:51:15.277801+0000","last_clean":"2024-09-13T17:51:15.277801+0000","last_became_active":"2024-09-13T17:50:38.612856+0000","last_became_peered":"2024-09-13T17:50:38.612856+0000","last_unstale":"2024-09-13T17:51:15.277801+0000","last_undegraded":"2024-09-13T17:51:15.277801+0000","last_fullsized":"2024-09-13T17:51:15.277801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:31:20.190996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270767+0000","last_change":"2024-09-13T17:50:34.582931+0000","last_active":"2024-09-13T17:51:15.270767+0000","last_peered":"2024-09-13T17:51:15.270767+0000","last_clean":"2024-09-13T17:51:15.270767+0000","last_became_active":"2024-09-13T17:50:34.582446+0000","last_became_peered":"2024-09-13T17:50:34.582446+0000","last_unstale":"2024-09-13T17:51:15.270767+0000","last_undegraded":"2024-09-13T17:51:15.270767+0000","last_fullsized":"2024-09-13T17:51:15.270767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:30.554785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:11.234327+0000","last_change":"2024-09-13T17:50:34.201237+0000","last_active":"2024-09-13T17:51:11.234327+0000","last_peered":"2024-09-13T17:51:11.234327+0000","last_clean":"2024-09-13T17:51:11.234327+0000","last_became_active":"2024-09-13T17:50:34.200873+0000","last_became_peered":"2024-09-13T17:50:34.200873+0000","last_unstale":"2024-09-13T17:51:11.234327+0000","last_undegraded":"2024-09-13T17:51:11.234327+0000","last_fullsized":"2024-09-13T17:51:11.234327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:02.711342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":169,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.984391+0000","last_change":"2024-09-13T17:50:22.095367+0000","last_active":"2024-09-13T17:50:38.984391+0000","last_peered":"2024-09-13T17:50:38.984391+0000","last_clean":"2024-09-13T17:50:38.984391+0000","last_became_active":"2024-09-13T17:50:22.093734+0000","last_became_peered":"2024-09-13T17:50:22.093734+0000","last_unstale":"2024-09-13T17:50:38.984391+0000","last_undegraded":"2024-09-13T17:50:38.984391+0000","last_fullsized":"2024-09-13T17:50:38.984391+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_clean_scrub_stamp":"2024-09-13T17:50:21.057812+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:23.531613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.014187+0000","last_change":"2024-09-13T17:50:38.618880+0000","last_active":"2024-09-13T17:50:41.014187+0000","last_peered":"2024-09-13T17:50:41.014187+0000","last_clean":"2024-09-13T17:50:41.014187+0000","last_became_active":"2024-09-13T17:50:38.618488+0000","last_became_peered":"2024-09-13T17:50:38.618488+0000","last_unstale":"2024-09-13T17:50:41.014187+0000","last_undegraded":"2024-09-13T17:50:41.014187+0000","last_fullsized":"2024-09-13T17:50:41.014187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:03.431876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.985276+0000","last_change":"2024-09-13T17:50:34.585881+0000","last_active":"2024-09-13T17:50:38.985276+0000","last_peered":"2024-09-13T17:50:38.985276+0000","last_clean":"2024-09-13T17:50:38.985276+0000","last_became_active":"2024-09-13T17:50:34.585166+0000","last_became_peered":"2024-09-13T17:50:34.585166+0000","last_unstale":"2024-09-13T17:50:38.985276+0000","last_undegraded":"2024-09-13T17:50:38.985276+0000","last_fullsized":"2024-09-13T17:50:38.985276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:06.654658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279549+0000","last_change":"2024-09-13T17:50:34.210934+0000","last_active":"2024-09-13T17:51:15.279549+0000","last_peered":"2024-09-13T17:51:15.279549+0000","last_clean":"2024-09-13T17:51:15.279549+0000","last_became_active":"2024-09-13T17:50:34.210589+0000","last_became_peered":"2024-09-13T17:50:34.210589+0000","last_unstale":"2024-09-13T17:51:15.279549+0000","last_undegraded":"2024-09-13T17:51:15.279549+0000","last_fullsized":"2024-09-13T17:51:15.279549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:26.213295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"peering","last_fresh":"2024-09-13T17:51:15.275839+0000","last_change":"2024-09-13T17:51:15.275971+0000","last_active":"2024-09-13T17:51:15.275839+0000","last_peered":"2024-09-13T17:50:41.014101+0000","last_clean":"2024-09-13T17:50:41.014101+0000","last_became_active":"2024-09-13T17:50:38.615358+0000","last_became_peered":"2024-09-13T17:50:38.615358+0000","last_unstale":"2024-09-13T17:51:15.275839+0000","last_undegraded":"2024-09-13T17:51:15.275839+0000","last_fullsized":"2024-09-13T17:51:15.275839+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:31.449312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1,2,7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.195149+0000","last_change":"2024-09-13T17:50:34.590169+0000","last_active":"2024-09-13T17:50:39.195149+0000","last_peered":"2024-09-13T17:50:39.195149+0000","last_clean":"2024-09-13T17:50:39.195149+0000","last_became_active":"2024-09-13T17:50:34.589434+0000","last_became_peered":"2024-09-13T17:50:34.589434+0000","last_unstale":"2024-09-13T17:50:39.195149+0000","last_undegraded":"2024-09-13T17:50:39.195149+0000","last_fullsized":"2024-09-13T17:50:39.195149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:12.545412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"25'1","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281689+0000","last_change":"2024-09-13T17:50:34.204269+0000","last_active":"2024-09-13T17:51:15.281689+0000","last_peered":"2024-09-13T17:51:15.281689+0000","last_clean":"2024-09-13T17:51:15.281689+0000","last_became_active":"2024-09-13T17:50:34.203874+0000","last_became_peered":"2024-09-13T17:50:34.203874+0000","last_unstale":"2024-09-13T17:51:15.281689+0000","last_undegraded":"2024-09-13T17:51:15.281689+0000","last_fullsized":"2024-09-13T17:51:15.281689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:10.410267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278070+0000","last_change":"2024-09-13T17:50:38.615989+0000","last_active":"2024-09-13T17:51:15.278070+0000","last_peered":"2024-09-13T17:51:15.278070+0000","last_clean":"2024-09-13T17:51:15.278070+0000","last_became_active":"2024-09-13T17:50:38.615530+0000","last_became_peered":"2024-09-13T17:50:38.615530+0000","last_unstale":"2024-09-13T17:51:15.278070+0000","last_undegraded":"2024-09-13T17:51:15.278070+0000","last_fullsized":"2024-09-13T17:51:15.278070+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:41.064092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.985714+0000","last_change":"2024-09-13T17:50:34.595120+0000","last_active":"2024-09-13T17:50:38.985714+0000","last_peered":"2024-09-13T17:50:38.985714+0000","last_clean":"2024-09-13T17:50:38.985714+0000","last_became_active":"2024-09-13T17:50:34.594842+0000","last_became_peered":"2024-09-13T17:50:34.594842+0000","last_unstale":"2024-09-13T17:50:38.985714+0000","last_undegraded":"2024-09-13T17:50:38.985714+0000","last_fullsized":"2024-09-13T17:50:38.985714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:15:38.982524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"25'2","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279806+0000","last_change":"2024-09-13T17:50:34.199957+0000","last_active":"2024-09-13T17:51:15.279806+0000","last_peered":"2024-09-13T17:51:15.279806+0000","last_clean":"2024-09-13T17:51:15.279806+0000","last_became_active":"2024-09-13T17:50:34.199650+0000","last_became_peered":"2024-09-13T17:50:34.199650+0000","last_unstale":"2024-09-13T17:51:15.279806+0000","last_undegraded":"2024-09-13T17:51:15.279806+0000","last_fullsized":"2024-09-13T17:51:15.279806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:26.929549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.013644+0000","last_change":"2024-09-13T17:50:38.609731+0000","last_active":"2024-09-13T17:50:41.013644+0000","last_peered":"2024-09-13T17:50:41.013644+0000","last_clean":"2024-09-13T17:50:41.013644+0000","last_became_active":"2024-09-13T17:50:38.609354+0000","last_became_peered":"2024-09-13T17:50:38.609354+0000","last_unstale":"2024-09-13T17:50:41.013644+0000","last_undegraded":"2024-09-13T17:50:41.013644+0000","last_fullsized":"2024-09-13T17:50:41.013644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:14.142369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.985412+0000","last_change":"2024-09-13T17:50:34.595146+0000","last_active":"2024-09-13T17:50:38.985412+0000","last_peered":"2024-09-13T17:50:38.985412+0000","last_clean":"2024-09-13T17:50:38.985412+0000","last_became_active":"2024-09-13T17:50:34.594876+0000","last_became_peered":"2024-09-13T17:50:34.594876+0000","last_unstale":"2024-09-13T17:50:38.985412+0000","last_undegraded":"2024-09-13T17:50:38.985412+0000","last_fullsized":"2024-09-13T17:50:38.985412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:26:10.638438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"25'1","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.582895+0000","last_change":"2024-09-13T17:50:34.200640+0000","last_active":"2024-09-13T17:50:39.582895+0000","last_peered":"2024-09-13T17:50:39.582895+0000","last_clean":"2024-09-13T17:50:39.582895+0000","last_became_active":"2024-09-13T17:50:34.200197+0000","last_became_peered":"2024-09-13T17:50:34.200197+0000","last_unstale":"2024-09-13T17:50:39.582895+0000","last_undegraded":"2024-09-13T17:50:39.582895+0000","last_fullsized":"2024-09-13T17:50:39.582895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:37:53.039720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"peering","last_fresh":"2024-09-13T17:51:15.270506+0000","last_change":"2024-09-13T17:51:15.270770+0000","last_active":"2024-09-13T17:51:15.270506+0000","last_peered":"2024-09-13T17:50:41.029222+0000","last_clean":"2024-09-13T17:50:41.029222+0000","last_became_active":"2024-09-13T17:50:38.618491+0000","last_became_peered":"2024-09-13T17:50:38.618491+0000","last_unstale":"2024-09-13T17:51:15.270506+0000","last_undegraded":"2024-09-13T17:51:15.270506+0000","last_fullsized":"2024-09-13T17:51:15.270506+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:30:49.103522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3,6],"acting":[4,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,3,6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.984764+0000","last_change":"2024-09-13T17:50:34.585741+0000","last_active":"2024-09-13T17:50:38.984764+0000","last_peered":"2024-09-13T17:50:38.984764+0000","last_clean":"2024-09-13T17:50:38.984764+0000","last_became_active":"2024-09-13T17:50:34.585464+0000","last_became_peered":"2024-09-13T17:50:34.585464+0000","last_unstale":"2024-09-13T17:50:38.984764+0000","last_undegraded":"2024-09-13T17:50:38.984764+0000","last_fullsized":"2024-09-13T17:50:38.984764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:59:22.031459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"25'15","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278697+0000","last_change":"2024-09-13T17:50:34.201272+0000","last_active":"2024-09-13T17:51:15.278697+0000","last_peered":"2024-09-13T17:51:15.278697+0000","last_clean":"2024-09-13T17:51:15.278697+0000","last_became_active":"2024-09-13T17:50:34.200936+0000","last_became_peered":"2024-09-13T17:50:34.200936+0000","last_unstale":"2024-09-13T17:51:15.278697+0000","last_undegraded":"2024-09-13T17:51:15.278697+0000","last_fullsized":"2024-09-13T17:51:15.278697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:52.094879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277648+0000","last_change":"2024-09-13T17:50:38.615229+0000","last_active":"2024-09-13T17:51:15.277648+0000","last_peered":"2024-09-13T17:51:15.277648+0000","last_clean":"2024-09-13T17:51:15.277648+0000","last_became_active":"2024-09-13T17:50:38.614739+0000","last_became_peered":"2024-09-13T17:50:38.614739+0000","last_unstale":"2024-09-13T17:51:15.277648+0000","last_undegraded":"2024-09-13T17:51:15.277648+0000","last_fullsized":"2024-09-13T17:51:15.277648+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:18.619529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.982245+0000","last_change":"2024-09-13T17:50:34.588848+0000","last_active":"2024-09-13T17:50:38.982245+0000","last_peered":"2024-09-13T17:50:38.982245+0000","last_clean":"2024-09-13T17:50:38.982245+0000","last_became_active":"2024-09-13T17:50:34.588565+0000","last_became_peered":"2024-09-13T17:50:34.588565+0000","last_unstale":"2024-09-13T17:50:38.982245+0000","last_undegraded":"2024-09-13T17:50:38.982245+0000","last_fullsized":"2024-09-13T17:50:38.982245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:47.927865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:11.235969+0000","last_change":"2024-09-13T17:50:34.199008+0000","last_active":"2024-09-13T17:51:11.235969+0000","last_peered":"2024-09-13T17:51:11.235969+0000","last_clean":"2024-09-13T17:51:11.235969+0000","last_became_active":"2024-09-13T17:50:34.197839+0000","last_became_peered":"2024-09-13T17:50:34.197839+0000","last_unstale":"2024-09-13T17:51:11.235969+0000","last_undegraded":"2024-09-13T17:51:11.235969+0000","last_fullsized":"2024-09-13T17:51:11.235969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:51.430710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272409+0000","last_change":"2024-09-13T17:50:38.611080+0000","last_active":"2024-09-13T17:51:15.272409+0000","last_peered":"2024-09-13T17:51:15.272409+0000","last_clean":"2024-09-13T17:51:15.272409+0000","last_became_active":"2024-09-13T17:50:38.610688+0000","last_became_peered":"2024-09-13T17:50:38.610688+0000","last_unstale":"2024-09-13T17:51:15.272409+0000","last_undegraded":"2024-09-13T17:51:15.272409+0000","last_fullsized":"2024-09-13T17:51:15.272409+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:28.527617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.984724+0000","last_change":"2024-09-13T17:50:34.594763+0000","last_active":"2024-09-13T17:50:38.984724+0000","last_peered":"2024-09-13T17:50:38.984724+0000","last_clean":"2024-09-13T17:50:38.984724+0000","last_became_active":"2024-09-13T17:50:34.594471+0000","last_became_peered":"2024-09-13T17:50:34.594471+0000","last_unstale":"2024-09-13T17:50:38.984724+0000","last_undegraded":"2024-09-13T17:50:38.984724+0000","last_fullsized":"2024-09-13T17:50:38.984724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:20:58.718271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:06.145238+0000","last_change":"2024-09-13T17:50:34.209432+0000","last_active":"2024-09-13T17:51:06.145238+0000","last_peered":"2024-09-13T17:51:06.145238+0000","last_clean":"2024-09-13T17:51:06.145238+0000","last_became_active":"2024-09-13T17:50:34.208817+0000","last_became_peered":"2024-09-13T17:50:34.208817+0000","last_unstale":"2024-09-13T17:51:06.145238+0000","last_undegraded":"2024-09-13T17:51:06.145238+0000","last_fullsized":"2024-09-13T17:51:06.145238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:49.587528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272159+0000","last_change":"2024-09-13T17:50:38.609461+0000","last_active":"2024-09-13T17:51:15.272159+0000","last_peered":"2024-09-13T17:51:15.272159+0000","last_clean":"2024-09-13T17:51:15.272159+0000","last_became_active":"2024-09-13T17:50:38.609128+0000","last_became_peered":"2024-09-13T17:50:38.609128+0000","last_unstale":"2024-09-13T17:51:15.272159+0000","last_undegraded":"2024-09-13T17:51:15.272159+0000","last_fullsized":"2024-09-13T17:51:15.272159+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:02.286825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.283064+0000","last_change":"2024-09-13T17:50:34.592970+0000","last_active":"2024-09-13T17:51:15.283064+0000","last_peered":"2024-09-13T17:51:15.283064+0000","last_clean":"2024-09-13T17:51:15.283064+0000","last_became_active":"2024-09-13T17:50:34.592669+0000","last_became_peered":"2024-09-13T17:50:34.592669+0000","last_unstale":"2024-09-13T17:51:15.283064+0000","last_undegraded":"2024-09-13T17:51:15.283064+0000","last_fullsized":"2024-09-13T17:51:15.283064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:52.031261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282599+0000","last_change":"2024-09-13T17:50:34.200798+0000","last_active":"2024-09-13T17:51:15.282599+0000","last_peered":"2024-09-13T17:51:15.282599+0000","last_clean":"2024-09-13T17:51:15.282599+0000","last_became_active":"2024-09-13T17:50:34.200529+0000","last_became_peered":"2024-09-13T17:50:34.200529+0000","last_unstale":"2024-09-13T17:51:15.282599+0000","last_undegraded":"2024-09-13T17:51:15.282599+0000","last_fullsized":"2024-09-13T17:51:15.282599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:26.334296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.013911+0000","last_change":"2024-09-13T17:50:38.606287+0000","last_active":"2024-09-13T17:50:41.013911+0000","last_peered":"2024-09-13T17:50:41.013911+0000","last_clean":"2024-09-13T17:50:41.013911+0000","last_became_active":"2024-09-13T17:50:38.605826+0000","last_became_peered":"2024-09-13T17:50:38.605826+0000","last_unstale":"2024-09-13T17:50:41.013911+0000","last_undegraded":"2024-09-13T17:50:41.013911+0000","last_fullsized":"2024-09-13T17:50:41.013911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:18.709454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.985113+0000","last_change":"2024-09-13T17:50:34.591579+0000","last_active":"2024-09-13T17:50:38.985113+0000","last_peered":"2024-09-13T17:50:38.985113+0000","last_clean":"2024-09-13T17:50:38.985113+0000","last_became_active":"2024-09-13T17:50:34.586165+0000","last_became_peered":"2024-09-13T17:50:34.586165+0000","last_unstale":"2024-09-13T17:50:38.985113+0000","last_undegraded":"2024-09-13T17:50:38.985113+0000","last_fullsized":"2024-09-13T17:50:38.985113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:56:19.205176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'11","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:06.146224+0000","last_change":"2024-09-13T17:50:34.203218+0000","last_active":"2024-09-13T17:51:06.146224+0000","last_peered":"2024-09-13T17:51:06.146224+0000","last_clean":"2024-09-13T17:51:06.146224+0000","last_became_active":"2024-09-13T17:50:34.202948+0000","last_became_peered":"2024-09-13T17:50:34.202948+0000","last_unstale":"2024-09-13T17:51:06.146224+0000","last_undegraded":"2024-09-13T17:51:06.146224+0000","last_fullsized":"2024-09-13T17:51:06.146224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:46.535569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.280382+0000","last_change":"2024-09-13T17:50:38.610597+0000","last_active":"2024-09-13T17:51:15.280382+0000","last_peered":"2024-09-13T17:51:15.280382+0000","last_clean":"2024-09-13T17:51:15.280382+0000","last_became_active":"2024-09-13T17:50:38.610055+0000","last_became_peered":"2024-09-13T17:50:38.610055+0000","last_unstale":"2024-09-13T17:51:15.280382+0000","last_undegraded":"2024-09-13T17:51:15.280382+0000","last_fullsized":"2024-09-13T17:51:15.280382+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:55.754936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.502265+0000","last_change":"2024-09-13T17:50:34.593963+0000","last_active":"2024-09-13T17:50:39.502265+0000","last_peered":"2024-09-13T17:50:39.502265+0000","last_clean":"2024-09-13T17:50:39.502265+0000","last_became_active":"2024-09-13T17:50:34.593454+0000","last_became_peered":"2024-09-13T17:50:34.593454+0000","last_unstale":"2024-09-13T17:50:39.502265+0000","last_undegraded":"2024-09-13T17:50:39.502265+0000","last_fullsized":"2024-09-13T17:50:39.502265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:18.883521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281506+0000","last_change":"2024-09-13T17:50:34.207936+0000","last_active":"2024-09-13T17:51:15.281506+0000","last_peered":"2024-09-13T17:51:15.281506+0000","last_clean":"2024-09-13T17:51:15.281506+0000","last_became_active":"2024-09-13T17:50:34.207605+0000","last_became_peered":"2024-09-13T17:50:34.207605+0000","last_unstale":"2024-09-13T17:51:15.281506+0000","last_undegraded":"2024-09-13T17:51:15.281506+0000","last_fullsized":"2024-09-13T17:51:15.281506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:05.494406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.029376+0000","last_change":"2024-09-13T17:50:38.616297+0000","last_active":"2024-09-13T17:50:41.029376+0000","last_peered":"2024-09-13T17:50:41.029376+0000","last_clean":"2024-09-13T17:50:41.029376+0000","last_became_active":"2024-09-13T17:50:38.614165+0000","last_became_peered":"2024-09-13T17:50:38.614165+0000","last_unstale":"2024-09-13T17:50:41.029376+0000","last_undegraded":"2024-09-13T17:50:41.029376+0000","last_fullsized":"2024-09-13T17:50:41.029376+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:11.357026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278023+0000","last_change":"2024-09-13T17:50:34.583001+0000","last_active":"2024-09-13T17:51:15.278023+0000","last_peered":"2024-09-13T17:51:15.278023+0000","last_clean":"2024-09-13T17:51:15.278023+0000","last_became_active":"2024-09-13T17:50:34.582529+0000","last_became_peered":"2024-09-13T17:50:34.582529+0000","last_unstale":"2024-09-13T17:51:15.278023+0000","last_undegraded":"2024-09-13T17:51:15.278023+0000","last_fullsized":"2024-09-13T17:51:15.278023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:05.287181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:06.143515+0000","last_change":"2024-09-13T17:50:34.205138+0000","last_active":"2024-09-13T17:51:06.143515+0000","last_peered":"2024-09-13T17:51:06.143515+0000","last_clean":"2024-09-13T17:51:06.143515+0000","last_became_active":"2024-09-13T17:50:34.203360+0000","last_became_peered":"2024-09-13T17:50:34.203360+0000","last_unstale":"2024-09-13T17:51:06.143515+0000","last_undegraded":"2024-09-13T17:51:06.143515+0000","last_fullsized":"2024-09-13T17:51:06.143515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:35.756963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272372+0000","last_change":"2024-09-13T17:50:38.611036+0000","last_active":"2024-09-13T17:51:15.272372+0000","last_peered":"2024-09-13T17:51:15.272372+0000","last_clean":"2024-09-13T17:51:15.272372+0000","last_became_active":"2024-09-13T17:50:38.610692+0000","last_became_peered":"2024-09-13T17:50:38.610692+0000","last_unstale":"2024-09-13T17:51:15.272372+0000","last_undegraded":"2024-09-13T17:51:15.272372+0000","last_fullsized":"2024-09-13T17:51:15.272372+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:10.147998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.368023+0000","last_change":"2024-09-13T17:51:16.365295+0000","last_active":"2024-09-13T17:51:16.368023+0000","last_peered":"2024-09-13T17:51:16.368023+0000","last_clean":"2024-09-13T17:51:16.368023+0000","last_became_active":"2024-09-13T17:51:16.364437+0000","last_became_peered":"2024-09-13T17:51:16.364437+0000","last_unstale":"2024-09-13T17:51:16.368023+0000","last_undegraded":"2024-09-13T17:51:16.368023+0000","last_fullsized":"2024-09-13T17:51:16.368023+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:23.208899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:06.146374+0000","last_change":"2024-09-13T17:50:34.204376+0000","last_active":"2024-09-13T17:51:06.146374+0000","last_peered":"2024-09-13T17:51:06.146374+0000","last_clean":"2024-09-13T17:51:06.146374+0000","last_became_active":"2024-09-13T17:50:34.204003+0000","last_became_peered":"2024-09-13T17:50:34.204003+0000","last_unstale":"2024-09-13T17:51:06.146374+0000","last_undegraded":"2024-09-13T17:51:06.146374+0000","last_fullsized":"2024-09-13T17:51:06.146374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:14.023990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270270+0000","last_change":"2024-09-13T17:50:38.606144+0000","last_active":"2024-09-13T17:51:15.270270+0000","last_peered":"2024-09-13T17:51:15.270270+0000","last_clean":"2024-09-13T17:51:15.270270+0000","last_became_active":"2024-09-13T17:50:38.605646+0000","last_became_peered":"2024-09-13T17:50:38.605646+0000","last_unstale":"2024-09-13T17:51:15.270270+0000","last_undegraded":"2024-09-13T17:51:15.270270+0000","last_fullsized":"2024-09-13T17:51:15.270270+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:07.120075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277435+0000","last_change":"2024-09-13T17:50:38.618855+0000","last_active":"2024-09-13T17:51:15.277435+0000","last_peered":"2024-09-13T17:51:15.277435+0000","last_clean":"2024-09-13T17:51:15.277435+0000","last_became_active":"2024-09-13T17:50:38.618449+0000","last_became_peered":"2024-09-13T17:50:38.618449+0000","last_unstale":"2024-09-13T17:51:15.277435+0000","last_undegraded":"2024-09-13T17:51:15.277435+0000","last_fullsized":"2024-09-13T17:51:15.277435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:29:01.114087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.280511+0000","last_change":"2024-09-13T17:50:34.590079+0000","last_active":"2024-09-13T17:51:15.280511+0000","last_peered":"2024-09-13T17:51:15.280511+0000","last_clean":"2024-09-13T17:51:15.280511+0000","last_became_active":"2024-09-13T17:50:34.589396+0000","last_became_peered":"2024-09-13T17:50:34.589396+0000","last_unstale":"2024-09-13T17:51:15.280511+0000","last_undegraded":"2024-09-13T17:51:15.280511+0000","last_fullsized":"2024-09-13T17:51:15.280511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:03.308341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:06.143408+0000","last_change":"2024-09-13T17:50:34.206575+0000","last_active":"2024-09-13T17:51:06.143408+0000","last_peered":"2024-09-13T17:51:06.143408+0000","last_clean":"2024-09-13T17:51:06.143408+0000","last_became_active":"2024-09-13T17:50:34.206308+0000","last_became_peered":"2024-09-13T17:50:34.206308+0000","last_unstale":"2024-09-13T17:51:06.143408+0000","last_undegraded":"2024-09-13T17:51:06.143408+0000","last_fullsized":"2024-09-13T17:51:06.143408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:34.766538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:41.014256+0000","last_change":"2024-09-13T17:50:38.618131+0000","last_active":"2024-09-13T17:50:41.014256+0000","last_peered":"2024-09-13T17:50:41.014256+0000","last_clean":"2024-09-13T17:50:41.014256+0000","last_became_active":"2024-09-13T17:50:38.617611+0000","last_became_peered":"2024-09-13T17:50:38.617611+0000","last_unstale":"2024-09-13T17:50:41.014256+0000","last_undegraded":"2024-09-13T17:50:41.014256+0000","last_fullsized":"2024-09-13T17:50:41.014256+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:30.746445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272535+0000","last_change":"2024-09-13T17:50:34.582153+0000","last_active":"2024-09-13T17:51:15.272535+0000","last_peered":"2024-09-13T17:51:15.272535+0000","last_clean":"2024-09-13T17:51:15.272535+0000","last_became_active":"2024-09-13T17:50:34.581708+0000","last_became_peered":"2024-09-13T17:50:34.581708+0000","last_unstale":"2024-09-13T17:51:15.272535+0000","last_undegraded":"2024-09-13T17:51:15.272535+0000","last_fullsized":"2024-09-13T17:51:15.272535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:22.238810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270828+0000","last_change":"2024-09-13T17:50:34.205489+0000","last_active":"2024-09-13T17:51:15.270828+0000","last_peered":"2024-09-13T17:51:15.270828+0000","last_clean":"2024-09-13T17:51:15.270828+0000","last_became_active":"2024-09-13T17:50:34.205202+0000","last_became_peered":"2024-09-13T17:50:34.205202+0000","last_unstale":"2024-09-13T17:51:15.270828+0000","last_undegraded":"2024-09-13T17:51:15.270828+0000","last_fullsized":"2024-09-13T17:51:15.270828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:54.450276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:51:11.234223+0000","last_change":"2024-09-13T17:50:34.202470+0000","last_active":"2024-09-13T17:51:11.234223+0000","last_peered":"2024-09-13T17:51:11.234223+0000","last_clean":"2024-09-13T17:51:11.234223+0000","last_became_active":"2024-09-13T17:50:34.202163+0000","last_became_peered":"2024-09-13T17:50:34.202163+0000","last_unstale":"2024-09-13T17:51:11.234223+0000","last_undegraded":"2024-09-13T17:51:11.234223+0000","last_fullsized":"2024-09-13T17:51:11.234223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:03.472932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282749+0000","last_change":"2024-09-13T17:50:34.590947+0000","last_active":"2024-09-13T17:51:15.282749+0000","last_peered":"2024-09-13T17:51:15.282749+0000","last_clean":"2024-09-13T17:51:15.282749+0000","last_became_active":"2024-09-13T17:50:34.590497+0000","last_became_peered":"2024-09-13T17:50:34.590497+0000","last_unstale":"2024-09-13T17:51:15.282749+0000","last_undegraded":"2024-09-13T17:51:15.282749+0000","last_fullsized":"2024-09-13T17:51:15.282749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:56:27.848092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279328+0000","last_change":"2024-09-13T17:50:38.618518+0000","last_active":"2024-09-13T17:51:15.279328+0000","last_peered":"2024-09-13T17:51:15.279328+0000","last_clean":"2024-09-13T17:51:15.279328+0000","last_became_active":"2024-09-13T17:50:38.618108+0000","last_became_peered":"2024-09-13T17:50:38.618108+0000","last_unstale":"2024-09-13T17:51:15.279328+0000","last_undegraded":"2024-09-13T17:51:15.279328+0000","last_fullsized":"2024-09-13T17:51:15.279328+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:42.260237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"25'5","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271153+0000","last_change":"2024-09-13T17:50:34.218459+0000","last_active":"2024-09-13T17:51:15.271153+0000","last_peered":"2024-09-13T17:51:15.271153+0000","last_clean":"2024-09-13T17:51:15.271153+0000","last_became_active":"2024-09-13T17:50:34.218087+0000","last_became_peered":"2024-09-13T17:50:34.218087+0000","last_unstale":"2024-09-13T17:51:15.271153+0000","last_undegraded":"2024-09-13T17:51:15.271153+0000","last_fullsized":"2024-09-13T17:51:15.271153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:56.914544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.501733+0000","last_change":"2024-09-13T17:50:34.583845+0000","last_active":"2024-09-13T17:50:39.501733+0000","last_peered":"2024-09-13T17:50:39.501733+0000","last_clean":"2024-09-13T17:50:39.501733+0000","last_became_active":"2024-09-13T17:50:34.583283+0000","last_became_peered":"2024-09-13T17:50:34.583283+0000","last_unstale":"2024-09-13T17:50:39.501733+0000","last_undegraded":"2024-09-13T17:50:39.501733+0000","last_fullsized":"2024-09-13T17:50:39.501733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:55.630285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271562+0000","last_change":"2024-09-13T17:50:38.611987+0000","last_active":"2024-09-13T17:51:15.271562+0000","last_peered":"2024-09-13T17:51:15.271562+0000","last_clean":"2024-09-13T17:51:15.271562+0000","last_became_active":"2024-09-13T17:50:38.611636+0000","last_became_peered":"2024-09-13T17:50:38.611636+0000","last_unstale":"2024-09-13T17:51:15.271562+0000","last_undegraded":"2024-09-13T17:51:15.271562+0000","last_fullsized":"2024-09-13T17:51:15.271562+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:41.111562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.981989+0000","last_change":"2024-09-13T17:50:34.588927+0000","last_active":"2024-09-13T17:50:38.981989+0000","last_peered":"2024-09-13T17:50:38.981989+0000","last_clean":"2024-09-13T17:50:38.981989+0000","last_became_active":"2024-09-13T17:50:34.588651+0000","last_became_peered":"2024-09-13T17:50:34.588651+0000","last_unstale":"2024-09-13T17:50:38.981989+0000","last_undegraded":"2024-09-13T17:50:38.981989+0000","last_fullsized":"2024-09-13T17:50:38.981989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:49.429670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278614+0000","last_change":"2024-09-13T17:50:34.583795+0000","last_active":"2024-09-13T17:51:15.278614+0000","last_peered":"2024-09-13T17:51:15.278614+0000","last_clean":"2024-09-13T17:51:15.278614+0000","last_became_active":"2024-09-13T17:50:34.583493+0000","last_became_peered":"2024-09-13T17:50:34.583493+0000","last_unstale":"2024-09-13T17:51:15.278614+0000","last_undegraded":"2024-09-13T17:51:15.278614+0000","last_fullsized":"2024-09-13T17:51:15.278614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:30.463932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.194789+0000","last_change":"2024-09-13T17:50:34.587624+0000","last_active":"2024-09-13T17:50:39.194789+0000","last_peered":"2024-09-13T17:50:39.194789+0000","last_clean":"2024-09-13T17:50:39.194789+0000","last_became_active":"2024-09-13T17:50:34.587119+0000","last_became_peered":"2024-09-13T17:50:34.587119+0000","last_unstale":"2024-09-13T17:50:39.194789+0000","last_undegraded":"2024-09-13T17:50:39.194789+0000","last_fullsized":"2024-09-13T17:50:39.194789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:16.783899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278177+0000","last_change":"2024-09-13T17:50:34.581923+0000","last_active":"2024-09-13T17:51:15.278177+0000","last_peered":"2024-09-13T17:51:15.278177+0000","last_clean":"2024-09-13T17:51:15.278177+0000","last_became_active":"2024-09-13T17:50:34.581628+0000","last_became_peered":"2024-09-13T17:50:34.581628+0000","last_unstale":"2024-09-13T17:51:15.278177+0000","last_undegraded":"2024-09-13T17:51:15.278177+0000","last_fullsized":"2024-09-13T17:51:15.278177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:28:55.936864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272460+0000","last_change":"2024-09-13T17:50:34.595037+0000","last_active":"2024-09-13T17:51:15.272460+0000","last_peered":"2024-09-13T17:51:15.272460+0000","last_clean":"2024-09-13T17:51:15.272460+0000","last_became_active":"2024-09-13T17:50:34.594857+0000","last_became_peered":"2024-09-13T17:50:34.594857+0000","last_unstale":"2024-09-13T17:51:15.272460+0000","last_undegraded":"2024-09-13T17:51:15.272460+0000","last_fullsized":"2024-09-13T17:51:15.272460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:11.420413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281006+0000","last_change":"2024-09-13T17:50:34.589651+0000","last_active":"2024-09-13T17:51:15.281006+0000","last_peered":"2024-09-13T17:51:15.281006+0000","last_clean":"2024-09-13T17:51:15.281006+0000","last_became_active":"2024-09-13T17:50:34.589354+0000","last_became_peered":"2024-09-13T17:50:34.589354+0000","last_unstale":"2024-09-13T17:51:15.281006+0000","last_undegraded":"2024-09-13T17:51:15.281006+0000","last_fullsized":"2024-09-13T17:51:15.281006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:31.294048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.983352+0000","last_change":"2024-09-13T17:50:34.597949+0000","last_active":"2024-09-13T17:50:38.983352+0000","last_peered":"2024-09-13T17:50:38.983352+0000","last_clean":"2024-09-13T17:50:38.983352+0000","last_became_active":"2024-09-13T17:50:34.597621+0000","last_became_peered":"2024-09-13T17:50:34.597621+0000","last_unstale":"2024-09-13T17:50:38.983352+0000","last_undegraded":"2024-09-13T17:50:38.983352+0000","last_fullsized":"2024-09-13T17:50:38.983352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:23.133462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272676+0000","last_change":"2024-09-13T17:50:34.589893+0000","last_active":"2024-09-13T17:51:15.272676+0000","last_peered":"2024-09-13T17:51:15.272676+0000","last_clean":"2024-09-13T17:51:15.272676+0000","last_became_active":"2024-09-13T17:50:34.589626+0000","last_became_peered":"2024-09-13T17:50:34.589626+0000","last_unstale":"2024-09-13T17:51:15.272676+0000","last_undegraded":"2024-09-13T17:51:15.272676+0000","last_fullsized":"2024-09-13T17:51:15.272676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:41.321025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.194245+0000","last_change":"2024-09-13T17:50:34.590046+0000","last_active":"2024-09-13T17:50:39.194245+0000","last_peered":"2024-09-13T17:50:39.194245+0000","last_clean":"2024-09-13T17:50:39.194245+0000","last_became_active":"2024-09-13T17:50:34.589757+0000","last_became_peered":"2024-09-13T17:50:34.589757+0000","last_unstale":"2024-09-13T17:50:39.194245+0000","last_undegraded":"2024-09-13T17:50:39.194245+0000","last_fullsized":"2024-09-13T17:50:39.194245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:28.239550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270567+0000","last_change":"2024-09-13T17:50:34.595518+0000","last_active":"2024-09-13T17:51:15.270567+0000","last_peered":"2024-09-13T17:51:15.270567+0000","last_clean":"2024-09-13T17:51:15.270567+0000","last_became_active":"2024-09-13T17:50:34.595242+0000","last_became_peered":"2024-09-13T17:50:34.595242+0000","last_unstale":"2024-09-13T17:51:15.270567+0000","last_undegraded":"2024-09-13T17:51:15.270567+0000","last_fullsized":"2024-09-13T17:51:15.270567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:26.149384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.194664+0000","last_change":"2024-09-13T17:50:34.593183+0000","last_active":"2024-09-13T17:50:39.194664+0000","last_peered":"2024-09-13T17:50:39.194664+0000","last_clean":"2024-09-13T17:50:39.194664+0000","last_became_active":"2024-09-13T17:50:34.592893+0000","last_became_peered":"2024-09-13T17:50:34.592893+0000","last_unstale":"2024-09-13T17:50:39.194664+0000","last_undegraded":"2024-09-13T17:50:39.194664+0000","last_fullsized":"2024-09-13T17:50:39.194664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:19.459505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278000+0000","last_change":"2024-09-13T17:50:34.594679+0000","last_active":"2024-09-13T17:51:15.278000+0000","last_peered":"2024-09-13T17:51:15.278000+0000","last_clean":"2024-09-13T17:51:15.278000+0000","last_became_active":"2024-09-13T17:50:34.593750+0000","last_became_peered":"2024-09-13T17:50:34.593750+0000","last_unstale":"2024-09-13T17:51:15.278000+0000","last_undegraded":"2024-09-13T17:51:15.278000+0000","last_fullsized":"2024-09-13T17:51:15.278000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:02.135577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.983701+0000","last_change":"2024-09-13T17:50:34.598736+0000","last_active":"2024-09-13T17:50:38.983701+0000","last_peered":"2024-09-13T17:50:38.983701+0000","last_clean":"2024-09-13T17:50:38.983701+0000","last_became_active":"2024-09-13T17:50:34.598433+0000","last_became_peered":"2024-09-13T17:50:34.598433+0000","last_unstale":"2024-09-13T17:50:38.983701+0000","last_undegraded":"2024-09-13T17:50:38.983701+0000","last_fullsized":"2024-09-13T17:50:38.983701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:08.975139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.985564+0000","last_change":"2024-09-13T17:50:34.591795+0000","last_active":"2024-09-13T17:50:38.985564+0000","last_peered":"2024-09-13T17:50:38.985564+0000","last_clean":"2024-09-13T17:50:38.985564+0000","last_became_active":"2024-09-13T17:50:34.591286+0000","last_became_peered":"2024-09-13T17:50:34.591286+0000","last_unstale":"2024-09-13T17:50:38.985564+0000","last_undegraded":"2024-09-13T17:50:38.985564+0000","last_fullsized":"2024-09-13T17:50:38.985564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:31.580247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279687+0000","last_change":"2024-09-13T17:50:34.587774+0000","last_active":"2024-09-13T17:51:15.279687+0000","last_peered":"2024-09-13T17:51:15.279687+0000","last_clean":"2024-09-13T17:51:15.279687+0000","last_became_active":"2024-09-13T17:50:34.587506+0000","last_became_peered":"2024-09-13T17:50:34.587506+0000","last_unstale":"2024-09-13T17:51:15.279687+0000","last_undegraded":"2024-09-13T17:51:15.279687+0000","last_fullsized":"2024-09-13T17:51:15.279687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:46.275528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.195748+0000","last_change":"2024-09-13T17:50:34.596100+0000","last_active":"2024-09-13T17:50:39.195748+0000","last_peered":"2024-09-13T17:50:39.195748+0000","last_clean":"2024-09-13T17:50:39.195748+0000","last_became_active":"2024-09-13T17:50:34.595798+0000","last_became_peered":"2024-09-13T17:50:34.595798+0000","last_unstale":"2024-09-13T17:50:39.195748+0000","last_undegraded":"2024-09-13T17:50:39.195748+0000","last_fullsized":"2024-09-13T17:50:39.195748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:49.543845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.194125+0000","last_change":"2024-09-13T17:50:34.584818+0000","last_active":"2024-09-13T17:50:39.194125+0000","last_peered":"2024-09-13T17:50:39.194125+0000","last_clean":"2024-09-13T17:50:39.194125+0000","last_became_active":"2024-09-13T17:50:34.584516+0000","last_became_peered":"2024-09-13T17:50:34.584516+0000","last_unstale":"2024-09-13T17:50:39.194125+0000","last_undegraded":"2024-09-13T17:50:39.194125+0000","last_fullsized":"2024-09-13T17:50:39.194125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:52.402024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281649+0000","last_change":"2024-09-13T17:50:34.598886+0000","last_active":"2024-09-13T17:51:15.281649+0000","last_peered":"2024-09-13T17:51:15.281649+0000","last_clean":"2024-09-13T17:51:15.281649+0000","last_became_active":"2024-09-13T17:50:34.598648+0000","last_became_peered":"2024-09-13T17:50:34.598648+0000","last_unstale":"2024-09-13T17:51:15.281649+0000","last_undegraded":"2024-09-13T17:51:15.281649+0000","last_fullsized":"2024-09-13T17:51:15.281649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:00:43.063223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282938+0000","last_change":"2024-09-13T17:50:34.582649+0000","last_active":"2024-09-13T17:51:15.282938+0000","last_peered":"2024-09-13T17:51:15.282938+0000","last_clean":"2024-09-13T17:51:15.282938+0000","last_became_active":"2024-09-13T17:50:34.581699+0000","last_became_peered":"2024-09-13T17:50:34.581699+0000","last_unstale":"2024-09-13T17:51:15.282938+0000","last_undegraded":"2024-09-13T17:51:15.282938+0000","last_fullsized":"2024-09-13T17:51:15.282938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:18.959303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270646+0000","last_change":"2024-09-13T17:50:34.587484+0000","last_active":"2024-09-13T17:51:15.270646+0000","last_peered":"2024-09-13T17:51:15.270646+0000","last_clean":"2024-09-13T17:51:15.270646+0000","last_became_active":"2024-09-13T17:50:34.587217+0000","last_became_peered":"2024-09-13T17:50:34.587217+0000","last_unstale":"2024-09-13T17:51:15.270646+0000","last_undegraded":"2024-09-13T17:51:15.270646+0000","last_fullsized":"2024-09-13T17:51:15.270646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:24:19.172607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.502178+0000","last_change":"2024-09-13T17:50:34.592871+0000","last_active":"2024-09-13T17:50:39.502178+0000","last_peered":"2024-09-13T17:50:39.502178+0000","last_clean":"2024-09-13T17:50:39.502178+0000","last_became_active":"2024-09-13T17:50:34.588513+0000","last_became_peered":"2024-09-13T17:50:34.588513+0000","last_unstale":"2024-09-13T17:50:39.502178+0000","last_undegraded":"2024-09-13T17:50:39.502178+0000","last_fullsized":"2024-09-13T17:50:39.502178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:33.131420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.983592+0000","last_change":"2024-09-13T17:50:34.585713+0000","last_active":"2024-09-13T17:50:38.983592+0000","last_peered":"2024-09-13T17:50:38.983592+0000","last_clean":"2024-09-13T17:50:38.983592+0000","last_became_active":"2024-09-13T17:50:34.585438+0000","last_became_peered":"2024-09-13T17:50:34.585438+0000","last_unstale":"2024-09-13T17:50:38.983592+0000","last_undegraded":"2024-09-13T17:50:38.983592+0000","last_fullsized":"2024-09-13T17:50:38.983592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:56:49.905393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.983306+0000","last_change":"2024-09-13T17:50:34.589628+0000","last_active":"2024-09-13T17:50:38.983306+0000","last_peered":"2024-09-13T17:50:38.983306+0000","last_clean":"2024-09-13T17:50:38.983306+0000","last_became_active":"2024-09-13T17:50:34.587665+0000","last_became_peered":"2024-09-13T17:50:34.587665+0000","last_unstale":"2024-09-13T17:50:38.983306+0000","last_undegraded":"2024-09-13T17:50:38.983306+0000","last_fullsized":"2024-09-13T17:50:38.983306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:15.518264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.985649+0000","last_change":"2024-09-13T17:50:34.591776+0000","last_active":"2024-09-13T17:50:38.985649+0000","last_peered":"2024-09-13T17:50:38.985649+0000","last_clean":"2024-09-13T17:50:38.985649+0000","last_became_active":"2024-09-13T17:50:34.591496+0000","last_became_peered":"2024-09-13T17:50:34.591496+0000","last_unstale":"2024-09-13T17:50:38.985649+0000","last_undegraded":"2024-09-13T17:50:38.985649+0000","last_fullsized":"2024-09-13T17:50:38.985649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:48.651999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.283047+0000","last_change":"2024-09-13T17:51:16.282372+0000","last_active":"2024-09-13T17:51:16.283047+0000","last_peered":"2024-09-13T17:51:16.283047+0000","last_clean":"2024-09-13T17:51:16.283047+0000","last_became_active":"2024-09-13T17:51:16.282065+0000","last_became_peered":"2024-09-13T17:51:16.282065+0000","last_unstale":"2024-09-13T17:51:16.283047+0000","last_undegraded":"2024-09-13T17:51:16.283047+0000","last_fullsized":"2024-09-13T17:51:16.283047+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:49.291224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.984603+0000","last_change":"2024-09-13T17:50:34.585531+0000","last_active":"2024-09-13T17:50:38.984603+0000","last_peered":"2024-09-13T17:50:38.984603+0000","last_clean":"2024-09-13T17:50:38.984603+0000","last_became_active":"2024-09-13T17:50:34.585226+0000","last_became_peered":"2024-09-13T17:50:34.585226+0000","last_unstale":"2024-09-13T17:50:38.984603+0000","last_undegraded":"2024-09-13T17:50:38.984603+0000","last_fullsized":"2024-09-13T17:50:38.984603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:17.105028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.194517+0000","last_change":"2024-09-13T17:50:34.586750+0000","last_active":"2024-09-13T17:50:39.194517+0000","last_peered":"2024-09-13T17:50:39.194517+0000","last_clean":"2024-09-13T17:50:39.194517+0000","last_became_active":"2024-09-13T17:50:34.584564+0000","last_became_peered":"2024-09-13T17:50:34.584564+0000","last_unstale":"2024-09-13T17:50:39.194517+0000","last_undegraded":"2024-09-13T17:50:39.194517+0000","last_fullsized":"2024-09-13T17:50:39.194517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:54.033257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.984499+0000","last_change":"2024-09-13T17:50:34.585938+0000","last_active":"2024-09-13T17:50:38.984499+0000","last_peered":"2024-09-13T17:50:38.984499+0000","last_clean":"2024-09-13T17:50:38.984499+0000","last_became_active":"2024-09-13T17:50:34.585584+0000","last_became_peered":"2024-09-13T17:50:34.585584+0000","last_unstale":"2024-09-13T17:50:38.984499+0000","last_undegraded":"2024-09-13T17:50:38.984499+0000","last_fullsized":"2024-09-13T17:50:38.984499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:32.623468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:39.194917+0000","last_change":"2024-09-13T17:50:34.588762+0000","last_active":"2024-09-13T17:50:39.194917+0000","last_peered":"2024-09-13T17:50:39.194917+0000","last_clean":"2024-09-13T17:50:39.194917+0000","last_became_active":"2024-09-13T17:50:34.588489+0000","last_became_peered":"2024-09-13T17:50:34.588489+0000","last_unstale":"2024-09-13T17:50:39.194917+0000","last_undegraded":"2024-09-13T17:50:39.194917+0000","last_fullsized":"2024-09-13T17:50:39.194917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:38.266297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277059+0000","last_change":"2024-09-13T17:50:34.583889+0000","last_active":"2024-09-13T17:51:15.277059+0000","last_peered":"2024-09-13T17:51:15.277059+0000","last_clean":"2024-09-13T17:51:15.277059+0000","last_became_active":"2024-09-13T17:50:34.583551+0000","last_became_peered":"2024-09-13T17:50:34.583551+0000","last_unstale":"2024-09-13T17:51:15.277059+0000","last_undegraded":"2024-09-13T17:51:15.277059+0000","last_fullsized":"2024-09-13T17:51:15.277059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:19.888626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281343+0000","last_change":"2024-09-13T17:50:34.585617+0000","last_active":"2024-09-13T17:51:15.281343+0000","last_peered":"2024-09-13T17:51:15.281343+0000","last_clean":"2024-09-13T17:51:15.281343+0000","last_became_active":"2024-09-13T17:50:34.585340+0000","last_became_peered":"2024-09-13T17:50:34.585340+0000","last_unstale":"2024-09-13T17:51:15.281343+0000","last_undegraded":"2024-09-13T17:51:15.281343+0000","last_fullsized":"2024-09-13T17:51:15.281343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:18.331045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:50:38.985639+0000","last_change":"2024-09-13T17:50:34.592365+0000","last_active":"2024-09-13T17:50:38.985639+0000","last_peered":"2024-09-13T17:50:38.985639+0000","last_clean":"2024-09-13T17:50:38.985639+0000","last_became_active":"2024-09-13T17:50:34.592046+0000","last_became_peered":"2024-09-13T17:50:34.592046+0000","last_unstale":"2024-09-13T17:50:38.985639+0000","last_undegraded":"2024-09-13T17:50:38.985639+0000","last_fullsized":"2024-09-13T17:50:38.985639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:54.283683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7309,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":14618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509454,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":113480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4870000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]}]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":303104,"data_stored":116742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]}]},{"osd":1,"up_from":15,"seq":64424509454,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":109965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]}]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":892928,"data_stored":706952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5790000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4660000000000002}]}]},{"osd":7,"up_from":14,"seq":60129542157,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":872448,"data_stored":693591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.196}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.234}]}]},{"osd":6,"up_from":14,"seq":60129542157,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":278528,"data_stored":106380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.198}]}]},{"osd":5,"up_from":14,"seq":60129542157,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":290816,"data_stored":110399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542157,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":286720,"data_stored":109687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:51:23.017 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:51:23.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.126+0000 7fbbe6758700 1 Processor -- start 2024-09-13T17:51:23.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.126+0000 7fbbe6758700 1 -- start start 2024-09-13T17:51:23.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbe6758700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbbe0155f90 0x7fbbe00c7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:23.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbe6758700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe00c5180 con 0x7fbbe006e480 2024-09-13T17:51:23.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbe6758700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe0070fb0 con 0x7fbbe006e860 2024-09-13T17:51:23.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbe6758700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe0156410 con 0x7fbbe0155f90 2024-09-13T17:51:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbe5756700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbbe006e480 0x7fbbe014f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44532/0 (socket says 172.21.15.40:44532) 2024-09-13T17:51:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbe5756700 1 -- 172.21.15.40:0/854394074 learned_addr learned my addr 172.21.15.40:0/854394074 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbe5f57700 1 --2- 172.21.15.40:0/854394074 >> v2:172.21.15.43:3300/0 conn(0x7fbbe0155f90 0x7fbbe00c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbd7fff700 1 -- 172.21.15.40:0/854394074 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2083820040 0 0) 0x7fbbe00c5180 con 0x7fbbe006e480 2024-09-13T17:51:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbd7fff700 1 -- 172.21.15.40:0/854394074 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbcc0035d0 con 0x7fbbe006e480 2024-09-13T17:51:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbd7fff700 1 -- 172.21.15.40:0/854394074 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4226282471 0 0) 0x7fbbcc0035d0 con 0x7fbbe006e480 2024-09-13T17:51:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbd7fff700 1 -- 172.21.15.40:0/854394074 >> v2:172.21.15.43:3300/0 conn(0x7fbbe0155f90 msgr2=0x7fbbe00c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbd7fff700 1 --2- 172.21.15.40:0/854394074 >> v2:172.21.15.43:3300/0 conn(0x7fbbe0155f90 0x7fbbe00c7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbd7fff700 1 -- 172.21.15.40:0/854394074 >> v1:172.21.15.43:6789/0 conn(0x7fbbe006e860 legacy=0x7fbbe0155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbd7fff700 1 -- 172.21.15.40:0/854394074 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe01566f0 con 0x7fbbe006e480 2024-09-13T17:51:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbd7fff700 1 -- 172.21.15.40:0/854394074 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fbbdc001670 con 0x7fbbe006e480 2024-09-13T17:51:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbd7fff700 1 -- 172.21.15.40:0/854394074 >> v1:172.21.15.40:6789/0 conn(0x7fbbe006e480 legacy=0x7fbbe014f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbd7fff700 1 --2- 172.21.15.40:0/854394074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbcc003a20 0x7fbbcc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbd7fff700 1 --2- 172.21.15.40:0/854394074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbcc004870 0x7fbbcc004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbd7fff700 1 --2- 172.21.15.40:0/854394074 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbbcc005230 0x7fbbcc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbd7fff700 1 -- 172.21.15.40:0/854394074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe0156410 con 0x7fbbcc003a20 2024-09-13T17:51:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbd7fff700 1 -- 172.21.15.40:0/854394074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc00c0e0 con 0x7fbbcc004870 2024-09-13T17:51:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbd7fff700 1 -- 172.21.15.40:0/854394074 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc00c2a0 con 0x7fbbcc005230 2024-09-13T17:51:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbe5756700 1 --2- 172.21.15.40:0/854394074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbcc003a20 0x7fbbcc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbe4f55700 1 --2- 172.21.15.40:0/854394074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbcc004870 0x7fbbcc004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbe5f57700 1 --2- 172.21.15.40:0/854394074 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbbcc005230 0x7fbbcc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbe5756700 1 -- 172.21.15.40:0/854394074 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbbcc005230 msgr2=0x7fbbcc00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbe5756700 1 --2- 172.21.15.40:0/854394074 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbbcc005230 0x7fbbcc00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbe5756700 1 -- 172.21.15.40:0/854394074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbcc004870 msgr2=0x7fbbcc004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbe5756700 1 --2- 172.21.15.40:0/854394074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbcc004870 0x7fbbcc004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbe5756700 1 -- 172.21.15.40:0/854394074 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbcc0035d0 con 0x7fbbcc003a20 2024-09-13T17:51:23.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbe5f57700 1 -- 172.21.15.40:0/854394074 reap_dead start 2024-09-13T17:51:23.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbe4f55700 1 --2- 172.21.15.40:0/854394074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbcc004870 0x7fbbcc004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:23.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbe5756700 1 --2- 172.21.15.40:0/854394074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbcc003a20 0x7fbbcc003e40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00c2a0 tx=0x7fbbcc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295d12941569c882 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:23.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbd7fff700 1 -- 172.21.15.40:0/854394074 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc007c30 con 0x7fbbcc003a20 2024-09-13T17:51:23.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbd7fff700 1 -- 172.21.15.40:0/854394074 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbdc00cd80 con 0x7fbbcc003a20 2024-09-13T17:51:23.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.130+0000 7fbbd7fff700 1 -- 172.21.15.40:0/854394074 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc005bc0 con 0x7fbbcc003a20 2024-09-13T17:51:23.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.134+0000 7fbbe6758700 1 -- 172.21.15.40:0/854394074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbcc003a20 msgr2=0x7fbbcc003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:23.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.134+0000 7fbbe6758700 1 --2- 172.21.15.40:0/854394074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbcc003a20 0x7fbbcc003e40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00c2a0 tx=0x7fbbcc0058a0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.134+0000 7fbbe6758700 1 -- 172.21.15.40:0/854394074 shutdown_connections 2024-09-13T17:51:23.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.134+0000 7fbbe6758700 1 --2- 172.21.15.40:0/854394074 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbcc003a20 0x7fbbcc003e40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.134+0000 7fbbe6758700 1 -- 172.21.15.40:0/854394074 >> 172.21.15.40:0/854394074 conn(0x7fbbe00c7d70 msgr2=0x7fbbe00c8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:23.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.134+0000 7fbbe6758700 1 -- 172.21.15.40:0/854394074 shutdown_connections 2024-09-13T17:51:23.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.134+0000 7fbbe6758700 1 -- 172.21.15.40:0/854394074 wait complete. 2024-09-13T17:51:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.134+0000 7fbbe6758700 1 Processor -- start 2024-09-13T17:51:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.134+0000 7fbbe6758700 1 -- start start 2024-09-13T17:51:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.134+0000 7fbbe6758700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbcc003a20 0x7fbbe0144200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.134+0000 7fbbe6758700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbe0144740 0x7fbbe014cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.134+0000 7fbbe6758700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbbe014d170 0x7fbbe01f2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.134+0000 7fbbe6758700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe014e870 con 0x7fbbcc003a20 2024-09-13T17:51:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.134+0000 7fbbe6758700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe014f110 con 0x7fbbe0144740 2024-09-13T17:51:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.134+0000 7fbbe6758700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe014f820 con 0x7fbbe014d170 2024-09-13T17:51:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.138+0000 7fbbe5756700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbcc003a20 0x7fbbe0144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.138+0000 7fbbe5756700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbcc003a20 0x7fbbe0144200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55110/0 (socket says 172.21.15.40:55110) 2024-09-13T17:51:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.138+0000 7fbbe5756700 1 -- 172.21.15.40:0/2121332785 learned_addr learned my addr 172.21.15.40:0/2121332785 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:23.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.138+0000 7fbbe4f55700 1 --2- 172.21.15.40:0/2121332785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbe0144740 0x7fbbe014cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:23.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.138+0000 7fbbe5f57700 1 --2- 172.21.15.40:0/2121332785 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbbe014d170 0x7fbbe01f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:23.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.138+0000 7fbbe5756700 1 -- 172.21.15.40:0/2121332785 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbbe014d170 msgr2=0x7fbbe01f2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:23.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.138+0000 7fbbe5756700 1 --2- 172.21.15.40:0/2121332785 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbbe014d170 0x7fbbe01f2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.138+0000 7fbbe5756700 1 -- 172.21.15.40:0/2121332785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbe0144740 msgr2=0x7fbbe014cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:23.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.138+0000 7fbbe5756700 1 --2- 172.21.15.40:0/2121332785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbe0144740 0x7fbbe014cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.138+0000 7fbbe5756700 1 -- 172.21.15.40:0/2121332785 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe014fae0 con 0x7fbbcc003a20 2024-09-13T17:51:23.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.138+0000 7fbbe5756700 1 --2- 172.21.15.40:0/2121332785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbcc003a20 0x7fbbe0144200 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00c890 tx=0x7fbbe0070f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:23.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.138+0000 7fbbd5ffb700 1 -- 172.21.15.40:0/2121332785 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc004ca0 con 0x7fbbcc003a20 2024-09-13T17:51:23.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.138+0000 7fbbd5ffb700 1 -- 172.21.15.40:0/2121332785 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbdc004e00 con 0x7fbbcc003a20 2024-09-13T17:51:23.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.138+0000 7fbbe6758700 1 -- 172.21.15.40:0/2121332785 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe014fdc0 con 0x7fbbcc003a20 2024-09-13T17:51:23.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.138+0000 7fbbd5ffb700 1 -- 172.21.15.40:0/2121332785 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc007310 con 0x7fbbcc003a20 2024-09-13T17:51:23.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.138+0000 7fbbe6758700 1 -- 172.21.15.40:0/2121332785 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe014eda0 con 0x7fbbcc003a20 2024-09-13T17:51:23.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.138+0000 7fbbe6758700 1 -- 172.21.15.40:0/2121332785 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe014ef60 con 0x7fbbcc003a20 2024-09-13T17:51:23.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.138+0000 7fbbd5ffb700 1 -- 172.21.15.40:0/2121332785 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbbdc007470 con 0x7fbbcc003a20 2024-09-13T17:51:23.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.142+0000 7fbbd5ffb700 1 --2- 172.21.15.40:0/2121332785 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbbc4066760 0x7fbbc4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:23.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.142+0000 7fbbd5ffb700 1 -- 172.21.15.40:0/2121332785 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbbdc085590 con 0x7fbbcc003a20 2024-09-13T17:51:23.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.142+0000 7fbbe4f55700 1 --2- 172.21.15.40:0/2121332785 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbbc4066760 0x7fbbc4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:23.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.142+0000 7fbbe4f55700 1 --2- 172.21.15.40:0/2121332785 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbbc4066760 0x7fbbc4068c20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0007fd0 tx=0x7fbbd0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:23.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.150+0000 7fbbd5ffb700 1 -- 172.21.15.40:0/2121332785 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbbdc0526f0 con 0x7fbbcc003a20 2024-09-13T17:51:23.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.314+0000 7fbbe6758700 1 -- 172.21.15.40:0/2121332785 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbbe0070c40 con 0x7fbbc4066760 2024-09-13T17:51:23.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.326+0000 7fbbd5ffb700 1 -- 172.21.15.40:0/2121332785 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+349592 (secure 0 0 0) 0x7fbbd0006880 con 0x7fbbc4066760 2024-09-13T17:51:23.335 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:23.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.330+0000 7fbbe6758700 1 -- 172.21.15.40:0/2121332785 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbbc4066760 msgr2=0x7fbbc4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:23.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.330+0000 7fbbe6758700 1 --2- 172.21.15.40:0/2121332785 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbbc4066760 0x7fbbc4068c20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0007fd0 tx=0x7fbbd0009040 comp rx=0 tx=0).stop 2024-09-13T17:51:23.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.330+0000 7fbbe6758700 1 -- 172.21.15.40:0/2121332785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbcc003a20 msgr2=0x7fbbe0144200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:23.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.330+0000 7fbbe6758700 1 --2- 172.21.15.40:0/2121332785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbcc003a20 0x7fbbe0144200 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00c890 tx=0x7fbbe0070f20 comp rx=0 tx=0).stop 2024-09-13T17:51:23.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.330+0000 7fbbe6758700 1 -- 172.21.15.40:0/2121332785 shutdown_connections 2024-09-13T17:51:23.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.330+0000 7fbbe6758700 1 --2- 172.21.15.40:0/2121332785 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbbc4066760 0x7fbbc4068c20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.330+0000 7fbbe6758700 1 --2- 172.21.15.40:0/2121332785 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbbe014d170 0x7fbbe01f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.330+0000 7fbbe6758700 1 --2- 172.21.15.40:0/2121332785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbbe0144740 0x7fbbe014cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.330+0000 7fbbe6758700 1 --2- 172.21.15.40:0/2121332785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbbcc003a20 0x7fbbe0144200 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.330+0000 7fbbe6758700 1 -- 172.21.15.40:0/2121332785 >> 172.21.15.40:0/2121332785 conn(0x7fbbe00c7d70 msgr2=0x7fbbe006d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:23.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.330+0000 7fbbe6758700 1 -- 172.21.15.40:0/2121332785 shutdown_connections 2024-09-13T17:51:23.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.330+0000 7fbbe6758700 1 -- 172.21.15.40:0/2121332785 wait complete. 2024-09-13T17:51:23.338 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-13T17:51:23.349 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2024-09-13T17:51:23.013833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598119,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":288,"num_write_kb":2459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236076,"kb_used_data":3528,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738644,"statfs":{"total":773094113280,"available":772852371456,"internally_reserved":0,"allocated":3612672,"data_stored":2123630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1016,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002740"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282601+0000","last_change":"2024-09-13T17:50:34.212158+0000","last_active":"2024-09-13T17:51:15.282601+0000","last_peered":"2024-09-13T17:51:15.282601+0000","last_clean":"2024-09-13T17:51:15.282601+0000","last_became_active":"2024-09-13T17:50:34.211867+0000","last_became_peered":"2024-09-13T17:50:34.211867+0000","last_unstale":"2024-09-13T17:51:15.282601+0000","last_undegraded":"2024-09-13T17:51:15.282601+0000","last_fullsized":"2024-09-13T17:51:15.282601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:14.671095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271433+0000","last_change":"2024-09-13T17:50:34.587396+0000","last_active":"2024-09-13T17:51:15.271433+0000","last_peered":"2024-09-13T17:51:15.271433+0000","last_clean":"2024-09-13T17:51:15.271433+0000","last_became_active":"2024-09-13T17:50:34.587108+0000","last_became_peered":"2024-09-13T17:50:34.587108+0000","last_unstale":"2024-09-13T17:51:15.271433+0000","last_undegraded":"2024-09-13T17:51:15.271433+0000","last_fullsized":"2024-09-13T17:51:15.271433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:39:16.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270087+0000","last_change":"2024-09-13T17:50:38.615059+0000","last_active":"2024-09-13T17:51:15.270087+0000","last_peered":"2024-09-13T17:51:15.270087+0000","last_clean":"2024-09-13T17:51:15.270087+0000","last_became_active":"2024-09-13T17:50:38.614723+0000","last_became_peered":"2024-09-13T17:50:38.614723+0000","last_unstale":"2024-09-13T17:51:15.270087+0000","last_undegraded":"2024-09-13T17:51:15.270087+0000","last_fullsized":"2024-09-13T17:51:15.270087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:23:01.902181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271688+0000","last_change":"2024-09-13T17:50:34.204760+0000","last_active":"2024-09-13T17:51:15.271688+0000","last_peered":"2024-09-13T17:51:15.271688+0000","last_clean":"2024-09-13T17:51:15.271688+0000","last_became_active":"2024-09-13T17:50:34.202930+0000","last_became_peered":"2024-09-13T17:50:34.202930+0000","last_unstale":"2024-09-13T17:51:15.271688+0000","last_undegraded":"2024-09-13T17:51:15.271688+0000","last_fullsized":"2024-09-13T17:51:15.271688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:59.784701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.366352+0000","last_change":"2024-09-13T17:51:16.366352+0000","last_active":"2024-09-13T17:51:16.366352+0000","last_peered":"2024-09-13T17:51:16.366352+0000","last_clean":"2024-09-13T17:51:16.366352+0000","last_became_active":"2024-09-13T17:51:16.365492+0000","last_became_peered":"2024-09-13T17:51:16.365492+0000","last_unstale":"2024-09-13T17:51:16.366352+0000","last_undegraded":"2024-09-13T17:51:16.366352+0000","last_fullsized":"2024-09-13T17:51:16.366352+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:42.192182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270184+0000","last_change":"2024-09-13T17:50:38.618751+0000","last_active":"2024-09-13T17:51:15.270184+0000","last_peered":"2024-09-13T17:51:15.270184+0000","last_clean":"2024-09-13T17:51:15.270184+0000","last_became_active":"2024-09-13T17:50:38.618423+0000","last_became_peered":"2024-09-13T17:50:38.618423+0000","last_unstale":"2024-09-13T17:51:15.270184+0000","last_undegraded":"2024-09-13T17:51:15.270184+0000","last_fullsized":"2024-09-13T17:51:15.270184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:50:02.418785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"25'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279312+0000","last_change":"2024-09-13T17:50:34.214972+0000","last_active":"2024-09-13T17:51:15.279312+0000","last_peered":"2024-09-13T17:51:15.279312+0000","last_clean":"2024-09-13T17:51:15.279312+0000","last_became_active":"2024-09-13T17:50:34.214652+0000","last_became_peered":"2024-09-13T17:50:34.214652+0000","last_unstale":"2024-09-13T17:51:15.279312+0000","last_undegraded":"2024-09-13T17:51:15.279312+0000","last_fullsized":"2024-09-13T17:51:15.279312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:57.176854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.282672+0000","last_change":"2024-09-13T17:51:16.281831+0000","last_active":"2024-09-13T17:51:16.282672+0000","last_peered":"2024-09-13T17:51:16.282672+0000","last_clean":"2024-09-13T17:51:16.282672+0000","last_became_active":"2024-09-13T17:51:16.281549+0000","last_became_peered":"2024-09-13T17:51:16.281549+0000","last_unstale":"2024-09-13T17:51:16.282672+0000","last_undegraded":"2024-09-13T17:51:16.282672+0000","last_fullsized":"2024-09-13T17:51:16.282672+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:43.894437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.274476+0000","last_change":"2024-09-13T17:50:38.616219+0000","last_active":"2024-09-13T17:51:15.274476+0000","last_peered":"2024-09-13T17:51:15.274476+0000","last_clean":"2024-09-13T17:51:15.274476+0000","last_became_active":"2024-09-13T17:50:38.614329+0000","last_became_peered":"2024-09-13T17:50:38.614329+0000","last_unstale":"2024-09-13T17:51:15.274476+0000","last_undegraded":"2024-09-13T17:51:15.274476+0000","last_fullsized":"2024-09-13T17:51:15.274476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:25.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"25'2","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279079+0000","last_change":"2024-09-13T17:50:34.216123+0000","last_active":"2024-09-13T17:51:15.279079+0000","last_peered":"2024-09-13T17:51:15.279079+0000","last_clean":"2024-09-13T17:51:15.279079+0000","last_became_active":"2024-09-13T17:50:34.215661+0000","last_became_peered":"2024-09-13T17:50:34.215661+0000","last_unstale":"2024-09-13T17:51:15.279079+0000","last_undegraded":"2024-09-13T17:51:15.279079+0000","last_fullsized":"2024-09-13T17:51:15.279079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:20.895037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270446+0000","last_change":"2024-09-13T17:50:34.599673+0000","last_active":"2024-09-13T17:51:15.270446+0000","last_peered":"2024-09-13T17:51:15.270446+0000","last_clean":"2024-09-13T17:51:15.270446+0000","last_became_active":"2024-09-13T17:50:34.599409+0000","last_became_peered":"2024-09-13T17:50:34.599409+0000","last_unstale":"2024-09-13T17:51:15.270446+0000","last_undegraded":"2024-09-13T17:51:15.270446+0000","last_fullsized":"2024-09-13T17:51:15.270446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:03:31.394385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.274077+0000","last_change":"2024-09-13T17:50:38.616317+0000","last_active":"2024-09-13T17:51:15.274077+0000","last_peered":"2024-09-13T17:51:15.274077+0000","last_clean":"2024-09-13T17:51:15.274077+0000","last_became_active":"2024-09-13T17:50:38.614311+0000","last_became_peered":"2024-09-13T17:50:38.614311+0000","last_unstale":"2024-09-13T17:51:15.274077+0000","last_undegraded":"2024-09-13T17:51:15.274077+0000","last_fullsized":"2024-09-13T17:51:15.274077+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:29.221522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275452+0000","last_change":"2024-09-13T17:50:34.599490+0000","last_active":"2024-09-13T17:51:15.275452+0000","last_peered":"2024-09-13T17:51:15.275452+0000","last_clean":"2024-09-13T17:51:15.275452+0000","last_became_active":"2024-09-13T17:50:34.599207+0000","last_became_peered":"2024-09-13T17:50:34.599207+0000","last_unstale":"2024-09-13T17:51:15.275452+0000","last_undegraded":"2024-09-13T17:51:15.275452+0000","last_fullsized":"2024-09-13T17:51:15.275452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:11.178161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278915+0000","last_change":"2024-09-13T17:50:34.215697+0000","last_active":"2024-09-13T17:51:15.278915+0000","last_peered":"2024-09-13T17:51:15.278915+0000","last_clean":"2024-09-13T17:51:15.278915+0000","last_became_active":"2024-09-13T17:50:34.215409+0000","last_became_peered":"2024-09-13T17:50:34.215409+0000","last_unstale":"2024-09-13T17:51:15.278915+0000","last_undegraded":"2024-09-13T17:51:15.278915+0000","last_fullsized":"2024-09-13T17:51:15.278915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:21.609567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275240+0000","last_change":"2024-09-13T17:50:38.611817+0000","last_active":"2024-09-13T17:51:15.275240+0000","last_peered":"2024-09-13T17:51:15.275240+0000","last_clean":"2024-09-13T17:51:15.275240+0000","last_became_active":"2024-09-13T17:50:38.611263+0000","last_became_peered":"2024-09-13T17:50:38.611263+0000","last_unstale":"2024-09-13T17:51:15.275240+0000","last_undegraded":"2024-09-13T17:51:15.275240+0000","last_fullsized":"2024-09-13T17:51:15.275240+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:56:47.466672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271037+0000","last_change":"2024-09-13T17:50:34.586307+0000","last_active":"2024-09-13T17:51:15.271037+0000","last_peered":"2024-09-13T17:51:15.271037+0000","last_clean":"2024-09-13T17:51:15.271037+0000","last_became_active":"2024-09-13T17:50:34.585831+0000","last_became_peered":"2024-09-13T17:50:34.585831+0000","last_unstale":"2024-09-13T17:51:15.271037+0000","last_undegraded":"2024-09-13T17:51:15.271037+0000","last_fullsized":"2024-09-13T17:51:15.271037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:40.912352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.280577+0000","last_change":"2024-09-13T17:50:34.211850+0000","last_active":"2024-09-13T17:51:15.280577+0000","last_peered":"2024-09-13T17:51:15.280577+0000","last_clean":"2024-09-13T17:51:15.280577+0000","last_became_active":"2024-09-13T17:50:34.211594+0000","last_became_peered":"2024-09-13T17:50:34.211594+0000","last_unstale":"2024-09-13T17:51:15.280577+0000","last_undegraded":"2024-09-13T17:51:15.280577+0000","last_fullsized":"2024-09-13T17:51:15.280577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:35:02.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276065+0000","last_change":"2024-09-13T17:50:38.613576+0000","last_active":"2024-09-13T17:51:15.276065+0000","last_peered":"2024-09-13T17:51:15.276065+0000","last_clean":"2024-09-13T17:51:15.276065+0000","last_became_active":"2024-09-13T17:50:38.612577+0000","last_became_peered":"2024-09-13T17:50:38.612577+0000","last_unstale":"2024-09-13T17:51:15.276065+0000","last_undegraded":"2024-09-13T17:51:15.276065+0000","last_fullsized":"2024-09-13T17:51:15.276065+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:25.551060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272679+0000","last_change":"2024-09-13T17:50:34.587273+0000","last_active":"2024-09-13T17:51:15.272679+0000","last_peered":"2024-09-13T17:51:15.272679+0000","last_clean":"2024-09-13T17:51:15.272679+0000","last_became_active":"2024-09-13T17:50:34.587027+0000","last_became_peered":"2024-09-13T17:50:34.587027+0000","last_unstale":"2024-09-13T17:51:15.272679+0000","last_undegraded":"2024-09-13T17:51:15.272679+0000","last_fullsized":"2024-09-13T17:51:15.272679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:58.826737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.280692+0000","last_change":"2024-09-13T17:50:34.211845+0000","last_active":"2024-09-13T17:51:15.280692+0000","last_peered":"2024-09-13T17:51:15.280692+0000","last_clean":"2024-09-13T17:51:15.280692+0000","last_became_active":"2024-09-13T17:50:34.211568+0000","last_became_peered":"2024-09-13T17:50:34.211568+0000","last_unstale":"2024-09-13T17:51:15.280692+0000","last_undegraded":"2024-09-13T17:51:15.280692+0000","last_fullsized":"2024-09-13T17:51:15.280692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:56.348984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281699+0000","last_change":"2024-09-13T17:50:38.610292+0000","last_active":"2024-09-13T17:51:15.281699+0000","last_peered":"2024-09-13T17:51:15.281699+0000","last_clean":"2024-09-13T17:51:15.281699+0000","last_became_active":"2024-09-13T17:50:38.609832+0000","last_became_peered":"2024-09-13T17:50:38.609832+0000","last_unstale":"2024-09-13T17:51:15.281699+0000","last_undegraded":"2024-09-13T17:51:15.281699+0000","last_fullsized":"2024-09-13T17:51:15.281699+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:13.783068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.273946+0000","last_change":"2024-09-13T17:50:34.593566+0000","last_active":"2024-09-13T17:51:15.273946+0000","last_peered":"2024-09-13T17:51:15.273946+0000","last_clean":"2024-09-13T17:51:15.273946+0000","last_became_active":"2024-09-13T17:50:34.588794+0000","last_became_peered":"2024-09-13T17:50:34.588794+0000","last_unstale":"2024-09-13T17:51:15.273946+0000","last_undegraded":"2024-09-13T17:51:15.273946+0000","last_fullsized":"2024-09-13T17:51:15.273946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:13.192453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277322+0000","last_change":"2024-09-13T17:50:34.214638+0000","last_active":"2024-09-13T17:51:15.277322+0000","last_peered":"2024-09-13T17:51:15.277322+0000","last_clean":"2024-09-13T17:51:15.277322+0000","last_became_active":"2024-09-13T17:50:34.214292+0000","last_became_peered":"2024-09-13T17:50:34.214292+0000","last_unstale":"2024-09-13T17:51:15.277322+0000","last_undegraded":"2024-09-13T17:51:15.277322+0000","last_fullsized":"2024-09-13T17:51:15.277322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:57.377104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275146+0000","last_change":"2024-09-13T17:50:38.613368+0000","last_active":"2024-09-13T17:51:15.275146+0000","last_peered":"2024-09-13T17:51:15.275146+0000","last_clean":"2024-09-13T17:51:15.275146+0000","last_became_active":"2024-09-13T17:50:38.613010+0000","last_became_peered":"2024-09-13T17:50:38.613010+0000","last_unstale":"2024-09-13T17:51:15.275146+0000","last_undegraded":"2024-09-13T17:51:15.275146+0000","last_fullsized":"2024-09-13T17:51:15.275146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:42.706625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282530+0000","last_change":"2024-09-13T17:50:34.582327+0000","last_active":"2024-09-13T17:51:15.282530+0000","last_peered":"2024-09-13T17:51:15.282530+0000","last_clean":"2024-09-13T17:51:15.282530+0000","last_became_active":"2024-09-13T17:50:34.582050+0000","last_became_peered":"2024-09-13T17:50:34.582050+0000","last_unstale":"2024-09-13T17:51:15.282530+0000","last_undegraded":"2024-09-13T17:51:15.282530+0000","last_fullsized":"2024-09-13T17:51:15.282530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:52.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277006+0000","last_change":"2024-09-13T17:50:34.211512+0000","last_active":"2024-09-13T17:51:15.277006+0000","last_peered":"2024-09-13T17:51:15.277006+0000","last_clean":"2024-09-13T17:51:15.277006+0000","last_became_active":"2024-09-13T17:50:34.211230+0000","last_became_peered":"2024-09-13T17:50:34.211230+0000","last_unstale":"2024-09-13T17:51:15.277006+0000","last_undegraded":"2024-09-13T17:51:15.277006+0000","last_fullsized":"2024-09-13T17:51:15.277006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:40.707777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272462+0000","last_change":"2024-09-13T17:50:38.608255+0000","last_active":"2024-09-13T17:51:15.272462+0000","last_peered":"2024-09-13T17:51:15.272462+0000","last_clean":"2024-09-13T17:51:15.272462+0000","last_became_active":"2024-09-13T17:50:38.606697+0000","last_became_peered":"2024-09-13T17:50:38.606697+0000","last_unstale":"2024-09-13T17:51:15.272462+0000","last_undegraded":"2024-09-13T17:51:15.272462+0000","last_fullsized":"2024-09-13T17:51:15.272462+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:47.265007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272916+0000","last_change":"2024-09-13T17:50:34.594476+0000","last_active":"2024-09-13T17:51:15.272916+0000","last_peered":"2024-09-13T17:51:15.272916+0000","last_clean":"2024-09-13T17:51:15.272916+0000","last_became_active":"2024-09-13T17:50:34.594215+0000","last_became_peered":"2024-09-13T17:50:34.594215+0000","last_unstale":"2024-09-13T17:51:15.272916+0000","last_undegraded":"2024-09-13T17:51:15.272916+0000","last_fullsized":"2024-09-13T17:51:15.272916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:02:27.528019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281479+0000","last_change":"2024-09-13T17:50:34.204018+0000","last_active":"2024-09-13T17:51:15.281479+0000","last_peered":"2024-09-13T17:51:15.281479+0000","last_clean":"2024-09-13T17:51:15.281479+0000","last_became_active":"2024-09-13T17:50:34.203702+0000","last_became_peered":"2024-09-13T17:50:34.203702+0000","last_unstale":"2024-09-13T17:51:15.281479+0000","last_undegraded":"2024-09-13T17:51:15.281479+0000","last_fullsized":"2024-09-13T17:51:15.281479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:44.431226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.365782+0000","last_change":"2024-09-13T17:51:16.365782+0000","last_active":"2024-09-13T17:51:16.365782+0000","last_peered":"2024-09-13T17:51:16.365782+0000","last_clean":"2024-09-13T17:51:16.365782+0000","last_became_active":"2024-09-13T17:51:16.364540+0000","last_became_peered":"2024-09-13T17:51:16.364540+0000","last_unstale":"2024-09-13T17:51:16.365782+0000","last_undegraded":"2024-09-13T17:51:16.365782+0000","last_fullsized":"2024-09-13T17:51:16.365782+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:18.753557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281218+0000","last_change":"2024-09-13T17:50:34.586802+0000","last_active":"2024-09-13T17:51:15.281218+0000","last_peered":"2024-09-13T17:51:15.281218+0000","last_clean":"2024-09-13T17:51:15.281218+0000","last_became_active":"2024-09-13T17:50:34.586552+0000","last_became_peered":"2024-09-13T17:50:34.586552+0000","last_unstale":"2024-09-13T17:51:15.281218+0000","last_undegraded":"2024-09-13T17:51:15.281218+0000","last_fullsized":"2024-09-13T17:51:15.281218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:43.383975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279434+0000","last_change":"2024-09-13T17:50:34.217083+0000","last_active":"2024-09-13T17:51:15.279434+0000","last_peered":"2024-09-13T17:51:15.279434+0000","last_clean":"2024-09-13T17:51:15.279434+0000","last_became_active":"2024-09-13T17:50:34.216758+0000","last_became_peered":"2024-09-13T17:50:34.216758+0000","last_unstale":"2024-09-13T17:51:15.279434+0000","last_undegraded":"2024-09-13T17:51:15.279434+0000","last_fullsized":"2024-09-13T17:51:15.279434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:36.473942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277926+0000","last_change":"2024-09-13T17:50:38.614667+0000","last_active":"2024-09-13T17:51:15.277926+0000","last_peered":"2024-09-13T17:51:15.277926+0000","last_clean":"2024-09-13T17:51:15.277926+0000","last_became_active":"2024-09-13T17:50:38.614284+0000","last_became_peered":"2024-09-13T17:50:38.614284+0000","last_unstale":"2024-09-13T17:51:15.277926+0000","last_undegraded":"2024-09-13T17:51:15.277926+0000","last_fullsized":"2024-09-13T17:51:15.277926+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:29.616794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270498+0000","last_change":"2024-09-13T17:50:34.588603+0000","last_active":"2024-09-13T17:51:15.270498+0000","last_peered":"2024-09-13T17:51:15.270498+0000","last_clean":"2024-09-13T17:51:15.270498+0000","last_became_active":"2024-09-13T17:50:34.588317+0000","last_became_peered":"2024-09-13T17:50:34.588317+0000","last_unstale":"2024-09-13T17:51:15.270498+0000","last_undegraded":"2024-09-13T17:51:15.270498+0000","last_fullsized":"2024-09-13T17:51:15.270498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:03.743130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281302+0000","last_change":"2024-09-13T17:50:34.204335+0000","last_active":"2024-09-13T17:51:15.281302+0000","last_peered":"2024-09-13T17:51:15.281302+0000","last_clean":"2024-09-13T17:51:15.281302+0000","last_became_active":"2024-09-13T17:50:34.204060+0000","last_became_peered":"2024-09-13T17:50:34.204060+0000","last_unstale":"2024-09-13T17:51:15.281302+0000","last_undegraded":"2024-09-13T17:51:15.281302+0000","last_fullsized":"2024-09-13T17:51:15.281302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:54:41.831344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279638+0000","last_change":"2024-09-13T17:50:38.619590+0000","last_active":"2024-09-13T17:51:15.279638+0000","last_peered":"2024-09-13T17:51:15.279638+0000","last_clean":"2024-09-13T17:51:15.279638+0000","last_became_active":"2024-09-13T17:50:38.617257+0000","last_became_peered":"2024-09-13T17:50:38.617257+0000","last_unstale":"2024-09-13T17:51:15.279638+0000","last_undegraded":"2024-09-13T17:51:15.279638+0000","last_fullsized":"2024-09-13T17:51:15.279638+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:34.870291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275058+0000","last_change":"2024-09-13T17:50:34.588466+0000","last_active":"2024-09-13T17:51:15.275058+0000","last_peered":"2024-09-13T17:51:15.275058+0000","last_clean":"2024-09-13T17:51:15.275058+0000","last_became_active":"2024-09-13T17:50:34.588063+0000","last_became_peered":"2024-09-13T17:50:34.588063+0000","last_unstale":"2024-09-13T17:51:15.275058+0000","last_undegraded":"2024-09-13T17:51:15.275058+0000","last_fullsized":"2024-09-13T17:51:15.275058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:43:00.669203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270012+0000","last_change":"2024-09-13T17:50:34.209239+0000","last_active":"2024-09-13T17:51:15.270012+0000","last_peered":"2024-09-13T17:51:15.270012+0000","last_clean":"2024-09-13T17:51:15.270012+0000","last_became_active":"2024-09-13T17:50:34.208196+0000","last_became_peered":"2024-09-13T17:50:34.208196+0000","last_unstale":"2024-09-13T17:51:15.270012+0000","last_undegraded":"2024-09-13T17:51:15.270012+0000","last_fullsized":"2024-09-13T17:51:15.270012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:39.253917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281566+0000","last_change":"2024-09-13T17:50:38.611021+0000","last_active":"2024-09-13T17:51:15.281566+0000","last_peered":"2024-09-13T17:51:15.281566+0000","last_clean":"2024-09-13T17:51:15.281566+0000","last_became_active":"2024-09-13T17:50:38.609794+0000","last_became_peered":"2024-09-13T17:50:38.609794+0000","last_unstale":"2024-09-13T17:51:15.281566+0000","last_undegraded":"2024-09-13T17:51:15.281566+0000","last_fullsized":"2024-09-13T17:51:15.281566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:59.980236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281553+0000","last_change":"2024-09-13T17:50:34.584290+0000","last_active":"2024-09-13T17:51:15.281553+0000","last_peered":"2024-09-13T17:51:15.281553+0000","last_clean":"2024-09-13T17:51:15.281553+0000","last_became_active":"2024-09-13T17:50:34.584023+0000","last_became_peered":"2024-09-13T17:50:34.584023+0000","last_unstale":"2024-09-13T17:51:15.281553+0000","last_undegraded":"2024-09-13T17:51:15.281553+0000","last_fullsized":"2024-09-13T17:51:15.281553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:11.304986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"25'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278819+0000","last_change":"2024-09-13T17:50:34.198932+0000","last_active":"2024-09-13T17:51:15.278819+0000","last_peered":"2024-09-13T17:51:15.278819+0000","last_clean":"2024-09-13T17:51:15.278819+0000","last_became_active":"2024-09-13T17:50:34.198603+0000","last_became_peered":"2024-09-13T17:50:34.198603+0000","last_unstale":"2024-09-13T17:51:15.278819+0000","last_undegraded":"2024-09-13T17:51:15.278819+0000","last_fullsized":"2024-09-13T17:51:15.278819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:00.917126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281714+0000","last_change":"2024-09-13T17:50:38.611326+0000","last_active":"2024-09-13T17:51:15.281714+0000","last_peered":"2024-09-13T17:51:15.281714+0000","last_clean":"2024-09-13T17:51:15.281714+0000","last_became_active":"2024-09-13T17:50:38.610900+0000","last_became_peered":"2024-09-13T17:50:38.610900+0000","last_unstale":"2024-09-13T17:51:15.281714+0000","last_undegraded":"2024-09-13T17:51:15.281714+0000","last_fullsized":"2024-09-13T17:51:15.281714+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:17:09.994736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276990+0000","last_change":"2024-09-13T17:50:34.594482+0000","last_active":"2024-09-13T17:51:15.276990+0000","last_peered":"2024-09-13T17:51:15.276990+0000","last_clean":"2024-09-13T17:51:15.276990+0000","last_became_active":"2024-09-13T17:50:34.594201+0000","last_became_peered":"2024-09-13T17:50:34.594201+0000","last_unstale":"2024-09-13T17:51:15.276990+0000","last_undegraded":"2024-09-13T17:51:15.276990+0000","last_fullsized":"2024-09-13T17:51:15.276990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:06.697507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277494+0000","last_change":"2024-09-13T17:50:34.218376+0000","last_active":"2024-09-13T17:51:15.277494+0000","last_peered":"2024-09-13T17:51:15.277494+0000","last_clean":"2024-09-13T17:51:15.277494+0000","last_became_active":"2024-09-13T17:50:34.218069+0000","last_became_peered":"2024-09-13T17:50:34.218069+0000","last_unstale":"2024-09-13T17:51:15.277494+0000","last_undegraded":"2024-09-13T17:51:15.277494+0000","last_fullsized":"2024-09-13T17:51:15.277494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:53:54.288969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277801+0000","last_change":"2024-09-13T17:50:38.613206+0000","last_active":"2024-09-13T17:51:15.277801+0000","last_peered":"2024-09-13T17:51:15.277801+0000","last_clean":"2024-09-13T17:51:15.277801+0000","last_became_active":"2024-09-13T17:50:38.612856+0000","last_became_peered":"2024-09-13T17:50:38.612856+0000","last_unstale":"2024-09-13T17:51:15.277801+0000","last_undegraded":"2024-09-13T17:51:15.277801+0000","last_fullsized":"2024-09-13T17:51:15.277801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:31:20.190996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270767+0000","last_change":"2024-09-13T17:50:34.582931+0000","last_active":"2024-09-13T17:51:15.270767+0000","last_peered":"2024-09-13T17:51:15.270767+0000","last_clean":"2024-09-13T17:51:15.270767+0000","last_became_active":"2024-09-13T17:50:34.582446+0000","last_became_peered":"2024-09-13T17:50:34.582446+0000","last_unstale":"2024-09-13T17:51:15.270767+0000","last_undegraded":"2024-09-13T17:51:15.270767+0000","last_fullsized":"2024-09-13T17:51:15.270767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:30.554785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270225+0000","last_change":"2024-09-13T17:50:34.201237+0000","last_active":"2024-09-13T17:51:15.270225+0000","last_peered":"2024-09-13T17:51:15.270225+0000","last_clean":"2024-09-13T17:51:15.270225+0000","last_became_active":"2024-09-13T17:50:34.200873+0000","last_became_peered":"2024-09-13T17:50:34.200873+0000","last_unstale":"2024-09-13T17:51:15.270225+0000","last_undegraded":"2024-09-13T17:51:15.270225+0000","last_fullsized":"2024-09-13T17:51:15.270225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:02.711342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277589+0000","last_change":"2024-09-13T17:50:22.095367+0000","last_active":"2024-09-13T17:51:15.277589+0000","last_peered":"2024-09-13T17:51:15.277589+0000","last_clean":"2024-09-13T17:51:15.277589+0000","last_became_active":"2024-09-13T17:50:22.093734+0000","last_became_peered":"2024-09-13T17:50:22.093734+0000","last_unstale":"2024-09-13T17:51:15.277589+0000","last_undegraded":"2024-09-13T17:51:15.277589+0000","last_fullsized":"2024-09-13T17:51:15.277589+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_clean_scrub_stamp":"2024-09-13T17:50:21.057812+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:23.531613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276680+0000","last_change":"2024-09-13T17:50:38.618880+0000","last_active":"2024-09-13T17:51:15.276680+0000","last_peered":"2024-09-13T17:51:15.276680+0000","last_clean":"2024-09-13T17:51:15.276680+0000","last_became_active":"2024-09-13T17:50:38.618488+0000","last_became_peered":"2024-09-13T17:50:38.618488+0000","last_unstale":"2024-09-13T17:51:15.276680+0000","last_undegraded":"2024-09-13T17:51:15.276680+0000","last_fullsized":"2024-09-13T17:51:15.276680+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:03.431876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279141+0000","last_change":"2024-09-13T17:50:34.585881+0000","last_active":"2024-09-13T17:51:15.279141+0000","last_peered":"2024-09-13T17:51:15.279141+0000","last_clean":"2024-09-13T17:51:15.279141+0000","last_became_active":"2024-09-13T17:50:34.585166+0000","last_became_peered":"2024-09-13T17:50:34.585166+0000","last_unstale":"2024-09-13T17:51:15.279141+0000","last_undegraded":"2024-09-13T17:51:15.279141+0000","last_fullsized":"2024-09-13T17:51:15.279141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:06.654658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279549+0000","last_change":"2024-09-13T17:50:34.210934+0000","last_active":"2024-09-13T17:51:15.279549+0000","last_peered":"2024-09-13T17:51:15.279549+0000","last_clean":"2024-09-13T17:51:15.279549+0000","last_became_active":"2024-09-13T17:50:34.210589+0000","last_became_peered":"2024-09-13T17:50:34.210589+0000","last_unstale":"2024-09-13T17:51:15.279549+0000","last_undegraded":"2024-09-13T17:51:15.279549+0000","last_fullsized":"2024-09-13T17:51:15.279549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:26.213295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.364911+0000","last_change":"2024-09-13T17:51:16.364911+0000","last_active":"2024-09-13T17:51:16.364911+0000","last_peered":"2024-09-13T17:51:16.364911+0000","last_clean":"2024-09-13T17:51:16.364911+0000","last_became_active":"2024-09-13T17:51:16.363974+0000","last_became_peered":"2024-09-13T17:51:16.363974+0000","last_unstale":"2024-09-13T17:51:16.364911+0000","last_undegraded":"2024-09-13T17:51:16.364911+0000","last_fullsized":"2024-09-13T17:51:16.364911+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:12.730197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271986+0000","last_change":"2024-09-13T17:50:34.590169+0000","last_active":"2024-09-13T17:51:15.271986+0000","last_peered":"2024-09-13T17:51:15.271986+0000","last_clean":"2024-09-13T17:51:15.271986+0000","last_became_active":"2024-09-13T17:50:34.589434+0000","last_became_peered":"2024-09-13T17:50:34.589434+0000","last_unstale":"2024-09-13T17:51:15.271986+0000","last_undegraded":"2024-09-13T17:51:15.271986+0000","last_fullsized":"2024-09-13T17:51:15.271986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:12.545412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"25'1","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281689+0000","last_change":"2024-09-13T17:50:34.204269+0000","last_active":"2024-09-13T17:51:15.281689+0000","last_peered":"2024-09-13T17:51:15.281689+0000","last_clean":"2024-09-13T17:51:15.281689+0000","last_became_active":"2024-09-13T17:50:34.203874+0000","last_became_peered":"2024-09-13T17:50:34.203874+0000","last_unstale":"2024-09-13T17:51:15.281689+0000","last_undegraded":"2024-09-13T17:51:15.281689+0000","last_fullsized":"2024-09-13T17:51:15.281689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:10.410267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278070+0000","last_change":"2024-09-13T17:50:38.615989+0000","last_active":"2024-09-13T17:51:15.278070+0000","last_peered":"2024-09-13T17:51:15.278070+0000","last_clean":"2024-09-13T17:51:15.278070+0000","last_became_active":"2024-09-13T17:50:38.615530+0000","last_became_peered":"2024-09-13T17:50:38.615530+0000","last_unstale":"2024-09-13T17:51:15.278070+0000","last_undegraded":"2024-09-13T17:51:15.278070+0000","last_fullsized":"2024-09-13T17:51:15.278070+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:41.064092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279600+0000","last_change":"2024-09-13T17:50:34.595120+0000","last_active":"2024-09-13T17:51:15.279600+0000","last_peered":"2024-09-13T17:51:15.279600+0000","last_clean":"2024-09-13T17:51:15.279600+0000","last_became_active":"2024-09-13T17:50:34.594842+0000","last_became_peered":"2024-09-13T17:50:34.594842+0000","last_unstale":"2024-09-13T17:51:15.279600+0000","last_undegraded":"2024-09-13T17:51:15.279600+0000","last_fullsized":"2024-09-13T17:51:15.279600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:15:38.982524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"25'2","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279806+0000","last_change":"2024-09-13T17:50:34.199957+0000","last_active":"2024-09-13T17:51:15.279806+0000","last_peered":"2024-09-13T17:51:15.279806+0000","last_clean":"2024-09-13T17:51:15.279806+0000","last_became_active":"2024-09-13T17:50:34.199650+0000","last_became_peered":"2024-09-13T17:50:34.199650+0000","last_unstale":"2024-09-13T17:51:15.279806+0000","last_undegraded":"2024-09-13T17:51:15.279806+0000","last_fullsized":"2024-09-13T17:51:15.279806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:26.929549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269062+0000","last_change":"2024-09-13T17:50:38.609731+0000","last_active":"2024-09-13T17:51:15.269062+0000","last_peered":"2024-09-13T17:51:15.269062+0000","last_clean":"2024-09-13T17:51:15.269062+0000","last_became_active":"2024-09-13T17:50:38.609354+0000","last_became_peered":"2024-09-13T17:50:38.609354+0000","last_unstale":"2024-09-13T17:51:15.269062+0000","last_undegraded":"2024-09-13T17:51:15.269062+0000","last_fullsized":"2024-09-13T17:51:15.269062+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:14.142369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279334+0000","last_change":"2024-09-13T17:50:34.595146+0000","last_active":"2024-09-13T17:51:15.279334+0000","last_peered":"2024-09-13T17:51:15.279334+0000","last_clean":"2024-09-13T17:51:15.279334+0000","last_became_active":"2024-09-13T17:50:34.594876+0000","last_became_peered":"2024-09-13T17:50:34.594876+0000","last_unstale":"2024-09-13T17:51:15.279334+0000","last_undegraded":"2024-09-13T17:51:15.279334+0000","last_fullsized":"2024-09-13T17:51:15.279334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:26:10.638438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'2","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.278874+0000","last_change":"2024-09-13T17:50:34.200640+0000","last_active":"2024-09-13T17:51:16.278874+0000","last_peered":"2024-09-13T17:51:16.278874+0000","last_clean":"2024-09-13T17:51:16.278874+0000","last_became_active":"2024-09-13T17:50:34.200197+0000","last_became_peered":"2024-09-13T17:50:34.200197+0000","last_unstale":"2024-09-13T17:51:16.278874+0000","last_undegraded":"2024-09-13T17:51:16.278874+0000","last_fullsized":"2024-09-13T17:51:16.278874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:37:53.039720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.281995+0000","last_change":"2024-09-13T17:51:16.281995+0000","last_active":"2024-09-13T17:51:16.281995+0000","last_peered":"2024-09-13T17:51:16.281995+0000","last_clean":"2024-09-13T17:51:16.281995+0000","last_became_active":"2024-09-13T17:51:16.281476+0000","last_became_peered":"2024-09-13T17:51:16.281476+0000","last_unstale":"2024-09-13T17:51:16.281995+0000","last_undegraded":"2024-09-13T17:51:16.281995+0000","last_fullsized":"2024-09-13T17:51:16.281995+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:21:56.254641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3,6],"acting":[4,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278650+0000","last_change":"2024-09-13T17:50:34.585741+0000","last_active":"2024-09-13T17:51:15.278650+0000","last_peered":"2024-09-13T17:51:15.278650+0000","last_clean":"2024-09-13T17:51:15.278650+0000","last_became_active":"2024-09-13T17:50:34.585464+0000","last_became_peered":"2024-09-13T17:50:34.585464+0000","last_unstale":"2024-09-13T17:51:15.278650+0000","last_undegraded":"2024-09-13T17:51:15.278650+0000","last_fullsized":"2024-09-13T17:51:15.278650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:59:22.031459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"25'15","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278697+0000","last_change":"2024-09-13T17:50:34.201272+0000","last_active":"2024-09-13T17:51:15.278697+0000","last_peered":"2024-09-13T17:51:15.278697+0000","last_clean":"2024-09-13T17:51:15.278697+0000","last_became_active":"2024-09-13T17:50:34.200936+0000","last_became_peered":"2024-09-13T17:50:34.200936+0000","last_unstale":"2024-09-13T17:51:15.278697+0000","last_undegraded":"2024-09-13T17:51:15.278697+0000","last_fullsized":"2024-09-13T17:51:15.278697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:52.094879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277648+0000","last_change":"2024-09-13T17:50:38.615229+0000","last_active":"2024-09-13T17:51:15.277648+0000","last_peered":"2024-09-13T17:51:15.277648+0000","last_clean":"2024-09-13T17:51:15.277648+0000","last_became_active":"2024-09-13T17:50:38.614739+0000","last_became_peered":"2024-09-13T17:50:38.614739+0000","last_unstale":"2024-09-13T17:51:15.277648+0000","last_undegraded":"2024-09-13T17:51:15.277648+0000","last_fullsized":"2024-09-13T17:51:15.277648+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:18.619529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275934+0000","last_change":"2024-09-13T17:50:34.588848+0000","last_active":"2024-09-13T17:51:15.275934+0000","last_peered":"2024-09-13T17:51:15.275934+0000","last_clean":"2024-09-13T17:51:15.275934+0000","last_became_active":"2024-09-13T17:50:34.588565+0000","last_became_peered":"2024-09-13T17:50:34.588565+0000","last_unstale":"2024-09-13T17:51:15.275934+0000","last_undegraded":"2024-09-13T17:51:15.275934+0000","last_fullsized":"2024-09-13T17:51:15.275934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:47.927865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278190+0000","last_change":"2024-09-13T17:50:34.199008+0000","last_active":"2024-09-13T17:51:15.278190+0000","last_peered":"2024-09-13T17:51:15.278190+0000","last_clean":"2024-09-13T17:51:15.278190+0000","last_became_active":"2024-09-13T17:50:34.197839+0000","last_became_peered":"2024-09-13T17:50:34.197839+0000","last_unstale":"2024-09-13T17:51:15.278190+0000","last_undegraded":"2024-09-13T17:51:15.278190+0000","last_fullsized":"2024-09-13T17:51:15.278190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:51.430710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272409+0000","last_change":"2024-09-13T17:50:38.611080+0000","last_active":"2024-09-13T17:51:15.272409+0000","last_peered":"2024-09-13T17:51:15.272409+0000","last_clean":"2024-09-13T17:51:15.272409+0000","last_became_active":"2024-09-13T17:50:38.610688+0000","last_became_peered":"2024-09-13T17:50:38.610688+0000","last_unstale":"2024-09-13T17:51:15.272409+0000","last_undegraded":"2024-09-13T17:51:15.272409+0000","last_fullsized":"2024-09-13T17:51:15.272409+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:28.527617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278851+0000","last_change":"2024-09-13T17:50:34.594763+0000","last_active":"2024-09-13T17:51:15.278851+0000","last_peered":"2024-09-13T17:51:15.278851+0000","last_clean":"2024-09-13T17:51:15.278851+0000","last_became_active":"2024-09-13T17:50:34.594471+0000","last_became_peered":"2024-09-13T17:50:34.594471+0000","last_unstale":"2024-09-13T17:51:15.278851+0000","last_undegraded":"2024-09-13T17:51:15.278851+0000","last_fullsized":"2024-09-13T17:51:15.278851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:20:58.718271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269946+0000","last_change":"2024-09-13T17:50:34.209432+0000","last_active":"2024-09-13T17:51:15.269946+0000","last_peered":"2024-09-13T17:51:15.269946+0000","last_clean":"2024-09-13T17:51:15.269946+0000","last_became_active":"2024-09-13T17:50:34.208817+0000","last_became_peered":"2024-09-13T17:50:34.208817+0000","last_unstale":"2024-09-13T17:51:15.269946+0000","last_undegraded":"2024-09-13T17:51:15.269946+0000","last_fullsized":"2024-09-13T17:51:15.269946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:49.587528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272159+0000","last_change":"2024-09-13T17:50:38.609461+0000","last_active":"2024-09-13T17:51:15.272159+0000","last_peered":"2024-09-13T17:51:15.272159+0000","last_clean":"2024-09-13T17:51:15.272159+0000","last_became_active":"2024-09-13T17:50:38.609128+0000","last_became_peered":"2024-09-13T17:50:38.609128+0000","last_unstale":"2024-09-13T17:51:15.272159+0000","last_undegraded":"2024-09-13T17:51:15.272159+0000","last_fullsized":"2024-09-13T17:51:15.272159+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:02.286825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.283064+0000","last_change":"2024-09-13T17:50:34.592970+0000","last_active":"2024-09-13T17:51:15.283064+0000","last_peered":"2024-09-13T17:51:15.283064+0000","last_clean":"2024-09-13T17:51:15.283064+0000","last_became_active":"2024-09-13T17:50:34.592669+0000","last_became_peered":"2024-09-13T17:50:34.592669+0000","last_unstale":"2024-09-13T17:51:15.283064+0000","last_undegraded":"2024-09-13T17:51:15.283064+0000","last_fullsized":"2024-09-13T17:51:15.283064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:52.031261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282599+0000","last_change":"2024-09-13T17:50:34.200798+0000","last_active":"2024-09-13T17:51:15.282599+0000","last_peered":"2024-09-13T17:51:15.282599+0000","last_clean":"2024-09-13T17:51:15.282599+0000","last_became_active":"2024-09-13T17:50:34.200529+0000","last_became_peered":"2024-09-13T17:50:34.200529+0000","last_unstale":"2024-09-13T17:51:15.282599+0000","last_undegraded":"2024-09-13T17:51:15.282599+0000","last_fullsized":"2024-09-13T17:51:15.282599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:26.334296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270388+0000","last_change":"2024-09-13T17:50:38.606287+0000","last_active":"2024-09-13T17:51:15.270388+0000","last_peered":"2024-09-13T17:51:15.270388+0000","last_clean":"2024-09-13T17:51:15.270388+0000","last_became_active":"2024-09-13T17:50:38.605826+0000","last_became_peered":"2024-09-13T17:50:38.605826+0000","last_unstale":"2024-09-13T17:51:15.270388+0000","last_undegraded":"2024-09-13T17:51:15.270388+0000","last_fullsized":"2024-09-13T17:51:15.270388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:18.709454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279140+0000","last_change":"2024-09-13T17:50:34.591579+0000","last_active":"2024-09-13T17:51:15.279140+0000","last_peered":"2024-09-13T17:51:15.279140+0000","last_clean":"2024-09-13T17:51:15.279140+0000","last_became_active":"2024-09-13T17:50:34.586165+0000","last_became_peered":"2024-09-13T17:50:34.586165+0000","last_unstale":"2024-09-13T17:51:15.279140+0000","last_undegraded":"2024-09-13T17:51:15.279140+0000","last_fullsized":"2024-09-13T17:51:15.279140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:56:19.205176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'11","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277643+0000","last_change":"2024-09-13T17:50:34.203218+0000","last_active":"2024-09-13T17:51:15.277643+0000","last_peered":"2024-09-13T17:51:15.277643+0000","last_clean":"2024-09-13T17:51:15.277643+0000","last_became_active":"2024-09-13T17:50:34.202948+0000","last_became_peered":"2024-09-13T17:50:34.202948+0000","last_unstale":"2024-09-13T17:51:15.277643+0000","last_undegraded":"2024-09-13T17:51:15.277643+0000","last_fullsized":"2024-09-13T17:51:15.277643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:46.535569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.280382+0000","last_change":"2024-09-13T17:50:38.610597+0000","last_active":"2024-09-13T17:51:15.280382+0000","last_peered":"2024-09-13T17:51:15.280382+0000","last_clean":"2024-09-13T17:51:15.280382+0000","last_became_active":"2024-09-13T17:50:38.610055+0000","last_became_peered":"2024-09-13T17:50:38.610055+0000","last_unstale":"2024-09-13T17:51:15.280382+0000","last_undegraded":"2024-09-13T17:51:15.280382+0000","last_fullsized":"2024-09-13T17:51:15.280382+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:55.754936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275920+0000","last_change":"2024-09-13T17:50:34.593963+0000","last_active":"2024-09-13T17:51:15.275920+0000","last_peered":"2024-09-13T17:51:15.275920+0000","last_clean":"2024-09-13T17:51:15.275920+0000","last_became_active":"2024-09-13T17:50:34.593454+0000","last_became_peered":"2024-09-13T17:50:34.593454+0000","last_unstale":"2024-09-13T17:51:15.275920+0000","last_undegraded":"2024-09-13T17:51:15.275920+0000","last_fullsized":"2024-09-13T17:51:15.275920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:18.883521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281506+0000","last_change":"2024-09-13T17:50:34.207936+0000","last_active":"2024-09-13T17:51:15.281506+0000","last_peered":"2024-09-13T17:51:15.281506+0000","last_clean":"2024-09-13T17:51:15.281506+0000","last_became_active":"2024-09-13T17:50:34.207605+0000","last_became_peered":"2024-09-13T17:50:34.207605+0000","last_unstale":"2024-09-13T17:51:15.281506+0000","last_undegraded":"2024-09-13T17:51:15.281506+0000","last_fullsized":"2024-09-13T17:51:15.281506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:05.494406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.274690+0000","last_change":"2024-09-13T17:50:38.616297+0000","last_active":"2024-09-13T17:51:15.274690+0000","last_peered":"2024-09-13T17:51:15.274690+0000","last_clean":"2024-09-13T17:51:15.274690+0000","last_became_active":"2024-09-13T17:50:38.614165+0000","last_became_peered":"2024-09-13T17:50:38.614165+0000","last_unstale":"2024-09-13T17:51:15.274690+0000","last_undegraded":"2024-09-13T17:51:15.274690+0000","last_fullsized":"2024-09-13T17:51:15.274690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:11.357026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278023+0000","last_change":"2024-09-13T17:50:34.583001+0000","last_active":"2024-09-13T17:51:15.278023+0000","last_peered":"2024-09-13T17:51:15.278023+0000","last_clean":"2024-09-13T17:51:15.278023+0000","last_became_active":"2024-09-13T17:50:34.582529+0000","last_became_peered":"2024-09-13T17:50:34.582529+0000","last_unstale":"2024-09-13T17:51:15.278023+0000","last_undegraded":"2024-09-13T17:51:15.278023+0000","last_fullsized":"2024-09-13T17:51:15.278023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:05.287181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271281+0000","last_change":"2024-09-13T17:50:34.205138+0000","last_active":"2024-09-13T17:51:15.271281+0000","last_peered":"2024-09-13T17:51:15.271281+0000","last_clean":"2024-09-13T17:51:15.271281+0000","last_became_active":"2024-09-13T17:50:34.203360+0000","last_became_peered":"2024-09-13T17:50:34.203360+0000","last_unstale":"2024-09-13T17:51:15.271281+0000","last_undegraded":"2024-09-13T17:51:15.271281+0000","last_fullsized":"2024-09-13T17:51:15.271281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:35.756963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272372+0000","last_change":"2024-09-13T17:50:38.611036+0000","last_active":"2024-09-13T17:51:15.272372+0000","last_peered":"2024-09-13T17:51:15.272372+0000","last_clean":"2024-09-13T17:51:15.272372+0000","last_became_active":"2024-09-13T17:50:38.610692+0000","last_became_peered":"2024-09-13T17:50:38.610692+0000","last_unstale":"2024-09-13T17:51:15.272372+0000","last_undegraded":"2024-09-13T17:51:15.272372+0000","last_fullsized":"2024-09-13T17:51:15.272372+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:10.147998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.368023+0000","last_change":"2024-09-13T17:51:16.365295+0000","last_active":"2024-09-13T17:51:16.368023+0000","last_peered":"2024-09-13T17:51:16.368023+0000","last_clean":"2024-09-13T17:51:16.368023+0000","last_became_active":"2024-09-13T17:51:16.364437+0000","last_became_peered":"2024-09-13T17:51:16.364437+0000","last_unstale":"2024-09-13T17:51:16.368023+0000","last_undegraded":"2024-09-13T17:51:16.368023+0000","last_fullsized":"2024-09-13T17:51:16.368023+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:23.208899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276819+0000","last_change":"2024-09-13T17:50:34.204376+0000","last_active":"2024-09-13T17:51:15.276819+0000","last_peered":"2024-09-13T17:51:15.276819+0000","last_clean":"2024-09-13T17:51:15.276819+0000","last_became_active":"2024-09-13T17:50:34.204003+0000","last_became_peered":"2024-09-13T17:50:34.204003+0000","last_unstale":"2024-09-13T17:51:15.276819+0000","last_undegraded":"2024-09-13T17:51:15.276819+0000","last_fullsized":"2024-09-13T17:51:15.276819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:14.023990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270270+0000","last_change":"2024-09-13T17:50:38.606144+0000","last_active":"2024-09-13T17:51:15.270270+0000","last_peered":"2024-09-13T17:51:15.270270+0000","last_clean":"2024-09-13T17:51:15.270270+0000","last_became_active":"2024-09-13T17:50:38.605646+0000","last_became_peered":"2024-09-13T17:50:38.605646+0000","last_unstale":"2024-09-13T17:51:15.270270+0000","last_undegraded":"2024-09-13T17:51:15.270270+0000","last_fullsized":"2024-09-13T17:51:15.270270+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:07.120075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277435+0000","last_change":"2024-09-13T17:50:38.618855+0000","last_active":"2024-09-13T17:51:15.277435+0000","last_peered":"2024-09-13T17:51:15.277435+0000","last_clean":"2024-09-13T17:51:15.277435+0000","last_became_active":"2024-09-13T17:50:38.618449+0000","last_became_peered":"2024-09-13T17:50:38.618449+0000","last_unstale":"2024-09-13T17:51:15.277435+0000","last_undegraded":"2024-09-13T17:51:15.277435+0000","last_fullsized":"2024-09-13T17:51:15.277435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:29:01.114087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.280511+0000","last_change":"2024-09-13T17:50:34.590079+0000","last_active":"2024-09-13T17:51:15.280511+0000","last_peered":"2024-09-13T17:51:15.280511+0000","last_clean":"2024-09-13T17:51:15.280511+0000","last_became_active":"2024-09-13T17:50:34.589396+0000","last_became_peered":"2024-09-13T17:50:34.589396+0000","last_unstale":"2024-09-13T17:51:15.280511+0000","last_undegraded":"2024-09-13T17:51:15.280511+0000","last_fullsized":"2024-09-13T17:51:15.280511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:03.308341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269447+0000","last_change":"2024-09-13T17:50:34.206575+0000","last_active":"2024-09-13T17:51:15.269447+0000","last_peered":"2024-09-13T17:51:15.269447+0000","last_clean":"2024-09-13T17:51:15.269447+0000","last_became_active":"2024-09-13T17:50:34.206308+0000","last_became_peered":"2024-09-13T17:50:34.206308+0000","last_unstale":"2024-09-13T17:51:15.269447+0000","last_undegraded":"2024-09-13T17:51:15.269447+0000","last_fullsized":"2024-09-13T17:51:15.269447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:34.766538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276602+0000","last_change":"2024-09-13T17:50:38.618131+0000","last_active":"2024-09-13T17:51:15.276602+0000","last_peered":"2024-09-13T17:51:15.276602+0000","last_clean":"2024-09-13T17:51:15.276602+0000","last_became_active":"2024-09-13T17:50:38.617611+0000","last_became_peered":"2024-09-13T17:50:38.617611+0000","last_unstale":"2024-09-13T17:51:15.276602+0000","last_undegraded":"2024-09-13T17:51:15.276602+0000","last_fullsized":"2024-09-13T17:51:15.276602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:30.746445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272535+0000","last_change":"2024-09-13T17:50:34.582153+0000","last_active":"2024-09-13T17:51:15.272535+0000","last_peered":"2024-09-13T17:51:15.272535+0000","last_clean":"2024-09-13T17:51:15.272535+0000","last_became_active":"2024-09-13T17:50:34.581708+0000","last_became_peered":"2024-09-13T17:50:34.581708+0000","last_unstale":"2024-09-13T17:51:15.272535+0000","last_undegraded":"2024-09-13T17:51:15.272535+0000","last_fullsized":"2024-09-13T17:51:15.272535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:22.238810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270828+0000","last_change":"2024-09-13T17:50:34.205489+0000","last_active":"2024-09-13T17:51:15.270828+0000","last_peered":"2024-09-13T17:51:15.270828+0000","last_clean":"2024-09-13T17:51:15.270828+0000","last_became_active":"2024-09-13T17:50:34.205202+0000","last_became_peered":"2024-09-13T17:50:34.205202+0000","last_unstale":"2024-09-13T17:51:15.270828+0000","last_undegraded":"2024-09-13T17:51:15.270828+0000","last_fullsized":"2024-09-13T17:51:15.270828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:54.450276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276330+0000","last_change":"2024-09-13T17:50:34.202470+0000","last_active":"2024-09-13T17:51:15.276330+0000","last_peered":"2024-09-13T17:51:15.276330+0000","last_clean":"2024-09-13T17:51:15.276330+0000","last_became_active":"2024-09-13T17:50:34.202163+0000","last_became_peered":"2024-09-13T17:50:34.202163+0000","last_unstale":"2024-09-13T17:51:15.276330+0000","last_undegraded":"2024-09-13T17:51:15.276330+0000","last_fullsized":"2024-09-13T17:51:15.276330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:03.472932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282749+0000","last_change":"2024-09-13T17:50:34.590947+0000","last_active":"2024-09-13T17:51:15.282749+0000","last_peered":"2024-09-13T17:51:15.282749+0000","last_clean":"2024-09-13T17:51:15.282749+0000","last_became_active":"2024-09-13T17:50:34.590497+0000","last_became_peered":"2024-09-13T17:50:34.590497+0000","last_unstale":"2024-09-13T17:51:15.282749+0000","last_undegraded":"2024-09-13T17:51:15.282749+0000","last_fullsized":"2024-09-13T17:51:15.282749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:56:27.848092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279328+0000","last_change":"2024-09-13T17:50:38.618518+0000","last_active":"2024-09-13T17:51:15.279328+0000","last_peered":"2024-09-13T17:51:15.279328+0000","last_clean":"2024-09-13T17:51:15.279328+0000","last_became_active":"2024-09-13T17:50:38.618108+0000","last_became_peered":"2024-09-13T17:50:38.618108+0000","last_unstale":"2024-09-13T17:51:15.279328+0000","last_undegraded":"2024-09-13T17:51:15.279328+0000","last_fullsized":"2024-09-13T17:51:15.279328+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:42.260237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'6","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.273983+0000","last_change":"2024-09-13T17:50:34.218459+0000","last_active":"2024-09-13T17:51:16.273983+0000","last_peered":"2024-09-13T17:51:16.273983+0000","last_clean":"2024-09-13T17:51:16.273983+0000","last_became_active":"2024-09-13T17:50:34.218087+0000","last_became_peered":"2024-09-13T17:50:34.218087+0000","last_unstale":"2024-09-13T17:51:16.273983+0000","last_undegraded":"2024-09-13T17:51:16.273983+0000","last_fullsized":"2024-09-13T17:51:16.273983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:56.914544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1016,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.274322+0000","last_change":"2024-09-13T17:50:34.583845+0000","last_active":"2024-09-13T17:51:15.274322+0000","last_peered":"2024-09-13T17:51:15.274322+0000","last_clean":"2024-09-13T17:51:15.274322+0000","last_became_active":"2024-09-13T17:50:34.583283+0000","last_became_peered":"2024-09-13T17:50:34.583283+0000","last_unstale":"2024-09-13T17:51:15.274322+0000","last_undegraded":"2024-09-13T17:51:15.274322+0000","last_fullsized":"2024-09-13T17:51:15.274322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:55.630285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271562+0000","last_change":"2024-09-13T17:50:38.611987+0000","last_active":"2024-09-13T17:51:15.271562+0000","last_peered":"2024-09-13T17:51:15.271562+0000","last_clean":"2024-09-13T17:51:15.271562+0000","last_became_active":"2024-09-13T17:50:38.611636+0000","last_became_peered":"2024-09-13T17:50:38.611636+0000","last_unstale":"2024-09-13T17:51:15.271562+0000","last_undegraded":"2024-09-13T17:51:15.271562+0000","last_fullsized":"2024-09-13T17:51:15.271562+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:41.111562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275726+0000","last_change":"2024-09-13T17:50:34.588927+0000","last_active":"2024-09-13T17:51:15.275726+0000","last_peered":"2024-09-13T17:51:15.275726+0000","last_clean":"2024-09-13T17:51:15.275726+0000","last_became_active":"2024-09-13T17:50:34.588651+0000","last_became_peered":"2024-09-13T17:50:34.588651+0000","last_unstale":"2024-09-13T17:51:15.275726+0000","last_undegraded":"2024-09-13T17:51:15.275726+0000","last_fullsized":"2024-09-13T17:51:15.275726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:49.429670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278614+0000","last_change":"2024-09-13T17:50:34.583795+0000","last_active":"2024-09-13T17:51:15.278614+0000","last_peered":"2024-09-13T17:51:15.278614+0000","last_clean":"2024-09-13T17:51:15.278614+0000","last_became_active":"2024-09-13T17:50:34.583493+0000","last_became_peered":"2024-09-13T17:50:34.583493+0000","last_unstale":"2024-09-13T17:51:15.278614+0000","last_undegraded":"2024-09-13T17:51:15.278614+0000","last_fullsized":"2024-09-13T17:51:15.278614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:30.463932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269876+0000","last_change":"2024-09-13T17:50:34.587624+0000","last_active":"2024-09-13T17:51:15.269876+0000","last_peered":"2024-09-13T17:51:15.269876+0000","last_clean":"2024-09-13T17:51:15.269876+0000","last_became_active":"2024-09-13T17:50:34.587119+0000","last_became_peered":"2024-09-13T17:50:34.587119+0000","last_unstale":"2024-09-13T17:51:15.269876+0000","last_undegraded":"2024-09-13T17:51:15.269876+0000","last_fullsized":"2024-09-13T17:51:15.269876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:16.783899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278177+0000","last_change":"2024-09-13T17:50:34.581923+0000","last_active":"2024-09-13T17:51:15.278177+0000","last_peered":"2024-09-13T17:51:15.278177+0000","last_clean":"2024-09-13T17:51:15.278177+0000","last_became_active":"2024-09-13T17:50:34.581628+0000","last_became_peered":"2024-09-13T17:50:34.581628+0000","last_unstale":"2024-09-13T17:51:15.278177+0000","last_undegraded":"2024-09-13T17:51:15.278177+0000","last_fullsized":"2024-09-13T17:51:15.278177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:28:55.936864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272460+0000","last_change":"2024-09-13T17:50:34.595037+0000","last_active":"2024-09-13T17:51:15.272460+0000","last_peered":"2024-09-13T17:51:15.272460+0000","last_clean":"2024-09-13T17:51:15.272460+0000","last_became_active":"2024-09-13T17:50:34.594857+0000","last_became_peered":"2024-09-13T17:50:34.594857+0000","last_unstale":"2024-09-13T17:51:15.272460+0000","last_undegraded":"2024-09-13T17:51:15.272460+0000","last_fullsized":"2024-09-13T17:51:15.272460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:11.420413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281006+0000","last_change":"2024-09-13T17:50:34.589651+0000","last_active":"2024-09-13T17:51:15.281006+0000","last_peered":"2024-09-13T17:51:15.281006+0000","last_clean":"2024-09-13T17:51:15.281006+0000","last_became_active":"2024-09-13T17:50:34.589354+0000","last_became_peered":"2024-09-13T17:50:34.589354+0000","last_unstale":"2024-09-13T17:51:15.281006+0000","last_undegraded":"2024-09-13T17:51:15.281006+0000","last_fullsized":"2024-09-13T17:51:15.281006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:31.294048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277272+0000","last_change":"2024-09-13T17:50:34.597949+0000","last_active":"2024-09-13T17:51:15.277272+0000","last_peered":"2024-09-13T17:51:15.277272+0000","last_clean":"2024-09-13T17:51:15.277272+0000","last_became_active":"2024-09-13T17:50:34.597621+0000","last_became_peered":"2024-09-13T17:50:34.597621+0000","last_unstale":"2024-09-13T17:51:15.277272+0000","last_undegraded":"2024-09-13T17:51:15.277272+0000","last_fullsized":"2024-09-13T17:51:15.277272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:23.133462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272676+0000","last_change":"2024-09-13T17:50:34.589893+0000","last_active":"2024-09-13T17:51:15.272676+0000","last_peered":"2024-09-13T17:51:15.272676+0000","last_clean":"2024-09-13T17:51:15.272676+0000","last_became_active":"2024-09-13T17:50:34.589626+0000","last_became_peered":"2024-09-13T17:50:34.589626+0000","last_unstale":"2024-09-13T17:51:15.272676+0000","last_undegraded":"2024-09-13T17:51:15.272676+0000","last_fullsized":"2024-09-13T17:51:15.272676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:41.321025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269328+0000","last_change":"2024-09-13T17:50:34.590046+0000","last_active":"2024-09-13T17:51:15.269328+0000","last_peered":"2024-09-13T17:51:15.269328+0000","last_clean":"2024-09-13T17:51:15.269328+0000","last_became_active":"2024-09-13T17:50:34.589757+0000","last_became_peered":"2024-09-13T17:50:34.589757+0000","last_unstale":"2024-09-13T17:51:15.269328+0000","last_undegraded":"2024-09-13T17:51:15.269328+0000","last_fullsized":"2024-09-13T17:51:15.269328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:28.239550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270567+0000","last_change":"2024-09-13T17:50:34.595518+0000","last_active":"2024-09-13T17:51:15.270567+0000","last_peered":"2024-09-13T17:51:15.270567+0000","last_clean":"2024-09-13T17:51:15.270567+0000","last_became_active":"2024-09-13T17:50:34.595242+0000","last_became_peered":"2024-09-13T17:50:34.595242+0000","last_unstale":"2024-09-13T17:51:15.270567+0000","last_undegraded":"2024-09-13T17:51:15.270567+0000","last_fullsized":"2024-09-13T17:51:15.270567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:26.149384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269781+0000","last_change":"2024-09-13T17:50:34.593183+0000","last_active":"2024-09-13T17:51:15.269781+0000","last_peered":"2024-09-13T17:51:15.269781+0000","last_clean":"2024-09-13T17:51:15.269781+0000","last_became_active":"2024-09-13T17:50:34.592893+0000","last_became_peered":"2024-09-13T17:50:34.592893+0000","last_unstale":"2024-09-13T17:51:15.269781+0000","last_undegraded":"2024-09-13T17:51:15.269781+0000","last_fullsized":"2024-09-13T17:51:15.269781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:19.459505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278000+0000","last_change":"2024-09-13T17:50:34.594679+0000","last_active":"2024-09-13T17:51:15.278000+0000","last_peered":"2024-09-13T17:51:15.278000+0000","last_clean":"2024-09-13T17:51:15.278000+0000","last_became_active":"2024-09-13T17:50:34.593750+0000","last_became_peered":"2024-09-13T17:50:34.593750+0000","last_unstale":"2024-09-13T17:51:15.278000+0000","last_undegraded":"2024-09-13T17:51:15.278000+0000","last_fullsized":"2024-09-13T17:51:15.278000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:02.135577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277577+0000","last_change":"2024-09-13T17:50:34.598736+0000","last_active":"2024-09-13T17:51:15.277577+0000","last_peered":"2024-09-13T17:51:15.277577+0000","last_clean":"2024-09-13T17:51:15.277577+0000","last_became_active":"2024-09-13T17:50:34.598433+0000","last_became_peered":"2024-09-13T17:50:34.598433+0000","last_unstale":"2024-09-13T17:51:15.277577+0000","last_undegraded":"2024-09-13T17:51:15.277577+0000","last_fullsized":"2024-09-13T17:51:15.277577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:08.975139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279323+0000","last_change":"2024-09-13T17:50:34.591795+0000","last_active":"2024-09-13T17:51:15.279323+0000","last_peered":"2024-09-13T17:51:15.279323+0000","last_clean":"2024-09-13T17:51:15.279323+0000","last_became_active":"2024-09-13T17:50:34.591286+0000","last_became_peered":"2024-09-13T17:50:34.591286+0000","last_unstale":"2024-09-13T17:51:15.279323+0000","last_undegraded":"2024-09-13T17:51:15.279323+0000","last_fullsized":"2024-09-13T17:51:15.279323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:31.580247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279687+0000","last_change":"2024-09-13T17:50:34.587774+0000","last_active":"2024-09-13T17:51:15.279687+0000","last_peered":"2024-09-13T17:51:15.279687+0000","last_clean":"2024-09-13T17:51:15.279687+0000","last_became_active":"2024-09-13T17:50:34.587506+0000","last_became_peered":"2024-09-13T17:50:34.587506+0000","last_unstale":"2024-09-13T17:51:15.279687+0000","last_undegraded":"2024-09-13T17:51:15.279687+0000","last_fullsized":"2024-09-13T17:51:15.279687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:46.275528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271196+0000","last_change":"2024-09-13T17:50:34.596100+0000","last_active":"2024-09-13T17:51:15.271196+0000","last_peered":"2024-09-13T17:51:15.271196+0000","last_clean":"2024-09-13T17:51:15.271196+0000","last_became_active":"2024-09-13T17:50:34.595798+0000","last_became_peered":"2024-09-13T17:50:34.595798+0000","last_unstale":"2024-09-13T17:51:15.271196+0000","last_undegraded":"2024-09-13T17:51:15.271196+0000","last_fullsized":"2024-09-13T17:51:15.271196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:49.543845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269217+0000","last_change":"2024-09-13T17:50:34.584818+0000","last_active":"2024-09-13T17:51:15.269217+0000","last_peered":"2024-09-13T17:51:15.269217+0000","last_clean":"2024-09-13T17:51:15.269217+0000","last_became_active":"2024-09-13T17:50:34.584516+0000","last_became_peered":"2024-09-13T17:50:34.584516+0000","last_unstale":"2024-09-13T17:51:15.269217+0000","last_undegraded":"2024-09-13T17:51:15.269217+0000","last_fullsized":"2024-09-13T17:51:15.269217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:52.402024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281649+0000","last_change":"2024-09-13T17:50:34.598886+0000","last_active":"2024-09-13T17:51:15.281649+0000","last_peered":"2024-09-13T17:51:15.281649+0000","last_clean":"2024-09-13T17:51:15.281649+0000","last_became_active":"2024-09-13T17:50:34.598648+0000","last_became_peered":"2024-09-13T17:50:34.598648+0000","last_unstale":"2024-09-13T17:51:15.281649+0000","last_undegraded":"2024-09-13T17:51:15.281649+0000","last_fullsized":"2024-09-13T17:51:15.281649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:00:43.063223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282938+0000","last_change":"2024-09-13T17:50:34.582649+0000","last_active":"2024-09-13T17:51:15.282938+0000","last_peered":"2024-09-13T17:51:15.282938+0000","last_clean":"2024-09-13T17:51:15.282938+0000","last_became_active":"2024-09-13T17:50:34.581699+0000","last_became_peered":"2024-09-13T17:50:34.581699+0000","last_unstale":"2024-09-13T17:51:15.282938+0000","last_undegraded":"2024-09-13T17:51:15.282938+0000","last_fullsized":"2024-09-13T17:51:15.282938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:18.959303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270646+0000","last_change":"2024-09-13T17:50:34.587484+0000","last_active":"2024-09-13T17:51:15.270646+0000","last_peered":"2024-09-13T17:51:15.270646+0000","last_clean":"2024-09-13T17:51:15.270646+0000","last_became_active":"2024-09-13T17:50:34.587217+0000","last_became_peered":"2024-09-13T17:50:34.587217+0000","last_unstale":"2024-09-13T17:51:15.270646+0000","last_undegraded":"2024-09-13T17:51:15.270646+0000","last_fullsized":"2024-09-13T17:51:15.270646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:24:19.172607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275806+0000","last_change":"2024-09-13T17:50:34.592871+0000","last_active":"2024-09-13T17:51:15.275806+0000","last_peered":"2024-09-13T17:51:15.275806+0000","last_clean":"2024-09-13T17:51:15.275806+0000","last_became_active":"2024-09-13T17:50:34.588513+0000","last_became_peered":"2024-09-13T17:50:34.588513+0000","last_unstale":"2024-09-13T17:51:15.275806+0000","last_undegraded":"2024-09-13T17:51:15.275806+0000","last_fullsized":"2024-09-13T17:51:15.275806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:33.131420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276633+0000","last_change":"2024-09-13T17:50:34.585713+0000","last_active":"2024-09-13T17:51:15.276633+0000","last_peered":"2024-09-13T17:51:15.276633+0000","last_clean":"2024-09-13T17:51:15.276633+0000","last_became_active":"2024-09-13T17:50:34.585438+0000","last_became_peered":"2024-09-13T17:50:34.585438+0000","last_unstale":"2024-09-13T17:51:15.276633+0000","last_undegraded":"2024-09-13T17:51:15.276633+0000","last_fullsized":"2024-09-13T17:51:15.276633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:56:49.905393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277185+0000","last_change":"2024-09-13T17:50:34.589628+0000","last_active":"2024-09-13T17:51:15.277185+0000","last_peered":"2024-09-13T17:51:15.277185+0000","last_clean":"2024-09-13T17:51:15.277185+0000","last_became_active":"2024-09-13T17:50:34.587665+0000","last_became_peered":"2024-09-13T17:50:34.587665+0000","last_unstale":"2024-09-13T17:51:15.277185+0000","last_undegraded":"2024-09-13T17:51:15.277185+0000","last_fullsized":"2024-09-13T17:51:15.277185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:15.518264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279579+0000","last_change":"2024-09-13T17:50:34.591776+0000","last_active":"2024-09-13T17:51:15.279579+0000","last_peered":"2024-09-13T17:51:15.279579+0000","last_clean":"2024-09-13T17:51:15.279579+0000","last_became_active":"2024-09-13T17:50:34.591496+0000","last_became_peered":"2024-09-13T17:50:34.591496+0000","last_unstale":"2024-09-13T17:51:15.279579+0000","last_undegraded":"2024-09-13T17:51:15.279579+0000","last_fullsized":"2024-09-13T17:51:15.279579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:48.651999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.283047+0000","last_change":"2024-09-13T17:51:16.282372+0000","last_active":"2024-09-13T17:51:16.283047+0000","last_peered":"2024-09-13T17:51:16.283047+0000","last_clean":"2024-09-13T17:51:16.283047+0000","last_became_active":"2024-09-13T17:51:16.282065+0000","last_became_peered":"2024-09-13T17:51:16.282065+0000","last_unstale":"2024-09-13T17:51:16.283047+0000","last_undegraded":"2024-09-13T17:51:16.283047+0000","last_fullsized":"2024-09-13T17:51:16.283047+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:49.291224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278625+0000","last_change":"2024-09-13T17:50:34.585531+0000","last_active":"2024-09-13T17:51:15.278625+0000","last_peered":"2024-09-13T17:51:15.278625+0000","last_clean":"2024-09-13T17:51:15.278625+0000","last_became_active":"2024-09-13T17:50:34.585226+0000","last_became_peered":"2024-09-13T17:50:34.585226+0000","last_unstale":"2024-09-13T17:51:15.278625+0000","last_undegraded":"2024-09-13T17:51:15.278625+0000","last_fullsized":"2024-09-13T17:51:15.278625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:17.105028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269683+0000","last_change":"2024-09-13T17:50:34.586750+0000","last_active":"2024-09-13T17:51:15.269683+0000","last_peered":"2024-09-13T17:51:15.269683+0000","last_clean":"2024-09-13T17:51:15.269683+0000","last_became_active":"2024-09-13T17:50:34.584564+0000","last_became_peered":"2024-09-13T17:50:34.584564+0000","last_unstale":"2024-09-13T17:51:15.269683+0000","last_undegraded":"2024-09-13T17:51:15.269683+0000","last_fullsized":"2024-09-13T17:51:15.269683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:54.033257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278769+0000","last_change":"2024-09-13T17:50:34.585938+0000","last_active":"2024-09-13T17:51:15.278769+0000","last_peered":"2024-09-13T17:51:15.278769+0000","last_clean":"2024-09-13T17:51:15.278769+0000","last_became_active":"2024-09-13T17:50:34.585584+0000","last_became_peered":"2024-09-13T17:50:34.585584+0000","last_unstale":"2024-09-13T17:51:15.278769+0000","last_undegraded":"2024-09-13T17:51:15.278769+0000","last_fullsized":"2024-09-13T17:51:15.278769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:32.623468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271015+0000","last_change":"2024-09-13T17:50:34.588762+0000","last_active":"2024-09-13T17:51:15.271015+0000","last_peered":"2024-09-13T17:51:15.271015+0000","last_clean":"2024-09-13T17:51:15.271015+0000","last_became_active":"2024-09-13T17:50:34.588489+0000","last_became_peered":"2024-09-13T17:50:34.588489+0000","last_unstale":"2024-09-13T17:51:15.271015+0000","last_undegraded":"2024-09-13T17:51:15.271015+0000","last_fullsized":"2024-09-13T17:51:15.271015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:38.266297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277059+0000","last_change":"2024-09-13T17:50:34.583889+0000","last_active":"2024-09-13T17:51:15.277059+0000","last_peered":"2024-09-13T17:51:15.277059+0000","last_clean":"2024-09-13T17:51:15.277059+0000","last_became_active":"2024-09-13T17:50:34.583551+0000","last_became_peered":"2024-09-13T17:50:34.583551+0000","last_unstale":"2024-09-13T17:51:15.277059+0000","last_undegraded":"2024-09-13T17:51:15.277059+0000","last_fullsized":"2024-09-13T17:51:15.277059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:19.888626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281343+0000","last_change":"2024-09-13T17:50:34.585617+0000","last_active":"2024-09-13T17:51:15.281343+0000","last_peered":"2024-09-13T17:51:15.281343+0000","last_clean":"2024-09-13T17:51:15.281343+0000","last_became_active":"2024-09-13T17:50:34.585340+0000","last_became_peered":"2024-09-13T17:50:34.585340+0000","last_unstale":"2024-09-13T17:51:15.281343+0000","last_undegraded":"2024-09-13T17:51:15.281343+0000","last_fullsized":"2024-09-13T17:51:15.281343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:18.331045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279542+0000","last_change":"2024-09-13T17:50:34.592365+0000","last_active":"2024-09-13T17:51:15.279542+0000","last_peered":"2024-09-13T17:51:15.279542+0000","last_clean":"2024-09-13T17:51:15.279542+0000","last_became_active":"2024-09-13T17:50:34.592046+0000","last_became_peered":"2024-09-13T17:50:34.592046+0000","last_unstale":"2024-09-13T17:51:15.279542+0000","last_undegraded":"2024-09-13T17:51:15.279542+0000","last_fullsized":"2024-09-13T17:51:15.279542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:54.283683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7751,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":15502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":307200,"data_stored":120825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4870000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]}]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":303104,"data_stored":116742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":307200,"data_stored":116868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":892928,"data_stored":706952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.5009999999999999,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.60199999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.528,"5min":2.528,"15min":2.528},"last":0.70299999999999996}]},{"osd":3,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.51600000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.5790000000000002,"5min":2.5790000000000002,"15min":2.5790000000000002},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.79500000000000004}]},{"osd":5,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.3700000000000001,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.5499999999999998,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":0.74099999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":1.022},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":0.92000000000000004}]},{"osd":7,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.4660000000000002,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.1599999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":897024,"data_stored":708002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":303104,"data_stored":120349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":4,"last update":"Fri Sep 13 17:51:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.90700000000000003}]},{"osd":5,"last update":"Fri Sep 13 17:51:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.995,"5min":0.995,"15min":0.995},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.296}]},{"osd":7,"last update":"Fri Sep 13 17:51:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.071},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.3999999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":303104,"data_stored":117302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5820000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542158,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":299008,"data_stored":116590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:51:23.355 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-13T17:51:23.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.462+0000 7fa73359e700 1 Processor -- start 2024-09-13T17:51:23.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.462+0000 7fa73359e700 1 -- start start 2024-09-13T17:51:23.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.462+0000 7fa73359e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa734072350 0x7fa734150d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:23.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.462+0000 7fa73359e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7340c55c0 con 0x7fa734071f70 2024-09-13T17:51:23.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.462+0000 7fa73359e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa734157710 con 0x7fa7340c7200 2024-09-13T17:51:23.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.462+0000 7fa73359e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa73406a390 con 0x7fa734072350 2024-09-13T17:51:23.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.462+0000 7fa73259c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa734071f70 0x7fa7340c5d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44546/0 (socket says 172.21.15.40:44546) 2024-09-13T17:51:23.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.462+0000 7fa73259c700 1 -- 172.21.15.40:0/3750946381 learned_addr learned my addr 172.21.15.40:0/3750946381 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:23.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.462+0000 7fa731d9b700 1 --2- 172.21.15.40:0/3750946381 >> v2:172.21.15.43:3300/0 conn(0x7fa734072350 0x7fa734150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:23.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.462+0000 7fa73159a700 1 -- 172.21.15.40:0/3750946381 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2802436505 0 0) 0x7fa7340c55c0 con 0x7fa734071f70 2024-09-13T17:51:23.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.462+0000 7fa73159a700 1 -- 172.21.15.40:0/3750946381 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa71c0035d0 con 0x7fa734071f70 2024-09-13T17:51:23.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.462+0000 7fa731d9b700 1 -- 172.21.15.40:0/3750946381 >> v1:172.21.15.43:6789/0 conn(0x7fa7340c7200 legacy=0x7fa734161ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:23.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.462+0000 7fa731d9b700 1 -- 172.21.15.40:0/3750946381 >> v1:172.21.15.40:6789/0 conn(0x7fa734071f70 legacy=0x7fa7340c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:23.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.462+0000 7fa731d9b700 1 -- 172.21.15.40:0/3750946381 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7341579d0 con 0x7fa734072350 2024-09-13T17:51:23.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.462+0000 7fa731d9b700 1 --2- 172.21.15.40:0/3750946381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa734072350 0x7fa734150d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa724009850 tx=0x7fa724009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=438d3b56cc31b664 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:23.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.462+0000 7fa73159a700 1 -- 172.21.15.40:0/3750946381 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724011830 con 0x7fa734072350 2024-09-13T17:51:23.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.466+0000 7fa73159a700 1 -- 172.21.15.40:0/3750946381 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa724011990 con 0x7fa734072350 2024-09-13T17:51:23.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.466+0000 7fa73159a700 1 -- 172.21.15.40:0/3750946381 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724011d40 con 0x7fa734072350 2024-09-13T17:51:23.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.466+0000 7fa73359e700 1 -- 172.21.15.40:0/3750946381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa734072350 msgr2=0x7fa734150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:23.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.466+0000 7fa73359e700 1 --2- 172.21.15.40:0/3750946381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa734072350 0x7fa734150d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa724009850 tx=0x7fa724009b60 comp rx=0 tx=0).stop 2024-09-13T17:51:23.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.466+0000 7fa73359e700 1 -- 172.21.15.40:0/3750946381 shutdown_connections 2024-09-13T17:51:23.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.466+0000 7fa73359e700 1 --2- 172.21.15.40:0/3750946381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa734072350 0x7fa734150d40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.466+0000 7fa73359e700 1 -- 172.21.15.40:0/3750946381 >> 172.21.15.40:0/3750946381 conn(0x7fa7340cabc0 msgr2=0x7fa7340cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:23.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.466+0000 7fa73359e700 1 -- 172.21.15.40:0/3750946381 shutdown_connections 2024-09-13T17:51:23.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.466+0000 7fa73359e700 1 -- 172.21.15.40:0/3750946381 wait complete. 2024-09-13T17:51:23.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.466+0000 7fa73359e700 1 Processor -- start 2024-09-13T17:51:23.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.466+0000 7fa73359e700 1 -- start start 2024-09-13T17:51:23.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa73359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa734071f70 0x7fa734148f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:23.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa73359e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa734072350 0x7fa734149490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:23.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa73359e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7340c7200 0x7fa73414a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:23.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa73359e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa734148ac0 con 0x7fa734071f70 2024-09-13T17:51:23.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa73359e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa734147700 con 0x7fa7340c7200 2024-09-13T17:51:23.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa73259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa734071f70 0x7fa734148f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:23.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa73359e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa734147460 con 0x7fa734072350 2024-09-13T17:51:23.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa73259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa734071f70 0x7fa734148f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55122/0 (socket says 172.21.15.40:55122) 2024-09-13T17:51:23.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa73259c700 1 -- 172.21.15.40:0/2914507431 learned_addr learned my addr 172.21.15.40:0/2914507431 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:23.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa731d9b700 1 --2- 172.21.15.40:0/2914507431 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa734072350 0x7fa734149490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:23.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa732d9d700 1 --2- 172.21.15.40:0/2914507431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7340c7200 0x7fa73414a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:23.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa73259c700 1 -- 172.21.15.40:0/2914507431 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa734072350 msgr2=0x7fa734149490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:23.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa73259c700 1 --2- 172.21.15.40:0/2914507431 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa734072350 0x7fa734149490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa73259c700 1 -- 172.21.15.40:0/2914507431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7340c7200 msgr2=0x7fa73414a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:23.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa73259c700 1 --2- 172.21.15.40:0/2914507431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7340c7200 0x7fa73414a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa73259c700 1 -- 172.21.15.40:0/2914507431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7240094b0 con 0x7fa734071f70 2024-09-13T17:51:23.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa73259c700 1 --2- 172.21.15.40:0/2914507431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa734071f70 0x7fa734148f50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa72800c730 tx=0x7fa72800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:23.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa722ffd700 1 -- 172.21.15.40:0/2914507431 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728005d90 con 0x7fa734071f70 2024-09-13T17:51:23.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa722ffd700 1 -- 172.21.15.40:0/2914507431 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7280070d0 con 0x7fa734071f70 2024-09-13T17:51:23.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa73359e700 1 -- 172.21.15.40:0/2914507431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa734148190 con 0x7fa734071f70 2024-09-13T17:51:23.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa722ffd700 1 -- 172.21.15.40:0/2914507431 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728007460 con 0x7fa734071f70 2024-09-13T17:51:23.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.470+0000 7fa73359e700 1 -- 172.21.15.40:0/2914507431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa734148750 con 0x7fa734071f70 2024-09-13T17:51:23.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.474+0000 7fa722ffd700 1 -- 172.21.15.40:0/2914507431 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa7280075c0 con 0x7fa734071f70 2024-09-13T17:51:23.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.474+0000 7fa73359e700 1 -- 172.21.15.40:0/2914507431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73414cfc0 con 0x7fa734071f70 2024-09-13T17:51:23.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.474+0000 7fa722ffd700 1 --2- 172.21.15.40:0/2914507431 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa714066760 0x7fa714068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:23.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.474+0000 7fa731d9b700 1 --2- 172.21.15.40:0/2914507431 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa714066760 0x7fa714068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:23.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.474+0000 7fa722ffd700 1 -- 172.21.15.40:0/2914507431 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa728083ac0 con 0x7fa734071f70 2024-09-13T17:51:23.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.474+0000 7fa731d9b700 1 --2- 172.21.15.40:0/2914507431 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa714066760 0x7fa714068c20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa7240090b0 tx=0x7fa724008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:23.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.482+0000 7fa722ffd700 1 -- 172.21.15.40:0/2914507431 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa728050940 con 0x7fa734071f70 2024-09-13T17:51:23.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.714+0000 7fa73359e700 1 -- 172.21.15.40:0/2914507431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa7340cb6e0 con 0x7fa734071f70 2024-09-13T17:51:23.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.714+0000 7fa722ffd700 1 -- 172.21.15.40:0/2914507431 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1963 (secure 0 0 0) 0x7fa7280177c0 con 0x7fa734071f70 2024-09-13T17:51:23.720 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:23.720 INFO:teuthology.orchestra.run.smithi040.stdout:{"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":74,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":27,"num_osds":8,"num_up_osds":8,"osd_up_since":1726249821,"num_in_osds":8,"osd_in_since":1726249812,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129}],"num_pgs":129,"num_pools":4,"num_objects":62,"data_bytes":598119,"bytes_used":241741824,"bytes_avail":772852371456,"bytes_total":773094113280,"write_bytes_sec":1151,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":15,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"f","status":"up:active","gid":4279},{"filesystem_id":1,"rank":1,"name":"d","status":"up:active","gid":4285},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4288}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-13T17:51:23.014470+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-13T17:51:23.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.718+0000 7fa73359e700 1 -- 172.21.15.40:0/2914507431 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa714066760 msgr2=0x7fa714068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:23.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.718+0000 7fa73359e700 1 --2- 172.21.15.40:0/2914507431 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa714066760 0x7fa714068c20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa7240090b0 tx=0x7fa724008030 comp rx=0 tx=0).stop 2024-09-13T17:51:23.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.718+0000 7fa73359e700 1 -- 172.21.15.40:0/2914507431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa734071f70 msgr2=0x7fa734148f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:23.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.718+0000 7fa73359e700 1 --2- 172.21.15.40:0/2914507431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa734071f70 0x7fa734148f50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa72800c730 tx=0x7fa72800ca40 comp rx=0 tx=0).stop 2024-09-13T17:51:23.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.718+0000 7fa73359e700 1 -- 172.21.15.40:0/2914507431 shutdown_connections 2024-09-13T17:51:23.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.718+0000 7fa73359e700 1 --2- 172.21.15.40:0/2914507431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7340c7200 0x7fa73414a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.718+0000 7fa73359e700 1 --2- 172.21.15.40:0/2914507431 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa714066760 0x7fa714068c20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.718+0000 7fa73359e700 1 --2- 172.21.15.40:0/2914507431 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa734072350 0x7fa734149490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.718+0000 7fa73359e700 1 --2- 172.21.15.40:0/2914507431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa734071f70 0x7fa734148f50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.718+0000 7fa73359e700 1 -- 172.21.15.40:0/2914507431 >> 172.21.15.40:0/2914507431 conn(0x7fa7340cabc0 msgr2=0x7fa73414d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:23.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.718+0000 7fa73359e700 1 -- 172.21.15.40:0/2914507431 shutdown_connections 2024-09-13T17:51:23.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.718+0000 7fa73359e700 1 -- 172.21.15.40:0/2914507431 wait complete. 2024-09-13T17:51:23.735 INFO:tasks.mon_thrash.ceph_manager:no progress seen, keeping timeout for now 2024-09-13T17:51:23.735 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:51:23.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.842+0000 7f1ed2039700 1 Processor -- start 2024-09-13T17:51:23.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.842+0000 7f1ed2039700 1 -- start start 2024-09-13T17:51:23.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.842+0000 7f1ed2039700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ec40d5640 0x7f1ec4045700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:23.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.842+0000 7f1ed2039700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec40d5a20 con 0x7f1ec4050820 2024-09-13T17:51:23.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.842+0000 7f1ed2039700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec40dbe40 con 0x7f1ec40464b0 2024-09-13T17:51:23.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.842+0000 7f1ed2039700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec4035950 con 0x7f1ec40d5640 2024-09-13T17:51:23.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.842+0000 7f1ed0836700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1ec4050820 0x7f1ec4034b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44548/0 (socket says 172.21.15.40:44548) 2024-09-13T17:51:23.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.842+0000 7f1ed0836700 1 -- 172.21.15.40:0/2684802738 learned_addr learned my addr 172.21.15.40:0/2684802738 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:23.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.846+0000 7f1ed1037700 1 --2- 172.21.15.40:0/2684802738 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ec40d5640 0x7f1ec4045700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:23.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.846+0000 7f1ebffff700 1 -- 172.21.15.40:0/2684802738 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2677531197 0 0) 0x7f1ec40d5a20 con 0x7f1ec4050820 2024-09-13T17:51:23.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.846+0000 7f1ebffff700 1 -- 172.21.15.40:0/2684802738 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eb80035d0 con 0x7f1ec4050820 2024-09-13T17:51:23.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.846+0000 7f1ed1037700 1 -- 172.21.15.40:0/2684802738 >> v1:172.21.15.43:6789/0 conn(0x7f1ec40464b0 legacy=0x7f1ec4035240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:23.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.846+0000 7f1ed1037700 1 -- 172.21.15.40:0/2684802738 >> v1:172.21.15.40:6789/0 conn(0x7f1ec4050820 legacy=0x7f1ec4034b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:23.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.846+0000 7f1ed1037700 1 -- 172.21.15.40:0/2684802738 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec40dc120 con 0x7f1ec40d5640 2024-09-13T17:51:23.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.846+0000 7f1ed1037700 1 --2- 172.21.15.40:0/2684802738 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ec40d5640 0x7f1ec4045700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8009880 tx=0x7f1ec8009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=997c77368efdc23c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:23.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.846+0000 7f1ebffff700 1 -- 172.21.15.40:0/2684802738 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8011830 con 0x7f1ec40d5640 2024-09-13T17:51:23.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.846+0000 7f1ebffff700 1 -- 172.21.15.40:0/2684802738 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec8011990 con 0x7f1ec40d5640 2024-09-13T17:51:23.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.846+0000 7f1ebffff700 1 -- 172.21.15.40:0/2684802738 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8011d20 con 0x7f1ec40d5640 2024-09-13T17:51:23.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.846+0000 7f1ed2039700 1 -- 172.21.15.40:0/2684802738 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ec40d5640 msgr2=0x7f1ec4045700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:23.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.846+0000 7f1ed2039700 1 --2- 172.21.15.40:0/2684802738 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ec40d5640 0x7f1ec4045700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8009880 tx=0x7f1ec8009b90 comp rx=0 tx=0).stop 2024-09-13T17:51:23.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.846+0000 7f1ed2039700 1 -- 172.21.15.40:0/2684802738 shutdown_connections 2024-09-13T17:51:23.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.846+0000 7f1ed2039700 1 --2- 172.21.15.40:0/2684802738 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ec40d5640 0x7f1ec4045700 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.846+0000 7f1ed2039700 1 -- 172.21.15.40:0/2684802738 >> 172.21.15.40:0/2684802738 conn(0x7f1ec404fd00 msgr2=0x7f1ec4050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:23.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.846+0000 7f1ed2039700 1 -- 172.21.15.40:0/2684802738 shutdown_connections 2024-09-13T17:51:23.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.846+0000 7f1ed2039700 1 -- 172.21.15.40:0/2684802738 wait complete. 2024-09-13T17:51:23.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed2039700 1 Processor -- start 2024-09-13T17:51:23.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed2039700 1 -- start start 2024-09-13T17:51:23.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed2039700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ec40464b0 0x7f1ec40dd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:23.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed2039700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec4050820 0x7f1ec40dedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:23.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed2039700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ec40d5640 0x7f1ec40df310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:23.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed2039700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec4175660 con 0x7f1ec4050820 2024-09-13T17:51:23.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed2039700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec4175100 con 0x7f1ec40464b0 2024-09-13T17:51:23.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed2039700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec40ce4f0 con 0x7f1ec40d5640 2024-09-13T17:51:23.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed0836700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec4050820 0x7f1ec40dedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:23.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed0836700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec4050820 0x7f1ec40dedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55132/0 (socket says 172.21.15.40:55132) 2024-09-13T17:51:23.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed1037700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ec40464b0 0x7f1ec40dd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:23.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed0836700 1 -- 172.21.15.40:0/1165047571 learned_addr learned my addr 172.21.15.40:0/1165047571 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:23.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed1838700 1 --2- 172.21.15.40:0/1165047571 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ec40d5640 0x7f1ec40df310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:23.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed0836700 1 -- 172.21.15.40:0/1165047571 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ec40d5640 msgr2=0x7f1ec40df310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:23.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed0836700 1 --2- 172.21.15.40:0/1165047571 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ec40d5640 0x7f1ec40df310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed0836700 1 -- 172.21.15.40:0/1165047571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ec40464b0 msgr2=0x7f1ec40dd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:23.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed0836700 1 --2- 172.21.15.40:0/1165047571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ec40464b0 0x7f1ec40dd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:23.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed0836700 1 -- 172.21.15.40:0/1165047571 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec80094b0 con 0x7f1ec4050820 2024-09-13T17:51:23.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed0836700 1 --2- 172.21.15.40:0/1165047571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec4050820 0x7f1ec40dedd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000c4f0 tx=0x7f1ec000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:23.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ebdffb700 1 -- 172.21.15.40:0/1165047571 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0005d90 con 0x7f1ec4050820 2024-09-13T17:51:23.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ebdffb700 1 -- 172.21.15.40:0/1165047571 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec00070d0 con 0x7f1ec4050820 2024-09-13T17:51:23.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed2039700 1 -- 172.21.15.40:0/1165047571 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec40de440 con 0x7f1ec4050820 2024-09-13T17:51:23.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ebdffb700 1 -- 172.21.15.40:0/1165047571 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0007460 con 0x7f1ec4050820 2024-09-13T17:51:23.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.850+0000 7f1ed2039700 1 -- 172.21.15.40:0/1165047571 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec40dc580 con 0x7f1ec4050820 2024-09-13T17:51:23.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.854+0000 7f1ed2039700 1 -- 172.21.15.40:0/1165047571 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec4045700 con 0x7f1ec4050820 2024-09-13T17:51:23.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.854+0000 7f1ebdffb700 1 -- 172.21.15.40:0/1165047571 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1ec00075c0 con 0x7f1ec4050820 2024-09-13T17:51:23.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.854+0000 7f1ebdffb700 1 --2- 172.21.15.40:0/1165047571 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1eb0066730 0x7f1eb0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:23.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.854+0000 7f1ebdffb700 1 -- 172.21.15.40:0/1165047571 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1ec00838f0 con 0x7f1ec4050820 2024-09-13T17:51:23.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.854+0000 7f1ed1037700 1 --2- 172.21.15.40:0/1165047571 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1eb0066730 0x7f1eb0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:23.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.854+0000 7f1ed1037700 1 --2- 172.21.15.40:0/1165047571 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1eb0066730 0x7f1eb0068bf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8000c00 tx=0x7f1ec8010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:23.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:23.866+0000 7f1ebdffb700 1 -- 172.21.15.40:0/1165047571 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1ec00508a0 con 0x7f1ec4050820 2024-09-13T17:51:24.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:24.034+0000 7f1ed2039700 1 -- 172.21.15.40:0/1165047571 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ec40060c0 con 0x7f1eb0066730 2024-09-13T17:51:24.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:24.050+0000 7f1ebdffb700 1 -- 172.21.15.40:0/1165047571 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+349592 (secure 0 0 0) 0x7f1ec8011c00 con 0x7f1eb0066730 2024-09-13T17:51:24.065 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:24.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:24.054+0000 7f1ed2039700 1 -- 172.21.15.40:0/1165047571 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1eb0066730 msgr2=0x7f1eb0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:24.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:24.054+0000 7f1ed2039700 1 --2- 172.21.15.40:0/1165047571 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1eb0066730 0x7f1eb0068bf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8000c00 tx=0x7f1ec8010000 comp rx=0 tx=0).stop 2024-09-13T17:51:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:24.054+0000 7f1ed2039700 1 -- 172.21.15.40:0/1165047571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec4050820 msgr2=0x7f1ec40dedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:24.054+0000 7f1ed2039700 1 --2- 172.21.15.40:0/1165047571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec4050820 0x7f1ec40dedd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000c4f0 tx=0x7f1ec000c800 comp rx=0 tx=0).stop 2024-09-13T17:51:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:24.058+0000 7f1ed2039700 1 -- 172.21.15.40:0/1165047571 shutdown_connections 2024-09-13T17:51:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:24.058+0000 7f1ed2039700 1 --2- 172.21.15.40:0/1165047571 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1eb0066730 0x7f1eb0068bf0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:24.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:24.058+0000 7f1ed2039700 1 --2- 172.21.15.40:0/1165047571 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ec40d5640 0x7f1ec40df310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:24.058+0000 7f1ed2039700 1 --2- 172.21.15.40:0/1165047571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec4050820 0x7f1ec40dedd0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:24.058+0000 7f1ed2039700 1 --2- 172.21.15.40:0/1165047571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ec40464b0 0x7f1ec40dd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:24.058+0000 7f1ed2039700 1 -- 172.21.15.40:0/1165047571 >> 172.21.15.40:0/1165047571 conn(0x7f1ec404fd00 msgr2=0x7f1ec40351a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:24.058+0000 7f1ed2039700 1 -- 172.21.15.40:0/1165047571 shutdown_connections 2024-09-13T17:51:24.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:24.058+0000 7f1ed2039700 1 -- 172.21.15.40:0/1165047571 wait complete. 2024-09-13T17:51:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-13T17:51:24.077 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2024-09-13T17:51:23.013833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598119,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":288,"num_write_kb":2459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236076,"kb_used_data":3528,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738644,"statfs":{"total":773094113280,"available":772852371456,"internally_reserved":0,"allocated":3612672,"data_stored":2123630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1016,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002740"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282601+0000","last_change":"2024-09-13T17:50:34.212158+0000","last_active":"2024-09-13T17:51:15.282601+0000","last_peered":"2024-09-13T17:51:15.282601+0000","last_clean":"2024-09-13T17:51:15.282601+0000","last_became_active":"2024-09-13T17:50:34.211867+0000","last_became_peered":"2024-09-13T17:50:34.211867+0000","last_unstale":"2024-09-13T17:51:15.282601+0000","last_undegraded":"2024-09-13T17:51:15.282601+0000","last_fullsized":"2024-09-13T17:51:15.282601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:14.671095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271433+0000","last_change":"2024-09-13T17:50:34.587396+0000","last_active":"2024-09-13T17:51:15.271433+0000","last_peered":"2024-09-13T17:51:15.271433+0000","last_clean":"2024-09-13T17:51:15.271433+0000","last_became_active":"2024-09-13T17:50:34.587108+0000","last_became_peered":"2024-09-13T17:50:34.587108+0000","last_unstale":"2024-09-13T17:51:15.271433+0000","last_undegraded":"2024-09-13T17:51:15.271433+0000","last_fullsized":"2024-09-13T17:51:15.271433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:39:16.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270087+0000","last_change":"2024-09-13T17:50:38.615059+0000","last_active":"2024-09-13T17:51:15.270087+0000","last_peered":"2024-09-13T17:51:15.270087+0000","last_clean":"2024-09-13T17:51:15.270087+0000","last_became_active":"2024-09-13T17:50:38.614723+0000","last_became_peered":"2024-09-13T17:50:38.614723+0000","last_unstale":"2024-09-13T17:51:15.270087+0000","last_undegraded":"2024-09-13T17:51:15.270087+0000","last_fullsized":"2024-09-13T17:51:15.270087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:23:01.902181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271688+0000","last_change":"2024-09-13T17:50:34.204760+0000","last_active":"2024-09-13T17:51:15.271688+0000","last_peered":"2024-09-13T17:51:15.271688+0000","last_clean":"2024-09-13T17:51:15.271688+0000","last_became_active":"2024-09-13T17:50:34.202930+0000","last_became_peered":"2024-09-13T17:50:34.202930+0000","last_unstale":"2024-09-13T17:51:15.271688+0000","last_undegraded":"2024-09-13T17:51:15.271688+0000","last_fullsized":"2024-09-13T17:51:15.271688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:59.784701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.366352+0000","last_change":"2024-09-13T17:51:16.366352+0000","last_active":"2024-09-13T17:51:16.366352+0000","last_peered":"2024-09-13T17:51:16.366352+0000","last_clean":"2024-09-13T17:51:16.366352+0000","last_became_active":"2024-09-13T17:51:16.365492+0000","last_became_peered":"2024-09-13T17:51:16.365492+0000","last_unstale":"2024-09-13T17:51:16.366352+0000","last_undegraded":"2024-09-13T17:51:16.366352+0000","last_fullsized":"2024-09-13T17:51:16.366352+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:42.192182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270184+0000","last_change":"2024-09-13T17:50:38.618751+0000","last_active":"2024-09-13T17:51:15.270184+0000","last_peered":"2024-09-13T17:51:15.270184+0000","last_clean":"2024-09-13T17:51:15.270184+0000","last_became_active":"2024-09-13T17:50:38.618423+0000","last_became_peered":"2024-09-13T17:50:38.618423+0000","last_unstale":"2024-09-13T17:51:15.270184+0000","last_undegraded":"2024-09-13T17:51:15.270184+0000","last_fullsized":"2024-09-13T17:51:15.270184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:50:02.418785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"25'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279312+0000","last_change":"2024-09-13T17:50:34.214972+0000","last_active":"2024-09-13T17:51:15.279312+0000","last_peered":"2024-09-13T17:51:15.279312+0000","last_clean":"2024-09-13T17:51:15.279312+0000","last_became_active":"2024-09-13T17:50:34.214652+0000","last_became_peered":"2024-09-13T17:50:34.214652+0000","last_unstale":"2024-09-13T17:51:15.279312+0000","last_undegraded":"2024-09-13T17:51:15.279312+0000","last_fullsized":"2024-09-13T17:51:15.279312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:57.176854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.282672+0000","last_change":"2024-09-13T17:51:16.281831+0000","last_active":"2024-09-13T17:51:16.282672+0000","last_peered":"2024-09-13T17:51:16.282672+0000","last_clean":"2024-09-13T17:51:16.282672+0000","last_became_active":"2024-09-13T17:51:16.281549+0000","last_became_peered":"2024-09-13T17:51:16.281549+0000","last_unstale":"2024-09-13T17:51:16.282672+0000","last_undegraded":"2024-09-13T17:51:16.282672+0000","last_fullsized":"2024-09-13T17:51:16.282672+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:43.894437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.274476+0000","last_change":"2024-09-13T17:50:38.616219+0000","last_active":"2024-09-13T17:51:15.274476+0000","last_peered":"2024-09-13T17:51:15.274476+0000","last_clean":"2024-09-13T17:51:15.274476+0000","last_became_active":"2024-09-13T17:50:38.614329+0000","last_became_peered":"2024-09-13T17:50:38.614329+0000","last_unstale":"2024-09-13T17:51:15.274476+0000","last_undegraded":"2024-09-13T17:51:15.274476+0000","last_fullsized":"2024-09-13T17:51:15.274476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:25.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"25'2","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279079+0000","last_change":"2024-09-13T17:50:34.216123+0000","last_active":"2024-09-13T17:51:15.279079+0000","last_peered":"2024-09-13T17:51:15.279079+0000","last_clean":"2024-09-13T17:51:15.279079+0000","last_became_active":"2024-09-13T17:50:34.215661+0000","last_became_peered":"2024-09-13T17:50:34.215661+0000","last_unstale":"2024-09-13T17:51:15.279079+0000","last_undegraded":"2024-09-13T17:51:15.279079+0000","last_fullsized":"2024-09-13T17:51:15.279079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:20.895037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270446+0000","last_change":"2024-09-13T17:50:34.599673+0000","last_active":"2024-09-13T17:51:15.270446+0000","last_peered":"2024-09-13T17:51:15.270446+0000","last_clean":"2024-09-13T17:51:15.270446+0000","last_became_active":"2024-09-13T17:50:34.599409+0000","last_became_peered":"2024-09-13T17:50:34.599409+0000","last_unstale":"2024-09-13T17:51:15.270446+0000","last_undegraded":"2024-09-13T17:51:15.270446+0000","last_fullsized":"2024-09-13T17:51:15.270446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:03:31.394385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.274077+0000","last_change":"2024-09-13T17:50:38.616317+0000","last_active":"2024-09-13T17:51:15.274077+0000","last_peered":"2024-09-13T17:51:15.274077+0000","last_clean":"2024-09-13T17:51:15.274077+0000","last_became_active":"2024-09-13T17:50:38.614311+0000","last_became_peered":"2024-09-13T17:50:38.614311+0000","last_unstale":"2024-09-13T17:51:15.274077+0000","last_undegraded":"2024-09-13T17:51:15.274077+0000","last_fullsized":"2024-09-13T17:51:15.274077+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:29.221522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275452+0000","last_change":"2024-09-13T17:50:34.599490+0000","last_active":"2024-09-13T17:51:15.275452+0000","last_peered":"2024-09-13T17:51:15.275452+0000","last_clean":"2024-09-13T17:51:15.275452+0000","last_became_active":"2024-09-13T17:50:34.599207+0000","last_became_peered":"2024-09-13T17:50:34.599207+0000","last_unstale":"2024-09-13T17:51:15.275452+0000","last_undegraded":"2024-09-13T17:51:15.275452+0000","last_fullsized":"2024-09-13T17:51:15.275452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:11.178161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278915+0000","last_change":"2024-09-13T17:50:34.215697+0000","last_active":"2024-09-13T17:51:15.278915+0000","last_peered":"2024-09-13T17:51:15.278915+0000","last_clean":"2024-09-13T17:51:15.278915+0000","last_became_active":"2024-09-13T17:50:34.215409+0000","last_became_peered":"2024-09-13T17:50:34.215409+0000","last_unstale":"2024-09-13T17:51:15.278915+0000","last_undegraded":"2024-09-13T17:51:15.278915+0000","last_fullsized":"2024-09-13T17:51:15.278915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:21.609567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275240+0000","last_change":"2024-09-13T17:50:38.611817+0000","last_active":"2024-09-13T17:51:15.275240+0000","last_peered":"2024-09-13T17:51:15.275240+0000","last_clean":"2024-09-13T17:51:15.275240+0000","last_became_active":"2024-09-13T17:50:38.611263+0000","last_became_peered":"2024-09-13T17:50:38.611263+0000","last_unstale":"2024-09-13T17:51:15.275240+0000","last_undegraded":"2024-09-13T17:51:15.275240+0000","last_fullsized":"2024-09-13T17:51:15.275240+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:56:47.466672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271037+0000","last_change":"2024-09-13T17:50:34.586307+0000","last_active":"2024-09-13T17:51:15.271037+0000","last_peered":"2024-09-13T17:51:15.271037+0000","last_clean":"2024-09-13T17:51:15.271037+0000","last_became_active":"2024-09-13T17:50:34.585831+0000","last_became_peered":"2024-09-13T17:50:34.585831+0000","last_unstale":"2024-09-13T17:51:15.271037+0000","last_undegraded":"2024-09-13T17:51:15.271037+0000","last_fullsized":"2024-09-13T17:51:15.271037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:40.912352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.280577+0000","last_change":"2024-09-13T17:50:34.211850+0000","last_active":"2024-09-13T17:51:15.280577+0000","last_peered":"2024-09-13T17:51:15.280577+0000","last_clean":"2024-09-13T17:51:15.280577+0000","last_became_active":"2024-09-13T17:50:34.211594+0000","last_became_peered":"2024-09-13T17:50:34.211594+0000","last_unstale":"2024-09-13T17:51:15.280577+0000","last_undegraded":"2024-09-13T17:51:15.280577+0000","last_fullsized":"2024-09-13T17:51:15.280577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:35:02.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276065+0000","last_change":"2024-09-13T17:50:38.613576+0000","last_active":"2024-09-13T17:51:15.276065+0000","last_peered":"2024-09-13T17:51:15.276065+0000","last_clean":"2024-09-13T17:51:15.276065+0000","last_became_active":"2024-09-13T17:50:38.612577+0000","last_became_peered":"2024-09-13T17:50:38.612577+0000","last_unstale":"2024-09-13T17:51:15.276065+0000","last_undegraded":"2024-09-13T17:51:15.276065+0000","last_fullsized":"2024-09-13T17:51:15.276065+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:25.551060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272679+0000","last_change":"2024-09-13T17:50:34.587273+0000","last_active":"2024-09-13T17:51:15.272679+0000","last_peered":"2024-09-13T17:51:15.272679+0000","last_clean":"2024-09-13T17:51:15.272679+0000","last_became_active":"2024-09-13T17:50:34.587027+0000","last_became_peered":"2024-09-13T17:50:34.587027+0000","last_unstale":"2024-09-13T17:51:15.272679+0000","last_undegraded":"2024-09-13T17:51:15.272679+0000","last_fullsized":"2024-09-13T17:51:15.272679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:58.826737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.280692+0000","last_change":"2024-09-13T17:50:34.211845+0000","last_active":"2024-09-13T17:51:15.280692+0000","last_peered":"2024-09-13T17:51:15.280692+0000","last_clean":"2024-09-13T17:51:15.280692+0000","last_became_active":"2024-09-13T17:50:34.211568+0000","last_became_peered":"2024-09-13T17:50:34.211568+0000","last_unstale":"2024-09-13T17:51:15.280692+0000","last_undegraded":"2024-09-13T17:51:15.280692+0000","last_fullsized":"2024-09-13T17:51:15.280692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:56.348984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281699+0000","last_change":"2024-09-13T17:50:38.610292+0000","last_active":"2024-09-13T17:51:15.281699+0000","last_peered":"2024-09-13T17:51:15.281699+0000","last_clean":"2024-09-13T17:51:15.281699+0000","last_became_active":"2024-09-13T17:50:38.609832+0000","last_became_peered":"2024-09-13T17:50:38.609832+0000","last_unstale":"2024-09-13T17:51:15.281699+0000","last_undegraded":"2024-09-13T17:51:15.281699+0000","last_fullsized":"2024-09-13T17:51:15.281699+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:13.783068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.273946+0000","last_change":"2024-09-13T17:50:34.593566+0000","last_active":"2024-09-13T17:51:15.273946+0000","last_peered":"2024-09-13T17:51:15.273946+0000","last_clean":"2024-09-13T17:51:15.273946+0000","last_became_active":"2024-09-13T17:50:34.588794+0000","last_became_peered":"2024-09-13T17:50:34.588794+0000","last_unstale":"2024-09-13T17:51:15.273946+0000","last_undegraded":"2024-09-13T17:51:15.273946+0000","last_fullsized":"2024-09-13T17:51:15.273946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:13.192453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277322+0000","last_change":"2024-09-13T17:50:34.214638+0000","last_active":"2024-09-13T17:51:15.277322+0000","last_peered":"2024-09-13T17:51:15.277322+0000","last_clean":"2024-09-13T17:51:15.277322+0000","last_became_active":"2024-09-13T17:50:34.214292+0000","last_became_peered":"2024-09-13T17:50:34.214292+0000","last_unstale":"2024-09-13T17:51:15.277322+0000","last_undegraded":"2024-09-13T17:51:15.277322+0000","last_fullsized":"2024-09-13T17:51:15.277322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:57.377104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275146+0000","last_change":"2024-09-13T17:50:38.613368+0000","last_active":"2024-09-13T17:51:15.275146+0000","last_peered":"2024-09-13T17:51:15.275146+0000","last_clean":"2024-09-13T17:51:15.275146+0000","last_became_active":"2024-09-13T17:50:38.613010+0000","last_became_peered":"2024-09-13T17:50:38.613010+0000","last_unstale":"2024-09-13T17:51:15.275146+0000","last_undegraded":"2024-09-13T17:51:15.275146+0000","last_fullsized":"2024-09-13T17:51:15.275146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:42.706625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282530+0000","last_change":"2024-09-13T17:50:34.582327+0000","last_active":"2024-09-13T17:51:15.282530+0000","last_peered":"2024-09-13T17:51:15.282530+0000","last_clean":"2024-09-13T17:51:15.282530+0000","last_became_active":"2024-09-13T17:50:34.582050+0000","last_became_peered":"2024-09-13T17:50:34.582050+0000","last_unstale":"2024-09-13T17:51:15.282530+0000","last_undegraded":"2024-09-13T17:51:15.282530+0000","last_fullsized":"2024-09-13T17:51:15.282530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:52.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277006+0000","last_change":"2024-09-13T17:50:34.211512+0000","last_active":"2024-09-13T17:51:15.277006+0000","last_peered":"2024-09-13T17:51:15.277006+0000","last_clean":"2024-09-13T17:51:15.277006+0000","last_became_active":"2024-09-13T17:50:34.211230+0000","last_became_peered":"2024-09-13T17:50:34.211230+0000","last_unstale":"2024-09-13T17:51:15.277006+0000","last_undegraded":"2024-09-13T17:51:15.277006+0000","last_fullsized":"2024-09-13T17:51:15.277006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:40.707777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272462+0000","last_change":"2024-09-13T17:50:38.608255+0000","last_active":"2024-09-13T17:51:15.272462+0000","last_peered":"2024-09-13T17:51:15.272462+0000","last_clean":"2024-09-13T17:51:15.272462+0000","last_became_active":"2024-09-13T17:50:38.606697+0000","last_became_peered":"2024-09-13T17:50:38.606697+0000","last_unstale":"2024-09-13T17:51:15.272462+0000","last_undegraded":"2024-09-13T17:51:15.272462+0000","last_fullsized":"2024-09-13T17:51:15.272462+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:47.265007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272916+0000","last_change":"2024-09-13T17:50:34.594476+0000","last_active":"2024-09-13T17:51:15.272916+0000","last_peered":"2024-09-13T17:51:15.272916+0000","last_clean":"2024-09-13T17:51:15.272916+0000","last_became_active":"2024-09-13T17:50:34.594215+0000","last_became_peered":"2024-09-13T17:50:34.594215+0000","last_unstale":"2024-09-13T17:51:15.272916+0000","last_undegraded":"2024-09-13T17:51:15.272916+0000","last_fullsized":"2024-09-13T17:51:15.272916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:02:27.528019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281479+0000","last_change":"2024-09-13T17:50:34.204018+0000","last_active":"2024-09-13T17:51:15.281479+0000","last_peered":"2024-09-13T17:51:15.281479+0000","last_clean":"2024-09-13T17:51:15.281479+0000","last_became_active":"2024-09-13T17:50:34.203702+0000","last_became_peered":"2024-09-13T17:50:34.203702+0000","last_unstale":"2024-09-13T17:51:15.281479+0000","last_undegraded":"2024-09-13T17:51:15.281479+0000","last_fullsized":"2024-09-13T17:51:15.281479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:44.431226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.365782+0000","last_change":"2024-09-13T17:51:16.365782+0000","last_active":"2024-09-13T17:51:16.365782+0000","last_peered":"2024-09-13T17:51:16.365782+0000","last_clean":"2024-09-13T17:51:16.365782+0000","last_became_active":"2024-09-13T17:51:16.364540+0000","last_became_peered":"2024-09-13T17:51:16.364540+0000","last_unstale":"2024-09-13T17:51:16.365782+0000","last_undegraded":"2024-09-13T17:51:16.365782+0000","last_fullsized":"2024-09-13T17:51:16.365782+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:18.753557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281218+0000","last_change":"2024-09-13T17:50:34.586802+0000","last_active":"2024-09-13T17:51:15.281218+0000","last_peered":"2024-09-13T17:51:15.281218+0000","last_clean":"2024-09-13T17:51:15.281218+0000","last_became_active":"2024-09-13T17:50:34.586552+0000","last_became_peered":"2024-09-13T17:50:34.586552+0000","last_unstale":"2024-09-13T17:51:15.281218+0000","last_undegraded":"2024-09-13T17:51:15.281218+0000","last_fullsized":"2024-09-13T17:51:15.281218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:43.383975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279434+0000","last_change":"2024-09-13T17:50:34.217083+0000","last_active":"2024-09-13T17:51:15.279434+0000","last_peered":"2024-09-13T17:51:15.279434+0000","last_clean":"2024-09-13T17:51:15.279434+0000","last_became_active":"2024-09-13T17:50:34.216758+0000","last_became_peered":"2024-09-13T17:50:34.216758+0000","last_unstale":"2024-09-13T17:51:15.279434+0000","last_undegraded":"2024-09-13T17:51:15.279434+0000","last_fullsized":"2024-09-13T17:51:15.279434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:36.473942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277926+0000","last_change":"2024-09-13T17:50:38.614667+0000","last_active":"2024-09-13T17:51:15.277926+0000","last_peered":"2024-09-13T17:51:15.277926+0000","last_clean":"2024-09-13T17:51:15.277926+0000","last_became_active":"2024-09-13T17:50:38.614284+0000","last_became_peered":"2024-09-13T17:50:38.614284+0000","last_unstale":"2024-09-13T17:51:15.277926+0000","last_undegraded":"2024-09-13T17:51:15.277926+0000","last_fullsized":"2024-09-13T17:51:15.277926+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:29.616794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270498+0000","last_change":"2024-09-13T17:50:34.588603+0000","last_active":"2024-09-13T17:51:15.270498+0000","last_peered":"2024-09-13T17:51:15.270498+0000","last_clean":"2024-09-13T17:51:15.270498+0000","last_became_active":"2024-09-13T17:50:34.588317+0000","last_became_peered":"2024-09-13T17:50:34.588317+0000","last_unstale":"2024-09-13T17:51:15.270498+0000","last_undegraded":"2024-09-13T17:51:15.270498+0000","last_fullsized":"2024-09-13T17:51:15.270498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:03.743130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281302+0000","last_change":"2024-09-13T17:50:34.204335+0000","last_active":"2024-09-13T17:51:15.281302+0000","last_peered":"2024-09-13T17:51:15.281302+0000","last_clean":"2024-09-13T17:51:15.281302+0000","last_became_active":"2024-09-13T17:50:34.204060+0000","last_became_peered":"2024-09-13T17:50:34.204060+0000","last_unstale":"2024-09-13T17:51:15.281302+0000","last_undegraded":"2024-09-13T17:51:15.281302+0000","last_fullsized":"2024-09-13T17:51:15.281302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:54:41.831344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279638+0000","last_change":"2024-09-13T17:50:38.619590+0000","last_active":"2024-09-13T17:51:15.279638+0000","last_peered":"2024-09-13T17:51:15.279638+0000","last_clean":"2024-09-13T17:51:15.279638+0000","last_became_active":"2024-09-13T17:50:38.617257+0000","last_became_peered":"2024-09-13T17:50:38.617257+0000","last_unstale":"2024-09-13T17:51:15.279638+0000","last_undegraded":"2024-09-13T17:51:15.279638+0000","last_fullsized":"2024-09-13T17:51:15.279638+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:34.870291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275058+0000","last_change":"2024-09-13T17:50:34.588466+0000","last_active":"2024-09-13T17:51:15.275058+0000","last_peered":"2024-09-13T17:51:15.275058+0000","last_clean":"2024-09-13T17:51:15.275058+0000","last_became_active":"2024-09-13T17:50:34.588063+0000","last_became_peered":"2024-09-13T17:50:34.588063+0000","last_unstale":"2024-09-13T17:51:15.275058+0000","last_undegraded":"2024-09-13T17:51:15.275058+0000","last_fullsized":"2024-09-13T17:51:15.275058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:43:00.669203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270012+0000","last_change":"2024-09-13T17:50:34.209239+0000","last_active":"2024-09-13T17:51:15.270012+0000","last_peered":"2024-09-13T17:51:15.270012+0000","last_clean":"2024-09-13T17:51:15.270012+0000","last_became_active":"2024-09-13T17:50:34.208196+0000","last_became_peered":"2024-09-13T17:50:34.208196+0000","last_unstale":"2024-09-13T17:51:15.270012+0000","last_undegraded":"2024-09-13T17:51:15.270012+0000","last_fullsized":"2024-09-13T17:51:15.270012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:39.253917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281566+0000","last_change":"2024-09-13T17:50:38.611021+0000","last_active":"2024-09-13T17:51:15.281566+0000","last_peered":"2024-09-13T17:51:15.281566+0000","last_clean":"2024-09-13T17:51:15.281566+0000","last_became_active":"2024-09-13T17:50:38.609794+0000","last_became_peered":"2024-09-13T17:50:38.609794+0000","last_unstale":"2024-09-13T17:51:15.281566+0000","last_undegraded":"2024-09-13T17:51:15.281566+0000","last_fullsized":"2024-09-13T17:51:15.281566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:59.980236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281553+0000","last_change":"2024-09-13T17:50:34.584290+0000","last_active":"2024-09-13T17:51:15.281553+0000","last_peered":"2024-09-13T17:51:15.281553+0000","last_clean":"2024-09-13T17:51:15.281553+0000","last_became_active":"2024-09-13T17:50:34.584023+0000","last_became_peered":"2024-09-13T17:50:34.584023+0000","last_unstale":"2024-09-13T17:51:15.281553+0000","last_undegraded":"2024-09-13T17:51:15.281553+0000","last_fullsized":"2024-09-13T17:51:15.281553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:11.304986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"25'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278819+0000","last_change":"2024-09-13T17:50:34.198932+0000","last_active":"2024-09-13T17:51:15.278819+0000","last_peered":"2024-09-13T17:51:15.278819+0000","last_clean":"2024-09-13T17:51:15.278819+0000","last_became_active":"2024-09-13T17:50:34.198603+0000","last_became_peered":"2024-09-13T17:50:34.198603+0000","last_unstale":"2024-09-13T17:51:15.278819+0000","last_undegraded":"2024-09-13T17:51:15.278819+0000","last_fullsized":"2024-09-13T17:51:15.278819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:00.917126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281714+0000","last_change":"2024-09-13T17:50:38.611326+0000","last_active":"2024-09-13T17:51:15.281714+0000","last_peered":"2024-09-13T17:51:15.281714+0000","last_clean":"2024-09-13T17:51:15.281714+0000","last_became_active":"2024-09-13T17:50:38.610900+0000","last_became_peered":"2024-09-13T17:50:38.610900+0000","last_unstale":"2024-09-13T17:51:15.281714+0000","last_undegraded":"2024-09-13T17:51:15.281714+0000","last_fullsized":"2024-09-13T17:51:15.281714+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:17:09.994736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276990+0000","last_change":"2024-09-13T17:50:34.594482+0000","last_active":"2024-09-13T17:51:15.276990+0000","last_peered":"2024-09-13T17:51:15.276990+0000","last_clean":"2024-09-13T17:51:15.276990+0000","last_became_active":"2024-09-13T17:50:34.594201+0000","last_became_peered":"2024-09-13T17:50:34.594201+0000","last_unstale":"2024-09-13T17:51:15.276990+0000","last_undegraded":"2024-09-13T17:51:15.276990+0000","last_fullsized":"2024-09-13T17:51:15.276990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:06.697507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277494+0000","last_change":"2024-09-13T17:50:34.218376+0000","last_active":"2024-09-13T17:51:15.277494+0000","last_peered":"2024-09-13T17:51:15.277494+0000","last_clean":"2024-09-13T17:51:15.277494+0000","last_became_active":"2024-09-13T17:50:34.218069+0000","last_became_peered":"2024-09-13T17:50:34.218069+0000","last_unstale":"2024-09-13T17:51:15.277494+0000","last_undegraded":"2024-09-13T17:51:15.277494+0000","last_fullsized":"2024-09-13T17:51:15.277494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:53:54.288969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277801+0000","last_change":"2024-09-13T17:50:38.613206+0000","last_active":"2024-09-13T17:51:15.277801+0000","last_peered":"2024-09-13T17:51:15.277801+0000","last_clean":"2024-09-13T17:51:15.277801+0000","last_became_active":"2024-09-13T17:50:38.612856+0000","last_became_peered":"2024-09-13T17:50:38.612856+0000","last_unstale":"2024-09-13T17:51:15.277801+0000","last_undegraded":"2024-09-13T17:51:15.277801+0000","last_fullsized":"2024-09-13T17:51:15.277801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:31:20.190996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270767+0000","last_change":"2024-09-13T17:50:34.582931+0000","last_active":"2024-09-13T17:51:15.270767+0000","last_peered":"2024-09-13T17:51:15.270767+0000","last_clean":"2024-09-13T17:51:15.270767+0000","last_became_active":"2024-09-13T17:50:34.582446+0000","last_became_peered":"2024-09-13T17:50:34.582446+0000","last_unstale":"2024-09-13T17:51:15.270767+0000","last_undegraded":"2024-09-13T17:51:15.270767+0000","last_fullsized":"2024-09-13T17:51:15.270767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:30.554785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270225+0000","last_change":"2024-09-13T17:50:34.201237+0000","last_active":"2024-09-13T17:51:15.270225+0000","last_peered":"2024-09-13T17:51:15.270225+0000","last_clean":"2024-09-13T17:51:15.270225+0000","last_became_active":"2024-09-13T17:50:34.200873+0000","last_became_peered":"2024-09-13T17:50:34.200873+0000","last_unstale":"2024-09-13T17:51:15.270225+0000","last_undegraded":"2024-09-13T17:51:15.270225+0000","last_fullsized":"2024-09-13T17:51:15.270225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:02.711342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277589+0000","last_change":"2024-09-13T17:50:22.095367+0000","last_active":"2024-09-13T17:51:15.277589+0000","last_peered":"2024-09-13T17:51:15.277589+0000","last_clean":"2024-09-13T17:51:15.277589+0000","last_became_active":"2024-09-13T17:50:22.093734+0000","last_became_peered":"2024-09-13T17:50:22.093734+0000","last_unstale":"2024-09-13T17:51:15.277589+0000","last_undegraded":"2024-09-13T17:51:15.277589+0000","last_fullsized":"2024-09-13T17:51:15.277589+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_clean_scrub_stamp":"2024-09-13T17:50:21.057812+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:23.531613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276680+0000","last_change":"2024-09-13T17:50:38.618880+0000","last_active":"2024-09-13T17:51:15.276680+0000","last_peered":"2024-09-13T17:51:15.276680+0000","last_clean":"2024-09-13T17:51:15.276680+0000","last_became_active":"2024-09-13T17:50:38.618488+0000","last_became_peered":"2024-09-13T17:50:38.618488+0000","last_unstale":"2024-09-13T17:51:15.276680+0000","last_undegraded":"2024-09-13T17:51:15.276680+0000","last_fullsized":"2024-09-13T17:51:15.276680+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:03.431876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279141+0000","last_change":"2024-09-13T17:50:34.585881+0000","last_active":"2024-09-13T17:51:15.279141+0000","last_peered":"2024-09-13T17:51:15.279141+0000","last_clean":"2024-09-13T17:51:15.279141+0000","last_became_active":"2024-09-13T17:50:34.585166+0000","last_became_peered":"2024-09-13T17:50:34.585166+0000","last_unstale":"2024-09-13T17:51:15.279141+0000","last_undegraded":"2024-09-13T17:51:15.279141+0000","last_fullsized":"2024-09-13T17:51:15.279141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:06.654658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279549+0000","last_change":"2024-09-13T17:50:34.210934+0000","last_active":"2024-09-13T17:51:15.279549+0000","last_peered":"2024-09-13T17:51:15.279549+0000","last_clean":"2024-09-13T17:51:15.279549+0000","last_became_active":"2024-09-13T17:50:34.210589+0000","last_became_peered":"2024-09-13T17:50:34.210589+0000","last_unstale":"2024-09-13T17:51:15.279549+0000","last_undegraded":"2024-09-13T17:51:15.279549+0000","last_fullsized":"2024-09-13T17:51:15.279549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:26.213295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.364911+0000","last_change":"2024-09-13T17:51:16.364911+0000","last_active":"2024-09-13T17:51:16.364911+0000","last_peered":"2024-09-13T17:51:16.364911+0000","last_clean":"2024-09-13T17:51:16.364911+0000","last_became_active":"2024-09-13T17:51:16.363974+0000","last_became_peered":"2024-09-13T17:51:16.363974+0000","last_unstale":"2024-09-13T17:51:16.364911+0000","last_undegraded":"2024-09-13T17:51:16.364911+0000","last_fullsized":"2024-09-13T17:51:16.364911+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:12.730197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271986+0000","last_change":"2024-09-13T17:50:34.590169+0000","last_active":"2024-09-13T17:51:15.271986+0000","last_peered":"2024-09-13T17:51:15.271986+0000","last_clean":"2024-09-13T17:51:15.271986+0000","last_became_active":"2024-09-13T17:50:34.589434+0000","last_became_peered":"2024-09-13T17:50:34.589434+0000","last_unstale":"2024-09-13T17:51:15.271986+0000","last_undegraded":"2024-09-13T17:51:15.271986+0000","last_fullsized":"2024-09-13T17:51:15.271986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:12.545412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"25'1","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281689+0000","last_change":"2024-09-13T17:50:34.204269+0000","last_active":"2024-09-13T17:51:15.281689+0000","last_peered":"2024-09-13T17:51:15.281689+0000","last_clean":"2024-09-13T17:51:15.281689+0000","last_became_active":"2024-09-13T17:50:34.203874+0000","last_became_peered":"2024-09-13T17:50:34.203874+0000","last_unstale":"2024-09-13T17:51:15.281689+0000","last_undegraded":"2024-09-13T17:51:15.281689+0000","last_fullsized":"2024-09-13T17:51:15.281689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:10.410267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278070+0000","last_change":"2024-09-13T17:50:38.615989+0000","last_active":"2024-09-13T17:51:15.278070+0000","last_peered":"2024-09-13T17:51:15.278070+0000","last_clean":"2024-09-13T17:51:15.278070+0000","last_became_active":"2024-09-13T17:50:38.615530+0000","last_became_peered":"2024-09-13T17:50:38.615530+0000","last_unstale":"2024-09-13T17:51:15.278070+0000","last_undegraded":"2024-09-13T17:51:15.278070+0000","last_fullsized":"2024-09-13T17:51:15.278070+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:41.064092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279600+0000","last_change":"2024-09-13T17:50:34.595120+0000","last_active":"2024-09-13T17:51:15.279600+0000","last_peered":"2024-09-13T17:51:15.279600+0000","last_clean":"2024-09-13T17:51:15.279600+0000","last_became_active":"2024-09-13T17:50:34.594842+0000","last_became_peered":"2024-09-13T17:50:34.594842+0000","last_unstale":"2024-09-13T17:51:15.279600+0000","last_undegraded":"2024-09-13T17:51:15.279600+0000","last_fullsized":"2024-09-13T17:51:15.279600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:15:38.982524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"25'2","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279806+0000","last_change":"2024-09-13T17:50:34.199957+0000","last_active":"2024-09-13T17:51:15.279806+0000","last_peered":"2024-09-13T17:51:15.279806+0000","last_clean":"2024-09-13T17:51:15.279806+0000","last_became_active":"2024-09-13T17:50:34.199650+0000","last_became_peered":"2024-09-13T17:50:34.199650+0000","last_unstale":"2024-09-13T17:51:15.279806+0000","last_undegraded":"2024-09-13T17:51:15.279806+0000","last_fullsized":"2024-09-13T17:51:15.279806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:26.929549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269062+0000","last_change":"2024-09-13T17:50:38.609731+0000","last_active":"2024-09-13T17:51:15.269062+0000","last_peered":"2024-09-13T17:51:15.269062+0000","last_clean":"2024-09-13T17:51:15.269062+0000","last_became_active":"2024-09-13T17:50:38.609354+0000","last_became_peered":"2024-09-13T17:50:38.609354+0000","last_unstale":"2024-09-13T17:51:15.269062+0000","last_undegraded":"2024-09-13T17:51:15.269062+0000","last_fullsized":"2024-09-13T17:51:15.269062+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:14.142369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279334+0000","last_change":"2024-09-13T17:50:34.595146+0000","last_active":"2024-09-13T17:51:15.279334+0000","last_peered":"2024-09-13T17:51:15.279334+0000","last_clean":"2024-09-13T17:51:15.279334+0000","last_became_active":"2024-09-13T17:50:34.594876+0000","last_became_peered":"2024-09-13T17:50:34.594876+0000","last_unstale":"2024-09-13T17:51:15.279334+0000","last_undegraded":"2024-09-13T17:51:15.279334+0000","last_fullsized":"2024-09-13T17:51:15.279334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:26:10.638438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'2","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.278874+0000","last_change":"2024-09-13T17:50:34.200640+0000","last_active":"2024-09-13T17:51:16.278874+0000","last_peered":"2024-09-13T17:51:16.278874+0000","last_clean":"2024-09-13T17:51:16.278874+0000","last_became_active":"2024-09-13T17:50:34.200197+0000","last_became_peered":"2024-09-13T17:50:34.200197+0000","last_unstale":"2024-09-13T17:51:16.278874+0000","last_undegraded":"2024-09-13T17:51:16.278874+0000","last_fullsized":"2024-09-13T17:51:16.278874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:37:53.039720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.281995+0000","last_change":"2024-09-13T17:51:16.281995+0000","last_active":"2024-09-13T17:51:16.281995+0000","last_peered":"2024-09-13T17:51:16.281995+0000","last_clean":"2024-09-13T17:51:16.281995+0000","last_became_active":"2024-09-13T17:51:16.281476+0000","last_became_peered":"2024-09-13T17:51:16.281476+0000","last_unstale":"2024-09-13T17:51:16.281995+0000","last_undegraded":"2024-09-13T17:51:16.281995+0000","last_fullsized":"2024-09-13T17:51:16.281995+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:21:56.254641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3,6],"acting":[4,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278650+0000","last_change":"2024-09-13T17:50:34.585741+0000","last_active":"2024-09-13T17:51:15.278650+0000","last_peered":"2024-09-13T17:51:15.278650+0000","last_clean":"2024-09-13T17:51:15.278650+0000","last_became_active":"2024-09-13T17:50:34.585464+0000","last_became_peered":"2024-09-13T17:50:34.585464+0000","last_unstale":"2024-09-13T17:51:15.278650+0000","last_undegraded":"2024-09-13T17:51:15.278650+0000","last_fullsized":"2024-09-13T17:51:15.278650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:59:22.031459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"25'15","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278697+0000","last_change":"2024-09-13T17:50:34.201272+0000","last_active":"2024-09-13T17:51:15.278697+0000","last_peered":"2024-09-13T17:51:15.278697+0000","last_clean":"2024-09-13T17:51:15.278697+0000","last_became_active":"2024-09-13T17:50:34.200936+0000","last_became_peered":"2024-09-13T17:50:34.200936+0000","last_unstale":"2024-09-13T17:51:15.278697+0000","last_undegraded":"2024-09-13T17:51:15.278697+0000","last_fullsized":"2024-09-13T17:51:15.278697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:52.094879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277648+0000","last_change":"2024-09-13T17:50:38.615229+0000","last_active":"2024-09-13T17:51:15.277648+0000","last_peered":"2024-09-13T17:51:15.277648+0000","last_clean":"2024-09-13T17:51:15.277648+0000","last_became_active":"2024-09-13T17:50:38.614739+0000","last_became_peered":"2024-09-13T17:50:38.614739+0000","last_unstale":"2024-09-13T17:51:15.277648+0000","last_undegraded":"2024-09-13T17:51:15.277648+0000","last_fullsized":"2024-09-13T17:51:15.277648+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:18.619529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275934+0000","last_change":"2024-09-13T17:50:34.588848+0000","last_active":"2024-09-13T17:51:15.275934+0000","last_peered":"2024-09-13T17:51:15.275934+0000","last_clean":"2024-09-13T17:51:15.275934+0000","last_became_active":"2024-09-13T17:50:34.588565+0000","last_became_peered":"2024-09-13T17:50:34.588565+0000","last_unstale":"2024-09-13T17:51:15.275934+0000","last_undegraded":"2024-09-13T17:51:15.275934+0000","last_fullsized":"2024-09-13T17:51:15.275934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:47.927865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278190+0000","last_change":"2024-09-13T17:50:34.199008+0000","last_active":"2024-09-13T17:51:15.278190+0000","last_peered":"2024-09-13T17:51:15.278190+0000","last_clean":"2024-09-13T17:51:15.278190+0000","last_became_active":"2024-09-13T17:50:34.197839+0000","last_became_peered":"2024-09-13T17:50:34.197839+0000","last_unstale":"2024-09-13T17:51:15.278190+0000","last_undegraded":"2024-09-13T17:51:15.278190+0000","last_fullsized":"2024-09-13T17:51:15.278190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:51.430710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272409+0000","last_change":"2024-09-13T17:50:38.611080+0000","last_active":"2024-09-13T17:51:15.272409+0000","last_peered":"2024-09-13T17:51:15.272409+0000","last_clean":"2024-09-13T17:51:15.272409+0000","last_became_active":"2024-09-13T17:50:38.610688+0000","last_became_peered":"2024-09-13T17:50:38.610688+0000","last_unstale":"2024-09-13T17:51:15.272409+0000","last_undegraded":"2024-09-13T17:51:15.272409+0000","last_fullsized":"2024-09-13T17:51:15.272409+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:28.527617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278851+0000","last_change":"2024-09-13T17:50:34.594763+0000","last_active":"2024-09-13T17:51:15.278851+0000","last_peered":"2024-09-13T17:51:15.278851+0000","last_clean":"2024-09-13T17:51:15.278851+0000","last_became_active":"2024-09-13T17:50:34.594471+0000","last_became_peered":"2024-09-13T17:50:34.594471+0000","last_unstale":"2024-09-13T17:51:15.278851+0000","last_undegraded":"2024-09-13T17:51:15.278851+0000","last_fullsized":"2024-09-13T17:51:15.278851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:20:58.718271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269946+0000","last_change":"2024-09-13T17:50:34.209432+0000","last_active":"2024-09-13T17:51:15.269946+0000","last_peered":"2024-09-13T17:51:15.269946+0000","last_clean":"2024-09-13T17:51:15.269946+0000","last_became_active":"2024-09-13T17:50:34.208817+0000","last_became_peered":"2024-09-13T17:50:34.208817+0000","last_unstale":"2024-09-13T17:51:15.269946+0000","last_undegraded":"2024-09-13T17:51:15.269946+0000","last_fullsized":"2024-09-13T17:51:15.269946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:49.587528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272159+0000","last_change":"2024-09-13T17:50:38.609461+0000","last_active":"2024-09-13T17:51:15.272159+0000","last_peered":"2024-09-13T17:51:15.272159+0000","last_clean":"2024-09-13T17:51:15.272159+0000","last_became_active":"2024-09-13T17:50:38.609128+0000","last_became_peered":"2024-09-13T17:50:38.609128+0000","last_unstale":"2024-09-13T17:51:15.272159+0000","last_undegraded":"2024-09-13T17:51:15.272159+0000","last_fullsized":"2024-09-13T17:51:15.272159+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:02.286825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.283064+0000","last_change":"2024-09-13T17:50:34.592970+0000","last_active":"2024-09-13T17:51:15.283064+0000","last_peered":"2024-09-13T17:51:15.283064+0000","last_clean":"2024-09-13T17:51:15.283064+0000","last_became_active":"2024-09-13T17:50:34.592669+0000","last_became_peered":"2024-09-13T17:50:34.592669+0000","last_unstale":"2024-09-13T17:51:15.283064+0000","last_undegraded":"2024-09-13T17:51:15.283064+0000","last_fullsized":"2024-09-13T17:51:15.283064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:52.031261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282599+0000","last_change":"2024-09-13T17:50:34.200798+0000","last_active":"2024-09-13T17:51:15.282599+0000","last_peered":"2024-09-13T17:51:15.282599+0000","last_clean":"2024-09-13T17:51:15.282599+0000","last_became_active":"2024-09-13T17:50:34.200529+0000","last_became_peered":"2024-09-13T17:50:34.200529+0000","last_unstale":"2024-09-13T17:51:15.282599+0000","last_undegraded":"2024-09-13T17:51:15.282599+0000","last_fullsized":"2024-09-13T17:51:15.282599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:26.334296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270388+0000","last_change":"2024-09-13T17:50:38.606287+0000","last_active":"2024-09-13T17:51:15.270388+0000","last_peered":"2024-09-13T17:51:15.270388+0000","last_clean":"2024-09-13T17:51:15.270388+0000","last_became_active":"2024-09-13T17:50:38.605826+0000","last_became_peered":"2024-09-13T17:50:38.605826+0000","last_unstale":"2024-09-13T17:51:15.270388+0000","last_undegraded":"2024-09-13T17:51:15.270388+0000","last_fullsized":"2024-09-13T17:51:15.270388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:18.709454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279140+0000","last_change":"2024-09-13T17:50:34.591579+0000","last_active":"2024-09-13T17:51:15.279140+0000","last_peered":"2024-09-13T17:51:15.279140+0000","last_clean":"2024-09-13T17:51:15.279140+0000","last_became_active":"2024-09-13T17:50:34.586165+0000","last_became_peered":"2024-09-13T17:50:34.586165+0000","last_unstale":"2024-09-13T17:51:15.279140+0000","last_undegraded":"2024-09-13T17:51:15.279140+0000","last_fullsized":"2024-09-13T17:51:15.279140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:56:19.205176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'11","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277643+0000","last_change":"2024-09-13T17:50:34.203218+0000","last_active":"2024-09-13T17:51:15.277643+0000","last_peered":"2024-09-13T17:51:15.277643+0000","last_clean":"2024-09-13T17:51:15.277643+0000","last_became_active":"2024-09-13T17:50:34.202948+0000","last_became_peered":"2024-09-13T17:50:34.202948+0000","last_unstale":"2024-09-13T17:51:15.277643+0000","last_undegraded":"2024-09-13T17:51:15.277643+0000","last_fullsized":"2024-09-13T17:51:15.277643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:46.535569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.280382+0000","last_change":"2024-09-13T17:50:38.610597+0000","last_active":"2024-09-13T17:51:15.280382+0000","last_peered":"2024-09-13T17:51:15.280382+0000","last_clean":"2024-09-13T17:51:15.280382+0000","last_became_active":"2024-09-13T17:50:38.610055+0000","last_became_peered":"2024-09-13T17:50:38.610055+0000","last_unstale":"2024-09-13T17:51:15.280382+0000","last_undegraded":"2024-09-13T17:51:15.280382+0000","last_fullsized":"2024-09-13T17:51:15.280382+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:55.754936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275920+0000","last_change":"2024-09-13T17:50:34.593963+0000","last_active":"2024-09-13T17:51:15.275920+0000","last_peered":"2024-09-13T17:51:15.275920+0000","last_clean":"2024-09-13T17:51:15.275920+0000","last_became_active":"2024-09-13T17:50:34.593454+0000","last_became_peered":"2024-09-13T17:50:34.593454+0000","last_unstale":"2024-09-13T17:51:15.275920+0000","last_undegraded":"2024-09-13T17:51:15.275920+0000","last_fullsized":"2024-09-13T17:51:15.275920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:18.883521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281506+0000","last_change":"2024-09-13T17:50:34.207936+0000","last_active":"2024-09-13T17:51:15.281506+0000","last_peered":"2024-09-13T17:51:15.281506+0000","last_clean":"2024-09-13T17:51:15.281506+0000","last_became_active":"2024-09-13T17:50:34.207605+0000","last_became_peered":"2024-09-13T17:50:34.207605+0000","last_unstale":"2024-09-13T17:51:15.281506+0000","last_undegraded":"2024-09-13T17:51:15.281506+0000","last_fullsized":"2024-09-13T17:51:15.281506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:05.494406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.274690+0000","last_change":"2024-09-13T17:50:38.616297+0000","last_active":"2024-09-13T17:51:15.274690+0000","last_peered":"2024-09-13T17:51:15.274690+0000","last_clean":"2024-09-13T17:51:15.274690+0000","last_became_active":"2024-09-13T17:50:38.614165+0000","last_became_peered":"2024-09-13T17:50:38.614165+0000","last_unstale":"2024-09-13T17:51:15.274690+0000","last_undegraded":"2024-09-13T17:51:15.274690+0000","last_fullsized":"2024-09-13T17:51:15.274690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:11.357026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278023+0000","last_change":"2024-09-13T17:50:34.583001+0000","last_active":"2024-09-13T17:51:15.278023+0000","last_peered":"2024-09-13T17:51:15.278023+0000","last_clean":"2024-09-13T17:51:15.278023+0000","last_became_active":"2024-09-13T17:50:34.582529+0000","last_became_peered":"2024-09-13T17:50:34.582529+0000","last_unstale":"2024-09-13T17:51:15.278023+0000","last_undegraded":"2024-09-13T17:51:15.278023+0000","last_fullsized":"2024-09-13T17:51:15.278023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:05.287181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271281+0000","last_change":"2024-09-13T17:50:34.205138+0000","last_active":"2024-09-13T17:51:15.271281+0000","last_peered":"2024-09-13T17:51:15.271281+0000","last_clean":"2024-09-13T17:51:15.271281+0000","last_became_active":"2024-09-13T17:50:34.203360+0000","last_became_peered":"2024-09-13T17:50:34.203360+0000","last_unstale":"2024-09-13T17:51:15.271281+0000","last_undegraded":"2024-09-13T17:51:15.271281+0000","last_fullsized":"2024-09-13T17:51:15.271281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:35.756963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272372+0000","last_change":"2024-09-13T17:50:38.611036+0000","last_active":"2024-09-13T17:51:15.272372+0000","last_peered":"2024-09-13T17:51:15.272372+0000","last_clean":"2024-09-13T17:51:15.272372+0000","last_became_active":"2024-09-13T17:50:38.610692+0000","last_became_peered":"2024-09-13T17:50:38.610692+0000","last_unstale":"2024-09-13T17:51:15.272372+0000","last_undegraded":"2024-09-13T17:51:15.272372+0000","last_fullsized":"2024-09-13T17:51:15.272372+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:10.147998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.368023+0000","last_change":"2024-09-13T17:51:16.365295+0000","last_active":"2024-09-13T17:51:16.368023+0000","last_peered":"2024-09-13T17:51:16.368023+0000","last_clean":"2024-09-13T17:51:16.368023+0000","last_became_active":"2024-09-13T17:51:16.364437+0000","last_became_peered":"2024-09-13T17:51:16.364437+0000","last_unstale":"2024-09-13T17:51:16.368023+0000","last_undegraded":"2024-09-13T17:51:16.368023+0000","last_fullsized":"2024-09-13T17:51:16.368023+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:23.208899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276819+0000","last_change":"2024-09-13T17:50:34.204376+0000","last_active":"2024-09-13T17:51:15.276819+0000","last_peered":"2024-09-13T17:51:15.276819+0000","last_clean":"2024-09-13T17:51:15.276819+0000","last_became_active":"2024-09-13T17:50:34.204003+0000","last_became_peered":"2024-09-13T17:50:34.204003+0000","last_unstale":"2024-09-13T17:51:15.276819+0000","last_undegraded":"2024-09-13T17:51:15.276819+0000","last_fullsized":"2024-09-13T17:51:15.276819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:14.023990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270270+0000","last_change":"2024-09-13T17:50:38.606144+0000","last_active":"2024-09-13T17:51:15.270270+0000","last_peered":"2024-09-13T17:51:15.270270+0000","last_clean":"2024-09-13T17:51:15.270270+0000","last_became_active":"2024-09-13T17:50:38.605646+0000","last_became_peered":"2024-09-13T17:50:38.605646+0000","last_unstale":"2024-09-13T17:51:15.270270+0000","last_undegraded":"2024-09-13T17:51:15.270270+0000","last_fullsized":"2024-09-13T17:51:15.270270+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:07.120075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277435+0000","last_change":"2024-09-13T17:50:38.618855+0000","last_active":"2024-09-13T17:51:15.277435+0000","last_peered":"2024-09-13T17:51:15.277435+0000","last_clean":"2024-09-13T17:51:15.277435+0000","last_became_active":"2024-09-13T17:50:38.618449+0000","last_became_peered":"2024-09-13T17:50:38.618449+0000","last_unstale":"2024-09-13T17:51:15.277435+0000","last_undegraded":"2024-09-13T17:51:15.277435+0000","last_fullsized":"2024-09-13T17:51:15.277435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:29:01.114087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.280511+0000","last_change":"2024-09-13T17:50:34.590079+0000","last_active":"2024-09-13T17:51:15.280511+0000","last_peered":"2024-09-13T17:51:15.280511+0000","last_clean":"2024-09-13T17:51:15.280511+0000","last_became_active":"2024-09-13T17:50:34.589396+0000","last_became_peered":"2024-09-13T17:50:34.589396+0000","last_unstale":"2024-09-13T17:51:15.280511+0000","last_undegraded":"2024-09-13T17:51:15.280511+0000","last_fullsized":"2024-09-13T17:51:15.280511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:03.308341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269447+0000","last_change":"2024-09-13T17:50:34.206575+0000","last_active":"2024-09-13T17:51:15.269447+0000","last_peered":"2024-09-13T17:51:15.269447+0000","last_clean":"2024-09-13T17:51:15.269447+0000","last_became_active":"2024-09-13T17:50:34.206308+0000","last_became_peered":"2024-09-13T17:50:34.206308+0000","last_unstale":"2024-09-13T17:51:15.269447+0000","last_undegraded":"2024-09-13T17:51:15.269447+0000","last_fullsized":"2024-09-13T17:51:15.269447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:34.766538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276602+0000","last_change":"2024-09-13T17:50:38.618131+0000","last_active":"2024-09-13T17:51:15.276602+0000","last_peered":"2024-09-13T17:51:15.276602+0000","last_clean":"2024-09-13T17:51:15.276602+0000","last_became_active":"2024-09-13T17:50:38.617611+0000","last_became_peered":"2024-09-13T17:50:38.617611+0000","last_unstale":"2024-09-13T17:51:15.276602+0000","last_undegraded":"2024-09-13T17:51:15.276602+0000","last_fullsized":"2024-09-13T17:51:15.276602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:30.746445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272535+0000","last_change":"2024-09-13T17:50:34.582153+0000","last_active":"2024-09-13T17:51:15.272535+0000","last_peered":"2024-09-13T17:51:15.272535+0000","last_clean":"2024-09-13T17:51:15.272535+0000","last_became_active":"2024-09-13T17:50:34.581708+0000","last_became_peered":"2024-09-13T17:50:34.581708+0000","last_unstale":"2024-09-13T17:51:15.272535+0000","last_undegraded":"2024-09-13T17:51:15.272535+0000","last_fullsized":"2024-09-13T17:51:15.272535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:22.238810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270828+0000","last_change":"2024-09-13T17:50:34.205489+0000","last_active":"2024-09-13T17:51:15.270828+0000","last_peered":"2024-09-13T17:51:15.270828+0000","last_clean":"2024-09-13T17:51:15.270828+0000","last_became_active":"2024-09-13T17:50:34.205202+0000","last_became_peered":"2024-09-13T17:50:34.205202+0000","last_unstale":"2024-09-13T17:51:15.270828+0000","last_undegraded":"2024-09-13T17:51:15.270828+0000","last_fullsized":"2024-09-13T17:51:15.270828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:54.450276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276330+0000","last_change":"2024-09-13T17:50:34.202470+0000","last_active":"2024-09-13T17:51:15.276330+0000","last_peered":"2024-09-13T17:51:15.276330+0000","last_clean":"2024-09-13T17:51:15.276330+0000","last_became_active":"2024-09-13T17:50:34.202163+0000","last_became_peered":"2024-09-13T17:50:34.202163+0000","last_unstale":"2024-09-13T17:51:15.276330+0000","last_undegraded":"2024-09-13T17:51:15.276330+0000","last_fullsized":"2024-09-13T17:51:15.276330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:03.472932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282749+0000","last_change":"2024-09-13T17:50:34.590947+0000","last_active":"2024-09-13T17:51:15.282749+0000","last_peered":"2024-09-13T17:51:15.282749+0000","last_clean":"2024-09-13T17:51:15.282749+0000","last_became_active":"2024-09-13T17:50:34.590497+0000","last_became_peered":"2024-09-13T17:50:34.590497+0000","last_unstale":"2024-09-13T17:51:15.282749+0000","last_undegraded":"2024-09-13T17:51:15.282749+0000","last_fullsized":"2024-09-13T17:51:15.282749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:56:27.848092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279328+0000","last_change":"2024-09-13T17:50:38.618518+0000","last_active":"2024-09-13T17:51:15.279328+0000","last_peered":"2024-09-13T17:51:15.279328+0000","last_clean":"2024-09-13T17:51:15.279328+0000","last_became_active":"2024-09-13T17:50:38.618108+0000","last_became_peered":"2024-09-13T17:50:38.618108+0000","last_unstale":"2024-09-13T17:51:15.279328+0000","last_undegraded":"2024-09-13T17:51:15.279328+0000","last_fullsized":"2024-09-13T17:51:15.279328+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:42.260237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'6","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.273983+0000","last_change":"2024-09-13T17:50:34.218459+0000","last_active":"2024-09-13T17:51:16.273983+0000","last_peered":"2024-09-13T17:51:16.273983+0000","last_clean":"2024-09-13T17:51:16.273983+0000","last_became_active":"2024-09-13T17:50:34.218087+0000","last_became_peered":"2024-09-13T17:50:34.218087+0000","last_unstale":"2024-09-13T17:51:16.273983+0000","last_undegraded":"2024-09-13T17:51:16.273983+0000","last_fullsized":"2024-09-13T17:51:16.273983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:56.914544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1016,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.274322+0000","last_change":"2024-09-13T17:50:34.583845+0000","last_active":"2024-09-13T17:51:15.274322+0000","last_peered":"2024-09-13T17:51:15.274322+0000","last_clean":"2024-09-13T17:51:15.274322+0000","last_became_active":"2024-09-13T17:50:34.583283+0000","last_became_peered":"2024-09-13T17:50:34.583283+0000","last_unstale":"2024-09-13T17:51:15.274322+0000","last_undegraded":"2024-09-13T17:51:15.274322+0000","last_fullsized":"2024-09-13T17:51:15.274322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:55.630285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271562+0000","last_change":"2024-09-13T17:50:38.611987+0000","last_active":"2024-09-13T17:51:15.271562+0000","last_peered":"2024-09-13T17:51:15.271562+0000","last_clean":"2024-09-13T17:51:15.271562+0000","last_became_active":"2024-09-13T17:50:38.611636+0000","last_became_peered":"2024-09-13T17:50:38.611636+0000","last_unstale":"2024-09-13T17:51:15.271562+0000","last_undegraded":"2024-09-13T17:51:15.271562+0000","last_fullsized":"2024-09-13T17:51:15.271562+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:41.111562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275726+0000","last_change":"2024-09-13T17:50:34.588927+0000","last_active":"2024-09-13T17:51:15.275726+0000","last_peered":"2024-09-13T17:51:15.275726+0000","last_clean":"2024-09-13T17:51:15.275726+0000","last_became_active":"2024-09-13T17:50:34.588651+0000","last_became_peered":"2024-09-13T17:50:34.588651+0000","last_unstale":"2024-09-13T17:51:15.275726+0000","last_undegraded":"2024-09-13T17:51:15.275726+0000","last_fullsized":"2024-09-13T17:51:15.275726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:49.429670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278614+0000","last_change":"2024-09-13T17:50:34.583795+0000","last_active":"2024-09-13T17:51:15.278614+0000","last_peered":"2024-09-13T17:51:15.278614+0000","last_clean":"2024-09-13T17:51:15.278614+0000","last_became_active":"2024-09-13T17:50:34.583493+0000","last_became_peered":"2024-09-13T17:50:34.583493+0000","last_unstale":"2024-09-13T17:51:15.278614+0000","last_undegraded":"2024-09-13T17:51:15.278614+0000","last_fullsized":"2024-09-13T17:51:15.278614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:30.463932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269876+0000","last_change":"2024-09-13T17:50:34.587624+0000","last_active":"2024-09-13T17:51:15.269876+0000","last_peered":"2024-09-13T17:51:15.269876+0000","last_clean":"2024-09-13T17:51:15.269876+0000","last_became_active":"2024-09-13T17:50:34.587119+0000","last_became_peered":"2024-09-13T17:50:34.587119+0000","last_unstale":"2024-09-13T17:51:15.269876+0000","last_undegraded":"2024-09-13T17:51:15.269876+0000","last_fullsized":"2024-09-13T17:51:15.269876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:16.783899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278177+0000","last_change":"2024-09-13T17:50:34.581923+0000","last_active":"2024-09-13T17:51:15.278177+0000","last_peered":"2024-09-13T17:51:15.278177+0000","last_clean":"2024-09-13T17:51:15.278177+0000","last_became_active":"2024-09-13T17:50:34.581628+0000","last_became_peered":"2024-09-13T17:50:34.581628+0000","last_unstale":"2024-09-13T17:51:15.278177+0000","last_undegraded":"2024-09-13T17:51:15.278177+0000","last_fullsized":"2024-09-13T17:51:15.278177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:28:55.936864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272460+0000","last_change":"2024-09-13T17:50:34.595037+0000","last_active":"2024-09-13T17:51:15.272460+0000","last_peered":"2024-09-13T17:51:15.272460+0000","last_clean":"2024-09-13T17:51:15.272460+0000","last_became_active":"2024-09-13T17:50:34.594857+0000","last_became_peered":"2024-09-13T17:50:34.594857+0000","last_unstale":"2024-09-13T17:51:15.272460+0000","last_undegraded":"2024-09-13T17:51:15.272460+0000","last_fullsized":"2024-09-13T17:51:15.272460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:11.420413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281006+0000","last_change":"2024-09-13T17:50:34.589651+0000","last_active":"2024-09-13T17:51:15.281006+0000","last_peered":"2024-09-13T17:51:15.281006+0000","last_clean":"2024-09-13T17:51:15.281006+0000","last_became_active":"2024-09-13T17:50:34.589354+0000","last_became_peered":"2024-09-13T17:50:34.589354+0000","last_unstale":"2024-09-13T17:51:15.281006+0000","last_undegraded":"2024-09-13T17:51:15.281006+0000","last_fullsized":"2024-09-13T17:51:15.281006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:31.294048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277272+0000","last_change":"2024-09-13T17:50:34.597949+0000","last_active":"2024-09-13T17:51:15.277272+0000","last_peered":"2024-09-13T17:51:15.277272+0000","last_clean":"2024-09-13T17:51:15.277272+0000","last_became_active":"2024-09-13T17:50:34.597621+0000","last_became_peered":"2024-09-13T17:50:34.597621+0000","last_unstale":"2024-09-13T17:51:15.277272+0000","last_undegraded":"2024-09-13T17:51:15.277272+0000","last_fullsized":"2024-09-13T17:51:15.277272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:23.133462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272676+0000","last_change":"2024-09-13T17:50:34.589893+0000","last_active":"2024-09-13T17:51:15.272676+0000","last_peered":"2024-09-13T17:51:15.272676+0000","last_clean":"2024-09-13T17:51:15.272676+0000","last_became_active":"2024-09-13T17:50:34.589626+0000","last_became_peered":"2024-09-13T17:50:34.589626+0000","last_unstale":"2024-09-13T17:51:15.272676+0000","last_undegraded":"2024-09-13T17:51:15.272676+0000","last_fullsized":"2024-09-13T17:51:15.272676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:41.321025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269328+0000","last_change":"2024-09-13T17:50:34.590046+0000","last_active":"2024-09-13T17:51:15.269328+0000","last_peered":"2024-09-13T17:51:15.269328+0000","last_clean":"2024-09-13T17:51:15.269328+0000","last_became_active":"2024-09-13T17:50:34.589757+0000","last_became_peered":"2024-09-13T17:50:34.589757+0000","last_unstale":"2024-09-13T17:51:15.269328+0000","last_undegraded":"2024-09-13T17:51:15.269328+0000","last_fullsized":"2024-09-13T17:51:15.269328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:28.239550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270567+0000","last_change":"2024-09-13T17:50:34.595518+0000","last_active":"2024-09-13T17:51:15.270567+0000","last_peered":"2024-09-13T17:51:15.270567+0000","last_clean":"2024-09-13T17:51:15.270567+0000","last_became_active":"2024-09-13T17:50:34.595242+0000","last_became_peered":"2024-09-13T17:50:34.595242+0000","last_unstale":"2024-09-13T17:51:15.270567+0000","last_undegraded":"2024-09-13T17:51:15.270567+0000","last_fullsized":"2024-09-13T17:51:15.270567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:26.149384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269781+0000","last_change":"2024-09-13T17:50:34.593183+0000","last_active":"2024-09-13T17:51:15.269781+0000","last_peered":"2024-09-13T17:51:15.269781+0000","last_clean":"2024-09-13T17:51:15.269781+0000","last_became_active":"2024-09-13T17:50:34.592893+0000","last_became_peered":"2024-09-13T17:50:34.592893+0000","last_unstale":"2024-09-13T17:51:15.269781+0000","last_undegraded":"2024-09-13T17:51:15.269781+0000","last_fullsized":"2024-09-13T17:51:15.269781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:19.459505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278000+0000","last_change":"2024-09-13T17:50:34.594679+0000","last_active":"2024-09-13T17:51:15.278000+0000","last_peered":"2024-09-13T17:51:15.278000+0000","last_clean":"2024-09-13T17:51:15.278000+0000","last_became_active":"2024-09-13T17:50:34.593750+0000","last_became_peered":"2024-09-13T17:50:34.593750+0000","last_unstale":"2024-09-13T17:51:15.278000+0000","last_undegraded":"2024-09-13T17:51:15.278000+0000","last_fullsized":"2024-09-13T17:51:15.278000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:02.135577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277577+0000","last_change":"2024-09-13T17:50:34.598736+0000","last_active":"2024-09-13T17:51:15.277577+0000","last_peered":"2024-09-13T17:51:15.277577+0000","last_clean":"2024-09-13T17:51:15.277577+0000","last_became_active":"2024-09-13T17:50:34.598433+0000","last_became_peered":"2024-09-13T17:50:34.598433+0000","last_unstale":"2024-09-13T17:51:15.277577+0000","last_undegraded":"2024-09-13T17:51:15.277577+0000","last_fullsized":"2024-09-13T17:51:15.277577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:08.975139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279323+0000","last_change":"2024-09-13T17:50:34.591795+0000","last_active":"2024-09-13T17:51:15.279323+0000","last_peered":"2024-09-13T17:51:15.279323+0000","last_clean":"2024-09-13T17:51:15.279323+0000","last_became_active":"2024-09-13T17:50:34.591286+0000","last_became_peered":"2024-09-13T17:50:34.591286+0000","last_unstale":"2024-09-13T17:51:15.279323+0000","last_undegraded":"2024-09-13T17:51:15.279323+0000","last_fullsized":"2024-09-13T17:51:15.279323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:31.580247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279687+0000","last_change":"2024-09-13T17:50:34.587774+0000","last_active":"2024-09-13T17:51:15.279687+0000","last_peered":"2024-09-13T17:51:15.279687+0000","last_clean":"2024-09-13T17:51:15.279687+0000","last_became_active":"2024-09-13T17:50:34.587506+0000","last_became_peered":"2024-09-13T17:50:34.587506+0000","last_unstale":"2024-09-13T17:51:15.279687+0000","last_undegraded":"2024-09-13T17:51:15.279687+0000","last_fullsized":"2024-09-13T17:51:15.279687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:46.275528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271196+0000","last_change":"2024-09-13T17:50:34.596100+0000","last_active":"2024-09-13T17:51:15.271196+0000","last_peered":"2024-09-13T17:51:15.271196+0000","last_clean":"2024-09-13T17:51:15.271196+0000","last_became_active":"2024-09-13T17:50:34.595798+0000","last_became_peered":"2024-09-13T17:50:34.595798+0000","last_unstale":"2024-09-13T17:51:15.271196+0000","last_undegraded":"2024-09-13T17:51:15.271196+0000","last_fullsized":"2024-09-13T17:51:15.271196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:49.543845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269217+0000","last_change":"2024-09-13T17:50:34.584818+0000","last_active":"2024-09-13T17:51:15.269217+0000","last_peered":"2024-09-13T17:51:15.269217+0000","last_clean":"2024-09-13T17:51:15.269217+0000","last_became_active":"2024-09-13T17:50:34.584516+0000","last_became_peered":"2024-09-13T17:50:34.584516+0000","last_unstale":"2024-09-13T17:51:15.269217+0000","last_undegraded":"2024-09-13T17:51:15.269217+0000","last_fullsized":"2024-09-13T17:51:15.269217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:52.402024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281649+0000","last_change":"2024-09-13T17:50:34.598886+0000","last_active":"2024-09-13T17:51:15.281649+0000","last_peered":"2024-09-13T17:51:15.281649+0000","last_clean":"2024-09-13T17:51:15.281649+0000","last_became_active":"2024-09-13T17:50:34.598648+0000","last_became_peered":"2024-09-13T17:50:34.598648+0000","last_unstale":"2024-09-13T17:51:15.281649+0000","last_undegraded":"2024-09-13T17:51:15.281649+0000","last_fullsized":"2024-09-13T17:51:15.281649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:00:43.063223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282938+0000","last_change":"2024-09-13T17:50:34.582649+0000","last_active":"2024-09-13T17:51:15.282938+0000","last_peered":"2024-09-13T17:51:15.282938+0000","last_clean":"2024-09-13T17:51:15.282938+0000","last_became_active":"2024-09-13T17:50:34.581699+0000","last_became_peered":"2024-09-13T17:50:34.581699+0000","last_unstale":"2024-09-13T17:51:15.282938+0000","last_undegraded":"2024-09-13T17:51:15.282938+0000","last_fullsized":"2024-09-13T17:51:15.282938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:18.959303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270646+0000","last_change":"2024-09-13T17:50:34.587484+0000","last_active":"2024-09-13T17:51:15.270646+0000","last_peered":"2024-09-13T17:51:15.270646+0000","last_clean":"2024-09-13T17:51:15.270646+0000","last_became_active":"2024-09-13T17:50:34.587217+0000","last_became_peered":"2024-09-13T17:50:34.587217+0000","last_unstale":"2024-09-13T17:51:15.270646+0000","last_undegraded":"2024-09-13T17:51:15.270646+0000","last_fullsized":"2024-09-13T17:51:15.270646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:24:19.172607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275806+0000","last_change":"2024-09-13T17:50:34.592871+0000","last_active":"2024-09-13T17:51:15.275806+0000","last_peered":"2024-09-13T17:51:15.275806+0000","last_clean":"2024-09-13T17:51:15.275806+0000","last_became_active":"2024-09-13T17:50:34.588513+0000","last_became_peered":"2024-09-13T17:50:34.588513+0000","last_unstale":"2024-09-13T17:51:15.275806+0000","last_undegraded":"2024-09-13T17:51:15.275806+0000","last_fullsized":"2024-09-13T17:51:15.275806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:33.131420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276633+0000","last_change":"2024-09-13T17:50:34.585713+0000","last_active":"2024-09-13T17:51:15.276633+0000","last_peered":"2024-09-13T17:51:15.276633+0000","last_clean":"2024-09-13T17:51:15.276633+0000","last_became_active":"2024-09-13T17:50:34.585438+0000","last_became_peered":"2024-09-13T17:50:34.585438+0000","last_unstale":"2024-09-13T17:51:15.276633+0000","last_undegraded":"2024-09-13T17:51:15.276633+0000","last_fullsized":"2024-09-13T17:51:15.276633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:56:49.905393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277185+0000","last_change":"2024-09-13T17:50:34.589628+0000","last_active":"2024-09-13T17:51:15.277185+0000","last_peered":"2024-09-13T17:51:15.277185+0000","last_clean":"2024-09-13T17:51:15.277185+0000","last_became_active":"2024-09-13T17:50:34.587665+0000","last_became_peered":"2024-09-13T17:50:34.587665+0000","last_unstale":"2024-09-13T17:51:15.277185+0000","last_undegraded":"2024-09-13T17:51:15.277185+0000","last_fullsized":"2024-09-13T17:51:15.277185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:15.518264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279579+0000","last_change":"2024-09-13T17:50:34.591776+0000","last_active":"2024-09-13T17:51:15.279579+0000","last_peered":"2024-09-13T17:51:15.279579+0000","last_clean":"2024-09-13T17:51:15.279579+0000","last_became_active":"2024-09-13T17:50:34.591496+0000","last_became_peered":"2024-09-13T17:50:34.591496+0000","last_unstale":"2024-09-13T17:51:15.279579+0000","last_undegraded":"2024-09-13T17:51:15.279579+0000","last_fullsized":"2024-09-13T17:51:15.279579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:48.651999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.283047+0000","last_change":"2024-09-13T17:51:16.282372+0000","last_active":"2024-09-13T17:51:16.283047+0000","last_peered":"2024-09-13T17:51:16.283047+0000","last_clean":"2024-09-13T17:51:16.283047+0000","last_became_active":"2024-09-13T17:51:16.282065+0000","last_became_peered":"2024-09-13T17:51:16.282065+0000","last_unstale":"2024-09-13T17:51:16.283047+0000","last_undegraded":"2024-09-13T17:51:16.283047+0000","last_fullsized":"2024-09-13T17:51:16.283047+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:49.291224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278625+0000","last_change":"2024-09-13T17:50:34.585531+0000","last_active":"2024-09-13T17:51:15.278625+0000","last_peered":"2024-09-13T17:51:15.278625+0000","last_clean":"2024-09-13T17:51:15.278625+0000","last_became_active":"2024-09-13T17:50:34.585226+0000","last_became_peered":"2024-09-13T17:50:34.585226+0000","last_unstale":"2024-09-13T17:51:15.278625+0000","last_undegraded":"2024-09-13T17:51:15.278625+0000","last_fullsized":"2024-09-13T17:51:15.278625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:17.105028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269683+0000","last_change":"2024-09-13T17:50:34.586750+0000","last_active":"2024-09-13T17:51:15.269683+0000","last_peered":"2024-09-13T17:51:15.269683+0000","last_clean":"2024-09-13T17:51:15.269683+0000","last_became_active":"2024-09-13T17:50:34.584564+0000","last_became_peered":"2024-09-13T17:50:34.584564+0000","last_unstale":"2024-09-13T17:51:15.269683+0000","last_undegraded":"2024-09-13T17:51:15.269683+0000","last_fullsized":"2024-09-13T17:51:15.269683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:54.033257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278769+0000","last_change":"2024-09-13T17:50:34.585938+0000","last_active":"2024-09-13T17:51:15.278769+0000","last_peered":"2024-09-13T17:51:15.278769+0000","last_clean":"2024-09-13T17:51:15.278769+0000","last_became_active":"2024-09-13T17:50:34.585584+0000","last_became_peered":"2024-09-13T17:50:34.585584+0000","last_unstale":"2024-09-13T17:51:15.278769+0000","last_undegraded":"2024-09-13T17:51:15.278769+0000","last_fullsized":"2024-09-13T17:51:15.278769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:32.623468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271015+0000","last_change":"2024-09-13T17:50:34.588762+0000","last_active":"2024-09-13T17:51:15.271015+0000","last_peered":"2024-09-13T17:51:15.271015+0000","last_clean":"2024-09-13T17:51:15.271015+0000","last_became_active":"2024-09-13T17:50:34.588489+0000","last_became_peered":"2024-09-13T17:50:34.588489+0000","last_unstale":"2024-09-13T17:51:15.271015+0000","last_undegraded":"2024-09-13T17:51:15.271015+0000","last_fullsized":"2024-09-13T17:51:15.271015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:38.266297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277059+0000","last_change":"2024-09-13T17:50:34.583889+0000","last_active":"2024-09-13T17:51:15.277059+0000","last_peered":"2024-09-13T17:51:15.277059+0000","last_clean":"2024-09-13T17:51:15.277059+0000","last_became_active":"2024-09-13T17:50:34.583551+0000","last_became_peered":"2024-09-13T17:50:34.583551+0000","last_unstale":"2024-09-13T17:51:15.277059+0000","last_undegraded":"2024-09-13T17:51:15.277059+0000","last_fullsized":"2024-09-13T17:51:15.277059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:19.888626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281343+0000","last_change":"2024-09-13T17:50:34.585617+0000","last_active":"2024-09-13T17:51:15.281343+0000","last_peered":"2024-09-13T17:51:15.281343+0000","last_clean":"2024-09-13T17:51:15.281343+0000","last_became_active":"2024-09-13T17:50:34.585340+0000","last_became_peered":"2024-09-13T17:50:34.585340+0000","last_unstale":"2024-09-13T17:51:15.281343+0000","last_undegraded":"2024-09-13T17:51:15.281343+0000","last_fullsized":"2024-09-13T17:51:15.281343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:18.331045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279542+0000","last_change":"2024-09-13T17:50:34.592365+0000","last_active":"2024-09-13T17:51:15.279542+0000","last_peered":"2024-09-13T17:51:15.279542+0000","last_clean":"2024-09-13T17:51:15.279542+0000","last_became_active":"2024-09-13T17:50:34.592046+0000","last_became_peered":"2024-09-13T17:50:34.592046+0000","last_unstale":"2024-09-13T17:51:15.279542+0000","last_undegraded":"2024-09-13T17:51:15.279542+0000","last_fullsized":"2024-09-13T17:51:15.279542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:54.283683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7751,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":15502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":307200,"data_stored":120825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4870000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]}]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":303104,"data_stored":116742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":307200,"data_stored":116868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":892928,"data_stored":706952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.5009999999999999,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.60199999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.528,"5min":2.528,"15min":2.528},"last":0.70299999999999996}]},{"osd":3,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.51600000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.5790000000000002,"5min":2.5790000000000002,"15min":2.5790000000000002},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.79500000000000004}]},{"osd":5,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.3700000000000001,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.5499999999999998,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":0.74099999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":1.022},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":0.92000000000000004}]},{"osd":7,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.4660000000000002,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.1599999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":897024,"data_stored":708002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2649999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":303104,"data_stored":120349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":4,"last update":"Fri Sep 13 17:51:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.90700000000000003}]},{"osd":5,"last update":"Fri Sep 13 17:51:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.995,"5min":0.995,"15min":0.995},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.296}]},{"osd":7,"last update":"Fri Sep 13 17:51:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.071},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.3999999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":303104,"data_stored":117302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5820000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1919999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542158,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":299008,"data_stored":116590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0720000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:51:27.086 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:51:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.202+0000 7fc032e28700 1 Processor -- start 2024-09-13T17:51:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.202+0000 7fc032e28700 1 -- start start 2024-09-13T17:51:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.202+0000 7fc032e28700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc02c153840 0x7fc02c0c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.202+0000 7fc032e28700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc02c0c9e20 0x7fc02c0cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc032e28700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc02c0cc060 con 0x7fc02c0cc4f0 2024-09-13T17:51:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc032e28700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc02c0b2840 con 0x7fc02c0c9e20 2024-09-13T17:51:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc032e28700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c0ca870 con 0x7fc02c153840 2024-09-13T17:51:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc031625700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc02c0cc4f0 0x7fc02c15d410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44554/0 (socket says 172.21.15.40:44554) 2024-09-13T17:51:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc031625700 1 -- 172.21.15.40:0/1488569725 learned_addr learned my addr 172.21.15.40:0/1488569725 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc032627700 1 --2- 172.21.15.40:0/1488569725 >> v2:172.21.15.43:3300/0 conn(0x7fc02c0c9e20 0x7fc02c0cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:27.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc031e26700 1 --2- 172.21.15.40:0/1488569725 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc02c153840 0x7fc02c0c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc030e24700 1 -- 172.21.15.40:0/1488569725 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 437891826 0 0) 0x7fc02c0cc060 con 0x7fc02c0cc4f0 2024-09-13T17:51:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc030e24700 1 -- 172.21.15.40:0/1488569725 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0180035d0 con 0x7fc02c0cc4f0 2024-09-13T17:51:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc032627700 1 -- 172.21.15.40:0/1488569725 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc02c153840 msgr2=0x7fc02c0c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc030e24700 1 -- 172.21.15.40:0/1488569725 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 938059254 0 0) 0x7fc0180035d0 con 0x7fc02c0cc4f0 2024-09-13T17:51:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc032627700 1 --2- 172.21.15.40:0/1488569725 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc02c153840 0x7fc02c0c3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc032627700 1 -- 172.21.15.40:0/1488569725 >> v1:172.21.15.40:6789/0 conn(0x7fc02c0cc4f0 legacy=0x7fc02c15d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc032627700 1 -- 172.21.15.40:0/1488569725 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c153c20 con 0x7fc02c0c9e20 2024-09-13T17:51:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc032627700 1 --2- 172.21.15.40:0/1488569725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc02c0c9e20 0x7fc02c0cf260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc024009d90 tx=0x7fc02400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=892fc0f5178d7e15 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc030e24700 1 -- 172.21.15.40:0/1488569725 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02400e070 con 0x7fc02c0c9e20 2024-09-13T17:51:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc030e24700 1 -- 172.21.15.40:0/1488569725 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc024003a40 con 0x7fc02c0c9e20 2024-09-13T17:51:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc030e24700 1 -- 172.21.15.40:0/1488569725 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028001670 con 0x7fc02c0c9e20 2024-09-13T17:51:27.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc032e28700 1 -- 172.21.15.40:0/1488569725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc02c0c9e20 msgr2=0x7fc02c0cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:27.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc032e28700 1 --2- 172.21.15.40:0/1488569725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc02c0c9e20 0x7fc02c0cf260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc024009d90 tx=0x7fc02400b330 comp rx=0 tx=0).stop 2024-09-13T17:51:27.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc032e28700 1 -- 172.21.15.40:0/1488569725 shutdown_connections 2024-09-13T17:51:27.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc032e28700 1 --2- 172.21.15.40:0/1488569725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc02c0c9e20 0x7fc02c0cf260 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc032e28700 1 --2- 172.21.15.40:0/1488569725 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc02c153840 0x7fc02c0c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc032e28700 1 -- 172.21.15.40:0/1488569725 >> 172.21.15.40:0/1488569725 conn(0x7fc02c06c800 msgr2=0x7fc02c06cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:27.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc032e28700 1 -- 172.21.15.40:0/1488569725 shutdown_connections 2024-09-13T17:51:27.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.206+0000 7fc032e28700 1 -- 172.21.15.40:0/1488569725 wait complete. 2024-09-13T17:51:27.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.210+0000 7fc032e28700 1 Processor -- start 2024-09-13T17:51:27.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.210+0000 7fc032e28700 1 -- start start 2024-09-13T17:51:27.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc032e28700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc02c0c9e20 0x7fc02c149270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:27.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc032e28700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc02c0cc4f0 0x7fc02c1e26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:27.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc032e28700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc02c153840 0x7fc02c1e2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:27.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc032e28700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c1e3d10 con 0x7fc02c0cc4f0 2024-09-13T17:51:27.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc032e28700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c1e0c40 con 0x7fc02c153840 2024-09-13T17:51:27.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc032e28700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02c1e2240 con 0x7fc02c0c9e20 2024-09-13T17:51:27.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc031625700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc02c0cc4f0 0x7fc02c1e26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:27.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc031625700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc02c0cc4f0 0x7fc02c1e26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55138/0 (socket says 172.21.15.40:55138) 2024-09-13T17:51:27.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc031625700 1 -- 172.21.15.40:0/337555157 learned_addr learned my addr 172.21.15.40:0/337555157 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:27.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc031e26700 1 --2- 172.21.15.40:0/337555157 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc02c0c9e20 0x7fc02c149270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:27.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc032627700 1 --2- 172.21.15.40:0/337555157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc02c153840 0x7fc02c1e2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:27.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc031625700 1 -- 172.21.15.40:0/337555157 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc02c0c9e20 msgr2=0x7fc02c149270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:27.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc031625700 1 --2- 172.21.15.40:0/337555157 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc02c0c9e20 0x7fc02c149270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc031625700 1 -- 172.21.15.40:0/337555157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc02c153840 msgr2=0x7fc02c1e2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:27.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc031625700 1 --2- 172.21.15.40:0/337555157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc02c153840 0x7fc02c1e2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc031625700 1 -- 172.21.15.40:0/337555157 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc024009a70 con 0x7fc02c0cc4f0 2024-09-13T17:51:27.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc031625700 1 --2- 172.21.15.40:0/337555157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc02c0cc4f0 0x7fc02c1e26d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc02800c730 tx=0x7fc02800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:27.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc0227fc700 1 -- 172.21.15.40:0/337555157 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028004ca0 con 0x7fc02c0cc4f0 2024-09-13T17:51:27.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc032e28700 1 -- 172.21.15.40:0/337555157 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc02c1e4ea0 con 0x7fc02c0cc4f0 2024-09-13T17:51:27.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc0227fc700 1 -- 172.21.15.40:0/337555157 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc028004e00 con 0x7fc02c0cc4f0 2024-09-13T17:51:27.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc032e28700 1 -- 172.21.15.40:0/337555157 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc02c1e1150 con 0x7fc02c0cc4f0 2024-09-13T17:51:27.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc0227fc700 1 -- 172.21.15.40:0/337555157 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0280072d0 con 0x7fc02c0cc4f0 2024-09-13T17:51:27.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.214+0000 7fc032e28700 1 -- 172.21.15.40:0/337555157 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc02c0cbcf0 con 0x7fc02c0cc4f0 2024-09-13T17:51:27.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.218+0000 7fc0227fc700 1 -- 172.21.15.40:0/337555157 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc028007430 con 0x7fc02c0cc4f0 2024-09-13T17:51:27.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.218+0000 7fc0227fc700 1 --2- 172.21.15.40:0/337555157 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc010066730 0x7fc010068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:27.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.218+0000 7fc0227fc700 1 -- 172.21.15.40:0/337555157 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc028084090 con 0x7fc02c0cc4f0 2024-09-13T17:51:27.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.218+0000 7fc031e26700 1 --2- 172.21.15.40:0/337555157 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc010066730 0x7fc010068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:27.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.218+0000 7fc031e26700 1 --2- 172.21.15.40:0/337555157 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc010066730 0x7fc010068bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0c4f60 tx=0x7fc01c009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:27.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.226+0000 7fc0227fc700 1 -- 172.21.15.40:0/337555157 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc028050fa0 con 0x7fc02c0cc4f0 2024-09-13T17:51:27.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.398+0000 7fc032e28700 1 -- 172.21.15.40:0/337555157 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc02c074810 con 0x7fc010066730 2024-09-13T17:51:27.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.414+0000 7fc0227fc700 1 -- 172.21.15.40:0/337555157 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+357802 (secure 0 0 0) 0x7fc02c074810 con 0x7fc010066730 2024-09-13T17:51:27.425 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:27.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.418+0000 7fc032e28700 1 -- 172.21.15.40:0/337555157 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc010066730 msgr2=0x7fc010068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:27.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.418+0000 7fc032e28700 1 --2- 172.21.15.40:0/337555157 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc010066730 0x7fc010068bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0c4f60 tx=0x7fc01c009240 comp rx=0 tx=0).stop 2024-09-13T17:51:27.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.418+0000 7fc032e28700 1 -- 172.21.15.40:0/337555157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc02c0cc4f0 msgr2=0x7fc02c1e26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:27.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.418+0000 7fc032e28700 1 --2- 172.21.15.40:0/337555157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc02c0cc4f0 0x7fc02c1e26d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc02800c730 tx=0x7fc02800ca40 comp rx=0 tx=0).stop 2024-09-13T17:51:27.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.418+0000 7fc032e28700 1 -- 172.21.15.40:0/337555157 shutdown_connections 2024-09-13T17:51:27.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.418+0000 7fc032e28700 1 --2- 172.21.15.40:0/337555157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc02c153840 0x7fc02c1e2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.418+0000 7fc032e28700 1 --2- 172.21.15.40:0/337555157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc02c0cc4f0 0x7fc02c1e26d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.418+0000 7fc032e28700 1 --2- 172.21.15.40:0/337555157 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc010066730 0x7fc010068bf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.418+0000 7fc032e28700 1 --2- 172.21.15.40:0/337555157 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc02c0c9e20 0x7fc02c149270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.418+0000 7fc032e28700 1 -- 172.21.15.40:0/337555157 >> 172.21.15.40:0/337555157 conn(0x7fc02c06c800 msgr2=0x7fc02c0b4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:27.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.418+0000 7fc032e28700 1 -- 172.21.15.40:0/337555157 shutdown_connections 2024-09-13T17:51:27.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.418+0000 7fc032e28700 1 -- 172.21.15.40:0/337555157 wait complete. 2024-09-13T17:51:27.432 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-13T17:51:27.440 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":53,"stamp":"2024-09-13T17:51:27.015628+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598119,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":288,"num_write_kb":2459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236076,"kb_used_data":3528,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738644,"statfs":{"total":773094113280,"available":772852371456,"internally_reserved":0,"allocated":3612672,"data_stored":2123630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":442,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.745106"},"pg_stats":[{"pgid":"2.1f","version":"25'9","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282601+0000","last_change":"2024-09-13T17:50:34.212158+0000","last_active":"2024-09-13T17:51:15.282601+0000","last_peered":"2024-09-13T17:51:15.282601+0000","last_clean":"2024-09-13T17:51:15.282601+0000","last_became_active":"2024-09-13T17:50:34.211867+0000","last_became_peered":"2024-09-13T17:50:34.211867+0000","last_unstale":"2024-09-13T17:51:15.282601+0000","last_undegraded":"2024-09-13T17:51:15.282601+0000","last_fullsized":"2024-09-13T17:51:15.282601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:14.671095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271433+0000","last_change":"2024-09-13T17:50:34.587396+0000","last_active":"2024-09-13T17:51:15.271433+0000","last_peered":"2024-09-13T17:51:15.271433+0000","last_clean":"2024-09-13T17:51:15.271433+0000","last_became_active":"2024-09-13T17:50:34.587108+0000","last_became_peered":"2024-09-13T17:50:34.587108+0000","last_unstale":"2024-09-13T17:51:15.271433+0000","last_undegraded":"2024-09-13T17:51:15.271433+0000","last_fullsized":"2024-09-13T17:51:15.271433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:39:16.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270087+0000","last_change":"2024-09-13T17:50:38.615059+0000","last_active":"2024-09-13T17:51:15.270087+0000","last_peered":"2024-09-13T17:51:15.270087+0000","last_clean":"2024-09-13T17:51:15.270087+0000","last_became_active":"2024-09-13T17:50:38.614723+0000","last_became_peered":"2024-09-13T17:50:38.614723+0000","last_unstale":"2024-09-13T17:51:15.270087+0000","last_undegraded":"2024-09-13T17:51:15.270087+0000","last_fullsized":"2024-09-13T17:51:15.270087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:23:01.902181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"25'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271688+0000","last_change":"2024-09-13T17:50:34.204760+0000","last_active":"2024-09-13T17:51:15.271688+0000","last_peered":"2024-09-13T17:51:15.271688+0000","last_clean":"2024-09-13T17:51:15.271688+0000","last_became_active":"2024-09-13T17:50:34.202930+0000","last_became_peered":"2024-09-13T17:50:34.202930+0000","last_unstale":"2024-09-13T17:51:15.271688+0000","last_undegraded":"2024-09-13T17:51:15.271688+0000","last_fullsized":"2024-09-13T17:51:15.271688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:59.784701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.366352+0000","last_change":"2024-09-13T17:51:16.366352+0000","last_active":"2024-09-13T17:51:16.366352+0000","last_peered":"2024-09-13T17:51:16.366352+0000","last_clean":"2024-09-13T17:51:16.366352+0000","last_became_active":"2024-09-13T17:51:16.365492+0000","last_became_peered":"2024-09-13T17:51:16.365492+0000","last_unstale":"2024-09-13T17:51:16.366352+0000","last_undegraded":"2024-09-13T17:51:16.366352+0000","last_fullsized":"2024-09-13T17:51:16.366352+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:42.192182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270184+0000","last_change":"2024-09-13T17:50:38.618751+0000","last_active":"2024-09-13T17:51:15.270184+0000","last_peered":"2024-09-13T17:51:15.270184+0000","last_clean":"2024-09-13T17:51:15.270184+0000","last_became_active":"2024-09-13T17:50:38.618423+0000","last_became_peered":"2024-09-13T17:50:38.618423+0000","last_unstale":"2024-09-13T17:51:15.270184+0000","last_undegraded":"2024-09-13T17:51:15.270184+0000","last_fullsized":"2024-09-13T17:51:15.270184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:50:02.418785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"25'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279312+0000","last_change":"2024-09-13T17:50:34.214972+0000","last_active":"2024-09-13T17:51:15.279312+0000","last_peered":"2024-09-13T17:51:15.279312+0000","last_clean":"2024-09-13T17:51:15.279312+0000","last_became_active":"2024-09-13T17:50:34.214652+0000","last_became_peered":"2024-09-13T17:50:34.214652+0000","last_unstale":"2024-09-13T17:51:15.279312+0000","last_undegraded":"2024-09-13T17:51:15.279312+0000","last_fullsized":"2024-09-13T17:51:15.279312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:57.176854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.282672+0000","last_change":"2024-09-13T17:51:16.281831+0000","last_active":"2024-09-13T17:51:16.282672+0000","last_peered":"2024-09-13T17:51:16.282672+0000","last_clean":"2024-09-13T17:51:16.282672+0000","last_became_active":"2024-09-13T17:51:16.281549+0000","last_became_peered":"2024-09-13T17:51:16.281549+0000","last_unstale":"2024-09-13T17:51:16.282672+0000","last_undegraded":"2024-09-13T17:51:16.282672+0000","last_fullsized":"2024-09-13T17:51:16.282672+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:43.894437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.274476+0000","last_change":"2024-09-13T17:50:38.616219+0000","last_active":"2024-09-13T17:51:15.274476+0000","last_peered":"2024-09-13T17:51:15.274476+0000","last_clean":"2024-09-13T17:51:15.274476+0000","last_became_active":"2024-09-13T17:50:38.614329+0000","last_became_peered":"2024-09-13T17:50:38.614329+0000","last_unstale":"2024-09-13T17:51:15.274476+0000","last_undegraded":"2024-09-13T17:51:15.274476+0000","last_fullsized":"2024-09-13T17:51:15.274476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:25.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"25'2","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279079+0000","last_change":"2024-09-13T17:50:34.216123+0000","last_active":"2024-09-13T17:51:15.279079+0000","last_peered":"2024-09-13T17:51:15.279079+0000","last_clean":"2024-09-13T17:51:15.279079+0000","last_became_active":"2024-09-13T17:50:34.215661+0000","last_became_peered":"2024-09-13T17:50:34.215661+0000","last_unstale":"2024-09-13T17:51:15.279079+0000","last_undegraded":"2024-09-13T17:51:15.279079+0000","last_fullsized":"2024-09-13T17:51:15.279079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:20.895037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270446+0000","last_change":"2024-09-13T17:50:34.599673+0000","last_active":"2024-09-13T17:51:15.270446+0000","last_peered":"2024-09-13T17:51:15.270446+0000","last_clean":"2024-09-13T17:51:15.270446+0000","last_became_active":"2024-09-13T17:50:34.599409+0000","last_became_peered":"2024-09-13T17:50:34.599409+0000","last_unstale":"2024-09-13T17:51:15.270446+0000","last_undegraded":"2024-09-13T17:51:15.270446+0000","last_fullsized":"2024-09-13T17:51:15.270446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:03:31.394385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.274077+0000","last_change":"2024-09-13T17:50:38.616317+0000","last_active":"2024-09-13T17:51:15.274077+0000","last_peered":"2024-09-13T17:51:15.274077+0000","last_clean":"2024-09-13T17:51:15.274077+0000","last_became_active":"2024-09-13T17:50:38.614311+0000","last_became_peered":"2024-09-13T17:50:38.614311+0000","last_unstale":"2024-09-13T17:51:15.274077+0000","last_undegraded":"2024-09-13T17:51:15.274077+0000","last_fullsized":"2024-09-13T17:51:15.274077+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:29.221522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275452+0000","last_change":"2024-09-13T17:50:34.599490+0000","last_active":"2024-09-13T17:51:15.275452+0000","last_peered":"2024-09-13T17:51:15.275452+0000","last_clean":"2024-09-13T17:51:15.275452+0000","last_became_active":"2024-09-13T17:50:34.599207+0000","last_became_peered":"2024-09-13T17:50:34.599207+0000","last_unstale":"2024-09-13T17:51:15.275452+0000","last_undegraded":"2024-09-13T17:51:15.275452+0000","last_fullsized":"2024-09-13T17:51:15.275452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:11.178161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"25'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278915+0000","last_change":"2024-09-13T17:50:34.215697+0000","last_active":"2024-09-13T17:51:15.278915+0000","last_peered":"2024-09-13T17:51:15.278915+0000","last_clean":"2024-09-13T17:51:15.278915+0000","last_became_active":"2024-09-13T17:50:34.215409+0000","last_became_peered":"2024-09-13T17:50:34.215409+0000","last_unstale":"2024-09-13T17:51:15.278915+0000","last_undegraded":"2024-09-13T17:51:15.278915+0000","last_fullsized":"2024-09-13T17:51:15.278915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:21.609567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275240+0000","last_change":"2024-09-13T17:50:38.611817+0000","last_active":"2024-09-13T17:51:15.275240+0000","last_peered":"2024-09-13T17:51:15.275240+0000","last_clean":"2024-09-13T17:51:15.275240+0000","last_became_active":"2024-09-13T17:50:38.611263+0000","last_became_peered":"2024-09-13T17:50:38.611263+0000","last_unstale":"2024-09-13T17:51:15.275240+0000","last_undegraded":"2024-09-13T17:51:15.275240+0000","last_fullsized":"2024-09-13T17:51:15.275240+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:56:47.466672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271037+0000","last_change":"2024-09-13T17:50:34.586307+0000","last_active":"2024-09-13T17:51:15.271037+0000","last_peered":"2024-09-13T17:51:15.271037+0000","last_clean":"2024-09-13T17:51:15.271037+0000","last_became_active":"2024-09-13T17:50:34.585831+0000","last_became_peered":"2024-09-13T17:50:34.585831+0000","last_unstale":"2024-09-13T17:51:15.271037+0000","last_undegraded":"2024-09-13T17:51:15.271037+0000","last_fullsized":"2024-09-13T17:51:15.271037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:40.912352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"25'1","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.280577+0000","last_change":"2024-09-13T17:50:34.211850+0000","last_active":"2024-09-13T17:51:15.280577+0000","last_peered":"2024-09-13T17:51:15.280577+0000","last_clean":"2024-09-13T17:51:15.280577+0000","last_became_active":"2024-09-13T17:50:34.211594+0000","last_became_peered":"2024-09-13T17:50:34.211594+0000","last_unstale":"2024-09-13T17:51:15.280577+0000","last_undegraded":"2024-09-13T17:51:15.280577+0000","last_fullsized":"2024-09-13T17:51:15.280577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:35:02.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276065+0000","last_change":"2024-09-13T17:50:38.613576+0000","last_active":"2024-09-13T17:51:15.276065+0000","last_peered":"2024-09-13T17:51:15.276065+0000","last_clean":"2024-09-13T17:51:15.276065+0000","last_became_active":"2024-09-13T17:50:38.612577+0000","last_became_peered":"2024-09-13T17:50:38.612577+0000","last_unstale":"2024-09-13T17:51:15.276065+0000","last_undegraded":"2024-09-13T17:51:15.276065+0000","last_fullsized":"2024-09-13T17:51:15.276065+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:25.551060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272679+0000","last_change":"2024-09-13T17:50:34.587273+0000","last_active":"2024-09-13T17:51:15.272679+0000","last_peered":"2024-09-13T17:51:15.272679+0000","last_clean":"2024-09-13T17:51:15.272679+0000","last_became_active":"2024-09-13T17:50:34.587027+0000","last_became_peered":"2024-09-13T17:50:34.587027+0000","last_unstale":"2024-09-13T17:51:15.272679+0000","last_undegraded":"2024-09-13T17:51:15.272679+0000","last_fullsized":"2024-09-13T17:51:15.272679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:58.826737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.280692+0000","last_change":"2024-09-13T17:50:34.211845+0000","last_active":"2024-09-13T17:51:15.280692+0000","last_peered":"2024-09-13T17:51:15.280692+0000","last_clean":"2024-09-13T17:51:15.280692+0000","last_became_active":"2024-09-13T17:50:34.211568+0000","last_became_peered":"2024-09-13T17:50:34.211568+0000","last_unstale":"2024-09-13T17:51:15.280692+0000","last_undegraded":"2024-09-13T17:51:15.280692+0000","last_fullsized":"2024-09-13T17:51:15.280692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:56.348984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281699+0000","last_change":"2024-09-13T17:50:38.610292+0000","last_active":"2024-09-13T17:51:15.281699+0000","last_peered":"2024-09-13T17:51:15.281699+0000","last_clean":"2024-09-13T17:51:15.281699+0000","last_became_active":"2024-09-13T17:50:38.609832+0000","last_became_peered":"2024-09-13T17:50:38.609832+0000","last_unstale":"2024-09-13T17:51:15.281699+0000","last_undegraded":"2024-09-13T17:51:15.281699+0000","last_fullsized":"2024-09-13T17:51:15.281699+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:13.783068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.273946+0000","last_change":"2024-09-13T17:50:34.593566+0000","last_active":"2024-09-13T17:51:15.273946+0000","last_peered":"2024-09-13T17:51:15.273946+0000","last_clean":"2024-09-13T17:51:15.273946+0000","last_became_active":"2024-09-13T17:50:34.588794+0000","last_became_peered":"2024-09-13T17:50:34.588794+0000","last_unstale":"2024-09-13T17:51:15.273946+0000","last_undegraded":"2024-09-13T17:51:15.273946+0000","last_fullsized":"2024-09-13T17:51:15.273946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:13.192453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"25'2","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277322+0000","last_change":"2024-09-13T17:50:34.214638+0000","last_active":"2024-09-13T17:51:15.277322+0000","last_peered":"2024-09-13T17:51:15.277322+0000","last_clean":"2024-09-13T17:51:15.277322+0000","last_became_active":"2024-09-13T17:50:34.214292+0000","last_became_peered":"2024-09-13T17:50:34.214292+0000","last_unstale":"2024-09-13T17:51:15.277322+0000","last_undegraded":"2024-09-13T17:51:15.277322+0000","last_fullsized":"2024-09-13T17:51:15.277322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:57.377104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275146+0000","last_change":"2024-09-13T17:50:38.613368+0000","last_active":"2024-09-13T17:51:15.275146+0000","last_peered":"2024-09-13T17:51:15.275146+0000","last_clean":"2024-09-13T17:51:15.275146+0000","last_became_active":"2024-09-13T17:50:38.613010+0000","last_became_peered":"2024-09-13T17:50:38.613010+0000","last_unstale":"2024-09-13T17:51:15.275146+0000","last_undegraded":"2024-09-13T17:51:15.275146+0000","last_fullsized":"2024-09-13T17:51:15.275146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:42.706625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282530+0000","last_change":"2024-09-13T17:50:34.582327+0000","last_active":"2024-09-13T17:51:15.282530+0000","last_peered":"2024-09-13T17:51:15.282530+0000","last_clean":"2024-09-13T17:51:15.282530+0000","last_became_active":"2024-09-13T17:50:34.582050+0000","last_became_peered":"2024-09-13T17:50:34.582050+0000","last_unstale":"2024-09-13T17:51:15.282530+0000","last_undegraded":"2024-09-13T17:51:15.282530+0000","last_fullsized":"2024-09-13T17:51:15.282530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:52.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"25'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277006+0000","last_change":"2024-09-13T17:50:34.211512+0000","last_active":"2024-09-13T17:51:15.277006+0000","last_peered":"2024-09-13T17:51:15.277006+0000","last_clean":"2024-09-13T17:51:15.277006+0000","last_became_active":"2024-09-13T17:50:34.211230+0000","last_became_peered":"2024-09-13T17:50:34.211230+0000","last_unstale":"2024-09-13T17:51:15.277006+0000","last_undegraded":"2024-09-13T17:51:15.277006+0000","last_fullsized":"2024-09-13T17:51:15.277006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:40.707777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272462+0000","last_change":"2024-09-13T17:50:38.608255+0000","last_active":"2024-09-13T17:51:15.272462+0000","last_peered":"2024-09-13T17:51:15.272462+0000","last_clean":"2024-09-13T17:51:15.272462+0000","last_became_active":"2024-09-13T17:50:38.606697+0000","last_became_peered":"2024-09-13T17:50:38.606697+0000","last_unstale":"2024-09-13T17:51:15.272462+0000","last_undegraded":"2024-09-13T17:51:15.272462+0000","last_fullsized":"2024-09-13T17:51:15.272462+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:47.265007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272916+0000","last_change":"2024-09-13T17:50:34.594476+0000","last_active":"2024-09-13T17:51:15.272916+0000","last_peered":"2024-09-13T17:51:15.272916+0000","last_clean":"2024-09-13T17:51:15.272916+0000","last_became_active":"2024-09-13T17:50:34.594215+0000","last_became_peered":"2024-09-13T17:50:34.594215+0000","last_unstale":"2024-09-13T17:51:15.272916+0000","last_undegraded":"2024-09-13T17:51:15.272916+0000","last_fullsized":"2024-09-13T17:51:15.272916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:02:27.528019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281479+0000","last_change":"2024-09-13T17:50:34.204018+0000","last_active":"2024-09-13T17:51:15.281479+0000","last_peered":"2024-09-13T17:51:15.281479+0000","last_clean":"2024-09-13T17:51:15.281479+0000","last_became_active":"2024-09-13T17:50:34.203702+0000","last_became_peered":"2024-09-13T17:50:34.203702+0000","last_unstale":"2024-09-13T17:51:15.281479+0000","last_undegraded":"2024-09-13T17:51:15.281479+0000","last_fullsized":"2024-09-13T17:51:15.281479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:44.431226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.365782+0000","last_change":"2024-09-13T17:51:16.365782+0000","last_active":"2024-09-13T17:51:16.365782+0000","last_peered":"2024-09-13T17:51:16.365782+0000","last_clean":"2024-09-13T17:51:16.365782+0000","last_became_active":"2024-09-13T17:51:16.364540+0000","last_became_peered":"2024-09-13T17:51:16.364540+0000","last_unstale":"2024-09-13T17:51:16.365782+0000","last_undegraded":"2024-09-13T17:51:16.365782+0000","last_fullsized":"2024-09-13T17:51:16.365782+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:18.753557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281218+0000","last_change":"2024-09-13T17:50:34.586802+0000","last_active":"2024-09-13T17:51:15.281218+0000","last_peered":"2024-09-13T17:51:15.281218+0000","last_clean":"2024-09-13T17:51:15.281218+0000","last_became_active":"2024-09-13T17:50:34.586552+0000","last_became_peered":"2024-09-13T17:50:34.586552+0000","last_unstale":"2024-09-13T17:51:15.281218+0000","last_undegraded":"2024-09-13T17:51:15.281218+0000","last_fullsized":"2024-09-13T17:51:15.281218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:43.383975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"25'1","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279434+0000","last_change":"2024-09-13T17:50:34.217083+0000","last_active":"2024-09-13T17:51:15.279434+0000","last_peered":"2024-09-13T17:51:15.279434+0000","last_clean":"2024-09-13T17:51:15.279434+0000","last_became_active":"2024-09-13T17:50:34.216758+0000","last_became_peered":"2024-09-13T17:50:34.216758+0000","last_unstale":"2024-09-13T17:51:15.279434+0000","last_undegraded":"2024-09-13T17:51:15.279434+0000","last_fullsized":"2024-09-13T17:51:15.279434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:36.473942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277926+0000","last_change":"2024-09-13T17:50:38.614667+0000","last_active":"2024-09-13T17:51:15.277926+0000","last_peered":"2024-09-13T17:51:15.277926+0000","last_clean":"2024-09-13T17:51:15.277926+0000","last_became_active":"2024-09-13T17:50:38.614284+0000","last_became_peered":"2024-09-13T17:50:38.614284+0000","last_unstale":"2024-09-13T17:51:15.277926+0000","last_undegraded":"2024-09-13T17:51:15.277926+0000","last_fullsized":"2024-09-13T17:51:15.277926+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:29.616794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270498+0000","last_change":"2024-09-13T17:50:34.588603+0000","last_active":"2024-09-13T17:51:15.270498+0000","last_peered":"2024-09-13T17:51:15.270498+0000","last_clean":"2024-09-13T17:51:15.270498+0000","last_became_active":"2024-09-13T17:50:34.588317+0000","last_became_peered":"2024-09-13T17:50:34.588317+0000","last_unstale":"2024-09-13T17:51:15.270498+0000","last_undegraded":"2024-09-13T17:51:15.270498+0000","last_fullsized":"2024-09-13T17:51:15.270498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:03.743130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281302+0000","last_change":"2024-09-13T17:50:34.204335+0000","last_active":"2024-09-13T17:51:15.281302+0000","last_peered":"2024-09-13T17:51:15.281302+0000","last_clean":"2024-09-13T17:51:15.281302+0000","last_became_active":"2024-09-13T17:50:34.204060+0000","last_became_peered":"2024-09-13T17:50:34.204060+0000","last_unstale":"2024-09-13T17:51:15.281302+0000","last_undegraded":"2024-09-13T17:51:15.281302+0000","last_fullsized":"2024-09-13T17:51:15.281302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:54:41.831344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279638+0000","last_change":"2024-09-13T17:50:38.619590+0000","last_active":"2024-09-13T17:51:15.279638+0000","last_peered":"2024-09-13T17:51:15.279638+0000","last_clean":"2024-09-13T17:51:15.279638+0000","last_became_active":"2024-09-13T17:50:38.617257+0000","last_became_peered":"2024-09-13T17:50:38.617257+0000","last_unstale":"2024-09-13T17:51:15.279638+0000","last_undegraded":"2024-09-13T17:51:15.279638+0000","last_fullsized":"2024-09-13T17:51:15.279638+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:34.870291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275058+0000","last_change":"2024-09-13T17:50:34.588466+0000","last_active":"2024-09-13T17:51:15.275058+0000","last_peered":"2024-09-13T17:51:15.275058+0000","last_clean":"2024-09-13T17:51:15.275058+0000","last_became_active":"2024-09-13T17:50:34.588063+0000","last_became_peered":"2024-09-13T17:50:34.588063+0000","last_unstale":"2024-09-13T17:51:15.275058+0000","last_undegraded":"2024-09-13T17:51:15.275058+0000","last_fullsized":"2024-09-13T17:51:15.275058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:43:00.669203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"25'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270012+0000","last_change":"2024-09-13T17:50:34.209239+0000","last_active":"2024-09-13T17:51:15.270012+0000","last_peered":"2024-09-13T17:51:15.270012+0000","last_clean":"2024-09-13T17:51:15.270012+0000","last_became_active":"2024-09-13T17:50:34.208196+0000","last_became_peered":"2024-09-13T17:50:34.208196+0000","last_unstale":"2024-09-13T17:51:15.270012+0000","last_undegraded":"2024-09-13T17:51:15.270012+0000","last_fullsized":"2024-09-13T17:51:15.270012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:39.253917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281566+0000","last_change":"2024-09-13T17:50:38.611021+0000","last_active":"2024-09-13T17:51:15.281566+0000","last_peered":"2024-09-13T17:51:15.281566+0000","last_clean":"2024-09-13T17:51:15.281566+0000","last_became_active":"2024-09-13T17:50:38.609794+0000","last_became_peered":"2024-09-13T17:50:38.609794+0000","last_unstale":"2024-09-13T17:51:15.281566+0000","last_undegraded":"2024-09-13T17:51:15.281566+0000","last_fullsized":"2024-09-13T17:51:15.281566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:59.980236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281553+0000","last_change":"2024-09-13T17:50:34.584290+0000","last_active":"2024-09-13T17:51:15.281553+0000","last_peered":"2024-09-13T17:51:15.281553+0000","last_clean":"2024-09-13T17:51:15.281553+0000","last_became_active":"2024-09-13T17:50:34.584023+0000","last_became_peered":"2024-09-13T17:50:34.584023+0000","last_unstale":"2024-09-13T17:51:15.281553+0000","last_undegraded":"2024-09-13T17:51:15.281553+0000","last_fullsized":"2024-09-13T17:51:15.281553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:11.304986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"25'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278819+0000","last_change":"2024-09-13T17:50:34.198932+0000","last_active":"2024-09-13T17:51:15.278819+0000","last_peered":"2024-09-13T17:51:15.278819+0000","last_clean":"2024-09-13T17:51:15.278819+0000","last_became_active":"2024-09-13T17:50:34.198603+0000","last_became_peered":"2024-09-13T17:50:34.198603+0000","last_unstale":"2024-09-13T17:51:15.278819+0000","last_undegraded":"2024-09-13T17:51:15.278819+0000","last_fullsized":"2024-09-13T17:51:15.278819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:00.917126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281714+0000","last_change":"2024-09-13T17:50:38.611326+0000","last_active":"2024-09-13T17:51:15.281714+0000","last_peered":"2024-09-13T17:51:15.281714+0000","last_clean":"2024-09-13T17:51:15.281714+0000","last_became_active":"2024-09-13T17:50:38.610900+0000","last_became_peered":"2024-09-13T17:50:38.610900+0000","last_unstale":"2024-09-13T17:51:15.281714+0000","last_undegraded":"2024-09-13T17:51:15.281714+0000","last_fullsized":"2024-09-13T17:51:15.281714+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:17:09.994736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276990+0000","last_change":"2024-09-13T17:50:34.594482+0000","last_active":"2024-09-13T17:51:15.276990+0000","last_peered":"2024-09-13T17:51:15.276990+0000","last_clean":"2024-09-13T17:51:15.276990+0000","last_became_active":"2024-09-13T17:50:34.594201+0000","last_became_peered":"2024-09-13T17:50:34.594201+0000","last_unstale":"2024-09-13T17:51:15.276990+0000","last_undegraded":"2024-09-13T17:51:15.276990+0000","last_fullsized":"2024-09-13T17:51:15.276990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:06.697507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277494+0000","last_change":"2024-09-13T17:50:34.218376+0000","last_active":"2024-09-13T17:51:15.277494+0000","last_peered":"2024-09-13T17:51:15.277494+0000","last_clean":"2024-09-13T17:51:15.277494+0000","last_became_active":"2024-09-13T17:50:34.218069+0000","last_became_peered":"2024-09-13T17:50:34.218069+0000","last_unstale":"2024-09-13T17:51:15.277494+0000","last_undegraded":"2024-09-13T17:51:15.277494+0000","last_fullsized":"2024-09-13T17:51:15.277494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:53:54.288969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277801+0000","last_change":"2024-09-13T17:50:38.613206+0000","last_active":"2024-09-13T17:51:15.277801+0000","last_peered":"2024-09-13T17:51:15.277801+0000","last_clean":"2024-09-13T17:51:15.277801+0000","last_became_active":"2024-09-13T17:50:38.612856+0000","last_became_peered":"2024-09-13T17:50:38.612856+0000","last_unstale":"2024-09-13T17:51:15.277801+0000","last_undegraded":"2024-09-13T17:51:15.277801+0000","last_fullsized":"2024-09-13T17:51:15.277801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:31:20.190996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270767+0000","last_change":"2024-09-13T17:50:34.582931+0000","last_active":"2024-09-13T17:51:15.270767+0000","last_peered":"2024-09-13T17:51:15.270767+0000","last_clean":"2024-09-13T17:51:15.270767+0000","last_became_active":"2024-09-13T17:50:34.582446+0000","last_became_peered":"2024-09-13T17:50:34.582446+0000","last_unstale":"2024-09-13T17:51:15.270767+0000","last_undegraded":"2024-09-13T17:51:15.270767+0000","last_fullsized":"2024-09-13T17:51:15.270767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:30.554785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"25'3","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270225+0000","last_change":"2024-09-13T17:50:34.201237+0000","last_active":"2024-09-13T17:51:15.270225+0000","last_peered":"2024-09-13T17:51:15.270225+0000","last_clean":"2024-09-13T17:51:15.270225+0000","last_became_active":"2024-09-13T17:50:34.200873+0000","last_became_peered":"2024-09-13T17:50:34.200873+0000","last_unstale":"2024-09-13T17:51:15.270225+0000","last_undegraded":"2024-09-13T17:51:15.270225+0000","last_fullsized":"2024-09-13T17:51:15.270225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:02.711342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277589+0000","last_change":"2024-09-13T17:50:22.095367+0000","last_active":"2024-09-13T17:51:15.277589+0000","last_peered":"2024-09-13T17:51:15.277589+0000","last_clean":"2024-09-13T17:51:15.277589+0000","last_became_active":"2024-09-13T17:50:22.093734+0000","last_became_peered":"2024-09-13T17:50:22.093734+0000","last_unstale":"2024-09-13T17:51:15.277589+0000","last_undegraded":"2024-09-13T17:51:15.277589+0000","last_fullsized":"2024-09-13T17:51:15.277589+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_clean_scrub_stamp":"2024-09-13T17:50:21.057812+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:23.531613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276680+0000","last_change":"2024-09-13T17:50:38.618880+0000","last_active":"2024-09-13T17:51:15.276680+0000","last_peered":"2024-09-13T17:51:15.276680+0000","last_clean":"2024-09-13T17:51:15.276680+0000","last_became_active":"2024-09-13T17:50:38.618488+0000","last_became_peered":"2024-09-13T17:50:38.618488+0000","last_unstale":"2024-09-13T17:51:15.276680+0000","last_undegraded":"2024-09-13T17:51:15.276680+0000","last_fullsized":"2024-09-13T17:51:15.276680+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:03.431876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279141+0000","last_change":"2024-09-13T17:50:34.585881+0000","last_active":"2024-09-13T17:51:15.279141+0000","last_peered":"2024-09-13T17:51:15.279141+0000","last_clean":"2024-09-13T17:51:15.279141+0000","last_became_active":"2024-09-13T17:50:34.585166+0000","last_became_peered":"2024-09-13T17:50:34.585166+0000","last_unstale":"2024-09-13T17:51:15.279141+0000","last_undegraded":"2024-09-13T17:51:15.279141+0000","last_fullsized":"2024-09-13T17:51:15.279141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:06.654658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279549+0000","last_change":"2024-09-13T17:50:34.210934+0000","last_active":"2024-09-13T17:51:15.279549+0000","last_peered":"2024-09-13T17:51:15.279549+0000","last_clean":"2024-09-13T17:51:15.279549+0000","last_became_active":"2024-09-13T17:50:34.210589+0000","last_became_peered":"2024-09-13T17:50:34.210589+0000","last_unstale":"2024-09-13T17:51:15.279549+0000","last_undegraded":"2024-09-13T17:51:15.279549+0000","last_fullsized":"2024-09-13T17:51:15.279549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:26.213295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.364911+0000","last_change":"2024-09-13T17:51:16.364911+0000","last_active":"2024-09-13T17:51:16.364911+0000","last_peered":"2024-09-13T17:51:16.364911+0000","last_clean":"2024-09-13T17:51:16.364911+0000","last_became_active":"2024-09-13T17:51:16.363974+0000","last_became_peered":"2024-09-13T17:51:16.363974+0000","last_unstale":"2024-09-13T17:51:16.364911+0000","last_undegraded":"2024-09-13T17:51:16.364911+0000","last_fullsized":"2024-09-13T17:51:16.364911+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:12.730197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271986+0000","last_change":"2024-09-13T17:50:34.590169+0000","last_active":"2024-09-13T17:51:15.271986+0000","last_peered":"2024-09-13T17:51:15.271986+0000","last_clean":"2024-09-13T17:51:15.271986+0000","last_became_active":"2024-09-13T17:50:34.589434+0000","last_became_peered":"2024-09-13T17:50:34.589434+0000","last_unstale":"2024-09-13T17:51:15.271986+0000","last_undegraded":"2024-09-13T17:51:15.271986+0000","last_fullsized":"2024-09-13T17:51:15.271986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:12.545412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"25'1","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281689+0000","last_change":"2024-09-13T17:50:34.204269+0000","last_active":"2024-09-13T17:51:15.281689+0000","last_peered":"2024-09-13T17:51:15.281689+0000","last_clean":"2024-09-13T17:51:15.281689+0000","last_became_active":"2024-09-13T17:50:34.203874+0000","last_became_peered":"2024-09-13T17:50:34.203874+0000","last_unstale":"2024-09-13T17:51:15.281689+0000","last_undegraded":"2024-09-13T17:51:15.281689+0000","last_fullsized":"2024-09-13T17:51:15.281689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:10.410267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278070+0000","last_change":"2024-09-13T17:50:38.615989+0000","last_active":"2024-09-13T17:51:15.278070+0000","last_peered":"2024-09-13T17:51:15.278070+0000","last_clean":"2024-09-13T17:51:15.278070+0000","last_became_active":"2024-09-13T17:50:38.615530+0000","last_became_peered":"2024-09-13T17:50:38.615530+0000","last_unstale":"2024-09-13T17:51:15.278070+0000","last_undegraded":"2024-09-13T17:51:15.278070+0000","last_fullsized":"2024-09-13T17:51:15.278070+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:41.064092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279600+0000","last_change":"2024-09-13T17:50:34.595120+0000","last_active":"2024-09-13T17:51:15.279600+0000","last_peered":"2024-09-13T17:51:15.279600+0000","last_clean":"2024-09-13T17:51:15.279600+0000","last_became_active":"2024-09-13T17:50:34.594842+0000","last_became_peered":"2024-09-13T17:50:34.594842+0000","last_unstale":"2024-09-13T17:51:15.279600+0000","last_undegraded":"2024-09-13T17:51:15.279600+0000","last_fullsized":"2024-09-13T17:51:15.279600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:15:38.982524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"25'2","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279806+0000","last_change":"2024-09-13T17:50:34.199957+0000","last_active":"2024-09-13T17:51:15.279806+0000","last_peered":"2024-09-13T17:51:15.279806+0000","last_clean":"2024-09-13T17:51:15.279806+0000","last_became_active":"2024-09-13T17:50:34.199650+0000","last_became_peered":"2024-09-13T17:50:34.199650+0000","last_unstale":"2024-09-13T17:51:15.279806+0000","last_undegraded":"2024-09-13T17:51:15.279806+0000","last_fullsized":"2024-09-13T17:51:15.279806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:26.929549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269062+0000","last_change":"2024-09-13T17:50:38.609731+0000","last_active":"2024-09-13T17:51:15.269062+0000","last_peered":"2024-09-13T17:51:15.269062+0000","last_clean":"2024-09-13T17:51:15.269062+0000","last_became_active":"2024-09-13T17:50:38.609354+0000","last_became_peered":"2024-09-13T17:50:38.609354+0000","last_unstale":"2024-09-13T17:51:15.269062+0000","last_undegraded":"2024-09-13T17:51:15.269062+0000","last_fullsized":"2024-09-13T17:51:15.269062+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:14.142369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279334+0000","last_change":"2024-09-13T17:50:34.595146+0000","last_active":"2024-09-13T17:51:15.279334+0000","last_peered":"2024-09-13T17:51:15.279334+0000","last_clean":"2024-09-13T17:51:15.279334+0000","last_became_active":"2024-09-13T17:50:34.594876+0000","last_became_peered":"2024-09-13T17:50:34.594876+0000","last_unstale":"2024-09-13T17:51:15.279334+0000","last_undegraded":"2024-09-13T17:51:15.279334+0000","last_fullsized":"2024-09-13T17:51:15.279334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:26:10.638438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'2","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.278874+0000","last_change":"2024-09-13T17:50:34.200640+0000","last_active":"2024-09-13T17:51:16.278874+0000","last_peered":"2024-09-13T17:51:16.278874+0000","last_clean":"2024-09-13T17:51:16.278874+0000","last_became_active":"2024-09-13T17:50:34.200197+0000","last_became_peered":"2024-09-13T17:50:34.200197+0000","last_unstale":"2024-09-13T17:51:16.278874+0000","last_undegraded":"2024-09-13T17:51:16.278874+0000","last_fullsized":"2024-09-13T17:51:16.278874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:37:53.039720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.281995+0000","last_change":"2024-09-13T17:51:16.281995+0000","last_active":"2024-09-13T17:51:16.281995+0000","last_peered":"2024-09-13T17:51:16.281995+0000","last_clean":"2024-09-13T17:51:16.281995+0000","last_became_active":"2024-09-13T17:51:16.281476+0000","last_became_peered":"2024-09-13T17:51:16.281476+0000","last_unstale":"2024-09-13T17:51:16.281995+0000","last_undegraded":"2024-09-13T17:51:16.281995+0000","last_fullsized":"2024-09-13T17:51:16.281995+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:21:56.254641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3,6],"acting":[4,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278650+0000","last_change":"2024-09-13T17:50:34.585741+0000","last_active":"2024-09-13T17:51:15.278650+0000","last_peered":"2024-09-13T17:51:15.278650+0000","last_clean":"2024-09-13T17:51:15.278650+0000","last_became_active":"2024-09-13T17:50:34.585464+0000","last_became_peered":"2024-09-13T17:50:34.585464+0000","last_unstale":"2024-09-13T17:51:15.278650+0000","last_undegraded":"2024-09-13T17:51:15.278650+0000","last_fullsized":"2024-09-13T17:51:15.278650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:59:22.031459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"25'15","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278697+0000","last_change":"2024-09-13T17:50:34.201272+0000","last_active":"2024-09-13T17:51:15.278697+0000","last_peered":"2024-09-13T17:51:15.278697+0000","last_clean":"2024-09-13T17:51:15.278697+0000","last_became_active":"2024-09-13T17:50:34.200936+0000","last_became_peered":"2024-09-13T17:50:34.200936+0000","last_unstale":"2024-09-13T17:51:15.278697+0000","last_undegraded":"2024-09-13T17:51:15.278697+0000","last_fullsized":"2024-09-13T17:51:15.278697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:52.094879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277648+0000","last_change":"2024-09-13T17:50:38.615229+0000","last_active":"2024-09-13T17:51:15.277648+0000","last_peered":"2024-09-13T17:51:15.277648+0000","last_clean":"2024-09-13T17:51:15.277648+0000","last_became_active":"2024-09-13T17:50:38.614739+0000","last_became_peered":"2024-09-13T17:50:38.614739+0000","last_unstale":"2024-09-13T17:51:15.277648+0000","last_undegraded":"2024-09-13T17:51:15.277648+0000","last_fullsized":"2024-09-13T17:51:15.277648+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:18.619529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275934+0000","last_change":"2024-09-13T17:50:34.588848+0000","last_active":"2024-09-13T17:51:15.275934+0000","last_peered":"2024-09-13T17:51:15.275934+0000","last_clean":"2024-09-13T17:51:15.275934+0000","last_became_active":"2024-09-13T17:50:34.588565+0000","last_became_peered":"2024-09-13T17:50:34.588565+0000","last_unstale":"2024-09-13T17:51:15.275934+0000","last_undegraded":"2024-09-13T17:51:15.275934+0000","last_fullsized":"2024-09-13T17:51:15.275934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:47.927865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278190+0000","last_change":"2024-09-13T17:50:34.199008+0000","last_active":"2024-09-13T17:51:15.278190+0000","last_peered":"2024-09-13T17:51:15.278190+0000","last_clean":"2024-09-13T17:51:15.278190+0000","last_became_active":"2024-09-13T17:50:34.197839+0000","last_became_peered":"2024-09-13T17:50:34.197839+0000","last_unstale":"2024-09-13T17:51:15.278190+0000","last_undegraded":"2024-09-13T17:51:15.278190+0000","last_fullsized":"2024-09-13T17:51:15.278190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:51.430710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272409+0000","last_change":"2024-09-13T17:50:38.611080+0000","last_active":"2024-09-13T17:51:15.272409+0000","last_peered":"2024-09-13T17:51:15.272409+0000","last_clean":"2024-09-13T17:51:15.272409+0000","last_became_active":"2024-09-13T17:50:38.610688+0000","last_became_peered":"2024-09-13T17:50:38.610688+0000","last_unstale":"2024-09-13T17:51:15.272409+0000","last_undegraded":"2024-09-13T17:51:15.272409+0000","last_fullsized":"2024-09-13T17:51:15.272409+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:28.527617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278851+0000","last_change":"2024-09-13T17:50:34.594763+0000","last_active":"2024-09-13T17:51:15.278851+0000","last_peered":"2024-09-13T17:51:15.278851+0000","last_clean":"2024-09-13T17:51:15.278851+0000","last_became_active":"2024-09-13T17:50:34.594471+0000","last_became_peered":"2024-09-13T17:50:34.594471+0000","last_unstale":"2024-09-13T17:51:15.278851+0000","last_undegraded":"2024-09-13T17:51:15.278851+0000","last_fullsized":"2024-09-13T17:51:15.278851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:20:58.718271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"25'3","reported_seq":19,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269946+0000","last_change":"2024-09-13T17:50:34.209432+0000","last_active":"2024-09-13T17:51:15.269946+0000","last_peered":"2024-09-13T17:51:15.269946+0000","last_clean":"2024-09-13T17:51:15.269946+0000","last_became_active":"2024-09-13T17:50:34.208817+0000","last_became_peered":"2024-09-13T17:50:34.208817+0000","last_unstale":"2024-09-13T17:51:15.269946+0000","last_undegraded":"2024-09-13T17:51:15.269946+0000","last_fullsized":"2024-09-13T17:51:15.269946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:49.587528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272159+0000","last_change":"2024-09-13T17:50:38.609461+0000","last_active":"2024-09-13T17:51:15.272159+0000","last_peered":"2024-09-13T17:51:15.272159+0000","last_clean":"2024-09-13T17:51:15.272159+0000","last_became_active":"2024-09-13T17:50:38.609128+0000","last_became_peered":"2024-09-13T17:50:38.609128+0000","last_unstale":"2024-09-13T17:51:15.272159+0000","last_undegraded":"2024-09-13T17:51:15.272159+0000","last_fullsized":"2024-09-13T17:51:15.272159+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:02.286825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.283064+0000","last_change":"2024-09-13T17:50:34.592970+0000","last_active":"2024-09-13T17:51:15.283064+0000","last_peered":"2024-09-13T17:51:15.283064+0000","last_clean":"2024-09-13T17:51:15.283064+0000","last_became_active":"2024-09-13T17:50:34.592669+0000","last_became_peered":"2024-09-13T17:50:34.592669+0000","last_unstale":"2024-09-13T17:51:15.283064+0000","last_undegraded":"2024-09-13T17:51:15.283064+0000","last_fullsized":"2024-09-13T17:51:15.283064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:52.031261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'6","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282599+0000","last_change":"2024-09-13T17:50:34.200798+0000","last_active":"2024-09-13T17:51:15.282599+0000","last_peered":"2024-09-13T17:51:15.282599+0000","last_clean":"2024-09-13T17:51:15.282599+0000","last_became_active":"2024-09-13T17:50:34.200529+0000","last_became_peered":"2024-09-13T17:50:34.200529+0000","last_unstale":"2024-09-13T17:51:15.282599+0000","last_undegraded":"2024-09-13T17:51:15.282599+0000","last_fullsized":"2024-09-13T17:51:15.282599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:26.334296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270388+0000","last_change":"2024-09-13T17:50:38.606287+0000","last_active":"2024-09-13T17:51:15.270388+0000","last_peered":"2024-09-13T17:51:15.270388+0000","last_clean":"2024-09-13T17:51:15.270388+0000","last_became_active":"2024-09-13T17:50:38.605826+0000","last_became_peered":"2024-09-13T17:50:38.605826+0000","last_unstale":"2024-09-13T17:51:15.270388+0000","last_undegraded":"2024-09-13T17:51:15.270388+0000","last_fullsized":"2024-09-13T17:51:15.270388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:18.709454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279140+0000","last_change":"2024-09-13T17:50:34.591579+0000","last_active":"2024-09-13T17:51:15.279140+0000","last_peered":"2024-09-13T17:51:15.279140+0000","last_clean":"2024-09-13T17:51:15.279140+0000","last_became_active":"2024-09-13T17:50:34.586165+0000","last_became_peered":"2024-09-13T17:50:34.586165+0000","last_unstale":"2024-09-13T17:51:15.279140+0000","last_undegraded":"2024-09-13T17:51:15.279140+0000","last_fullsized":"2024-09-13T17:51:15.279140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:56:19.205176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'11","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277643+0000","last_change":"2024-09-13T17:50:34.203218+0000","last_active":"2024-09-13T17:51:15.277643+0000","last_peered":"2024-09-13T17:51:15.277643+0000","last_clean":"2024-09-13T17:51:15.277643+0000","last_became_active":"2024-09-13T17:50:34.202948+0000","last_became_peered":"2024-09-13T17:50:34.202948+0000","last_unstale":"2024-09-13T17:51:15.277643+0000","last_undegraded":"2024-09-13T17:51:15.277643+0000","last_fullsized":"2024-09-13T17:51:15.277643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:46.535569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3795,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.280382+0000","last_change":"2024-09-13T17:50:38.610597+0000","last_active":"2024-09-13T17:51:15.280382+0000","last_peered":"2024-09-13T17:51:15.280382+0000","last_clean":"2024-09-13T17:51:15.280382+0000","last_became_active":"2024-09-13T17:50:38.610055+0000","last_became_peered":"2024-09-13T17:50:38.610055+0000","last_unstale":"2024-09-13T17:51:15.280382+0000","last_undegraded":"2024-09-13T17:51:15.280382+0000","last_fullsized":"2024-09-13T17:51:15.280382+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:55.754936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275920+0000","last_change":"2024-09-13T17:50:34.593963+0000","last_active":"2024-09-13T17:51:15.275920+0000","last_peered":"2024-09-13T17:51:15.275920+0000","last_clean":"2024-09-13T17:51:15.275920+0000","last_became_active":"2024-09-13T17:50:34.593454+0000","last_became_peered":"2024-09-13T17:50:34.593454+0000","last_unstale":"2024-09-13T17:51:15.275920+0000","last_undegraded":"2024-09-13T17:51:15.275920+0000","last_fullsized":"2024-09-13T17:51:15.275920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:18.883521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"25'4","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281506+0000","last_change":"2024-09-13T17:50:34.207936+0000","last_active":"2024-09-13T17:51:15.281506+0000","last_peered":"2024-09-13T17:51:15.281506+0000","last_clean":"2024-09-13T17:51:15.281506+0000","last_became_active":"2024-09-13T17:50:34.207605+0000","last_became_peered":"2024-09-13T17:50:34.207605+0000","last_unstale":"2024-09-13T17:51:15.281506+0000","last_undegraded":"2024-09-13T17:51:15.281506+0000","last_fullsized":"2024-09-13T17:51:15.281506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:05.494406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.274690+0000","last_change":"2024-09-13T17:50:38.616297+0000","last_active":"2024-09-13T17:51:15.274690+0000","last_peered":"2024-09-13T17:51:15.274690+0000","last_clean":"2024-09-13T17:51:15.274690+0000","last_became_active":"2024-09-13T17:50:38.614165+0000","last_became_peered":"2024-09-13T17:50:38.614165+0000","last_unstale":"2024-09-13T17:51:15.274690+0000","last_undegraded":"2024-09-13T17:51:15.274690+0000","last_fullsized":"2024-09-13T17:51:15.274690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:11.357026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278023+0000","last_change":"2024-09-13T17:50:34.583001+0000","last_active":"2024-09-13T17:51:15.278023+0000","last_peered":"2024-09-13T17:51:15.278023+0000","last_clean":"2024-09-13T17:51:15.278023+0000","last_became_active":"2024-09-13T17:50:34.582529+0000","last_became_peered":"2024-09-13T17:50:34.582529+0000","last_unstale":"2024-09-13T17:51:15.278023+0000","last_undegraded":"2024-09-13T17:51:15.278023+0000","last_fullsized":"2024-09-13T17:51:15.278023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:05.287181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"25'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271281+0000","last_change":"2024-09-13T17:50:34.205138+0000","last_active":"2024-09-13T17:51:15.271281+0000","last_peered":"2024-09-13T17:51:15.271281+0000","last_clean":"2024-09-13T17:51:15.271281+0000","last_became_active":"2024-09-13T17:50:34.203360+0000","last_became_peered":"2024-09-13T17:50:34.203360+0000","last_unstale":"2024-09-13T17:51:15.271281+0000","last_undegraded":"2024-09-13T17:51:15.271281+0000","last_fullsized":"2024-09-13T17:51:15.271281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:35.756963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272372+0000","last_change":"2024-09-13T17:50:38.611036+0000","last_active":"2024-09-13T17:51:15.272372+0000","last_peered":"2024-09-13T17:51:15.272372+0000","last_clean":"2024-09-13T17:51:15.272372+0000","last_became_active":"2024-09-13T17:50:38.610692+0000","last_became_peered":"2024-09-13T17:50:38.610692+0000","last_unstale":"2024-09-13T17:51:15.272372+0000","last_undegraded":"2024-09-13T17:51:15.272372+0000","last_fullsized":"2024-09-13T17:51:15.272372+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:10.147998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.368023+0000","last_change":"2024-09-13T17:51:16.365295+0000","last_active":"2024-09-13T17:51:16.368023+0000","last_peered":"2024-09-13T17:51:16.368023+0000","last_clean":"2024-09-13T17:51:16.368023+0000","last_became_active":"2024-09-13T17:51:16.364437+0000","last_became_peered":"2024-09-13T17:51:16.364437+0000","last_unstale":"2024-09-13T17:51:16.368023+0000","last_undegraded":"2024-09-13T17:51:16.368023+0000","last_fullsized":"2024-09-13T17:51:16.368023+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:23.208899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276819+0000","last_change":"2024-09-13T17:50:34.204376+0000","last_active":"2024-09-13T17:51:15.276819+0000","last_peered":"2024-09-13T17:51:15.276819+0000","last_clean":"2024-09-13T17:51:15.276819+0000","last_became_active":"2024-09-13T17:50:34.204003+0000","last_became_peered":"2024-09-13T17:50:34.204003+0000","last_unstale":"2024-09-13T17:51:15.276819+0000","last_undegraded":"2024-09-13T17:51:15.276819+0000","last_fullsized":"2024-09-13T17:51:15.276819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:14.023990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270270+0000","last_change":"2024-09-13T17:50:38.606144+0000","last_active":"2024-09-13T17:51:15.270270+0000","last_peered":"2024-09-13T17:51:15.270270+0000","last_clean":"2024-09-13T17:51:15.270270+0000","last_became_active":"2024-09-13T17:50:38.605646+0000","last_became_peered":"2024-09-13T17:50:38.605646+0000","last_unstale":"2024-09-13T17:51:15.270270+0000","last_undegraded":"2024-09-13T17:51:15.270270+0000","last_fullsized":"2024-09-13T17:51:15.270270+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:07.120075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277435+0000","last_change":"2024-09-13T17:50:38.618855+0000","last_active":"2024-09-13T17:51:15.277435+0000","last_peered":"2024-09-13T17:51:15.277435+0000","last_clean":"2024-09-13T17:51:15.277435+0000","last_became_active":"2024-09-13T17:50:38.618449+0000","last_became_peered":"2024-09-13T17:50:38.618449+0000","last_unstale":"2024-09-13T17:51:15.277435+0000","last_undegraded":"2024-09-13T17:51:15.277435+0000","last_fullsized":"2024-09-13T17:51:15.277435+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:29:01.114087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.280511+0000","last_change":"2024-09-13T17:50:34.590079+0000","last_active":"2024-09-13T17:51:15.280511+0000","last_peered":"2024-09-13T17:51:15.280511+0000","last_clean":"2024-09-13T17:51:15.280511+0000","last_became_active":"2024-09-13T17:50:34.589396+0000","last_became_peered":"2024-09-13T17:50:34.589396+0000","last_unstale":"2024-09-13T17:51:15.280511+0000","last_undegraded":"2024-09-13T17:51:15.280511+0000","last_fullsized":"2024-09-13T17:51:15.280511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:03.308341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269447+0000","last_change":"2024-09-13T17:50:34.206575+0000","last_active":"2024-09-13T17:51:15.269447+0000","last_peered":"2024-09-13T17:51:15.269447+0000","last_clean":"2024-09-13T17:51:15.269447+0000","last_became_active":"2024-09-13T17:50:34.206308+0000","last_became_peered":"2024-09-13T17:50:34.206308+0000","last_unstale":"2024-09-13T17:51:15.269447+0000","last_undegraded":"2024-09-13T17:51:15.269447+0000","last_fullsized":"2024-09-13T17:51:15.269447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:34.766538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276602+0000","last_change":"2024-09-13T17:50:38.618131+0000","last_active":"2024-09-13T17:51:15.276602+0000","last_peered":"2024-09-13T17:51:15.276602+0000","last_clean":"2024-09-13T17:51:15.276602+0000","last_became_active":"2024-09-13T17:50:38.617611+0000","last_became_peered":"2024-09-13T17:50:38.617611+0000","last_unstale":"2024-09-13T17:51:15.276602+0000","last_undegraded":"2024-09-13T17:51:15.276602+0000","last_fullsized":"2024-09-13T17:51:15.276602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:30.746445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272535+0000","last_change":"2024-09-13T17:50:34.582153+0000","last_active":"2024-09-13T17:51:15.272535+0000","last_peered":"2024-09-13T17:51:15.272535+0000","last_clean":"2024-09-13T17:51:15.272535+0000","last_became_active":"2024-09-13T17:50:34.581708+0000","last_became_peered":"2024-09-13T17:50:34.581708+0000","last_unstale":"2024-09-13T17:51:15.272535+0000","last_undegraded":"2024-09-13T17:51:15.272535+0000","last_fullsized":"2024-09-13T17:51:15.272535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:22.238810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"25'2","reported_seq":18,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270828+0000","last_change":"2024-09-13T17:50:34.205489+0000","last_active":"2024-09-13T17:51:15.270828+0000","last_peered":"2024-09-13T17:51:15.270828+0000","last_clean":"2024-09-13T17:51:15.270828+0000","last_became_active":"2024-09-13T17:50:34.205202+0000","last_became_peered":"2024-09-13T17:50:34.205202+0000","last_unstale":"2024-09-13T17:51:15.270828+0000","last_undegraded":"2024-09-13T17:51:15.270828+0000","last_fullsized":"2024-09-13T17:51:15.270828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:54.450276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"25'3","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276330+0000","last_change":"2024-09-13T17:50:34.202470+0000","last_active":"2024-09-13T17:51:15.276330+0000","last_peered":"2024-09-13T17:51:15.276330+0000","last_clean":"2024-09-13T17:51:15.276330+0000","last_became_active":"2024-09-13T17:50:34.202163+0000","last_became_peered":"2024-09-13T17:50:34.202163+0000","last_unstale":"2024-09-13T17:51:15.276330+0000","last_undegraded":"2024-09-13T17:51:15.276330+0000","last_fullsized":"2024-09-13T17:51:15.276330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:03.472932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282749+0000","last_change":"2024-09-13T17:50:34.590947+0000","last_active":"2024-09-13T17:51:15.282749+0000","last_peered":"2024-09-13T17:51:15.282749+0000","last_clean":"2024-09-13T17:51:15.282749+0000","last_became_active":"2024-09-13T17:50:34.590497+0000","last_became_peered":"2024-09-13T17:50:34.590497+0000","last_unstale":"2024-09-13T17:51:15.282749+0000","last_undegraded":"2024-09-13T17:51:15.282749+0000","last_fullsized":"2024-09-13T17:51:15.282749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:56:27.848092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279328+0000","last_change":"2024-09-13T17:50:38.618518+0000","last_active":"2024-09-13T17:51:15.279328+0000","last_peered":"2024-09-13T17:51:15.279328+0000","last_clean":"2024-09-13T17:51:15.279328+0000","last_became_active":"2024-09-13T17:50:38.618108+0000","last_became_peered":"2024-09-13T17:50:38.618108+0000","last_unstale":"2024-09-13T17:51:15.279328+0000","last_undegraded":"2024-09-13T17:51:15.279328+0000","last_fullsized":"2024-09-13T17:51:15.279328+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:42.260237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'6","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.273983+0000","last_change":"2024-09-13T17:50:34.218459+0000","last_active":"2024-09-13T17:51:16.273983+0000","last_peered":"2024-09-13T17:51:16.273983+0000","last_clean":"2024-09-13T17:51:16.273983+0000","last_became_active":"2024-09-13T17:50:34.218087+0000","last_became_peered":"2024-09-13T17:50:34.218087+0000","last_unstale":"2024-09-13T17:51:16.273983+0000","last_undegraded":"2024-09-13T17:51:16.273983+0000","last_fullsized":"2024-09-13T17:51:16.273983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:56.914544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1016,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.274322+0000","last_change":"2024-09-13T17:50:34.583845+0000","last_active":"2024-09-13T17:51:15.274322+0000","last_peered":"2024-09-13T17:51:15.274322+0000","last_clean":"2024-09-13T17:51:15.274322+0000","last_became_active":"2024-09-13T17:50:34.583283+0000","last_became_peered":"2024-09-13T17:50:34.583283+0000","last_unstale":"2024-09-13T17:51:15.274322+0000","last_undegraded":"2024-09-13T17:51:15.274322+0000","last_fullsized":"2024-09-13T17:51:15.274322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:55.630285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271562+0000","last_change":"2024-09-13T17:50:38.611987+0000","last_active":"2024-09-13T17:51:15.271562+0000","last_peered":"2024-09-13T17:51:15.271562+0000","last_clean":"2024-09-13T17:51:15.271562+0000","last_became_active":"2024-09-13T17:50:38.611636+0000","last_became_peered":"2024-09-13T17:50:38.611636+0000","last_unstale":"2024-09-13T17:51:15.271562+0000","last_undegraded":"2024-09-13T17:51:15.271562+0000","last_fullsized":"2024-09-13T17:51:15.271562+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:41.111562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275726+0000","last_change":"2024-09-13T17:50:34.588927+0000","last_active":"2024-09-13T17:51:15.275726+0000","last_peered":"2024-09-13T17:51:15.275726+0000","last_clean":"2024-09-13T17:51:15.275726+0000","last_became_active":"2024-09-13T17:50:34.588651+0000","last_became_peered":"2024-09-13T17:50:34.588651+0000","last_unstale":"2024-09-13T17:51:15.275726+0000","last_undegraded":"2024-09-13T17:51:15.275726+0000","last_fullsized":"2024-09-13T17:51:15.275726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:49.429670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278614+0000","last_change":"2024-09-13T17:50:34.583795+0000","last_active":"2024-09-13T17:51:15.278614+0000","last_peered":"2024-09-13T17:51:15.278614+0000","last_clean":"2024-09-13T17:51:15.278614+0000","last_became_active":"2024-09-13T17:50:34.583493+0000","last_became_peered":"2024-09-13T17:50:34.583493+0000","last_unstale":"2024-09-13T17:51:15.278614+0000","last_undegraded":"2024-09-13T17:51:15.278614+0000","last_fullsized":"2024-09-13T17:51:15.278614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:30.463932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269876+0000","last_change":"2024-09-13T17:50:34.587624+0000","last_active":"2024-09-13T17:51:15.269876+0000","last_peered":"2024-09-13T17:51:15.269876+0000","last_clean":"2024-09-13T17:51:15.269876+0000","last_became_active":"2024-09-13T17:50:34.587119+0000","last_became_peered":"2024-09-13T17:50:34.587119+0000","last_unstale":"2024-09-13T17:51:15.269876+0000","last_undegraded":"2024-09-13T17:51:15.269876+0000","last_fullsized":"2024-09-13T17:51:15.269876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:16.783899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278177+0000","last_change":"2024-09-13T17:50:34.581923+0000","last_active":"2024-09-13T17:51:15.278177+0000","last_peered":"2024-09-13T17:51:15.278177+0000","last_clean":"2024-09-13T17:51:15.278177+0000","last_became_active":"2024-09-13T17:50:34.581628+0000","last_became_peered":"2024-09-13T17:50:34.581628+0000","last_unstale":"2024-09-13T17:51:15.278177+0000","last_undegraded":"2024-09-13T17:51:15.278177+0000","last_fullsized":"2024-09-13T17:51:15.278177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:28:55.936864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272460+0000","last_change":"2024-09-13T17:50:34.595037+0000","last_active":"2024-09-13T17:51:15.272460+0000","last_peered":"2024-09-13T17:51:15.272460+0000","last_clean":"2024-09-13T17:51:15.272460+0000","last_became_active":"2024-09-13T17:50:34.594857+0000","last_became_peered":"2024-09-13T17:50:34.594857+0000","last_unstale":"2024-09-13T17:51:15.272460+0000","last_undegraded":"2024-09-13T17:51:15.272460+0000","last_fullsized":"2024-09-13T17:51:15.272460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:11.420413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281006+0000","last_change":"2024-09-13T17:50:34.589651+0000","last_active":"2024-09-13T17:51:15.281006+0000","last_peered":"2024-09-13T17:51:15.281006+0000","last_clean":"2024-09-13T17:51:15.281006+0000","last_became_active":"2024-09-13T17:50:34.589354+0000","last_became_peered":"2024-09-13T17:50:34.589354+0000","last_unstale":"2024-09-13T17:51:15.281006+0000","last_undegraded":"2024-09-13T17:51:15.281006+0000","last_fullsized":"2024-09-13T17:51:15.281006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:31.294048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277272+0000","last_change":"2024-09-13T17:50:34.597949+0000","last_active":"2024-09-13T17:51:15.277272+0000","last_peered":"2024-09-13T17:51:15.277272+0000","last_clean":"2024-09-13T17:51:15.277272+0000","last_became_active":"2024-09-13T17:50:34.597621+0000","last_became_peered":"2024-09-13T17:50:34.597621+0000","last_unstale":"2024-09-13T17:51:15.277272+0000","last_undegraded":"2024-09-13T17:51:15.277272+0000","last_fullsized":"2024-09-13T17:51:15.277272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:23.133462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.272676+0000","last_change":"2024-09-13T17:50:34.589893+0000","last_active":"2024-09-13T17:51:15.272676+0000","last_peered":"2024-09-13T17:51:15.272676+0000","last_clean":"2024-09-13T17:51:15.272676+0000","last_became_active":"2024-09-13T17:50:34.589626+0000","last_became_peered":"2024-09-13T17:50:34.589626+0000","last_unstale":"2024-09-13T17:51:15.272676+0000","last_undegraded":"2024-09-13T17:51:15.272676+0000","last_fullsized":"2024-09-13T17:51:15.272676+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:41.321025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269328+0000","last_change":"2024-09-13T17:50:34.590046+0000","last_active":"2024-09-13T17:51:15.269328+0000","last_peered":"2024-09-13T17:51:15.269328+0000","last_clean":"2024-09-13T17:51:15.269328+0000","last_became_active":"2024-09-13T17:50:34.589757+0000","last_became_peered":"2024-09-13T17:50:34.589757+0000","last_unstale":"2024-09-13T17:51:15.269328+0000","last_undegraded":"2024-09-13T17:51:15.269328+0000","last_fullsized":"2024-09-13T17:51:15.269328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:28.239550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270567+0000","last_change":"2024-09-13T17:50:34.595518+0000","last_active":"2024-09-13T17:51:15.270567+0000","last_peered":"2024-09-13T17:51:15.270567+0000","last_clean":"2024-09-13T17:51:15.270567+0000","last_became_active":"2024-09-13T17:50:34.595242+0000","last_became_peered":"2024-09-13T17:50:34.595242+0000","last_unstale":"2024-09-13T17:51:15.270567+0000","last_undegraded":"2024-09-13T17:51:15.270567+0000","last_fullsized":"2024-09-13T17:51:15.270567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:26.149384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269781+0000","last_change":"2024-09-13T17:50:34.593183+0000","last_active":"2024-09-13T17:51:15.269781+0000","last_peered":"2024-09-13T17:51:15.269781+0000","last_clean":"2024-09-13T17:51:15.269781+0000","last_became_active":"2024-09-13T17:50:34.592893+0000","last_became_peered":"2024-09-13T17:50:34.592893+0000","last_unstale":"2024-09-13T17:51:15.269781+0000","last_undegraded":"2024-09-13T17:51:15.269781+0000","last_fullsized":"2024-09-13T17:51:15.269781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:19.459505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278000+0000","last_change":"2024-09-13T17:50:34.594679+0000","last_active":"2024-09-13T17:51:15.278000+0000","last_peered":"2024-09-13T17:51:15.278000+0000","last_clean":"2024-09-13T17:51:15.278000+0000","last_became_active":"2024-09-13T17:50:34.593750+0000","last_became_peered":"2024-09-13T17:50:34.593750+0000","last_unstale":"2024-09-13T17:51:15.278000+0000","last_undegraded":"2024-09-13T17:51:15.278000+0000","last_fullsized":"2024-09-13T17:51:15.278000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:02.135577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277577+0000","last_change":"2024-09-13T17:50:34.598736+0000","last_active":"2024-09-13T17:51:15.277577+0000","last_peered":"2024-09-13T17:51:15.277577+0000","last_clean":"2024-09-13T17:51:15.277577+0000","last_became_active":"2024-09-13T17:50:34.598433+0000","last_became_peered":"2024-09-13T17:50:34.598433+0000","last_unstale":"2024-09-13T17:51:15.277577+0000","last_undegraded":"2024-09-13T17:51:15.277577+0000","last_fullsized":"2024-09-13T17:51:15.277577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:08.975139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279323+0000","last_change":"2024-09-13T17:50:34.591795+0000","last_active":"2024-09-13T17:51:15.279323+0000","last_peered":"2024-09-13T17:51:15.279323+0000","last_clean":"2024-09-13T17:51:15.279323+0000","last_became_active":"2024-09-13T17:50:34.591286+0000","last_became_peered":"2024-09-13T17:50:34.591286+0000","last_unstale":"2024-09-13T17:51:15.279323+0000","last_undegraded":"2024-09-13T17:51:15.279323+0000","last_fullsized":"2024-09-13T17:51:15.279323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:31.580247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279687+0000","last_change":"2024-09-13T17:50:34.587774+0000","last_active":"2024-09-13T17:51:15.279687+0000","last_peered":"2024-09-13T17:51:15.279687+0000","last_clean":"2024-09-13T17:51:15.279687+0000","last_became_active":"2024-09-13T17:50:34.587506+0000","last_became_peered":"2024-09-13T17:50:34.587506+0000","last_unstale":"2024-09-13T17:51:15.279687+0000","last_undegraded":"2024-09-13T17:51:15.279687+0000","last_fullsized":"2024-09-13T17:51:15.279687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:46.275528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271196+0000","last_change":"2024-09-13T17:50:34.596100+0000","last_active":"2024-09-13T17:51:15.271196+0000","last_peered":"2024-09-13T17:51:15.271196+0000","last_clean":"2024-09-13T17:51:15.271196+0000","last_became_active":"2024-09-13T17:50:34.595798+0000","last_became_peered":"2024-09-13T17:50:34.595798+0000","last_unstale":"2024-09-13T17:51:15.271196+0000","last_undegraded":"2024-09-13T17:51:15.271196+0000","last_fullsized":"2024-09-13T17:51:15.271196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:49.543845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269217+0000","last_change":"2024-09-13T17:50:34.584818+0000","last_active":"2024-09-13T17:51:15.269217+0000","last_peered":"2024-09-13T17:51:15.269217+0000","last_clean":"2024-09-13T17:51:15.269217+0000","last_became_active":"2024-09-13T17:50:34.584516+0000","last_became_peered":"2024-09-13T17:50:34.584516+0000","last_unstale":"2024-09-13T17:51:15.269217+0000","last_undegraded":"2024-09-13T17:51:15.269217+0000","last_fullsized":"2024-09-13T17:51:15.269217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:52.402024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281649+0000","last_change":"2024-09-13T17:50:34.598886+0000","last_active":"2024-09-13T17:51:15.281649+0000","last_peered":"2024-09-13T17:51:15.281649+0000","last_clean":"2024-09-13T17:51:15.281649+0000","last_became_active":"2024-09-13T17:50:34.598648+0000","last_became_peered":"2024-09-13T17:50:34.598648+0000","last_unstale":"2024-09-13T17:51:15.281649+0000","last_undegraded":"2024-09-13T17:51:15.281649+0000","last_fullsized":"2024-09-13T17:51:15.281649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:00:43.063223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.282938+0000","last_change":"2024-09-13T17:50:34.582649+0000","last_active":"2024-09-13T17:51:15.282938+0000","last_peered":"2024-09-13T17:51:15.282938+0000","last_clean":"2024-09-13T17:51:15.282938+0000","last_became_active":"2024-09-13T17:50:34.581699+0000","last_became_peered":"2024-09-13T17:50:34.581699+0000","last_unstale":"2024-09-13T17:51:15.282938+0000","last_undegraded":"2024-09-13T17:51:15.282938+0000","last_fullsized":"2024-09-13T17:51:15.282938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:18.959303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.270646+0000","last_change":"2024-09-13T17:50:34.587484+0000","last_active":"2024-09-13T17:51:15.270646+0000","last_peered":"2024-09-13T17:51:15.270646+0000","last_clean":"2024-09-13T17:51:15.270646+0000","last_became_active":"2024-09-13T17:50:34.587217+0000","last_became_peered":"2024-09-13T17:50:34.587217+0000","last_unstale":"2024-09-13T17:51:15.270646+0000","last_undegraded":"2024-09-13T17:51:15.270646+0000","last_fullsized":"2024-09-13T17:51:15.270646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:24:19.172607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.275806+0000","last_change":"2024-09-13T17:50:34.592871+0000","last_active":"2024-09-13T17:51:15.275806+0000","last_peered":"2024-09-13T17:51:15.275806+0000","last_clean":"2024-09-13T17:51:15.275806+0000","last_became_active":"2024-09-13T17:50:34.588513+0000","last_became_peered":"2024-09-13T17:50:34.588513+0000","last_unstale":"2024-09-13T17:51:15.275806+0000","last_undegraded":"2024-09-13T17:51:15.275806+0000","last_fullsized":"2024-09-13T17:51:15.275806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:33.131420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.276633+0000","last_change":"2024-09-13T17:50:34.585713+0000","last_active":"2024-09-13T17:51:15.276633+0000","last_peered":"2024-09-13T17:51:15.276633+0000","last_clean":"2024-09-13T17:51:15.276633+0000","last_became_active":"2024-09-13T17:50:34.585438+0000","last_became_peered":"2024-09-13T17:50:34.585438+0000","last_unstale":"2024-09-13T17:51:15.276633+0000","last_undegraded":"2024-09-13T17:51:15.276633+0000","last_fullsized":"2024-09-13T17:51:15.276633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:56:49.905393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277185+0000","last_change":"2024-09-13T17:50:34.589628+0000","last_active":"2024-09-13T17:51:15.277185+0000","last_peered":"2024-09-13T17:51:15.277185+0000","last_clean":"2024-09-13T17:51:15.277185+0000","last_became_active":"2024-09-13T17:50:34.587665+0000","last_became_peered":"2024-09-13T17:50:34.587665+0000","last_unstale":"2024-09-13T17:51:15.277185+0000","last_undegraded":"2024-09-13T17:51:15.277185+0000","last_fullsized":"2024-09-13T17:51:15.277185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:15.518264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279579+0000","last_change":"2024-09-13T17:50:34.591776+0000","last_active":"2024-09-13T17:51:15.279579+0000","last_peered":"2024-09-13T17:51:15.279579+0000","last_clean":"2024-09-13T17:51:15.279579+0000","last_became_active":"2024-09-13T17:50:34.591496+0000","last_became_peered":"2024-09-13T17:50:34.591496+0000","last_unstale":"2024-09-13T17:51:15.279579+0000","last_undegraded":"2024-09-13T17:51:15.279579+0000","last_fullsized":"2024-09-13T17:51:15.279579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:48.651999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T17:51:16.283047+0000","last_change":"2024-09-13T17:51:16.282372+0000","last_active":"2024-09-13T17:51:16.283047+0000","last_peered":"2024-09-13T17:51:16.283047+0000","last_clean":"2024-09-13T17:51:16.283047+0000","last_became_active":"2024-09-13T17:51:16.282065+0000","last_became_peered":"2024-09-13T17:51:16.282065+0000","last_unstale":"2024-09-13T17:51:16.283047+0000","last_undegraded":"2024-09-13T17:51:16.283047+0000","last_fullsized":"2024-09-13T17:51:16.283047+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:49.291224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278625+0000","last_change":"2024-09-13T17:50:34.585531+0000","last_active":"2024-09-13T17:51:15.278625+0000","last_peered":"2024-09-13T17:51:15.278625+0000","last_clean":"2024-09-13T17:51:15.278625+0000","last_became_active":"2024-09-13T17:50:34.585226+0000","last_became_peered":"2024-09-13T17:50:34.585226+0000","last_unstale":"2024-09-13T17:51:15.278625+0000","last_undegraded":"2024-09-13T17:51:15.278625+0000","last_fullsized":"2024-09-13T17:51:15.278625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:17.105028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.269683+0000","last_change":"2024-09-13T17:50:34.586750+0000","last_active":"2024-09-13T17:51:15.269683+0000","last_peered":"2024-09-13T17:51:15.269683+0000","last_clean":"2024-09-13T17:51:15.269683+0000","last_became_active":"2024-09-13T17:50:34.584564+0000","last_became_peered":"2024-09-13T17:50:34.584564+0000","last_unstale":"2024-09-13T17:51:15.269683+0000","last_undegraded":"2024-09-13T17:51:15.269683+0000","last_fullsized":"2024-09-13T17:51:15.269683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:54.033257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.278769+0000","last_change":"2024-09-13T17:50:34.585938+0000","last_active":"2024-09-13T17:51:15.278769+0000","last_peered":"2024-09-13T17:51:15.278769+0000","last_clean":"2024-09-13T17:51:15.278769+0000","last_became_active":"2024-09-13T17:50:34.585584+0000","last_became_peered":"2024-09-13T17:50:34.585584+0000","last_unstale":"2024-09-13T17:51:15.278769+0000","last_undegraded":"2024-09-13T17:51:15.278769+0000","last_fullsized":"2024-09-13T17:51:15.278769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:32.623468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.271015+0000","last_change":"2024-09-13T17:50:34.588762+0000","last_active":"2024-09-13T17:51:15.271015+0000","last_peered":"2024-09-13T17:51:15.271015+0000","last_clean":"2024-09-13T17:51:15.271015+0000","last_became_active":"2024-09-13T17:50:34.588489+0000","last_became_peered":"2024-09-13T17:50:34.588489+0000","last_unstale":"2024-09-13T17:51:15.271015+0000","last_undegraded":"2024-09-13T17:51:15.271015+0000","last_fullsized":"2024-09-13T17:51:15.271015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:38.266297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.277059+0000","last_change":"2024-09-13T17:50:34.583889+0000","last_active":"2024-09-13T17:51:15.277059+0000","last_peered":"2024-09-13T17:51:15.277059+0000","last_clean":"2024-09-13T17:51:15.277059+0000","last_became_active":"2024-09-13T17:50:34.583551+0000","last_became_peered":"2024-09-13T17:50:34.583551+0000","last_unstale":"2024-09-13T17:51:15.277059+0000","last_undegraded":"2024-09-13T17:51:15.277059+0000","last_fullsized":"2024-09-13T17:51:15.277059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:19.888626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.281343+0000","last_change":"2024-09-13T17:50:34.585617+0000","last_active":"2024-09-13T17:51:15.281343+0000","last_peered":"2024-09-13T17:51:15.281343+0000","last_clean":"2024-09-13T17:51:15.281343+0000","last_became_active":"2024-09-13T17:50:34.585340+0000","last_became_peered":"2024-09-13T17:50:34.585340+0000","last_unstale":"2024-09-13T17:51:15.281343+0000","last_undegraded":"2024-09-13T17:51:15.281343+0000","last_fullsized":"2024-09-13T17:51:15.281343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:18.331045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-13T17:51:15.279542+0000","last_change":"2024-09-13T17:50:34.592365+0000","last_active":"2024-09-13T17:51:15.279542+0000","last_peered":"2024-09-13T17:51:15.279542+0000","last_clean":"2024-09-13T17:51:15.279542+0000","last_became_active":"2024-09-13T17:50:34.592046+0000","last_became_peered":"2024-09-13T17:50:34.592046+0000","last_unstale":"2024-09-13T17:51:15.279542+0000","last_undegraded":"2024-09-13T17:51:15.279542+0000","last_fullsized":"2024-09-13T17:51:15.279542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:54.283683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7751,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":15502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":307200,"data_stored":120825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4870000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.401}]}]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":303104,"data_stored":116742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.335}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4079999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":307200,"data_stored":116868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":892928,"data_stored":706952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.5009999999999999,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.60199999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.528,"5min":2.528,"15min":2.528},"last":0.70299999999999996}]},{"osd":3,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.51600000000000001}]},{"osd":4,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.5790000000000002,"5min":2.5790000000000002,"15min":2.5790000000000002},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.79500000000000004}]},{"osd":5,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.3700000000000001,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.5499999999999998,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":0.74099999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":1.022},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":0.92000000000000004}]},{"osd":7,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.4660000000000002,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.1599999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542159,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":897024,"data_stored":708002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:51:25 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.92300000000000004}]},{"osd":1,"last update":"Fri Sep 13 17:51:25 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":1.1910000000000001},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":1.0029999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:51:25 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":1.03}]},{"osd":3,"last update":"Fri Sep 13 17:51:25 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.71899999999999997}]},{"osd":4,"last update":"Fri Sep 13 17:51:25 2024","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.016,"15min":1.016},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.706,"5min":1.706,"15min":1.706},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.056,"5min":1.056,"15min":1.056},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":0.873}]},{"osd":5,"last update":"Fri Sep 13 17:51:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.0820000000000001}]},{"osd":6,"last update":"Fri Sep 13 17:51:25 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.123}]}]},{"osd":6,"up_from":14,"seq":60129542159,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":303104,"data_stored":120349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":4,"last update":"Fri Sep 13 17:51:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.90700000000000003}]},{"osd":5,"last update":"Fri Sep 13 17:51:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.995,"5min":0.995,"15min":0.995},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.296}]},{"osd":7,"last update":"Fri Sep 13 17:51:20 2024","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.071},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.3999999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542159,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":303104,"data_stored":117302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:51:25 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.8780000000000001,"5min":2.8780000000000001,"15min":2.8780000000000001},"last":1.083}]},{"osd":1,"last update":"Fri Sep 13 17:51:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.5859999999999999,"5min":2.5859999999999999,"15min":2.5859999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":1.133}]},{"osd":2,"last update":"Fri Sep 13 17:51:25 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.96099999999999997}]},{"osd":3,"last update":"Fri Sep 13 17:51:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.6970000000000001,"5min":2.6970000000000001,"15min":2.6970000000000001},"last":1.27},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.84099999999999997}]},{"osd":4,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.9689999999999999,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":0.878},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.02}]},{"osd":6,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":3.0150000000000001,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":1.242}]},{"osd":7,"last update":"Fri Sep 13 17:51:21 2024","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":1.2},{"interface":"front","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":3.121,"5min":3.121,"15min":3.121},"last":1.3680000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542159,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":299008,"data_stored":116590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:51:22 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.73599999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.83999999999999997}]},{"osd":2,"last update":"Fri Sep 13 17:51:22 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.113},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.81000000000000005}]},{"osd":3,"last update":"Fri Sep 13 17:51:22 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.85099999999999998}]},{"osd":5,"last update":"Fri Sep 13 17:51:22 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.64800000000000002},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.2709999999999999}]},{"osd":6,"last update":"Fri Sep 13 17:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.196},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.91100000000000003}]},{"osd":7,"last update":"Fri Sep 13 17:51:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.93200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:51:27.447 INFO:tasks.mon_thrash.ceph_manager:clean! 2024-09-13T17:51:27.448 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:51:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.562+0000 7f9e88e62700 1 Processor -- start 2024-09-13T17:51:27.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.562+0000 7f9e88e62700 1 -- start start 2024-09-13T17:51:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.562+0000 7f9e88e62700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9e84153810 0x7f9e840ca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.562+0000 7f9e88e62700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e8406d400 con 0x7f9e840cafd0 2024-09-13T17:51:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.562+0000 7f9e88e62700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e8406ce40 con 0x7f9e840c3da0 2024-09-13T17:51:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.562+0000 7f9e88e62700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e8406d120 con 0x7f9e84153810 2024-09-13T17:51:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.562+0000 7f9e83fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9e840cafd0 0x7f9e8406a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44562/0 (socket says 172.21.15.40:44562) 2024-09-13T17:51:27.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.562+0000 7f9e837fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9e84153810 0x7f9e840ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:27.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.566+0000 7f9e83fff700 1 -- 172.21.15.40:0/1145615328 learned_addr learned my addr 172.21.15.40:0/1145615328 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:27.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.566+0000 7f9e837fe700 1 -- 172.21.15.40:0/1145615328 >> v1:172.21.15.43:6789/0 conn(0x7f9e840c3da0 legacy=0x7f9e84159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:27.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.566+0000 7f9e837fe700 1 -- 172.21.15.40:0/1145615328 >> v1:172.21.15.40:6789/0 conn(0x7f9e840cafd0 legacy=0x7f9e8406a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:27.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.566+0000 7f9e827fc700 1 -- 172.21.15.40:0/1145615328 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 717206814 0 0) 0x7f9e8406d400 con 0x7f9e840cafd0 2024-09-13T17:51:27.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.566+0000 7f9e837fe700 1 -- 172.21.15.40:0/1145615328 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e840c98b0 con 0x7f9e84153810 2024-09-13T17:51:27.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.566+0000 7f9e837fe700 1 --2- 172.21.15.40:0/1145615328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e84153810 0x7f9e840ca390 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009850 tx=0x7f9e74009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c70e2b29cc6b1e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:27.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.566+0000 7f9e827fc700 1 -- 172.21.15.40:0/1145615328 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e74011830 con 0x7f9e84153810 2024-09-13T17:51:27.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.566+0000 7f9e827fc700 1 -- 172.21.15.40:0/1145615328 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e74011990 con 0x7f9e84153810 2024-09-13T17:51:27.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.566+0000 7f9e827fc700 1 -- 172.21.15.40:0/1145615328 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e74011d20 con 0x7f9e84153810 2024-09-13T17:51:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.566+0000 7f9e88e62700 1 -- 172.21.15.40:0/1145615328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e84153810 msgr2=0x7f9e840ca390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.566+0000 7f9e88e62700 1 --2- 172.21.15.40:0/1145615328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e84153810 0x7f9e840ca390 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009850 tx=0x7f9e74009b60 comp rx=0 tx=0).stop 2024-09-13T17:51:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.566+0000 7f9e88e62700 1 -- 172.21.15.40:0/1145615328 shutdown_connections 2024-09-13T17:51:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.566+0000 7f9e88e62700 1 --2- 172.21.15.40:0/1145615328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e84153810 0x7f9e840ca390 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.566+0000 7f9e88e62700 1 -- 172.21.15.40:0/1145615328 >> 172.21.15.40:0/1145615328 conn(0x7f9e840cbca0 msgr2=0x7f9e840cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.566+0000 7f9e88e62700 1 -- 172.21.15.40:0/1145615328 shutdown_connections 2024-09-13T17:51:27.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.566+0000 7f9e88e62700 1 -- 172.21.15.40:0/1145615328 wait complete. 2024-09-13T17:51:27.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.570+0000 7f9e88e62700 1 Processor -- start 2024-09-13T17:51:27.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.570+0000 7f9e88e62700 1 -- start start 2024-09-13T17:51:27.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.570+0000 7f9e88e62700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e840c3da0 0x7f9e841f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:27.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.570+0000 7f9e88e62700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e840cafd0 0x7f9e841e4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:27.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.570+0000 7f9e88e62700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9e84153810 0x7f9e84156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:27.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.570+0000 7f9e88e62700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e841eebd0 con 0x7f9e840c3da0 2024-09-13T17:51:27.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.570+0000 7f9e88e62700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e840c9100 con 0x7f9e840cafd0 2024-09-13T17:51:27.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.570+0000 7f9e88e62700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e841f3090 con 0x7f9e84153810 2024-09-13T17:51:27.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.574+0000 7f9e837fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e840c3da0 0x7f9e841f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:27.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.574+0000 7f9e837fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e840c3da0 0x7f9e841f2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55152/0 (socket says 172.21.15.40:55152) 2024-09-13T17:51:27.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.574+0000 7f9e837fe700 1 -- 172.21.15.40:0/1103372868 learned_addr learned my addr 172.21.15.40:0/1103372868 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:27.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.574+0000 7f9e82ffd700 1 --2- 172.21.15.40:0/1103372868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e840cafd0 0x7f9e841e4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:27.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.574+0000 7f9e83fff700 1 --2- 172.21.15.40:0/1103372868 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9e84153810 0x7f9e84156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:27.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.574+0000 7f9e837fe700 1 -- 172.21.15.40:0/1103372868 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9e84153810 msgr2=0x7f9e84156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:27.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.574+0000 7f9e837fe700 1 --2- 172.21.15.40:0/1103372868 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9e84153810 0x7f9e84156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.574+0000 7f9e837fe700 1 -- 172.21.15.40:0/1103372868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e840cafd0 msgr2=0x7f9e841e4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:27.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.574+0000 7f9e837fe700 1 --2- 172.21.15.40:0/1103372868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e840cafd0 0x7f9e841e4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.574+0000 7f9e837fe700 1 -- 172.21.15.40:0/1103372868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e740094b0 con 0x7f9e840c3da0 2024-09-13T17:51:27.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.574+0000 7f9e83fff700 1 --2- 172.21.15.40:0/1103372868 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9e84153810 0x7f9e84156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:27.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.574+0000 7f9e837fe700 1 --2- 172.21.15.40:0/1103372868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e840c3da0 0x7f9e841f2b50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9e740090b0 tx=0x7f9e74004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:27.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.574+0000 7f9e7bfff700 1 -- 172.21.15.40:0/1103372868 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7400bb20 con 0x7f9e840c3da0 2024-09-13T17:51:27.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.574+0000 7f9e7bfff700 1 -- 172.21.15.40:0/1103372868 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7401e070 con 0x7f9e840c3da0 2024-09-13T17:51:27.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.574+0000 7f9e88e62700 1 -- 172.21.15.40:0/1103372868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e841ef3f0 con 0x7f9e840c3da0 2024-09-13T17:51:27.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.574+0000 7f9e7bfff700 1 -- 172.21.15.40:0/1103372868 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7400bec0 con 0x7f9e840c3da0 2024-09-13T17:51:27.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.574+0000 7f9e88e62700 1 -- 172.21.15.40:0/1103372868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e841e4c90 con 0x7f9e840c3da0 2024-09-13T17:51:27.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.574+0000 7f9e88e62700 1 -- 172.21.15.40:0/1103372868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e840cb400 con 0x7f9e840c3da0 2024-09-13T17:51:27.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.578+0000 7f9e7bfff700 1 -- 172.21.15.40:0/1103372868 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9e7402d440 con 0x7f9e840c3da0 2024-09-13T17:51:27.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.578+0000 7f9e7bfff700 1 --2- 172.21.15.40:0/1103372868 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e64066730 0x7f9e64068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:27.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.578+0000 7f9e7bfff700 1 -- 172.21.15.40:0/1103372868 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9e74085890 con 0x7f9e840c3da0 2024-09-13T17:51:27.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.578+0000 7f9e82ffd700 1 --2- 172.21.15.40:0/1103372868 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e64066730 0x7f9e64068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:27.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.578+0000 7f9e82ffd700 1 --2- 172.21.15.40:0/1103372868 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e64066730 0x7f9e64068bf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9e840cd0e0 tx=0x7f9e6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:27.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.586+0000 7f9e7bfff700 1 -- 172.21.15.40:0/1103372868 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9e740528d0 con 0x7f9e840c3da0 2024-09-13T17:51:27.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.758+0000 7f9e88e62700 1 -- 172.21.15.40:0/1103372868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e840c5bd0 con 0x7f9e840c3da0 2024-09-13T17:51:27.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.762+0000 7f9e7bfff700 1 -- 172.21.15.40:0/1103372868 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18650 (secure 0 0 0) 0x7f9e74055ef0 con 0x7f9e840c3da0 2024-09-13T17:51:27.767 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:27.767 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":27,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:51:16.266176+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:50:32.740336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:50:33.552914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:50:36.623639+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:51:27.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.762+0000 7f9e88e62700 1 -- 172.21.15.40:0/1103372868 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e64066730 msgr2=0x7f9e64068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:27.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.762+0000 7f9e88e62700 1 --2- 172.21.15.40:0/1103372868 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e64066730 0x7f9e64068bf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9e840cd0e0 tx=0x7f9e6c009040 comp rx=0 tx=0).stop 2024-09-13T17:51:27.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.762+0000 7f9e88e62700 1 -- 172.21.15.40:0/1103372868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e840c3da0 msgr2=0x7f9e841f2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:27.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.762+0000 7f9e88e62700 1 --2- 172.21.15.40:0/1103372868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e840c3da0 0x7f9e841f2b50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9e740090b0 tx=0x7f9e74004770 comp rx=0 tx=0).stop 2024-09-13T17:51:27.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.762+0000 7f9e88e62700 1 -- 172.21.15.40:0/1103372868 shutdown_connections 2024-09-13T17:51:27.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.762+0000 7f9e88e62700 1 --2- 172.21.15.40:0/1103372868 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e64066730 0x7f9e64068bf0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.762+0000 7f9e88e62700 1 --2- 172.21.15.40:0/1103372868 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9e84153810 0x7f9e84156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.762+0000 7f9e88e62700 1 --2- 172.21.15.40:0/1103372868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e840cafd0 0x7f9e841e4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.762+0000 7f9e88e62700 1 --2- 172.21.15.40:0/1103372868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e840c3da0 0x7f9e841f2b50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.762+0000 7f9e88e62700 1 -- 172.21.15.40:0/1103372868 >> 172.21.15.40:0/1103372868 conn(0x7f9e840cbca0 msgr2=0x7f9e840c5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:27.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.762+0000 7f9e88e62700 1 -- 172.21.15.40:0/1103372868 shutdown_connections 2024-09-13T17:51:27.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.762+0000 7f9e88e62700 1 -- 172.21.15.40:0/1103372868 wait complete. 2024-09-13T17:51:27.784 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:50:21.057171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:50:32.740336+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:50:33.552914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:50:36.623639+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T17:51:27.785 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:51:27.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.902+0000 7f8ed7fff700 1 Processor -- start 2024-09-13T17:51:27.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.902+0000 7f8ed7fff700 1 -- start start 2024-09-13T17:51:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.902+0000 7f8ed7fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8ed814fdd0 0x7f8ed80c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.902+0000 7f8ed7fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8ed80c0a10 0x7f8ed80c0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.902+0000 7f8ed7fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed8159eb0 con 0x7f8ed80c72e0 2024-09-13T17:51:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.902+0000 7f8ed7fff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed80c76e0 con 0x7f8ed80c0a10 2024-09-13T17:51:27.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.902+0000 7f8ed7fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed80c13e0 con 0x7f8ed814fdd0 2024-09-13T17:51:27.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed77fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8ed80c0a10 0x7f8ed80c0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:27.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed6ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8ed814fdd0 0x7f8ed80c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:27.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed67fc700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8ed80c72e0 0x7f8ed80c59d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:37130/0 (socket says 172.21.15.40:37130) 2024-09-13T17:51:27.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed67fc700 1 -- 172.21.15.40:0/149236543 learned_addr learned my addr 172.21.15.40:0/149236543 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:27.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed77fe700 1 -- 172.21.15.40:0/149236543 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8ed814fdd0 msgr2=0x7f8ed80c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:27.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed77fe700 1 --2- 172.21.15.40:0/149236543 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8ed814fdd0 0x7f8ed80c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed77fe700 1 -- 172.21.15.40:0/149236543 >> v1:172.21.15.43:6789/0 conn(0x7f8ed80c72e0 legacy=0x7f8ed80c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:27.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed77fe700 1 -- 172.21.15.40:0/149236543 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed8158fc0 con 0x7f8ed80c0a10 2024-09-13T17:51:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed5ffb700 1 -- 172.21.15.40:0/149236543 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3076134618 0 0) 0x7f8ed8159eb0 con 0x7f8ed80c72e0 2024-09-13T17:51:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed77fe700 1 --2- 172.21.15.40:0/149236543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed80c0a10 0x7f8ed80c0ea0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000bfa0 tx=0x7f8ed000d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=735efe16f57cf63c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed5ffb700 1 -- 172.21.15.40:0/149236543 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0010070 con 0x7f8ed80c0a10 2024-09-13T17:51:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed5ffb700 1 -- 172.21.15.40:0/149236543 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed000da00 con 0x7f8ed80c0a10 2024-09-13T17:51:27.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed5ffb700 1 -- 172.21.15.40:0/149236543 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed000ddb0 con 0x7f8ed80c0a10 2024-09-13T17:51:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed7fff700 1 -- 172.21.15.40:0/149236543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed80c0a10 msgr2=0x7f8ed80c0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed7fff700 1 --2- 172.21.15.40:0/149236543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed80c0a10 0x7f8ed80c0ea0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000bfa0 tx=0x7f8ed000d330 comp rx=0 tx=0).stop 2024-09-13T17:51:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed7fff700 1 -- 172.21.15.40:0/149236543 shutdown_connections 2024-09-13T17:51:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed7fff700 1 --2- 172.21.15.40:0/149236543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed80c0a10 0x7f8ed80c0ea0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed7fff700 1 --2- 172.21.15.40:0/149236543 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8ed814fdd0 0x7f8ed80c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed7fff700 1 -- 172.21.15.40:0/149236543 >> 172.21.15.40:0/149236543 conn(0x7f8ed8066090 msgr2=0x7f8ed80664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed7fff700 1 -- 172.21.15.40:0/149236543 shutdown_connections 2024-09-13T17:51:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.906+0000 7f8ed7fff700 1 -- 172.21.15.40:0/149236543 wait complete. 2024-09-13T17:51:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed7fff700 1 Processor -- start 2024-09-13T17:51:27.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed7fff700 1 -- start start 2024-09-13T17:51:27.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed80c0a10 0x7f8ed80afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:27.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed7fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8ed80c72e0 0x7f8ed80ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:27.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed814fdd0 0x7f8ed80aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:27.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed7fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed8151b50 con 0x7f8ed814fdd0 2024-09-13T17:51:27.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed7fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed81502a0 con 0x7f8ed80c0a10 2024-09-13T17:51:27.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed7fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed8152810 con 0x7f8ed80c72e0 2024-09-13T17:51:27.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed814fdd0 0x7f8ed80aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:27.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed6ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed80c0a10 0x7f8ed80afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:27.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed814fdd0 0x7f8ed80aee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55162/0 (socket says 172.21.15.40:55162) 2024-09-13T17:51:27.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed6ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed80c0a10 0x7f8ed80afd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:35722/0 (socket says 172.21.15.40:35722) 2024-09-13T17:51:27.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed67fc700 1 --2- 172.21.15.40:0/1338217321 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8ed80c72e0 0x7f8ed80ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed77fe700 1 -- 172.21.15.40:0/1338217321 learned_addr learned my addr 172.21.15.40:0/1338217321 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed77fe700 1 -- 172.21.15.40:0/1338217321 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8ed80c72e0 msgr2=0x7f8ed80ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed77fe700 1 --2- 172.21.15.40:0/1338217321 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8ed80c72e0 0x7f8ed80ae950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed77fe700 1 -- 172.21.15.40:0/1338217321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed80c0a10 msgr2=0x7f8ed80afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed77fe700 1 --2- 172.21.15.40:0/1338217321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed80c0a10 0x7f8ed80afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed77fe700 1 -- 172.21.15.40:0/1338217321 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed000bc80 con 0x7f8ed814fdd0 2024-09-13T17:51:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed77fe700 1 --2- 172.21.15.40:0/1338217321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed814fdd0 0x7f8ed80aee90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0004270 tx=0x7f8ed0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ecf7fe700 1 -- 172.21.15.40:0/1338217321 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0004490 con 0x7f8ed814fdd0 2024-09-13T17:51:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ecf7fe700 1 -- 172.21.15.40:0/1338217321 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed00045f0 con 0x7f8ed814fdd0 2024-09-13T17:51:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed7fff700 1 -- 172.21.15.40:0/1338217321 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed81504d0 con 0x7f8ed814fdd0 2024-09-13T17:51:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ecf7fe700 1 -- 172.21.15.40:0/1338217321 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed000a420 con 0x7f8ed814fdd0 2024-09-13T17:51:27.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.910+0000 7f8ed7fff700 1 -- 172.21.15.40:0/1338217321 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed81521c0 con 0x7f8ed814fdd0 2024-09-13T17:51:27.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.914+0000 7f8ed7fff700 1 -- 172.21.15.40:0/1338217321 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed814aed0 con 0x7f8ed814fdd0 2024-09-13T17:51:27.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.914+0000 7f8ecf7fe700 1 -- 172.21.15.40:0/1338217321 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8ed000a580 con 0x7f8ed814fdd0 2024-09-13T17:51:27.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.914+0000 7f8ecf7fe700 1 --2- 172.21.15.40:0/1338217321 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ebc066730 0x7f8ebc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:27.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.914+0000 7f8ecf7fe700 1 -- 172.21.15.40:0/1338217321 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8ed0086210 con 0x7f8ed814fdd0 2024-09-13T17:51:27.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.914+0000 7f8ed6ffd700 1 --2- 172.21.15.40:0/1338217321 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ebc066730 0x7f8ebc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:27.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.918+0000 7f8ed6ffd700 1 --2- 172.21.15.40:0/1338217321 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ebc066730 0x7f8ebc068bf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009b50 tx=0x7f8ec8009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:27.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:27.926+0000 7f8ecf7fe700 1 -- 172.21.15.40:0/1338217321 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8ed0053120 con 0x7f8ed814fdd0 2024-09-13T17:51:28.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.094+0000 7f8ed7fff700 1 -- 172.21.15.40:0/1338217321 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ed8070400 con 0x7f8ed814fdd0 2024-09-13T17:51:28.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.094+0000 7f8ecf7fe700 1 -- 172.21.15.40:0/1338217321 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8ed000b480 con 0x7f8ed814fdd0 2024-09-13T17:51:28.099 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-09-13T17:51:28.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.094+0000 7f8ed7fff700 1 -- 172.21.15.40:0/1338217321 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ebc066730 msgr2=0x7f8ebc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:28.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.094+0000 7f8ed7fff700 1 --2- 172.21.15.40:0/1338217321 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ebc066730 0x7f8ebc068bf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009b50 tx=0x7f8ec8009240 comp rx=0 tx=0).stop 2024-09-13T17:51:28.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.094+0000 7f8ed7fff700 1 -- 172.21.15.40:0/1338217321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed814fdd0 msgr2=0x7f8ed80aee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:28.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.094+0000 7f8ed7fff700 1 --2- 172.21.15.40:0/1338217321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed814fdd0 0x7f8ed80aee90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0004270 tx=0x7f8ed0004af0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.094+0000 7f8ed7fff700 1 -- 172.21.15.40:0/1338217321 shutdown_connections 2024-09-13T17:51:28.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.094+0000 7f8ed7fff700 1 --2- 172.21.15.40:0/1338217321 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ed814fdd0 0x7f8ed80aee90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.098+0000 7f8ed7fff700 1 --2- 172.21.15.40:0/1338217321 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ebc066730 0x7f8ebc068bf0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.098+0000 7f8ed7fff700 1 --2- 172.21.15.40:0/1338217321 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8ed80c72e0 0x7f8ed80ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.098+0000 7f8ed7fff700 1 --2- 172.21.15.40:0/1338217321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ed80c0a10 0x7f8ed80afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.098+0000 7f8ed7fff700 1 -- 172.21.15.40:0/1338217321 >> 172.21.15.40:0/1338217321 conn(0x7f8ed8066090 msgr2=0x7f8ed814a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:28.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.098+0000 7f8ed7fff700 1 -- 172.21.15.40:0/1338217321 shutdown_connections 2024-09-13T17:51:28.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.098+0000 7f8ed7fff700 1 -- 172.21.15.40:0/1338217321 wait complete. 2024-09-13T17:51:28.114 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:51:28.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.226+0000 7fa8a545e700 1 Processor -- start 2024-09-13T17:51:28.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa8a545e700 1 -- start start 2024-09-13T17:51:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa8a545e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa8a00cf1a0 0x7fa8a015a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa8a545e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa8a00cc250 0x7fa8a00c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa8a545e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a015a7a0 con 0x7fa8a00cf580 2024-09-13T17:51:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa8a545e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a00cbdc0 con 0x7fa8a00cc250 2024-09-13T17:51:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa8a545e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a0153f80 con 0x7fa8a00cf1a0 2024-09-13T17:51:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89f7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa8a00cf580 0x7fa8a00c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44576/0 (socket says 172.21.15.40:44576) 2024-09-13T17:51:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89f7fe700 1 -- 172.21.15.40:0/3882407538 learned_addr learned my addr 172.21.15.40:0/3882407538 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89ffff700 1 --2- 172.21.15.40:0/3882407538 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa8a00cf1a0 0x7fa8a015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa8a4c5d700 1 --2- 172.21.15.40:0/3882407538 >> v2:172.21.15.43:3300/0 conn(0x7fa8a00cc250 0x7fa8a00c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89effd700 1 -- 172.21.15.40:0/3882407538 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963813612 0 0) 0x7fa888001110 con 0x7fa8a00cf580 2024-09-13T17:51:28.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89effd700 1 -- 172.21.15.40:0/3882407538 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa88c0035d0 con 0x7fa8a00cf580 2024-09-13T17:51:28.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89effd700 1 -- 172.21.15.40:0/3882407538 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4151402959 0 0) 0x7fa88c0035d0 con 0x7fa8a00cf580 2024-09-13T17:51:28.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89effd700 1 -- 172.21.15.40:0/3882407538 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa8a00cf1a0 msgr2=0x7fa8a015a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:28.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89effd700 1 --2- 172.21.15.40:0/3882407538 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa8a00cf1a0 0x7fa8a015a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89effd700 1 -- 172.21.15.40:0/3882407538 >> v2:172.21.15.43:3300/0 conn(0x7fa8a00cc250 msgr2=0x7fa8a00c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:28.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89effd700 1 --2- 172.21.15.40:0/3882407538 >> v2:172.21.15.43:3300/0 conn(0x7fa8a00cc250 0x7fa8a00c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89effd700 1 -- 172.21.15.40:0/3882407538 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a00caed0 con 0x7fa8a00cf580 2024-09-13T17:51:28.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89effd700 1 -- 172.21.15.40:0/3882407538 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fa888001800 con 0x7fa8a00cf580 2024-09-13T17:51:28.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89effd700 1 -- 172.21.15.40:0/3882407538 >> v1:172.21.15.40:6789/0 conn(0x7fa8a00cf580 legacy=0x7fa8a00c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:28.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89effd700 1 --2- 172.21.15.40:0/3882407538 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa8a00cf1a0 0x7fa88c003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89effd700 1 --2- 172.21.15.40:0/3882407538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c003fa0 0x7fa88c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89effd700 1 --2- 172.21.15.40:0/3882407538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa88c004db0 0x7fa88c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89effd700 1 -- 172.21.15.40:0/3882407538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00cbdc0 con 0x7fa88c003fa0 2024-09-13T17:51:28.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89effd700 1 -- 172.21.15.40:0/3882407538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a0153f80 con 0x7fa88c004db0 2024-09-13T17:51:28.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89effd700 1 -- 172.21.15.40:0/3882407538 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c00ba60 con 0x7fa8a00cf1a0 2024-09-13T17:51:28.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89f7fe700 1 --2- 172.21.15.40:0/3882407538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c003fa0 0x7fa88c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa8a4c5d700 1 --2- 172.21.15.40:0/3882407538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa88c004db0 0x7fa88c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89ffff700 1 --2- 172.21.15.40:0/3882407538 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa8a00cf1a0 0x7fa88c003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89f7fe700 1 -- 172.21.15.40:0/3882407538 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa8a00cf1a0 msgr2=0x7fa88c003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89f7fe700 1 --2- 172.21.15.40:0/3882407538 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa8a00cf1a0 0x7fa88c003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89f7fe700 1 -- 172.21.15.40:0/3882407538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa88c004db0 msgr2=0x7fa88c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89f7fe700 1 --2- 172.21.15.40:0/3882407538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa88c004db0 0x7fa88c005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89f7fe700 1 -- 172.21.15.40:0/3882407538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88c0035d0 con 0x7fa88c003fa0 2024-09-13T17:51:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89f7fe700 1 --2- 172.21.15.40:0/3882407538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c003fa0 0x7fa88c004870 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa8a008d080 tx=0x7fa88c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da013753a3b82fa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89effd700 1 -- 172.21.15.40:0/3882407538 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88800ce00 con 0x7fa88c003fa0 2024-09-13T17:51:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89effd700 1 -- 172.21.15.40:0/3882407538 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa888005980 con 0x7fa88c003fa0 2024-09-13T17:51:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.230+0000 7fa89effd700 1 -- 172.21.15.40:0/3882407538 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888005d30 con 0x7fa88c003fa0 2024-09-13T17:51:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.234+0000 7fa8a545e700 1 -- 172.21.15.40:0/3882407538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c003fa0 msgr2=0x7fa88c004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.234+0000 7fa8a545e700 1 --2- 172.21.15.40:0/3882407538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c003fa0 0x7fa88c004870 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa8a008d080 tx=0x7fa88c0058a0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.234+0000 7fa8a4c5d700 1 -- 172.21.15.40:0/3882407538 reap_dead start 2024-09-13T17:51:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.234+0000 7fa8a545e700 1 -- 172.21.15.40:0/3882407538 shutdown_connections 2024-09-13T17:51:28.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.234+0000 7fa8a545e700 1 -- 172.21.15.40:0/3882407538 >> 172.21.15.40:0/3882407538 conn(0x7fa8a0066090 msgr2=0x7fa8a0066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:28.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.234+0000 7fa8a545e700 1 -- 172.21.15.40:0/3882407538 shutdown_connections 2024-09-13T17:51:28.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.234+0000 7fa8a545e700 1 -- 172.21.15.40:0/3882407538 wait complete. 2024-09-13T17:51:28.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.234+0000 7fa8a545e700 1 Processor -- start 2024-09-13T17:51:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.234+0000 7fa8a545e700 1 -- start start 2024-09-13T17:51:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.234+0000 7fa8a545e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c003fa0 0x7fa8a01e3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.234+0000 7fa8a545e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8a01e6940 0x7fa8a01e5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.234+0000 7fa8a545e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa8a00cdc40 0x7fa8a01e60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.234+0000 7fa8a545e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00c5450 con 0x7fa88c003fa0 2024-09-13T17:51:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.234+0000 7fa8a545e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a01e6610 con 0x7fa8a01e6940 2024-09-13T17:51:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.234+0000 7fa8a545e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a00c5980 con 0x7fa8a00cdc40 2024-09-13T17:51:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.234+0000 7fa89ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c003fa0 0x7fa8a01e3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.234+0000 7fa89ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c003fa0 0x7fa8a01e3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55182/0 (socket says 172.21.15.40:55182) 2024-09-13T17:51:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.234+0000 7fa89ffff700 1 -- 172.21.15.40:0/2810703644 learned_addr learned my addr 172.21.15.40:0/2810703644 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.238+0000 7fa89f7fe700 1 --2- 172.21.15.40:0/2810703644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8a01e6940 0x7fa8a01e5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.238+0000 7fa8a4c5d700 1 --2- 172.21.15.40:0/2810703644 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa8a00cdc40 0x7fa8a01e60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.238+0000 7fa89ffff700 1 -- 172.21.15.40:0/2810703644 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa8a00cdc40 msgr2=0x7fa8a01e60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:28.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.238+0000 7fa89ffff700 1 --2- 172.21.15.40:0/2810703644 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa8a00cdc40 0x7fa8a01e60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.238+0000 7fa89ffff700 1 -- 172.21.15.40:0/2810703644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8a01e6940 msgr2=0x7fa8a01e5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:28.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.238+0000 7fa89ffff700 1 --2- 172.21.15.40:0/2810703644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8a01e6940 0x7fa8a01e5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.238+0000 7fa89ffff700 1 -- 172.21.15.40:0/2810703644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a01547d0 con 0x7fa88c003fa0 2024-09-13T17:51:28.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.238+0000 7fa8a4c5d700 1 --2- 172.21.15.40:0/2810703644 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa8a00cdc40 0x7fa8a01e60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:51:28.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.238+0000 7fa89f7fe700 1 --2- 172.21.15.40:0/2810703644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8a01e6940 0x7fa8a01e5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:51:28.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.238+0000 7fa89ffff700 1 --2- 172.21.15.40:0/2810703644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c003fa0 0x7fa8a01e3c20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa898011e30 tx=0x7fa89800f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:28.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.238+0000 7fa89cff9700 1 -- 172.21.15.40:0/2810703644 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89800e040 con 0x7fa88c003fa0 2024-09-13T17:51:28.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.238+0000 7fa8a545e700 1 -- 172.21.15.40:0/2810703644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a00c56a0 con 0x7fa88c003fa0 2024-09-13T17:51:28.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.238+0000 7fa89cff9700 1 -- 172.21.15.40:0/2810703644 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89800f040 con 0x7fa88c003fa0 2024-09-13T17:51:28.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.238+0000 7fa89cff9700 1 -- 172.21.15.40:0/2810703644 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa898009c00 con 0x7fa88c003fa0 2024-09-13T17:51:28.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.238+0000 7fa8a545e700 1 -- 172.21.15.40:0/2810703644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a0154480 con 0x7fa88c003fa0 2024-09-13T17:51:28.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.238+0000 7fa89cff9700 1 -- 172.21.15.40:0/2810703644 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa898015630 con 0x7fa88c003fa0 2024-09-13T17:51:28.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.238+0000 7fa8a545e700 1 -- 172.21.15.40:0/2810703644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a0067a40 con 0x7fa88c003fa0 2024-09-13T17:51:28.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.238+0000 7fa89cff9700 1 --2- 172.21.15.40:0/2810703644 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa880066730 0x7fa880068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.238+0000 7fa89cff9700 1 -- 172.21.15.40:0/2810703644 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa8980851f0 con 0x7fa88c003fa0 2024-09-13T17:51:28.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.242+0000 7fa89f7fe700 1 --2- 172.21.15.40:0/2810703644 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa880066730 0x7fa880068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.242+0000 7fa89f7fe700 1 --2- 172.21.15.40:0/2810703644 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa880066730 0x7fa880068bf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa8a01e6d40 tx=0x7fa88800a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:28.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.246+0000 7fa89cff9700 1 -- 172.21.15.40:0/2810703644 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa898052120 con 0x7fa88c003fa0 2024-09-13T17:51:28.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.418+0000 7fa8a545e700 1 -- 172.21.15.40:0/2810703644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa8a00c66c0 con 0x7fa88c003fa0 2024-09-13T17:51:28.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.418+0000 7fa89cff9700 1 -- 172.21.15.40:0/2810703644 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa898055740 con 0x7fa88c003fa0 2024-09-13T17:51:28.424 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-09-13T17:51:28.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.422+0000 7fa8a545e700 1 -- 172.21.15.40:0/2810703644 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa880066730 msgr2=0x7fa880068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:28.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.422+0000 7fa8a545e700 1 --2- 172.21.15.40:0/2810703644 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa880066730 0x7fa880068bf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa8a01e6d40 tx=0x7fa88800a000 comp rx=0 tx=0).stop 2024-09-13T17:51:28.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.422+0000 7fa8a545e700 1 -- 172.21.15.40:0/2810703644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c003fa0 msgr2=0x7fa8a01e3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:28.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.422+0000 7fa8a545e700 1 --2- 172.21.15.40:0/2810703644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c003fa0 0x7fa8a01e3c20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa898011e30 tx=0x7fa89800f490 comp rx=0 tx=0).stop 2024-09-13T17:51:28.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.422+0000 7fa8a545e700 1 -- 172.21.15.40:0/2810703644 shutdown_connections 2024-09-13T17:51:28.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.422+0000 7fa8a545e700 1 --2- 172.21.15.40:0/2810703644 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa880066730 0x7fa880068bf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.422+0000 7fa8a545e700 1 --2- 172.21.15.40:0/2810703644 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa8a00cdc40 0x7fa8a01e60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.422+0000 7fa8a545e700 1 --2- 172.21.15.40:0/2810703644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8a01e6940 0x7fa8a01e5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.422+0000 7fa8a545e700 1 --2- 172.21.15.40:0/2810703644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa88c003fa0 0x7fa8a01e3c20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.422+0000 7fa8a545e700 1 -- 172.21.15.40:0/2810703644 >> 172.21.15.40:0/2810703644 conn(0x7fa8a0066090 msgr2=0x7fa8a00b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:28.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.422+0000 7fa8a545e700 1 -- 172.21.15.40:0/2810703644 shutdown_connections 2024-09-13T17:51:28.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.422+0000 7fa8a545e700 1 -- 172.21.15.40:0/2810703644 wait complete. 2024-09-13T17:51:28.439 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:51:28.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.550+0000 7fc1dbfff700 1 Processor -- start 2024-09-13T17:51:28.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.550+0000 7fc1dbfff700 1 -- start start 2024-09-13T17:51:28.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.550+0000 7fc1dbfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc1dc159540 0x7fc1dc0c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.550+0000 7fc1dbfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc1dc0ca3c0 0x7fc1dc0c5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.550+0000 7fc1dbfff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1dc15a490 con 0x7fc1dc153a10 2024-09-13T17:51:28.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.550+0000 7fc1dbfff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1dc159940 con 0x7fc1dc159540 2024-09-13T17:51:28.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.550+0000 7fc1dbfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1dc159bf0 con 0x7fc1dc0ca3c0 2024-09-13T17:51:28.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1da7fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc1dc159540 0x7fc1dc0c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1da7fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc1dc159540 0x7fc1dc0c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55192/0 (socket says 172.21.15.40:55192) 2024-09-13T17:51:28.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1daffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc1dc153a10 0x7fc1dc0c4030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44582/0 (socket says 172.21.15.40:44582) 2024-09-13T17:51:28.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1da7fc700 1 -- 172.21.15.40:0/3297599749 learned_addr learned my addr 172.21.15.40:0/3297599749 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:28.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1db7fe700 1 --2- 172.21.15.40:0/3297599749 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc1dc0ca3c0 0x7fc1dc0c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1da7fc700 1 -- 172.21.15.40:0/3297599749 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc1dc0ca3c0 msgr2=0x7fc1dc0c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:28.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1da7fc700 1 --2- 172.21.15.40:0/3297599749 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc1dc0ca3c0 0x7fc1dc0c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1da7fc700 1 -- 172.21.15.40:0/3297599749 >> v1:172.21.15.40:6789/0 conn(0x7fc1dc153a10 legacy=0x7fc1dc0c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1da7fc700 1 -- 172.21.15.40:0/3297599749 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1dc159ed0 con 0x7fc1dc159540 2024-09-13T17:51:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1db7fe700 1 --2- 172.21.15.40:0/3297599749 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc1dc0ca3c0 0x7fc1dc0c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1da7fc700 1 --2- 172.21.15.40:0/3297599749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1dc159540 0x7fc1dc0c9e80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009910 tx=0x7fc1cc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e87760be4a33c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1d9ffb700 1 -- 172.21.15.40:0/3297599749 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc00ba60 con 0x7fc1dc159540 2024-09-13T17:51:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1d9ffb700 1 -- 172.21.15.40:0/3297599749 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1cc00bbc0 con 0x7fc1dc159540 2024-09-13T17:51:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1d9ffb700 1 -- 172.21.15.40:0/3297599749 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc009e60 con 0x7fc1dc159540 2024-09-13T17:51:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1dbfff700 1 -- 172.21.15.40:0/3297599749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1dc159540 msgr2=0x7fc1dc0c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1dbfff700 1 --2- 172.21.15.40:0/3297599749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1dc159540 0x7fc1dc0c9e80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009910 tx=0x7fc1cc009c20 comp rx=0 tx=0).stop 2024-09-13T17:51:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1dbfff700 1 -- 172.21.15.40:0/3297599749 shutdown_connections 2024-09-13T17:51:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1dbfff700 1 --2- 172.21.15.40:0/3297599749 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc1dc0ca3c0 0x7fc1dc0c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1dbfff700 1 --2- 172.21.15.40:0/3297599749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1dc159540 0x7fc1dc0c9e80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1dbfff700 1 -- 172.21.15.40:0/3297599749 >> 172.21.15.40:0/3297599749 conn(0x7fc1dc066270 msgr2=0x7fc1dc066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:28.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1dbfff700 1 -- 172.21.15.40:0/3297599749 shutdown_connections 2024-09-13T17:51:28.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1dbfff700 1 -- 172.21.15.40:0/3297599749 wait complete. 2024-09-13T17:51:28.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.554+0000 7fc1dbfff700 1 Processor -- start 2024-09-13T17:51:28.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1dbfff700 1 -- start start 2024-09-13T17:51:28.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1dbfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1dc0ca3c0 0x7fc1dc1491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1dbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1dc153a10 0x7fc1dc1ef460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1dbfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc1dc159540 0x7fc1dc1458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1dbfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1dc1f3300 con 0x7fc1dc153a10 2024-09-13T17:51:28.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1dbfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1dc157f10 con 0x7fc1dc0ca3c0 2024-09-13T17:51:28.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1dbfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1dc1e2b20 con 0x7fc1dc159540 2024-09-13T17:51:28.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1da7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1dc153a10 0x7fc1dc1ef460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1da7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1dc153a10 0x7fc1dc1ef460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55204/0 (socket says 172.21.15.40:55204) 2024-09-13T17:51:28.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1da7fc700 1 -- 172.21.15.40:0/1077493816 learned_addr learned my addr 172.21.15.40:0/1077493816 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:28.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1daffd700 1 --2- 172.21.15.40:0/1077493816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1dc0ca3c0 0x7fc1dc1491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1db7fe700 1 --2- 172.21.15.40:0/1077493816 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc1dc159540 0x7fc1dc1458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1da7fc700 1 -- 172.21.15.40:0/1077493816 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc1dc159540 msgr2=0x7fc1dc1458b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:28.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1da7fc700 1 --2- 172.21.15.40:0/1077493816 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc1dc159540 0x7fc1dc1458b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1da7fc700 1 -- 172.21.15.40:0/1077493816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1dc0ca3c0 msgr2=0x7fc1dc1491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:28.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1da7fc700 1 --2- 172.21.15.40:0/1077493816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1dc0ca3c0 0x7fc1dc1491d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1da7fc700 1 -- 172.21.15.40:0/1077493816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1cc009570 con 0x7fc1dc153a10 2024-09-13T17:51:28.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1da7fc700 1 --2- 172.21.15.40:0/1077493816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1dc153a10 0x7fc1dc1ef460 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc000c00 tx=0x7fc1cc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:28.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1c37fe700 1 -- 172.21.15.40:0/1077493816 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc004500 con 0x7fc1dc153a10 2024-09-13T17:51:28.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1c37fe700 1 -- 172.21.15.40:0/1077493816 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1cc0036c0 con 0x7fc1dc153a10 2024-09-13T17:51:28.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1dbfff700 1 -- 172.21.15.40:0/1077493816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1dc1e2de0 con 0x7fc1dc153a10 2024-09-13T17:51:28.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1c37fe700 1 -- 172.21.15.40:0/1077493816 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc003a50 con 0x7fc1dc153a10 2024-09-13T17:51:28.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.558+0000 7fc1dbfff700 1 -- 172.21.15.40:0/1077493816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1dc1f2cb0 con 0x7fc1dc153a10 2024-09-13T17:51:28.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.562+0000 7fc1dbfff700 1 -- 172.21.15.40:0/1077493816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1dc1f34b0 con 0x7fc1dc153a10 2024-09-13T17:51:28.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.562+0000 7fc1c37fe700 1 -- 172.21.15.40:0/1077493816 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc1cc003bb0 con 0x7fc1dc153a10 2024-09-13T17:51:28.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.562+0000 7fc1c37fe700 1 --2- 172.21.15.40:0/1077493816 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc1c8066760 0x7fc1c8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.562+0000 7fc1daffd700 1 --2- 172.21.15.40:0/1077493816 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc1c8066760 0x7fc1c8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.562+0000 7fc1c37fe700 1 -- 172.21.15.40:0/1077493816 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc1cc021030 con 0x7fc1dc153a10 2024-09-13T17:51:28.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.562+0000 7fc1daffd700 1 --2- 172.21.15.40:0/1077493816 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc1c8066760 0x7fc1c8068c20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0000f60 tx=0x7fc1d0007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:28.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.570+0000 7fc1c37fe700 1 -- 172.21.15.40:0/1077493816 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc1dc1f34b0 con 0x7fc1dc153a10 2024-09-13T17:51:28.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.738+0000 7fc1dbfff700 1 -- 172.21.15.40:0/1077493816 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc1dc0c9e80 con 0x7fc1dc153a10 2024-09-13T17:51:28.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.738+0000 7fc1c37fe700 1 -- 172.21.15.40:0/1077493816 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc1cc02b020 con 0x7fc1dc153a10 2024-09-13T17:51:28.744 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-09-13T17:51:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.742+0000 7fc1dbfff700 1 -- 172.21.15.40:0/1077493816 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc1c8066760 msgr2=0x7fc1c8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.742+0000 7fc1dbfff700 1 --2- 172.21.15.40:0/1077493816 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc1c8066760 0x7fc1c8068c20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0000f60 tx=0x7fc1d0007d30 comp rx=0 tx=0).stop 2024-09-13T17:51:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.742+0000 7fc1dbfff700 1 -- 172.21.15.40:0/1077493816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1dc153a10 msgr2=0x7fc1dc1ef460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:28.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.742+0000 7fc1dbfff700 1 --2- 172.21.15.40:0/1077493816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1dc153a10 0x7fc1dc1ef460 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc000c00 tx=0x7fc1cc004770 comp rx=0 tx=0).stop 2024-09-13T17:51:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.742+0000 7fc1dbfff700 1 -- 172.21.15.40:0/1077493816 shutdown_connections 2024-09-13T17:51:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.742+0000 7fc1dbfff700 1 --2- 172.21.15.40:0/1077493816 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc1c8066760 0x7fc1c8068c20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.742+0000 7fc1dbfff700 1 --2- 172.21.15.40:0/1077493816 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc1dc159540 0x7fc1dc1458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.742+0000 7fc1dbfff700 1 --2- 172.21.15.40:0/1077493816 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc1dc153a10 0x7fc1dc1ef460 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.742+0000 7fc1dbfff700 1 --2- 172.21.15.40:0/1077493816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1dc0ca3c0 0x7fc1dc1491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.742+0000 7fc1dbfff700 1 -- 172.21.15.40:0/1077493816 >> 172.21.15.40:0/1077493816 conn(0x7fc1dc066270 msgr2=0x7fc1dc0c5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:28.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.742+0000 7fc1dbfff700 1 -- 172.21.15.40:0/1077493816 shutdown_connections 2024-09-13T17:51:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.742+0000 7fc1dbfff700 1 -- 172.21.15.40:0/1077493816 wait complete. 2024-09-13T17:51:28.759 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T17:51:28.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.878+0000 7f37974b4700 1 Processor -- start 2024-09-13T17:51:28.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.878+0000 7f37974b4700 1 -- start start 2024-09-13T17:51:28.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.878+0000 7f37974b4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f37900cf580 0x7f379015a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.878+0000 7f37974b4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37900cc250 0x7f37900c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.878+0000 7f37974b4700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f379015a7a0 con 0x7f37900cf1a0 2024-09-13T17:51:28.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.878+0000 7f37974b4700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37900cbdc0 con 0x7f37900cf580 2024-09-13T17:51:28.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.878+0000 7f37974b4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3790153f80 con 0x7f37900cc250 2024-09-13T17:51:28.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f3795cb1700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f37900cf580 0x7f379015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f3796cb3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37900cc250 0x7f37900c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f37964b2700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f37900cf1a0 0x7f37900c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:37132/0 (socket says 172.21.15.40:37132) 2024-09-13T17:51:28.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f3795cb1700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f37900cf580 0x7f379015a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:35764/0 (socket says 172.21.15.40:35764) 2024-09-13T17:51:28.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f3795cb1700 1 -- 172.21.15.40:0/2759097007 learned_addr learned my addr 172.21.15.40:0/2759097007 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:28.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f3795cb1700 1 -- 172.21.15.40:0/2759097007 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37900cc250 msgr2=0x7f37900c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:28.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f3795cb1700 1 --2- 172.21.15.40:0/2759097007 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37900cc250 0x7f37900c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f3795cb1700 1 -- 172.21.15.40:0/2759097007 >> v1:172.21.15.43:6789/0 conn(0x7f37900cf1a0 legacy=0x7f37900c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:28.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f3795cb1700 1 -- 172.21.15.40:0/2759097007 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37900caed0 con 0x7f37900cf580 2024-09-13T17:51:28.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f37954b0700 1 -- 172.21.15.40:0/2759097007 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588266918 0 0) 0x7f3780001110 con 0x7f37900cf1a0 2024-09-13T17:51:28.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f3796cb3700 1 --2- 172.21.15.40:0/2759097007 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37900cc250 0x7f37900c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:28.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f3795cb1700 1 --2- 172.21.15.40:0/2759097007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37900cf580 0x7f379015a260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f378c009910 tx=0x7f378c009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e1bd982745c0e5a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:28.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f37954b0700 1 -- 172.21.15.40:0/2759097007 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c011830 con 0x7f37900cf580 2024-09-13T17:51:28.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f37954b0700 1 -- 172.21.15.40:0/2759097007 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378c011990 con 0x7f37900cf580 2024-09-13T17:51:28.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f37954b0700 1 -- 172.21.15.40:0/2759097007 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c011d40 con 0x7f37900cf580 2024-09-13T17:51:28.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f37974b4700 1 -- 172.21.15.40:0/2759097007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37900cf580 msgr2=0x7f379015a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:28.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f37974b4700 1 --2- 172.21.15.40:0/2759097007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37900cf580 0x7f379015a260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f378c009910 tx=0x7f378c009c20 comp rx=0 tx=0).stop 2024-09-13T17:51:28.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f37974b4700 1 -- 172.21.15.40:0/2759097007 shutdown_connections 2024-09-13T17:51:28.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f37974b4700 1 --2- 172.21.15.40:0/2759097007 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37900cc250 0x7f37900c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f37974b4700 1 --2- 172.21.15.40:0/2759097007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37900cf580 0x7f379015a260 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f37974b4700 1 -- 172.21.15.40:0/2759097007 >> 172.21.15.40:0/2759097007 conn(0x7f3790066090 msgr2=0x7f3790066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:28.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f37974b4700 1 -- 172.21.15.40:0/2759097007 shutdown_connections 2024-09-13T17:51:28.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.882+0000 7f37974b4700 1 -- 172.21.15.40:0/2759097007 wait complete. 2024-09-13T17:51:28.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f37974b4700 1 Processor -- start 2024-09-13T17:51:28.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f37974b4700 1 -- start start 2024-09-13T17:51:28.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f37974b4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37900cc250 0x7f379015abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f37974b4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37900cf1a0 0x7f37900c3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f37974b4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37900cf580 0x7f379014e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f37974b4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37901571d0 con 0x7f37900cf1a0 2024-09-13T17:51:28.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f37974b4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37900c5580 con 0x7f37900cf580 2024-09-13T17:51:28.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f37974b4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37900c2f20 con 0x7f37900cc250 2024-09-13T17:51:28.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f3795cb1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37900cf1a0 0x7f37900c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f3795cb1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37900cf1a0 0x7f37900c3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55210/0 (socket says 172.21.15.40:55210) 2024-09-13T17:51:28.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f3795cb1700 1 -- 172.21.15.40:0/3452095032 learned_addr learned my addr 172.21.15.40:0/3452095032 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:28.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f37964b2700 1 --2- 172.21.15.40:0/3452095032 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37900cc250 0x7f379015abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f3796cb3700 1 --2- 172.21.15.40:0/3452095032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37900cf580 0x7f379014e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f3795cb1700 1 -- 172.21.15.40:0/3452095032 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37900cc250 msgr2=0x7f379015abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:28.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f3795cb1700 1 --2- 172.21.15.40:0/3452095032 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37900cc250 0x7f379015abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f3795cb1700 1 -- 172.21.15.40:0/3452095032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37900cf580 msgr2=0x7f379014e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:28.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f3795cb1700 1 --2- 172.21.15.40:0/3452095032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37900cf580 0x7f379014e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:28.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f3795cb1700 1 -- 172.21.15.40:0/3452095032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3788009a70 con 0x7f37900cf1a0 2024-09-13T17:51:28.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f37964b2700 1 --2- 172.21.15.40:0/3452095032 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37900cc250 0x7f379015abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:28.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f3796cb3700 1 --2- 172.21.15.40:0/3452095032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37900cf580 0x7f379014e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:51:28.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.886+0000 7f3795cb1700 1 --2- 172.21.15.40:0/3452095032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37900cf1a0 0x7f37900c3160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f378c014040 tx=0x7f378c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:28.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.890+0000 7f3786ffd700 1 -- 172.21.15.40:0/3452095032 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c00ba70 con 0x7f37900cf1a0 2024-09-13T17:51:28.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.890+0000 7f3786ffd700 1 -- 172.21.15.40:0/3452095032 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378c01e070 con 0x7f37900cf1a0 2024-09-13T17:51:28.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.890+0000 7f37974b4700 1 -- 172.21.15.40:0/3452095032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f378c009570 con 0x7f37900cf1a0 2024-09-13T17:51:28.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.890+0000 7f3786ffd700 1 -- 172.21.15.40:0/3452095032 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c00be50 con 0x7f37900cf1a0 2024-09-13T17:51:28.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.890+0000 7f37974b4700 1 -- 172.21.15.40:0/3452095032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3790156c10 con 0x7f37900cf1a0 2024-09-13T17:51:28.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.890+0000 7f37974b4700 1 -- 172.21.15.40:0/3452095032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3790067a40 con 0x7f37900cf1a0 2024-09-13T17:51:28.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.890+0000 7f3786ffd700 1 -- 172.21.15.40:0/3452095032 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f378c02d450 con 0x7f37900cf1a0 2024-09-13T17:51:28.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.890+0000 7f3786ffd700 1 --2- 172.21.15.40:0/3452095032 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3774066760 0x7f3774068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:28.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.890+0000 7f3786ffd700 1 -- 172.21.15.40:0/3452095032 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f378c085630 con 0x7f37900cf1a0 2024-09-13T17:51:28.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.890+0000 7f37964b2700 1 --2- 172.21.15.40:0/3452095032 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3774066760 0x7f3774068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:28.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.894+0000 7f37964b2700 1 --2- 172.21.15.40:0/3452095032 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3774066760 0x7f3774068c20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3780000f60 tx=0x7f3780007b00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:28.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:28.898+0000 7f3786ffd700 1 -- 172.21.15.40:0/3452095032 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f378c0524d0 con 0x7f37900cf1a0 2024-09-13T17:51:29.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.074+0000 7f37974b4700 1 -- 172.21.15.40:0/3452095032 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f37900cc6e0 con 0x7f37900cf1a0 2024-09-13T17:51:29.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.078+0000 7f3786ffd700 1 -- 172.21.15.40:0/3452095032 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f378c055af0 con 0x7f37900cf1a0 2024-09-13T17:51:29.081 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-09-13T17:51:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.078+0000 7f37974b4700 1 -- 172.21.15.40:0/3452095032 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3774066760 msgr2=0x7f3774068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.078+0000 7f37974b4700 1 --2- 172.21.15.40:0/3452095032 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3774066760 0x7f3774068c20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3780000f60 tx=0x7f3780007b00 comp rx=0 tx=0).stop 2024-09-13T17:51:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.082+0000 7f37974b4700 1 -- 172.21.15.40:0/3452095032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37900cf1a0 msgr2=0x7f37900c3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.082+0000 7f37974b4700 1 --2- 172.21.15.40:0/3452095032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37900cf1a0 0x7f37900c3160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f378c014040 tx=0x7f378c004770 comp rx=0 tx=0).stop 2024-09-13T17:51:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.082+0000 7f37974b4700 1 -- 172.21.15.40:0/3452095032 shutdown_connections 2024-09-13T17:51:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.082+0000 7f37974b4700 1 --2- 172.21.15.40:0/3452095032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37900cf580 0x7f379014e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.082+0000 7f37974b4700 1 --2- 172.21.15.40:0/3452095032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37900cf1a0 0x7f37900c3160 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.082+0000 7f37974b4700 1 --2- 172.21.15.40:0/3452095032 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3774066760 0x7f3774068c20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.082+0000 7f37974b4700 1 --2- 172.21.15.40:0/3452095032 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37900cc250 0x7f379015abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.082+0000 7f37974b4700 1 -- 172.21.15.40:0/3452095032 >> 172.21.15.40:0/3452095032 conn(0x7f3790066090 msgr2=0x7f3790066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.082+0000 7f37974b4700 1 -- 172.21.15.40:0/3452095032 shutdown_connections 2024-09-13T17:51:29.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.082+0000 7f37974b4700 1 -- 172.21.15.40:0/3452095032 wait complete. 2024-09-13T17:51:29.101 DEBUG:tasks.mon_thrash:Yielding 2024-09-13T17:51:29.101 INFO:teuthology.run_tasks:Running task workunit... 2024-09-13T17:51:29.115 INFO:tasks.workunit:Pulling workunits from ref 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:51:29.115 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-13T17:51:29.115 INFO:tasks.workunit:timeout=3h 2024-09-13T17:51:29.115 INFO:tasks.workunit:cleanup=True 2024-09-13T17:51:29.116 DEBUG:teuthology.orchestra.run.smithi040:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-13T17:51:29.116 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:51:29.123 INFO:teuthology.orchestra.run.smithi040.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-13T17:51:29.123 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-13T17:51:29.123 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 34h/52d Inode: 1 Links: 2 2024-09-13T17:51:29.123 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:51:29.123 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-13T17:51:29.123 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-09-13 17:50:39.568574682 +0000 2024-09-13T17:51:29.123 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-09-13 17:50:47.797997920 +0000 2024-09-13T17:51:29.123 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2024-09-13T17:51:29.124 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-13T17:51:29.124 DEBUG:teuthology.orchestra.run.smithi040:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-13T17:51:29.201 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:51:29.208 INFO:tasks.workunit.client.0.smithi040.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-13T17:51:29.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.222+0000 7f9d2bfff700 1 Processor -- start 2024-09-13T17:51:29.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.222+0000 7f9d2bfff700 1 -- start start 2024-09-13T17:51:29.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.222+0000 7f9d2bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9d2c1500c0 0x7f9d2c0c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:29.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.222+0000 7f9d2bfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9d2c072250 0x7f9d2c0c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:29.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.222+0000 7f9d2bfff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2c0cba20 con 0x7f9d2c070500 2024-09-13T17:51:29.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.222+0000 7f9d2bfff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d2c1560e0 con 0x7f9d2c072250 2024-09-13T17:51:29.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.222+0000 7f9d2bfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c1563c0 con 0x7f9d2c1500c0 2024-09-13T17:51:29.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.222+0000 7f9d2affd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9d2c1500c0 0x7f9d2c0c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:29.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.222+0000 7f9d2b7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9d2c072250 0x7f9d2c0c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:29.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.222+0000 7f9d2affd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9d2c1500c0 0x7f9d2c0c66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:59862/0 (socket says 172.21.15.40:59862) 2024-09-13T17:51:29.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.222+0000 7f9d227fc700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9d2c070500 0x7f9d2c071b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:37148/0 (socket says 172.21.15.40:37148) 2024-09-13T17:51:29.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.222+0000 7f9d2affd700 1 -- 172.21.15.40:0/2437489986 learned_addr learned my addr 172.21.15.40:0/2437489986 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:29.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.222+0000 7f9d2b7fe700 1 -- 172.21.15.40:0/2437489986 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9d2c1500c0 msgr2=0x7f9d2c0c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:29.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.222+0000 7f9d2b7fe700 1 --2- 172.21.15.40:0/2437489986 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9d2c1500c0 0x7f9d2c0c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.222+0000 7f9d2b7fe700 1 -- 172.21.15.40:0/2437489986 >> v1:172.21.15.43:6789/0 conn(0x7f9d2c070500 legacy=0x7f9d2c071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:29.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.222+0000 7f9d2b7fe700 1 -- 172.21.15.40:0/2437489986 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c1566a0 con 0x7f9d2c072250 2024-09-13T17:51:29.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.222+0000 7f9d2affd700 1 --2- 172.21.15.40:0/2437489986 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9d2c1500c0 0x7f9d2c0c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:29.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.222+0000 7f9d2a7fc700 1 -- 172.21.15.40:0/2437489986 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392574326 0 0) 0x7f9d2c0cba20 con 0x7f9d2c070500 2024-09-13T17:51:29.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.222+0000 7f9d2b7fe700 1 --2- 172.21.15.40:0/2437489986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2c072250 0x7f9d2c0c6c10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400bfa0 tx=0x7f9d2400d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7ef68e1ade3c7c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:29.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2a7fc700 1 -- 172.21.15.40:0/2437489986 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24010070 con 0x7f9d2c072250 2024-09-13T17:51:29.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2a7fc700 1 -- 172.21.15.40:0/2437489986 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2400b2e0 con 0x7f9d2c072250 2024-09-13T17:51:29.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2a7fc700 1 -- 172.21.15.40:0/2437489986 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2400dc40 con 0x7f9d2c072250 2024-09-13T17:51:29.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2bfff700 1 -- 172.21.15.40:0/2437489986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2c072250 msgr2=0x7f9d2c0c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:29.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2bfff700 1 --2- 172.21.15.40:0/2437489986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2c072250 0x7f9d2c0c6c10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400bfa0 tx=0x7f9d2400d330 comp rx=0 tx=0).stop 2024-09-13T17:51:29.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2bfff700 1 -- 172.21.15.40:0/2437489986 shutdown_connections 2024-09-13T17:51:29.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2bfff700 1 --2- 172.21.15.40:0/2437489986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2c072250 0x7f9d2c0c6c10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2bfff700 1 --2- 172.21.15.40:0/2437489986 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9d2c1500c0 0x7f9d2c0c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2bfff700 1 -- 172.21.15.40:0/2437489986 >> 172.21.15.40:0/2437489986 conn(0x7f9d2c0cabc0 msgr2=0x7f9d2c0cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:29.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2bfff700 1 -- 172.21.15.40:0/2437489986 shutdown_connections 2024-09-13T17:51:29.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2bfff700 1 -- 172.21.15.40:0/2437489986 wait complete. 2024-09-13T17:51:29.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2bfff700 1 Processor -- start 2024-09-13T17:51:29.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2bfff700 1 -- start start 2024-09-13T17:51:29.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2c070500 0x7f9d2c0ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:29.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9d2c072250 0x7f9d2c1e0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:29.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d2c1500c0 0x7f9d2c1ea530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:29.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2bfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0c6cf0 con 0x7f9d2c1500c0 2024-09-13T17:51:29.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2bfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c1ee500 con 0x7f9d2c070500 2024-09-13T17:51:29.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2bfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0c85c0 con 0x7f9d2c072250 2024-09-13T17:51:29.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d2c1500c0 0x7f9d2c1ea530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:29.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d2c1500c0 0x7f9d2c1ea530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55214/0 (socket says 172.21.15.40:55214) 2024-09-13T17:51:29.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2b7fe700 1 -- 172.21.15.40:0/1947850330 learned_addr learned my addr 172.21.15.40:0/1947850330 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:29.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d2affd700 1 --2- 172.21.15.40:0/1947850330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2c070500 0x7f9d2c0ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:29.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.226+0000 7f9d227fc700 1 --2- 172.21.15.40:0/1947850330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9d2c072250 0x7f9d2c1e0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:29.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.230+0000 7f9d2b7fe700 1 -- 172.21.15.40:0/1947850330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9d2c072250 msgr2=0x7f9d2c1e0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:29.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.230+0000 7f9d2b7fe700 1 --2- 172.21.15.40:0/1947850330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9d2c072250 0x7f9d2c1e0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.230+0000 7f9d2b7fe700 1 -- 172.21.15.40:0/1947850330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2c070500 msgr2=0x7f9d2c0ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:29.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.230+0000 7f9d2b7fe700 1 --2- 172.21.15.40:0/1947850330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2c070500 0x7f9d2c0ca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.230+0000 7f9d2b7fe700 1 -- 172.21.15.40:0/1947850330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c009570 con 0x7f9d2c1500c0 2024-09-13T17:51:29.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.230+0000 7f9d2b7fe700 1 --2- 172.21.15.40:0/1947850330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d2c1500c0 0x7f9d2c1ea530 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400bfd0 tx=0x7f9d24004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.230+0000 7f9d23fff700 1 -- 172.21.15.40:0/1947850330 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24003e30 con 0x7f9d2c1500c0 2024-09-13T17:51:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.230+0000 7f9d2bfff700 1 -- 172.21.15.40:0/1947850330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2400bc80 con 0x7f9d2c1500c0 2024-09-13T17:51:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.230+0000 7f9d2bfff700 1 -- 172.21.15.40:0/1947850330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2c0c82e0 con 0x7f9d2c1500c0 2024-09-13T17:51:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.230+0000 7f9d23fff700 1 -- 172.21.15.40:0/1947850330 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d24004250 con 0x7f9d2c1500c0 2024-09-13T17:51:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.230+0000 7f9d23fff700 1 -- 172.21.15.40:0/1947850330 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d240045c0 con 0x7f9d2c1500c0 2024-09-13T17:51:29.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.230+0000 7f9d2bfff700 1 -- 172.21.15.40:0/1947850330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d10005310 con 0x7f9d2c1500c0 2024-09-13T17:51:29.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.230+0000 7f9d23fff700 1 -- 172.21.15.40:0/1947850330 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9d24004720 con 0x7f9d2c1500c0 2024-09-13T17:51:29.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.234+0000 7f9d23fff700 1 --2- 172.21.15.40:0/1947850330 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9d0c066760 0x7f9d0c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:29.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.234+0000 7f9d23fff700 1 -- 172.21.15.40:0/1947850330 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9d24087a30 con 0x7f9d2c1500c0 2024-09-13T17:51:29.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.234+0000 7f9d2affd700 1 --2- 172.21.15.40:0/1947850330 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9d0c066760 0x7f9d0c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:29.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.234+0000 7f9d2affd700 1 --2- 172.21.15.40:0/1947850330 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9d0c066760 0x7f9d0c068c20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c006d50 tx=0x7f9d1c0092e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:29.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.238+0000 7f9d23fff700 1 -- 172.21.15.40:0/1947850330 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9d24054960 con 0x7f9d2c1500c0 2024-09-13T17:51:29.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.454+0000 7f9d2bfff700 1 -- 172.21.15.40:0/1947850330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d100055c0 con 0x7f9d2c1500c0 2024-09-13T17:51:29.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.454+0000 7f9d23fff700 1 -- 172.21.15.40:0/1947850330 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6268 (secure 0 0 0) 0x7f9d24057f80 con 0x7f9d2c1500c0 2024-09-13T17:51:29.460 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T17:51:29.460 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6839/1861594206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1861594206},{"type":"v1","addr":"172.21.15.40:6839","nonce":1861594206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1548394691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1548394691},{"type":"v1","addr":"172.21.15.40:6835","nonce":1548394691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/486611242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":486611242},{"type":"v1","addr":"172.21.15.40:6837","nonce":486611242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:50:34.562623+0000","modified":"2024-09-13T17:51:12.236489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4288,"mds_1":4285,"mds_2":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"f","rank":2,"incarnation":14,"state":"up:active","state_seq":12,"addr":"172.21.15.43:6833/526975022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":526975022},{"type":"v1","addr":"172.21.15.43:6833","nonce":526975022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4285":{"gid":4285,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.43:6835/2586910944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2586910944},{"type":"v1","addr":"172.21.15.43:6835","nonce":2586910944}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/334160033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":334160033},{"type":"v1","addr":"172.21.15.43:6837","nonce":334160033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:51:29.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.458+0000 7f9d2bfff700 1 -- 172.21.15.40:0/1947850330 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9d0c066760 msgr2=0x7f9d0c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:29.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.458+0000 7f9d2bfff700 1 --2- 172.21.15.40:0/1947850330 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9d0c066760 0x7f9d0c068c20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c006d50 tx=0x7f9d1c0092e0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.458+0000 7f9d2bfff700 1 -- 172.21.15.40:0/1947850330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d2c1500c0 msgr2=0x7f9d2c1ea530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:29.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.458+0000 7f9d2bfff700 1 --2- 172.21.15.40:0/1947850330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d2c1500c0 0x7f9d2c1ea530 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400bfd0 tx=0x7f9d24004af0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.458+0000 7f9d2bfff700 1 -- 172.21.15.40:0/1947850330 shutdown_connections 2024-09-13T17:51:29.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.458+0000 7f9d2bfff700 1 --2- 172.21.15.40:0/1947850330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d2c1500c0 0x7f9d2c1ea530 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.458+0000 7f9d2bfff700 1 --2- 172.21.15.40:0/1947850330 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9d0c066760 0x7f9d0c068c20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.458+0000 7f9d2bfff700 1 --2- 172.21.15.40:0/1947850330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9d2c072250 0x7f9d2c1e0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.458+0000 7f9d2bfff700 1 --2- 172.21.15.40:0/1947850330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2c070500 0x7f9d2c0ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.458+0000 7f9d2bfff700 1 -- 172.21.15.40:0/1947850330 >> 172.21.15.40:0/1947850330 conn(0x7f9d2c0cabc0 msgr2=0x7f9d2c0c3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:29.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.458+0000 7f9d2bfff700 1 -- 172.21.15.40:0/1947850330 shutdown_connections 2024-09-13T17:51:29.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.458+0000 7f9d2bfff700 1 -- 172.21.15.40:0/1947850330 wait complete. 2024-09-13T17:51:29.464 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 15 2024-09-13T17:51:29.475 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2024-09-13T17:51:29.475 INFO:tasks.mon_thrash.mon_thrasher:seed: 1726249887, revive delay: 20.0, thrash delay: 10.0 thrash many: False, maintain quorum: True store thrash: False, probability: 50 freeze mon: prob 10.0 duration 15.0 2024-09-13T17:51:29.475 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T17:51:29.476 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:51:29.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9a99e700 1 Processor -- start 2024-09-13T17:51:29.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9a99e700 1 -- start start 2024-09-13T17:51:29.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9a99e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe940c79c0 0x7fbe940c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:29.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9a99e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbe9406e100 0x7fbe9414f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:29.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9a99e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe94155ff0 con 0x7fbe940c75e0 2024-09-13T17:51:29.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9a99e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe941562d0 con 0x7fbe9406e100 2024-09-13T17:51:29.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9a99e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe941565b0 con 0x7fbe940c79c0 2024-09-13T17:51:29.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9919b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe940c79c0 0x7fbe940c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:29.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9999c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbe940c75e0 0x7fbe941557b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:37150/0 (socket says 172.21.15.40:37150) 2024-09-13T17:51:29.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9999c700 1 -- 172.21.15.40:0/2597180037 learned_addr learned my addr 172.21.15.40:0/2597180037 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:29.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9a19d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbe9406e100 0x7fbe9414f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:29.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9899a700 1 -- 172.21.15.40:0/2597180037 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1928979117 0 0) 0x7fbe94155ff0 con 0x7fbe940c75e0 2024-09-13T17:51:29.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9899a700 1 -- 172.21.15.40:0/2597180037 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe7c0035d0 con 0x7fbe940c75e0 2024-09-13T17:51:29.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9919b700 1 -- 172.21.15.40:0/2597180037 >> v2:172.21.15.43:3300/0 conn(0x7fbe9406e100 msgr2=0x7fbe9414f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:29.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9919b700 1 --2- 172.21.15.40:0/2597180037 >> v2:172.21.15.43:3300/0 conn(0x7fbe9406e100 0x7fbe9414f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9919b700 1 -- 172.21.15.40:0/2597180037 >> v1:172.21.15.43:6789/0 conn(0x7fbe940c75e0 legacy=0x7fbe941557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:29.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9919b700 1 -- 172.21.15.40:0/2597180037 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9406d620 con 0x7fbe940c79c0 2024-09-13T17:51:29.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9a19d700 1 --2- 172.21.15.40:0/2597180037 >> v2:172.21.15.43:3300/0 conn(0x7fbe9406e100 0x7fbe9414f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:29.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9919b700 1 --2- 172.21.15.40:0/2597180037 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe940c79c0 0x7fbe940c6b40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009d90 tx=0x7fbe9000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6d37bc31179db7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:29.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9899a700 1 -- 172.21.15.40:0/2597180037 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9000e070 con 0x7fbe940c79c0 2024-09-13T17:51:29.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9899a700 1 -- 172.21.15.40:0/2597180037 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9000bb20 con 0x7fbe940c79c0 2024-09-13T17:51:29.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.582+0000 7fbe9899a700 1 -- 172.21.15.40:0/2597180037 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9000be90 con 0x7fbe940c79c0 2024-09-13T17:51:29.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9a99e700 1 -- 172.21.15.40:0/2597180037 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe940c79c0 msgr2=0x7fbe940c6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:29.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9a99e700 1 --2- 172.21.15.40:0/2597180037 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe940c79c0 0x7fbe940c6b40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009d90 tx=0x7fbe9000b330 comp rx=0 tx=0).stop 2024-09-13T17:51:29.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9a99e700 1 -- 172.21.15.40:0/2597180037 shutdown_connections 2024-09-13T17:51:29.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9a99e700 1 --2- 172.21.15.40:0/2597180037 >> v2:172.21.15.43:3300/0 conn(0x7fbe9406e100 0x7fbe9414f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9a99e700 1 --2- 172.21.15.40:0/2597180037 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe940c79c0 0x7fbe940c6b40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9a99e700 1 -- 172.21.15.40:0/2597180037 >> 172.21.15.40:0/2597180037 conn(0x7fbe940cb450 msgr2=0x7fbe940cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:29.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9a99e700 1 -- 172.21.15.40:0/2597180037 shutdown_connections 2024-09-13T17:51:29.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9a99e700 1 -- 172.21.15.40:0/2597180037 wait complete. 2024-09-13T17:51:29.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9a99e700 1 Processor -- start 2024-09-13T17:51:29.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9a99e700 1 -- start start 2024-09-13T17:51:29.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9a99e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe9406e100 0x7fbe940c4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:29.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9a99e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe940c75e0 0x7fbe941e01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:29.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9a99e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe940c79c0 0x7fbe941edbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:29.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9a99e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe940c5550 con 0x7fbe940c75e0 2024-09-13T17:51:29.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9a99e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe940c5810 con 0x7fbe940c79c0 2024-09-13T17:51:29.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9a99e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe940cabe0 con 0x7fbe9406e100 2024-09-13T17:51:29.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9919b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe940c75e0 0x7fbe941e01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:29.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9919b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe940c75e0 0x7fbe941e01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55226/0 (socket says 172.21.15.40:55226) 2024-09-13T17:51:29.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9919b700 1 -- 172.21.15.40:0/3507854486 learned_addr learned my addr 172.21.15.40:0/3507854486 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:29.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9999c700 1 --2- 172.21.15.40:0/3507854486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe9406e100 0x7fbe940c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:29.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.586+0000 7fbe9a19d700 1 --2- 172.21.15.40:0/3507854486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe940c79c0 0x7fbe941edbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.590+0000 7fbe9919b700 1 -- 172.21.15.40:0/3507854486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe9406e100 msgr2=0x7fbe940c4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.590+0000 7fbe9919b700 1 --2- 172.21.15.40:0/3507854486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe9406e100 0x7fbe940c4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.590+0000 7fbe9919b700 1 -- 172.21.15.40:0/3507854486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe940c79c0 msgr2=0x7fbe941edbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.590+0000 7fbe9919b700 1 --2- 172.21.15.40:0/3507854486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe940c79c0 0x7fbe941edbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.590+0000 7fbe9919b700 1 -- 172.21.15.40:0/3507854486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c0094b0 con 0x7fbe940c75e0 2024-09-13T17:51:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.590+0000 7fbe9919b700 1 --2- 172.21.15.40:0/3507854486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe940c75e0 0x7fbe941e01c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000b370 tx=0x7fbe90004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.590+0000 7fbe8a7fc700 1 -- 172.21.15.40:0/3507854486 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90007000 con 0x7fbe940c75e0 2024-09-13T17:51:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.590+0000 7fbe8a7fc700 1 -- 172.21.15.40:0/3507854486 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe90020070 con 0x7fbe940c75e0 2024-09-13T17:51:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.590+0000 7fbe8a7fc700 1 -- 172.21.15.40:0/3507854486 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe900276c0 con 0x7fbe940c75e0 2024-09-13T17:51:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.590+0000 7fbe9a99e700 1 -- 172.21.15.40:0/3507854486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe90009a70 con 0x7fbe940c75e0 2024-09-13T17:51:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.590+0000 7fbe9a99e700 1 -- 172.21.15.40:0/3507854486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe94158b90 con 0x7fbe940c75e0 2024-09-13T17:51:29.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.590+0000 7fbe9a99e700 1 -- 172.21.15.40:0/3507854486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe940cb100 con 0x7fbe940c75e0 2024-09-13T17:51:29.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.590+0000 7fbe8a7fc700 1 -- 172.21.15.40:0/3507854486 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbe90027820 con 0x7fbe940c75e0 2024-09-13T17:51:29.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.594+0000 7fbe8a7fc700 1 --2- 172.21.15.40:0/3507854486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbe80066730 0x7fbe80068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:29.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.594+0000 7fbe9999c700 1 --2- 172.21.15.40:0/3507854486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbe80066730 0x7fbe80068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:29.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.594+0000 7fbe8a7fc700 1 -- 172.21.15.40:0/3507854486 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbe90022070 con 0x7fbe940c75e0 2024-09-13T17:51:29.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.594+0000 7fbe9999c700 1 --2- 172.21.15.40:0/3507854486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbe80066730 0x7fbe80068bf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbe84007fd0 tx=0x7fbe84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:29.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.598+0000 7fbe8a7fc700 1 -- 172.21.15.40:0/3507854486 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbe900533b0 con 0x7fbe940c75e0 2024-09-13T17:51:29.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.830+0000 7fbe9a99e700 1 -- 172.21.15.40:0/3507854486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbe941589d0 con 0x7fbe940c75e0 2024-09-13T17:51:29.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.830+0000 7fbe8a7fc700 1 -- 172.21.15.40:0/3507854486 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7fbe900569d0 con 0x7fbe940c75e0 2024-09-13T17:51:29.835 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":80,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:51:29.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.830+0000 7fbe9a99e700 1 -- 172.21.15.40:0/3507854486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbe80066730 msgr2=0x7fbe80068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:29.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.830+0000 7fbe9a99e700 1 --2- 172.21.15.40:0/3507854486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbe80066730 0x7fbe80068bf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbe84007fd0 tx=0x7fbe84009040 comp rx=0 tx=0).stop 2024-09-13T17:51:29.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.830+0000 7fbe9a99e700 1 -- 172.21.15.40:0/3507854486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe940c75e0 msgr2=0x7fbe941e01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:29.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.830+0000 7fbe9a99e700 1 --2- 172.21.15.40:0/3507854486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe940c75e0 0x7fbe941e01c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000b370 tx=0x7fbe90004af0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.834+0000 7fbe9a99e700 1 -- 172.21.15.40:0/3507854486 shutdown_connections 2024-09-13T17:51:29.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.834+0000 7fbe9a99e700 1 --2- 172.21.15.40:0/3507854486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe940c79c0 0x7fbe941edbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.834+0000 7fbe9a99e700 1 --2- 172.21.15.40:0/3507854486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe940c75e0 0x7fbe941e01c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.834+0000 7fbe9a99e700 1 --2- 172.21.15.40:0/3507854486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbe80066730 0x7fbe80068bf0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.834+0000 7fbe9a99e700 1 --2- 172.21.15.40:0/3507854486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe9406e100 0x7fbe940c4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.834+0000 7fbe9a99e700 1 -- 172.21.15.40:0/3507854486 >> 172.21.15.40:0/3507854486 conn(0x7fbe940cb450 msgr2=0x7fbe940cb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:29.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.834+0000 7fbe9a99e700 1 -- 172.21.15.40:0/3507854486 shutdown_connections 2024-09-13T17:51:29.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.834+0000 7fbe9a99e700 1 -- 172.21.15.40:0/3507854486 wait complete. 2024-09-13T17:51:29.850 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T17:51:29.850 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T17:51:29.851 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T17:51:29.851 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:51:29.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.962+0000 7ffb6b59e700 1 Processor -- start 2024-09-13T17:51:29.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.962+0000 7ffb6b59e700 1 -- start start 2024-09-13T17:51:29.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.962+0000 7ffb6b59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb6c0cf1a0 0x7ffb6c15a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:29.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.962+0000 7ffb6b59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffb6c0cc250 0x7ffb6c0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:29.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.962+0000 7ffb6b59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb6c15a7a0 con 0x7ffb6c0cf580 2024-09-13T17:51:29.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.962+0000 7ffb6b59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb6c0cbdc0 con 0x7ffb6c0cc250 2024-09-13T17:51:29.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.962+0000 7ffb6b59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c153f80 con 0x7ffb6c0cf1a0 2024-09-13T17:51:29.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.962+0000 7ffb6ad9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffb6c0cc250 0x7ffb6c0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:29.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.962+0000 7ffb6ad9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffb6c0cc250 0x7ffb6c0c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55238/0 (socket says 172.21.15.40:55238) 2024-09-13T17:51:29.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.962+0000 7ffb6ad9d700 1 -- 172.21.15.40:0/2898148678 learned_addr learned my addr 172.21.15.40:0/2898148678 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:29.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.962+0000 7ffb69d9b700 1 --1- 172.21.15.40:0/2898148678 >> v1:172.21.15.40:6789/0 conn(0x7ffb6c0cf580 0x7ffb6c0c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44586/0 (socket says 172.21.15.40:44586) 2024-09-13T17:51:29.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.962+0000 7ffb6a59c700 1 --2- 172.21.15.40:0/2898148678 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb6c0cf1a0 0x7ffb6c15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:29.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.962+0000 7ffb6ad9d700 1 -- 172.21.15.40:0/2898148678 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb6c0cf1a0 msgr2=0x7ffb6c15a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:29.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.962+0000 7ffb6ad9d700 1 --2- 172.21.15.40:0/2898148678 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb6c0cf1a0 0x7ffb6c15a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.962+0000 7ffb6ad9d700 1 -- 172.21.15.40:0/2898148678 >> v1:172.21.15.40:6789/0 conn(0x7ffb6c0cf580 legacy=0x7ffb6c0c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:29.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.962+0000 7ffb6ad9d700 1 -- 172.21.15.40:0/2898148678 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c0caed0 con 0x7ffb6c0cc250 2024-09-13T17:51:29.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.962+0000 7ffb6ad9d700 1 --2- 172.21.15.40:0/2898148678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c0cc250 0x7ffb6c0c8aa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009e50 tx=0x7ffb6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b7a50ade8e2770 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:29.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.962+0000 7ffb6959a700 1 -- 172.21.15.40:0/2898148678 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6400e070 con 0x7ffb6c0cc250 2024-09-13T17:51:29.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.962+0000 7ffb6959a700 1 -- 172.21.15.40:0/2898148678 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6400b910 con 0x7ffb6c0cc250 2024-09-13T17:51:29.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.962+0000 7ffb6959a700 1 -- 172.21.15.40:0/2898148678 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6400bcc0 con 0x7ffb6c0cc250 2024-09-13T17:51:29.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.966+0000 7ffb6b59e700 1 -- 172.21.15.40:0/2898148678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c0cc250 msgr2=0x7ffb6c0c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:29.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.966+0000 7ffb6b59e700 1 --2- 172.21.15.40:0/2898148678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c0cc250 0x7ffb6c0c8aa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009e50 tx=0x7ffb6400b330 comp rx=0 tx=0).stop 2024-09-13T17:51:29.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.966+0000 7ffb6b59e700 1 -- 172.21.15.40:0/2898148678 shutdown_connections 2024-09-13T17:51:29.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.966+0000 7ffb6b59e700 1 --2- 172.21.15.40:0/2898148678 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c0cc250 0x7ffb6c0c8aa0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.966+0000 7ffb6b59e700 1 --2- 172.21.15.40:0/2898148678 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb6c0cf1a0 0x7ffb6c15a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.966+0000 7ffb6b59e700 1 -- 172.21.15.40:0/2898148678 >> 172.21.15.40:0/2898148678 conn(0x7ffb6c066090 msgr2=0x7ffb6c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:29.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.966+0000 7ffb6b59e700 1 -- 172.21.15.40:0/2898148678 shutdown_connections 2024-09-13T17:51:29.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.966+0000 7ffb6b59e700 1 -- 172.21.15.40:0/2898148678 wait complete. 2024-09-13T17:51:29.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.966+0000 7ffb6b59e700 1 Processor -- start 2024-09-13T17:51:29.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.966+0000 7ffb6b59e700 1 -- start start 2024-09-13T17:51:29.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6b59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c0cc250 0x7ffb6c159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:29.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6b59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb6c0cf1a0 0x7ffb6c1eee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:29.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6b59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb6c0cf580 0x7ffb6c1ef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:29.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6b59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c1ee9e0 con 0x7ffb6c0cc250 2024-09-13T17:51:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6b59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c15a190 con 0x7ffb6c0cf580 2024-09-13T17:51:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6b59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c0b2bb0 con 0x7ffb6c0cf1a0 2024-09-13T17:51:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c0cc250 0x7ffb6c159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c0cc250 0x7ffb6c159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55250/0 (socket says 172.21.15.40:55250) 2024-09-13T17:51:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6a59c700 1 -- 172.21.15.40:0/687328664 learned_addr learned my addr 172.21.15.40:0/687328664 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb69d9b700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb6c0cf1a0 0x7ffb6c1eee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6ad9d700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb6c0cf580 0x7ffb6c1ef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6a59c700 1 -- 172.21.15.40:0/687328664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb6c0cf1a0 msgr2=0x7ffb6c1eee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6a59c700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb6c0cf1a0 0x7ffb6c1eee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6a59c700 1 -- 172.21.15.40:0/687328664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb6c0cf580 msgr2=0x7ffb6c1ef3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6a59c700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb6c0cf580 0x7ffb6c1ef3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6a59c700 1 -- 172.21.15.40:0/687328664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb64009b30 con 0x7ffb6c0cc250 2024-09-13T17:51:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6ad9d700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb6c0cf580 0x7ffb6c1ef3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:51:29.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6a59c700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c0cc250 0x7ffb6c159390 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00e9d0 tx=0x7ffb5c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:29.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb5affd700 1 -- 172.21.15.40:0/687328664 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c009720 con 0x7ffb6c0cc250 2024-09-13T17:51:29.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb5affd700 1 -- 172.21.15.40:0/687328664 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5c00c850 con 0x7ffb6c0cc250 2024-09-13T17:51:29.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6b59e700 1 -- 172.21.15.40:0/687328664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6c0b2e90 con 0x7ffb6c0cc250 2024-09-13T17:51:29.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb5affd700 1 -- 172.21.15.40:0/687328664 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c00cbc0 con 0x7ffb6c0cc250 2024-09-13T17:51:29.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6b59e700 1 -- 172.21.15.40:0/687328664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6c15a3e0 con 0x7ffb6c0cc250 2024-09-13T17:51:29.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6b59e700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c0b3ac0 0x7ffb6c1f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:29.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb6b59e700 1 -- 172.21.15.40:0/687328664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c074750 con 0x7ffb6c0b3ac0 2024-09-13T17:51:29.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.970+0000 7ffb69d9b700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c0b3ac0 0x7ffb6c1f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:29.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.974+0000 7ffb69d9b700 1 -- 172.21.15.40:0/687328664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb6c0cb770 con 0x7ffb6c0b3ac0 2024-09-13T17:51:29.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.974+0000 7ffb5affd700 1 -- 172.21.15.40:0/687328664 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffb5c00cd20 con 0x7ffb6c0cc250 2024-09-13T17:51:29.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.974+0000 7ffb69d9b700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c0b3ac0 0x7ffb6c1f1db0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c1ec470 tx=0x7ffb600016b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:29.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.974+0000 7ffb5affd700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffb4c066730 0x7ffb4c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.974+0000 7ffb5affd700 1 -- 172.21.15.40:0/687328664 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ffb5c0845a0 con 0x7ffb6c0cc250 2024-09-13T17:51:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.974+0000 7ffb6ad9d700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffb4c066730 0x7ffb4c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.974+0000 7ffb5affd700 1 -- 172.21.15.40:0/687328664 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60001d30 con 0x7ffb6c0b3ac0 2024-09-13T17:51:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.974+0000 7ffb5affd700 1 -- 172.21.15.40:0/687328664 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ffb600114f0 con 0x7ffb6c0b3ac0 2024-09-13T17:51:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.974+0000 7ffb5affd700 1 -- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c0b3ac0 msgr2=0x7ffb6c1f1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.974+0000 7ffb5affd700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c0b3ac0 0x7ffb6c1f1db0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c1ec470 tx=0x7ffb600016b0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.974+0000 7ffb5affd700 1 -- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c0b3ac0 msgr2=0x7ffb6c1f1db0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:51:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.974+0000 7ffb5affd700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c0b3ac0 0x7ffb6c1f1db0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.974+0000 7ffb6ad9d700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffb4c066730 0x7ffb4c068bf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c159e30 tx=0x7ffb64009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:29.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.986+0000 7ffb6b59e700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c1f22f0 0x7ffb6c1f2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:29.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.986+0000 7ffb6b59e700 1 -- 172.21.15.40:0/687328664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c0c48e0 con 0x7ffb6c1f22f0 2024-09-13T17:51:29.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.986+0000 7ffb69d9b700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c1f22f0 0x7ffb6c1f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:29.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.986+0000 7ffb69d9b700 1 -- 172.21.15.40:0/687328664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ffb6c0cb490 con 0x7ffb6c1f22f0 2024-09-13T17:51:29.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.986+0000 7ffb69d9b700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c1f22f0 0x7ffb6c1f2700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0ca0b0 tx=0x7ffb60001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:29.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.986+0000 7ffb5affd700 1 -- 172.21.15.40:0/687328664 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb60011930 con 0x7ffb6c1f22f0 2024-09-13T17:51:29.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.986+0000 7ffb5affd700 1 -- 172.21.15.40:0/687328664 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4783 (secure 0 0 0) 0x7ffb60011a90 con 0x7ffb6c1f22f0 2024-09-13T17:51:29.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.986+0000 7ffb5affd700 1 -- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c1f22f0 msgr2=0x7ffb6c1f2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:29.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.986+0000 7ffb5affd700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c1f22f0 0x7ffb6c1f2700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0ca0b0 tx=0x7ffb60001080 comp rx=0 tx=0).stop 2024-09-13T17:51:29.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.986+0000 7ffb5affd700 1 -- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c1f22f0 msgr2=0x7ffb6c1f2700 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:51:29.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.986+0000 7ffb5affd700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c1f22f0 0x7ffb6c1f2700 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:29.993 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:51:29.993 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:51:29.993 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:51:29.993 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:51:29.993 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 8, 2024-09-13T17:51:29.994 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:51:29.994 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:51:29.994 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:51:29.994 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:51:29.994 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:29.994 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 80, 2024-09-13T17:51:29.994 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:51:29.994 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:51:29.994 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:51:29.994 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:51:29.994 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:51:29.994 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:51:29.994 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:51:29.994 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:51:29.995 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:51:29.995 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:51:29.995 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:51:29.995 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:51:29.995 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:51:29.995 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:29.995 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:51:29.995 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:51:29.995 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:51:29.995 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:51:29.995 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:51:29.995 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:51:29.995 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:51:29.995 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:51:29.996 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:51:29.996 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:51:29.996 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:51:29.996 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:51:29.996 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:29.996 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:29.996 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:51:29.996 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:51:29.996 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:51:29.996 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:51:29.996 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:51:29.996 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:51:29.996 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:51:29.996 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:51:29.997 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:51:29.997 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:51:29.997 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:51:29.997 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:51:29.997 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:51:29.997 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:51:29.997 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:51:29.998 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:51:29.998 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:51:29.998 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:51:29.998 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:51:29.998 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:51:29.999 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:51:29.999 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:51:29.999 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:51:29.999 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:51:29.999 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:51:29.999 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:51:30.000 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:51:30.000 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:30.000 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:51:30.000 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.000 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:51:30.000 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.000 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:51:30.001 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:51:30.001 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:51:30.001 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:51:30.001 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.001 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:51:30.001 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:51:30.002 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:30.002 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.002 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.002 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:51:30.002 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:51:30.003 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:30.003 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.003 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:30.003 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.003 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:51:30.003 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:51:30.004 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:51:30.004 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:51:30.005 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:51:30.005 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.005 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.005 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:51:30.005 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:51:30.006 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:51:30.006 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:51:30.006 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.006 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:51:30.006 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:51:30.007 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:30.007 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.007 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.007 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:51:30.007 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:51:30.007 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:30.008 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.008 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:30.008 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.008 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:51:30.008 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:51:30.008 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:51:30.009 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:51:30.009 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:51:30.009 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.009 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.009 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:51:30.009 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:51:30.010 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:51:30.010 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:51:30.010 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.010 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:51:30.010 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:51:30.011 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:30.011 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.011 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.011 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:51:30.011 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:51:30.011 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:30.011 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.012 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:30.012 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.012 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:51:30.012 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:51:30.012 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:51:30.013 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:51:30.013 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:51:30.013 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.013 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:30.013 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.013 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:51:30.014 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:51:30.014 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.014 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:30.014 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:30.014 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:51:30.014 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.015 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:30.015 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:51:30.015 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.015 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:30.015 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:30.015 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:51:30.016 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.016 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:30.016 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:51:30.016 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.016 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:30.016 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:30.017 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T17:51:30.017 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.017 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:30.017 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:51:30.017 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.017 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:30.017 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:30.018 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:51:30.018 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.018 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:30.018 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.018 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:51:30.018 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:51:30.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.986+0000 7ffb6b59e700 1 -- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffb4c066730 msgr2=0x7ffb4c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.986+0000 7ffb6b59e700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffb4c066730 0x7ffb4c068bf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c159e30 tx=0x7ffb64009000 comp rx=0 tx=0).stop 2024-09-13T17:51:30.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.986+0000 7ffb6b59e700 1 -- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c0cc250 msgr2=0x7ffb6c159390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.986+0000 7ffb6b59e700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c0cc250 0x7ffb6c159390 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00e9d0 tx=0x7ffb5c00ed90 comp rx=0 tx=0).stop 2024-09-13T17:51:30.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.986+0000 7ffb6ad9d700 1 -- 172.21.15.40:0/687328664 reap_dead start 2024-09-13T17:51:30.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.990+0000 7ffb6b59e700 1 -- 172.21.15.40:0/687328664 shutdown_connections 2024-09-13T17:51:30.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.990+0000 7ffb6b59e700 1 --2- 172.21.15.40:0/687328664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb6c0cc250 0x7ffb6c159390 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.990+0000 7ffb6b59e700 1 -- 172.21.15.40:0/687328664 >> 172.21.15.40:0/687328664 conn(0x7ffb6c066090 msgr2=0x7ffb6c0c7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:30.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.990+0000 7ffb6b59e700 1 -- 172.21.15.40:0/687328664 shutdown_connections 2024-09-13T17:51:30.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:29.990+0000 7ffb6b59e700 1 -- 172.21.15.40:0/687328664 wait complete. 2024-09-13T17:51:30.023 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:51:30.023 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T17:51:30.023 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:51:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.142+0000 7f88c63a6700 1 Processor -- start 2024-09-13T17:51:30.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.142+0000 7f88c63a6700 1 -- start start 2024-09-13T17:51:30.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c63a6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88c00b0d70 0x7f88c0157f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c63a6700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f88c006a950 0x7f88c006ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c63a6700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c0158440 con 0x7f88c0151750 2024-09-13T17:51:30.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c63a6700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c006a4c0 con 0x7f88c006a950 2024-09-13T17:51:30.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c63a6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c006b340 con 0x7f88c00b0d70 2024-09-13T17:51:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c4943700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f88c006a950 0x7f88c006ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c4943700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f88c006a950 0x7f88c006ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55264/0 (socket says 172.21.15.40:55264) 2024-09-13T17:51:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c4943700 1 -- 172.21.15.40:0/2662288137 learned_addr learned my addr 172.21.15.40:0/2662288137 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88bf7fe700 1 --2- 172.21.15.40:0/2662288137 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88c00b0d70 0x7f88c0157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c4943700 1 -- 172.21.15.40:0/2662288137 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88c00b0d70 msgr2=0x7f88c0157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c4943700 1 --2- 172.21.15.40:0/2662288137 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88c00b0d70 0x7f88c0157f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c4943700 1 -- 172.21.15.40:0/2662288137 >> v1:172.21.15.43:6789/0 conn(0x7f88c0151750 legacy=0x7f88c006fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c4943700 1 -- 172.21.15.40:0/2662288137 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c006cf60 con 0x7f88c006a950 2024-09-13T17:51:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c4943700 1 --2- 172.21.15.40:0/2662288137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c006a950 0x7f88c006ae00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009e50 tx=0x7f88b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b958d8488a5e69c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88beffd700 1 -- 172.21.15.40:0/2662288137 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b800e070 con 0x7f88c006a950 2024-09-13T17:51:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88beffd700 1 -- 172.21.15.40:0/2662288137 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b800b910 con 0x7f88c006a950 2024-09-13T17:51:30.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88beffd700 1 -- 172.21.15.40:0/2662288137 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b800bcc0 con 0x7f88c006a950 2024-09-13T17:51:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c63a6700 1 -- 172.21.15.40:0/2662288137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c006a950 msgr2=0x7f88c006ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c63a6700 1 --2- 172.21.15.40:0/2662288137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c006a950 0x7f88c006ae00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009e50 tx=0x7f88b800b330 comp rx=0 tx=0).stop 2024-09-13T17:51:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c63a6700 1 -- 172.21.15.40:0/2662288137 shutdown_connections 2024-09-13T17:51:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c63a6700 1 --2- 172.21.15.40:0/2662288137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c006a950 0x7f88c006ae00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c63a6700 1 --2- 172.21.15.40:0/2662288137 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88c00b0d70 0x7f88c0157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c63a6700 1 -- 172.21.15.40:0/2662288137 >> 172.21.15.40:0/2662288137 conn(0x7f88c006d5b0 msgr2=0x7f88c006d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c63a6700 1 -- 172.21.15.40:0/2662288137 shutdown_connections 2024-09-13T17:51:30.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.146+0000 7f88c63a6700 1 -- 172.21.15.40:0/2662288137 wait complete. 2024-09-13T17:51:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.150+0000 7f88c63a6700 1 Processor -- start 2024-09-13T17:51:30.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.150+0000 7f88c63a6700 1 -- start start 2024-09-13T17:51:30.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88c63a6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88c006a950 0x7f88c01449a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88c63a6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c00b0d70 0x7f88c0143720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88c63a6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c0151750 0x7f88c0143c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88c63a6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c01503b0 con 0x7f88c0151750 2024-09-13T17:51:30.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88c63a6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c014dd10 con 0x7f88c00b0d70 2024-09-13T17:51:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88c63a6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c01ed3b0 con 0x7f88c006a950 2024-09-13T17:51:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88c4943700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c0151750 0x7f88c0143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88c4943700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c0151750 0x7f88c0143c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55270/0 (socket says 172.21.15.40:55270) 2024-09-13T17:51:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88c4943700 1 -- 172.21.15.40:0/2196208160 learned_addr learned my addr 172.21.15.40:0/2196208160 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88bf7fe700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c00b0d70 0x7f88c0143720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88bffff700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88c006a950 0x7f88c01449a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88c4943700 1 -- 172.21.15.40:0/2196208160 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88c006a950 msgr2=0x7f88c01449a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88c4943700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88c006a950 0x7f88c01449a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88c4943700 1 -- 172.21.15.40:0/2196208160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c00b0d70 msgr2=0x7f88c0143720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88c4943700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c00b0d70 0x7f88c0143720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88c4943700 1 -- 172.21.15.40:0/2196208160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b8009b30 con 0x7f88c0151750 2024-09-13T17:51:30.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88c4943700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c0151750 0x7f88c0143c60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f88c006f5d0 tx=0x7f88b8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88bcff9700 1 -- 172.21.15.40:0/2196208160 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b80048c0 con 0x7f88c0151750 2024-09-13T17:51:30.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88bcff9700 1 -- 172.21.15.40:0/2196208160 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b80099b0 con 0x7f88c0151750 2024-09-13T17:51:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88bcff9700 1 -- 172.21.15.40:0/2196208160 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b801b670 con 0x7f88c0151750 2024-09-13T17:51:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88c63a6700 1 -- 172.21.15.40:0/2196208160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c01505c0 con 0x7f88c0151750 2024-09-13T17:51:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88c63a6700 1 -- 172.21.15.40:0/2196208160 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c014e670 con 0x7f88c0151750 2024-09-13T17:51:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88c63a6700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c014dfb0 0x7f88c01ed890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.154+0000 7f88c63a6700 1 -- 172.21.15.40:0/2196208160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c0067840 con 0x7f88c014dfb0 2024-09-13T17:51:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.158+0000 7f88bffff700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c014dfb0 0x7f88c01ed890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.158+0000 7f88bffff700 1 -- 172.21.15.40:0/2196208160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88c0071240 con 0x7f88c014dfb0 2024-09-13T17:51:30.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.158+0000 7f88bcff9700 1 -- 172.21.15.40:0/2196208160 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f88b801b7d0 con 0x7f88c0151750 2024-09-13T17:51:30.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.158+0000 7f88bffff700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c014dfb0 0x7f88c01ed890 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f88b400f540 tx=0x7f88b400f900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:30.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.158+0000 7f88bcff9700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f88ac066730 0x7f88ac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.158+0000 7f88bcff9700 1 -- 172.21.15.40:0/2196208160 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f88b8085d10 con 0x7f88c0151750 2024-09-13T17:51:30.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.158+0000 7f88bf7fe700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f88ac066730 0x7f88ac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.158+0000 7f88bcff9700 1 -- 172.21.15.40:0/2196208160 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4005320 con 0x7f88c014dfb0 2024-09-13T17:51:30.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.158+0000 7f88bcff9700 1 -- 172.21.15.40:0/2196208160 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f88b4005480 con 0x7f88c014dfb0 2024-09-13T17:51:30.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.158+0000 7f88bcff9700 1 -- 172.21.15.40:0/2196208160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c014dfb0 msgr2=0x7f88c01ed890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.158+0000 7f88bcff9700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c014dfb0 0x7f88c01ed890 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f88b400f540 tx=0x7f88b400f900 comp rx=0 tx=0).stop 2024-09-13T17:51:30.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.158+0000 7f88bcff9700 1 -- 172.21.15.40:0/2196208160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c014dfb0 msgr2=0x7f88c01ed890 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:51:30.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.158+0000 7f88bcff9700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c014dfb0 0x7f88c01ed890 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.158+0000 7f88bf7fe700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f88ac066730 0x7f88ac068bf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f88c00c3cd0 tx=0x7f88b0009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:30.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.170+0000 7f88c63a6700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c00b0940 0x7f88c01eddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.170+0000 7f88c63a6700 1 -- 172.21.15.40:0/2196208160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c01ee310 con 0x7f88c00b0940 2024-09-13T17:51:30.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.170+0000 7f88bffff700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c00b0940 0x7f88c01eddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.174+0000 7f88bffff700 1 -- 172.21.15.40:0/2196208160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f88c0070f60 con 0x7f88c00b0940 2024-09-13T17:51:30.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.174+0000 7f88bffff700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c00b0940 0x7f88c01eddd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f88b400f540 tx=0x7f88b4007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:30.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.174+0000 7f88bcff9700 1 -- 172.21.15.40:0/2196208160 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4023070 con 0x7f88c00b0940 2024-09-13T17:51:30.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.174+0000 7f88bcff9700 1 -- 172.21.15.40:0/2196208160 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4947 (secure 0 0 0) 0x7f88b401c6d0 con 0x7f88c00b0940 2024-09-13T17:51:30.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.174+0000 7f88bcff9700 1 -- 172.21.15.40:0/2196208160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c00b0940 msgr2=0x7f88c01eddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.174+0000 7f88bcff9700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c00b0940 0x7f88c01eddd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f88b400f540 tx=0x7f88b4007f90 comp rx=0 tx=0).stop 2024-09-13T17:51:30.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.174+0000 7f88bcff9700 1 -- 172.21.15.40:0/2196208160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c00b0940 msgr2=0x7f88c01eddd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:51:30.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.174+0000 7f88bcff9700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c00b0940 0x7f88c01eddd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.180 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:51:30.180 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:51:30.180 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:51:30.180 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:51:30.180 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 8, 2024-09-13T17:51:30.180 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:51:30.180 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:51:30.180 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:51:30.180 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:51:30.180 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:30.180 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 80, 2024-09-13T17:51:30.181 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:51:30.181 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:51:30.181 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:51:30.181 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:51:30.181 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:51:30.181 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:51:30.181 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:51:30.181 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:51:30.181 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:51:30.181 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:51:30.181 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:51:30.181 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:51:30.181 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:51:30.182 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:30.182 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:51:30.182 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:51:30.182 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:51:30.182 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:51:30.182 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:51:30.182 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:51:30.182 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:51:30.182 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:51:30.182 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:51:30.182 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:51:30.182 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:51:30.182 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:51:30.182 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:30.183 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.183 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:51:30.183 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:51:30.183 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:51:30.183 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:51:30.183 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:51:30.183 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:51:30.183 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:51:30.183 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:51:30.183 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:51:30.183 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:51:30.183 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:51:30.183 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:51:30.184 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:51:30.184 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:51:30.184 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:51:30.184 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:51:30.184 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:51:30.184 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:51:30.184 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:51:30.184 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:51:30.184 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:51:30.184 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:51:30.184 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:51:30.184 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:51:30.184 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:51:30.185 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:51:30.185 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:51:30.185 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:30.185 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:51:30.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.185 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:51:30.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.185 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:51:30.185 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:51:30.185 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:51:30.185 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:51:30.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.185 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:51:30.186 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:51:30.186 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:30.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.186 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:51:30.186 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:51:30.186 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:30.186 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.186 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:30.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.186 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:51:30.186 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:51:30.186 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:51:30.186 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:51:30.187 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:51:30.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.187 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.187 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:51:30.187 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:51:30.187 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:51:30.187 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:51:30.187 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.187 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:51:30.187 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:51:30.187 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:30.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.187 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.188 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:51:30.188 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:51:30.188 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:30.188 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.188 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:30.188 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.188 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:51:30.188 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:51:30.188 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:51:30.188 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:51:30.188 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:51:30.188 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.188 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.188 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:51:30.189 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:51:30.189 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:51:30.189 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:51:30.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.189 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:51:30.189 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:51:30.189 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:30.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.189 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:51:30.189 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:51:30.189 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:30.189 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.190 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:30.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.190 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:51:30.190 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:51:30.190 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:51:30.190 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:51:30.190 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:51:30.190 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.190 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:30.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.190 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:51:30.190 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:51:30.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.191 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:30.191 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:30.191 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:51:30.191 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.191 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:30.191 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:51:30.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.191 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:30.191 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:30.191 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T17:51:30.191 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.191 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:30.192 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:51:30.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.192 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:30.192 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:30.192 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:51:30.192 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.192 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:30.192 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:51:30.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.192 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:30.192 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:30.192 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:51:30.192 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.193 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:30.193 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:51:30.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.193 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:30.193 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:30.193 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:51:30.193 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.193 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:30.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.193 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:51:30.193 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:51:30.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.174+0000 7f88c63a6700 1 -- 172.21.15.40:0/2196208160 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f88ac066730 msgr2=0x7f88ac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.174+0000 7f88c63a6700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f88ac066730 0x7f88ac068bf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f88c00c3cd0 tx=0x7f88b0009240 comp rx=0 tx=0).stop 2024-09-13T17:51:30.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.174+0000 7f88c63a6700 1 -- 172.21.15.40:0/2196208160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c0151750 msgr2=0x7f88c0143c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.174+0000 7f88c4943700 1 -- 172.21.15.40:0/2196208160 reap_dead start 2024-09-13T17:51:30.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.174+0000 7f88c63a6700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c0151750 0x7f88c0143c60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f88c006f5d0 tx=0x7f88b8004af0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.174+0000 7f88c63a6700 1 -- 172.21.15.40:0/2196208160 shutdown_connections 2024-09-13T17:51:30.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.174+0000 7f88c63a6700 1 --2- 172.21.15.40:0/2196208160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88c0151750 0x7f88c0143c60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.174+0000 7f88c63a6700 1 -- 172.21.15.40:0/2196208160 >> 172.21.15.40:0/2196208160 conn(0x7f88c006d5b0 msgr2=0x7f88c00c9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:30.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.174+0000 7f88c63a6700 1 -- 172.21.15.40:0/2196208160 shutdown_connections 2024-09-13T17:51:30.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.174+0000 7f88c63a6700 1 -- 172.21.15.40:0/2196208160 wait complete. 2024-09-13T17:51:30.196 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:51:30.196 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T17:51:30.196 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:51:30.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.318+0000 7fcc973c6700 1 Processor -- start 2024-09-13T17:51:30.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.318+0000 7fcc973c6700 1 -- start start 2024-09-13T17:51:30.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.318+0000 7fcc973c6700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcc9006d7b0 0x7fcc9006e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.318+0000 7fcc973c6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc9006db90 0x7fcc9014f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.318+0000 7fcc973c6700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc90155ad0 con 0x7fcc9006eab0 2024-09-13T17:51:30.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.318+0000 7fcc973c6700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc90155db0 con 0x7fcc9006d7b0 2024-09-13T17:51:30.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.318+0000 7fcc973c6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc90156090 con 0x7fcc9006db90 2024-09-13T17:51:30.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.318+0000 7fcc95bc3700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcc9006eab0 0x7fcc90071430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44592/0 (socket says 172.21.15.40:44592) 2024-09-13T17:51:30.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.318+0000 7fcc95bc3700 1 -- 172.21.15.40:0/2253346443 learned_addr learned my addr 172.21.15.40:0/2253346443 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:30.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.318+0000 7fcc963c4700 1 --2- 172.21.15.40:0/2253346443 >> v2:172.21.15.43:3300/0 conn(0x7fcc9006d7b0 0x7fcc9006e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.318+0000 7fcc96bc5700 1 --2- 172.21.15.40:0/2253346443 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc9006db90 0x7fcc9014f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc953c2700 1 -- 172.21.15.40:0/2253346443 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2371321640 0 0) 0x7fcc90155ad0 con 0x7fcc9006eab0 2024-09-13T17:51:30.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc953c2700 1 -- 172.21.15.40:0/2253346443 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc7c0035d0 con 0x7fcc9006eab0 2024-09-13T17:51:30.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc96bc5700 1 -- 172.21.15.40:0/2253346443 >> v2:172.21.15.43:3300/0 conn(0x7fcc9006d7b0 msgr2=0x7fcc9006e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:30.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc96bc5700 1 --2- 172.21.15.40:0/2253346443 >> v2:172.21.15.43:3300/0 conn(0x7fcc9006d7b0 0x7fcc9006e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc96bc5700 1 -- 172.21.15.40:0/2253346443 >> v1:172.21.15.40:6789/0 conn(0x7fcc9006eab0 legacy=0x7fcc90071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc96bc5700 1 -- 172.21.15.40:0/2253346443 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc90156370 con 0x7fcc9006db90 2024-09-13T17:51:30.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc963c4700 1 --2- 172.21.15.40:0/2253346443 >> v2:172.21.15.43:3300/0 conn(0x7fcc9006d7b0 0x7fcc9006e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:30.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc96bc5700 1 --2- 172.21.15.40:0/2253346443 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc9006db90 0x7fcc9014f5d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009d90 tx=0x7fcc8800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cb1280132af167d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:30.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc953c2700 1 -- 172.21.15.40:0/2253346443 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8800e070 con 0x7fcc9006db90 2024-09-13T17:51:30.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc953c2700 1 -- 172.21.15.40:0/2253346443 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8800bb20 con 0x7fcc9006db90 2024-09-13T17:51:30.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc953c2700 1 -- 172.21.15.40:0/2253346443 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88012430 con 0x7fcc9006db90 2024-09-13T17:51:30.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc973c6700 1 -- 172.21.15.40:0/2253346443 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc9006db90 msgr2=0x7fcc9014f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc973c6700 1 --2- 172.21.15.40:0/2253346443 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc9006db90 0x7fcc9014f5d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009d90 tx=0x7fcc8800b330 comp rx=0 tx=0).stop 2024-09-13T17:51:30.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc973c6700 1 -- 172.21.15.40:0/2253346443 shutdown_connections 2024-09-13T17:51:30.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc973c6700 1 --2- 172.21.15.40:0/2253346443 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc9006db90 0x7fcc9014f5d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc973c6700 1 --2- 172.21.15.40:0/2253346443 >> v2:172.21.15.43:3300/0 conn(0x7fcc9006d7b0 0x7fcc9006e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc973c6700 1 -- 172.21.15.40:0/2253346443 >> 172.21.15.40:0/2253346443 conn(0x7fcc900cb450 msgr2=0x7fcc900cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:30.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc973c6700 1 -- 172.21.15.40:0/2253346443 shutdown_connections 2024-09-13T17:51:30.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc973c6700 1 -- 172.21.15.40:0/2253346443 wait complete. 2024-09-13T17:51:30.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc973c6700 1 Processor -- start 2024-09-13T17:51:30.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.322+0000 7fcc973c6700 1 -- start start 2024-09-13T17:51:30.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc973c6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc9006d7b0 0x7fcc900c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc973c6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc9006db90 0x7fcc9014f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc973c6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc9006eab0 0x7fcc901f2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc973c6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc901e0f90 con 0x7fcc9006db90 2024-09-13T17:51:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc973c6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc90157f30 con 0x7fcc9006d7b0 2024-09-13T17:51:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc973c6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc900c3f50 con 0x7fcc9006eab0 2024-09-13T17:51:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc95bc3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc9006db90 0x7fcc9014f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc95bc3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc9006db90 0x7fcc9014f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55276/0 (socket says 172.21.15.40:55276) 2024-09-13T17:51:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc95bc3700 1 -- 172.21.15.40:0/1173417595 learned_addr learned my addr 172.21.15.40:0/1173417595 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc963c4700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc9006d7b0 0x7fcc900c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc96bc5700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc9006eab0 0x7fcc901f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc95bc3700 1 -- 172.21.15.40:0/1173417595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc9006eab0 msgr2=0x7fcc901f2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc95bc3700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc9006eab0 0x7fcc901f2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc95bc3700 1 -- 172.21.15.40:0/1173417595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc9006d7b0 msgr2=0x7fcc900c3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc95bc3700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc9006d7b0 0x7fcc900c3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc95bc3700 1 -- 172.21.15.40:0/1173417595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8c00a650 con 0x7fcc9006db90 2024-09-13T17:51:30.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc95bc3700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc9006db90 0x7fcc9014f820 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000c530 tx=0x7fcc8000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:30.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc86ffd700 1 -- 172.21.15.40:0/1173417595 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc80005d90 con 0x7fcc9006db90 2024-09-13T17:51:30.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc86ffd700 1 -- 172.21.15.40:0/1173417595 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc800070d0 con 0x7fcc9006db90 2024-09-13T17:51:30.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc973c6700 1 -- 172.21.15.40:0/1173417595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc88009a70 con 0x7fcc9006db90 2024-09-13T17:51:30.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc86ffd700 1 -- 172.21.15.40:0/1173417595 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc80007480 con 0x7fcc9006db90 2024-09-13T17:51:30.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc973c6700 1 -- 172.21.15.40:0/1173417595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc901dfe70 con 0x7fcc9006db90 2024-09-13T17:51:30.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc973c6700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc900c4680 0x7fcc901f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc973c6700 1 -- 172.21.15.40:0/1173417595 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc900675d0 con 0x7fcc900c4680 2024-09-13T17:51:30.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.326+0000 7fcc963c4700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc900c4680 0x7fcc901f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.330+0000 7fcc963c4700 1 -- 172.21.15.40:0/1173417595 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc901f2370 con 0x7fcc900c4680 2024-09-13T17:51:30.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.330+0000 7fcc963c4700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc900c4680 0x7fcc901f1e30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c011a20 tx=0x7fcc8c011de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:30.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.330+0000 7fcc86ffd700 1 -- 172.21.15.40:0/1173417595 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c00fc20 con 0x7fcc900c4680 2024-09-13T17:51:30.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.330+0000 7fcc86ffd700 1 -- 172.21.15.40:0/1173417595 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcc800075e0 con 0x7fcc9006db90 2024-09-13T17:51:30.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.330+0000 7fcc86ffd700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcc74066770 0x7fcc74068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.330+0000 7fcc86ffd700 1 -- 172.21.15.40:0/1173417595 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcc80083990 con 0x7fcc9006db90 2024-09-13T17:51:30.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.330+0000 7fcc96bc5700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcc74066770 0x7fcc74068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.330+0000 7fcc86ffd700 1 -- 172.21.15.40:0/1173417595 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fcc8c00fd80 con 0x7fcc900c4680 2024-09-13T17:51:30.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.330+0000 7fcc86ffd700 1 -- 172.21.15.40:0/1173417595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc900c4680 msgr2=0x7fcc901f1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.330+0000 7fcc86ffd700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc900c4680 0x7fcc901f1e30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c011a20 tx=0x7fcc8c011de0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.330+0000 7fcc86ffd700 1 -- 172.21.15.40:0/1173417595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc900c4680 msgr2=0x7fcc901f1e30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:51:30.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.330+0000 7fcc86ffd700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc900c4680 0x7fcc901f1e30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.330+0000 7fcc96bc5700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcc74066770 0x7fcc74068c30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcc9006dfa0 tx=0x7fcc88011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:30.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.342+0000 7fcc973c6700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc901e0060 0x7fcc901469f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.342+0000 7fcc973c6700 1 -- 172.21.15.40:0/1173417595 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc90070ea0 con 0x7fcc901e0060 2024-09-13T17:51:30.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.342+0000 7fcc963c4700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc901e0060 0x7fcc901469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.342+0000 7fcc963c4700 1 -- 172.21.15.40:0/1173417595 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcc900c7940 con 0x7fcc901e0060 2024-09-13T17:51:30.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.342+0000 7fcc963c4700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc901e0060 0x7fcc901469f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c011a20 tx=0x7fcc8c004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:30.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.346+0000 7fcc86ffd700 1 -- 172.21.15.40:0/1173417595 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c020040 con 0x7fcc901e0060 2024-09-13T17:51:30.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.346+0000 7fcc86ffd700 1 -- 172.21.15.40:0/1173417595 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4781 (secure 0 0 0) 0x7fcc8c025070 con 0x7fcc901e0060 2024-09-13T17:51:30.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.346+0000 7fcc86ffd700 1 -- 172.21.15.40:0/1173417595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc901e0060 msgr2=0x7fcc901469f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.346+0000 7fcc86ffd700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc901e0060 0x7fcc901469f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c011a20 tx=0x7fcc8c004770 comp rx=0 tx=0).stop 2024-09-13T17:51:30.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.346+0000 7fcc86ffd700 1 -- 172.21.15.40:0/1173417595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc901e0060 msgr2=0x7fcc901469f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:51:30.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.346+0000 7fcc86ffd700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc901e0060 0x7fcc901469f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.352 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:51:30.352 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:51:30.352 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:51:30.352 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:51:30.353 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 8, 2024-09-13T17:51:30.353 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:51:30.353 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:51:30.353 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:51:30.353 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:51:30.353 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:30.354 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 80, 2024-09-13T17:51:30.354 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:51:30.354 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:51:30.354 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:51:30.354 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:51:30.354 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:51:30.355 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:51:30.355 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:51:30.355 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:51:30.355 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:51:30.355 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:51:30.355 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:51:30.356 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:51:30.356 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:51:30.356 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:30.356 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:51:30.356 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:51:30.356 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:51:30.357 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:51:30.357 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:51:30.357 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:51:30.357 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:51:30.357 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:51:30.357 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:51:30.358 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:51:30.358 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:51:30.358 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:51:30.358 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:30.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.358 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:51:30.359 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:51:30.359 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:51:30.359 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:51:30.359 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:51:30.359 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:51:30.360 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:51:30.360 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:51:30.360 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:51:30.360 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:51:30.360 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:51:30.360 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:51:30.360 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:51:30.361 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:51:30.361 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:51:30.361 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:51:30.361 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:51:30.361 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:51:30.362 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:51:30.362 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:51:30.362 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:51:30.362 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:51:30.362 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:51:30.362 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:51:30.362 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:51:30.363 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:51:30.363 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:51:30.363 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:30.363 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:51:30.363 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.363 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:51:30.364 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.364 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:51:30.364 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:51:30.364 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:51:30.364 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:51:30.365 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.365 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:51:30.365 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:51:30.365 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:30.365 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.365 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.366 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:51:30.366 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:51:30.366 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:30.366 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.366 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:30.366 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.367 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:51:30.367 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:51:30.367 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:51:30.367 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:51:30.367 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:51:30.367 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.368 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.368 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:51:30.368 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:51:30.368 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:51:30.368 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:51:30.369 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.369 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:51:30.369 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:51:30.369 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:30.369 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.369 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.370 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:51:30.370 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:51:30.370 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:30.370 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.370 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:30.370 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.371 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:51:30.371 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:51:30.371 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:51:30.371 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:51:30.371 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:51:30.371 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.372 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.372 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:51:30.372 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:51:30.372 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:51:30.372 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:51:30.373 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.373 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:51:30.373 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:51:30.373 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:30.373 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.373 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.374 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:51:30.374 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:51:30.374 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:30.374 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.374 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:30.374 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.375 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:51:30.375 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:51:30.375 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:51:30.375 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:51:30.375 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:51:30.376 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.376 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:30.376 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.376 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:51:30.376 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:51:30.376 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.376 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:30.377 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:30.377 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:51:30.377 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.377 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:30.377 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:51:30.377 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.378 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:30.378 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:30.378 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:51:30.378 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.378 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:30.378 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:51:30.379 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.379 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:30.379 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:30.379 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:51:30.379 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.379 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:30.380 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:51:30.380 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:30.380 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:30.380 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:30.380 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:51:30.381 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:30.381 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:30.381 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:30.381 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:51:30.381 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:51:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.346+0000 7fcc973c6700 1 -- 172.21.15.40:0/1173417595 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcc74066770 msgr2=0x7fcc74068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.346+0000 7fcc973c6700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcc74066770 0x7fcc74068c30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcc9006dfa0 tx=0x7fcc88011000 comp rx=0 tx=0).stop 2024-09-13T17:51:30.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.346+0000 7fcc973c6700 1 -- 172.21.15.40:0/1173417595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc9006db90 msgr2=0x7fcc9014f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.346+0000 7fcc973c6700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc9006db90 0x7fcc9014f820 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000c530 tx=0x7fcc8000c840 comp rx=0 tx=0).stop 2024-09-13T17:51:30.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.346+0000 7fcc96bc5700 1 -- 172.21.15.40:0/1173417595 reap_dead start 2024-09-13T17:51:30.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.346+0000 7fcc973c6700 1 -- 172.21.15.40:0/1173417595 shutdown_connections 2024-09-13T17:51:30.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.346+0000 7fcc973c6700 1 --2- 172.21.15.40:0/1173417595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc9006db90 0x7fcc9014f820 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.346+0000 7fcc973c6700 1 -- 172.21.15.40:0/1173417595 >> 172.21.15.40:0/1173417595 conn(0x7fcc900cb450 msgr2=0x7fcc901df630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:30.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.346+0000 7fcc973c6700 1 -- 172.21.15.40:0/1173417595 shutdown_connections 2024-09-13T17:51:30.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.346+0000 7fcc973c6700 1 -- 172.21.15.40:0/1173417595 wait complete. 2024-09-13T17:51:30.385 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:51:30.386 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-13T17:51:30.386 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T17:51:30.386 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-13T17:51:30.386 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T17:51:30.387 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T17:51:30.387 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:51:30.422 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T17:51:30.423 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T17:51:30.423 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:51:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.578+0000 7f4ebfaa6700 1 Processor -- start 2024-09-13T17:51:30.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.578+0000 7f4ebfaa6700 1 -- start start 2024-09-13T17:51:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.582+0000 7f4ebfaa6700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4eb8070c30 0x7f4eb80c8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.582+0000 7f4ebfaa6700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb80c6870 con 0x7f4eb806a600 2024-09-13T17:51:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.582+0000 7f4ebd041700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f4eb8070c30 msgr2=0x7f4eb80c8320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:51:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.582+0000 7f4ebfaa6700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb80c6e30 con 0x7f4eb806a9e0 2024-09-13T17:51:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.582+0000 7f4ebfaa6700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb806b210 con 0x7f4eb8070c30 2024-09-13T17:51:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.582+0000 7f4ebd842700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f4eb806a600 legacy=0x7f4eb81514e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:51:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.582+0000 7f4ebd041700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4eb8070c30 0x7f4eb80c8320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:51:30.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.582+0000 7f4ebe043700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4eb806a9e0 0x7f4eb806cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:37166/0 (socket says 172.21.15.40:37166) 2024-09-13T17:51:30.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.582+0000 7f4ebe043700 1 -- 172.21.15.40:0/185744294 learned_addr learned my addr 172.21.15.40:0/185744294 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:30.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.582+0000 7f4ebc840700 1 -- 172.21.15.40:0/185744294 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 827919073 0 0) 0x7f4eb80c6e30 con 0x7f4eb806a9e0 2024-09-13T17:51:30.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.582+0000 7f4ebc840700 1 -- 172.21.15.40:0/185744294 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ea40035d0 con 0x7f4eb806a9e0 2024-09-13T17:51:30.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.582+0000 7f4ebc840700 1 -- 172.21.15.40:0/185744294 <== mon.1 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 600560015 0 0) 0x7f4ea40035d0 con 0x7f4eb806a9e0 2024-09-13T17:51:30.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.582+0000 7f4ebc840700 1 -- 172.21.15.40:0/185744294 >> v2:172.21.15.40:3300/0 conn(0x7f4eb8070c30 msgr2=0x7f4eb80c8320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:51:30.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.582+0000 7f4ebc840700 1 --2- 172.21.15.40:0/185744294 >> v2:172.21.15.40:3300/0 conn(0x7f4eb8070c30 0x7f4eb80c8320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.582+0000 7f4ebc840700 1 -- 172.21.15.40:0/185744294 >> v1:172.21.15.40:6789/0 conn(0x7f4eb806a600 legacy=0x7f4eb81514e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:51:30.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.582+0000 7f4ebc840700 1 -- 172.21.15.40:0/185744294 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb8158160 con 0x7f4eb806a9e0 2024-09-13T17:51:30.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.582+0000 7f4ebc840700 1 -- 172.21.15.40:0/185744294 <== mon.1 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f4eb00019f0 con 0x7f4eb806a9e0 2024-09-13T17:51:30.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebc840700 1 -- 172.21.15.40:0/185744294 >> v1:172.21.15.43:6789/0 conn(0x7f4eb806a9e0 legacy=0x7f4eb806cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebc840700 1 --2- 172.21.15.40:0/185744294 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4ea4003a20 0x7f4ea4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebc840700 1 --2- 172.21.15.40:0/185744294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ea4004870 0x7f4ea4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebc840700 1 --2- 172.21.15.40:0/185744294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea4005350 0x7f4ea400fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebc840700 1 -- 172.21.15.40:0/185744294 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb806b210 con 0x7f4ea4004870 2024-09-13T17:51:30.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebc840700 1 -- 172.21.15.40:0/185744294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea4010660 con 0x7f4ea4005350 2024-09-13T17:51:30.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebc840700 1 -- 172.21.15.40:0/185744294 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea4010820 con 0x7f4ea4003a20 2024-09-13T17:51:30.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebd041700 1 -- 172.21.15.40:0/185744294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ea4004870 msgr2=0x7f4ea4004cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:51:30.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebd041700 1 --2- 172.21.15.40:0/185744294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ea4004870 0x7f4ea4004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:51:30.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebd842700 1 --2- 172.21.15.40:0/185744294 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4ea4003a20 0x7f4ea4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebe043700 1 --2- 172.21.15.40:0/185744294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea4005350 0x7f4ea400fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebd842700 1 -- 172.21.15.40:0/185744294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea4005350 msgr2=0x7f4ea400fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:30.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebd842700 1 --2- 172.21.15.40:0/185744294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ea4005350 0x7f4ea400fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebd842700 1 -- 172.21.15.40:0/185744294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ea4004870 msgr2=0x7f4ea4004cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:51:30.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebd842700 1 --2- 172.21.15.40:0/185744294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ea4004870 0x7f4ea4004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebd842700 1 -- 172.21.15.40:0/185744294 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea40035d0 con 0x7f4ea4003a20 2024-09-13T17:51:30.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebe043700 1 -- 172.21.15.40:0/185744294 reap_dead start 2024-09-13T17:51:30.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebd842700 1 --2- 172.21.15.40:0/185744294 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4ea4003a20 0x7f4ea4003e20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4003a30 tx=0x7f4ea40058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae05f307591a254 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:30.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebc840700 1 -- 172.21.15.40:0/185744294 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb40098d0 con 0x7f4ea4003a20 2024-09-13T17:51:30.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebc840700 1 -- 172.21.15.40:0/185744294 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb400bd60 con 0x7f4ea4003a20 2024-09-13T17:51:30.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.586+0000 7f4ebc840700 1 -- 172.21.15.40:0/185744294 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb4003ca0 con 0x7f4ea4003a20 2024-09-13T17:51:30.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebfaa6700 1 -- 172.21.15.40:0/185744294 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4ea4003a20 msgr2=0x7f4ea4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebfaa6700 1 --2- 172.21.15.40:0/185744294 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4ea4003a20 0x7f4ea4003e20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4003a30 tx=0x7f4ea40058a0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebfaa6700 1 -- 172.21.15.40:0/185744294 shutdown_connections 2024-09-13T17:51:30.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebfaa6700 1 --2- 172.21.15.40:0/185744294 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4ea4003a20 0x7f4ea4003e20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebfaa6700 1 -- 172.21.15.40:0/185744294 >> 172.21.15.40:0/185744294 conn(0x7f4eb806e620 msgr2=0x7f4eb806ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:30.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebfaa6700 1 -- 172.21.15.40:0/185744294 shutdown_connections 2024-09-13T17:51:30.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebfaa6700 1 -- 172.21.15.40:0/185744294 wait complete. 2024-09-13T17:51:30.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebfaa6700 1 Processor -- start 2024-09-13T17:51:30.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebfaa6700 1 -- start start 2024-09-13T17:51:30.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebfaa6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4eb81e0b70 0x7f4eb81edd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebfaa6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4eb81de6d0 0x7f4eb81ee260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebfaa6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eb8148670 0x7f4eb814f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebfaa6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb81568a0 con 0x7f4eb81e0b70 2024-09-13T17:51:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebfaa6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb81ee930 con 0x7f4eb8148670 2024-09-13T17:51:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebfaa6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb81eebf0 con 0x7f4eb81de6d0 2024-09-13T17:51:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebd842700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4eb81e0b70 msgr2=0x7f4eb81edd20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:51:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebd842700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4eb81e0b70 0x7f4eb81edd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:51:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebd041700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4eb81de6d0 0x7f4eb81ee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebd041700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4eb81de6d0 0x7f4eb81ee260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:60018/0 (socket says 172.21.15.40:60018) 2024-09-13T17:51:30.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebd041700 1 -- 172.21.15.40:0/769129122 learned_addr learned my addr 172.21.15.40:0/769129122 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:30.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.590+0000 7f4ebe043700 1 --2- 172.21.15.40:0/769129122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eb8148670 0x7f4eb814f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.594+0000 7f4ebd041700 1 -- 172.21.15.40:0/769129122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eb8148670 msgr2=0x7f4eb814f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.594+0000 7f4ebd041700 1 --2- 172.21.15.40:0/769129122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eb8148670 0x7f4eb814f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.594+0000 7f4ebd041700 1 -- 172.21.15.40:0/769129122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4eb81e0b70 msgr2=0x7f4eb81edd20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:51:30.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.594+0000 7f4ebd041700 1 --2- 172.21.15.40:0/769129122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4eb81e0b70 0x7f4eb81edd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.594+0000 7f4ebd041700 1 -- 172.21.15.40:0/769129122 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb80c3fc0 con 0x7f4eb81de6d0 2024-09-13T17:51:30.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.594+0000 7f4ebd041700 1 --2- 172.21.15.40:0/769129122 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4eb81de6d0 0x7f4eb81ee260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800b780 tx=0x7f4ea800bb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:30.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.594+0000 7f4eae7fc700 1 -- 172.21.15.40:0/769129122 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8005070 con 0x7f4eb81de6d0 2024-09-13T17:51:30.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.594+0000 7f4eae7fc700 1 -- 172.21.15.40:0/769129122 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea80051f0 con 0x7f4eb81de6d0 2024-09-13T17:51:30.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.594+0000 7f4ebfaa6700 1 -- 172.21.15.40:0/769129122 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb80c42a0 con 0x7f4eb81de6d0 2024-09-13T17:51:30.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.594+0000 7f4eae7fc700 1 -- 172.21.15.40:0/769129122 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80055a0 con 0x7f4eb81de6d0 2024-09-13T17:51:30.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.594+0000 7f4ebfaa6700 1 -- 172.21.15.40:0/769129122 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb81e0f50 con 0x7f4eb81de6d0 2024-09-13T17:51:30.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.594+0000 7f4ebfaa6700 1 -- 172.21.15.40:0/769129122 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb80c7050 con 0x7f4eb81de6d0 2024-09-13T17:51:30.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.594+0000 7f4eae7fc700 1 -- 172.21.15.40:0/769129122 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4ea8005700 con 0x7f4eb81de6d0 2024-09-13T17:51:30.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.594+0000 7f4eae7fc700 1 --2- 172.21.15.40:0/769129122 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4e9c066710 0x7f4e9c068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:30.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.594+0000 7f4eae7fc700 1 -- 172.21.15.40:0/769129122 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4ea8083f30 con 0x7f4eb81de6d0 2024-09-13T17:51:30.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.594+0000 7f4ebd842700 1 --2- 172.21.15.40:0/769129122 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4e9c066710 0x7f4e9c068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:30.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.598+0000 7f4ebd842700 1 --2- 172.21.15.40:0/769129122 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4e9c066710 0x7f4e9c068bd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4eb80c3160 tx=0x7f4eb4019040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:30.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.602+0000 7f4eae7fc700 1 -- 172.21.15.40:0/769129122 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4ea8050f70 con 0x7f4eb81de6d0 2024-09-13T17:51:30.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.838+0000 7f4ebfaa6700 1 -- 172.21.15.40:0/769129122 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4eb80c5fb0 con 0x7f4eb81de6d0 2024-09-13T17:51:30.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.838+0000 7f4eae7fc700 1 -- 172.21.15.40:0/769129122 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7f4ea8054590 con 0x7f4eb81de6d0 2024-09-13T17:51:30.845 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":81,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:51:30.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.842+0000 7f4ebfaa6700 1 -- 172.21.15.40:0/769129122 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4e9c066710 msgr2=0x7f4e9c068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.842+0000 7f4ebfaa6700 1 --2- 172.21.15.40:0/769129122 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4e9c066710 0x7f4e9c068bd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4eb80c3160 tx=0x7f4eb4019040 comp rx=0 tx=0).stop 2024-09-13T17:51:30.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.842+0000 7f4ebfaa6700 1 -- 172.21.15.40:0/769129122 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4eb81de6d0 msgr2=0x7f4eb81ee260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:30.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.842+0000 7f4ebfaa6700 1 --2- 172.21.15.40:0/769129122 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4eb81de6d0 0x7f4eb81ee260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4ea800b780 tx=0x7f4ea800bb40 comp rx=0 tx=0).stop 2024-09-13T17:51:30.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.842+0000 7f4ebfaa6700 1 -- 172.21.15.40:0/769129122 shutdown_connections 2024-09-13T17:51:30.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.842+0000 7f4ebfaa6700 1 --2- 172.21.15.40:0/769129122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4eb8148670 0x7f4eb814f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.842+0000 7f4ebfaa6700 1 --2- 172.21.15.40:0/769129122 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4e9c066710 0x7f4e9c068bd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.842+0000 7f4ebfaa6700 1 --2- 172.21.15.40:0/769129122 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4eb81de6d0 0x7f4eb81ee260 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.842+0000 7f4ebfaa6700 1 --2- 172.21.15.40:0/769129122 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4eb81e0b70 0x7f4eb81edd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:30.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.842+0000 7f4ebfaa6700 1 -- 172.21.15.40:0/769129122 >> 172.21.15.40:0/769129122 conn(0x7f4eb806e620 msgr2=0x7f4eb80c72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:30.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.842+0000 7f4ebfaa6700 1 -- 172.21.15.40:0/769129122 shutdown_connections 2024-09-13T17:51:30.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:30.842+0000 7f4ebfaa6700 1 -- 172.21.15.40:0/769129122 wait complete. 2024-09-13T17:51:33.650 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:51:33.652 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:51:33.861 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:51:33.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.974+0000 7fbe92372700 1 Processor -- start 2024-09-13T17:51:33.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.978+0000 7fbe92372700 1 -- start start 2024-09-13T17:51:33.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.978+0000 7fbe92372700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe8c0c55a0 0x7fbe8c06ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:33.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.978+0000 7fbe92372700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbe8c0b1d60 0x7fbe8c15aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:33.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.978+0000 7fbe92372700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8c071100 con 0x7fbe8c0c5980 2024-09-13T17:51:33.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.978+0000 7fbe92372700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe8c06f1a0 con 0x7fbe8c0b1d60 2024-09-13T17:51:33.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.978+0000 7fbe92372700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c070b80 con 0x7fbe8c0c55a0 2024-09-13T17:51:33.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.978+0000 7fbe90b6f700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7fbe8c0b1d60 msgr2=0x7fbe8c15aa30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:51:33.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.978+0000 7fbe91b71700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7fbe8c0c5980 legacy=0x7fbe8c150ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:51:33.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.978+0000 7fbe90b6f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbe8c0b1d60 0x7fbe8c15aa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:51:33.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.978+0000 7fbe91370700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe8c0c55a0 0x7fbe8c06ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:33.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.978+0000 7fbe91370700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe8c0c55a0 0x7fbe8c06ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:47926/0 (socket says 172.21.15.40:47926) 2024-09-13T17:51:33.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.978+0000 7fbe91370700 1 -- 172.21.15.40:0/925336230 learned_addr learned my addr 172.21.15.40:0/925336230 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:33.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.978+0000 7fbe91370700 1 -- 172.21.15.40:0/925336230 >> v2:172.21.15.40:3300/0 conn(0x7fbe8c0b1d60 msgr2=0x7fbe8c15aa30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:51:33.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.978+0000 7fbe91370700 1 --2- 172.21.15.40:0/925336230 >> v2:172.21.15.40:3300/0 conn(0x7fbe8c0b1d60 0x7fbe8c15aa30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:33.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.978+0000 7fbe91370700 1 -- 172.21.15.40:0/925336230 >> v1:172.21.15.40:6789/0 conn(0x7fbe8c0c5980 legacy=0x7fbe8c150ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:51:33.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.978+0000 7fbe91370700 1 -- 172.21.15.40:0/925336230 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c070e20 con 0x7fbe8c0c55a0 2024-09-13T17:51:33.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.978+0000 7fbe91370700 1 --2- 172.21.15.40:0/925336230 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe8c0c55a0 0x7fbe8c06ec60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c009910 tx=0x7fbe7c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bbdbab57b6533d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:33.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.978+0000 7fbe83fff700 1 -- 172.21.15.40:0/925336230 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c004230 con 0x7fbe8c0c55a0 2024-09-13T17:51:33.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.978+0000 7fbe83fff700 1 -- 172.21.15.40:0/925336230 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7c004390 con 0x7fbe8c0c55a0 2024-09-13T17:51:33.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.978+0000 7fbe83fff700 1 -- 172.21.15.40:0/925336230 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c009e60 con 0x7fbe8c0c55a0 2024-09-13T17:51:33.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe92372700 1 -- 172.21.15.40:0/925336230 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe8c0c55a0 msgr2=0x7fbe8c06ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:33.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe92372700 1 --2- 172.21.15.40:0/925336230 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe8c0c55a0 0x7fbe8c06ec60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c009910 tx=0x7fbe7c009c20 comp rx=0 tx=0).stop 2024-09-13T17:51:33.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe92372700 1 -- 172.21.15.40:0/925336230 shutdown_connections 2024-09-13T17:51:33.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe92372700 1 --2- 172.21.15.40:0/925336230 >> v2:172.21.15.40:3300/0 conn(0x7fbe8c0b1d60 0x7fbe8c15aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:33.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe92372700 1 --2- 172.21.15.40:0/925336230 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe8c0c55a0 0x7fbe8c06ec60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:33.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe92372700 1 -- 172.21.15.40:0/925336230 >> 172.21.15.40:0/925336230 conn(0x7fbe8c0cbd80 msgr2=0x7fbe8c0cc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:33.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe92372700 1 -- 172.21.15.40:0/925336230 shutdown_connections 2024-09-13T17:51:33.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe92372700 1 -- 172.21.15.40:0/925336230 wait complete. 2024-09-13T17:51:33.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe92372700 1 Processor -- start 2024-09-13T17:51:33.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe92372700 1 -- start start 2024-09-13T17:51:33.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe92372700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe8c0b1d60 0x7fbe8c06b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:33.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe92372700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0c55a0 0x7fbe8c155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:33.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe92372700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe8c0c5980 0x7fbe8c1563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:33.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe92372700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c155760 con 0x7fbe8c0b1d60 2024-09-13T17:51:33.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe92372700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c1f2af0 con 0x7fbe8c0c55a0 2024-09-13T17:51:33.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe92372700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c1f3670 con 0x7fbe8c0c5980 2024-09-13T17:51:33.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe91370700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe8c0b1d60 msgr2=0x7fbe8c06b740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:51:33.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe91370700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe8c0b1d60 0x7fbe8c06b740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:51:33.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe91b71700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe8c0c5980 0x7fbe8c1563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:33.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe90b6f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0c55a0 0x7fbe8c155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:33.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe91b71700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe8c0c5980 0x7fbe8c1563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:47940/0 (socket says 172.21.15.40:47940) 2024-09-13T17:51:33.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe90b6f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0c55a0 0x7fbe8c155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:40400/0 (socket says 172.21.15.40:40400) 2024-09-13T17:51:33.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.982+0000 7fbe91b71700 1 -- 172.21.15.40:0/4120269758 learned_addr learned my addr 172.21.15.40:0/4120269758 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:33.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.986+0000 7fbe91b71700 1 -- 172.21.15.40:0/4120269758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0c55a0 msgr2=0x7fbe8c155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:33.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.986+0000 7fbe91b71700 1 --2- 172.21.15.40:0/4120269758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0c55a0 0x7fbe8c155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:33.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.986+0000 7fbe91b71700 1 -- 172.21.15.40:0/4120269758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe8c0b1d60 msgr2=0x7fbe8c06b740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:51:33.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.986+0000 7fbe91b71700 1 --2- 172.21.15.40:0/4120269758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe8c0b1d60 0x7fbe8c06b740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:33.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.986+0000 7fbe91b71700 1 -- 172.21.15.40:0/4120269758 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7c009570 con 0x7fbe8c0c5980 2024-09-13T17:51:33.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.986+0000 7fbe90b6f700 1 --2- 172.21.15.40:0/4120269758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0c55a0 0x7fbe8c155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:33.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.986+0000 7fbe91b71700 1 --2- 172.21.15.40:0/4120269758 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe8c0c5980 0x7fbe8c1563d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400be90 tx=0x7fbe84009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:33.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.986+0000 7fbe81ffb700 1 -- 172.21.15.40:0/4120269758 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8400cd10 con 0x7fbe8c0c5980 2024-09-13T17:51:33.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.986+0000 7fbe81ffb700 1 -- 172.21.15.40:0/4120269758 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe84015470 con 0x7fbe8c0c5980 2024-09-13T17:51:33.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.986+0000 7fbe81ffb700 1 -- 172.21.15.40:0/4120269758 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84015820 con 0x7fbe8c0c5980 2024-09-13T17:51:33.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.986+0000 7fbe92372700 1 -- 172.21.15.40:0/4120269758 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8c155950 con 0x7fbe8c0c5980 2024-09-13T17:51:33.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.986+0000 7fbe92372700 1 -- 172.21.15.40:0/4120269758 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8c1f2d40 con 0x7fbe8c0c5980 2024-09-13T17:51:33.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.986+0000 7fbe92372700 1 -- 172.21.15.40:0/4120269758 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe8c071530 con 0x7fbe8c0c5980 2024-09-13T17:51:33.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.990+0000 7fbe81ffb700 1 -- 172.21.15.40:0/4120269758 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbe84012070 con 0x7fbe8c0c5980 2024-09-13T17:51:33.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.990+0000 7fbe81ffb700 1 --2- 172.21.15.40:0/4120269758 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbe700666e0 0x7fbe70068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:33.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.990+0000 7fbe81ffb700 1 -- 172.21.15.40:0/4120269758 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbe84084b00 con 0x7fbe8c0c5980 2024-09-13T17:51:33.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.990+0000 7fbe91370700 1 --2- 172.21.15.40:0/4120269758 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbe700666e0 0x7fbe70068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:33.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.990+0000 7fbe91370700 1 --2- 172.21.15.40:0/4120269758 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbe700666e0 0x7fbe70068ba0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c009910 tx=0x7fbe7c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:34.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:33.998+0000 7fbe81ffb700 1 -- 172.21.15.40:0/4120269758 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbe84051aa0 con 0x7fbe8c0c5980 2024-09-13T17:51:34.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:34.230+0000 7fbe92372700 1 -- 172.21.15.40:0/4120269758 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbe8c14e950 con 0x7fbe8c0c5980 2024-09-13T17:51:34.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:34.230+0000 7fbe81ffb700 1 -- 172.21.15.40:0/4120269758 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7fbe8401a070 con 0x7fbe8c0c5980 2024-09-13T17:51:34.236 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":84,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:51:34.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:34.234+0000 7fbe92372700 1 -- 172.21.15.40:0/4120269758 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbe700666e0 msgr2=0x7fbe70068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:34.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:34.234+0000 7fbe92372700 1 --2- 172.21.15.40:0/4120269758 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbe700666e0 0x7fbe70068ba0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c009910 tx=0x7fbe7c008030 comp rx=0 tx=0).stop 2024-09-13T17:51:34.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:34.234+0000 7fbe92372700 1 -- 172.21.15.40:0/4120269758 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe8c0c5980 msgr2=0x7fbe8c1563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:34.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:34.234+0000 7fbe92372700 1 --2- 172.21.15.40:0/4120269758 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe8c0c5980 0x7fbe8c1563d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400be90 tx=0x7fbe84009f30 comp rx=0 tx=0).stop 2024-09-13T17:51:34.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:34.234+0000 7fbe92372700 1 -- 172.21.15.40:0/4120269758 shutdown_connections 2024-09-13T17:51:34.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:34.234+0000 7fbe92372700 1 --2- 172.21.15.40:0/4120269758 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbe700666e0 0x7fbe70068ba0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:34.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:34.234+0000 7fbe92372700 1 --2- 172.21.15.40:0/4120269758 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe8c0c5980 0x7fbe8c1563d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:34.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:34.234+0000 7fbe92372700 1 --2- 172.21.15.40:0/4120269758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0c55a0 0x7fbe8c155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:34.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:34.234+0000 7fbe92372700 1 --2- 172.21.15.40:0/4120269758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe8c0b1d60 0x7fbe8c06b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:34.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:34.234+0000 7fbe92372700 1 -- 172.21.15.40:0/4120269758 >> 172.21.15.40:0/4120269758 conn(0x7fbe8c0cbd80 msgr2=0x7fbe8c071740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:34.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:34.234+0000 7fbe92372700 1 -- 172.21.15.40:0/4120269758 shutdown_connections 2024-09-13T17:51:34.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:34.234+0000 7fbe92372700 1 -- 172.21.15.40:0/4120269758 wait complete. 2024-09-13T17:51:37.252 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:51:37.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.370+0000 7f1f38219700 1 Processor -- start 2024-09-13T17:51:37.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.370+0000 7f1f38219700 1 -- start start 2024-09-13T17:51:37.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.370+0000 7f1f38219700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f3006d570 0x7f1f301502b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:37.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.370+0000 7f1f38219700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f3006fb20 0x7f1f301507f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:37.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.370+0000 7f1f38219700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f30071bf0 con 0x7f1f3006d950 2024-09-13T17:51:37.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.370+0000 7f1f38219700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f30071330 con 0x7f1f3006d570 2024-09-13T17:51:37.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.370+0000 7f1f38219700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f300c73a0 con 0x7f1f3006fb20 2024-09-13T17:51:37.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.370+0000 7f1f367b6700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f1f3006d950 legacy=0x7f1f300c04f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:51:37.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.370+0000 7f1f35fb5700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f1f3006d570 msgr2=0x7f1f301502b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:51:37.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.370+0000 7f1f35fb5700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f3006d570 0x7f1f301502b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:51:37.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.370+0000 7f1f357b4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f3006fb20 0x7f1f301507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:37.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.370+0000 7f1f357b4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f3006fb20 0x7f1f301507f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:40416/0 (socket says 172.21.15.40:40416) 2024-09-13T17:51:37.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.370+0000 7f1f357b4700 1 -- 172.21.15.40:0/775063497 learned_addr learned my addr 172.21.15.40:0/775063497 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:37.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f357b4700 1 -- 172.21.15.40:0/775063497 >> v2:172.21.15.40:3300/0 conn(0x7f1f3006d570 msgr2=0x7f1f301502b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:51:37.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f357b4700 1 --2- 172.21.15.40:0/775063497 >> v2:172.21.15.40:3300/0 conn(0x7f1f3006d570 0x7f1f301502b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:37.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f357b4700 1 -- 172.21.15.40:0/775063497 >> v1:172.21.15.40:6789/0 conn(0x7f1f3006d950 legacy=0x7f1f300c04f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:51:37.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f357b4700 1 -- 172.21.15.40:0/775063497 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f300715a0 con 0x7f1f3006fb20 2024-09-13T17:51:37.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f357b4700 1 --2- 172.21.15.40:0/775063497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3006fb20 0x7f1f301507f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1f30070ce0 tx=0x7f1f24009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df0da3d1f4c945fd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:37.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f34fb3700 1 -- 172.21.15.40:0/775063497 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f24004230 con 0x7f1f3006fb20 2024-09-13T17:51:37.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f34fb3700 1 -- 172.21.15.40:0/775063497 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f24004390 con 0x7f1f3006fb20 2024-09-13T17:51:37.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f34fb3700 1 -- 172.21.15.40:0/775063497 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f240036c0 con 0x7f1f3006fb20 2024-09-13T17:51:37.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f38219700 1 -- 172.21.15.40:0/775063497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3006fb20 msgr2=0x7f1f301507f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:37.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f38219700 1 --2- 172.21.15.40:0/775063497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3006fb20 0x7f1f301507f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1f30070ce0 tx=0x7f1f24009bf0 comp rx=0 tx=0).stop 2024-09-13T17:51:37.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f38219700 1 -- 172.21.15.40:0/775063497 shutdown_connections 2024-09-13T17:51:37.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f38219700 1 --2- 172.21.15.40:0/775063497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3006fb20 0x7f1f301507f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:37.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f38219700 1 --2- 172.21.15.40:0/775063497 >> v2:172.21.15.40:3300/0 conn(0x7f1f3006d570 0x7f1f301502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:37.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f38219700 1 -- 172.21.15.40:0/775063497 >> 172.21.15.40:0/775063497 conn(0x7f1f300cadd0 msgr2=0x7f1f300cb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:37.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f38219700 1 -- 172.21.15.40:0/775063497 shutdown_connections 2024-09-13T17:51:37.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f38219700 1 -- 172.21.15.40:0/775063497 wait complete. 2024-09-13T17:51:37.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f38219700 1 Processor -- start 2024-09-13T17:51:37.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f38219700 1 -- start start 2024-09-13T17:51:37.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f38219700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f3006d570 0x7f1f3013f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:37.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f38219700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f3006d950 0x7f1f30145fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:37.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f38219700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3006fb20 0x7f1f30146500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:37.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f38219700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f300ca4b0 con 0x7f1f3006d950 2024-09-13T17:51:37.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f38219700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f301ebee0 con 0x7f1f3006fb20 2024-09-13T17:51:37.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f38219700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f301eea00 con 0x7f1f3006d570 2024-09-13T17:51:37.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f357b4700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f3006d950 msgr2=0x7f1f30145fc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:51:37.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.374+0000 7f1f357b4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f3006d950 0x7f1f30145fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:51:37.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.378+0000 7f1f367b6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3006fb20 0x7f1f30146500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:37.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.378+0000 7f1f35fb5700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f3006d570 0x7f1f3013f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:37.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.378+0000 7f1f367b6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3006fb20 0x7f1f30146500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:40420/0 (socket says 172.21.15.40:40420) 2024-09-13T17:51:37.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.378+0000 7f1f367b6700 1 -- 172.21.15.40:0/1704714749 learned_addr learned my addr 172.21.15.40:0/1704714749 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:37.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.378+0000 7f1f367b6700 1 -- 172.21.15.40:0/1704714749 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f3006d570 msgr2=0x7f1f3013f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:37.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.378+0000 7f1f367b6700 1 --2- 172.21.15.40:0/1704714749 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f3006d570 0x7f1f3013f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:37.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.378+0000 7f1f367b6700 1 -- 172.21.15.40:0/1704714749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f3006d950 msgr2=0x7f1f30145fc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:51:37.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.378+0000 7f1f367b6700 1 --2- 172.21.15.40:0/1704714749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f3006d950 0x7f1f30145fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:37.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.378+0000 7f1f367b6700 1 -- 172.21.15.40:0/1704714749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f24009570 con 0x7f1f3006fb20 2024-09-13T17:51:37.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.378+0000 7f1f35fb5700 1 --2- 172.21.15.40:0/1704714749 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f3006d570 0x7f1f3013f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:37.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.378+0000 7f1f367b6700 1 --2- 172.21.15.40:0/1704714749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3006fb20 0x7f1f30146500 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800be20 tx=0x7f1f28009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:37.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.378+0000 7f1f227fc700 1 -- 172.21.15.40:0/1704714749 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f28016630 con 0x7f1f3006fb20 2024-09-13T17:51:37.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.378+0000 7f1f227fc700 1 -- 172.21.15.40:0/1704714749 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f280167b0 con 0x7f1f3006fb20 2024-09-13T17:51:37.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.378+0000 7f1f38219700 1 -- 172.21.15.40:0/1704714749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f300c8bd0 con 0x7f1f3006fb20 2024-09-13T17:51:37.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.378+0000 7f1f227fc700 1 -- 172.21.15.40:0/1704714749 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f28016b60 con 0x7f1f3006fb20 2024-09-13T17:51:37.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.378+0000 7f1f38219700 1 -- 172.21.15.40:0/1704714749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f301eb5b0 con 0x7f1f3006fb20 2024-09-13T17:51:37.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.382+0000 7f1f38219700 1 -- 172.21.15.40:0/1704714749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f300bfe80 con 0x7f1f3006fb20 2024-09-13T17:51:37.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.382+0000 7f1f227fc700 1 -- 172.21.15.40:0/1704714749 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1f28008070 con 0x7f1f3006fb20 2024-09-13T17:51:37.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.382+0000 7f1f227fc700 1 --2- 172.21.15.40:0/1704714749 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f140666e0 0x7f1f14068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:37.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.382+0000 7f1f227fc700 1 -- 172.21.15.40:0/1704714749 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1f28084d50 con 0x7f1f3006fb20 2024-09-13T17:51:37.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.382+0000 7f1f35fb5700 1 --2- 172.21.15.40:0/1704714749 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f140666e0 0x7f1f14068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:37.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.382+0000 7f1f35fb5700 1 --2- 172.21.15.40:0/1704714749 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f140666e0 0x7f1f14068ba0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c003a30 tx=0x7f1f2c003c40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:37.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.394+0000 7f1f227fc700 1 -- 172.21.15.40:0/1704714749 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1f28051bd0 con 0x7f1f3006fb20 2024-09-13T17:51:37.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.626+0000 7f1f38219700 1 -- 172.21.15.40:0/1704714749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1f301561a0 con 0x7f1f3006fb20 2024-09-13T17:51:37.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.630+0000 7f1f227fc700 1 -- 172.21.15.40:0/1704714749 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7f1f2801b080 con 0x7f1f3006fb20 2024-09-13T17:51:37.634 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":88,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:51:37.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.630+0000 7f1f38219700 1 -- 172.21.15.40:0/1704714749 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f140666e0 msgr2=0x7f1f14068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:37.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.630+0000 7f1f38219700 1 --2- 172.21.15.40:0/1704714749 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f140666e0 0x7f1f14068ba0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c003a30 tx=0x7f1f2c003c40 comp rx=0 tx=0).stop 2024-09-13T17:51:37.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.630+0000 7f1f38219700 1 -- 172.21.15.40:0/1704714749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3006fb20 msgr2=0x7f1f30146500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:37.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.630+0000 7f1f38219700 1 --2- 172.21.15.40:0/1704714749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3006fb20 0x7f1f30146500 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800be20 tx=0x7f1f28009ee0 comp rx=0 tx=0).stop 2024-09-13T17:51:37.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.630+0000 7f1f38219700 1 -- 172.21.15.40:0/1704714749 shutdown_connections 2024-09-13T17:51:37.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.630+0000 7f1f38219700 1 --2- 172.21.15.40:0/1704714749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f3006fb20 0x7f1f30146500 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:37.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.630+0000 7f1f38219700 1 --2- 172.21.15.40:0/1704714749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f3006d950 0x7f1f30145fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:37.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.630+0000 7f1f38219700 1 --2- 172.21.15.40:0/1704714749 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f140666e0 0x7f1f14068ba0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:37.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.630+0000 7f1f38219700 1 --2- 172.21.15.40:0/1704714749 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f3006d570 0x7f1f3013f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:37.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.630+0000 7f1f38219700 1 -- 172.21.15.40:0/1704714749 >> 172.21.15.40:0/1704714749 conn(0x7f1f300cadd0 msgr2=0x7f1f30150080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:37.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.630+0000 7f1f38219700 1 -- 172.21.15.40:0/1704714749 shutdown_connections 2024-09-13T17:51:37.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:37.630+0000 7f1f38219700 1 -- 172.21.15.40:0/1704714749 wait complete. 2024-09-13T17:51:40.651 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:51:40.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.766+0000 7f1f630c7700 1 Processor -- start 2024-09-13T17:51:40.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.766+0000 7f1f630c7700 1 -- start start 2024-09-13T17:51:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.766+0000 7f1f630c7700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 0x7f1f5c1502b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.766+0000 7f1f630c7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 0x7f1f5c1507f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.766+0000 7f1f630c7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f5c06d950 0x7f1f5c0c04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.766+0000 7f1f630c7700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f5c071bf0 con 0x7f1f5c06d950 2024-09-13T17:51:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.766+0000 7f1f630c7700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f5c071330 con 0x7f1f5c06d570 2024-09-13T17:51:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.766+0000 7f1f630c7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0c73a0 con 0x7f1f5c06fb20 2024-09-13T17:51:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.766+0000 7f1f61664700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f1f5c06d950 msgr2=0x7f1f5c0c04f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:51:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.766+0000 7f1f61664700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f5c06d950 0x7f1f5c0c04f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:51:40.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.770+0000 7f1f5bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 0x7f1f5c1507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.770+0000 7f1f5bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 0x7f1f5c1507f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:47960/0 (socket says 172.21.15.40:47960) 2024-09-13T17:51:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.770+0000 7f1f5bfff700 1 -- 172.21.15.40:0/1398845951 learned_addr learned my addr 172.21.15.40:0/1398845951 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.770+0000 7f1f60e63700 1 --2- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 0x7f1f5c1502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.770+0000 7f1f5bfff700 1 -- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 msgr2=0x7f1f5c1507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:51:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.770+0000 7f1f5bfff700 1 -- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 msgr2=0x7f1f5c1507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:51:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.770+0000 7f1f5bfff700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 0x7f1f5c1507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:51:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.770+0000 7f1f5bfff700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 0x7f1f5c1507f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:51:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.770+0000 7f1f60e63700 1 -- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 msgr2=0x7f1f5c1502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:51:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.770+0000 7f1f60e63700 1 -- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 msgr2=0x7f1f5c1502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:51:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.770+0000 7f1f60e63700 1 --2- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 0x7f1f5c1502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:51:40.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.770+0000 7f1f60e63700 1 --2- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 0x7f1f5c1502b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:51:40.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.970+0000 7f1f61664700 1 -- 172.21.15.40:0/1398845951 >> v2:172.21.15.40:3300/0 conn(0x7f1f5c06d950 msgr2=0x7f1f5c0c04f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:51:40.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.970+0000 7f1f61664700 1 --2- 172.21.15.40:0/1398845951 >> v2:172.21.15.40:3300/0 conn(0x7f1f5c06d950 0x7f1f5c0c04f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:51:40.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.974+0000 7f1f60e63700 1 --2- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 0x7f1f5c1502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:40.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.974+0000 7f1f5bfff700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 0x7f1f5c1507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:40.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.974+0000 7f1f60e63700 1 -- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 msgr2=0x7f1f5c1502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:51:40.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.974+0000 7f1f60e63700 1 -- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 msgr2=0x7f1f5c1502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:51:40.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.974+0000 7f1f60e63700 1 --2- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 0x7f1f5c1502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:51:40.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.974+0000 7f1f60e63700 1 --2- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 0x7f1f5c1502b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:51:40.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.974+0000 7f1f5bfff700 1 -- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 msgr2=0x7f1f5c1507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:51:40.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.974+0000 7f1f5bfff700 1 -- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 msgr2=0x7f1f5c1507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:51:40.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.974+0000 7f1f5bfff700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 0x7f1f5c1507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:51:40.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:40.974+0000 7f1f5bfff700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 0x7f1f5c1507f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:51:41.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:41.374+0000 7f1f61664700 1 -- 172.21.15.40:0/1398845951 >> v2:172.21.15.40:3300/0 conn(0x7f1f5c06d950 msgr2=0x7f1f5c0c04f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:51:41.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:41.374+0000 7f1f61664700 1 --2- 172.21.15.40:0/1398845951 >> v2:172.21.15.40:3300/0 conn(0x7f1f5c06d950 0x7f1f5c0c04f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:51:41.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:41.378+0000 7f1f5bfff700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 0x7f1f5c1507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:41.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:41.378+0000 7f1f60e63700 1 --2- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 0x7f1f5c1502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:41.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:41.378+0000 7f1f5bfff700 1 -- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 msgr2=0x7f1f5c1507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:51:41.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:41.378+0000 7f1f5bfff700 1 -- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 msgr2=0x7f1f5c1507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:51:41.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:41.378+0000 7f1f5bfff700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 0x7f1f5c1507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:51:41.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:41.378+0000 7f1f5bfff700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 0x7f1f5c1507f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:51:41.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:41.378+0000 7f1f60e63700 1 -- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 msgr2=0x7f1f5c1502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:51:41.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:41.378+0000 7f1f60e63700 1 -- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 msgr2=0x7f1f5c1502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:51:41.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:41.378+0000 7f1f60e63700 1 --2- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 0x7f1f5c1502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:51:41.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:41.378+0000 7f1f60e63700 1 --2- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 0x7f1f5c1502b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:51:42.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:42.178+0000 7f1f61664700 1 -- 172.21.15.40:0/1398845951 >> v2:172.21.15.40:3300/0 conn(0x7f1f5c06d950 msgr2=0x7f1f5c0c04f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:51:42.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:42.178+0000 7f1f61664700 1 --2- 172.21.15.40:0/1398845951 >> v2:172.21.15.40:3300/0 conn(0x7f1f5c06d950 0x7f1f5c0c04f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:51:42.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:42.182+0000 7f1f5bfff700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 0x7f1f5c1507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:42.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:42.182+0000 7f1f60e63700 1 --2- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 0x7f1f5c1502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:42.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:42.182+0000 7f1f5bfff700 1 -- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 msgr2=0x7f1f5c1507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:51:42.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:42.182+0000 7f1f5bfff700 1 -- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 msgr2=0x7f1f5c1507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:51:42.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:42.182+0000 7f1f5bfff700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 0x7f1f5c1507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:51:42.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:42.182+0000 7f1f60e63700 1 -- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 msgr2=0x7f1f5c1502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:51:42.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:42.182+0000 7f1f60e63700 1 -- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 msgr2=0x7f1f5c1502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:51:42.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:42.182+0000 7f1f60e63700 1 --2- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 0x7f1f5c1502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:51:42.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:42.182+0000 7f1f5bfff700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 0x7f1f5c1507f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:51:42.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:42.182+0000 7f1f60e63700 1 --2- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 0x7f1f5c1502b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:51:43.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:43.766+0000 7f1f5a7fc700 1 -- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 msgr2=0x7f1f5c1507f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:51:43.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:43.766+0000 7f1f5a7fc700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 0x7f1f5c1507f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:43.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:43.766+0000 7f1f5a7fc700 1 -- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 msgr2=0x7f1f5c1502b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:51:43.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:43.766+0000 7f1f5a7fc700 1 --2- 172.21.15.40:0/1398845951 >> v2:172.21.15.43:3300/0 conn(0x7f1f5c06d570 0x7f1f5c1502b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:43.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:43.766+0000 7f1f5a7fc700 1 -- 172.21.15.40:0/1398845951 >> v2:172.21.15.40:3300/0 conn(0x7f1f5c06d950 msgr2=0x7f1f5c0c04f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:51:43.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:43.766+0000 7f1f5a7fc700 1 --2- 172.21.15.40:0/1398845951 >> v2:172.21.15.40:3300/0 conn(0x7f1f5c06d950 0x7f1f5c0c04f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:43.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:43.770+0000 7f1f5a7fc700 1 -- 172.21.15.40:0/1398845951 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5c0715a0 con 0x7f1f400073a0 2024-09-13T17:51:43.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:43.770+0000 7f1f5a7fc700 1 -- 172.21.15.40:0/1398845951 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f4000c110 con 0x7f1f400036f0 2024-09-13T17:51:43.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:43.770+0000 7f1f5bfff700 1 -- 172.21.15.40:0/1398845951 >> v1:172.21.15.40:6789/0 conn(0x7f1f400073a0 legacy=0x7f1f40009860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:51:43.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:43.974+0000 7f1f5bfff700 1 -- 172.21.15.40:0/1398845951 >> v1:172.21.15.40:6789/0 conn(0x7f1f400073a0 legacy=0x7f1f40009860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:51:44.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:44.377+0000 7f1f5bfff700 1 -- 172.21.15.40:0/1398845951 >> v1:172.21.15.40:6789/0 conn(0x7f1f400073a0 legacy=0x7f1f40009860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:51:45.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.181+0000 7f1f5bfff700 1 -- 172.21.15.40:0/1398845951 >> v1:172.21.15.40:6789/0 conn(0x7f1f400073a0 legacy=0x7f1f40009860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:51:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.357+0000 7f1f5b7fe700 1 -- 172.21.15.40:0/1398845951 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487897013 0 0) 0x7f1f4000c110 con 0x7f1f400036f0 2024-09-13T17:51:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.357+0000 7f1f5b7fe700 1 -- 172.21.15.40:0/1398845951 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f4c0035d0 con 0x7f1f400036f0 2024-09-13T17:51:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.361+0000 7f1f5b7fe700 1 -- 172.21.15.40:0/1398845951 <== mon.1 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3833789327 0 0) 0x7f1f4c0035d0 con 0x7f1f400036f0 2024-09-13T17:51:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.361+0000 7f1f5b7fe700 1 -- 172.21.15.40:0/1398845951 >> v1:172.21.15.40:6789/0 conn(0x7f1f400073a0 legacy=0x7f1f40009860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:51:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.361+0000 7f1f5b7fe700 1 -- 172.21.15.40:0/1398845951 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4000d310 con 0x7f1f400036f0 2024-09-13T17:51:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.361+0000 7f1f5b7fe700 1 -- 172.21.15.40:0/1398845951 <== mon.1 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f1f5000c4f0 con 0x7f1f400036f0 2024-09-13T17:51:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.361+0000 7f1f5b7fe700 1 -- 172.21.15.40:0/1398845951 >> v1:172.21.15.43:6789/0 conn(0x7f1f400036f0 legacy=0x7f1f40005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.361+0000 7f1f61664700 1 -- 172.21.15.40:0/1398845951 reap_dead start 2024-09-13T17:51:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.361+0000 7f1f5b7fe700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 0x7f1f4c003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.361+0000 7f1f5b7fe700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c0040f0 0x7f1f4c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.361+0000 7f1f5b7fe700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f4c004e00 0x7f1f4c005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.361+0000 7f1f5b7fe700 1 -- 172.21.15.40:0/1398845951 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c00bcb0 con 0x7f1f4c004e00 2024-09-13T17:51:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.361+0000 7f1f5b7fe700 1 -- 172.21.15.40:0/1398845951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c00be20 con 0x7f1f4c0040f0 2024-09-13T17:51:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.361+0000 7f1f5b7fe700 1 -- 172.21.15.40:0/1398845951 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c00bfe0 con 0x7f1f5c06fb20 2024-09-13T17:51:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.361+0000 7f1f60e63700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 0x7f1f4c003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.361+0000 7f1f61664700 1 -- 172.21.15.40:0/1398845951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f4c004e00 msgr2=0x7f1f4c005280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:51:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.361+0000 7f1f61664700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f4c004e00 0x7f1f4c005280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:51:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.361+0000 7f1f5bfff700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c0040f0 0x7f1f4c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.361+0000 7f1f60e63700 1 -- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c0040f0 msgr2=0x7f1f4c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:51:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.361+0000 7f1f60e63700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c0040f0 0x7f1f4c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.365+0000 7f1f60e63700 1 -- 172.21.15.40:0/1398845951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f4c004e00 msgr2=0x7f1f4c005280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:51:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.365+0000 7f1f60e63700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f4c004e00 0x7f1f4c005280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.365+0000 7f1f60e63700 1 -- 172.21.15.40:0/1398845951 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c0035d0 con 0x7f1f5c06fb20 2024-09-13T17:51:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.365+0000 7f1f5bfff700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c0040f0 0x7f1f4c004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.365+0000 7f1f60e63700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 0x7f1f4c003a20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000ce00 tx=0x7f1f4c0058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f12d0d0644b3e0e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.365+0000 7f1f5b7fe700 1 -- 172.21.15.40:0/1398845951 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50014ce0 con 0x7f1f5c06fb20 2024-09-13T17:51:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.365+0000 7f1f5b7fe700 1 -- 172.21.15.40:0/1398845951 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5000c040 con 0x7f1f5c06fb20 2024-09-13T17:51:45.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.365+0000 7f1f5b7fe700 1 -- 172.21.15.40:0/1398845951 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50004270 con 0x7f1f5c06fb20 2024-09-13T17:51:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.369+0000 7f1f630c7700 1 -- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 msgr2=0x7f1f4c003a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:45.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.369+0000 7f1f630c7700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 0x7f1f4c003a20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000ce00 tx=0x7f1f4c0058a0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.369+0000 7f1f630c7700 1 -- 172.21.15.40:0/1398845951 shutdown_connections 2024-09-13T17:51:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.369+0000 7f1f630c7700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f4c004e00 0x7f1f4c005280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.369+0000 7f1f630c7700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c0040f0 0x7f1f4c004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.369+0000 7f1f630c7700 1 --2- 172.21.15.40:0/1398845951 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f5c06fb20 0x7f1f4c003a20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.369+0000 7f1f630c7700 1 -- 172.21.15.40:0/1398845951 >> 172.21.15.40:0/1398845951 conn(0x7f1f5c0cadd0 msgr2=0x7f1f5c0cb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.369+0000 7f1f630c7700 1 -- 172.21.15.40:0/1398845951 shutdown_connections 2024-09-13T17:51:45.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.369+0000 7f1f630c7700 1 -- 172.21.15.40:0/1398845951 wait complete. 2024-09-13T17:51:45.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f630c7700 1 Processor -- start 2024-09-13T17:51:45.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f630c7700 1 -- start start 2024-09-13T17:51:45.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f630c7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c0040f0 0x7f1f5c14c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f630c7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f4c004e00 0x7f1f5c0c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f630c7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f5c06fb20 0x7f1f5c1eee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f630c7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0c6cb0 con 0x7f1f5c06fb20 2024-09-13T17:51:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f630c7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c0ae4b0 con 0x7f1f4c0040f0 2024-09-13T17:51:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f630c7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c14c580 con 0x7f1f4c004e00 2024-09-13T17:51:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f61664700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f5c06fb20 msgr2=0x7f1f5c1eee10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:51:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f61664700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f5c06fb20 0x7f1f5c1eee10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:51:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f5bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f4c004e00 0x7f1f5c0c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f60e63700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c0040f0 0x7f1f5c14c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f60e63700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c0040f0 0x7f1f5c14c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:45002/0 (socket says 172.21.15.40:45002) 2024-09-13T17:51:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f5bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f4c004e00 0x7f1f5c0c6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:40976/0 (socket says 172.21.15.40:40976) 2024-09-13T17:51:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f60e63700 1 -- 172.21.15.40:0/1468242040 learned_addr learned my addr 172.21.15.40:0/1468242040 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:45.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f60e63700 1 -- 172.21.15.40:0/1468242040 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f4c004e00 msgr2=0x7f1f5c0c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:45.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f60e63700 1 --2- 172.21.15.40:0/1468242040 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f4c004e00 0x7f1f5c0c6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f60e63700 1 -- 172.21.15.40:0/1468242040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f5c06fb20 msgr2=0x7f1f5c1eee10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:51:45.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f60e63700 1 --2- 172.21.15.40:0/1468242040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f5c06fb20 0x7f1f5c1eee10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f60e63700 1 -- 172.21.15.40:0/1468242040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c1ee950 con 0x7f1f4c0040f0 2024-09-13T17:51:45.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.373+0000 7f1f5bfff700 1 --2- 172.21.15.40:0/1468242040 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f4c004e00 0x7f1f5c0c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:45.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.377+0000 7f1f60e63700 1 --2- 172.21.15.40:0/1468242040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c0040f0 0x7f1f5c14c040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f500148b0 tx=0x7f1f50004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:45.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.377+0000 7f1f597fa700 1 -- 172.21.15.40:0/1468242040 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50026070 con 0x7f1f4c0040f0 2024-09-13T17:51:45.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.377+0000 7f1f597fa700 1 -- 172.21.15.40:0/1468242040 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f50006c80 con 0x7f1f4c0040f0 2024-09-13T17:51:45.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.377+0000 7f1f630c7700 1 -- 172.21.15.40:0/1468242040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c14c780 con 0x7f1f4c0040f0 2024-09-13T17:51:45.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.377+0000 7f1f597fa700 1 -- 172.21.15.40:0/1468242040 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f50004480 con 0x7f1f4c0040f0 2024-09-13T17:51:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.377+0000 7f1f630c7700 1 -- 172.21.15.40:0/1468242040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5c0adc10 con 0x7f1f4c0040f0 2024-09-13T17:51:45.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.377+0000 7f1f630c7700 1 -- 172.21.15.40:0/1468242040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5c156a10 con 0x7f1f4c0040f0 2024-09-13T17:51:45.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.381+0000 7f1f597fa700 1 -- 172.21.15.40:0/1468242040 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1f5000ca10 con 0x7f1f4c0040f0 2024-09-13T17:51:45.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.381+0000 7f1f597fa700 1 --2- 172.21.15.40:0/1468242040 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f40066f70 0x7f1f40069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.381+0000 7f1f597fa700 1 -- 172.21.15.40:0/1468242040 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1f50000e40 con 0x7f1f4c0040f0 2024-09-13T17:51:45.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.381+0000 7f1f5bfff700 1 --2- 172.21.15.40:0/1468242040 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f40066f70 0x7f1f40069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:45.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.381+0000 7f1f5bfff700 1 --2- 172.21.15.40:0/1468242040 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f40066f70 0x7f1f40069430 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c1492d0 tx=0x7f1f48014040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:45.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.389+0000 7f1f597fa700 1 -- 172.21.15.40:0/1468242040 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1f50087e60 con 0x7f1f4c0040f0 2024-09-13T17:51:45.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.629+0000 7f1f630c7700 1 -- 172.21.15.40:0/1468242040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1f5c156a10 con 0x7f1f4c0040f0 2024-09-13T17:51:45.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.633+0000 7f1f597fa700 1 -- 172.21.15.40:0/1468242040 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7f1f50052cd0 con 0x7f1f4c0040f0 2024-09-13T17:51:45.638 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":10,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:51:45.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.637+0000 7f1f630c7700 1 -- 172.21.15.40:0/1468242040 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f40066f70 msgr2=0x7f1f40069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:45.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.637+0000 7f1f630c7700 1 --2- 172.21.15.40:0/1468242040 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f40066f70 0x7f1f40069430 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c1492d0 tx=0x7f1f48014040 comp rx=0 tx=0).stop 2024-09-13T17:51:45.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.637+0000 7f1f630c7700 1 -- 172.21.15.40:0/1468242040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c0040f0 msgr2=0x7f1f5c14c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:45.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.637+0000 7f1f630c7700 1 --2- 172.21.15.40:0/1468242040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c0040f0 0x7f1f5c14c040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f500148b0 tx=0x7f1f50004770 comp rx=0 tx=0).stop 2024-09-13T17:51:45.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.637+0000 7f1f630c7700 1 -- 172.21.15.40:0/1468242040 shutdown_connections 2024-09-13T17:51:45.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.637+0000 7f1f630c7700 1 --2- 172.21.15.40:0/1468242040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f5c06fb20 0x7f1f5c1eee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.637+0000 7f1f630c7700 1 --2- 172.21.15.40:0/1468242040 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f40066f70 0x7f1f40069430 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.637+0000 7f1f630c7700 1 --2- 172.21.15.40:0/1468242040 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f4c004e00 0x7f1f5c0c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.637+0000 7f1f630c7700 1 --2- 172.21.15.40:0/1468242040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c0040f0 0x7f1f5c14c040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.637+0000 7f1f630c7700 1 -- 172.21.15.40:0/1468242040 >> 172.21.15.40:0/1468242040 conn(0x7f1f5c0cadd0 msgr2=0x7f1f5c0720b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:45.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.637+0000 7f1f630c7700 1 -- 172.21.15.40:0/1468242040 shutdown_connections 2024-09-13T17:51:45.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.637+0000 7f1f630c7700 1 -- 172.21.15.40:0/1468242040 wait complete. 2024-09-13T17:51:45.660 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T17:51:45.660 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-13T17:51:45.660 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:51:45.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.769+0000 7fd6ea60c700 1 Processor -- start 2024-09-13T17:51:45.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.769+0000 7fd6ea60c700 1 -- start start 2024-09-13T17:51:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6ea60c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd6e406ac20 0x7fd6e406b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6ea60c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd6e40c1110 0x7fd6e406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6ea60c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e40c16f0 con 0x7fd6e40bfe80 2024-09-13T17:51:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6ea60c700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e406b540 con 0x7fd6e406ac20 2024-09-13T17:51:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6ea60c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e406cfe0 con 0x7fd6e40c1110 2024-09-13T17:51:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6e37fe700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7fd6e40bfe80 legacy=0x7fd6e40715e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:51:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6e3fff700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7fd6e406ac20 msgr2=0x7fd6e406b000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:51:45.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6e3fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd6e406ac20 0x7fd6e406b000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:51:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6e8ba9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd6e40c1110 0x7fd6e406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6e8ba9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd6e40c1110 0x7fd6e406fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:40978/0 (socket says 172.21.15.40:40978) 2024-09-13T17:51:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6e8ba9700 1 -- 172.21.15.40:0/21063252 learned_addr learned my addr 172.21.15.40:0/21063252 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6e8ba9700 1 -- 172.21.15.40:0/21063252 >> v2:172.21.15.40:3300/0 conn(0x7fd6e406ac20 msgr2=0x7fd6e406b000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:51:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6e8ba9700 1 --2- 172.21.15.40:0/21063252 >> v2:172.21.15.40:3300/0 conn(0x7fd6e406ac20 0x7fd6e406b000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6e8ba9700 1 -- 172.21.15.40:0/21063252 >> v1:172.21.15.40:6789/0 conn(0x7fd6e40bfe80 legacy=0x7fd6e40715e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:51:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6e8ba9700 1 -- 172.21.15.40:0/21063252 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e40c19d0 con 0x7fd6e40c1110 2024-09-13T17:51:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6e8ba9700 1 --2- 172.21.15.40:0/21063252 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd6e40c1110 0x7fd6e406fbc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009e50 tx=0x7fd6dc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb2c5fa62fc95b59 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6e2ffd700 1 -- 172.21.15.40:0/21063252 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc00e070 con 0x7fd6e40c1110 2024-09-13T17:51:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6e2ffd700 1 -- 172.21.15.40:0/21063252 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc003a40 con 0x7fd6e40c1110 2024-09-13T17:51:45.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6e2ffd700 1 -- 172.21.15.40:0/21063252 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc003df0 con 0x7fd6e40c1110 2024-09-13T17:51:45.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6ea60c700 1 -- 172.21.15.40:0/21063252 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd6e40c1110 msgr2=0x7fd6e406fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:45.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6ea60c700 1 --2- 172.21.15.40:0/21063252 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd6e40c1110 0x7fd6e406fbc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009e50 tx=0x7fd6dc00b330 comp rx=0 tx=0).stop 2024-09-13T17:51:45.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6ea60c700 1 -- 172.21.15.40:0/21063252 shutdown_connections 2024-09-13T17:51:45.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6ea60c700 1 --2- 172.21.15.40:0/21063252 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd6e40c1110 0x7fd6e406fbc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6ea60c700 1 --2- 172.21.15.40:0/21063252 >> v2:172.21.15.40:3300/0 conn(0x7fd6e406ac20 0x7fd6e406b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6ea60c700 1 -- 172.21.15.40:0/21063252 >> 172.21.15.40:0/21063252 conn(0x7fd6e406d5b0 msgr2=0x7fd6e406d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:45.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6ea60c700 1 -- 172.21.15.40:0/21063252 shutdown_connections 2024-09-13T17:51:45.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.773+0000 7fd6ea60c700 1 -- 172.21.15.40:0/21063252 wait complete. 2024-09-13T17:51:45.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6ea60c700 1 Processor -- start 2024-09-13T17:51:45.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6ea60c700 1 -- start start 2024-09-13T17:51:45.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6ea60c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6e406ac20 0x7fd6e40c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6ea60c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e40bfe80 0x7fd6e40c1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6ea60c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd6e40c1110 0x7fd6e40c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6ea60c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e41f13b0 con 0x7fd6e406ac20 2024-09-13T17:51:45.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6ea60c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e40ca5a0 con 0x7fd6e40bfe80 2024-09-13T17:51:45.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6ea60c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e40ca860 con 0x7fd6e40c1110 2024-09-13T17:51:45.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6e3fff700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6e406ac20 msgr2=0x7fd6e40c9990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:51:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6e3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6e406ac20 0x7fd6e40c9990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:51:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6e37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e40bfe80 0x7fd6e40c1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6e8ba9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd6e40c1110 0x7fd6e40c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6e37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e40bfe80 0x7fd6e40c1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:45016/0 (socket says 172.21.15.40:45016) 2024-09-13T17:51:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6e37fe700 1 -- 172.21.15.40:0/3831220805 learned_addr learned my addr 172.21.15.40:0/3831220805 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:45.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6e8ba9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd6e40c1110 0x7fd6e40c3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:40980/0 (socket says 172.21.15.40:40980) 2024-09-13T17:51:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6e37fe700 1 -- 172.21.15.40:0/3831220805 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd6e40c1110 msgr2=0x7fd6e40c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6e37fe700 1 --2- 172.21.15.40:0/3831220805 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd6e40c1110 0x7fd6e40c3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6e37fe700 1 -- 172.21.15.40:0/3831220805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6e406ac20 msgr2=0x7fd6e40c9990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:51:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6e37fe700 1 --2- 172.21.15.40:0/3831220805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6e406ac20 0x7fd6e40c9990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6e37fe700 1 -- 172.21.15.40:0/3831220805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc009b30 con 0x7fd6e40bfe80 2024-09-13T17:51:45.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6e8ba9700 1 --2- 172.21.15.40:0/3831220805 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd6e40c1110 0x7fd6e40c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:51:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6e37fe700 1 --2- 172.21.15.40:0/3831220805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e40bfe80 0x7fd6e40c1e60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00b780 tx=0x7fd6cc00ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.777+0000 7fd6e0ff9700 1 -- 172.21.15.40:0/3831220805 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc003e70 con 0x7fd6e40bfe80 2024-09-13T17:51:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.781+0000 7fd6ea60c700 1 -- 172.21.15.40:0/3831220805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e40c8ed0 con 0x7fd6e40bfe80 2024-09-13T17:51:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.781+0000 7fd6e0ff9700 1 -- 172.21.15.40:0/3831220805 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6cc005070 con 0x7fd6e40bfe80 2024-09-13T17:51:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.781+0000 7fd6e0ff9700 1 -- 172.21.15.40:0/3831220805 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc005420 con 0x7fd6e40bfe80 2024-09-13T17:51:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.781+0000 7fd6ea60c700 1 -- 172.21.15.40:0/3831220805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e40c9490 con 0x7fd6e40bfe80 2024-09-13T17:51:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.781+0000 7fd6ea60c700 1 --2- 172.21.15.40:0/3831220805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e40c42b0 0x7fd6e4141050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.781+0000 7fd6ea60c700 1 -- 172.21.15.40:0/3831220805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e4067840 con 0x7fd6e40c42b0 2024-09-13T17:51:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.781+0000 7fd6e3fff700 1 --2- 172.21.15.40:0/3831220805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e40c42b0 0x7fd6e4141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.781+0000 7fd6e3fff700 1 -- 172.21.15.40:0/3831220805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd6e40c0aa0 con 0x7fd6e40c42b0 2024-09-13T17:51:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.781+0000 7fd6e3fff700 1 --2- 172.21.15.40:0/3831220805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e40c42b0 0x7fd6e4141050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400bcc0 tx=0x7fd6d4009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.781+0000 7fd6e0ff9700 1 -- 172.21.15.40:0/3831220805 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd6cc005580 con 0x7fd6e40bfe80 2024-09-13T17:51:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.781+0000 7fd6e0ff9700 1 --2- 172.21.15.40:0/3831220805 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd6c40666e0 0x7fd6c4068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.781+0000 7fd6e0ff9700 1 -- 172.21.15.40:0/3831220805 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd6cc084040 con 0x7fd6e40bfe80 2024-09-13T17:51:45.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.781+0000 7fd6e0ff9700 1 -- 172.21.15.40:0/3831220805 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4003ab0 con 0x7fd6e40c42b0 2024-09-13T17:51:45.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.781+0000 7fd6e8ba9700 1 --2- 172.21.15.40:0/3831220805 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd6c40666e0 0x7fd6c4068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:45.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.781+0000 7fd6e0ff9700 1 -- 172.21.15.40:0/3831220805 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd6d4003c10 con 0x7fd6e40c42b0 2024-09-13T17:51:45.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.781+0000 7fd6e0ff9700 1 -- 172.21.15.40:0/3831220805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e40c42b0 msgr2=0x7fd6e4141050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:45.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.781+0000 7fd6e0ff9700 1 --2- 172.21.15.40:0/3831220805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e40c42b0 0x7fd6e4141050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400bcc0 tx=0x7fd6d4009f30 comp rx=0 tx=0).stop 2024-09-13T17:51:45.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.781+0000 7fd6e0ff9700 1 -- 172.21.15.40:0/3831220805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e40c42b0 msgr2=0x7fd6e4141050 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:51:45.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.781+0000 7fd6e0ff9700 1 --2- 172.21.15.40:0/3831220805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e40c42b0 0x7fd6e4141050 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.785+0000 7fd6e8ba9700 1 --2- 172.21.15.40:0/3831220805 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd6c40666e0 0x7fd6c4068ba0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc015040 tx=0x7fd6dc009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:45.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.793+0000 7fd6ea60c700 1 --2- 172.21.15.40:0/3831220805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e4141590 0x7fd6e413f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.793+0000 7fd6ea60c700 1 -- 172.21.15.40:0/3831220805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e4070100 con 0x7fd6e4141590 2024-09-13T17:51:45.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.793+0000 7fd6e3fff700 1 --2- 172.21.15.40:0/3831220805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e4141590 0x7fd6e413f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:45.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.793+0000 7fd6e3fff700 1 -- 172.21.15.40:0/3831220805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd6e40c0d80 con 0x7fd6e4141590 2024-09-13T17:51:45.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.797+0000 7fd6e3fff700 1 --2- 172.21.15.40:0/3831220805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e4141590 0x7fd6e413f9e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4014000 tx=0x7fd6d4005ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:45.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.797+0000 7fd6e0ff9700 1 -- 172.21.15.40:0/3831220805 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d401a990 con 0x7fd6e4141590 2024-09-13T17:51:45.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.797+0000 7fd6e0ff9700 1 -- 172.21.15.40:0/3831220805 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4938 (secure 0 0 0) 0x7fd6d401aaf0 con 0x7fd6e4141590 2024-09-13T17:51:45.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.797+0000 7fd6e0ff9700 1 -- 172.21.15.40:0/3831220805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e4141590 msgr2=0x7fd6e413f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:45.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.797+0000 7fd6e0ff9700 1 --2- 172.21.15.40:0/3831220805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e4141590 0x7fd6e413f9e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4014000 tx=0x7fd6d4005ad0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.797+0000 7fd6e0ff9700 1 -- 172.21.15.40:0/3831220805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e4141590 msgr2=0x7fd6e413f9e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:51:45.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.797+0000 7fd6e0ff9700 1 --2- 172.21.15.40:0/3831220805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e4141590 0x7fd6e413f9e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.803 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:51:45.803 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:51:45.803 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:51:45.803 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:51:45.803 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 10, 2024-09-13T17:51:45.804 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:51:45.804 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:51:45.804 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:51:45.804 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:45.804 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T17:51:45.804 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:51:45.804 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:51:45.804 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:51:45.804 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:51:45.804 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:51:45.804 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:51:45.804 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:51:45.804 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:51:45.805 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:51:45.805 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:51:45.805 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:51:45.805 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:51:45.805 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:51:45.805 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:45.805 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:51:45.805 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:51:45.805 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:51:45.805 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:51:45.805 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:51:45.805 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:51:45.805 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:51:45.806 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:51:45.806 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:51:45.806 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:51:45.806 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:51:45.806 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:51:45.806 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:45.806 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.806 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:51:45.806 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:51:45.806 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:51:45.806 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:51:45.806 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:51:45.806 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:51:45.806 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:51:45.807 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:51:45.807 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:51:45.807 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:51:45.807 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:51:45.807 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:51:45.807 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:51:45.807 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:51:45.807 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:51:45.807 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:51:45.807 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:51:45.807 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:51:45.808 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:51:45.808 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:51:45.808 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:51:45.808 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:51:45.808 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:51:45.808 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:51:45.808 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:51:45.808 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:51:45.808 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:51:45.808 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:45.808 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:51:45.809 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.809 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:51:45.809 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.809 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:51:45.809 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:51:45.809 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:51:45.809 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:51:45.809 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.809 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:51:45.809 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:51:45.809 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:45.809 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.810 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.810 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:51:45.810 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:51:45.810 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:45.810 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:45.810 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:45.810 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.810 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:51:45.810 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:51:45.810 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:51:45.810 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:51:45.811 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:51:45.811 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.811 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.811 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:51:45.811 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:51:45.811 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:51:45.811 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:51:45.811 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.811 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:51:45.811 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:51:45.811 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:45.811 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.812 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.812 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:51:45.812 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:51:45.812 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:45.812 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:45.812 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:45.812 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.812 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:51:45.812 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:51:45.812 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:51:45.813 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:51:45.813 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:51:45.813 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.813 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.813 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:51:45.813 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:51:45.813 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:51:45.813 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:51:45.813 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.813 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:51:45.813 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:51:45.814 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:45.814 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.814 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.814 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:51:45.814 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:51:45.814 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:45.814 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:45.814 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:45.814 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.814 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:51:45.814 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:51:45.815 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:51:45.815 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:51:45.815 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:51:45.815 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:45.815 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:45.815 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.815 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:51:45.815 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:51:45.815 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.815 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:45.816 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:45.816 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:51:45.816 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:45.816 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:45.816 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:51:45.816 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.816 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:45.816 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:45.816 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T17:51:45.816 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:45.816 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:45.817 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:51:45.817 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.817 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:45.817 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:45.817 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T17:51:45.817 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:45.817 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:45.817 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:51:45.817 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.817 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:45.817 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:45.818 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:51:45.818 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:45.818 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:45.818 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:51:45.818 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.818 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:45.818 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:45.818 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:51:45.818 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:45.818 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:45.818 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.819 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:51:45.819 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:51:45.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.797+0000 7fd6ea60c700 1 -- 172.21.15.40:0/3831220805 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd6c40666e0 msgr2=0x7fd6c4068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:45.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.797+0000 7fd6ea60c700 1 --2- 172.21.15.40:0/3831220805 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd6c40666e0 0x7fd6c4068ba0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc015040 tx=0x7fd6dc009000 comp rx=0 tx=0).stop 2024-09-13T17:51:45.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.797+0000 7fd6ea60c700 1 -- 172.21.15.40:0/3831220805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e40bfe80 msgr2=0x7fd6e40c1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:45.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.797+0000 7fd6ea60c700 1 --2- 172.21.15.40:0/3831220805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6e40bfe80 0x7fd6e40c1e60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00b780 tx=0x7fd6cc00ba90 comp rx=0 tx=0).stop 2024-09-13T17:51:45.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.797+0000 7fd6e8ba9700 1 -- 172.21.15.40:0/3831220805 reap_dead start 2024-09-13T17:51:45.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.797+0000 7fd6e8ba9700 1 -- 172.21.15.40:0/3831220805 reap_dead start 2024-09-13T17:51:45.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.797+0000 7fd6ea60c700 1 -- 172.21.15.40:0/3831220805 shutdown_connections 2024-09-13T17:51:45.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.797+0000 7fd6ea60c700 1 -- 172.21.15.40:0/3831220805 >> 172.21.15.40:0/3831220805 conn(0x7fd6e406d5b0 msgr2=0x7fd6e40c69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:45.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.797+0000 7fd6ea60c700 1 -- 172.21.15.40:0/3831220805 shutdown_connections 2024-09-13T17:51:45.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.797+0000 7fd6ea60c700 1 -- 172.21.15.40:0/3831220805 wait complete. 2024-09-13T17:51:45.821 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T17:51:45.821 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T17:51:45.821 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:51:45.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.933+0000 7f3709bcc700 1 Processor -- start 2024-09-13T17:51:45.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.933+0000 7f3709bcc700 1 -- start start 2024-09-13T17:51:45.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.933+0000 7f3709bcc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f37040c7f60 0x7f3704072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.933+0000 7f3709bcc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37040c7320 0x7f370415ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.933+0000 7f3709bcc700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37040c5290 con 0x7f37040c8340 2024-09-13T17:51:45.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.933+0000 7f3709bcc700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3704071390 con 0x7f37040c7320 2024-09-13T17:51:45.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.933+0000 7f3709bcc700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37040727f0 con 0x7f37040c7f60 2024-09-13T17:51:45.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.933+0000 7f3703fff700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f37040c7320 msgr2=0x7f370415ab50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:51:45.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.933+0000 7f3703fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37040c7320 0x7f370415ab50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:51:45.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.933+0000 7f3708bca700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f37040c7f60 0x7f3704072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:45.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.933+0000 7f37093cb700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f37040c8340 0x7f3704151190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:46336/0 (socket says 172.21.15.40:46336) 2024-09-13T17:51:45.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.933+0000 7f37093cb700 1 -- 172.21.15.40:0/3443970058 learned_addr learned my addr 172.21.15.40:0/3443970058 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:51:45.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3708bca700 1 -- 172.21.15.40:0/3443970058 >> v2:172.21.15.40:3300/0 conn(0x7f37040c7320 msgr2=0x7f370415ab50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:51:45.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3708bca700 1 --2- 172.21.15.40:0/3443970058 >> v2:172.21.15.40:3300/0 conn(0x7f37040c7320 0x7f370415ab50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3708bca700 1 -- 172.21.15.40:0/3443970058 >> v1:172.21.15.43:6789/0 conn(0x7f37040c8340 legacy=0x7f3704151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:45.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f37037fe700 1 -- 172.21.15.40:0/3443970058 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3336034008 0 0) 0x7f37040c5290 con 0x7f37040c8340 2024-09-13T17:51:45.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3708bca700 1 -- 172.21.15.40:0/3443970058 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37040c5870 con 0x7f37040c7f60 2024-09-13T17:51:45.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3708bca700 1 --2- 172.21.15.40:0/3443970058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37040c7f60 0x7f3704072200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f37040c2150 tx=0x7f36ec009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=806f43610fa66c53 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:45.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f37037fe700 1 -- 172.21.15.40:0/3443970058 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec011830 con 0x7f37040c7f60 2024-09-13T17:51:45.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f37037fe700 1 -- 172.21.15.40:0/3443970058 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ec011990 con 0x7f37040c7f60 2024-09-13T17:51:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f37037fe700 1 -- 172.21.15.40:0/3443970058 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec011d40 con 0x7f37040c7f60 2024-09-13T17:51:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3709bcc700 1 -- 172.21.15.40:0/3443970058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37040c7f60 msgr2=0x7f3704072200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3709bcc700 1 --2- 172.21.15.40:0/3443970058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37040c7f60 0x7f3704072200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f37040c2150 tx=0x7f36ec009b30 comp rx=0 tx=0).stop 2024-09-13T17:51:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3709bcc700 1 -- 172.21.15.40:0/3443970058 shutdown_connections 2024-09-13T17:51:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3709bcc700 1 --2- 172.21.15.40:0/3443970058 >> v2:172.21.15.40:3300/0 conn(0x7f37040c7320 0x7f370415ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3709bcc700 1 --2- 172.21.15.40:0/3443970058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37040c7f60 0x7f3704072200 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3709bcc700 1 -- 172.21.15.40:0/3443970058 >> 172.21.15.40:0/3443970058 conn(0x7f3704066090 msgr2=0x7f37040664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3709bcc700 1 -- 172.21.15.40:0/3443970058 shutdown_connections 2024-09-13T17:51:45.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3709bcc700 1 -- 172.21.15.40:0/3443970058 wait complete. 2024-09-13T17:51:45.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3709bcc700 1 Processor -- start 2024-09-13T17:51:45.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3709bcc700 1 -- start start 2024-09-13T17:51:45.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3709bcc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37040c7320 0x7f37040c2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3709bcc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37040c7f60 0x7f37041e2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3709bcc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37040c8340 0x7f37040ade10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3709bcc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37040c1780 con 0x7f37040c7320 2024-09-13T17:51:45.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3709bcc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37040c0c20 con 0x7f37040c8340 2024-09-13T17:51:45.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3709bcc700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37040c0ee0 con 0x7f37040c7f60 2024-09-13T17:51:45.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3708bca700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37040c7320 msgr2=0x7f37040c2aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:51:45.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.937+0000 7f3708bca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37040c7320 0x7f37040c2aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:51:45.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.941+0000 7f3703fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37040c7f60 0x7f37041e2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:45.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.941+0000 7f3703fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37040c7f60 0x7f37041e2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:40990/0 (socket says 172.21.15.40:40990) 2024-09-13T17:51:45.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.941+0000 7f3703fff700 1 -- 172.21.15.40:0/1656795106 learned_addr learned my addr 172.21.15.40:0/1656795106 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:51:45.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.941+0000 7f37093cb700 1 --2- 172.21.15.40:0/1656795106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37040c8340 0x7f37040ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:45.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.941+0000 7f3703fff700 1 -- 172.21.15.40:0/1656795106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37040c8340 msgr2=0x7f37040ade10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:45.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.941+0000 7f3703fff700 1 --2- 172.21.15.40:0/1656795106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37040c8340 0x7f37040ade10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.941+0000 7f3703fff700 1 -- 172.21.15.40:0/1656795106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37040c7320 msgr2=0x7f37040c2aa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:51:45.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.941+0000 7f3703fff700 1 --2- 172.21.15.40:0/1656795106 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37040c7320 0x7f37040c2aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.941+0000 7f3703fff700 1 -- 172.21.15.40:0/1656795106 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec0094b0 con 0x7f37040c7f60 2024-09-13T17:51:45.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.941+0000 7f3703fff700 1 --2- 172.21.15.40:0/1656795106 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37040c7f60 0x7f37041e2d70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f36f400b9e0 tx=0x7f36f400bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:45.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.941+0000 7f37017fa700 1 -- 172.21.15.40:0/1656795106 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f400c9b0 con 0x7f37040c7f60 2024-09-13T17:51:45.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.941+0000 7f37017fa700 1 -- 172.21.15.40:0/1656795106 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f400cb30 con 0x7f37040c7f60 2024-09-13T17:51:45.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.941+0000 7f37017fa700 1 -- 172.21.15.40:0/1656795106 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f4015470 con 0x7f37040c7f60 2024-09-13T17:51:45.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.941+0000 7f3709bcc700 1 -- 172.21.15.40:0/1656795106 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37040c11c0 con 0x7f37040c7f60 2024-09-13T17:51:45.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.941+0000 7f3709bcc700 1 -- 172.21.15.40:0/1656795106 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37040c22c0 con 0x7f37040c7f60 2024-09-13T17:51:45.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.941+0000 7f3709bcc700 1 --2- 172.21.15.40:0/1656795106 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37040c24e0 0x7f37040ae350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.941+0000 7f3709bcc700 1 -- 172.21.15.40:0/1656795106 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3704067720 con 0x7f37040c24e0 2024-09-13T17:51:45.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.941+0000 7f3708bca700 1 --2- 172.21.15.40:0/1656795106 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37040c24e0 0x7f37040ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:45.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.941+0000 7f3708bca700 1 -- 172.21.15.40:0/1656795106 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f37040c6110 con 0x7f37040c24e0 2024-09-13T17:51:45.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.945+0000 7f37017fa700 1 -- 172.21.15.40:0/1656795106 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f36f4012070 con 0x7f37040c7f60 2024-09-13T17:51:45.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.945+0000 7f3708bca700 1 --2- 172.21.15.40:0/1656795106 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37040c24e0 0x7f37040ae350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f37040c2150 tx=0x7f36ec004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:45.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.945+0000 7f37017fa700 1 --2- 172.21.15.40:0/1656795106 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f36e40666e0 0x7f36e4068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.945+0000 7f37017fa700 1 -- 172.21.15.40:0/1656795106 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f36f40842a0 con 0x7f37040c7f60 2024-09-13T17:51:45.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.945+0000 7f37017fa700 1 -- 172.21.15.40:0/1656795106 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec01ab50 con 0x7f37040c24e0 2024-09-13T17:51:45.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.945+0000 7f37093cb700 1 --2- 172.21.15.40:0/1656795106 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f36e40666e0 0x7f36e4068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:45.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.945+0000 7f37017fa700 1 -- 172.21.15.40:0/1656795106 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f36ec01f070 con 0x7f37040c24e0 2024-09-13T17:51:45.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.945+0000 7f37017fa700 1 -- 172.21.15.40:0/1656795106 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37040c24e0 msgr2=0x7f37040ae350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:45.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.945+0000 7f37017fa700 1 --2- 172.21.15.40:0/1656795106 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37040c24e0 0x7f37040ae350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f37040c2150 tx=0x7f36ec004770 comp rx=0 tx=0).stop 2024-09-13T17:51:45.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.945+0000 7f37017fa700 1 -- 172.21.15.40:0/1656795106 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37040c24e0 msgr2=0x7f37040ae350 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:51:45.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.945+0000 7f37017fa700 1 --2- 172.21.15.40:0/1656795106 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37040c24e0 0x7f37040ae350 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.945+0000 7f37093cb700 1 --2- 172.21.15.40:0/1656795106 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f36e40666e0 0x7f36e4068ba0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f36fc000f60 tx=0x7f36fc007e10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.953+0000 7f3709bcc700 1 --2- 172.21.15.40:0/1656795106 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37041589a0 0x7f37040c34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:51:45.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.953+0000 7f3709bcc700 1 -- 172.21.15.40:0/1656795106 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37040c5e30 con 0x7f37041589a0 2024-09-13T17:51:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.953+0000 7f3708bca700 1 --2- 172.21.15.40:0/1656795106 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37041589a0 0x7f37040c34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:51:45.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.953+0000 7f3708bca700 1 -- 172.21.15.40:0/1656795106 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f37040c63f0 con 0x7f37041589a0 2024-09-13T17:51:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.953+0000 7f3708bca700 1 --2- 172.21.15.40:0/1656795106 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37041589a0 0x7f37040c34a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f36ec000c00 tx=0x7f36ec009240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:51:45.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.953+0000 7f37017fa700 1 -- 172.21.15.40:0/1656795106 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec02c680 con 0x7f37041589a0 2024-09-13T17:51:45.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.953+0000 7f37017fa700 1 -- 172.21.15.40:0/1656795106 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4770 (secure 0 0 0) 0x7f36ec02c7e0 con 0x7f37041589a0 2024-09-13T17:51:45.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.953+0000 7f37017fa700 1 -- 172.21.15.40:0/1656795106 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37041589a0 msgr2=0x7f37040c34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:45.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.953+0000 7f37017fa700 1 --2- 172.21.15.40:0/1656795106 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37041589a0 0x7f37040c34a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f36ec000c00 tx=0x7f36ec009240 comp rx=0 tx=0).stop 2024-09-13T17:51:45.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.953+0000 7f37017fa700 1 -- 172.21.15.40:0/1656795106 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37041589a0 msgr2=0x7f37040c34a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:51:45.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.953+0000 7f37017fa700 1 --2- 172.21.15.40:0/1656795106 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37041589a0 0x7f37040c34a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.962 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:51:45.962 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:51:45.962 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:51:45.962 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:51:45.962 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 10, 2024-09-13T17:51:45.962 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:51:45.962 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:51:45.962 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:51:45.962 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:45.962 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T17:51:45.963 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:51:45.963 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:51:45.963 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:51:45.963 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:51:45.963 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:51:45.963 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:51:45.963 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:51:45.963 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:51:45.963 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:51:45.963 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:51:45.963 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:51:45.963 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:51:45.963 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:51:45.963 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:45.964 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:51:45.964 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:51:45.964 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:51:45.964 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:51:45.964 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:51:45.964 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:51:45.964 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:51:45.964 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:51:45.964 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:51:45.964 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:51:45.964 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:51:45.964 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:51:45.964 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:45.965 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.965 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:51:45.965 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:51:45.965 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:51:45.965 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:51:45.965 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:51:45.965 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:51:45.965 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:51:45.965 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:51:45.965 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:51:45.965 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:51:45.965 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:51:45.965 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:51:45.966 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:51:45.966 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:51:45.966 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:51:45.966 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:51:45.966 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:51:45.966 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:51:45.966 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:51:45.966 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:51:45.966 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:51:45.966 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:51:45.966 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:51:45.966 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:51:45.967 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:51:45.967 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:51:45.968 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:51:45.968 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:45.968 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:51:45.968 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.968 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:51:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:51:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:51:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:51:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:51:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:51:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:51:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:51:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:51:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:51:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:51:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:51:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:51:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:51:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:51:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:51:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:51:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:51:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:51:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:51:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:51:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:51:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:51:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:51:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:51:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:51:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:51:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:51:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:51:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:51:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:51:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:51:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:51:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:51:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:51:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:51:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:51:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:51:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:51:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:51:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:51:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:51:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:51:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:51:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:51:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:51:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:51:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T17:51:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:51:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:51:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:51:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:51:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:51:45.977 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:51:45.977 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:51:45.977 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:51:45.977 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:51:45.977 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:51:45.977 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:51:45.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.957+0000 7f3709bcc700 1 -- 172.21.15.40:0/1656795106 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f36e40666e0 msgr2=0x7f36e4068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:45.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.957+0000 7f3709bcc700 1 --2- 172.21.15.40:0/1656795106 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f36e40666e0 0x7f36e4068ba0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f36fc000f60 tx=0x7f36fc007e10 comp rx=0 tx=0).stop 2024-09-13T17:51:45.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.957+0000 7f3709bcc700 1 -- 172.21.15.40:0/1656795106 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37040c7f60 msgr2=0x7f37041e2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:51:45.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.957+0000 7f3709bcc700 1 --2- 172.21.15.40:0/1656795106 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37040c7f60 0x7f37041e2d70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f36f400b9e0 tx=0x7f36f400bcf0 comp rx=0 tx=0).stop 2024-09-13T17:51:45.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.957+0000 7f37093cb700 1 -- 172.21.15.40:0/1656795106 reap_dead start 2024-09-13T17:51:45.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.957+0000 7f37093cb700 1 -- 172.21.15.40:0/1656795106 reap_dead start 2024-09-13T17:51:45.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.957+0000 7f3709bcc700 1 -- 172.21.15.40:0/1656795106 shutdown_connections 2024-09-13T17:51:45.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.957+0000 7f3709bcc700 1 -- 172.21.15.40:0/1656795106 >> 172.21.15.40:0/1656795106 conn(0x7f3704066090 msgr2=0x7f37040c4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:51:45.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.957+0000 7f3709bcc700 1 -- 172.21.15.40:0/1656795106 shutdown_connections 2024-09-13T17:51:45.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:51:45.957+0000 7f3709bcc700 1 -- 172.21.15.40:0/1656795106 wait complete. 2024-09-13T17:51:45.980 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T17:51:45.981 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T17:52:03.725 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:52:03.728 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:52:05.981 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T17:52:05.981 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-13T17:52:05.982 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T17:52:05.982 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T17:52:05.984 INFO:tasks.ceph.mon.a:Started 2024-09-13T17:52:05.984 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T17:52:05.984 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:52:06.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.109+0000 7faa034bd700 1 Processor -- start 2024-09-13T17:52:06.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.109+0000 7faa034bd700 1 -- start start 2024-09-13T17:52:06.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.113+0000 7faa034bd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa9fc071f70 0x7fa9fc0c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.113+0000 7faa034bd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa9fc072350 0x7fa9fc150d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.113+0000 7faa034bd700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9fc0c55c0 con 0x7fa9fc0c7200 2024-09-13T17:52:06.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.113+0000 7faa034bd700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9fc06a390 con 0x7fa9fc071f70 2024-09-13T17:52:06.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.113+0000 7faa034bd700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc157710 con 0x7fa9fc072350 2024-09-13T17:52:06.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.113+0000 7faa024bb700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa9fc071f70 0x7fa9fc0c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.113+0000 7faa024bb700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa9fc071f70 0x7fa9fc0c5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57896/0 (socket says 172.21.15.40:57896) 2024-09-13T17:52:06.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.113+0000 7faa024bb700 1 -- 172.21.15.40:0/2407732523 learned_addr learned my addr 172.21.15.40:0/2407732523 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:06.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.113+0000 7faa024bb700 1 -- 172.21.15.40:0/2407732523 >> v2:172.21.15.40:3300/0 conn(0x7fa9fc071f70 msgr2=0x7fa9fc0c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:52:06.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.113+0000 7faa024bb700 1 -- 172.21.15.40:0/2407732523 >> v2:172.21.15.40:3300/0 conn(0x7fa9fc071f70 msgr2=0x7fa9fc0c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:52:06.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.113+0000 7faa024bb700 1 --2- 172.21.15.40:0/2407732523 >> v2:172.21.15.40:3300/0 conn(0x7fa9fc071f70 0x7fa9fc0c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:52:06.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.113+0000 7faa024bb700 1 --2- 172.21.15.40:0/2407732523 >> v2:172.21.15.40:3300/0 conn(0x7fa9fc071f70 0x7fa9fc0c5d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:52:06.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.113+0000 7faa01cba700 1 --2- 172.21.15.40:0/2407732523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa9fc072350 0x7fa9fc150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.113+0000 7faa01cba700 1 -- 172.21.15.40:0/2407732523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa9fc072350 msgr2=0x7fa9fc150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:52:06.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.113+0000 7faa01cba700 1 -- 172.21.15.40:0/2407732523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa9fc072350 msgr2=0x7fa9fc150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:52:06.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.113+0000 7faa01cba700 1 --2- 172.21.15.40:0/2407732523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa9fc072350 0x7fa9fc150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:52:06.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.113+0000 7faa01cba700 1 --2- 172.21.15.40:0/2407732523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa9fc072350 0x7fa9fc150d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:52:06.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.141+0000 7faa014b9700 1 -- 172.21.15.40:0/2407732523 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4056697944 0 0) 0x7fa9fc0c55c0 con 0x7fa9fc0c7200 2024-09-13T17:52:06.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.141+0000 7faa014b9700 1 -- 172.21.15.40:0/2407732523 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9e80035d0 con 0x7fa9fc0c7200 2024-09-13T17:52:06.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.141+0000 7faa014b9700 1 -- 172.21.15.40:0/2407732523 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 242634868 0 0) 0x7fa9e80035d0 con 0x7fa9fc0c7200 2024-09-13T17:52:06.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.141+0000 7faa014b9700 1 -- 172.21.15.40:0/2407732523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa9fc072350 msgr2=0x7fa9fc150d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:52:06.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.141+0000 7faa014b9700 1 --2- 172.21.15.40:0/2407732523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa9fc072350 0x7fa9fc150d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.141+0000 7faa014b9700 1 -- 172.21.15.40:0/2407732523 >> v2:172.21.15.40:3300/0 conn(0x7fa9fc071f70 msgr2=0x7fa9fc0c5d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:52:06.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.141+0000 7faa014b9700 1 --2- 172.21.15.40:0/2407732523 >> v2:172.21.15.40:3300/0 conn(0x7fa9fc071f70 0x7fa9fc0c5d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.141+0000 7faa014b9700 1 -- 172.21.15.40:0/2407732523 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9fc1579d0 con 0x7fa9fc0c7200 2024-09-13T17:52:06.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.141+0000 7faa014b9700 1 -- 172.21.15.40:0/2407732523 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fa9f8001670 con 0x7fa9fc0c7200 2024-09-13T17:52:06.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa014b9700 1 -- 172.21.15.40:0/2407732523 >> v1:172.21.15.40:6789/0 conn(0x7fa9fc0c7200 legacy=0x7fa9fc15a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa014b9700 1 --2- 172.21.15.40:0/2407732523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9e8003a20 0x7fa9e8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa014b9700 1 --2- 172.21.15.40:0/2407732523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa9fc072350 0x7fa9e8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa014b9700 1 --2- 172.21.15.40:0/2407732523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9e8004db0 0x7fa9e8005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa014b9700 1 -- 172.21.15.40:0/2407732523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc157710 con 0x7fa9e8004db0 2024-09-13T17:52:06.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa014b9700 1 -- 172.21.15.40:0/2407732523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e800bae0 con 0x7fa9e8003a20 2024-09-13T17:52:06.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa014b9700 1 -- 172.21.15.40:0/2407732523 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e800bc20 con 0x7fa9fc072350 2024-09-13T17:52:06.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa02cbc700 1 --2- 172.21.15.40:0/2407732523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9e8004db0 0x7fa9e8005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa024bb700 1 --2- 172.21.15.40:0/2407732523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9e8003a20 0x7fa9e8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa01cba700 1 --2- 172.21.15.40:0/2407732523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa9fc072350 0x7fa9e8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa02cbc700 1 -- 172.21.15.40:0/2407732523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa9fc072350 msgr2=0x7fa9e8004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:52:06.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa02cbc700 1 --2- 172.21.15.40:0/2407732523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa9fc072350 0x7fa9e8004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa02cbc700 1 -- 172.21.15.40:0/2407732523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9e8003a20 msgr2=0x7fa9e8003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:52:06.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa02cbc700 1 --2- 172.21.15.40:0/2407732523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9e8003a20 0x7fa9e8003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa02cbc700 1 -- 172.21.15.40:0/2407732523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e80035d0 con 0x7fa9e8004db0 2024-09-13T17:52:06.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa02cbc700 1 --2- 172.21.15.40:0/2407732523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9e8004db0 0x7fa9e8005230 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800c240 tx=0x7fa9e80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3e92d02539ef07 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:06.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa014b9700 1 -- 172.21.15.40:0/2407732523 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8007b80 con 0x7fa9e8004db0 2024-09-13T17:52:06.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa014b9700 1 -- 172.21.15.40:0/2407732523 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f800cd40 con 0x7fa9e8004db0 2024-09-13T17:52:06.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa014b9700 1 -- 172.21.15.40:0/2407732523 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8012660 con 0x7fa9e8004db0 2024-09-13T17:52:06.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa034bd700 1 -- 172.21.15.40:0/2407732523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9e8004db0 msgr2=0x7fa9e8005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa034bd700 1 --2- 172.21.15.40:0/2407732523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9e8004db0 0x7fa9e8005230 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800c240 tx=0x7fa9e80058a0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa02cbc700 1 -- 172.21.15.40:0/2407732523 reap_dead start 2024-09-13T17:52:06.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa034bd700 1 -- 172.21.15.40:0/2407732523 shutdown_connections 2024-09-13T17:52:06.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa034bd700 1 -- 172.21.15.40:0/2407732523 >> 172.21.15.40:0/2407732523 conn(0x7fa9fc0cabc0 msgr2=0x7fa9fc0cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:06.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa034bd700 1 -- 172.21.15.40:0/2407732523 shutdown_connections 2024-09-13T17:52:06.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.145+0000 7faa034bd700 1 -- 172.21.15.40:0/2407732523 wait complete. 2024-09-13T17:52:06.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa034bd700 1 Processor -- start 2024-09-13T17:52:06.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa034bd700 1 -- start start 2024-09-13T17:52:06.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa034bd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa9fc14ca20 0x7fa9fc155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa034bd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9fc14ce00 0x7fa9fc155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa034bd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9fc1520c0 0x7fa9fc152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa034bd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc071170 con 0x7fa9fc14ce00 2024-09-13T17:52:06.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa034bd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc154ca0 con 0x7fa9fc1520c0 2024-09-13T17:52:06.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa034bd700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc0719a0 con 0x7fa9fc14ca20 2024-09-13T17:52:06.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa01cba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9fc14ce00 0x7fa9fc155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa01cba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9fc14ce00 0x7fa9fc155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57910/0 (socket says 172.21.15.40:57910) 2024-09-13T17:52:06.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa01cba700 1 -- 172.21.15.40:0/1002435527 learned_addr learned my addr 172.21.15.40:0/1002435527 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:06.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa024bb700 1 --2- 172.21.15.40:0/1002435527 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa9fc14ca20 0x7fa9fc155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa02cbc700 1 --2- 172.21.15.40:0/1002435527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9fc1520c0 0x7fa9fc152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa01cba700 1 -- 172.21.15.40:0/1002435527 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa9fc14ca20 msgr2=0x7fa9fc155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa01cba700 1 --2- 172.21.15.40:0/1002435527 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa9fc14ca20 0x7fa9fc155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa01cba700 1 -- 172.21.15.40:0/1002435527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9fc1520c0 msgr2=0x7fa9fc152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa01cba700 1 --2- 172.21.15.40:0/1002435527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9fc1520c0 0x7fa9fc152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa01cba700 1 -- 172.21.15.40:0/1002435527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9fc1519e0 con 0x7fa9fc14ce00 2024-09-13T17:52:06.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa01cba700 1 --2- 172.21.15.40:0/1002435527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9fc14ce00 0x7fa9fc155a70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec0116b0 tx=0x7fa9ec0119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:06.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7fa9f2ffd700 1 -- 172.21.15.40:0/1002435527 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec00c070 con 0x7fa9fc14ce00 2024-09-13T17:52:06.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7fa9f2ffd700 1 -- 172.21.15.40:0/1002435527 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ec015870 con 0x7fa9fc14ce00 2024-09-13T17:52:06.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa034bd700 1 -- 172.21.15.40:0/1002435527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9fc157280 con 0x7fa9fc14ce00 2024-09-13T17:52:06.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7faa034bd700 1 -- 172.21.15.40:0/1002435527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9fc151ca0 con 0x7fa9fc14ce00 2024-09-13T17:52:06.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.149+0000 7fa9f2ffd700 1 -- 172.21.15.40:0/1002435527 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec0095a0 con 0x7fa9fc14ce00 2024-09-13T17:52:06.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.153+0000 7fa9f2ffd700 1 -- 172.21.15.40:0/1002435527 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa9ec015c20 con 0x7fa9fc14ce00 2024-09-13T17:52:06.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.153+0000 7fa9f2ffd700 1 --2- 172.21.15.40:0/1002435527 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa9e0066730 0x7fa9e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.153+0000 7faa024bb700 1 --2- 172.21.15.40:0/1002435527 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa9e0066730 0x7fa9e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.153+0000 7fa9f2ffd700 1 -- 172.21.15.40:0/1002435527 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa9ec0845f0 con 0x7fa9fc14ce00 2024-09-13T17:52:06.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.153+0000 7faa024bb700 1 --2- 172.21.15.40:0/1002435527 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa9e0066730 0x7fa9e0068bf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40096d0 tx=0x7fa9f4009100 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:06.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.157+0000 7faa034bd700 1 -- 172.21.15.40:0/1002435527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e4005310 con 0x7fa9fc14ce00 2024-09-13T17:52:06.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.161+0000 7fa9f2ffd700 1 -- 172.21.15.40:0/1002435527 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa9ec051630 con 0x7fa9fc14ce00 2024-09-13T17:52:06.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.385+0000 7faa034bd700 1 -- 172.21.15.40:0/1002435527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa9e40055c0 con 0x7fa9fc14ce00 2024-09-13T17:52:06.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.385+0000 7fa9f2ffd700 1 -- 172.21.15.40:0/1002435527 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7fa9ec023020 con 0x7fa9fc14ce00 2024-09-13T17:52:06.387 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:52:06.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.389+0000 7faa034bd700 1 -- 172.21.15.40:0/1002435527 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa9e0066730 msgr2=0x7fa9e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.389+0000 7faa034bd700 1 --2- 172.21.15.40:0/1002435527 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa9e0066730 0x7fa9e0068bf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40096d0 tx=0x7fa9f4009100 comp rx=0 tx=0).stop 2024-09-13T17:52:06.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.389+0000 7faa034bd700 1 -- 172.21.15.40:0/1002435527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9fc14ce00 msgr2=0x7fa9fc155a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.389+0000 7faa034bd700 1 --2- 172.21.15.40:0/1002435527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9fc14ce00 0x7fa9fc155a70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec0116b0 tx=0x7fa9ec0119c0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.389+0000 7faa034bd700 1 -- 172.21.15.40:0/1002435527 shutdown_connections 2024-09-13T17:52:06.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.389+0000 7faa034bd700 1 --2- 172.21.15.40:0/1002435527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9fc1520c0 0x7fa9fc152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.389+0000 7faa034bd700 1 --2- 172.21.15.40:0/1002435527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa9fc14ce00 0x7fa9fc155a70 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.389+0000 7faa034bd700 1 --2- 172.21.15.40:0/1002435527 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa9e0066730 0x7fa9e0068bf0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.389+0000 7faa034bd700 1 --2- 172.21.15.40:0/1002435527 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa9fc14ca20 0x7fa9fc155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.389+0000 7faa034bd700 1 -- 172.21.15.40:0/1002435527 >> 172.21.15.40:0/1002435527 conn(0x7fa9fc0cabc0 msgr2=0x7fa9fc0c5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:06.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.389+0000 7faa034bd700 1 -- 172.21.15.40:0/1002435527 shutdown_connections 2024-09-13T17:52:06.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.389+0000 7faa034bd700 1 -- 172.21.15.40:0/1002435527 wait complete. 2024-09-13T17:52:06.401 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T17:52:06.401 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T17:52:06.401 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:52:06.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee6e727700 1 Processor -- start 2024-09-13T17:52:06.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee6e727700 1 -- start start 2024-09-13T17:52:06.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee6e727700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fee68153720 0x7fee680ca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee6e727700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fee680cf9b0 0x7fee6806c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee6e727700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee68159e50 con 0x7fee6806c5b0 2024-09-13T17:52:06.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee6e727700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee680c5ea0 con 0x7fee680cf9b0 2024-09-13T17:52:06.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee6e727700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6815d390 con 0x7fee68153720 2024-09-13T17:52:06.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee6df26700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fee6806c5b0 0x7fee6806aac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34590/0 (socket says 172.21.15.40:34590) 2024-09-13T17:52:06.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee6df26700 1 -- 172.21.15.40:0/697612611 learned_addr learned my addr 172.21.15.40:0/697612611 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:52:06.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee6d725700 1 --2- 172.21.15.40:0/697612611 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fee68153720 0x7fee680ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee6cf24700 1 --2- 172.21.15.40:0/697612611 >> v2:172.21.15.43:3300/0 conn(0x7fee680cf9b0 0x7fee6806c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee5ffff700 1 -- 172.21.15.40:0/697612611 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 218772739 0 0) 0x7fee68159e50 con 0x7fee6806c5b0 2024-09-13T17:52:06.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee5ffff700 1 -- 172.21.15.40:0/697612611 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee540035d0 con 0x7fee6806c5b0 2024-09-13T17:52:06.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee5ffff700 1 -- 172.21.15.40:0/697612611 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2691133986 0 0) 0x7fee540035d0 con 0x7fee6806c5b0 2024-09-13T17:52:06.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee5ffff700 1 -- 172.21.15.40:0/697612611 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fee68153720 msgr2=0x7fee680ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:52:06.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee5ffff700 1 --2- 172.21.15.40:0/697612611 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fee68153720 0x7fee680ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee5ffff700 1 -- 172.21.15.40:0/697612611 >> v2:172.21.15.43:3300/0 conn(0x7fee680cf9b0 msgr2=0x7fee6806c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:52:06.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee5ffff700 1 --2- 172.21.15.40:0/697612611 >> v2:172.21.15.43:3300/0 conn(0x7fee680cf9b0 0x7fee6806c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee5ffff700 1 -- 172.21.15.40:0/697612611 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6815a080 con 0x7fee6806c5b0 2024-09-13T17:52:06.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee5ffff700 1 -- 172.21.15.40:0/697612611 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fee600019f0 con 0x7fee6806c5b0 2024-09-13T17:52:06.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee5ffff700 1 -- 172.21.15.40:0/697612611 >> v1:172.21.15.40:6789/0 conn(0x7fee6806c5b0 legacy=0x7fee6806aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee5ffff700 1 --2- 172.21.15.40:0/697612611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee54003a20 0x7fee54003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee5ffff700 1 --2- 172.21.15.40:0/697612611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee54004870 0x7fee54004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee5ffff700 1 --2- 172.21.15.40:0/697612611 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fee68153720 0x7fee54005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee5ffff700 1 -- 172.21.15.40:0/697612611 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee680c5ea0 con 0x7fee54004870 2024-09-13T17:52:06.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee5ffff700 1 -- 172.21.15.40:0/697612611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6815d390 con 0x7fee54003a20 2024-09-13T17:52:06.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee5ffff700 1 -- 172.21.15.40:0/697612611 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5400ba30 con 0x7fee68153720 2024-09-13T17:52:06.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee6cf24700 1 --2- 172.21.15.40:0/697612611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee54004870 0x7fee54004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee6d725700 1 --2- 172.21.15.40:0/697612611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee54003a20 0x7fee54003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee6df26700 1 --2- 172.21.15.40:0/697612611 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fee68153720 0x7fee54005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee6cf24700 1 -- 172.21.15.40:0/697612611 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fee68153720 msgr2=0x7fee54005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:52:06.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee6cf24700 1 --2- 172.21.15.40:0/697612611 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fee68153720 0x7fee54005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee6cf24700 1 -- 172.21.15.40:0/697612611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee54003a20 msgr2=0x7fee54003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:52:06.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee6cf24700 1 --2- 172.21.15.40:0/697612611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee54003a20 0x7fee54003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee6cf24700 1 -- 172.21.15.40:0/697612611 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee540035d0 con 0x7fee54004870 2024-09-13T17:52:06.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee6cf24700 1 --2- 172.21.15.40:0/697612611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee54004870 0x7fee54004cf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fee58009b40 tx=0x7fee540058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292414395ac7b0f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:06.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee5ffff700 1 -- 172.21.15.40:0/697612611 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5800e6c0 con 0x7fee54004870 2024-09-13T17:52:06.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee5ffff700 1 -- 172.21.15.40:0/697612611 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5800e820 con 0x7fee54004870 2024-09-13T17:52:06.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.525+0000 7fee5ffff700 1 -- 172.21.15.40:0/697612611 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5800eb90 con 0x7fee54004870 2024-09-13T17:52:06.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.529+0000 7fee6e727700 1 -- 172.21.15.40:0/697612611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee54004870 msgr2=0x7fee54004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.529+0000 7fee6e727700 1 --2- 172.21.15.40:0/697612611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee54004870 0x7fee54004cf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fee58009b40 tx=0x7fee540058a0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.529+0000 7fee6df26700 1 -- 172.21.15.40:0/697612611 reap_dead start 2024-09-13T17:52:06.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.529+0000 7fee6e727700 1 -- 172.21.15.40:0/697612611 shutdown_connections 2024-09-13T17:52:06.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.529+0000 7fee6e727700 1 -- 172.21.15.40:0/697612611 >> 172.21.15.40:0/697612611 conn(0x7fee680666c0 msgr2=0x7fee680cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:06.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.529+0000 7fee6e727700 1 -- 172.21.15.40:0/697612611 shutdown_connections 2024-09-13T17:52:06.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.529+0000 7fee6e727700 1 -- 172.21.15.40:0/697612611 wait complete. 2024-09-13T17:52:06.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.529+0000 7fee6e727700 1 Processor -- start 2024-09-13T17:52:06.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.529+0000 7fee6e727700 1 -- start start 2024-09-13T17:52:06.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.529+0000 7fee6e727700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee68158a70 0x7fee6815aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.529+0000 7fee6e727700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fee68156a10 0x7fee6815af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.529+0000 7fee6e727700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee681f2b60 0x7fee681f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.529+0000 7fee6e727700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6815a590 con 0x7fee681f2b60 2024-09-13T17:52:06.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.529+0000 7fee6e727700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee68154750 con 0x7fee68158a70 2024-09-13T17:52:06.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.529+0000 7fee6df26700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee681f2b60 0x7fee681f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.529+0000 7fee6e727700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6815c2c0 con 0x7fee68156a10 2024-09-13T17:52:06.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.529+0000 7fee6df26700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee681f2b60 0x7fee681f2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57928/0 (socket says 172.21.15.40:57928) 2024-09-13T17:52:06.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.529+0000 7fee6df26700 1 -- 172.21.15.40:0/3033827734 learned_addr learned my addr 172.21.15.40:0/3033827734 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:06.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee6d725700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee68158a70 0x7fee6815aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee6cf24700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fee68156a10 0x7fee6815af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee6df26700 1 -- 172.21.15.40:0/3033827734 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fee68156a10 msgr2=0x7fee6815af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee6df26700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fee68156a10 0x7fee6815af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee6df26700 1 -- 172.21.15.40:0/3033827734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee68158a70 msgr2=0x7fee6815aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee6df26700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee68158a70 0x7fee6815aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee6df26700 1 -- 172.21.15.40:0/3033827734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee681549f0 con 0x7fee681f2b60 2024-09-13T17:52:06.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee6cf24700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fee68156a10 0x7fee6815af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:52:06.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee6df26700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee681f2b60 0x7fee681f2fe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fee60000f80 tx=0x7fee60007cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:06.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee5dffb700 1 -- 172.21.15.40:0/3033827734 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6000e8f0 con 0x7fee681f2b60 2024-09-13T17:52:06.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee5dffb700 1 -- 172.21.15.40:0/3033827734 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6000ea70 con 0x7fee681f2b60 2024-09-13T17:52:06.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee6e727700 1 -- 172.21.15.40:0/3033827734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee68154cd0 con 0x7fee681f2b60 2024-09-13T17:52:06.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee5dffb700 1 -- 172.21.15.40:0/3033827734 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6000ee20 con 0x7fee681f2b60 2024-09-13T17:52:06.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee6e727700 1 -- 172.21.15.40:0/3033827734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee68155290 con 0x7fee681f2b60 2024-09-13T17:52:06.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee6e727700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee681f3520 0x7fee681ef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee6e727700 1 -- 172.21.15.40:0/3033827734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee68074750 con 0x7fee681f3520 2024-09-13T17:52:06.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee6d725700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee681f3520 0x7fee681ef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee5dffb700 1 -- 172.21.15.40:0/3033827734 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fee6001d460 con 0x7fee681f2b60 2024-09-13T17:52:06.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee6d725700 1 -- 172.21.15.40:0/3033827734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee680c3ff0 con 0x7fee681f3520 2024-09-13T17:52:06.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee6d725700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee681f3520 0x7fee681ef5b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fee640146f0 tx=0x7fee64014ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:06.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee5dffb700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fee54067d00 0x7fee5406a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee6cf24700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fee54067d00 0x7fee5406a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.533+0000 7fee5dffb700 1 -- 172.21.15.40:0/3033827734 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fee60083f20 con 0x7fee681f2b60 2024-09-13T17:52:06.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.537+0000 7fee5dffb700 1 -- 172.21.15.40:0/3033827734 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64004230 con 0x7fee681f3520 2024-09-13T17:52:06.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.537+0000 7fee5dffb700 1 -- 172.21.15.40:0/3033827734 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fee64004390 con 0x7fee681f3520 2024-09-13T17:52:06.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.537+0000 7fee5dffb700 1 -- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee681f3520 msgr2=0x7fee681ef5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.537+0000 7fee5dffb700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee681f3520 0x7fee681ef5b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fee640146f0 tx=0x7fee64014ab0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.537+0000 7fee5dffb700 1 -- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee681f3520 msgr2=0x7fee681ef5b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:52:06.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.537+0000 7fee5dffb700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee681f3520 0x7fee681ef5b0 secure :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fee640146f0 tx=0x7fee64014ab0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.537+0000 7fee6cf24700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fee54067d00 0x7fee5406a1c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fee68067bd0 tx=0x7fee5801e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:06.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee6e727700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee480028f0 0x7fee48004d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee6e727700 1 -- 172.21.15.40:0/3033827734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee48007dd0 con 0x7fee480028f0 2024-09-13T17:52:06.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee6d725700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee480028f0 0x7fee48004d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee6d725700 1 -- 172.21.15.40:0/3033827734 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fee48007f90 con 0x7fee480028f0 2024-09-13T17:52:06.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee6d725700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee480028f0 0x7fee48004d90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fee640146f0 tx=0x7fee640047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:06.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee5dffb700 1 -- 172.21.15.40:0/3033827734 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64022050 con 0x7fee480028f0 2024-09-13T17:52:06.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee5dffb700 1 -- 172.21.15.40:0/3033827734 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4451 (secure 0 0 0) 0x7fee64027070 con 0x7fee480028f0 2024-09-13T17:52:06.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee5dffb700 1 -- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee480028f0 msgr2=0x7fee48004d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee5dffb700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee480028f0 0x7fee48004d90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fee640146f0 tx=0x7fee640047e0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee5dffb700 1 -- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee480028f0 msgr2=0x7fee48004d90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:52:06.552 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:52:06.552 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:52:06.552 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:52:06.552 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:52:06.552 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 14, 2024-09-13T17:52:06.553 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:52:06.553 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:52:06.553 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:52:06.553 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:52:06.553 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:06.553 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T17:52:06.553 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:52:06.553 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:52:06.553 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:52:06.553 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:06.553 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:06.553 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:06.553 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:06.553 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:06.554 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:06.554 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:06.554 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:06.554 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:06.554 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:06.554 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:06.554 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:52:06.554 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:52:06.554 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:06.554 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:06.554 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:06.554 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:06.554 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:06.554 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:06.555 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:06.555 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:06.555 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:06.555 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:06.555 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:06.555 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.555 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:52:06.555 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:52:06.555 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:52:06.555 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:52:06.555 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:52:06.555 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:52:06.555 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:52:06.555 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:52:06.556 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:52:06.556 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:52:06.556 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:52:06.556 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:52:06.556 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:52:06.556 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:52:06.556 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:52:06.556 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:52:06.556 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:52:06.556 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:06.556 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:06.556 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:06.556 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:06.556 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:06.557 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:06.557 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:06.557 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:06.557 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:06.557 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:06.557 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:06.557 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:52:06.557 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.557 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:52:06.557 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.557 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:52:06.557 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:52:06.557 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:06.557 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:06.558 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.558 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:06.558 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:52:06.558 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:06.558 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.558 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.558 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:06.558 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:52:06.558 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:06.558 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.558 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:06.558 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.558 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:52:06.558 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:52:06.559 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:06.559 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:06.559 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:06.559 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.559 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.559 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:52:06.559 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:52:06.559 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:06.559 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:06.559 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.559 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:06.559 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:52:06.559 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:06.559 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.559 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.560 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:06.560 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:52:06.560 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:06.560 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.560 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:06.560 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.560 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:52:06.560 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:52:06.560 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:06.560 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:06.560 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:06.560 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.560 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.561 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:52:06.561 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:52:06.561 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:06.561 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:06.561 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.561 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:06.561 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:52:06.561 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:06.561 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.561 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.561 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:06.561 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:52:06.561 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:06.561 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.561 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:06.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.562 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:52:06.562 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:52:06.562 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:06.562 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:06.562 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:06.562 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.562 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:06.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.562 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:52:06.562 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:52:06.562 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.562 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:06.562 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:52:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:52:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:52:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:52:06.563 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:52:06.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee5dffb700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee480028f0 0x7fee48004d90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee6e727700 1 -- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fee54067d00 msgr2=0x7fee5406a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee6e727700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fee54067d00 0x7fee5406a1c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fee68067bd0 tx=0x7fee5801e040 comp rx=0 tx=0).stop 2024-09-13T17:52:06.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee6df26700 1 -- 172.21.15.40:0/3033827734 reap_dead start 2024-09-13T17:52:06.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee6e727700 1 -- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee681f2b60 msgr2=0x7fee681f2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee6e727700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee681f2b60 0x7fee681f2fe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fee60000f80 tx=0x7fee60007cb0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee6e727700 1 -- 172.21.15.40:0/3033827734 shutdown_connections 2024-09-13T17:52:06.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee6e727700 1 --2- 172.21.15.40:0/3033827734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee681f2b60 0x7fee681f2fe0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee6e727700 1 -- 172.21.15.40:0/3033827734 >> 172.21.15.40:0/3033827734 conn(0x7fee680666c0 msgr2=0x7fee680c4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:06.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee6e727700 1 -- 172.21.15.40:0/3033827734 shutdown_connections 2024-09-13T17:52:06.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.549+0000 7fee6e727700 1 -- 172.21.15.40:0/3033827734 wait complete. 2024-09-13T17:52:06.566 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:52:06.566 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T17:52:06.566 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:52:06.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d89c4700 1 Processor -- start 2024-09-13T17:52:06.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d89c4700 1 -- start start 2024-09-13T17:52:06.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d89c4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd5d4151580 0x7fd5d40c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d89c4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5d4157f10 0x7fd5d40c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d89c4700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d4070ae0 con 0x7fd5d40b1fb0 2024-09-13T17:52:06.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d89c4700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5d4070d30 con 0x7fd5d4151580 2024-09-13T17:52:06.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d89c4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d40c29c0 con 0x7fd5d4157f10 2024-09-13T17:52:06.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d2ffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd5d40b1fb0 0x7fd5d4157800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34592/0 (socket says 172.21.15.40:34592) 2024-09-13T17:52:06.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d2ffd700 1 -- 172.21.15.40:0/3995867807 learned_addr learned my addr 172.21.15.40:0/3995867807 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:52:06.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d37fe700 1 --2- 172.21.15.40:0/3995867807 >> v2:172.21.15.43:3300/0 conn(0x7fd5d4151580 0x7fd5d40c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d3fff700 1 --2- 172.21.15.40:0/3995867807 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5d4157f10 0x7fd5d40c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d27fc700 1 -- 172.21.15.40:0/3995867807 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1295707765 0 0) 0x7fd5d4070ae0 con 0x7fd5d40b1fb0 2024-09-13T17:52:06.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d27fc700 1 -- 172.21.15.40:0/3995867807 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5bc0035d0 con 0x7fd5d40b1fb0 2024-09-13T17:52:06.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d27fc700 1 -- 172.21.15.40:0/3995867807 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2455124043 0 0) 0x7fd5bc0035d0 con 0x7fd5d40b1fb0 2024-09-13T17:52:06.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d37fe700 1 -- 172.21.15.40:0/3995867807 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5d4157f10 msgr2=0x7fd5d40c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:52:06.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d37fe700 1 --2- 172.21.15.40:0/3995867807 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5d4157f10 0x7fd5d40c57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d37fe700 1 -- 172.21.15.40:0/3995867807 >> v1:172.21.15.40:6789/0 conn(0x7fd5d40b1fb0 legacy=0x7fd5d4157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d37fe700 1 -- 172.21.15.40:0/3995867807 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d40c2ca0 con 0x7fd5d4151580 2024-09-13T17:52:06.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d3fff700 1 --2- 172.21.15.40:0/3995867807 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5d4157f10 0x7fd5d40c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:52:06.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d37fe700 1 --2- 172.21.15.40:0/3995867807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d4151580 0x7fd5d40c52b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009850 tx=0x7fd5c4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9bfd9781dc0f62e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:06.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.685+0000 7fd5d27fc700 1 -- 172.21.15.40:0/3995867807 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8001670 con 0x7fd5d4151580 2024-09-13T17:52:06.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d27fc700 1 -- 172.21.15.40:0/3995867807 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c4004030 con 0x7fd5d4151580 2024-09-13T17:52:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d27fc700 1 -- 172.21.15.40:0/3995867807 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c40043e0 con 0x7fd5d4151580 2024-09-13T17:52:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d89c4700 1 -- 172.21.15.40:0/3995867807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d4151580 msgr2=0x7fd5d40c52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d89c4700 1 --2- 172.21.15.40:0/3995867807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d4151580 0x7fd5d40c52b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009850 tx=0x7fd5c4009b60 comp rx=0 tx=0).stop 2024-09-13T17:52:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d89c4700 1 -- 172.21.15.40:0/3995867807 shutdown_connections 2024-09-13T17:52:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d89c4700 1 --2- 172.21.15.40:0/3995867807 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5d4157f10 0x7fd5d40c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d89c4700 1 --2- 172.21.15.40:0/3995867807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d4151580 0x7fd5d40c52b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d89c4700 1 -- 172.21.15.40:0/3995867807 >> 172.21.15.40:0/3995867807 conn(0x7fd5d406e6d0 msgr2=0x7fd5d406eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d89c4700 1 -- 172.21.15.40:0/3995867807 shutdown_connections 2024-09-13T17:52:06.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d89c4700 1 -- 172.21.15.40:0/3995867807 wait complete. 2024-09-13T17:52:06.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d89c4700 1 Processor -- start 2024-09-13T17:52:06.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d89c4700 1 -- start start 2024-09-13T17:52:06.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d89c4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d40b1fb0 0x7fd5d40cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d89c4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5d4151580 0x7fd5d4155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d89c4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d4157f10 0x7fd5d4147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d89c4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d4147d70 con 0x7fd5d4157f10 2024-09-13T17:52:06.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d89c4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d41502f0 con 0x7fd5d40b1fb0 2024-09-13T17:52:06.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d89c4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d4156100 con 0x7fd5d4151580 2024-09-13T17:52:06.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d4157f10 0x7fd5d4147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d4157f10 0x7fd5d4147830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57952/0 (socket says 172.21.15.40:57952) 2024-09-13T17:52:06.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.689+0000 7fd5d3fff700 1 -- 172.21.15.40:0/956354580 learned_addr learned my addr 172.21.15.40:0/956354580 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:06.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5d2ffd700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5d4151580 0x7fd5d4155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5d37fe700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d40b1fb0 0x7fd5d40cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5d3fff700 1 -- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5d4151580 msgr2=0x7fd5d4155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5d3fff700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5d4151580 0x7fd5d4155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5d3fff700 1 -- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d40b1fb0 msgr2=0x7fd5d40cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5d3fff700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d40b1fb0 0x7fd5d40cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5d3fff700 1 -- 172.21.15.40:0/956354580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5cc009a70 con 0x7fd5d4157f10 2024-09-13T17:52:06.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5d37fe700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d40b1fb0 0x7fd5d40cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:52:06.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5d3fff700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d4157f10 0x7fd5d4147830 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00efc0 tx=0x7fd5cc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:06.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5bbfff700 1 -- 172.21.15.40:0/956354580 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc00cb40 con 0x7fd5d4157f10 2024-09-13T17:52:06.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5bbfff700 1 -- 172.21.15.40:0/956354580 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5cc00f040 con 0x7fd5d4157f10 2024-09-13T17:52:06.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5d89c4700 1 -- 172.21.15.40:0/956354580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c40094b0 con 0x7fd5d4157f10 2024-09-13T17:52:06.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5bbfff700 1 -- 172.21.15.40:0/956354580 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc003a40 con 0x7fd5d4157f10 2024-09-13T17:52:06.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5d89c4700 1 -- 172.21.15.40:0/956354580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d41ece80 con 0x7fd5d4157f10 2024-09-13T17:52:06.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5d89c4700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d41ed0a0 0x7fd5d40c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5d89c4700 1 -- 172.21.15.40:0/956354580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d40675e0 con 0x7fd5d41ed0a0 2024-09-13T17:52:06.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5d37fe700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d41ed0a0 0x7fd5d40c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5bbfff700 1 -- 172.21.15.40:0/956354580 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd5cc014480 con 0x7fd5d4157f10 2024-09-13T17:52:06.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5d37fe700 1 -- 172.21.15.40:0/956354580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5d4151960 con 0x7fd5d41ed0a0 2024-09-13T17:52:06.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5d37fe700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d41ed0a0 0x7fd5d40c86f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009850 tx=0x7fd5c4004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:06.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5bbfff700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd5c0066730 0x7fd5c0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5bbfff700 1 -- 172.21.15.40:0/956354580 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd5cc084f50 con 0x7fd5d4157f10 2024-09-13T17:52:06.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5d2ffd700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd5c0066730 0x7fd5c0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.693+0000 7fd5bbfff700 1 -- 172.21.15.40:0/956354580 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c401f070 con 0x7fd5d41ed0a0 2024-09-13T17:52:06.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.697+0000 7fd5d2ffd700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd5c0066730 0x7fd5c0068bf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4071720 tx=0x7fd5c8001230 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:06.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.697+0000 7fd5bbfff700 1 -- 172.21.15.40:0/956354580 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd5c40038a0 con 0x7fd5d41ed0a0 2024-09-13T17:52:06.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.697+0000 7fd5bbfff700 1 -- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d41ed0a0 msgr2=0x7fd5d40c86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.697+0000 7fd5bbfff700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d41ed0a0 0x7fd5d40c86f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009850 tx=0x7fd5c4004770 comp rx=0 tx=0).stop 2024-09-13T17:52:06.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.697+0000 7fd5bbfff700 1 -- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d41ed0a0 msgr2=0x7fd5d40c86f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:52:06.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.697+0000 7fd5bbfff700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d41ed0a0 0x7fd5d40c86f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.705+0000 7fd5d89c4700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d40c8c30 0x7fd5d41efe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.705+0000 7fd5d89c4700 1 -- 172.21.15.40:0/956354580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d4071840 con 0x7fd5d40c8c30 2024-09-13T17:52:06.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.705+0000 7fd5d37fe700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d40c8c30 0x7fd5d41efe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.705+0000 7fd5d37fe700 1 -- 172.21.15.40:0/956354580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd5d40c9040 con 0x7fd5d40c8c30 2024-09-13T17:52:06.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.705+0000 7fd5d37fe700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d40c8c30 0x7fd5d41efe80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009ba0 tx=0x7fd5c4011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.705+0000 7fd5bbfff700 1 -- 172.21.15.40:0/956354580 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c402cab0 con 0x7fd5d40c8c30 2024-09-13T17:52:06.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.705+0000 7fd5bbfff700 1 -- 172.21.15.40:0/956354580 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4947 (secure 0 0 0) 0x7fd5c402cc10 con 0x7fd5d40c8c30 2024-09-13T17:52:06.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.705+0000 7fd5bbfff700 1 -- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d40c8c30 msgr2=0x7fd5d41efe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.705+0000 7fd5bbfff700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d40c8c30 0x7fd5d41efe80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009ba0 tx=0x7fd5c4011000 comp rx=0 tx=0).stop 2024-09-13T17:52:06.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.705+0000 7fd5bbfff700 1 -- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d40c8c30 msgr2=0x7fd5d41efe80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:52:06.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.705+0000 7fd5bbfff700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d40c8c30 0x7fd5d41efe80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.710 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:52:06.710 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:52:06.710 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:52:06.710 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:52:06.710 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 14, 2024-09-13T17:52:06.711 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:52:06.711 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:52:06.711 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:52:06.711 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:52:06.711 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:06.711 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T17:52:06.711 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:52:06.711 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:52:06.711 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:52:06.711 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:06.711 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:06.711 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:06.711 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:06.711 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:06.712 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:06.712 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:06.712 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:06.712 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:06.712 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:06.712 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:06.712 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:52:06.712 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:52:06.712 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:06.712 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:06.712 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:06.712 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:06.712 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:06.712 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:06.713 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:06.713 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:06.713 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:06.713 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:06.713 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:06.713 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.713 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:52:06.713 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:52:06.713 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:52:06.713 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:52:06.714 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:52:06.714 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:52:06.714 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:52:06.714 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:52:06.714 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:52:06.714 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:52:06.714 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:52:06.714 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:52:06.714 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:52:06.714 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:52:06.714 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:52:06.714 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:52:06.715 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:52:06.715 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:06.715 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:06.715 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:06.715 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:06.715 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:06.715 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:06.715 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:06.715 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:06.715 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:06.715 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:06.715 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:06.715 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:52:06.715 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.716 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:52:06.716 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.716 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:52:06.716 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:52:06.716 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:06.716 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:06.716 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.716 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:06.716 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:52:06.716 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:06.716 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.716 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.716 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:06.716 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:52:06.717 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:06.717 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.717 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:06.717 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.717 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:52:06.717 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:52:06.717 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:06.717 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:06.717 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:06.717 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.717 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.717 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:52:06.717 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:52:06.717 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:06.718 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:06.718 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.718 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:06.718 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:52:06.718 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:06.718 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.718 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.718 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:06.718 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:52:06.718 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:06.718 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.718 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:06.718 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.719 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:52:06.719 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:52:06.719 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:06.719 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:06.719 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:06.719 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.719 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.719 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:52:06.719 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:52:06.719 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:06.719 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:06.719 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.719 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:06.719 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:52:06.719 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:06.720 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.720 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.720 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:06.720 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:52:06.720 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:06.720 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.720 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:06.720 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.720 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:52:06.720 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:52:06.720 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:06.720 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:06.720 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:06.721 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.721 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:06.721 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.721 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:52:06.721 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:52:06.721 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.721 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:06.721 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:06.721 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:52:06.721 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.721 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:06.721 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:52:06.722 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.722 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:06.722 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:06.722 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:52:06.722 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.722 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:06.722 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:52:06.722 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.722 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:06.722 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:06.722 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T17:52:06.722 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.722 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:06.722 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:52:06.723 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.723 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:06.723 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:06.723 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:52:06.723 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.723 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:06.723 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:52:06.723 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.723 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:06.723 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:06.723 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:52:06.723 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.723 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:06.723 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.723 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:52:06.724 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:52:06.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.709+0000 7fd5d89c4700 1 -- 172.21.15.40:0/956354580 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd5c0066730 msgr2=0x7fd5c0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.709+0000 7fd5d89c4700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd5c0066730 0x7fd5c0068bf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4071720 tx=0x7fd5c8001230 comp rx=0 tx=0).stop 2024-09-13T17:52:06.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.709+0000 7fd5d89c4700 1 -- 172.21.15.40:0/956354580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d4157f10 msgr2=0x7fd5d4147830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.709+0000 7fd5d3fff700 1 -- 172.21.15.40:0/956354580 reap_dead start 2024-09-13T17:52:06.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.709+0000 7fd5d89c4700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d4157f10 0x7fd5d4147830 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00efc0 tx=0x7fd5cc00c3a0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.709+0000 7fd5d89c4700 1 -- 172.21.15.40:0/956354580 shutdown_connections 2024-09-13T17:52:06.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.709+0000 7fd5d89c4700 1 --2- 172.21.15.40:0/956354580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d4157f10 0x7fd5d4147830 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.709+0000 7fd5d89c4700 1 -- 172.21.15.40:0/956354580 >> 172.21.15.40:0/956354580 conn(0x7fd5d406e6d0 msgr2=0x7fd5d40c52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:06.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.709+0000 7fd5d89c4700 1 -- 172.21.15.40:0/956354580 shutdown_connections 2024-09-13T17:52:06.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.709+0000 7fd5d89c4700 1 -- 172.21.15.40:0/956354580 wait complete. 2024-09-13T17:52:06.726 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:52:06.726 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T17:52:06.726 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:52:06.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde75c6f700 1 Processor -- start 2024-09-13T17:52:06.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde75c6f700 1 -- start start 2024-09-13T17:52:06.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde75c6f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde701503c0 0x7fde700c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde75c6f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fde700c8500 0x7fde700c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde75c6f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde700c8f80 con 0x7fde700c71c0 2024-09-13T17:52:06.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde75c6f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde700c2a80 con 0x7fde700c8500 2024-09-13T17:52:06.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde75c6f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde701507a0 con 0x7fde701503c0 2024-09-13T17:52:06.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde6ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fde700c8500 0x7fde700c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde6ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fde700c8500 0x7fde700c6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57966/0 (socket says 172.21.15.40:57966) 2024-09-13T17:52:06.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde6ffff700 1 -- 172.21.15.40:0/1683293178 learned_addr learned my addr 172.21.15.40:0/1683293178 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:06.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde74c6d700 1 --2- 172.21.15.40:0/1683293178 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde701503c0 0x7fde700c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde6ffff700 1 -- 172.21.15.40:0/1683293178 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde701503c0 msgr2=0x7fde700c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:52:06.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde6ffff700 1 --2- 172.21.15.40:0/1683293178 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde701503c0 0x7fde700c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde6ffff700 1 -- 172.21.15.40:0/1683293178 >> v1:172.21.15.43:6789/0 conn(0x7fde700c71c0 legacy=0x7fde70159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde6ffff700 1 -- 172.21.15.40:0/1683293178 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde700c8a40 con 0x7fde700c8500 2024-09-13T17:52:06.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde74c6d700 1 --2- 172.21.15.40:0/1683293178 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde701503c0 0x7fde700c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:52:06.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde6ffff700 1 --2- 172.21.15.40:0/1683293178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde700c8500 0x7fde700c6c80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009910 tx=0x7fde5c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ca4d6faa1ac4b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:06.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde6f7fe700 1 -- 172.21.15.40:0/1683293178 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c00ba60 con 0x7fde700c8500 2024-09-13T17:52:06.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde6f7fe700 1 -- 172.21.15.40:0/1683293178 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c00bbc0 con 0x7fde700c8500 2024-09-13T17:52:06.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde6f7fe700 1 -- 172.21.15.40:0/1683293178 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c009e60 con 0x7fde700c8500 2024-09-13T17:52:06.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde75c6f700 1 -- 172.21.15.40:0/1683293178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde700c8500 msgr2=0x7fde700c6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde75c6f700 1 --2- 172.21.15.40:0/1683293178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde700c8500 0x7fde700c6c80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009910 tx=0x7fde5c009c20 comp rx=0 tx=0).stop 2024-09-13T17:52:06.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde75c6f700 1 -- 172.21.15.40:0/1683293178 shutdown_connections 2024-09-13T17:52:06.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde75c6f700 1 --2- 172.21.15.40:0/1683293178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde700c8500 0x7fde700c6c80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde75c6f700 1 --2- 172.21.15.40:0/1683293178 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde701503c0 0x7fde700c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde75c6f700 1 -- 172.21.15.40:0/1683293178 >> 172.21.15.40:0/1683293178 conn(0x7fde7006e6d0 msgr2=0x7fde7006eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:06.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde75c6f700 1 -- 172.21.15.40:0/1683293178 shutdown_connections 2024-09-13T17:52:06.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.849+0000 7fde75c6f700 1 -- 172.21.15.40:0/1683293178 wait complete. 2024-09-13T17:52:06.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.853+0000 7fde75c6f700 1 Processor -- start 2024-09-13T17:52:06.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.853+0000 7fde75c6f700 1 -- start start 2024-09-13T17:52:06.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.853+0000 7fde75c6f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde700c71c0 0x7fde701eefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.853+0000 7fde75c6f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde700c8500 0x7fde700cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.853+0000 7fde75c6f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde701503c0 0x7fde701519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.853+0000 7fde75c6f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde70157900 con 0x7fde701503c0 2024-09-13T17:52:06.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.853+0000 7fde75c6f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde701ef4e0 con 0x7fde700c71c0 2024-09-13T17:52:06.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.853+0000 7fde75c6f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde701eb9c0 con 0x7fde700c8500 2024-09-13T17:52:06.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.853+0000 7fde7546e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde701503c0 0x7fde701519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.853+0000 7fde7546e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde701503c0 0x7fde701519c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57984/0 (socket says 172.21.15.40:57984) 2024-09-13T17:52:06.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.853+0000 7fde7546e700 1 -- 172.21.15.40:0/2767080428 learned_addr learned my addr 172.21.15.40:0/2767080428 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:06.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.853+0000 7fde74c6d700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde700c71c0 0x7fde701eefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.853+0000 7fde6ffff700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde700c8500 0x7fde700cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.853+0000 7fde7546e700 1 -- 172.21.15.40:0/2767080428 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde700c8500 msgr2=0x7fde700cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.853+0000 7fde7546e700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde700c8500 0x7fde700cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.853+0000 7fde7546e700 1 -- 172.21.15.40:0/2767080428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde700c71c0 msgr2=0x7fde701eefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.853+0000 7fde7546e700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde700c71c0 0x7fde701eefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.853+0000 7fde7546e700 1 -- 172.21.15.40:0/2767080428 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c009570 con 0x7fde701503c0 2024-09-13T17:52:06.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.853+0000 7fde7546e700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde701503c0 0x7fde701519c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fde6800cc80 tx=0x7fde6800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:06.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.853+0000 7fde6d7fa700 1 -- 172.21.15.40:0/2767080428 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde68003c20 con 0x7fde701503c0 2024-09-13T17:52:06.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.857+0000 7fde6d7fa700 1 -- 172.21.15.40:0/2767080428 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde68003d80 con 0x7fde701503c0 2024-09-13T17:52:06.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.857+0000 7fde75c6f700 1 -- 172.21.15.40:0/2767080428 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde700c3430 con 0x7fde701503c0 2024-09-13T17:52:06.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.857+0000 7fde6d7fa700 1 -- 172.21.15.40:0/2767080428 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6800aa40 con 0x7fde701503c0 2024-09-13T17:52:06.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.857+0000 7fde75c6f700 1 -- 172.21.15.40:0/2767080428 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde701eb370 con 0x7fde701503c0 2024-09-13T17:52:06.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.857+0000 7fde75c6f700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde701eb500 0x7fde700c3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.857+0000 7fde75c6f700 1 -- 172.21.15.40:0/2767080428 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde700675e0 con 0x7fde701eb500 2024-09-13T17:52:06.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.857+0000 7fde74c6d700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde701eb500 0x7fde700c3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.857+0000 7fde6d7fa700 1 -- 172.21.15.40:0/2767080428 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fde6800aba0 con 0x7fde701503c0 2024-09-13T17:52:06.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.857+0000 7fde6d7fa700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fde54066760 0x7fde54068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.857+0000 7fde74c6d700 1 -- 172.21.15.40:0/2767080428 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde700c2710 con 0x7fde701eb500 2024-09-13T17:52:06.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.857+0000 7fde6d7fa700 1 -- 172.21.15.40:0/2767080428 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fde680844c0 con 0x7fde701503c0 2024-09-13T17:52:06.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.857+0000 7fde6ffff700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fde54066760 0x7fde54068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.857+0000 7fde74c6d700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde701eb500 0x7fde700c3c60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fde64011600 tx=0x7fde640119c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:06.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.857+0000 7fde6d7fa700 1 -- 172.21.15.40:0/2767080428 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64004550 con 0x7fde701eb500 2024-09-13T17:52:06.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.857+0000 7fde6ffff700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fde54066760 0x7fde54068c20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009c60 tx=0x7fde5c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:06.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.857+0000 7fde6d7fa700 1 -- 172.21.15.40:0/2767080428 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fde640036c0 con 0x7fde701eb500 2024-09-13T17:52:06.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.857+0000 7fde6d7fa700 1 -- 172.21.15.40:0/2767080428 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde701eb500 msgr2=0x7fde700c3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.857+0000 7fde6d7fa700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde701eb500 0x7fde700c3c60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fde64011600 tx=0x7fde640119c0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.857+0000 7fde6d7fa700 1 -- 172.21.15.40:0/2767080428 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde701eb500 msgr2=0x7fde700c3c60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:52:06.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.857+0000 7fde6d7fa700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde701eb500 0x7fde700c3c60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.869+0000 7fde75c6f700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde700c2f10 0x7fde700c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.869+0000 7fde75c6f700 1 -- 172.21.15.40:0/2767080428 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde700c8ce0 con 0x7fde700c2f10 2024-09-13T17:52:06.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.873+0000 7fde74c6d700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde700c2f10 0x7fde700c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.873+0000 7fde74c6d700 1 -- 172.21.15.40:0/2767080428 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fde700b1e60 con 0x7fde700c2f10 2024-09-13T17:52:06.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.873+0000 7fde74c6d700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde700c2f10 0x7fde700c41a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fde64011600 tx=0x7fde64004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:06.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.873+0000 7fde6d7fa700 1 -- 172.21.15.40:0/2767080428 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64025070 con 0x7fde700c2f10 2024-09-13T17:52:06.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.873+0000 7fde6d7fa700 1 -- 172.21.15.40:0/2767080428 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4781 (secure 0 0 0) 0x7fde640202e0 con 0x7fde700c2f10 2024-09-13T17:52:06.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.873+0000 7fde6d7fa700 1 -- 172.21.15.40:0/2767080428 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde700c2f10 msgr2=0x7fde700c41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.873+0000 7fde6d7fa700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde700c2f10 0x7fde700c41a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fde64011600 tx=0x7fde64004770 comp rx=0 tx=0).stop 2024-09-13T17:52:06.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.873+0000 7fde6d7fa700 1 -- 172.21.15.40:0/2767080428 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde700c2f10 msgr2=0x7fde700c41a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:52:06.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.873+0000 7fde6d7fa700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fde700c2f10 0x7fde700c41a0 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fde64011600 tx=0x7fde64004770 comp rx=0 tx=0).stop 2024-09-13T17:52:06.876 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:52:06.876 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:52:06.876 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:52:06.876 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:52:06.876 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 14, 2024-09-13T17:52:06.876 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:52:06.876 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:52:06.876 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:52:06.877 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:52:06.877 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:06.877 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T17:52:06.877 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:52:06.877 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:52:06.877 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:52:06.877 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:06.877 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:06.877 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:06.877 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:06.877 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:06.877 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:06.877 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:06.878 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:06.878 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:06.878 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:06.878 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:06.878 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:52:06.878 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:52:06.878 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:06.878 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:06.878 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:06.878 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:06.878 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:06.878 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:06.878 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:06.879 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:06.879 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:06.879 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:06.879 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:06.879 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.879 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:52:06.879 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:52:06.879 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:52:06.879 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:52:06.879 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:52:06.879 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:52:06.879 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:52:06.879 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:52:06.879 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:52:06.880 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:52:06.880 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:52:06.880 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:52:06.880 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:52:06.880 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:52:06.880 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:52:06.880 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:52:06.880 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:52:06.880 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:06.880 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:06.880 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:06.880 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:06.880 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:06.881 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:06.881 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:06.881 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:06.881 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:06.881 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:06.881 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:06.881 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:52:06.881 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.881 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:52:06.881 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.881 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:52:06.881 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:52:06.881 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:06.882 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:06.882 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.882 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:06.882 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:52:06.882 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:06.882 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.882 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.882 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:06.882 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:52:06.882 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:06.882 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.882 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:06.882 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.882 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:52:06.883 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:52:06.883 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:06.883 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:06.883 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:06.883 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.883 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.883 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:52:06.883 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:52:06.883 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:06.883 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:06.883 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.883 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:06.883 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:52:06.883 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:06.884 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.884 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.884 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:06.884 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:52:06.884 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:06.884 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.884 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:06.884 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.884 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:52:06.884 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:52:06.884 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:06.884 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:06.885 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:06.885 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.885 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.885 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:52:06.885 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:52:06.885 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:06.885 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:06.885 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.885 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:06.885 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:52:06.885 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:06.885 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.885 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.885 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:06.886 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:52:06.886 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:06.886 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.886 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:06.886 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.886 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:52:06.886 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:52:06.886 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:06.886 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:06.886 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:06.886 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.886 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:06.886 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.886 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:52:06.887 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:52:06.887 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.887 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:06.887 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:06.887 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:52:06.887 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.887 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:06.887 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:52:06.887 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.887 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:06.887 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:06.887 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:52:06.887 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.887 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:06.888 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:52:06.888 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.888 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:06.888 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:06.888 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T17:52:06.888 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.888 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:06.888 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:52:06.888 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:06.888 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:06.888 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:06.888 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:52:06.888 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:06.888 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:06.889 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:06.889 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:52:06.889 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:52:06.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.873+0000 7fde75c6f700 1 -- 172.21.15.40:0/2767080428 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fde54066760 msgr2=0x7fde54068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.873+0000 7fde75c6f700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fde54066760 0x7fde54068c20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009c60 tx=0x7fde5c008030 comp rx=0 tx=0).stop 2024-09-13T17:52:06.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.873+0000 7fde75c6f700 1 -- 172.21.15.40:0/2767080428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde701503c0 msgr2=0x7fde701519c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.873+0000 7fde7546e700 1 -- 172.21.15.40:0/2767080428 reap_dead start 2024-09-13T17:52:06.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.873+0000 7fde75c6f700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde701503c0 0x7fde701519c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fde6800cc80 tx=0x7fde6800cf90 comp rx=0 tx=0).stop 2024-09-13T17:52:06.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.873+0000 7fde75c6f700 1 -- 172.21.15.40:0/2767080428 shutdown_connections 2024-09-13T17:52:06.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.873+0000 7fde75c6f700 1 --2- 172.21.15.40:0/2767080428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fde701503c0 0x7fde701519c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.873+0000 7fde75c6f700 1 -- 172.21.15.40:0/2767080428 >> 172.21.15.40:0/2767080428 conn(0x7fde7006e6d0 msgr2=0x7fde70153d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:06.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.873+0000 7fde75c6f700 1 -- 172.21.15.40:0/2767080428 shutdown_connections 2024-09-13T17:52:06.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.873+0000 7fde75c6f700 1 -- 172.21.15.40:0/2767080428 wait complete. 2024-09-13T17:52:06.891 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:52:06.891 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T17:52:06.891 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T17:52:06.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b971e700 1 Processor -- start 2024-09-13T17:52:06.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b971e700 1 -- start start 2024-09-13T17:52:06.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b971e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1b40cc4f0 0x7fa1b40c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b971e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa1b40c9e20 0x7fa1b40cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b971e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b40cc060 con 0x7fa1b4153840 2024-09-13T17:52:06.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b971e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b40b2840 con 0x7fa1b40cc4f0 2024-09-13T17:52:06.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b971e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b40ca870 con 0x7fa1b40c9e20 2024-09-13T17:52:06.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b37fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1b40cc4f0 0x7fa1b40c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b37fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa1b40cc4f0 0x7fa1b40c3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57998/0 (socket says 172.21.15.40:57998) 2024-09-13T17:52:06.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b37fe700 1 -- 172.21.15.40:0/819824953 learned_addr learned my addr 172.21.15.40:0/819824953 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:06.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b8f1d700 1 --2- 172.21.15.40:0/819824953 >> v2:172.21.15.43:3300/0 conn(0x7fa1b40c9e20 0x7fa1b40cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b37fe700 1 -- 172.21.15.40:0/819824953 >> v2:172.21.15.43:3300/0 conn(0x7fa1b40c9e20 msgr2=0x7fa1b40cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:52:06.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b37fe700 1 --2- 172.21.15.40:0/819824953 >> v2:172.21.15.43:3300/0 conn(0x7fa1b40c9e20 0x7fa1b40cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b37fe700 1 -- 172.21.15.40:0/819824953 >> v1:172.21.15.43:6789/0 conn(0x7fa1b4153840 legacy=0x7fa1b415d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b37fe700 1 -- 172.21.15.40:0/819824953 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b4153c20 con 0x7fa1b40cc4f0 2024-09-13T17:52:06.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b37fe700 1 --2- 172.21.15.40:0/819824953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1b40cc4f0 0x7fa1b40c3eb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa1b406c5b0 tx=0x7fa1a8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90656d6ba452033b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:06.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b2ffd700 1 -- 172.21.15.40:0/819824953 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8011830 con 0x7fa1b40cc4f0 2024-09-13T17:52:06.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b2ffd700 1 -- 172.21.15.40:0/819824953 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a8011990 con 0x7fa1b40cc4f0 2024-09-13T17:52:06.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b2ffd700 1 -- 172.21.15.40:0/819824953 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8011d20 con 0x7fa1b40cc4f0 2024-09-13T17:52:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b971e700 1 -- 172.21.15.40:0/819824953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1b40cc4f0 msgr2=0x7fa1b40c3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b971e700 1 --2- 172.21.15.40:0/819824953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1b40cc4f0 0x7fa1b40c3eb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa1b406c5b0 tx=0x7fa1a8009bf0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b971e700 1 -- 172.21.15.40:0/819824953 shutdown_connections 2024-09-13T17:52:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b971e700 1 --2- 172.21.15.40:0/819824953 >> v2:172.21.15.43:3300/0 conn(0x7fa1b40c9e20 0x7fa1b40cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b971e700 1 --2- 172.21.15.40:0/819824953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1b40cc4f0 0x7fa1b40c3eb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b971e700 1 -- 172.21.15.40:0/819824953 >> 172.21.15.40:0/819824953 conn(0x7fa1b406c800 msgr2=0x7fa1b406cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b971e700 1 -- 172.21.15.40:0/819824953 shutdown_connections 2024-09-13T17:52:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.989+0000 7fa1b971e700 1 -- 172.21.15.40:0/819824953 wait complete. 2024-09-13T17:52:06.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b971e700 1 Processor -- start 2024-09-13T17:52:06.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b971e700 1 -- start start 2024-09-13T17:52:06.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b971e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1b40c9e20 0x7fa1b414eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b971e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b40cc4f0 0x7fa1b41f2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b971e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1b4153840 0x7fa1b4149300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:06.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b971e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b41f2410 con 0x7fa1b4153840 2024-09-13T17:52:06.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b971e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b4154b00 con 0x7fa1b40cc4f0 2024-09-13T17:52:06.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b971e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b41f2100 con 0x7fa1b40c9e20 2024-09-13T17:52:06.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b8f1d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1b4153840 0x7fa1b4149300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b8f1d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1b4153840 0x7fa1b4149300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58004/0 (socket says 172.21.15.40:58004) 2024-09-13T17:52:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b8f1d700 1 -- 172.21.15.40:0/2098234048 learned_addr learned my addr 172.21.15.40:0/2098234048 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b3fff700 1 --2- 172.21.15.40:0/2098234048 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1b40c9e20 0x7fa1b414eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b37fe700 1 --2- 172.21.15.40:0/2098234048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b40cc4f0 0x7fa1b41f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b8f1d700 1 -- 172.21.15.40:0/2098234048 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1b40c9e20 msgr2=0x7fa1b414eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b8f1d700 1 --2- 172.21.15.40:0/2098234048 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1b40c9e20 0x7fa1b414eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b8f1d700 1 -- 172.21.15.40:0/2098234048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b40cc4f0 msgr2=0x7fa1b41f2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b8f1d700 1 --2- 172.21.15.40:0/2098234048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b40cc4f0 0x7fa1b41f2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b8f1d700 1 -- 172.21.15.40:0/2098234048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a8009570 con 0x7fa1b4153840 2024-09-13T17:52:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b37fe700 1 --2- 172.21.15.40:0/2098234048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b40cc4f0 0x7fa1b41f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:52:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b8f1d700 1 --2- 172.21.15.40:0/2098234048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1b4153840 0x7fa1b4149300 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00efc0 tx=0x7fa1ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b0ff9700 1 -- 172.21.15.40:0/2098234048 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac00cd40 con 0x7fa1b4153840 2024-09-13T17:52:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b0ff9700 1 -- 172.21.15.40:0/2098234048 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ac00f040 con 0x7fa1b4153840 2024-09-13T17:52:06.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b971e700 1 -- 172.21.15.40:0/2098234048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b4153f80 con 0x7fa1b4153840 2024-09-13T17:52:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b0ff9700 1 -- 172.21.15.40:0/2098234048 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac0146c0 con 0x7fa1b4153840 2024-09-13T17:52:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.993+0000 7fa1b971e700 1 -- 172.21.15.40:0/2098234048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b4154540 con 0x7fa1b4153840 2024-09-13T17:52:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.997+0000 7fa1b0ff9700 1 -- 172.21.15.40:0/2098234048 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa1ac014820 con 0x7fa1b4153840 2024-09-13T17:52:06.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.997+0000 7fa1b0ff9700 1 --2- 172.21.15.40:0/2098234048 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1a0066760 0x7fa1a0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:07.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.997+0000 7fa1b0ff9700 1 -- 172.21.15.40:0/2098234048 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa1ac084e60 con 0x7fa1b4153840 2024-09-13T17:52:07.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.997+0000 7fa1b3fff700 1 --2- 172.21.15.40:0/2098234048 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1a0066760 0x7fa1a0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:07.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:06.997+0000 7fa1b3fff700 1 --2- 172.21.15.40:0/2098234048 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1a0066760 0x7fa1a0068c20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4000f60 tx=0x7fa1a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:07.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:07.001+0000 7fa19a7fc700 1 -- 172.21.15.40:0/2098234048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b41f2550 con 0x7fa1b4153840 2024-09-13T17:52:07.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:07.009+0000 7fa1b0ff9700 1 -- 172.21.15.40:0/2098234048 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa1ac0b0db0 con 0x7fa1b4153840 2024-09-13T17:52:07.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:07.233+0000 7fa19a7fc700 1 -- 172.21.15.40:0/2098234048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fa1b40cbcf0 con 0x7fa1b4153840 2024-09-13T17:52:07.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:07.233+0000 7fa1b0ff9700 1 -- 172.21.15.40:0/2098234048 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fa1b41f2550 con 0x7fa1b4153840 2024-09-13T17:52:07.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:07.237+0000 7fa19a7fc700 1 -- 172.21.15.40:0/2098234048 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1a0066760 msgr2=0x7fa1a0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:07.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:07.237+0000 7fa19a7fc700 1 --2- 172.21.15.40:0/2098234048 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1a0066760 0x7fa1a0068c20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4000f60 tx=0x7fa1a4009040 comp rx=0 tx=0).stop 2024-09-13T17:52:07.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:07.237+0000 7fa19a7fc700 1 -- 172.21.15.40:0/2098234048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1b4153840 msgr2=0x7fa1b4149300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:07.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:07.237+0000 7fa19a7fc700 1 --2- 172.21.15.40:0/2098234048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1b4153840 0x7fa1b4149300 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00efc0 tx=0x7fa1ac00c3a0 comp rx=0 tx=0).stop 2024-09-13T17:52:07.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:07.237+0000 7fa19a7fc700 1 -- 172.21.15.40:0/2098234048 shutdown_connections 2024-09-13T17:52:07.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:07.237+0000 7fa19a7fc700 1 --2- 172.21.15.40:0/2098234048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1b4153840 0x7fa1b4149300 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:07.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:07.237+0000 7fa19a7fc700 1 --2- 172.21.15.40:0/2098234048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1b40cc4f0 0x7fa1b41f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:07.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:07.237+0000 7fa19a7fc700 1 --2- 172.21.15.40:0/2098234048 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1a0066760 0x7fa1a0068c20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:07.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:07.237+0000 7fa19a7fc700 1 --2- 172.21.15.40:0/2098234048 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1b40c9e20 0x7fa1b414eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:07.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:07.237+0000 7fa19a7fc700 1 -- 172.21.15.40:0/2098234048 >> 172.21.15.40:0/2098234048 conn(0x7fa1b406c800 msgr2=0x7fa1b406cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:07.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:07.237+0000 7fa19a7fc700 1 -- 172.21.15.40:0/2098234048 shutdown_connections 2024-09-13T17:52:07.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:07.237+0000 7fa19a7fc700 1 -- 172.21.15.40:0/2098234048 wait complete. 2024-09-13T17:52:07.250 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T17:52:17.251 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T17:52:17.252 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:52:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f2753fff700 1 Processor -- start 2024-09-13T17:52:17.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f2753fff700 1 -- start start 2024-09-13T17:52:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f2753fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27540702d0 0x7f27540c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f2753fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f27540c8af0 0x7f27540c7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f2753fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27540c5920 con 0x7f2754150390 2024-09-13T17:52:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f2753fff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27540c3430 con 0x7f27540c8af0 2024-09-13T17:52:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f2753fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27540c2aa0 con 0x7f27540702d0 2024-09-13T17:52:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f274bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27540702d0 0x7f27540c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f27537fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f27540c8af0 0x7f27540c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f274bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27540702d0 0x7f27540c6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:34322/0 (socket says 172.21.15.40:34322) 2024-09-13T17:52:17.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f274bfff700 1 -- 172.21.15.40:0/4154971504 learned_addr learned my addr 172.21.15.40:0/4154971504 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f274bfff700 1 -- 172.21.15.40:0/4154971504 >> v2:172.21.15.43:3300/0 conn(0x7f27540c8af0 msgr2=0x7f27540c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:52:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f274bfff700 1 --2- 172.21.15.40:0/4154971504 >> v2:172.21.15.43:3300/0 conn(0x7f27540c8af0 0x7f27540c7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f274bfff700 1 -- 172.21.15.40:0/4154971504 >> v1:172.21.15.43:6789/0 conn(0x7f2754150390 legacy=0x7f2754156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f274bfff700 1 -- 172.21.15.40:0/4154971504 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27540c5640 con 0x7f27540702d0 2024-09-13T17:52:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f274bfff700 1 --2- 172.21.15.40:0/4154971504 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27540702d0 0x7f27540c6ef0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f273c009910 tx=0x7f273c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec688ab9d5390c66 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f27527fc700 1 -- 172.21.15.40:0/4154971504 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c011830 con 0x7f27540702d0 2024-09-13T17:52:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f27527fc700 1 -- 172.21.15.40:0/4154971504 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273c011990 con 0x7f27540702d0 2024-09-13T17:52:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f27527fc700 1 -- 172.21.15.40:0/4154971504 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273c011d40 con 0x7f27540702d0 2024-09-13T17:52:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f2753fff700 1 -- 172.21.15.40:0/4154971504 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27540702d0 msgr2=0x7f27540c6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f2753fff700 1 --2- 172.21.15.40:0/4154971504 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27540702d0 0x7f27540c6ef0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f273c009910 tx=0x7f273c009c20 comp rx=0 tx=0).stop 2024-09-13T17:52:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f2753fff700 1 -- 172.21.15.40:0/4154971504 shutdown_connections 2024-09-13T17:52:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f2753fff700 1 --2- 172.21.15.40:0/4154971504 >> v2:172.21.15.43:3300/0 conn(0x7f27540c8af0 0x7f27540c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f2753fff700 1 --2- 172.21.15.40:0/4154971504 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27540702d0 0x7f27540c6ef0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f2753fff700 1 -- 172.21.15.40:0/4154971504 >> 172.21.15.40:0/4154971504 conn(0x7f275406c880 msgr2=0x7f275406a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f2753fff700 1 -- 172.21.15.40:0/4154971504 shutdown_connections 2024-09-13T17:52:17.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.376+0000 7f2753fff700 1 -- 172.21.15.40:0/4154971504 wait complete. 2024-09-13T17:52:17.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2753fff700 1 Processor -- start 2024-09-13T17:52:17.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2753fff700 1 -- start start 2024-09-13T17:52:17.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2753fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27540702d0 0x7f275414dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2753fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27540c8af0 0x7f2754144f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2753fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2754150390 0x7f2754071150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2753fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27541476b0 con 0x7f27540702d0 2024-09-13T17:52:17.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2753fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2754146a30 con 0x7f2754150390 2024-09-13T17:52:17.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2753fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2754146d10 con 0x7f27540c8af0 2024-09-13T17:52:17.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2752ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27540702d0 0x7f275414dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2752ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27540702d0 0x7f275414dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60906/0 (socket says 172.21.15.40:60906) 2024-09-13T17:52:17.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2752ffd700 1 -- 172.21.15.40:0/3932679894 learned_addr learned my addr 172.21.15.40:0/3932679894 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:17.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f274bfff700 1 --2- 172.21.15.40:0/3932679894 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27540c8af0 0x7f2754144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f27537fe700 1 --2- 172.21.15.40:0/3932679894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2754150390 0x7f2754071150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2752ffd700 1 -- 172.21.15.40:0/3932679894 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27540c8af0 msgr2=0x7f2754144f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2752ffd700 1 --2- 172.21.15.40:0/3932679894 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27540c8af0 0x7f2754144f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2752ffd700 1 -- 172.21.15.40:0/3932679894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2754150390 msgr2=0x7f2754071150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2752ffd700 1 --2- 172.21.15.40:0/3932679894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2754150390 0x7f2754071150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2752ffd700 1 -- 172.21.15.40:0/3932679894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c009570 con 0x7f27540702d0 2024-09-13T17:52:17.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f27537fe700 1 --2- 172.21.15.40:0/3932679894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2754150390 0x7f2754071150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:52:17.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2752ffd700 1 --2- 172.21.15.40:0/3932679894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27540702d0 0x7f275414dbb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f274400c500 tx=0x7f274400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:17.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2750ff9700 1 -- 172.21.15.40:0/3932679894 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27440052d0 con 0x7f27540702d0 2024-09-13T17:52:17.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2753fff700 1 -- 172.21.15.40:0/3932679894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2754146ff0 con 0x7f27540702d0 2024-09-13T17:52:17.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2753fff700 1 -- 172.21.15.40:0/3932679894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2754145460 con 0x7f27540702d0 2024-09-13T17:52:17.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2750ff9700 1 -- 172.21.15.40:0/3932679894 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2744005430 con 0x7f27540702d0 2024-09-13T17:52:17.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.380+0000 7f2750ff9700 1 -- 172.21.15.40:0/3932679894 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27440055a0 con 0x7f27540702d0 2024-09-13T17:52:17.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.384+0000 7f2753fff700 1 -- 172.21.15.40:0/3932679894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2738005310 con 0x7f27540702d0 2024-09-13T17:52:17.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.384+0000 7f2750ff9700 1 -- 172.21.15.40:0/3932679894 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f27440038a0 con 0x7f27540702d0 2024-09-13T17:52:17.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.384+0000 7f2750ff9700 1 --2- 172.21.15.40:0/3932679894 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2734066760 0x7f2734068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.384+0000 7f274bfff700 1 --2- 172.21.15.40:0/3932679894 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2734066760 0x7f2734068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.384+0000 7f2750ff9700 1 -- 172.21.15.40:0/3932679894 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2744083760 con 0x7f27540702d0 2024-09-13T17:52:17.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.384+0000 7f274bfff700 1 --2- 172.21.15.40:0/3932679894 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2734066760 0x7f2734068c20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f27540725a0 tx=0x7f273c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:17.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.388+0000 7f2750ff9700 1 -- 172.21.15.40:0/3932679894 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2744050850 con 0x7f27540702d0 2024-09-13T17:52:17.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.616+0000 7f2753fff700 1 -- 172.21.15.40:0/3932679894 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f27380055c0 con 0x7f27540702d0 2024-09-13T17:52:17.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.616+0000 7f2750ff9700 1 -- 172.21.15.40:0/3932679894 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f27440011a0 con 0x7f27540702d0 2024-09-13T17:52:17.621 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:52:17.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.620+0000 7f2753fff700 1 -- 172.21.15.40:0/3932679894 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2734066760 msgr2=0x7f2734068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.620+0000 7f2753fff700 1 --2- 172.21.15.40:0/3932679894 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2734066760 0x7f2734068c20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f27540725a0 tx=0x7f273c008030 comp rx=0 tx=0).stop 2024-09-13T17:52:17.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.620+0000 7f2753fff700 1 -- 172.21.15.40:0/3932679894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27540702d0 msgr2=0x7f275414dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.620+0000 7f2753fff700 1 --2- 172.21.15.40:0/3932679894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27540702d0 0x7f275414dbb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f274400c500 tx=0x7f274400c8c0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.620+0000 7f2753fff700 1 -- 172.21.15.40:0/3932679894 shutdown_connections 2024-09-13T17:52:17.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.620+0000 7f2753fff700 1 --2- 172.21.15.40:0/3932679894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2754150390 0x7f2754071150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.620+0000 7f2753fff700 1 --2- 172.21.15.40:0/3932679894 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2734066760 0x7f2734068c20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.620+0000 7f2753fff700 1 --2- 172.21.15.40:0/3932679894 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27540c8af0 0x7f2754144f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.620+0000 7f2753fff700 1 --2- 172.21.15.40:0/3932679894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27540702d0 0x7f275414dbb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.620+0000 7f2753fff700 1 -- 172.21.15.40:0/3932679894 >> 172.21.15.40:0/3932679894 conn(0x7f275406c880 msgr2=0x7f27540c3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:17.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.620+0000 7f2753fff700 1 -- 172.21.15.40:0/3932679894 shutdown_connections 2024-09-13T17:52:17.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.620+0000 7f2753fff700 1 -- 172.21.15.40:0/3932679894 wait complete. 2024-09-13T17:52:17.633 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T17:52:17.634 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T17:52:17.634 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T17:52:17.634 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:52:17.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.736+0000 7fd052f15700 1 Processor -- start 2024-09-13T17:52:17.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.736+0000 7fd052f15700 1 -- start start 2024-09-13T17:52:17.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.736+0000 7fd052f15700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd04c151050 0x7fd04c066220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.736+0000 7fd052f15700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd04c157d40 0x7fd04c0c2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.736+0000 7fd052f15700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04c1578b0 con 0x7fd04c0cb2a0 2024-09-13T17:52:17.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.736+0000 7fd052f15700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04c157090 con 0x7fd04c151050 2024-09-13T17:52:17.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.736+0000 7fd052f15700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c0c6630 con 0x7fd04c157d40 2024-09-13T17:52:17.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.736+0000 7fd051f13700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd04c151050 0x7fd04c066220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.736+0000 7fd051f13700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd04c151050 0x7fd04c066220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60918/0 (socket says 172.21.15.40:60918) 2024-09-13T17:52:17.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.736+0000 7fd051f13700 1 -- 172.21.15.40:0/1208110580 learned_addr learned my addr 172.21.15.40:0/1208110580 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:17.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.736+0000 7fd051712700 1 --2- 172.21.15.40:0/1208110580 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd04c157d40 0x7fd04c0c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.736+0000 7fd051f13700 1 -- 172.21.15.40:0/1208110580 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd04c157d40 msgr2=0x7fd04c0c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:52:17.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.736+0000 7fd051f13700 1 --2- 172.21.15.40:0/1208110580 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd04c157d40 0x7fd04c0c2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.736+0000 7fd051f13700 1 -- 172.21.15.40:0/1208110580 >> v1:172.21.15.43:6789/0 conn(0x7fd04c0cb2a0 legacy=0x7fd04c0cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.736+0000 7fd051f13700 1 -- 172.21.15.40:0/1208110580 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c06b290 con 0x7fd04c151050 2024-09-13T17:52:17.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.736+0000 7fd051f13700 1 --2- 172.21.15.40:0/1208110580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd04c151050 0x7fd04c066220 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd048009910 tx=0x7fd048009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb23f3e67e3747df server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:17.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.736+0000 7fd050f11700 1 -- 172.21.15.40:0/1208110580 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04800ba60 con 0x7fd04c151050 2024-09-13T17:52:17.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.736+0000 7fd050f11700 1 -- 172.21.15.40:0/1208110580 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04800bbc0 con 0x7fd04c151050 2024-09-13T17:52:17.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.736+0000 7fd050f11700 1 -- 172.21.15.40:0/1208110580 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048009e60 con 0x7fd04c151050 2024-09-13T17:52:17.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd052f15700 1 -- 172.21.15.40:0/1208110580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd04c151050 msgr2=0x7fd04c066220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd052f15700 1 --2- 172.21.15.40:0/1208110580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd04c151050 0x7fd04c066220 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd048009910 tx=0x7fd048009c20 comp rx=0 tx=0).stop 2024-09-13T17:52:17.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd052f15700 1 -- 172.21.15.40:0/1208110580 shutdown_connections 2024-09-13T17:52:17.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd052f15700 1 --2- 172.21.15.40:0/1208110580 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd04c157d40 0x7fd04c0c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd052f15700 1 --2- 172.21.15.40:0/1208110580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd04c151050 0x7fd04c066220 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd052f15700 1 -- 172.21.15.40:0/1208110580 >> 172.21.15.40:0/1208110580 conn(0x7fd04c06a390 msgr2=0x7fd04c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:17.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd052f15700 1 -- 172.21.15.40:0/1208110580 shutdown_connections 2024-09-13T17:52:17.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd052f15700 1 -- 172.21.15.40:0/1208110580 wait complete. 2024-09-13T17:52:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd052f15700 1 Processor -- start 2024-09-13T17:52:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd052f15700 1 -- start start 2024-09-13T17:52:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd052f15700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd04c0cb2a0 0x7fd04c1ef8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd052f15700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd04c151050 0x7fd04c0711b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd052f15700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c157d40 0x7fd04c155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd052f15700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c14cf20 con 0x7fd04c0cb2a0 2024-09-13T17:52:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd052f15700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c1efe20 con 0x7fd04c157d40 2024-09-13T17:52:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd052f15700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c070430 con 0x7fd04c151050 2024-09-13T17:52:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd051f13700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd04c0cb2a0 0x7fd04c1ef8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd051f13700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd04c0cb2a0 0x7fd04c1ef8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60928/0 (socket says 172.21.15.40:60928) 2024-09-13T17:52:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd051f13700 1 -- 172.21.15.40:0/1104872904 learned_addr learned my addr 172.21.15.40:0/1104872904 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd051712700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd04c151050 0x7fd04c0711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd052714700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c157d40 0x7fd04c155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd051f13700 1 -- 172.21.15.40:0/1104872904 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd04c151050 msgr2=0x7fd04c0711b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd051f13700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd04c151050 0x7fd04c0711b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd051f13700 1 -- 172.21.15.40:0/1104872904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c157d40 msgr2=0x7fd04c155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd051f13700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c157d40 0x7fd04c155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd051f13700 1 -- 172.21.15.40:0/1104872904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd048009570 con 0x7fd04c0cb2a0 2024-09-13T17:52:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd051f13700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd04c0cb2a0 0x7fd04c1ef8e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd048009c60 tx=0x7fd048004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd0427fc700 1 -- 172.21.15.40:0/1104872904 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048004550 con 0x7fd04c0cb2a0 2024-09-13T17:52:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.740+0000 7fd0427fc700 1 -- 172.21.15.40:0/1104872904 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04801f070 con 0x7fd04c0cb2a0 2024-09-13T17:52:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.744+0000 7fd0427fc700 1 -- 172.21.15.40:0/1104872904 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048003900 con 0x7fd04c0cb2a0 2024-09-13T17:52:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.744+0000 7fd052f15700 1 -- 172.21.15.40:0/1104872904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04c0716f0 con 0x7fd04c0cb2a0 2024-09-13T17:52:17.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.744+0000 7fd052f15700 1 -- 172.21.15.40:0/1104872904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04c0706d0 con 0x7fd04c0cb2a0 2024-09-13T17:52:17.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.744+0000 7fd052f15700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd04c1f02d0 0x7fd04c0708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.744+0000 7fd052f15700 1 -- 172.21.15.40:0/1104872904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c067600 con 0x7fd04c1f02d0 2024-09-13T17:52:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.744+0000 7fd051712700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd04c1f02d0 0x7fd04c0708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.744+0000 7fd051712700 1 -- 172.21.15.40:0/1104872904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd04c0c2d90 con 0x7fd04c1f02d0 2024-09-13T17:52:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.744+0000 7fd051712700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd04c1f02d0 0x7fd04c0708e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd04c156db0 tx=0x7fd03c00f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.744+0000 7fd0427fc700 1 -- 172.21.15.40:0/1104872904 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c014870 con 0x7fd04c1f02d0 2024-09-13T17:52:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.744+0000 7fd0427fc700 1 -- 172.21.15.40:0/1104872904 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd048003a60 con 0x7fd04c0cb2a0 2024-09-13T17:52:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.744+0000 7fd0427fc700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd0380668c0 0x7fd038068d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.744+0000 7fd0427fc700 1 -- 172.21.15.40:0/1104872904 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd048021070 con 0x7fd04c0cb2a0 2024-09-13T17:52:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.744+0000 7fd052714700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd0380668c0 0x7fd038068d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.744+0000 7fd0427fc700 1 -- 172.21.15.40:0/1104872904 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd03c00fe90 con 0x7fd04c1f02d0 2024-09-13T17:52:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.744+0000 7fd0427fc700 1 -- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd04c1f02d0 msgr2=0x7fd04c0708e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.744+0000 7fd0427fc700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd04c1f02d0 0x7fd04c0708e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd04c156db0 tx=0x7fd03c00f810 comp rx=0 tx=0).stop 2024-09-13T17:52:17.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.744+0000 7fd0427fc700 1 -- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd04c1f02d0 msgr2=0x7fd04c0708e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:52:17.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.744+0000 7fd0427fc700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd04c1f02d0 0x7fd04c0708e0 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd04c156db0 tx=0x7fd03c00f810 comp rx=0 tx=0).stop 2024-09-13T17:52:17.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.744+0000 7fd052714700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd0380668c0 0x7fd038068d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd0440012e0 tx=0x7fd044009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:17.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.756+0000 7fd052f15700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0340028f0 0x7fd034004d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.756+0000 7fd052f15700 1 -- 172.21.15.40:0/1104872904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd034007dd0 con 0x7fd0340028f0 2024-09-13T17:52:17.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.756+0000 7fd051712700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0340028f0 0x7fd034004d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.756+0000 7fd051712700 1 -- 172.21.15.40:0/1104872904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd034007f90 con 0x7fd0340028f0 2024-09-13T17:52:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.756+0000 7fd051712700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0340028f0 0x7fd034004d90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd03c016000 tx=0x7fd03c0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.756+0000 7fd0427fc700 1 -- 172.21.15.40:0/1104872904 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c004230 con 0x7fd0340028f0 2024-09-13T17:52:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.756+0000 7fd0427fc700 1 -- 172.21.15.40:0/1104872904 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4452 (secure 0 0 0) 0x7fd03c01f2e0 con 0x7fd0340028f0 2024-09-13T17:52:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.756+0000 7fd0427fc700 1 -- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0340028f0 msgr2=0x7fd034004d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.756+0000 7fd0427fc700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0340028f0 0x7fd034004d90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd03c016000 tx=0x7fd03c0047e0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.756+0000 7fd0427fc700 1 -- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0340028f0 msgr2=0x7fd034004d90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:52:17.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.756+0000 7fd0427fc700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0340028f0 0x7fd034004d90 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd03c016000 tx=0x7fd03c0047e0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.763 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:52:17.763 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:52:17.763 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:52:17.763 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:52:17.763 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 14, 2024-09-13T17:52:17.763 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:52:17.763 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:52:17.763 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:52:17.763 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:52:17.764 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:17.764 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 11, 2024-09-13T17:52:17.764 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:52:17.764 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:52:17.764 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:52:17.764 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:17.764 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:17.764 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:17.764 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:17.764 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:17.764 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:17.764 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:17.764 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:17.765 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:17.765 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:17.765 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:17.765 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:52:17.765 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:52:17.765 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:17.765 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:17.765 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:17.765 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:17.765 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:17.765 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:17.765 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:17.765 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:17.765 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:17.766 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:17.766 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:17.766 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.766 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:52:17.766 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:52:17.766 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:52:17.766 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:52:17.766 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:52:17.766 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:52:17.766 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:52:17.767 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:52:17.767 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:52:17.767 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:52:17.767 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:52:17.767 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:52:17.767 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:52:17.767 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:52:17.767 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:52:17.767 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:52:17.767 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:52:17.767 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:17.767 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:17.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:17.767 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:17.768 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:17.768 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:17.768 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:17.768 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:17.768 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:17.768 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:17.768 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:17.768 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:52:17.768 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.768 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:52:17.768 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.768 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:52:17.768 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:52:17.768 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:17.769 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:17.769 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.769 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:17.769 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:52:17.769 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:17.769 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.769 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.769 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:17.769 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:52:17.769 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:17.769 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:17.769 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:17.769 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.769 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:52:17.770 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:52:17.770 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:17.770 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:17.770 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:17.770 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.770 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.770 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:52:17.770 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:52:17.770 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:17.770 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:17.770 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.770 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:17.770 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:52:17.770 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:17.771 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.771 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.771 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:17.771 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:52:17.771 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:17.771 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:17.771 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:17.771 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.771 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:52:17.772 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:52:17.772 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:17.772 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:17.772 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:17.772 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.772 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.772 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:52:17.772 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:52:17.772 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:17.772 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:17.772 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.772 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:17.772 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:52:17.773 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:17.773 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.773 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.773 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:17.773 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:52:17.773 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:17.773 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:17.773 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:17.773 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.773 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:52:17.773 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:52:17.773 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:17.773 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:17.773 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:17.774 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:17.774 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:17.774 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.774 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:52:17.774 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:52:17.774 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.774 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:17.774 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:17.774 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:52:17.774 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:17.774 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:17.774 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:52:17.774 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.774 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:17.775 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:17.775 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:52:17.775 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:17.775 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:17.775 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.775 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:52:17.775 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:52:17.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.760+0000 7fd052f15700 1 -- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd0380668c0 msgr2=0x7fd038068d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.760+0000 7fd052f15700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd0380668c0 0x7fd038068d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd0440012e0 tx=0x7fd044009040 comp rx=0 tx=0).stop 2024-09-13T17:52:17.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.760+0000 7fd052f15700 1 -- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd04c0cb2a0 msgr2=0x7fd04c1ef8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.760+0000 7fd052714700 1 -- 172.21.15.40:0/1104872904 reap_dead start 2024-09-13T17:52:17.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.760+0000 7fd052f15700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd04c0cb2a0 0x7fd04c1ef8e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd048009c60 tx=0x7fd048004770 comp rx=0 tx=0).stop 2024-09-13T17:52:17.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.760+0000 7fd052f15700 1 -- 172.21.15.40:0/1104872904 shutdown_connections 2024-09-13T17:52:17.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.760+0000 7fd052f15700 1 --2- 172.21.15.40:0/1104872904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd04c0cb2a0 0x7fd04c1ef8e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.760+0000 7fd052f15700 1 -- 172.21.15.40:0/1104872904 >> 172.21.15.40:0/1104872904 conn(0x7fd04c06a390 msgr2=0x7fd04c14d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:17.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.760+0000 7fd052f15700 1 -- 172.21.15.40:0/1104872904 shutdown_connections 2024-09-13T17:52:17.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.760+0000 7fd052f15700 1 -- 172.21.15.40:0/1104872904 wait complete. 2024-09-13T17:52:17.778 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:52:17.778 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T17:52:17.778 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:52:17.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.892+0000 7fc45080f700 1 Processor -- start 2024-09-13T17:52:17.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.892+0000 7fc45080f700 1 -- start start 2024-09-13T17:52:17.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.892+0000 7fc45080f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc44c153730 0x7fc44c159f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.892+0000 7fc45080f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc44c0c7fb0 0x7fc44c0c8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.892+0000 7fc45080f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc44c0c8b70 0x7fc44c15cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.892+0000 7fc45080f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44c15a450 con 0x7fc44c153730 2024-09-13T17:52:17.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.892+0000 7fc45080f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44c070ae0 con 0x7fc44c0c7fb0 2024-09-13T17:52:17.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.892+0000 7fc45080f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c071010 con 0x7fc44c0c8b70 2024-09-13T17:52:17.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.892+0000 7fc44b7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc44c153730 0x7fc44c159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.892+0000 7fc44b7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc44c153730 0x7fc44c159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60956/0 (socket says 172.21.15.40:60956) 2024-09-13T17:52:17.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.892+0000 7fc44b7fe700 1 -- 172.21.15.40:0/2625389107 learned_addr learned my addr 172.21.15.40:0/2625389107 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:17.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.892+0000 7fc44bfff700 1 --2- 172.21.15.40:0/2625389107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc44c0c8b70 0x7fc44c15cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.892+0000 7fc44affd700 1 --2- 172.21.15.40:0/2625389107 >> v2:172.21.15.43:3300/0 conn(0x7fc44c0c7fb0 0x7fc44c0c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.892+0000 7fc44b7fe700 1 -- 172.21.15.40:0/2625389107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc44c0c8b70 msgr2=0x7fc44c15cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:52:17.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.892+0000 7fc44b7fe700 1 --2- 172.21.15.40:0/2625389107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc44c0c8b70 0x7fc44c15cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.892+0000 7fc44b7fe700 1 -- 172.21.15.40:0/2625389107 >> v2:172.21.15.43:3300/0 conn(0x7fc44c0c7fb0 msgr2=0x7fc44c0c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:52:17.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.892+0000 7fc44b7fe700 1 --2- 172.21.15.40:0/2625389107 >> v2:172.21.15.43:3300/0 conn(0x7fc44c0c7fb0 0x7fc44c0c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.892+0000 7fc44b7fe700 1 -- 172.21.15.40:0/2625389107 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c072420 con 0x7fc44c153730 2024-09-13T17:52:17.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.892+0000 7fc44bfff700 1 --2- 172.21.15.40:0/2625389107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc44c0c8b70 0x7fc44c15cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:52:17.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc44b7fe700 1 --2- 172.21.15.40:0/2625389107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc44c153730 0x7fc44c159f10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc44c1570c0 tx=0x7fc43c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4d0942c4030a9c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:17.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc44a7fc700 1 -- 172.21.15.40:0/2625389107 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c00b8a0 con 0x7fc44c153730 2024-09-13T17:52:17.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc44a7fc700 1 -- 172.21.15.40:0/2625389107 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c00ba00 con 0x7fc44c153730 2024-09-13T17:52:17.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc44a7fc700 1 -- 172.21.15.40:0/2625389107 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c00bd90 con 0x7fc44c153730 2024-09-13T17:52:17.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc45080f700 1 -- 172.21.15.40:0/2625389107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc44c153730 msgr2=0x7fc44c159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc45080f700 1 --2- 172.21.15.40:0/2625389107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc44c153730 0x7fc44c159f10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc44c1570c0 tx=0x7fc43c009bf0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc45080f700 1 -- 172.21.15.40:0/2625389107 shutdown_connections 2024-09-13T17:52:17.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc45080f700 1 --2- 172.21.15.40:0/2625389107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc44c0c8b70 0x7fc44c15cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc45080f700 1 --2- 172.21.15.40:0/2625389107 >> v2:172.21.15.43:3300/0 conn(0x7fc44c0c7fb0 0x7fc44c0c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc45080f700 1 --2- 172.21.15.40:0/2625389107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc44c153730 0x7fc44c159f10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc45080f700 1 -- 172.21.15.40:0/2625389107 >> 172.21.15.40:0/2625389107 conn(0x7fc44c06e6d0 msgr2=0x7fc44c06eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:17.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc45080f700 1 -- 172.21.15.40:0/2625389107 shutdown_connections 2024-09-13T17:52:17.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc45080f700 1 -- 172.21.15.40:0/2625389107 wait complete. 2024-09-13T17:52:17.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc45080f700 1 Processor -- start 2024-09-13T17:52:17.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc45080f700 1 -- start start 2024-09-13T17:52:17.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc45080f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc44c0c7fb0 0x7fc44c155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc45080f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc44c0c8b70 0x7fc44c0adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc45080f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c153730 0x7fc44c1f1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc45080f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0c3660 con 0x7fc44c0c8b70 2024-09-13T17:52:17.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc45080f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c1f0520 con 0x7fc44c153730 2024-09-13T17:52:17.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.896+0000 7fc45080f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c1f0dc0 con 0x7fc44c0c7fb0 2024-09-13T17:52:17.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc44affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc44c0c8b70 0x7fc44c0adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc44affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc44c0c8b70 0x7fc44c0adb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60966/0 (socket says 172.21.15.40:60966) 2024-09-13T17:52:17.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc44affd700 1 -- 172.21.15.40:0/1654436492 learned_addr learned my addr 172.21.15.40:0/1654436492 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:17.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc44bfff700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c153730 0x7fc44c1f1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc44b7fe700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc44c0c7fb0 0x7fc44c155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc44affd700 1 -- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc44c0c7fb0 msgr2=0x7fc44c155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc44affd700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc44c0c7fb0 0x7fc44c155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc44affd700 1 -- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c153730 msgr2=0x7fc44c1f1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc44affd700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c153730 0x7fc44c1f1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc44affd700 1 -- 172.21.15.40:0/1654436492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c009570 con 0x7fc44c0c8b70 2024-09-13T17:52:17.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc44bfff700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c153730 0x7fc44c1f1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:52:17.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc44affd700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc44c0c8b70 0x7fc44c0adb00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc44000e810 tx=0x7fc44000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:17.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc433fff700 1 -- 172.21.15.40:0/1654436492 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc440013610 con 0x7fc44c0c8b70 2024-09-13T17:52:17.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc433fff700 1 -- 172.21.15.40:0/1654436492 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc440013790 con 0x7fc44c0c8b70 2024-09-13T17:52:17.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc45080f700 1 -- 172.21.15.40:0/1654436492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44c0c3240 con 0x7fc44c0c8b70 2024-09-13T17:52:17.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc433fff700 1 -- 172.21.15.40:0/1654436492 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc440013b40 con 0x7fc44c0c8b70 2024-09-13T17:52:17.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc45080f700 1 -- 172.21.15.40:0/1654436492 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44c156520 con 0x7fc44c0c8b70 2024-09-13T17:52:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc45080f700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c154d40 0x7fc44c1f1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc45080f700 1 -- 172.21.15.40:0/1654436492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c067600 con 0x7fc44c154d40 2024-09-13T17:52:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc44b7fe700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c154d40 0x7fc44c1f1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc44b7fe700 1 -- 172.21.15.40:0/1654436492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc44c072140 con 0x7fc44c154d40 2024-09-13T17:52:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc433fff700 1 -- 172.21.15.40:0/1654436492 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc440013ca0 con 0x7fc44c0c8b70 2024-09-13T17:52:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc44b7fe700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c154d40 0x7fc44c1f1790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc44c1570c0 tx=0x7fc43c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc433fff700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc434066890 0x7fc434068d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc433fff700 1 -- 172.21.15.40:0/1654436492 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc4400841d0 con 0x7fc44c0c8b70 2024-09-13T17:52:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc44bfff700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc434066890 0x7fc434068d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.900+0000 7fc433fff700 1 -- 172.21.15.40:0/1654436492 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c0038c0 con 0x7fc44c154d40 2024-09-13T17:52:17.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.904+0000 7fc433fff700 1 -- 172.21.15.40:0/1654436492 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc43c003a20 con 0x7fc44c154d40 2024-09-13T17:52:17.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.904+0000 7fc433fff700 1 -- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c154d40 msgr2=0x7fc44c1f1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.904+0000 7fc433fff700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c154d40 0x7fc44c1f1790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc44c1570c0 tx=0x7fc43c004770 comp rx=0 tx=0).stop 2024-09-13T17:52:17.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.904+0000 7fc433fff700 1 -- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c154d40 msgr2=0x7fc44c1f1790 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:52:17.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.904+0000 7fc433fff700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c154d40 0x7fc44c1f1790 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.904+0000 7fc44bfff700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc434066890 0x7fc434068d50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc444009bc0 tx=0x7fc444003ae0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:17.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.912+0000 7fc45080f700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c1f0660 0x7fc44c150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:17.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.916+0000 7fc45080f700 1 -- 172.21.15.40:0/1654436492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c071e60 con 0x7fc44c1f0660 2024-09-13T17:52:17.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.916+0000 7fc44b7fe700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c1f0660 0x7fc44c150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:17.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.916+0000 7fc44b7fe700 1 -- 172.21.15.40:0/1654436492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc44c072920 con 0x7fc44c1f0660 2024-09-13T17:52:17.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.916+0000 7fc44b7fe700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c1f0660 0x7fc44c150420 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc43c000c00 tx=0x7fc43c011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:17.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.916+0000 7fc433fff700 1 -- 172.21.15.40:0/1654436492 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c035440 con 0x7fc44c1f0660 2024-09-13T17:52:17.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.916+0000 7fc433fff700 1 -- 172.21.15.40:0/1654436492 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7fc43c024440 con 0x7fc44c1f0660 2024-09-13T17:52:17.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.916+0000 7fc433fff700 1 -- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c1f0660 msgr2=0x7fc44c150420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.916+0000 7fc433fff700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c1f0660 0x7fc44c150420 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc43c000c00 tx=0x7fc43c011000 comp rx=0 tx=0).stop 2024-09-13T17:52:17.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.916+0000 7fc433fff700 1 -- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c1f0660 msgr2=0x7fc44c150420 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:52:17.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.916+0000 7fc433fff700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c1f0660 0x7fc44c150420 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.921 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:52:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:52:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:52:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:52:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 14, 2024-09-13T17:52:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:52:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:52:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:52:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:52:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 11, 2024-09-13T17:52:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:52:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:52:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:52:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:17.922 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:17.923 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:52:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:52:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.924 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:52:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:52:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:52:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:52:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:52:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:52:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:52:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:52:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:52:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:52:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:52:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:52:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:52:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:52:17.925 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:52:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:52:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:52:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:17.926 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:52:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:52:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:52:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:52:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:52:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.927 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:52:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:52:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:52:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:52:17.928 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:52:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:52:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:52:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:17.929 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:52:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:52:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:52:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:52:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:17.930 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:52:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:52:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:17.931 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:52:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:52:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:52:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:52:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:17.932 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:52:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:52:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:52:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:52:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T17:52:17.933 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:52:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:52:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:52:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:52:17.934 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:17.935 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:17.935 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:17.935 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:52:17.935 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:52:17.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.916+0000 7fc45080f700 1 -- 172.21.15.40:0/1654436492 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc434066890 msgr2=0x7fc434068d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.916+0000 7fc45080f700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc434066890 0x7fc434068d50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc444009bc0 tx=0x7fc444003ae0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.916+0000 7fc45080f700 1 -- 172.21.15.40:0/1654436492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc44c0c8b70 msgr2=0x7fc44c0adb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.916+0000 7fc45080f700 1 --2- 172.21.15.40:0/1654436492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc44c0c8b70 0x7fc44c0adb00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc44000e810 tx=0x7fc44000ebd0 comp rx=0 tx=0).stop 2024-09-13T17:52:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.916+0000 7fc44bfff700 1 -- 172.21.15.40:0/1654436492 reap_dead start 2024-09-13T17:52:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.916+0000 7fc44bfff700 1 -- 172.21.15.40:0/1654436492 reap_dead start 2024-09-13T17:52:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.916+0000 7fc45080f700 1 -- 172.21.15.40:0/1654436492 shutdown_connections 2024-09-13T17:52:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.916+0000 7fc45080f700 1 -- 172.21.15.40:0/1654436492 >> 172.21.15.40:0/1654436492 conn(0x7fc44c06e6d0 msgr2=0x7fc44c0cab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.916+0000 7fc45080f700 1 -- 172.21.15.40:0/1654436492 shutdown_connections 2024-09-13T17:52:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:17.916+0000 7fc45080f700 1 -- 172.21.15.40:0/1654436492 wait complete. 2024-09-13T17:52:17.937 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:52:17.937 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T17:52:17.938 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:52:18.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d7fff700 1 Processor -- start 2024-09-13T17:52:18.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d7fff700 1 -- start start 2024-09-13T17:52:18.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa3d8153720 0x7fa3d80ca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:18.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d7fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3d8161d40 0x7fa3d8164130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:18.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d7fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d8159e50 con 0x7fa3d80cf9b0 2024-09-13T17:52:18.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d7fff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d80c5ea0 con 0x7fa3d8153720 2024-09-13T17:52:18.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d7fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d815d390 con 0x7fa3d8161d40 2024-09-13T17:52:18.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d67fc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa3d80cf9b0 0x7fa3d806c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47376/0 (socket says 172.21.15.40:47376) 2024-09-13T17:52:18.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d67fc700 1 -- 172.21.15.40:0/503917771 learned_addr learned my addr 172.21.15.40:0/503917771 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:52:18.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d6ffd700 1 --2- 172.21.15.40:0/503917771 >> v2:172.21.15.40:3300/0 conn(0x7fa3d8153720 0x7fa3d80ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:18.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d77fe700 1 --2- 172.21.15.40:0/503917771 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3d8161d40 0x7fa3d8164130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:18.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d6ffd700 1 -- 172.21.15.40:0/503917771 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3d8161d40 msgr2=0x7fa3d8164130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:52:18.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d6ffd700 1 --2- 172.21.15.40:0/503917771 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3d8161d40 0x7fa3d8164130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d6ffd700 1 -- 172.21.15.40:0/503917771 >> v1:172.21.15.40:6789/0 conn(0x7fa3d80cf9b0 legacy=0x7fa3d806c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:18.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d6ffd700 1 -- 172.21.15.40:0/503917771 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d815a080 con 0x7fa3d8153720 2024-09-13T17:52:18.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d77fe700 1 --2- 172.21.15.40:0/503917771 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3d8161d40 0x7fa3d8164130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:52:18.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d6ffd700 1 --2- 172.21.15.40:0/503917771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8153720 0x7fa3d80ca510 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009850 tx=0x7fa3c8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7b43532315f407 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:18.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d5ffb700 1 -- 172.21.15.40:0/503917771 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8011830 con 0x7fa3d8153720 2024-09-13T17:52:18.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d5ffb700 1 -- 172.21.15.40:0/503917771 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c8011990 con 0x7fa3d8153720 2024-09-13T17:52:18.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d5ffb700 1 -- 172.21.15.40:0/503917771 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8011d20 con 0x7fa3d8153720 2024-09-13T17:52:18.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d7fff700 1 -- 172.21.15.40:0/503917771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8153720 msgr2=0x7fa3d80ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:18.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d7fff700 1 --2- 172.21.15.40:0/503917771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8153720 0x7fa3d80ca510 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009850 tx=0x7fa3c8009b60 comp rx=0 tx=0).stop 2024-09-13T17:52:18.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d7fff700 1 -- 172.21.15.40:0/503917771 shutdown_connections 2024-09-13T17:52:18.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d7fff700 1 --2- 172.21.15.40:0/503917771 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3d8161d40 0x7fa3d8164130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d7fff700 1 --2- 172.21.15.40:0/503917771 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8153720 0x7fa3d80ca510 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d7fff700 1 -- 172.21.15.40:0/503917771 >> 172.21.15.40:0/503917771 conn(0x7fa3d80666c0 msgr2=0x7fa3d80cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:18.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d7fff700 1 -- 172.21.15.40:0/503917771 shutdown_connections 2024-09-13T17:52:18.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.040+0000 7fa3d7fff700 1 -- 172.21.15.40:0/503917771 wait complete. 2024-09-13T17:52:18.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d7fff700 1 Processor -- start 2024-09-13T17:52:18.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d7fff700 1 -- start start 2024-09-13T17:52:18.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d7fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3d80cf9b0 0x7fa3d80c7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:18.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8153720 0x7fa3d8154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:18.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3d8161d40 0x7fa3d81ef510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:18.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d7fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d80cf050 con 0x7fa3d8153720 2024-09-13T17:52:18.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d7fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d8156a50 con 0x7fa3d8161d40 2024-09-13T17:52:18.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d7fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d8153de0 con 0x7fa3d80cf9b0 2024-09-13T17:52:18.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8153720 0x7fa3d8154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:18.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8153720 0x7fa3d8154e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60982/0 (socket says 172.21.15.40:60982) 2024-09-13T17:52:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d67fc700 1 -- 172.21.15.40:0/2539670087 learned_addr learned my addr 172.21.15.40:0/2539670087 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d6ffd700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3d80cf9b0 0x7fa3d80c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d77fe700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3d8161d40 0x7fa3d81ef510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d67fc700 1 -- 172.21.15.40:0/2539670087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3d80cf9b0 msgr2=0x7fa3d80c7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d67fc700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3d80cf9b0 0x7fa3d80c7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d67fc700 1 -- 172.21.15.40:0/2539670087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3d8161d40 msgr2=0x7fa3d81ef510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d67fc700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3d8161d40 0x7fa3d81ef510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d67fc700 1 -- 172.21.15.40:0/2539670087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c80094b0 con 0x7fa3d8153720 2024-09-13T17:52:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d67fc700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8153720 0x7fa3d8154e30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc001eb0 tx=0x7fa3cc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3bf7fe700 1 -- 172.21.15.40:0/2539670087 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc005d90 con 0x7fa3d8153720 2024-09-13T17:52:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3bf7fe700 1 -- 172.21.15.40:0/2539670087 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc0070d0 con 0x7fa3d8153720 2024-09-13T17:52:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d7fff700 1 -- 172.21.15.40:0/2539670087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d81540c0 con 0x7fa3d8153720 2024-09-13T17:52:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3d7fff700 1 -- 172.21.15.40:0/2539670087 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d80cf2a0 con 0x7fa3d8153720 2024-09-13T17:52:18.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.044+0000 7fa3bf7fe700 1 -- 172.21.15.40:0/2539670087 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc007440 con 0x7fa3d8153720 2024-09-13T17:52:18.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.048+0000 7fa3bf7fe700 1 -- 172.21.15.40:0/2539670087 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa3cc0075a0 con 0x7fa3d8153720 2024-09-13T17:52:18.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.048+0000 7fa3bf7fe700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa3c4066730 0x7fa3c4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:18.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.048+0000 7fa3bf7fe700 1 -- 172.21.15.40:0/2539670087 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa3cc0837f0 con 0x7fa3d8153720 2024-09-13T17:52:18.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.048+0000 7fa3d6ffd700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa3c4066730 0x7fa3c4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:18.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.048+0000 7fa3d6ffd700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa3c4066730 0x7fa3c4068bf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009fd0 tx=0x7fa3c8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:18.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.052+0000 7fa3d7fff700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3b8002a40 0x7fa3b8004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:18.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.052+0000 7fa3d7fff700 1 -- 172.21.15.40:0/2539670087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b8007f40 con 0x7fa3b8002a40 2024-09-13T17:52:18.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.052+0000 7fa3d77fe700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3b8002a40 0x7fa3b8004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:18.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.052+0000 7fa3d77fe700 1 -- 172.21.15.40:0/2539670087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3b8008100 con 0x7fa3b8002a40 2024-09-13T17:52:18.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.052+0000 7fa3d77fe700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3b8002a40 0x7fa3b8004f00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80cff70 tx=0x7fa3d000c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:18.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.052+0000 7fa3bf7fe700 1 -- 172.21.15.40:0/2539670087 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d000ce50 con 0x7fa3b8002a40 2024-09-13T17:52:18.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.052+0000 7fa3bf7fe700 1 -- 172.21.15.40:0/2539670087 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa3d000a410 con 0x7fa3b8002a40 2024-09-13T17:52:18.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.052+0000 7fa3bf7fe700 1 -- 172.21.15.40:0/2539670087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3b8002a40 msgr2=0x7fa3b8004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:18.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.052+0000 7fa3bf7fe700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3b8002a40 0x7fa3b8004f00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80cff70 tx=0x7fa3d000c3c0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.052+0000 7fa3bf7fe700 1 -- 172.21.15.40:0/2539670087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3b8002a40 msgr2=0x7fa3b8004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:52:18.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.052+0000 7fa3bf7fe700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3b8002a40 0x7fa3b8004f00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.064+0000 7fa3d7fff700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3b8007020 0x7fa3b8007490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:18.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.064+0000 7fa3d7fff700 1 -- 172.21.15.40:0/2539670087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b800a400 con 0x7fa3b8007020 2024-09-13T17:52:18.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.064+0000 7fa3d77fe700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3b8007020 0x7fa3b8007490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:18.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.064+0000 7fa3d77fe700 1 -- 172.21.15.40:0/2539670087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa3b800a5c0 con 0x7fa3b8007020 2024-09-13T17:52:18.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.068+0000 7fa3d77fe700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3b8007020 0x7fa3b8007490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80cff70 tx=0x7fa3d0004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:18.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.068+0000 7fa3bf7fe700 1 -- 172.21.15.40:0/2539670087 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0018aa0 con 0x7fa3b8007020 2024-09-13T17:52:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.068+0000 7fa3bf7fe700 1 -- 172.21.15.40:0/2539670087 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4782 (secure 0 0 0) 0x7fa3d0022070 con 0x7fa3b8007020 2024-09-13T17:52:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.068+0000 7fa3bf7fe700 1 -- 172.21.15.40:0/2539670087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3b8007020 msgr2=0x7fa3b8007490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.068+0000 7fa3bf7fe700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3b8007020 0x7fa3b8007490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80cff70 tx=0x7fa3d0004af0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.068+0000 7fa3bf7fe700 1 -- 172.21.15.40:0/2539670087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3b8007020 msgr2=0x7fa3b8007490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:52:18.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.068+0000 7fa3bf7fe700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa3b8007020 0x7fa3b8007490 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.072 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:52:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:52:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:52:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:52:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 14, 2024-09-13T17:52:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:52:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:52:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:52:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:52:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 11, 2024-09-13T17:52:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:52:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:52:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:52:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:52:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:52:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:52:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:52:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:52:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:52:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:52:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:52:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:52:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:52:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:52:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:52:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:52:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:52:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:52:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:52:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:52:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:52:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:52:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:52:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:52:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:52:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:52:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:52:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:52:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:52:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:52:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:52:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:52:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:52:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:52:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:52:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:52:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:52:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:52:18.083 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:18.083 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:18.083 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:18.083 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:18.083 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:52:18.083 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:18.083 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:18.083 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:18.083 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:18.083 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:52:18.083 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:18.083 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:18.083 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:18.083 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:18.084 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:52:18.084 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:52:18.084 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:18.084 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:18.084 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:18.084 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:18.084 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:18.084 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:18.084 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:52:18.084 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:52:18.084 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:18.084 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:18.084 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:18.084 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:52:18.085 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:18.085 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:18.085 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:52:18.085 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:18.085 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:18.085 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:18.085 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:52:18.085 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:18.085 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:18.085 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:52:18.085 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:18.085 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:18.085 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:18.085 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T17:52:18.086 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:18.086 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:18.086 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:52:18.086 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:18.086 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:18.086 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:18.086 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:52:18.086 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:18.086 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:18.086 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:18.086 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:52:18.086 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:52:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.068+0000 7fa3d7fff700 1 -- 172.21.15.40:0/2539670087 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa3c4066730 msgr2=0x7fa3c4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.068+0000 7fa3d7fff700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa3c4066730 0x7fa3c4068bf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009fd0 tx=0x7fa3c8008030 comp rx=0 tx=0).stop 2024-09-13T17:52:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.068+0000 7fa3d7fff700 1 -- 172.21.15.40:0/2539670087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8153720 msgr2=0x7fa3d8154e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.068+0000 7fa3d7fff700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8153720 0x7fa3d8154e30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc001eb0 tx=0x7fa3cc00c860 comp rx=0 tx=0).stop 2024-09-13T17:52:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.068+0000 7fa3d77fe700 1 -- 172.21.15.40:0/2539670087 reap_dead start 2024-09-13T17:52:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.068+0000 7fa3d7fff700 1 -- 172.21.15.40:0/2539670087 shutdown_connections 2024-09-13T17:52:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.068+0000 7fa3d7fff700 1 --2- 172.21.15.40:0/2539670087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa3d8153720 0x7fa3d8154e30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.068+0000 7fa3d7fff700 1 -- 172.21.15.40:0/2539670087 >> 172.21.15.40:0/2539670087 conn(0x7fa3d80666c0 msgr2=0x7fa3d806c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:18.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.068+0000 7fa3d7fff700 1 -- 172.21.15.40:0/2539670087 shutdown_connections 2024-09-13T17:52:18.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.068+0000 7fa3d7fff700 1 -- 172.21.15.40:0/2539670087 wait complete. 2024-09-13T17:52:18.089 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:52:18.089 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-13T17:52:18.089 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T17:52:18.089 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-13T17:52:18.089 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-13T17:52:18.090 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T17:52:18.090 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:52:18.131 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T17:52:18.131 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T17:52:18.131 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:52:18.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.236+0000 7f890141e700 1 Processor -- start 2024-09-13T17:52:18.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.236+0000 7f890141e700 1 -- start start 2024-09-13T17:52:18.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f890141e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f88fc153880 0x7f88fc0c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:18.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f890141e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88fc0cb2f0 0x7f88fc0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:18.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f890141e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f88fc0cf3c0 0x7f88fc15cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:18.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f890141e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88fc159770 con 0x7f88fc153880 2024-09-13T17:52:18.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f890141e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88fc159a50 con 0x7f88fc0cf3c0 2024-09-13T17:52:18.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f890141e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc159d30 con 0x7f88fc0cb2f0 2024-09-13T17:52:18.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f88fbfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f88fc153880 0x7f88fc0c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:18.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f88fbfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f88fc153880 0x7f88fc0c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32806/0 (socket says 172.21.15.40:32806) 2024-09-13T17:52:18.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f88fbfff700 1 -- 172.21.15.40:0/1061796649 learned_addr learned my addr 172.21.15.40:0/1061796649 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:18.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f88f3fff700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88fc0cb2f0 msgr2=0x7f88fc0cee80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:52:18.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f88f3fff700 1 --2- 172.21.15.40:0/1061796649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88fc0cb2f0 0x7f88fc0cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:52:18.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f8900c1d700 1 --2- 172.21.15.40:0/1061796649 >> v2:172.21.15.43:3300/0 conn(0x7f88fc0cf3c0 0x7f88fc15cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:18.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f88fbfff700 1 -- 172.21.15.40:0/1061796649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88fc0cb2f0 msgr2=0x7f88fc0cee80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:52:18.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f88fbfff700 1 --2- 172.21.15.40:0/1061796649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88fc0cb2f0 0x7f88fc0cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f88fbfff700 1 -- 172.21.15.40:0/1061796649 >> v2:172.21.15.43:3300/0 conn(0x7f88fc0cf3c0 msgr2=0x7f88fc15cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:52:18.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f88fbfff700 1 --2- 172.21.15.40:0/1061796649 >> v2:172.21.15.43:3300/0 conn(0x7f88fc0cf3c0 0x7f88fc15cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f88fbfff700 1 -- 172.21.15.40:0/1061796649 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc15a010 con 0x7f88fc153880 2024-09-13T17:52:18.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f8900c1d700 1 --2- 172.21.15.40:0/1061796649 >> v2:172.21.15.43:3300/0 conn(0x7f88fc0cf3c0 0x7f88fc15cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:52:18.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f88fbfff700 1 --2- 172.21.15.40:0/1061796649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88fc153880 0x7f88fc0c9ef0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0cffc0 tx=0x7f88ec009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23450a0a8618ee03 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:18.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f88fb7fe700 1 -- 172.21.15.40:0/1061796649 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec00b8a0 con 0x7f88fc153880 2024-09-13T17:52:18.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f88fb7fe700 1 -- 172.21.15.40:0/1061796649 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ec00ba00 con 0x7f88fc153880 2024-09-13T17:52:18.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f88fb7fe700 1 -- 172.21.15.40:0/1061796649 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec00bd90 con 0x7f88fc153880 2024-09-13T17:52:18.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f890141e700 1 -- 172.21.15.40:0/1061796649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88fc153880 msgr2=0x7f88fc0c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:18.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f890141e700 1 --2- 172.21.15.40:0/1061796649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88fc153880 0x7f88fc0c9ef0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0cffc0 tx=0x7f88ec009bf0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f890141e700 1 -- 172.21.15.40:0/1061796649 shutdown_connections 2024-09-13T17:52:18.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f890141e700 1 --2- 172.21.15.40:0/1061796649 >> v2:172.21.15.43:3300/0 conn(0x7f88fc0cf3c0 0x7f88fc15cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f890141e700 1 --2- 172.21.15.40:0/1061796649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88fc0cb2f0 0x7f88fc0cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f890141e700 1 --2- 172.21.15.40:0/1061796649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88fc153880 0x7f88fc0c9ef0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f890141e700 1 -- 172.21.15.40:0/1061796649 >> 172.21.15.40:0/1061796649 conn(0x7f88fc06c5b0 msgr2=0x7f88fc06c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:18.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f890141e700 1 -- 172.21.15.40:0/1061796649 shutdown_connections 2024-09-13T17:52:18.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f890141e700 1 -- 172.21.15.40:0/1061796649 wait complete. 2024-09-13T17:52:18.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f890141e700 1 Processor -- start 2024-09-13T17:52:18.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.240+0000 7f890141e700 1 -- start start 2024-09-13T17:52:18.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f890141e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88fc0cb2f0 0x7f88fc149490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:18.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f890141e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88fc0cf3c0 0x7f88fc0ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:18.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f890141e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88fc153880 0x7f88fc1f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:18.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f890141e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc15cbf0 con 0x7f88fc0cf3c0 2024-09-13T17:52:18.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f890141e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc149070 con 0x7f88fc0cb2f0 2024-09-13T17:52:18.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f890141e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88fc0cec80 con 0x7f88fc153880 2024-09-13T17:52:18.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f8900c1d700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88fc153880 msgr2=0x7f88fc1f1de0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:52:18.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f8900c1d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88fc153880 0x7f88fc1f1de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:52:18.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f88f3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88fc0cf3c0 0x7f88fc0ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:18.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f88f3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88fc0cf3c0 0x7f88fc0ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32822/0 (socket says 172.21.15.40:32822) 2024-09-13T17:52:18.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f88f3fff700 1 -- 172.21.15.40:0/1110675072 learned_addr learned my addr 172.21.15.40:0/1110675072 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:18.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f88fbfff700 1 --2- 172.21.15.40:0/1110675072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88fc0cb2f0 0x7f88fc149490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:18.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f88f3fff700 1 -- 172.21.15.40:0/1110675072 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88fc153880 msgr2=0x7f88fc1f1de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:52:18.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f88f3fff700 1 --2- 172.21.15.40:0/1110675072 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88fc153880 0x7f88fc1f1de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f88f3fff700 1 -- 172.21.15.40:0/1110675072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88fc0cb2f0 msgr2=0x7f88fc149490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:18.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f88f3fff700 1 --2- 172.21.15.40:0/1110675072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88fc0cb2f0 0x7f88fc149490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f88f3fff700 1 -- 172.21.15.40:0/1110675072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ec009570 con 0x7f88fc0cf3c0 2024-09-13T17:52:18.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f88fbfff700 1 --2- 172.21.15.40:0/1110675072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88fc0cb2f0 0x7f88fc149490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:52:18.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f88f3fff700 1 --2- 172.21.15.40:0/1110675072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88fc0cf3c0 0x7f88fc0ce030 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f88e400b780 tx=0x7f88e400bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:18.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f88f97fa700 1 -- 172.21.15.40:0/1110675072 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e4003e60 con 0x7f88fc0cf3c0 2024-09-13T17:52:18.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f88f97fa700 1 -- 172.21.15.40:0/1110675072 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e4005070 con 0x7f88fc0cf3c0 2024-09-13T17:52:18.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f890141e700 1 -- 172.21.15.40:0/1110675072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88fc0c8e80 con 0x7f88fc0cf3c0 2024-09-13T17:52:18.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f88f97fa700 1 -- 172.21.15.40:0/1110675072 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e4005420 con 0x7f88fc0cf3c0 2024-09-13T17:52:18.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f890141e700 1 -- 172.21.15.40:0/1110675072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88fc15c910 con 0x7f88fc0cf3c0 2024-09-13T17:52:18.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f890141e700 1 -- 172.21.15.40:0/1110675072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88fc1491b0 con 0x7f88fc0cf3c0 2024-09-13T17:52:18.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.244+0000 7f88f97fa700 1 -- 172.21.15.40:0/1110675072 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f88e4005580 con 0x7f88fc0cf3c0 2024-09-13T17:52:18.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.248+0000 7f88f97fa700 1 --2- 172.21.15.40:0/1110675072 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f88dc0666e0 0x7f88dc068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:18.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.248+0000 7f88f97fa700 1 -- 172.21.15.40:0/1110675072 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f88e4083a60 con 0x7f88fc0cf3c0 2024-09-13T17:52:18.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.248+0000 7f88fbfff700 1 --2- 172.21.15.40:0/1110675072 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f88dc0666e0 0x7f88dc068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:18.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.248+0000 7f88fbfff700 1 --2- 172.21.15.40:0/1110675072 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f88dc0666e0 0x7f88dc068ba0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f88ec014040 tx=0x7f88ec008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:18.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.256+0000 7f88f97fa700 1 -- 172.21.15.40:0/1110675072 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f88fc1491b0 con 0x7f88fc0cf3c0 2024-09-13T17:52:18.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.472+0000 7f890141e700 1 -- 172.21.15.40:0/1110675072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f88fc1491b0 con 0x7f88fc0cf3c0 2024-09-13T17:52:18.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.476+0000 7f88f97fa700 1 -- 172.21.15.40:0/1110675072 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f88fc1491b0 con 0x7f88fc0cf3c0 2024-09-13T17:52:18.479 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:52:18.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.476+0000 7f890141e700 1 -- 172.21.15.40:0/1110675072 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f88dc0666e0 msgr2=0x7f88dc068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.476+0000 7f890141e700 1 --2- 172.21.15.40:0/1110675072 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f88dc0666e0 0x7f88dc068ba0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f88ec014040 tx=0x7f88ec008030 comp rx=0 tx=0).stop 2024-09-13T17:52:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.476+0000 7f890141e700 1 -- 172.21.15.40:0/1110675072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88fc0cf3c0 msgr2=0x7f88fc0ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.476+0000 7f890141e700 1 --2- 172.21.15.40:0/1110675072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88fc0cf3c0 0x7f88fc0ce030 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f88e400b780 tx=0x7f88e400bb40 comp rx=0 tx=0).stop 2024-09-13T17:52:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.476+0000 7f890141e700 1 -- 172.21.15.40:0/1110675072 shutdown_connections 2024-09-13T17:52:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.476+0000 7f890141e700 1 --2- 172.21.15.40:0/1110675072 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f88dc0666e0 0x7f88dc068ba0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.476+0000 7f890141e700 1 --2- 172.21.15.40:0/1110675072 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88fc153880 0x7f88fc1f1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.476+0000 7f890141e700 1 --2- 172.21.15.40:0/1110675072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88fc0cf3c0 0x7f88fc0ce030 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.476+0000 7f890141e700 1 --2- 172.21.15.40:0/1110675072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88fc0cb2f0 0x7f88fc149490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.476+0000 7f890141e700 1 -- 172.21.15.40:0/1110675072 >> 172.21.15.40:0/1110675072 conn(0x7f88fc06c5b0 msgr2=0x7f88fc0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.476+0000 7f890141e700 1 -- 172.21.15.40:0/1110675072 shutdown_connections 2024-09-13T17:52:18.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:18.476+0000 7f890141e700 1 -- 172.21.15.40:0/1110675072 wait complete. 2024-09-13T17:52:21.491 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:52:21.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.588+0000 7f417326f700 1 Processor -- start 2024-09-13T17:52:21.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.588+0000 7f417326f700 1 -- start start 2024-09-13T17:52:21.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.588+0000 7f417326f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f416c0cc4f0 0x7f416c0c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:21.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.588+0000 7f417326f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f416c0c9e20 0x7f416c0cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:21.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.588+0000 7f417326f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f416c0cc060 con 0x7f416c153840 2024-09-13T17:52:21.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.588+0000 7f417326f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f416c0b2840 con 0x7f416c0c9e20 2024-09-13T17:52:21.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.588+0000 7f417326f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f416c0ca870 con 0x7f416c0cc4f0 2024-09-13T17:52:21.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f4172a6e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f416c0c9e20 0x7f416c0cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:21.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f4172a6e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f416c0c9e20 0x7f416c0cf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52088/0 (socket says 172.21.15.40:52088) 2024-09-13T17:52:21.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f4172a6e700 1 -- 172.21.15.40:0/3197157746 learned_addr learned my addr 172.21.15.40:0/3197157746 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f4171a6c700 1 --2- 172.21.15.40:0/3197157746 >> v2:172.21.15.43:3300/0 conn(0x7f416c0cc4f0 0x7f416c0c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f4172a6e700 1 -- 172.21.15.40:0/3197157746 >> v2:172.21.15.43:3300/0 conn(0x7f416c0cc4f0 msgr2=0x7f416c0c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:52:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f4172a6e700 1 --2- 172.21.15.40:0/3197157746 >> v2:172.21.15.43:3300/0 conn(0x7f416c0cc4f0 0x7f416c0c3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f4172a6e700 1 -- 172.21.15.40:0/3197157746 >> v1:172.21.15.40:6789/0 conn(0x7f416c153840 legacy=0x7f416c15d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f4172a6e700 1 -- 172.21.15.40:0/3197157746 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c153c20 con 0x7f416c0c9e20 2024-09-13T17:52:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f4172a6e700 1 --2- 172.21.15.40:0/3197157746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0c9e20 0x7f416c0cf260 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f416c0c9550 tx=0x7f416400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da363102cee8183 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f417126b700 1 -- 172.21.15.40:0/3197157746 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416400e070 con 0x7f416c0c9e20 2024-09-13T17:52:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f417126b700 1 -- 172.21.15.40:0/3197157746 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416400b9c0 con 0x7f416c0c9e20 2024-09-13T17:52:21.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f417126b700 1 -- 172.21.15.40:0/3197157746 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416400bd70 con 0x7f416c0c9e20 2024-09-13T17:52:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f417326f700 1 -- 172.21.15.40:0/3197157746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0c9e20 msgr2=0x7f416c0cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f417326f700 1 --2- 172.21.15.40:0/3197157746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0c9e20 0x7f416c0cf260 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f416c0c9550 tx=0x7f416400b330 comp rx=0 tx=0).stop 2024-09-13T17:52:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f417326f700 1 -- 172.21.15.40:0/3197157746 shutdown_connections 2024-09-13T17:52:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f417326f700 1 --2- 172.21.15.40:0/3197157746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0c9e20 0x7f416c0cf260 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:21.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f417326f700 1 --2- 172.21.15.40:0/3197157746 >> v2:172.21.15.43:3300/0 conn(0x7f416c0cc4f0 0x7f416c0c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:21.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f417326f700 1 -- 172.21.15.40:0/3197157746 >> 172.21.15.40:0/3197157746 conn(0x7f416c06c800 msgr2=0x7f416c06cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:21.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f417326f700 1 -- 172.21.15.40:0/3197157746 shutdown_connections 2024-09-13T17:52:21.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f417326f700 1 -- 172.21.15.40:0/3197157746 wait complete. 2024-09-13T17:52:21.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f417326f700 1 Processor -- start 2024-09-13T17:52:21.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f417326f700 1 -- start start 2024-09-13T17:52:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.592+0000 7f417326f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0c9e20 0x7f416c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f417326f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f416c0cc4f0 0x7f416c1505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f417326f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f416c153840 0x7f416c1495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f417326f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c155120 con 0x7f416c0c9e20 2024-09-13T17:52:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f417326f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c1542d0 con 0x7f416c0cc4f0 2024-09-13T17:52:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f417326f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416c1545b0 con 0x7f416c153840 2024-09-13T17:52:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f417226d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0c9e20 0x7f416c0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f4172a6e700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f416c153840 msgr2=0x7f416c1495c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:52:21.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f417226d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0c9e20 0x7f416c0b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52098/0 (socket says 172.21.15.40:52098) 2024-09-13T17:52:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f417226d700 1 -- 172.21.15.40:0/421339945 learned_addr learned my addr 172.21.15.40:0/421339945 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f4172a6e700 1 --2- 172.21.15.40:0/421339945 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f416c153840 0x7f416c1495c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:52:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f4171a6c700 1 --2- 172.21.15.40:0/421339945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f416c0cc4f0 0x7f416c1505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f417226d700 1 -- 172.21.15.40:0/421339945 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f416c153840 msgr2=0x7f416c1495c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:52:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f417226d700 1 --2- 172.21.15.40:0/421339945 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f416c153840 0x7f416c1495c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f417226d700 1 -- 172.21.15.40:0/421339945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f416c0cc4f0 msgr2=0x7f416c1505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f417226d700 1 --2- 172.21.15.40:0/421339945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f416c0cc4f0 0x7f416c1505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f417226d700 1 -- 172.21.15.40:0/421339945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4164009b30 con 0x7f416c0c9e20 2024-09-13T17:52:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f417226d700 1 --2- 172.21.15.40:0/421339945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0c9e20 0x7f416c0b4320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f416800c500 tx=0x7f416800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f4162ffd700 1 -- 172.21.15.40:0/421339945 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41680052d0 con 0x7f416c0c9e20 2024-09-13T17:52:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f4162ffd700 1 -- 172.21.15.40:0/421339945 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4168005430 con 0x7f416c0c9e20 2024-09-13T17:52:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f417326f700 1 -- 172.21.15.40:0/421339945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416c14fb10 con 0x7f416c0c9e20 2024-09-13T17:52:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f4162ffd700 1 -- 172.21.15.40:0/421339945 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41680055a0 con 0x7f416c0c9e20 2024-09-13T17:52:21.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f417326f700 1 -- 172.21.15.40:0/421339945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416c1500d0 con 0x7f416c0c9e20 2024-09-13T17:52:21.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f4162ffd700 1 -- 172.21.15.40:0/421339945 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f41680038e0 con 0x7f416c0c9e20 2024-09-13T17:52:21.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f4162ffd700 1 --2- 172.21.15.40:0/421339945 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f41500666e0 0x7f4150068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:21.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f4162ffd700 1 -- 172.21.15.40:0/421339945 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4168083970 con 0x7f416c0c9e20 2024-09-13T17:52:21.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f4171a6c700 1 --2- 172.21.15.40:0/421339945 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f41500666e0 0x7f4150068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:21.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.596+0000 7f4171a6c700 1 --2- 172.21.15.40:0/421339945 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f41500666e0 0x7f4150068ba0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f416c1e5880 tx=0x7f415c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:21.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.600+0000 7f417326f700 1 -- 172.21.15.40:0/421339945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416c0cbcf0 con 0x7f416c0c9e20 2024-09-13T17:52:21.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.608+0000 7f4162ffd700 1 -- 172.21.15.40:0/421339945 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4168050930 con 0x7f416c0c9e20 2024-09-13T17:52:21.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.844+0000 7f417326f700 1 -- 172.21.15.40:0/421339945 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f416c0ca9d0 con 0x7f416c0c9e20 2024-09-13T17:52:21.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.844+0000 7f4162ffd700 1 -- 172.21.15.40:0/421339945 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f4168017020 con 0x7f416c0c9e20 2024-09-13T17:52:21.849 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:52:21.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.848+0000 7f417326f700 1 -- 172.21.15.40:0/421339945 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f41500666e0 msgr2=0x7f4150068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:21.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.848+0000 7f417326f700 1 --2- 172.21.15.40:0/421339945 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f41500666e0 0x7f4150068ba0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f416c1e5880 tx=0x7f415c008040 comp rx=0 tx=0).stop 2024-09-13T17:52:21.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.848+0000 7f417326f700 1 -- 172.21.15.40:0/421339945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0c9e20 msgr2=0x7f416c0b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:21.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.848+0000 7f417326f700 1 --2- 172.21.15.40:0/421339945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0c9e20 0x7f416c0b4320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f416800c500 tx=0x7f416800c8c0 comp rx=0 tx=0).stop 2024-09-13T17:52:21.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.848+0000 7f417326f700 1 -- 172.21.15.40:0/421339945 shutdown_connections 2024-09-13T17:52:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.848+0000 7f417326f700 1 --2- 172.21.15.40:0/421339945 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f41500666e0 0x7f4150068ba0 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f416c1e5880 tx=0x7f415c008040 comp rx=0 tx=0).stop 2024-09-13T17:52:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.848+0000 7f417326f700 1 --2- 172.21.15.40:0/421339945 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f416c153840 0x7f416c1495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.848+0000 7f417326f700 1 --2- 172.21.15.40:0/421339945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f416c0cc4f0 0x7f416c1505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.848+0000 7f417326f700 1 --2- 172.21.15.40:0/421339945 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f416c0c9e20 0x7f416c0b4320 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.848+0000 7f417326f700 1 -- 172.21.15.40:0/421339945 >> 172.21.15.40:0/421339945 conn(0x7f416c06c800 msgr2=0x7f416c14dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.848+0000 7f417326f700 1 -- 172.21.15.40:0/421339945 shutdown_connections 2024-09-13T17:52:21.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:21.848+0000 7f417326f700 1 -- 172.21.15.40:0/421339945 wait complete. 2024-09-13T17:52:24.865 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:52:24.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c5a55f700 1 Processor -- start 2024-09-13T17:52:24.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c5a55f700 1 -- start start 2024-09-13T17:52:24.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c5a55f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c54072240 0x7f4c540c0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:24.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c5a55f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4c540c28f0 0x7f4c541565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:24.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c5a55f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c540b2210 con 0x7f4c54071e60 2024-09-13T17:52:24.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c5a55f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c540c1780 con 0x7f4c540c28f0 2024-09-13T17:52:24.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c5a55f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c540c1a60 con 0x7f4c54072240 2024-09-13T17:52:24.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c58d5c700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c54072240 msgr2=0x7f4c540c0e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:52:24.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c58d5c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c54072240 0x7f4c540c0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:52:24.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c59d5e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4c540c28f0 0x7f4c541565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:24.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c5955d700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4c54071e60 0x7f4c540c32e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:44408/0 (socket says 172.21.15.40:44408) 2024-09-13T17:52:24.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c59d5e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4c540c28f0 0x7f4c541565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:55554/0 (socket says 172.21.15.40:55554) 2024-09-13T17:52:24.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c5955d700 1 -- 172.21.15.40:0/95639928 learned_addr learned my addr 172.21.15.40:0/95639928 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:52:24.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c59d5e700 1 -- 172.21.15.40:0/95639928 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c54072240 msgr2=0x7f4c540c0e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:52:24.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c59d5e700 1 --2- 172.21.15.40:0/95639928 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c54072240 0x7f4c540c0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:24.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c59d5e700 1 -- 172.21.15.40:0/95639928 >> v1:172.21.15.43:6789/0 conn(0x7f4c54071e60 legacy=0x7f4c540c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:24.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c59d5e700 1 -- 172.21.15.40:0/95639928 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c540c5d70 con 0x7f4c540c28f0 2024-09-13T17:52:24.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c59d5e700 1 --2- 172.21.15.40:0/95639928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c540c28f0 0x7f4c541565c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009d90 tx=0x7f4c4c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f433041e31de98f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:24.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c4bfff700 1 -- 172.21.15.40:0/95639928 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c00e070 con 0x7f4c540c28f0 2024-09-13T17:52:24.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c4bfff700 1 -- 172.21.15.40:0/95639928 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4c00bbd0 con 0x7f4c540c28f0 2024-09-13T17:52:24.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.968+0000 7f4c4bfff700 1 -- 172.21.15.40:0/95639928 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c012430 con 0x7f4c540c28f0 2024-09-13T17:52:24.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c5a55f700 1 -- 172.21.15.40:0/95639928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c540c28f0 msgr2=0x7f4c541565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:24.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c5a55f700 1 --2- 172.21.15.40:0/95639928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c540c28f0 0x7f4c541565c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009d90 tx=0x7f4c4c00b330 comp rx=0 tx=0).stop 2024-09-13T17:52:24.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c5a55f700 1 -- 172.21.15.40:0/95639928 shutdown_connections 2024-09-13T17:52:24.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c5a55f700 1 --2- 172.21.15.40:0/95639928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c540c28f0 0x7f4c541565c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:24.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c5a55f700 1 --2- 172.21.15.40:0/95639928 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c54072240 0x7f4c540c0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:24.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c5a55f700 1 -- 172.21.15.40:0/95639928 >> 172.21.15.40:0/95639928 conn(0x7f4c5406e4f0 msgr2=0x7f4c5406e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:24.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c5a55f700 1 -- 172.21.15.40:0/95639928 shutdown_connections 2024-09-13T17:52:24.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c5a55f700 1 -- 172.21.15.40:0/95639928 wait complete. 2024-09-13T17:52:24.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c5a55f700 1 Processor -- start 2024-09-13T17:52:24.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c5a55f700 1 -- start start 2024-09-13T17:52:24.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c5a55f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c54071e60 0x7f4c5414dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:24.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c5a55f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c54072240 0x7f4c541ef5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:24.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c5a55f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c540c28f0 0x7f4c54148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:24.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c5a55f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5406d5d0 con 0x7f4c54072240 2024-09-13T17:52:24.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c5a55f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c54156640 con 0x7f4c54071e60 2024-09-13T17:52:24.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c5a55f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5414a6f0 con 0x7f4c540c28f0 2024-09-13T17:52:24.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c58d5c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c54072240 0x7f4c541ef5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:24.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c59d5e700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c540c28f0 msgr2=0x7f4c54148650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:52:24.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c59d5e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c540c28f0 0x7f4c54148650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:52:24.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c58d5c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c54072240 0x7f4c541ef5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52104/0 (socket says 172.21.15.40:52104) 2024-09-13T17:52:24.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c58d5c700 1 -- 172.21.15.40:0/70169608 learned_addr learned my addr 172.21.15.40:0/70169608 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:24.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c5955d700 1 --2- 172.21.15.40:0/70169608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c54071e60 0x7f4c5414dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:24.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c58d5c700 1 -- 172.21.15.40:0/70169608 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c540c28f0 msgr2=0x7f4c54148650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:52:24.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c58d5c700 1 --2- 172.21.15.40:0/70169608 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c540c28f0 0x7f4c54148650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:24.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c58d5c700 1 -- 172.21.15.40:0/70169608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c54071e60 msgr2=0x7f4c5414dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:24.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c58d5c700 1 --2- 172.21.15.40:0/70169608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c54071e60 0x7f4c5414dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:24.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.972+0000 7f4c58d5c700 1 -- 172.21.15.40:0/70169608 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c009a70 con 0x7f4c54072240 2024-09-13T17:52:24.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.976+0000 7f4c58d5c700 1 --2- 172.21.15.40:0/70169608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c54072240 0x7f4c541ef5c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400b7b0 tx=0x7f4c4400bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:24.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.976+0000 7f4c49ffb700 1 -- 172.21.15.40:0/70169608 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c44003e60 con 0x7f4c54072240 2024-09-13T17:52:24.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.976+0000 7f4c49ffb700 1 -- 172.21.15.40:0/70169608 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c44005070 con 0x7f4c54072240 2024-09-13T17:52:24.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.976+0000 7f4c5a55f700 1 -- 172.21.15.40:0/70169608 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c540c7ab0 con 0x7f4c54072240 2024-09-13T17:52:24.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.976+0000 7f4c49ffb700 1 -- 172.21.15.40:0/70169608 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c44005440 con 0x7f4c54072240 2024-09-13T17:52:24.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.976+0000 7f4c5a55f700 1 -- 172.21.15.40:0/70169608 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c5406dac0 con 0x7f4c54072240 2024-09-13T17:52:24.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.976+0000 7f4c49ffb700 1 -- 172.21.15.40:0/70169608 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4c440055a0 con 0x7f4c54072240 2024-09-13T17:52:24.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.976+0000 7f4c49ffb700 1 --2- 172.21.15.40:0/70169608 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4c400667d0 0x7f4c40068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:24.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.976+0000 7f4c49ffb700 1 -- 172.21.15.40:0/70169608 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4c44083fc0 con 0x7f4c54072240 2024-09-13T17:52:24.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.976+0000 7f4c5955d700 1 --2- 172.21.15.40:0/70169608 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4c400667d0 0x7f4c40068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:24.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.976+0000 7f4c5955d700 1 --2- 172.21.15.40:0/70169608 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4c400667d0 0x7f4c40068c90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4c50005930 tx=0x7f4c50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:24.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.980+0000 7f4c5a55f700 1 -- 172.21.15.40:0/70169608 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c540c0e90 con 0x7f4c54072240 2024-09-13T17:52:24.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:24.988+0000 7f4c49ffb700 1 -- 172.21.15.40:0/70169608 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4c44050ee0 con 0x7f4c54072240 2024-09-13T17:52:25.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:25.212+0000 7f4c5a55f700 1 -- 172.21.15.40:0/70169608 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4c54158590 con 0x7f4c54072240 2024-09-13T17:52:25.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:25.212+0000 7f4c49ffb700 1 -- 172.21.15.40:0/70169608 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f4c44054500 con 0x7f4c54072240 2024-09-13T17:52:25.217 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:52:25.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:25.216+0000 7f4c5a55f700 1 -- 172.21.15.40:0/70169608 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4c400667d0 msgr2=0x7f4c40068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:25.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:25.216+0000 7f4c5a55f700 1 --2- 172.21.15.40:0/70169608 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4c400667d0 0x7f4c40068c90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4c50005930 tx=0x7f4c50009040 comp rx=0 tx=0).stop 2024-09-13T17:52:25.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:25.216+0000 7f4c5a55f700 1 -- 172.21.15.40:0/70169608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c54072240 msgr2=0x7f4c541ef5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:25.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:25.216+0000 7f4c5a55f700 1 --2- 172.21.15.40:0/70169608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c54072240 0x7f4c541ef5c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400b7b0 tx=0x7f4c4400bac0 comp rx=0 tx=0).stop 2024-09-13T17:52:25.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:25.216+0000 7f4c5a55f700 1 -- 172.21.15.40:0/70169608 shutdown_connections 2024-09-13T17:52:25.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:25.216+0000 7f4c5a55f700 1 --2- 172.21.15.40:0/70169608 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4c400667d0 0x7f4c40068c90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:25.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:25.216+0000 7f4c5a55f700 1 --2- 172.21.15.40:0/70169608 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c540c28f0 0x7f4c54148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:25.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:25.216+0000 7f4c5a55f700 1 --2- 172.21.15.40:0/70169608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c54072240 0x7f4c541ef5c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:25.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:25.216+0000 7f4c5a55f700 1 --2- 172.21.15.40:0/70169608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c54071e60 0x7f4c5414dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:25.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:25.216+0000 7f4c5a55f700 1 -- 172.21.15.40:0/70169608 >> 172.21.15.40:0/70169608 conn(0x7f4c5406e4f0 msgr2=0x7f4c540ca490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:25.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:25.216+0000 7f4c5a55f700 1 -- 172.21.15.40:0/70169608 shutdown_connections 2024-09-13T17:52:25.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:25.216+0000 7f4c5a55f700 1 -- 172.21.15.40:0/70169608 wait complete. 2024-09-13T17:52:28.234 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:52:28.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.356+0000 7fed19b1f700 1 Processor -- start 2024-09-13T17:52:28.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.356+0000 7fed19b1f700 1 -- start start 2024-09-13T17:52:28.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.360+0000 7fed19b1f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed14150770 0x7fed14156280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:28.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.360+0000 7fed19b1f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 0x7fed141567c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:28.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.360+0000 7fed19b1f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed14072390 con 0x7fed1406b4a0 2024-09-13T17:52:28.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.360+0000 7fed19b1f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed14072820 con 0x7fed140c8510 2024-09-13T17:52:28.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.360+0000 7fed19b1f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed14071d80 con 0x7fed14150770 2024-09-13T17:52:28.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.360+0000 7fed137fe700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed14150770 msgr2=0x7fed14156280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:52:28.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.360+0000 7fed137fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed14150770 0x7fed14156280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:52:28.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.360+0000 7fed0bfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 0x7fed141567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:28.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.360+0000 7fed13fff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fed1406b4a0 0x7fed1406cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:44410/0 (socket says 172.21.15.40:44410) 2024-09-13T17:52:28.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.360+0000 7fed0bfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 0x7fed141567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:55582/0 (socket says 172.21.15.40:55582) 2024-09-13T17:52:28.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.360+0000 7fed13fff700 1 -- 172.21.15.40:0/2653712460 learned_addr learned my addr 172.21.15.40:0/2653712460 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:52:28.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.360+0000 7fed0bfff700 1 -- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 msgr2=0x7fed141567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:52:28.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.360+0000 7fed0bfff700 1 -- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 msgr2=0x7fed141567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:52:28.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.360+0000 7fed0bfff700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 0x7fed141567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:52:28.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.360+0000 7fed0bfff700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 0x7fed141567c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:52:28.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.564+0000 7fed137fe700 1 -- 172.21.15.40:0/2653712460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed14150770 msgr2=0x7fed14156280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:52:28.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.564+0000 7fed137fe700 1 --2- 172.21.15.40:0/2653712460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed14150770 0x7fed14156280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:52:28.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.564+0000 7fed0bfff700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 0x7fed141567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:28.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.564+0000 7fed0bfff700 1 -- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 msgr2=0x7fed141567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:52:28.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.564+0000 7fed0bfff700 1 -- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 msgr2=0x7fed141567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:52:28.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.564+0000 7fed0bfff700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 0x7fed141567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:52:28.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.564+0000 7fed0bfff700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 0x7fed141567c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:52:28.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.968+0000 7fed137fe700 1 -- 172.21.15.40:0/2653712460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed14150770 msgr2=0x7fed14156280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:52:28.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.968+0000 7fed137fe700 1 --2- 172.21.15.40:0/2653712460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed14150770 0x7fed14156280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:52:28.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.968+0000 7fed0bfff700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 0x7fed141567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:28.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.968+0000 7fed0bfff700 1 -- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 msgr2=0x7fed141567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:52:28.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.968+0000 7fed0bfff700 1 -- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 msgr2=0x7fed141567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:52:28.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.968+0000 7fed0bfff700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 0x7fed141567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:52:28.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:28.968+0000 7fed0bfff700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 0x7fed141567c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:52:29.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:29.772+0000 7fed137fe700 1 -- 172.21.15.40:0/2653712460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed14150770 msgr2=0x7fed14156280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:52:29.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:29.772+0000 7fed137fe700 1 --2- 172.21.15.40:0/2653712460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed14150770 0x7fed14156280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:52:29.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:29.772+0000 7fed0bfff700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 0x7fed141567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:29.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:29.772+0000 7fed0bfff700 1 -- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 msgr2=0x7fed141567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:52:29.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:29.772+0000 7fed0bfff700 1 -- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 msgr2=0x7fed141567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:52:29.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:29.772+0000 7fed0bfff700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 0x7fed141567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:52:29.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:29.772+0000 7fed0bfff700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 0x7fed141567c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:52:31.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.360+0000 7fed11ffb700 1 -- 172.21.15.40:0/2653712460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed14150770 msgr2=0x7fed14156280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:52:31.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.360+0000 7fed11ffb700 1 --2- 172.21.15.40:0/2653712460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed14150770 0x7fed14156280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:31.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.360+0000 7fed11ffb700 1 -- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 msgr2=0x7fed141567c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:52:31.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.360+0000 7fed11ffb700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.43:3300/0 conn(0x7fed140c8510 0x7fed141567c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:31.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.360+0000 7fed11ffb700 1 -- 172.21.15.40:0/2653712460 >> v1:172.21.15.43:6789/0 conn(0x7fed1406b4a0 legacy=0x7fed1406cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:31.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.360+0000 7fed11ffb700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.40:3300/0 conn(0x7fecf4003760 0x7fecf4005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:31.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.360+0000 7fed11ffb700 1 -- 172.21.15.40:0/2653712460 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed1406dc50 con 0x7fecf4006210 2024-09-13T17:52:31.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.360+0000 7fed11ffb700 1 -- 172.21.15.40:0/2653712460 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed14072820 con 0x7fecf4003760 2024-09-13T17:52:31.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.360+0000 7fed137fe700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.40:3300/0 conn(0x7fecf4003760 0x7fecf4005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:31.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.360+0000 7fed137fe700 1 -- 172.21.15.40:0/2653712460 >> v2:172.21.15.40:3300/0 conn(0x7fecf4003760 msgr2=0x7fecf4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:52:31.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.360+0000 7fed137fe700 1 -- 172.21.15.40:0/2653712460 >> v2:172.21.15.40:3300/0 conn(0x7fecf4003760 msgr2=0x7fecf4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:52:31.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.360+0000 7fed137fe700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.40:3300/0 conn(0x7fecf4003760 0x7fecf4005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:52:31.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.360+0000 7fed137fe700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.40:3300/0 conn(0x7fecf4003760 0x7fecf4005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:52:31.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.564+0000 7fed137fe700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.40:3300/0 conn(0x7fecf4003760 0x7fecf4005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:31.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.564+0000 7fed137fe700 1 -- 172.21.15.40:0/2653712460 >> v2:172.21.15.40:3300/0 conn(0x7fecf4003760 msgr2=0x7fecf4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:52:31.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.564+0000 7fed137fe700 1 -- 172.21.15.40:0/2653712460 >> v2:172.21.15.40:3300/0 conn(0x7fecf4003760 msgr2=0x7fecf4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:52:31.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.564+0000 7fed137fe700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.40:3300/0 conn(0x7fecf4003760 0x7fecf4005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:52:31.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.564+0000 7fed137fe700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.40:3300/0 conn(0x7fecf4003760 0x7fecf4005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:52:31.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.968+0000 7fed137fe700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.40:3300/0 conn(0x7fecf4003760 0x7fecf4005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:31.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.968+0000 7fed137fe700 1 -- 172.21.15.40:0/2653712460 >> v2:172.21.15.40:3300/0 conn(0x7fecf4003760 msgr2=0x7fecf4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:52:31.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.968+0000 7fed137fe700 1 -- 172.21.15.40:0/2653712460 >> v2:172.21.15.40:3300/0 conn(0x7fecf4003760 msgr2=0x7fecf4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:52:31.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.968+0000 7fed137fe700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.40:3300/0 conn(0x7fecf4003760 0x7fecf4005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:52:31.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:31.968+0000 7fed137fe700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.40:3300/0 conn(0x7fecf4003760 0x7fecf4005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:52:32.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.276+0000 7fed12ffd700 1 -- 172.21.15.40:0/2653712460 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3898525390 0 0) 0x7fed1406dc50 con 0x7fecf4006210 2024-09-13T17:52:32.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.276+0000 7fed12ffd700 1 -- 172.21.15.40:0/2653712460 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed000035d0 con 0x7fecf4006210 2024-09-13T17:52:32.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.276+0000 7fed12ffd700 1 -- 172.21.15.40:0/2653712460 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 486422956 0 0) 0x7fed000035d0 con 0x7fecf4006210 2024-09-13T17:52:32.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.276+0000 7fed12ffd700 1 -- 172.21.15.40:0/2653712460 >> v2:172.21.15.40:3300/0 conn(0x7fecf4003760 msgr2=0x7fecf4005bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:52:32.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.276+0000 7fed12ffd700 1 --2- 172.21.15.40:0/2653712460 >> v2:172.21.15.40:3300/0 conn(0x7fecf4003760 0x7fecf4005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.276+0000 7fed12ffd700 1 -- 172.21.15.40:0/2653712460 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf400afc0 con 0x7fecf4006210 2024-09-13T17:52:32.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.276+0000 7fed12ffd700 1 -- 172.21.15.40:0/2653712460 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fecfc00c4f0 con 0x7fecf4006210 2024-09-13T17:52:32.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.276+0000 7fed12ffd700 1 -- 172.21.15.40:0/2653712460 >> v1:172.21.15.40:6789/0 conn(0x7fecf4006210 legacy=0x7fecf40086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:32.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.276+0000 7fed12ffd700 1 --2- 172.21.15.40:0/2653712460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed00003a20 0x7fed00003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.276+0000 7fed13fff700 1 -- 172.21.15.40:0/2653712460 reap_dead start 2024-09-13T17:52:32.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.276+0000 7fed12ffd700 1 --2- 172.21.15.40:0/2653712460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed14150770 0x7fed00004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.276+0000 7fed12ffd700 1 --2- 172.21.15.40:0/2653712460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed00004db0 0x7fed00005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.276+0000 7fed12ffd700 1 -- 172.21.15.40:0/2653712460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed14072820 con 0x7fed00003a20 2024-09-13T17:52:32.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.276+0000 7fed12ffd700 1 -- 172.21.15.40:0/2653712460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0000bbf0 con 0x7fed00004db0 2024-09-13T17:52:32.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.276+0000 7fed12ffd700 1 -- 172.21.15.40:0/2653712460 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed0000bdb0 con 0x7fed14150770 2024-09-13T17:52:32.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.276+0000 7fed0bfff700 1 -- 172.21.15.40:0/2653712460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed14150770 msgr2=0x7fed00004870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:52:32.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.276+0000 7fed0bfff700 1 --2- 172.21.15.40:0/2653712460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed14150770 0x7fed00004870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:52:32.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.276+0000 7fed13fff700 1 --2- 172.21.15.40:0/2653712460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed00004db0 0x7fed00005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:32.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.280+0000 7fed13fff700 1 -- 172.21.15.40:0/2653712460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed14150770 msgr2=0x7fed00004870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:52:32.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.280+0000 7fed13fff700 1 --2- 172.21.15.40:0/2653712460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed14150770 0x7fed00004870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.280+0000 7fed13fff700 1 -- 172.21.15.40:0/2653712460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed00003a20 msgr2=0x7fed00003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:52:32.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.280+0000 7fed13fff700 1 --2- 172.21.15.40:0/2653712460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed00003a20 0x7fed00003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.280+0000 7fed13fff700 1 -- 172.21.15.40:0/2653712460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed000035d0 con 0x7fed00004db0 2024-09-13T17:52:32.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.280+0000 7fed13fff700 1 --2- 172.21.15.40:0/2653712460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed00004db0 0x7fed00005230 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fed1406d890 tx=0x7fed000058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=896817037bdfca00 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:32.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.280+0000 7fed12ffd700 1 -- 172.21.15.40:0/2653712460 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c001c60 con 0x7fed00004db0 2024-09-13T17:52:32.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.280+0000 7fed12ffd700 1 -- 172.21.15.40:0/2653712460 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0c00c860 con 0x7fed00004db0 2024-09-13T17:52:32.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.280+0000 7fed12ffd700 1 -- 172.21.15.40:0/2653712460 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c00cbf0 con 0x7fed00004db0 2024-09-13T17:52:32.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.280+0000 7fed19b1f700 1 -- 172.21.15.40:0/2653712460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed00004db0 msgr2=0x7fed00005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:32.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.280+0000 7fed19b1f700 1 --2- 172.21.15.40:0/2653712460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed00004db0 0x7fed00005230 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fed1406d890 tx=0x7fed000058a0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.280+0000 7fed19b1f700 1 -- 172.21.15.40:0/2653712460 shutdown_connections 2024-09-13T17:52:32.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.280+0000 7fed19b1f700 1 --2- 172.21.15.40:0/2653712460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed00004db0 0x7fed00005230 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.280+0000 7fed19b1f700 1 --2- 172.21.15.40:0/2653712460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed14150770 0x7fed00004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.280+0000 7fed19b1f700 1 --2- 172.21.15.40:0/2653712460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed00003a20 0x7fed00003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.280+0000 7fed19b1f700 1 -- 172.21.15.40:0/2653712460 >> 172.21.15.40:0/2653712460 conn(0x7fed140c8ac0 msgr2=0x7fed140c8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:32.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.280+0000 7fed19b1f700 1 -- 172.21.15.40:0/2653712460 shutdown_connections 2024-09-13T17:52:32.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.280+0000 7fed19b1f700 1 -- 172.21.15.40:0/2653712460 wait complete. 2024-09-13T17:52:32.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.284+0000 7fed19b1f700 1 Processor -- start 2024-09-13T17:52:32.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.284+0000 7fed19b1f700 1 -- start start 2024-09-13T17:52:32.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.284+0000 7fed19b1f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed00003a20 0x7fed14154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed19b1f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed00004db0 0x7fed14146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed19b1f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed14150770 0x7fed141eb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed19b1f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed141ebf00 con 0x7fed14150770 2024-09-13T17:52:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed19b1f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed14154600 con 0x7fed00004db0 2024-09-13T17:52:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed19b1f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed14146260 con 0x7fed00003a20 2024-09-13T17:52:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed137fe700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed00003a20 msgr2=0x7fed14154a90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:52:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed137fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed00003a20 0x7fed14154a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:52:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed13fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed14150770 0x7fed141eb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed13fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed14150770 0x7fed141eb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39266/0 (socket says 172.21.15.40:39266) 2024-09-13T17:52:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed13fff700 1 -- 172.21.15.40:0/289732369 learned_addr learned my addr 172.21.15.40:0/289732369 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:32.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed0bfff700 1 --2- 172.21.15.40:0/289732369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed00004db0 0x7fed14146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed13fff700 1 -- 172.21.15.40:0/289732369 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed00003a20 msgr2=0x7fed14154a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:52:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed13fff700 1 --2- 172.21.15.40:0/289732369 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed00003a20 0x7fed14154a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed13fff700 1 -- 172.21.15.40:0/289732369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed00004db0 msgr2=0x7fed14146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed13fff700 1 --2- 172.21.15.40:0/289732369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed00004db0 0x7fed14146830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed13fff700 1 -- 172.21.15.40:0/289732369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed14156fa0 con 0x7fed14150770 2024-09-13T17:52:32.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed13fff700 1 --2- 172.21.15.40:0/289732369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed14150770 0x7fed141eb9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009f10 tx=0x7fed0c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:32.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed10ff9700 1 -- 172.21.15.40:0/289732369 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c012810 con 0x7fed14150770 2024-09-13T17:52:32.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed10ff9700 1 -- 172.21.15.40:0/289732369 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0c0129b0 con 0x7fed14150770 2024-09-13T17:52:32.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed19b1f700 1 -- 172.21.15.40:0/289732369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed14145ca0 con 0x7fed14150770 2024-09-13T17:52:32.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed10ff9700 1 -- 172.21.15.40:0/289732369 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c012d60 con 0x7fed14150770 2024-09-13T17:52:32.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.288+0000 7fed19b1f700 1 -- 172.21.15.40:0/289732369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed14153fb0 con 0x7fed14150770 2024-09-13T17:52:32.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.292+0000 7fed19b1f700 1 -- 172.21.15.40:0/289732369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed1414e5d0 con 0x7fed14150770 2024-09-13T17:52:32.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.292+0000 7fed10ff9700 1 -- 172.21.15.40:0/289732369 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fed0c019070 con 0x7fed14150770 2024-09-13T17:52:32.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.292+0000 7fed10ff9700 1 --2- 172.21.15.40:0/289732369 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fecf4066f30 0x7fecf40693f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.292+0000 7fed137fe700 1 --2- 172.21.15.40:0/289732369 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fecf4066f30 0x7fecf40693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:32.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.292+0000 7fed10ff9700 1 -- 172.21.15.40:0/289732369 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fed0c0856a0 con 0x7fed14150770 2024-09-13T17:52:32.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.292+0000 7fed137fe700 1 --2- 172.21.15.40:0/289732369 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fecf4066f30 0x7fecf40693f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fed040039e0 tx=0x7fed04005040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:32.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.300+0000 7fed10ff9700 1 -- 172.21.15.40:0/289732369 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fed0c052660 con 0x7fed14150770 2024-09-13T17:52:32.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.528+0000 7fed19b1f700 1 -- 172.21.15.40:0/289732369 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fed1406dad0 con 0x7fed14150770 2024-09-13T17:52:32.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.528+0000 7fed10ff9700 1 -- 172.21.15.40:0/289732369 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7fed0c055c80 con 0x7fed14150770 2024-09-13T17:52:32.534 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":18,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:52:32.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.532+0000 7fed19b1f700 1 -- 172.21.15.40:0/289732369 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fecf4066f30 msgr2=0x7fecf40693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:32.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.532+0000 7fed19b1f700 1 --2- 172.21.15.40:0/289732369 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fecf4066f30 0x7fecf40693f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fed040039e0 tx=0x7fed04005040 comp rx=0 tx=0).stop 2024-09-13T17:52:32.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.532+0000 7fed19b1f700 1 -- 172.21.15.40:0/289732369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed14150770 msgr2=0x7fed141eb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:32.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.532+0000 7fed19b1f700 1 --2- 172.21.15.40:0/289732369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed14150770 0x7fed141eb9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009f10 tx=0x7fed0c0014a0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.532+0000 7fed19b1f700 1 -- 172.21.15.40:0/289732369 shutdown_connections 2024-09-13T17:52:32.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.532+0000 7fed19b1f700 1 --2- 172.21.15.40:0/289732369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fed14150770 0x7fed141eb9c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.532+0000 7fed19b1f700 1 --2- 172.21.15.40:0/289732369 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed00004db0 0x7fed14146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.532+0000 7fed19b1f700 1 --2- 172.21.15.40:0/289732369 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fecf4066f30 0x7fecf40693f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.532+0000 7fed19b1f700 1 --2- 172.21.15.40:0/289732369 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fed00003a20 0x7fed14154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.532+0000 7fed19b1f700 1 -- 172.21.15.40:0/289732369 >> 172.21.15.40:0/289732369 conn(0x7fed140c8ac0 msgr2=0x7fed1414d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:32.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.532+0000 7fed19b1f700 1 -- 172.21.15.40:0/289732369 shutdown_connections 2024-09-13T17:52:32.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.532+0000 7fed19b1f700 1 -- 172.21.15.40:0/289732369 wait complete. 2024-09-13T17:52:32.548 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T17:52:32.548 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-13T17:52:32.548 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:52:32.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.656+0000 7f0133fff700 1 Processor -- start 2024-09-13T17:52:32.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.656+0000 7f0133fff700 1 -- start start 2024-09-13T17:52:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.656+0000 7f0133fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f01341515a0 0x7f013406e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.656+0000 7f0133fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f01340c3de0 0x7f01340cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.656+0000 7f0133fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f013406e580 con 0x7f013406dde0 2024-09-13T17:52:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.656+0000 7f0133fff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0134151980 con 0x7f01340c3de0 2024-09-13T17:52:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.656+0000 7f0133fff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01340c3930 con 0x7f01341515a0 2024-09-13T17:52:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.656+0000 7f01337fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f01340c3de0 0x7f01340cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.656+0000 7f01337fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f01340c3de0 0x7f01340cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39276/0 (socket says 172.21.15.40:39276) 2024-09-13T17:52:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.656+0000 7f01337fe700 1 -- 172.21.15.40:0/139095547 learned_addr learned my addr 172.21.15.40:0/139095547 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.656+0000 7f0132ffd700 1 --2- 172.21.15.40:0/139095547 >> v2:172.21.15.43:3300/0 conn(0x7f01341515a0 0x7f013406e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:32.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.656+0000 7f01337fe700 1 -- 172.21.15.40:0/139095547 >> v2:172.21.15.43:3300/0 conn(0x7f01341515a0 msgr2=0x7f013406e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:52:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.656+0000 7f01337fe700 1 --2- 172.21.15.40:0/139095547 >> v2:172.21.15.43:3300/0 conn(0x7f01341515a0 0x7f013406e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.656+0000 7f01337fe700 1 -- 172.21.15.40:0/139095547 >> v1:172.21.15.43:6789/0 conn(0x7f013406dde0 legacy=0x7f013415a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.656+0000 7f01337fe700 1 -- 172.21.15.40:0/139095547 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01340c2020 con 0x7f01340c3de0 2024-09-13T17:52:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.660+0000 7f01337fe700 1 --2- 172.21.15.40:0/139095547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01340c3de0 0x7f01340cd3d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f01340c2210 tx=0x7f012c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a169f56fe3950e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.660+0000 7f0131ffb700 1 -- 172.21.15.40:0/139095547 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c00e070 con 0x7f01340c3de0 2024-09-13T17:52:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.660+0000 7f0131ffb700 1 -- 172.21.15.40:0/139095547 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012c003a40 con 0x7f01340c3de0 2024-09-13T17:52:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.660+0000 7f0131ffb700 1 -- 172.21.15.40:0/139095547 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c003df0 con 0x7f01340c3de0 2024-09-13T17:52:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.660+0000 7f0133fff700 1 -- 172.21.15.40:0/139095547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01340c3de0 msgr2=0x7f01340cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.660+0000 7f0133fff700 1 --2- 172.21.15.40:0/139095547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01340c3de0 0x7f01340cd3d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f01340c2210 tx=0x7f012c00b330 comp rx=0 tx=0).stop 2024-09-13T17:52:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.660+0000 7f0133fff700 1 -- 172.21.15.40:0/139095547 shutdown_connections 2024-09-13T17:52:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.660+0000 7f0133fff700 1 --2- 172.21.15.40:0/139095547 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01340c3de0 0x7f01340cd3d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.660+0000 7f0133fff700 1 --2- 172.21.15.40:0/139095547 >> v2:172.21.15.43:3300/0 conn(0x7f01341515a0 0x7f013406e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.660+0000 7f0133fff700 1 -- 172.21.15.40:0/139095547 >> 172.21.15.40:0/139095547 conn(0x7f0134072070 msgr2=0x7f0134072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.660+0000 7f0133fff700 1 -- 172.21.15.40:0/139095547 shutdown_connections 2024-09-13T17:52:32.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.660+0000 7f0133fff700 1 -- 172.21.15.40:0/139095547 wait complete. 2024-09-13T17:52:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.660+0000 7f0133fff700 1 Processor -- start 2024-09-13T17:52:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.660+0000 7f0133fff700 1 -- start start 2024-09-13T17:52:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.660+0000 7f0133fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013406dde0 0x7f013414cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.660+0000 7f0133fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f01340c3de0 0x7f01340cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.660+0000 7f0133fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01341515a0 0x7f01341f0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.660+0000 7f0133fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013414c150 con 0x7f013406dde0 2024-09-13T17:52:32.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.660+0000 7f0133fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01341509f0 con 0x7f01341515a0 2024-09-13T17:52:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.660+0000 7f0133fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0134150cd0 con 0x7f01340c3de0 2024-09-13T17:52:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f0132ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013406dde0 0x7f013414cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f01327fc700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f01340c3de0 msgr2=0x7f01340cbf40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:52:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f0132ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013406dde0 0x7f013414cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39280/0 (socket says 172.21.15.40:39280) 2024-09-13T17:52:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f0132ffd700 1 -- 172.21.15.40:0/157190341 learned_addr learned my addr 172.21.15.40:0/157190341 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f01327fc700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f01340c3de0 0x7f01340cbf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:52:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f01337fe700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01341515a0 0x7f01341f0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f0132ffd700 1 -- 172.21.15.40:0/157190341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f01340c3de0 msgr2=0x7f01340cbf40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:52:32.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f0132ffd700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f01340c3de0 0x7f01340cbf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f0132ffd700 1 -- 172.21.15.40:0/157190341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01341515a0 msgr2=0x7f01341f0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f0132ffd700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01341515a0 0x7f01341f0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f0132ffd700 1 -- 172.21.15.40:0/157190341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012c009b30 con 0x7f013406dde0 2024-09-13T17:52:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f01337fe700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01341515a0 0x7f01341f0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:52:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f0132ffd700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013406dde0 0x7f013414cf50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f012400d800 tx=0x7f012400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f011bfff700 1 -- 172.21.15.40:0/157190341 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01240125f0 con 0x7f013406dde0 2024-09-13T17:52:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f011bfff700 1 -- 172.21.15.40:0/157190341 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0124012750 con 0x7f013406dde0 2024-09-13T17:52:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f0133fff700 1 -- 172.21.15.40:0/157190341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f013414b820 con 0x7f013406dde0 2024-09-13T17:52:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f011bfff700 1 -- 172.21.15.40:0/157190341 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0124012ac0 con 0x7f013406dde0 2024-09-13T17:52:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f0133fff700 1 -- 172.21.15.40:0/157190341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f013414bde0 con 0x7f013406dde0 2024-09-13T17:52:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f0133fff700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013406e740 0x7f01341f0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f0133fff700 1 -- 172.21.15.40:0/157190341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0134074750 con 0x7f013406e740 2024-09-13T17:52:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f01327fc700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013406e740 0x7f01341f0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:32.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f01327fc700 1 -- 172.21.15.40:0/157190341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0134158020 con 0x7f013406e740 2024-09-13T17:52:32.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f01327fc700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013406e740 0x7f01341f0a50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0134154680 tx=0x7f012800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:32.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f011bfff700 1 -- 172.21.15.40:0/157190341 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0128005c80 con 0x7f013406e740 2024-09-13T17:52:32.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.664+0000 7f011bfff700 1 -- 172.21.15.40:0/157190341 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0124012c20 con 0x7f013406dde0 2024-09-13T17:52:32.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.668+0000 7f011bfff700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0120066800 0x7f0120068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.668+0000 7f01337fe700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0120066800 0x7f0120068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:32.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.668+0000 7f011bfff700 1 -- 172.21.15.40:0/157190341 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f012800cea0 con 0x7f013406e740 2024-09-13T17:52:32.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.668+0000 7f011bfff700 1 -- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013406e740 msgr2=0x7f01341f0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:32.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.668+0000 7f011bfff700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013406e740 0x7f01341f0a50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0134154680 tx=0x7f012800c7e0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.668+0000 7f011bfff700 1 -- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013406e740 msgr2=0x7f01341f0a50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:52:32.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.668+0000 7f011bfff700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013406e740 0x7f01341f0a50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.668+0000 7f01337fe700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0120066800 0x7f0120068cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f013414ceb0 tx=0x7f012c009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:32.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.672+0000 7f011bfff700 1 -- 172.21.15.40:0/157190341 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0124084790 con 0x7f013406dde0 2024-09-13T17:52:32.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.680+0000 7f0133fff700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01341548c0 0x7f01341403d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.680+0000 7f0133fff700 1 -- 172.21.15.40:0/157190341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0134157d40 con 0x7f01341548c0 2024-09-13T17:52:32.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.680+0000 7f01327fc700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01341548c0 0x7f01341403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:32.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.680+0000 7f01327fc700 1 -- 172.21.15.40:0/157190341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f01340c7880 con 0x7f01341548c0 2024-09-13T17:52:32.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.680+0000 7f01327fc700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01341548c0 0x7f01341403d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0128011000 tx=0x7f0128000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:32.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.680+0000 7f011bfff700 1 -- 172.21.15.40:0/157190341 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012801a330 con 0x7f01341548c0 2024-09-13T17:52:32.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.680+0000 7f011bfff700 1 -- 172.21.15.40:0/157190341 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4606 (secure 0 0 0) 0x7f0128007730 con 0x7f01341548c0 2024-09-13T17:52:32.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.680+0000 7f011bfff700 1 -- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01341548c0 msgr2=0x7f01341403d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:32.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.680+0000 7f011bfff700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01341548c0 0x7f01341403d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0128011000 tx=0x7f0128000ed0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.680+0000 7f011bfff700 1 -- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01341548c0 msgr2=0x7f01341403d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:52:32.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.680+0000 7f011bfff700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f01341548c0 0x7f01341403d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.686 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:52:32.687 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:52:32.687 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:52:32.687 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:52:32.687 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 18, 2024-09-13T17:52:32.687 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:52:32.687 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:52:32.687 INFO:teuthology.orchestra.run.smithi040.stdout: 1 2024-09-13T17:52:32.687 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:32.687 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T17:52:32.688 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:52:32.689 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:52:32.689 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:52:32.689 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:32.689 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:32.689 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:32.689 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:32.689 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:32.689 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:32.689 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:32.689 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:32.689 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:32.690 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:32.690 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:32.690 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:52:32.690 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:52:32.690 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:32.690 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:32.690 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:32.690 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:32.690 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:32.690 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:32.690 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:32.690 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:32.690 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:32.690 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:32.691 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:32.691 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.691 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:52:32.691 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:52:32.691 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:52:32.691 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:52:32.691 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:52:32.691 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:52:32.691 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:52:32.691 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:52:32.691 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:52:32.691 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:52:32.691 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:52:32.692 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:52:32.692 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:52:32.692 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:52:32.692 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:52:32.692 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:52:32.692 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:52:32.692 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:32.692 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:32.692 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:32.692 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:32.692 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:32.692 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:32.692 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:32.693 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:32.693 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:32.693 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:32.693 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:32.693 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:52:32.693 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.693 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:52:32.693 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.693 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:52:32.693 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:52:32.693 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:32.693 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:32.693 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.693 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:32.693 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:52:32.694 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:32.694 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.694 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.694 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:32.694 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:52:32.694 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:32.694 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:32.694 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:32.694 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.694 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:52:32.694 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:52:32.694 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:32.694 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:32.694 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:32.695 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.695 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.695 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:52:32.695 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:52:32.695 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:32.695 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:32.695 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.695 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:32.695 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:52:32.695 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:32.695 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.695 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.695 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:32.696 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:52:32.696 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:32.696 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:32.696 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:32.696 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.696 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:52:32.696 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:52:32.696 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:32.696 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:32.696 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:32.696 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.696 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.696 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:52:32.697 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:52:32.697 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:32.697 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:32.697 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.697 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:32.697 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:52:32.697 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:32.697 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.697 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.697 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:32.697 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:52:32.697 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:32.697 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:32.697 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:32.698 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.698 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:52:32.698 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:52:32.698 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:32.698 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:32.698 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:32.698 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:32.698 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:32.698 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.698 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:52:32.698 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:52:32.698 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.698 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:32.698 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:32.699 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:52:32.699 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:32.699 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:32.699 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:52:32.699 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.699 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:32.699 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:32.699 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:52:32.699 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:32.699 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:32.699 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:52:32.699 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.699 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:32.700 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:32.700 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:52:32.700 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:32.700 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:32.700 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.700 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:52:32.700 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:52:32.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.680+0000 7f0133fff700 1 -- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0120066800 msgr2=0x7f0120068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:32.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.680+0000 7f0133fff700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0120066800 0x7f0120068cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f013414ceb0 tx=0x7f012c009000 comp rx=0 tx=0).stop 2024-09-13T17:52:32.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.680+0000 7f0133fff700 1 -- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013406dde0 msgr2=0x7f013414cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:32.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.680+0000 7f0133fff700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013406dde0 0x7f013414cf50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f012400d800 tx=0x7f012400dbc0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.680+0000 7f01337fe700 1 -- 172.21.15.40:0/157190341 reap_dead start 2024-09-13T17:52:32.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.680+0000 7f0133fff700 1 -- 172.21.15.40:0/157190341 shutdown_connections 2024-09-13T17:52:32.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.680+0000 7f0133fff700 1 --2- 172.21.15.40:0/157190341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f013406dde0 0x7f013414cf50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.680+0000 7f0133fff700 1 -- 172.21.15.40:0/157190341 >> 172.21.15.40:0/157190341 conn(0x7f0134072070 msgr2=0x7f0134154e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:32.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.684+0000 7f0133fff700 1 -- 172.21.15.40:0/157190341 shutdown_connections 2024-09-13T17:52:32.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.684+0000 7f0133fff700 1 -- 172.21.15.40:0/157190341 wait complete. 2024-09-13T17:52:32.702 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T17:52:32.703 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-13T17:52:32.703 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:52:32.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.808+0000 7fd040d24700 1 Processor -- start 2024-09-13T17:52:32.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.808+0000 7fd040d24700 1 -- start start 2024-09-13T17:52:32.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.808+0000 7fd040d24700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd03c1503c0 0x7fd03c0c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.808+0000 7fd040d24700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd03c159630 0x7fd03c159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.808+0000 7fd040d24700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd03c0c8f80 con 0x7fd03c0c8500 2024-09-13T17:52:32.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.808+0000 7fd040d24700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03c0c2a80 con 0x7fd03c1503c0 2024-09-13T17:52:32.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.808+0000 7fd040d24700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c15a020 con 0x7fd03c159630 2024-09-13T17:52:32.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.808+0000 7fd03b7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd03c1503c0 0x7fd03c0c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.808+0000 7fd03bfff700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd03c159630 msgr2=0x7fd03c159ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:52:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.808+0000 7fd03b7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd03c1503c0 0x7fd03c0c79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39318/0 (socket says 172.21.15.40:39318) 2024-09-13T17:52:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.808+0000 7fd03bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd03c159630 0x7fd03c159ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:52:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.808+0000 7fd03b7fe700 1 -- 172.21.15.40:0/1611220143 learned_addr learned my addr 172.21.15.40:0/1611220143 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.808+0000 7fd03b7fe700 1 -- 172.21.15.40:0/1611220143 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd03c159630 msgr2=0x7fd03c159ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:52:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.808+0000 7fd03b7fe700 1 --2- 172.21.15.40:0/1611220143 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd03c159630 0x7fd03c159ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.808+0000 7fd03b7fe700 1 -- 172.21.15.40:0/1611220143 >> v1:172.21.15.43:6789/0 conn(0x7fd03c0c8500 legacy=0x7fd03c0c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.808+0000 7fd03b7fe700 1 -- 172.21.15.40:0/1611220143 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03c1507a0 con 0x7fd03c1503c0 2024-09-13T17:52:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.808+0000 7fd03b7fe700 1 --2- 172.21.15.40:0/1611220143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd03c1503c0 0x7fd03c0c79c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd034009e50 tx=0x7fd03400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ec74d81824f58f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.808+0000 7fd03a7fc700 1 -- 172.21.15.40:0/1611220143 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03400e070 con 0x7fd03c1503c0 2024-09-13T17:52:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.808+0000 7fd03a7fc700 1 -- 172.21.15.40:0/1611220143 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03400b9c0 con 0x7fd03c1503c0 2024-09-13T17:52:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.808+0000 7fd03a7fc700 1 -- 172.21.15.40:0/1611220143 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03400bd70 con 0x7fd03c1503c0 2024-09-13T17:52:32.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd040d24700 1 -- 172.21.15.40:0/1611220143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd03c1503c0 msgr2=0x7fd03c0c79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:32.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd040d24700 1 --2- 172.21.15.40:0/1611220143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd03c1503c0 0x7fd03c0c79c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd034009e50 tx=0x7fd03400b330 comp rx=0 tx=0).stop 2024-09-13T17:52:32.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd040d24700 1 -- 172.21.15.40:0/1611220143 shutdown_connections 2024-09-13T17:52:32.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd040d24700 1 --2- 172.21.15.40:0/1611220143 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd03c159630 0x7fd03c159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd040d24700 1 --2- 172.21.15.40:0/1611220143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd03c1503c0 0x7fd03c0c79c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd040d24700 1 -- 172.21.15.40:0/1611220143 >> 172.21.15.40:0/1611220143 conn(0x7fd03c06e6d0 msgr2=0x7fd03c06eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:32.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd040d24700 1 -- 172.21.15.40:0/1611220143 shutdown_connections 2024-09-13T17:52:32.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd040d24700 1 -- 172.21.15.40:0/1611220143 wait complete. 2024-09-13T17:52:32.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd040d24700 1 Processor -- start 2024-09-13T17:52:32.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd040d24700 1 -- start start 2024-09-13T17:52:32.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd040d24700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd03c0c8500 0x7fd03c06a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd040d24700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c1503c0 0x7fd03c0c2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd040d24700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd03c159630 0x7fd03c0c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd040d24700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c1ed930 con 0x7fd03c0c8500 2024-09-13T17:52:32.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd040d24700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c1f1060 con 0x7fd03c1503c0 2024-09-13T17:52:32.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd040d24700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c1f12c0 con 0x7fd03c159630 2024-09-13T17:52:32.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd03b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd03c0c8500 0x7fd03c06a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:32.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd03bfff700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd03c159630 msgr2=0x7fd03c0c4f90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:52:32.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd03bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd03c159630 0x7fd03c0c4f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:52:32.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd03b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd03c0c8500 0x7fd03c06a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39334/0 (socket says 172.21.15.40:39334) 2024-09-13T17:52:32.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd03b7fe700 1 -- 172.21.15.40:0/3270566391 learned_addr learned my addr 172.21.15.40:0/3270566391 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:32.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd03affd700 1 --2- 172.21.15.40:0/3270566391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c1503c0 0x7fd03c0c2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:32.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd03b7fe700 1 -- 172.21.15.40:0/3270566391 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd03c159630 msgr2=0x7fd03c0c4f90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:52:32.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd03b7fe700 1 --2- 172.21.15.40:0/3270566391 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd03c159630 0x7fd03c0c4f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd03b7fe700 1 -- 172.21.15.40:0/3270566391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c1503c0 msgr2=0x7fd03c0c2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:32.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd03b7fe700 1 --2- 172.21.15.40:0/3270566391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c1503c0 0x7fd03c0c2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd03b7fe700 1 -- 172.21.15.40:0/3270566391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd034009b30 con 0x7fd03c0c8500 2024-09-13T17:52:32.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd03b7fe700 1 --2- 172.21.15.40:0/3270566391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd03c0c8500 0x7fd03c06a600 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd034000f80 tx=0x7fd034004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:32.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd023fff700 1 -- 172.21.15.40:0/3270566391 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0340092e0 con 0x7fd03c0c8500 2024-09-13T17:52:32.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd023fff700 1 -- 172.21.15.40:0/3270566391 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd034004970 con 0x7fd03c0c8500 2024-09-13T17:52:32.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd040d24700 1 -- 172.21.15.40:0/3270566391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03c1ed270 con 0x7fd03c0c8500 2024-09-13T17:52:32.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd023fff700 1 -- 172.21.15.40:0/3270566391 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03401c660 con 0x7fd03c0c8500 2024-09-13T17:52:32.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.812+0000 7fd040d24700 1 -- 172.21.15.40:0/3270566391 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03c1ecf60 con 0x7fd03c0c8500 2024-09-13T17:52:32.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.816+0000 7fd040d24700 1 --2- 172.21.15.40:0/3270566391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c0c54d0 0x7fd03c0c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.816+0000 7fd040d24700 1 -- 172.21.15.40:0/3270566391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c0675e0 con 0x7fd03c0c54d0 2024-09-13T17:52:32.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.816+0000 7fd03affd700 1 --2- 172.21.15.40:0/3270566391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c0c54d0 0x7fd03c0c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:32.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.816+0000 7fd023fff700 1 -- 172.21.15.40:0/3270566391 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd03401c7c0 con 0x7fd03c0c8500 2024-09-13T17:52:32.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.816+0000 7fd03affd700 1 -- 172.21.15.40:0/3270566391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd03c0c8ce0 con 0x7fd03c0c54d0 2024-09-13T17:52:32.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.816+0000 7fd03affd700 1 --2- 172.21.15.40:0/3270566391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c0c54d0 0x7fd03c0c39d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd030000f60 tx=0x7fd03000fb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:32.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.816+0000 7fd023fff700 1 --2- 172.21.15.40:0/3270566391 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd0280666e0 0x7fd028068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.816+0000 7fd023fff700 1 -- 172.21.15.40:0/3270566391 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd034085f30 con 0x7fd03c0c8500 2024-09-13T17:52:32.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.816+0000 7fd03bfff700 1 --2- 172.21.15.40:0/3270566391 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd0280666e0 0x7fd028068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:32.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.816+0000 7fd023fff700 1 -- 172.21.15.40:0/3270566391 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0300052e0 con 0x7fd03c0c54d0 2024-09-13T17:52:32.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.816+0000 7fd03bfff700 1 --2- 172.21.15.40:0/3270566391 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd0280666e0 0x7fd028068ba0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd02c0039e0 tx=0x7fd02c006d10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:32.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.816+0000 7fd023fff700 1 -- 172.21.15.40:0/3270566391 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd030005440 con 0x7fd03c0c54d0 2024-09-13T17:52:32.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.816+0000 7fd023fff700 1 -- 172.21.15.40:0/3270566391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c0c54d0 msgr2=0x7fd03c0c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:32.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.816+0000 7fd023fff700 1 --2- 172.21.15.40:0/3270566391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c0c54d0 0x7fd03c0c39d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd030000f60 tx=0x7fd03000fb00 comp rx=0 tx=0).stop 2024-09-13T17:52:32.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.816+0000 7fd023fff700 1 -- 172.21.15.40:0/3270566391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c0c54d0 msgr2=0x7fd03c0c39d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:52:32.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.816+0000 7fd023fff700 1 --2- 172.21.15.40:0/3270566391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c0c54d0 0x7fd03c0c39d0 secure :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd030000f60 tx=0x7fd03000fb00 comp rx=0 tx=0).stop 2024-09-13T17:52:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.828+0000 7fd040d24700 1 --2- 172.21.15.40:0/3270566391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c0c5b10 0x7fd03c0c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.828+0000 7fd040d24700 1 -- 172.21.15.40:0/3270566391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c1ed400 con 0x7fd03c0c5b10 2024-09-13T17:52:32.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.828+0000 7fd03affd700 1 --2- 172.21.15.40:0/3270566391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c0c5b10 0x7fd03c0c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:32.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.828+0000 7fd03affd700 1 -- 172.21.15.40:0/3270566391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd03c0c8a40 con 0x7fd03c0c5b10 2024-09-13T17:52:32.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.828+0000 7fd03affd700 1 --2- 172.21.15.40:0/3270566391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c0c5b10 0x7fd03c0c3f10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd030017000 tx=0x7fd030001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:32.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.828+0000 7fd023fff700 1 -- 172.21.15.40:0/3270566391 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030023070 con 0x7fd03c0c5b10 2024-09-13T17:52:32.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.832+0000 7fd023fff700 1 -- 172.21.15.40:0/3270566391 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4936 (secure 0 0 0) 0x7fd030003d00 con 0x7fd03c0c5b10 2024-09-13T17:52:32.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.832+0000 7fd023fff700 1 -- 172.21.15.40:0/3270566391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c0c5b10 msgr2=0x7fd03c0c3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:32.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.832+0000 7fd023fff700 1 --2- 172.21.15.40:0/3270566391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c0c5b10 0x7fd03c0c3f10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd030017000 tx=0x7fd030001120 comp rx=0 tx=0).stop 2024-09-13T17:52:32.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.832+0000 7fd023fff700 1 -- 172.21.15.40:0/3270566391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c0c5b10 msgr2=0x7fd03c0c3f10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:52:32.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.832+0000 7fd023fff700 1 --2- 172.21.15.40:0/3270566391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c0c5b10 0x7fd03c0c3f10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.835 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:52:32.836 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:52:32.836 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:52:32.836 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:52:32.836 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 18, 2024-09-13T17:52:32.836 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:52:32.836 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:52:32.837 INFO:teuthology.orchestra.run.smithi040.stdout: 1 2024-09-13T17:52:32.837 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:32.837 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T17:52:32.837 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:52:32.838 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:52:32.838 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:52:32.838 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:32.838 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:32.838 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:32.838 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:32.838 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:32.838 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:32.839 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:32.839 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:32.839 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:32.839 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:32.839 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:32.839 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:52:32.839 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:52:32.839 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:32.839 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:32.839 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:32.839 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:32.840 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:32.840 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:32.840 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:32.840 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:32.840 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:32.840 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:32.840 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:32.840 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.840 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:52:32.840 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:52:32.840 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:52:32.840 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:52:32.841 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:52:32.841 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:52:32.841 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:52:32.841 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:52:32.841 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:52:32.841 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:52:32.841 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:52:32.841 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:52:32.841 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:52:32.841 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:52:32.841 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:52:32.842 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:52:32.842 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:52:32.842 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:52:32.842 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:52:32.842 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:52:32.842 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:52:32.842 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:52:32.842 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:52:32.842 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:52:32.842 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:52:32.842 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:52:32.842 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:52:32.843 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:32.843 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:52:32.843 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.843 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:52:32.843 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.843 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:52:32.843 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:52:32.843 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:32.843 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:32.843 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.843 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:32.844 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:52:32.844 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:32.844 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.844 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.844 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:32.844 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:52:32.844 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:32.844 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:32.844 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:32.844 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.844 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:52:32.844 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:52:32.845 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:32.845 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:32.845 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:32.845 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.845 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.845 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:52:32.845 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:52:32.845 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:32.845 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:32.845 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.845 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:32.845 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:52:32.846 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:32.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.846 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.846 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:32.846 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:52:32.846 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:32.846 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:32.846 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:32.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.846 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:52:32.846 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:52:32.846 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:32.847 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:32.847 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:32.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.847 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.847 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:52:32.847 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:52:32.847 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:52:32.847 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:52:32.847 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.847 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:52:32.847 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:52:32.847 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:32.848 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.848 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.848 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:52:32.848 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:52:32.848 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:52:32.848 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:32.848 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:32.848 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.848 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:52:32.848 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:52:32.848 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:52:32.848 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:52:32.849 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:52:32.849 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:32.849 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:32.849 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.849 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:52:32.849 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:52:32.849 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.849 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:32.849 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:32.849 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:52:32.849 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:32.849 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:32.850 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:52:32.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.850 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:32.850 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:32.850 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:52:32.850 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:32.850 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:32.850 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:52:32.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.850 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:32.850 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:32.851 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:52:32.851 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:32.851 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:32.851 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:52:32.851 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.851 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:32.851 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:32.851 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:52:32.851 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:32.851 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:52:32.851 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:52:32.851 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:52:32.852 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:52:32.852 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:52:32.852 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:52:32.852 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:52:32.852 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:52:32.852 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:52:32.852 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:52:32.852 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:52:32.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.832+0000 7fd040d24700 1 -- 172.21.15.40:0/3270566391 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd0280666e0 msgr2=0x7fd028068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:32.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.832+0000 7fd040d24700 1 --2- 172.21.15.40:0/3270566391 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd0280666e0 0x7fd028068ba0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd02c0039e0 tx=0x7fd02c006d10 comp rx=0 tx=0).stop 2024-09-13T17:52:32.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.832+0000 7fd040d24700 1 -- 172.21.15.40:0/3270566391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd03c0c8500 msgr2=0x7fd03c06a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:32.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.832+0000 7fd040d24700 1 --2- 172.21.15.40:0/3270566391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd03c0c8500 0x7fd03c06a600 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd034000f80 tx=0x7fd034004af0 comp rx=0 tx=0).stop 2024-09-13T17:52:32.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.832+0000 7fd03bfff700 1 -- 172.21.15.40:0/3270566391 reap_dead start 2024-09-13T17:52:32.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.832+0000 7fd03bfff700 1 -- 172.21.15.40:0/3270566391 reap_dead start 2024-09-13T17:52:32.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.832+0000 7fd040d24700 1 -- 172.21.15.40:0/3270566391 shutdown_connections 2024-09-13T17:52:32.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.832+0000 7fd040d24700 1 -- 172.21.15.40:0/3270566391 >> 172.21.15.40:0/3270566391 conn(0x7fd03c06e6d0 msgr2=0x7fd03c0709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:32.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.832+0000 7fd040d24700 1 -- 172.21.15.40:0/3270566391 shutdown_connections 2024-09-13T17:52:32.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:32.832+0000 7fd040d24700 1 -- 172.21.15.40:0/3270566391 wait complete. 2024-09-13T17:52:32.855 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T17:52:32.855 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T17:52:32.933 INFO:tasks.workunit.client.0.smithi040.stderr:Note: switching to '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'. 2024-09-13T17:52:32.933 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-09-13T17:52:32.933 INFO:tasks.workunit.client.0.smithi040.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-13T17:52:32.934 INFO:tasks.workunit.client.0.smithi040.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-13T17:52:32.934 INFO:tasks.workunit.client.0.smithi040.stderr:state without impacting any branches by switching back to a branch. 2024-09-13T17:52:32.934 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-09-13T17:52:32.934 INFO:tasks.workunit.client.0.smithi040.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-13T17:52:32.934 INFO:tasks.workunit.client.0.smithi040.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-13T17:52:32.934 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-09-13T17:52:32.934 INFO:tasks.workunit.client.0.smithi040.stderr: git switch -c 2024-09-13T17:52:32.934 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-09-13T17:52:32.934 INFO:tasks.workunit.client.0.smithi040.stderr:Or undo this operation with: 2024-09-13T17:52:32.934 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-09-13T17:52:32.934 INFO:tasks.workunit.client.0.smithi040.stderr: git switch - 2024-09-13T17:52:32.934 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-09-13T17:52:32.934 INFO:tasks.workunit.client.0.smithi040.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-13T17:52:32.934 INFO:tasks.workunit.client.0.smithi040.stderr: 2024-09-13T17:52:32.935 INFO:tasks.workunit.client.0.smithi040.stderr:HEAD is now at 6c6b083f148 Merge PR #56690 into wip-rishabh-testing-20240910.100730-reef 2024-09-13T17:52:32.940 DEBUG:teuthology.orchestra.run.smithi040:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-13T17:52:32.974 INFO:tasks.workunit.client.0.smithi040.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-13T17:52:32.977 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-13T17:52:32.978 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-13T17:52:33.791 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:52:33.795 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:52:33.993 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-13T17:52:34.090 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-13T17:52:34.155 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-13T17:52:34.157 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-13T17:52:34.157 INFO:tasks.workunit.client.0.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-13T17:52:34.225 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-13T17:52:34.231 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T17:52:34.231 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-13T17:52:34.286 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-09-13T17:52:34.287 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-09-13T17:52:34.287 DEBUG:teuthology.orchestra.run.smithi040:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-09-13T17:52:34.335 INFO:tasks.workunit.client.0.smithi040.stderr:Cloning into 'ffsb'... 2024-09-13T17:52:39.038 INFO:tasks.workunit.client.0.smithi040.stderr:Updating files: 25% (15/59) Updating files: 27% (16/59) Updating files: 28% (17/59) Updating files: 30% (18/59) Updating files: 32% (19/59) Updating files: 33% (20/59) Updating files: 35% (21/59) Updating files: 37% (22/59) Updating files: 38% (23/59) Updating files: 40% (24/59) Updating files: 42% (25/59) Updating files: 44% (26/59) Updating files: 45% (27/59) Updating files: 47% (28/59) Updating files: 49% (29/59) Updating files: 50% (30/59) Updating files: 52% (31/59) Updating files: 54% (32/59) Updating files: 55% (33/59) Updating files: 57% (34/59) Updating files: 59% (35/59) Updating files: 61% (36/59) Updating files: 62% (37/59) Updating files: 64% (38/59) Updating files: 66% (39/59) Updating files: 67% (40/59) Updating files: 69% (41/59) Updating files: 71% (42/59) Updating files: 72% (43/59) Updating files: 74% (44/59) Updating files: 76% (45/59) Updating files: 77% (46/59) Updating files: 79% (47/59) Updating files: 81% (48/59) Updating files: 83% (49/59) Updating files: 84% (50/59) Updating files: 86% (51/59) Updating files: 88% (52/59) Updating files: 89% (53/59) Updating files: 91% (54/59) Updating files: 93% (55/59) Updating files: 94% (56/59) Updating files: 96% (57/59) Updating files: 98% (58/59) Updating files: 100% (59/59) Updating files: 100% (59/59), done. 2024-09-13T17:52:39.470 INFO:tasks.workunit.client.0.smithi040.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-13T17:52:40.491 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether build environment is sane... yes 2024-09-13T17:52:40.529 INFO:tasks.workunit.client.0.smithi040.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-13T17:52:40.531 INFO:tasks.workunit.client.0.smithi040.stdout:checking for gawk... gawk 2024-09-13T17:52:40.582 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether make sets $(MAKE)... yes 2024-09-13T17:52:40.698 INFO:tasks.workunit.client.0.smithi040.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-13T17:52:40.699 INFO:tasks.workunit.client.0.smithi040.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-13T17:52:40.741 INFO:tasks.workunit.client.0.smithi040.stdout:checking for style of include used by make... GNU 2024-09-13T17:52:40.752 INFO:tasks.workunit.client.0.smithi040.stdout:checking for gcc... gcc 2024-09-13T17:52:40.954 INFO:tasks.workunit.client.0.smithi040.stdout:checking for C compiler default output file name... a.out 2024-09-13T17:52:40.961 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether the C compiler works... yes 2024-09-13T17:52:40.971 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether we are cross compiling... no 2024-09-13T17:52:41.932 INFO:tasks.workunit.client.0.smithi040.stdout:checking for suffix of executables... 2024-09-13T17:52:42.018 INFO:tasks.workunit.client.0.smithi040.stdout:checking for suffix of object files... o 2024-09-13T17:52:42.113 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether we are using the GNU C compiler... yes 2024-09-13T17:52:42.194 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether gcc accepts -g... yes 2024-09-13T17:52:42.350 INFO:tasks.workunit.client.0.smithi040.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-13T17:52:42.666 INFO:tasks.workunit.client.0.smithi040.stdout:checking dependency style of gcc... gcc3 2024-09-13T17:52:43.149 INFO:tasks.workunit.client.0.smithi040.stdout:checking how to run the C preprocessor... gcc -E 2024-09-13T17:52:43.261 INFO:tasks.workunit.client.0.smithi040.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-13T17:52:43.266 INFO:tasks.workunit.client.0.smithi040.stdout:checking for egrep... /usr/bin/grep -E 2024-09-13T17:52:43.295 INFO:tasks.workunit.client.0.smithi040.stdout:checking for AIX... no 2024-09-13T17:52:43.340 INFO:tasks.workunit.client.0.smithi040.stdout:checking for gcc... (cached) gcc 2024-09-13T17:52:43.374 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-13T17:52:43.377 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-13T17:52:43.379 INFO:tasks.workunit.client.0.smithi040.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-13T17:52:43.381 INFO:tasks.workunit.client.0.smithi040.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-13T17:52:43.545 INFO:tasks.workunit.client.0.smithi040.stdout:checking for main in -lm... yes 2024-09-13T17:52:43.706 INFO:tasks.workunit.client.0.smithi040.stdout:checking for main in -lpthread... yes 2024-09-13T17:52:44.120 INFO:tasks.workunit.client.0.smithi040.stdout:checking for ANSI C header files... yes 2024-09-13T17:52:44.249 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-13T17:52:44.374 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sys/types.h... yes 2024-09-13T17:52:44.517 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sys/stat.h... yes 2024-09-13T17:52:44.650 INFO:tasks.workunit.client.0.smithi040.stdout:checking for stdlib.h... yes 2024-09-13T17:52:44.793 INFO:tasks.workunit.client.0.smithi040.stdout:checking for string.h... yes 2024-09-13T17:52:44.933 INFO:tasks.workunit.client.0.smithi040.stdout:checking for memory.h... yes 2024-09-13T17:52:45.070 INFO:tasks.workunit.client.0.smithi040.stdout:checking for strings.h... yes 2024-09-13T17:52:45.214 INFO:tasks.workunit.client.0.smithi040.stdout:checking for inttypes.h... yes 2024-09-13T17:52:45.357 INFO:tasks.workunit.client.0.smithi040.stdout:checking for stdint.h... yes 2024-09-13T17:52:45.850 INFO:tasks.workunit.client.0.smithi040.stdout:checking for unistd.h... yes 2024-09-13T17:52:46.033 INFO:tasks.workunit.client.0.smithi040.stdout:checking pthread.h usability... yes 2024-09-13T17:52:46.095 INFO:tasks.workunit.client.0.smithi040.stdout:checking pthread.h presence... yes 2024-09-13T17:52:46.097 INFO:tasks.workunit.client.0.smithi040.stdout:checking for pthread.h... yes 2024-09-13T17:52:46.271 INFO:tasks.workunit.client.0.smithi040.stdout:checking fcntl.h usability... yes 2024-09-13T17:52:46.337 INFO:tasks.workunit.client.0.smithi040.stdout:checking fcntl.h presence... yes 2024-09-13T17:52:46.339 INFO:tasks.workunit.client.0.smithi040.stdout:checking for fcntl.h... yes 2024-09-13T17:52:46.510 INFO:tasks.workunit.client.0.smithi040.stdout:checking limits.h usability... yes 2024-09-13T17:52:46.564 INFO:tasks.workunit.client.0.smithi040.stdout:checking limits.h presence... yes 2024-09-13T17:52:46.566 INFO:tasks.workunit.client.0.smithi040.stdout:checking for limits.h... yes 2024-09-13T17:52:46.622 INFO:tasks.workunit.client.0.smithi040.stdout:checking for stdint.h... (cached) yes 2024-09-13T17:52:46.817 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/time.h usability... yes 2024-09-13T17:52:46.873 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/time.h presence... yes 2024-09-13T17:52:46.876 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sys/time.h... yes 2024-09-13T17:52:46.933 INFO:tasks.workunit.client.0.smithi040.stdout:checking for unistd.h... (cached) yes 2024-09-13T17:52:47.097 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/vfs.h usability... yes 2024-09-13T17:52:47.145 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/vfs.h presence... yes 2024-09-13T17:52:47.146 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sys/vfs.h... yes 2024-09-13T17:52:48.284 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/limits.h usability... no 2024-09-13T17:52:48.342 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/limits.h presence... no 2024-09-13T17:52:48.344 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sys/limits.h... no 2024-09-13T17:52:48.449 INFO:tasks.workunit.client.0.smithi040.stdout:checking for an ANSI C-conforming const... yes 2024-09-13T17:52:48.546 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-13T17:52:48.973 INFO:tasks.workunit.client.0.smithi040.stdout:checking for function prototypes... yes 2024-09-13T17:52:49.093 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether setvbuf arguments are reversed... no 2024-09-13T17:52:49.286 INFO:tasks.workunit.client.0.smithi040.stdout:checking for system... yes 2024-09-13T17:52:49.510 INFO:tasks.workunit.client.0.smithi040.stdout:checking for gettimeofday... yes 2024-09-13T17:52:50.629 INFO:tasks.workunit.client.0.smithi040.stdout:checking for mkdir... yes 2024-09-13T17:52:50.838 INFO:tasks.workunit.client.0.smithi040.stdout:checking for strerror... yes 2024-09-13T17:52:51.059 INFO:tasks.workunit.client.0.smithi040.stdout:checking for open64... yes 2024-09-13T17:52:51.260 INFO:tasks.workunit.client.0.smithi040.stdout:checking for stat64... yes 2024-09-13T17:52:51.479 INFO:tasks.workunit.client.0.smithi040.stdout:checking for fseeko64... yes 2024-09-13T17:52:51.701 INFO:tasks.workunit.client.0.smithi040.stdout:checking for lrand48_r... yes 2024-09-13T17:52:51.912 INFO:tasks.workunit.client.0.smithi040.stdout:checking for srand48_r... yes 2024-09-13T17:52:52.001 INFO:tasks.workunit.client.0.smithi040.stdout:configure: creating ./config.status 2024-09-13T17:52:52.344 INFO:tasks.workunit.client.0.smithi040.stdout:config.status: creating Makefile 2024-09-13T17:52:52.401 INFO:tasks.workunit.client.0.smithi040.stdout:config.status: creating config.h 2024-09-13T17:52:52.466 INFO:tasks.workunit.client.0.smithi040.stdout:config.status: executing depfiles commands 2024-09-13T17:52:52.634 INFO:tasks.workunit.client.0.smithi040.stdout:config.status: executing default commands 2024-09-13T17:52:52.856 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-13T17:52:52.856 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-13T17:52:52.856 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T17:52:52.857 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T17:52:52.859 INFO:tasks.ceph.mon.c:Started 2024-09-13T17:52:52.859 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T17:52:52.860 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:52:52.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.975+0000 7f6e795f4700 1 Processor -- start 2024-09-13T17:52:52.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.975+0000 7f6e795f4700 1 -- start start 2024-09-13T17:52:52.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e795f4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6e740c70d0 0x7f6e740c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:52.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e795f4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6e74070c40 0x7f6e74070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:52.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e795f4700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7415a480 con 0x7f6e740c74b0 2024-09-13T17:52:52.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e795f4700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e74159620 con 0x7f6e740c70d0 2024-09-13T17:52:52.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e795f4700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e74159870 con 0x7f6e74070c40 2024-09-13T17:52:52.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e73fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6e740c70d0 0x7f6e740c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:52.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e73fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6e740c70d0 0x7f6e740c87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34444/0 (socket says 172.21.15.40:34444) 2024-09-13T17:52:52.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e73fff700 1 -- 172.21.15.40:0/2683055628 learned_addr learned my addr 172.21.15.40:0/2683055628 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:52.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e78df3700 1 --2- 172.21.15.40:0/2683055628 >> v2:172.21.15.43:3300/0 conn(0x7f6e74070c40 0x7f6e74070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:52.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e73fff700 1 -- 172.21.15.40:0/2683055628 >> v2:172.21.15.43:3300/0 conn(0x7f6e74070c40 msgr2=0x7f6e74070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:52:52.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e73fff700 1 --2- 172.21.15.40:0/2683055628 >> v2:172.21.15.43:3300/0 conn(0x7f6e74070c40 0x7f6e74070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:52.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e73fff700 1 -- 172.21.15.40:0/2683055628 >> v1:172.21.15.40:6789/0 conn(0x7f6e740c74b0 legacy=0x7f6e74156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:52.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e73fff700 1 -- 172.21.15.40:0/2683055628 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e74159b50 con 0x7f6e740c70d0 2024-09-13T17:52:52.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e72ffd700 1 -- 172.21.15.40:0/2683055628 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2068702265 0 0) 0x7f6e7415a480 con 0x7f6e740c74b0 2024-09-13T17:52:52.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e78df3700 1 --2- 172.21.15.40:0/2683055628 >> v2:172.21.15.43:3300/0 conn(0x7f6e74070c40 0x7f6e74070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:52:52.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e73fff700 1 --2- 172.21.15.40:0/2683055628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e740c70d0 0x7f6e740c87f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6e740c7890 tx=0x7f6e64009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7430196e3321788 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:52.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e72ffd700 1 -- 172.21.15.40:0/2683055628 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64011830 con 0x7f6e740c70d0 2024-09-13T17:52:52.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e72ffd700 1 -- 172.21.15.40:0/2683055628 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e64011990 con 0x7f6e740c70d0 2024-09-13T17:52:52.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e72ffd700 1 -- 172.21.15.40:0/2683055628 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64011d20 con 0x7f6e740c70d0 2024-09-13T17:52:52.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e795f4700 1 -- 172.21.15.40:0/2683055628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e740c70d0 msgr2=0x7f6e740c87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:52.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e795f4700 1 --2- 172.21.15.40:0/2683055628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e740c70d0 0x7f6e740c87f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6e740c7890 tx=0x7f6e64009bf0 comp rx=0 tx=0).stop 2024-09-13T17:52:52.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e795f4700 1 -- 172.21.15.40:0/2683055628 shutdown_connections 2024-09-13T17:52:52.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e795f4700 1 --2- 172.21.15.40:0/2683055628 >> v2:172.21.15.43:3300/0 conn(0x7f6e74070c40 0x7f6e74070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:52.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e795f4700 1 --2- 172.21.15.40:0/2683055628 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e740c70d0 0x7f6e740c87f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:52.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e795f4700 1 -- 172.21.15.40:0/2683055628 >> 172.21.15.40:0/2683055628 conn(0x7f6e7406a390 msgr2=0x7f6e7406a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:52.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e795f4700 1 -- 172.21.15.40:0/2683055628 shutdown_connections 2024-09-13T17:52:52.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.979+0000 7f6e795f4700 1 -- 172.21.15.40:0/2683055628 wait complete. 2024-09-13T17:52:52.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.983+0000 7f6e795f4700 1 Processor -- start 2024-09-13T17:52:52.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.983+0000 7f6e795f4700 1 -- start start 2024-09-13T17:52:52.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e795f4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e74070c40 0x7f6e74155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:52.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e795f4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e740c70d0 0x7f6e741efdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:52.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e795f4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e740c74b0 0x7f6e74148b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:52.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e795f4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e74158ff0 con 0x7f6e740c70d0 2024-09-13T17:52:52.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e795f4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e740cbb60 con 0x7f6e740c74b0 2024-09-13T17:52:52.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e795f4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e74159410 con 0x7f6e74070c40 2024-09-13T17:52:52.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e78df3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e740c74b0 0x7f6e74148b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:52.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e737fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e740c70d0 0x7f6e741efdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:52.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e78df3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e740c74b0 0x7f6e74148b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:38270/0 (socket says 172.21.15.40:38270) 2024-09-13T17:52:52.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e73fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e74070c40 0x7f6e74155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:52.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e78df3700 1 -- 172.21.15.40:0/3615638125 learned_addr learned my addr 172.21.15.40:0/3615638125 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:52:52.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e73fff700 1 -- 172.21.15.40:0/3615638125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e74070c40 msgr2=0x7f6e74155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T17:52:52.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e73fff700 1 -- 172.21.15.40:0/3615638125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e74070c40 msgr2=0x7f6e74155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:52:52.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e73fff700 1 --2- 172.21.15.40:0/3615638125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e74070c40 0x7f6e74155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:52:52.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e73fff700 1 --2- 172.21.15.40:0/3615638125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e74070c40 0x7f6e74155290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:52:52.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e737fe700 1 -- 172.21.15.40:0/3615638125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e74070c40 msgr2=0x7f6e74155290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:52:52.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e737fe700 1 --2- 172.21.15.40:0/3615638125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e74070c40 0x7f6e74155290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:52.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e737fe700 1 -- 172.21.15.40:0/3615638125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e740c74b0 msgr2=0x7f6e74148b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:52.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e737fe700 1 --2- 172.21.15.40:0/3615638125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e740c74b0 0x7f6e74148b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:52.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e737fe700 1 -- 172.21.15.40:0/3615638125 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c009a70 con 0x7f6e740c70d0 2024-09-13T17:52:52.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e737fe700 1 --2- 172.21.15.40:0/3615638125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e740c70d0 0x7f6e741efdf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800c410 tx=0x7f6e6800c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:52.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e70ff9700 1 -- 172.21.15.40:0/3615638125 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e680054e0 con 0x7f6e740c70d0 2024-09-13T17:52:52.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e70ff9700 1 -- 172.21.15.40:0/3615638125 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e680036a0 con 0x7f6e740c70d0 2024-09-13T17:52:52.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e795f4700 1 -- 172.21.15.40:0/3615638125 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e64009570 con 0x7f6e740c70d0 2024-09-13T17:52:52.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e70ff9700 1 -- 172.21.15.40:0/3615638125 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68003a50 con 0x7f6e740c70d0 2024-09-13T17:52:52.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.987+0000 7f6e795f4700 1 -- 172.21.15.40:0/3615638125 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e74153a70 con 0x7f6e740c70d0 2024-09-13T17:52:52.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.991+0000 7f6e795f4700 1 -- 172.21.15.40:0/3615638125 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e74070290 con 0x7f6e740c70d0 2024-09-13T17:52:52.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.991+0000 7f6e70ff9700 1 -- 172.21.15.40:0/3615638125 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6e68003bb0 con 0x7f6e740c70d0 2024-09-13T17:52:52.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.991+0000 7f6e70ff9700 1 --2- 172.21.15.40:0/3615638125 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e60066730 0x7f6e60068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:52:52.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.991+0000 7f6e70ff9700 1 -- 172.21.15.40:0/3615638125 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6e68083920 con 0x7f6e740c70d0 2024-09-13T17:52:52.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.991+0000 7f6e73fff700 1 --2- 172.21.15.40:0/3615638125 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e60066730 0x7f6e60068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:52:52.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.991+0000 7f6e73fff700 1 --2- 172.21.15.40:0/3615638125 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e60066730 0x7f6e60068bf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6e64000c00 tx=0x7f6e64008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:52:53.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:52.995+0000 7f6e70ff9700 1 -- 172.21.15.40:0/3615638125 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6e68050830 con 0x7f6e740c70d0 2024-09-13T17:52:53.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:53.243+0000 7f6e795f4700 1 -- 172.21.15.40:0/3615638125 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f6e7414f660 con 0x7f6e740c70d0 2024-09-13T17:52:53.608 INFO:tasks.workunit.client.0.smithi040.stdout:cd . && /bin/bash /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-09-13T17:52:53.625 INFO:tasks.workunit.client.0.smithi040.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-09-13T17:52:53.629 INFO:tasks.workunit.client.0.smithi040.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-09-13T17:52:53.632 INFO:tasks.workunit.client.0.smithi040.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-09-13T17:52:53.632 INFO:tasks.workunit.client.0.smithi040.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-09-13T17:52:53.633 INFO:tasks.workunit.client.0.smithi040.stderr: any GNU archive site. 2024-09-13T17:52:53.642 INFO:tasks.workunit.client.0.smithi040.stdout: cd . && /bin/bash /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-09-13T17:52:53.655 INFO:tasks.workunit.client.0.smithi040.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-09-13T17:52:53.658 INFO:tasks.workunit.client.0.smithi040.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-09-13T17:52:53.658 INFO:tasks.workunit.client.0.smithi040.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-09-13T17:52:53.658 INFO:tasks.workunit.client.0.smithi040.stderr: You might want to install the `Automake' and `Perl' packages. 2024-09-13T17:52:53.658 INFO:tasks.workunit.client.0.smithi040.stderr: Grab them from any GNU archive site. 2024-09-13T17:52:58.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:58.199+0000 7f6e70ff9700 1 -- 172.21.15.40:0/3615638125 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7f6e68003e60 con 0x7f6e740c70d0 2024-09-13T17:52:58.204 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":22,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:52:58.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:58.203+0000 7f6e795f4700 1 -- 172.21.15.40:0/3615638125 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e60066730 msgr2=0x7f6e60068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:58.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:58.203+0000 7f6e795f4700 1 --2- 172.21.15.40:0/3615638125 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e60066730 0x7f6e60068bf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6e64000c00 tx=0x7f6e64008030 comp rx=0 tx=0).stop 2024-09-13T17:52:58.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:58.203+0000 7f6e795f4700 1 -- 172.21.15.40:0/3615638125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e740c70d0 msgr2=0x7f6e741efdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:52:58.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:58.203+0000 7f6e795f4700 1 --2- 172.21.15.40:0/3615638125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e740c70d0 0x7f6e741efdf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800c410 tx=0x7f6e6800c720 comp rx=0 tx=0).stop 2024-09-13T17:52:58.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:58.203+0000 7f6e795f4700 1 -- 172.21.15.40:0/3615638125 shutdown_connections 2024-09-13T17:52:58.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:58.203+0000 7f6e795f4700 1 --2- 172.21.15.40:0/3615638125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e740c74b0 0x7f6e74148b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:58.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:58.203+0000 7f6e795f4700 1 --2- 172.21.15.40:0/3615638125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e740c70d0 0x7f6e741efdf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:58.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:58.203+0000 7f6e795f4700 1 --2- 172.21.15.40:0/3615638125 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e60066730 0x7f6e60068bf0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:58.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:58.203+0000 7f6e795f4700 1 --2- 172.21.15.40:0/3615638125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e74070c40 0x7f6e74155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:52:58.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:58.203+0000 7f6e795f4700 1 -- 172.21.15.40:0/3615638125 >> 172.21.15.40:0/3615638125 conn(0x7f6e7406a390 msgr2=0x7f6e740c8660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:52:58.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:58.203+0000 7f6e795f4700 1 -- 172.21.15.40:0/3615638125 shutdown_connections 2024-09-13T17:52:58.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:52:58.203+0000 7f6e795f4700 1 -- 172.21.15.40:0/3615638125 wait complete. 2024-09-13T17:52:58.738 INFO:tasks.workunit.client.0.smithi040.stdout:cd . && /bin/bash /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-09-13T17:52:59.044 INFO:tasks.workunit.client.0.smithi040.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-09-13T17:52:59.044 INFO:tasks.workunit.client.0.smithi040.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-09-13T17:52:59.044 INFO:tasks.workunit.client.0.smithi040.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-09-13T17:52:59.045 INFO:tasks.workunit.client.0.smithi040.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-09-13T17:52:59.253 INFO:tasks.workunit.client.0.smithi040.stdout:/bin/bash ./config.status --recheck 2024-09-13T17:53:00.019 INFO:tasks.workunit.client.0.smithi040.stdout:running CONFIG_SHELL=/bin/bash /bin/bash ./configure --no-create --no-recursion 2024-09-13T17:53:01.229 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:53:01.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c6ca7c700 1 Processor -- start 2024-09-13T17:53:01.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c6ca7c700 1 -- start start 2024-09-13T17:53:01.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c6ca7c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1c60036150 0x7f1c60007840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c6ca7c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c60033260 0x7f1c600e5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c6ca7c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c600db8a0 con 0x7f1c60051190 2024-09-13T17:53:01.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c6ca7c700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c600067c0 con 0x7f1c60036150 2024-09-13T17:53:01.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c6ca7c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c60006aa0 con 0x7f1c60033260 2024-09-13T17:53:01.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c66ffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1c60051190 0x7f1c6004c3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50074/0 (socket says 172.21.15.40:50074) 2024-09-13T17:53:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c66ffd700 1 -- 172.21.15.40:0/2445251967 learned_addr learned my addr 172.21.15.40:0/2445251967 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:53:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c677fe700 1 --2- 172.21.15.40:0/2445251967 >> v2:172.21.15.40:3300/0 conn(0x7f1c60036150 0x7f1c60007840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c67fff700 1 --2- 172.21.15.40:0/2445251967 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c60033260 0x7f1c600e5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c67fff700 1 -- 172.21.15.40:0/2445251967 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c60033260 msgr2=0x7f1c600e5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:53:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c67fff700 1 -- 172.21.15.40:0/2445251967 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c60033260 msgr2=0x7f1c600e5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:53:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c67fff700 1 --2- 172.21.15.40:0/2445251967 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c60033260 0x7f1c600e5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c667fc700 1 -- 172.21.15.40:0/2445251967 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139145415 0 0) 0x7f1c600db8a0 con 0x7f1c60051190 2024-09-13T17:53:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c67fff700 1 --2- 172.21.15.40:0/2445251967 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c60033260 0x7f1c600e5380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c667fc700 1 -- 172.21.15.40:0/2445251967 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c500035d0 con 0x7f1c60051190 2024-09-13T17:53:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c677fe700 1 -- 172.21.15.40:0/2445251967 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c60033260 msgr2=0x7f1c600e5380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:53:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c677fe700 1 --2- 172.21.15.40:0/2445251967 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c60033260 0x7f1c600e5380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c677fe700 1 -- 172.21.15.40:0/2445251967 >> v1:172.21.15.40:6789/0 conn(0x7f1c60051190 legacy=0x7f1c6004c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c677fe700 1 -- 172.21.15.40:0/2445251967 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c60006d80 con 0x7f1c60036150 2024-09-13T17:53:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c677fe700 1 --2- 172.21.15.40:0/2445251967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c60036150 0x7f1c60007840 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1c580077b0 tx=0x7f1c58007ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62e96cf80a68e8b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c667fc700 1 -- 172.21.15.40:0/2445251967 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5800a070 con 0x7f1c60036150 2024-09-13T17:53:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c667fc700 1 -- 172.21.15.40:0/2445251967 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5800f7b0 con 0x7f1c60036150 2024-09-13T17:53:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.355+0000 7f1c667fc700 1 -- 172.21.15.40:0/2445251967 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5800fb40 con 0x7f1c60036150 2024-09-13T17:53:01.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.359+0000 7f1c6ca7c700 1 -- 172.21.15.40:0/2445251967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c60036150 msgr2=0x7f1c60007840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.359+0000 7f1c6ca7c700 1 --2- 172.21.15.40:0/2445251967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c60036150 0x7f1c60007840 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1c580077b0 tx=0x7f1c58007ac0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.359+0000 7f1c6ca7c700 1 -- 172.21.15.40:0/2445251967 shutdown_connections 2024-09-13T17:53:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.359+0000 7f1c6ca7c700 1 --2- 172.21.15.40:0/2445251967 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c60033260 0x7f1c600e5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.359+0000 7f1c6ca7c700 1 --2- 172.21.15.40:0/2445251967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c60036150 0x7f1c60007840 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.359+0000 7f1c6ca7c700 1 -- 172.21.15.40:0/2445251967 >> 172.21.15.40:0/2445251967 conn(0x7f1c600505e0 msgr2=0x7f1c600509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.359+0000 7f1c6ca7c700 1 -- 172.21.15.40:0/2445251967 shutdown_connections 2024-09-13T17:53:01.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.359+0000 7f1c6ca7c700 1 -- 172.21.15.40:0/2445251967 wait complete. 2024-09-13T17:53:01.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.359+0000 7f1c6ca7c700 1 Processor -- start 2024-09-13T17:53:01.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.359+0000 7f1c6ca7c700 1 -- start start 2024-09-13T17:53:01.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c6ca7c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c60033260 0x7f1c6004e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c6ca7c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c60036150 0x7f1c60170370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c6ca7c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c60051190 0x7f1c601708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c6ca7c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6004a0a0 con 0x7f1c60051190 2024-09-13T17:53:01.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c6ca7c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c60174840 con 0x7f1c60033260 2024-09-13T17:53:01.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c6ca7c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6004a2d0 con 0x7f1c60036150 2024-09-13T17:53:01.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c67fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c60051190 0x7f1c601708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c67fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c60051190 0x7f1c601708b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49356/0 (socket says 172.21.15.40:49356) 2024-09-13T17:53:01.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c67fff700 1 -- 172.21.15.40:0/1802687595 learned_addr learned my addr 172.21.15.40:0/1802687595 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:01.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c66ffd700 1 --2- 172.21.15.40:0/1802687595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c60036150 0x7f1c60170370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c677fe700 1 --2- 172.21.15.40:0/1802687595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c60033260 0x7f1c6004e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c66ffd700 1 -- 172.21.15.40:0/1802687595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c60036150 msgr2=0x7f1c60170370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T17:53:01.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c66ffd700 1 -- 172.21.15.40:0/1802687595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c60036150 msgr2=0x7f1c60170370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:53:01.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c66ffd700 1 --2- 172.21.15.40:0/1802687595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c60036150 0x7f1c60170370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:01.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c66ffd700 1 --2- 172.21.15.40:0/1802687595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c60036150 0x7f1c60170370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:01.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c67fff700 1 -- 172.21.15.40:0/1802687595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c60036150 msgr2=0x7f1c60170370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:53:01.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c67fff700 1 --2- 172.21.15.40:0/1802687595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c60036150 0x7f1c60170370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c67fff700 1 -- 172.21.15.40:0/1802687595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c60033260 msgr2=0x7f1c6004e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c67fff700 1 --2- 172.21.15.40:0/1802687595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c60033260 0x7f1c6004e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c67fff700 1 -- 172.21.15.40:0/1802687595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c580073c0 con 0x7f1c60051190 2024-09-13T17:53:01.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c67fff700 1 --2- 172.21.15.40:0/1802687595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c60051190 0x7f1c601708b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1c6804f940 tx=0x7f1c68067dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:01.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c4ffff700 1 -- 172.21.15.40:0/1802687595 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68067720 con 0x7f1c60051190 2024-09-13T17:53:01.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c4ffff700 1 -- 172.21.15.40:0/1802687595 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c68067880 con 0x7f1c60051190 2024-09-13T17:53:01.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c4ffff700 1 -- 172.21.15.40:0/1802687595 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68072310 con 0x7f1c60051190 2024-09-13T17:53:01.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c6ca7c700 1 -- 172.21.15.40:0/1802687595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c60173a80 con 0x7f1c60051190 2024-09-13T17:53:01.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.363+0000 7f1c6ca7c700 1 -- 172.21.15.40:0/1802687595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c60174040 con 0x7f1c60051190 2024-09-13T17:53:01.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.367+0000 7f1c6ca7c700 1 -- 172.21.15.40:0/1802687595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c6004b9b0 con 0x7f1c60051190 2024-09-13T17:53:01.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.367+0000 7f1c4ffff700 1 -- 172.21.15.40:0/1802687595 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1c68072470 con 0x7f1c60051190 2024-09-13T17:53:01.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.367+0000 7f1c4ffff700 1 --2- 172.21.15.40:0/1802687595 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1c54066730 0x7f1c54068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.367+0000 7f1c4ffff700 1 -- 172.21.15.40:0/1802687595 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1c680fed20 con 0x7f1c60051190 2024-09-13T17:53:01.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.367+0000 7f1c677fe700 1 --2- 172.21.15.40:0/1802687595 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1c54066730 0x7f1c54068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.367+0000 7f1c677fe700 1 --2- 172.21.15.40:0/1802687595 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1c54066730 0x7f1c54068bf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1c58012040 tx=0x7f1c5800a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:01.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.371+0000 7f1c4ffff700 1 -- 172.21.15.40:0/1802687595 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1c680cf6d0 con 0x7f1c60051190 2024-09-13T17:53:01.450 INFO:tasks.workunit.client.0.smithi040.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-13T17:53:01.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.611+0000 7f1c6ca7c700 1 -- 172.21.15.40:0/1802687595 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1c600db450 con 0x7f1c60051190 2024-09-13T17:53:01.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.615+0000 7f1c4ffff700 1 -- 172.21.15.40:0/1802687595 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7f1c6809a070 con 0x7f1c60051190 2024-09-13T17:53:01.619 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":24,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:53:01.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.615+0000 7f1c6ca7c700 1 -- 172.21.15.40:0/1802687595 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1c54066730 msgr2=0x7f1c54068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.615+0000 7f1c6ca7c700 1 --2- 172.21.15.40:0/1802687595 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1c54066730 0x7f1c54068bf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1c58012040 tx=0x7f1c5800a000 comp rx=0 tx=0).stop 2024-09-13T17:53:01.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.615+0000 7f1c6ca7c700 1 -- 172.21.15.40:0/1802687595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c60051190 msgr2=0x7f1c601708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.615+0000 7f1c6ca7c700 1 --2- 172.21.15.40:0/1802687595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c60051190 0x7f1c601708b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1c6804f940 tx=0x7f1c68067dd0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.615+0000 7f1c6ca7c700 1 -- 172.21.15.40:0/1802687595 shutdown_connections 2024-09-13T17:53:01.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.615+0000 7f1c6ca7c700 1 --2- 172.21.15.40:0/1802687595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c60051190 0x7f1c601708b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.615+0000 7f1c6ca7c700 1 --2- 172.21.15.40:0/1802687595 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1c54066730 0x7f1c54068bf0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.615+0000 7f1c6ca7c700 1 --2- 172.21.15.40:0/1802687595 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c60036150 0x7f1c60170370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.615+0000 7f1c6ca7c700 1 --2- 172.21.15.40:0/1802687595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c60033260 0x7f1c6004e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.615+0000 7f1c6ca7c700 1 -- 172.21.15.40:0/1802687595 >> 172.21.15.40:0/1802687595 conn(0x7f1c600505e0 msgr2=0x7f1c6004f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:01.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.615+0000 7f1c6ca7c700 1 -- 172.21.15.40:0/1802687595 shutdown_connections 2024-09-13T17:53:01.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.615+0000 7f1c6ca7c700 1 -- 172.21.15.40:0/1802687595 wait complete. 2024-09-13T17:53:01.634 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T17:53:01.634 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T17:53:01.634 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:53:01.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.743+0000 7fce1a566700 1 Processor -- start 2024-09-13T17:53:01.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.743+0000 7fce1a566700 1 -- start start 2024-09-13T17:53:01.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.743+0000 7fce1a566700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fce140c8340 0x7fce14072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.743+0000 7fce1a566700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce140c5290 con 0x7fce140c7f60 2024-09-13T17:53:01.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.743+0000 7fce1a566700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce140727f0 con 0x7fce140c7320 2024-09-13T17:53:01.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.743+0000 7fce1a566700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce14071390 con 0x7fce140c8340 2024-09-13T17:53:01.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.743+0000 7fce19564700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fce140c7f60 0x7fce1415ab50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50086/0 (socket says 172.21.15.40:50086) 2024-09-13T17:53:01.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.743+0000 7fce19564700 1 -- 172.21.15.40:0/3025315652 learned_addr learned my addr 172.21.15.40:0/3025315652 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:53:01.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.743+0000 7fce19d65700 1 --2- 172.21.15.40:0/3025315652 >> v2:172.21.15.43:3300/0 conn(0x7fce140c8340 0x7fce14072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.743+0000 7fce0bfff700 1 -- 172.21.15.40:0/3025315652 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1932069060 0 0) 0x7fce140c5290 con 0x7fce140c7f60 2024-09-13T17:53:01.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.743+0000 7fce0bfff700 1 -- 172.21.15.40:0/3025315652 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce000035d0 con 0x7fce140c7f60 2024-09-13T17:53:01.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.743+0000 7fce19d65700 1 -- 172.21.15.40:0/3025315652 >> v1:172.21.15.43:6789/0 conn(0x7fce140c7320 legacy=0x7fce14151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.743+0000 7fce19d65700 1 -- 172.21.15.40:0/3025315652 >> v1:172.21.15.40:6789/0 conn(0x7fce140c7f60 legacy=0x7fce1415ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.743+0000 7fce19d65700 1 -- 172.21.15.40:0/3025315652 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce140c5870 con 0x7fce140c8340 2024-09-13T17:53:01.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.743+0000 7fce19d65700 1 --2- 172.21.15.40:0/3025315652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce140c8340 0x7fce14072200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fce0c009850 tx=0x7fce0c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88519f27ac0b8f4a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:01.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.747+0000 7fce0bfff700 1 -- 172.21.15.40:0/3025315652 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c011830 con 0x7fce140c8340 2024-09-13T17:53:01.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.747+0000 7fce0bfff700 1 -- 172.21.15.40:0/3025315652 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0c011990 con 0x7fce140c8340 2024-09-13T17:53:01.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.747+0000 7fce0bfff700 1 -- 172.21.15.40:0/3025315652 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c011d40 con 0x7fce140c8340 2024-09-13T17:53:01.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.747+0000 7fce1a566700 1 -- 172.21.15.40:0/3025315652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce140c8340 msgr2=0x7fce14072200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.747+0000 7fce1a566700 1 --2- 172.21.15.40:0/3025315652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce140c8340 0x7fce14072200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fce0c009850 tx=0x7fce0c009b60 comp rx=0 tx=0).stop 2024-09-13T17:53:01.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.747+0000 7fce1a566700 1 -- 172.21.15.40:0/3025315652 shutdown_connections 2024-09-13T17:53:01.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.747+0000 7fce1a566700 1 --2- 172.21.15.40:0/3025315652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce140c8340 0x7fce14072200 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.747+0000 7fce1a566700 1 -- 172.21.15.40:0/3025315652 >> 172.21.15.40:0/3025315652 conn(0x7fce14066090 msgr2=0x7fce140664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:01.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.747+0000 7fce1a566700 1 -- 172.21.15.40:0/3025315652 shutdown_connections 2024-09-13T17:53:01.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.747+0000 7fce1a566700 1 -- 172.21.15.40:0/3025315652 wait complete. 2024-09-13T17:53:01.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.747+0000 7fce1a566700 1 Processor -- start 2024-09-13T17:53:01.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.747+0000 7fce1a566700 1 -- start start 2024-09-13T17:53:01.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.747+0000 7fce1a566700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce140c7320 0x7fce140c0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.747+0000 7fce1a566700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce140c7f60 0x7fce141f2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.747+0000 7fce1a566700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce140c8340 0x7fce141e1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.747+0000 7fce1a566700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce141e1c60 con 0x7fce140c7f60 2024-09-13T17:53:01.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.747+0000 7fce1a566700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce14144500 con 0x7fce140c7320 2024-09-13T17:53:01.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.747+0000 7fce1a566700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce14143ca0 con 0x7fce140c8340 2024-09-13T17:53:01.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce18d63700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce140c7f60 0x7fce141f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce18d63700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce140c7f60 0x7fce141f2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49372/0 (socket says 172.21.15.40:49372) 2024-09-13T17:53:01.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce18d63700 1 -- 172.21.15.40:0/3515553551 learned_addr learned my addr 172.21.15.40:0/3515553551 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:01.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce19564700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce140c7320 0x7fce140c0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce19d65700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce140c8340 0x7fce141e1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce19d65700 1 -- 172.21.15.40:0/3515553551 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce140c8340 msgr2=0x7fce141e1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-13T17:53:01.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce19d65700 1 -- 172.21.15.40:0/3515553551 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce140c8340 msgr2=0x7fce141e1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:53:01.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce19d65700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce140c8340 0x7fce141e1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:01.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce19d65700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce140c8340 0x7fce141e1720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:01.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce18d63700 1 -- 172.21.15.40:0/3515553551 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce140c8340 msgr2=0x7fce141e1720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:53:01.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce18d63700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce140c8340 0x7fce141e1720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce18d63700 1 -- 172.21.15.40:0/3515553551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce140c7320 msgr2=0x7fce140c0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce18d63700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce140c7320 0x7fce140c0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce18d63700 1 -- 172.21.15.40:0/3515553551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0c0094b0 con 0x7fce140c7f60 2024-09-13T17:53:01.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce18d63700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce140c7f60 0x7fce141f2340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fce10001ed0 tx=0x7fce100021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:01.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce09ffb700 1 -- 172.21.15.40:0/3515553551 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce10003c20 con 0x7fce140c7f60 2024-09-13T17:53:01.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce09ffb700 1 -- 172.21.15.40:0/3515553551 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce10003d80 con 0x7fce140c7f60 2024-09-13T17:53:01.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce1a566700 1 -- 172.21.15.40:0/3515553551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce14143f40 con 0x7fce140c7f60 2024-09-13T17:53:01.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce09ffb700 1 -- 172.21.15.40:0/3515553551 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1000abf0 con 0x7fce140c7f60 2024-09-13T17:53:01.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce1a566700 1 -- 172.21.15.40:0/3515553551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce140c0470 con 0x7fce140c7f60 2024-09-13T17:53:01.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.751+0000 7fce09ffb700 1 -- 172.21.15.40:0/3515553551 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fce1000ad50 con 0x7fce140c7f60 2024-09-13T17:53:01.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.755+0000 7fce1a566700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce141e0840 0x7fce141e2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.755+0000 7fce1a566700 1 -- 172.21.15.40:0/3515553551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce14067720 con 0x7fce141e0840 2024-09-13T17:53:01.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.755+0000 7fce19564700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce141e0840 0x7fce141e2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.755+0000 7fce09ffb700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcdf8066730 0x7fcdf8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.755+0000 7fce09ffb700 1 -- 172.21.15.40:0/3515553551 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fce10084650 con 0x7fce140c7f60 2024-09-13T17:53:01.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.755+0000 7fce19564700 1 -- 172.21.15.40:0/3515553551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fce140c6110 con 0x7fce141e0840 2024-09-13T17:53:01.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.755+0000 7fce19d65700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcdf8066730 0x7fcdf8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.755+0000 7fce19564700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce141e0840 0x7fce141e2530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fce040011e0 tx=0x7fce04001950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:01.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.755+0000 7fce09ffb700 1 -- 172.21.15.40:0/3515553551 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce04012560 con 0x7fce141e0840 2024-09-13T17:53:01.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.755+0000 7fce19d65700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcdf8066730 0x7fcdf8068bf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fce0c009850 tx=0x7fce0c01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:01.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.755+0000 7fce09ffb700 1 -- 172.21.15.40:0/3515553551 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fce040126c0 con 0x7fce141e0840 2024-09-13T17:53:01.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.755+0000 7fce09ffb700 1 -- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce141e0840 msgr2=0x7fce141e2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.755+0000 7fce09ffb700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce141e0840 0x7fce141e2530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fce040011e0 tx=0x7fce04001950 comp rx=0 tx=0).stop 2024-09-13T17:53:01.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.755+0000 7fce09ffb700 1 -- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce141e0840 msgr2=0x7fce141e2530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:53:01.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.755+0000 7fce09ffb700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce141e0840 0x7fce141e2530 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fce040011e0 tx=0x7fce04001950 comp rx=0 tx=0).stop 2024-09-13T17:53:01.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.767+0000 7fce1a566700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce140c1c00 0x7fce14141570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.767+0000 7fce1a566700 1 -- 172.21.15.40:0/3515553551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce140c5e30 con 0x7fce140c1c00 2024-09-13T17:53:01.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.767+0000 7fce19564700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce140c1c00 0x7fce14141570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.767+0000 7fce19564700 1 -- 172.21.15.40:0/3515553551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fce140c63f0 con 0x7fce140c1c00 2024-09-13T17:53:01.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.767+0000 7fce19564700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce140c1c00 0x7fce14141570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fce04001eb0 tx=0x7fce04001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:01.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.771+0000 7fce09ffb700 1 -- 172.21.15.40:0/3515553551 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce04022070 con 0x7fce140c1c00 2024-09-13T17:53:01.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.771+0000 7fce09ffb700 1 -- 172.21.15.40:0/3515553551 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4617 (secure 0 0 0) 0x7fce0401d240 con 0x7fce140c1c00 2024-09-13T17:53:01.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.771+0000 7fce09ffb700 1 -- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce140c1c00 msgr2=0x7fce14141570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.771+0000 7fce09ffb700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce140c1c00 0x7fce14141570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fce04001eb0 tx=0x7fce04001150 comp rx=0 tx=0).stop 2024-09-13T17:53:01.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.771+0000 7fce09ffb700 1 -- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce140c1c00 msgr2=0x7fce14141570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:53:01.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.771+0000 7fce09ffb700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce140c1c00 0x7fce14141570 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.776 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:53:01.776 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:53:01.776 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:53:01.777 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:53:01.777 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 24, 2024-09-13T17:53:01.777 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:53:01.777 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:53:01.777 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:53:01.777 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:53:01.777 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:01.777 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 3, 2024-09-13T17:53:01.777 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:53:01.777 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:53:01.777 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:53:01.777 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:01.777 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:01.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:01.778 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:01.778 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:01.778 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:01.778 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:01.778 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:01.778 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:01.778 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:01.778 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:01.778 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:53:01.778 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:53:01.778 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:01.778 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:01.778 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:01.779 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:01.779 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:01.779 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:01.779 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:01.779 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:01.779 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:01.779 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:01.780 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:01.780 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.780 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:53:01.780 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:53:01.780 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:53:01.781 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:53:01.781 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:53:01.781 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:53:01.781 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:53:01.781 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:53:01.782 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:53:01.782 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:53:01.782 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:53:01.782 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:53:01.783 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:53:01.783 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:53:01.783 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:53:01.783 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:53:01.783 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:53:01.784 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:01.784 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:01.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:01.784 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:01.785 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:01.785 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:01.785 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:01.785 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:01.785 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:01.786 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:01.786 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:01.786 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:53:01.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:53:01.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.787 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:53:01.787 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:53:01.787 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:01.788 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:01.788 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.788 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:01.788 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:53:01.789 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:01.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.789 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.789 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:01.790 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:53:01.790 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:01.790 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:01.790 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:01.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.791 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:53:01.791 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:53:01.791 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:01.791 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:01.791 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:01.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.791 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.791 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:53:01.791 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:53:01.791 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:01.792 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:01.792 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.792 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:01.792 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:53:01.792 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:01.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.792 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.792 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:01.792 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:53:01.792 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:01.792 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:01.792 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:01.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.793 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:53:01.793 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:53:01.793 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:01.793 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:01.793 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:01.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.793 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.793 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:53:01.793 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:53:01.793 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:01.793 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:01.793 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.793 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:01.794 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:53:01.794 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:01.794 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.794 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.794 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:01.794 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:53:01.794 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:01.794 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:01.794 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:01.794 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.794 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:53:01.795 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:53:01.795 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:01.795 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:01.795 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:01.795 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:01.795 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:01.795 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.795 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:53:01.795 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:53:01.795 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.795 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:01.795 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:01.796 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:53:01.796 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:01.796 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:01.796 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:53:01.796 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.796 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:01.796 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:01.796 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:53:01.796 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:01.796 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:01.797 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:53:01.797 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.797 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:01.797 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:01.797 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:53:01.797 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:01.797 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:01.797 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.797 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:53:01.797 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:53:01.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.771+0000 7fce1a566700 1 -- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcdf8066730 msgr2=0x7fcdf8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.771+0000 7fce1a566700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcdf8066730 0x7fcdf8068bf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fce0c009850 tx=0x7fce0c01c040 comp rx=0 tx=0).stop 2024-09-13T17:53:01.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.771+0000 7fce1a566700 1 -- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce140c7f60 msgr2=0x7fce141f2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.771+0000 7fce1a566700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce140c7f60 0x7fce141f2340 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fce10001ed0 tx=0x7fce100021e0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.771+0000 7fce19d65700 1 -- 172.21.15.40:0/3515553551 reap_dead start 2024-09-13T17:53:01.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.771+0000 7fce1a566700 1 -- 172.21.15.40:0/3515553551 shutdown_connections 2024-09-13T17:53:01.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.771+0000 7fce1a566700 1 --2- 172.21.15.40:0/3515553551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce140c7f60 0x7fce141f2340 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.771+0000 7fce1a566700 1 -- 172.21.15.40:0/3515553551 >> 172.21.15.40:0/3515553551 conn(0x7fce14066090 msgr2=0x7fce14150f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:01.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.771+0000 7fce1a566700 1 -- 172.21.15.40:0/3515553551 shutdown_connections 2024-09-13T17:53:01.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.771+0000 7fce1a566700 1 -- 172.21.15.40:0/3515553551 wait complete. 2024-09-13T17:53:01.800 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:53:01.800 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T17:53:01.800 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:53:01.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.919+0000 7f9e1e0d9700 1 Processor -- start 2024-09-13T17:53:01.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.919+0000 7f9e1e0d9700 1 -- start start 2024-09-13T17:53:01.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.919+0000 7f9e1e0d9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9e18150520 0x7f9e1806e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.919+0000 7f9e1e0d9700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9e18071180 0x7f9e18070770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.919+0000 7f9e1e0d9700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e18066460 con 0x7f9e18150900 2024-09-13T17:53:01.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.919+0000 7f9e1e0d9700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e180c3830 con 0x7f9e18150520 2024-09-13T17:53:01.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.919+0000 7f9e1e0d9700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e18066130 con 0x7f9e18071180 2024-09-13T17:53:01.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.919+0000 7f9e1d0d7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9e18150520 0x7f9e1806e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.919+0000 7f9e1d0d7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9e18150520 0x7f9e1806e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49394/0 (socket says 172.21.15.40:49394) 2024-09-13T17:53:01.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.919+0000 7f9e1d0d7700 1 -- 172.21.15.40:0/3569634504 learned_addr learned my addr 172.21.15.40:0/3569634504 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:01.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.919+0000 7f9e1d8d8700 1 --2- 172.21.15.40:0/3569634504 >> v2:172.21.15.43:3300/0 conn(0x7f9e18071180 0x7f9e18070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.923+0000 7f9e1d0d7700 1 -- 172.21.15.40:0/3569634504 >> v2:172.21.15.43:3300/0 conn(0x7f9e18071180 msgr2=0x7f9e18070770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:53:01.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.923+0000 7f9e1d0d7700 1 --2- 172.21.15.40:0/3569634504 >> v2:172.21.15.43:3300/0 conn(0x7f9e18071180 0x7f9e18070770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.923+0000 7f9e1d0d7700 1 -- 172.21.15.40:0/3569634504 >> v1:172.21.15.43:6789/0 conn(0x7f9e18150900 legacy=0x7f9e180c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.923+0000 7f9e1d0d7700 1 -- 172.21.15.40:0/3569634504 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e18070e80 con 0x7f9e18150520 2024-09-13T17:53:01.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.923+0000 7f9e1d0d7700 1 --2- 172.21.15.40:0/3569634504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e18150520 0x7f9e1806e860 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9e1806b450 tx=0x7f9e14009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a03200afe7bdc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:01.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.923+0000 7f9e0ffff700 1 -- 172.21.15.40:0/3569634504 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14011830 con 0x7f9e18150520 2024-09-13T17:53:01.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.923+0000 7f9e0ffff700 1 -- 172.21.15.40:0/3569634504 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e14011990 con 0x7f9e18150520 2024-09-13T17:53:01.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.923+0000 7f9e0ffff700 1 -- 172.21.15.40:0/3569634504 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14011d20 con 0x7f9e18150520 2024-09-13T17:53:01.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.923+0000 7f9e1e0d9700 1 -- 172.21.15.40:0/3569634504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e18150520 msgr2=0x7f9e1806e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.923+0000 7f9e1e0d9700 1 --2- 172.21.15.40:0/3569634504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e18150520 0x7f9e1806e860 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9e1806b450 tx=0x7f9e14009bf0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.923+0000 7f9e1e0d9700 1 -- 172.21.15.40:0/3569634504 shutdown_connections 2024-09-13T17:53:01.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.923+0000 7f9e1e0d9700 1 --2- 172.21.15.40:0/3569634504 >> v2:172.21.15.43:3300/0 conn(0x7f9e18071180 0x7f9e18070770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.923+0000 7f9e1e0d9700 1 --2- 172.21.15.40:0/3569634504 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e18150520 0x7f9e1806e860 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.923+0000 7f9e1e0d9700 1 -- 172.21.15.40:0/3569634504 >> 172.21.15.40:0/3569634504 conn(0x7f9e180cb450 msgr2=0x7f9e180cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:01.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.923+0000 7f9e1e0d9700 1 -- 172.21.15.40:0/3569634504 shutdown_connections 2024-09-13T17:53:01.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.923+0000 7f9e1e0d9700 1 -- 172.21.15.40:0/3569634504 wait complete. 2024-09-13T17:53:01.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.927+0000 7f9e1e0d9700 1 Processor -- start 2024-09-13T17:53:01.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.927+0000 7f9e1e0d9700 1 -- start start 2024-09-13T17:53:01.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.927+0000 7f9e1e0d9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e18071180 0x7f9e181eee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.927+0000 7f9e1e0d9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9e18150520 0x7f9e1806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.927+0000 7f9e1e0d9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e1814bc90 0x7f9e181f2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.927+0000 7f9e1e0d9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e181ef350 con 0x7f9e18071180 2024-09-13T17:53:01.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.927+0000 7f9e1e0d9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e181e20f0 con 0x7f9e1814bc90 2024-09-13T17:53:01.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.927+0000 7f9e1e0d9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e18157300 con 0x7f9e18150520 2024-09-13T17:53:01.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.927+0000 7f9e1d0d7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e18071180 0x7f9e181eee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.927+0000 7f9e1d0d7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e18071180 0x7f9e181eee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49404/0 (socket says 172.21.15.40:49404) 2024-09-13T17:53:01.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.927+0000 7f9e1d0d7700 1 -- 172.21.15.40:0/2580779460 learned_addr learned my addr 172.21.15.40:0/2580779460 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:01.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.927+0000 7f9e1d8d8700 1 --2- 172.21.15.40:0/2580779460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e1814bc90 0x7f9e181f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.927+0000 7f9e1c8d6700 1 --2- 172.21.15.40:0/2580779460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9e18150520 0x7f9e1806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e1d0d7700 1 -- 172.21.15.40:0/2580779460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9e18150520 msgr2=0x7f9e1806d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e1d0d7700 1 --2- 172.21.15.40:0/2580779460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9e18150520 0x7f9e1806d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e1d0d7700 1 -- 172.21.15.40:0/2580779460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e1814bc90 msgr2=0x7f9e181f2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e1d0d7700 1 --2- 172.21.15.40:0/2580779460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e1814bc90 0x7f9e181f2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e1d0d7700 1 -- 172.21.15.40:0/2580779460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e14009570 con 0x7f9e18071180 2024-09-13T17:53:01.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e1d0d7700 1 --2- 172.21.15.40:0/2580779460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e18071180 0x7f9e181eee10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009c30 tx=0x7f9e14004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:01.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e0dffb700 1 -- 172.21.15.40:0/2580779460 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e140045d0 con 0x7f9e18071180 2024-09-13T17:53:01.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e0dffb700 1 -- 172.21.15.40:0/2580779460 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1401f070 con 0x7f9e18071180 2024-09-13T17:53:01.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e0dffb700 1 -- 172.21.15.40:0/2580779460 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14003900 con 0x7f9e18071180 2024-09-13T17:53:01.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e1e0d9700 1 -- 172.21.15.40:0/2580779460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e1814f4d0 con 0x7f9e18071180 2024-09-13T17:53:01.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e1e0d9700 1 -- 172.21.15.40:0/2580779460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e181e2300 con 0x7f9e18071180 2024-09-13T17:53:01.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e1e0d9700 1 --2- 172.21.15.40:0/2580779460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e1814efc0 0x7f9e1814f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e1e0d9700 1 -- 172.21.15.40:0/2580779460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e180675d0 con 0x7f9e1814efc0 2024-09-13T17:53:01.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e1c8d6700 1 --2- 172.21.15.40:0/2580779460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e1814efc0 0x7f9e1814f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e0dffb700 1 -- 172.21.15.40:0/2580779460 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9e14003a60 con 0x7f9e18071180 2024-09-13T17:53:01.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e0dffb700 1 --2- 172.21.15.40:0/2580779460 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9dfc066760 0x7f9dfc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e1c8d6700 1 -- 172.21.15.40:0/2580779460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e180c2ae0 con 0x7f9e1814efc0 2024-09-13T17:53:01.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e0dffb700 1 -- 172.21.15.40:0/2580779460 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9e14021070 con 0x7f9e18071180 2024-09-13T17:53:01.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e1d8d8700 1 --2- 172.21.15.40:0/2580779460 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9dfc066760 0x7f9dfc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e1c8d6700 1 --2- 172.21.15.40:0/2580779460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e1814efc0 0x7f9e1814f940 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9e1814c150 tx=0x7f9e0800e790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:01.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.931+0000 7f9e0dffb700 1 -- 172.21.15.40:0/2580779460 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0800ee40 con 0x7f9e1814efc0 2024-09-13T17:53:01.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.935+0000 7f9e1d8d8700 1 --2- 172.21.15.40:0/2580779460 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9dfc066760 0x7f9dfc068c20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9e1814c090 tx=0x7f9e1000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:01.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.935+0000 7f9e0dffb700 1 -- 172.21.15.40:0/2580779460 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9e080052e0 con 0x7f9e1814efc0 2024-09-13T17:53:01.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.935+0000 7f9e0dffb700 1 -- 172.21.15.40:0/2580779460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e1814efc0 msgr2=0x7f9e1814f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.935+0000 7f9e0dffb700 1 --2- 172.21.15.40:0/2580779460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e1814efc0 0x7f9e1814f940 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9e1814c150 tx=0x7f9e0800e790 comp rx=0 tx=0).stop 2024-09-13T17:53:01.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.935+0000 7f9e0dffb700 1 -- 172.21.15.40:0/2580779460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e1814efc0 msgr2=0x7f9e1814f940 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:53:01.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.935+0000 7f9e0dffb700 1 --2- 172.21.15.40:0/2580779460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e1814efc0 0x7f9e1814f940 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.939+0000 7f9e1e0d9700 1 --2- 172.21.15.40:0/2580779460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e18150900 0x7f9e181f28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:01.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.939+0000 7f9e1e0d9700 1 -- 172.21.15.40:0/2580779460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e18070490 con 0x7f9e18150900 2024-09-13T17:53:01.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.939+0000 7f9e1c8d6700 1 --2- 172.21.15.40:0/2580779460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e18150900 0x7f9e181f28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:01.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.943+0000 7f9e1c8d6700 1 -- 172.21.15.40:0/2580779460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9e180c56f0 con 0x7f9e18150900 2024-09-13T17:53:01.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.943+0000 7f9e1c8d6700 1 --2- 172.21.15.40:0/2580779460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e18150900 0x7f9e181f28c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9e1814c150 tx=0x7f9e08000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:01.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.943+0000 7f9e0dffb700 1 -- 172.21.15.40:0/2580779460 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e08021070 con 0x7f9e18150900 2024-09-13T17:53:01.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.943+0000 7f9e0dffb700 1 -- 172.21.15.40:0/2580779460 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4947 (secure 0 0 0) 0x7f9e0801ce60 con 0x7f9e18150900 2024-09-13T17:53:01.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.943+0000 7f9e0dffb700 1 -- 172.21.15.40:0/2580779460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e18150900 msgr2=0x7f9e181f28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.943+0000 7f9e0dffb700 1 --2- 172.21.15.40:0/2580779460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e18150900 0x7f9e181f28c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9e1814c150 tx=0x7f9e08000ed0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.943+0000 7f9e0dffb700 1 -- 172.21.15.40:0/2580779460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e18150900 msgr2=0x7f9e181f28c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:53:01.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.943+0000 7f9e0dffb700 1 --2- 172.21.15.40:0/2580779460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e18150900 0x7f9e181f28c0 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9e1814c150 tx=0x7f9e08000ed0 comp rx=0 tx=0).stop 2024-09-13T17:53:01.949 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:53:01.949 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:53:01.949 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:53:01.949 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:53:01.949 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 24, 2024-09-13T17:53:01.949 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:53:01.950 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:53:01.950 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:53:01.950 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:53:01.950 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:01.950 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 3, 2024-09-13T17:53:01.950 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:53:01.950 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:53:01.950 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:53:01.950 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:01.950 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:01.950 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:01.951 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:01.951 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:01.951 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:01.951 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:01.951 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:01.951 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:01.951 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:01.951 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:01.951 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:53:01.951 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:53:01.951 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:01.951 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:01.952 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:01.952 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:01.952 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:01.952 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:01.952 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:01.952 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:01.952 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:01.952 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:01.952 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:01.952 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.952 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:53:01.952 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:53:01.953 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:53:01.953 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:53:01.953 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:53:01.953 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:53:01.953 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:53:01.953 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:53:01.953 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:53:01.953 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:53:01.953 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:53:01.953 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:53:01.953 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:53:01.953 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:53:01.954 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:53:01.954 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:53:01.954 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:53:01.954 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:01.954 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:01.954 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:01.954 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:01.954 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:01.954 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:01.954 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:01.954 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:53:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:53:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:53:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:53:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:01.955 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:53:01.956 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:01.956 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.956 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.956 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:01.956 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:53:01.956 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:01.956 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:01.956 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:01.956 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.956 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:53:01.956 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:53:01.956 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:01.956 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:01.956 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:01.957 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.957 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.957 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:53:01.957 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:53:01.957 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:01.957 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:01.957 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.957 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:01.957 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:53:01.957 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:01.957 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.957 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.957 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:01.957 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:53:01.958 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:01.958 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:01.958 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:01.958 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.958 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:53:01.958 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:53:01.958 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:01.958 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:01.958 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:01.958 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.958 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.958 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:53:01.959 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:53:01.959 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:01.959 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:01.959 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.959 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:01.959 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:53:01.959 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:01.959 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.959 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.959 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:01.959 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:53:01.959 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:01.960 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:01.960 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:01.960 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.960 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:53:01.960 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:53:01.960 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:01.960 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:01.960 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:01.960 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:01.960 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:01.960 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.961 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:53:01.961 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:53:01.961 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.961 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:01.961 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:01.961 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:53:01.961 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:01.961 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:01.961 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:53:01.961 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.961 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:01.962 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:01.962 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:53:01.962 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:01.962 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:01.962 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:53:01.962 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.962 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:01.962 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:01.962 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:53:01.962 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:01.963 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:01.963 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:53:01.963 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.963 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:01.963 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:01.963 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:53:01.963 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:01.963 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:01.963 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:53:01.963 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:01.963 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:01.963 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:01.964 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:53:01.964 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:01.964 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:01.964 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:01.964 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:53:01.964 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:53:01.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.943+0000 7f9e1e0d9700 1 -- 172.21.15.40:0/2580779460 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9dfc066760 msgr2=0x7f9dfc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.943+0000 7f9e1e0d9700 1 --2- 172.21.15.40:0/2580779460 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9dfc066760 0x7f9dfc068c20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9e1814c090 tx=0x7f9e1000b040 comp rx=0 tx=0).stop 2024-09-13T17:53:01.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.943+0000 7f9e1e0d9700 1 -- 172.21.15.40:0/2580779460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e18071180 msgr2=0x7f9e181eee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:01.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.943+0000 7f9e1e0d9700 1 --2- 172.21.15.40:0/2580779460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9e18071180 0x7f9e181eee10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009c30 tx=0x7f9e14004770 comp rx=0 tx=0).stop 2024-09-13T17:53:01.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.943+0000 7f9e1d8d8700 1 -- 172.21.15.40:0/2580779460 reap_dead start 2024-09-13T17:53:01.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.943+0000 7f9e1d8d8700 1 -- 172.21.15.40:0/2580779460 reap_dead start 2024-09-13T17:53:01.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.943+0000 7f9e1e0d9700 1 -- 172.21.15.40:0/2580779460 shutdown_connections 2024-09-13T17:53:01.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.943+0000 7f9e1e0d9700 1 -- 172.21.15.40:0/2580779460 >> 172.21.15.40:0/2580779460 conn(0x7f9e180cb450 msgr2=0x7f9e180c5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:01.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.943+0000 7f9e1e0d9700 1 -- 172.21.15.40:0/2580779460 shutdown_connections 2024-09-13T17:53:01.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:01.943+0000 7f9e1e0d9700 1 -- 172.21.15.40:0/2580779460 wait complete. 2024-09-13T17:53:01.967 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:53:01.967 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T17:53:01.967 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:53:02.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.083+0000 7fd16007b700 1 Processor -- start 2024-09-13T17:53:02.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.083+0000 7fd16007b700 1 -- start start 2024-09-13T17:53:02.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd16007b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd158157d10 0x7fd1580b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:02.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd16007b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd158161c60 0x7fd1580c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:02.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd16007b700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1580b1000 con 0x7fd15806afb0 2024-09-13T17:53:02.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd16007b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1580729e0 con 0x7fd158157d10 2024-09-13T17:53:02.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd16007b700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1580c6c20 con 0x7fd158161c60 2024-09-13T17:53:02.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd15e618700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd15806afb0 0x7fd15806a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50094/0 (socket says 172.21.15.40:50094) 2024-09-13T17:53:02.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd15e618700 1 -- 172.21.15.40:0/3237123183 learned_addr learned my addr 172.21.15.40:0/3237123183 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:53:02.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd15de17700 1 --2- 172.21.15.40:0/3237123183 >> v2:172.21.15.40:3300/0 conn(0x7fd158157d10 0x7fd1580b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:02.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd15d616700 1 --2- 172.21.15.40:0/3237123183 >> v2:172.21.15.43:3300/0 conn(0x7fd158161c60 0x7fd1580c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:02.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd15ce15700 1 -- 172.21.15.40:0/3237123183 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329716537 0 0) 0x7fd1580b1000 con 0x7fd15806afb0 2024-09-13T17:53:02.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd15ce15700 1 -- 172.21.15.40:0/3237123183 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1440035d0 con 0x7fd15806afb0 2024-09-13T17:53:02.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd15de17700 1 -- 172.21.15.40:0/3237123183 >> v2:172.21.15.43:3300/0 conn(0x7fd158161c60 msgr2=0x7fd1580c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:53:02.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd15de17700 1 --2- 172.21.15.40:0/3237123183 >> v2:172.21.15.43:3300/0 conn(0x7fd158161c60 0x7fd1580c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:02.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd15de17700 1 -- 172.21.15.40:0/3237123183 >> v1:172.21.15.40:6789/0 conn(0x7fd15806afb0 legacy=0x7fd15806a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:02.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd15de17700 1 -- 172.21.15.40:0/3237123183 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1581617d0 con 0x7fd158157d10 2024-09-13T17:53:02.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd15d616700 1 --2- 172.21.15.40:0/3237123183 >> v2:172.21.15.43:3300/0 conn(0x7fd158161c60 0x7fd1580c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:53:02.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd15de17700 1 --2- 172.21.15.40:0/3237123183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd158157d10 0x7fd1580b0ac0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd15815b590 tx=0x7fd154009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657f007f53f8d210 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:02.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd15ce15700 1 -- 172.21.15.40:0/3237123183 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd154011830 con 0x7fd158157d10 2024-09-13T17:53:02.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd15ce15700 1 -- 172.21.15.40:0/3237123183 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd154011990 con 0x7fd158157d10 2024-09-13T17:53:02.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd15ce15700 1 -- 172.21.15.40:0/3237123183 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd154011d20 con 0x7fd158157d10 2024-09-13T17:53:02.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd16007b700 1 -- 172.21.15.40:0/3237123183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd158157d10 msgr2=0x7fd1580b0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:02.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd16007b700 1 --2- 172.21.15.40:0/3237123183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd158157d10 0x7fd1580b0ac0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd15815b590 tx=0x7fd154009b30 comp rx=0 tx=0).stop 2024-09-13T17:53:02.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd16007b700 1 -- 172.21.15.40:0/3237123183 shutdown_connections 2024-09-13T17:53:02.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd16007b700 1 --2- 172.21.15.40:0/3237123183 >> v2:172.21.15.43:3300/0 conn(0x7fd158161c60 0x7fd1580c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:02.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd16007b700 1 --2- 172.21.15.40:0/3237123183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd158157d10 0x7fd1580b0ac0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:02.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd16007b700 1 -- 172.21.15.40:0/3237123183 >> 172.21.15.40:0/3237123183 conn(0x7fd15806d5b0 msgr2=0x7fd15806d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:02.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd16007b700 1 -- 172.21.15.40:0/3237123183 shutdown_connections 2024-09-13T17:53:02.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.087+0000 7fd16007b700 1 -- 172.21.15.40:0/3237123183 wait complete. 2024-09-13T17:53:02.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.091+0000 7fd16007b700 1 Processor -- start 2024-09-13T17:53:02.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.091+0000 7fd16007b700 1 -- start start 2024-09-13T17:53:02.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.091+0000 7fd16007b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd15806afb0 0x7fd1581f6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:02.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.091+0000 7fd16007b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd158157d10 0x7fd1580affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:02.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.091+0000 7fd16007b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd158161c60 0x7fd15815cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:02.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.091+0000 7fd16007b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1581f7160 con 0x7fd158161c60 2024-09-13T17:53:02.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.091+0000 7fd16007b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15815cb90 con 0x7fd15806afb0 2024-09-13T17:53:02.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.091+0000 7fd16007b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1581f66c0 con 0x7fd158157d10 2024-09-13T17:53:02.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.091+0000 7fd15de17700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd15806afb0 0x7fd1581f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:02.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.091+0000 7fd15d616700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd158157d10 0x7fd1580affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:02.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.091+0000 7fd15de17700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd15806afb0 0x7fd1581f6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:51286/0 (socket says 172.21.15.40:51286) 2024-09-13T17:53:02.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.091+0000 7fd15de17700 1 -- 172.21.15.40:0/829979363 learned_addr learned my addr 172.21.15.40:0/829979363 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:02.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.091+0000 7fd15e618700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd158161c60 0x7fd15815cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:02.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.091+0000 7fd15d616700 1 -- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd158157d10 msgr2=0x7fd1580affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T17:53:02.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.091+0000 7fd15d616700 1 -- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd158157d10 msgr2=0x7fd1580affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:53:02.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.091+0000 7fd15d616700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd158157d10 0x7fd1580affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:02.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.091+0000 7fd15d616700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd158157d10 0x7fd1580affa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:02.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.095+0000 7fd15de17700 1 -- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd158157d10 msgr2=0x7fd1580affa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:53:02.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.095+0000 7fd15de17700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd158157d10 0x7fd1580affa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:02.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.095+0000 7fd15de17700 1 -- 172.21.15.40:0/829979363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd158161c60 msgr2=0x7fd15815cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:02.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.095+0000 7fd15de17700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd158161c60 0x7fd15815cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:02.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.095+0000 7fd15de17700 1 -- 172.21.15.40:0/829979363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1480094b0 con 0x7fd15806afb0 2024-09-13T17:53:02.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.095+0000 7fd15de17700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd15806afb0 0x7fd1581f6180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd154009b70 tx=0x7fd154004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:02.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.095+0000 7fd14e7fc700 1 -- 172.21.15.40:0/829979363 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd154004400 con 0x7fd15806afb0 2024-09-13T17:53:02.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.095+0000 7fd14e7fc700 1 -- 172.21.15.40:0/829979363 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15401f070 con 0x7fd15806afb0 2024-09-13T17:53:02.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.095+0000 7fd16007b700 1 -- 172.21.15.40:0/829979363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1540094b0 con 0x7fd15806afb0 2024-09-13T17:53:02.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.095+0000 7fd14e7fc700 1 -- 172.21.15.40:0/829979363 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1540036c0 con 0x7fd15806afb0 2024-09-13T17:53:02.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.095+0000 7fd16007b700 1 -- 172.21.15.40:0/829979363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1581f6ba0 con 0x7fd15806afb0 2024-09-13T17:53:02.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.095+0000 7fd16007b700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 0x7fd15815d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:02.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.095+0000 7fd16007b700 1 -- 172.21.15.40:0/829979363 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd158067840 con 0x7fd1580c2590 2024-09-13T17:53:02.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.095+0000 7fd15d616700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 0x7fd15815d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:02.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.095+0000 7fd15d616700 1 -- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 msgr2=0x7fd15815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-09-13T17:53:02.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.095+0000 7fd15d616700 1 -- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 msgr2=0x7fd15815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:53:02.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.095+0000 7fd15d616700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 0x7fd15815d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:02.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.095+0000 7fd15d616700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 0x7fd15815d4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:02.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.099+0000 7fd14e7fc700 1 -- 172.21.15.40:0/829979363 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd154021030 con 0x7fd15806afb0 2024-09-13T17:53:02.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.099+0000 7fd14e7fc700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd13c066730 0x7fd13c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:02.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.099+0000 7fd14e7fc700 1 -- 172.21.15.40:0/829979363 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd154085b90 con 0x7fd15806afb0 2024-09-13T17:53:02.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.099+0000 7fd15e618700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd13c066730 0x7fd13c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:02.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.099+0000 7fd15e618700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd13c066730 0x7fd13c068bf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd150001560 tx=0x7fd150001a10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:02.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.299+0000 7fd15d616700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 0x7fd15815d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:02.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.299+0000 7fd15d616700 1 -- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 msgr2=0x7fd15815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-13T17:53:02.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.299+0000 7fd15d616700 1 -- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 msgr2=0x7fd15815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:53:02.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.299+0000 7fd15d616700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 0x7fd15815d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:02.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.299+0000 7fd15d616700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 0x7fd15815d4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:53:02.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.703+0000 7fd15d616700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 0x7fd15815d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:02.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.703+0000 7fd15d616700 1 -- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 msgr2=0x7fd15815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-13T17:53:02.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.703+0000 7fd15d616700 1 -- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 msgr2=0x7fd15815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:53:02.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.703+0000 7fd15d616700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 0x7fd15815d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:02.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:02.703+0000 7fd15d616700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 0x7fd15815d4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:53:02.851 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether build environment is sane... yes 2024-09-13T17:53:02.908 INFO:tasks.workunit.client.0.smithi040.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-13T17:53:02.941 INFO:tasks.workunit.client.0.smithi040.stdout:checking for gawk... gawk 2024-09-13T17:53:03.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.507+0000 7fd15d616700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 0x7fd15815d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:03.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.507+0000 7fd15d616700 1 -- 172.21.15.40:0/829979363 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd158160ee0 con 0x7fd1580c2590 2024-09-13T17:53:03.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.507+0000 7fd15d616700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 0x7fd15815d4f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd1580ca600 tx=0x7fd148017ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:03.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.507+0000 7fd14e7fc700 1 -- 172.21.15.40:0/829979363 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1480097e0 con 0x7fd1580c2590 2024-09-13T17:53:03.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.507+0000 7fd14e7fc700 1 -- 172.21.15.40:0/829979363 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd14801c070 con 0x7fd1580c2590 2024-09-13T17:53:03.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.507+0000 7fd14e7fc700 1 -- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 msgr2=0x7fd15815d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:03.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.507+0000 7fd14e7fc700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 0x7fd15815d4f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd1580ca600 tx=0x7fd148017ca0 comp rx=0 tx=0).stop 2024-09-13T17:53:03.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.507+0000 7fd14e7fc700 1 -- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 msgr2=0x7fd15815d4f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:53:03.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.507+0000 7fd14e7fc700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd1580c2590 0x7fd15815d4f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:03.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.523+0000 7fd16007b700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd15815c610 0x7fd158156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:03.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.523+0000 7fd16007b700 1 -- 172.21.15.40:0/829979363 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1581580f0 con 0x7fd15815c610 2024-09-13T17:53:03.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.523+0000 7fd15d616700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd15815c610 0x7fd158156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:03.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.523+0000 7fd15d616700 1 -- 172.21.15.40:0/829979363 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd1580c7150 con 0x7fd15815c610 2024-09-13T17:53:03.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.523+0000 7fd15d616700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd15815c610 0x7fd158156e50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd148022000 tx=0x7fd1480047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:03.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.527+0000 7fd14e7fc700 1 -- 172.21.15.40:0/829979363 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd148006750 con 0x7fd15815c610 2024-09-13T17:53:03.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.527+0000 7fd14e7fc700 1 -- 172.21.15.40:0/829979363 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4449 (secure 0 0 0) 0x7fd148005ed0 con 0x7fd15815c610 2024-09-13T17:53:03.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.527+0000 7fd14e7fc700 1 -- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd15815c610 msgr2=0x7fd158156e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:03.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.527+0000 7fd14e7fc700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd15815c610 0x7fd158156e50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd148022000 tx=0x7fd1480047e0 comp rx=0 tx=0).stop 2024-09-13T17:53:03.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.527+0000 7fd14e7fc700 1 -- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd15815c610 msgr2=0x7fd158156e50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:53:03.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.527+0000 7fd14e7fc700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd15815c610 0x7fd158156e50 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd148022000 tx=0x7fd1480047e0 comp rx=0 tx=0).stop 2024-09-13T17:53:03.532 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:53:03.532 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:53:03.532 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:53:03.532 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:53:03.532 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 24, 2024-09-13T17:53:03.532 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:53:03.532 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:53:03.532 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:53:03.533 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:53:03.533 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:03.533 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 4, 2024-09-13T17:53:03.533 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:53:03.533 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:53:03.533 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:53:03.533 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:03.533 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:03.533 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:03.533 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:03.533 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:03.533 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:03.534 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:03.534 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:03.534 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:03.534 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:03.534 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:03.534 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:53:03.534 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:53:03.534 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:03.534 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:03.534 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:03.534 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:03.534 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:03.534 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:03.534 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:03.535 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:03.535 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:03.535 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:03.535 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:03.535 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:03.535 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:53:03.535 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:53:03.535 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:53:03.535 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:53:03.535 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:53:03.535 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:53:03.535 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:53:03.535 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:53:03.535 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:53:03.535 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:53:03.536 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:53:03.536 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:53:03.536 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:53:03.536 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:53:03.536 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:53:03.536 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:53:03.536 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:53:03.536 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:03.536 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:03.536 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:03.536 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:03.536 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:03.536 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:03.537 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:03.537 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:03.537 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:03.537 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:03.537 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:03.537 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:53:03.537 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:03.537 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:53:03.537 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:03.537 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:53:03.537 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:53:03.537 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:03.537 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:03.538 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:03.538 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:03.538 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:53:03.538 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:03.538 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:03.538 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:03.538 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:03.538 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:53:03.538 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:03.539 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:03.539 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:03.539 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:03.539 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:53:03.539 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:53:03.539 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:03.539 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:03.539 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:03.539 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:03.539 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:03.539 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:53:03.539 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:53:03.539 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:03.540 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:03.540 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:03.540 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:03.540 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:53:03.540 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:03.540 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:03.540 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:03.540 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:03.540 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:53:03.540 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:03.541 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:03.541 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:03.541 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:03.541 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:53:03.541 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:53:03.541 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:03.541 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:03.541 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:03.541 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:03.541 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:03.542 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:53:03.542 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:53:03.542 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:03.542 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:03.542 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:03.542 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:03.542 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:53:03.542 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:03.542 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:03.543 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:03.543 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:03.543 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:53:03.543 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:03.543 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:03.543 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:03.543 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:03.543 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:53:03.543 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:53:03.543 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:03.543 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:03.543 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:03.543 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:03.543 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:03.544 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:03.544 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:53:03.544 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:53:03.544 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:03.544 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:03.544 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:03.544 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:53:03.544 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:03.544 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:03.544 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:53:03.544 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:03.544 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:03.545 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:03.545 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:53:03.545 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:03.545 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:03.545 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:03.545 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:53:03.545 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:53:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.527+0000 7fd16007b700 1 -- 172.21.15.40:0/829979363 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd13c066730 msgr2=0x7fd13c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.527+0000 7fd16007b700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd13c066730 0x7fd13c068bf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd150001560 tx=0x7fd150001a10 comp rx=0 tx=0).stop 2024-09-13T17:53:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.527+0000 7fd16007b700 1 -- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd15806afb0 msgr2=0x7fd1581f6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.527+0000 7fd16007b700 1 --2- 172.21.15.40:0/829979363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd15806afb0 0x7fd1581f6180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd154009b70 tx=0x7fd154004770 comp rx=0 tx=0).stop 2024-09-13T17:53:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.527+0000 7fd15e618700 1 -- 172.21.15.40:0/829979363 reap_dead start 2024-09-13T17:53:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.527+0000 7fd15e618700 1 -- 172.21.15.40:0/829979363 reap_dead start 2024-09-13T17:53:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.527+0000 7fd16007b700 1 -- 172.21.15.40:0/829979363 shutdown_connections 2024-09-13T17:53:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.527+0000 7fd16007b700 1 -- 172.21.15.40:0/829979363 >> 172.21.15.40:0/829979363 conn(0x7fd15806d5b0 msgr2=0x7fd15815a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.527+0000 7fd16007b700 1 -- 172.21.15.40:0/829979363 shutdown_connections 2024-09-13T17:53:03.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.527+0000 7fd16007b700 1 -- 172.21.15.40:0/829979363 wait complete. 2024-09-13T17:53:03.548 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:53:03.548 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T17:53:03.548 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T17:53:03.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a3cf7700 1 Processor -- start 2024-09-13T17:53:03.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a3cf7700 1 -- start start 2024-09-13T17:53:03.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a3cf7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f429c0c9e30 0x7f429c0c8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:03.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a3cf7700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f429c15dbe0 con 0x7f429c0c9300 2024-09-13T17:53:03.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a3cf7700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f429c15dea0 con 0x7f429c0ca210 2024-09-13T17:53:03.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a3cf7700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f429c06d210 con 0x7f429c0c9e30 2024-09-13T17:53:03.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a1a93700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f429c0c9e30 0x7f429c0c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:03.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a2294700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f429c0c9300 0x7f429c06faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50110/0 (socket says 172.21.15.40:50110) 2024-09-13T17:53:03.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a1a93700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f429c0c9e30 0x7f429c0c8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49444/0 (socket says 172.21.15.40:49444) 2024-09-13T17:53:03.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a2294700 1 -- 172.21.15.40:0/3635232553 learned_addr learned my addr 172.21.15.40:0/3635232553 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:53:03.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a1a93700 1 -- 172.21.15.40:0/3635232553 >> v1:172.21.15.43:6789/0 conn(0x7f429c0ca210 legacy=0x7f429c157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:03.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a1a93700 1 -- 172.21.15.40:0/3635232553 >> v1:172.21.15.40:6789/0 conn(0x7f429c0c9300 legacy=0x7f429c06faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:03.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a1a93700 1 -- 172.21.15.40:0/3635232553 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429c15e180 con 0x7f429c0c9e30 2024-09-13T17:53:03.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a1a93700 1 --2- 172.21.15.40:0/3635232553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429c0c9e30 0x7f429c0c8290 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4290009850 tx=0x7f4290009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4197297b53c37 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:03.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a0a91700 1 -- 172.21.15.40:0/3635232553 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4290011830 con 0x7f429c0c9e30 2024-09-13T17:53:03.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a0a91700 1 -- 172.21.15.40:0/3635232553 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4290011990 con 0x7f429c0c9e30 2024-09-13T17:53:03.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a0a91700 1 -- 172.21.15.40:0/3635232553 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4290011d20 con 0x7f429c0c9e30 2024-09-13T17:53:03.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a3cf7700 1 -- 172.21.15.40:0/3635232553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429c0c9e30 msgr2=0x7f429c0c8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:03.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a3cf7700 1 --2- 172.21.15.40:0/3635232553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429c0c9e30 0x7f429c0c8290 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4290009850 tx=0x7f4290009b60 comp rx=0 tx=0).stop 2024-09-13T17:53:03.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a3cf7700 1 -- 172.21.15.40:0/3635232553 shutdown_connections 2024-09-13T17:53:03.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a3cf7700 1 --2- 172.21.15.40:0/3635232553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429c0c9e30 0x7f429c0c8290 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:03.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a3cf7700 1 -- 172.21.15.40:0/3635232553 >> 172.21.15.40:0/3635232553 conn(0x7f429c066270 msgr2=0x7f429c066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:03.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a3cf7700 1 -- 172.21.15.40:0/3635232553 shutdown_connections 2024-09-13T17:53:03.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.663+0000 7f42a3cf7700 1 -- 172.21.15.40:0/3635232553 wait complete. 2024-09-13T17:53:03.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a3cf7700 1 Processor -- start 2024-09-13T17:53:03.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a3cf7700 1 -- start start 2024-09-13T17:53:03.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a3cf7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429c0c9300 0x7f429c1e69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:03.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a3cf7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f429c0c9e30 0x7f429c1f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:03.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a3cf7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429c0ca210 0x7f429c15a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:03.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a3cf7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c1f31f0 con 0x7f429c0ca210 2024-09-13T17:53:03.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a3cf7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429c157070 con 0x7f429c0c9e30 2024-09-13T17:53:03.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a3cf7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f429c0c4e60 con 0x7f429c0c9300 2024-09-13T17:53:03.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a2294700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429c0ca210 0x7f429c15a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:03.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a2294700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429c0ca210 0x7f429c15a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49452/0 (socket says 172.21.15.40:49452) 2024-09-13T17:53:03.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a2294700 1 -- 172.21.15.40:0/2998261562 learned_addr learned my addr 172.21.15.40:0/2998261562 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:03.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a1a93700 1 --2- 172.21.15.40:0/2998261562 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429c0c9300 0x7f429c1e69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:03.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a1292700 1 --2- 172.21.15.40:0/2998261562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f429c0c9e30 0x7f429c1f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:03.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a2294700 1 -- 172.21.15.40:0/2998261562 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429c0c9300 msgr2=0x7f429c1e69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:03.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a2294700 1 --2- 172.21.15.40:0/2998261562 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429c0c9300 0x7f429c1e69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:03.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a2294700 1 -- 172.21.15.40:0/2998261562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f429c0c9e30 msgr2=0x7f429c1f7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:03.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a2294700 1 --2- 172.21.15.40:0/2998261562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f429c0c9e30 0x7f429c1f7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:03.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a2294700 1 -- 172.21.15.40:0/2998261562 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42900094b0 con 0x7f429c0ca210 2024-09-13T17:53:03.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a1a93700 1 --2- 172.21.15.40:0/2998261562 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429c0c9300 0x7f429c1e69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:53:03.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a2294700 1 --2- 172.21.15.40:0/2998261562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429c0ca210 0x7f429c15a330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f429400c420 tx=0x7f429400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:03.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f429a7fc700 1 -- 172.21.15.40:0/2998261562 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294005d90 con 0x7f429c0ca210 2024-09-13T17:53:03.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f429a7fc700 1 -- 172.21.15.40:0/2998261562 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42940070d0 con 0x7f429c0ca210 2024-09-13T17:53:03.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f429a7fc700 1 -- 172.21.15.40:0/2998261562 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294007440 con 0x7f429c0ca210 2024-09-13T17:53:03.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.667+0000 7f42a3cf7700 1 -- 172.21.15.40:0/2998261562 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f429c0cd000 con 0x7f429c0ca210 2024-09-13T17:53:03.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.671+0000 7f42a3cf7700 1 -- 172.21.15.40:0/2998261562 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429c0cd880 con 0x7f429c0ca210 2024-09-13T17:53:03.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.671+0000 7f429a7fc700 1 -- 172.21.15.40:0/2998261562 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f42940075a0 con 0x7f429c0ca210 2024-09-13T17:53:03.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.671+0000 7f42a3cf7700 1 -- 172.21.15.40:0/2998261562 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429c06eb50 con 0x7f429c0ca210 2024-09-13T17:53:03.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.671+0000 7f429a7fc700 1 --2- 172.21.15.40:0/2998261562 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4288066760 0x7f4288068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:03.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.671+0000 7f429a7fc700 1 -- 172.21.15.40:0/2998261562 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4294083880 con 0x7f429c0ca210 2024-09-13T17:53:03.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.671+0000 7f42a1a93700 1 --2- 172.21.15.40:0/2998261562 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4288066760 0x7f4288068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:03.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.671+0000 7f42a1a93700 1 --2- 172.21.15.40:0/2998261562 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4288066760 0x7f4288068c20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4290014040 tx=0x7f4290008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:03.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.679+0000 7f429a7fc700 1 -- 172.21.15.40:0/2998261562 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4294050850 con 0x7f429c0ca210 2024-09-13T17:53:03.782 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether make sets $(MAKE)... yes 2024-09-13T17:53:03.926 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.919+0000 7f42a3cf7700 1 -- 172.21.15.40:0/2998261562 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f429c0b2fb0 con 0x7f429c0ca210 2024-09-13T17:53:03.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.923+0000 7f429a7fc700 1 -- 172.21.15.40:0/2998261562 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f4294017080 con 0x7f429c0ca210 2024-09-13T17:53:03.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.927+0000 7f42a3cf7700 1 -- 172.21.15.40:0/2998261562 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4288066760 msgr2=0x7f4288068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:03.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.927+0000 7f42a3cf7700 1 --2- 172.21.15.40:0/2998261562 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4288066760 0x7f4288068c20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4290014040 tx=0x7f4290008030 comp rx=0 tx=0).stop 2024-09-13T17:53:03.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.927+0000 7f42a3cf7700 1 -- 172.21.15.40:0/2998261562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429c0ca210 msgr2=0x7f429c15a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:03.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.927+0000 7f42a3cf7700 1 --2- 172.21.15.40:0/2998261562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429c0ca210 0x7f429c15a330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f429400c420 tx=0x7f429400c7e0 comp rx=0 tx=0).stop 2024-09-13T17:53:03.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.927+0000 7f42a3cf7700 1 -- 172.21.15.40:0/2998261562 shutdown_connections 2024-09-13T17:53:03.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.927+0000 7f42a3cf7700 1 --2- 172.21.15.40:0/2998261562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429c0ca210 0x7f429c15a330 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:03.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.927+0000 7f42a3cf7700 1 --2- 172.21.15.40:0/2998261562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f429c0c9e30 0x7f429c1f7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:03.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.927+0000 7f42a3cf7700 1 --2- 172.21.15.40:0/2998261562 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4288066760 0x7f4288068c20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:03.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.927+0000 7f42a3cf7700 1 --2- 172.21.15.40:0/2998261562 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429c0c9300 0x7f429c1e69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:03.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.927+0000 7f42a3cf7700 1 -- 172.21.15.40:0/2998261562 >> 172.21.15.40:0/2998261562 conn(0x7f429c066270 msgr2=0x7f429c0c2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:03.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.927+0000 7f42a3cf7700 1 -- 172.21.15.40:0/2998261562 shutdown_connections 2024-09-13T17:53:03.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:03.927+0000 7f42a3cf7700 1 -- 172.21.15.40:0/2998261562 wait complete. 2024-09-13T17:53:03.943 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T17:53:03.984 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:53:03.986 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:53:04.599 INFO:tasks.workunit.client.0.smithi040.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-13T17:53:05.199 INFO:tasks.workunit.client.0.smithi040.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-13T17:53:05.281 INFO:tasks.workunit.client.0.smithi040.stdout:checking for style of include used by make... GNU 2024-09-13T17:53:06.617 INFO:tasks.workunit.client.0.smithi040.stdout:checking for gcc... gcc 2024-09-13T17:53:07.062 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether the C compiler works... yes 2024-09-13T17:53:07.090 INFO:tasks.workunit.client.0.smithi040.stdout:checking for C compiler default output file name... a.out 2024-09-13T17:53:07.287 INFO:tasks.workunit.client.0.smithi040.stdout:checking for suffix of executables... 2024-09-13T17:53:07.499 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether we are cross compiling... no 2024-09-13T17:53:08.532 INFO:tasks.workunit.client.0.smithi040.stdout:checking for suffix of object files... o 2024-09-13T17:53:09.261 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether we are using the GNU C compiler... yes 2024-09-13T17:53:09.422 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether gcc accepts -g... yes 2024-09-13T17:53:09.597 INFO:tasks.workunit.client.0.smithi040.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-13T17:53:09.927 INFO:tasks.workunit.client.0.smithi040.stdout:checking dependency style of gcc... gcc3 2024-09-13T17:53:10.174 INFO:tasks.workunit.client.0.smithi040.stdout:checking how to run the C preprocessor... gcc -E 2024-09-13T17:53:10.860 INFO:tasks.workunit.client.0.smithi040.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-13T17:53:10.886 INFO:tasks.workunit.client.0.smithi040.stdout:checking for egrep... /usr/bin/grep -E 2024-09-13T17:53:11.379 INFO:tasks.workunit.client.0.smithi040.stdout:checking for ANSI C header files... yes 2024-09-13T17:53:11.590 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sys/types.h... yes 2024-09-13T17:53:12.677 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sys/stat.h... yes 2024-09-13T17:53:13.945 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T17:53:13.945 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:53:13.991 INFO:tasks.workunit.client.0.smithi040.stdout:checking for stdlib.h... yes 2024-09-13T17:53:14.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.063+0000 7f66ca450700 1 Processor -- start 2024-09-13T17:53:14.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.063+0000 7f66ca450700 1 -- start start 2024-09-13T17:53:14.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.063+0000 7f66ca450700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f66c40c3520 0x7f66c41580f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.063+0000 7f66ca450700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66c4066420 0x7f66c415aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.063+0000 7f66ca450700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c40c3ef0 con 0x7f66c40c3900 2024-09-13T17:53:14.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.063+0000 7f66ca450700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c4157cd0 con 0x7f66c40c3520 2024-09-13T17:53:14.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.063+0000 7f66ca450700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c40c59d0 con 0x7f66c4066420 2024-09-13T17:53:14.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.063+0000 7f66c37fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66c4066420 0x7f66c415aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.063+0000 7f66c37fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66c4066420 0x7f66c415aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:55384/0 (socket says 172.21.15.40:55384) 2024-09-13T17:53:14.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.063+0000 7f66c37fe700 1 -- 172.21.15.40:0/4233961520 learned_addr learned my addr 172.21.15.40:0/4233961520 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:14.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.063+0000 7f66c3fff700 1 --2- 172.21.15.40:0/4233961520 >> v2:172.21.15.43:3300/0 conn(0x7f66c40c3520 0x7f66c41580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.067+0000 7f66c37fe700 1 -- 172.21.15.40:0/4233961520 >> v2:172.21.15.43:3300/0 conn(0x7f66c40c3520 msgr2=0x7f66c41580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:53:14.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.067+0000 7f66c37fe700 1 --2- 172.21.15.40:0/4233961520 >> v2:172.21.15.43:3300/0 conn(0x7f66c40c3520 0x7f66c41580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.067+0000 7f66c37fe700 1 -- 172.21.15.40:0/4233961520 >> v1:172.21.15.43:6789/0 conn(0x7f66c40c3900 legacy=0x7f66c4151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.067+0000 7f66c37fe700 1 -- 172.21.15.40:0/4233961520 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c40c5410 con 0x7f66c4066420 2024-09-13T17:53:14.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.067+0000 7f66c37fe700 1 --2- 172.21.15.40:0/4233961520 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66c4066420 0x7f66c415aaf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009910 tx=0x7f66b8009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0ddd25a82c6d68 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:14.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.067+0000 7f66c2ffd700 1 -- 172.21.15.40:0/4233961520 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8011830 con 0x7f66c4066420 2024-09-13T17:53:14.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.067+0000 7f66c2ffd700 1 -- 172.21.15.40:0/4233961520 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b8011990 con 0x7f66c4066420 2024-09-13T17:53:14.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.067+0000 7f66c2ffd700 1 -- 172.21.15.40:0/4233961520 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8011d40 con 0x7f66c4066420 2024-09-13T17:53:14.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.067+0000 7f66ca450700 1 -- 172.21.15.40:0/4233961520 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66c4066420 msgr2=0x7f66c415aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.067+0000 7f66ca450700 1 --2- 172.21.15.40:0/4233961520 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66c4066420 0x7f66c415aaf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009910 tx=0x7f66b8009c20 comp rx=0 tx=0).stop 2024-09-13T17:53:14.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.067+0000 7f66ca450700 1 -- 172.21.15.40:0/4233961520 shutdown_connections 2024-09-13T17:53:14.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.067+0000 7f66ca450700 1 --2- 172.21.15.40:0/4233961520 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66c4066420 0x7f66c415aaf0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.067+0000 7f66ca450700 1 --2- 172.21.15.40:0/4233961520 >> v2:172.21.15.43:3300/0 conn(0x7f66c40c3520 0x7f66c41580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.067+0000 7f66ca450700 1 -- 172.21.15.40:0/4233961520 >> 172.21.15.40:0/4233961520 conn(0x7f66c406a390 msgr2=0x7f66c406a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:14.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.067+0000 7f66ca450700 1 -- 172.21.15.40:0/4233961520 shutdown_connections 2024-09-13T17:53:14.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.067+0000 7f66ca450700 1 -- 172.21.15.40:0/4233961520 wait complete. 2024-09-13T17:53:14.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.067+0000 7f66ca450700 1 Processor -- start 2024-09-13T17:53:14.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.067+0000 7f66ca450700 1 -- start start 2024-09-13T17:53:14.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66ca450700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c4066420 0x7f66c41ed9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66ca450700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66c40c3520 0x7f66c41e2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66ca450700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66c40c3900 0x7f66c4152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66ca450700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c4154650 con 0x7f66c40c3520 2024-09-13T17:53:14.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66ca450700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c41edf20 con 0x7f66c4066420 2024-09-13T17:53:14.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66ca450700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c4151e90 con 0x7f66c40c3900 2024-09-13T17:53:14.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66c37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66c40c3520 0x7f66c41e2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66c37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66c40c3520 0x7f66c41e2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39042/0 (socket says 172.21.15.40:39042) 2024-09-13T17:53:14.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66c37fe700 1 -- 172.21.15.40:0/984731497 learned_addr learned my addr 172.21.15.40:0/984731497 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:14.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66c3fff700 1 --2- 172.21.15.40:0/984731497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c4066420 0x7f66c41ed9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66c89ed700 1 --2- 172.21.15.40:0/984731497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66c40c3900 0x7f66c4152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66c37fe700 1 -- 172.21.15.40:0/984731497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66c40c3900 msgr2=0x7f66c4152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66c37fe700 1 --2- 172.21.15.40:0/984731497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66c40c3900 0x7f66c4152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66c37fe700 1 -- 172.21.15.40:0/984731497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c4066420 msgr2=0x7f66c41ed9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66c37fe700 1 --2- 172.21.15.40:0/984731497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c4066420 0x7f66c41ed9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66c37fe700 1 -- 172.21.15.40:0/984731497 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b8009570 con 0x7f66c40c3520 2024-09-13T17:53:14.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66c37fe700 1 --2- 172.21.15.40:0/984731497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66c40c3520 0x7f66c41e2d50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f66b8013000 tx=0x7f66b8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:14.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66c0ff9700 1 -- 172.21.15.40:0/984731497 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8004510 con 0x7f66c40c3520 2024-09-13T17:53:14.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66c0ff9700 1 -- 172.21.15.40:0/984731497 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b801e070 con 0x7f66c40c3520 2024-09-13T17:53:14.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66ca450700 1 -- 172.21.15.40:0/984731497 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c4153ad0 con 0x7f66c40c3520 2024-09-13T17:53:14.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66c0ff9700 1 -- 172.21.15.40:0/984731497 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b80038e0 con 0x7f66c40c3520 2024-09-13T17:53:14.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66ca450700 1 -- 172.21.15.40:0/984731497 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c4154090 con 0x7f66c40c3520 2024-09-13T17:53:14.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.071+0000 7f66ca450700 1 -- 172.21.15.40:0/984731497 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c40c4670 con 0x7f66c40c3520 2024-09-13T17:53:14.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.075+0000 7f66c0ff9700 1 -- 172.21.15.40:0/984731497 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f66b8003a40 con 0x7f66c40c3520 2024-09-13T17:53:14.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.075+0000 7f66c0ff9700 1 --2- 172.21.15.40:0/984731497 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f66b0066730 0x7f66b0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.075+0000 7f66c0ff9700 1 -- 172.21.15.40:0/984731497 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f66b80851b0 con 0x7f66c40c3520 2024-09-13T17:53:14.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.075+0000 7f66c3fff700 1 --2- 172.21.15.40:0/984731497 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f66b0066730 0x7f66b0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.075+0000 7f66c3fff700 1 --2- 172.21.15.40:0/984731497 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f66b0066730 0x7f66b0068bf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009bf0 tx=0x7f66b4009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:14.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.079+0000 7f66c0ff9700 1 -- 172.21.15.40:0/984731497 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f66b8052150 con 0x7f66c40c3520 2024-09-13T17:53:14.177 INFO:tasks.workunit.client.0.smithi040.stdout:checking for string.h... yes 2024-09-13T17:53:14.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.331+0000 7f66ca450700 1 -- 172.21.15.40:0/984731497 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f66c40c8990 con 0x7f66c40c3520 2024-09-13T17:53:14.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.335+0000 7f66c0ff9700 1 -- 172.21.15.40:0/984731497 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f66b8020620 con 0x7f66c40c3520 2024-09-13T17:53:14.339 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":24,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:53:14.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.335+0000 7f66ca450700 1 -- 172.21.15.40:0/984731497 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f66b0066730 msgr2=0x7f66b0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.335+0000 7f66ca450700 1 --2- 172.21.15.40:0/984731497 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f66b0066730 0x7f66b0068bf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009bf0 tx=0x7f66b4009240 comp rx=0 tx=0).stop 2024-09-13T17:53:14.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.335+0000 7f66ca450700 1 -- 172.21.15.40:0/984731497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66c40c3520 msgr2=0x7f66c41e2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.335+0000 7f66ca450700 1 --2- 172.21.15.40:0/984731497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66c40c3520 0x7f66c41e2d50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f66b8013000 tx=0x7f66b8004770 comp rx=0 tx=0).stop 2024-09-13T17:53:14.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.335+0000 7f66ca450700 1 -- 172.21.15.40:0/984731497 shutdown_connections 2024-09-13T17:53:14.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.335+0000 7f66ca450700 1 --2- 172.21.15.40:0/984731497 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f66b0066730 0x7f66b0068bf0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.335+0000 7f66ca450700 1 --2- 172.21.15.40:0/984731497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66c40c3900 0x7f66c4152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.335+0000 7f66ca450700 1 --2- 172.21.15.40:0/984731497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66c40c3520 0x7f66c41e2d50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.335+0000 7f66ca450700 1 --2- 172.21.15.40:0/984731497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c4066420 0x7f66c41ed9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.335+0000 7f66ca450700 1 -- 172.21.15.40:0/984731497 >> 172.21.15.40:0/984731497 conn(0x7f66c406a390 msgr2=0x7f66c4155500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:14.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.335+0000 7f66ca450700 1 -- 172.21.15.40:0/984731497 shutdown_connections 2024-09-13T17:53:14.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.335+0000 7f66ca450700 1 -- 172.21.15.40:0/984731497 wait complete. 2024-09-13T17:53:14.355 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T17:53:14.355 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T17:53:14.356 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T17:53:14.356 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:53:14.356 INFO:tasks.workunit.client.0.smithi040.stdout:checking for memory.h... yes 2024-09-13T17:53:14.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.467+0000 7fb45cf82700 1 Processor -- start 2024-09-13T17:53:14.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.467+0000 7fb45cf82700 1 -- start start 2024-09-13T17:53:14.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.467+0000 7fb45cf82700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb4581503f0 0x7fb4580c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.467+0000 7fb45cf82700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4580c6d20 con 0x7fb458156710 2024-09-13T17:53:14.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.467+0000 7fb45cf82700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb458159770 con 0x7fb458070a70 2024-09-13T17:53:14.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.467+0000 7fb45cf82700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb458159a30 con 0x7fb4581503f0 2024-09-13T17:53:14.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.467+0000 7fb457fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb458156710 0x7fb458156b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44628/0 (socket says 172.21.15.40:44628) 2024-09-13T17:53:14.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.467+0000 7fb457fff700 1 -- 172.21.15.40:0/1433798714 learned_addr learned my addr 172.21.15.40:0/1433798714 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:53:14.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.467+0000 7fb4577fe700 1 --2- 172.21.15.40:0/1433798714 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb4581503f0 0x7fb4580c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.467+0000 7fb4567fc700 1 -- 172.21.15.40:0/1433798714 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077292867 0 0) 0x7fb4580c6d20 con 0x7fb458156710 2024-09-13T17:53:14.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.467+0000 7fb4567fc700 1 -- 172.21.15.40:0/1433798714 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4440035d0 con 0x7fb458156710 2024-09-13T17:53:14.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.467+0000 7fb4567fc700 1 -- 172.21.15.40:0/1433798714 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1002488620 0 0) 0x7fb458159770 con 0x7fb458070a70 2024-09-13T17:53:14.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.467+0000 7fb4567fc700 1 -- 172.21.15.40:0/1433798714 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4580c6d20 con 0x7fb458070a70 2024-09-13T17:53:14.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.467+0000 7fb4577fe700 1 -- 172.21.15.40:0/1433798714 >> v1:172.21.15.43:6789/0 conn(0x7fb458070a70 legacy=0x7fb458071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.467+0000 7fb4577fe700 1 -- 172.21.15.40:0/1433798714 >> v1:172.21.15.40:6789/0 conn(0x7fb458156710 legacy=0x7fb458156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.467+0000 7fb4577fe700 1 -- 172.21.15.40:0/1433798714 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb458159d10 con 0x7fb4581503f0 2024-09-13T17:53:14.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.467+0000 7fb4577fe700 1 --2- 172.21.15.40:0/1433798714 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb4581503f0 0x7fb4580c67e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb450009dc0 tx=0x7fb45000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2837c87babb7d2c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:14.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.471+0000 7fb4567fc700 1 -- 172.21.15.40:0/1433798714 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45000e070 con 0x7fb4581503f0 2024-09-13T17:53:14.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.471+0000 7fb4567fc700 1 -- 172.21.15.40:0/1433798714 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45000bb20 con 0x7fb4581503f0 2024-09-13T17:53:14.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.471+0000 7fb4567fc700 1 -- 172.21.15.40:0/1433798714 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45000be90 con 0x7fb4581503f0 2024-09-13T17:53:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.471+0000 7fb45cf82700 1 -- 172.21.15.40:0/1433798714 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb4581503f0 msgr2=0x7fb4580c67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.471+0000 7fb45cf82700 1 --2- 172.21.15.40:0/1433798714 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb4581503f0 0x7fb4580c67e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb450009dc0 tx=0x7fb45000b330 comp rx=0 tx=0).stop 2024-09-13T17:53:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.471+0000 7fb45cf82700 1 -- 172.21.15.40:0/1433798714 shutdown_connections 2024-09-13T17:53:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.471+0000 7fb45cf82700 1 --2- 172.21.15.40:0/1433798714 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb4581503f0 0x7fb4580c67e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.471+0000 7fb45cf82700 1 -- 172.21.15.40:0/1433798714 >> 172.21.15.40:0/1433798714 conn(0x7fb4580cc7f0 msgr2=0x7fb4580c88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.471+0000 7fb45cf82700 1 -- 172.21.15.40:0/1433798714 shutdown_connections 2024-09-13T17:53:14.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.471+0000 7fb45cf82700 1 -- 172.21.15.40:0/1433798714 wait complete. 2024-09-13T17:53:14.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb45cf82700 1 Processor -- start 2024-09-13T17:53:14.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb45cf82700 1 -- start start 2024-09-13T17:53:14.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb45cf82700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb458070a70 0x7fb45814f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb45cf82700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4581503f0 0x7fb4581f2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb45cf82700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb458156710 0x7fb4581f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb45cf82700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4581eecb0 con 0x7fb4581503f0 2024-09-13T17:53:14.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb45cf82700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4581f22a0 con 0x7fb458070a70 2024-09-13T17:53:14.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb45cf82700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4580cbc10 con 0x7fb458156710 2024-09-13T17:53:14.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb456ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4581503f0 0x7fb4581f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb456ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4581503f0 0x7fb4581f2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39052/0 (socket says 172.21.15.40:39052) 2024-09-13T17:53:14.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb456ffd700 1 -- 172.21.15.40:0/2025368620 learned_addr learned my addr 172.21.15.40:0/2025368620 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:14.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb4577fe700 1 --2- 172.21.15.40:0/2025368620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb458070a70 0x7fb45814f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb457fff700 1 --2- 172.21.15.40:0/2025368620 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb458156710 0x7fb4581f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb456ffd700 1 -- 172.21.15.40:0/2025368620 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb458156710 msgr2=0x7fb4581f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb456ffd700 1 --2- 172.21.15.40:0/2025368620 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb458156710 0x7fb4581f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb456ffd700 1 -- 172.21.15.40:0/2025368620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb458070a70 msgr2=0x7fb45814f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb456ffd700 1 --2- 172.21.15.40:0/2025368620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb458070a70 0x7fb45814f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb456ffd700 1 -- 172.21.15.40:0/2025368620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb450009a70 con 0x7fb4581503f0 2024-09-13T17:53:14.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb456ffd700 1 --2- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4581503f0 0x7fb4581f2730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb44000c580 tx=0x7fb44000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:14.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb44ffff700 1 -- 172.21.15.40:0/2025368620 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4400052d0 con 0x7fb4581503f0 2024-09-13T17:53:14.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb44ffff700 1 -- 172.21.15.40:0/2025368620 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb440005430 con 0x7fb4581503f0 2024-09-13T17:53:14.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb45cf82700 1 -- 172.21.15.40:0/2025368620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4581eeea0 con 0x7fb4581503f0 2024-09-13T17:53:14.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb45cf82700 1 -- 172.21.15.40:0/2025368620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4581ef460 con 0x7fb4581503f0 2024-09-13T17:53:14.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.475+0000 7fb44ffff700 1 -- 172.21.15.40:0/2025368620 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4400055a0 con 0x7fb4581503f0 2024-09-13T17:53:14.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.479+0000 7fb45cf82700 1 --2- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43c002a40 0x7fb43c004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.479+0000 7fb45cf82700 1 -- 172.21.15.40:0/2025368620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c007f40 con 0x7fb43c002a40 2024-09-13T17:53:14.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.479+0000 7fb4577fe700 1 --2- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43c002a40 0x7fb43c004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.479+0000 7fb4577fe700 1 -- 172.21.15.40:0/2025368620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb43c008100 con 0x7fb43c002a40 2024-09-13T17:53:14.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.479+0000 7fb4577fe700 1 --2- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43c002a40 0x7fb43c004f00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb450009dc0 tx=0x7fb450004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:14.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.479+0000 7fb44ffff700 1 -- 172.21.15.40:0/2025368620 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb450025400 con 0x7fb43c002a40 2024-09-13T17:53:14.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.479+0000 7fb44ffff700 1 -- 172.21.15.40:0/2025368620 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb4400038e0 con 0x7fb4581503f0 2024-09-13T17:53:14.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.479+0000 7fb44ffff700 1 --2- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb4380667d0 0x7fb438068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.479+0000 7fb44ffff700 1 -- 172.21.15.40:0/2025368620 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb440083850 con 0x7fb4581503f0 2024-09-13T17:53:14.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.479+0000 7fb457fff700 1 --2- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb4380667d0 0x7fb438068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.479+0000 7fb44ffff700 1 -- 172.21.15.40:0/2025368620 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb450025560 con 0x7fb43c002a40 2024-09-13T17:53:14.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.479+0000 7fb44ffff700 1 -- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43c002a40 msgr2=0x7fb43c004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.479+0000 7fb44ffff700 1 --2- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43c002a40 0x7fb43c004f00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb450009dc0 tx=0x7fb450004b60 comp rx=0 tx=0).stop 2024-09-13T17:53:14.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.479+0000 7fb44ffff700 1 -- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43c002a40 msgr2=0x7fb43c004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:53:14.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.479+0000 7fb44ffff700 1 --2- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43c002a40 0x7fb43c004f00 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb450009dc0 tx=0x7fb450004b60 comp rx=0 tx=0).stop 2024-09-13T17:53:14.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.479+0000 7fb457fff700 1 --2- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb4380667d0 0x7fb438068c90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb458158380 tx=0x7fb448007d60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:14.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb45cf82700 1 --2- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43c007040 0x7fb43c0074d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb45cf82700 1 -- 172.21.15.40:0/2025368620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43c00a400 con 0x7fb43c007040 2024-09-13T17:53:14.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb4577fe700 1 --2- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43c007040 0x7fb43c0074d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb4577fe700 1 -- 172.21.15.40:0/2025368620 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb43c00a5c0 con 0x7fb43c007040 2024-09-13T17:53:14.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb4577fe700 1 --2- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43c007040 0x7fb43c0074d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb450009f30 tx=0x7fb450004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:14.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb44ffff700 1 -- 172.21.15.40:0/2025368620 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45002b2e0 con 0x7fb43c007040 2024-09-13T17:53:14.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb44ffff700 1 -- 172.21.15.40:0/2025368620 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4618 (secure 0 0 0) 0x7fb45002b620 con 0x7fb43c007040 2024-09-13T17:53:14.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb44ffff700 1 -- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43c007040 msgr2=0x7fb43c0074d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb44ffff700 1 --2- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43c007040 0x7fb43c0074d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb450009f30 tx=0x7fb450004af0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb44ffff700 1 -- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43c007040 msgr2=0x7fb43c0074d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:53:14.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb44ffff700 1 --2- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb43c007040 0x7fb43c0074d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.500 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:53:14.500 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:53:14.501 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:53:14.501 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:53:14.501 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 24, 2024-09-13T17:53:14.501 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:53:14.501 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:53:14.502 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:53:14.502 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:53:14.502 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:14.502 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 15, 2024-09-13T17:53:14.502 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:53:14.502 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:53:14.503 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:53:14.503 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:14.503 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:14.503 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:14.503 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:14.503 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:14.503 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:14.504 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:14.504 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:14.504 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:14.504 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:14.504 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:14.504 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:53:14.504 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:53:14.504 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:14.504 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:14.504 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:14.504 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:14.504 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:14.505 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:14.505 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:14.505 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:14.505 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:14.505 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:14.505 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:14.505 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.505 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:53:14.505 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:53:14.505 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:53:14.505 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:53:14.505 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:53:14.505 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:53:14.505 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:53:14.506 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:53:14.506 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:53:14.506 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:53:14.506 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:53:14.506 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:53:14.506 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:53:14.506 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:53:14.506 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:53:14.506 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:53:14.506 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:53:14.506 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:14.506 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:14.506 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:14.506 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:14.507 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:14.507 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:14.507 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:14.507 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:14.507 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:14.507 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:14.507 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:14.507 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:53:14.507 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.507 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:53:14.507 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.507 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:53:14.507 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:53:14.508 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:14.508 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:14.508 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.508 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:14.508 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:53:14.508 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:14.508 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.508 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.508 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:14.508 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:53:14.508 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:14.508 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.508 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:14.508 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.509 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:53:14.509 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:53:14.509 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:14.509 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:14.509 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:14.509 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.509 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.509 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:53:14.509 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:53:14.509 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:14.509 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:14.509 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.509 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:14.510 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:53:14.510 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:14.510 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.510 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.510 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:14.510 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:53:14.510 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:14.510 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.511 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:14.512 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.512 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:53:14.512 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:53:14.512 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:14.512 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:14.512 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:14.512 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.512 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.512 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:53:14.512 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:53:14.512 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:14.512 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:14.512 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.513 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:14.513 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:53:14.513 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:14.513 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.513 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.513 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:14.513 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:53:14.514 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:14.514 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.514 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:14.514 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.514 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:53:14.514 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:53:14.515 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:14.515 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:14.515 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:14.515 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.515 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:14.515 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.515 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:53:14.515 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:53:14.515 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.515 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:14.515 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:14.515 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:53:14.516 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.516 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:14.516 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:53:14.516 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.516 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:14.516 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:14.516 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:53:14.516 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.516 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:14.516 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:53:14.516 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.516 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:14.517 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:14.517 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:53:14.517 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.517 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:14.517 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.518 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:53:14.518 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:53:14.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb45cf82700 1 -- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb4380667d0 msgr2=0x7fb438068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb45cf82700 1 --2- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb4380667d0 0x7fb438068c90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb458158380 tx=0x7fb448007d60 comp rx=0 tx=0).stop 2024-09-13T17:53:14.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb45cf82700 1 -- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4581503f0 msgr2=0x7fb4581f2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb45cf82700 1 --2- 172.21.15.40:0/2025368620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4581503f0 0x7fb4581f2730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb44000c580 tx=0x7fb44000c890 comp rx=0 tx=0).stop 2024-09-13T17:53:14.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb457fff700 1 -- 172.21.15.40:0/2025368620 reap_dead start 2024-09-13T17:53:14.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb457fff700 1 -- 172.21.15.40:0/2025368620 reap_dead start 2024-09-13T17:53:14.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb45cf82700 1 -- 172.21.15.40:0/2025368620 shutdown_connections 2024-09-13T17:53:14.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb45cf82700 1 -- 172.21.15.40:0/2025368620 >> 172.21.15.40:0/2025368620 conn(0x7fb4580cc7f0 msgr2=0x7fb458156b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb45cf82700 1 -- 172.21.15.40:0/2025368620 shutdown_connections 2024-09-13T17:53:14.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.491+0000 7fb45cf82700 1 -- 172.21.15.40:0/2025368620 wait complete. 2024-09-13T17:53:14.521 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:53:14.521 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T17:53:14.521 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:53:14.541 INFO:tasks.workunit.client.0.smithi040.stdout:checking for strings.h... yes 2024-09-13T17:53:14.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.635+0000 7f2ce359e700 1 Processor -- start 2024-09-13T17:53:14.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.635+0000 7f2ce359e700 1 -- start start 2024-09-13T17:53:14.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.635+0000 7f2ce359e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ce4153730 0x7f2ce4159f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce359e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2ce415cfe0 0x7f2ce415d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce359e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce415a450 con 0x7f2ce40c7fb0 2024-09-13T17:53:14.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce359e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce415d9d0 con 0x7f2ce415cfe0 2024-09-13T17:53:14.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce359e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce4070ae0 con 0x7f2ce4153730 2024-09-13T17:53:14.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce259c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ce4153730 0x7f2ce4159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce2d9d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2ce415cfe0 0x7f2ce415d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce259c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ce4153730 0x7f2ce4159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:55430/0 (socket says 172.21.15.40:55430) 2024-09-13T17:53:14.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce2d9d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2ce415cfe0 0x7f2ce415d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:35380/0 (socket says 172.21.15.40:35380) 2024-09-13T17:53:14.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce2d9d700 1 -- 172.21.15.40:0/2619302834 learned_addr learned my addr 172.21.15.40:0/2619302834 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:14.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce2d9d700 1 -- 172.21.15.40:0/2619302834 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ce4153730 msgr2=0x7f2ce4159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:53:14.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce2d9d700 1 --2- 172.21.15.40:0/2619302834 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ce4153730 0x7f2ce4159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce2d9d700 1 -- 172.21.15.40:0/2619302834 >> v1:172.21.15.43:6789/0 conn(0x7f2ce40c7fb0 legacy=0x7f2ce40c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce2d9d700 1 -- 172.21.15.40:0/2619302834 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce4071010 con 0x7f2ce415cfe0 2024-09-13T17:53:14.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce2d9d700 1 --2- 172.21.15.40:0/2619302834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce415cfe0 0x7f2ce415d490 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009e50 tx=0x7f2cdc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bb72b603e25497a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:14.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce159a700 1 -- 172.21.15.40:0/2619302834 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc00e070 con 0x7f2ce415cfe0 2024-09-13T17:53:14.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce159a700 1 -- 172.21.15.40:0/2619302834 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc00b910 con 0x7f2ce415cfe0 2024-09-13T17:53:14.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce159a700 1 -- 172.21.15.40:0/2619302834 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc00bcc0 con 0x7f2ce415cfe0 2024-09-13T17:53:14.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce359e700 1 -- 172.21.15.40:0/2619302834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce415cfe0 msgr2=0x7f2ce415d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce359e700 1 --2- 172.21.15.40:0/2619302834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce415cfe0 0x7f2ce415d490 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009e50 tx=0x7f2cdc00b330 comp rx=0 tx=0).stop 2024-09-13T17:53:14.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce359e700 1 -- 172.21.15.40:0/2619302834 shutdown_connections 2024-09-13T17:53:14.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce359e700 1 --2- 172.21.15.40:0/2619302834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce415cfe0 0x7f2ce415d490 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce359e700 1 --2- 172.21.15.40:0/2619302834 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ce4153730 0x7f2ce4159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce359e700 1 -- 172.21.15.40:0/2619302834 >> 172.21.15.40:0/2619302834 conn(0x7f2ce406e6d0 msgr2=0x7f2ce406eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:14.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce359e700 1 -- 172.21.15.40:0/2619302834 shutdown_connections 2024-09-13T17:53:14.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.639+0000 7f2ce359e700 1 -- 172.21.15.40:0/2619302834 wait complete. 2024-09-13T17:53:14.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce359e700 1 Processor -- start 2024-09-13T17:53:14.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce359e700 1 -- start start 2024-09-13T17:53:14.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ce40c7fb0 0x7f2ce4156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce359e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce4153730 0x7f2ce41f2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce359e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ce415cfe0 0x7f2ce41eff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce359e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce41f2d80 con 0x7f2ce40c7fb0 2024-09-13T17:53:14.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce359e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce4155ae0 con 0x7f2ce4153730 2024-09-13T17:53:14.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce359e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce41f2aa0 con 0x7f2ce415cfe0 2024-09-13T17:53:14.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ce40c7fb0 0x7f2ce4156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ce40c7fb0 0x7f2ce4156760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39090/0 (socket says 172.21.15.40:39090) 2024-09-13T17:53:14.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce259c700 1 -- 172.21.15.40:0/2646685813 learned_addr learned my addr 172.21.15.40:0/2646685813 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:14.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce2d9d700 1 --2- 172.21.15.40:0/2646685813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ce415cfe0 0x7f2ce41eff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce1d9b700 1 --2- 172.21.15.40:0/2646685813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce4153730 0x7f2ce41f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce259c700 1 -- 172.21.15.40:0/2646685813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ce415cfe0 msgr2=0x7f2ce41eff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce259c700 1 --2- 172.21.15.40:0/2646685813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2ce415cfe0 0x7f2ce41eff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce259c700 1 -- 172.21.15.40:0/2646685813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce4153730 msgr2=0x7f2ce41f2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce259c700 1 --2- 172.21.15.40:0/2646685813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce4153730 0x7f2ce41f2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce259c700 1 -- 172.21.15.40:0/2646685813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc009b30 con 0x7f2ce40c7fb0 2024-09-13T17:53:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce259c700 1 --2- 172.21.15.40:0/2646685813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ce40c7fb0 0x7f2ce4156760 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400e9d0 tx=0x7f2cd400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2cd2ffd700 1 -- 172.21.15.40:0/2646685813 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4009720 con 0x7f2ce40c7fb0 2024-09-13T17:53:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2cd2ffd700 1 -- 172.21.15.40:0/2646685813 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd400ee70 con 0x7f2ce40c7fb0 2024-09-13T17:53:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce359e700 1 -- 172.21.15.40:0/2646685813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce4155800 con 0x7f2ce40c7fb0 2024-09-13T17:53:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2cd2ffd700 1 -- 172.21.15.40:0/2646685813 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd400ca50 con 0x7f2ce40c7fb0 2024-09-13T17:53:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.643+0000 7f2ce359e700 1 -- 172.21.15.40:0/2646685813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce40c5e00 con 0x7f2ce40c7fb0 2024-09-13T17:53:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.647+0000 7f2ce359e700 1 --2- 172.21.15.40:0/2646685813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce41f0490 0x7f2ce4153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.647+0000 7f2ce359e700 1 -- 172.21.15.40:0/2646685813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce40c59e0 con 0x7f2ce41f0490 2024-09-13T17:53:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.647+0000 7f2ce1d9b700 1 --2- 172.21.15.40:0/2646685813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce41f0490 0x7f2ce4153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.647+0000 7f2cd2ffd700 1 -- 172.21.15.40:0/2646685813 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2cd400cbb0 con 0x7f2ce40c7fb0 2024-09-13T17:53:14.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.647+0000 7f2ce1d9b700 1 -- 172.21.15.40:0/2646685813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ce4152c30 con 0x7f2ce41f0490 2024-09-13T17:53:14.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.647+0000 7f2cd2ffd700 1 --2- 172.21.15.40:0/2646685813 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2cc4066730 0x7f2cc4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.647+0000 7f2cd2ffd700 1 -- 172.21.15.40:0/2646685813 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2cd4084530 con 0x7f2ce40c7fb0 2024-09-13T17:53:14.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.647+0000 7f2ce1d9b700 1 --2- 172.21.15.40:0/2646685813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce41f0490 0x7f2ce4153c40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2ce406eb20 tx=0x7f2cd800fb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:14.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.647+0000 7f2ce2d9d700 1 --2- 172.21.15.40:0/2646685813 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2cc4066730 0x7f2cc4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.647+0000 7f2cd2ffd700 1 -- 172.21.15.40:0/2646685813 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd80052e0 con 0x7f2ce41f0490 2024-09-13T17:53:14.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.647+0000 7f2ce2d9d700 1 --2- 172.21.15.40:0/2646685813 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2cc4066730 0x7f2cc4068bf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00b370 tx=0x7f2cdc009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:14.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.647+0000 7f2cd2ffd700 1 -- 172.21.15.40:0/2646685813 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f2cd8005440 con 0x7f2ce41f0490 2024-09-13T17:53:14.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.647+0000 7f2cd2ffd700 1 -- 172.21.15.40:0/2646685813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce41f0490 msgr2=0x7f2ce4153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.647+0000 7f2cd2ffd700 1 --2- 172.21.15.40:0/2646685813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce41f0490 0x7f2ce4153c40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2ce406eb20 tx=0x7f2cd800fb00 comp rx=0 tx=0).stop 2024-09-13T17:53:14.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.647+0000 7f2cd2ffd700 1 -- 172.21.15.40:0/2646685813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce41f0490 msgr2=0x7f2ce4153c40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:53:14.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.647+0000 7f2cd2ffd700 1 --2- 172.21.15.40:0/2646685813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce41f0490 0x7f2ce4153c40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.655+0000 7f2ce359e700 1 --2- 172.21.15.40:0/2646685813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce4154180 0x7f2ce4154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.655+0000 7f2ce359e700 1 -- 172.21.15.40:0/2646685813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce4067600 con 0x7f2ce4154180 2024-09-13T17:53:14.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.659+0000 7f2ce1d9b700 1 --2- 172.21.15.40:0/2646685813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce4154180 0x7f2ce4154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.659+0000 7f2ce1d9b700 1 -- 172.21.15.40:0/2646685813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2ce40b1d40 con 0x7f2ce4154180 2024-09-13T17:53:14.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.659+0000 7f2ce1d9b700 1 --2- 172.21.15.40:0/2646685813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce4154180 0x7f2ce4154590 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8017000 tx=0x7f2cd8001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:14.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.659+0000 7f2cd2ffd700 1 -- 172.21.15.40:0/2646685813 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8023070 con 0x7f2ce4154180 2024-09-13T17:53:14.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.659+0000 7f2cd2ffd700 1 -- 172.21.15.40:0/2646685813 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7f2cd8003e30 con 0x7f2ce4154180 2024-09-13T17:53:14.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.659+0000 7f2cd2ffd700 1 -- 172.21.15.40:0/2646685813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce4154180 msgr2=0x7f2ce4154590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.659+0000 7f2cd2ffd700 1 --2- 172.21.15.40:0/2646685813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce4154180 0x7f2ce4154590 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8017000 tx=0x7f2cd8001120 comp rx=0 tx=0).stop 2024-09-13T17:53:14.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.659+0000 7f2cd2ffd700 1 -- 172.21.15.40:0/2646685813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce4154180 msgr2=0x7f2ce4154590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:53:14.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.659+0000 7f2cd2ffd700 1 --2- 172.21.15.40:0/2646685813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ce4154180 0x7f2ce4154590 secure :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8017000 tx=0x7f2cd8001120 comp rx=0 tx=0).stop 2024-09-13T17:53:14.666 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:53:14.667 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:53:14.667 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:53:14.667 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:53:14.667 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 24, 2024-09-13T17:53:14.667 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:53:14.667 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:53:14.667 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:53:14.667 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:53:14.667 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:14.667 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 16, 2024-09-13T17:53:14.667 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:53:14.667 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:53:14.667 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:53:14.667 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:14.668 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:14.668 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:14.668 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:14.668 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:14.668 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:14.668 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:14.668 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:14.668 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:14.668 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:14.668 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:14.668 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:53:14.668 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:53:14.669 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:14.669 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:14.669 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:14.669 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:14.669 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:14.669 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:14.669 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:14.669 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:14.669 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:14.669 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:14.669 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:14.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.669 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:53:14.670 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:53:14.670 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:53:14.670 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:53:14.670 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:53:14.670 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:53:14.670 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:53:14.670 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:53:14.670 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:53:14.670 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:53:14.670 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:53:14.670 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:53:14.670 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:53:14.670 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:53:14.671 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:53:14.671 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:53:14.671 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:53:14.671 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:14.671 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:14.671 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:14.671 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:14.671 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:14.671 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:14.671 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:14.671 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:14.671 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:14.671 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:14.672 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:14.672 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:53:14.672 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.672 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:53:14.672 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.672 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:53:14.672 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:53:14.672 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:14.672 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:14.672 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.672 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:14.672 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:53:14.672 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:14.673 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.673 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.673 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:14.673 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:53:14.673 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:14.673 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.673 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:14.673 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.673 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:53:14.673 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:53:14.673 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:14.673 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:14.674 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:14.674 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.674 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.674 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:53:14.674 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:53:14.674 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:14.674 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:14.674 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.674 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:14.674 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:53:14.674 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:14.674 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.674 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.674 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:14.675 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:53:14.675 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:14.675 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.675 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:14.675 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.675 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:53:14.675 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:53:14.675 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:14.675 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:14.675 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:14.675 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.675 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.676 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:53:14.676 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:53:14.676 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:14.676 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:14.676 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.676 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:14.676 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:53:14.676 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:14.676 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.676 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.676 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:14.676 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:53:14.676 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:14.676 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.677 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:14.677 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.677 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:53:14.677 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:53:14.677 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:14.677 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:14.677 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:14.677 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.677 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:14.677 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.677 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:53:14.677 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:53:14.678 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.678 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:14.678 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:14.678 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:53:14.678 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.678 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:14.678 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:53:14.679 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.679 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:14.679 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:14.679 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:53:14.679 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.679 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:14.679 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:53:14.679 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.680 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:14.680 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:14.680 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:53:14.680 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.680 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:14.680 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:53:14.681 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.681 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:14.681 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:14.681 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:53:14.681 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.681 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:14.681 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:53:14.681 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.681 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:14.681 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:14.682 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:53:14.682 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.682 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:14.682 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.682 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:53:14.682 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:53:14.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.659+0000 7f2ce359e700 1 -- 172.21.15.40:0/2646685813 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2cc4066730 msgr2=0x7f2cc4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.659+0000 7f2ce359e700 1 --2- 172.21.15.40:0/2646685813 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2cc4066730 0x7f2cc4068bf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00b370 tx=0x7f2cdc009000 comp rx=0 tx=0).stop 2024-09-13T17:53:14.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.659+0000 7f2ce359e700 1 -- 172.21.15.40:0/2646685813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ce40c7fb0 msgr2=0x7f2ce4156760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.659+0000 7f2ce359e700 1 --2- 172.21.15.40:0/2646685813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ce40c7fb0 0x7f2ce4156760 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400e9d0 tx=0x7f2cd400ece0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.659+0000 7f2ce2d9d700 1 -- 172.21.15.40:0/2646685813 reap_dead start 2024-09-13T17:53:14.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.659+0000 7f2ce2d9d700 1 -- 172.21.15.40:0/2646685813 reap_dead start 2024-09-13T17:53:14.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.659+0000 7f2ce359e700 1 -- 172.21.15.40:0/2646685813 shutdown_connections 2024-09-13T17:53:14.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.659+0000 7f2ce359e700 1 -- 172.21.15.40:0/2646685813 >> 172.21.15.40:0/2646685813 conn(0x7f2ce406e6d0 msgr2=0x7f2ce406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:14.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.659+0000 7f2ce359e700 1 -- 172.21.15.40:0/2646685813 shutdown_connections 2024-09-13T17:53:14.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.659+0000 7f2ce359e700 1 -- 172.21.15.40:0/2646685813 wait complete. 2024-09-13T17:53:14.685 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:53:14.685 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T17:53:14.685 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:53:14.722 INFO:tasks.workunit.client.0.smithi040.stdout:checking for inttypes.h... yes 2024-09-13T17:53:14.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.839+0000 7f0f9235f700 1 Processor -- start 2024-09-13T17:53:14.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.839+0000 7f0f9235f700 1 -- start start 2024-09-13T17:53:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.839+0000 7f0f9235f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0f8c0c7fb0 0x7f0f8c159f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.839+0000 7f0f9235f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c15cfe0 0x7f0f8c15d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.839+0000 7f0f9235f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f8c15a450 con 0x7f0f8c153730 2024-09-13T17:53:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.839+0000 7f0f9235f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f8c070ae0 con 0x7f0f8c0c7fb0 2024-09-13T17:53:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.839+0000 7f0f9235f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c071010 con 0x7f0f8c15cfe0 2024-09-13T17:53:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.839+0000 7f0f90b5c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0f8c0c7fb0 0x7f0f8c159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.839+0000 7f0f9135d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0f8c153730 0x7f0f8c0c8630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44630/0 (socket says 172.21.15.40:44630) 2024-09-13T17:53:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.839+0000 7f0f9135d700 1 -- 172.21.15.40:0/514886486 learned_addr learned my addr 172.21.15.40:0/514886486 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:53:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.839+0000 7f0f91b5e700 1 --2- 172.21.15.40:0/514886486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c15cfe0 0x7f0f8c15d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.839+0000 7f0f90b5c700 1 -- 172.21.15.40:0/514886486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c15cfe0 msgr2=0x7f0f8c15d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:53:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.839+0000 7f0f90b5c700 1 --2- 172.21.15.40:0/514886486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c15cfe0 0x7f0f8c15d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.839+0000 7f0f90b5c700 1 -- 172.21.15.40:0/514886486 >> v1:172.21.15.40:6789/0 conn(0x7f0f8c153730 legacy=0x7f0f8c0c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.839+0000 7f0f90b5c700 1 -- 172.21.15.40:0/514886486 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c15d9b0 con 0x7f0f8c0c7fb0 2024-09-13T17:53:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.839+0000 7f0f7ffff700 1 -- 172.21.15.40:0/514886486 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944513010 0 0) 0x7f0f88001110 con 0x7f0f8c153730 2024-09-13T17:53:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.839+0000 7f0f90b5c700 1 --2- 172.21.15.40:0/514886486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f8c0c7fb0 0x7f0f8c159f10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009850 tx=0x7f0f80009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477639d227c47c9a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f7ffff700 1 -- 172.21.15.40:0/514886486 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80011830 con 0x7f0f8c0c7fb0 2024-09-13T17:53:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f7ffff700 1 -- 172.21.15.40:0/514886486 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f80011990 con 0x7f0f8c0c7fb0 2024-09-13T17:53:14.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f7ffff700 1 -- 172.21.15.40:0/514886486 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80011d40 con 0x7f0f8c0c7fb0 2024-09-13T17:53:14.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f9235f700 1 -- 172.21.15.40:0/514886486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f8c0c7fb0 msgr2=0x7f0f8c159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f9235f700 1 --2- 172.21.15.40:0/514886486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f8c0c7fb0 0x7f0f8c159f10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009850 tx=0x7f0f80009b60 comp rx=0 tx=0).stop 2024-09-13T17:53:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f9235f700 1 -- 172.21.15.40:0/514886486 shutdown_connections 2024-09-13T17:53:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f9235f700 1 --2- 172.21.15.40:0/514886486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c15cfe0 0x7f0f8c15d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f9235f700 1 --2- 172.21.15.40:0/514886486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f8c0c7fb0 0x7f0f8c159f10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f9235f700 1 -- 172.21.15.40:0/514886486 >> 172.21.15.40:0/514886486 conn(0x7f0f8c06e6d0 msgr2=0x7f0f8c06eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f9235f700 1 -- 172.21.15.40:0/514886486 shutdown_connections 2024-09-13T17:53:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f9235f700 1 -- 172.21.15.40:0/514886486 wait complete. 2024-09-13T17:53:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f9235f700 1 Processor -- start 2024-09-13T17:53:14.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f9235f700 1 -- start start 2024-09-13T17:53:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f9235f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f8c0c7fb0 0x7f0f8c0c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f9235f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f8c153730 0x7f0f8c14cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f9235f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c15cfe0 0x7f0f8c1570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f9235f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c158850 con 0x7f0f8c153730 2024-09-13T17:53:14.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f9235f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c1e2ea0 con 0x7f0f8c0c7fb0 2024-09-13T17:53:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f9235f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c0ca050 con 0x7f0f8c15cfe0 2024-09-13T17:53:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f90b5c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f8c153730 0x7f0f8c14cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f90b5c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f8c153730 0x7f0f8c14cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39118/0 (socket says 172.21.15.40:39118) 2024-09-13T17:53:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f90b5c700 1 -- 172.21.15.40:0/2893628064 learned_addr learned my addr 172.21.15.40:0/2893628064 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f91b5e700 1 --2- 172.21.15.40:0/2893628064 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c15cfe0 0x7f0f8c1570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.843+0000 7f0f9135d700 1 --2- 172.21.15.40:0/2893628064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f8c0c7fb0 0x7f0f8c0c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f90b5c700 1 -- 172.21.15.40:0/2893628064 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c15cfe0 msgr2=0x7f0f8c1570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f90b5c700 1 --2- 172.21.15.40:0/2893628064 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c15cfe0 0x7f0f8c1570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f90b5c700 1 -- 172.21.15.40:0/2893628064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f8c0c7fb0 msgr2=0x7f0f8c0c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f90b5c700 1 --2- 172.21.15.40:0/2893628064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f8c0c7fb0 0x7f0f8c0c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f90b5c700 1 -- 172.21.15.40:0/2893628064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f800094b0 con 0x7f0f8c153730 2024-09-13T17:53:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f90b5c700 1 --2- 172.21.15.40:0/2893628064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f8c153730 0x7f0f8c14cea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f800090b0 tx=0x7f0f80004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f7dffb700 1 -- 172.21.15.40:0/2893628064 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8001aac0 con 0x7f0f8c153730 2024-09-13T17:53:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f7dffb700 1 -- 172.21.15.40:0/2893628064 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8001f070 con 0x7f0f8c153730 2024-09-13T17:53:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f9235f700 1 -- 172.21.15.40:0/2893628064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8c158de0 con 0x7f0f8c153730 2024-09-13T17:53:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f7dffb700 1 -- 172.21.15.40:0/2893628064 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8001ae60 con 0x7f0f8c153730 2024-09-13T17:53:14.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f9235f700 1 -- 172.21.15.40:0/2893628064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8c158290 con 0x7f0f8c153730 2024-09-13T17:53:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f9235f700 1 --2- 172.21.15.40:0/2893628064 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c158450 0x7f0f8c157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f9235f700 1 -- 172.21.15.40:0/2893628064 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c159f10 con 0x7f0f8c158450 2024-09-13T17:53:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f9135d700 1 --2- 172.21.15.40:0/2893628064 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c158450 0x7f0f8c157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f7dffb700 1 -- 172.21.15.40:0/2893628064 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0f8001b430 con 0x7f0f8c153730 2024-09-13T17:53:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f9135d700 1 -- 172.21.15.40:0/2893628064 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f8c0675e0 con 0x7f0f8c158450 2024-09-13T17:53:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f9135d700 1 --2- 172.21.15.40:0/2893628064 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c158450 0x7f0f8c157630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0f88001670 tx=0x7f0f88001a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:14.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f7dffb700 1 --2- 172.21.15.40:0/2893628064 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0f70066730 0x7f0f70068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f91b5e700 1 --2- 172.21.15.40:0/2893628064 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0f70066730 0x7f0f70068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f7dffb700 1 -- 172.21.15.40:0/2893628064 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0f800855c0 con 0x7f0f8c153730 2024-09-13T17:53:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.847+0000 7f0f7dffb700 1 -- 172.21.15.40:0/2893628064 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88004aa0 con 0x7f0f8c158450 2024-09-13T17:53:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.851+0000 7f0f7dffb700 1 -- 172.21.15.40:0/2893628064 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0f88004c00 con 0x7f0f8c158450 2024-09-13T17:53:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.851+0000 7f0f7dffb700 1 -- 172.21.15.40:0/2893628064 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c158450 msgr2=0x7f0f8c157630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.851+0000 7f0f7dffb700 1 --2- 172.21.15.40:0/2893628064 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c158450 0x7f0f8c157630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0f88001670 tx=0x7f0f88001a30 comp rx=0 tx=0).stop 2024-09-13T17:53:14.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.851+0000 7f0f7dffb700 1 -- 172.21.15.40:0/2893628064 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c158450 msgr2=0x7f0f8c157630 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:53:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.851+0000 7f0f7dffb700 1 --2- 172.21.15.40:0/2893628064 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c158450 0x7f0f8c157630 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.851+0000 7f0f91b5e700 1 --2- 172.21.15.40:0/2893628064 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0f70066730 0x7f0f70068bf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0f840073f0 tx=0x7f0f8400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:14.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.863+0000 7f0f9235f700 1 --2- 172.21.15.40:0/2893628064 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c1e4f30 0x7f0f8c1f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:14.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.863+0000 7f0f9235f700 1 -- 172.21.15.40:0/2893628064 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c1e2cc0 con 0x7f0f8c1e4f30 2024-09-13T17:53:14.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.863+0000 7f0f9135d700 1 --2- 172.21.15.40:0/2893628064 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c1e4f30 0x7f0f8c1f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:14.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.863+0000 7f0f9135d700 1 -- 172.21.15.40:0/2893628064 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0f8c071e60 con 0x7f0f8c1e4f30 2024-09-13T17:53:14.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.867+0000 7f0f9135d700 1 --2- 172.21.15.40:0/2893628064 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c1e4f30 0x7f0f8c1f2590 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0f88015000 tx=0x7f0f88001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:14.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.867+0000 7f0f7dffb700 1 -- 172.21.15.40:0/2893628064 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88022070 con 0x7f0f8c1e4f30 2024-09-13T17:53:14.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.867+0000 7f0f7dffb700 1 -- 172.21.15.40:0/2893628064 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4450 (secure 0 0 0) 0x7f0f8801d320 con 0x7f0f8c1e4f30 2024-09-13T17:53:14.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.867+0000 7f0f7dffb700 1 -- 172.21.15.40:0/2893628064 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c1e4f30 msgr2=0x7f0f8c1f2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.867+0000 7f0f7dffb700 1 --2- 172.21.15.40:0/2893628064 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c1e4f30 0x7f0f8c1f2590 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0f88015000 tx=0x7f0f88001080 comp rx=0 tx=0).stop 2024-09-13T17:53:14.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.867+0000 7f0f7dffb700 1 -- 172.21.15.40:0/2893628064 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c1e4f30 msgr2=0x7f0f8c1f2590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:53:14.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.867+0000 7f0f7dffb700 1 --2- 172.21.15.40:0/2893628064 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0f8c1e4f30 0x7f0f8c1f2590 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:14.872 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:53:14.873 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:53:14.873 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:53:14.873 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:53:14.873 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 24, 2024-09-13T17:53:14.873 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:53:14.873 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:53:14.873 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:53:14.873 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:53:14.874 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:14.874 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 16, 2024-09-13T17:53:14.874 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:53:14.874 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:53:14.874 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:53:14.874 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:14.874 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:14.874 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:14.874 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:14.874 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:14.874 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:14.874 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:14.875 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:14.875 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:14.875 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:14.875 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:14.875 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:53:14.875 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:53:14.875 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:14.875 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:14.875 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:14.875 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:14.875 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:14.875 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:14.875 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:14.875 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:14.876 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:14.876 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:14.876 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:14.876 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.876 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:53:14.876 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:53:14.876 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:53:14.876 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:53:14.876 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:53:14.876 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:53:14.876 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:53:14.876 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:53:14.876 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:53:14.876 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:53:14.877 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:53:14.877 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:53:14.877 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:53:14.877 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:53:14.877 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:53:14.877 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:53:14.877 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:53:14.877 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:14.877 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:14.877 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:14.877 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:14.877 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:14.877 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:14.877 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:14.878 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:14.878 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:14.878 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:14.878 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:14.878 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:53:14.878 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.878 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:53:14.878 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.878 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:53:14.878 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:53:14.878 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:14.878 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:14.878 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.879 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:14.879 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:53:14.879 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:14.879 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.879 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.879 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:14.879 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:53:14.879 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:14.879 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.879 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:14.879 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.879 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:53:14.879 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:53:14.879 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:14.879 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:14.880 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:14.880 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.880 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.880 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:53:14.880 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:53:14.880 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:14.880 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:14.880 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.880 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:14.880 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:53:14.880 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:14.880 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.880 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.881 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:14.881 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:53:14.881 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:14.881 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.881 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:14.881 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.881 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:53:14.881 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:53:14.881 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:14.881 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:14.881 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:14.881 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.882 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.882 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:53:14.882 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:53:14.882 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:14.882 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:14.882 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.882 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:14.882 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:53:14.882 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:14.882 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.882 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.882 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:14.882 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:53:14.882 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:14.883 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.883 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:14.883 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.883 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:53:14.883 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:53:14.883 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:14.883 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:14.883 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:14.883 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.883 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:14.883 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.884 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:53:14.884 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:53:14.884 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.884 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:14.884 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:14.884 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:53:14.884 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.884 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:14.885 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:53:14.885 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:14.885 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:14.885 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:14.885 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:53:14.885 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:14.885 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:14.885 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:14.885 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:53:14.886 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:53:14.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.867+0000 7f0f9235f700 1 -- 172.21.15.40:0/2893628064 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0f70066730 msgr2=0x7f0f70068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.867+0000 7f0f9235f700 1 --2- 172.21.15.40:0/2893628064 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0f70066730 0x7f0f70068bf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0f840073f0 tx=0x7f0f8400b040 comp rx=0 tx=0).stop 2024-09-13T17:53:14.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.867+0000 7f0f9235f700 1 -- 172.21.15.40:0/2893628064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f8c153730 msgr2=0x7f0f8c14cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:14.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.867+0000 7f0f9235f700 1 --2- 172.21.15.40:0/2893628064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f8c153730 0x7f0f8c14cea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f800090b0 tx=0x7f0f80004770 comp rx=0 tx=0).stop 2024-09-13T17:53:14.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.867+0000 7f0f91b5e700 1 -- 172.21.15.40:0/2893628064 reap_dead start 2024-09-13T17:53:14.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.867+0000 7f0f91b5e700 1 -- 172.21.15.40:0/2893628064 reap_dead start 2024-09-13T17:53:14.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.867+0000 7f0f9235f700 1 -- 172.21.15.40:0/2893628064 shutdown_connections 2024-09-13T17:53:14.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.867+0000 7f0f9235f700 1 -- 172.21.15.40:0/2893628064 >> 172.21.15.40:0/2893628064 conn(0x7f0f8c06e6d0 msgr2=0x7f0f8c06eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:14.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.867+0000 7f0f9235f700 1 -- 172.21.15.40:0/2893628064 shutdown_connections 2024-09-13T17:53:14.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:14.867+0000 7f0f9235f700 1 -- 172.21.15.40:0/2893628064 wait complete. 2024-09-13T17:53:14.889 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:53:14.889 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-13T17:53:14.890 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T17:53:14.890 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-13T17:53:14.890 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-13T17:53:14.890 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T17:53:14.890 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:53:14.914 INFO:tasks.workunit.client.0.smithi040.stdout:checking for stdint.h... yes 2024-09-13T17:53:14.932 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T17:53:14.932 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T17:53:14.932 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:53:15.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.043+0000 7f2dd9bd2700 1 Processor -- start 2024-09-13T17:53:15.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.043+0000 7f2dd9bd2700 1 -- start start 2024-09-13T17:53:15.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd9bd2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2dd4153840 0x7f2dd40c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:15.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd9bd2700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2dd40c9e20 0x7f2dd40cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:15.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd9bd2700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd40cc060 con 0x7f2dd40cc4f0 2024-09-13T17:53:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd9bd2700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd40b2840 con 0x7f2dd40c9e20 2024-09-13T17:53:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd9bd2700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd40ca870 con 0x7f2dd4153840 2024-09-13T17:53:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd3fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2dd40cc4f0 0x7f2dd415d410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44636/0 (socket says 172.21.15.40:44636) 2024-09-13T17:53:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd3fff700 1 -- 172.21.15.40:0/13298900 learned_addr learned my addr 172.21.15.40:0/13298900 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:53:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd93d1700 1 --2- 172.21.15.40:0/13298900 >> v2:172.21.15.40:3300/0 conn(0x7f2dd40c9e20 0x7f2dd40cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd8bd0700 1 -- 172.21.15.40:0/13298900 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2dd4153840 msgr2=0x7f2dd40c3eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:53:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd8bd0700 1 --2- 172.21.15.40:0/13298900 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2dd4153840 0x7f2dd40c3eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd93d1700 1 -- 172.21.15.40:0/13298900 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2dd4153840 msgr2=0x7f2dd40c3eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:53:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd93d1700 1 --2- 172.21.15.40:0/13298900 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2dd4153840 0x7f2dd40c3eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd93d1700 1 -- 172.21.15.40:0/13298900 >> v1:172.21.15.40:6789/0 conn(0x7f2dd40cc4f0 legacy=0x7f2dd415d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd93d1700 1 -- 172.21.15.40:0/13298900 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd4153c20 con 0x7f2dd40c9e20 2024-09-13T17:53:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd93d1700 1 --2- 172.21.15.40:0/13298900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dd40c9e20 0x7f2dd40cf260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009d90 tx=0x7f2dcc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf1816ea90c9bff server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:15.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd37fe700 1 -- 172.21.15.40:0/13298900 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc00e070 con 0x7f2dd40c9e20 2024-09-13T17:53:15.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd37fe700 1 -- 172.21.15.40:0/13298900 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dcc00bbd0 con 0x7f2dd40c9e20 2024-09-13T17:53:15.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd37fe700 1 -- 172.21.15.40:0/13298900 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc012430 con 0x7f2dd40c9e20 2024-09-13T17:53:15.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd9bd2700 1 -- 172.21.15.40:0/13298900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dd40c9e20 msgr2=0x7f2dd40cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:15.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd9bd2700 1 --2- 172.21.15.40:0/13298900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dd40c9e20 0x7f2dd40cf260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009d90 tx=0x7f2dcc00b330 comp rx=0 tx=0).stop 2024-09-13T17:53:15.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd9bd2700 1 -- 172.21.15.40:0/13298900 shutdown_connections 2024-09-13T17:53:15.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd9bd2700 1 --2- 172.21.15.40:0/13298900 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dd40c9e20 0x7f2dd40cf260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:15.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd9bd2700 1 --2- 172.21.15.40:0/13298900 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2dd4153840 0x7f2dd40c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:15.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd9bd2700 1 -- 172.21.15.40:0/13298900 >> 172.21.15.40:0/13298900 conn(0x7f2dd406c800 msgr2=0x7f2dd406cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:15.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.047+0000 7f2dd9bd2700 1 -- 172.21.15.40:0/13298900 shutdown_connections 2024-09-13T17:53:15.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd9bd2700 1 -- 172.21.15.40:0/13298900 wait complete. 2024-09-13T17:53:15.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd9bd2700 1 Processor -- start 2024-09-13T17:53:15.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd9bd2700 1 -- start start 2024-09-13T17:53:15.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd9bd2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dd40c9e20 0x7f2dd40c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd9bd2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2dd40cc4f0 0x7f2dd40c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd9bd2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dd4153840 0x7f2dd4154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd9bd2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd4154100 con 0x7f2dd40c9e20 2024-09-13T17:53:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd9bd2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd4156910 con 0x7f2dd4153840 2024-09-13T17:53:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd9bd2700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd40c8b50 con 0x7f2dd40cc4f0 2024-09-13T17:53:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd3fff700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2dd40cc4f0 msgr2=0x7f2dd40c7a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:53:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2dd40cc4f0 0x7f2dd40c7a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd8bd0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dd40c9e20 0x7f2dd40c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd8bd0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dd40c9e20 0x7f2dd40c74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39136/0 (socket says 172.21.15.40:39136) 2024-09-13T17:53:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd8bd0700 1 -- 172.21.15.40:0/1758556385 learned_addr learned my addr 172.21.15.40:0/1758556385 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd93d1700 1 --2- 172.21.15.40:0/1758556385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dd4153840 0x7f2dd4154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd8bd0700 1 -- 172.21.15.40:0/1758556385 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2dd40cc4f0 msgr2=0x7f2dd40c7a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:53:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd8bd0700 1 --2- 172.21.15.40:0/1758556385 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2dd40cc4f0 0x7f2dd40c7a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:15.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd8bd0700 1 -- 172.21.15.40:0/1758556385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dd4153840 msgr2=0x7f2dd4154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:15.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd8bd0700 1 --2- 172.21.15.40:0/1758556385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dd4153840 0x7f2dd4154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:15.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd8bd0700 1 -- 172.21.15.40:0/1758556385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc009a70 con 0x7f2dd40c9e20 2024-09-13T17:53:15.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.051+0000 7f2dd8bd0700 1 --2- 172.21.15.40:0/1758556385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dd40c9e20 0x7f2dd40c74f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800b780 tx=0x7f2dc800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:15.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.055+0000 7f2dd17fa700 1 -- 172.21.15.40:0/1758556385 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8003e70 con 0x7f2dd40c9e20 2024-09-13T17:53:15.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.055+0000 7f2dd17fa700 1 -- 172.21.15.40:0/1758556385 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc8005070 con 0x7f2dd40c9e20 2024-09-13T17:53:15.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.055+0000 7f2dd9bd2700 1 -- 172.21.15.40:0/1758556385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd4156bd0 con 0x7f2dd40c9e20 2024-09-13T17:53:15.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.055+0000 7f2dd17fa700 1 -- 172.21.15.40:0/1758556385 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8005420 con 0x7f2dd40c9e20 2024-09-13T17:53:15.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.055+0000 7f2dd9bd2700 1 -- 172.21.15.40:0/1758556385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd40c90f0 con 0x7f2dd40c9e20 2024-09-13T17:53:15.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.055+0000 7f2dd9bd2700 1 -- 172.21.15.40:0/1758556385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db4005310 con 0x7f2dd40c9e20 2024-09-13T17:53:15.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.059+0000 7f2dd17fa700 1 -- 172.21.15.40:0/1758556385 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2dc8005580 con 0x7f2dd40c9e20 2024-09-13T17:53:15.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.059+0000 7f2dd17fa700 1 --2- 172.21.15.40:0/1758556385 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2dc00666e0 0x7f2dc0068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:15.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.059+0000 7f2dd17fa700 1 -- 172.21.15.40:0/1758556385 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2dc8083ec0 con 0x7f2dd40c9e20 2024-09-13T17:53:15.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.059+0000 7f2dd3fff700 1 --2- 172.21.15.40:0/1758556385 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2dc00666e0 0x7f2dc0068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:15.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.059+0000 7f2dd3fff700 1 --2- 172.21.15.40:0/1758556385 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2dc00666e0 0x7f2dc0068ba0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4152a40 tx=0x7f2dc4007bb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:15.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.063+0000 7f2dd17fa700 1 -- 172.21.15.40:0/1758556385 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2dc8050e70 con 0x7f2dd40c9e20 2024-09-13T17:53:15.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.303+0000 7f2dd9bd2700 1 -- 172.21.15.40:0/1758556385 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2db40055c0 con 0x7f2dd40c9e20 2024-09-13T17:53:15.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.303+0000 7f2dd17fa700 1 -- 172.21.15.40:0/1758556385 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f2dc8005830 con 0x7f2dd40c9e20 2024-09-13T17:53:15.308 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":24,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:53:15.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.303+0000 7f2dd9bd2700 1 -- 172.21.15.40:0/1758556385 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2dc00666e0 msgr2=0x7f2dc0068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:15.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.303+0000 7f2dd9bd2700 1 --2- 172.21.15.40:0/1758556385 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2dc00666e0 0x7f2dc0068ba0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4152a40 tx=0x7f2dc4007bb0 comp rx=0 tx=0).stop 2024-09-13T17:53:15.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.303+0000 7f2dd9bd2700 1 -- 172.21.15.40:0/1758556385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dd40c9e20 msgr2=0x7f2dd40c74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:15.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.303+0000 7f2dd9bd2700 1 --2- 172.21.15.40:0/1758556385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dd40c9e20 0x7f2dd40c74f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800b780 tx=0x7f2dc800ba90 comp rx=0 tx=0).stop 2024-09-13T17:53:15.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.303+0000 7f2dd9bd2700 1 -- 172.21.15.40:0/1758556385 shutdown_connections 2024-09-13T17:53:15.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.303+0000 7f2dd9bd2700 1 --2- 172.21.15.40:0/1758556385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2dd4153840 0x7f2dd4154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:15.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.303+0000 7f2dd9bd2700 1 --2- 172.21.15.40:0/1758556385 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2dc00666e0 0x7f2dc0068ba0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:15.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.303+0000 7f2dd9bd2700 1 --2- 172.21.15.40:0/1758556385 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2dd40cc4f0 0x7f2dd40c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:15.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.303+0000 7f2dd9bd2700 1 --2- 172.21.15.40:0/1758556385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2dd40c9e20 0x7f2dd40c74f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:15.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.303+0000 7f2dd9bd2700 1 -- 172.21.15.40:0/1758556385 >> 172.21.15.40:0/1758556385 conn(0x7f2dd406c800 msgr2=0x7f2dd40b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:15.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.303+0000 7f2dd9bd2700 1 -- 172.21.15.40:0/1758556385 shutdown_connections 2024-09-13T17:53:15.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:15.303+0000 7f2dd9bd2700 1 -- 172.21.15.40:0/1758556385 wait complete. 2024-09-13T17:53:16.592 INFO:tasks.workunit.client.0.smithi040.stdout:checking for unistd.h... yes 2024-09-13T17:53:17.045 INFO:tasks.workunit.client.0.smithi040.stdout:checking minix/config.h usability... no 2024-09-13T17:53:17.204 INFO:tasks.workunit.client.0.smithi040.stdout:checking minix/config.h presence... no 2024-09-13T17:53:17.232 INFO:tasks.workunit.client.0.smithi040.stdout:checking for minix/config.h... no 2024-09-13T17:53:17.634 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-09-13T17:53:18.322 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:53:18.351 INFO:tasks.workunit.client.0.smithi040.stdout:checking for gcc... (cached) gcc 2024-09-13T17:53:18.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd25ffff700 1 Processor -- start 2024-09-13T17:53:18.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd25ffff700 1 -- start start 2024-09-13T17:53:18.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd25ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd260153810 0x7fd2600ca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:18.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd25ffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2600c3da0 0x7fd260159d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:18.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd25ffff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26006d400 con 0x7fd2600cafd0 2024-09-13T17:53:18.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd25ffff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd26015a260 con 0x7fd260153810 2024-09-13T17:53:18.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd25ffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26006ce40 con 0x7fd2600c3da0 2024-09-13T17:53:18.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd25effd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd260153810 0x7fd2600ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:18.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd2567fc700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2600c3da0 msgr2=0x7fd260159d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:53:18.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd25f7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd2600cafd0 0x7fd26006a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44642/0 (socket says 172.21.15.40:44642) 2024-09-13T17:53:18.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd2567fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2600c3da0 0x7fd260159d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:18.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd25f7fe700 1 -- 172.21.15.40:0/3316625633 learned_addr learned my addr 172.21.15.40:0/3316625633 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:53:18.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd25effd700 1 -- 172.21.15.40:0/3316625633 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2600c3da0 msgr2=0x7fd260159d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:53:18.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd25effd700 1 --2- 172.21.15.40:0/3316625633 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2600c3da0 0x7fd260159d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:18.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd25effd700 1 -- 172.21.15.40:0/3316625633 >> v1:172.21.15.40:6789/0 conn(0x7fd2600cafd0 legacy=0x7fd26006a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:18.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd25effd700 1 -- 172.21.15.40:0/3316625633 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26006d120 con 0x7fd260153810 2024-09-13T17:53:18.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd25e7fc700 1 -- 172.21.15.40:0/3316625633 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131000559 0 0) 0x7fd26006d400 con 0x7fd2600cafd0 2024-09-13T17:53:18.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd25effd700 1 --2- 172.21.15.40:0/3316625633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd260153810 0x7fd2600ca390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd250009850 tx=0x7fd250009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e645f58c5f9e29 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:18.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd25e7fc700 1 -- 172.21.15.40:0/3316625633 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250011830 con 0x7fd260153810 2024-09-13T17:53:18.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd25e7fc700 1 -- 172.21.15.40:0/3316625633 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd250011990 con 0x7fd260153810 2024-09-13T17:53:18.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.443+0000 7fd25e7fc700 1 -- 172.21.15.40:0/3316625633 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250011d20 con 0x7fd260153810 2024-09-13T17:53:18.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd25ffff700 1 -- 172.21.15.40:0/3316625633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd260153810 msgr2=0x7fd2600ca390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:18.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd25ffff700 1 --2- 172.21.15.40:0/3316625633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd260153810 0x7fd2600ca390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd250009850 tx=0x7fd250009b60 comp rx=0 tx=0).stop 2024-09-13T17:53:18.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd25ffff700 1 -- 172.21.15.40:0/3316625633 shutdown_connections 2024-09-13T17:53:18.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd25ffff700 1 --2- 172.21.15.40:0/3316625633 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2600c3da0 0x7fd260159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:18.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd25ffff700 1 --2- 172.21.15.40:0/3316625633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd260153810 0x7fd2600ca390 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:18.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd25ffff700 1 -- 172.21.15.40:0/3316625633 >> 172.21.15.40:0/3316625633 conn(0x7fd2600cbca0 msgr2=0x7fd2600cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:18.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd25ffff700 1 -- 172.21.15.40:0/3316625633 shutdown_connections 2024-09-13T17:53:18.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd25ffff700 1 -- 172.21.15.40:0/3316625633 wait complete. 2024-09-13T17:53:18.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd25ffff700 1 Processor -- start 2024-09-13T17:53:18.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd25ffff700 1 -- start start 2024-09-13T17:53:18.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd25ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2600c3da0 0x7fd2601f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:18.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd25ffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2600cafd0 0x7fd2601e6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:18.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd25ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd260153810 0x7fd2601563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:18.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd25ffff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd260153ca0 con 0x7fd260153810 2024-09-13T17:53:18.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd25ffff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2601efcc0 con 0x7fd2600c3da0 2024-09-13T17:53:18.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd25ffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2601e5500 con 0x7fd2600cafd0 2024-09-13T17:53:18.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd2567fc700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2600cafd0 msgr2=0x7fd2601e6080 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:53:18.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd2567fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2600cafd0 0x7fd2601e6080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:18.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd25f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd260153810 0x7fd2601563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:18.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd25f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd260153810 0x7fd2601563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39162/0 (socket says 172.21.15.40:39162) 2024-09-13T17:53:18.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd25f7fe700 1 -- 172.21.15.40:0/1480026820 learned_addr learned my addr 172.21.15.40:0/1480026820 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:18.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.447+0000 7fd25effd700 1 --2- 172.21.15.40:0/1480026820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2600c3da0 0x7fd2601f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:18.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.451+0000 7fd25f7fe700 1 -- 172.21.15.40:0/1480026820 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2600cafd0 msgr2=0x7fd2601e6080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:53:18.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.451+0000 7fd25f7fe700 1 --2- 172.21.15.40:0/1480026820 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2600cafd0 0x7fd2601e6080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:18.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.451+0000 7fd25f7fe700 1 -- 172.21.15.40:0/1480026820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2600c3da0 msgr2=0x7fd2601f2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:18.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.451+0000 7fd25f7fe700 1 --2- 172.21.15.40:0/1480026820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2600c3da0 0x7fd2601f2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:18.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.451+0000 7fd25f7fe700 1 -- 172.21.15.40:0/1480026820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2500094b0 con 0x7fd260153810 2024-09-13T17:53:18.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.451+0000 7fd25f7fe700 1 --2- 172.21.15.40:0/1480026820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd260153810 0x7fd2601563b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd25800caf0 tx=0x7fd25800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:18.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.451+0000 7fd25cff9700 1 -- 172.21.15.40:0/1480026820 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25800aaa0 con 0x7fd260153810 2024-09-13T17:53:18.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.451+0000 7fd25cff9700 1 -- 172.21.15.40:0/1480026820 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25800ac20 con 0x7fd260153810 2024-09-13T17:53:18.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.451+0000 7fd25ffff700 1 -- 172.21.15.40:0/1480026820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2601f2750 con 0x7fd260153810 2024-09-13T17:53:18.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.451+0000 7fd25cff9700 1 -- 172.21.15.40:0/1480026820 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2580070d0 con 0x7fd260153810 2024-09-13T17:53:18.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.451+0000 7fd25ffff700 1 -- 172.21.15.40:0/1480026820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2600b3040 con 0x7fd260153810 2024-09-13T17:53:18.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.451+0000 7fd25ffff700 1 -- 172.21.15.40:0/1480026820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2601f28e0 con 0x7fd260153810 2024-09-13T17:53:18.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.451+0000 7fd25cff9700 1 -- 172.21.15.40:0/1480026820 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd258007230 con 0x7fd260153810 2024-09-13T17:53:18.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.451+0000 7fd25cff9700 1 --2- 172.21.15.40:0/1480026820 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd2400666e0 0x7fd240068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:18.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.451+0000 7fd25cff9700 1 -- 172.21.15.40:0/1480026820 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd258083e30 con 0x7fd260153810 2024-09-13T17:53:18.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.455+0000 7fd25effd700 1 --2- 172.21.15.40:0/1480026820 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd2400666e0 0x7fd240068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:18.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.455+0000 7fd25effd700 1 --2- 172.21.15.40:0/1480026820 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd2400666e0 0x7fd240068ba0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd2500090b0 tx=0x7fd25001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:18.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.463+0000 7fd25cff9700 1 -- 172.21.15.40:0/1480026820 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd258050e10 con 0x7fd260153810 2024-09-13T17:53:18.644 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-13T17:53:18.660 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-13T17:53:18.680 INFO:tasks.workunit.client.0.smithi040.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-13T17:53:18.703 INFO:tasks.workunit.client.0.smithi040.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-13T17:53:18.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.719+0000 7fd25ffff700 1 -- 172.21.15.40:0/1480026820 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd2600cb460 con 0x7fd260153810 2024-09-13T17:53:18.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.719+0000 7fd25cff9700 1 -- 172.21.15.40:0/1480026820 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fd258018020 con 0x7fd260153810 2024-09-13T17:53:18.726 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":24,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:53:18.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.723+0000 7fd25ffff700 1 -- 172.21.15.40:0/1480026820 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd2400666e0 msgr2=0x7fd240068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:18.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.727+0000 7fd25ffff700 1 --2- 172.21.15.40:0/1480026820 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd2400666e0 0x7fd240068ba0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd2500090b0 tx=0x7fd25001c040 comp rx=0 tx=0).stop 2024-09-13T17:53:18.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.727+0000 7fd25ffff700 1 -- 172.21.15.40:0/1480026820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd260153810 msgr2=0x7fd2601563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:18.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.727+0000 7fd25ffff700 1 --2- 172.21.15.40:0/1480026820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd260153810 0x7fd2601563b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd25800caf0 tx=0x7fd25800ce00 comp rx=0 tx=0).stop 2024-09-13T17:53:18.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.727+0000 7fd25ffff700 1 -- 172.21.15.40:0/1480026820 shutdown_connections 2024-09-13T17:53:18.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.727+0000 7fd25ffff700 1 --2- 172.21.15.40:0/1480026820 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd260153810 0x7fd2601563b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:18.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.727+0000 7fd25ffff700 1 --2- 172.21.15.40:0/1480026820 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd2400666e0 0x7fd240068ba0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:18.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.727+0000 7fd25ffff700 1 --2- 172.21.15.40:0/1480026820 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd2600cafd0 0x7fd2601e6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:18.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.727+0000 7fd25ffff700 1 --2- 172.21.15.40:0/1480026820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2600c3da0 0x7fd2601f2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:18.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.727+0000 7fd25ffff700 1 -- 172.21.15.40:0/1480026820 >> 172.21.15.40:0/1480026820 conn(0x7fd2600cbca0 msgr2=0x7fd2600cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:18.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.727+0000 7fd25ffff700 1 -- 172.21.15.40:0/1480026820 shutdown_connections 2024-09-13T17:53:18.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:18.727+0000 7fd25ffff700 1 -- 172.21.15.40:0/1480026820 wait complete. 2024-09-13T17:53:18.912 INFO:tasks.workunit.client.0.smithi040.stdout:checking for main in -lm... yes 2024-09-13T17:53:19.623 INFO:tasks.workunit.client.0.smithi040.stdout:checking for main in -lpthread... yes 2024-09-13T17:53:19.651 INFO:tasks.workunit.client.0.smithi040.stdout:checking for ANSI C header files... (cached) yes 2024-09-13T17:53:20.029 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-13T17:53:20.458 INFO:tasks.workunit.client.0.smithi040.stdout:checking pthread.h usability... yes 2024-09-13T17:53:20.590 INFO:tasks.workunit.client.0.smithi040.stdout:checking pthread.h presence... yes 2024-09-13T17:53:20.615 INFO:tasks.workunit.client.0.smithi040.stdout:checking for pthread.h... yes 2024-09-13T17:53:20.842 INFO:tasks.workunit.client.0.smithi040.stdout:checking fcntl.h usability... yes 2024-09-13T17:53:21.744 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:53:21.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.862+0000 7ff43a6bf700 1 Processor -- start 2024-09-13T17:53:21.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.862+0000 7ff43a6bf700 1 -- start start 2024-09-13T17:53:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.862+0000 7ff43a6bf700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff42c0dc4d0 0x7ff42c044be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.862+0000 7ff43a6bf700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff42c0509b0 0x7ff42c045120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.862+0000 7ff43a6bf700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff42c0dc980 con 0x7ff42c0d6750 2024-09-13T17:53:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.862+0000 7ff43a6bf700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff42c0dd500 con 0x7ff42c0dc4d0 2024-09-13T17:53:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.862+0000 7ff43a6bf700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c0dcc60 con 0x7ff42c0509b0 2024-09-13T17:53:21.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.862+0000 7ff439ebe700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff42c0509b0 msgr2=0x7ff42c045120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:53:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.862+0000 7ff439ebe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff42c0509b0 0x7ff42c045120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.862+0000 7ff438ebc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff42c0dc4d0 0x7ff42c044be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.862+0000 7ff438ebc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff42c0dc4d0 0x7ff42c044be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:48796/0 (socket says 172.21.15.40:48796) 2024-09-13T17:53:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.862+0000 7ff438ebc700 1 -- 172.21.15.40:0/822465665 learned_addr learned my addr 172.21.15.40:0/822465665 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:21.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.862+0000 7ff438ebc700 1 -- 172.21.15.40:0/822465665 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff42c0509b0 msgr2=0x7ff42c045120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:53:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.862+0000 7ff438ebc700 1 --2- 172.21.15.40:0/822465665 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff42c0509b0 0x7ff42c045120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.862+0000 7ff438ebc700 1 -- 172.21.15.40:0/822465665 >> v1:172.21.15.43:6789/0 conn(0x7ff42c0d6750 legacy=0x7ff42c0e0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.862+0000 7ff438ebc700 1 -- 172.21.15.40:0/822465665 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42c0dcf40 con 0x7ff42c0dc4d0 2024-09-13T17:53:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.862+0000 7ff438ebc700 1 --2- 172.21.15.40:0/822465665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff42c0dc4d0 0x7ff42c044be0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff424009910 tx=0x7ff424009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=193bf970fbfb4933 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.862+0000 7ff42bfff700 1 -- 172.21.15.40:0/822465665 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42400ba60 con 0x7ff42c0dc4d0 2024-09-13T17:53:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.862+0000 7ff42bfff700 1 -- 172.21.15.40:0/822465665 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42400bbc0 con 0x7ff42c0dc4d0 2024-09-13T17:53:21.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.862+0000 7ff42bfff700 1 -- 172.21.15.40:0/822465665 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff424009e60 con 0x7ff42c0dc4d0 2024-09-13T17:53:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.866+0000 7ff43a6bf700 1 -- 172.21.15.40:0/822465665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff42c0dc4d0 msgr2=0x7ff42c044be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.866+0000 7ff43a6bf700 1 --2- 172.21.15.40:0/822465665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff42c0dc4d0 0x7ff42c044be0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff424009910 tx=0x7ff424009c20 comp rx=0 tx=0).stop 2024-09-13T17:53:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.866+0000 7ff43a6bf700 1 -- 172.21.15.40:0/822465665 shutdown_connections 2024-09-13T17:53:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.866+0000 7ff43a6bf700 1 --2- 172.21.15.40:0/822465665 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff42c0509b0 0x7ff42c045120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.866+0000 7ff43a6bf700 1 --2- 172.21.15.40:0/822465665 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff42c0dc4d0 0x7ff42c044be0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.866+0000 7ff43a6bf700 1 -- 172.21.15.40:0/822465665 >> 172.21.15.40:0/822465665 conn(0x7ff42c04fd00 msgr2=0x7ff42c050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.866+0000 7ff43a6bf700 1 -- 172.21.15.40:0/822465665 shutdown_connections 2024-09-13T17:53:21.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.866+0000 7ff43a6bf700 1 -- 172.21.15.40:0/822465665 wait complete. 2024-09-13T17:53:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff43a6bf700 1 Processor -- start 2024-09-13T17:53:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff43a6bf700 1 -- start start 2024-09-13T17:53:21.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff43a6bf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff42c0509b0 0x7ff42c175620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:21.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff43a6bf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff42c0d6750 0x7ff42c0487d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:21.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff43a6bf700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff42c0dc4d0 0x7ff42c04e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:21.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff43a6bf700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c166500 con 0x7ff42c0d6750 2024-09-13T17:53:21.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff43a6bf700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c175b60 con 0x7ff42c0509b0 2024-09-13T17:53:21.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff43a6bf700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c049f10 con 0x7ff42c0dc4d0 2024-09-13T17:53:21.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff439ebe700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff42c0dc4d0 msgr2=0x7ff42c04e2f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:53:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff439ebe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff42c0dc4d0 0x7ff42c04e2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff438ebc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff42c0d6750 0x7ff42c0487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff438ebc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff42c0d6750 0x7ff42c0487d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50064/0 (socket says 172.21.15.40:50064) 2024-09-13T17:53:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff438ebc700 1 -- 172.21.15.40:0/468586852 learned_addr learned my addr 172.21.15.40:0/468586852 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff4396bd700 1 --2- 172.21.15.40:0/468586852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff42c0509b0 0x7ff42c175620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff438ebc700 1 -- 172.21.15.40:0/468586852 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff42c0dc4d0 msgr2=0x7ff42c04e2f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:53:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff438ebc700 1 --2- 172.21.15.40:0/468586852 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff42c0dc4d0 0x7ff42c04e2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff438ebc700 1 -- 172.21.15.40:0/468586852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff42c0509b0 msgr2=0x7ff42c175620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff438ebc700 1 --2- 172.21.15.40:0/468586852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff42c0509b0 0x7ff42c175620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff438ebc700 1 -- 172.21.15.40:0/468586852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff424009570 con 0x7ff42c0d6750 2024-09-13T17:53:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff438ebc700 1 --2- 172.21.15.40:0/468586852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff42c0d6750 0x7ff42c0487d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff424009c60 tx=0x7ff424004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff429ffb700 1 -- 172.21.15.40:0/468586852 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff424012a00 con 0x7ff42c0d6750 2024-09-13T17:53:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff429ffb700 1 -- 172.21.15.40:0/468586852 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42401e070 con 0x7ff42c0d6750 2024-09-13T17:53:21.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff43a6bf700 1 -- 172.21.15.40:0/468586852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42c166770 con 0x7ff42c0d6750 2024-09-13T17:53:21.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff429ffb700 1 -- 172.21.15.40:0/468586852 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff424012d60 con 0x7ff42c0d6750 2024-09-13T17:53:21.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.870+0000 7ff43a6bf700 1 -- 172.21.15.40:0/468586852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff42c047480 con 0x7ff42c0d6750 2024-09-13T17:53:21.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.874+0000 7ff43a6bf700 1 -- 172.21.15.40:0/468586852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff42c047000 con 0x7ff42c0d6750 2024-09-13T17:53:21.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.874+0000 7ff429ffb700 1 -- 172.21.15.40:0/468586852 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff42402d450 con 0x7ff42c0d6750 2024-09-13T17:53:21.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.874+0000 7ff429ffb700 1 --2- 172.21.15.40:0/468586852 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff4180666e0 0x7ff418068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:21.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.874+0000 7ff429ffb700 1 -- 172.21.15.40:0/468586852 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff4240856a0 con 0x7ff42c0d6750 2024-09-13T17:53:21.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.874+0000 7ff4396bd700 1 --2- 172.21.15.40:0/468586852 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff4180666e0 0x7ff418068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:21.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.874+0000 7ff4396bd700 1 --2- 172.21.15.40:0/468586852 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff4180666e0 0x7ff418068ba0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff430000f60 tx=0x7ff430009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:21.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:21.882+0000 7ff429ffb700 1 -- 172.21.15.40:0/468586852 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff42c047000 con 0x7ff42c0d6750 2024-09-13T17:53:22.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:22.118+0000 7ff43a6bf700 1 -- 172.21.15.40:0/468586852 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff42c047000 con 0x7ff42c0d6750 2024-09-13T17:53:22.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:22.118+0000 7ff429ffb700 1 -- 172.21.15.40:0/468586852 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7ff42c047000 con 0x7ff42c0d6750 2024-09-13T17:53:22.127 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":24,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:53:22.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:22.122+0000 7ff43a6bf700 1 -- 172.21.15.40:0/468586852 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff4180666e0 msgr2=0x7ff418068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:22.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:22.122+0000 7ff43a6bf700 1 --2- 172.21.15.40:0/468586852 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff4180666e0 0x7ff418068ba0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff430000f60 tx=0x7ff430009040 comp rx=0 tx=0).stop 2024-09-13T17:53:22.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:22.122+0000 7ff43a6bf700 1 -- 172.21.15.40:0/468586852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff42c0d6750 msgr2=0x7ff42c0487d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:22.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:22.122+0000 7ff43a6bf700 1 --2- 172.21.15.40:0/468586852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff42c0d6750 0x7ff42c0487d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff424009c60 tx=0x7ff424004770 comp rx=0 tx=0).stop 2024-09-13T17:53:22.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:22.122+0000 7ff43a6bf700 1 -- 172.21.15.40:0/468586852 shutdown_connections 2024-09-13T17:53:22.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:22.122+0000 7ff43a6bf700 1 --2- 172.21.15.40:0/468586852 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff4180666e0 0x7ff418068ba0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:22.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:22.122+0000 7ff43a6bf700 1 --2- 172.21.15.40:0/468586852 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff42c0dc4d0 0x7ff42c04e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:22.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:22.122+0000 7ff43a6bf700 1 --2- 172.21.15.40:0/468586852 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff42c0d6750 0x7ff42c0487d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:22.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:22.122+0000 7ff43a6bf700 1 --2- 172.21.15.40:0/468586852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff42c0509b0 0x7ff42c175620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:22.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:22.122+0000 7ff43a6bf700 1 -- 172.21.15.40:0/468586852 >> 172.21.15.40:0/468586852 conn(0x7ff42c04fd00 msgr2=0x7ff42c0d4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:22.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:22.122+0000 7ff43a6bf700 1 -- 172.21.15.40:0/468586852 shutdown_connections 2024-09-13T17:53:22.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:22.122+0000 7ff43a6bf700 1 -- 172.21.15.40:0/468586852 wait complete. 2024-09-13T17:53:22.210 INFO:tasks.workunit.client.0.smithi040.stdout:checking fcntl.h presence... yes 2024-09-13T17:53:22.228 INFO:tasks.workunit.client.0.smithi040.stdout:checking for fcntl.h... yes 2024-09-13T17:53:22.618 INFO:tasks.workunit.client.0.smithi040.stdout:checking limits.h usability... yes 2024-09-13T17:53:23.681 INFO:tasks.workunit.client.0.smithi040.stdout:checking limits.h presence... yes 2024-09-13T17:53:23.708 INFO:tasks.workunit.client.0.smithi040.stdout:checking for limits.h... yes 2024-09-13T17:53:23.782 INFO:tasks.workunit.client.0.smithi040.stdout:checking for stdint.h... (cached) yes 2024-09-13T17:53:24.015 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/time.h usability... yes 2024-09-13T17:53:24.145 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/time.h presence... yes 2024-09-13T17:53:24.170 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sys/time.h... yes 2024-09-13T17:53:24.231 INFO:tasks.workunit.client.0.smithi040.stdout:checking for unistd.h... (cached) yes 2024-09-13T17:53:24.453 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/vfs.h usability... yes 2024-09-13T17:53:24.885 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/vfs.h presence... yes 2024-09-13T17:53:24.960 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sys/vfs.h... yes 2024-09-13T17:53:25.142 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:53:25.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.262+0000 7f1daae18700 1 Processor -- start 2024-09-13T17:53:25.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.262+0000 7f1daae18700 1 -- start start 2024-09-13T17:53:25.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.262+0000 7f1daae18700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 0x7f1da415acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:25.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.262+0000 7f1daae18700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 0x7f1da415b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:25.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.262+0000 7f1daae18700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da415a840 con 0x7f1da41512c0 2024-09-13T17:53:25.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.262+0000 7f1daae18700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da41516a0 con 0x7f1da40b2d00 2024-09-13T17:53:25.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.262+0000 7f1daae18700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da41580a0 con 0x7f1da4070120 2024-09-13T17:53:25.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.262+0000 7f1da9615700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 0x7f1da415acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:25.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.262+0000 7f1da9615700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 0x7f1da415acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50076/0 (socket says 172.21.15.40:50076) 2024-09-13T17:53:25.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.262+0000 7f1da9615700 1 -- 172.21.15.40:0/434251568 learned_addr learned my addr 172.21.15.40:0/434251568 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:25.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.262+0000 7f1da9615700 1 -- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 msgr2=0x7f1da415acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:53:25.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.262+0000 7f1da9615700 1 -- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 msgr2=0x7f1da415acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:53:25.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.262+0000 7f1da9615700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 0x7f1da415acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:25.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.262+0000 7f1daa617700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 0x7f1da415b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:25.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.262+0000 7f1da9615700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 0x7f1da415acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:25.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.262+0000 7f1daa617700 1 -- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 msgr2=0x7f1da415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:53:25.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.262+0000 7f1daa617700 1 -- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 msgr2=0x7f1da415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:53:25.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.262+0000 7f1daa617700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 0x7f1da415b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:25.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.262+0000 7f1daa617700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 0x7f1da415b210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.466+0000 7f1da9615700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 0x7f1da415acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.466+0000 7f1daa617700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 0x7f1da415b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.466+0000 7f1da9615700 1 -- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 msgr2=0x7f1da415acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:53:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.466+0000 7f1da9615700 1 -- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 msgr2=0x7f1da415acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:53:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.466+0000 7f1da9615700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 0x7f1da415acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.466+0000 7f1da9615700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 0x7f1da415acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:53:25.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.466+0000 7f1daa617700 1 -- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 msgr2=0x7f1da415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:53:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.466+0000 7f1daa617700 1 -- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 msgr2=0x7f1da415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:53:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.466+0000 7f1daa617700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 0x7f1da415b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:25.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.466+0000 7f1daa617700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 0x7f1da415b210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:53:25.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.870+0000 7f1da9615700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 0x7f1da415acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:25.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.870+0000 7f1daa617700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 0x7f1da415b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:25.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.870+0000 7f1da9615700 1 -- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 msgr2=0x7f1da415acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:53:25.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.870+0000 7f1da9615700 1 -- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 msgr2=0x7f1da415acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:53:25.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.870+0000 7f1da9615700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 0x7f1da415acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:25.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.870+0000 7f1daa617700 1 -- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 msgr2=0x7f1da415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:53:25.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.870+0000 7f1daa617700 1 -- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 msgr2=0x7f1da415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:53:25.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.870+0000 7f1daa617700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 0x7f1da415b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:25.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.870+0000 7f1da9615700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 0x7f1da415acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:53:25.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:25.870+0000 7f1daa617700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 0x7f1da415b210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:53:26.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:26.674+0000 7f1da9615700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 0x7f1da415acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:26.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:26.674+0000 7f1daa617700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 0x7f1da415b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:26.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:26.674+0000 7f1da9615700 1 -- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 msgr2=0x7f1da415acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:53:26.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:26.674+0000 7f1da9615700 1 -- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 msgr2=0x7f1da415acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:53:26.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:26.674+0000 7f1da9615700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 0x7f1da415acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:26.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:26.674+0000 7f1daa617700 1 -- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 msgr2=0x7f1da415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:53:26.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:26.674+0000 7f1daa617700 1 -- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 msgr2=0x7f1da415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:53:26.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:26.674+0000 7f1daa617700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 0x7f1da415b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:26.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:26.674+0000 7f1da9615700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 0x7f1da415acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:53:26.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:26.674+0000 7f1daa617700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 0x7f1da415b210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:53:28.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:28.266+0000 7f1d9b7fe700 1 -- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 msgr2=0x7f1da415b210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:53:28.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:28.266+0000 7f1d9b7fe700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.43:3300/0 conn(0x7f1da4070120 0x7f1da415b210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:28.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:28.266+0000 7f1d9b7fe700 1 -- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 msgr2=0x7f1da415acd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:53:28.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:28.266+0000 7f1d9b7fe700 1 --2- 172.21.15.40:0/434251568 >> v2:172.21.15.40:3300/0 conn(0x7f1da40b2d00 0x7f1da415acd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:28.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:28.266+0000 7f1d9b7fe700 1 -- 172.21.15.40:0/434251568 >> v1:172.21.15.43:6789/0 conn(0x7f1da41512c0 legacy=0x7f1da40c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:28.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:28.266+0000 7f1d9b7fe700 1 --2- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d88007340 0x7f1d880097e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:28.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:28.266+0000 7f1d9b7fe700 1 -- 172.21.15.40:0/434251568 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da4157520 con 0x7f1d88003710 2024-09-13T17:53:28.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:28.266+0000 7f1d9b7fe700 1 -- 172.21.15.40:0/434251568 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da41516a0 con 0x7f1d88007340 2024-09-13T17:53:28.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:28.266+0000 7f1da9615700 1 -- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d88007340 msgr2=0x7f1d880097e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:53:28.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:28.266+0000 7f1da9615700 1 --2- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d88007340 0x7f1d880097e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:28.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:28.470+0000 7f1da9615700 1 -- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d88007340 msgr2=0x7f1d880097e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:53:28.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:28.470+0000 7f1da9615700 1 --2- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d88007340 0x7f1d880097e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:53:28.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:28.874+0000 7f1da9615700 1 -- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d88007340 msgr2=0x7f1d880097e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:53:28.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:28.874+0000 7f1da9615700 1 --2- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d88007340 0x7f1d880097e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:53:29.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.166+0000 7f1da8e14700 1 -- 172.21.15.40:0/434251568 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1743104119 0 0) 0x7f1da4157520 con 0x7f1d88003710 2024-09-13T17:53:29.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.166+0000 7f1da8e14700 1 -- 172.21.15.40:0/434251568 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d900035d0 con 0x7f1d88003710 2024-09-13T17:53:29.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.166+0000 7f1da8e14700 1 -- 172.21.15.40:0/434251568 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3895302273 0 0) 0x7f1d900035d0 con 0x7f1d88003710 2024-09-13T17:53:29.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.166+0000 7f1da8e14700 1 -- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d88007340 msgr2=0x7f1d880097e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:53:29.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.166+0000 7f1da8e14700 1 --2- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d88007340 0x7f1d880097e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.170+0000 7f1da8e14700 1 -- 172.21.15.40:0/434251568 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8800afa0 con 0x7f1d88003710 2024-09-13T17:53:29.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.170+0000 7f1da8e14700 1 -- 172.21.15.40:0/434251568 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f1da00016b0 con 0x7f1d88003710 2024-09-13T17:53:29.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.170+0000 7f1da8e14700 1 -- 172.21.15.40:0/434251568 >> v1:172.21.15.40:6789/0 conn(0x7f1d88003710 legacy=0x7f1d88005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:29.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.170+0000 7f1daa617700 1 -- 172.21.15.40:0/434251568 reap_dead start 2024-09-13T17:53:29.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.170+0000 7f1da8e14700 1 --2- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d90003a20 0x7f1d90003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.170+0000 7f1da8e14700 1 --2- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d90004870 0x7f1d90004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.170+0000 7f1da8e14700 1 --2- 172.21.15.40:0/434251568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d90005350 0x7f1d9000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.170+0000 7f1da8e14700 1 -- 172.21.15.40:0/434251568 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da41516a0 con 0x7f1d90005350 2024-09-13T17:53:29.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.170+0000 7f1da8e14700 1 -- 172.21.15.40:0/434251568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9000c2a0 con 0x7f1d90003a20 2024-09-13T17:53:29.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.170+0000 7f1da8e14700 1 -- 172.21.15.40:0/434251568 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9000c460 con 0x7f1d90004870 2024-09-13T17:53:29.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.170+0000 7f1da9e16700 1 --2- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d90003a20 0x7f1d90003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:29.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.170+0000 7f1da9615700 1 -- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d90004870 msgr2=0x7f1d90004cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:53:29.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.170+0000 7f1da9615700 1 --2- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d90004870 0x7f1d90004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:29.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.170+0000 7f1da9e16700 1 -- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d90004870 msgr2=0x7f1d90004cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:53:29.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.170+0000 7f1da9e16700 1 --2- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d90004870 0x7f1d90004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.170+0000 7f1da9e16700 1 -- 172.21.15.40:0/434251568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d90005350 msgr2=0x7f1d9000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:53:29.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.170+0000 7f1da9e16700 1 --2- 172.21.15.40:0/434251568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d90005350 0x7f1d9000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.170+0000 7f1da9e16700 1 -- 172.21.15.40:0/434251568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d900035d0 con 0x7f1d90003a20 2024-09-13T17:53:29.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.170+0000 7f1da9e16700 1 --2- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d90003a20 0x7f1d90003e20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1da000c0e0 tx=0x7f1d900058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19c1e0dea81d7989 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:29.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.178+0000 7f1da8e14700 1 -- 172.21.15.40:0/434251568 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0007ae0 con 0x7f1d90003a20 2024-09-13T17:53:29.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.178+0000 7f1da8e14700 1 -- 172.21.15.40:0/434251568 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da000cc80 con 0x7f1d90003a20 2024-09-13T17:53:29.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.178+0000 7f1da8e14700 1 -- 172.21.15.40:0/434251568 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0012420 con 0x7f1d90003a20 2024-09-13T17:53:29.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1daae18700 1 -- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d90003a20 msgr2=0x7f1d90003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:29.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1daae18700 1 --2- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d90003a20 0x7f1d90003e20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1da000c0e0 tx=0x7f1d900058a0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1daae18700 1 -- 172.21.15.40:0/434251568 shutdown_connections 2024-09-13T17:53:29.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1daae18700 1 --2- 172.21.15.40:0/434251568 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d90005350 0x7f1d9000ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1daae18700 1 --2- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d90004870 0x7f1d90004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1daae18700 1 --2- 172.21.15.40:0/434251568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d90003a20 0x7f1d90003e20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1daae18700 1 -- 172.21.15.40:0/434251568 >> 172.21.15.40:0/434251568 conn(0x7f1da40ccef0 msgr2=0x7f1da40cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:29.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1daae18700 1 -- 172.21.15.40:0/434251568 shutdown_connections 2024-09-13T17:53:29.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1daae18700 1 -- 172.21.15.40:0/434251568 wait complete. 2024-09-13T17:53:29.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1daae18700 1 Processor -- start 2024-09-13T17:53:29.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1daae18700 1 -- start start 2024-09-13T17:53:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1daae18700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d90003a20 0x7f1da406b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1daae18700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d90004870 0x7f1da40cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1daae18700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d90005350 0x7f1da41447a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1daae18700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da41f3270 con 0x7f1d90003a20 2024-09-13T17:53:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1daae18700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da406b9e0 con 0x7f1d90005350 2024-09-13T17:53:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1daae18700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da41f2d10 con 0x7f1d90004870 2024-09-13T17:53:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1da9615700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d90004870 msgr2=0x7f1da40cc630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:53:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1da9e16700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d90003a20 0x7f1da406b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1da9615700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d90004870 0x7f1da40cc630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1da9e16700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d90003a20 0x7f1da406b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50120/0 (socket says 172.21.15.40:50120) 2024-09-13T17:53:29.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1da9e16700 1 -- 172.21.15.40:0/3523333134 learned_addr learned my addr 172.21.15.40:0/3523333134 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:29.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.182+0000 7f1daa617700 1 --2- 172.21.15.40:0/3523333134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d90005350 0x7f1da41447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:29.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.186+0000 7f1da9e16700 1 -- 172.21.15.40:0/3523333134 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d90004870 msgr2=0x7f1da40cc630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:53:29.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.186+0000 7f1da9e16700 1 --2- 172.21.15.40:0/3523333134 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d90004870 0x7f1da40cc630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.186+0000 7f1da9e16700 1 -- 172.21.15.40:0/3523333134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d90005350 msgr2=0x7f1da41447a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:29.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.186+0000 7f1da9e16700 1 --2- 172.21.15.40:0/3523333134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d90005350 0x7f1da41447a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.186+0000 7f1da9e16700 1 -- 172.21.15.40:0/3523333134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da41f2f90 con 0x7f1d90003a20 2024-09-13T17:53:29.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.186+0000 7f1da9e16700 1 --2- 172.21.15.40:0/3523333134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d90003a20 0x7f1da406b4a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000c430 tx=0x7f1da0001960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:29.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.186+0000 7f1d9a7fc700 1 -- 172.21.15.40:0/3523333134 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0026450 con 0x7f1d90003a20 2024-09-13T17:53:29.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.186+0000 7f1d9a7fc700 1 -- 172.21.15.40:0/3523333134 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da00265b0 con 0x7f1d90003a20 2024-09-13T17:53:29.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.186+0000 7f1d9a7fc700 1 -- 172.21.15.40:0/3523333134 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0026940 con 0x7f1d90003a20 2024-09-13T17:53:29.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.186+0000 7f1daae18700 1 -- 172.21.15.40:0/3523333134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da41eef60 con 0x7f1d90003a20 2024-09-13T17:53:29.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.186+0000 7f1daae18700 1 -- 172.21.15.40:0/3523333134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da41ef520 con 0x7f1d90003a20 2024-09-13T17:53:29.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.186+0000 7f1daae18700 1 -- 172.21.15.40:0/3523333134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da41eedc0 con 0x7f1d90003a20 2024-09-13T17:53:29.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.186+0000 7f1d9a7fc700 1 -- 172.21.15.40:0/3523333134 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1da0026aa0 con 0x7f1d90003a20 2024-09-13T17:53:29.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.186+0000 7f1d9a7fc700 1 --2- 172.21.15.40:0/3523333134 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d88066f70 0x7f1d88069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.186+0000 7f1d9a7fc700 1 -- 172.21.15.40:0/3523333134 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1da00856a0 con 0x7f1d90003a20 2024-09-13T17:53:29.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.186+0000 7f1da9615700 1 --2- 172.21.15.40:0/3523333134 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d88066f70 0x7f1d88069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:29.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.190+0000 7f1da9615700 1 --2- 172.21.15.40:0/3523333134 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d88066f70 0x7f1d88069430 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1da415a4a0 tx=0x7f1d94008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:29.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.194+0000 7f1d9a7fc700 1 -- 172.21.15.40:0/3523333134 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1da41eedc0 con 0x7f1d90003a20 2024-09-13T17:53:29.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.426+0000 7f1daae18700 1 -- 172.21.15.40:0/3523333134 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1da41eedc0 con 0x7f1d90003a20 2024-09-13T17:53:29.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.426+0000 7f1d9a7fc700 1 -- 172.21.15.40:0/3523333134 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7f1da41eedc0 con 0x7f1d90003a20 2024-09-13T17:53:29.434 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":28,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:53:29.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.430+0000 7f1daae18700 1 -- 172.21.15.40:0/3523333134 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d88066f70 msgr2=0x7f1d88069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:29.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.430+0000 7f1daae18700 1 --2- 172.21.15.40:0/3523333134 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d88066f70 0x7f1d88069430 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1da415a4a0 tx=0x7f1d94008040 comp rx=0 tx=0).stop 2024-09-13T17:53:29.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.430+0000 7f1daae18700 1 -- 172.21.15.40:0/3523333134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d90003a20 msgr2=0x7f1da406b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:29.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.430+0000 7f1daae18700 1 --2- 172.21.15.40:0/3523333134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d90003a20 0x7f1da406b4a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000c430 tx=0x7f1da0001960 comp rx=0 tx=0).stop 2024-09-13T17:53:29.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.430+0000 7f1daae18700 1 -- 172.21.15.40:0/3523333134 shutdown_connections 2024-09-13T17:53:29.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.430+0000 7f1daae18700 1 --2- 172.21.15.40:0/3523333134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d90005350 0x7f1da41447a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.430+0000 7f1daae18700 1 --2- 172.21.15.40:0/3523333134 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d88066f70 0x7f1d88069430 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.430+0000 7f1daae18700 1 --2- 172.21.15.40:0/3523333134 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d90004870 0x7f1da40cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.430+0000 7f1daae18700 1 --2- 172.21.15.40:0/3523333134 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d90003a20 0x7f1da406b4a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.430+0000 7f1daae18700 1 -- 172.21.15.40:0/3523333134 >> 172.21.15.40:0/3523333134 conn(0x7f1da40ccef0 msgr2=0x7f1da40c8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:29.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.430+0000 7f1daae18700 1 -- 172.21.15.40:0/3523333134 shutdown_connections 2024-09-13T17:53:29.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.430+0000 7f1daae18700 1 -- 172.21.15.40:0/3523333134 wait complete. 2024-09-13T17:53:29.449 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T17:53:29.449 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-13T17:53:29.449 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:53:29.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.554+0000 7f8babfff700 1 Processor -- start 2024-09-13T17:53:29.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.554+0000 7f8babfff700 1 -- start start 2024-09-13T17:53:29.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8babfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8bac0cf1a0 0x7f8bac15a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8babfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8bac0cf580 0x7f8bac0c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8babfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8bac0cc250 0x7f8bac0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8babfff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bac15a7a0 con 0x7f8bac0cc250 2024-09-13T17:53:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8babfff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bac0cbdc0 con 0x7f8bac0cf1a0 2024-09-13T17:53:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8babfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac153f80 con 0x7f8bac0cf580 2024-09-13T17:53:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8baa7fc700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8bac0cf580 msgr2=0x7f8bac0c9fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:53:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8baa7fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8bac0cf580 0x7f8bac0c9fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8bab7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8bac0cc250 0x7f8bac0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8bab7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8bac0cc250 0x7f8bac0c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50136/0 (socket says 172.21.15.40:50136) 2024-09-13T17:53:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8bab7fe700 1 -- 172.21.15.40:0/334218952 learned_addr learned my addr 172.21.15.40:0/334218952 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8baaffd700 1 --2- 172.21.15.40:0/334218952 >> v2:172.21.15.43:3300/0 conn(0x7f8bac0cf1a0 0x7f8bac15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:29.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8bab7fe700 1 -- 172.21.15.40:0/334218952 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8bac0cf580 msgr2=0x7f8bac0c9fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:53:29.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8bab7fe700 1 --2- 172.21.15.40:0/334218952 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8bac0cf580 0x7f8bac0c9fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8bab7fe700 1 -- 172.21.15.40:0/334218952 >> v2:172.21.15.43:3300/0 conn(0x7f8bac0cf1a0 msgr2=0x7f8bac15a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:53:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8bab7fe700 1 --2- 172.21.15.40:0/334218952 >> v2:172.21.15.43:3300/0 conn(0x7f8bac0cf1a0 0x7f8bac15a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8bab7fe700 1 -- 172.21.15.40:0/334218952 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bac0caed0 con 0x7f8bac0cc250 2024-09-13T17:53:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8bab7fe700 1 --2- 172.21.15.40:0/334218952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0cc250 0x7f8bac0c8aa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8bac067020 tx=0x7f8ba400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f557f3c36d2d4ecf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8ba9ffb700 1 -- 172.21.15.40:0/334218952 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba400e070 con 0x7f8bac0cc250 2024-09-13T17:53:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8ba9ffb700 1 -- 172.21.15.40:0/334218952 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba400b910 con 0x7f8bac0cc250 2024-09-13T17:53:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8ba9ffb700 1 -- 172.21.15.40:0/334218952 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba400bcc0 con 0x7f8bac0cc250 2024-09-13T17:53:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8babfff700 1 -- 172.21.15.40:0/334218952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0cc250 msgr2=0x7f8bac0c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8babfff700 1 --2- 172.21.15.40:0/334218952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0cc250 0x7f8bac0c8aa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8bac067020 tx=0x7f8ba400b330 comp rx=0 tx=0).stop 2024-09-13T17:53:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8babfff700 1 -- 172.21.15.40:0/334218952 shutdown_connections 2024-09-13T17:53:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8babfff700 1 --2- 172.21.15.40:0/334218952 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0cc250 0x7f8bac0c8aa0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8babfff700 1 --2- 172.21.15.40:0/334218952 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8bac0cf580 0x7f8bac0c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8babfff700 1 --2- 172.21.15.40:0/334218952 >> v2:172.21.15.43:3300/0 conn(0x7f8bac0cf1a0 0x7f8bac15a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8babfff700 1 -- 172.21.15.40:0/334218952 >> 172.21.15.40:0/334218952 conn(0x7f8bac066090 msgr2=0x7f8bac066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:29.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8babfff700 1 -- 172.21.15.40:0/334218952 shutdown_connections 2024-09-13T17:53:29.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.558+0000 7f8babfff700 1 -- 172.21.15.40:0/334218952 wait complete. 2024-09-13T17:53:29.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8babfff700 1 Processor -- start 2024-09-13T17:53:29.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8babfff700 1 -- start start 2024-09-13T17:53:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8babfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0cc250 0x7f8bac0c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8babfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bac0cf1a0 0x7f8bac0ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8babfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8bac0cf580 0x7f8bac155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8babfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac0c6dd0 con 0x7f8bac0cc250 2024-09-13T17:53:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8babfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac0c7f10 con 0x7f8bac0cf1a0 2024-09-13T17:53:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8babfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac1ee870 con 0x7f8bac0cf580 2024-09-13T17:53:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8bab7fe700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8bac0cf580 msgr2=0x7f8bac155b40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:53:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8baaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0cc250 0x7f8bac0c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8bab7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8bac0cf580 0x7f8bac155b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8baaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0cc250 0x7f8bac0c81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50138/0 (socket says 172.21.15.40:50138) 2024-09-13T17:53:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8baaffd700 1 -- 172.21.15.40:0/2609328250 learned_addr learned my addr 172.21.15.40:0/2609328250 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8baa7fc700 1 --2- 172.21.15.40:0/2609328250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bac0cf1a0 0x7f8bac0ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8baaffd700 1 -- 172.21.15.40:0/2609328250 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8bac0cf580 msgr2=0x7f8bac155b40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:53:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8baaffd700 1 --2- 172.21.15.40:0/2609328250 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8bac0cf580 0x7f8bac155b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8baaffd700 1 -- 172.21.15.40:0/2609328250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bac0cf1a0 msgr2=0x7f8bac0ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8baaffd700 1 --2- 172.21.15.40:0/2609328250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bac0cf1a0 0x7f8bac0ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8baaffd700 1 -- 172.21.15.40:0/2609328250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba4009b30 con 0x7f8bac0cc250 2024-09-13T17:53:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8baaffd700 1 --2- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0cc250 0x7f8bac0c81b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00e9d0 tx=0x7f8b9c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8ba37fe700 1 -- 172.21.15.40:0/2609328250 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c009720 con 0x7f8bac0cc250 2024-09-13T17:53:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8ba37fe700 1 -- 172.21.15.40:0/2609328250 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b9c00c850 con 0x7f8bac0cc250 2024-09-13T17:53:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.562+0000 7f8babfff700 1 -- 172.21.15.40:0/2609328250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bac155340 con 0x7f8bac0cc250 2024-09-13T17:53:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.566+0000 7f8ba37fe700 1 -- 172.21.15.40:0/2609328250 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c00cbc0 con 0x7f8bac0cc250 2024-09-13T17:53:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.566+0000 7f8babfff700 1 -- 172.21.15.40:0/2609328250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bac0ce6f0 con 0x7f8bac0cc250 2024-09-13T17:53:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.566+0000 7f8babfff700 1 --2- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac1508c0 0x7f8bac156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.566+0000 7f8babfff700 1 -- 172.21.15.40:0/2609328250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac074750 con 0x7f8bac1508c0 2024-09-13T17:53:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.566+0000 7f8baa7fc700 1 --2- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac1508c0 0x7f8bac156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.566+0000 7f8baa7fc700 1 -- 172.21.15.40:0/2609328250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8bac0cb770 con 0x7f8bac1508c0 2024-09-13T17:53:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.566+0000 7f8baa7fc700 1 --2- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac1508c0 0x7f8bac156080 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0cc830 tx=0x7f8b94009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.566+0000 7f8ba37fe700 1 -- 172.21.15.40:0/2609328250 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b94012600 con 0x7f8bac1508c0 2024-09-13T17:53:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.566+0000 7f8ba37fe700 1 -- 172.21.15.40:0/2609328250 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8b94012760 con 0x7f8bac1508c0 2024-09-13T17:53:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.566+0000 7f8ba37fe700 1 -- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac1508c0 msgr2=0x7f8bac156080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.566+0000 7f8ba37fe700 1 --2- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac1508c0 0x7f8bac156080 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0cc830 tx=0x7f8b94009ab0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.566+0000 7f8ba37fe700 1 -- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac1508c0 msgr2=0x7f8bac156080 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:53:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.566+0000 7f8ba37fe700 1 --2- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac1508c0 0x7f8bac156080 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0cc830 tx=0x7f8b94009ab0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.566+0000 7f8ba37fe700 1 -- 172.21.15.40:0/2609328250 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8b9c00cd20 con 0x7f8bac0cc250 2024-09-13T17:53:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.566+0000 7f8ba37fe700 1 --2- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8b8c0664b0 0x7f8b8c068970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.566+0000 7f8ba37fe700 1 -- 172.21.15.40:0/2609328250 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8b9c0845a0 con 0x7f8bac0cc250 2024-09-13T17:53:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.566+0000 7f8baa7fc700 1 --2- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8b8c0664b0 0x7f8b8c068970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.570+0000 7f8baa7fc700 1 --2- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8b8c0664b0 0x7f8b8c068970 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0cc830 tx=0x7f8b94008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.578+0000 7f8babfff700 1 --2- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac14d140 0x7f8bac14d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.578+0000 7f8babfff700 1 -- 172.21.15.40:0/2609328250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac150420 con 0x7f8bac14d140 2024-09-13T17:53:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.578+0000 7f8bab7fe700 1 --2- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac14d140 0x7f8bac14d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:29.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.578+0000 7f8bab7fe700 1 -- 172.21.15.40:0/2609328250 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8bac0cb490 con 0x7f8bac14d140 2024-09-13T17:53:29.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.578+0000 7f8bab7fe700 1 --2- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac14d140 0x7f8bac14d550 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8bac067020 tx=0x7f8ba4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:29.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.578+0000 7f8ba37fe700 1 -- 172.21.15.40:0/2609328250 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba40099b0 con 0x7f8bac14d140 2024-09-13T17:53:29.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.578+0000 7f8ba37fe700 1 -- 172.21.15.40:0/2609328250 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4606 (secure 0 0 0) 0x7f8ba40092e0 con 0x7f8bac14d140 2024-09-13T17:53:29.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.578+0000 7f8ba37fe700 1 -- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac14d140 msgr2=0x7f8bac14d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:29.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.578+0000 7f8ba37fe700 1 --2- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac14d140 0x7f8bac14d550 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8bac067020 tx=0x7f8ba4004af0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.578+0000 7f8ba37fe700 1 -- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac14d140 msgr2=0x7f8bac14d550 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:53:29.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.578+0000 7f8ba37fe700 1 --2- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac14d140 0x7f8bac14d550 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.587 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:53:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:53:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:53:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:53:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 28, 2024-09-13T17:53:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:53:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:53:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: 1 2024-09-13T17:53:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T17:53:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:53:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:53:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:53:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:53:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:53:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:53:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:53:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:53:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:53:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:53:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:53:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:53:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:53:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:53:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:53:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:53:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:53:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:53:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:53:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:53:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:53:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:53:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:53:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:53:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:53:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:53:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:53:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:53:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:53:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:53:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:53:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:53:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:53:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:53:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:53:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:53:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:53:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:53:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:53:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:53:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:53:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:53:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:53:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:53:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:53:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:53:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.599 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:29.599 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:29.599 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:53:29.599 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:29.599 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:29.599 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:53:29.599 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.599 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:29.599 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:29.599 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:53:29.599 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:29.599 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:29.599 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.599 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:53:29.600 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:53:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.582+0000 7f8babfff700 1 -- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8b8c0664b0 msgr2=0x7f8b8c068970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.582+0000 7f8babfff700 1 --2- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8b8c0664b0 0x7f8b8c068970 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0cc830 tx=0x7f8b94008030 comp rx=0 tx=0).stop 2024-09-13T17:53:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.582+0000 7f8babfff700 1 -- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0cc250 msgr2=0x7f8bac0c81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.582+0000 7f8babfff700 1 --2- 172.21.15.40:0/2609328250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bac0cc250 0x7f8bac0c81b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00e9d0 tx=0x7f8b9c00ece0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.582+0000 7f8bab7fe700 1 -- 172.21.15.40:0/2609328250 reap_dead start 2024-09-13T17:53:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.582+0000 7f8bab7fe700 1 -- 172.21.15.40:0/2609328250 reap_dead start 2024-09-13T17:53:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.582+0000 7f8babfff700 1 -- 172.21.15.40:0/2609328250 shutdown_connections 2024-09-13T17:53:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.582+0000 7f8babfff700 1 -- 172.21.15.40:0/2609328250 >> 172.21.15.40:0/2609328250 conn(0x7f8bac066090 msgr2=0x7f8bac066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:29.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.582+0000 7f8babfff700 1 -- 172.21.15.40:0/2609328250 shutdown_connections 2024-09-13T17:53:29.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.582+0000 7f8babfff700 1 -- 172.21.15.40:0/2609328250 wait complete. 2024-09-13T17:53:29.602 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T17:53:29.602 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-13T17:53:29.602 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:53:29.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff96dca5700 1 Processor -- start 2024-09-13T17:53:29.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff96dca5700 1 -- start start 2024-09-13T17:53:29.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff96dca5700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff9680c8050 0x7ff9681562e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff96dca5700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff9680c8430 0x7ff9680c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff96dca5700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff968159f40 con 0x7ff968070a60 2024-09-13T17:53:29.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff96dca5700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff968156820 con 0x7ff9680c8430 2024-09-13T17:53:29.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff96dca5700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9680c6ee0 con 0x7ff9680c8050 2024-09-13T17:53:29.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff967fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff9680c8430 0x7ff9680c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:29.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff967fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff9680c8430 0x7ff9680c66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50172/0 (socket says 172.21.15.40:50172) 2024-09-13T17:53:29.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff967fff700 1 -- 172.21.15.40:0/1938113986 learned_addr learned my addr 172.21.15.40:0/1938113986 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:29.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff96cca3700 1 --2- 172.21.15.40:0/1938113986 >> v2:172.21.15.43:3300/0 conn(0x7ff9680c8050 0x7ff9681562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff967fff700 1 -- 172.21.15.40:0/1938113986 >> v2:172.21.15.43:3300/0 conn(0x7ff9680c8050 msgr2=0x7ff9681562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:53:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff967fff700 1 --2- 172.21.15.40:0/1938113986 >> v2:172.21.15.43:3300/0 conn(0x7ff9680c8050 0x7ff9681562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff967fff700 1 -- 172.21.15.40:0/1938113986 >> v1:172.21.15.40:6789/0 conn(0x7ff968070a60 legacy=0x7ff968150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff967fff700 1 -- 172.21.15.40:0/1938113986 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9680c71c0 con 0x7ff9680c8430 2024-09-13T17:53:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff9677fe700 1 -- 172.21.15.40:0/1938113986 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761964750 0 0) 0x7ff968159f40 con 0x7ff968070a60 2024-09-13T17:53:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff967fff700 1 --2- 172.21.15.40:0/1938113986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680c8430 0x7ff9680c66a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff9680cbe70 tx=0x7ff958009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b78d86d67e66a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff9677fe700 1 -- 172.21.15.40:0/1938113986 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff958011830 con 0x7ff9680c8430 2024-09-13T17:53:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff9677fe700 1 -- 172.21.15.40:0/1938113986 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff958011990 con 0x7ff9680c8430 2024-09-13T17:53:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff9677fe700 1 -- 172.21.15.40:0/1938113986 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff958011d20 con 0x7ff9680c8430 2024-09-13T17:53:29.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff96dca5700 1 -- 172.21.15.40:0/1938113986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680c8430 msgr2=0x7ff9680c66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:29.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff96dca5700 1 --2- 172.21.15.40:0/1938113986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680c8430 0x7ff9680c66a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff9680cbe70 tx=0x7ff958009bf0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff96dca5700 1 -- 172.21.15.40:0/1938113986 shutdown_connections 2024-09-13T17:53:29.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff96dca5700 1 --2- 172.21.15.40:0/1938113986 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680c8430 0x7ff9680c66a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff96dca5700 1 --2- 172.21.15.40:0/1938113986 >> v2:172.21.15.43:3300/0 conn(0x7ff9680c8050 0x7ff9681562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff96dca5700 1 -- 172.21.15.40:0/1938113986 >> 172.21.15.40:0/1938113986 conn(0x7ff968066090 msgr2=0x7ff9680664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff96dca5700 1 -- 172.21.15.40:0/1938113986 shutdown_connections 2024-09-13T17:53:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.714+0000 7ff96dca5700 1 -- 172.21.15.40:0/1938113986 wait complete. 2024-09-13T17:53:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff96dca5700 1 Processor -- start 2024-09-13T17:53:29.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff96dca5700 1 -- start start 2024-09-13T17:53:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff96dca5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff968070a60 0x7ff9680cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff96dca5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680c8050 0x7ff9681f0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff96dca5700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9680c8430 0x7ff9681ee8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff96dca5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff968149b30 con 0x7ff9680c8050 2024-09-13T17:53:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff96dca5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96814a080 con 0x7ff968070a60 2024-09-13T17:53:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff96dca5700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff968149da0 con 0x7ff9680c8430 2024-09-13T17:53:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff967fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680c8050 0x7ff9681f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff96d4a4700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9680c8430 msgr2=0x7ff9681ee8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:53:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff967fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680c8050 0x7ff9681f0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50174/0 (socket says 172.21.15.40:50174) 2024-09-13T17:53:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff967fff700 1 -- 172.21.15.40:0/3286877435 learned_addr learned my addr 172.21.15.40:0/3286877435 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff96d4a4700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9680c8430 0x7ff9681ee8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff96cca3700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff968070a60 0x7ff9680cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff967fff700 1 -- 172.21.15.40:0/3286877435 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9680c8430 msgr2=0x7ff9681ee8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:53:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff967fff700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9680c8430 0x7ff9681ee8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff967fff700 1 -- 172.21.15.40:0/3286877435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff968070a60 msgr2=0x7ff9680cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff967fff700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff968070a60 0x7ff9680cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff967fff700 1 -- 172.21.15.40:0/3286877435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff958009570 con 0x7ff9680c8050 2024-09-13T17:53:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff967fff700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680c8050 0x7ff9681f0310 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff958014040 tx=0x7ff958004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:29.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff9657fa700 1 -- 172.21.15.40:0/3286877435 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff958004550 con 0x7ff9680c8050 2024-09-13T17:53:29.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff9657fa700 1 -- 172.21.15.40:0/3286877435 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95801e070 con 0x7ff9680c8050 2024-09-13T17:53:29.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff96dca5700 1 -- 172.21.15.40:0/3286877435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9681ee0d0 con 0x7ff9680c8050 2024-09-13T17:53:29.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff9657fa700 1 -- 172.21.15.40:0/3286877435 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9580038c0 con 0x7ff9680c8050 2024-09-13T17:53:29.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.718+0000 7ff96dca5700 1 -- 172.21.15.40:0/3286877435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9681f0d90 con 0x7ff9680c8050 2024-09-13T17:53:29.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.722+0000 7ff96dca5700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9681f0850 0x7ff9681eee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.722+0000 7ff96dca5700 1 -- 172.21.15.40:0/3286877435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff968067720 con 0x7ff9681f0850 2024-09-13T17:53:29.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.722+0000 7ff9657fa700 1 -- 172.21.15.40:0/3286877435 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff958003a20 con 0x7ff9680c8050 2024-09-13T17:53:29.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.722+0000 7ff96cca3700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9681f0850 0x7ff9681eee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.722+0000 7ff9657fa700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff9540666e0 0x7ff954068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.722+0000 7ff9657fa700 1 -- 172.21.15.40:0/3286877435 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff958085480 con 0x7ff9680c8050 2024-09-13T17:53:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.722+0000 7ff96d4a4700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff9540666e0 0x7ff954068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.722+0000 7ff96cca3700 1 -- 172.21.15.40:0/3286877435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff9680c6be0 con 0x7ff9681f0850 2024-09-13T17:53:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.722+0000 7ff96d4a4700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff9540666e0 0x7ff954068ba0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff9680c0130 tx=0x7ff960001780 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.722+0000 7ff96cca3700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9681f0850 0x7ff9681eee10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff95c011830 tx=0x7ff95c011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.722+0000 7ff9657fa700 1 -- 172.21.15.40:0/3286877435 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c004230 con 0x7ff9681f0850 2024-09-13T17:53:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.722+0000 7ff9657fa700 1 -- 172.21.15.40:0/3286877435 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff95c004390 con 0x7ff9681f0850 2024-09-13T17:53:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.722+0000 7ff9657fa700 1 -- 172.21.15.40:0/3286877435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9681f0850 msgr2=0x7ff9681eee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:29.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.722+0000 7ff9657fa700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9681f0850 0x7ff9681eee10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff95c011830 tx=0x7ff95c011bf0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.722+0000 7ff9657fa700 1 -- 172.21.15.40:0/3286877435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9681f0850 msgr2=0x7ff9681eee10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:53:29.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.722+0000 7ff9657fa700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9681f0850 0x7ff9681eee10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.730+0000 7ff96dca5700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff968147c90 0x7ff9681480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:29.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.730+0000 7ff96dca5700 1 -- 172.21.15.40:0/3286877435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff968159bd0 con 0x7ff968147c90 2024-09-13T17:53:29.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.730+0000 7ff96cca3700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff968147c90 0x7ff9681480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:29.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.734+0000 7ff96cca3700 1 -- 172.21.15.40:0/3286877435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff9680c74a0 con 0x7ff968147c90 2024-09-13T17:53:29.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.734+0000 7ff96cca3700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff968147c90 0x7ff9681480a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff95c011830 tx=0x7ff95c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:29.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.734+0000 7ff9657fa700 1 -- 172.21.15.40:0/3286877435 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c024070 con 0x7ff968147c90 2024-09-13T17:53:29.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.734+0000 7ff9657fa700 1 -- 172.21.15.40:0/3286877435 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4936 (secure 0 0 0) 0x7ff95c015c60 con 0x7ff968147c90 2024-09-13T17:53:29.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.734+0000 7ff9657fa700 1 -- 172.21.15.40:0/3286877435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff968147c90 msgr2=0x7ff9681480a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:29.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.734+0000 7ff9657fa700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff968147c90 0x7ff9681480a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff95c011830 tx=0x7ff95c004770 comp rx=0 tx=0).stop 2024-09-13T17:53:29.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.734+0000 7ff9657fa700 1 -- 172.21.15.40:0/3286877435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff968147c90 msgr2=0x7ff9681480a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:53:29.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.734+0000 7ff9657fa700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff968147c90 0x7ff9681480a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.742 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:53:29.742 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:53:29.742 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:53:29.742 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:53:29.742 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 28, 2024-09-13T17:53:29.742 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:53:29.742 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:53:29.742 INFO:teuthology.orchestra.run.smithi040.stdout: 1 2024-09-13T17:53:29.742 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:29.742 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T17:53:29.742 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:53:29.742 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:53:29.743 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:53:29.743 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:29.743 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:29.743 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:29.743 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:29.743 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:29.743 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:29.743 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:29.743 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:29.743 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:29.743 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:29.743 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:29.743 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:53:29.743 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:53:29.743 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:29.744 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:29.744 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:29.744 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:29.744 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:29.744 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:29.744 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:29.744 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:29.744 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:29.744 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:29.744 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:29.744 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.744 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:53:29.744 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:53:29.744 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:53:29.745 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:53:29.745 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:53:29.745 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:53:29.745 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:53:29.745 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:53:29.745 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:53:29.745 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:53:29.745 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:53:29.745 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:53:29.745 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:53:29.745 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:53:29.745 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:53:29.745 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:53:29.745 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:53:29.746 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:29.746 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:29.746 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:29.746 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:29.746 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:29.746 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:29.746 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:29.746 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:29.746 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:29.746 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:29.746 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:29.746 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:53:29.746 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.746 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:53:29.746 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:53:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:53:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:53:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:53:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:53:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:53:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:53:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:53:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:53:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:53:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:53:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:53:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:53:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:53:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:53:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:53:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:53:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:53:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:53:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:53:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:53:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:53:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:53:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:53:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:53:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:53:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:53:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:53:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:53:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:53:29.755 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:53:29.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.734+0000 7ff96dca5700 1 -- 172.21.15.40:0/3286877435 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff9540666e0 msgr2=0x7ff954068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:29.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.734+0000 7ff96dca5700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff9540666e0 0x7ff954068ba0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff9680c0130 tx=0x7ff960001780 comp rx=0 tx=0).stop 2024-09-13T17:53:29.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.734+0000 7ff96dca5700 1 -- 172.21.15.40:0/3286877435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680c8050 msgr2=0x7ff9681f0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:29.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.734+0000 7ff96d4a4700 1 -- 172.21.15.40:0/3286877435 reap_dead start 2024-09-13T17:53:29.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.734+0000 7ff96dca5700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680c8050 0x7ff9681f0310 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff958014040 tx=0x7ff958004770 comp rx=0 tx=0).stop 2024-09-13T17:53:29.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.734+0000 7ff96dca5700 1 -- 172.21.15.40:0/3286877435 shutdown_connections 2024-09-13T17:53:29.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.734+0000 7ff96dca5700 1 --2- 172.21.15.40:0/3286877435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9680c8050 0x7ff9681f0310 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:29.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.734+0000 7ff96dca5700 1 -- 172.21.15.40:0/3286877435 >> 172.21.15.40:0/3286877435 conn(0x7ff968066090 msgr2=0x7ff96814dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:29.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.734+0000 7ff96dca5700 1 -- 172.21.15.40:0/3286877435 shutdown_connections 2024-09-13T17:53:29.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:29.734+0000 7ff96dca5700 1 -- 172.21.15.40:0/3286877435 wait complete. 2024-09-13T17:53:29.757 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T17:53:29.757 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T17:53:30.135 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/limits.h usability... no 2024-09-13T17:53:30.300 INFO:tasks.workunit.client.0.smithi040.stdout:checking sys/limits.h presence... no 2024-09-13T17:53:30.329 INFO:tasks.workunit.client.0.smithi040.stdout:checking for sys/limits.h... no 2024-09-13T17:53:30.494 INFO:tasks.workunit.client.0.smithi040.stdout:checking for an ANSI C-conforming const... yes 2024-09-13T17:53:30.906 INFO:tasks.workunit.client.0.smithi040.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-13T17:53:31.839 INFO:tasks.workunit.client.0.smithi040.stdout:checking for system... yes 2024-09-13T17:53:32.117 INFO:tasks.workunit.client.0.smithi040.stdout:checking for gettimeofday... yes 2024-09-13T17:53:32.379 INFO:tasks.workunit.client.0.smithi040.stdout:checking for mkdir... yes 2024-09-13T17:53:33.093 INFO:tasks.workunit.client.0.smithi040.stdout:checking for strerror... yes 2024-09-13T17:53:33.360 INFO:tasks.workunit.client.0.smithi040.stdout:checking for open64... yes 2024-09-13T17:53:33.975 INFO:tasks.workunit.client.0.smithi040.stdout:checking for stat64... yes 2024-09-13T17:53:34.029 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:53:34.031 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:53:34.233 INFO:tasks.workunit.client.0.smithi040.stdout:checking for fseeko64... yes 2024-09-13T17:53:35.506 INFO:tasks.workunit.client.0.smithi040.stdout:checking for lrand48_r... yes 2024-09-13T17:53:36.315 INFO:tasks.workunit.client.0.smithi040.stdout:checking for srand48_r... yes 2024-09-13T17:53:36.407 INFO:tasks.workunit.client.0.smithi040.stdout:configure: creating ./config.status 2024-09-13T17:53:39.276 INFO:tasks.workunit.client.0.smithi040.stdout: /bin/bash ./config.status 2024-09-13T17:53:39.496 INFO:tasks.workunit.client.0.smithi040.stdout:config.status: creating Makefile 2024-09-13T17:53:39.581 INFO:tasks.workunit.client.0.smithi040.stdout:config.status: creating config.h 2024-09-13T17:53:39.671 INFO:tasks.workunit.client.0.smithi040.stdout:config.status: executing depfiles commands 2024-09-13T17:53:39.740 INFO:tasks.workunit.client.0.smithi040.stdout:config.status: executing default commands 2024-09-13T17:53:39.849 INFO:tasks.workunit.client.0.smithi040.stdout:cd . && /bin/bash /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-09-13T17:53:40.756 INFO:tasks.workunit.client.0.smithi040.stdout:rm -f stamp-h1 2024-09-13T17:53:40.770 INFO:tasks.workunit.client.0.smithi040.stdout:touch config.h.in 2024-09-13T17:53:40.785 INFO:tasks.workunit.client.0.smithi040.stdout:cd . && /bin/bash ./config.status config.h 2024-09-13T17:53:42.337 INFO:tasks.workunit.client.0.smithi040.stdout:config.status: creating config.h 2024-09-13T17:53:42.443 INFO:tasks.workunit.client.0.smithi040.stdout:make all-am 2024-09-13T17:53:42.508 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-13T17:53:42.508 INFO:tasks.workunit.client.0.smithi040.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-09-13T17:53:43.028 INFO:tasks.workunit.client.0.smithi040.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-09-13T17:53:43.055 INFO:tasks.workunit.client.0.smithi040.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-09-13T17:53:43.409 INFO:tasks.workunit.client.0.smithi040.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-09-13T17:53:43.428 INFO:tasks.workunit.client.0.smithi040.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-09-13T17:53:44.560 INFO:tasks.workunit.client.0.smithi040.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-09-13T17:53:44.582 INFO:tasks.workunit.client.0.smithi040.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-09-13T17:53:45.011 INFO:tasks.workunit.client.0.smithi040.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-09-13T17:53:45.030 INFO:tasks.workunit.client.0.smithi040.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-09-13T17:53:45.467 INFO:tasks.workunit.client.0.smithi040.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-09-13T17:53:45.483 INFO:tasks.workunit.client.0.smithi040.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-09-13T17:53:45.813 INFO:tasks.workunit.client.0.smithi040.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-09-13T17:53:45.836 INFO:tasks.workunit.client.0.smithi040.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-09-13T17:53:46.949 INFO:tasks.workunit.client.0.smithi040.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-09-13T17:53:46.969 INFO:tasks.workunit.client.0.smithi040.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-09-13T17:53:47.230 INFO:tasks.workunit.client.0.smithi040.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-09-13T17:53:47.250 INFO:tasks.workunit.client.0.smithi040.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-09-13T17:53:48.228 INFO:tasks.workunit.client.0.smithi040.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-09-13T17:53:48.245 INFO:tasks.workunit.client.0.smithi040.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-09-13T17:53:48.630 INFO:tasks.workunit.client.0.smithi040.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-09-13T17:53:48.650 INFO:tasks.workunit.client.0.smithi040.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-09-13T17:53:49.144 INFO:tasks.workunit.client.0.smithi040.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-09-13T17:53:49.166 INFO:tasks.workunit.client.0.smithi040.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-09-13T17:53:49.492 INFO:tasks.workunit.client.0.smithi040.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-09-13T17:53:49.511 INFO:tasks.workunit.client.0.smithi040.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-09-13T17:53:49.757 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-13T17:53:49.758 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-13T17:53:49.758 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T17:53:49.758 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T17:53:49.761 INFO:tasks.ceph.mon.c:Started 2024-09-13T17:53:49.761 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T17:53:49.761 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:53:49.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.874+0000 7f05f575d700 1 Processor -- start 2024-09-13T17:53:49.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.874+0000 7f05f575d700 1 -- start start 2024-09-13T17:53:49.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05f575d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f05e80d6750 0x7f05e80e0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:49.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05f575d700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e80dc980 con 0x7f05e80509b0 2024-09-13T17:53:49.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05f575d700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e80dcc60 con 0x7f05e80dc4d0 2024-09-13T17:53:49.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05f575d700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e80dd500 con 0x7f05e80d6750 2024-09-13T17:53:49.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05f4f5c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f05e80509b0 0x7f05e80e6e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42544/0 (socket says 172.21.15.40:42544) 2024-09-13T17:53:49.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05f4f5c700 1 -- 172.21.15.40:0/1989741435 learned_addr learned my addr 172.21.15.40:0/1989741435 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:53:49.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05effff700 1 --2- 172.21.15.40:0/1989741435 >> v2:172.21.15.43:3300/0 conn(0x7f05e80d6750 0x7f05e80e0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:49.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05eeffd700 1 -- 172.21.15.40:0/1989741435 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1242206476 0 0) 0x7f05e80dc980 con 0x7f05e80509b0 2024-09-13T17:53:49.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05eeffd700 1 -- 172.21.15.40:0/1989741435 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05e00035d0 con 0x7f05e80509b0 2024-09-13T17:53:49.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05eeffd700 1 -- 172.21.15.40:0/1989741435 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2512522127 0 0) 0x7f05e80dcc60 con 0x7f05e80dc4d0 2024-09-13T17:53:49.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05eeffd700 1 -- 172.21.15.40:0/1989741435 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05e80dc980 con 0x7f05e80dc4d0 2024-09-13T17:53:49.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05eeffd700 1 -- 172.21.15.40:0/1989741435 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1159505660 0 0) 0x7f05e00035d0 con 0x7f05e80509b0 2024-09-13T17:53:49.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05eeffd700 1 -- 172.21.15.40:0/1989741435 >> v2:172.21.15.43:3300/0 conn(0x7f05e80d6750 msgr2=0x7f05e80e0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:53:49.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05eeffd700 1 --2- 172.21.15.40:0/1989741435 >> v2:172.21.15.43:3300/0 conn(0x7f05e80d6750 0x7f05e80e0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:49.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05eeffd700 1 -- 172.21.15.40:0/1989741435 >> v1:172.21.15.43:6789/0 conn(0x7f05e80dc4d0 legacy=0x7f05e8044be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:49.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05eeffd700 1 -- 172.21.15.40:0/1989741435 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e80dcf40 con 0x7f05e80509b0 2024-09-13T17:53:49.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05eeffd700 1 -- 172.21.15.40:0/1989741435 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f05f0067210 con 0x7f05e80509b0 2024-09-13T17:53:49.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05eeffd700 1 -- 172.21.15.40:0/1989741435 >> v1:172.21.15.40:6789/0 conn(0x7f05e80509b0 legacy=0x7f05e80e6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:49.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05eeffd700 1 --2- 172.21.15.40:0/1989741435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e0003bb0 0x7f05e0003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:49.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05eeffd700 1 --2- 172.21.15.40:0/1989741435 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05e0004870 0x7f05e0004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:49.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05eeffd700 1 --2- 172.21.15.40:0/1989741435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05e0005230 0x7f05e000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:49.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05eeffd700 1 -- 172.21.15.40:0/1989741435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e000bf70 con 0x7f05e0005230 2024-09-13T17:53:49.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05eeffd700 1 -- 172.21.15.40:0/1989741435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e000c0b0 con 0x7f05e0003bb0 2024-09-13T17:53:49.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05eeffd700 1 -- 172.21.15.40:0/1989741435 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e000c220 con 0x7f05e0004870 2024-09-13T17:53:49.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05f4f5c700 1 --2- 172.21.15.40:0/1989741435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05e0005230 0x7f05e000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:49.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05effff700 1 --2- 172.21.15.40:0/1989741435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e0003bb0 0x7f05e0003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:49.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05ef7fe700 1 --2- 172.21.15.40:0/1989741435 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05e0004870 0x7f05e0004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:49.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05effff700 1 -- 172.21.15.40:0/1989741435 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05e0004870 msgr2=0x7f05e0004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:53:49.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05effff700 1 --2- 172.21.15.40:0/1989741435 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05e0004870 0x7f05e0004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:49.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05effff700 1 -- 172.21.15.40:0/1989741435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05e0005230 msgr2=0x7f05e000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:53:49.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05effff700 1 --2- 172.21.15.40:0/1989741435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05e0005230 0x7f05e000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:49.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05effff700 1 -- 172.21.15.40:0/1989741435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e00035d0 con 0x7f05e0003bb0 2024-09-13T17:53:49.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05ef7fe700 1 --2- 172.21.15.40:0/1989741435 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05e0004870 0x7f05e0004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:53:49.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05f4f5c700 1 --2- 172.21.15.40:0/1989741435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05e0005230 0x7f05e000ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:53:49.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05f4f5c700 1 -- 172.21.15.40:0/1989741435 reap_dead start 2024-09-13T17:53:49.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05effff700 1 --2- 172.21.15.40:0/1989741435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e0003bb0 0x7f05e0003fb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009b70 tx=0x7f05e00058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=459263320514906a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:49.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05eeffd700 1 -- 172.21.15.40:0/1989741435 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e400e6c0 con 0x7f05e0003bb0 2024-09-13T17:53:49.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05eeffd700 1 -- 172.21.15.40:0/1989741435 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e400e820 con 0x7f05e0003bb0 2024-09-13T17:53:49.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.878+0000 7f05eeffd700 1 -- 172.21.15.40:0/1989741435 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e400eb90 con 0x7f05e0003bb0 2024-09-13T17:53:49.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.882+0000 7f05f575d700 1 -- 172.21.15.40:0/1989741435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e0003bb0 msgr2=0x7f05e0003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:49.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.882+0000 7f05f575d700 1 --2- 172.21.15.40:0/1989741435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e0003bb0 0x7f05e0003fb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009b70 tx=0x7f05e00058a0 comp rx=0 tx=0).stop 2024-09-13T17:53:49.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.882+0000 7f05f575d700 1 -- 172.21.15.40:0/1989741435 shutdown_connections 2024-09-13T17:53:49.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.882+0000 7f05f575d700 1 --2- 172.21.15.40:0/1989741435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e0003bb0 0x7f05e0003fb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:49.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.882+0000 7f05f575d700 1 -- 172.21.15.40:0/1989741435 >> 172.21.15.40:0/1989741435 conn(0x7f05e804fd00 msgr2=0x7f05e8050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:49.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.882+0000 7f05f575d700 1 -- 172.21.15.40:0/1989741435 shutdown_connections 2024-09-13T17:53:49.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.882+0000 7f05f575d700 1 -- 172.21.15.40:0/1989741435 wait complete. 2024-09-13T17:53:49.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.882+0000 7f05f575d700 1 Processor -- start 2024-09-13T17:53:49.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.882+0000 7f05f575d700 1 -- start start 2024-09-13T17:53:49.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05f575d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e0003bb0 0x7f05e8047a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:49.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05f575d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05e80d53e0 0x7f05e81733f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:49.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05f575d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05e8173930 0x7f05e804bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:49.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05f575d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e8049f60 con 0x7f05e80d53e0 2024-09-13T17:53:49.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05f575d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e8173db0 con 0x7f05e0003bb0 2024-09-13T17:53:49.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05f575d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e8033ab0 con 0x7f05e8173930 2024-09-13T17:53:49.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05ef7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05e80d53e0 0x7f05e81733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:49.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05ef7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05e80d53e0 0x7f05e81733f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36142/0 (socket says 172.21.15.40:36142) 2024-09-13T17:53:49.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05ef7fe700 1 -- 172.21.15.40:0/3793152388 learned_addr learned my addr 172.21.15.40:0/3793152388 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:49.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05effff700 1 --2- 172.21.15.40:0/3793152388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e0003bb0 0x7f05e8047a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:49.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05f4f5c700 1 --2- 172.21.15.40:0/3793152388 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05e8173930 0x7f05e804bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:49.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05f4f5c700 1 -- 172.21.15.40:0/3793152388 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05e8173930 msgr2=0x7f05e804bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-13T17:53:49.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05f4f5c700 1 -- 172.21.15.40:0/3793152388 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05e8173930 msgr2=0x7f05e804bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:53:49.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05f4f5c700 1 --2- 172.21.15.40:0/3793152388 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05e8173930 0x7f05e804bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:49.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05f4f5c700 1 --2- 172.21.15.40:0/3793152388 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05e8173930 0x7f05e804bcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:49.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05ef7fe700 1 -- 172.21.15.40:0/3793152388 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05e8173930 msgr2=0x7f05e804bcb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:53:49.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05ef7fe700 1 --2- 172.21.15.40:0/3793152388 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05e8173930 0x7f05e804bcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:49.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05ef7fe700 1 -- 172.21.15.40:0/3793152388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e0003bb0 msgr2=0x7f05e8047a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:49.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05ef7fe700 1 --2- 172.21.15.40:0/3793152388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e0003bb0 0x7f05e8047a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:49.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05ef7fe700 1 -- 172.21.15.40:0/3793152388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e80d6cb0 con 0x7f05e80d53e0 2024-09-13T17:53:49.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05ef7fe700 1 --2- 172.21.15.40:0/3793152388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05e80d53e0 0x7f05e81733f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00f8c0 tx=0x7f05dc00fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:49.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05ecff9700 1 -- 172.21.15.40:0/3793152388 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc018070 con 0x7f05e80d53e0 2024-09-13T17:53:49.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05ecff9700 1 -- 172.21.15.40:0/3793152388 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05dc0052e0 con 0x7f05e80d53e0 2024-09-13T17:53:49.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05f575d700 1 -- 172.21.15.40:0/3793152388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e8033d50 con 0x7f05e80d53e0 2024-09-13T17:53:49.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05ecff9700 1 -- 172.21.15.40:0/3793152388 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc005490 con 0x7f05e80d53e0 2024-09-13T17:53:49.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05f575d700 1 -- 172.21.15.40:0/3793152388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e80498a0 con 0x7f05e80d53e0 2024-09-13T17:53:49.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.886+0000 7f05f575d700 1 -- 172.21.15.40:0/3793152388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e8035de0 con 0x7f05e80d53e0 2024-09-13T17:53:49.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.890+0000 7f05ecff9700 1 -- 172.21.15.40:0/3793152388 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f05dc006880 con 0x7f05e80d53e0 2024-09-13T17:53:49.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.890+0000 7f05ecff9700 1 --2- 172.21.15.40:0/3793152388 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f05d4066760 0x7f05d4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:49.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.890+0000 7f05ecff9700 1 -- 172.21.15.40:0/3793152388 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f05dc083af0 con 0x7f05e80d53e0 2024-09-13T17:53:49.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.890+0000 7f05effff700 1 --2- 172.21.15.40:0/3793152388 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f05d4066760 0x7f05d4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:49.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.890+0000 7f05effff700 1 --2- 172.21.15.40:0/3793152388 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f05d4066760 0x7f05d4068c20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f05e40090b0 tx=0x7f05e401e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:49.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:49.898+0000 7f05ecff9700 1 -- 172.21.15.40:0/3793152388 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f05dc050b30 con 0x7f05e80d53e0 2024-09-13T17:53:50.047 INFO:tasks.workunit.client.0.smithi040.stderr:ffsb_op.c: In function 'print_op_results': 2024-09-13T17:53:50.047 INFO:tasks.workunit.client.0.smithi040.stderr:ffsb_op.c:83:23: warning: embedded '\0' in format [-Wformat-contains-nul] 2024-09-13T17:53:50.047 INFO:tasks.workunit.client.0.smithi040.stderr: 83 | sprintf(buf, "%s/sec\0", buf); 2024-09-13T17:53:50.048 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~ 2024-09-13T17:53:50.048 INFO:tasks.workunit.client.0.smithi040.stderr:ffsb_op.c:86:19: warning: embedded '\0' in format [-Wformat-contains-nul] 2024-09-13T17:53:50.048 INFO:tasks.workunit.client.0.smithi040.stderr: 86 | sprintf(buf, "NA\0"); 2024-09-13T17:53:50.048 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~ 2024-09-13T17:53:50.074 INFO:tasks.workunit.client.0.smithi040.stderr:ffsb_op.c: In function 'print_results': 2024-09-13T17:53:50.074 INFO:tasks.workunit.client.0.smithi040.stderr:ffsb_op.c:83:19: warning: '/sec' directive writing 4 bytes into a region of size between 1 and 256 [-Wformat-overflow=] 2024-09-13T17:53:50.074 INFO:tasks.workunit.client.0.smithi040.stderr: 83 | sprintf(buf, "%s/sec\0", buf); 2024-09-13T17:53:50.074 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~ 2024-09-13T17:53:50.074 INFO:tasks.workunit.client.0.smithi040.stderr:In file included from /usr/include/stdio.h:867, 2024-09-13T17:53:50.074 INFO:tasks.workunit.client.0.smithi040.stderr: from ffsb_op.c:20: 2024-09-13T17:53:50.074 INFO:tasks.workunit.client.0.smithi040.stderr:/usr/include/x86_64-linux-gnu/bits/stdio2.h:36:10: note: '__builtin___sprintf_chk' output between 5 and 260 bytes into a destination of size 256 2024-09-13T17:53:50.074 INFO:tasks.workunit.client.0.smithi040.stderr: 36 | return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1, 2024-09-13T17:53:50.074 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T17:53:50.074 INFO:tasks.workunit.client.0.smithi040.stderr: 37 | __bos (__s), __fmt, __va_arg_pack ()); 2024-09-13T17:53:50.075 INFO:tasks.workunit.client.0.smithi040.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T17:53:50.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:50.138+0000 7f05f575d700 1 -- 172.21.15.40:0/3793152388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f05e804aa40 con 0x7f05e80d53e0 2024-09-13T17:53:50.658 INFO:tasks.workunit.client.0.smithi040.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-09-13T17:53:50.674 INFO:tasks.workunit.client.0.smithi040.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-09-13T17:53:52.713 INFO:tasks.workunit.client.0.smithi040.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-09-13T17:53:52.733 INFO:tasks.workunit.client.0.smithi040.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-09-13T17:53:53.898 INFO:tasks.workunit.client.0.smithi040.stderr:In file included from parser.h:22, 2024-09-13T17:53:53.898 INFO:tasks.workunit.client.0.smithi040.stderr: from parser.c:25: 2024-09-13T17:53:53.898 INFO:tasks.workunit.client.0.smithi040.stderr:parser.c: In function 'init_filesys': 2024-09-13T17:53:53.901 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T17:53:53.901 INFO:tasks.workunit.client.0.smithi040.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T17:53:53.901 INFO:tasks.workunit.client.0.smithi040.stderr: | ^ 2024-09-13T17:53:53.901 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2024-09-13T17:53:53.902 INFO:tasks.workunit.client.0.smithi040.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T17:53:53.902 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~ 2024-09-13T17:53:53.902 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2024-09-13T17:53:53.902 INFO:tasks.workunit.client.0.smithi040.stderr: 36 | container_of(ptr, type, member) 2024-09-13T17:53:53.902 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~ 2024-09-13T17:53:53.902 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:39:13: note: in expansion of macro 'list_entry' 2024-09-13T17:53:53.902 INFO:tasks.workunit.client.0.smithi040.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-13T17:53:53.902 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~ 2024-09-13T17:53:53.902 INFO:tasks.workunit.client.0.smithi040.stderr:parser.c:934:3: note: in expansion of macro 'list_for_each_entry' 2024-09-13T17:53:53.902 INFO:tasks.workunit.client.0.smithi040.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-13T17:53:53.902 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T17:53:53.902 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T17:53:53.903 INFO:tasks.workunit.client.0.smithi040.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T17:53:53.903 INFO:tasks.workunit.client.0.smithi040.stderr: | ^ 2024-09-13T17:53:53.903 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2024-09-13T17:53:53.903 INFO:tasks.workunit.client.0.smithi040.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T17:53:53.903 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~ 2024-09-13T17:53:53.903 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2024-09-13T17:53:53.903 INFO:tasks.workunit.client.0.smithi040.stderr: 36 | container_of(ptr, type, member) 2024-09-13T17:53:53.903 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~ 2024-09-13T17:53:53.903 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:41:13: note: in expansion of macro 'list_entry' 2024-09-13T17:53:53.903 INFO:tasks.workunit.client.0.smithi040.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-13T17:53:53.903 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~ 2024-09-13T17:53:53.903 INFO:tasks.workunit.client.0.smithi040.stderr:parser.c:934:3: note: in expansion of macro 'list_for_each_entry' 2024-09-13T17:53:53.903 INFO:tasks.workunit.client.0.smithi040.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-13T17:53:53.904 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T17:53:53.904 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T17:53:53.904 INFO:tasks.workunit.client.0.smithi040.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T17:53:53.904 INFO:tasks.workunit.client.0.smithi040.stderr: | ^ 2024-09-13T17:53:53.904 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2024-09-13T17:53:53.904 INFO:tasks.workunit.client.0.smithi040.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T17:53:53.904 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~ 2024-09-13T17:53:53.904 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2024-09-13T17:53:53.904 INFO:tasks.workunit.client.0.smithi040.stderr: 36 | container_of(ptr, type, member) 2024-09-13T17:53:53.904 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~ 2024-09-13T17:53:53.904 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:39:13: note: in expansion of macro 'list_entry' 2024-09-13T17:53:53.904 INFO:tasks.workunit.client.0.smithi040.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-13T17:53:53.904 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~ 2024-09-13T17:53:53.905 INFO:tasks.workunit.client.0.smithi040.stderr:parser.c:941:3: note: in expansion of macro 'list_for_each_entry' 2024-09-13T17:53:53.905 INFO:tasks.workunit.client.0.smithi040.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-13T17:53:53.905 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T17:53:53.905 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T17:53:53.905 INFO:tasks.workunit.client.0.smithi040.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T17:53:53.905 INFO:tasks.workunit.client.0.smithi040.stderr: | ^ 2024-09-13T17:53:53.905 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2024-09-13T17:53:53.905 INFO:tasks.workunit.client.0.smithi040.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T17:53:53.905 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~ 2024-09-13T17:53:53.905 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2024-09-13T17:53:53.905 INFO:tasks.workunit.client.0.smithi040.stderr: 36 | container_of(ptr, type, member) 2024-09-13T17:53:53.905 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~ 2024-09-13T17:53:53.905 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:41:13: note: in expansion of macro 'list_entry' 2024-09-13T17:53:53.906 INFO:tasks.workunit.client.0.smithi040.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-13T17:53:53.906 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~ 2024-09-13T17:53:53.906 INFO:tasks.workunit.client.0.smithi040.stderr:parser.c:941:3: note: in expansion of macro 'list_for_each_entry' 2024-09-13T17:53:53.906 INFO:tasks.workunit.client.0.smithi040.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-13T17:53:53.906 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T17:53:53.906 INFO:tasks.workunit.client.0.smithi040.stderr:parser.c: In function 'init_tg_stats': 2024-09-13T17:53:53.906 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T17:53:53.906 INFO:tasks.workunit.client.0.smithi040.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T17:53:53.906 INFO:tasks.workunit.client.0.smithi040.stderr: | ^ 2024-09-13T17:53:53.906 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2024-09-13T17:53:53.906 INFO:tasks.workunit.client.0.smithi040.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T17:53:53.906 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~ 2024-09-13T17:53:53.907 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2024-09-13T17:53:53.907 INFO:tasks.workunit.client.0.smithi040.stderr: 36 | container_of(ptr, type, member) 2024-09-13T17:53:53.907 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~ 2024-09-13T17:53:53.907 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:39:13: note: in expansion of macro 'list_entry' 2024-09-13T17:53:53.907 INFO:tasks.workunit.client.0.smithi040.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-13T17:53:53.907 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~ 2024-09-13T17:53:53.907 INFO:tasks.workunit.client.0.smithi040.stderr:parser.c:971:6: note: in expansion of macro 'list_for_each_entry' 2024-09-13T17:53:53.908 INFO:tasks.workunit.client.0.smithi040.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-13T17:53:53.908 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T17:53:53.908 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T17:53:53.908 INFO:tasks.workunit.client.0.smithi040.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T17:53:53.908 INFO:tasks.workunit.client.0.smithi040.stderr: | ^ 2024-09-13T17:53:53.908 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2024-09-13T17:53:53.908 INFO:tasks.workunit.client.0.smithi040.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T17:53:53.908 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~ 2024-09-13T17:53:53.908 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2024-09-13T17:53:53.908 INFO:tasks.workunit.client.0.smithi040.stderr: 36 | container_of(ptr, type, member) 2024-09-13T17:53:53.908 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~ 2024-09-13T17:53:53.908 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:41:13: note: in expansion of macro 'list_entry' 2024-09-13T17:53:53.908 INFO:tasks.workunit.client.0.smithi040.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-13T17:53:53.908 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~ 2024-09-13T17:53:53.909 INFO:tasks.workunit.client.0.smithi040.stderr:parser.c:971:6: note: in expansion of macro 'list_for_each_entry' 2024-09-13T17:53:53.909 INFO:tasks.workunit.client.0.smithi040.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-13T17:53:53.909 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T17:53:53.909 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T17:53:53.909 INFO:tasks.workunit.client.0.smithi040.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T17:53:53.909 INFO:tasks.workunit.client.0.smithi040.stderr: | ^ 2024-09-13T17:53:53.909 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2024-09-13T17:53:53.909 INFO:tasks.workunit.client.0.smithi040.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T17:53:53.909 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~ 2024-09-13T17:53:53.909 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2024-09-13T17:53:53.909 INFO:tasks.workunit.client.0.smithi040.stderr: 36 | container_of(ptr, type, member) 2024-09-13T17:53:53.909 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~ 2024-09-13T17:53:53.909 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:39:13: note: in expansion of macro 'list_entry' 2024-09-13T17:53:53.909 INFO:tasks.workunit.client.0.smithi040.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-13T17:53:53.910 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~ 2024-09-13T17:53:53.910 INFO:tasks.workunit.client.0.smithi040.stderr:parser.c:980:6: note: in expansion of macro 'list_for_each_entry' 2024-09-13T17:53:53.910 INFO:tasks.workunit.client.0.smithi040.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-13T17:53:53.910 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T17:53:53.910 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-13T17:53:53.910 INFO:tasks.workunit.client.0.smithi040.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-13T17:53:53.910 INFO:tasks.workunit.client.0.smithi040.stderr: | ^ 2024-09-13T17:53:53.910 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:33:28: note: in expansion of macro 'offsetof' 2024-09-13T17:53:53.910 INFO:tasks.workunit.client.0.smithi040.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-13T17:53:53.910 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~ 2024-09-13T17:53:53.911 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:36:2: note: in expansion of macro 'container_of' 2024-09-13T17:53:53.911 INFO:tasks.workunit.client.0.smithi040.stderr: 36 | container_of(ptr, type, member) 2024-09-13T17:53:53.911 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~ 2024-09-13T17:53:53.911 INFO:tasks.workunit.client.0.smithi040.stderr:list.h:41:13: note: in expansion of macro 'list_entry' 2024-09-13T17:53:53.911 INFO:tasks.workunit.client.0.smithi040.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-13T17:53:53.911 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~ 2024-09-13T17:53:53.911 INFO:tasks.workunit.client.0.smithi040.stderr:parser.c:980:6: note: in expansion of macro 'list_for_each_entry' 2024-09-13T17:53:53.911 INFO:tasks.workunit.client.0.smithi040.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-13T17:53:53.911 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-13T17:53:53.978 INFO:tasks.workunit.client.0.smithi040.stderr:In file included from /usr/include/string.h:495, 2024-09-13T17:53:53.979 INFO:tasks.workunit.client.0.smithi040.stderr: from parser.c:19: 2024-09-13T17:53:53.979 INFO:tasks.workunit.client.0.smithi040.stderr:In function 'memcpy', 2024-09-13T17:53:53.979 INFO:tasks.workunit.client.0.smithi040.stderr: inlined from 'size64_convert' at parser.c:119:2: 2024-09-13T17:53:53.979 INFO:tasks.workunit.client.0.smithi040.stderr:/usr/include/x86_64-linux-gnu/bits/string_fortified.h:34:10: warning: '__builtin_memcpy' reading 3 bytes from a region of size 2 [-Wstringop-overflow=] 2024-09-13T17:53:53.979 INFO:tasks.workunit.client.0.smithi040.stderr: 34 | return __builtin___memcpy_chk (__dest, __src, __len, __bos0 (__dest)); 2024-09-13T17:53:53.979 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T17:53:54.048 INFO:tasks.workunit.client.0.smithi040.stderr:In function 'strncpy', 2024-09-13T17:53:54.048 INFO:tasks.workunit.client.0.smithi040.stderr: inlined from 'set_weight' at parser.c:623:4: 2024-09-13T17:53:54.048 INFO:tasks.workunit.client.0.smithi040.stderr:/usr/include/x86_64-linux-gnu/bits/string_fortified.h:106:10: warning: '__builtin_strncpy' specified bound depends on the length of the source argument [-Wstringop-overflow=] 2024-09-13T17:53:54.048 INFO:tasks.workunit.client.0.smithi040.stderr: 106 | return __builtin___strncpy_chk (__dest, __src, __len, __bos (__dest)); 2024-09-13T17:53:54.048 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T17:53:54.048 INFO:tasks.workunit.client.0.smithi040.stderr:parser.c: In function 'set_weight': 2024-09-13T17:53:54.049 INFO:tasks.workunit.client.0.smithi040.stderr:parser.c:620:10: note: length computed here 2024-09-13T17:53:54.049 INFO:tasks.workunit.client.0.smithi040.stderr: 620 | len = strlen(tmp_config->name); 2024-09-13T17:53:54.049 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T17:53:54.053 INFO:tasks.workunit.client.0.smithi040.stderr:In file included from /usr/include/string.h:495, 2024-09-13T17:53:54.053 INFO:tasks.workunit.client.0.smithi040.stderr: from parser.c:19: 2024-09-13T17:53:54.053 INFO:tasks.workunit.client.0.smithi040.stderr:In function 'strncpy', 2024-09-13T17:53:54.053 INFO:tasks.workunit.client.0.smithi040.stderr: inlined from 'get_weight_total' at parser.c:645:4: 2024-09-13T17:53:54.054 INFO:tasks.workunit.client.0.smithi040.stderr:/usr/include/x86_64-linux-gnu/bits/string_fortified.h:106:10: warning: '__builtin_strncpy' specified bound depends on the length of the source argument [-Wstringop-overflow=] 2024-09-13T17:53:54.054 INFO:tasks.workunit.client.0.smithi040.stderr: 106 | return __builtin___strncpy_chk (__dest, __src, __len, __bos (__dest)); 2024-09-13T17:53:54.054 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T17:53:54.054 INFO:tasks.workunit.client.0.smithi040.stderr:parser.c: In function 'get_weight_total': 2024-09-13T17:53:54.054 INFO:tasks.workunit.client.0.smithi040.stderr:parser.c:642:10: note: length computed here 2024-09-13T17:53:54.054 INFO:tasks.workunit.client.0.smithi040.stderr: 642 | len = strlen(tmp_config->name); 2024-09-13T17:53:54.054 INFO:tasks.workunit.client.0.smithi040.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-13T17:53:54.234 INFO:tasks.workunit.client.0.smithi040.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-09-13T17:53:54.254 INFO:tasks.workunit.client.0.smithi040.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-09-13T17:53:55.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:55.098+0000 7f05ecff9700 1 -- 172.21.15.40:0/3793152388 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7f05dc0015b0 con 0x7f05e80d53e0 2024-09-13T17:53:55.106 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":32,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:53:55.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:55.102+0000 7f05f575d700 1 -- 172.21.15.40:0/3793152388 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f05d4066760 msgr2=0x7f05d4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:55.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:55.102+0000 7f05f575d700 1 --2- 172.21.15.40:0/3793152388 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f05d4066760 0x7f05d4068c20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f05e40090b0 tx=0x7f05e401e040 comp rx=0 tx=0).stop 2024-09-13T17:53:55.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:55.102+0000 7f05f575d700 1 -- 172.21.15.40:0/3793152388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05e80d53e0 msgr2=0x7f05e81733f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:55.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:55.102+0000 7f05f575d700 1 --2- 172.21.15.40:0/3793152388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05e80d53e0 0x7f05e81733f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00f8c0 tx=0x7f05dc00fbd0 comp rx=0 tx=0).stop 2024-09-13T17:53:55.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:55.102+0000 7f05f575d700 1 -- 172.21.15.40:0/3793152388 shutdown_connections 2024-09-13T17:53:55.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:55.102+0000 7f05f575d700 1 --2- 172.21.15.40:0/3793152388 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f05d4066760 0x7f05d4068c20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:55.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:55.102+0000 7f05f575d700 1 --2- 172.21.15.40:0/3793152388 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05e8173930 0x7f05e804bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:55.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:55.102+0000 7f05f575d700 1 --2- 172.21.15.40:0/3793152388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05e80d53e0 0x7f05e81733f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:55.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:55.102+0000 7f05f575d700 1 --2- 172.21.15.40:0/3793152388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05e0003bb0 0x7f05e8047a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:55.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:55.102+0000 7f05f575d700 1 -- 172.21.15.40:0/3793152388 >> 172.21.15.40:0/3793152388 conn(0x7f05e804fd00 msgr2=0x7f05e80dbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:55.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:55.106+0000 7f05f575d700 1 -- 172.21.15.40:0/3793152388 shutdown_connections 2024-09-13T17:53:55.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:55.106+0000 7f05f575d700 1 -- 172.21.15.40:0/3793152388 wait complete. 2024-09-13T17:53:57.518 INFO:tasks.workunit.client.0.smithi040.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-09-13T17:53:57.540 INFO:tasks.workunit.client.0.smithi040.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-09-13T17:53:57.878 INFO:tasks.workunit.client.0.smithi040.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-09-13T17:53:57.897 INFO:tasks.workunit.client.0.smithi040.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-09-13T17:53:58.060 INFO:tasks.workunit.client.0.smithi040.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-09-13T17:53:58.085 INFO:tasks.workunit.client.0.smithi040.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-09-13T17:53:58.129 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:53:58.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.246+0000 7fd0b7fff700 1 Processor -- start 2024-09-13T17:53:58.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.246+0000 7fd0b7fff700 1 -- start start 2024-09-13T17:53:58.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd0b80c5360 0x7fd0b80c5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b7fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b8153be0 con 0x7fd0b8153800 2024-09-13T17:53:58.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b7fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b815d100 con 0x7fd0b815c9c0 2024-09-13T17:53:58.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b7fff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b80ca4a0 con 0x7fd0b80c5360 2024-09-13T17:53:58.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b6ffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd0b8153800 0x7fd0b80c9d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53180/0 (socket says 172.21.15.40:53180) 2024-09-13T17:53:58.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b6ffd700 1 -- 172.21.15.40:0/4217151832 learned_addr learned my addr 172.21.15.40:0/4217151832 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:53:58.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b77fe700 1 --2- 172.21.15.40:0/4217151832 >> v2:172.21.15.40:3300/0 conn(0x7fd0b80c5360 0x7fd0b80c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b77fe700 1 -- 172.21.15.40:0/4217151832 >> v1:172.21.15.43:6789/0 conn(0x7fd0b815c9c0 legacy=0x7fd0b80cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b77fe700 1 -- 172.21.15.40:0/4217151832 >> v1:172.21.15.40:6789/0 conn(0x7fd0b8153800 legacy=0x7fd0b80c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b77fe700 1 -- 172.21.15.40:0/4217151832 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b815d3e0 con 0x7fd0b80c5360 2024-09-13T17:53:58.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b77fe700 1 --2- 172.21.15.40:0/4217151832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0b80c5360 0x7fd0b80c5770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009d90 tx=0x7fd0b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca917f510265c2d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:58.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b67fc700 1 -- 172.21.15.40:0/4217151832 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b000e070 con 0x7fd0b80c5360 2024-09-13T17:53:58.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b67fc700 1 -- 172.21.15.40:0/4217151832 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b000bbd0 con 0x7fd0b80c5360 2024-09-13T17:53:58.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b67fc700 1 -- 172.21.15.40:0/4217151832 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0012430 con 0x7fd0b80c5360 2024-09-13T17:53:58.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b7fff700 1 -- 172.21.15.40:0/4217151832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0b80c5360 msgr2=0x7fd0b80c5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b7fff700 1 --2- 172.21.15.40:0/4217151832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0b80c5360 0x7fd0b80c5770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0009d90 tx=0x7fd0b000b330 comp rx=0 tx=0).stop 2024-09-13T17:53:58.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b7fff700 1 -- 172.21.15.40:0/4217151832 shutdown_connections 2024-09-13T17:53:58.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b7fff700 1 --2- 172.21.15.40:0/4217151832 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0b80c5360 0x7fd0b80c5770 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b7fff700 1 -- 172.21.15.40:0/4217151832 >> 172.21.15.40:0/4217151832 conn(0x7fd0b8066090 msgr2=0x7fd0b80664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:58.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b7fff700 1 -- 172.21.15.40:0/4217151832 shutdown_connections 2024-09-13T17:53:58.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.250+0000 7fd0b7fff700 1 -- 172.21.15.40:0/4217151832 wait complete. 2024-09-13T17:53:58.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b7fff700 1 Processor -- start 2024-09-13T17:53:58.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b7fff700 1 -- start start 2024-09-13T17:53:58.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0b80c5360 0x7fd0b80b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b7fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0b8153800 0x7fd0b8154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b815c9c0 0x7fd0b81548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b7fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b80ceae0 con 0x7fd0b80c5360 2024-09-13T17:53:58.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b7fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b80b4f90 con 0x7fd0b815c9c0 2024-09-13T17:53:58.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b7fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b80b3d40 con 0x7fd0b8153800 2024-09-13T17:53:58.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0b80c5360 0x7fd0b80b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0b80c5360 0x7fd0b80b33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56960/0 (socket says 172.21.15.40:56960) 2024-09-13T17:53:58.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b6ffd700 1 -- 172.21.15.40:0/3166459019 learned_addr learned my addr 172.21.15.40:0/3166459019 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:58.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b77fe700 1 --2- 172.21.15.40:0/3166459019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b815c9c0 0x7fd0b81548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0affff700 1 --2- 172.21.15.40:0/3166459019 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0b8153800 0x7fd0b8154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0affff700 1 -- 172.21.15.40:0/3166459019 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0b8153800 msgr2=0x7fd0b8154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T17:53:58.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0affff700 1 -- 172.21.15.40:0/3166459019 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0b8153800 msgr2=0x7fd0b8154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:53:58.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0affff700 1 --2- 172.21.15.40:0/3166459019 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0b8153800 0x7fd0b8154380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:58.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b6ffd700 1 -- 172.21.15.40:0/3166459019 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0b8153800 msgr2=0x7fd0b8154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0affff700 1 --2- 172.21.15.40:0/3166459019 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0b8153800 0x7fd0b8154380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:58.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b6ffd700 1 --2- 172.21.15.40:0/3166459019 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0b8153800 0x7fd0b8154380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b6ffd700 1 -- 172.21.15.40:0/3166459019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b815c9c0 msgr2=0x7fd0b81548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b6ffd700 1 --2- 172.21.15.40:0/3166459019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b815c9c0 0x7fd0b81548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b6ffd700 1 -- 172.21.15.40:0/3166459019 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b0009a70 con 0x7fd0b80c5360 2024-09-13T17:53:58.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b6ffd700 1 --2- 172.21.15.40:0/3166459019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0b80c5360 0x7fd0b80b33a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800c700 tx=0x7fd0a800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:58.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b4ff9700 1 -- 172.21.15.40:0/3166459019 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8005d90 con 0x7fd0b80c5360 2024-09-13T17:53:58.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b4ff9700 1 -- 172.21.15.40:0/3166459019 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a80070d0 con 0x7fd0b80c5360 2024-09-13T17:53:58.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b4ff9700 1 -- 172.21.15.40:0/3166459019 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8007440 con 0x7fd0b80c5360 2024-09-13T17:53:58.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b7fff700 1 -- 172.21.15.40:0/3166459019 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b81ef3f0 con 0x7fd0b80c5360 2024-09-13T17:53:58.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.254+0000 7fd0b7fff700 1 -- 172.21.15.40:0/3166459019 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b80cecc0 con 0x7fd0b80c5360 2024-09-13T17:53:58.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.258+0000 7fd0b7fff700 1 -- 172.21.15.40:0/3166459019 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b80b1da0 con 0x7fd0b80c5360 2024-09-13T17:53:58.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.258+0000 7fd0b4ff9700 1 -- 172.21.15.40:0/3166459019 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd0a80075a0 con 0x7fd0b80c5360 2024-09-13T17:53:58.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.258+0000 7fd0b4ff9700 1 --2- 172.21.15.40:0/3166459019 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd098066730 0x7fd098068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.258+0000 7fd0b4ff9700 1 -- 172.21.15.40:0/3166459019 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd0a80838c0 con 0x7fd0b80c5360 2024-09-13T17:53:58.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.258+0000 7fd0affff700 1 --2- 172.21.15.40:0/3166459019 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd098066730 0x7fd098068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.258+0000 7fd0affff700 1 --2- 172.21.15.40:0/3166459019 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd098066730 0x7fd098068bf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd0b815c420 tx=0x7fd0a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:58.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.266+0000 7fd0b4ff9700 1 -- 172.21.15.40:0/3166459019 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd0a80508a0 con 0x7fd0b80c5360 2024-09-13T17:53:58.458 INFO:tasks.workunit.client.0.smithi040.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-13T17:53:58.468 INFO:tasks.workunit.client.0.smithi040.stdout:FFSB version 6.0-RC2 started 2024-09-13T17:53:58.468 INFO:tasks.workunit.client.0.smithi040.stdout: 2024-09-13T17:53:58.473 INFO:tasks.workunit.client.0.smithi040.stdout:benchmark time = 300 2024-09-13T17:53:58.473 INFO:tasks.workunit.client.0.smithi040.stdout:ThreadGroup 0 2024-09-13T17:53:58.473 INFO:tasks.workunit.client.0.smithi040.stdout:================ 2024-09-13T17:53:58.473 INFO:tasks.workunit.client.0.smithi040.stdout: num_threads = 32 2024-09-13T17:53:58.473 INFO:tasks.workunit.client.0.smithi040.stdout: 2024-09-13T17:53:58.473 INFO:tasks.workunit.client.0.smithi040.stdout: read_random = off 2024-09-13T17:53:58.473 INFO:tasks.workunit.client.0.smithi040.stdout: read_size = 0 (0B) 2024-09-13T17:53:58.473 INFO:tasks.workunit.client.0.smithi040.stdout: read_blocksize = 0 (0B) 2024-09-13T17:53:58.473 INFO:tasks.workunit.client.0.smithi040.stdout: read_skip = off 2024-09-13T17:53:58.474 INFO:tasks.workunit.client.0.smithi040.stdout: read_skipsize = 0 (0B) 2024-09-13T17:53:58.474 INFO:tasks.workunit.client.0.smithi040.stdout: 2024-09-13T17:53:58.474 INFO:tasks.workunit.client.0.smithi040.stdout: write_random = on 2024-09-13T17:53:58.474 INFO:tasks.workunit.client.0.smithi040.stdout: write_size = 5242880 (5MB) 2024-09-13T17:53:58.474 INFO:tasks.workunit.client.0.smithi040.stdout: fsync_file = 0 2024-09-13T17:53:58.474 INFO:tasks.workunit.client.0.smithi040.stdout: write_blocksize = 4096 (4KB) 2024-09-13T17:53:58.474 INFO:tasks.workunit.client.0.smithi040.stdout: wait time = 0 2024-09-13T17:53:58.474 INFO:tasks.workunit.client.0.smithi040.stdout: 2024-09-13T17:53:58.474 INFO:tasks.workunit.client.0.smithi040.stdout: op weights 2024-09-13T17:53:58.474 INFO:tasks.workunit.client.0.smithi040.stdout: read = 0 (0.00%) 2024-09-13T17:53:58.474 INFO:tasks.workunit.client.0.smithi040.stdout: readall = 0 (0.00%) 2024-09-13T17:53:58.474 INFO:tasks.workunit.client.0.smithi040.stdout: write = 1 (100.00%) 2024-09-13T17:53:58.474 INFO:tasks.workunit.client.0.smithi040.stdout: create = 0 (0.00%) 2024-09-13T17:53:58.474 INFO:tasks.workunit.client.0.smithi040.stdout: append = 0 (0.00%) 2024-09-13T17:53:58.475 INFO:tasks.workunit.client.0.smithi040.stdout: delete = 0 (0.00%) 2024-09-13T17:53:58.475 INFO:tasks.workunit.client.0.smithi040.stdout: metaop = 0 (0.00%) 2024-09-13T17:53:58.475 INFO:tasks.workunit.client.0.smithi040.stdout: createdir = 0 (0.00%) 2024-09-13T17:53:58.475 INFO:tasks.workunit.client.0.smithi040.stdout: stat = 0 (0.00%) 2024-09-13T17:53:58.475 INFO:tasks.workunit.client.0.smithi040.stdout: writeall = 0 (0.00%) 2024-09-13T17:53:58.475 INFO:tasks.workunit.client.0.smithi040.stdout: writeall_fsync = 0 (0.00%) 2024-09-13T17:53:58.475 INFO:tasks.workunit.client.0.smithi040.stdout: open_close = 0 (0.00%) 2024-09-13T17:53:58.475 INFO:tasks.workunit.client.0.smithi040.stdout: write_fsync = 0 (0.00%) 2024-09-13T17:53:58.475 INFO:tasks.workunit.client.0.smithi040.stdout: create_fsync = 0 (0.00%) 2024-09-13T17:53:58.475 INFO:tasks.workunit.client.0.smithi040.stdout: append_fsync = 0 (0.00%) 2024-09-13T17:53:58.475 INFO:tasks.workunit.client.0.smithi040.stdout: 2024-09-13T17:53:58.475 INFO:tasks.workunit.client.0.smithi040.stdout:FileSystem . 2024-09-13T17:53:58.475 INFO:tasks.workunit.client.0.smithi040.stdout:========== 2024-09-13T17:53:58.475 INFO:tasks.workunit.client.0.smithi040.stdout: num_dirs = 0 2024-09-13T17:53:58.476 INFO:tasks.workunit.client.0.smithi040.stdout: starting files = 128 2024-09-13T17:53:58.476 INFO:tasks.workunit.client.0.smithi040.stdout: 2024-09-13T17:53:58.476 INFO:tasks.workunit.client.0.smithi040.stdout: min file size = 104857600 (100MB) 2024-09-13T17:53:58.476 INFO:tasks.workunit.client.0.smithi040.stdout: max file size = 104857600 (100MB) 2024-09-13T17:53:58.476 INFO:tasks.workunit.client.0.smithi040.stdout: directio = off 2024-09-13T17:53:58.476 INFO:tasks.workunit.client.0.smithi040.stdout: alignedio = on 2024-09-13T17:53:58.476 INFO:tasks.workunit.client.0.smithi040.stdout: bufferedio = off 2024-09-13T17:53:58.476 INFO:tasks.workunit.client.0.smithi040.stdout: 2024-09-13T17:53:58.476 INFO:tasks.workunit.client.0.smithi040.stdout: aging is off 2024-09-13T17:53:58.477 INFO:tasks.workunit.client.0.smithi040.stdout: current utilization = 0.04% 2024-09-13T17:53:58.477 INFO:tasks.workunit.client.0.smithi040.stdout: 2024-09-13T17:53:58.477 INFO:tasks.workunit.client.0.smithi040.stdout:checking existing fs: . 2024-09-13T17:53:58.477 INFO:tasks.workunit.client.0.smithi040.stdout:recreating new fileset 2024-09-13T17:53:58.477 INFO:tasks.workunit.client.0.smithi040.stderr:opendir: No such file or directory 2024-09-13T17:53:58.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.530+0000 7fd0b7fff700 1 -- 172.21.15.40:0/3166459019 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd0b80c5ef0 con 0x7fd0b80c5360 2024-09-13T17:53:58.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.530+0000 7fd0b4ff9700 1 -- 172.21.15.40:0/3166459019 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7fd0a8053ec0 con 0x7fd0b80c5360 2024-09-13T17:53:58.537 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":34,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:53:58.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.534+0000 7fd0b7fff700 1 -- 172.21.15.40:0/3166459019 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd098066730 msgr2=0x7fd098068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.534+0000 7fd0b7fff700 1 --2- 172.21.15.40:0/3166459019 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd098066730 0x7fd098068bf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd0b815c420 tx=0x7fd0a0009040 comp rx=0 tx=0).stop 2024-09-13T17:53:58.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.534+0000 7fd0b7fff700 1 -- 172.21.15.40:0/3166459019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0b80c5360 msgr2=0x7fd0b80b33a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.534+0000 7fd0b7fff700 1 --2- 172.21.15.40:0/3166459019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0b80c5360 0x7fd0b80b33a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800c700 tx=0x7fd0a800ca10 comp rx=0 tx=0).stop 2024-09-13T17:53:58.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.534+0000 7fd0b7fff700 1 -- 172.21.15.40:0/3166459019 shutdown_connections 2024-09-13T17:53:58.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.534+0000 7fd0b7fff700 1 --2- 172.21.15.40:0/3166459019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0b815c9c0 0x7fd0b81548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.534+0000 7fd0b7fff700 1 --2- 172.21.15.40:0/3166459019 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd098066730 0x7fd098068bf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.534+0000 7fd0b7fff700 1 --2- 172.21.15.40:0/3166459019 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0b8153800 0x7fd0b8154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.534+0000 7fd0b7fff700 1 --2- 172.21.15.40:0/3166459019 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0b80c5360 0x7fd0b80b33a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.534+0000 7fd0b7fff700 1 -- 172.21.15.40:0/3166459019 >> 172.21.15.40:0/3166459019 conn(0x7fd0b8066090 msgr2=0x7fd0b8151990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:58.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.534+0000 7fd0b7fff700 1 -- 172.21.15.40:0/3166459019 shutdown_connections 2024-09-13T17:53:58.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.534+0000 7fd0b7fff700 1 -- 172.21.15.40:0/3166459019 wait complete. 2024-09-13T17:53:58.558 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T17:53:58.558 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T17:53:58.558 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:53:58.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.662+0000 7fdffc957700 1 Processor -- start 2024-09-13T17:53:58.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdffc957700 1 -- start start 2024-09-13T17:53:58.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdffc957700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdff8150690 0x7fdff806b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdffc957700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff80c2140 con 0x7fdff80c0e90 2024-09-13T17:53:58.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdffc957700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff80c1850 con 0x7fdff806da80 2024-09-13T17:53:58.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdffc957700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff806b9e0 con 0x7fdff8150690 2024-09-13T17:53:58.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff759e700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdff80c0e90 0x7fdff8160a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53190/0 (socket says 172.21.15.40:53190) 2024-09-13T17:53:58.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff759e700 1 -- 172.21.15.40:0/390550778 learned_addr learned my addr 172.21.15.40:0/390550778 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:53:58.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff6d9d700 1 --2- 172.21.15.40:0/390550778 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdff8150690 0x7fdff806b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff5d9b700 1 -- 172.21.15.40:0/390550778 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2048533175 0 0) 0x7fdff80c2140 con 0x7fdff80c0e90 2024-09-13T17:53:58.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff5d9b700 1 -- 172.21.15.40:0/390550778 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfe00035d0 con 0x7fdff80c0e90 2024-09-13T17:53:58.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff6d9d700 1 -- 172.21.15.40:0/390550778 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdff8150690 msgr2=0x7fdff806b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:53:58.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff6d9d700 1 -- 172.21.15.40:0/390550778 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdff8150690 msgr2=0x7fdff806b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:53:58.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff6d9d700 1 --2- 172.21.15.40:0/390550778 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdff8150690 0x7fdff806b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:58.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff6d9d700 1 --2- 172.21.15.40:0/390550778 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdff8150690 0x7fdff806b4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:58.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff5d9b700 1 -- 172.21.15.40:0/390550778 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225195621 0 0) 0x7fdff80c1850 con 0x7fdff806da80 2024-09-13T17:53:58.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff5d9b700 1 -- 172.21.15.40:0/390550778 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdff80c2140 con 0x7fdff806da80 2024-09-13T17:53:58.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff5d9b700 1 -- 172.21.15.40:0/390550778 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1182500629 0 0) 0x7fdfe00035d0 con 0x7fdff80c0e90 2024-09-13T17:53:58.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff5d9b700 1 -- 172.21.15.40:0/390550778 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdff8150690 msgr2=0x7fdff806b4a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:53:58.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff5d9b700 1 --2- 172.21.15.40:0/390550778 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdff8150690 0x7fdff806b4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff5d9b700 1 -- 172.21.15.40:0/390550778 >> v1:172.21.15.43:6789/0 conn(0x7fdff806da80 legacy=0x7fdff8071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff5d9b700 1 -- 172.21.15.40:0/390550778 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff80c1e60 con 0x7fdff80c0e90 2024-09-13T17:53:58.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff5d9b700 1 -- 172.21.15.40:0/390550778 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fdff00019f0 con 0x7fdff80c0e90 2024-09-13T17:53:58.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff5d9b700 1 -- 172.21.15.40:0/390550778 >> v1:172.21.15.40:6789/0 conn(0x7fdff80c0e90 legacy=0x7fdff8160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff5d9b700 1 --2- 172.21.15.40:0/390550778 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdff8150690 0x7fdfe0003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff5d9b700 1 --2- 172.21.15.40:0/390550778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfe0004140 0x7fdfe0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff5d9b700 1 --2- 172.21.15.40:0/390550778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe0004db0 0x7fdfe0005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff5d9b700 1 -- 172.21.15.40:0/390550778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff806b9e0 con 0x7fdfe0004140 2024-09-13T17:53:58.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff5d9b700 1 -- 172.21.15.40:0/390550778 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe000bbc0 con 0x7fdfe0004db0 2024-09-13T17:53:58.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff5d9b700 1 -- 172.21.15.40:0/390550778 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe000bd80 con 0x7fdff8150690 2024-09-13T17:53:58.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff659c700 1 --2- 172.21.15.40:0/390550778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfe0004140 0x7fdfe0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff6d9d700 1 --2- 172.21.15.40:0/390550778 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdff8150690 0x7fdfe0003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff759e700 1 --2- 172.21.15.40:0/390550778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe0004db0 0x7fdfe0005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff659c700 1 -- 172.21.15.40:0/390550778 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdff8150690 msgr2=0x7fdfe0003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:53:58.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff659c700 1 --2- 172.21.15.40:0/390550778 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdff8150690 0x7fdfe0003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff659c700 1 -- 172.21.15.40:0/390550778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe0004db0 msgr2=0x7fdfe0005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:53:58.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff659c700 1 --2- 172.21.15.40:0/390550778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe0004db0 0x7fdfe0005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff659c700 1 -- 172.21.15.40:0/390550778 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe00035d0 con 0x7fdfe0004140 2024-09-13T17:53:58.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff759e700 1 --2- 172.21.15.40:0/390550778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe0004db0 0x7fdfe0005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:53:58.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.666+0000 7fdff659c700 1 --2- 172.21.15.40:0/390550778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfe0004140 0x7fdfe0004870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdfec007fd0 tx=0x7fdfe00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb076013568b958 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:58.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.670+0000 7fdff5d9b700 1 -- 172.21.15.40:0/390550778 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec001450 con 0x7fdfe0004140 2024-09-13T17:53:58.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.670+0000 7fdff5d9b700 1 -- 172.21.15.40:0/390550778 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfec0052f0 con 0x7fdfe0004140 2024-09-13T17:53:58.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.670+0000 7fdff5d9b700 1 -- 172.21.15.40:0/390550778 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec001d60 con 0x7fdfe0004140 2024-09-13T17:53:58.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.670+0000 7fdffc957700 1 -- 172.21.15.40:0/390550778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfe0004140 msgr2=0x7fdfe0004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.670+0000 7fdffc957700 1 --2- 172.21.15.40:0/390550778 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfe0004140 0x7fdfe0004870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdfec007fd0 tx=0x7fdfe00058a0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.670+0000 7fdff759e700 1 -- 172.21.15.40:0/390550778 reap_dead start 2024-09-13T17:53:58.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.670+0000 7fdffc957700 1 -- 172.21.15.40:0/390550778 shutdown_connections 2024-09-13T17:53:58.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.670+0000 7fdffc957700 1 -- 172.21.15.40:0/390550778 >> 172.21.15.40:0/390550778 conn(0x7fdff806e4f0 msgr2=0x7fdff806e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:58.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.670+0000 7fdffc957700 1 -- 172.21.15.40:0/390550778 shutdown_connections 2024-09-13T17:53:58.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.670+0000 7fdffc957700 1 -- 172.21.15.40:0/390550778 wait complete. 2024-09-13T17:53:58.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.670+0000 7fdffc957700 1 Processor -- start 2024-09-13T17:53:58.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.670+0000 7fdffc957700 1 -- start start 2024-09-13T17:53:58.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdffc957700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdff813c960 0x7fdff81f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdffc957700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdff813cd40 0x7fdff81ef490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdffc957700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff81ef9d0 0x7fdff81efe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdffc957700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff81f2ea0 con 0x7fdff81ef9d0 2024-09-13T17:53:58.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdffc957700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff8151910 con 0x7fdff813c960 2024-09-13T17:53:58.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdffc957700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff813b0c0 con 0x7fdff813cd40 2024-09-13T17:53:58.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdff759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff81ef9d0 0x7fdff81efe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdff759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff81ef9d0 0x7fdff81efe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56974/0 (socket says 172.21.15.40:56974) 2024-09-13T17:53:58.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdff759e700 1 -- 172.21.15.40:0/2031064693 learned_addr learned my addr 172.21.15.40:0/2031064693 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:58.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdff6d9d700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdff813c960 0x7fdff81f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdff659c700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdff813cd40 0x7fdff81ef490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdff659c700 1 -- 172.21.15.40:0/2031064693 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdff813cd40 msgr2=0x7fdff81ef490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T17:53:58.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdff659c700 1 -- 172.21.15.40:0/2031064693 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdff813cd40 msgr2=0x7fdff81ef490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:53:58.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdff659c700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdff813cd40 0x7fdff81ef490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:58.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdff659c700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdff813cd40 0x7fdff81ef490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:58.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdff759e700 1 -- 172.21.15.40:0/2031064693 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdff813cd40 msgr2=0x7fdff81ef490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:53:58.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdff759e700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdff813cd40 0x7fdff81ef490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdff759e700 1 -- 172.21.15.40:0/2031064693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdff813c960 msgr2=0x7fdff81f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdff759e700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdff813c960 0x7fdff81f32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdff759e700 1 -- 172.21.15.40:0/2031064693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff80c2610 con 0x7fdff81ef9d0 2024-09-13T17:53:58.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdff759e700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff81ef9d0 0x7fdff81efe50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdff0000f80 tx=0x7fdff0007cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:58.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdfdf7fe700 1 -- 172.21.15.40:0/2031064693 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff000e8d0 con 0x7fdff81ef9d0 2024-09-13T17:53:58.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdfdf7fe700 1 -- 172.21.15.40:0/2031064693 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff000ea50 con 0x7fdff81ef9d0 2024-09-13T17:53:58.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdffc957700 1 -- 172.21.15.40:0/2031064693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff813ae00 con 0x7fdff81ef9d0 2024-09-13T17:53:58.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdfdf7fe700 1 -- 172.21.15.40:0/2031064693 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff000ee00 con 0x7fdff81ef9d0 2024-09-13T17:53:58.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdffc957700 1 -- 172.21.15.40:0/2031064693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff81f2ac0 con 0x7fdff81ef9d0 2024-09-13T17:53:58.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdffc957700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff80c64d0 0x7fdff80c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdffc957700 1 -- 172.21.15.40:0/2031064693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff80676b0 con 0x7fdff80c64d0 2024-09-13T17:53:58.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdff6d9d700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff80c64d0 0x7fdff80c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.674+0000 7fdff6d9d700 1 -- 172.21.15.40:0/2031064693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdff80c7d00 con 0x7fdff80c64d0 2024-09-13T17:53:58.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.678+0000 7fdff6d9d700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff80c64d0 0x7fdff80c68b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8009100 tx=0x7fdfe8009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:58.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.678+0000 7fdfdf7fe700 1 -- 172.21.15.40:0/2031064693 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8016870 con 0x7fdff80c64d0 2024-09-13T17:53:58.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.678+0000 7fdfdf7fe700 1 -- 172.21.15.40:0/2031064693 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdff001d460 con 0x7fdff81ef9d0 2024-09-13T17:53:58.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.678+0000 7fdfdf7fe700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdfe4066800 0x7fdfe4068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.678+0000 7fdff659c700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdfe4066800 0x7fdfe4068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.678+0000 7fdfdf7fe700 1 -- 172.21.15.40:0/2031064693 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdff0084700 con 0x7fdff81ef9d0 2024-09-13T17:53:58.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.678+0000 7fdfdf7fe700 1 -- 172.21.15.40:0/2031064693 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fdfe8003e70 con 0x7fdff80c64d0 2024-09-13T17:53:58.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.678+0000 7fdfdf7fe700 1 -- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff80c64d0 msgr2=0x7fdff80c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.678+0000 7fdfdf7fe700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff80c64d0 0x7fdff80c68b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8009100 tx=0x7fdfe8009940 comp rx=0 tx=0).stop 2024-09-13T17:53:58.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.678+0000 7fdfdf7fe700 1 -- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff80c64d0 msgr2=0x7fdff80c68b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:53:58.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.678+0000 7fdfdf7fe700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff80c64d0 0x7fdff80c68b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.678+0000 7fdff659c700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdfe4066800 0x7fdfe4068cc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdff804ff70 tx=0x7fdfec00a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:58.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdffc957700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff80c6df0 0x7fdff80c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdffc957700 1 -- 172.21.15.40:0/2031064693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff81f2fe0 con 0x7fdff80c6df0 2024-09-13T17:53:58.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdff6d9d700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff80c6df0 0x7fdff80c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdff6d9d700 1 -- 172.21.15.40:0/2031064693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdff81657c0 con 0x7fdff80c6df0 2024-09-13T17:53:58.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdff6d9d700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff80c6df0 0x7fdff80c9be0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdfe801c000 tx=0x7fdfe80047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:58.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdfdf7fe700 1 -- 172.21.15.40:0/2031064693 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe80157d0 con 0x7fdff80c6df0 2024-09-13T17:53:58.697 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:53:58.697 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:53:58.697 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:53:58.698 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:53:58.698 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 34, 2024-09-13T17:53:58.698 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:53:58.698 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:53:58.698 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:53:58.698 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:53:58.698 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:58.698 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 3, 2024-09-13T17:53:58.698 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:53:58.698 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:53:58.698 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:53:58.698 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:58.698 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:58.699 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:58.699 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:58.699 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:58.699 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:58.699 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:58.699 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:58.699 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:58.699 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:58.699 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:58.699 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:53:58.699 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:53:58.699 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:58.699 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:58.699 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:58.699 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:58.700 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:58.700 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:58.700 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:58.700 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:58.700 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:58.700 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:58.700 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:58.700 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.700 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:53:58.700 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:53:58.700 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:53:58.700 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:53:58.700 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:53:58.701 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:53:58.701 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:53:58.701 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:53:58.701 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:53:58.701 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:53:58.701 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:53:58.701 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:53:58.701 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:53:58.701 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:53:58.701 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:53:58.701 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:53:58.701 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:53:58.701 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:58.701 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:58.702 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:58.702 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:58.702 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:58.702 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:58.702 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:58.702 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:58.702 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:58.702 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:58.702 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:58.702 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:53:58.702 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.702 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:53:58.702 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.703 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:53:58.703 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:53:58.703 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:58.703 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:58.703 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.703 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:58.703 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:53:58.703 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:58.703 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.703 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.703 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:58.703 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:53:58.703 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:58.703 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:58.704 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:58.704 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.704 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:53:58.704 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:53:58.704 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:58.704 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:58.704 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:58.704 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.704 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.704 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:53:58.704 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:53:58.704 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:58.704 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:58.704 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.705 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:58.705 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:53:58.705 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:58.705 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.705 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.705 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:58.705 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:53:58.705 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:58.705 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:58.705 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:58.705 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.705 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:53:58.705 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:53:58.705 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:58.706 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:58.706 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:58.706 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.706 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.706 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:53:58.706 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:53:58.706 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:58.706 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:58.706 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.706 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:58.706 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:53:58.706 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:58.706 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.706 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.707 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:58.707 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:53:58.707 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:58.707 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:58.707 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:58.707 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.707 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:53:58.707 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:53:58.707 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:58.707 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:58.707 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:58.707 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:58.707 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:58.707 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.708 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:53:58.708 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:53:58.708 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.708 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:58.708 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:58.708 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:53:58.708 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:58.708 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:58.708 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:53:58.708 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.708 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:58.708 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:58.708 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:53:58.709 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:58.709 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:58.709 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:53:58.709 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.709 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:58.709 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:58.709 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:53:58.709 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:58.709 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:58.709 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.709 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:53:58.709 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:53:58.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdfdf7fe700 1 -- 172.21.15.40:0/2031064693 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4617 (secure 0 0 0) 0x7fdfe8015930 con 0x7fdff80c6df0 2024-09-13T17:53:58.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdfdf7fe700 1 -- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff80c6df0 msgr2=0x7fdff80c9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdfdf7fe700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff80c6df0 0x7fdff80c9be0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdfe801c000 tx=0x7fdfe80047e0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdfdf7fe700 1 -- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff80c6df0 msgr2=0x7fdff80c9be0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:53:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdfdf7fe700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff80c6df0 0x7fdff80c9be0 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdfe801c000 tx=0x7fdfe80047e0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdffc957700 1 -- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdfe4066800 msgr2=0x7fdfe4068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdffc957700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdfe4066800 0x7fdfe4068cc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdff804ff70 tx=0x7fdfec00a000 comp rx=0 tx=0).stop 2024-09-13T17:53:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdffc957700 1 -- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff81ef9d0 msgr2=0x7fdff81efe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdffc957700 1 --2- 172.21.15.40:0/2031064693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdff81ef9d0 0x7fdff81efe50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdff0000f80 tx=0x7fdff0007cb0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdff759e700 1 -- 172.21.15.40:0/2031064693 reap_dead start 2024-09-13T17:53:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdff759e700 1 -- 172.21.15.40:0/2031064693 reap_dead start 2024-09-13T17:53:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdffc957700 1 -- 172.21.15.40:0/2031064693 shutdown_connections 2024-09-13T17:53:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdffc957700 1 -- 172.21.15.40:0/2031064693 >> 172.21.15.40:0/2031064693 conn(0x7fdff806e4f0 msgr2=0x7fdff814f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdffc957700 1 -- 172.21.15.40:0/2031064693 shutdown_connections 2024-09-13T17:53:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.690+0000 7fdffc957700 1 -- 172.21.15.40:0/2031064693 wait complete. 2024-09-13T17:53:58.712 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:53:58.712 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T17:53:58.713 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:53:58.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.814+0000 7f6078830700 1 Processor -- start 2024-09-13T17:53:58.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.814+0000 7f6078830700 1 -- start start 2024-09-13T17:53:58.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f6078830700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f60740c6190 0x7f60740711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f6078830700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607415aea0 con 0x7f6074153150 2024-09-13T17:53:58.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f6078830700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6074159e30 con 0x7f60740724d0 2024-09-13T17:53:58.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f6078830700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607415a0d0 con 0x7f60740c6190 2024-09-13T17:53:58.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f6072d9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6074153150 0x7f607415a790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53198/0 (socket says 172.21.15.40:53198) 2024-09-13T17:53:58.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f6072d9d700 1 -- 172.21.15.40:0/3012236751 learned_addr learned my addr 172.21.15.40:0/3012236751 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:53:58.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f607359e700 1 --2- 172.21.15.40:0/3012236751 >> v2:172.21.15.43:3300/0 conn(0x7f60740c6190 0x7f60740711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f607259c700 1 -- 172.21.15.40:0/3012236751 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 458024146 0 0) 0x7f607415aea0 con 0x7f6074153150 2024-09-13T17:53:58.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f607259c700 1 -- 172.21.15.40:0/3012236751 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60600035d0 con 0x7f6074153150 2024-09-13T17:53:58.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f607359e700 1 -- 172.21.15.40:0/3012236751 >> v1:172.21.15.43:6789/0 conn(0x7f60740724d0 legacy=0x7f6074070aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f607359e700 1 -- 172.21.15.40:0/3012236751 >> v1:172.21.15.40:6789/0 conn(0x7f6074153150 legacy=0x7f607415a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f607259c700 1 -- 172.21.15.40:0/3012236751 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1190421831 0 0) 0x7f60600035d0 con 0x7f6074153150 2024-09-13T17:53:58.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f607359e700 1 -- 172.21.15.40:0/3012236751 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60740cb2a0 con 0x7f60740c6190 2024-09-13T17:53:58.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f607359e700 1 --2- 172.21.15.40:0/3012236751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60740c6190 0x7f60740711b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f606c009d90 tx=0x7f606c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cfea7bab1718c73 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:58.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f607259c700 1 -- 172.21.15.40:0/3012236751 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c00e070 con 0x7f60740c6190 2024-09-13T17:53:58.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f607259c700 1 -- 172.21.15.40:0/3012236751 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606c003a40 con 0x7f60740c6190 2024-09-13T17:53:58.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f607259c700 1 -- 172.21.15.40:0/3012236751 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c001670 con 0x7f60740c6190 2024-09-13T17:53:58.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f6078830700 1 -- 172.21.15.40:0/3012236751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60740c6190 msgr2=0x7f60740711b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f6078830700 1 --2- 172.21.15.40:0/3012236751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60740c6190 0x7f60740711b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f606c009d90 tx=0x7f606c00b330 comp rx=0 tx=0).stop 2024-09-13T17:53:58.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f6078830700 1 -- 172.21.15.40:0/3012236751 shutdown_connections 2024-09-13T17:53:58.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f6078830700 1 --2- 172.21.15.40:0/3012236751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60740c6190 0x7f60740711b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f6078830700 1 -- 172.21.15.40:0/3012236751 >> 172.21.15.40:0/3012236751 conn(0x7f607406a390 msgr2=0x7f607406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:58.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f6078830700 1 -- 172.21.15.40:0/3012236751 shutdown_connections 2024-09-13T17:53:58.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.818+0000 7f6078830700 1 -- 172.21.15.40:0/3012236751 wait complete. 2024-09-13T17:53:58.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.822+0000 7f6078830700 1 Processor -- start 2024-09-13T17:53:58.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.822+0000 7f6078830700 1 -- start start 2024-09-13T17:53:58.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.822+0000 7f6078830700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60740724d0 0x7f6074154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.822+0000 7f6078830700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60740c6190 0x7f607414be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.822+0000 7f6078830700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6074153150 0x7f60741f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.822+0000 7f6078830700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6074155330 con 0x7f60740724d0 2024-09-13T17:53:58.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.822+0000 7f6078830700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60741e9000 con 0x7f6074153150 2024-09-13T17:53:58.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.822+0000 7f6078830700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60740c4880 con 0x7f60740c6190 2024-09-13T17:53:58.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.822+0000 7f6072d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60740724d0 0x7f6074154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.822+0000 7f6072d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60740724d0 0x7f6074154df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56990/0 (socket says 172.21.15.40:56990) 2024-09-13T17:53:58.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.822+0000 7f6072d9d700 1 -- 172.21.15.40:0/2696948333 learned_addr learned my addr 172.21.15.40:0/2696948333 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:58.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.822+0000 7f606bfff700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60740c6190 0x7f607414be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.822+0000 7f607359e700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6074153150 0x7f60741f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.822+0000 7f606bfff700 1 -- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60740c6190 msgr2=0x7f607414be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T17:53:58.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.822+0000 7f606bfff700 1 -- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60740c6190 msgr2=0x7f607414be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:53:58.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.822+0000 7f606bfff700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60740c6190 0x7f607414be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:58.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.822+0000 7f6072d9d700 1 -- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60740c6190 msgr2=0x7f607414be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.822+0000 7f606bfff700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60740c6190 0x7f607414be80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:58.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.822+0000 7f6072d9d700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60740c6190 0x7f607414be80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.822+0000 7f6072d9d700 1 -- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6074153150 msgr2=0x7f60741f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.826+0000 7f6072d9d700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6074153150 0x7f60741f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.826+0000 7f6072d9d700 1 -- 172.21.15.40:0/2696948333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c009a70 con 0x7f60740724d0 2024-09-13T17:53:58.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.826+0000 7f607359e700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6074153150 0x7f60741f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:53:58.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.826+0000 7f6072d9d700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60740724d0 0x7f6074154df0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f605c00c540 tx=0x7f605c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:58.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.826+0000 7f606b7fe700 1 -- 172.21.15.40:0/2696948333 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c004ca0 con 0x7f60740724d0 2024-09-13T17:53:58.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.826+0000 7f606b7fe700 1 -- 172.21.15.40:0/2696948333 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f605c004e00 con 0x7f60740724d0 2024-09-13T17:53:58.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.826+0000 7f606b7fe700 1 -- 172.21.15.40:0/2696948333 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c007310 con 0x7f60740724d0 2024-09-13T17:53:58.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.826+0000 7f6078830700 1 -- 172.21.15.40:0/2696948333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60741ef610 con 0x7f60740724d0 2024-09-13T17:53:58.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.826+0000 7f6078830700 1 -- 172.21.15.40:0/2696948333 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6074153d50 con 0x7f60740724d0 2024-09-13T17:53:58.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.826+0000 7f6078830700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6058002a40 0x7f6058004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.826+0000 7f6078830700 1 -- 172.21.15.40:0/2696948333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6058007f40 con 0x7f6058002a40 2024-09-13T17:53:58.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.826+0000 7f606b7fe700 1 -- 172.21.15.40:0/2696948333 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f605c007470 con 0x7f60740724d0 2024-09-13T17:53:58.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.826+0000 7f606bfff700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6058002a40 0x7f6058004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.830+0000 7f606b7fe700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6054066730 0x7f6054068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.830+0000 7f606b7fe700 1 -- 172.21.15.40:0/2696948333 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f605c0841d0 con 0x7f60740724d0 2024-09-13T17:53:58.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.830+0000 7f607359e700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6054066730 0x7f6054068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.830+0000 7f606bfff700 1 -- 172.21.15.40:0/2696948333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6058008100 con 0x7f6058002a40 2024-09-13T17:53:58.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.830+0000 7f606bfff700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6058002a40 0x7f6058004f00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f607415c6c0 tx=0x7f606400e860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:58.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.830+0000 7f607359e700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6054066730 0x7f6054068bf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f606c00b370 tx=0x7f606c011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:58.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.830+0000 7f606b7fe700 1 -- 172.21.15.40:0/2696948333 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60640052e0 con 0x7f6058002a40 2024-09-13T17:53:58.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.830+0000 7f606b7fe700 1 -- 172.21.15.40:0/2696948333 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6064005440 con 0x7f6058002a40 2024-09-13T17:53:58.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.830+0000 7f606b7fe700 1 -- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6058002a40 msgr2=0x7f6058004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.830+0000 7f606b7fe700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6058002a40 0x7f6058004f00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f607415c6c0 tx=0x7f606400e860 comp rx=0 tx=0).stop 2024-09-13T17:53:58.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.830+0000 7f606b7fe700 1 -- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6058002a40 msgr2=0x7f6058004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:53:58.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.830+0000 7f606b7fe700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6058002a40 0x7f6058004f00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.838+0000 7f6078830700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6058006fd0 0x7f6058007440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.838+0000 7f6078830700 1 -- 172.21.15.40:0/2696948333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605800a400 con 0x7f6058006fd0 2024-09-13T17:53:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.838+0000 7f606bfff700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6058006fd0 0x7f6058007440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.838+0000 7f606bfff700 1 -- 172.21.15.40:0/2696948333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f605800a5c0 con 0x7f6058006fd0 2024-09-13T17:53:58.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.838+0000 7f606bfff700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6058006fd0 0x7f6058007440 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6064015000 tx=0x7f6064001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:58.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.838+0000 7f606b7fe700 1 -- 172.21.15.40:0/2696948333 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6064021070 con 0x7f6058006fd0 2024-09-13T17:53:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.838+0000 7f606b7fe700 1 -- 172.21.15.40:0/2696948333 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4947 (secure 0 0 0) 0x7f6064003e10 con 0x7f6058006fd0 2024-09-13T17:53:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.838+0000 7f606b7fe700 1 -- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6058006fd0 msgr2=0x7f6058007440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.838+0000 7f606b7fe700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6058006fd0 0x7f6058007440 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6064015000 tx=0x7f6064001120 comp rx=0 tx=0).stop 2024-09-13T17:53:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.838+0000 7f606b7fe700 1 -- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6058006fd0 msgr2=0x7f6058007440 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:53:58.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.838+0000 7f606b7fe700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6058006fd0 0x7f6058007440 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.847 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:53:58.847 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:53:58.848 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:53:58.848 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:53:58.848 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 34, 2024-09-13T17:53:58.848 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:53:58.848 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:53:58.848 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:53:58.848 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:53:58.848 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:58.848 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 3, 2024-09-13T17:53:58.848 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:53:58.848 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:53:58.849 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:53:58.849 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:58.849 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:58.849 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:58.849 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:58.849 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:58.849 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:58.849 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:58.849 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:58.849 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:58.849 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:58.849 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:58.849 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:53:58.849 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:53:58.850 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:58.850 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:58.850 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:58.850 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:58.850 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:58.850 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:58.851 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:58.851 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:58.851 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:58.851 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:58.851 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:58.851 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.851 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:53:58.851 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:53:58.851 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:53:58.851 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:53:58.851 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:53:58.851 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:53:58.851 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:53:58.851 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:53:58.852 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:53:58.852 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:53:58.852 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:53:58.852 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:53:58.852 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:53:58.852 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:53:58.852 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:53:58.852 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:53:58.852 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:53:58.852 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:53:58.852 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:53:58.852 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:53:58.852 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:53:58.852 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:53:58.853 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:53:58.853 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:53:58.853 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:53:58.853 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:53:58.853 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:53:58.853 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:58.853 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:53:58.853 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.853 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:53:58.853 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.853 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:53:58.853 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:53:58.853 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:58.854 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:58.854 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.854 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:58.854 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:53:58.854 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:58.854 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.854 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.854 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:58.854 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:53:58.854 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:58.854 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:58.854 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:58.854 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.854 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:53:58.855 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:53:58.855 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:58.855 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:58.855 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:58.855 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.855 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.855 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:53:58.855 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:53:58.855 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:58.855 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:58.855 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.855 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:58.855 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:53:58.855 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:58.855 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.856 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.856 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:58.856 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:53:58.856 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:58.856 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:58.856 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:58.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.856 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:53:58.856 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:53:58.856 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:58.856 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:58.856 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:58.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.857 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.857 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:53:58.857 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:53:58.857 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:53:58.857 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:53:58.857 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.857 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:53:58.857 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:53:58.857 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:58.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.857 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.857 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:53:58.857 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:53:58.857 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:53:58.858 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:58.858 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:58.858 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.858 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:53:58.858 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:53:58.858 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:53:58.858 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:53:58.858 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:53:58.858 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:58.858 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:58.858 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.858 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:53:58.858 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:53:58.858 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.858 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:58.859 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:58.859 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:53:58.859 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:58.859 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:58.859 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:53:58.859 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.859 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:58.859 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:58.859 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:53:58.859 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:58.859 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:58.859 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:53:58.859 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.859 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:58.859 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:58.860 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:53:58.860 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:58.860 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:58.860 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:53:58.860 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.860 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:58.860 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:58.860 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:53:58.860 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:58.860 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:53:58.860 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:53:58.860 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:53:58.860 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:53:58.861 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:53:58.861 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:53:58.861 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:53:58.861 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:53:58.861 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:53:58.861 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:53:58.861 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:53:58.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.842+0000 7f6078830700 1 -- 172.21.15.40:0/2696948333 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6054066730 msgr2=0x7f6054068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.842+0000 7f6078830700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6054066730 0x7f6054068bf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f606c00b370 tx=0x7f606c011000 comp rx=0 tx=0).stop 2024-09-13T17:53:58.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.842+0000 7f6078830700 1 -- 172.21.15.40:0/2696948333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60740724d0 msgr2=0x7f6074154df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.842+0000 7f6078830700 1 --2- 172.21.15.40:0/2696948333 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60740724d0 0x7f6074154df0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f605c00c540 tx=0x7f605c00c900 comp rx=0 tx=0).stop 2024-09-13T17:53:58.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.842+0000 7f607359e700 1 -- 172.21.15.40:0/2696948333 reap_dead start 2024-09-13T17:53:58.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.842+0000 7f607359e700 1 -- 172.21.15.40:0/2696948333 reap_dead start 2024-09-13T17:53:58.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.842+0000 7f6078830700 1 -- 172.21.15.40:0/2696948333 shutdown_connections 2024-09-13T17:53:58.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.842+0000 7f6078830700 1 -- 172.21.15.40:0/2696948333 >> 172.21.15.40:0/2696948333 conn(0x7f607406a390 msgr2=0x7f60740c9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:58.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.842+0000 7f6078830700 1 -- 172.21.15.40:0/2696948333 shutdown_connections 2024-09-13T17:53:58.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.842+0000 7f6078830700 1 -- 172.21.15.40:0/2696948333 wait complete. 2024-09-13T17:53:58.864 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:53:58.864 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T17:53:58.864 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:53:58.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.970+0000 7f0b2b59e700 1 Processor -- start 2024-09-13T17:53:58.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.970+0000 7f0b2b59e700 1 -- start start 2024-09-13T17:53:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2b59e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0b2c0cf1a0 0x7f0b2c15a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2b59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c0cc250 0x7f0b2c0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2b59e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b2c15a7a0 con 0x7f0b2c0cf580 2024-09-13T17:53:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2b59e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b2c0cbdc0 con 0x7f0b2c0cf1a0 2024-09-13T17:53:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2b59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c153f80 con 0x7f0b2c0cc250 2024-09-13T17:53:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2a59c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0b2c0cf1a0 0x7f0b2c15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2ad9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c0cc250 0x7f0b2c0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2ad9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c0cc250 0x7f0b2c0c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:58192/0 (socket says 172.21.15.40:58192) 2024-09-13T17:53:58.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2ad9d700 1 -- 172.21.15.40:0/641933548 learned_addr learned my addr 172.21.15.40:0/641933548 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:58.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b29d9b700 1 --1- 172.21.15.40:0/641933548 >> v1:172.21.15.43:6789/0 conn(0x7f0b2c0cf580 0x7f0b2c0c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:54592/0 (socket says 172.21.15.40:54592) 2024-09-13T17:53:58.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2ad9d700 1 -- 172.21.15.40:0/641933548 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c0cc250 msgr2=0x7f0b2c0c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:53:58.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2ad9d700 1 -- 172.21.15.40:0/641933548 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c0cc250 msgr2=0x7f0b2c0c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:53:58.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2ad9d700 1 --2- 172.21.15.40:0/641933548 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c0cc250 0x7f0b2c0c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:58.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2ad9d700 1 --2- 172.21.15.40:0/641933548 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c0cc250 0x7f0b2c0c8aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:58.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2a59c700 1 -- 172.21.15.40:0/641933548 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c0cc250 msgr2=0x7f0b2c0c8aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:53:58.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2a59c700 1 --2- 172.21.15.40:0/641933548 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c0cc250 0x7f0b2c0c8aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2a59c700 1 -- 172.21.15.40:0/641933548 >> v1:172.21.15.43:6789/0 conn(0x7f0b2c0cf580 legacy=0x7f0b2c0c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2959a700 1 -- 172.21.15.40:0/641933548 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2646881236 0 0) 0x7f0b20001110 con 0x7f0b2c0cf580 2024-09-13T17:53:58.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2a59c700 1 -- 172.21.15.40:0/641933548 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2c0caed0 con 0x7f0b2c0cf1a0 2024-09-13T17:53:58.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2a59c700 1 --2- 172.21.15.40:0/641933548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c0cf1a0 0x7f0b2c15a260 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009d90 tx=0x7f0b1c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7acf303985bd31b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:58.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2959a700 1 -- 172.21.15.40:0/641933548 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c00e070 con 0x7f0b2c0cf1a0 2024-09-13T17:53:58.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2959a700 1 -- 172.21.15.40:0/641933548 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1c00bbd0 con 0x7f0b2c0cf1a0 2024-09-13T17:53:58.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.974+0000 7f0b2959a700 1 -- 172.21.15.40:0/641933548 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c012430 con 0x7f0b2c0cf1a0 2024-09-13T17:53:58.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.978+0000 7f0b2b59e700 1 -- 172.21.15.40:0/641933548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c0cf1a0 msgr2=0x7f0b2c15a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.978+0000 7f0b2b59e700 1 --2- 172.21.15.40:0/641933548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c0cf1a0 0x7f0b2c15a260 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009d90 tx=0x7f0b1c00b330 comp rx=0 tx=0).stop 2024-09-13T17:53:58.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.978+0000 7f0b2b59e700 1 -- 172.21.15.40:0/641933548 shutdown_connections 2024-09-13T17:53:58.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.978+0000 7f0b2b59e700 1 --2- 172.21.15.40:0/641933548 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c0cc250 0x7f0b2c0c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.978+0000 7f0b2b59e700 1 --2- 172.21.15.40:0/641933548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c0cf1a0 0x7f0b2c15a260 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.978+0000 7f0b2b59e700 1 -- 172.21.15.40:0/641933548 >> 172.21.15.40:0/641933548 conn(0x7f0b2c066090 msgr2=0x7f0b2c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:53:58.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.978+0000 7f0b2b59e700 1 -- 172.21.15.40:0/641933548 shutdown_connections 2024-09-13T17:53:58.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.978+0000 7f0b2b59e700 1 -- 172.21.15.40:0/641933548 wait complete. 2024-09-13T17:53:58.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.978+0000 7f0b2b59e700 1 Processor -- start 2024-09-13T17:53:58.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.978+0000 7f0b2b59e700 1 -- start start 2024-09-13T17:53:58.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2b59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b2c0cc250 0x7f0b2c1ef050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2b59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c0cf580 0x7f0b2c1e5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2b59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1f3240 0x7f0b2c1e61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2b59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c157210 con 0x7f0b2c0cc250 2024-09-13T17:53:58.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2b59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c1f2e20 con 0x7f0b2c0cf580 2024-09-13T17:53:58.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2b59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c156a20 con 0x7f0b2c1f3240 2024-09-13T17:53:58.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b2c0cc250 0x7f0b2c1ef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b2c0cc250 0x7f0b2c1ef050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57006/0 (socket says 172.21.15.40:57006) 2024-09-13T17:53:58.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2a59c700 1 -- 172.21.15.40:0/3358434502 learned_addr learned my addr 172.21.15.40:0/3358434502 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:53:58.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b29d9b700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c0cf580 0x7f0b2c1e5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2ad9d700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1f3240 0x7f0b2c1e61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2ad9d700 1 -- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1f3240 msgr2=0x7f0b2c1e61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-13T17:53:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2ad9d700 1 -- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1f3240 msgr2=0x7f0b2c1e61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:53:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2ad9d700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1f3240 0x7f0b2c1e61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2ad9d700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1f3240 0x7f0b2c1e61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2a59c700 1 -- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1f3240 msgr2=0x7f0b2c1e61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:53:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2a59c700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1f3240 0x7f0b2c1e61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2a59c700 1 -- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c0cf580 msgr2=0x7f0b2c1e5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:53:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2a59c700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b2c0cf580 0x7f0b2c1e5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:53:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2a59c700 1 -- 172.21.15.40:0/3358434502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1c009a70 con 0x7f0b2c0cc250 2024-09-13T17:53:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2a59c700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b2c0cc250 0x7f0b2c1ef050 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c013000 tx=0x7f0b1c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b1affd700 1 -- 172.21.15.40:0/3358434502 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c012430 con 0x7f0b2c0cc250 2024-09-13T17:53:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b1affd700 1 -- 172.21.15.40:0/3358434502 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1c020070 con 0x7f0b2c0cc250 2024-09-13T17:53:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2b59e700 1 -- 172.21.15.40:0/3358434502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b2c156c50 con 0x7f0b2c0cc250 2024-09-13T17:53:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b1affd700 1 -- 172.21.15.40:0/3358434502 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c004280 con 0x7f0b2c0cc250 2024-09-13T17:53:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2b59e700 1 -- 172.21.15.40:0/3358434502 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b2c1f2760 con 0x7f0b2c0cc250 2024-09-13T17:53:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2b59e700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 0x7f0b2c152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b2b59e700 1 -- 172.21.15.40:0/3358434502 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c074750 con 0x7f0b2c1e19f0 2024-09-13T17:53:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b29d9b700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 0x7f0b2c152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b29d9b700 1 -- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 msgr2=0x7f0b2c152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-09-13T17:53:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b29d9b700 1 -- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 msgr2=0x7f0b2c152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:53:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b29d9b700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 0x7f0b2c152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.982+0000 7f0b29d9b700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 0x7f0b2c152cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:53:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.986+0000 7f0b1affd700 1 -- 172.21.15.40:0/3358434502 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0b1c0043e0 con 0x7f0b2c0cc250 2024-09-13T17:53:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.986+0000 7f0b1affd700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0b0c066730 0x7f0b0c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:53:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.986+0000 7f0b1affd700 1 -- 172.21.15.40:0/3358434502 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0b1c022070 con 0x7f0b2c0cc250 2024-09-13T17:53:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.986+0000 7f0b2ad9d700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0b0c066730 0x7f0b0c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:58.986+0000 7f0b2ad9d700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0b0c066730 0x7f0b0c068bf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0b240091a0 tx=0x7f0b24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:53:59.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:59.186+0000 7f0b29d9b700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 0x7f0b2c152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:59.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:59.186+0000 7f0b29d9b700 1 -- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 msgr2=0x7f0b2c152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-13T17:53:59.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:59.186+0000 7f0b29d9b700 1 -- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 msgr2=0x7f0b2c152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:53:59.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:59.186+0000 7f0b29d9b700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 0x7f0b2c152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:59.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:59.186+0000 7f0b29d9b700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 0x7f0b2c152cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:53:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:59.590+0000 7f0b29d9b700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 0x7f0b2c152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:53:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:59.590+0000 7f0b29d9b700 1 -- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 msgr2=0x7f0b2c152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-13T17:53:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:59.590+0000 7f0b29d9b700 1 -- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 msgr2=0x7f0b2c152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:53:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:59.590+0000 7f0b29d9b700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 0x7f0b2c152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:53:59.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:53:59.590+0000 7f0b29d9b700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 0x7f0b2c152cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:54:00.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.394+0000 7f0b29d9b700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 0x7f0b2c152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:00.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.394+0000 7f0b29d9b700 1 -- 172.21.15.40:0/3358434502 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b2c0cb770 con 0x7f0b2c1e19f0 2024-09-13T17:54:00.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.394+0000 7f0b29d9b700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 0x7f0b2c152cc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0c5320 tx=0x7f0b20001870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:00.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.394+0000 7f0b1affd700 1 -- 172.21.15.40:0/3358434502 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b20017640 con 0x7f0b2c1e19f0 2024-09-13T17:54:00.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.394+0000 7f0b1affd700 1 -- 172.21.15.40:0/3358434502 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0b200177a0 con 0x7f0b2c1e19f0 2024-09-13T17:54:00.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.394+0000 7f0b1affd700 1 -- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 msgr2=0x7f0b2c152cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:00.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.394+0000 7f0b1affd700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 0x7f0b2c152cc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0c5320 tx=0x7f0b20001870 comp rx=0 tx=0).stop 2024-09-13T17:54:00.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.394+0000 7f0b1affd700 1 -- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 msgr2=0x7f0b2c152cc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:54:00.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.394+0000 7f0b1affd700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c1e19f0 0x7f0b2c152cc0 secure :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0c5320 tx=0x7f0b20001870 comp rx=0 tx=0).stop 2024-09-13T17:54:00.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.406+0000 7f0b2b59e700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c153200 0x7f0b2c1551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:00.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.406+0000 7f0b2b59e700 1 -- 172.21.15.40:0/3358434502 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c0cb490 con 0x7f0b2c153200 2024-09-13T17:54:00.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.406+0000 7f0b29d9b700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c153200 0x7f0b2c1551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:00.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.410+0000 7f0b29d9b700 1 -- 172.21.15.40:0/3358434502 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0b2c0cba50 con 0x7f0b2c153200 2024-09-13T17:54:00.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.410+0000 7f0b29d9b700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c153200 0x7f0b2c1551a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0b1870 tx=0x7f0b20000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:00.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.410+0000 7f0b1affd700 1 -- 172.21.15.40:0/3358434502 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b20027070 con 0x7f0b2c153200 2024-09-13T17:54:00.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.410+0000 7f0b1affd700 1 -- 172.21.15.40:0/3358434502 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4449 (secure 0 0 0) 0x7f0b20017e80 con 0x7f0b2c153200 2024-09-13T17:54:00.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.410+0000 7f0b1affd700 1 -- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c153200 msgr2=0x7f0b2c1551a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:00.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.410+0000 7f0b1affd700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c153200 0x7f0b2c1551a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0b1870 tx=0x7f0b20000ed0 comp rx=0 tx=0).stop 2024-09-13T17:54:00.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.410+0000 7f0b1affd700 1 -- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c153200 msgr2=0x7f0b2c1551a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:54:00.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.410+0000 7f0b1affd700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b2c153200 0x7f0b2c1551a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:00.418 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:54:00.418 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:54:00.418 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:54:00.418 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:54:00.418 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 34, 2024-09-13T17:54:00.418 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:54:00.418 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:54:00.418 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:54:00.418 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:54:00.418 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:00.419 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 4, 2024-09-13T17:54:00.419 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:54:00.419 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:54:00.419 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:54:00.419 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:00.419 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:00.419 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:00.419 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:00.419 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:00.419 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:00.419 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:00.419 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:00.419 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:00.419 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:00.420 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:00.420 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:54:00.420 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:54:00.420 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:00.420 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:00.420 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:00.420 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:00.420 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:00.420 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:00.420 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:00.420 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:00.420 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:00.420 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:00.421 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:00.421 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:00.421 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:54:00.421 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:54:00.421 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:54:00.421 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:54:00.421 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:54:00.421 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:54:00.421 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:54:00.421 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:54:00.421 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:54:00.421 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:54:00.421 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:54:00.421 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:54:00.422 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:54:00.422 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:54:00.422 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:54:00.422 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:54:00.422 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:54:00.422 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:00.422 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:00.422 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:00.422 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:00.422 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:00.422 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:00.422 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:00.422 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:00.423 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:00.423 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:00.423 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:00.423 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:54:00.423 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:00.423 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:54:00.423 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:00.423 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:54:00.423 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:54:00.423 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:00.423 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:00.423 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:00.423 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:00.424 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:54:00.424 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:00.424 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:00.424 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:00.424 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:00.424 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:54:00.424 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:00.424 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:00.424 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:00.424 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:00.424 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:54:00.424 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:54:00.424 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:00.425 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:00.425 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:00.425 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:00.425 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:00.425 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:54:00.425 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:54:00.425 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:00.425 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:00.425 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:00.425 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:00.425 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:54:00.425 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:00.425 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:00.425 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:00.426 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:00.426 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:54:00.426 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:00.426 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:00.426 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:00.426 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:00.426 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:54:00.426 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:54:00.426 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:00.426 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:00.426 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:00.426 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:00.426 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:00.426 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:54:00.427 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:54:00.427 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:00.427 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:00.427 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:00.427 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:00.427 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:54:00.427 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:00.427 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:00.427 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:00.427 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:00.427 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:54:00.427 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:00.427 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:00.428 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:00.428 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:00.428 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:54:00.428 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:54:00.428 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:00.428 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:00.428 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:00.428 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:00.428 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:00.428 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:00.428 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:54:00.428 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:54:00.428 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:00.428 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:00.429 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:00.429 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:54:00.429 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:00.429 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:00.429 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:54:00.429 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:00.429 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:00.429 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:00.429 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:54:00.429 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:00.429 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:00.429 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:00.429 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:54:00.429 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:54:00.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.410+0000 7f0b2b59e700 1 -- 172.21.15.40:0/3358434502 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0b0c066730 msgr2=0x7f0b0c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:00.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.410+0000 7f0b2b59e700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0b0c066730 0x7f0b0c068bf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0b240091a0 tx=0x7f0b24008040 comp rx=0 tx=0).stop 2024-09-13T17:54:00.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.410+0000 7f0b2b59e700 1 -- 172.21.15.40:0/3358434502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b2c0cc250 msgr2=0x7f0b2c1ef050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:00.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.410+0000 7f0b2ad9d700 1 -- 172.21.15.40:0/3358434502 reap_dead start 2024-09-13T17:54:00.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.410+0000 7f0b2b59e700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b2c0cc250 0x7f0b2c1ef050 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c013000 tx=0x7f0b1c004af0 comp rx=0 tx=0).stop 2024-09-13T17:54:00.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.410+0000 7f0b2b59e700 1 -- 172.21.15.40:0/3358434502 shutdown_connections 2024-09-13T17:54:00.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.410+0000 7f0b2b59e700 1 --2- 172.21.15.40:0/3358434502 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b2c0cc250 0x7f0b2c1ef050 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:00.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.410+0000 7f0b2b59e700 1 -- 172.21.15.40:0/3358434502 >> 172.21.15.40:0/3358434502 conn(0x7f0b2c066090 msgr2=0x7f0b2c0cd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:00.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.410+0000 7f0b2b59e700 1 -- 172.21.15.40:0/3358434502 shutdown_connections 2024-09-13T17:54:00.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.410+0000 7f0b2b59e700 1 -- 172.21.15.40:0/3358434502 wait complete. 2024-09-13T17:54:00.434 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:54:00.434 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T17:54:00.434 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T17:54:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.550+0000 7f065f59e700 1 Processor -- start 2024-09-13T17:54:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.550+0000 7f065f59e700 1 -- start start 2024-09-13T17:54:00.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.550+0000 7f065f59e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0660071e60 0x7f06601571d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:00.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.550+0000 7f065f59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0660072240 0x7f06600c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:00.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.550+0000 7f065f59e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06600c67e0 con 0x7f0660157710 2024-09-13T17:54:00.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.550+0000 7f065f59e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06600c4d80 con 0x7f0660071e60 2024-09-13T17:54:00.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.550+0000 7f065f59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06600c3420 con 0x7f0660072240 2024-09-13T17:54:00.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.550+0000 7f065dd9b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0660072240 0x7f06600c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:00.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.550+0000 7f065e59c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0660071e60 0x7f06601571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:00.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.550+0000 7f065dd9b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0660072240 0x7f06600c51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:58252/0 (socket says 172.21.15.40:58252) 2024-09-13T17:54:00.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.550+0000 7f065dd9b700 1 -- 172.21.15.40:0/3822813952 learned_addr learned my addr 172.21.15.40:0/3822813952 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:00.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.550+0000 7f065dd9b700 1 -- 172.21.15.40:0/3822813952 >> v2:172.21.15.43:3300/0 conn(0x7f0660071e60 msgr2=0x7f06601571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:00.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.550+0000 7f065dd9b700 1 --2- 172.21.15.40:0/3822813952 >> v2:172.21.15.43:3300/0 conn(0x7f0660071e60 0x7f06601571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.550+0000 7f065dd9b700 1 -- 172.21.15.40:0/3822813952 >> v1:172.21.15.43:6789/0 conn(0x7f0660157710 legacy=0x7f0660151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:00.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.550+0000 7f065dd9b700 1 -- 172.21.15.40:0/3822813952 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0660157c80 con 0x7f0660072240 2024-09-13T17:54:00.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.554+0000 7f065dd9b700 1 --2- 172.21.15.40:0/3822813952 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0660072240 0x7f06600c51a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0654009910 tx=0x7f0654009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=222ecbc8d3cf27d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:00.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.554+0000 7f065d59a700 1 -- 172.21.15.40:0/3822813952 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654011830 con 0x7f0660072240 2024-09-13T17:54:00.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.554+0000 7f065d59a700 1 -- 172.21.15.40:0/3822813952 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0654011990 con 0x7f0660072240 2024-09-13T17:54:00.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.554+0000 7f065d59a700 1 -- 172.21.15.40:0/3822813952 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654011d40 con 0x7f0660072240 2024-09-13T17:54:00.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.554+0000 7f065f59e700 1 -- 172.21.15.40:0/3822813952 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0660072240 msgr2=0x7f06600c51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:00.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.554+0000 7f065f59e700 1 --2- 172.21.15.40:0/3822813952 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0660072240 0x7f06600c51a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0654009910 tx=0x7f0654009c20 comp rx=0 tx=0).stop 2024-09-13T17:54:00.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.554+0000 7f065f59e700 1 -- 172.21.15.40:0/3822813952 shutdown_connections 2024-09-13T17:54:00.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.554+0000 7f065f59e700 1 --2- 172.21.15.40:0/3822813952 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0660072240 0x7f06600c51a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:00.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.554+0000 7f065f59e700 1 --2- 172.21.15.40:0/3822813952 >> v2:172.21.15.43:3300/0 conn(0x7f0660071e60 0x7f06601571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:00.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.554+0000 7f065f59e700 1 -- 172.21.15.40:0/3822813952 >> 172.21.15.40:0/3822813952 conn(0x7f066006e4f0 msgr2=0x7f066006e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:00.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.554+0000 7f065f59e700 1 -- 172.21.15.40:0/3822813952 shutdown_connections 2024-09-13T17:54:00.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.554+0000 7f065f59e700 1 -- 172.21.15.40:0/3822813952 wait complete. 2024-09-13T17:54:00.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.554+0000 7f065f59e700 1 Processor -- start 2024-09-13T17:54:00.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.554+0000 7f065f59e700 1 -- start start 2024-09-13T17:54:00.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.554+0000 7f065f59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0660071e60 0x7f0660146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:00.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.554+0000 7f065f59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0660072240 0x7f0660155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:00.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.554+0000 7f065f59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0660157710 0x7f0660155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:00.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.554+0000 7f065f59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06600c2820 con 0x7f0660072240 2024-09-13T17:54:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.554+0000 7f065f59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066006ce20 con 0x7f0660071e60 2024-09-13T17:54:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.554+0000 7f065f59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06600c2ac0 con 0x7f0660157710 2024-09-13T17:54:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.558+0000 7f065dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0660072240 0x7f0660155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.558+0000 7f065dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0660072240 0x7f0660155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57018/0 (socket says 172.21.15.40:57018) 2024-09-13T17:54:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.558+0000 7f065e59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0660071e60 0x7f0660146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.558+0000 7f065dd9b700 1 -- 172.21.15.40:0/3968089599 learned_addr learned my addr 172.21.15.40:0/3968089599 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.558+0000 7f065ed9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0660157710 0x7f0660155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.558+0000 7f065dd9b700 1 -- 172.21.15.40:0/3968089599 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0660157710 msgr2=0x7f0660155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.558+0000 7f065dd9b700 1 --2- 172.21.15.40:0/3968089599 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0660157710 0x7f0660155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.558+0000 7f065dd9b700 1 -- 172.21.15.40:0/3968089599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0660071e60 msgr2=0x7f0660146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.558+0000 7f065dd9b700 1 --2- 172.21.15.40:0/3968089599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0660071e60 0x7f0660146a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.558+0000 7f065dd9b700 1 -- 172.21.15.40:0/3968089599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0654009570 con 0x7f0660072240 2024-09-13T17:54:00.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.558+0000 7f065dd9b700 1 --2- 172.21.15.40:0/3968089599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0660072240 0x7f0660155400 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0654000c00 tx=0x7f0654004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.558+0000 7f064effd700 1 -- 172.21.15.40:0/3968089599 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654004510 con 0x7f0660072240 2024-09-13T17:54:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.558+0000 7f064effd700 1 -- 172.21.15.40:0/3968089599 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06540036c0 con 0x7f0660072240 2024-09-13T17:54:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.558+0000 7f065f59e700 1 -- 172.21.15.40:0/3968089599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066006d0c0 con 0x7f0660072240 2024-09-13T17:54:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.558+0000 7f064effd700 1 -- 172.21.15.40:0/3968089599 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654003a70 con 0x7f0660072240 2024-09-13T17:54:00.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.558+0000 7f065f59e700 1 -- 172.21.15.40:0/3968089599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066006d680 con 0x7f0660072240 2024-09-13T17:54:00.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.558+0000 7f065f59e700 1 -- 172.21.15.40:0/3968089599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0644005310 con 0x7f0660072240 2024-09-13T17:54:00.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.558+0000 7f064effd700 1 -- 172.21.15.40:0/3968089599 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0654003bd0 con 0x7f0660072240 2024-09-13T17:54:00.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.562+0000 7f064effd700 1 --2- 172.21.15.40:0/3968089599 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0640066760 0x7f0640068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:00.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.562+0000 7f064effd700 1 -- 172.21.15.40:0/3968089599 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0654021030 con 0x7f0660072240 2024-09-13T17:54:00.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.562+0000 7f065e59c700 1 --2- 172.21.15.40:0/3968089599 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0640066760 0x7f0640068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:00.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.562+0000 7f065e59c700 1 --2- 172.21.15.40:0/3968089599 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0640066760 0x7f0640068c20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0650009bf0 tx=0x7f0650009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:00.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.566+0000 7f064effd700 1 -- 172.21.15.40:0/3968089599 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0654052a00 con 0x7f0660072240 2024-09-13T17:54:00.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.850+0000 7f065f59e700 1 -- 172.21.15.40:0/3968089599 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f0644005890 con 0x7f0660072240 2024-09-13T17:54:00.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.850+0000 7f064effd700 1 -- 172.21.15.40:0/3968089599 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f0654015080 con 0x7f0660072240 2024-09-13T17:54:00.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.854+0000 7f065f59e700 1 -- 172.21.15.40:0/3968089599 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0640066760 msgr2=0x7f0640068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:00.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.854+0000 7f065f59e700 1 --2- 172.21.15.40:0/3968089599 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0640066760 0x7f0640068c20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0650009bf0 tx=0x7f0650009240 comp rx=0 tx=0).stop 2024-09-13T17:54:00.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.854+0000 7f065f59e700 1 -- 172.21.15.40:0/3968089599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0660072240 msgr2=0x7f0660155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:00.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.854+0000 7f065f59e700 1 --2- 172.21.15.40:0/3968089599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0660072240 0x7f0660155400 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0654000c00 tx=0x7f0654004770 comp rx=0 tx=0).stop 2024-09-13T17:54:00.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.854+0000 7f065f59e700 1 -- 172.21.15.40:0/3968089599 shutdown_connections 2024-09-13T17:54:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.854+0000 7f065f59e700 1 --2- 172.21.15.40:0/3968089599 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0640066760 0x7f0640068c20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.854+0000 7f065f59e700 1 --2- 172.21.15.40:0/3968089599 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0660157710 0x7f0660155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.854+0000 7f065f59e700 1 --2- 172.21.15.40:0/3968089599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0660072240 0x7f0660155400 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.854+0000 7f065f59e700 1 --2- 172.21.15.40:0/3968089599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0660071e60 0x7f0660146a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.854+0000 7f065f59e700 1 -- 172.21.15.40:0/3968089599 >> 172.21.15.40:0/3968089599 conn(0x7f066006e4f0 msgr2=0x7f066006e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.854+0000 7f065f59e700 1 -- 172.21.15.40:0/3968089599 shutdown_connections 2024-09-13T17:54:00.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:00.854+0000 7f065f59e700 1 -- 172.21.15.40:0/3968089599 wait complete. 2024-09-13T17:54:00.873 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T17:54:04.166 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:54:04.168 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:54:10.874 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T17:54:10.874 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:54:10.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.989+0000 7f581f0a0700 1 Processor -- start 2024-09-13T17:54:10.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.989+0000 7f581f0a0700 1 -- start start 2024-09-13T17:54:11.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581f0a0700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f581000b270 0x7f58100e01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581f0a0700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58100470a0 con 0x7f58100d65b0 2024-09-13T17:54:11.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581f0a0700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5810047380 con 0x7f581000b760 2024-09-13T17:54:11.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581f0a0700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581000a640 con 0x7f581000b270 2024-09-13T17:54:11.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581e09e700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f58100d65b0 0x7f5810008860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43826/0 (socket says 172.21.15.40:43826) 2024-09-13T17:54:11.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581e09e700 1 -- 172.21.15.40:0/1782723430 learned_addr learned my addr 172.21.15.40:0/1782723430 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:54:11.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581d89d700 1 --2- 172.21.15.40:0/1782723430 >> v2:172.21.15.43:3300/0 conn(0x7f581000b270 0x7f58100e01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581d09c700 1 -- 172.21.15.40:0/1782723430 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 237989136 0 0) 0x7f58100470a0 con 0x7f58100d65b0 2024-09-13T17:54:11.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581d09c700 1 -- 172.21.15.40:0/1782723430 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58040035d0 con 0x7f58100d65b0 2024-09-13T17:54:11.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581d09c700 1 -- 172.21.15.40:0/1782723430 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2246029467 0 0) 0x7f58040035d0 con 0x7f58100d65b0 2024-09-13T17:54:11.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581d09c700 1 -- 172.21.15.40:0/1782723430 >> v2:172.21.15.43:3300/0 conn(0x7f581000b270 msgr2=0x7f58100e01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:11.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581d09c700 1 --2- 172.21.15.40:0/1782723430 >> v2:172.21.15.43:3300/0 conn(0x7f581000b270 0x7f58100e01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581d09c700 1 -- 172.21.15.40:0/1782723430 >> v1:172.21.15.43:6789/0 conn(0x7f581000b760 legacy=0x7f58100e6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581d09c700 1 -- 172.21.15.40:0/1782723430 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5810047660 con 0x7f58100d65b0 2024-09-13T17:54:11.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581d09c700 1 -- 172.21.15.40:0/1782723430 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f5808001670 con 0x7f58100d65b0 2024-09-13T17:54:11.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581d09c700 1 -- 172.21.15.40:0/1782723430 >> v1:172.21.15.40:6789/0 conn(0x7f58100d65b0 legacy=0x7f5810008860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581d09c700 1 --2- 172.21.15.40:0/1782723430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5804003a20 0x7f5804003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581d09c700 1 --2- 172.21.15.40:0/1782723430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5804004870 0x7f5804004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581d09c700 1 --2- 172.21.15.40:0/1782723430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5804005230 0x7f580400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581d09c700 1 -- 172.21.15.40:0/1782723430 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581000a640 con 0x7f5804003a20 2024-09-13T17:54:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581d09c700 1 -- 172.21.15.40:0/1782723430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580400c080 con 0x7f5804004870 2024-09-13T17:54:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581d09c700 1 -- 172.21.15.40:0/1782723430 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580400c240 con 0x7f5804005230 2024-09-13T17:54:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581d89d700 1 --2- 172.21.15.40:0/1782723430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5804004870 0x7f5804004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581e09e700 1 --2- 172.21.15.40:0/1782723430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5804003a20 0x7f5804003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581e89f700 1 --2- 172.21.15.40:0/1782723430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5804005230 0x7f580400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581e09e700 1 -- 172.21.15.40:0/1782723430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5804005230 msgr2=0x7f580400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581e09e700 1 --2- 172.21.15.40:0/1782723430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5804005230 0x7f580400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581e09e700 1 -- 172.21.15.40:0/1782723430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5804004870 msgr2=0x7f5804004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581e09e700 1 --2- 172.21.15.40:0/1782723430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5804004870 0x7f5804004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581e09e700 1 -- 172.21.15.40:0/1782723430 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58040035d0 con 0x7f5804003a20 2024-09-13T17:54:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.993+0000 7f581e89f700 1 -- 172.21.15.40:0/1782723430 reap_dead start 2024-09-13T17:54:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.997+0000 7f581e09e700 1 --2- 172.21.15.40:0/1782723430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5804003a20 0x7f5804003e40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f580800c2a0 tx=0x7f58040058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c8f58cffc1909d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.997+0000 7f581d09c700 1 -- 172.21.15.40:0/1782723430 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808007c80 con 0x7f5804003a20 2024-09-13T17:54:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.997+0000 7f581d09c700 1 -- 172.21.15.40:0/1782723430 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580800cd60 con 0x7f5804003a20 2024-09-13T17:54:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.997+0000 7f581d09c700 1 -- 172.21.15.40:0/1782723430 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808005bc0 con 0x7f5804003a20 2024-09-13T17:54:11.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.997+0000 7f581f0a0700 1 -- 172.21.15.40:0/1782723430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5804003a20 msgr2=0x7f5804003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.997+0000 7f581f0a0700 1 --2- 172.21.15.40:0/1782723430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5804003a20 0x7f5804003e40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f580800c2a0 tx=0x7f58040058a0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.997+0000 7f581f0a0700 1 -- 172.21.15.40:0/1782723430 shutdown_connections 2024-09-13T17:54:11.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.997+0000 7f581f0a0700 1 --2- 172.21.15.40:0/1782723430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5804003a20 0x7f5804003e40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.997+0000 7f581f0a0700 1 -- 172.21.15.40:0/1782723430 >> 172.21.15.40:0/1782723430 conn(0x7f581000a780 msgr2=0x7f581000ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:11.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.997+0000 7f581f0a0700 1 -- 172.21.15.40:0/1782723430 shutdown_connections 2024-09-13T17:54:11.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.997+0000 7f581f0a0700 1 -- 172.21.15.40:0/1782723430 wait complete. 2024-09-13T17:54:11.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.997+0000 7f581f0a0700 1 Processor -- start 2024-09-13T17:54:11.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:10.997+0000 7f581f0a0700 1 -- start start 2024-09-13T17:54:11.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f581f0a0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5804003a20 0x7f5810175610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f581f0a0700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5810006710 0x7f5810177f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f581f0a0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5810006bf0 0x7f58101694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f581f0a0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5810008a80 con 0x7f5804003a20 2024-09-13T17:54:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f581f0a0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5810034a60 con 0x7f5810006bf0 2024-09-13T17:54:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f581f0a0700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5810175180 con 0x7f5810006710 2024-09-13T17:54:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f581e09e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5804003a20 0x7f5810175610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f581e09e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5804003a20 0x7f5810175610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51526/0 (socket says 172.21.15.40:51526) 2024-09-13T17:54:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f581e09e700 1 -- 172.21.15.40:0/3002453204 learned_addr learned my addr 172.21.15.40:0/3002453204 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f581d89d700 1 --2- 172.21.15.40:0/3002453204 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5810006710 0x7f5810177f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f581e89f700 1 --2- 172.21.15.40:0/3002453204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5810006bf0 0x7f58101694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f581e09e700 1 -- 172.21.15.40:0/3002453204 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5810006710 msgr2=0x7f5810177f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f581e09e700 1 --2- 172.21.15.40:0/3002453204 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5810006710 0x7f5810177f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f581e09e700 1 -- 172.21.15.40:0/3002453204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5810006bf0 msgr2=0x7f58101694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f581e09e700 1 --2- 172.21.15.40:0/3002453204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5810006bf0 0x7f58101694e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f581e09e700 1 -- 172.21.15.40:0/3002453204 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58100359e0 con 0x7f5804003a20 2024-09-13T17:54:11.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f581e09e700 1 --2- 172.21.15.40:0/3002453204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5804003a20 0x7f5810175610 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f580800c890 tx=0x7f5808001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f580effd700 1 -- 172.21.15.40:0/3002453204 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808004ca0 con 0x7f5804003a20 2024-09-13T17:54:11.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f580effd700 1 -- 172.21.15.40:0/3002453204 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5808004e20 con 0x7f5804003a20 2024-09-13T17:54:11.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f581f0a0700 1 -- 172.21.15.40:0/3002453204 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5810035cc0 con 0x7f5804003a20 2024-09-13T17:54:11.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f580effd700 1 -- 172.21.15.40:0/3002453204 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808007310 con 0x7f5804003a20 2024-09-13T17:54:11.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f581f0a0700 1 -- 172.21.15.40:0/3002453204 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5810174e10 con 0x7f5804003a20 2024-09-13T17:54:11.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f581f0a0700 1 -- 172.21.15.40:0/3002453204 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5810174fa0 con 0x7f5804003a20 2024-09-13T17:54:11.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.001+0000 7f580effd700 1 -- 172.21.15.40:0/3002453204 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5808007470 con 0x7f5804003a20 2024-09-13T17:54:11.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.005+0000 7f580effd700 1 --2- 172.21.15.40:0/3002453204 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f57fc066730 0x7f57fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.005+0000 7f581d89d700 1 --2- 172.21.15.40:0/3002453204 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f57fc066730 0x7f57fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.005+0000 7f580effd700 1 -- 172.21.15.40:0/3002453204 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5808085460 con 0x7f5804003a20 2024-09-13T17:54:11.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.005+0000 7f581d89d700 1 --2- 172.21.15.40:0/3002453204 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f57fc066730 0x7f57fc068bf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f58100dcfd0 tx=0x7f5814009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.009+0000 7f580effd700 1 -- 172.21.15.40:0/3002453204 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5808052420 con 0x7f5804003a20 2024-09-13T17:54:11.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.265+0000 7f581f0a0700 1 -- 172.21.15.40:0/3002453204 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f58100361c0 con 0x7f5804003a20 2024-09-13T17:54:11.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.265+0000 7f580effd700 1 -- 172.21.15.40:0/3002453204 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f5808007de0 con 0x7f5804003a20 2024-09-13T17:54:11.275 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":34,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:54:11.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.269+0000 7f581f0a0700 1 -- 172.21.15.40:0/3002453204 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f57fc066730 msgr2=0x7f57fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.269+0000 7f581f0a0700 1 --2- 172.21.15.40:0/3002453204 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f57fc066730 0x7f57fc068bf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f58100dcfd0 tx=0x7f5814009240 comp rx=0 tx=0).stop 2024-09-13T17:54:11.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.269+0000 7f581f0a0700 1 -- 172.21.15.40:0/3002453204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5804003a20 msgr2=0x7f5810175610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.269+0000 7f581f0a0700 1 --2- 172.21.15.40:0/3002453204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5804003a20 0x7f5810175610 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f580800c890 tx=0x7f5808001a00 comp rx=0 tx=0).stop 2024-09-13T17:54:11.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.269+0000 7f581f0a0700 1 -- 172.21.15.40:0/3002453204 shutdown_connections 2024-09-13T17:54:11.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.269+0000 7f581f0a0700 1 --2- 172.21.15.40:0/3002453204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5810006bf0 0x7f58101694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.269+0000 7f581f0a0700 1 --2- 172.21.15.40:0/3002453204 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f57fc066730 0x7f57fc068bf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.269+0000 7f581f0a0700 1 --2- 172.21.15.40:0/3002453204 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5810006710 0x7f5810177f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.269+0000 7f581f0a0700 1 --2- 172.21.15.40:0/3002453204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5804003a20 0x7f5810175610 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.269+0000 7f581f0a0700 1 -- 172.21.15.40:0/3002453204 >> 172.21.15.40:0/3002453204 conn(0x7f581000a780 msgr2=0x7f5810050700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:11.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.269+0000 7f581f0a0700 1 -- 172.21.15.40:0/3002453204 shutdown_connections 2024-09-13T17:54:11.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.269+0000 7f581f0a0700 1 -- 172.21.15.40:0/3002453204 wait complete. 2024-09-13T17:54:11.290 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T17:54:11.290 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T17:54:11.290 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T17:54:11.290 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:54:11.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe73fff700 1 Processor -- start 2024-09-13T17:54:11.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe73fff700 1 -- start start 2024-09-13T17:54:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe73fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efe6c0d8920 0x7efe6c049090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe73fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe6c0e1c00 0x7efe6c0e2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe73fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6c054df0 con 0x7efe6c007480 2024-09-13T17:54:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe73fff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe6c050a10 con 0x7efe6c0d8920 2024-09-13T17:54:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe73fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c006ff0 con 0x7efe6c0e1c00 2024-09-13T17:54:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe727fc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efe6c007480 0x7efe6c04efb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57096/0 (socket says 172.21.15.40:57096) 2024-09-13T17:54:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe727fc700 1 -- 172.21.15.40:0/1091764129 learned_addr learned my addr 172.21.15.40:0/1091764129 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:54:11.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe72ffd700 1 --2- 172.21.15.40:0/1091764129 >> v2:172.21.15.43:3300/0 conn(0x7efe6c0d8920 0x7efe6c049090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe737fe700 1 --2- 172.21.15.40:0/1091764129 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe6c0e1c00 0x7efe6c0e2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe71ffb700 1 -- 172.21.15.40:0/1091764129 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 19134881 0 0) 0x7efe6c054df0 con 0x7efe6c007480 2024-09-13T17:54:11.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe71ffb700 1 -- 172.21.15.40:0/1091764129 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe5c0035d0 con 0x7efe6c007480 2024-09-13T17:54:11.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe71ffb700 1 -- 172.21.15.40:0/1091764129 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1907561473 0 0) 0x7efe5c0035d0 con 0x7efe6c007480 2024-09-13T17:54:11.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe71ffb700 1 -- 172.21.15.40:0/1091764129 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe6c0e1c00 msgr2=0x7efe6c0e2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:11.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe71ffb700 1 --2- 172.21.15.40:0/1091764129 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe6c0e1c00 0x7efe6c0e2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe71ffb700 1 -- 172.21.15.40:0/1091764129 >> v2:172.21.15.43:3300/0 conn(0x7efe6c0d8920 msgr2=0x7efe6c049090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:11.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe71ffb700 1 --2- 172.21.15.40:0/1091764129 >> v2:172.21.15.43:3300/0 conn(0x7efe6c0d8920 0x7efe6c049090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe71ffb700 1 -- 172.21.15.40:0/1091764129 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c0495d0 con 0x7efe6c007480 2024-09-13T17:54:11.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe71ffb700 1 -- 172.21.15.40:0/1091764129 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7efe640017f0 con 0x7efe6c007480 2024-09-13T17:54:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe71ffb700 1 -- 172.21.15.40:0/1091764129 >> v1:172.21.15.40:6789/0 conn(0x7efe6c007480 legacy=0x7efe6c04efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe71ffb700 1 --2- 172.21.15.40:0/1091764129 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe6c0e1c00 0x7efe5c003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe71ffb700 1 --2- 172.21.15.40:0/1091764129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe5c003fa0 0x7efe5c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe71ffb700 1 --2- 172.21.15.40:0/1091764129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe5c004db0 0x7efe5c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe71ffb700 1 -- 172.21.15.40:0/1091764129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c050a10 con 0x7efe5c003fa0 2024-09-13T17:54:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe71ffb700 1 -- 172.21.15.40:0/1091764129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c006ff0 con 0x7efe5c004db0 2024-09-13T17:54:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe71ffb700 1 -- 172.21.15.40:0/1091764129 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c00bab0 con 0x7efe6c0e1c00 2024-09-13T17:54:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe727fc700 1 --2- 172.21.15.40:0/1091764129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe5c003fa0 0x7efe5c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe72ffd700 1 --2- 172.21.15.40:0/1091764129 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe6c0e1c00 0x7efe5c003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe737fe700 1 --2- 172.21.15.40:0/1091764129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe5c004db0 0x7efe5c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe727fc700 1 -- 172.21.15.40:0/1091764129 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe6c0e1c00 msgr2=0x7efe5c003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe727fc700 1 --2- 172.21.15.40:0/1091764129 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe6c0e1c00 0x7efe5c003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe727fc700 1 -- 172.21.15.40:0/1091764129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe5c004db0 msgr2=0x7efe5c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:11.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe727fc700 1 --2- 172.21.15.40:0/1091764129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe5c004db0 0x7efe5c005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe727fc700 1 -- 172.21.15.40:0/1091764129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5c0035d0 con 0x7efe5c003fa0 2024-09-13T17:54:11.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe737fe700 1 --2- 172.21.15.40:0/1091764129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe5c004db0 0x7efe5c005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:54:11.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe727fc700 1 --2- 172.21.15.40:0/1091764129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe5c003fa0 0x7efe5c004870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efe6400c320 tx=0x7efe5c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119f277b0935ae48 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe71ffb700 1 -- 172.21.15.40:0/1091764129 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6400ce60 con 0x7efe5c003fa0 2024-09-13T17:54:11.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe71ffb700 1 -- 172.21.15.40:0/1091764129 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe640070d0 con 0x7efe5c003fa0 2024-09-13T17:54:11.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.413+0000 7efe71ffb700 1 -- 172.21.15.40:0/1091764129 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe64007480 con 0x7efe5c003fa0 2024-09-13T17:54:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.417+0000 7efe73fff700 1 -- 172.21.15.40:0/1091764129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe5c003fa0 msgr2=0x7efe5c004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.417+0000 7efe73fff700 1 --2- 172.21.15.40:0/1091764129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe5c003fa0 0x7efe5c004870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efe6400c320 tx=0x7efe5c0058a0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.417+0000 7efe737fe700 1 -- 172.21.15.40:0/1091764129 reap_dead start 2024-09-13T17:54:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.417+0000 7efe73fff700 1 -- 172.21.15.40:0/1091764129 shutdown_connections 2024-09-13T17:54:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.417+0000 7efe73fff700 1 -- 172.21.15.40:0/1091764129 >> 172.21.15.40:0/1091764129 conn(0x7efe6c054110 msgr2=0x7efe6c054520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.417+0000 7efe73fff700 1 -- 172.21.15.40:0/1091764129 shutdown_connections 2024-09-13T17:54:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.417+0000 7efe73fff700 1 -- 172.21.15.40:0/1091764129 wait complete. 2024-09-13T17:54:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.417+0000 7efe73fff700 1 Processor -- start 2024-09-13T17:54:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.417+0000 7efe73fff700 1 -- start start 2024-09-13T17:54:11.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe73fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe6c0cf120 0x7efe6c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe73fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe6c0cf500 0x7efe6c053670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe73fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c0e15d0 0x7efe6c0d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe73fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c0cda30 con 0x7efe6c0e15d0 2024-09-13T17:54:11.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe73fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c0cdcd0 con 0x7efe6c0cf500 2024-09-13T17:54:11.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe73fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c04d230 con 0x7efe6c0cf120 2024-09-13T17:54:11.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe737fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c0e15d0 0x7efe6c0d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe737fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c0e15d0 0x7efe6c0d0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59248/0 (socket says 172.21.15.40:59248) 2024-09-13T17:54:11.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe737fe700 1 -- 172.21.15.40:0/1499844460 learned_addr learned my addr 172.21.15.40:0/1499844460 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe72ffd700 1 --2- 172.21.15.40:0/1499844460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe6c0cf120 0x7efe6c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe727fc700 1 --2- 172.21.15.40:0/1499844460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe6c0cf500 0x7efe6c053670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe737fe700 1 -- 172.21.15.40:0/1499844460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe6c0cf120 msgr2=0x7efe6c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe737fe700 1 --2- 172.21.15.40:0/1499844460 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe6c0cf120 0x7efe6c0ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe737fe700 1 -- 172.21.15.40:0/1499844460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe6c0cf500 msgr2=0x7efe6c053670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe737fe700 1 --2- 172.21.15.40:0/1499844460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe6c0cf500 0x7efe6c053670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe737fe700 1 -- 172.21.15.40:0/1499844460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c04d4a0 con 0x7efe6c0e15d0 2024-09-13T17:54:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe737fe700 1 --2- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c0e15d0 0x7efe6c0d0de0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efe7404ff70 tx=0x7efe740680b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe5b7fe700 1 -- 172.21.15.40:0/1499844460 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe74067720 con 0x7efe6c0e15d0 2024-09-13T17:54:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe5b7fe700 1 -- 172.21.15.40:0/1499844460 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe74091040 con 0x7efe6c0e15d0 2024-09-13T17:54:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe73fff700 1 -- 172.21.15.40:0/1499844460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6c04d780 con 0x7efe6c0e15d0 2024-09-13T17:54:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe5b7fe700 1 -- 172.21.15.40:0/1499844460 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe74072150 con 0x7efe6c0e15d0 2024-09-13T17:54:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe73fff700 1 -- 172.21.15.40:0/1499844460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe6c0ccd90 con 0x7efe6c0e15d0 2024-09-13T17:54:11.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe73fff700 1 --2- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c0d1320 0x7efe6c0d1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe73fff700 1 -- 172.21.15.40:0/1499844460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c0084b0 con 0x7efe6c0d1320 2024-09-13T17:54:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.421+0000 7efe72ffd700 1 --2- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c0d1320 0x7efe6c0d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.425+0000 7efe72ffd700 1 -- 172.21.15.40:0/1499844460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efe6c0d6140 con 0x7efe6c0d1320 2024-09-13T17:54:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.425+0000 7efe72ffd700 1 --2- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c0d1320 0x7efe6c0d1700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efe68014970 tx=0x7efe68014d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.425+0000 7efe5b7fe700 1 -- 172.21.15.40:0/1499844460 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68012950 con 0x7efe6c0d1320 2024-09-13T17:54:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.425+0000 7efe5b7fe700 1 -- 172.21.15.40:0/1499844460 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efe740724c0 con 0x7efe6c0e15d0 2024-09-13T17:54:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.425+0000 7efe5b7fe700 1 --2- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7efe60066800 0x7efe60068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.425+0000 7efe727fc700 1 --2- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7efe60066800 0x7efe60068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.425+0000 7efe5b7fe700 1 -- 172.21.15.40:0/1499844460 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7efe740fe8f0 con 0x7efe6c0e15d0 2024-09-13T17:54:11.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.425+0000 7efe5b7fe700 1 -- 172.21.15.40:0/1499844460 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7efe68012ab0 con 0x7efe6c0d1320 2024-09-13T17:54:11.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.425+0000 7efe5b7fe700 1 -- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c0d1320 msgr2=0x7efe6c0d1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.425+0000 7efe5b7fe700 1 --2- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c0d1320 0x7efe6c0d1700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efe68014970 tx=0x7efe68014d30 comp rx=0 tx=0).stop 2024-09-13T17:54:11.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.425+0000 7efe5b7fe700 1 -- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c0d1320 msgr2=0x7efe6c0d1700 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:54:11.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.425+0000 7efe5b7fe700 1 --2- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c0d1320 0x7efe6c0d1700 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efe68014970 tx=0x7efe68014d30 comp rx=0 tx=0).stop 2024-09-13T17:54:11.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.425+0000 7efe727fc700 1 --2- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7efe60066800 0x7efe60068cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efe6c003fb0 tx=0x7efe6400a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.433+0000 7efe73fff700 1 --2- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c0cb4d0 0x7efe6c0cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.437+0000 7efe73fff700 1 -- 172.21.15.40:0/1499844460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c04cc50 con 0x7efe6c0cb4d0 2024-09-13T17:54:11.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.437+0000 7efe72ffd700 1 --2- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c0cb4d0 0x7efe6c0cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.437+0000 7efe72ffd700 1 -- 172.21.15.40:0/1499844460 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7efe6c0069e0 con 0x7efe6c0cb4d0 2024-09-13T17:54:11.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.437+0000 7efe72ffd700 1 --2- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c0cb4d0 0x7efe6c0cb8e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efe68014970 tx=0x7efe680047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.437+0000 7efe5b7fe700 1 -- 172.21.15.40:0/1499844460 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68004220 con 0x7efe6c0cb4d0 2024-09-13T17:54:11.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.437+0000 7efe5b7fe700 1 -- 172.21.15.40:0/1499844460 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4618 (secure 0 0 0) 0x7efe68004380 con 0x7efe6c0cb4d0 2024-09-13T17:54:11.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.437+0000 7efe5b7fe700 1 -- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c0cb4d0 msgr2=0x7efe6c0cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.437+0000 7efe5b7fe700 1 --2- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c0cb4d0 0x7efe6c0cb8e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efe68014970 tx=0x7efe680047e0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.437+0000 7efe5b7fe700 1 -- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c0cb4d0 msgr2=0x7efe6c0cb8e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:54:11.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.437+0000 7efe5b7fe700 1 --2- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c0cb4d0 0x7efe6c0cb8e0 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efe68014970 tx=0x7efe680047e0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.446 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:54:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:54:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:54:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:54:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 34, 2024-09-13T17:54:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:54:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:54:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:54:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:54:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 15, 2024-09-13T17:54:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:54:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:54:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:54:11.447 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:54:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:54:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:11.448 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:54:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:54:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:54:11.449 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:54:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:54:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:54:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:54:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:54:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:54:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:54:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:54:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:54:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:54:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:54:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:54:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:54:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:54:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:11.450 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:54:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:54:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.451 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:54:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:54:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:54:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:54:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.452 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:54:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:54:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:54:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:54:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:11.453 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:54:11.454 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:11.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.454 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.454 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:11.454 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:54:11.454 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:11.454 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.454 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:11.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.454 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:54:11.454 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:54:11.454 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:11.454 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:11.454 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:54:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:54:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:54:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:11.455 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:54:11.456 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:11.456 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.456 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:11.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.456 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:54:11.456 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:54:11.456 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:11.456 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:54:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:54:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:54:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:54:11.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.458 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:11.458 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:11.458 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:54:11.458 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.458 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:11.458 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:54:11.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.458 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:11.458 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:11.458 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:54:11.458 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.458 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:11.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.458 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:54:11.458 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:54:11.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.437+0000 7efe73fff700 1 -- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7efe60066800 msgr2=0x7efe60068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.437+0000 7efe73fff700 1 --2- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7efe60066800 0x7efe60068cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efe6c003fb0 tx=0x7efe6400a000 comp rx=0 tx=0).stop 2024-09-13T17:54:11.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.437+0000 7efe73fff700 1 -- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c0e15d0 msgr2=0x7efe6c0d0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.437+0000 7efe73fff700 1 --2- 172.21.15.40:0/1499844460 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe6c0e15d0 0x7efe6c0d0de0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efe7404ff70 tx=0x7efe740680b0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.437+0000 7efe737fe700 1 -- 172.21.15.40:0/1499844460 reap_dead start 2024-09-13T17:54:11.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.437+0000 7efe737fe700 1 -- 172.21.15.40:0/1499844460 reap_dead start 2024-09-13T17:54:11.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.437+0000 7efe73fff700 1 -- 172.21.15.40:0/1499844460 shutdown_connections 2024-09-13T17:54:11.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.437+0000 7efe73fff700 1 -- 172.21.15.40:0/1499844460 >> 172.21.15.40:0/1499844460 conn(0x7efe6c054110 msgr2=0x7efe6c0d56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:11.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.437+0000 7efe73fff700 1 -- 172.21.15.40:0/1499844460 shutdown_connections 2024-09-13T17:54:11.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.437+0000 7efe73fff700 1 -- 172.21.15.40:0/1499844460 wait complete. 2024-09-13T17:54:11.461 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:54:11.461 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T17:54:11.461 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:54:11.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.573+0000 7f793f59e700 1 Processor -- start 2024-09-13T17:54:11.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.573+0000 7f793f59e700 1 -- start start 2024-09-13T17:54:11.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.573+0000 7f793f59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79380d68b0 0x7f79380dd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.573+0000 7f793f59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f793800daf0 0x7f793800df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.573+0000 7f793f59e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79380e0c60 con 0x7f793804e490 2024-09-13T17:54:11.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.573+0000 7f793f59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79380dfcc0 con 0x7f79380d68b0 2024-09-13T17:54:11.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.573+0000 7f793f59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793800e440 con 0x7f793800daf0 2024-09-13T17:54:11.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.573+0000 7f793e59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79380d68b0 0x7f79380dd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.573+0000 7f793e59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79380d68b0 0x7f79380dd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59262/0 (socket says 172.21.15.40:59262) 2024-09-13T17:54:11.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.573+0000 7f793e59c700 1 -- 172.21.15.40:0/3357171879 learned_addr learned my addr 172.21.15.40:0/3357171879 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:11.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.573+0000 7f793ed9d700 1 --2- 172.21.15.40:0/3357171879 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f793800daf0 0x7f793800df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.573+0000 7f793e59c700 1 -- 172.21.15.40:0/3357171879 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f793800daf0 msgr2=0x7f793800df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.573+0000 7f793e59c700 1 --2- 172.21.15.40:0/3357171879 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f793800daf0 0x7f793800df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.573+0000 7f793e59c700 1 -- 172.21.15.40:0/3357171879 >> v1:172.21.15.43:6789/0 conn(0x7f793804e490 legacy=0x7f79380361a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.573+0000 7f793e59c700 1 -- 172.21.15.40:0/3357171879 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79380dd690 con 0x7f79380d68b0 2024-09-13T17:54:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.577+0000 7f793e59c700 1 --2- 172.21.15.40:0/3357171879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79380d68b0 0x7f79380dd150 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7930009910 tx=0x7f7930009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60f8197cfd133f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.577+0000 7f793d59a700 1 -- 172.21.15.40:0/3357171879 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793000ba60 con 0x7f79380d68b0 2024-09-13T17:54:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.577+0000 7f793d59a700 1 -- 172.21.15.40:0/3357171879 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793000bbc0 con 0x7f79380d68b0 2024-09-13T17:54:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.577+0000 7f793d59a700 1 -- 172.21.15.40:0/3357171879 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7930009e60 con 0x7f79380d68b0 2024-09-13T17:54:11.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.577+0000 7f793f59e700 1 -- 172.21.15.40:0/3357171879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79380d68b0 msgr2=0x7f79380dd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.577+0000 7f793f59e700 1 --2- 172.21.15.40:0/3357171879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79380d68b0 0x7f79380dd150 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7930009910 tx=0x7f7930009c20 comp rx=0 tx=0).stop 2024-09-13T17:54:11.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.577+0000 7f793f59e700 1 -- 172.21.15.40:0/3357171879 shutdown_connections 2024-09-13T17:54:11.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.577+0000 7f793f59e700 1 --2- 172.21.15.40:0/3357171879 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f793800daf0 0x7f793800df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.577+0000 7f793f59e700 1 --2- 172.21.15.40:0/3357171879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79380d68b0 0x7f79380dd150 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.577+0000 7f793f59e700 1 -- 172.21.15.40:0/3357171879 >> 172.21.15.40:0/3357171879 conn(0x7f7938009a80 msgr2=0x7f7938009e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:11.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.577+0000 7f793f59e700 1 -- 172.21.15.40:0/3357171879 shutdown_connections 2024-09-13T17:54:11.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.577+0000 7f793f59e700 1 -- 172.21.15.40:0/3357171879 wait complete. 2024-09-13T17:54:11.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793f59e700 1 Processor -- start 2024-09-13T17:54:11.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793f59e700 1 -- start start 2024-09-13T17:54:11.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793f59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f793800daf0 0x7f79380deb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793f59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f793804e490 0x7f7938175f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793f59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79380d68b0 0x7f7938176440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793f59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7938050730 con 0x7f79380d68b0 2024-09-13T17:54:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793f59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7938034800 con 0x7f793800daf0 2024-09-13T17:54:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793f59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7938034ac0 con 0x7f793804e490 2024-09-13T17:54:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79380d68b0 0x7f7938176440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79380d68b0 0x7f7938176440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59264/0 (socket says 172.21.15.40:59264) 2024-09-13T17:54:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793ed9d700 1 -- 172.21.15.40:0/4212976966 learned_addr learned my addr 172.21.15.40:0/4212976966 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793e59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f793800daf0 0x7f79380deb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793dd9b700 1 --2- 172.21.15.40:0/4212976966 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f793804e490 0x7f7938175f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793ed9d700 1 -- 172.21.15.40:0/4212976966 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f793804e490 msgr2=0x7f7938175f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793ed9d700 1 --2- 172.21.15.40:0/4212976966 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f793804e490 0x7f7938175f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793ed9d700 1 -- 172.21.15.40:0/4212976966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f793800daf0 msgr2=0x7f79380deb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793ed9d700 1 --2- 172.21.15.40:0/4212976966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f793800daf0 0x7f79380deb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793ed9d700 1 -- 172.21.15.40:0/4212976966 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7930009570 con 0x7f79380d68b0 2024-09-13T17:54:11.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793ed9d700 1 --2- 172.21.15.40:0/4212976966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79380d68b0 0x7f7938176440 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f794004f940 tx=0x7f79400680b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f792effd700 1 -- 172.21.15.40:0/4212976966 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7940067930 con 0x7f79380d68b0 2024-09-13T17:54:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f792effd700 1 -- 172.21.15.40:0/4212976966 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7940072070 con 0x7f79380d68b0 2024-09-13T17:54:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793f59e700 1 -- 172.21.15.40:0/4212976966 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79380509e0 con 0x7f79380d68b0 2024-09-13T17:54:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f792effd700 1 -- 172.21.15.40:0/4212976966 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79400723e0 con 0x7f79380d68b0 2024-09-13T17:54:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.581+0000 7f793f59e700 1 -- 172.21.15.40:0/4212976966 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79381679b0 con 0x7f79380d68b0 2024-09-13T17:54:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.585+0000 7f793f59e700 1 --2- 172.21.15.40:0/4212976966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7938167d80 0x7f7938165b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.585+0000 7f793f59e700 1 -- 172.21.15.40:0/4212976966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7938168160 con 0x7f7938167d80 2024-09-13T17:54:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.585+0000 7f793e59c700 1 --2- 172.21.15.40:0/4212976966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7938167d80 0x7f7938165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.585+0000 7f792effd700 1 -- 172.21.15.40:0/4212976966 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7940072540 con 0x7f79380d68b0 2024-09-13T17:54:11.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.585+0000 7f793e59c700 1 -- 172.21.15.40:0/4212976966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f79380e0180 con 0x7f7938167d80 2024-09-13T17:54:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.585+0000 7f793e59c700 1 --2- 172.21.15.40:0/4212976966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7938167d80 0x7f7938165b20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7930009910 tx=0x7f7930004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.585+0000 7f792effd700 1 --2- 172.21.15.40:0/4212976966 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7920066730 0x7f7920068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.585+0000 7f793dd9b700 1 --2- 172.21.15.40:0/4212976966 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7920066730 0x7f7920068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.585+0000 7f792effd700 1 -- 172.21.15.40:0/4212976966 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f79400feca0 con 0x7f79380d68b0 2024-09-13T17:54:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.585+0000 7f792effd700 1 -- 172.21.15.40:0/4212976966 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793001f070 con 0x7f7938167d80 2024-09-13T17:54:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.585+0000 7f793dd9b700 1 --2- 172.21.15.40:0/4212976966 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7920066730 0x7f7920068bf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f79380d57c0 tx=0x7f7934009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.585+0000 7f792effd700 1 -- 172.21.15.40:0/4212976966 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f79300036c0 con 0x7f7938167d80 2024-09-13T17:54:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.585+0000 7f792effd700 1 -- 172.21.15.40:0/4212976966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7938167d80 msgr2=0x7f7938165b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.585+0000 7f792effd700 1 --2- 172.21.15.40:0/4212976966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7938167d80 0x7f7938165b20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7930009910 tx=0x7f7930004770 comp rx=0 tx=0).stop 2024-09-13T17:54:11.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.585+0000 7f792effd700 1 -- 172.21.15.40:0/4212976966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7938167d80 msgr2=0x7f7938165b20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:54:11.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.585+0000 7f792effd700 1 --2- 172.21.15.40:0/4212976966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7938167d80 0x7f7938165b20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.597+0000 7f793f59e700 1 --2- 172.21.15.40:0/4212976966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7938050e30 0x7f7938166060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.597+0000 7f793f59e700 1 -- 172.21.15.40:0/4212976966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79380074c0 con 0x7f7938050e30 2024-09-13T17:54:11.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.597+0000 7f793e59c700 1 --2- 172.21.15.40:0/4212976966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7938050e30 0x7f7938166060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.597+0000 7f793e59c700 1 -- 172.21.15.40:0/4212976966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f79380dfea0 con 0x7f7938050e30 2024-09-13T17:54:11.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.597+0000 7f793e59c700 1 --2- 172.21.15.40:0/4212976966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7938050e30 0x7f7938166060 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7930000c00 tx=0x7f79300092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.597+0000 7f792effd700 1 -- 172.21.15.40:0/4212976966 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793000bed0 con 0x7f7938050e30 2024-09-13T17:54:11.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.597+0000 7f792effd700 1 -- 172.21.15.40:0/4212976966 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7f793002b300 con 0x7f7938050e30 2024-09-13T17:54:11.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.597+0000 7f792effd700 1 -- 172.21.15.40:0/4212976966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7938050e30 msgr2=0x7f7938166060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.597+0000 7f792effd700 1 --2- 172.21.15.40:0/4212976966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7938050e30 0x7f7938166060 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7930000c00 tx=0x7f79300092e0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.597+0000 7f792effd700 1 -- 172.21.15.40:0/4212976966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7938050e30 msgr2=0x7f7938166060 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:54:11.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.597+0000 7f792effd700 1 --2- 172.21.15.40:0/4212976966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7938050e30 0x7f7938166060 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.607 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:54:11.607 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:54:11.608 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:54:11.608 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:54:11.608 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 34, 2024-09-13T17:54:11.608 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:54:11.608 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:54:11.608 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:54:11.608 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:54:11.608 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:11.608 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 16, 2024-09-13T17:54:11.608 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:54:11.608 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:54:11.608 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:54:11.608 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:11.609 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:11.609 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:11.609 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:11.609 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:11.609 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:11.609 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:11.609 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:11.609 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:11.609 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:11.612 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:11.612 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:54:11.612 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:54:11.612 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:11.613 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:11.613 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:11.613 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:11.613 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:11.613 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:11.613 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:11.613 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:11.613 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:11.613 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:11.613 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:11.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.613 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:54:11.614 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:54:11.614 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:54:11.614 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:54:11.614 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:54:11.614 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:54:11.614 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:54:11.614 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:54:11.614 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:54:11.614 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:54:11.614 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:54:11.614 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:54:11.614 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:54:11.615 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:54:11.615 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:54:11.615 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:54:11.615 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:54:11.615 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:11.615 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:11.615 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:11.615 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:11.615 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:11.615 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:11.615 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:11.615 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:11.616 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:11.616 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:11.616 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:11.616 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:54:11.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.616 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:54:11.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.616 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:54:11.616 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:54:11.616 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:11.616 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:11.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.617 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:11.617 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:54:11.617 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:11.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.617 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:11.617 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:54:11.617 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:11.617 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.617 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:11.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.617 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:54:11.618 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:54:11.618 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:11.618 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:11.618 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:11.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.618 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:54:11.618 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:54:11.618 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:11.618 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:11.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.618 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:11.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:54:11.619 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:11.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.619 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:11.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:54:11.619 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:11.619 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.619 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:11.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:54:11.620 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:54:11.620 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:11.620 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:11.620 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:11.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.620 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:54:11.620 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:54:11.620 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:11.620 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:11.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.620 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:11.621 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:54:11.621 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:11.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.621 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:11.621 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:54:11.621 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:11.621 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.621 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:11.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.621 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:54:11.621 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:54:11.622 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:11.622 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:11.622 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:11.622 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.622 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:11.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.622 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:54:11.622 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:54:11.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.622 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:11.622 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:11.623 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:54:11.623 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.623 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:11.623 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:54:11.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.623 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:11.623 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:11.623 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:54:11.623 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.623 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:11.623 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:54:11.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.624 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:11.624 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:11.624 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:54:11.624 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.624 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:11.624 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:54:11.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.624 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:11.624 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:11.624 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:54:11.624 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.625 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:11.625 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:54:11.625 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.625 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:11.625 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:11.625 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:54:11.625 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.625 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:11.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.625 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:54:11.625 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:54:11.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.601+0000 7f793f59e700 1 -- 172.21.15.40:0/4212976966 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7920066730 msgr2=0x7f7920068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.601+0000 7f793f59e700 1 --2- 172.21.15.40:0/4212976966 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7920066730 0x7f7920068bf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f79380d57c0 tx=0x7f7934009040 comp rx=0 tx=0).stop 2024-09-13T17:54:11.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.601+0000 7f793f59e700 1 -- 172.21.15.40:0/4212976966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79380d68b0 msgr2=0x7f7938176440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.601+0000 7f793f59e700 1 --2- 172.21.15.40:0/4212976966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79380d68b0 0x7f7938176440 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f794004f940 tx=0x7f79400680b0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.601+0000 7f793ed9d700 1 -- 172.21.15.40:0/4212976966 reap_dead start 2024-09-13T17:54:11.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.601+0000 7f793ed9d700 1 -- 172.21.15.40:0/4212976966 reap_dead start 2024-09-13T17:54:11.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.601+0000 7f793f59e700 1 -- 172.21.15.40:0/4212976966 shutdown_connections 2024-09-13T17:54:11.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.601+0000 7f793f59e700 1 -- 172.21.15.40:0/4212976966 >> 172.21.15.40:0/4212976966 conn(0x7f7938009a80 msgr2=0x7f79380db230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:11.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.601+0000 7f793f59e700 1 -- 172.21.15.40:0/4212976966 shutdown_connections 2024-09-13T17:54:11.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.601+0000 7f793f59e700 1 -- 172.21.15.40:0/4212976966 wait complete. 2024-09-13T17:54:11.628 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:54:11.628 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T17:54:11.628 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:54:11.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.729+0000 7f76762ec700 1 Processor -- start 2024-09-13T17:54:11.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.729+0000 7f76762ec700 1 -- start start 2024-09-13T17:54:11.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f76762ec700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f76700b0d70 0x7f7670157f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f76762ec700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f767006a950 0x7f767006ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f76762ec700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7670158440 con 0x7f7670151750 2024-09-13T17:54:11.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f76762ec700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f767006a4c0 con 0x7f76700b0d70 2024-09-13T17:54:11.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f76762ec700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f767006b320 con 0x7f767006a950 2024-09-13T17:54:11.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f766f7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f76700b0d70 0x7f7670157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f766f7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f76700b0d70 0x7f7670157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59272/0 (socket says 172.21.15.40:59272) 2024-09-13T17:54:11.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f766f7fe700 1 -- 172.21.15.40:0/70692790 learned_addr learned my addr 172.21.15.40:0/70692790 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:11.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f7674889700 1 --2- 172.21.15.40:0/70692790 >> v2:172.21.15.43:3300/0 conn(0x7f767006a950 0x7f767006ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f766f7fe700 1 -- 172.21.15.40:0/70692790 >> v2:172.21.15.43:3300/0 conn(0x7f767006a950 msgr2=0x7f767006ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:11.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f766f7fe700 1 --2- 172.21.15.40:0/70692790 >> v2:172.21.15.43:3300/0 conn(0x7f767006a950 0x7f767006ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f766f7fe700 1 -- 172.21.15.40:0/70692790 >> v1:172.21.15.43:6789/0 conn(0x7f7670151750 legacy=0x7f767006fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f766f7fe700 1 -- 172.21.15.40:0/70692790 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767006cf60 con 0x7f76700b0d70 2024-09-13T17:54:11.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f766f7fe700 1 --2- 172.21.15.40:0/70692790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76700b0d70 0x7f7670157f00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7670072820 tx=0x7f7664009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5642e6beab6bcf4b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f766effd700 1 -- 172.21.15.40:0/70692790 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766400b8a0 con 0x7f76700b0d70 2024-09-13T17:54:11.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f766effd700 1 -- 172.21.15.40:0/70692790 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766400ba00 con 0x7f76700b0d70 2024-09-13T17:54:11.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f766effd700 1 -- 172.21.15.40:0/70692790 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766400bd90 con 0x7f76700b0d70 2024-09-13T17:54:11.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f76762ec700 1 -- 172.21.15.40:0/70692790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76700b0d70 msgr2=0x7f7670157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f76762ec700 1 --2- 172.21.15.40:0/70692790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76700b0d70 0x7f7670157f00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7670072820 tx=0x7f7664009bf0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f76762ec700 1 -- 172.21.15.40:0/70692790 shutdown_connections 2024-09-13T17:54:11.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f76762ec700 1 --2- 172.21.15.40:0/70692790 >> v2:172.21.15.43:3300/0 conn(0x7f767006a950 0x7f767006ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f76762ec700 1 --2- 172.21.15.40:0/70692790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76700b0d70 0x7f7670157f00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f76762ec700 1 -- 172.21.15.40:0/70692790 >> 172.21.15.40:0/70692790 conn(0x7f767006d5b0 msgr2=0x7f767006d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:11.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f76762ec700 1 -- 172.21.15.40:0/70692790 shutdown_connections 2024-09-13T17:54:11.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.733+0000 7f76762ec700 1 -- 172.21.15.40:0/70692790 wait complete. 2024-09-13T17:54:11.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f76762ec700 1 Processor -- start 2024-09-13T17:54:11.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f76762ec700 1 -- start start 2024-09-13T17:54:11.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f76762ec700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f767006a950 0x7f76701ee610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f76762ec700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76700b0d70 0x7f7670156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f76762ec700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7670151750 0x7f76701571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f76762ec700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7670158240 con 0x7f7670151750 2024-09-13T17:54:11.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f76762ec700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76701566d0 con 0x7f767006a950 2024-09-13T17:54:11.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f76762ec700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7670157cc0 con 0x7f76700b0d70 2024-09-13T17:54:11.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f7674889700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7670151750 0x7f76701571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f7674889700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7670151750 0x7f76701571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59282/0 (socket says 172.21.15.40:59282) 2024-09-13T17:54:11.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f7674889700 1 -- 172.21.15.40:0/1736804469 learned_addr learned my addr 172.21.15.40:0/1736804469 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:11.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f766f7fe700 1 --2- 172.21.15.40:0/1736804469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76700b0d70 0x7f7670156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f766ffff700 1 --2- 172.21.15.40:0/1736804469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f767006a950 0x7f76701ee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f7674889700 1 -- 172.21.15.40:0/1736804469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76700b0d70 msgr2=0x7f7670156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f7674889700 1 --2- 172.21.15.40:0/1736804469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76700b0d70 0x7f7670156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f7674889700 1 -- 172.21.15.40:0/1736804469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f767006a950 msgr2=0x7f76701ee610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f7674889700 1 --2- 172.21.15.40:0/1736804469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f767006a950 0x7f76701ee610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f7674889700 1 -- 172.21.15.40:0/1736804469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7664009570 con 0x7f7670151750 2024-09-13T17:54:11.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f7674889700 1 --2- 172.21.15.40:0/1736804469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7670151750 0x7f76701571b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7668007e60 tx=0x7f766800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f766cff9700 1 -- 172.21.15.40:0/1736804469 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766800ed30 con 0x7f7670151750 2024-09-13T17:54:11.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f766cff9700 1 -- 172.21.15.40:0/1736804469 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7668014420 con 0x7f7670151750 2024-09-13T17:54:11.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f76762ec700 1 -- 172.21.15.40:0/1736804469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76701ee0c0 con 0x7f7670151750 2024-09-13T17:54:11.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f766cff9700 1 -- 172.21.15.40:0/1736804469 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76680147d0 con 0x7f7670151750 2024-09-13T17:54:11.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.737+0000 7f76762ec700 1 -- 172.21.15.40:0/1736804469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76701556b0 con 0x7f7670151750 2024-09-13T17:54:11.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.741+0000 7f76762ec700 1 --2- 172.21.15.40:0/1736804469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76701edc20 0x7f7670155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.741+0000 7f76762ec700 1 -- 172.21.15.40:0/1736804469 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7670067840 con 0x7f76701edc20 2024-09-13T17:54:11.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.741+0000 7f766ffff700 1 --2- 172.21.15.40:0/1736804469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76701edc20 0x7f7670155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.741+0000 7f766ffff700 1 -- 172.21.15.40:0/1736804469 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7670071240 con 0x7f76701edc20 2024-09-13T17:54:11.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.741+0000 7f766cff9700 1 -- 172.21.15.40:0/1736804469 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7668014930 con 0x7f7670151750 2024-09-13T17:54:11.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.741+0000 7f766ffff700 1 --2- 172.21.15.40:0/1736804469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76701edc20 0x7f7670155840 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f766000f540 tx=0x7f766000f900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.741+0000 7f766cff9700 1 --2- 172.21.15.40:0/1736804469 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f765c066730 0x7f765c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.741+0000 7f766cff9700 1 -- 172.21.15.40:0/1736804469 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7668084a90 con 0x7f7670151750 2024-09-13T17:54:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.741+0000 7f766f7fe700 1 --2- 172.21.15.40:0/1736804469 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f765c066730 0x7f765c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.741+0000 7f766cff9700 1 -- 172.21.15.40:0/1736804469 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7660005320 con 0x7f76701edc20 2024-09-13T17:54:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.741+0000 7f766cff9700 1 -- 172.21.15.40:0/1736804469 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7660005480 con 0x7f76701edc20 2024-09-13T17:54:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.741+0000 7f766cff9700 1 -- 172.21.15.40:0/1736804469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76701edc20 msgr2=0x7f7670155840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.741+0000 7f766cff9700 1 --2- 172.21.15.40:0/1736804469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76701edc20 0x7f7670155840 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f766000f540 tx=0x7f766000f900 comp rx=0 tx=0).stop 2024-09-13T17:54:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.741+0000 7f766cff9700 1 -- 172.21.15.40:0/1736804469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76701edc20 msgr2=0x7f7670155840 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:54:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.741+0000 7f766cff9700 1 --2- 172.21.15.40:0/1736804469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76701edc20 0x7f7670155840 secure :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f766000f540 tx=0x7f766000f900 comp rx=0 tx=0).stop 2024-09-13T17:54:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.741+0000 7f766f7fe700 1 --2- 172.21.15.40:0/1736804469 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f765c066730 0x7f765c068bf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7664000c00 tx=0x7f7664008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.753+0000 7f76762ec700 1 --2- 172.21.15.40:0/1736804469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7670149460 0x7f7670149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.753+0000 7f76762ec700 1 -- 172.21.15.40:0/1736804469 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7670070f60 con 0x7f7670149460 2024-09-13T17:54:11.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.753+0000 7f766ffff700 1 --2- 172.21.15.40:0/1736804469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7670149460 0x7f7670149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.753+0000 7f766ffff700 1 -- 172.21.15.40:0/1736804469 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7670071520 con 0x7f7670149460 2024-09-13T17:54:11.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.753+0000 7f766ffff700 1 --2- 172.21.15.40:0/1736804469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7670149460 0x7f7670149870 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f766000f540 tx=0x7f7660007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.753+0000 7f766cff9700 1 -- 172.21.15.40:0/1736804469 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7660023070 con 0x7f7670149460 2024-09-13T17:54:11.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.753+0000 7f766cff9700 1 -- 172.21.15.40:0/1736804469 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4450 (secure 0 0 0) 0x7f7660007630 con 0x7f7670149460 2024-09-13T17:54:11.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.753+0000 7f766cff9700 1 -- 172.21.15.40:0/1736804469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7670149460 msgr2=0x7f7670149870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.753+0000 7f766cff9700 1 --2- 172.21.15.40:0/1736804469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7670149460 0x7f7670149870 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f766000f540 tx=0x7f7660007f90 comp rx=0 tx=0).stop 2024-09-13T17:54:11.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.753+0000 7f766cff9700 1 -- 172.21.15.40:0/1736804469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7670149460 msgr2=0x7f7670149870 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:54:11.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.753+0000 7f766cff9700 1 --2- 172.21.15.40:0/1736804469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7670149460 0x7f7670149870 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.764 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:54:11.764 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:54:11.764 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:54:11.764 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:54:11.764 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 34, 2024-09-13T17:54:11.764 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:54:11.764 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:54:11.764 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:54:11.764 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:54:11.764 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:11.764 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 16, 2024-09-13T17:54:11.764 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:54:11.765 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:54:11.765 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:54:11.765 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:11.765 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:11.765 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:11.765 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:11.765 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:11.765 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:11.765 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:11.765 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:11.765 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:11.765 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:11.765 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:11.765 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:54:11.766 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:54:11.766 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:11.766 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:11.766 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:11.766 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:11.766 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:11.766 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:11.766 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:11.766 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:11.766 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:11.766 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:11.766 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:11.766 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.766 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:54:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:54:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:54:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:54:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:54:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:54:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:54:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:54:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:54:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:54:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:54:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:54:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:54:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:54:11.767 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:54:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:54:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:54:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:11.768 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:54:11.769 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:54:11.769 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.769 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:54:11.769 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:54:11.769 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:11.769 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:11.769 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.769 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:11.769 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:54:11.769 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:11.769 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.769 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.770 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:11.770 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:54:11.770 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:11.770 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.770 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:11.770 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.770 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:54:11.770 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:54:11.770 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:11.770 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:11.770 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:11.770 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.770 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.770 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:54:11.771 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:54:11.771 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:11.771 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:11.771 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.771 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:11.771 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:54:11.771 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:11.771 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.771 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.771 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:11.771 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:54:11.771 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:11.771 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.771 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:11.771 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.772 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:54:11.772 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:54:11.772 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:11.772 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:11.772 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:11.772 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.772 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.772 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:54:11.772 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:54:11.772 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:11.772 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:11.772 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.772 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:11.772 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:54:11.773 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:11.773 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.773 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.773 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:11.773 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:54:11.773 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:11.773 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.773 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:11.773 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.773 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:54:11.773 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:54:11.773 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:11.773 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:11.773 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:11.774 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.774 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:11.774 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.774 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:54:11.774 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:54:11.774 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.774 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:11.774 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:11.774 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:54:11.774 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.774 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:11.774 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:54:11.774 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:11.774 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:11.774 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:11.775 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:54:11.775 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:11.775 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:11.775 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:11.775 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:54:11.775 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:54:11.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.757+0000 7f76762ec700 1 -- 172.21.15.40:0/1736804469 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f765c066730 msgr2=0x7f765c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.757+0000 7f76762ec700 1 --2- 172.21.15.40:0/1736804469 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f765c066730 0x7f765c068bf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7664000c00 tx=0x7f7664008030 comp rx=0 tx=0).stop 2024-09-13T17:54:11.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.757+0000 7f76762ec700 1 -- 172.21.15.40:0/1736804469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7670151750 msgr2=0x7f76701571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.757+0000 7f76762ec700 1 --2- 172.21.15.40:0/1736804469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7670151750 0x7f76701571b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7668007e60 tx=0x7f766800e3c0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.757+0000 7f7674889700 1 -- 172.21.15.40:0/1736804469 reap_dead start 2024-09-13T17:54:11.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.757+0000 7f7674889700 1 -- 172.21.15.40:0/1736804469 reap_dead start 2024-09-13T17:54:11.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.757+0000 7f76762ec700 1 -- 172.21.15.40:0/1736804469 shutdown_connections 2024-09-13T17:54:11.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.757+0000 7f76762ec700 1 -- 172.21.15.40:0/1736804469 >> 172.21.15.40:0/1736804469 conn(0x7f767006d5b0 msgr2=0x7f76700c40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:11.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.757+0000 7f76762ec700 1 -- 172.21.15.40:0/1736804469 shutdown_connections 2024-09-13T17:54:11.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.757+0000 7f76762ec700 1 -- 172.21.15.40:0/1736804469 wait complete. 2024-09-13T17:54:11.777 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:54:11.778 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-13T17:54:11.778 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T17:54:11.778 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-13T17:54:11.778 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-13T17:54:11.778 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T17:54:11.778 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:54:11.835 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T17:54:11.836 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T17:54:11.836 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:54:11.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.949+0000 7f48929ee700 1 Processor -- start 2024-09-13T17:54:11.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.949+0000 7f48929ee700 1 -- start start 2024-09-13T17:54:11.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.949+0000 7f48929ee700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f488c1503d0 0x7f488c156940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.949+0000 7f48929ee700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f488c15e960 0x7f488c160d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.949+0000 7f48929ee700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f488c15a520 con 0x7f488c06b4a0 2024-09-13T17:54:11.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.949+0000 7f48929ee700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488c1564b0 con 0x7f488c1503d0 2024-09-13T17:54:11.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.949+0000 7f48929ee700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c15a000 con 0x7f488c15e960 2024-09-13T17:54:11.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.949+0000 7f48919ec700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7f488c1503d0 msgr2=0x7f488c156940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:54:11.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.949+0000 7f48919ec700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f488c1503d0 0x7f488c156940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:54:11.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.949+0000 7f48911eb700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f488c06b4a0 0x7f488c071bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57098/0 (socket says 172.21.15.40:57098) 2024-09-13T17:54:11.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.949+0000 7f48911eb700 1 -- 172.21.15.40:0/296128362 learned_addr learned my addr 172.21.15.40:0/296128362 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:54:11.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.949+0000 7f48921ed700 1 --2- 172.21.15.40:0/296128362 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f488c15e960 0x7f488c160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48909ea700 1 -- 172.21.15.40:0/296128362 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3100333217 0 0) 0x7f488c15a520 con 0x7f488c06b4a0 2024-09-13T17:54:11.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48909ea700 1 -- 172.21.15.40:0/296128362 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48780035d0 con 0x7f488c06b4a0 2024-09-13T17:54:11.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48921ed700 1 -- 172.21.15.40:0/296128362 >> v2:172.21.15.43:3300/0 conn(0x7f488c1503d0 msgr2=0x7f488c156940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:54:11.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48921ed700 1 --2- 172.21.15.40:0/296128362 >> v2:172.21.15.43:3300/0 conn(0x7f488c1503d0 0x7f488c156940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48921ed700 1 -- 172.21.15.40:0/296128362 >> v1:172.21.15.40:6789/0 conn(0x7f488c06b4a0 legacy=0x7f488c071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48921ed700 1 -- 172.21.15.40:0/296128362 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c15a280 con 0x7f488c15e960 2024-09-13T17:54:11.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48921ed700 1 --2- 172.21.15.40:0/296128362 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f488c15e960 0x7f488c160d50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4884009d90 tx=0x7f488400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=609beaa13623e0e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48909ea700 1 -- 172.21.15.40:0/296128362 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400e070 con 0x7f488c15e960 2024-09-13T17:54:11.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48909ea700 1 -- 172.21.15.40:0/296128362 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488400bb20 con 0x7f488c15e960 2024-09-13T17:54:11.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48909ea700 1 -- 172.21.15.40:0/296128362 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884012430 con 0x7f488c15e960 2024-09-13T17:54:11.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48929ee700 1 -- 172.21.15.40:0/296128362 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f488c15e960 msgr2=0x7f488c160d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48929ee700 1 --2- 172.21.15.40:0/296128362 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f488c15e960 0x7f488c160d50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4884009d90 tx=0x7f488400b330 comp rx=0 tx=0).stop 2024-09-13T17:54:11.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48929ee700 1 -- 172.21.15.40:0/296128362 shutdown_connections 2024-09-13T17:54:11.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48929ee700 1 --2- 172.21.15.40:0/296128362 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f488c15e960 0x7f488c160d50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48929ee700 1 --2- 172.21.15.40:0/296128362 >> v2:172.21.15.43:3300/0 conn(0x7f488c1503d0 0x7f488c156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48929ee700 1 -- 172.21.15.40:0/296128362 >> 172.21.15.40:0/296128362 conn(0x7f488c06e4f0 msgr2=0x7f488c06e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:11.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48929ee700 1 -- 172.21.15.40:0/296128362 shutdown_connections 2024-09-13T17:54:11.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48929ee700 1 -- 172.21.15.40:0/296128362 wait complete. 2024-09-13T17:54:11.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48929ee700 1 Processor -- start 2024-09-13T17:54:11.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48929ee700 1 -- start start 2024-09-13T17:54:11.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48929ee700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c06b4a0 0x7f488c1f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48929ee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f488c1503d0 0x7f488c0ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48929ee700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f488c15e960 0x7f488c142760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48929ee700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c1f21c0 con 0x7f488c1503d0 2024-09-13T17:54:11.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.953+0000 7f48929ee700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c0c9ff0 con 0x7f488c06b4a0 2024-09-13T17:54:11.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.957+0000 7f48929ee700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c153720 con 0x7f488c15e960 2024-09-13T17:54:11.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.957+0000 7f48919ec700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c06b4a0 msgr2=0x7f488c1f1c80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:54:11.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.957+0000 7f48919ec700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c06b4a0 0x7f488c1f1c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:54:11.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.957+0000 7f48911eb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f488c1503d0 0x7f488c0ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.957+0000 7f48911eb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f488c1503d0 0x7f488c0ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59438/0 (socket says 172.21.15.40:59438) 2024-09-13T17:54:11.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.957+0000 7f48911eb700 1 -- 172.21.15.40:0/2460001288 learned_addr learned my addr 172.21.15.40:0/2460001288 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.957+0000 7f48921ed700 1 --2- 172.21.15.40:0/2460001288 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f488c15e960 0x7f488c142760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.957+0000 7f48911eb700 1 -- 172.21.15.40:0/2460001288 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f488c15e960 msgr2=0x7f488c142760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.957+0000 7f48911eb700 1 --2- 172.21.15.40:0/2460001288 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f488c15e960 0x7f488c142760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.957+0000 7f48911eb700 1 -- 172.21.15.40:0/2460001288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c06b4a0 msgr2=0x7f488c1f1c80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:54:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.957+0000 7f48911eb700 1 --2- 172.21.15.40:0/2460001288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c06b4a0 0x7f488c1f1c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.957+0000 7f48911eb700 1 -- 172.21.15.40:0/2460001288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4884009a70 con 0x7f488c1503d0 2024-09-13T17:54:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.957+0000 7f48911eb700 1 --2- 172.21.15.40:0/2460001288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f488c1503d0 0x7f488c0ca480 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f487c00c700 tx=0x7f487c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.957+0000 7f48827fc700 1 -- 172.21.15.40:0/2460001288 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487c005d90 con 0x7f488c1503d0 2024-09-13T17:54:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.957+0000 7f48827fc700 1 -- 172.21.15.40:0/2460001288 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487c0070d0 con 0x7f488c1503d0 2024-09-13T17:54:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.957+0000 7f48827fc700 1 -- 172.21.15.40:0/2460001288 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487c007480 con 0x7f488c1503d0 2024-09-13T17:54:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.957+0000 7f48929ee700 1 -- 172.21.15.40:0/2460001288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488c1539c0 con 0x7f488c1503d0 2024-09-13T17:54:11.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.957+0000 7f48929ee700 1 -- 172.21.15.40:0/2460001288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c0c9c80 con 0x7f488c1503d0 2024-09-13T17:54:11.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.957+0000 7f48929ee700 1 -- 172.21.15.40:0/2460001288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488c0c8330 con 0x7f488c1503d0 2024-09-13T17:54:11.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.961+0000 7f48827fc700 1 -- 172.21.15.40:0/2460001288 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f487c0075e0 con 0x7f488c1503d0 2024-09-13T17:54:11.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.961+0000 7f48827fc700 1 --2- 172.21.15.40:0/2460001288 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f48700666e0 0x7f4870068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:11.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.961+0000 7f48827fc700 1 -- 172.21.15.40:0/2460001288 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f487c083e20 con 0x7f488c1503d0 2024-09-13T17:54:11.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.961+0000 7f48919ec700 1 --2- 172.21.15.40:0/2460001288 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f48700666e0 0x7f4870068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:11.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.961+0000 7f48919ec700 1 --2- 172.21.15.40:0/2460001288 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f48700666e0 0x7f4870068ba0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f48880039e0 tx=0x7f4888009270 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:11.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:11.965+0000 7f48827fc700 1 -- 172.21.15.40:0/2460001288 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f487c050eb0 con 0x7f488c1503d0 2024-09-13T17:54:12.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:12.217+0000 7f48929ee700 1 -- 172.21.15.40:0/2460001288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f488c14d990 con 0x7f488c1503d0 2024-09-13T17:54:12.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:12.217+0000 7f48827fc700 1 -- 172.21.15.40:0/2460001288 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f487c017080 con 0x7f488c1503d0 2024-09-13T17:54:12.229 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":34,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:54:12.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:12.225+0000 7f48929ee700 1 -- 172.21.15.40:0/2460001288 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f48700666e0 msgr2=0x7f4870068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:12.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:12.225+0000 7f48929ee700 1 --2- 172.21.15.40:0/2460001288 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f48700666e0 0x7f4870068ba0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f48880039e0 tx=0x7f4888009270 comp rx=0 tx=0).stop 2024-09-13T17:54:12.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:12.225+0000 7f48929ee700 1 -- 172.21.15.40:0/2460001288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f488c1503d0 msgr2=0x7f488c0ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:12.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:12.225+0000 7f48929ee700 1 --2- 172.21.15.40:0/2460001288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f488c1503d0 0x7f488c0ca480 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f487c00c700 tx=0x7f487c00ca10 comp rx=0 tx=0).stop 2024-09-13T17:54:12.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:12.225+0000 7f48929ee700 1 -- 172.21.15.40:0/2460001288 shutdown_connections 2024-09-13T17:54:12.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:12.225+0000 7f48929ee700 1 --2- 172.21.15.40:0/2460001288 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f48700666e0 0x7f4870068ba0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:12.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:12.225+0000 7f48929ee700 1 --2- 172.21.15.40:0/2460001288 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f488c15e960 0x7f488c142760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:12.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:12.225+0000 7f48929ee700 1 --2- 172.21.15.40:0/2460001288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f488c1503d0 0x7f488c0ca480 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:12.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:12.225+0000 7f48929ee700 1 --2- 172.21.15.40:0/2460001288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c06b4a0 0x7f488c1f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:12.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:12.225+0000 7f48929ee700 1 -- 172.21.15.40:0/2460001288 >> 172.21.15.40:0/2460001288 conn(0x7f488c06e4f0 msgr2=0x7f488c071c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:12.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:12.225+0000 7f48929ee700 1 -- 172.21.15.40:0/2460001288 shutdown_connections 2024-09-13T17:54:12.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:12.225+0000 7f48929ee700 1 -- 172.21.15.40:0/2460001288 wait complete. 2024-09-13T17:54:15.249 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:54:15.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.365+0000 7f53023e3700 1 Processor -- start 2024-09-13T17:54:15.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.365+0000 7f53023e3700 1 -- start start 2024-09-13T17:54:15.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.365+0000 7f53023e3700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f52fc1512c0 0x7f52fc0c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:15.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.365+0000 7f53023e3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52fc15b3e0 0x7f52fc0c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:15.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.365+0000 7f53023e3700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52fc15a840 con 0x7f52fc0b2d00 2024-09-13T17:54:15.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.365+0000 7f53023e3700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52fc1516a0 con 0x7f52fc1512c0 2024-09-13T17:54:15.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.365+0000 7f53023e3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc1580a0 con 0x7f52fc15b3e0 2024-09-13T17:54:15.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.365+0000 7f53013e1700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7f52fc1512c0 msgr2=0x7f52fc0c78f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:54:15.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.365+0000 7f53013e1700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f52fc1512c0 0x7f52fc0c78f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:54:15.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.365+0000 7f5300be0700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f52fc0b2d00 0x7f52fc15acd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57114/0 (socket says 172.21.15.40:57114) 2024-09-13T17:54:15.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.365+0000 7f5300be0700 1 -- 172.21.15.40:0/560227660 learned_addr learned my addr 172.21.15.40:0/560227660 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:54:15.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.365+0000 7f5301be2700 1 --2- 172.21.15.40:0/560227660 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52fc15b3e0 0x7f52fc0c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:15.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.369+0000 7f52f3fff700 1 -- 172.21.15.40:0/560227660 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3671924353 0 0) 0x7f52fc15a840 con 0x7f52fc0b2d00 2024-09-13T17:54:15.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.369+0000 7f52f3fff700 1 -- 172.21.15.40:0/560227660 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52e80035d0 con 0x7f52fc0b2d00 2024-09-13T17:54:15.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.369+0000 7f5301be2700 1 -- 172.21.15.40:0/560227660 >> v2:172.21.15.43:3300/0 conn(0x7f52fc1512c0 msgr2=0x7f52fc0c78f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:54:15.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.369+0000 7f5301be2700 1 --2- 172.21.15.40:0/560227660 >> v2:172.21.15.43:3300/0 conn(0x7f52fc1512c0 0x7f52fc0c78f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:15.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.369+0000 7f5301be2700 1 -- 172.21.15.40:0/560227660 >> v1:172.21.15.40:6789/0 conn(0x7f52fc0b2d00 legacy=0x7f52fc15acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:15.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.369+0000 7f5301be2700 1 -- 172.21.15.40:0/560227660 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc157520 con 0x7f52fc15b3e0 2024-09-13T17:54:15.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.369+0000 7f5301be2700 1 --2- 172.21.15.40:0/560227660 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52fc15b3e0 0x7f52fc0c7e30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009d90 tx=0x7f52f400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25796d88f8276e57 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:15.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.369+0000 7f52f3fff700 1 -- 172.21.15.40:0/560227660 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f400e070 con 0x7f52fc15b3e0 2024-09-13T17:54:15.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.369+0000 7f52f3fff700 1 -- 172.21.15.40:0/560227660 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f400bb20 con 0x7f52fc15b3e0 2024-09-13T17:54:15.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.369+0000 7f52f3fff700 1 -- 172.21.15.40:0/560227660 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f400be90 con 0x7f52fc15b3e0 2024-09-13T17:54:15.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.369+0000 7f53023e3700 1 -- 172.21.15.40:0/560227660 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52fc15b3e0 msgr2=0x7f52fc0c7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:15.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.369+0000 7f53023e3700 1 --2- 172.21.15.40:0/560227660 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52fc15b3e0 0x7f52fc0c7e30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009d90 tx=0x7f52f400b330 comp rx=0 tx=0).stop 2024-09-13T17:54:15.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.369+0000 7f53023e3700 1 -- 172.21.15.40:0/560227660 shutdown_connections 2024-09-13T17:54:15.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.369+0000 7f53023e3700 1 --2- 172.21.15.40:0/560227660 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52fc15b3e0 0x7f52fc0c7e30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:15.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.369+0000 7f53023e3700 1 --2- 172.21.15.40:0/560227660 >> v2:172.21.15.43:3300/0 conn(0x7f52fc1512c0 0x7f52fc0c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:15.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.369+0000 7f53023e3700 1 -- 172.21.15.40:0/560227660 >> 172.21.15.40:0/560227660 conn(0x7f52fc0ccef0 msgr2=0x7f52fc0cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:15.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.369+0000 7f53023e3700 1 -- 172.21.15.40:0/560227660 shutdown_connections 2024-09-13T17:54:15.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.369+0000 7f53023e3700 1 -- 172.21.15.40:0/560227660 wait complete. 2024-09-13T17:54:15.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.373+0000 7f53023e3700 1 Processor -- start 2024-09-13T17:54:15.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.373+0000 7f53023e3700 1 -- start start 2024-09-13T17:54:15.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.373+0000 7f53023e3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52fc0b2d00 0x7f52fc1480f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:15.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.373+0000 7f53023e3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52fc1512c0 0x7f52fc14d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:15.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.373+0000 7f53023e3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52fc15b3e0 0x7f52fc0cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:15.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.373+0000 7f53023e3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc0c6d30 con 0x7f52fc1512c0 2024-09-13T17:54:15.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.373+0000 7f53023e3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc14dbd0 con 0x7f52fc15b3e0 2024-09-13T17:54:15.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.373+0000 7f53023e3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc0c6690 con 0x7f52fc0b2d00 2024-09-13T17:54:15.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.373+0000 7f5301be2700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52fc15b3e0 msgr2=0x7f52fc0cb880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:54:15.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.373+0000 7f5300be0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52fc1512c0 0x7f52fc14d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:15.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.373+0000 7f5301be2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52fc15b3e0 0x7f52fc0cb880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:54:15.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.373+0000 7f5300be0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52fc1512c0 0x7f52fc14d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59452/0 (socket says 172.21.15.40:59452) 2024-09-13T17:54:15.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.373+0000 7f5300be0700 1 -- 172.21.15.40:0/2106286985 learned_addr learned my addr 172.21.15.40:0/2106286985 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:15.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.373+0000 7f53013e1700 1 --2- 172.21.15.40:0/2106286985 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52fc0b2d00 0x7f52fc1480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:15.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.373+0000 7f5300be0700 1 -- 172.21.15.40:0/2106286985 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52fc0b2d00 msgr2=0x7f52fc1480f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:15.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.373+0000 7f5300be0700 1 --2- 172.21.15.40:0/2106286985 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52fc0b2d00 0x7f52fc1480f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:15.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.373+0000 7f5300be0700 1 -- 172.21.15.40:0/2106286985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52fc15b3e0 msgr2=0x7f52fc0cb880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:54:15.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.373+0000 7f5300be0700 1 --2- 172.21.15.40:0/2106286985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52fc15b3e0 0x7f52fc0cb880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:15.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.373+0000 7f5300be0700 1 -- 172.21.15.40:0/2106286985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f4009a70 con 0x7f52fc1512c0 2024-09-13T17:54:15.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.373+0000 7f5300be0700 1 --2- 172.21.15.40:0/2106286985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52fc1512c0 0x7f52fc14d690 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00c4d0 tx=0x7f52ec00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:15.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.377+0000 7f52f1ffb700 1 -- 172.21.15.40:0/2106286985 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec0011d0 con 0x7f52fc1512c0 2024-09-13T17:54:15.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.377+0000 7f52f1ffb700 1 -- 172.21.15.40:0/2106286985 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec005d90 con 0x7f52fc1512c0 2024-09-13T17:54:15.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.377+0000 7f53023e3700 1 -- 172.21.15.40:0/2106286985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52fc0c6fa0 con 0x7f52fc1512c0 2024-09-13T17:54:15.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.377+0000 7f52f1ffb700 1 -- 172.21.15.40:0/2106286985 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec0072d0 con 0x7f52fc1512c0 2024-09-13T17:54:15.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.377+0000 7f53023e3700 1 -- 172.21.15.40:0/2106286985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52fc0cbdc0 con 0x7f52fc1512c0 2024-09-13T17:54:15.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.377+0000 7f52f1ffb700 1 -- 172.21.15.40:0/2106286985 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f52ec007430 con 0x7f52fc1512c0 2024-09-13T17:54:15.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.377+0000 7f53023e3700 1 -- 172.21.15.40:0/2106286985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52fc157120 con 0x7f52fc1512c0 2024-09-13T17:54:15.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.377+0000 7f52f1ffb700 1 --2- 172.21.15.40:0/2106286985 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f52e00666e0 0x7f52e0068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:15.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.377+0000 7f52f1ffb700 1 -- 172.21.15.40:0/2106286985 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f52ec0837e0 con 0x7f52fc1512c0 2024-09-13T17:54:15.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.377+0000 7f53013e1700 1 --2- 172.21.15.40:0/2106286985 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f52e00666e0 0x7f52e0068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:15.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.377+0000 7f53013e1700 1 --2- 172.21.15.40:0/2106286985 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f52e00666e0 0x7f52e0068ba0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f52f80039e0 tx=0x7f52f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:15.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.385+0000 7f52f1ffb700 1 -- 172.21.15.40:0/2106286985 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f52ec0508d0 con 0x7f52fc1512c0 2024-09-13T17:54:15.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.653+0000 7f53023e3700 1 -- 172.21.15.40:0/2106286985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f52fc070640 con 0x7f52fc1512c0 2024-09-13T17:54:15.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.653+0000 7f52f1ffb700 1 -- 172.21.15.40:0/2106286985 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f52ec017080 con 0x7f52fc1512c0 2024-09-13T17:54:15.657 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":34,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:54:15.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.653+0000 7f53023e3700 1 -- 172.21.15.40:0/2106286985 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f52e00666e0 msgr2=0x7f52e0068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:15.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.653+0000 7f53023e3700 1 --2- 172.21.15.40:0/2106286985 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f52e00666e0 0x7f52e0068ba0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f52f80039e0 tx=0x7f52f8008040 comp rx=0 tx=0).stop 2024-09-13T17:54:15.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.653+0000 7f53023e3700 1 -- 172.21.15.40:0/2106286985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52fc1512c0 msgr2=0x7f52fc14d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:15.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.653+0000 7f53023e3700 1 --2- 172.21.15.40:0/2106286985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52fc1512c0 0x7f52fc14d690 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00c4d0 tx=0x7f52ec00c890 comp rx=0 tx=0).stop 2024-09-13T17:54:15.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.653+0000 7f53023e3700 1 -- 172.21.15.40:0/2106286985 shutdown_connections 2024-09-13T17:54:15.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.653+0000 7f53023e3700 1 --2- 172.21.15.40:0/2106286985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52fc15b3e0 0x7f52fc0cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:15.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.653+0000 7f53023e3700 1 --2- 172.21.15.40:0/2106286985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52fc1512c0 0x7f52fc14d690 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:15.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.653+0000 7f53023e3700 1 --2- 172.21.15.40:0/2106286985 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f52e00666e0 0x7f52e0068ba0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:15.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.653+0000 7f53023e3700 1 --2- 172.21.15.40:0/2106286985 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52fc0b2d00 0x7f52fc1480f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:15.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.653+0000 7f53023e3700 1 -- 172.21.15.40:0/2106286985 >> 172.21.15.40:0/2106286985 conn(0x7f52fc0ccef0 msgr2=0x7f52fc0c27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:15.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.653+0000 7f53023e3700 1 -- 172.21.15.40:0/2106286985 shutdown_connections 2024-09-13T17:54:15.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:15.657+0000 7f53023e3700 1 -- 172.21.15.40:0/2106286985 wait complete. 2024-09-13T17:54:18.674 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:54:18.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.793+0000 7f2b455b6700 1 Processor -- start 2024-09-13T17:54:18.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.793+0000 7f2b455b6700 1 -- start start 2024-09-13T17:54:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.793+0000 7f2b455b6700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2b4006b4a0 0x7f2b40071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.793+0000 7f2b455b6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2b40072130 0x7f2b40160970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.793+0000 7f2b455b6700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b4015a520 con 0x7f2b401503d0 2024-09-13T17:54:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.793+0000 7f2b455b6700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b40157050 con 0x7f2b4006b4a0 2024-09-13T17:54:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.793+0000 7f2b455b6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b401564b0 con 0x7f2b40072130 2024-09-13T17:54:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.793+0000 7f2b3ffff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2b401503d0 0x7f2b40156940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57122/0 (socket says 172.21.15.40:57122) 2024-09-13T17:54:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.793+0000 7f2b3ffff700 1 -- 172.21.15.40:0/2735834950 learned_addr learned my addr 172.21.15.40:0/2735834950 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:54:18.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.793+0000 7f2b3f7fe700 1 -- 172.21.15.40:0/2735834950 >> v2:172.21.15.43:3300/0 conn(0x7f2b4006b4a0 msgr2=0x7f2b40071bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:54:18.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.793+0000 7f2b3f7fe700 1 --2- 172.21.15.40:0/2735834950 >> v2:172.21.15.43:3300/0 conn(0x7f2b4006b4a0 0x7f2b40071bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:54:18.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.793+0000 7f2b44db5700 1 --2- 172.21.15.40:0/2735834950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2b40072130 0x7f2b40160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:18.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.793+0000 7f2b44db5700 1 -- 172.21.15.40:0/2735834950 >> v2:172.21.15.43:3300/0 conn(0x7f2b4006b4a0 msgr2=0x7f2b40071bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:54:18.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.793+0000 7f2b44db5700 1 --2- 172.21.15.40:0/2735834950 >> v2:172.21.15.43:3300/0 conn(0x7f2b4006b4a0 0x7f2b40071bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.793+0000 7f2b44db5700 1 -- 172.21.15.40:0/2735834950 >> v1:172.21.15.40:6789/0 conn(0x7f2b401503d0 legacy=0x7f2b40156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.793+0000 7f2b44db5700 1 -- 172.21.15.40:0/2735834950 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b4015a000 con 0x7f2b40072130 2024-09-13T17:54:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.793+0000 7f2b44db5700 1 --2- 172.21.15.40:0/2735834950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2b40072130 0x7f2b40160970 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009d90 tx=0x7f2b3800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=614e5a0a38bad0f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.793+0000 7f2b3effd700 1 -- 172.21.15.40:0/2735834950 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3800e070 con 0x7f2b40072130 2024-09-13T17:54:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.797+0000 7f2b3effd700 1 -- 172.21.15.40:0/2735834950 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3800bbd0 con 0x7f2b40072130 2024-09-13T17:54:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.797+0000 7f2b3effd700 1 -- 172.21.15.40:0/2735834950 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38012430 con 0x7f2b40072130 2024-09-13T17:54:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.797+0000 7f2b455b6700 1 -- 172.21.15.40:0/2735834950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2b40072130 msgr2=0x7f2b40160970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.797+0000 7f2b455b6700 1 --2- 172.21.15.40:0/2735834950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2b40072130 0x7f2b40160970 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009d90 tx=0x7f2b3800b330 comp rx=0 tx=0).stop 2024-09-13T17:54:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.797+0000 7f2b455b6700 1 -- 172.21.15.40:0/2735834950 shutdown_connections 2024-09-13T17:54:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.797+0000 7f2b455b6700 1 --2- 172.21.15.40:0/2735834950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2b40072130 0x7f2b40160970 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.797+0000 7f2b455b6700 1 --2- 172.21.15.40:0/2735834950 >> v2:172.21.15.43:3300/0 conn(0x7f2b4006b4a0 0x7f2b40071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.797+0000 7f2b455b6700 1 -- 172.21.15.40:0/2735834950 >> 172.21.15.40:0/2735834950 conn(0x7f2b4006e4f0 msgr2=0x7f2b4006e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.797+0000 7f2b455b6700 1 -- 172.21.15.40:0/2735834950 shutdown_connections 2024-09-13T17:54:18.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.797+0000 7f2b455b6700 1 -- 172.21.15.40:0/2735834950 wait complete. 2024-09-13T17:54:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.797+0000 7f2b455b6700 1 Processor -- start 2024-09-13T17:54:18.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.797+0000 7f2b455b6700 1 -- start start 2024-09-13T17:54:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b455b6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b4006b4a0 0x7f2b40155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b455b6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b40072130 0x7f2b40153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b455b6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2b401503d0 0x7f2b401ec320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b455b6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b401ef620 con 0x7f2b40072130 2024-09-13T17:54:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b455b6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b401ef870 con 0x7f2b4006b4a0 2024-09-13T17:54:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b455b6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b401efb50 con 0x7f2b401503d0 2024-09-13T17:54:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b3ffff700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b4006b4a0 msgr2=0x7f2b40155500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:54:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b3ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b4006b4a0 0x7f2b40155500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:54:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b3f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b40072130 0x7f2b40153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b3f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b40072130 0x7f2b40153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59462/0 (socket says 172.21.15.40:59462) 2024-09-13T17:54:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b3f7fe700 1 -- 172.21.15.40:0/1192978870 learned_addr learned my addr 172.21.15.40:0/1192978870 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:18.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b44db5700 1 --2- 172.21.15.40:0/1192978870 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2b401503d0 0x7f2b401ec320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b3f7fe700 1 -- 172.21.15.40:0/1192978870 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2b401503d0 msgr2=0x7f2b401ec320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b3f7fe700 1 --2- 172.21.15.40:0/1192978870 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2b401503d0 0x7f2b401ec320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b3f7fe700 1 -- 172.21.15.40:0/1192978870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b4006b4a0 msgr2=0x7f2b40155500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:54:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b3f7fe700 1 --2- 172.21.15.40:0/1192978870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b4006b4a0 0x7f2b40155500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:18.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b3f7fe700 1 -- 172.21.15.40:0/1192978870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b38009a70 con 0x7f2b40072130 2024-09-13T17:54:18.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b3f7fe700 1 --2- 172.21.15.40:0/1192978870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b40072130 0x7f2b40153300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400dad0 tx=0x7f2b3400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:18.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b3cff9700 1 -- 172.21.15.40:0/1192978870 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b34003e70 con 0x7f2b40072130 2024-09-13T17:54:18.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b3cff9700 1 -- 172.21.15.40:0/1192978870 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3400e8c0 con 0x7f2b40072130 2024-09-13T17:54:18.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b455b6700 1 -- 172.21.15.40:0/1192978870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b401efe30 con 0x7f2b40072130 2024-09-13T17:54:18.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b3cff9700 1 -- 172.21.15.40:0/1192978870 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3400ec70 con 0x7f2b40072130 2024-09-13T17:54:18.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b455b6700 1 -- 172.21.15.40:0/1192978870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b401f03f0 con 0x7f2b40072130 2024-09-13T17:54:18.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.801+0000 7f2b455b6700 1 -- 172.21.15.40:0/1192978870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b401ec0e0 con 0x7f2b40072130 2024-09-13T17:54:18.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.805+0000 7f2b3cff9700 1 -- 172.21.15.40:0/1192978870 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2b34014070 con 0x7f2b40072130 2024-09-13T17:54:18.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.805+0000 7f2b3cff9700 1 --2- 172.21.15.40:0/1192978870 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2b2c0666e0 0x7f2b2c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:18.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.805+0000 7f2b3cff9700 1 -- 172.21.15.40:0/1192978870 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2b34086240 con 0x7f2b40072130 2024-09-13T17:54:18.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.805+0000 7f2b3ffff700 1 --2- 172.21.15.40:0/1192978870 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2b2c0666e0 0x7f2b2c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:18.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.805+0000 7f2b3ffff700 1 --2- 172.21.15.40:0/1192978870 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2b2c0666e0 0x7f2b2c068ba0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2b300048c0 tx=0x7f2b30007b90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:18.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:18.813+0000 7f2b3cff9700 1 -- 172.21.15.40:0/1192978870 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2b340531f0 con 0x7f2b40072130 2024-09-13T17:54:19.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:19.109+0000 7f2b455b6700 1 -- 172.21.15.40:0/1192978870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2b401507b0 con 0x7f2b40072130 2024-09-13T17:54:19.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:19.109+0000 7f2b3cff9700 1 -- 172.21.15.40:0/1192978870 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f2b34056810 con 0x7f2b40072130 2024-09-13T17:54:19.114 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":34,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:54:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:19.113+0000 7f2b455b6700 1 -- 172.21.15.40:0/1192978870 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2b2c0666e0 msgr2=0x7f2b2c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:19.113+0000 7f2b455b6700 1 --2- 172.21.15.40:0/1192978870 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2b2c0666e0 0x7f2b2c068ba0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2b300048c0 tx=0x7f2b30007b90 comp rx=0 tx=0).stop 2024-09-13T17:54:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:19.113+0000 7f2b455b6700 1 -- 172.21.15.40:0/1192978870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b40072130 msgr2=0x7f2b40153300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:19.113+0000 7f2b455b6700 1 --2- 172.21.15.40:0/1192978870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b40072130 0x7f2b40153300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400dad0 tx=0x7f2b3400dde0 comp rx=0 tx=0).stop 2024-09-13T17:54:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:19.113+0000 7f2b455b6700 1 -- 172.21.15.40:0/1192978870 shutdown_connections 2024-09-13T17:54:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:19.113+0000 7f2b455b6700 1 --2- 172.21.15.40:0/1192978870 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2b2c0666e0 0x7f2b2c068ba0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:19.113+0000 7f2b455b6700 1 --2- 172.21.15.40:0/1192978870 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2b401503d0 0x7f2b401ec320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:19.113+0000 7f2b455b6700 1 --2- 172.21.15.40:0/1192978870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b40072130 0x7f2b40153300 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:19.113+0000 7f2b455b6700 1 --2- 172.21.15.40:0/1192978870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b4006b4a0 0x7f2b40155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:19.113+0000 7f2b455b6700 1 -- 172.21.15.40:0/1192978870 >> 172.21.15.40:0/1192978870 conn(0x7f2b4006e4f0 msgr2=0x7f2b400c1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:19.113+0000 7f2b455b6700 1 -- 172.21.15.40:0/1192978870 shutdown_connections 2024-09-13T17:54:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:19.113+0000 7f2b455b6700 1 -- 172.21.15.40:0/1192978870 wait complete. 2024-09-13T17:54:22.135 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:54:22.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.253+0000 7fb231ab6700 1 Processor -- start 2024-09-13T17:54:22.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.253+0000 7fb231ab6700 1 -- start start 2024-09-13T17:54:22.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.253+0000 7fb231ab6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 0x7fb22c06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:22.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.253+0000 7fb231ab6700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb22c153830 0x7fb22c15d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:22.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.253+0000 7fb231ab6700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb22c06eb10 con 0x7fb22c15cfa0 2024-09-13T17:54:22.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.253+0000 7fb231ab6700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb22c0c7bb0 con 0x7fb22c153830 2024-09-13T17:54:22.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.253+0000 7fb231ab6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c06b4c0 con 0x7fb22c15cbc0 2024-09-13T17:54:22.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.253+0000 7fb2312b5700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7fb22c153830 msgr2=0x7fb22c15d3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:54:22.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.253+0000 7fb22bfff700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7fb22c15cfa0 legacy=0x7fb22c153120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T17:54:22.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.253+0000 7fb2312b5700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb22c153830 0x7fb22c15d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:54:22.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.253+0000 7fb230ab4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 0x7fb22c06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:22.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.253+0000 7fb230ab4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 0x7fb22c06e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:57140/0 (socket says 172.21.15.40:57140) 2024-09-13T17:54:22.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.253+0000 7fb230ab4700 1 -- 172.21.15.40:0/1286247127 learned_addr learned my addr 172.21.15.40:0/1286247127 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:22.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.253+0000 7fb230ab4700 1 -- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 msgr2=0x7fb22c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:54:22.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.253+0000 7fb230ab4700 1 -- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 msgr2=0x7fb22c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:54:22.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.253+0000 7fb230ab4700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 0x7fb22c06e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:54:22.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.253+0000 7fb230ab4700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 0x7fb22c06e5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:54:22.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.457+0000 7fb22bfff700 1 -- 172.21.15.40:0/1286247127 >> v1:172.21.15.43:6789/0 conn(0x7fb22c15cfa0 legacy=0x7fb22c153120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T17:54:22.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.457+0000 7fb2312b5700 1 -- 172.21.15.40:0/1286247127 >> v2:172.21.15.43:3300/0 conn(0x7fb22c153830 msgr2=0x7fb22c15d3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:54:22.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.457+0000 7fb2312b5700 1 --2- 172.21.15.40:0/1286247127 >> v2:172.21.15.43:3300/0 conn(0x7fb22c153830 0x7fb22c15d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:54:22.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.457+0000 7fb230ab4700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 0x7fb22c06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:22.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.457+0000 7fb230ab4700 1 -- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 msgr2=0x7fb22c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:54:22.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.457+0000 7fb230ab4700 1 -- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 msgr2=0x7fb22c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:54:22.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.457+0000 7fb230ab4700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 0x7fb22c06e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:54:22.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.457+0000 7fb230ab4700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 0x7fb22c06e5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:54:22.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.861+0000 7fb22bfff700 1 -- 172.21.15.40:0/1286247127 >> v1:172.21.15.43:6789/0 conn(0x7fb22c15cfa0 legacy=0x7fb22c153120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T17:54:22.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.861+0000 7fb2312b5700 1 -- 172.21.15.40:0/1286247127 >> v2:172.21.15.43:3300/0 conn(0x7fb22c153830 msgr2=0x7fb22c15d3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:54:22.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.861+0000 7fb2312b5700 1 --2- 172.21.15.40:0/1286247127 >> v2:172.21.15.43:3300/0 conn(0x7fb22c153830 0x7fb22c15d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:54:22.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.861+0000 7fb230ab4700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 0x7fb22c06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:22.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.861+0000 7fb230ab4700 1 -- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 msgr2=0x7fb22c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:54:22.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.861+0000 7fb230ab4700 1 -- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 msgr2=0x7fb22c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:54:22.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.861+0000 7fb230ab4700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 0x7fb22c06e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:54:22.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:22.861+0000 7fb230ab4700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 0x7fb22c06e5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:54:23.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:23.665+0000 7fb2312b5700 1 -- 172.21.15.40:0/1286247127 >> v2:172.21.15.43:3300/0 conn(0x7fb22c153830 msgr2=0x7fb22c15d3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:54:23.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:23.665+0000 7fb22bfff700 1 -- 172.21.15.40:0/1286247127 >> v1:172.21.15.43:6789/0 conn(0x7fb22c15cfa0 legacy=0x7fb22c153120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T17:54:23.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:23.665+0000 7fb2312b5700 1 --2- 172.21.15.40:0/1286247127 >> v2:172.21.15.43:3300/0 conn(0x7fb22c153830 0x7fb22c15d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:54:23.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:23.665+0000 7fb230ab4700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 0x7fb22c06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:23.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:23.665+0000 7fb230ab4700 1 -- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 msgr2=0x7fb22c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:54:23.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:23.665+0000 7fb230ab4700 1 -- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 msgr2=0x7fb22c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:54:23.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:23.665+0000 7fb230ab4700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 0x7fb22c06e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:54:23.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:23.665+0000 7fb230ab4700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 0x7fb22c06e5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:54:25.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.253+0000 7fb22a7fc700 1 -- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 msgr2=0x7fb22c06e5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:54:25.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.253+0000 7fb22a7fc700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb22c15cbc0 0x7fb22c06e5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:25.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.253+0000 7fb22a7fc700 1 -- 172.21.15.40:0/1286247127 >> v2:172.21.15.43:3300/0 conn(0x7fb22c153830 msgr2=0x7fb22c15d3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:54:25.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.253+0000 7fb22a7fc700 1 --2- 172.21.15.40:0/1286247127 >> v2:172.21.15.43:3300/0 conn(0x7fb22c153830 0x7fb22c15d3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:25.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.253+0000 7fb22a7fc700 1 -- 172.21.15.40:0/1286247127 >> v1:172.21.15.43:6789/0 conn(0x7fb22c15cfa0 legacy=0x7fb22c153120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:54:25.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.253+0000 7fb22a7fc700 1 --2- 172.21.15.40:0/1286247127 >> v2:172.21.15.40:3300/0 conn(0x7fb210007460 0x7fb210009920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:25.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.253+0000 7fb22a7fc700 1 -- 172.21.15.40:0/1286247127 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb22c06b7a0 con 0x7fb210003720 2024-09-13T17:54:25.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.253+0000 7fb22a7fc700 1 -- 172.21.15.40:0/1286247127 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb22c0c7bb0 con 0x7fb210007460 2024-09-13T17:54:25.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.253+0000 7fb22bfff700 1 --2- 172.21.15.40:0/1286247127 >> v2:172.21.15.40:3300/0 conn(0x7fb210007460 0x7fb210009920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:25.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.257+0000 7fb22bfff700 1 -- 172.21.15.40:0/1286247127 >> v2:172.21.15.40:3300/0 conn(0x7fb210007460 msgr2=0x7fb210009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:54:25.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.257+0000 7fb22bfff700 1 -- 172.21.15.40:0/1286247127 >> v2:172.21.15.40:3300/0 conn(0x7fb210007460 msgr2=0x7fb210009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:54:25.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.257+0000 7fb22bfff700 1 --2- 172.21.15.40:0/1286247127 >> v2:172.21.15.40:3300/0 conn(0x7fb210007460 0x7fb210009920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:54:25.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.257+0000 7fb22bfff700 1 --2- 172.21.15.40:0/1286247127 >> v2:172.21.15.40:3300/0 conn(0x7fb210007460 0x7fb210009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:54:25.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.461+0000 7fb22bfff700 1 --2- 172.21.15.40:0/1286247127 >> v2:172.21.15.40:3300/0 conn(0x7fb210007460 0x7fb210009920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:25.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.461+0000 7fb22bfff700 1 -- 172.21.15.40:0/1286247127 >> v2:172.21.15.40:3300/0 conn(0x7fb210007460 msgr2=0x7fb210009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:54:25.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.461+0000 7fb22bfff700 1 -- 172.21.15.40:0/1286247127 >> v2:172.21.15.40:3300/0 conn(0x7fb210007460 msgr2=0x7fb210009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:54:25.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.461+0000 7fb22bfff700 1 --2- 172.21.15.40:0/1286247127 >> v2:172.21.15.40:3300/0 conn(0x7fb210007460 0x7fb210009920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:54:25.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.461+0000 7fb22bfff700 1 --2- 172.21.15.40:0/1286247127 >> v2:172.21.15.40:3300/0 conn(0x7fb210007460 0x7fb210009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:54:25.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.865+0000 7fb22bfff700 1 --2- 172.21.15.40:0/1286247127 >> v2:172.21.15.40:3300/0 conn(0x7fb210007460 0x7fb210009920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:25.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.865+0000 7fb22bfff700 1 -- 172.21.15.40:0/1286247127 >> v2:172.21.15.40:3300/0 conn(0x7fb210007460 msgr2=0x7fb210009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:54:25.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.865+0000 7fb22bfff700 1 -- 172.21.15.40:0/1286247127 >> v2:172.21.15.40:3300/0 conn(0x7fb210007460 msgr2=0x7fb210009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:54:25.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.865+0000 7fb22bfff700 1 --2- 172.21.15.40:0/1286247127 >> v2:172.21.15.40:3300/0 conn(0x7fb210007460 0x7fb210009920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:54:25.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:25.865+0000 7fb22bfff700 1 --2- 172.21.15.40:0/1286247127 >> v2:172.21.15.40:3300/0 conn(0x7fb210007460 0x7fb210009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:54:26.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.273+0000 7fb22b7fe700 1 -- 172.21.15.40:0/1286247127 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296853683 0 0) 0x7fb22c06b7a0 con 0x7fb210003720 2024-09-13T17:54:26.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.273+0000 7fb22b7fe700 1 -- 172.21.15.40:0/1286247127 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb21c0035d0 con 0x7fb210003720 2024-09-13T17:54:26.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.273+0000 7fb22b7fe700 1 -- 172.21.15.40:0/1286247127 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4108386274 0 0) 0x7fb21c0035d0 con 0x7fb210003720 2024-09-13T17:54:26.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.273+0000 7fb22b7fe700 1 -- 172.21.15.40:0/1286247127 >> v2:172.21.15.40:3300/0 conn(0x7fb210007460 msgr2=0x7fb210009920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:54:26.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.273+0000 7fb22b7fe700 1 --2- 172.21.15.40:0/1286247127 >> v2:172.21.15.40:3300/0 conn(0x7fb210007460 0x7fb210009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.273+0000 7fb22b7fe700 1 -- 172.21.15.40:0/1286247127 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22c06eb10 con 0x7fb210003720 2024-09-13T17:54:26.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.273+0000 7fb22b7fe700 1 -- 172.21.15.40:0/1286247127 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fb22000c570 con 0x7fb210003720 2024-09-13T17:54:26.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb22b7fe700 1 -- 172.21.15.40:0/1286247127 >> v1:172.21.15.40:6789/0 conn(0x7fb210003720 legacy=0x7fb210005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb2312b5700 1 -- 172.21.15.40:0/1286247127 reap_dead start 2024-09-13T17:54:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb22b7fe700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb21c003a20 0x7fb21c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb22b7fe700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb21c004870 0x7fb21c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb22b7fe700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb21c005280 0x7fb21c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb22b7fe700 1 -- 172.21.15.40:0/1286247127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c0c7bb0 con 0x7fb21c005280 2024-09-13T17:54:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb22b7fe700 1 -- 172.21.15.40:0/1286247127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c00c1b0 con 0x7fb21c004870 2024-09-13T17:54:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb22b7fe700 1 -- 172.21.15.40:0/1286247127 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c00c370 con 0x7fb21c003a20 2024-09-13T17:54:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb22bfff700 1 -- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb21c004870 msgr2=0x7fb21c004cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:54:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb230ab4700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb21c003a20 0x7fb21c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb22bfff700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb21c004870 0x7fb21c004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:54:26.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb2312b5700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb21c005280 0x7fb21c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:26.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb2312b5700 1 -- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb21c003a20 msgr2=0x7fb21c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:26.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb2312b5700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb21c003a20 0x7fb21c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb2312b5700 1 -- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb21c004870 msgr2=0x7fb21c004cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:54:26.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb2312b5700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb21c004870 0x7fb21c004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb2312b5700 1 -- 172.21.15.40:0/1286247127 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21c0035d0 con 0x7fb21c005280 2024-09-13T17:54:26.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb230ab4700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb21c003a20 0x7fb21c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:54:26.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb2312b5700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb21c005280 0x7fb21c00ba30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb210009ee0 tx=0x7fb21c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75684dadcfe83ef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:26.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb22b7fe700 1 -- 172.21.15.40:0/1286247127 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb224007940 con 0x7fb21c005280 2024-09-13T17:54:26.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb22b7fe700 1 -- 172.21.15.40:0/1286247127 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22400cca0 con 0x7fb21c005280 2024-09-13T17:54:26.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb22b7fe700 1 -- 172.21.15.40:0/1286247127 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22400f480 con 0x7fb21c005280 2024-09-13T17:54:26.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb231ab6700 1 -- 172.21.15.40:0/1286247127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb21c005280 msgr2=0x7fb21c00ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:26.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb231ab6700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb21c005280 0x7fb21c00ba30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb210009ee0 tx=0x7fb21c0058a0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb231ab6700 1 -- 172.21.15.40:0/1286247127 shutdown_connections 2024-09-13T17:54:26.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb231ab6700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb21c005280 0x7fb21c00ba30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb231ab6700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb21c004870 0x7fb21c004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb231ab6700 1 --2- 172.21.15.40:0/1286247127 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb21c003a20 0x7fb21c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb231ab6700 1 -- 172.21.15.40:0/1286247127 >> 172.21.15.40:0/1286247127 conn(0x7fb22c0ccef0 msgr2=0x7fb22c0cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:26.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb231ab6700 1 -- 172.21.15.40:0/1286247127 shutdown_connections 2024-09-13T17:54:26.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.277+0000 7fb231ab6700 1 -- 172.21.15.40:0/1286247127 wait complete. 2024-09-13T17:54:26.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.281+0000 7fb231ab6700 1 Processor -- start 2024-09-13T17:54:26.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.281+0000 7fb231ab6700 1 -- start start 2024-09-13T17:54:26.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.281+0000 7fb231ab6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb21c003a20 0x7fb22c15c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.281+0000 7fb231ab6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb21c004870 0x7fb22c159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.281+0000 7fb231ab6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb21c005280 0x7fb22c15b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.281+0000 7fb231ab6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c1efa10 con 0x7fb21c004870 2024-09-13T17:54:26.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.281+0000 7fb231ab6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c1eeb20 con 0x7fb21c005280 2024-09-13T17:54:26.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.281+0000 7fb231ab6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c1eee00 con 0x7fb21c003a20 2024-09-13T17:54:26.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.281+0000 7fb2312b5700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb21c005280 msgr2=0x7fb22c15b7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:54:26.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.281+0000 7fb2312b5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb21c005280 0x7fb22c15b7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:54:26.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.281+0000 7fb22bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb21c004870 0x7fb22c159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:26.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.281+0000 7fb22bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb21c004870 0x7fb22c159e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38422/0 (socket says 172.21.15.40:38422) 2024-09-13T17:54:26.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.281+0000 7fb22bfff700 1 -- 172.21.15.40:0/3902607843 learned_addr learned my addr 172.21.15.40:0/3902607843 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:26.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.281+0000 7fb230ab4700 1 --2- 172.21.15.40:0/3902607843 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb21c003a20 0x7fb22c15c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:26.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.285+0000 7fb22bfff700 1 -- 172.21.15.40:0/3902607843 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb21c003a20 msgr2=0x7fb22c15c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:26.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.285+0000 7fb22bfff700 1 --2- 172.21.15.40:0/3902607843 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb21c003a20 0x7fb22c15c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.285+0000 7fb22bfff700 1 -- 172.21.15.40:0/3902607843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb21c005280 msgr2=0x7fb22c15b7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:54:26.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.285+0000 7fb22bfff700 1 --2- 172.21.15.40:0/3902607843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb21c005280 0x7fb22c15b7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.285+0000 7fb22bfff700 1 -- 172.21.15.40:0/3902607843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22c1ef0e0 con 0x7fb21c004870 2024-09-13T17:54:26.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.285+0000 7fb22bfff700 1 --2- 172.21.15.40:0/3902607843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb21c004870 0x7fb22c159e00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb21800be60 tx=0x7fb218005330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:26.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.285+0000 7fb2297fa700 1 -- 172.21.15.40:0/3902607843 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb218016610 con 0x7fb21c004870 2024-09-13T17:54:26.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.285+0000 7fb2297fa700 1 -- 172.21.15.40:0/3902607843 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb218011050 con 0x7fb21c004870 2024-09-13T17:54:26.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.285+0000 7fb231ab6700 1 -- 172.21.15.40:0/3902607843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb22c1ef3c0 con 0x7fb21c004870 2024-09-13T17:54:26.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.285+0000 7fb2297fa700 1 -- 172.21.15.40:0/3902607843 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2180169f0 con 0x7fb21c004870 2024-09-13T17:54:26.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.285+0000 7fb231ab6700 1 -- 172.21.15.40:0/3902607843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb22c0c96c0 con 0x7fb21c004870 2024-09-13T17:54:26.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.285+0000 7fb2297fa700 1 -- 172.21.15.40:0/3902607843 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb21800a3e0 con 0x7fb21c004870 2024-09-13T17:54:26.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.285+0000 7fb2297fa700 1 --2- 172.21.15.40:0/3902607843 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb210066da0 0x7fb210069260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.285+0000 7fb231ab6700 1 -- 172.21.15.40:0/3902607843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb22c06bd40 con 0x7fb21c004870 2024-09-13T17:54:26.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.285+0000 7fb2297fa700 1 -- 172.21.15.40:0/3902607843 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb218084b60 con 0x7fb21c004870 2024-09-13T17:54:26.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.285+0000 7fb230ab4700 1 --2- 172.21.15.40:0/3902607843 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb210066da0 0x7fb210069260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:26.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.289+0000 7fb230ab4700 1 --2- 172.21.15.40:0/3902607843 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb210066da0 0x7fb210069260 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb220000c00 tx=0x7fb220008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:26.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.293+0000 7fb2297fa700 1 -- 172.21.15.40:0/3902607843 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb22c06bd40 con 0x7fb21c004870 2024-09-13T17:54:26.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.565+0000 7fb231ab6700 1 -- 172.21.15.40:0/3902607843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb22c06bd40 con 0x7fb21c004870 2024-09-13T17:54:26.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.569+0000 7fb2297fa700 1 -- 172.21.15.40:0/3902607843 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7fb22c06bd40 con 0x7fb21c004870 2024-09-13T17:54:26.572 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":38,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:54:26.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.569+0000 7fb231ab6700 1 -- 172.21.15.40:0/3902607843 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb210066da0 msgr2=0x7fb210069260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:26.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.569+0000 7fb231ab6700 1 --2- 172.21.15.40:0/3902607843 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb210066da0 0x7fb210069260 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb220000c00 tx=0x7fb220008040 comp rx=0 tx=0).stop 2024-09-13T17:54:26.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.569+0000 7fb231ab6700 1 -- 172.21.15.40:0/3902607843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb21c004870 msgr2=0x7fb22c159e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:26.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.569+0000 7fb231ab6700 1 --2- 172.21.15.40:0/3902607843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb21c004870 0x7fb22c159e00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb21800be60 tx=0x7fb218005330 comp rx=0 tx=0).stop 2024-09-13T17:54:26.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.569+0000 7fb231ab6700 1 -- 172.21.15.40:0/3902607843 shutdown_connections 2024-09-13T17:54:26.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.569+0000 7fb231ab6700 1 --2- 172.21.15.40:0/3902607843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb21c005280 0x7fb22c15b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.569+0000 7fb231ab6700 1 --2- 172.21.15.40:0/3902607843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb21c004870 0x7fb22c159e00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.569+0000 7fb231ab6700 1 --2- 172.21.15.40:0/3902607843 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb210066da0 0x7fb210069260 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.569+0000 7fb231ab6700 1 --2- 172.21.15.40:0/3902607843 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb21c003a20 0x7fb22c15c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.569+0000 7fb231ab6700 1 -- 172.21.15.40:0/3902607843 >> 172.21.15.40:0/3902607843 conn(0x7fb22c0ccef0 msgr2=0x7fb22c0c99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:26.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.569+0000 7fb231ab6700 1 -- 172.21.15.40:0/3902607843 shutdown_connections 2024-09-13T17:54:26.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.569+0000 7fb231ab6700 1 -- 172.21.15.40:0/3902607843 wait complete. 2024-09-13T17:54:26.595 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T17:54:26.595 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-13T17:54:26.595 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:54:26.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.705+0000 7fa2c759e700 1 Processor -- start 2024-09-13T17:54:26.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.705+0000 7fa2c759e700 1 -- start start 2024-09-13T17:54:26.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c759e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa2c814fce0 0x7fa2c81562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c759e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c80c99e0 con 0x7fa2c80cb3b0 2024-09-13T17:54:26.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c759e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c80c9f60 con 0x7fa2c80cad30 2024-09-13T17:54:26.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c759e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c81500c0 con 0x7fa2c814fce0 2024-09-13T17:54:26.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c659c700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7fa2c814fce0 msgr2=0x7fa2c81562f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:54:26.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c659c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa2c814fce0 0x7fa2c81562f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:54:26.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c6d9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa2c80cb3b0 0x7fa2c8160220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32900/0 (socket says 172.21.15.40:32900) 2024-09-13T17:54:26.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c6d9d700 1 -- 172.21.15.40:0/453764606 learned_addr learned my addr 172.21.15.40:0/453764606 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:54:26.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c5d9b700 1 -- 172.21.15.40:0/453764606 >> v1:172.21.15.43:6789/0 conn(0x7fa2c80cad30 legacy=0x7fa2c8156d90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T17:54:26.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c559a700 1 -- 172.21.15.40:0/453764606 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150625223 0 0) 0x7fa2c80c99e0 con 0x7fa2c80cb3b0 2024-09-13T17:54:26.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c559a700 1 -- 172.21.15.40:0/453764606 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2b00035d0 con 0x7fa2c80cb3b0 2024-09-13T17:54:26.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c559a700 1 -- 172.21.15.40:0/453764606 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3157944344 0 0) 0x7fa2b00035d0 con 0x7fa2c80cb3b0 2024-09-13T17:54:26.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c559a700 1 -- 172.21.15.40:0/453764606 >> v2:172.21.15.43:3300/0 conn(0x7fa2c814fce0 msgr2=0x7fa2c81562f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:54:26.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c559a700 1 --2- 172.21.15.40:0/453764606 >> v2:172.21.15.43:3300/0 conn(0x7fa2c814fce0 0x7fa2c81562f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c559a700 1 -- 172.21.15.40:0/453764606 >> v1:172.21.15.43:6789/0 conn(0x7fa2c80cad30 legacy=0x7fa2c8156d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:54:26.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c559a700 1 -- 172.21.15.40:0/453764606 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c8159cf0 con 0x7fa2c80cb3b0 2024-09-13T17:54:26.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c559a700 1 -- 172.21.15.40:0/453764606 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fa2c00019f0 con 0x7fa2c80cb3b0 2024-09-13T17:54:26.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c559a700 1 -- 172.21.15.40:0/453764606 >> v1:172.21.15.40:6789/0 conn(0x7fa2c80cb3b0 legacy=0x7fa2c8160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:26.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c559a700 1 --2- 172.21.15.40:0/453764606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b0003a20 0x7fa2b0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c559a700 1 --2- 172.21.15.40:0/453764606 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa2b0004870 0x7fa2b0004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c559a700 1 --2- 172.21.15.40:0/453764606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b0005280 0x7fa2b000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c559a700 1 -- 172.21.15.40:0/453764606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c81500c0 con 0x7fa2b0005280 2024-09-13T17:54:26.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c559a700 1 -- 172.21.15.40:0/453764606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b000c190 con 0x7fa2b0003a20 2024-09-13T17:54:26.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c559a700 1 -- 172.21.15.40:0/453764606 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b000c350 con 0x7fa2b0004870 2024-09-13T17:54:26.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c659c700 1 -- 172.21.15.40:0/453764606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b0003a20 msgr2=0x7fa2b0003e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:54:26.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c659c700 1 --2- 172.21.15.40:0/453764606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b0003a20 0x7fa2b0003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:54:26.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c6d9d700 1 --2- 172.21.15.40:0/453764606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b0005280 0x7fa2b000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:26.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c5d9b700 1 --2- 172.21.15.40:0/453764606 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa2b0004870 0x7fa2b0004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:26.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c6d9d700 1 -- 172.21.15.40:0/453764606 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa2b0004870 msgr2=0x7fa2b0004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:26.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c6d9d700 1 --2- 172.21.15.40:0/453764606 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa2b0004870 0x7fa2b0004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c6d9d700 1 -- 172.21.15.40:0/453764606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b0003a20 msgr2=0x7fa2b0003e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:54:26.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c6d9d700 1 --2- 172.21.15.40:0/453764606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2b0003a20 0x7fa2b0003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c6d9d700 1 -- 172.21.15.40:0/453764606 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b00035d0 con 0x7fa2b0005280 2024-09-13T17:54:26.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c6d9d700 1 -- 172.21.15.40:0/453764606 reap_dead start 2024-09-13T17:54:26.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c6d9d700 1 --2- 172.21.15.40:0/453764606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b0005280 0x7fa2b000ba30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000a810 tx=0x7fa2b00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c025cc7ce2da2a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:26.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c559a700 1 -- 172.21.15.40:0/453764606 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c000a2d0 con 0x7fa2b0005280 2024-09-13T17:54:26.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c559a700 1 -- 172.21.15.40:0/453764606 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c00072d0 con 0x7fa2b0005280 2024-09-13T17:54:26.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.709+0000 7fa2c559a700 1 -- 172.21.15.40:0/453764606 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0007680 con 0x7fa2b0005280 2024-09-13T17:54:26.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.713+0000 7fa2c759e700 1 -- 172.21.15.40:0/453764606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b0005280 msgr2=0x7fa2b000ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:26.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.713+0000 7fa2c759e700 1 --2- 172.21.15.40:0/453764606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b0005280 0x7fa2b000ba30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa2c000a810 tx=0x7fa2b00058a0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.713+0000 7fa2c759e700 1 -- 172.21.15.40:0/453764606 shutdown_connections 2024-09-13T17:54:26.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.713+0000 7fa2c759e700 1 --2- 172.21.15.40:0/453764606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2b0005280 0x7fa2b000ba30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.713+0000 7fa2c759e700 1 -- 172.21.15.40:0/453764606 >> 172.21.15.40:0/453764606 conn(0x7fa2c806dcf0 msgr2=0x7fa2c806e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:26.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.713+0000 7fa2c759e700 1 -- 172.21.15.40:0/453764606 shutdown_connections 2024-09-13T17:54:26.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.713+0000 7fa2c759e700 1 -- 172.21.15.40:0/453764606 wait complete. 2024-09-13T17:54:26.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.713+0000 7fa2c759e700 1 Processor -- start 2024-09-13T17:54:26.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.713+0000 7fa2c759e700 1 -- start start 2024-09-13T17:54:26.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa2b0005280 0x7fa2c80715d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2c8143810 0x7fa2c80c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c80c6010 0x7fa2c80c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c8158d10 con 0x7fa2c80c6010 2024-09-13T17:54:26.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c8156f80 con 0x7fa2c8143810 2024-09-13T17:54:26.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c8142e40 con 0x7fa2b0005280 2024-09-13T17:54:26.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c5d9b700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2c8143810 msgr2=0x7fa2c80c5ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:54:26.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c80c6010 0x7fa2c80c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:26.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c5d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2c8143810 0x7fa2c80c5ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:54:26.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c80c6010 0x7fa2c80c66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38430/0 (socket says 172.21.15.40:38430) 2024-09-13T17:54:26.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c6d9d700 1 -- 172.21.15.40:0/3132576956 learned_addr learned my addr 172.21.15.40:0/3132576956 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:26.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c659c700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa2b0005280 0x7fa2c80715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:26.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c6d9d700 1 -- 172.21.15.40:0/3132576956 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa2b0005280 msgr2=0x7fa2c80715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:26.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c6d9d700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa2b0005280 0x7fa2c80715d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c6d9d700 1 -- 172.21.15.40:0/3132576956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2c8143810 msgr2=0x7fa2c80c5ad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:54:26.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c6d9d700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2c8143810 0x7fa2c80c5ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c6d9d700 1 -- 172.21.15.40:0/3132576956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c8141b30 con 0x7fa2c80c6010 2024-09-13T17:54:26.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c659c700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa2b0005280 0x7fa2c80715d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:54:26.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c6d9d700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c80c6010 0x7fa2c80c66e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa2c81534f0 tx=0x7fa2c81558c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:26.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2b6ffd700 1 -- 172.21.15.40:0/3132576956 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0025030 con 0x7fa2c80c6010 2024-09-13T17:54:26.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2b6ffd700 1 -- 172.21.15.40:0/3132576956 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c000ae70 con 0x7fa2c80c6010 2024-09-13T17:54:26.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c759e700 1 -- 172.21.15.40:0/3132576956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c814e7e0 con 0x7fa2c80c6010 2024-09-13T17:54:26.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2b6ffd700 1 -- 172.21.15.40:0/3132576956 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c000a2d0 con 0x7fa2c80c6010 2024-09-13T17:54:26.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c759e700 1 -- 172.21.15.40:0/3132576956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c8142080 con 0x7fa2c80c6010 2024-09-13T17:54:26.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c759e700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2ac002a40 0x7fa2ac004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c759e700 1 -- 172.21.15.40:0/3132576956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac007f40 con 0x7fa2ac002a40 2024-09-13T17:54:26.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c659c700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2ac002a40 0x7fa2ac004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:26.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c659c700 1 -- 172.21.15.40:0/3132576956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa2ac008100 con 0x7fa2ac002a40 2024-09-13T17:54:26.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.717+0000 7fa2c659c700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2ac002a40 0x7fa2ac004f00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80039e0 tx=0x7fa2b8009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:26.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.721+0000 7fa2b6ffd700 1 -- 172.21.15.40:0/3132576956 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b800dda0 con 0x7fa2ac002a40 2024-09-13T17:54:26.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.721+0000 7fa2b6ffd700 1 -- 172.21.15.40:0/3132576956 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa2b8015420 con 0x7fa2ac002a40 2024-09-13T17:54:26.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.721+0000 7fa2b6ffd700 1 -- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2ac002a40 msgr2=0x7fa2ac004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:26.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.721+0000 7fa2b6ffd700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2ac002a40 0x7fa2ac004f00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80039e0 tx=0x7fa2b8009eb0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.721+0000 7fa2b6ffd700 1 -- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2ac002a40 msgr2=0x7fa2ac004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:54:26.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.721+0000 7fa2b6ffd700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2ac002a40 0x7fa2ac004f00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.721+0000 7fa2b6ffd700 1 -- 172.21.15.40:0/3132576956 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa2c0028070 con 0x7fa2c80c6010 2024-09-13T17:54:26.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.721+0000 7fa2b6ffd700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa2a80663e0 0x7fa2a80688a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.721+0000 7fa2b6ffd700 1 -- 172.21.15.40:0/3132576956 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa2c0026070 con 0x7fa2c80c6010 2024-09-13T17:54:26.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.721+0000 7fa2c659c700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa2a80663e0 0x7fa2a80688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:26.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.721+0000 7fa2c659c700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa2a80663e0 0x7fa2a80688a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80039e0 tx=0x7fa2b8011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:26.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.733+0000 7fa2c759e700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2ac006fd0 0x7fa2ac007420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.733+0000 7fa2c759e700 1 -- 172.21.15.40:0/3132576956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac00a400 con 0x7fa2ac006fd0 2024-09-13T17:54:26.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.733+0000 7fa2c5d9b700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2ac006fd0 0x7fa2ac007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:26.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.733+0000 7fa2c5d9b700 1 -- 172.21.15.40:0/3132576956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa2ac00a5c0 con 0x7fa2ac006fd0 2024-09-13T17:54:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.733+0000 7fa2c5d9b700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2ac006fd0 0x7fa2ac007420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0039e0 tx=0x7fa2bc00b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.733+0000 7fa2b6ffd700 1 -- 172.21.15.40:0/3132576956 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc00de10 con 0x7fa2ac006fd0 2024-09-13T17:54:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.733+0000 7fa2b6ffd700 1 -- 172.21.15.40:0/3132576956 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4938 (secure 0 0 0) 0x7fa2bc003a60 con 0x7fa2ac006fd0 2024-09-13T17:54:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.733+0000 7fa2b6ffd700 1 -- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2ac006fd0 msgr2=0x7fa2ac007420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.733+0000 7fa2b6ffd700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2ac006fd0 0x7fa2ac007420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0039e0 tx=0x7fa2bc00b5f0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.733+0000 7fa2b6ffd700 1 -- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2ac006fd0 msgr2=0x7fa2ac007420 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:54:26.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.733+0000 7fa2b6ffd700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2ac006fd0 0x7fa2ac007420 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0039e0 tx=0x7fa2bc00b5f0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.740 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:54:26.740 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:54:26.740 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:54:26.740 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:54:26.740 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 38, 2024-09-13T17:54:26.740 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:54:26.740 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:54:26.740 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:54:26.740 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:26.740 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T17:54:26.741 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:54:26.741 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:54:26.741 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:54:26.741 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:26.741 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:26.741 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:26.741 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:26.741 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:26.741 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:26.741 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:26.741 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:26.741 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:26.741 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:26.741 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:26.741 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:54:26.742 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:54:26.742 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:26.742 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:26.742 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:26.742 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:26.742 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:26.742 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:26.742 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:26.742 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:26.742 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:26.742 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:26.742 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:26.742 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.742 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:54:26.742 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:54:26.743 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:54:26.743 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:54:26.743 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:54:26.743 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:54:26.743 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:54:26.743 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:54:26.743 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:54:26.743 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:54:26.743 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:54:26.743 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:54:26.743 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:54:26.743 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:54:26.743 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:54:26.743 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:54:26.743 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:54:26.744 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:26.744 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:26.744 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:26.744 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:26.744 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:26.744 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:26.744 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:26.744 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:26.744 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:26.744 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:26.744 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:26.744 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:54:26.744 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.744 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:54:26.745 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.745 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:54:26.745 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:54:26.745 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:26.745 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:26.745 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.745 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:26.745 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:54:26.745 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:26.748 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.748 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.748 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:26.748 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:54:26.748 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:26.748 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:26.748 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:26.748 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.749 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:54:26.749 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:54:26.749 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:26.749 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:26.749 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:26.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.749 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.749 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:54:26.749 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:54:26.749 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:26.749 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:26.749 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.749 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:26.749 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:54:26.750 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:26.750 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.750 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.750 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:26.750 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:54:26.750 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:26.750 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:26.750 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:26.750 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.750 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:54:26.750 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:54:26.750 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:26.750 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:26.750 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:26.750 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.751 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.751 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:54:26.751 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:54:26.751 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:26.751 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:26.751 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.751 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:26.751 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:54:26.751 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:26.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.751 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.751 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:26.751 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:54:26.751 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:26.752 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:26.752 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:26.752 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.752 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:54:26.752 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:54:26.752 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:26.752 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:26.752 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:26.752 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:26.752 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:26.752 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.752 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:54:26.752 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:54:26.752 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.752 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:26.753 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:26.753 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:54:26.753 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:26.753 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:26.753 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:54:26.753 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.753 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:26.753 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:26.753 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:54:26.753 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:26.753 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:26.753 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:54:26.753 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.753 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:26.754 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:26.754 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:54:26.754 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:26.754 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:26.754 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:54:26.754 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.754 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:26.754 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:26.754 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:54:26.754 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:26.754 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:26.754 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:54:26.754 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.754 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:26.755 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:26.755 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:54:26.755 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:26.755 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:26.755 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.755 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:54:26.755 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:54:26.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.737+0000 7fa2c759e700 1 -- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa2a80663e0 msgr2=0x7fa2a80688a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:26.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.737+0000 7fa2c759e700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa2a80663e0 0x7fa2a80688a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80039e0 tx=0x7fa2b8011000 comp rx=0 tx=0).stop 2024-09-13T17:54:26.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.737+0000 7fa2c759e700 1 -- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c80c6010 msgr2=0x7fa2c80c66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:26.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.737+0000 7fa2c6d9d700 1 -- 172.21.15.40:0/3132576956 reap_dead start 2024-09-13T17:54:26.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.737+0000 7fa2c759e700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c80c6010 0x7fa2c80c66e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa2c81534f0 tx=0x7fa2c81558c0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.737+0000 7fa2c759e700 1 -- 172.21.15.40:0/3132576956 shutdown_connections 2024-09-13T17:54:26.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.737+0000 7fa2c759e700 1 --2- 172.21.15.40:0/3132576956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2c80c6010 0x7fa2c80c66e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.737+0000 7fa2c759e700 1 -- 172.21.15.40:0/3132576956 >> 172.21.15.40:0/3132576956 conn(0x7fa2c806dcf0 msgr2=0x7fa2c81548d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:26.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.737+0000 7fa2c759e700 1 -- 172.21.15.40:0/3132576956 shutdown_connections 2024-09-13T17:54:26.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.737+0000 7fa2c759e700 1 -- 172.21.15.40:0/3132576956 wait complete. 2024-09-13T17:54:26.757 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T17:54:26.757 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T17:54:26.758 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:54:26.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f8855ca2700 1 Processor -- start 2024-09-13T17:54:26.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f8855ca2700 1 -- start start 2024-09-13T17:54:26.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f8855ca2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f884804ca90 0x7f88480df120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f8855ca2700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8848046f00 0x7f8848006880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f8855ca2700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8848045480 con 0x7f88480d8b00 2024-09-13T17:54:26.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f8855ca2700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88480456b0 con 0x7f8848046f00 2024-09-13T17:54:26.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f8855ca2700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8848045950 con 0x7f884804ca90 2024-09-13T17:54:26.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f8854ca0700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7f88480d8b00 legacy=0x7f884804af60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T17:54:26.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f88554a1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8848046f00 0x7f8848006880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:26.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f88554a1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8848046f00 0x7f8848006880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38460/0 (socket says 172.21.15.40:38460) 2024-09-13T17:54:26.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f88554a1700 1 -- 172.21.15.40:0/2400510372 learned_addr learned my addr 172.21.15.40:0/2400510372 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:26.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f884ffff700 1 --2- 172.21.15.40:0/2400510372 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f884804ca90 0x7f88480df120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:26.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f88554a1700 1 -- 172.21.15.40:0/2400510372 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f884804ca90 msgr2=0x7f88480df120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:26.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f88554a1700 1 --2- 172.21.15.40:0/2400510372 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f884804ca90 0x7f88480df120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f88554a1700 1 -- 172.21.15.40:0/2400510372 >> v1:172.21.15.43:6789/0 conn(0x7f88480d8b00 legacy=0x7f884804af60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:54:26.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f88554a1700 1 -- 172.21.15.40:0/2400510372 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8848046140 con 0x7f8848046f00 2024-09-13T17:54:26.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f884ffff700 1 --2- 172.21.15.40:0/2400510372 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f884804ca90 0x7f88480df120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:54:26.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f88554a1700 1 --2- 172.21.15.40:0/2400510372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8848046f00 0x7f8848006880 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8850063990 tx=0x7f8850069e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9e93a79568c852 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:26.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f884f7fe700 1 -- 172.21.15.40:0/2400510372 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850067dd0 con 0x7f8848046f00 2024-09-13T17:54:26.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f884f7fe700 1 -- 172.21.15.40:0/2400510372 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8850067f30 con 0x7f8848046f00 2024-09-13T17:54:26.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.885+0000 7f884f7fe700 1 -- 172.21.15.40:0/2400510372 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8850074720 con 0x7f8848046f00 2024-09-13T17:54:26.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.889+0000 7f8855ca2700 1 -- 172.21.15.40:0/2400510372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8848046f00 msgr2=0x7f8848006880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:26.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.889+0000 7f8855ca2700 1 --2- 172.21.15.40:0/2400510372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8848046f00 0x7f8848006880 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8850063990 tx=0x7f8850069e70 comp rx=0 tx=0).stop 2024-09-13T17:54:26.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.889+0000 7f8855ca2700 1 -- 172.21.15.40:0/2400510372 shutdown_connections 2024-09-13T17:54:26.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.889+0000 7f8855ca2700 1 --2- 172.21.15.40:0/2400510372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8848046f00 0x7f8848006880 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.889+0000 7f8855ca2700 1 --2- 172.21.15.40:0/2400510372 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f884804ca90 0x7f88480df120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.889+0000 7f8855ca2700 1 -- 172.21.15.40:0/2400510372 >> 172.21.15.40:0/2400510372 conn(0x7f88480505e0 msgr2=0x7f88480509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:26.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.889+0000 7f8855ca2700 1 -- 172.21.15.40:0/2400510372 shutdown_connections 2024-09-13T17:54:26.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.889+0000 7f8855ca2700 1 -- 172.21.15.40:0/2400510372 wait complete. 2024-09-13T17:54:26.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.889+0000 7f8855ca2700 1 Processor -- start 2024-09-13T17:54:26.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.889+0000 7f8855ca2700 1 -- start start 2024-09-13T17:54:26.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f8855ca2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8848046f00 0x7f8848167e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f8855ca2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f884804ca90 0x7f88480d2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f8855ca2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88480d8b00 0x7f88480dc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f8855ca2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8848034ee0 con 0x7f884804ca90 2024-09-13T17:54:26.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f8855ca2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8848035e40 con 0x7f8848046f00 2024-09-13T17:54:26.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f8855ca2700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8848035910 con 0x7f88480d8b00 2024-09-13T17:54:26.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f8854ca0700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8848046f00 msgr2=0x7f8848167e40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:54:26.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f884ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f884804ca90 0x7f88480d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:26.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f8854ca0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8848046f00 0x7f8848167e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:54:26.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f884ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f884804ca90 0x7f88480d2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38464/0 (socket says 172.21.15.40:38464) 2024-09-13T17:54:26.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f884ffff700 1 -- 172.21.15.40:0/3867109330 learned_addr learned my addr 172.21.15.40:0/3867109330 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:26.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f88554a1700 1 --2- 172.21.15.40:0/3867109330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88480d8b00 0x7f88480dc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:26.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f884ffff700 1 -- 172.21.15.40:0/3867109330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88480d8b00 msgr2=0x7f88480dc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:26.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f884ffff700 1 --2- 172.21.15.40:0/3867109330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88480d8b00 0x7f88480dc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f884ffff700 1 -- 172.21.15.40:0/3867109330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8848046f00 msgr2=0x7f8848167e40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:54:26.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f884ffff700 1 --2- 172.21.15.40:0/3867109330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8848046f00 0x7f8848167e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f884ffff700 1 -- 172.21.15.40:0/3867109330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885004f5c0 con 0x7f884804ca90 2024-09-13T17:54:26.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f884ffff700 1 --2- 172.21.15.40:0/3867109330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f884804ca90 0x7f88480d2160 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f883c00e760 tx=0x7f883c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:26.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f884d7fa700 1 -- 172.21.15.40:0/3867109330 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c004230 con 0x7f884804ca90 2024-09-13T17:54:26.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f884d7fa700 1 -- 172.21.15.40:0/3867109330 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883c0043b0 con 0x7f884804ca90 2024-09-13T17:54:26.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f8855ca2700 1 -- 172.21.15.40:0/3867109330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8848035b60 con 0x7f884804ca90 2024-09-13T17:54:26.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f884d7fa700 1 -- 172.21.15.40:0/3867109330 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c0036c0 con 0x7f884804ca90 2024-09-13T17:54:26.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f8855ca2700 1 -- 172.21.15.40:0/3867109330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8848034820 con 0x7f884804ca90 2024-09-13T17:54:26.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f8855ca2700 1 --2- 172.21.15.40:0/3867109330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88480dca50 0x7f88480d6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f8855ca2700 1 -- 172.21.15.40:0/3867109330 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8848008d50 con 0x7f88480dca50 2024-09-13T17:54:26.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f8854ca0700 1 --2- 172.21.15.40:0/3867109330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88480dca50 0x7f88480d6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:26.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.893+0000 7f884d7fa700 1 -- 172.21.15.40:0/3867109330 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f883c003820 con 0x7f884804ca90 2024-09-13T17:54:26.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.897+0000 7f8854ca0700 1 -- 172.21.15.40:0/3867109330 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88480469e0 con 0x7f88480dca50 2024-09-13T17:54:26.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.897+0000 7f8854ca0700 1 --2- 172.21.15.40:0/3867109330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88480dca50 0x7f88480d6320 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f884400b470 tx=0x7f884400b830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:26.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.897+0000 7f884d7fa700 1 --2- 172.21.15.40:0/3867109330 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f88340666e0 0x7f8834068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.897+0000 7f88554a1700 1 --2- 172.21.15.40:0/3867109330 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f88340666e0 0x7f8834068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:26.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.897+0000 7f884d7fa700 1 -- 172.21.15.40:0/3867109330 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f883c083f90 con 0x7f884804ca90 2024-09-13T17:54:26.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.897+0000 7f884d7fa700 1 -- 172.21.15.40:0/3867109330 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844005bb0 con 0x7f88480dca50 2024-09-13T17:54:26.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.897+0000 7f88554a1700 1 --2- 172.21.15.40:0/3867109330 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f88340666e0 0x7f8834068ba0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f884800a050 tx=0x7f885006d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:26.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.897+0000 7f884d7fa700 1 -- 172.21.15.40:0/3867109330 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8844005d10 con 0x7f88480dca50 2024-09-13T17:54:26.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.897+0000 7f884d7fa700 1 -- 172.21.15.40:0/3867109330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88480dca50 msgr2=0x7f88480d6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:26.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.897+0000 7f884d7fa700 1 --2- 172.21.15.40:0/3867109330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88480dca50 0x7f88480d6320 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f884400b470 tx=0x7f884400b830 comp rx=0 tx=0).stop 2024-09-13T17:54:26.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.897+0000 7f884d7fa700 1 -- 172.21.15.40:0/3867109330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88480dca50 msgr2=0x7f88480d6320 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:54:26.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.897+0000 7f884d7fa700 1 --2- 172.21.15.40:0/3867109330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88480dca50 0x7f88480d6320 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.905+0000 7f8855ca2700 1 --2- 172.21.15.40:0/3867109330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8848033d90 0x7f88480d6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:26.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.905+0000 7f8855ca2700 1 -- 172.21.15.40:0/3867109330 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8848046700 con 0x7f8848033d90 2024-09-13T17:54:26.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.905+0000 7f8854ca0700 1 --2- 172.21.15.40:0/3867109330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8848033d90 0x7f88480d6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:26.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.905+0000 7f8854ca0700 1 -- 172.21.15.40:0/3867109330 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8848036600 con 0x7f8848033d90 2024-09-13T17:54:26.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.905+0000 7f8854ca0700 1 --2- 172.21.15.40:0/3867109330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8848033d90 0x7f88480d6860 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f884400b470 tx=0x7f8844005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:26.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.905+0000 7f884d7fa700 1 -- 172.21.15.40:0/3867109330 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884401f070 con 0x7f8848033d90 2024-09-13T17:54:26.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.909+0000 7f884d7fa700 1 -- 172.21.15.40:0/3867109330 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4770 (secure 0 0 0) 0x7f884401a320 con 0x7f8848033d90 2024-09-13T17:54:26.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.909+0000 7f884d7fa700 1 -- 172.21.15.40:0/3867109330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8848033d90 msgr2=0x7f88480d6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:26.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.909+0000 7f884d7fa700 1 --2- 172.21.15.40:0/3867109330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8848033d90 0x7f88480d6860 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f884400b470 tx=0x7f8844005ad0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.909+0000 7f884d7fa700 1 -- 172.21.15.40:0/3867109330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8848033d90 msgr2=0x7f88480d6860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:54:26.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.909+0000 7f884d7fa700 1 --2- 172.21.15.40:0/3867109330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8848033d90 0x7f88480d6860 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f884400b470 tx=0x7f8844005ad0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.912 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:54:26.912 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:54:26.912 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:54:26.912 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:54:26.912 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 38, 2024-09-13T17:54:26.913 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:54:26.913 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:54:26.913 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:54:26.913 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:26.913 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T17:54:26.913 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:54:26.913 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:54:26.913 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:54:26.913 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:26.913 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:26.913 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:26.913 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:26.913 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:26.914 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:26.914 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:26.914 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:26.914 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:26.914 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:26.914 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:26.914 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:54:26.914 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:54:26.915 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:26.915 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:26.915 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:26.915 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:26.915 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:26.915 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:26.915 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:26.915 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:26.915 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:26.915 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:26.915 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:26.916 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.916 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:54:26.916 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:54:26.916 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:54:26.916 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:54:26.916 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:54:26.916 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:54:26.916 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:54:26.916 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:54:26.916 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:54:26.916 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:54:26.916 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:54:26.916 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:54:26.917 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:54:26.917 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:54:26.917 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:54:26.917 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:54:26.917 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:54:26.917 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:26.917 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:26.917 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:26.917 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:26.917 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:26.917 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:26.917 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:26.917 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:26.917 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:26.918 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:26.918 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:26.918 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:54:26.918 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.918 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:54:26.918 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.918 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:54:26.918 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:54:26.918 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:26.918 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:26.918 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.918 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:26.918 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:54:26.918 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:26.919 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.919 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.919 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:26.919 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:54:26.919 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:26.919 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:26.919 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:26.919 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.919 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:54:26.919 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:54:26.919 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:26.919 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:26.919 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:26.919 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.920 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.920 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:54:26.920 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:54:26.920 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:26.920 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:26.920 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.920 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:26.920 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:54:26.920 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:26.920 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.920 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.920 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:26.920 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:54:26.920 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:26.921 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:26.921 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:26.921 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.921 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:54:26.921 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:54:26.921 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:26.921 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:26.921 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:26.921 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.921 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.921 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:54:26.921 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:54:26.921 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:26.921 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:26.922 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.922 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:26.922 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:54:26.922 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:26.922 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.922 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.922 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:26.922 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:54:26.923 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:26.923 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:26.923 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:26.923 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.923 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:54:26.923 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:54:26.923 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:26.923 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:26.923 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:26.923 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:26.923 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:26.923 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.924 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:54:26.924 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:54:26.924 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.924 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:26.924 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:26.924 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:54:26.924 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:26.924 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:26.924 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:54:26.924 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.924 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:26.924 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:26.924 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:54:26.924 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:26.925 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:26.925 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:54:26.925 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.925 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:26.925 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:26.925 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:54:26.925 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:26.925 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:26.925 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:54:26.925 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:26.925 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:26.925 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:26.925 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:54:26.926 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:26.926 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:26.926 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:26.926 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:54:26.926 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:54:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.909+0000 7f8855ca2700 1 -- 172.21.15.40:0/3867109330 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f88340666e0 msgr2=0x7f8834068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.909+0000 7f8855ca2700 1 --2- 172.21.15.40:0/3867109330 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f88340666e0 0x7f8834068ba0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f884800a050 tx=0x7f885006d040 comp rx=0 tx=0).stop 2024-09-13T17:54:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.909+0000 7f8855ca2700 1 -- 172.21.15.40:0/3867109330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f884804ca90 msgr2=0x7f88480d2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.909+0000 7f8855ca2700 1 --2- 172.21.15.40:0/3867109330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f884804ca90 0x7f88480d2160 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f883c00e760 tx=0x7f883c00eb20 comp rx=0 tx=0).stop 2024-09-13T17:54:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.909+0000 7f88554a1700 1 -- 172.21.15.40:0/3867109330 reap_dead start 2024-09-13T17:54:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.909+0000 7f8855ca2700 1 -- 172.21.15.40:0/3867109330 shutdown_connections 2024-09-13T17:54:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.909+0000 7f8855ca2700 1 --2- 172.21.15.40:0/3867109330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f884804ca90 0x7f88480d2160 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.909+0000 7f8855ca2700 1 -- 172.21.15.40:0/3867109330 >> 172.21.15.40:0/3867109330 conn(0x7f88480505e0 msgr2=0x7f88480d73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:26.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.909+0000 7f8855ca2700 1 -- 172.21.15.40:0/3867109330 shutdown_connections 2024-09-13T17:54:26.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:26.909+0000 7f8855ca2700 1 -- 172.21.15.40:0/3867109330 wait complete. 2024-09-13T17:54:26.929 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T17:54:26.930 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T17:54:34.223 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:54:34.226 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:54:46.931 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-13T17:54:46.931 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-13T17:54:46.931 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T17:54:46.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T17:54:46.935 INFO:tasks.ceph.mon.b:Started 2024-09-13T17:54:46.935 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T17:54:46.935 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:54:47.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.052+0000 7f587f59e700 1 Processor -- start 2024-09-13T17:54:47.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.052+0000 7f587f59e700 1 -- start start 2024-09-13T17:54:47.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.052+0000 7f587f59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5880072240 0x7f58800c0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:47.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.052+0000 7f587f59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58800c28f0 0x7f58801565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:47.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.052+0000 7f587f59e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58800b2210 con 0x7f5880071e60 2024-09-13T17:54:47.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.052+0000 7f587f59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58800c1780 con 0x7f58800c28f0 2024-09-13T17:54:47.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.052+0000 7f587f59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58800c1a60 con 0x7f5880072240 2024-09-13T17:54:47.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.052+0000 7f587ed9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58800c28f0 0x7f58801565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:47.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.052+0000 7f587e59c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5880071e60 0x7f58800c32e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:51960/0 (socket says 172.21.15.40:51960) 2024-09-13T17:54:47.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.052+0000 7f587e59c700 1 -- 172.21.15.40:0/556751993 learned_addr learned my addr 172.21.15.40:0/556751993 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:54:47.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.052+0000 7f587dd9b700 1 --2- 172.21.15.40:0/556751993 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5880072240 0x7f58800c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:47.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.056+0000 7f587ed9d700 1 -- 172.21.15.40:0/556751993 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5880072240 msgr2=0x7f58800c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:47.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.056+0000 7f587ed9d700 1 --2- 172.21.15.40:0/556751993 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5880072240 0x7f58800c0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:47.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.056+0000 7f587ed9d700 1 -- 172.21.15.40:0/556751993 >> v1:172.21.15.43:6789/0 conn(0x7f5880071e60 legacy=0x7f58800c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:47.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.056+0000 7f587ed9d700 1 -- 172.21.15.40:0/556751993 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58800c5d70 con 0x7f58800c28f0 2024-09-13T17:54:47.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.056+0000 7f587ed9d700 1 --2- 172.21.15.40:0/556751993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58800c28f0 0x7f58801565c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5878009d90 tx=0x7f587800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f76cdf283bc5a2d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:47.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.056+0000 7f587d59a700 1 -- 172.21.15.40:0/556751993 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587800e070 con 0x7f58800c28f0 2024-09-13T17:54:47.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.056+0000 7f587d59a700 1 -- 172.21.15.40:0/556751993 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587800bb20 con 0x7f58800c28f0 2024-09-13T17:54:47.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.056+0000 7f587d59a700 1 -- 172.21.15.40:0/556751993 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878012430 con 0x7f58800c28f0 2024-09-13T17:54:47.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.056+0000 7f587f59e700 1 -- 172.21.15.40:0/556751993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58800c28f0 msgr2=0x7f58801565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:47.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.056+0000 7f587f59e700 1 --2- 172.21.15.40:0/556751993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58800c28f0 0x7f58801565c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5878009d90 tx=0x7f587800b330 comp rx=0 tx=0).stop 2024-09-13T17:54:47.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.056+0000 7f587f59e700 1 -- 172.21.15.40:0/556751993 shutdown_connections 2024-09-13T17:54:47.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.056+0000 7f587f59e700 1 --2- 172.21.15.40:0/556751993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58800c28f0 0x7f58801565c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:47.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.056+0000 7f587f59e700 1 --2- 172.21.15.40:0/556751993 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5880072240 0x7f58800c0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:47.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.056+0000 7f587f59e700 1 -- 172.21.15.40:0/556751993 >> 172.21.15.40:0/556751993 conn(0x7f588006e4f0 msgr2=0x7f588006e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:47.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.056+0000 7f587f59e700 1 -- 172.21.15.40:0/556751993 shutdown_connections 2024-09-13T17:54:47.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.056+0000 7f587f59e700 1 -- 172.21.15.40:0/556751993 wait complete. 2024-09-13T17:54:47.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.056+0000 7f587f59e700 1 Processor -- start 2024-09-13T17:54:47.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.056+0000 7f587f59e700 1 -- start start 2024-09-13T17:54:47.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587f59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5880071e60 0x7f58801f04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:47.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587f59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5880072240 0x7f58800c7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:47.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587f59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f58800c28f0 0x7f58801efab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:47.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587f59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58801efff0 con 0x7f5880071e60 2024-09-13T17:54:47.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587f59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58800c6ba0 con 0x7f5880072240 2024-09-13T17:54:47.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587f59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58800c7ce0 con 0x7f58800c28f0 2024-09-13T17:54:47.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5880071e60 0x7f58801f04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:47.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5880071e60 0x7f58801f04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53190/0 (socket says 172.21.15.40:53190) 2024-09-13T17:54:47.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587e59c700 1 -- 172.21.15.40:0/3215725055 learned_addr learned my addr 172.21.15.40:0/3215725055 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:47.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587dd9b700 1 --2- 172.21.15.40:0/3215725055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5880072240 0x7f58800c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:47.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587ed9d700 1 --2- 172.21.15.40:0/3215725055 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f58800c28f0 0x7f58801efab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:47.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587dd9b700 1 -- 172.21.15.40:0/3215725055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5880072240 msgr2=0x7f58800c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T17:54:47.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587dd9b700 1 -- 172.21.15.40:0/3215725055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5880072240 msgr2=0x7f58800c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:54:47.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587dd9b700 1 --2- 172.21.15.40:0/3215725055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5880072240 0x7f58800c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:54:47.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587dd9b700 1 --2- 172.21.15.40:0/3215725055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5880072240 0x7f58800c7030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:54:47.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587e59c700 1 -- 172.21.15.40:0/3215725055 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f58800c28f0 msgr2=0x7f58801efab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:47.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587e59c700 1 --2- 172.21.15.40:0/3215725055 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f58800c28f0 0x7f58801efab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:47.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587e59c700 1 -- 172.21.15.40:0/3215725055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5880072240 msgr2=0x7f58800c7030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:54:47.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587e59c700 1 --2- 172.21.15.40:0/3215725055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5880072240 0x7f58800c7030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:47.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587e59c700 1 -- 172.21.15.40:0/3215725055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5878009a70 con 0x7f5880071e60 2024-09-13T17:54:47.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587e59c700 1 --2- 172.21.15.40:0/3215725055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5880071e60 0x7f58801f04a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5870001eb0 tx=0x7f587000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:47.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f586effd700 1 -- 172.21.15.40:0/3215725055 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870005d90 con 0x7f5880071e60 2024-09-13T17:54:47.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f586effd700 1 -- 172.21.15.40:0/3215725055 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58700070d0 con 0x7f5880071e60 2024-09-13T17:54:47.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f586effd700 1 -- 172.21.15.40:0/3215725055 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870007440 con 0x7f5880071e60 2024-09-13T17:54:47.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587f59e700 1 -- 172.21.15.40:0/3215725055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58800c7fc0 con 0x7f5880071e60 2024-09-13T17:54:47.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.060+0000 7f587f59e700 1 -- 172.21.15.40:0/3215725055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58800c6550 con 0x7f5880071e60 2024-09-13T17:54:47.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.064+0000 7f587f59e700 1 -- 172.21.15.40:0/3215725055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58800c0e90 con 0x7f5880071e60 2024-09-13T17:54:47.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.064+0000 7f586effd700 1 -- 172.21.15.40:0/3215725055 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f58700075a0 con 0x7f5880071e60 2024-09-13T17:54:47.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.064+0000 7f586effd700 1 --2- 172.21.15.40:0/3215725055 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5860066760 0x7f5860068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:47.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.064+0000 7f586effd700 1 -- 172.21.15.40:0/3215725055 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f58700838a0 con 0x7f5880071e60 2024-09-13T17:54:47.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.064+0000 7f587dd9b700 1 --2- 172.21.15.40:0/3215725055 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5860066760 0x7f5860068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:47.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.064+0000 7f587dd9b700 1 --2- 172.21.15.40:0/3215725055 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5860066760 0x7f5860068c20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f588014ead0 tx=0x7f5874008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:47.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.072+0000 7f586effd700 1 -- 172.21.15.40:0/3215725055 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5870050870 con 0x7f5880071e60 2024-09-13T17:54:47.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.332+0000 7f587f59e700 1 -- 172.21.15.40:0/3215725055 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5880152210 con 0x7f5880071e60 2024-09-13T17:54:47.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.332+0000 7f586effd700 1 -- 172.21.15.40:0/3215725055 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f5870017080 con 0x7f5880071e60 2024-09-13T17:54:47.337 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":38,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:54:47.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.332+0000 7f587f59e700 1 -- 172.21.15.40:0/3215725055 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5860066760 msgr2=0x7f5860068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:47.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.332+0000 7f587f59e700 1 --2- 172.21.15.40:0/3215725055 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5860066760 0x7f5860068c20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f588014ead0 tx=0x7f5874008040 comp rx=0 tx=0).stop 2024-09-13T17:54:47.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.332+0000 7f587f59e700 1 -- 172.21.15.40:0/3215725055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5880071e60 msgr2=0x7f58801f04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:47.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.332+0000 7f587f59e700 1 --2- 172.21.15.40:0/3215725055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5880071e60 0x7f58801f04a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5870001eb0 tx=0x7f587000c7b0 comp rx=0 tx=0).stop 2024-09-13T17:54:47.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.332+0000 7f587f59e700 1 -- 172.21.15.40:0/3215725055 shutdown_connections 2024-09-13T17:54:47.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.332+0000 7f587f59e700 1 --2- 172.21.15.40:0/3215725055 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5860066760 0x7f5860068c20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:47.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.332+0000 7f587f59e700 1 --2- 172.21.15.40:0/3215725055 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f58800c28f0 0x7f58801efab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:47.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.332+0000 7f587f59e700 1 --2- 172.21.15.40:0/3215725055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5880072240 0x7f58800c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:47.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.332+0000 7f587f59e700 1 --2- 172.21.15.40:0/3215725055 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5880071e60 0x7f58801f04a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:47.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.332+0000 7f587f59e700 1 -- 172.21.15.40:0/3215725055 >> 172.21.15.40:0/3215725055 conn(0x7f588006e4f0 msgr2=0x7f5880158c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:47.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.332+0000 7f587f59e700 1 -- 172.21.15.40:0/3215725055 shutdown_connections 2024-09-13T17:54:47.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:47.332+0000 7f587f59e700 1 -- 172.21.15.40:0/3215725055 wait complete. 2024-09-13T17:54:50.353 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:54:50.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.460+0000 7f220a8c3700 1 Processor -- start 2024-09-13T17:54:50.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.460+0000 7f220a8c3700 1 -- start start 2024-09-13T17:54:50.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.460+0000 7f220a8c3700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21fc04e490 0x7f21fc0361a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:50.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.460+0000 7f220a8c3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21fc0366e0 0x7f21fc00daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:50.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f220a8c3700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21fc0e0c60 con 0x7f21fc0d68b0 2024-09-13T17:54:50.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f220a8c3700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21fc0dfcc0 con 0x7f21fc04e490 2024-09-13T17:54:50.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f220a8c3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc04d370 con 0x7f21fc0366e0 2024-09-13T17:54:50.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f22090c0700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21fc04e490 0x7f21fc0361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:50.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f22090c0700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21fc04e490 0x7f21fc0361a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53202/0 (socket says 172.21.15.40:53202) 2024-09-13T17:54:50.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f22090c0700 1 -- 172.21.15.40:0/670333125 learned_addr learned my addr 172.21.15.40:0/670333125 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:50.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f220a0c2700 1 --2- 172.21.15.40:0/670333125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21fc0366e0 0x7f21fc00daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:50.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f22090c0700 1 -- 172.21.15.40:0/670333125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21fc0366e0 msgr2=0x7f21fc00daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:50.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f22090c0700 1 --2- 172.21.15.40:0/670333125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21fc0366e0 0x7f21fc00daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:50.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f22090c0700 1 -- 172.21.15.40:0/670333125 >> v1:172.21.15.40:6789/0 conn(0x7f21fc0d68b0 legacy=0x7f21fc0dd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:50.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f22090c0700 1 -- 172.21.15.40:0/670333125 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21fc0dfea0 con 0x7f21fc04e490 2024-09-13T17:54:50.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f220a0c2700 1 --2- 172.21.15.40:0/670333125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21fc0366e0 0x7f21fc00daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:54:50.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f22090c0700 1 --2- 172.21.15.40:0/670333125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc04e490 0x7f21fc0361a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009910 tx=0x7f21f4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449c91437398d4fd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:50.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f22088bf700 1 -- 172.21.15.40:0/670333125 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f400ba60 con 0x7f21fc04e490 2024-09-13T17:54:50.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f22088bf700 1 -- 172.21.15.40:0/670333125 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f400bbc0 con 0x7f21fc04e490 2024-09-13T17:54:50.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f22088bf700 1 -- 172.21.15.40:0/670333125 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4009e60 con 0x7f21fc04e490 2024-09-13T17:54:50.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f220a8c3700 1 -- 172.21.15.40:0/670333125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc04e490 msgr2=0x7f21fc0361a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:50.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f220a8c3700 1 --2- 172.21.15.40:0/670333125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc04e490 0x7f21fc0361a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009910 tx=0x7f21f4009c20 comp rx=0 tx=0).stop 2024-09-13T17:54:50.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f220a8c3700 1 -- 172.21.15.40:0/670333125 shutdown_connections 2024-09-13T17:54:50.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f220a8c3700 1 --2- 172.21.15.40:0/670333125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21fc0366e0 0x7f21fc00daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:50.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f220a8c3700 1 --2- 172.21.15.40:0/670333125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc04e490 0x7f21fc0361a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:50.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f220a8c3700 1 -- 172.21.15.40:0/670333125 >> 172.21.15.40:0/670333125 conn(0x7f21fc009a80 msgr2=0x7f21fc009e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:50.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f220a8c3700 1 -- 172.21.15.40:0/670333125 shutdown_connections 2024-09-13T17:54:50.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f220a8c3700 1 -- 172.21.15.40:0/670333125 wait complete. 2024-09-13T17:54:50.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.464+0000 7f220a8c3700 1 Processor -- start 2024-09-13T17:54:50.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f220a8c3700 1 -- start start 2024-09-13T17:54:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f220a8c3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21fc0366e0 0x7f21fc0db5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f220a8c3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21fc04e490 0x7f21fc0dee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f220a8c3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc0d68b0 0x7f21fc0df390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f220a8c3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc052900 con 0x7f21fc0d68b0 2024-09-13T17:54:50.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f220a8c3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc0cc350 con 0x7f21fc04e490 2024-09-13T17:54:50.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f220a8c3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc052620 con 0x7f21fc0366e0 2024-09-13T17:54:50.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f220a0c2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc0d68b0 0x7f21fc0df390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:50.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f220a0c2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc0d68b0 0x7f21fc0df390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53214/0 (socket says 172.21.15.40:53214) 2024-09-13T17:54:50.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f220a0c2700 1 -- 172.21.15.40:0/724937367 learned_addr learned my addr 172.21.15.40:0/724937367 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:50.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f22090c0700 1 --2- 172.21.15.40:0/724937367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21fc04e490 0x7f21fc0dee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:50.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f22098c1700 1 --2- 172.21.15.40:0/724937367 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21fc0366e0 0x7f21fc0db5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:50.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f220a0c2700 1 -- 172.21.15.40:0/724937367 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21fc0366e0 msgr2=0x7f21fc0db5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:50.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f220a0c2700 1 --2- 172.21.15.40:0/724937367 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21fc0366e0 0x7f21fc0db5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:50.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f220a0c2700 1 -- 172.21.15.40:0/724937367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21fc04e490 msgr2=0x7f21fc0dee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:50.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f220a0c2700 1 --2- 172.21.15.40:0/724937367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21fc04e490 0x7f21fc0dee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:50.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f220a0c2700 1 -- 172.21.15.40:0/724937367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f4009570 con 0x7f21fc0d68b0 2024-09-13T17:54:50.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f220a0c2700 1 --2- 172.21.15.40:0/724937367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc0d68b0 0x7f21fc0df390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2204072e80 tx=0x7f22040680b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:50.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f21fa7fc700 1 -- 172.21.15.40:0/724937367 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2204067930 con 0x7f21fc0d68b0 2024-09-13T17:54:50.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f21fa7fc700 1 -- 172.21.15.40:0/724937367 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2204072050 con 0x7f21fc0d68b0 2024-09-13T17:54:50.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f21fa7fc700 1 -- 172.21.15.40:0/724937367 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22040723e0 con 0x7f21fc0d68b0 2024-09-13T17:54:50.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f220a8c3700 1 -- 172.21.15.40:0/724937367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21fc0cc0b0 con 0x7f21fc0d68b0 2024-09-13T17:54:50.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.468+0000 7f220a8c3700 1 -- 172.21.15.40:0/724937367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21fc0cba90 con 0x7f21fc0d68b0 2024-09-13T17:54:50.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.472+0000 7f220a8c3700 1 -- 172.21.15.40:0/724937367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21fc052a90 con 0x7f21fc0d68b0 2024-09-13T17:54:50.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.472+0000 7f21fa7fc700 1 -- 172.21.15.40:0/724937367 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2204072540 con 0x7f21fc0d68b0 2024-09-13T17:54:50.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.472+0000 7f21fa7fc700 1 --2- 172.21.15.40:0/724937367 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f21e8066730 0x7f21e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:50.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.472+0000 7f21fa7fc700 1 -- 172.21.15.40:0/724937367 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f22040ff050 con 0x7f21fc0d68b0 2024-09-13T17:54:50.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.472+0000 7f22098c1700 1 --2- 172.21.15.40:0/724937367 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f21e8066730 0x7f21e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:50.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.476+0000 7f22098c1700 1 --2- 172.21.15.40:0/724937367 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f21e8066730 0x7f21e8068bf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2200000f60 tx=0x7f2200009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:50.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.480+0000 7f21fa7fc700 1 -- 172.21.15.40:0/724937367 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f22040cbf80 con 0x7f21fc0d68b0 2024-09-13T17:54:50.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.724+0000 7f220a8c3700 1 -- 172.21.15.40:0/724937367 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f21fc0d6c90 con 0x7f21fc0d68b0 2024-09-13T17:54:50.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.724+0000 7f21fa7fc700 1 -- 172.21.15.40:0/724937367 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f22040cf5a0 con 0x7f21fc0d68b0 2024-09-13T17:54:50.731 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":38,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:54:50.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.728+0000 7f220a8c3700 1 -- 172.21.15.40:0/724937367 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f21e8066730 msgr2=0x7f21e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:50.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.728+0000 7f220a8c3700 1 --2- 172.21.15.40:0/724937367 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f21e8066730 0x7f21e8068bf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2200000f60 tx=0x7f2200009040 comp rx=0 tx=0).stop 2024-09-13T17:54:50.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.728+0000 7f220a8c3700 1 -- 172.21.15.40:0/724937367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc0d68b0 msgr2=0x7f21fc0df390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:50.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.728+0000 7f220a8c3700 1 --2- 172.21.15.40:0/724937367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc0d68b0 0x7f21fc0df390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2204072e80 tx=0x7f22040680b0 comp rx=0 tx=0).stop 2024-09-13T17:54:50.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.728+0000 7f220a8c3700 1 -- 172.21.15.40:0/724937367 shutdown_connections 2024-09-13T17:54:50.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.728+0000 7f220a8c3700 1 --2- 172.21.15.40:0/724937367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21fc0d68b0 0x7f21fc0df390 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:50.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.728+0000 7f220a8c3700 1 --2- 172.21.15.40:0/724937367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21fc04e490 0x7f21fc0dee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:50.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.728+0000 7f220a8c3700 1 --2- 172.21.15.40:0/724937367 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f21e8066730 0x7f21e8068bf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:50.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.728+0000 7f220a8c3700 1 --2- 172.21.15.40:0/724937367 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21fc0366e0 0x7f21fc0db5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:50.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.728+0000 7f220a8c3700 1 -- 172.21.15.40:0/724937367 >> 172.21.15.40:0/724937367 conn(0x7f21fc009a80 msgr2=0x7f21fc04ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:50.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.728+0000 7f220a8c3700 1 -- 172.21.15.40:0/724937367 shutdown_connections 2024-09-13T17:54:50.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:50.728+0000 7f220a8c3700 1 -- 172.21.15.40:0/724937367 wait complete. 2024-09-13T17:54:53.748 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:54:53.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.868+0000 7fac4eb46700 1 Processor -- start 2024-09-13T17:54:53.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.868+0000 7fac4eb46700 1 -- start start 2024-09-13T17:54:53.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.868+0000 7fac4eb46700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac40051190 0x7fac4004c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:53.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.868+0000 7fac4eb46700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac40007f50 con 0x7fac40033260 2024-09-13T17:54:53.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.868+0000 7fac4eb46700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac400db8a0 con 0x7fac40036150 2024-09-13T17:54:53.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.868+0000 7fac4eb46700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac400067c0 con 0x7fac40051190 2024-09-13T17:54:53.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.868+0000 7fac4e345700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fac40033260 0x7fac400e5380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35852/0 (socket says 172.21.15.40:35852) 2024-09-13T17:54:53.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.868+0000 7fac4e345700 1 -- 172.21.15.40:0/3036098829 learned_addr learned my addr 172.21.15.40:0/3036098829 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:54:53.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.868+0000 7fac4d343700 1 --2- 172.21.15.40:0/3036098829 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac40051190 0x7fac4004c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:53.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.872+0000 7fac4cb42700 1 -- 172.21.15.40:0/3036098829 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666288017 0 0) 0x7fac40007f50 con 0x7fac40033260 2024-09-13T17:54:53.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.872+0000 7fac4cb42700 1 -- 172.21.15.40:0/3036098829 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac340035d0 con 0x7fac40033260 2024-09-13T17:54:53.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.872+0000 7fac4d343700 1 -- 172.21.15.40:0/3036098829 >> v1:172.21.15.43:6789/0 conn(0x7fac40036150 legacy=0x7fac40007840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:53.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.872+0000 7fac4d343700 1 -- 172.21.15.40:0/3036098829 >> v1:172.21.15.40:6789/0 conn(0x7fac40033260 legacy=0x7fac400e5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:53.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.872+0000 7fac4d343700 1 -- 172.21.15.40:0/3036098829 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac40006aa0 con 0x7fac40051190 2024-09-13T17:54:53.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.872+0000 7fac4d343700 1 --2- 172.21.15.40:0/3036098829 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac40051190 0x7fac4004c3a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fac3800aa20 tx=0x7fac3800ad30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f7db6e56eeeb0cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:53.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.872+0000 7fac4cb42700 1 -- 172.21.15.40:0/3036098829 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3800cb70 con 0x7fac40051190 2024-09-13T17:54:53.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.872+0000 7fac4cb42700 1 -- 172.21.15.40:0/3036098829 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3800ccd0 con 0x7fac40051190 2024-09-13T17:54:53.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.872+0000 7fac4cb42700 1 -- 172.21.15.40:0/3036098829 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac38017640 con 0x7fac40051190 2024-09-13T17:54:53.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.872+0000 7fac4eb46700 1 -- 172.21.15.40:0/3036098829 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac40051190 msgr2=0x7fac4004c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:53.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.872+0000 7fac4eb46700 1 --2- 172.21.15.40:0/3036098829 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac40051190 0x7fac4004c3a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fac3800aa20 tx=0x7fac3800ad30 comp rx=0 tx=0).stop 2024-09-13T17:54:53.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.872+0000 7fac4eb46700 1 -- 172.21.15.40:0/3036098829 shutdown_connections 2024-09-13T17:54:53.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.872+0000 7fac4eb46700 1 --2- 172.21.15.40:0/3036098829 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac40051190 0x7fac4004c3a0 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fac3800aa20 tx=0x7fac3800ad30 comp rx=0 tx=0).stop 2024-09-13T17:54:53.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.872+0000 7fac4eb46700 1 -- 172.21.15.40:0/3036098829 >> 172.21.15.40:0/3036098829 conn(0x7fac400505e0 msgr2=0x7fac400509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:53.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.872+0000 7fac4eb46700 1 -- 172.21.15.40:0/3036098829 shutdown_connections 2024-09-13T17:54:53.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.872+0000 7fac4eb46700 1 -- 172.21.15.40:0/3036098829 wait complete. 2024-09-13T17:54:53.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4eb46700 1 Processor -- start 2024-09-13T17:54:53.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4eb46700 1 -- start start 2024-09-13T17:54:53.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4eb46700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac40033260 0x7fac4004a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:53.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4eb46700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac40036150 0x7fac400478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:53.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4eb46700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac4004fb20 0x7fac400ce460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:53.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4eb46700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4004edd0 con 0x7fac40036150 2024-09-13T17:54:53.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4eb46700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac40049ee0 con 0x7fac4004fb20 2024-09-13T17:54:53.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4eb46700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac400495f0 con 0x7fac40033260 2024-09-13T17:54:53.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4d343700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac40036150 0x7fac400478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:53.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4d343700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac40036150 0x7fac400478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37596/0 (socket says 172.21.15.40:37596) 2024-09-13T17:54:53.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4db44700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac40033260 0x7fac4004a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:53.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4d343700 1 -- 172.21.15.40:0/3368174042 learned_addr learned my addr 172.21.15.40:0/3368174042 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:53.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4e345700 1 --2- 172.21.15.40:0/3368174042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac4004fb20 0x7fac400ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:53.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4d343700 1 -- 172.21.15.40:0/3368174042 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac40033260 msgr2=0x7fac4004a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:53.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4d343700 1 --2- 172.21.15.40:0/3368174042 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac40033260 0x7fac4004a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:53.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4d343700 1 -- 172.21.15.40:0/3368174042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac4004fb20 msgr2=0x7fac400ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:53.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4d343700 1 --2- 172.21.15.40:0/3368174042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac4004fb20 0x7fac400ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:53.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4d343700 1 -- 172.21.15.40:0/3368174042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3800a650 con 0x7fac40036150 2024-09-13T17:54:53.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4d343700 1 --2- 172.21.15.40:0/3368174042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac40036150 0x7fac400478e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fac38019040 tx=0x7fac38004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:53.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac3e7fc700 1 -- 172.21.15.40:0/3368174042 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3800aee0 con 0x7fac40036150 2024-09-13T17:54:53.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac3e7fc700 1 -- 172.21.15.40:0/3368174042 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac38017630 con 0x7fac40036150 2024-09-13T17:54:53.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4eb46700 1 -- 172.21.15.40:0/3368174042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac4004f000 con 0x7fac40036150 2024-09-13T17:54:53.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac3e7fc700 1 -- 172.21.15.40:0/3368174042 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac38004270 con 0x7fac40036150 2024-09-13T17:54:53.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.876+0000 7fac4eb46700 1 -- 172.21.15.40:0/3368174042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac40049b70 con 0x7fac40036150 2024-09-13T17:54:53.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.880+0000 7fac4eb46700 1 -- 172.21.15.40:0/3368174042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac4004ac30 con 0x7fac40036150 2024-09-13T17:54:53.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.880+0000 7fac3e7fc700 1 -- 172.21.15.40:0/3368174042 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fac380043d0 con 0x7fac40036150 2024-09-13T17:54:53.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.880+0000 7fac3e7fc700 1 --2- 172.21.15.40:0/3368174042 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fac2c066730 0x7fac2c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:53.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.880+0000 7fac4db44700 1 --2- 172.21.15.40:0/3368174042 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fac2c066730 0x7fac2c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:53.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.880+0000 7fac3e7fc700 1 -- 172.21.15.40:0/3368174042 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fac38013070 con 0x7fac40036150 2024-09-13T17:54:53.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.880+0000 7fac4db44700 1 --2- 172.21.15.40:0/3368174042 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fac2c066730 0x7fac2c068bf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fac440017d0 tx=0x7fac44009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:53.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:53.888+0000 7fac3e7fc700 1 -- 172.21.15.40:0/3368174042 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fac38052c40 con 0x7fac40036150 2024-09-13T17:54:54.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.152+0000 7fac4eb46700 1 -- 172.21.15.40:0/3368174042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fac400d9bf0 con 0x7fac40036150 2024-09-13T17:54:54.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.152+0000 7fac3e7fc700 1 -- 172.21.15.40:0/3368174042 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7fac38056260 con 0x7fac40036150 2024-09-13T17:54:54.158 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:54:54.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.156+0000 7fac4eb46700 1 -- 172.21.15.40:0/3368174042 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fac2c066730 msgr2=0x7fac2c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.156+0000 7fac4eb46700 1 --2- 172.21.15.40:0/3368174042 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fac2c066730 0x7fac2c068bf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fac440017d0 tx=0x7fac44009040 comp rx=0 tx=0).stop 2024-09-13T17:54:54.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.156+0000 7fac4eb46700 1 -- 172.21.15.40:0/3368174042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac40036150 msgr2=0x7fac400478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.156+0000 7fac4eb46700 1 --2- 172.21.15.40:0/3368174042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac40036150 0x7fac400478e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fac38019040 tx=0x7fac38004770 comp rx=0 tx=0).stop 2024-09-13T17:54:54.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.156+0000 7fac4eb46700 1 -- 172.21.15.40:0/3368174042 shutdown_connections 2024-09-13T17:54:54.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.156+0000 7fac4eb46700 1 --2- 172.21.15.40:0/3368174042 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac4004fb20 0x7fac400ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.156+0000 7fac4eb46700 1 --2- 172.21.15.40:0/3368174042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac40036150 0x7fac400478e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.156+0000 7fac4eb46700 1 --2- 172.21.15.40:0/3368174042 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fac2c066730 0x7fac2c068bf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.156+0000 7fac4eb46700 1 --2- 172.21.15.40:0/3368174042 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac40033260 0x7fac4004a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.156+0000 7fac4eb46700 1 -- 172.21.15.40:0/3368174042 >> 172.21.15.40:0/3368174042 conn(0x7fac400505e0 msgr2=0x7fac4004c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:54.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.156+0000 7fac4eb46700 1 -- 172.21.15.40:0/3368174042 shutdown_connections 2024-09-13T17:54:54.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.156+0000 7fac4eb46700 1 -- 172.21.15.40:0/3368174042 wait complete. 2024-09-13T17:54:54.175 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T17:54:54.175 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T17:54:54.175 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:54:54.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f676caaa700 1 Processor -- start 2024-09-13T17:54:54.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f676caaa700 1 -- start start 2024-09-13T17:54:54.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f676caaa700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f67680cc620 0x7f6768153520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f676caaa700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f676815d620 0x7f6768153a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f676caaa700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6768159780 con 0x7f67680cc240 2024-09-13T17:54:54.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f676caaa700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6768153fa0 con 0x7f676815d620 2024-09-13T17:54:54.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f676caaa700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6768159a40 con 0x7f67680cc620 2024-09-13T17:54:54.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f6766d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f676815d620 0x7f6768153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f6766d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f676815d620 0x7f6768153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37610/0 (socket says 172.21.15.40:37610) 2024-09-13T17:54:54.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f6766d9d700 1 -- 172.21.15.40:0/1848834377 learned_addr learned my addr 172.21.15.40:0/1848834377 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f675dd9b700 1 --2- 172.21.15.40:0/1848834377 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f67680cc620 0x7f6768153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f6766d9d700 1 -- 172.21.15.40:0/1848834377 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f67680cc620 msgr2=0x7f6768153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f6766d9d700 1 --2- 172.21.15.40:0/1848834377 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f67680cc620 0x7f6768153520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f6766d9d700 1 -- 172.21.15.40:0/1848834377 >> v1:172.21.15.43:6789/0 conn(0x7f67680cc240 legacy=0x7f676815cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f6766d9d700 1 -- 172.21.15.40:0/1848834377 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6768159d20 con 0x7f676815d620 2024-09-13T17:54:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f6766d9d700 1 --2- 172.21.15.40:0/1848834377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f676815d620 0x7f6768153a60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6760009e80 tx=0x7f676000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a7c297c714cd69 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f6765d9b700 1 -- 172.21.15.40:0/1848834377 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676000e070 con 0x7f676815d620 2024-09-13T17:54:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f6765d9b700 1 -- 172.21.15.40:0/1848834377 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676000b8d0 con 0x7f676815d620 2024-09-13T17:54:54.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f6765d9b700 1 -- 172.21.15.40:0/1848834377 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676000bc80 con 0x7f676815d620 2024-09-13T17:54:54.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f676caaa700 1 -- 172.21.15.40:0/1848834377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f676815d620 msgr2=0x7f6768153a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f676caaa700 1 --2- 172.21.15.40:0/1848834377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f676815d620 0x7f6768153a60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6760009e80 tx=0x7f676000b330 comp rx=0 tx=0).stop 2024-09-13T17:54:54.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f676caaa700 1 -- 172.21.15.40:0/1848834377 shutdown_connections 2024-09-13T17:54:54.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f676caaa700 1 --2- 172.21.15.40:0/1848834377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f676815d620 0x7f6768153a60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f676caaa700 1 --2- 172.21.15.40:0/1848834377 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f67680cc620 0x7f6768153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f676caaa700 1 -- 172.21.15.40:0/1848834377 >> 172.21.15.40:0/1848834377 conn(0x7f6768066480 msgr2=0x7f6768066890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:54.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.276+0000 7f676caaa700 1 -- 172.21.15.40:0/1848834377 shutdown_connections 2024-09-13T17:54:54.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.280+0000 7f676caaa700 1 -- 172.21.15.40:0/1848834377 wait complete. 2024-09-13T17:54:54.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.280+0000 7f676caaa700 1 Processor -- start 2024-09-13T17:54:54.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.280+0000 7f676caaa700 1 -- start start 2024-09-13T17:54:54.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f676caaa700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67680cc240 0x7f676815b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f676caaa700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f67680cc620 0x7f67680c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f676caaa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f676815d620 0x7f67680c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f676caaa700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67680c86f0 con 0x7f676815d620 2024-09-13T17:54:54.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f676caaa700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67680c8990 con 0x7f67680cc240 2024-09-13T17:54:54.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f676caaa700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67680c8c50 con 0x7f67680cc620 2024-09-13T17:54:54.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f6766d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f676815d620 0x7f67680c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f6766d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f676815d620 0x7f67680c6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37612/0 (socket says 172.21.15.40:37612) 2024-09-13T17:54:54.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f6766d9d700 1 -- 172.21.15.40:0/3976114440 learned_addr learned my addr 172.21.15.40:0/3976114440 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:54.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f676659c700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67680cc240 0x7f676815b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f675dd9b700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f67680cc620 0x7f67680c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f6766d9d700 1 -- 172.21.15.40:0/3976114440 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f67680cc620 msgr2=0x7f67680c79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f6766d9d700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f67680cc620 0x7f67680c79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f6766d9d700 1 -- 172.21.15.40:0/3976114440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67680cc240 msgr2=0x7f676815b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f6766d9d700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67680cc240 0x7f676815b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f6766d9d700 1 -- 172.21.15.40:0/3976114440 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6760009b30 con 0x7f676815d620 2024-09-13T17:54:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f6766d9d700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f676815d620 0x7f67680c6680 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6760012000 tx=0x7f6760004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f675f7fe700 1 -- 172.21.15.40:0/3976114440 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760004800 con 0x7f676815d620 2024-09-13T17:54:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f675f7fe700 1 -- 172.21.15.40:0/3976114440 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67600092e0 con 0x7f676815d620 2024-09-13T17:54:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f676caaa700 1 -- 172.21.15.40:0/3976114440 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67680c8f30 con 0x7f676815d620 2024-09-13T17:54:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f675f7fe700 1 -- 172.21.15.40:0/3976114440 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676001b650 con 0x7f676815d620 2024-09-13T17:54:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f676caaa700 1 -- 172.21.15.40:0/3976114440 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67680c7490 con 0x7f676815d620 2024-09-13T17:54:54.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f676caaa700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67680b2c10 0x7f676814cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f676caaa700 1 -- 172.21.15.40:0/3976114440 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6768067520 con 0x7f67680b2c10 2024-09-13T17:54:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f676659c700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67680b2c10 0x7f676814cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f676659c700 1 -- 172.21.15.40:0/3976114440 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f676815a5c0 con 0x7f67680b2c10 2024-09-13T17:54:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.284+0000 7f676659c700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67680b2c10 0x7f676814cbe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f675800f770 tx=0x7f675800fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:54.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.288+0000 7f675f7fe700 1 -- 172.21.15.40:0/3976114440 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67580052e0 con 0x7f67680b2c10 2024-09-13T17:54:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.288+0000 7f675f7fe700 1 -- 172.21.15.40:0/3976114440 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6758005440 con 0x7f67680b2c10 2024-09-13T17:54:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.288+0000 7f675f7fe700 1 -- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67680b2c10 msgr2=0x7f676814cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.288+0000 7f675f7fe700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67680b2c10 0x7f676814cbe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f675800f770 tx=0x7f675800fb30 comp rx=0 tx=0).stop 2024-09-13T17:54:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.288+0000 7f675f7fe700 1 -- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67680b2c10 msgr2=0x7f676814cbe0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:54:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.288+0000 7f675f7fe700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67680b2c10 0x7f676814cbe0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.288+0000 7f675f7fe700 1 -- 172.21.15.40:0/3976114440 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f676001b7b0 con 0x7f676815d620 2024-09-13T17:54:54.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.288+0000 7f675f7fe700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6748066440 0x7f6748068900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.288+0000 7f675f7fe700 1 -- 172.21.15.40:0/3976114440 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f67600856c0 con 0x7f676815d620 2024-09-13T17:54:54.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.288+0000 7f676659c700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6748066440 0x7f6748068900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.288+0000 7f676659c700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6748066440 0x7f6748068900 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f675800f770 tx=0x7f6758005000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:54.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f676caaa700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67680c90c0 0x7f676814d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f676caaa700 1 -- 172.21.15.40:0/3976114440 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676815a2e0 con 0x7f67680c90c0 2024-09-13T17:54:54.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f675dd9b700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67680c90c0 0x7f676814d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f675dd9b700 1 -- 172.21.15.40:0/3976114440 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f67680cad10 con 0x7f67680c90c0 2024-09-13T17:54:54.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f675dd9b700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67680c90c0 0x7f676814d120 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6768158e40 tx=0x7f6750011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:54.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f675f7fe700 1 -- 172.21.15.40:0/3976114440 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6750004030 con 0x7f67680c90c0 2024-09-13T17:54:54.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f675f7fe700 1 -- 172.21.15.40:0/3976114440 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4949 (secure 0 0 0) 0x7f6750004190 con 0x7f67680c90c0 2024-09-13T17:54:54.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f675f7fe700 1 -- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67680c90c0 msgr2=0x7f676814d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f675f7fe700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67680c90c0 0x7f676814d120 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6768158e40 tx=0x7f6750011d30 comp rx=0 tx=0).stop 2024-09-13T17:54:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f675f7fe700 1 -- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67680c90c0 msgr2=0x7f676814d120 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:54:54.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f675f7fe700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67680c90c0 0x7f676814d120 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.307 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:54:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:54:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:54:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:54:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 40, 2024-09-13T17:54:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:54:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:54:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:54:54.307 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:54:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T17:54:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:54:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:54:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:54:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:54.308 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:54:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:54:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:54.309 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:54:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:54:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:54:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:54:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:54:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:54:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:54:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:54:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:54:54.310 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:54:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:54:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:54:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:54:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:54:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:54:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:54:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:54:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:54.311 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:54:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:54:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:54:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:54:54.312 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:54:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:54:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:54.313 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:54:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:54:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:54:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:54:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.314 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:54:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:54:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:54:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:54:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:54.315 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:54:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:54:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:54:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.316 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:54:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:54:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:54:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:54.317 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:54:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:54:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:54:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:54:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:54.318 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:54:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:54:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:54:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:54:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:54.319 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:54:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:54:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:54:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.320 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:54:54.321 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:54:54.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f676caaa700 1 -- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6748066440 msgr2=0x7f6748068900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f676caaa700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6748066440 0x7f6748068900 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f675800f770 tx=0x7f6758005000 comp rx=0 tx=0).stop 2024-09-13T17:54:54.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f676caaa700 1 -- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f676815d620 msgr2=0x7f67680c6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f6766d9d700 1 -- 172.21.15.40:0/3976114440 reap_dead start 2024-09-13T17:54:54.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f676caaa700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f676815d620 0x7f67680c6680 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6760012000 tx=0x7f6760004af0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f676caaa700 1 -- 172.21.15.40:0/3976114440 shutdown_connections 2024-09-13T17:54:54.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f676caaa700 1 --2- 172.21.15.40:0/3976114440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f676815d620 0x7f67680c6680 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f676caaa700 1 -- 172.21.15.40:0/3976114440 >> 172.21.15.40:0/3976114440 conn(0x7f6768066480 msgr2=0x7f67681e29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:54.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f676caaa700 1 -- 172.21.15.40:0/3976114440 shutdown_connections 2024-09-13T17:54:54.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.300+0000 7f676caaa700 1 -- 172.21.15.40:0/3976114440 wait complete. 2024-09-13T17:54:54.323 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:54:54.323 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T17:54:54.324 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:54:54.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.440+0000 7f064ade6700 1 Processor -- start 2024-09-13T17:54:54.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.440+0000 7f064ade6700 1 -- start start 2024-09-13T17:54:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.440+0000 7f064ade6700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f06440b1b60 0x7f06440c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.440+0000 7f064ade6700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06441596a0 con 0x7f0644153c30 2024-09-13T17:54:54.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.440+0000 7f064ade6700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0644159960 con 0x7f06440ca440 2024-09-13T17:54:54.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.440+0000 7f064ade6700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0644159c40 con 0x7f06440b1b60 2024-09-13T17:54:54.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.440+0000 7f0649de4700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0644153c30 0x7f06440cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35860/0 (socket says 172.21.15.40:35860) 2024-09-13T17:54:54.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.440+0000 7f0649de4700 1 -- 172.21.15.40:0/2049416743 learned_addr learned my addr 172.21.15.40:0/2049416743 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:54:54.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.440+0000 7f06495e3700 1 --2- 172.21.15.40:0/2049416743 >> v2:172.21.15.43:3300/0 conn(0x7f06440b1b60 0x7f06440c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.440+0000 7f0648de2700 1 -- 172.21.15.40:0/2049416743 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404611815 0 0) 0x7f06441596a0 con 0x7f0644153c30 2024-09-13T17:54:54.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.440+0000 7f0648de2700 1 -- 172.21.15.40:0/2049416743 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06300035d0 con 0x7f0644153c30 2024-09-13T17:54:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0648de2700 1 -- 172.21.15.40:0/2049416743 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1515227246 0 0) 0x7f06300035d0 con 0x7f0644153c30 2024-09-13T17:54:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0648de2700 1 -- 172.21.15.40:0/2049416743 >> v2:172.21.15.43:3300/0 conn(0x7f06440b1b60 msgr2=0x7f06440c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0648de2700 1 --2- 172.21.15.40:0/2049416743 >> v2:172.21.15.43:3300/0 conn(0x7f06440b1b60 0x7f06440c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0648de2700 1 -- 172.21.15.40:0/2049416743 >> v1:172.21.15.43:6789/0 conn(0x7f06440ca440 legacy=0x7f06440cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0648de2700 1 -- 172.21.15.40:0/2049416743 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0644159f20 con 0x7f0644153c30 2024-09-13T17:54:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0648de2700 1 -- 172.21.15.40:0/2049416743 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f0640001670 con 0x7f0644153c30 2024-09-13T17:54:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0648de2700 1 -- 172.21.15.40:0/2049416743 >> v1:172.21.15.40:6789/0 conn(0x7f0644153c30 legacy=0x7f06440cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0648de2700 1 --2- 172.21.15.40:0/2049416743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0630003a20 0x7f0630003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0648de2700 1 --2- 172.21.15.40:0/2049416743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0630004870 0x7f0630004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0648de2700 1 --2- 172.21.15.40:0/2049416743 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0630005230 0x7f063000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0648de2700 1 -- 172.21.15.40:0/2049416743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0644159c40 con 0x7f0630003a20 2024-09-13T17:54:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0648de2700 1 -- 172.21.15.40:0/2049416743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063000c1d0 con 0x7f0630004870 2024-09-13T17:54:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0648de2700 1 -- 172.21.15.40:0/2049416743 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063000c390 con 0x7f0630005230 2024-09-13T17:54:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0649de4700 1 --2- 172.21.15.40:0/2049416743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0630003a20 0x7f0630003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f064a5e5700 1 --2- 172.21.15.40:0/2049416743 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0630005230 0x7f063000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0649de4700 1 -- 172.21.15.40:0/2049416743 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0630005230 msgr2=0x7f063000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0649de4700 1 --2- 172.21.15.40:0/2049416743 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0630005230 0x7f063000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0649de4700 1 -- 172.21.15.40:0/2049416743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0630004870 msgr2=0x7f0630004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f06495e3700 1 --2- 172.21.15.40:0/2049416743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0630004870 0x7f0630004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0649de4700 1 --2- 172.21.15.40:0/2049416743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0630004870 0x7f0630004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f064a5e5700 1 -- 172.21.15.40:0/2049416743 reap_dead start 2024-09-13T17:54:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0649de4700 1 -- 172.21.15.40:0/2049416743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06300035d0 con 0x7f0630003a20 2024-09-13T17:54:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f06495e3700 1 --2- 172.21.15.40:0/2049416743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0630004870 0x7f0630004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:54:54.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0649de4700 1 --2- 172.21.15.40:0/2049416743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0630003a20 0x7f0630003e40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f064000c320 tx=0x7f06300058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a22c375f83907f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:54.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0648de2700 1 -- 172.21.15.40:0/2049416743 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0640007c30 con 0x7f0630003a20 2024-09-13T17:54:54.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0648de2700 1 -- 172.21.15.40:0/2049416743 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064000ce60 con 0x7f0630003a20 2024-09-13T17:54:54.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f0648de2700 1 -- 172.21.15.40:0/2049416743 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0640005bc0 con 0x7f0630003a20 2024-09-13T17:54:54.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f064ade6700 1 -- 172.21.15.40:0/2049416743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0630003a20 msgr2=0x7f0630003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f064ade6700 1 --2- 172.21.15.40:0/2049416743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0630003a20 0x7f0630003e40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f064000c320 tx=0x7f06300058a0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f064ade6700 1 -- 172.21.15.40:0/2049416743 shutdown_connections 2024-09-13T17:54:54.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f064ade6700 1 --2- 172.21.15.40:0/2049416743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0630003a20 0x7f0630003e40 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f064000c320 tx=0x7f06300058a0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f064ade6700 1 -- 172.21.15.40:0/2049416743 >> 172.21.15.40:0/2049416743 conn(0x7f0644066090 msgr2=0x7f06440664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:54.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f064ade6700 1 -- 172.21.15.40:0/2049416743 shutdown_connections 2024-09-13T17:54:54.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.444+0000 7f064ade6700 1 -- 172.21.15.40:0/2049416743 wait complete. 2024-09-13T17:54:54.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f064ade6700 1 Processor -- start 2024-09-13T17:54:54.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f064ade6700 1 -- start start 2024-09-13T17:54:54.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f064ade6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06440b4810 0x7f06441ef170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f064ade6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06440b4bf0 0x7f064414d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f064ade6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06441f32b0 0x7f064414d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f064ade6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06441f2e90 con 0x7f06440b4810 2024-09-13T17:54:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f064ade6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06441ef6b0 con 0x7f06441f32b0 2024-09-13T17:54:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f064ade6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06440cd760 con 0x7f06440b4bf0 2024-09-13T17:54:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f0649de4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06440b4810 0x7f06441ef170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f0649de4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06440b4810 0x7f06441ef170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37666/0 (socket says 172.21.15.40:37666) 2024-09-13T17:54:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f0649de4700 1 -- 172.21.15.40:0/4069910940 learned_addr learned my addr 172.21.15.40:0/4069910940 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f06495e3700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06440b4bf0 0x7f064414d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f064a5e5700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06441f32b0 0x7f064414d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f0649de4700 1 -- 172.21.15.40:0/4069910940 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06440b4bf0 msgr2=0x7f064414d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f0649de4700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f06440b4bf0 0x7f064414d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f0649de4700 1 -- 172.21.15.40:0/4069910940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06441f32b0 msgr2=0x7f064414d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f0649de4700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06441f32b0 0x7f064414d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f0649de4700 1 -- 172.21.15.40:0/4069910940 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06440cd1a0 con 0x7f06440b4810 2024-09-13T17:54:54.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f0649de4700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06440b4810 0x7f06441ef170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0640011000 tx=0x7f0640001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:54.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f063a7fc700 1 -- 172.21.15.40:0/4069910940 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0640000ed0 con 0x7f06440b4810 2024-09-13T17:54:54.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f063a7fc700 1 -- 172.21.15.40:0/4069910940 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0640004e50 con 0x7f06440b4810 2024-09-13T17:54:54.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f063a7fc700 1 -- 172.21.15.40:0/4069910940 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0640007310 con 0x7f06440b4810 2024-09-13T17:54:54.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f064ade6700 1 -- 172.21.15.40:0/4069910940 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06440cfa30 con 0x7f06440b4810 2024-09-13T17:54:54.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.448+0000 7f064ade6700 1 -- 172.21.15.40:0/4069910940 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06441f2810 con 0x7f06440b4810 2024-09-13T17:54:54.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.452+0000 7f064ade6700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06441f29d0 0x7f06440c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.452+0000 7f064ade6700 1 -- 172.21.15.40:0/4069910940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0644074750 con 0x7f06441f29d0 2024-09-13T17:54:54.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.452+0000 7f06495e3700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06441f29d0 0x7f06440c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.452+0000 7f06495e3700 1 -- 172.21.15.40:0/4069910940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f064406ccb0 con 0x7f06441f29d0 2024-09-13T17:54:54.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.452+0000 7f06495e3700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06441f29d0 0x7f06440c7570 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f06441f02a0 tx=0x7f06340139c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:54.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.452+0000 7f063a7fc700 1 -- 172.21.15.40:0/4069910940 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0640007470 con 0x7f06440b4810 2024-09-13T17:54:54.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.452+0000 7f063a7fc700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0628066760 0x7f0628068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.452+0000 7f064a5e5700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0628066760 0x7f0628068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.452+0000 7f063a7fc700 1 -- 172.21.15.40:0/4069910940 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634016980 con 0x7f06441f29d0 2024-09-13T17:54:54.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.452+0000 7f063a7fc700 1 -- 172.21.15.40:0/4069910940 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0640085430 con 0x7f06440b4810 2024-09-13T17:54:54.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.452+0000 7f063a7fc700 1 -- 172.21.15.40:0/4069910940 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0634016ae0 con 0x7f06441f29d0 2024-09-13T17:54:54.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.452+0000 7f063a7fc700 1 -- 172.21.15.40:0/4069910940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06441f29d0 msgr2=0x7f06440c7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.452+0000 7f063a7fc700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06441f29d0 0x7f06440c7570 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f06441f02a0 tx=0x7f06340139c0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.452+0000 7f063a7fc700 1 -- 172.21.15.40:0/4069910940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06441f29d0 msgr2=0x7f06440c7570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:54:54.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.452+0000 7f063a7fc700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06441f29d0 0x7f06440c7570 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.452+0000 7f064a5e5700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0628066760 0x7f0628068c20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f06440ca820 tx=0x7f063c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:54.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.460+0000 7f064ade6700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06440c7ab0 0x7f06440c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.460+0000 7f064ade6700 1 -- 172.21.15.40:0/4069910940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064415a4e0 con 0x7f06440c7ab0 2024-09-13T17:54:54.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.460+0000 7f06495e3700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06440c7ab0 0x7f06440c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.464+0000 7f06495e3700 1 -- 172.21.15.40:0/4069910940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f06440b1f90 con 0x7f06440c7ab0 2024-09-13T17:54:54.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.464+0000 7f06495e3700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06440c7ab0 0x7f06440c7ec0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f063401a000 tx=0x7f0634004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:54.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.464+0000 7f063a7fc700 1 -- 172.21.15.40:0/4069910940 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634025040 con 0x7f06440c7ab0 2024-09-13T17:54:54.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.464+0000 7f063a7fc700 1 -- 172.21.15.40:0/4069910940 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4449 (secure 0 0 0) 0x7f0634015a40 con 0x7f06440c7ab0 2024-09-13T17:54:54.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.464+0000 7f063a7fc700 1 -- 172.21.15.40:0/4069910940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06440c7ab0 msgr2=0x7f06440c7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.464+0000 7f063a7fc700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06440c7ab0 0x7f06440c7ec0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f063401a000 tx=0x7f0634004770 comp rx=0 tx=0).stop 2024-09-13T17:54:54.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.464+0000 7f063a7fc700 1 -- 172.21.15.40:0/4069910940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06440c7ab0 msgr2=0x7f06440c7ec0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:54:54.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.464+0000 7f063a7fc700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06440c7ab0 0x7f06440c7ec0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.470 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:54:54.470 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:54:54.470 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:54:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:54:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 40, 2024-09-13T17:54:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:54:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:54:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:54:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:54:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T17:54:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:54:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:54:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:54:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:54.471 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:54:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:54:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:54.472 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:54:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:54:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:54:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:54:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:54:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:54:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:54:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:54:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:54:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:54:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:54:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:54:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:54:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:54:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:54:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:54:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:54:54.474 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:54:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:54:54.475 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:54:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:54:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:54:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:54:54.476 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:54.477 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.477 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:54.477 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.477 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:54:54.477 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:54:54.477 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:54.477 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:54.477 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:54.477 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.477 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.477 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:54:54.477 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:54:54.477 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:54.477 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:54:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:54:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:54:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:54:54.478 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:54:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:54:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:54:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:54.479 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:54:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:54:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:54:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.480 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:54:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:54:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:54:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:54:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:54.481 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:54.482 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:54:54.482 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.482 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:54.482 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.482 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:54:54.482 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:54:54.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.464+0000 7f064ade6700 1 -- 172.21.15.40:0/4069910940 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0628066760 msgr2=0x7f0628068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.464+0000 7f064ade6700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0628066760 0x7f0628068c20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f06440ca820 tx=0x7f063c009040 comp rx=0 tx=0).stop 2024-09-13T17:54:54.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.464+0000 7f064ade6700 1 -- 172.21.15.40:0/4069910940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06440b4810 msgr2=0x7f06441ef170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.464+0000 7f064ade6700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06440b4810 0x7f06441ef170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0640011000 tx=0x7f0640001a00 comp rx=0 tx=0).stop 2024-09-13T17:54:54.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.464+0000 7f064a5e5700 1 -- 172.21.15.40:0/4069910940 reap_dead start 2024-09-13T17:54:54.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.464+0000 7f064ade6700 1 -- 172.21.15.40:0/4069910940 shutdown_connections 2024-09-13T17:54:54.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.464+0000 7f064ade6700 1 --2- 172.21.15.40:0/4069910940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06440b4810 0x7f06441ef170 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.464+0000 7f064ade6700 1 -- 172.21.15.40:0/4069910940 >> 172.21.15.40:0/4069910940 conn(0x7f0644066090 msgr2=0x7f06440cc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:54.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.464+0000 7f064ade6700 1 -- 172.21.15.40:0/4069910940 shutdown_connections 2024-09-13T17:54:54.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.464+0000 7f064ade6700 1 -- 172.21.15.40:0/4069910940 wait complete. 2024-09-13T17:54:54.487 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:54:54.487 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T17:54:54.487 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:54:54.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.612+0000 7f61af16e700 1 Processor -- start 2024-09-13T17:54:54.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.612+0000 7f61af16e700 1 -- start start 2024-09-13T17:54:54.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.612+0000 7f61af16e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80cf9b0 0x7f61a806c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.612+0000 7f61af16e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f61a806c5b0 0x7f61a8163d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.612+0000 7f61af16e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a8159e50 con 0x7f61a8153720 2024-09-13T17:54:54.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.612+0000 7f61af16e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a80c5ea0 con 0x7f61a806c5b0 2024-09-13T17:54:54.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.612+0000 7f61af16e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a815d390 con 0x7f61a80cf9b0 2024-09-13T17:54:54.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.612+0000 7f61ae16c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f61a8153720 0x7f61a80ca510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35872/0 (socket says 172.21.15.40:35872) 2024-09-13T17:54:54.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.612+0000 7f61ae16c700 1 -- 172.21.15.40:0/3976417160 learned_addr learned my addr 172.21.15.40:0/3976417160 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:54:54.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.612+0000 7f61ae96d700 1 --2- 172.21.15.40:0/3976417160 >> v2:172.21.15.40:3300/0 conn(0x7f61a806c5b0 0x7f61a8163d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.612+0000 7f61ad96b700 1 --2- 172.21.15.40:0/3976417160 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80cf9b0 0x7f61a806c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.612+0000 7f61ad16a700 1 -- 172.21.15.40:0/3976417160 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2794514377 0 0) 0x7f61a8159e50 con 0x7f61a8153720 2024-09-13T17:54:54.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.612+0000 7f61ad16a700 1 -- 172.21.15.40:0/3976417160 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61940035d0 con 0x7f61a8153720 2024-09-13T17:54:54.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.612+0000 7f61ae96d700 1 -- 172.21.15.40:0/3976417160 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80cf9b0 msgr2=0x7f61a806c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:54.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.612+0000 7f61ae96d700 1 --2- 172.21.15.40:0/3976417160 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80cf9b0 0x7f61a806c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.612+0000 7f61ae96d700 1 -- 172.21.15.40:0/3976417160 >> v1:172.21.15.40:6789/0 conn(0x7f61a8153720 legacy=0x7f61a80ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.612+0000 7f61ae96d700 1 -- 172.21.15.40:0/3976417160 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a80cac20 con 0x7f61a806c5b0 2024-09-13T17:54:54.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.612+0000 7f61ae96d700 1 --2- 172.21.15.40:0/3976417160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61a806c5b0 0x7f61a8163d50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f61a0009d90 tx=0x7f61a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290c392f1ecc41fd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:54.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.612+0000 7f61ad16a700 1 -- 172.21.15.40:0/3976417160 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a000e070 con 0x7f61a806c5b0 2024-09-13T17:54:54.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.616+0000 7f61ad16a700 1 -- 172.21.15.40:0/3976417160 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a000bb20 con 0x7f61a806c5b0 2024-09-13T17:54:54.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.616+0000 7f61ad16a700 1 -- 172.21.15.40:0/3976417160 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a0012430 con 0x7f61a806c5b0 2024-09-13T17:54:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.616+0000 7f61af16e700 1 -- 172.21.15.40:0/3976417160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61a806c5b0 msgr2=0x7f61a8163d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.616+0000 7f61af16e700 1 --2- 172.21.15.40:0/3976417160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61a806c5b0 0x7f61a8163d50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f61a0009d90 tx=0x7f61a000b330 comp rx=0 tx=0).stop 2024-09-13T17:54:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.616+0000 7f61af16e700 1 -- 172.21.15.40:0/3976417160 shutdown_connections 2024-09-13T17:54:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.616+0000 7f61af16e700 1 --2- 172.21.15.40:0/3976417160 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61a806c5b0 0x7f61a8163d50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.616+0000 7f61af16e700 1 --2- 172.21.15.40:0/3976417160 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80cf9b0 0x7f61a806c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.616+0000 7f61af16e700 1 -- 172.21.15.40:0/3976417160 >> 172.21.15.40:0/3976417160 conn(0x7f61a80666c0 msgr2=0x7f61a80cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:54.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.616+0000 7f61af16e700 1 -- 172.21.15.40:0/3976417160 shutdown_connections 2024-09-13T17:54:54.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.616+0000 7f61af16e700 1 -- 172.21.15.40:0/3976417160 wait complete. 2024-09-13T17:54:54.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.616+0000 7f61af16e700 1 Processor -- start 2024-09-13T17:54:54.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.616+0000 7f61af16e700 1 -- start start 2024-09-13T17:54:54.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.620+0000 7f61af16e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61a806c5b0 0x7f61a815c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.620+0000 7f61af16e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80cf9b0 0x7f61a80c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.620+0000 7f61af16e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61a8153720 0x7f61a81fc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.620+0000 7f61af16e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a80c7c70 con 0x7f61a8153720 2024-09-13T17:54:54.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.620+0000 7f61af16e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a81fbd10 con 0x7f61a806c5b0 2024-09-13T17:54:54.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.620+0000 7f61af16e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a81fbff0 con 0x7f61a80cf9b0 2024-09-13T17:54:54.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.620+0000 7f61ae16c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61a806c5b0 0x7f61a815c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.620+0000 7f61ae96d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61a8153720 0x7f61a81fc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.620+0000 7f61ae96d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61a8153720 0x7f61a81fc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37682/0 (socket says 172.21.15.40:37682) 2024-09-13T17:54:54.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.620+0000 7f61ae96d700 1 -- 172.21.15.40:0/732802400 learned_addr learned my addr 172.21.15.40:0/732802400 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:54.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.620+0000 7f61ad96b700 1 --2- 172.21.15.40:0/732802400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80cf9b0 0x7f61a80c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.620+0000 7f61ae96d700 1 -- 172.21.15.40:0/732802400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80cf9b0 msgr2=0x7f61a80c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.620+0000 7f61ae96d700 1 --2- 172.21.15.40:0/732802400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80cf9b0 0x7f61a80c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.620+0000 7f61ae96d700 1 -- 172.21.15.40:0/732802400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61a806c5b0 msgr2=0x7f61a815c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.620+0000 7f61ae96d700 1 --2- 172.21.15.40:0/732802400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61a806c5b0 0x7f61a815c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.620+0000 7f61ae96d700 1 -- 172.21.15.40:0/732802400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a0009a70 con 0x7f61a8153720 2024-09-13T17:54:54.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.620+0000 7f61ae96d700 1 --2- 172.21.15.40:0/732802400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61a8153720 0x7f61a81fc480 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f61a0014040 tx=0x7f61a0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:54.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.620+0000 7f619effd700 1 -- 172.21.15.40:0/732802400 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a0012430 con 0x7f61a8153720 2024-09-13T17:54:54.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.624+0000 7f619effd700 1 -- 172.21.15.40:0/732802400 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a0020070 con 0x7f61a8153720 2024-09-13T17:54:54.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.624+0000 7f619effd700 1 -- 172.21.15.40:0/732802400 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a00098f0 con 0x7f61a8153720 2024-09-13T17:54:54.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.624+0000 7f61af16e700 1 -- 172.21.15.40:0/732802400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a80c6560 con 0x7f61a8153720 2024-09-13T17:54:54.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.624+0000 7f61af16e700 1 -- 172.21.15.40:0/732802400 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a80cf030 con 0x7f61a8153720 2024-09-13T17:54:54.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.624+0000 7f61af16e700 1 --2- 172.21.15.40:0/732802400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80c3f60 0x7f61a815ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.624+0000 7f61af16e700 1 -- 172.21.15.40:0/732802400 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a8074750 con 0x7f61a80c3f60 2024-09-13T17:54:54.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.624+0000 7f61ae16c700 1 --2- 172.21.15.40:0/732802400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80c3f60 0x7f61a815ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.624+0000 7f619effd700 1 -- 172.21.15.40:0/732802400 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f61a0004040 con 0x7f61a8153720 2024-09-13T17:54:54.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.624+0000 7f61ae16c700 1 -- 172.21.15.40:0/732802400 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f61a8153b00 con 0x7f61a80c3f60 2024-09-13T17:54:54.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.624+0000 7f619effd700 1 --2- 172.21.15.40:0/732802400 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f618c066730 0x7f618c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.624+0000 7f619effd700 1 -- 172.21.15.40:0/732802400 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f61a0022070 con 0x7f61a8153720 2024-09-13T17:54:54.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.624+0000 7f61ad96b700 1 --2- 172.21.15.40:0/732802400 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f618c066730 0x7f618c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.624+0000 7f61ae16c700 1 --2- 172.21.15.40:0/732802400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80c3f60 0x7f61a815ab80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f61a40011e0 tx=0x7f61a40018b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:54.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.624+0000 7f619effd700 1 -- 172.21.15.40:0/732802400 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a4012560 con 0x7f61a80c3f60 2024-09-13T17:54:54.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.628+0000 7f61ad96b700 1 --2- 172.21.15.40:0/732802400 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f618c066730 0x7f618c068bf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f61a814e880 tx=0x7f6198008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:54.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.628+0000 7f619effd700 1 -- 172.21.15.40:0/732802400 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f61a40126c0 con 0x7f61a80c3f60 2024-09-13T17:54:54.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.628+0000 7f619effd700 1 -- 172.21.15.40:0/732802400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80c3f60 msgr2=0x7f61a815ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.628+0000 7f619effd700 1 --2- 172.21.15.40:0/732802400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80c3f60 0x7f61a815ab80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f61a40011e0 tx=0x7f61a40018b0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.628+0000 7f619effd700 1 -- 172.21.15.40:0/732802400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80c3f60 msgr2=0x7f61a815ab80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:54:54.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.628+0000 7f619effd700 1 --2- 172.21.15.40:0/732802400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80c3f60 0x7f61a815ab80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.632+0000 7f61af16e700 1 --2- 172.21.15.40:0/732802400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80c7330 0x7f61a81f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.632+0000 7f61af16e700 1 -- 172.21.15.40:0/732802400 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a815a340 con 0x7f61a80c7330 2024-09-13T17:54:54.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.632+0000 7f61ae16c700 1 --2- 172.21.15.40:0/732802400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80c7330 0x7f61a81f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.636+0000 7f61ae16c700 1 -- 172.21.15.40:0/732802400 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f61a815ca60 con 0x7f61a80c7330 2024-09-13T17:54:54.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.636+0000 7f61ae16c700 1 --2- 172.21.15.40:0/732802400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80c7330 0x7f61a81f8e10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f61a4006bb0 tx=0x7f61a4001150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:54.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.636+0000 7f619effd700 1 -- 172.21.15.40:0/732802400 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a4022070 con 0x7f61a80c7330 2024-09-13T17:54:54.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.636+0000 7f619effd700 1 -- 172.21.15.40:0/732802400 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4781 (secure 0 0 0) 0x7f61a4004aa0 con 0x7f61a80c7330 2024-09-13T17:54:54.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.636+0000 7f619effd700 1 -- 172.21.15.40:0/732802400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80c7330 msgr2=0x7f61a81f8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.636+0000 7f619effd700 1 --2- 172.21.15.40:0/732802400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80c7330 0x7f61a81f8e10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f61a4006bb0 tx=0x7f61a4001150 comp rx=0 tx=0).stop 2024-09-13T17:54:54.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.636+0000 7f619effd700 1 -- 172.21.15.40:0/732802400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80c7330 msgr2=0x7f61a81f8e10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:54:54.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.636+0000 7f619effd700 1 --2- 172.21.15.40:0/732802400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61a80c7330 0x7f61a81f8e10 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f61a4006bb0 tx=0x7f61a4001150 comp rx=0 tx=0).stop 2024-09-13T17:54:54.642 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:54:54.642 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:54:54.642 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:54:54.642 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:54:54.642 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 40, 2024-09-13T17:54:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:54:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:54:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:54:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:54:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T17:54:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:54:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:54:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:54:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:54.643 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:54.644 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:54.644 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:54.644 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:54.644 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:54.644 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:54.644 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:54.644 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:54.644 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:54:54.644 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:54:54.644 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:54.644 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:54:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:54:54.645 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:54:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:54:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:54:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:54:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:54:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:54:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:54:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:54:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:54:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:54:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:54:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:54:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:54:54.646 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:54:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:54:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:54:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:54:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:54:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:54:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:54:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:54:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:54:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:54:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:54:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:54:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:54:54.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:54:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:54:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:54:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:54:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.648 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:54:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:54:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:54:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.649 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:54:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:54:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:54:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:54:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.650 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:54.651 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.651 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:54:54.651 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:54:54.651 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:54.651 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:54.651 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:54.651 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.651 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.651 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:54:54.651 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:54:54.651 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:54:54.651 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:54:54.651 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:54:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:54:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:54:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:54:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:54:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:54:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:54:54.652 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:54:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:54:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:54:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:54:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:54:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:54:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:54.653 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:54:54.654 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.654 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:54.654 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:54.654 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:54:54.654 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.654 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:54.654 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:54:54.654 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.654 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:54.654 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:54.654 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:54:54.654 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.654 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:54:54.654 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:54:54.655 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:54:54.655 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:54:54.655 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:54:54.655 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:54:54.655 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:54:54.655 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:54:54.655 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:54:54.655 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:54:54.655 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:54:54.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.636+0000 7f61af16e700 1 -- 172.21.15.40:0/732802400 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f618c066730 msgr2=0x7f618c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.636+0000 7f61af16e700 1 --2- 172.21.15.40:0/732802400 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f618c066730 0x7f618c068bf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f61a814e880 tx=0x7f6198008040 comp rx=0 tx=0).stop 2024-09-13T17:54:54.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.636+0000 7f61af16e700 1 -- 172.21.15.40:0/732802400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61a8153720 msgr2=0x7f61a81fc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.636+0000 7f61ae96d700 1 -- 172.21.15.40:0/732802400 reap_dead start 2024-09-13T17:54:54.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.636+0000 7f61af16e700 1 --2- 172.21.15.40:0/732802400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61a8153720 0x7f61a81fc480 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f61a0014040 tx=0x7f61a0004af0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.636+0000 7f61af16e700 1 -- 172.21.15.40:0/732802400 shutdown_connections 2024-09-13T17:54:54.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.636+0000 7f61af16e700 1 --2- 172.21.15.40:0/732802400 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61a8153720 0x7f61a81fc480 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.636+0000 7f61af16e700 1 -- 172.21.15.40:0/732802400 >> 172.21.15.40:0/732802400 conn(0x7f61a80666c0 msgr2=0x7f61a8152970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:54.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.636+0000 7f61af16e700 1 -- 172.21.15.40:0/732802400 shutdown_connections 2024-09-13T17:54:54.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.636+0000 7f61af16e700 1 -- 172.21.15.40:0/732802400 wait complete. 2024-09-13T17:54:54.657 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:54:54.658 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T17:54:54.658 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T17:54:54.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffacb59e700 1 Processor -- start 2024-09-13T17:54:54.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffacb59e700 1 -- start start 2024-09-13T17:54:54.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffacb59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffacc066300 0x7ffacc0c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffacb59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffacc0666e0 0x7ffacc150f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffacb59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffacc158040 con 0x7ffacc0b1ad0 2024-09-13T17:54:54.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffacb59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffacc157430 con 0x7ffacc066300 2024-09-13T17:54:54.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffacb59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc157710 con 0x7ffacc0666e0 2024-09-13T17:54:54.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffaca59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffacc066300 0x7ffacc0c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffacad9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ffacc0b1ad0 0x7ffacc0cab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:35878/0 (socket says 172.21.15.40:35878) 2024-09-13T17:54:54.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffaca59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffacc066300 0x7ffacc0c9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37696/0 (socket says 172.21.15.40:37696) 2024-09-13T17:54:54.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffacad9d700 1 -- 172.21.15.40:0/2628517353 learned_addr learned my addr 172.21.15.40:0/2628517353 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:54:54.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffac3fff700 1 --2- 172.21.15.40:0/2628517353 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffacc0666e0 0x7ffacc150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffaca59c700 1 -- 172.21.15.40:0/2628517353 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffacc0666e0 msgr2=0x7ffacc150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:54:54.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffaca59c700 1 --2- 172.21.15.40:0/2628517353 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffacc0666e0 0x7ffacc150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffaca59c700 1 -- 172.21.15.40:0/2628517353 >> v1:172.21.15.40:6789/0 conn(0x7ffacc0b1ad0 legacy=0x7ffacc0cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffaca59c700 1 -- 172.21.15.40:0/2628517353 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffacc1579f0 con 0x7ffacc066300 2024-09-13T17:54:54.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffac9d9b700 1 -- 172.21.15.40:0/2628517353 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127879670 0 0) 0x7ffacc158040 con 0x7ffacc0b1ad0 2024-09-13T17:54:54.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffac3fff700 1 --2- 172.21.15.40:0/2628517353 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffacc0666e0 0x7ffacc150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:54:54.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffaca59c700 1 --2- 172.21.15.40:0/2628517353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffacc066300 0x7ffacc0c9be0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009850 tx=0x7ffabc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec69997fe016a3a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:54.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffac9d9b700 1 -- 172.21.15.40:0/2628517353 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc011830 con 0x7ffacc066300 2024-09-13T17:54:54.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffac9d9b700 1 -- 172.21.15.40:0/2628517353 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffabc011990 con 0x7ffacc066300 2024-09-13T17:54:54.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffac9d9b700 1 -- 172.21.15.40:0/2628517353 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc011d20 con 0x7ffacc066300 2024-09-13T17:54:54.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffacb59e700 1 -- 172.21.15.40:0/2628517353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffacc066300 msgr2=0x7ffacc0c9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffacb59e700 1 --2- 172.21.15.40:0/2628517353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffacc066300 0x7ffacc0c9be0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009850 tx=0x7ffabc009b60 comp rx=0 tx=0).stop 2024-09-13T17:54:54.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffacb59e700 1 -- 172.21.15.40:0/2628517353 shutdown_connections 2024-09-13T17:54:54.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffacb59e700 1 --2- 172.21.15.40:0/2628517353 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffacc0666e0 0x7ffacc150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffacb59e700 1 --2- 172.21.15.40:0/2628517353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffacc066300 0x7ffacc0c9be0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffacb59e700 1 -- 172.21.15.40:0/2628517353 >> 172.21.15.40:0/2628517353 conn(0x7ffacc0c8fc0 msgr2=0x7ffacc06cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:54.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.772+0000 7ffacb59e700 1 -- 172.21.15.40:0/2628517353 shutdown_connections 2024-09-13T17:54:54.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffacb59e700 1 -- 172.21.15.40:0/2628517353 wait complete. 2024-09-13T17:54:54.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffacb59e700 1 Processor -- start 2024-09-13T17:54:54.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffacb59e700 1 -- start start 2024-09-13T17:54:54.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffacb59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffacc066300 0x7ffacc14bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffacb59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffacc0666e0 0x7ffacc147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffacb59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffacc0b1ad0 0x7ffacc071d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffacb59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc146690 con 0x7ffacc066300 2024-09-13T17:54:54.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffacb59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc147380 con 0x7ffacc0666e0 2024-09-13T17:54:54.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffacb59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc147d50 con 0x7ffacc0b1ad0 2024-09-13T17:54:54.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffaca59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffacc066300 0x7ffacc14bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffaca59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffacc066300 0x7ffacc14bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37706/0 (socket says 172.21.15.40:37706) 2024-09-13T17:54:54.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffaca59c700 1 -- 172.21.15.40:0/3397798310 learned_addr learned my addr 172.21.15.40:0/3397798310 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:54:54.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffacad9d700 1 --2- 172.21.15.40:0/3397798310 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffacc0b1ad0 0x7ffacc071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffac3fff700 1 --2- 172.21.15.40:0/3397798310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffacc0666e0 0x7ffacc147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffaca59c700 1 -- 172.21.15.40:0/3397798310 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffacc0b1ad0 msgr2=0x7ffacc071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffaca59c700 1 --2- 172.21.15.40:0/3397798310 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffacc0b1ad0 0x7ffacc071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffaca59c700 1 -- 172.21.15.40:0/3397798310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffacc0666e0 msgr2=0x7ffacc147810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:54.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffaca59c700 1 --2- 172.21.15.40:0/3397798310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffacc0666e0 0x7ffacc147810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:54.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffaca59c700 1 -- 172.21.15.40:0/3397798310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc0094b0 con 0x7ffacc066300 2024-09-13T17:54:54.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffac3fff700 1 --2- 172.21.15.40:0/3397798310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffacc0666e0 0x7ffacc147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:54:54.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.776+0000 7ffaca59c700 1 --2- 172.21.15.40:0/3397798310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffacc066300 0x7ffacc14bb10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0c9830 tx=0x7ffabc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:54.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.780+0000 7ffac2ffd700 1 -- 172.21.15.40:0/3397798310 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc004480 con 0x7ffacc066300 2024-09-13T17:54:54.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.780+0000 7ffac2ffd700 1 -- 172.21.15.40:0/3397798310 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffabc01f070 con 0x7ffacc066300 2024-09-13T17:54:54.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.780+0000 7ffacb59e700 1 -- 172.21.15.40:0/3397798310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffacc1484f0 con 0x7ffacc066300 2024-09-13T17:54:54.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.780+0000 7ffac2ffd700 1 -- 172.21.15.40:0/3397798310 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc0038c0 con 0x7ffacc066300 2024-09-13T17:54:54.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.780+0000 7ffacb59e700 1 -- 172.21.15.40:0/3397798310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffacc14a2f0 con 0x7ffacc066300 2024-09-13T17:54:54.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.780+0000 7ffacb59e700 1 -- 172.21.15.40:0/3397798310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffacc0c9be0 con 0x7ffacc066300 2024-09-13T17:54:54.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.780+0000 7ffac2ffd700 1 -- 172.21.15.40:0/3397798310 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffabc021030 con 0x7ffacc066300 2024-09-13T17:54:54.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.780+0000 7ffac2ffd700 1 --2- 172.21.15.40:0/3397798310 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffaac066730 0x7ffaac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:54:54.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.780+0000 7ffac2ffd700 1 -- 172.21.15.40:0/3397798310 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ffabc085570 con 0x7ffacc066300 2024-09-13T17:54:54.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.780+0000 7ffac3fff700 1 --2- 172.21.15.40:0/3397798310 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffaac066730 0x7ffaac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:54:54.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.780+0000 7ffac3fff700 1 --2- 172.21.15.40:0/3397798310 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffaac066730 0x7ffaac068bf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0726d0 tx=0x7ffab4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:54:54.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:54.788+0000 7ffac2ffd700 1 -- 172.21.15.40:0/3397798310 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ffabc052520 con 0x7ffacc066300 2024-09-13T17:54:55.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:55.056+0000 7ffacb59e700 1 -- 172.21.15.40:0/3397798310 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7ffacc1552f0 con 0x7ffacc066300 2024-09-13T17:54:55.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:55.056+0000 7ffac2ffd700 1 -- 172.21.15.40:0/3397798310 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7ffabc055b40 con 0x7ffacc066300 2024-09-13T17:54:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:55.060+0000 7ffacb59e700 1 -- 172.21.15.40:0/3397798310 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffaac066730 msgr2=0x7ffaac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:55.060+0000 7ffacb59e700 1 --2- 172.21.15.40:0/3397798310 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffaac066730 0x7ffaac068bf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0726d0 tx=0x7ffab4008040 comp rx=0 tx=0).stop 2024-09-13T17:54:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:55.060+0000 7ffacb59e700 1 -- 172.21.15.40:0/3397798310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffacc066300 msgr2=0x7ffacc14bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:54:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:55.060+0000 7ffacb59e700 1 --2- 172.21.15.40:0/3397798310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffacc066300 0x7ffacc14bb10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffacc0c9830 tx=0x7ffabc004770 comp rx=0 tx=0).stop 2024-09-13T17:54:55.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:55.060+0000 7ffacb59e700 1 -- 172.21.15.40:0/3397798310 shutdown_connections 2024-09-13T17:54:55.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:55.060+0000 7ffacb59e700 1 --2- 172.21.15.40:0/3397798310 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffaac066730 0x7ffaac068bf0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:55.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:55.060+0000 7ffacb59e700 1 --2- 172.21.15.40:0/3397798310 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffacc0b1ad0 0x7ffacc071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:55.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:55.060+0000 7ffacb59e700 1 --2- 172.21.15.40:0/3397798310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffacc0666e0 0x7ffacc147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:55.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:55.060+0000 7ffacb59e700 1 --2- 172.21.15.40:0/3397798310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffacc066300 0x7ffacc14bb10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:54:55.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:55.060+0000 7ffacb59e700 1 -- 172.21.15.40:0/3397798310 >> 172.21.15.40:0/3397798310 conn(0x7ffacc0c8fc0 msgr2=0x7ffacc06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:54:55.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:55.060+0000 7ffacb59e700 1 -- 172.21.15.40:0/3397798310 shutdown_connections 2024-09-13T17:54:55.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:54:55.060+0000 7ffacb59e700 1 -- 172.21.15.40:0/3397798310 wait complete. 2024-09-13T17:54:55.082 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T17:55:04.294 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:55:04.297 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:55:05.083 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T17:55:05.083 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:55:05.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa770b68700 1 Processor -- start 2024-09-13T17:55:05.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa770b68700 1 -- start start 2024-09-13T17:55:05.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa770b68700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa76c0c3d80 0x7fa76c06b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa770b68700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa76c0c7880 0x7fa76c06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa770b68700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa76c06f7d0 con 0x7fa76c153870 2024-09-13T17:55:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa770b68700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa76c06e6b0 con 0x7fa76c0c7880 2024-09-13T17:55:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa770b68700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa76c0c1ea0 con 0x7fa76c0c3d80 2024-09-13T17:55:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa76bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa76c0c7880 0x7fa76c06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa76bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa76c0c7880 0x7fa76c06b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39898/0 (socket says 172.21.15.40:39898) 2024-09-13T17:55:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa76bfff700 1 -- 172.21.15.40:0/2267325080 learned_addr learned my addr 172.21.15.40:0/2267325080 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa76affd700 1 --2- 172.21.15.40:0/2267325080 >> v2:172.21.15.43:3300/0 conn(0x7fa76c0c3d80 0x7fa76c06b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa76bfff700 1 -- 172.21.15.40:0/2267325080 >> v2:172.21.15.43:3300/0 conn(0x7fa76c0c3d80 msgr2=0x7fa76c06b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:55:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa76bfff700 1 --2- 172.21.15.40:0/2267325080 >> v2:172.21.15.43:3300/0 conn(0x7fa76c0c3d80 0x7fa76c06b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa76bfff700 1 -- 172.21.15.40:0/2267325080 >> v1:172.21.15.43:6789/0 conn(0x7fa76c153870 legacy=0x7fa76c06fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:55:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa76bfff700 1 -- 172.21.15.40:0/2267325080 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76c06e130 con 0x7fa76c0c7880 2024-09-13T17:55:05.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa76affd700 1 --2- 172.21.15.40:0/2267325080 >> v2:172.21.15.43:3300/0 conn(0x7fa76c0c3d80 0x7fa76c06b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:55:05.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa76bfff700 1 --2- 172.21.15.40:0/2267325080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa76c0c7880 0x7fa76c06b9e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0cdf40 tx=0x7fa76400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e176093892db53e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:05.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa76a7fc700 1 -- 172.21.15.40:0/2267325080 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76400e070 con 0x7fa76c0c7880 2024-09-13T17:55:05.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa76a7fc700 1 -- 172.21.15.40:0/2267325080 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa764003a40 con 0x7fa76c0c7880 2024-09-13T17:55:05.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa76a7fc700 1 -- 172.21.15.40:0/2267325080 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764003df0 con 0x7fa76c0c7880 2024-09-13T17:55:05.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa770b68700 1 -- 172.21.15.40:0/2267325080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa76c0c7880 msgr2=0x7fa76c06b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa770b68700 1 --2- 172.21.15.40:0/2267325080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa76c0c7880 0x7fa76c06b9e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0cdf40 tx=0x7fa76400b330 comp rx=0 tx=0).stop 2024-09-13T17:55:05.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa770b68700 1 -- 172.21.15.40:0/2267325080 shutdown_connections 2024-09-13T17:55:05.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa770b68700 1 --2- 172.21.15.40:0/2267325080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa76c0c7880 0x7fa76c06b9e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa770b68700 1 --2- 172.21.15.40:0/2267325080 >> v2:172.21.15.43:3300/0 conn(0x7fa76c0c3d80 0x7fa76c06b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa770b68700 1 -- 172.21.15.40:0/2267325080 >> 172.21.15.40:0/2267325080 conn(0x7fa76c066090 msgr2=0x7fa76c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:05.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa770b68700 1 -- 172.21.15.40:0/2267325080 shutdown_connections 2024-09-13T17:55:05.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.196+0000 7fa770b68700 1 -- 172.21.15.40:0/2267325080 wait complete. 2024-09-13T17:55:05.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.200+0000 7fa770b68700 1 Processor -- start 2024-09-13T17:55:05.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.200+0000 7fa770b68700 1 -- start start 2024-09-13T17:55:05.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.200+0000 7fa770b68700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa76c0c3d80 0x7fa76c1f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.200+0000 7fa770b68700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa76c0c7880 0x7fa76c1557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.200+0000 7fa770b68700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa76c153870 0x7fa76c0c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.200+0000 7fa770b68700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c155070 con 0x7fa76c0c7880 2024-09-13T17:55:05.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.200+0000 7fa770b68700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c0c6940 con 0x7fa76c153870 2024-09-13T17:55:05.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.200+0000 7fa770b68700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c14a070 con 0x7fa76c0c3d80 2024-09-13T17:55:05.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.204+0000 7fa76affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa76c0c7880 0x7fa76c1557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.204+0000 7fa76affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa76c0c7880 0x7fa76c1557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39908/0 (socket says 172.21.15.40:39908) 2024-09-13T17:55:05.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.204+0000 7fa76affd700 1 -- 172.21.15.40:0/178388992 learned_addr learned my addr 172.21.15.40:0/178388992 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:05.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.204+0000 7fa76b7fe700 1 --2- 172.21.15.40:0/178388992 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa76c0c3d80 0x7fa76c1f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.204+0000 7fa76bfff700 1 --2- 172.21.15.40:0/178388992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa76c153870 0x7fa76c0c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.204+0000 7fa76affd700 1 -- 172.21.15.40:0/178388992 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa76c0c3d80 msgr2=0x7fa76c1f23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.204+0000 7fa76affd700 1 --2- 172.21.15.40:0/178388992 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa76c0c3d80 0x7fa76c1f23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.204+0000 7fa76affd700 1 -- 172.21.15.40:0/178388992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa76c153870 msgr2=0x7fa76c0c5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.204+0000 7fa76affd700 1 --2- 172.21.15.40:0/178388992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa76c153870 0x7fa76c0c5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.204+0000 7fa76affd700 1 -- 172.21.15.40:0/178388992 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa764009b30 con 0x7fa76c0c7880 2024-09-13T17:55:05.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.204+0000 7fa76affd700 1 --2- 172.21.15.40:0/178388992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa76c0c7880 0x7fa76c1557e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa76000c770 tx=0x7fa76000ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:05.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.204+0000 7fa753fff700 1 -- 172.21.15.40:0/178388992 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760011800 con 0x7fa76c0c7880 2024-09-13T17:55:05.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.204+0000 7fa753fff700 1 -- 172.21.15.40:0/178388992 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa760011960 con 0x7fa76c0c7880 2024-09-13T17:55:05.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.204+0000 7fa770b68700 1 -- 172.21.15.40:0/178388992 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa76c0c6bb0 con 0x7fa76c0c7880 2024-09-13T17:55:05.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.204+0000 7fa753fff700 1 -- 172.21.15.40:0/178388992 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760011d10 con 0x7fa76c0c7880 2024-09-13T17:55:05.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.204+0000 7fa770b68700 1 -- 172.21.15.40:0/178388992 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa76c0c7170 con 0x7fa76c0c7880 2024-09-13T17:55:05.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.204+0000 7fa770b68700 1 -- 172.21.15.40:0/178388992 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa76c154e50 con 0x7fa76c0c7880 2024-09-13T17:55:05.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.204+0000 7fa753fff700 1 -- 172.21.15.40:0/178388992 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa760018360 con 0x7fa76c0c7880 2024-09-13T17:55:05.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.208+0000 7fa753fff700 1 --2- 172.21.15.40:0/178388992 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa758066730 0x7fa758068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.208+0000 7fa753fff700 1 -- 172.21.15.40:0/178388992 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa7600844f0 con 0x7fa76c0c7880 2024-09-13T17:55:05.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.208+0000 7fa76b7fe700 1 --2- 172.21.15.40:0/178388992 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa758066730 0x7fa758068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.208+0000 7fa76b7fe700 1 --2- 172.21.15.40:0/178388992 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa758066730 0x7fa758068bf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa75c009980 tx=0x7fa75c0090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:05.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.212+0000 7fa753fff700 1 -- 172.21.15.40:0/178388992 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa760051410 con 0x7fa76c0c7880 2024-09-13T17:55:05.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.484+0000 7fa770b68700 1 -- 172.21.15.40:0/178388992 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa76c06ec40 con 0x7fa76c0c7880 2024-09-13T17:55:05.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.484+0000 7fa753fff700 1 -- 172.21.15.40:0/178388992 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fa76001f620 con 0x7fa76c0c7880 2024-09-13T17:55:05.491 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:55:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.488+0000 7fa770b68700 1 -- 172.21.15.40:0/178388992 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa758066730 msgr2=0x7fa758068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.488+0000 7fa770b68700 1 --2- 172.21.15.40:0/178388992 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa758066730 0x7fa758068bf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa75c009980 tx=0x7fa75c0090e0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.488+0000 7fa770b68700 1 -- 172.21.15.40:0/178388992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa76c0c7880 msgr2=0x7fa76c1557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.488+0000 7fa770b68700 1 --2- 172.21.15.40:0/178388992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa76c0c7880 0x7fa76c1557e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa76000c770 tx=0x7fa76000ca80 comp rx=0 tx=0).stop 2024-09-13T17:55:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.488+0000 7fa770b68700 1 -- 172.21.15.40:0/178388992 shutdown_connections 2024-09-13T17:55:05.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.488+0000 7fa770b68700 1 --2- 172.21.15.40:0/178388992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa76c153870 0x7fa76c0c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.488+0000 7fa770b68700 1 --2- 172.21.15.40:0/178388992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa76c0c7880 0x7fa76c1557e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.488+0000 7fa770b68700 1 --2- 172.21.15.40:0/178388992 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa758066730 0x7fa758068bf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.488+0000 7fa770b68700 1 --2- 172.21.15.40:0/178388992 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa76c0c3d80 0x7fa76c1f23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.488+0000 7fa770b68700 1 -- 172.21.15.40:0/178388992 >> 172.21.15.40:0/178388992 conn(0x7fa76c066090 msgr2=0x7fa76c06b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:05.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.488+0000 7fa770b68700 1 -- 172.21.15.40:0/178388992 shutdown_connections 2024-09-13T17:55:05.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.488+0000 7fa770b68700 1 -- 172.21.15.40:0/178388992 wait complete. 2024-09-13T17:55:05.507 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T17:55:05.507 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T17:55:05.507 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T17:55:05.507 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:55:05.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.636+0000 7f3281ddc700 1 Processor -- start 2024-09-13T17:55:05.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.636+0000 7f3281ddc700 1 -- start start 2024-09-13T17:55:05.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.636+0000 7f3281ddc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f32740d87a0 0x7f32740df110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.636+0000 7f3281ddc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f32740469c0 0x7f3274046e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.636+0000 7f3281ddc700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3274047610 con 0x7f3274007b60 2024-09-13T17:55:05.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.636+0000 7f3281ddc700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f327404bef0 con 0x7f32740d87a0 2024-09-13T17:55:05.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.636+0000 7f3281ddc700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3274006b50 con 0x7f32740469c0 2024-09-13T17:55:05.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f327bfff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3274007b60 0x7f327404b7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56932/0 (socket says 172.21.15.40:56932) 2024-09-13T17:55:05.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f327bfff700 1 -- 172.21.15.40:0/3842645831 learned_addr learned my addr 172.21.15.40:0/3842645831 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:55:05.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f32815db700 1 --2- 172.21.15.40:0/3842645831 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f32740469c0 0x7f3274046e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f3280dda700 1 --2- 172.21.15.40:0/3842645831 >> v2:172.21.15.43:3300/0 conn(0x7f32740d87a0 0x7f32740df110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f327b7fe700 1 -- 172.21.15.40:0/3842645831 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4056203992 0 0) 0x7f3274047610 con 0x7f3274007b60 2024-09-13T17:55:05.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f327b7fe700 1 -- 172.21.15.40:0/3842645831 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f326c0035d0 con 0x7f3274007b60 2024-09-13T17:55:05.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f3280dda700 1 -- 172.21.15.40:0/3842645831 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f32740469c0 msgr2=0x7f3274046e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:55:05.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f3280dda700 1 --2- 172.21.15.40:0/3842645831 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f32740469c0 0x7f3274046e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f3280dda700 1 -- 172.21.15.40:0/3842645831 >> v1:172.21.15.40:6789/0 conn(0x7f3274007b60 legacy=0x7f327404b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f3280dda700 1 -- 172.21.15.40:0/3842645831 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32740076d0 con 0x7f32740d87a0 2024-09-13T17:55:05.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f32815db700 1 --2- 172.21.15.40:0/3842645831 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f32740469c0 0x7f3274046e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:55:05.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f3280dda700 1 --2- 172.21.15.40:0/3842645831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32740d87a0 0x7f32740df110 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3270009850 tx=0x7f3270009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5052dd0d9f3a63a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:05.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f327b7fe700 1 -- 172.21.15.40:0/3842645831 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3270011830 con 0x7f32740d87a0 2024-09-13T17:55:05.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f327b7fe700 1 -- 172.21.15.40:0/3842645831 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3270011990 con 0x7f32740d87a0 2024-09-13T17:55:05.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f327b7fe700 1 -- 172.21.15.40:0/3842645831 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3270011d20 con 0x7f32740d87a0 2024-09-13T17:55:05.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f3281ddc700 1 -- 172.21.15.40:0/3842645831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32740d87a0 msgr2=0x7f32740df110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f3281ddc700 1 --2- 172.21.15.40:0/3842645831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32740d87a0 0x7f32740df110 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3270009850 tx=0x7f3270009b60 comp rx=0 tx=0).stop 2024-09-13T17:55:05.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f3281ddc700 1 -- 172.21.15.40:0/3842645831 shutdown_connections 2024-09-13T17:55:05.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f3281ddc700 1 --2- 172.21.15.40:0/3842645831 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f32740469c0 0x7f3274046e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f3281ddc700 1 --2- 172.21.15.40:0/3842645831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32740d87a0 0x7f32740df110 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f3281ddc700 1 -- 172.21.15.40:0/3842645831 >> 172.21.15.40:0/3842645831 conn(0x7f3274050e60 msgr2=0x7f3274051270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:05.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f3281ddc700 1 -- 172.21.15.40:0/3842645831 shutdown_connections 2024-09-13T17:55:05.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f3281ddc700 1 -- 172.21.15.40:0/3842645831 wait complete. 2024-09-13T17:55:05.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f3281ddc700 1 Processor -- start 2024-09-13T17:55:05.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.640+0000 7f3281ddc700 1 -- start start 2024-09-13T17:55:05.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f3281ddc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32740469c0 0x7f32740d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f3281ddc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32740d87a0 0x7f32740d9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f3281ddc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f32740ced00 0x7f32740d1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f3281ddc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32740d9bc0 con 0x7f32740469c0 2024-09-13T17:55:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f3281ddc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32740d9030 con 0x7f32740d87a0 2024-09-13T17:55:05.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f3281ddc700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32740d0370 con 0x7f32740ced00 2024-09-13T17:55:05.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f3280dda700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32740469c0 0x7f32740d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f3280dda700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32740469c0 0x7f32740d04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39918/0 (socket says 172.21.15.40:39918) 2024-09-13T17:55:05.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f3280dda700 1 -- 172.21.15.40:0/1461414743 learned_addr learned my addr 172.21.15.40:0/1461414743 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:05.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f327bfff700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32740d87a0 0x7f32740d9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f32815db700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f32740ced00 0x7f32740d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f3280dda700 1 -- 172.21.15.40:0/1461414743 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f32740ced00 msgr2=0x7f32740d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f3280dda700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f32740ced00 0x7f32740d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f3280dda700 1 -- 172.21.15.40:0/1461414743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32740d87a0 msgr2=0x7f32740d9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f3280dda700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32740d87a0 0x7f32740d9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f3280dda700 1 -- 172.21.15.40:0/1461414743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c04f5c0 con 0x7f32740469c0 2024-09-13T17:55:05.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f3280dda700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32740469c0 0x7f32740d04f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3270009eb0 tx=0x7f3270004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:05.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f32797fa700 1 -- 172.21.15.40:0/1461414743 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3270006ea0 con 0x7f32740469c0 2024-09-13T17:55:05.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f32797fa700 1 -- 172.21.15.40:0/1461414743 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327001f070 con 0x7f32740469c0 2024-09-13T17:55:05.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f32797fa700 1 -- 172.21.15.40:0/1461414743 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32700036c0 con 0x7f32740469c0 2024-09-13T17:55:05.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f3281ddc700 1 -- 172.21.15.40:0/1461414743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32700094b0 con 0x7f32740469c0 2024-09-13T17:55:05.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f3281ddc700 1 -- 172.21.15.40:0/1461414743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327404a7b0 con 0x7f32740469c0 2024-09-13T17:55:05.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f3281ddc700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3274007b60 0x7f32740d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f3281ddc700 1 -- 172.21.15.40:0/1461414743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32740095f0 con 0x7f3274007b60 2024-09-13T17:55:05.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.644+0000 7f327bfff700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3274007b60 0x7f32740d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.648+0000 7f327bfff700 1 -- 172.21.15.40:0/1461414743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f32740df650 con 0x7f3274007b60 2024-09-13T17:55:05.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.648+0000 7f32797fa700 1 -- 172.21.15.40:0/1461414743 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3270003840 con 0x7f32740469c0 2024-09-13T17:55:05.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.648+0000 7f327bfff700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3274007b60 0x7f32740d1ac0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f32740d7d40 tx=0x7f32680018b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:05.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.648+0000 7f32797fa700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f32600667b0 0x7f3260068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.648+0000 7f32797fa700 1 -- 172.21.15.40:0/1461414743 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3270021070 con 0x7f32740469c0 2024-09-13T17:55:05.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.648+0000 7f32815db700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f32600667b0 0x7f3260068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.648+0000 7f32797fa700 1 -- 172.21.15.40:0/1461414743 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3268012620 con 0x7f3274007b60 2024-09-13T17:55:05.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.648+0000 7f32797fa700 1 -- 172.21.15.40:0/1461414743 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3268012780 con 0x7f3274007b60 2024-09-13T17:55:05.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.648+0000 7f32797fa700 1 -- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3274007b60 msgr2=0x7f32740d1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.648+0000 7f32797fa700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3274007b60 0x7f32740d1ac0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f32740d7d40 tx=0x7f32680018b0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.648+0000 7f32797fa700 1 -- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3274007b60 msgr2=0x7f32740d1ac0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:55:05.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.648+0000 7f32797fa700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3274007b60 0x7f32740d1ac0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.648+0000 7f32815db700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f32600667b0 0x7f3260068c70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f327c0641b0 tx=0x7f327c06e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:05.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.656+0000 7f3281ddc700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f327404a9a0 0x7f32740cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.656+0000 7f3281ddc700 1 -- 172.21.15.40:0/1461414743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32740d8b80 con 0x7f327404a9a0 2024-09-13T17:55:05.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.656+0000 7f327bfff700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f327404a9a0 0x7f32740cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.656+0000 7f327bfff700 1 -- 172.21.15.40:0/1461414743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3274006da0 con 0x7f327404a9a0 2024-09-13T17:55:05.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.656+0000 7f327bfff700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f327404a9a0 0x7f32740cf4c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3274051f40 tx=0x7f3268001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:05.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.656+0000 7f32797fa700 1 -- 172.21.15.40:0/1461414743 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3268022070 con 0x7f327404a9a0 2024-09-13T17:55:05.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.656+0000 7f32797fa700 1 -- 172.21.15.40:0/1461414743 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4950 (secure 0 0 0) 0x7f3268004aa0 con 0x7f327404a9a0 2024-09-13T17:55:05.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.656+0000 7f32797fa700 1 -- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f327404a9a0 msgr2=0x7f32740cf4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.656+0000 7f32797fa700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f327404a9a0 0x7f32740cf4c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3274051f40 tx=0x7f3268001150 comp rx=0 tx=0).stop 2024-09-13T17:55:05.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.656+0000 7f32797fa700 1 -- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f327404a9a0 msgr2=0x7f32740cf4c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:55:05.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.656+0000 7f32797fa700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f327404a9a0 0x7f32740cf4c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.671 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:55:05.671 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:55:05.671 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:55:05.671 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:55:05.671 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 40, 2024-09-13T17:55:05.672 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:55:05.672 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:55:05.672 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:55:05.672 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:55:05.672 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:05.673 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 13, 2024-09-13T17:55:05.673 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:55:05.673 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:55:05.673 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:55:05.673 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:55:05.673 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:55:05.674 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:55:05.674 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:55:05.674 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:55:05.674 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:55:05.674 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:55:05.674 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:55:05.675 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:55:05.675 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:55:05.675 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:05.675 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:55:05.675 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:55:05.675 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:55:05.676 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:55:05.676 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:55:05.676 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:55:05.676 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:55:05.676 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:55:05.677 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:55:05.677 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:55:05.677 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:55:05.677 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:55:05.677 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:05.677 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.678 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:55:05.678 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:55:05.678 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:55:05.678 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:55:05.678 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:55:05.678 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:55:05.679 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:55:05.679 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:55:05.679 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:55:05.679 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:55:05.679 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:55:05.680 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:55:05.680 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:55:05.681 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:55:05.681 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:55:05.681 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:55:05.681 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:55:05.681 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:55:05.681 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:55:05.681 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:55:05.682 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:55:05.682 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:55:05.682 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:55:05.682 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:55:05.682 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:55:05.683 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:55:05.683 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:55:05.683 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:05.683 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:55:05.683 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.683 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:55:05.684 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.684 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:55:05.684 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:55:05.684 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:55:05.684 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:55:05.684 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.685 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:55:05.685 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:55:05.685 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:05.685 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.685 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.685 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:55:05.685 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:55:05.686 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:05.686 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:05.686 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:05.686 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.686 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:55:05.686 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:55:05.686 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:55:05.686 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:55:05.686 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:55:05.687 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.687 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.687 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:55:05.687 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:55:05.687 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:55:05.687 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:55:05.687 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.687 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:55:05.687 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:55:05.687 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:05.687 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.687 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.687 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:55:05.687 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:55:05.687 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:05.688 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:05.688 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:05.688 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.688 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:55:05.688 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:55:05.688 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:55:05.688 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:55:05.688 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:55:05.688 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.688 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.688 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:55:05.688 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:55:05.688 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:55:05.689 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:55:05.689 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.689 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:55:05.689 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:55:05.689 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:05.689 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.689 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.689 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:55:05.689 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:55:05.689 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:05.689 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:05.689 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:05.689 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.689 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:55:05.690 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:55:05.690 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:55:05.690 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:55:05.690 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:55:05.690 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:05.690 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:05.690 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.690 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:55:05.690 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:55:05.690 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.690 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:55:05.690 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:55:05.690 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:55:05.690 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:05.691 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:05.691 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:55:05.691 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.691 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:55:05.691 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:55:05.691 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:55:05.691 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:05.691 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:05.691 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:55:05.691 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.691 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:55:05.691 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:55:05.691 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:55:05.691 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:05.692 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:05.692 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:55:05.692 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.692 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:55:05.692 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:55:05.692 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:55:05.692 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:05.692 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:05.692 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:55:05.692 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.692 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:55:05.692 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:55:05.692 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:55:05.693 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:05.693 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:05.693 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.693 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:55:05.693 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:55:05.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.660+0000 7f3281ddc700 1 -- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f32600667b0 msgr2=0x7f3260068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.660+0000 7f3281ddc700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f32600667b0 0x7f3260068c70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f327c0641b0 tx=0x7f327c06e040 comp rx=0 tx=0).stop 2024-09-13T17:55:05.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.660+0000 7f3281ddc700 1 -- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32740469c0 msgr2=0x7f32740d04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.660+0000 7f32815db700 1 -- 172.21.15.40:0/1461414743 reap_dead start 2024-09-13T17:55:05.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.660+0000 7f3281ddc700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32740469c0 0x7f32740d04f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3270009eb0 tx=0x7f3270004770 comp rx=0 tx=0).stop 2024-09-13T17:55:05.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.660+0000 7f3281ddc700 1 -- 172.21.15.40:0/1461414743 shutdown_connections 2024-09-13T17:55:05.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.660+0000 7f3281ddc700 1 --2- 172.21.15.40:0/1461414743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32740469c0 0x7f32740d04f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.660+0000 7f3281ddc700 1 -- 172.21.15.40:0/1461414743 >> 172.21.15.40:0/1461414743 conn(0x7f3274050e60 msgr2=0x7f32740d2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:05.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.660+0000 7f3281ddc700 1 -- 172.21.15.40:0/1461414743 shutdown_connections 2024-09-13T17:55:05.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.660+0000 7f3281ddc700 1 -- 172.21.15.40:0/1461414743 wait complete. 2024-09-13T17:55:05.696 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:55:05.697 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T17:55:05.697 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:55:05.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.824+0000 7fad4f266700 1 Processor -- start 2024-09-13T17:55:05.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.824+0000 7fad4f266700 1 -- start start 2024-09-13T17:55:05.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4f266700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad4814fa30 0x7fad48158e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4f266700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad48159ca0 con 0x7fad480c6c40 2024-09-13T17:55:05.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4f266700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad48159400 con 0x7fad4814fe10 2024-09-13T17:55:05.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4f266700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad48072100 con 0x7fad4814fa30 2024-09-13T17:55:05.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4ea65700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fad480c6c40 0x7fad480c7bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56940/0 (socket says 172.21.15.40:56940) 2024-09-13T17:55:05.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4e264700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad4814fa30 0x7fad48158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4da63700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fad4814fe10 0x7fad481561a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:34348/0 (socket says 172.21.15.40:34348) 2024-09-13T17:55:05.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4ea65700 1 -- 172.21.15.40:0/3018766682 learned_addr learned my addr 172.21.15.40:0/3018766682 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:55:05.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4d262700 1 -- 172.21.15.40:0/3018766682 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678815530 0 0) 0x7fad48159ca0 con 0x7fad480c6c40 2024-09-13T17:55:05.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4d262700 1 -- 172.21.15.40:0/3018766682 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad340035d0 con 0x7fad480c6c40 2024-09-13T17:55:05.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4d262700 1 -- 172.21.15.40:0/3018766682 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3119434459 0 0) 0x7fad48159400 con 0x7fad4814fe10 2024-09-13T17:55:05.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4d262700 1 -- 172.21.15.40:0/3018766682 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad48159ca0 con 0x7fad4814fe10 2024-09-13T17:55:05.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4d262700 1 -- 172.21.15.40:0/3018766682 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2505949688 0 0) 0x7fad340035d0 con 0x7fad480c6c40 2024-09-13T17:55:05.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4d262700 1 -- 172.21.15.40:0/3018766682 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad4814fa30 msgr2=0x7fad48158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:55:05.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4d262700 1 --2- 172.21.15.40:0/3018766682 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad4814fa30 0x7fad48158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4d262700 1 -- 172.21.15.40:0/3018766682 >> v1:172.21.15.43:6789/0 conn(0x7fad4814fe10 legacy=0x7fad481561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4d262700 1 -- 172.21.15.40:0/3018766682 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad481596e0 con 0x7fad480c6c40 2024-09-13T17:55:05.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4d262700 1 -- 172.21.15.40:0/3018766682 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fad400019f0 con 0x7fad480c6c40 2024-09-13T17:55:05.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4d262700 1 -- 172.21.15.40:0/3018766682 >> v1:172.21.15.40:6789/0 conn(0x7fad480c6c40 legacy=0x7fad480c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4d262700 1 --2- 172.21.15.40:0/3018766682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34004c40 0x7fad34005040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4d262700 1 --2- 172.21.15.40:0/3018766682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad34003810 0x7fad34003c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4d262700 1 --2- 172.21.15.40:0/3018766682 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad4814fa30 0x7fad340041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4d262700 1 -- 172.21.15.40:0/3018766682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad48072100 con 0x7fad34004c40 2024-09-13T17:55:05.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4d262700 1 -- 172.21.15.40:0/3018766682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3400bae0 con 0x7fad34003810 2024-09-13T17:55:05.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4d262700 1 -- 172.21.15.40:0/3018766682 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3400bc70 con 0x7fad4814fa30 2024-09-13T17:55:05.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4e264700 1 --2- 172.21.15.40:0/3018766682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34004c40 0x7fad34005040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4da63700 1 --2- 172.21.15.40:0/3018766682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad34003810 0x7fad34003c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4e264700 1 -- 172.21.15.40:0/3018766682 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad4814fa30 msgr2=0x7fad340041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:55:05.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4e264700 1 --2- 172.21.15.40:0/3018766682 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad4814fa30 0x7fad340041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4e264700 1 -- 172.21.15.40:0/3018766682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad34003810 msgr2=0x7fad34003c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:55:05.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4e264700 1 --2- 172.21.15.40:0/3018766682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad34003810 0x7fad34003c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4e264700 1 -- 172.21.15.40:0/3018766682 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad340035d0 con 0x7fad34004c40 2024-09-13T17:55:05.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4e264700 1 --2- 172.21.15.40:0/3018766682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34004c40 0x7fad34005040 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fad4400ab70 tx=0x7fad340058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90d0fc90cde2c4a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:05.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4d262700 1 -- 172.21.15.40:0/3018766682 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4400f9d0 con 0x7fad34004c40 2024-09-13T17:55:05.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4d262700 1 -- 172.21.15.40:0/3018766682 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4400fb30 con 0x7fad34004c40 2024-09-13T17:55:05.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.828+0000 7fad4d262700 1 -- 172.21.15.40:0/3018766682 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4400fea0 con 0x7fad34004c40 2024-09-13T17:55:05.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.832+0000 7fad4f266700 1 -- 172.21.15.40:0/3018766682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34004c40 msgr2=0x7fad34005040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.832+0000 7fad4f266700 1 --2- 172.21.15.40:0/3018766682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34004c40 0x7fad34005040 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fad4400ab70 tx=0x7fad340058a0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.832+0000 7fad4ea65700 1 -- 172.21.15.40:0/3018766682 reap_dead start 2024-09-13T17:55:05.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.832+0000 7fad4f266700 1 -- 172.21.15.40:0/3018766682 shutdown_connections 2024-09-13T17:55:05.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.832+0000 7fad4f266700 1 -- 172.21.15.40:0/3018766682 >> 172.21.15.40:0/3018766682 conn(0x7fad4806a390 msgr2=0x7fad4806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:05.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.832+0000 7fad4f266700 1 -- 172.21.15.40:0/3018766682 shutdown_connections 2024-09-13T17:55:05.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.832+0000 7fad4f266700 1 -- 172.21.15.40:0/3018766682 wait complete. 2024-09-13T17:55:05.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.832+0000 7fad4f266700 1 Processor -- start 2024-09-13T17:55:05.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.832+0000 7fad4f266700 1 -- start start 2024-09-13T17:55:05.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.832+0000 7fad4f266700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34004c40 0x7fad480c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.832+0000 7fad4f266700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad480c1dd0 0x7fad481ec7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.832+0000 7fad4f266700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad481ecd20 0x7fad4814bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.832+0000 7fad4f266700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4814d420 con 0x7fad34004c40 2024-09-13T17:55:05.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.832+0000 7fad4f266700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad48152d60 con 0x7fad481ecd20 2024-09-13T17:55:05.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.832+0000 7fad4f266700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad48153040 con 0x7fad480c1dd0 2024-09-13T17:55:05.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad4e264700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34004c40 0x7fad480c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad4e264700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34004c40 0x7fad480c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39954/0 (socket says 172.21.15.40:39954) 2024-09-13T17:55:05.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad4e264700 1 -- 172.21.15.40:0/2876337104 learned_addr learned my addr 172.21.15.40:0/2876337104 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:05.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad4ea65700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad481ecd20 0x7fad4814bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad4da63700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad480c1dd0 0x7fad481ec7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad4e264700 1 -- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad480c1dd0 msgr2=0x7fad481ec7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad4e264700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad480c1dd0 0x7fad481ec7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad4e264700 1 -- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad481ecd20 msgr2=0x7fad4814bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad4e264700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad481ecd20 0x7fad4814bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad4e264700 1 -- 172.21.15.40:0/2876337104 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad480b87f0 con 0x7fad34004c40 2024-09-13T17:55:05.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad4da63700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad480c1dd0 0x7fad481ec7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:55:05.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad4e264700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34004c40 0x7fad480c7af0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fad34005810 tx=0x7fad440047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:05.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad3effd700 1 -- 172.21.15.40:0/2876337104 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad440038c0 con 0x7fad34004c40 2024-09-13T17:55:05.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad3effd700 1 -- 172.21.15.40:0/2876337104 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad44003a40 con 0x7fad34004c40 2024-09-13T17:55:05.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad4f266700 1 -- 172.21.15.40:0/2876337104 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad4814d420 con 0x7fad34004c40 2024-09-13T17:55:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad3effd700 1 -- 172.21.15.40:0/2876337104 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad44003df0 con 0x7fad34004c40 2024-09-13T17:55:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad4f266700 1 -- 172.21.15.40:0/2876337104 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad480b8db0 con 0x7fad34004c40 2024-09-13T17:55:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad4f266700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad4814c1a0 0x7fad481efd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad4f266700 1 -- 172.21.15.40:0/2876337104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad480b8980 con 0x7fad4814c1a0 2024-09-13T17:55:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad4da63700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad4814c1a0 0x7fad481efd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad4da63700 1 -- 172.21.15.40:0/2876337104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad480b8670 con 0x7fad4814c1a0 2024-09-13T17:55:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.836+0000 7fad4da63700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad4814c1a0 0x7fad481efd50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fad4814ed70 tx=0x7fad38001730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.840+0000 7fad3effd700 1 -- 172.21.15.40:0/2876337104 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fad4401b400 con 0x7fad34004c40 2024-09-13T17:55:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.840+0000 7fad3effd700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad2c06aae0 0x7fad2c06cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.840+0000 7fad3effd700 1 -- 172.21.15.40:0/2876337104 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fad440856b0 con 0x7fad34004c40 2024-09-13T17:55:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.840+0000 7fad4ea65700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad2c06aae0 0x7fad2c06cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.840+0000 7fad3effd700 1 -- 172.21.15.40:0/2876337104 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad38001cf0 con 0x7fad4814c1a0 2024-09-13T17:55:05.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.840+0000 7fad3effd700 1 -- 172.21.15.40:0/2876337104 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fad38013570 con 0x7fad4814c1a0 2024-09-13T17:55:05.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.840+0000 7fad3effd700 1 -- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad4814c1a0 msgr2=0x7fad481efd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.840+0000 7fad3effd700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad4814c1a0 0x7fad481efd50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fad4814ed70 tx=0x7fad38001730 comp rx=0 tx=0).stop 2024-09-13T17:55:05.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.840+0000 7fad3effd700 1 -- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad4814c1a0 msgr2=0x7fad481efd50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:55:05.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.840+0000 7fad3effd700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad4814c1a0 0x7fad481efd50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.840+0000 7fad4ea65700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad2c06aae0 0x7fad2c06cfa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fad3400be00 tx=0x7fad40001a00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.852+0000 7fad4f266700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad481f0290 0x7fad481f06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.852+0000 7fad4f266700 1 -- 172.21.15.40:0/2876337104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad48067600 con 0x7fad481f0290 2024-09-13T17:55:05.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.852+0000 7fad4da63700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad481f0290 0x7fad481f06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.852+0000 7fad4da63700 1 -- 172.21.15.40:0/2876337104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fad48155c80 con 0x7fad481f0290 2024-09-13T17:55:05.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.852+0000 7fad4da63700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad481f0290 0x7fad481f06a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fad4814ed70 tx=0x7fad38007b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:05.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.852+0000 7fad3effd700 1 -- 172.21.15.40:0/2876337104 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad38023070 con 0x7fad481f0290 2024-09-13T17:55:05.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.852+0000 7fad3effd700 1 -- 172.21.15.40:0/2876337104 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4450 (secure 0 0 0) 0x7fad38027410 con 0x7fad481f0290 2024-09-13T17:55:05.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.852+0000 7fad3effd700 1 -- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad481f0290 msgr2=0x7fad481f06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.852+0000 7fad3effd700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad481f0290 0x7fad481f06a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fad4814ed70 tx=0x7fad38007b50 comp rx=0 tx=0).stop 2024-09-13T17:55:05.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.852+0000 7fad3effd700 1 -- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad481f0290 msgr2=0x7fad481f06a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:55:05.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.852+0000 7fad3effd700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad481f0290 0x7fad481f06a0 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fad4814ed70 tx=0x7fad38007b50 comp rx=0 tx=0).stop 2024-09-13T17:55:05.860 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:55:05.861 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:55:05.861 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:55:05.861 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:55:05.861 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 40, 2024-09-13T17:55:05.861 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:55:05.861 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:55:05.861 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:55:05.861 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:55:05.861 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:05.862 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 13, 2024-09-13T17:55:05.862 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:55:05.862 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:55:05.862 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:55:05.862 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:55:05.862 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:55:05.862 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:55:05.862 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:55:05.862 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:55:05.862 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:55:05.862 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:55:05.863 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:55:05.863 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:55:05.863 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:55:05.863 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:05.863 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:55:05.863 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:55:05.863 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:55:05.863 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:55:05.863 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:55:05.863 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:55:05.863 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:55:05.863 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:55:05.864 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:55:05.864 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:55:05.864 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:55:05.864 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:55:05.864 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:05.864 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.864 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:55:05.864 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:55:05.864 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:55:05.864 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:55:05.865 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:55:05.865 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:55:05.865 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:55:05.865 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:55:05.865 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:55:05.865 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:55:05.865 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:55:05.865 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:55:05.865 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:55:05.865 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:55:05.866 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:55:05.866 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:55:05.866 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:55:05.866 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:55:05.866 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:55:05.866 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:55:05.866 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:55:05.866 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:55:05.866 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:55:05.866 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:55:05.866 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:55:05.866 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:55:05.866 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:55:05.867 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:05.867 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:55:05.867 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.867 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:55:05.867 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.867 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:55:05.867 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:55:05.867 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:55:05.867 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:55:05.867 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.867 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:55:05.867 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:55:05.867 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:05.868 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.868 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.868 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:55:05.868 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:55:05.868 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:05.868 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:05.868 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:05.868 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.868 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:55:05.868 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:55:05.868 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:55:05.868 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:55:05.868 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:55:05.869 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.869 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.869 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:55:05.869 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:55:05.869 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:55:05.869 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:55:05.869 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.870 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:55:05.870 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:55:05.870 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:05.870 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.870 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.870 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:55:05.870 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:55:05.870 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:05.870 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:05.870 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:05.870 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.870 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:55:05.871 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:55:05.871 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:55:05.871 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:55:05.871 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:55:05.871 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.871 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.871 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:55:05.871 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:55:05.871 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:55:05.871 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:55:05.871 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.871 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:55:05.871 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:55:05.871 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:05.872 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.872 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.872 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:55:05.872 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:55:05.872 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:05.872 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:05.872 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:05.872 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.872 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:55:05.872 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:55:05.872 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:55:05.872 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:55:05.872 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:55:05.873 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:05.873 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:05.873 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.873 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:55:05.873 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:55:05.873 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.873 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:55:05.873 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:55:05.873 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:55:05.873 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:05.873 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:05.873 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:55:05.873 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:05.873 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:55:05.874 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:55:05.874 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:55:05.874 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:05.874 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:05.874 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:05.874 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:55:05.874 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:55:05.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.856+0000 7fad4f266700 1 -- 172.21.15.40:0/2876337104 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad2c06aae0 msgr2=0x7fad2c06cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.856+0000 7fad4f266700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad2c06aae0 0x7fad2c06cfa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fad3400be00 tx=0x7fad40001a00 comp rx=0 tx=0).stop 2024-09-13T17:55:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.856+0000 7fad4f266700 1 -- 172.21.15.40:0/2876337104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34004c40 msgr2=0x7fad480c7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.856+0000 7fad4f266700 1 --2- 172.21.15.40:0/2876337104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad34004c40 0x7fad480c7af0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fad34005810 tx=0x7fad440047e0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.856+0000 7fad4ea65700 1 -- 172.21.15.40:0/2876337104 reap_dead start 2024-09-13T17:55:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.856+0000 7fad4ea65700 1 -- 172.21.15.40:0/2876337104 reap_dead start 2024-09-13T17:55:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.856+0000 7fad4f266700 1 -- 172.21.15.40:0/2876337104 shutdown_connections 2024-09-13T17:55:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.856+0000 7fad4f266700 1 -- 172.21.15.40:0/2876337104 >> 172.21.15.40:0/2876337104 conn(0x7fad4806a390 msgr2=0x7fad480735e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.856+0000 7fad4f266700 1 -- 172.21.15.40:0/2876337104 shutdown_connections 2024-09-13T17:55:05.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.856+0000 7fad4f266700 1 -- 172.21.15.40:0/2876337104 wait complete. 2024-09-13T17:55:05.877 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:55:05.877 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T17:55:05.877 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:55:05.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.976+0000 7fa4ca964700 1 Processor -- start 2024-09-13T17:55:05.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.976+0000 7fa4ca964700 1 -- start start 2024-09-13T17:55:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4ca964700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c415d6f0 0x7fa4c4159f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4ca964700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c415a450 con 0x7fa4c4153730 2024-09-13T17:55:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4ca964700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c4070ae0 con 0x7fa4c40c7fb0 2024-09-13T17:55:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4ca964700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c4071010 con 0x7fa4c415d6f0 2024-09-13T17:55:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c9962700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa4c4153730 0x7fa4c40c8630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56946/0 (socket says 172.21.15.40:56946) 2024-09-13T17:55:05.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c9962700 1 -- 172.21.15.40:0/4178433551 learned_addr learned my addr 172.21.15.40:0/4178433551 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:55:05.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4ca163700 1 --2- 172.21.15.40:0/4178433551 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c415d6f0 0x7fa4c4159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c8960700 1 -- 172.21.15.40:0/4178433551 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100198357 0 0) 0x7fa4c0001110 con 0x7fa4c4153730 2024-09-13T17:55:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c8960700 1 -- 172.21.15.40:0/4178433551 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4b00035d0 con 0x7fa4c4153730 2024-09-13T17:55:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c8960700 1 -- 172.21.15.40:0/4178433551 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017498035 0 0) 0x7fa4c4070ae0 con 0x7fa4c40c7fb0 2024-09-13T17:55:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c8960700 1 -- 172.21.15.40:0/4178433551 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4c0001110 con 0x7fa4c40c7fb0 2024-09-13T17:55:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c8960700 1 -- 172.21.15.40:0/4178433551 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4259418248 0 0) 0x7fa4b00035d0 con 0x7fa4c4153730 2024-09-13T17:55:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c8960700 1 -- 172.21.15.40:0/4178433551 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c415d6f0 msgr2=0x7fa4c4159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:55:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c8960700 1 --2- 172.21.15.40:0/4178433551 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c415d6f0 0x7fa4c4159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c8960700 1 -- 172.21.15.40:0/4178433551 >> v1:172.21.15.43:6789/0 conn(0x7fa4c40c7fb0 legacy=0x7fa4c415cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c8960700 1 -- 172.21.15.40:0/4178433551 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c4072420 con 0x7fa4c4153730 2024-09-13T17:55:05.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c8960700 1 -- 172.21.15.40:0/4178433551 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fa4c0001800 con 0x7fa4c4153730 2024-09-13T17:55:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4ca163700 1 --2- 172.21.15.40:0/4178433551 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c415d6f0 0x7fa4c4159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:55:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c8960700 1 -- 172.21.15.40:0/4178433551 >> v1:172.21.15.40:6789/0 conn(0x7fa4c4153730 legacy=0x7fa4c40c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c8960700 1 --2- 172.21.15.40:0/4178433551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b0003be0 0x7fa4b0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c8960700 1 --2- 172.21.15.40:0/4178433551 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c415d6f0 0x7fa4b0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c8960700 1 --2- 172.21.15.40:0/4178433551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4b0004db0 0x7fa4b0005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c8960700 1 -- 172.21.15.40:0/4178433551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c4071010 con 0x7fa4b0003be0 2024-09-13T17:55:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c8960700 1 -- 172.21.15.40:0/4178433551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b000bae0 con 0x7fa4b0004db0 2024-09-13T17:55:05.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c8960700 1 -- 172.21.15.40:0/4178433551 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b000bca0 con 0x7fa4c415d6f0 2024-09-13T17:55:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c9962700 1 --2- 172.21.15.40:0/4178433551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b0003be0 0x7fa4b0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4ca163700 1 --2- 172.21.15.40:0/4178433551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4b0004db0 0x7fa4b0005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c9161700 1 --2- 172.21.15.40:0/4178433551 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c415d6f0 0x7fa4b0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c9962700 1 -- 172.21.15.40:0/4178433551 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c415d6f0 msgr2=0x7fa4b0004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:55:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c9962700 1 --2- 172.21.15.40:0/4178433551 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c415d6f0 0x7fa4b0004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c9962700 1 -- 172.21.15.40:0/4178433551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4b0004db0 msgr2=0x7fa4b0005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:55:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c9962700 1 --2- 172.21.15.40:0/4178433551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4b0004db0 0x7fa4b0005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c9962700 1 -- 172.21.15.40:0/4178433551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4bc009a70 con 0x7fa4b0003be0 2024-09-13T17:55:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c9962700 1 --2- 172.21.15.40:0/4178433551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b0003be0 0x7fa4b0003fe0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0012950 tx=0x7fa4b00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f636ead5479331 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:05.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c8960700 1 -- 172.21.15.40:0/4178433551 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0017070 con 0x7fa4b0003be0 2024-09-13T17:55:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c8960700 1 -- 172.21.15.40:0/4178433551 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c0005980 con 0x7fa4b0003be0 2024-09-13T17:55:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.980+0000 7fa4c8960700 1 -- 172.21.15.40:0/4178433551 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0005d30 con 0x7fa4b0003be0 2024-09-13T17:55:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.984+0000 7fa4ca964700 1 -- 172.21.15.40:0/4178433551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b0003be0 msgr2=0x7fa4b0003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.984+0000 7fa4ca964700 1 --2- 172.21.15.40:0/4178433551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b0003be0 0x7fa4b0003fe0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0012950 tx=0x7fa4b00058a0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.984+0000 7fa4ca163700 1 -- 172.21.15.40:0/4178433551 reap_dead start 2024-09-13T17:55:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.984+0000 7fa4ca964700 1 -- 172.21.15.40:0/4178433551 shutdown_connections 2024-09-13T17:55:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.984+0000 7fa4ca964700 1 -- 172.21.15.40:0/4178433551 >> 172.21.15.40:0/4178433551 conn(0x7fa4c406e6d0 msgr2=0x7fa4c406eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.984+0000 7fa4ca964700 1 -- 172.21.15.40:0/4178433551 shutdown_connections 2024-09-13T17:55:05.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.984+0000 7fa4ca964700 1 -- 172.21.15.40:0/4178433551 wait complete. 2024-09-13T17:55:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.984+0000 7fa4ca964700 1 Processor -- start 2024-09-13T17:55:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.984+0000 7fa4ca964700 1 -- start start 2024-09-13T17:55:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4ca964700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b0003be0 0x7fa4c414df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4ca964700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c41eb970 0x7fa4c41ec3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4ca964700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c414be70 0x7fa4c4148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4ca964700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c41ebf00 con 0x7fa4b0003be0 2024-09-13T17:55:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4ca964700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c414d6e0 con 0x7fa4c414be70 2024-09-13T17:55:05.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4ca964700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c40ca120 con 0x7fa4c41eb970 2024-09-13T17:55:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4c9962700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b0003be0 0x7fa4c414df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4c9962700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b0003be0 0x7fa4c414df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39968/0 (socket says 172.21.15.40:39968) 2024-09-13T17:55:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4c9962700 1 -- 172.21.15.40:0/3187309505 learned_addr learned my addr 172.21.15.40:0/3187309505 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4c9161700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c41eb970 0x7fa4c41ec3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4ca163700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c414be70 0x7fa4c4148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4c9962700 1 -- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c41eb970 msgr2=0x7fa4c41ec3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4c9962700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c41eb970 0x7fa4c41ec3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4c9962700 1 -- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c414be70 msgr2=0x7fa4c4148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4c9962700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c414be70 0x7fa4c4148410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4c9962700 1 -- 172.21.15.40:0/3187309505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c414cad0 con 0x7fa4b0003be0 2024-09-13T17:55:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4ca163700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c414be70 0x7fa4c4148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:55:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4c9962700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b0003be0 0x7fa4c414df80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0012f40 tx=0x7fa4c406a990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:05.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4ba7fc700 1 -- 172.21.15.40:0/3187309505 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0004a90 con 0x7fa4b0003be0 2024-09-13T17:55:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4ba7fc700 1 -- 172.21.15.40:0/3187309505 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c0004bf0 con 0x7fa4b0003be0 2024-09-13T17:55:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4ca964700 1 -- 172.21.15.40:0/3187309505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c414cdb0 con 0x7fa4b0003be0 2024-09-13T17:55:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4ba7fc700 1 -- 172.21.15.40:0/3187309505 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0001c50 con 0x7fa4b0003be0 2024-09-13T17:55:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4ca964700 1 -- 172.21.15.40:0/3187309505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c414d370 con 0x7fa4b0003be0 2024-09-13T17:55:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4ca964700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c414c250 0x7fa4c4148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4ca964700 1 -- 172.21.15.40:0/3187309505 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c4067600 con 0x7fa4c414c250 2024-09-13T17:55:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4c9161700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c414c250 0x7fa4c4148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4c9161700 1 -- 172.21.15.40:0/3187309505 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa4c4072140 con 0x7fa4c414c250 2024-09-13T17:55:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.988+0000 7fa4c9161700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c414c250 0x7fa4c4148950 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4000c00 tx=0x7fa4b4001a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.992+0000 7fa4ba7fc700 1 -- 172.21.15.40:0/3187309505 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4019070 con 0x7fa4c414c250 2024-09-13T17:55:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.992+0000 7fa4ba7fc700 1 -- 172.21.15.40:0/3187309505 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa4c0010850 con 0x7fa4b0003be0 2024-09-13T17:55:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.992+0000 7fa4ba7fc700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4a80667d0 0x7fa4a8068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.992+0000 7fa4ba7fc700 1 -- 172.21.15.40:0/3187309505 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa4c008cdb0 con 0x7fa4b0003be0 2024-09-13T17:55:05.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.992+0000 7fa4ca163700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4a80667d0 0x7fa4a8068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.992+0000 7fa4ba7fc700 1 -- 172.21.15.40:0/3187309505 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa4b4006880 con 0x7fa4c414c250 2024-09-13T17:55:06.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.992+0000 7fa4ba7fc700 1 -- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c414c250 msgr2=0x7fa4c4148950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:06.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.992+0000 7fa4ba7fc700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c414c250 0x7fa4c4148950 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4000c00 tx=0x7fa4b4001a30 comp rx=0 tx=0).stop 2024-09-13T17:55:06.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.992+0000 7fa4ba7fc700 1 -- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c414c250 msgr2=0x7fa4c4148950 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:55:06.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.992+0000 7fa4ba7fc700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c414c250 0x7fa4c4148950 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:06.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:05.992+0000 7fa4ca163700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4a80667d0 0x7fa4a8068c90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa4c40ccc60 tx=0x7fa4bc00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:06.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.004+0000 7fa4ca964700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c41ee4b0 0x7fa4c41ee8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:06.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.004+0000 7fa4ca964700 1 -- 172.21.15.40:0/3187309505 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c4071e60 con 0x7fa4c41ee4b0 2024-09-13T17:55:06.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.004+0000 7fa4c9161700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c41ee4b0 0x7fa4c41ee8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.004+0000 7fa4c9161700 1 -- 172.21.15.40:0/3187309505 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa4c415cac0 con 0x7fa4c41ee4b0 2024-09-13T17:55:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.004+0000 7fa4c9161700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c41ee4b0 0x7fa4c41ee8c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4000c00 tx=0x7fa4b4007d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:06.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.004+0000 7fa4ba7fc700 1 -- 172.21.15.40:0/3187309505 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4024070 con 0x7fa4c41ee4b0 2024-09-13T17:55:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.008+0000 7fa4ba7fc700 1 -- 172.21.15.40:0/3187309505 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4782 (secure 0 0 0) 0x7fa4b4007230 con 0x7fa4c41ee4b0 2024-09-13T17:55:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.008+0000 7fa4ba7fc700 1 -- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c41ee4b0 msgr2=0x7fa4c41ee8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.008+0000 7fa4ba7fc700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c41ee4b0 0x7fa4c41ee8c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4000c00 tx=0x7fa4b4007d70 comp rx=0 tx=0).stop 2024-09-13T17:55:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.008+0000 7fa4ba7fc700 1 -- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c41ee4b0 msgr2=0x7fa4c41ee8c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:55:06.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.008+0000 7fa4ba7fc700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c41ee4b0 0x7fa4c41ee8c0 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4000c00 tx=0x7fa4b4007d70 comp rx=0 tx=0).stop 2024-09-13T17:55:06.013 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:55:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:55:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:55:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:55:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 40, 2024-09-13T17:55:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:55:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:55:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:55:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:55:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 13, 2024-09-13T17:55:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:55:06.013 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:55:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:55:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:55:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:55:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:55:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:55:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:55:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:55:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:55:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:55:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:55:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:55:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:06.014 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:55:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:55:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:55:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:55:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:55:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:55:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:55:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:55:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:55:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:55:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:55:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:55:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:06.015 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:06.016 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:55:06.016 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:55:06.016 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:55:06.016 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:55:06.016 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:55:06.016 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:55:06.016 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:55:06.016 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:55:06.016 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:55:06.016 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:55:06.017 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:55:06.017 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:55:06.017 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:55:06.017 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:55:06.017 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:55:06.017 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:55:06.017 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:55:06.017 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:55:06.017 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:55:06.017 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:55:06.018 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:55:06.018 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:55:06.018 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:55:06.018 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:55:06.018 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:55:06.018 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:55:06.018 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:55:06.018 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:06.018 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:55:06.018 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:55:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:55:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:55:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:55:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:55:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:55:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:55:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:06.019 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:55:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:55:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:55:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:55:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:55:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:55:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:55:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:06.020 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:55:06.021 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:55:06.021 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:55:06.021 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:55:06.021 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:06.021 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:55:06.021 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:55:06.021 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:06.021 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:06.021 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:55:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:55:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:55:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:55:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:55:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:55:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:55:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:06.022 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:55:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:55:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:55:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:55:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:55:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:55:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:55:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:55:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:06.023 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:55:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:55:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:55:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:55:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:55:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:55:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:55:06.024 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:55:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:55:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:55:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:55:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:55:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:55:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:55:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:55:06.025 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:55:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:55:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:55:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:55:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:55:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:55:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:55:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:06.026 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:55:06.027 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:55:06.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.008+0000 7fa4ca964700 1 -- 172.21.15.40:0/3187309505 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4a80667d0 msgr2=0x7fa4a8068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:06.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.008+0000 7fa4ca964700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4a80667d0 0x7fa4a8068c90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa4c40ccc60 tx=0x7fa4bc00b040 comp rx=0 tx=0).stop 2024-09-13T17:55:06.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.008+0000 7fa4ca964700 1 -- 172.21.15.40:0/3187309505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b0003be0 msgr2=0x7fa4c414df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:06.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.008+0000 7fa4ca964700 1 --2- 172.21.15.40:0/3187309505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4b0003be0 0x7fa4c414df80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0012f40 tx=0x7fa4c406a990 comp rx=0 tx=0).stop 2024-09-13T17:55:06.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.008+0000 7fa4ca163700 1 -- 172.21.15.40:0/3187309505 reap_dead start 2024-09-13T17:55:06.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.008+0000 7fa4ca163700 1 -- 172.21.15.40:0/3187309505 reap_dead start 2024-09-13T17:55:06.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.008+0000 7fa4ca964700 1 -- 172.21.15.40:0/3187309505 shutdown_connections 2024-09-13T17:55:06.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.008+0000 7fa4ca964700 1 -- 172.21.15.40:0/3187309505 >> 172.21.15.40:0/3187309505 conn(0x7fa4c406e6d0 msgr2=0x7fa4c40c6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:06.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.008+0000 7fa4ca964700 1 -- 172.21.15.40:0/3187309505 shutdown_connections 2024-09-13T17:55:06.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.008+0000 7fa4ca964700 1 -- 172.21.15.40:0/3187309505 wait complete. 2024-09-13T17:55:06.029 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:55:06.029 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-13T17:55:06.029 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T17:55:06.030 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-13T17:55:06.030 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T17:55:06.030 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T17:55:06.030 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:55:06.072 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T17:55:06.072 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T17:55:06.072 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:55:06.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.180+0000 7f3cd5364700 1 Processor -- start 2024-09-13T17:55:06.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.180+0000 7f3cd5364700 1 -- start start 2024-09-13T17:55:06.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3cd5364700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3cd0153880 0x7f3cd00c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:06.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3cd5364700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3cd015cf60 0x7f3cd015d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:06.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3cd5364700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd015d930 con 0x7f3cd00cb2f0 2024-09-13T17:55:06.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3cd5364700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd0159770 con 0x7f3cd0153880 2024-09-13T17:55:06.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3cd5364700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd0159a50 con 0x7f3cd015cf60 2024-09-13T17:55:06.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3ccffff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3cd0153880 0x7f3cd00c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:06.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3ccffff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3cd0153880 0x7f3cd00c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:42376/0 (socket says 172.21.15.40:42376) 2024-09-13T17:55:06.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3ccffff700 1 -- 172.21.15.40:0/2259672198 learned_addr learned my addr 172.21.15.40:0/2259672198 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:06.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3cd4b63700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3cd015cf60 0x7f3cd015d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:06.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3ccffff700 1 -- 172.21.15.40:0/2259672198 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3cd015cf60 msgr2=0x7f3cd015d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:55:06.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3ccffff700 1 --2- 172.21.15.40:0/2259672198 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3cd015cf60 0x7f3cd015d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:06.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3ccffff700 1 -- 172.21.15.40:0/2259672198 >> v1:172.21.15.43:6789/0 conn(0x7f3cd00cb2f0 legacy=0x7f3cd00cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:06.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3ccffff700 1 -- 172.21.15.40:0/2259672198 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd0159d30 con 0x7f3cd0153880 2024-09-13T17:55:06.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3ccffff700 1 --2- 172.21.15.40:0/2259672198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cd0153880 0x7f3cd00c9ef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009e50 tx=0x7f3cc800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=408c7a7e8b4b6ce0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:06.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3cceffd700 1 -- 172.21.15.40:0/2259672198 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc800e070 con 0x7f3cd0153880 2024-09-13T17:55:06.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3cceffd700 1 -- 172.21.15.40:0/2259672198 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc800b910 con 0x7f3cd0153880 2024-09-13T17:55:06.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3cceffd700 1 -- 172.21.15.40:0/2259672198 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc800bcc0 con 0x7f3cd0153880 2024-09-13T17:55:06.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3cd5364700 1 -- 172.21.15.40:0/2259672198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cd0153880 msgr2=0x7f3cd00c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:06.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3cd5364700 1 --2- 172.21.15.40:0/2259672198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cd0153880 0x7f3cd00c9ef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009e50 tx=0x7f3cc800b330 comp rx=0 tx=0).stop 2024-09-13T17:55:06.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3cd5364700 1 -- 172.21.15.40:0/2259672198 shutdown_connections 2024-09-13T17:55:06.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3cd5364700 1 --2- 172.21.15.40:0/2259672198 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3cd015cf60 0x7f3cd015d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:06.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3cd5364700 1 --2- 172.21.15.40:0/2259672198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cd0153880 0x7f3cd00c9ef0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:06.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3cd5364700 1 -- 172.21.15.40:0/2259672198 >> 172.21.15.40:0/2259672198 conn(0x7f3cd006c5b0 msgr2=0x7f3cd006c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:06.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3cd5364700 1 -- 172.21.15.40:0/2259672198 shutdown_connections 2024-09-13T17:55:06.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.184+0000 7f3cd5364700 1 -- 172.21.15.40:0/2259672198 wait complete. 2024-09-13T17:55:06.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3cd5364700 1 Processor -- start 2024-09-13T17:55:06.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3cd5364700 1 -- start start 2024-09-13T17:55:06.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3cd5364700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cd00cb2f0 0x7f3cd015b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:06.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3cd5364700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cd0153880 0x7f3cd0149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:06.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3cd5364700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3cd015cf60 0x7f3cd01f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:06.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3cd5364700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd015bbf0 con 0x7f3cd0153880 2024-09-13T17:55:06.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3cd5364700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd00b41c0 con 0x7f3cd00cb2f0 2024-09-13T17:55:06.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3cd5364700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd01f2050 con 0x7f3cd015cf60 2024-09-13T17:55:06.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3ccf7fe700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cd0153880 msgr2=0x7f3cd0149060 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:55:06.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3ccf7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cd0153880 0x7f3cd0149060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:06.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3ccffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cd00cb2f0 0x7f3cd015b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:06.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3ccffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cd00cb2f0 0x7f3cd015b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:42386/0 (socket says 172.21.15.40:42386) 2024-09-13T17:55:06.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3ccffff700 1 -- 172.21.15.40:0/1388293227 learned_addr learned my addr 172.21.15.40:0/1388293227 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:06.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3cd4b63700 1 --2- 172.21.15.40:0/1388293227 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3cd015cf60 0x7f3cd01f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:06.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3ccffff700 1 -- 172.21.15.40:0/1388293227 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3cd015cf60 msgr2=0x7f3cd01f27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:06.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3ccffff700 1 --2- 172.21.15.40:0/1388293227 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3cd015cf60 0x7f3cd01f27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:06.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3ccffff700 1 -- 172.21.15.40:0/1388293227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cd0153880 msgr2=0x7f3cd0149060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:55:06.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3ccffff700 1 --2- 172.21.15.40:0/1388293227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cd0153880 0x7f3cd0149060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:06.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3ccffff700 1 -- 172.21.15.40:0/1388293227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc8009b30 con 0x7f3cd00cb2f0 2024-09-13T17:55:06.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3ccffff700 1 --2- 172.21.15.40:0/1388293227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cd00cb2f0 0x7f3cd015b460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00cca90 tx=0x7f3cc8004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:06.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3cccff9700 1 -- 172.21.15.40:0/1388293227 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8004930 con 0x7f3cd00cb2f0 2024-09-13T17:55:06.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3cd5364700 1 -- 172.21.15.40:0/1388293227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd00b47c0 con 0x7f3cd00cb2f0 2024-09-13T17:55:06.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3cccff9700 1 -- 172.21.15.40:0/1388293227 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc80092e0 con 0x7f3cd00cb2f0 2024-09-13T17:55:06.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3cccff9700 1 -- 172.21.15.40:0/1388293227 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc801c660 con 0x7f3cd00cb2f0 2024-09-13T17:55:06.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.188+0000 7f3cd5364700 1 -- 172.21.15.40:0/1388293227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd01495a0 con 0x7f3cd00cb2f0 2024-09-13T17:55:06.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.192+0000 7f3cd5364700 1 -- 172.21.15.40:0/1388293227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd01f2500 con 0x7f3cd00cb2f0 2024-09-13T17:55:06.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.192+0000 7f3cccff9700 1 -- 172.21.15.40:0/1388293227 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3cc801c7c0 con 0x7f3cd00cb2f0 2024-09-13T17:55:06.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.192+0000 7f3cccff9700 1 --2- 172.21.15.40:0/1388293227 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3cbc0666e0 0x7f3cbc068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:06.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.192+0000 7f3cccff9700 1 -- 172.21.15.40:0/1388293227 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3cc8086200 con 0x7f3cd00cb2f0 2024-09-13T17:55:06.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.192+0000 7f3ccf7fe700 1 --2- 172.21.15.40:0/1388293227 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3cbc0666e0 0x7f3cbc068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:06.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.192+0000 7f3ccf7fe700 1 --2- 172.21.15.40:0/1388293227 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3cbc0666e0 0x7f3cbc068ba0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0007fd0 tx=0x7f3cc0007ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:06.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.204+0000 7f3cccff9700 1 -- 172.21.15.40:0/1388293227 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3cd01f2500 con 0x7f3cd00cb2f0 2024-09-13T17:55:06.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.472+0000 7f3cd5364700 1 -- 172.21.15.40:0/1388293227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3cd01f2500 con 0x7f3cd00cb2f0 2024-09-13T17:55:06.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.472+0000 7f3cccff9700 1 -- 172.21.15.40:0/1388293227 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f3cd01f2500 con 0x7f3cd00cb2f0 2024-09-13T17:55:06.480 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:55:06.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.476+0000 7f3cd5364700 1 -- 172.21.15.40:0/1388293227 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3cbc0666e0 msgr2=0x7f3cbc068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:06.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.476+0000 7f3cd5364700 1 --2- 172.21.15.40:0/1388293227 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3cbc0666e0 0x7f3cbc068ba0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0007fd0 tx=0x7f3cc0007ac0 comp rx=0 tx=0).stop 2024-09-13T17:55:06.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.476+0000 7f3cd5364700 1 -- 172.21.15.40:0/1388293227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cd00cb2f0 msgr2=0x7f3cd015b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:06.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.476+0000 7f3cd5364700 1 --2- 172.21.15.40:0/1388293227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cd00cb2f0 0x7f3cd015b460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00cca90 tx=0x7f3cc8004af0 comp rx=0 tx=0).stop 2024-09-13T17:55:06.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.476+0000 7f3cd5364700 1 -- 172.21.15.40:0/1388293227 shutdown_connections 2024-09-13T17:55:06.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.476+0000 7f3cd5364700 1 --2- 172.21.15.40:0/1388293227 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3cbc0666e0 0x7f3cbc068ba0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:06.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.476+0000 7f3cd5364700 1 --2- 172.21.15.40:0/1388293227 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3cd015cf60 0x7f3cd01f27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:06.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.476+0000 7f3cd5364700 1 --2- 172.21.15.40:0/1388293227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3cd0153880 0x7f3cd0149060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:06.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.476+0000 7f3cd5364700 1 --2- 172.21.15.40:0/1388293227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3cd00cb2f0 0x7f3cd015b460 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:06.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.476+0000 7f3cd5364700 1 -- 172.21.15.40:0/1388293227 >> 172.21.15.40:0/1388293227 conn(0x7f3cd006c5b0 msgr2=0x7f3cd014e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:06.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.476+0000 7f3cd5364700 1 -- 172.21.15.40:0/1388293227 shutdown_connections 2024-09-13T17:55:06.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:06.476+0000 7f3cd5364700 1 -- 172.21.15.40:0/1388293227 wait complete. 2024-09-13T17:55:09.500 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:55:09.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.616+0000 7fc8289b7700 1 Processor -- start 2024-09-13T17:55:09.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.616+0000 7fc8289b7700 1 -- start start 2024-09-13T17:55:09.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.616+0000 7fc8289b7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc8241500c0 0x7fc8240c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:09.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.616+0000 7fc8289b7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc824072250 0x7fc8240c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:09.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.616+0000 7fc8289b7700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8240cba20 con 0x7fc824070500 2024-09-13T17:55:09.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.616+0000 7fc8289b7700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8241560e0 con 0x7fc8241500c0 2024-09-13T17:55:09.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.616+0000 7fc8289b7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8241563c0 con 0x7fc824072250 2024-09-13T17:55:09.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.616+0000 7fc81bfff700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7fc824070500 legacy=0x7fc824071b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:55:09.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.616+0000 7fc8237fe700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7fc8241500c0 msgr2=0x7fc8240c66d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:55:09.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.616+0000 7fc8237fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc8241500c0 0x7fc8240c66d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:09.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.616+0000 7fc823fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc824072250 0x7fc8240c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:09.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.616+0000 7fc823fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc824072250 0x7fc8240c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:33308/0 (socket says 172.21.15.40:33308) 2024-09-13T17:55:09.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.616+0000 7fc823fff700 1 -- 172.21.15.40:0/2043933030 learned_addr learned my addr 172.21.15.40:0/2043933030 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:09.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.616+0000 7fc823fff700 1 -- 172.21.15.40:0/2043933030 >> v2:172.21.15.40:3300/0 conn(0x7fc8241500c0 msgr2=0x7fc8240c66d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:55:09.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.616+0000 7fc823fff700 1 --2- 172.21.15.40:0/2043933030 >> v2:172.21.15.40:3300/0 conn(0x7fc8241500c0 0x7fc8240c66d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:09.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.616+0000 7fc823fff700 1 -- 172.21.15.40:0/2043933030 >> v1:172.21.15.40:6789/0 conn(0x7fc824070500 legacy=0x7fc824071b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:55:09.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.616+0000 7fc823fff700 1 -- 172.21.15.40:0/2043933030 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8241566a0 con 0x7fc824072250 2024-09-13T17:55:09.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.620+0000 7fc823fff700 1 --2- 172.21.15.40:0/2043933030 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc824072250 0x7fc8240c6c10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00d150 tx=0x7fc81c00d460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54a5bff24923f106 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:09.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.620+0000 7fc822ffd700 1 -- 172.21.15.40:0/2043933030 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c010070 con 0x7fc824072250 2024-09-13T17:55:09.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.620+0000 7fc822ffd700 1 -- 172.21.15.40:0/2043933030 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81c00dd80 con 0x7fc824072250 2024-09-13T17:55:09.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.620+0000 7fc822ffd700 1 -- 172.21.15.40:0/2043933030 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c00b310 con 0x7fc824072250 2024-09-13T17:55:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.620+0000 7fc8289b7700 1 -- 172.21.15.40:0/2043933030 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc824072250 msgr2=0x7fc8240c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.620+0000 7fc8289b7700 1 --2- 172.21.15.40:0/2043933030 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc824072250 0x7fc8240c6c10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00d150 tx=0x7fc81c00d460 comp rx=0 tx=0).stop 2024-09-13T17:55:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.620+0000 7fc8289b7700 1 -- 172.21.15.40:0/2043933030 shutdown_connections 2024-09-13T17:55:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.620+0000 7fc8289b7700 1 --2- 172.21.15.40:0/2043933030 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc824072250 0x7fc8240c6c10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.620+0000 7fc8289b7700 1 --2- 172.21.15.40:0/2043933030 >> v2:172.21.15.40:3300/0 conn(0x7fc8241500c0 0x7fc8240c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.620+0000 7fc8289b7700 1 -- 172.21.15.40:0/2043933030 >> 172.21.15.40:0/2043933030 conn(0x7fc8240cabc0 msgr2=0x7fc8240cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.620+0000 7fc8289b7700 1 -- 172.21.15.40:0/2043933030 shutdown_connections 2024-09-13T17:55:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.620+0000 7fc8289b7700 1 -- 172.21.15.40:0/2043933030 wait complete. 2024-09-13T17:55:09.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.620+0000 7fc8289b7700 1 Processor -- start 2024-09-13T17:55:09.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.620+0000 7fc8289b7700 1 -- start start 2024-09-13T17:55:09.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc8289b7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc824070500 0x7fc8241dc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:09.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc8289b7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc824072250 0x7fc8241eef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:09.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc8289b7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8241500c0 0x7fc8241ef470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:09.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc8289b7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8241f3070 con 0x7fc8241500c0 2024-09-13T17:55:09.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc8289b7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc824155a80 con 0x7fc824072250 2024-09-13T17:55:09.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc8289b7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc824155d60 con 0x7fc824070500 2024-09-13T17:55:09.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc823fff700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8241500c0 msgr2=0x7fc8241ef470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:55:09.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc823fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8241500c0 0x7fc8241ef470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:09.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc8237fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc824070500 0x7fc8241dc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:09.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc81bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc824072250 0x7fc8241eef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:09.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc8237fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc824070500 0x7fc8241dc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:33320/0 (socket says 172.21.15.40:33320) 2024-09-13T17:55:09.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc8237fe700 1 -- 172.21.15.40:0/262352269 learned_addr learned my addr 172.21.15.40:0/262352269 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:09.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc8237fe700 1 -- 172.21.15.40:0/262352269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc824072250 msgr2=0x7fc8241eef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:09.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc8237fe700 1 --2- 172.21.15.40:0/262352269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc824072250 0x7fc8241eef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:09.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc8237fe700 1 -- 172.21.15.40:0/262352269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8241500c0 msgr2=0x7fc8241ef470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:55:09.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc8237fe700 1 --2- 172.21.15.40:0/262352269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8241500c0 0x7fc8241ef470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:09.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc8237fe700 1 -- 172.21.15.40:0/262352269 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c00bd40 con 0x7fc824070500 2024-09-13T17:55:09.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc8237fe700 1 --2- 172.21.15.40:0/262352269 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc824070500 0x7fc8241dc2b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00b570 tx=0x7fc80c00b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:09.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc820ff9700 1 -- 172.21.15.40:0/262352269 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c003e70 con 0x7fc824070500 2024-09-13T17:55:09.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc820ff9700 1 -- 172.21.15.40:0/262352269 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80c005070 con 0x7fc824070500 2024-09-13T17:55:09.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc8289b7700 1 -- 172.21.15.40:0/262352269 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc82406ac70 con 0x7fc824070500 2024-09-13T17:55:09.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc820ff9700 1 -- 172.21.15.40:0/262352269 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c005440 con 0x7fc824070500 2024-09-13T17:55:09.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc8289b7700 1 -- 172.21.15.40:0/262352269 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc82406b230 con 0x7fc824070500 2024-09-13T17:55:09.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.624+0000 7fc8289b7700 1 -- 172.21.15.40:0/262352269 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8240c4410 con 0x7fc824070500 2024-09-13T17:55:09.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.628+0000 7fc820ff9700 1 -- 172.21.15.40:0/262352269 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc80c0055a0 con 0x7fc824070500 2024-09-13T17:55:09.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.628+0000 7fc820ff9700 1 --2- 172.21.15.40:0/262352269 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8040666e0 0x7fc804068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:09.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.628+0000 7fc820ff9700 1 -- 172.21.15.40:0/262352269 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc80c083d00 con 0x7fc824070500 2024-09-13T17:55:09.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.628+0000 7fc81bfff700 1 --2- 172.21.15.40:0/262352269 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8040666e0 0x7fc804068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:09.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.628+0000 7fc81bfff700 1 --2- 172.21.15.40:0/262352269 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8040666e0 0x7fc804068ba0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc814009470 tx=0x7fc814009400 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:09.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.636+0000 7fc820ff9700 1 -- 172.21.15.40:0/262352269 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc80c050d40 con 0x7fc824070500 2024-09-13T17:55:09.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.888+0000 7fc8289b7700 1 -- 172.21.15.40:0/262352269 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc8240c7360 con 0x7fc824070500 2024-09-13T17:55:09.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.888+0000 7fc820ff9700 1 -- 172.21.15.40:0/262352269 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fc80c016620 con 0x7fc824070500 2024-09-13T17:55:09.894 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:55:09.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.892+0000 7fc8289b7700 1 -- 172.21.15.40:0/262352269 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8040666e0 msgr2=0x7fc804068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:09.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.892+0000 7fc8289b7700 1 --2- 172.21.15.40:0/262352269 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8040666e0 0x7fc804068ba0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc814009470 tx=0x7fc814009400 comp rx=0 tx=0).stop 2024-09-13T17:55:09.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.892+0000 7fc8289b7700 1 -- 172.21.15.40:0/262352269 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc824070500 msgr2=0x7fc8241dc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:09.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.892+0000 7fc8289b7700 1 --2- 172.21.15.40:0/262352269 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc824070500 0x7fc8241dc2b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00b570 tx=0x7fc80c00b880 comp rx=0 tx=0).stop 2024-09-13T17:55:09.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.892+0000 7fc8289b7700 1 -- 172.21.15.40:0/262352269 shutdown_connections 2024-09-13T17:55:09.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.892+0000 7fc8289b7700 1 --2- 172.21.15.40:0/262352269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8241500c0 0x7fc8241ef470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:09.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.892+0000 7fc8289b7700 1 --2- 172.21.15.40:0/262352269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc824072250 0x7fc8241eef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:09.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.892+0000 7fc8289b7700 1 --2- 172.21.15.40:0/262352269 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8040666e0 0x7fc804068ba0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:09.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.892+0000 7fc8289b7700 1 --2- 172.21.15.40:0/262352269 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc824070500 0x7fc8241dc2b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:09.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.892+0000 7fc8289b7700 1 -- 172.21.15.40:0/262352269 >> 172.21.15.40:0/262352269 conn(0x7fc8240cabc0 msgr2=0x7fc8240c52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:09.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.892+0000 7fc8289b7700 1 -- 172.21.15.40:0/262352269 shutdown_connections 2024-09-13T17:55:09.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:09.892+0000 7fc8289b7700 1 -- 172.21.15.40:0/262352269 wait complete. 2024-09-13T17:55:12.910 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:55:13.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.032+0000 7f8396278700 1 Processor -- start 2024-09-13T17:55:13.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.032+0000 7f8396278700 1 -- start start 2024-09-13T17:55:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.032+0000 7f8396278700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8390151470 0x7f83900c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.032+0000 7f8396278700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8390071bf0 0x7f8390072020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.032+0000 7f8396278700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83900c4520 con 0x7f83900b1bd0 2024-09-13T17:55:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.032+0000 7f8396278700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83900c4dc0 con 0x7f8390151470 2024-09-13T17:55:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.032+0000 7f8396278700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8390157cf0 con 0x7f8390071bf0 2024-09-13T17:55:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.032+0000 7f8394a75700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f83900b1bd0 legacy=0x7f83901571d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:55:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.032+0000 7f8395276700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f8390151470 msgr2=0x7f83900c5250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:55:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.032+0000 7f8395276700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8390151470 0x7f83900c5250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.032+0000 7f8395a77700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8390071bf0 0x7f8390072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:13.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.032+0000 7f8395a77700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8390071bf0 0x7f8390072020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:44622/0 (socket says 172.21.15.40:44622) 2024-09-13T17:55:13.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.032+0000 7f8395a77700 1 -- 172.21.15.40:0/2682309358 learned_addr learned my addr 172.21.15.40:0/2682309358 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:13.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.036+0000 7f8395a77700 1 -- 172.21.15.40:0/2682309358 >> v2:172.21.15.40:3300/0 conn(0x7f8390151470 msgr2=0x7f83900c5250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:55:13.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.036+0000 7f8395a77700 1 --2- 172.21.15.40:0/2682309358 >> v2:172.21.15.40:3300/0 conn(0x7f8390151470 0x7f83900c5250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:13.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.036+0000 7f8395a77700 1 -- 172.21.15.40:0/2682309358 >> v1:172.21.15.40:6789/0 conn(0x7f83900b1bd0 legacy=0x7f83901571d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:55:13.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.036+0000 7f8395a77700 1 -- 172.21.15.40:0/2682309358 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8390157fd0 con 0x7f8390071bf0 2024-09-13T17:55:13.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.036+0000 7f8395a77700 1 --2- 172.21.15.40:0/2682309358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8390071bf0 0x7f8390072020 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8390070b70 tx=0x7f838800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57b662e8af5732ec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:13.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.036+0000 7f8387fff700 1 -- 172.21.15.40:0/2682309358 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838800e070 con 0x7f8390071bf0 2024-09-13T17:55:13.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.036+0000 7f8387fff700 1 -- 172.21.15.40:0/2682309358 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8388003a40 con 0x7f8390071bf0 2024-09-13T17:55:13.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.036+0000 7f8387fff700 1 -- 172.21.15.40:0/2682309358 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388003df0 con 0x7f8390071bf0 2024-09-13T17:55:13.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.036+0000 7f8396278700 1 -- 172.21.15.40:0/2682309358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8390071bf0 msgr2=0x7f8390072020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:13.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.036+0000 7f8396278700 1 --2- 172.21.15.40:0/2682309358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8390071bf0 0x7f8390072020 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8390070b70 tx=0x7f838800b330 comp rx=0 tx=0).stop 2024-09-13T17:55:13.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.036+0000 7f8396278700 1 -- 172.21.15.40:0/2682309358 shutdown_connections 2024-09-13T17:55:13.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.036+0000 7f8396278700 1 --2- 172.21.15.40:0/2682309358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8390071bf0 0x7f8390072020 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:13.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.036+0000 7f8396278700 1 --2- 172.21.15.40:0/2682309358 >> v2:172.21.15.40:3300/0 conn(0x7f8390151470 0x7f83900c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:13.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.036+0000 7f8396278700 1 -- 172.21.15.40:0/2682309358 >> 172.21.15.40:0/2682309358 conn(0x7f839006e4f0 msgr2=0x7f839006e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:13.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.036+0000 7f8396278700 1 -- 172.21.15.40:0/2682309358 shutdown_connections 2024-09-13T17:55:13.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.036+0000 7f8396278700 1 -- 172.21.15.40:0/2682309358 wait complete. 2024-09-13T17:55:13.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.036+0000 7f8396278700 1 Processor -- start 2024-09-13T17:55:13.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.036+0000 7f8396278700 1 -- start start 2024-09-13T17:55:13.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8396278700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8390071bf0 0x7f839015a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:13.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8396278700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83900b1bd0 0x7f839006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8396278700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8390151470 0x7f83901f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8396278700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83901ef460 con 0x7f8390071bf0 2024-09-13T17:55:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8396278700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83901ef740 con 0x7f83900b1bd0 2024-09-13T17:55:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8396278700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83901efa20 con 0x7f8390151470 2024-09-13T17:55:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8395276700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8390071bf0 msgr2=0x7f839015a520 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:55:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8395276700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8390071bf0 0x7f839015a520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8394a75700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83900b1bd0 0x7f839006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8394a75700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83900b1bd0 0x7f839006d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:44632/0 (socket says 172.21.15.40:44632) 2024-09-13T17:55:13.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8394a75700 1 -- 172.21.15.40:0/835597645 learned_addr learned my addr 172.21.15.40:0/835597645 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8395a77700 1 --2- 172.21.15.40:0/835597645 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8390151470 0x7f83901f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8394a75700 1 -- 172.21.15.40:0/835597645 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8390151470 msgr2=0x7f83901f2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8394a75700 1 --2- 172.21.15.40:0/835597645 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8390151470 0x7f83901f2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8394a75700 1 -- 172.21.15.40:0/835597645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8390071bf0 msgr2=0x7f839015a520 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:55:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8394a75700 1 --2- 172.21.15.40:0/835597645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8390071bf0 0x7f839015a520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8394a75700 1 -- 172.21.15.40:0/835597645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8388009b30 con 0x7f83900b1bd0 2024-09-13T17:55:13.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8394a75700 1 --2- 172.21.15.40:0/835597645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83900b1bd0 0x7f839006d350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f838000b780 tx=0x7f838000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8385ffb700 1 -- 172.21.15.40:0/835597645 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380003e70 con 0x7f83900b1bd0 2024-09-13T17:55:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8396278700 1 -- 172.21.15.40:0/835597645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83901efd00 con 0x7f83900b1bd0 2024-09-13T17:55:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8385ffb700 1 -- 172.21.15.40:0/835597645 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8380005070 con 0x7f83900b1bd0 2024-09-13T17:55:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8385ffb700 1 -- 172.21.15.40:0/835597645 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380005420 con 0x7f83900b1bd0 2024-09-13T17:55:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8396278700 1 -- 172.21.15.40:0/835597645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f839015aa60 con 0x7f83900b1bd0 2024-09-13T17:55:13.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.040+0000 7f8396278700 1 -- 172.21.15.40:0/835597645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83901f2540 con 0x7f83900b1bd0 2024-09-13T17:55:13.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.044+0000 7f8385ffb700 1 -- 172.21.15.40:0/835597645 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8380005580 con 0x7f83900b1bd0 2024-09-13T17:55:13.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.044+0000 7f8385ffb700 1 --2- 172.21.15.40:0/835597645 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f83740666e0 0x7f8374068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:13.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.044+0000 7f8395276700 1 --2- 172.21.15.40:0/835597645 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f83740666e0 0x7f8374068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:13.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.044+0000 7f8385ffb700 1 -- 172.21.15.40:0/835597645 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8380084670 con 0x7f83900b1bd0 2024-09-13T17:55:13.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.044+0000 7f8395276700 1 --2- 172.21.15.40:0/835597645 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f83740666e0 0x7f8374068ba0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f838c006fd0 tx=0x7f838c006de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:13.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.056+0000 7f8385ffb700 1 -- 172.21.15.40:0/835597645 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8380051590 con 0x7f83900b1bd0 2024-09-13T17:55:13.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.296+0000 7f8396278700 1 -- 172.21.15.40:0/835597645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f83901f2540 con 0x7f83900b1bd0 2024-09-13T17:55:13.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.296+0000 7f8385ffb700 1 -- 172.21.15.40:0/835597645 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f83901f2540 con 0x7f83900b1bd0 2024-09-13T17:55:13.302 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:55:13.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.296+0000 7f8396278700 1 -- 172.21.15.40:0/835597645 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f83740666e0 msgr2=0x7f8374068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:13.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.296+0000 7f8396278700 1 --2- 172.21.15.40:0/835597645 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f83740666e0 0x7f8374068ba0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f838c006fd0 tx=0x7f838c006de0 comp rx=0 tx=0).stop 2024-09-13T17:55:13.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.296+0000 7f8396278700 1 -- 172.21.15.40:0/835597645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83900b1bd0 msgr2=0x7f839006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:13.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.296+0000 7f8396278700 1 --2- 172.21.15.40:0/835597645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83900b1bd0 0x7f839006d350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f838000b780 tx=0x7f838000ba90 comp rx=0 tx=0).stop 2024-09-13T17:55:13.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.296+0000 7f8396278700 1 -- 172.21.15.40:0/835597645 shutdown_connections 2024-09-13T17:55:13.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.296+0000 7f8396278700 1 --2- 172.21.15.40:0/835597645 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f83740666e0 0x7f8374068ba0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:13.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.296+0000 7f8396278700 1 --2- 172.21.15.40:0/835597645 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8390151470 0x7f83901f2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:13.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.296+0000 7f8396278700 1 --2- 172.21.15.40:0/835597645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83900b1bd0 0x7f839006d350 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:13.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.296+0000 7f8396278700 1 --2- 172.21.15.40:0/835597645 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8390071bf0 0x7f839015a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:13.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.296+0000 7f8396278700 1 -- 172.21.15.40:0/835597645 >> 172.21.15.40:0/835597645 conn(0x7f839006e4f0 msgr2=0x7f839014ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:13.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.300+0000 7f8396278700 1 -- 172.21.15.40:0/835597645 shutdown_connections 2024-09-13T17:55:13.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:13.300+0000 7f8396278700 1 -- 172.21.15.40:0/835597645 wait complete. 2024-09-13T17:55:16.322 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:55:16.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:16.436+0000 7feef5a44700 1 Processor -- start 2024-09-13T17:55:16.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:16.436+0000 7feef5a44700 1 -- start start 2024-09-13T17:55:16.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:16.440+0000 7feef5a44700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feef00cb2f0 0x7feef00cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:16.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:16.440+0000 7feef5a44700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef0159770 con 0x7feef0153880 2024-09-13T17:55:16.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:16.440+0000 7feef5a44700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef0159a50 con 0x7feef00cf3c0 2024-09-13T17:55:16.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:16.440+0000 7feef5a44700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef0159d30 con 0x7feef00cb2f0 2024-09-13T17:55:16.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:16.440+0000 7feeeffff700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7feef00cb2f0 msgr2=0x7feef00cee80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:55:16.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:16.440+0000 7feef4a42700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7feef0153880 legacy=0x7feef00c9ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:55:16.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:16.440+0000 7feeeffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feef00cb2f0 0x7feef00cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:16.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:16.440+0000 7feef5243700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7feef00cf3c0 0x7feef015cf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:55620/0 (socket says 172.21.15.40:55620) 2024-09-13T17:55:16.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:16.440+0000 7feef5243700 1 -- 172.21.15.40:0/1775951266 learned_addr learned my addr 172.21.15.40:0/1775951266 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:55:16.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:16.644+0000 7feef4a42700 1 -- 172.21.15.40:0/1775951266 >> v1:172.21.15.40:6789/0 conn(0x7feef0153880 legacy=0x7feef00c9ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:55:16.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:16.644+0000 7feeeffff700 1 -- 172.21.15.40:0/1775951266 >> v2:172.21.15.40:3300/0 conn(0x7feef00cb2f0 msgr2=0x7feef00cee80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:55:16.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:16.644+0000 7feeeffff700 1 --2- 172.21.15.40:0/1775951266 >> v2:172.21.15.40:3300/0 conn(0x7feef00cb2f0 0x7feef00cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:55:17.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:17.048+0000 7feef4a42700 1 -- 172.21.15.40:0/1775951266 >> v1:172.21.15.40:6789/0 conn(0x7feef0153880 legacy=0x7feef00c9ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:55:17.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:17.048+0000 7feeeffff700 1 -- 172.21.15.40:0/1775951266 >> v2:172.21.15.40:3300/0 conn(0x7feef00cb2f0 msgr2=0x7feef00cee80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:55:17.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:17.048+0000 7feeeffff700 1 --2- 172.21.15.40:0/1775951266 >> v2:172.21.15.40:3300/0 conn(0x7feef00cb2f0 0x7feef00cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:55:17.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:17.852+0000 7feef4a42700 1 -- 172.21.15.40:0/1775951266 >> v1:172.21.15.40:6789/0 conn(0x7feef0153880 legacy=0x7feef00c9ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:55:17.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:17.852+0000 7feeeffff700 1 -- 172.21.15.40:0/1775951266 >> v2:172.21.15.40:3300/0 conn(0x7feef00cb2f0 msgr2=0x7feef00cee80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:55:17.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:17.852+0000 7feeeffff700 1 --2- 172.21.15.40:0/1775951266 >> v2:172.21.15.40:3300/0 conn(0x7feef00cb2f0 0x7feef00cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:55:19.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.440+0000 7feeee7fc700 1 -- 172.21.15.40:0/1775951266 >> v2:172.21.15.40:3300/0 conn(0x7feef00cb2f0 msgr2=0x7feef00cee80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:55:19.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.440+0000 7feeee7fc700 1 --2- 172.21.15.40:0/1775951266 >> v2:172.21.15.40:3300/0 conn(0x7feef00cb2f0 0x7feef00cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:19.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.440+0000 7feeee7fc700 1 -- 172.21.15.40:0/1775951266 >> v1:172.21.15.43:6789/0 conn(0x7feef00cf3c0 legacy=0x7feef015cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:19.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.440+0000 7feeee7fc700 1 -- 172.21.15.40:0/1775951266 >> v1:172.21.15.40:6789/0 conn(0x7feef0153880 legacy=0x7feef00c9ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:55:19.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.440+0000 7feeee7fc700 1 --2- 172.21.15.40:0/1775951266 >> v2:172.21.15.43:3300/0 conn(0x7feedc003740 0x7feedc005ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:19.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.440+0000 7feeee7fc700 1 --2- 172.21.15.40:0/1775951266 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 0x7feedc008650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:19.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.440+0000 7feeee7fc700 1 -- 172.21.15.40:0/1775951266 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef0159d30 con 0x7feedc003740 2024-09-13T17:55:19.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.440+0000 7feeee7fc700 1 -- 172.21.15.40:0/1775951266 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc008d70 con 0x7feedc006190 2024-09-13T17:55:19.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.440+0000 7feef4a42700 1 --2- 172.21.15.40:0/1775951266 >> v2:172.21.15.43:3300/0 conn(0x7feedc003740 0x7feedc005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:19.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.440+0000 7feeeffff700 1 --2- 172.21.15.40:0/1775951266 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 0x7feedc008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:19.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.440+0000 7feef4a42700 1 -- 172.21.15.40:0/1775951266 >> v2:172.21.15.43:3300/0 conn(0x7feedc003740 msgr2=0x7feedc005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:55:19.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.440+0000 7feef4a42700 1 -- 172.21.15.40:0/1775951266 >> v2:172.21.15.43:3300/0 conn(0x7feedc003740 msgr2=0x7feedc005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:55:19.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.440+0000 7feef4a42700 1 --2- 172.21.15.40:0/1775951266 >> v2:172.21.15.43:3300/0 conn(0x7feedc003740 0x7feedc005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:55:19.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.440+0000 7feeeffff700 1 -- 172.21.15.40:0/1775951266 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 msgr2=0x7feedc008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:55:19.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.440+0000 7feeeffff700 1 -- 172.21.15.40:0/1775951266 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 msgr2=0x7feedc008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:55:19.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.440+0000 7feeeffff700 1 --2- 172.21.15.40:0/1775951266 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 0x7feedc008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:55:19.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.440+0000 7feef4a42700 1 --2- 172.21.15.40:0/1775951266 >> v2:172.21.15.43:3300/0 conn(0x7feedc003740 0x7feedc005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:19.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.440+0000 7feeeffff700 1 --2- 172.21.15.40:0/1775951266 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 0x7feedc008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:19.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.644+0000 7feeeffff700 1 --2- 172.21.15.40:0/1775951266 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 0x7feedc008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:19.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.644+0000 7feef4a42700 1 --2- 172.21.15.40:0/1775951266 >> v2:172.21.15.43:3300/0 conn(0x7feedc003740 0x7feedc005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.644+0000 7feeeffff700 1 -- 172.21.15.40:0/1775951266 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 msgr2=0x7feedc008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:55:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.644+0000 7feeeffff700 1 -- 172.21.15.40:0/1775951266 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 msgr2=0x7feedc008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:55:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.644+0000 7feeeffff700 1 --2- 172.21.15.40:0/1775951266 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 0x7feedc008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:55:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.644+0000 7feeeffff700 1 --2- 172.21.15.40:0/1775951266 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 0x7feedc008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:55:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.644+0000 7feef4a42700 1 -- 172.21.15.40:0/1775951266 >> v2:172.21.15.43:3300/0 conn(0x7feedc003740 msgr2=0x7feedc005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:55:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.644+0000 7feef4a42700 1 -- 172.21.15.40:0/1775951266 >> v2:172.21.15.43:3300/0 conn(0x7feedc003740 msgr2=0x7feedc005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:55:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.644+0000 7feef4a42700 1 --2- 172.21.15.40:0/1775951266 >> v2:172.21.15.43:3300/0 conn(0x7feedc003740 0x7feedc005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:55:19.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:19.644+0000 7feef4a42700 1 --2- 172.21.15.40:0/1775951266 >> v2:172.21.15.43:3300/0 conn(0x7feedc003740 0x7feedc005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:55:20.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.048+0000 7feeeffff700 1 --2- 172.21.15.40:0/1775951266 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 0x7feedc008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:20.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.048+0000 7feef4a42700 1 --2- 172.21.15.40:0/1775951266 >> v2:172.21.15.43:3300/0 conn(0x7feedc003740 0x7feedc005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:20.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.048+0000 7feef4a42700 1 -- 172.21.15.40:0/1775951266 >> v2:172.21.15.43:3300/0 conn(0x7feedc003740 msgr2=0x7feedc005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:55:20.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.048+0000 7feef4a42700 1 -- 172.21.15.40:0/1775951266 >> v2:172.21.15.43:3300/0 conn(0x7feedc003740 msgr2=0x7feedc005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:55:20.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.048+0000 7feef4a42700 1 --2- 172.21.15.40:0/1775951266 >> v2:172.21.15.43:3300/0 conn(0x7feedc003740 0x7feedc005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:55:20.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.048+0000 7feef4a42700 1 --2- 172.21.15.40:0/1775951266 >> v2:172.21.15.43:3300/0 conn(0x7feedc003740 0x7feedc005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:55:20.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.048+0000 7feeeffff700 1 -- 172.21.15.40:0/1775951266 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 msgr2=0x7feedc008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:55:20.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.048+0000 7feeeffff700 1 -- 172.21.15.40:0/1775951266 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 msgr2=0x7feedc008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:55:20.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.048+0000 7feeeffff700 1 --2- 172.21.15.40:0/1775951266 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 0x7feedc008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:55:20.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.048+0000 7feeeffff700 1 --2- 172.21.15.40:0/1775951266 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 0x7feedc008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:55:20.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.852+0000 7feeeffff700 1 --2- 172.21.15.40:0/1775951266 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 0x7feedc008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:20.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.852+0000 7feef4a42700 1 --2- 172.21.15.40:0/1775951266 >> v2:172.21.15.43:3300/0 conn(0x7feedc003740 0x7feedc005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:20.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.852+0000 7feeeffff700 1 -- 172.21.15.40:0/1775951266 >> v2:172.21.15.43:3300/0 conn(0x7feedc003740 msgr2=0x7feedc005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:55:20.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.852+0000 7feeeffff700 1 --2- 172.21.15.40:0/1775951266 >> v2:172.21.15.43:3300/0 conn(0x7feedc003740 0x7feedc005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:20.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.852+0000 7feeeffff700 1 -- 172.21.15.40:0/1775951266 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef015a010 con 0x7feedc006190 2024-09-13T17:55:20.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.852+0000 7feeeffff700 1 --2- 172.21.15.40:0/1775951266 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 0x7feedc008650 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feee4003a30 tx=0x7feee400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3e3da96d8bbde7c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:20.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.852+0000 7feeef7fe700 1 -- 172.21.15.40:0/1775951266 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4013070 con 0x7feedc006190 2024-09-13T17:55:20.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.852+0000 7feeef7fe700 1 -- 172.21.15.40:0/1775951266 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee40050c0 con 0x7feedc006190 2024-09-13T17:55:20.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.852+0000 7feeef7fe700 1 -- 172.21.15.40:0/1775951266 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4005450 con 0x7feedc006190 2024-09-13T17:55:20.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.852+0000 7feef5a44700 1 -- 172.21.15.40:0/1775951266 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 msgr2=0x7feedc008650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:20.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.852+0000 7feef5a44700 1 --2- 172.21.15.40:0/1775951266 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 0x7feedc008650 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feee4003a30 tx=0x7feee400e720 comp rx=0 tx=0).stop 2024-09-13T17:55:20.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.852+0000 7feef5243700 1 -- 172.21.15.40:0/1775951266 reap_dead start 2024-09-13T17:55:20.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.852+0000 7feef5a44700 1 -- 172.21.15.40:0/1775951266 shutdown_connections 2024-09-13T17:55:20.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.852+0000 7feef5a44700 1 -- 172.21.15.40:0/1775951266 >> 172.21.15.40:0/1775951266 conn(0x7feef006c5b0 msgr2=0x7feef006c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:20.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.852+0000 7feef5a44700 1 -- 172.21.15.40:0/1775951266 shutdown_connections 2024-09-13T17:55:20.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.852+0000 7feef5a44700 1 -- 172.21.15.40:0/1775951266 wait complete. 2024-09-13T17:55:20.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.856+0000 7feef5a44700 1 Processor -- start 2024-09-13T17:55:20.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.856+0000 7feef5a44700 1 -- start start 2024-09-13T17:55:20.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.856+0000 7feef5a44700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 0x7feef0149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:20.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.856+0000 7feef5a44700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef014e530 0x7feef014fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:20.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.856+0000 7feef5a44700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef014f650 0x7feef01501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:20.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.856+0000 7feef5a44700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef0149c90 con 0x7feef014e530 2024-09-13T17:55:20.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.856+0000 7feef5a44700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef0149ec0 con 0x7feef014f650 2024-09-13T17:55:20.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.856+0000 7feef5a44700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef01ef500 con 0x7feedc006190 2024-09-13T17:55:20.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.856+0000 7feeeffff700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef014e530 msgr2=0x7feef014fca0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:55:20.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.856+0000 7feeeffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef014e530 0x7feef014fca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:20.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.860+0000 7feef5243700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef014f650 0x7feef01501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:20.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.860+0000 7feef4a42700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 0x7feef0149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:20.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.860+0000 7feef5243700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef014f650 0x7feef01501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:44684/0 (socket says 172.21.15.40:44684) 2024-09-13T17:55:20.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.860+0000 7feef4a42700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 0x7feef0149360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:39598/0 (socket says 172.21.15.40:39598) 2024-09-13T17:55:20.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.860+0000 7feef5243700 1 -- 172.21.15.40:0/2656326936 learned_addr learned my addr 172.21.15.40:0/2656326936 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:20.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.860+0000 7feef5243700 1 -- 172.21.15.40:0/2656326936 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 msgr2=0x7feef0149360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:20.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.860+0000 7feef5243700 1 --2- 172.21.15.40:0/2656326936 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 0x7feef0149360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:20.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.860+0000 7feef5243700 1 -- 172.21.15.40:0/2656326936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef014e530 msgr2=0x7feef014fca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:55:20.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.860+0000 7feef5243700 1 --2- 172.21.15.40:0/2656326936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef014e530 0x7feef014fca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:20.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.860+0000 7feef5243700 1 -- 172.21.15.40:0/2656326936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee4009ad0 con 0x7feef014f650 2024-09-13T17:55:20.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.860+0000 7feef5243700 1 --2- 172.21.15.40:0/2656326936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef014f650 0x7feef01501e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feedc006160 tx=0x7feee800ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:20.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.860+0000 7feeed7fa700 1 -- 172.21.15.40:0/2656326936 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee8007480 con 0x7feef014f650 2024-09-13T17:55:20.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.860+0000 7feeed7fa700 1 -- 172.21.15.40:0/2656326936 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee80075e0 con 0x7feef014f650 2024-09-13T17:55:20.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.860+0000 7feef5a44700 1 -- 172.21.15.40:0/2656326936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef01ef7e0 con 0x7feef014f650 2024-09-13T17:55:20.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.860+0000 7feeed7fa700 1 -- 172.21.15.40:0/2656326936 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee8007950 con 0x7feef014f650 2024-09-13T17:55:20.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.860+0000 7feef5a44700 1 -- 172.21.15.40:0/2656326936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef01efda0 con 0x7feef014f650 2024-09-13T17:55:20.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.860+0000 7feef5a44700 1 -- 172.21.15.40:0/2656326936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef00cee80 con 0x7feef014f650 2024-09-13T17:55:20.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.864+0000 7feeed7fa700 1 -- 172.21.15.40:0/2656326936 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feee80123b0 con 0x7feef014f650 2024-09-13T17:55:20.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.864+0000 7feeed7fa700 1 --2- 172.21.15.40:0/2656326936 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7feedc067560 0x7feedc069a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:20.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.864+0000 7feef4a42700 1 --2- 172.21.15.40:0/2656326936 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7feedc067560 0x7feedc069a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:20.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.864+0000 7feeed7fa700 1 -- 172.21.15.40:0/2656326936 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7feee8083dd0 con 0x7feef014f650 2024-09-13T17:55:20.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.864+0000 7feef4a42700 1 --2- 172.21.15.40:0/2656326936 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7feedc067560 0x7feedc069a20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7feee0003a30 tx=0x7feee0005040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:20.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:20.876+0000 7feeed7fa700 1 -- 172.21.15.40:0/2656326936 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7feee8054780 con 0x7feef014f650 2024-09-13T17:55:21.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.132+0000 7feef5a44700 1 -- 172.21.15.40:0/2656326936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7feef00caa50 con 0x7feef014f650 2024-09-13T17:55:21.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.132+0000 7feeed7fa700 1 -- 172.21.15.40:0/2656326936 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7feee8085610 con 0x7feef014f650 2024-09-13T17:55:21.137 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":42,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:55:21.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.132+0000 7feef5a44700 1 -- 172.21.15.40:0/2656326936 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7feedc067560 msgr2=0x7feedc069a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:21.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.132+0000 7feef5a44700 1 --2- 172.21.15.40:0/2656326936 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7feedc067560 0x7feedc069a20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7feee0003a30 tx=0x7feee0005040 comp rx=0 tx=0).stop 2024-09-13T17:55:21.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.132+0000 7feef5a44700 1 -- 172.21.15.40:0/2656326936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef014f650 msgr2=0x7feef01501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:21.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.132+0000 7feef5a44700 1 --2- 172.21.15.40:0/2656326936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef014f650 0x7feef01501e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7feedc006160 tx=0x7feee800ca70 comp rx=0 tx=0).stop 2024-09-13T17:55:21.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.132+0000 7feef5a44700 1 -- 172.21.15.40:0/2656326936 shutdown_connections 2024-09-13T17:55:21.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.132+0000 7feef5a44700 1 --2- 172.21.15.40:0/2656326936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef014f650 0x7feef01501e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.132+0000 7feef5a44700 1 --2- 172.21.15.40:0/2656326936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feef014e530 0x7feef014fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.132+0000 7feef5a44700 1 --2- 172.21.15.40:0/2656326936 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7feedc067560 0x7feedc069a20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.132+0000 7feef5a44700 1 --2- 172.21.15.40:0/2656326936 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feedc006190 0x7feef0149360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.132+0000 7feef5a44700 1 -- 172.21.15.40:0/2656326936 >> 172.21.15.40:0/2656326936 conn(0x7feef006c5b0 msgr2=0x7feef00cb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:21.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.136+0000 7feef5a44700 1 -- 172.21.15.40:0/2656326936 shutdown_connections 2024-09-13T17:55:21.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.136+0000 7feef5a44700 1 -- 172.21.15.40:0/2656326936 wait complete. 2024-09-13T17:55:21.154 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T17:55:21.154 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-13T17:55:21.154 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:55:21.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.260+0000 7f7a00f6a700 1 Processor -- start 2024-09-13T17:55:21.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.260+0000 7f7a00f6a700 1 -- start start 2024-09-13T17:55:21.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.260+0000 7f7a00f6a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79f40d8670 0x7f79f40500a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:21.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.260+0000 7f7a00f6a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f79f4006b60 0x7f79f4006f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:21.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.260+0000 7f7a00f6a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79f40e1cf0 0x7f79f40e2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:21.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.260+0000 7f7a00f6a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f4049000 con 0x7f79f40e1cf0 2024-09-13T17:55:21.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.260+0000 7f7a00f6a700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f404f720 con 0x7f79f4006b60 2024-09-13T17:55:21.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.260+0000 7f7a00f6a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f40505e0 con 0x7f79f40d8670 2024-09-13T17:55:21.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.260+0000 7f79fbfff700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f79f40e1cf0 msgr2=0x7f79f40e2160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:55:21.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.260+0000 7f79fbfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f79f40e1cf0 0x7f79f40e2160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:21.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f79fb7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79f40d8670 0x7f79f40500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:21.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f79fb7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79f40d8670 0x7f79f40500a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:39614/0 (socket says 172.21.15.40:39614) 2024-09-13T17:55:21.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f79fb7fe700 1 -- 172.21.15.40:0/1234028522 learned_addr learned my addr 172.21.15.40:0/1234028522 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:21.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f79faffd700 1 --2- 172.21.15.40:0/1234028522 >> v2:172.21.15.43:3300/0 conn(0x7f79f4006b60 0x7f79f4006f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:21.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f79fb7fe700 1 -- 172.21.15.40:0/1234028522 >> v2:172.21.15.43:3300/0 conn(0x7f79f4006b60 msgr2=0x7f79f4006f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:55:21.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f79fb7fe700 1 --2- 172.21.15.40:0/1234028522 >> v2:172.21.15.43:3300/0 conn(0x7f79f4006b60 0x7f79f4006f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f79fb7fe700 1 -- 172.21.15.40:0/1234028522 >> v2:172.21.15.40:3300/0 conn(0x7f79f40e1cf0 msgr2=0x7f79f40e2160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:55:21.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f79fb7fe700 1 --2- 172.21.15.40:0/1234028522 >> v2:172.21.15.40:3300/0 conn(0x7f79f40e1cf0 0x7f79f40e2160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f79fb7fe700 1 -- 172.21.15.40:0/1234028522 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f404f4f0 con 0x7f79f40d8670 2024-09-13T17:55:21.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f79fb7fe700 1 --2- 172.21.15.40:0/1234028522 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79f40d8670 0x7f79f40500a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009670 tx=0x7f79f0009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e37f583becf92bf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:21.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f79fa7fc700 1 -- 172.21.15.40:0/1234028522 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0011830 con 0x7f79f40d8670 2024-09-13T17:55:21.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f79fa7fc700 1 -- 172.21.15.40:0/1234028522 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f0011990 con 0x7f79f40d8670 2024-09-13T17:55:21.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f79fa7fc700 1 -- 172.21.15.40:0/1234028522 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0011d20 con 0x7f79f40d8670 2024-09-13T17:55:21.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f7a00f6a700 1 -- 172.21.15.40:0/1234028522 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79f40d8670 msgr2=0x7f79f40500a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:21.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f7a00f6a700 1 --2- 172.21.15.40:0/1234028522 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79f40d8670 0x7f79f40500a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009670 tx=0x7f79f0009bf0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f7a00f6a700 1 -- 172.21.15.40:0/1234028522 shutdown_connections 2024-09-13T17:55:21.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f7a00f6a700 1 --2- 172.21.15.40:0/1234028522 >> v2:172.21.15.40:3300/0 conn(0x7f79f40e1cf0 0x7f79f40e2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f7a00f6a700 1 --2- 172.21.15.40:0/1234028522 >> v2:172.21.15.43:3300/0 conn(0x7f79f4006b60 0x7f79f4006f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f7a00f6a700 1 --2- 172.21.15.40:0/1234028522 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79f40d8670 0x7f79f40500a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f7a00f6a700 1 -- 172.21.15.40:0/1234028522 >> 172.21.15.40:0/1234028522 conn(0x7f79f4054120 msgr2=0x7f79f4054500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:21.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f7a00f6a700 1 -- 172.21.15.40:0/1234028522 shutdown_connections 2024-09-13T17:55:21.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.264+0000 7f7a00f6a700 1 -- 172.21.15.40:0/1234028522 wait complete. 2024-09-13T17:55:21.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f7a00f6a700 1 Processor -- start 2024-09-13T17:55:21.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f7a00f6a700 1 -- start start 2024-09-13T17:55:21.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f7a00f6a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79f4006b60 0x7f79f40da330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:21.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f7a00f6a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d8670 0x7f79f40d1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:21.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f7a00f6a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79f40e1cf0 0x7f79f4177ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:21.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f79fb7fe700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79f4006b60 msgr2=0x7f79f40da330 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:55:21.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f7a00f6a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f40da870 con 0x7f79f4006b60 2024-09-13T17:55:21.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f7a00f6a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f40d93a0 con 0x7f79f40d8670 2024-09-13T17:55:21.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f7a00f6a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f40d5010 con 0x7f79f40e1cf0 2024-09-13T17:55:21.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f79fb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79f4006b60 0x7f79f40da330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:21.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f79faffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d8670 0x7f79f40d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:21.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f79fbfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79f40e1cf0 0x7f79f4177ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:21.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f79faffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d8670 0x7f79f40d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:44696/0 (socket says 172.21.15.40:44696) 2024-09-13T17:55:21.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f79faffd700 1 -- 172.21.15.40:0/3354914347 learned_addr learned my addr 172.21.15.40:0/3354914347 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:21.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f79faffd700 1 -- 172.21.15.40:0/3354914347 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79f40e1cf0 msgr2=0x7f79f4177ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:21.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f79faffd700 1 --2- 172.21.15.40:0/3354914347 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79f40e1cf0 0x7f79f4177ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f79faffd700 1 -- 172.21.15.40:0/3354914347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79f4006b60 msgr2=0x7f79f40da330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:55:21.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f79faffd700 1 --2- 172.21.15.40:0/3354914347 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79f4006b60 0x7f79f40da330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f79faffd700 1 -- 172.21.15.40:0/3354914347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f0009570 con 0x7f79f40d8670 2024-09-13T17:55:21.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f79faffd700 1 --2- 172.21.15.40:0/3354914347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d8670 0x7f79f40d1650 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009fd0 tx=0x7f79e800eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:21.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f79e7fff700 1 -- 172.21.15.40:0/3354914347 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e80095a0 con 0x7f79f40d8670 2024-09-13T17:55:21.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f79e7fff700 1 -- 172.21.15.40:0/3354914347 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e8013680 con 0x7f79f40d8670 2024-09-13T17:55:21.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f7a00f6a700 1 -- 172.21.15.40:0/3354914347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f40d52f0 con 0x7f79f40d8670 2024-09-13T17:55:21.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f79e7fff700 1 -- 172.21.15.40:0/3354914347 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8013a10 con 0x7f79f40d8670 2024-09-13T17:55:21.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.268+0000 7f7a00f6a700 1 -- 172.21.15.40:0/3354914347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f40d9680 con 0x7f79f40d8670 2024-09-13T17:55:21.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.272+0000 7f7a00f6a700 1 --2- 172.21.15.40:0/3354914347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d28a0 0x7f79f4178420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:21.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.272+0000 7f7a00f6a700 1 -- 172.21.15.40:0/3354914347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f40084b0 con 0x7f79f40d28a0 2024-09-13T17:55:21.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.272+0000 7f79fb7fe700 1 --2- 172.21.15.40:0/3354914347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d28a0 0x7f79f4178420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:21.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.272+0000 7f79fb7fe700 1 -- 172.21.15.40:0/3354914347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f79f4007200 con 0x7f79f40d28a0 2024-09-13T17:55:21.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.272+0000 7f79fb7fe700 1 --2- 172.21.15.40:0/3354914347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d28a0 0x7f79f4178420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009670 tx=0x7f79f0004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:21.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.272+0000 7f79e7fff700 1 -- 172.21.15.40:0/3354914347 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0004450 con 0x7f79f40d28a0 2024-09-13T17:55:21.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.272+0000 7f79e7fff700 1 -- 172.21.15.40:0/3354914347 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f79e8013b70 con 0x7f79f40d8670 2024-09-13T17:55:21.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.272+0000 7f79e7fff700 1 --2- 172.21.15.40:0/3354914347 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f79e00667d0 0x7f79e0068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:21.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.272+0000 7f79fbfff700 1 --2- 172.21.15.40:0/3354914347 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f79e00667d0 0x7f79e0068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:21.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.272+0000 7f79e7fff700 1 -- 172.21.15.40:0/3354914347 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f79e8084730 con 0x7f79f40d8670 2024-09-13T17:55:21.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.276+0000 7f79e7fff700 1 -- 172.21.15.40:0/3354914347 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f79f00045b0 con 0x7f79f40d28a0 2024-09-13T17:55:21.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.276+0000 7f79e7fff700 1 -- 172.21.15.40:0/3354914347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d28a0 msgr2=0x7f79f4178420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:21.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.276+0000 7f79e7fff700 1 --2- 172.21.15.40:0/3354914347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d28a0 0x7f79f4178420 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009670 tx=0x7f79f0004770 comp rx=0 tx=0).stop 2024-09-13T17:55:21.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.276+0000 7f79e7fff700 1 -- 172.21.15.40:0/3354914347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d28a0 msgr2=0x7f79f4178420 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:55:21.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.276+0000 7f79e7fff700 1 --2- 172.21.15.40:0/3354914347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d28a0 0x7f79f4178420 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.276+0000 7f79fbfff700 1 --2- 172.21.15.40:0/3354914347 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f79e00667d0 0x7f79e0068c90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f79fc04f480 tx=0x7f79fc065ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:21.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f7a00f6a700 1 --2- 172.21.15.40:0/3354914347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d5480 0x7f79f40cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:21.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f7a00f6a700 1 -- 172.21.15.40:0/3354914347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f40e1b80 con 0x7f79f40d5480 2024-09-13T17:55:21.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f79fb7fe700 1 --2- 172.21.15.40:0/3354914347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d5480 0x7f79f40cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:21.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f79fb7fe700 1 -- 172.21.15.40:0/3354914347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f79f404f210 con 0x7f79f40d5480 2024-09-13T17:55:21.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f79fb7fe700 1 --2- 172.21.15.40:0/3354914347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d5480 0x7f79f40cbab0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009190 tx=0x7f79f00092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:21.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f79e7fff700 1 -- 172.21.15.40:0/3354914347 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0027b20 con 0x7f79f40d5480 2024-09-13T17:55:21.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f79e7fff700 1 -- 172.21.15.40:0/3354914347 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4938 (secure 0 0 0) 0x7f79f001b430 con 0x7f79f40d5480 2024-09-13T17:55:21.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f79e7fff700 1 -- 172.21.15.40:0/3354914347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d5480 msgr2=0x7f79f40cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:21.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f79e7fff700 1 --2- 172.21.15.40:0/3354914347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d5480 0x7f79f40cbab0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009190 tx=0x7f79f00092e0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f79e7fff700 1 -- 172.21.15.40:0/3354914347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d5480 msgr2=0x7f79f40cbab0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:55:21.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f79e7fff700 1 --2- 172.21.15.40:0/3354914347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d5480 0x7f79f40cbab0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.295 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:55:21.295 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:55:21.295 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:55:21.295 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:55:21.295 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 42, 2024-09-13T17:55:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:55:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:55:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:55:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T17:55:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:55:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:55:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:55:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:55:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:55:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:55:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:55:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:55:21.296 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:55:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:55:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:55:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:55:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:55:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:55:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:55:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:55:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:55:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:55:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:55:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:55:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:55:21.297 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:55:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:55:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:55:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:55:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:55:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:55:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:55:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:55:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:55:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:55:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:55:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:55:21.298 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:55:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:55:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:55:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:55:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:55:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:55:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:55:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:55:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:55:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:55:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:55:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:55:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:55:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:55:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:55:21.299 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:55:21.300 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:55:21.300 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:55:21.300 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:55:21.300 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:21.300 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:55:21.300 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.300 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:55:21.300 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.300 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:55:21.300 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:55:21.300 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:55:21.300 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:55:21.300 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.300 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:55:21.301 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:55:21.301 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:21.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.301 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.301 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:55:21.301 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:55:21.301 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:21.301 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:21.301 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:21.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.301 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:55:21.301 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:55:21.301 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:55:21.301 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:55:21.301 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:55:21.302 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.302 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.302 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:55:21.302 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:55:21.302 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:55:21.302 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:55:21.302 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.302 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:55:21.302 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:55:21.302 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:21.302 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.302 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.302 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:55:21.302 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:55:21.303 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:21.303 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:21.303 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:21.303 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.303 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:55:21.303 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:55:21.303 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:55:21.303 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:55:21.303 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:55:21.303 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.303 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.303 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:55:21.303 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:55:21.303 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:55:21.304 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:55:21.304 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.304 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:55:21.304 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:55:21.304 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:21.304 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.304 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.304 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:55:21.304 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:55:21.304 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:21.304 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:21.304 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:21.304 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.304 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:55:21.305 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:55:21.305 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:55:21.305 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:55:21.305 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:55:21.305 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:21.305 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:21.305 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.305 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:55:21.305 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:55:21.305 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.305 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:55:21.305 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:55:21.305 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:55:21.305 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:21.306 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:21.306 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:55:21.306 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.306 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:55:21.306 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:55:21.306 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:55:21.306 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:21.306 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:21.306 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:55:21.306 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.306 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:55:21.306 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:55:21.306 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:55:21.306 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:21.306 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:21.307 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:55:21.307 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.307 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:55:21.307 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:55:21.307 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:55:21.307 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:21.307 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:21.307 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:55:21.307 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.307 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:55:21.307 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:55:21.307 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:55:21.307 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:21.307 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:21.308 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.308 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:55:21.308 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:55:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f7a00f6a700 1 -- 172.21.15.40:0/3354914347 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f79e00667d0 msgr2=0x7f79e0068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f7a00f6a700 1 --2- 172.21.15.40:0/3354914347 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f79e00667d0 0x7f79e0068c90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f79fc04f480 tx=0x7f79fc065ea0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f7a00f6a700 1 -- 172.21.15.40:0/3354914347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d8670 msgr2=0x7f79f40d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f79fbfff700 1 -- 172.21.15.40:0/3354914347 reap_dead start 2024-09-13T17:55:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f7a00f6a700 1 --2- 172.21.15.40:0/3354914347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d8670 0x7f79f40d1650 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f79e8009fd0 tx=0x7f79e800eb20 comp rx=0 tx=0).stop 2024-09-13T17:55:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f7a00f6a700 1 -- 172.21.15.40:0/3354914347 shutdown_connections 2024-09-13T17:55:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f7a00f6a700 1 --2- 172.21.15.40:0/3354914347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79f40d8670 0x7f79f40d1650 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f7a00f6a700 1 -- 172.21.15.40:0/3354914347 >> 172.21.15.40:0/3354914347 conn(0x7f79f4054120 msgr2=0x7f79f4038be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f7a00f6a700 1 -- 172.21.15.40:0/3354914347 shutdown_connections 2024-09-13T17:55:21.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.288+0000 7f7a00f6a700 1 -- 172.21.15.40:0/3354914347 wait complete. 2024-09-13T17:55:21.310 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T17:55:21.310 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T17:55:21.310 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:55:21.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.424+0000 7f243ffff700 1 Processor -- start 2024-09-13T17:55:21.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.424+0000 7f243ffff700 1 -- start start 2024-09-13T17:55:21.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.424+0000 7f243ffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2440151580 0x7f24400c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:21.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.424+0000 7f243ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f24400b1fb0 0x7f24400c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:21.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.424+0000 7f243ffff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2440070ae0 con 0x7f2440071260 2024-09-13T17:55:21.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.424+0000 7f243ffff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2440070d30 con 0x7f24400b1fb0 2024-09-13T17:55:21.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.424+0000 7f243ffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24400c29c0 con 0x7f2440151580 2024-09-13T17:55:21.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.424+0000 7f243e7fc700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f24400b1fb0 msgr2=0x7f24400c57f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:55:21.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.424+0000 7f243f7fe700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f2440071260 legacy=0x7f2440157800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:55:21.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.424+0000 7f243e7fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f24400b1fb0 0x7f24400c57f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.424+0000 7f243effd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2440151580 0x7f24400c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.424+0000 7f243effd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2440151580 0x7f24400c52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:56270/0 (socket says 172.21.15.40:56270) 2024-09-13T17:55:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.424+0000 7f243effd700 1 -- 172.21.15.40:0/1433394067 learned_addr learned my addr 172.21.15.40:0/1433394067 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.428+0000 7f243effd700 1 -- 172.21.15.40:0/1433394067 >> v2:172.21.15.40:3300/0 conn(0x7f24400b1fb0 msgr2=0x7f24400c57f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:55:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.428+0000 7f243effd700 1 --2- 172.21.15.40:0/1433394067 >> v2:172.21.15.40:3300/0 conn(0x7f24400b1fb0 0x7f24400c57f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.428+0000 7f243effd700 1 -- 172.21.15.40:0/1433394067 >> v1:172.21.15.40:6789/0 conn(0x7f2440071260 legacy=0x7f2440157800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:55:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.428+0000 7f243effd700 1 -- 172.21.15.40:0/1433394067 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24400c2ca0 con 0x7f2440151580 2024-09-13T17:55:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.428+0000 7f243effd700 1 --2- 172.21.15.40:0/1433394067 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2440151580 0x7f24400c52b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2430009910 tx=0x7f2430009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60c3aab418eb1f88 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.428+0000 7f243dffb700 1 -- 172.21.15.40:0/1433394067 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430004230 con 0x7f2440151580 2024-09-13T17:55:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.428+0000 7f243dffb700 1 -- 172.21.15.40:0/1433394067 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2430004390 con 0x7f2440151580 2024-09-13T17:55:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.428+0000 7f243dffb700 1 -- 172.21.15.40:0/1433394067 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430009e60 con 0x7f2440151580 2024-09-13T17:55:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.428+0000 7f243ffff700 1 -- 172.21.15.40:0/1433394067 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2440151580 msgr2=0x7f24400c52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:21.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.428+0000 7f243ffff700 1 --2- 172.21.15.40:0/1433394067 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2440151580 0x7f24400c52b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2430009910 tx=0x7f2430009c20 comp rx=0 tx=0).stop 2024-09-13T17:55:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.428+0000 7f243ffff700 1 -- 172.21.15.40:0/1433394067 shutdown_connections 2024-09-13T17:55:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.428+0000 7f243ffff700 1 --2- 172.21.15.40:0/1433394067 >> v2:172.21.15.40:3300/0 conn(0x7f24400b1fb0 0x7f24400c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.428+0000 7f243ffff700 1 --2- 172.21.15.40:0/1433394067 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2440151580 0x7f24400c52b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.428+0000 7f243ffff700 1 -- 172.21.15.40:0/1433394067 >> 172.21.15.40:0/1433394067 conn(0x7f244006e6d0 msgr2=0x7f244006eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.428+0000 7f243ffff700 1 -- 172.21.15.40:0/1433394067 shutdown_connections 2024-09-13T17:55:21.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.428+0000 7f243ffff700 1 -- 172.21.15.40:0/1433394067 wait complete. 2024-09-13T17:55:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.428+0000 7f243ffff700 1 Processor -- start 2024-09-13T17:55:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.428+0000 7f243ffff700 1 -- start start 2024-09-13T17:55:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243ffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2440071260 0x7f2440159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24400b1fb0 0x7f2440159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2440151580 0x7f2440147400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:21.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243ffff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24400ae2a0 con 0x7f24400b1fb0 2024-09-13T17:55:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243ffff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2440158e20 con 0x7f2440151580 2024-09-13T17:55:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243ffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24400af0f0 con 0x7f2440071260 2024-09-13T17:55:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243e7fc700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24400b1fb0 msgr2=0x7f2440159240 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:55:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24400b1fb0 0x7f2440159240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243effd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2440071260 0x7f2440159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2440151580 0x7f2440147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243effd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2440071260 0x7f2440159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:56284/0 (socket says 172.21.15.40:56284) 2024-09-13T17:55:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243effd700 1 -- 172.21.15.40:0/3304433582 learned_addr learned my addr 172.21.15.40:0/3304433582 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243effd700 1 -- 172.21.15.40:0/3304433582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2440151580 msgr2=0x7f2440147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243effd700 1 --2- 172.21.15.40:0/3304433582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2440151580 0x7f2440147400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243effd700 1 -- 172.21.15.40:0/3304433582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24400b1fb0 msgr2=0x7f2440159240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:55:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243effd700 1 --2- 172.21.15.40:0/3304433582 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24400b1fb0 0x7f2440159240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243effd700 1 -- 172.21.15.40:0/3304433582 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2430009570 con 0x7f2440071260 2024-09-13T17:55:21.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243effd700 1 --2- 172.21.15.40:0/3304433582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2440071260 0x7f2440159ef0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2430004000 tx=0x7f2430004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f24277fe700 1 -- 172.21.15.40:0/3304433582 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24300036c0 con 0x7f2440071260 2024-09-13T17:55:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f24277fe700 1 -- 172.21.15.40:0/3304433582 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2430003820 con 0x7f2440071260 2024-09-13T17:55:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243ffff700 1 -- 172.21.15.40:0/3304433582 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24400aecc0 con 0x7f2440071260 2024-09-13T17:55:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f24277fe700 1 -- 172.21.15.40:0/3304433582 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430003b90 con 0x7f2440071260 2024-09-13T17:55:21.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243ffff700 1 -- 172.21.15.40:0/3304433582 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2440158480 con 0x7f2440071260 2024-09-13T17:55:21.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f24277fe700 1 -- 172.21.15.40:0/3304433582 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2430003cf0 con 0x7f2440071260 2024-09-13T17:55:21.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243ffff700 1 --2- 172.21.15.40:0/3304433582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2440159780 0x7f2440147940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:21.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.432+0000 7f243ffff700 1 -- 172.21.15.40:0/3304433582 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2440067600 con 0x7f2440159780 2024-09-13T17:55:21.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.436+0000 7f24277fe700 1 --2- 172.21.15.40:0/3304433582 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f242c0666e0 0x7f242c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.436+0000 7f243e7fc700 1 --2- 172.21.15.40:0/3304433582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2440159780 0x7f2440147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.436+0000 7f24277fe700 1 -- 172.21.15.40:0/3304433582 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2430085c80 con 0x7f2440071260 2024-09-13T17:55:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.436+0000 7f243f7fe700 1 --2- 172.21.15.40:0/3304433582 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f242c0666e0 0x7f242c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.436+0000 7f243e7fc700 1 -- 172.21.15.40:0/3304433582 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2440151960 con 0x7f2440159780 2024-09-13T17:55:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.436+0000 7f243e7fc700 1 --2- 172.21.15.40:0/3304433582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2440159780 0x7f2440147940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f244006eb20 tx=0x7f2434009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.436+0000 7f243f7fe700 1 --2- 172.21.15.40:0/3304433582 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f242c0666e0 0x7f242c068ba0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2438003c90 tx=0x7f243800d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.436+0000 7f24277fe700 1 -- 172.21.15.40:0/3304433582 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243400ca80 con 0x7f2440159780 2024-09-13T17:55:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.436+0000 7f24277fe700 1 -- 172.21.15.40:0/3304433582 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f243400cbe0 con 0x7f2440159780 2024-09-13T17:55:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.436+0000 7f24277fe700 1 -- 172.21.15.40:0/3304433582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2440159780 msgr2=0x7f2440147940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.436+0000 7f24277fe700 1 --2- 172.21.15.40:0/3304433582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2440159780 0x7f2440147940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f244006eb20 tx=0x7f2434009f10 comp rx=0 tx=0).stop 2024-09-13T17:55:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.436+0000 7f24277fe700 1 -- 172.21.15.40:0/3304433582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2440159780 msgr2=0x7f2440147940 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:55:21.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.436+0000 7f24277fe700 1 --2- 172.21.15.40:0/3304433582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2440159780 0x7f2440147940 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.444+0000 7f243ffff700 1 --2- 172.21.15.40:0/3304433582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24400c8aa0 0x7f2440158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:21.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.444+0000 7f243ffff700 1 -- 172.21.15.40:0/3304433582 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2440071840 con 0x7f24400c8aa0 2024-09-13T17:55:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.444+0000 7f243e7fc700 1 --2- 172.21.15.40:0/3304433582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24400c8aa0 0x7f2440158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.444+0000 7f243e7fc700 1 -- 172.21.15.40:0/3304433582 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2440159090 con 0x7f24400c8aa0 2024-09-13T17:55:21.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.444+0000 7f243e7fc700 1 --2- 172.21.15.40:0/3304433582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24400c8aa0 0x7f2440158640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f244006eb20 tx=0x7f2434005ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.444+0000 7f24277fe700 1 -- 172.21.15.40:0/3304433582 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434019070 con 0x7f24400c8aa0 2024-09-13T17:55:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.444+0000 7f24277fe700 1 -- 172.21.15.40:0/3304433582 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4770 (secure 0 0 0) 0x7f243401a990 con 0x7f24400c8aa0 2024-09-13T17:55:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.444+0000 7f24277fe700 1 -- 172.21.15.40:0/3304433582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24400c8aa0 msgr2=0x7f2440158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.444+0000 7f24277fe700 1 --2- 172.21.15.40:0/3304433582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24400c8aa0 0x7f2440158640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f244006eb20 tx=0x7f2434005ac0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.444+0000 7f24277fe700 1 -- 172.21.15.40:0/3304433582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24400c8aa0 msgr2=0x7f2440158640 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:55:21.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.444+0000 7f24277fe700 1 --2- 172.21.15.40:0/3304433582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24400c8aa0 0x7f2440158640 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:21.452 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:55:21.453 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:55:21.453 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:55:21.453 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:55:21.453 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 42, 2024-09-13T17:55:21.453 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:55:21.453 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:55:21.453 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:55:21.453 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:21.453 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T17:55:21.453 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:55:21.453 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:55:21.453 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:55:21.453 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:55:21.454 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:55:21.454 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:55:21.454 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:55:21.454 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:55:21.454 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:55:21.454 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:55:21.454 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:55:21.454 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:55:21.454 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:55:21.454 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:21.454 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:55:21.454 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:55:21.454 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:55:21.454 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:55:21.455 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:55:21.455 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:55:21.455 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:55:21.455 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:55:21.455 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:55:21.455 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:55:21.455 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:55:21.455 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:55:21.455 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:21.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.455 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:55:21.455 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:55:21.455 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:55:21.455 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:55:21.456 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:55:21.456 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:55:21.456 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:55:21.456 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:55:21.456 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:55:21.456 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:55:21.456 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:55:21.456 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:55:21.456 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:55:21.456 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:55:21.456 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:55:21.456 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:55:21.456 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:55:21.457 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:55:21.457 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:55:21.457 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:55:21.457 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:55:21.457 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:55:21.457 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:55:21.457 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:55:21.457 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:55:21.457 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:55:21.457 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:55:21.457 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:21.457 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:55:21.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.457 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:55:21.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.458 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:55:21.458 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:55:21.458 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:55:21.458 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:55:21.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.458 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:55:21.458 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:55:21.458 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:21.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.458 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:55:21.458 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:55:21.458 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:21.459 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:21.459 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:21.459 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.459 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:55:21.459 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:55:21.459 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:55:21.459 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:55:21.459 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:55:21.459 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.459 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.459 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:55:21.459 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:55:21.459 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:55:21.459 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:55:21.460 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.460 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:55:21.460 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:55:21.460 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:21.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.460 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.460 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:55:21.460 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:55:21.460 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:21.460 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:21.460 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:21.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.460 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:55:21.460 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:55:21.461 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:55:21.461 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:55:21.461 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:55:21.461 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.461 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.461 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:55:21.461 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:55:21.461 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:55:21.461 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:55:21.461 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.461 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:55:21.461 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:55:21.461 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:21.461 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.461 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.462 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:55:21.462 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:55:21.462 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:55:21.462 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:21.462 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:21.462 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.462 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:55:21.462 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:55:21.462 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:55:21.462 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:55:21.462 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:55:21.462 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:21.462 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:21.462 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.463 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:55:21.463 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:55:21.463 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.463 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:55:21.463 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:55:21.463 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:55:21.463 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:21.463 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:21.463 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:55:21.463 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.463 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:55:21.463 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:55:21.463 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T17:55:21.463 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:21.464 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:21.464 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:55:21.464 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.464 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:55:21.464 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:55:21.464 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T17:55:21.464 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:21.464 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:55:21.464 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:55:21.464 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:55:21.464 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:55:21.464 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:55:21.464 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:55:21.464 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:55:21.465 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:55:21.465 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:55:21.465 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:55:21.465 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:55:21.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.448+0000 7f243ffff700 1 -- 172.21.15.40:0/3304433582 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f242c0666e0 msgr2=0x7f242c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:21.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.448+0000 7f243ffff700 1 --2- 172.21.15.40:0/3304433582 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f242c0666e0 0x7f242c068ba0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2438003c90 tx=0x7f243800d040 comp rx=0 tx=0).stop 2024-09-13T17:55:21.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.448+0000 7f243ffff700 1 -- 172.21.15.40:0/3304433582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2440071260 msgr2=0x7f2440159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:21.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.448+0000 7f243ffff700 1 --2- 172.21.15.40:0/3304433582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2440071260 0x7f2440159ef0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2430004000 tx=0x7f2430004770 comp rx=0 tx=0).stop 2024-09-13T17:55:21.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.448+0000 7f243f7fe700 1 -- 172.21.15.40:0/3304433582 reap_dead start 2024-09-13T17:55:21.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.448+0000 7f243f7fe700 1 -- 172.21.15.40:0/3304433582 reap_dead start 2024-09-13T17:55:21.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.448+0000 7f243ffff700 1 -- 172.21.15.40:0/3304433582 shutdown_connections 2024-09-13T17:55:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.448+0000 7f243ffff700 1 -- 172.21.15.40:0/3304433582 >> 172.21.15.40:0/3304433582 conn(0x7f244006e6d0 msgr2=0x7f24400c8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.448+0000 7f243ffff700 1 -- 172.21.15.40:0/3304433582 shutdown_connections 2024-09-13T17:55:21.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:21.448+0000 7f243ffff700 1 -- 172.21.15.40:0/3304433582 wait complete. 2024-09-13T17:55:21.467 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T17:55:21.467 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T17:55:34.627 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:55:34.629 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:55:41.468 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T17:55:41.468 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-13T17:55:41.468 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T17:55:41.468 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T17:55:41.471 INFO:tasks.ceph.mon.a:Started 2024-09-13T17:55:41.471 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T17:55:41.471 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:55:41.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.583+0000 7f290a321700 1 Processor -- start 2024-09-13T17:55:41.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.583+0000 7f290a321700 1 -- start start 2024-09-13T17:55:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f290a321700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2904071f70 0x7f29040c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f290a321700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2904072350 0x7f2904150d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f290a321700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29040c55c0 con 0x7f29040c7200 2024-09-13T17:55:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f290a321700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f290406a390 con 0x7f2904072350 2024-09-13T17:55:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f290a321700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2904157710 con 0x7f2904071f70 2024-09-13T17:55:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f2908b1e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2904072350 0x7f2904150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:41.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f2909b20700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f29040c7200 0x7f290415a700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42012/0 (socket says 172.21.15.40:42012) 2024-09-13T17:55:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f2909b20700 1 -- 172.21.15.40:0/2272678409 learned_addr learned my addr 172.21.15.40:0/2272678409 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:55:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f2908b1e700 1 --2- 172.21.15.40:0/2272678409 >> v2:172.21.15.40:3300/0 conn(0x7f2904072350 0x7f2904150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53564/0 (socket says 172.21.15.40:53564) 2024-09-13T17:55:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f2908b1e700 1 -- 172.21.15.40:0/2272678409 >> v2:172.21.15.40:3300/0 conn(0x7f2904072350 msgr2=0x7f2904150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:55:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f2908b1e700 1 -- 172.21.15.40:0/2272678409 >> v2:172.21.15.40:3300/0 conn(0x7f2904072350 msgr2=0x7f2904150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:55:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f2908b1e700 1 --2- 172.21.15.40:0/2272678409 >> v2:172.21.15.40:3300/0 conn(0x7f2904072350 0x7f2904150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:55:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f290931f700 1 --2- 172.21.15.40:0/2272678409 >> v2:172.21.15.43:3300/0 conn(0x7f2904071f70 0x7f29040c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f2908b1e700 1 --2- 172.21.15.40:0/2272678409 >> v2:172.21.15.40:3300/0 conn(0x7f2904072350 0x7f2904150d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f290931f700 1 -- 172.21.15.40:0/2272678409 >> v2:172.21.15.40:3300/0 conn(0x7f2904072350 msgr2=0x7f2904150d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:55:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f290931f700 1 --2- 172.21.15.40:0/2272678409 >> v2:172.21.15.40:3300/0 conn(0x7f2904072350 0x7f2904150d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f290931f700 1 -- 172.21.15.40:0/2272678409 >> v1:172.21.15.40:6789/0 conn(0x7f29040c7200 legacy=0x7f290415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:41.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f290931f700 1 -- 172.21.15.40:0/2272678409 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29041579d0 con 0x7f2904071f70 2024-09-13T17:55:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f290931f700 1 --2- 172.21.15.40:0/2272678409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2904071f70 0x7f29040c5d30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f29040672a0 tx=0x7f28f4009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27dbf47713ae3b95 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f28fbfff700 1 -- 172.21.15.40:0/2272678409 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4011830 con 0x7f2904071f70 2024-09-13T17:55:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f28fbfff700 1 -- 172.21.15.40:0/2272678409 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f4011990 con 0x7f2904071f70 2024-09-13T17:55:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f28fbfff700 1 -- 172.21.15.40:0/2272678409 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4011d40 con 0x7f2904071f70 2024-09-13T17:55:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f290a321700 1 -- 172.21.15.40:0/2272678409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2904071f70 msgr2=0x7f29040c5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f290a321700 1 --2- 172.21.15.40:0/2272678409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2904071f70 0x7f29040c5d30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f29040672a0 tx=0x7f28f4009b30 comp rx=0 tx=0).stop 2024-09-13T17:55:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f290a321700 1 -- 172.21.15.40:0/2272678409 shutdown_connections 2024-09-13T17:55:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f290a321700 1 --2- 172.21.15.40:0/2272678409 >> v2:172.21.15.40:3300/0 conn(0x7f2904072350 0x7f2904150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:41.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f290a321700 1 --2- 172.21.15.40:0/2272678409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2904071f70 0x7f29040c5d30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:41.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f290a321700 1 -- 172.21.15.40:0/2272678409 >> 172.21.15.40:0/2272678409 conn(0x7f29040cabc0 msgr2=0x7f29040cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:41.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f290a321700 1 -- 172.21.15.40:0/2272678409 shutdown_connections 2024-09-13T17:55:41.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.587+0000 7f290a321700 1 -- 172.21.15.40:0/2272678409 wait complete. 2024-09-13T17:55:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f290a321700 1 Processor -- start 2024-09-13T17:55:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f290a321700 1 -- start start 2024-09-13T17:55:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f290a321700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2904071f70 0x7f290414a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f290a321700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2904072350 0x7f2904145e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f290a321700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29040c7200 0x7f2904146340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f290a321700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2904156a50 con 0x7f2904071f70 2024-09-13T17:55:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f290a321700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2904147ef0 con 0x7f29040c7200 2024-09-13T17:55:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f290a321700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29041ecc30 con 0x7f2904072350 2024-09-13T17:55:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f290931f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2904071f70 0x7f290414a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:41.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f2909b20700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29040c7200 0x7f2904146340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f290931f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2904071f70 0x7f290414a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53574/0 (socket says 172.21.15.40:53574) 2024-09-13T17:55:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f290931f700 1 -- 172.21.15.40:0/2456851589 learned_addr learned my addr 172.21.15.40:0/2456851589 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f290931f700 1 -- 172.21.15.40:0/2456851589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2904071f70 msgr2=0x7f290414a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T17:55:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f290931f700 1 -- 172.21.15.40:0/2456851589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2904071f70 msgr2=0x7f290414a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:55:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f290931f700 1 --2- 172.21.15.40:0/2456851589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2904071f70 0x7f290414a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:55:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f2908b1e700 1 --2- 172.21.15.40:0/2456851589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2904072350 0x7f2904145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f290931f700 1 --2- 172.21.15.40:0/2456851589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2904071f70 0x7f290414a5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f2909b20700 1 -- 172.21.15.40:0/2456851589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2904072350 msgr2=0x7f2904145e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f2909b20700 1 --2- 172.21.15.40:0/2456851589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2904072350 0x7f2904145e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f2909b20700 1 -- 172.21.15.40:0/2456851589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2904071f70 msgr2=0x7f290414a5e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:55:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f2909b20700 1 --2- 172.21.15.40:0/2456851589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2904071f70 0x7f290414a5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f2909b20700 1 -- 172.21.15.40:0/2456851589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f40094b0 con 0x7f29040c7200 2024-09-13T17:55:41.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.591+0000 7f2909b20700 1 --2- 172.21.15.40:0/2456851589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29040c7200 0x7f2904146340 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00caa0 tx=0x7f28fc00cad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.595+0000 7f28f9ffb700 1 -- 172.21.15.40:0/2456851589 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc0074e0 con 0x7f29040c7200 2024-09-13T17:55:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.595+0000 7f28f9ffb700 1 -- 172.21.15.40:0/2456851589 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28fc007640 con 0x7f29040c7200 2024-09-13T17:55:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.595+0000 7f290a321700 1 -- 172.21.15.40:0/2456851589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29041ecef0 con 0x7f29040c7200 2024-09-13T17:55:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.595+0000 7f28f9ffb700 1 -- 172.21.15.40:0/2456851589 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc0079b0 con 0x7f29040c7200 2024-09-13T17:55:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.595+0000 7f290a321700 1 -- 172.21.15.40:0/2456851589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29041ed4b0 con 0x7f29040c7200 2024-09-13T17:55:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.595+0000 7f290a321700 1 -- 172.21.15.40:0/2456851589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f290414e570 con 0x7f29040c7200 2024-09-13T17:55:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.595+0000 7f28f9ffb700 1 -- 172.21.15.40:0/2456851589 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f28fc01b3b0 con 0x7f29040c7200 2024-09-13T17:55:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.595+0000 7f28f9ffb700 1 --2- 172.21.15.40:0/2456851589 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f28e8066760 0x7f28e8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.595+0000 7f28f9ffb700 1 -- 172.21.15.40:0/2456851589 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f28fc083ca0 con 0x7f29040c7200 2024-09-13T17:55:41.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.595+0000 7f290931f700 1 --2- 172.21.15.40:0/2456851589 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f28e8066760 0x7f28e8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:41.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.599+0000 7f290931f700 1 --2- 172.21.15.40:0/2456851589 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f28e8066760 0x7f28e8068c20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f28f4000c00 tx=0x7f28f4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:41.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.607+0000 7f28f9ffb700 1 -- 172.21.15.40:0/2456851589 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f28fc050d00 con 0x7f29040c7200 2024-09-13T17:55:41.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.863+0000 7f290a321700 1 -- 172.21.15.40:0/2456851589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f29041e0f00 con 0x7f29040c7200 2024-09-13T17:55:41.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.863+0000 7f28f9ffb700 1 -- 172.21.15.40:0/2456851589 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f28fc017080 con 0x7f29040c7200 2024-09-13T17:55:41.872 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":42,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:55:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.867+0000 7f290a321700 1 -- 172.21.15.40:0/2456851589 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f28e8066760 msgr2=0x7f28e8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.867+0000 7f290a321700 1 --2- 172.21.15.40:0/2456851589 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f28e8066760 0x7f28e8068c20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f28f4000c00 tx=0x7f28f4008030 comp rx=0 tx=0).stop 2024-09-13T17:55:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.867+0000 7f290a321700 1 -- 172.21.15.40:0/2456851589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29040c7200 msgr2=0x7f2904146340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.867+0000 7f290a321700 1 --2- 172.21.15.40:0/2456851589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29040c7200 0x7f2904146340 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00caa0 tx=0x7f28fc00cad0 comp rx=0 tx=0).stop 2024-09-13T17:55:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.867+0000 7f290a321700 1 -- 172.21.15.40:0/2456851589 shutdown_connections 2024-09-13T17:55:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.867+0000 7f290a321700 1 --2- 172.21.15.40:0/2456851589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29040c7200 0x7f2904146340 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.867+0000 7f290a321700 1 --2- 172.21.15.40:0/2456851589 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f28e8066760 0x7f28e8068c20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.867+0000 7f290a321700 1 --2- 172.21.15.40:0/2456851589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2904072350 0x7f2904145e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.867+0000 7f290a321700 1 --2- 172.21.15.40:0/2456851589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2904071f70 0x7f290414a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.867+0000 7f290a321700 1 -- 172.21.15.40:0/2456851589 >> 172.21.15.40:0/2456851589 conn(0x7f29040cabc0 msgr2=0x7f290414fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.867+0000 7f290a321700 1 -- 172.21.15.40:0/2456851589 shutdown_connections 2024-09-13T17:55:41.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:41.867+0000 7f290a321700 1 -- 172.21.15.40:0/2456851589 wait complete. 2024-09-13T17:55:44.888 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:55:45.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:44.999+0000 7f37a32d1700 1 Processor -- start 2024-09-13T17:55:45.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:44.999+0000 7f37a32d1700 1 -- start start 2024-09-13T17:55:45.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a32d1700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f379c156bc0 0x7f379c15a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:45.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a32d1700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f379c15a5a0 con 0x7f379c071ec0 2024-09-13T17:55:45.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a32d1700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f379c0c6b40 con 0x7f379c1502f0 2024-09-13T17:55:45.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a32d1700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f379c0c3060 con 0x7f379c156bc0 2024-09-13T17:55:45.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a1ace700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f379c071ec0 0x7f379c1564b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42024/0 (socket says 172.21.15.40:42024) 2024-09-13T17:55:45.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a1ace700 1 -- 172.21.15.40:0/4292864873 learned_addr learned my addr 172.21.15.40:0/4292864873 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:55:45.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a2ad0700 1 --2- 172.21.15.40:0/4292864873 >> v2:172.21.15.43:3300/0 conn(0x7f379c156bc0 0x7f379c15a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:45.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a2ad0700 1 -- 172.21.15.40:0/4292864873 >> v1:172.21.15.43:6789/0 conn(0x7f379c1502f0 legacy=0x7f379c0c2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:45.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a2ad0700 1 -- 172.21.15.40:0/4292864873 >> v1:172.21.15.40:6789/0 conn(0x7f379c071ec0 legacy=0x7f379c1564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:45.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a2ad0700 1 -- 172.21.15.40:0/4292864873 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c0c6e20 con 0x7f379c156bc0 2024-09-13T17:55:45.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a2ad0700 1 --2- 172.21.15.40:0/4292864873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f379c156bc0 0x7f379c15a060 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3794009d90 tx=0x7f379400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7d530d6dba13ef8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:45.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a12cd700 1 -- 172.21.15.40:0/4292864873 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379400e070 con 0x7f379c156bc0 2024-09-13T17:55:45.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a12cd700 1 -- 172.21.15.40:0/4292864873 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379400bbd0 con 0x7f379c156bc0 2024-09-13T17:55:45.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a12cd700 1 -- 172.21.15.40:0/4292864873 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3794012430 con 0x7f379c156bc0 2024-09-13T17:55:45.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a32d1700 1 -- 172.21.15.40:0/4292864873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f379c156bc0 msgr2=0x7f379c15a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:45.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a32d1700 1 --2- 172.21.15.40:0/4292864873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f379c156bc0 0x7f379c15a060 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3794009d90 tx=0x7f379400b330 comp rx=0 tx=0).stop 2024-09-13T17:55:45.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a32d1700 1 -- 172.21.15.40:0/4292864873 shutdown_connections 2024-09-13T17:55:45.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a32d1700 1 --2- 172.21.15.40:0/4292864873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f379c156bc0 0x7f379c15a060 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:45.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a32d1700 1 -- 172.21.15.40:0/4292864873 >> 172.21.15.40:0/4292864873 conn(0x7f379c06e480 msgr2=0x7f379c06e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:45.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a32d1700 1 -- 172.21.15.40:0/4292864873 shutdown_connections 2024-09-13T17:55:45.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.003+0000 7f37a32d1700 1 -- 172.21.15.40:0/4292864873 wait complete. 2024-09-13T17:55:45.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.007+0000 7f37a32d1700 1 Processor -- start 2024-09-13T17:55:45.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.007+0000 7f37a32d1700 1 -- start start 2024-09-13T17:55:45.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.007+0000 7f37a32d1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f379c071ec0 0x7f379c158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:45.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.007+0000 7f37a32d1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f379c1502f0 0x7f379c155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:45.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.007+0000 7f37a32d1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f379c156bc0 0x7f379c1f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:45.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.007+0000 7f37a32d1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c1577a0 con 0x7f379c1502f0 2024-09-13T17:55:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.007+0000 7f37a32d1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c1f3000 con 0x7f379c156bc0 2024-09-13T17:55:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.007+0000 7f37a32d1700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379c06b070 con 0x7f379c071ec0 2024-09-13T17:55:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.007+0000 7f37a1ace700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f379c1502f0 0x7f379c155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.007+0000 7f37a1ace700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f379c1502f0 0x7f379c155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53584/0 (socket says 172.21.15.40:53584) 2024-09-13T17:55:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.007+0000 7f37a1ace700 1 -- 172.21.15.40:0/887383724 learned_addr learned my addr 172.21.15.40:0/887383724 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.007+0000 7f37a1ace700 1 -- 172.21.15.40:0/887383724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f379c1502f0 msgr2=0x7f379c155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-13T17:55:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.007+0000 7f37a1ace700 1 -- 172.21.15.40:0/887383724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f379c1502f0 msgr2=0x7f379c155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:55:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.007+0000 7f37a1ace700 1 --2- 172.21.15.40:0/887383724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f379c1502f0 0x7f379c155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:55:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.007+0000 7f37a22cf700 1 --2- 172.21.15.40:0/887383724 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f379c071ec0 0x7f379c158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.007+0000 7f37a2ad0700 1 --2- 172.21.15.40:0/887383724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f379c156bc0 0x7f379c1f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:45.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.007+0000 7f37a1ace700 1 --2- 172.21.15.40:0/887383724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f379c1502f0 0x7f379c155050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:45.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.011+0000 7f37a22cf700 1 -- 172.21.15.40:0/887383724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f379c156bc0 msgr2=0x7f379c1f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.011+0000 7f37a22cf700 1 --2- 172.21.15.40:0/887383724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f379c156bc0 0x7f379c1f3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.011+0000 7f37a22cf700 1 -- 172.21.15.40:0/887383724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f379c1502f0 msgr2=0x7f379c155050 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:55:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.011+0000 7f37a22cf700 1 --2- 172.21.15.40:0/887383724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f379c1502f0 0x7f379c155050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.011+0000 7f37a22cf700 1 -- 172.21.15.40:0/887383724 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3794009a70 con 0x7f379c071ec0 2024-09-13T17:55:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.011+0000 7f37a2ad0700 1 --2- 172.21.15.40:0/887383724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f379c156bc0 0x7f379c1f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:55:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.011+0000 7f37a22cf700 1 --2- 172.21.15.40:0/887383724 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f379c071ec0 0x7f379c158280 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f379800c480 tx=0x7f379800c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.011+0000 7f3792ffd700 1 -- 172.21.15.40:0/887383724 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37980052d0 con 0x7f379c071ec0 2024-09-13T17:55:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.011+0000 7f3792ffd700 1 -- 172.21.15.40:0/887383724 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3798005430 con 0x7f379c071ec0 2024-09-13T17:55:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.011+0000 7f3792ffd700 1 -- 172.21.15.40:0/887383724 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37980055a0 con 0x7f379c071ec0 2024-09-13T17:55:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.011+0000 7f37a32d1700 1 -- 172.21.15.40:0/887383724 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379c1ef860 con 0x7f379c071ec0 2024-09-13T17:55:45.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.011+0000 7f37a32d1700 1 -- 172.21.15.40:0/887383724 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379c06a510 con 0x7f379c071ec0 2024-09-13T17:55:45.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.011+0000 7f37a32d1700 1 -- 172.21.15.40:0/887383724 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379c1506d0 con 0x7f379c071ec0 2024-09-13T17:55:45.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.011+0000 7f3792ffd700 1 -- 172.21.15.40:0/887383724 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f37980038e0 con 0x7f379c071ec0 2024-09-13T17:55:45.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.015+0000 7f3792ffd700 1 --2- 172.21.15.40:0/887383724 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3780066730 0x7f3780068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:45.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.015+0000 7f3792ffd700 1 -- 172.21.15.40:0/887383724 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3798083800 con 0x7f379c071ec0 2024-09-13T17:55:45.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.015+0000 7f37a1ace700 1 --2- 172.21.15.40:0/887383724 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3780066730 0x7f3780068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:45.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.015+0000 7f37a1ace700 1 --2- 172.21.15.40:0/887383724 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3780066730 0x7f3780068bf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f378c007fd0 tx=0x7f378c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.019+0000 7f3792ffd700 1 -- 172.21.15.40:0/887383724 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3798050840 con 0x7f379c071ec0 2024-09-13T17:55:45.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.279+0000 7f37a32d1700 1 -- 172.21.15.40:0/887383724 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f379c0b1600 con 0x7f379c071ec0 2024-09-13T17:55:45.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.279+0000 7f3792ffd700 1 -- 172.21.15.40:0/887383724 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f3798017610 con 0x7f379c071ec0 2024-09-13T17:55:45.288 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":42,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":24,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:55:45.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.283+0000 7f37a32d1700 1 -- 172.21.15.40:0/887383724 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3780066730 msgr2=0x7f3780068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:45.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.283+0000 7f37a32d1700 1 --2- 172.21.15.40:0/887383724 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3780066730 0x7f3780068bf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f378c007fd0 tx=0x7f378c009040 comp rx=0 tx=0).stop 2024-09-13T17:55:45.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.283+0000 7f37a32d1700 1 -- 172.21.15.40:0/887383724 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f379c071ec0 msgr2=0x7f379c158280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:45.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.283+0000 7f37a32d1700 1 --2- 172.21.15.40:0/887383724 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f379c071ec0 0x7f379c158280 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f379800c480 tx=0x7f379800c790 comp rx=0 tx=0).stop 2024-09-13T17:55:45.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.283+0000 7f37a32d1700 1 -- 172.21.15.40:0/887383724 shutdown_connections 2024-09-13T17:55:45.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.283+0000 7f37a32d1700 1 --2- 172.21.15.40:0/887383724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f379c156bc0 0x7f379c1f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:45.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.283+0000 7f37a32d1700 1 --2- 172.21.15.40:0/887383724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f379c1502f0 0x7f379c155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:45.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.283+0000 7f37a32d1700 1 --2- 172.21.15.40:0/887383724 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3780066730 0x7f3780068bf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:45.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.283+0000 7f37a32d1700 1 --2- 172.21.15.40:0/887383724 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f379c071ec0 0x7f379c158280 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:45.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.283+0000 7f37a32d1700 1 -- 172.21.15.40:0/887383724 >> 172.21.15.40:0/887383724 conn(0x7f379c06e480 msgr2=0x7f379c149d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:45.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.283+0000 7f37a32d1700 1 -- 172.21.15.40:0/887383724 shutdown_connections 2024-09-13T17:55:45.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:45.283+0000 7f37a32d1700 1 -- 172.21.15.40:0/887383724 wait complete. 2024-09-13T17:55:48.309 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:55:48.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.431+0000 7fc8e8bd3700 1 Processor -- start 2024-09-13T17:55:48.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.431+0000 7fc8e8bd3700 1 -- start start 2024-09-13T17:55:48.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.431+0000 7fc8e8bd3700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc8e40c9ef0 0x7fc8e415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:48.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.431+0000 7fc8e8bd3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8e40ca2d0 0x7fc8e415d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:48.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.431+0000 7fc8e8bd3700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e415cd10 con 0x7fc8e40c7880 2024-09-13T17:55:48.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.431+0000 7fc8e8bd3700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e415a720 con 0x7fc8e40c9ef0 2024-09-13T17:55:48.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.431+0000 7fc8e8bd3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e406ecd0 con 0x7fc8e40ca2d0 2024-09-13T17:55:48.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.431+0000 7fc8e2d9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc8e40c7880 0x7fc8e4153500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42028/0 (socket says 172.21.15.40:42028) 2024-09-13T17:55:48.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.431+0000 7fc8e2d9d700 1 -- 172.21.15.40:0/2257270785 learned_addr learned my addr 172.21.15.40:0/2257270785 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:55:48.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.431+0000 7fc8e259c700 1 --2- 172.21.15.40:0/2257270785 >> v2:172.21.15.43:3300/0 conn(0x7fc8e40c9ef0 0x7fc8e415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:48.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.431+0000 7fc8e1d9b700 1 --2- 172.21.15.40:0/2257270785 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8e40ca2d0 0x7fc8e415d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:48.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.431+0000 7fc8e259c700 1 -- 172.21.15.40:0/2257270785 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8e40ca2d0 msgr2=0x7fc8e415d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:55:48.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.431+0000 7fc8e259c700 1 --2- 172.21.15.40:0/2257270785 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8e40ca2d0 0x7fc8e415d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:48.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.431+0000 7fc8e259c700 1 -- 172.21.15.40:0/2257270785 >> v1:172.21.15.40:6789/0 conn(0x7fc8e40c7880 legacy=0x7fc8e4153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:48.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.431+0000 7fc8e259c700 1 -- 172.21.15.40:0/2257270785 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e406efb0 con 0x7fc8e40c9ef0 2024-09-13T17:55:48.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.431+0000 7fc8e259c700 1 --2- 172.21.15.40:0/2257270785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e40c9ef0 0x7fc8e415a1e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009880 tx=0x7fc8d4009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0b717e6a62b94c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:48.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.431+0000 7fc8e159a700 1 -- 172.21.15.40:0/2257270785 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4011830 con 0x7fc8e40c9ef0 2024-09-13T17:55:48.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e159a700 1 -- 172.21.15.40:0/2257270785 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d4011990 con 0x7fc8e40c9ef0 2024-09-13T17:55:48.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e159a700 1 -- 172.21.15.40:0/2257270785 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4011d20 con 0x7fc8e40c9ef0 2024-09-13T17:55:48.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e8bd3700 1 -- 172.21.15.40:0/2257270785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e40c9ef0 msgr2=0x7fc8e415a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:48.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e8bd3700 1 --2- 172.21.15.40:0/2257270785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e40c9ef0 0x7fc8e415a1e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009880 tx=0x7fc8d4009b90 comp rx=0 tx=0).stop 2024-09-13T17:55:48.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e8bd3700 1 -- 172.21.15.40:0/2257270785 shutdown_connections 2024-09-13T17:55:48.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e8bd3700 1 --2- 172.21.15.40:0/2257270785 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8e40ca2d0 0x7fc8e415d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:48.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e8bd3700 1 --2- 172.21.15.40:0/2257270785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e40c9ef0 0x7fc8e415a1e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:48.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e8bd3700 1 -- 172.21.15.40:0/2257270785 >> 172.21.15.40:0/2257270785 conn(0x7fc8e4066270 msgr2=0x7fc8e4066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:48.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e8bd3700 1 -- 172.21.15.40:0/2257270785 shutdown_connections 2024-09-13T17:55:48.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e8bd3700 1 -- 172.21.15.40:0/2257270785 wait complete. 2024-09-13T17:55:48.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e8bd3700 1 Processor -- start 2024-09-13T17:55:48.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e8bd3700 1 -- start start 2024-09-13T17:55:48.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e8bd3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e40c7880 0x7fc8e41591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:48.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e8bd3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8e40c9ef0 0x7fc8e4146aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:48.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e8bd3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e40ca2d0 0x7fc8e4146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:48.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e8bd3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40cd600 con 0x7fc8e40c7880 2024-09-13T17:55:48.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e8bd3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40c4280 con 0x7fc8e40ca2d0 2024-09-13T17:55:48.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e8bd3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40caf60 con 0x7fc8e40c9ef0 2024-09-13T17:55:48.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e40c7880 0x7fc8e41591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:48.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e40c7880 0x7fc8e41591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53586/0 (socket says 172.21.15.40:53586) 2024-09-13T17:55:48.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e259c700 1 -- 172.21.15.40:0/3156161090 learned_addr learned my addr 172.21.15.40:0/3156161090 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:48.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e1d9b700 1 --2- 172.21.15.40:0/3156161090 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8e40c9ef0 0x7fc8e4146aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:48.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e259c700 1 -- 172.21.15.40:0/3156161090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e40c7880 msgr2=0x7fc8e41591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T17:55:48.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e2d9d700 1 --2- 172.21.15.40:0/3156161090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e40ca2d0 0x7fc8e4146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:48.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e259c700 1 -- 172.21.15.40:0/3156161090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e40c7880 msgr2=0x7fc8e41591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:55:48.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e259c700 1 --2- 172.21.15.40:0/3156161090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e40c7880 0x7fc8e41591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:55:48.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.435+0000 7fc8e259c700 1 --2- 172.21.15.40:0/3156161090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e40c7880 0x7fc8e41591f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:48.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.439+0000 7fc8e2d9d700 1 -- 172.21.15.40:0/3156161090 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8e40c9ef0 msgr2=0x7fc8e4146aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:48.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.439+0000 7fc8e2d9d700 1 --2- 172.21.15.40:0/3156161090 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8e40c9ef0 0x7fc8e4146aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:48.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.439+0000 7fc8e2d9d700 1 -- 172.21.15.40:0/3156161090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e40c7880 msgr2=0x7fc8e41591f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:55:48.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.439+0000 7fc8e2d9d700 1 --2- 172.21.15.40:0/3156161090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e40c7880 0x7fc8e41591f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:48.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.439+0000 7fc8e2d9d700 1 -- 172.21.15.40:0/3156161090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d40094b0 con 0x7fc8e40ca2d0 2024-09-13T17:55:48.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.439+0000 7fc8e2d9d700 1 --2- 172.21.15.40:0/3156161090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e40ca2d0 0x7fc8e4146fe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00ca70 tx=0x7fc8dc00caa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:48.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.439+0000 7fc8d2ffd700 1 -- 172.21.15.40:0/3156161090 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc007480 con 0x7fc8e40ca2d0 2024-09-13T17:55:48.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.439+0000 7fc8d2ffd700 1 -- 172.21.15.40:0/3156161090 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8dc0075e0 con 0x7fc8e40ca2d0 2024-09-13T17:55:48.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.439+0000 7fc8d2ffd700 1 -- 172.21.15.40:0/3156161090 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc007950 con 0x7fc8e40ca2d0 2024-09-13T17:55:48.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.439+0000 7fc8e8bd3700 1 -- 172.21.15.40:0/3156161090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e40cac80 con 0x7fc8e40ca2d0 2024-09-13T17:55:48.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.439+0000 7fc8e8bd3700 1 -- 172.21.15.40:0/3156161090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e40cbf20 con 0x7fc8e40ca2d0 2024-09-13T17:55:48.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.439+0000 7fc8e8bd3700 1 -- 172.21.15.40:0/3156161090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e40c44a0 con 0x7fc8e40ca2d0 2024-09-13T17:55:48.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.439+0000 7fc8d2ffd700 1 -- 172.21.15.40:0/3156161090 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc8dc01b3b0 con 0x7fc8e40ca2d0 2024-09-13T17:55:48.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.443+0000 7fc8d2ffd700 1 --2- 172.21.15.40:0/3156161090 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8c4066760 0x7fc8c4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:48.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.443+0000 7fc8d2ffd700 1 -- 172.21.15.40:0/3156161090 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc8dc083c00 con 0x7fc8e40ca2d0 2024-09-13T17:55:48.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.443+0000 7fc8e259c700 1 --2- 172.21.15.40:0/3156161090 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8c4066760 0x7fc8c4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:48.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.443+0000 7fc8e259c700 1 --2- 172.21.15.40:0/3156161090 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8c4066760 0x7fc8c4068c20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009fd0 tx=0x7fc8d4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:48.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.451+0000 7fc8d2ffd700 1 -- 172.21.15.40:0/3156161090 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc8e40c44a0 con 0x7fc8e40ca2d0 2024-09-13T17:55:48.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.715+0000 7fc8e8bd3700 1 -- 172.21.15.40:0/3156161090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc8e40c44a0 con 0x7fc8e40ca2d0 2024-09-13T17:55:48.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.715+0000 7fc8d2ffd700 1 -- 172.21.15.40:0/3156161090 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7fc8e40c44a0 con 0x7fc8e40ca2d0 2024-09-13T17:55:48.723 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":42,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":28,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:55:48.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.719+0000 7fc8e8bd3700 1 -- 172.21.15.40:0/3156161090 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8c4066760 msgr2=0x7fc8c4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:48.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.719+0000 7fc8e8bd3700 1 --2- 172.21.15.40:0/3156161090 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8c4066760 0x7fc8c4068c20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009fd0 tx=0x7fc8d4008030 comp rx=0 tx=0).stop 2024-09-13T17:55:48.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.719+0000 7fc8e8bd3700 1 -- 172.21.15.40:0/3156161090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e40ca2d0 msgr2=0x7fc8e4146fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:48.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.719+0000 7fc8e8bd3700 1 --2- 172.21.15.40:0/3156161090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e40ca2d0 0x7fc8e4146fe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00ca70 tx=0x7fc8dc00caa0 comp rx=0 tx=0).stop 2024-09-13T17:55:48.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.719+0000 7fc8e8bd3700 1 -- 172.21.15.40:0/3156161090 shutdown_connections 2024-09-13T17:55:48.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.719+0000 7fc8e8bd3700 1 --2- 172.21.15.40:0/3156161090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e40ca2d0 0x7fc8e4146fe0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:48.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.719+0000 7fc8e8bd3700 1 --2- 172.21.15.40:0/3156161090 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8c4066760 0x7fc8c4068c20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:48.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.719+0000 7fc8e8bd3700 1 --2- 172.21.15.40:0/3156161090 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8e40c9ef0 0x7fc8e4146aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:48.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.719+0000 7fc8e8bd3700 1 --2- 172.21.15.40:0/3156161090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e40c7880 0x7fc8e41591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:48.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.719+0000 7fc8e8bd3700 1 -- 172.21.15.40:0/3156161090 >> 172.21.15.40:0/3156161090 conn(0x7fc8e4066270 msgr2=0x7fc8e415c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:48.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.719+0000 7fc8e8bd3700 1 -- 172.21.15.40:0/3156161090 shutdown_connections 2024-09-13T17:55:48.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:48.719+0000 7fc8e8bd3700 1 -- 172.21.15.40:0/3156161090 wait complete. 2024-09-13T17:55:51.744 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:55:51.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.871+0000 7ffb02153700 1 Processor -- start 2024-09-13T17:55:51.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.871+0000 7ffb02153700 1 -- start start 2024-09-13T17:55:51.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.871+0000 7ffb02153700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffafc159010 0x7ffafc0c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:51.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.871+0000 7ffb02153700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffafc0c8ab0 0x7ffafc0c2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:51.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.871+0000 7ffb02153700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffafc0c7400 con 0x7ffafc0c2a40 2024-09-13T17:55:51.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.871+0000 7ffb02153700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffafc0c4ae0 con 0x7ffafc0c8ab0 2024-09-13T17:55:51.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.871+0000 7ffb02153700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffafc0c6990 con 0x7ffafc159010 2024-09-13T17:55:51.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.871+0000 7ffb00950700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffafc0c8ab0 0x7ffafc0c2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:51.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.871+0000 7ffb00950700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffafc0c8ab0 0x7ffafc0c2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41840/0 (socket says 172.21.15.40:41840) 2024-09-13T17:55:51.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.871+0000 7ffb00950700 1 -- 172.21.15.40:0/4234179570 learned_addr learned my addr 172.21.15.40:0/4234179570 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:51.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.871+0000 7ffb00950700 1 -- 172.21.15.40:0/4234179570 >> v2:172.21.15.40:3300/0 conn(0x7ffafc0c8ab0 msgr2=0x7ffafc0c2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:55:51.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.871+0000 7ffb00950700 1 -- 172.21.15.40:0/4234179570 >> v2:172.21.15.40:3300/0 conn(0x7ffafc0c8ab0 msgr2=0x7ffafc0c2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:55:51.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.871+0000 7ffb00950700 1 --2- 172.21.15.40:0/4234179570 >> v2:172.21.15.40:3300/0 conn(0x7ffafc0c8ab0 0x7ffafc0c2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:55:51.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.871+0000 7ffb00950700 1 --2- 172.21.15.40:0/4234179570 >> v2:172.21.15.40:3300/0 conn(0x7ffafc0c8ab0 0x7ffafc0c2500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:51.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.871+0000 7ffb01151700 1 --2- 172.21.15.40:0/4234179570 >> v2:172.21.15.43:3300/0 conn(0x7ffafc159010 0x7ffafc0c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:51.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.875+0000 7ffb01151700 1 -- 172.21.15.40:0/4234179570 >> v2:172.21.15.40:3300/0 conn(0x7ffafc0c8ab0 msgr2=0x7ffafc0c2500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:55:51.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.875+0000 7ffb01151700 1 --2- 172.21.15.40:0/4234179570 >> v2:172.21.15.40:3300/0 conn(0x7ffafc0c8ab0 0x7ffafc0c2500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:51.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.875+0000 7ffb01151700 1 -- 172.21.15.40:0/4234179570 >> v1:172.21.15.43:6789/0 conn(0x7ffafc0c2a40 legacy=0x7ffafc0c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:51.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.875+0000 7ffb01151700 1 -- 172.21.15.40:0/4234179570 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc0c4520 con 0x7ffafc159010 2024-09-13T17:55:51.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.875+0000 7ffb01151700 1 --2- 172.21.15.40:0/4234179570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc159010 0x7ffafc0c6dc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffafc071610 tx=0x7ffaf8009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a5ff0ff380fcba8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:51.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.875+0000 7ffaf3fff700 1 -- 172.21.15.40:0/4234179570 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8011830 con 0x7ffafc159010 2024-09-13T17:55:51.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.875+0000 7ffaf3fff700 1 -- 172.21.15.40:0/4234179570 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf8011990 con 0x7ffafc159010 2024-09-13T17:55:51.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.875+0000 7ffaf3fff700 1 -- 172.21.15.40:0/4234179570 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8011d40 con 0x7ffafc159010 2024-09-13T17:55:51.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.875+0000 7ffb02153700 1 -- 172.21.15.40:0/4234179570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc159010 msgr2=0x7ffafc0c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:51.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.875+0000 7ffb02153700 1 --2- 172.21.15.40:0/4234179570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc159010 0x7ffafc0c6dc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffafc071610 tx=0x7ffaf8009b30 comp rx=0 tx=0).stop 2024-09-13T17:55:51.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.875+0000 7ffb02153700 1 -- 172.21.15.40:0/4234179570 shutdown_connections 2024-09-13T17:55:51.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.875+0000 7ffb02153700 1 --2- 172.21.15.40:0/4234179570 >> v2:172.21.15.40:3300/0 conn(0x7ffafc0c8ab0 0x7ffafc0c2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:51.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.875+0000 7ffb02153700 1 --2- 172.21.15.40:0/4234179570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc159010 0x7ffafc0c6dc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:51.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.875+0000 7ffb02153700 1 -- 172.21.15.40:0/4234179570 >> 172.21.15.40:0/4234179570 conn(0x7ffafc0ccef0 msgr2=0x7ffafc0cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:51.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.875+0000 7ffb02153700 1 -- 172.21.15.40:0/4234179570 shutdown_connections 2024-09-13T17:55:51.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.875+0000 7ffb02153700 1 -- 172.21.15.40:0/4234179570 wait complete. 2024-09-13T17:55:51.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb02153700 1 Processor -- start 2024-09-13T17:55:51.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb02153700 1 -- start start 2024-09-13T17:55:51.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb02153700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffafc0c2a40 0x7ffafc1efcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:51.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb02153700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffafc0c8ab0 0x7ffafc156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:51.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb02153700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc159010 0x7ffafc157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:51.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb02153700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc0cb440 con 0x7ffafc0c2a40 2024-09-13T17:55:51.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb02153700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc0cb970 con 0x7ffafc159010 2024-09-13T17:55:51.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb02153700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc1ef820 con 0x7ffafc0c8ab0 2024-09-13T17:55:51.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb01151700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffafc0c2a40 0x7ffafc1efcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:51.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb01151700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffafc0c2a40 0x7ffafc1efcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41848/0 (socket says 172.21.15.40:41848) 2024-09-13T17:55:51.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb01151700 1 -- 172.21.15.40:0/2478654364 learned_addr learned my addr 172.21.15.40:0/2478654364 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:51.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb01151700 1 -- 172.21.15.40:0/2478654364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffafc0c2a40 msgr2=0x7ffafc1efcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T17:55:51.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb01151700 1 -- 172.21.15.40:0/2478654364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffafc0c2a40 msgr2=0x7ffafc1efcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:55:51.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb01151700 1 --2- 172.21.15.40:0/2478654364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffafc0c2a40 0x7ffafc1efcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:55:51.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb01151700 1 --2- 172.21.15.40:0/2478654364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffafc0c2a40 0x7ffafc1efcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:51.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb00950700 1 --2- 172.21.15.40:0/2478654364 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffafc0c8ab0 0x7ffafc156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:51.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb01952700 1 --2- 172.21.15.40:0/2478654364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc159010 0x7ffafc157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:51.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb01952700 1 -- 172.21.15.40:0/2478654364 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffafc0c8ab0 msgr2=0x7ffafc156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:51.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb01952700 1 --2- 172.21.15.40:0/2478654364 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffafc0c8ab0 0x7ffafc156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:51.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb01952700 1 -- 172.21.15.40:0/2478654364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffafc0c2a40 msgr2=0x7ffafc1efcb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:55:51.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb01952700 1 --2- 172.21.15.40:0/2478654364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffafc0c2a40 0x7ffafc1efcb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:51.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb01952700 1 -- 172.21.15.40:0/2478654364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf80094b0 con 0x7ffafc159010 2024-09-13T17:55:51.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.879+0000 7ffb00950700 1 --2- 172.21.15.40:0/2478654364 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffafc0c8ab0 0x7ffafc156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:55:51.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.883+0000 7ffb01952700 1 --2- 172.21.15.40:0/2478654364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc159010 0x7ffafc157af0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400caa0 tx=0x7ffaf400cdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:51.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.883+0000 7ffaf1ffb700 1 -- 172.21.15.40:0/2478654364 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4003c20 con 0x7ffafc159010 2024-09-13T17:55:51.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.883+0000 7ffaf1ffb700 1 -- 172.21.15.40:0/2478654364 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf4003d80 con 0x7ffafc159010 2024-09-13T17:55:51.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.883+0000 7ffb02153700 1 -- 172.21.15.40:0/2478654364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffafc0cbc30 con 0x7ffafc159010 2024-09-13T17:55:51.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.883+0000 7ffaf1ffb700 1 -- 172.21.15.40:0/2478654364 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400a9e0 con 0x7ffafc159010 2024-09-13T17:55:51.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.883+0000 7ffb02153700 1 -- 172.21.15.40:0/2478654364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffafc0cb690 con 0x7ffafc159010 2024-09-13T17:55:51.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.883+0000 7ffb02153700 1 -- 172.21.15.40:0/2478654364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffafc0c95a0 con 0x7ffafc159010 2024-09-13T17:55:51.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.883+0000 7ffaf1ffb700 1 -- 172.21.15.40:0/2478654364 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffaf400ab40 con 0x7ffafc159010 2024-09-13T17:55:51.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.883+0000 7ffaf1ffb700 1 --2- 172.21.15.40:0/2478654364 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffae0066730 0x7ffae0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:51.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.887+0000 7ffaf1ffb700 1 -- 172.21.15.40:0/2478654364 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ffaf4084d80 con 0x7ffafc159010 2024-09-13T17:55:51.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.887+0000 7ffb01151700 1 --2- 172.21.15.40:0/2478654364 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffae0066730 0x7ffae0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:51.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.887+0000 7ffb01151700 1 --2- 172.21.15.40:0/2478654364 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffae0066730 0x7ffae0068bf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8000c00 tx=0x7ffaf8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:51.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:51.895+0000 7ffaf1ffb700 1 -- 172.21.15.40:0/2478654364 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ffaf4051e10 con 0x7ffafc159010 2024-09-13T17:55:52.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:52.163+0000 7ffb02153700 1 -- 172.21.15.40:0/2478654364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ffafc0c3a10 con 0x7ffafc159010 2024-09-13T17:55:52.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:52.167+0000 7ffaf1ffb700 1 -- 172.21.15.40:0/2478654364 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7ffaf4055430 con 0x7ffafc159010 2024-09-13T17:55:52.173 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":42,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":31,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:55:52.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:52.167+0000 7ffb02153700 1 -- 172.21.15.40:0/2478654364 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffae0066730 msgr2=0x7ffae0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:52.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:52.167+0000 7ffb02153700 1 --2- 172.21.15.40:0/2478654364 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffae0066730 0x7ffae0068bf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8000c00 tx=0x7ffaf8008030 comp rx=0 tx=0).stop 2024-09-13T17:55:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:52.167+0000 7ffb02153700 1 -- 172.21.15.40:0/2478654364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc159010 msgr2=0x7ffafc157af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:52.167+0000 7ffb02153700 1 --2- 172.21.15.40:0/2478654364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc159010 0x7ffafc157af0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400caa0 tx=0x7ffaf400cdb0 comp rx=0 tx=0).stop 2024-09-13T17:55:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:52.171+0000 7ffb02153700 1 -- 172.21.15.40:0/2478654364 shutdown_connections 2024-09-13T17:55:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:52.171+0000 7ffb02153700 1 --2- 172.21.15.40:0/2478654364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc159010 0x7ffafc157af0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:52.171+0000 7ffb02153700 1 --2- 172.21.15.40:0/2478654364 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffae0066730 0x7ffae0068bf0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:52.171+0000 7ffb02153700 1 --2- 172.21.15.40:0/2478654364 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffafc0c8ab0 0x7ffafc156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:52.171+0000 7ffb02153700 1 --2- 172.21.15.40:0/2478654364 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffafc0c2a40 0x7ffafc1efcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:52.171+0000 7ffb02153700 1 -- 172.21.15.40:0/2478654364 >> 172.21.15.40:0/2478654364 conn(0x7ffafc0ccef0 msgr2=0x7ffafc0c1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:52.171+0000 7ffb02153700 1 -- 172.21.15.40:0/2478654364 shutdown_connections 2024-09-13T17:55:52.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:52.171+0000 7ffb02153700 1 -- 172.21.15.40:0/2478654364 wait complete. 2024-09-13T17:55:55.194 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:55:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.311+0000 7faac1c45700 1 Processor -- start 2024-09-13T17:55:55.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.311+0000 7faac1c45700 1 -- start start 2024-09-13T17:55:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.311+0000 7faac1c45700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faab40d8920 0x7faab4049090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.311+0000 7faac1c45700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab4054df0 con 0x7faab40e1c00 2024-09-13T17:55:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.311+0000 7faac1c45700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab4006ff0 con 0x7faab4007480 2024-09-13T17:55:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.311+0000 7faac1c45700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab4050a10 con 0x7faab40d8920 2024-09-13T17:55:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.311+0000 7faac1444700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7faab40e1c00 0x7faab40e2070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43356/0 (socket says 172.21.15.40:43356) 2024-09-13T17:55:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.311+0000 7faac1444700 1 -- 172.21.15.40:0/1079292820 learned_addr learned my addr 172.21.15.40:0/1079292820 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:55:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.311+0000 7faac0c43700 1 --2- 172.21.15.40:0/1079292820 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faab40d8920 0x7faab4049090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:55.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.315+0000 7faabb7fe700 1 -- 172.21.15.40:0/1079292820 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559228761 0 0) 0x7faab4006ff0 con 0x7faab4007480 2024-09-13T17:55:55.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.315+0000 7faabb7fe700 1 -- 172.21.15.40:0/1079292820 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaac0035d0 con 0x7faab4007480 2024-09-13T17:55:55.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.315+0000 7faac0c43700 1 -- 172.21.15.40:0/1079292820 >> v1:172.21.15.43:6789/0 conn(0x7faab4007480 legacy=0x7faab404efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:55.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.315+0000 7faac0c43700 1 -- 172.21.15.40:0/1079292820 >> v1:172.21.15.40:6789/0 conn(0x7faab40e1c00 legacy=0x7faab40e2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:55.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.315+0000 7faac0c43700 1 -- 172.21.15.40:0/1079292820 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab40495d0 con 0x7faab40d8920 2024-09-13T17:55:55.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.315+0000 7faac0c43700 1 --2- 172.21.15.40:0/1079292820 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faab40d8920 0x7faab4049090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faab0009880 tx=0x7faab0009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff16f2d47ae8c6b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.315+0000 7faabb7fe700 1 -- 172.21.15.40:0/1079292820 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab0011830 con 0x7faab40d8920 2024-09-13T17:55:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.315+0000 7faabb7fe700 1 -- 172.21.15.40:0/1079292820 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab0011990 con 0x7faab40d8920 2024-09-13T17:55:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.315+0000 7faabb7fe700 1 -- 172.21.15.40:0/1079292820 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab0011d20 con 0x7faab40d8920 2024-09-13T17:55:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.315+0000 7faac1c45700 1 -- 172.21.15.40:0/1079292820 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faab40d8920 msgr2=0x7faab4049090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.315+0000 7faac1c45700 1 --2- 172.21.15.40:0/1079292820 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faab40d8920 0x7faab4049090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faab0009880 tx=0x7faab0009b90 comp rx=0 tx=0).stop 2024-09-13T17:55:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.315+0000 7faac1c45700 1 -- 172.21.15.40:0/1079292820 shutdown_connections 2024-09-13T17:55:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.315+0000 7faac1c45700 1 --2- 172.21.15.40:0/1079292820 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faab40d8920 0x7faab4049090 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:55.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.315+0000 7faac1c45700 1 -- 172.21.15.40:0/1079292820 >> 172.21.15.40:0/1079292820 conn(0x7faab4054110 msgr2=0x7faab4054520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.315+0000 7faac1c45700 1 -- 172.21.15.40:0/1079292820 shutdown_connections 2024-09-13T17:55:55.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.315+0000 7faac1c45700 1 -- 172.21.15.40:0/1079292820 wait complete. 2024-09-13T17:55:55.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.315+0000 7faac1c45700 1 Processor -- start 2024-09-13T17:55:55.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.315+0000 7faac1c45700 1 -- start start 2024-09-13T17:55:55.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faac1c45700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faab4007480 0x7faab40395c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:55.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faac1c45700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faab40d8920 0x7faab4174660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:55.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faac1c45700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faab40e1c00 0x7faab4174ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:55.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faac1c45700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab4174240 con 0x7faab4007480 2024-09-13T17:55:55.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faac1c45700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab40d0480 con 0x7faab40e1c00 2024-09-13T17:55:55.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faac1c45700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab40cd690 con 0x7faab40d8920 2024-09-13T17:55:55.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faac0c43700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faab4007480 0x7faab40395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:55.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faac0c43700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faab4007480 0x7faab40395c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41852/0 (socket says 172.21.15.40:41852) 2024-09-13T17:55:55.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faac0c43700 1 -- 172.21.15.40:0/3098459807 learned_addr learned my addr 172.21.15.40:0/3098459807 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:55.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faabbfff700 1 --2- 172.21.15.40:0/3098459807 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faab40d8920 0x7faab4174660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:55.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faac1444700 1 --2- 172.21.15.40:0/3098459807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faab40e1c00 0x7faab4174ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:55.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faac0c43700 1 -- 172.21.15.40:0/3098459807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faab4007480 msgr2=0x7faab40395c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T17:55:55.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faac0c43700 1 -- 172.21.15.40:0/3098459807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faab4007480 msgr2=0x7faab40395c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:55:55.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faac0c43700 1 --2- 172.21.15.40:0/3098459807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faab4007480 0x7faab40395c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:55:55.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faac0c43700 1 --2- 172.21.15.40:0/3098459807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faab4007480 0x7faab40395c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:55.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faabbfff700 1 -- 172.21.15.40:0/3098459807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faab40e1c00 msgr2=0x7faab4174ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:55.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faabbfff700 1 --2- 172.21.15.40:0/3098459807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faab40e1c00 0x7faab4174ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:55.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faabbfff700 1 -- 172.21.15.40:0/3098459807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faab4007480 msgr2=0x7faab40395c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:55:55.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faabbfff700 1 --2- 172.21.15.40:0/3098459807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faab4007480 0x7faab40395c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:55.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faabbfff700 1 -- 172.21.15.40:0/3098459807 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab00094b0 con 0x7faab40d8920 2024-09-13T17:55:55.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faabbfff700 1 --2- 172.21.15.40:0/3098459807 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faab40d8920 0x7faab4174660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faaa800c500 tx=0x7faaa800c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:55.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faab97fa700 1 -- 172.21.15.40:0/3098459807 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa80052d0 con 0x7faab40d8920 2024-09-13T17:55:55.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faab97fa700 1 -- 172.21.15.40:0/3098459807 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa8005430 con 0x7faab40d8920 2024-09-13T17:55:55.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faac1c45700 1 -- 172.21.15.40:0/3098459807 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab40cd970 con 0x7faab40d8920 2024-09-13T17:55:55.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faab97fa700 1 -- 172.21.15.40:0/3098459807 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa8003930 con 0x7faab40d8920 2024-09-13T17:55:55.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faac1c45700 1 -- 172.21.15.40:0/3098459807 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab40d0180 con 0x7faab40d8920 2024-09-13T17:55:55.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.319+0000 7faac1c45700 1 -- 172.21.15.40:0/3098459807 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab41744b0 con 0x7faab40d8920 2024-09-13T17:55:55.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.323+0000 7faab97fa700 1 -- 172.21.15.40:0/3098459807 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faaa8003a90 con 0x7faab40d8920 2024-09-13T17:55:55.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.323+0000 7faab97fa700 1 --2- 172.21.15.40:0/3098459807 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faaa0066730 0x7faaa0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:55.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.323+0000 7faab97fa700 1 -- 172.21.15.40:0/3098459807 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7faaa80838d0 con 0x7faab40d8920 2024-09-13T17:55:55.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.323+0000 7faac0c43700 1 --2- 172.21.15.40:0/3098459807 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faaa0066730 0x7faaa0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:55.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.323+0000 7faac0c43700 1 --2- 172.21.15.40:0/3098459807 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faaa0066730 0x7faaa0068bf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faab00097d0 tx=0x7faab0010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:55.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.331+0000 7faab97fa700 1 -- 172.21.15.40:0/3098459807 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7faaa8050910 con 0x7faab40d8920 2024-09-13T17:55:55.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.599+0000 7faac1c45700 1 -- 172.21.15.40:0/3098459807 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7faab40cd7d0 con 0x7faab40d8920 2024-09-13T17:55:55.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.599+0000 7faab97fa700 1 -- 172.21.15.40:0/3098459807 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7faaa8017610 con 0x7faab40d8920 2024-09-13T17:55:55.607 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":42,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":34,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:55:55.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.603+0000 7faac1c45700 1 -- 172.21.15.40:0/3098459807 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faaa0066730 msgr2=0x7faaa0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:55.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.603+0000 7faac1c45700 1 --2- 172.21.15.40:0/3098459807 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faaa0066730 0x7faaa0068bf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faab00097d0 tx=0x7faab0010000 comp rx=0 tx=0).stop 2024-09-13T17:55:55.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.603+0000 7faac1c45700 1 -- 172.21.15.40:0/3098459807 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faab40d8920 msgr2=0x7faab4174660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:55.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.603+0000 7faac1c45700 1 --2- 172.21.15.40:0/3098459807 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faab40d8920 0x7faab4174660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faaa800c500 tx=0x7faaa800c810 comp rx=0 tx=0).stop 2024-09-13T17:55:55.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.603+0000 7faac1c45700 1 -- 172.21.15.40:0/3098459807 shutdown_connections 2024-09-13T17:55:55.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.603+0000 7faac1c45700 1 --2- 172.21.15.40:0/3098459807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faab40e1c00 0x7faab4174ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:55.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.603+0000 7faac1c45700 1 --2- 172.21.15.40:0/3098459807 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faaa0066730 0x7faaa0068bf0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:55.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.603+0000 7faac1c45700 1 --2- 172.21.15.40:0/3098459807 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faab40d8920 0x7faab4174660 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:55.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.603+0000 7faac1c45700 1 --2- 172.21.15.40:0/3098459807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faab4007480 0x7faab40395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:55.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.603+0000 7faac1c45700 1 -- 172.21.15.40:0/3098459807 >> 172.21.15.40:0/3098459807 conn(0x7faab4054110 msgr2=0x7faab4036d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:55.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.603+0000 7faac1c45700 1 -- 172.21.15.40:0/3098459807 shutdown_connections 2024-09-13T17:55:55.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:55.603+0000 7faac1c45700 1 -- 172.21.15.40:0/3098459807 wait complete. 2024-09-13T17:55:58.625 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:55:58.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.731+0000 7fa76b9d1700 1 Processor -- start 2024-09-13T17:55:58.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.731+0000 7fa76b9d1700 1 -- start start 2024-09-13T17:55:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.731+0000 7fa76b9d1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa764156bd0 0x7fa76406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.731+0000 7fa76b9d1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa7640b11b0 0x7fa764163ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:58.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.731+0000 7fa76b9d1700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7640c7400 con 0x7fa764072760 2024-09-13T17:55:58.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.731+0000 7fa76b9d1700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7640c24f0 con 0x7fa7640b11b0 2024-09-13T17:55:58.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.731+0000 7fa76b9d1700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7640c8130 con 0x7fa764156bd0 2024-09-13T17:55:58.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.731+0000 7fa769f6e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa7640b11b0 0x7fa764163ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:58.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.731+0000 7fa769f6e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa7640b11b0 0x7fa764163ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41868/0 (socket says 172.21.15.40:41868) 2024-09-13T17:55:58.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.731+0000 7fa769f6e700 1 -- 172.21.15.40:0/3487810204 learned_addr learned my addr 172.21.15.40:0/3487810204 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:58.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.731+0000 7fa76976d700 1 --2- 172.21.15.40:0/3487810204 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa764156bd0 0x7fa76406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:58.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.731+0000 7fa769f6e700 1 -- 172.21.15.40:0/3487810204 >> v2:172.21.15.40:3300/0 conn(0x7fa7640b11b0 msgr2=0x7fa764163ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:55:58.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.731+0000 7fa769f6e700 1 -- 172.21.15.40:0/3487810204 >> v2:172.21.15.40:3300/0 conn(0x7fa7640b11b0 msgr2=0x7fa764163ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:55:58.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.731+0000 7fa769f6e700 1 --2- 172.21.15.40:0/3487810204 >> v2:172.21.15.40:3300/0 conn(0x7fa7640b11b0 0x7fa764163ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:55:58.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.731+0000 7fa769f6e700 1 --2- 172.21.15.40:0/3487810204 >> v2:172.21.15.40:3300/0 conn(0x7fa7640b11b0 0x7fa764163ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:58.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.735+0000 7fa76976d700 1 -- 172.21.15.40:0/3487810204 >> v2:172.21.15.40:3300/0 conn(0x7fa7640b11b0 msgr2=0x7fa764163ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:55:58.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.735+0000 7fa76976d700 1 --2- 172.21.15.40:0/3487810204 >> v2:172.21.15.40:3300/0 conn(0x7fa7640b11b0 0x7fa764163ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:58.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.735+0000 7fa76976d700 1 -- 172.21.15.40:0/3487810204 >> v1:172.21.15.43:6789/0 conn(0x7fa764072760 legacy=0x7fa7640c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:58.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.735+0000 7fa75bfff700 1 -- 172.21.15.40:0/3487810204 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462719371 0 0) 0x7fa7640c7400 con 0x7fa764072760 2024-09-13T17:55:58.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.735+0000 7fa76976d700 1 -- 172.21.15.40:0/3487810204 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa764072090 con 0x7fa764156bd0 2024-09-13T17:55:58.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.735+0000 7fa76976d700 1 --2- 172.21.15.40:0/3487810204 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa764156bd0 0x7fa76406e3c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa760009850 tx=0x7fa760009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b25d28403a89684b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:58.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.735+0000 7fa75bfff700 1 -- 172.21.15.40:0/3487810204 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760011830 con 0x7fa764156bd0 2024-09-13T17:55:58.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.735+0000 7fa75bfff700 1 -- 172.21.15.40:0/3487810204 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa760011990 con 0x7fa764156bd0 2024-09-13T17:55:58.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.735+0000 7fa75bfff700 1 -- 172.21.15.40:0/3487810204 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760011d20 con 0x7fa764156bd0 2024-09-13T17:55:58.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.735+0000 7fa76b9d1700 1 -- 172.21.15.40:0/3487810204 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa764156bd0 msgr2=0x7fa76406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:58.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.735+0000 7fa76b9d1700 1 --2- 172.21.15.40:0/3487810204 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa764156bd0 0x7fa76406e3c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa760009850 tx=0x7fa760009b60 comp rx=0 tx=0).stop 2024-09-13T17:55:58.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.735+0000 7fa76b9d1700 1 -- 172.21.15.40:0/3487810204 shutdown_connections 2024-09-13T17:55:58.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.735+0000 7fa76b9d1700 1 --2- 172.21.15.40:0/3487810204 >> v2:172.21.15.40:3300/0 conn(0x7fa7640b11b0 0x7fa764163ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:58.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.735+0000 7fa76b9d1700 1 --2- 172.21.15.40:0/3487810204 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa764156bd0 0x7fa76406e3c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:58.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.735+0000 7fa76b9d1700 1 -- 172.21.15.40:0/3487810204 >> 172.21.15.40:0/3487810204 conn(0x7fa7640666c0 msgr2=0x7fa7640cbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:58.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.735+0000 7fa76b9d1700 1 -- 172.21.15.40:0/3487810204 shutdown_connections 2024-09-13T17:55:58.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.735+0000 7fa76b9d1700 1 -- 172.21.15.40:0/3487810204 wait complete. 2024-09-13T17:55:58.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa76b9d1700 1 Processor -- start 2024-09-13T17:55:58.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa76b9d1700 1 -- start start 2024-09-13T17:55:58.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa76b9d1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa764072760 0x7fa7640afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:58.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa76b9d1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7640b11b0 0x7fa7640ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:58.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa76b9d1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa764156bd0 0x7fa7640c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:58.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa76b9d1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7640ae510 con 0x7fa764072760 2024-09-13T17:55:58.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa76b9d1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7641f0320 con 0x7fa764156bd0 2024-09-13T17:55:58.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa76b9d1700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7641eeeb0 con 0x7fa7640b11b0 2024-09-13T17:55:58.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa76976d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa764072760 0x7fa7640afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:58.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa76976d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa764072760 0x7fa7640afb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41880/0 (socket says 172.21.15.40:41880) 2024-09-13T17:55:58.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa76976d700 1 -- 172.21.15.40:0/719438633 learned_addr learned my addr 172.21.15.40:0/719438633 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:55:58.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa76976d700 1 -- 172.21.15.40:0/719438633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa764072760 msgr2=0x7fa7640afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T17:55:58.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa769f6e700 1 --2- 172.21.15.40:0/719438633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa764156bd0 0x7fa7640c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:58.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa76976d700 1 -- 172.21.15.40:0/719438633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa764072760 msgr2=0x7fa7640afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:55:58.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa76976d700 1 --2- 172.21.15.40:0/719438633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa764072760 0x7fa7640afb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:55:58.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa768f6c700 1 --2- 172.21.15.40:0/719438633 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7640b11b0 0x7fa7640ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa76976d700 1 --2- 172.21.15.40:0/719438633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa764072760 0x7fa7640afb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:55:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa769f6e700 1 -- 172.21.15.40:0/719438633 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7640b11b0 msgr2=0x7fa7640ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa769f6e700 1 --2- 172.21.15.40:0/719438633 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7640b11b0 0x7fa7640ae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa769f6e700 1 -- 172.21.15.40:0/719438633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa764072760 msgr2=0x7fa7640afb00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:55:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa769f6e700 1 --2- 172.21.15.40:0/719438633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa764072760 0x7fa7640afb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa769f6e700 1 -- 172.21.15.40:0/719438633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7600094b0 con 0x7fa764156bd0 2024-09-13T17:55:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa768f6c700 1 --2- 172.21.15.40:0/719438633 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7640b11b0 0x7fa7640ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:55:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa769f6e700 1 --2- 172.21.15.40:0/719438633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa764156bd0 0x7fa7640c57c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa75c007a60 tx=0x7fa75c007d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa759ffb700 1 -- 172.21.15.40:0/719438633 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c00d9d0 con 0x7fa764156bd0 2024-09-13T17:55:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa759ffb700 1 -- 172.21.15.40:0/719438633 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c00f040 con 0x7fa764156bd0 2024-09-13T17:55:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa76b9d1700 1 -- 172.21.15.40:0/719438633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7641ef150 con 0x7fa764156bd0 2024-09-13T17:55:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa759ffb700 1 -- 172.21.15.40:0/719438633 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c00dd70 con 0x7fa764156bd0 2024-09-13T17:55:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.739+0000 7fa76b9d1700 1 -- 172.21.15.40:0/719438633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7640ad940 con 0x7fa764156bd0 2024-09-13T17:55:58.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.743+0000 7fa76b9d1700 1 -- 172.21.15.40:0/719438633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7640ae650 con 0x7fa764156bd0 2024-09-13T17:55:58.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.743+0000 7fa759ffb700 1 -- 172.21.15.40:0/719438633 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa75c01d360 con 0x7fa764156bd0 2024-09-13T17:55:58.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.743+0000 7fa759ffb700 1 --2- 172.21.15.40:0/719438633 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa748066730 0x7fa748068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:55:58.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.743+0000 7fa759ffb700 1 -- 172.21.15.40:0/719438633 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa75c0840b0 con 0x7fa764156bd0 2024-09-13T17:55:58.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.743+0000 7fa76976d700 1 --2- 172.21.15.40:0/719438633 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa748066730 0x7fa748068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:55:58.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.743+0000 7fa76976d700 1 --2- 172.21.15.40:0/719438633 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa748066730 0x7fa748068bf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa760009ba0 tx=0x7fa760008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:55:58.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:58.755+0000 7fa759ffb700 1 -- 172.21.15.40:0/719438633 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa7640ae650 con 0x7fa764156bd0 2024-09-13T17:55:59.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:59.055+0000 7fa76b9d1700 1 -- 172.21.15.40:0/719438633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa7640ae650 con 0x7fa764156bd0 2024-09-13T17:55:59.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:59.055+0000 7fa759ffb700 1 -- 172.21.15.40:0/719438633 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7fa7640ae650 con 0x7fa764156bd0 2024-09-13T17:55:59.062 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":42,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":38,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:55:59.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:59.055+0000 7fa76b9d1700 1 -- 172.21.15.40:0/719438633 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa748066730 msgr2=0x7fa748068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:59.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:59.055+0000 7fa76b9d1700 1 --2- 172.21.15.40:0/719438633 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa748066730 0x7fa748068bf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa760009ba0 tx=0x7fa760008030 comp rx=0 tx=0).stop 2024-09-13T17:55:59.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:59.055+0000 7fa76b9d1700 1 -- 172.21.15.40:0/719438633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa764156bd0 msgr2=0x7fa7640c57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:55:59.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:59.055+0000 7fa76b9d1700 1 --2- 172.21.15.40:0/719438633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa764156bd0 0x7fa7640c57c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa75c007a60 tx=0x7fa75c007d70 comp rx=0 tx=0).stop 2024-09-13T17:55:59.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:59.055+0000 7fa76b9d1700 1 -- 172.21.15.40:0/719438633 shutdown_connections 2024-09-13T17:55:59.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:59.055+0000 7fa76b9d1700 1 --2- 172.21.15.40:0/719438633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa764156bd0 0x7fa7640c57c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:59.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:59.055+0000 7fa76b9d1700 1 --2- 172.21.15.40:0/719438633 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa748066730 0x7fa748068bf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:59.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:59.055+0000 7fa76b9d1700 1 --2- 172.21.15.40:0/719438633 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7640b11b0 0x7fa7640ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:59.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:59.055+0000 7fa76b9d1700 1 --2- 172.21.15.40:0/719438633 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa764072760 0x7fa7640afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:55:59.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:59.055+0000 7fa76b9d1700 1 -- 172.21.15.40:0/719438633 >> 172.21.15.40:0/719438633 conn(0x7fa7640666c0 msgr2=0x7fa764155b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:55:59.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:59.055+0000 7fa76b9d1700 1 -- 172.21.15.40:0/719438633 shutdown_connections 2024-09-13T17:55:59.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:55:59.055+0000 7fa76b9d1700 1 -- 172.21.15.40:0/719438633 wait complete. 2024-09-13T17:56:02.083 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:56:02.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.199+0000 7fdf64b5b700 1 Processor -- start 2024-09-13T17:56:02.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.199+0000 7fdf64b5b700 1 -- start start 2024-09-13T17:56:02.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf64b5b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdf600cc130 0x7fdf6015a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:02.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf64b5b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf600cc510 0x7fdf600ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:02.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf64b5b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdf600b3410 0x7fdf60153540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:02.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf64b5b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf6015a760 con 0x7fdf600b3410 2024-09-13T17:56:02.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf64b5b700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf600cae10 con 0x7fdf600cc130 2024-09-13T17:56:02.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf64b5b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6015ce80 con 0x7fdf600cc510 2024-09-13T17:56:02.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf5ed9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdf600b3410 0x7fdf60153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:02.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf5ed9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdf600b3410 0x7fdf60153540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47650/0 (socket says 172.21.15.40:47650) 2024-09-13T17:56:02.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf5ed9d700 1 -- 172.21.15.40:0/2014965081 learned_addr learned my addr 172.21.15.40:0/2014965081 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:02.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf57fff700 1 --2- 172.21.15.40:0/2014965081 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf600cc510 0x7fdf600ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:02.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf5e59c700 1 --2- 172.21.15.40:0/2014965081 >> v2:172.21.15.43:3300/0 conn(0x7fdf600cc130 0x7fdf6015a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:02.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf5ed9d700 1 -- 172.21.15.40:0/2014965081 >> v2:172.21.15.40:3300/0 conn(0x7fdf600b3410 msgr2=0x7fdf60153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:56:02.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf5ed9d700 1 -- 172.21.15.40:0/2014965081 >> v2:172.21.15.40:3300/0 conn(0x7fdf600b3410 msgr2=0x7fdf60153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:56:02.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf5ed9d700 1 --2- 172.21.15.40:0/2014965081 >> v2:172.21.15.40:3300/0 conn(0x7fdf600b3410 0x7fdf60153540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:56:02.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf5ed9d700 1 --2- 172.21.15.40:0/2014965081 >> v2:172.21.15.40:3300/0 conn(0x7fdf600b3410 0x7fdf60153540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:02.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf57fff700 1 -- 172.21.15.40:0/2014965081 >> v2:172.21.15.43:3300/0 conn(0x7fdf600cc130 msgr2=0x7fdf6015a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:56:02.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf57fff700 1 --2- 172.21.15.40:0/2014965081 >> v2:172.21.15.43:3300/0 conn(0x7fdf600cc130 0x7fdf6015a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:02.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf57fff700 1 -- 172.21.15.40:0/2014965081 >> v2:172.21.15.40:3300/0 conn(0x7fdf600b3410 msgr2=0x7fdf60153540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:56:02.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf57fff700 1 --2- 172.21.15.40:0/2014965081 >> v2:172.21.15.40:3300/0 conn(0x7fdf600b3410 0x7fdf60153540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:02.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf57fff700 1 -- 172.21.15.40:0/2014965081 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf600cab30 con 0x7fdf600cc510 2024-09-13T17:56:02.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf57fff700 1 --2- 172.21.15.40:0/2014965081 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf600cc510 0x7fdf600ca1a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdf600c9c50 tx=0x7fdf48009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67fb01d80081c357 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:02.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf5dd9b700 1 -- 172.21.15.40:0/2014965081 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48011830 con 0x7fdf600cc510 2024-09-13T17:56:02.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf5dd9b700 1 -- 172.21.15.40:0/2014965081 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf48011990 con 0x7fdf600cc510 2024-09-13T17:56:02.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf5dd9b700 1 -- 172.21.15.40:0/2014965081 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf48011d20 con 0x7fdf600cc510 2024-09-13T17:56:02.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf64b5b700 1 -- 172.21.15.40:0/2014965081 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf600cc510 msgr2=0x7fdf600ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:02.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf64b5b700 1 --2- 172.21.15.40:0/2014965081 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf600cc510 0x7fdf600ca1a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdf600c9c50 tx=0x7fdf48009b30 comp rx=0 tx=0).stop 2024-09-13T17:56:02.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf64b5b700 1 -- 172.21.15.40:0/2014965081 shutdown_connections 2024-09-13T17:56:02.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf64b5b700 1 --2- 172.21.15.40:0/2014965081 >> v2:172.21.15.40:3300/0 conn(0x7fdf600b3410 0x7fdf60153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:02.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf64b5b700 1 --2- 172.21.15.40:0/2014965081 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf600cc510 0x7fdf600ca1a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:02.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf64b5b700 1 --2- 172.21.15.40:0/2014965081 >> v2:172.21.15.43:3300/0 conn(0x7fdf600cc130 0x7fdf6015a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:02.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf64b5b700 1 -- 172.21.15.40:0/2014965081 >> 172.21.15.40:0/2014965081 conn(0x7fdf60066270 msgr2=0x7fdf60066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:02.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf64b5b700 1 -- 172.21.15.40:0/2014965081 shutdown_connections 2024-09-13T17:56:02.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.203+0000 7fdf64b5b700 1 -- 172.21.15.40:0/2014965081 wait complete. 2024-09-13T17:56:02.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.207+0000 7fdf64b5b700 1 Processor -- start 2024-09-13T17:56:02.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.207+0000 7fdf64b5b700 1 -- start start 2024-09-13T17:56:02.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf64b5b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf600b3410 0x7fdf60150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:02.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf64b5b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf600cc130 0x7fdf60158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:02.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf64b5b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf600cc510 0x7fdf60159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:02.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf64b5b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf601581b0 con 0x7fdf600b3410 2024-09-13T17:56:02.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf64b5b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600b3f10 con 0x7fdf600cc510 2024-09-13T17:56:02.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf64b5b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf601575a0 con 0x7fdf600cc130 2024-09-13T17:56:02.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf5e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf600b3410 0x7fdf60150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:02.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf5e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf600b3410 0x7fdf60150fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47656/0 (socket says 172.21.15.40:47656) 2024-09-13T17:56:02.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf5e59c700 1 -- 172.21.15.40:0/2091499309 learned_addr learned my addr 172.21.15.40:0/2091499309 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:02.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf57fff700 1 --2- 172.21.15.40:0/2091499309 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf600cc130 0x7fdf60158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:02.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf5e59c700 1 -- 172.21.15.40:0/2091499309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf600b3410 msgr2=0x7fdf60150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T17:56:02.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf5e59c700 1 -- 172.21.15.40:0/2091499309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf600b3410 msgr2=0x7fdf60150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:56:02.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf5e59c700 1 --2- 172.21.15.40:0/2091499309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf600b3410 0x7fdf60150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:56:02.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf5ed9d700 1 --2- 172.21.15.40:0/2091499309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf600cc510 0x7fdf60159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:02.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf5e59c700 1 --2- 172.21.15.40:0/2091499309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf600b3410 0x7fdf60150fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:02.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf5ed9d700 1 -- 172.21.15.40:0/2091499309 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf600cc130 msgr2=0x7fdf60158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:02.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf5ed9d700 1 --2- 172.21.15.40:0/2091499309 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf600cc130 0x7fdf60158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:02.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf5ed9d700 1 -- 172.21.15.40:0/2091499309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf600b3410 msgr2=0x7fdf60150fd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:56:02.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf5ed9d700 1 --2- 172.21.15.40:0/2091499309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf600b3410 0x7fdf60150fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:02.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf5ed9d700 1 -- 172.21.15.40:0/2091499309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf480094b0 con 0x7fdf600cc510 2024-09-13T17:56:02.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf5ed9d700 1 --2- 172.21.15.40:0/2091499309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf600cc510 0x7fdf60159010 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009fd0 tx=0x7fdf58007d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:02.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf56ffd700 1 -- 172.21.15.40:0/2091499309 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58003c40 con 0x7fdf600cc510 2024-09-13T17:56:02.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf56ffd700 1 -- 172.21.15.40:0/2091499309 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5800f040 con 0x7fdf600cc510 2024-09-13T17:56:02.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf64b5b700 1 -- 172.21.15.40:0/2091499309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf60157880 con 0x7fdf600cc510 2024-09-13T17:56:02.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf56ffd700 1 -- 172.21.15.40:0/2091499309 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5800d7d0 con 0x7fdf600cc510 2024-09-13T17:56:02.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.211+0000 7fdf64b5b700 1 -- 172.21.15.40:0/2091499309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf60157e40 con 0x7fdf600cc510 2024-09-13T17:56:02.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.215+0000 7fdf64b5b700 1 -- 172.21.15.40:0/2091499309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf600b38a0 con 0x7fdf600cc510 2024-09-13T17:56:02.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.215+0000 7fdf56ffd700 1 -- 172.21.15.40:0/2091499309 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdf5800d930 con 0x7fdf600cc510 2024-09-13T17:56:02.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.215+0000 7fdf56ffd700 1 --2- 172.21.15.40:0/2091499309 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf40066730 0x7fdf40068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:02.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.215+0000 7fdf56ffd700 1 -- 172.21.15.40:0/2091499309 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdf58084880 con 0x7fdf600cc510 2024-09-13T17:56:02.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.215+0000 7fdf5e59c700 1 --2- 172.21.15.40:0/2091499309 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf40066730 0x7fdf40068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:02.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.215+0000 7fdf5e59c700 1 --2- 172.21.15.40:0/2091499309 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf40066730 0x7fdf40068bf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009670 tx=0x7fdf50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:02.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.223+0000 7fdf56ffd700 1 -- 172.21.15.40:0/2091499309 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdf58051830 con 0x7fdf600cc510 2024-09-13T17:56:02.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.483+0000 7fdf64b5b700 1 -- 172.21.15.40:0/2091499309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdf601498e0 con 0x7fdf600cc510 2024-09-13T17:56:02.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.483+0000 7fdf56ffd700 1 -- 172.21.15.40:0/2091499309 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7fdf58023080 con 0x7fdf600cc510 2024-09-13T17:56:02.490 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":42,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":41,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:56:02.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.487+0000 7fdf64b5b700 1 -- 172.21.15.40:0/2091499309 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf40066730 msgr2=0x7fdf40068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:02.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.487+0000 7fdf64b5b700 1 --2- 172.21.15.40:0/2091499309 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf40066730 0x7fdf40068bf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009670 tx=0x7fdf50008040 comp rx=0 tx=0).stop 2024-09-13T17:56:02.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.487+0000 7fdf64b5b700 1 -- 172.21.15.40:0/2091499309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf600cc510 msgr2=0x7fdf60159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:02.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.487+0000 7fdf64b5b700 1 --2- 172.21.15.40:0/2091499309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf600cc510 0x7fdf60159010 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009fd0 tx=0x7fdf58007d40 comp rx=0 tx=0).stop 2024-09-13T17:56:02.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.487+0000 7fdf64b5b700 1 -- 172.21.15.40:0/2091499309 shutdown_connections 2024-09-13T17:56:02.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.487+0000 7fdf64b5b700 1 --2- 172.21.15.40:0/2091499309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf600cc510 0x7fdf60159010 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:02.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.487+0000 7fdf64b5b700 1 --2- 172.21.15.40:0/2091499309 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf40066730 0x7fdf40068bf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:02.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.487+0000 7fdf64b5b700 1 --2- 172.21.15.40:0/2091499309 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf600cc130 0x7fdf60158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:02.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.487+0000 7fdf64b5b700 1 --2- 172.21.15.40:0/2091499309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf600b3410 0x7fdf60150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:02.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.487+0000 7fdf64b5b700 1 -- 172.21.15.40:0/2091499309 >> 172.21.15.40:0/2091499309 conn(0x7fdf60066270 msgr2=0x7fdf600c9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:02.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.487+0000 7fdf64b5b700 1 -- 172.21.15.40:0/2091499309 shutdown_connections 2024-09-13T17:56:02.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:02.487+0000 7fdf64b5b700 1 -- 172.21.15.40:0/2091499309 wait complete. 2024-09-13T17:56:04.707 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:56:04.710 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:56:05.507 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:56:05.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.623+0000 7f69b3fff700 1 Processor -- start 2024-09-13T17:56:05.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.623+0000 7f69b3fff700 1 -- start start 2024-09-13T17:56:05.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.623+0000 7f69b3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69b406b4a0 0x7f69b4071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:05.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.623+0000 7f69b3fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b415a520 con 0x7f69b41503d0 2024-09-13T17:56:05.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.623+0000 7f69b3fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b41564b0 con 0x7f69b4072130 2024-09-13T17:56:05.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.623+0000 7f69b3fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b4157050 con 0x7f69b406b4a0 2024-09-13T17:56:05.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.623+0000 7f69b2ffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f69b41503d0 0x7f69b4156940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55578/0 (socket says 172.21.15.40:55578) 2024-09-13T17:56:05.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.623+0000 7f69b2ffd700 1 -- 172.21.15.40:0/186275980 learned_addr learned my addr 172.21.15.40:0/186275980 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:56:05.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.623+0000 7f69b27fc700 1 --2- 172.21.15.40:0/186275980 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69b406b4a0 0x7f69b4071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:05.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.623+0000 7f69b1ffb700 1 -- 172.21.15.40:0/186275980 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180968472 0 0) 0x7f69b41564b0 con 0x7f69b4072130 2024-09-13T17:56:05.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.623+0000 7f69b1ffb700 1 -- 172.21.15.40:0/186275980 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f699c0035d0 con 0x7f69b4072130 2024-09-13T17:56:05.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.627+0000 7f69b27fc700 1 -- 172.21.15.40:0/186275980 >> v1:172.21.15.43:6789/0 conn(0x7f69b4072130 legacy=0x7f69b4160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:05.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.627+0000 7f69b27fc700 1 -- 172.21.15.40:0/186275980 >> v1:172.21.15.40:6789/0 conn(0x7f69b41503d0 legacy=0x7f69b4156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:05.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.627+0000 7f69b27fc700 1 -- 172.21.15.40:0/186275980 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b415a000 con 0x7f69b406b4a0 2024-09-13T17:56:05.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.627+0000 7f69b27fc700 1 --2- 172.21.15.40:0/186275980 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69b406b4a0 0x7f69b4071bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009880 tx=0x7f69a8009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f18021b5604d68d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:05.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.627+0000 7f69b1ffb700 1 -- 172.21.15.40:0/186275980 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a8011830 con 0x7f69b406b4a0 2024-09-13T17:56:05.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.627+0000 7f69b1ffb700 1 -- 172.21.15.40:0/186275980 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a8011990 con 0x7f69b406b4a0 2024-09-13T17:56:05.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.627+0000 7f69b1ffb700 1 -- 172.21.15.40:0/186275980 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a8011d00 con 0x7f69b406b4a0 2024-09-13T17:56:05.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.627+0000 7f69b3fff700 1 -- 172.21.15.40:0/186275980 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69b406b4a0 msgr2=0x7f69b4071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:05.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.627+0000 7f69b3fff700 1 --2- 172.21.15.40:0/186275980 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69b406b4a0 0x7f69b4071bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009880 tx=0x7f69a8009b90 comp rx=0 tx=0).stop 2024-09-13T17:56:05.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.627+0000 7f69b3fff700 1 -- 172.21.15.40:0/186275980 shutdown_connections 2024-09-13T17:56:05.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.627+0000 7f69b3fff700 1 --2- 172.21.15.40:0/186275980 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69b406b4a0 0x7f69b4071bf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:05.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.627+0000 7f69b3fff700 1 -- 172.21.15.40:0/186275980 >> 172.21.15.40:0/186275980 conn(0x7f69b406e4f0 msgr2=0x7f69b406e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:05.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.627+0000 7f69b3fff700 1 -- 172.21.15.40:0/186275980 shutdown_connections 2024-09-13T17:56:05.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.627+0000 7f69b3fff700 1 -- 172.21.15.40:0/186275980 wait complete. 2024-09-13T17:56:05.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.627+0000 7f69b3fff700 1 Processor -- start 2024-09-13T17:56:05.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.627+0000 7f69b3fff700 1 -- start start 2024-09-13T17:56:05.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b3fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69b406b4a0 0x7f69b414c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69b4072130 0x7f69b41f0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69b41503d0 0x7f69b41460e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b3fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b40caae0 con 0x7f69b4072130 2024-09-13T17:56:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b3fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b40cb980 con 0x7f69b406b4a0 2024-09-13T17:56:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b3fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b4142c20 con 0x7f69b41503d0 2024-09-13T17:56:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b27fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69b4072130 0x7f69b41f0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b27fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69b4072130 0x7f69b41f0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47676/0 (socket says 172.21.15.40:47676) 2024-09-13T17:56:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b27fc700 1 -- 172.21.15.40:0/658463703 learned_addr learned my addr 172.21.15.40:0/658463703 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b2ffd700 1 --2- 172.21.15.40:0/658463703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69b406b4a0 0x7f69b414c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b27fc700 1 -- 172.21.15.40:0/658463703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69b4072130 msgr2=0x7f69b41f0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T17:56:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b27fc700 1 -- 172.21.15.40:0/658463703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69b4072130 msgr2=0x7f69b41f0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:56:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b27fc700 1 --2- 172.21.15.40:0/658463703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69b4072130 0x7f69b41f0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:56:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b27fc700 1 --2- 172.21.15.40:0/658463703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69b4072130 0x7f69b41f0a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:05.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b37fe700 1 --2- 172.21.15.40:0/658463703 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69b41503d0 0x7f69b41460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:05.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b2ffd700 1 -- 172.21.15.40:0/658463703 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69b41503d0 msgr2=0x7f69b41460e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:05.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b2ffd700 1 --2- 172.21.15.40:0/658463703 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69b41503d0 0x7f69b41460e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:05.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b2ffd700 1 -- 172.21.15.40:0/658463703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69b4072130 msgr2=0x7f69b41f0a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:56:05.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b2ffd700 1 --2- 172.21.15.40:0/658463703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69b4072130 0x7f69b41f0a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:05.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b2ffd700 1 -- 172.21.15.40:0/658463703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a80094b0 con 0x7f69b406b4a0 2024-09-13T17:56:05.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b2ffd700 1 --2- 172.21.15.40:0/658463703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69b406b4a0 0x7f69b414c320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f69a400c430 tx=0x7f69a400c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:05.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f699b7fe700 1 -- 172.21.15.40:0/658463703 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a4005d90 con 0x7f69b406b4a0 2024-09-13T17:56:05.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f699b7fe700 1 -- 172.21.15.40:0/658463703 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a40070d0 con 0x7f69b406b4a0 2024-09-13T17:56:05.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b3fff700 1 -- 172.21.15.40:0/658463703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b41f0fd0 con 0x7f69b406b4a0 2024-09-13T17:56:05.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f699b7fe700 1 -- 172.21.15.40:0/658463703 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a4007440 con 0x7f69b406b4a0 2024-09-13T17:56:05.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.631+0000 7f69b3fff700 1 -- 172.21.15.40:0/658463703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b41f0290 con 0x7f69b406b4a0 2024-09-13T17:56:05.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.635+0000 7f69b3fff700 1 -- 172.21.15.40:0/658463703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b41507b0 con 0x7f69b406b4a0 2024-09-13T17:56:05.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.635+0000 7f699b7fe700 1 -- 172.21.15.40:0/658463703 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f69a40075a0 con 0x7f69b406b4a0 2024-09-13T17:56:05.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.635+0000 7f699b7fe700 1 --2- 172.21.15.40:0/658463703 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f69a0066730 0x7f69a0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:05.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.635+0000 7f699b7fe700 1 -- 172.21.15.40:0/658463703 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f69a4083d80 con 0x7f69b406b4a0 2024-09-13T17:56:05.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.635+0000 7f69b27fc700 1 --2- 172.21.15.40:0/658463703 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f69a0066730 0x7f69a0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:05.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.635+0000 7f69b27fc700 1 --2- 172.21.15.40:0/658463703 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f69a0066730 0x7f69a0068bf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f69b414c860 tx=0x7f69a8010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:05.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.643+0000 7f699b7fe700 1 -- 172.21.15.40:0/658463703 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f69a4050de0 con 0x7f69b406b4a0 2024-09-13T17:56:05.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.899+0000 7f69b3fff700 1 -- 172.21.15.40:0/658463703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f69b40c2e30 con 0x7f69b406b4a0 2024-09-13T17:56:05.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.899+0000 7f699b7fe700 1 -- 172.21.15.40:0/658463703 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f69a4017080 con 0x7f69b406b4a0 2024-09-13T17:56:05.906 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":42,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":45,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:56:05.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.899+0000 7f69b3fff700 1 -- 172.21.15.40:0/658463703 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f69a0066730 msgr2=0x7f69a0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:05.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.899+0000 7f69b3fff700 1 --2- 172.21.15.40:0/658463703 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f69a0066730 0x7f69a0068bf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f69b414c860 tx=0x7f69a8010000 comp rx=0 tx=0).stop 2024-09-13T17:56:05.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.899+0000 7f69b3fff700 1 -- 172.21.15.40:0/658463703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69b406b4a0 msgr2=0x7f69b414c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:05.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.899+0000 7f69b3fff700 1 --2- 172.21.15.40:0/658463703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69b406b4a0 0x7f69b414c320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f69a400c430 tx=0x7f69a400c740 comp rx=0 tx=0).stop 2024-09-13T17:56:05.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.899+0000 7f69b3fff700 1 -- 172.21.15.40:0/658463703 shutdown_connections 2024-09-13T17:56:05.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.899+0000 7f69b3fff700 1 --2- 172.21.15.40:0/658463703 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f69a0066730 0x7f69a0068bf0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:05.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.899+0000 7f69b3fff700 1 --2- 172.21.15.40:0/658463703 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69b41503d0 0x7f69b41460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:05.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.899+0000 7f69b3fff700 1 --2- 172.21.15.40:0/658463703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69b4072130 0x7f69b41f0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:05.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.899+0000 7f69b3fff700 1 --2- 172.21.15.40:0/658463703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69b406b4a0 0x7f69b414c320 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:05.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.899+0000 7f69b3fff700 1 -- 172.21.15.40:0/658463703 >> 172.21.15.40:0/658463703 conn(0x7f69b406e4f0 msgr2=0x7f69b406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:05.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.899+0000 7f69b3fff700 1 -- 172.21.15.40:0/658463703 shutdown_connections 2024-09-13T17:56:05.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:05.899+0000 7f69b3fff700 1 -- 172.21.15.40:0/658463703 wait complete. 2024-09-13T17:56:08.922 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:56:09.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.043+0000 7f50b9da3700 1 Processor -- start 2024-09-13T17:56:09.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.043+0000 7f50b9da3700 1 -- start start 2024-09-13T17:56:09.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b9da3700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f50b4072350 0x7f50b4150d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b9da3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f50b40c7200 0x7f50b4151280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b8da1700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f50b4071f70 0x7f50b40c5d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55588/0 (socket says 172.21.15.40:55588) 2024-09-13T17:56:09.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b9da3700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b40c55c0 con 0x7f50b4071f70 2024-09-13T17:56:09.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b8da1700 1 -- 172.21.15.40:0/2846980692 learned_addr learned my addr 172.21.15.40:0/2846980692 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:56:09.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b9da3700 1 -- 172.21.15.40:0/2846980692 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50b406a390 con 0x7f50b4072350 2024-09-13T17:56:09.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b9da3700 1 -- 172.21.15.40:0/2846980692 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b4157710 con 0x7f50b40c7200 2024-09-13T17:56:09.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b3fff700 1 --2- 172.21.15.40:0/2846980692 >> v2:172.21.15.43:3300/0 conn(0x7f50b4072350 0x7f50b4150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b95a2700 1 --2- 172.21.15.40:0/2846980692 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f50b40c7200 0x7f50b4151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b37fe700 1 -- 172.21.15.40:0/2846980692 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181094769 0 0) 0x7f50b40c55c0 con 0x7f50b4071f70 2024-09-13T17:56:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b37fe700 1 -- 172.21.15.40:0/2846980692 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50a40035d0 con 0x7f50b4071f70 2024-09-13T17:56:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b37fe700 1 -- 172.21.15.40:0/2846980692 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2960003231 0 0) 0x7f50a40035d0 con 0x7f50b4071f70 2024-09-13T17:56:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b37fe700 1 -- 172.21.15.40:0/2846980692 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f50b40c7200 msgr2=0x7f50b4151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:56:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b37fe700 1 --2- 172.21.15.40:0/2846980692 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f50b40c7200 0x7f50b4151280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b37fe700 1 -- 172.21.15.40:0/2846980692 >> v2:172.21.15.43:3300/0 conn(0x7f50b4072350 msgr2=0x7f50b4150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:56:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b37fe700 1 --2- 172.21.15.40:0/2846980692 >> v2:172.21.15.43:3300/0 conn(0x7f50b4072350 0x7f50b4150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b37fe700 1 -- 172.21.15.40:0/2846980692 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b41579d0 con 0x7f50b4071f70 2024-09-13T17:56:09.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b37fe700 1 -- 172.21.15.40:0/2846980692 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f50a0001670 con 0x7f50b4071f70 2024-09-13T17:56:09.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b37fe700 1 -- 172.21.15.40:0/2846980692 >> v1:172.21.15.40:6789/0 conn(0x7f50b4071f70 legacy=0x7f50b40c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b37fe700 1 --2- 172.21.15.40:0/2846980692 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f50b40c7200 0x7f50a4003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b37fe700 1 --2- 172.21.15.40:0/2846980692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50a4003fa0 0x7f50a4004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b37fe700 1 --2- 172.21.15.40:0/2846980692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a4004db0 0x7f50a4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b37fe700 1 -- 172.21.15.40:0/2846980692 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b4157710 con 0x7f50a4003fa0 2024-09-13T17:56:09.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b37fe700 1 -- 172.21.15.40:0/2846980692 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a400ba30 con 0x7f50a4004db0 2024-09-13T17:56:09.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b37fe700 1 -- 172.21.15.40:0/2846980692 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a400bb70 con 0x7f50b40c7200 2024-09-13T17:56:09.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b3fff700 1 --2- 172.21.15.40:0/2846980692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50a4003fa0 0x7f50a4004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b8da1700 1 --2- 172.21.15.40:0/2846980692 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f50b40c7200 0x7f50a4003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b95a2700 1 --2- 172.21.15.40:0/2846980692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a4004db0 0x7f50a4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b3fff700 1 -- 172.21.15.40:0/2846980692 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f50b40c7200 msgr2=0x7f50a4003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:56:09.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b3fff700 1 --2- 172.21.15.40:0/2846980692 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f50b40c7200 0x7f50a4003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b3fff700 1 -- 172.21.15.40:0/2846980692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a4004db0 msgr2=0x7f50a4005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:56:09.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b3fff700 1 --2- 172.21.15.40:0/2846980692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a4004db0 0x7f50a4005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b3fff700 1 -- 172.21.15.40:0/2846980692 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a40035d0 con 0x7f50a4003fa0 2024-09-13T17:56:09.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b3fff700 1 --2- 172.21.15.40:0/2846980692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50a4003fa0 0x7f50a4004870 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f50a8009b70 tx=0x7f50a40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f1bd8c59e37966 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b37fe700 1 -- 172.21.15.40:0/2846980692 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a800e660 con 0x7f50a4003fa0 2024-09-13T17:56:09.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b37fe700 1 -- 172.21.15.40:0/2846980692 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a800e7c0 con 0x7f50a4003fa0 2024-09-13T17:56:09.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.047+0000 7f50b37fe700 1 -- 172.21.15.40:0/2846980692 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a800eb30 con 0x7f50a4003fa0 2024-09-13T17:56:09.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b9da3700 1 -- 172.21.15.40:0/2846980692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50a4003fa0 msgr2=0x7f50a4004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b9da3700 1 --2- 172.21.15.40:0/2846980692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50a4003fa0 0x7f50a4004870 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f50a8009b70 tx=0x7f50a40058a0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b95a2700 1 -- 172.21.15.40:0/2846980692 reap_dead start 2024-09-13T17:56:09.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b9da3700 1 -- 172.21.15.40:0/2846980692 shutdown_connections 2024-09-13T17:56:09.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b9da3700 1 -- 172.21.15.40:0/2846980692 >> 172.21.15.40:0/2846980692 conn(0x7f50b40cabc0 msgr2=0x7f50b40cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:09.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b9da3700 1 -- 172.21.15.40:0/2846980692 shutdown_connections 2024-09-13T17:56:09.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b9da3700 1 -- 172.21.15.40:0/2846980692 wait complete. 2024-09-13T17:56:09.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b9da3700 1 Processor -- start 2024-09-13T17:56:09.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b9da3700 1 -- start start 2024-09-13T17:56:09.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b9da3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a4003fa0 0x7f50b4151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b9da3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50b4071880 0x7f50b41f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b9da3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f50b41425d0 0x7f50b4146460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b9da3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b41ef520 con 0x7f50b4071880 2024-09-13T17:56:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b9da3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b4152470 con 0x7f50a4003fa0 2024-09-13T17:56:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b9da3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b4071280 con 0x7f50b41425d0 2024-09-13T17:56:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50b4071880 0x7f50b41f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50b4071880 0x7f50b41f2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47718/0 (socket says 172.21.15.40:47718) 2024-09-13T17:56:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b3fff700 1 -- 172.21.15.40:0/1610698886 learned_addr learned my addr 172.21.15.40:0/1610698886 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b8da1700 1 --2- 172.21.15.40:0/1610698886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a4003fa0 0x7f50b4151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b95a2700 1 --2- 172.21.15.40:0/1610698886 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f50b41425d0 0x7f50b4146460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b3fff700 1 -- 172.21.15.40:0/1610698886 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f50b41425d0 msgr2=0x7f50b4146460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b3fff700 1 --2- 172.21.15.40:0/1610698886 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f50b41425d0 0x7f50b4146460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b3fff700 1 -- 172.21.15.40:0/1610698886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a4003fa0 msgr2=0x7f50b4151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b3fff700 1 --2- 172.21.15.40:0/1610698886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a4003fa0 0x7f50b4151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b3fff700 1 -- 172.21.15.40:0/1610698886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b41eeba0 con 0x7f50b4071880 2024-09-13T17:56:09.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b95a2700 1 --2- 172.21.15.40:0/1610698886 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f50b41425d0 0x7f50b4146460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:56:09.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.051+0000 7f50b3fff700 1 --2- 172.21.15.40:0/1610698886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50b4071880 0x7f50b41f2aa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f50a4004710 tx=0x7f50a80047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.055+0000 7f50b17fa700 1 -- 172.21.15.40:0/1610698886 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a800e660 con 0x7f50b4071880 2024-09-13T17:56:09.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.055+0000 7f50b9da3700 1 -- 172.21.15.40:0/1610698886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b41f2fe0 con 0x7f50b4071880 2024-09-13T17:56:09.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.055+0000 7f50b17fa700 1 -- 172.21.15.40:0/1610698886 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a8013e00 con 0x7f50b4071880 2024-09-13T17:56:09.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.055+0000 7f50b17fa700 1 -- 172.21.15.40:0/1610698886 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8004290 con 0x7f50b4071880 2024-09-13T17:56:09.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.055+0000 7f50b9da3700 1 -- 172.21.15.40:0/1610698886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b41eee60 con 0x7f50b4071880 2024-09-13T17:56:09.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.055+0000 7f50b9da3700 1 -- 172.21.15.40:0/1610698886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b40cb6e0 con 0x7f50b4071880 2024-09-13T17:56:09.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.055+0000 7f50b17fa700 1 -- 172.21.15.40:0/1610698886 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f50a80043f0 con 0x7f50b4071880 2024-09-13T17:56:09.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.055+0000 7f50b17fa700 1 --2- 172.21.15.40:0/1610698886 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5098066730 0x7f5098068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.055+0000 7f50b17fa700 1 -- 172.21.15.40:0/1610698886 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f50a8085a30 con 0x7f50b4071880 2024-09-13T17:56:09.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.055+0000 7f50b8da1700 1 --2- 172.21.15.40:0/1610698886 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5098066730 0x7f5098068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.055+0000 7f50b8da1700 1 --2- 172.21.15.40:0/1610698886 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5098066730 0x7f5098068bf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f50a0000c00 tx=0x7f50a00079a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.063+0000 7f50b17fa700 1 -- 172.21.15.40:0/1610698886 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f50a8052a70 con 0x7f50b4071880 2024-09-13T17:56:09.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.331+0000 7f50b9da3700 1 -- 172.21.15.40:0/1610698886 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f50b414e260 con 0x7f50b4071880 2024-09-13T17:56:09.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.331+0000 7f50b17fa700 1 -- 172.21.15.40:0/1610698886 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7f50a8056090 con 0x7f50b4071880 2024-09-13T17:56:09.337 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:56:09.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.331+0000 7f50b9da3700 1 -- 172.21.15.40:0/1610698886 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5098066730 msgr2=0x7f5098068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.331+0000 7f50b9da3700 1 --2- 172.21.15.40:0/1610698886 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5098066730 0x7f5098068bf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f50a0000c00 tx=0x7f50a00079a0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.331+0000 7f50b9da3700 1 -- 172.21.15.40:0/1610698886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50b4071880 msgr2=0x7f50b41f2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.331+0000 7f50b9da3700 1 --2- 172.21.15.40:0/1610698886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50b4071880 0x7f50b41f2aa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f50a4004710 tx=0x7f50a80047e0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.331+0000 7f50b9da3700 1 -- 172.21.15.40:0/1610698886 shutdown_connections 2024-09-13T17:56:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.331+0000 7f50b9da3700 1 --2- 172.21.15.40:0/1610698886 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5098066730 0x7f5098068bf0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.331+0000 7f50b9da3700 1 --2- 172.21.15.40:0/1610698886 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f50b41425d0 0x7f50b4146460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.331+0000 7f50b9da3700 1 --2- 172.21.15.40:0/1610698886 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50b4071880 0x7f50b41f2aa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.331+0000 7f50b9da3700 1 --2- 172.21.15.40:0/1610698886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a4003fa0 0x7f50b4151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.331+0000 7f50b9da3700 1 -- 172.21.15.40:0/1610698886 >> 172.21.15.40:0/1610698886 conn(0x7f50b40cabc0 msgr2=0x7f50b40c9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.331+0000 7f50b9da3700 1 -- 172.21.15.40:0/1610698886 shutdown_connections 2024-09-13T17:56:09.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.331+0000 7f50b9da3700 1 -- 172.21.15.40:0/1610698886 wait complete. 2024-09-13T17:56:09.353 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T17:56:09.353 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T17:56:09.353 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:56:09.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.455+0000 7f58a4ae5700 1 Processor -- start 2024-09-13T17:56:09.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.455+0000 7f58a4ae5700 1 -- start start 2024-09-13T17:56:09.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.455+0000 7f58a4ae5700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f58a00cf1a0 0x7f58a015a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.455+0000 7f58a4ae5700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58a00cc250 0x7f58a00c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.455+0000 7f58a4ae5700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a015a7a0 con 0x7f58a00cf580 2024-09-13T17:56:09.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.455+0000 7f58a4ae5700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a00cbdc0 con 0x7f58a00cc250 2024-09-13T17:56:09.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.455+0000 7f58a4ae5700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a0153f80 con 0x7f58a00cf1a0 2024-09-13T17:56:09.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.455+0000 7f589ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58a00cc250 0x7f58a00c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.455+0000 7f5897fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f58a00cf580 0x7f58a00c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55604/0 (socket says 172.21.15.40:55604) 2024-09-13T17:56:09.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.455+0000 7f589ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58a00cc250 0x7f58a00c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47734/0 (socket says 172.21.15.40:47734) 2024-09-13T17:56:09.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.455+0000 7f5897fff700 1 -- 172.21.15.40:0/734361518 learned_addr learned my addr 172.21.15.40:0/734361518 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:56:09.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.459+0000 7f589f7fe700 1 --2- 172.21.15.40:0/734361518 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f58a00cf1a0 0x7f58a015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.459+0000 7f589ffff700 1 -- 172.21.15.40:0/734361518 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f58a00cf1a0 msgr2=0x7f58a015a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:56:09.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.459+0000 7f589ffff700 1 --2- 172.21.15.40:0/734361518 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f58a00cf1a0 0x7f58a015a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.459+0000 7f589ffff700 1 -- 172.21.15.40:0/734361518 >> v1:172.21.15.40:6789/0 conn(0x7f58a00cf580 legacy=0x7f58a00c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.459+0000 7f589ffff700 1 -- 172.21.15.40:0/734361518 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a00caed0 con 0x7f58a00cc250 2024-09-13T17:56:09.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.459+0000 7f589effd700 1 -- 172.21.15.40:0/734361518 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3308254440 0 0) 0x7f5888001110 con 0x7f58a00cf580 2024-09-13T17:56:09.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.459+0000 7f589ffff700 1 --2- 172.21.15.40:0/734361518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00cc250 0x7f58a00c8aa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5898009d90 tx=0x7f589800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae30d20b55d8206 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.459+0000 7f589effd700 1 -- 172.21.15.40:0/734361518 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589800e070 con 0x7f58a00cc250 2024-09-13T17:56:09.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.459+0000 7f589effd700 1 -- 172.21.15.40:0/734361518 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589800bb20 con 0x7f58a00cc250 2024-09-13T17:56:09.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.459+0000 7f589effd700 1 -- 172.21.15.40:0/734361518 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898012430 con 0x7f58a00cc250 2024-09-13T17:56:09.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.459+0000 7f58a4ae5700 1 -- 172.21.15.40:0/734361518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00cc250 msgr2=0x7f58a00c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.459+0000 7f58a4ae5700 1 --2- 172.21.15.40:0/734361518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00cc250 0x7f58a00c8aa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5898009d90 tx=0x7f589800b330 comp rx=0 tx=0).stop 2024-09-13T17:56:09.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.459+0000 7f58a4ae5700 1 -- 172.21.15.40:0/734361518 shutdown_connections 2024-09-13T17:56:09.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.459+0000 7f58a4ae5700 1 --2- 172.21.15.40:0/734361518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00cc250 0x7f58a00c8aa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.459+0000 7f58a4ae5700 1 --2- 172.21.15.40:0/734361518 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f58a00cf1a0 0x7f58a015a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.459+0000 7f58a4ae5700 1 -- 172.21.15.40:0/734361518 >> 172.21.15.40:0/734361518 conn(0x7f58a0066090 msgr2=0x7f58a0066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:09.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.459+0000 7f58a4ae5700 1 -- 172.21.15.40:0/734361518 shutdown_connections 2024-09-13T17:56:09.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.459+0000 7f58a4ae5700 1 -- 172.21.15.40:0/734361518 wait complete. 2024-09-13T17:56:09.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.459+0000 7f58a4ae5700 1 Processor -- start 2024-09-13T17:56:09.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.459+0000 7f58a4ae5700 1 -- start start 2024-09-13T17:56:09.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f58a4ae5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58a00cc250 0x7f58a00c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f58a4ae5700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f58a00cf1a0 0x7f58a01ef4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f58a4ae5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00cf580 0x7f58a00c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f58a4ae5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00c5130 con 0x7f58a00cf580 2024-09-13T17:56:09.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f58a4ae5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a0152a80 con 0x7f58a00cc250 2024-09-13T17:56:09.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f58a4ae5700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a01ef9e0 con 0x7f58a00cf1a0 2024-09-13T17:56:09.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f589ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00cf580 0x7f58a00c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f589ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00cf580 0x7f58a00c5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47748/0 (socket says 172.21.15.40:47748) 2024-09-13T17:56:09.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f589ffff700 1 -- 172.21.15.40:0/1943071235 learned_addr learned my addr 172.21.15.40:0/1943071235 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:09.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f5897fff700 1 --2- 172.21.15.40:0/1943071235 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f58a00cf1a0 0x7f58a01ef4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f589f7fe700 1 --2- 172.21.15.40:0/1943071235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58a00cc250 0x7f58a00c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f589ffff700 1 -- 172.21.15.40:0/1943071235 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f58a00cf1a0 msgr2=0x7f58a01ef4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f589ffff700 1 --2- 172.21.15.40:0/1943071235 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f58a00cf1a0 0x7f58a01ef4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f589ffff700 1 -- 172.21.15.40:0/1943071235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58a00cc250 msgr2=0x7f58a00c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f589ffff700 1 --2- 172.21.15.40:0/1943071235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58a00cc250 0x7f58a00c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f589ffff700 1 -- 172.21.15.40:0/1943071235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5898009a70 con 0x7f58a00cf580 2024-09-13T17:56:09.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f589ffff700 1 --2- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00cf580 0x7f58a00c5700 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5898014040 tx=0x7f5898004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f589cff9700 1 -- 172.21.15.40:0/1943071235 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898012430 con 0x7f58a00cf580 2024-09-13T17:56:09.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f589cff9700 1 -- 172.21.15.40:0/1943071235 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5898020070 con 0x7f58a00cf580 2024-09-13T17:56:09.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f58a4ae5700 1 -- 172.21.15.40:0/1943071235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a00c29b0 con 0x7f58a00cf580 2024-09-13T17:56:09.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f589cff9700 1 -- 172.21.15.40:0/1943071235 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898004280 con 0x7f58a00cf580 2024-09-13T17:56:09.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.463+0000 7f58a4ae5700 1 -- 172.21.15.40:0/1943071235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a00c3250 con 0x7f58a00cf580 2024-09-13T17:56:09.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.467+0000 7f58a4ae5700 1 --2- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00c2b40 0x7f58a00c36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.467+0000 7f58a4ae5700 1 -- 172.21.15.40:0/1943071235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a0074750 con 0x7f58a00c2b40 2024-09-13T17:56:09.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.467+0000 7f589f7fe700 1 --2- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00c2b40 0x7f58a00c36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.467+0000 7f589f7fe700 1 -- 172.21.15.40:0/1943071235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f58a00cb770 con 0x7f58a00c2b40 2024-09-13T17:56:09.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.467+0000 7f589f7fe700 1 --2- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00c2b40 0x7f58a00c36c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f58900119d0 tx=0x7f5890011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.467+0000 7f589cff9700 1 -- 172.21.15.40:0/1943071235 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890004240 con 0x7f58a00c2b40 2024-09-13T17:56:09.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.467+0000 7f589cff9700 1 -- 172.21.15.40:0/1943071235 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f58980043e0 con 0x7f58a00cf580 2024-09-13T17:56:09.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.467+0000 7f589cff9700 1 --2- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5880066800 0x7f5880068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.467+0000 7f589cff9700 1 -- 172.21.15.40:0/1943071235 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f58900043a0 con 0x7f58a00c2b40 2024-09-13T17:56:09.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.467+0000 7f5897fff700 1 --2- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5880066800 0x7f5880068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.467+0000 7f589cff9700 1 -- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00c2b40 msgr2=0x7f58a00c36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.467+0000 7f589cff9700 1 --2- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00c2b40 0x7f58a00c36c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f58900119d0 tx=0x7f5890011d90 comp rx=0 tx=0).stop 2024-09-13T17:56:09.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.467+0000 7f589cff9700 1 -- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00c2b40 msgr2=0x7f58a00c36c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:56:09.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.467+0000 7f589cff9700 1 --2- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00c2b40 0x7f58a00c36c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.467+0000 7f589cff9700 1 -- 172.21.15.40:0/1943071235 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5898022070 con 0x7f58a00cf580 2024-09-13T17:56:09.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.467+0000 7f5897fff700 1 --2- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5880066800 0x7f5880068cc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f58a00cee90 tx=0x7f5888007e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.475+0000 7f58a4ae5700 1 --2- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00c3c00 0x7f58a01f13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.475+0000 7f58a4ae5700 1 -- 172.21.15.40:0/1943071235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00cb490 con 0x7f58a00c3c00 2024-09-13T17:56:09.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.475+0000 7f589f7fe700 1 --2- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00c3c00 0x7f58a01f13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.479+0000 7f589f7fe700 1 -- 172.21.15.40:0/1943071235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f58a00cba50 con 0x7f58a00c3c00 2024-09-13T17:56:09.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.479+0000 7f589f7fe700 1 --2- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00c3c00 0x7f58a01f13f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f58900119d0 tx=0x7f5890004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.479+0000 7f589cff9700 1 -- 172.21.15.40:0/1943071235 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589001e450 con 0x7f58a00c3c00 2024-09-13T17:56:09.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.479+0000 7f589cff9700 1 -- 172.21.15.40:0/1943071235 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4617 (secure 0 0 0) 0x7f58900283a0 con 0x7f58a00c3c00 2024-09-13T17:56:09.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.479+0000 7f589cff9700 1 -- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00c3c00 msgr2=0x7f58a01f13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.479+0000 7f589cff9700 1 --2- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00c3c00 0x7f58a01f13f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f58900119d0 tx=0x7f5890004770 comp rx=0 tx=0).stop 2024-09-13T17:56:09.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.479+0000 7f589cff9700 1 -- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00c3c00 msgr2=0x7f58a01f13f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:56:09.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.479+0000 7f589cff9700 1 --2- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00c3c00 0x7f58a01f13f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.486 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:56:09.487 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:56:09.487 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:56:09.487 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:56:09.487 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 46, 2024-09-13T17:56:09.487 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:56:09.487 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:56:09.487 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:56:09.487 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:56:09.487 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:09.487 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T17:56:09.487 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:56:09.487 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:56:09.487 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:56:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:56:09.488 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:56:09.489 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:09.489 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:09.489 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:09.489 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:09.489 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:09.489 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:09.489 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:09.489 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:09.489 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:09.490 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:09.490 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:09.490 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.490 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:56:09.490 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:56:09.490 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:56:09.490 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:56:09.490 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:56:09.490 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:56:09.490 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:56:09.491 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:56:09.491 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:56:09.491 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:56:09.491 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:56:09.491 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:56:09.491 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:56:09.491 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:56:09.491 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:56:09.491 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:56:09.492 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:56:09.492 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:09.492 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:09.492 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:09.492 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:09.492 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:09.492 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:09.492 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:09.492 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:09.492 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:09.492 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:09.492 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:09.492 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:56:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:56:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:56:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:56:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:56:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.493 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:56:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:56:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:56:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:56:09.494 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:56:09.495 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:09.495 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:09.495 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.495 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:09.495 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:56:09.495 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:09.495 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.495 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.495 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:09.495 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:56:09.495 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:56:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:56:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:56:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:56:09.496 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:56:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:56:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:56:09.497 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:56:09.498 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:09.498 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:09.498 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:09.498 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.498 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:09.498 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.498 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:56:09.498 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:56:09.498 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.498 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:09.498 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:09.498 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:56:09.499 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.499 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:09.499 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:56:09.499 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.499 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:09.499 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:09.499 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:56:09.499 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.499 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:09.500 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:56:09.500 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.500 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:09.500 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:09.500 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:56:09.500 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.500 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:09.500 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.501 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:56:09.501 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:56:09.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.479+0000 7f58a4ae5700 1 -- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5880066800 msgr2=0x7f5880068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.479+0000 7f58a4ae5700 1 --2- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5880066800 0x7f5880068cc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f58a00cee90 tx=0x7f5888007e90 comp rx=0 tx=0).stop 2024-09-13T17:56:09.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.479+0000 7f58a4ae5700 1 -- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00cf580 msgr2=0x7f58a00c5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.479+0000 7f58a4ae5700 1 --2- 172.21.15.40:0/1943071235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58a00cf580 0x7f58a00c5700 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5898014040 tx=0x7f5898004af0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.479+0000 7f589ffff700 1 -- 172.21.15.40:0/1943071235 reap_dead start 2024-09-13T17:56:09.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.479+0000 7f589ffff700 1 -- 172.21.15.40:0/1943071235 reap_dead start 2024-09-13T17:56:09.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.479+0000 7f58a4ae5700 1 -- 172.21.15.40:0/1943071235 shutdown_connections 2024-09-13T17:56:09.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.479+0000 7f58a4ae5700 1 -- 172.21.15.40:0/1943071235 >> 172.21.15.40:0/1943071235 conn(0x7f58a0066090 msgr2=0x7f58a00c7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:09.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.479+0000 7f58a4ae5700 1 -- 172.21.15.40:0/1943071235 shutdown_connections 2024-09-13T17:56:09.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.479+0000 7f58a4ae5700 1 -- 172.21.15.40:0/1943071235 wait complete. 2024-09-13T17:56:09.503 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:56:09.503 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T17:56:09.504 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:56:09.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.615+0000 7f3b598c9700 1 Processor -- start 2024-09-13T17:56:09.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.615+0000 7f3b598c9700 1 -- start start 2024-09-13T17:56:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.615+0000 7f3b598c9700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3b4c0d8b40 0x7f3b4c049090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.615+0000 7f3b598c9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b4c04f220 0x7f3b4c0e2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.615+0000 7f3b598c9700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b4c050390 con 0x7f3b4c007430 2024-09-13T17:56:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.615+0000 7f3b598c9700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b4c0508e0 con 0x7f3b4c0d8b40 2024-09-13T17:56:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.615+0000 7f3b598c9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c0df6f0 con 0x7f3b4c04f220 2024-09-13T17:56:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.615+0000 7f3b53fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3b4c007430 0x7f3b4c0e1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55614/0 (socket says 172.21.15.40:55614) 2024-09-13T17:56:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.615+0000 7f3b53fff700 1 -- 172.21.15.40:0/1953118233 learned_addr learned my addr 172.21.15.40:0/1953118233 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:56:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.615+0000 7f3b590c8700 1 --2- 172.21.15.40:0/1953118233 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b4c04f220 0x7f3b4c0e2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.615+0000 7f3b588c7700 1 --2- 172.21.15.40:0/1953118233 >> v2:172.21.15.43:3300/0 conn(0x7f3b4c0d8b40 0x7f3b4c049090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.615+0000 7f3b537fe700 1 -- 172.21.15.40:0/1953118233 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094486396 0 0) 0x7f3b4c050390 con 0x7f3b4c007430 2024-09-13T17:56:09.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.619+0000 7f3b537fe700 1 -- 172.21.15.40:0/1953118233 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b440035d0 con 0x7f3b4c007430 2024-09-13T17:56:09.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.619+0000 7f3b537fe700 1 -- 172.21.15.40:0/1953118233 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2742647396 0 0) 0x7f3b440035d0 con 0x7f3b4c007430 2024-09-13T17:56:09.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.619+0000 7f3b590c8700 1 -- 172.21.15.40:0/1953118233 >> v2:172.21.15.43:3300/0 conn(0x7f3b4c0d8b40 msgr2=0x7f3b4c049090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:56:09.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.619+0000 7f3b590c8700 1 --2- 172.21.15.40:0/1953118233 >> v2:172.21.15.43:3300/0 conn(0x7f3b4c0d8b40 0x7f3b4c049090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.619+0000 7f3b590c8700 1 -- 172.21.15.40:0/1953118233 >> v1:172.21.15.40:6789/0 conn(0x7f3b4c007430 legacy=0x7f3b4c0e1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.619+0000 7f3b590c8700 1 -- 172.21.15.40:0/1953118233 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c055030 con 0x7f3b4c04f220 2024-09-13T17:56:09.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.619+0000 7f3b590c8700 1 --2- 172.21.15.40:0/1953118233 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b4c04f220 0x7f3b4c0e2360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3b54069620 tx=0x7f3b54069e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba5277838f01a2d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.619+0000 7f3b537fe700 1 -- 172.21.15.40:0/1953118233 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40001670 con 0x7f3b4c04f220 2024-09-13T17:56:09.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.619+0000 7f3b537fe700 1 -- 172.21.15.40:0/1953118233 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b540689b0 con 0x7f3b4c04f220 2024-09-13T17:56:09.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.619+0000 7f3b537fe700 1 -- 172.21.15.40:0/1953118233 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b54067fd0 con 0x7f3b4c04f220 2024-09-13T17:56:09.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.619+0000 7f3b598c9700 1 -- 172.21.15.40:0/1953118233 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b4c04f220 msgr2=0x7f3b4c0e2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.619+0000 7f3b598c9700 1 --2- 172.21.15.40:0/1953118233 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b4c04f220 0x7f3b4c0e2360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3b54069620 tx=0x7f3b54069e70 comp rx=0 tx=0).stop 2024-09-13T17:56:09.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.619+0000 7f3b598c9700 1 -- 172.21.15.40:0/1953118233 shutdown_connections 2024-09-13T17:56:09.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.619+0000 7f3b598c9700 1 --2- 172.21.15.40:0/1953118233 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b4c04f220 0x7f3b4c0e2360 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.619+0000 7f3b598c9700 1 --2- 172.21.15.40:0/1953118233 >> v2:172.21.15.43:3300/0 conn(0x7f3b4c0d8b40 0x7f3b4c049090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.619+0000 7f3b598c9700 1 -- 172.21.15.40:0/1953118233 >> 172.21.15.40:0/1953118233 conn(0x7f3b4c054110 msgr2=0x7f3b4c054520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:09.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.619+0000 7f3b598c9700 1 -- 172.21.15.40:0/1953118233 shutdown_connections 2024-09-13T17:56:09.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.619+0000 7f3b598c9700 1 -- 172.21.15.40:0/1953118233 wait complete. 2024-09-13T17:56:09.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.623+0000 7f3b598c9700 1 Processor -- start 2024-09-13T17:56:09.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.623+0000 7f3b598c9700 1 -- start start 2024-09-13T17:56:09.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.623+0000 7f3b598c9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b4c007430 0x7f3b4c052d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.623+0000 7f3b598c9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4c04f220 0x7f3b4c053260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.623+0000 7f3b598c9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b4c0d8b40 0x7f3b4c0d9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.623+0000 7f3b598c9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c0db0b0 con 0x7f3b4c0d8b40 2024-09-13T17:56:09.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.623+0000 7f3b598c9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c0c95a0 con 0x7f3b4c04f220 2024-09-13T17:56:09.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.623+0000 7f3b598c9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c052440 con 0x7f3b4c007430 2024-09-13T17:56:09.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.623+0000 7f3b590c8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b4c0d8b40 0x7f3b4c0d9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.623+0000 7f3b590c8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b4c0d8b40 0x7f3b4c0d9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47774/0 (socket says 172.21.15.40:47774) 2024-09-13T17:56:09.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.623+0000 7f3b590c8700 1 -- 172.21.15.40:0/2523090913 learned_addr learned my addr 172.21.15.40:0/2523090913 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:09.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.623+0000 7f3b588c7700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b4c007430 0x7f3b4c052d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.623+0000 7f3b53fff700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4c04f220 0x7f3b4c053260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.623+0000 7f3b590c8700 1 -- 172.21.15.40:0/2523090913 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b4c007430 msgr2=0x7f3b4c052d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.623+0000 7f3b590c8700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b4c007430 0x7f3b4c052d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.623+0000 7f3b590c8700 1 -- 172.21.15.40:0/2523090913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4c04f220 msgr2=0x7f3b4c053260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.623+0000 7f3b590c8700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4c04f220 0x7f3b4c053260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.623+0000 7f3b590c8700 1 -- 172.21.15.40:0/2523090913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5404f5c0 con 0x7f3b4c0d8b40 2024-09-13T17:56:09.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.623+0000 7f3b590c8700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b4c0d8b40 0x7f3b4c0d9200 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b54067bd0 tx=0x7f3b54068470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.623+0000 7f3b517fa700 1 -- 172.21.15.40:0/2523090913 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b54067720 con 0x7f3b4c0d8b40 2024-09-13T17:56:09.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.627+0000 7f3b598c9700 1 -- 172.21.15.40:0/2523090913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b4c175160 con 0x7f3b4c0d8b40 2024-09-13T17:56:09.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.627+0000 7f3b517fa700 1 -- 172.21.15.40:0/2523090913 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b54067880 con 0x7f3b4c0d8b40 2024-09-13T17:56:09.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.627+0000 7f3b517fa700 1 -- 172.21.15.40:0/2523090913 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b54072390 con 0x7f3b4c0d8b40 2024-09-13T17:56:09.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.627+0000 7f3b598c9700 1 -- 172.21.15.40:0/2523090913 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4c175440 con 0x7f3b4c0d8b40 2024-09-13T17:56:09.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.627+0000 7f3b598c9700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4c0d9740 0x7f3b4c0c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.627+0000 7f3b598c9700 1 -- 172.21.15.40:0/2523090913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c0084b0 con 0x7f3b4c0d9740 2024-09-13T17:56:09.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.627+0000 7f3b517fa700 1 -- 172.21.15.40:0/2523090913 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3b54091020 con 0x7f3b4c0d8b40 2024-09-13T17:56:09.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.627+0000 7f3b588c7700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4c0d9740 0x7f3b4c0c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.627+0000 7f3b517fa700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3b38066730 0x7f3b38068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.627+0000 7f3b588c7700 1 -- 172.21.15.40:0/2523090913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3b4c006c30 con 0x7f3b4c0d9740 2024-09-13T17:56:09.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.627+0000 7f3b517fa700 1 -- 172.21.15.40:0/2523090913 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3b540ffa40 con 0x7f3b4c0d8b40 2024-09-13T17:56:09.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.627+0000 7f3b53fff700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3b38066730 0x7f3b38068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.627+0000 7f3b588c7700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4c0d9740 0x7f3b4c0c5950 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3b480119d0 tx=0x7f3b48011d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.627+0000 7f3b517fa700 1 -- 172.21.15.40:0/2523090913 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b48004240 con 0x7f3b4c0d9740 2024-09-13T17:56:09.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.627+0000 7f3b53fff700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3b38066730 0x7f3b38068bf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0d6420 tx=0x7f3b40001e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.631+0000 7f3b517fa700 1 -- 172.21.15.40:0/2523090913 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3b480043a0 con 0x7f3b4c0d9740 2024-09-13T17:56:09.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.631+0000 7f3b517fa700 1 -- 172.21.15.40:0/2523090913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4c0d9740 msgr2=0x7f3b4c0c5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.631+0000 7f3b517fa700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4c0d9740 0x7f3b4c0c5950 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3b480119d0 tx=0x7f3b48011d90 comp rx=0 tx=0).stop 2024-09-13T17:56:09.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.631+0000 7f3b517fa700 1 -- 172.21.15.40:0/2523090913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4c0d9740 msgr2=0x7f3b4c0c5950 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:56:09.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.631+0000 7f3b517fa700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4c0d9740 0x7f3b4c0c5950 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.639+0000 7f3b598c9700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4c0c5e90 0x7f3b4c177ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.639+0000 7f3b598c9700 1 -- 172.21.15.40:0/2523090913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4c1752f0 con 0x7f3b4c0c5e90 2024-09-13T17:56:09.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.639+0000 7f3b588c7700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4c0c5e90 0x7f3b4c177ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.639+0000 7f3b588c7700 1 -- 172.21.15.40:0/2523090913 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3b4c006950 con 0x7f3b4c0c5e90 2024-09-13T17:56:09.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.639+0000 7f3b588c7700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4c0c5e90 0x7f3b4c177ad0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3b480119d0 tx=0x7f3b48004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.639+0000 7f3b517fa700 1 -- 172.21.15.40:0/2523090913 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4801e650 con 0x7f3b4c0c5e90 2024-09-13T17:56:09.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.643+0000 7f3b517fa700 1 -- 172.21.15.40:0/2523090913 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4947 (secure 0 0 0) 0x7f3b480283a0 con 0x7f3b4c0c5e90 2024-09-13T17:56:09.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.643+0000 7f3b517fa700 1 -- 172.21.15.40:0/2523090913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4c0c5e90 msgr2=0x7f3b4c177ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.643+0000 7f3b517fa700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4c0c5e90 0x7f3b4c177ad0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3b480119d0 tx=0x7f3b48004770 comp rx=0 tx=0).stop 2024-09-13T17:56:09.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.643+0000 7f3b517fa700 1 -- 172.21.15.40:0/2523090913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4c0c5e90 msgr2=0x7f3b4c177ad0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:56:09.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.643+0000 7f3b517fa700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b4c0c5e90 0x7f3b4c177ad0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.650 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:56:09.650 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:56:09.651 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:56:09.651 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:56:09.651 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 46, 2024-09-13T17:56:09.651 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:56:09.651 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:56:09.652 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:56:09.652 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:56:09.652 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:09.652 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T17:56:09.652 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:56:09.653 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:56:09.653 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:56:09.654 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:09.654 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:09.654 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:09.654 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:09.654 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:09.654 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:09.654 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:09.654 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:09.654 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:09.654 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:09.654 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:09.654 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:56:09.655 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:56:09.655 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:09.655 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:09.655 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:09.655 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:09.655 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:09.655 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:09.655 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:09.655 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:09.655 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:09.655 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:09.655 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:09.655 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.655 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:56:09.656 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:56:09.656 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:56:09.656 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:56:09.656 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:56:09.656 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:56:09.656 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:56:09.656 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:56:09.656 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:56:09.657 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:56:09.657 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:56:09.657 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:56:09.657 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:56:09.657 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:56:09.657 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:56:09.657 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:56:09.657 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:56:09.657 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:09.657 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:09.657 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:09.658 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:09.658 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:09.658 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:09.658 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:09.658 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:09.658 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:09.658 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:09.658 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:09.658 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:56:09.658 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.658 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:56:09.658 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.658 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:56:09.658 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:56:09.658 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:09.659 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:09.659 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.659 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:09.659 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:56:09.659 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:09.659 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.659 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.659 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:09.659 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:56:09.659 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:09.659 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.659 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:09.660 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.660 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:56:09.660 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:56:09.660 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:09.660 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:09.660 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:09.660 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.660 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.661 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:56:09.661 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:56:09.661 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:09.661 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:09.661 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.661 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:09.661 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:56:09.664 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:09.664 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.664 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.664 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:09.664 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:56:09.664 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:09.664 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.664 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:09.665 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.665 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:56:09.665 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:56:09.665 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:09.665 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:09.665 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:09.665 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.665 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.665 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:56:09.665 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:56:09.665 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:09.665 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:09.665 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.665 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:09.666 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:56:09.666 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:09.666 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.666 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.666 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:09.666 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:56:09.666 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:09.666 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.666 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:09.666 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.667 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:56:09.667 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:56:09.667 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:09.667 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:09.667 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:09.667 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.667 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:09.667 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.667 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:56:09.667 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:56:09.667 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.667 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:09.667 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:09.668 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:56:09.668 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.668 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:09.668 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:56:09.668 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.668 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:09.668 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:09.668 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:56:09.668 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.669 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:09.669 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:56:09.669 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.669 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:09.669 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:09.669 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:56:09.670 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.670 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:09.670 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:56:09.670 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.670 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:09.670 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:09.671 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:56:09.671 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.671 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:09.671 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:56:09.671 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.671 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:09.671 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:09.671 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:56:09.671 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.672 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:09.672 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.672 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:56:09.672 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:56:09.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.643+0000 7f3b598c9700 1 -- 172.21.15.40:0/2523090913 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3b38066730 msgr2=0x7f3b38068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.643+0000 7f3b598c9700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3b38066730 0x7f3b38068bf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0d6420 tx=0x7f3b40001e40 comp rx=0 tx=0).stop 2024-09-13T17:56:09.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.643+0000 7f3b598c9700 1 -- 172.21.15.40:0/2523090913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b4c0d8b40 msgr2=0x7f3b4c0d9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.643+0000 7f3b590c8700 1 -- 172.21.15.40:0/2523090913 reap_dead start 2024-09-13T17:56:09.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.643+0000 7f3b598c9700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b4c0d8b40 0x7f3b4c0d9200 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b54067bd0 tx=0x7f3b54068470 comp rx=0 tx=0).stop 2024-09-13T17:56:09.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.643+0000 7f3b598c9700 1 -- 172.21.15.40:0/2523090913 shutdown_connections 2024-09-13T17:56:09.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.643+0000 7f3b598c9700 1 --2- 172.21.15.40:0/2523090913 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b4c0d8b40 0x7f3b4c0d9200 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.643+0000 7f3b598c9700 1 -- 172.21.15.40:0/2523090913 >> 172.21.15.40:0/2523090913 conn(0x7f3b4c054110 msgr2=0x7f3b4c0d7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:09.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.643+0000 7f3b598c9700 1 -- 172.21.15.40:0/2523090913 shutdown_connections 2024-09-13T17:56:09.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.643+0000 7f3b598c9700 1 -- 172.21.15.40:0/2523090913 wait complete. 2024-09-13T17:56:09.675 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:56:09.675 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T17:56:09.675 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:56:09.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.787+0000 7f1f8142e700 1 Processor -- start 2024-09-13T17:56:09.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.787+0000 7f1f8142e700 1 -- start start 2024-09-13T17:56:09.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f8142e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c066090 0x7f1f7c0c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f8142e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f7c0c2950 0x7f1f7c157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f8142e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7c0c51b0 con 0x7f1f7c1513a0 2024-09-13T17:56:09.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f8142e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f7c0c6690 con 0x7f1f7c0c2950 2024-09-13T17:56:09.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f8142e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c0c2e00 con 0x7f1f7c066090 2024-09-13T17:56:09.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f7bfff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1f7c1513a0 0x7f1f7c0c7490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55626/0 (socket says 172.21.15.40:55626) 2024-09-13T17:56:09.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f80c2d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f7c0c2950 0x7f1f7c157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f7bfff700 1 -- 172.21.15.40:0/2229381651 learned_addr learned my addr 172.21.15.40:0/2229381651 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:56:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f7b7fe700 1 --2- 172.21.15.40:0/2229381651 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c066090 0x7f1f7c0c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f80c2d700 1 -- 172.21.15.40:0/2229381651 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c066090 msgr2=0x7f1f7c0c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:56:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f80c2d700 1 --2- 172.21.15.40:0/2229381651 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c066090 0x7f1f7c0c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f80c2d700 1 -- 172.21.15.40:0/2229381651 >> v1:172.21.15.40:6789/0 conn(0x7f1f7c1513a0 legacy=0x7f1f7c0c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f80c2d700 1 -- 172.21.15.40:0/2229381651 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c0c5470 con 0x7f1f7c0c2950 2024-09-13T17:56:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f7affd700 1 -- 172.21.15.40:0/2229381651 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4113657784 0 0) 0x7f1f7c0c51b0 con 0x7f1f7c1513a0 2024-09-13T17:56:09.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f80c2d700 1 --2- 172.21.15.40:0/2229381651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f7c0c2950 0x7f1f7c157cc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009d90 tx=0x7f1f7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5093852cc802c01 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f7affd700 1 -- 172.21.15.40:0/2229381651 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7400e070 con 0x7f1f7c0c2950 2024-09-13T17:56:09.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f7affd700 1 -- 172.21.15.40:0/2229381651 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7400bb20 con 0x7f1f7c0c2950 2024-09-13T17:56:09.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f7affd700 1 -- 172.21.15.40:0/2229381651 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f74012430 con 0x7f1f7c0c2950 2024-09-13T17:56:09.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f8142e700 1 -- 172.21.15.40:0/2229381651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f7c0c2950 msgr2=0x7f1f7c157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f8142e700 1 --2- 172.21.15.40:0/2229381651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f7c0c2950 0x7f1f7c157cc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009d90 tx=0x7f1f7400b330 comp rx=0 tx=0).stop 2024-09-13T17:56:09.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f8142e700 1 -- 172.21.15.40:0/2229381651 shutdown_connections 2024-09-13T17:56:09.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f8142e700 1 --2- 172.21.15.40:0/2229381651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f7c0c2950 0x7f1f7c157cc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f8142e700 1 --2- 172.21.15.40:0/2229381651 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c066090 0x7f1f7c0c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f8142e700 1 -- 172.21.15.40:0/2229381651 >> 172.21.15.40:0/2229381651 conn(0x7f1f7c0cb450 msgr2=0x7f1f7c0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:09.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f8142e700 1 -- 172.21.15.40:0/2229381651 shutdown_connections 2024-09-13T17:56:09.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.791+0000 7f1f8142e700 1 -- 172.21.15.40:0/2229381651 wait complete. 2024-09-13T17:56:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.795+0000 7f1f8142e700 1 Processor -- start 2024-09-13T17:56:09.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.795+0000 7f1f8142e700 1 -- start start 2024-09-13T17:56:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.795+0000 7f1f8142e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f7c066090 0x7f1f7c0ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.795+0000 7f1f8142e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f7c0c2950 0x7f1f7c1f2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.795+0000 7f1f8142e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c1513a0 0x7f1f7c146cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.795+0000 7f1f8142e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c1e4450 con 0x7f1f7c0c2950 2024-09-13T17:56:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.795+0000 7f1f8142e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c1e38d0 con 0x7f1f7c066090 2024-09-13T17:56:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.795+0000 7f1f8142e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c1e49d0 con 0x7f1f7c1513a0 2024-09-13T17:56:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.795+0000 7f1f7b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f7c0c2950 0x7f1f7c1f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.795+0000 7f1f7b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f7c0c2950 0x7f1f7c1f2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47782/0 (socket says 172.21.15.40:47782) 2024-09-13T17:56:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.795+0000 7f1f7b7fe700 1 -- 172.21.15.40:0/27870938 learned_addr learned my addr 172.21.15.40:0/27870938 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.795+0000 7f1f7bfff700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f7c066090 0x7f1f7c0ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.795+0000 7f1f80c2d700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c1513a0 0x7f1f7c146cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.795+0000 7f1f7b7fe700 1 -- 172.21.15.40:0/27870938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c1513a0 msgr2=0x7f1f7c146cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.795+0000 7f1f7b7fe700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c1513a0 0x7f1f7c146cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.795+0000 7f1f7b7fe700 1 -- 172.21.15.40:0/27870938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f7c066090 msgr2=0x7f1f7c0ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.795+0000 7f1f7b7fe700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f7c066090 0x7f1f7c0ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.795+0000 7f1f7b7fe700 1 -- 172.21.15.40:0/27870938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f74009a70 con 0x7f1f7c0c2950 2024-09-13T17:56:09.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.795+0000 7f1f7b7fe700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f7c0c2950 0x7f1f7c1f2410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000e9a0 tx=0x7f1f7000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f78ff9700 1 -- 172.21.15.40:0/27870938 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70009720 con 0x7f1f7c0c2950 2024-09-13T17:56:09.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f78ff9700 1 -- 172.21.15.40:0/27870938 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7000c690 con 0x7f1f7c0c2950 2024-09-13T17:56:09.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f8142e700 1 -- 172.21.15.40:0/27870938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7c1e4e60 con 0x7f1f7c0c2950 2024-09-13T17:56:09.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f78ff9700 1 -- 172.21.15.40:0/27870938 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7000ca20 con 0x7f1f7c0c2950 2024-09-13T17:56:09.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f8142e700 1 -- 172.21.15.40:0/27870938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7c1e1780 con 0x7f1f7c0c2950 2024-09-13T17:56:09.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f8142e700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c1f2950 0x7f1f7c154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f8142e700 1 -- 172.21.15.40:0/27870938 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c0675d0 con 0x7f1f7c1f2950 2024-09-13T17:56:09.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f7bfff700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c1f2950 0x7f1f7c154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f7bfff700 1 -- 172.21.15.40:0/27870938 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f7c0c62b0 con 0x7f1f7c1f2950 2024-09-13T17:56:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f78ff9700 1 -- 172.21.15.40:0/27870938 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1f7000cc00 con 0x7f1f7c0c2950 2024-09-13T17:56:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f7bfff700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c1f2950 0x7f1f7c154750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0014e0 tx=0x7f1f6c0018a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f78ff9700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f68066730 0x7f1f68068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f78ff9700 1 -- 172.21.15.40:0/27870938 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1f700845c0 con 0x7f1f7c0c2950 2024-09-13T17:56:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f80c2d700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f68066730 0x7f1f68068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f78ff9700 1 -- 172.21.15.40:0/27870938 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c017070 con 0x7f1f7c1f2950 2024-09-13T17:56:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f78ff9700 1 -- 172.21.15.40:0/27870938 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f1f6c012560 con 0x7f1f7c1f2950 2024-09-13T17:56:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f78ff9700 1 -- 172.21.15.40:0/27870938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c1f2950 msgr2=0x7f1f7c154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f78ff9700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c1f2950 0x7f1f7c154750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0014e0 tx=0x7f1f6c0018a0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f78ff9700 1 -- 172.21.15.40:0/27870938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c1f2950 msgr2=0x7f1f7c154750 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:56:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f78ff9700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c1f2950 0x7f1f7c154750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.799+0000 7f1f80c2d700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f68066730 0x7f1f68068bf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1f74014040 tx=0x7f1f74011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.815+0000 7f1f8142e700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c151c20 0x7f1f7c1471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.815+0000 7f1f8142e700 1 -- 172.21.15.40:0/27870938 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c1e4150 con 0x7f1f7c151c20 2024-09-13T17:56:09.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.815+0000 7f1f7bfff700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c151c20 0x7f1f7c1471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.815+0000 7f1f7bfff700 1 -- 172.21.15.40:0/27870938 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1f7c0c5fd0 con 0x7f1f7c151c20 2024-09-13T17:56:09.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.815+0000 7f1f7bfff700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c151c20 0x7f1f7c1471f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c016000 tx=0x7f1f6c001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.815+0000 7f1f78ff9700 1 -- 172.21.15.40:0/27870938 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c022070 con 0x7f1f7c151c20 2024-09-13T17:56:09.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.815+0000 7f1f78ff9700 1 -- 172.21.15.40:0/27870938 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4781 (secure 0 0 0) 0x7f1f6c01de20 con 0x7f1f7c151c20 2024-09-13T17:56:09.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.815+0000 7f1f78ff9700 1 -- 172.21.15.40:0/27870938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c151c20 msgr2=0x7f1f7c1471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.815+0000 7f1f78ff9700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c151c20 0x7f1f7c1471f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c016000 tx=0x7f1f6c001080 comp rx=0 tx=0).stop 2024-09-13T17:56:09.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.815+0000 7f1f78ff9700 1 -- 172.21.15.40:0/27870938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c151c20 msgr2=0x7f1f7c1471f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:56:09.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.815+0000 7f1f78ff9700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f7c151c20 0x7f1f7c1471f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.825 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:56:09.825 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:56:09.825 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:56:09.825 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:56:09.825 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 46, 2024-09-13T17:56:09.825 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:56:09.825 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:56:09.825 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:56:09.825 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:56:09.825 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:09.825 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T17:56:09.825 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:56:09.825 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:56:09.826 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:56:09.826 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:09.826 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:09.826 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:09.826 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:09.826 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:09.826 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:09.826 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:09.826 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:09.826 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:09.826 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:09.826 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:09.826 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:56:09.826 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:56:09.826 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:09.827 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:09.827 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:09.827 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:09.827 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:09.827 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:09.827 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:09.827 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:09.827 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:09.827 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:09.827 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:09.827 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.827 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:56:09.827 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:56:09.827 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:56:09.828 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:56:09.828 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:56:09.828 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:56:09.828 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:56:09.828 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:56:09.828 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:56:09.828 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:56:09.828 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:56:09.828 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:56:09.828 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:56:09.828 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:56:09.828 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:56:09.828 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:56:09.828 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:56:09.829 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:09.829 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:09.829 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:09.829 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:09.829 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:09.829 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:09.829 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:09.829 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:09.829 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:09.829 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:09.829 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:09.829 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:56:09.829 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.829 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:56:09.830 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.830 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:56:09.830 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:56:09.830 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:09.830 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:09.830 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.830 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:09.830 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:56:09.830 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:09.830 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.830 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.830 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:09.830 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:56:09.830 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:09.831 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.831 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:09.831 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.831 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:56:09.831 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:56:09.831 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:09.831 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:09.831 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:09.831 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.831 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.831 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:56:09.831 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:56:09.831 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:09.832 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:09.832 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.832 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:09.832 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:56:09.832 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:09.832 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.832 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.832 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:09.832 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:56:09.832 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:09.832 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.832 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:09.832 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.832 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:56:09.833 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:56:09.833 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:09.833 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:09.833 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:09.833 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.833 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.833 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:56:09.833 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:56:09.833 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:09.833 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:09.833 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.833 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:09.833 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:56:09.833 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:09.834 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.834 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.834 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:09.834 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:56:09.834 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:09.834 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.834 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:09.834 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.834 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:56:09.834 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:56:09.834 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:09.834 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:09.834 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:09.834 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.835 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:09.835 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.835 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:56:09.835 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:56:09.835 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.835 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:09.835 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:09.835 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:56:09.835 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.835 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:09.835 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:56:09.835 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.835 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:09.835 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:09.836 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:56:09.836 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.836 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:09.836 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:56:09.836 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.836 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:09.836 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:09.836 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T17:56:09.836 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.836 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:09.836 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:56:09.836 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:09.836 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:09.837 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:09.837 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:56:09.837 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:09.837 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:09.837 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:09.837 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:56:09.837 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:56:09.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.815+0000 7f1f8142e700 1 -- 172.21.15.40:0/27870938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f68066730 msgr2=0x7f1f68068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.815+0000 7f1f8142e700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f68066730 0x7f1f68068bf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1f74014040 tx=0x7f1f74011000 comp rx=0 tx=0).stop 2024-09-13T17:56:09.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.815+0000 7f1f80c2d700 1 -- 172.21.15.40:0/27870938 reap_dead start 2024-09-13T17:56:09.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.815+0000 7f1f8142e700 1 -- 172.21.15.40:0/27870938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f7c0c2950 msgr2=0x7f1f7c1f2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.815+0000 7f1f8142e700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f7c0c2950 0x7f1f7c1f2410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000e9a0 tx=0x7f1f7000ecb0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.815+0000 7f1f8142e700 1 -- 172.21.15.40:0/27870938 shutdown_connections 2024-09-13T17:56:09.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.815+0000 7f1f8142e700 1 --2- 172.21.15.40:0/27870938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f7c0c2950 0x7f1f7c1f2410 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.815+0000 7f1f8142e700 1 -- 172.21.15.40:0/27870938 >> 172.21.15.40:0/27870938 conn(0x7f1f7c0cb450 msgr2=0x7f1f7c0cb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:09.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.819+0000 7f1f8142e700 1 -- 172.21.15.40:0/27870938 shutdown_connections 2024-09-13T17:56:09.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.819+0000 7f1f8142e700 1 -- 172.21.15.40:0/27870938 wait complete. 2024-09-13T17:56:09.839 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:56:09.839 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T17:56:09.839 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T17:56:09.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.943+0000 7f89288c6700 1 Processor -- start 2024-09-13T17:56:09.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.943+0000 7f89288c6700 1 -- start start 2024-09-13T17:56:09.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f89288c6700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f892406afd0 0x7f8924071f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f89288c6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f89240c2e60 0x7f89240706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f89288c6700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89240717d0 con 0x7f892414fb00 2024-09-13T17:56:09.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f89288c6700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8924071b20 con 0x7f892406afd0 2024-09-13T17:56:09.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f89288c6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892406add0 con 0x7f89240c2e60 2024-09-13T17:56:09.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f8922d9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f892414fb00 0x7f8924158ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55628/0 (socket says 172.21.15.40:55628) 2024-09-13T17:56:09.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f8922d9d700 1 -- 172.21.15.40:0/3237041859 learned_addr learned my addr 172.21.15.40:0/3237041859 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:56:09.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f892359e700 1 --2- 172.21.15.40:0/3237041859 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f89240c2e60 0x7f89240706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f891bfff700 1 --2- 172.21.15.40:0/3237041859 >> v2:172.21.15.43:3300/0 conn(0x7f892406afd0 0x7f8924071f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f892259c700 1 -- 172.21.15.40:0/3237041859 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1992118653 0 0) 0x7f89240717d0 con 0x7f892414fb00 2024-09-13T17:56:09.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f892259c700 1 -- 172.21.15.40:0/3237041859 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89100046e0 con 0x7f892414fb00 2024-09-13T17:56:09.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f892259c700 1 -- 172.21.15.40:0/3237041859 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3163207899 0 0) 0x7f89100046e0 con 0x7f892414fb00 2024-09-13T17:56:09.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f892259c700 1 -- 172.21.15.40:0/3237041859 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f89240c2e60 msgr2=0x7f89240706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:56:09.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f892259c700 1 --2- 172.21.15.40:0/3237041859 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f89240c2e60 0x7f89240706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f892259c700 1 -- 172.21.15.40:0/3237041859 >> v2:172.21.15.43:3300/0 conn(0x7f892406afd0 msgr2=0x7f8924071f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:56:09.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f892259c700 1 --2- 172.21.15.40:0/3237041859 >> v2:172.21.15.43:3300/0 conn(0x7f892406afd0 0x7f8924071f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f892259c700 1 -- 172.21.15.40:0/3237041859 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89240c32d0 con 0x7f892414fb00 2024-09-13T17:56:09.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f892259c700 1 -- 172.21.15.40:0/3237041859 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f8914001670 con 0x7f892414fb00 2024-09-13T17:56:09.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f892259c700 1 -- 172.21.15.40:0/3237041859 >> v1:172.21.15.40:6789/0 conn(0x7f892414fb00 legacy=0x7f8924158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f892259c700 1 --2- 172.21.15.40:0/3237041859 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f89240c2e60 0x7f8910004b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f892259c700 1 --2- 172.21.15.40:0/3237041859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89100050e0 0x7f8910005980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f891bfff700 1 --2- 172.21.15.40:0/3237041859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89100050e0 0x7f8910005980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f892259c700 1 --2- 172.21.15.40:0/3237041859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8910005ec0 0x7f8910006340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f892259c700 1 -- 172.21.15.40:0/3237041859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8924071b20 con 0x7f89100050e0 2024-09-13T17:56:09.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f892259c700 1 -- 172.21.15.40:0/3237041859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892406add0 con 0x7f8910005ec0 2024-09-13T17:56:09.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f892259c700 1 -- 172.21.15.40:0/3237041859 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891000cb40 con 0x7f89240c2e60 2024-09-13T17:56:09.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f8922d9d700 1 --2- 172.21.15.40:0/3237041859 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f89240c2e60 0x7f8910004b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f892359e700 1 --2- 172.21.15.40:0/3237041859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8910005ec0 0x7f8910006340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f891bfff700 1 -- 172.21.15.40:0/3237041859 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f89240c2e60 msgr2=0x7f8910004b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:56:09.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f891bfff700 1 --2- 172.21.15.40:0/3237041859 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f89240c2e60 0x7f8910004b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f891bfff700 1 -- 172.21.15.40:0/3237041859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8910005ec0 msgr2=0x7f8910006340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:56:09.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f891bfff700 1 --2- 172.21.15.40:0/3237041859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8910005ec0 0x7f8910006340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.947+0000 7f891bfff700 1 -- 172.21.15.40:0/3237041859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89100046e0 con 0x7f89100050e0 2024-09-13T17:56:09.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.951+0000 7f891bfff700 1 --2- 172.21.15.40:0/3237041859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89100050e0 0x7f8910005980 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f890c009b70 tx=0x7f89100069b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dec3ca72954b645 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.951+0000 7f892259c700 1 -- 172.21.15.40:0/3237041859 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c00e6c0 con 0x7f89100050e0 2024-09-13T17:56:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.951+0000 7f892259c700 1 -- 172.21.15.40:0/3237041859 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890c00e820 con 0x7f89100050e0 2024-09-13T17:56:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.951+0000 7f892259c700 1 -- 172.21.15.40:0/3237041859 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c00eb90 con 0x7f89100050e0 2024-09-13T17:56:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.951+0000 7f89288c6700 1 -- 172.21.15.40:0/3237041859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89100050e0 msgr2=0x7f8910005980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.951+0000 7f89288c6700 1 --2- 172.21.15.40:0/3237041859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89100050e0 0x7f8910005980 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f890c009b70 tx=0x7f89100069b0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.951+0000 7f892359e700 1 -- 172.21.15.40:0/3237041859 reap_dead start 2024-09-13T17:56:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.951+0000 7f89288c6700 1 -- 172.21.15.40:0/3237041859 shutdown_connections 2024-09-13T17:56:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.951+0000 7f89288c6700 1 -- 172.21.15.40:0/3237041859 >> 172.21.15.40:0/3237041859 conn(0x7f89240cabc0 msgr2=0x7f89240cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.951+0000 7f89288c6700 1 -- 172.21.15.40:0/3237041859 shutdown_connections 2024-09-13T17:56:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.951+0000 7f89288c6700 1 -- 172.21.15.40:0/3237041859 wait complete. 2024-09-13T17:56:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.951+0000 7f89288c6700 1 Processor -- start 2024-09-13T17:56:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.951+0000 7f89288c6700 1 -- start start 2024-09-13T17:56:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f89288c6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89100050e0 0x7f89241540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f89288c6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f892414adc0 0x7f89241585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f89288c6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89241eda40 0x7f89240c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f89288c6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892414ac40 con 0x7f89100050e0 2024-09-13T17:56:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f89288c6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89240b8500 con 0x7f89241eda40 2024-09-13T17:56:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f89288c6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8924158160 con 0x7f892414adc0 2024-09-13T17:56:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f8922d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89100050e0 0x7f89241540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f8922d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89100050e0 0x7f89241540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47796/0 (socket says 172.21.15.40:47796) 2024-09-13T17:56:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f8922d9d700 1 -- 172.21.15.40:0/3782235081 learned_addr learned my addr 172.21.15.40:0/3782235081 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f891bfff700 1 --2- 172.21.15.40:0/3782235081 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f892414adc0 0x7f89241585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f892359e700 1 --2- 172.21.15.40:0/3782235081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89241eda40 0x7f89240c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f8922d9d700 1 -- 172.21.15.40:0/3782235081 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f892414adc0 msgr2=0x7f89241585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f8922d9d700 1 --2- 172.21.15.40:0/3782235081 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f892414adc0 0x7f89241585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f8922d9d700 1 -- 172.21.15.40:0/3782235081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89241eda40 msgr2=0x7f89240c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:09.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f8922d9d700 1 --2- 172.21.15.40:0/3782235081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89241eda40 0x7f89240c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:09.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f8922d9d700 1 -- 172.21.15.40:0/3782235081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89241591f0 con 0x7f89100050e0 2024-09-13T17:56:09.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f8922d9d700 1 --2- 172.21.15.40:0/3782235081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89100050e0 0x7f89241540a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f89140072f0 tx=0x7f89140076b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f891b7fe700 1 -- 172.21.15.40:0/3782235081 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891400f870 con 0x7f89100050e0 2024-09-13T17:56:09.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f89288c6700 1 -- 172.21.15.40:0/3782235081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89241594d0 con 0x7f89100050e0 2024-09-13T17:56:09.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f891b7fe700 1 -- 172.21.15.40:0/3782235081 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891400f9f0 con 0x7f89100050e0 2024-09-13T17:56:09.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f891b7fe700 1 -- 172.21.15.40:0/3782235081 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891400fda0 con 0x7f89100050e0 2024-09-13T17:56:09.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f89288c6700 1 -- 172.21.15.40:0/3782235081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8924157b10 con 0x7f89100050e0 2024-09-13T17:56:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.955+0000 7f89288c6700 1 -- 172.21.15.40:0/3782235081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f892414fee0 con 0x7f89100050e0 2024-09-13T17:56:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.959+0000 7f891b7fe700 1 -- 172.21.15.40:0/3782235081 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f891401d360 con 0x7f89100050e0 2024-09-13T17:56:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.959+0000 7f891b7fe700 1 --2- 172.21.15.40:0/3782235081 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8904066760 0x7f8904068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.959+0000 7f891b7fe700 1 -- 172.21.15.40:0/3782235081 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8914083de0 con 0x7f89100050e0 2024-09-13T17:56:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.959+0000 7f891bfff700 1 --2- 172.21.15.40:0/3782235081 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8904066760 0x7f8904068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.959+0000 7f891bfff700 1 --2- 172.21.15.40:0/3782235081 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8904066760 0x7f8904068c20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f890c0090b0 tx=0x7f890c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:09.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:09.967+0000 7f891b7fe700 1 -- 172.21.15.40:0/3782235081 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8914050c80 con 0x7f89100050e0 2024-09-13T17:56:10.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:10.223+0000 7f89288c6700 1 -- 172.21.15.40:0/3782235081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f89240c84e0 con 0x7f89100050e0 2024-09-13T17:56:10.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:10.223+0000 7f891b7fe700 1 -- 172.21.15.40:0/3782235081 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f89140542a0 con 0x7f89100050e0 2024-09-13T17:56:10.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:10.227+0000 7f89288c6700 1 -- 172.21.15.40:0/3782235081 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8904066760 msgr2=0x7f8904068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:10.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:10.227+0000 7f89288c6700 1 --2- 172.21.15.40:0/3782235081 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8904066760 0x7f8904068c20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f890c0090b0 tx=0x7f890c008030 comp rx=0 tx=0).stop 2024-09-13T17:56:10.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:10.227+0000 7f89288c6700 1 -- 172.21.15.40:0/3782235081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89100050e0 msgr2=0x7f89241540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:10.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:10.227+0000 7f89288c6700 1 --2- 172.21.15.40:0/3782235081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89100050e0 0x7f89241540a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f89140072f0 tx=0x7f89140076b0 comp rx=0 tx=0).stop 2024-09-13T17:56:10.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:10.227+0000 7f89288c6700 1 -- 172.21.15.40:0/3782235081 shutdown_connections 2024-09-13T17:56:10.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:10.227+0000 7f89288c6700 1 --2- 172.21.15.40:0/3782235081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89241eda40 0x7f89240c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:10.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:10.227+0000 7f89288c6700 1 --2- 172.21.15.40:0/3782235081 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8904066760 0x7f8904068c20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:10.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:10.227+0000 7f89288c6700 1 --2- 172.21.15.40:0/3782235081 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f892414adc0 0x7f89241585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:10.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:10.227+0000 7f89288c6700 1 --2- 172.21.15.40:0/3782235081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89100050e0 0x7f89241540a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:10.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:10.227+0000 7f89288c6700 1 -- 172.21.15.40:0/3782235081 >> 172.21.15.40:0/3782235081 conn(0x7f89240cabc0 msgr2=0x7f89240c4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:10.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:10.227+0000 7f89288c6700 1 -- 172.21.15.40:0/3782235081 shutdown_connections 2024-09-13T17:56:10.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:10.227+0000 7f89288c6700 1 -- 172.21.15.40:0/3782235081 wait complete. 2024-09-13T17:56:10.252 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T17:56:20.253 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T17:56:20.253 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:56:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f768dfb4700 1 Processor -- start 2024-09-13T17:56:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f768dfb4700 1 -- start start 2024-09-13T17:56:20.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f768dfb4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f76880ca070 0x7f76880c1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f768dfb4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f76880c2450 0x7f7688159950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f768dfb4700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f768815da20 con 0x7f7688156bd0 2024-09-13T17:56:20.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f768dfb4700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f768806f080 con 0x7f76880ca070 2024-09-13T17:56:20.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f768dfb4700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f768815cbc0 con 0x7f76880c2450 2024-09-13T17:56:20.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f7687fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f76880ca070 0x7f76880c1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f7687fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f76880ca070 0x7f76880c1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40246/0 (socket says 172.21.15.40:40246) 2024-09-13T17:56:20.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f7687fff700 1 -- 172.21.15.40:0/2660885484 learned_addr learned my addr 172.21.15.40:0/2660885484 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:20.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f768d7b3700 1 --2- 172.21.15.40:0/2660885484 >> v2:172.21.15.43:3300/0 conn(0x7f76880c2450 0x7f7688159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f7687fff700 1 -- 172.21.15.40:0/2660885484 >> v2:172.21.15.43:3300/0 conn(0x7f76880c2450 msgr2=0x7f7688159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:56:20.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f7687fff700 1 --2- 172.21.15.40:0/2660885484 >> v2:172.21.15.43:3300/0 conn(0x7f76880c2450 0x7f7688159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f7687fff700 1 -- 172.21.15.40:0/2660885484 >> v1:172.21.15.43:6789/0 conn(0x7f7688156bd0 legacy=0x7f768806e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:56:20.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f7687fff700 1 -- 172.21.15.40:0/2660885484 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768815cea0 con 0x7f76880ca070 2024-09-13T17:56:20.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f768d7b3700 1 --2- 172.21.15.40:0/2660885484 >> v2:172.21.15.43:3300/0 conn(0x7f76880c2450 0x7f7688159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:56:20.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f7687fff700 1 --2- 172.21.15.40:0/2660885484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76880ca070 0x7f76880c1f10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f76880b1f50 tx=0x7f7674009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e4002dea035645 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:20.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f76877fe700 1 -- 172.21.15.40:0/2660885484 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7674004230 con 0x7f76880ca070 2024-09-13T17:56:20.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f76877fe700 1 -- 172.21.15.40:0/2660885484 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7674004390 con 0x7f76880ca070 2024-09-13T17:56:20.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f76877fe700 1 -- 172.21.15.40:0/2660885484 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76740036c0 con 0x7f76880ca070 2024-09-13T17:56:20.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f768dfb4700 1 -- 172.21.15.40:0/2660885484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76880ca070 msgr2=0x7f76880c1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.362+0000 7f768dfb4700 1 --2- 172.21.15.40:0/2660885484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76880ca070 0x7f76880c1f10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f76880b1f50 tx=0x7f7674009bf0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f768dfb4700 1 -- 172.21.15.40:0/2660885484 shutdown_connections 2024-09-13T17:56:20.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f768dfb4700 1 --2- 172.21.15.40:0/2660885484 >> v2:172.21.15.43:3300/0 conn(0x7f76880c2450 0x7f7688159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f768dfb4700 1 --2- 172.21.15.40:0/2660885484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76880ca070 0x7f76880c1f10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f768dfb4700 1 -- 172.21.15.40:0/2660885484 >> 172.21.15.40:0/2660885484 conn(0x7f7688066090 msgr2=0x7f7688066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:20.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f768dfb4700 1 -- 172.21.15.40:0/2660885484 shutdown_connections 2024-09-13T17:56:20.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f768dfb4700 1 -- 172.21.15.40:0/2660885484 wait complete. 2024-09-13T17:56:20.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f768dfb4700 1 Processor -- start 2024-09-13T17:56:20.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f768dfb4700 1 -- start start 2024-09-13T17:56:20.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f768dfb4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76880c2450 0x7f76881efd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f768dfb4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76880ca070 0x7f7688151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f768dfb4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7688156bd0 0x7f768814c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f768dfb4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76881ef910 con 0x7f76880ca070 2024-09-13T17:56:20.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f768dfb4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7688159420 con 0x7f76880c2450 2024-09-13T17:56:20.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f768dfb4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76881ef290 con 0x7f7688156bd0 2024-09-13T17:56:20.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f7687fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76880ca070 0x7f7688151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f7687fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76880ca070 0x7f7688151d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40254/0 (socket says 172.21.15.40:40254) 2024-09-13T17:56:20.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f7687fff700 1 -- 172.21.15.40:0/1993455380 learned_addr learned my addr 172.21.15.40:0/1993455380 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:20.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f768cfb2700 1 --2- 172.21.15.40:0/1993455380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76880c2450 0x7f76881efd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f768d7b3700 1 --2- 172.21.15.40:0/1993455380 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7688156bd0 0x7f768814c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f7687fff700 1 -- 172.21.15.40:0/1993455380 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7688156bd0 msgr2=0x7f768814c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f7687fff700 1 --2- 172.21.15.40:0/1993455380 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7688156bd0 0x7f768814c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f7687fff700 1 -- 172.21.15.40:0/1993455380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76880c2450 msgr2=0x7f76881efd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f7687fff700 1 --2- 172.21.15.40:0/1993455380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76880c2450 0x7f76881efd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.366+0000 7f7687fff700 1 -- 172.21.15.40:0/1993455380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7674009570 con 0x7f76880ca070 2024-09-13T17:56:20.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.370+0000 7f7687fff700 1 --2- 172.21.15.40:0/1993455380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76880ca070 0x7f7688151d40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7674004000 tx=0x7f7674004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:20.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.370+0000 7f76857fa700 1 -- 172.21.15.40:0/1993455380 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7674003d00 con 0x7f76880ca070 2024-09-13T17:56:20.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.370+0000 7f76857fa700 1 -- 172.21.15.40:0/1993455380 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767401f070 con 0x7f76880ca070 2024-09-13T17:56:20.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.370+0000 7f768dfb4700 1 -- 172.21.15.40:0/1993455380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76881f3280 con 0x7f76880ca070 2024-09-13T17:56:20.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.370+0000 7f76857fa700 1 -- 172.21.15.40:0/1993455380 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767400b8a0 con 0x7f76880ca070 2024-09-13T17:56:20.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.370+0000 7f768dfb4700 1 -- 172.21.15.40:0/1993455380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76881f3560 con 0x7f76880ca070 2024-09-13T17:56:20.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.370+0000 7f768dfb4700 1 -- 172.21.15.40:0/1993455380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76881efa50 con 0x7f76880ca070 2024-09-13T17:56:20.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.370+0000 7f76857fa700 1 -- 172.21.15.40:0/1993455380 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7674021030 con 0x7f76880ca070 2024-09-13T17:56:20.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.370+0000 7f76857fa700 1 --2- 172.21.15.40:0/1993455380 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f766c066730 0x7f766c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.370+0000 7f76857fa700 1 -- 172.21.15.40:0/1993455380 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7674085650 con 0x7f76880ca070 2024-09-13T17:56:20.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.374+0000 7f768cfb2700 1 --2- 172.21.15.40:0/1993455380 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f766c066730 0x7f766c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.374+0000 7f768cfb2700 1 --2- 172.21.15.40:0/1993455380 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f766c066730 0x7f766c068bf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f767c009980 tx=0x7f767c0090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:20.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.378+0000 7f76857fa700 1 -- 172.21.15.40:0/1993455380 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f76881efa50 con 0x7f76880ca070 2024-09-13T17:56:20.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.638+0000 7f768dfb4700 1 -- 172.21.15.40:0/1993455380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f76881efa50 con 0x7f76880ca070 2024-09-13T17:56:20.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.642+0000 7f76857fa700 1 -- 172.21.15.40:0/1993455380 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f76881efa50 con 0x7f76880ca070 2024-09-13T17:56:20.649 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:56:20.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.642+0000 7f768dfb4700 1 -- 172.21.15.40:0/1993455380 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f766c066730 msgr2=0x7f766c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.642+0000 7f768dfb4700 1 --2- 172.21.15.40:0/1993455380 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f766c066730 0x7f766c068bf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f767c009980 tx=0x7f767c0090e0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.642+0000 7f768dfb4700 1 -- 172.21.15.40:0/1993455380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76880ca070 msgr2=0x7f7688151d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.642+0000 7f768dfb4700 1 --2- 172.21.15.40:0/1993455380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76880ca070 0x7f7688151d40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7674004000 tx=0x7f7674004770 comp rx=0 tx=0).stop 2024-09-13T17:56:20.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.642+0000 7f768dfb4700 1 -- 172.21.15.40:0/1993455380 shutdown_connections 2024-09-13T17:56:20.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.642+0000 7f768dfb4700 1 --2- 172.21.15.40:0/1993455380 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f766c066730 0x7f766c068bf0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.642+0000 7f768dfb4700 1 --2- 172.21.15.40:0/1993455380 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7688156bd0 0x7f768814c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.642+0000 7f768dfb4700 1 --2- 172.21.15.40:0/1993455380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76880ca070 0x7f7688151d40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.642+0000 7f768dfb4700 1 --2- 172.21.15.40:0/1993455380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76880c2450 0x7f76881efd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.642+0000 7f768dfb4700 1 -- 172.21.15.40:0/1993455380 >> 172.21.15.40:0/1993455380 conn(0x7f7688066090 msgr2=0x7f7688155e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:20.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.642+0000 7f768dfb4700 1 -- 172.21.15.40:0/1993455380 shutdown_connections 2024-09-13T17:56:20.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.642+0000 7f768dfb4700 1 -- 172.21.15.40:0/1993455380 wait complete. 2024-09-13T17:56:20.664 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T17:56:20.664 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T17:56:20.665 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T17:56:20.665 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:56:20.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.770+0000 7f4186b28700 1 Processor -- start 2024-09-13T17:56:20.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.770+0000 7f4186b28700 1 -- start start 2024-09-13T17:56:20.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4186b28700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f41801512e0 0x7f418015a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4186b28700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f41800c7310 0x7f418006afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4186b28700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4180157f00 0x7f41800c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4186b28700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4180158310 con 0x7f41800c7310 2024-09-13T17:56:20.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4186b28700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41800724e0 con 0x7f41801512e0 2024-09-13T17:56:20.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4186b28700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4180157420 con 0x7f4180157f00 2024-09-13T17:56:20.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4185325700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f41800c7310 0x7f418006afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4185325700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f41800c7310 0x7f418006afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40258/0 (socket says 172.21.15.40:40258) 2024-09-13T17:56:20.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4185325700 1 -- 172.21.15.40:0/3905937048 learned_addr learned my addr 172.21.15.40:0/3905937048 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:20.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4185b26700 1 --2- 172.21.15.40:0/3905937048 >> v2:172.21.15.43:3300/0 conn(0x7f41801512e0 0x7f418015a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4186327700 1 --2- 172.21.15.40:0/3905937048 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4180157f00 0x7f41800c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4185325700 1 -- 172.21.15.40:0/3905937048 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4180157f00 msgr2=0x7f41800c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:56:20.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4185325700 1 --2- 172.21.15.40:0/3905937048 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4180157f00 0x7f41800c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4185325700 1 -- 172.21.15.40:0/3905937048 >> v2:172.21.15.43:3300/0 conn(0x7f41801512e0 msgr2=0x7f418015a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:56:20.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4185325700 1 --2- 172.21.15.40:0/3905937048 >> v2:172.21.15.43:3300/0 conn(0x7f41801512e0 0x7f418015a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4185325700 1 -- 172.21.15.40:0/3905937048 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4180157700 con 0x7f41800c7310 2024-09-13T17:56:20.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4185325700 1 --2- 172.21.15.40:0/3905937048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41800c7310 0x7f418006afe0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f41800cbde0 tx=0x7f4170009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8690d08502465c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:20.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4184b24700 1 -- 172.21.15.40:0/3905937048 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170011830 con 0x7f41800c7310 2024-09-13T17:56:20.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4184b24700 1 -- 172.21.15.40:0/3905937048 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4170011990 con 0x7f41800c7310 2024-09-13T17:56:20.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4184b24700 1 -- 172.21.15.40:0/3905937048 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170011d20 con 0x7f41800c7310 2024-09-13T17:56:20.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4186b28700 1 -- 172.21.15.40:0/3905937048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41800c7310 msgr2=0x7f418006afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4186b28700 1 --2- 172.21.15.40:0/3905937048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41800c7310 0x7f418006afe0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f41800cbde0 tx=0x7f4170009bf0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4186b28700 1 -- 172.21.15.40:0/3905937048 shutdown_connections 2024-09-13T17:56:20.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4186b28700 1 --2- 172.21.15.40:0/3905937048 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4180157f00 0x7f41800c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4186b28700 1 --2- 172.21.15.40:0/3905937048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41800c7310 0x7f418006afe0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4186b28700 1 --2- 172.21.15.40:0/3905937048 >> v2:172.21.15.43:3300/0 conn(0x7f41801512e0 0x7f418015a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4186b28700 1 -- 172.21.15.40:0/3905937048 >> 172.21.15.40:0/3905937048 conn(0x7f4180066090 msgr2=0x7f41800664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:20.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4186b28700 1 -- 172.21.15.40:0/3905937048 shutdown_connections 2024-09-13T17:56:20.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.774+0000 7f4186b28700 1 -- 172.21.15.40:0/3905937048 wait complete. 2024-09-13T17:56:20.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.778+0000 7f4186b28700 1 Processor -- start 2024-09-13T17:56:20.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.778+0000 7f4186b28700 1 -- start start 2024-09-13T17:56:20.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.778+0000 7f4186b28700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41800c7310 0x7f41800ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.778+0000 7f4186b28700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41801512e0 0x7f41801598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.778+0000 7f4186b28700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4180157f00 0x7f41800ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.778+0000 7f4186b28700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41800aec20 con 0x7f41800c7310 2024-09-13T17:56:20.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.778+0000 7f4186b28700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41800c9d20 con 0x7f4180157f00 2024-09-13T17:56:20.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.778+0000 7f4186b28700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41800aefa0 con 0x7f41801512e0 2024-09-13T17:56:20.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.778+0000 7f4185b26700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41800c7310 0x7f41800ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.778+0000 7f4185b26700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41800c7310 0x7f41800ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40262/0 (socket says 172.21.15.40:40262) 2024-09-13T17:56:20.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.778+0000 7f4185b26700 1 -- 172.21.15.40:0/630237350 learned_addr learned my addr 172.21.15.40:0/630237350 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:20.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.778+0000 7f4186327700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4180157f00 0x7f41800ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.778+0000 7f4185325700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41801512e0 0x7f41801598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.778+0000 7f4185b26700 1 -- 172.21.15.40:0/630237350 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41801512e0 msgr2=0x7f41801598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.778+0000 7f4185b26700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41801512e0 0x7f41801598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.778+0000 7f4185b26700 1 -- 172.21.15.40:0/630237350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4180157f00 msgr2=0x7f41800ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.778+0000 7f4185b26700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4180157f00 0x7f41800ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.778+0000 7f4185b26700 1 -- 172.21.15.40:0/630237350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4170009570 con 0x7f41800c7310 2024-09-13T17:56:20.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.778+0000 7f4185b26700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41800c7310 0x7f41800ae6e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f417c00e940 tx=0x7f417c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:20.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f41767fc700 1 -- 172.21.15.40:0/630237350 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c013680 con 0x7f41800c7310 2024-09-13T17:56:20.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f41767fc700 1 -- 172.21.15.40:0/630237350 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417c0137e0 con 0x7f41800c7310 2024-09-13T17:56:20.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f4186b28700 1 -- 172.21.15.40:0/630237350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41800aff10 con 0x7f41800c7310 2024-09-13T17:56:20.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f41767fc700 1 -- 172.21.15.40:0/630237350 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c013b70 con 0x7f41800c7310 2024-09-13T17:56:20.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f4186b28700 1 -- 172.21.15.40:0/630237350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41800c9f30 con 0x7f41800c7310 2024-09-13T17:56:20.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f4186b28700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168002a40 0x7f4168004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f4186b28700 1 -- 172.21.15.40:0/630237350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4168007f40 con 0x7f4168002a40 2024-09-13T17:56:20.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f4185325700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168002a40 0x7f4168004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f4185325700 1 -- 172.21.15.40:0/630237350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4168008100 con 0x7f4168002a40 2024-09-13T17:56:20.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f4185325700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168002a40 0x7f4168004f00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f41800cbde0 tx=0x7f4170004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:20.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f41767fc700 1 -- 172.21.15.40:0/630237350 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170020070 con 0x7f4168002a40 2024-09-13T17:56:20.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f41767fc700 1 -- 172.21.15.40:0/630237350 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f417c013cd0 con 0x7f41800c7310 2024-09-13T17:56:20.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f41767fc700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4164066800 0x7f4164068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f41767fc700 1 -- 172.21.15.40:0/630237350 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f417c0842f0 con 0x7f41800c7310 2024-09-13T17:56:20.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f4186327700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4164066800 0x7f4164068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f41767fc700 1 -- 172.21.15.40:0/630237350 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4170004250 con 0x7f4168002a40 2024-09-13T17:56:20.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f41767fc700 1 -- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168002a40 msgr2=0x7f4168004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f41767fc700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168002a40 0x7f4168004f00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f41800cbde0 tx=0x7f4170004770 comp rx=0 tx=0).stop 2024-09-13T17:56:20.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f41767fc700 1 -- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168002a40 msgr2=0x7f4168004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:56:20.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f41767fc700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168002a40 0x7f4168004f00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.782+0000 7f4186327700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4164066800 0x7f4164068cc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f41800cbcd0 tx=0x7f417800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:20.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f4186b28700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168006fd0 0x7f4168007460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f4186b28700 1 -- 172.21.15.40:0/630237350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416800a4a0 con 0x7f4168006fd0 2024-09-13T17:56:20.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f4185325700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168006fd0 0x7f4168007460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f4185325700 1 -- 172.21.15.40:0/630237350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f41680079a0 con 0x7f4168006fd0 2024-09-13T17:56:20.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f4185325700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168006fd0 0x7f4168007460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4170009150 tx=0x7f4170011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:20.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f41767fc700 1 -- 172.21.15.40:0/630237350 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417002b880 con 0x7f4168006fd0 2024-09-13T17:56:20.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f41767fc700 1 -- 172.21.15.40:0/630237350 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4618 (secure 0 0 0) 0x7f417002f070 con 0x7f4168006fd0 2024-09-13T17:56:20.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f41767fc700 1 -- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168006fd0 msgr2=0x7f4168007460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f41767fc700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168006fd0 0x7f4168007460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4170009150 tx=0x7f4170011000 comp rx=0 tx=0).stop 2024-09-13T17:56:20.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f41767fc700 1 -- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168006fd0 msgr2=0x7f4168007460 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:56:20.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f41767fc700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4168006fd0 0x7f4168007460 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.803 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:56:20.804 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:56:20.804 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:56:20.804 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:56:20.804 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 46, 2024-09-13T17:56:20.804 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:56:20.804 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:56:20.804 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:56:20.804 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:56:20.804 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:20.804 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 13, 2024-09-13T17:56:20.804 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:56:20.804 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:56:20.805 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:56:20.805 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:20.805 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:20.805 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:20.805 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:20.805 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:20.805 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:20.805 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:20.805 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:20.805 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:20.805 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:20.805 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:20.805 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:56:20.805 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:56:20.806 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:20.806 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:20.806 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:20.806 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:20.806 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:20.806 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:20.806 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:20.806 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:20.806 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:20.806 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:20.806 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:20.806 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.806 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:56:20.806 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:56:20.807 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:56:20.807 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:56:20.807 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:56:20.807 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:56:20.807 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:56:20.807 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:56:20.807 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:56:20.807 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:56:20.807 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:56:20.807 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:56:20.807 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:56:20.807 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:56:20.807 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:56:20.808 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:56:20.808 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:56:20.808 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:20.808 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:20.808 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:20.808 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:20.809 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:20.809 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:20.809 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:20.809 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:20.809 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:20.809 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:20.809 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:20.809 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:56:20.809 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.809 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:56:20.809 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.809 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:56:20.809 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:56:20.810 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:20.810 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:20.810 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.810 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:20.810 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:56:20.810 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:20.810 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.810 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.810 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:20.810 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:56:20.810 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:20.810 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:20.810 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:20.810 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.811 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:56:20.811 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:56:20.811 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:20.811 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:20.811 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:20.811 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.811 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.811 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:56:20.811 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:56:20.811 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:20.811 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:20.811 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.811 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:20.812 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:56:20.812 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:20.812 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.812 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.812 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:20.812 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:56:20.812 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:20.812 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:20.812 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:20.812 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.812 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:56:20.812 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:56:20.812 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:20.812 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:20.813 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:20.813 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.813 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.813 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:56:20.813 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:56:20.813 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:20.813 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:20.813 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.813 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:20.813 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:56:20.813 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:20.813 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.813 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.813 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:20.814 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:56:20.814 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:20.814 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:20.814 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:20.814 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.814 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:56:20.814 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:56:20.814 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:20.814 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:20.814 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:20.814 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:20.814 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:20.814 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.814 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:56:20.815 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:56:20.815 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.815 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:20.815 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:20.815 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:56:20.815 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:20.815 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:20.815 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:56:20.815 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.815 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:20.815 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:20.815 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:56:20.815 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:20.815 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:20.816 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:56:20.816 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.816 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:20.816 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:20.816 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:56:20.816 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:20.816 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:20.816 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.816 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:56:20.816 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:56:20.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f4186b28700 1 -- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4164066800 msgr2=0x7f4164068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f4186b28700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4164066800 0x7f4164068cc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f41800cbcd0 tx=0x7f417800b040 comp rx=0 tx=0).stop 2024-09-13T17:56:20.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f4186b28700 1 -- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41800c7310 msgr2=0x7f41800ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f4186b28700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41800c7310 0x7f41800ae6e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f417c00e940 tx=0x7f417c00ec50 comp rx=0 tx=0).stop 2024-09-13T17:56:20.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f4186327700 1 -- 172.21.15.40:0/630237350 reap_dead start 2024-09-13T17:56:20.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f4186b28700 1 -- 172.21.15.40:0/630237350 shutdown_connections 2024-09-13T17:56:20.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f4186b28700 1 --2- 172.21.15.40:0/630237350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41800c7310 0x7f41800ae6e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f4186b28700 1 -- 172.21.15.40:0/630237350 >> 172.21.15.40:0/630237350 conn(0x7f4180066090 msgr2=0x7f41800c7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:20.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f4186b28700 1 -- 172.21.15.40:0/630237350 shutdown_connections 2024-09-13T17:56:20.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.794+0000 7f4186b28700 1 -- 172.21.15.40:0/630237350 wait complete. 2024-09-13T17:56:20.823 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:56:20.823 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T17:56:20.823 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:56:20.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.926+0000 7facd28e8700 1 Processor -- start 2024-09-13T17:56:20.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.926+0000 7facd28e8700 1 -- start start 2024-09-13T17:56:20.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.926+0000 7facd28e8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7facc40dc4d0 0x7facc4044be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.926+0000 7facd28e8700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc40dc980 con 0x7facc40d6750 2024-09-13T17:56:20.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.926+0000 7facd28e8700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc40dcc60 con 0x7facc40509b0 2024-09-13T17:56:20.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.926+0000 7facd28e8700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc40dd500 con 0x7facc40dc4d0 2024-09-13T17:56:20.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.926+0000 7facd10e5700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7facc40dc4d0 0x7facc4044be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.926+0000 7facd10e5700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7facc40dc4d0 0x7facc4044be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40278/0 (socket says 172.21.15.40:40278) 2024-09-13T17:56:20.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.926+0000 7facd10e5700 1 -- 172.21.15.40:0/4035658691 learned_addr learned my addr 172.21.15.40:0/4035658691 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:20.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.926+0000 7facd10e5700 1 -- 172.21.15.40:0/4035658691 >> v1:172.21.15.43:6789/0 conn(0x7facc40509b0 legacy=0x7facc40e6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.926+0000 7facd10e5700 1 -- 172.21.15.40:0/4035658691 >> v1:172.21.15.40:6789/0 conn(0x7facc40d6750 legacy=0x7facc40e0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.926+0000 7facd08e4700 1 -- 172.21.15.40:0/4035658691 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601565563 0 0) 0x7facc40dc980 con 0x7facc40d6750 2024-09-13T17:56:20.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.926+0000 7facd10e5700 1 -- 172.21.15.40:0/4035658691 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc40dcf40 con 0x7facc40dc4d0 2024-09-13T17:56:20.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.926+0000 7facd10e5700 1 --2- 172.21.15.40:0/4035658691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc40dc4d0 0x7facc4044be0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7facc8009910 tx=0x7facc8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57863c1efd929e56 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:20.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.926+0000 7facd08e4700 1 -- 172.21.15.40:0/4035658691 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc800ba60 con 0x7facc40dc4d0 2024-09-13T17:56:20.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.926+0000 7facd08e4700 1 -- 172.21.15.40:0/4035658691 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc800bbc0 con 0x7facc40dc4d0 2024-09-13T17:56:20.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.926+0000 7facd08e4700 1 -- 172.21.15.40:0/4035658691 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc8009e60 con 0x7facc40dc4d0 2024-09-13T17:56:20.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd28e8700 1 -- 172.21.15.40:0/4035658691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc40dc4d0 msgr2=0x7facc4044be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd28e8700 1 --2- 172.21.15.40:0/4035658691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc40dc4d0 0x7facc4044be0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7facc8009910 tx=0x7facc8009c20 comp rx=0 tx=0).stop 2024-09-13T17:56:20.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd28e8700 1 -- 172.21.15.40:0/4035658691 shutdown_connections 2024-09-13T17:56:20.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd28e8700 1 --2- 172.21.15.40:0/4035658691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc40dc4d0 0x7facc4044be0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd28e8700 1 -- 172.21.15.40:0/4035658691 >> 172.21.15.40:0/4035658691 conn(0x7facc404fd00 msgr2=0x7facc4050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:20.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd28e8700 1 -- 172.21.15.40:0/4035658691 shutdown_connections 2024-09-13T17:56:20.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd28e8700 1 -- 172.21.15.40:0/4035658691 wait complete. 2024-09-13T17:56:20.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd28e8700 1 Processor -- start 2024-09-13T17:56:20.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd28e8700 1 -- start start 2024-09-13T17:56:20.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd28e8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc40509b0 0x7facc4034590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd28e8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc40d6750 0x7facc404e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd28e8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7facc40dc4d0 0x7facc404ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd28e8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc404f4b0 con 0x7facc40d6750 2024-09-13T17:56:20.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd28e8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc404de20 con 0x7facc40509b0 2024-09-13T17:56:20.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd28e8700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc40c9150 con 0x7facc40dc4d0 2024-09-13T17:56:20.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd10e5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc40d6750 0x7facc404e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd10e5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc40d6750 0x7facc404e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40294/0 (socket says 172.21.15.40:40294) 2024-09-13T17:56:20.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd10e5700 1 -- 172.21.15.40:0/4232755550 learned_addr learned my addr 172.21.15.40:0/4232755550 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:20.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd18e6700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc40509b0 0x7facc4034590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd20e7700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7facc40dc4d0 0x7facc404ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd10e5700 1 -- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7facc40dc4d0 msgr2=0x7facc404ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd10e5700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7facc40dc4d0 0x7facc404ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd10e5700 1 -- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc40509b0 msgr2=0x7facc4034590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd10e5700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc40509b0 0x7facc4034590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd10e5700 1 -- 172.21.15.40:0/4232755550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc8009570 con 0x7facc40d6750 2024-09-13T17:56:20.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.930+0000 7facd18e6700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc40509b0 0x7facc4034590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:56:20.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facd10e5700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc40d6750 0x7facc404e7f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7facc8009c60 tx=0x7facc8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:20.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facc27fc700 1 -- 172.21.15.40:0/4232755550 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc80036c0 con 0x7facc40d6750 2024-09-13T17:56:20.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facc27fc700 1 -- 172.21.15.40:0/4232755550 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc801f070 con 0x7facc40d6750 2024-09-13T17:56:20.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facd28e8700 1 -- 172.21.15.40:0/4232755550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc40c9430 con 0x7facc40d6750 2024-09-13T17:56:20.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facc27fc700 1 -- 172.21.15.40:0/4232755550 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc8003a60 con 0x7facc40d6750 2024-09-13T17:56:20.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facd28e8700 1 -- 172.21.15.40:0/4232755550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc404f9c0 con 0x7facc40d6750 2024-09-13T17:56:20.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facd28e8700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc40cc1e0 0x7facc40cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facd28e8700 1 -- 172.21.15.40:0/4232755550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc40084b0 con 0x7facc40cc1e0 2024-09-13T17:56:20.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facd18e6700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc40cc1e0 0x7facc40cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facc27fc700 1 -- 172.21.15.40:0/4232755550 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7facc8003bc0 con 0x7facc40d6750 2024-09-13T17:56:20.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facd18e6700 1 -- 172.21.15.40:0/4232755550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7facc40dfc20 con 0x7facc40cc1e0 2024-09-13T17:56:20.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facd18e6700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc40cc1e0 0x7facc40cc5c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7facbc0011e0 tx=0x7facbc00ea90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:20.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facc27fc700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7facb0066730 0x7facb0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facc27fc700 1 -- 172.21.15.40:0/4232755550 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7facc8021070 con 0x7facc40d6750 2024-09-13T17:56:20.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facd20e7700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7facb0066730 0x7facb0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facc27fc700 1 -- 172.21.15.40:0/4232755550 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc0052e0 con 0x7facc40cc1e0 2024-09-13T17:56:20.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facd20e7700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7facb0066730 0x7facb0068bf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faccc063cf0 tx=0x7faccc065ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:20.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facc27fc700 1 -- 172.21.15.40:0/4232755550 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7facbc005440 con 0x7facc40cc1e0 2024-09-13T17:56:20.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facc27fc700 1 -- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc40cc1e0 msgr2=0x7facc40cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facc27fc700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc40cc1e0 0x7facc40cc5c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7facbc0011e0 tx=0x7facbc00ea90 comp rx=0 tx=0).stop 2024-09-13T17:56:20.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facc27fc700 1 -- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc40cc1e0 msgr2=0x7facc40cc5c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:56:20.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.934+0000 7facc27fc700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc40cc1e0 0x7facc40cc5c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.950+0000 7facd28e8700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc40c3440 0x7facc40c3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:20.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.950+0000 7facd28e8700 1 -- 172.21.15.40:0/4232755550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc404fb50 con 0x7facc40c3440 2024-09-13T17:56:20.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.950+0000 7facd18e6700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc40c3440 0x7facc40c3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:20.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.950+0000 7facd18e6700 1 -- 172.21.15.40:0/4232755550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7facc40cbfa0 con 0x7facc40c3440 2024-09-13T17:56:20.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.950+0000 7facd18e6700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc40c3440 0x7facc40c3850 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7facbc016000 tx=0x7facbc001080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:20.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.950+0000 7facc27fc700 1 -- 172.21.15.40:0/4232755550 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc021070 con 0x7facc40c3440 2024-09-13T17:56:20.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.954+0000 7facc27fc700 1 -- 172.21.15.40:0/4232755550 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7facbc0038b0 con 0x7facc40c3440 2024-09-13T17:56:20.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.954+0000 7facc27fc700 1 -- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc40c3440 msgr2=0x7facc40c3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.954+0000 7facc27fc700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc40c3440 0x7facc40c3850 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7facbc016000 tx=0x7facbc001080 comp rx=0 tx=0).stop 2024-09-13T17:56:20.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.954+0000 7facc27fc700 1 -- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc40c3440 msgr2=0x7facc40c3850 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:56:20.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.954+0000 7facc27fc700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc40c3440 0x7facc40c3850 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.962 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:56:20.962 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:56:20.962 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:56:20.962 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:56:20.962 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 46, 2024-09-13T17:56:20.962 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:56:20.962 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:56:20.962 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:56:20.962 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:56:20.963 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:20.963 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 13, 2024-09-13T17:56:20.963 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:56:20.963 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:56:20.963 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:56:20.963 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:20.963 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:20.963 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:20.963 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:20.963 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:20.963 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:20.963 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:20.963 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:20.963 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:20.963 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:20.964 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:20.964 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:56:20.964 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:56:20.964 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:20.964 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:20.964 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:20.964 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:20.964 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:20.964 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:20.964 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:20.964 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:20.965 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:20.965 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:20.965 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:20.965 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.965 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:56:20.965 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:56:20.965 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:56:20.965 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:56:20.965 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:56:20.965 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:56:20.965 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:56:20.965 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:56:20.965 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:56:20.966 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:56:20.966 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:56:20.966 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:56:20.966 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:56:20.966 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:56:20.966 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:56:20.966 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:56:20.966 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:56:20.966 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:20.966 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:20.966 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:20.966 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:20.966 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:20.967 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:20.967 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:20.967 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:20.967 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:20.967 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:20.967 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:20.967 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:56:20.967 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.967 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:56:20.967 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.967 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:56:20.968 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:56:20.968 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:20.968 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:20.968 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.968 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:20.968 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:56:20.968 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:20.968 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.968 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.968 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:20.968 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:56:20.968 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:20.968 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:20.968 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:20.969 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.969 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:56:20.969 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:56:20.969 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:20.969 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:20.969 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:20.969 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.969 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.969 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:56:20.969 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:56:20.969 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:20.969 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:20.970 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.970 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:20.970 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:56:20.970 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:20.970 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.970 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.970 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:20.970 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:56:20.970 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:20.970 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:20.970 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:20.970 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.970 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:56:20.971 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:56:20.971 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:20.971 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:20.971 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:20.971 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.971 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.971 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:56:20.971 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:56:20.971 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:20.971 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:20.971 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.971 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:20.971 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:56:20.971 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:20.972 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.972 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.972 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:20.972 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:56:20.972 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:20.972 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:20.972 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:20.973 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.973 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:56:20.973 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:56:20.973 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:20.973 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:20.973 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:20.974 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:20.974 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:20.974 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.974 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:56:20.974 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:56:20.974 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.975 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:20.975 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:20.975 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:56:20.975 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:20.975 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:20.975 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:56:20.976 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.976 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:20.976 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:20.976 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:56:20.976 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:20.976 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:20.977 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:56:20.977 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.977 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:20.977 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:20.977 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:56:20.978 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:20.978 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:20.978 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:56:20.978 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.978 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:20.978 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:20.979 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:56:20.979 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:20.979 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:20.979 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:56:20.979 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:20.979 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:20.980 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:20.980 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:56:20.980 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:20.980 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:20.980 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:20.980 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:56:20.981 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:56:20.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.954+0000 7facd28e8700 1 -- 172.21.15.40:0/4232755550 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7facb0066730 msgr2=0x7facb0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.954+0000 7facd28e8700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7facb0066730 0x7facb0068bf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faccc063cf0 tx=0x7faccc065ea0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.954+0000 7facd28e8700 1 -- 172.21.15.40:0/4232755550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc40d6750 msgr2=0x7facc404e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:20.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.954+0000 7facd20e7700 1 -- 172.21.15.40:0/4232755550 reap_dead start 2024-09-13T17:56:20.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.954+0000 7facd28e8700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc40d6750 0x7facc404e7f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7facc8009c60 tx=0x7facc8004770 comp rx=0 tx=0).stop 2024-09-13T17:56:20.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.954+0000 7facd28e8700 1 -- 172.21.15.40:0/4232755550 shutdown_connections 2024-09-13T17:56:20.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.954+0000 7facd28e8700 1 --2- 172.21.15.40:0/4232755550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7facc40d6750 0x7facc404e7f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:20.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.954+0000 7facd28e8700 1 -- 172.21.15.40:0/4232755550 >> 172.21.15.40:0/4232755550 conn(0x7facc404fd00 msgr2=0x7facc4005b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:20.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.954+0000 7facd28e8700 1 -- 172.21.15.40:0/4232755550 shutdown_connections 2024-09-13T17:56:20.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:20.954+0000 7facd28e8700 1 -- 172.21.15.40:0/4232755550 wait complete. 2024-09-13T17:56:20.985 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:56:20.985 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T17:56:20.985 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:56:21.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.098+0000 7fdbe7fff700 1 Processor -- start 2024-09-13T17:56:21.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.098+0000 7fdbe7fff700 1 -- start start 2024-09-13T17:56:21.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe7fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe8066220 0x7fdbe80cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:21.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe7fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe81578b0 con 0x7fdbe8151050 2024-09-13T17:56:21.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe7fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe806b290 con 0x7fdbe8157d40 2024-09-13T17:56:21.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe7fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe8157090 con 0x7fdbe8066220 2024-09-13T17:56:21.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe6ffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdbe8151050 0x7fdbe80c2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50450/0 (socket says 172.21.15.40:50450) 2024-09-13T17:56:21.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe6ffd700 1 -- 172.21.15.40:0/2443497217 learned_addr learned my addr 172.21.15.40:0/2443497217 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:56:21.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe77fe700 1 --2- 172.21.15.40:0/2443497217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe8066220 0x7fdbe80cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:21.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe5ffb700 1 -- 172.21.15.40:0/2443497217 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 823298322 0 0) 0x7fdbe81578b0 con 0x7fdbe8151050 2024-09-13T17:56:21.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe5ffb700 1 -- 172.21.15.40:0/2443497217 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbd00035d0 con 0x7fdbe8151050 2024-09-13T17:56:21.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe5ffb700 1 -- 172.21.15.40:0/2443497217 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929252771 0 0) 0x7fdbe806b290 con 0x7fdbe8157d40 2024-09-13T17:56:21.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe5ffb700 1 -- 172.21.15.40:0/2443497217 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbe81578b0 con 0x7fdbe8157d40 2024-09-13T17:56:21.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe77fe700 1 -- 172.21.15.40:0/2443497217 >> v1:172.21.15.43:6789/0 conn(0x7fdbe8157d40 legacy=0x7fdbe80cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:21.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe77fe700 1 -- 172.21.15.40:0/2443497217 >> v1:172.21.15.40:6789/0 conn(0x7fdbe8151050 legacy=0x7fdbe80c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:21.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe77fe700 1 -- 172.21.15.40:0/2443497217 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe81572a0 con 0x7fdbe8066220 2024-09-13T17:56:21.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe77fe700 1 --2- 172.21.15.40:0/2443497217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe8066220 0x7fdbe80cb9b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdbe81571d0 tx=0x7fdbe000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e345164d0c659484 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:21.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe5ffb700 1 -- 172.21.15.40:0/2443497217 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0016070 con 0x7fdbe8066220 2024-09-13T17:56:21.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe5ffb700 1 -- 172.21.15.40:0/2443497217 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe000bb20 con 0x7fdbe8066220 2024-09-13T17:56:21.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe5ffb700 1 -- 172.21.15.40:0/2443497217 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe000e070 con 0x7fdbe8066220 2024-09-13T17:56:21.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe7fff700 1 -- 172.21.15.40:0/2443497217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe8066220 msgr2=0x7fdbe80cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:21.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe7fff700 1 --2- 172.21.15.40:0/2443497217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe8066220 0x7fdbe80cb9b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdbe81571d0 tx=0x7fdbe000b330 comp rx=0 tx=0).stop 2024-09-13T17:56:21.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe7fff700 1 -- 172.21.15.40:0/2443497217 shutdown_connections 2024-09-13T17:56:21.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe7fff700 1 --2- 172.21.15.40:0/2443497217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe8066220 0x7fdbe80cb9b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:21.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe7fff700 1 -- 172.21.15.40:0/2443497217 >> 172.21.15.40:0/2443497217 conn(0x7fdbe806a390 msgr2=0x7fdbe806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:21.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe7fff700 1 -- 172.21.15.40:0/2443497217 shutdown_connections 2024-09-13T17:56:21.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.102+0000 7fdbe7fff700 1 -- 172.21.15.40:0/2443497217 wait complete. 2024-09-13T17:56:21.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe7fff700 1 Processor -- start 2024-09-13T17:56:21.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe7fff700 1 -- start start 2024-09-13T17:56:21.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe7fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe8066220 0x7fdbe8149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:21.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe8151050 0x7fdbe80c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:21.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbe8157d40 0x7fdbe81ee200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:21.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe7fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe8148d00 con 0x7fdbe8157d40 2024-09-13T17:56:21.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe7fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe8149190 con 0x7fdbe8151050 2024-09-13T17:56:21.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe7fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe81480f0 con 0x7fdbe8066220 2024-09-13T17:56:21.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe67fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe8151050 0x7fdbe80c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:21.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbe8157d40 0x7fdbe81ee200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:21.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe6ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe8066220 0x7fdbe8149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:21.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe67fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe8151050 0x7fdbe80c7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:36504/0 (socket says 172.21.15.40:36504) 2024-09-13T17:56:21.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe67fc700 1 -- 172.21.15.40:0/2959015147 learned_addr learned my addr 172.21.15.40:0/2959015147 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:21.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe77fe700 1 -- 172.21.15.40:0/2959015147 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe8066220 msgr2=0x7fdbe8149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:21.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe77fe700 1 --2- 172.21.15.40:0/2959015147 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe8066220 0x7fdbe8149dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:21.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe77fe700 1 -- 172.21.15.40:0/2959015147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe8151050 msgr2=0x7fdbe80c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:21.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe77fe700 1 --2- 172.21.15.40:0/2959015147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbe8151050 0x7fdbe80c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:21.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe77fe700 1 -- 172.21.15.40:0/2959015147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe0009a70 con 0x7fdbe8157d40 2024-09-13T17:56:21.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe77fe700 1 --2- 172.21.15.40:0/2959015147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbe8157d40 0x7fdbe81ee200 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0015000 tx=0x7fdbe0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:21.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbcffff700 1 -- 172.21.15.40:0/2959015147 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0016040 con 0x7fdbe8157d40 2024-09-13T17:56:21.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbcffff700 1 -- 172.21.15.40:0/2959015147 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe0004860 con 0x7fdbe8157d40 2024-09-13T17:56:21.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe7fff700 1 -- 172.21.15.40:0/2959015147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe81483d0 con 0x7fdbe8157d40 2024-09-13T17:56:21.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbcffff700 1 -- 172.21.15.40:0/2959015147 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe001b680 con 0x7fdbe8157d40 2024-09-13T17:56:21.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.106+0000 7fdbe7fff700 1 -- 172.21.15.40:0/2959015147 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe8148990 con 0x7fdbe8157d40 2024-09-13T17:56:21.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.110+0000 7fdbe7fff700 1 --2- 172.21.15.40:0/2959015147 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe8147a10 0x7fdbe81f0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:21.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.110+0000 7fdbe7fff700 1 -- 172.21.15.40:0/2959015147 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe8067600 con 0x7fdbe8147a10 2024-09-13T17:56:21.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.110+0000 7fdbe6ffd700 1 --2- 172.21.15.40:0/2959015147 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe8147a10 0x7fdbe81f0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:21.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.110+0000 7fdbe6ffd700 1 -- 172.21.15.40:0/2959015147 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdbe80c6350 con 0x7fdbe8147a10 2024-09-13T17:56:21.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.110+0000 7fdbcffff700 1 -- 172.21.15.40:0/2959015147 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdbe001b7e0 con 0x7fdbe8157d40 2024-09-13T17:56:21.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.110+0000 7fdbe6ffd700 1 --2- 172.21.15.40:0/2959015147 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe8147a10 0x7fdbe81f0300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8007fd0 tx=0x7fdbd8001970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:21.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.110+0000 7fdbcffff700 1 --2- 172.21.15.40:0/2959015147 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdbd4066730 0x7fdbd4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:21.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.110+0000 7fdbcffff700 1 -- 172.21.15.40:0/2959015147 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdbe0085720 con 0x7fdbe8157d40 2024-09-13T17:56:21.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.110+0000 7fdbe67fc700 1 --2- 172.21.15.40:0/2959015147 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdbd4066730 0x7fdbd4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:21.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.110+0000 7fdbcffff700 1 -- 172.21.15.40:0/2959015147 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8005b70 con 0x7fdbe8147a10 2024-09-13T17:56:21.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.110+0000 7fdbcffff700 1 -- 172.21.15.40:0/2959015147 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fdbd8005cd0 con 0x7fdbe8147a10 2024-09-13T17:56:21.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.110+0000 7fdbcffff700 1 -- 172.21.15.40:0/2959015147 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe8147a10 msgr2=0x7fdbe81f0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:21.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.110+0000 7fdbcffff700 1 --2- 172.21.15.40:0/2959015147 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe8147a10 0x7fdbe81f0300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8007fd0 tx=0x7fdbd8001970 comp rx=0 tx=0).stop 2024-09-13T17:56:21.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.110+0000 7fdbcffff700 1 -- 172.21.15.40:0/2959015147 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe8147a10 msgr2=0x7fdbe81f0300 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:56:21.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.110+0000 7fdbcffff700 1 --2- 172.21.15.40:0/2959015147 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe8147a10 0x7fdbe81f0300 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:21.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.110+0000 7fdbe67fc700 1 --2- 172.21.15.40:0/2959015147 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdbd4066730 0x7fdbd4068bf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdbe81e2920 tx=0x7fdbdc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:21.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.122+0000 7fdbe7fff700 1 --2- 172.21.15.40:0/2959015147 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe814a820 0x7fdbe81f0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:21.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.122+0000 7fdbe7fff700 1 -- 172.21.15.40:0/2959015147 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe8149650 con 0x7fdbe814a820 2024-09-13T17:56:21.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.122+0000 7fdbe6ffd700 1 --2- 172.21.15.40:0/2959015147 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe814a820 0x7fdbe81f0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:21.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.122+0000 7fdbe6ffd700 1 -- 172.21.15.40:0/2959015147 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdbe80c6630 con 0x7fdbe814a820 2024-09-13T17:56:21.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.122+0000 7fdbe6ffd700 1 --2- 172.21.15.40:0/2959015147 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe814a820 0x7fdbe81f0840 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8016000 tx=0x7fdbd8001150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:21.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.122+0000 7fdbcffff700 1 -- 172.21.15.40:0/2959015147 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8022070 con 0x7fdbe814a820 2024-09-13T17:56:21.132 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:56:21.132 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:56:21.132 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:56:21.132 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:56:21.132 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 46, 2024-09-13T17:56:21.132 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:56:21.132 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:56:21.133 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:56:21.133 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:56:21.133 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:21.133 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 14, 2024-09-13T17:56:21.133 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:56:21.133 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:56:21.133 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:56:21.133 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:21.133 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:21.133 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:21.133 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:21.133 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:21.133 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:21.133 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:21.134 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:21.134 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:21.134 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:21.134 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:21.134 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:56:21.134 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:56:21.134 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:21.134 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:21.134 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:21.134 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:21.134 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:21.134 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:21.134 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:21.135 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:21.135 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:21.135 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:21.135 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:21.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:21.135 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:56:21.135 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:56:21.135 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:56:21.135 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:56:21.135 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:56:21.135 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:56:21.135 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:56:21.135 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:56:21.136 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:56:21.136 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:56:21.136 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:56:21.136 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:56:21.136 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:56:21.136 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:56:21.136 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:56:21.136 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:56:21.136 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:56:21.136 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:21.136 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:21.136 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:21.136 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:21.136 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:21.137 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:21.137 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:21.137 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:21.137 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:21.137 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:21.137 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:21.137 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:56:21.137 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:21.137 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:56:21.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:21.137 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:56:21.137 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:56:21.137 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:21.137 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:21.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:21.138 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:21.138 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:56:21.138 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:21.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:21.138 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:21.138 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:21.138 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:56:21.138 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:21.138 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:21.138 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:21.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:21.138 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:56:21.138 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:56:21.139 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:21.139 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:21.139 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:21.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:21.139 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:21.139 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:56:21.139 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:56:21.139 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:21.139 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:21.139 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:21.139 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:21.139 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:56:21.139 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:21.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:21.140 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:21.140 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:21.140 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:56:21.140 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:21.140 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:21.140 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:21.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:21.140 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:56:21.140 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:56:21.140 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:21.140 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:21.140 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:21.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:21.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:21.141 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:56:21.141 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:56:21.141 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:21.141 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:21.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:21.141 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:21.141 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:56:21.141 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:21.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:21.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:21.141 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:21.141 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:56:21.142 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:21.142 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:21.142 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:21.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:21.142 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:56:21.142 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:56:21.142 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:21.142 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:21.142 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:21.142 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:21.142 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:21.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:21.142 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:56:21.143 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:56:21.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:21.143 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:21.143 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:21.143 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:56:21.143 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:21.143 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:21.143 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:56:21.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:21.143 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:21.143 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:21.143 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:56:21.143 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:21.143 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:21.144 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:56:21.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:21.144 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:21.144 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:21.144 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T17:56:21.144 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:21.144 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:21.144 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:56:21.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:21.144 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:21.144 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:21.144 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:56:21.144 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:21.145 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:21.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:21.145 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:56:21.145 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:56:21.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.122+0000 7fdbcffff700 1 -- 172.21.15.40:0/2959015147 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4782 (secure 0 0 0) 0x7fdbd80066b0 con 0x7fdbe814a820 2024-09-13T17:56:21.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.122+0000 7fdbcffff700 1 -- 172.21.15.40:0/2959015147 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe814a820 msgr2=0x7fdbe81f0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:21.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.122+0000 7fdbcffff700 1 --2- 172.21.15.40:0/2959015147 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe814a820 0x7fdbe81f0840 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8016000 tx=0x7fdbd8001150 comp rx=0 tx=0).stop 2024-09-13T17:56:21.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.122+0000 7fdbcffff700 1 -- 172.21.15.40:0/2959015147 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe814a820 msgr2=0x7fdbe81f0840 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:56:21.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.122+0000 7fdbcffff700 1 --2- 172.21.15.40:0/2959015147 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdbe814a820 0x7fdbe81f0840 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:21.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.122+0000 7fdbe7fff700 1 -- 172.21.15.40:0/2959015147 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdbd4066730 msgr2=0x7fdbd4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:21.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.122+0000 7fdbe7fff700 1 --2- 172.21.15.40:0/2959015147 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdbd4066730 0x7fdbd4068bf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdbe81e2920 tx=0x7fdbdc009040 comp rx=0 tx=0).stop 2024-09-13T17:56:21.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.126+0000 7fdbe77fe700 1 -- 172.21.15.40:0/2959015147 reap_dead start 2024-09-13T17:56:21.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.126+0000 7fdbe7fff700 1 -- 172.21.15.40:0/2959015147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbe8157d40 msgr2=0x7fdbe81ee200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:21.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.126+0000 7fdbe7fff700 1 --2- 172.21.15.40:0/2959015147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbe8157d40 0x7fdbe81ee200 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0015000 tx=0x7fdbe0004af0 comp rx=0 tx=0).stop 2024-09-13T17:56:21.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.126+0000 7fdbe7fff700 1 -- 172.21.15.40:0/2959015147 shutdown_connections 2024-09-13T17:56:21.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.126+0000 7fdbe7fff700 1 --2- 172.21.15.40:0/2959015147 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbe8157d40 0x7fdbe81ee200 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:21.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.126+0000 7fdbe7fff700 1 -- 172.21.15.40:0/2959015147 >> 172.21.15.40:0/2959015147 conn(0x7fdbe806a390 msgr2=0x7fdbe8150220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:21.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.126+0000 7fdbe7fff700 1 -- 172.21.15.40:0/2959015147 shutdown_connections 2024-09-13T17:56:21.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.126+0000 7fdbe7fff700 1 -- 172.21.15.40:0/2959015147 wait complete. 2024-09-13T17:56:21.148 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:56:21.148 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-13T17:56:21.148 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T17:56:21.148 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-13T17:56:21.148 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-13T17:56:21.148 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T17:56:21.149 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:56:21.200 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T17:56:21.200 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T17:56:21.201 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:56:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.310+0000 7f711d0c7700 1 Processor -- start 2024-09-13T17:56:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.310+0000 7f711d0c7700 1 -- start start 2024-09-13T17:56:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.310+0000 7f711d0c7700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f711815e1a0 0x7f711806a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.310+0000 7f711d0c7700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7118071690 con 0x7f711806a8d0 2024-09-13T17:56:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.310+0000 7f711d0c7700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7118154620 con 0x7f7118154240 2024-09-13T17:56:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.310+0000 7f711d0c7700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71180721e0 con 0x7f711815e1a0 2024-09-13T17:56:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.310+0000 7f71177fe700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7f711815e1a0 msgr2=0x7f711806a390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:56:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.310+0000 7f7117fff700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7f7118154240 legacy=0x7f711815ec60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T17:56:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.310+0000 7f71177fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f711815e1a0 0x7f711806a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.310+0000 7f711c8c6700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f711806a8d0 0x7f71181680f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50466/0 (socket says 172.21.15.40:50466) 2024-09-13T17:56:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f711c8c6700 1 -- 172.21.15.40:0/2011220503 learned_addr learned my addr 172.21.15.40:0/2011220503 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:56:21.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f7116ffd700 1 -- 172.21.15.40:0/2011220503 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 677295507 0 0) 0x7f7118071690 con 0x7f711806a8d0 2024-09-13T17:56:21.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f7116ffd700 1 -- 172.21.15.40:0/2011220503 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71040035d0 con 0x7f711806a8d0 2024-09-13T17:56:21.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f7116ffd700 1 -- 172.21.15.40:0/2011220503 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2038700889 0 0) 0x7f71040035d0 con 0x7f711806a8d0 2024-09-13T17:56:21.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f7116ffd700 1 -- 172.21.15.40:0/2011220503 >> v2:172.21.15.43:3300/0 conn(0x7f711815e1a0 msgr2=0x7f711806a390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:56:21.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f7116ffd700 1 --2- 172.21.15.40:0/2011220503 >> v2:172.21.15.43:3300/0 conn(0x7f711815e1a0 0x7f711806a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:21.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f7116ffd700 1 -- 172.21.15.40:0/2011220503 >> v1:172.21.15.43:6789/0 conn(0x7f7118154240 legacy=0x7f711815ec60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:56:21.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f7116ffd700 1 -- 172.21.15.40:0/2011220503 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7118071b20 con 0x7f711806a8d0 2024-09-13T17:56:21.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f7116ffd700 1 -- 172.21.15.40:0/2011220503 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f7110001670 con 0x7f711806a8d0 2024-09-13T17:56:21.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f7116ffd700 1 -- 172.21.15.40:0/2011220503 >> v1:172.21.15.40:6789/0 conn(0x7f711806a8d0 legacy=0x7f71181680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f7116ffd700 1 --2- 172.21.15.40:0/2011220503 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7104003a20 0x7f7104003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f7116ffd700 1 --2- 172.21.15.40:0/2011220503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7104004870 0x7f7104004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f7116ffd700 1 --2- 172.21.15.40:0/2011220503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71040052a0 0x7f710400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f7116ffd700 1 -- 172.21.15.40:0/2011220503 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71180721e0 con 0x7f7104004870 2024-09-13T17:56:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f7116ffd700 1 -- 172.21.15.40:0/2011220503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710400c1f0 con 0x7f71040052a0 2024-09-13T17:56:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f7116ffd700 1 -- 172.21.15.40:0/2011220503 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710400c3b0 con 0x7f7104003a20 2024-09-13T17:56:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f711c8c6700 1 -- 172.21.15.40:0/2011220503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71040052a0 msgr2=0x7f710400ba30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:56:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f711c8c6700 1 --2- 172.21.15.40:0/2011220503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71040052a0 0x7f710400ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f71177fe700 1 --2- 172.21.15.40:0/2011220503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7104004870 0x7f7104004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f7117fff700 1 --2- 172.21.15.40:0/2011220503 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7104003a20 0x7f7104003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f71177fe700 1 -- 172.21.15.40:0/2011220503 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7104003a20 msgr2=0x7f7104003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:56:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f71177fe700 1 --2- 172.21.15.40:0/2011220503 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7104003a20 0x7f7104003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f71177fe700 1 -- 172.21.15.40:0/2011220503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71040052a0 msgr2=0x7f710400ba30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:56:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f71177fe700 1 --2- 172.21.15.40:0/2011220503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71040052a0 0x7f710400ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f71177fe700 1 -- 172.21.15.40:0/2011220503 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71040035d0 con 0x7f7104004870 2024-09-13T17:56:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f711c8c6700 1 -- 172.21.15.40:0/2011220503 reap_dead start 2024-09-13T17:56:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f7117fff700 1 --2- 172.21.15.40:0/2011220503 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7104003a20 0x7f7104003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:56:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f71177fe700 1 --2- 172.21.15.40:0/2011220503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7104004870 0x7f7104004cf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7100009dc0 tx=0x7f71040058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89878f461fbe54f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f7116ffd700 1 -- 172.21.15.40:0/2011220503 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100009860 con 0x7f7104004870 2024-09-13T17:56:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f7116ffd700 1 -- 172.21.15.40:0/2011220503 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710000bd60 con 0x7f7104004870 2024-09-13T17:56:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f7116ffd700 1 -- 172.21.15.40:0/2011220503 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7100015640 con 0x7f7104004870 2024-09-13T17:56:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f711d0c7700 1 -- 172.21.15.40:0/2011220503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7104004870 msgr2=0x7f7104004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f711d0c7700 1 --2- 172.21.15.40:0/2011220503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7104004870 0x7f7104004cf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7100009dc0 tx=0x7f71040058a0 comp rx=0 tx=0).stop 2024-09-13T17:56:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f711d0c7700 1 -- 172.21.15.40:0/2011220503 shutdown_connections 2024-09-13T17:56:21.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f711d0c7700 1 --2- 172.21.15.40:0/2011220503 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7104004870 0x7f7104004cf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:21.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f711d0c7700 1 -- 172.21.15.40:0/2011220503 >> 172.21.15.40:0/2011220503 conn(0x7f71180cabc0 msgr2=0x7f71180cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:21.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f711d0c7700 1 -- 172.21.15.40:0/2011220503 shutdown_connections 2024-09-13T17:56:21.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.314+0000 7f711d0c7700 1 -- 172.21.15.40:0/2011220503 wait complete. 2024-09-13T17:56:21.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.318+0000 7f711d0c7700 1 Processor -- start 2024-09-13T17:56:21.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.318+0000 7f711d0c7700 1 -- start start 2024-09-13T17:56:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.318+0000 7f711d0c7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7104004870 0x7f711815a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.318+0000 7f711d0c7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181f6fd0 0x7f711815c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.318+0000 7f711d0c7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71181f44c0 0x7f71181f4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.318+0000 7f711d0c7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71180c7b60 con 0x7f71181f44c0 2024-09-13T17:56:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.318+0000 7f711d0c7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71181e4b40 con 0x7f7104004870 2024-09-13T17:56:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.318+0000 7f711d0c7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71181e3850 con 0x7f71181f6fd0 2024-09-13T17:56:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.318+0000 7f7117fff700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7104004870 msgr2=0x7f711815a920 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:56:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.318+0000 7f7117fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7104004870 0x7f711815a920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.318+0000 7f711c8c6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71181f44c0 0x7f71181f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.318+0000 7f71177fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181f6fd0 0x7f711815c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.318+0000 7f711c8c6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71181f44c0 0x7f71181f4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40368/0 (socket says 172.21.15.40:40368) 2024-09-13T17:56:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.318+0000 7f711c8c6700 1 -- 172.21.15.40:0/1901241441 learned_addr learned my addr 172.21.15.40:0/1901241441 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.322+0000 7f711c8c6700 1 -- 172.21.15.40:0/1901241441 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181f6fd0 msgr2=0x7f711815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.322+0000 7f711c8c6700 1 --2- 172.21.15.40:0/1901241441 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181f6fd0 0x7f711815c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.322+0000 7f711c8c6700 1 -- 172.21.15.40:0/1901241441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7104004870 msgr2=0x7f711815a920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:56:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.322+0000 7f711c8c6700 1 --2- 172.21.15.40:0/1901241441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7104004870 0x7f711815a920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.322+0000 7f711c8c6700 1 -- 172.21.15.40:0/1901241441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71181e46b0 con 0x7f71181f44c0 2024-09-13T17:56:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.322+0000 7f711c8c6700 1 --2- 172.21.15.40:0/1901241441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71181f44c0 0x7f71181f4940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f711000c700 tx=0x7f711000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.322+0000 7f7114ff9700 1 -- 172.21.15.40:0/1901241441 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7110004ca0 con 0x7f71181f44c0 2024-09-13T17:56:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.322+0000 7f7114ff9700 1 -- 172.21.15.40:0/1901241441 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7110004e00 con 0x7f71181f44c0 2024-09-13T17:56:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.322+0000 7f711d0c7700 1 -- 172.21.15.40:0/1901241441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71181e3aa0 con 0x7f71181f44c0 2024-09-13T17:56:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.322+0000 7f7114ff9700 1 -- 172.21.15.40:0/1901241441 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7110007270 con 0x7f71181f44c0 2024-09-13T17:56:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.322+0000 7f711d0c7700 1 -- 172.21.15.40:0/1901241441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71181e4060 con 0x7f71181f44c0 2024-09-13T17:56:21.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.322+0000 7f711d0c7700 1 -- 172.21.15.40:0/1901241441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7118071320 con 0x7f71181f44c0 2024-09-13T17:56:21.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.322+0000 7f7114ff9700 1 -- 172.21.15.40:0/1901241441 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f71100073d0 con 0x7f71181f44c0 2024-09-13T17:56:21.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.322+0000 7f7114ff9700 1 --2- 172.21.15.40:0/1901241441 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70f8066710 0x7f70f8068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:21.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.322+0000 7f7114ff9700 1 -- 172.21.15.40:0/1901241441 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f71100838d0 con 0x7f71181f44c0 2024-09-13T17:56:21.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.322+0000 7f7117fff700 1 --2- 172.21.15.40:0/1901241441 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70f8066710 0x7f70f8068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:21.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.326+0000 7f7117fff700 1 --2- 172.21.15.40:0/1901241441 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70f8066710 0x7f70f8068bd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f711804f940 tx=0x7f7108008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:21.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.330+0000 7f7114ff9700 1 -- 172.21.15.40:0/1901241441 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7110054280 con 0x7f71181f44c0 2024-09-13T17:56:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.590+0000 7f711d0c7700 1 -- 172.21.15.40:0/1901241441 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f711806b370 con 0x7f71181f44c0 2024-09-13T17:56:21.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.590+0000 7f7114ff9700 1 -- 172.21.15.40:0/1901241441 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f7110081e60 con 0x7f71181f44c0 2024-09-13T17:56:21.598 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:56:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.590+0000 7f711d0c7700 1 -- 172.21.15.40:0/1901241441 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70f8066710 msgr2=0x7f70f8068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.590+0000 7f711d0c7700 1 --2- 172.21.15.40:0/1901241441 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70f8066710 0x7f70f8068bd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f711804f940 tx=0x7f7108008040 comp rx=0 tx=0).stop 2024-09-13T17:56:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.590+0000 7f711d0c7700 1 -- 172.21.15.40:0/1901241441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71181f44c0 msgr2=0x7f71181f4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.590+0000 7f711d0c7700 1 --2- 172.21.15.40:0/1901241441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71181f44c0 0x7f71181f4940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f711000c700 tx=0x7f711000ca10 comp rx=0 tx=0).stop 2024-09-13T17:56:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.590+0000 7f711d0c7700 1 -- 172.21.15.40:0/1901241441 shutdown_connections 2024-09-13T17:56:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.590+0000 7f711d0c7700 1 --2- 172.21.15.40:0/1901241441 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71181f44c0 0x7f71181f4940 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.590+0000 7f711d0c7700 1 --2- 172.21.15.40:0/1901241441 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70f8066710 0x7f70f8068bd0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.590+0000 7f711d0c7700 1 --2- 172.21.15.40:0/1901241441 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181f6fd0 0x7f711815c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.594+0000 7f711d0c7700 1 --2- 172.21.15.40:0/1901241441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7104004870 0x7f711815a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.594+0000 7f711d0c7700 1 -- 172.21.15.40:0/1901241441 >> 172.21.15.40:0/1901241441 conn(0x7f71180cabc0 msgr2=0x7f71180c40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:21.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.594+0000 7f711d0c7700 1 -- 172.21.15.40:0/1901241441 shutdown_connections 2024-09-13T17:56:21.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:21.594+0000 7f711d0c7700 1 -- 172.21.15.40:0/1901241441 wait complete. 2024-09-13T17:56:22.106 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-09-13T17:56:24.614 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:56:24.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcdb0a36700 1 Processor -- start 2024-09-13T17:56:24.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcdb0a36700 1 -- start start 2024-09-13T17:56:24.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcdb0a36700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcdac153800 0x7fcdac15c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:24.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcdb0a36700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcdac0c9d90 0x7fcdac0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:24.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcdb0a36700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdac153be0 con 0x7fcdac0ca190 2024-09-13T17:56:24.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcdb0a36700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdac15cf00 con 0x7fcdac0c9d90 2024-09-13T17:56:24.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcdb0a36700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdac0ca770 con 0x7fcdac153800 2024-09-13T17:56:24.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcdab59e700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7fcdac0ca190 legacy=0x7fcdac0c5360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T17:56:24.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcdaad9d700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7fcdac153800 msgr2=0x7fcdac15c9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:56:24.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcdaa59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcdac0c9d90 0x7fcdac0cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:24.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcdaad9d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcdac153800 0x7fcdac15c9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:24.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcdaa59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcdac0c9d90 0x7fcdac0cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58732/0 (socket says 172.21.15.40:58732) 2024-09-13T17:56:24.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcdaa59c700 1 -- 172.21.15.40:0/2121901428 learned_addr learned my addr 172.21.15.40:0/2121901428 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:24.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcdaa59c700 1 -- 172.21.15.40:0/2121901428 >> v2:172.21.15.43:3300/0 conn(0x7fcdac153800 msgr2=0x7fcdac15c9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:56:24.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcdaa59c700 1 --2- 172.21.15.40:0/2121901428 >> v2:172.21.15.43:3300/0 conn(0x7fcdac153800 0x7fcdac15c9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:24.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcdaa59c700 1 -- 172.21.15.40:0/2121901428 >> v1:172.21.15.43:6789/0 conn(0x7fcdac0ca190 legacy=0x7fcdac0c5360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:56:24.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcdaa59c700 1 -- 172.21.15.40:0/2121901428 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac15d100 con 0x7fcdac0c9d90 2024-09-13T17:56:24.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcdaa59c700 1 --2- 172.21.15.40:0/2121901428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac0c9d90 0x7fcdac0cee80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcdac08c5d0 tx=0x7fcda0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8638dda7e5df1e23 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:24.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcda9d9b700 1 -- 172.21.15.40:0/2121901428 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0004230 con 0x7fcdac0c9d90 2024-09-13T17:56:24.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcda9d9b700 1 -- 172.21.15.40:0/2121901428 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda0004390 con 0x7fcdac0c9d90 2024-09-13T17:56:24.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.734+0000 7fcda9d9b700 1 -- 172.21.15.40:0/2121901428 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda00036c0 con 0x7fcdac0c9d90 2024-09-13T17:56:24.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.738+0000 7fcdb0a36700 1 -- 172.21.15.40:0/2121901428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac0c9d90 msgr2=0x7fcdac0cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:24.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.738+0000 7fcdb0a36700 1 --2- 172.21.15.40:0/2121901428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac0c9d90 0x7fcdac0cee80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcdac08c5d0 tx=0x7fcda0009bf0 comp rx=0 tx=0).stop 2024-09-13T17:56:24.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.738+0000 7fcdb0a36700 1 -- 172.21.15.40:0/2121901428 shutdown_connections 2024-09-13T17:56:24.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.738+0000 7fcdb0a36700 1 --2- 172.21.15.40:0/2121901428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac0c9d90 0x7fcdac0cee80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:24.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.738+0000 7fcdb0a36700 1 --2- 172.21.15.40:0/2121901428 >> v2:172.21.15.43:3300/0 conn(0x7fcdac153800 0x7fcdac15c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:24.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.738+0000 7fcdb0a36700 1 -- 172.21.15.40:0/2121901428 >> 172.21.15.40:0/2121901428 conn(0x7fcdac066090 msgr2=0x7fcdac0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:24.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.738+0000 7fcdb0a36700 1 -- 172.21.15.40:0/2121901428 shutdown_connections 2024-09-13T17:56:24.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.738+0000 7fcdb0a36700 1 -- 172.21.15.40:0/2121901428 wait complete. 2024-09-13T17:56:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.738+0000 7fcdb0a36700 1 Processor -- start 2024-09-13T17:56:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.738+0000 7fcdb0a36700 1 -- start start 2024-09-13T17:56:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.738+0000 7fcdb0a36700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcdac0c9d90 0x7fcdac154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.738+0000 7fcdb0a36700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdac0ca190 0x7fcdac1497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.738+0000 7fcdb0a36700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac153800 0x7fcdac149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.738+0000 7fcdb0a36700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac154100 con 0x7fcdac153800 2024-09-13T17:56:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.738+0000 7fcdb0a36700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac14f230 con 0x7fcdac0ca190 2024-09-13T17:56:24.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.738+0000 7fcdb0a36700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac153e60 con 0x7fcdac0c9d90 2024-09-13T17:56:24.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.742+0000 7fcdaa59c700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdac0ca190 msgr2=0x7fcdac1497c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:56:24.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.742+0000 7fcdaa59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdac0ca190 0x7fcdac1497c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:24.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.742+0000 7fcdab59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac153800 0x7fcdac149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:24.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.742+0000 7fcdab59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac153800 0x7fcdac149d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58740/0 (socket says 172.21.15.40:58740) 2024-09-13T17:56:24.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.742+0000 7fcdab59e700 1 -- 172.21.15.40:0/3001657235 learned_addr learned my addr 172.21.15.40:0/3001657235 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:24.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.742+0000 7fcdaad9d700 1 --2- 172.21.15.40:0/3001657235 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcdac0c9d90 0x7fcdac154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:24.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.742+0000 7fcdab59e700 1 -- 172.21.15.40:0/3001657235 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcdac0c9d90 msgr2=0x7fcdac154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:24.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.742+0000 7fcdab59e700 1 --2- 172.21.15.40:0/3001657235 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcdac0c9d90 0x7fcdac154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:24.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.742+0000 7fcdab59e700 1 -- 172.21.15.40:0/3001657235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdac0ca190 msgr2=0x7fcdac1497c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:56:24.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.742+0000 7fcdab59e700 1 --2- 172.21.15.40:0/3001657235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdac0ca190 0x7fcdac1497c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:24.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.742+0000 7fcdab59e700 1 -- 172.21.15.40:0/3001657235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda0009570 con 0x7fcdac153800 2024-09-13T17:56:24.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.742+0000 7fcdab59e700 1 --2- 172.21.15.40:0/3001657235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac153800 0x7fcdac149d00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcda400b9c0 tx=0x7fcda400bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:24.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.742+0000 7fcd937fe700 1 -- 172.21.15.40:0/3001657235 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda400d8e0 con 0x7fcdac153800 2024-09-13T17:56:24.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.742+0000 7fcd937fe700 1 -- 172.21.15.40:0/3001657235 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda4006e90 con 0x7fcdac153800 2024-09-13T17:56:24.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.742+0000 7fcdb0a36700 1 -- 172.21.15.40:0/3001657235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdac15c760 con 0x7fcdac153800 2024-09-13T17:56:24.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.742+0000 7fcd937fe700 1 -- 172.21.15.40:0/3001657235 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda400dcc0 con 0x7fcdac153800 2024-09-13T17:56:24.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.742+0000 7fcdb0a36700 1 -- 172.21.15.40:0/3001657235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdac14f480 con 0x7fcdac153800 2024-09-13T17:56:24.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.742+0000 7fcd917fa700 1 -- 172.21.15.40:0/3001657235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd84005310 con 0x7fcdac153800 2024-09-13T17:56:24.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.742+0000 7fcd937fe700 1 -- 172.21.15.40:0/3001657235 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcda4015400 con 0x7fcdac153800 2024-09-13T17:56:24.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.746+0000 7fcd937fe700 1 --2- 172.21.15.40:0/3001657235 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcd98066710 0x7fcd98068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:24.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.746+0000 7fcdaad9d700 1 --2- 172.21.15.40:0/3001657235 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcd98066710 0x7fcd98068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:24.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.746+0000 7fcd937fe700 1 -- 172.21.15.40:0/3001657235 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcda4084330 con 0x7fcdac153800 2024-09-13T17:56:24.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.746+0000 7fcdaad9d700 1 --2- 172.21.15.40:0/3001657235 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcd98066710 0x7fcd98068bd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c003960 tx=0x7fcd9c0091c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:24.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:24.750+0000 7fcd937fe700 1 -- 172.21.15.40:0/3001657235 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcda4051370 con 0x7fcdac153800 2024-09-13T17:56:25.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:25.002+0000 7fcd917fa700 1 -- 172.21.15.40:0/3001657235 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fcd840055c0 con 0x7fcdac153800 2024-09-13T17:56:25.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:25.002+0000 7fcd937fe700 1 -- 172.21.15.40:0/3001657235 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fcda4054990 con 0x7fcdac153800 2024-09-13T17:56:25.012 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:56:25.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:25.006+0000 7fcd917fa700 1 -- 172.21.15.40:0/3001657235 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcd98066710 msgr2=0x7fcd98068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:25.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:25.006+0000 7fcd917fa700 1 --2- 172.21.15.40:0/3001657235 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcd98066710 0x7fcd98068bd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c003960 tx=0x7fcd9c0091c0 comp rx=0 tx=0).stop 2024-09-13T17:56:25.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:25.006+0000 7fcd917fa700 1 -- 172.21.15.40:0/3001657235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac153800 msgr2=0x7fcdac149d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:25.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:25.006+0000 7fcd917fa700 1 --2- 172.21.15.40:0/3001657235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac153800 0x7fcdac149d00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcda400b9c0 tx=0x7fcda400bcd0 comp rx=0 tx=0).stop 2024-09-13T17:56:25.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:25.006+0000 7fcd917fa700 1 -- 172.21.15.40:0/3001657235 shutdown_connections 2024-09-13T17:56:25.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:25.006+0000 7fcd917fa700 1 --2- 172.21.15.40:0/3001657235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac153800 0x7fcdac149d00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:25.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:25.006+0000 7fcd917fa700 1 --2- 172.21.15.40:0/3001657235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdac0ca190 0x7fcdac1497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:25.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:25.006+0000 7fcd917fa700 1 --2- 172.21.15.40:0/3001657235 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcd98066710 0x7fcd98068bd0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:25.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:25.006+0000 7fcd917fa700 1 --2- 172.21.15.40:0/3001657235 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcdac0c9d90 0x7fcdac154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:25.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:25.006+0000 7fcd917fa700 1 -- 172.21.15.40:0/3001657235 >> 172.21.15.40:0/3001657235 conn(0x7fcdac066090 msgr2=0x7fcdac0cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:25.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:25.006+0000 7fcd917fa700 1 -- 172.21.15.40:0/3001657235 shutdown_connections 2024-09-13T17:56:25.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:25.006+0000 7fcd917fa700 1 -- 172.21.15.40:0/3001657235 wait complete. 2024-09-13T17:56:28.028 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:56:28.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.150+0000 7ff38a04f700 1 Processor -- start 2024-09-13T17:56:28.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.150+0000 7ff38a04f700 1 -- start start 2024-09-13T17:56:28.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.150+0000 7ff38a04f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff384153880 0x7ff3840c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:28.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.150+0000 7ff38a04f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff3840cb2f0 0x7ff3840cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:28.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.150+0000 7ff38a04f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff384159770 con 0x7ff3840cf3c0 2024-09-13T17:56:28.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.150+0000 7ff38a04f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff384159a50 con 0x7ff3840cb2f0 2024-09-13T17:56:28.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.150+0000 7ff38a04f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff384159d30 con 0x7ff384153880 2024-09-13T17:56:28.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.150+0000 7ff38884c700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7ff3840cb2f0 msgr2=0x7ff3840cee80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:56:28.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.150+0000 7ff38984e700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7ff3840cf3c0 legacy=0x7ff38415cf60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T17:56:28.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.150+0000 7ff38884c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff3840cb2f0 0x7ff3840cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:28.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.150+0000 7ff38904d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff384153880 0x7ff3840c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:28.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.150+0000 7ff38904d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff384153880 0x7ff3840c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:45174/0 (socket says 172.21.15.40:45174) 2024-09-13T17:56:28.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.150+0000 7ff38904d700 1 -- 172.21.15.40:0/4166420755 learned_addr learned my addr 172.21.15.40:0/4166420755 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:28.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.150+0000 7ff38904d700 1 -- 172.21.15.40:0/4166420755 >> v2:172.21.15.43:3300/0 conn(0x7ff3840cb2f0 msgr2=0x7ff3840cee80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:56:28.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.150+0000 7ff38904d700 1 --2- 172.21.15.40:0/4166420755 >> v2:172.21.15.43:3300/0 conn(0x7ff3840cb2f0 0x7ff3840cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:28.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.150+0000 7ff38904d700 1 -- 172.21.15.40:0/4166420755 >> v1:172.21.15.43:6789/0 conn(0x7ff3840cf3c0 legacy=0x7ff38415cf60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:56:28.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.150+0000 7ff38904d700 1 -- 172.21.15.40:0/4166420755 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38415a010 con 0x7ff384153880 2024-09-13T17:56:28.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.150+0000 7ff38904d700 1 --2- 172.21.15.40:0/4166420755 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff384153880 0x7ff3840c9ef0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff380009910 tx=0x7ff380009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecf3c3583b7044a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:28.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff37bfff700 1 -- 172.21.15.40:0/4166420755 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff380004230 con 0x7ff384153880 2024-09-13T17:56:28.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff37bfff700 1 -- 172.21.15.40:0/4166420755 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff380004390 con 0x7ff384153880 2024-09-13T17:56:28.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff37bfff700 1 -- 172.21.15.40:0/4166420755 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff380009e60 con 0x7ff384153880 2024-09-13T17:56:28.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38a04f700 1 -- 172.21.15.40:0/4166420755 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff384153880 msgr2=0x7ff3840c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:28.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38a04f700 1 --2- 172.21.15.40:0/4166420755 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff384153880 0x7ff3840c9ef0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff380009910 tx=0x7ff380009c20 comp rx=0 tx=0).stop 2024-09-13T17:56:28.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38a04f700 1 -- 172.21.15.40:0/4166420755 shutdown_connections 2024-09-13T17:56:28.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38a04f700 1 --2- 172.21.15.40:0/4166420755 >> v2:172.21.15.43:3300/0 conn(0x7ff3840cb2f0 0x7ff3840cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:28.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38a04f700 1 --2- 172.21.15.40:0/4166420755 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff384153880 0x7ff3840c9ef0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:28.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38a04f700 1 -- 172.21.15.40:0/4166420755 >> 172.21.15.40:0/4166420755 conn(0x7ff38406c5b0 msgr2=0x7ff38406c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:28.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38a04f700 1 -- 172.21.15.40:0/4166420755 shutdown_connections 2024-09-13T17:56:28.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38a04f700 1 -- 172.21.15.40:0/4166420755 wait complete. 2024-09-13T17:56:28.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38a04f700 1 Processor -- start 2024-09-13T17:56:28.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38a04f700 1 -- start start 2024-09-13T17:56:28.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38a04f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3840cb2f0 0x7ff38415a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:28.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38a04f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3840cf3c0 0x7ff3841efb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:28.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38a04f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff384153880 0x7ff3840c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:28.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38a04f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38415ace0 con 0x7ff3840cf3c0 2024-09-13T17:56:28.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38a04f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3841f0060 con 0x7ff384153880 2024-09-13T17:56:28.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38a04f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38415b480 con 0x7ff3840cb2f0 2024-09-13T17:56:28.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38884c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3840cf3c0 0x7ff3841efb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:28.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38984e700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff384153880 msgr2=0x7ff3840c6e90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:56:28.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38984e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff384153880 0x7ff3840c6e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:28.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38884c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3840cf3c0 0x7ff3841efb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58752/0 (socket says 172.21.15.40:58752) 2024-09-13T17:56:28.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.154+0000 7ff38884c700 1 -- 172.21.15.40:0/4111539412 learned_addr learned my addr 172.21.15.40:0/4111539412 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:28.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.158+0000 7ff38904d700 1 --2- 172.21.15.40:0/4111539412 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3840cb2f0 0x7ff38415a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.158+0000 7ff38904d700 1 -- 172.21.15.40:0/4111539412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff384153880 msgr2=0x7ff3840c6e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:56:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.158+0000 7ff38904d700 1 --2- 172.21.15.40:0/4111539412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff384153880 0x7ff3840c6e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.158+0000 7ff38904d700 1 -- 172.21.15.40:0/4111539412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3840cf3c0 msgr2=0x7ff3841efb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.158+0000 7ff38904d700 1 --2- 172.21.15.40:0/4111539412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3840cf3c0 0x7ff3841efb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.158+0000 7ff38904d700 1 -- 172.21.15.40:0/4111539412 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff380009570 con 0x7ff3840cb2f0 2024-09-13T17:56:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.158+0000 7ff38904d700 1 --2- 172.21.15.40:0/4111539412 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3840cb2f0 0x7ff38415a7a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff3840cffc0 tx=0x7ff380004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.158+0000 7ff379ffb700 1 -- 172.21.15.40:0/4111539412 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3800038c0 con 0x7ff3840cb2f0 2024-09-13T17:56:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.158+0000 7ff379ffb700 1 -- 172.21.15.40:0/4111539412 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff380003a20 con 0x7ff3840cb2f0 2024-09-13T17:56:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.158+0000 7ff38a04f700 1 -- 172.21.15.40:0/4111539412 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3840c73d0 con 0x7ff3840cb2f0 2024-09-13T17:56:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.158+0000 7ff379ffb700 1 -- 172.21.15.40:0/4111539412 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff380003dd0 con 0x7ff3840cb2f0 2024-09-13T17:56:28.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.158+0000 7ff38a04f700 1 -- 172.21.15.40:0/4111539412 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38415b1a0 con 0x7ff3840cb2f0 2024-09-13T17:56:28.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.158+0000 7ff38a04f700 1 -- 172.21.15.40:0/4111539412 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3841530e0 con 0x7ff3840cb2f0 2024-09-13T17:56:28.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.158+0000 7ff379ffb700 1 -- 172.21.15.40:0/4111539412 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff38001b440 con 0x7ff3840cb2f0 2024-09-13T17:56:28.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.162+0000 7ff379ffb700 1 --2- 172.21.15.40:0/4111539412 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff3680666e0 0x7ff368068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:28.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.162+0000 7ff379ffb700 1 -- 172.21.15.40:0/4111539412 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff380085c70 con 0x7ff3840cb2f0 2024-09-13T17:56:28.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.162+0000 7ff38884c700 1 --2- 172.21.15.40:0/4111539412 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff3680666e0 0x7ff368068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:28.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.162+0000 7ff38884c700 1 --2- 172.21.15.40:0/4111539412 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff3680666e0 0x7ff368068ba0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff3740094d0 tx=0x7ff374009460 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:28.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.170+0000 7ff379ffb700 1 -- 172.21.15.40:0/4111539412 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff380052cb0 con 0x7ff3840cb2f0 2024-09-13T17:56:28.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.422+0000 7ff38a04f700 1 -- 172.21.15.40:0/4111539412 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff3840cee80 con 0x7ff3840cb2f0 2024-09-13T17:56:28.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.422+0000 7ff379ffb700 1 -- 172.21.15.40:0/4111539412 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7ff380028080 con 0x7ff3840cb2f0 2024-09-13T17:56:28.431 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:56:28.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.426+0000 7ff38a04f700 1 -- 172.21.15.40:0/4111539412 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff3680666e0 msgr2=0x7ff368068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:28.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.426+0000 7ff38a04f700 1 --2- 172.21.15.40:0/4111539412 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff3680666e0 0x7ff368068ba0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff3740094d0 tx=0x7ff374009460 comp rx=0 tx=0).stop 2024-09-13T17:56:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.426+0000 7ff38a04f700 1 -- 172.21.15.40:0/4111539412 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3840cb2f0 msgr2=0x7ff38415a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.426+0000 7ff38a04f700 1 --2- 172.21.15.40:0/4111539412 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3840cb2f0 0x7ff38415a7a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff3840cffc0 tx=0x7ff380004770 comp rx=0 tx=0).stop 2024-09-13T17:56:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.426+0000 7ff38a04f700 1 -- 172.21.15.40:0/4111539412 shutdown_connections 2024-09-13T17:56:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.426+0000 7ff38a04f700 1 --2- 172.21.15.40:0/4111539412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff384153880 0x7ff3840c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.426+0000 7ff38a04f700 1 --2- 172.21.15.40:0/4111539412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3840cf3c0 0x7ff3841efb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.426+0000 7ff38a04f700 1 --2- 172.21.15.40:0/4111539412 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff3680666e0 0x7ff368068ba0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.426+0000 7ff38a04f700 1 --2- 172.21.15.40:0/4111539412 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3840cb2f0 0x7ff38415a7a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.426+0000 7ff38a04f700 1 -- 172.21.15.40:0/4111539412 >> 172.21.15.40:0/4111539412 conn(0x7ff38406c5b0 msgr2=0x7ff3840c9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.426+0000 7ff38a04f700 1 -- 172.21.15.40:0/4111539412 shutdown_connections 2024-09-13T17:56:28.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:28.426+0000 7ff38a04f700 1 -- 172.21.15.40:0/4111539412 wait complete. 2024-09-13T17:56:28.921 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-09-13T17:56:31.446 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:56:31.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.558+0000 7fdfbb59e700 1 Processor -- start 2024-09-13T17:56:31.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.558+0000 7fdfbb59e700 1 -- start start 2024-09-13T17:56:31.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.558+0000 7fdfbb59e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdfbc153840 0x7fdfbc0c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:31.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.558+0000 7fdfbb59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 0x7fdfbc0cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:31.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.558+0000 7fdfbb59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfbc0cc060 con 0x7fdfbc0cc4f0 2024-09-13T17:56:31.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.558+0000 7fdfbb59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfbc0b2840 con 0x7fdfbc0c9e20 2024-09-13T17:56:31.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.558+0000 7fdfbb59e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfbc0ca870 con 0x7fdfbc153840 2024-09-13T17:56:31.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.558+0000 7fdfbad9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 0x7fdfbc0cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:31.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.558+0000 7fdfb3fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdfbc0cc4f0 0x7fdfbc15d410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45356/0 (socket says 172.21.15.40:45356) 2024-09-13T17:56:31.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.558+0000 7fdfb3fff700 1 -- 172.21.15.40:0/4275301449 learned_addr learned my addr 172.21.15.40:0/4275301449 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:56:31.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.558+0000 7fdfba59c700 1 -- 172.21.15.40:0/4275301449 >> v2:172.21.15.43:3300/0 conn(0x7fdfbc153840 msgr2=0x7fdfbc0c3eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:56:31.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.558+0000 7fdfbad9d700 1 -- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 msgr2=0x7fdfbc0cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:56:31.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.558+0000 7fdfbad9d700 1 -- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 msgr2=0x7fdfbc0cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:56:31.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.558+0000 7fdfbad9d700 1 --2- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 0x7fdfbc0cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:56:31.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.558+0000 7fdfba59c700 1 --2- 172.21.15.40:0/4275301449 >> v2:172.21.15.43:3300/0 conn(0x7fdfbc153840 0x7fdfbc0c3eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:31.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.558+0000 7fdfbad9d700 1 --2- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 0x7fdfbc0cf260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:31.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.762+0000 7fdfba59c700 1 -- 172.21.15.40:0/4275301449 >> v2:172.21.15.43:3300/0 conn(0x7fdfbc153840 msgr2=0x7fdfbc0c3eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:56:31.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.762+0000 7fdfba59c700 1 --2- 172.21.15.40:0/4275301449 >> v2:172.21.15.43:3300/0 conn(0x7fdfbc153840 0x7fdfbc0c3eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:56:31.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.762+0000 7fdfbad9d700 1 --2- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 0x7fdfbc0cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:31.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.762+0000 7fdfbad9d700 1 -- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 msgr2=0x7fdfbc0cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:56:31.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.762+0000 7fdfbad9d700 1 -- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 msgr2=0x7fdfbc0cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:56:31.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.762+0000 7fdfbad9d700 1 --2- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 0x7fdfbc0cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:56:31.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:31.762+0000 7fdfbad9d700 1 --2- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 0x7fdfbc0cf260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:56:32.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:32.166+0000 7fdfba59c700 1 -- 172.21.15.40:0/4275301449 >> v2:172.21.15.43:3300/0 conn(0x7fdfbc153840 msgr2=0x7fdfbc0c3eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:56:32.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:32.166+0000 7fdfba59c700 1 --2- 172.21.15.40:0/4275301449 >> v2:172.21.15.43:3300/0 conn(0x7fdfbc153840 0x7fdfbc0c3eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:56:32.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:32.166+0000 7fdfbad9d700 1 --2- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 0x7fdfbc0cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:32.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:32.166+0000 7fdfbad9d700 1 -- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 msgr2=0x7fdfbc0cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:56:32.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:32.166+0000 7fdfbad9d700 1 -- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 msgr2=0x7fdfbc0cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:56:32.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:32.166+0000 7fdfbad9d700 1 --2- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 0x7fdfbc0cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:56:32.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:32.166+0000 7fdfbad9d700 1 --2- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 0x7fdfbc0cf260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:56:32.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:32.970+0000 7fdfba59c700 1 -- 172.21.15.40:0/4275301449 >> v2:172.21.15.43:3300/0 conn(0x7fdfbc153840 msgr2=0x7fdfbc0c3eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:56:32.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:32.970+0000 7fdfba59c700 1 --2- 172.21.15.40:0/4275301449 >> v2:172.21.15.43:3300/0 conn(0x7fdfbc153840 0x7fdfbc0c3eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:56:32.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:32.970+0000 7fdfbad9d700 1 --2- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 0x7fdfbc0cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:32.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:32.970+0000 7fdfbad9d700 1 -- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 msgr2=0x7fdfbc0cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:56:32.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:32.970+0000 7fdfbad9d700 1 -- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 msgr2=0x7fdfbc0cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:56:32.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:32.970+0000 7fdfbad9d700 1 --2- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 0x7fdfbc0cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:56:32.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:32.970+0000 7fdfbad9d700 1 --2- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 0x7fdfbc0cf260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:56:34.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:34.558+0000 7fdfb8d99700 1 -- 172.21.15.40:0/4275301449 >> v2:172.21.15.43:3300/0 conn(0x7fdfbc153840 msgr2=0x7fdfbc0c3eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:56:34.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:34.558+0000 7fdfb8d99700 1 --2- 172.21.15.40:0/4275301449 >> v2:172.21.15.43:3300/0 conn(0x7fdfbc153840 0x7fdfbc0c3eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:34.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:34.558+0000 7fdfb8d99700 1 -- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 msgr2=0x7fdfbc0cf260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:56:34.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:34.558+0000 7fdfb8d99700 1 --2- 172.21.15.40:0/4275301449 >> v2:172.21.15.40:3300/0 conn(0x7fdfbc0c9e20 0x7fdfbc0cf260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:34.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:34.558+0000 7fdfb8d99700 1 -- 172.21.15.40:0/4275301449 >> v1:172.21.15.40:6789/0 conn(0x7fdfbc0cc4f0 legacy=0x7fdfbc15d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:34.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:34.558+0000 7fdfb8d99700 1 --2- 172.21.15.40:0/4275301449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf9c0073f0 0x7fdf9c0098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:34.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:34.558+0000 7fdfb8d99700 1 -- 172.21.15.40:0/4275301449 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfbc153c20 con 0x7fdf9c003710 2024-09-13T17:56:34.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:34.558+0000 7fdfb8d99700 1 -- 172.21.15.40:0/4275301449 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc0ca870 con 0x7fdf9c0073f0 2024-09-13T17:56:34.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:34.558+0000 7fdfba59c700 1 -- 172.21.15.40:0/4275301449 >> v1:172.21.15.43:6789/0 conn(0x7fdf9c003710 legacy=0x7fdf9c005b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T17:56:34.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:34.558+0000 7fdfb3fff700 1 --2- 172.21.15.40:0/4275301449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf9c0073f0 0x7fdf9c0098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:34.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:34.558+0000 7fdfb3fff700 1 -- 172.21.15.40:0/4275301449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf9c0073f0 msgr2=0x7fdf9c0098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:56:34.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:34.558+0000 7fdfb3fff700 1 -- 172.21.15.40:0/4275301449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf9c0073f0 msgr2=0x7fdf9c0098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:56:34.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:34.558+0000 7fdfb3fff700 1 --2- 172.21.15.40:0/4275301449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf9c0073f0 0x7fdf9c0098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:56:34.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:34.558+0000 7fdfb3fff700 1 --2- 172.21.15.40:0/4275301449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf9c0073f0 0x7fdf9c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:34.754 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:56:34.756 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:56:34.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:34.762+0000 7fdfba59c700 1 -- 172.21.15.40:0/4275301449 >> v1:172.21.15.43:6789/0 conn(0x7fdf9c003710 legacy=0x7fdf9c005b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T17:56:34.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:34.762+0000 7fdfb3fff700 1 --2- 172.21.15.40:0/4275301449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf9c0073f0 0x7fdf9c0098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:34.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:34.762+0000 7fdfb3fff700 1 -- 172.21.15.40:0/4275301449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf9c0073f0 msgr2=0x7fdf9c0098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:56:34.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:34.762+0000 7fdfb3fff700 1 -- 172.21.15.40:0/4275301449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf9c0073f0 msgr2=0x7fdf9c0098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:56:34.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:34.762+0000 7fdfb3fff700 1 --2- 172.21.15.40:0/4275301449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf9c0073f0 0x7fdf9c0098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:56:34.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:34.762+0000 7fdfb3fff700 1 --2- 172.21.15.40:0/4275301449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf9c0073f0 0x7fdf9c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:56:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.166+0000 7fdfba59c700 1 -- 172.21.15.40:0/4275301449 >> v1:172.21.15.43:6789/0 conn(0x7fdf9c003710 legacy=0x7fdf9c005b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T17:56:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.166+0000 7fdfb3fff700 1 --2- 172.21.15.40:0/4275301449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf9c0073f0 0x7fdf9c0098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.166+0000 7fdfb3fff700 1 -- 172.21.15.40:0/4275301449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf9c0073f0 msgr2=0x7fdf9c0098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:56:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.166+0000 7fdfb3fff700 1 -- 172.21.15.40:0/4275301449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf9c0073f0 msgr2=0x7fdf9c0098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:56:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.166+0000 7fdfb3fff700 1 --2- 172.21.15.40:0/4275301449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf9c0073f0 0x7fdf9c0098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:56:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.166+0000 7fdfb3fff700 1 --2- 172.21.15.40:0/4275301449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf9c0073f0 0x7fdf9c0098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:56:35.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.970+0000 7fdfba59c700 1 -- 172.21.15.40:0/4275301449 >> v1:172.21.15.43:6789/0 conn(0x7fdf9c003710 legacy=0x7fdf9c005b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T17:56:35.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.970+0000 7fdfb3fff700 1 --2- 172.21.15.40:0/4275301449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf9c0073f0 0x7fdf9c0098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:35.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.970+0000 7fdfb3fff700 1 -- 172.21.15.40:0/4275301449 >> v1:172.21.15.43:6789/0 conn(0x7fdf9c003710 legacy=0x7fdf9c005b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:56:35.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.970+0000 7fdfb3fff700 1 -- 172.21.15.40:0/4275301449 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c00b0b0 con 0x7fdf9c0073f0 2024-09-13T17:56:35.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.970+0000 7fdfb3fff700 1 --2- 172.21.15.40:0/4275301449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf9c0073f0 0x7fdf9c0098b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4005d90 tx=0x7fdfa4007330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=693314ad8c5a76ce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:35.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.970+0000 7fdfb9d9b700 1 -- 172.21.15.40:0/4275301449 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4005e80 con 0x7fdf9c0073f0 2024-09-13T17:56:35.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.970+0000 7fdfb9d9b700 1 -- 172.21.15.40:0/4275301449 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa4016480 con 0x7fdf9c0073f0 2024-09-13T17:56:35.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.970+0000 7fdfb9d9b700 1 -- 172.21.15.40:0/4275301449 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa40167f0 con 0x7fdf9c0073f0 2024-09-13T17:56:35.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.970+0000 7fdfbb59e700 1 -- 172.21.15.40:0/4275301449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf9c0073f0 msgr2=0x7fdf9c0098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:35.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.970+0000 7fdfbb59e700 1 --2- 172.21.15.40:0/4275301449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf9c0073f0 0x7fdf9c0098b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4005d90 tx=0x7fdfa4007330 comp rx=0 tx=0).stop 2024-09-13T17:56:35.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.970+0000 7fdfbad9d700 1 -- 172.21.15.40:0/4275301449 reap_dead start 2024-09-13T17:56:35.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.970+0000 7fdfbb59e700 1 -- 172.21.15.40:0/4275301449 shutdown_connections 2024-09-13T17:56:35.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.970+0000 7fdfbb59e700 1 -- 172.21.15.40:0/4275301449 >> 172.21.15.40:0/4275301449 conn(0x7fdfbc06c800 msgr2=0x7fdfbc06cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:35.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.974+0000 7fdfbb59e700 1 -- 172.21.15.40:0/4275301449 shutdown_connections 2024-09-13T17:56:35.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.974+0000 7fdfbb59e700 1 -- 172.21.15.40:0/4275301449 wait complete. 2024-09-13T17:56:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.974+0000 7fdfbb59e700 1 Processor -- start 2024-09-13T17:56:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.974+0000 7fdfbb59e700 1 -- start start 2024-09-13T17:56:35.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.974+0000 7fdfbb59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9c0073f0 0x7fdfbc14a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:35.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.974+0000 7fdfbb59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfbc14bc00 0x7fdfbc14ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:35.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.974+0000 7fdfbb59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdfbc0c8640 0x7fdfbc0c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:35.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.974+0000 7fdfbb59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc156e50 con 0x7fdfbc14bc00 2024-09-13T17:56:35.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.974+0000 7fdfbb59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc14d2e0 con 0x7fdf9c0073f0 2024-09-13T17:56:35.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.974+0000 7fdfbb59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc0c9000 con 0x7fdfbc0c8640 2024-09-13T17:56:35.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.978+0000 7fdfba59c700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9c0073f0 msgr2=0x7fdfbc14a640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:56:35.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.978+0000 7fdfba59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9c0073f0 0x7fdfbc14a640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:35.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.978+0000 7fdfb3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfbc14bc00 0x7fdfbc14ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:35.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.978+0000 7fdfb3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfbc14bc00 0x7fdfbc14ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60506/0 (socket says 172.21.15.40:60506) 2024-09-13T17:56:35.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.978+0000 7fdfb3fff700 1 -- 172.21.15.40:0/3610484737 learned_addr learned my addr 172.21.15.40:0/3610484737 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:35.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.978+0000 7fdfbad9d700 1 --2- 172.21.15.40:0/3610484737 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdfbc0c8640 0x7fdfbc0c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:35.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.978+0000 7fdfb3fff700 1 -- 172.21.15.40:0/3610484737 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdfbc0c8640 msgr2=0x7fdfbc0c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:35.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.978+0000 7fdfb3fff700 1 --2- 172.21.15.40:0/3610484737 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdfbc0c8640 0x7fdfbc0c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:35.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.978+0000 7fdfb3fff700 1 -- 172.21.15.40:0/3610484737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9c0073f0 msgr2=0x7fdfbc14a640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:56:35.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.978+0000 7fdfb3fff700 1 --2- 172.21.15.40:0/3610484737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9c0073f0 0x7fdfbc14a640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:35.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.978+0000 7fdfb3fff700 1 -- 172.21.15.40:0/3610484737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa400f040 con 0x7fdfbc14bc00 2024-09-13T17:56:35.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.978+0000 7fdfb3fff700 1 --2- 172.21.15.40:0/3610484737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfbc14bc00 0x7fdfbc14ab80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4017000 tx=0x7fdfa4001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:35.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.978+0000 7fdfb2ffd700 1 -- 172.21.15.40:0/3610484737 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4021820 con 0x7fdfbc14bc00 2024-09-13T17:56:35.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.978+0000 7fdfb2ffd700 1 -- 172.21.15.40:0/3610484737 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa4021980 con 0x7fdfbc14bc00 2024-09-13T17:56:35.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.978+0000 7fdfbb59e700 1 -- 172.21.15.40:0/3610484737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfbc06d070 con 0x7fdfbc14bc00 2024-09-13T17:56:35.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.978+0000 7fdfb2ffd700 1 -- 172.21.15.40:0/3610484737 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4021d30 con 0x7fdfbc14bc00 2024-09-13T17:56:35.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.978+0000 7fdfbb59e700 1 -- 172.21.15.40:0/3610484737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfbc156b70 con 0x7fdfbc14bc00 2024-09-13T17:56:35.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.978+0000 7fdfbb59e700 1 -- 172.21.15.40:0/3610484737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfbc0cbcf0 con 0x7fdfbc14bc00 2024-09-13T17:56:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.982+0000 7fdfb2ffd700 1 -- 172.21.15.40:0/3610484737 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdfa401f400 con 0x7fdfbc14bc00 2024-09-13T17:56:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.982+0000 7fdfb2ffd700 1 --2- 172.21.15.40:0/3610484737 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf9c067520 0x7fdf9c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.982+0000 7fdfb2ffd700 1 -- 172.21.15.40:0/3610484737 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdfa4086440 con 0x7fdfbc14bc00 2024-09-13T17:56:35.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.982+0000 7fdfba59c700 1 --2- 172.21.15.40:0/3610484737 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf9c067520 0x7fdf9c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:35.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.982+0000 7fdfba59c700 1 --2- 172.21.15.40:0/3610484737 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf9c067520 0x7fdf9c0699e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0039e0 tx=0x7fdfac0070d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:35.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:35.990+0000 7fdfb2ffd700 1 -- 172.21.15.40:0/3610484737 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdfa40535a0 con 0x7fdfbc14bc00 2024-09-13T17:56:36.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.278+0000 7fdfbb59e700 1 -- 172.21.15.40:0/3610484737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdfbc0ca9d0 con 0x7fdfbc14bc00 2024-09-13T17:56:36.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.278+0000 7fdfb2ffd700 1 -- 172.21.15.40:0/3610484737 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7fdfa4056bc0 con 0x7fdfbc14bc00 2024-09-13T17:56:36.288 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":50,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:56:36.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.282+0000 7fdfbb59e700 1 -- 172.21.15.40:0/3610484737 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf9c067520 msgr2=0x7fdf9c0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:36.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.282+0000 7fdfbb59e700 1 --2- 172.21.15.40:0/3610484737 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf9c067520 0x7fdf9c0699e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0039e0 tx=0x7fdfac0070d0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.282+0000 7fdfbb59e700 1 -- 172.21.15.40:0/3610484737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfbc14bc00 msgr2=0x7fdfbc14ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:36.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.282+0000 7fdfbb59e700 1 --2- 172.21.15.40:0/3610484737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfbc14bc00 0x7fdfbc14ab80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4017000 tx=0x7fdfa4001190 comp rx=0 tx=0).stop 2024-09-13T17:56:36.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.282+0000 7fdfbb59e700 1 -- 172.21.15.40:0/3610484737 shutdown_connections 2024-09-13T17:56:36.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.282+0000 7fdfbb59e700 1 --2- 172.21.15.40:0/3610484737 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf9c067520 0x7fdf9c0699e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.282+0000 7fdfbb59e700 1 --2- 172.21.15.40:0/3610484737 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdfbc0c8640 0x7fdfbc0c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.282+0000 7fdfbb59e700 1 --2- 172.21.15.40:0/3610484737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfbc14bc00 0x7fdfbc14ab80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.282+0000 7fdfbb59e700 1 --2- 172.21.15.40:0/3610484737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9c0073f0 0x7fdfbc14a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.282+0000 7fdfbb59e700 1 -- 172.21.15.40:0/3610484737 >> 172.21.15.40:0/3610484737 conn(0x7fdfbc06c800 msgr2=0x7fdfbc14f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:36.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.282+0000 7fdfbb59e700 1 -- 172.21.15.40:0/3610484737 shutdown_connections 2024-09-13T17:56:36.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.282+0000 7fdfbb59e700 1 -- 172.21.15.40:0/3610484737 wait complete. 2024-09-13T17:56:36.302 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T17:56:36.302 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-13T17:56:36.302 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:56:36.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880ebe6700 1 Processor -- start 2024-09-13T17:56:36.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880ebe6700 1 -- start start 2024-09-13T17:56:36.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880ebe6700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f88000477b0 0x7f8800047bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:36.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880ebe6700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88000e1cc0 con 0x7f88000088d0 2024-09-13T17:56:36.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880ebe6700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88000e1f80 con 0x7f88000d8ad0 2024-09-13T17:56:36.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880ebe6700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88000e2260 con 0x7f88000477b0 2024-09-13T17:56:36.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880d3e3700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7f88000477b0 msgr2=0x7f8800047bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:56:36.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880e3e5700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f88000088d0 0x7f8800008ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45368/0 (socket says 172.21.15.40:45368) 2024-09-13T17:56:36.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880e3e5700 1 -- 172.21.15.40:0/611810132 learned_addr learned my addr 172.21.15.40:0/611810132 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:56:36.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880d3e3700 1 --2- 172.21.15.40:0/611810132 >> v2:172.21.15.43:3300/0 conn(0x7f88000477b0 0x7f8800047bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:36.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880dbe4700 1 -- 172.21.15.40:0/611810132 >> v1:172.21.15.43:6789/0 conn(0x7f88000d8ad0 legacy=0x7f880004cd00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T17:56:36.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880cbe2700 1 -- 172.21.15.40:0/611810132 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1595077617 0 0) 0x7f88000e1cc0 con 0x7f88000088d0 2024-09-13T17:56:36.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880cbe2700 1 -- 172.21.15.40:0/611810132 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87f40035d0 con 0x7f88000088d0 2024-09-13T17:56:36.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880cbe2700 1 -- 172.21.15.40:0/611810132 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1084472098 0 0) 0x7f87f40035d0 con 0x7f88000088d0 2024-09-13T17:56:36.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880cbe2700 1 -- 172.21.15.40:0/611810132 >> v2:172.21.15.43:3300/0 conn(0x7f88000477b0 msgr2=0x7f8800047bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:56:36.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880cbe2700 1 --2- 172.21.15.40:0/611810132 >> v2:172.21.15.43:3300/0 conn(0x7f88000477b0 0x7f8800047bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880cbe2700 1 -- 172.21.15.40:0/611810132 >> v1:172.21.15.43:6789/0 conn(0x7f88000d8ad0 legacy=0x7f880004cd00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:56:36.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880cbe2700 1 -- 172.21.15.40:0/611810132 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88000e2540 con 0x7f88000088d0 2024-09-13T17:56:36.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880cbe2700 1 -- 172.21.15.40:0/611810132 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f8808067210 con 0x7f88000088d0 2024-09-13T17:56:36.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880cbe2700 1 -- 172.21.15.40:0/611810132 >> v1:172.21.15.40:6789/0 conn(0x7f88000088d0 legacy=0x7f8800008ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:36.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880cbe2700 1 --2- 172.21.15.40:0/611810132 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f87f4003a20 0x7f87f4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:36.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880cbe2700 1 --2- 172.21.15.40:0/611810132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f4004870 0x7f87f4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:36.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880cbe2700 1 --2- 172.21.15.40:0/611810132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87f4005230 0x7f87f400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:36.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880cbe2700 1 -- 172.21.15.40:0/611810132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88000e2260 con 0x7f87f4005230 2024-09-13T17:56:36.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880cbe2700 1 -- 172.21.15.40:0/611810132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f400c160 con 0x7f87f4004870 2024-09-13T17:56:36.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880cbe2700 1 -- 172.21.15.40:0/611810132 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f400c320 con 0x7f87f4003a20 2024-09-13T17:56:36.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880d3e3700 1 -- 172.21.15.40:0/611810132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f4004870 msgr2=0x7f87f4004cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:56:36.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880d3e3700 1 --2- 172.21.15.40:0/611810132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f4004870 0x7f87f4004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:36.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.410+0000 7f880e3e5700 1 --2- 172.21.15.40:0/611810132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87f4005230 0x7f87f400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:36.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880dbe4700 1 --2- 172.21.15.40:0/611810132 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f87f4003a20 0x7f87f4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:36.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880e3e5700 1 -- 172.21.15.40:0/611810132 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f87f4003a20 msgr2=0x7f87f4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:56:36.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880e3e5700 1 --2- 172.21.15.40:0/611810132 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f87f4003a20 0x7f87f4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880e3e5700 1 -- 172.21.15.40:0/611810132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f4004870 msgr2=0x7f87f4004cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:56:36.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880e3e5700 1 --2- 172.21.15.40:0/611810132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f4004870 0x7f87f4004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880e3e5700 1 -- 172.21.15.40:0/611810132 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f40035d0 con 0x7f87f4005230 2024-09-13T17:56:36.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880dbe4700 1 --2- 172.21.15.40:0/611810132 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f87f4003a20 0x7f87f4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:56:36.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880e3e5700 1 -- 172.21.15.40:0/611810132 reap_dead start 2024-09-13T17:56:36.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880e3e5700 1 --2- 172.21.15.40:0/611810132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87f4005230 0x7f87f400ba30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f880802ce60 tx=0x7f87f40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca0033d47eceb7c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:36.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880cbe2700 1 -- 172.21.15.40:0/611810132 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808074600 con 0x7f87f4005230 2024-09-13T17:56:36.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880cbe2700 1 -- 172.21.15.40:0/611810132 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8808074780 con 0x7f87f4005230 2024-09-13T17:56:36.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880cbe2700 1 -- 172.21.15.40:0/611810132 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808067760 con 0x7f87f4005230 2024-09-13T17:56:36.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880ebe6700 1 -- 172.21.15.40:0/611810132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87f4005230 msgr2=0x7f87f400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:36.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880ebe6700 1 --2- 172.21.15.40:0/611810132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87f4005230 0x7f87f400ba30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f880802ce60 tx=0x7f87f40058a0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880ebe6700 1 -- 172.21.15.40:0/611810132 shutdown_connections 2024-09-13T17:56:36.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880ebe6700 1 --2- 172.21.15.40:0/611810132 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87f4005230 0x7f87f400ba30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880ebe6700 1 -- 172.21.15.40:0/611810132 >> 172.21.15.40:0/611810132 conn(0x7f880000dc40 msgr2=0x7f880000e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:36.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880ebe6700 1 -- 172.21.15.40:0/611810132 shutdown_connections 2024-09-13T17:56:36.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880ebe6700 1 -- 172.21.15.40:0/611810132 wait complete. 2024-09-13T17:56:36.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880ebe6700 1 Processor -- start 2024-09-13T17:56:36.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.414+0000 7f880ebe6700 1 -- start start 2024-09-13T17:56:36.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880ebe6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87f4005230 0x7f8800178220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:36.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880ebe6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88000cfb10 0x7f8800052130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:36.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880ebe6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88000d0340 0x7f88000d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:36.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880ebe6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88000d0d00 con 0x7f87f4005230 2024-09-13T17:56:36.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880ebe6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88001745f0 con 0x7f88000d0340 2024-09-13T17:56:36.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880ebe6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88001748d0 con 0x7f88000cfb10 2024-09-13T17:56:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880dbe4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87f4005230 0x7f8800178220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880e3e5700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88000d0340 msgr2=0x7f88000d07c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:56:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880e3e5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88000d0340 0x7f88000d07c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880dbe4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87f4005230 0x7f8800178220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60516/0 (socket says 172.21.15.40:60516) 2024-09-13T17:56:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880dbe4700 1 -- 172.21.15.40:0/1289321846 learned_addr learned my addr 172.21.15.40:0/1289321846 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880d3e3700 1 --2- 172.21.15.40:0/1289321846 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88000cfb10 0x7f8800052130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880dbe4700 1 -- 172.21.15.40:0/1289321846 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88000cfb10 msgr2=0x7f8800052130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880dbe4700 1 --2- 172.21.15.40:0/1289321846 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88000cfb10 0x7f8800052130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880dbe4700 1 -- 172.21.15.40:0/1289321846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88000d0340 msgr2=0x7f88000d07c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:56:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880dbe4700 1 --2- 172.21.15.40:0/1289321846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88000d0340 0x7f88000d07c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880dbe4700 1 -- 172.21.15.40:0/1289321846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8800174bb0 con 0x7f87f4005230 2024-09-13T17:56:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880dbe4700 1 --2- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87f4005230 0x7f8800178220 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f880400dd70 tx=0x7f880400b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f87fe7fc700 1 -- 172.21.15.40:0/1289321846 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8804003c60 con 0x7f87f4005230 2024-09-13T17:56:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f87fe7fc700 1 -- 172.21.15.40:0/1289321846 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8804003dc0 con 0x7f87f4005230 2024-09-13T17:56:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880ebe6700 1 -- 172.21.15.40:0/1289321846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8800174e90 con 0x7f87f4005230 2024-09-13T17:56:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f87fe7fc700 1 -- 172.21.15.40:0/1289321846 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8804005310 con 0x7f87f4005230 2024-09-13T17:56:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880ebe6700 1 -- 172.21.15.40:0/1289321846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88000cf880 con 0x7f87f4005230 2024-09-13T17:56:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880ebe6700 1 --2- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8800033f30 0x7f8800034310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880ebe6700 1 -- 172.21.15.40:0/1289321846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88000074a0 con 0x7f8800033f30 2024-09-13T17:56:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.418+0000 7f880d3e3700 1 --2- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8800033f30 0x7f8800034310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.422+0000 7f880d3e3700 1 -- 172.21.15.40:0/1289321846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88000471c0 con 0x7f8800033f30 2024-09-13T17:56:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.422+0000 7f880d3e3700 1 --2- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8800033f30 0x7f8800034310 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f87f80039e0 tx=0x7f87f8009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.422+0000 7f87fe7fc700 1 -- 172.21.15.40:0/1289321846 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8804005470 con 0x7f87f4005230 2024-09-13T17:56:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.422+0000 7f87fe7fc700 1 --2- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f87ec066760 0x7f87ec068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.422+0000 7f87fe7fc700 1 -- 172.21.15.40:0/1289321846 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f88040847c0 con 0x7f87f4005230 2024-09-13T17:56:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.422+0000 7f880e3e5700 1 --2- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f87ec066760 0x7f87ec068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:36.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.422+0000 7f87fe7fc700 1 -- 172.21.15.40:0/1289321846 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8012600 con 0x7f8800033f30 2024-09-13T17:56:36.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.422+0000 7f87fe7fc700 1 -- 172.21.15.40:0/1289321846 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f87f8012760 con 0x7f8800033f30 2024-09-13T17:56:36.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.422+0000 7f87fe7fc700 1 -- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8800033f30 msgr2=0x7f8800034310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:36.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.422+0000 7f87fe7fc700 1 --2- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8800033f30 0x7f8800034310 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f87f80039e0 tx=0x7f87f8009aa0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.422+0000 7f87fe7fc700 1 -- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8800033f30 msgr2=0x7f8800034310 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:56:36.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.422+0000 7f87fe7fc700 1 --2- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8800033f30 0x7f8800034310 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.422+0000 7f880e3e5700 1 --2- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f87ec066760 0x7f87ec068c20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f87f400c130 tx=0x7f8808071050 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f880ebe6700 1 --2- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8800031980 0x7f8800031d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f880ebe6700 1 -- 172.21.15.40:0/1289321846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88000cfef0 con 0x7f8800031980 2024-09-13T17:56:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f880d3e3700 1 --2- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8800031980 0x7f8800031d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:36.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f880d3e3700 1 -- 172.21.15.40:0/1289321846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f88000e2b00 con 0x7f8800031980 2024-09-13T17:56:36.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f880d3e3700 1 --2- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8800031980 0x7f8800031d60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f87f80039e0 tx=0x7f87f8005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:36.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f87fe7fc700 1 -- 172.21.15.40:0/1289321846 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8023070 con 0x7f8800031980 2024-09-13T17:56:36.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f87fe7fc700 1 -- 172.21.15.40:0/1289321846 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4938 (secure 0 0 0) 0x7f87f801b740 con 0x7f8800031980 2024-09-13T17:56:36.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f87fe7fc700 1 -- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8800031980 msgr2=0x7f8800031d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:36.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f87fe7fc700 1 --2- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8800031980 0x7f8800031d60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f87f80039e0 tx=0x7f87f8005a80 comp rx=0 tx=0).stop 2024-09-13T17:56:36.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f87fe7fc700 1 -- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8800031980 msgr2=0x7f8800031d60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:56:36.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f87fe7fc700 1 --2- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8800031980 0x7f8800031d60 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f87f80039e0 tx=0x7f87f8005a80 comp rx=0 tx=0).stop 2024-09-13T17:56:36.444 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:56:36.444 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:56:36.444 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:56:36.444 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:56:36.444 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 50, 2024-09-13T17:56:36.444 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:56:36.444 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:56:36.445 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:56:36.445 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:36.445 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T17:56:36.445 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:56:36.445 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:56:36.445 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:56:36.445 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:36.445 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:36.445 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:36.445 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:36.445 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:36.445 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:36.445 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:36.446 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:36.446 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:36.446 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:36.446 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:36.446 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:56:36.446 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:56:36.446 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:36.446 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:36.446 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:36.446 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:36.447 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:36.447 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:36.447 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:36.447 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:36.447 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:36.447 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:36.447 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:36.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.447 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:56:36.447 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:56:36.447 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:56:36.447 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:56:36.447 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:56:36.447 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:56:36.448 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:56:36.448 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:56:36.448 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:56:36.448 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:56:36.448 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:56:36.448 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:56:36.448 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:56:36.448 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:56:36.448 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:56:36.448 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:56:36.448 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:56:36.448 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:36.448 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:36.448 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:36.449 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:36.449 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:36.449 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:36.449 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:36.449 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:36.449 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:36.449 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:36.449 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:36.449 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:56:36.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.449 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:56:36.449 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.449 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:56:36.449 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:56:36.450 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:36.450 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:36.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.450 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:36.450 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:56:36.450 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:36.450 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.450 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:36.450 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:56:36.450 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:36.450 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:36.450 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:36.450 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.451 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:56:36.451 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:56:36.451 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:36.451 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:36.451 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:36.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.451 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:56:36.451 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:56:36.451 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:36.451 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:36.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.451 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:36.451 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:56:36.451 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:36.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.452 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.452 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:36.452 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:56:36.452 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:36.452 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:36.452 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:36.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.452 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:56:36.452 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:56:36.452 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:36.452 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:36.453 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:36.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.453 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:56:36.453 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:56:36.453 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:36.453 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:36.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.453 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:36.453 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:56:36.453 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:36.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.454 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:36.454 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:56:36.454 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:36.454 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:36.454 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:36.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.454 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:56:36.454 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:56:36.454 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:36.454 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:36.454 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:36.454 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:36.454 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:36.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.454 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:56:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:56:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:56:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:56:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:56:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:56:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:56:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:56:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:56:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:56:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:56:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:56:36.457 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:56:36.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f880ebe6700 1 -- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f87ec066760 msgr2=0x7f87ec068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:36.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f880ebe6700 1 --2- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f87ec066760 0x7f87ec068c20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f87f400c130 tx=0x7f8808071050 comp rx=0 tx=0).stop 2024-09-13T17:56:36.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f880ebe6700 1 -- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87f4005230 msgr2=0x7f8800178220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:36.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f880ebe6700 1 --2- 172.21.15.40:0/1289321846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87f4005230 0x7f8800178220 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f880400dd70 tx=0x7f880400b5f0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f880e3e5700 1 -- 172.21.15.40:0/1289321846 reap_dead start 2024-09-13T17:56:36.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f880e3e5700 1 -- 172.21.15.40:0/1289321846 reap_dead start 2024-09-13T17:56:36.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f880ebe6700 1 -- 172.21.15.40:0/1289321846 shutdown_connections 2024-09-13T17:56:36.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f880ebe6700 1 -- 172.21.15.40:0/1289321846 >> 172.21.15.40:0/1289321846 conn(0x7f880000dc40 msgr2=0x7f88000d69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:36.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f880ebe6700 1 -- 172.21.15.40:0/1289321846 shutdown_connections 2024-09-13T17:56:36.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.434+0000 7f880ebe6700 1 -- 172.21.15.40:0/1289321846 wait complete. 2024-09-13T17:56:36.460 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T17:56:36.460 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T17:56:36.460 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:56:36.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f5690a6e700 1 Processor -- start 2024-09-13T17:56:36.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f5690a6e700 1 -- start start 2024-09-13T17:56:36.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f5690a6e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5684045a40 0x7f5684045e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:36.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f5690a6e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5684045480 0x7f56840e5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:36.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f5690a6e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56840334f0 con 0x7f5684007c70 2024-09-13T17:56:36.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f5690a6e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5684033a00 con 0x7f5684045a40 2024-09-13T17:56:36.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f5690a6e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5684046360 con 0x7f5684045480 2024-09-13T17:56:36.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f568b7fe700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7f5684045a40 msgr2=0x7f5684045e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:56:36.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f568b7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5684045a40 0x7f5684045e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:36.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f568affd700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7f5684007c70 legacy=0x7f56840df190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T17:56:36.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f568bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5684045480 0x7f56840e5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:36.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f568bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5684045480 0x7f56840e5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:58324/0 (socket says 172.21.15.40:58324) 2024-09-13T17:56:36.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f568bfff700 1 -- 172.21.15.40:0/2666195297 learned_addr learned my addr 172.21.15.40:0/2666195297 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:36.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f568bfff700 1 -- 172.21.15.40:0/2666195297 >> v2:172.21.15.43:3300/0 conn(0x7f5684045a40 msgr2=0x7f5684045e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:56:36.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f568bfff700 1 --2- 172.21.15.40:0/2666195297 >> v2:172.21.15.43:3300/0 conn(0x7f5684045a40 0x7f5684045e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f568bfff700 1 -- 172.21.15.40:0/2666195297 >> v1:172.21.15.43:6789/0 conn(0x7f5684007c70 legacy=0x7f56840df190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:56:36.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f568bfff700 1 -- 172.21.15.40:0/2666195297 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5684033720 con 0x7f5684045480 2024-09-13T17:56:36.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f568bfff700 1 --2- 172.21.15.40:0/2666195297 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5684045480 0x7f56840e5d20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f568c063990 tx=0x7f568c069e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19142f809424d805 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:36.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f568a7fc700 1 -- 172.21.15.40:0/2666195297 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c068a10 con 0x7f5684045480 2024-09-13T17:56:36.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f568a7fc700 1 -- 172.21.15.40:0/2666195297 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568c067dd0 con 0x7f5684045480 2024-09-13T17:56:36.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.570+0000 7f568a7fc700 1 -- 172.21.15.40:0/2666195297 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c0744e0 con 0x7f5684045480 2024-09-13T17:56:36.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.574+0000 7f5690a6e700 1 -- 172.21.15.40:0/2666195297 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5684045480 msgr2=0x7f56840e5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:36.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.574+0000 7f5690a6e700 1 --2- 172.21.15.40:0/2666195297 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5684045480 0x7f56840e5d20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f568c063990 tx=0x7f568c069e70 comp rx=0 tx=0).stop 2024-09-13T17:56:36.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.574+0000 7f5690a6e700 1 -- 172.21.15.40:0/2666195297 shutdown_connections 2024-09-13T17:56:36.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.574+0000 7f5690a6e700 1 --2- 172.21.15.40:0/2666195297 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5684045480 0x7f56840e5d20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.574+0000 7f5690a6e700 1 --2- 172.21.15.40:0/2666195297 >> v2:172.21.15.43:3300/0 conn(0x7f5684045a40 0x7f5684045e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.574+0000 7f5690a6e700 1 -- 172.21.15.40:0/2666195297 >> 172.21.15.40:0/2666195297 conn(0x7f56840505e0 msgr2=0x7f56840509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:36.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.574+0000 7f5690a6e700 1 -- 172.21.15.40:0/2666195297 shutdown_connections 2024-09-13T17:56:36.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.574+0000 7f5690a6e700 1 -- 172.21.15.40:0/2666195297 wait complete. 2024-09-13T17:56:36.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.574+0000 7f5690a6e700 1 Processor -- start 2024-09-13T17:56:36.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.574+0000 7f5690a6e700 1 -- start start 2024-09-13T17:56:36.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f5690a6e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5684007c70 0x7f56840caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:36.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f5690a6e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5684045480 0x7f56840dd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f5690a6e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5684045a40 0x7f56840cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f5690a6e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56840c8cc0 con 0x7f5684045a40 2024-09-13T17:56:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f5690a6e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56840cb780 con 0x7f5684007c70 2024-09-13T17:56:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f5690a6e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56840d1700 con 0x7f5684045480 2024-09-13T17:56:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f568b7fe700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5684007c70 msgr2=0x7f56840caa50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T17:56:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f568b7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5684007c70 0x7f56840caa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f568bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5684045a40 0x7f56840cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f568bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5684045a40 0x7f56840cd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60544/0 (socket says 172.21.15.40:60544) 2024-09-13T17:56:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f568bfff700 1 -- 172.21.15.40:0/4060752801 learned_addr learned my addr 172.21.15.40:0/4060752801 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f568affd700 1 --2- 172.21.15.40:0/4060752801 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5684045480 0x7f56840dd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f568bfff700 1 -- 172.21.15.40:0/4060752801 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5684045480 msgr2=0x7f56840dd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f568bfff700 1 --2- 172.21.15.40:0/4060752801 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5684045480 0x7f56840dd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f568bfff700 1 -- 172.21.15.40:0/4060752801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5684007c70 msgr2=0x7f56840caa50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:56:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f568bfff700 1 --2- 172.21.15.40:0/4060752801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5684007c70 0x7f56840caa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f568bfff700 1 -- 172.21.15.40:0/4060752801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c04f5c0 con 0x7f5684045a40 2024-09-13T17:56:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f568bfff700 1 --2- 172.21.15.40:0/4060752801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5684045a40 0x7f56840cd6a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f568c0641b0 tx=0x7f568c068470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f5677fff700 1 -- 172.21.15.40:0/4060752801 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c072150 con 0x7f5684045a40 2024-09-13T17:56:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f5677fff700 1 -- 172.21.15.40:0/4060752801 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568c0722b0 con 0x7f5684045a40 2024-09-13T17:56:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f5690a6e700 1 -- 172.21.15.40:0/4060752801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56840d1980 con 0x7f5684045a40 2024-09-13T17:56:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f5677fff700 1 -- 172.21.15.40:0/4060752801 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c072440 con 0x7f5684045a40 2024-09-13T17:56:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f5690a6e700 1 -- 172.21.15.40:0/4060752801 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5684175170 con 0x7f5684045a40 2024-09-13T17:56:36.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f5690a6e700 1 --2- 172.21.15.40:0/4060752801 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56840cb380 0x7f56840c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:36.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f5690a6e700 1 -- 172.21.15.40:0/4060752801 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5684008d50 con 0x7f56840cb380 2024-09-13T17:56:36.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f568b7fe700 1 --2- 172.21.15.40:0/4060752801 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56840cb380 0x7f56840c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:36.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.578+0000 7f5677fff700 1 -- 172.21.15.40:0/4060752801 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f568c0725a0 con 0x7f5684045a40 2024-09-13T17:56:36.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.582+0000 7f568b7fe700 1 -- 172.21.15.40:0/4060752801 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5684034ac0 con 0x7f56840cb380 2024-09-13T17:56:36.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.582+0000 7f568b7fe700 1 --2- 172.21.15.40:0/4060752801 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56840cb380 0x7f56840c7df0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5680003a30 tx=0x7f5680009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:36.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.582+0000 7f5677fff700 1 --2- 172.21.15.40:0/4060752801 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f56700666e0 0x7f5670068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:36.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.582+0000 7f5677fff700 1 -- 172.21.15.40:0/4060752801 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f568c0ffb70 con 0x7f5684045a40 2024-09-13T17:56:36.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.582+0000 7f568affd700 1 --2- 172.21.15.40:0/4060752801 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f56700666e0 0x7f5670068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:36.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.582+0000 7f5677fff700 1 -- 172.21.15.40:0/4060752801 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568000cb00 con 0x7f56840cb380 2024-09-13T17:56:36.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.582+0000 7f568affd700 1 --2- 172.21.15.40:0/4060752801 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f56700666e0 0x7f5670068ba0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5678009470 tx=0x7f5678009400 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:36.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.582+0000 7f5677fff700 1 -- 172.21.15.40:0/4060752801 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f568000cc60 con 0x7f56840cb380 2024-09-13T17:56:36.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.582+0000 7f5677fff700 1 -- 172.21.15.40:0/4060752801 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56840cb380 msgr2=0x7f56840c7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.582+0000 7f5677fff700 1 --2- 172.21.15.40:0/4060752801 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56840cb380 0x7f56840c7df0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5680003a30 tx=0x7f5680009f30 comp rx=0 tx=0).stop 2024-09-13T17:56:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.582+0000 7f5677fff700 1 -- 172.21.15.40:0/4060752801 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56840cb380 msgr2=0x7f56840c7df0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:56:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.582+0000 7f5677fff700 1 --2- 172.21.15.40:0/4060752801 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56840cb380 0x7f56840c7df0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.594+0000 7f5690a6e700 1 --2- 172.21.15.40:0/4060752801 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56840cdbe0 0x7f56840c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:36.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.594+0000 7f5690a6e700 1 -- 172.21.15.40:0/4060752801 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56840de990 con 0x7f56840cdbe0 2024-09-13T17:56:36.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.594+0000 7f568b7fe700 1 --2- 172.21.15.40:0/4060752801 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56840cdbe0 0x7f56840c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:36.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.594+0000 7f568b7fe700 1 -- 172.21.15.40:0/4060752801 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f56840062a0 con 0x7f56840cdbe0 2024-09-13T17:56:36.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.594+0000 7f568b7fe700 1 --2- 172.21.15.40:0/4060752801 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56840cdbe0 0x7f56840c8330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5680009fd0 tx=0x7f5680005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:36.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.594+0000 7f5677fff700 1 -- 172.21.15.40:0/4060752801 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5680005280 con 0x7f56840cdbe0 2024-09-13T17:56:36.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.594+0000 7f5677fff700 1 -- 172.21.15.40:0/4060752801 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4770 (secure 0 0 0) 0x7f56800053e0 con 0x7f56840cdbe0 2024-09-13T17:56:36.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.594+0000 7f5677fff700 1 -- 172.21.15.40:0/4060752801 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56840cdbe0 msgr2=0x7f56840c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:36.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.594+0000 7f5677fff700 1 --2- 172.21.15.40:0/4060752801 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56840cdbe0 0x7f56840c8330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5680009fd0 tx=0x7f5680005b20 comp rx=0 tx=0).stop 2024-09-13T17:56:36.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.594+0000 7f5677fff700 1 -- 172.21.15.40:0/4060752801 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56840cdbe0 msgr2=0x7f56840c8330 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:56:36.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.594+0000 7f5677fff700 1 --2- 172.21.15.40:0/4060752801 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56840cdbe0 0x7f56840c8330 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.605 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:56:36.605 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:56:36.605 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:56:36.605 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:56:36.605 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 50, 2024-09-13T17:56:36.605 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:56:36.605 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:56:36.605 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:56:36.605 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:36.605 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T17:56:36.606 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:56:36.606 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:56:36.606 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:56:36.606 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:36.606 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:36.606 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:36.606 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:36.606 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:36.606 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:36.606 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:36.606 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:36.606 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:36.606 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:36.606 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:36.607 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:56:36.607 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:56:36.607 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:36.607 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:36.607 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:36.607 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:36.607 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:36.607 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:36.607 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:36.607 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:36.607 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:36.607 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:36.607 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:36.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.608 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:56:36.608 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:56:36.608 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:56:36.608 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:56:36.608 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:56:36.608 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:56:36.608 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:56:36.608 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:56:36.608 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:56:36.608 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:56:36.608 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:56:36.608 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:56:36.609 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:56:36.609 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:56:36.609 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:56:36.609 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:56:36.609 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:56:36.609 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:56:36.609 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:56:36.609 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:56:36.609 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:56:36.609 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:56:36.609 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:56:36.609 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:56:36.609 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:56:36.609 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:56:36.610 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:56:36.610 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:36.610 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:56:36.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.610 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:56:36.610 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.610 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:56:36.610 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:56:36.610 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:36.610 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:36.610 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.610 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:56:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:56:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:56:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:56:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:56:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:56:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:56:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:56:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:56:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:56:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:56:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:56:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:56:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:56:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:56:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:56:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:56:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:56:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:56:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:56:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:56:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:56:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:56:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:56:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:56:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:56:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:56:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:56:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:56:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:56:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T17:56:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:56:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:56:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:56:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:56:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:56:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:56:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:56:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:56:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:56:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:56:36.617 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:56:36.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.594+0000 7f5690a6e700 1 -- 172.21.15.40:0/4060752801 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f56700666e0 msgr2=0x7f5670068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:36.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.594+0000 7f5690a6e700 1 --2- 172.21.15.40:0/4060752801 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f56700666e0 0x7f5670068ba0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5678009470 tx=0x7f5678009400 comp rx=0 tx=0).stop 2024-09-13T17:56:36.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.594+0000 7f5690a6e700 1 -- 172.21.15.40:0/4060752801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5684045a40 msgr2=0x7f56840cd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:36.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.594+0000 7f568bfff700 1 -- 172.21.15.40:0/4060752801 reap_dead start 2024-09-13T17:56:36.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.594+0000 7f5690a6e700 1 --2- 172.21.15.40:0/4060752801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5684045a40 0x7f56840cd6a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f568c0641b0 tx=0x7f568c068470 comp rx=0 tx=0).stop 2024-09-13T17:56:36.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.598+0000 7f5690a6e700 1 -- 172.21.15.40:0/4060752801 shutdown_connections 2024-09-13T17:56:36.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.598+0000 7f5690a6e700 1 --2- 172.21.15.40:0/4060752801 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5684045a40 0x7f56840cd6a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:36.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.598+0000 7f5690a6e700 1 -- 172.21.15.40:0/4060752801 >> 172.21.15.40:0/4060752801 conn(0x7f56840505e0 msgr2=0x7f5684006590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:36.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.598+0000 7f5690a6e700 1 -- 172.21.15.40:0/4060752801 shutdown_connections 2024-09-13T17:56:36.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:36.598+0000 7f5690a6e700 1 -- 172.21.15.40:0/4060752801 wait complete. 2024-09-13T17:56:36.620 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T17:56:36.620 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T17:56:56.620 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-13T17:56:56.621 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-13T17:56:56.621 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T17:56:56.621 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T17:56:56.624 INFO:tasks.ceph.mon.b:Started 2024-09-13T17:56:56.624 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T17:56:56.625 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:56:56.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.733+0000 7ff2b759e700 1 Processor -- start 2024-09-13T17:56:56.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.733+0000 7ff2b759e700 1 -- start start 2024-09-13T17:56:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.733+0000 7ff2b759e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff2b806ea60 0x7ff2b8070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.733+0000 7ff2b759e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff2b8071620 0x7ff2b806d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.733+0000 7ff2b759e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b814fdd0 con 0x7ff2b814f9f0 2024-09-13T17:56:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.733+0000 7ff2b759e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b8070920 con 0x7ff2b806ea60 2024-09-13T17:56:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.733+0000 7ff2b659c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff2b814f9f0 0x7ff2b8070ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57846/0 (socket says 172.21.15.40:57846) 2024-09-13T17:56:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.733+0000 7ff2b759e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b81564d0 con 0x7ff2b8071620 2024-09-13T17:56:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.733+0000 7ff2b5d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff2b806ea60 0x7ff2b8070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.733+0000 7ff2b659c700 1 -- 172.21.15.40:0/3250222498 learned_addr learned my addr 172.21.15.40:0/3250222498 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:56:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.733+0000 7ff2b6d9d700 1 --2- 172.21.15.40:0/3250222498 >> v2:172.21.15.43:3300/0 conn(0x7ff2b8071620 0x7ff2b806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:56.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b6d9d700 1 -- 172.21.15.40:0/3250222498 >> v2:172.21.15.43:3300/0 conn(0x7ff2b8071620 msgr2=0x7ff2b806d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:56:56.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b6d9d700 1 -- 172.21.15.40:0/3250222498 >> v2:172.21.15.43:3300/0 conn(0x7ff2b8071620 msgr2=0x7ff2b806d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:56:56.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b6d9d700 1 --2- 172.21.15.40:0/3250222498 >> v2:172.21.15.43:3300/0 conn(0x7ff2b8071620 0x7ff2b806d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:56:56.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b6d9d700 1 --2- 172.21.15.40:0/3250222498 >> v2:172.21.15.43:3300/0 conn(0x7ff2b8071620 0x7ff2b806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:56.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b5d9b700 1 -- 172.21.15.40:0/3250222498 >> v2:172.21.15.43:3300/0 conn(0x7ff2b8071620 msgr2=0x7ff2b806d570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:56:56.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b5d9b700 1 --2- 172.21.15.40:0/3250222498 >> v2:172.21.15.43:3300/0 conn(0x7ff2b8071620 0x7ff2b806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:56.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b5d9b700 1 -- 172.21.15.40:0/3250222498 >> v1:172.21.15.40:6789/0 conn(0x7ff2b814f9f0 legacy=0x7ff2b8070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:56.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b559a700 1 -- 172.21.15.40:0/3250222498 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4111890224 0 0) 0x7ff2b814fdd0 con 0x7ff2b814f9f0 2024-09-13T17:56:56.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b5d9b700 1 -- 172.21.15.40:0/3250222498 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b81558b0 con 0x7ff2b806ea60 2024-09-13T17:56:56.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b5d9b700 1 --2- 172.21.15.40:0/3250222498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b806ea60 0x7ff2b8070290 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80c8ee0 tx=0x7ff2ac009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117d288c99edf1b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:56.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b559a700 1 -- 172.21.15.40:0/3250222498 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac011830 con 0x7ff2b806ea60 2024-09-13T17:56:56.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b559a700 1 -- 172.21.15.40:0/3250222498 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ac011990 con 0x7ff2b806ea60 2024-09-13T17:56:56.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b559a700 1 -- 172.21.15.40:0/3250222498 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac011d40 con 0x7ff2b806ea60 2024-09-13T17:56:56.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b759e700 1 -- 172.21.15.40:0/3250222498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b806ea60 msgr2=0x7ff2b8070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:56.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b759e700 1 --2- 172.21.15.40:0/3250222498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b806ea60 0x7ff2b8070290 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80c8ee0 tx=0x7ff2ac009b30 comp rx=0 tx=0).stop 2024-09-13T17:56:56.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b759e700 1 -- 172.21.15.40:0/3250222498 shutdown_connections 2024-09-13T17:56:56.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b759e700 1 --2- 172.21.15.40:0/3250222498 >> v2:172.21.15.43:3300/0 conn(0x7ff2b8071620 0x7ff2b806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:56.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b759e700 1 --2- 172.21.15.40:0/3250222498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b806ea60 0x7ff2b8070290 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:56.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b759e700 1 -- 172.21.15.40:0/3250222498 >> 172.21.15.40:0/3250222498 conn(0x7ff2b806a390 msgr2=0x7ff2b806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:56.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b759e700 1 -- 172.21.15.40:0/3250222498 shutdown_connections 2024-09-13T17:56:56.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.737+0000 7ff2b759e700 1 -- 172.21.15.40:0/3250222498 wait complete. 2024-09-13T17:56:56.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b759e700 1 Processor -- start 2024-09-13T17:56:56.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b759e700 1 -- start start 2024-09-13T17:56:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2b806ea60 0x7ff2b814b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b8071620 0x7ff2b8154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff2b814f9f0 0x7ff2b8154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b80713b0 con 0x7ff2b8071620 2024-09-13T17:56:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b814b560 con 0x7ff2b806ea60 2024-09-13T17:56:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b8150e00 con 0x7ff2b814f9f0 2024-09-13T17:56:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b5d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b8071620 0x7ff2b8154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2b806ea60 0x7ff2b814b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b5d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b8071620 0x7ff2b8154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38056/0 (socket says 172.21.15.40:38056) 2024-09-13T17:56:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b5d9b700 1 -- 172.21.15.40:0/3210868590 learned_addr learned my addr 172.21.15.40:0/3210868590 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:56:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b6d9d700 1 --2- 172.21.15.40:0/3210868590 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff2b814f9f0 0x7ff2b8154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:56.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b659c700 1 -- 172.21.15.40:0/3210868590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2b806ea60 msgr2=0x7ff2b814b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T17:56:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b659c700 1 -- 172.21.15.40:0/3210868590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2b806ea60 msgr2=0x7ff2b814b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:56:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b659c700 1 --2- 172.21.15.40:0/3210868590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2b806ea60 0x7ff2b814b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:56:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b659c700 1 --2- 172.21.15.40:0/3210868590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2b806ea60 0x7ff2b814b020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:56:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b5d9b700 1 -- 172.21.15.40:0/3210868590 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff2b814f9f0 msgr2=0x7ff2b8154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b5d9b700 1 --2- 172.21.15.40:0/3210868590 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff2b814f9f0 0x7ff2b8154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b5d9b700 1 -- 172.21.15.40:0/3210868590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2b806ea60 msgr2=0x7ff2b814b020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:56:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b5d9b700 1 --2- 172.21.15.40:0/3210868590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2b806ea60 0x7ff2b814b020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b5d9b700 1 -- 172.21.15.40:0/3210868590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ac0094b0 con 0x7ff2b8071620 2024-09-13T17:56:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b5d9b700 1 --2- 172.21.15.40:0/3210868590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b8071620 0x7ff2b8154580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac000c00 tx=0x7ff2ac004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2a6ffd700 1 -- 172.21.15.40:0/3210868590 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac01aa10 con 0x7ff2b8071620 2024-09-13T17:56:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2a6ffd700 1 -- 172.21.15.40:0/3210868590 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ac01f070 con 0x7ff2b8071620 2024-09-13T17:56:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b759e700 1 -- 172.21.15.40:0/3210868590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b81ea8b0 con 0x7ff2b8071620 2024-09-13T17:56:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2a6ffd700 1 -- 172.21.15.40:0/3210868590 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac01add0 con 0x7ff2b8071620 2024-09-13T17:56:56.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.741+0000 7ff2b759e700 1 -- 172.21.15.40:0/3210868590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b81eae70 con 0x7ff2b8071620 2024-09-13T17:56:56.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.745+0000 7ff2b759e700 1 -- 172.21.15.40:0/3210868590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b8070290 con 0x7ff2b8071620 2024-09-13T17:56:56.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.745+0000 7ff2a6ffd700 1 -- 172.21.15.40:0/3210868590 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff2ac01b430 con 0x7ff2b8071620 2024-09-13T17:56:56.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.745+0000 7ff2a6ffd700 1 --2- 172.21.15.40:0/3210868590 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff298066760 0x7ff298068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:56:56.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.745+0000 7ff2a6ffd700 1 -- 172.21.15.40:0/3210868590 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff2ac085620 con 0x7ff2b8071620 2024-09-13T17:56:56.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.745+0000 7ff2b659c700 1 --2- 172.21.15.40:0/3210868590 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff298066760 0x7ff298068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:56:56.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.745+0000 7ff2b659c700 1 --2- 172.21.15.40:0/3210868590 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff298066760 0x7ff298068c20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8001030 tx=0x7ff2a8009f30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:56:56.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:56.753+0000 7ff2a6ffd700 1 -- 172.21.15.40:0/3210868590 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff2ac0525d0 con 0x7ff2b8071620 2024-09-13T17:56:57.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:57.001+0000 7ff2b759e700 1 -- 172.21.15.40:0/3210868590 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff2b80c6ab0 con 0x7ff2b8071620 2024-09-13T17:56:57.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:57.001+0000 7ff2a6ffd700 1 -- 172.21.15.40:0/3210868590 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7ff2ac021610 con 0x7ff2b8071620 2024-09-13T17:56:57.013 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":50,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:56:57.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:57.005+0000 7ff2b759e700 1 -- 172.21.15.40:0/3210868590 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff298066760 msgr2=0x7ff298068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:57.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:57.005+0000 7ff2b759e700 1 --2- 172.21.15.40:0/3210868590 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff298066760 0x7ff298068c20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8001030 tx=0x7ff2a8009f30 comp rx=0 tx=0).stop 2024-09-13T17:56:57.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:57.005+0000 7ff2b759e700 1 -- 172.21.15.40:0/3210868590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b8071620 msgr2=0x7ff2b8154580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:56:57.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:57.005+0000 7ff2b759e700 1 --2- 172.21.15.40:0/3210868590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b8071620 0x7ff2b8154580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac000c00 tx=0x7ff2ac004770 comp rx=0 tx=0).stop 2024-09-13T17:56:57.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:57.005+0000 7ff2b759e700 1 -- 172.21.15.40:0/3210868590 shutdown_connections 2024-09-13T17:56:57.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:57.005+0000 7ff2b759e700 1 --2- 172.21.15.40:0/3210868590 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff298066760 0x7ff298068c20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:57.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:57.005+0000 7ff2b759e700 1 --2- 172.21.15.40:0/3210868590 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff2b814f9f0 0x7ff2b8154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:57.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:57.005+0000 7ff2b759e700 1 --2- 172.21.15.40:0/3210868590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2b8071620 0x7ff2b8154580 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:57.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:57.005+0000 7ff2b759e700 1 --2- 172.21.15.40:0/3210868590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2b806ea60 0x7ff2b814b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:56:57.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:57.005+0000 7ff2b759e700 1 -- 172.21.15.40:0/3210868590 >> 172.21.15.40:0/3210868590 conn(0x7ff2b806a390 msgr2=0x7ff2b80c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:56:57.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:57.005+0000 7ff2b759e700 1 -- 172.21.15.40:0/3210868590 shutdown_connections 2024-09-13T17:56:57.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:56:57.005+0000 7ff2b759e700 1 -- 172.21.15.40:0/3210868590 wait complete. 2024-09-13T17:57:00.029 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:57:00.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.141+0000 7f12bbfff700 1 Processor -- start 2024-09-13T17:57:00.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.141+0000 7f12bbfff700 1 -- start start 2024-09-13T17:57:00.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12bbfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f12b4050820 0x7f12b4035240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:00.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12bbfff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b40d5a20 con 0x7f12b4045e10 2024-09-13T17:57:00.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12bbfff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b40dbe40 con 0x7f12b40d5640 2024-09-13T17:57:00.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12bbfff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b40dc120 con 0x7f12b4050820 2024-09-13T17:57:00.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12bb7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f12b4050820 0x7f12b4035240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:00.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12baffd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f12b40d5640 0x7f12b4045700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:38602/0 (socket says 172.21.15.40:38602) 2024-09-13T17:57:00.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12baffd700 1 -- 172.21.15.40:0/4020542167 learned_addr learned my addr 172.21.15.40:0/4020542167 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:57:00.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12ba7fc700 1 --1- 172.21.15.40:0/4020542167 >> v1:172.21.15.40:6789/0 conn(0x7f12b4045e10 0x7f12b4034b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57850/0 (socket says 172.21.15.40:57850) 2024-09-13T17:57:00.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12bb7fe700 1 -- 172.21.15.40:0/4020542167 >> v1:172.21.15.43:6789/0 conn(0x7f12b40d5640 legacy=0x7f12b4045700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:00.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12bb7fe700 1 -- 172.21.15.40:0/4020542167 >> v1:172.21.15.40:6789/0 conn(0x7f12b4045e10 legacy=0x7f12b4034b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:00.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12bb7fe700 1 -- 172.21.15.40:0/4020542167 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b4005b00 con 0x7f12b4050820 2024-09-13T17:57:00.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12bb7fe700 1 --2- 172.21.15.40:0/4020542167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12b4050820 0x7f12b4035240 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f12bc069620 tx=0x7f12bc069e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250f6e9c9c10b80d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:00.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12b9ffb700 1 -- 172.21.15.40:0/4020542167 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc067fd0 con 0x7f12b4050820 2024-09-13T17:57:00.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12b9ffb700 1 -- 172.21.15.40:0/4020542167 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12bc068a70 con 0x7f12b4050820 2024-09-13T17:57:00.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12b9ffb700 1 -- 172.21.15.40:0/4020542167 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc074700 con 0x7f12b4050820 2024-09-13T17:57:00.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12bbfff700 1 -- 172.21.15.40:0/4020542167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12b4050820 msgr2=0x7f12b4035240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:00.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12bbfff700 1 --2- 172.21.15.40:0/4020542167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12b4050820 0x7f12b4035240 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f12bc069620 tx=0x7f12bc069e70 comp rx=0 tx=0).stop 2024-09-13T17:57:00.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12bbfff700 1 -- 172.21.15.40:0/4020542167 shutdown_connections 2024-09-13T17:57:00.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12bbfff700 1 --2- 172.21.15.40:0/4020542167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12b4050820 0x7f12b4035240 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:00.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12bbfff700 1 -- 172.21.15.40:0/4020542167 >> 172.21.15.40:0/4020542167 conn(0x7f12b404fd00 msgr2=0x7f12b4050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:00.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12bbfff700 1 -- 172.21.15.40:0/4020542167 shutdown_connections 2024-09-13T17:57:00.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.145+0000 7f12bbfff700 1 -- 172.21.15.40:0/4020542167 wait complete. 2024-09-13T17:57:00.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12bbfff700 1 Processor -- start 2024-09-13T17:57:00.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12bbfff700 1 -- start start 2024-09-13T17:57:00.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12bbfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12b4045e10 0x7f12b4170980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:00.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12bbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12b4050820 0x7f12b4173f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:00.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12bbfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f12b40d5640 0x7f12b4166b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:00.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12bbfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b4171970 con 0x7f12b4050820 2024-09-13T17:57:00.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12bbfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b40d1530 con 0x7f12b4045e10 2024-09-13T17:57:00.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12bbfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b4174570 con 0x7f12b40d5640 2024-09-13T17:57:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12ba7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12b4050820 0x7f12b4173f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12ba7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12b4050820 0x7f12b4173f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38076/0 (socket says 172.21.15.40:38076) 2024-09-13T17:57:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12ba7fc700 1 -- 172.21.15.40:0/1035850873 learned_addr learned my addr 172.21.15.40:0/1035850873 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12baffd700 1 --2- 172.21.15.40:0/1035850873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12b4045e10 0x7f12b4170980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12bb7fe700 1 --2- 172.21.15.40:0/1035850873 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f12b40d5640 0x7f12b4166b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12baffd700 1 -- 172.21.15.40:0/1035850873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12b4045e10 msgr2=0x7f12b4170980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T17:57:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12baffd700 1 -- 172.21.15.40:0/1035850873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12b4045e10 msgr2=0x7f12b4170980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:57:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12baffd700 1 --2- 172.21.15.40:0/1035850873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12b4045e10 0x7f12b4170980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:57:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12baffd700 1 --2- 172.21.15.40:0/1035850873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12b4045e10 0x7f12b4170980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:57:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12ba7fc700 1 -- 172.21.15.40:0/1035850873 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f12b40d5640 msgr2=0x7f12b4166b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12ba7fc700 1 --2- 172.21.15.40:0/1035850873 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f12b40d5640 0x7f12b4166b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12ba7fc700 1 -- 172.21.15.40:0/1035850873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12b4045e10 msgr2=0x7f12b4170980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:57:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12ba7fc700 1 --2- 172.21.15.40:0/1035850873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12b4045e10 0x7f12b4170980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12ba7fc700 1 -- 172.21.15.40:0/1035850873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12bc04f5c0 con 0x7f12b4050820 2024-09-13T17:57:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.149+0000 7f12bb7fe700 1 --2- 172.21.15.40:0/1035850873 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f12b40d5640 0x7f12b4166b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.153+0000 7f12ba7fc700 1 --2- 172.21.15.40:0/1035850873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12b4050820 0x7f12b4173f30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f12b000c4d0 tx=0x7f12b000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.153+0000 7f12a37fe700 1 -- 172.21.15.40:0/1035850873 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0005cd0 con 0x7f12b4050820 2024-09-13T17:57:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.153+0000 7f12a37fe700 1 -- 172.21.15.40:0/1035850873 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b0005e30 con 0x7f12b4050820 2024-09-13T17:57:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.153+0000 7f12bbfff700 1 -- 172.21.15.40:0/1035850873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b4174850 con 0x7f12b4050820 2024-09-13T17:57:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.153+0000 7f12a37fe700 1 -- 172.21.15.40:0/1035850873 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b00072d0 con 0x7f12b4050820 2024-09-13T17:57:00.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.153+0000 7f12bbfff700 1 -- 172.21.15.40:0/1035850873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b40d11c0 con 0x7f12b4050820 2024-09-13T17:57:00.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.153+0000 7f12bbfff700 1 -- 172.21.15.40:0/1035850873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b4050590 con 0x7f12b4050820 2024-09-13T17:57:00.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.153+0000 7f12a37fe700 1 -- 172.21.15.40:0/1035850873 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f12b0007430 con 0x7f12b4050820 2024-09-13T17:57:00.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.153+0000 7f12a37fe700 1 --2- 172.21.15.40:0/1035850873 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f12a8066760 0x7f12a8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:00.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.153+0000 7f12a37fe700 1 -- 172.21.15.40:0/1035850873 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f12b0083910 con 0x7f12b4050820 2024-09-13T17:57:00.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.153+0000 7f12baffd700 1 --2- 172.21.15.40:0/1035850873 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f12a8066760 0x7f12a8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:00.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.153+0000 7f12baffd700 1 --2- 172.21.15.40:0/1035850873 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f12a8066760 0x7f12a8068c20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f12ac005930 tx=0x7f12ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:00.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.161+0000 7f12a37fe700 1 -- 172.21.15.40:0/1035850873 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f12b00508e0 con 0x7f12b4050820 2024-09-13T17:57:00.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.429+0000 7f12bbfff700 1 -- 172.21.15.40:0/1035850873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f12b4035c80 con 0x7f12b4050820 2024-09-13T17:57:00.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.429+0000 7f12a37fe700 1 -- 172.21.15.40:0/1035850873 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f12b0017020 con 0x7f12b4050820 2024-09-13T17:57:00.439 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":50,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:57:00.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.433+0000 7f12bbfff700 1 -- 172.21.15.40:0/1035850873 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f12a8066760 msgr2=0x7f12a8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:00.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.433+0000 7f12bbfff700 1 --2- 172.21.15.40:0/1035850873 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f12a8066760 0x7f12a8068c20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f12ac005930 tx=0x7f12ac009040 comp rx=0 tx=0).stop 2024-09-13T17:57:00.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.433+0000 7f12bbfff700 1 -- 172.21.15.40:0/1035850873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12b4050820 msgr2=0x7f12b4173f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:00.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.433+0000 7f12bbfff700 1 --2- 172.21.15.40:0/1035850873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12b4050820 0x7f12b4173f30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f12b000c4d0 tx=0x7f12b000c890 comp rx=0 tx=0).stop 2024-09-13T17:57:00.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.433+0000 7f12bbfff700 1 -- 172.21.15.40:0/1035850873 shutdown_connections 2024-09-13T17:57:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.433+0000 7f12bbfff700 1 --2- 172.21.15.40:0/1035850873 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f12a8066760 0x7f12a8068c20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.433+0000 7f12bbfff700 1 --2- 172.21.15.40:0/1035850873 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f12b40d5640 0x7f12b4166b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.433+0000 7f12bbfff700 1 --2- 172.21.15.40:0/1035850873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12b4050820 0x7f12b4173f30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.433+0000 7f12bbfff700 1 --2- 172.21.15.40:0/1035850873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12b4045e10 0x7f12b4170980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.433+0000 7f12bbfff700 1 -- 172.21.15.40:0/1035850873 >> 172.21.15.40:0/1035850873 conn(0x7f12b404fd00 msgr2=0x7f12b404c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.433+0000 7f12bbfff700 1 -- 172.21.15.40:0/1035850873 shutdown_connections 2024-09-13T17:57:00.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:00.433+0000 7f12bbfff700 1 -- 172.21.15.40:0/1035850873 wait complete. 2024-09-13T17:57:03.456 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:57:03.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe6397700 1 Processor -- start 2024-09-13T17:57:03.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe6397700 1 -- start start 2024-09-13T17:57:03.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe6397700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7fe0153880 0x7f7fe00c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe6397700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7fe00cb2f0 0x7f7fe00cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe6397700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe0159770 con 0x7f7fe00cf3c0 2024-09-13T17:57:03.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe6397700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe0159a50 con 0x7f7fe00cb2f0 2024-09-13T17:57:03.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe6397700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe0159d30 con 0x7f7fe0153880 2024-09-13T17:57:03.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe4b94700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7fe00cb2f0 0x7f7fe00cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe4b94700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7fe00cb2f0 0x7f7fe00cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59620/0 (socket says 172.21.15.40:59620) 2024-09-13T17:57:03.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe4b94700 1 -- 172.21.15.40:0/4034885615 learned_addr learned my addr 172.21.15.40:0/4034885615 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:03.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe5b96700 1 --1- 172.21.15.40:0/4034885615 >> v1:172.21.15.43:6789/0 conn(0x7f7fe00cf3c0 0x7f7fe015cf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:55276/0 (socket says 172.21.15.40:55276) 2024-09-13T17:57:03.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe5395700 1 --2- 172.21.15.40:0/4034885615 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7fe0153880 0x7f7fe00c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe4b94700 1 -- 172.21.15.40:0/4034885615 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7fe0153880 msgr2=0x7f7fe00c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:03.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fd7fff700 1 -- 172.21.15.40:0/4034885615 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078442498 0 0) 0x7f7fe0159770 con 0x7f7fe00cf3c0 2024-09-13T17:57:03.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe4b94700 1 --2- 172.21.15.40:0/4034885615 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7fe0153880 0x7f7fe00c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe4b94700 1 -- 172.21.15.40:0/4034885615 >> v1:172.21.15.43:6789/0 conn(0x7f7fe00cf3c0 legacy=0x7f7fe015cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe4b94700 1 -- 172.21.15.40:0/4034885615 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe015a010 con 0x7f7fe00cb2f0 2024-09-13T17:57:03.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe4b94700 1 --2- 172.21.15.40:0/4034885615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe00cb2f0 0x7f7fe00cee80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009910 tx=0x7f7fd0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b99bf274742e64b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:03.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fd7fff700 1 -- 172.21.15.40:0/4034885615 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0011830 con 0x7f7fe00cb2f0 2024-09-13T17:57:03.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fd7fff700 1 -- 172.21.15.40:0/4034885615 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd0011990 con 0x7f7fe00cb2f0 2024-09-13T17:57:03.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fd7fff700 1 -- 172.21.15.40:0/4034885615 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0011d40 con 0x7f7fe00cb2f0 2024-09-13T17:57:03.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe6397700 1 -- 172.21.15.40:0/4034885615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe00cb2f0 msgr2=0x7f7fe00cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe6397700 1 --2- 172.21.15.40:0/4034885615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe00cb2f0 0x7f7fe00cee80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009910 tx=0x7f7fd0009c20 comp rx=0 tx=0).stop 2024-09-13T17:57:03.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe6397700 1 -- 172.21.15.40:0/4034885615 shutdown_connections 2024-09-13T17:57:03.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe6397700 1 --2- 172.21.15.40:0/4034885615 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe00cb2f0 0x7f7fe00cee80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe6397700 1 --2- 172.21.15.40:0/4034885615 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7fe0153880 0x7f7fe00c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe6397700 1 -- 172.21.15.40:0/4034885615 >> 172.21.15.40:0/4034885615 conn(0x7f7fe006c5b0 msgr2=0x7f7fe006c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:03.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe6397700 1 -- 172.21.15.40:0/4034885615 shutdown_connections 2024-09-13T17:57:03.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.565+0000 7f7fe6397700 1 -- 172.21.15.40:0/4034885615 wait complete. 2024-09-13T17:57:03.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.569+0000 7f7fe6397700 1 Processor -- start 2024-09-13T17:57:03.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.569+0000 7f7fe6397700 1 -- start start 2024-09-13T17:57:03.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.569+0000 7f7fe6397700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fe00cb2f0 0x7f7fe0143c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.569+0000 7f7fe6397700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe00cf3c0 0x7f7fe0142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.569+0000 7f7fe6397700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7fe0153880 0x7f7fe01434e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.569+0000 7f7fe6397700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe01478e0 con 0x7f7fe00cf3c0 2024-09-13T17:57:03.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.569+0000 7f7fe6397700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe0141510 con 0x7f7fe00cb2f0 2024-09-13T17:57:03.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.569+0000 7f7fe6397700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe0141960 con 0x7f7fe0153880 2024-09-13T17:57:03.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.569+0000 7f7fe4b94700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe00cf3c0 0x7f7fe0142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.569+0000 7f7fe4b94700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe00cf3c0 0x7f7fe0142fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59636/0 (socket says 172.21.15.40:59636) 2024-09-13T17:57:03.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.569+0000 7f7fe4b94700 1 -- 172.21.15.40:0/1523714919 learned_addr learned my addr 172.21.15.40:0/1523714919 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:03.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.569+0000 7f7fe5395700 1 --2- 172.21.15.40:0/1523714919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fe00cb2f0 0x7f7fe0143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.569+0000 7f7fe5b96700 1 --2- 172.21.15.40:0/1523714919 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7fe0153880 0x7f7fe01434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.569+0000 7f7fe4b94700 1 -- 172.21.15.40:0/1523714919 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7fe0153880 msgr2=0x7f7fe01434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.569+0000 7f7fe4b94700 1 --2- 172.21.15.40:0/1523714919 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7fe0153880 0x7f7fe01434e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.569+0000 7f7fe4b94700 1 -- 172.21.15.40:0/1523714919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fe00cb2f0 msgr2=0x7f7fe0143c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.569+0000 7f7fe4b94700 1 --2- 172.21.15.40:0/1523714919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fe00cb2f0 0x7f7fe0143c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.569+0000 7f7fe4b94700 1 -- 172.21.15.40:0/1523714919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd0009570 con 0x7f7fe00cf3c0 2024-09-13T17:57:03.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.569+0000 7f7fe5b96700 1 --2- 172.21.15.40:0/1523714919 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7fe0153880 0x7f7fe01434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:57:03.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.573+0000 7f7fe4b94700 1 --2- 172.21.15.40:0/1523714919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe00cf3c0 0x7f7fe0142fa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0013000 tx=0x7f7fd0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:03.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.573+0000 7f7fd5ffb700 1 -- 172.21.15.40:0/1523714919 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd000b9f0 con 0x7f7fe00cf3c0 2024-09-13T17:57:03.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.573+0000 7f7fe6397700 1 -- 172.21.15.40:0/1523714919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe015c1f0 con 0x7f7fe00cf3c0 2024-09-13T17:57:03.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.573+0000 7f7fd5ffb700 1 -- 172.21.15.40:0/1523714919 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd001f070 con 0x7f7fe00cf3c0 2024-09-13T17:57:03.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.573+0000 7f7fd5ffb700 1 -- 172.21.15.40:0/1523714919 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd000bd70 con 0x7f7fe00cf3c0 2024-09-13T17:57:03.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.573+0000 7f7fe6397700 1 -- 172.21.15.40:0/1523714919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe015c830 con 0x7f7fe00cf3c0 2024-09-13T17:57:03.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.573+0000 7f7fe6397700 1 -- 172.21.15.40:0/1523714919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe00cee80 con 0x7f7fe00cf3c0 2024-09-13T17:57:03.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.573+0000 7f7fd5ffb700 1 -- 172.21.15.40:0/1523714919 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7fd001a430 con 0x7f7fe00cf3c0 2024-09-13T17:57:03.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.573+0000 7f7fd5ffb700 1 --2- 172.21.15.40:0/1523714919 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7fc4066760 0x7f7fc4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.573+0000 7f7fd5ffb700 1 -- 172.21.15.40:0/1523714919 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7fd0021070 con 0x7f7fe00cf3c0 2024-09-13T17:57:03.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.573+0000 7f7fe5395700 1 --2- 172.21.15.40:0/1523714919 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7fc4066760 0x7f7fc4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.577+0000 7f7fe5395700 1 --2- 172.21.15.40:0/1523714919 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7fc4066760 0x7f7fc4068c20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009b50 tx=0x7f7fdc009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:03.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.581+0000 7f7fd5ffb700 1 -- 172.21.15.40:0/1523714919 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7fd0052ba0 con 0x7f7fe00cf3c0 2024-09-13T17:57:03.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.845+0000 7f7fe6397700 1 -- 172.21.15.40:0/1523714919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f7fe00cf910 con 0x7f7fe00cf3c0 2024-09-13T17:57:03.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.845+0000 7f7fd5ffb700 1 -- 172.21.15.40:0/1523714919 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7f7fd00561c0 con 0x7f7fe00cf3c0 2024-09-13T17:57:03.856 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":52,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:57:03.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.849+0000 7f7fe6397700 1 -- 172.21.15.40:0/1523714919 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7fc4066760 msgr2=0x7f7fc4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.849+0000 7f7fe6397700 1 --2- 172.21.15.40:0/1523714919 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7fc4066760 0x7f7fc4068c20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009b50 tx=0x7f7fdc009240 comp rx=0 tx=0).stop 2024-09-13T17:57:03.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.849+0000 7f7fe6397700 1 -- 172.21.15.40:0/1523714919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe00cf3c0 msgr2=0x7f7fe0142fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.849+0000 7f7fe6397700 1 --2- 172.21.15.40:0/1523714919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe00cf3c0 0x7f7fe0142fa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0013000 tx=0x7f7fd0004770 comp rx=0 tx=0).stop 2024-09-13T17:57:03.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.849+0000 7f7fe6397700 1 -- 172.21.15.40:0/1523714919 shutdown_connections 2024-09-13T17:57:03.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.849+0000 7f7fe6397700 1 --2- 172.21.15.40:0/1523714919 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7fc4066760 0x7f7fc4068c20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.849+0000 7f7fe6397700 1 --2- 172.21.15.40:0/1523714919 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7fe0153880 0x7f7fe01434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.849+0000 7f7fe6397700 1 --2- 172.21.15.40:0/1523714919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7fe00cf3c0 0x7f7fe0142fa0 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0013000 tx=0x7f7fd0004770 comp rx=0 tx=0).stop 2024-09-13T17:57:03.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.849+0000 7f7fe6397700 1 --2- 172.21.15.40:0/1523714919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fe00cb2f0 0x7f7fe0143c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.849+0000 7f7fe6397700 1 -- 172.21.15.40:0/1523714919 >> 172.21.15.40:0/1523714919 conn(0x7f7fe006c5b0 msgr2=0x7f7fe0152920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:03.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.849+0000 7f7fe6397700 1 -- 172.21.15.40:0/1523714919 shutdown_connections 2024-09-13T17:57:03.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.849+0000 7f7fe6397700 1 -- 172.21.15.40:0/1523714919 wait complete. 2024-09-13T17:57:03.872 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T17:57:03.872 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T17:57:03.872 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:57:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.973+0000 7f02ec971700 1 Processor -- start 2024-09-13T17:57:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.973+0000 7f02ec971700 1 -- start start 2024-09-13T17:57:03.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.973+0000 7f02ec971700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02e80cc620 0x7f02e8153520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.973+0000 7f02ec971700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e8159780 con 0x7f02e815d620 2024-09-13T17:57:03.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.973+0000 7f02ec971700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e8159a40 con 0x7f02e80cc240 2024-09-13T17:57:03.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.973+0000 7f02ec971700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e8159d20 con 0x7f02e80cc620 2024-09-13T17:57:03.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.973+0000 7f02e6d9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f02e815d620 0x7f02e80c3fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47126/0 (socket says 172.21.15.40:47126) 2024-09-13T17:57:03.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.973+0000 7f02e6d9d700 1 -- 172.21.15.40:0/2947565389 learned_addr learned my addr 172.21.15.40:0/2947565389 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:57:03.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.973+0000 7f02dffff700 1 --2- 172.21.15.40:0/2947565389 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02e80cc620 0x7f02e8153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.973+0000 7f02e5d9b700 1 -- 172.21.15.40:0/2947565389 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718282747 0 0) 0x7f02e8159780 con 0x7f02e815d620 2024-09-13T17:57:03.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.973+0000 7f02e5d9b700 1 -- 172.21.15.40:0/2947565389 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02c80035d0 con 0x7f02e815d620 2024-09-13T17:57:03.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.973+0000 7f02e5d9b700 1 -- 172.21.15.40:0/2947565389 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3429303395 0 0) 0x7f02e8159a40 con 0x7f02e80cc240 2024-09-13T17:57:03.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.973+0000 7f02e5d9b700 1 -- 172.21.15.40:0/2947565389 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02e8159780 con 0x7f02e80cc240 2024-09-13T17:57:03.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.973+0000 7f02e5d9b700 1 -- 172.21.15.40:0/2947565389 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3839237216 0 0) 0x7f02c80035d0 con 0x7f02e815d620 2024-09-13T17:57:03.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.973+0000 7f02dffff700 1 -- 172.21.15.40:0/2947565389 >> v1:172.21.15.43:6789/0 conn(0x7f02e80cc240 legacy=0x7f02e815cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.973+0000 7f02dffff700 1 -- 172.21.15.40:0/2947565389 >> v1:172.21.15.40:6789/0 conn(0x7f02e815d620 legacy=0x7f02e80c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.973+0000 7f02dffff700 1 -- 172.21.15.40:0/2947565389 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e815a000 con 0x7f02e80cc620 2024-09-13T17:57:03.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.973+0000 7f02dffff700 1 --2- 172.21.15.40:0/2947565389 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02e80cc620 0x7f02e8153520 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f02d00098b0 tx=0x7f02d0009bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=537624b914f2a4af server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:03.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.977+0000 7f02e5d9b700 1 -- 172.21.15.40:0/2947565389 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e00019f0 con 0x7f02e80cc620 2024-09-13T17:57:03.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.977+0000 7f02e5d9b700 1 -- 172.21.15.40:0/2947565389 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d0004030 con 0x7f02e80cc620 2024-09-13T17:57:03.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.977+0000 7f02e5d9b700 1 -- 172.21.15.40:0/2947565389 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d00043a0 con 0x7f02e80cc620 2024-09-13T17:57:03.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.977+0000 7f02ec971700 1 -- 172.21.15.40:0/2947565389 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02e80cc620 msgr2=0x7f02e8153520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.977+0000 7f02ec971700 1 --2- 172.21.15.40:0/2947565389 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02e80cc620 0x7f02e8153520 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f02d00098b0 tx=0x7f02d0009bc0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.977+0000 7f02ec971700 1 -- 172.21.15.40:0/2947565389 shutdown_connections 2024-09-13T17:57:03.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.977+0000 7f02ec971700 1 --2- 172.21.15.40:0/2947565389 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02e80cc620 0x7f02e8153520 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.977+0000 7f02ec971700 1 -- 172.21.15.40:0/2947565389 >> 172.21.15.40:0/2947565389 conn(0x7f02e8066480 msgr2=0x7f02e8066890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:03.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.977+0000 7f02ec971700 1 -- 172.21.15.40:0/2947565389 shutdown_connections 2024-09-13T17:57:03.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.977+0000 7f02ec971700 1 -- 172.21.15.40:0/2947565389 wait complete. 2024-09-13T17:57:03.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02ec971700 1 Processor -- start 2024-09-13T17:57:03.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02ec971700 1 -- start start 2024-09-13T17:57:03.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02ec971700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02e80cc240 0x7f02e814ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02ec971700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02e80cc620 0x7f02e80ced40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02ec971700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02e815d620 0x7f02e80cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02ec971700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e814dfb0 con 0x7f02e80cc620 2024-09-13T17:57:03.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02ec971700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e814ebf0 con 0x7f02e80cc240 2024-09-13T17:57:03.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02ec971700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e814f320 con 0x7f02e815d620 2024-09-13T17:57:03.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02dffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02e80cc620 0x7f02e80ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02dffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02e80cc620 0x7f02e80ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59650/0 (socket says 172.21.15.40:59650) 2024-09-13T17:57:03.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02dffff700 1 -- 172.21.15.40:0/4155515738 learned_addr learned my addr 172.21.15.40:0/4155515738 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:03.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02e6d9d700 1 --2- 172.21.15.40:0/4155515738 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02e815d620 0x7f02e80cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02e659c700 1 --2- 172.21.15.40:0/4155515738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02e80cc240 0x7f02e814ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02dffff700 1 -- 172.21.15.40:0/4155515738 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02e815d620 msgr2=0x7f02e80cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02dffff700 1 --2- 172.21.15.40:0/4155515738 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02e815d620 0x7f02e80cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02dffff700 1 -- 172.21.15.40:0/4155515738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02e80cc240 msgr2=0x7f02e814ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02dffff700 1 --2- 172.21.15.40:0/4155515738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02e80cc240 0x7f02e814ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:03.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02dffff700 1 -- 172.21.15.40:0/4155515738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d00094b0 con 0x7f02e80cc620 2024-09-13T17:57:03.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02dffff700 1 --2- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02e80cc620 0x7f02e80ced40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f02d0004000 tx=0x7f02d0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:03.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02deffd700 1 -- 172.21.15.40:0/4155515738 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0003b70 con 0x7f02e80cc620 2024-09-13T17:57:03.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02deffd700 1 -- 172.21.15.40:0/4155515738 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d001f070 con 0x7f02e80cc620 2024-09-13T17:57:03.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02ec971700 1 -- 172.21.15.40:0/4155515738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e80ce3d0 con 0x7f02e80cc620 2024-09-13T17:57:03.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02deffd700 1 -- 172.21.15.40:0/4155515738 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0011630 con 0x7f02e80cc620 2024-09-13T17:57:03.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.981+0000 7f02ec971700 1 -- 172.21.15.40:0/4155515738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e80cd2a0 con 0x7f02e80cc620 2024-09-13T17:57:03.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.985+0000 7f02ec971700 1 --2- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c0002a40 0x7f02c0004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.985+0000 7f02ec971700 1 -- 172.21.15.40:0/4155515738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c0007f40 con 0x7f02c0002a40 2024-09-13T17:57:03.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.985+0000 7f02e659c700 1 --2- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c0002a40 0x7f02c0004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.985+0000 7f02e659c700 1 -- 172.21.15.40:0/4155515738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02c0008100 con 0x7f02c0002a40 2024-09-13T17:57:03.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.985+0000 7f02deffd700 1 -- 172.21.15.40:0/4155515738 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f02d0021030 con 0x7f02e80cc620 2024-09-13T17:57:03.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.985+0000 7f02e659c700 1 --2- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c0002a40 0x7f02c0004f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f02d800f770 tx=0x7f02d800fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:03.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.985+0000 7f02deffd700 1 --2- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f02cc066730 0x7f02cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:03.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.985+0000 7f02deffd700 1 -- 172.21.15.40:0/4155515738 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f02d00857f0 con 0x7f02e80cc620 2024-09-13T17:57:03.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.985+0000 7f02e6d9d700 1 --2- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f02cc066730 0x7f02cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:03.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.985+0000 7f02deffd700 1 -- 172.21.15.40:0/4155515738 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d8005320 con 0x7f02c0002a40 2024-09-13T17:57:03.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.985+0000 7f02deffd700 1 -- 172.21.15.40:0/4155515738 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f02d8005480 con 0x7f02c0002a40 2024-09-13T17:57:03.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.985+0000 7f02deffd700 1 -- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c0002a40 msgr2=0x7f02c0004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:03.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.985+0000 7f02deffd700 1 --2- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c0002a40 0x7f02c0004f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f02d800f770 tx=0x7f02d800fb30 comp rx=0 tx=0).stop 2024-09-13T17:57:03.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.985+0000 7f02e6d9d700 1 --2- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f02cc066730 0x7f02cc068bf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f02e0001560 tx=0x7f02e0001970 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:03.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.985+0000 7f02deffd700 1 -- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c0002a40 msgr2=0x7f02c0004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:57:03.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.985+0000 7f02deffd700 1 --2- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c0002a40 0x7f02c0004f00 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f02d800f770 tx=0x7f02d800fb30 comp rx=0 tx=0).stop 2024-09-13T17:57:04.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.997+0000 7f02ec971700 1 --2- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c0006ef0 0x7f02c0007360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.997+0000 7f02ec971700 1 -- 172.21.15.40:0/4155515738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c000a4a0 con 0x7f02c0006ef0 2024-09-13T17:57:04.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.997+0000 7f02e659c700 1 --2- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c0006ef0 0x7f02c0007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.997+0000 7f02e659c700 1 -- 172.21.15.40:0/4155515738 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f02c00078a0 con 0x7f02c0006ef0 2024-09-13T17:57:04.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.997+0000 7f02e659c700 1 --2- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c0006ef0 0x7f02c0007360 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f02d8016000 tx=0x7f02d8001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:04.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:03.997+0000 7f02deffd700 1 -- 172.21.15.40:0/4155515738 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d8022070 con 0x7f02c0006ef0 2024-09-13T17:57:04.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.001+0000 7f02deffd700 1 -- 172.21.15.40:0/4155515738 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4949 (secure 0 0 0) 0x7f02d8007520 con 0x7f02c0006ef0 2024-09-13T17:57:04.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.001+0000 7f02deffd700 1 -- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c0006ef0 msgr2=0x7f02c0007360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.001+0000 7f02deffd700 1 --2- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c0006ef0 0x7f02c0007360 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f02d8016000 tx=0x7f02d8001150 comp rx=0 tx=0).stop 2024-09-13T17:57:04.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.001+0000 7f02deffd700 1 -- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c0006ef0 msgr2=0x7f02c0007360 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:57:04.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.001+0000 7f02deffd700 1 --2- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02c0006ef0 0x7f02c0007360 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.010 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:57:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:57:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:57:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:57:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 52, 2024-09-13T17:57:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:57:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:57:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:57:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:57:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T17:57:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:57:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:57:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:57:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:04.012 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:04.012 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:04.012 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:04.012 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:04.012 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:04.012 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:04.012 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:57:04.012 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:57:04.012 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:04.012 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:04.012 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:04.012 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:04.012 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:04.013 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:04.013 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:04.013 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:04.013 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:04.013 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:04.013 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:04.013 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.013 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:57:04.013 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:57:04.013 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:57:04.013 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:57:04.013 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:57:04.013 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:57:04.013 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:57:04.014 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:57:04.014 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:57:04.014 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:57:04.014 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:57:04.014 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:57:04.014 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:57:04.014 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:57:04.014 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:57:04.014 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:57:04.014 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:57:04.014 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:04.014 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:04.015 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:04.015 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:04.015 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:04.015 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:04.015 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:04.015 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:04.015 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:04.015 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:04.015 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:04.015 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:57:04.015 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.015 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:57:04.015 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.016 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:57:04.016 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:57:04.017 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:04.017 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:04.017 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.017 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:04.017 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:57:04.017 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:04.017 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.017 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.017 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:04.017 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:57:04.017 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:04.017 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.018 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:04.018 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.018 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:57:04.018 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:57:04.018 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:04.018 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:04.018 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:04.018 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.018 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.018 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:57:04.018 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:57:04.018 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:04.018 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:04.018 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.019 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:04.019 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:57:04.019 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:04.019 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.019 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.019 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:04.019 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:57:04.019 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:04.019 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.019 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:04.019 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.019 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:57:04.019 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:57:04.019 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:04.019 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:04.019 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:04.020 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.020 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.020 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:57:04.020 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:57:04.020 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:04.020 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:04.020 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.020 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:04.020 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:57:04.020 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:04.020 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.020 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.020 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:04.021 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:57:04.021 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:04.021 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.021 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:04.021 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.021 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:57:04.021 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:57:04.021 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:04.021 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:04.021 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:04.021 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.021 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:04.021 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.022 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:57:04.022 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:57:04.022 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.022 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:04.022 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:04.022 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:57:04.022 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.022 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:04.022 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:57:04.022 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.022 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:04.022 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:04.022 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:57:04.022 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.023 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:04.023 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:57:04.023 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.023 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:04.023 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:04.023 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:57:04.023 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.023 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:04.023 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:57:04.023 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.023 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:04.023 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:04.023 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:57:04.023 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.023 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:04.024 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:57:04.024 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.024 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:04.024 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:04.024 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:57:04.024 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.024 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:04.024 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.024 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:57:04.024 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:57:04.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.001+0000 7f02ec971700 1 -- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f02cc066730 msgr2=0x7f02cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.001+0000 7f02ec971700 1 --2- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f02cc066730 0x7f02cc068bf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f02e0001560 tx=0x7f02e0001970 comp rx=0 tx=0).stop 2024-09-13T17:57:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.001+0000 7f02ec971700 1 -- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02e80cc620 msgr2=0x7f02e80ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.001+0000 7f02ec971700 1 --2- 172.21.15.40:0/4155515738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02e80cc620 0x7f02e80ced40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f02d0004000 tx=0x7f02d0004770 comp rx=0 tx=0).stop 2024-09-13T17:57:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.001+0000 7f02e6d9d700 1 -- 172.21.15.40:0/4155515738 reap_dead start 2024-09-13T17:57:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.001+0000 7f02e6d9d700 1 -- 172.21.15.40:0/4155515738 reap_dead start 2024-09-13T17:57:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.001+0000 7f02ec971700 1 -- 172.21.15.40:0/4155515738 shutdown_connections 2024-09-13T17:57:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.001+0000 7f02ec971700 1 -- 172.21.15.40:0/4155515738 >> 172.21.15.40:0/4155515738 conn(0x7f02e8066480 msgr2=0x7f02e8152580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.001+0000 7f02ec971700 1 -- 172.21.15.40:0/4155515738 shutdown_connections 2024-09-13T17:57:04.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.001+0000 7f02ec971700 1 -- 172.21.15.40:0/4155515738 wait complete. 2024-09-13T17:57:04.026 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:57:04.026 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T17:57:04.027 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:57:04.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.133+0000 7fd128b3c700 1 Processor -- start 2024-09-13T17:57:04.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.133+0000 7fd128b3c700 1 -- start start 2024-09-13T17:57:04.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.133+0000 7fd128b3c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd11c04e400 0x7fd11c0361a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.133+0000 7fd128b3c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd11c0d6d90 con 0x7fd11c0366e0 2024-09-13T17:57:04.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.133+0000 7fd128b3c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd11c0e0bb0 con 0x7fd11c0d69b0 2024-09-13T17:57:04.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.133+0000 7fd128b3c700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd11c04d3d0 con 0x7fd11c04e400 2024-09-13T17:57:04.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.133+0000 7fd123fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd11c0366e0 0x7fd11c00daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47132/0 (socket says 172.21.15.40:47132) 2024-09-13T17:57:04.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.133+0000 7fd123fff700 1 -- 172.21.15.40:0/890035162 learned_addr learned my addr 172.21.15.40:0/890035162 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:57:04.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd122ffd700 1 --2- 172.21.15.40:0/890035162 >> v2:172.21.15.43:3300/0 conn(0x7fd11c04e400 0x7fd11c0361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd1227fc700 1 -- 172.21.15.40:0/890035162 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345584970 0 0) 0x7fd11c0d6d90 con 0x7fd11c0366e0 2024-09-13T17:57:04.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd1227fc700 1 -- 172.21.15.40:0/890035162 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1140035d0 con 0x7fd11c0366e0 2024-09-13T17:57:04.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd1227fc700 1 -- 172.21.15.40:0/890035162 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 86928695 0 0) 0x7fd1140035d0 con 0x7fd11c0366e0 2024-09-13T17:57:04.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd1227fc700 1 -- 172.21.15.40:0/890035162 >> v2:172.21.15.43:3300/0 conn(0x7fd11c04e400 msgr2=0x7fd11c0361a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:04.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd1227fc700 1 --2- 172.21.15.40:0/890035162 >> v2:172.21.15.43:3300/0 conn(0x7fd11c04e400 0x7fd11c0361a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd1227fc700 1 -- 172.21.15.40:0/890035162 >> v1:172.21.15.43:6789/0 conn(0x7fd11c0d69b0 legacy=0x7fd11c0dd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd1227fc700 1 -- 172.21.15.40:0/890035162 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11c0dfdf0 con 0x7fd11c0366e0 2024-09-13T17:57:04.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd1227fc700 1 -- 172.21.15.40:0/890035162 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fd124067210 con 0x7fd11c0366e0 2024-09-13T17:57:04.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd1227fc700 1 -- 172.21.15.40:0/890035162 >> v1:172.21.15.40:6789/0 conn(0x7fd11c0366e0 legacy=0x7fd11c00daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd1227fc700 1 --2- 172.21.15.40:0/890035162 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd114003a20 0x7fd114003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd1227fc700 1 --2- 172.21.15.40:0/890035162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1140043c0 0x7fd114004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd1227fc700 1 --2- 172.21.15.40:0/890035162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd114005090 0x7fd11400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd1227fc700 1 -- 172.21.15.40:0/890035162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c04d3d0 con 0x7fd1140043c0 2024-09-13T17:57:04.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd1227fc700 1 -- 172.21.15.40:0/890035162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11400bfd0 con 0x7fd114005090 2024-09-13T17:57:04.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd1227fc700 1 -- 172.21.15.40:0/890035162 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd11400c190 con 0x7fd114003a20 2024-09-13T17:57:04.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd122ffd700 1 --2- 172.21.15.40:0/890035162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1140043c0 0x7fd114004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd1237fe700 1 --2- 172.21.15.40:0/890035162 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd114003a20 0x7fd114003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd123fff700 1 --2- 172.21.15.40:0/890035162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd114005090 0x7fd11400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd122ffd700 1 -- 172.21.15.40:0/890035162 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd114003a20 msgr2=0x7fd114003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:04.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd122ffd700 1 --2- 172.21.15.40:0/890035162 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd114003a20 0x7fd114003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd122ffd700 1 -- 172.21.15.40:0/890035162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd114005090 msgr2=0x7fd11400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:04.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd122ffd700 1 --2- 172.21.15.40:0/890035162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd114005090 0x7fd11400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd122ffd700 1 -- 172.21.15.40:0/890035162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1140035d0 con 0x7fd1140043c0 2024-09-13T17:57:04.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd123fff700 1 -- 172.21.15.40:0/890035162 reap_dead start 2024-09-13T17:57:04.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd122ffd700 1 --2- 172.21.15.40:0/890035162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1140043c0 0x7fd114004890 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd110009b70 tx=0x7fd1140058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d49a6f5e7bbedde server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:04.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd1227fc700 1 -- 172.21.15.40:0/890035162 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11000e6c0 con 0x7fd1140043c0 2024-09-13T17:57:04.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd1227fc700 1 -- 172.21.15.40:0/890035162 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11000e820 con 0x7fd1140043c0 2024-09-13T17:57:04.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd1227fc700 1 -- 172.21.15.40:0/890035162 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11000eb90 con 0x7fd1140043c0 2024-09-13T17:57:04.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd128b3c700 1 -- 172.21.15.40:0/890035162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1140043c0 msgr2=0x7fd114004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd128b3c700 1 --2- 172.21.15.40:0/890035162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1140043c0 0x7fd114004890 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd110009b70 tx=0x7fd1140058a0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd128b3c700 1 -- 172.21.15.40:0/890035162 shutdown_connections 2024-09-13T17:57:04.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd128b3c700 1 --2- 172.21.15.40:0/890035162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1140043c0 0x7fd114004890 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd128b3c700 1 -- 172.21.15.40:0/890035162 >> 172.21.15.40:0/890035162 conn(0x7fd11c009a60 msgr2=0x7fd11c009e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:04.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd128b3c700 1 -- 172.21.15.40:0/890035162 shutdown_connections 2024-09-13T17:57:04.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.137+0000 7fd128b3c700 1 -- 172.21.15.40:0/890035162 wait complete. 2024-09-13T17:57:04.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.141+0000 7fd128b3c700 1 Processor -- start 2024-09-13T17:57:04.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.141+0000 7fd128b3c700 1 -- start start 2024-09-13T17:57:04.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.141+0000 7fd128b3c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1140043c0 0x7fd11c173570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.141+0000 7fd128b3c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd11c04b1c0 0x7fd11c04b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.141+0000 7fd128b3c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd11c171af0 0x7fd11c171f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.141+0000 7fd128b3c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c04bb60 con 0x7fd11c04b1c0 2024-09-13T17:57:04.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.141+0000 7fd128b3c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c009350 con 0x7fd1140043c0 2024-09-13T17:57:04.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.141+0000 7fd128b3c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c0d9bc0 con 0x7fd11c171af0 2024-09-13T17:57:04.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.141+0000 7fd122ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd11c04b1c0 0x7fd11c04b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.141+0000 7fd122ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd11c04b1c0 0x7fd11c04b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59700/0 (socket says 172.21.15.40:59700) 2024-09-13T17:57:04.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.141+0000 7fd122ffd700 1 -- 172.21.15.40:0/174166567 learned_addr learned my addr 172.21.15.40:0/174166567 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:04.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.141+0000 7fd1237fe700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1140043c0 0x7fd11c173570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.141+0000 7fd123fff700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd11c171af0 0x7fd11c171f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.141+0000 7fd122ffd700 1 -- 172.21.15.40:0/174166567 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd11c171af0 msgr2=0x7fd11c171f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.141+0000 7fd122ffd700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd11c171af0 0x7fd11c171f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.141+0000 7fd122ffd700 1 -- 172.21.15.40:0/174166567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1140043c0 msgr2=0x7fd11c173570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.141+0000 7fd122ffd700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1140043c0 0x7fd11c173570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.141+0000 7fd122ffd700 1 -- 172.21.15.40:0/174166567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11c04bef0 con 0x7fd11c04b1c0 2024-09-13T17:57:04.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.145+0000 7fd122ffd700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd11c04b1c0 0x7fd11c04b620 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd11400bfa0 tx=0x7fd1100047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:04.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.145+0000 7fd10ffff700 1 -- 172.21.15.40:0/174166567 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11000e6c0 con 0x7fd11c04b1c0 2024-09-13T17:57:04.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.145+0000 7fd10ffff700 1 -- 172.21.15.40:0/174166567 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd110003af0 con 0x7fd11c04b1c0 2024-09-13T17:57:04.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.145+0000 7fd10ffff700 1 -- 172.21.15.40:0/174166567 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd110003ea0 con 0x7fd11c04b1c0 2024-09-13T17:57:04.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.145+0000 7fd128b3c700 1 -- 172.21.15.40:0/174166567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd11c1724b0 con 0x7fd11c04b1c0 2024-09-13T17:57:04.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.145+0000 7fd128b3c700 1 -- 172.21.15.40:0/174166567 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd11c009840 con 0x7fd11c04b1c0 2024-09-13T17:57:04.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.145+0000 7fd128b3c700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c1714e0 0x7fd11c049a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.145+0000 7fd128b3c700 1 -- 172.21.15.40:0/174166567 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c0074a0 con 0x7fd11c1714e0 2024-09-13T17:57:04.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.145+0000 7fd1237fe700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c1714e0 0x7fd11c049a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.145+0000 7fd10ffff700 1 -- 172.21.15.40:0/174166567 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd110013470 con 0x7fd11c04b1c0 2024-09-13T17:57:04.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.145+0000 7fd1237fe700 1 -- 172.21.15.40:0/174166567 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd11c0e0690 con 0x7fd11c1714e0 2024-09-13T17:57:04.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.145+0000 7fd1237fe700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c1714e0 0x7fd11c049a70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00e8c0 tx=0x7fd118012bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:04.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.145+0000 7fd10ffff700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd108066730 0x7fd108068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.145+0000 7fd10ffff700 1 -- 172.21.15.40:0/174166567 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd110085890 con 0x7fd11c04b1c0 2024-09-13T17:57:04.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.145+0000 7fd123fff700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd108066730 0x7fd108068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.145+0000 7fd10ffff700 1 -- 172.21.15.40:0/174166567 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1180053a0 con 0x7fd11c1714e0 2024-09-13T17:57:04.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.149+0000 7fd123fff700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd108066730 0x7fd108068bf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd124072e80 tx=0x7fd124065ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:04.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.149+0000 7fd10ffff700 1 -- 172.21.15.40:0/174166567 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd118005500 con 0x7fd11c1714e0 2024-09-13T17:57:04.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.149+0000 7fd10ffff700 1 -- 172.21.15.40:0/174166567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c1714e0 msgr2=0x7fd11c049a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.149+0000 7fd10ffff700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c1714e0 0x7fd11c049a70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00e8c0 tx=0x7fd118012bf0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.149+0000 7fd10ffff700 1 -- 172.21.15.40:0/174166567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c1714e0 msgr2=0x7fd11c049a70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:57:04.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.149+0000 7fd10ffff700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c1714e0 0x7fd11c049a70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.157+0000 7fd128b3c700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c0cc390 0x7fd11c0cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.157+0000 7fd128b3c700 1 -- 172.21.15.40:0/174166567 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c0e03b0 con 0x7fd11c0cc390 2024-09-13T17:57:04.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.157+0000 7fd1237fe700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c0cc390 0x7fd11c0cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.161+0000 7fd1237fe700 1 -- 172.21.15.40:0/174166567 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd11c04cf90 con 0x7fd11c0cc390 2024-09-13T17:57:04.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.161+0000 7fd1237fe700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c0cc390 0x7fd11c0cc7a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00e8c0 tx=0x7fd118001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:04.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.161+0000 7fd10ffff700 1 -- 172.21.15.40:0/174166567 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118025070 con 0x7fd11c0cc390 2024-09-13T17:57:04.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.161+0000 7fd10ffff700 1 -- 172.21.15.40:0/174166567 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4449 (secure 0 0 0) 0x7fd1180200d0 con 0x7fd11c0cc390 2024-09-13T17:57:04.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.161+0000 7fd10ffff700 1 -- 172.21.15.40:0/174166567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c0cc390 msgr2=0x7fd11c0cc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.161+0000 7fd10ffff700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c0cc390 0x7fd11c0cc7a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00e8c0 tx=0x7fd118001120 comp rx=0 tx=0).stop 2024-09-13T17:57:04.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.161+0000 7fd10ffff700 1 -- 172.21.15.40:0/174166567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c0cc390 msgr2=0x7fd11c0cc7a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:57:04.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.161+0000 7fd10ffff700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd11c0cc390 0x7fd11c0cc7a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.171 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:57:04.172 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:57:04.172 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:57:04.172 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:57:04.172 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 52, 2024-09-13T17:57:04.172 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:57:04.172 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:57:04.172 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:57:04.172 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:57:04.172 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:04.172 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T17:57:04.172 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:57:04.173 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:57:04.173 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:57:04.173 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:04.173 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:04.173 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:04.173 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:04.173 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:04.173 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:04.173 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:04.173 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:04.173 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:04.173 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:04.173 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:04.173 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:57:04.174 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:57:04.174 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:04.174 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:04.174 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:04.174 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:04.174 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:04.174 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:04.174 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:04.174 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:04.175 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:04.175 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:04.175 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:04.175 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.175 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:57:04.175 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:57:04.175 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:57:04.175 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:57:04.175 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:57:04.175 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:57:04.175 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:57:04.176 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:57:04.176 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:57:04.176 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:57:04.176 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:57:04.176 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:57:04.176 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:57:04.176 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:57:04.176 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:57:04.176 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:57:04.176 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:57:04.176 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:04.176 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:04.177 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:04.177 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:04.177 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:04.177 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:04.177 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:04.177 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:04.177 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:04.177 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:04.177 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:04.177 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:57:04.177 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.177 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:57:04.177 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.177 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:57:04.178 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:57:04.178 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:04.178 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:04.178 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.178 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:04.178 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:57:04.178 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:04.178 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.178 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.178 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:04.178 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:57:04.178 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:04.178 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.178 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:04.179 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.179 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:57:04.179 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:57:04.179 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:04.179 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:04.179 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:04.179 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.179 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.179 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:57:04.179 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:57:04.179 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:04.179 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:04.179 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.179 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:04.180 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:57:04.180 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:04.180 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.180 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.180 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:04.180 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:57:04.180 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:04.180 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.180 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:04.180 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.180 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:57:04.180 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:57:04.180 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:04.181 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:04.181 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:04.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.181 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.181 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:57:04.181 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:57:04.181 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:04.181 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:04.181 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.181 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:04.181 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:57:04.181 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:04.181 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.181 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.181 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:04.182 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:57:04.182 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:04.182 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.182 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:04.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.182 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:57:04.182 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:57:04.182 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:04.182 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:04.182 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:04.182 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.182 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:04.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.183 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:57:04.183 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:57:04.183 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.183 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:04.183 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:04.183 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:57:04.183 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.183 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:04.183 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:57:04.183 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.183 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:04.183 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:04.183 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:57:04.183 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.184 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:04.184 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.184 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:57:04.184 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:57:04.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.161+0000 7fd128b3c700 1 -- 172.21.15.40:0/174166567 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd108066730 msgr2=0x7fd108068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.161+0000 7fd128b3c700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd108066730 0x7fd108068bf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd124072e80 tx=0x7fd124065ea0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.161+0000 7fd128b3c700 1 -- 172.21.15.40:0/174166567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd11c04b1c0 msgr2=0x7fd11c04b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.161+0000 7fd123fff700 1 -- 172.21.15.40:0/174166567 reap_dead start 2024-09-13T17:57:04.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.161+0000 7fd128b3c700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd11c04b1c0 0x7fd11c04b620 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd11400bfa0 tx=0x7fd1100047e0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.161+0000 7fd128b3c700 1 -- 172.21.15.40:0/174166567 shutdown_connections 2024-09-13T17:57:04.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.161+0000 7fd128b3c700 1 --2- 172.21.15.40:0/174166567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd11c04b1c0 0x7fd11c04b620 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.161+0000 7fd128b3c700 1 -- 172.21.15.40:0/174166567 >> 172.21.15.40:0/174166567 conn(0x7fd11c009a60 msgr2=0x7fd11c051f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:04.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.161+0000 7fd128b3c700 1 -- 172.21.15.40:0/174166567 shutdown_connections 2024-09-13T17:57:04.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.161+0000 7fd128b3c700 1 -- 172.21.15.40:0/174166567 wait complete. 2024-09-13T17:57:04.186 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:57:04.186 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T17:57:04.186 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:57:04.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.301+0000 7f6a7328c700 1 Processor -- start 2024-09-13T17:57:04.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a7328c700 1 -- start start 2024-09-13T17:57:04.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a7328c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6a6c06e7f0 0x7f6a6c159f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a7328c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6a6c153330 0x7f6a6c1537a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a7328c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6c153ce0 con 0x7f6a6c06e410 2024-09-13T17:57:04.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a7328c700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6c15d9a0 con 0x7f6a6c153330 2024-09-13T17:57:04.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a7328c700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6c072240 con 0x7f6a6c06e7f0 2024-09-13T17:57:04.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a72a8b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6a6c153330 0x7f6a6c1537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a72a8b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6a6c153330 0x7f6a6c1537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59702/0 (socket says 172.21.15.40:59702) 2024-09-13T17:57:04.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a72a8b700 1 -- 172.21.15.40:0/3988302576 learned_addr learned my addr 172.21.15.40:0/3988302576 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:04.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a71a89700 1 --2- 172.21.15.40:0/3988302576 >> v2:172.21.15.43:3300/0 conn(0x7f6a6c06e7f0 0x7f6a6c159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a72a8b700 1 -- 172.21.15.40:0/3988302576 >> v2:172.21.15.43:3300/0 conn(0x7f6a6c06e7f0 msgr2=0x7f6a6c159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:04.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a72a8b700 1 --2- 172.21.15.40:0/3988302576 >> v2:172.21.15.43:3300/0 conn(0x7f6a6c06e7f0 0x7f6a6c159f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a72a8b700 1 -- 172.21.15.40:0/3988302576 >> v1:172.21.15.43:6789/0 conn(0x7f6a6c06e410 legacy=0x7f6a6c0c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a72a8b700 1 -- 172.21.15.40:0/3988302576 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c15a470 con 0x7f6a6c153330 2024-09-13T17:57:04.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a72a8b700 1 --2- 172.21.15.40:0/3988302576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a6c153330 0x7f6a6c1537a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c0ccbe0 tx=0x7f6a5c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a39b02419d8296 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:04.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a71288700 1 -- 172.21.15.40:0/3988302576 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c00b8a0 con 0x7f6a6c153330 2024-09-13T17:57:04.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a71288700 1 -- 172.21.15.40:0/3988302576 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5c00ba00 con 0x7f6a6c153330 2024-09-13T17:57:04.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a71288700 1 -- 172.21.15.40:0/3988302576 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c00bd90 con 0x7f6a6c153330 2024-09-13T17:57:04.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a7328c700 1 -- 172.21.15.40:0/3988302576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a6c153330 msgr2=0x7f6a6c1537a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a7328c700 1 --2- 172.21.15.40:0/3988302576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a6c153330 0x7f6a6c1537a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c0ccbe0 tx=0x7f6a5c009bf0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a7328c700 1 -- 172.21.15.40:0/3988302576 shutdown_connections 2024-09-13T17:57:04.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a7328c700 1 --2- 172.21.15.40:0/3988302576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a6c153330 0x7f6a6c1537a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a7328c700 1 --2- 172.21.15.40:0/3988302576 >> v2:172.21.15.43:3300/0 conn(0x7f6a6c06e7f0 0x7f6a6c159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a7328c700 1 -- 172.21.15.40:0/3988302576 >> 172.21.15.40:0/3988302576 conn(0x7f6a6c0cbc40 msgr2=0x7f6a6c0cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:04.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a7328c700 1 -- 172.21.15.40:0/3988302576 shutdown_connections 2024-09-13T17:57:04.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.305+0000 7f6a7328c700 1 -- 172.21.15.40:0/3988302576 wait complete. 2024-09-13T17:57:04.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.309+0000 7f6a7328c700 1 Processor -- start 2024-09-13T17:57:04.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.309+0000 7f6a7328c700 1 -- start start 2024-09-13T17:57:04.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a7328c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a6c06e410 0x7f6a6c14eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a7328c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6c06e7f0 0x7f6a6c0c4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a7328c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a6c153330 0x7f6a6c0c3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a7328c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c14f3e0 con 0x7f6a6c06e410 2024-09-13T17:57:04.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a7328c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0c4a90 con 0x7f6a6c153330 2024-09-13T17:57:04.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a7328c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c14dd40 con 0x7f6a6c06e7f0 2024-09-13T17:57:04.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a7228a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a6c06e410 0x7f6a6c14eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a7228a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a6c06e410 0x7f6a6c14eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59706/0 (socket says 172.21.15.40:59706) 2024-09-13T17:57:04.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a72a8b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a6c153330 0x7f6a6c0c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a71a89700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6c06e7f0 0x7f6a6c0c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a7228a700 1 -- 172.21.15.40:0/1114368920 learned_addr learned my addr 172.21.15.40:0/1114368920 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:04.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a72a8b700 1 -- 172.21.15.40:0/1114368920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6c06e7f0 msgr2=0x7f6a6c0c4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a72a8b700 1 --2- 172.21.15.40:0/1114368920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6c06e7f0 0x7f6a6c0c4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a72a8b700 1 -- 172.21.15.40:0/1114368920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a6c06e410 msgr2=0x7f6a6c14eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a72a8b700 1 --2- 172.21.15.40:0/1114368920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a6c06e410 0x7f6a6c14eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a72a8b700 1 -- 172.21.15.40:0/1114368920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c009570 con 0x7f6a6c153330 2024-09-13T17:57:04.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a72a8b700 1 --2- 172.21.15.40:0/1114368920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a6c153330 0x7f6a6c0c3210 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c016000 tx=0x7f6a5c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:04.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a62ffd700 1 -- 172.21.15.40:0/1114368920 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c0038c0 con 0x7f6a6c153330 2024-09-13T17:57:04.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a7328c700 1 -- 172.21.15.40:0/1114368920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6c14e020 con 0x7f6a6c153330 2024-09-13T17:57:04.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a62ffd700 1 -- 172.21.15.40:0/1114368920 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5c01e070 con 0x7f6a6c153330 2024-09-13T17:57:04.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a7328c700 1 -- 172.21.15.40:0/1114368920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6c150ed0 con 0x7f6a6c153330 2024-09-13T17:57:04.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.313+0000 7f6a62ffd700 1 -- 172.21.15.40:0/1114368920 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c003c60 con 0x7f6a6c153330 2024-09-13T17:57:04.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.317+0000 7f6a7328c700 1 --2- 172.21.15.40:0/1114368920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6c0c4e80 0x7f6a6c0c5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.317+0000 7f6a7328c700 1 -- 172.21.15.40:0/1114368920 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c067570 con 0x7f6a6c0c4e80 2024-09-13T17:57:04.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.317+0000 7f6a7228a700 1 --2- 172.21.15.40:0/1114368920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6c0c4e80 0x7f6a6c0c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.317+0000 7f6a7228a700 1 -- 172.21.15.40:0/1114368920 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6a6c15d100 con 0x7f6a6c0c4e80 2024-09-13T17:57:04.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.317+0000 7f6a7228a700 1 --2- 172.21.15.40:0/1114368920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6c0c4e80 0x7f6a6c0c5260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6a64002070 tx=0x7f6a6400d430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:04.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.317+0000 7f6a62ffd700 1 -- 172.21.15.40:0/1114368920 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6400dc90 con 0x7f6a6c0c4e80 2024-09-13T17:57:04.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.317+0000 7f6a62ffd700 1 -- 172.21.15.40:0/1114368920 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6a5c025480 con 0x7f6a6c153330 2024-09-13T17:57:04.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.317+0000 7f6a62ffd700 1 --2- 172.21.15.40:0/1114368920 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6a500667d0 0x7f6a50068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.317+0000 7f6a62ffd700 1 -- 172.21.15.40:0/1114368920 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6a5c020070 con 0x7f6a6c153330 2024-09-13T17:57:04.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.317+0000 7f6a71a89700 1 --2- 172.21.15.40:0/1114368920 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6a500667d0 0x7f6a50068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.317+0000 7f6a62ffd700 1 -- 172.21.15.40:0/1114368920 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6a6400ddf0 con 0x7f6a6c0c4e80 2024-09-13T17:57:04.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.317+0000 7f6a62ffd700 1 -- 172.21.15.40:0/1114368920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6c0c4e80 msgr2=0x7f6a6c0c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.317+0000 7f6a62ffd700 1 --2- 172.21.15.40:0/1114368920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6c0c4e80 0x7f6a6c0c5260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6a64002070 tx=0x7f6a6400d430 comp rx=0 tx=0).stop 2024-09-13T17:57:04.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.317+0000 7f6a62ffd700 1 -- 172.21.15.40:0/1114368920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6c0c4e80 msgr2=0x7f6a6c0c5260 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:57:04.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.317+0000 7f6a62ffd700 1 --2- 172.21.15.40:0/1114368920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6c0c4e80 0x7f6a6c0c5260 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.321+0000 7f6a71a89700 1 --2- 172.21.15.40:0/1114368920 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6a500667d0 0x7f6a50068c90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009a00 tx=0x7f6a68008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:04.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a7328c700 1 --2- 172.21.15.40:0/1114368920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6c15b6b0 0x7f6a6c15bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a7328c700 1 -- 172.21.15.40:0/1114368920 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c15cb80 con 0x7f6a6c15b6b0 2024-09-13T17:57:04.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a7228a700 1 --2- 172.21.15.40:0/1114368920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6c15b6b0 0x7f6a6c15bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a7228a700 1 -- 172.21.15.40:0/1114368920 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6a6c15d3e0 con 0x7f6a6c15b6b0 2024-09-13T17:57:04.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a7228a700 1 --2- 172.21.15.40:0/1114368920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6c15b6b0 0x7f6a6c15bac0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6a64002070 tx=0x7f6a64001250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:04.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a62ffd700 1 -- 172.21.15.40:0/1114368920 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a64023070 con 0x7f6a6c15b6b0 2024-09-13T17:57:04.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a62ffd700 1 -- 172.21.15.40:0/1114368920 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4781 (secure 0 0 0) 0x7f6a640052e0 con 0x7f6a6c15b6b0 2024-09-13T17:57:04.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a62ffd700 1 -- 172.21.15.40:0/1114368920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6c15b6b0 msgr2=0x7f6a6c15bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a62ffd700 1 --2- 172.21.15.40:0/1114368920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6c15b6b0 0x7f6a6c15bac0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6a64002070 tx=0x7f6a64001250 comp rx=0 tx=0).stop 2024-09-13T17:57:04.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a62ffd700 1 -- 172.21.15.40:0/1114368920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6c15b6b0 msgr2=0x7f6a6c15bac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:57:04.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a62ffd700 1 --2- 172.21.15.40:0/1114368920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6c15b6b0 0x7f6a6c15bac0 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6a64002070 tx=0x7f6a64001250 comp rx=0 tx=0).stop 2024-09-13T17:57:04.341 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:57:04.341 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:57:04.341 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:57:04.341 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:57:04.341 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 52, 2024-09-13T17:57:04.341 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:57:04.341 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:57:04.341 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:57:04.341 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:57:04.341 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:04.341 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T17:57:04.341 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:57:04.342 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:57:04.342 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:57:04.342 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:04.342 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:04.342 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:04.342 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:04.342 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:04.342 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:04.342 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:04.342 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:04.342 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:04.342 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:04.342 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:04.342 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:57:04.343 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:57:04.343 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:04.343 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:04.343 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:04.343 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:04.343 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:04.343 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:04.343 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:04.343 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:04.343 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:04.343 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:04.343 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:04.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.343 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:57:04.344 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:57:04.344 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:57:04.344 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:57:04.344 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:57:04.344 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:57:04.344 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:57:04.344 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:57:04.344 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:57:04.344 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:57:04.344 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:57:04.345 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:57:04.345 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:57:04.345 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:57:04.345 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:57:04.345 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:57:04.345 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:57:04.345 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:04.345 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:04.345 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:04.345 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:04.345 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:04.345 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:04.346 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:04.346 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:04.346 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:04.346 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:04.346 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:04.346 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:57:04.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.346 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:57:04.346 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.346 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:57:04.346 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:57:04.346 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:04.346 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:04.347 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.347 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:04.347 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:57:04.347 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:04.347 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.347 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.347 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:04.347 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:57:04.347 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:04.347 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.347 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:04.347 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.348 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:57:04.348 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:57:04.348 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:04.348 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:04.348 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:04.348 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.348 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:57:04.348 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:57:04.348 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:04.348 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:04.348 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.348 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:04.349 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:57:04.349 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:04.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.349 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.349 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:04.349 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:57:04.349 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:04.349 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.349 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:04.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.349 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:57:04.350 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:57:04.350 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:04.350 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:04.350 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:04.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.350 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.350 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:57:04.350 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:57:04.350 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:04.350 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:04.350 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.350 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:04.350 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:57:04.351 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:04.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.351 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:04.351 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:57:04.351 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:04.351 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.351 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:04.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.351 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:57:04.351 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:57:04.351 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:04.351 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:04.352 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:04.352 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.352 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:04.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.352 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:57:04.352 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:57:04.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.352 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:04.352 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:04.352 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:57:04.352 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.352 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:04.352 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:57:04.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.353 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:04.353 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:04.353 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:57:04.353 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.353 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:04.353 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:57:04.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.353 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:04.353 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:04.353 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T17:57:04.353 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.353 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:04.353 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:57:04.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:04.354 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:04.354 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:04.354 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:57:04.354 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:04.354 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:04.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:04.354 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:57:04.354 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:57:04.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a7328c700 1 -- 172.21.15.40:0/1114368920 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6a500667d0 msgr2=0x7f6a50068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a7328c700 1 --2- 172.21.15.40:0/1114368920 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6a500667d0 0x7f6a50068c90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009a00 tx=0x7f6a68008040 comp rx=0 tx=0).stop 2024-09-13T17:57:04.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a7328c700 1 -- 172.21.15.40:0/1114368920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a6c153330 msgr2=0x7f6a6c0c3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a7328c700 1 --2- 172.21.15.40:0/1114368920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a6c153330 0x7f6a6c0c3210 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c016000 tx=0x7f6a5c004770 comp rx=0 tx=0).stop 2024-09-13T17:57:04.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a72a8b700 1 -- 172.21.15.40:0/1114368920 reap_dead start 2024-09-13T17:57:04.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a72a8b700 1 -- 172.21.15.40:0/1114368920 reap_dead start 2024-09-13T17:57:04.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a7328c700 1 -- 172.21.15.40:0/1114368920 shutdown_connections 2024-09-13T17:57:04.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a7328c700 1 -- 172.21.15.40:0/1114368920 >> 172.21.15.40:0/1114368920 conn(0x7f6a6c0cbc40 msgr2=0x7f6a6c0ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:04.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a7328c700 1 -- 172.21.15.40:0/1114368920 shutdown_connections 2024-09-13T17:57:04.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.329+0000 7f6a7328c700 1 -- 172.21.15.40:0/1114368920 wait complete. 2024-09-13T17:57:04.356 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:57:04.356 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T17:57:04.356 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T17:57:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.457+0000 7f51faf43700 1 Processor -- start 2024-09-13T17:57:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.457+0000 7f51faf43700 1 -- start start 2024-09-13T17:57:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.457+0000 7f51faf43700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f51f4159890 0x7f51f40722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.457+0000 7f51faf43700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51f40c78a0 0x7f51f40c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.457+0000 7f51faf43700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f406b870 con 0x7f51f40702e0 2024-09-13T17:57:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.457+0000 7f51faf43700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f406ed60 con 0x7f51f4159890 2024-09-13T17:57:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.457+0000 7f51faf43700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f4071dc0 con 0x7f51f40c78a0 2024-09-13T17:57:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.457+0000 7f51f9f41700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f51f4159890 0x7f51f40722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.457+0000 7f51fa742700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51f40c78a0 0x7f51f40c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.457+0000 7f51f9f41700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f51f4159890 0x7f51f40722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:50838/0 (socket says 172.21.15.40:50838) 2024-09-13T17:57:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.457+0000 7f51fa742700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51f40c78a0 0x7f51f40c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:37040/0 (socket says 172.21.15.40:37040) 2024-09-13T17:57:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.457+0000 7f51f9f41700 1 -- 172.21.15.40:0/381945548 learned_addr learned my addr 172.21.15.40:0/381945548 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:04.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.457+0000 7f51fa742700 1 -- 172.21.15.40:0/381945548 >> v2:172.21.15.43:3300/0 conn(0x7f51f4159890 msgr2=0x7f51f40722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:04.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.457+0000 7f51fa742700 1 --2- 172.21.15.40:0/381945548 >> v2:172.21.15.43:3300/0 conn(0x7f51f4159890 0x7f51f40722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.457+0000 7f51fa742700 1 -- 172.21.15.40:0/381945548 >> v1:172.21.15.43:6789/0 conn(0x7f51f40702e0 legacy=0x7f51f4159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.457+0000 7f51fa742700 1 -- 172.21.15.40:0/381945548 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f4070cd0 con 0x7f51f40c78a0 2024-09-13T17:57:04.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.457+0000 7f51fa742700 1 --2- 172.21.15.40:0/381945548 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51f40c78a0 0x7f51f40c5c70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009d90 tx=0x7f51ec00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eff524d5ceb9f022 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:04.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.457+0000 7f51f8f3f700 1 -- 172.21.15.40:0/381945548 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec00e070 con 0x7f51f40c78a0 2024-09-13T17:57:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.457+0000 7f51f8f3f700 1 -- 172.21.15.40:0/381945548 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec00bb20 con 0x7f51f40c78a0 2024-09-13T17:57:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.457+0000 7f51f8f3f700 1 -- 172.21.15.40:0/381945548 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec012430 con 0x7f51f40c78a0 2024-09-13T17:57:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.461+0000 7f51faf43700 1 -- 172.21.15.40:0/381945548 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51f40c78a0 msgr2=0x7f51f40c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.461+0000 7f51faf43700 1 --2- 172.21.15.40:0/381945548 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51f40c78a0 0x7f51f40c5c70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009d90 tx=0x7f51ec00b330 comp rx=0 tx=0).stop 2024-09-13T17:57:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.461+0000 7f51faf43700 1 -- 172.21.15.40:0/381945548 shutdown_connections 2024-09-13T17:57:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.461+0000 7f51faf43700 1 --2- 172.21.15.40:0/381945548 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51f40c78a0 0x7f51f40c5c70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.461+0000 7f51faf43700 1 --2- 172.21.15.40:0/381945548 >> v2:172.21.15.43:3300/0 conn(0x7f51f4159890 0x7f51f40722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.461+0000 7f51faf43700 1 -- 172.21.15.40:0/381945548 >> 172.21.15.40:0/381945548 conn(0x7f51f40c7d70 msgr2=0x7f51f40c8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.461+0000 7f51faf43700 1 -- 172.21.15.40:0/381945548 shutdown_connections 2024-09-13T17:57:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.461+0000 7f51faf43700 1 -- 172.21.15.40:0/381945548 wait complete. 2024-09-13T17:57:04.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.461+0000 7f51faf43700 1 Processor -- start 2024-09-13T17:57:04.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.461+0000 7f51faf43700 1 -- start start 2024-09-13T17:57:04.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51faf43700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51f40702e0 0x7f51f41447d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51faf43700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51f40c78a0 0x7f51f4141c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51faf43700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f4159890 0x7f51f412dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51faf43700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f41443b0 con 0x7f51f40c78a0 2024-09-13T17:57:04.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51faf43700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f41f2e60 con 0x7f51f4159890 2024-09-13T17:57:04.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51faf43700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f41f2900 con 0x7f51f40702e0 2024-09-13T17:57:04.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51f9740700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51f40c78a0 0x7f51f4141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51f9740700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51f40c78a0 0x7f51f4141c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59712/0 (socket says 172.21.15.40:59712) 2024-09-13T17:57:04.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51f9740700 1 -- 172.21.15.40:0/2691298776 learned_addr learned my addr 172.21.15.40:0/2691298776 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:04.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51fa742700 1 --2- 172.21.15.40:0/2691298776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f4159890 0x7f51f412dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51f9f41700 1 --2- 172.21.15.40:0/2691298776 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51f40702e0 0x7f51f41447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51f9740700 1 -- 172.21.15.40:0/2691298776 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51f40702e0 msgr2=0x7f51f41447d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51f9740700 1 --2- 172.21.15.40:0/2691298776 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51f40702e0 0x7f51f41447d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51f9740700 1 -- 172.21.15.40:0/2691298776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f4159890 msgr2=0x7f51f412dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51f9740700 1 --2- 172.21.15.40:0/2691298776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f4159890 0x7f51f412dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51f9740700 1 -- 172.21.15.40:0/2691298776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec009a70 con 0x7f51f40c78a0 2024-09-13T17:57:04.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51f9740700 1 --2- 172.21.15.40:0/2691298776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51f40c78a0 0x7f51f4141c70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f51e400c530 tx=0x7f51e400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:04.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51ea7fc700 1 -- 172.21.15.40:0/2691298776 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e40052d0 con 0x7f51f40c78a0 2024-09-13T17:57:04.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51ea7fc700 1 -- 172.21.15.40:0/2691298776 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e4005430 con 0x7f51f40c78a0 2024-09-13T17:57:04.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51faf43700 1 -- 172.21.15.40:0/2691298776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f41453e0 con 0x7f51f40c78a0 2024-09-13T17:57:04.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51ea7fc700 1 -- 172.21.15.40:0/2691298776 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e40055a0 con 0x7f51f40c78a0 2024-09-13T17:57:04.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.465+0000 7f51faf43700 1 -- 172.21.15.40:0/2691298776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f414e8f0 con 0x7f51f40c78a0 2024-09-13T17:57:04.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.469+0000 7f51faf43700 1 -- 172.21.15.40:0/2691298776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f41444f0 con 0x7f51f40c78a0 2024-09-13T17:57:04.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.469+0000 7f51ea7fc700 1 -- 172.21.15.40:0/2691298776 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f51e40038e0 con 0x7f51f40c78a0 2024-09-13T17:57:04.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.469+0000 7f51ea7fc700 1 --2- 172.21.15.40:0/2691298776 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f51d8066760 0x7f51d8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:04.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.469+0000 7f51ea7fc700 1 -- 172.21.15.40:0/2691298776 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f51e40839f0 con 0x7f51f40c78a0 2024-09-13T17:57:04.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.469+0000 7f51f9f41700 1 --2- 172.21.15.40:0/2691298776 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f51d8066760 0x7f51d8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:04.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.469+0000 7f51f9f41700 1 --2- 172.21.15.40:0/2691298776 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f51d8066760 0x7f51d8068c20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009b50 tx=0x7f51f00092e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:04.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.477+0000 7f51ea7fc700 1 -- 172.21.15.40:0/2691298776 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f51f41444f0 con 0x7f51f40c78a0 2024-09-13T17:57:04.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.777+0000 7f51faf43700 1 -- 172.21.15.40:0/2691298776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f51f41444f0 con 0x7f51f40c78a0 2024-09-13T17:57:04.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.781+0000 7f51ea7fc700 1 -- 172.21.15.40:0/2691298776 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f51f41444f0 con 0x7f51f40c78a0 2024-09-13T17:57:04.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.781+0000 7f51faf43700 1 -- 172.21.15.40:0/2691298776 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f51d8066760 msgr2=0x7f51d8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.781+0000 7f51faf43700 1 --2- 172.21.15.40:0/2691298776 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f51d8066760 0x7f51d8068c20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009b50 tx=0x7f51f00092e0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.781+0000 7f51faf43700 1 -- 172.21.15.40:0/2691298776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51f40c78a0 msgr2=0x7f51f4141c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:04.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.781+0000 7f51faf43700 1 --2- 172.21.15.40:0/2691298776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51f40c78a0 0x7f51f4141c70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f51e400c530 tx=0x7f51e400c840 comp rx=0 tx=0).stop 2024-09-13T17:57:04.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.781+0000 7f51faf43700 1 -- 172.21.15.40:0/2691298776 shutdown_connections 2024-09-13T17:57:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.781+0000 7f51faf43700 1 --2- 172.21.15.40:0/2691298776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51f4159890 0x7f51f412dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.781+0000 7f51faf43700 1 --2- 172.21.15.40:0/2691298776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51f40c78a0 0x7f51f4141c70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.785+0000 7f51faf43700 1 --2- 172.21.15.40:0/2691298776 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f51d8066760 0x7f51d8068c20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.785+0000 7f51faf43700 1 --2- 172.21.15.40:0/2691298776 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51f40702e0 0x7f51f41447d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.785+0000 7f51faf43700 1 -- 172.21.15.40:0/2691298776 >> 172.21.15.40:0/2691298776 conn(0x7f51f40c7d70 msgr2=0x7f51f40c6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.785+0000 7f51faf43700 1 -- 172.21.15.40:0/2691298776 shutdown_connections 2024-09-13T17:57:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:04.785+0000 7f51faf43700 1 -- 172.21.15.40:0/2691298776 wait complete. 2024-09-13T17:57:04.807 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T17:57:04.809 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:57:04.811 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:57:14.808 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T17:57:14.809 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:57:14.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b759e700 1 Processor -- start 2024-09-13T17:57:14.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b759e700 1 -- start start 2024-09-13T17:57:14.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f22b81503d0 0x7f22b8156940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:14.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b759e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22b815e960 0x7f22b8160d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:14.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b759e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b815a520 con 0x7f22b806b4a0 2024-09-13T17:57:14.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b759e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b81564b0 con 0x7f22b815e960 2024-09-13T17:57:14.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b815a000 con 0x7f22b81503d0 2024-09-13T17:57:14.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b5d9b700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f22b806b4a0 0x7f22b8071bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46398/0 (socket says 172.21.15.40:46398) 2024-09-13T17:57:14.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b5d9b700 1 -- 172.21.15.40:0/2448825922 learned_addr learned my addr 172.21.15.40:0/2448825922 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:57:14.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b6d9d700 1 --2- 172.21.15.40:0/2448825922 >> v2:172.21.15.43:3300/0 conn(0x7f22b815e960 0x7f22b8160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:14.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b659c700 1 --2- 172.21.15.40:0/2448825922 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f22b81503d0 0x7f22b8156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:14.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b559a700 1 -- 172.21.15.40:0/2448825922 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2232536089 0 0) 0x7f22b815a520 con 0x7f22b806b4a0 2024-09-13T17:57:14.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b559a700 1 -- 172.21.15.40:0/2448825922 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22a00035d0 con 0x7f22b806b4a0 2024-09-13T17:57:14.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b559a700 1 -- 172.21.15.40:0/2448825922 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4227110651 0 0) 0x7f22a00035d0 con 0x7f22b806b4a0 2024-09-13T17:57:14.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b559a700 1 -- 172.21.15.40:0/2448825922 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f22b81503d0 msgr2=0x7f22b8156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:14.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b559a700 1 --2- 172.21.15.40:0/2448825922 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f22b81503d0 0x7f22b8156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:14.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b559a700 1 -- 172.21.15.40:0/2448825922 >> v2:172.21.15.43:3300/0 conn(0x7f22b815e960 msgr2=0x7f22b8160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:14.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b559a700 1 --2- 172.21.15.40:0/2448825922 >> v2:172.21.15.43:3300/0 conn(0x7f22b815e960 0x7f22b8160d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:14.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b559a700 1 -- 172.21.15.40:0/2448825922 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b815a280 con 0x7f22b806b4a0 2024-09-13T17:57:14.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b559a700 1 -- 172.21.15.40:0/2448825922 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f22ac001670 con 0x7f22b806b4a0 2024-09-13T17:57:14.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b559a700 1 -- 172.21.15.40:0/2448825922 >> v1:172.21.15.40:6789/0 conn(0x7f22b806b4a0 legacy=0x7f22b8071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:14.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b559a700 1 --2- 172.21.15.40:0/2448825922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22a0003a20 0x7f22a0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:14.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b559a700 1 --2- 172.21.15.40:0/2448825922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22a0004870 0x7f22a0004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:14.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b559a700 1 --2- 172.21.15.40:0/2448825922 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f22b81503d0 0x7f22a0005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:14.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b559a700 1 -- 172.21.15.40:0/2448825922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b81564b0 con 0x7f22a0004870 2024-09-13T17:57:14.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b559a700 1 -- 172.21.15.40:0/2448825922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b815a000 con 0x7f22a0003a20 2024-09-13T17:57:14.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b559a700 1 -- 172.21.15.40:0/2448825922 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a000ba30 con 0x7f22b81503d0 2024-09-13T17:57:14.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b5d9b700 1 --2- 172.21.15.40:0/2448825922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22a0004870 0x7f22a0004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:14.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.941+0000 7f22b6d9d700 1 --2- 172.21.15.40:0/2448825922 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f22b81503d0 0x7f22a0005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:14.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.945+0000 7f22b659c700 1 --2- 172.21.15.40:0/2448825922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22a0003a20 0x7f22a0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:14.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.945+0000 7f22b5d9b700 1 -- 172.21.15.40:0/2448825922 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f22b81503d0 msgr2=0x7f22a0005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:14.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.945+0000 7f22b5d9b700 1 --2- 172.21.15.40:0/2448825922 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f22b81503d0 0x7f22a0005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:14.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.945+0000 7f22b5d9b700 1 -- 172.21.15.40:0/2448825922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22a0003a20 msgr2=0x7f22a0003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:14.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.945+0000 7f22b5d9b700 1 --2- 172.21.15.40:0/2448825922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22a0003a20 0x7f22a0003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:14.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.945+0000 7f22b5d9b700 1 -- 172.21.15.40:0/2448825922 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a00035d0 con 0x7f22a0004870 2024-09-13T17:57:14.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.945+0000 7f22b659c700 1 --2- 172.21.15.40:0/2448825922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22a0003a20 0x7f22a0003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:57:14.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.945+0000 7f22b5d9b700 1 --2- 172.21.15.40:0/2448825922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22a0004870 0x7f22a0004cf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00c2f0 tx=0x7f22a00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec18fd890901157 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:14.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.945+0000 7f22b559a700 1 -- 172.21.15.40:0/2448825922 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac007c60 con 0x7f22a0004870 2024-09-13T17:57:14.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.945+0000 7f22b559a700 1 -- 172.21.15.40:0/2448825922 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ac00ce30 con 0x7f22a0004870 2024-09-13T17:57:14.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.945+0000 7f22b559a700 1 -- 172.21.15.40:0/2448825922 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac005bc0 con 0x7f22a0004870 2024-09-13T17:57:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.945+0000 7f22b759e700 1 -- 172.21.15.40:0/2448825922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22a0004870 msgr2=0x7f22a0004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.945+0000 7f22b759e700 1 --2- 172.21.15.40:0/2448825922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22a0004870 0x7f22a0004cf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00c2f0 tx=0x7f22a00058a0 comp rx=0 tx=0).stop 2024-09-13T17:57:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.945+0000 7f22b6d9d700 1 -- 172.21.15.40:0/2448825922 reap_dead start 2024-09-13T17:57:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.945+0000 7f22b759e700 1 -- 172.21.15.40:0/2448825922 shutdown_connections 2024-09-13T17:57:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.945+0000 7f22b759e700 1 -- 172.21.15.40:0/2448825922 >> 172.21.15.40:0/2448825922 conn(0x7f22b806e4f0 msgr2=0x7f22b806e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.945+0000 7f22b759e700 1 -- 172.21.15.40:0/2448825922 shutdown_connections 2024-09-13T17:57:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.945+0000 7f22b759e700 1 -- 172.21.15.40:0/2448825922 wait complete. 2024-09-13T17:57:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.945+0000 7f22b759e700 1 Processor -- start 2024-09-13T17:57:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.945+0000 7f22b759e700 1 -- start start 2024-09-13T17:57:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22b759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22b8141cd0 0x7f22b8153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22b759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22b81effe0 0x7f22b8154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22b759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f22b81420b0 0x7f22b806a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22b759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b8144940 con 0x7f22b8141cd0 2024-09-13T17:57:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22b759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b8146020 con 0x7f22b81effe0 2024-09-13T17:57:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22b759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b8144dd0 con 0x7f22b81420b0 2024-09-13T17:57:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22b659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22b8141cd0 0x7f22b8153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22b659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22b8141cd0 0x7f22b8153d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55622/0 (socket says 172.21.15.40:55622) 2024-09-13T17:57:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22b659c700 1 -- 172.21.15.40:0/2703096465 learned_addr learned my addr 172.21.15.40:0/2703096465 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22b5d9b700 1 --2- 172.21.15.40:0/2703096465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22b81effe0 0x7f22b8154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22b659c700 1 -- 172.21.15.40:0/2703096465 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f22b81420b0 msgr2=0x7f22b806a390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:57:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22b659c700 1 --2- 172.21.15.40:0/2703096465 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f22b81420b0 0x7f22b806a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22b659c700 1 -- 172.21.15.40:0/2703096465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22b81effe0 msgr2=0x7f22b8154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22b659c700 1 --2- 172.21.15.40:0/2703096465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22b81effe0 0x7f22b8154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22b659c700 1 -- 172.21.15.40:0/2703096465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b8143ae0 con 0x7f22b8141cd0 2024-09-13T17:57:14.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22b659c700 1 --2- 172.21.15.40:0/2703096465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22b8141cd0 0x7f22b8153d30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f22a80108f0 tx=0x7f22a8010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:14.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22a6ffd700 1 -- 172.21.15.40:0/2703096465 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8004020 con 0x7f22b8141cd0 2024-09-13T17:57:14.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22a6ffd700 1 -- 172.21.15.40:0/2703096465 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a8004180 con 0x7f22b8141cd0 2024-09-13T17:57:14.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22b759e700 1 -- 172.21.15.40:0/2703096465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b8143d30 con 0x7f22b8141cd0 2024-09-13T17:57:14.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22a6ffd700 1 -- 172.21.15.40:0/2703096465 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8004530 con 0x7f22b8141cd0 2024-09-13T17:57:14.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.949+0000 7f22b759e700 1 -- 172.21.15.40:0/2703096465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b81440c0 con 0x7f22b8141cd0 2024-09-13T17:57:14.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.953+0000 7f22b759e700 1 -- 172.21.15.40:0/2703096465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b81560d0 con 0x7f22b8141cd0 2024-09-13T17:57:14.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.953+0000 7f22a6ffd700 1 -- 172.21.15.40:0/2703096465 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f22a80036c0 con 0x7f22b8141cd0 2024-09-13T17:57:14.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.953+0000 7f22a6ffd700 1 --2- 172.21.15.40:0/2703096465 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2298066760 0x7f2298068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:14.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.953+0000 7f22a6ffd700 1 -- 172.21.15.40:0/2703096465 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f22a80845c0 con 0x7f22b8141cd0 2024-09-13T17:57:14.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.953+0000 7f22b5d9b700 1 --2- 172.21.15.40:0/2703096465 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2298066760 0x7f2298068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:14.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.953+0000 7f22b5d9b700 1 --2- 172.21.15.40:0/2703096465 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2298066760 0x7f2298068c20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f22b804f940 tx=0x7f22ac006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:14.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:14.961+0000 7f22a6ffd700 1 -- 172.21.15.40:0/2703096465 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f22a8051570 con 0x7f22b8141cd0 2024-09-13T17:57:15.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.237+0000 7f22b759e700 1 -- 172.21.15.40:0/2703096465 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f22b80c68e0 con 0x7f22b8141cd0 2024-09-13T17:57:15.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.237+0000 7f22a6ffd700 1 -- 172.21.15.40:0/2703096465 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f22a8054b90 con 0x7f22b8141cd0 2024-09-13T17:57:15.242 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":52,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:57:15.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.237+0000 7f22b759e700 1 -- 172.21.15.40:0/2703096465 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2298066760 msgr2=0x7f2298068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.237+0000 7f22b759e700 1 --2- 172.21.15.40:0/2703096465 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2298066760 0x7f2298068c20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f22b804f940 tx=0x7f22ac006040 comp rx=0 tx=0).stop 2024-09-13T17:57:15.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.237+0000 7f22b759e700 1 -- 172.21.15.40:0/2703096465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22b8141cd0 msgr2=0x7f22b8153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.237+0000 7f22b759e700 1 --2- 172.21.15.40:0/2703096465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22b8141cd0 0x7f22b8153d30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f22a80108f0 tx=0x7f22a8010c00 comp rx=0 tx=0).stop 2024-09-13T17:57:15.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.237+0000 7f22b759e700 1 -- 172.21.15.40:0/2703096465 shutdown_connections 2024-09-13T17:57:15.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.237+0000 7f22b759e700 1 --2- 172.21.15.40:0/2703096465 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2298066760 0x7f2298068c20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.237+0000 7f22b759e700 1 --2- 172.21.15.40:0/2703096465 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f22b81420b0 0x7f22b806a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.237+0000 7f22b759e700 1 --2- 172.21.15.40:0/2703096465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22b81effe0 0x7f22b8154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.237+0000 7f22b759e700 1 --2- 172.21.15.40:0/2703096465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22b8141cd0 0x7f22b8153d30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.237+0000 7f22b759e700 1 -- 172.21.15.40:0/2703096465 >> 172.21.15.40:0/2703096465 conn(0x7f22b806e4f0 msgr2=0x7f22b8071d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:15.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.237+0000 7f22b759e700 1 -- 172.21.15.40:0/2703096465 shutdown_connections 2024-09-13T17:57:15.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.237+0000 7f22b759e700 1 -- 172.21.15.40:0/2703096465 wait complete. 2024-09-13T17:57:15.256 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T17:57:15.257 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T17:57:15.257 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T17:57:15.257 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:57:15.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74b66d9700 1 Processor -- start 2024-09-13T17:57:15.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74b66d9700 1 -- start start 2024-09-13T17:57:15.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74b66d9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f74b006dde0 0x7f74b006e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74b66d9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f74b00c3de0 0x7f74b00cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74b66d9700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b006e580 con 0x7f74b01515a0 2024-09-13T17:57:15.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74b66d9700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b0151980 con 0x7f74b00c3de0 2024-09-13T17:57:15.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74b66d9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b00c3930 con 0x7f74b006dde0 2024-09-13T17:57:15.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74b5ed8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f74b00c3de0 0x7f74b00cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74b5ed8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f74b00c3de0 0x7f74b00cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55638/0 (socket says 172.21.15.40:55638) 2024-09-13T17:57:15.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74b5ed8700 1 -- 172.21.15.40:0/286798926 learned_addr learned my addr 172.21.15.40:0/286798926 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:15.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74b4ed6700 1 --2- 172.21.15.40:0/286798926 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f74b006dde0 0x7f74b006e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74b5ed8700 1 -- 172.21.15.40:0/286798926 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f74b006dde0 msgr2=0x7f74b006e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:15.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74b5ed8700 1 --2- 172.21.15.40:0/286798926 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f74b006dde0 0x7f74b006e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74b5ed8700 1 -- 172.21.15.40:0/286798926 >> v1:172.21.15.43:6789/0 conn(0x7f74b01515a0 legacy=0x7f74b015a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:15.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74b5ed8700 1 -- 172.21.15.40:0/286798926 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b00c2020 con 0x7f74b00c3de0 2024-09-13T17:57:15.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74b5ed8700 1 --2- 172.21.15.40:0/286798926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b00c3de0 0x7f74b00cd3d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f74a8009e50 tx=0x7f74a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d579861be4c4838e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:15.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74a7fff700 1 -- 172.21.15.40:0/286798926 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8003c40 con 0x7f74b00c3de0 2024-09-13T17:57:15.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74a7fff700 1 -- 172.21.15.40:0/286798926 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a8003da0 con 0x7f74b00c3de0 2024-09-13T17:57:15.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74a7fff700 1 -- 172.21.15.40:0/286798926 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a800bbc0 con 0x7f74b00c3de0 2024-09-13T17:57:15.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74b66d9700 1 -- 172.21.15.40:0/286798926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b00c3de0 msgr2=0x7f74b00cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.369+0000 7f74b66d9700 1 --2- 172.21.15.40:0/286798926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b00c3de0 0x7f74b00cd3d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f74a8009e50 tx=0x7f74a800b330 comp rx=0 tx=0).stop 2024-09-13T17:57:15.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.373+0000 7f74b66d9700 1 -- 172.21.15.40:0/286798926 shutdown_connections 2024-09-13T17:57:15.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.373+0000 7f74b66d9700 1 --2- 172.21.15.40:0/286798926 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b00c3de0 0x7f74b00cd3d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.373+0000 7f74b66d9700 1 --2- 172.21.15.40:0/286798926 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f74b006dde0 0x7f74b006e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.373+0000 7f74b66d9700 1 -- 172.21.15.40:0/286798926 >> 172.21.15.40:0/286798926 conn(0x7f74b0072070 msgr2=0x7f74b0072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:15.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.373+0000 7f74b66d9700 1 -- 172.21.15.40:0/286798926 shutdown_connections 2024-09-13T17:57:15.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.373+0000 7f74b66d9700 1 -- 172.21.15.40:0/286798926 wait complete. 2024-09-13T17:57:15.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.373+0000 7f74b66d9700 1 Processor -- start 2024-09-13T17:57:15.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.373+0000 7f74b66d9700 1 -- start start 2024-09-13T17:57:15.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.373+0000 7f74b66d9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f74b006dde0 0x7f74b013ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.373+0000 7f74b66d9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b00c3de0 0x7f74b006c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.373+0000 7f74b66d9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74b01515a0 0x7f74b006cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.373+0000 7f74b66d9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b014b7d0 con 0x7f74b00c3de0 2024-09-13T17:57:15.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.373+0000 7f74b66d9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b0149fe0 con 0x7f74b01515a0 2024-09-13T17:57:15.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.373+0000 7f74b66d9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b014a280 con 0x7f74b006dde0 2024-09-13T17:57:15.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.373+0000 7f74b4ed6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b00c3de0 0x7f74b006c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.373+0000 7f74b56d7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f74b006dde0 0x7f74b013ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.373+0000 7f74b4ed6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b00c3de0 0x7f74b006c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55650/0 (socket says 172.21.15.40:55650) 2024-09-13T17:57:15.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.373+0000 7f74b4ed6700 1 -- 172.21.15.40:0/365340183 learned_addr learned my addr 172.21.15.40:0/365340183 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:15.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.373+0000 7f74b5ed8700 1 --2- 172.21.15.40:0/365340183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74b01515a0 0x7f74b006cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.377+0000 7f74b4ed6700 1 -- 172.21.15.40:0/365340183 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f74b006dde0 msgr2=0x7f74b013ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.377+0000 7f74b4ed6700 1 --2- 172.21.15.40:0/365340183 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f74b006dde0 0x7f74b013ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.377+0000 7f74b4ed6700 1 -- 172.21.15.40:0/365340183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74b01515a0 msgr2=0x7f74b006cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.377+0000 7f74b4ed6700 1 --2- 172.21.15.40:0/365340183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74b01515a0 0x7f74b006cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.377+0000 7f74b4ed6700 1 -- 172.21.15.40:0/365340183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a8009b30 con 0x7f74b00c3de0 2024-09-13T17:57:15.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.377+0000 7f74b4ed6700 1 --2- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b00c3de0 0x7f74b006c620 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f74a000e7e0 tx=0x7f74a000eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:15.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.377+0000 7f74a5ffb700 1 -- 172.21.15.40:0/365340183 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a000c070 con 0x7f74b00c3de0 2024-09-13T17:57:15.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.377+0000 7f74a5ffb700 1 -- 172.21.15.40:0/365340183 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a0013610 con 0x7f74b00c3de0 2024-09-13T17:57:15.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.377+0000 7f74b66d9700 1 -- 172.21.15.40:0/365340183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b014ac50 con 0x7f74b00c3de0 2024-09-13T17:57:15.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.377+0000 7f74a5ffb700 1 -- 172.21.15.40:0/365340183 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a00139a0 con 0x7f74b00c3de0 2024-09-13T17:57:15.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.377+0000 7f74b66d9700 1 -- 172.21.15.40:0/365340183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b014b210 con 0x7f74b00c3de0 2024-09-13T17:57:15.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.377+0000 7f74b66d9700 1 --2- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b014b3d0 0x7f74b01ee8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.377+0000 7f74b66d9700 1 -- 172.21.15.40:0/365340183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b0074750 con 0x7f74b014b3d0 2024-09-13T17:57:15.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.377+0000 7f74b56d7700 1 --2- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b014b3d0 0x7f74b01ee8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.377+0000 7f74b56d7700 1 -- 172.21.15.40:0/365340183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74b006ffe0 con 0x7f74b014b3d0 2024-09-13T17:57:15.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.377+0000 7f74b56d7700 1 --2- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b014b3d0 0x7f74b01ee8c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00e7a0 tx=0x7f74ac00eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:15.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.377+0000 7f74a5ffb700 1 -- 172.21.15.40:0/365340183 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f74a0013b00 con 0x7f74b00c3de0 2024-09-13T17:57:15.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.381+0000 7f74a5ffb700 1 --2- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7494066730 0x7f7494068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.381+0000 7f74a5ffb700 1 -- 172.21.15.40:0/365340183 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac004080 con 0x7f74b014b3d0 2024-09-13T17:57:15.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.381+0000 7f74b5ed8700 1 --2- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7494066730 0x7f7494068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.381+0000 7f74a5ffb700 1 -- 172.21.15.40:0/365340183 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f74a00841c0 con 0x7f74b00c3de0 2024-09-13T17:57:15.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.381+0000 7f74a5ffb700 1 -- 172.21.15.40:0/365340183 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f74ac0041e0 con 0x7f74b014b3d0 2024-09-13T17:57:15.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.381+0000 7f74a5ffb700 1 -- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b014b3d0 msgr2=0x7f74b01ee8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.381+0000 7f74a5ffb700 1 --2- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b014b3d0 0x7f74b01ee8c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00e7a0 tx=0x7f74ac00eb60 comp rx=0 tx=0).stop 2024-09-13T17:57:15.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.381+0000 7f74a5ffb700 1 -- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b014b3d0 msgr2=0x7f74b01ee8c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:57:15.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.381+0000 7f74a5ffb700 1 --2- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b014b3d0 0x7f74b01ee8c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.381+0000 7f74b5ed8700 1 --2- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7494066730 0x7f7494068bf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f74a800b370 tx=0x7f74a8014000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:15.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.389+0000 7f74b66d9700 1 --2- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b006e740 0x7f74b01eee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.389+0000 7f74b66d9700 1 -- 172.21.15.40:0/365340183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b0158020 con 0x7f74b006e740 2024-09-13T17:57:15.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.389+0000 7f74b56d7700 1 --2- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b006e740 0x7f74b01eee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.389+0000 7f74b56d7700 1 -- 172.21.15.40:0/365340183 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f74b0157d40 con 0x7f74b006e740 2024-09-13T17:57:15.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.393+0000 7f74b56d7700 1 --2- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b006e740 0x7f74b01eee00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00e7a0 tx=0x7f74ac0047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:15.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.393+0000 7f74a5ffb700 1 -- 172.21.15.40:0/365340183 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac005e00 con 0x7f74b006e740 2024-09-13T17:57:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.393+0000 7f74a5ffb700 1 -- 172.21.15.40:0/365340183 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4950 (secure 0 0 0) 0x7f74ac003b30 con 0x7f74b006e740 2024-09-13T17:57:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.393+0000 7f74a5ffb700 1 -- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b006e740 msgr2=0x7f74b01eee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.393+0000 7f74a5ffb700 1 --2- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b006e740 0x7f74b01eee00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00e7a0 tx=0x7f74ac0047a0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.393+0000 7f74a5ffb700 1 -- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b006e740 msgr2=0x7f74b01eee00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:57:15.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.393+0000 7f74a5ffb700 1 --2- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b006e740 0x7f74b01eee00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.398 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:57:15.399 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:57:15.399 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:57:15.399 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:57:15.399 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 52, 2024-09-13T17:57:15.399 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:57:15.399 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:57:15.399 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:57:15.399 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:57:15.399 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:15.399 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 13, 2024-09-13T17:57:15.399 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:57:15.399 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:57:15.400 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:57:15.400 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:15.400 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:15.400 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:15.400 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:15.400 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:15.400 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:15.400 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:15.400 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:15.400 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:15.400 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:15.400 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:15.400 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:57:15.401 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:57:15.401 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:15.401 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:15.401 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:15.401 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:15.401 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:15.401 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:15.401 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:15.401 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:15.401 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:15.401 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:15.401 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:15.401 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.401 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:57:15.402 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:57:15.402 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:57:15.402 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:57:15.402 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:57:15.402 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:57:15.402 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:57:15.402 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:57:15.402 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:57:15.402 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:57:15.402 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:57:15.402 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:57:15.402 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:57:15.402 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:57:15.403 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:57:15.403 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:57:15.403 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:57:15.403 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:15.403 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:15.403 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:15.403 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:15.403 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:15.403 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:15.403 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:15.403 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:15.403 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:15.403 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:15.403 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:15.403 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:57:15.404 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.404 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:57:15.404 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.404 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:57:15.404 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:57:15.404 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:15.404 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:15.404 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.404 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:15.404 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:57:15.404 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:15.404 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.404 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.404 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:15.405 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:57:15.405 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:15.405 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.405 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:15.405 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.405 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:57:15.405 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:57:15.405 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:15.405 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:15.405 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:15.405 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.405 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.405 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:57:15.405 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:57:15.406 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:15.406 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:15.406 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.406 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:15.406 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:57:15.406 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:15.406 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.406 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.406 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:15.406 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:57:15.406 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:15.406 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.406 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:15.407 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.407 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:57:15.407 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:57:15.407 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:15.407 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:15.407 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:15.407 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.407 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.407 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:57:15.407 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:57:15.407 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:15.407 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:15.407 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.407 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:15.408 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:57:15.408 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:15.408 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.408 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.408 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:15.408 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:57:15.408 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:15.408 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.408 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:15.408 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.408 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:57:15.408 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:57:15.408 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:15.409 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:15.409 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:15.409 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.409 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:15.409 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.409 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:57:15.409 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:57:15.409 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.409 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:15.409 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:15.409 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:57:15.409 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.409 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:15.409 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:57:15.410 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.410 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:15.410 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:15.410 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:57:15.410 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.410 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:15.410 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:57:15.410 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.410 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:15.410 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:15.410 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:57:15.410 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.410 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:15.410 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:57:15.411 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.411 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:15.411 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:15.411 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:57:15.411 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.411 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:15.411 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:57:15.411 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.411 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:15.411 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:15.411 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:57:15.411 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.411 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:15.411 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.412 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:57:15.412 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:57:15.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.393+0000 7f74b66d9700 1 -- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7494066730 msgr2=0x7f7494068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.393+0000 7f74b66d9700 1 --2- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7494066730 0x7f7494068bf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f74a800b370 tx=0x7f74a8014000 comp rx=0 tx=0).stop 2024-09-13T17:57:15.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.393+0000 7f74b66d9700 1 -- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b00c3de0 msgr2=0x7f74b006c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.393+0000 7f74b66d9700 1 --2- 172.21.15.40:0/365340183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74b00c3de0 0x7f74b006c620 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f74a000e7e0 tx=0x7f74a000eaf0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.393+0000 7f74b5ed8700 1 -- 172.21.15.40:0/365340183 reap_dead start 2024-09-13T17:57:15.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.393+0000 7f74b5ed8700 1 -- 172.21.15.40:0/365340183 reap_dead start 2024-09-13T17:57:15.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.393+0000 7f74b66d9700 1 -- 172.21.15.40:0/365340183 shutdown_connections 2024-09-13T17:57:15.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.393+0000 7f74b66d9700 1 -- 172.21.15.40:0/365340183 >> 172.21.15.40:0/365340183 conn(0x7f74b0072070 msgr2=0x7f74b00c25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:15.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.393+0000 7f74b66d9700 1 -- 172.21.15.40:0/365340183 shutdown_connections 2024-09-13T17:57:15.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.393+0000 7f74b66d9700 1 -- 172.21.15.40:0/365340183 wait complete. 2024-09-13T17:57:15.414 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:57:15.414 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T17:57:15.415 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:57:15.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.517+0000 7f9fa8b01700 1 Processor -- start 2024-09-13T17:57:15.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.517+0000 7f9fa8b01700 1 -- start start 2024-09-13T17:57:15.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa8b01700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9fa40cc620 0x7f9fa4153520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa8b01700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa4159780 con 0x7f9fa40cc240 2024-09-13T17:57:15.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa8b01700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa4159a40 con 0x7f9fa415d620 2024-09-13T17:57:15.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa8b01700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa4159d20 con 0x7f9fa40cc620 2024-09-13T17:57:15.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa259c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9fa40cc240 0x7f9fa415cf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46410/0 (socket says 172.21.15.40:46410) 2024-09-13T17:57:15.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa259c700 1 -- 172.21.15.40:0/1828895347 learned_addr learned my addr 172.21.15.40:0/1828895347 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:57:15.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa1d9b700 1 --2- 172.21.15.40:0/1828895347 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9fa40cc620 0x7f9fa4153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa159a700 1 -- 172.21.15.40:0/1828895347 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922046120 0 0) 0x7f9fa4159780 con 0x7f9fa40cc240 2024-09-13T17:57:15.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa159a700 1 -- 172.21.15.40:0/1828895347 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f8c0035d0 con 0x7f9fa40cc240 2024-09-13T17:57:15.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa159a700 1 -- 172.21.15.40:0/1828895347 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2078926577 0 0) 0x7f9fa4159a40 con 0x7f9fa415d620 2024-09-13T17:57:15.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa159a700 1 -- 172.21.15.40:0/1828895347 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fa4159780 con 0x7f9fa415d620 2024-09-13T17:57:15.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa1d9b700 1 -- 172.21.15.40:0/1828895347 >> v1:172.21.15.43:6789/0 conn(0x7f9fa415d620 legacy=0x7f9fa40c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa1d9b700 1 -- 172.21.15.40:0/1828895347 >> v1:172.21.15.40:6789/0 conn(0x7f9fa40cc240 legacy=0x7f9fa415cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa159a700 1 -- 172.21.15.40:0/1828895347 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1588623514 0 0) 0x7f9f8c0035d0 con 0x7f9fa40cc240 2024-09-13T17:57:15.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa1d9b700 1 -- 172.21.15.40:0/1828895347 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa415a000 con 0x7f9fa40cc620 2024-09-13T17:57:15.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa1d9b700 1 --2- 172.21.15.40:0/1828895347 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9fa40cc620 0x7f9fa4153520 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9f980098b0 tx=0x7f9f98009bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6ddfbdc540f19f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:15.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa159a700 1 -- 172.21.15.40:0/1828895347 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f94001670 con 0x7f9fa40cc620 2024-09-13T17:57:15.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa159a700 1 -- 172.21.15.40:0/1828895347 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f98004030 con 0x7f9fa40cc620 2024-09-13T17:57:15.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa159a700 1 -- 172.21.15.40:0/1828895347 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f980043e0 con 0x7f9fa40cc620 2024-09-13T17:57:15.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa8b01700 1 -- 172.21.15.40:0/1828895347 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9fa40cc620 msgr2=0x7f9fa4153520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa8b01700 1 --2- 172.21.15.40:0/1828895347 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9fa40cc620 0x7f9fa4153520 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9f980098b0 tx=0x7f9f98009bc0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa8b01700 1 -- 172.21.15.40:0/1828895347 shutdown_connections 2024-09-13T17:57:15.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa8b01700 1 --2- 172.21.15.40:0/1828895347 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9fa40cc620 0x7f9fa4153520 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa8b01700 1 -- 172.21.15.40:0/1828895347 >> 172.21.15.40:0/1828895347 conn(0x7f9fa4066480 msgr2=0x7f9fa4066890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:15.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa8b01700 1 -- 172.21.15.40:0/1828895347 shutdown_connections 2024-09-13T17:57:15.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.521+0000 7f9fa8b01700 1 -- 172.21.15.40:0/1828895347 wait complete. 2024-09-13T17:57:15.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa8b01700 1 Processor -- start 2024-09-13T17:57:15.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa8b01700 1 -- start start 2024-09-13T17:57:15.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa8b01700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa40cc240 0x7f9fa414fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa8b01700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa40cc620 0x7f9fa41e3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa8b01700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9fa415d620 0x7f9fa40cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa8b01700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa40cf440 con 0x7f9fa40cc240 2024-09-13T17:57:15.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa8b01700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa4149410 con 0x7f9fa40cc620 2024-09-13T17:57:15.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa8b01700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa41506f0 con 0x7f9fa415d620 2024-09-13T17:57:15.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa40cc240 0x7f9fa414fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa40cc240 0x7f9fa414fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55678/0 (socket says 172.21.15.40:55678) 2024-09-13T17:57:15.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa259c700 1 -- 172.21.15.40:0/1012809424 learned_addr learned my addr 172.21.15.40:0/1012809424 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:15.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa2d9d700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9fa415d620 0x7f9fa40cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa1d9b700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa40cc620 0x7f9fa41e3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa259c700 1 -- 172.21.15.40:0/1012809424 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9fa415d620 msgr2=0x7f9fa40cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa259c700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9fa415d620 0x7f9fa40cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa259c700 1 -- 172.21.15.40:0/1012809424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa40cc620 msgr2=0x7f9fa41e3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa259c700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa40cc620 0x7f9fa41e3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa259c700 1 -- 172.21.15.40:0/1012809424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f980094b0 con 0x7f9fa40cc240 2024-09-13T17:57:15.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa259c700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa40cc240 0x7f9fa414fd60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400c6d0 tx=0x7f9f9400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:15.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9f92ffd700 1 -- 172.21.15.40:0/1012809424 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f94004ca0 con 0x7f9fa40cc240 2024-09-13T17:57:15.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9f92ffd700 1 -- 172.21.15.40:0/1012809424 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f94004e00 con 0x7f9fa40cc240 2024-09-13T17:57:15.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa8b01700 1 -- 172.21.15.40:0/1012809424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa41502a0 con 0x7f9fa40cc240 2024-09-13T17:57:15.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9f92ffd700 1 -- 172.21.15.40:0/1012809424 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f94007310 con 0x7f9fa40cc240 2024-09-13T17:57:15.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.525+0000 7f9fa8b01700 1 -- 172.21.15.40:0/1012809424 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa40cf7a0 con 0x7f9fa40cc240 2024-09-13T17:57:15.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.529+0000 7f9fa8b01700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa40cfb40 0x7f9fa40cda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.529+0000 7f9fa8b01700 1 -- 172.21.15.40:0/1012809424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa4067520 con 0x7f9fa40cfb40 2024-09-13T17:57:15.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.529+0000 7f9fa1d9b700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa40cfb40 0x7f9fa40cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.529+0000 7f9fa1d9b700 1 -- 172.21.15.40:0/1012809424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9fa40b1fc0 con 0x7f9fa40cfb40 2024-09-13T17:57:15.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.529+0000 7f9fa1d9b700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa40cfb40 0x7f9fa40cda20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4074470 tx=0x7f9f98004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:15.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.529+0000 7f9f92ffd700 1 -- 172.21.15.40:0/1012809424 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9f94007470 con 0x7f9fa40cc240 2024-09-13T17:57:15.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.529+0000 7f9f92ffd700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9f84066730 0x7f9f84068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.529+0000 7f9fa2d9d700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9f84066730 0x7f9f84068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.529+0000 7f9f92ffd700 1 -- 172.21.15.40:0/1012809424 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9f94084150 con 0x7f9fa40cc240 2024-09-13T17:57:15.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.529+0000 7f9f92ffd700 1 -- 172.21.15.40:0/1012809424 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f98006ea0 con 0x7f9fa40cfb40 2024-09-13T17:57:15.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.529+0000 7f9f92ffd700 1 -- 172.21.15.40:0/1012809424 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9f98003e20 con 0x7f9fa40cfb40 2024-09-13T17:57:15.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.529+0000 7f9f92ffd700 1 -- 172.21.15.40:0/1012809424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa40cfb40 msgr2=0x7f9fa40cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.529+0000 7f9f92ffd700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa40cfb40 0x7f9fa40cda20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4074470 tx=0x7f9f98004770 comp rx=0 tx=0).stop 2024-09-13T17:57:15.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.529+0000 7f9f92ffd700 1 -- 172.21.15.40:0/1012809424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa40cfb40 msgr2=0x7f9fa40cda20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:57:15.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.529+0000 7f9f92ffd700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa40cfb40 0x7f9fa40cda20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.533+0000 7f9fa2d9d700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9f84066730 0x7f9f84068bf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9fa414cb10 tx=0x7f9f9c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:15.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.541+0000 7f9fa8b01700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa41f2cf0 0x7f9fa41f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.541+0000 7f9fa8b01700 1 -- 172.21.15.40:0/1012809424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa40b2c10 con 0x7f9fa41f2cf0 2024-09-13T17:57:15.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.541+0000 7f9fa1d9b700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa41f2cf0 0x7f9fa41f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.541+0000 7f9fa1d9b700 1 -- 172.21.15.40:0/1012809424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9fa4074240 con 0x7f9fa41f2cf0 2024-09-13T17:57:15.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.541+0000 7f9fa1d9b700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa41f2cf0 0x7f9fa41f30d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9fa40ca440 tx=0x7f9f98011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:15.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.541+0000 7f9f92ffd700 1 -- 172.21.15.40:0/1012809424 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9801bbe0 con 0x7f9fa41f2cf0 2024-09-13T17:57:15.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.545+0000 7f9f92ffd700 1 -- 172.21.15.40:0/1012809424 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4450 (secure 0 0 0) 0x7f9f9801bd40 con 0x7f9fa41f2cf0 2024-09-13T17:57:15.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.545+0000 7f9f92ffd700 1 -- 172.21.15.40:0/1012809424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa41f2cf0 msgr2=0x7f9fa41f30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.545+0000 7f9f92ffd700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa41f2cf0 0x7f9fa41f30d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9fa40ca440 tx=0x7f9f98011000 comp rx=0 tx=0).stop 2024-09-13T17:57:15.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.545+0000 7f9f92ffd700 1 -- 172.21.15.40:0/1012809424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa41f2cf0 msgr2=0x7f9fa41f30d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:57:15.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.545+0000 7f9f92ffd700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fa41f2cf0 0x7f9fa41f30d0 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9fa40ca440 tx=0x7f9f98011000 comp rx=0 tx=0).stop 2024-09-13T17:57:15.550 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:57:15.550 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:57:15.550 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:57:15.550 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:57:15.551 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 52, 2024-09-13T17:57:15.551 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:57:15.551 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:57:15.551 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:57:15.551 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:57:15.551 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:15.551 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 13, 2024-09-13T17:57:15.551 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:57:15.551 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:57:15.551 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:57:15.551 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:15.551 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:15.551 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:15.551 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:15.552 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:15.552 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:15.552 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:15.552 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:15.552 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:15.552 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:15.552 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:15.552 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:57:15.552 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:57:15.552 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:15.552 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:15.552 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:15.552 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:15.553 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:15.553 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:15.553 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:15.553 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:15.553 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:15.553 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:15.553 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:15.553 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.553 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:57:15.553 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:57:15.553 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:57:15.553 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:57:15.553 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:57:15.553 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:57:15.554 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:57:15.554 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:57:15.554 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:57:15.554 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:57:15.554 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:57:15.554 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:57:15.554 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:57:15.554 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:57:15.554 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:57:15.554 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:57:15.554 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:57:15.554 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:15.554 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:15.554 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:15.555 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:15.555 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:15.555 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:15.555 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:15.555 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:15.555 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:15.555 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:15.555 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:15.555 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:57:15.555 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.555 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:57:15.555 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.555 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:57:15.555 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:57:15.555 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:15.556 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:15.556 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.556 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:15.556 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:57:15.556 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:15.556 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.556 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.556 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:15.556 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:57:15.556 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:15.556 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.556 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:15.556 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.556 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:57:15.557 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:57:15.557 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:15.557 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:15.557 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:15.557 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.557 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.557 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:57:15.557 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:57:15.557 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:15.557 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:15.557 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.557 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:15.557 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:57:15.557 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:15.557 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.558 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.558 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:15.558 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:57:15.558 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:15.558 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.558 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:15.558 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.558 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:57:15.559 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:57:15.559 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:15.559 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:15.559 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:15.559 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.559 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.559 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:57:15.559 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:57:15.559 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:15.559 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:15.559 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.559 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:15.559 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:57:15.559 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:15.560 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.560 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.560 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:15.560 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:57:15.560 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:15.560 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.560 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:15.560 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.560 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:57:15.560 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:57:15.560 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:15.560 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:15.560 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:15.561 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.561 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:15.561 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.561 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:57:15.561 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:57:15.561 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.561 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:15.561 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:15.561 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:57:15.561 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.561 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:15.561 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:57:15.561 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.561 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:15.562 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:15.562 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:57:15.562 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.562 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:15.562 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.562 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:57:15.562 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:57:15.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.545+0000 7f9fa8b01700 1 -- 172.21.15.40:0/1012809424 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9f84066730 msgr2=0x7f9f84068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.545+0000 7f9fa8b01700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9f84066730 0x7f9f84068bf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9fa414cb10 tx=0x7f9f9c009040 comp rx=0 tx=0).stop 2024-09-13T17:57:15.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.545+0000 7f9fa8b01700 1 -- 172.21.15.40:0/1012809424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa40cc240 msgr2=0x7f9fa414fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.545+0000 7f9fa8b01700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa40cc240 0x7f9fa414fd60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400c6d0 tx=0x7f9f9400c9e0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.545+0000 7f9fa2d9d700 1 -- 172.21.15.40:0/1012809424 reap_dead start 2024-09-13T17:57:15.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.545+0000 7f9fa8b01700 1 -- 172.21.15.40:0/1012809424 shutdown_connections 2024-09-13T17:57:15.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.545+0000 7f9fa8b01700 1 --2- 172.21.15.40:0/1012809424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9fa40cc240 0x7f9fa414fd60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.545+0000 7f9fa8b01700 1 -- 172.21.15.40:0/1012809424 >> 172.21.15.40:0/1012809424 conn(0x7f9fa4066480 msgr2=0x7f9fa41573e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:15.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.545+0000 7f9fa8b01700 1 -- 172.21.15.40:0/1012809424 shutdown_connections 2024-09-13T17:57:15.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.545+0000 7f9fa8b01700 1 -- 172.21.15.40:0/1012809424 wait complete. 2024-09-13T17:57:15.566 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:57:15.566 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T17:57:15.566 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:57:15.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.677+0000 7fd5aa4c3700 1 Processor -- start 2024-09-13T17:57:15.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.677+0000 7fd5aa4c3700 1 -- start start 2024-09-13T17:57:15.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.677+0000 7fd5aa4c3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a406dde0 0x7fd5a406e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.677+0000 7fd5aa4c3700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd5a40c3de0 0x7fd5a40cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.677+0000 7fd5aa4c3700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5a406e580 con 0x7fd5a41515a0 2024-09-13T17:57:15.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.677+0000 7fd5aa4c3700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a4151980 con 0x7fd5a40c3de0 2024-09-13T17:57:15.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.677+0000 7fd5aa4c3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a40c3930 con 0x7fd5a406dde0 2024-09-13T17:57:15.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd5a94c1700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd5a41515a0 0x7fd5a415a810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46414/0 (socket says 172.21.15.40:46414) 2024-09-13T17:57:15.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd5a94c1700 1 -- 172.21.15.40:0/3762358924 learned_addr learned my addr 172.21.15.40:0/3762358924 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:57:15.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd5a9cc2700 1 --2- 172.21.15.40:0/3762358924 >> v2:172.21.15.40:3300/0 conn(0x7fd5a40c3de0 0x7fd5a40cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd5a8cc0700 1 --2- 172.21.15.40:0/3762358924 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a406dde0 0x7fd5a406e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd59bfff700 1 -- 172.21.15.40:0/3762358924 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1356683531 0 0) 0x7fd5a406e580 con 0x7fd5a41515a0 2024-09-13T17:57:15.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd59bfff700 1 -- 172.21.15.40:0/3762358924 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5900035d0 con 0x7fd5a41515a0 2024-09-13T17:57:15.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd5a9cc2700 1 -- 172.21.15.40:0/3762358924 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a406dde0 msgr2=0x7fd5a406e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:15.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd5a9cc2700 1 --2- 172.21.15.40:0/3762358924 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a406dde0 0x7fd5a406e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd5a9cc2700 1 -- 172.21.15.40:0/3762358924 >> v1:172.21.15.40:6789/0 conn(0x7fd5a41515a0 legacy=0x7fd5a415a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd5a9cc2700 1 -- 172.21.15.40:0/3762358924 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a40c2020 con 0x7fd5a40c3de0 2024-09-13T17:57:15.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd5a9cc2700 1 --2- 172.21.15.40:0/3762358924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5a40c3de0 0x7fd5a40cd3d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009d90 tx=0x7fd59c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cc92f73a5a3edb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:15.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd59bfff700 1 -- 172.21.15.40:0/3762358924 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c00e070 con 0x7fd5a40c3de0 2024-09-13T17:57:15.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd59bfff700 1 -- 172.21.15.40:0/3762358924 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c00bb20 con 0x7fd5a40c3de0 2024-09-13T17:57:15.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd59bfff700 1 -- 172.21.15.40:0/3762358924 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c012430 con 0x7fd5a40c3de0 2024-09-13T17:57:15.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd5aa4c3700 1 -- 172.21.15.40:0/3762358924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5a40c3de0 msgr2=0x7fd5a40cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd5aa4c3700 1 --2- 172.21.15.40:0/3762358924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5a40c3de0 0x7fd5a40cd3d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009d90 tx=0x7fd59c00b330 comp rx=0 tx=0).stop 2024-09-13T17:57:15.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd5aa4c3700 1 -- 172.21.15.40:0/3762358924 shutdown_connections 2024-09-13T17:57:15.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd5aa4c3700 1 --2- 172.21.15.40:0/3762358924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5a40c3de0 0x7fd5a40cd3d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd5aa4c3700 1 --2- 172.21.15.40:0/3762358924 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a406dde0 0x7fd5a406e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd5aa4c3700 1 -- 172.21.15.40:0/3762358924 >> 172.21.15.40:0/3762358924 conn(0x7fd5a4072070 msgr2=0x7fd5a4072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:15.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd5aa4c3700 1 -- 172.21.15.40:0/3762358924 shutdown_connections 2024-09-13T17:57:15.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.681+0000 7fd5aa4c3700 1 -- 172.21.15.40:0/3762358924 wait complete. 2024-09-13T17:57:15.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5aa4c3700 1 Processor -- start 2024-09-13T17:57:15.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5aa4c3700 1 -- start start 2024-09-13T17:57:15.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5aa4c3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5a406dde0 0x7fd5a40c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5aa4c3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a40c3de0 0x7fd5a41eab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5aa4c3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5a41515a0 0x7fd5a41eb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5aa4c3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a40c7350 con 0x7fd5a41515a0 2024-09-13T17:57:15.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5aa4c3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a41efc10 con 0x7fd5a406dde0 2024-09-13T17:57:15.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5aa4c3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a41eed60 con 0x7fd5a40c3de0 2024-09-13T17:57:15.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5a9cc2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5a41515a0 0x7fd5a41eb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5a9cc2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5a41515a0 0x7fd5a41eb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55696/0 (socket says 172.21.15.40:55696) 2024-09-13T17:57:15.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5a9cc2700 1 -- 172.21.15.40:0/1536658911 learned_addr learned my addr 172.21.15.40:0/1536658911 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:15.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5a8cc0700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a40c3de0 0x7fd5a41eab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5a94c1700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5a406dde0 0x7fd5a40c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5a9cc2700 1 -- 172.21.15.40:0/1536658911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a40c3de0 msgr2=0x7fd5a41eab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5a9cc2700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a40c3de0 0x7fd5a41eab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5a9cc2700 1 -- 172.21.15.40:0/1536658911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5a406dde0 msgr2=0x7fd5a40c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5a9cc2700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5a406dde0 0x7fd5a40c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5a9cc2700 1 -- 172.21.15.40:0/1536658911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59c009a70 con 0x7fd5a41515a0 2024-09-13T17:57:15.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5a9cc2700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5a41515a0 0x7fd5a41eb050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00b370 tx=0x7fd59c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:15.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd599ffb700 1 -- 172.21.15.40:0/1536658911 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c012430 con 0x7fd5a41515a0 2024-09-13T17:57:15.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd599ffb700 1 -- 172.21.15.40:0/1536658911 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c020070 con 0x7fd5a41515a0 2024-09-13T17:57:15.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5aa4c3700 1 -- 172.21.15.40:0/1536658911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a41ef000 con 0x7fd5a41515a0 2024-09-13T17:57:15.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd599ffb700 1 -- 172.21.15.40:0/1536658911 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c0098f0 con 0x7fd5a41515a0 2024-09-13T17:57:15.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.685+0000 7fd5aa4c3700 1 -- 172.21.15.40:0/1536658911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a41ef5c0 con 0x7fd5a41515a0 2024-09-13T17:57:15.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.689+0000 7fd5aa4c3700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a41ef780 0x7fd5a414a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.689+0000 7fd5aa4c3700 1 -- 172.21.15.40:0/1536658911 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a4074750 con 0x7fd5a41ef780 2024-09-13T17:57:15.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.689+0000 7fd5a94c1700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a41ef780 0x7fd5a414a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.689+0000 7fd599ffb700 1 -- 172.21.15.40:0/1536658911 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd59c004040 con 0x7fd5a41515a0 2024-09-13T17:57:15.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.689+0000 7fd5a94c1700 1 -- 172.21.15.40:0/1536658911 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5a4158020 con 0x7fd5a41ef780 2024-09-13T17:57:15.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.689+0000 7fd599ffb700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd588066730 0x7fd588068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.689+0000 7fd5a8cc0700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd588066730 0x7fd588068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.689+0000 7fd599ffb700 1 -- 172.21.15.40:0/1536658911 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd59c022070 con 0x7fd5a41515a0 2024-09-13T17:57:15.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.689+0000 7fd5a94c1700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a41ef780 0x7fd5a414a240 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd5a00011e0 tx=0x7fd5a00018b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:15.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.689+0000 7fd599ffb700 1 -- 172.21.15.40:0/1536658911 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0012560 con 0x7fd5a41ef780 2024-09-13T17:57:15.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.689+0000 7fd5a8cc0700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd588066730 0x7fd588068bf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd5a406ff80 tx=0x7fd594009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:15.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.689+0000 7fd599ffb700 1 -- 172.21.15.40:0/1536658911 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd5a00126c0 con 0x7fd5a41ef780 2024-09-13T17:57:15.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.689+0000 7fd599ffb700 1 -- 172.21.15.40:0/1536658911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a41ef780 msgr2=0x7fd5a414a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.689+0000 7fd599ffb700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a41ef780 0x7fd5a414a240 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd5a00011e0 tx=0x7fd5a00018b0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.689+0000 7fd599ffb700 1 -- 172.21.15.40:0/1536658911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a41ef780 msgr2=0x7fd5a414a240 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:57:15.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.689+0000 7fd599ffb700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a41ef780 0x7fd5a414a240 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.701+0000 7fd5aa4c3700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a406e740 0x7fd5a414a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.701+0000 7fd5aa4c3700 1 -- 172.21.15.40:0/1536658911 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a4157d40 con 0x7fd5a406e740 2024-09-13T17:57:15.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.701+0000 7fd5a94c1700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a406e740 0x7fd5a414a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.701+0000 7fd5a94c1700 1 -- 172.21.15.40:0/1536658911 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd5a40c7880 con 0x7fd5a406e740 2024-09-13T17:57:15.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.701+0000 7fd5a94c1700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a406e740 0x7fd5a414a780 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0006bb0 tx=0x7fd5a0001150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:15.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.701+0000 7fd599ffb700 1 -- 172.21.15.40:0/1536658911 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0022070 con 0x7fd5a406e740 2024-09-13T17:57:15.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.701+0000 7fd599ffb700 1 -- 172.21.15.40:0/1536658911 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4782 (secure 0 0 0) 0x7fd5a0004bb0 con 0x7fd5a406e740 2024-09-13T17:57:15.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.701+0000 7fd599ffb700 1 -- 172.21.15.40:0/1536658911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a406e740 msgr2=0x7fd5a414a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.701+0000 7fd599ffb700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a406e740 0x7fd5a414a780 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0006bb0 tx=0x7fd5a0001150 comp rx=0 tx=0).stop 2024-09-13T17:57:15.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.701+0000 7fd599ffb700 1 -- 172.21.15.40:0/1536658911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a406e740 msgr2=0x7fd5a414a780 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:57:15.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.701+0000 7fd599ffb700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd5a406e740 0x7fd5a414a780 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0006bb0 tx=0x7fd5a0001150 comp rx=0 tx=0).stop 2024-09-13T17:57:15.709 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:57:15.709 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:57:15.709 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:57:15.709 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:57:15.709 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 52, 2024-09-13T17:57:15.709 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:57:15.710 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:57:15.710 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:57:15.710 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:57:15.710 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:15.710 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 14, 2024-09-13T17:57:15.710 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:57:15.710 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:57:15.710 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:57:15.710 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:15.710 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:15.710 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:15.710 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:15.710 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:15.710 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:15.711 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:15.711 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:15.711 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:15.711 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:15.711 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:15.711 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:57:15.711 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:57:15.711 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:15.711 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:15.711 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:15.711 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:15.711 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:15.711 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:15.711 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:15.712 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:15.712 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:15.712 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:15.712 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:15.712 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.712 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:57:15.712 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:57:15.712 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:57:15.712 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:57:15.712 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:57:15.712 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:57:15.712 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:57:15.712 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:57:15.712 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:57:15.713 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:57:15.713 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:57:15.713 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:57:15.713 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:57:15.713 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:57:15.713 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:57:15.713 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:57:15.713 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:57:15.713 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:15.713 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:15.713 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:15.713 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:15.713 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:15.714 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:15.714 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:15.714 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:15.714 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:15.714 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:15.714 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:15.714 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:57:15.714 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.714 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:57:15.714 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.714 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:57:15.714 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:57:15.714 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:15.714 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:15.715 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.715 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:15.715 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:57:15.715 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:15.715 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.715 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.715 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:15.715 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:57:15.715 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:15.715 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.715 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:15.715 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.715 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:57:15.715 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:57:15.716 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:15.716 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:15.716 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:15.716 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.716 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.716 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:57:15.716 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:57:15.716 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:15.716 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:15.716 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.716 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:15.716 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:57:15.716 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:15.717 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.717 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.717 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:15.717 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:57:15.717 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:15.717 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.717 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:15.717 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.717 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:57:15.717 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:57:15.720 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:15.720 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:15.720 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:15.720 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.720 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.720 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:57:15.721 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:57:15.721 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:15.721 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:15.721 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.721 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:15.721 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:57:15.721 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:15.721 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.721 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.721 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:15.721 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:57:15.721 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:15.721 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.722 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:15.722 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.722 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:57:15.722 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:57:15.722 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:15.722 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:15.722 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:15.722 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.722 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:15.722 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.722 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:57:15.722 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:57:15.722 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.722 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:15.723 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:15.723 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:57:15.723 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.723 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:15.723 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:57:15.723 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.723 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:15.723 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:15.723 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T17:57:15.723 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.723 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:15.723 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:57:15.723 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.723 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:15.723 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:15.724 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T17:57:15.724 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.724 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:15.724 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:57:15.724 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:15.724 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:15.724 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:15.724 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:57:15.724 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:15.724 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:15.724 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:15.724 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:57:15.724 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:57:15.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.701+0000 7fd5aa4c3700 1 -- 172.21.15.40:0/1536658911 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd588066730 msgr2=0x7fd588068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.701+0000 7fd5aa4c3700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd588066730 0x7fd588068bf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd5a406ff80 tx=0x7fd594009240 comp rx=0 tx=0).stop 2024-09-13T17:57:15.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.701+0000 7fd5aa4c3700 1 -- 172.21.15.40:0/1536658911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5a41515a0 msgr2=0x7fd5a41eb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.701+0000 7fd5a9cc2700 1 -- 172.21.15.40:0/1536658911 reap_dead start 2024-09-13T17:57:15.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.701+0000 7fd5aa4c3700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5a41515a0 0x7fd5a41eb050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00b370 tx=0x7fd59c004af0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.705+0000 7fd5aa4c3700 1 -- 172.21.15.40:0/1536658911 shutdown_connections 2024-09-13T17:57:15.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.705+0000 7fd5aa4c3700 1 --2- 172.21.15.40:0/1536658911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5a41515a0 0x7fd5a41eb050 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.705+0000 7fd5aa4c3700 1 -- 172.21.15.40:0/1536658911 >> 172.21.15.40:0/1536658911 conn(0x7fd5a4072070 msgr2=0x7fd5a414edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:15.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.705+0000 7fd5aa4c3700 1 -- 172.21.15.40:0/1536658911 shutdown_connections 2024-09-13T17:57:15.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.705+0000 7fd5aa4c3700 1 -- 172.21.15.40:0/1536658911 wait complete. 2024-09-13T17:57:15.727 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:57:15.727 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-13T17:57:15.727 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T17:57:15.727 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-13T17:57:15.727 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T17:57:15.727 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T17:57:15.727 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:57:15.769 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T17:57:15.769 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T17:57:15.770 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:57:15.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.889+0000 7f82d2dd4700 1 Processor -- start 2024-09-13T17:57:15.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.889+0000 7f82d2dd4700 1 -- start start 2024-09-13T17:57:15.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d2dd4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f82c40d4ba0 0x7f82c404a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d2dd4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f82c404abc0 0x7f82c4006390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d2dd4700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c4046730 con 0x7f82c40dea90 2024-09-13T17:57:15.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d2dd4700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c4007020 con 0x7f82c404abc0 2024-09-13T17:57:15.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d2dd4700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c4046bc0 con 0x7f82c40d4ba0 2024-09-13T17:57:15.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d25d3700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f82c40dea90 legacy=0x7f82c40daaf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:57:15.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d15d1700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f82c404abc0 msgr2=0x7f82c4006390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:57:15.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d15d1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f82c404abc0 0x7f82c4006390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:57:15.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d1dd2700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f82c40d4ba0 0x7f82c404a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d1dd2700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f82c40d4ba0 0x7f82c404a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:55156/0 (socket says 172.21.15.40:55156) 2024-09-13T17:57:15.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d1dd2700 1 -- 172.21.15.40:0/1854154893 learned_addr learned my addr 172.21.15.40:0/1854154893 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:15.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d1dd2700 1 -- 172.21.15.40:0/1854154893 >> v2:172.21.15.40:3300/0 conn(0x7f82c404abc0 msgr2=0x7f82c4006390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:57:15.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d1dd2700 1 --2- 172.21.15.40:0/1854154893 >> v2:172.21.15.40:3300/0 conn(0x7f82c404abc0 0x7f82c4006390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d1dd2700 1 -- 172.21.15.40:0/1854154893 >> v1:172.21.15.40:6789/0 conn(0x7f82c40dea90 legacy=0x7f82c40daaf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:57:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d1dd2700 1 -- 172.21.15.40:0/1854154893 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c40defd0 con 0x7f82c40d4ba0 2024-09-13T17:57:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d1dd2700 1 --2- 172.21.15.40:0/1854154893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c40d4ba0 0x7f82c404a680 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f82c400d440 tx=0x7f82c8009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=572ad2239bea48c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d0dd0700 1 -- 172.21.15.40:0/1854154893 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c8004230 con 0x7f82c40d4ba0 2024-09-13T17:57:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d0dd0700 1 -- 172.21.15.40:0/1854154893 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c8004390 con 0x7f82c40d4ba0 2024-09-13T17:57:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d0dd0700 1 -- 172.21.15.40:0/1854154893 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c80036c0 con 0x7f82c40d4ba0 2024-09-13T17:57:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d2dd4700 1 -- 172.21.15.40:0/1854154893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c40d4ba0 msgr2=0x7f82c404a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d2dd4700 1 --2- 172.21.15.40:0/1854154893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c40d4ba0 0x7f82c404a680 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f82c400d440 tx=0x7f82c8009bf0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d2dd4700 1 -- 172.21.15.40:0/1854154893 shutdown_connections 2024-09-13T17:57:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d2dd4700 1 --2- 172.21.15.40:0/1854154893 >> v2:172.21.15.40:3300/0 conn(0x7f82c404abc0 0x7f82c4006390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d2dd4700 1 --2- 172.21.15.40:0/1854154893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c40d4ba0 0x7f82c404a680 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.893+0000 7f82d2dd4700 1 -- 172.21.15.40:0/1854154893 >> 172.21.15.40:0/1854154893 conn(0x7f82c404fd00 msgr2=0x7f82c4050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d2dd4700 1 -- 172.21.15.40:0/1854154893 shutdown_connections 2024-09-13T17:57:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d2dd4700 1 -- 172.21.15.40:0/1854154893 wait complete. 2024-09-13T17:57:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d2dd4700 1 Processor -- start 2024-09-13T17:57:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d2dd4700 1 -- start start 2024-09-13T17:57:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d2dd4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c404abc0 0x7f82c4035000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d2dd4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82c40d4ba0 0x7f82c4179830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d2dd4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82c40dea90 0x7f82c4049990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d2dd4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c417c370 con 0x7f82c40d4ba0 2024-09-13T17:57:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d2dd4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c416dd00 con 0x7f82c404abc0 2024-09-13T17:57:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d2dd4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c417b9d0 con 0x7f82c40dea90 2024-09-13T17:57:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d15d1700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82c40d4ba0 msgr2=0x7f82c4179830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:57:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d15d1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82c40d4ba0 0x7f82c4179830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:57:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d1dd2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c404abc0 0x7f82c4035000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d25d3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82c40dea90 0x7f82c4049990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d1dd2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c404abc0 0x7f82c4035000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:55172/0 (socket says 172.21.15.40:55172) 2024-09-13T17:57:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d1dd2700 1 -- 172.21.15.40:0/3894572657 learned_addr learned my addr 172.21.15.40:0/3894572657 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d25d3700 1 -- 172.21.15.40:0/3894572657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c404abc0 msgr2=0x7f82c4035000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d25d3700 1 --2- 172.21.15.40:0/3894572657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c404abc0 0x7f82c4035000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d25d3700 1 -- 172.21.15.40:0/3894572657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82c40d4ba0 msgr2=0x7f82c4179830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:57:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d25d3700 1 --2- 172.21.15.40:0/3894572657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82c40d4ba0 0x7f82c4179830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d25d3700 1 -- 172.21.15.40:0/3894572657 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c8009570 con 0x7f82c40dea90 2024-09-13T17:57:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d25d3700 1 --2- 172.21.15.40:0/3894572657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82c40dea90 0x7f82c4049990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f82cc063cf0 tx=0x7f82cc0680b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82c27fc700 1 -- 172.21.15.40:0/3894572657 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc0720c0 con 0x7f82c40dea90 2024-09-13T17:57:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82c27fc700 1 -- 172.21.15.40:0/3894572657 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82cc072240 con 0x7f82c40dea90 2024-09-13T17:57:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.897+0000 7f82d2dd4700 1 -- 172.21.15.40:0/3894572657 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c417bcb0 con 0x7f82c40dea90 2024-09-13T17:57:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.901+0000 7f82c27fc700 1 -- 172.21.15.40:0/3894572657 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc072630 con 0x7f82c40dea90 2024-09-13T17:57:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.901+0000 7f82d2dd4700 1 -- 172.21.15.40:0/3894572657 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c417c790 con 0x7f82c40dea90 2024-09-13T17:57:15.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.901+0000 7f82d2dd4700 1 -- 172.21.15.40:0/3894572657 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c417c4b0 con 0x7f82c40dea90 2024-09-13T17:57:15.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.901+0000 7f82c27fc700 1 -- 172.21.15.40:0/3894572657 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f82cc091020 con 0x7f82c40dea90 2024-09-13T17:57:15.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.901+0000 7f82c27fc700 1 --2- 172.21.15.40:0/3894572657 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f82b00666e0 0x7f82b0068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:15.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.901+0000 7f82c27fc700 1 -- 172.21.15.40:0/3894572657 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f82cc0fee40 con 0x7f82c40dea90 2024-09-13T17:57:15.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.901+0000 7f82d1dd2700 1 --2- 172.21.15.40:0/3894572657 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f82b00666e0 0x7f82b0068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:15.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.905+0000 7f82d1dd2700 1 --2- 172.21.15.40:0/3894572657 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f82b00666e0 0x7f82b0068ba0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009150 tx=0x7f82c8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:15.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:15.913+0000 7f82c27fc700 1 -- 172.21.15.40:0/3894572657 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f82cc0cf880 con 0x7f82c40dea90 2024-09-13T17:57:16.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:16.177+0000 7f82d2dd4700 1 -- 172.21.15.40:0/3894572657 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f82c417c4b0 con 0x7f82c40dea90 2024-09-13T17:57:16.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:16.177+0000 7f82c27fc700 1 -- 172.21.15.40:0/3894572657 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f82c417c4b0 con 0x7f82c40dea90 2024-09-13T17:57:16.183 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":52,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:57:16.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:16.177+0000 7f82d2dd4700 1 -- 172.21.15.40:0/3894572657 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f82b00666e0 msgr2=0x7f82b0068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:16.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:16.177+0000 7f82d2dd4700 1 --2- 172.21.15.40:0/3894572657 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f82b00666e0 0x7f82b0068ba0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009150 tx=0x7f82c8008030 comp rx=0 tx=0).stop 2024-09-13T17:57:16.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:16.177+0000 7f82d2dd4700 1 -- 172.21.15.40:0/3894572657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82c40dea90 msgr2=0x7f82c4049990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:16.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:16.177+0000 7f82d2dd4700 1 --2- 172.21.15.40:0/3894572657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82c40dea90 0x7f82c4049990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f82cc063cf0 tx=0x7f82cc0680b0 comp rx=0 tx=0).stop 2024-09-13T17:57:16.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:16.181+0000 7f82d2dd4700 1 -- 172.21.15.40:0/3894572657 shutdown_connections 2024-09-13T17:57:16.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:16.181+0000 7f82d2dd4700 1 --2- 172.21.15.40:0/3894572657 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f82b00666e0 0x7f82b0068ba0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:16.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:16.181+0000 7f82d2dd4700 1 --2- 172.21.15.40:0/3894572657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82c40dea90 0x7f82c4049990 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:16.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:16.181+0000 7f82d2dd4700 1 --2- 172.21.15.40:0/3894572657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82c40d4ba0 0x7f82c4179830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:16.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:16.181+0000 7f82d2dd4700 1 --2- 172.21.15.40:0/3894572657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c404abc0 0x7f82c4035000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:16.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:16.181+0000 7f82d2dd4700 1 -- 172.21.15.40:0/3894572657 >> 172.21.15.40:0/3894572657 conn(0x7f82c404fd00 msgr2=0x7f82c40d7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:16.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:16.181+0000 7f82d2dd4700 1 -- 172.21.15.40:0/3894572657 shutdown_connections 2024-09-13T17:57:16.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:16.181+0000 7f82d2dd4700 1 -- 172.21.15.40:0/3894572657 wait complete. 2024-09-13T17:57:19.199 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:57:19.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.321+0000 7fbe248de700 1 Processor -- start 2024-09-13T17:57:19.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.321+0000 7fbe248de700 1 -- start start 2024-09-13T17:57:19.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.321+0000 7fbe248de700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbe20153840 0x7fbe200c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:19.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.321+0000 7fbe248de700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe200c9e20 0x7fbe200cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:19.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.321+0000 7fbe248de700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe200cc060 con 0x7fbe200cc4f0 2024-09-13T17:57:19.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.321+0000 7fbe248de700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe200b2840 con 0x7fbe20153840 2024-09-13T17:57:19.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.321+0000 7fbe248de700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe200ca870 con 0x7fbe200c9e20 2024-09-13T17:57:19.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.321+0000 7fbe1ffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe200c9e20 0x7fbe200cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:19.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.321+0000 7fbe1f7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbe20153840 0x7fbe200c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:19.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.321+0000 7fbe1ffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe200c9e20 0x7fbe200cf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:47910/0 (socket says 172.21.15.40:47910) 2024-09-13T17:57:19.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.321+0000 7fbe1ffff700 1 -- 172.21.15.40:0/3346794495 learned_addr learned my addr 172.21.15.40:0/3346794495 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:19.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.321+0000 7fbe1ffff700 1 -- 172.21.15.40:0/3346794495 >> v2:172.21.15.43:3300/0 conn(0x7fbe20153840 msgr2=0x7fbe200c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:19.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.321+0000 7fbe1ffff700 1 --2- 172.21.15.40:0/3346794495 >> v2:172.21.15.43:3300/0 conn(0x7fbe20153840 0x7fbe200c3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.321+0000 7fbe1ffff700 1 -- 172.21.15.40:0/3346794495 >> v1:172.21.15.43:6789/0 conn(0x7fbe200cc4f0 legacy=0x7fbe2015d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:19.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.321+0000 7fbe1ffff700 1 -- 172.21.15.40:0/3346794495 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe20153c20 con 0x7fbe200c9e20 2024-09-13T17:57:19.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.321+0000 7fbe1ffff700 1 --2- 172.21.15.40:0/3346794495 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe200c9e20 0x7fbe200cf260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009e50 tx=0x7fbe1800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=971bab9b56d293fe server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:19.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe1e7fc700 1 -- 172.21.15.40:0/3346794495 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1800e070 con 0x7fbe200c9e20 2024-09-13T17:57:19.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe1e7fc700 1 -- 172.21.15.40:0/3346794495 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1800b910 con 0x7fbe200c9e20 2024-09-13T17:57:19.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe1e7fc700 1 -- 172.21.15.40:0/3346794495 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1800bcc0 con 0x7fbe200c9e20 2024-09-13T17:57:19.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe248de700 1 -- 172.21.15.40:0/3346794495 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe200c9e20 msgr2=0x7fbe200cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:19.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe248de700 1 --2- 172.21.15.40:0/3346794495 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe200c9e20 0x7fbe200cf260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009e50 tx=0x7fbe1800b330 comp rx=0 tx=0).stop 2024-09-13T17:57:19.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe248de700 1 -- 172.21.15.40:0/3346794495 shutdown_connections 2024-09-13T17:57:19.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe248de700 1 --2- 172.21.15.40:0/3346794495 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe200c9e20 0x7fbe200cf260 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe248de700 1 --2- 172.21.15.40:0/3346794495 >> v2:172.21.15.43:3300/0 conn(0x7fbe20153840 0x7fbe200c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe248de700 1 -- 172.21.15.40:0/3346794495 >> 172.21.15.40:0/3346794495 conn(0x7fbe2006c800 msgr2=0x7fbe2006cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:19.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe248de700 1 -- 172.21.15.40:0/3346794495 shutdown_connections 2024-09-13T17:57:19.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe248de700 1 -- 172.21.15.40:0/3346794495 wait complete. 2024-09-13T17:57:19.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe248de700 1 Processor -- start 2024-09-13T17:57:19.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe248de700 1 -- start start 2024-09-13T17:57:19.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe248de700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe200c9e20 0x7fbe20143d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:19.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe248de700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe200cc4f0 0x7fbe200c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:19.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe248de700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe20153840 0x7fbe20148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:19.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe248de700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe200c7340 con 0x7fbe200c9e20 2024-09-13T17:57:19.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe248de700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe201442d0 con 0x7fbe200cc4f0 2024-09-13T17:57:19.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe248de700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe200ce3e0 con 0x7fbe20153840 2024-09-13T17:57:19.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe1f7fe700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe200c9e20 msgr2=0x7fbe20143d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:57:19.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe1f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe200c9e20 0x7fbe20143d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:57:19.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe1effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe200cc4f0 0x7fbe200c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:19.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe1effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe200cc4f0 0x7fbe200c6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:55176/0 (socket says 172.21.15.40:55176) 2024-09-13T17:57:19.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe1effd700 1 -- 172.21.15.40:0/2483214903 learned_addr learned my addr 172.21.15.40:0/2483214903 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:19.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.325+0000 7fbe1ffff700 1 --2- 172.21.15.40:0/2483214903 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe20153840 0x7fbe20148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.329+0000 7fbe1effd700 1 -- 172.21.15.40:0/2483214903 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe20153840 msgr2=0x7fbe20148dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.329+0000 7fbe1effd700 1 --2- 172.21.15.40:0/2483214903 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe20153840 0x7fbe20148dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.329+0000 7fbe1effd700 1 -- 172.21.15.40:0/2483214903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe200c9e20 msgr2=0x7fbe20143d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:57:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.329+0000 7fbe1effd700 1 --2- 172.21.15.40:0/2483214903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe200c9e20 0x7fbe20143d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.329+0000 7fbe1effd700 1 -- 172.21.15.40:0/2483214903 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe18009b30 con 0x7fbe200cc4f0 2024-09-13T17:57:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.329+0000 7fbe1effd700 1 --2- 172.21.15.40:0/2483214903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe200cc4f0 0x7fbe200c6260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400c530 tx=0x7fbe1400c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.329+0000 7fbe07fff700 1 -- 172.21.15.40:0/2483214903 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe140052d0 con 0x7fbe200cc4f0 2024-09-13T17:57:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.329+0000 7fbe07fff700 1 -- 172.21.15.40:0/2483214903 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe14005430 con 0x7fbe200cc4f0 2024-09-13T17:57:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.329+0000 7fbe248de700 1 -- 172.21.15.40:0/2483214903 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe201449c0 con 0x7fbe200cc4f0 2024-09-13T17:57:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.329+0000 7fbe07fff700 1 -- 172.21.15.40:0/2483214903 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe140055a0 con 0x7fbe200cc4f0 2024-09-13T17:57:19.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.329+0000 7fbe248de700 1 -- 172.21.15.40:0/2483214903 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe200c7dd0 con 0x7fbe200cc4f0 2024-09-13T17:57:19.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.333+0000 7fbe07fff700 1 -- 172.21.15.40:0/2483214903 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbe140038e0 con 0x7fbe200cc4f0 2024-09-13T17:57:19.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.333+0000 7fbe248de700 1 -- 172.21.15.40:0/2483214903 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe200cbcf0 con 0x7fbe200cc4f0 2024-09-13T17:57:19.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.333+0000 7fbe07fff700 1 --2- 172.21.15.40:0/2483214903 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbe0c0666e0 0x7fbe0c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:19.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.333+0000 7fbe07fff700 1 -- 172.21.15.40:0/2483214903 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbe14083890 con 0x7fbe200cc4f0 2024-09-13T17:57:19.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.333+0000 7fbe1f7fe700 1 --2- 172.21.15.40:0/2483214903 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbe0c0666e0 0x7fbe0c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:19.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.333+0000 7fbe1f7fe700 1 --2- 172.21.15.40:0/2483214903 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbe0c0666e0 0x7fbe0c068ba0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009500 tx=0x7fbe10009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:19.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.345+0000 7fbe07fff700 1 -- 172.21.15.40:0/2483214903 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbe140508d0 con 0x7fbe200cc4f0 2024-09-13T17:57:19.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.633+0000 7fbe248de700 1 -- 172.21.15.40:0/2483214903 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbe20151060 con 0x7fbe200cc4f0 2024-09-13T17:57:19.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.633+0000 7fbe07fff700 1 -- 172.21.15.40:0/2483214903 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fbe14017620 con 0x7fbe200cc4f0 2024-09-13T17:57:19.640 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":52,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:57:19.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.637+0000 7fbe248de700 1 -- 172.21.15.40:0/2483214903 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbe0c0666e0 msgr2=0x7fbe0c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:19.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.637+0000 7fbe248de700 1 --2- 172.21.15.40:0/2483214903 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbe0c0666e0 0x7fbe0c068ba0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009500 tx=0x7fbe10009240 comp rx=0 tx=0).stop 2024-09-13T17:57:19.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.637+0000 7fbe248de700 1 -- 172.21.15.40:0/2483214903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe200cc4f0 msgr2=0x7fbe200c6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:19.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.637+0000 7fbe248de700 1 --2- 172.21.15.40:0/2483214903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe200cc4f0 0x7fbe200c6260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400c530 tx=0x7fbe1400c840 comp rx=0 tx=0).stop 2024-09-13T17:57:19.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.637+0000 7fbe248de700 1 -- 172.21.15.40:0/2483214903 shutdown_connections 2024-09-13T17:57:19.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.637+0000 7fbe248de700 1 --2- 172.21.15.40:0/2483214903 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbe0c0666e0 0x7fbe0c068ba0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.637+0000 7fbe248de700 1 --2- 172.21.15.40:0/2483214903 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbe20153840 0x7fbe20148dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.637+0000 7fbe248de700 1 --2- 172.21.15.40:0/2483214903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe200cc4f0 0x7fbe200c6260 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.637+0000 7fbe248de700 1 --2- 172.21.15.40:0/2483214903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe200c9e20 0x7fbe20143d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:19.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.637+0000 7fbe248de700 1 -- 172.21.15.40:0/2483214903 >> 172.21.15.40:0/2483214903 conn(0x7fbe2006c800 msgr2=0x7fbe20146920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:19.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.637+0000 7fbe248de700 1 -- 172.21.15.40:0/2483214903 shutdown_connections 2024-09-13T17:57:19.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:19.637+0000 7fbe248de700 1 -- 172.21.15.40:0/2483214903 wait complete. 2024-09-13T17:57:22.657 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:57:22.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.781+0000 7fcc989d0700 1 Processor -- start 2024-09-13T17:57:22.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.781+0000 7fcc989d0700 1 -- start start 2024-09-13T17:57:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc989d0700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcc941534e0 0x7fcc94159d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc989d0700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcc94070290 0x7fcc9406d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc989d0700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc9406e960 0x7fcc9406d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc989d0700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc940c6db0 con 0x7fcc94070290 2024-09-13T17:57:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc989d0700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc940c6920 con 0x7fcc941534e0 2024-09-13T17:57:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc989d0700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9415a260 con 0x7fcc9406e960 2024-09-13T17:57:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc9259c700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7fcc94070290 msgr2=0x7fcc9406d490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:57:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc9259c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcc94070290 0x7fcc9406d490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:57:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc92d9d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcc941534e0 0x7fcc94159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc9359e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc9406e960 0x7fcc9406d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc92d9d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcc941534e0 0x7fcc94159d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:53864/0 (socket says 172.21.15.40:53864) 2024-09-13T17:57:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc92d9d700 1 -- 172.21.15.40:0/2794214678 learned_addr learned my addr 172.21.15.40:0/2794214678 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc92d9d700 1 -- 172.21.15.40:0/2794214678 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc9406e960 msgr2=0x7fcc9406d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc92d9d700 1 --2- 172.21.15.40:0/2794214678 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc9406e960 0x7fcc9406d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc92d9d700 1 -- 172.21.15.40:0/2794214678 >> v2:172.21.15.40:3300/0 conn(0x7fcc94070290 msgr2=0x7fcc9406d490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:57:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc92d9d700 1 --2- 172.21.15.40:0/2794214678 >> v2:172.21.15.40:3300/0 conn(0x7fcc94070290 0x7fcc9406d490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc92d9d700 1 -- 172.21.15.40:0/2794214678 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc940c5d50 con 0x7fcc941534e0 2024-09-13T17:57:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc9359e700 1 --2- 172.21.15.40:0/2794214678 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc9406e960 0x7fcc9406d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc92d9d700 1 --2- 172.21.15.40:0/2794214678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc941534e0 0x7fcc94159d20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcc94074470 tx=0x7fcc84009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c77d96776e500229 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc91d9b700 1 -- 172.21.15.40:0/2794214678 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84011830 con 0x7fcc941534e0 2024-09-13T17:57:22.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc91d9b700 1 -- 172.21.15.40:0/2794214678 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc84011990 con 0x7fcc941534e0 2024-09-13T17:57:22.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc91d9b700 1 -- 172.21.15.40:0/2794214678 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84011d20 con 0x7fcc941534e0 2024-09-13T17:57:22.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc989d0700 1 -- 172.21.15.40:0/2794214678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc941534e0 msgr2=0x7fcc94159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:22.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc989d0700 1 --2- 172.21.15.40:0/2794214678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc941534e0 0x7fcc94159d20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcc94074470 tx=0x7fcc84009bf0 comp rx=0 tx=0).stop 2024-09-13T17:57:22.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc989d0700 1 -- 172.21.15.40:0/2794214678 shutdown_connections 2024-09-13T17:57:22.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc989d0700 1 --2- 172.21.15.40:0/2794214678 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc9406e960 0x7fcc9406d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:22.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc989d0700 1 --2- 172.21.15.40:0/2794214678 >> v2:172.21.15.40:3300/0 conn(0x7fcc94070290 0x7fcc9406d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:22.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc989d0700 1 --2- 172.21.15.40:0/2794214678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc941534e0 0x7fcc94159d20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:22.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc989d0700 1 -- 172.21.15.40:0/2794214678 >> 172.21.15.40:0/2794214678 conn(0x7fcc9406a390 msgr2=0x7fcc9406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:22.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc989d0700 1 -- 172.21.15.40:0/2794214678 shutdown_connections 2024-09-13T17:57:22.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.785+0000 7fcc989d0700 1 -- 172.21.15.40:0/2794214678 wait complete. 2024-09-13T17:57:22.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.789+0000 7fcc989d0700 1 Processor -- start 2024-09-13T17:57:22.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.789+0000 7fcc989d0700 1 -- start start 2024-09-13T17:57:22.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.789+0000 7fcc989d0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc9406e960 0x7fcc9414e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:22.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.789+0000 7fcc989d0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc94070290 0x7fcc940cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:22.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.789+0000 7fcc989d0700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc941534e0 0x7fcc940c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:22.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.789+0000 7fcc92d9d700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc9406e960 msgr2=0x7fcc9414e670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:57:22.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.789+0000 7fcc989d0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc941e6170 con 0x7fcc9406e960 2024-09-13T17:57:22.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.789+0000 7fcc92d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc9406e960 0x7fcc9414e670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:57:22.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.789+0000 7fcc989d0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc940cbf70 con 0x7fcc94070290 2024-09-13T17:57:22.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.789+0000 7fcc989d0700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9414af30 con 0x7fcc941534e0 2024-09-13T17:57:22.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.789+0000 7fcc9259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc94070290 0x7fcc940cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:22.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.789+0000 7fcc9259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc94070290 0x7fcc940cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:53870/0 (socket says 172.21.15.40:53870) 2024-09-13T17:57:22.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.789+0000 7fcc9259c700 1 -- 172.21.15.40:0/1638624026 learned_addr learned my addr 172.21.15.40:0/1638624026 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:22.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.789+0000 7fcc9359e700 1 --2- 172.21.15.40:0/1638624026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc941534e0 0x7fcc940c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:22.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.793+0000 7fcc9259c700 1 -- 172.21.15.40:0/1638624026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc941534e0 msgr2=0x7fcc940c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:22.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.793+0000 7fcc9259c700 1 --2- 172.21.15.40:0/1638624026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc941534e0 0x7fcc940c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:22.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.793+0000 7fcc9259c700 1 -- 172.21.15.40:0/1638624026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc9406e960 msgr2=0x7fcc9414e670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:57:22.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.793+0000 7fcc9259c700 1 --2- 172.21.15.40:0/1638624026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc9406e960 0x7fcc9414e670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:22.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.793+0000 7fcc9259c700 1 -- 172.21.15.40:0/1638624026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8c009a70 con 0x7fcc94070290 2024-09-13T17:57:22.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.793+0000 7fcc9259c700 1 --2- 172.21.15.40:0/1638624026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc94070290 0x7fcc940cb500 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800b780 tx=0x7fcc8800bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:22.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.793+0000 7fcc7b7fe700 1 -- 172.21.15.40:0/1638624026 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88003e70 con 0x7fcc94070290 2024-09-13T17:57:22.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.793+0000 7fcc7b7fe700 1 -- 172.21.15.40:0/1638624026 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc88005070 con 0x7fcc94070290 2024-09-13T17:57:22.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.793+0000 7fcc989d0700 1 -- 172.21.15.40:0/1638624026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc84009570 con 0x7fcc94070290 2024-09-13T17:57:22.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.793+0000 7fcc7b7fe700 1 -- 172.21.15.40:0/1638624026 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88005420 con 0x7fcc94070290 2024-09-13T17:57:22.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.793+0000 7fcc989d0700 1 -- 172.21.15.40:0/1638624026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc941e6680 con 0x7fcc94070290 2024-09-13T17:57:22.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.793+0000 7fcc7b7fe700 1 -- 172.21.15.40:0/1638624026 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcc88005580 con 0x7fcc94070290 2024-09-13T17:57:22.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.793+0000 7fcc989d0700 1 -- 172.21.15.40:0/1638624026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc940c8c20 con 0x7fcc94070290 2024-09-13T17:57:22.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.793+0000 7fcc7b7fe700 1 --2- 172.21.15.40:0/1638624026 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcc800666e0 0x7fcc80068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:22.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.797+0000 7fcc7b7fe700 1 -- 172.21.15.40:0/1638624026 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcc880840f0 con 0x7fcc94070290 2024-09-13T17:57:22.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.797+0000 7fcc92d9d700 1 --2- 172.21.15.40:0/1638624026 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcc800666e0 0x7fcc80068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:22.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.797+0000 7fcc92d9d700 1 --2- 172.21.15.40:0/1638624026 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcc800666e0 0x7fcc80068ba0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009fd0 tx=0x7fcc84008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:22.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:22.805+0000 7fcc7b7fe700 1 -- 172.21.15.40:0/1638624026 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcc88051010 con 0x7fcc94070290 2024-09-13T17:57:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:23.069+0000 7fcc989d0700 1 -- 172.21.15.40:0/1638624026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fcc94153950 con 0x7fcc94070290 2024-09-13T17:57:23.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:23.069+0000 7fcc7b7fe700 1 -- 172.21.15.40:0/1638624026 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fcc88005830 con 0x7fcc94070290 2024-09-13T17:57:23.077 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":52,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:57:23.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:23.073+0000 7fcc989d0700 1 -- 172.21.15.40:0/1638624026 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcc800666e0 msgr2=0x7fcc80068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:23.073+0000 7fcc989d0700 1 --2- 172.21.15.40:0/1638624026 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcc800666e0 0x7fcc80068ba0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009fd0 tx=0x7fcc84008030 comp rx=0 tx=0).stop 2024-09-13T17:57:23.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:23.073+0000 7fcc989d0700 1 -- 172.21.15.40:0/1638624026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc94070290 msgr2=0x7fcc940cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:23.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:23.073+0000 7fcc989d0700 1 --2- 172.21.15.40:0/1638624026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc94070290 0x7fcc940cb500 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800b780 tx=0x7fcc8800bb40 comp rx=0 tx=0).stop 2024-09-13T17:57:23.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:23.073+0000 7fcc989d0700 1 -- 172.21.15.40:0/1638624026 shutdown_connections 2024-09-13T17:57:23.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:23.073+0000 7fcc989d0700 1 --2- 172.21.15.40:0/1638624026 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcc800666e0 0x7fcc80068ba0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:23.073+0000 7fcc989d0700 1 --2- 172.21.15.40:0/1638624026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcc941534e0 0x7fcc940c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:23.073+0000 7fcc989d0700 1 --2- 172.21.15.40:0/1638624026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc94070290 0x7fcc940cb500 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:23.073+0000 7fcc989d0700 1 --2- 172.21.15.40:0/1638624026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc9406e960 0x7fcc9414e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:23.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:23.073+0000 7fcc989d0700 1 -- 172.21.15.40:0/1638624026 >> 172.21.15.40:0/1638624026 conn(0x7fcc9406a390 msgr2=0x7fcc940c2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:23.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:23.073+0000 7fcc989d0700 1 -- 172.21.15.40:0/1638624026 shutdown_connections 2024-09-13T17:57:23.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:23.073+0000 7fcc989d0700 1 -- 172.21.15.40:0/1638624026 wait complete. 2024-09-13T17:57:26.093 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:57:26.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:26.213+0000 7fdaee8b2700 1 Processor -- start 2024-09-13T17:57:26.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:26.213+0000 7fdaee8b2700 1 -- start start 2024-09-13T17:57:26.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:26.213+0000 7fdaee8b2700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdae815cf60 0x7fdae815d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:26.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:26.213+0000 7fdaee8b2700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae815d910 con 0x7fdae80cb2f0 2024-09-13T17:57:26.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:26.213+0000 7fdaee8b2700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae8159770 con 0x7fdae8153880 2024-09-13T17:57:26.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:26.213+0000 7fdaee8b2700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdae8159a50 con 0x7fdae815cf60 2024-09-13T17:57:26.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:26.213+0000 7fdaed0af700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7fdae80cb2f0 legacy=0x7fdae80cee80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:57:26.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:26.213+0000 7fdaee0b1700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7fdae815cf60 msgr2=0x7fdae815d3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:57:26.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:26.213+0000 7fdaee0b1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdae815cf60 0x7fdae815d3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:57:26.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:26.213+0000 7fdaed8b0700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdae8153880 0x7fdae80c9ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:49994/0 (socket says 172.21.15.40:49994) 2024-09-13T17:57:26.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:26.213+0000 7fdaed8b0700 1 -- 172.21.15.40:0/375946290 learned_addr learned my addr 172.21.15.40:0/375946290 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:57:26.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:26.417+0000 7fdaee0b1700 1 -- 172.21.15.40:0/375946290 >> v2:172.21.15.40:3300/0 conn(0x7fdae815cf60 msgr2=0x7fdae815d3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:57:26.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:26.417+0000 7fdaed0af700 1 -- 172.21.15.40:0/375946290 >> v1:172.21.15.40:6789/0 conn(0x7fdae80cb2f0 legacy=0x7fdae80cee80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:57:26.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:26.417+0000 7fdaee0b1700 1 --2- 172.21.15.40:0/375946290 >> v2:172.21.15.40:3300/0 conn(0x7fdae815cf60 0x7fdae815d3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:57:26.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:26.821+0000 7fdaed0af700 1 -- 172.21.15.40:0/375946290 >> v1:172.21.15.40:6789/0 conn(0x7fdae80cb2f0 legacy=0x7fdae80cee80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:57:26.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:26.821+0000 7fdaee0b1700 1 -- 172.21.15.40:0/375946290 >> v2:172.21.15.40:3300/0 conn(0x7fdae815cf60 msgr2=0x7fdae815d3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:57:26.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:26.821+0000 7fdaee0b1700 1 --2- 172.21.15.40:0/375946290 >> v2:172.21.15.40:3300/0 conn(0x7fdae815cf60 0x7fdae815d3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:57:27.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:27.625+0000 7fdaed0af700 1 -- 172.21.15.40:0/375946290 >> v1:172.21.15.40:6789/0 conn(0x7fdae80cb2f0 legacy=0x7fdae80cee80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:57:27.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:27.625+0000 7fdaee0b1700 1 -- 172.21.15.40:0/375946290 >> v2:172.21.15.40:3300/0 conn(0x7fdae815cf60 msgr2=0x7fdae815d3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:57:27.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:27.625+0000 7fdaee0b1700 1 --2- 172.21.15.40:0/375946290 >> v2:172.21.15.40:3300/0 conn(0x7fdae815cf60 0x7fdae815d3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:57:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.213+0000 7fdae77fe700 1 -- 172.21.15.40:0/375946290 >> v2:172.21.15.40:3300/0 conn(0x7fdae815cf60 msgr2=0x7fdae815d3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:57:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.213+0000 7fdae77fe700 1 --2- 172.21.15.40:0/375946290 >> v2:172.21.15.40:3300/0 conn(0x7fdae815cf60 0x7fdae815d3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:29.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.213+0000 7fdae77fe700 1 -- 172.21.15.40:0/375946290 >> v1:172.21.15.43:6789/0 conn(0x7fdae8153880 legacy=0x7fdae80c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.213+0000 7fdae77fe700 1 -- 172.21.15.40:0/375946290 >> v1:172.21.15.40:6789/0 conn(0x7fdae80cb2f0 legacy=0x7fdae80cee80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:57:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.213+0000 7fdae77fe700 1 --2- 172.21.15.40:0/375946290 >> v2:172.21.15.43:3300/0 conn(0x7fdad4003740 0x7fdad4005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.213+0000 7fdae77fe700 1 --2- 172.21.15.40:0/375946290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdad4006190 0x7fdad4008650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.213+0000 7fdae77fe700 1 -- 172.21.15.40:0/375946290 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdae8159a50 con 0x7fdad4003740 2024-09-13T17:57:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.213+0000 7fdae77fe700 1 -- 172.21.15.40:0/375946290 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad4008d70 con 0x7fdad4006190 2024-09-13T17:57:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.213+0000 7fdaed8b0700 1 --2- 172.21.15.40:0/375946290 >> v2:172.21.15.43:3300/0 conn(0x7fdad4003740 0x7fdad4005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.213+0000 7fdaed0af700 1 --2- 172.21.15.40:0/375946290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdad4006190 0x7fdad4008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.213+0000 7fdaed0af700 1 -- 172.21.15.40:0/375946290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdad4006190 msgr2=0x7fdad4008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:57:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.213+0000 7fdaed8b0700 1 -- 172.21.15.40:0/375946290 >> v2:172.21.15.43:3300/0 conn(0x7fdad4003740 msgr2=0x7fdad4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:57:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.213+0000 7fdaed0af700 1 -- 172.21.15.40:0/375946290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdad4006190 msgr2=0x7fdad4008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:57:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.213+0000 7fdaed0af700 1 --2- 172.21.15.40:0/375946290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdad4006190 0x7fdad4008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:57:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.213+0000 7fdaed8b0700 1 -- 172.21.15.40:0/375946290 >> v2:172.21.15.43:3300/0 conn(0x7fdad4003740 msgr2=0x7fdad4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:57:29.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.213+0000 7fdaed8b0700 1 --2- 172.21.15.40:0/375946290 >> v2:172.21.15.43:3300/0 conn(0x7fdad4003740 0x7fdad4005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:57:29.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.213+0000 7fdaed0af700 1 --2- 172.21.15.40:0/375946290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdad4006190 0x7fdad4008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:57:29.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.213+0000 7fdaed8b0700 1 --2- 172.21.15.40:0/375946290 >> v2:172.21.15.43:3300/0 conn(0x7fdad4003740 0x7fdad4005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:57:29.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.417+0000 7fdaed8b0700 1 --2- 172.21.15.40:0/375946290 >> v2:172.21.15.43:3300/0 conn(0x7fdad4003740 0x7fdad4005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:29.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.417+0000 7fdaed0af700 1 --2- 172.21.15.40:0/375946290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdad4006190 0x7fdad4008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:29.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.417+0000 7fdaed8b0700 1 -- 172.21.15.40:0/375946290 >> v2:172.21.15.43:3300/0 conn(0x7fdad4003740 msgr2=0x7fdad4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:57:29.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.417+0000 7fdaed8b0700 1 -- 172.21.15.40:0/375946290 >> v2:172.21.15.43:3300/0 conn(0x7fdad4003740 msgr2=0x7fdad4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:57:29.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.417+0000 7fdaed8b0700 1 --2- 172.21.15.40:0/375946290 >> v2:172.21.15.43:3300/0 conn(0x7fdad4003740 0x7fdad4005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:57:29.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.417+0000 7fdaed8b0700 1 --2- 172.21.15.40:0/375946290 >> v2:172.21.15.43:3300/0 conn(0x7fdad4003740 0x7fdad4005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:57:29.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.417+0000 7fdaed0af700 1 -- 172.21.15.40:0/375946290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdad4006190 msgr2=0x7fdad4008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:57:29.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.417+0000 7fdaed0af700 1 -- 172.21.15.40:0/375946290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdad4006190 msgr2=0x7fdad4008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:57:29.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.417+0000 7fdaed0af700 1 --2- 172.21.15.40:0/375946290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdad4006190 0x7fdad4008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:57:29.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.417+0000 7fdaed0af700 1 --2- 172.21.15.40:0/375946290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdad4006190 0x7fdad4008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:57:29.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.821+0000 7fdaed8b0700 1 --2- 172.21.15.40:0/375946290 >> v2:172.21.15.43:3300/0 conn(0x7fdad4003740 0x7fdad4005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:29.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.821+0000 7fdaed0af700 1 --2- 172.21.15.40:0/375946290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdad4006190 0x7fdad4008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:29.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.821+0000 7fdaed0af700 1 -- 172.21.15.40:0/375946290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdad4006190 msgr2=0x7fdad4008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:57:29.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.821+0000 7fdaed0af700 1 -- 172.21.15.40:0/375946290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdad4006190 msgr2=0x7fdad4008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:57:29.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.821+0000 7fdaed0af700 1 --2- 172.21.15.40:0/375946290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdad4006190 0x7fdad4008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:57:29.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.821+0000 7fdaed8b0700 1 -- 172.21.15.40:0/375946290 >> v2:172.21.15.43:3300/0 conn(0x7fdad4003740 msgr2=0x7fdad4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:57:29.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.821+0000 7fdaed8b0700 1 -- 172.21.15.40:0/375946290 >> v2:172.21.15.43:3300/0 conn(0x7fdad4003740 msgr2=0x7fdad4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:57:29.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.821+0000 7fdaed8b0700 1 --2- 172.21.15.40:0/375946290 >> v2:172.21.15.43:3300/0 conn(0x7fdad4003740 0x7fdad4005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:57:29.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.821+0000 7fdaed0af700 1 --2- 172.21.15.40:0/375946290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdad4006190 0x7fdad4008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:57:29.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:29.821+0000 7fdaed8b0700 1 --2- 172.21.15.40:0/375946290 >> v2:172.21.15.43:3300/0 conn(0x7fdad4003740 0x7fdad4005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:57:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.625+0000 7fdaed0af700 1 --2- 172.21.15.40:0/375946290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdad4006190 0x7fdad4008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.625+0000 7fdaed8b0700 1 --2- 172.21.15.40:0/375946290 >> v2:172.21.15.43:3300/0 conn(0x7fdad4003740 0x7fdad4005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.625+0000 7fdaed0af700 1 -- 172.21.15.40:0/375946290 >> v2:172.21.15.43:3300/0 conn(0x7fdad4003740 msgr2=0x7fdad4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:30.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.625+0000 7fdaed0af700 1 --2- 172.21.15.40:0/375946290 >> v2:172.21.15.43:3300/0 conn(0x7fdad4003740 0x7fdad4005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.625+0000 7fdaed0af700 1 -- 172.21.15.40:0/375946290 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae8159d30 con 0x7fdad4006190 2024-09-13T17:57:30.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.625+0000 7fdaed8b0700 1 --2- 172.21.15.40:0/375946290 >> v2:172.21.15.43:3300/0 conn(0x7fdad4003740 0x7fdad4005bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:30.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.625+0000 7fdaed0af700 1 --2- 172.21.15.40:0/375946290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdad4006190 0x7fdad4008650 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdadc003a30 tx=0x7fdadc009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e323f5240eda0388 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:30.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.625+0000 7fdaec8ae700 1 -- 172.21.15.40:0/375946290 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc013070 con 0x7fdad4006190 2024-09-13T17:57:30.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.625+0000 7fdaec8ae700 1 -- 172.21.15.40:0/375946290 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdadc00e970 con 0x7fdad4006190 2024-09-13T17:57:30.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.625+0000 7fdaec8ae700 1 -- 172.21.15.40:0/375946290 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc00ed00 con 0x7fdad4006190 2024-09-13T17:57:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.625+0000 7fdaee8b2700 1 -- 172.21.15.40:0/375946290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdad4006190 msgr2=0x7fdad4008650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.625+0000 7fdaee8b2700 1 --2- 172.21.15.40:0/375946290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdad4006190 0x7fdad4008650 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdadc003a30 tx=0x7fdadc009ed0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.625+0000 7fdaee0b1700 1 -- 172.21.15.40:0/375946290 reap_dead start 2024-09-13T17:57:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.625+0000 7fdaee8b2700 1 -- 172.21.15.40:0/375946290 shutdown_connections 2024-09-13T17:57:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.625+0000 7fdaee8b2700 1 -- 172.21.15.40:0/375946290 >> 172.21.15.40:0/375946290 conn(0x7fdae806c5b0 msgr2=0x7fdae806c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.625+0000 7fdaee8b2700 1 -- 172.21.15.40:0/375946290 shutdown_connections 2024-09-13T17:57:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.625+0000 7fdaee8b2700 1 -- 172.21.15.40:0/375946290 wait complete. 2024-09-13T17:57:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.629+0000 7fdaee8b2700 1 Processor -- start 2024-09-13T17:57:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.629+0000 7fdaee8b2700 1 -- start start 2024-09-13T17:57:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdaee8b2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdae815b9f0 0x7fdae8156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdaee8b2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdae815bdd0 0x7fdae80c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdaee8b2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdae80c8580 0x7fdae80c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdaee8b2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae81f0520 con 0x7fdae80c8580 2024-09-13T17:57:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdaee8b2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae81ef700 con 0x7fdae815bdd0 2024-09-13T17:57:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdaee8b2700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae81ef9a0 con 0x7fdae815b9f0 2024-09-13T17:57:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdaee0b1700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdae80c8580 msgr2=0x7fdae80c6630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:57:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdaee0b1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdae80c8580 0x7fdae80c6630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:57:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdaed0af700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdae815bdd0 0x7fdae80c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdaed8b0700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdae815b9f0 0x7fdae8156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdaed0af700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdae815bdd0 0x7fdae80c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:53910/0 (socket says 172.21.15.40:53910) 2024-09-13T17:57:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdaed0af700 1 -- 172.21.15.40:0/1391660045 learned_addr learned my addr 172.21.15.40:0/1391660045 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdaed0af700 1 -- 172.21.15.40:0/1391660045 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdae815b9f0 msgr2=0x7fdae8156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdaed0af700 1 --2- 172.21.15.40:0/1391660045 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdae815b9f0 0x7fdae8156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdaed0af700 1 -- 172.21.15.40:0/1391660045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdae80c8580 msgr2=0x7fdae80c6630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:57:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdaed0af700 1 --2- 172.21.15.40:0/1391660045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdae80c8580 0x7fdae80c6630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdaed0af700 1 -- 172.21.15.40:0/1391660045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc006e70 con 0x7fdae815bdd0 2024-09-13T17:57:30.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdaed0af700 1 --2- 172.21.15.40:0/1391660045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdae815bdd0 0x7fdae80c8040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdadc015040 tx=0x7fdadc005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:30.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdae67fc700 1 -- 172.21.15.40:0/1391660045 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc01f070 con 0x7fdae815bdd0 2024-09-13T17:57:30.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdae67fc700 1 -- 172.21.15.40:0/1391660045 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdadc018630 con 0x7fdae815bdd0 2024-09-13T17:57:30.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdaee8b2700 1 -- 172.21.15.40:0/1391660045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae81efc80 con 0x7fdae815bdd0 2024-09-13T17:57:30.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdaee8b2700 1 -- 172.21.15.40:0/1391660045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae81f0240 con 0x7fdae815bdd0 2024-09-13T17:57:30.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.633+0000 7fdae67fc700 1 -- 172.21.15.40:0/1391660045 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc0189e0 con 0x7fdae815bdd0 2024-09-13T17:57:30.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.637+0000 7fdaee8b2700 1 -- 172.21.15.40:0/1391660045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac8005310 con 0x7fdae815bdd0 2024-09-13T17:57:30.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.637+0000 7fdae67fc700 1 -- 172.21.15.40:0/1391660045 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdadc018b40 con 0x7fdae815bdd0 2024-09-13T17:57:30.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.637+0000 7fdae67fc700 1 --2- 172.21.15.40:0/1391660045 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdad4067430 0x7fdad40698f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:30.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.637+0000 7fdae67fc700 1 -- 172.21.15.40:0/1391660045 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdadc085530 con 0x7fdae815bdd0 2024-09-13T17:57:30.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.637+0000 7fdaed8b0700 1 --2- 172.21.15.40:0/1391660045 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdad4067430 0x7fdad40698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:30.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.637+0000 7fdaed8b0700 1 --2- 172.21.15.40:0/1391660045 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdad4067430 0x7fdad40698f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdad8004c70 tx=0x7fdad8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:30.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.649+0000 7fdae67fc700 1 -- 172.21.15.40:0/1391660045 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdadc052650 con 0x7fdae815bdd0 2024-09-13T17:57:30.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.921+0000 7fdaee8b2700 1 -- 172.21.15.40:0/1391660045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdac80055c0 con 0x7fdae815bdd0 2024-09-13T17:57:30.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.925+0000 7fdae67fc700 1 -- 172.21.15.40:0/1391660045 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7fdadc055c70 con 0x7fdae815bdd0 2024-09-13T17:57:30.930 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":54,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:57:30.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.929+0000 7fdaee8b2700 1 -- 172.21.15.40:0/1391660045 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdad4067430 msgr2=0x7fdad40698f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.929+0000 7fdaee8b2700 1 --2- 172.21.15.40:0/1391660045 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdad4067430 0x7fdad40698f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdad8004c70 tx=0x7fdad8009040 comp rx=0 tx=0).stop 2024-09-13T17:57:30.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.929+0000 7fdaee8b2700 1 -- 172.21.15.40:0/1391660045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdae815bdd0 msgr2=0x7fdae80c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:30.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.929+0000 7fdaee8b2700 1 --2- 172.21.15.40:0/1391660045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdae815bdd0 0x7fdae80c8040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdadc015040 tx=0x7fdadc005d90 comp rx=0 tx=0).stop 2024-09-13T17:57:30.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.929+0000 7fdaee8b2700 1 -- 172.21.15.40:0/1391660045 shutdown_connections 2024-09-13T17:57:30.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.929+0000 7fdaee8b2700 1 --2- 172.21.15.40:0/1391660045 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdae80c8580 0x7fdae80c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.929+0000 7fdaee8b2700 1 --2- 172.21.15.40:0/1391660045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdae815bdd0 0x7fdae80c8040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.929+0000 7fdaee8b2700 1 --2- 172.21.15.40:0/1391660045 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdad4067430 0x7fdad40698f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.929+0000 7fdaee8b2700 1 --2- 172.21.15.40:0/1391660045 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdae815b9f0 0x7fdae8156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:30.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.929+0000 7fdaee8b2700 1 -- 172.21.15.40:0/1391660045 >> 172.21.15.40:0/1391660045 conn(0x7fdae806c5b0 msgr2=0x7fdae80b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:30.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.929+0000 7fdaee8b2700 1 -- 172.21.15.40:0/1391660045 shutdown_connections 2024-09-13T17:57:30.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:30.929+0000 7fdaee8b2700 1 -- 172.21.15.40:0/1391660045 wait complete. 2024-09-13T17:57:30.951 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T17:57:30.952 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-13T17:57:30.952 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:57:31.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72f4beb700 1 Processor -- start 2024-09-13T17:57:31.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72f4beb700 1 -- start start 2024-09-13T17:57:31.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72f4beb700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f72e804d0f0 0x7f72e80e0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72f4beb700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f72e8046df0 0x7f72e804de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72f4beb700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e80dd3c0 con 0x7f72e80d6670 2024-09-13T17:57:31.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72f4beb700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e80e0180 con 0x7f72e8046df0 2024-09-13T17:57:31.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72f4beb700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e80dcb80 con 0x7f72e804d0f0 2024-09-13T17:57:31.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72ef7fe700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f72e80d6670 legacy=0x7f72e80dc470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:57:31.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72effff700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f72e8046df0 msgr2=0x7f72e804de20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:57:31.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72effff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f72e8046df0 0x7f72e804de20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:57:31.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72eeffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f72e804d0f0 0x7f72e80e0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72eeffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f72e804d0f0 0x7f72e80e0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:53912/0 (socket says 172.21.15.40:53912) 2024-09-13T17:57:31.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72eeffd700 1 -- 172.21.15.40:0/1054787179 learned_addr learned my addr 172.21.15.40:0/1054787179 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:31.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72eeffd700 1 -- 172.21.15.40:0/1054787179 >> v2:172.21.15.40:3300/0 conn(0x7f72e8046df0 msgr2=0x7f72e804de20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:57:31.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72eeffd700 1 --2- 172.21.15.40:0/1054787179 >> v2:172.21.15.40:3300/0 conn(0x7f72e8046df0 0x7f72e804de20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72eeffd700 1 -- 172.21.15.40:0/1054787179 >> v1:172.21.15.40:6789/0 conn(0x7f72e80d6670 legacy=0x7f72e80dc470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:57:31.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72eeffd700 1 -- 172.21.15.40:0/1054787179 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e8047220 con 0x7f72e804d0f0 2024-09-13T17:57:31.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72eeffd700 1 --2- 172.21.15.40:0/1054787179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e804d0f0 0x7f72e80e0610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f72e80da0a0 tx=0x7f72dc009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e026e04e3008789b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:31.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72ee7fc700 1 -- 172.21.15.40:0/1054787179 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc004230 con 0x7f72e804d0f0 2024-09-13T17:57:31.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72ee7fc700 1 -- 172.21.15.40:0/1054787179 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72dc004390 con 0x7f72e804d0f0 2024-09-13T17:57:31.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.065+0000 7f72ee7fc700 1 -- 172.21.15.40:0/1054787179 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc0036c0 con 0x7f72e804d0f0 2024-09-13T17:57:31.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72f4beb700 1 -- 172.21.15.40:0/1054787179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e804d0f0 msgr2=0x7f72e80e0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72f4beb700 1 --2- 172.21.15.40:0/1054787179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e804d0f0 0x7f72e80e0610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f72e80da0a0 tx=0x7f72dc009bf0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72f4beb700 1 -- 172.21.15.40:0/1054787179 shutdown_connections 2024-09-13T17:57:31.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72f4beb700 1 --2- 172.21.15.40:0/1054787179 >> v2:172.21.15.40:3300/0 conn(0x7f72e8046df0 0x7f72e804de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72f4beb700 1 --2- 172.21.15.40:0/1054787179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e804d0f0 0x7f72e80e0610 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72f4beb700 1 -- 172.21.15.40:0/1054787179 >> 172.21.15.40:0/1054787179 conn(0x7f72e800dc40 msgr2=0x7f72e800e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:31.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72f4beb700 1 -- 172.21.15.40:0/1054787179 shutdown_connections 2024-09-13T17:57:31.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72f4beb700 1 -- 172.21.15.40:0/1054787179 wait complete. 2024-09-13T17:57:31.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72f4beb700 1 Processor -- start 2024-09-13T17:57:31.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72f4beb700 1 -- start start 2024-09-13T17:57:31.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72f4beb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e8046df0 0x7f72e8049970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72f4beb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72e804d0f0 0x7f72e8049eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72f4beb700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72e80d6670 0x7f72e80db4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72f4beb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e80d2e90 con 0x7f72e804d0f0 2024-09-13T17:57:31.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72f4beb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e804bc80 con 0x7f72e8046df0 2024-09-13T17:57:31.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72f4beb700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e804b9c0 con 0x7f72e80d6670 2024-09-13T17:57:31.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72eeffd700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72e804d0f0 msgr2=0x7f72e8049eb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:57:31.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72eeffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72e804d0f0 0x7f72e8049eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:57:31.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72ef7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e8046df0 0x7f72e8049970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72ef7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e8046df0 0x7f72e8049970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:53922/0 (socket says 172.21.15.40:53922) 2024-09-13T17:57:31.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72ef7fe700 1 -- 172.21.15.40:0/3129868461 learned_addr learned my addr 172.21.15.40:0/3129868461 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:31.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.069+0000 7f72effff700 1 --2- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72e80d6670 0x7f72e80db4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.073+0000 7f72ef7fe700 1 -- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72e80d6670 msgr2=0x7f72e80db4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.073+0000 7f72ef7fe700 1 --2- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72e80d6670 0x7f72e80db4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.073+0000 7f72ef7fe700 1 -- 172.21.15.40:0/3129868461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72e804d0f0 msgr2=0x7f72e8049eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:57:31.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.073+0000 7f72ef7fe700 1 --2- 172.21.15.40:0/3129868461 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72e804d0f0 0x7f72e8049eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.073+0000 7f72ef7fe700 1 -- 172.21.15.40:0/3129868461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc009570 con 0x7f72e8046df0 2024-09-13T17:57:31.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.073+0000 7f72ef7fe700 1 --2- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e8046df0 0x7f72e8049970 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f72e400b570 tx=0x7f72e400b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:31.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.073+0000 7f72dbfff700 1 -- 172.21.15.40:0/3129868461 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e4003e70 con 0x7f72e8046df0 2024-09-13T17:57:31.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.073+0000 7f72dbfff700 1 -- 172.21.15.40:0/3129868461 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e4005070 con 0x7f72e8046df0 2024-09-13T17:57:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.073+0000 7f72dbfff700 1 -- 172.21.15.40:0/3129868461 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e4005440 con 0x7f72e8046df0 2024-09-13T17:57:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.073+0000 7f72f4beb700 1 -- 172.21.15.40:0/3129868461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e80d15f0 con 0x7f72e8046df0 2024-09-13T17:57:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.073+0000 7f72f4beb700 1 -- 172.21.15.40:0/3129868461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e804beb0 con 0x7f72e8046df0 2024-09-13T17:57:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.073+0000 7f72f4beb700 1 --2- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e80d11e0 0x7f72e80dba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.073+0000 7f72f4beb700 1 -- 172.21.15.40:0/3129868461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e80074a0 con 0x7f72e80d11e0 2024-09-13T17:57:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.073+0000 7f72eeffd700 1 --2- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e80d11e0 0x7f72e80dba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.073+0000 7f72eeffd700 1 -- 172.21.15.40:0/3129868461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f72e80dfb30 con 0x7f72e80d11e0 2024-09-13T17:57:31.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.073+0000 7f72dbfff700 1 -- 172.21.15.40:0/3129868461 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f72e40055a0 con 0x7f72e8046df0 2024-09-13T17:57:31.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.073+0000 7f72eeffd700 1 --2- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e80d11e0 0x7f72e80dba00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f72dc004700 tx=0x7f72dc004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:31.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.073+0000 7f72dbfff700 1 --2- 172.21.15.40:0/3129868461 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f72d40666e0 0x7f72d4068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.077+0000 7f72dbfff700 1 -- 172.21.15.40:0/3129868461 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f72e4083990 con 0x7f72e8046df0 2024-09-13T17:57:31.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.077+0000 7f72effff700 1 --2- 172.21.15.40:0/3129868461 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f72d40666e0 0x7f72d4068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.077+0000 7f72dbfff700 1 -- 172.21.15.40:0/3129868461 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc00b660 con 0x7f72e80d11e0 2024-09-13T17:57:31.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.077+0000 7f72dbfff700 1 -- 172.21.15.40:0/3129868461 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f72dc00b7c0 con 0x7f72e80d11e0 2024-09-13T17:57:31.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.077+0000 7f72effff700 1 --2- 172.21.15.40:0/3129868461 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f72d40666e0 0x7f72d4068ba0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f72f004f480 tx=0x7f72f0065ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:31.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.077+0000 7f72dbfff700 1 -- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e80d11e0 msgr2=0x7f72e80dba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.077+0000 7f72dbfff700 1 --2- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e80d11e0 0x7f72e80dba00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f72dc004700 tx=0x7f72dc004770 comp rx=0 tx=0).stop 2024-09-13T17:57:31.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.077+0000 7f72dbfff700 1 -- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e80d11e0 msgr2=0x7f72e80dba00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:57:31.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.077+0000 7f72dbfff700 1 --2- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e80d11e0 0x7f72e80dba00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.089+0000 7f72f4beb700 1 --2- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e80d0d40 0x7f72e80de620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.089+0000 7f72f4beb700 1 -- 172.21.15.40:0/3129868461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e80dd0e0 con 0x7f72e80d0d40 2024-09-13T17:57:31.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.089+0000 7f72eeffd700 1 --2- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e80d0d40 0x7f72e80de620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.089+0000 7f72eeffd700 1 -- 172.21.15.40:0/3129868461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f72e80dfe10 con 0x7f72e80d0d40 2024-09-13T17:57:31.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.093+0000 7f72eeffd700 1 --2- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e80d0d40 0x7f72e80de620 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f72dc000c00 tx=0x7f72dc0092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:31.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.093+0000 7f72dbfff700 1 -- 172.21.15.40:0/3129868461 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc02a780 con 0x7f72e80d0d40 2024-09-13T17:57:31.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.093+0000 7f72dbfff700 1 -- 172.21.15.40:0/3129868461 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4938 (secure 0 0 0) 0x7f72dc02a8e0 con 0x7f72e80d0d40 2024-09-13T17:57:31.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.093+0000 7f72dbfff700 1 -- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e80d0d40 msgr2=0x7f72e80de620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.093+0000 7f72dbfff700 1 --2- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e80d0d40 0x7f72e80de620 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f72dc000c00 tx=0x7f72dc0092e0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.093+0000 7f72dbfff700 1 -- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e80d0d40 msgr2=0x7f72e80de620 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:57:31.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.093+0000 7f72dbfff700 1 --2- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e80d0d40 0x7f72e80de620 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.099 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:57:31.100 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:57:31.100 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:57:31.100 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:57:31.100 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 54, 2024-09-13T17:57:31.100 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:57:31.100 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:57:31.100 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:57:31.100 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:31.100 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T17:57:31.100 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:57:31.100 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:57:31.100 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:57:31.101 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:31.101 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:31.101 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:31.101 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:31.101 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:31.101 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:31.101 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:31.101 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:31.101 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:31.101 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:31.101 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:31.101 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:57:31.101 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:57:31.102 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:31.102 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:31.102 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:31.102 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:31.102 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:31.102 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:31.102 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:31.102 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:31.102 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:31.102 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:31.102 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:31.102 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.102 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:57:31.102 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:57:31.103 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:57:31.103 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:57:31.103 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:57:31.103 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:57:31.103 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:57:31.104 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:57:31.104 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:57:31.104 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:57:31.104 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:57:31.104 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:57:31.104 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:57:31.104 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:57:31.104 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:57:31.104 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:57:31.104 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:57:31.104 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:31.104 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:31.104 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:57:31.105 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:31.106 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:31.107 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:57:31.108 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:31.109 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.110 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.110 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:31.110 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:57:31.110 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:31.110 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:31.110 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:31.110 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.110 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:57:31.110 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:57:31.110 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:31.110 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:31.110 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:31.110 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:31.110 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:31.111 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.111 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:57:31.111 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:57:31.111 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.111 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:31.111 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:31.111 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:57:31.111 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:31.111 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:31.111 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:57:31.111 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.111 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:31.111 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:31.112 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:57:31.112 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:31.112 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:31.112 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:57:31.112 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.112 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:31.112 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:31.112 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:57:31.112 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:31.112 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:31.112 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:57:31.112 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.112 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:31.113 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:31.113 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:57:31.113 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:31.113 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:31.113 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:57:31.113 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.113 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:31.113 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:31.113 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:57:31.113 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:31.113 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:31.113 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.113 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:57:31.113 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:57:31.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.093+0000 7f72f4beb700 1 -- 172.21.15.40:0/3129868461 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f72d40666e0 msgr2=0x7f72d4068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.093+0000 7f72f4beb700 1 --2- 172.21.15.40:0/3129868461 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f72d40666e0 0x7f72d4068ba0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f72f004f480 tx=0x7f72f0065ea0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.093+0000 7f72f4beb700 1 -- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e8046df0 msgr2=0x7f72e8049970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.093+0000 7f72effff700 1 -- 172.21.15.40:0/3129868461 reap_dead start 2024-09-13T17:57:31.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.093+0000 7f72f4beb700 1 --2- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e8046df0 0x7f72e8049970 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f72e400b570 tx=0x7f72e400b880 comp rx=0 tx=0).stop 2024-09-13T17:57:31.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.093+0000 7f72f4beb700 1 -- 172.21.15.40:0/3129868461 shutdown_connections 2024-09-13T17:57:31.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.093+0000 7f72f4beb700 1 --2- 172.21.15.40:0/3129868461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e8046df0 0x7f72e8049970 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.093+0000 7f72f4beb700 1 -- 172.21.15.40:0/3129868461 >> 172.21.15.40:0/3129868461 conn(0x7f72e800dc40 msgr2=0x7f72e8047410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:31.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.093+0000 7f72f4beb700 1 -- 172.21.15.40:0/3129868461 shutdown_connections 2024-09-13T17:57:31.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.093+0000 7f72f4beb700 1 -- 172.21.15.40:0/3129868461 wait complete. 2024-09-13T17:57:31.116 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T17:57:31.116 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T17:57:31.116 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:57:31.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.237+0000 7f0af4ee2700 1 Processor -- start 2024-09-13T17:57:31.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.237+0000 7f0af4ee2700 1 -- start start 2024-09-13T17:57:31.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.241+0000 7f0af4ee2700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0af0156b40 0x7f0af00c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.241+0000 7f0af4ee2700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0af00b1fc0 0x7f0af00c5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.241+0000 7f0af4ee2700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af00c3940 con 0x7f0af00c1990 2024-09-13T17:57:31.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.241+0000 7f0af4ee2700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af00c5c10 con 0x7f0af00b1fc0 2024-09-13T17:57:31.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.241+0000 7f0af4ee2700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af00c4580 con 0x7f0af0156b40 2024-09-13T17:57:31.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.241+0000 7f0aeeffd700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f0af00b1fc0 msgr2=0x7f0af00c5380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:57:31.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.241+0000 7f0aeeffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0af00b1fc0 0x7f0af00c5380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:57:31.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.241+0000 7f0aef7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0af0156b40 0x7f0af00c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.241+0000 7f0aef7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0af0156b40 0x7f0af00c4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:53942/0 (socket says 172.21.15.40:53942) 2024-09-13T17:57:31.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.241+0000 7f0aef7fe700 1 -- 172.21.15.40:0/1957387476 learned_addr learned my addr 172.21.15.40:0/1957387476 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:31.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.241+0000 7f0aef7fe700 1 -- 172.21.15.40:0/1957387476 >> v2:172.21.15.40:3300/0 conn(0x7f0af00b1fc0 msgr2=0x7f0af00c5380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:57:31.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.241+0000 7f0aef7fe700 1 --2- 172.21.15.40:0/1957387476 >> v2:172.21.15.40:3300/0 conn(0x7f0af00b1fc0 0x7f0af00c5380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.241+0000 7f0aef7fe700 1 -- 172.21.15.40:0/1957387476 >> v1:172.21.15.43:6789/0 conn(0x7f0af00c1990 legacy=0x7f0af0071d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.241+0000 7f0aee7fc700 1 -- 172.21.15.40:0/1957387476 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3855683584 0 0) 0x7f0ae8001490 con 0x7f0af00c1990 2024-09-13T17:57:31.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.241+0000 7f0aef7fe700 1 -- 172.21.15.40:0/1957387476 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af00c12b0 con 0x7f0af0156b40 2024-09-13T17:57:31.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.241+0000 7f0aef7fe700 1 --2- 172.21.15.40:0/1957387476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af0156b40 0x7f0af00c4e40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0af00708a0 tx=0x7f0ae0009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67c2c20727975db3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:31.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.241+0000 7f0aee7fc700 1 -- 172.21.15.40:0/1957387476 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae000b8a0 con 0x7f0af0156b40 2024-09-13T17:57:31.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.241+0000 7f0aee7fc700 1 -- 172.21.15.40:0/1957387476 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae000ba00 con 0x7f0af0156b40 2024-09-13T17:57:31.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.241+0000 7f0aee7fc700 1 -- 172.21.15.40:0/1957387476 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae000bd90 con 0x7f0af0156b40 2024-09-13T17:57:31.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.245+0000 7f0af4ee2700 1 -- 172.21.15.40:0/1957387476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af0156b40 msgr2=0x7f0af00c4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.245+0000 7f0af4ee2700 1 --2- 172.21.15.40:0/1957387476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af0156b40 0x7f0af00c4e40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0af00708a0 tx=0x7f0ae0009bf0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.245+0000 7f0af4ee2700 1 -- 172.21.15.40:0/1957387476 shutdown_connections 2024-09-13T17:57:31.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.245+0000 7f0af4ee2700 1 --2- 172.21.15.40:0/1957387476 >> v2:172.21.15.40:3300/0 conn(0x7f0af00b1fc0 0x7f0af00c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.245+0000 7f0af4ee2700 1 --2- 172.21.15.40:0/1957387476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af0156b40 0x7f0af00c4e40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.245+0000 7f0af4ee2700 1 -- 172.21.15.40:0/1957387476 >> 172.21.15.40:0/1957387476 conn(0x7f0af006e4f0 msgr2=0x7f0af006e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:31.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.245+0000 7f0af4ee2700 1 -- 172.21.15.40:0/1957387476 shutdown_connections 2024-09-13T17:57:31.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.245+0000 7f0af4ee2700 1 -- 172.21.15.40:0/1957387476 wait complete. 2024-09-13T17:57:31.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.245+0000 7f0af4ee2700 1 Processor -- start 2024-09-13T17:57:31.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.245+0000 7f0af4ee2700 1 -- start start 2024-09-13T17:57:31.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.245+0000 7f0af4ee2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af00b1fc0 0x7f0af01eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.245+0000 7f0af4ee2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af00c1990 0x7f0af0151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.245+0000 7f0af4ee2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0af0156b40 0x7f0af014da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.245+0000 7f0af4ee2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af01f2c40 con 0x7f0af00c1990 2024-09-13T17:57:31.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.245+0000 7f0af4ee2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af006c570 con 0x7f0af00b1fc0 2024-09-13T17:57:31.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.245+0000 7f0af4ee2700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af006c830 con 0x7f0af0156b40 2024-09-13T17:57:31.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.245+0000 7f0aeeffd700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af00c1990 msgr2=0x7f0af0151230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:57:31.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.245+0000 7f0aeeffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af00c1990 0x7f0af0151230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:57:31.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.249+0000 7f0aef7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af00b1fc0 0x7f0af01eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.249+0000 7f0aef7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af00b1fc0 0x7f0af01eeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:53944/0 (socket says 172.21.15.40:53944) 2024-09-13T17:57:31.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.249+0000 7f0aef7fe700 1 -- 172.21.15.40:0/1996882642 learned_addr learned my addr 172.21.15.40:0/1996882642 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:31.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.249+0000 7f0aef7fe700 1 -- 172.21.15.40:0/1996882642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0af0156b40 msgr2=0x7f0af014da80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:57:31.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.249+0000 7f0aef7fe700 1 --2- 172.21.15.40:0/1996882642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0af0156b40 0x7f0af014da80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.249+0000 7f0aef7fe700 1 -- 172.21.15.40:0/1996882642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af00c1990 msgr2=0x7f0af0151230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:57:31.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.249+0000 7f0aef7fe700 1 --2- 172.21.15.40:0/1996882642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0af00c1990 0x7f0af0151230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.249+0000 7f0aef7fe700 1 -- 172.21.15.40:0/1996882642 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae0009570 con 0x7f0af00b1fc0 2024-09-13T17:57:31.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.249+0000 7f0aef7fe700 1 --2- 172.21.15.40:0/1996882642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af00b1fc0 0x7f0af01eeee0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0000c00 tx=0x7f0ae0004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:31.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.249+0000 7f0ad7fff700 1 -- 172.21.15.40:0/1996882642 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0011e00 con 0x7f0af00b1fc0 2024-09-13T17:57:31.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.249+0000 7f0ad7fff700 1 -- 172.21.15.40:0/1996882642 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae001f070 con 0x7f0af00b1fc0 2024-09-13T17:57:31.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.249+0000 7f0af4ee2700 1 -- 172.21.15.40:0/1996882642 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af01f25f0 con 0x7f0af00b1fc0 2024-09-13T17:57:31.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.249+0000 7f0ad7fff700 1 -- 172.21.15.40:0/1996882642 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae00266a0 con 0x7f0af00b1fc0 2024-09-13T17:57:31.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.249+0000 7f0af4ee2700 1 -- 172.21.15.40:0/1996882642 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af0155820 con 0x7f0af00b1fc0 2024-09-13T17:57:31.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.249+0000 7f0af4ee2700 1 --2- 172.21.15.40:0/1996882642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0af006b730 0x7f0af014dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.249+0000 7f0af4ee2700 1 -- 172.21.15.40:0/1996882642 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af00676b0 con 0x7f0af006b730 2024-09-13T17:57:31.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.249+0000 7f0aeeffd700 1 --2- 172.21.15.40:0/1996882642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0af006b730 0x7f0af014dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.249+0000 7f0aeeffd700 1 -- 172.21.15.40:0/1996882642 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0af00c5df0 con 0x7f0af006b730 2024-09-13T17:57:31.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.253+0000 7f0aeeffd700 1 --2- 172.21.15.40:0/1996882642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0af006b730 0x7f0af014dfc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0af015d770 tx=0x7f0ae400b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:31.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.253+0000 7f0ad7fff700 1 -- 172.21.15.40:0/1996882642 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0ae001a430 con 0x7f0af00b1fc0 2024-09-13T17:57:31.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.253+0000 7f0ad7fff700 1 --2- 172.21.15.40:0/1996882642 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0adc0666e0 0x7f0adc068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.253+0000 7f0ad7fff700 1 -- 172.21.15.40:0/1996882642 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0ae0021070 con 0x7f0af00b1fc0 2024-09-13T17:57:31.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.253+0000 7f0ad7fff700 1 -- 172.21.15.40:0/1996882642 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4003e70 con 0x7f0af006b730 2024-09-13T17:57:31.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.253+0000 7f0ad7fff700 1 -- 172.21.15.40:0/1996882642 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0ae4005070 con 0x7f0af006b730 2024-09-13T17:57:31.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.253+0000 7f0aeffff700 1 --2- 172.21.15.40:0/1996882642 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0adc0666e0 0x7f0adc068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.253+0000 7f0ad7fff700 1 -- 172.21.15.40:0/1996882642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0af006b730 msgr2=0x7f0af014dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.253+0000 7f0ad7fff700 1 --2- 172.21.15.40:0/1996882642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0af006b730 0x7f0af014dfc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0af015d770 tx=0x7f0ae400b800 comp rx=0 tx=0).stop 2024-09-13T17:57:31.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.253+0000 7f0ad7fff700 1 -- 172.21.15.40:0/1996882642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0af006b730 msgr2=0x7f0af014dfc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:57:31.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.253+0000 7f0ad7fff700 1 --2- 172.21.15.40:0/1996882642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0af006b730 0x7f0af014dfc0 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0af015d770 tx=0x7f0ae400b800 comp rx=0 tx=0).stop 2024-09-13T17:57:31.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.253+0000 7f0aeffff700 1 --2- 172.21.15.40:0/1996882642 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0adc0666e0 0x7f0adc068ba0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80016f0 tx=0x7f0ae80017c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:31.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.265+0000 7f0af4ee2700 1 --2- 172.21.15.40:0/1996882642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0af0155f10 0x7f0af014f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:31.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.265+0000 7f0af4ee2700 1 -- 172.21.15.40:0/1996882642 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af00b1770 con 0x7f0af0155f10 2024-09-13T17:57:31.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.265+0000 7f0aeeffd700 1 --2- 172.21.15.40:0/1996882642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0af0155f10 0x7f0af014f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:31.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.265+0000 7f0aeeffd700 1 -- 172.21.15.40:0/1996882642 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0af00c3660 con 0x7f0af0155f10 2024-09-13T17:57:31.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.265+0000 7f0aeeffd700 1 --2- 172.21.15.40:0/1996882642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0af0155f10 0x7f0af014f950 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0af006ce80 tx=0x7f0ae4005a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:31.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.265+0000 7f0ad7fff700 1 -- 172.21.15.40:0/1996882642 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae401e070 con 0x7f0af0155f10 2024-09-13T17:57:31.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.265+0000 7f0ad7fff700 1 -- 172.21.15.40:0/1996882642 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4936 (secure 0 0 0) 0x7f0ae4019c70 con 0x7f0af0155f10 2024-09-13T17:57:31.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.265+0000 7f0ad7fff700 1 -- 172.21.15.40:0/1996882642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0af0155f10 msgr2=0x7f0af014f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.265+0000 7f0ad7fff700 1 --2- 172.21.15.40:0/1996882642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0af0155f10 0x7f0af014f950 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0af006ce80 tx=0x7f0ae4005a80 comp rx=0 tx=0).stop 2024-09-13T17:57:31.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.265+0000 7f0ad7fff700 1 -- 172.21.15.40:0/1996882642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0af0155f10 msgr2=0x7f0af014f950 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:57:31.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.265+0000 7f0ad7fff700 1 --2- 172.21.15.40:0/1996882642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0af0155f10 0x7f0af014f950 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.274 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:57:31.274 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:57:31.274 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:57:31.274 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:57:31.274 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 54, 2024-09-13T17:57:31.274 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:57:31.274 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:57:31.274 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:57:31.274 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:31.274 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T17:57:31.275 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:57:31.275 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:57:31.275 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:57:31.275 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:31.275 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:31.275 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:31.275 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:31.275 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:31.275 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:31.275 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:31.275 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:31.275 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:31.275 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:31.275 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:31.276 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:57:31.276 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:57:31.276 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:31.276 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:31.276 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:31.276 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:31.276 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:31.276 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:31.276 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:31.276 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:31.276 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:31.276 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:31.276 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:31.276 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.277 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:57:31.277 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:57:31.277 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:57:31.277 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:57:31.277 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:57:31.277 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:57:31.277 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:57:31.277 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:57:31.277 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:57:31.277 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:57:31.277 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:57:31.277 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:57:31.277 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:57:31.277 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:57:31.278 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:57:31.278 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:57:31.278 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:57:31.278 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:57:31.278 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:57:31.278 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:57:31.278 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:57:31.278 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:57:31.278 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:57:31.278 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:57:31.278 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:57:31.278 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:57:31.278 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:57:31.278 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:31.279 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:57:31.279 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.279 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:57:31.279 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.279 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:57:31.279 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:57:31.279 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:31.279 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:31.279 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.279 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:31.279 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:57:31.279 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:31.279 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.279 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.280 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:31.280 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:57:31.280 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:31.280 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:31.280 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:31.280 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.280 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:57:31.280 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:57:31.280 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:31.280 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:31.280 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:31.280 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.280 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.281 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:57:31.281 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:57:31.281 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:31.281 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:31.281 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.281 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:31.281 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:57:31.281 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:31.281 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.281 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.281 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:31.281 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:57:31.281 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:31.281 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:31.282 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:31.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.282 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:57:31.282 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:57:31.282 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:31.282 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:31.282 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:31.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.282 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.282 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:57:31.282 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:57:31.282 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:57:31.282 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:57:31.282 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.283 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:57:31.283 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:57:31.283 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:31.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.283 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.283 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:57:31.283 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:57:31.283 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:57:31.283 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:31.283 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:31.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.283 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:57:31.283 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:57:31.283 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:57:31.284 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:57:31.284 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:57:31.284 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:31.284 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:31.284 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.284 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:57:31.284 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:57:31.284 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.284 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:31.284 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:31.284 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:57:31.284 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:31.284 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:31.285 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:57:31.285 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.285 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:31.285 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:31.285 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:57:31.285 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:31.285 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:31.285 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:57:31.285 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.285 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:31.285 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:31.285 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T17:57:31.285 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:31.285 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:31.286 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:57:31.286 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.286 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:31.286 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:31.286 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:57:31.286 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:31.286 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:57:31.286 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:57:31.286 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:57:31.286 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:57:31.286 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:57:31.286 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:57:31.286 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:57:31.286 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:57:31.286 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:57:31.287 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:57:31.287 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:57:31.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.269+0000 7f0af4ee2700 1 -- 172.21.15.40:0/1996882642 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0adc0666e0 msgr2=0x7f0adc068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.269+0000 7f0af4ee2700 1 --2- 172.21.15.40:0/1996882642 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0adc0666e0 0x7f0adc068ba0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80016f0 tx=0x7f0ae80017c0 comp rx=0 tx=0).stop 2024-09-13T17:57:31.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.269+0000 7f0af4ee2700 1 -- 172.21.15.40:0/1996882642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af00b1fc0 msgr2=0x7f0af01eeee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:31.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.269+0000 7f0af4ee2700 1 --2- 172.21.15.40:0/1996882642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0af00b1fc0 0x7f0af01eeee0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0000c00 tx=0x7f0ae0004770 comp rx=0 tx=0).stop 2024-09-13T17:57:31.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.269+0000 7f0aeffff700 1 -- 172.21.15.40:0/1996882642 reap_dead start 2024-09-13T17:57:31.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.269+0000 7f0aeffff700 1 -- 172.21.15.40:0/1996882642 reap_dead start 2024-09-13T17:57:31.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.269+0000 7f0af4ee2700 1 -- 172.21.15.40:0/1996882642 shutdown_connections 2024-09-13T17:57:31.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.269+0000 7f0af4ee2700 1 -- 172.21.15.40:0/1996882642 >> 172.21.15.40:0/1996882642 conn(0x7f0af006e4f0 msgr2=0x7f0af00ca360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:31.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.269+0000 7f0af4ee2700 1 -- 172.21.15.40:0/1996882642 shutdown_connections 2024-09-13T17:57:31.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:31.269+0000 7f0af4ee2700 1 -- 172.21.15.40:0/1996882642 wait complete. 2024-09-13T17:57:31.289 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T17:57:31.289 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T17:57:34.857 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:57:34.859 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:57:51.290 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T17:57:51.290 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-13T17:57:51.291 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T17:57:51.291 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T17:57:51.294 INFO:tasks.ceph.mon.a:Started 2024-09-13T17:57:51.294 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T17:57:51.294 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:57:51.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.420+0000 7f0d1a6f8700 1 Processor -- start 2024-09-13T17:57:51.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.420+0000 7f0d1a6f8700 1 -- start start 2024-09-13T17:57:51.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.420+0000 7f0d1a6f8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0d140c6d80 0x7f0d14155a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.420+0000 7f0d1a6f8700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0d140cb520 0x7f0d14155f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.420+0000 7f0d1a6f8700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d14156650 con 0x7f0d140c69a0 2024-09-13T17:57:51.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.420+0000 7f0d1a6f8700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d14157650 con 0x7f0d140cb520 2024-09-13T17:57:51.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.420+0000 7f0d1a6f8700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d141590e0 con 0x7f0d140c6d80 2024-09-13T17:57:51.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.420+0000 7f0d18c95700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0d140cb520 0x7f0d14155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.420+0000 7f0d137fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0d140c6d80 0x7f0d14155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.420+0000 7f0d18c95700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0d140cb520 0x7f0d14155f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:55822/0 (socket says 172.21.15.40:55822) 2024-09-13T17:57:51.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.420+0000 7f0d18c95700 1 -- 172.21.15.40:0/3752470615 learned_addr learned my addr 172.21.15.40:0/3752470615 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:51.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.420+0000 7f0d18c95700 1 -- 172.21.15.40:0/3752470615 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0d140c6d80 msgr2=0x7f0d14155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:51.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.420+0000 7f0d18c95700 1 --2- 172.21.15.40:0/3752470615 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0d140c6d80 0x7f0d14155a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.420+0000 7f0d18c95700 1 -- 172.21.15.40:0/3752470615 >> v1:172.21.15.43:6789/0 conn(0x7f0d140c69a0 legacy=0x7f0d14156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.420+0000 7f0d18c95700 1 -- 172.21.15.40:0/3752470615 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d141593c0 con 0x7f0d140cb520 2024-09-13T17:57:51.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.420+0000 7f0d18c95700 1 --2- 172.21.15.40:0/3752470615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d140cb520 0x7f0d14155f90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00d150 tx=0x7f0d0c00d460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50d518bb69427236 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:51.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.420+0000 7f0d12ffd700 1 -- 172.21.15.40:0/3752470615 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c010070 con 0x7f0d140cb520 2024-09-13T17:57:51.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.420+0000 7f0d12ffd700 1 -- 172.21.15.40:0/3752470615 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c00db00 con 0x7f0d140cb520 2024-09-13T17:57:51.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.420+0000 7f0d12ffd700 1 -- 172.21.15.40:0/3752470615 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c016400 con 0x7f0d140cb520 2024-09-13T17:57:51.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d1a6f8700 1 -- 172.21.15.40:0/3752470615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d140cb520 msgr2=0x7f0d14155f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d1a6f8700 1 --2- 172.21.15.40:0/3752470615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d140cb520 0x7f0d14155f90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00d150 tx=0x7f0d0c00d460 comp rx=0 tx=0).stop 2024-09-13T17:57:51.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d1a6f8700 1 -- 172.21.15.40:0/3752470615 shutdown_connections 2024-09-13T17:57:51.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d1a6f8700 1 --2- 172.21.15.40:0/3752470615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d140cb520 0x7f0d14155f90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d1a6f8700 1 --2- 172.21.15.40:0/3752470615 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0d140c6d80 0x7f0d14155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d1a6f8700 1 -- 172.21.15.40:0/3752470615 >> 172.21.15.40:0/3752470615 conn(0x7f0d1406a390 msgr2=0x7f0d1406a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:51.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d1a6f8700 1 -- 172.21.15.40:0/3752470615 shutdown_connections 2024-09-13T17:57:51.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d1a6f8700 1 -- 172.21.15.40:0/3752470615 wait complete. 2024-09-13T17:57:51.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d1a6f8700 1 Processor -- start 2024-09-13T17:57:51.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d1a6f8700 1 -- start start 2024-09-13T17:57:51.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d1a6f8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0d140c69a0 0x7f0d140c4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d1a6f8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d140c6d80 0x7f0d141eab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d1a6f8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d140cb520 0x7f0d141eb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d1a6f8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140c5610 con 0x7f0d140c6d80 2024-09-13T17:57:51.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d1a6f8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140719e0 con 0x7f0d140cb520 2024-09-13T17:57:51.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d1a6f8700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140c5000 con 0x7f0d140c69a0 2024-09-13T17:57:51.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d137fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d140c6d80 0x7f0d141eab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d137fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d140c6d80 0x7f0d141eab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42170/0 (socket says 172.21.15.40:42170) 2024-09-13T17:57:51.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d137fe700 1 -- 172.21.15.40:0/1638124029 learned_addr learned my addr 172.21.15.40:0/1638124029 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:51.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d18c95700 1 --2- 172.21.15.40:0/1638124029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d140cb520 0x7f0d141eb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d137fe700 1 -- 172.21.15.40:0/1638124029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d140c6d80 msgr2=0x7f0d141eab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T17:57:51.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d137fe700 1 -- 172.21.15.40:0/1638124029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d140c6d80 msgr2=0x7f0d141eab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:57:51.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d137fe700 1 --2- 172.21.15.40:0/1638124029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d140c6d80 0x7f0d141eab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:57:51.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d13fff700 1 --2- 172.21.15.40:0/1638124029 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0d140c69a0 0x7f0d140c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.424+0000 7f0d137fe700 1 --2- 172.21.15.40:0/1638124029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d140c6d80 0x7f0d141eab00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:57:51.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.428+0000 7f0d18c95700 1 -- 172.21.15.40:0/1638124029 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0d140c69a0 msgr2=0x7f0d140c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.428+0000 7f0d18c95700 1 --2- 172.21.15.40:0/1638124029 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0d140c69a0 0x7f0d140c4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.428+0000 7f0d18c95700 1 -- 172.21.15.40:0/1638124029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d140c6d80 msgr2=0x7f0d141eab00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:57:51.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.428+0000 7f0d18c95700 1 --2- 172.21.15.40:0/1638124029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d140c6d80 0x7f0d141eab00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.428+0000 7f0d18c95700 1 -- 172.21.15.40:0/1638124029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0c00bd40 con 0x7f0d140cb520 2024-09-13T17:57:51.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.428+0000 7f0d18c95700 1 --2- 172.21.15.40:0/1638124029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d140cb520 0x7f0d141eb040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c019000 tx=0x7f0d0c004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:51.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.428+0000 7f0d10ff9700 1 -- 172.21.15.40:0/1638124029 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c025070 con 0x7f0d140cb520 2024-09-13T17:57:51.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.428+0000 7f0d10ff9700 1 -- 172.21.15.40:0/1638124029 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c00b020 con 0x7f0d140cb520 2024-09-13T17:57:51.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.428+0000 7f0d1a6f8700 1 -- 172.21.15.40:0/1638124029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d140c3cb0 con 0x7f0d140cb520 2024-09-13T17:57:51.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.428+0000 7f0d1a6f8700 1 -- 172.21.15.40:0/1638124029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d140c52a0 con 0x7f0d140cb520 2024-09-13T17:57:51.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.428+0000 7f0d10ff9700 1 -- 172.21.15.40:0/1638124029 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c016400 con 0x7f0d140cb520 2024-09-13T17:57:51.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.428+0000 7f0d1a6f8700 1 -- 172.21.15.40:0/1638124029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf4005310 con 0x7f0d140cb520 2024-09-13T17:57:51.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.428+0000 7f0d10ff9700 1 -- 172.21.15.40:0/1638124029 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0d0c00b3d0 con 0x7f0d140cb520 2024-09-13T17:57:51.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.432+0000 7f0d10ff9700 1 --2- 172.21.15.40:0/1638124029 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0d00066730 0x7f0d00068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:51.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.432+0000 7f0d13fff700 1 --2- 172.21.15.40:0/1638124029 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0d00066730 0x7f0d00068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:51.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.432+0000 7f0d10ff9700 1 -- 172.21.15.40:0/1638124029 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0d0c088a30 con 0x7f0d140cb520 2024-09-13T17:57:51.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.432+0000 7f0d13fff700 1 --2- 172.21.15.40:0/1638124029 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0d00066730 0x7f0d00068bf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0d04000f60 tx=0x7f0d04009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:51.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.440+0000 7f0d10ff9700 1 -- 172.21.15.40:0/1638124029 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0d0c055be0 con 0x7f0d140cb520 2024-09-13T17:57:51.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.692+0000 7f0d1a6f8700 1 -- 172.21.15.40:0/1638124029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0cf4005890 con 0x7f0d140cb520 2024-09-13T17:57:51.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.692+0000 7f0d10ff9700 1 -- 172.21.15.40:0/1638124029 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f0d0c059200 con 0x7f0d140cb520 2024-09-13T17:57:51.702 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":54,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:57:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.696+0000 7f0d1a6f8700 1 -- 172.21.15.40:0/1638124029 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0d00066730 msgr2=0x7f0d00068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.696+0000 7f0d1a6f8700 1 --2- 172.21.15.40:0/1638124029 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0d00066730 0x7f0d00068bf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0d04000f60 tx=0x7f0d04009040 comp rx=0 tx=0).stop 2024-09-13T17:57:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.696+0000 7f0d1a6f8700 1 -- 172.21.15.40:0/1638124029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d140cb520 msgr2=0x7f0d141eb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.696+0000 7f0d1a6f8700 1 --2- 172.21.15.40:0/1638124029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d140cb520 0x7f0d141eb040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c019000 tx=0x7f0d0c004af0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.696+0000 7f0d1a6f8700 1 -- 172.21.15.40:0/1638124029 shutdown_connections 2024-09-13T17:57:51.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.696+0000 7f0d1a6f8700 1 --2- 172.21.15.40:0/1638124029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d140cb520 0x7f0d141eb040 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.696+0000 7f0d1a6f8700 1 --2- 172.21.15.40:0/1638124029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0d140c6d80 0x7f0d141eab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.696+0000 7f0d1a6f8700 1 --2- 172.21.15.40:0/1638124029 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0d00066730 0x7f0d00068bf0 secure :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0d04000f60 tx=0x7f0d04009040 comp rx=0 tx=0).stop 2024-09-13T17:57:51.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.696+0000 7f0d1a6f8700 1 --2- 172.21.15.40:0/1638124029 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0d140c69a0 0x7f0d140c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:51.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.696+0000 7f0d1a6f8700 1 -- 172.21.15.40:0/1638124029 >> 172.21.15.40:0/1638124029 conn(0x7f0d1406a390 msgr2=0x7f0d140c7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:51.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.696+0000 7f0d1a6f8700 1 -- 172.21.15.40:0/1638124029 shutdown_connections 2024-09-13T17:57:51.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:51.696+0000 7f0d1a6f8700 1 -- 172.21.15.40:0/1638124029 wait complete. 2024-09-13T17:57:54.718 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:57:54.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.844+0000 7f8865519700 1 Processor -- start 2024-09-13T17:57:54.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.844+0000 7f8865519700 1 -- start start 2024-09-13T17:57:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.844+0000 7f8865519700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88600702e0 0x7f88600722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.844+0000 7f8865519700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f886006b870 con 0x7f8860159890 2024-09-13T17:57:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.844+0000 7f8865519700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8860071dc0 con 0x7f88600c78a0 2024-09-13T17:57:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.844+0000 7f8865519700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886006ed60 con 0x7f88600702e0 2024-09-13T17:57:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.848+0000 7f885ffff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8860159890 0x7f8860159c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55364/0 (socket says 172.21.15.40:55364) 2024-09-13T17:57:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.848+0000 7f885ffff700 1 -- 172.21.15.40:0/180080926 learned_addr learned my addr 172.21.15.40:0/180080926 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:57:54.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.848+0000 7f885f7fe700 1 --2- 172.21.15.40:0/180080926 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88600702e0 0x7f88600722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.848+0000 7f885effd700 1 -- 172.21.15.40:0/180080926 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2679924134 0 0) 0x7f8860071dc0 con 0x7f88600c78a0 2024-09-13T17:57:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.848+0000 7f885effd700 1 -- 172.21.15.40:0/180080926 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88480035d0 con 0x7f88600c78a0 2024-09-13T17:57:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.848+0000 7f885f7fe700 1 -- 172.21.15.40:0/180080926 >> v1:172.21.15.43:6789/0 conn(0x7f88600c78a0 legacy=0x7f886014fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.848+0000 7f885f7fe700 1 -- 172.21.15.40:0/180080926 >> v1:172.21.15.40:6789/0 conn(0x7f8860159890 legacy=0x7f8860159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.848+0000 7f885f7fe700 1 -- 172.21.15.40:0/180080926 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8860070cd0 con 0x7f88600702e0 2024-09-13T17:57:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.848+0000 7f885f7fe700 1 --2- 172.21.15.40:0/180080926 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88600702e0 0x7f88600722c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8854009880 tx=0x7f8854009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7298788b2601844 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:54.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.848+0000 7f885effd700 1 -- 172.21.15.40:0/180080926 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8854011830 con 0x7f88600702e0 2024-09-13T17:57:54.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.848+0000 7f885effd700 1 -- 172.21.15.40:0/180080926 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8854011990 con 0x7f88600702e0 2024-09-13T17:57:54.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.848+0000 7f885effd700 1 -- 172.21.15.40:0/180080926 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8854011d20 con 0x7f88600702e0 2024-09-13T17:57:54.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.848+0000 7f8865519700 1 -- 172.21.15.40:0/180080926 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88600702e0 msgr2=0x7f88600722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.848+0000 7f8865519700 1 --2- 172.21.15.40:0/180080926 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88600702e0 0x7f88600722c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8854009880 tx=0x7f8854009b90 comp rx=0 tx=0).stop 2024-09-13T17:57:54.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.848+0000 7f8865519700 1 -- 172.21.15.40:0/180080926 shutdown_connections 2024-09-13T17:57:54.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.848+0000 7f8865519700 1 --2- 172.21.15.40:0/180080926 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88600702e0 0x7f88600722c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.848+0000 7f8865519700 1 -- 172.21.15.40:0/180080926 >> 172.21.15.40:0/180080926 conn(0x7f88600c7d70 msgr2=0x7f88600c8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:54.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.848+0000 7f8865519700 1 -- 172.21.15.40:0/180080926 shutdown_connections 2024-09-13T17:57:54.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.848+0000 7f8865519700 1 -- 172.21.15.40:0/180080926 wait complete. 2024-09-13T17:57:54.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f8865519700 1 Processor -- start 2024-09-13T17:57:54.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f8865519700 1 -- start start 2024-09-13T17:57:54.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f8865519700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88600702e0 0x7f8860152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f8865519700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88600c78a0 0x7f886006d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f8865519700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8860159890 0x7f88601f2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f8865519700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88601532e0 con 0x7f88600c78a0 2024-09-13T17:57:54.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f8865519700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8860153000 con 0x7f88600702e0 2024-09-13T17:57:54.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f8865519700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8860146da0 con 0x7f8860159890 2024-09-13T17:57:54.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f885f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88600c78a0 0x7f886006d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f885f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88600c78a0 0x7f886006d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42186/0 (socket says 172.21.15.40:42186) 2024-09-13T17:57:54.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f885f7fe700 1 -- 172.21.15.40:0/2815269549 learned_addr learned my addr 172.21.15.40:0/2815269549 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:54.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f885ffff700 1 --2- 172.21.15.40:0/2815269549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88600702e0 0x7f8860152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f8864d18700 1 --2- 172.21.15.40:0/2815269549 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8860159890 0x7f88601f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f885f7fe700 1 -- 172.21.15.40:0/2815269549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88600c78a0 msgr2=0x7f886006d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-13T17:57:54.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f885f7fe700 1 -- 172.21.15.40:0/2815269549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88600c78a0 msgr2=0x7f886006d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:57:54.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f885f7fe700 1 --2- 172.21.15.40:0/2815269549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88600c78a0 0x7f886006d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:57:54.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f885f7fe700 1 --2- 172.21.15.40:0/2815269549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88600c78a0 0x7f886006d860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:57:54.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f8864d18700 1 -- 172.21.15.40:0/2815269549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88600702e0 msgr2=0x7f8860152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:54.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f8864d18700 1 --2- 172.21.15.40:0/2815269549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88600702e0 0x7f8860152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f8864d18700 1 -- 172.21.15.40:0/2815269549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88600c78a0 msgr2=0x7f886006d860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:57:54.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f8864d18700 1 --2- 172.21.15.40:0/2815269549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88600c78a0 0x7f886006d860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:54.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f8864d18700 1 -- 172.21.15.40:0/2815269549 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88540094b0 con 0x7f8860159890 2024-09-13T17:57:54.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f885ffff700 1 --2- 172.21.15.40:0/2815269549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88600702e0 0x7f8860152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:54.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.852+0000 7f8864d18700 1 --2- 172.21.15.40:0/2815269549 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8860159890 0x7f88601f2460 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f885800cb90 tx=0x7f885800cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:54.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.856+0000 7f885cff9700 1 -- 172.21.15.40:0/2815269549 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8858003c20 con 0x7f8860159890 2024-09-13T17:57:54.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.856+0000 7f885cff9700 1 -- 172.21.15.40:0/2815269549 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8858003d80 con 0x7f8860159890 2024-09-13T17:57:54.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.856+0000 7f8865519700 1 -- 172.21.15.40:0/2815269549 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8860147080 con 0x7f8860159890 2024-09-13T17:57:54.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.856+0000 7f885cff9700 1 -- 172.21.15.40:0/2815269549 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885800aaa0 con 0x7f8860159890 2024-09-13T17:57:54.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.856+0000 7f8865519700 1 -- 172.21.15.40:0/2815269549 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8860147640 con 0x7f8860159890 2024-09-13T17:57:54.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.856+0000 7f8865519700 1 -- 172.21.15.40:0/2815269549 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88600722c0 con 0x7f8860159890 2024-09-13T17:57:54.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.856+0000 7f885cff9700 1 -- 172.21.15.40:0/2815269549 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f885800ac00 con 0x7f8860159890 2024-09-13T17:57:54.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.856+0000 7f885cff9700 1 --2- 172.21.15.40:0/2815269549 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f884c066760 0x7f884c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:54.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.856+0000 7f885ffff700 1 --2- 172.21.15.40:0/2815269549 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f884c066760 0x7f884c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:54.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.860+0000 7f885cff9700 1 -- 172.21.15.40:0/2815269549 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f88580844d0 con 0x7f8860159890 2024-09-13T17:57:54.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.860+0000 7f885ffff700 1 --2- 172.21.15.40:0/2815269549 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f884c066760 0x7f884c068c20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8850005930 tx=0x7f8850009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:54.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:54.872+0000 7f885cff9700 1 -- 172.21.15.40:0/2815269549 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8858051480 con 0x7f8860159890 2024-09-13T17:57:55.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:55.120+0000 7f8865519700 1 -- 172.21.15.40:0/2815269549 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8860150530 con 0x7f8860159890 2024-09-13T17:57:55.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:55.124+0000 7f885cff9700 1 -- 172.21.15.40:0/2815269549 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f8858054aa0 con 0x7f8860159890 2024-09-13T17:57:55.131 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":54,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":24,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:57:55.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:55.124+0000 7f8865519700 1 -- 172.21.15.40:0/2815269549 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f884c066760 msgr2=0x7f884c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:55.124+0000 7f8865519700 1 --2- 172.21.15.40:0/2815269549 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f884c066760 0x7f884c068c20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8850005930 tx=0x7f8850009040 comp rx=0 tx=0).stop 2024-09-13T17:57:55.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:55.124+0000 7f8865519700 1 -- 172.21.15.40:0/2815269549 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8860159890 msgr2=0x7f88601f2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:55.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:55.124+0000 7f8865519700 1 --2- 172.21.15.40:0/2815269549 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8860159890 0x7f88601f2460 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f885800cb90 tx=0x7f885800cea0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:55.124+0000 7f8865519700 1 -- 172.21.15.40:0/2815269549 shutdown_connections 2024-09-13T17:57:55.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:55.124+0000 7f8865519700 1 --2- 172.21.15.40:0/2815269549 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f884c066760 0x7f884c068c20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:55.124+0000 7f8865519700 1 --2- 172.21.15.40:0/2815269549 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8860159890 0x7f88601f2460 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:55.124+0000 7f8865519700 1 --2- 172.21.15.40:0/2815269549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88600c78a0 0x7f886006d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:55.124+0000 7f8865519700 1 --2- 172.21.15.40:0/2815269549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88600702e0 0x7f8860152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:55.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:55.124+0000 7f8865519700 1 -- 172.21.15.40:0/2815269549 >> 172.21.15.40:0/2815269549 conn(0x7f88600c7d70 msgr2=0x7f8860159c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:55.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:55.124+0000 7f8865519700 1 -- 172.21.15.40:0/2815269549 shutdown_connections 2024-09-13T17:57:55.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:55.124+0000 7f8865519700 1 -- 172.21.15.40:0/2815269549 wait complete. 2024-09-13T17:57:58.146 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:57:58.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.256+0000 7f0991472700 1 Processor -- start 2024-09-13T17:57:58.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.256+0000 7f0991472700 1 -- start start 2024-09-13T17:57:58.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0991472700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f098c159890 0x7f098c0722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:58.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0991472700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f098c0702e0 0x7f098c0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:58.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0991472700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f098c0c78a0 0x7f098c159c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:58.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0991472700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098c06ed60 con 0x7f098c0c78a0 2024-09-13T17:57:58.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0991472700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098c06b870 con 0x7f098c0702e0 2024-09-13T17:57:58.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0991472700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098c071dc0 con 0x7f098c159890 2024-09-13T17:57:58.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0990c71700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f098c0c78a0 0x7f098c159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:58.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0990c71700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f098c0c78a0 0x7f098c159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42190/0 (socket says 172.21.15.40:42190) 2024-09-13T17:57:58.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0990c71700 1 -- 172.21.15.40:0/4220230510 learned_addr learned my addr 172.21.15.40:0/4220230510 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:58.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0990c71700 1 -- 172.21.15.40:0/4220230510 >> v2:172.21.15.40:3300/0 conn(0x7f098c0c78a0 msgr2=0x7f098c159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:57:58.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0990c71700 1 -- 172.21.15.40:0/4220230510 >> v2:172.21.15.40:3300/0 conn(0x7f098c0c78a0 msgr2=0x7f098c159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:57:58.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0990c71700 1 --2- 172.21.15.40:0/4220230510 >> v2:172.21.15.40:3300/0 conn(0x7f098c0c78a0 0x7f098c159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:57:58.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0990c71700 1 --2- 172.21.15.40:0/4220230510 >> v2:172.21.15.40:3300/0 conn(0x7f098c0c78a0 0x7f098c159c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:57:58.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f098b7fe700 1 --2- 172.21.15.40:0/4220230510 >> v2:172.21.15.43:3300/0 conn(0x7f098c0702e0 0x7f098c0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:58.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f098bfff700 1 --2- 172.21.15.40:0/4220230510 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f098c159890 0x7f098c0722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:58.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f098b7fe700 1 -- 172.21.15.40:0/4220230510 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f098c159890 msgr2=0x7f098c0722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:57:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f098b7fe700 1 --2- 172.21.15.40:0/4220230510 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f098c159890 0x7f098c0722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f098b7fe700 1 -- 172.21.15.40:0/4220230510 >> v2:172.21.15.40:3300/0 conn(0x7f098c0c78a0 msgr2=0x7f098c159c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:57:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f098b7fe700 1 --2- 172.21.15.40:0/4220230510 >> v2:172.21.15.40:3300/0 conn(0x7f098c0c78a0 0x7f098c159c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f098b7fe700 1 -- 172.21.15.40:0/4220230510 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c070cd0 con 0x7f098c0702e0 2024-09-13T17:57:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f098b7fe700 1 --2- 172.21.15.40:0/4220230510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f098c0702e0 0x7f098c0c5c70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f098c0cc3b0 tx=0x7f0980009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c49f51665cbaa43 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f098affd700 1 -- 172.21.15.40:0/4220230510 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0980011830 con 0x7f098c0702e0 2024-09-13T17:57:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f098affd700 1 -- 172.21.15.40:0/4220230510 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0980011990 con 0x7f098c0702e0 2024-09-13T17:57:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f098affd700 1 -- 172.21.15.40:0/4220230510 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0980011d20 con 0x7f098c0702e0 2024-09-13T17:57:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0991472700 1 -- 172.21.15.40:0/4220230510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f098c0702e0 msgr2=0x7f098c0c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0991472700 1 --2- 172.21.15.40:0/4220230510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f098c0702e0 0x7f098c0c5c70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f098c0cc3b0 tx=0x7f0980009b30 comp rx=0 tx=0).stop 2024-09-13T17:57:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0991472700 1 -- 172.21.15.40:0/4220230510 shutdown_connections 2024-09-13T17:57:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0991472700 1 --2- 172.21.15.40:0/4220230510 >> v2:172.21.15.40:3300/0 conn(0x7f098c0c78a0 0x7f098c159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:58.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0991472700 1 --2- 172.21.15.40:0/4220230510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f098c0702e0 0x7f098c0c5c70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:58.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0991472700 1 --2- 172.21.15.40:0/4220230510 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f098c159890 0x7f098c0722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:58.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0991472700 1 -- 172.21.15.40:0/4220230510 >> 172.21.15.40:0/4220230510 conn(0x7f098c0c7d70 msgr2=0x7f098c0c8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:58.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0991472700 1 -- 172.21.15.40:0/4220230510 shutdown_connections 2024-09-13T17:57:58.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.260+0000 7f0991472700 1 -- 172.21.15.40:0/4220230510 wait complete. 2024-09-13T17:57:58.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f0991472700 1 Processor -- start 2024-09-13T17:57:58.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f0991472700 1 -- start start 2024-09-13T17:57:58.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f0991472700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f098c0702e0 0x7f098c06e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:58.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f0991472700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f098c0c78a0 0x7f098c0b8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:58.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f0991472700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f098c159890 0x7f098c0b91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:58.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f0991472700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c0c16b0 con 0x7f098c0c78a0 2024-09-13T17:57:58.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f0991472700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c1ed690 con 0x7f098c159890 2024-09-13T17:57:58.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f0991472700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098c1ed900 con 0x7f098c0702e0 2024-09-13T17:57:58.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f098b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f098c0c78a0 0x7f098c0b8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:58.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f098b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f098c0c78a0 0x7f098c0b8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42206/0 (socket says 172.21.15.40:42206) 2024-09-13T17:57:58.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f098b7fe700 1 -- 172.21.15.40:0/2102479764 learned_addr learned my addr 172.21.15.40:0/2102479764 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:57:58.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f098b7fe700 1 -- 172.21.15.40:0/2102479764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f098c0c78a0 msgr2=0x7f098c0b8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T17:57:58.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f098b7fe700 1 -- 172.21.15.40:0/2102479764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f098c0c78a0 msgr2=0x7f098c0b8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:57:58.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f098b7fe700 1 --2- 172.21.15.40:0/2102479764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f098c0c78a0 0x7f098c0b8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:57:58.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f0990c71700 1 --2- 172.21.15.40:0/2102479764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f098c159890 0x7f098c0b91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:58.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f098b7fe700 1 --2- 172.21.15.40:0/2102479764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f098c0c78a0 0x7f098c0b8160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:57:58.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f098bfff700 1 --2- 172.21.15.40:0/2102479764 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f098c0702e0 0x7f098c06e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:58.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f0990c71700 1 -- 172.21.15.40:0/2102479764 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f098c0702e0 msgr2=0x7f098c06e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:58.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f0990c71700 1 --2- 172.21.15.40:0/2102479764 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f098c0702e0 0x7f098c06e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:58.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f0990c71700 1 -- 172.21.15.40:0/2102479764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f098c0c78a0 msgr2=0x7f098c0b8160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:57:58.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f0990c71700 1 --2- 172.21.15.40:0/2102479764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f098c0c78a0 0x7f098c0b8160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:58.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f0990c71700 1 -- 172.21.15.40:0/2102479764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09800094b0 con 0x7f098c159890 2024-09-13T17:57:58.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f098bfff700 1 --2- 172.21.15.40:0/2102479764 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f098c0702e0 0x7f098c06e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:57:58.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.264+0000 7f0990c71700 1 --2- 172.21.15.40:0/2102479764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f098c159890 0x7f098c0b91f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f098c0c5930 tx=0x7f0984007d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:58.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.268+0000 7f0988ff9700 1 -- 172.21.15.40:0/2102479764 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098400d9d0 con 0x7f098c159890 2024-09-13T17:57:58.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.268+0000 7f0991472700 1 -- 172.21.15.40:0/2102479764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f098c0c1900 con 0x7f098c159890 2024-09-13T17:57:58.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.268+0000 7f0988ff9700 1 -- 172.21.15.40:0/2102479764 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098400f040 con 0x7f098c159890 2024-09-13T17:57:58.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.268+0000 7f0988ff9700 1 -- 172.21.15.40:0/2102479764 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098400dd30 con 0x7f098c159890 2024-09-13T17:57:58.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.268+0000 7f0991472700 1 -- 172.21.15.40:0/2102479764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f098c0c8930 con 0x7f098c159890 2024-09-13T17:57:58.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.268+0000 7f0991472700 1 -- 172.21.15.40:0/2102479764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f098c0c1e20 con 0x7f098c159890 2024-09-13T17:57:58.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.268+0000 7f0988ff9700 1 -- 172.21.15.40:0/2102479764 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f098401e3b0 con 0x7f098c159890 2024-09-13T17:57:58.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.268+0000 7f0988ff9700 1 --2- 172.21.15.40:0/2102479764 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0978066730 0x7f0978068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:57:58.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.268+0000 7f098bfff700 1 --2- 172.21.15.40:0/2102479764 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0978066730 0x7f0978068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:57:58.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.268+0000 7f0988ff9700 1 -- 172.21.15.40:0/2102479764 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f09840843f0 con 0x7f098c159890 2024-09-13T17:57:58.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.268+0000 7f098bfff700 1 --2- 172.21.15.40:0/2102479764 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0978066730 0x7f0978068bf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f097c009ec0 tx=0x7f097c009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:57:58.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.276+0000 7f0988ff9700 1 -- 172.21.15.40:0/2102479764 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f09840513c0 con 0x7f098c159890 2024-09-13T17:57:58.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.536+0000 7f0991472700 1 -- 172.21.15.40:0/2102479764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f098c0c1e20 con 0x7f098c159890 2024-09-13T17:57:58.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.536+0000 7f0988ff9700 1 -- 172.21.15.40:0/2102479764 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f098c0c1e20 con 0x7f098c159890 2024-09-13T17:57:58.543 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":54,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":28,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:57:58.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.536+0000 7f0991472700 1 -- 172.21.15.40:0/2102479764 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0978066730 msgr2=0x7f0978068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:58.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.536+0000 7f0991472700 1 --2- 172.21.15.40:0/2102479764 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0978066730 0x7f0978068bf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f097c009ec0 tx=0x7f097c009240 comp rx=0 tx=0).stop 2024-09-13T17:57:58.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.536+0000 7f0991472700 1 -- 172.21.15.40:0/2102479764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f098c159890 msgr2=0x7f098c0b91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:57:58.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.536+0000 7f0991472700 1 --2- 172.21.15.40:0/2102479764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f098c159890 0x7f098c0b91f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f098c0c5930 tx=0x7f0984007d40 comp rx=0 tx=0).stop 2024-09-13T17:57:58.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.536+0000 7f0991472700 1 -- 172.21.15.40:0/2102479764 shutdown_connections 2024-09-13T17:57:58.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.536+0000 7f0991472700 1 --2- 172.21.15.40:0/2102479764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f098c159890 0x7f098c0b91f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:58.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.536+0000 7f0991472700 1 --2- 172.21.15.40:0/2102479764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f098c0c78a0 0x7f098c0b8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:58.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.536+0000 7f0991472700 1 --2- 172.21.15.40:0/2102479764 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0978066730 0x7f0978068bf0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:58.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.536+0000 7f0991472700 1 --2- 172.21.15.40:0/2102479764 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f098c0702e0 0x7f098c06e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:57:58.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.536+0000 7f0991472700 1 -- 172.21.15.40:0/2102479764 >> 172.21.15.40:0/2102479764 conn(0x7f098c0c7d70 msgr2=0x7f098c0c3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:57:58.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.536+0000 7f0991472700 1 -- 172.21.15.40:0/2102479764 shutdown_connections 2024-09-13T17:57:58.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:57:58.536+0000 7f0991472700 1 -- 172.21.15.40:0/2102479764 wait complete. 2024-09-13T17:58:01.561 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:58:01.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.676+0000 7fb5a1647700 1 Processor -- start 2024-09-13T17:58:01.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.676+0000 7fb5a1647700 1 -- start start 2024-09-13T17:58:01.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.676+0000 7fb5a1647700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb59c0c4c90 0x7fb59c159670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:01.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.676+0000 7fb5a1647700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb59c070370 0x7fb59c159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:01.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.676+0000 7fb5a1647700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59c0710a0 con 0x7fb59c150250 2024-09-13T17:58:01.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.676+0000 7fb5a1647700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb59c072900 con 0x7fb59c0c4c90 2024-09-13T17:58:01.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.676+0000 7fb5a1647700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb59c071d80 con 0x7fb59c070370 2024-09-13T17:58:01.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.676+0000 7fb59b7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb59c0c4c90 0x7fb59c159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:01.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.676+0000 7fb59bfff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb59c150250 0x7fb59c0c4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55750/0 (socket says 172.21.15.40:55750) 2024-09-13T17:58:01.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.676+0000 7fb59bfff700 1 -- 172.21.15.40:0/2379998568 learned_addr learned my addr 172.21.15.40:0/2379998568 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:58:01.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.676+0000 7fb59b7fe700 1 -- 172.21.15.40:0/2379998568 >> v2:172.21.15.40:3300/0 conn(0x7fb59c0c4c90 msgr2=0x7fb59c159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:58:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.676+0000 7fb59b7fe700 1 -- 172.21.15.40:0/2379998568 >> v2:172.21.15.40:3300/0 conn(0x7fb59c0c4c90 msgr2=0x7fb59c159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.676+0000 7fb59b7fe700 1 --2- 172.21.15.40:0/2379998568 >> v2:172.21.15.40:3300/0 conn(0x7fb59c0c4c90 0x7fb59c159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.676+0000 7fb59b7fe700 1 --2- 172.21.15.40:0/2379998568 >> v2:172.21.15.40:3300/0 conn(0x7fb59c0c4c90 0x7fb59c159670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:01.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.676+0000 7fb5a0e46700 1 --2- 172.21.15.40:0/2379998568 >> v2:172.21.15.43:3300/0 conn(0x7fb59c070370 0x7fb59c159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:01.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.676+0000 7fb5a0e46700 1 -- 172.21.15.40:0/2379998568 >> v2:172.21.15.40:3300/0 conn(0x7fb59c0c4c90 msgr2=0x7fb59c159670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:58:01.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.676+0000 7fb5a0e46700 1 --2- 172.21.15.40:0/2379998568 >> v2:172.21.15.40:3300/0 conn(0x7fb59c0c4c90 0x7fb59c159670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:01.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.676+0000 7fb5a0e46700 1 -- 172.21.15.40:0/2379998568 >> v1:172.21.15.40:6789/0 conn(0x7fb59c150250 legacy=0x7fb59c0c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:01.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.676+0000 7fb5a0e46700 1 -- 172.21.15.40:0/2379998568 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c072060 con 0x7fb59c070370 2024-09-13T17:58:01.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.676+0000 7fb5a0e46700 1 --2- 172.21.15.40:0/2379998568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb59c070370 0x7fb59c159bb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0c5b80 tx=0x7fb59400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=621409451a8aa9cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:01.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.680+0000 7fb59affd700 1 -- 172.21.15.40:0/2379998568 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59400bdd0 con 0x7fb59c070370 2024-09-13T17:58:01.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.680+0000 7fb59affd700 1 -- 172.21.15.40:0/2379998568 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb594011460 con 0x7fb59c070370 2024-09-13T17:58:01.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.680+0000 7fb59affd700 1 -- 172.21.15.40:0/2379998568 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb594011810 con 0x7fb59c070370 2024-09-13T17:58:01.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.680+0000 7fb5a1647700 1 -- 172.21.15.40:0/2379998568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb59c070370 msgr2=0x7fb59c159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:01.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.680+0000 7fb5a1647700 1 --2- 172.21.15.40:0/2379998568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb59c070370 0x7fb59c159bb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0c5b80 tx=0x7fb59400b330 comp rx=0 tx=0).stop 2024-09-13T17:58:01.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.680+0000 7fb5a1647700 1 -- 172.21.15.40:0/2379998568 shutdown_connections 2024-09-13T17:58:01.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.680+0000 7fb5a1647700 1 --2- 172.21.15.40:0/2379998568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb59c070370 0x7fb59c159bb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:01.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.680+0000 7fb5a1647700 1 --2- 172.21.15.40:0/2379998568 >> v2:172.21.15.40:3300/0 conn(0x7fb59c0c4c90 0x7fb59c159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:01.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.680+0000 7fb5a1647700 1 -- 172.21.15.40:0/2379998568 >> 172.21.15.40:0/2379998568 conn(0x7fb59c0cb4c0 msgr2=0x7fb59c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:01.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.680+0000 7fb5a1647700 1 -- 172.21.15.40:0/2379998568 shutdown_connections 2024-09-13T17:58:01.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.680+0000 7fb5a1647700 1 -- 172.21.15.40:0/2379998568 wait complete. 2024-09-13T17:58:01.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb5a1647700 1 Processor -- start 2024-09-13T17:58:01.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb5a1647700 1 -- start start 2024-09-13T17:58:01.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb5a1647700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb59c070370 0x7fb59c147760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:01.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb5a1647700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb59c0c4c90 0x7fb59c148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:01.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb5a1647700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb59c150250 0x7fb59c0c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:01.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb5a1647700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c0c5d30 con 0x7fb59c0c4c90 2024-09-13T17:58:01.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb5a1647700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c0c6b90 con 0x7fb59c070370 2024-09-13T17:58:01.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb5a1647700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c0c6300 con 0x7fb59c150250 2024-09-13T17:58:01.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb59b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb59c0c4c90 0x7fb59c148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:01.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb59b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb59c0c4c90 0x7fb59c148670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32994/0 (socket says 172.21.15.40:32994) 2024-09-13T17:58:01.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb59b7fe700 1 -- 172.21.15.40:0/3238664003 learned_addr learned my addr 172.21.15.40:0/3238664003 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:01.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb59bfff700 1 --2- 172.21.15.40:0/3238664003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb59c070370 0x7fb59c147760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:01.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb59b7fe700 1 -- 172.21.15.40:0/3238664003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb59c0c4c90 msgr2=0x7fb59c148670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T17:58:01.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb5a0e46700 1 --2- 172.21.15.40:0/3238664003 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb59c150250 0x7fb59c0c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:01.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb59b7fe700 1 -- 172.21.15.40:0/3238664003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb59c0c4c90 msgr2=0x7fb59c148670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:58:01.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb59b7fe700 1 --2- 172.21.15.40:0/3238664003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb59c0c4c90 0x7fb59c148670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:01.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb59b7fe700 1 --2- 172.21.15.40:0/3238664003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb59c0c4c90 0x7fb59c148670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:01.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb5a0e46700 1 -- 172.21.15.40:0/3238664003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb59c070370 msgr2=0x7fb59c147760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:01.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb5a0e46700 1 --2- 172.21.15.40:0/3238664003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb59c070370 0x7fb59c147760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:01.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb5a0e46700 1 -- 172.21.15.40:0/3238664003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb59c0c4c90 msgr2=0x7fb59c148670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:58:01.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb5a0e46700 1 --2- 172.21.15.40:0/3238664003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb59c0c4c90 0x7fb59c148670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:01.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb5a0e46700 1 -- 172.21.15.40:0/3238664003 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb594009a70 con 0x7fb59c150250 2024-09-13T17:58:01.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb5a0e46700 1 --2- 172.21.15.40:0/3238664003 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb59c150250 0x7fb59c0c7070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb594018000 tx=0x7fb594004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:01.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb598ff9700 1 -- 172.21.15.40:0/3238664003 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb594011dd0 con 0x7fb59c150250 2024-09-13T17:58:01.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.684+0000 7fb598ff9700 1 -- 172.21.15.40:0/3238664003 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb59401e070 con 0x7fb59c150250 2024-09-13T17:58:01.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.688+0000 7fb5a1647700 1 -- 172.21.15.40:0/3238664003 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59c0c65c0 con 0x7fb59c150250 2024-09-13T17:58:01.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.688+0000 7fb598ff9700 1 -- 172.21.15.40:0/3238664003 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb594004280 con 0x7fb59c150250 2024-09-13T17:58:01.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.688+0000 7fb5a1647700 1 -- 172.21.15.40:0/3238664003 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59c146aa0 con 0x7fb59c150250 2024-09-13T17:58:01.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.688+0000 7fb5a1647700 1 -- 172.21.15.40:0/3238664003 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59c070c70 con 0x7fb59c150250 2024-09-13T17:58:01.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.688+0000 7fb598ff9700 1 -- 172.21.15.40:0/3238664003 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb5940043e0 con 0x7fb59c150250 2024-09-13T17:58:01.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.688+0000 7fb598ff9700 1 --2- 172.21.15.40:0/3238664003 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb588066760 0x7fb588068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:01.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.688+0000 7fb59bfff700 1 --2- 172.21.15.40:0/3238664003 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb588066760 0x7fb588068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:01.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.688+0000 7fb598ff9700 1 -- 172.21.15.40:0/3238664003 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb594088c10 con 0x7fb59c150250 2024-09-13T17:58:01.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.692+0000 7fb59bfff700 1 --2- 172.21.15.40:0/3238664003 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb588066760 0x7fb588068c20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb58c000f60 tx=0x7fb58c007f30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:01.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.700+0000 7fb598ff9700 1 -- 172.21.15.40:0/3238664003 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb594055ce0 con 0x7fb59c150250 2024-09-13T17:58:01.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.964+0000 7fb5a1647700 1 -- 172.21.15.40:0/3238664003 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb59c145710 con 0x7fb59c150250 2024-09-13T17:58:01.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.964+0000 7fb598ff9700 1 -- 172.21.15.40:0/3238664003 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7fb594059300 con 0x7fb59c150250 2024-09-13T17:58:01.973 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":54,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":31,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:58:01.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.968+0000 7fb5a1647700 1 -- 172.21.15.40:0/3238664003 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb588066760 msgr2=0x7fb588068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:01.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.968+0000 7fb5a1647700 1 --2- 172.21.15.40:0/3238664003 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb588066760 0x7fb588068c20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb58c000f60 tx=0x7fb58c007f30 comp rx=0 tx=0).stop 2024-09-13T17:58:01.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.968+0000 7fb5a1647700 1 -- 172.21.15.40:0/3238664003 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb59c150250 msgr2=0x7fb59c0c7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:01.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.968+0000 7fb5a1647700 1 --2- 172.21.15.40:0/3238664003 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb59c150250 0x7fb59c0c7070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb594018000 tx=0x7fb594004af0 comp rx=0 tx=0).stop 2024-09-13T17:58:01.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.968+0000 7fb5a1647700 1 -- 172.21.15.40:0/3238664003 shutdown_connections 2024-09-13T17:58:01.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.968+0000 7fb5a1647700 1 --2- 172.21.15.40:0/3238664003 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb588066760 0x7fb588068c20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:01.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.968+0000 7fb5a1647700 1 --2- 172.21.15.40:0/3238664003 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb59c150250 0x7fb59c0c7070 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:01.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.968+0000 7fb5a1647700 1 --2- 172.21.15.40:0/3238664003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb59c0c4c90 0x7fb59c148670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:01.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.968+0000 7fb5a1647700 1 --2- 172.21.15.40:0/3238664003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb59c070370 0x7fb59c147760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:01.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.968+0000 7fb5a1647700 1 -- 172.21.15.40:0/3238664003 >> 172.21.15.40:0/3238664003 conn(0x7fb59c0cb4c0 msgr2=0x7fb59c0c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:01.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.968+0000 7fb5a1647700 1 -- 172.21.15.40:0/3238664003 shutdown_connections 2024-09-13T17:58:01.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:01.968+0000 7fb5a1647700 1 -- 172.21.15.40:0/3238664003 wait complete. 2024-09-13T17:58:04.957 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:58:04.959 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:58:04.989 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:58:05.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.156+0000 7fc308fe9700 1 Processor -- start 2024-09-13T17:58:05.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.156+0000 7fc308fe9700 1 -- start start 2024-09-13T17:58:05.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.160+0000 7fc308fe9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 0x7fc2fc04b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:05.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.160+0000 7fc308fe9700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 0x7fc2fc046e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:05.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.160+0000 7fc308fe9700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2fc047610 con 0x7fc2fc0d87a0 2024-09-13T17:58:05.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.160+0000 7fc308fe9700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2fc006b50 con 0x7fc2fc0469c0 2024-09-13T17:58:05.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.160+0000 7fc308fe9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc0076d0 con 0x7fc2fc007b60 2024-09-13T17:58:05.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.160+0000 7fc3037fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc2fc0d87a0 0x7fc2fc0df110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55762/0 (socket says 172.21.15.40:55762) 2024-09-13T17:58:05.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.160+0000 7fc3037fe700 1 -- 172.21.15.40:0/36623815 learned_addr learned my addr 172.21.15.40:0/36623815 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:58:05.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.160+0000 7fc303fff700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 0x7fc2fc046e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:05.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.160+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 0x7fc2fc04b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:05.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.160+0000 7fc303fff700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 msgr2=0x7fc2fc046e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:58:05.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.160+0000 7fc303fff700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 msgr2=0x7fc2fc046e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:05.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.160+0000 7fc303fff700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 0x7fc2fc046e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:05.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.160+0000 7fc303fff700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 0x7fc2fc046e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:05.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.160+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 msgr2=0x7fc2fc04b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:58:05.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.160+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 msgr2=0x7fc2fc04b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:05.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.160+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 0x7fc2fc04b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:05.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.160+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 0x7fc2fc04b7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:05.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.364+0000 7fc303fff700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 0x7fc2fc046e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:05.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.364+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 0x7fc2fc04b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:05.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.364+0000 7fc303fff700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 msgr2=0x7fc2fc046e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:58:05.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.364+0000 7fc303fff700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 msgr2=0x7fc2fc046e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:05.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.364+0000 7fc303fff700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 0x7fc2fc046e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:05.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.364+0000 7fc303fff700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 0x7fc2fc046e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:58:05.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.364+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 msgr2=0x7fc2fc04b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:58:05.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.364+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 msgr2=0x7fc2fc04b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:05.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.364+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 0x7fc2fc04b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:05.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.364+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 0x7fc2fc04b7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:58:05.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.768+0000 7fc303fff700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 0x7fc2fc046e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:05.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.768+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 0x7fc2fc04b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:05.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.768+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 msgr2=0x7fc2fc04b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:58:05.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.768+0000 7fc303fff700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 msgr2=0x7fc2fc046e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:58:05.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.768+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 msgr2=0x7fc2fc04b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:05.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.768+0000 7fc303fff700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 msgr2=0x7fc2fc046e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:05.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.768+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 0x7fc2fc04b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:05.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.768+0000 7fc303fff700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 0x7fc2fc046e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:05.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.768+0000 7fc303fff700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 0x7fc2fc046e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:58:05.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:05.768+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 0x7fc2fc04b7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:58:06.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:06.408+0000 7fc3037fe700 1 -- 172.21.15.40:0/36623815 >> v1:172.21.15.40:6789/0 conn(0x7fc2fc0d87a0 legacy=0x7fc2fc0df110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-13T17:58:06.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:06.408+0000 7fc3037fe700 1 -- 172.21.15.40:0/36623815 >> v1:172.21.15.40:6789/0 conn(0x7fc2fc0d87a0 legacy=0x7fc2fc0df110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:58:06.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:06.408+0000 7fc3037fe700 1 --1- 172.21.15.40:0/36623815 >> v1:172.21.15.40:6789/0 conn(0x7fc2fc0d87a0 0x7fc2fc0df110 :-1 s=OPENED pgs=30 cs=1 l=1).handle_message read tag failed 2024-09-13T17:58:06.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:06.408+0000 7fc3037fe700 1 --1- 172.21.15.40:0/36623815 >> v1:172.21.15.40:6789/0 conn(0x7fc2fc0d87a0 0x7fc2fc0df110 :-1 s=OPENED pgs=30 cs=1 l=1).fault on lossy channel, failing 2024-09-13T17:58:06.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:06.572+0000 7fc303fff700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 0x7fc2fc046e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:06.572+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 0x7fc2fc04b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:06.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:06.572+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 msgr2=0x7fc2fc04b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:58:06.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:06.572+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 msgr2=0x7fc2fc04b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:06.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:06.572+0000 7fc303fff700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 msgr2=0x7fc2fc046e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:58:06.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:06.572+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 0x7fc2fc04b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:06.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:06.572+0000 7fc303fff700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 msgr2=0x7fc2fc046e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:06.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:06.572+0000 7fc303fff700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 0x7fc2fc046e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:06.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:06.572+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 0x7fc2fc04b7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:58:06.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:06.572+0000 7fc303fff700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 0x7fc2fc046e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:58:08.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.160+0000 7fc3017fa700 1 -- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 msgr2=0x7fc2fc04b7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:58:08.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.160+0000 7fc3017fa700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc007b60 0x7fc2fc04b7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.160+0000 7fc3017fa700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 msgr2=0x7fc2fc046e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:58:08.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.160+0000 7fc3017fa700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2fc0469c0 0x7fc2fc046e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:08.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.160+0000 7fc3017fa700 1 -- 172.21.15.40:0/36623815 >> v1:172.21.15.40:6789/0 conn(0x7fc2fc0d87a0 legacy=0x7fc2fc0df110 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:58:08.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.160+0000 7fc3017fa700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 0x7fc2f00097c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:08.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.160+0000 7fc3017fa700 1 -- 172.21.15.40:0/36623815 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2fc0d8b80 con 0x7fc2f0003650 2024-09-13T17:58:08.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.160+0000 7fc3017fa700 1 -- 172.21.15.40:0/36623815 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2fc006b50 con 0x7fc2f0007300 2024-09-13T17:58:08.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.160+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 0x7fc2f00097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.160+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 msgr2=0x7fc2f00097c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:58:08.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.160+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 msgr2=0x7fc2f00097c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:08.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.160+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 0x7fc2f00097c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:08.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.160+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 0x7fc2f00097c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:08.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.364+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 0x7fc2f00097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.364+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 msgr2=0x7fc2f00097c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:58:08.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.364+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 msgr2=0x7fc2f00097c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:08.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.364+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 0x7fc2f00097c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:08.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.364+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 0x7fc2f00097c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:58:08.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.768+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 0x7fc2f00097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:08.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.768+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 msgr2=0x7fc2f00097c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:58:08.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.768+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 msgr2=0x7fc2f00097c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:08.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.768+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 0x7fc2f00097c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:08.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:08.768+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 0x7fc2f00097c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:58:09.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:09.572+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 0x7fc2f00097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:09.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:09.572+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 msgr2=0x7fc2f00097c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:58:09.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:09.572+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 msgr2=0x7fc2f00097c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:09.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:09.572+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 0x7fc2f00097c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:09.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:09.572+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 0x7fc2f00097c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:58:11.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.160+0000 7fc3017fa700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 msgr2=0x7fc2f00097c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:58:11.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.160+0000 7fc3017fa700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.40:3300/0 conn(0x7fc2f0007300 0x7fc2f00097c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:11.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.160+0000 7fc3017fa700 1 -- 172.21.15.40:0/36623815 >> v1:172.21.15.43:6789/0 conn(0x7fc2f0003650 legacy=0x7fc2f0005ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:11.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.160+0000 7fc303fff700 1 -- 172.21.15.40:0/36623815 reap_dead start 2024-09-13T17:58:11.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.160+0000 7fc3017fa700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 0x7fc2f00097a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:11.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.160+0000 7fc3017fa700 1 -- 172.21.15.40:0/36623815 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f000af80 con 0x7fc2fc0d87a0 2024-09-13T17:58:11.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.160+0000 7fc3017fa700 1 -- 172.21.15.40:0/36623815 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f00066b0 con 0x7fc2f0006200 2024-09-13T17:58:11.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.160+0000 7fc3017fa700 1 -- 172.21.15.40:0/36623815 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2fc006b50 con 0x7fc2f000a650 2024-09-13T17:58:11.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.160+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 0x7fc2f00097a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.160+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 msgr2=0x7fc2f00097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:58:11.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.160+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 msgr2=0x7fc2f00097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:11.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.160+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 0x7fc2f00097a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:11.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.160+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 0x7fc2f00097a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.364+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 0x7fc2f00097a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.364+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 msgr2=0x7fc2f00097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:58:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.364+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 msgr2=0x7fc2f00097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.364+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 0x7fc2f00097a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:11.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.364+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 0x7fc2f00097a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:58:11.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.768+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 0x7fc2f00097a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:11.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.768+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 msgr2=0x7fc2f00097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:58:11.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.768+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 msgr2=0x7fc2f00097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:11.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.768+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 0x7fc2f00097a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:11.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:11.768+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 0x7fc2f00097a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:58:12.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:12.572+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 0x7fc2f00097a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:12.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:12.572+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 msgr2=0x7fc2f00097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:58:12.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:12.572+0000 7fc302ffd700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 msgr2=0x7fc2f00097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:12.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:12.572+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 0x7fc2f00097a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:12.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:12.572+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 0x7fc2f00097a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:58:14.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.080+0000 7fc3027fc700 1 -- 172.21.15.40:0/36623815 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272635546 0 0) 0x7fc2f00066b0 con 0x7fc2f0006200 2024-09-13T17:58:14.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.080+0000 7fc3027fc700 1 -- 172.21.15.40:0/36623815 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2ec0035d0 con 0x7fc2f0006200 2024-09-13T17:58:14.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3027fc700 1 -- 172.21.15.40:0/36623815 <== mon.1 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1665035248 0 0) 0x7fc2ec0035d0 con 0x7fc2f0006200 2024-09-13T17:58:14.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3027fc700 1 -- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 msgr2=0x7fc2f00097a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:58:14.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3027fc700 1 --2- 172.21.15.40:0/36623815 >> v2:172.21.15.43:3300/0 conn(0x7fc2f000a650 0x7fc2f00097a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3027fc700 1 -- 172.21.15.40:0/36623815 >> v1:172.21.15.40:6789/0 conn(0x7fc2fc0d87a0 legacy=0x7fc2f0009f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3027fc700 1 -- 172.21.15.40:0/36623815 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f0006840 con 0x7fc2f0006200 2024-09-13T17:58:14.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3027fc700 1 -- 172.21.15.40:0/36623815 <== mon.1 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fc30404ec20 con 0x7fc2f0006200 2024-09-13T17:58:14.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3027fc700 1 -- 172.21.15.40:0/36623815 >> v1:172.21.15.43:6789/0 conn(0x7fc2f0006200 legacy=0x7fc2f0005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3027fc700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ec003a20 0x7fc2ec003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3027fc700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2ec004870 0x7fc2ec004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3027fc700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2ec005280 0x7fc2ec00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3027fc700 1 -- 172.21.15.40:0/36623815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc006b50 con 0x7fc2ec003a20 2024-09-13T17:58:14.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3027fc700 1 -- 172.21.15.40:0/36623815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec00c1d0 con 0x7fc2ec004870 2024-09-13T17:58:14.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3027fc700 1 -- 172.21.15.40:0/36623815 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec00c390 con 0x7fc2ec005280 2024-09-13T17:58:14.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3037fe700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ec003a20 0x7fc2ec003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2ec004870 0x7fc2ec004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc303fff700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2ec005280 0x7fc2ec00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3037fe700 1 -- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2ec005280 msgr2=0x7fc2ec00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:14.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3037fe700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2ec005280 0x7fc2ec00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3037fe700 1 -- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2ec004870 msgr2=0x7fc2ec004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:14.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3037fe700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2ec004870 0x7fc2ec004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3037fe700 1 -- 172.21.15.40:0/36623815 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec0035d0 con 0x7fc2ec003a20 2024-09-13T17:58:14.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc303fff700 1 -- 172.21.15.40:0/36623815 reap_dead start 2024-09-13T17:58:14.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc302ffd700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2ec004870 0x7fc2ec004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:14.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3037fe700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ec003a20 0x7fc2ec003e40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8001a90 tx=0x7fc2ec0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ac1bf49e10ed68 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:14.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3027fc700 1 -- 172.21.15.40:0/36623815 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8004b50 con 0x7fc2ec003a20 2024-09-13T17:58:14.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3027fc700 1 -- 172.21.15.40:0/36623815 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f8004cb0 con 0x7fc2ec003a20 2024-09-13T17:58:14.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.088+0000 7fc3027fc700 1 -- 172.21.15.40:0/36623815 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8007390 con 0x7fc2ec003a20 2024-09-13T17:58:14.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.092+0000 7fc308fe9700 1 -- 172.21.15.40:0/36623815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ec003a20 msgr2=0x7fc2ec003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.092+0000 7fc308fe9700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ec003a20 0x7fc2ec003e40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8001a90 tx=0x7fc2ec0058a0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.092+0000 7fc308fe9700 1 -- 172.21.15.40:0/36623815 shutdown_connections 2024-09-13T17:58:14.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.092+0000 7fc308fe9700 1 --2- 172.21.15.40:0/36623815 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ec003a20 0x7fc2ec003e40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.092+0000 7fc308fe9700 1 -- 172.21.15.40:0/36623815 >> 172.21.15.40:0/36623815 conn(0x7fc2fc050e60 msgr2=0x7fc2fc051270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:14.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.092+0000 7fc308fe9700 1 -- 172.21.15.40:0/36623815 shutdown_connections 2024-09-13T17:58:14.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.092+0000 7fc308fe9700 1 -- 172.21.15.40:0/36623815 wait complete. 2024-09-13T17:58:14.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.092+0000 7fc308fe9700 1 Processor -- start 2024-09-13T17:58:14.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.092+0000 7fc308fe9700 1 -- start start 2024-09-13T17:58:14.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc308fe9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ec003a20 0x7fc2fc0d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc308fe9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc0e27f0 0x7fc2fc0d1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc308fe9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2fc04afb0 0x7fc2fc0da830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc308fe9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc175080 con 0x7fc2ec003a20 2024-09-13T17:58:14.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc308fe9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc0d2610 con 0x7fc2fc04afb0 2024-09-13T17:58:14.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc308fe9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc1741f0 con 0x7fc2fc0e27f0 2024-09-13T17:58:14.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc3037fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ec003a20 0x7fc2fc0d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc3037fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ec003a20 0x7fc2fc0d19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59538/0 (socket says 172.21.15.40:59538) 2024-09-13T17:58:14.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc3037fe700 1 -- 172.21.15.40:0/2194461242 learned_addr learned my addr 172.21.15.40:0/2194461242 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:14.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc302ffd700 1 --2- 172.21.15.40:0/2194461242 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc0e27f0 0x7fc2fc0d1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc303fff700 1 --2- 172.21.15.40:0/2194461242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2fc04afb0 0x7fc2fc0da830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc3037fe700 1 -- 172.21.15.40:0/2194461242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ec003a20 msgr2=0x7fc2fc0d19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T17:58:14.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc3037fe700 1 -- 172.21.15.40:0/2194461242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ec003a20 msgr2=0x7fc2fc0d19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:58:14.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc3037fe700 1 --2- 172.21.15.40:0/2194461242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ec003a20 0x7fc2fc0d19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:14.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc3037fe700 1 --2- 172.21.15.40:0/2194461242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ec003a20 0x7fc2fc0d19d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:14.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc303fff700 1 -- 172.21.15.40:0/2194461242 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc0e27f0 msgr2=0x7fc2fc0d1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc303fff700 1 --2- 172.21.15.40:0/2194461242 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc0e27f0 0x7fc2fc0d1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc303fff700 1 -- 172.21.15.40:0/2194461242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ec003a20 msgr2=0x7fc2fc0d19d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:58:14.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc303fff700 1 --2- 172.21.15.40:0/2194461242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ec003a20 0x7fc2fc0d19d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc303fff700 1 -- 172.21.15.40:0/2194461242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc174400 con 0x7fc2fc04afb0 2024-09-13T17:58:14.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc302ffd700 1 --2- 172.21.15.40:0/2194461242 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc0e27f0 0x7fc2fc0d1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:14.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc303fff700 1 --2- 172.21.15.40:0/2194461242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2fc04afb0 0x7fc2fc0da830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc30402ce60 tx=0x7fc3040680b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:14.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc2ebfff700 1 -- 172.21.15.40:0/2194461242 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc304067720 con 0x7fc2fc04afb0 2024-09-13T17:58:14.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc2ebfff700 1 -- 172.21.15.40:0/2194461242 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc304072040 con 0x7fc2fc04afb0 2024-09-13T17:58:14.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc308fe9700 1 -- 172.21.15.40:0/2194461242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2fc1746e0 con 0x7fc2fc04afb0 2024-09-13T17:58:14.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc308fe9700 1 -- 172.21.15.40:0/2194461242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2fc174ca0 con 0x7fc2fc04afb0 2024-09-13T17:58:14.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc2ebfff700 1 -- 172.21.15.40:0/2194461242 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc304074840 con 0x7fc2fc04afb0 2024-09-13T17:58:14.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.096+0000 7fc308fe9700 1 -- 172.21.15.40:0/2194461242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e4005310 con 0x7fc2fc04afb0 2024-09-13T17:58:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.100+0000 7fc2ebfff700 1 -- 172.21.15.40:0/2194461242 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc304095070 con 0x7fc2fc04afb0 2024-09-13T17:58:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.100+0000 7fc2ebfff700 1 --2- 172.21.15.40:0/2194461242 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc2f0067d40 0x7fc2f006a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.100+0000 7fc3037fe700 1 --2- 172.21.15.40:0/2194461242 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc2f0067d40 0x7fc2f006a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.100+0000 7fc2ebfff700 1 -- 172.21.15.40:0/2194461242 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc3040fe440 con 0x7fc2fc04afb0 2024-09-13T17:58:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.100+0000 7fc3037fe700 1 --2- 172.21.15.40:0/2194461242 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc2f0067d40 0x7fc2f006a200 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8001050 tx=0x7fc2f8016040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:14.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.104+0000 7fc2ebfff700 1 -- 172.21.15.40:0/2194461242 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc3040cb530 con 0x7fc2fc04afb0 2024-09-13T17:58:14.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.372+0000 7fc308fe9700 1 -- 172.21.15.40:0/2194461242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc2e40055c0 con 0x7fc2fc04afb0 2024-09-13T17:58:14.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.372+0000 7fc2ebfff700 1 -- 172.21.15.40:0/2194461242 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7fc3040ceb50 con 0x7fc2fc04afb0 2024-09-13T17:58:14.381 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":60,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:58:14.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.376+0000 7fc308fe9700 1 -- 172.21.15.40:0/2194461242 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc2f0067d40 msgr2=0x7fc2f006a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.376+0000 7fc308fe9700 1 --2- 172.21.15.40:0/2194461242 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc2f0067d40 0x7fc2f006a200 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8001050 tx=0x7fc2f8016040 comp rx=0 tx=0).stop 2024-09-13T17:58:14.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.376+0000 7fc308fe9700 1 -- 172.21.15.40:0/2194461242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2fc04afb0 msgr2=0x7fc2fc0da830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:14.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.376+0000 7fc308fe9700 1 --2- 172.21.15.40:0/2194461242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2fc04afb0 0x7fc2fc0da830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc30402ce60 tx=0x7fc3040680b0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.376+0000 7fc308fe9700 1 -- 172.21.15.40:0/2194461242 shutdown_connections 2024-09-13T17:58:14.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.376+0000 7fc308fe9700 1 --2- 172.21.15.40:0/2194461242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2fc04afb0 0x7fc2fc0da830 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.376+0000 7fc308fe9700 1 --2- 172.21.15.40:0/2194461242 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc2f0067d40 0x7fc2f006a200 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.376+0000 7fc308fe9700 1 --2- 172.21.15.40:0/2194461242 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2fc0e27f0 0x7fc2fc0d1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.376+0000 7fc308fe9700 1 --2- 172.21.15.40:0/2194461242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2ec003a20 0x7fc2fc0d19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:14.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.376+0000 7fc308fe9700 1 -- 172.21.15.40:0/2194461242 >> 172.21.15.40:0/2194461242 conn(0x7fc2fc050e60 msgr2=0x7fc2fc0d7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:14.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.376+0000 7fc308fe9700 1 -- 172.21.15.40:0/2194461242 shutdown_connections 2024-09-13T17:58:14.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:14.376+0000 7fc308fe9700 1 -- 172.21.15.40:0/2194461242 wait complete. 2024-09-13T17:58:17.401 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:58:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.512+0000 7f3c6d3c4700 1 Processor -- start 2024-09-13T17:58:17.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.512+0000 7f3c6d3c4700 1 -- start start 2024-09-13T17:58:17.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c6d3c4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c68071e60 0x7f3c680c0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c6d3c4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c68072240 0x7f3c680c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c6d3c4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3c680c28f0 0x7f3c681565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c6d3c4700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c680b2210 con 0x7f3c68071e60 2024-09-13T17:58:17.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c6d3c4700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c680c3820 con 0x7f3c680c28f0 2024-09-13T17:58:17.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c6d3c4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c680c1780 con 0x7f3c68072240 2024-09-13T17:58:17.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c67fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c68071e60 0x7f3c680c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c67fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c68071e60 0x7f3c680c0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59554/0 (socket says 172.21.15.40:59554) 2024-09-13T17:58:17.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c67fff700 1 -- 172.21.15.40:0/1177426069 learned_addr learned my addr 172.21.15.40:0/1177426069 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:17.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c677fe700 1 --2- 172.21.15.40:0/1177426069 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c68072240 0x7f3c680c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c6cbc3700 1 --2- 172.21.15.40:0/1177426069 >> v2:172.21.15.43:3300/0 conn(0x7f3c680c28f0 0x7f3c681565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c67fff700 1 -- 172.21.15.40:0/1177426069 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c68072240 msgr2=0x7f3c680c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:17.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c67fff700 1 --2- 172.21.15.40:0/1177426069 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c68072240 0x7f3c680c32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c67fff700 1 -- 172.21.15.40:0/1177426069 >> v2:172.21.15.43:3300/0 conn(0x7f3c680c28f0 msgr2=0x7f3c681565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:17.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c67fff700 1 --2- 172.21.15.40:0/1177426069 >> v2:172.21.15.43:3300/0 conn(0x7f3c680c28f0 0x7f3c681565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c67fff700 1 -- 172.21.15.40:0/1177426069 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c680c1a60 con 0x7f3c68071e60 2024-09-13T17:58:17.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c67fff700 1 --2- 172.21.15.40:0/1177426069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c68071e60 0x7f3c680c0e90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3c68066eb0 tx=0x7f3c58009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8341c67f9998f189 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:17.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c66ffd700 1 -- 172.21.15.40:0/1177426069 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58011830 con 0x7f3c68071e60 2024-09-13T17:58:17.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c66ffd700 1 -- 172.21.15.40:0/1177426069 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c58011990 con 0x7f3c68071e60 2024-09-13T17:58:17.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c66ffd700 1 -- 172.21.15.40:0/1177426069 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58011d20 con 0x7f3c68071e60 2024-09-13T17:58:17.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c6d3c4700 1 -- 172.21.15.40:0/1177426069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c68071e60 msgr2=0x7f3c680c0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c6d3c4700 1 --2- 172.21.15.40:0/1177426069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c68071e60 0x7f3c680c0e90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3c68066eb0 tx=0x7f3c58009bf0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c6d3c4700 1 -- 172.21.15.40:0/1177426069 shutdown_connections 2024-09-13T17:58:17.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c6d3c4700 1 --2- 172.21.15.40:0/1177426069 >> v2:172.21.15.43:3300/0 conn(0x7f3c680c28f0 0x7f3c681565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c6d3c4700 1 --2- 172.21.15.40:0/1177426069 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c68072240 0x7f3c680c32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c6d3c4700 1 --2- 172.21.15.40:0/1177426069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c68071e60 0x7f3c680c0e90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c6d3c4700 1 -- 172.21.15.40:0/1177426069 >> 172.21.15.40:0/1177426069 conn(0x7f3c6806e4f0 msgr2=0x7f3c6806e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:17.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c6d3c4700 1 -- 172.21.15.40:0/1177426069 shutdown_connections 2024-09-13T17:58:17.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.516+0000 7f3c6d3c4700 1 -- 172.21.15.40:0/1177426069 wait complete. 2024-09-13T17:58:17.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c6d3c4700 1 Processor -- start 2024-09-13T17:58:17.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c6d3c4700 1 -- start start 2024-09-13T17:58:17.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c6d3c4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c68071e60 0x7f3c6814dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c6d3c4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c68072240 0x7f3c681e8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c6d3c4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c680c28f0 0x7f3c681e8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c6d3c4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c681efa30 con 0x7f3c68071e60 2024-09-13T17:58:17.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c6d3c4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c681f21a0 con 0x7f3c680c28f0 2024-09-13T17:58:17.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c6d3c4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c681f2480 con 0x7f3c68072240 2024-09-13T17:58:17.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c67fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c68071e60 0x7f3c6814dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c67fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c68071e60 0x7f3c6814dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59562/0 (socket says 172.21.15.40:59562) 2024-09-13T17:58:17.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c67fff700 1 -- 172.21.15.40:0/3589568753 learned_addr learned my addr 172.21.15.40:0/3589568753 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:17.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c6cbc3700 1 --2- 172.21.15.40:0/3589568753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c680c28f0 0x7f3c681e8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c677fe700 1 --2- 172.21.15.40:0/3589568753 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c68072240 0x7f3c681e8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c67fff700 1 -- 172.21.15.40:0/3589568753 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c68072240 msgr2=0x7f3c681e8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c67fff700 1 --2- 172.21.15.40:0/3589568753 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c68072240 0x7f3c681e8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c67fff700 1 -- 172.21.15.40:0/3589568753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c680c28f0 msgr2=0x7f3c681e8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c67fff700 1 --2- 172.21.15.40:0/3589568753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c680c28f0 0x7f3c681e8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c67fff700 1 -- 172.21.15.40:0/3589568753 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c58009570 con 0x7f3c68071e60 2024-09-13T17:58:17.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c67fff700 1 --2- 172.21.15.40:0/3589568753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c68071e60 0x7f3c6814dcb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3c58014040 tx=0x7f3c58004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:17.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c64ff9700 1 -- 172.21.15.40:0/3589568753 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58004450 con 0x7f3c68071e60 2024-09-13T17:58:17.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c64ff9700 1 -- 172.21.15.40:0/3589568753 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5801e070 con 0x7f3c68071e60 2024-09-13T17:58:17.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c6d3c4700 1 -- 172.21.15.40:0/3589568753 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c681f2760 con 0x7f3c68071e60 2024-09-13T17:58:17.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c64ff9700 1 -- 172.21.15.40:0/3589568753 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58003900 con 0x7f3c68071e60 2024-09-13T17:58:17.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.520+0000 7f3c6d3c4700 1 -- 172.21.15.40:0/3589568753 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c681f2d20 con 0x7f3c68071e60 2024-09-13T17:58:17.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.524+0000 7f3c6d3c4700 1 -- 172.21.15.40:0/3589568753 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c681f2f10 con 0x7f3c68071e60 2024-09-13T17:58:17.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.524+0000 7f3c64ff9700 1 -- 172.21.15.40:0/3589568753 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3c58003a60 con 0x7f3c68071e60 2024-09-13T17:58:17.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.524+0000 7f3c64ff9700 1 --2- 172.21.15.40:0/3589568753 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3c54066760 0x7f3c54068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.524+0000 7f3c677fe700 1 --2- 172.21.15.40:0/3589568753 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3c54066760 0x7f3c54068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.524+0000 7f3c64ff9700 1 -- 172.21.15.40:0/3589568753 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3c58084fb0 con 0x7f3c68071e60 2024-09-13T17:58:17.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.524+0000 7f3c677fe700 1 --2- 172.21.15.40:0/3589568753 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3c54066760 0x7f3c54068c20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3c681f3120 tx=0x7f3c5c009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:17.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.532+0000 7f3c64ff9700 1 -- 172.21.15.40:0/3589568753 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3c58051ff0 con 0x7f3c68071e60 2024-09-13T17:58:17.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.796+0000 7f3c6d3c4700 1 -- 172.21.15.40:0/3589568753 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3c680c2da0 con 0x7f3c68071e60 2024-09-13T17:58:17.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.796+0000 7f3c64ff9700 1 -- 172.21.15.40:0/3589568753 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7f3c58020070 con 0x7f3c68071e60 2024-09-13T17:58:17.805 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":62,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:58:17.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.800+0000 7f3c6d3c4700 1 -- 172.21.15.40:0/3589568753 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3c54066760 msgr2=0x7f3c54068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.800+0000 7f3c6d3c4700 1 --2- 172.21.15.40:0/3589568753 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3c54066760 0x7f3c54068c20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3c681f3120 tx=0x7f3c5c009240 comp rx=0 tx=0).stop 2024-09-13T17:58:17.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.800+0000 7f3c6d3c4700 1 -- 172.21.15.40:0/3589568753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c68071e60 msgr2=0x7f3c6814dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.800+0000 7f3c6d3c4700 1 --2- 172.21.15.40:0/3589568753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c68071e60 0x7f3c6814dcb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3c58014040 tx=0x7f3c58004770 comp rx=0 tx=0).stop 2024-09-13T17:58:17.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.800+0000 7f3c6d3c4700 1 -- 172.21.15.40:0/3589568753 shutdown_connections 2024-09-13T17:58:17.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.800+0000 7f3c6d3c4700 1 --2- 172.21.15.40:0/3589568753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c680c28f0 0x7f3c681e8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.800+0000 7f3c6d3c4700 1 --2- 172.21.15.40:0/3589568753 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3c54066760 0x7f3c54068c20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.800+0000 7f3c6d3c4700 1 --2- 172.21.15.40:0/3589568753 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c68072240 0x7f3c681e8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.800+0000 7f3c6d3c4700 1 --2- 172.21.15.40:0/3589568753 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c68071e60 0x7f3c6814dcb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.800+0000 7f3c6d3c4700 1 -- 172.21.15.40:0/3589568753 >> 172.21.15.40:0/3589568753 conn(0x7f3c6806e4f0 msgr2=0x7f3c680cb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:17.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.800+0000 7f3c6d3c4700 1 -- 172.21.15.40:0/3589568753 shutdown_connections 2024-09-13T17:58:17.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.800+0000 7f3c6d3c4700 1 -- 172.21.15.40:0/3589568753 wait complete. 2024-09-13T17:58:17.824 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T17:58:17.824 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T17:58:17.824 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:58:17.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.936+0000 7f3976578700 1 Processor -- start 2024-09-13T17:58:17.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.936+0000 7f3976578700 1 -- start start 2024-09-13T17:58:17.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.936+0000 7f3976578700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39700b1c60 0x7f39700c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.936+0000 7f3976578700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3970070ae0 con 0x7f397015d8a0 2024-09-13T17:58:17.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.936+0000 7f3976578700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3970070d80 con 0x7f39701577d0 2024-09-13T17:58:17.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.936+0000 7f3976578700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3970071020 con 0x7f39700b1c60 2024-09-13T17:58:17.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.936+0000 7f3974d75700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f397015d8a0 0x7f39700c8630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50364/0 (socket says 172.21.15.40:50364) 2024-09-13T17:58:17.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.936+0000 7f3974d75700 1 -- 172.21.15.40:0/2280179096 learned_addr learned my addr 172.21.15.40:0/2280179096 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:58:17.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.936+0000 7f3975d77700 1 --2- 172.21.15.40:0/2280179096 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39700b1c60 0x7f39700c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.936+0000 7f3975d77700 1 -- 172.21.15.40:0/2280179096 >> v1:172.21.15.43:6789/0 conn(0x7f39701577d0 legacy=0x7f39701616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.936+0000 7f3975d77700 1 -- 172.21.15.40:0/2280179096 >> v1:172.21.15.40:6789/0 conn(0x7f397015d8a0 legacy=0x7f39700c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.936+0000 7f3975d77700 1 -- 172.21.15.40:0/2280179096 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39700712c0 con 0x7f39700b1c60 2024-09-13T17:58:17.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.936+0000 7f3975d77700 1 --2- 172.21.15.40:0/2280179096 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39700b1c60 0x7f39700c7fb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3960009880 tx=0x7f3960009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d5c03885269a7ca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:17.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.940+0000 7f3967fff700 1 -- 172.21.15.40:0/2280179096 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3960011830 con 0x7f39700b1c60 2024-09-13T17:58:17.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.940+0000 7f3967fff700 1 -- 172.21.15.40:0/2280179096 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3960011990 con 0x7f39700b1c60 2024-09-13T17:58:17.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.940+0000 7f3967fff700 1 -- 172.21.15.40:0/2280179096 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3960011d20 con 0x7f39700b1c60 2024-09-13T17:58:17.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.940+0000 7f3976578700 1 -- 172.21.15.40:0/2280179096 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39700b1c60 msgr2=0x7f39700c7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.940+0000 7f3976578700 1 --2- 172.21.15.40:0/2280179096 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39700b1c60 0x7f39700c7fb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3960009880 tx=0x7f3960009b90 comp rx=0 tx=0).stop 2024-09-13T17:58:17.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.940+0000 7f3976578700 1 -- 172.21.15.40:0/2280179096 shutdown_connections 2024-09-13T17:58:17.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.940+0000 7f3976578700 1 --2- 172.21.15.40:0/2280179096 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39700b1c60 0x7f39700c7fb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.940+0000 7f3976578700 1 -- 172.21.15.40:0/2280179096 >> 172.21.15.40:0/2280179096 conn(0x7f397006e6d0 msgr2=0x7f397006eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:17.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.940+0000 7f3976578700 1 -- 172.21.15.40:0/2280179096 shutdown_connections 2024-09-13T17:58:17.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.940+0000 7f3976578700 1 -- 172.21.15.40:0/2280179096 wait complete. 2024-09-13T17:58:17.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3976578700 1 Processor -- start 2024-09-13T17:58:17.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3976578700 1 -- start start 2024-09-13T17:58:17.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3976578700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39700b1c60 0x7f39701f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3976578700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39701577d0 0x7f39700cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3976578700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f397015d8a0 0x7f39701f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3976578700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39701f36e0 con 0x7f39700b1c60 2024-09-13T17:58:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3976578700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397015ff30 con 0x7f397015d8a0 2024-09-13T17:58:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3976578700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39701f7720 con 0x7f39701577d0 2024-09-13T17:58:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3975576700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39700b1c60 0x7f39701f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3975576700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39700b1c60 0x7f39701f71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59566/0 (socket says 172.21.15.40:59566) 2024-09-13T17:58:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3975576700 1 -- 172.21.15.40:0/2542193423 learned_addr learned my addr 172.21.15.40:0/2542193423 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3975d77700 1 --2- 172.21.15.40:0/2542193423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f397015d8a0 0x7f39701f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3974d75700 1 --2- 172.21.15.40:0/2542193423 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39701577d0 0x7f39700cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3975576700 1 -- 172.21.15.40:0/2542193423 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39701577d0 msgr2=0x7f39700cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3975576700 1 --2- 172.21.15.40:0/2542193423 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39701577d0 0x7f39700cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3975576700 1 -- 172.21.15.40:0/2542193423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f397015d8a0 msgr2=0x7f39701f3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3975576700 1 --2- 172.21.15.40:0/2542193423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f397015d8a0 0x7f39701f3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3975576700 1 -- 172.21.15.40:0/2542193423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39600094b0 con 0x7f39700b1c60 2024-09-13T17:58:17.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3975576700 1 --2- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39700b1c60 0x7f39701f71e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3968001ed0 tx=0x7f39680021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:17.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3965ffb700 1 -- 172.21.15.40:0/2542193423 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3968003c20 con 0x7f39700b1c60 2024-09-13T17:58:17.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3965ffb700 1 -- 172.21.15.40:0/2542193423 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3968003d80 con 0x7f39700b1c60 2024-09-13T17:58:17.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3976578700 1 -- 172.21.15.40:0/2542193423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39701f3370 con 0x7f39700b1c60 2024-09-13T17:58:17.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3965ffb700 1 -- 172.21.15.40:0/2542193423 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396800aae0 con 0x7f39700b1c60 2024-09-13T17:58:17.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3976578700 1 -- 172.21.15.40:0/2542193423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3970160160 con 0x7f39700b1c60 2024-09-13T17:58:17.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3976578700 1 --2- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3970150470 0x7f3970160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3976578700 1 -- 172.21.15.40:0/2542193423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39700675e0 con 0x7f3970150470 2024-09-13T17:58:17.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.944+0000 7f3974d75700 1 --2- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3970150470 0x7f3970160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.948+0000 7f3974d75700 1 -- 172.21.15.40:0/2542193423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3970160eb0 con 0x7f3970150470 2024-09-13T17:58:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.948+0000 7f3965ffb700 1 -- 172.21.15.40:0/2542193423 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f396800ac40 con 0x7f39700b1c60 2024-09-13T17:58:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.948+0000 7f3974d75700 1 --2- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3970150470 0x7f3970160700 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3970153f30 tx=0x7f396c00fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.948+0000 7f3965ffb700 1 --2- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3954066730 0x7f3954068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.948+0000 7f3965ffb700 1 -- 172.21.15.40:0/2542193423 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3968084560 con 0x7f39700b1c60 2024-09-13T17:58:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.948+0000 7f3975d77700 1 --2- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3954066730 0x7f3954068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.948+0000 7f3965ffb700 1 -- 172.21.15.40:0/2542193423 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c005b70 con 0x7f3970150470 2024-09-13T17:58:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.948+0000 7f3965ffb700 1 -- 172.21.15.40:0/2542193423 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f396c005cd0 con 0x7f3970150470 2024-09-13T17:58:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.948+0000 7f3965ffb700 1 -- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3970150470 msgr2=0x7f3970160700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.948+0000 7f3965ffb700 1 --2- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3970150470 0x7f3970160700 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3970153f30 tx=0x7f396c00fb00 comp rx=0 tx=0).stop 2024-09-13T17:58:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.948+0000 7f3965ffb700 1 -- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3970150470 msgr2=0x7f3970160700 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:58:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.948+0000 7f3965ffb700 1 --2- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3970150470 0x7f3970160700 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.948+0000 7f3975d77700 1 --2- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3954066730 0x7f3954068bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f39600090b0 tx=0x7f3960008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:17.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3976578700 1 --2- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39700c36e0 0x7f3970071cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:17.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3976578700 1 -- 172.21.15.40:0/2542193423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3970071840 con 0x7f39700c36e0 2024-09-13T17:58:17.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3974d75700 1 --2- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39700c36e0 0x7f3970071cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:17.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3974d75700 1 -- 172.21.15.40:0/2542193423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3970161190 con 0x7f39700c36e0 2024-09-13T17:58:17.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3974d75700 1 --2- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39700c36e0 0x7f3970071cb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3970067720 tx=0x7f396c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:17.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3965ffb700 1 -- 172.21.15.40:0/2542193423 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c023070 con 0x7f39700c36e0 2024-09-13T17:58:17.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3965ffb700 1 -- 172.21.15.40:0/2542193423 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4451 (secure 0 0 0) 0x7f396c005e40 con 0x7f39700c36e0 2024-09-13T17:58:17.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3965ffb700 1 -- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39700c36e0 msgr2=0x7f3970071cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3965ffb700 1 --2- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39700c36e0 0x7f3970071cb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3970067720 tx=0x7f396c0058a0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3965ffb700 1 -- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39700c36e0 msgr2=0x7f3970071cb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:58:17.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3965ffb700 1 --2- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39700c36e0 0x7f3970071cb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.965 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:58:17.965 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:58:17.965 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:58:17.965 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:58:17.965 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 62, 2024-09-13T17:58:17.965 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:58:17.966 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:58:17.966 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:58:17.966 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:58:17.966 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:17.966 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T17:58:17.966 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:58:17.966 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:58:17.966 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:58:17.966 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:17.966 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:17.966 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:17.966 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:17.966 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:17.966 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:17.967 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:17.967 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:17.967 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:17.967 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:17.967 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:17.967 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:58:17.967 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:58:17.967 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:17.967 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:17.967 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:17.967 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:17.967 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:17.967 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:17.967 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:17.968 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:17.968 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:17.968 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:17.968 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:17.968 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:17.968 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:58:17.968 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:58:17.968 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:58:17.968 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:58:17.968 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:58:17.968 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:58:17.968 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:58:17.968 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:58:17.969 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:58:17.969 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:58:17.969 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:58:17.969 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:58:17.969 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:58:17.969 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:58:17.969 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:58:17.969 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:58:17.969 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:58:17.969 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:17.969 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:17.969 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:17.969 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:17.970 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:17.970 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:17.970 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:17.970 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:17.970 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:17.970 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:17.970 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:17.970 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:58:17.970 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:17.970 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:58:17.970 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:17.970 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:58:17.970 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:58:17.971 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:17.971 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:17.971 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:17.971 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:17.971 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:58:17.971 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:17.971 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:17.971 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:17.971 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:17.971 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:58:17.971 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:17.971 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:17.971 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:17.971 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:17.972 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:58:17.972 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:58:17.972 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:17.972 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:17.972 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:17.972 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:17.972 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:17.972 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:58:17.972 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:58:17.972 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:17.972 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:17.972 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:17.972 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:17.972 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:58:17.973 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:17.973 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:17.973 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:17.973 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:17.973 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:58:17.973 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:17.973 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:17.973 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:17.973 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:17.973 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:58:17.973 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:58:17.973 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:17.973 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:17.973 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:17.973 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:17.974 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:17.974 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:58:17.974 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:58:17.974 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:17.974 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:17.974 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:17.974 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:17.974 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:58:17.974 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:17.974 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:17.974 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:17.974 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:17.975 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:58:17.975 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:17.975 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:17.975 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:17.975 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:17.975 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:58:17.975 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:58:17.975 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:17.975 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:17.975 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:17.975 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:17.975 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:17.975 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:17.975 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:58:17.976 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:58:17.976 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:17.976 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:17.976 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:17.976 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:17.976 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:17.976 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:17.976 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:58:17.976 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:17.976 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:17.976 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:17.976 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:58:17.977 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:17.977 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:17.977 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:17.977 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:58:17.977 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:58:17.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3976578700 1 -- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3954066730 msgr2=0x7f3954068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3976578700 1 --2- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3954066730 0x7f3954068bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f39600090b0 tx=0x7f3960008030 comp rx=0 tx=0).stop 2024-09-13T17:58:17.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3976578700 1 -- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39700b1c60 msgr2=0x7f39701f71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:17.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3976578700 1 --2- 172.21.15.40:0/2542193423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39700b1c60 0x7f39701f71e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3968001ed0 tx=0x7f39680021e0 comp rx=0 tx=0).stop 2024-09-13T17:58:17.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3975d77700 1 -- 172.21.15.40:0/2542193423 reap_dead start 2024-09-13T17:58:17.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3975d77700 1 -- 172.21.15.40:0/2542193423 reap_dead start 2024-09-13T17:58:17.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3976578700 1 -- 172.21.15.40:0/2542193423 shutdown_connections 2024-09-13T17:58:17.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3976578700 1 -- 172.21.15.40:0/2542193423 >> 172.21.15.40:0/2542193423 conn(0x7f397006e6d0 msgr2=0x7f39700c6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:17.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3976578700 1 -- 172.21.15.40:0/2542193423 shutdown_connections 2024-09-13T17:58:17.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:17.956+0000 7f3976578700 1 -- 172.21.15.40:0/2542193423 wait complete. 2024-09-13T17:58:17.979 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:58:17.979 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T17:58:17.979 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:58:18.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.084+0000 7f43e5d68700 1 Processor -- start 2024-09-13T17:58:18.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.084+0000 7f43e5d68700 1 -- start start 2024-09-13T17:58:18.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e5d68700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f43e00cf1a0 0x7f43e015a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e5d68700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43e00cf580 0x7f43e00c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e5d68700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e015a7a0 con 0x7f43e00cc250 2024-09-13T17:58:18.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e5d68700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e00cbdc0 con 0x7f43e00cf1a0 2024-09-13T17:58:18.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e5d68700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e0153f80 con 0x7f43e00cf580 2024-09-13T17:58:18.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e5567700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f43e00cc250 0x7f43e00c8aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50370/0 (socket says 172.21.15.40:50370) 2024-09-13T17:58:18.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e5567700 1 -- 172.21.15.40:0/2848796007 learned_addr learned my addr 172.21.15.40:0/2848796007 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:58:18.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e4d66700 1 --2- 172.21.15.40:0/2848796007 >> v2:172.21.15.43:3300/0 conn(0x7f43e00cf1a0 0x7f43e015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43dffff700 1 --2- 172.21.15.40:0/2848796007 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43e00cf580 0x7f43e00c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43df7fe700 1 -- 172.21.15.40:0/2848796007 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457392515 0 0) 0x7f43d8001490 con 0x7f43e00cc250 2024-09-13T17:58:18.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43df7fe700 1 -- 172.21.15.40:0/2848796007 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43d00035d0 con 0x7f43e00cc250 2024-09-13T17:58:18.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e4d66700 1 -- 172.21.15.40:0/2848796007 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43e00cf580 msgr2=0x7f43e00c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:18.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e4d66700 1 --2- 172.21.15.40:0/2848796007 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43e00cf580 0x7f43e00c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e4d66700 1 -- 172.21.15.40:0/2848796007 >> v1:172.21.15.40:6789/0 conn(0x7f43e00cc250 legacy=0x7f43e00c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e4d66700 1 -- 172.21.15.40:0/2848796007 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e00caed0 con 0x7f43e00cf1a0 2024-09-13T17:58:18.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e4d66700 1 --2- 172.21.15.40:0/2848796007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e00cf1a0 0x7f43e015a260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009850 tx=0x7f43d4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cea134aa38b612e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43df7fe700 1 -- 172.21.15.40:0/2848796007 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d4011830 con 0x7f43e00cf1a0 2024-09-13T17:58:18.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43df7fe700 1 -- 172.21.15.40:0/2848796007 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d4011990 con 0x7f43e00cf1a0 2024-09-13T17:58:18.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43df7fe700 1 -- 172.21.15.40:0/2848796007 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d4011d00 con 0x7f43e00cf1a0 2024-09-13T17:58:18.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e5d68700 1 -- 172.21.15.40:0/2848796007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e00cf1a0 msgr2=0x7f43e015a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e5d68700 1 --2- 172.21.15.40:0/2848796007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e00cf1a0 0x7f43e015a260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009850 tx=0x7f43d4009b60 comp rx=0 tx=0).stop 2024-09-13T17:58:18.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e5d68700 1 -- 172.21.15.40:0/2848796007 shutdown_connections 2024-09-13T17:58:18.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e5d68700 1 --2- 172.21.15.40:0/2848796007 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43e00cf580 0x7f43e00c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e5d68700 1 --2- 172.21.15.40:0/2848796007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e00cf1a0 0x7f43e015a260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e5d68700 1 -- 172.21.15.40:0/2848796007 >> 172.21.15.40:0/2848796007 conn(0x7f43e0066090 msgr2=0x7f43e0066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:18.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e5d68700 1 -- 172.21.15.40:0/2848796007 shutdown_connections 2024-09-13T17:58:18.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.088+0000 7f43e5d68700 1 -- 172.21.15.40:0/2848796007 wait complete. 2024-09-13T17:58:18.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43e5d68700 1 Processor -- start 2024-09-13T17:58:18.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43e5d68700 1 -- start start 2024-09-13T17:58:18.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43e5d68700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e00cc250 0x7f43e01594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43e5d68700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e00cf1a0 0x7f43e00b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43e5d68700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43e00cf580 0x7f43e01f2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43e5d68700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e01f2080 con 0x7f43e00cf1a0 2024-09-13T17:58:18.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43e5d68700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e01599f0 con 0x7f43e00cc250 2024-09-13T17:58:18.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43e5d68700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e01510b0 con 0x7f43e00cf580 2024-09-13T17:58:18.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43dffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e00cf1a0 0x7f43e00b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43dffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e00cf1a0 0x7f43e00b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59600/0 (socket says 172.21.15.40:59600) 2024-09-13T17:58:18.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43dffff700 1 -- 172.21.15.40:0/913755562 learned_addr learned my addr 172.21.15.40:0/913755562 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:18.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43e4d66700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e00cc250 0x7f43e01594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43e5567700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43e00cf580 0x7f43e01f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43dffff700 1 -- 172.21.15.40:0/913755562 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43e00cf580 msgr2=0x7f43e01f2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43dffff700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43e00cf580 0x7f43e01f2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43dffff700 1 -- 172.21.15.40:0/913755562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e00cc250 msgr2=0x7f43e01594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43dffff700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e00cc250 0x7f43e01594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43dffff700 1 -- 172.21.15.40:0/913755562 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d40094b0 con 0x7f43e00cf1a0 2024-09-13T17:58:18.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43dffff700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e00cf1a0 0x7f43e00b1d30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009fd0 tx=0x7f43cc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43dd7fa700 1 -- 172.21.15.40:0/913755562 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0095a0 con 0x7f43e00cf1a0 2024-09-13T17:58:18.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43dd7fa700 1 -- 172.21.15.40:0/913755562 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43cc013680 con 0x7f43e00cf1a0 2024-09-13T17:58:18.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43e5d68700 1 -- 172.21.15.40:0/913755562 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e0151390 con 0x7f43e00cf1a0 2024-09-13T17:58:18.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43dd7fa700 1 -- 172.21.15.40:0/913755562 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc013a30 con 0x7f43e00cf1a0 2024-09-13T17:58:18.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.092+0000 7f43e5d68700 1 -- 172.21.15.40:0/913755562 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e0151950 con 0x7f43e00cf1a0 2024-09-13T17:58:18.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.096+0000 7f43e5d68700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c0002a40 0x7f43c0004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.096+0000 7f43e5d68700 1 -- 172.21.15.40:0/913755562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c0007f40 con 0x7f43c0002a40 2024-09-13T17:58:18.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.096+0000 7f43dd7fa700 1 -- 172.21.15.40:0/913755562 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f43cc013b90 con 0x7f43e00cf1a0 2024-09-13T17:58:18.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.096+0000 7f43e4d66700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c0002a40 0x7f43c0004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.096+0000 7f43dd7fa700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f43c4066760 0x7f43c4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.096+0000 7f43e4d66700 1 -- 172.21.15.40:0/913755562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f43c0008100 con 0x7f43c0002a40 2024-09-13T17:58:18.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.096+0000 7f43dd7fa700 1 -- 172.21.15.40:0/913755562 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f43cc0842f0 con 0x7f43e00cf1a0 2024-09-13T17:58:18.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.096+0000 7f43e5567700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f43c4066760 0x7f43c4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.096+0000 7f43e4d66700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c0002a40 0x7f43c0004f00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009850 tx=0x7f43d4004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.096+0000 7f43e5567700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f43c4066760 0x7f43c4068c20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f43d80015f0 tx=0x7f43d8001a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.096+0000 7f43dd7fa700 1 -- 172.21.15.40:0/913755562 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d4020070 con 0x7f43c0002a40 2024-09-13T17:58:18.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.096+0000 7f43dd7fa700 1 -- 172.21.15.40:0/913755562 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f43d40040f0 con 0x7f43c0002a40 2024-09-13T17:58:18.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.096+0000 7f43dd7fa700 1 -- 172.21.15.40:0/913755562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c0002a40 msgr2=0x7f43c0004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.096+0000 7f43dd7fa700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c0002a40 0x7f43c0004f00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009850 tx=0x7f43d4004770 comp rx=0 tx=0).stop 2024-09-13T17:58:18.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.096+0000 7f43dd7fa700 1 -- 172.21.15.40:0/913755562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c0002a40 msgr2=0x7f43c0004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:58:18.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.096+0000 7f43dd7fa700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c0002a40 0x7f43c0004f00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.108+0000 7f43e5d68700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c0006ee0 0x7f43c0007350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.108+0000 7f43e5d68700 1 -- 172.21.15.40:0/913755562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c000a4a0 con 0x7f43c0006ee0 2024-09-13T17:58:18.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.108+0000 7f43e4d66700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c0006ee0 0x7f43c0007350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.112+0000 7f43e4d66700 1 -- 172.21.15.40:0/913755562 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f43c0007890 con 0x7f43c0006ee0 2024-09-13T17:58:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.112+0000 7f43e4d66700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c0006ee0 0x7f43c0007350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009ba0 tx=0x7f43d4011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.112+0000 7f43dd7fa700 1 -- 172.21.15.40:0/913755562 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d402c940 con 0x7f43c0006ee0 2024-09-13T17:58:18.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.112+0000 7f43dd7fa700 1 -- 172.21.15.40:0/913755562 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4947 (secure 0 0 0) 0x7f43d401b430 con 0x7f43c0006ee0 2024-09-13T17:58:18.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.112+0000 7f43dd7fa700 1 -- 172.21.15.40:0/913755562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c0006ee0 msgr2=0x7f43c0007350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.112+0000 7f43dd7fa700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c0006ee0 0x7f43c0007350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009ba0 tx=0x7f43d4011000 comp rx=0 tx=0).stop 2024-09-13T17:58:18.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.112+0000 7f43dd7fa700 1 -- 172.21.15.40:0/913755562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c0006ee0 msgr2=0x7f43c0007350 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:58:18.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.112+0000 7f43dd7fa700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43c0006ee0 0x7f43c0007350 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009ba0 tx=0x7f43d4011000 comp rx=0 tx=0).stop 2024-09-13T17:58:18.122 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:58:18.122 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:58:18.122 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:58:18.122 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:58:18.122 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 62, 2024-09-13T17:58:18.122 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:58:18.122 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:58:18.123 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:58:18.123 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:58:18.123 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:18.123 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T17:58:18.123 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:58:18.123 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:58:18.123 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:58:18.123 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:18.124 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:18.124 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:18.124 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:18.124 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:18.124 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:18.124 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:18.124 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:18.124 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:18.124 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:18.124 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:18.124 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:58:18.124 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:58:18.124 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:18.125 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:18.125 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:18.125 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:18.125 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:18.125 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:18.125 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:18.125 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:18.125 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:18.125 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:18.125 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:18.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.125 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:58:18.125 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:58:18.125 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:58:18.126 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:58:18.126 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:58:18.126 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:58:18.126 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:58:18.126 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:58:18.126 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:58:18.126 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:58:18.126 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:58:18.126 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:58:18.126 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:58:18.126 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:58:18.126 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:58:18.126 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:58:18.126 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:58:18.127 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:18.127 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:18.127 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:18.127 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:18.127 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:18.127 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:18.127 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:18.127 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:18.127 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:18.127 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:18.127 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:18.127 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:58:18.127 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.128 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:58:18.128 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.128 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:58:18.128 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:58:18.128 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:18.128 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:18.128 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.128 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:18.128 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:58:18.128 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:18.128 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.129 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.129 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:18.129 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:58:18.129 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:18.129 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:18.129 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:18.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.129 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:58:18.129 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:58:18.129 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:18.129 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:18.129 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:18.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:58:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:58:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:58:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:58:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:18.130 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:18.131 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:18.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.131 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:58:18.131 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:58:18.131 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:18.131 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:18.131 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:18.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.131 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:58:18.131 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:58:18.131 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:18.131 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:18.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.132 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:18.132 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:58:18.132 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:18.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.132 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:18.132 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:58:18.132 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:18.132 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:18.132 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:18.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.132 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:58:18.133 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:58:18.133 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:18.133 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:18.133 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:18.133 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:18.133 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:18.133 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.133 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:58:18.133 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:58:18.133 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.133 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:18.133 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:18.134 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:18.134 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:18.134 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:18.134 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:58:18.134 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.134 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:18.134 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:18.134 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:18.134 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:18.134 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:18.134 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:58:18.134 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.134 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:18.134 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:18.134 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:18.135 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:18.135 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:18.135 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:58:18.135 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.135 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:18.135 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:18.135 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:18.135 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:18.135 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:18.135 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:58:18.135 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.135 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:18.135 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:18.135 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:18.136 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:18.136 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:18.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.136 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:58:18.136 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:58:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.112+0000 7f43e5d68700 1 -- 172.21.15.40:0/913755562 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f43c4066760 msgr2=0x7f43c4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.112+0000 7f43e5d68700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f43c4066760 0x7f43c4068c20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f43d80015f0 tx=0x7f43d8001a20 comp rx=0 tx=0).stop 2024-09-13T17:58:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.112+0000 7f43e5567700 1 -- 172.21.15.40:0/913755562 reap_dead start 2024-09-13T17:58:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.112+0000 7f43e5d68700 1 -- 172.21.15.40:0/913755562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e00cf1a0 msgr2=0x7f43e00b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.112+0000 7f43e5d68700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e00cf1a0 0x7f43e00b1d30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009fd0 tx=0x7f43cc00eb20 comp rx=0 tx=0).stop 2024-09-13T17:58:18.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.112+0000 7f43e5d68700 1 -- 172.21.15.40:0/913755562 shutdown_connections 2024-09-13T17:58:18.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.112+0000 7f43e5d68700 1 --2- 172.21.15.40:0/913755562 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43e00cf1a0 0x7f43e00b1d30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.112+0000 7f43e5d68700 1 -- 172.21.15.40:0/913755562 >> 172.21.15.40:0/913755562 conn(0x7f43e0066090 msgr2=0x7f43e00c9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:18.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.112+0000 7f43e5d68700 1 -- 172.21.15.40:0/913755562 shutdown_connections 2024-09-13T17:58:18.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.112+0000 7f43e5d68700 1 -- 172.21.15.40:0/913755562 wait complete. 2024-09-13T17:58:18.138 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:58:18.138 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T17:58:18.138 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:58:18.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.256+0000 7f0e3d968700 1 Processor -- start 2024-09-13T17:58:18.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.256+0000 7f0e3d968700 1 -- start start 2024-09-13T17:58:18.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e3d968700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e3815a260 0x7f0e380c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e3d968700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e380cbdc0 con 0x7f0e380cf580 2024-09-13T17:58:18.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e3d968700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e38153f80 con 0x7f0e380cf1a0 2024-09-13T17:58:18.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e3d968700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e380caed0 con 0x7f0e3815a260 2024-09-13T17:58:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e37fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0e380cf580 0x7f0e380cc250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50386/0 (socket says 172.21.15.40:50386) 2024-09-13T17:58:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e37fff700 1 -- 172.21.15.40:0/693935056 learned_addr learned my addr 172.21.15.40:0/693935056 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:58:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e3d167700 1 --2- 172.21.15.40:0/693935056 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e3815a260 0x7f0e380c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e377fe700 1 -- 172.21.15.40:0/693935056 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725235190 0 0) 0x7f0e380cbdc0 con 0x7f0e380cf580 2024-09-13T17:58:18.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e377fe700 1 -- 172.21.15.40:0/693935056 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e280035d0 con 0x7f0e380cf580 2024-09-13T17:58:18.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e377fe700 1 -- 172.21.15.40:0/693935056 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 220846621 0 0) 0x7f0e280035d0 con 0x7f0e380cf580 2024-09-13T17:58:18.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e377fe700 1 -- 172.21.15.40:0/693935056 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e3815a260 msgr2=0x7f0e380c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:18.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e377fe700 1 --2- 172.21.15.40:0/693935056 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e3815a260 0x7f0e380c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e377fe700 1 -- 172.21.15.40:0/693935056 >> v1:172.21.15.43:6789/0 conn(0x7f0e380cf1a0 legacy=0x7f0e380c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e377fe700 1 -- 172.21.15.40:0/693935056 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e380cb1b0 con 0x7f0e380cf580 2024-09-13T17:58:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e377fe700 1 -- 172.21.15.40:0/693935056 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f0e24001670 con 0x7f0e380cf580 2024-09-13T17:58:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e377fe700 1 -- 172.21.15.40:0/693935056 >> v1:172.21.15.40:6789/0 conn(0x7f0e380cf580 legacy=0x7f0e380cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e377fe700 1 --2- 172.21.15.40:0/693935056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e28003a50 0x7f0e28003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e377fe700 1 --2- 172.21.15.40:0/693935056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e28004400 0x7f0e280048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e377fe700 1 --2- 172.21.15.40:0/693935056 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e3815a260 0x7f0e28005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e377fe700 1 -- 172.21.15.40:0/693935056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e380caed0 con 0x7f0e28004400 2024-09-13T17:58:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e377fe700 1 -- 172.21.15.40:0/693935056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2800ba30 con 0x7f0e28003a50 2024-09-13T17:58:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e377fe700 1 -- 172.21.15.40:0/693935056 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2800bbf0 con 0x7f0e3815a260 2024-09-13T17:58:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e37fff700 1 --2- 172.21.15.40:0/693935056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e28004400 0x7f0e280048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e3d167700 1 --2- 172.21.15.40:0/693935056 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e3815a260 0x7f0e28005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e3c966700 1 --2- 172.21.15.40:0/693935056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e28003a50 0x7f0e28003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e37fff700 1 -- 172.21.15.40:0/693935056 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e3815a260 msgr2=0x7f0e28005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:18.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e37fff700 1 --2- 172.21.15.40:0/693935056 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e3815a260 0x7f0e28005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e37fff700 1 -- 172.21.15.40:0/693935056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e28003a50 msgr2=0x7f0e28003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:18.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e37fff700 1 --2- 172.21.15.40:0/693935056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e28003a50 0x7f0e28003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e37fff700 1 -- 172.21.15.40:0/693935056 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e280035d0 con 0x7f0e28004400 2024-09-13T17:58:18.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e37fff700 1 --2- 172.21.15.40:0/693935056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e28004400 0x7f0e280048d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0e380cf990 tx=0x7f0e280058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c287b833aea182d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e377fe700 1 -- 172.21.15.40:0/693935056 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2400ce00 con 0x7f0e28004400 2024-09-13T17:58:18.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e377fe700 1 -- 172.21.15.40:0/693935056 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e24005980 con 0x7f0e28004400 2024-09-13T17:58:18.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.260+0000 7f0e377fe700 1 -- 172.21.15.40:0/693935056 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24005d30 con 0x7f0e28004400 2024-09-13T17:58:18.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.264+0000 7f0e3d968700 1 -- 172.21.15.40:0/693935056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e28004400 msgr2=0x7f0e280048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.264+0000 7f0e3d968700 1 --2- 172.21.15.40:0/693935056 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e28004400 0x7f0e280048d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0e380cf990 tx=0x7f0e280058a0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.264+0000 7f0e3d167700 1 -- 172.21.15.40:0/693935056 reap_dead start 2024-09-13T17:58:18.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.264+0000 7f0e3d968700 1 -- 172.21.15.40:0/693935056 shutdown_connections 2024-09-13T17:58:18.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.264+0000 7f0e3d968700 1 -- 172.21.15.40:0/693935056 >> 172.21.15.40:0/693935056 conn(0x7f0e38066090 msgr2=0x7f0e38066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:18.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.264+0000 7f0e3d968700 1 -- 172.21.15.40:0/693935056 shutdown_connections 2024-09-13T17:58:18.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.264+0000 7f0e3d968700 1 -- 172.21.15.40:0/693935056 wait complete. 2024-09-13T17:58:18.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.264+0000 7f0e3d968700 1 Processor -- start 2024-09-13T17:58:18.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.264+0000 7f0e3d968700 1 -- start start 2024-09-13T17:58:18.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.264+0000 7f0e3d968700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e28004400 0x7f0e38158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.264+0000 7f0e3d968700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e38151340 0x7f0e38151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.264+0000 7f0e3d968700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e381ee8d0 0x7f0e381eed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.264+0000 7f0e3d968700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e380c3ab0 con 0x7f0e38151340 2024-09-13T17:58:18.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.264+0000 7f0e3d968700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e381ef290 con 0x7f0e381ee8d0 2024-09-13T17:58:18.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.264+0000 7f0e3d968700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e380c31c0 con 0x7f0e28004400 2024-09-13T17:58:18.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.264+0000 7f0e37fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e38151340 0x7f0e38151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.264+0000 7f0e37fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e38151340 0x7f0e38151780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59622/0 (socket says 172.21.15.40:59622) 2024-09-13T17:58:18.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.264+0000 7f0e37fff700 1 -- 172.21.15.40:0/1142408742 learned_addr learned my addr 172.21.15.40:0/1142408742 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:18.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e3d167700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e381ee8d0 0x7f0e381eed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e3c966700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e28004400 0x7f0e38158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e37fff700 1 -- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e28004400 msgr2=0x7f0e38158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e37fff700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e28004400 0x7f0e38158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e37fff700 1 -- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e381ee8d0 msgr2=0x7f0e381eed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e37fff700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e381ee8d0 0x7f0e381eed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e37fff700 1 -- 172.21.15.40:0/1142408742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e380c2ba0 con 0x7f0e38151340 2024-09-13T17:58:18.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e3c966700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e28004400 0x7f0e38158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:18.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e3d167700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e381ee8d0 0x7f0e381eed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:18.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e37fff700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e38151340 0x7f0e38151780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400c8b0 tx=0x7f0e24001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e357fa700 1 -- 172.21.15.40:0/1142408742 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24007c40 con 0x7f0e38151340 2024-09-13T17:58:18.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e357fa700 1 -- 172.21.15.40:0/1142408742 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e24004890 con 0x7f0e38151340 2024-09-13T17:58:18.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e3d968700 1 -- 172.21.15.40:0/1142408742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e380c3ab0 con 0x7f0e38151340 2024-09-13T17:58:18.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e357fa700 1 -- 172.21.15.40:0/1142408742 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24004c00 con 0x7f0e38151340 2024-09-13T17:58:18.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e3d968700 1 -- 172.21.15.40:0/1142408742 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e380c3460 con 0x7f0e38151340 2024-09-13T17:58:18.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e3d968700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e380c3620 0x7f0e380c4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e3d968700 1 -- 172.21.15.40:0/1142408742 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e38074750 con 0x7f0e380c3620 2024-09-13T17:58:18.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e3c966700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e380c3620 0x7f0e380c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e3c966700 1 -- 172.21.15.40:0/1142408742 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0e380cba50 con 0x7f0e380c3620 2024-09-13T17:58:18.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e3c966700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e380c3620 0x7f0e380c4350 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0015e0 tx=0x7f0e2c0019a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e357fa700 1 -- 172.21.15.40:0/1142408742 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0e240070d0 con 0x7f0e38151340 2024-09-13T17:58:18.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e357fa700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0e1c066730 0x7f0e1c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e357fa700 1 -- 172.21.15.40:0/1142408742 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0e24085b80 con 0x7f0e38151340 2024-09-13T17:58:18.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.268+0000 7f0e3d167700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0e1c066730 0x7f0e1c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.272+0000 7f0e357fa700 1 -- 172.21.15.40:0/1142408742 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c019070 con 0x7f0e380c3620 2024-09-13T17:58:18.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.272+0000 7f0e357fa700 1 -- 172.21.15.40:0/1142408742 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0e2c013600 con 0x7f0e380c3620 2024-09-13T17:58:18.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.272+0000 7f0e357fa700 1 -- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e380c3620 msgr2=0x7f0e380c4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.272+0000 7f0e357fa700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e380c3620 0x7f0e380c4350 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0015e0 tx=0x7f0e2c0019a0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.272+0000 7f0e357fa700 1 -- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e380c3620 msgr2=0x7f0e380c4350 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:58:18.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.272+0000 7f0e357fa700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e380c3620 0x7f0e380c4350 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.272+0000 7f0e3d167700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0e1c066730 0x7f0e1c068bf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0e380ccb30 tx=0x7f0e3000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.280+0000 7f0e3d968700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e380c4890 0x7f0e381f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.280+0000 7f0e3d968700 1 -- 172.21.15.40:0/1142408742 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e380c3300 con 0x7f0e380c4890 2024-09-13T17:58:18.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.280+0000 7f0e3c966700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e380c4890 0x7f0e381f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.280+0000 7f0e3c966700 1 -- 172.21.15.40:0/1142408742 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0e380cb770 con 0x7f0e380c4890 2024-09-13T17:58:18.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.280+0000 7f0e3c966700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e380c4890 0x7f0e381f1dc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c001eb0 tx=0x7f0e2c007d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.280+0000 7f0e357fa700 1 -- 172.21.15.40:0/1142408742 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c023070 con 0x7f0e380c4890 2024-09-13T17:58:18.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.280+0000 7f0e357fa700 1 -- 172.21.15.40:0/1142408742 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4947 (secure 0 0 0) 0x7f0e2c013e00 con 0x7f0e380c4890 2024-09-13T17:58:18.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.280+0000 7f0e357fa700 1 -- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e380c4890 msgr2=0x7f0e381f1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.280+0000 7f0e357fa700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e380c4890 0x7f0e381f1dc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c001eb0 tx=0x7f0e2c007d50 comp rx=0 tx=0).stop 2024-09-13T17:58:18.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.280+0000 7f0e357fa700 1 -- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e380c4890 msgr2=0x7f0e381f1dc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:58:18.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.280+0000 7f0e357fa700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0e380c4890 0x7f0e381f1dc0 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c001eb0 tx=0x7f0e2c007d50 comp rx=0 tx=0).stop 2024-09-13T17:58:18.290 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:58:18.290 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:58:18.290 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:58:18.290 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:58:18.290 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 62, 2024-09-13T17:58:18.290 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:58:18.290 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:58:18.290 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:58:18.290 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:58:18.291 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:18.291 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T17:58:18.291 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:58:18.291 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:58:18.291 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:58:18.291 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:18.291 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:18.291 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:18.291 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:18.291 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:18.291 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:18.291 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:18.291 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:18.292 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:18.292 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:18.292 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:18.292 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:58:18.292 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:58:18.292 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:18.292 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:18.292 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:18.292 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:18.292 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:18.292 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:18.292 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:18.292 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:18.293 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:18.293 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:18.293 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:18.293 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.293 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:58:18.293 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:58:18.293 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:58:18.293 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:58:18.293 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:58:18.293 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:58:18.293 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:58:18.293 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:58:18.293 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:58:18.293 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:58:18.293 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:58:18.294 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:58:18.294 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:58:18.294 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:58:18.294 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:58:18.294 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:58:18.294 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:58:18.294 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:18.294 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:18.294 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:18.294 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:18.294 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:18.294 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:18.294 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:18.294 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:18.295 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:18.295 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:18.295 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:18.295 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:58:18.295 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.295 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:58:18.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.295 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:58:18.295 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:58:18.295 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:18.295 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:18.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.295 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:18.295 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:58:18.296 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:18.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.296 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.296 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:18.296 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:58:18.296 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:18.296 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:18.296 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:18.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.296 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:58:18.296 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:58:18.297 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:18.297 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:18.297 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:18.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.297 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:58:18.297 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:58:18.297 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:18.297 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:18.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.297 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:18.297 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:58:18.297 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:18.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.298 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.298 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:18.298 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:58:18.298 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:18.298 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:18.298 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:18.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.298 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:58:18.298 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:58:18.298 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:18.298 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:18.298 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:18.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.298 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.299 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:58:18.299 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:58:18.299 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:18.299 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:18.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.299 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:18.299 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:58:18.299 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:18.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.299 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:18.299 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:58:18.299 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:18.299 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:18.300 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:18.300 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.300 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:58:18.300 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:58:18.300 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:18.300 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:18.300 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:18.300 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:18.300 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:18.300 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.300 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:58:18.300 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:58:18.300 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.301 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:18.301 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:18.301 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:18.301 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:18.301 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:18.301 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:58:18.301 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.301 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:18.301 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:18.301 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:58:18.301 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:18.301 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:18.301 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:58:18.301 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.301 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:18.302 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:18.302 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T17:58:18.302 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:18.302 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:18.302 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:58:18.302 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.302 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:18.302 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:18.302 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:58:18.302 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:18.302 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:18.302 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:58:18.302 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:18.303 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:18.303 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:18.303 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:18.303 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:18.303 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:18.303 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:18.303 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:58:18.303 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:58:18.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.284+0000 7f0e3d968700 1 -- 172.21.15.40:0/1142408742 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0e1c066730 msgr2=0x7f0e1c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.284+0000 7f0e3d968700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0e1c066730 0x7f0e1c068bf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0e380ccb30 tx=0x7f0e3000b040 comp rx=0 tx=0).stop 2024-09-13T17:58:18.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.284+0000 7f0e3d968700 1 -- 172.21.15.40:0/1142408742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e38151340 msgr2=0x7f0e38151780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.284+0000 7f0e3d968700 1 --2- 172.21.15.40:0/1142408742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e38151340 0x7f0e38151780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400c8b0 tx=0x7f0e24001a00 comp rx=0 tx=0).stop 2024-09-13T17:58:18.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.284+0000 7f0e3d167700 1 -- 172.21.15.40:0/1142408742 reap_dead start 2024-09-13T17:58:18.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.284+0000 7f0e3d167700 1 -- 172.21.15.40:0/1142408742 reap_dead start 2024-09-13T17:58:18.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.284+0000 7f0e3d968700 1 -- 172.21.15.40:0/1142408742 shutdown_connections 2024-09-13T17:58:18.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.284+0000 7f0e3d968700 1 -- 172.21.15.40:0/1142408742 >> 172.21.15.40:0/1142408742 conn(0x7f0e38066090 msgr2=0x7f0e3815c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:18.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.284+0000 7f0e3d968700 1 -- 172.21.15.40:0/1142408742 shutdown_connections 2024-09-13T17:58:18.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.284+0000 7f0e3d968700 1 -- 172.21.15.40:0/1142408742 wait complete. 2024-09-13T17:58:18.305 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:58:18.306 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T17:58:18.306 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T17:58:18.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.408+0000 7fca2e595700 1 Processor -- start 2024-09-13T17:58:18.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.408+0000 7fca2e595700 1 -- start start 2024-09-13T17:58:18.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.408+0000 7fca2e595700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fca20046f90 0x7fca2004af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.408+0000 7fca2e595700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca20007640 con 0x7fca2004ca40 2024-09-13T17:58:18.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.408+0000 7fca2e595700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca20045470 con 0x7fca200d8a20 2024-09-13T17:58:18.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.408+0000 7fca2e595700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca200473a0 con 0x7fca20046f90 2024-09-13T17:58:18.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.408+0000 7fca2cd92700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fca2004ca40 0x7fca200e6f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50396/0 (socket says 172.21.15.40:50396) 2024-09-13T17:58:18.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.408+0000 7fca2cd92700 1 -- 172.21.15.40:0/64885434 learned_addr learned my addr 172.21.15.40:0/64885434 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:58:18.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.408+0000 7fca2dd94700 1 --2- 172.21.15.40:0/64885434 >> v2:172.21.15.43:3300/0 conn(0x7fca20046f90 0x7fca2004af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.408+0000 7fca1ffff700 1 -- 172.21.15.40:0/64885434 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 40530384 0 0) 0x7fca20007640 con 0x7fca2004ca40 2024-09-13T17:58:18.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.408+0000 7fca1ffff700 1 -- 172.21.15.40:0/64885434 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca140035d0 con 0x7fca2004ca40 2024-09-13T17:58:18.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.408+0000 7fca2dd94700 1 -- 172.21.15.40:0/64885434 >> v1:172.21.15.43:6789/0 conn(0x7fca200d8a20 legacy=0x7fca20006920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.408+0000 7fca2dd94700 1 -- 172.21.15.40:0/64885434 >> v1:172.21.15.40:6789/0 conn(0x7fca2004ca40 legacy=0x7fca200e6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.408+0000 7fca1ffff700 1 -- 172.21.15.40:0/64885434 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2939602147 0 0) 0x7fca140035d0 con 0x7fca2004ca40 2024-09-13T17:58:18.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.408+0000 7fca2dd94700 1 -- 172.21.15.40:0/64885434 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca20046b00 con 0x7fca20046f90 2024-09-13T17:58:18.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.408+0000 7fca2dd94700 1 --2- 172.21.15.40:0/64885434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca20046f90 0x7fca2004af10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fca18009850 tx=0x7fca18009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca3fd1d91ed813ac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.412+0000 7fca1ffff700 1 -- 172.21.15.40:0/64885434 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18011830 con 0x7fca20046f90 2024-09-13T17:58:18.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.412+0000 7fca1ffff700 1 -- 172.21.15.40:0/64885434 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca18011990 con 0x7fca20046f90 2024-09-13T17:58:18.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.412+0000 7fca1ffff700 1 -- 172.21.15.40:0/64885434 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18011d00 con 0x7fca20046f90 2024-09-13T17:58:18.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.412+0000 7fca2e595700 1 -- 172.21.15.40:0/64885434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca20046f90 msgr2=0x7fca2004af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.412+0000 7fca2e595700 1 --2- 172.21.15.40:0/64885434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca20046f90 0x7fca2004af10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fca18009850 tx=0x7fca18009b60 comp rx=0 tx=0).stop 2024-09-13T17:58:18.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.412+0000 7fca2e595700 1 -- 172.21.15.40:0/64885434 shutdown_connections 2024-09-13T17:58:18.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.412+0000 7fca2e595700 1 --2- 172.21.15.40:0/64885434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca20046f90 0x7fca2004af10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.412+0000 7fca2e595700 1 -- 172.21.15.40:0/64885434 >> 172.21.15.40:0/64885434 conn(0x7fca20050590 msgr2=0x7fca200509a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:18.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.412+0000 7fca2e595700 1 -- 172.21.15.40:0/64885434 shutdown_connections 2024-09-13T17:58:18.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.412+0000 7fca2e595700 1 -- 172.21.15.40:0/64885434 wait complete. 2024-09-13T17:58:18.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.412+0000 7fca2e595700 1 Processor -- start 2024-09-13T17:58:18.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.412+0000 7fca2e595700 1 -- start start 2024-09-13T17:58:18.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2e595700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca20046f90 0x7fca20031aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2e595700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca2004ca40 0x7fca201782d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2e595700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fca200d8a20 0x7fca2004f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2e595700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca20178810 con 0x7fca20046f90 2024-09-13T17:58:18.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2e595700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca20031520 con 0x7fca2004ca40 2024-09-13T17:58:18.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2e595700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca200317c0 con 0x7fca200d8a20 2024-09-13T17:58:18.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2d593700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca20046f90 0x7fca20031aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2d593700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca20046f90 0x7fca20031aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59624/0 (socket says 172.21.15.40:59624) 2024-09-13T17:58:18.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2d593700 1 -- 172.21.15.40:0/1470725775 learned_addr learned my addr 172.21.15.40:0/1470725775 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:18.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2dd94700 1 --2- 172.21.15.40:0/1470725775 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fca200d8a20 0x7fca2004f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2cd92700 1 --2- 172.21.15.40:0/1470725775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca2004ca40 0x7fca201782d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2d593700 1 -- 172.21.15.40:0/1470725775 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fca200d8a20 msgr2=0x7fca2004f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2d593700 1 --2- 172.21.15.40:0/1470725775 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fca200d8a20 0x7fca2004f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2d593700 1 -- 172.21.15.40:0/1470725775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca2004ca40 msgr2=0x7fca201782d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2d593700 1 --2- 172.21.15.40:0/1470725775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca2004ca40 0x7fca201782d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2d593700 1 -- 172.21.15.40:0/1470725775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca180094b0 con 0x7fca20046f90 2024-09-13T17:58:18.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2d593700 1 --2- 172.21.15.40:0/1470725775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca20046f90 0x7fca20031aa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fca2804f480 tx=0x7fca280680b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2dd94700 1 --2- 172.21.15.40:0/1470725775 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fca200d8a20 0x7fca2004f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:58:18.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca1dffb700 1 -- 172.21.15.40:0/1470725775 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca28067930 con 0x7fca20046f90 2024-09-13T17:58:18.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2e595700 1 -- 172.21.15.40:0/1470725775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca20030130 con 0x7fca20046f90 2024-09-13T17:58:18.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca1dffb700 1 -- 172.21.15.40:0/1470725775 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca28072150 con 0x7fca20046f90 2024-09-13T17:58:18.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca1dffb700 1 -- 172.21.15.40:0/1470725775 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca28072540 con 0x7fca20046f90 2024-09-13T17:58:18.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2e595700 1 -- 172.21.15.40:0/1470725775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca20047d80 con 0x7fca20046f90 2024-09-13T17:58:18.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.416+0000 7fca2e595700 1 -- 172.21.15.40:0/1470725775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca200459a0 con 0x7fca20046f90 2024-09-13T17:58:18.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.420+0000 7fca1dffb700 1 -- 172.21.15.40:0/1470725775 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fca28091020 con 0x7fca20046f90 2024-09-13T17:58:18.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.420+0000 7fca1dffb700 1 --2- 172.21.15.40:0/1470725775 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fca0c066730 0x7fca0c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:18.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.420+0000 7fca1dffb700 1 -- 172.21.15.40:0/1470725775 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fca280fe840 con 0x7fca20046f90 2024-09-13T17:58:18.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.420+0000 7fca2cd92700 1 --2- 172.21.15.40:0/1470725775 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fca0c066730 0x7fca0c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:18.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.420+0000 7fca2cd92700 1 --2- 172.21.15.40:0/1470725775 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fca0c066730 0x7fca0c068bf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fca200de400 tx=0x7fca24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:18.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.424+0000 7fca1dffb700 1 -- 172.21.15.40:0/1470725775 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fca280cb800 con 0x7fca20046f90 2024-09-13T17:58:18.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.691+0000 7fca2e595700 1 -- 172.21.15.40:0/1470725775 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fca200d75a0 con 0x7fca20046f90 2024-09-13T17:58:18.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.691+0000 7fca1dffb700 1 -- 172.21.15.40:0/1470725775 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fca280cee20 con 0x7fca20046f90 2024-09-13T17:58:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.695+0000 7fca2e595700 1 -- 172.21.15.40:0/1470725775 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fca0c066730 msgr2=0x7fca0c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.695+0000 7fca2e595700 1 --2- 172.21.15.40:0/1470725775 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fca0c066730 0x7fca0c068bf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fca200de400 tx=0x7fca24009040 comp rx=0 tx=0).stop 2024-09-13T17:58:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.695+0000 7fca2e595700 1 -- 172.21.15.40:0/1470725775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca20046f90 msgr2=0x7fca20031aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.695+0000 7fca2e595700 1 --2- 172.21.15.40:0/1470725775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca20046f90 0x7fca20031aa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fca2804f480 tx=0x7fca280680b0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.695+0000 7fca2e595700 1 -- 172.21.15.40:0/1470725775 shutdown_connections 2024-09-13T17:58:18.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.695+0000 7fca2e595700 1 --2- 172.21.15.40:0/1470725775 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fca0c066730 0x7fca0c068bf0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.695+0000 7fca2e595700 1 --2- 172.21.15.40:0/1470725775 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fca200d8a20 0x7fca2004f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.695+0000 7fca2e595700 1 --2- 172.21.15.40:0/1470725775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca2004ca40 0x7fca201782d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.695+0000 7fca2e595700 1 --2- 172.21.15.40:0/1470725775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca20046f90 0x7fca20031aa0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:18.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.695+0000 7fca2e595700 1 -- 172.21.15.40:0/1470725775 >> 172.21.15.40:0/1470725775 conn(0x7fca20050590 msgr2=0x7fca2004b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:18.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.695+0000 7fca2e595700 1 -- 172.21.15.40:0/1470725775 shutdown_connections 2024-09-13T17:58:18.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:18.695+0000 7fca2e595700 1 -- 172.21.15.40:0/1470725775 wait complete. 2024-09-13T17:58:18.719 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T17:58:28.720 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T17:58:28.720 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:58:28.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.847+0000 7f8f599d3700 1 Processor -- start 2024-09-13T17:58:28.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.847+0000 7f8f599d3700 1 -- start start 2024-09-13T17:58:28.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.847+0000 7f8f599d3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f4c00b270 0x7f8f4c0e01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.847+0000 7f8f599d3700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f4c0470a0 con 0x7f8f4c00b760 2024-09-13T17:58:28.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.847+0000 7f8f599d3700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f4c047380 con 0x7f8f4c0d65b0 2024-09-13T17:58:28.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.847+0000 7f8f599d3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c00a640 con 0x7f8f4c00b270 2024-09-13T17:58:28.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.847+0000 7f8f591d2700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8f4c00b760 0x7f8f4c0e6f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56668/0 (socket says 172.21.15.40:56668) 2024-09-13T17:58:28.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.847+0000 7f8f591d2700 1 -- 172.21.15.40:0/1139592717 learned_addr learned my addr 172.21.15.40:0/1139592717 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:58:28.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.847+0000 7f8f53fff700 1 --2- 172.21.15.40:0/1139592717 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f4c00b270 0x7f8f4c0e01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.851+0000 7f8f537fe700 1 -- 172.21.15.40:0/1139592717 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1281828544 0 0) 0x7f8f4c0470a0 con 0x7f8f4c00b760 2024-09-13T17:58:28.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.851+0000 7f8f537fe700 1 -- 172.21.15.40:0/1139592717 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f440035d0 con 0x7f8f4c00b760 2024-09-13T17:58:28.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.851+0000 7f8f537fe700 1 -- 172.21.15.40:0/1139592717 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322239592 0 0) 0x7f8f4c047380 con 0x7f8f4c0d65b0 2024-09-13T17:58:28.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.851+0000 7f8f537fe700 1 -- 172.21.15.40:0/1139592717 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f4c0470a0 con 0x7f8f4c0d65b0 2024-09-13T17:58:28.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.851+0000 7f8f53fff700 1 -- 172.21.15.40:0/1139592717 >> v1:172.21.15.43:6789/0 conn(0x7f8f4c0d65b0 legacy=0x7f8f4c008860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.851+0000 7f8f53fff700 1 -- 172.21.15.40:0/1139592717 >> v1:172.21.15.40:6789/0 conn(0x7f8f4c00b760 legacy=0x7f8f4c0e6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.851+0000 7f8f53fff700 1 -- 172.21.15.40:0/1139592717 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c047660 con 0x7f8f4c00b270 2024-09-13T17:58:28.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.851+0000 7f8f53fff700 1 --2- 172.21.15.40:0/1139592717 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f4c00b270 0x7f8f4c0e01e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009880 tx=0x7f8f40009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f1a66df4046df22 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:28.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.851+0000 7f8f537fe700 1 -- 172.21.15.40:0/1139592717 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40011830 con 0x7f8f4c00b270 2024-09-13T17:58:28.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.851+0000 7f8f537fe700 1 -- 172.21.15.40:0/1139592717 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f40011990 con 0x7f8f4c00b270 2024-09-13T17:58:28.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.851+0000 7f8f537fe700 1 -- 172.21.15.40:0/1139592717 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40011d20 con 0x7f8f4c00b270 2024-09-13T17:58:28.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.851+0000 7f8f599d3700 1 -- 172.21.15.40:0/1139592717 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f4c00b270 msgr2=0x7f8f4c0e01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.851+0000 7f8f599d3700 1 --2- 172.21.15.40:0/1139592717 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f4c00b270 0x7f8f4c0e01e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009880 tx=0x7f8f40009b90 comp rx=0 tx=0).stop 2024-09-13T17:58:28.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.851+0000 7f8f599d3700 1 -- 172.21.15.40:0/1139592717 shutdown_connections 2024-09-13T17:58:28.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.851+0000 7f8f599d3700 1 --2- 172.21.15.40:0/1139592717 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f4c00b270 0x7f8f4c0e01e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.851+0000 7f8f599d3700 1 -- 172.21.15.40:0/1139592717 >> 172.21.15.40:0/1139592717 conn(0x7f8f4c00a780 msgr2=0x7f8f4c00ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:28.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.851+0000 7f8f599d3700 1 -- 172.21.15.40:0/1139592717 shutdown_connections 2024-09-13T17:58:28.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.851+0000 7f8f599d3700 1 -- 172.21.15.40:0/1139592717 wait complete. 2024-09-13T17:58:28.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.851+0000 7f8f599d3700 1 Processor -- start 2024-09-13T17:58:28.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.851+0000 7f8f599d3700 1 -- start start 2024-09-13T17:58:28.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f599d3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f4c00b270 0x7f8f4c0d8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f599d3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f4c00b760 0x7f8f4c0dd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f599d3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f4c0d65b0 0x7f8f4c0d7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f599d3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c173380 con 0x7f8f4c00b760 2024-09-13T17:58:28.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f599d3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c0d6c80 con 0x7f8f4c0d65b0 2024-09-13T17:58:28.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f599d3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c0d6f20 con 0x7f8f4c00b270 2024-09-13T17:58:28.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f53fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f4c00b760 0x7f8f4c0dd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f53fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f4c00b760 0x7f8f4c0dd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41496/0 (socket says 172.21.15.40:41496) 2024-09-13T17:58:28.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f53fff700 1 -- 172.21.15.40:0/1789030736 learned_addr learned my addr 172.21.15.40:0/1789030736 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:28.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f589d1700 1 --2- 172.21.15.40:0/1789030736 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f4c00b270 0x7f8f4c0d8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f591d2700 1 --2- 172.21.15.40:0/1789030736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f4c0d65b0 0x7f8f4c0d7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f53fff700 1 -- 172.21.15.40:0/1789030736 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f4c00b270 msgr2=0x7f8f4c0d8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f53fff700 1 --2- 172.21.15.40:0/1789030736 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f4c00b270 0x7f8f4c0d8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f53fff700 1 -- 172.21.15.40:0/1789030736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f4c0d65b0 msgr2=0x7f8f4c0d7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:28.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f53fff700 1 --2- 172.21.15.40:0/1789030736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f4c0d65b0 0x7f8f4c0d7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:28.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f53fff700 1 -- 172.21.15.40:0/1789030736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f400094b0 con 0x7f8f4c00b760 2024-09-13T17:58:28.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f53fff700 1 --2- 172.21.15.40:0/1789030736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f4c00b760 0x7f8f4c0dd780 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009bd0 tx=0x7f8f40004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:28.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f517fa700 1 -- 172.21.15.40:0/1789030736 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40004450 con 0x7f8f4c00b760 2024-09-13T17:58:28.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f517fa700 1 -- 172.21.15.40:0/1789030736 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f400045d0 con 0x7f8f4c00b760 2024-09-13T17:58:28.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f599d3700 1 -- 172.21.15.40:0/1789030736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4c172e40 con 0x7f8f4c00b760 2024-09-13T17:58:28.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f517fa700 1 -- 172.21.15.40:0/1789030736 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40003900 con 0x7f8f4c00b760 2024-09-13T17:58:28.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f599d3700 1 -- 172.21.15.40:0/1789030736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4c04ad70 con 0x7f8f4c00b760 2024-09-13T17:58:28.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f599d3700 1 -- 172.21.15.40:0/1789030736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4c0d7090 con 0x7f8f4c00b760 2024-09-13T17:58:28.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.855+0000 7f8f517fa700 1 -- 172.21.15.40:0/1789030736 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8f40003a60 con 0x7f8f4c00b760 2024-09-13T17:58:28.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.859+0000 7f8f517fa700 1 --2- 172.21.15.40:0/1789030736 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8f38066760 0x7f8f38068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:28.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.859+0000 7f8f517fa700 1 -- 172.21.15.40:0/1789030736 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8f400856c0 con 0x7f8f4c00b760 2024-09-13T17:58:28.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.859+0000 7f8f589d1700 1 --2- 172.21.15.40:0/1789030736 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8f38066760 0x7f8f38068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:28.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.859+0000 7f8f589d1700 1 --2- 172.21.15.40:0/1789030736 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8f38066760 0x7f8f38068c20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8f480011e0 tx=0x7f8f48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:28.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:28.867+0000 7f8f517fa700 1 -- 172.21.15.40:0/1789030736 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8f400525e0 con 0x7f8f4c00b760 2024-09-13T17:58:29.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.123+0000 7f8f599d3700 1 -- 172.21.15.40:0/1789030736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8f4c0361c0 con 0x7f8f4c00b760 2024-09-13T17:58:29.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.127+0000 7f8f517fa700 1 -- 172.21.15.40:0/1789030736 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f8f40072e60 con 0x7f8f4c00b760 2024-09-13T17:58:29.134 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":62,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:58:29.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.127+0000 7f8f599d3700 1 -- 172.21.15.40:0/1789030736 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8f38066760 msgr2=0x7f8f38068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.127+0000 7f8f599d3700 1 --2- 172.21.15.40:0/1789030736 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8f38066760 0x7f8f38068c20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8f480011e0 tx=0x7f8f48009040 comp rx=0 tx=0).stop 2024-09-13T17:58:29.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.127+0000 7f8f599d3700 1 -- 172.21.15.40:0/1789030736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f4c00b760 msgr2=0x7f8f4c0dd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.127+0000 7f8f599d3700 1 --2- 172.21.15.40:0/1789030736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f4c00b760 0x7f8f4c0dd780 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009bd0 tx=0x7f8f40004770 comp rx=0 tx=0).stop 2024-09-13T17:58:29.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.127+0000 7f8f599d3700 1 -- 172.21.15.40:0/1789030736 shutdown_connections 2024-09-13T17:58:29.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.127+0000 7f8f599d3700 1 --2- 172.21.15.40:0/1789030736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f4c0d65b0 0x7f8f4c0d7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.127+0000 7f8f599d3700 1 --2- 172.21.15.40:0/1789030736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f4c00b760 0x7f8f4c0dd780 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.127+0000 7f8f599d3700 1 --2- 172.21.15.40:0/1789030736 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8f38066760 0x7f8f38068c20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.127+0000 7f8f599d3700 1 --2- 172.21.15.40:0/1789030736 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f4c00b270 0x7f8f4c0d8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.127+0000 7f8f599d3700 1 -- 172.21.15.40:0/1789030736 >> 172.21.15.40:0/1789030736 conn(0x7f8f4c00a780 msgr2=0x7f8f4c0d2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:29.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.127+0000 7f8f599d3700 1 -- 172.21.15.40:0/1789030736 shutdown_connections 2024-09-13T17:58:29.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.127+0000 7f8f599d3700 1 -- 172.21.15.40:0/1789030736 wait complete. 2024-09-13T17:58:29.151 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T17:58:29.152 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T17:58:29.152 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T17:58:29.152 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:58:29.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.255+0000 7f8bc0d40700 1 Processor -- start 2024-09-13T17:58:29.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.255+0000 7f8bc0d40700 1 -- start start 2024-09-13T17:58:29.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.255+0000 7f8bc0d40700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8bbc1503c0 0x7f8bbc0c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.255+0000 7f8bc0d40700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8bbc0c8500 0x7f8bbc0c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.255+0000 7f8bc0d40700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bbc0c8f80 con 0x7f8bbc0c71c0 2024-09-13T17:58:29.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.255+0000 7f8bc0d40700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bbc0c2a80 con 0x7f8bbc0c8500 2024-09-13T17:58:29.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.255+0000 7f8bc0d40700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc1507a0 con 0x7f8bbc1503c0 2024-09-13T17:58:29.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bbaffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8bbc0c8500 0x7f8bbc0c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bbbfff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8bbc0c71c0 0x7f8bbc159630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56670/0 (socket says 172.21.15.40:56670) 2024-09-13T17:58:29.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bbbfff700 1 -- 172.21.15.40:0/3006004331 learned_addr learned my addr 172.21.15.40:0/3006004331 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:58:29.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bbb7fe700 1 --2- 172.21.15.40:0/3006004331 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8bbc1503c0 0x7f8bbc0c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bbaffd700 1 -- 172.21.15.40:0/3006004331 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8bbc1503c0 msgr2=0x7f8bbc0c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:29.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bbaffd700 1 --2- 172.21.15.40:0/3006004331 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8bbc1503c0 0x7f8bbc0c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bbaffd700 1 -- 172.21.15.40:0/3006004331 >> v1:172.21.15.40:6789/0 conn(0x7f8bbc0c71c0 legacy=0x7f8bbc159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bba7fc700 1 -- 172.21.15.40:0/3006004331 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2878207712 0 0) 0x7f8bbc0c8f80 con 0x7f8bbc0c71c0 2024-09-13T17:58:29.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bbaffd700 1 -- 172.21.15.40:0/3006004331 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc0c8a40 con 0x7f8bbc0c8500 2024-09-13T17:58:29.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bbaffd700 1 --2- 172.21.15.40:0/3006004331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0c8500 0x7f8bbc0c6c80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009850 tx=0x7f8bb0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352807715961f522 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bba7fc700 1 -- 172.21.15.40:0/3006004331 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0011830 con 0x7f8bbc0c8500 2024-09-13T17:58:29.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bba7fc700 1 -- 172.21.15.40:0/3006004331 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb0011990 con 0x7f8bbc0c8500 2024-09-13T17:58:29.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bba7fc700 1 -- 172.21.15.40:0/3006004331 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0011d40 con 0x7f8bbc0c8500 2024-09-13T17:58:29.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bc0d40700 1 -- 172.21.15.40:0/3006004331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0c8500 msgr2=0x7f8bbc0c6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bc0d40700 1 --2- 172.21.15.40:0/3006004331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0c8500 0x7f8bbc0c6c80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009850 tx=0x7f8bb0009b60 comp rx=0 tx=0).stop 2024-09-13T17:58:29.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bc0d40700 1 -- 172.21.15.40:0/3006004331 shutdown_connections 2024-09-13T17:58:29.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bc0d40700 1 --2- 172.21.15.40:0/3006004331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0c8500 0x7f8bbc0c6c80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bc0d40700 1 --2- 172.21.15.40:0/3006004331 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8bbc1503c0 0x7f8bbc0c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bc0d40700 1 -- 172.21.15.40:0/3006004331 >> 172.21.15.40:0/3006004331 conn(0x7f8bbc06e6d0 msgr2=0x7f8bbc06eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:29.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bc0d40700 1 -- 172.21.15.40:0/3006004331 shutdown_connections 2024-09-13T17:58:29.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.259+0000 7f8bc0d40700 1 -- 172.21.15.40:0/3006004331 wait complete. 2024-09-13T17:58:29.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bc0d40700 1 Processor -- start 2024-09-13T17:58:29.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bc0d40700 1 -- start start 2024-09-13T17:58:29.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bc0d40700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8bbc0c71c0 0x7f8bbc14eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bc0d40700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc1503c0 0x7f8bbc145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bc0d40700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bbc1e25e0 0x7f8bbc146010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bc0d40700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc146990 con 0x7f8bbc1503c0 2024-09-13T17:58:29.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bc0d40700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc0c5840 con 0x7f8bbc1e25e0 2024-09-13T17:58:29.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bc0d40700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc0c3b20 con 0x7f8bbc0c71c0 2024-09-13T17:58:29.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bbaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc1503c0 0x7f8bbc145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bbaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc1503c0 0x7f8bbc145ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41516/0 (socket says 172.21.15.40:41516) 2024-09-13T17:58:29.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bbaffd700 1 -- 172.21.15.40:0/1526518296 learned_addr learned my addr 172.21.15.40:0/1526518296 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:29.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bbbfff700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bbc1e25e0 0x7f8bbc146010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bbb7fe700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8bbc0c71c0 0x7f8bbc14eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bbaffd700 1 -- 172.21.15.40:0/1526518296 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8bbc0c71c0 msgr2=0x7f8bbc14eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bbaffd700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8bbc0c71c0 0x7f8bbc14eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bbaffd700 1 -- 172.21.15.40:0/1526518296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bbc1e25e0 msgr2=0x7f8bbc146010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bbaffd700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bbc1e25e0 0x7f8bbc146010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bbaffd700 1 -- 172.21.15.40:0/1526518296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb00094b0 con 0x7f8bbc1503c0 2024-09-13T17:58:29.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bbaffd700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc1503c0 0x7f8bbc145ad0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8bb00090b0 tx=0x7f8bb0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8ba3fff700 1 -- 172.21.15.40:0/1526518296 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0004550 con 0x7f8bbc1503c0 2024-09-13T17:58:29.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8ba3fff700 1 -- 172.21.15.40:0/1526518296 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb001f070 con 0x7f8bbc1503c0 2024-09-13T17:58:29.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bc0d40700 1 -- 172.21.15.40:0/1526518296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bbc145640 con 0x7f8bbc1503c0 2024-09-13T17:58:29.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8ba3fff700 1 -- 172.21.15.40:0/1526518296 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0003900 con 0x7f8bbc1503c0 2024-09-13T17:58:29.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.263+0000 7f8bc0d40700 1 -- 172.21.15.40:0/1526518296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bbc1e30e0 con 0x7f8bbc1503c0 2024-09-13T17:58:29.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.267+0000 7f8bc0d40700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0c5bc0 0x7f8bbc1e3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.267+0000 7f8bc0d40700 1 -- 172.21.15.40:0/1526518296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc0675e0 con 0x7f8bbc0c5bc0 2024-09-13T17:58:29.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.267+0000 7f8bbb7fe700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0c5bc0 0x7f8bbc1e3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.267+0000 7f8bbb7fe700 1 -- 172.21.15.40:0/1526518296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8bbc0c2710 con 0x7f8bbc0c5bc0 2024-09-13T17:58:29.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.267+0000 7f8bbb7fe700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0c5bc0 0x7f8bbc1e3270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0119d0 tx=0x7f8bac011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.267+0000 7f8ba3fff700 1 -- 172.21.15.40:0/1526518296 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8bb0003a60 con 0x7f8bbc1503c0 2024-09-13T17:58:29.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.267+0000 7f8ba3fff700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ba8066730 0x7f8ba8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.267+0000 7f8ba3fff700 1 -- 172.21.15.40:0/1526518296 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8bb0021070 con 0x7f8bbc1503c0 2024-09-13T17:58:29.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.267+0000 7f8bbbfff700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ba8066730 0x7f8ba8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.267+0000 7f8ba3fff700 1 -- 172.21.15.40:0/1526518296 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac004240 con 0x7f8bbc0c5bc0 2024-09-13T17:58:29.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.267+0000 7f8ba3fff700 1 -- 172.21.15.40:0/1526518296 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8bac0043a0 con 0x7f8bbc0c5bc0 2024-09-13T17:58:29.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.267+0000 7f8ba3fff700 1 -- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0c5bc0 msgr2=0x7f8bbc1e3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.267+0000 7f8ba3fff700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0c5bc0 0x7f8bbc1e3270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0119d0 tx=0x7f8bac011d90 comp rx=0 tx=0).stop 2024-09-13T17:58:29.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.267+0000 7f8ba3fff700 1 -- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0c5bc0 msgr2=0x7f8bbc1e3270 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:58:29.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.267+0000 7f8ba3fff700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0c5bc0 0x7f8bbc1e3270 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.267+0000 7f8bbbfff700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ba8066730 0x7f8ba8068bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4001d10 tx=0x7f8bb4001ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.279+0000 7f8bc0d40700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0c33d0 0x7f8bbc1f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.279+0000 7f8bc0d40700 1 -- 172.21.15.40:0/1526518296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc0c8ce0 con 0x7f8bbc0c33d0 2024-09-13T17:58:29.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.279+0000 7f8bbb7fe700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0c33d0 0x7f8bbc1f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.279+0000 7f8bbb7fe700 1 -- 172.21.15.40:0/1526518296 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8bbc0b1e60 con 0x7f8bbc0c33d0 2024-09-13T17:58:29.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.279+0000 7f8bbb7fe700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0c33d0 0x7f8bbc1f2a80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0119d0 tx=0x7f8bac004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.283+0000 7f8ba3fff700 1 -- 172.21.15.40:0/1526518296 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac01e450 con 0x7f8bbc0c33d0 2024-09-13T17:58:29.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.283+0000 7f8ba3fff700 1 -- 172.21.15.40:0/1526518296 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4452 (secure 0 0 0) 0x7f8bac0283a0 con 0x7f8bbc0c33d0 2024-09-13T17:58:29.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.283+0000 7f8ba3fff700 1 -- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0c33d0 msgr2=0x7f8bbc1f2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.283+0000 7f8ba3fff700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0c33d0 0x7f8bbc1f2a80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0119d0 tx=0x7f8bac004770 comp rx=0 tx=0).stop 2024-09-13T17:58:29.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.283+0000 7f8ba3fff700 1 -- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0c33d0 msgr2=0x7f8bbc1f2a80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:58:29.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.283+0000 7f8ba3fff700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc0c33d0 0x7f8bbc1f2a80 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0119d0 tx=0x7f8bac004770 comp rx=0 tx=0).stop 2024-09-13T17:58:29.291 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:58:29.291 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:58:29.291 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:58:29.291 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:58:29.291 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 62, 2024-09-13T17:58:29.291 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:58:29.292 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:58:29.292 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:58:29.292 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:58:29.292 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:29.292 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 12, 2024-09-13T17:58:29.292 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:58:29.292 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:58:29.292 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:58:29.292 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:29.292 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:29.292 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:29.292 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:29.292 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:29.293 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:29.293 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:29.293 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:29.293 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:29.293 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:29.293 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:29.293 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:58:29.293 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:58:29.293 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:29.293 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:29.293 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:29.294 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:29.294 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:29.294 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:29.294 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:29.294 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:29.294 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:29.294 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:29.294 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:29.294 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.294 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:58:29.294 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:58:29.294 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:58:29.294 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:58:29.295 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:58:29.295 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:58:29.295 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:58:29.295 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:58:29.295 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:58:29.295 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:58:29.295 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:58:29.295 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:58:29.295 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:58:29.295 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:58:29.295 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:58:29.295 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:58:29.295 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:58:29.296 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:29.296 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:29.296 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:29.296 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:29.296 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:29.296 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:29.296 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:29.296 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:29.296 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:29.296 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:29.296 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:29.296 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:58:29.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.296 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:58:29.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.297 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:58:29.297 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:58:29.297 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:29.297 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:29.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.297 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:29.297 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:58:29.298 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:29.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.298 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.298 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:29.298 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:58:29.298 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:29.298 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.298 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:29.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.298 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:58:29.298 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:58:29.298 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:29.298 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:29.298 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:29.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.299 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:58:29.299 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:58:29.299 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:29.299 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:29.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.299 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:29.299 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:58:29.299 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:29.300 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.300 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.300 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:29.300 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:58:29.300 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:29.300 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.300 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:29.300 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.300 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:58:29.300 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:58:29.300 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:29.300 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:29.300 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:29.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.301 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.301 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:58:29.301 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:58:29.301 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:29.301 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:29.301 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.301 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:29.301 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:58:29.301 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:29.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.301 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.301 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:29.301 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:58:29.302 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:29.302 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.302 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:29.302 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.302 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:58:29.302 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:58:29.302 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:29.302 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:29.302 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:29.302 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.302 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:29.302 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.302 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:58:29.302 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:58:29.303 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.303 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:29.303 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:29.303 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:29.303 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.303 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:29.303 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:58:29.303 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.303 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:29.303 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:29.303 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:58:29.303 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.303 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:29.303 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.304 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:58:29.304 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:58:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.283+0000 7f8bc0d40700 1 -- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ba8066730 msgr2=0x7f8ba8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.283+0000 7f8bc0d40700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ba8066730 0x7f8ba8068bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4001d10 tx=0x7f8bb4001ca0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.283+0000 7f8bc0d40700 1 -- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc1503c0 msgr2=0x7f8bbc145ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.283+0000 7f8bbbfff700 1 -- 172.21.15.40:0/1526518296 reap_dead start 2024-09-13T17:58:29.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.283+0000 7f8bc0d40700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc1503c0 0x7f8bbc145ad0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8bb00090b0 tx=0x7f8bb0004770 comp rx=0 tx=0).stop 2024-09-13T17:58:29.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.283+0000 7f8bc0d40700 1 -- 172.21.15.40:0/1526518296 shutdown_connections 2024-09-13T17:58:29.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.283+0000 7f8bc0d40700 1 --2- 172.21.15.40:0/1526518296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bbc1503c0 0x7f8bbc145ad0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.283+0000 7f8bc0d40700 1 -- 172.21.15.40:0/1526518296 >> 172.21.15.40:0/1526518296 conn(0x7f8bbc06e6d0 msgr2=0x7f8bbc155430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:29.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.283+0000 7f8bc0d40700 1 -- 172.21.15.40:0/1526518296 shutdown_connections 2024-09-13T17:58:29.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.283+0000 7f8bc0d40700 1 -- 172.21.15.40:0/1526518296 wait complete. 2024-09-13T17:58:29.306 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:58:29.306 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T17:58:29.306 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:58:29.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.411+0000 7f6be3fff700 1 Processor -- start 2024-09-13T17:58:29.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.411+0000 7f6be3fff700 1 -- start start 2024-09-13T17:58:29.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be3fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6be4156ed0 0x7f6be415cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6be40666e0 0x7f6be415d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be3fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be40c82c0 con 0x7f6be4066300 2024-09-13T17:58:29.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be3fff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be40c8560 con 0x7f6be4156ed0 2024-09-13T17:58:29.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be3fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be406a530 con 0x7f6be40666e0 2024-09-13T17:58:29.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be37fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6be40666e0 0x7f6be415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be27fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6be4156ed0 0x7f6be415cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be37fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6be40666e0 0x7f6be415d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:46280/0 (socket says 172.21.15.40:46280) 2024-09-13T17:58:29.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be37fe700 1 -- 172.21.15.40:0/731953292 learned_addr learned my addr 172.21.15.40:0/731953292 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:29.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be37fe700 1 -- 172.21.15.40:0/731953292 >> v2:172.21.15.43:3300/0 conn(0x7f6be4156ed0 msgr2=0x7f6be415cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:29.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be37fe700 1 --2- 172.21.15.40:0/731953292 >> v2:172.21.15.43:3300/0 conn(0x7f6be4156ed0 0x7f6be415cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be37fe700 1 -- 172.21.15.40:0/731953292 >> v1:172.21.15.43:6789/0 conn(0x7f6be4066300 legacy=0x7f6be41567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be37fe700 1 -- 172.21.15.40:0/731953292 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be406a810 con 0x7f6be40666e0 2024-09-13T17:58:29.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be27fc700 1 --2- 172.21.15.40:0/731953292 >> v2:172.21.15.43:3300/0 conn(0x7f6be4156ed0 0x7f6be415cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:29.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be37fe700 1 --2- 172.21.15.40:0/731953292 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6be40666e0 0x7f6be415d060 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009e50 tx=0x7f6bdc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22183a8640f42301 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be1ffb700 1 -- 172.21.15.40:0/731953292 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc00e070 con 0x7f6be40666e0 2024-09-13T17:58:29.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be1ffb700 1 -- 172.21.15.40:0/731953292 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bdc00b910 con 0x7f6be40666e0 2024-09-13T17:58:29.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be1ffb700 1 -- 172.21.15.40:0/731953292 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc00bcc0 con 0x7f6be40666e0 2024-09-13T17:58:29.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be3fff700 1 -- 172.21.15.40:0/731953292 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6be40666e0 msgr2=0x7f6be415d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be3fff700 1 --2- 172.21.15.40:0/731953292 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6be40666e0 0x7f6be415d060 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009e50 tx=0x7f6bdc00b330 comp rx=0 tx=0).stop 2024-09-13T17:58:29.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be3fff700 1 -- 172.21.15.40:0/731953292 shutdown_connections 2024-09-13T17:58:29.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be3fff700 1 --2- 172.21.15.40:0/731953292 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6be40666e0 0x7f6be415d060 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be3fff700 1 --2- 172.21.15.40:0/731953292 >> v2:172.21.15.43:3300/0 conn(0x7f6be4156ed0 0x7f6be415cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be3fff700 1 -- 172.21.15.40:0/731953292 >> 172.21.15.40:0/731953292 conn(0x7f6be40cc9c0 msgr2=0x7f6be40c8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:29.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be3fff700 1 -- 172.21.15.40:0/731953292 shutdown_connections 2024-09-13T17:58:29.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.415+0000 7f6be3fff700 1 -- 172.21.15.40:0/731953292 wait complete. 2024-09-13T17:58:29.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be3fff700 1 Processor -- start 2024-09-13T17:58:29.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be3fff700 1 -- start start 2024-09-13T17:58:29.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be3fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be4066300 0x7f6be414beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6be40666e0 0x7f6be41e4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6be4156ed0 0x7f6be41efc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be3fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be41e1870 con 0x7f6be40666e0 2024-09-13T17:58:29.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be3fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be41e1b50 con 0x7f6be4066300 2024-09-13T17:58:29.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be3fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be41e1e30 con 0x7f6be4156ed0 2024-09-13T17:58:29.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be27fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6be40666e0 0x7f6be41e4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be2ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be4066300 0x7f6be414beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be37fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6be4156ed0 0x7f6be41efc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be2ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be4066300 0x7f6be414beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:32808/0 (socket says 172.21.15.40:32808) 2024-09-13T17:58:29.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be2ffd700 1 -- 172.21.15.40:0/1121339928 learned_addr learned my addr 172.21.15.40:0/1121339928 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:29.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be27fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6be40666e0 0x7f6be41e4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41552/0 (socket says 172.21.15.40:41552) 2024-09-13T17:58:29.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be37fe700 1 -- 172.21.15.40:0/1121339928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be4066300 msgr2=0x7f6be414beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be37fe700 1 --2- 172.21.15.40:0/1121339928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be4066300 0x7f6be414beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be37fe700 1 -- 172.21.15.40:0/1121339928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6be40666e0 msgr2=0x7f6be41e4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be37fe700 1 --2- 172.21.15.40:0/1121339928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6be40666e0 0x7f6be41e4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be37fe700 1 -- 172.21.15.40:0/1121339928 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd80094b0 con 0x7f6be4156ed0 2024-09-13T17:58:29.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be2ffd700 1 --2- 172.21.15.40:0/1121339928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be4066300 0x7f6be414beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:29.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be27fc700 1 --2- 172.21.15.40:0/1121339928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6be40666e0 0x7f6be41e4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:29.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.419+0000 7f6be37fe700 1 --2- 172.21.15.40:0/1121339928 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6be4156ed0 0x7f6be41efc40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc015000 tx=0x7f6bdc004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.423+0000 7f6bcb7fe700 1 -- 172.21.15.40:0/1121339928 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc004780 con 0x7f6be4156ed0 2024-09-13T17:58:29.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.423+0000 7f6bcb7fe700 1 -- 172.21.15.40:0/1121339928 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bdc0092e0 con 0x7f6be4156ed0 2024-09-13T17:58:29.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.423+0000 7f6be3fff700 1 -- 172.21.15.40:0/1121339928 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bdc009b30 con 0x7f6be4156ed0 2024-09-13T17:58:29.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.423+0000 7f6bcb7fe700 1 -- 172.21.15.40:0/1121339928 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc01c440 con 0x7f6be4156ed0 2024-09-13T17:58:29.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.423+0000 7f6be3fff700 1 -- 172.21.15.40:0/1121339928 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be41ef720 con 0x7f6be4156ed0 2024-09-13T17:58:29.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.423+0000 7f6be3fff700 1 --2- 172.21.15.40:0/1121339928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be41e0040 0x7f6be41f0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.423+0000 7f6be3fff700 1 -- 172.21.15.40:0/1121339928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be4067520 con 0x7f6be41e0040 2024-09-13T17:58:29.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.423+0000 7f6be2ffd700 1 --2- 172.21.15.40:0/1121339928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be41e0040 0x7f6be41f0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.423+0000 7f6be2ffd700 1 -- 172.21.15.40:0/1121339928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6be40c6870 con 0x7f6be41e0040 2024-09-13T17:58:29.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.423+0000 7f6be2ffd700 1 --2- 172.21.15.40:0/1121339928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be41e0040 0x7f6be41f0180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400fae0 tx=0x7f6bd400fea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.423+0000 7f6bcb7fe700 1 -- 172.21.15.40:0/1121339928 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4017070 con 0x7f6be41e0040 2024-09-13T17:58:29.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.423+0000 7f6bcb7fe700 1 -- 172.21.15.40:0/1121339928 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6bdc01c5a0 con 0x7f6be4156ed0 2024-09-13T17:58:29.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.423+0000 7f6bcb7fe700 1 --2- 172.21.15.40:0/1121339928 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6bd00667d0 0x7f6bd0068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.423+0000 7f6be27fc700 1 --2- 172.21.15.40:0/1121339928 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6bd00667d0 0x7f6bd0068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.423+0000 7f6bcb7fe700 1 -- 172.21.15.40:0/1121339928 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6bdc085980 con 0x7f6be4156ed0 2024-09-13T17:58:29.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.427+0000 7f6bcb7fe700 1 -- 172.21.15.40:0/1121339928 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6bd40052e0 con 0x7f6be41e0040 2024-09-13T17:58:29.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.427+0000 7f6bcb7fe700 1 -- 172.21.15.40:0/1121339928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be41e0040 msgr2=0x7f6be41f0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.427+0000 7f6bcb7fe700 1 --2- 172.21.15.40:0/1121339928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be41e0040 0x7f6be41f0180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400fae0 tx=0x7f6bd400fea0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.427+0000 7f6bcb7fe700 1 -- 172.21.15.40:0/1121339928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be41e0040 msgr2=0x7f6be41f0180 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:58:29.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.427+0000 7f6bcb7fe700 1 --2- 172.21.15.40:0/1121339928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be41e0040 0x7f6be41f0180 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.427+0000 7f6be27fc700 1 --2- 172.21.15.40:0/1121339928 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6bd00667d0 0x7f6bd0068c90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6be4066f80 tx=0x7f6bd8009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.439+0000 7f6be3fff700 1 --2- 172.21.15.40:0/1121339928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be41e4550 0x7f6be41464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.439+0000 7f6be3fff700 1 -- 172.21.15.40:0/1121339928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be41e1fc0 con 0x7f6be41e4550 2024-09-13T17:58:29.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.439+0000 7f6be2ffd700 1 --2- 172.21.15.40:0/1121339928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be41e4550 0x7f6be41464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.439+0000 7f6be2ffd700 1 -- 172.21.15.40:0/1121339928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6be406dfa0 con 0x7f6be41e4550 2024-09-13T17:58:29.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.439+0000 7f6be2ffd700 1 --2- 172.21.15.40:0/1121339928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be41e4550 0x7f6be41464a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400fae0 tx=0x7f6bd4001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.439+0000 7f6bcb7fe700 1 -- 172.21.15.40:0/1121339928 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4023070 con 0x7f6be41e4550 2024-09-13T17:58:29.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.439+0000 7f6bcb7fe700 1 -- 172.21.15.40:0/1121339928 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7f6bd401edc0 con 0x7f6be41e4550 2024-09-13T17:58:29.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.439+0000 7f6bcb7fe700 1 -- 172.21.15.40:0/1121339928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be41e4550 msgr2=0x7f6be41464a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.439+0000 7f6bcb7fe700 1 --2- 172.21.15.40:0/1121339928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be41e4550 0x7f6be41464a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400fae0 tx=0x7f6bd4001120 comp rx=0 tx=0).stop 2024-09-13T17:58:29.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.439+0000 7f6bcb7fe700 1 -- 172.21.15.40:0/1121339928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be41e4550 msgr2=0x7f6be41464a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:58:29.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.439+0000 7f6bcb7fe700 1 --2- 172.21.15.40:0/1121339928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6be41e4550 0x7f6be41464a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.450 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:58:29.450 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:58:29.450 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:58:29.451 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:58:29.451 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 62, 2024-09-13T17:58:29.451 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:58:29.451 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:58:29.451 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:58:29.451 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:58:29.451 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:29.451 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 12, 2024-09-13T17:58:29.451 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:58:29.451 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:58:29.451 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:58:29.451 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:29.451 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:29.451 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:29.451 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:29.452 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:29.452 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:29.452 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:29.452 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:29.452 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:29.452 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:29.452 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:29.452 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:58:29.452 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:58:29.452 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:29.452 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:29.452 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:29.452 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:29.452 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:29.453 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:29.453 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:29.453 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:29.453 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:29.453 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:29.453 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:29.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.453 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:58:29.453 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:58:29.453 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:58:29.453 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:58:29.453 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:58:29.453 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:58:29.453 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:58:29.454 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:58:29.454 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:58:29.454 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:58:29.454 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:58:29.454 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:58:29.454 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:58:29.454 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:58:29.454 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:58:29.454 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:58:29.454 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:58:29.454 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:29.454 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:29.454 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:29.454 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:29.455 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:29.455 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:29.455 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:29.455 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:29.455 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:29.455 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:29.455 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:29.455 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:58:29.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.455 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:58:29.455 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.455 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:58:29.455 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:58:29.455 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:29.455 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:29.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.456 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:29.456 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:58:29.456 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:29.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.456 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:29.456 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:58:29.456 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:29.456 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.456 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:29.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.456 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:58:29.456 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:58:29.457 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:29.457 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:29.457 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:29.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.457 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:58:29.457 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:58:29.457 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:29.457 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:29.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.457 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:29.457 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:58:29.457 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:29.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.458 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:29.458 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:58:29.458 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:29.458 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.458 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:29.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.458 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:58:29.458 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:58:29.458 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:29.458 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:29.458 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:29.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.458 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:58:29.459 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:58:29.459 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:29.459 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:29.459 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.459 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:29.459 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:58:29.459 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:29.459 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.459 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.459 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:29.459 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:58:29.459 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:29.459 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.459 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:29.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.460 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:58:29.460 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:58:29.460 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:29.460 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:29.460 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:29.460 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.460 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:29.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.460 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:58:29.460 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:58:29.460 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.460 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:29.460 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:29.460 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:29.461 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.461 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:29.461 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:58:29.461 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.461 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:29.461 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:29.461 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:29.461 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.461 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:29.461 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:58:29.461 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.461 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:29.461 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:29.461 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:29.462 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.462 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:29.462 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:58:29.462 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.462 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:29.462 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:29.462 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:29.462 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.462 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:29.462 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:58:29.462 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.462 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:29.462 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:29.462 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:29.463 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.463 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:29.463 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.463 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:58:29.463 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:58:29.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.443+0000 7f6be3fff700 1 -- 172.21.15.40:0/1121339928 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6bd00667d0 msgr2=0x7f6bd0068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.443+0000 7f6be3fff700 1 --2- 172.21.15.40:0/1121339928 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6bd00667d0 0x7f6bd0068c90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6be4066f80 tx=0x7f6bd8009240 comp rx=0 tx=0).stop 2024-09-13T17:58:29.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.443+0000 7f6be3fff700 1 -- 172.21.15.40:0/1121339928 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6be4156ed0 msgr2=0x7f6be41efc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.443+0000 7f6be3fff700 1 --2- 172.21.15.40:0/1121339928 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6be4156ed0 0x7f6be41efc40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc015000 tx=0x7f6bdc004af0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.443+0000 7f6be37fe700 1 -- 172.21.15.40:0/1121339928 reap_dead start 2024-09-13T17:58:29.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.443+0000 7f6be37fe700 1 -- 172.21.15.40:0/1121339928 reap_dead start 2024-09-13T17:58:29.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.443+0000 7f6be3fff700 1 -- 172.21.15.40:0/1121339928 shutdown_connections 2024-09-13T17:58:29.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.443+0000 7f6be3fff700 1 -- 172.21.15.40:0/1121339928 >> 172.21.15.40:0/1121339928 conn(0x7f6be40cc9c0 msgr2=0x7f6be41558e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:29.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.443+0000 7f6be3fff700 1 -- 172.21.15.40:0/1121339928 shutdown_connections 2024-09-13T17:58:29.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.443+0000 7f6be3fff700 1 -- 172.21.15.40:0/1121339928 wait complete. 2024-09-13T17:58:29.466 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:58:29.466 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T17:58:29.466 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:58:29.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.575+0000 7fd9ad7d4700 1 Processor -- start 2024-09-13T17:58:29.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.575+0000 7fd9ad7d4700 1 -- start start 2024-09-13T17:58:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.575+0000 7fd9ad7d4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a8150900 0x7fd9a806e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.575+0000 7fd9ad7d4700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9a8066460 con 0x7fd9a8150520 2024-09-13T17:58:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.575+0000 7fd9ad7d4700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9a8066130 con 0x7fd9a8071180 2024-09-13T17:58:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.575+0000 7fd9ad7d4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80c3830 con 0x7fd9a8150900 2024-09-13T17:58:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.575+0000 7fd9a7fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd9a8150520 0x7fd9a80c4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56686/0 (socket says 172.21.15.40:56686) 2024-09-13T17:58:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.575+0000 7fd9a7fff700 1 -- 172.21.15.40:0/2679749144 learned_addr learned my addr 172.21.15.40:0/2679749144 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:58:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.575+0000 7fd9a77fe700 1 --2- 172.21.15.40:0/2679749144 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a8150900 0x7fd9a806e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.575+0000 7fd9a6ffd700 1 -- 172.21.15.40:0/2679749144 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2835041761 0 0) 0x7fd9a8066460 con 0x7fd9a8150520 2024-09-13T17:58:29.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.575+0000 7fd9a6ffd700 1 -- 172.21.15.40:0/2679749144 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9900035d0 con 0x7fd9a8150520 2024-09-13T17:58:29.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.575+0000 7fd9a6ffd700 1 -- 172.21.15.40:0/2679749144 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408149833 0 0) 0x7fd9a8066130 con 0x7fd9a8071180 2024-09-13T17:58:29.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.575+0000 7fd9a6ffd700 1 -- 172.21.15.40:0/2679749144 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9a8066460 con 0x7fd9a8071180 2024-09-13T17:58:29.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.575+0000 7fd9a77fe700 1 -- 172.21.15.40:0/2679749144 >> v1:172.21.15.43:6789/0 conn(0x7fd9a8071180 legacy=0x7fd9a8156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.575+0000 7fd9a77fe700 1 -- 172.21.15.40:0/2679749144 >> v1:172.21.15.40:6789/0 conn(0x7fd9a8150520 legacy=0x7fd9a80c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.575+0000 7fd9a77fe700 1 -- 172.21.15.40:0/2679749144 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a8070e80 con 0x7fd9a8150900 2024-09-13T17:58:29.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.575+0000 7fd9a77fe700 1 --2- 172.21.15.40:0/2679749144 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a8150900 0x7fd9a806e860 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0098b0 tx=0x7fd99c009bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a7251f5a4fcefe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.579+0000 7fd9a6ffd700 1 -- 172.21.15.40:0/2679749144 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c011830 con 0x7fd9a8150900 2024-09-13T17:58:29.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.579+0000 7fd9a6ffd700 1 -- 172.21.15.40:0/2679749144 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99c011990 con 0x7fd9a8150900 2024-09-13T17:58:29.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.579+0000 7fd9a6ffd700 1 -- 172.21.15.40:0/2679749144 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c011d20 con 0x7fd9a8150900 2024-09-13T17:58:29.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.579+0000 7fd9ad7d4700 1 -- 172.21.15.40:0/2679749144 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a8150900 msgr2=0x7fd9a806e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.579+0000 7fd9ad7d4700 1 --2- 172.21.15.40:0/2679749144 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a8150900 0x7fd9a806e860 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0098b0 tx=0x7fd99c009bc0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.579+0000 7fd9ad7d4700 1 -- 172.21.15.40:0/2679749144 shutdown_connections 2024-09-13T17:58:29.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.579+0000 7fd9ad7d4700 1 --2- 172.21.15.40:0/2679749144 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a8150900 0x7fd9a806e860 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.579+0000 7fd9ad7d4700 1 -- 172.21.15.40:0/2679749144 >> 172.21.15.40:0/2679749144 conn(0x7fd9a80cb450 msgr2=0x7fd9a80cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:29.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.579+0000 7fd9ad7d4700 1 -- 172.21.15.40:0/2679749144 shutdown_connections 2024-09-13T17:58:29.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.579+0000 7fd9ad7d4700 1 -- 172.21.15.40:0/2679749144 wait complete. 2024-09-13T17:58:29.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.579+0000 7fd9ad7d4700 1 Processor -- start 2024-09-13T17:58:29.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.579+0000 7fd9ad7d4700 1 -- start start 2024-09-13T17:58:29.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9ad7d4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9a8071180 0x7fd9a81420d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9ad7d4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a8150520 0x7fd9a814e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9ad7d4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a80cabb0 0x7fd9a814e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9ad7d4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a814dd40 con 0x7fd9a8071180 2024-09-13T17:58:29.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9ad7d4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a8147660 con 0x7fd9a8150520 2024-09-13T17:58:29.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9ad7d4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a81574d0 con 0x7fd9a80cabb0 2024-09-13T17:58:29.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9a8071180 0x7fd9a81420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9a8071180 0x7fd9a81420d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41560/0 (socket says 172.21.15.40:41560) 2024-09-13T17:58:29.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9a7fff700 1 -- 172.21.15.40:0/3909177359 learned_addr learned my addr 172.21.15.40:0/3909177359 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:29.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9a77fe700 1 --2- 172.21.15.40:0/3909177359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a8150520 0x7fd9a814e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9acfd3700 1 --2- 172.21.15.40:0/3909177359 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a80cabb0 0x7fd9a814e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9a7fff700 1 -- 172.21.15.40:0/3909177359 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a80cabb0 msgr2=0x7fd9a814e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9a7fff700 1 --2- 172.21.15.40:0/3909177359 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a80cabb0 0x7fd9a814e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9a7fff700 1 -- 172.21.15.40:0/3909177359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a8150520 msgr2=0x7fd9a814e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9a7fff700 1 --2- 172.21.15.40:0/3909177359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a8150520 0x7fd9a814e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9a7fff700 1 -- 172.21.15.40:0/3909177359 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c0094b0 con 0x7fd9a8071180 2024-09-13T17:58:29.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9a7fff700 1 --2- 172.21.15.40:0/3909177359 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9a8071180 0x7fd9a81420d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd99800c590 tx=0x7fd99800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9a4ff9700 1 -- 172.21.15.40:0/3909177359 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd998005d90 con 0x7fd9a8071180 2024-09-13T17:58:29.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9a4ff9700 1 -- 172.21.15.40:0/3909177359 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9980070d0 con 0x7fd9a8071180 2024-09-13T17:58:29.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9ad7d4700 1 -- 172.21.15.40:0/3909177359 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a8149b50 con 0x7fd9a8071180 2024-09-13T17:58:29.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9a4ff9700 1 -- 172.21.15.40:0/3909177359 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd998007440 con 0x7fd9a8071180 2024-09-13T17:58:29.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9ad7d4700 1 -- 172.21.15.40:0/3909177359 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a80ca100 con 0x7fd9a8071180 2024-09-13T17:58:29.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9ad7d4700 1 --2- 172.21.15.40:0/3909177359 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a8150900 0x7fd9a80ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9ad7d4700 1 -- 172.21.15.40:0/3909177359 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80675d0 con 0x7fd9a8150900 2024-09-13T17:58:29.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.583+0000 7fd9a77fe700 1 --2- 172.21.15.40:0/3909177359 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a8150900 0x7fd9a80ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.587+0000 7fd9a77fe700 1 -- 172.21.15.40:0/3909177359 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9a80c2ae0 con 0x7fd9a8150900 2024-09-13T17:58:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.587+0000 7fd9a4ff9700 1 -- 172.21.15.40:0/3909177359 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd9980075a0 con 0x7fd9a8071180 2024-09-13T17:58:29.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.587+0000 7fd9a77fe700 1 --2- 172.21.15.40:0/3909177359 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a8150900 0x7fd9a80ca290 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80c3d30 tx=0x7fd99c0047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.587+0000 7fd9a4ff9700 1 --2- 172.21.15.40:0/3909177359 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd994066730 0x7fd994068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.587+0000 7fd9acfd3700 1 --2- 172.21.15.40:0/3909177359 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd994066730 0x7fd994068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.587+0000 7fd9a4ff9700 1 -- 172.21.15.40:0/3909177359 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd998083a00 con 0x7fd9a8071180 2024-09-13T17:58:29.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.587+0000 7fd9a4ff9700 1 -- 172.21.15.40:0/3909177359 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c01a6b0 con 0x7fd9a8150900 2024-09-13T17:58:29.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.587+0000 7fd9a4ff9700 1 -- 172.21.15.40:0/3909177359 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd99c01a810 con 0x7fd9a8150900 2024-09-13T17:58:29.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.587+0000 7fd9a4ff9700 1 -- 172.21.15.40:0/3909177359 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a8150900 msgr2=0x7fd9a80ca290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.587+0000 7fd9a4ff9700 1 --2- 172.21.15.40:0/3909177359 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a8150900 0x7fd9a80ca290 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80c3d30 tx=0x7fd99c0047e0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.587+0000 7fd9a4ff9700 1 -- 172.21.15.40:0/3909177359 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a8150900 msgr2=0x7fd9a80ca290 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:58:29.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.587+0000 7fd9a4ff9700 1 --2- 172.21.15.40:0/3909177359 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a8150900 0x7fd9a80ca290 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.587+0000 7fd9acfd3700 1 --2- 172.21.15.40:0/3909177359 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd994066730 0x7fd994068bf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0001560 tx=0x7fd9a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9ad7d4700 1 --2- 172.21.15.40:0/3909177359 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a813f980 0x7fd9a813fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9ad7d4700 1 -- 172.21.15.40:0/3909177359 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a8147300 con 0x7fd9a813f980 2024-09-13T17:58:29.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9a77fe700 1 --2- 172.21.15.40:0/3909177359 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a813f980 0x7fd9a813fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9a77fe700 1 -- 172.21.15.40:0/3909177359 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd9a8070490 con 0x7fd9a813f980 2024-09-13T17:58:29.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9a77fe700 1 --2- 172.21.15.40:0/3909177359 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a813f980 0x7fd9a813fd90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0097d0 tx=0x7fd99c004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9a4ff9700 1 -- 172.21.15.40:0/3909177359 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c003e60 con 0x7fd9a813f980 2024-09-13T17:58:29.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9a4ff9700 1 -- 172.21.15.40:0/3909177359 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7fd99c004590 con 0x7fd9a813f980 2024-09-13T17:58:29.609 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:58:29.609 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:58:29.609 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:58:29.609 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:58:29.609 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 62, 2024-09-13T17:58:29.609 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:58:29.609 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:58:29.609 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:58:29.609 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:58:29.609 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:29.609 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 12, 2024-09-13T17:58:29.609 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:58:29.610 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:58:29.610 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:58:29.610 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:29.610 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:29.610 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:29.610 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:29.610 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:29.610 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:29.610 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:29.610 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:29.610 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:29.610 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:29.610 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:29.610 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:58:29.611 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:58:29.611 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:29.611 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:29.611 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:29.611 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:29.611 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:29.611 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:29.611 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:29.611 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:29.612 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:29.612 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:29.612 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:29.612 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.612 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:58:29.612 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:58:29.612 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:58:29.612 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:58:29.613 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:58:29.613 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:58:29.613 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:58:29.613 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:58:29.613 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:58:29.613 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:58:29.613 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:58:29.613 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:58:29.613 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:58:29.613 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:58:29.613 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:58:29.613 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:58:29.613 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:58:29.614 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:29.614 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:29.614 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:29.614 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:29.614 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:29.614 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:29.614 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:29.614 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:29.614 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:29.614 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:29.614 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:29.614 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:58:29.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.615 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:58:29.615 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.615 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:58:29.615 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:58:29.615 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:29.615 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:29.615 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.615 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:29.615 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:58:29.615 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:29.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.615 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.615 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:29.615 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:58:29.616 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:29.616 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.616 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:29.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.616 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:58:29.616 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:58:29.616 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:29.616 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:29.616 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:29.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.616 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:58:29.616 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:58:29.617 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:29.617 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:29.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.617 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:29.617 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:58:29.617 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:29.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.617 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:29.617 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:58:29.617 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:29.617 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.617 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:29.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.618 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:58:29.618 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:58:29.618 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:29.618 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:29.618 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:29.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.618 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:58:29.618 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:58:29.618 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:29.618 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:29.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.619 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:29.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:58:29.619 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:29.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.619 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:29.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:58:29.619 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:29.619 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.619 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:29.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:58:29.620 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:58:29.620 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:29.620 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:29.620 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:29.620 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.620 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:29.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.620 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:58:29.620 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:58:29.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.620 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:29.620 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:29.620 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:29.621 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.621 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:29.621 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:58:29.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.621 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:29.621 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:29.621 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:58:29.621 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.621 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:29.621 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:58:29.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.621 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:29.621 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:29.622 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T17:58:29.622 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.622 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:29.622 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:58:29.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.622 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:29.622 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:29.622 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:58:29.622 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.622 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:29.622 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:58:29.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:29.622 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:29.623 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:29.623 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:29.623 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:29.623 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:29.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:29.623 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:58:29.623 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:58:29.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9a4ff9700 1 -- 172.21.15.40:0/3909177359 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a813f980 msgr2=0x7fd9a813fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9a4ff9700 1 --2- 172.21.15.40:0/3909177359 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a813f980 0x7fd9a813fd90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0097d0 tx=0x7fd99c004770 comp rx=0 tx=0).stop 2024-09-13T17:58:29.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9a4ff9700 1 -- 172.21.15.40:0/3909177359 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a813f980 msgr2=0x7fd9a813fd90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:58:29.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9a4ff9700 1 --2- 172.21.15.40:0/3909177359 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9a813f980 0x7fd9a813fd90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9ad7d4700 1 -- 172.21.15.40:0/3909177359 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd994066730 msgr2=0x7fd994068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9ad7d4700 1 --2- 172.21.15.40:0/3909177359 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd994066730 0x7fd994068bf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0001560 tx=0x7fd9a0009040 comp rx=0 tx=0).stop 2024-09-13T17:58:29.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9ad7d4700 1 -- 172.21.15.40:0/3909177359 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9a8071180 msgr2=0x7fd9a81420d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9ad7d4700 1 --2- 172.21.15.40:0/3909177359 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9a8071180 0x7fd9a81420d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd99800c590 tx=0x7fd99800c950 comp rx=0 tx=0).stop 2024-09-13T17:58:29.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9acfd3700 1 -- 172.21.15.40:0/3909177359 reap_dead start 2024-09-13T17:58:29.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9acfd3700 1 -- 172.21.15.40:0/3909177359 reap_dead start 2024-09-13T17:58:29.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9ad7d4700 1 -- 172.21.15.40:0/3909177359 shutdown_connections 2024-09-13T17:58:29.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9ad7d4700 1 -- 172.21.15.40:0/3909177359 >> 172.21.15.40:0/3909177359 conn(0x7fd9a80cb450 msgr2=0x7fd9a8142f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:29.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9ad7d4700 1 -- 172.21.15.40:0/3909177359 shutdown_connections 2024-09-13T17:58:29.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.599+0000 7fd9ad7d4700 1 -- 172.21.15.40:0/3909177359 wait complete. 2024-09-13T17:58:29.626 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:58:29.626 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-13T17:58:29.626 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T17:58:29.626 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-13T17:58:29.626 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T17:58:29.626 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T17:58:29.627 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:58:29.670 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T17:58:29.670 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T17:58:29.670 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:58:29.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.787+0000 7f108b59e700 1 Processor -- start 2024-09-13T17:58:29.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f108b59e700 1 -- start start 2024-09-13T17:58:29.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f108b59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f108c0c4210 0x7f108c0c4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f108b59e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f108c14fad0 0x7f108c14ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f108a59c700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f108c159840 legacy=0x7f108c159c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:58:29.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f108b59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108c0663d0 con 0x7f108c159840 2024-09-13T17:58:29.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f108b59e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108c070d40 con 0x7f108c14fad0 2024-09-13T17:58:29.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f108b59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c15a330 con 0x7f108c0c4210 2024-09-13T17:58:29.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f108ad9d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f108c14fad0 0x7f108c14ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f1089d9b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f108c0c4210 0x7f108c0c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f1089d9b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f108c0c4210 0x7f108c0c4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:46314/0 (socket says 172.21.15.40:46314) 2024-09-13T17:58:29.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f108ad9d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f108c14fad0 0x7f108c14ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:32838/0 (socket says 172.21.15.40:32838) 2024-09-13T17:58:29.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f1089d9b700 1 -- 172.21.15.40:0/3794308511 learned_addr learned my addr 172.21.15.40:0/3794308511 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:29.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f1089d9b700 1 -- 172.21.15.40:0/3794308511 >> v2:172.21.15.43:3300/0 conn(0x7f108c14fad0 msgr2=0x7f108c14ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:29.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f1089d9b700 1 --2- 172.21.15.40:0/3794308511 >> v2:172.21.15.43:3300/0 conn(0x7f108c14fad0 0x7f108c14ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f1089d9b700 1 -- 172.21.15.40:0/3794308511 >> v1:172.21.15.40:6789/0 conn(0x7f108c159840 legacy=0x7f108c159c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:58:29.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f1089d9b700 1 -- 172.21.15.40:0/3794308511 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c070fe0 con 0x7f108c0c4210 2024-09-13T17:58:29.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f108ad9d700 1 --2- 172.21.15.40:0/3794308511 >> v2:172.21.15.43:3300/0 conn(0x7f108c14fad0 0x7f108c14ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:29.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f1089d9b700 1 --2- 172.21.15.40:0/3794308511 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f108c0c4210 0x7f108c0c4610 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1080009910 tx=0x7f1080009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b711e6e91458c353 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f108959a700 1 -- 172.21.15.40:0/3794308511 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1080004230 con 0x7f108c0c4210 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f108959a700 1 -- 172.21.15.40:0/3794308511 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1080004390 con 0x7f108c0c4210 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.791+0000 7f108959a700 1 -- 172.21.15.40:0/3794308511 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10800036c0 con 0x7f108c0c4210 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.795+0000 7f108b59e700 1 -- 172.21.15.40:0/3794308511 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f108c0c4210 msgr2=0x7f108c0c4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.795+0000 7f108b59e700 1 --2- 172.21.15.40:0/3794308511 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f108c0c4210 0x7f108c0c4610 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1080009910 tx=0x7f1080009c20 comp rx=0 tx=0).stop 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.795+0000 7f108b59e700 1 -- 172.21.15.40:0/3794308511 shutdown_connections 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.795+0000 7f108b59e700 1 --2- 172.21.15.40:0/3794308511 >> v2:172.21.15.43:3300/0 conn(0x7f108c14fad0 0x7f108c14ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.795+0000 7f108b59e700 1 --2- 172.21.15.40:0/3794308511 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f108c0c4210 0x7f108c0c4610 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.795+0000 7f108b59e700 1 -- 172.21.15.40:0/3794308511 >> 172.21.15.40:0/3794308511 conn(0x7f108c06a390 msgr2=0x7f108c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.795+0000 7f108b59e700 1 -- 172.21.15.40:0/3794308511 shutdown_connections 2024-09-13T17:58:29.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.795+0000 7f108b59e700 1 -- 172.21.15.40:0/3794308511 wait complete. 2024-09-13T17:58:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.795+0000 7f108b59e700 1 Processor -- start 2024-09-13T17:58:29.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.795+0000 7f108b59e700 1 -- start start 2024-09-13T17:58:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108b59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f108c0c4210 0x7f108c1ef860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108b59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f108c14fad0 0x7f108c1dffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108b59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f108c159840 0x7f108c1e04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108b59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c1efda0 con 0x7f108c14fad0 2024-09-13T17:58:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108b59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c14ca70 con 0x7f108c159840 2024-09-13T17:58:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108b59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c14cd10 con 0x7f108c0c4210 2024-09-13T17:58:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f1089d9b700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f108c14fad0 msgr2=0x7f108c1dffb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:58:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f1089d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f108c14fad0 0x7f108c1dffb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108ad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f108c159840 0x7f108c1e04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108a59c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f108c0c4210 0x7f108c1ef860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108ad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f108c159840 0x7f108c1e04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:32840/0 (socket says 172.21.15.40:32840) 2024-09-13T17:58:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108a59c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f108c0c4210 0x7f108c1ef860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:46316/0 (socket says 172.21.15.40:46316) 2024-09-13T17:58:29.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108ad9d700 1 -- 172.21.15.40:0/2016681474 learned_addr learned my addr 172.21.15.40:0/2016681474 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108ad9d700 1 -- 172.21.15.40:0/2016681474 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f108c0c4210 msgr2=0x7f108c1ef860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108ad9d700 1 --2- 172.21.15.40:0/2016681474 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f108c0c4210 0x7f108c1ef860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108ad9d700 1 -- 172.21.15.40:0/2016681474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f108c14fad0 msgr2=0x7f108c1dffb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:58:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108ad9d700 1 --2- 172.21.15.40:0/2016681474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f108c14fad0 0x7f108c1dffb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108ad9d700 1 -- 172.21.15.40:0/2016681474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1084009a70 con 0x7f108c159840 2024-09-13T17:58:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108a59c700 1 --2- 172.21.15.40:0/2016681474 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f108c0c4210 0x7f108c1ef860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108ad9d700 1 --2- 172.21.15.40:0/2016681474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f108c159840 0x7f108c1e04f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f108400efc0 tx=0x7f108400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f107affd700 1 -- 172.21.15.40:0/2016681474 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108400cb40 con 0x7f108c159840 2024-09-13T17:58:29.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f107affd700 1 -- 172.21.15.40:0/2016681474 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108400f040 con 0x7f108c159840 2024-09-13T17:58:29.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108b59e700 1 -- 172.21.15.40:0/2016681474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1080009570 con 0x7f108c159840 2024-09-13T17:58:29.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108b59e700 1 -- 172.21.15.40:0/2016681474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f108c1df4d0 con 0x7f108c159840 2024-09-13T17:58:29.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f107affd700 1 -- 172.21.15.40:0/2016681474 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1084003a40 con 0x7f108c159840 2024-09-13T17:58:29.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.799+0000 7f108b59e700 1 -- 172.21.15.40:0/2016681474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1070005310 con 0x7f108c159840 2024-09-13T17:58:29.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.803+0000 7f107affd700 1 -- 172.21.15.40:0/2016681474 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f108403b480 con 0x7f108c159840 2024-09-13T17:58:29.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.803+0000 7f107affd700 1 --2- 172.21.15.40:0/2016681474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f106c066710 0x7f106c068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:29.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.803+0000 7f108a59c700 1 --2- 172.21.15.40:0/2016681474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f106c066710 0x7f106c068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:29.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.803+0000 7f107affd700 1 -- 172.21.15.40:0/2016681474 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1084085420 con 0x7f108c159840 2024-09-13T17:58:29.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.803+0000 7f108a59c700 1 --2- 172.21.15.40:0/2016681474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f106c066710 0x7f106c068bd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f107c003a30 tx=0x7f107c003c40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:29.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:29.811+0000 7f107affd700 1 -- 172.21.15.40:0/2016681474 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1084052540 con 0x7f108c159840 2024-09-13T17:58:30.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:30.079+0000 7f108b59e700 1 -- 172.21.15.40:0/2016681474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1070005890 con 0x7f108c159840 2024-09-13T17:58:30.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:30.079+0000 7f107affd700 1 -- 172.21.15.40:0/2016681474 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f1084055b60 con 0x7f108c159840 2024-09-13T17:58:30.089 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":62,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:58:30.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:30.083+0000 7f108b59e700 1 -- 172.21.15.40:0/2016681474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f106c066710 msgr2=0x7f106c068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:30.083+0000 7f108b59e700 1 --2- 172.21.15.40:0/2016681474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f106c066710 0x7f106c068bd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f107c003a30 tx=0x7f107c003c40 comp rx=0 tx=0).stop 2024-09-13T17:58:30.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:30.083+0000 7f108b59e700 1 -- 172.21.15.40:0/2016681474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f108c159840 msgr2=0x7f108c1e04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:30.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:30.083+0000 7f108b59e700 1 --2- 172.21.15.40:0/2016681474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f108c159840 0x7f108c1e04f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f108400efc0 tx=0x7f108400c3a0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:30.083+0000 7f108b59e700 1 -- 172.21.15.40:0/2016681474 shutdown_connections 2024-09-13T17:58:30.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:30.083+0000 7f108b59e700 1 --2- 172.21.15.40:0/2016681474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f108c159840 0x7f108c1e04f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:30.083+0000 7f108b59e700 1 --2- 172.21.15.40:0/2016681474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f108c14fad0 0x7f108c1dffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:30.083+0000 7f108b59e700 1 --2- 172.21.15.40:0/2016681474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f106c066710 0x7f106c068bd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:30.083+0000 7f108b59e700 1 --2- 172.21.15.40:0/2016681474 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f108c0c4210 0x7f108c1ef860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:30.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:30.083+0000 7f108b59e700 1 -- 172.21.15.40:0/2016681474 >> 172.21.15.40:0/2016681474 conn(0x7f108c06a390 msgr2=0x7f108c14ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:30.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:30.083+0000 7f108b59e700 1 -- 172.21.15.40:0/2016681474 shutdown_connections 2024-09-13T17:58:30.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:30.083+0000 7f108b59e700 1 -- 172.21.15.40:0/2016681474 wait complete. 2024-09-13T17:58:33.105 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:58:33.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.219+0000 7f159f293700 1 Processor -- start 2024-09-13T17:58:33.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.219+0000 7f159f293700 1 -- start start 2024-09-13T17:58:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.219+0000 7f159f293700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1598153840 0x7f15980c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.219+0000 7f159f293700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f15980c9e20 0x7f15980cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.219+0000 7f159f293700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15980cc060 con 0x7f15980cc4f0 2024-09-13T17:58:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.219+0000 7f159f293700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15980b2840 con 0x7f15980c9e20 2024-09-13T17:58:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.219+0000 7f159f293700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15980ca870 con 0x7f1598153840 2024-09-13T17:58:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.219+0000 7f159da90700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f15980cc4f0 legacy=0x7f159815d410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:58:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.219+0000 7f159ea92700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f15980c9e20 msgr2=0x7f15980cf260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:58:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.219+0000 7f159ea92700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f15980c9e20 0x7f15980cf260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.219+0000 7f159e291700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1598153840 0x7f15980c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.219+0000 7f159e291700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1598153840 0x7f15980c3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:39978/0 (socket says 172.21.15.40:39978) 2024-09-13T17:58:33.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.219+0000 7f159e291700 1 -- 172.21.15.40:0/1108676463 learned_addr learned my addr 172.21.15.40:0/1108676463 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:33.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.223+0000 7f159e291700 1 -- 172.21.15.40:0/1108676463 >> v2:172.21.15.40:3300/0 conn(0x7f15980c9e20 msgr2=0x7f15980cf260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:58:33.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.223+0000 7f159e291700 1 --2- 172.21.15.40:0/1108676463 >> v2:172.21.15.40:3300/0 conn(0x7f15980c9e20 0x7f15980cf260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:33.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.223+0000 7f159e291700 1 -- 172.21.15.40:0/1108676463 >> v1:172.21.15.40:6789/0 conn(0x7f15980cc4f0 legacy=0x7f159815d410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:58:33.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.223+0000 7f159e291700 1 -- 172.21.15.40:0/1108676463 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1598153c20 con 0x7f1598153840 2024-09-13T17:58:33.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.223+0000 7f159e291700 1 --2- 172.21.15.40:0/1108676463 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1598153840 0x7f15980c3eb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1588009e50 tx=0x7f158800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7067cc969bb3e28c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:33.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.223+0000 7f159d28f700 1 -- 172.21.15.40:0/1108676463 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158800e070 con 0x7f1598153840 2024-09-13T17:58:33.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.223+0000 7f159d28f700 1 -- 172.21.15.40:0/1108676463 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1588003a40 con 0x7f1598153840 2024-09-13T17:58:33.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.223+0000 7f159d28f700 1 -- 172.21.15.40:0/1108676463 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588003df0 con 0x7f1598153840 2024-09-13T17:58:33.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.223+0000 7f159f293700 1 -- 172.21.15.40:0/1108676463 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1598153840 msgr2=0x7f15980c3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:33.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.223+0000 7f159f293700 1 --2- 172.21.15.40:0/1108676463 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1598153840 0x7f15980c3eb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1588009e50 tx=0x7f158800b330 comp rx=0 tx=0).stop 2024-09-13T17:58:33.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.223+0000 7f159f293700 1 -- 172.21.15.40:0/1108676463 shutdown_connections 2024-09-13T17:58:33.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.223+0000 7f159f293700 1 --2- 172.21.15.40:0/1108676463 >> v2:172.21.15.40:3300/0 conn(0x7f15980c9e20 0x7f15980cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:33.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.223+0000 7f159f293700 1 --2- 172.21.15.40:0/1108676463 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1598153840 0x7f15980c3eb0 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1588009e50 tx=0x7f158800b330 comp rx=0 tx=0).stop 2024-09-13T17:58:33.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.223+0000 7f159f293700 1 -- 172.21.15.40:0/1108676463 >> 172.21.15.40:0/1108676463 conn(0x7f159806c800 msgr2=0x7f159806cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:33.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.223+0000 7f159f293700 1 -- 172.21.15.40:0/1108676463 shutdown_connections 2024-09-13T17:58:33.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.223+0000 7f159f293700 1 -- 172.21.15.40:0/1108676463 wait complete. 2024-09-13T17:58:33.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159f293700 1 Processor -- start 2024-09-13T17:58:33.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159f293700 1 -- start start 2024-09-13T17:58:33.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159f293700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15980c9e20 0x7f15980ce360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:33.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159f293700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15980cc4f0 0x7f15981f02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:33.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159f293700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15980c8550 0x7f15980b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:33.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159f293700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15980c6aa0 con 0x7f15980c9e20 2024-09-13T17:58:33.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159f293700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159814a3a0 con 0x7f15980c8550 2024-09-13T17:58:33.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159e291700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15980c9e20 msgr2=0x7f15980ce360 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:58:33.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159f293700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15980c5280 con 0x7f15980cc4f0 2024-09-13T17:58:33.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159e291700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15980c9e20 0x7f15980ce360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:33.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159ea92700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15980c8550 0x7f15980b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:33.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159da90700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15980cc4f0 0x7f15981f02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:33.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159ea92700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15980c8550 0x7f15980b4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:57946/0 (socket says 172.21.15.40:57946) 2024-09-13T17:58:33.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159da90700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15980cc4f0 0x7f15981f02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:39990/0 (socket says 172.21.15.40:39990) 2024-09-13T17:58:33.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159ea92700 1 -- 172.21.15.40:0/504050781 learned_addr learned my addr 172.21.15.40:0/504050781 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:33.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159ea92700 1 -- 172.21.15.40:0/504050781 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15980cc4f0 msgr2=0x7f15981f02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:33.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159ea92700 1 --2- 172.21.15.40:0/504050781 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15980cc4f0 0x7f15981f02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:33.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159ea92700 1 -- 172.21.15.40:0/504050781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15980c9e20 msgr2=0x7f15980ce360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:58:33.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159ea92700 1 --2- 172.21.15.40:0/504050781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15980c9e20 0x7f15980ce360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:33.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159ea92700 1 -- 172.21.15.40:0/504050781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1588009b30 con 0x7f15980c8550 2024-09-13T17:58:33.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f159ea92700 1 --2- 172.21.15.40:0/504050781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15980c8550 0x7f15980b4130 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f159000b5b0 tx=0x7f159000b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:33.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.227+0000 7f1596ffd700 1 -- 172.21.15.40:0/504050781 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590003e70 con 0x7f15980c8550 2024-09-13T17:58:33.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.231+0000 7f1596ffd700 1 -- 172.21.15.40:0/504050781 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1590005070 con 0x7f15980c8550 2024-09-13T17:58:33.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.231+0000 7f159f293700 1 -- 172.21.15.40:0/504050781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15981f0830 con 0x7f15980c8550 2024-09-13T17:58:33.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.231+0000 7f159f293700 1 -- 172.21.15.40:0/504050781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15981e4dd0 con 0x7f15980c8550 2024-09-13T17:58:33.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.231+0000 7f1596ffd700 1 -- 172.21.15.40:0/504050781 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590005420 con 0x7f15980c8550 2024-09-13T17:58:33.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.231+0000 7f159f293700 1 -- 172.21.15.40:0/504050781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15980c5f80 con 0x7f15980c8550 2024-09-13T17:58:33.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.231+0000 7f1596ffd700 1 -- 172.21.15.40:0/504050781 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1590005580 con 0x7f15980c8550 2024-09-13T17:58:33.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.231+0000 7f1596ffd700 1 --2- 172.21.15.40:0/504050781 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1584066710 0x7f1584068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:33.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.231+0000 7f1596ffd700 1 -- 172.21.15.40:0/504050781 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1590084380 con 0x7f15980c8550 2024-09-13T17:58:33.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.231+0000 7f159e291700 1 --2- 172.21.15.40:0/504050781 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1584066710 0x7f1584068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:33.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.235+0000 7f159e291700 1 --2- 172.21.15.40:0/504050781 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1584066710 0x7f1584068bd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f159808cb60 tx=0x7f1588009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:33.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.243+0000 7f1596ffd700 1 -- 172.21.15.40:0/504050781 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1590051380 con 0x7f15980c8550 2024-09-13T17:58:33.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.503+0000 7f159f293700 1 -- 172.21.15.40:0/504050781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f15981f0830 con 0x7f15980c8550 2024-09-13T17:58:33.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.503+0000 7f1596ffd700 1 -- 172.21.15.40:0/504050781 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f15981f0830 con 0x7f15980c8550 2024-09-13T17:58:33.512 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":62,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:58:33.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.507+0000 7f159f293700 1 -- 172.21.15.40:0/504050781 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1584066710 msgr2=0x7f1584068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:33.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.507+0000 7f159f293700 1 --2- 172.21.15.40:0/504050781 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1584066710 0x7f1584068bd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f159808cb60 tx=0x7f1588009000 comp rx=0 tx=0).stop 2024-09-13T17:58:33.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.507+0000 7f159f293700 1 -- 172.21.15.40:0/504050781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15980c8550 msgr2=0x7f15980b4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:33.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.507+0000 7f159f293700 1 --2- 172.21.15.40:0/504050781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15980c8550 0x7f15980b4130 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f159000b5b0 tx=0x7f159000b8c0 comp rx=0 tx=0).stop 2024-09-13T17:58:33.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.507+0000 7f159f293700 1 -- 172.21.15.40:0/504050781 shutdown_connections 2024-09-13T17:58:33.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.507+0000 7f159f293700 1 --2- 172.21.15.40:0/504050781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15980c8550 0x7f15980b4130 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:33.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.507+0000 7f159f293700 1 --2- 172.21.15.40:0/504050781 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1584066710 0x7f1584068bd0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:33.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.507+0000 7f159f293700 1 --2- 172.21.15.40:0/504050781 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15980cc4f0 0x7f15981f02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:33.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.507+0000 7f159f293700 1 --2- 172.21.15.40:0/504050781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15980c9e20 0x7f15980ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:33.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.507+0000 7f159f293700 1 -- 172.21.15.40:0/504050781 >> 172.21.15.40:0/504050781 conn(0x7f159806c800 msgr2=0x7f159814efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:33.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.507+0000 7f159f293700 1 -- 172.21.15.40:0/504050781 shutdown_connections 2024-09-13T17:58:33.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:33.507+0000 7f159f293700 1 -- 172.21.15.40:0/504050781 wait complete. 2024-09-13T17:58:35.035 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:58:35.037 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:58:36.533 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:58:36.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.643+0000 7f5a688ac700 1 Processor -- start 2024-09-13T17:58:36.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.643+0000 7f5a688ac700 1 -- start start 2024-09-13T17:58:36.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a688ac700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a640c8050 0x7f5a6415e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:36.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a688ac700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5a64070190 0x7f5a641575a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:36.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a688ac700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a64066770 con 0x7f5a640c8430 2024-09-13T17:58:36.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a688ac700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a6415e680 con 0x7f5a64070190 2024-09-13T17:58:36.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a688ac700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a64161b80 con 0x7f5a640c8050 2024-09-13T17:58:36.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a62d9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a640c8050 0x7f5a6415e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:36.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a6359e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5a64070190 0x7f5a641575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:36.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a62d9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a640c8050 0x7f5a6415e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:39998/0 (socket says 172.21.15.40:39998) 2024-09-13T17:58:36.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a62d9d700 1 -- 172.21.15.40:0/1417360565 learned_addr learned my addr 172.21.15.40:0/1417360565 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:36.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a62d9d700 1 -- 172.21.15.40:0/1417360565 >> v2:172.21.15.43:3300/0 conn(0x7f5a64070190 msgr2=0x7f5a641575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:36.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a62d9d700 1 --2- 172.21.15.40:0/1417360565 >> v2:172.21.15.43:3300/0 conn(0x7f5a64070190 0x7f5a641575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:36.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a62d9d700 1 -- 172.21.15.40:0/1417360565 >> v1:172.21.15.43:6789/0 conn(0x7f5a640c8430 legacy=0x7f5a640c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:36.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a62d9d700 1 -- 172.21.15.40:0/1417360565 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a64066490 con 0x7f5a640c8050 2024-09-13T17:58:36.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a62d9d700 1 --2- 172.21.15.40:0/1417360565 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a640c8050 0x7f5a6415e140 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5a54009910 tx=0x7f5a54009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f93a4b42df65bb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:36.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a61d9b700 1 -- 172.21.15.40:0/1417360565 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54011830 con 0x7f5a640c8050 2024-09-13T17:58:36.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a61d9b700 1 -- 172.21.15.40:0/1417360565 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a54011990 con 0x7f5a640c8050 2024-09-13T17:58:36.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a61d9b700 1 -- 172.21.15.40:0/1417360565 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54011d40 con 0x7f5a640c8050 2024-09-13T17:58:36.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a688ac700 1 -- 172.21.15.40:0/1417360565 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a640c8050 msgr2=0x7f5a6415e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:36.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a688ac700 1 --2- 172.21.15.40:0/1417360565 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a640c8050 0x7f5a6415e140 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5a54009910 tx=0x7f5a54009c20 comp rx=0 tx=0).stop 2024-09-13T17:58:36.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a688ac700 1 -- 172.21.15.40:0/1417360565 shutdown_connections 2024-09-13T17:58:36.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a688ac700 1 --2- 172.21.15.40:0/1417360565 >> v2:172.21.15.43:3300/0 conn(0x7f5a64070190 0x7f5a641575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:36.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a688ac700 1 --2- 172.21.15.40:0/1417360565 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a640c8050 0x7f5a6415e140 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:36.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a688ac700 1 -- 172.21.15.40:0/1417360565 >> 172.21.15.40:0/1417360565 conn(0x7f5a6406a390 msgr2=0x7f5a6406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:36.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a688ac700 1 -- 172.21.15.40:0/1417360565 shutdown_connections 2024-09-13T17:58:36.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.647+0000 7f5a688ac700 1 -- 172.21.15.40:0/1417360565 wait complete. 2024-09-13T17:58:36.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a688ac700 1 Processor -- start 2024-09-13T17:58:36.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a688ac700 1 -- start start 2024-09-13T17:58:36.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a688ac700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a64070190 0x7f5a6415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:36.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a688ac700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a640c8050 0x7f5a641e6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:36.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a688ac700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a640c8430 0x7f5a6413f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:36.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a688ac700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a641f3cd0 con 0x7f5a640c8430 2024-09-13T17:58:36.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a688ac700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a64140b40 con 0x7f5a640c8050 2024-09-13T17:58:36.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a688ac700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a64140e20 con 0x7f5a64070190 2024-09-13T17:58:36.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a6359e700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a640c8430 msgr2=0x7f5a6413f250 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:58:36.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a6359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a640c8430 0x7f5a6413f250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:36.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a6259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a640c8050 0x7f5a641e6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:36.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a62d9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a64070190 0x7f5a6415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:36.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a6259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a640c8050 0x7f5a641e6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:57974/0 (socket says 172.21.15.40:57974) 2024-09-13T17:58:36.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a6259c700 1 -- 172.21.15.40:0/105826938 learned_addr learned my addr 172.21.15.40:0/105826938 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:36.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a6259c700 1 -- 172.21.15.40:0/105826938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a64070190 msgr2=0x7f5a6415fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:36.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a6259c700 1 --2- 172.21.15.40:0/105826938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a64070190 0x7f5a6415fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:36.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a6259c700 1 -- 172.21.15.40:0/105826938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a640c8430 msgr2=0x7f5a6413f250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:58:36.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a6259c700 1 --2- 172.21.15.40:0/105826938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a640c8430 0x7f5a6413f250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:36.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a6259c700 1 -- 172.21.15.40:0/105826938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a54009570 con 0x7f5a640c8050 2024-09-13T17:58:36.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a62d9d700 1 --2- 172.21.15.40:0/105826938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a64070190 0x7f5a6415fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:36.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.651+0000 7f5a6259c700 1 --2- 172.21.15.40:0/105826938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a640c8050 0x7f5a641e6a90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5a5800c500 tx=0x7f5a5800c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:36.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.655+0000 7f5a4b7fe700 1 -- 172.21.15.40:0/105826938 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a580052d0 con 0x7f5a640c8050 2024-09-13T17:58:36.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.655+0000 7f5a4b7fe700 1 -- 172.21.15.40:0/105826938 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a58005450 con 0x7f5a640c8050 2024-09-13T17:58:36.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.655+0000 7f5a4b7fe700 1 -- 172.21.15.40:0/105826938 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a580038e0 con 0x7f5a640c8050 2024-09-13T17:58:36.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.655+0000 7f5a688ac700 1 -- 172.21.15.40:0/105826938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a641f3330 con 0x7f5a640c8050 2024-09-13T17:58:36.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.655+0000 7f5a688ac700 1 -- 172.21.15.40:0/105826938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a641f38f0 con 0x7f5a640c8050 2024-09-13T17:58:36.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.655+0000 7f5a688ac700 1 -- 172.21.15.40:0/105826938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a641f3e10 con 0x7f5a640c8050 2024-09-13T17:58:36.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.655+0000 7f5a4b7fe700 1 -- 172.21.15.40:0/105826938 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5a58003a40 con 0x7f5a640c8050 2024-09-13T17:58:36.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.655+0000 7f5a4b7fe700 1 --2- 172.21.15.40:0/105826938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5a50066710 0x7f5a50068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:36.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.655+0000 7f5a4b7fe700 1 -- 172.21.15.40:0/105826938 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5a58083920 con 0x7f5a640c8050 2024-09-13T17:58:36.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.655+0000 7f5a62d9d700 1 --2- 172.21.15.40:0/105826938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5a50066710 0x7f5a50068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:36.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.659+0000 7f5a62d9d700 1 --2- 172.21.15.40:0/105826938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5a50066710 0x7f5a50068bd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5a54000c00 tx=0x7f5a54008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:36.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.663+0000 7f5a4b7fe700 1 -- 172.21.15.40:0/105826938 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5a580509b0 con 0x7f5a640c8050 2024-09-13T17:58:36.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.923+0000 7f5a688ac700 1 -- 172.21.15.40:0/105826938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5a641f3e10 con 0x7f5a640c8050 2024-09-13T17:58:36.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.923+0000 7f5a4b7fe700 1 -- 172.21.15.40:0/105826938 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f5a58017620 con 0x7f5a640c8050 2024-09-13T17:58:36.933 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":62,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:58:36.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.927+0000 7f5a688ac700 1 -- 172.21.15.40:0/105826938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5a50066710 msgr2=0x7f5a50068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:36.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.927+0000 7f5a688ac700 1 --2- 172.21.15.40:0/105826938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5a50066710 0x7f5a50068bd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5a54000c00 tx=0x7f5a54008030 comp rx=0 tx=0).stop 2024-09-13T17:58:36.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.927+0000 7f5a688ac700 1 -- 172.21.15.40:0/105826938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a640c8050 msgr2=0x7f5a641e6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:36.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.927+0000 7f5a688ac700 1 --2- 172.21.15.40:0/105826938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a640c8050 0x7f5a641e6a90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5a5800c500 tx=0x7f5a5800c810 comp rx=0 tx=0).stop 2024-09-13T17:58:36.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.927+0000 7f5a688ac700 1 -- 172.21.15.40:0/105826938 shutdown_connections 2024-09-13T17:58:36.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.927+0000 7f5a688ac700 1 --2- 172.21.15.40:0/105826938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a640c8430 0x7f5a6413f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:36.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.927+0000 7f5a688ac700 1 --2- 172.21.15.40:0/105826938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a640c8050 0x7f5a641e6a90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:36.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.927+0000 7f5a688ac700 1 --2- 172.21.15.40:0/105826938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5a50066710 0x7f5a50068bd0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:36.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.927+0000 7f5a688ac700 1 --2- 172.21.15.40:0/105826938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a64070190 0x7f5a6415fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:36.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.927+0000 7f5a688ac700 1 -- 172.21.15.40:0/105826938 >> 172.21.15.40:0/105826938 conn(0x7f5a6406a390 msgr2=0x7f5a640c65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:36.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.927+0000 7f5a688ac700 1 -- 172.21.15.40:0/105826938 shutdown_connections 2024-09-13T17:58:36.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:36.927+0000 7f5a688ac700 1 -- 172.21.15.40:0/105826938 wait complete. 2024-09-13T17:58:39.952 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:58:40.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.063+0000 7f81dffff700 1 Processor -- start 2024-09-13T17:58:40.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.063+0000 7f81dffff700 1 -- start start 2024-09-13T17:58:40.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.063+0000 7f81dffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 0x7f81d804d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.063+0000 7f81dffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f81d8047480 0x7f81d80db4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:40.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.063+0000 7f81dffff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d8036ec0 con 0x7f81d80478a0 2024-09-13T17:58:40.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.063+0000 7f81dffff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d8046ff0 con 0x7f81d8047480 2024-09-13T17:58:40.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.063+0000 7f81dffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d80469e0 con 0x7f81d80d5740 2024-09-13T17:58:40.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.063+0000 7f81de7fc700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f81d8047480 msgr2=0x7f81d80db4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:58:40.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.063+0000 7f81de7fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f81d8047480 0x7f81d80db4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:40.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.063+0000 7f81df7fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f81d80478a0 0x7f81d804bc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:50208/0 (socket says 172.21.15.40:50208) 2024-09-13T17:58:40.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.063+0000 7f81deffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 0x7f81d804d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.063+0000 7f81df7fe700 1 -- 172.21.15.40:0/433292430 learned_addr learned my addr 172.21.15.40:0/433292430 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:58:40.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.063+0000 7f81deffd700 1 -- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 msgr2=0x7f81d804d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:58:40.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.063+0000 7f81deffd700 1 -- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 msgr2=0x7f81d804d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:40.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.063+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 0x7f81d804d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:40.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.063+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 0x7f81d804d070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:40.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.267+0000 7f81de7fc700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.40:3300/0 conn(0x7f81d8047480 msgr2=0x7f81d80db4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:58:40.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.267+0000 7f81de7fc700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.40:3300/0 conn(0x7f81d8047480 0x7f81d80db4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:58:40.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.267+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 0x7f81d804d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.267+0000 7f81deffd700 1 -- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 msgr2=0x7f81d804d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:58:40.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.267+0000 7f81deffd700 1 -- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 msgr2=0x7f81d804d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:40.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.267+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 0x7f81d804d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:40.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.267+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 0x7f81d804d070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:58:40.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.671+0000 7f81de7fc700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.40:3300/0 conn(0x7f81d8047480 msgr2=0x7f81d80db4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:58:40.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.671+0000 7f81de7fc700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.40:3300/0 conn(0x7f81d8047480 0x7f81d80db4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:58:40.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.671+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 0x7f81d804d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:40.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.671+0000 7f81deffd700 1 -- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 msgr2=0x7f81d804d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:58:40.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.671+0000 7f81deffd700 1 -- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 msgr2=0x7f81d804d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:40.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.671+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 0x7f81d804d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:40.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:40.671+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 0x7f81d804d070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:58:41.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:41.475+0000 7f81de7fc700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.40:3300/0 conn(0x7f81d8047480 msgr2=0x7f81d80db4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:58:41.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:41.475+0000 7f81de7fc700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.40:3300/0 conn(0x7f81d8047480 0x7f81d80db4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:58:41.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:41.475+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 0x7f81d804d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:41.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:41.475+0000 7f81deffd700 1 -- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 msgr2=0x7f81d804d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:58:41.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:41.475+0000 7f81deffd700 1 -- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 msgr2=0x7f81d804d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:41.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:41.475+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 0x7f81d804d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:41.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:41.475+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 0x7f81d804d070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:58:43.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.063+0000 7f81dcff9700 1 -- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 msgr2=0x7f81d804d070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:58:43.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.063+0000 7f81dcff9700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d80d5740 0x7f81d804d070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.063+0000 7f81dcff9700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.40:3300/0 conn(0x7f81d8047480 msgr2=0x7f81d80db4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:58:43.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.063+0000 7f81dcff9700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.40:3300/0 conn(0x7f81d8047480 0x7f81d80db4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:43.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.063+0000 7f81dcff9700 1 -- 172.21.15.40:0/433292430 >> v1:172.21.15.43:6789/0 conn(0x7f81d80478a0 legacy=0x7f81d804bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:43.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.063+0000 7f81dcff9700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 0x7f81cc005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:43.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.063+0000 7f81dcff9700 1 -- 172.21.15.40:0/433292430 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d80dbe00 con 0x7f81cc006210 2024-09-13T17:58:43.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.063+0000 7f81dcff9700 1 -- 172.21.15.40:0/433292430 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d8046ff0 con 0x7f81cc003740 2024-09-13T17:58:43.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.063+0000 7f81de7fc700 1 -- 172.21.15.40:0/433292430 >> v1:172.21.15.40:6789/0 conn(0x7f81cc006210 legacy=0x7f81cc0086b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:58:43.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.063+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 0x7f81cc005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:43.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.063+0000 7f81deffd700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 msgr2=0x7f81cc005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:58:43.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.063+0000 7f81deffd700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 msgr2=0x7f81cc005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:43.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.063+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 0x7f81cc005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:43.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.063+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 0x7f81cc005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:43.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.267+0000 7f81de7fc700 1 -- 172.21.15.40:0/433292430 >> v1:172.21.15.40:6789/0 conn(0x7f81cc006210 legacy=0x7f81cc0086b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:58:43.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.267+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 0x7f81cc005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:43.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.267+0000 7f81deffd700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 msgr2=0x7f81cc005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:58:43.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.267+0000 7f81deffd700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 msgr2=0x7f81cc005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:43.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.267+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 0x7f81cc005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:43.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.267+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 0x7f81cc005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:58:43.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.671+0000 7f81de7fc700 1 -- 172.21.15.40:0/433292430 >> v1:172.21.15.40:6789/0 conn(0x7f81cc006210 legacy=0x7f81cc0086b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:58:43.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.671+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 0x7f81cc005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:43.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.671+0000 7f81deffd700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 msgr2=0x7f81cc005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:58:43.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.671+0000 7f81deffd700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 msgr2=0x7f81cc005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:43.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.671+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 0x7f81cc005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:43.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:43.671+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 0x7f81cc005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:58:44.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:44.475+0000 7f81de7fc700 1 -- 172.21.15.40:0/433292430 >> v1:172.21.15.40:6789/0 conn(0x7f81cc006210 legacy=0x7f81cc0086b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:58:44.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:44.475+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 0x7f81cc005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:44.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:44.475+0000 7f81deffd700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 msgr2=0x7f81cc005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:58:44.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:44.475+0000 7f81deffd700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 msgr2=0x7f81cc005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:44.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:44.475+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 0x7f81cc005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:44.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:44.475+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 0x7f81cc005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:58:46.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.067+0000 7f81dcff9700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 msgr2=0x7f81cc005bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:58:46.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.067+0000 7f81dcff9700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc003740 0x7f81cc005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:46.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.067+0000 7f81dcff9700 1 -- 172.21.15.40:0/433292430 >> v1:172.21.15.40:6789/0 conn(0x7f81cc006210 legacy=0x7f81cc0086b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:58:46.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.067+0000 7f81df7fe700 1 -- 172.21.15.40:0/433292430 reap_dead start 2024-09-13T17:58:46.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.067+0000 7f81dcff9700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 0x7f81cc005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:46.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.067+0000 7f81dcff9700 1 -- 172.21.15.40:0/433292430 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81cc00aff0 con 0x7f81cc00a640 2024-09-13T17:58:46.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.067+0000 7f81dcff9700 1 -- 172.21.15.40:0/433292430 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d80dbe00 con 0x7f81d80478a0 2024-09-13T17:58:46.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.067+0000 7f81dcff9700 1 -- 172.21.15.40:0/433292430 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d8046ff0 con 0x7f81cc00aae0 2024-09-13T17:58:46.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.067+0000 7f81de7fc700 1 -- 172.21.15.40:0/433292430 >> v1:172.21.15.40:6789/0 conn(0x7f81cc00a640 legacy=0x7f81cc0066a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:58:46.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.067+0000 7f81df7fe700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 0x7f81cc005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:46.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.067+0000 7f81df7fe700 1 -- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 msgr2=0x7f81cc005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:58:46.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.067+0000 7f81df7fe700 1 -- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 msgr2=0x7f81cc005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:46.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.067+0000 7f81df7fe700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 0x7f81cc005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:46.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.067+0000 7f81df7fe700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 0x7f81cc005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:46.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.271+0000 7f81de7fc700 1 -- 172.21.15.40:0/433292430 >> v1:172.21.15.40:6789/0 conn(0x7f81cc00a640 legacy=0x7f81cc0066a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:58:46.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.271+0000 7f81df7fe700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 0x7f81cc005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:46.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.271+0000 7f81df7fe700 1 -- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 msgr2=0x7f81cc005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:58:46.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.271+0000 7f81df7fe700 1 -- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 msgr2=0x7f81cc005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:46.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.271+0000 7f81df7fe700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 0x7f81cc005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:46.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.271+0000 7f81df7fe700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 0x7f81cc005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:58:46.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.675+0000 7f81de7fc700 1 -- 172.21.15.40:0/433292430 >> v1:172.21.15.40:6789/0 conn(0x7f81cc00a640 legacy=0x7f81cc0066a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:58:46.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.675+0000 7f81df7fe700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 0x7f81cc005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:46.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.675+0000 7f81df7fe700 1 -- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 msgr2=0x7f81cc005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T17:58:46.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.675+0000 7f81df7fe700 1 -- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 msgr2=0x7f81cc005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:46.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.675+0000 7f81df7fe700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 0x7f81cc005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:46.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:46.675+0000 7f81df7fe700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 0x7f81cc005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:58:47.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:47.479+0000 7f81de7fc700 1 -- 172.21.15.40:0/433292430 >> v1:172.21.15.40:6789/0 conn(0x7f81cc00a640 legacy=0x7f81cc0066a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:58:47.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:47.479+0000 7f81df7fe700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 0x7f81cc005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:47.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:47.479+0000 7f81df7fe700 1 -- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 msgr2=0x7f81cc005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:58:47.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:47.479+0000 7f81df7fe700 1 -- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 msgr2=0x7f81cc005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:47.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:47.479+0000 7f81df7fe700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 0x7f81cc005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:47.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:47.479+0000 7f81df7fe700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 0x7f81cc005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:58:49.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.063+0000 7f81dcff9700 1 -- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 msgr2=0x7f81cc005bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:58:49.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.063+0000 7f81dcff9700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81cc00aae0 0x7f81cc005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:49.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.063+0000 7f81dcff9700 1 -- 172.21.15.40:0/433292430 >> v1:172.21.15.43:6789/0 conn(0x7f81d80478a0 legacy=0x7f81cc009f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:49.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.063+0000 7f81dcff9700 1 -- 172.21.15.40:0/433292430 >> v1:172.21.15.40:6789/0 conn(0x7f81cc00a640 legacy=0x7f81cc0066a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:58:49.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.063+0000 7f81dcff9700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.40:3300/0 conn(0x7f81cc00f680 0x7f81cc00fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:49.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.063+0000 7f81dcff9700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc010160 0x7f81cc00e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:49.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.063+0000 7f81dcff9700 1 -- 172.21.15.40:0/433292430 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d8046ff0 con 0x7f81cc00f680 2024-09-13T17:58:49.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.063+0000 7f81dcff9700 1 -- 172.21.15.40:0/433292430 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d80469e0 con 0x7f81cc010160 2024-09-13T17:58:49.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.063+0000 7f81deffd700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.40:3300/0 conn(0x7f81cc00f680 msgr2=0x7f81cc00fa90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:58:49.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.063+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.40:3300/0 conn(0x7f81cc00f680 0x7f81cc00fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:49.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.063+0000 7f81de7fc700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc010160 0x7f81cc00e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:49.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.067+0000 7f81de7fc700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc010160 msgr2=0x7f81cc00e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:58:49.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.067+0000 7f81de7fc700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc010160 msgr2=0x7f81cc00e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:49.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.067+0000 7f81de7fc700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc010160 0x7f81cc00e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:49.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.067+0000 7f81de7fc700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc010160 0x7f81cc00e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:49.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.267+0000 7f81deffd700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.40:3300/0 conn(0x7f81cc00f680 msgr2=0x7f81cc00fa90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:58:49.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.267+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.40:3300/0 conn(0x7f81cc00f680 0x7f81cc00fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:58:49.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.271+0000 7f81de7fc700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc010160 0x7f81cc00e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:49.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.271+0000 7f81de7fc700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc010160 msgr2=0x7f81cc00e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:58:49.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.271+0000 7f81de7fc700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc010160 msgr2=0x7f81cc00e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:58:49.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.271+0000 7f81de7fc700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc010160 0x7f81cc00e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:58:49.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.271+0000 7f81de7fc700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc010160 0x7f81cc00e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:58:49.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.671+0000 7f81deffd700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.40:3300/0 conn(0x7f81cc00f680 msgr2=0x7f81cc00fa90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:58:49.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.671+0000 7f81deffd700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.40:3300/0 conn(0x7f81cc00f680 0x7f81cc00fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:58:49.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.675+0000 7f81de7fc700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.43:3300/0 conn(0x7f81cc010160 0x7f81cc00e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:49.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.675+0000 7f81de7fc700 1 -- 172.21.15.40:0/433292430 >> v2:172.21.15.40:3300/0 conn(0x7f81cc00f680 msgr2=0x7f81cc00fa90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:58:49.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.675+0000 7f81de7fc700 1 --2- 172.21.15.40:0/433292430 >> v2:172.21.15.40:3300/0 conn(0x7f81cc00f680 0x7f81cc00fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:49.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.675+0000 7f81de7fc700 1 -- 172.21.15.40:0/433292430 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc010650 con 0x7f81cc010160 2024-09-13T17:58:49.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.675+0000 7f81de7fc700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81cc010160 0x7f81cc00e510 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f81cc010130 tx=0x7f81d4011620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79685acefea3f19d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:49.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.675+0000 7f81ddffb700 1 -- 172.21.15.40:0/433292430 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d4013420 con 0x7f81cc010160 2024-09-13T17:58:49.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.675+0000 7f81ddffb700 1 -- 172.21.15.40:0/433292430 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d4013580 con 0x7f81cc010160 2024-09-13T17:58:49.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.675+0000 7f81ddffb700 1 -- 172.21.15.40:0/433292430 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d4013930 con 0x7f81cc010160 2024-09-13T17:58:49.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.675+0000 7f81dffff700 1 -- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81cc010160 msgr2=0x7f81cc00e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:49.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.675+0000 7f81dffff700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81cc010160 0x7f81cc00e510 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f81cc010130 tx=0x7f81d4011620 comp rx=0 tx=0).stop 2024-09-13T17:58:49.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.675+0000 7f81df7fe700 1 -- 172.21.15.40:0/433292430 reap_dead start 2024-09-13T17:58:49.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.675+0000 7f81dffff700 1 -- 172.21.15.40:0/433292430 shutdown_connections 2024-09-13T17:58:49.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.675+0000 7f81dffff700 1 --2- 172.21.15.40:0/433292430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81cc010160 0x7f81cc00e510 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:49.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.675+0000 7f81dffff700 1 -- 172.21.15.40:0/433292430 >> 172.21.15.40:0/433292430 conn(0x7f81d8050e10 msgr2=0x7f81d8051220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:49.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.679+0000 7f81dffff700 1 -- 172.21.15.40:0/433292430 shutdown_connections 2024-09-13T17:58:49.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.679+0000 7f81dffff700 1 -- 172.21.15.40:0/433292430 wait complete. 2024-09-13T17:58:49.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.679+0000 7f81dffff700 1 Processor -- start 2024-09-13T17:58:49.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.679+0000 7f81dffff700 1 -- start start 2024-09-13T17:58:49.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81dffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81d80d0bf0 0x7f81d8174c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81dffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d8007e90 0x7f81d8175500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81dffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81d80d0fd0 0x7f81d80066a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81dffff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d8178010 con 0x7f81d80d0fd0 2024-09-13T17:58:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81dffff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d81747f0 con 0x7f81d80d0bf0 2024-09-13T17:58:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81dffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d81784d0 con 0x7f81d8007e90 2024-09-13T17:58:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81df7fe700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81d80d0fd0 msgr2=0x7f81d80066a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:58:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81df7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81d80d0fd0 0x7f81d80066a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81deffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81d80d0bf0 0x7f81d8174c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81de7fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d8007e90 0x7f81d8175500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81deffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81d80d0bf0 0x7f81d8174c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:34018/0 (socket says 172.21.15.40:34018) 2024-09-13T17:58:49.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81deffd700 1 -- 172.21.15.40:0/1376938499 learned_addr learned my addr 172.21.15.40:0/1376938499 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:49.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81de7fc700 1 -- 172.21.15.40:0/1376938499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81d80d0bf0 msgr2=0x7f81d8174c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:49.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81de7fc700 1 --2- 172.21.15.40:0/1376938499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81d80d0bf0 0x7f81d8174c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:49.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81de7fc700 1 -- 172.21.15.40:0/1376938499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81d80d0fd0 msgr2=0x7f81d80066a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:58:49.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81de7fc700 1 --2- 172.21.15.40:0/1376938499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81d80d0fd0 0x7f81d80066a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:49.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81de7fc700 1 -- 172.21.15.40:0/1376938499 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d4007b70 con 0x7f81d8007e90 2024-09-13T17:58:49.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81deffd700 1 --2- 172.21.15.40:0/1376938499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81d80d0bf0 0x7f81d8174c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:49.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81de7fc700 1 --2- 172.21.15.40:0/1376938499 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d8007e90 0x7f81d8175500 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00f5f0 tx=0x7f81d4007450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:49.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81c7fff700 1 -- 172.21.15.40:0/1376938499 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d401c450 con 0x7f81d8007e90 2024-09-13T17:58:49.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.683+0000 7f81c7fff700 1 -- 172.21.15.40:0/1376938499 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d401c5b0 con 0x7f81d8007e90 2024-09-13T17:58:49.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.687+0000 7f81dffff700 1 -- 172.21.15.40:0/1376938499 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d81781f0 con 0x7f81d8007e90 2024-09-13T17:58:49.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.687+0000 7f81c7fff700 1 -- 172.21.15.40:0/1376938499 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d401c940 con 0x7f81d8007e90 2024-09-13T17:58:49.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.687+0000 7f81dffff700 1 -- 172.21.15.40:0/1376938499 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d8174480 con 0x7f81d8007e90 2024-09-13T17:58:49.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.687+0000 7f81dffff700 1 -- 172.21.15.40:0/1376938499 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d804d070 con 0x7f81d8007e90 2024-09-13T17:58:49.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.687+0000 7f81c7fff700 1 -- 172.21.15.40:0/1376938499 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f81d401caa0 con 0x7f81d8007e90 2024-09-13T17:58:49.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.691+0000 7f81c7fff700 1 --2- 172.21.15.40:0/1376938499 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f81cc067930 0x7f81cc069df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:49.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.691+0000 7f81c7fff700 1 -- 172.21.15.40:0/1376938499 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f81d4009040 con 0x7f81d8007e90 2024-09-13T17:58:49.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.691+0000 7f81deffd700 1 --2- 172.21.15.40:0/1376938499 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f81cc067930 0x7f81cc069df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:49.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.691+0000 7f81deffd700 1 --2- 172.21.15.40:0/1376938499 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f81cc067930 0x7f81cc069df0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f81d0016500 tx=0x7f81d001a040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:49.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.703+0000 7f81c7fff700 1 -- 172.21.15.40:0/1376938499 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f81d4053b70 con 0x7f81d8007e90 2024-09-13T17:58:49.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.939+0000 7f81dffff700 1 -- 172.21.15.40:0/1376938499 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f81d8174610 con 0x7f81d8007e90 2024-09-13T17:58:49.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.939+0000 7f81c7fff700 1 -- 172.21.15.40:0/1376938499 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7f81d4017610 con 0x7f81d8007e90 2024-09-13T17:58:49.948 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":66,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:58:49.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.943+0000 7f81dffff700 1 -- 172.21.15.40:0/1376938499 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f81cc067930 msgr2=0x7f81cc069df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:49.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.943+0000 7f81dffff700 1 --2- 172.21.15.40:0/1376938499 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f81cc067930 0x7f81cc069df0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f81d0016500 tx=0x7f81d001a040 comp rx=0 tx=0).stop 2024-09-13T17:58:49.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.943+0000 7f81dffff700 1 -- 172.21.15.40:0/1376938499 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d8007e90 msgr2=0x7f81d8175500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:49.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.943+0000 7f81dffff700 1 --2- 172.21.15.40:0/1376938499 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d8007e90 0x7f81d8175500 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00f5f0 tx=0x7f81d4007450 comp rx=0 tx=0).stop 2024-09-13T17:58:49.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.943+0000 7f81dffff700 1 -- 172.21.15.40:0/1376938499 shutdown_connections 2024-09-13T17:58:49.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.943+0000 7f81dffff700 1 --2- 172.21.15.40:0/1376938499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f81d80d0fd0 0x7f81d80066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:49.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.943+0000 7f81dffff700 1 --2- 172.21.15.40:0/1376938499 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f81cc067930 0x7f81cc069df0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:49.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.943+0000 7f81dffff700 1 --2- 172.21.15.40:0/1376938499 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f81d8007e90 0x7f81d8175500 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:49.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.943+0000 7f81dffff700 1 --2- 172.21.15.40:0/1376938499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81d80d0bf0 0x7f81d8174c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:49.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.943+0000 7f81dffff700 1 -- 172.21.15.40:0/1376938499 >> 172.21.15.40:0/1376938499 conn(0x7f81d8050e10 msgr2=0x7f81d804dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:49.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.943+0000 7f81dffff700 1 -- 172.21.15.40:0/1376938499 shutdown_connections 2024-09-13T17:58:49.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:49.943+0000 7f81dffff700 1 -- 172.21.15.40:0/1376938499 wait complete. 2024-09-13T17:58:49.964 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T17:58:49.964 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-13T17:58:49.964 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:58:50.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.071+0000 7f9012647700 1 Processor -- start 2024-09-13T17:58:50.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.071+0000 7f9012647700 1 -- start start 2024-09-13T17:58:50.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9012647700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f900c151580 0x7f900c0c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9011645700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f900c151580 msgr2=0x7f900c0c52b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:58:50.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9011645700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f900c151580 0x7f900c0c52b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:50.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9012647700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f900c157f10 0x7f900c0c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9012647700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f900c070ae0 con 0x7f900c0b1fb0 2024-09-13T17:58:50.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9012647700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900c070d30 con 0x7f900c151580 2024-09-13T17:58:50.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9010e44700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f900c0b1fb0 legacy=0x7f900c157800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:58:50.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9012647700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900c0c29c0 con 0x7f900c157f10 2024-09-13T17:58:50.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9011e46700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f900c157f10 0x7f900c0c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9011e46700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f900c157f10 0x7f900c0c57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:34028/0 (socket says 172.21.15.40:34028) 2024-09-13T17:58:50.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9011e46700 1 -- 172.21.15.40:0/1248043417 learned_addr learned my addr 172.21.15.40:0/1248043417 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:50.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9011e46700 1 -- 172.21.15.40:0/1248043417 >> v2:172.21.15.40:3300/0 conn(0x7f900c151580 msgr2=0x7f900c0c52b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:58:50.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9011e46700 1 --2- 172.21.15.40:0/1248043417 >> v2:172.21.15.40:3300/0 conn(0x7f900c151580 0x7f900c0c52b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9011e46700 1 -- 172.21.15.40:0/1248043417 >> v1:172.21.15.40:6789/0 conn(0x7f900c0b1fb0 legacy=0x7f900c157800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:58:50.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9011e46700 1 -- 172.21.15.40:0/1248043417 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c0c2ca0 con 0x7f900c157f10 2024-09-13T17:58:50.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9011e46700 1 --2- 172.21.15.40:0/1248043417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f900c157f10 0x7f900c0c57f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f900c0c9a50 tx=0x7f900400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b06fc78833753c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:50.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9003fff700 1 -- 172.21.15.40:0/1248043417 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900400e070 con 0x7f900c157f10 2024-09-13T17:58:50.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9003fff700 1 -- 172.21.15.40:0/1248043417 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9004003a40 con 0x7f900c157f10 2024-09-13T17:58:50.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9003fff700 1 -- 172.21.15.40:0/1248043417 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9004003df0 con 0x7f900c157f10 2024-09-13T17:58:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9012647700 1 -- 172.21.15.40:0/1248043417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f900c157f10 msgr2=0x7f900c0c57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9012647700 1 --2- 172.21.15.40:0/1248043417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f900c157f10 0x7f900c0c57f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f900c0c9a50 tx=0x7f900400b330 comp rx=0 tx=0).stop 2024-09-13T17:58:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9012647700 1 -- 172.21.15.40:0/1248043417 shutdown_connections 2024-09-13T17:58:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9012647700 1 --2- 172.21.15.40:0/1248043417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f900c157f10 0x7f900c0c57f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9012647700 1 --2- 172.21.15.40:0/1248043417 >> v2:172.21.15.40:3300/0 conn(0x7f900c151580 0x7f900c0c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9012647700 1 -- 172.21.15.40:0/1248043417 >> 172.21.15.40:0/1248043417 conn(0x7f900c06e6d0 msgr2=0x7f900c06eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9012647700 1 -- 172.21.15.40:0/1248043417 shutdown_connections 2024-09-13T17:58:50.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.075+0000 7f9012647700 1 -- 172.21.15.40:0/1248043417 wait complete. 2024-09-13T17:58:50.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9012647700 1 Processor -- start 2024-09-13T17:58:50.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9012647700 1 -- start start 2024-09-13T17:58:50.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9012647700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f900c0b1fb0 0x7f900c1e1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9012647700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f900c151580 0x7f900c1e1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9012647700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f900c157f10 0x7f900c0c4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9012647700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0c81e0 con 0x7f900c157f10 2024-09-13T17:58:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9012647700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0c7ad0 con 0x7f900c151580 2024-09-13T17:58:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9012647700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0c7450 con 0x7f900c0b1fb0 2024-09-13T17:58:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9011e46700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f900c157f10 msgr2=0x7f900c0c4370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:58:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9011e46700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f900c157f10 0x7f900c0c4370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9011645700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f900c0b1fb0 0x7f900c1e1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9010e44700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f900c151580 0x7f900c1e1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9011645700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f900c0b1fb0 0x7f900c1e1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:54732/0 (socket says 172.21.15.40:54732) 2024-09-13T17:58:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9011645700 1 -- 172.21.15.40:0/3050212269 learned_addr learned my addr 172.21.15.40:0/3050212269 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:50.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9011645700 1 -- 172.21.15.40:0/3050212269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f900c151580 msgr2=0x7f900c1e1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9011645700 1 --2- 172.21.15.40:0/3050212269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f900c151580 0x7f900c1e1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9011645700 1 -- 172.21.15.40:0/3050212269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f900c157f10 msgr2=0x7f900c0c4370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:58:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9011645700 1 --2- 172.21.15.40:0/3050212269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f900c157f10 0x7f900c0c4370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9011645700 1 -- 172.21.15.40:0/3050212269 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9004009b30 con 0x7f900c0b1fb0 2024-09-13T17:58:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9010e44700 1 --2- 172.21.15.40:0/3050212269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f900c151580 0x7f900c1e1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:58:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9011645700 1 --2- 172.21.15.40:0/3050212269 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f900c0b1fb0 0x7f900c1e1990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f900800b780 tx=0x7f900800ba90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9001ffb700 1 -- 172.21.15.40:0/3050212269 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008003e70 con 0x7f900c0b1fb0 2024-09-13T17:58:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9001ffb700 1 -- 172.21.15.40:0/3050212269 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9008005070 con 0x7f900c0b1fb0 2024-09-13T17:58:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9012647700 1 -- 172.21.15.40:0/3050212269 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900c0c7170 con 0x7f900c0b1fb0 2024-09-13T17:58:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9012647700 1 -- 172.21.15.40:0/3050212269 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900c1ef510 con 0x7f900c0b1fb0 2024-09-13T17:58:50.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.079+0000 7f9001ffb700 1 -- 172.21.15.40:0/3050212269 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008005420 con 0x7f900c0b1fb0 2024-09-13T17:58:50.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.083+0000 7f9012647700 1 --2- 172.21.15.40:0/3050212269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fec002a40 0x7f8fec004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.083+0000 7f9012647700 1 -- 172.21.15.40:0/3050212269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec007f40 con 0x7f8fec002a40 2024-09-13T17:58:50.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.083+0000 7f9010e44700 1 --2- 172.21.15.40:0/3050212269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fec002a40 0x7f8fec004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.083+0000 7f9010e44700 1 -- 172.21.15.40:0/3050212269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8fec008100 con 0x7f8fec002a40 2024-09-13T17:58:50.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.083+0000 7f9001ffb700 1 -- 172.21.15.40:0/3050212269 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9008005580 con 0x7f900c0b1fb0 2024-09-13T17:58:50.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.083+0000 7f9010e44700 1 --2- 172.21.15.40:0/3050212269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fec002a40 0x7f8fec004f00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f900c071f30 tx=0x7f8ffc005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:50.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.083+0000 7f9001ffb700 1 --2- 172.21.15.40:0/3050212269 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ff00666e0 0x7f8ff0068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.083+0000 7f9001ffb700 1 -- 172.21.15.40:0/3050212269 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9008083fd0 con 0x7f900c0b1fb0 2024-09-13T17:58:50.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.083+0000 7f9011e46700 1 --2- 172.21.15.40:0/3050212269 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ff00666e0 0x7f8ff0068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.083+0000 7f9001ffb700 1 -- 172.21.15.40:0/3050212269 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc00e820 con 0x7f8fec002a40 2024-09-13T17:58:50.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.083+0000 7f9001ffb700 1 -- 172.21.15.40:0/3050212269 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8ffc00e980 con 0x7f8fec002a40 2024-09-13T17:58:50.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.083+0000 7f9001ffb700 1 -- 172.21.15.40:0/3050212269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fec002a40 msgr2=0x7f8fec004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.083+0000 7f9001ffb700 1 --2- 172.21.15.40:0/3050212269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fec002a40 0x7f8fec004f00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f900c071f30 tx=0x7f8ffc005770 comp rx=0 tx=0).stop 2024-09-13T17:58:50.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.083+0000 7f9001ffb700 1 -- 172.21.15.40:0/3050212269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fec002a40 msgr2=0x7f8fec004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:58:50.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.083+0000 7f9001ffb700 1 --2- 172.21.15.40:0/3050212269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fec002a40 0x7f8fec004f00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.083+0000 7f9011e46700 1 --2- 172.21.15.40:0/3050212269 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ff00666e0 0x7f8ff0068ba0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f900400b370 tx=0x7f9004009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:50.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.095+0000 7f9012647700 1 --2- 172.21.15.40:0/3050212269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fec006ee0 0x7f8fec007330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.095+0000 7f9012647700 1 -- 172.21.15.40:0/3050212269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec00a4a0 con 0x7f8fec006ee0 2024-09-13T17:58:50.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.095+0000 7f9010e44700 1 --2- 172.21.15.40:0/3050212269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fec006ee0 0x7f8fec007330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.099+0000 7f9010e44700 1 -- 172.21.15.40:0/3050212269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8fec007870 con 0x7f8fec006ee0 2024-09-13T17:58:50.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.099+0000 7f9010e44700 1 --2- 172.21.15.40:0/3050212269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fec006ee0 0x7f8fec007330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc018000 tx=0x7f8ffc005a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:50.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.099+0000 7f9001ffb700 1 -- 172.21.15.40:0/3050212269 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc005cf0 con 0x7f8fec006ee0 2024-09-13T17:58:50.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.099+0000 7f9001ffb700 1 -- 172.21.15.40:0/3050212269 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4938 (secure 0 0 0) 0x7f8ffc005e50 con 0x7f8fec006ee0 2024-09-13T17:58:50.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.099+0000 7f9001ffb700 1 -- 172.21.15.40:0/3050212269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fec006ee0 msgr2=0x7f8fec007330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.099+0000 7f9001ffb700 1 --2- 172.21.15.40:0/3050212269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fec006ee0 0x7f8fec007330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc018000 tx=0x7f8ffc005a80 comp rx=0 tx=0).stop 2024-09-13T17:58:50.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.099+0000 7f9001ffb700 1 -- 172.21.15.40:0/3050212269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fec006ee0 msgr2=0x7f8fec007330 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:58:50.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.099+0000 7f9001ffb700 1 --2- 172.21.15.40:0/3050212269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fec006ee0 0x7f8fec007330 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.108 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:58:50.108 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:58:50.109 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:58:50.109 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:58:50.109 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 66, 2024-09-13T17:58:50.109 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:58:50.109 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:58:50.109 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:58:50.109 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:50.109 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T17:58:50.109 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:58:50.109 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:58:50.109 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:58:50.110 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:50.110 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:50.110 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:50.110 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:50.110 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:50.110 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:50.110 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:50.110 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:50.110 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:50.110 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:50.110 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:50.111 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:58:50.111 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:58:50.111 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:50.111 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:50.111 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:50.111 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:50.111 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:50.111 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:50.111 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:50.111 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:50.112 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:50.112 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:50.112 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:50.112 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.112 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:58:50.112 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:58:50.112 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:58:50.112 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:58:50.112 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:58:50.112 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:58:50.112 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:58:50.112 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:58:50.113 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:58:50.113 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:58:50.113 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:58:50.113 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:58:50.113 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:58:50.113 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:58:50.113 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:58:50.113 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:58:50.113 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:58:50.113 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:50.113 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:50.114 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:50.114 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:50.114 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:50.114 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:50.114 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:50.114 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:50.114 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:50.114 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:50.114 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:50.114 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:58:50.114 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.114 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:58:50.115 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.115 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:58:50.115 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:58:50.115 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:50.115 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:50.115 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.115 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:50.115 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:58:50.115 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:50.115 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.116 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.116 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:50.116 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:58:50.116 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:50.116 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:50.116 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:50.116 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.116 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:58:50.116 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:58:50.116 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:50.116 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:50.117 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:50.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.117 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.117 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:58:50.117 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:58:50.117 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:50.117 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:50.117 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.117 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:50.117 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:58:50.117 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:50.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.118 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:50.118 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:58:50.118 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:50.118 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:50.118 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:50.118 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.118 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:58:50.118 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:58:50.118 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:50.118 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:50.118 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:50.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.119 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:58:50.119 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:58:50.119 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:50.119 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:50.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.119 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:50.119 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:58:50.119 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:50.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.120 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.120 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:50.120 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:58:50.120 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:50.120 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:50.120 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:50.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.120 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:58:50.120 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:58:50.120 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:50.120 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:50.121 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:50.121 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:50.121 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:50.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.121 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:58:50.121 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:58:50.121 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.121 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:50.121 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:50.121 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:50.121 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:50.121 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:50.122 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:58:50.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.122 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:50.122 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:50.122 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:50.122 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:50.122 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:50.122 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:58:50.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.122 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:50.122 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:50.123 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:50.123 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:50.123 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:50.123 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:58:50.123 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.123 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:50.123 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:50.123 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:50.123 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:50.123 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:50.123 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:58:50.123 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.124 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:50.124 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:50.124 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:50.124 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:50.124 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:50.124 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.124 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:58:50.124 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:58:50.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.099+0000 7f9012647700 1 -- 172.21.15.40:0/3050212269 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ff00666e0 msgr2=0x7f8ff0068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.099+0000 7f9012647700 1 --2- 172.21.15.40:0/3050212269 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8ff00666e0 0x7f8ff0068ba0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f900400b370 tx=0x7f9004009000 comp rx=0 tx=0).stop 2024-09-13T17:58:50.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.099+0000 7f9012647700 1 -- 172.21.15.40:0/3050212269 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f900c0b1fb0 msgr2=0x7f900c1e1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.099+0000 7f9012647700 1 --2- 172.21.15.40:0/3050212269 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f900c0b1fb0 0x7f900c1e1990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f900800b780 tx=0x7f900800ba90 comp rx=0 tx=0).stop 2024-09-13T17:58:50.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.099+0000 7f9011e46700 1 -- 172.21.15.40:0/3050212269 reap_dead start 2024-09-13T17:58:50.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.099+0000 7f9011e46700 1 -- 172.21.15.40:0/3050212269 reap_dead start 2024-09-13T17:58:50.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.099+0000 7f9012647700 1 -- 172.21.15.40:0/3050212269 shutdown_connections 2024-09-13T17:58:50.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.099+0000 7f9012647700 1 -- 172.21.15.40:0/3050212269 >> 172.21.15.40:0/3050212269 conn(0x7f900c06e6d0 msgr2=0x7f900c155230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:50.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.099+0000 7f9012647700 1 -- 172.21.15.40:0/3050212269 shutdown_connections 2024-09-13T17:58:50.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.099+0000 7f9012647700 1 -- 172.21.15.40:0/3050212269 wait complete. 2024-09-13T17:58:50.126 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T17:58:50.127 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T17:58:50.127 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:58:50.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ef100700 1 Processor -- start 2024-09-13T17:58:50.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ef100700 1 -- start start 2024-09-13T17:58:50.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ef100700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb5e80cf9b0 0x7fb5e806c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ef100700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e8159e50 con 0x7fb5e806c5b0 2024-09-13T17:58:50.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ef100700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e815d390 con 0x7fb5e8153720 2024-09-13T17:58:50.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ef100700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e80c5ea0 con 0x7fb5e80cf9b0 2024-09-13T17:58:50.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ed8fd700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7fb5e80cf9b0 msgr2=0x7fb5e806c070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:58:50.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ee8ff700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7fb5e806c5b0 legacy=0x7fb5e8163d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T17:58:50.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ed8fd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb5e80cf9b0 0x7fb5e806c070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:50.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ee0fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb5e8153720 0x7fb5e80ca510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:43144/0 (socket says 172.21.15.40:43144) 2024-09-13T17:58:50.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ee0fe700 1 -- 172.21.15.40:0/2999113653 learned_addr learned my addr 172.21.15.40:0/2999113653 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:58:50.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ed0fc700 1 -- 172.21.15.40:0/2999113653 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3649800105 0 0) 0x7fb5e815d390 con 0x7fb5e8153720 2024-09-13T17:58:50.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ed0fc700 1 -- 172.21.15.40:0/2999113653 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5d40035d0 con 0x7fb5e8153720 2024-09-13T17:58:50.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ed0fc700 1 -- 172.21.15.40:0/2999113653 <== mon.1 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 981987427 0 0) 0x7fb5d40035d0 con 0x7fb5e8153720 2024-09-13T17:58:50.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ed0fc700 1 -- 172.21.15.40:0/2999113653 >> v2:172.21.15.40:3300/0 conn(0x7fb5e80cf9b0 msgr2=0x7fb5e806c070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:58:50.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ed0fc700 1 --2- 172.21.15.40:0/2999113653 >> v2:172.21.15.40:3300/0 conn(0x7fb5e80cf9b0 0x7fb5e806c070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ed0fc700 1 -- 172.21.15.40:0/2999113653 >> v1:172.21.15.40:6789/0 conn(0x7fb5e806c5b0 legacy=0x7fb5e8163d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:58:50.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ed0fc700 1 -- 172.21.15.40:0/2999113653 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e80cac20 con 0x7fb5e8153720 2024-09-13T17:58:50.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ed0fc700 1 -- 172.21.15.40:0/2999113653 <== mon.1 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fb5e4001670 con 0x7fb5e8153720 2024-09-13T17:58:50.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ed0fc700 1 -- 172.21.15.40:0/2999113653 >> v1:172.21.15.43:6789/0 conn(0x7fb5e8153720 legacy=0x7fb5e80ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ed0fc700 1 --2- 172.21.15.40:0/2999113653 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5d4003a20 0x7fb5d4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ed0fc700 1 --2- 172.21.15.40:0/2999113653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5d4004870 0x7fb5d4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ed0fc700 1 --2- 172.21.15.40:0/2999113653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5d4005280 0x7fb5d400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ed0fc700 1 -- 172.21.15.40:0/2999113653 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e80c5ea0 con 0x7fb5d4005280 2024-09-13T17:58:50.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ed0fc700 1 -- 172.21.15.40:0/2999113653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d400c190 con 0x7fb5d4004870 2024-09-13T17:58:50.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ed0fc700 1 -- 172.21.15.40:0/2999113653 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d400c350 con 0x7fb5d4003a20 2024-09-13T17:58:50.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ee8ff700 1 -- 172.21.15.40:0/2999113653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5d4005280 msgr2=0x7fb5d400ba30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:58:50.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ee8ff700 1 --2- 172.21.15.40:0/2999113653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5d4005280 0x7fb5d400ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:50.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ed8fd700 1 --2- 172.21.15.40:0/2999113653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5d4004870 0x7fb5d4004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.243+0000 7fb5ee0fe700 1 --2- 172.21.15.40:0/2999113653 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5d4003a20 0x7fb5d4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.247+0000 7fb5ed8fd700 1 -- 172.21.15.40:0/2999113653 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5d4003a20 msgr2=0x7fb5d4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:58:50.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.247+0000 7fb5ed8fd700 1 --2- 172.21.15.40:0/2999113653 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5d4003a20 0x7fb5d4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.247+0000 7fb5ed8fd700 1 -- 172.21.15.40:0/2999113653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5d4005280 msgr2=0x7fb5d400ba30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:58:50.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.247+0000 7fb5ed8fd700 1 --2- 172.21.15.40:0/2999113653 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5d4005280 0x7fb5d400ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.247+0000 7fb5ed8fd700 1 -- 172.21.15.40:0/2999113653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d40035d0 con 0x7fb5d4004870 2024-09-13T17:58:50.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.247+0000 7fb5ee8ff700 1 -- 172.21.15.40:0/2999113653 reap_dead start 2024-09-13T17:58:50.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.247+0000 7fb5ed8fd700 1 --2- 172.21.15.40:0/2999113653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5d4004870 0x7fb5d4004cf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009e60 tx=0x7fb5d40058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2bf6bc4ff7daa13 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:50.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.247+0000 7fb5ed0fc700 1 -- 172.21.15.40:0/2999113653 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d80098d0 con 0x7fb5d4004870 2024-09-13T17:58:50.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.247+0000 7fb5ed0fc700 1 -- 172.21.15.40:0/2999113653 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d800bcf0 con 0x7fb5d4004870 2024-09-13T17:58:50.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.247+0000 7fb5ed0fc700 1 -- 172.21.15.40:0/2999113653 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8003ca0 con 0x7fb5d4004870 2024-09-13T17:58:50.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.247+0000 7fb5ef100700 1 -- 172.21.15.40:0/2999113653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5d4004870 msgr2=0x7fb5d4004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.247+0000 7fb5ef100700 1 --2- 172.21.15.40:0/2999113653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5d4004870 0x7fb5d4004cf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009e60 tx=0x7fb5d40058a0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.247+0000 7fb5ef100700 1 -- 172.21.15.40:0/2999113653 shutdown_connections 2024-09-13T17:58:50.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.247+0000 7fb5ef100700 1 --2- 172.21.15.40:0/2999113653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5d4004870 0x7fb5d4004cf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.247+0000 7fb5ef100700 1 -- 172.21.15.40:0/2999113653 >> 172.21.15.40:0/2999113653 conn(0x7fb5e80666c0 msgr2=0x7fb5e80cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:50.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.247+0000 7fb5ef100700 1 -- 172.21.15.40:0/2999113653 shutdown_connections 2024-09-13T17:58:50.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.247+0000 7fb5ef100700 1 -- 172.21.15.40:0/2999113653 wait complete. 2024-09-13T17:58:50.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.247+0000 7fb5ef100700 1 Processor -- start 2024-09-13T17:58:50.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.247+0000 7fb5ef100700 1 -- start start 2024-09-13T17:58:50.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.247+0000 7fb5ef100700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5d4004870 0x7fb5e814e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ef100700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e814fb20 0x7fb5e81572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ef100700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e815a590 0x7fb5e8149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ef100700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e81effb0 con 0x7fb5e814fb20 2024-09-13T17:58:50.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ef100700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e8150310 con 0x7fb5e815a590 2024-09-13T17:58:50.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ef100700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e81505f0 con 0x7fb5d4004870 2024-09-13T17:58:50.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ed8fd700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e814fb20 msgr2=0x7fb5e81572a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T17:58:50.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ed8fd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e814fb20 0x7fb5e81572a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:58:50.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ee8ff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e815a590 0x7fb5e8149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ee0fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5d4004870 0x7fb5e814e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ee8ff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e815a590 0x7fb5e8149630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:34056/0 (socket says 172.21.15.40:34056) 2024-09-13T17:58:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ee8ff700 1 -- 172.21.15.40:0/1575156683 learned_addr learned my addr 172.21.15.40:0/1575156683 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:58:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ee8ff700 1 -- 172.21.15.40:0/1575156683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5d4004870 msgr2=0x7fb5e814e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ee8ff700 1 --2- 172.21.15.40:0/1575156683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5d4004870 0x7fb5e814e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ee8ff700 1 -- 172.21.15.40:0/1575156683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e814fb20 msgr2=0x7fb5e81572a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:58:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ee8ff700 1 --2- 172.21.15.40:0/1575156683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5e814fb20 0x7fb5e81572a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ee8ff700 1 -- 172.21.15.40:0/1575156683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e81ef960 con 0x7fb5e815a590 2024-09-13T17:58:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ee8ff700 1 --2- 172.21.15.40:0/1575156683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e815a590 0x7fb5e8149630 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb5e806d2e0 tx=0x7fb5e000bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5deffd700 1 -- 172.21.15.40:0/1575156683 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e000c9f0 con 0x7fb5e815a590 2024-09-13T17:58:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5deffd700 1 -- 172.21.15.40:0/1575156683 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e000cb70 con 0x7fb5e815a590 2024-09-13T17:58:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ef100700 1 -- 172.21.15.40:0/1575156683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e81efc40 con 0x7fb5e815a590 2024-09-13T17:58:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5deffd700 1 -- 172.21.15.40:0/1575156683 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0015470 con 0x7fb5e815a590 2024-09-13T17:58:50.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ef100700 1 -- 172.21.15.40:0/1575156683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e814ffa0 con 0x7fb5e815a590 2024-09-13T17:58:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ef100700 1 --2- 172.21.15.40:0/1575156683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5e815aa10 0x7fb5e81f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ef100700 1 -- 172.21.15.40:0/1575156683 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e8074750 con 0x7fb5e815aa10 2024-09-13T17:58:50.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ee0fe700 1 --2- 172.21.15.40:0/1575156683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5e815aa10 0x7fb5e81f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.251+0000 7fb5ee0fe700 1 -- 172.21.15.40:0/1575156683 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb5e8153b00 con 0x7fb5e815aa10 2024-09-13T17:58:50.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.255+0000 7fb5ee0fe700 1 --2- 172.21.15.40:0/1575156683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5e815aa10 0x7fb5e81f2bc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4007250 tx=0x7fb5e4007610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:50.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.255+0000 7fb5deffd700 1 -- 172.21.15.40:0/1575156683 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4007850 con 0x7fb5e815aa10 2024-09-13T17:58:50.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.255+0000 7fb5deffd700 1 -- 172.21.15.40:0/1575156683 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb5e0012070 con 0x7fb5e815a590 2024-09-13T17:58:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.255+0000 7fb5deffd700 1 --2- 172.21.15.40:0/1575156683 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb5cc066770 0x7fb5cc068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.255+0000 7fb5deffd700 1 -- 172.21.15.40:0/1575156683 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb5e0083fe0 con 0x7fb5e815a590 2024-09-13T17:58:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.255+0000 7fb5ed8fd700 1 --2- 172.21.15.40:0/1575156683 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb5cc066770 0x7fb5cc068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.255+0000 7fb5deffd700 1 -- 172.21.15.40:0/1575156683 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb5e4012840 con 0x7fb5e815aa10 2024-09-13T17:58:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.255+0000 7fb5deffd700 1 -- 172.21.15.40:0/1575156683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5e815aa10 msgr2=0x7fb5e81f2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.255+0000 7fb5deffd700 1 --2- 172.21.15.40:0/1575156683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5e815aa10 0x7fb5e81f2bc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4007250 tx=0x7fb5e4007610 comp rx=0 tx=0).stop 2024-09-13T17:58:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.255+0000 7fb5deffd700 1 -- 172.21.15.40:0/1575156683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5e815aa10 msgr2=0x7fb5e81f2bc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:58:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.255+0000 7fb5deffd700 1 --2- 172.21.15.40:0/1575156683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5e815aa10 0x7fb5e81f2bc0 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4007250 tx=0x7fb5e4007610 comp rx=0 tx=0).stop 2024-09-13T17:58:50.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.255+0000 7fb5ed8fd700 1 --2- 172.21.15.40:0/1575156683 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb5cc066770 0x7fb5cc068c30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb5e814a560 tx=0x7fb5d8019040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:50.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.263+0000 7fb5ef100700 1 --2- 172.21.15.40:0/1575156683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5e81f3100 0x7fb5e81f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:58:50.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.263+0000 7fb5ef100700 1 -- 172.21.15.40:0/1575156683 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e81efe30 con 0x7fb5e81f3100 2024-09-13T17:58:50.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.263+0000 7fb5ee0fe700 1 --2- 172.21.15.40:0/1575156683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5e81f3100 0x7fb5e81f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:58:50.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.263+0000 7fb5ee0fe700 1 -- 172.21.15.40:0/1575156683 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb5e815a340 con 0x7fb5e81f3100 2024-09-13T17:58:50.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.263+0000 7fb5ee0fe700 1 --2- 172.21.15.40:0/1575156683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5e81f3100 0x7fb5e81f3510 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4007250 tx=0x7fb5e40019c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:58:50.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.263+0000 7fb5deffd700 1 -- 172.21.15.40:0/1575156683 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4024070 con 0x7fb5e81f3100 2024-09-13T17:58:50.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.267+0000 7fb5deffd700 1 -- 172.21.15.40:0/1575156683 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4936 (secure 0 0 0) 0x7fb5e4015650 con 0x7fb5e81f3100 2024-09-13T17:58:50.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.267+0000 7fb5deffd700 1 -- 172.21.15.40:0/1575156683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5e81f3100 msgr2=0x7fb5e81f3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.267+0000 7fb5deffd700 1 --2- 172.21.15.40:0/1575156683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5e81f3100 0x7fb5e81f3510 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4007250 tx=0x7fb5e40019c0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.267+0000 7fb5deffd700 1 -- 172.21.15.40:0/1575156683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5e81f3100 msgr2=0x7fb5e81f3510 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:58:50.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.267+0000 7fb5deffd700 1 --2- 172.21.15.40:0/1575156683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5e81f3100 0x7fb5e81f3510 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:58:50.275 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:58:50.275 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:58:50.275 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:58:50.275 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:58:50.275 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 66, 2024-09-13T17:58:50.275 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:58:50.275 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:58:50.275 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:58:50.276 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:50.276 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T17:58:50.276 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:58:50.276 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:58:50.276 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:58:50.276 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:50.276 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:50.276 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:50.276 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:50.276 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:50.276 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:50.277 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:50.277 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:50.277 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:50.277 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:50.277 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:50.277 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:58:50.277 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:58:50.277 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:50.277 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:50.277 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:50.277 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:50.278 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:50.278 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:50.278 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:50.278 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:50.278 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:50.278 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:50.278 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:50.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.278 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:58:50.278 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:58:50.278 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:58:50.278 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:58:50.279 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:58:50.279 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:58:50.279 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:58:50.279 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:58:50.279 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:58:50.279 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:58:50.279 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:58:50.279 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:58:50.279 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:58:50.279 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:58:50.279 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:58:50.279 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:58:50.280 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:58:50.280 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:58:50.280 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:58:50.280 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:58:50.280 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:58:50.280 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:58:50.280 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:58:50.280 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:58:50.280 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:58:50.280 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:58:50.280 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:58:50.280 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:50.281 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:58:50.281 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.281 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:58:50.281 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.281 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:58:50.281 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:58:50.281 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:50.281 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:50.281 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.281 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:50.281 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:58:50.281 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:50.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.282 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.282 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:50.282 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:58:50.282 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:50.282 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:50.282 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:50.282 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.282 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:58:50.282 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:58:50.282 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:50.283 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:50.283 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:50.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.283 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.283 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:58:50.283 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:58:50.283 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:50.283 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:50.283 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.283 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:50.283 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:58:50.283 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:50.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.284 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.284 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:50.284 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:58:50.284 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:50.284 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:50.284 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:50.284 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.284 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:58:50.284 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:58:50.284 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:50.284 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:50.284 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:50.285 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.285 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.285 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:58:50.285 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:58:50.285 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:58:50.285 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:58:50.285 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.285 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:58:50.285 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:58:50.285 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:50.285 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.285 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.286 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:58:50.286 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:58:50.286 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:58:50.286 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:50.286 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:50.286 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.286 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:58:50.286 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:58:50.286 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:58:50.286 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:58:50.286 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:58:50.286 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:50.287 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:50.287 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.287 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:58:50.287 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:58:50.287 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.287 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:50.287 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:50.287 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:50.287 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:50.287 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:50.287 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:58:50.287 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.288 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:50.288 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:50.288 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:58:50.288 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:50.288 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:50.288 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:58:50.288 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.288 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:50.288 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:50.288 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T17:58:50.288 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:50.288 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:50.289 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:58:50.289 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.289 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:50.289 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:50.289 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:58:50.289 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:50.289 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:58:50.289 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:58:50.289 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:58:50.289 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:58:50.289 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:58:50.289 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:58:50.290 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:58:50.290 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:58:50.290 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:58:50.290 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:58:50.290 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:58:50.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.267+0000 7fb5ef100700 1 -- 172.21.15.40:0/1575156683 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb5cc066770 msgr2=0x7fb5cc068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.267+0000 7fb5ef100700 1 --2- 172.21.15.40:0/1575156683 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb5cc066770 0x7fb5cc068c30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb5e814a560 tx=0x7fb5d8019040 comp rx=0 tx=0).stop 2024-09-13T17:58:50.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.267+0000 7fb5ef100700 1 -- 172.21.15.40:0/1575156683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e815a590 msgr2=0x7fb5e8149630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:58:50.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.267+0000 7fb5ef100700 1 --2- 172.21.15.40:0/1575156683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e815a590 0x7fb5e8149630 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb5e806d2e0 tx=0x7fb5e000bd50 comp rx=0 tx=0).stop 2024-09-13T17:58:50.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.267+0000 7fb5ee8ff700 1 -- 172.21.15.40:0/1575156683 reap_dead start 2024-09-13T17:58:50.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.267+0000 7fb5ee8ff700 1 -- 172.21.15.40:0/1575156683 reap_dead start 2024-09-13T17:58:50.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.267+0000 7fb5ef100700 1 -- 172.21.15.40:0/1575156683 shutdown_connections 2024-09-13T17:58:50.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.267+0000 7fb5ef100700 1 -- 172.21.15.40:0/1575156683 >> 172.21.15.40:0/1575156683 conn(0x7fb5e80666c0 msgr2=0x7fb5e80cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:58:50.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.267+0000 7fb5ef100700 1 -- 172.21.15.40:0/1575156683 shutdown_connections 2024-09-13T17:58:50.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:58:50.267+0000 7fb5ef100700 1 -- 172.21.15.40:0/1575156683 wait complete. 2024-09-13T17:58:50.292 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T17:58:50.293 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T17:59:05.109 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:59:05.112 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:59:10.293 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T17:59:10.293 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-13T17:59:10.294 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T17:59:10.294 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T17:59:10.296 INFO:tasks.ceph.mon.a:Started 2024-09-13T17:59:10.296 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T17:59:10.297 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:59:10.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.410+0000 7f08cffff700 1 Processor -- start 2024-09-13T17:59:10.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.410+0000 7f08cffff700 1 -- start start 2024-09-13T17:59:10.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.410+0000 7f08cffff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f08d0152fe0 0x7f08d00bfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.410+0000 7f08cffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f08d00c73d0 0x7f08d006ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.410+0000 7f08cffff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d01533c0 con 0x7f08d006b160 2024-09-13T17:59:10.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.410+0000 7f08cffff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d006fec0 con 0x7f08d00c73d0 2024-09-13T17:59:10.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.410+0000 7f08cffff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d006fc00 con 0x7f08d0152fe0 2024-09-13T17:59:10.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.410+0000 7f08c7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f08d00c73d0 0x7f08d006ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:10.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.410+0000 7f08cf7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f08d006b160 0x7f08d00c07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47168/0 (socket says 172.21.15.40:47168) 2024-09-13T17:59:10.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.410+0000 7f08cf7fe700 1 -- 172.21.15.40:0/2247576534 learned_addr learned my addr 172.21.15.40:0/2247576534 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:59:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.410+0000 7f08c7fff700 1 -- 172.21.15.40:0/2247576534 >> v2:172.21.15.40:3300/0 conn(0x7f08d00c73d0 msgr2=0x7f08d006ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T17:59:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.410+0000 7f08c7fff700 1 -- 172.21.15.40:0/2247576534 >> v2:172.21.15.40:3300/0 conn(0x7f08d00c73d0 msgr2=0x7f08d006ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:59:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.410+0000 7f08c7fff700 1 --2- 172.21.15.40:0/2247576534 >> v2:172.21.15.40:3300/0 conn(0x7f08d00c73d0 0x7f08d006ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:59:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.410+0000 7f08c7fff700 1 --2- 172.21.15.40:0/2247576534 >> v2:172.21.15.40:3300/0 conn(0x7f08d00c73d0 0x7f08d006ac20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:59:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.410+0000 7f08ceffd700 1 --2- 172.21.15.40:0/2247576534 >> v2:172.21.15.43:3300/0 conn(0x7f08d0152fe0 0x7f08d00bfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.410+0000 7f08ceffd700 1 -- 172.21.15.40:0/2247576534 >> v2:172.21.15.40:3300/0 conn(0x7f08d00c73d0 msgr2=0x7f08d006ac20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:59:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.410+0000 7f08ceffd700 1 --2- 172.21.15.40:0/2247576534 >> v2:172.21.15.40:3300/0 conn(0x7f08d00c73d0 0x7f08d006ac20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.410+0000 7f08ceffd700 1 -- 172.21.15.40:0/2247576534 >> v1:172.21.15.40:6789/0 conn(0x7f08d006b160 legacy=0x7f08d00c07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.410+0000 7f08ceffd700 1 -- 172.21.15.40:0/2247576534 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d0071ef0 con 0x7f08d0152fe0 2024-09-13T17:59:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.414+0000 7f08ceffd700 1 --2- 172.21.15.40:0/2247576534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d0152fe0 0x7f08d00bfe80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f08d00720e0 tx=0x7f08c0009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7f6608e20a34c86 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.414+0000 7f08ce7fc700 1 -- 172.21.15.40:0/2247576534 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0004020 con 0x7f08d0152fe0 2024-09-13T17:59:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.414+0000 7f08ce7fc700 1 -- 172.21.15.40:0/2247576534 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c0004180 con 0x7f08d0152fe0 2024-09-13T17:59:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.414+0000 7f08ce7fc700 1 -- 172.21.15.40:0/2247576534 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0004530 con 0x7f08d0152fe0 2024-09-13T17:59:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.414+0000 7f08cffff700 1 -- 172.21.15.40:0/2247576534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d0152fe0 msgr2=0x7f08d00bfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.414+0000 7f08cffff700 1 --2- 172.21.15.40:0/2247576534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d0152fe0 0x7f08d00bfe80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f08d00720e0 tx=0x7f08c0009b30 comp rx=0 tx=0).stop 2024-09-13T17:59:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.414+0000 7f08cffff700 1 -- 172.21.15.40:0/2247576534 shutdown_connections 2024-09-13T17:59:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.414+0000 7f08cffff700 1 --2- 172.21.15.40:0/2247576534 >> v2:172.21.15.40:3300/0 conn(0x7f08d00c73d0 0x7f08d006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.414+0000 7f08cffff700 1 --2- 172.21.15.40:0/2247576534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d0152fe0 0x7f08d00bfe80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.414+0000 7f08cffff700 1 -- 172.21.15.40:0/2247576534 >> 172.21.15.40:0/2247576534 conn(0x7f08d006d360 msgr2=0x7f08d006d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.414+0000 7f08cffff700 1 -- 172.21.15.40:0/2247576534 shutdown_connections 2024-09-13T17:59:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.414+0000 7f08cffff700 1 -- 172.21.15.40:0/2247576534 wait complete. 2024-09-13T17:59:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.414+0000 7f08cffff700 1 Processor -- start 2024-09-13T17:59:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.414+0000 7f08cffff700 1 -- start start 2024-09-13T17:59:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08cffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f08d006b160 0x7f08d0148a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08cffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d00c73d0 0x7f08d0145b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08cffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d0152fe0 0x7f08d0146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08cffff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d00b0b40 con 0x7f08d00c73d0 2024-09-13T17:59:10.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08cffff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d01425e0 con 0x7f08d0152fe0 2024-09-13T17:59:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08cffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d0149b50 con 0x7f08d006b160 2024-09-13T17:59:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08c7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d00c73d0 0x7f08d0145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08c7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d00c73d0 0x7f08d0145b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48334/0 (socket says 172.21.15.40:48334) 2024-09-13T17:59:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08c7fff700 1 -- 172.21.15.40:0/3334164962 learned_addr learned my addr 172.21.15.40:0/3334164962 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08ceffd700 1 --2- 172.21.15.40:0/3334164962 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f08d006b160 0x7f08d0148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08c7fff700 1 -- 172.21.15.40:0/3334164962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d00c73d0 msgr2=0x7f08d0145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T17:59:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08c7fff700 1 -- 172.21.15.40:0/3334164962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d00c73d0 msgr2=0x7f08d0145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:59:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08c7fff700 1 --2- 172.21.15.40:0/3334164962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d00c73d0 0x7f08d0145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:59:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08cf7fe700 1 --2- 172.21.15.40:0/3334164962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d0152fe0 0x7f08d0146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08c7fff700 1 --2- 172.21.15.40:0/3334164962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d00c73d0 0x7f08d0145b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:59:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08ceffd700 1 -- 172.21.15.40:0/3334164962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d0152fe0 msgr2=0x7f08d0146ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08ceffd700 1 --2- 172.21.15.40:0/3334164962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d0152fe0 0x7f08d0146ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08ceffd700 1 -- 172.21.15.40:0/3334164962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d00c73d0 msgr2=0x7f08d0145b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:59:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08ceffd700 1 --2- 172.21.15.40:0/3334164962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d00c73d0 0x7f08d0145b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08ceffd700 1 -- 172.21.15.40:0/3334164962 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c00094b0 con 0x7f08d006b160 2024-09-13T17:59:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08ceffd700 1 --2- 172.21.15.40:0/3334164962 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f08d006b160 0x7f08d0148a70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f08c0015000 tx=0x7f08c0004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08c77fe700 1 -- 172.21.15.40:0/3334164962 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0003cf0 con 0x7f08d006b160 2024-09-13T17:59:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08c77fe700 1 -- 172.21.15.40:0/3334164962 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c0003e50 con 0x7f08d006b160 2024-09-13T17:59:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08cffff700 1 -- 172.21.15.40:0/3334164962 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d0142b30 con 0x7f08d006b160 2024-09-13T17:59:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08c77fe700 1 -- 172.21.15.40:0/3334164962 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0011850 con 0x7f08d006b160 2024-09-13T17:59:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.418+0000 7f08cffff700 1 -- 172.21.15.40:0/3334164962 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d01466f0 con 0x7f08d006b160 2024-09-13T17:59:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.422+0000 7f08cffff700 1 -- 172.21.15.40:0/3334164962 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d00c5c20 con 0x7f08d006b160 2024-09-13T17:59:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.422+0000 7f08c77fe700 1 -- 172.21.15.40:0/3334164962 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f08c00119b0 con 0x7f08d006b160 2024-09-13T17:59:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.422+0000 7f08c77fe700 1 --2- 172.21.15.40:0/3334164962 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f08b0066760 0x7f08b0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.422+0000 7f08c77fe700 1 -- 172.21.15.40:0/3334164962 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f08c0085710 con 0x7f08d006b160 2024-09-13T17:59:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.422+0000 7f08c7fff700 1 --2- 172.21.15.40:0/3334164962 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f08b0066760 0x7f08b0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.426+0000 7f08c7fff700 1 --2- 172.21.15.40:0/3334164962 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f08b0066760 0x7f08b0068c20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f08d006d7f0 tx=0x7f08b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:10.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.434+0000 7f08c77fe700 1 -- 172.21.15.40:0/3334164962 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f08c00528a0 con 0x7f08d006b160 2024-09-13T17:59:10.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.722+0000 7f08cffff700 1 -- 172.21.15.40:0/3334164962 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f08d006dbc0 con 0x7f08d006b160 2024-09-13T17:59:10.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.722+0000 7f08c77fe700 1 -- 172.21.15.40:0/3334164962 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f08c0055ec0 con 0x7f08d006b160 2024-09-13T17:59:10.731 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":66,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:59:10.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.722+0000 7f08cffff700 1 -- 172.21.15.40:0/3334164962 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f08b0066760 msgr2=0x7f08b0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.722+0000 7f08cffff700 1 --2- 172.21.15.40:0/3334164962 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f08b0066760 0x7f08b0068c20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f08d006d7f0 tx=0x7f08b8008040 comp rx=0 tx=0).stop 2024-09-13T17:59:10.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.722+0000 7f08cffff700 1 -- 172.21.15.40:0/3334164962 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f08d006b160 msgr2=0x7f08d0148a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:10.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.722+0000 7f08cffff700 1 --2- 172.21.15.40:0/3334164962 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f08d006b160 0x7f08d0148a70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f08c0015000 tx=0x7f08c0004770 comp rx=0 tx=0).stop 2024-09-13T17:59:10.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.722+0000 7f08cffff700 1 -- 172.21.15.40:0/3334164962 shutdown_connections 2024-09-13T17:59:10.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.722+0000 7f08cffff700 1 --2- 172.21.15.40:0/3334164962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d0152fe0 0x7f08d0146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.722+0000 7f08cffff700 1 --2- 172.21.15.40:0/3334164962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d00c73d0 0x7f08d0145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.722+0000 7f08cffff700 1 --2- 172.21.15.40:0/3334164962 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f08b0066760 0x7f08b0068c20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.722+0000 7f08cffff700 1 --2- 172.21.15.40:0/3334164962 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f08d006b160 0x7f08d0148a70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:10.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.722+0000 7f08cffff700 1 -- 172.21.15.40:0/3334164962 >> 172.21.15.40:0/3334164962 conn(0x7f08d006d360 msgr2=0x7f08d0150900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:10.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.722+0000 7f08cffff700 1 -- 172.21.15.40:0/3334164962 shutdown_connections 2024-09-13T17:59:10.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:10.726+0000 7f08cffff700 1 -- 172.21.15.40:0/3334164962 wait complete. 2024-09-13T17:59:13.747 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:59:13.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.850+0000 7f43891a2700 1 Processor -- start 2024-09-13T17:59:13.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.850+0000 7f43891a2700 1 -- start start 2024-09-13T17:59:13.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.850+0000 7f43891a2700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f43841503d0 0x7f43840c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:13.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.850+0000 7f43891a2700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43840c1580 con 0x7f43840c67e0 2024-09-13T17:59:13.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.850+0000 7f43891a2700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43840c2a50 con 0x7f43840b11b0 2024-09-13T17:59:13.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.850+0000 7f43891a2700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43840c2480 con 0x7f43841503d0 2024-09-13T17:59:13.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.850+0000 7f437bfff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f43840c67e0 0x7f43840c7200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50076/0 (socket says 172.21.15.40:50076) 2024-09-13T17:59:13.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.850+0000 7f437bfff700 1 -- 172.21.15.40:0/2652411358 learned_addr learned my addr 172.21.15.40:0/2652411358 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:59:13.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.854+0000 7f4383fff700 1 --2- 172.21.15.40:0/2652411358 >> v2:172.21.15.43:3300/0 conn(0x7f43841503d0 0x7f43840c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:13.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.854+0000 7f4383fff700 1 -- 172.21.15.40:0/2652411358 >> v1:172.21.15.43:6789/0 conn(0x7f43840b11b0 legacy=0x7f4384163dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:13.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.854+0000 7f4383fff700 1 -- 172.21.15.40:0/2652411358 >> v1:172.21.15.40:6789/0 conn(0x7f43840c67e0 legacy=0x7f43840c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:13.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.854+0000 7f4383fff700 1 -- 172.21.15.40:0/2652411358 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43840c2cc0 con 0x7f43841503d0 2024-09-13T17:59:13.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.854+0000 7f4383fff700 1 --2- 172.21.15.40:0/2652411358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43841503d0 0x7f43840c7d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4374009850 tx=0x7f4374009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7ca38587b4d52fa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:13.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.854+0000 7f43837fe700 1 -- 172.21.15.40:0/2652411358 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4374011830 con 0x7f43841503d0 2024-09-13T17:59:13.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.854+0000 7f43837fe700 1 -- 172.21.15.40:0/2652411358 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4374011990 con 0x7f43841503d0 2024-09-13T17:59:13.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.854+0000 7f43837fe700 1 -- 172.21.15.40:0/2652411358 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4374011d20 con 0x7f43841503d0 2024-09-13T17:59:13.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.854+0000 7f43891a2700 1 -- 172.21.15.40:0/2652411358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43841503d0 msgr2=0x7f43840c7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:13.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.854+0000 7f43891a2700 1 --2- 172.21.15.40:0/2652411358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43841503d0 0x7f43840c7d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4374009850 tx=0x7f4374009b60 comp rx=0 tx=0).stop 2024-09-13T17:59:13.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.854+0000 7f43891a2700 1 -- 172.21.15.40:0/2652411358 shutdown_connections 2024-09-13T17:59:13.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.854+0000 7f43891a2700 1 --2- 172.21.15.40:0/2652411358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43841503d0 0x7f43840c7d80 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4374009850 tx=0x7f4374009b60 comp rx=0 tx=0).stop 2024-09-13T17:59:13.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.854+0000 7f43891a2700 1 -- 172.21.15.40:0/2652411358 >> 172.21.15.40:0/2652411358 conn(0x7f43840c88e0 msgr2=0x7f43840c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:13.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.854+0000 7f43891a2700 1 -- 172.21.15.40:0/2652411358 shutdown_connections 2024-09-13T17:59:13.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.854+0000 7f43891a2700 1 -- 172.21.15.40:0/2652411358 wait complete. 2024-09-13T17:59:13.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f43891a2700 1 Processor -- start 2024-09-13T17:59:13.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f43891a2700 1 -- start start 2024-09-13T17:59:13.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f43891a2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43840b11b0 0x7f43840addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:13.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f43891a2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43840c67e0 0x7f4384158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:13.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f43891a2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f438406ba10 0x7f4384159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:13.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f43891a2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438406a710 con 0x7f43840b11b0 2024-09-13T17:59:13.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f43891a2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43840ad920 con 0x7f438406ba10 2024-09-13T17:59:13.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f43891a2700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43840ae4e0 con 0x7f43840c67e0 2024-09-13T17:59:13.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f4383fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43840b11b0 0x7f43840addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:13.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f4383fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43840b11b0 0x7f43840addc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38152/0 (socket says 172.21.15.40:38152) 2024-09-13T17:59:13.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f4383fff700 1 -- 172.21.15.40:0/2486881352 learned_addr learned my addr 172.21.15.40:0/2486881352 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:13.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f43889a1700 1 --2- 172.21.15.40:0/2486881352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f438406ba10 0x7f4384159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:13.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f4383fff700 1 -- 172.21.15.40:0/2486881352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43840b11b0 msgr2=0x7f43840addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T17:59:13.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f4383fff700 1 -- 172.21.15.40:0/2486881352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43840b11b0 msgr2=0x7f43840addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T17:59:13.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f4383fff700 1 --2- 172.21.15.40:0/2486881352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43840b11b0 0x7f43840addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:59:13.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f437bfff700 1 --2- 172.21.15.40:0/2486881352 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43840c67e0 0x7f4384158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:13.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f4383fff700 1 --2- 172.21.15.40:0/2486881352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43840b11b0 0x7f43840addc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:59:13.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f437bfff700 1 -- 172.21.15.40:0/2486881352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f438406ba10 msgr2=0x7f4384159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:13.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f437bfff700 1 --2- 172.21.15.40:0/2486881352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f438406ba10 0x7f4384159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:13.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f437bfff700 1 -- 172.21.15.40:0/2486881352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43840b11b0 msgr2=0x7f43840addc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:59:13.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f437bfff700 1 --2- 172.21.15.40:0/2486881352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43840b11b0 0x7f43840addc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:13.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f437bfff700 1 -- 172.21.15.40:0/2486881352 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43740094b0 con 0x7f43840c67e0 2024-09-13T17:59:13.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f43889a1700 1 --2- 172.21.15.40:0/2486881352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f438406ba10 0x7f4384159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:59:13.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f437bfff700 1 --2- 172.21.15.40:0/2486881352 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43840c67e0 0x7f4384158980 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f436c00c460 tx=0x7f436c00c770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:13.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.858+0000 7f43817fa700 1 -- 172.21.15.40:0/2486881352 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c00cea0 con 0x7f43840c67e0 2024-09-13T17:59:13.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.862+0000 7f43891a2700 1 -- 172.21.15.40:0/2486881352 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438406c220 con 0x7f43840c67e0 2024-09-13T17:59:13.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.862+0000 7f43817fa700 1 -- 172.21.15.40:0/2486881352 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436c001140 con 0x7f43840c67e0 2024-09-13T17:59:13.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.862+0000 7f43891a2700 1 -- 172.21.15.40:0/2486881352 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4384157880 con 0x7f43840c67e0 2024-09-13T17:59:13.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.862+0000 7f43817fa700 1 -- 172.21.15.40:0/2486881352 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436c0070d0 con 0x7f43840c67e0 2024-09-13T17:59:13.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.862+0000 7f43891a2700 1 -- 172.21.15.40:0/2486881352 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4384156e00 con 0x7f43840c67e0 2024-09-13T17:59:13.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.862+0000 7f43817fa700 1 -- 172.21.15.40:0/2486881352 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f436c007230 con 0x7f43840c67e0 2024-09-13T17:59:13.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.862+0000 7f43817fa700 1 --2- 172.21.15.40:0/2486881352 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4364066730 0x7f4364068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:13.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.862+0000 7f43817fa700 1 -- 172.21.15.40:0/2486881352 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f436c083680 con 0x7f43840c67e0 2024-09-13T17:59:13.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.862+0000 7f4383fff700 1 --2- 172.21.15.40:0/2486881352 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4364066730 0x7f4364068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:13.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.866+0000 7f4383fff700 1 --2- 172.21.15.40:0/2486881352 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4364066730 0x7f4364068bf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4374009ba0 tx=0x7f4374008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:13.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:13.874+0000 7f43817fa700 1 -- 172.21.15.40:0/2486881352 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f436c0506f0 con 0x7f43840c67e0 2024-09-13T17:59:14.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:14.138+0000 7f43891a2700 1 -- 172.21.15.40:0/2486881352 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f43840c1710 con 0x7f43840c67e0 2024-09-13T17:59:14.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:14.138+0000 7f43817fa700 1 -- 172.21.15.40:0/2486881352 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f436c053d10 con 0x7f43840c67e0 2024-09-13T17:59:14.149 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":66,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":24,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:59:14.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:14.142+0000 7f43891a2700 1 -- 172.21.15.40:0/2486881352 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4364066730 msgr2=0x7f4364068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:14.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:14.142+0000 7f43891a2700 1 --2- 172.21.15.40:0/2486881352 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4364066730 0x7f4364068bf0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4374009ba0 tx=0x7f4374008030 comp rx=0 tx=0).stop 2024-09-13T17:59:14.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:14.142+0000 7f43891a2700 1 -- 172.21.15.40:0/2486881352 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43840c67e0 msgr2=0x7f4384158980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:14.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:14.142+0000 7f43891a2700 1 --2- 172.21.15.40:0/2486881352 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43840c67e0 0x7f4384158980 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f436c00c460 tx=0x7f436c00c770 comp rx=0 tx=0).stop 2024-09-13T17:59:14.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:14.142+0000 7f43891a2700 1 -- 172.21.15.40:0/2486881352 shutdown_connections 2024-09-13T17:59:14.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:14.142+0000 7f43891a2700 1 --2- 172.21.15.40:0/2486881352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f438406ba10 0x7f4384159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:14.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:14.142+0000 7f43891a2700 1 --2- 172.21.15.40:0/2486881352 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4364066730 0x7f4364068bf0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:14.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:14.142+0000 7f43891a2700 1 --2- 172.21.15.40:0/2486881352 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43840c67e0 0x7f4384158980 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:14.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:14.142+0000 7f43891a2700 1 --2- 172.21.15.40:0/2486881352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43840b11b0 0x7f43840addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:14.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:14.142+0000 7f43891a2700 1 -- 172.21.15.40:0/2486881352 >> 172.21.15.40:0/2486881352 conn(0x7f43840c88e0 msgr2=0x7f4384071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:14.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:14.142+0000 7f43891a2700 1 -- 172.21.15.40:0/2486881352 shutdown_connections 2024-09-13T17:59:14.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:14.142+0000 7f43891a2700 1 -- 172.21.15.40:0/2486881352 wait complete. 2024-09-13T17:59:17.165 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:59:17.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.286+0000 7fa4c9657700 1 Processor -- start 2024-09-13T17:59:17.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.286+0000 7fa4c9657700 1 -- start start 2024-09-13T17:59:17.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c9657700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4c4153630 0x7fa4c415d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c9657700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa4c40c1ea0 0x7fa4c4070b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c9657700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c40c0e90 con 0x7fa4c40c67e0 2024-09-13T17:59:17.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c9657700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c406f5f0 con 0x7fa4c40c1ea0 2024-09-13T17:59:17.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c9657700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c40c8360 con 0x7fa4c4153630 2024-09-13T17:59:17.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c37fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa4c40c1ea0 0x7fa4c4070b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c8e56700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa4c40c67e0 0x7fa4c40c7500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50092/0 (socket says 172.21.15.40:50092) 2024-09-13T17:59:17.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c8e56700 1 -- 172.21.15.40:0/1778083175 learned_addr learned my addr 172.21.15.40:0/1778083175 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:59:17.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c3fff700 1 --2- 172.21.15.40:0/1778083175 >> v2:172.21.15.43:3300/0 conn(0x7fa4c4153630 0x7fa4c415d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c37fe700 1 -- 172.21.15.40:0/1778083175 >> v2:172.21.15.43:3300/0 conn(0x7fa4c4153630 msgr2=0x7fa4c415d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:17.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c37fe700 1 --2- 172.21.15.40:0/1778083175 >> v2:172.21.15.43:3300/0 conn(0x7fa4c4153630 0x7fa4c415d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c37fe700 1 -- 172.21.15.40:0/1778083175 >> v1:172.21.15.40:6789/0 conn(0x7fa4c40c67e0 legacy=0x7fa4c40c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c37fe700 1 -- 172.21.15.40:0/1778083175 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c40c10e0 con 0x7fa4c40c1ea0 2024-09-13T17:59:17.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c37fe700 1 --2- 172.21.15.40:0/1778083175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c40c1ea0 0x7fa4c4070b90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009850 tx=0x7fa4b8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930eacc284a7da1c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:17.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c2ffd700 1 -- 172.21.15.40:0/1778083175 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8011830 con 0x7fa4c40c1ea0 2024-09-13T17:59:17.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c2ffd700 1 -- 172.21.15.40:0/1778083175 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b8011990 con 0x7fa4c40c1ea0 2024-09-13T17:59:17.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c2ffd700 1 -- 172.21.15.40:0/1778083175 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8011d20 con 0x7fa4c40c1ea0 2024-09-13T17:59:17.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c9657700 1 -- 172.21.15.40:0/1778083175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c40c1ea0 msgr2=0x7fa4c4070b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c9657700 1 --2- 172.21.15.40:0/1778083175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c40c1ea0 0x7fa4c4070b90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009850 tx=0x7fa4b8009b60 comp rx=0 tx=0).stop 2024-09-13T17:59:17.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c9657700 1 -- 172.21.15.40:0/1778083175 shutdown_connections 2024-09-13T17:59:17.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c9657700 1 --2- 172.21.15.40:0/1778083175 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c40c1ea0 0x7fa4c4070b90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c9657700 1 --2- 172.21.15.40:0/1778083175 >> v2:172.21.15.43:3300/0 conn(0x7fa4c4153630 0x7fa4c415d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c9657700 1 -- 172.21.15.40:0/1778083175 >> 172.21.15.40:0/1778083175 conn(0x7fa4c406e480 msgr2=0x7fa4c406e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:17.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c9657700 1 -- 172.21.15.40:0/1778083175 shutdown_connections 2024-09-13T17:59:17.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.290+0000 7fa4c9657700 1 -- 172.21.15.40:0/1778083175 wait complete. 2024-09-13T17:59:17.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c9657700 1 Processor -- start 2024-09-13T17:59:17.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c9657700 1 -- start start 2024-09-13T17:59:17.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c9657700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c40c1ea0 0x7fa4c4071230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c9657700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c40c67e0 0x7fa4c41473f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c9657700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c4153630 0x7fa4c40c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c9657700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c414b190 con 0x7fa4c40c67e0 2024-09-13T17:59:17.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c9657700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c4147930 con 0x7fa4c4153630 2024-09-13T17:59:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c9657700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c40718c0 con 0x7fa4c40c1ea0 2024-09-13T17:59:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c40c67e0 0x7fa4c41473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c40c67e0 0x7fa4c41473f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38184/0 (socket says 172.21.15.40:38184) 2024-09-13T17:59:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c37fe700 1 -- 172.21.15.40:0/3773060357 learned_addr learned my addr 172.21.15.40:0/3773060357 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c3fff700 1 --2- 172.21.15.40:0/3773060357 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c40c1ea0 0x7fa4c4071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c8e56700 1 --2- 172.21.15.40:0/3773060357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c4153630 0x7fa4c40c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c37fe700 1 -- 172.21.15.40:0/3773060357 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c40c1ea0 msgr2=0x7fa4c4071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c37fe700 1 --2- 172.21.15.40:0/3773060357 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c40c1ea0 0x7fa4c4071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c37fe700 1 -- 172.21.15.40:0/3773060357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c4153630 msgr2=0x7fa4c40c53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c37fe700 1 --2- 172.21.15.40:0/3773060357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c4153630 0x7fa4c40c53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c37fe700 1 -- 172.21.15.40:0/3773060357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b80094b0 con 0x7fa4c40c67e0 2024-09-13T17:59:17.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c37fe700 1 --2- 172.21.15.40:0/3773060357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c40c67e0 0x7fa4c41473f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8000c00 tx=0x7fa4b8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c0ff9700 1 -- 172.21.15.40:0/3773060357 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8004400 con 0x7fa4c40c67e0 2024-09-13T17:59:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c0ff9700 1 -- 172.21.15.40:0/3773060357 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b801f070 con 0x7fa4c40c67e0 2024-09-13T17:59:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c9657700 1 -- 172.21.15.40:0/3773060357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c4151780 con 0x7fa4c40c67e0 2024-09-13T17:59:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c0ff9700 1 -- 172.21.15.40:0/3773060357 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b80036c0 con 0x7fa4c40c67e0 2024-09-13T17:59:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.294+0000 7fa4c9657700 1 -- 172.21.15.40:0/3773060357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c414a7f0 con 0x7fa4c40c67e0 2024-09-13T17:59:17.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.298+0000 7fa4c9657700 1 -- 172.21.15.40:0/3773060357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c4153a10 con 0x7fa4c40c67e0 2024-09-13T17:59:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.298+0000 7fa4c0ff9700 1 -- 172.21.15.40:0/3773060357 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa4b8021030 con 0x7fa4c40c67e0 2024-09-13T17:59:17.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.298+0000 7fa4c0ff9700 1 --2- 172.21.15.40:0/3773060357 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4b0066730 0x7fa4b0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.298+0000 7fa4c0ff9700 1 -- 172.21.15.40:0/3773060357 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa4b80854e0 con 0x7fa4c40c67e0 2024-09-13T17:59:17.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.298+0000 7fa4c3fff700 1 --2- 172.21.15.40:0/3773060357 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4b0066730 0x7fa4b0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.298+0000 7fa4c3fff700 1 --2- 172.21.15.40:0/3773060357 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4b0066730 0x7fa4b0068bf0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4007a60 tx=0x7fa4b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:17.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.310+0000 7fa4c0ff9700 1 -- 172.21.15.40:0/3773060357 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa4b8052490 con 0x7fa4c40c67e0 2024-09-13T17:59:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.570+0000 7fa4c9657700 1 -- 172.21.15.40:0/3773060357 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa4c414a000 con 0x7fa4c40c67e0 2024-09-13T17:59:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.570+0000 7fa4c0ff9700 1 -- 172.21.15.40:0/3773060357 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7fa4b8015620 con 0x7fa4c40c67e0 2024-09-13T17:59:17.581 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":70,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:59:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.574+0000 7fa4c9657700 1 -- 172.21.15.40:0/3773060357 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4b0066730 msgr2=0x7fa4b0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.574+0000 7fa4c9657700 1 --2- 172.21.15.40:0/3773060357 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4b0066730 0x7fa4b0068bf0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4007a60 tx=0x7fa4b4008040 comp rx=0 tx=0).stop 2024-09-13T17:59:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.574+0000 7fa4c9657700 1 -- 172.21.15.40:0/3773060357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c40c67e0 msgr2=0x7fa4c41473f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.574+0000 7fa4c9657700 1 --2- 172.21.15.40:0/3773060357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c40c67e0 0x7fa4c41473f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8000c00 tx=0x7fa4b8004770 comp rx=0 tx=0).stop 2024-09-13T17:59:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.574+0000 7fa4c9657700 1 -- 172.21.15.40:0/3773060357 shutdown_connections 2024-09-13T17:59:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.574+0000 7fa4c9657700 1 --2- 172.21.15.40:0/3773060357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4c4153630 0x7fa4c40c53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.574+0000 7fa4c9657700 1 --2- 172.21.15.40:0/3773060357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4c40c67e0 0x7fa4c41473f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.574+0000 7fa4c9657700 1 --2- 172.21.15.40:0/3773060357 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4b0066730 0x7fa4b0068bf0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.574+0000 7fa4c9657700 1 --2- 172.21.15.40:0/3773060357 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4c40c1ea0 0x7fa4c4071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.574+0000 7fa4c9657700 1 -- 172.21.15.40:0/3773060357 >> 172.21.15.40:0/3773060357 conn(0x7fa4c406e480 msgr2=0x7fa4c4150e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.574+0000 7fa4c9657700 1 -- 172.21.15.40:0/3773060357 shutdown_connections 2024-09-13T17:59:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.574+0000 7fa4c9657700 1 -- 172.21.15.40:0/3773060357 wait complete. 2024-09-13T17:59:17.598 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T17:59:17.598 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T17:59:17.599 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:59:17.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.710+0000 7f9eecc5b700 1 Processor -- start 2024-09-13T17:59:17.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.710+0000 7f9eecc5b700 1 -- start start 2024-09-13T17:59:17.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.710+0000 7f9eecc5b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ee80c7d30 0x7f9ee80ca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.710+0000 7f9eecc5b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9ee80cad40 0x7f9ee80c8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.710+0000 7f9eecc5b700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee8155b20 con 0x7f9ee814f7d0 2024-09-13T17:59:17.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9eecc5b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee80c8650 con 0x7f9ee80cad40 2024-09-13T17:59:17.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9eecc5b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee80cab20 con 0x7f9ee80c7d30 2024-09-13T17:59:17.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9ee6ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9ee80cad40 0x7f9ee80c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9ee7fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9ee814f7d0 0x7f9ee8155fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50108/0 (socket says 172.21.15.40:50108) 2024-09-13T17:59:17.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9ee7fff700 1 -- 172.21.15.40:0/3926421713 learned_addr learned my addr 172.21.15.40:0/3926421713 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:59:17.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9ee77fe700 1 --2- 172.21.15.40:0/3926421713 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ee80c7d30 0x7f9ee80ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9ee67fc700 1 -- 172.21.15.40:0/3926421713 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639617308 0 0) 0x7f9ee8155b20 con 0x7f9ee814f7d0 2024-09-13T17:59:17.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9ee67fc700 1 -- 172.21.15.40:0/3926421713 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ed00035d0 con 0x7f9ee814f7d0 2024-09-13T17:59:17.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9ee6ffd700 1 -- 172.21.15.40:0/3926421713 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ee80c7d30 msgr2=0x7f9ee80ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:17.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9ee6ffd700 1 --2- 172.21.15.40:0/3926421713 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ee80c7d30 0x7f9ee80ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9ee6ffd700 1 -- 172.21.15.40:0/3926421713 >> v1:172.21.15.40:6789/0 conn(0x7f9ee814f7d0 legacy=0x7f9ee8155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9ee6ffd700 1 -- 172.21.15.40:0/3926421713 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee8159e80 con 0x7f9ee80cad40 2024-09-13T17:59:17.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9ee6ffd700 1 --2- 172.21.15.40:0/3926421713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee80cad40 0x7f9ee80c8110 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009850 tx=0x7f9edc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d1ec373c8fb809 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:17.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9ee67fc700 1 -- 172.21.15.40:0/3926421713 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc011830 con 0x7f9ee80cad40 2024-09-13T17:59:17.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9ee67fc700 1 -- 172.21.15.40:0/3926421713 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9edc011990 con 0x7f9ee80cad40 2024-09-13T17:59:17.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9ee67fc700 1 -- 172.21.15.40:0/3926421713 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc011d40 con 0x7f9ee80cad40 2024-09-13T17:59:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9eecc5b700 1 -- 172.21.15.40:0/3926421713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee80cad40 msgr2=0x7f9ee80c8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9eecc5b700 1 --2- 172.21.15.40:0/3926421713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee80cad40 0x7f9ee80c8110 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009850 tx=0x7f9edc009b60 comp rx=0 tx=0).stop 2024-09-13T17:59:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9eecc5b700 1 -- 172.21.15.40:0/3926421713 shutdown_connections 2024-09-13T17:59:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9eecc5b700 1 --2- 172.21.15.40:0/3926421713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee80cad40 0x7f9ee80c8110 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9eecc5b700 1 --2- 172.21.15.40:0/3926421713 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ee80c7d30 0x7f9ee80ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9eecc5b700 1 -- 172.21.15.40:0/3926421713 >> 172.21.15.40:0/3926421713 conn(0x7f9ee806a390 msgr2=0x7f9ee806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9eecc5b700 1 -- 172.21.15.40:0/3926421713 shutdown_connections 2024-09-13T17:59:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9eecc5b700 1 -- 172.21.15.40:0/3926421713 wait complete. 2024-09-13T17:59:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9eecc5b700 1 Processor -- start 2024-09-13T17:59:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.714+0000 7f9eecc5b700 1 -- start start 2024-09-13T17:59:17.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9eecc5b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee80c7d30 0x7f9ee80b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9eecc5b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ee80cad40 0x7f9ee8150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9eecc5b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee814f7d0 0x7f9ee8070c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9eecc5b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee80c21a0 con 0x7f9ee80c7d30 2024-09-13T17:59:17.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9eecc5b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee8155910 con 0x7f9ee814f7d0 2024-09-13T17:59:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9eecc5b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee80b9130 con 0x7f9ee80cad40 2024-09-13T17:59:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9ee77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee80c7d30 0x7f9ee80b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9ee77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee80c7d30 0x7f9ee80b87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38202/0 (socket says 172.21.15.40:38202) 2024-09-13T17:59:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9ee77fe700 1 -- 172.21.15.40:0/2159000025 learned_addr learned my addr 172.21.15.40:0/2159000025 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9ee6ffd700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ee80cad40 0x7f9ee8150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9ee7fff700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee814f7d0 0x7f9ee8070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9ee77fe700 1 -- 172.21.15.40:0/2159000025 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ee80cad40 msgr2=0x7f9ee8150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9ee77fe700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ee80cad40 0x7f9ee8150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9ee77fe700 1 -- 172.21.15.40:0/2159000025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee814f7d0 msgr2=0x7f9ee8070c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9ee77fe700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee814f7d0 0x7f9ee8070c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9ee77fe700 1 -- 172.21.15.40:0/2159000025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9edc0094b0 con 0x7f9ee80c7d30 2024-09-13T17:59:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9ee7fff700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee814f7d0 0x7f9ee8070c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T17:59:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9ee77fe700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee80c7d30 0x7f9ee80b87e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8010e20 tx=0x7f9ed800e640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9ecffff700 1 -- 172.21.15.40:0/2159000025 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed800b840 con 0x7f9ee80c7d30 2024-09-13T17:59:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9ecffff700 1 -- 172.21.15.40:0/2159000025 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed8011450 con 0x7f9ee80c7d30 2024-09-13T17:59:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9ecffff700 1 -- 172.21.15.40:0/2159000025 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed80117c0 con 0x7f9ee80c7d30 2024-09-13T17:59:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9eecc5b700 1 -- 172.21.15.40:0/2159000025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee81519a0 con 0x7f9ee80c7d30 2024-09-13T17:59:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9eecc5b700 1 -- 172.21.15.40:0/2159000025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee8155070 con 0x7f9ee80c7d30 2024-09-13T17:59:17.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9eecc5b700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee8155410 0x7f9ee8151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9eecc5b700 1 -- 172.21.15.40:0/2159000025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee8067600 con 0x7f9ee8155410 2024-09-13T17:59:17.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9ee6ffd700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee8155410 0x7f9ee8151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.718+0000 7f9ecffff700 1 -- 172.21.15.40:0/2159000025 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9ed8011920 con 0x7f9ee80c7d30 2024-09-13T17:59:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.722+0000 7f9ee6ffd700 1 -- 172.21.15.40:0/2159000025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ee81595e0 con 0x7f9ee8155410 2024-09-13T17:59:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.722+0000 7f9ecffff700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9ed4066730 0x7f9ed4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.722+0000 7f9ecffff700 1 -- 172.21.15.40:0/2159000025 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9ed8086ef0 con 0x7f9ee80c7d30 2024-09-13T17:59:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.722+0000 7f9ee6ffd700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee8155410 0x7f9ee8151e80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8153540 tx=0x7f9edc0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.722+0000 7f9ee7fff700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9ed4066730 0x7f9ed4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.722+0000 7f9ecffff700 1 -- 172.21.15.40:0/2159000025 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc028070 con 0x7f9ee8155410 2024-09-13T17:59:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.722+0000 7f9ee7fff700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9ed4066730 0x7f9ed4068bf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8150e60 tx=0x7f9ee0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.722+0000 7f9ecffff700 1 -- 172.21.15.40:0/2159000025 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9edc004250 con 0x7f9ee8155410 2024-09-13T17:59:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.722+0000 7f9ecffff700 1 -- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee8155410 msgr2=0x7f9ee8151e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.722+0000 7f9ecffff700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee8155410 0x7f9ee8151e80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8153540 tx=0x7f9edc0047e0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.722+0000 7f9ecffff700 1 -- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee8155410 msgr2=0x7f9ee8151e80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:59:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.722+0000 7f9ecffff700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee8155410 0x7f9ee8151e80 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9eecc5b700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee81523c0 0x7f9ee814c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9eecc5b700 1 -- 172.21.15.40:0/2159000025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee8159080 con 0x7f9ee81523c0 2024-09-13T17:59:17.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9ee6ffd700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee81523c0 0x7f9ee814c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9ee6ffd700 1 -- 172.21.15.40:0/2159000025 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9ee81598c0 con 0x7f9ee81523c0 2024-09-13T17:59:17.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9ee6ffd700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee81523c0 0x7f9ee814c9d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009ba0 tx=0x7f9edc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:17.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9ecffff700 1 -- 172.21.15.40:0/2159000025 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc02cb80 con 0x7f9ee81523c0 2024-09-13T17:59:17.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9ecffff700 1 -- 172.21.15.40:0/2159000025 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4451 (secure 0 0 0) 0x7f9edc030070 con 0x7f9ee81523c0 2024-09-13T17:59:17.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9ecffff700 1 -- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee81523c0 msgr2=0x7f9ee814c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9ecffff700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee81523c0 0x7f9ee814c9d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009ba0 tx=0x7f9edc004770 comp rx=0 tx=0).stop 2024-09-13T17:59:17.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9ecffff700 1 -- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee81523c0 msgr2=0x7f9ee814c9d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:59:17.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9ecffff700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee81523c0 0x7f9ee814c9d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.742 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:59:17.742 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:59:17.742 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:59:17.742 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:59:17.742 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 70, 2024-09-13T17:59:17.742 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:59:17.743 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:59:17.743 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:59:17.743 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:59:17.743 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:17.743 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T17:59:17.743 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:59:17.743 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:59:17.743 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:59:17.743 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:17.743 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:17.743 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:17.743 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:17.743 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:17.744 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:17.744 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:17.744 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:17.744 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:17.744 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:17.744 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:17.744 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:59:17.744 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:59:17.744 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:17.744 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:17.745 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:17.745 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:17.745 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:17.745 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:17.745 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:17.746 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:17.746 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:17.746 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:17.746 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:17.746 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.746 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:59:17.746 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:59:17.746 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:59:17.746 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:59:17.746 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:59:17.746 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:59:17.746 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:59:17.746 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:59:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:59:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:59:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:59:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:59:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:59:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:59:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:59:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:59:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:59:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:59:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:59:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:59:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:59:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:59:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:59:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:59:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:59:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:59:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:59:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:59:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:59:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:59:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:59:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:59:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:59:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:17.753 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:17.753 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.753 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:17.753 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:59:17.753 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:17.753 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:59:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:59:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:59:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:59:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:59:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:59:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:59:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:59:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:59:17.756 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:59:17.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9eecc5b700 1 -- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9ed4066730 msgr2=0x7f9ed4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9eecc5b700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9ed4066730 0x7f9ed4068bf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8150e60 tx=0x7f9ee0009040 comp rx=0 tx=0).stop 2024-09-13T17:59:17.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9eecc5b700 1 -- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee80c7d30 msgr2=0x7f9ee80b87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9eecc5b700 1 --2- 172.21.15.40:0/2159000025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ee80c7d30 0x7f9ee80b87e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8010e20 tx=0x7f9ed800e640 comp rx=0 tx=0).stop 2024-09-13T17:59:17.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9ee7fff700 1 -- 172.21.15.40:0/2159000025 reap_dead start 2024-09-13T17:59:17.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9ee7fff700 1 -- 172.21.15.40:0/2159000025 reap_dead start 2024-09-13T17:59:17.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9eecc5b700 1 -- 172.21.15.40:0/2159000025 shutdown_connections 2024-09-13T17:59:17.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9eecc5b700 1 -- 172.21.15.40:0/2159000025 >> 172.21.15.40:0/2159000025 conn(0x7f9ee806a390 msgr2=0x7f9ee814e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:17.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9eecc5b700 1 -- 172.21.15.40:0/2159000025 shutdown_connections 2024-09-13T17:59:17.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.730+0000 7f9eecc5b700 1 -- 172.21.15.40:0/2159000025 wait complete. 2024-09-13T17:59:17.761 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:59:17.761 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T17:59:17.761 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:59:17.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.870+0000 7f7c0ffff700 1 Processor -- start 2024-09-13T17:59:17.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.870+0000 7f7c0ffff700 1 -- start start 2024-09-13T17:59:17.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.870+0000 7f7c0ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7c100cf1a0 0x7f7c1015a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.870+0000 7f7c0ffff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c1015a7a0 con 0x7f7c100cc250 2024-09-13T17:59:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.870+0000 7f7c0ffff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c100cbdc0 con 0x7f7c100cf580 2024-09-13T17:59:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.870+0000 7f7c0ffff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c10153f80 con 0x7f7c100cf1a0 2024-09-13T17:59:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.870+0000 7f7c0effd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7c100cf1a0 0x7f7c1015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.870+0000 7f7c0f7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7c100cc250 0x7f7c100c8aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50110/0 (socket says 172.21.15.40:50110) 2024-09-13T17:59:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.870+0000 7f7c0f7fe700 1 -- 172.21.15.40:0/1963743887 learned_addr learned my addr 172.21.15.40:0/1963743887 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:59:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.870+0000 7f7c0e7fc700 1 -- 172.21.15.40:0/1963743887 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1945847898 0 0) 0x7f7c08001490 con 0x7f7c100cc250 2024-09-13T17:59:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.870+0000 7f7c0e7fc700 1 -- 172.21.15.40:0/1963743887 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bfc0035d0 con 0x7f7c100cc250 2024-09-13T17:59:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.870+0000 7f7c0effd700 1 -- 172.21.15.40:0/1963743887 >> v1:172.21.15.43:6789/0 conn(0x7f7c100cf580 legacy=0x7f7c100c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.870+0000 7f7c0effd700 1 -- 172.21.15.40:0/1963743887 >> v1:172.21.15.40:6789/0 conn(0x7f7c100cc250 legacy=0x7f7c100c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.870+0000 7f7c0effd700 1 -- 172.21.15.40:0/1963743887 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c100caed0 con 0x7f7c100cf1a0 2024-09-13T17:59:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.870+0000 7f7c0effd700 1 --2- 172.21.15.40:0/1963743887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c100cf1a0 0x7f7c1015a260 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009850 tx=0x7f7c00009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7111bfb2c839a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.870+0000 7f7c0e7fc700 1 -- 172.21.15.40:0/1963743887 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c00011830 con 0x7f7c100cf1a0 2024-09-13T17:59:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.870+0000 7f7c0e7fc700 1 -- 172.21.15.40:0/1963743887 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c00011990 con 0x7f7c100cf1a0 2024-09-13T17:59:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.870+0000 7f7c0e7fc700 1 -- 172.21.15.40:0/1963743887 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c00011d00 con 0x7f7c100cf1a0 2024-09-13T17:59:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c0ffff700 1 -- 172.21.15.40:0/1963743887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c100cf1a0 msgr2=0x7f7c1015a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c0ffff700 1 --2- 172.21.15.40:0/1963743887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c100cf1a0 0x7f7c1015a260 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009850 tx=0x7f7c00009b60 comp rx=0 tx=0).stop 2024-09-13T17:59:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c0ffff700 1 -- 172.21.15.40:0/1963743887 shutdown_connections 2024-09-13T17:59:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c0ffff700 1 --2- 172.21.15.40:0/1963743887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c100cf1a0 0x7f7c1015a260 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c0ffff700 1 -- 172.21.15.40:0/1963743887 >> 172.21.15.40:0/1963743887 conn(0x7f7c10066090 msgr2=0x7f7c10066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c0ffff700 1 -- 172.21.15.40:0/1963743887 shutdown_connections 2024-09-13T17:59:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c0ffff700 1 -- 172.21.15.40:0/1963743887 wait complete. 2024-09-13T17:59:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c0ffff700 1 Processor -- start 2024-09-13T17:59:17.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c0ffff700 1 -- start start 2024-09-13T17:59:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c0ffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7c100cc250 0x7f7c10156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c0ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c100cf1a0 0x7f7c10150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c0ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c100cf580 0x7f7c101eec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c0ffff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c10155f90 con 0x7f7c100cf1a0 2024-09-13T17:59:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c0ffff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c101f2d50 con 0x7f7c100cf580 2024-09-13T17:59:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c0ffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c10156eb0 con 0x7f7c100cc250 2024-09-13T17:59:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c07fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c100cf1a0 0x7f7c10150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c07fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c100cf1a0 0x7f7c10150a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38248/0 (socket says 172.21.15.40:38248) 2024-09-13T17:59:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c07fff700 1 -- 172.21.15.40:0/382995023 learned_addr learned my addr 172.21.15.40:0/382995023 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c0f7fe700 1 --2- 172.21.15.40:0/382995023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c100cf580 0x7f7c101eec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c0effd700 1 --2- 172.21.15.40:0/382995023 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7c100cc250 0x7f7c10156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c07fff700 1 -- 172.21.15.40:0/382995023 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7c100cc250 msgr2=0x7f7c10156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c07fff700 1 --2- 172.21.15.40:0/382995023 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7c100cc250 0x7f7c10156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c07fff700 1 -- 172.21.15.40:0/382995023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c100cf580 msgr2=0x7f7c101eec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c07fff700 1 --2- 172.21.15.40:0/382995023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c100cf580 0x7f7c101eec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.874+0000 7f7c07fff700 1 -- 172.21.15.40:0/382995023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c000094b0 con 0x7f7c100cf1a0 2024-09-13T17:59:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.878+0000 7f7c07fff700 1 --2- 172.21.15.40:0/382995023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c100cf1a0 0x7f7c10150a80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800c430 tx=0x7f7bf800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.878+0000 7f7c077fe700 1 -- 172.21.15.40:0/382995023 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf80052d0 con 0x7f7c100cf1a0 2024-09-13T17:59:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.878+0000 7f7c077fe700 1 -- 172.21.15.40:0/382995023 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf8005430 con 0x7f7c100cf1a0 2024-09-13T17:59:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.878+0000 7f7c0ffff700 1 -- 172.21.15.40:0/382995023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c10157140 con 0x7f7c100cf1a0 2024-09-13T17:59:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.878+0000 7f7c077fe700 1 -- 172.21.15.40:0/382995023 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf80055a0 con 0x7f7c100cf1a0 2024-09-13T17:59:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.878+0000 7f7c0ffff700 1 -- 172.21.15.40:0/382995023 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c101f2420 con 0x7f7c100cf1a0 2024-09-13T17:59:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.878+0000 7f7c077fe700 1 -- 172.21.15.40:0/382995023 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7bf80036a0 con 0x7f7c100cf1a0 2024-09-13T17:59:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.878+0000 7f7c077fe700 1 --2- 172.21.15.40:0/382995023 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7bf0066730 0x7f7bf0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.878+0000 7f7c077fe700 1 -- 172.21.15.40:0/382995023 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7bf80837e0 con 0x7f7c100cf1a0 2024-09-13T17:59:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.878+0000 7f7c0effd700 1 --2- 172.21.15.40:0/382995023 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7bf0066730 0x7f7bf0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.878+0000 7f7c0ffff700 1 --2- 172.21.15.40:0/382995023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c101ef170 0x7f7c101f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.878+0000 7f7c0ffff700 1 -- 172.21.15.40:0/382995023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c10074750 con 0x7f7c101ef170 2024-09-13T17:59:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.878+0000 7f7c0effd700 1 --2- 172.21.15.40:0/382995023 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7bf0066730 0x7f7bf0068bf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009ba0 tx=0x7f7c00008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.882+0000 7f7c0f7fe700 1 --2- 172.21.15.40:0/382995023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c101ef170 0x7f7c101f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.882+0000 7f7c0f7fe700 1 -- 172.21.15.40:0/382995023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c100cb490 con 0x7f7c101ef170 2024-09-13T17:59:17.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.882+0000 7f7c0f7fe700 1 --2- 172.21.15.40:0/382995023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c101ef170 0x7f7c101f25b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7c080016f0 tx=0x7f7c0800ec40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:17.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.882+0000 7f7c077fe700 1 -- 172.21.15.40:0/382995023 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0800ee80 con 0x7f7c101ef170 2024-09-13T17:59:17.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.886+0000 7f7c077fe700 1 -- 172.21.15.40:0/382995023 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7c0800c730 con 0x7f7c101ef170 2024-09-13T17:59:17.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.886+0000 7f7c077fe700 1 -- 172.21.15.40:0/382995023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c101ef170 msgr2=0x7f7c101f25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.886+0000 7f7c077fe700 1 --2- 172.21.15.40:0/382995023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c101ef170 0x7f7c101f25b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7c080016f0 tx=0x7f7c0800ec40 comp rx=0 tx=0).stop 2024-09-13T17:59:17.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.886+0000 7f7c077fe700 1 -- 172.21.15.40:0/382995023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c101ef170 msgr2=0x7f7c101f25b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:59:17.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.886+0000 7f7c077fe700 1 --2- 172.21.15.40:0/382995023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c101ef170 0x7f7c101f25b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.894+0000 7f7c0ffff700 1 --2- 172.21.15.40:0/382995023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c1014f1a0 0x7f7c1014f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:17.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.894+0000 7f7c0ffff700 1 -- 172.21.15.40:0/382995023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c100cba50 con 0x7f7c1014f1a0 2024-09-13T17:59:17.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.894+0000 7f7c0f7fe700 1 --2- 172.21.15.40:0/382995023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c1014f1a0 0x7f7c1014f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:17.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.894+0000 7f7c0f7fe700 1 -- 172.21.15.40:0/382995023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7c10067a40 con 0x7f7c1014f1a0 2024-09-13T17:59:17.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.898+0000 7f7c0f7fe700 1 --2- 172.21.15.40:0/382995023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c1014f1a0 0x7f7c1014f5b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7c080016f0 tx=0x7f7c08001660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.898+0000 7f7c077fe700 1 -- 172.21.15.40:0/382995023 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c08028640 con 0x7f7c1014f1a0 2024-09-13T17:59:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.898+0000 7f7c077fe700 1 -- 172.21.15.40:0/382995023 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4947 (secure 0 0 0) 0x7f7c08013420 con 0x7f7c1014f1a0 2024-09-13T17:59:17.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.898+0000 7f7c077fe700 1 -- 172.21.15.40:0/382995023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c1014f1a0 msgr2=0x7f7c1014f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.898+0000 7f7c077fe700 1 --2- 172.21.15.40:0/382995023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c1014f1a0 0x7f7c1014f5b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7c080016f0 tx=0x7f7c08001660 comp rx=0 tx=0).stop 2024-09-13T17:59:17.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.898+0000 7f7c077fe700 1 -- 172.21.15.40:0/382995023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c1014f1a0 msgr2=0x7f7c1014f5b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:59:17.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.898+0000 7f7c077fe700 1 --2- 172.21.15.40:0/382995023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c1014f1a0 0x7f7c1014f5b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.908 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:59:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:59:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:59:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:59:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 70, 2024-09-13T17:59:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:59:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:59:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:59:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:59:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T17:59:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:59:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:59:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:59:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:59:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:59:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:59:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:59:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:59:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:59:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:59:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:59:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:59:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:59:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:59:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:59:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:59:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:59:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:59:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:59:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:59:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:59:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:59:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:59:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:59:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:59:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:59:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:59:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:59:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:59:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:59:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:59:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:59:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:59:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:59:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:59:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:59:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:59:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:59:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:59:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:59:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:59:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:59:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:59:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:59:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:59:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:59:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:59:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:59:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:59:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:59:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:59:17.920 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:59:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:59:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:17.921 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:59:17.921 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:59:17.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.898+0000 7f7c0ffff700 1 -- 172.21.15.40:0/382995023 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7bf0066730 msgr2=0x7f7bf0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.898+0000 7f7c0ffff700 1 --2- 172.21.15.40:0/382995023 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7bf0066730 0x7f7bf0068bf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009ba0 tx=0x7f7c00008030 comp rx=0 tx=0).stop 2024-09-13T17:59:17.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.898+0000 7f7c0ffff700 1 -- 172.21.15.40:0/382995023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c100cf1a0 msgr2=0x7f7c10150a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:17.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.898+0000 7f7c0ffff700 1 --2- 172.21.15.40:0/382995023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c100cf1a0 0x7f7c10150a80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800c430 tx=0x7f7bf800c7f0 comp rx=0 tx=0).stop 2024-09-13T17:59:17.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.898+0000 7f7c0f7fe700 1 -- 172.21.15.40:0/382995023 reap_dead start 2024-09-13T17:59:17.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.898+0000 7f7c0ffff700 1 -- 172.21.15.40:0/382995023 shutdown_connections 2024-09-13T17:59:17.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.898+0000 7f7c0ffff700 1 -- 172.21.15.40:0/382995023 >> 172.21.15.40:0/382995023 conn(0x7f7c10066090 msgr2=0x7f7c100ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:17.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.898+0000 7f7c0ffff700 1 -- 172.21.15.40:0/382995023 shutdown_connections 2024-09-13T17:59:17.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:17.898+0000 7f7c0ffff700 1 -- 172.21.15.40:0/382995023 wait complete. 2024-09-13T17:59:17.924 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:59:17.924 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T17:59:17.924 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:59:18.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.034+0000 7f47e29cb700 1 Processor -- start 2024-09-13T17:59:18.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.034+0000 7f47e29cb700 1 -- start start 2024-09-13T17:59:18.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.034+0000 7f47e29cb700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc14fe90 0x7f47dc0c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.034+0000 7f47e29cb700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47dc0c9f00 con 0x7f47dc066420 2024-09-13T17:59:18.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.034+0000 7f47e29cb700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47dc159670 con 0x7f47dc15d2d0 2024-09-13T17:59:18.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.034+0000 7f47e29cb700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc15a2f0 con 0x7f47dc14fe90 2024-09-13T17:59:18.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.034+0000 7f47e11c8700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f47dc066420 0x7f47dc15cbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50114/0 (socket says 172.21.15.40:50114) 2024-09-13T17:59:18.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.034+0000 7f47e11c8700 1 -- 172.21.15.40:0/1059845005 learned_addr learned my addr 172.21.15.40:0/1059845005 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:59:18.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.034+0000 7f47e19c9700 1 --2- 172.21.15.40:0/1059845005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc14fe90 0x7f47dc0c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.034+0000 7f47e09c7700 1 -- 172.21.15.40:0/1059845005 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346629849 0 0) 0x7f47dc0c9f00 con 0x7f47dc066420 2024-09-13T17:59:18.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.034+0000 7f47e09c7700 1 -- 172.21.15.40:0/1059845005 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47c80035d0 con 0x7f47dc066420 2024-09-13T17:59:18.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.034+0000 7f47e09c7700 1 -- 172.21.15.40:0/1059845005 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332532809 0 0) 0x7f47dc159670 con 0x7f47dc15d2d0 2024-09-13T17:59:18.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.034+0000 7f47e09c7700 1 -- 172.21.15.40:0/1059845005 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47dc0c9f00 con 0x7f47dc15d2d0 2024-09-13T17:59:18.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.034+0000 7f47e19c9700 1 -- 172.21.15.40:0/1059845005 >> v1:172.21.15.43:6789/0 conn(0x7f47dc15d2d0 legacy=0x7f47dc163e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.034+0000 7f47e19c9700 1 -- 172.21.15.40:0/1059845005 >> v1:172.21.15.40:6789/0 conn(0x7f47dc066420 legacy=0x7f47dc15cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.034+0000 7f47e19c9700 1 -- 172.21.15.40:0/1059845005 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47dc159950 con 0x7f47dc14fe90 2024-09-13T17:59:18.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.034+0000 7f47e19c9700 1 --2- 172.21.15.40:0/1059845005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc14fe90 0x7f47dc0c7d30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009880 tx=0x7f47cc009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93f7c5f30fecf490 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:18.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.034+0000 7f47e09c7700 1 -- 172.21.15.40:0/1059845005 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d8001670 con 0x7f47dc14fe90 2024-09-13T17:59:18.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.034+0000 7f47e09c7700 1 -- 172.21.15.40:0/1059845005 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47cc010590 con 0x7f47dc14fe90 2024-09-13T17:59:18.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.038+0000 7f47e09c7700 1 -- 172.21.15.40:0/1059845005 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc010900 con 0x7f47dc14fe90 2024-09-13T17:59:18.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.038+0000 7f47e29cb700 1 -- 172.21.15.40:0/1059845005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc14fe90 msgr2=0x7f47dc0c7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.038+0000 7f47e29cb700 1 --2- 172.21.15.40:0/1059845005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc14fe90 0x7f47dc0c7d30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009880 tx=0x7f47cc009b90 comp rx=0 tx=0).stop 2024-09-13T17:59:18.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.038+0000 7f47e29cb700 1 -- 172.21.15.40:0/1059845005 shutdown_connections 2024-09-13T17:59:18.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.038+0000 7f47e29cb700 1 --2- 172.21.15.40:0/1059845005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc14fe90 0x7f47dc0c7d30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.038+0000 7f47e29cb700 1 -- 172.21.15.40:0/1059845005 >> 172.21.15.40:0/1059845005 conn(0x7f47dc06a390 msgr2=0x7f47dc06a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:18.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.038+0000 7f47e29cb700 1 -- 172.21.15.40:0/1059845005 shutdown_connections 2024-09-13T17:59:18.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.038+0000 7f47e29cb700 1 -- 172.21.15.40:0/1059845005 wait complete. 2024-09-13T17:59:18.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.038+0000 7f47e29cb700 1 Processor -- start 2024-09-13T17:59:18.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.038+0000 7f47e29cb700 1 -- start start 2024-09-13T17:59:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.038+0000 7f47e29cb700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc066420 0x7f47dc0c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.038+0000 7f47e29cb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47dc14fe90 0x7f47dc157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.038+0000 7f47e29cb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47dc15d2d0 0x7f47dc1ef320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.038+0000 7f47e29cb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc156700 con 0x7f47dc15d2d0 2024-09-13T17:59:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.038+0000 7f47e29cb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc1f2620 con 0x7f47dc14fe90 2024-09-13T17:59:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.038+0000 7f47e29cb700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc0c8070 con 0x7f47dc066420 2024-09-13T17:59:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.038+0000 7f47e21ca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47dc15d2d0 0x7f47dc1ef320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.038+0000 7f47e19c9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc066420 0x7f47dc0c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.038+0000 7f47e11c8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47dc14fe90 0x7f47dc157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47e11c8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47dc14fe90 0x7f47dc157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:57252/0 (socket says 172.21.15.40:57252) 2024-09-13T17:59:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47e11c8700 1 -- 172.21.15.40:0/361127425 learned_addr learned my addr 172.21.15.40:0/361127425 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:18.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47e21ca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47dc15d2d0 0x7f47dc1ef320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38256/0 (socket says 172.21.15.40:38256) 2024-09-13T17:59:18.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47e19c9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc066420 0x7f47dc0c7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:60644/0 (socket says 172.21.15.40:60644) 2024-09-13T17:59:18.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47e21ca700 1 -- 172.21.15.40:0/361127425 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc066420 msgr2=0x7f47dc0c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47e21ca700 1 --2- 172.21.15.40:0/361127425 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc066420 0x7f47dc0c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47e21ca700 1 -- 172.21.15.40:0/361127425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47dc14fe90 msgr2=0x7f47dc157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47e21ca700 1 --2- 172.21.15.40:0/361127425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47dc14fe90 0x7f47dc157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47e21ca700 1 -- 172.21.15.40:0/361127425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc0094b0 con 0x7f47dc15d2d0 2024-09-13T17:59:18.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47e11c8700 1 --2- 172.21.15.40:0/361127425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47dc14fe90 0x7f47dc157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:59:18.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47e21ca700 1 --2- 172.21.15.40:0/361127425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47dc15d2d0 0x7f47dc1ef320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f47d400cbf0 tx=0x7f47d400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:18.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47d27fc700 1 -- 172.21.15.40:0/361127425 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d4003c20 con 0x7f47dc15d2d0 2024-09-13T17:59:18.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47d27fc700 1 -- 172.21.15.40:0/361127425 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d4003d80 con 0x7f47dc15d2d0 2024-09-13T17:59:18.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47e29cb700 1 -- 172.21.15.40:0/361127425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47dc1f2890 con 0x7f47dc15d2d0 2024-09-13T17:59:18.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47d27fc700 1 -- 172.21.15.40:0/361127425 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d400aa30 con 0x7f47dc15d2d0 2024-09-13T17:59:18.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47e29cb700 1 -- 172.21.15.40:0/361127425 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47dc1f2e50 con 0x7f47dc15d2d0 2024-09-13T17:59:18.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47e29cb700 1 --2- 172.21.15.40:0/361127425 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc152f90 0x7f47dc1ef860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47e29cb700 1 -- 172.21.15.40:0/361127425 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc067640 con 0x7f47dc152f90 2024-09-13T17:59:18.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47e19c9700 1 --2- 172.21.15.40:0/361127425 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc152f90 0x7f47dc1ef860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47e19c9700 1 -- 172.21.15.40:0/361127425 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47dc0c27e0 con 0x7f47dc152f90 2024-09-13T17:59:18.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47d27fc700 1 -- 172.21.15.40:0/361127425 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f47d400ab90 con 0x7f47dc15d2d0 2024-09-13T17:59:18.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.042+0000 7f47e19c9700 1 --2- 172.21.15.40:0/361127425 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc152f90 0x7f47dc1ef860 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009880 tx=0x7f47cc0047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:18.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.046+0000 7f47d27fc700 1 --2- 172.21.15.40:0/361127425 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f47c0066760 0x7f47c0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.046+0000 7f47e11c8700 1 --2- 172.21.15.40:0/361127425 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f47c0066760 0x7f47c0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.046+0000 7f47d27fc700 1 -- 172.21.15.40:0/361127425 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f47d4084590 con 0x7f47dc15d2d0 2024-09-13T17:59:18.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.046+0000 7f47d27fc700 1 -- 172.21.15.40:0/361127425 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc004220 con 0x7f47dc152f90 2024-09-13T17:59:18.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.046+0000 7f47d27fc700 1 -- 172.21.15.40:0/361127425 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f47cc004380 con 0x7f47dc152f90 2024-09-13T17:59:18.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.046+0000 7f47d27fc700 1 -- 172.21.15.40:0/361127425 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc152f90 msgr2=0x7f47dc1ef860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.046+0000 7f47d27fc700 1 --2- 172.21.15.40:0/361127425 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc152f90 0x7f47dc1ef860 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009880 tx=0x7f47cc0047e0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.046+0000 7f47d27fc700 1 -- 172.21.15.40:0/361127425 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc152f90 msgr2=0x7f47dc1ef860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:59:18.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.046+0000 7f47d27fc700 1 --2- 172.21.15.40:0/361127425 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc152f90 0x7f47dc1ef860 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.046+0000 7f47e11c8700 1 --2- 172.21.15.40:0/361127425 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f47c0066760 0x7f47c0068c20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0ca1c0 tx=0x7f47d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:18.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.054+0000 7f47e29cb700 1 --2- 172.21.15.40:0/361127425 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc1f2fe0 0x7f47dc14e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.058+0000 7f47e29cb700 1 -- 172.21.15.40:0/361127425 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc159f10 con 0x7f47dc1f2fe0 2024-09-13T17:59:18.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.058+0000 7f47e19c9700 1 --2- 172.21.15.40:0/361127425 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc1f2fe0 0x7f47dc14e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.058+0000 7f47e19c9700 1 -- 172.21.15.40:0/361127425 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f47dc0c9b00 con 0x7f47dc1f2fe0 2024-09-13T17:59:18.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.058+0000 7f47e19c9700 1 --2- 172.21.15.40:0/361127425 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc1f2fe0 0x7f47dc14e250 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009bd0 tx=0x7f47cc004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:18.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.058+0000 7f47d27fc700 1 -- 172.21.15.40:0/361127425 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc02c720 con 0x7f47dc1f2fe0 2024-09-13T17:59:18.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.058+0000 7f47d27fc700 1 -- 172.21.15.40:0/361127425 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4947 (secure 0 0 0) 0x7f47cc02c880 con 0x7f47dc1f2fe0 2024-09-13T17:59:18.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.058+0000 7f47d27fc700 1 -- 172.21.15.40:0/361127425 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc1f2fe0 msgr2=0x7f47dc14e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.058+0000 7f47d27fc700 1 --2- 172.21.15.40:0/361127425 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc1f2fe0 0x7f47dc14e250 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009bd0 tx=0x7f47cc004770 comp rx=0 tx=0).stop 2024-09-13T17:59:18.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.058+0000 7f47d27fc700 1 -- 172.21.15.40:0/361127425 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc1f2fe0 msgr2=0x7f47dc14e250 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:59:18.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.058+0000 7f47d27fc700 1 --2- 172.21.15.40:0/361127425 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47dc1f2fe0 0x7f47dc14e250 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.069 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:59:18.069 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:59:18.069 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:59:18.069 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:59:18.069 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 70, 2024-09-13T17:59:18.070 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:59:18.070 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:59:18.070 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:59:18.070 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:59:18.070 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:18.070 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T17:59:18.070 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:59:18.070 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:59:18.070 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:59:18.070 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:18.070 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:18.070 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:18.070 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:18.070 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:18.071 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:18.071 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:18.071 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:18.071 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:18.071 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:18.071 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:18.071 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:59:18.071 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:59:18.071 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:18.071 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:18.071 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:18.071 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:18.071 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:59:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:59:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:59:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:59:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:59:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:59:18.072 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:59:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:59:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:59:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:59:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:59:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:59:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:59:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:59:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:59:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:59:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:59:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:18.073 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:59:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:59:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:59:18.074 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:59:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:59:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:59:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:18.075 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:59:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:59:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:59:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:59:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:18.076 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:59:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:59:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:59:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:59:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:18.077 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:59:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:59:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:59:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:18.078 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:59:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:59:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:59:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:59:18.079 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:59:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:59:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:59:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:59:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:18.080 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:59:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T17:59:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:59:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:59:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:18.081 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:59:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:59:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:18.082 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:59:18.082 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:59:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.058+0000 7f47e29cb700 1 -- 172.21.15.40:0/361127425 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f47c0066760 msgr2=0x7f47c0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.058+0000 7f47e29cb700 1 --2- 172.21.15.40:0/361127425 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f47c0066760 0x7f47c0068c20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0ca1c0 tx=0x7f47d8009040 comp rx=0 tx=0).stop 2024-09-13T17:59:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.058+0000 7f47e29cb700 1 -- 172.21.15.40:0/361127425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47dc15d2d0 msgr2=0x7f47dc1ef320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.058+0000 7f47e21ca700 1 -- 172.21.15.40:0/361127425 reap_dead start 2024-09-13T17:59:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.058+0000 7f47e29cb700 1 --2- 172.21.15.40:0/361127425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47dc15d2d0 0x7f47dc1ef320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f47d400cbf0 tx=0x7f47d400cf00 comp rx=0 tx=0).stop 2024-09-13T17:59:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.058+0000 7f47e29cb700 1 -- 172.21.15.40:0/361127425 shutdown_connections 2024-09-13T17:59:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.058+0000 7f47e29cb700 1 --2- 172.21.15.40:0/361127425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47dc15d2d0 0x7f47dc1ef320 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.058+0000 7f47e29cb700 1 -- 172.21.15.40:0/361127425 >> 172.21.15.40:0/361127425 conn(0x7f47dc06a390 msgr2=0x7f47dc15cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.058+0000 7f47e29cb700 1 -- 172.21.15.40:0/361127425 shutdown_connections 2024-09-13T17:59:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.058+0000 7f47e29cb700 1 -- 172.21.15.40:0/361127425 wait complete. 2024-09-13T17:59:18.084 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:59:18.085 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T17:59:18.085 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T17:59:18.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.190+0000 7fcfae240700 1 Processor -- start 2024-09-13T17:59:18.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.190+0000 7fcfae240700 1 -- start start 2024-09-13T17:59:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfae240700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcfa80b2b70 0x7fcfa806f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfae240700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa8070390 con 0x7fcfa81512c0 2024-09-13T17:59:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfae240700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa80c80c0 con 0x7fcfa806fee0 2024-09-13T17:59:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfae240700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa806e780 con 0x7fcfa80b2b70 2024-09-13T17:59:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfad23e700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcfa81512c0 0x7fcfa8157b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50130/0 (socket says 172.21.15.40:50130) 2024-09-13T17:59:18.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfad23e700 1 -- 172.21.15.40:0/4286938136 learned_addr learned my addr 172.21.15.40:0/4286938136 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:59:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcf9ffff700 1 -- 172.21.15.40:0/4286938136 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1127383653 0 0) 0x7fcfa8070390 con 0x7fcfa81512c0 2024-09-13T17:59:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcf9ffff700 1 -- 172.21.15.40:0/4286938136 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf940035d0 con 0x7fcfa81512c0 2024-09-13T17:59:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfaca3d700 1 --2- 172.21.15.40:0/4286938136 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcfa80b2b70 0x7fcfa806f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcf9ffff700 1 -- 172.21.15.40:0/4286938136 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3833440697 0 0) 0x7fcf940035d0 con 0x7fcfa81512c0 2024-09-13T17:59:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcf9ffff700 1 -- 172.21.15.40:0/4286938136 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcfa80b2b70 msgr2=0x7fcfa806f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcf9ffff700 1 --2- 172.21.15.40:0/4286938136 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcfa80b2b70 0x7fcfa806f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcf9ffff700 1 -- 172.21.15.40:0/4286938136 >> v1:172.21.15.43:6789/0 conn(0x7fcfa806fee0 legacy=0x7fcfa80c1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcf9ffff700 1 -- 172.21.15.40:0/4286938136 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa806e1c0 con 0x7fcfa81512c0 2024-09-13T17:59:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcf9ffff700 1 -- 172.21.15.40:0/4286938136 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fcfa4001670 con 0x7fcfa81512c0 2024-09-13T17:59:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcf9ffff700 1 -- 172.21.15.40:0/4286938136 >> v1:172.21.15.40:6789/0 conn(0x7fcfa81512c0 legacy=0x7fcfa8157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcf9ffff700 1 --2- 172.21.15.40:0/4286938136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf94003a50 0x7fcf94003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcf9ffff700 1 --2- 172.21.15.40:0/4286938136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf94004400 0x7fcf940048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcf9ffff700 1 --2- 172.21.15.40:0/4286938136 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf940050d0 0x7fcf9400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcf9ffff700 1 -- 172.21.15.40:0/4286938136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa806e780 con 0x7fcf94003a50 2024-09-13T17:59:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcf9ffff700 1 -- 172.21.15.40:0/4286938136 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9400bf70 con 0x7fcf94004400 2024-09-13T17:59:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcf9ffff700 1 -- 172.21.15.40:0/4286938136 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9400c0b0 con 0x7fcf940050d0 2024-09-13T17:59:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfad23e700 1 --2- 172.21.15.40:0/4286938136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf94003a50 0x7fcf94003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfada3f700 1 --2- 172.21.15.40:0/4286938136 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf940050d0 0x7fcf9400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfaca3d700 1 --2- 172.21.15.40:0/4286938136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf94004400 0x7fcf940048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfad23e700 1 -- 172.21.15.40:0/4286938136 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf940050d0 msgr2=0x7fcf9400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfad23e700 1 --2- 172.21.15.40:0/4286938136 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf940050d0 0x7fcf9400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfad23e700 1 -- 172.21.15.40:0/4286938136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf94004400 msgr2=0x7fcf940048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfad23e700 1 --2- 172.21.15.40:0/4286938136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf94004400 0x7fcf940048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfad23e700 1 -- 172.21.15.40:0/4286938136 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf940035d0 con 0x7fcf94003a50 2024-09-13T17:59:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfad23e700 1 --2- 172.21.15.40:0/4286938136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf94003a50 0x7fcf94003e30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcfa400c2f0 tx=0x7fcf940058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21386f957b6a6717 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcf9ffff700 1 -- 172.21.15.40:0/4286938136 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa400ce30 con 0x7fcf94003a50 2024-09-13T17:59:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcf9ffff700 1 -- 172.21.15.40:0/4286938136 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa4005980 con 0x7fcf94003a50 2024-09-13T17:59:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcf9ffff700 1 -- 172.21.15.40:0/4286938136 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4005d30 con 0x7fcf94003a50 2024-09-13T17:59:18.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfae240700 1 -- 172.21.15.40:0/4286938136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf94003a50 msgr2=0x7fcf94003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfae240700 1 --2- 172.21.15.40:0/4286938136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf94003a50 0x7fcf94003e30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcfa400c2f0 tx=0x7fcf940058a0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfada3f700 1 -- 172.21.15.40:0/4286938136 reap_dead start 2024-09-13T17:59:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfae240700 1 -- 172.21.15.40:0/4286938136 shutdown_connections 2024-09-13T17:59:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.194+0000 7fcfae240700 1 -- 172.21.15.40:0/4286938136 >> 172.21.15.40:0/4286938136 conn(0x7fcfa8066090 msgr2=0x7fcfa8066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfae240700 1 -- 172.21.15.40:0/4286938136 shutdown_connections 2024-09-13T17:59:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfae240700 1 -- 172.21.15.40:0/4286938136 wait complete. 2024-09-13T17:59:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfae240700 1 Processor -- start 2024-09-13T17:59:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfae240700 1 -- start start 2024-09-13T17:59:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfae240700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf94003a50 0x7fcfa8155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfae240700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcfa806d2d0 0x7fcfa814be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfae240700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcfa8156680 0x7fcfa8156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfae240700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa8155250 con 0x7fcfa806d2d0 2024-09-13T17:59:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfae240700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa806ba30 con 0x7fcf94003a50 2024-09-13T17:59:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfae240700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa8154d20 con 0x7fcfa8156680 2024-09-13T17:59:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfaca3d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcfa806d2d0 0x7fcfa814be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfaca3d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcfa806d2d0 0x7fcfa814be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38266/0 (socket says 172.21.15.40:38266) 2024-09-13T17:59:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfaca3d700 1 -- 172.21.15.40:0/2466117263 learned_addr learned my addr 172.21.15.40:0/2466117263 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfad23e700 1 --2- 172.21.15.40:0/2466117263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf94003a50 0x7fcfa8155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfada3f700 1 --2- 172.21.15.40:0/2466117263 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcfa8156680 0x7fcfa8156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfaca3d700 1 -- 172.21.15.40:0/2466117263 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcfa8156680 msgr2=0x7fcfa8156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfaca3d700 1 --2- 172.21.15.40:0/2466117263 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcfa8156680 0x7fcfa8156b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfaca3d700 1 -- 172.21.15.40:0/2466117263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf94003a50 msgr2=0x7fcfa8155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfaca3d700 1 --2- 172.21.15.40:0/2466117263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf94003a50 0x7fcfa8155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfaca3d700 1 -- 172.21.15.40:0/2466117263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa8153fb0 con 0x7fcfa806d2d0 2024-09-13T17:59:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfaca3d700 1 --2- 172.21.15.40:0/2466117263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcfa806d2d0 0x7fcfa814be90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcf980118a0 tx=0x7fcf98011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcf9dffb700 1 -- 172.21.15.40:0/2466117263 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf98004300 con 0x7fcfa806d2d0 2024-09-13T17:59:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcf9dffb700 1 -- 172.21.15.40:0/2466117263 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf98004460 con 0x7fcfa806d2d0 2024-09-13T17:59:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfae240700 1 -- 172.21.15.40:0/2466117263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa806b750 con 0x7fcfa806d2d0 2024-09-13T17:59:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcf9dffb700 1 -- 172.21.15.40:0/2466117263 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf980045d0 con 0x7fcfa806d2d0 2024-09-13T17:59:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.198+0000 7fcfae240700 1 -- 172.21.15.40:0/2466117263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa8154f70 con 0x7fcfa806d2d0 2024-09-13T17:59:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.202+0000 7fcf9dffb700 1 -- 172.21.15.40:0/2466117263 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcf98003900 con 0x7fcfa806d2d0 2024-09-13T17:59:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.202+0000 7fcfae240700 1 -- 172.21.15.40:0/2466117263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa8154a80 con 0x7fcfa806d2d0 2024-09-13T17:59:18.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.202+0000 7fcf9dffb700 1 --2- 172.21.15.40:0/2466117263 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcf8c066730 0x7fcf8c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:18.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.202+0000 7fcf9dffb700 1 -- 172.21.15.40:0/2466117263 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcf98084730 con 0x7fcfa806d2d0 2024-09-13T17:59:18.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.202+0000 7fcfad23e700 1 --2- 172.21.15.40:0/2466117263 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcf8c066730 0x7fcf8c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:18.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.202+0000 7fcfad23e700 1 --2- 172.21.15.40:0/2466117263 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcf8c066730 0x7fcf8c068bf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4001930 tx=0x7fcfa4007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:18.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.206+0000 7fcf9dffb700 1 -- 172.21.15.40:0/2466117263 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcf980516e0 con 0x7fcfa806d2d0 2024-09-13T17:59:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.478+0000 7fcfae240700 1 -- 172.21.15.40:0/2466117263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fcfa8154140 con 0x7fcfa806d2d0 2024-09-13T17:59:18.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.478+0000 7fcf9dffb700 1 -- 172.21.15.40:0/2466117263 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fcf98019080 con 0x7fcfa806d2d0 2024-09-13T17:59:18.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.482+0000 7fcfae240700 1 -- 172.21.15.40:0/2466117263 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcf8c066730 msgr2=0x7fcf8c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.482+0000 7fcfae240700 1 --2- 172.21.15.40:0/2466117263 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcf8c066730 0x7fcf8c068bf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4001930 tx=0x7fcfa4007000 comp rx=0 tx=0).stop 2024-09-13T17:59:18.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.482+0000 7fcfae240700 1 -- 172.21.15.40:0/2466117263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcfa806d2d0 msgr2=0x7fcfa814be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:18.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.482+0000 7fcfae240700 1 --2- 172.21.15.40:0/2466117263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcfa806d2d0 0x7fcfa814be90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcf980118a0 tx=0x7fcf98011bb0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.482+0000 7fcfae240700 1 -- 172.21.15.40:0/2466117263 shutdown_connections 2024-09-13T17:59:18.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.482+0000 7fcfae240700 1 --2- 172.21.15.40:0/2466117263 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcf8c066730 0x7fcf8c068bf0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.482+0000 7fcfae240700 1 --2- 172.21.15.40:0/2466117263 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcfa8156680 0x7fcfa8156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.482+0000 7fcfae240700 1 --2- 172.21.15.40:0/2466117263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcfa806d2d0 0x7fcfa814be90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.482+0000 7fcfae240700 1 --2- 172.21.15.40:0/2466117263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf94003a50 0x7fcfa8155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:18.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.482+0000 7fcfae240700 1 -- 172.21.15.40:0/2466117263 >> 172.21.15.40:0/2466117263 conn(0x7fcfa8066090 msgr2=0x7fcfa814eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:18.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.482+0000 7fcfae240700 1 -- 172.21.15.40:0/2466117263 shutdown_connections 2024-09-13T17:59:18.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:18.482+0000 7fcfae240700 1 -- 172.21.15.40:0/2466117263 wait complete. 2024-09-13T17:59:18.507 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T17:59:28.508 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T17:59:28.508 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:59:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.614+0000 7f27f54c2700 1 Processor -- start 2024-09-13T17:59:28.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.614+0000 7f27f54c2700 1 -- start start 2024-09-13T17:59:28.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.614+0000 7f27f54c2700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f27f0153880 0x7f27f00c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:28.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.614+0000 7f27f54c2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27f00cb2f0 0x7f27f00cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:28.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.614+0000 7f27f54c2700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f0159770 con 0x7f27f00cf3c0 2024-09-13T17:59:28.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.614+0000 7f27f54c2700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f0159a50 con 0x7f27f0153880 2024-09-13T17:59:28.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.614+0000 7f27f54c2700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f0159d30 con 0x7f27f00cb2f0 2024-09-13T17:59:28.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.614+0000 7f27f4cc1700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f27f00cf3c0 0x7f27f015cf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58304/0 (socket says 172.21.15.40:58304) 2024-09-13T17:59:28.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.614+0000 7f27f4cc1700 1 -- 172.21.15.40:0/1679674206 learned_addr learned my addr 172.21.15.40:0/1679674206 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:59:28.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.614+0000 7f27effff700 1 --2- 172.21.15.40:0/1679674206 >> v2:172.21.15.40:3300/0 conn(0x7f27f0153880 0x7f27f00c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:28.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.614+0000 7f27e7fff700 1 --2- 172.21.15.40:0/1679674206 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27f00cb2f0 0x7f27f00cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:28.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.614+0000 7f27effff700 1 -- 172.21.15.40:0/1679674206 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27f00cb2f0 msgr2=0x7f27f00cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:28.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.614+0000 7f27effff700 1 --2- 172.21.15.40:0/1679674206 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27f00cb2f0 0x7f27f00cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:28.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.614+0000 7f27effff700 1 -- 172.21.15.40:0/1679674206 >> v1:172.21.15.40:6789/0 conn(0x7f27f00cf3c0 legacy=0x7f27f015cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:28.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.614+0000 7f27ef7fe700 1 -- 172.21.15.40:0/1679674206 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 976123005 0 0) 0x7f27f0159770 con 0x7f27f00cf3c0 2024-09-13T17:59:28.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.614+0000 7f27effff700 1 -- 172.21.15.40:0/1679674206 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f015a010 con 0x7f27f0153880 2024-09-13T17:59:28.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.614+0000 7f27effff700 1 --2- 172.21.15.40:0/1679674206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27f0153880 0x7f27f00c9ef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009850 tx=0x7f27e0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977b830e7dc6b49e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:28.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.614+0000 7f27ef7fe700 1 -- 172.21.15.40:0/1679674206 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0011830 con 0x7f27f0153880 2024-09-13T17:59:28.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.614+0000 7f27ef7fe700 1 -- 172.21.15.40:0/1679674206 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e0011990 con 0x7f27f0153880 2024-09-13T17:59:28.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.614+0000 7f27ef7fe700 1 -- 172.21.15.40:0/1679674206 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0011d40 con 0x7f27f0153880 2024-09-13T17:59:28.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.618+0000 7f27f54c2700 1 -- 172.21.15.40:0/1679674206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27f0153880 msgr2=0x7f27f00c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:28.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.618+0000 7f27f54c2700 1 --2- 172.21.15.40:0/1679674206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27f0153880 0x7f27f00c9ef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009850 tx=0x7f27e0009b60 comp rx=0 tx=0).stop 2024-09-13T17:59:28.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.618+0000 7f27f54c2700 1 -- 172.21.15.40:0/1679674206 shutdown_connections 2024-09-13T17:59:28.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.618+0000 7f27f54c2700 1 --2- 172.21.15.40:0/1679674206 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27f00cb2f0 0x7f27f00cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:28.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.618+0000 7f27f54c2700 1 --2- 172.21.15.40:0/1679674206 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27f0153880 0x7f27f00c9ef0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:28.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.618+0000 7f27f54c2700 1 -- 172.21.15.40:0/1679674206 >> 172.21.15.40:0/1679674206 conn(0x7f27f006c5b0 msgr2=0x7f27f006c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:28.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.618+0000 7f27f54c2700 1 -- 172.21.15.40:0/1679674206 shutdown_connections 2024-09-13T17:59:28.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.618+0000 7f27f54c2700 1 -- 172.21.15.40:0/1679674206 wait complete. 2024-09-13T17:59:28.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.618+0000 7f27f54c2700 1 Processor -- start 2024-09-13T17:59:28.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.618+0000 7f27f54c2700 1 -- start start 2024-09-13T17:59:28.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.618+0000 7f27f54c2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27f00cb2f0 0x7f27f015c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:28.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.618+0000 7f27f54c2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27f00cf3c0 0x7f27f00b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:28.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.618+0000 7f27f54c2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27f0153880 0x7f27f00b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:28.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.618+0000 7f27f54c2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f00c52f0 con 0x7f27f00cb2f0 2024-09-13T17:59:28.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.618+0000 7f27f54c2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f015be40 con 0x7f27f00cf3c0 2024-09-13T17:59:28.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.618+0000 7f27f54c2700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f00b40a0 con 0x7f27f0153880 2024-09-13T17:59:28.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.618+0000 7f27effff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27f00cb2f0 0x7f27f015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:28.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.618+0000 7f27effff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27f00cb2f0 0x7f27f015c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36906/0 (socket says 172.21.15.40:36906) 2024-09-13T17:59:28.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.618+0000 7f27effff700 1 -- 172.21.15.40:0/731219585 learned_addr learned my addr 172.21.15.40:0/731219585 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:28.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.622+0000 7f27f4cc1700 1 --2- 172.21.15.40:0/731219585 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27f0153880 0x7f27f00b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:28.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.622+0000 7f27e7fff700 1 --2- 172.21.15.40:0/731219585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27f00cf3c0 0x7f27f00b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:28.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.622+0000 7f27effff700 1 -- 172.21.15.40:0/731219585 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27f0153880 msgr2=0x7f27f00b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:28.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.622+0000 7f27effff700 1 --2- 172.21.15.40:0/731219585 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27f0153880 0x7f27f00b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:28.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.622+0000 7f27effff700 1 -- 172.21.15.40:0/731219585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27f00cf3c0 msgr2=0x7f27f00b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:28.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.622+0000 7f27effff700 1 --2- 172.21.15.40:0/731219585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27f00cf3c0 0x7f27f00b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:28.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.622+0000 7f27effff700 1 -- 172.21.15.40:0/731219585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e00094b0 con 0x7f27f00cb2f0 2024-09-13T17:59:28.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.622+0000 7f27effff700 1 --2- 172.21.15.40:0/731219585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27f00cb2f0 0x7f27f015c220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f27e0014040 tx=0x7f27e0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:28.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.622+0000 7f27ed7fa700 1 -- 172.21.15.40:0/731219585 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0019980 con 0x7f27f00cb2f0 2024-09-13T17:59:28.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.622+0000 7f27ed7fa700 1 -- 172.21.15.40:0/731219585 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e001e070 con 0x7f27f00cb2f0 2024-09-13T17:59:28.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.622+0000 7f27f54c2700 1 -- 172.21.15.40:0/731219585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f015ba20 con 0x7f27f00cb2f0 2024-09-13T17:59:28.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.622+0000 7f27ed7fa700 1 -- 172.21.15.40:0/731219585 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0019d20 con 0x7f27f00cb2f0 2024-09-13T17:59:28.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.622+0000 7f27f54c2700 1 -- 172.21.15.40:0/731219585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f00c59c0 con 0x7f27f00cb2f0 2024-09-13T17:59:28.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.622+0000 7f27f54c2700 1 -- 172.21.15.40:0/731219585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f0152830 con 0x7f27f00cb2f0 2024-09-13T17:59:28.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.622+0000 7f27ed7fa700 1 -- 172.21.15.40:0/731219585 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f27e002d450 con 0x7f27f00cb2f0 2024-09-13T17:59:28.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.626+0000 7f27ed7fa700 1 --2- 172.21.15.40:0/731219585 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f27d0066730 0x7f27d0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:28.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.626+0000 7f27ed7fa700 1 -- 172.21.15.40:0/731219585 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f27e0085490 con 0x7f27f00cb2f0 2024-09-13T17:59:28.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.626+0000 7f27e7fff700 1 --2- 172.21.15.40:0/731219585 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f27d0066730 0x7f27d0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:28.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.626+0000 7f27e7fff700 1 --2- 172.21.15.40:0/731219585 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f27d0066730 0x7f27d0068bf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f27f01596a0 tx=0x7f27d8009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:28.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.630+0000 7f27ed7fa700 1 -- 172.21.15.40:0/731219585 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f27e00524d0 con 0x7f27f00cb2f0 2024-09-13T17:59:28.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.898+0000 7f27f54c2700 1 -- 172.21.15.40:0/731219585 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f27f00cee80 con 0x7f27f00cb2f0 2024-09-13T17:59:28.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.902+0000 7f27ed7fa700 1 -- 172.21.15.40:0/731219585 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f27e0020620 con 0x7f27f00cb2f0 2024-09-13T17:59:28.910 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":70,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:59:28.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.902+0000 7f27f54c2700 1 -- 172.21.15.40:0/731219585 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f27d0066730 msgr2=0x7f27d0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:28.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.902+0000 7f27f54c2700 1 --2- 172.21.15.40:0/731219585 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f27d0066730 0x7f27d0068bf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f27f01596a0 tx=0x7f27d8009240 comp rx=0 tx=0).stop 2024-09-13T17:59:28.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.902+0000 7f27f54c2700 1 -- 172.21.15.40:0/731219585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27f00cb2f0 msgr2=0x7f27f015c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:28.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.902+0000 7f27f54c2700 1 --2- 172.21.15.40:0/731219585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27f00cb2f0 0x7f27f015c220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f27e0014040 tx=0x7f27e0004770 comp rx=0 tx=0).stop 2024-09-13T17:59:28.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.902+0000 7f27f54c2700 1 -- 172.21.15.40:0/731219585 shutdown_connections 2024-09-13T17:59:28.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.902+0000 7f27f54c2700 1 --2- 172.21.15.40:0/731219585 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f27d0066730 0x7f27d0068bf0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:28.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.902+0000 7f27f54c2700 1 --2- 172.21.15.40:0/731219585 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27f0153880 0x7f27f00b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:28.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.902+0000 7f27f54c2700 1 --2- 172.21.15.40:0/731219585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27f00cf3c0 0x7f27f00b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:28.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.902+0000 7f27f54c2700 1 --2- 172.21.15.40:0/731219585 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27f00cb2f0 0x7f27f015c220 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:28.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.902+0000 7f27f54c2700 1 -- 172.21.15.40:0/731219585 >> 172.21.15.40:0/731219585 conn(0x7f27f006c5b0 msgr2=0x7f27f0152ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:28.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.902+0000 7f27f54c2700 1 -- 172.21.15.40:0/731219585 shutdown_connections 2024-09-13T17:59:28.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:28.902+0000 7f27f54c2700 1 -- 172.21.15.40:0/731219585 wait complete. 2024-09-13T17:59:28.926 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T17:59:28.926 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T17:59:28.926 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T17:59:28.926 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:59:29.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.030+0000 7fbb60ab2700 1 Processor -- start 2024-09-13T17:59:29.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.030+0000 7fbb60ab2700 1 -- start start 2024-09-13T17:59:29.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.030+0000 7fbb60ab2700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbb5c153720 0x7fbb5c0ca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.030+0000 7fbb60ab2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbb5c0cf9b0 0x7fbb5c06c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.030+0000 7fbb60ab2700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb5c159e50 con 0x7fbb5c06c5b0 2024-09-13T17:59:29.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.030+0000 7fbb60ab2700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb5c0c5ea0 con 0x7fbb5c153720 2024-09-13T17:59:29.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.030+0000 7fbb60ab2700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c15d390 con 0x7fbb5c0cf9b0 2024-09-13T17:59:29.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb5affd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbb5c0cf9b0 0x7fbb5c06c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb5b7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbb5c153720 0x7fbb5c0ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb5affd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbb5c0cf9b0 0x7fbb5c06c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:38434/0 (socket says 172.21.15.40:38434) 2024-09-13T17:59:29.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb5affd700 1 -- 172.21.15.40:0/1150214326 learned_addr learned my addr 172.21.15.40:0/1150214326 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:29.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb5b7fe700 1 --2- 172.21.15.40:0/1150214326 >> v2:172.21.15.43:3300/0 conn(0x7fbb5c153720 0x7fbb5c0ca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:60694/0 (socket says 172.21.15.40:60694) 2024-09-13T17:59:29.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb5affd700 1 -- 172.21.15.40:0/1150214326 >> v2:172.21.15.43:3300/0 conn(0x7fbb5c153720 msgr2=0x7fbb5c0ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:29.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb5affd700 1 --2- 172.21.15.40:0/1150214326 >> v2:172.21.15.43:3300/0 conn(0x7fbb5c153720 0x7fbb5c0ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb5affd700 1 -- 172.21.15.40:0/1150214326 >> v1:172.21.15.43:6789/0 conn(0x7fbb5c06c5b0 legacy=0x7fbb5c06aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb5affd700 1 -- 172.21.15.40:0/1150214326 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c15a080 con 0x7fbb5c0cf9b0 2024-09-13T17:59:29.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb5affd700 1 --2- 172.21.15.40:0/1150214326 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbb5c0cf9b0 0x7fbb5c06c070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009910 tx=0x7fbb50009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c82de58ad9bd975b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:29.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb5a7fc700 1 -- 172.21.15.40:0/1150214326 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50011830 con 0x7fbb5c0cf9b0 2024-09-13T17:59:29.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb5a7fc700 1 -- 172.21.15.40:0/1150214326 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb50011990 con 0x7fbb5c0cf9b0 2024-09-13T17:59:29.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb5a7fc700 1 -- 172.21.15.40:0/1150214326 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50011d40 con 0x7fbb5c0cf9b0 2024-09-13T17:59:29.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb60ab2700 1 -- 172.21.15.40:0/1150214326 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbb5c0cf9b0 msgr2=0x7fbb5c06c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb60ab2700 1 --2- 172.21.15.40:0/1150214326 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbb5c0cf9b0 0x7fbb5c06c070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009910 tx=0x7fbb50009c20 comp rx=0 tx=0).stop 2024-09-13T17:59:29.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb60ab2700 1 -- 172.21.15.40:0/1150214326 shutdown_connections 2024-09-13T17:59:29.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb60ab2700 1 --2- 172.21.15.40:0/1150214326 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbb5c0cf9b0 0x7fbb5c06c070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb60ab2700 1 --2- 172.21.15.40:0/1150214326 >> v2:172.21.15.43:3300/0 conn(0x7fbb5c153720 0x7fbb5c0ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb60ab2700 1 -- 172.21.15.40:0/1150214326 >> 172.21.15.40:0/1150214326 conn(0x7fbb5c0666c0 msgr2=0x7fbb5c0cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:29.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb60ab2700 1 -- 172.21.15.40:0/1150214326 shutdown_connections 2024-09-13T17:59:29.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.034+0000 7fbb60ab2700 1 -- 172.21.15.40:0/1150214326 wait complete. 2024-09-13T17:59:29.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.038+0000 7fbb60ab2700 1 Processor -- start 2024-09-13T17:59:29.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.038+0000 7fbb60ab2700 1 -- start start 2024-09-13T17:59:29.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.038+0000 7fbb60ab2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbb5c06c5b0 0x7fbb5c0cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.038+0000 7fbb60ab2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c0cf9b0 0x7fbb5c0b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.038+0000 7fbb60ab2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb5c153720 0x7fbb5c1f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.038+0000 7fbb60ab2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c0cd830 con 0x7fbb5c0cf9b0 2024-09-13T17:59:29.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.038+0000 7fbb60ab2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c0cedf0 con 0x7fbb5c153720 2024-09-13T17:59:29.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.038+0000 7fbb60ab2700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c0ce2b0 con 0x7fbb5c06c5b0 2024-09-13T17:59:29.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.038+0000 7fbb5affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c0cf9b0 0x7fbb5c0b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.038+0000 7fbb5b7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbb5c06c5b0 0x7fbb5c0cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.038+0000 7fbb5affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c0cf9b0 0x7fbb5c0b2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36910/0 (socket says 172.21.15.40:36910) 2024-09-13T17:59:29.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.038+0000 7fbb5affd700 1 -- 172.21.15.40:0/1888416259 learned_addr learned my addr 172.21.15.40:0/1888416259 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:29.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.038+0000 7fbb5bfff700 1 --2- 172.21.15.40:0/1888416259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb5c153720 0x7fbb5c1f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.038+0000 7fbb5affd700 1 -- 172.21.15.40:0/1888416259 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbb5c06c5b0 msgr2=0x7fbb5c0cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.038+0000 7fbb5affd700 1 --2- 172.21.15.40:0/1888416259 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbb5c06c5b0 0x7fbb5c0cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.038+0000 7fbb5affd700 1 -- 172.21.15.40:0/1888416259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb5c153720 msgr2=0x7fbb5c1f2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.038+0000 7fbb5affd700 1 --2- 172.21.15.40:0/1888416259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb5c153720 0x7fbb5c1f2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.038+0000 7fbb5affd700 1 -- 172.21.15.40:0/1888416259 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb50009570 con 0x7fbb5c0cf9b0 2024-09-13T17:59:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.042+0000 7fbb5affd700 1 --2- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c0cf9b0 0x7fbb5c0b2e20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009c60 tx=0x7fbb50004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.042+0000 7fbb43fff700 1 -- 172.21.15.40:0/1888416259 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50004300 con 0x7fbb5c0cf9b0 2024-09-13T17:59:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.042+0000 7fbb43fff700 1 -- 172.21.15.40:0/1888416259 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5001f070 con 0x7fbb5c0cf9b0 2024-09-13T17:59:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.042+0000 7fbb60ab2700 1 -- 172.21.15.40:0/1888416259 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb5c0cce60 con 0x7fbb5c0cf9b0 2024-09-13T17:59:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.042+0000 7fbb43fff700 1 -- 172.21.15.40:0/1888416259 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb500036c0 con 0x7fbb5c0cf9b0 2024-09-13T17:59:29.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.042+0000 7fbb60ab2700 1 -- 172.21.15.40:0/1888416259 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5c0ce830 con 0x7fbb5c0cf9b0 2024-09-13T17:59:29.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.042+0000 7fbb60ab2700 1 --2- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c0ce9f0 0x7fbb5c1f3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.042+0000 7fbb60ab2700 1 -- 172.21.15.40:0/1888416259 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c1f3560 con 0x7fbb5c0ce9f0 2024-09-13T17:59:29.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.042+0000 7fbb5b7fe700 1 --2- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c0ce9f0 0x7fbb5c1f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.042+0000 7fbb5b7fe700 1 -- 172.21.15.40:0/1888416259 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb5c15ca60 con 0x7fbb5c0ce9f0 2024-09-13T17:59:29.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.042+0000 7fbb5b7fe700 1 --2- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c0ce9f0 0x7fbb5c1f3020 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0119d0 tx=0x7fbb4c011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:29.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.042+0000 7fbb43fff700 1 -- 172.21.15.40:0/1888416259 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c004240 con 0x7fbb5c0ce9f0 2024-09-13T17:59:29.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.042+0000 7fbb43fff700 1 -- 172.21.15.40:0/1888416259 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbb50021030 con 0x7fbb5c0cf9b0 2024-09-13T17:59:29.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.042+0000 7fbb43fff700 1 --2- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbb480667d0 0x7fbb48068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.042+0000 7fbb43fff700 1 -- 172.21.15.40:0/1888416259 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbb50085710 con 0x7fbb5c0cf9b0 2024-09-13T17:59:29.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.042+0000 7fbb5bfff700 1 --2- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbb480667d0 0x7fbb48068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.042+0000 7fbb43fff700 1 -- 172.21.15.40:0/1888416259 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fbb4c0043a0 con 0x7fbb5c0ce9f0 2024-09-13T17:59:29.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.042+0000 7fbb43fff700 1 -- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c0ce9f0 msgr2=0x7fbb5c1f3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.042+0000 7fbb43fff700 1 --2- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c0ce9f0 0x7fbb5c1f3020 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c0119d0 tx=0x7fbb4c011d90 comp rx=0 tx=0).stop 2024-09-13T17:59:29.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.042+0000 7fbb43fff700 1 -- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c0ce9f0 msgr2=0x7fbb5c1f3020 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:59:29.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.046+0000 7fbb43fff700 1 --2- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c0ce9f0 0x7fbb5c1f3020 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.046+0000 7fbb5bfff700 1 --2- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbb480667d0 0x7fbb48068c90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbb540012e0 tx=0x7fbb54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:29.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb60ab2700 1 --2- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c15c2d0 0x7fbb5c1e36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb60ab2700 1 -- 172.21.15.40:0/1888416259 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c074750 con 0x7fbb5c15c2d0 2024-09-13T17:59:29.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb5b7fe700 1 --2- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c15c2d0 0x7fbb5c1e36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb5b7fe700 1 -- 172.21.15.40:0/1888416259 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbb5c153b00 con 0x7fbb5c15c2d0 2024-09-13T17:59:29.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb5b7fe700 1 --2- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c15c2d0 0x7fbb5c1e36c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c01b000 tx=0x7fbb4c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:29.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb43fff700 1 -- 172.21.15.40:0/1888416259 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c01e450 con 0x7fbb5c15c2d0 2024-09-13T17:59:29.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb43fff700 1 -- 172.21.15.40:0/1888416259 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4452 (secure 0 0 0) 0x7fbb4c0283a0 con 0x7fbb5c15c2d0 2024-09-13T17:59:29.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb43fff700 1 -- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c15c2d0 msgr2=0x7fbb5c1e36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb43fff700 1 --2- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c15c2d0 0x7fbb5c1e36c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c01b000 tx=0x7fbb4c004770 comp rx=0 tx=0).stop 2024-09-13T17:59:29.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb43fff700 1 -- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c15c2d0 msgr2=0x7fbb5c1e36c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:59:29.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb43fff700 1 --2- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c15c2d0 0x7fbb5c1e36c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.066 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:59:29.066 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:59:29.066 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:59:29.066 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:59:29.066 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 70, 2024-09-13T17:59:29.066 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:59:29.066 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:59:29.066 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:59:29.066 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:59:29.066 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:29.066 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 12, 2024-09-13T17:59:29.066 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:59:29.066 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:59:29.067 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:59:29.067 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:29.067 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:29.067 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:29.067 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:29.067 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:29.067 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:29.067 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:29.067 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:29.067 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:29.067 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:29.067 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:29.067 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:59:29.068 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:59:29.068 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:29.068 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:29.068 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:29.068 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:29.068 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:29.068 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:29.068 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:29.068 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:29.068 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:29.068 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:29.068 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:29.068 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.069 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:59:29.069 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:59:29.069 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:59:29.069 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:59:29.069 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:59:29.069 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:59:29.069 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:59:29.069 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:59:29.069 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:59:29.069 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:59:29.069 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:59:29.069 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:59:29.069 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:59:29.070 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:59:29.070 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:59:29.070 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:59:29.070 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:59:29.070 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:29.070 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:29.070 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:29.070 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:29.070 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:29.070 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:29.070 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:29.070 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:29.070 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:29.071 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:29.071 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:29.071 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:59:29.071 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.071 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:59:29.071 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.071 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:59:29.071 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:59:29.071 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:29.071 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:29.071 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.071 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:29.071 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:59:29.071 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:29.072 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.072 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.072 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:29.072 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:59:29.072 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:29.072 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.072 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:29.072 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.072 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:59:29.072 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:59:29.072 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:29.072 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:29.072 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:29.072 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.073 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.073 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:59:29.073 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:59:29.073 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:29.073 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:29.073 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.073 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:29.073 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:59:29.073 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:29.073 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.073 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.073 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:29.073 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:59:29.073 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:29.074 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.074 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:29.074 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.074 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:59:29.074 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:59:29.074 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:29.074 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:29.074 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:29.074 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.074 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.074 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:59:29.074 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:59:29.074 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:29.074 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:29.075 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.075 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:29.075 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:59:29.075 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:29.075 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.075 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.075 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:29.075 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:59:29.075 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:29.075 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.075 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:29.075 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.075 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:59:29.075 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:59:29.075 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:29.076 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:29.076 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:29.076 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.076 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:29.076 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.076 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:59:29.076 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:59:29.076 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.076 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:29.076 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:29.076 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:59:29.076 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.076 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:29.076 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:59:29.077 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.077 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:29.077 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:29.077 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:59:29.077 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.077 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:29.077 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.077 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:59:29.077 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:59:29.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb60ab2700 1 -- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbb480667d0 msgr2=0x7fbb48068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb60ab2700 1 --2- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbb480667d0 0x7fbb48068c90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbb540012e0 tx=0x7fbb54009040 comp rx=0 tx=0).stop 2024-09-13T17:59:29.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb60ab2700 1 -- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c0cf9b0 msgr2=0x7fbb5c0b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb5bfff700 1 -- 172.21.15.40:0/1888416259 reap_dead start 2024-09-13T17:59:29.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb60ab2700 1 --2- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c0cf9b0 0x7fbb5c0b2e20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009c60 tx=0x7fbb50004770 comp rx=0 tx=0).stop 2024-09-13T17:59:29.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb60ab2700 1 -- 172.21.15.40:0/1888416259 shutdown_connections 2024-09-13T17:59:29.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb60ab2700 1 --2- 172.21.15.40:0/1888416259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb5c0cf9b0 0x7fbb5c0b2e20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb60ab2700 1 -- 172.21.15.40:0/1888416259 >> 172.21.15.40:0/1888416259 conn(0x7fbb5c0666c0 msgr2=0x7fbb5c151e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:29.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb60ab2700 1 -- 172.21.15.40:0/1888416259 shutdown_connections 2024-09-13T17:59:29.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.054+0000 7fbb60ab2700 1 -- 172.21.15.40:0/1888416259 wait complete. 2024-09-13T17:59:29.079 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:59:29.080 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T17:59:29.080 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:59:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.190+0000 7f429989a700 1 Processor -- start 2024-09-13T17:59:29.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.190+0000 7f429989a700 1 -- start start 2024-09-13T17:59:29.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.194+0000 7f429989a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f429414fa30 0x7f4294158e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.194+0000 7f429989a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f429414fe10 0x7f42940c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.194+0000 7f429989a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4294159ca0 con 0x7f42941561a0 2024-09-13T17:59:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.194+0000 7f429989a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4294072100 con 0x7f429414fa30 2024-09-13T17:59:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.194+0000 7f429989a700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4294159400 con 0x7f429414fe10 2024-09-13T17:59:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.194+0000 7f4298898700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f429414fa30 0x7f4294158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.194+0000 7f4299099700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f42941561a0 0x7f42940c7bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58312/0 (socket says 172.21.15.40:58312) 2024-09-13T17:59:29.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.194+0000 7f4299099700 1 -- 172.21.15.40:0/2059101429 learned_addr learned my addr 172.21.15.40:0/2059101429 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:59:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.194+0000 7f4293fff700 1 --2- 172.21.15.40:0/2059101429 >> v2:172.21.15.43:3300/0 conn(0x7f429414fe10 0x7f42940c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.194+0000 7f4298898700 1 -- 172.21.15.40:0/2059101429 >> v2:172.21.15.43:3300/0 conn(0x7f429414fe10 msgr2=0x7f42940c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.194+0000 7f4298898700 1 --2- 172.21.15.40:0/2059101429 >> v2:172.21.15.43:3300/0 conn(0x7f429414fe10 0x7f42940c6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.194+0000 7f4298898700 1 -- 172.21.15.40:0/2059101429 >> v1:172.21.15.40:6789/0 conn(0x7f42941561a0 legacy=0x7f42940c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.194+0000 7f4298898700 1 -- 172.21.15.40:0/2059101429 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42941596e0 con 0x7f429414fa30 2024-09-13T17:59:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.194+0000 7f4293fff700 1 --2- 172.21.15.40:0/2059101429 >> v2:172.21.15.43:3300/0 conn(0x7f429414fe10 0x7f42940c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T17:59:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.194+0000 7f4298898700 1 --2- 172.21.15.40:0/2059101429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429414fa30 0x7f4294158e00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f42941534c0 tx=0x7f4280009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9488d95df6cc5313 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.194+0000 7f42937fe700 1 -- 172.21.15.40:0/2059101429 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280011830 con 0x7f429414fa30 2024-09-13T17:59:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.194+0000 7f42937fe700 1 -- 172.21.15.40:0/2059101429 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4280011990 con 0x7f429414fa30 2024-09-13T17:59:29.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.194+0000 7f42937fe700 1 -- 172.21.15.40:0/2059101429 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280011d20 con 0x7f429414fa30 2024-09-13T17:59:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.194+0000 7f429989a700 1 -- 172.21.15.40:0/2059101429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429414fa30 msgr2=0x7f4294158e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.194+0000 7f429989a700 1 --2- 172.21.15.40:0/2059101429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429414fa30 0x7f4294158e00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f42941534c0 tx=0x7f4280009b60 comp rx=0 tx=0).stop 2024-09-13T17:59:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.198+0000 7f429989a700 1 -- 172.21.15.40:0/2059101429 shutdown_connections 2024-09-13T17:59:29.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.198+0000 7f429989a700 1 --2- 172.21.15.40:0/2059101429 >> v2:172.21.15.43:3300/0 conn(0x7f429414fe10 0x7f42940c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.198+0000 7f429989a700 1 --2- 172.21.15.40:0/2059101429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f429414fa30 0x7f4294158e00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.198+0000 7f429989a700 1 -- 172.21.15.40:0/2059101429 >> 172.21.15.40:0/2059101429 conn(0x7f429406a390 msgr2=0x7f429406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:29.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.198+0000 7f429989a700 1 -- 172.21.15.40:0/2059101429 shutdown_connections 2024-09-13T17:59:29.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.198+0000 7f429989a700 1 -- 172.21.15.40:0/2059101429 wait complete. 2024-09-13T17:59:29.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.198+0000 7f429989a700 1 Processor -- start 2024-09-13T17:59:29.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.198+0000 7f429989a700 1 -- start start 2024-09-13T17:59:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.198+0000 7f429989a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429414fa30 0x7f42940cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.198+0000 7f429989a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42941561a0 0x7f42940cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.198+0000 7f429989a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4294147a40 0x7f4294147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.198+0000 7f429989a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4294158870 con 0x7f4294147a40 2024-09-13T17:59:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.198+0000 7f429989a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42941419c0 con 0x7f42941561a0 2024-09-13T17:59:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.198+0000 7f429989a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42940c2190 con 0x7f429414fa30 2024-09-13T17:59:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.198+0000 7f4299099700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4294147a40 0x7f4294147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.198+0000 7f4299099700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4294147a40 0x7f4294147ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36946/0 (socket says 172.21.15.40:36946) 2024-09-13T17:59:29.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.198+0000 7f4299099700 1 -- 172.21.15.40:0/3011976996 learned_addr learned my addr 172.21.15.40:0/3011976996 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.198+0000 7f4293fff700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42941561a0 0x7f42940cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.198+0000 7f4298898700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429414fa30 0x7f42940cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.202+0000 7f4299099700 1 -- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429414fa30 msgr2=0x7f42940cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.202+0000 7f4299099700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f429414fa30 0x7f42940cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.202+0000 7f4299099700 1 -- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42941561a0 msgr2=0x7f42940cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.202+0000 7f4299099700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42941561a0 0x7f42940cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.202+0000 7f4299099700 1 -- 172.21.15.40:0/3011976996 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42800094b0 con 0x7f4294147a40 2024-09-13T17:59:29.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.202+0000 7f4293fff700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42941561a0 0x7f42940cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:59:29.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.202+0000 7f4299099700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4294147a40 0x7f4294147ec0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f428c00cbc0 tx=0x7f428c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:29.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.202+0000 7f42917fa700 1 -- 172.21.15.40:0/3011976996 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c007480 con 0x7f4294147a40 2024-09-13T17:59:29.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.202+0000 7f429989a700 1 -- 172.21.15.40:0/3011976996 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42941416e0 con 0x7f4294147a40 2024-09-13T17:59:29.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.202+0000 7f429989a700 1 -- 172.21.15.40:0/3011976996 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4294153840 con 0x7f4294147a40 2024-09-13T17:59:29.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.202+0000 7f42917fa700 1 -- 172.21.15.40:0/3011976996 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428c0075e0 con 0x7f4294147a40 2024-09-13T17:59:29.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.202+0000 7f42917fa700 1 -- 172.21.15.40:0/3011976996 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c007950 con 0x7f4294147a40 2024-09-13T17:59:29.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.202+0000 7f429989a700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4274002a40 0x7f4274004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.202+0000 7f429989a700 1 -- 172.21.15.40:0/3011976996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4274007f40 con 0x7f4274002a40 2024-09-13T17:59:29.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.202+0000 7f4298898700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4274002a40 0x7f4274004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.202+0000 7f4298898700 1 -- 172.21.15.40:0/3011976996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4274008100 con 0x7f4274002a40 2024-09-13T17:59:29.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.206+0000 7f42917fa700 1 -- 172.21.15.40:0/3011976996 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f428c012400 con 0x7f4294147a40 2024-09-13T17:59:29.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.206+0000 7f4298898700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4274002a40 0x7f4274004f00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f42941534c0 tx=0x7f42800047e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.206+0000 7f42917fa700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4278066730 0x7f4278068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.206+0000 7f42917fa700 1 -- 172.21.15.40:0/3011976996 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f428c083c70 con 0x7f4294147a40 2024-09-13T17:59:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.206+0000 7f4293fff700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4278066730 0x7f4278068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.206+0000 7f42917fa700 1 -- 172.21.15.40:0/3011976996 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280028070 con 0x7f4274002a40 2024-09-13T17:59:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.206+0000 7f42917fa700 1 -- 172.21.15.40:0/3011976996 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f42800036c0 con 0x7f4274002a40 2024-09-13T17:59:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.206+0000 7f42917fa700 1 -- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4274002a40 msgr2=0x7f4274004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.206+0000 7f42917fa700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4274002a40 0x7f4274004f00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f42941534c0 tx=0x7f42800047e0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.206+0000 7f42917fa700 1 -- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4274002a40 msgr2=0x7f4274004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:59:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.206+0000 7f42917fa700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4274002a40 0x7f4274004f00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.206+0000 7f4293fff700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4278066730 0x7f4278068bf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f429414f690 tx=0x7f4288008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:29.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.218+0000 7f429989a700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4274006f30 0x7f42740073a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.218+0000 7f429989a700 1 -- 172.21.15.40:0/3011976996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427400a4a0 con 0x7f4274006f30 2024-09-13T17:59:29.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.218+0000 7f4298898700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4274006f30 0x7f42740073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.218+0000 7f4298898700 1 -- 172.21.15.40:0/3011976996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f42740078e0 con 0x7f4274006f30 2024-09-13T17:59:29.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.218+0000 7f4298898700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4274006f30 0x7f42740073a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4280009fd0 tx=0x7f4280004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:29.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.218+0000 7f42917fa700 1 -- 172.21.15.40:0/3011976996 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428002c730 con 0x7f4274006f30 2024-09-13T17:59:29.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.218+0000 7f42917fa700 1 -- 172.21.15.40:0/3011976996 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7f428002c890 con 0x7f4274006f30 2024-09-13T17:59:29.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.218+0000 7f42917fa700 1 -- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4274006f30 msgr2=0x7f42740073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.218+0000 7f42917fa700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4274006f30 0x7f42740073a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4280009fd0 tx=0x7f4280004770 comp rx=0 tx=0).stop 2024-09-13T17:59:29.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.218+0000 7f42917fa700 1 -- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4274006f30 msgr2=0x7f42740073a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:59:29.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.218+0000 7f42917fa700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4274006f30 0x7f42740073a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.231 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:59:29.231 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:59:29.231 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:59:29.231 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:59:29.231 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 70, 2024-09-13T17:59:29.231 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:59:29.231 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:59:29.231 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:59:29.231 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:59:29.231 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:29.232 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 12, 2024-09-13T17:59:29.232 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:59:29.232 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:59:29.232 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:59:29.232 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:29.232 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:29.232 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:29.232 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:29.232 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:29.233 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:29.233 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:29.233 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:29.233 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:29.233 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:29.233 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:29.233 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:59:29.233 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:59:29.233 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:29.233 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:29.233 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:29.233 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:29.233 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:29.233 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:29.234 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:29.234 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:29.234 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:29.234 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:29.234 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:29.234 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.234 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:59:29.234 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:59:29.234 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:59:29.234 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:59:29.234 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:59:29.234 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:59:29.234 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:59:29.234 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:59:29.235 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:59:29.235 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:59:29.235 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:59:29.235 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:59:29.235 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:59:29.235 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:59:29.235 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:59:29.235 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:59:29.235 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:59:29.235 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:29.235 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:29.235 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:29.235 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:29.236 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:29.236 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:29.236 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:29.236 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:29.236 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:29.236 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:29.236 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:29.236 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:59:29.236 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.236 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:59:29.236 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.236 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:59:29.236 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:59:29.237 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:29.237 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:29.237 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.237 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:29.237 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:59:29.237 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:29.237 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.237 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.237 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:29.237 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:59:29.237 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:29.237 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.237 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:29.237 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.238 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:59:29.238 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:59:29.238 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:29.238 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:29.238 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:29.238 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.238 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.238 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:59:29.238 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:59:29.238 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:29.238 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:29.238 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.238 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:29.239 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:59:29.239 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:29.239 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.239 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.239 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:29.239 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:59:29.239 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:29.239 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.239 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:29.239 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.239 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:59:29.239 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:59:29.239 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:29.239 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:29.240 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:29.240 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.240 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.240 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:59:29.240 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:59:29.240 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:29.240 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:29.240 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.240 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:29.240 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:59:29.240 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:29.240 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.240 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.241 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:29.241 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:59:29.241 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:29.241 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.241 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:29.241 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.241 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:59:29.241 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:59:29.241 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:29.241 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:29.241 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:29.241 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.241 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:29.241 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.242 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:59:29.242 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:59:29.242 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.242 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:29.242 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:29.242 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:59:29.242 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.242 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:29.242 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:59:29.242 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.242 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:29.242 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:29.242 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:59:29.242 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.243 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:29.243 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:59:29.243 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.243 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:29.243 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:29.243 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:59:29.243 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.243 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:29.243 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:59:29.243 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.243 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:29.243 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:29.243 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:59:29.244 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.244 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:29.244 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:59:29.244 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.244 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:29.244 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:29.244 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:59:29.244 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.244 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:29.244 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.244 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:59:29.244 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:59:29.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.222+0000 7f429989a700 1 -- 172.21.15.40:0/3011976996 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4278066730 msgr2=0x7f4278068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.222+0000 7f429989a700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4278066730 0x7f4278068bf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f429414f690 tx=0x7f4288008040 comp rx=0 tx=0).stop 2024-09-13T17:59:29.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.222+0000 7f429989a700 1 -- 172.21.15.40:0/3011976996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4294147a40 msgr2=0x7f4294147ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.222+0000 7f429989a700 1 --2- 172.21.15.40:0/3011976996 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4294147a40 0x7f4294147ec0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f428c00cbc0 tx=0x7f428c00cbf0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.222+0000 7f4299099700 1 -- 172.21.15.40:0/3011976996 reap_dead start 2024-09-13T17:59:29.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.222+0000 7f4299099700 1 -- 172.21.15.40:0/3011976996 reap_dead start 2024-09-13T17:59:29.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.222+0000 7f429989a700 1 -- 172.21.15.40:0/3011976996 shutdown_connections 2024-09-13T17:59:29.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.222+0000 7f429989a700 1 -- 172.21.15.40:0/3011976996 >> 172.21.15.40:0/3011976996 conn(0x7f429406a390 msgr2=0x7f42940c5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:29.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.222+0000 7f429989a700 1 -- 172.21.15.40:0/3011976996 shutdown_connections 2024-09-13T17:59:29.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.222+0000 7f429989a700 1 -- 172.21.15.40:0/3011976996 wait complete. 2024-09-13T17:59:29.247 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:59:29.247 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T17:59:29.247 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T17:59:29.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.350+0000 7f05b8e18700 1 Processor -- start 2024-09-13T17:59:29.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.350+0000 7f05b8e18700 1 -- start start 2024-09-13T17:59:29.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b8e18700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b41515a0 0x7f05b406e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b8e18700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f05b40c3de0 0x7f05b40cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b8e18700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b406e580 con 0x7f05b406dde0 2024-09-13T17:59:29.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b8e18700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05b4151980 con 0x7f05b40c3de0 2024-09-13T17:59:29.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b8e18700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b40c3930 con 0x7f05b41515a0 2024-09-13T17:59:29.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b3fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f05b40c3de0 0x7f05b40cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b3fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f05b40c3de0 0x7f05b40cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36954/0 (socket says 172.21.15.40:36954) 2024-09-13T17:59:29.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b3fff700 1 -- 172.21.15.40:0/3729846212 learned_addr learned my addr 172.21.15.40:0/3729846212 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:29.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b37fe700 1 --2- 172.21.15.40:0/3729846212 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b41515a0 0x7f05b406e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b3fff700 1 -- 172.21.15.40:0/3729846212 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b41515a0 msgr2=0x7f05b406e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:29.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b3fff700 1 --2- 172.21.15.40:0/3729846212 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b41515a0 0x7f05b406e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b3fff700 1 -- 172.21.15.40:0/3729846212 >> v1:172.21.15.43:6789/0 conn(0x7f05b406dde0 legacy=0x7f05b415a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b3fff700 1 -- 172.21.15.40:0/3729846212 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b40c2020 con 0x7f05b40c3de0 2024-09-13T17:59:29.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b3fff700 1 --2- 172.21.15.40:0/3729846212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05b40c3de0 0x7f05b40cd3d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f05ac009e50 tx=0x7f05ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72706daa9c9fb06a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:29.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b27fc700 1 -- 172.21.15.40:0/3729846212 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac00e070 con 0x7f05b40c3de0 2024-09-13T17:59:29.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b27fc700 1 -- 172.21.15.40:0/3729846212 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac00b910 con 0x7f05b40c3de0 2024-09-13T17:59:29.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b27fc700 1 -- 172.21.15.40:0/3729846212 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac00bcc0 con 0x7f05b40c3de0 2024-09-13T17:59:29.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b8e18700 1 -- 172.21.15.40:0/3729846212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05b40c3de0 msgr2=0x7f05b40cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b8e18700 1 --2- 172.21.15.40:0/3729846212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05b40c3de0 0x7f05b40cd3d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f05ac009e50 tx=0x7f05ac00b330 comp rx=0 tx=0).stop 2024-09-13T17:59:29.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b8e18700 1 -- 172.21.15.40:0/3729846212 shutdown_connections 2024-09-13T17:59:29.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b8e18700 1 --2- 172.21.15.40:0/3729846212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05b40c3de0 0x7f05b40cd3d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b8e18700 1 --2- 172.21.15.40:0/3729846212 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b41515a0 0x7f05b406e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b8e18700 1 -- 172.21.15.40:0/3729846212 >> 172.21.15.40:0/3729846212 conn(0x7f05b4072070 msgr2=0x7f05b4072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:29.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b8e18700 1 -- 172.21.15.40:0/3729846212 shutdown_connections 2024-09-13T17:59:29.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.354+0000 7f05b8e18700 1 -- 172.21.15.40:0/3729846212 wait complete. 2024-09-13T17:59:29.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b8e18700 1 Processor -- start 2024-09-13T17:59:29.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b8e18700 1 -- start start 2024-09-13T17:59:29.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b8e18700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b406dde0 0x7f05b41460e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b8e18700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05b40c3de0 0x7f05b40c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b8e18700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b41515a0 0x7f05b40c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b8e18700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b40c94a0 con 0x7f05b40c3de0 2024-09-13T17:59:29.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b8e18700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b40c96d0 con 0x7f05b406dde0 2024-09-13T17:59:29.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b8e18700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b4155ed0 con 0x7f05b41515a0 2024-09-13T17:59:29.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05b40c3de0 0x7f05b40c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05b40c3de0 0x7f05b40c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36960/0 (socket says 172.21.15.40:36960) 2024-09-13T17:59:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b2ffd700 1 -- 172.21.15.40:0/3053945891 learned_addr learned my addr 172.21.15.40:0/3053945891 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b37fe700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b406dde0 0x7f05b41460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b3fff700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b41515a0 0x7f05b40c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b2ffd700 1 -- 172.21.15.40:0/3053945891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b41515a0 msgr2=0x7f05b40c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b2ffd700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b41515a0 0x7f05b40c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b2ffd700 1 -- 172.21.15.40:0/3053945891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b406dde0 msgr2=0x7f05b41460e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b2ffd700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05b406dde0 0x7f05b41460e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b2ffd700 1 -- 172.21.15.40:0/3053945891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac009b30 con 0x7f05b40c3de0 2024-09-13T17:59:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b2ffd700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05b40c3de0 0x7f05b40c8a20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f05a800c530 tx=0x7f05a800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f059bfff700 1 -- 172.21.15.40:0/3053945891 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a80052d0 con 0x7f05b40c3de0 2024-09-13T17:59:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f059bfff700 1 -- 172.21.15.40:0/3053945891 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a8005450 con 0x7f05b40c3de0 2024-09-13T17:59:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b8e18700 1 -- 172.21.15.40:0/3053945891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b4154e60 con 0x7f05b40c3de0 2024-09-13T17:59:29.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f059bfff700 1 -- 172.21.15.40:0/3053945891 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a80038e0 con 0x7f05b40c3de0 2024-09-13T17:59:29.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.358+0000 7f05b8e18700 1 -- 172.21.15.40:0/3053945891 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b40c9920 con 0x7f05b40c3de0 2024-09-13T17:59:29.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.362+0000 7f05b8e18700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b406e740 0x7f05b41ee180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.362+0000 7f05b8e18700 1 -- 172.21.15.40:0/3053945891 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b4074750 con 0x7f05b406e740 2024-09-13T17:59:29.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.362+0000 7f05b37fe700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b406e740 0x7f05b41ee180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.362+0000 7f059bfff700 1 -- 172.21.15.40:0/3053945891 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f05a8003a40 con 0x7f05b40c3de0 2024-09-13T17:59:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.362+0000 7f05b37fe700 1 -- 172.21.15.40:0/3053945891 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f05b4158020 con 0x7f05b406e740 2024-09-13T17:59:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.362+0000 7f05b37fe700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b406e740 0x7f05b41ee180 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f05a40119d0 tx=0x7f05a4011d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.362+0000 7f059bfff700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f05a0066730 0x7f05a0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.362+0000 7f05b3fff700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f05a0066730 0x7f05a0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.362+0000 7f059bfff700 1 -- 172.21.15.40:0/3053945891 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f05a8083930 con 0x7f05b40c3de0 2024-09-13T17:59:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.362+0000 7f059bfff700 1 -- 172.21.15.40:0/3053945891 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4004240 con 0x7f05b406e740 2024-09-13T17:59:29.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.362+0000 7f05b3fff700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f05a0066730 0x7f05a0068bf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f05b40c4250 tx=0x7f05ac009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:29.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.366+0000 7f059bfff700 1 -- 172.21.15.40:0/3053945891 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f05a40043a0 con 0x7f05b406e740 2024-09-13T17:59:29.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.366+0000 7f059bfff700 1 -- 172.21.15.40:0/3053945891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b406e740 msgr2=0x7f05b41ee180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.366+0000 7f059bfff700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b406e740 0x7f05b41ee180 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f05a40119d0 tx=0x7f05a4011d90 comp rx=0 tx=0).stop 2024-09-13T17:59:29.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.366+0000 7f059bfff700 1 -- 172.21.15.40:0/3053945891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b406e740 msgr2=0x7f05b41ee180 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:59:29.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.366+0000 7f059bfff700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b406e740 0x7f05b41ee180 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f05a40119d0 tx=0x7f05a4011d90 comp rx=0 tx=0).stop 2024-09-13T17:59:29.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.378+0000 7f05b8e18700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b41ee6c0 0x7f05b41eead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.378+0000 7f05b8e18700 1 -- 172.21.15.40:0/3053945891 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b4157d40 con 0x7f05b41ee6c0 2024-09-13T17:59:29.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.382+0000 7f05b37fe700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b41ee6c0 0x7f05b41eead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.382+0000 7f05b37fe700 1 -- 172.21.15.40:0/3053945891 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f05b40c7880 con 0x7f05b41ee6c0 2024-09-13T17:59:29.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.382+0000 7f05b37fe700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b41ee6c0 0x7f05b41eead0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f05a401b000 tx=0x7f05a4004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:29.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.382+0000 7f059bfff700 1 -- 172.21.15.40:0/3053945891 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a400fd40 con 0x7f05b41ee6c0 2024-09-13T17:59:29.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.382+0000 7f059bfff700 1 -- 172.21.15.40:0/3053945891 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7f05a400fea0 con 0x7f05b41ee6c0 2024-09-13T17:59:29.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.382+0000 7f059bfff700 1 -- 172.21.15.40:0/3053945891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b41ee6c0 msgr2=0x7f05b41eead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.382+0000 7f059bfff700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b41ee6c0 0x7f05b41eead0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f05a401b000 tx=0x7f05a4004770 comp rx=0 tx=0).stop 2024-09-13T17:59:29.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.382+0000 7f059bfff700 1 -- 172.21.15.40:0/3053945891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b41ee6c0 msgr2=0x7f05b41eead0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:59:29.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.382+0000 7f059bfff700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05b41ee6c0 0x7f05b41eead0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.392 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:59:29.393 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:59:29.393 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:59:29.393 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:59:29.393 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 70, 2024-09-13T17:59:29.393 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:59:29.393 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:59:29.393 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T17:59:29.393 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T17:59:29.393 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:29.393 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 12, 2024-09-13T17:59:29.393 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:59:29.393 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:59:29.394 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:59:29.394 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:29.394 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:29.394 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:29.394 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:29.394 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:29.394 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:29.394 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:29.394 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:29.394 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:29.394 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:29.394 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:29.394 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:59:29.395 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:59:29.395 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:29.395 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:29.395 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:29.395 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:29.395 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:29.395 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:29.395 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:29.395 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:29.395 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:29.395 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:29.395 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:29.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.396 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:59:29.396 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:59:29.396 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:59:29.396 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:59:29.396 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:59:29.396 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:59:29.396 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:59:29.396 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:59:29.396 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:59:29.396 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:59:29.396 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:59:29.396 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:59:29.396 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:59:29.396 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:59:29.397 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:59:29.397 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:59:29.397 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:59:29.397 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:29.397 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:29.397 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:29.397 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:29.397 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:29.397 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:29.397 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:29.397 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:29.397 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:29.397 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:29.398 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:29.398 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:59:29.398 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.398 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:59:29.398 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.398 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:59:29.398 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:59:29.398 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:29.398 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:29.398 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.398 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:29.398 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:59:29.398 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:29.398 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.399 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.399 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:29.399 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:59:29.399 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:29.399 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.399 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:29.399 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.399 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:59:29.399 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:59:29.399 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:29.399 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:29.399 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:29.399 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.400 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.400 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:59:29.400 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:59:29.400 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:29.400 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:29.400 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.400 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:29.400 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:59:29.400 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:29.400 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.400 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.400 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:29.400 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:59:29.401 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:29.401 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.401 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:29.401 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.401 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:59:29.401 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:59:29.401 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:29.401 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:29.401 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:29.401 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.401 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.401 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:59:29.401 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:59:29.401 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:29.402 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:29.402 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.402 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:29.402 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:59:29.402 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:29.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.402 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.402 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:29.402 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:59:29.402 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:29.402 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.402 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:29.402 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.403 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:59:29.403 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:59:29.403 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:29.403 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:29.403 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:29.403 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.403 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:29.403 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.403 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:59:29.403 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:59:29.403 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.403 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:29.403 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:29.404 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:59:29.404 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.404 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:29.404 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:59:29.404 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.404 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:29.404 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:29.404 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T17:59:29.404 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.404 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:29.404 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:59:29.404 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.405 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:29.405 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:29.405 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T17:59:29.405 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.405 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:29.405 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:59:29.405 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.405 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:29.405 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:29.405 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:59:29.405 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.405 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:29.405 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:59:29.405 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:29.406 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:29.406 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:29.406 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:59:29.406 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:29.406 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:29.406 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:29.406 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:59:29.406 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:59:29.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.382+0000 7f05b8e18700 1 -- 172.21.15.40:0/3053945891 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f05a0066730 msgr2=0x7f05a0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.382+0000 7f05b8e18700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f05a0066730 0x7f05a0068bf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f05b40c4250 tx=0x7f05ac009000 comp rx=0 tx=0).stop 2024-09-13T17:59:29.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.382+0000 7f05b8e18700 1 -- 172.21.15.40:0/3053945891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05b40c3de0 msgr2=0x7f05b40c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.382+0000 7f05b8e18700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05b40c3de0 0x7f05b40c8a20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f05a800c530 tx=0x7f05a800c840 comp rx=0 tx=0).stop 2024-09-13T17:59:29.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.382+0000 7f05b3fff700 1 -- 172.21.15.40:0/3053945891 reap_dead start 2024-09-13T17:59:29.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.382+0000 7f05b8e18700 1 -- 172.21.15.40:0/3053945891 shutdown_connections 2024-09-13T17:59:29.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.382+0000 7f05b8e18700 1 --2- 172.21.15.40:0/3053945891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05b40c3de0 0x7f05b40c8a20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.382+0000 7f05b8e18700 1 -- 172.21.15.40:0/3053945891 >> 172.21.15.40:0/3053945891 conn(0x7f05b4072070 msgr2=0x7f05b4148300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:29.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.382+0000 7f05b8e18700 1 -- 172.21.15.40:0/3053945891 shutdown_connections 2024-09-13T17:59:29.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.382+0000 7f05b8e18700 1 -- 172.21.15.40:0/3053945891 wait complete. 2024-09-13T17:59:29.408 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T17:59:29.409 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-13T17:59:29.409 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T17:59:29.409 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-13T17:59:29.409 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-13T17:59:29.409 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T17:59:29.409 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:59:29.459 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T17:59:29.459 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T17:59:29.459 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:59:29.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.562+0000 7f65bcc83700 1 Processor -- start 2024-09-13T17:59:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.562+0000 7f65bcc83700 1 -- start start 2024-09-13T17:59:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65bcc83700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f65b81514b0 0x7f65b80c2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65bcc83700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f65b80c7400 0x7f65b80c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65bcc83700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65b815a8b0 con 0x7f65b81510d0 2024-09-13T17:59:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65bcc83700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65b8066390 con 0x7f65b81514b0 2024-09-13T17:59:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65bcc83700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b815ab20 con 0x7f65b80c7400 2024-09-13T17:59:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65b6ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f65b81514b0 0x7f65b80c2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65b6ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f65b81514b0 0x7f65b80c2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37086/0 (socket says 172.21.15.40:37086) 2024-09-13T17:59:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65b6ffd700 1 -- 172.21.15.40:0/1618090726 learned_addr learned my addr 172.21.15.40:0/1618090726 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65b7fff700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f65b80c7400 msgr2=0x7f65b80c4dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:59:29.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65b7fff700 1 --2- 172.21.15.40:0/1618090726 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f65b80c7400 0x7f65b80c4dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:59:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65b6ffd700 1 -- 172.21.15.40:0/1618090726 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f65b80c7400 msgr2=0x7f65b80c4dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:59:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65b6ffd700 1 --2- 172.21.15.40:0/1618090726 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f65b80c7400 0x7f65b80c4dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65b6ffd700 1 -- 172.21.15.40:0/1618090726 >> v1:172.21.15.40:6789/0 conn(0x7f65b81510d0 legacy=0x7f65b8157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65b6ffd700 1 -- 172.21.15.40:0/1618090726 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b815ae00 con 0x7f65b81514b0 2024-09-13T17:59:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65b6ffd700 1 --2- 172.21.15.40:0/1618090726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65b81514b0 0x7f65b80c2770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009910 tx=0x7f65ac009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2398e25c86076c0d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:29.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65b67fc700 1 -- 172.21.15.40:0/1618090726 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac00ba60 con 0x7f65b81514b0 2024-09-13T17:59:29.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65b67fc700 1 -- 172.21.15.40:0/1618090726 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ac00bbc0 con 0x7f65b81514b0 2024-09-13T17:59:29.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65b67fc700 1 -- 172.21.15.40:0/1618090726 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac009e60 con 0x7f65b81514b0 2024-09-13T17:59:29.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65bcc83700 1 -- 172.21.15.40:0/1618090726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65b81514b0 msgr2=0x7f65b80c2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65bcc83700 1 --2- 172.21.15.40:0/1618090726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65b81514b0 0x7f65b80c2770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009910 tx=0x7f65ac009c20 comp rx=0 tx=0).stop 2024-09-13T17:59:29.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65bcc83700 1 -- 172.21.15.40:0/1618090726 shutdown_connections 2024-09-13T17:59:29.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65bcc83700 1 --2- 172.21.15.40:0/1618090726 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f65b80c7400 0x7f65b80c4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65bcc83700 1 --2- 172.21.15.40:0/1618090726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65b81514b0 0x7f65b80c2770 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65bcc83700 1 -- 172.21.15.40:0/1618090726 >> 172.21.15.40:0/1618090726 conn(0x7f65b80c7d70 msgr2=0x7f65b80c8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:29.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.566+0000 7f65bcc83700 1 -- 172.21.15.40:0/1618090726 shutdown_connections 2024-09-13T17:59:29.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.570+0000 7f65bcc83700 1 -- 172.21.15.40:0/1618090726 wait complete. 2024-09-13T17:59:29.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.570+0000 7f65bcc83700 1 Processor -- start 2024-09-13T17:59:29.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.570+0000 7f65bcc83700 1 -- start start 2024-09-13T17:59:29.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.570+0000 7f65bcc83700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65b80c7400 0x7f65b8155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.570+0000 7f65bcc83700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65b81510d0 0x7f65b8148bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.570+0000 7f65bcc83700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f65b81514b0 0x7f65b8156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.570+0000 7f65bcc83700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b8149440 con 0x7f65b81510d0 2024-09-13T17:59:29.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.570+0000 7f65bcc83700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b8147ed0 con 0x7f65b80c7400 2024-09-13T17:59:29.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.570+0000 7f65bcc83700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b81f1050 con 0x7f65b81514b0 2024-09-13T17:59:29.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.570+0000 7f65b7fff700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f65b81514b0 msgr2=0x7f65b8156220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:59:29.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.570+0000 7f65b6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65b81510d0 0x7f65b8148bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.570+0000 7f65b7fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f65b81514b0 0x7f65b8156220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:59:29.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.570+0000 7f65b6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65b81510d0 0x7f65b8148bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37098/0 (socket says 172.21.15.40:37098) 2024-09-13T17:59:29.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.570+0000 7f65b6ffd700 1 -- 172.21.15.40:0/242304116 learned_addr learned my addr 172.21.15.40:0/242304116 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:29.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.570+0000 7f65b77fe700 1 --2- 172.21.15.40:0/242304116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65b80c7400 0x7f65b8155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.570+0000 7f65b6ffd700 1 -- 172.21.15.40:0/242304116 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f65b81514b0 msgr2=0x7f65b8156220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:59:29.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.570+0000 7f65b6ffd700 1 --2- 172.21.15.40:0/242304116 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f65b81514b0 0x7f65b8156220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.570+0000 7f65b6ffd700 1 -- 172.21.15.40:0/242304116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65b80c7400 msgr2=0x7f65b8155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.570+0000 7f65b6ffd700 1 --2- 172.21.15.40:0/242304116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65b80c7400 0x7f65b8155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.570+0000 7f65b6ffd700 1 -- 172.21.15.40:0/242304116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ac009570 con 0x7f65b81510d0 2024-09-13T17:59:29.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.574+0000 7f65b6ffd700 1 --2- 172.21.15.40:0/242304116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65b81510d0 0x7f65b8148bc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009c60 tx=0x7f65ac004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:29.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.574+0000 7f659ffff700 1 -- 172.21.15.40:0/242304116 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac012a00 con 0x7f65b81510d0 2024-09-13T17:59:29.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.574+0000 7f659ffff700 1 -- 172.21.15.40:0/242304116 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ac01e070 con 0x7f65b81510d0 2024-09-13T17:59:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.574+0000 7f65bcc83700 1 -- 172.21.15.40:0/242304116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65b8148730 con 0x7f65b81510d0 2024-09-13T17:59:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.574+0000 7f659ffff700 1 -- 172.21.15.40:0/242304116 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac012d60 con 0x7f65b81510d0 2024-09-13T17:59:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.574+0000 7f65bcc83700 1 -- 172.21.15.40:0/242304116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b8143cb0 con 0x7f65b81510d0 2024-09-13T17:59:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.574+0000 7f65bcc83700 1 -- 172.21.15.40:0/242304116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b81488c0 con 0x7f65b81510d0 2024-09-13T17:59:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.574+0000 7f659ffff700 1 -- 172.21.15.40:0/242304116 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f65ac02d450 con 0x7f65b81510d0 2024-09-13T17:59:29.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.574+0000 7f659ffff700 1 --2- 172.21.15.40:0/242304116 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f65a40666e0 0x7f65a4068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:29.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.574+0000 7f65b77fe700 1 --2- 172.21.15.40:0/242304116 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f65a40666e0 0x7f65a4068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:29.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.574+0000 7f659ffff700 1 -- 172.21.15.40:0/242304116 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f65ac085660 con 0x7f65b81510d0 2024-09-13T17:59:29.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.578+0000 7f65b77fe700 1 --2- 172.21.15.40:0/242304116 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f65a40666e0 0x7f65a4068ba0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f65a8000f60 tx=0x7f65a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:29.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.582+0000 7f659ffff700 1 -- 172.21.15.40:0/242304116 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f65ac052610 con 0x7f65b81510d0 2024-09-13T17:59:29.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.862+0000 7f65bcc83700 1 -- 172.21.15.40:0/242304116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f65b815b600 con 0x7f65b81510d0 2024-09-13T17:59:29.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.866+0000 7f659ffff700 1 -- 172.21.15.40:0/242304116 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f65ac055c30 con 0x7f65b81510d0 2024-09-13T17:59:29.875 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":70,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:59:29.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.866+0000 7f65bcc83700 1 -- 172.21.15.40:0/242304116 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f65a40666e0 msgr2=0x7f65a4068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.866+0000 7f65bcc83700 1 --2- 172.21.15.40:0/242304116 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f65a40666e0 0x7f65a4068ba0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f65a8000f60 tx=0x7f65a8009040 comp rx=0 tx=0).stop 2024-09-13T17:59:29.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.866+0000 7f65bcc83700 1 -- 172.21.15.40:0/242304116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65b81510d0 msgr2=0x7f65b8148bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:29.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.866+0000 7f65bcc83700 1 --2- 172.21.15.40:0/242304116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65b81510d0 0x7f65b8148bc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009c60 tx=0x7f65ac004770 comp rx=0 tx=0).stop 2024-09-13T17:59:29.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.866+0000 7f65bcc83700 1 -- 172.21.15.40:0/242304116 shutdown_connections 2024-09-13T17:59:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.866+0000 7f65bcc83700 1 --2- 172.21.15.40:0/242304116 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f65a40666e0 0x7f65a4068ba0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.866+0000 7f65bcc83700 1 --2- 172.21.15.40:0/242304116 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f65b81514b0 0x7f65b8156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.866+0000 7f65bcc83700 1 --2- 172.21.15.40:0/242304116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65b81510d0 0x7f65b8148bc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.866+0000 7f65bcc83700 1 --2- 172.21.15.40:0/242304116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65b80c7400 0x7f65b8155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.866+0000 7f65bcc83700 1 -- 172.21.15.40:0/242304116 >> 172.21.15.40:0/242304116 conn(0x7f65b80c7d70 msgr2=0x7f65b806d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.866+0000 7f65bcc83700 1 -- 172.21.15.40:0/242304116 shutdown_connections 2024-09-13T17:59:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:29.866+0000 7f65bcc83700 1 -- 172.21.15.40:0/242304116 wait complete. 2024-09-13T17:59:32.898 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:59:33.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.010+0000 7fd7349bd700 1 Processor -- start 2024-09-13T17:59:33.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.010+0000 7fd7349bd700 1 -- start start 2024-09-13T17:59:33.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.010+0000 7fd7349bd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd730071e60 0x7fd7300c0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:33.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.010+0000 7fd7349bd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd730072240 0x7fd7300c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:33.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.010+0000 7fd7349bd700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7300b2210 con 0x7fd7300c28f0 2024-09-13T17:59:33.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.010+0000 7fd7349bd700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7300c3820 con 0x7fd730071e60 2024-09-13T17:59:33.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.010+0000 7fd7349bd700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7300c1780 con 0x7fd730072240 2024-09-13T17:59:33.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.010+0000 7fd72e59c700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd730072240 msgr2=0x7fd7300c32e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:59:33.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.010+0000 7fd72e59c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd730072240 0x7fd7300c32e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:59:33.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.010+0000 7fd72ed9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd730071e60 0x7fd7300c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:33.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.010+0000 7fd72ed9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd730071e60 0x7fd7300c0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39392/0 (socket says 172.21.15.40:39392) 2024-09-13T17:59:33.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.010+0000 7fd72ed9d700 1 -- 172.21.15.40:0/1976842586 learned_addr learned my addr 172.21.15.40:0/1976842586 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:33.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.010+0000 7fd72ed9d700 1 -- 172.21.15.40:0/1976842586 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd730072240 msgr2=0x7fd7300c32e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:59:33.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.010+0000 7fd72ed9d700 1 --2- 172.21.15.40:0/1976842586 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd730072240 0x7fd7300c32e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:33.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.010+0000 7fd72ed9d700 1 -- 172.21.15.40:0/1976842586 >> v1:172.21.15.40:6789/0 conn(0x7fd7300c28f0 legacy=0x7fd7301565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:33.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.010+0000 7fd72ed9d700 1 -- 172.21.15.40:0/1976842586 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7300c1a60 con 0x7fd730071e60 2024-09-13T17:59:33.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.014+0000 7fd72ed9d700 1 --2- 172.21.15.40:0/1976842586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd730071e60 0x7fd7300c0e90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd720009910 tx=0x7fd720009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f564367a36dc3cc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:33.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.014+0000 7fd72dd9b700 1 -- 172.21.15.40:0/1976842586 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72000ba60 con 0x7fd730071e60 2024-09-13T17:59:33.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.014+0000 7fd72dd9b700 1 -- 172.21.15.40:0/1976842586 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72000bbc0 con 0x7fd730071e60 2024-09-13T17:59:33.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.014+0000 7fd72dd9b700 1 -- 172.21.15.40:0/1976842586 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd720009e60 con 0x7fd730071e60 2024-09-13T17:59:33.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.014+0000 7fd7349bd700 1 -- 172.21.15.40:0/1976842586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd730071e60 msgr2=0x7fd7300c0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:33.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.014+0000 7fd7349bd700 1 --2- 172.21.15.40:0/1976842586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd730071e60 0x7fd7300c0e90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd720009910 tx=0x7fd720009c20 comp rx=0 tx=0).stop 2024-09-13T17:59:33.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.014+0000 7fd7349bd700 1 -- 172.21.15.40:0/1976842586 shutdown_connections 2024-09-13T17:59:33.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.014+0000 7fd7349bd700 1 --2- 172.21.15.40:0/1976842586 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd730072240 0x7fd7300c32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:33.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.014+0000 7fd7349bd700 1 --2- 172.21.15.40:0/1976842586 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd730071e60 0x7fd7300c0e90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:33.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.014+0000 7fd7349bd700 1 -- 172.21.15.40:0/1976842586 >> 172.21.15.40:0/1976842586 conn(0x7fd73006e4f0 msgr2=0x7fd73006e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:33.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.014+0000 7fd7349bd700 1 -- 172.21.15.40:0/1976842586 shutdown_connections 2024-09-13T17:59:33.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.014+0000 7fd7349bd700 1 -- 172.21.15.40:0/1976842586 wait complete. 2024-09-13T17:59:33.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.014+0000 7fd7349bd700 1 Processor -- start 2024-09-13T17:59:33.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.014+0000 7fd7349bd700 1 -- start start 2024-09-13T17:59:33.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd7349bd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd730071e60 0x7fd7301483c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:33.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd7349bd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd730072240 0x7fd730157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:33.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd7349bd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd7300c28f0 0x7fd730157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:33.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd7349bd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7300c8670 con 0x7fd730072240 2024-09-13T17:59:33.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd7349bd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7300c7540 con 0x7fd730071e60 2024-09-13T17:59:33.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd7349bd700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd730152920 con 0x7fd7300c28f0 2024-09-13T17:59:33.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd72f59e700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd7300c28f0 msgr2=0x7fd730157590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:59:33.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd72f59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd7300c28f0 0x7fd730157590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:59:33.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd72e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd730072240 0x7fd730157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:33.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd72e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd730072240 0x7fd730157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39408/0 (socket says 172.21.15.40:39408) 2024-09-13T17:59:33.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd72e59c700 1 -- 172.21.15.40:0/693518644 learned_addr learned my addr 172.21.15.40:0/693518644 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:33.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd72ed9d700 1 --2- 172.21.15.40:0/693518644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd730071e60 0x7fd7301483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:33.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd72e59c700 1 -- 172.21.15.40:0/693518644 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd7300c28f0 msgr2=0x7fd730157590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:59:33.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd72e59c700 1 --2- 172.21.15.40:0/693518644 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd7300c28f0 0x7fd730157590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:33.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd72e59c700 1 -- 172.21.15.40:0/693518644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd730071e60 msgr2=0x7fd7301483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:33.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd72e59c700 1 --2- 172.21.15.40:0/693518644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd730071e60 0x7fd7301483c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:33.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd72e59c700 1 -- 172.21.15.40:0/693518644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd720009570 con 0x7fd730072240 2024-09-13T17:59:33.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd72e59c700 1 --2- 172.21.15.40:0/693518644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd730072240 0x7fd730157050 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd72400b780 tx=0x7fd72400ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:33.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd7177fe700 1 -- 172.21.15.40:0/693518644 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd724003e60 con 0x7fd730072240 2024-09-13T17:59:33.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd7177fe700 1 -- 172.21.15.40:0/693518644 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd724005070 con 0x7fd730072240 2024-09-13T17:59:33.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd7349bd700 1 -- 172.21.15.40:0/693518644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7300c7810 con 0x7fd730072240 2024-09-13T17:59:33.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd7177fe700 1 -- 172.21.15.40:0/693518644 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd724005440 con 0x7fd730072240 2024-09-13T17:59:33.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd7349bd700 1 -- 172.21.15.40:0/693518644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7300c6f80 con 0x7fd730072240 2024-09-13T17:59:33.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.018+0000 7fd7349bd700 1 -- 172.21.15.40:0/693518644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7300c2da0 con 0x7fd730072240 2024-09-13T17:59:33.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.022+0000 7fd7177fe700 1 -- 172.21.15.40:0/693518644 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd7240055a0 con 0x7fd730072240 2024-09-13T17:59:33.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.022+0000 7fd7177fe700 1 --2- 172.21.15.40:0/693518644 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd71c066760 0x7fd71c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:33.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.022+0000 7fd7177fe700 1 -- 172.21.15.40:0/693518644 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd724083dd0 con 0x7fd730072240 2024-09-13T17:59:33.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.022+0000 7fd72ed9d700 1 --2- 172.21.15.40:0/693518644 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd71c066760 0x7fd71c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:33.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.022+0000 7fd72ed9d700 1 --2- 172.21.15.40:0/693518644 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd71c066760 0x7fd71c068c20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd720014040 tx=0x7fd720008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:33.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.030+0000 7fd7177fe700 1 -- 172.21.15.40:0/693518644 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd724050e60 con 0x7fd730072240 2024-09-13T17:59:33.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.278+0000 7fd7349bd700 1 -- 172.21.15.40:0/693518644 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd730156850 con 0x7fd730072240 2024-09-13T17:59:33.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.282+0000 7fd7177fe700 1 -- 172.21.15.40:0/693518644 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fd724054480 con 0x7fd730072240 2024-09-13T17:59:33.291 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":70,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:59:33.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.282+0000 7fd7349bd700 1 -- 172.21.15.40:0/693518644 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd71c066760 msgr2=0x7fd71c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:33.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.282+0000 7fd7349bd700 1 --2- 172.21.15.40:0/693518644 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd71c066760 0x7fd71c068c20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd720014040 tx=0x7fd720008030 comp rx=0 tx=0).stop 2024-09-13T17:59:33.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.282+0000 7fd7349bd700 1 -- 172.21.15.40:0/693518644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd730072240 msgr2=0x7fd730157050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:33.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.282+0000 7fd7349bd700 1 --2- 172.21.15.40:0/693518644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd730072240 0x7fd730157050 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd72400b780 tx=0x7fd72400ba90 comp rx=0 tx=0).stop 2024-09-13T17:59:33.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.282+0000 7fd7349bd700 1 -- 172.21.15.40:0/693518644 shutdown_connections 2024-09-13T17:59:33.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.282+0000 7fd7349bd700 1 --2- 172.21.15.40:0/693518644 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd71c066760 0x7fd71c068c20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:33.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.282+0000 7fd7349bd700 1 --2- 172.21.15.40:0/693518644 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd7300c28f0 0x7fd730157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:33.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.282+0000 7fd7349bd700 1 --2- 172.21.15.40:0/693518644 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd730072240 0x7fd730157050 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:33.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.282+0000 7fd7349bd700 1 --2- 172.21.15.40:0/693518644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd730071e60 0x7fd7301483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:33.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.282+0000 7fd7349bd700 1 -- 172.21.15.40:0/693518644 >> 172.21.15.40:0/693518644 conn(0x7fd73006e4f0 msgr2=0x7fd7300cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:33.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.282+0000 7fd7349bd700 1 -- 172.21.15.40:0/693518644 shutdown_connections 2024-09-13T17:59:33.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:33.282+0000 7fd7349bd700 1 -- 172.21.15.40:0/693518644 wait complete. 2024-09-13T17:59:35.161 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:59:35.163 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:59:36.307 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:59:36.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.414+0000 7f0bef59e700 1 Processor -- start 2024-09-13T17:59:36.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.414+0000 7f0bef59e700 1 -- start start 2024-09-13T17:59:36.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.414+0000 7f0bef59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0bf006e790 0x7f0bf006b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:36.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.414+0000 7f0bef59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf015caf0 con 0x7f0bf0159de0 2024-09-13T17:59:36.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.414+0000 7f0bef59e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf015cdd0 con 0x7f0bf0153540 2024-09-13T17:59:36.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.414+0000 7f0bef59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf015a390 con 0x7f0bf006e790 2024-09-13T17:59:36.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.414+0000 7f0beed9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0bf0159de0 0x7f0bf0163ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40218/0 (socket says 172.21.15.40:40218) 2024-09-13T17:59:36.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.414+0000 7f0be7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0bf006e790 0x7f0bf006b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:36.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.414+0000 7f0beed9d700 1 -- 172.21.15.40:0/3992626230 learned_addr learned my addr 172.21.15.40:0/3992626230 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:59:36.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.414+0000 7f0be7fff700 1 -- 172.21.15.40:0/3992626230 >> v1:172.21.15.43:6789/0 conn(0x7f0bf0153540 legacy=0x7f0bf00c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:36.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.414+0000 7f0be7fff700 1 -- 172.21.15.40:0/3992626230 >> v1:172.21.15.40:6789/0 conn(0x7f0bf0159de0 legacy=0x7f0bf0163ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:36.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.414+0000 7f0be7fff700 1 -- 172.21.15.40:0/3992626230 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf015d0b0 con 0x7f0bf006e790 2024-09-13T17:59:36.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.414+0000 7f0be7fff700 1 --2- 172.21.15.40:0/3992626230 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bf006e790 0x7f0bf006b5a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8009850 tx=0x7f0bd8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9129ed486a8d10e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:36.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.414+0000 7f0bedd9b700 1 -- 172.21.15.40:0/3992626230 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8011830 con 0x7f0bf006e790 2024-09-13T17:59:36.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.418+0000 7f0bedd9b700 1 -- 172.21.15.40:0/3992626230 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd8011990 con 0x7f0bf006e790 2024-09-13T17:59:36.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.418+0000 7f0bedd9b700 1 -- 172.21.15.40:0/3992626230 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8011d20 con 0x7f0bf006e790 2024-09-13T17:59:36.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.418+0000 7f0bef59e700 1 -- 172.21.15.40:0/3992626230 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bf006e790 msgr2=0x7f0bf006b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:36.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.418+0000 7f0bef59e700 1 --2- 172.21.15.40:0/3992626230 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bf006e790 0x7f0bf006b5a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8009850 tx=0x7f0bd8009b60 comp rx=0 tx=0).stop 2024-09-13T17:59:36.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.418+0000 7f0bef59e700 1 -- 172.21.15.40:0/3992626230 shutdown_connections 2024-09-13T17:59:36.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.418+0000 7f0bef59e700 1 --2- 172.21.15.40:0/3992626230 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bf006e790 0x7f0bf006b5a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:36.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.418+0000 7f0bef59e700 1 -- 172.21.15.40:0/3992626230 >> 172.21.15.40:0/3992626230 conn(0x7f0bf0066480 msgr2=0x7f0bf0066890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:36.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.418+0000 7f0bef59e700 1 -- 172.21.15.40:0/3992626230 shutdown_connections 2024-09-13T17:59:36.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.418+0000 7f0bef59e700 1 -- 172.21.15.40:0/3992626230 wait complete. 2024-09-13T17:59:36.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.418+0000 7f0bef59e700 1 Processor -- start 2024-09-13T17:59:36.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.418+0000 7f0bef59e700 1 -- start start 2024-09-13T17:59:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.418+0000 7f0bef59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bf006e790 0x7f0bf01f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.418+0000 7f0bef59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0bf0153540 0x7f0bf014ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.418+0000 7f0bef59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bf0159de0 0x7f0bf014f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.418+0000 7f0bef59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf014e860 con 0x7f0bf006e790 2024-09-13T17:59:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.418+0000 7f0bef59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf014dbb0 con 0x7f0bf0159de0 2024-09-13T17:59:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.418+0000 7f0bef59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf00cae00 con 0x7f0bf0153540 2024-09-13T17:59:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.418+0000 7f0be7fff700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0bf0153540 msgr2=0x7f0bf014ecf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:59:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.418+0000 7f0bee59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bf006e790 0x7f0bf01f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.418+0000 7f0be7fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0bf0153540 0x7f0bf014ecf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:59:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.422+0000 7f0bee59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bf006e790 0x7f0bf01f2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39418/0 (socket says 172.21.15.40:39418) 2024-09-13T17:59:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.422+0000 7f0bee59c700 1 -- 172.21.15.40:0/1313899861 learned_addr learned my addr 172.21.15.40:0/1313899861 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.422+0000 7f0beed9d700 1 --2- 172.21.15.40:0/1313899861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bf0159de0 0x7f0bf014f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:36.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.422+0000 7f0bee59c700 1 -- 172.21.15.40:0/1313899861 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0bf0153540 msgr2=0x7f0bf014ecf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:59:36.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.422+0000 7f0bee59c700 1 --2- 172.21.15.40:0/1313899861 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0bf0153540 0x7f0bf014ecf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:36.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.422+0000 7f0bee59c700 1 -- 172.21.15.40:0/1313899861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bf0159de0 msgr2=0x7f0bf014f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:36.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.422+0000 7f0bee59c700 1 --2- 172.21.15.40:0/1313899861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bf0159de0 0x7f0bf014f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:36.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.422+0000 7f0bee59c700 1 -- 172.21.15.40:0/1313899861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd80094b0 con 0x7f0bf006e790 2024-09-13T17:59:36.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.422+0000 7f0bee59c700 1 --2- 172.21.15.40:0/1313899861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bf006e790 0x7f0bf01f2bb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0be000c500 tx=0x7f0be000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:36.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.422+0000 7f0be6ffd700 1 -- 172.21.15.40:0/1313899861 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be00052d0 con 0x7f0bf006e790 2024-09-13T17:59:36.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.422+0000 7f0be6ffd700 1 -- 172.21.15.40:0/1313899861 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be0005430 con 0x7f0bf006e790 2024-09-13T17:59:36.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.422+0000 7f0bef59e700 1 -- 172.21.15.40:0/1313899861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf01f36b0 con 0x7f0bf006e790 2024-09-13T17:59:36.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.422+0000 7f0be6ffd700 1 -- 172.21.15.40:0/1313899861 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be00055a0 con 0x7f0bf006e790 2024-09-13T17:59:36.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.422+0000 7f0bef59e700 1 -- 172.21.15.40:0/1313899861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf014e210 con 0x7f0bf006e790 2024-09-13T17:59:36.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.422+0000 7f0bef59e700 1 -- 172.21.15.40:0/1313899861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf00b3080 con 0x7f0bf006e790 2024-09-13T17:59:36.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.422+0000 7f0be6ffd700 1 -- 172.21.15.40:0/1313899861 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0be00038e0 con 0x7f0bf006e790 2024-09-13T17:59:36.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.422+0000 7f0be6ffd700 1 --2- 172.21.15.40:0/1313899861 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0bd00666e0 0x7f0bd0068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:36.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.422+0000 7f0be6ffd700 1 -- 172.21.15.40:0/1313899861 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0be0083900 con 0x7f0bf006e790 2024-09-13T17:59:36.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.422+0000 7f0be7fff700 1 --2- 172.21.15.40:0/1313899861 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0bd00666e0 0x7f0bd0068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:36.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.426+0000 7f0be7fff700 1 --2- 172.21.15.40:0/1313899861 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0bd00666e0 0x7f0bd0068ba0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0158fd0 tx=0x7f0bd8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.430+0000 7f0be6ffd700 1 -- 172.21.15.40:0/1313899861 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0be00508b0 con 0x7f0bf006e790 2024-09-13T17:59:36.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.686+0000 7f0bef59e700 1 -- 172.21.15.40:0/1313899861 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0bf015c830 con 0x7f0bf006e790 2024-09-13T17:59:36.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.690+0000 7f0be6ffd700 1 -- 172.21.15.40:0/1313899861 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f0be0017620 con 0x7f0bf006e790 2024-09-13T17:59:36.699 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":70,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:59:36.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.690+0000 7f0bef59e700 1 -- 172.21.15.40:0/1313899861 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0bd00666e0 msgr2=0x7f0bd0068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:36.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.690+0000 7f0bef59e700 1 --2- 172.21.15.40:0/1313899861 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0bd00666e0 0x7f0bd0068ba0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0158fd0 tx=0x7f0bd8008030 comp rx=0 tx=0).stop 2024-09-13T17:59:36.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.690+0000 7f0bef59e700 1 -- 172.21.15.40:0/1313899861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bf006e790 msgr2=0x7f0bf01f2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:36.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.690+0000 7f0bef59e700 1 --2- 172.21.15.40:0/1313899861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bf006e790 0x7f0bf01f2bb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0be000c500 tx=0x7f0be000c810 comp rx=0 tx=0).stop 2024-09-13T17:59:36.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.690+0000 7f0bef59e700 1 -- 172.21.15.40:0/1313899861 shutdown_connections 2024-09-13T17:59:36.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.690+0000 7f0bef59e700 1 --2- 172.21.15.40:0/1313899861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0bf0159de0 0x7f0bf014f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:36.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.690+0000 7f0bef59e700 1 --2- 172.21.15.40:0/1313899861 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0bd00666e0 0x7f0bd0068ba0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:36.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.690+0000 7f0bef59e700 1 --2- 172.21.15.40:0/1313899861 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0bf0153540 0x7f0bf014ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:36.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.690+0000 7f0bef59e700 1 --2- 172.21.15.40:0/1313899861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0bf006e790 0x7f0bf01f2bb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:36.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.690+0000 7f0bef59e700 1 -- 172.21.15.40:0/1313899861 >> 172.21.15.40:0/1313899861 conn(0x7f0bf0066480 msgr2=0x7f0bf00c8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:36.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.690+0000 7f0bef59e700 1 -- 172.21.15.40:0/1313899861 shutdown_connections 2024-09-13T17:59:36.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:36.690+0000 7f0bef59e700 1 -- 172.21.15.40:0/1313899861 wait complete. 2024-09-13T17:59:39.720 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T17:59:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:39.842+0000 7ffa066ec700 1 Processor -- start 2024-09-13T17:59:39.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:39.842+0000 7ffa066ec700 1 -- start start 2024-09-13T17:59:39.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:39.842+0000 7ffa066ec700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 0x7ffa000cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:39.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:39.842+0000 7ffa066ec700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa00153be0 con 0x7ffa000cf3c0 2024-09-13T17:59:39.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:39.842+0000 7ffa066ec700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa0015d100 con 0x7ffa00153800 2024-09-13T17:59:39.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:39.842+0000 7ffa066ec700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa000ca4a0 con 0x7ffa0015c9c0 2024-09-13T17:59:39.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:39.842+0000 7ffa04ee9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 0x7ffa000cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:39.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:39.842+0000 7ffa04ee9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 0x7ffa000cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39422/0 (socket says 172.21.15.40:39422) 2024-09-13T17:59:39.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:39.842+0000 7ffa04ee9700 1 -- 172.21.15.40:0/2221306544 learned_addr learned my addr 172.21.15.40:0/2221306544 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:39.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:39.842+0000 7ffa05eeb700 1 --1- 172.21.15.40:0/2221306544 >> v1:172.21.15.40:6789/0 conn(0x7ffa000cf3c0 0x7ffa000c5360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40232/0 (socket says 172.21.15.40:40232) 2024-09-13T17:59:39.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:39.842+0000 7ffa04ee9700 1 -- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 msgr2=0x7ffa000cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:59:39.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:39.842+0000 7ffa04ee9700 1 -- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 msgr2=0x7ffa000cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:59:39.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:39.842+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 0x7ffa000cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:59:39.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:39.842+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 0x7ffa000cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:59:40.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:40.046+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 0x7ffa000cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:40.046+0000 7ffa04ee9700 1 -- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 msgr2=0x7ffa000cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:59:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:40.046+0000 7ffa04ee9700 1 -- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 msgr2=0x7ffa000cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:59:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:40.046+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 0x7ffa000cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:59:40.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:40.046+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 0x7ffa000cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:59:40.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:40.450+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 0x7ffa000cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:40.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:40.450+0000 7ffa04ee9700 1 -- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 msgr2=0x7ffa000cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:59:40.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:40.450+0000 7ffa04ee9700 1 -- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 msgr2=0x7ffa000cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:59:40.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:40.450+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 0x7ffa000cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:59:40.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:40.450+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 0x7ffa000cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:59:41.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:41.254+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 0x7ffa000cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:41.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:41.254+0000 7ffa04ee9700 1 -- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 msgr2=0x7ffa000cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:59:41.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:41.254+0000 7ffa04ee9700 1 -- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 msgr2=0x7ffa000cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:59:41.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:41.254+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 0x7ffa000cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:59:41.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:41.254+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 0x7ffa000cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:59:42.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:42.842+0000 7ff9f6ffd700 1 -- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 msgr2=0x7ffa000cee80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:59:42.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:42.842+0000 7ff9f6ffd700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.40:3300/0 conn(0x7ffa0015c9c0 0x7ffa000cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:42.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:42.842+0000 7ff9f6ffd700 1 -- 172.21.15.40:0/2221306544 >> v1:172.21.15.43:6789/0 conn(0x7ffa00153800 legacy=0x7ffa000c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:42.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:42.842+0000 7ff9f6ffd700 1 -- 172.21.15.40:0/2221306544 >> v1:172.21.15.40:6789/0 conn(0x7ffa000cf3c0 legacy=0x7ffa000c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:42.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:42.842+0000 7ff9f6ffd700 1 --2- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4003790 0x7ff9e4005be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:42.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:42.842+0000 7ff9f6ffd700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 0x7ff9e4008780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:42.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:42.842+0000 7ff9f6ffd700 1 -- 172.21.15.40:0/2221306544 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa000ca4a0 con 0x7ff9e4006300 2024-09-13T17:59:42.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:42.842+0000 7ff9f6ffd700 1 -- 172.21.15.40:0/2221306544 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e4008e30 con 0x7ff9e4003790 2024-09-13T17:59:42.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:42.842+0000 7ffa056ea700 1 -- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4003790 msgr2=0x7ff9e4005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:59:42.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:42.842+0000 7ffa056ea700 1 --2- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4003790 0x7ff9e4005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:59:42.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:42.842+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 0x7ff9e4008780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:42.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:42.842+0000 7ffa04ee9700 1 -- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 msgr2=0x7ff9e4008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:59:42.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:42.842+0000 7ffa04ee9700 1 -- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 msgr2=0x7ff9e4008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:59:42.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:42.842+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 0x7ff9e4008780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:59:42.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:42.842+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 0x7ff9e4008780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:59:43.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:43.046+0000 7ffa056ea700 1 -- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4003790 msgr2=0x7ff9e4005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:59:43.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:43.046+0000 7ffa056ea700 1 --2- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4003790 0x7ff9e4005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:59:43.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:43.046+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 0x7ff9e4008780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:43.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:43.046+0000 7ffa04ee9700 1 -- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 msgr2=0x7ff9e4008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:59:43.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:43.046+0000 7ffa04ee9700 1 -- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 msgr2=0x7ff9e4008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:59:43.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:43.046+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 0x7ff9e4008780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:59:43.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:43.046+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 0x7ff9e4008780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:59:43.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:43.450+0000 7ffa056ea700 1 -- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4003790 msgr2=0x7ff9e4005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:59:43.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:43.450+0000 7ffa056ea700 1 --2- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4003790 0x7ff9e4005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:59:43.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:43.450+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 0x7ff9e4008780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:43.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:43.450+0000 7ffa04ee9700 1 -- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 msgr2=0x7ff9e4008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:59:43.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:43.450+0000 7ffa04ee9700 1 -- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 msgr2=0x7ff9e4008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:59:43.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:43.450+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 0x7ff9e4008780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:59:43.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:43.450+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 0x7ff9e4008780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:59:44.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:44.254+0000 7ffa056ea700 1 -- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4003790 msgr2=0x7ff9e4005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:59:44.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:44.254+0000 7ffa056ea700 1 --2- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4003790 0x7ff9e4005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:59:44.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:44.254+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 0x7ff9e4008780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:44.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:44.254+0000 7ffa04ee9700 1 -- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 msgr2=0x7ff9e4008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:59:44.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:44.254+0000 7ffa04ee9700 1 -- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 msgr2=0x7ff9e4008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:59:44.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:44.254+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 0x7ff9e4008780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:59:44.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:44.254+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 0x7ff9e4008780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T17:59:45.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.841+0000 7ff9f6ffd700 1 -- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4003790 msgr2=0x7ff9e4005be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:59:45.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.841+0000 7ff9f6ffd700 1 --2- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4003790 0x7ff9e4005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:45.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.841+0000 7ff9f6ffd700 1 -- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 msgr2=0x7ff9e4008780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:59:45.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.841+0000 7ff9f6ffd700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4006300 0x7ff9e4008780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:45.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.841+0000 7ffa05eeb700 1 -- 172.21.15.40:0/2221306544 reap_dead start 2024-09-13T17:59:45.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.841+0000 7ff9f6ffd700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4003bd0 0x7ff9e4006770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:45.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.841+0000 7ff9f6ffd700 1 --2- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4006cb0 0x7ff9e40070f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:45.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.841+0000 7ff9f6ffd700 1 -- 172.21.15.40:0/2221306544 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa0015d3e0 con 0x7ff9e4007630 2024-09-13T17:59:45.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.841+0000 7ff9f6ffd700 1 -- 172.21.15.40:0/2221306544 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa000ca4a0 con 0x7ff9e4003bd0 2024-09-13T17:59:45.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.841+0000 7ff9f6ffd700 1 -- 172.21.15.40:0/2221306544 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e4008e30 con 0x7ff9e4006cb0 2024-09-13T17:59:45.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.841+0000 7ffa04ee9700 1 -- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4006cb0 msgr2=0x7ff9e40070f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:59:45.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.841+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4006cb0 0x7ff9e40070f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:59:45.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.841+0000 7ffa056ea700 1 --2- 172.21.15.40:0/2221306544 >> v2:172.21.15.43:3300/0 conn(0x7ff9e4003bd0 0x7ff9e4006770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:45.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.841+0000 7ffa056ea700 1 -- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4006cb0 msgr2=0x7ff9e40070f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:59:45.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.841+0000 7ffa056ea700 1 --2- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4006cb0 0x7ff9e40070f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:45.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.841+0000 7ffa056ea700 1 -- 172.21.15.40:0/2221306544 >> v1:172.21.15.43:6789/0 conn(0x7ff9e4007630 legacy=0x7ff9e4007a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:45.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.841+0000 7ffa056ea700 1 -- 172.21.15.40:0/2221306544 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e40084f0 con 0x7ff9e4003bd0 2024-09-13T17:59:45.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.841+0000 7ffa056ea700 1 --2- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e4003bd0 0x7ff9e4006770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0011d0 tx=0x7ff9fc001c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69c6cb8e4da235ca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:45.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.845+0000 7ff9f7fff700 1 -- 172.21.15.40:0/2221306544 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc005c80 con 0x7ff9e4003bd0 2024-09-13T17:59:45.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.845+0000 7ff9f7fff700 1 -- 172.21.15.40:0/2221306544 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9fc005e00 con 0x7ff9e4003bd0 2024-09-13T17:59:45.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.845+0000 7ff9f7fff700 1 -- 172.21.15.40:0/2221306544 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc007310 con 0x7ff9e4003bd0 2024-09-13T17:59:45.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.845+0000 7ffa066ec700 1 -- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e4003bd0 msgr2=0x7ff9e4006770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:45.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.845+0000 7ffa066ec700 1 --2- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e4003bd0 0x7ff9e4006770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0011d0 tx=0x7ff9fc001c60 comp rx=0 tx=0).stop 2024-09-13T17:59:45.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.845+0000 7ffa066ec700 1 -- 172.21.15.40:0/2221306544 shutdown_connections 2024-09-13T17:59:45.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.845+0000 7ffa066ec700 1 --2- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4006cb0 0x7ff9e40070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:45.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.845+0000 7ffa066ec700 1 --2- 172.21.15.40:0/2221306544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e4003bd0 0x7ff9e4006770 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:45.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.845+0000 7ffa066ec700 1 -- 172.21.15.40:0/2221306544 >> 172.21.15.40:0/2221306544 conn(0x7ffa00066090 msgr2=0x7ffa000664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:45.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.845+0000 7ffa066ec700 1 -- 172.21.15.40:0/2221306544 shutdown_connections 2024-09-13T17:59:45.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.845+0000 7ffa066ec700 1 -- 172.21.15.40:0/2221306544 wait complete. 2024-09-13T17:59:45.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.849+0000 7ffa066ec700 1 Processor -- start 2024-09-13T17:59:45.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.849+0000 7ffa066ec700 1 -- start start 2024-09-13T17:59:45.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.849+0000 7ffa066ec700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4003bd0 0x7ffa001f2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:45.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.849+0000 7ffa066ec700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e4006cb0 0x7ffa0014deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:45.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.849+0000 7ffa066ec700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e4007630 0x7ffa0014e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:45.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.849+0000 7ffa066ec700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000cf7d0 con 0x7ff9e4007630 2024-09-13T17:59:45.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.849+0000 7ffa066ec700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0015c6d0 con 0x7ff9e4006cb0 2024-09-13T17:59:45.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.849+0000 7ffa066ec700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0014f950 con 0x7ff9e4003bd0 2024-09-13T17:59:45.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.849+0000 7ffa056ea700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4003bd0 msgr2=0x7ffa001f2270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:59:45.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.849+0000 7ffa056ea700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4003bd0 0x7ffa001f2270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:59:45.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.849+0000 7ffa05eeb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e4007630 0x7ffa0014e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:45.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.849+0000 7ffa04ee9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e4006cb0 0x7ffa0014deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:45.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.849+0000 7ffa05eeb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e4007630 0x7ffa0014e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54298/0 (socket says 172.21.15.40:54298) 2024-09-13T17:59:45.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.849+0000 7ffa05eeb700 1 -- 172.21.15.40:0/1301890027 learned_addr learned my addr 172.21.15.40:0/1301890027 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:45.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.853+0000 7ffa04ee9700 1 -- 172.21.15.40:0/1301890027 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4003bd0 msgr2=0x7ffa001f2270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:59:45.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.853+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/1301890027 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4003bd0 0x7ffa001f2270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:45.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.853+0000 7ffa04ee9700 1 -- 172.21.15.40:0/1301890027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e4007630 msgr2=0x7ffa0014e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:45.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.853+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/1301890027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e4007630 0x7ffa0014e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:45.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.853+0000 7ffa04ee9700 1 -- 172.21.15.40:0/1301890027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0015d100 con 0x7ff9e4006cb0 2024-09-13T17:59:45.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.853+0000 7ffa05eeb700 1 --2- 172.21.15.40:0/1301890027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e4007630 0x7ffa0014e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:59:45.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.853+0000 7ffa04ee9700 1 --2- 172.21.15.40:0/1301890027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e4006cb0 0x7ffa0014deb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0016790 tx=0x7ff9f0016aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:45.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.853+0000 7ff9f5ffb700 1 -- 172.21.15.40:0/1301890027 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f001e070 con 0x7ff9e4006cb0 2024-09-13T17:59:45.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.853+0000 7ff9f5ffb700 1 -- 172.21.15.40:0/1301890027 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f000c040 con 0x7ff9e4006cb0 2024-09-13T17:59:45.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.853+0000 7ffa066ec700 1 -- 172.21.15.40:0/1301890027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa000b4e10 con 0x7ff9e4006cb0 2024-09-13T17:59:45.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.853+0000 7ff9f5ffb700 1 -- 172.21.15.40:0/1301890027 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0004260 con 0x7ff9e4006cb0 2024-09-13T17:59:45.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.853+0000 7ffa066ec700 1 -- 172.21.15.40:0/1301890027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0014f720 con 0x7ff9e4006cb0 2024-09-13T17:59:45.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.853+0000 7ffa066ec700 1 -- 172.21.15.40:0/1301890027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa000cee80 con 0x7ff9e4006cb0 2024-09-13T17:59:45.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.857+0000 7ff9f5ffb700 1 -- 172.21.15.40:0/1301890027 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff9f00043c0 con 0x7ff9e4006cb0 2024-09-13T17:59:45.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.857+0000 7ff9f5ffb700 1 --2- 172.21.15.40:0/1301890027 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff9e40684d0 0x7ff9e406a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:45.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.857+0000 7ff9f5ffb700 1 -- 172.21.15.40:0/1301890027 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff9f0084e10 con 0x7ff9e4006cb0 2024-09-13T17:59:45.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.857+0000 7ffa056ea700 1 --2- 172.21.15.40:0/1301890027 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff9e40684d0 0x7ff9e406a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:45.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.857+0000 7ffa056ea700 1 --2- 172.21.15.40:0/1301890027 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff9e40684d0 0x7ff9e406a990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc015040 tx=0x7ff9fc007f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:45.869+0000 7ff9f5ffb700 1 -- 172.21.15.40:0/1301890027 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff9f0055860 con 0x7ff9e4006cb0 2024-09-13T17:59:46.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.133+0000 7ffa066ec700 1 -- 172.21.15.40:0/1301890027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ffa000c5ef0 con 0x7ff9e4006cb0 2024-09-13T17:59:46.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.133+0000 7ff9f5ffb700 1 -- 172.21.15.40:0/1301890027 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7ff9f00553f0 con 0x7ff9e4006cb0 2024-09-13T17:59:46.144 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":74,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T17:59:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.133+0000 7ffa066ec700 1 -- 172.21.15.40:0/1301890027 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff9e40684d0 msgr2=0x7ff9e406a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.133+0000 7ffa066ec700 1 --2- 172.21.15.40:0/1301890027 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff9e40684d0 0x7ff9e406a990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc015040 tx=0x7ff9fc007f90 comp rx=0 tx=0).stop 2024-09-13T17:59:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.133+0000 7ffa066ec700 1 -- 172.21.15.40:0/1301890027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e4006cb0 msgr2=0x7ffa0014deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.133+0000 7ffa066ec700 1 --2- 172.21.15.40:0/1301890027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e4006cb0 0x7ffa0014deb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0016790 tx=0x7ff9f0016aa0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.137+0000 7ffa066ec700 1 -- 172.21.15.40:0/1301890027 shutdown_connections 2024-09-13T17:59:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.137+0000 7ffa066ec700 1 --2- 172.21.15.40:0/1301890027 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9e4007630 0x7ffa0014e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.137+0000 7ffa066ec700 1 --2- 172.21.15.40:0/1301890027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e4006cb0 0x7ffa0014deb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.137+0000 7ffa066ec700 1 --2- 172.21.15.40:0/1301890027 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff9e40684d0 0x7ff9e406a990 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.137+0000 7ffa066ec700 1 --2- 172.21.15.40:0/1301890027 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff9e4003bd0 0x7ffa001f2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.137+0000 7ffa066ec700 1 -- 172.21.15.40:0/1301890027 >> 172.21.15.40:0/1301890027 conn(0x7ffa00066090 msgr2=0x7ffa00066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:46.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.137+0000 7ffa066ec700 1 -- 172.21.15.40:0/1301890027 shutdown_connections 2024-09-13T17:59:46.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.137+0000 7ffa066ec700 1 -- 172.21.15.40:0/1301890027 wait complete. 2024-09-13T17:59:46.160 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T17:59:46.160 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-13T17:59:46.161 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T17:59:46.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.273+0000 7fcc17d6a700 1 Processor -- start 2024-09-13T17:59:46.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.273+0000 7fcc17d6a700 1 -- start start 2024-09-13T17:59:46.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.273+0000 7fcc17d6a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcc101518a0 0x7fcc101589b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:46.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc17d6a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc10151c80 con 0x7fcc100c8380 2024-09-13T17:59:46.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc17d6a700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc100c8830 con 0x7fcc1006ddb0 2024-09-13T17:59:46.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc17d6a700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc100c6a60 con 0x7fcc101518a0 2024-09-13T17:59:46.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc15305700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcc100c8380 0x7fcc10071ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59838/0 (socket says 172.21.15.40:59838) 2024-09-13T17:59:46.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc15305700 1 -- 172.21.15.40:0/1069573651 learned_addr learned my addr 172.21.15.40:0/1069573651 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T17:59:46.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc15b06700 1 --2- 172.21.15.40:0/1069573651 >> v2:172.21.15.43:3300/0 conn(0x7fcc101518a0 0x7fcc101589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:46.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc14b04700 1 -- 172.21.15.40:0/1069573651 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180020110 0 0) 0x7fcc10151c80 con 0x7fcc100c8380 2024-09-13T17:59:46.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc14b04700 1 -- 172.21.15.40:0/1069573651 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbfc0035d0 con 0x7fcc100c8380 2024-09-13T17:59:46.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc14b04700 1 -- 172.21.15.40:0/1069573651 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2807004401 0 0) 0x7fcbfc0035d0 con 0x7fcc100c8380 2024-09-13T17:59:46.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc14b04700 1 -- 172.21.15.40:0/1069573651 >> v2:172.21.15.43:3300/0 conn(0x7fcc101518a0 msgr2=0x7fcc101589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:46.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc14b04700 1 --2- 172.21.15.40:0/1069573651 >> v2:172.21.15.43:3300/0 conn(0x7fcc101518a0 0x7fcc101589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc14b04700 1 -- 172.21.15.40:0/1069573651 >> v1:172.21.15.43:6789/0 conn(0x7fcc1006ddb0 legacy=0x7fcc10161de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:46.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc14b04700 1 -- 172.21.15.40:0/1069573651 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc100c6d40 con 0x7fcc100c8380 2024-09-13T17:59:46.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc14b04700 1 -- 172.21.15.40:0/1069573651 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fcc00001670 con 0x7fcc100c8380 2024-09-13T17:59:46.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc14b04700 1 -- 172.21.15.40:0/1069573651 >> v1:172.21.15.40:6789/0 conn(0x7fcc100c8380 legacy=0x7fcc10071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:46.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc14b04700 1 --2- 172.21.15.40:0/1069573651 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcbfc003a20 0x7fcbfc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:46.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc14b04700 1 --2- 172.21.15.40:0/1069573651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbfc004870 0x7fcbfc004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc14b04700 1 --2- 172.21.15.40:0/1069573651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbfc005230 0x7fcbfc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc14b04700 1 -- 172.21.15.40:0/1069573651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc100c6a60 con 0x7fcbfc004870 2024-09-13T17:59:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc14b04700 1 -- 172.21.15.40:0/1069573651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc00c0d0 con 0x7fcbfc005230 2024-09-13T17:59:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc14b04700 1 -- 172.21.15.40:0/1069573651 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc00c290 con 0x7fcbfc003a20 2024-09-13T17:59:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc15b06700 1 -- 172.21.15.40:0/1069573651 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcbfc003a20 msgr2=0x7fcbfc003e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:59:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc15b06700 1 --2- 172.21.15.40:0/1069573651 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcbfc003a20 0x7fcbfc003e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:59:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc16307700 1 --2- 172.21.15.40:0/1069573651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbfc005230 0x7fcbfc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc15305700 1 --2- 172.21.15.40:0/1069573651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbfc004870 0x7fcbfc004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc15305700 1 -- 172.21.15.40:0/1069573651 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcbfc003a20 msgr2=0x7fcbfc003e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:59:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc15305700 1 --2- 172.21.15.40:0/1069573651 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcbfc003a20 0x7fcbfc003e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc15305700 1 -- 172.21.15.40:0/1069573651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbfc005230 msgr2=0x7fcbfc00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc15305700 1 --2- 172.21.15.40:0/1069573651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbfc005230 0x7fcbfc00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc15305700 1 -- 172.21.15.40:0/1069573651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc0035d0 con 0x7fcbfc004870 2024-09-13T17:59:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc16307700 1 -- 172.21.15.40:0/1069573651 reap_dead start 2024-09-13T17:59:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc15305700 1 --2- 172.21.15.40:0/1069573651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbfc004870 0x7fcbfc004cf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000c2a0 tx=0x7fcbfc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e5fe1da38aef0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc14b04700 1 -- 172.21.15.40:0/1069573651 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00007c30 con 0x7fcbfc004870 2024-09-13T17:59:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc14b04700 1 -- 172.21.15.40:0/1069573651 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0000ce00 con 0x7fcbfc004870 2024-09-13T17:59:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc14b04700 1 -- 172.21.15.40:0/1069573651 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00005bc0 con 0x7fcbfc004870 2024-09-13T17:59:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc17d6a700 1 -- 172.21.15.40:0/1069573651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbfc004870 msgr2=0x7fcbfc004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc17d6a700 1 --2- 172.21.15.40:0/1069573651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbfc004870 0x7fcbfc004cf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000c2a0 tx=0x7fcbfc0058a0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc17d6a700 1 -- 172.21.15.40:0/1069573651 shutdown_connections 2024-09-13T17:59:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc17d6a700 1 --2- 172.21.15.40:0/1069573651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcbfc004870 0x7fcbfc004cf0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc17d6a700 1 -- 172.21.15.40:0/1069573651 >> 172.21.15.40:0/1069573651 conn(0x7fcc100c8ac0 msgr2=0x7fcc100c8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc17d6a700 1 -- 172.21.15.40:0/1069573651 shutdown_connections 2024-09-13T17:59:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.277+0000 7fcc17d6a700 1 -- 172.21.15.40:0/1069573651 wait complete. 2024-09-13T17:59:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.281+0000 7fcc17d6a700 1 Processor -- start 2024-09-13T17:59:46.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.281+0000 7fcc17d6a700 1 -- start start 2024-09-13T17:59:46.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.281+0000 7fcc17d6a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcbfc004870 0x7fcc10150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:46.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.281+0000 7fcc17d6a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc1014d9c0 0x7fcc1014f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:46.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.281+0000 7fcc17d6a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc1014c6c0 0x7fcc1014cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:46.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.281+0000 7fcc17d6a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1014c4f0 con 0x7fcc1014d9c0 2024-09-13T17:59:46.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.281+0000 7fcc17d6a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc100b13e0 con 0x7fcc1014c6c0 2024-09-13T17:59:46.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.281+0000 7fcc17d6a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1014bcb0 con 0x7fcbfc004870 2024-09-13T17:59:46.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.281+0000 7fcc15b06700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcbfc004870 msgr2=0x7fcc10150b60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:59:46.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.281+0000 7fcc15b06700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcbfc004870 0x7fcc10150b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:59:46.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.281+0000 7fcc15305700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc1014d9c0 0x7fcc1014f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:46.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.281+0000 7fcc15305700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc1014d9c0 0x7fcc1014f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54324/0 (socket says 172.21.15.40:54324) 2024-09-13T17:59:46.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.281+0000 7fcc15305700 1 -- 172.21.15.40:0/1281400311 learned_addr learned my addr 172.21.15.40:0/1281400311 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:46.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.281+0000 7fcc16307700 1 --2- 172.21.15.40:0/1281400311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc1014c6c0 0x7fcc1014cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:46.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.281+0000 7fcc15305700 1 -- 172.21.15.40:0/1281400311 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcbfc004870 msgr2=0x7fcc10150b60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:59:46.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.281+0000 7fcc15305700 1 --2- 172.21.15.40:0/1281400311 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcbfc004870 0x7fcc10150b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.281+0000 7fcc15305700 1 -- 172.21.15.40:0/1281400311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc1014c6c0 msgr2=0x7fcc1014cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:46.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.281+0000 7fcc15305700 1 --2- 172.21.15.40:0/1281400311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc1014c6c0 0x7fcc1014cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.281+0000 7fcc15305700 1 -- 172.21.15.40:0/1281400311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1015a5e0 con 0x7fcc1014d9c0 2024-09-13T17:59:46.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.281+0000 7fcc15305700 1 --2- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc1014d9c0 0x7fcc1014f130 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00c0a0 tx=0x7fcc10157540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:46.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.285+0000 7fcc067fc700 1 -- 172.21.15.40:0/1281400311 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00001670 con 0x7fcc1014d9c0 2024-09-13T17:59:46.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.285+0000 7fcc067fc700 1 -- 172.21.15.40:0/1281400311 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc000070d0 con 0x7fcc1014d9c0 2024-09-13T17:59:46.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.285+0000 7fcc17d6a700 1 -- 172.21.15.40:0/1281400311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc101e28e0 con 0x7fcc1014d9c0 2024-09-13T17:59:46.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.285+0000 7fcc067fc700 1 -- 172.21.15.40:0/1281400311 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00007480 con 0x7fcc1014d9c0 2024-09-13T17:59:46.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.285+0000 7fcc17d6a700 1 -- 172.21.15.40:0/1281400311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1014c210 con 0x7fcc1014d9c0 2024-09-13T17:59:46.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.285+0000 7fcc17d6a700 1 --2- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc10158ec0 0x7fcc101592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:46.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.285+0000 7fcc17d6a700 1 -- 172.21.15.40:0/1281400311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc10067840 con 0x7fcc10158ec0 2024-09-13T17:59:46.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.285+0000 7fcc15b06700 1 --2- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc10158ec0 0x7fcc101592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:46.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.285+0000 7fcc15b06700 1 -- 172.21.15.40:0/1281400311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc1014d090 con 0x7fcc10158ec0 2024-09-13T17:59:46.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.285+0000 7fcc15b06700 1 --2- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc10158ec0 0x7fcc101592a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00e8b0 tx=0x7fcc0c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:46.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.285+0000 7fcc067fc700 1 -- 172.21.15.40:0/1281400311 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcc000075e0 con 0x7fcc1014d9c0 2024-09-13T17:59:46.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.285+0000 7fcc067fc700 1 --2- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcbf4066680 0x7fcbf4068b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:46.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.285+0000 7fcc16307700 1 --2- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcbf4066680 0x7fcbf4068b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:46.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.285+0000 7fcc067fc700 1 -- 172.21.15.40:0/1281400311 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c009710 con 0x7fcc10158ec0 2024-09-13T17:59:46.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.285+0000 7fcc067fc700 1 -- 172.21.15.40:0/1281400311 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcc00085730 con 0x7fcc1014d9c0 2024-09-13T17:59:46.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.285+0000 7fcc067fc700 1 -- 172.21.15.40:0/1281400311 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fcc0c004020 con 0x7fcc10158ec0 2024-09-13T17:59:46.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.285+0000 7fcc067fc700 1 -- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc10158ec0 msgr2=0x7fcc101592a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:46.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.285+0000 7fcc067fc700 1 --2- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc10158ec0 0x7fcc101592a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00e8b0 tx=0x7fcc0c00ec70 comp rx=0 tx=0).stop 2024-09-13T17:59:46.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.285+0000 7fcc067fc700 1 -- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc10158ec0 msgr2=0x7fcc101592a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:59:46.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.285+0000 7fcc067fc700 1 --2- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc10158ec0 0x7fcc101592a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.289+0000 7fcc16307700 1 --2- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcbf4066680 0x7fcbf4068b40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcc1006e3f0 tx=0x7fcc08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:46.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc17d6a700 1 --2- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc101597e0 0x7fcc101e3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:46.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc17d6a700 1 -- 172.21.15.40:0/1281400311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc10070e00 con 0x7fcc101597e0 2024-09-13T17:59:46.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc15b06700 1 --2- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc101597e0 0x7fcc101e3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:46.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc15b06700 1 -- 172.21.15.40:0/1281400311 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcc100c7300 con 0x7fcc101597e0 2024-09-13T17:59:46.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc15b06700 1 --2- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc101597e0 0x7fcc101e3870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00e8b0 tx=0x7fcc0c0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:46.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc067fc700 1 -- 172.21.15.40:0/1281400311 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c022070 con 0x7fcc101597e0 2024-09-13T17:59:46.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc067fc700 1 -- 172.21.15.40:0/1281400311 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4772 (secure 0 0 0) 0x7fcc0c0045d0 con 0x7fcc101597e0 2024-09-13T17:59:46.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc067fc700 1 -- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc101597e0 msgr2=0x7fcc101e3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:46.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc067fc700 1 --2- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc101597e0 0x7fcc101e3870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00e8b0 tx=0x7fcc0c0047e0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc067fc700 1 -- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc101597e0 msgr2=0x7fcc101e3870 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:59:46.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc067fc700 1 --2- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc101597e0 0x7fcc101e3870 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.314 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:59:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:59:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:59:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T17:59:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 74, 2024-09-13T17:59:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:59:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:59:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: 1 2024-09-13T17:59:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T17:59:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:59:46.316 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:59:46.316 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:59:46.316 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:46.316 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:46.316 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:46.317 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:46.317 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:46.317 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:46.317 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:46.317 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:46.318 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:46.318 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:46.318 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:46.318 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:59:46.318 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:59:46.318 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:46.319 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:46.319 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:46.319 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:46.319 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:46.319 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:46.320 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:46.320 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:46.320 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:46.320 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:46.320 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:46.321 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.321 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:59:46.321 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:59:46.321 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:59:46.321 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:59:46.321 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:59:46.322 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:59:46.322 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:59:46.322 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:59:46.322 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:59:46.322 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:59:46.322 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:59:46.323 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:59:46.323 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:59:46.323 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:59:46.323 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:59:46.323 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:59:46.323 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:59:46.324 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:46.324 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:46.324 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:46.324 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:46.324 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:46.324 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:46.324 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:46.324 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:46.324 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:46.324 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:46.324 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:46.324 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:59:46.325 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.325 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:59:46.325 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.325 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:59:46.325 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:59:46.325 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:46.325 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:46.325 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.325 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:46.325 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:59:46.326 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:46.326 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.326 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.326 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:46.326 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:59:46.326 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:46.326 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:46.326 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:46.327 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.327 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:59:46.327 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:59:46.327 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:46.327 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:46.327 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:46.328 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.328 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.328 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:59:46.328 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:59:46.328 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:46.328 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:46.328 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.328 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:46.329 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:59:46.329 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:46.329 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.329 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.329 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:46.329 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:59:46.329 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:46.330 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:46.330 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:46.330 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.330 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:59:46.330 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:59:46.330 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:46.330 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:46.330 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:46.330 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.330 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.330 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:59:46.330 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:59:46.330 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:46.331 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:46.331 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.331 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:46.331 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:59:46.331 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:46.331 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.331 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.331 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:46.331 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:59:46.331 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:46.331 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:46.331 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:46.331 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.331 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:59:46.332 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:59:46.332 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:46.332 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:46.332 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:46.332 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:46.332 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:46.332 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.332 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:59:46.332 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:59:46.332 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.332 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:46.332 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:46.332 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:59:46.333 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:46.333 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:46.333 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:59:46.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.333 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:46.333 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:46.333 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T17:59:46.333 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:46.333 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:46.333 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:59:46.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.333 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:46.333 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:46.333 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:59:46.334 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:46.334 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:46.334 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:59:46.334 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.334 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:46.334 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:46.334 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T17:59:46.334 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:46.334 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:46.334 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.334 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:59:46.334 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:59:46.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc17d6a700 1 -- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcbf4066680 msgr2=0x7fcbf4068b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:46.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc17d6a700 1 --2- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcbf4066680 0x7fcbf4068b40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcc1006e3f0 tx=0x7fcc08009040 comp rx=0 tx=0).stop 2024-09-13T17:59:46.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc17d6a700 1 -- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc1014d9c0 msgr2=0x7fcc1014f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:46.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc17d6a700 1 --2- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc1014d9c0 0x7fcc1014f130 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00c0a0 tx=0x7fcc10157540 comp rx=0 tx=0).stop 2024-09-13T17:59:46.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc16307700 1 -- 172.21.15.40:0/1281400311 reap_dead start 2024-09-13T17:59:46.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc17d6a700 1 -- 172.21.15.40:0/1281400311 shutdown_connections 2024-09-13T17:59:46.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc17d6a700 1 --2- 172.21.15.40:0/1281400311 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc1014d9c0 0x7fcc1014f130 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc17d6a700 1 -- 172.21.15.40:0/1281400311 >> 172.21.15.40:0/1281400311 conn(0x7fcc100c8ac0 msgr2=0x7fcc100cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:46.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc17d6a700 1 -- 172.21.15.40:0/1281400311 shutdown_connections 2024-09-13T17:59:46.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.297+0000 7fcc17d6a700 1 -- 172.21.15.40:0/1281400311 wait complete. 2024-09-13T17:59:46.337 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T17:59:46.337 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-13T17:59:46.337 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T17:59:46.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.445+0000 7f3900995700 1 Processor -- start 2024-09-13T17:59:46.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.445+0000 7f3900995700 1 -- start start 2024-09-13T17:59:46.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.445+0000 7f3900995700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f38fc0cf9b0 0x7f38fc06c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:46.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.445+0000 7f3900995700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f38fc06c5b0 0x7f38fc163d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:46.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.445+0000 7f3900995700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38fc159e50 con 0x7f38fc153720 2024-09-13T17:59:46.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.445+0000 7f3900995700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc0c5ea0 con 0x7f38fc06c5b0 2024-09-13T17:59:46.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.445+0000 7f3900995700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc15d390 con 0x7f38fc0cf9b0 2024-09-13T17:59:46.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.445+0000 7f38fbfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f38fc06c5b0 0x7f38fc163d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:46.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.445+0000 7f38fbfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f38fc06c5b0 0x7f38fc163d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54358/0 (socket says 172.21.15.40:54358) 2024-09-13T17:59:46.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.445+0000 7f38fb7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f38fc153720 0x7f38fc0ca510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59842/0 (socket says 172.21.15.40:59842) 2024-09-13T17:59:46.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.445+0000 7f38fbfff700 1 -- 172.21.15.40:0/4127992603 learned_addr learned my addr 172.21.15.40:0/4127992603 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:46.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.445+0000 7f38faffd700 1 --2- 172.21.15.40:0/4127992603 >> v2:172.21.15.43:3300/0 conn(0x7f38fc0cf9b0 0x7f38fc06c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:46.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.445+0000 7f38fbfff700 1 -- 172.21.15.40:0/4127992603 >> v2:172.21.15.43:3300/0 conn(0x7f38fc0cf9b0 msgr2=0x7f38fc06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:59:46.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.445+0000 7f38fbfff700 1 --2- 172.21.15.40:0/4127992603 >> v2:172.21.15.43:3300/0 conn(0x7f38fc0cf9b0 0x7f38fc06c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.445+0000 7f38fbfff700 1 -- 172.21.15.40:0/4127992603 >> v1:172.21.15.40:6789/0 conn(0x7f38fc153720 legacy=0x7f38fc0ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:46.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.445+0000 7f38fbfff700 1 -- 172.21.15.40:0/4127992603 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc0cac20 con 0x7f38fc06c5b0 2024-09-13T17:59:46.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.445+0000 7f38fbfff700 1 --2- 172.21.15.40:0/4127992603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc06c5b0 0x7f38fc163d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0b1b20 tx=0x7f38f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d1518fd9415840 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f38fa7fc700 1 -- 172.21.15.40:0/4127992603 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f400e070 con 0x7f38fc06c5b0 2024-09-13T17:59:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f38fa7fc700 1 -- 172.21.15.40:0/4127992603 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f400b9c0 con 0x7f38fc06c5b0 2024-09-13T17:59:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f38fa7fc700 1 -- 172.21.15.40:0/4127992603 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f400bd70 con 0x7f38fc06c5b0 2024-09-13T17:59:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f3900995700 1 -- 172.21.15.40:0/4127992603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc06c5b0 msgr2=0x7f38fc163d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f3900995700 1 --2- 172.21.15.40:0/4127992603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc06c5b0 0x7f38fc163d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0b1b20 tx=0x7f38f400b330 comp rx=0 tx=0).stop 2024-09-13T17:59:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f3900995700 1 -- 172.21.15.40:0/4127992603 shutdown_connections 2024-09-13T17:59:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f3900995700 1 --2- 172.21.15.40:0/4127992603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc06c5b0 0x7f38fc163d50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f3900995700 1 --2- 172.21.15.40:0/4127992603 >> v2:172.21.15.43:3300/0 conn(0x7f38fc0cf9b0 0x7f38fc06c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f3900995700 1 -- 172.21.15.40:0/4127992603 >> 172.21.15.40:0/4127992603 conn(0x7f38fc0666c0 msgr2=0x7f38fc0cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:46.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f3900995700 1 -- 172.21.15.40:0/4127992603 shutdown_connections 2024-09-13T17:59:46.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f3900995700 1 -- 172.21.15.40:0/4127992603 wait complete. 2024-09-13T17:59:46.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f3900995700 1 Processor -- start 2024-09-13T17:59:46.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f3900995700 1 -- start start 2024-09-13T17:59:46.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f3900995700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc06c5b0 0x7f38fc15c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:46.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f3900995700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38fc0cf9b0 0x7f38fc1588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:46.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f3900995700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc153720 0x7f38fc1f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:46.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f3900995700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc154940 con 0x7f38fc153720 2024-09-13T17:59:46.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f3900995700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc15c8b0 con 0x7f38fc06c5b0 2024-09-13T17:59:46.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f3900995700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc155b50 con 0x7f38fc0cf9b0 2024-09-13T17:59:46.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f38faffd700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38fc0cf9b0 msgr2=0x7f38fc1588b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T17:59:46.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f38faffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38fc0cf9b0 0x7f38fc1588b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:59:46.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.449+0000 7f38fbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc153720 0x7f38fc1f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:46.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f38fbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc153720 0x7f38fc1f2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54372/0 (socket says 172.21.15.40:54372) 2024-09-13T17:59:46.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f38fbfff700 1 -- 172.21.15.40:0/639391338 learned_addr learned my addr 172.21.15.40:0/639391338 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T17:59:46.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f38fb7fe700 1 --2- 172.21.15.40:0/639391338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc06c5b0 0x7f38fc15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:46.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f38fbfff700 1 -- 172.21.15.40:0/639391338 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38fc0cf9b0 msgr2=0x7f38fc1588b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:59:46.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f38fbfff700 1 --2- 172.21.15.40:0/639391338 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38fc0cf9b0 0x7f38fc1588b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f38fbfff700 1 -- 172.21.15.40:0/639391338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc06c5b0 msgr2=0x7f38fc15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:46.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f38fbfff700 1 --2- 172.21.15.40:0/639391338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc06c5b0 0x7f38fc15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f38fbfff700 1 -- 172.21.15.40:0/639391338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f4009b30 con 0x7f38fc153720 2024-09-13T17:59:46.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f38fbfff700 1 --2- 172.21.15.40:0/639391338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc153720 0x7f38fc1f2710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f38fc06bb00 tx=0x7f38f4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:46.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f38f3fff700 1 -- 172.21.15.40:0/639391338 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4004930 con 0x7f38fc153720 2024-09-13T17:59:46.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f38f3fff700 1 -- 172.21.15.40:0/639391338 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f40099b0 con 0x7f38fc153720 2024-09-13T17:59:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f3900995700 1 -- 172.21.15.40:0/639391338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38fc153e70 con 0x7f38fc153720 2024-09-13T17:59:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f38f3fff700 1 -- 172.21.15.40:0/639391338 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f401c680 con 0x7f38fc153720 2024-09-13T17:59:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f3900995700 1 -- 172.21.15.40:0/639391338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38fc155df0 con 0x7f38fc153720 2024-09-13T17:59:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f3900995700 1 --2- 172.21.15.40:0/639391338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc154330 0x7f38fc14ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f3900995700 1 -- 172.21.15.40:0/639391338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc074750 con 0x7f38fc154330 2024-09-13T17:59:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f38fb7fe700 1 --2- 172.21.15.40:0/639391338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc154330 0x7f38fc14ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f38fb7fe700 1 -- 172.21.15.40:0/639391338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f38fc15d0c0 con 0x7f38fc154330 2024-09-13T17:59:46.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f38f3fff700 1 -- 172.21.15.40:0/639391338 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f38f401c7e0 con 0x7f38fc153720 2024-09-13T17:59:46.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f38fb7fe700 1 --2- 172.21.15.40:0/639391338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc154330 0x7f38fc14ca90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f38ec0014e0 tx=0x7f38ec0018a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:46.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f38f3fff700 1 --2- 172.21.15.40:0/639391338 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f38dc0666e0 0x7f38dc068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:46.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f38f3fff700 1 -- 172.21.15.40:0/639391338 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f38f4086110 con 0x7f38fc153720 2024-09-13T17:59:46.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.453+0000 7f38faffd700 1 --2- 172.21.15.40:0/639391338 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f38dc0666e0 0x7f38dc068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:46.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.457+0000 7f38f3fff700 1 -- 172.21.15.40:0/639391338 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec017070 con 0x7f38fc154330 2024-09-13T17:59:46.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.457+0000 7f38f3fff700 1 -- 172.21.15.40:0/639391338 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f38ec012560 con 0x7f38fc154330 2024-09-13T17:59:46.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.457+0000 7f38f3fff700 1 -- 172.21.15.40:0/639391338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc154330 msgr2=0x7f38fc14ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:46.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.457+0000 7f38f3fff700 1 --2- 172.21.15.40:0/639391338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc154330 0x7f38fc14ca90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f38ec0014e0 tx=0x7f38ec0018a0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.457+0000 7f38faffd700 1 --2- 172.21.15.40:0/639391338 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f38dc0666e0 0x7f38dc068ba0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0c46e0 tx=0x7f38e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:46.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.457+0000 7f38f3fff700 1 -- 172.21.15.40:0/639391338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc154330 msgr2=0x7f38fc14ca90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:59:46.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.457+0000 7f38f3fff700 1 --2- 172.21.15.40:0/639391338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc154330 0x7f38fc14ca90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.465+0000 7f3900995700 1 --2- 172.21.15.40:0/639391338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc155470 0x7f38fc14cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:59:46.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.465+0000 7f3900995700 1 -- 172.21.15.40:0/639391338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0c4d60 con 0x7f38fc155470 2024-09-13T17:59:46.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.465+0000 7f38fb7fe700 1 --2- 172.21.15.40:0/639391338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc155470 0x7f38fc14cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:59:46.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.465+0000 7f38fb7fe700 1 -- 172.21.15.40:0/639391338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f38fc153b00 con 0x7f38fc155470 2024-09-13T17:59:46.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.465+0000 7f38fb7fe700 1 --2- 172.21.15.40:0/639391338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc155470 0x7f38fc14cfd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f38ec015040 tx=0x7f38ec001080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:59:46.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.465+0000 7f38f3fff700 1 -- 172.21.15.40:0/639391338 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec023070 con 0x7f38fc155470 2024-09-13T17:59:46.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.465+0000 7f38f3fff700 1 -- 172.21.15.40:0/639391338 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4936 (secure 0 0 0) 0x7f38ec012e20 con 0x7f38fc155470 2024-09-13T17:59:46.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.465+0000 7f38f3fff700 1 -- 172.21.15.40:0/639391338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc155470 msgr2=0x7f38fc14cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:46.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.465+0000 7f38f3fff700 1 --2- 172.21.15.40:0/639391338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc155470 0x7f38fc14cfd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f38ec015040 tx=0x7f38ec001080 comp rx=0 tx=0).stop 2024-09-13T17:59:46.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.465+0000 7f38f3fff700 1 -- 172.21.15.40:0/639391338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc155470 msgr2=0x7f38fc14cfd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:59:46.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.465+0000 7f38f3fff700 1 --2- 172.21.15.40:0/639391338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc155470 0x7f38fc14cfd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.479 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T17:59:46.479 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:59:46.479 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:59:46.479 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T17:59:46.479 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 74, 2024-09-13T17:59:46.479 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T17:59:46.480 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T17:59:46.480 INFO:teuthology.orchestra.run.smithi040.stdout: 1 2024-09-13T17:59:46.480 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:46.480 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T17:59:46.480 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:59:46.480 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T17:59:46.481 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T17:59:46.481 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:46.481 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:46.481 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:46.481 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:46.481 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:46.481 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:46.481 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:46.481 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:46.481 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:46.481 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:46.481 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:46.481 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T17:59:46.482 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T17:59:46.482 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:46.482 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:46.482 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:46.482 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:46.482 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:46.482 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:46.482 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:46.482 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:46.482 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:46.482 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:46.482 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:46.482 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.482 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T17:59:46.482 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T17:59:46.483 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T17:59:46.483 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T17:59:46.483 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T17:59:46.483 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T17:59:46.483 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:59:46.483 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T17:59:46.483 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T17:59:46.483 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T17:59:46.483 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T17:59:46.483 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T17:59:46.483 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T17:59:46.483 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T17:59:46.483 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T17:59:46.483 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T17:59:46.484 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T17:59:46.484 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T17:59:46.484 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T17:59:46.484 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T17:59:46.484 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T17:59:46.484 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T17:59:46.484 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T17:59:46.484 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T17:59:46.484 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T17:59:46.484 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T17:59:46.484 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T17:59:46.485 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:46.485 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T17:59:46.485 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.485 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T17:59:46.485 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.485 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T17:59:46.485 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T17:59:46.485 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:46.485 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:46.485 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.485 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:46.485 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T17:59:46.486 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:46.486 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.486 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.486 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:46.486 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T17:59:46.486 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:46.486 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:46.486 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:46.486 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.486 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T17:59:46.486 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T17:59:46.486 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:46.487 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:46.487 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:46.487 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.487 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.487 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T17:59:46.487 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T17:59:46.487 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:46.487 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:46.487 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.487 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:46.487 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T17:59:46.487 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:46.488 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.488 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.488 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:46.488 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T17:59:46.488 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:46.488 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:46.489 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:46.489 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.489 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T17:59:46.489 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T17:59:46.489 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:46.489 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:46.490 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:46.490 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.490 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.490 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T17:59:46.490 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T17:59:46.490 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T17:59:46.491 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T17:59:46.491 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.491 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T17:59:46.491 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T17:59:46.491 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:46.492 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.492 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.492 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T17:59:46.492 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T17:59:46.492 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T17:59:46.492 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:46.493 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:46.493 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.493 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T17:59:46.493 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T17:59:46.493 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T17:59:46.494 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T17:59:46.494 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T17:59:46.494 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:46.494 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:46.494 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.494 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T17:59:46.495 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T17:59:46.495 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.495 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:46.495 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:46.495 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T17:59:46.495 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:46.496 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:46.496 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T17:59:46.496 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.496 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:46.496 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:46.497 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:59:46.497 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:46.497 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:46.497 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T17:59:46.497 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.497 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:46.498 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:46.498 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:59:46.498 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:46.498 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:46.498 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T17:59:46.498 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.499 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:46.499 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:46.499 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:59:46.499 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:46.499 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T17:59:46.499 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T17:59:46.500 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T17:59:46.500 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T17:59:46.500 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T17:59:46.500 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T17:59:46.500 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T17:59:46.501 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T17:59:46.501 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T17:59:46.501 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T17:59:46.501 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T17:59:46.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.469+0000 7f3900995700 1 -- 172.21.15.40:0/639391338 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f38dc0666e0 msgr2=0x7f38dc068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:46.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.469+0000 7f3900995700 1 --2- 172.21.15.40:0/639391338 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f38dc0666e0 0x7f38dc068ba0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0c46e0 tx=0x7f38e4008040 comp rx=0 tx=0).stop 2024-09-13T17:59:46.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.469+0000 7f3900995700 1 -- 172.21.15.40:0/639391338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc153720 msgr2=0x7f38fc1f2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:59:46.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.469+0000 7f38fbfff700 1 -- 172.21.15.40:0/639391338 reap_dead start 2024-09-13T17:59:46.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.469+0000 7f3900995700 1 --2- 172.21.15.40:0/639391338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc153720 0x7f38fc1f2710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f38fc06bb00 tx=0x7f38f4004af0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.469+0000 7f3900995700 1 -- 172.21.15.40:0/639391338 shutdown_connections 2024-09-13T17:59:46.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.469+0000 7f3900995700 1 --2- 172.21.15.40:0/639391338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc153720 0x7f38fc1f2710 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:59:46.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.469+0000 7f3900995700 1 -- 172.21.15.40:0/639391338 >> 172.21.15.40:0/639391338 conn(0x7f38fc0666c0 msgr2=0x7f38fc0cba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:59:46.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.469+0000 7f3900995700 1 -- 172.21.15.40:0/639391338 shutdown_connections 2024-09-13T17:59:46.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T17:59:46.469+0000 7f3900995700 1 -- 172.21.15.40:0/639391338 wait complete. 2024-09-13T17:59:46.505 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T17:59:46.505 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:00:05.208 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:00:05.211 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:00:06.506 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-13T18:00:06.506 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-13T18:00:06.506 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T18:00:06.506 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T18:00:06.509 INFO:tasks.ceph.mon.c:Started 2024-09-13T18:00:06.509 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:00:06.509 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:00:06.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.621+0000 7fe3174a8700 1 Processor -- start 2024-09-13T18:00:06.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.621+0000 7fe3174a8700 1 -- start start 2024-09-13T18:00:06.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3174a8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe310153990 0x7fe31006f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:06.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3174a8700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe31006ee30 0x7fe310159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:06.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3174a8700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3100c8680 con 0x7fe3100c7880 2024-09-13T18:00:06.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3174a8700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3100c40b0 con 0x7fe310153990 2024-09-13T18:00:06.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3174a8700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3100c8960 con 0x7fe31006ee30 2024-09-13T18:00:06.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3164a6700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe310153990 0x7fe31006f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:06.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3164a6700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe310153990 0x7fe31006f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47246/0 (socket says 172.21.15.40:47246) 2024-09-13T18:00:06.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3164a6700 1 -- 172.21.15.40:0/3578144658 learned_addr learned my addr 172.21.15.40:0/3578144658 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:06.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe315ca5700 1 --2- 172.21.15.40:0/3578144658 >> v2:172.21.15.43:3300/0 conn(0x7fe31006ee30 0x7fe310159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:06.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3164a6700 1 -- 172.21.15.40:0/3578144658 >> v2:172.21.15.43:3300/0 conn(0x7fe31006ee30 msgr2=0x7fe310159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:00:06.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3164a6700 1 --2- 172.21.15.40:0/3578144658 >> v2:172.21.15.43:3300/0 conn(0x7fe31006ee30 0x7fe310159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:06.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3164a6700 1 -- 172.21.15.40:0/3578144658 >> v1:172.21.15.40:6789/0 conn(0x7fe3100c7880 legacy=0x7fe31006b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:06.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3164a6700 1 -- 172.21.15.40:0/3578144658 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3100c8c40 con 0x7fe310153990 2024-09-13T18:00:06.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3164a6700 1 --2- 172.21.15.40:0/3578144658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe310153990 0x7fe31006f5a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe310066ec0 tx=0x7fe30c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e18592728ab69a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:06.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3154a4700 1 -- 172.21.15.40:0/3578144658 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c011830 con 0x7fe310153990 2024-09-13T18:00:06.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3154a4700 1 -- 172.21.15.40:0/3578144658 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30c011990 con 0x7fe310153990 2024-09-13T18:00:06.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3154a4700 1 -- 172.21.15.40:0/3578144658 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c011d20 con 0x7fe310153990 2024-09-13T18:00:06.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3174a8700 1 -- 172.21.15.40:0/3578144658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe310153990 msgr2=0x7fe31006f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:06.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3174a8700 1 --2- 172.21.15.40:0/3578144658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe310153990 0x7fe31006f5a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe310066ec0 tx=0x7fe30c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:00:06.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3174a8700 1 -- 172.21.15.40:0/3578144658 shutdown_connections 2024-09-13T18:00:06.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3174a8700 1 --2- 172.21.15.40:0/3578144658 >> v2:172.21.15.43:3300/0 conn(0x7fe31006ee30 0x7fe310159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:06.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3174a8700 1 --2- 172.21.15.40:0/3578144658 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe310153990 0x7fe31006f5a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:06.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3174a8700 1 -- 172.21.15.40:0/3578144658 >> 172.21.15.40:0/3578144658 conn(0x7fe310066090 msgr2=0x7fe310066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:06.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3174a8700 1 -- 172.21.15.40:0/3578144658 shutdown_connections 2024-09-13T18:00:06.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.625+0000 7fe3174a8700 1 -- 172.21.15.40:0/3578144658 wait complete. 2024-09-13T18:00:06.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe3174a8700 1 Processor -- start 2024-09-13T18:00:06.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe3174a8700 1 -- start start 2024-09-13T18:00:06.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe3174a8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe31006ee30 0x7fe310152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:06.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe3174a8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3100c7880 0x7fe31014e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:06.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe3174a8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe310153990 0x7fe3101eef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:06.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe3174a8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31014d7f0 con 0x7fe3100c7880 2024-09-13T18:00:06.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe3174a8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31014ee80 con 0x7fe310153990 2024-09-13T18:00:06.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe3174a8700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3101473a0 con 0x7fe31006ee30 2024-09-13T18:00:06.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe315ca5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3100c7880 0x7fe31014e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:06.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe315ca5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3100c7880 0x7fe31014e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47254/0 (socket says 172.21.15.40:47254) 2024-09-13T18:00:06.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe315ca5700 1 -- 172.21.15.40:0/3098410578 learned_addr learned my addr 172.21.15.40:0/3098410578 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:06.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe316ca7700 1 --2- 172.21.15.40:0/3098410578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe310153990 0x7fe3101eef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:06.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe3164a6700 1 --2- 172.21.15.40:0/3098410578 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe31006ee30 0x7fe310152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:06.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe315ca5700 1 -- 172.21.15.40:0/3098410578 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe31006ee30 msgr2=0x7fe310152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:06.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe315ca5700 1 --2- 172.21.15.40:0/3098410578 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe31006ee30 0x7fe310152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:06.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe315ca5700 1 -- 172.21.15.40:0/3098410578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe310153990 msgr2=0x7fe3101eef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:06.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe315ca5700 1 --2- 172.21.15.40:0/3098410578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe310153990 0x7fe3101eef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:06.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe315ca5700 1 -- 172.21.15.40:0/3098410578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c009570 con 0x7fe3100c7880 2024-09-13T18:00:06.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe316ca7700 1 --2- 172.21.15.40:0/3098410578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe310153990 0x7fe3101eef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:00:06.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe315ca5700 1 --2- 172.21.15.40:0/3098410578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3100c7880 0x7fe31014e940 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe30000e9d0 tx=0x7fe30000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:06.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe306ffd700 1 -- 172.21.15.40:0/3098410578 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe300009720 con 0x7fe3100c7880 2024-09-13T18:00:06.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe306ffd700 1 -- 172.21.15.40:0/3098410578 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30000c850 con 0x7fe3100c7880 2024-09-13T18:00:06.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe306ffd700 1 -- 172.21.15.40:0/3098410578 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30000cbc0 con 0x7fe3100c7880 2024-09-13T18:00:06.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe3174a8700 1 -- 172.21.15.40:0/3098410578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31014cd50 con 0x7fe3100c7880 2024-09-13T18:00:06.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.629+0000 7fe3174a8700 1 -- 172.21.15.40:0/3098410578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31014f0a0 con 0x7fe3100c7880 2024-09-13T18:00:06.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.633+0000 7fe3174a8700 1 -- 172.21.15.40:0/3098410578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f8005310 con 0x7fe3100c7880 2024-09-13T18:00:06.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.633+0000 7fe306ffd700 1 -- 172.21.15.40:0/3098410578 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe30000cd20 con 0x7fe3100c7880 2024-09-13T18:00:06.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.633+0000 7fe306ffd700 1 --2- 172.21.15.40:0/3098410578 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe2f4066730 0x7fe2f4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:06.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.633+0000 7fe306ffd700 1 -- 172.21.15.40:0/3098410578 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe300084930 con 0x7fe3100c7880 2024-09-13T18:00:06.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.633+0000 7fe3164a6700 1 --2- 172.21.15.40:0/3098410578 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe2f4066730 0x7fe2f4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:06.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.637+0000 7fe3164a6700 1 --2- 172.21.15.40:0/3098410578 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe2f4066730 0x7fe2f4068bf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe30c000c00 tx=0x7fe30c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:06.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.641+0000 7fe306ffd700 1 -- 172.21.15.40:0/3098410578 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe300051970 con 0x7fe3100c7880 2024-09-13T18:00:06.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.909+0000 7fe3174a8700 1 -- 172.21.15.40:0/3098410578 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe2f80055c0 con 0x7fe3100c7880 2024-09-13T18:00:06.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.909+0000 7fe306ffd700 1 -- 172.21.15.40:0/3098410578 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7fe300017610 con 0x7fe3100c7880 2024-09-13T18:00:06.916 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":74,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:00:06.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.909+0000 7fe3174a8700 1 -- 172.21.15.40:0/3098410578 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe2f4066730 msgr2=0x7fe2f4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:06.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.909+0000 7fe3174a8700 1 --2- 172.21.15.40:0/3098410578 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe2f4066730 0x7fe2f4068bf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe30c000c00 tx=0x7fe30c008030 comp rx=0 tx=0).stop 2024-09-13T18:00:06.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.909+0000 7fe3174a8700 1 -- 172.21.15.40:0/3098410578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3100c7880 msgr2=0x7fe31014e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:06.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.909+0000 7fe3174a8700 1 --2- 172.21.15.40:0/3098410578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3100c7880 0x7fe31014e940 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe30000e9d0 tx=0x7fe30000ed90 comp rx=0 tx=0).stop 2024-09-13T18:00:06.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.913+0000 7fe3174a8700 1 -- 172.21.15.40:0/3098410578 shutdown_connections 2024-09-13T18:00:06.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.913+0000 7fe3174a8700 1 --2- 172.21.15.40:0/3098410578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe310153990 0x7fe3101eef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:06.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.913+0000 7fe3174a8700 1 --2- 172.21.15.40:0/3098410578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3100c7880 0x7fe31014e940 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:06.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.913+0000 7fe3174a8700 1 --2- 172.21.15.40:0/3098410578 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe2f4066730 0x7fe2f4068bf0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:06.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.913+0000 7fe3174a8700 1 --2- 172.21.15.40:0/3098410578 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe31006ee30 0x7fe310152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:06.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.913+0000 7fe3174a8700 1 -- 172.21.15.40:0/3098410578 >> 172.21.15.40:0/3098410578 conn(0x7fe310066090 msgr2=0x7fe3100c26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:06.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.913+0000 7fe3174a8700 1 -- 172.21.15.40:0/3098410578 shutdown_connections 2024-09-13T18:00:06.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:06.913+0000 7fe3174a8700 1 -- 172.21.15.40:0/3098410578 wait complete. 2024-09-13T18:00:09.938 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:00:10.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.053+0000 7f35fdffe700 1 Processor -- start 2024-09-13T18:00:10.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.053+0000 7f35fdffe700 1 -- start start 2024-09-13T18:00:10.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.053+0000 7f35fdffe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f004ca90 0x7f35f0007560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.053+0000 7f35fdffe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f35f0007ba0 0x7f35f00dc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.053+0000 7f35fdffe700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f0005fa0 con 0x7f35f00d69c0 2024-09-13T18:00:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35fdffe700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f00368b0 con 0x7f35f0007ba0 2024-09-13T18:00:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35fdffe700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f00dc510 con 0x7f35f004ca90 2024-09-13T18:00:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35fd7fd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f35f0007ba0 0x7f35f00dc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35f7fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f004ca90 0x7f35f0007560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35fd7fd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f35f0007ba0 0x7f35f00dc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:36064/0 (socket says 172.21.15.40:36064) 2024-09-13T18:00:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35fd7fd700 1 -- 172.21.15.40:0/773044325 learned_addr learned my addr 172.21.15.40:0/773044325 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35f7fff700 1 -- 172.21.15.40:0/773044325 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f004ca90 msgr2=0x7f35f0007560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:00:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35f7fff700 1 -- 172.21.15.40:0/773044325 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f004ca90 msgr2=0x7f35f0007560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:00:10.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35f7fff700 1 --2- 172.21.15.40:0/773044325 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f004ca90 0x7f35f0007560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:10.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35f7fff700 1 --2- 172.21.15.40:0/773044325 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f004ca90 0x7f35f0007560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:10.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35fd7fd700 1 -- 172.21.15.40:0/773044325 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f004ca90 msgr2=0x7f35f0007560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:00:10.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35fd7fd700 1 --2- 172.21.15.40:0/773044325 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f004ca90 0x7f35f0007560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35fd7fd700 1 -- 172.21.15.40:0/773044325 >> v1:172.21.15.43:6789/0 conn(0x7f35f00d69c0 legacy=0x7f35f0033e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:10.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35fd7fd700 1 -- 172.21.15.40:0/773044325 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f0036080 con 0x7f35f0007ba0 2024-09-13T18:00:10.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35fd7fd700 1 --2- 172.21.15.40:0/773044325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f0007ba0 0x7f35f00dc960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f35f8063990 tx=0x7f35f8069e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e5db3d9da204478 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:10.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35f77fe700 1 -- 172.21.15.40:0/773044325 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8068a70 con 0x7f35f0007ba0 2024-09-13T18:00:10.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35f77fe700 1 -- 172.21.15.40:0/773044325 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f8067dd0 con 0x7f35f0007ba0 2024-09-13T18:00:10.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35f77fe700 1 -- 172.21.15.40:0/773044325 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f80744e0 con 0x7f35f0007ba0 2024-09-13T18:00:10.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35fdffe700 1 -- 172.21.15.40:0/773044325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f0007ba0 msgr2=0x7f35f00dc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:10.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35fdffe700 1 --2- 172.21.15.40:0/773044325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f0007ba0 0x7f35f00dc960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f35f8063990 tx=0x7f35f8069e70 comp rx=0 tx=0).stop 2024-09-13T18:00:10.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35fdffe700 1 -- 172.21.15.40:0/773044325 shutdown_connections 2024-09-13T18:00:10.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35fdffe700 1 --2- 172.21.15.40:0/773044325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f0007ba0 0x7f35f00dc960 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35fdffe700 1 --2- 172.21.15.40:0/773044325 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f004ca90 0x7f35f0007560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35fdffe700 1 -- 172.21.15.40:0/773044325 >> 172.21.15.40:0/773044325 conn(0x7f35f00505e0 msgr2=0x7f35f00509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:10.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35fdffe700 1 -- 172.21.15.40:0/773044325 shutdown_connections 2024-09-13T18:00:10.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.057+0000 7f35fdffe700 1 -- 172.21.15.40:0/773044325 wait complete. 2024-09-13T18:00:10.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35fdffe700 1 Processor -- start 2024-09-13T18:00:10.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35fdffe700 1 -- start start 2024-09-13T18:00:10.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35fdffe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f0007ba0 0x7f35f00d2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35fdffe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f004ca90 0x7f35f0177880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35fdffe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f00d69c0 0x7f35f00cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35fdffe700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f00480f0 con 0x7f35f00d69c0 2024-09-13T18:00:10.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35fdffe700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f0031990 con 0x7f35f004ca90 2024-09-13T18:00:10.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35fdffe700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f00309b0 con 0x7f35f0007ba0 2024-09-13T18:00:10.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35fd7fd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f00d69c0 0x7f35f00cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35fd7fd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f00d69c0 0x7f35f00cfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47270/0 (socket says 172.21.15.40:47270) 2024-09-13T18:00:10.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35fd7fd700 1 -- 172.21.15.40:0/2090424420 learned_addr learned my addr 172.21.15.40:0/2090424420 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:10.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35f7fff700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f004ca90 0x7f35f0177880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35fcffc700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f0007ba0 0x7f35f00d2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35fd7fd700 1 -- 172.21.15.40:0/2090424420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f00d69c0 msgr2=0x7f35f00cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-13T18:00:10.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35fd7fd700 1 -- 172.21.15.40:0/2090424420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f00d69c0 msgr2=0x7f35f00cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:00:10.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35fd7fd700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f00d69c0 0x7f35f00cfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:10.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35fd7fd700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f00d69c0 0x7f35f00cfee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:10.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35f7fff700 1 -- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f004ca90 msgr2=0x7f35f0177880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:00:10.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35f7fff700 1 -- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f004ca90 msgr2=0x7f35f0177880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:00:10.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35fcffc700 1 -- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f0007ba0 msgr2=0x7f35f00d2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:00:10.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35f7fff700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f004ca90 0x7f35f0177880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:10.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35fcffc700 1 -- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f0007ba0 msgr2=0x7f35f00d2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:00:10.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35fcffc700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f0007ba0 0x7f35f00d2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:10.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35f7fff700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f004ca90 0x7f35f0177880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:10.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.061+0000 7f35fcffc700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f0007ba0 0x7f35f00d2640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:10.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.265+0000 7f35fd7fd700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f00d69c0 0x7f35f00cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.265+0000 7f35f7fff700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f004ca90 0x7f35f0177880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.265+0000 7f35fcffc700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f0007ba0 0x7f35f00d2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.265+0000 7f35fcffc700 1 -- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f0007ba0 msgr2=0x7f35f00d2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:00:10.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.265+0000 7f35fcffc700 1 -- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f0007ba0 msgr2=0x7f35f00d2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:00:10.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.265+0000 7f35fcffc700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f0007ba0 0x7f35f00d2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:10.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.265+0000 7f35fd7fd700 1 -- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f0007ba0 msgr2=0x7f35f00d2640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:00:10.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.265+0000 7f35fcffc700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f0007ba0 0x7f35f00d2640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:00:10.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.265+0000 7f35fd7fd700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f0007ba0 0x7f35f00d2640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.265+0000 7f35fd7fd700 1 -- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f004ca90 msgr2=0x7f35f0177880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:10.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.265+0000 7f35fd7fd700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f004ca90 0x7f35f0177880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.265+0000 7f35fd7fd700 1 -- 172.21.15.40:0/2090424420 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f804f5c0 con 0x7f35f00d69c0 2024-09-13T18:00:10.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.265+0000 7f35fd7fd700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f00d69c0 0x7f35f00cfee0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f35f8063990 tx=0x7f35f8068470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:10.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.265+0000 7f35f57fa700 1 -- 172.21.15.40:0/2090424420 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f80677e0 con 0x7f35f00d69c0 2024-09-13T18:00:10.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.265+0000 7f35f57fa700 1 -- 172.21.15.40:0/2090424420 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f8067940 con 0x7f35f00d69c0 2024-09-13T18:00:10.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.265+0000 7f35fdffe700 1 -- 172.21.15.40:0/2090424420 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f00300b0 con 0x7f35f00d69c0 2024-09-13T18:00:10.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.265+0000 7f35f57fa700 1 -- 172.21.15.40:0/2090424420 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8072390 con 0x7f35f00d69c0 2024-09-13T18:00:10.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.265+0000 7f35fdffe700 1 -- 172.21.15.40:0/2090424420 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f0031620 con 0x7f35f00d69c0 2024-09-13T18:00:10.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.269+0000 7f35fdffe700 1 -- 172.21.15.40:0/2090424420 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f0007560 con 0x7f35f00d69c0 2024-09-13T18:00:10.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.269+0000 7f35f57fa700 1 -- 172.21.15.40:0/2090424420 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f35f80724f0 con 0x7f35f00d69c0 2024-09-13T18:00:10.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.269+0000 7f35f57fa700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f35e4066730 0x7f35e4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.269+0000 7f35f57fa700 1 -- 172.21.15.40:0/2090424420 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f35f8100510 con 0x7f35f00d69c0 2024-09-13T18:00:10.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.269+0000 7f35fcffc700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f35e4066730 0x7f35e4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.269+0000 7f35fcffc700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f35e4066730 0x7f35e4068bf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f35e8007fd0 tx=0x7f35e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:10.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.277+0000 7f35f57fa700 1 -- 172.21.15.40:0/2090424420 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f35f80cd5e0 con 0x7f35f00d69c0 2024-09-13T18:00:10.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.545+0000 7f35fdffe700 1 -- 172.21.15.40:0/2090424420 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f35f0006930 con 0x7f35f00d69c0 2024-09-13T18:00:10.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.549+0000 7f35f57fa700 1 -- 172.21.15.40:0/2090424420 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7f35f80a5620 con 0x7f35f00d69c0 2024-09-13T18:00:10.555 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":76,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:00:10.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.549+0000 7f35fdffe700 1 -- 172.21.15.40:0/2090424420 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f35e4066730 msgr2=0x7f35e4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:10.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.549+0000 7f35fdffe700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f35e4066730 0x7f35e4068bf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f35e8007fd0 tx=0x7f35e8009040 comp rx=0 tx=0).stop 2024-09-13T18:00:10.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.549+0000 7f35fdffe700 1 -- 172.21.15.40:0/2090424420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f00d69c0 msgr2=0x7f35f00cfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:10.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.549+0000 7f35fdffe700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f00d69c0 0x7f35f00cfee0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f35f8063990 tx=0x7f35f8068470 comp rx=0 tx=0).stop 2024-09-13T18:00:10.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.549+0000 7f35fdffe700 1 -- 172.21.15.40:0/2090424420 shutdown_connections 2024-09-13T18:00:10.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.549+0000 7f35fdffe700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f00d69c0 0x7f35f00cfee0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.549+0000 7f35fdffe700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f004ca90 0x7f35f0177880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.549+0000 7f35fdffe700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f35e4066730 0x7f35e4068bf0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.549+0000 7f35fdffe700 1 --2- 172.21.15.40:0/2090424420 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f0007ba0 0x7f35f00d2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.549+0000 7f35fdffe700 1 -- 172.21.15.40:0/2090424420 >> 172.21.15.40:0/2090424420 conn(0x7f35f00505e0 msgr2=0x7f35f00da800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:10.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.549+0000 7f35fdffe700 1 -- 172.21.15.40:0/2090424420 shutdown_connections 2024-09-13T18:00:10.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.549+0000 7f35fdffe700 1 -- 172.21.15.40:0/2090424420 wait complete. 2024-09-13T18:00:10.575 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:00:10.575 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:00:10.575 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:00:10.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090b49e700 1 Processor -- start 2024-09-13T18:00:10.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090b49e700 1 -- start start 2024-09-13T18:00:10.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090b49e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f09041503c0 0x7f09040c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090b49e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f09040c8500 0x7f09040c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090b49e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f09040c71c0 0x7f0904159630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090b49e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09040c8f80 con 0x7f09041503c0 2024-09-13T18:00:10.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090b49e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09040c2a80 con 0x7f09040c8500 2024-09-13T18:00:10.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090b49e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09041507a0 con 0x7f09040c71c0 2024-09-13T18:00:10.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090a49c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f09041503c0 0x7f09040c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090a49c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f09041503c0 0x7f09040c79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47294/0 (socket says 172.21.15.40:47294) 2024-09-13T18:00:10.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090a49c700 1 -- 172.21.15.40:0/1011277043 learned_addr learned my addr 172.21.15.40:0/1011277043 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:10.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f0909c9b700 1 --2- 172.21.15.40:0/1011277043 >> v2:172.21.15.43:3300/0 conn(0x7f09040c8500 0x7f09040c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090ac9d700 1 --2- 172.21.15.40:0/1011277043 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f09040c71c0 0x7f0904159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090ac9d700 1 -- 172.21.15.40:0/1011277043 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f09040c71c0 msgr2=0x7f0904159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:00:10.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090ac9d700 1 -- 172.21.15.40:0/1011277043 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f09040c71c0 msgr2=0x7f0904159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:00:10.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090ac9d700 1 --2- 172.21.15.40:0/1011277043 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f09040c71c0 0x7f0904159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:10.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090a49c700 1 -- 172.21.15.40:0/1011277043 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f09040c71c0 msgr2=0x7f0904159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:00:10.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090ac9d700 1 --2- 172.21.15.40:0/1011277043 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f09040c71c0 0x7f0904159630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:10.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090a49c700 1 --2- 172.21.15.40:0/1011277043 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f09040c71c0 0x7f0904159630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090a49c700 1 -- 172.21.15.40:0/1011277043 >> v2:172.21.15.43:3300/0 conn(0x7f09040c8500 msgr2=0x7f09040c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:00:10.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090a49c700 1 --2- 172.21.15.40:0/1011277043 >> v2:172.21.15.43:3300/0 conn(0x7f09040c8500 0x7f09040c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090a49c700 1 -- 172.21.15.40:0/1011277043 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09040c8a40 con 0x7f09041503c0 2024-09-13T18:00:10.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090a49c700 1 --2- 172.21.15.40:0/1011277043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09041503c0 0x7f09040c79c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0904072b70 tx=0x7f0900009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41dffcdaa92430d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:10.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090949a700 1 -- 172.21.15.40:0/1011277043 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0900011830 con 0x7f09041503c0 2024-09-13T18:00:10.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090949a700 1 -- 172.21.15.40:0/1011277043 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0900011990 con 0x7f09041503c0 2024-09-13T18:00:10.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.685+0000 7f090949a700 1 -- 172.21.15.40:0/1011277043 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0900011d20 con 0x7f09041503c0 2024-09-13T18:00:10.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090b49e700 1 -- 172.21.15.40:0/1011277043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09041503c0 msgr2=0x7f09040c79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:10.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090b49e700 1 --2- 172.21.15.40:0/1011277043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09041503c0 0x7f09040c79c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0904072b70 tx=0x7f0900009bf0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090b49e700 1 -- 172.21.15.40:0/1011277043 shutdown_connections 2024-09-13T18:00:10.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090b49e700 1 --2- 172.21.15.40:0/1011277043 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f09040c71c0 0x7f0904159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090b49e700 1 --2- 172.21.15.40:0/1011277043 >> v2:172.21.15.43:3300/0 conn(0x7f09040c8500 0x7f09040c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090b49e700 1 --2- 172.21.15.40:0/1011277043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09041503c0 0x7f09040c79c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090b49e700 1 -- 172.21.15.40:0/1011277043 >> 172.21.15.40:0/1011277043 conn(0x7f090406e6d0 msgr2=0x7f090406eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:10.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090b49e700 1 -- 172.21.15.40:0/1011277043 shutdown_connections 2024-09-13T18:00:10.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090b49e700 1 -- 172.21.15.40:0/1011277043 wait complete. 2024-09-13T18:00:10.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090b49e700 1 Processor -- start 2024-09-13T18:00:10.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090b49e700 1 -- start start 2024-09-13T18:00:10.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090b49e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09040c71c0 0x7f09040ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090b49e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f09040c8500 0x7f09040cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090b49e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09041503c0 0x7f090414d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090b49e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09040c3f70 con 0x7f09041503c0 2024-09-13T18:00:10.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090b49e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09040cb210 con 0x7f09040c71c0 2024-09-13T18:00:10.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090b49e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09041f1390 con 0x7f09040c8500 2024-09-13T18:00:10.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090ac9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09041503c0 0x7f090414d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090ac9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09041503c0 0x7f090414d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47302/0 (socket says 172.21.15.40:47302) 2024-09-13T18:00:10.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090ac9d700 1 -- 172.21.15.40:0/271503911 learned_addr learned my addr 172.21.15.40:0/271503911 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:10.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f0909c9b700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f09040c8500 0x7f09040cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090a49c700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09040c71c0 0x7f09040ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f0909c9b700 1 -- 172.21.15.40:0/271503911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f09040c8500 msgr2=0x7f09040cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:00:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f0909c9b700 1 -- 172.21.15.40:0/271503911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f09040c8500 msgr2=0x7f09040cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:00:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f0909c9b700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f09040c8500 0x7f09040cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f0909c9b700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f09040c8500 0x7f09040cb760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090ac9d700 1 -- 172.21.15.40:0/271503911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f09040c8500 msgr2=0x7f09040cb760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:00:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090ac9d700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f09040c8500 0x7f09040cb760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090ac9d700 1 -- 172.21.15.40:0/271503911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09040c71c0 msgr2=0x7f09040ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090ac9d700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09040c71c0 0x7f09040ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090ac9d700 1 -- 172.21.15.40:0/271503911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0900009570 con 0x7f09041503c0 2024-09-13T18:00:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.689+0000 7f090ac9d700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09041503c0 0x7f090414d280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f08fc007d10 tx=0x7f08fc00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f08faffd700 1 -- 172.21.15.40:0/271503911 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc00ed80 con 0x7f09041503c0 2024-09-13T18:00:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f08faffd700 1 -- 172.21.15.40:0/271503911 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc007e60 con 0x7f09041503c0 2024-09-13T18:00:10.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f090b49e700 1 -- 172.21.15.40:0/271503911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09041f0de0 con 0x7f09041503c0 2024-09-13T18:00:10.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f08faffd700 1 -- 172.21.15.40:0/271503911 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc014660 con 0x7f09041503c0 2024-09-13T18:00:10.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f090b49e700 1 -- 172.21.15.40:0/271503911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09040cac50 con 0x7f09041503c0 2024-09-13T18:00:10.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f090b49e700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09040c3970 0x7f090414f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f090b49e700 1 -- 172.21.15.40:0/271503911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0904067600 con 0x7f09040c3970 2024-09-13T18:00:10.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f090a49c700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09040c3970 0x7f090414f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f090a49c700 1 -- 172.21.15.40:0/271503911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09040c2710 con 0x7f09040c3970 2024-09-13T18:00:10.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f090a49c700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09040c3970 0x7f090414f2e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0904072b70 tx=0x7f0900004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:10.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f08faffd700 1 -- 172.21.15.40:0/271503911 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f08fc0147c0 con 0x7f09041503c0 2024-09-13T18:00:10.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f08faffd700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f08e8066730 0x7f08e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f08faffd700 1 -- 172.21.15.40:0/271503911 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f08fc0848d0 con 0x7f09041503c0 2024-09-13T18:00:10.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f0909c9b700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f08e8066730 0x7f08e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f08faffd700 1 -- 172.21.15.40:0/271503911 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090001f070 con 0x7f09040c3970 2024-09-13T18:00:10.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f08faffd700 1 -- 172.21.15.40:0/271503911 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0900004040 con 0x7f09040c3970 2024-09-13T18:00:10.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f08faffd700 1 -- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09040c3970 msgr2=0x7f090414f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:10.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f08faffd700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09040c3970 0x7f090414f2e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0904072b70 tx=0x7f0900004770 comp rx=0 tx=0).stop 2024-09-13T18:00:10.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f08faffd700 1 -- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09040c3970 msgr2=0x7f090414f2e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:00:10.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f08faffd700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09040c3970 0x7f090414f2e0 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0904072b70 tx=0x7f0900004770 comp rx=0 tx=0).stop 2024-09-13T18:00:10.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.693+0000 7f0909c9b700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f08e8066730 0x7f08e8068bf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f09041eaac0 tx=0x7f08f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:10.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.701+0000 7f090b49e700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09040c99a0 0x7f09040c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.701+0000 7f090b49e700 1 -- 172.21.15.40:0/271503911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09040cc960 con 0x7f09040c99a0 2024-09-13T18:00:10.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.705+0000 7f090a49c700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09040c99a0 0x7f09040c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.705+0000 7f090a49c700 1 -- 172.21.15.40:0/271503911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f09040c8ce0 con 0x7f09040c99a0 2024-09-13T18:00:10.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.705+0000 7f090a49c700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09040c99a0 0x7f09040c5490 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0900009c30 tx=0x7f0900011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:10.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.705+0000 7f08faffd700 1 -- 172.21.15.40:0/271503911 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0900003ca0 con 0x7f09040c99a0 2024-09-13T18:00:10.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.705+0000 7f08faffd700 1 -- 172.21.15.40:0/271503911 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4783 (secure 0 0 0) 0x7f0900003e00 con 0x7f09040c99a0 2024-09-13T18:00:10.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.705+0000 7f08faffd700 1 -- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09040c99a0 msgr2=0x7f09040c5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:10.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.705+0000 7f08faffd700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09040c99a0 0x7f09040c5490 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0900009c30 tx=0x7f0900011000 comp rx=0 tx=0).stop 2024-09-13T18:00:10.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.705+0000 7f08faffd700 1 -- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09040c99a0 msgr2=0x7f09040c5490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:00:10.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.705+0000 7f08faffd700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09040c99a0 0x7f09040c5490 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.713 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:00:10.713 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:00:10.713 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:00:10.713 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:00:10.713 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 76, 2024-09-13T18:00:10.713 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:00:10.714 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:00:10.714 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:00:10.714 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:00:10.714 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:10.714 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:00:10.714 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:00:10.714 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:00:10.714 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:00:10.714 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:10.714 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:10.714 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:10.714 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:10.714 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:10.714 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:10.715 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:10.715 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:10.715 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:10.715 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:10.715 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:10.715 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:00:10.715 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:00:10.715 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:10.715 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:10.715 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:10.715 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:10.715 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:10.715 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:10.716 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:10.716 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:10.716 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:10.716 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:10.716 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:10.716 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.716 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:00:10.716 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:00:10.716 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:00:10.716 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:00:10.716 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:00:10.716 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:00:10.716 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:00:10.716 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:00:10.717 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:00:10.717 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:00:10.717 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:00:10.717 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:00:10.717 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:00:10.717 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:00:10.717 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:00:10.717 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:00:10.717 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:00:10.717 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:10.717 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:10.717 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:10.717 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:10.718 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:10.718 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:10.718 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:10.718 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:10.718 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:10.718 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:10.718 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:10.718 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:00:10.718 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.718 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:00:10.718 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.718 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:00:10.718 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:00:10.719 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:10.719 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:10.719 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.719 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:10.719 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:00:10.719 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:10.719 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.719 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.719 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:10.719 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:00:10.719 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:10.719 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:10.719 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:10.720 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.720 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:00:10.720 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:00:10.720 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:10.720 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:10.720 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:10.720 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.720 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.720 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:00:10.720 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:00:10.720 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:10.720 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:10.720 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.720 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:10.721 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:00:10.721 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:10.721 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.721 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.721 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:10.721 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:00:10.721 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:10.721 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:10.721 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:10.721 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.721 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:00:10.721 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:00:10.721 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:10.722 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:10.722 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:10.722 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.722 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.722 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:00:10.722 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:00:10.722 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:10.722 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:10.722 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.722 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:10.722 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:00:10.722 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:10.722 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.722 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.723 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:10.723 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:00:10.723 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:10.723 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:10.723 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:10.723 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.723 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:00:10.723 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:00:10.723 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:10.723 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:10.723 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:10.723 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:10.723 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:10.724 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.724 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:00:10.724 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:00:10.724 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.724 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:10.724 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:10.724 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:00:10.724 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:10.724 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:10.724 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:00:10.724 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.724 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:10.724 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:10.724 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:00:10.725 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:10.725 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:10.725 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:00:10.725 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.725 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:10.725 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:10.725 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:00:10.725 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:10.725 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:10.725 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:00:10.725 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.725 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:10.725 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:10.726 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:00:10.726 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:10.726 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:10.726 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.726 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:00:10.726 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:00:10.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.705+0000 7f090b49e700 1 -- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f08e8066730 msgr2=0x7f08e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:10.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.705+0000 7f090b49e700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f08e8066730 0x7f08e8068bf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f09041eaac0 tx=0x7f08f4008040 comp rx=0 tx=0).stop 2024-09-13T18:00:10.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.705+0000 7f090b49e700 1 -- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09041503c0 msgr2=0x7f090414d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:10.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.705+0000 7f090b49e700 1 --2- 172.21.15.40:0/271503911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f09041503c0 0x7f090414d280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f08fc007d10 tx=0x7f08fc00e3c0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.705+0000 7f090ac9d700 1 -- 172.21.15.40:0/271503911 reap_dead start 2024-09-13T18:00:10.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.705+0000 7f090ac9d700 1 -- 172.21.15.40:0/271503911 reap_dead start 2024-09-13T18:00:10.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.705+0000 7f090b49e700 1 -- 172.21.15.40:0/271503911 shutdown_connections 2024-09-13T18:00:10.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.705+0000 7f090b49e700 1 -- 172.21.15.40:0/271503911 >> 172.21.15.40:0/271503911 conn(0x7f090406e6d0 msgr2=0x7f09040b1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:10.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.705+0000 7f090b49e700 1 -- 172.21.15.40:0/271503911 shutdown_connections 2024-09-13T18:00:10.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.705+0000 7f090b49e700 1 -- 172.21.15.40:0/271503911 wait complete. 2024-09-13T18:00:10.728 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:00:10.728 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:00:10.729 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:00:10.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.837+0000 7f86f5d33700 1 Processor -- start 2024-09-13T18:00:10.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.837+0000 7f86f5d33700 1 -- start start 2024-09-13T18:00:10.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.837+0000 7f86f5d33700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86f0157d40 0x7f86f0066220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.837+0000 7f86f5d33700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86f00cb2a0 0x7f86f00cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.837+0000 7f86f5d33700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f01578b0 con 0x7f86f0151050 2024-09-13T18:00:10.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.837+0000 7f86f5d33700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f0157090 con 0x7f86f0157d40 2024-09-13T18:00:10.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.837+0000 7f86f5d33700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f00c6630 con 0x7f86f00cb2a0 2024-09-13T18:00:10.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86e7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86f0157d40 0x7f86f0066220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86e7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86f0157d40 0x7f86f0066220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47326/0 (socket says 172.21.15.40:47326) 2024-09-13T18:00:10.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86e7fff700 1 -- 172.21.15.40:0/1569564275 learned_addr learned my addr 172.21.15.40:0/1569564275 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:10.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86f5532700 1 --2- 172.21.15.40:0/1569564275 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86f00cb2a0 0x7f86f00cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86f5532700 1 -- 172.21.15.40:0/1569564275 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86f00cb2a0 msgr2=0x7f86f00cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:00:10.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86f5532700 1 -- 172.21.15.40:0/1569564275 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86f00cb2a0 msgr2=0x7f86f00cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:00:10.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86e7fff700 1 -- 172.21.15.40:0/1569564275 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86f00cb2a0 msgr2=0x7f86f00cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:00:10.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86f5532700 1 --2- 172.21.15.40:0/1569564275 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86f00cb2a0 0x7f86f00cb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:10.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86f5532700 1 --2- 172.21.15.40:0/1569564275 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86f00cb2a0 0x7f86f00cb6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:10.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86e7fff700 1 --2- 172.21.15.40:0/1569564275 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86f00cb2a0 0x7f86f00cb6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86e7fff700 1 -- 172.21.15.40:0/1569564275 >> v1:172.21.15.43:6789/0 conn(0x7f86f0151050 legacy=0x7f86f00c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:10.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86e7fff700 1 -- 172.21.15.40:0/1569564275 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f006b290 con 0x7f86f0157d40 2024-09-13T18:00:10.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86e7fff700 1 --2- 172.21.15.40:0/1569564275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86f0157d40 0x7f86f0066220 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009910 tx=0x7f86e0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f9f53d84c210a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:10.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86e77fe700 1 -- 172.21.15.40:0/1569564275 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e000ba60 con 0x7f86f0157d40 2024-09-13T18:00:10.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86e77fe700 1 -- 172.21.15.40:0/1569564275 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e000bbc0 con 0x7f86f0157d40 2024-09-13T18:00:10.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86e77fe700 1 -- 172.21.15.40:0/1569564275 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0009e60 con 0x7f86f0157d40 2024-09-13T18:00:10.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86f5d33700 1 -- 172.21.15.40:0/1569564275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86f0157d40 msgr2=0x7f86f0066220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:10.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86f5d33700 1 --2- 172.21.15.40:0/1569564275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86f0157d40 0x7f86f0066220 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009910 tx=0x7f86e0009c20 comp rx=0 tx=0).stop 2024-09-13T18:00:10.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86f5d33700 1 -- 172.21.15.40:0/1569564275 shutdown_connections 2024-09-13T18:00:10.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86f5d33700 1 --2- 172.21.15.40:0/1569564275 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86f00cb2a0 0x7f86f00cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86f5d33700 1 --2- 172.21.15.40:0/1569564275 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86f0157d40 0x7f86f0066220 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86f5d33700 1 -- 172.21.15.40:0/1569564275 >> 172.21.15.40:0/1569564275 conn(0x7f86f006a390 msgr2=0x7f86f006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:10.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86f5d33700 1 -- 172.21.15.40:0/1569564275 shutdown_connections 2024-09-13T18:00:10.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.841+0000 7f86f5d33700 1 -- 172.21.15.40:0/1569564275 wait complete. 2024-09-13T18:00:10.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f5d33700 1 Processor -- start 2024-09-13T18:00:10.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f5d33700 1 -- start start 2024-09-13T18:00:10.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f5d33700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86f00cb2a0 0x7f86f01471b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f5d33700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f0151050 0x7f86f0071410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f5d33700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86f0157d40 0x7f86f0071950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f5d33700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f01ef8d0 con 0x7f86f0157d40 2024-09-13T18:00:10.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f5d33700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f01eec70 con 0x7f86f0151050 2024-09-13T18:00:10.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f5d33700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f01f2f00 con 0x7f86f00cb2a0 2024-09-13T18:00:10.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f5532700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86f0157d40 0x7f86f0071950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f5532700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86f0157d40 0x7f86f0071950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47340/0 (socket says 172.21.15.40:47340) 2024-09-13T18:00:10.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f5532700 1 -- 172.21.15.40:0/2970203754 learned_addr learned my addr 172.21.15.40:0/2970203754 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:10.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f4d31700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86f00cb2a0 0x7f86f01471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86e7fff700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f0151050 0x7f86f0071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f4d31700 1 -- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86f00cb2a0 msgr2=0x7f86f01471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:00:10.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f4d31700 1 -- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86f00cb2a0 msgr2=0x7f86f01471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:00:10.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f4d31700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86f00cb2a0 0x7f86f01471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:10.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f4d31700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86f00cb2a0 0x7f86f01471b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:10.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f5532700 1 -- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86f00cb2a0 msgr2=0x7f86f01471b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:00:10.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f5532700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f86f00cb2a0 0x7f86f01471b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f5532700 1 -- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f0151050 msgr2=0x7f86f0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:10.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f5532700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f0151050 0x7f86f0071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f5532700 1 -- 172.21.15.40:0/2970203754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e0009570 con 0x7f86f0157d40 2024-09-13T18:00:10.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f5532700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86f0157d40 0x7f86f0071950 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f86e8007e30 tx=0x7f86e800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:10.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86e57fa700 1 -- 172.21.15.40:0/2970203754 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e800ede0 con 0x7f86f0157d40 2024-09-13T18:00:10.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f5d33700 1 -- 172.21.15.40:0/2970203754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f01eef30 con 0x7f86f0157d40 2024-09-13T18:00:10.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86e57fa700 1 -- 172.21.15.40:0/2970203754 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e8014420 con 0x7f86f0157d40 2024-09-13T18:00:10.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86e57fa700 1 -- 172.21.15.40:0/2970203754 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e80147b0 con 0x7f86f0157d40 2024-09-13T18:00:10.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.845+0000 7f86f5d33700 1 -- 172.21.15.40:0/2970203754 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f01ef4f0 con 0x7f86f0157d40 2024-09-13T18:00:10.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.849+0000 7f86f5d33700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f014b7b0 0x7f86f014bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.849+0000 7f86f5d33700 1 -- 172.21.15.40:0/2970203754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f0067600 con 0x7f86f014b7b0 2024-09-13T18:00:10.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.849+0000 7f86f4d31700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f014b7b0 0x7f86f014bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.849+0000 7f86f4d31700 1 -- 172.21.15.40:0/2970203754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f86f0150380 con 0x7f86f014b7b0 2024-09-13T18:00:10.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.849+0000 7f86e57fa700 1 -- 172.21.15.40:0/2970203754 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f86e8014910 con 0x7f86f0157d40 2024-09-13T18:00:10.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.849+0000 7f86f4d31700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f014b7b0 0x7f86f014bb90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00e4d0 tx=0x7f86ec00e890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:10.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.849+0000 7f86e57fa700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86dc066730 0x7f86dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.849+0000 7f86e57fa700 1 -- 172.21.15.40:0/2970203754 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f86e8084cd0 con 0x7f86f0157d40 2024-09-13T18:00:10.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.849+0000 7f86e7fff700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86dc066730 0x7f86dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.849+0000 7f86e57fa700 1 -- 172.21.15.40:0/2970203754 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec0052e0 con 0x7f86f014b7b0 2024-09-13T18:00:10.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.849+0000 7f86e57fa700 1 -- 172.21.15.40:0/2970203754 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f86ec005440 con 0x7f86f014b7b0 2024-09-13T18:00:10.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.849+0000 7f86e7fff700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86dc066730 0x7f86dc068bf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009c60 tx=0x7f86e0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:10.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.849+0000 7f86e57fa700 1 -- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f014b7b0 msgr2=0x7f86f014bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:10.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.849+0000 7f86e57fa700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f014b7b0 0x7f86f014bb90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00e4d0 tx=0x7f86ec00e890 comp rx=0 tx=0).stop 2024-09-13T18:00:10.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.849+0000 7f86e57fa700 1 -- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f014b7b0 msgr2=0x7f86f014bb90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:00:10.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.849+0000 7f86e57fa700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f014b7b0 0x7f86f014bb90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.857+0000 7f86f5d33700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f01f21e0 0x7f86f01f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:10.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.857+0000 7f86f5d33700 1 -- 172.21.15.40:0/2970203754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f01ef6e0 con 0x7f86f01f21e0 2024-09-13T18:00:10.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.857+0000 7f86f4d31700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f01f21e0 0x7f86f01f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:10.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.861+0000 7f86f4d31700 1 -- 172.21.15.40:0/2970203754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f86f01572a0 con 0x7f86f01f21e0 2024-09-13T18:00:10.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.861+0000 7f86f4d31700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f01f21e0 0x7f86f01f25f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00e4d0 tx=0x7f86ec001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:10.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.861+0000 7f86e57fa700 1 -- 172.21.15.40:0/2970203754 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec021070 con 0x7f86f01f21e0 2024-09-13T18:00:10.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.861+0000 7f86e57fa700 1 -- 172.21.15.40:0/2970203754 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4947 (secure 0 0 0) 0x7f86ec01b680 con 0x7f86f01f21e0 2024-09-13T18:00:10.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.861+0000 7f86e57fa700 1 -- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f01f21e0 msgr2=0x7f86f01f25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:10.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.861+0000 7f86e57fa700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f01f21e0 0x7f86f01f25f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00e4d0 tx=0x7f86ec001120 comp rx=0 tx=0).stop 2024-09-13T18:00:10.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.861+0000 7f86e57fa700 1 -- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f01f21e0 msgr2=0x7f86f01f25f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:00:10.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.861+0000 7f86e57fa700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86f01f21e0 0x7f86f01f25f0 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00e4d0 tx=0x7f86ec001120 comp rx=0 tx=0).stop 2024-09-13T18:00:10.869 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:00:10.869 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:00:10.869 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:00:10.869 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:00:10.869 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 76, 2024-09-13T18:00:10.870 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:00:10.870 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:00:10.870 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:00:10.870 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:00:10.870 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:10.870 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:00:10.870 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:00:10.870 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:00:10.870 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:00:10.870 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:10.870 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:10.870 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:10.870 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:10.870 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:00:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:00:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:10.871 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:10.872 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:10.872 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:10.872 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:10.872 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:10.872 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:10.872 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.872 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:00:10.872 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:00:10.872 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:00:10.872 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:00:10.872 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:00:10.872 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:00:10.872 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:00:10.872 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:00:10.873 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:00:10.873 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:00:10.873 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:00:10.873 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:00:10.873 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:00:10.873 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:00:10.873 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:00:10.873 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:00:10.873 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:00:10.873 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:10.873 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:10.873 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:10.873 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:10.874 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:10.874 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:10.874 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:10.874 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:10.874 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:10.874 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:10.874 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:10.874 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:00:10.874 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.874 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:00:10.874 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.874 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:00:10.874 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:00:10.874 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:10.875 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:10.875 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.875 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:10.875 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:00:10.875 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:10.875 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.875 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.875 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:10.875 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:00:10.875 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:10.875 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:10.875 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:10.875 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.875 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:00:10.876 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:00:10.876 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:10.876 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:10.876 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:10.876 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.876 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.876 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:00:10.876 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:00:10.876 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:10.876 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:10.876 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.876 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:10.876 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:00:10.877 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:10.877 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.877 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.877 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:10.877 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:00:10.877 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:10.877 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:10.877 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:10.877 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.877 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:00:10.877 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:00:10.877 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:10.877 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:10.878 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:10.878 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.878 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.878 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:00:10.878 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:00:10.878 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:10.878 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:10.878 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.878 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:10.878 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:00:10.878 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:10.878 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.878 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.879 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:10.879 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:00:10.879 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:10.879 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:10.879 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:10.879 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.879 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:00:10.879 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:00:10.879 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:10.879 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:10.879 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:10.879 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:10.879 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:10.879 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.880 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:00:10.880 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:00:10.880 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.880 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:10.880 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:10.880 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:00:10.880 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:10.880 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:10.880 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:00:10.880 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.880 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:10.880 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:10.880 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:00:10.880 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:10.881 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:10.881 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:00:10.881 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.881 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:10.881 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:10.881 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:00:10.881 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:10.881 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:10.881 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:00:10.881 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.881 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:10.881 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:10.881 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:00:10.881 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:10.882 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:10.882 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:00:10.882 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:10.882 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:10.882 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:10.882 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:00:10.882 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:10.882 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:10.882 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:10.882 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:00:10.882 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:00:10.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.861+0000 7f86f5d33700 1 -- 172.21.15.40:0/2970203754 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86dc066730 msgr2=0x7f86dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:10.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.861+0000 7f86f5d33700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86dc066730 0x7f86dc068bf0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009c60 tx=0x7f86e0008030 comp rx=0 tx=0).stop 2024-09-13T18:00:10.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.861+0000 7f86f5d33700 1 -- 172.21.15.40:0/2970203754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86f0157d40 msgr2=0x7f86f0071950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:10.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.861+0000 7f86f5532700 1 -- 172.21.15.40:0/2970203754 reap_dead start 2024-09-13T18:00:10.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.861+0000 7f86f5d33700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86f0157d40 0x7f86f0071950 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f86e8007e30 tx=0x7f86e800e3c0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.861+0000 7f86f5d33700 1 -- 172.21.15.40:0/2970203754 shutdown_connections 2024-09-13T18:00:10.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.861+0000 7f86f5d33700 1 --2- 172.21.15.40:0/2970203754 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86f0157d40 0x7f86f0071950 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:10.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.861+0000 7f86f5d33700 1 -- 172.21.15.40:0/2970203754 >> 172.21.15.40:0/2970203754 conn(0x7f86f006a390 msgr2=0x7f86f014f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:10.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.861+0000 7f86f5d33700 1 -- 172.21.15.40:0/2970203754 shutdown_connections 2024-09-13T18:00:10.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.861+0000 7f86f5d33700 1 -- 172.21.15.40:0/2970203754 wait complete. 2024-09-13T18:00:10.885 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:00:10.885 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:00:10.885 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:00:11.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.993+0000 7f76fbfff700 1 Processor -- start 2024-09-13T18:00:11.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.993+0000 7f76fbfff700 1 -- start start 2024-09-13T18:00:11.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fbfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f40d8b40 0x7f76f4049090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fbfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f76f4007430 0x7f76f40e1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fbfff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76f4050390 con 0x7f76f404f220 2024-09-13T18:00:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fbfff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f40508e0 con 0x7f76f4007430 2024-09-13T18:00:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fbfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f40df6f0 con 0x7f76f40d8b40 2024-09-13T18:00:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fb7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f76f404f220 0x7f76f40e2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41952/0 (socket says 172.21.15.40:41952) 2024-09-13T18:00:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fb7fe700 1 -- 172.21.15.40:0/3584619033 learned_addr learned my addr 172.21.15.40:0/3584619033 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:00:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fa7fc700 1 --2- 172.21.15.40:0/3584619033 >> v2:172.21.15.40:3300/0 conn(0x7f76f4007430 0x7f76f40e1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:11.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76faffd700 1 --2- 172.21.15.40:0/3584619033 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f40d8b40 0x7f76f4049090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76faffd700 1 -- 172.21.15.40:0/3584619033 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f40d8b40 msgr2=0x7f76f4049090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:00:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76faffd700 1 -- 172.21.15.40:0/3584619033 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f40d8b40 msgr2=0x7f76f4049090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:00:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76faffd700 1 --2- 172.21.15.40:0/3584619033 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f40d8b40 0x7f76f4049090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76faffd700 1 --2- 172.21.15.40:0/3584619033 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f40d8b40 0x7f76f4049090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fa7fc700 1 -- 172.21.15.40:0/3584619033 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f40d8b40 msgr2=0x7f76f4049090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:00:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fa7fc700 1 --2- 172.21.15.40:0/3584619033 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f40d8b40 0x7f76f4049090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fa7fc700 1 -- 172.21.15.40:0/3584619033 >> v1:172.21.15.40:6789/0 conn(0x7f76f404f220 legacy=0x7f76f40e2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fa7fc700 1 -- 172.21.15.40:0/3584619033 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f4055030 con 0x7f76f4007430 2024-09-13T18:00:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fa7fc700 1 --2- 172.21.15.40:0/3584619033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f4007430 0x7f76f40e1c50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009850 tx=0x7f76f0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a903e7bfdd3e1f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76f9ffb700 1 -- 172.21.15.40:0/3584619033 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0011830 con 0x7f76f4007430 2024-09-13T18:00:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76f9ffb700 1 -- 172.21.15.40:0/3584619033 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f0011990 con 0x7f76f4007430 2024-09-13T18:00:11.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76f9ffb700 1 -- 172.21.15.40:0/3584619033 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0011d20 con 0x7f76f4007430 2024-09-13T18:00:11.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fbfff700 1 -- 172.21.15.40:0/3584619033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f4007430 msgr2=0x7f76f40e1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fbfff700 1 --2- 172.21.15.40:0/3584619033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f4007430 0x7f76f40e1c50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009850 tx=0x7f76f0009b60 comp rx=0 tx=0).stop 2024-09-13T18:00:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fbfff700 1 -- 172.21.15.40:0/3584619033 shutdown_connections 2024-09-13T18:00:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fbfff700 1 --2- 172.21.15.40:0/3584619033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f4007430 0x7f76f40e1c50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fbfff700 1 --2- 172.21.15.40:0/3584619033 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f40d8b40 0x7f76f4049090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fbfff700 1 -- 172.21.15.40:0/3584619033 >> 172.21.15.40:0/3584619033 conn(0x7f76f4054110 msgr2=0x7f76f4054520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fbfff700 1 -- 172.21.15.40:0/3584619033 shutdown_connections 2024-09-13T18:00:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fbfff700 1 -- 172.21.15.40:0/3584619033 wait complete. 2024-09-13T18:00:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:10.997+0000 7f76fbfff700 1 Processor -- start 2024-09-13T18:00:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76fbfff700 1 -- start start 2024-09-13T18:00:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76fbfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4007430 0x7f76f4165d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:11.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76fbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f404f220 0x7f76f41779b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:11.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76fbfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76f40d8b40 0x7f76f40ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:11.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76fbfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f4169260 con 0x7f76f404f220 2024-09-13T18:00:11.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76fbfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f4167a50 con 0x7f76f40d8b40 2024-09-13T18:00:11.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76fbfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f40dbfe0 con 0x7f76f4007430 2024-09-13T18:00:11.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76fa7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f404f220 0x7f76f41779b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:11.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76fa7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f404f220 0x7f76f41779b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47348/0 (socket says 172.21.15.40:47348) 2024-09-13T18:00:11.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76fa7fc700 1 -- 172.21.15.40:0/235358793 learned_addr learned my addr 172.21.15.40:0/235358793 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:11.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76fb7fe700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76f40d8b40 0x7f76f40ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:11.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76faffd700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4007430 0x7f76f4165d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:11.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76faffd700 1 -- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4007430 msgr2=0x7f76f4165d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:00:11.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76faffd700 1 -- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4007430 msgr2=0x7f76f4165d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:00:11.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76faffd700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4007430 0x7f76f4165d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:11.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76faffd700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4007430 0x7f76f4165d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:11.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76fa7fc700 1 -- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4007430 msgr2=0x7f76f4165d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:00:11.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76fa7fc700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4007430 0x7f76f4165d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:11.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76fa7fc700 1 -- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76f40d8b40 msgr2=0x7f76f40ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:11.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76fa7fc700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76f40d8b40 0x7f76f40ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:11.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76fa7fc700 1 -- 172.21.15.40:0/235358793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f00094b0 con 0x7f76f404f220 2024-09-13T18:00:11.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76fa7fc700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f404f220 0x7f76f41779b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f76f0000c00 tx=0x7f76f0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:11.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76e3fff700 1 -- 172.21.15.40:0/235358793 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0004480 con 0x7f76f404f220 2024-09-13T18:00:11.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76e3fff700 1 -- 172.21.15.40:0/235358793 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f001f070 con 0x7f76f404f220 2024-09-13T18:00:11.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76fbfff700 1 -- 172.21.15.40:0/235358793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f4167ca0 con 0x7f76f404f220 2024-09-13T18:00:11.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76e3fff700 1 -- 172.21.15.40:0/235358793 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0003900 con 0x7f76f404f220 2024-09-13T18:00:11.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.001+0000 7f76fbfff700 1 -- 172.21.15.40:0/235358793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f40db7e0 con 0x7f76f404f220 2024-09-13T18:00:11.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.005+0000 7f76fbfff700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 0x7f76f40ced90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:11.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.005+0000 7f76fbfff700 1 -- 172.21.15.40:0/235358793 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f40084b0 con 0x7f76f4177ef0 2024-09-13T18:00:11.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.005+0000 7f76faffd700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 0x7f76f40ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:11.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.005+0000 7f76faffd700 1 -- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 msgr2=0x7f76f40ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-09-13T18:00:11.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.005+0000 7f76faffd700 1 -- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 msgr2=0x7f76f40ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:00:11.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.005+0000 7f76faffd700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 0x7f76f40ced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:11.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.005+0000 7f76faffd700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 0x7f76f40ced90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:11.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.005+0000 7f76e3fff700 1 -- 172.21.15.40:0/235358793 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f76f0021030 con 0x7f76f404f220 2024-09-13T18:00:11.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.005+0000 7f76e3fff700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f76e8066730 0x7f76e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:11.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.005+0000 7f76e3fff700 1 -- 172.21.15.40:0/235358793 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f76f0085800 con 0x7f76f404f220 2024-09-13T18:00:11.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.005+0000 7f76fb7fe700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f76e8066730 0x7f76e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:11.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.005+0000 7f76fb7fe700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f76e8066730 0x7f76e8068bf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f76fc072e80 tx=0x7f76fc065ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:11.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.209+0000 7f76faffd700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 0x7f76f40ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:11.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.209+0000 7f76faffd700 1 -- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 msgr2=0x7f76f40ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-13T18:00:11.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.209+0000 7f76faffd700 1 -- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 msgr2=0x7f76f40ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:00:11.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.209+0000 7f76faffd700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 0x7f76f40ced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:11.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.209+0000 7f76faffd700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 0x7f76f40ced90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:00:11.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.613+0000 7f76faffd700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 0x7f76f40ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:11.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.613+0000 7f76faffd700 1 -- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 msgr2=0x7f76f40ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-13T18:00:11.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.613+0000 7f76faffd700 1 -- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 msgr2=0x7f76f40ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:00:11.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.613+0000 7f76faffd700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 0x7f76f40ced90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:11.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:11.613+0000 7f76faffd700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 0x7f76f40ced90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:00:12.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.417+0000 7f76faffd700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 0x7f76f40ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:12.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.417+0000 7f76faffd700 1 -- 172.21.15.40:0/235358793 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76f4007150 con 0x7f76f4177ef0 2024-09-13T18:00:12.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.417+0000 7f76faffd700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 0x7f76f40ced90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0166e0 tx=0x7f76ec016aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:12.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.417+0000 7f76e3fff700 1 -- 172.21.15.40:0/235358793 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec018920 con 0x7f76f4177ef0 2024-09-13T18:00:12.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.417+0000 7f76e3fff700 1 -- 172.21.15.40:0/235358793 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f76ec00c040 con 0x7f76f4177ef0 2024-09-13T18:00:12.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.417+0000 7f76e3fff700 1 -- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 msgr2=0x7f76f40ced90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:12.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.417+0000 7f76e3fff700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 0x7f76f40ced90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0166e0 tx=0x7f76ec016aa0 comp rx=0 tx=0).stop 2024-09-13T18:00:12.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.417+0000 7f76e3fff700 1 -- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 msgr2=0x7f76f40ced90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:00:12.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.417+0000 7f76e3fff700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f4177ef0 0x7f76f40ced90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:12.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.429+0000 7f76fbfff700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f40dab90 0x7f76f4049090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:12.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.429+0000 7f76fbfff700 1 -- 172.21.15.40:0/235358793 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f4006c30 con 0x7f76f40dab90 2024-09-13T18:00:12.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.429+0000 7f76faffd700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f40dab90 0x7f76f4049090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:12.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.429+0000 7f76faffd700 1 -- 172.21.15.40:0/235358793 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f76f4006950 con 0x7f76f40dab90 2024-09-13T18:00:12.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.429+0000 7f76faffd700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f40dab90 0x7f76f4049090 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0166e0 tx=0x7f76ec0047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:12.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.429+0000 7f76e3fff700 1 -- 172.21.15.40:0/235358793 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec027580 con 0x7f76f40dab90 2024-09-13T18:00:12.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.433+0000 7f76e3fff700 1 -- 172.21.15.40:0/235358793 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4449 (secure 0 0 0) 0x7f76ec02b070 con 0x7f76f40dab90 2024-09-13T18:00:12.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.433+0000 7f76e3fff700 1 -- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f40dab90 msgr2=0x7f76f4049090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:12.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.433+0000 7f76e3fff700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f40dab90 0x7f76f4049090 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0166e0 tx=0x7f76ec0047e0 comp rx=0 tx=0).stop 2024-09-13T18:00:12.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.433+0000 7f76e3fff700 1 -- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f40dab90 msgr2=0x7f76f4049090 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:00:12.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.433+0000 7f76e3fff700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76f40dab90 0x7f76f4049090 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0166e0 tx=0x7f76ec0047e0 comp rx=0 tx=0).stop 2024-09-13T18:00:12.439 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:00:12.440 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:00:12.440 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:00:12.440 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:00:12.440 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 76, 2024-09-13T18:00:12.440 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:00:12.440 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:00:12.440 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:00:12.440 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:00:12.440 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:12.440 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:00:12.440 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:00:12.440 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:00:12.441 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:00:12.441 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:12.441 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:12.441 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:12.441 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:12.441 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:12.441 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:12.441 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:12.441 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:12.441 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:12.441 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:12.441 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:12.441 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:00:12.441 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:00:12.442 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:12.442 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:12.442 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:12.442 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:12.442 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:12.442 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:12.442 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:12.442 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:12.442 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:12.442 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:12.442 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:12.442 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:12.442 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:00:12.442 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:00:12.443 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:00:12.443 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:00:12.443 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:00:12.443 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:00:12.443 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:00:12.443 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:00:12.443 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:00:12.443 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:00:12.443 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:00:12.443 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:00:12.443 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:00:12.443 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:00:12.443 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:00:12.443 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:00:12.444 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:00:12.444 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:12.444 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:12.444 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:12.444 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:12.444 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:12.444 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:12.444 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:12.444 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:12.444 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:12.444 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:12.444 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:12.444 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:00:12.445 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:12.445 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:00:12.445 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:12.445 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:00:12.445 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:00:12.445 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:12.445 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:12.445 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:12.445 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:12.445 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:00:12.445 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:12.445 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:12.445 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:12.445 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:12.446 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:00:12.446 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:12.446 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:12.446 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:12.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:12.446 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:00:12.446 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:00:12.446 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:12.446 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:12.446 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:12.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:12.446 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:12.446 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:00:12.446 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:00:12.447 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:12.447 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:12.447 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:12.447 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:12.447 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:00:12.447 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:12.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:12.447 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:12.447 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:12.447 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:00:12.447 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:12.447 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:12.447 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:12.448 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:12.448 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:00:12.448 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:00:12.448 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:12.448 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:12.448 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:12.448 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:12.448 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:12.448 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:00:12.448 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:00:12.448 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:12.448 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:12.448 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:12.448 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:12.449 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:00:12.449 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:12.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:12.449 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:12.449 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:12.449 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:00:12.449 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:12.449 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:12.449 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:12.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:12.449 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:00:12.449 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:00:12.449 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:12.450 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:12.450 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:12.450 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:12.450 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:12.450 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:12.450 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:00:12.450 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:00:12.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:12.450 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:12.450 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:12.450 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:00:12.450 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:12.450 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:12.450 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:00:12.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:12.451 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:12.451 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:12.451 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:00:12.451 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:12.451 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:12.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:12.451 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:00:12.451 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:00:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.433+0000 7f76fbfff700 1 -- 172.21.15.40:0/235358793 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f76e8066730 msgr2=0x7f76e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.433+0000 7f76fbfff700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f76e8066730 0x7f76e8068bf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f76fc072e80 tx=0x7f76fc065ea0 comp rx=0 tx=0).stop 2024-09-13T18:00:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.433+0000 7f76fbfff700 1 -- 172.21.15.40:0/235358793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f404f220 msgr2=0x7f76f41779b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.433+0000 7f76fb7fe700 1 -- 172.21.15.40:0/235358793 reap_dead start 2024-09-13T18:00:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.433+0000 7f76fbfff700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f404f220 0x7f76f41779b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f76f0000c00 tx=0x7f76f0004770 comp rx=0 tx=0).stop 2024-09-13T18:00:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.433+0000 7f76fbfff700 1 -- 172.21.15.40:0/235358793 shutdown_connections 2024-09-13T18:00:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.433+0000 7f76fbfff700 1 --2- 172.21.15.40:0/235358793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76f404f220 0x7f76f41779b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.433+0000 7f76fbfff700 1 -- 172.21.15.40:0/235358793 >> 172.21.15.40:0/235358793 conn(0x7f76f4054110 msgr2=0x7f76f4167140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.433+0000 7f76fbfff700 1 -- 172.21.15.40:0/235358793 shutdown_connections 2024-09-13T18:00:12.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.433+0000 7f76fbfff700 1 -- 172.21.15.40:0/235358793 wait complete. 2024-09-13T18:00:12.455 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:00:12.455 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:00:12.455 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:00:12.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.565+0000 7f78d217a700 1 Processor -- start 2024-09-13T18:00:12.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.565+0000 7f78d217a700 1 -- start start 2024-09-13T18:00:12.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d217a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f78cc153840 0x7f78cc0c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:12.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d217a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f78cc0c9e20 0x7f78cc0cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:12.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d217a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78cc0cc060 con 0x7f78cc0cc4f0 2024-09-13T18:00:12.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d217a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78cc0b2840 con 0x7f78cc0c9e20 2024-09-13T18:00:12.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d217a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc0ca870 con 0x7f78cc153840 2024-09-13T18:00:12.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d1979700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f78cc0c9e20 0x7f78cc0cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:12.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d1979700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f78cc0c9e20 0x7f78cc0cf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40804/0 (socket says 172.21.15.40:40804) 2024-09-13T18:00:12.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d1979700 1 -- 172.21.15.40:0/2398127849 learned_addr learned my addr 172.21.15.40:0/2398127849 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:12.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d1178700 1 --2- 172.21.15.40:0/2398127849 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f78cc153840 0x7f78cc0c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:12.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d1979700 1 -- 172.21.15.40:0/2398127849 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f78cc153840 msgr2=0x7f78cc0c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:00:12.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d1979700 1 --2- 172.21.15.40:0/2398127849 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f78cc153840 0x7f78cc0c3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:12.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d1979700 1 -- 172.21.15.40:0/2398127849 >> v1:172.21.15.40:6789/0 conn(0x7f78cc0cc4f0 legacy=0x7f78cc15d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:12.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d1979700 1 -- 172.21.15.40:0/2398127849 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78cc153c20 con 0x7f78cc0c9e20 2024-09-13T18:00:12.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d1979700 1 --2- 172.21.15.40:0/2398127849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc0c9e20 0x7f78cc0cf260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009e50 tx=0x7f78c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1d09b03c5adf7b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:12.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78c3fff700 1 -- 172.21.15.40:0/2398127849 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c400e070 con 0x7f78cc0c9e20 2024-09-13T18:00:12.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78c3fff700 1 -- 172.21.15.40:0/2398127849 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c400b910 con 0x7f78cc0c9e20 2024-09-13T18:00:12.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78c3fff700 1 -- 172.21.15.40:0/2398127849 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c400bcc0 con 0x7f78cc0c9e20 2024-09-13T18:00:12.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d217a700 1 -- 172.21.15.40:0/2398127849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc0c9e20 msgr2=0x7f78cc0cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:12.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d217a700 1 --2- 172.21.15.40:0/2398127849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc0c9e20 0x7f78cc0cf260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009e50 tx=0x7f78c400b330 comp rx=0 tx=0).stop 2024-09-13T18:00:12.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d217a700 1 -- 172.21.15.40:0/2398127849 shutdown_connections 2024-09-13T18:00:12.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d217a700 1 --2- 172.21.15.40:0/2398127849 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc0c9e20 0x7f78cc0cf260 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:12.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d217a700 1 --2- 172.21.15.40:0/2398127849 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f78cc153840 0x7f78cc0c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:12.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d217a700 1 -- 172.21.15.40:0/2398127849 >> 172.21.15.40:0/2398127849 conn(0x7f78cc06c800 msgr2=0x7f78cc06cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:12.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d217a700 1 -- 172.21.15.40:0/2398127849 shutdown_connections 2024-09-13T18:00:12.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.569+0000 7f78d217a700 1 -- 172.21.15.40:0/2398127849 wait complete. 2024-09-13T18:00:12.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d217a700 1 Processor -- start 2024-09-13T18:00:12.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d217a700 1 -- start start 2024-09-13T18:00:12.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d217a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc0c9e20 0x7f78cc1e3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:12.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d217a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f78cc0cc4f0 0x7f78cc155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d217a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78cc153840 0x7f78cc156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d217a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc14e5c0 con 0x7f78cc0c9e20 2024-09-13T18:00:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d217a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc14e180 con 0x7f78cc153840 2024-09-13T18:00:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d217a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc1f2a20 con 0x7f78cc0cc4f0 2024-09-13T18:00:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d1178700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc0c9e20 0x7f78cc1e3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d1178700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc0c9e20 0x7f78cc1e3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40806/0 (socket says 172.21.15.40:40806) 2024-09-13T18:00:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d1178700 1 -- 172.21.15.40:0/1544662955 learned_addr learned my addr 172.21.15.40:0/1544662955 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d0977700 1 --2- 172.21.15.40:0/1544662955 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f78cc0cc4f0 0x7f78cc155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d1979700 1 --2- 172.21.15.40:0/1544662955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78cc153840 0x7f78cc156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d1178700 1 -- 172.21.15.40:0/1544662955 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f78cc0cc4f0 msgr2=0x7f78cc155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d1178700 1 --2- 172.21.15.40:0/1544662955 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f78cc0cc4f0 0x7f78cc155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d1178700 1 -- 172.21.15.40:0/1544662955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78cc153840 msgr2=0x7f78cc156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:12.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d1178700 1 --2- 172.21.15.40:0/1544662955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78cc153840 0x7f78cc156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d1178700 1 -- 172.21.15.40:0/1544662955 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c4009b30 con 0x7f78cc0c9e20 2024-09-13T18:00:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d1178700 1 --2- 172.21.15.40:0/1544662955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc0c9e20 0x7f78cc1e3450 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f78c800e840 tx=0x7f78c800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78c1ffb700 1 -- 172.21.15.40:0/1544662955 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c80095a0 con 0x7f78cc0c9e20 2024-09-13T18:00:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78c1ffb700 1 -- 172.21.15.40:0/1544662955 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c800c690 con 0x7f78cc0c9e20 2024-09-13T18:00:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d217a700 1 -- 172.21.15.40:0/1544662955 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78cc1f2d00 con 0x7f78cc0c9e20 2024-09-13T18:00:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78c1ffb700 1 -- 172.21.15.40:0/1544662955 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c800ca00 con 0x7f78cc0c9e20 2024-09-13T18:00:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d217a700 1 -- 172.21.15.40:0/1544662955 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78cc1f32c0 con 0x7f78cc0c9e20 2024-09-13T18:00:12.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.573+0000 7f78d217a700 1 -- 172.21.15.40:0/1544662955 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78cc1f34e0 con 0x7f78cc0c9e20 2024-09-13T18:00:12.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.577+0000 7f78c1ffb700 1 -- 172.21.15.40:0/1544662955 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f78c800cb60 con 0x7f78cc0c9e20 2024-09-13T18:00:12.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.577+0000 7f78c1ffb700 1 --2- 172.21.15.40:0/1544662955 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f78b0066730 0x7f78b0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:12.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.577+0000 7f78c1ffb700 1 -- 172.21.15.40:0/1544662955 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f78c8084500 con 0x7f78cc0c9e20 2024-09-13T18:00:12.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.577+0000 7f78d0977700 1 --2- 172.21.15.40:0/1544662955 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f78b0066730 0x7f78b0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:12.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.577+0000 7f78d0977700 1 --2- 172.21.15.40:0/1544662955 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f78b0066730 0x7f78b0068bf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0c5dd0 tx=0x7f78bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:12.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.581+0000 7f78c1ffb700 1 -- 172.21.15.40:0/1544662955 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f78c80514b0 con 0x7f78cc0c9e20 2024-09-13T18:00:12.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.849+0000 7f78d217a700 1 -- 172.21.15.40:0/1544662955 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f78cc0cbcf0 con 0x7f78cc0c9e20 2024-09-13T18:00:12.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.849+0000 7f78c1ffb700 1 -- 172.21.15.40:0/1544662955 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f78c8054ad0 con 0x7f78cc0c9e20 2024-09-13T18:00:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.853+0000 7f78d217a700 1 -- 172.21.15.40:0/1544662955 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f78b0066730 msgr2=0x7f78b0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:12.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.853+0000 7f78d217a700 1 --2- 172.21.15.40:0/1544662955 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f78b0066730 0x7f78b0068bf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0c5dd0 tx=0x7f78bc009040 comp rx=0 tx=0).stop 2024-09-13T18:00:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.853+0000 7f78d217a700 1 -- 172.21.15.40:0/1544662955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc0c9e20 msgr2=0x7f78cc1e3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.853+0000 7f78d217a700 1 --2- 172.21.15.40:0/1544662955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc0c9e20 0x7f78cc1e3450 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f78c800e840 tx=0x7f78c800ec00 comp rx=0 tx=0).stop 2024-09-13T18:00:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.853+0000 7f78d217a700 1 -- 172.21.15.40:0/1544662955 shutdown_connections 2024-09-13T18:00:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.853+0000 7f78d217a700 1 --2- 172.21.15.40:0/1544662955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78cc153840 0x7f78cc156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.853+0000 7f78d217a700 1 --2- 172.21.15.40:0/1544662955 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f78b0066730 0x7f78b0068bf0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.853+0000 7f78d217a700 1 --2- 172.21.15.40:0/1544662955 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f78cc0cc4f0 0x7f78cc155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.853+0000 7f78d217a700 1 --2- 172.21.15.40:0/1544662955 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78cc0c9e20 0x7f78cc1e3450 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.853+0000 7f78d217a700 1 -- 172.21.15.40:0/1544662955 >> 172.21.15.40:0/1544662955 conn(0x7f78cc06c800 msgr2=0x7f78cc152b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.853+0000 7f78d217a700 1 -- 172.21.15.40:0/1544662955 shutdown_connections 2024-09-13T18:00:12.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:12.853+0000 7f78d217a700 1 -- 172.21.15.40:0/1544662955 wait complete. 2024-09-13T18:00:12.873 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:00:22.875 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:00:22.875 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:00:22.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.989+0000 7f7b1521c700 1 Processor -- start 2024-09-13T18:00:22.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.989+0000 7f7b1521c700 1 -- start start 2024-09-13T18:00:23.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.989+0000 7f7b1521c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b100c8520 0x7f7b10157340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b1521c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7b100c6a00 0x7f7b1015b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b1521c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b1015a930 con 0x7f7b100c8900 2024-09-13T18:00:23.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b1521c700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b1015b930 con 0x7f7b100c6a00 2024-09-13T18:00:23.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b1521c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b100c5f30 con 0x7f7b100c8520 2024-09-13T18:00:23.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b0f59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7b100c6a00 0x7f7b1015b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b0f59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7b100c6a00 0x7f7b1015b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48276/0 (socket says 172.21.15.40:48276) 2024-09-13T18:00:23.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b0ed9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b100c8520 0x7f7b10157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b0f59e700 1 -- 172.21.15.40:0/2604175923 learned_addr learned my addr 172.21.15.40:0/2604175923 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:23.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b0f59e700 1 -- 172.21.15.40:0/2604175923 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b100c8520 msgr2=0x7f7b10157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:00:23.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b0f59e700 1 --2- 172.21.15.40:0/2604175923 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b100c8520 0x7f7b10157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b0f59e700 1 -- 172.21.15.40:0/2604175923 >> v1:172.21.15.43:6789/0 conn(0x7f7b100c8900 legacy=0x7f7b100c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b0f59e700 1 -- 172.21.15.40:0/2604175923 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b100c6210 con 0x7f7b100c6a00 2024-09-13T18:00:23.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b0f59e700 1 --2- 172.21.15.40:0/2604175923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b100c6a00 0x7f7b1015b3f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009e50 tx=0x7f7b0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecb27ef9c2d32de server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b0dd9b700 1 -- 172.21.15.40:0/2604175923 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0800e070 con 0x7f7b100c6a00 2024-09-13T18:00:23.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b0dd9b700 1 -- 172.21.15.40:0/2604175923 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0800b910 con 0x7f7b100c6a00 2024-09-13T18:00:23.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b0dd9b700 1 -- 172.21.15.40:0/2604175923 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0800bcc0 con 0x7f7b100c6a00 2024-09-13T18:00:23.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b1521c700 1 -- 172.21.15.40:0/2604175923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b100c6a00 msgr2=0x7f7b1015b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b1521c700 1 --2- 172.21.15.40:0/2604175923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b100c6a00 0x7f7b1015b3f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009e50 tx=0x7f7b0800b330 comp rx=0 tx=0).stop 2024-09-13T18:00:23.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b1521c700 1 -- 172.21.15.40:0/2604175923 shutdown_connections 2024-09-13T18:00:23.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b1521c700 1 --2- 172.21.15.40:0/2604175923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b100c6a00 0x7f7b1015b3f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b1521c700 1 --2- 172.21.15.40:0/2604175923 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b100c8520 0x7f7b10157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b1521c700 1 -- 172.21.15.40:0/2604175923 >> 172.21.15.40:0/2604175923 conn(0x7f7b1006a570 msgr2=0x7f7b1006a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:23.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b1521c700 1 -- 172.21.15.40:0/2604175923 shutdown_connections 2024-09-13T18:00:23.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.993+0000 7f7b1521c700 1 -- 172.21.15.40:0/2604175923 wait complete. 2024-09-13T18:00:23.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b1521c700 1 Processor -- start 2024-09-13T18:00:23.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b1521c700 1 -- start start 2024-09-13T18:00:23.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b1521c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b100c6a00 0x7f7b10155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b1521c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b100c8520 0x7f7b101ef4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b1521c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b100c8900 0x7f7b1006e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b1521c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b100c9cc0 con 0x7f7b100c8520 2024-09-13T18:00:23.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b1521c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b10071c40 con 0x7f7b100c6a00 2024-09-13T18:00:23.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b1521c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b10155fa0 con 0x7f7b100c8900 2024-09-13T18:00:23.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b0e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b100c8520 0x7f7b101ef4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b0e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b100c8520 0x7f7b101ef4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48292/0 (socket says 172.21.15.40:48292) 2024-09-13T18:00:23.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b0e59c700 1 -- 172.21.15.40:0/1755205201 learned_addr learned my addr 172.21.15.40:0/1755205201 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:23.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b0ed9d700 1 --2- 172.21.15.40:0/1755205201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b100c6a00 0x7f7b10155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b0f59e700 1 --2- 172.21.15.40:0/1755205201 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b100c8900 0x7f7b1006e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b0e59c700 1 -- 172.21.15.40:0/1755205201 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b100c8900 msgr2=0x7f7b1006e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b0e59c700 1 --2- 172.21.15.40:0/1755205201 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b100c8900 0x7f7b1006e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b0e59c700 1 -- 172.21.15.40:0/1755205201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b100c6a00 msgr2=0x7f7b10155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b0e59c700 1 --2- 172.21.15.40:0/1755205201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b100c6a00 0x7f7b10155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b0e59c700 1 -- 172.21.15.40:0/1755205201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b08009b30 con 0x7f7b100c8520 2024-09-13T18:00:23.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b0e59c700 1 --2- 172.21.15.40:0/1755205201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b100c8520 0x7f7b101ef4c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400c500 tx=0x7f7b0400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7af77fe700 1 -- 172.21.15.40:0/1755205201 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b040052d0 con 0x7f7b100c8520 2024-09-13T18:00:23.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7af77fe700 1 -- 172.21.15.40:0/1755205201 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b04005430 con 0x7f7b100c8520 2024-09-13T18:00:23.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7af77fe700 1 -- 172.21.15.40:0/1755205201 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b040055a0 con 0x7f7b100c8520 2024-09-13T18:00:23.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b1521c700 1 -- 172.21.15.40:0/1755205201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b10156280 con 0x7f7b100c8520 2024-09-13T18:00:23.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:22.997+0000 7f7b1521c700 1 -- 172.21.15.40:0/1755205201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b100715f0 con 0x7f7b100c8520 2024-09-13T18:00:23.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.001+0000 7f7b1521c700 1 -- 172.21.15.40:0/1755205201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b10159300 con 0x7f7b100c8520 2024-09-13T18:00:23.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.001+0000 7f7af77fe700 1 -- 172.21.15.40:0/1755205201 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7b040038e0 con 0x7f7b100c8520 2024-09-13T18:00:23.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.001+0000 7f7af77fe700 1 --2- 172.21.15.40:0/1755205201 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7afc066730 0x7f7afc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.001+0000 7f7b0ed9d700 1 --2- 172.21.15.40:0/1755205201 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7afc066730 0x7f7afc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.001+0000 7f7af77fe700 1 -- 172.21.15.40:0/1755205201 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7b04083850 con 0x7f7b100c8520 2024-09-13T18:00:23.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.001+0000 7f7b0ed9d700 1 --2- 172.21.15.40:0/1755205201 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7afc066730 0x7f7afc068bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009bf0 tx=0x7f7b00009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.009+0000 7f7af77fe700 1 -- 172.21.15.40:0/1755205201 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7b04050890 con 0x7f7b100c8520 2024-09-13T18:00:23.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.277+0000 7f7b1521c700 1 -- 172.21.15.40:0/1755205201 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f7b10154d60 con 0x7f7b100c8520 2024-09-13T18:00:23.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.277+0000 7f7af77fe700 1 -- 172.21.15.40:0/1755205201 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f7b04017620 con 0x7f7b100c8520 2024-09-13T18:00:23.286 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":76,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:00:23.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.281+0000 7f7b1521c700 1 -- 172.21.15.40:0/1755205201 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7afc066730 msgr2=0x7f7afc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.285+0000 7f7b1521c700 1 --2- 172.21.15.40:0/1755205201 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7afc066730 0x7f7afc068bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009bf0 tx=0x7f7b00009240 comp rx=0 tx=0).stop 2024-09-13T18:00:23.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.285+0000 7f7b1521c700 1 -- 172.21.15.40:0/1755205201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b100c8520 msgr2=0x7f7b101ef4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.285+0000 7f7b1521c700 1 --2- 172.21.15.40:0/1755205201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b100c8520 0x7f7b101ef4c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400c500 tx=0x7f7b0400c810 comp rx=0 tx=0).stop 2024-09-13T18:00:23.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.285+0000 7f7b1521c700 1 -- 172.21.15.40:0/1755205201 shutdown_connections 2024-09-13T18:00:23.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.285+0000 7f7b1521c700 1 --2- 172.21.15.40:0/1755205201 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7afc066730 0x7f7afc068bf0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.285+0000 7f7b1521c700 1 --2- 172.21.15.40:0/1755205201 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b100c8900 0x7f7b1006e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.285+0000 7f7b1521c700 1 --2- 172.21.15.40:0/1755205201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b100c8520 0x7f7b101ef4c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.285+0000 7f7b1521c700 1 --2- 172.21.15.40:0/1755205201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b100c6a00 0x7f7b10155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.285+0000 7f7b1521c700 1 -- 172.21.15.40:0/1755205201 >> 172.21.15.40:0/1755205201 conn(0x7f7b1006a570 msgr2=0x7f7b10158a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:23.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.285+0000 7f7b1521c700 1 -- 172.21.15.40:0/1755205201 shutdown_connections 2024-09-13T18:00:23.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.285+0000 7f7b1521c700 1 -- 172.21.15.40:0/1755205201 wait complete. 2024-09-13T18:00:23.305 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:00:23.305 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T18:00:23.305 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:00:23.306 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:00:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f736177b700 1 Processor -- start 2024-09-13T18:00:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f736177b700 1 -- start start 2024-09-13T18:00:23.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f736177b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f73540d8ad0 0x7f735404cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f736177b700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73540e1cc0 con 0x7f7354047ec0 2024-09-13T18:00:23.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f736177b700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73540e1f80 con 0x7f735404d240 2024-09-13T18:00:23.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f736177b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f735404d640 con 0x7f73540d8ad0 2024-09-13T18:00:23.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f735bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f73540d8ad0 0x7f735404cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f735bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f73540d8ad0 0x7f735404cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48308/0 (socket says 172.21.15.40:48308) 2024-09-13T18:00:23.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f735bfff700 1 -- 172.21.15.40:0/2872759341 learned_addr learned my addr 172.21.15.40:0/2872759341 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:23.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f735bfff700 1 -- 172.21.15.40:0/2872759341 >> v1:172.21.15.43:6789/0 conn(0x7f735404d240 legacy=0x7f73540477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f735bfff700 1 -- 172.21.15.40:0/2872759341 >> v1:172.21.15.40:6789/0 conn(0x7f7354047ec0 legacy=0x7f73540088d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:00:23.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f735bfff700 1 -- 172.21.15.40:0/2872759341 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73540e2260 con 0x7f73540d8ad0 2024-09-13T18:00:23.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f735bfff700 1 --2- 172.21.15.40:0/2872759341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73540d8ad0 0x7f735404cd00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7350009910 tx=0x7f7350009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5e196038311fd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f735affd700 1 -- 172.21.15.40:0/2872759341 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350004230 con 0x7f73540d8ad0 2024-09-13T18:00:23.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f735affd700 1 -- 172.21.15.40:0/2872759341 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7350004390 con 0x7f73540d8ad0 2024-09-13T18:00:23.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f735affd700 1 -- 172.21.15.40:0/2872759341 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350009e60 con 0x7f73540d8ad0 2024-09-13T18:00:23.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f736177b700 1 -- 172.21.15.40:0/2872759341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73540d8ad0 msgr2=0x7f735404cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f736177b700 1 --2- 172.21.15.40:0/2872759341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73540d8ad0 0x7f735404cd00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7350009910 tx=0x7f7350009c20 comp rx=0 tx=0).stop 2024-09-13T18:00:23.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f736177b700 1 -- 172.21.15.40:0/2872759341 shutdown_connections 2024-09-13T18:00:23.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f736177b700 1 --2- 172.21.15.40:0/2872759341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73540d8ad0 0x7f735404cd00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f736177b700 1 -- 172.21.15.40:0/2872759341 >> 172.21.15.40:0/2872759341 conn(0x7f735400dc40 msgr2=0x7f735400e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:23.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f736177b700 1 -- 172.21.15.40:0/2872759341 shutdown_connections 2024-09-13T18:00:23.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.413+0000 7f736177b700 1 -- 172.21.15.40:0/2872759341 wait complete. 2024-09-13T18:00:23.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.417+0000 7f736177b700 1 Processor -- start 2024-09-13T18:00:23.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.417+0000 7f736177b700 1 -- start start 2024-09-13T18:00:23.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.417+0000 7f736177b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7354047ec0 0x7f73541747b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.417+0000 7f736177b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f735404d240 0x7f735404b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.417+0000 7f736177b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73540d8ad0 0x7f735404be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.417+0000 7f736177b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7354177da0 con 0x7f7354047ec0 2024-09-13T18:00:23.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.417+0000 7f736177b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73541782c0 con 0x7f735404d240 2024-09-13T18:00:23.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.417+0000 7f736177b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73540cf2f0 con 0x7f73540d8ad0 2024-09-13T18:00:23.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.417+0000 7f735bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7354047ec0 0x7f73541747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.417+0000 7f735bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7354047ec0 0x7f73541747b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48322/0 (socket says 172.21.15.40:48322) 2024-09-13T18:00:23.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.417+0000 7f735bfff700 1 -- 172.21.15.40:0/2482188466 learned_addr learned my addr 172.21.15.40:0/2482188466 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:23.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.417+0000 7f7360f7a700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73540d8ad0 0x7f735404be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.421+0000 7f735b7fe700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f735404d240 0x7f735404b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.421+0000 7f735bfff700 1 -- 172.21.15.40:0/2482188466 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73540d8ad0 msgr2=0x7f735404be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.421+0000 7f735bfff700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f73540d8ad0 0x7f735404be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.421+0000 7f735bfff700 1 -- 172.21.15.40:0/2482188466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f735404d240 msgr2=0x7f735404b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.421+0000 7f735bfff700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f735404d240 0x7f735404b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.421+0000 7f735bfff700 1 -- 172.21.15.40:0/2482188466 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7350009570 con 0x7f7354047ec0 2024-09-13T18:00:23.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.421+0000 7f735bfff700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7354047ec0 0x7f73541747b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7350004000 tx=0x7f7350004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.421+0000 7f7358ff9700 1 -- 172.21.15.40:0/2482188466 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350004500 con 0x7f7354047ec0 2024-09-13T18:00:23.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.421+0000 7f7358ff9700 1 -- 172.21.15.40:0/2482188466 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73500036c0 con 0x7f7354047ec0 2024-09-13T18:00:23.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.421+0000 7f736177b700 1 -- 172.21.15.40:0/2482188466 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7354173ff0 con 0x7f7354047ec0 2024-09-13T18:00:23.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.421+0000 7f7358ff9700 1 -- 172.21.15.40:0/2482188466 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350003a70 con 0x7f7354047ec0 2024-09-13T18:00:23.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.421+0000 7f736177b700 1 -- 172.21.15.40:0/2482188466 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7354177fe0 con 0x7f7354047ec0 2024-09-13T18:00:23.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.421+0000 7f736177b700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f735404ad30 0x7f7354174180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.421+0000 7f736177b700 1 -- 172.21.15.40:0/2482188466 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7354178170 con 0x7f735404ad30 2024-09-13T18:00:23.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.421+0000 7f735b7fe700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f735404ad30 0x7f7354174180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.421+0000 7f735b7fe700 1 -- 172.21.15.40:0/2482188466 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f73540e2b00 con 0x7f735404ad30 2024-09-13T18:00:23.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.421+0000 7f7358ff9700 1 -- 172.21.15.40:0/2482188466 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7350003bd0 con 0x7f7354047ec0 2024-09-13T18:00:23.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.421+0000 7f735b7fe700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f735404ad30 0x7f7354174180 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7354168250 tx=0x7f734800e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.425+0000 7f7358ff9700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7340066730 0x7f7340068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.425+0000 7f7358ff9700 1 -- 172.21.15.40:0/2482188466 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f73500857e0 con 0x7f7354047ec0 2024-09-13T18:00:23.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.425+0000 7f7360f7a700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7340066730 0x7f7340068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.425+0000 7f7358ff9700 1 -- 172.21.15.40:0/2482188466 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73480052e0 con 0x7f735404ad30 2024-09-13T18:00:23.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.425+0000 7f7358ff9700 1 -- 172.21.15.40:0/2482188466 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7348005440 con 0x7f735404ad30 2024-09-13T18:00:23.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.425+0000 7f7358ff9700 1 -- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f735404ad30 msgr2=0x7f7354174180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.425+0000 7f7358ff9700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f735404ad30 0x7f7354174180 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7354168250 tx=0x7f734800e860 comp rx=0 tx=0).stop 2024-09-13T18:00:23.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.425+0000 7f7358ff9700 1 -- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f735404ad30 msgr2=0x7f7354174180 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:00:23.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.425+0000 7f7358ff9700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f735404ad30 0x7f7354174180 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.425+0000 7f7360f7a700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7340066730 0x7f7340068bf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f735c04ffd0 tx=0x7f735c065ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.437+0000 7f736177b700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73540cde70 0x7f73540ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.437+0000 7f736177b700 1 -- 172.21.15.40:0/2482188466 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73540074a0 con 0x7f73540cde70 2024-09-13T18:00:23.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.437+0000 7f735b7fe700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73540cde70 0x7f73540ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.437+0000 7f735b7fe700 1 -- 172.21.15.40:0/2482188466 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f73540e2820 con 0x7f73540cde70 2024-09-13T18:00:23.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.437+0000 7f735b7fe700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73540cde70 0x7f73540ce280 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7348016000 tx=0x7f7348000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.437+0000 7f7358ff9700 1 -- 172.21.15.40:0/2482188466 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7348021070 con 0x7f73540cde70 2024-09-13T18:00:23.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.437+0000 7f7358ff9700 1 -- 172.21.15.40:0/2482188466 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4784 (secure 0 0 0) 0x7f7348001220 con 0x7f73540cde70 2024-09-13T18:00:23.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.437+0000 7f7358ff9700 1 -- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73540cde70 msgr2=0x7f73540ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.437+0000 7f7358ff9700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73540cde70 0x7f73540ce280 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7348016000 tx=0x7f7348000ed0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.437+0000 7f7358ff9700 1 -- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73540cde70 msgr2=0x7f73540ce280 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:00:23.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.437+0000 7f7358ff9700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73540cde70 0x7f73540ce280 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.447 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:00:23.447 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:00:23.447 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:00:23.447 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:00:23.448 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 76, 2024-09-13T18:00:23.448 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:00:23.448 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:00:23.448 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:00:23.448 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:00:23.448 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:23.448 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 13, 2024-09-13T18:00:23.448 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:00:23.448 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:00:23.448 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:00:23.448 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:23.448 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:23.449 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:23.449 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:23.449 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:23.449 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:23.449 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:23.449 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:23.449 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:23.449 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:23.449 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:23.449 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:00:23.449 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:00:23.449 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:23.449 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:23.450 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:23.450 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:23.450 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:23.450 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:23.450 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:23.450 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:23.450 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:23.450 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:23.450 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:23.450 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.450 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:00:23.450 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:00:23.450 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:00:23.450 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:00:23.451 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:00:23.451 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:00:23.451 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:00:23.451 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:00:23.451 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:00:23.451 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:00:23.451 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:00:23.451 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:00:23.451 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:00:23.451 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:00:23.451 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:00:23.451 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:00:23.451 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:00:23.452 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:23.452 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:23.452 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:23.452 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:23.452 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:23.452 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:23.452 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:23.452 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:23.452 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:23.452 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:23.452 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:23.452 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:00:23.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.452 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:00:23.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.453 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:00:23.453 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:00:23.453 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:23.453 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:23.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.453 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:23.453 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:00:23.453 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:23.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.453 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:23.453 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:00:23.453 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:23.454 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.454 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:23.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.454 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:00:23.454 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:00:23.454 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:23.454 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:23.454 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:23.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.454 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.454 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:00:23.454 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:00:23.454 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:23.455 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:23.455 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.455 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:23.455 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:00:23.455 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:23.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.455 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.455 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:23.455 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:00:23.455 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:23.455 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.455 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:23.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.455 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:00:23.456 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:00:23.456 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:23.456 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:23.456 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:23.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.456 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:00:23.456 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:00:23.456 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:23.456 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:23.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.456 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:23.456 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:00:23.456 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:23.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.457 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:23.457 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:00:23.457 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:23.457 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.457 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:23.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.457 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:00:23.457 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:00:23.457 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:23.457 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:23.457 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:23.457 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.458 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:23.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.458 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:00:23.458 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:00:23.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.458 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:23.458 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:23.458 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:00:23.458 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.458 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:23.458 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:00:23.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.458 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:23.458 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:23.459 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:00:23.459 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.459 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:23.459 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:00:23.459 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.459 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:23.459 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:23.459 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:00:23.459 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.459 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:23.459 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:00:23.459 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.459 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:23.459 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:23.460 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:00:23.460 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.460 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:23.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.460 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:00:23.460 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:00:23.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.441+0000 7f736177b700 1 -- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7340066730 msgr2=0x7f7340068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.441+0000 7f736177b700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7340066730 0x7f7340068bf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f735c04ffd0 tx=0x7f735c065ea0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.441+0000 7f736177b700 1 -- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7354047ec0 msgr2=0x7f73541747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.441+0000 7f7360f7a700 1 -- 172.21.15.40:0/2482188466 reap_dead start 2024-09-13T18:00:23.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.441+0000 7f736177b700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7354047ec0 0x7f73541747b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7350004000 tx=0x7f7350004770 comp rx=0 tx=0).stop 2024-09-13T18:00:23.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.441+0000 7f736177b700 1 -- 172.21.15.40:0/2482188466 shutdown_connections 2024-09-13T18:00:23.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.441+0000 7f736177b700 1 --2- 172.21.15.40:0/2482188466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7354047ec0 0x7f73541747b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.441+0000 7f736177b700 1 -- 172.21.15.40:0/2482188466 >> 172.21.15.40:0/2482188466 conn(0x7f735400dc40 msgr2=0x7f7354008740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:23.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.441+0000 7f736177b700 1 -- 172.21.15.40:0/2482188466 shutdown_connections 2024-09-13T18:00:23.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.441+0000 7f736177b700 1 -- 172.21.15.40:0/2482188466 wait complete. 2024-09-13T18:00:23.462 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:00:23.462 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:00:23.463 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:00:23.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.577+0000 7f6777fff700 1 Processor -- start 2024-09-13T18:00:23.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.577+0000 7f6777fff700 1 -- start start 2024-09-13T18:00:23.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.577+0000 7f6777fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6778153750 0x7f67780c6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.577+0000 7f6777fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f67780c2980 0x7f67780c8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.577+0000 7f6777fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f677815cb30 con 0x7f67780c8b40 2024-09-13T18:00:23.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.577+0000 7f6777fff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67781597c0 con 0x7f6778153750 2024-09-13T18:00:23.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.577+0000 7f6777fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6778159aa0 con 0x7f67780c2980 2024-09-13T18:00:23.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f67777fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f67780c8b40 0x7f677815cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59770/0 (socket says 172.21.15.40:59770) 2024-09-13T18:00:23.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f67777fe700 1 -- 172.21.15.40:0/1845884413 learned_addr learned my addr 172.21.15.40:0/1845884413 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:00:23.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f6776ffd700 1 --2- 172.21.15.40:0/1845884413 >> v2:172.21.15.43:3300/0 conn(0x7f6778153750 0x7f67780c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f67767fc700 1 --2- 172.21.15.40:0/1845884413 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f67780c2980 0x7f67780c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f6775ffb700 1 -- 172.21.15.40:0/1845884413 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363433341 0 0) 0x7f677815cb30 con 0x7f67780c8b40 2024-09-13T18:00:23.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f67767fc700 1 -- 172.21.15.40:0/1845884413 >> v2:172.21.15.43:3300/0 conn(0x7f6778153750 msgr2=0x7f67780c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:00:23.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f67767fc700 1 --2- 172.21.15.40:0/1845884413 >> v2:172.21.15.43:3300/0 conn(0x7f6778153750 0x7f67780c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f67767fc700 1 -- 172.21.15.40:0/1845884413 >> v1:172.21.15.40:6789/0 conn(0x7f67780c8b40 legacy=0x7f677815cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f67767fc700 1 -- 172.21.15.40:0/1845884413 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6778159d80 con 0x7f67780c2980 2024-09-13T18:00:23.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f67767fc700 1 --2- 172.21.15.40:0/1845884413 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f67780c2980 0x7f67780c8600 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f676c009850 tx=0x7f676c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c9dc14cbf860f54 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f6775ffb700 1 -- 172.21.15.40:0/1845884413 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c011830 con 0x7f67780c2980 2024-09-13T18:00:23.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f6775ffb700 1 -- 172.21.15.40:0/1845884413 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676c011990 con 0x7f67780c2980 2024-09-13T18:00:23.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f6775ffb700 1 -- 172.21.15.40:0/1845884413 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c011d40 con 0x7f67780c2980 2024-09-13T18:00:23.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f6777fff700 1 -- 172.21.15.40:0/1845884413 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f67780c2980 msgr2=0x7f67780c8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f6777fff700 1 --2- 172.21.15.40:0/1845884413 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f67780c2980 0x7f67780c8600 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f676c009850 tx=0x7f676c009b60 comp rx=0 tx=0).stop 2024-09-13T18:00:23.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f6777fff700 1 -- 172.21.15.40:0/1845884413 shutdown_connections 2024-09-13T18:00:23.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f6777fff700 1 --2- 172.21.15.40:0/1845884413 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f67780c2980 0x7f67780c8600 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f6777fff700 1 --2- 172.21.15.40:0/1845884413 >> v2:172.21.15.43:3300/0 conn(0x7f6778153750 0x7f67780c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f6777fff700 1 -- 172.21.15.40:0/1845884413 >> 172.21.15.40:0/1845884413 conn(0x7f677806e620 msgr2=0x7f677806ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:23.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f6777fff700 1 -- 172.21.15.40:0/1845884413 shutdown_connections 2024-09-13T18:00:23.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f6777fff700 1 -- 172.21.15.40:0/1845884413 wait complete. 2024-09-13T18:00:23.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f6777fff700 1 Processor -- start 2024-09-13T18:00:23.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.581+0000 7f6777fff700 1 -- start start 2024-09-13T18:00:23.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f6777fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f67780c8b40 0x7f677814d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f6777fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6778153750 0x7f677814f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f6777fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f677814fcc0 0x7f677815b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f6777fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67781f2d70 con 0x7f677814fcc0 2024-09-13T18:00:23.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f6777fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67781f32a0 con 0x7f6778153750 2024-09-13T18:00:23.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f6777fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67781ef5e0 con 0x7f67780c8b40 2024-09-13T18:00:23.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f67777fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f677814fcc0 0x7f677815b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f67777fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f677814fcc0 0x7f677815b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48354/0 (socket says 172.21.15.40:48354) 2024-09-13T18:00:23.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f67777fe700 1 -- 172.21.15.40:0/232882018 learned_addr learned my addr 172.21.15.40:0/232882018 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:23.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f67767fc700 1 --2- 172.21.15.40:0/232882018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6778153750 0x7f677814f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f6776ffd700 1 --2- 172.21.15.40:0/232882018 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f67780c8b40 0x7f677814d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f67777fe700 1 -- 172.21.15.40:0/232882018 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f67780c8b40 msgr2=0x7f677814d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f67777fe700 1 --2- 172.21.15.40:0/232882018 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f67780c8b40 0x7f677814d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f67777fe700 1 -- 172.21.15.40:0/232882018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6778153750 msgr2=0x7f677814f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f67777fe700 1 --2- 172.21.15.40:0/232882018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6778153750 0x7f677814f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f67777fe700 1 -- 172.21.15.40:0/232882018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676c0094b0 con 0x7f677814fcc0 2024-09-13T18:00:23.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f67777fe700 1 --2- 172.21.15.40:0/232882018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f677814fcc0 0x7f677815b930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f677000cca0 tx=0x7f677000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f675ffff700 1 -- 172.21.15.40:0/232882018 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677000ab70 con 0x7f677814fcc0 2024-09-13T18:00:23.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f675ffff700 1 -- 172.21.15.40:0/232882018 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677000acd0 con 0x7f677814fcc0 2024-09-13T18:00:23.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f6777fff700 1 -- 172.21.15.40:0/232882018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677814d910 con 0x7f677814fcc0 2024-09-13T18:00:23.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f675ffff700 1 -- 172.21.15.40:0/232882018 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6770007310 con 0x7f677814fcc0 2024-09-13T18:00:23.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f6777fff700 1 -- 172.21.15.40:0/232882018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67781eef90 con 0x7f677814fcc0 2024-09-13T18:00:23.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f6777fff700 1 --2- 172.21.15.40:0/232882018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6758002a40 0x7f6758004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f6777fff700 1 -- 172.21.15.40:0/232882018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6758007f40 con 0x7f6758002a40 2024-09-13T18:00:23.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.585+0000 7f6776ffd700 1 --2- 172.21.15.40:0/232882018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6758002a40 0x7f6758004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.589+0000 7f6776ffd700 1 -- 172.21.15.40:0/232882018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6758008100 con 0x7f6758002a40 2024-09-13T18:00:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.589+0000 7f675ffff700 1 -- 172.21.15.40:0/232882018 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6770007470 con 0x7f677814fcc0 2024-09-13T18:00:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.589+0000 7f6776ffd700 1 --2- 172.21.15.40:0/232882018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6758002a40 0x7f6758004f00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f67680119d0 tx=0x7f6768011d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.589+0000 7f675ffff700 1 --2- 172.21.15.40:0/232882018 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6764066730 0x7f6764068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.589+0000 7f67767fc700 1 --2- 172.21.15.40:0/232882018 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6764066730 0x7f6764068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.589+0000 7f675ffff700 1 -- 172.21.15.40:0/232882018 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f67700842e0 con 0x7f677814fcc0 2024-09-13T18:00:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.589+0000 7f675ffff700 1 -- 172.21.15.40:0/232882018 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6768004240 con 0x7f6758002a40 2024-09-13T18:00:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.589+0000 7f675ffff700 1 -- 172.21.15.40:0/232882018 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f67680043a0 con 0x7f6758002a40 2024-09-13T18:00:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.589+0000 7f675ffff700 1 -- 172.21.15.40:0/232882018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6758002a40 msgr2=0x7f6758004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.589+0000 7f675ffff700 1 --2- 172.21.15.40:0/232882018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6758002a40 0x7f6758004f00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f67680119d0 tx=0x7f6768011d90 comp rx=0 tx=0).stop 2024-09-13T18:00:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.589+0000 7f675ffff700 1 -- 172.21.15.40:0/232882018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6758002a40 msgr2=0x7f6758004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:00:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.589+0000 7f675ffff700 1 --2- 172.21.15.40:0/232882018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6758002a40 0x7f6758004f00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.589+0000 7f67767fc700 1 --2- 172.21.15.40:0/232882018 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6764066730 0x7f6764068bf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f67780c9770 tx=0x7f676c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.601+0000 7f6777fff700 1 --2- 172.21.15.40:0/232882018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6758007020 0x7f6758007490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.601+0000 7f6777fff700 1 -- 172.21.15.40:0/232882018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675800a400 con 0x7f6758007020 2024-09-13T18:00:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.601+0000 7f6776ffd700 1 --2- 172.21.15.40:0/232882018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6758007020 0x7f6758007490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.601+0000 7f6776ffd700 1 -- 172.21.15.40:0/232882018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f675800a5c0 con 0x7f6758007020 2024-09-13T18:00:23.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.601+0000 7f6776ffd700 1 --2- 172.21.15.40:0/232882018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6758007020 0x7f6758007490 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f67680119d0 tx=0x7f6768004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.601+0000 7f675ffff700 1 -- 172.21.15.40:0/232882018 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676801e450 con 0x7f6758007020 2024-09-13T18:00:23.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.601+0000 7f675ffff700 1 -- 172.21.15.40:0/232882018 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7f67680283a0 con 0x7f6758007020 2024-09-13T18:00:23.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.601+0000 7f675ffff700 1 -- 172.21.15.40:0/232882018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6758007020 msgr2=0x7f6758007490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.601+0000 7f675ffff700 1 --2- 172.21.15.40:0/232882018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6758007020 0x7f6758007490 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f67680119d0 tx=0x7f6768004770 comp rx=0 tx=0).stop 2024-09-13T18:00:23.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.601+0000 7f675ffff700 1 -- 172.21.15.40:0/232882018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6758007020 msgr2=0x7f6758007490 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:00:23.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.601+0000 7f675ffff700 1 --2- 172.21.15.40:0/232882018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6758007020 0x7f6758007490 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.611 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:00:23.611 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:00:23.612 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:00:23.612 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:00:23.612 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 76, 2024-09-13T18:00:23.612 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:00:23.612 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:00:23.612 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:00:23.612 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:00:23.612 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:23.612 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 13, 2024-09-13T18:00:23.612 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:00:23.612 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:00:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:00:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:23.613 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:00:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:00:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.614 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:00:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:00:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:00:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:00:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:00:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:00:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:00:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:00:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:00:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:00:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:00:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:00:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:00:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:00:23.615 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:00:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:00:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:00:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:23.616 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:00:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:00:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:00:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:00:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:23.617 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:00:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:00:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:00:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:00:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:23.618 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:00:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:00:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:00:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.619 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:00:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:00:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:00:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.620 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:00:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:00:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:00:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:23.621 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:00:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:00:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:00:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:00:23.622 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:00:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:00:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:00:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:00:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:23.623 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:00:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:00:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:00:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:00:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:23.624 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:00:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:00:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.625 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:00:23.625 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:00:23.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.605+0000 7f6777fff700 1 -- 172.21.15.40:0/232882018 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6764066730 msgr2=0x7f6764068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.605+0000 7f6777fff700 1 --2- 172.21.15.40:0/232882018 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6764066730 0x7f6764068bf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f67780c9770 tx=0x7f676c008030 comp rx=0 tx=0).stop 2024-09-13T18:00:23.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.605+0000 7f6777fff700 1 -- 172.21.15.40:0/232882018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f677814fcc0 msgr2=0x7f677815b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.605+0000 7f6777fff700 1 --2- 172.21.15.40:0/232882018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f677814fcc0 0x7f677815b930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f677000cca0 tx=0x7f677000a060 comp rx=0 tx=0).stop 2024-09-13T18:00:23.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.605+0000 7f67777fe700 1 -- 172.21.15.40:0/232882018 reap_dead start 2024-09-13T18:00:23.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.605+0000 7f67777fe700 1 -- 172.21.15.40:0/232882018 reap_dead start 2024-09-13T18:00:23.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.605+0000 7f6777fff700 1 -- 172.21.15.40:0/232882018 shutdown_connections 2024-09-13T18:00:23.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.605+0000 7f6777fff700 1 -- 172.21.15.40:0/232882018 >> 172.21.15.40:0/232882018 conn(0x7f677806e620 msgr2=0x7f677806ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:23.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.605+0000 7f6777fff700 1 -- 172.21.15.40:0/232882018 shutdown_connections 2024-09-13T18:00:23.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.605+0000 7f6777fff700 1 -- 172.21.15.40:0/232882018 wait complete. 2024-09-13T18:00:23.629 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:00:23.629 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:00:23.629 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:00:23.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.749+0000 7f3aa1239700 1 Processor -- start 2024-09-13T18:00:23.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.749+0000 7f3aa1239700 1 -- start start 2024-09-13T18:00:23.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.749+0000 7f3aa1239700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c0cf580 0x7f3a9c15a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.749+0000 7f3aa1239700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3a9c0cc250 0x7f3a9c0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.749+0000 7f3aa1239700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a9c15a7a0 con 0x7f3a9c0cf1a0 2024-09-13T18:00:23.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.749+0000 7f3aa1239700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9c0cbdc0 con 0x7f3a9c0cc250 2024-09-13T18:00:23.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.749+0000 7f3aa1239700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c153f80 con 0x7f3a9c0cf580 2024-09-13T18:00:23.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.749+0000 7f3a9b7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c0cf580 0x7f3a9c15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.749+0000 7f3aa0a38700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3a9c0cc250 0x7f3a9c0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.749+0000 7f3aa0a38700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3a9c0cc250 0x7f3a9c0c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:33178/0 (socket says 172.21.15.40:33178) 2024-09-13T18:00:23.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.749+0000 7f3aa0a38700 1 -- 172.21.15.40:0/3411508100 learned_addr learned my addr 172.21.15.40:0/3411508100 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:23.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.749+0000 7f3a9b7fe700 1 -- 172.21.15.40:0/3411508100 >> v2:172.21.15.43:3300/0 conn(0x7f3a9c0cc250 msgr2=0x7f3a9c0c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:00:23.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.749+0000 7f3a9b7fe700 1 --2- 172.21.15.40:0/3411508100 >> v2:172.21.15.43:3300/0 conn(0x7f3a9c0cc250 0x7f3a9c0c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.749+0000 7f3a9b7fe700 1 -- 172.21.15.40:0/3411508100 >> v1:172.21.15.43:6789/0 conn(0x7f3a9c0cf1a0 legacy=0x7f3a9c0c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.749+0000 7f3a9b7fe700 1 -- 172.21.15.40:0/3411508100 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c0caed0 con 0x7f3a9c0cf580 2024-09-13T18:00:23.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.749+0000 7f3a9b7fe700 1 --2- 172.21.15.40:0/3411508100 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c0cf580 0x7f3a9c15a260 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3a90009850 tx=0x7f3a90009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb890dfd57c99155 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3a9affd700 1 -- 172.21.15.40:0/3411508100 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a90011830 con 0x7f3a9c0cf580 2024-09-13T18:00:23.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3a9affd700 1 -- 172.21.15.40:0/3411508100 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a90011990 con 0x7f3a9c0cf580 2024-09-13T18:00:23.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3a9affd700 1 -- 172.21.15.40:0/3411508100 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a90011d40 con 0x7f3a9c0cf580 2024-09-13T18:00:23.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3aa1239700 1 -- 172.21.15.40:0/3411508100 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c0cf580 msgr2=0x7f3a9c15a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3aa1239700 1 --2- 172.21.15.40:0/3411508100 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c0cf580 0x7f3a9c15a260 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3a90009850 tx=0x7f3a90009b60 comp rx=0 tx=0).stop 2024-09-13T18:00:23.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3aa1239700 1 -- 172.21.15.40:0/3411508100 shutdown_connections 2024-09-13T18:00:23.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3aa1239700 1 --2- 172.21.15.40:0/3411508100 >> v2:172.21.15.43:3300/0 conn(0x7f3a9c0cc250 0x7f3a9c0c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3aa1239700 1 --2- 172.21.15.40:0/3411508100 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c0cf580 0x7f3a9c15a260 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3aa1239700 1 -- 172.21.15.40:0/3411508100 >> 172.21.15.40:0/3411508100 conn(0x7f3a9c066090 msgr2=0x7f3a9c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:23.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3aa1239700 1 -- 172.21.15.40:0/3411508100 shutdown_connections 2024-09-13T18:00:23.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3aa1239700 1 -- 172.21.15.40:0/3411508100 wait complete. 2024-09-13T18:00:23.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3aa1239700 1 Processor -- start 2024-09-13T18:00:23.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3aa1239700 1 -- start start 2024-09-13T18:00:23.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3aa1239700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c0cc250 0x7f3a9c156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3aa1239700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a9c0cf1a0 0x7f3a9c1547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3aa1239700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a9c0cf580 0x7f3a9c159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3aa1239700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c0b2fd0 con 0x7f3a9c0cf1a0 2024-09-13T18:00:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3aa1239700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c154310 con 0x7f3a9c0cf580 2024-09-13T18:00:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3aa1239700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c1571d0 con 0x7f3a9c0cc250 2024-09-13T18:00:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3a9b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a9c0cf1a0 0x7f3a9c1547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3a9b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a9c0cf1a0 0x7f3a9c1547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48370/0 (socket says 172.21.15.40:48370) 2024-09-13T18:00:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3a9b7fe700 1 -- 172.21.15.40:0/3731007604 learned_addr learned my addr 172.21.15.40:0/3731007604 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3a9bfff700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c0cc250 0x7f3a9c156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.753+0000 7f3aa0a38700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a9c0cf580 0x7f3a9c159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.757+0000 7f3a9b7fe700 1 -- 172.21.15.40:0/3731007604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c0cc250 msgr2=0x7f3a9c156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.757+0000 7f3a9b7fe700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c0cc250 0x7f3a9c156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.757+0000 7f3a9b7fe700 1 -- 172.21.15.40:0/3731007604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a9c0cf580 msgr2=0x7f3a9c159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.757+0000 7f3a9b7fe700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a9c0cf580 0x7f3a9c159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.757+0000 7f3a9b7fe700 1 -- 172.21.15.40:0/3731007604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a900094b0 con 0x7f3a9c0cf1a0 2024-09-13T18:00:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.757+0000 7f3a9b7fe700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a9c0cf1a0 0x7f3a9c1547a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3a90014040 tx=0x7f3a90004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.757+0000 7f3a98ff9700 1 -- 172.21.15.40:0/3731007604 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a900036c0 con 0x7f3a9c0cf1a0 2024-09-13T18:00:23.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.757+0000 7f3a98ff9700 1 -- 172.21.15.40:0/3731007604 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9001e070 con 0x7f3a9c0cf1a0 2024-09-13T18:00:23.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.757+0000 7f3aa1239700 1 -- 172.21.15.40:0/3731007604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9c0b2730 con 0x7f3a9c0cf1a0 2024-09-13T18:00:23.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.757+0000 7f3a98ff9700 1 -- 172.21.15.40:0/3731007604 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a90003a80 con 0x7f3a9c0cf1a0 2024-09-13T18:00:23.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.757+0000 7f3aa1239700 1 -- 172.21.15.40:0/3731007604 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9c0b2cf0 con 0x7f3a9c0cf1a0 2024-09-13T18:00:23.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.757+0000 7f3aa1239700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c0b3ae0 0x7f3a9c14d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.757+0000 7f3aa1239700 1 -- 172.21.15.40:0/3731007604 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c074750 con 0x7f3a9c0b3ae0 2024-09-13T18:00:23.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.757+0000 7f3a9bfff700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c0b3ae0 0x7f3a9c14d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.757+0000 7f3a98ff9700 1 -- 172.21.15.40:0/3731007604 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3a90003be0 con 0x7f3a9c0cf1a0 2024-09-13T18:00:23.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.757+0000 7f3a9bfff700 1 -- 172.21.15.40:0/3731007604 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a9c0c5f20 con 0x7f3a9c0b3ae0 2024-09-13T18:00:23.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.757+0000 7f3a9bfff700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c0b3ae0 0x7f3a9c14d1d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00f770 tx=0x7f3a8c00fb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.761+0000 7f3a98ff9700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3a88066730 0x7f3a88068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.761+0000 7f3aa0a38700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3a88066730 0x7f3a88068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.761+0000 7f3a98ff9700 1 -- 172.21.15.40:0/3731007604 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3a90020070 con 0x7f3a9c0cf1a0 2024-09-13T18:00:23.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.761+0000 7f3a98ff9700 1 -- 172.21.15.40:0/3731007604 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c0052e0 con 0x7f3a9c0b3ae0 2024-09-13T18:00:23.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.761+0000 7f3a98ff9700 1 -- 172.21.15.40:0/3731007604 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3a8c005440 con 0x7f3a9c0b3ae0 2024-09-13T18:00:23.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.761+0000 7f3a98ff9700 1 -- 172.21.15.40:0/3731007604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c0b3ae0 msgr2=0x7f3a9c14d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.761+0000 7f3a98ff9700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c0b3ae0 0x7f3a9c14d1d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00f770 tx=0x7f3a8c00fb30 comp rx=0 tx=0).stop 2024-09-13T18:00:23.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.761+0000 7f3a98ff9700 1 -- 172.21.15.40:0/3731007604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c0b3ae0 msgr2=0x7f3a9c14d1d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:00:23.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.761+0000 7f3a98ff9700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c0b3ae0 0x7f3a9c14d1d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.761+0000 7f3aa0a38700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3a88066730 0x7f3a88068bf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3a940113e0 tx=0x7f3a9400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.769+0000 7f3aa1239700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c14d710 0x7f3a9c152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.769+0000 7f3aa1239700 1 -- 172.21.15.40:0/3731007604 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c0cb770 con 0x7f3a9c14d710 2024-09-13T18:00:23.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.769+0000 7f3a9bfff700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c14d710 0x7f3a9c152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.773+0000 7f3a9bfff700 1 -- 172.21.15.40:0/3731007604 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3a9c0cb490 con 0x7f3a9c14d710 2024-09-13T18:00:23.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.773+0000 7f3a9bfff700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c14d710 0x7f3a9c152620 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00f770 tx=0x7f3a8c001120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.773+0000 7f3a98ff9700 1 -- 172.21.15.40:0/3731007604 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c023070 con 0x7f3a9c14d710 2024-09-13T18:00:23.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.773+0000 7f3a98ff9700 1 -- 172.21.15.40:0/3731007604 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4450 (secure 0 0 0) 0x7f3a8c027410 con 0x7f3a9c14d710 2024-09-13T18:00:23.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.773+0000 7f3a98ff9700 1 -- 172.21.15.40:0/3731007604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c14d710 msgr2=0x7f3a9c152620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.773+0000 7f3a98ff9700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c14d710 0x7f3a9c152620 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00f770 tx=0x7f3a8c001120 comp rx=0 tx=0).stop 2024-09-13T18:00:23.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.773+0000 7f3a98ff9700 1 -- 172.21.15.40:0/3731007604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c14d710 msgr2=0x7f3a9c152620 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:00:23.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.773+0000 7f3a98ff9700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9c14d710 0x7f3a9c152620 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.781 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:00:23.781 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:00:23.781 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:00:23.782 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:00:23.782 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 76, 2024-09-13T18:00:23.782 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:00:23.782 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:00:23.782 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:00:23.782 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:00:23.782 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:23.782 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 13, 2024-09-13T18:00:23.782 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:00:23.782 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:00:23.782 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:00:23.782 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:23.782 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:23.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:23.783 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:23.783 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:23.783 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:23.783 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:23.783 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:23.783 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:23.783 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:23.783 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:23.783 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:00:23.783 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:00:23.783 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:23.783 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:23.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:23.783 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:23.784 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:23.784 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:23.784 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:23.784 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:23.784 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:23.784 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:23.784 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:23.784 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.784 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:00:23.784 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:00:23.784 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:00:23.784 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:00:23.784 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:00:23.785 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:00:23.785 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:00:23.785 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:00:23.785 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:00:23.785 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:00:23.785 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:00:23.785 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:00:23.785 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:00:23.785 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:00:23.785 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:00:23.785 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:00:23.785 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:00:23.785 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:23.785 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:23.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:23.786 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:23.786 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:23.786 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:23.786 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:23.786 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:23.786 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:23.786 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:23.786 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:23.786 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:00:23.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:00:23.786 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.786 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:00:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:00:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:00:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:00:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.787 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:23.788 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.788 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:00:23.788 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:00:23.788 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:23.788 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:23.788 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:23.788 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.788 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.788 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:00:23.788 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:00:23.789 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:23.789 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:23.789 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.789 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:23.789 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:00:23.789 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:23.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.789 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.789 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:23.789 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:00:23.789 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:23.789 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.789 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:23.790 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.790 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:00:23.790 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:00:23.790 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:23.790 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:23.790 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:23.790 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.790 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.790 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:00:23.790 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:00:23.790 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:23.790 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:23.790 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.790 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:23.791 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:00:23.791 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:23.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.791 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.791 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:23.791 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:00:23.791 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:23.791 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.791 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:23.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.791 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:00:23.791 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:00:23.791 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:23.791 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:23.792 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:23.792 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.792 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:23.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.792 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:00:23.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:00:23.792 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.792 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:23.792 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:23.792 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:00:23.792 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.792 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:23.792 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:00:23.792 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:23.793 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:23.793 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:23.793 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:00:23.793 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:23.793 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:23.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:23.793 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:00:23.793 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:00:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.773+0000 7f3aa1239700 1 -- 172.21.15.40:0/3731007604 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3a88066730 msgr2=0x7f3a88068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.773+0000 7f3aa1239700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3a88066730 0x7f3a88068bf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3a940113e0 tx=0x7f3a9400b040 comp rx=0 tx=0).stop 2024-09-13T18:00:23.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.773+0000 7f3aa0a38700 1 -- 172.21.15.40:0/3731007604 reap_dead start 2024-09-13T18:00:23.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.773+0000 7f3aa1239700 1 -- 172.21.15.40:0/3731007604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a9c0cf1a0 msgr2=0x7f3a9c1547a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.773+0000 7f3aa1239700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a9c0cf1a0 0x7f3a9c1547a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3a90014040 tx=0x7f3a90004770 comp rx=0 tx=0).stop 2024-09-13T18:00:23.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.773+0000 7f3aa1239700 1 -- 172.21.15.40:0/3731007604 shutdown_connections 2024-09-13T18:00:23.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.773+0000 7f3aa1239700 1 --2- 172.21.15.40:0/3731007604 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a9c0cf1a0 0x7f3a9c1547a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.773+0000 7f3aa1239700 1 -- 172.21.15.40:0/3731007604 >> 172.21.15.40:0/3731007604 conn(0x7f3a9c066090 msgr2=0x7f3a9c0cd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:23.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.773+0000 7f3aa1239700 1 -- 172.21.15.40:0/3731007604 shutdown_connections 2024-09-13T18:00:23.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.773+0000 7f3aa1239700 1 -- 172.21.15.40:0/3731007604 wait complete. 2024-09-13T18:00:23.795 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:00:23.796 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-13T18:00:23.796 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T18:00:23.796 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-13T18:00:23.796 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-13T18:00:23.796 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T18:00:23.796 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:00:23.826 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T18:00:23.826 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T18:00:23.826 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:00:23.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.933+0000 7f370d4a9700 1 Processor -- start 2024-09-13T18:00:23.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.933+0000 7f370d4a9700 1 -- start start 2024-09-13T18:00:23.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.933+0000 7f370d4a9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3708159840 0x7f3708159c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.933+0000 7f370d4a9700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f37080c4920 0x7f370814fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.933+0000 7f3707fff700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3708159840 msgr2=0x7f3708159c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:00:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.933+0000 7f3707fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3708159840 0x7f3708159c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.933+0000 7f37077fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f370815a160 0x7f37080c4210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59778/0 (socket says 172.21.15.40:59778) 2024-09-13T18:00:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.933+0000 7f37077fe700 1 -- 172.21.15.40:0/2204983379 learned_addr learned my addr 172.21.15.40:0/2204983379 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:00:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.933+0000 7f370d4a9700 1 -- 172.21.15.40:0/2204983379 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37080663d0 con 0x7f370815a160 2024-09-13T18:00:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.933+0000 7f370d4a9700 1 -- 172.21.15.40:0/2204983379 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3708070d40 con 0x7f37080c4920 2024-09-13T18:00:23.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.933+0000 7f370d4a9700 1 -- 172.21.15.40:0/2204983379 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3708070fe0 con 0x7f3708159840 2024-09-13T18:00:23.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.937+0000 7f370cca8700 1 --2- 172.21.15.40:0/2204983379 >> v2:172.21.15.43:3300/0 conn(0x7f37080c4920 0x7f370814fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.937+0000 7f370cca8700 1 -- 172.21.15.40:0/2204983379 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3708159840 msgr2=0x7f3708159c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:00:23.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.937+0000 7f370cca8700 1 --2- 172.21.15.40:0/2204983379 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3708159840 0x7f3708159c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.937+0000 7f370cca8700 1 -- 172.21.15.40:0/2204983379 >> v1:172.21.15.40:6789/0 conn(0x7f370815a160 legacy=0x7f37080c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.937+0000 7f370cca8700 1 -- 172.21.15.40:0/2204983379 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37080712c0 con 0x7f37080c4920 2024-09-13T18:00:23.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.937+0000 7f370cca8700 1 --2- 172.21.15.40:0/2204983379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37080c4920 0x7f370814fad0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3700009d90 tx=0x7f370000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d8917d530d5915a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.941+0000 7f3706ffd700 1 -- 172.21.15.40:0/2204983379 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370000e070 con 0x7f37080c4920 2024-09-13T18:00:23.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.941+0000 7f3706ffd700 1 -- 172.21.15.40:0/2204983379 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370000bad0 con 0x7f37080c4920 2024-09-13T18:00:23.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.941+0000 7f3706ffd700 1 -- 172.21.15.40:0/2204983379 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370000be80 con 0x7f37080c4920 2024-09-13T18:00:23.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.941+0000 7f370d4a9700 1 -- 172.21.15.40:0/2204983379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37080c4920 msgr2=0x7f370814fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.941+0000 7f370d4a9700 1 --2- 172.21.15.40:0/2204983379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37080c4920 0x7f370814fad0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3700009d90 tx=0x7f370000b330 comp rx=0 tx=0).stop 2024-09-13T18:00:23.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.941+0000 7f370d4a9700 1 -- 172.21.15.40:0/2204983379 shutdown_connections 2024-09-13T18:00:23.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.941+0000 7f370d4a9700 1 --2- 172.21.15.40:0/2204983379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37080c4920 0x7f370814fad0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.941+0000 7f370d4a9700 1 --2- 172.21.15.40:0/2204983379 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3708159840 0x7f3708159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.941+0000 7f370d4a9700 1 -- 172.21.15.40:0/2204983379 >> 172.21.15.40:0/2204983379 conn(0x7f370806a390 msgr2=0x7f370806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:23.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.941+0000 7f370d4a9700 1 -- 172.21.15.40:0/2204983379 shutdown_connections 2024-09-13T18:00:23.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.941+0000 7f370d4a9700 1 -- 172.21.15.40:0/2204983379 wait complete. 2024-09-13T18:00:23.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.941+0000 7f370d4a9700 1 Processor -- start 2024-09-13T18:00:23.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.941+0000 7f370d4a9700 1 -- start start 2024-09-13T18:00:23.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f370d4a9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37080c4920 0x7f37080c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f370d4a9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3708159840 0x7f37081e3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f370d4a9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f370815a160 0x7f37081e3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f370d4a9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37081f3290 con 0x7f37080c4920 2024-09-13T18:00:23.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f370d4a9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3708149d30 con 0x7f3708159840 2024-09-13T18:00:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f370d4a9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37081f2650 con 0x7f370815a160 2024-09-13T18:00:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f370cca8700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f370815a160 msgr2=0x7f37081e3ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:00:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f3707fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37080c4920 0x7f37080c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f370cca8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f370815a160 0x7f37081e3ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f3707fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37080c4920 0x7f37080c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48372/0 (socket says 172.21.15.40:48372) 2024-09-13T18:00:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f3707fff700 1 -- 172.21.15.40:0/2800416674 learned_addr learned my addr 172.21.15.40:0/2800416674 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f37077fe700 1 --2- 172.21.15.40:0/2800416674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3708159840 0x7f37081e3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f3707fff700 1 -- 172.21.15.40:0/2800416674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f370815a160 msgr2=0x7f37081e3ff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:00:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f3707fff700 1 --2- 172.21.15.40:0/2800416674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f370815a160 0x7f37081e3ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f3707fff700 1 -- 172.21.15.40:0/2800416674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3708159840 msgr2=0x7f37081e3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f3707fff700 1 --2- 172.21.15.40:0/2800416674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3708159840 0x7f37081e3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f3707fff700 1 -- 172.21.15.40:0/2800416674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3700009a70 con 0x7f37080c4920 2024-09-13T18:00:23.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f3707fff700 1 --2- 172.21.15.40:0/2800416674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37080c4920 0x7f37080c72e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f36f800b780 tx=0x7f36f800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f3704ff9700 1 -- 172.21.15.40:0/2800416674 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f8003e70 con 0x7f37080c4920 2024-09-13T18:00:23.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f3704ff9700 1 -- 172.21.15.40:0/2800416674 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f8005070 con 0x7f37080c4920 2024-09-13T18:00:23.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f370d4a9700 1 -- 172.21.15.40:0/2800416674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37081f28f0 con 0x7f37080c4920 2024-09-13T18:00:23.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f3704ff9700 1 -- 172.21.15.40:0/2800416674 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f8005420 con 0x7f37080c4920 2024-09-13T18:00:23.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f370d4a9700 1 -- 172.21.15.40:0/2800416674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37081f2eb0 con 0x7f37080c4920 2024-09-13T18:00:23.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.945+0000 7f370d4a9700 1 -- 172.21.15.40:0/2800416674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3708154d90 con 0x7f37080c4920 2024-09-13T18:00:23.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.949+0000 7f3704ff9700 1 -- 172.21.15.40:0/2800416674 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f36f8005580 con 0x7f37080c4920 2024-09-13T18:00:23.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.949+0000 7f3704ff9700 1 --2- 172.21.15.40:0/2800416674 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f36f40666e0 0x7f36f4068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:23.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.949+0000 7f37077fe700 1 --2- 172.21.15.40:0/2800416674 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f36f40666e0 0x7f36f4068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:23.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.949+0000 7f3704ff9700 1 -- 172.21.15.40:0/2800416674 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f36f8083f70 con 0x7f37080c4920 2024-09-13T18:00:23.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.949+0000 7f37077fe700 1 --2- 172.21.15.40:0/2800416674 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f36f40666e0 0x7f36f4068ba0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f37081542f0 tx=0x7f36fc007ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:23.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:23.953+0000 7f3704ff9700 1 -- 172.21.15.40:0/2800416674 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f36f80b5db0 con 0x7f37080c4920 2024-09-13T18:00:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:24.197+0000 7f370d4a9700 1 -- 172.21.15.40:0/2800416674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3708159c20 con 0x7f37080c4920 2024-09-13T18:00:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:24.197+0000 7f3704ff9700 1 -- 172.21.15.40:0/2800416674 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f36f8050e90 con 0x7f37080c4920 2024-09-13T18:00:24.206 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":76,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:00:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:24.201+0000 7f370d4a9700 1 -- 172.21.15.40:0/2800416674 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f36f40666e0 msgr2=0x7f36f4068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:24.201+0000 7f370d4a9700 1 --2- 172.21.15.40:0/2800416674 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f36f40666e0 0x7f36f4068ba0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f37081542f0 tx=0x7f36fc007ed0 comp rx=0 tx=0).stop 2024-09-13T18:00:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:24.201+0000 7f370d4a9700 1 -- 172.21.15.40:0/2800416674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37080c4920 msgr2=0x7f37080c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:24.201+0000 7f370d4a9700 1 --2- 172.21.15.40:0/2800416674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37080c4920 0x7f37080c72e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f36f800b780 tx=0x7f36f800ba90 comp rx=0 tx=0).stop 2024-09-13T18:00:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:24.201+0000 7f370d4a9700 1 -- 172.21.15.40:0/2800416674 shutdown_connections 2024-09-13T18:00:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:24.201+0000 7f370d4a9700 1 --2- 172.21.15.40:0/2800416674 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f36f40666e0 0x7f36f4068ba0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:24.201+0000 7f370d4a9700 1 --2- 172.21.15.40:0/2800416674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f370815a160 0x7f37081e3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:24.201+0000 7f370d4a9700 1 --2- 172.21.15.40:0/2800416674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3708159840 0x7f37081e3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:24.201+0000 7f370d4a9700 1 --2- 172.21.15.40:0/2800416674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37080c4920 0x7f37080c72e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:24.201+0000 7f370d4a9700 1 -- 172.21.15.40:0/2800416674 >> 172.21.15.40:0/2800416674 conn(0x7f370806a390 msgr2=0x7f370814f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:24.201+0000 7f370d4a9700 1 -- 172.21.15.40:0/2800416674 shutdown_connections 2024-09-13T18:00:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:24.201+0000 7f370d4a9700 1 -- 172.21.15.40:0/2800416674 wait complete. 2024-09-13T18:00:27.228 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:00:27.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b57fff700 1 Processor -- start 2024-09-13T18:00:27.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b57fff700 1 -- start start 2024-09-13T18:00:27.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b57fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5b580c4210 0x7f5b580c4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b57fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5b5814fad0 0x7f5b5814ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b57fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b580663d0 con 0x7f5b58159840 2024-09-13T18:00:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b57fff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b58070d40 con 0x7f5b5814fad0 2024-09-13T18:00:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b57fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5815a330 con 0x7f5b580c4210 2024-09-13T18:00:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b577fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5b5814fad0 0x7f5b5814ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b567fc700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5b580c4210 msgr2=0x7f5b580c4610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:00:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b567fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5b580c4210 0x7f5b580c4610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b577fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5b5814fad0 0x7f5b5814ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48384/0 (socket says 172.21.15.40:48384) 2024-09-13T18:00:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b577fe700 1 -- 172.21.15.40:0/2869588000 learned_addr learned my addr 172.21.15.40:0/2869588000 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b577fe700 1 -- 172.21.15.40:0/2869588000 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5b580c4210 msgr2=0x7f5b580c4610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:00:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b577fe700 1 --2- 172.21.15.40:0/2869588000 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5b580c4210 0x7f5b580c4610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b577fe700 1 -- 172.21.15.40:0/2869588000 >> v1:172.21.15.40:6789/0 conn(0x7f5b58159840 legacy=0x7f5b58159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:27.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b577fe700 1 -- 172.21.15.40:0/2869588000 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b58070fe0 con 0x7f5b5814fad0 2024-09-13T18:00:27.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b577fe700 1 --2- 172.21.15.40:0/2869588000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b5814fad0 0x7f5b5814ff80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000d150 tx=0x7f5b5000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab85c003061544e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:27.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b55ffb700 1 -- 172.21.15.40:0/2869588000 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b50010070 con 0x7f5b5814fad0 2024-09-13T18:00:27.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b55ffb700 1 -- 172.21.15.40:0/2869588000 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5000ddc0 con 0x7f5b5814fad0 2024-09-13T18:00:27.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b55ffb700 1 -- 172.21.15.40:0/2869588000 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5000b310 con 0x7f5b5814fad0 2024-09-13T18:00:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b57fff700 1 -- 172.21.15.40:0/2869588000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b5814fad0 msgr2=0x7f5b5814ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b57fff700 1 --2- 172.21.15.40:0/2869588000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b5814fad0 0x7f5b5814ff80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000d150 tx=0x7f5b5000d460 comp rx=0 tx=0).stop 2024-09-13T18:00:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b57fff700 1 -- 172.21.15.40:0/2869588000 shutdown_connections 2024-09-13T18:00:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b57fff700 1 --2- 172.21.15.40:0/2869588000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b5814fad0 0x7f5b5814ff80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b57fff700 1 --2- 172.21.15.40:0/2869588000 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5b580c4210 0x7f5b580c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b57fff700 1 -- 172.21.15.40:0/2869588000 >> 172.21.15.40:0/2869588000 conn(0x7f5b5806a390 msgr2=0x7f5b5806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b57fff700 1 -- 172.21.15.40:0/2869588000 shutdown_connections 2024-09-13T18:00:27.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.341+0000 7f5b57fff700 1 -- 172.21.15.40:0/2869588000 wait complete. 2024-09-13T18:00:27.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b57fff700 1 Processor -- start 2024-09-13T18:00:27.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b57fff700 1 -- start start 2024-09-13T18:00:27.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b57fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b580c4210 0x7f5b580ca510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:27.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b57fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b5814fad0 0x7f5b580caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:27.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b57fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5b58159840 0x7f5b58157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:27.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b57fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b581df4e0 con 0x7f5b580c4210 2024-09-13T18:00:27.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b57fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b581eee80 con 0x7f5b5814fad0 2024-09-13T18:00:27.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b57fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b581571a0 con 0x7f5b58159840 2024-09-13T18:00:27.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b56ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b580c4210 0x7f5b580ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:27.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b577fe700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5b58159840 msgr2=0x7f5b58157f40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:00:27.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b577fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5b58159840 0x7f5b58157f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:27.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b56ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b580c4210 0x7f5b580ca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48394/0 (socket says 172.21.15.40:48394) 2024-09-13T18:00:27.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b56ffd700 1 -- 172.21.15.40:0/2166755065 learned_addr learned my addr 172.21.15.40:0/2166755065 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:27.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b567fc700 1 --2- 172.21.15.40:0/2166755065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b5814fad0 0x7f5b580caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:27.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b56ffd700 1 -- 172.21.15.40:0/2166755065 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5b58159840 msgr2=0x7f5b58157f40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:00:27.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b56ffd700 1 --2- 172.21.15.40:0/2166755065 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5b58159840 0x7f5b58157f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:27.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b56ffd700 1 -- 172.21.15.40:0/2166755065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b5814fad0 msgr2=0x7f5b580caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:27.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b56ffd700 1 --2- 172.21.15.40:0/2166755065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b5814fad0 0x7f5b580caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:27.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b56ffd700 1 -- 172.21.15.40:0/2166755065 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5000bd40 con 0x7f5b580c4210 2024-09-13T18:00:27.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b56ffd700 1 --2- 172.21.15.40:0/2166755065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b580c4210 0x7f5b580ca510 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c500 tx=0x7f5b4800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:27.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b3ffff700 1 -- 172.21.15.40:0/2166755065 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b480052d0 con 0x7f5b580c4210 2024-09-13T18:00:27.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b3ffff700 1 -- 172.21.15.40:0/2166755065 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b48005430 con 0x7f5b580c4210 2024-09-13T18:00:27.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b57fff700 1 -- 172.21.15.40:0/2166755065 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b581ef100 con 0x7f5b580c4210 2024-09-13T18:00:27.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b3ffff700 1 -- 172.21.15.40:0/2166755065 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b480055a0 con 0x7f5b580c4210 2024-09-13T18:00:27.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.345+0000 7f5b57fff700 1 -- 172.21.15.40:0/2166755065 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b58157740 con 0x7f5b580c4210 2024-09-13T18:00:27.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.349+0000 7f5b57fff700 1 -- 172.21.15.40:0/2166755065 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b580c3980 con 0x7f5b580c4210 2024-09-13T18:00:27.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.349+0000 7f5b3ffff700 1 -- 172.21.15.40:0/2166755065 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5b480038e0 con 0x7f5b580c4210 2024-09-13T18:00:27.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.349+0000 7f5b3ffff700 1 --2- 172.21.15.40:0/2166755065 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5b44066710 0x7f5b44068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:27.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.349+0000 7f5b567fc700 1 --2- 172.21.15.40:0/2166755065 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5b44066710 0x7f5b44068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:27.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.349+0000 7f5b3ffff700 1 -- 172.21.15.40:0/2166755065 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5b480838e0 con 0x7f5b580c4210 2024-09-13T18:00:27.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.349+0000 7f5b567fc700 1 --2- 172.21.15.40:0/2166755065 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5b44066710 0x7f5b44068bd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5b580cb250 tx=0x7f5b4c003c30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:27.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.357+0000 7f5b3ffff700 1 -- 172.21.15.40:0/2166755065 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5b580c3980 con 0x7f5b580c4210 2024-09-13T18:00:27.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.632+0000 7f5b57fff700 1 -- 172.21.15.40:0/2166755065 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f5b580c3980 con 0x7f5b580c4210 2024-09-13T18:00:27.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.632+0000 7f5b3ffff700 1 -- 172.21.15.40:0/2166755065 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f5b48017620 con 0x7f5b580c4210 2024-09-13T18:00:27.643 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":76,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:00:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.636+0000 7f5b57fff700 1 -- 172.21.15.40:0/2166755065 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5b44066710 msgr2=0x7f5b44068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.636+0000 7f5b57fff700 1 --2- 172.21.15.40:0/2166755065 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5b44066710 0x7f5b44068bd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5b580cb250 tx=0x7f5b4c003c30 comp rx=0 tx=0).stop 2024-09-13T18:00:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.636+0000 7f5b57fff700 1 -- 172.21.15.40:0/2166755065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b580c4210 msgr2=0x7f5b580ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.636+0000 7f5b57fff700 1 --2- 172.21.15.40:0/2166755065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b580c4210 0x7f5b580ca510 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800c500 tx=0x7f5b4800c810 comp rx=0 tx=0).stop 2024-09-13T18:00:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.636+0000 7f5b57fff700 1 -- 172.21.15.40:0/2166755065 shutdown_connections 2024-09-13T18:00:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.636+0000 7f5b57fff700 1 --2- 172.21.15.40:0/2166755065 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5b44066710 0x7f5b44068bd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:27.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.636+0000 7f5b57fff700 1 --2- 172.21.15.40:0/2166755065 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5b58159840 0x7f5b58157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:27.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.636+0000 7f5b57fff700 1 --2- 172.21.15.40:0/2166755065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b5814fad0 0x7f5b580caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:27.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.636+0000 7f5b57fff700 1 --2- 172.21.15.40:0/2166755065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5b580c4210 0x7f5b580ca510 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:27.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.636+0000 7f5b57fff700 1 -- 172.21.15.40:0/2166755065 >> 172.21.15.40:0/2166755065 conn(0x7f5b5806a390 msgr2=0x7f5b58155240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:27.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.636+0000 7f5b57fff700 1 -- 172.21.15.40:0/2166755065 shutdown_connections 2024-09-13T18:00:27.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:27.636+0000 7f5b57fff700 1 -- 172.21.15.40:0/2166755065 wait complete. 2024-09-13T18:00:30.661 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:00:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.768+0000 7f714e538700 1 Processor -- start 2024-09-13T18:00:30.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.768+0000 7f714e538700 1 -- start start 2024-09-13T18:00:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.768+0000 7f714e538700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7148071e60 0x7f71480c0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.768+0000 7f714e538700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71480c28f0 0x7f71481565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.768+0000 7f714e538700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71480b2210 con 0x7f7148072240 2024-09-13T18:00:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.768+0000 7f714e538700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71480c1780 con 0x7f71480c28f0 2024-09-13T18:00:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.768+0000 7f714e538700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71480c1a60 con 0x7f7148071e60 2024-09-13T18:00:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.768+0000 7f714dd37700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71480c28f0 0x7f71481565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.768+0000 7f714dd37700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71480c28f0 0x7f71481565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48402/0 (socket says 172.21.15.40:48402) 2024-09-13T18:00:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.768+0000 7f714dd37700 1 -- 172.21.15.40:0/2637305219 learned_addr learned my addr 172.21.15.40:0/2637305219 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.772+0000 7f714d536700 1 --2- 172.21.15.40:0/2637305219 >> v2:172.21.15.43:3300/0 conn(0x7f7148071e60 0x7f71480c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.772+0000 7f714dd37700 1 -- 172.21.15.40:0/2637305219 >> v2:172.21.15.43:3300/0 conn(0x7f7148071e60 msgr2=0x7f71480c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:00:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.772+0000 7f714dd37700 1 --2- 172.21.15.40:0/2637305219 >> v2:172.21.15.43:3300/0 conn(0x7f7148071e60 0x7f71480c0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.772+0000 7f714dd37700 1 -- 172.21.15.40:0/2637305219 >> v1:172.21.15.43:6789/0 conn(0x7f7148072240 legacy=0x7f71480c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.772+0000 7f714dd37700 1 -- 172.21.15.40:0/2637305219 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71480c5d70 con 0x7f71480c28f0 2024-09-13T18:00:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.772+0000 7f714d536700 1 --2- 172.21.15.40:0/2637305219 >> v2:172.21.15.43:3300/0 conn(0x7f7148071e60 0x7f71480c0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:00:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.772+0000 7f714dd37700 1 --2- 172.21.15.40:0/2637305219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71480c28f0 0x7f71481565c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7148070750 tx=0x7f714000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0eb42cd0733ff84 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.772+0000 7f713ffff700 1 -- 172.21.15.40:0/2637305219 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714000e070 con 0x7f71480c28f0 2024-09-13T18:00:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.772+0000 7f713ffff700 1 -- 172.21.15.40:0/2637305219 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714000b9c0 con 0x7f71480c28f0 2024-09-13T18:00:30.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.772+0000 7f713ffff700 1 -- 172.21.15.40:0/2637305219 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714000bd70 con 0x7f71480c28f0 2024-09-13T18:00:30.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.772+0000 7f714e538700 1 -- 172.21.15.40:0/2637305219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71480c28f0 msgr2=0x7f71481565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:30.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.772+0000 7f714e538700 1 --2- 172.21.15.40:0/2637305219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71480c28f0 0x7f71481565c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7148070750 tx=0x7f714000b330 comp rx=0 tx=0).stop 2024-09-13T18:00:30.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.772+0000 7f714e538700 1 -- 172.21.15.40:0/2637305219 shutdown_connections 2024-09-13T18:00:30.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.772+0000 7f714e538700 1 --2- 172.21.15.40:0/2637305219 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71480c28f0 0x7f71481565c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:30.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.772+0000 7f714e538700 1 --2- 172.21.15.40:0/2637305219 >> v2:172.21.15.43:3300/0 conn(0x7f7148071e60 0x7f71480c0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:30.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.772+0000 7f714e538700 1 -- 172.21.15.40:0/2637305219 >> 172.21.15.40:0/2637305219 conn(0x7f714806e4f0 msgr2=0x7f714806e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:30.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.772+0000 7f714e538700 1 -- 172.21.15.40:0/2637305219 shutdown_connections 2024-09-13T18:00:30.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.772+0000 7f714e538700 1 -- 172.21.15.40:0/2637305219 wait complete. 2024-09-13T18:00:30.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714e538700 1 Processor -- start 2024-09-13T18:00:30.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714e538700 1 -- start start 2024-09-13T18:00:30.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714e538700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7148071e60 0x7f714814ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:30.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714e538700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7148072240 0x7f71480ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:30.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714e538700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71480c28f0 0x7f71480ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:30.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714e538700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7148152ad0 con 0x7f71480c28f0 2024-09-13T18:00:30.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714e538700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71480c9d10 con 0x7f7148072240 2024-09-13T18:00:30.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714e538700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7148152cc0 con 0x7f7148071e60 2024-09-13T18:00:30.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714d536700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7148071e60 msgr2=0x7f714814ddb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:00:30.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714dd37700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71480c28f0 0x7f71480ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:30.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714dd37700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71480c28f0 0x7f71480ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48414/0 (socket says 172.21.15.40:48414) 2024-09-13T18:00:30.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714dd37700 1 -- 172.21.15.40:0/2567329021 learned_addr learned my addr 172.21.15.40:0/2567329021 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:30.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714d536700 1 --2- 172.21.15.40:0/2567329021 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7148071e60 0x7f714814ddb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:30.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714cd35700 1 --2- 172.21.15.40:0/2567329021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7148072240 0x7f71480ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:30.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714dd37700 1 -- 172.21.15.40:0/2567329021 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7148071e60 msgr2=0x7f714814ddb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:00:30.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714dd37700 1 --2- 172.21.15.40:0/2567329021 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7148071e60 0x7f714814ddb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:30.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714dd37700 1 -- 172.21.15.40:0/2567329021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7148072240 msgr2=0x7f71480ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:30.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714dd37700 1 --2- 172.21.15.40:0/2567329021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7148072240 0x7f71480ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:30.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714dd37700 1 -- 172.21.15.40:0/2567329021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7140009b30 con 0x7f71480c28f0 2024-09-13T18:00:30.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714dd37700 1 --2- 172.21.15.40:0/2567329021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71480c28f0 0x7f71480ca6e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f714000b730 tx=0x7f7140004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:30.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f713dffb700 1 -- 172.21.15.40:0/2567329021 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7140004250 con 0x7f71480c28f0 2024-09-13T18:00:30.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f713dffb700 1 -- 172.21.15.40:0/2567329021 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71400043b0 con 0x7f71480c28f0 2024-09-13T18:00:30.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.776+0000 7f714e538700 1 -- 172.21.15.40:0/2567329021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7148152fa0 con 0x7f71480c28f0 2024-09-13T18:00:30.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.780+0000 7f713dffb700 1 -- 172.21.15.40:0/2567329021 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7140004720 con 0x7f71480c28f0 2024-09-13T18:00:30.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.780+0000 7f714e538700 1 -- 172.21.15.40:0/2567329021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f714814c0f0 con 0x7f71480c28f0 2024-09-13T18:00:30.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.780+0000 7f714e538700 1 -- 172.21.15.40:0/2567329021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7128005310 con 0x7f71480c28f0 2024-09-13T18:00:30.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.780+0000 7f713dffb700 1 -- 172.21.15.40:0/2567329021 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f714001b430 con 0x7f71480c28f0 2024-09-13T18:00:30.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.780+0000 7f713dffb700 1 --2- 172.21.15.40:0/2567329021 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f712c0666e0 0x7f712c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:30.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.780+0000 7f713dffb700 1 -- 172.21.15.40:0/2567329021 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7140085660 con 0x7f71480c28f0 2024-09-13T18:00:30.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.780+0000 7f714d536700 1 --2- 172.21.15.40:0/2567329021 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f712c0666e0 0x7f712c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:30.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.780+0000 7f714d536700 1 --2- 172.21.15.40:0/2567329021 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f712c0666e0 0x7f712c068ba0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7144009180 tx=0x7f7144008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:30.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:30.788+0000 7f713dffb700 1 -- 172.21.15.40:0/2567329021 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f71400560a0 con 0x7f71480c28f0 2024-09-13T18:00:31.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:31.076+0000 7f714e538700 1 -- 172.21.15.40:0/2567329021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f71280055c0 con 0x7f71480c28f0 2024-09-13T18:00:31.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:31.080+0000 7f713dffb700 1 -- 172.21.15.40:0/2567329021 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f7140055c30 con 0x7f71480c28f0 2024-09-13T18:00:31.088 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":76,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:00:31.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:31.080+0000 7f714e538700 1 -- 172.21.15.40:0/2567329021 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f712c0666e0 msgr2=0x7f712c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:31.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:31.080+0000 7f714e538700 1 --2- 172.21.15.40:0/2567329021 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f712c0666e0 0x7f712c068ba0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7144009180 tx=0x7f7144008040 comp rx=0 tx=0).stop 2024-09-13T18:00:31.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:31.080+0000 7f714e538700 1 -- 172.21.15.40:0/2567329021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71480c28f0 msgr2=0x7f71480ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:31.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:31.080+0000 7f714e538700 1 --2- 172.21.15.40:0/2567329021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71480c28f0 0x7f71480ca6e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f714000b730 tx=0x7f7140004af0 comp rx=0 tx=0).stop 2024-09-13T18:00:31.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:31.080+0000 7f714e538700 1 -- 172.21.15.40:0/2567329021 shutdown_connections 2024-09-13T18:00:31.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:31.080+0000 7f714e538700 1 --2- 172.21.15.40:0/2567329021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71480c28f0 0x7f71480ca6e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:31.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:31.080+0000 7f714e538700 1 --2- 172.21.15.40:0/2567329021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7148072240 0x7f71480ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:31.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:31.080+0000 7f714e538700 1 --2- 172.21.15.40:0/2567329021 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f712c0666e0 0x7f712c068ba0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:31.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:31.080+0000 7f714e538700 1 --2- 172.21.15.40:0/2567329021 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7148071e60 0x7f714814ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:31.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:31.080+0000 7f714e538700 1 -- 172.21.15.40:0/2567329021 >> 172.21.15.40:0/2567329021 conn(0x7f714806e4f0 msgr2=0x7f71480c7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:31.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:31.080+0000 7f714e538700 1 -- 172.21.15.40:0/2567329021 shutdown_connections 2024-09-13T18:00:31.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:31.080+0000 7f714e538700 1 -- 172.21.15.40:0/2567329021 wait complete. 2024-09-13T18:00:34.106 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:00:34.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.216+0000 7ff712dcb700 1 Processor -- start 2024-09-13T18:00:34.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.216+0000 7ff712dcb700 1 -- start start 2024-09-13T18:00:34.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.220+0000 7ff712dcb700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 0x7ff70c156000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:34.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.220+0000 7ff712dcb700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff70c0b2240 con 0x7ff70c1504e0 2024-09-13T18:00:34.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.220+0000 7ff712dcb700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff70c159bc0 con 0x7ff70c0b1b60 2024-09-13T18:00:34.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.220+0000 7ff712dcb700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff70c15a4a0 con 0x7ff70c15a050 2024-09-13T18:00:34.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.220+0000 7ff711dc9700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff70c1504e0 0x7ff70c0c7900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45516/0 (socket says 172.21.15.40:45516) 2024-09-13T18:00:34.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.220+0000 7ff711dc9700 1 -- 172.21.15.40:0/3829569720 learned_addr learned my addr 172.21.15.40:0/3829569720 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:00:34.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.220+0000 7ff7125ca700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 0x7ff70c156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:34.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.220+0000 7ff7125ca700 1 -- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 msgr2=0x7ff70c156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:00:34.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.220+0000 7ff7125ca700 1 -- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 msgr2=0x7ff70c156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:00:34.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.220+0000 7ff7125ca700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 0x7ff70c156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:34.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.220+0000 7ff7125ca700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 0x7ff70c156000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:34.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.424+0000 7ff7125ca700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 0x7ff70c156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:34.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.424+0000 7ff7125ca700 1 -- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 msgr2=0x7ff70c156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:00:34.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.424+0000 7ff7125ca700 1 -- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 msgr2=0x7ff70c156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:00:34.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.424+0000 7ff7125ca700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 0x7ff70c156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:34.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.424+0000 7ff7125ca700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 0x7ff70c156000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:00:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.828+0000 7ff7125ca700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 0x7ff70c156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.828+0000 7ff7125ca700 1 -- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 msgr2=0x7ff70c156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:00:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.828+0000 7ff7125ca700 1 -- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 msgr2=0x7ff70c156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:00:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.828+0000 7ff7125ca700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 0x7ff70c156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:34.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:34.828+0000 7ff7125ca700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 0x7ff70c156000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:00:35.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:35.632+0000 7ff7125ca700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 0x7ff70c156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:35.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:35.632+0000 7ff7125ca700 1 -- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 msgr2=0x7ff70c156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:00:35.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:35.632+0000 7ff7125ca700 1 -- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 msgr2=0x7ff70c156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:00:35.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:35.632+0000 7ff7125ca700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 0x7ff70c156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:35.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:35.632+0000 7ff7125ca700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 0x7ff70c156000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:00:35.670 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:00:35.673 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:00:37.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.220+0000 7ff7037fe700 1 -- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 msgr2=0x7ff70c156000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:00:37.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.220+0000 7ff7037fe700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.43:3300/0 conn(0x7ff70c15a050 0x7ff70c156000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:37.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.220+0000 7ff7037fe700 1 -- 172.21.15.40:0/3829569720 >> v1:172.21.15.43:6789/0 conn(0x7ff70c0b1b60 legacy=0x7ff70c06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:37.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.220+0000 7ff7037fe700 1 -- 172.21.15.40:0/3829569720 >> v1:172.21.15.40:6789/0 conn(0x7ff70c1504e0 legacy=0x7ff70c0c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:37.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.220+0000 7ff7037fe700 1 --2- 172.21.15.40:0/3829569720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0003790 0x7ff6f0005be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:37.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.220+0000 7ff7037fe700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 0x7ff6f0008780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:37.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.220+0000 7ff7037fe700 1 -- 172.21.15.40:0/3829569720 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff70c15a4a0 con 0x7ff6f0006300 2024-09-13T18:00:37.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.220+0000 7ff7037fe700 1 -- 172.21.15.40:0/3829569720 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f000b040 con 0x7ff6f0003790 2024-09-13T18:00:37.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.220+0000 7ff711dc9700 1 -- 172.21.15.40:0/3829569720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0003790 msgr2=0x7ff6f0005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:00:37.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.220+0000 7ff711dc9700 1 --2- 172.21.15.40:0/3829569720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0003790 0x7ff6f0005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:37.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.220+0000 7ff7115c8700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 0x7ff6f0008780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:37.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.220+0000 7ff7115c8700 1 -- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 msgr2=0x7ff6f0008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:00:37.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.220+0000 7ff7115c8700 1 -- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 msgr2=0x7ff6f0008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:00:37.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.220+0000 7ff7115c8700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 0x7ff6f0008780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:37.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.220+0000 7ff7115c8700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 0x7ff6f0008780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:37.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.424+0000 7ff711dc9700 1 -- 172.21.15.40:0/3829569720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0003790 msgr2=0x7ff6f0005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:00:37.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.424+0000 7ff711dc9700 1 --2- 172.21.15.40:0/3829569720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0003790 0x7ff6f0005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:00:37.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.424+0000 7ff7115c8700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 0x7ff6f0008780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:37.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.424+0000 7ff7115c8700 1 -- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 msgr2=0x7ff6f0008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:00:37.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.424+0000 7ff7115c8700 1 -- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 msgr2=0x7ff6f0008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:00:37.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.424+0000 7ff7115c8700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 0x7ff6f0008780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:37.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.424+0000 7ff7115c8700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 0x7ff6f0008780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:00:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.828+0000 7ff711dc9700 1 -- 172.21.15.40:0/3829569720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0003790 msgr2=0x7ff6f0005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:00:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.828+0000 7ff7115c8700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 0x7ff6f0008780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.828+0000 7ff711dc9700 1 --2- 172.21.15.40:0/3829569720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0003790 0x7ff6f0005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:00:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.828+0000 7ff7115c8700 1 -- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 msgr2=0x7ff6f0008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:00:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.828+0000 7ff7115c8700 1 -- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 msgr2=0x7ff6f0008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:00:37.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.828+0000 7ff7115c8700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 0x7ff6f0008780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:37.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:37.828+0000 7ff7115c8700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 0x7ff6f0008780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:00:38.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:38.632+0000 7ff711dc9700 1 -- 172.21.15.40:0/3829569720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0003790 msgr2=0x7ff6f0005be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:00:38.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:38.632+0000 7ff7115c8700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 0x7ff6f0008780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:38.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:38.632+0000 7ff711dc9700 1 --2- 172.21.15.40:0/3829569720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0003790 0x7ff6f0005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:00:38.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:38.632+0000 7ff7115c8700 1 -- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 msgr2=0x7ff6f0008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:00:38.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:38.632+0000 7ff7115c8700 1 -- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 msgr2=0x7ff6f0008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:00:38.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:38.632+0000 7ff7115c8700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 0x7ff6f0008780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:00:38.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:38.632+0000 7ff7115c8700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 0x7ff6f0008780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:00:40.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff7037fe700 1 -- 172.21.15.40:0/3829569720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0003790 msgr2=0x7ff6f0005be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:00:40.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff7037fe700 1 --2- 172.21.15.40:0/3829569720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0003790 0x7ff6f0005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff7037fe700 1 -- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 msgr2=0x7ff6f0008780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:00:40.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff7037fe700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0006300 0x7ff6f0008780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff7125ca700 1 -- 172.21.15.40:0/3829569720 reap_dead start 2024-09-13T18:00:40.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff7037fe700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0003bd0 0x7ff6f0006770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff7037fe700 1 --2- 172.21.15.40:0/3829569720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0006cb0 0x7ff6f00070d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff7037fe700 1 -- 172.21.15.40:0/3829569720 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff70c0c6e20 con 0x7ff6f0007610 2024-09-13T18:00:40.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff7037fe700 1 -- 172.21.15.40:0/3829569720 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff70c15a4a0 con 0x7ff6f0003bd0 2024-09-13T18:00:40.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff7037fe700 1 -- 172.21.15.40:0/3829569720 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f000b040 con 0x7ff6f0006cb0 2024-09-13T18:00:40.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff7115c8700 1 -- 172.21.15.40:0/3829569720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0006cb0 msgr2=0x7ff6f00070d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:00:40.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff711dc9700 1 --2- 172.21.15.40:0/3829569720 >> v2:172.21.15.40:3300/0 conn(0x7ff6f0003bd0 0x7ff6f0006770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:40.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff7115c8700 1 --2- 172.21.15.40:0/3829569720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0006cb0 0x7ff6f00070d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:40.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff711dc9700 1 -- 172.21.15.40:0/3829569720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0006cb0 msgr2=0x7ff6f00070d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:00:40.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff711dc9700 1 --2- 172.21.15.40:0/3829569720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0006cb0 0x7ff6f00070d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff711dc9700 1 -- 172.21.15.40:0/3829569720 >> v1:172.21.15.40:6789/0 conn(0x7ff6f0007610 legacy=0x7ff6f0007a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:40.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff711dc9700 1 -- 172.21.15.40:0/3829569720 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f00084d0 con 0x7ff6f0003bd0 2024-09-13T18:00:40.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff711dc9700 1 --2- 172.21.15.40:0/3829569720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f0003bd0 0x7ff6f0006770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff708007fd0 tx=0x7ff708001d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a884c820a8cd34f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:40.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff710dc7700 1 -- 172.21.15.40:0/3829569720 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708005d90 con 0x7ff6f0003bd0 2024-09-13T18:00:40.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff710dc7700 1 -- 172.21.15.40:0/3829569720 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7080070d0 con 0x7ff6f0003bd0 2024-09-13T18:00:40.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff710dc7700 1 -- 172.21.15.40:0/3829569720 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708007460 con 0x7ff6f0003bd0 2024-09-13T18:00:40.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff712dcb700 1 -- 172.21.15.40:0/3829569720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f0003bd0 msgr2=0x7ff6f0006770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:40.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff712dcb700 1 --2- 172.21.15.40:0/3829569720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f0003bd0 0x7ff6f0006770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff708007fd0 tx=0x7ff708001d60 comp rx=0 tx=0).stop 2024-09-13T18:00:40.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff712dcb700 1 -- 172.21.15.40:0/3829569720 shutdown_connections 2024-09-13T18:00:40.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff712dcb700 1 --2- 172.21.15.40:0/3829569720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0006cb0 0x7ff6f00070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff712dcb700 1 --2- 172.21.15.40:0/3829569720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f0003bd0 0x7ff6f0006770 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.220+0000 7ff712dcb700 1 -- 172.21.15.40:0/3829569720 >> 172.21.15.40:0/3829569720 conn(0x7ff70c0cbd80 msgr2=0x7ff70c0cc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:40.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.224+0000 7ff712dcb700 1 -- 172.21.15.40:0/3829569720 shutdown_connections 2024-09-13T18:00:40.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.224+0000 7ff712dcb700 1 -- 172.21.15.40:0/3829569720 wait complete. 2024-09-13T18:00:40.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.224+0000 7ff712dcb700 1 Processor -- start 2024-09-13T18:00:40.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.224+0000 7ff712dcb700 1 -- start start 2024-09-13T18:00:40.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff712dcb700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0003bd0 0x7ff70c1eb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff712dcb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f0006cb0 0x7ff70c14ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff712dcb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f0007610 0x7ff70c152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff712dcb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c1432e0 con 0x7ff6f0007610 2024-09-13T18:00:40.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff712dcb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c1454b0 con 0x7ff6f0006cb0 2024-09-13T18:00:40.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff712dcb700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c1eada0 con 0x7ff6f0003bd0 2024-09-13T18:00:40.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff711dc9700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0003bd0 msgr2=0x7ff70c1eb840 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:00:40.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff711dc9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0003bd0 0x7ff70c1eb840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:40.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff7125ca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f0007610 0x7ff70c152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:40.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff7125ca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f0007610 0x7ff70c152e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57234/0 (socket says 172.21.15.40:57234) 2024-09-13T18:00:40.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff7125ca700 1 -- 172.21.15.40:0/2598854328 learned_addr learned my addr 172.21.15.40:0/2598854328 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:40.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff7115c8700 1 --2- 172.21.15.40:0/2598854328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f0006cb0 0x7ff70c14ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:40.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff7125ca700 1 -- 172.21.15.40:0/2598854328 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0003bd0 msgr2=0x7ff70c1eb840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:00:40.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff7125ca700 1 --2- 172.21.15.40:0/2598854328 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0003bd0 0x7ff70c1eb840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff7125ca700 1 -- 172.21.15.40:0/2598854328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f0006cb0 msgr2=0x7ff70c14ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:40.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff7125ca700 1 --2- 172.21.15.40:0/2598854328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f0006cb0 0x7ff70c14ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff7125ca700 1 -- 172.21.15.40:0/2598854328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70c0b2240 con 0x7ff6f0007610 2024-09-13T18:00:40.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff7125ca700 1 --2- 172.21.15.40:0/2598854328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f0007610 0x7ff70c152e30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff70c06ded0 tx=0x7ff704004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:40.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff7027fc700 1 -- 172.21.15.40:0/2598854328 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff704011b00 con 0x7ff6f0007610 2024-09-13T18:00:40.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff7027fc700 1 -- 172.21.15.40:0/2598854328 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70400f040 con 0x7ff6f0007610 2024-09-13T18:00:40.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff712dcb700 1 -- 172.21.15.40:0/2598854328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff70c1eed90 con 0x7ff6f0007610 2024-09-13T18:00:40.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff7027fc700 1 -- 172.21.15.40:0/2598854328 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff704011ea0 con 0x7ff6f0007610 2024-09-13T18:00:40.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.228+0000 7ff712dcb700 1 -- 172.21.15.40:0/2598854328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70c1eb320 con 0x7ff6f0007610 2024-09-13T18:00:40.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.232+0000 7ff712dcb700 1 -- 172.21.15.40:0/2598854328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff70c1eb540 con 0x7ff6f0007610 2024-09-13T18:00:40.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.232+0000 7ff7027fc700 1 -- 172.21.15.40:0/2598854328 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff704022460 con 0x7ff6f0007610 2024-09-13T18:00:40.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.232+0000 7ff7027fc700 1 --2- 172.21.15.40:0/2598854328 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff6f006a590 0x7ff6f006ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.232+0000 7ff7027fc700 1 -- 172.21.15.40:0/2598854328 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff704084d80 con 0x7ff6f0007610 2024-09-13T18:00:40.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.232+0000 7ff711dc9700 1 --2- 172.21.15.40:0/2598854328 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff6f006a590 0x7ff6f006ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:40.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.232+0000 7ff711dc9700 1 --2- 172.21.15.40:0/2598854328 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff6f006a590 0x7ff6f006ca50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff708000c00 tx=0x7ff70800a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:40.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.240+0000 7ff7027fc700 1 -- 172.21.15.40:0/2598854328 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff704051ca0 con 0x7ff6f0007610 2024-09-13T18:00:40.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.496+0000 7ff712dcb700 1 -- 172.21.15.40:0/2598854328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff70c0c0f30 con 0x7ff6f0007610 2024-09-13T18:00:40.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.500+0000 7ff7027fc700 1 -- 172.21.15.40:0/2598854328 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7ff704086610 con 0x7ff6f0007610 2024-09-13T18:00:40.507 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":80,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:00:40.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.500+0000 7ff712dcb700 1 -- 172.21.15.40:0/2598854328 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff6f006a590 msgr2=0x7ff6f006ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:40.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.500+0000 7ff712dcb700 1 --2- 172.21.15.40:0/2598854328 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff6f006a590 0x7ff6f006ca50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff708000c00 tx=0x7ff70800a000 comp rx=0 tx=0).stop 2024-09-13T18:00:40.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.500+0000 7ff712dcb700 1 -- 172.21.15.40:0/2598854328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f0007610 msgr2=0x7ff70c152e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:40.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.500+0000 7ff712dcb700 1 --2- 172.21.15.40:0/2598854328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f0007610 0x7ff70c152e30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff70c06ded0 tx=0x7ff704004940 comp rx=0 tx=0).stop 2024-09-13T18:00:40.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.500+0000 7ff712dcb700 1 -- 172.21.15.40:0/2598854328 shutdown_connections 2024-09-13T18:00:40.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.500+0000 7ff712dcb700 1 --2- 172.21.15.40:0/2598854328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6f0007610 0x7ff70c152e30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.500+0000 7ff712dcb700 1 --2- 172.21.15.40:0/2598854328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f0006cb0 0x7ff70c14ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.500+0000 7ff712dcb700 1 --2- 172.21.15.40:0/2598854328 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff6f006a590 0x7ff6f006ca50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.500+0000 7ff712dcb700 1 --2- 172.21.15.40:0/2598854328 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff6f0003bd0 0x7ff70c1eb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.500+0000 7ff712dcb700 1 -- 172.21.15.40:0/2598854328 >> 172.21.15.40:0/2598854328 conn(0x7ff70c0cbd80 msgr2=0x7ff70c0c9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:40.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.500+0000 7ff712dcb700 1 -- 172.21.15.40:0/2598854328 shutdown_connections 2024-09-13T18:00:40.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.500+0000 7ff712dcb700 1 -- 172.21.15.40:0/2598854328 wait complete. 2024-09-13T18:00:40.521 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T18:00:40.521 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-13T18:00:40.522 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:00:40.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94af13d700 1 Processor -- start 2024-09-13T18:00:40.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94af13d700 1 -- start start 2024-09-13T18:00:40.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94af13d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f94a80b0d70 0x7f94a8157f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94af13d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f94a806a950 0x7f94a806ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94af13d700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a8158440 con 0x7f94a8151750 2024-09-13T18:00:40.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94af13d700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a806a4c0 con 0x7f94a80b0d70 2024-09-13T18:00:40.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94af13d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a806b320 con 0x7f94a806a950 2024-09-13T18:00:40.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94ad6da700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f94a806a950 msgr2=0x7f94a806ade0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:00:40.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94aced9700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f94a8151750 0x7f94a806fbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45534/0 (socket says 172.21.15.40:45534) 2024-09-13T18:00:40.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94ad6da700 1 --2- 172.21.15.40:0/2794526538 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f94a806a950 0x7f94a806ade0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:40.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94aced9700 1 -- 172.21.15.40:0/2794526538 learned_addr learned my addr 172.21.15.40:0/2794526538 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:00:40.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a7fff700 1 --2- 172.21.15.40:0/2794526538 >> v2:172.21.15.43:3300/0 conn(0x7f94a80b0d70 0x7f94a8157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:40.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a77fe700 1 -- 172.21.15.40:0/2794526538 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824135249 0 0) 0x7f949c001110 con 0x7f94a8151750 2024-09-13T18:00:40.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a77fe700 1 -- 172.21.15.40:0/2794526538 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94980035d0 con 0x7f94a8151750 2024-09-13T18:00:40.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a77fe700 1 -- 172.21.15.40:0/2794526538 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1364401770 0 0) 0x7f94980035d0 con 0x7f94a8151750 2024-09-13T18:00:40.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a77fe700 1 -- 172.21.15.40:0/2794526538 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f94a806a950 msgr2=0x7f94a806ade0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:00:40.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a77fe700 1 --2- 172.21.15.40:0/2794526538 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f94a806a950 0x7f94a806ade0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a77fe700 1 -- 172.21.15.40:0/2794526538 >> v2:172.21.15.43:3300/0 conn(0x7f94a80b0d70 msgr2=0x7f94a8157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:00:40.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a77fe700 1 --2- 172.21.15.40:0/2794526538 >> v2:172.21.15.43:3300/0 conn(0x7f94a80b0d70 0x7f94a8157f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a77fe700 1 -- 172.21.15.40:0/2794526538 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a806cf60 con 0x7f94a8151750 2024-09-13T18:00:40.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a77fe700 1 -- 172.21.15.40:0/2794526538 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f949c001800 con 0x7f94a8151750 2024-09-13T18:00:40.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a77fe700 1 -- 172.21.15.40:0/2794526538 >> v1:172.21.15.40:6789/0 conn(0x7f94a8151750 legacy=0x7f94a806fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:40.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a77fe700 1 --2- 172.21.15.40:0/2794526538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9498003a20 0x7f9498003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a77fe700 1 --2- 172.21.15.40:0/2794526538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9498004870 0x7f9498004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a77fe700 1 --2- 172.21.15.40:0/2794526538 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f94a806a950 0x7f9498005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a77fe700 1 -- 172.21.15.40:0/2794526538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a806a4c0 con 0x7f9498004870 2024-09-13T18:00:40.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a77fe700 1 -- 172.21.15.40:0/2794526538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a806b320 con 0x7f9498003a20 2024-09-13T18:00:40.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a77fe700 1 -- 172.21.15.40:0/2794526538 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949800bb60 con 0x7f94a806a950 2024-09-13T18:00:40.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a7fff700 1 --2- 172.21.15.40:0/2794526538 >> v2:172.21.15.43:3300/0 conn(0x7f94a80b0d70 0x7f94a8157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:00:40.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94ad6da700 1 -- 172.21.15.40:0/2794526538 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f94a806a950 msgr2=0x7f9498005230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:00:40.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94ad6da700 1 --2- 172.21.15.40:0/2794526538 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f94a806a950 0x7f9498005230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:40.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a7fff700 1 --2- 172.21.15.40:0/2794526538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9498004870 0x7f9498004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:40.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94aced9700 1 --2- 172.21.15.40:0/2794526538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9498003a20 0x7f9498003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:40.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a7fff700 1 -- 172.21.15.40:0/2794526538 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f94a806a950 msgr2=0x7f9498005230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:00:40.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a7fff700 1 --2- 172.21.15.40:0/2794526538 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f94a806a950 0x7f9498005230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a7fff700 1 -- 172.21.15.40:0/2794526538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9498003a20 msgr2=0x7f9498003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:00:40.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a7fff700 1 --2- 172.21.15.40:0/2794526538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9498003a20 0x7f9498003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.624+0000 7f94a7fff700 1 -- 172.21.15.40:0/2794526538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94980035d0 con 0x7f9498004870 2024-09-13T18:00:40.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.628+0000 7f94a7fff700 1 --2- 172.21.15.40:0/2794526538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9498004870 0x7f9498004cf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9494009cf0 tx=0x7f94980058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19bdcae10fae2418 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:40.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.628+0000 7f94a77fe700 1 -- 172.21.15.40:0/2794526538 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9494004030 con 0x7f9498004870 2024-09-13T18:00:40.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.628+0000 7f94a77fe700 1 -- 172.21.15.40:0/2794526538 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9494004190 con 0x7f9498004870 2024-09-13T18:00:40.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.628+0000 7f94a77fe700 1 -- 172.21.15.40:0/2794526538 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9494004500 con 0x7f9498004870 2024-09-13T18:00:40.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.628+0000 7f94af13d700 1 -- 172.21.15.40:0/2794526538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9498004870 msgr2=0x7f9498004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:40.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.628+0000 7f94af13d700 1 --2- 172.21.15.40:0/2794526538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9498004870 0x7f9498004cf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9494009cf0 tx=0x7f94980058a0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.628+0000 7f94ad6da700 1 -- 172.21.15.40:0/2794526538 reap_dead start 2024-09-13T18:00:40.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.628+0000 7f94af13d700 1 -- 172.21.15.40:0/2794526538 shutdown_connections 2024-09-13T18:00:40.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.628+0000 7f94af13d700 1 -- 172.21.15.40:0/2794526538 >> 172.21.15.40:0/2794526538 conn(0x7f94a806d5b0 msgr2=0x7f94a806d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:40.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.628+0000 7f94af13d700 1 -- 172.21.15.40:0/2794526538 shutdown_connections 2024-09-13T18:00:40.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.628+0000 7f94af13d700 1 -- 172.21.15.40:0/2794526538 wait complete. 2024-09-13T18:00:40.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.628+0000 7f94af13d700 1 Processor -- start 2024-09-13T18:00:40.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.628+0000 7f94af13d700 1 -- start start 2024-09-13T18:00:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94af13d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9498004870 0x7f94a80c1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94af13d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f94a814dc10 0x7f94a8148a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94af13d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a81f2a40 0x7f94a81f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94af13d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a81499d0 con 0x7f94a81f2a40 2024-09-13T18:00:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94af13d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a81efe50 con 0x7f9498004870 2024-09-13T18:00:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94af13d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a80c0a20 con 0x7f94a814dc10 2024-09-13T18:00:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94a7fff700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f94a814dc10 msgr2=0x7f94a8148a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:00:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94a7fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f94a814dc10 0x7f94a8148a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94ad6da700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a81f2a40 0x7f94a81f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94aced9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9498004870 0x7f94a80c1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94ad6da700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a81f2a40 0x7f94a81f2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57258/0 (socket says 172.21.15.40:57258) 2024-09-13T18:00:40.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94ad6da700 1 -- 172.21.15.40:0/1801359773 learned_addr learned my addr 172.21.15.40:0/1801359773 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94ad6da700 1 -- 172.21.15.40:0/1801359773 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f94a814dc10 msgr2=0x7f94a8148a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:00:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94ad6da700 1 --2- 172.21.15.40:0/1801359773 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f94a814dc10 0x7f94a8148a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94ad6da700 1 -- 172.21.15.40:0/1801359773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9498004870 msgr2=0x7f94a80c1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94ad6da700 1 --2- 172.21.15.40:0/1801359773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9498004870 0x7f94a80c1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94ad6da700 1 -- 172.21.15.40:0/1801359773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a81ef4d0 con 0x7f94a81f2a40 2024-09-13T18:00:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94aced9700 1 --2- 172.21.15.40:0/1801359773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9498004870 0x7f94a80c1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:00:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94ad6da700 1 --2- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a81f2a40 0x7f94a81f2ea0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f94a80708e0 tx=0x7f94a000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94a57fa700 1 -- 172.21.15.40:0/1801359773 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a000cd10 con 0x7f94a81f2a40 2024-09-13T18:00:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94a57fa700 1 -- 172.21.15.40:0/1801359773 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a000ce70 con 0x7f94a81f2a40 2024-09-13T18:00:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94af13d700 1 -- 172.21.15.40:0/1801359773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a814f9c0 con 0x7f94a81f2a40 2024-09-13T18:00:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94a57fa700 1 -- 172.21.15.40:0/1801359773 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a0015690 con 0x7f94a81f2a40 2024-09-13T18:00:40.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94af13d700 1 -- 172.21.15.40:0/1801359773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a81efa70 con 0x7f94a81f2a40 2024-09-13T18:00:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94af13d700 1 --2- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a814dff0 0x7f94a81f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.632+0000 7f94af13d700 1 -- 172.21.15.40:0/1801359773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a8067840 con 0x7f94a814dff0 2024-09-13T18:00:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.636+0000 7f94aced9700 1 --2- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a814dff0 0x7f94a81f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:40.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.636+0000 7f94aced9700 1 -- 172.21.15.40:0/1801359773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f94a8071240 con 0x7f94a814dff0 2024-09-13T18:00:40.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.636+0000 7f94aced9700 1 --2- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a814dff0 0x7f94a81f33e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f949c001270 tx=0x7f949c0125a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:40.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.636+0000 7f94a57fa700 1 -- 172.21.15.40:0/1801359773 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c012c10 con 0x7f94a814dff0 2024-09-13T18:00:40.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.636+0000 7f94a57fa700 1 -- 172.21.15.40:0/1801359773 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f94a0012070 con 0x7f94a81f2a40 2024-09-13T18:00:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.636+0000 7f94a57fa700 1 --2- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f948c066800 0x7f948c068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.636+0000 7f94a57fa700 1 -- 172.21.15.40:0/1801359773 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f94a0084290 con 0x7f94a81f2a40 2024-09-13T18:00:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.636+0000 7f94a7fff700 1 --2- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f948c066800 0x7f948c068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.636+0000 7f94a57fa700 1 -- 172.21.15.40:0/1801359773 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f949c012d70 con 0x7f94a814dff0 2024-09-13T18:00:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.636+0000 7f94a57fa700 1 -- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a814dff0 msgr2=0x7f94a81f33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.636+0000 7f94a57fa700 1 --2- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a814dff0 0x7f94a81f33e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f949c001270 tx=0x7f949c0125a0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.636+0000 7f94a57fa700 1 -- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a814dff0 msgr2=0x7f94a81f33e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:00:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.636+0000 7f94a57fa700 1 --2- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a814dff0 0x7f94a81f33e0 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f949c001270 tx=0x7f949c0125a0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.636+0000 7f94a7fff700 1 --2- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f948c066800 0x7f948c068cc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f94a81f3b00 tx=0x7f9494008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:40.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.644+0000 7f94af13d700 1 --2- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a81402a0 0x7f94a81406b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.644+0000 7f94af13d700 1 -- 172.21.15.40:0/1801359773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a8070f60 con 0x7f94a81402a0 2024-09-13T18:00:40.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.644+0000 7f94aced9700 1 --2- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a81402a0 0x7f94a81406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:40.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.648+0000 7f94aced9700 1 -- 172.21.15.40:0/1801359773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f94a8071520 con 0x7f94a81402a0 2024-09-13T18:00:40.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.648+0000 7f94aced9700 1 --2- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a81402a0 0x7f94a81406b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f949c001270 tx=0x7f949c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:40.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.648+0000 7f94a57fa700 1 -- 172.21.15.40:0/1801359773 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c025070 con 0x7f94a81402a0 2024-09-13T18:00:40.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.648+0000 7f94a57fa700 1 -- 172.21.15.40:0/1801359773 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4772 (secure 0 0 0) 0x7f949c015b30 con 0x7f94a81402a0 2024-09-13T18:00:40.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.648+0000 7f94a57fa700 1 -- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a81402a0 msgr2=0x7f94a81406b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:40.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.648+0000 7f94a57fa700 1 --2- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a81402a0 0x7f94a81406b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f949c001270 tx=0x7f949c007a20 comp rx=0 tx=0).stop 2024-09-13T18:00:40.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.648+0000 7f94a57fa700 1 -- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a81402a0 msgr2=0x7f94a81406b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:00:40.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.648+0000 7f94a57fa700 1 --2- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a81402a0 0x7f94a81406b0 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f949c001270 tx=0x7f949c007a20 comp rx=0 tx=0).stop 2024-09-13T18:00:40.657 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:00:40.657 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:00:40.657 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:00:40.657 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:00:40.657 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 80, 2024-09-13T18:00:40.657 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:00:40.657 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:00:40.657 INFO:teuthology.orchestra.run.smithi040.stdout: 1 2024-09-13T18:00:40.658 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:40.658 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:00:40.658 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:00:40.658 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:00:40.658 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:00:40.658 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:40.658 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:40.658 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:40.658 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:40.658 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:40.658 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:40.658 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:40.658 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:40.658 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:40.659 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:40.659 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:40.659 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:00:40.659 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:00:40.659 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:40.659 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:40.659 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:40.659 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:40.659 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:40.659 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:40.659 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:40.659 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:40.659 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:40.659 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:40.660 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:40.660 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.660 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:00:40.660 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:00:40.660 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:00:40.660 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:00:40.660 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:00:40.660 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:00:40.660 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:00:40.660 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:00:40.660 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:00:40.660 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:00:40.660 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:00:40.660 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:00:40.661 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:00:40.661 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:00:40.661 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:00:40.661 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:00:40.661 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:00:40.661 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:40.661 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:40.661 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:40.661 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:40.661 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:40.661 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:40.661 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:40.661 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:40.661 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:40.661 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:40.662 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:40.662 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:00:40.662 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.662 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:00:40.662 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.662 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:00:40.662 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:00:40.662 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:40.662 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:40.662 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.662 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:40.662 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:00:40.662 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:40.662 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.663 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.663 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:40.663 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:00:40.663 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:40.663 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:40.663 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:40.663 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.663 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:00:40.663 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:00:40.663 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:40.663 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:40.663 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:40.663 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.663 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.663 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:00:40.664 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:00:40.664 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:40.664 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:40.664 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.664 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:40.664 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:00:40.664 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:40.664 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.664 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.664 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:40.664 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:00:40.664 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:40.664 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:40.664 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:40.665 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.665 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:00:40.665 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:00:40.665 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:40.665 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:40.665 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:40.665 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.665 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.665 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:00:40.665 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:00:40.665 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:40.665 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:40.665 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.665 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:40.666 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:00:40.666 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:40.666 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.666 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.666 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:40.666 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:00:40.666 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:40.666 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:40.666 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:40.666 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.666 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:00:40.666 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:00:40.666 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:40.666 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:40.667 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:40.667 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:40.667 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:40.667 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.667 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:00:40.667 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:00:40.667 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.667 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:40.667 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:40.667 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:00:40.667 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:40.667 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:40.667 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:00:40.667 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.667 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:40.668 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:40.668 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:00:40.668 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:40.668 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:40.668 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:00:40.668 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.668 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:40.668 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:40.668 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:00:40.668 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:40.668 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:40.668 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:00:40.668 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.669 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:40.669 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:40.669 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:00:40.669 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:40.669 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:40.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.669 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:00:40.669 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:00:40.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.648+0000 7f94af13d700 1 -- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f948c066800 msgr2=0x7f948c068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:40.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.648+0000 7f94af13d700 1 --2- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f948c066800 0x7f948c068cc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f94a81f3b00 tx=0x7f9494008030 comp rx=0 tx=0).stop 2024-09-13T18:00:40.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.648+0000 7f94ad6da700 1 -- 172.21.15.40:0/1801359773 reap_dead start 2024-09-13T18:00:40.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.648+0000 7f94af13d700 1 -- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a81f2a40 msgr2=0x7f94a81f2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:40.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.648+0000 7f94af13d700 1 --2- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a81f2a40 0x7f94a81f2ea0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f94a80708e0 tx=0x7f94a000bdd0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.648+0000 7f94af13d700 1 -- 172.21.15.40:0/1801359773 shutdown_connections 2024-09-13T18:00:40.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.648+0000 7f94af13d700 1 --2- 172.21.15.40:0/1801359773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94a81f2a40 0x7f94a81f2ea0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.648+0000 7f94af13d700 1 -- 172.21.15.40:0/1801359773 >> 172.21.15.40:0/1801359773 conn(0x7f94a806d5b0 msgr2=0x7f94a8071e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:40.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.648+0000 7f94af13d700 1 -- 172.21.15.40:0/1801359773 shutdown_connections 2024-09-13T18:00:40.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.648+0000 7f94af13d700 1 -- 172.21.15.40:0/1801359773 wait complete. 2024-09-13T18:00:40.672 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:00:40.672 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-13T18:00:40.672 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:00:40.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.784+0000 7f3df7fff700 1 Processor -- start 2024-09-13T18:00:40.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.784+0000 7f3df7fff700 1 -- start start 2024-09-13T18:00:40.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df7fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3df8153a90 0x7f3df80c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df7fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df815ce40 con 0x7f3df80704a0 2024-09-13T18:00:40.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df7fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df80c7d00 con 0x7f3df80c78f0 2024-09-13T18:00:40.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df7fff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df8153e70 con 0x7f3df8153a90 2024-09-13T18:00:40.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df6ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3df8153a90 0x7f3df80c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:40.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df6ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3df8153a90 0x7f3df80c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:39870/0 (socket says 172.21.15.40:39870) 2024-09-13T18:00:40.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df6ffd700 1 -- 172.21.15.40:0/299939098 learned_addr learned my addr 172.21.15.40:0/299939098 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:40.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df6ffd700 1 -- 172.21.15.40:0/299939098 >> v1:172.21.15.43:6789/0 conn(0x7f3df80c78f0 legacy=0x7f3df806f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:40.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df6ffd700 1 -- 172.21.15.40:0/299939098 >> v1:172.21.15.40:6789/0 conn(0x7f3df80704a0 legacy=0x7f3df815d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:40.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df67fc700 1 -- 172.21.15.40:0/299939098 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335923677 0 0) 0x7f3df815ce40 con 0x7f3df80704a0 2024-09-13T18:00:40.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df6ffd700 1 -- 172.21.15.40:0/299939098 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df80c88a0 con 0x7f3df8153a90 2024-09-13T18:00:40.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df6ffd700 1 --2- 172.21.15.40:0/299939098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df8153a90 0x7f3df80c94a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009910 tx=0x7f3de8009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b687573e84c6fbfc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:40.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df67fc700 1 -- 172.21.15.40:0/299939098 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de800ba60 con 0x7f3df8153a90 2024-09-13T18:00:40.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df67fc700 1 -- 172.21.15.40:0/299939098 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de800bbc0 con 0x7f3df8153a90 2024-09-13T18:00:40.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df67fc700 1 -- 172.21.15.40:0/299939098 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de8009e60 con 0x7f3df8153a90 2024-09-13T18:00:40.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df7fff700 1 -- 172.21.15.40:0/299939098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df8153a90 msgr2=0x7f3df80c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:40.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df7fff700 1 --2- 172.21.15.40:0/299939098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df8153a90 0x7f3df80c94a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009910 tx=0x7f3de8009c20 comp rx=0 tx=0).stop 2024-09-13T18:00:40.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df7fff700 1 -- 172.21.15.40:0/299939098 shutdown_connections 2024-09-13T18:00:40.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df7fff700 1 --2- 172.21.15.40:0/299939098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df8153a90 0x7f3df80c94a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df7fff700 1 -- 172.21.15.40:0/299939098 >> 172.21.15.40:0/299939098 conn(0x7f3df80ccef0 msgr2=0x7f3df80cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:40.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df7fff700 1 -- 172.21.15.40:0/299939098 shutdown_connections 2024-09-13T18:00:40.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.788+0000 7f3df7fff700 1 -- 172.21.15.40:0/299939098 wait complete. 2024-09-13T18:00:40.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df7fff700 1 Processor -- start 2024-09-13T18:00:40.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df7fff700 1 -- start start 2024-09-13T18:00:40.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df80704a0 0x7f3df80c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3df80c78f0 0x7f3df8150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df7fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3df8153a90 0x7f3df8150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df7fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df8150fc0 con 0x7f3df80c78f0 2024-09-13T18:00:40.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df7fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df81f3630 con 0x7f3df80704a0 2024-09-13T18:00:40.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df7fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df80c5a10 con 0x7f3df8153a90 2024-09-13T18:00:40.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df77fe700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3df8153a90 msgr2=0x7f3df8150a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:00:40.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df77fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3df8153a90 0x7f3df8150a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:00:40.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3deffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3df80c78f0 0x7f3df8150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:40.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df6ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df80704a0 0x7f3df80c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:40.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3deffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3df80c78f0 0x7f3df8150540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57288/0 (socket says 172.21.15.40:57288) 2024-09-13T18:00:40.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df6ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df80704a0 0x7f3df80c6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:39874/0 (socket says 172.21.15.40:39874) 2024-09-13T18:00:40.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df6ffd700 1 -- 172.21.15.40:0/2731528346 learned_addr learned my addr 172.21.15.40:0/2731528346 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:00:40.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df6ffd700 1 -- 172.21.15.40:0/2731528346 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3df8153a90 msgr2=0x7f3df8150a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:00:40.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df6ffd700 1 --2- 172.21.15.40:0/2731528346 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3df8153a90 0x7f3df8150a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df6ffd700 1 -- 172.21.15.40:0/2731528346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3df80c78f0 msgr2=0x7f3df8150540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:40.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df6ffd700 1 --2- 172.21.15.40:0/2731528346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3df80c78f0 0x7f3df8150540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df6ffd700 1 -- 172.21.15.40:0/2731528346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de8009570 con 0x7f3df80704a0 2024-09-13T18:00:40.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df6ffd700 1 --2- 172.21.15.40:0/2731528346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df80704a0 0x7f3df80c6a70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3de8000c00 tx=0x7f3de8004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:40.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df4ff9700 1 -- 172.21.15.40:0/2731528346 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de80119f0 con 0x7f3df80704a0 2024-09-13T18:00:40.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df4ff9700 1 -- 172.21.15.40:0/2731528346 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de801f070 con 0x7f3df80704a0 2024-09-13T18:00:40.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df7fff700 1 -- 172.21.15.40:0/2731528346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df806ef70 con 0x7f3df80704a0 2024-09-13T18:00:40.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df4ff9700 1 -- 172.21.15.40:0/2731528346 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de8011d90 con 0x7f3df80704a0 2024-09-13T18:00:40.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df7fff700 1 -- 172.21.15.40:0/2731528346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df80c5f90 con 0x7f3df80704a0 2024-09-13T18:00:40.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df7fff700 1 --2- 172.21.15.40:0/2731528346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df81f30c0 0x7f3df80c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.792+0000 7f3df7fff700 1 -- 172.21.15.40:0/2731528346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df8074830 con 0x7f3df81f30c0 2024-09-13T18:00:40.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.796+0000 7f3deffff700 1 --2- 172.21.15.40:0/2731528346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df81f30c0 0x7f3df80c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:40.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.796+0000 7f3deffff700 1 -- 172.21.15.40:0/2731528346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3df815a560 con 0x7f3df81f30c0 2024-09-13T18:00:40.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.796+0000 7f3df4ff9700 1 -- 172.21.15.40:0/2731528346 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3de8026480 con 0x7f3df80704a0 2024-09-13T18:00:40.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.796+0000 7f3deffff700 1 --2- 172.21.15.40:0/2731528346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df81f30c0 0x7f3df80c94a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3df80702b0 tx=0x7f3de000f8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:40.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.796+0000 7f3df4ff9700 1 --2- 172.21.15.40:0/2731528346 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3dd80666e0 0x7f3dd8068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.796+0000 7f3df77fe700 1 --2- 172.21.15.40:0/2731528346 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3dd80666e0 0x7f3dd8068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:40.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.796+0000 7f3df4ff9700 1 -- 172.21.15.40:0/2731528346 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3de8021070 con 0x7f3df80704a0 2024-09-13T18:00:40.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.796+0000 7f3df4ff9700 1 -- 172.21.15.40:0/2731528346 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de0005320 con 0x7f3df81f30c0 2024-09-13T18:00:40.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.796+0000 7f3df4ff9700 1 -- 172.21.15.40:0/2731528346 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3de0005480 con 0x7f3df81f30c0 2024-09-13T18:00:40.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.796+0000 7f3df4ff9700 1 -- 172.21.15.40:0/2731528346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df81f30c0 msgr2=0x7f3df80c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:40.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.796+0000 7f3df4ff9700 1 --2- 172.21.15.40:0/2731528346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df81f30c0 0x7f3df80c94a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3df80702b0 tx=0x7f3de000f8d0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.796+0000 7f3df4ff9700 1 -- 172.21.15.40:0/2731528346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df81f30c0 msgr2=0x7f3df80c94a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:00:40.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.796+0000 7f3df4ff9700 1 --2- 172.21.15.40:0/2731528346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df81f30c0 0x7f3df80c94a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.796+0000 7f3df77fe700 1 --2- 172.21.15.40:0/2731528346 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3dd80666e0 0x7f3dd8068ba0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3df00013b0 tx=0x7f3df0001770 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:40.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3df7fff700 1 --2- 172.21.15.40:0/2731528346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df80c99e0 0x7f3df80adc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:00:40.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3df7fff700 1 -- 172.21.15.40:0/2731528346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df80c8b40 con 0x7f3df80c99e0 2024-09-13T18:00:40.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3deffff700 1 --2- 172.21.15.40:0/2731528346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df80c99e0 0x7f3df80adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:00:40.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3deffff700 1 -- 172.21.15.40:0/2731528346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3df815d9e0 con 0x7f3df80c99e0 2024-09-13T18:00:40.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3deffff700 1 --2- 172.21.15.40:0/2731528346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df80c99e0 0x7f3df80adc20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3de0016000 tx=0x7f3de0000fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:00:40.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3df4ff9700 1 -- 172.21.15.40:0/2731528346 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de0022070 con 0x7f3df80c99e0 2024-09-13T18:00:40.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3df4ff9700 1 -- 172.21.15.40:0/2731528346 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4936 (secure 0 0 0) 0x7f3de001c310 con 0x7f3df80c99e0 2024-09-13T18:00:40.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3df4ff9700 1 -- 172.21.15.40:0/2731528346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df80c99e0 msgr2=0x7f3df80adc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:40.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3df4ff9700 1 --2- 172.21.15.40:0/2731528346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df80c99e0 0x7f3df80adc20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3de0016000 tx=0x7f3de0000fb0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3df4ff9700 1 -- 172.21.15.40:0/2731528346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df80c99e0 msgr2=0x7f3df80adc20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:00:40.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3df4ff9700 1 --2- 172.21.15.40:0/2731528346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df80c99e0 0x7f3df80adc20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:00:40.818 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:00:40.818 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:00:40.818 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:00:40.818 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:00:40.818 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 80, 2024-09-13T18:00:40.818 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:00:40.818 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:00:40.818 INFO:teuthology.orchestra.run.smithi040.stdout: 1 2024-09-13T18:00:40.818 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:40.818 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:00:40.818 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:00:40.818 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:00:40.818 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:00:40.819 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:40.819 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:40.819 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:40.819 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:40.819 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:40.819 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:40.819 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:40.819 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:40.819 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:40.819 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:40.819 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:40.819 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:00:40.819 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:00:40.820 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:40.820 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:40.820 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:40.820 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:40.820 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:40.820 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:40.820 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:40.820 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:40.820 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:40.820 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:40.820 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:40.820 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.820 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:00:40.820 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:00:40.821 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:00:40.821 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:00:40.821 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:00:40.821 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:00:40.821 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:00:40.821 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:00:40.821 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:00:40.821 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:00:40.821 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:00:40.821 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:00:40.821 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:00:40.821 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:00:40.821 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:00:40.822 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:00:40.822 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:00:40.822 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:00:40.822 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:00:40.822 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:00:40.822 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:00:40.822 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:00:40.822 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:00:40.822 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:00:40.822 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:00:40.822 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:00:40.823 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:00:40.823 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:40.823 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:00:40.823 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.823 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:00:40.823 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.823 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:00:40.823 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:00:40.823 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:40.823 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:40.823 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.823 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:40.823 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:00:40.823 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:40.824 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.824 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.824 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:40.824 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:00:40.824 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:40.824 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:40.824 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:40.824 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.824 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:00:40.824 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:00:40.824 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:40.824 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:40.824 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:40.825 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.825 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.825 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:00:40.825 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:00:40.825 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:40.825 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:40.825 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.825 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:40.825 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:00:40.825 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:40.825 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.825 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.825 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:40.826 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:00:40.826 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:40.826 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:40.826 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:40.826 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.826 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:00:40.826 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:00:40.826 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:40.826 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:40.826 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:40.827 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.827 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.827 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:00:40.827 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:00:40.827 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:00:40.827 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:00:40.827 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.827 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:00:40.827 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:00:40.827 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:40.827 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.827 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.827 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:00:40.827 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:00:40.828 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:00:40.828 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:40.828 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:40.828 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.828 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:00:40.828 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:00:40.828 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:00:40.828 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:00:40.828 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:00:40.828 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:40.828 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:40.828 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.828 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:00:40.829 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:00:40.829 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.829 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:40.829 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:40.829 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:00:40.829 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:40.829 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:40.829 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:00:40.829 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.829 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:40.829 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:40.829 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:00:40.829 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:40.829 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:40.830 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:00:40.830 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.830 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:40.830 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:40.830 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:00:40.830 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:40.830 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:40.830 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:00:40.830 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.830 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:40.830 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:40.830 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:00:40.830 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:40.830 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:00:40.831 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:00:40.831 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:00:40.831 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:00:40.831 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:00:40.831 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:00:40.831 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:00:40.831 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:00:40.831 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:00:40.831 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:00:40.831 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:00:40.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3df7fff700 1 -- 172.21.15.40:0/2731528346 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3dd80666e0 msgr2=0x7f3dd8068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:40.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3df7fff700 1 --2- 172.21.15.40:0/2731528346 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3dd80666e0 0x7f3dd8068ba0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3df00013b0 tx=0x7f3df0001770 comp rx=0 tx=0).stop 2024-09-13T18:00:40.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3df7fff700 1 -- 172.21.15.40:0/2731528346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df80704a0 msgr2=0x7f3df80c6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:00:40.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3df7fff700 1 --2- 172.21.15.40:0/2731528346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3df80704a0 0x7f3df80c6a70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3de8000c00 tx=0x7f3de8004770 comp rx=0 tx=0).stop 2024-09-13T18:00:40.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3df77fe700 1 -- 172.21.15.40:0/2731528346 reap_dead start 2024-09-13T18:00:40.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3df77fe700 1 -- 172.21.15.40:0/2731528346 reap_dead start 2024-09-13T18:00:40.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3df7fff700 1 -- 172.21.15.40:0/2731528346 shutdown_connections 2024-09-13T18:00:40.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3df7fff700 1 -- 172.21.15.40:0/2731528346 >> 172.21.15.40:0/2731528346 conn(0x7f3df80ccef0 msgr2=0x7f3df80c24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:00:40.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3df7fff700 1 -- 172.21.15.40:0/2731528346 shutdown_connections 2024-09-13T18:00:40.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:00:40.808+0000 7f3df7fff700 1 -- 172.21.15.40:0/2731528346 wait complete. 2024-09-13T18:00:40.833 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:00:40.834 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:01:00.835 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-13T18:01:00.835 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-13T18:01:00.835 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T18:01:00.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T18:01:00.838 INFO:tasks.ceph.mon.c:Started 2024-09-13T18:01:00.838 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:01:00.839 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:01:00.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.952+0000 7f211bfff700 1 Processor -- start 2024-09-13T18:01:00.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.952+0000 7f211bfff700 1 -- start start 2024-09-13T18:01:00.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.952+0000 7f211bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f211c159010 0x7f211c0c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:00.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.952+0000 7f211bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f211c0c8ab0 0x7f211c0c2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:00.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.952+0000 7f211bfff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f211c0c7400 con 0x7f211c0c2a40 2024-09-13T18:01:00.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.952+0000 7f211bfff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f211c0c4ae0 con 0x7f211c0c8ab0 2024-09-13T18:01:00.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.952+0000 7f211bfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211c0c6990 con 0x7f211c159010 2024-09-13T18:01:00.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.952+0000 7f211a7fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f211c0c8ab0 0x7f211c0c2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:00.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.952+0000 7f211a7fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f211c0c8ab0 0x7f211c0c2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38006/0 (socket says 172.21.15.40:38006) 2024-09-13T18:01:00.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.952+0000 7f211a7fc700 1 -- 172.21.15.40:0/2869501385 learned_addr learned my addr 172.21.15.40:0/2869501385 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:00.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.952+0000 7f211affd700 1 --2- 172.21.15.40:0/2869501385 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f211c159010 0x7f211c0c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:00.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.952+0000 7f211a7fc700 1 -- 172.21.15.40:0/2869501385 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f211c159010 msgr2=0x7f211c0c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:01:00.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.952+0000 7f211a7fc700 1 --2- 172.21.15.40:0/2869501385 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f211c159010 0x7f211c0c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:00.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.952+0000 7f211a7fc700 1 -- 172.21.15.40:0/2869501385 >> v1:172.21.15.43:6789/0 conn(0x7f211c0c2a40 legacy=0x7f211c0c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:00.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.952+0000 7f211a7fc700 1 -- 172.21.15.40:0/2869501385 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c0c4520 con 0x7f211c0c8ab0 2024-09-13T18:01:00.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.952+0000 7f211a7fc700 1 --2- 172.21.15.40:0/2869501385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c0c8ab0 0x7f211c0c2500 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2110009910 tx=0x7f2110009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbad19cdd44aef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.956+0000 7f2119ffb700 1 -- 172.21.15.40:0/2869501385 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211000ba60 con 0x7f211c0c8ab0 2024-09-13T18:01:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.956+0000 7f2119ffb700 1 -- 172.21.15.40:0/2869501385 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211000bbc0 con 0x7f211c0c8ab0 2024-09-13T18:01:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.956+0000 7f2119ffb700 1 -- 172.21.15.40:0/2869501385 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110009e60 con 0x7f211c0c8ab0 2024-09-13T18:01:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.956+0000 7f211bfff700 1 -- 172.21.15.40:0/2869501385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c0c8ab0 msgr2=0x7f211c0c2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.956+0000 7f211bfff700 1 --2- 172.21.15.40:0/2869501385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c0c8ab0 0x7f211c0c2500 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2110009910 tx=0x7f2110009c20 comp rx=0 tx=0).stop 2024-09-13T18:01:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.956+0000 7f211bfff700 1 -- 172.21.15.40:0/2869501385 shutdown_connections 2024-09-13T18:01:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.956+0000 7f211bfff700 1 --2- 172.21.15.40:0/2869501385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c0c8ab0 0x7f211c0c2500 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.956+0000 7f211bfff700 1 --2- 172.21.15.40:0/2869501385 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f211c159010 0x7f211c0c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.956+0000 7f211bfff700 1 -- 172.21.15.40:0/2869501385 >> 172.21.15.40:0/2869501385 conn(0x7f211c0ccef0 msgr2=0x7f211c0cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.956+0000 7f211bfff700 1 -- 172.21.15.40:0/2869501385 shutdown_connections 2024-09-13T18:01:00.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.956+0000 7f211bfff700 1 -- 172.21.15.40:0/2869501385 wait complete. 2024-09-13T18:01:00.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.956+0000 7f211bfff700 1 Processor -- start 2024-09-13T18:01:00.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.956+0000 7f211bfff700 1 -- start start 2024-09-13T18:01:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f211c0c2a40 0x7f211c14d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c0c8ab0 0x7f211c1ef280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f211c159010 0x7f211c1522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211bfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c14e570 con 0x7f211c0c8ab0 2024-09-13T18:01:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211bfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c1f3050 con 0x7f211c159010 2024-09-13T18:01:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211bfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211c14dff0 con 0x7f211c0c2a40 2024-09-13T18:01:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c0c8ab0 0x7f211c1ef280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211a7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c0c8ab0 0x7f211c1ef280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38016/0 (socket says 172.21.15.40:38016) 2024-09-13T18:01:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211a7fc700 1 -- 172.21.15.40:0/2186969759 learned_addr learned my addr 172.21.15.40:0/2186969759 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211affd700 1 --2- 172.21.15.40:0/2186969759 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f211c0c2a40 0x7f211c14d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:00.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211b7fe700 1 --2- 172.21.15.40:0/2186969759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f211c159010 0x7f211c1522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:00.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211affd700 1 -- 172.21.15.40:0/2186969759 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f211c0c2a40 msgr2=0x7f211c14d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:01:00.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211affd700 1 -- 172.21.15.40:0/2186969759 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f211c0c2a40 msgr2=0x7f211c14d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:01:00.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211a7fc700 1 -- 172.21.15.40:0/2186969759 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f211c0c2a40 msgr2=0x7f211c14d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:00.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211affd700 1 --2- 172.21.15.40:0/2186969759 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f211c0c2a40 0x7f211c14d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:01:00.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211affd700 1 --2- 172.21.15.40:0/2186969759 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f211c0c2a40 0x7f211c14d960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:00.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211a7fc700 1 --2- 172.21.15.40:0/2186969759 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f211c0c2a40 0x7f211c14d960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:00.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211a7fc700 1 -- 172.21.15.40:0/2186969759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f211c159010 msgr2=0x7f211c1522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:00.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211a7fc700 1 --2- 172.21.15.40:0/2186969759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f211c159010 0x7f211c1522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:00.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211a7fc700 1 -- 172.21.15.40:0/2186969759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2110009570 con 0x7f211c0c8ab0 2024-09-13T18:01:00.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211b7fe700 1 --2- 172.21.15.40:0/2186969759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f211c159010 0x7f211c1522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:01:00.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211a7fc700 1 --2- 172.21.15.40:0/2186969759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c0c8ab0 0x7f211c1ef280 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2110000c00 tx=0x7f2110004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:00.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f21037fe700 1 -- 172.21.15.40:0/2186969759 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110004520 con 0x7f211c0c8ab0 2024-09-13T18:01:00.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f21037fe700 1 -- 172.21.15.40:0/2186969759 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211001e070 con 0x7f211c0c8ab0 2024-09-13T18:01:00.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211bfff700 1 -- 172.21.15.40:0/2186969759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211c14e290 con 0x7f211c0c8ab0 2024-09-13T18:01:00.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211bfff700 1 -- 172.21.15.40:0/2186969759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211c1f2c70 con 0x7f211c0c8ab0 2024-09-13T18:01:00.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f21037fe700 1 -- 172.21.15.40:0/2186969759 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110003900 con 0x7f211c0c8ab0 2024-09-13T18:01:00.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.960+0000 7f211bfff700 1 -- 172.21.15.40:0/2186969759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f211c1f27f0 con 0x7f211c0c8ab0 2024-09-13T18:01:00.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.964+0000 7f21037fe700 1 -- 172.21.15.40:0/2186969759 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2110003a60 con 0x7f211c0c8ab0 2024-09-13T18:01:00.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.964+0000 7f21037fe700 1 --2- 172.21.15.40:0/2186969759 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2108066730 0x7f2108068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:00.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.964+0000 7f21037fe700 1 -- 172.21.15.40:0/2186969759 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2110020070 con 0x7f211c0c8ab0 2024-09-13T18:01:00.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.964+0000 7f211affd700 1 --2- 172.21.15.40:0/2186969759 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2108066730 0x7f2108068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:00.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.964+0000 7f211affd700 1 --2- 172.21.15.40:0/2186969759 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2108066730 0x7f2108068bf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f210c009910 tx=0x7f210c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:00.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:00.968+0000 7f21037fe700 1 -- 172.21.15.40:0/2186969759 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f211c1f27f0 con 0x7f211c0c8ab0 2024-09-13T18:01:01.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:01.220+0000 7f211bfff700 1 -- 172.21.15.40:0/2186969759 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f211c1f27f0 con 0x7f211c0c8ab0 2024-09-13T18:01:01.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:01.220+0000 7f21037fe700 1 -- 172.21.15.40:0/2186969759 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f211c1f27f0 con 0x7f211c0c8ab0 2024-09-13T18:01:01.231 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":80,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:01:01.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:01.224+0000 7f211bfff700 1 -- 172.21.15.40:0/2186969759 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2108066730 msgr2=0x7f2108068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:01.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:01.224+0000 7f211bfff700 1 --2- 172.21.15.40:0/2186969759 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2108066730 0x7f2108068bf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f210c009910 tx=0x7f210c008040 comp rx=0 tx=0).stop 2024-09-13T18:01:01.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:01.224+0000 7f211bfff700 1 -- 172.21.15.40:0/2186969759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c0c8ab0 msgr2=0x7f211c1ef280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:01.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:01.224+0000 7f211bfff700 1 --2- 172.21.15.40:0/2186969759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c0c8ab0 0x7f211c1ef280 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2110000c00 tx=0x7f2110004770 comp rx=0 tx=0).stop 2024-09-13T18:01:01.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:01.224+0000 7f211bfff700 1 -- 172.21.15.40:0/2186969759 shutdown_connections 2024-09-13T18:01:01.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:01.224+0000 7f211bfff700 1 --2- 172.21.15.40:0/2186969759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f211c159010 0x7f211c1522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:01.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:01.224+0000 7f211bfff700 1 --2- 172.21.15.40:0/2186969759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f211c0c8ab0 0x7f211c1ef280 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:01.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:01.224+0000 7f211bfff700 1 --2- 172.21.15.40:0/2186969759 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2108066730 0x7f2108068bf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:01.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:01.224+0000 7f211bfff700 1 --2- 172.21.15.40:0/2186969759 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f211c0c2a40 0x7f211c14d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:01.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:01.224+0000 7f211bfff700 1 -- 172.21.15.40:0/2186969759 >> 172.21.15.40:0/2186969759 conn(0x7f211c0ccef0 msgr2=0x7f211c154820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:01.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:01.224+0000 7f211bfff700 1 -- 172.21.15.40:0/2186969759 shutdown_connections 2024-09-13T18:01:01.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:01.224+0000 7f211bfff700 1 -- 172.21.15.40:0/2186969759 wait complete. 2024-09-13T18:01:04.247 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:01:04.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f2026aaa700 1 Processor -- start 2024-09-13T18:01:04.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f2026aaa700 1 -- start start 2024-09-13T18:01:04.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f2026aaa700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f20180d8920 0x7f2018049090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f2026aaa700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f20180e1c00 0x7f20180e2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f2026aaa700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2018054df0 con 0x7f2018007480 2024-09-13T18:01:04.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f2026aaa700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2018050a10 con 0x7f20180e1c00 2024-09-13T18:01:04.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f2026aaa700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2018006ff0 con 0x7f20180d8920 2024-09-13T18:01:04.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f2025aa8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f20180d8920 0x7f2018049090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f20262a9700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f20180e1c00 0x7f20180e2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f20262a9700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f20180e1c00 0x7f20180e2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:57016/0 (socket says 172.21.15.40:57016) 2024-09-13T18:01:04.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f20262a9700 1 -- 172.21.15.40:0/816080060 learned_addr learned my addr 172.21.15.40:0/816080060 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:04.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f2025aa8700 1 -- 172.21.15.40:0/816080060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f20180d8920 msgr2=0x7f2018049090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:01:04.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f2025aa8700 1 -- 172.21.15.40:0/816080060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f20180d8920 msgr2=0x7f2018049090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:01:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f2025aa8700 1 --2- 172.21.15.40:0/816080060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f20180d8920 0x7f2018049090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:01:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f2025aa8700 1 --2- 172.21.15.40:0/816080060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f20180d8920 0x7f2018049090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f20262a9700 1 -- 172.21.15.40:0/816080060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f20180d8920 msgr2=0x7f2018049090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:01:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f20262a9700 1 --2- 172.21.15.40:0/816080060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f20180d8920 0x7f2018049090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f20262a9700 1 -- 172.21.15.40:0/816080060 >> v1:172.21.15.43:6789/0 conn(0x7f2018007480 legacy=0x7f201804efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f20262a9700 1 -- 172.21.15.40:0/816080060 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20180495d0 con 0x7f20180e1c00 2024-09-13T18:01:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f20262a9700 1 --2- 172.21.15.40:0/816080060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20180e1c00 0x7f20180e2070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2020063990 tx=0x7f2020069e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7219a166190196c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f2024aa6700 1 -- 172.21.15.40:0/816080060 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020068a70 con 0x7f20180e1c00 2024-09-13T18:01:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f2024aa6700 1 -- 172.21.15.40:0/816080060 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2020067dd0 con 0x7f20180e1c00 2024-09-13T18:01:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.364+0000 7f2024aa6700 1 -- 172.21.15.40:0/816080060 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20200744e0 con 0x7f20180e1c00 2024-09-13T18:01:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.368+0000 7f2026aaa700 1 -- 172.21.15.40:0/816080060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20180e1c00 msgr2=0x7f20180e2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.368+0000 7f2026aaa700 1 --2- 172.21.15.40:0/816080060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20180e1c00 0x7f20180e2070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2020063990 tx=0x7f2020069e70 comp rx=0 tx=0).stop 2024-09-13T18:01:04.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.368+0000 7f2026aaa700 1 -- 172.21.15.40:0/816080060 shutdown_connections 2024-09-13T18:01:04.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.368+0000 7f2026aaa700 1 --2- 172.21.15.40:0/816080060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20180e1c00 0x7f20180e2070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.368+0000 7f2026aaa700 1 --2- 172.21.15.40:0/816080060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f20180d8920 0x7f2018049090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.368+0000 7f2026aaa700 1 -- 172.21.15.40:0/816080060 >> 172.21.15.40:0/816080060 conn(0x7f2018054110 msgr2=0x7f2018054520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:04.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.368+0000 7f2026aaa700 1 -- 172.21.15.40:0/816080060 shutdown_connections 2024-09-13T18:01:04.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.368+0000 7f2026aaa700 1 -- 172.21.15.40:0/816080060 wait complete. 2024-09-13T18:01:04.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.368+0000 7f2026aaa700 1 Processor -- start 2024-09-13T18:01:04.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.368+0000 7f2026aaa700 1 -- start start 2024-09-13T18:01:04.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.368+0000 7f2026aaa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2018007480 0x7f20180cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.368+0000 7f2026aaa700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20180d8920 0x7f20180d8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f2026aaa700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f20180e1c00 0x7f20180c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f2026aaa700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20180d9e90 con 0x7f2018007480 2024-09-13T18:01:04.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f2026aaa700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20180e1a50 con 0x7f20180d8920 2024-09-13T18:01:04.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f2026aaa700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201803a110 con 0x7f20180e1c00 2024-09-13T18:01:04.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f2025aa8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2018007480 0x7f20180cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f2025aa8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2018007480 0x7f20180cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54158/0 (socket says 172.21.15.40:54158) 2024-09-13T18:01:04.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f2025aa8700 1 -- 172.21.15.40:0/824644018 learned_addr learned my addr 172.21.15.40:0/824644018 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:04.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f20252a7700 1 --2- 172.21.15.40:0/824644018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20180d8920 0x7f20180d8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f2025aa8700 1 -- 172.21.15.40:0/824644018 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f20180e1c00 msgr2=0x7f20180c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f20262a9700 1 --2- 172.21.15.40:0/824644018 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f20180e1c00 0x7f20180c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f2025aa8700 1 --2- 172.21.15.40:0/824644018 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f20180e1c00 0x7f20180c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f2025aa8700 1 -- 172.21.15.40:0/824644018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20180d8920 msgr2=0x7f20180d8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f2025aa8700 1 --2- 172.21.15.40:0/824644018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20180d8920 0x7f20180d8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f2025aa8700 1 -- 172.21.15.40:0/824644018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202004f5c0 con 0x7f2018007480 2024-09-13T18:01:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f2025aa8700 1 --2- 172.21.15.40:0/824644018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2018007480 0x7f20180cddf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f201c00e7e0 tx=0x7f201c00eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f20167fc700 1 -- 172.21.15.40:0/824644018 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c013610 con 0x7f2018007480 2024-09-13T18:01:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f20167fc700 1 -- 172.21.15.40:0/824644018 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201c013770 con 0x7f2018007480 2024-09-13T18:01:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f20167fc700 1 -- 172.21.15.40:0/824644018 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c013b20 con 0x7f2018007480 2024-09-13T18:01:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f2026aaa700 1 -- 172.21.15.40:0/824644018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2018053da0 con 0x7f2018007480 2024-09-13T18:01:04.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f2026aaa700 1 -- 172.21.15.40:0/824644018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2018053530 con 0x7f2018007480 2024-09-13T18:01:04.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.372+0000 7f2026aaa700 1 -- 172.21.15.40:0/824644018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2018050730 con 0x7f2018007480 2024-09-13T18:01:04.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.376+0000 7f20167fc700 1 -- 172.21.15.40:0/824644018 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f201c013c80 con 0x7f2018007480 2024-09-13T18:01:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.376+0000 7f20167fc700 1 --2- 172.21.15.40:0/824644018 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2004066730 0x7f2004068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.376+0000 7f20167fc700 1 -- 172.21.15.40:0/824644018 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f201c0845a0 con 0x7f2018007480 2024-09-13T18:01:04.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.376+0000 7f20252a7700 1 --2- 172.21.15.40:0/824644018 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2004066730 0x7f2004068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.376+0000 7f20252a7700 1 --2- 172.21.15.40:0/824644018 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2004066730 0x7f2004068bf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2010007fd0 tx=0x7f2010009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:04.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.384+0000 7f20167fc700 1 -- 172.21.15.40:0/824644018 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f201c051550 con 0x7f2018007480 2024-09-13T18:01:04.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.648+0000 7f2026aaa700 1 -- 172.21.15.40:0/824644018 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f201804fb60 con 0x7f2018007480 2024-09-13T18:01:04.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.648+0000 7f20167fc700 1 -- 172.21.15.40:0/824644018 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7f201c054b70 con 0x7f2018007480 2024-09-13T18:01:04.656 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":82,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:01:04.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.648+0000 7f2026aaa700 1 -- 172.21.15.40:0/824644018 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2004066730 msgr2=0x7f2004068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.648+0000 7f2026aaa700 1 --2- 172.21.15.40:0/824644018 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2004066730 0x7f2004068bf0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2010007fd0 tx=0x7f2010009040 comp rx=0 tx=0).stop 2024-09-13T18:01:04.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.648+0000 7f2026aaa700 1 -- 172.21.15.40:0/824644018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2018007480 msgr2=0x7f20180cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.648+0000 7f2026aaa700 1 --2- 172.21.15.40:0/824644018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2018007480 0x7f20180cddf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f201c00e7e0 tx=0x7f201c00eba0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.648+0000 7f2026aaa700 1 -- 172.21.15.40:0/824644018 shutdown_connections 2024-09-13T18:01:04.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.648+0000 7f2026aaa700 1 --2- 172.21.15.40:0/824644018 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2004066730 0x7f2004068bf0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.648+0000 7f2026aaa700 1 --2- 172.21.15.40:0/824644018 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f20180e1c00 0x7f20180c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.648+0000 7f2026aaa700 1 --2- 172.21.15.40:0/824644018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20180d8920 0x7f20180d8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.648+0000 7f2026aaa700 1 --2- 172.21.15.40:0/824644018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2018007480 0x7f20180cddf0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.648+0000 7f2026aaa700 1 -- 172.21.15.40:0/824644018 >> 172.21.15.40:0/824644018 conn(0x7f2018054110 msgr2=0x7f2018049760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:04.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.648+0000 7f2026aaa700 1 -- 172.21.15.40:0/824644018 shutdown_connections 2024-09-13T18:01:04.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.648+0000 7f2026aaa700 1 -- 172.21.15.40:0/824644018 wait complete. 2024-09-13T18:01:04.670 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:01:04.670 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:01:04.670 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:01:04.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.776+0000 7f39f08f8700 1 Processor -- start 2024-09-13T18:01:04.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.776+0000 7f39f08f8700 1 -- start start 2024-09-13T18:01:04.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39f08f8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39ec1518c0 0x7f39ec157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39f08f8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f39ec070460 0x7f39ec157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39f08f8700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39ec0c8cc0 con 0x7f39ec1514e0 2024-09-13T18:01:04.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39f08f8700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39ec157390 con 0x7f39ec070460 2024-09-13T18:01:04.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39f08f8700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec0c8fa0 con 0x7f39ec1518c0 2024-09-13T18:01:04.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39eb59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f39ec070460 0x7f39ec157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39eb59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f39ec070460 0x7f39ec157fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54164/0 (socket says 172.21.15.40:54164) 2024-09-13T18:01:04.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39eb59e700 1 -- 172.21.15.40:0/3555419214 learned_addr learned my addr 172.21.15.40:0/3555419214 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:04.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39ea59c700 1 --2- 172.21.15.40:0/3555419214 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39ec1518c0 0x7f39ec157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39eb59e700 1 -- 172.21.15.40:0/3555419214 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39ec1518c0 msgr2=0x7f39ec157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:01:04.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39eb59e700 1 --2- 172.21.15.40:0/3555419214 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39ec1518c0 0x7f39ec157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39eb59e700 1 -- 172.21.15.40:0/3555419214 >> v1:172.21.15.40:6789/0 conn(0x7f39ec1514e0 legacy=0x7f39ec0b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39eb59e700 1 -- 172.21.15.40:0/3555419214 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ec0c9280 con 0x7f39ec070460 2024-09-13T18:01:04.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39eb59e700 1 --2- 172.21.15.40:0/3555419214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec070460 0x7f39ec157fe0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009e50 tx=0x7f39e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d4adf9128304ab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:04.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39e9d9b700 1 -- 172.21.15.40:0/3555419214 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e400e070 con 0x7f39ec070460 2024-09-13T18:01:04.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39e9d9b700 1 -- 172.21.15.40:0/3555419214 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e400b9c0 con 0x7f39ec070460 2024-09-13T18:01:04.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39e9d9b700 1 -- 172.21.15.40:0/3555419214 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e400bd70 con 0x7f39ec070460 2024-09-13T18:01:04.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39f08f8700 1 -- 172.21.15.40:0/3555419214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec070460 msgr2=0x7f39ec157fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39f08f8700 1 --2- 172.21.15.40:0/3555419214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec070460 0x7f39ec157fe0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009e50 tx=0x7f39e400b330 comp rx=0 tx=0).stop 2024-09-13T18:01:04.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39f08f8700 1 -- 172.21.15.40:0/3555419214 shutdown_connections 2024-09-13T18:01:04.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39f08f8700 1 --2- 172.21.15.40:0/3555419214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec070460 0x7f39ec157fe0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39f08f8700 1 --2- 172.21.15.40:0/3555419214 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39ec1518c0 0x7f39ec157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39f08f8700 1 -- 172.21.15.40:0/3555419214 >> 172.21.15.40:0/3555419214 conn(0x7f39ec0ccef0 msgr2=0x7f39ec0cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:04.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39f08f8700 1 -- 172.21.15.40:0/3555419214 shutdown_connections 2024-09-13T18:01:04.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.780+0000 7f39f08f8700 1 -- 172.21.15.40:0/3555419214 wait complete. 2024-09-13T18:01:04.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39f08f8700 1 Processor -- start 2024-09-13T18:01:04.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39f08f8700 1 -- start start 2024-09-13T18:01:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39f08f8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec070460 0x7f39ec0c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39f08f8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39ec1514e0 0x7f39ec150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39f08f8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ec1518c0 0x7f39ec1439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39f08f8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec152310 con 0x7f39ec070460 2024-09-13T18:01:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39f08f8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec14f2f0 con 0x7f39ec1518c0 2024-09-13T18:01:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39f08f8700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec14f5d0 con 0x7f39ec1514e0 2024-09-13T18:01:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39ead9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec070460 0x7f39ec0c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39ead9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec070460 0x7f39ec0c4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54172/0 (socket says 172.21.15.40:54172) 2024-09-13T18:01:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39ead9d700 1 -- 172.21.15.40:0/337699782 learned_addr learned my addr 172.21.15.40:0/337699782 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39eb59e700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ec1518c0 0x7f39ec1439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39ea59c700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39ec1514e0 0x7f39ec150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39ead9d700 1 -- 172.21.15.40:0/337699782 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39ec1514e0 msgr2=0x7f39ec150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39ead9d700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39ec1514e0 0x7f39ec150670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39ead9d700 1 -- 172.21.15.40:0/337699782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ec1518c0 msgr2=0x7f39ec1439e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39ead9d700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ec1518c0 0x7f39ec1439e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39ead9d700 1 -- 172.21.15.40:0/337699782 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e4009b30 con 0x7f39ec070460 2024-09-13T18:01:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39eb59e700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ec1518c0 0x7f39ec1439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:01:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39ead9d700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec070460 0x7f39ec0c4710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00c500 tx=0x7f39dc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39d37fe700 1 -- 172.21.15.40:0/337699782 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc0052d0 con 0x7f39ec070460 2024-09-13T18:01:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39d37fe700 1 -- 172.21.15.40:0/337699782 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39dc005430 con 0x7f39ec070460 2024-09-13T18:01:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39f08f8700 1 -- 172.21.15.40:0/337699782 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ec0c8a90 con 0x7f39ec070460 2024-09-13T18:01:04.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39d37fe700 1 -- 172.21.15.40:0/337699782 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc0055a0 con 0x7f39ec070460 2024-09-13T18:01:04.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.784+0000 7f39f08f8700 1 -- 172.21.15.40:0/337699782 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ec14fb90 con 0x7f39ec070460 2024-09-13T18:01:04.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.788+0000 7f39d37fe700 1 -- 172.21.15.40:0/337699782 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f39dc0038e0 con 0x7f39ec070460 2024-09-13T18:01:04.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.788+0000 7f39f08f8700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec1f0070 0x7f39ec14fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.788+0000 7f39f08f8700 1 -- 172.21.15.40:0/337699782 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec074830 con 0x7f39ec1f0070 2024-09-13T18:01:04.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.788+0000 7f39ea59c700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec1f0070 0x7f39ec14fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.788+0000 7f39d37fe700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f39d8066730 0x7f39d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.788+0000 7f39d37fe700 1 -- 172.21.15.40:0/337699782 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f39dc083a10 con 0x7f39ec070460 2024-09-13T18:01:04.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.788+0000 7f39eb59e700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f39d8066730 0x7f39d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.788+0000 7f39ea59c700 1 -- 172.21.15.40:0/337699782 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39ec0b2a90 con 0x7f39ec1f0070 2024-09-13T18:01:04.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.788+0000 7f39ea59c700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec1f0070 0x7f39ec14fdb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f39ec1446e0 tx=0x7f39e0010b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.788+0000 7f39d37fe700 1 -- 172.21.15.40:0/337699782 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0015880 con 0x7f39ec1f0070 2024-09-13T18:01:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.788+0000 7f39eb59e700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f39d8066730 0x7f39d8068bf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f39e400b370 tx=0x7f39e4009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.788+0000 7f39d37fe700 1 -- 172.21.15.40:0/337699782 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f39e000c050 con 0x7f39ec1f0070 2024-09-13T18:01:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.788+0000 7f39d37fe700 1 -- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec1f0070 msgr2=0x7f39ec14fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.788+0000 7f39d37fe700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec1f0070 0x7f39ec14fdb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f39ec1446e0 tx=0x7f39e0010b20 comp rx=0 tx=0).stop 2024-09-13T18:01:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.788+0000 7f39d37fe700 1 -- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec1f0070 msgr2=0x7f39ec14fdb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:01:04.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.788+0000 7f39d37fe700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec1f0070 0x7f39ec14fdb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39f08f8700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec1415c0 0x7f39ec1419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39f08f8700 1 -- 172.21.15.40:0/337699782 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec157580 con 0x7f39ec1415c0 2024-09-13T18:01:04.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39ea59c700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec1415c0 0x7f39ec1419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39ea59c700 1 -- 172.21.15.40:0/337699782 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f39ec067520 con 0x7f39ec1415c0 2024-09-13T18:01:04.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39ea59c700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec1415c0 0x7f39ec1419d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f39e001b000 tx=0x7f39e0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:04.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39d37fe700 1 -- 172.21.15.40:0/337699782 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0021ad0 con 0x7f39ec1415c0 2024-09-13T18:01:04.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39d37fe700 1 -- 172.21.15.40:0/337699782 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4783 (secure 0 0 0) 0x7f39e0021300 con 0x7f39ec1415c0 2024-09-13T18:01:04.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39d37fe700 1 -- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec1415c0 msgr2=0x7f39ec1419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39d37fe700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec1415c0 0x7f39ec1419d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f39e001b000 tx=0x7f39e0004770 comp rx=0 tx=0).stop 2024-09-13T18:01:04.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39d37fe700 1 -- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec1415c0 msgr2=0x7f39ec1419d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:01:04.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39d37fe700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec1415c0 0x7f39ec1419d0 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f39e001b000 tx=0x7f39e0004770 comp rx=0 tx=0).stop 2024-09-13T18:01:04.815 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:01:04.815 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:01:04.815 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:01:04.815 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:01:04.815 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 82, 2024-09-13T18:01:04.815 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:01:04.815 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:01:04.815 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:01:04.815 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:01:04.815 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:04.815 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:01:04.815 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:01:04.815 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:01:04.816 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:01:04.816 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:04.816 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:04.816 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:04.816 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:04.816 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:04.816 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:04.816 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:04.816 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:04.816 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:04.816 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:04.816 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:04.816 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:01:04.817 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:01:04.817 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:04.817 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:04.817 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:04.817 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:04.817 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:04.817 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:04.817 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:04.817 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:04.817 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:04.817 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:04.817 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:04.817 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.817 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:01:04.818 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:01:04.818 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:01:04.818 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:01:04.818 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:01:04.818 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:01:04.818 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:01:04.818 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:01:04.818 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:01:04.818 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:01:04.818 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:01:04.818 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:01:04.818 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:01:04.818 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:01:04.818 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:01:04.819 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:01:04.819 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:01:04.819 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:04.819 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:04.819 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:04.819 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:04.819 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:04.819 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:04.819 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:04.819 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:04.819 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:04.819 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:04.819 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:04.820 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:01:04.820 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.820 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:01:04.820 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.820 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:01:04.820 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:01:04.820 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:04.820 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:04.820 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.820 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:04.820 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:01:04.820 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:04.820 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.820 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.821 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:04.821 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:01:04.821 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:04.821 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:04.821 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:04.821 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.821 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:01:04.821 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:01:04.821 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:04.821 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:04.821 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:04.821 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.821 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.821 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:01:04.822 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:01:04.822 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:04.822 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:04.822 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.822 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:04.822 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:01:04.822 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:04.822 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.822 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.822 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:04.822 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:01:04.822 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:04.822 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:04.822 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:04.823 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.823 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:01:04.823 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:01:04.823 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:04.823 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:04.823 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:04.823 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.823 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.823 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:01:04.823 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:01:04.823 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:04.823 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:04.823 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.823 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:04.824 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:01:04.824 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:04.824 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.824 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.824 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:04.824 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:01:04.824 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:04.824 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:04.824 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:04.824 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.824 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:01:04.824 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:01:04.824 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:04.824 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:04.825 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:04.825 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:04.825 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:04.825 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.825 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:01:04.825 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:01:04.825 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.825 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:04.825 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:04.825 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:01:04.825 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:04.825 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:04.825 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:01:04.825 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.826 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:04.826 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:04.826 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:01:04.826 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:04.826 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:04.826 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:01:04.826 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.826 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:04.826 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:04.826 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:01:04.826 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:04.826 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:04.826 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:01:04.826 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.827 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:04.827 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:04.827 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:01:04.827 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:04.827 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:04.827 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.827 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:01:04.827 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:01:04.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39f08f8700 1 -- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f39d8066730 msgr2=0x7f39d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39f08f8700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f39d8066730 0x7f39d8068bf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f39e400b370 tx=0x7f39e4009000 comp rx=0 tx=0).stop 2024-09-13T18:01:04.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39f08f8700 1 -- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec070460 msgr2=0x7f39ec0c4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39f08f8700 1 --2- 172.21.15.40:0/337699782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39ec070460 0x7f39ec0c4710 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00c500 tx=0x7f39dc00c8c0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39eb59e700 1 -- 172.21.15.40:0/337699782 reap_dead start 2024-09-13T18:01:04.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39eb59e700 1 -- 172.21.15.40:0/337699782 reap_dead start 2024-09-13T18:01:04.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39f08f8700 1 -- 172.21.15.40:0/337699782 shutdown_connections 2024-09-13T18:01:04.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39f08f8700 1 -- 172.21.15.40:0/337699782 >> 172.21.15.40:0/337699782 conn(0x7f39ec0ccef0 msgr2=0x7f39ec06dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:04.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39f08f8700 1 -- 172.21.15.40:0/337699782 shutdown_connections 2024-09-13T18:01:04.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.804+0000 7f39f08f8700 1 -- 172.21.15.40:0/337699782 wait complete. 2024-09-13T18:01:04.830 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:01:04.830 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:01:04.830 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:01:04.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.944+0000 7f2422295700 1 Processor -- start 2024-09-13T18:01:04.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.944+0000 7f2422295700 1 -- start start 2024-09-13T18:01:04.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.944+0000 7f2422295700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f241c0cf1a0 0x7f241c15a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.944+0000 7f2422295700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f241c0cc250 0x7f241c0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.944+0000 7f2422295700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f241c15a7a0 con 0x7f241c0cf580 2024-09-13T18:01:04.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.944+0000 7f2422295700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f241c0cbdc0 con 0x7f241c0cc250 2024-09-13T18:01:04.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.944+0000 7f2422295700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f241c153f80 con 0x7f241c0cf1a0 2024-09-13T18:01:04.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.944+0000 7f2421a94700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f241c0cc250 0x7f241c0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.944+0000 7f2421293700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f241c0cf1a0 0x7f241c15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.944+0000 7f2421a94700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f241c0cc250 0x7f241c0c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54194/0 (socket says 172.21.15.40:54194) 2024-09-13T18:01:04.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.944+0000 7f2421a94700 1 -- 172.21.15.40:0/3085712205 learned_addr learned my addr 172.21.15.40:0/3085712205 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:04.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.944+0000 7f2421a94700 1 -- 172.21.15.40:0/3085712205 >> v2:172.21.15.43:3300/0 conn(0x7f241c0cf1a0 msgr2=0x7f241c15a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:01:04.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.944+0000 7f2421a94700 1 --2- 172.21.15.40:0/3085712205 >> v2:172.21.15.43:3300/0 conn(0x7f241c0cf1a0 0x7f241c15a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.944+0000 7f2421a94700 1 -- 172.21.15.40:0/3085712205 >> v1:172.21.15.43:6789/0 conn(0x7f241c0cf580 legacy=0x7f241c0c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.944+0000 7f2421a94700 1 -- 172.21.15.40:0/3085712205 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c0caed0 con 0x7f241c0cc250 2024-09-13T18:01:04.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.944+0000 7f2421a94700 1 --2- 172.21.15.40:0/3085712205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f241c0cc250 0x7f241c0c8aa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f241c067020 tx=0x7f241400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b24ba840412122 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:04.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.944+0000 7f2413fff700 1 -- 172.21.15.40:0/3085712205 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241400e070 con 0x7f241c0cc250 2024-09-13T18:01:04.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.944+0000 7f2413fff700 1 -- 172.21.15.40:0/3085712205 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241400b910 con 0x7f241c0cc250 2024-09-13T18:01:04.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.944+0000 7f2413fff700 1 -- 172.21.15.40:0/3085712205 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241400bcc0 con 0x7f241c0cc250 2024-09-13T18:01:04.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.948+0000 7f2422295700 1 -- 172.21.15.40:0/3085712205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f241c0cc250 msgr2=0x7f241c0c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.948+0000 7f2422295700 1 --2- 172.21.15.40:0/3085712205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f241c0cc250 0x7f241c0c8aa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f241c067020 tx=0x7f241400b330 comp rx=0 tx=0).stop 2024-09-13T18:01:04.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.948+0000 7f2422295700 1 -- 172.21.15.40:0/3085712205 shutdown_connections 2024-09-13T18:01:04.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.948+0000 7f2422295700 1 --2- 172.21.15.40:0/3085712205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f241c0cc250 0x7f241c0c8aa0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.948+0000 7f2422295700 1 --2- 172.21.15.40:0/3085712205 >> v2:172.21.15.43:3300/0 conn(0x7f241c0cf1a0 0x7f241c15a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.948+0000 7f2422295700 1 -- 172.21.15.40:0/3085712205 >> 172.21.15.40:0/3085712205 conn(0x7f241c066090 msgr2=0x7f241c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:04.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.948+0000 7f2422295700 1 -- 172.21.15.40:0/3085712205 shutdown_connections 2024-09-13T18:01:04.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.948+0000 7f2422295700 1 -- 172.21.15.40:0/3085712205 wait complete. 2024-09-13T18:01:04.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.948+0000 7f2422295700 1 Processor -- start 2024-09-13T18:01:04.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.948+0000 7f2422295700 1 -- start start 2024-09-13T18:01:04.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2422295700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f241c0cc250 0x7f241c1563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2422295700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f241c0cf1a0 0x7f241c1513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2422295700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f241c0cf580 0x7f241c151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2422295700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c159fe0 con 0x7f241c0cf580 2024-09-13T18:01:04.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2422295700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c159db0 con 0x7f241c0cc250 2024-09-13T18:01:04.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2422295700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241c159470 con 0x7f241c0cf1a0 2024-09-13T18:01:04.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2421a94700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f241c0cf580 0x7f241c151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2421a94700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f241c0cf580 0x7f241c151910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54196/0 (socket says 172.21.15.40:54196) 2024-09-13T18:01:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2421a94700 1 -- 172.21.15.40:0/2528645940 learned_addr learned my addr 172.21.15.40:0/2528645940 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2421293700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f241c0cc250 0x7f241c1563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2420a92700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f241c0cf1a0 0x7f241c1513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2420a92700 1 -- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f241c0cf1a0 msgr2=0x7f241c1513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:01:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2420a92700 1 -- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f241c0cf1a0 msgr2=0x7f241c1513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:01:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2420a92700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f241c0cf1a0 0x7f241c1513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:01:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2420a92700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f241c0cf1a0 0x7f241c1513d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2421a94700 1 -- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f241c0cf1a0 msgr2=0x7f241c1513d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:01:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2421a94700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f241c0cf1a0 0x7f241c1513d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2421a94700 1 -- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f241c0cc250 msgr2=0x7f241c1563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2421a94700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f241c0cc250 0x7f241c1563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2421a94700 1 -- 172.21.15.40:0/2528645940 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2414009b30 con 0x7f241c0cf580 2024-09-13T18:01:04.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2421a94700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f241c0cf580 0x7f241c151910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f241c08c270 tx=0x7f2414004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:04.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2411ffb700 1 -- 172.21.15.40:0/2528645940 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414004930 con 0x7f241c0cf580 2024-09-13T18:01:04.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2411ffb700 1 -- 172.21.15.40:0/2528645940 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24140092e0 con 0x7f241c0cf580 2024-09-13T18:01:04.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2422295700 1 -- 172.21.15.40:0/2528645940 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241c159710 con 0x7f241c0cf580 2024-09-13T18:01:04.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2411ffb700 1 -- 172.21.15.40:0/2528645940 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241401c660 con 0x7f241c0cf580 2024-09-13T18:01:04.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.952+0000 7f2422295700 1 -- 172.21.15.40:0/2528645940 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241c15a510 con 0x7f241c0cf580 2024-09-13T18:01:04.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.956+0000 7f2422295700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f241c0c57f0 0x7f241c1eec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.956+0000 7f2422295700 1 -- 172.21.15.40:0/2528645940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c074750 con 0x7f241c0c57f0 2024-09-13T18:01:04.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.956+0000 7f2411ffb700 1 -- 172.21.15.40:0/2528645940 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f241401c7c0 con 0x7f241c0cf580 2024-09-13T18:01:04.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.956+0000 7f2421293700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f241c0c57f0 0x7f241c1eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.956+0000 7f2411ffb700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2400066730 0x7f2400068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.956+0000 7f2421293700 1 -- 172.21.15.40:0/2528645940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f241c0cb770 con 0x7f241c0c57f0 2024-09-13T18:01:04.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.956+0000 7f2411ffb700 1 -- 172.21.15.40:0/2528645940 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2414085df0 con 0x7f241c0cf580 2024-09-13T18:01:04.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.956+0000 7f2420a92700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2400066730 0x7f2400068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.956+0000 7f2421293700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f241c0c57f0 0x7f241c1eec70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f240c0119d0 tx=0x7f240c011d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:04.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.956+0000 7f2411ffb700 1 -- 172.21.15.40:0/2528645940 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c004240 con 0x7f241c0c57f0 2024-09-13T18:01:04.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.956+0000 7f2420a92700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2400066730 0x7f2400068bf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f241c1ec550 tx=0x7f2418009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:04.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.956+0000 7f2411ffb700 1 -- 172.21.15.40:0/2528645940 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f240c0043a0 con 0x7f241c0c57f0 2024-09-13T18:01:04.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.956+0000 7f2411ffb700 1 -- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f241c0c57f0 msgr2=0x7f241c1eec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.956+0000 7f2411ffb700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f241c0c57f0 0x7f241c1eec70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f240c0119d0 tx=0x7f240c011d90 comp rx=0 tx=0).stop 2024-09-13T18:01:04.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.956+0000 7f2411ffb700 1 -- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f241c0c57f0 msgr2=0x7f241c1eec70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:01:04.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.956+0000 7f2411ffb700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f241c0c57f0 0x7f241c1eec70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.968+0000 7f2422295700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f241c0cea90 0x7f241c1ef1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:04.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.968+0000 7f2422295700 1 -- 172.21.15.40:0/2528645940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c0cb490 con 0x7f241c0cea90 2024-09-13T18:01:04.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.968+0000 7f2421293700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f241c0cea90 0x7f241c1ef1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:04.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.968+0000 7f2421293700 1 -- 172.21.15.40:0/2528645940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f241c0cba50 con 0x7f241c0cea90 2024-09-13T18:01:04.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.968+0000 7f2421293700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f241c0cea90 0x7f241c1ef1b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f240c01b000 tx=0x7f240c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:04.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.968+0000 7f2411ffb700 1 -- 172.21.15.40:0/2528645940 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c00fee0 con 0x7f241c0cea90 2024-09-13T18:01:04.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.972+0000 7f2411ffb700 1 -- 172.21.15.40:0/2528645940 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4947 (secure 0 0 0) 0x7f240c029400 con 0x7f241c0cea90 2024-09-13T18:01:04.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.972+0000 7f2411ffb700 1 -- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f241c0cea90 msgr2=0x7f241c1ef1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.972+0000 7f2411ffb700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f241c0cea90 0x7f241c1ef1b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f240c01b000 tx=0x7f240c004770 comp rx=0 tx=0).stop 2024-09-13T18:01:04.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.972+0000 7f2411ffb700 1 -- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f241c0cea90 msgr2=0x7f241c1ef1b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:01:04.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.972+0000 7f2411ffb700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f241c0cea90 0x7f241c1ef1b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.980 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:01:04.980 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:01:04.980 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:01:04.980 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:01:04.981 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 82, 2024-09-13T18:01:04.981 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:01:04.981 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:01:04.981 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:01:04.981 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:01:04.981 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:04.981 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:01:04.981 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:01:04.981 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:01:04.981 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:01:04.981 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:04.981 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:04.981 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:04.981 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:04.982 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:04.982 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:04.982 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:04.982 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:04.982 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:04.982 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:04.982 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:04.982 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:01:04.982 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:01:04.982 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:04.982 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:04.982 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:04.982 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:04.982 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:04.983 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:04.983 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:04.983 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:04.983 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:04.983 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:04.983 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:04.983 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.984 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:01:04.984 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:01:04.984 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:01:04.984 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:01:04.984 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:01:04.984 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:01:04.984 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:01:04.984 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:01:04.984 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:01:04.984 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:01:04.984 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:01:04.984 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:01:04.985 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:01:04.985 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:01:04.985 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:01:04.985 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:01:04.985 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:01:04.985 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:04.985 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:04.986 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:04.986 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:04.986 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:04.986 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:04.986 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:04.986 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:04.986 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:04.986 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:04.986 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:04.986 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:01:04.987 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.987 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:01:04.987 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.987 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:01:04.987 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:01:04.987 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:04.987 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:04.987 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.987 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:04.987 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:01:04.987 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:04.988 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.988 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.988 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:04.988 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:01:04.988 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:04.988 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:04.988 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:04.988 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.988 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:01:04.988 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:01:04.988 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:04.988 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:04.988 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:04.988 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.989 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.989 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:01:04.989 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:01:04.989 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:04.989 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:04.989 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.989 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:04.989 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:01:04.989 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:04.989 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.989 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.989 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:04.989 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:01:04.990 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:04.990 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:04.990 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:04.990 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.990 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:01:04.990 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:01:04.990 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:04.990 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:04.990 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:04.991 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.991 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.991 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:01:04.991 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:01:04.991 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:04.991 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:04.991 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.991 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:04.991 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:01:04.991 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:04.991 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.992 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.992 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:04.992 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:01:04.992 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:04.992 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:04.992 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:04.992 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.992 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:01:04.992 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:01:04.992 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:04.992 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:04.993 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:04.993 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:04.993 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:04.993 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.993 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:01:04.993 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:01:04.993 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.993 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:04.993 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:04.993 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:01:04.993 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:04.993 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:04.993 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:01:04.994 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.994 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:04.994 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:04.994 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:01:04.994 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:04.994 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:04.994 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:01:04.994 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.994 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:04.994 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:04.994 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:01:04.994 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:04.994 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:04.994 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:01:04.995 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.995 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:04.995 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:04.995 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:01:04.995 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:04.995 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:04.995 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:01:04.996 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:04.996 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:04.996 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:04.996 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:01:04.996 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:04.996 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:04.996 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:04.996 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:01:04.996 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:01:04.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.972+0000 7f2422295700 1 -- 172.21.15.40:0/2528645940 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2400066730 msgr2=0x7f2400068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.972+0000 7f2422295700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2400066730 0x7f2400068bf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f241c1ec550 tx=0x7f2418009040 comp rx=0 tx=0).stop 2024-09-13T18:01:04.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.972+0000 7f2422295700 1 -- 172.21.15.40:0/2528645940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f241c0cf580 msgr2=0x7f241c151910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:04.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.972+0000 7f2422295700 1 --2- 172.21.15.40:0/2528645940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f241c0cf580 0x7f241c151910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f241c08c270 tx=0x7f2414004af0 comp rx=0 tx=0).stop 2024-09-13T18:01:04.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.972+0000 7f2421a94700 1 -- 172.21.15.40:0/2528645940 reap_dead start 2024-09-13T18:01:04.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.972+0000 7f2421a94700 1 -- 172.21.15.40:0/2528645940 reap_dead start 2024-09-13T18:01:04.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.972+0000 7f2422295700 1 -- 172.21.15.40:0/2528645940 shutdown_connections 2024-09-13T18:01:04.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.972+0000 7f2422295700 1 -- 172.21.15.40:0/2528645940 >> 172.21.15.40:0/2528645940 conn(0x7f241c066090 msgr2=0x7f241c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:04.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.972+0000 7f2422295700 1 -- 172.21.15.40:0/2528645940 shutdown_connections 2024-09-13T18:01:04.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:04.972+0000 7f2422295700 1 -- 172.21.15.40:0/2528645940 wait complete. 2024-09-13T18:01:04.999 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:01:04.999 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:01:04.999 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:01:05.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f3870889700 1 Processor -- start 2024-09-13T18:01:05.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f3870889700 1 -- start start 2024-09-13T18:01:05.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f3870889700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f386c0c3060 0x7f386c066300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:05.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f3870889700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f386c0c56d0 con 0x7f386c15b0f0 2024-09-13T18:01:05.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f3870889700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f386c0c6860 con 0x7f386c151410 2024-09-13T18:01:05.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f3870889700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c0c49a0 con 0x7f386c0c3060 2024-09-13T18:01:05.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386bfff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f386c15b0f0 0x7f386c06dcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48660/0 (socket says 172.21.15.40:48660) 2024-09-13T18:01:05.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386bfff700 1 -- 172.21.15.40:0/2700726676 learned_addr learned my addr 172.21.15.40:0/2700726676 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:01:05.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386affd700 1 --2- 172.21.15.40:0/2700726676 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f386c0c3060 0x7f386c066300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:05.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386a7fc700 1 -- 172.21.15.40:0/2700726676 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317594617 0 0) 0x7f386c0c56d0 con 0x7f386c15b0f0 2024-09-13T18:01:05.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386a7fc700 1 -- 172.21.15.40:0/2700726676 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38540035d0 con 0x7f386c15b0f0 2024-09-13T18:01:05.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386affd700 1 -- 172.21.15.40:0/2700726676 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f386c0c3060 msgr2=0x7f386c066300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:01:05.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386affd700 1 -- 172.21.15.40:0/2700726676 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f386c0c3060 msgr2=0x7f386c066300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:01:05.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386affd700 1 --2- 172.21.15.40:0/2700726676 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f386c0c3060 0x7f386c066300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:01:05.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386affd700 1 --2- 172.21.15.40:0/2700726676 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f386c0c3060 0x7f386c066300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:05.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386a7fc700 1 -- 172.21.15.40:0/2700726676 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1721636396 0 0) 0x7f386c0c6860 con 0x7f386c151410 2024-09-13T18:01:05.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386a7fc700 1 -- 172.21.15.40:0/2700726676 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f386c0c56d0 con 0x7f386c151410 2024-09-13T18:01:05.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386a7fc700 1 -- 172.21.15.40:0/2700726676 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 717344973 0 0) 0x7f38540035d0 con 0x7f386c15b0f0 2024-09-13T18:01:05.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386a7fc700 1 -- 172.21.15.40:0/2700726676 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f386c0c3060 msgr2=0x7f386c066300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:01:05.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386a7fc700 1 --2- 172.21.15.40:0/2700726676 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f386c0c3060 0x7f386c066300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:05.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386a7fc700 1 -- 172.21.15.40:0/2700726676 >> v1:172.21.15.43:6789/0 conn(0x7f386c151410 legacy=0x7f386c0c2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:05.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386a7fc700 1 -- 172.21.15.40:0/2700726676 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c0c53a0 con 0x7f386c15b0f0 2024-09-13T18:01:05.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386a7fc700 1 -- 172.21.15.40:0/2700726676 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f38640019f0 con 0x7f386c15b0f0 2024-09-13T18:01:05.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386a7fc700 1 -- 172.21.15.40:0/2700726676 >> v1:172.21.15.40:6789/0 conn(0x7f386c15b0f0 legacy=0x7f386c06dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:05.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386a7fc700 1 --2- 172.21.15.40:0/2700726676 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f386c0c3060 0x7f3854003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:05.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386a7fc700 1 --2- 172.21.15.40:0/2700726676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3854004140 0x7f3854004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:05.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386a7fc700 1 --2- 172.21.15.40:0/2700726676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3854004db0 0x7f3854005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:05.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386a7fc700 1 -- 172.21.15.40:0/2700726676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c0c49a0 con 0x7f3854004140 2024-09-13T18:01:05.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386a7fc700 1 -- 172.21.15.40:0/2700726676 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385400bbc0 con 0x7f3854004db0 2024-09-13T18:01:05.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386a7fc700 1 -- 172.21.15.40:0/2700726676 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385400bd80 con 0x7f386c0c3060 2024-09-13T18:01:05.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386affd700 1 --2- 172.21.15.40:0/2700726676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3854004140 0x7f3854004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:05.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386b7fe700 1 --2- 172.21.15.40:0/2700726676 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f386c0c3060 0x7f3854003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:05.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.108+0000 7f386bfff700 1 --2- 172.21.15.40:0/2700726676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3854004db0 0x7f3854005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:05.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.112+0000 7f386affd700 1 -- 172.21.15.40:0/2700726676 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f386c0c3060 msgr2=0x7f3854003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:01:05.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.112+0000 7f386affd700 1 --2- 172.21.15.40:0/2700726676 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f386c0c3060 0x7f3854003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:05.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.112+0000 7f386affd700 1 -- 172.21.15.40:0/2700726676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3854004db0 msgr2=0x7f3854005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:01:05.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.112+0000 7f386affd700 1 --2- 172.21.15.40:0/2700726676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3854004db0 0x7f3854005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:05.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.112+0000 7f386affd700 1 -- 172.21.15.40:0/2700726676 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38540035d0 con 0x7f3854004140 2024-09-13T18:01:05.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.112+0000 7f386affd700 1 --2- 172.21.15.40:0/2700726676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3854004140 0x7f3854004870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f386c154d90 tx=0x7f38540058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1bf4d4985dc5b62 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:05.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.112+0000 7f386a7fc700 1 -- 172.21.15.40:0/2700726676 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c004230 con 0x7f3854004140 2024-09-13T18:01:05.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.112+0000 7f386a7fc700 1 -- 172.21.15.40:0/2700726676 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c004390 con 0x7f3854004140 2024-09-13T18:01:05.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.112+0000 7f386a7fc700 1 -- 172.21.15.40:0/2700726676 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c0036c0 con 0x7f3854004140 2024-09-13T18:01:05.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.112+0000 7f3870889700 1 -- 172.21.15.40:0/2700726676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3854004140 msgr2=0x7f3854004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:05.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.112+0000 7f3870889700 1 --2- 172.21.15.40:0/2700726676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3854004140 0x7f3854004870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f386c154d90 tx=0x7f38540058a0 comp rx=0 tx=0).stop 2024-09-13T18:01:05.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.112+0000 7f386bfff700 1 -- 172.21.15.40:0/2700726676 reap_dead start 2024-09-13T18:01:05.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.112+0000 7f3870889700 1 -- 172.21.15.40:0/2700726676 shutdown_connections 2024-09-13T18:01:05.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.112+0000 7f3870889700 1 -- 172.21.15.40:0/2700726676 >> 172.21.15.40:0/2700726676 conn(0x7f386c06cba0 msgr2=0x7f386c06a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:05.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.112+0000 7f3870889700 1 -- 172.21.15.40:0/2700726676 shutdown_connections 2024-09-13T18:01:05.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.112+0000 7f3870889700 1 -- 172.21.15.40:0/2700726676 wait complete. 2024-09-13T18:01:05.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f3870889700 1 Processor -- start 2024-09-13T18:01:05.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f3870889700 1 -- start start 2024-09-13T18:01:05.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f3870889700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3854004140 0x7f386c15a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:05.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f3870889700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f386c146a20 0x7f386c145580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:05.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f3870889700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f386c145ac0 0x7f386c0704f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:05.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f3870889700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c158920 con 0x7f386c146a20 2024-09-13T18:01:05.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f3870889700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c071710 con 0x7f386c145ac0 2024-09-13T18:01:05.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f3870889700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c1585b0 con 0x7f3854004140 2024-09-13T18:01:05.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f386affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f386c146a20 0x7f386c145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:05.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f386affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f386c146a20 0x7f386c145580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54216/0 (socket says 172.21.15.40:54216) 2024-09-13T18:01:05.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f386affd700 1 -- 172.21.15.40:0/3551067847 learned_addr learned my addr 172.21.15.40:0/3551067847 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:05.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f386bfff700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f386c145ac0 0x7f386c0704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:05.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f386b7fe700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3854004140 0x7f386c15a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:05.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f386b7fe700 1 -- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3854004140 msgr2=0x7f386c15a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:01:05.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f386b7fe700 1 -- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3854004140 msgr2=0x7f386c15a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:01:05.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f386b7fe700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3854004140 0x7f386c15a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:01:05.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f386b7fe700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3854004140 0x7f386c15a160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:05.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f386affd700 1 -- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3854004140 msgr2=0x7f386c15a160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:01:05.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f386affd700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3854004140 0x7f386c15a160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:05.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f386affd700 1 -- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f386c145ac0 msgr2=0x7f386c0704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:05.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f386affd700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f386c145ac0 0x7f386c0704f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:05.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f386affd700 1 -- 172.21.15.40:0/3551067847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c159610 con 0x7f386c146a20 2024-09-13T18:01:05.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f386affd700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f386c146a20 0x7f386c145580 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f385c014000 tx=0x7f385c0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:05.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f3853fff700 1 -- 172.21.15.40:0/3551067847 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c004030 con 0x7f386c146a20 2024-09-13T18:01:05.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f3853fff700 1 -- 172.21.15.40:0/3551067847 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c022070 con 0x7f386c146a20 2024-09-13T18:01:05.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f3870889700 1 -- 172.21.15.40:0/3551067847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386c149950 con 0x7f386c146a20 2024-09-13T18:01:05.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.116+0000 7f3853fff700 1 -- 172.21.15.40:0/3551067847 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c027670 con 0x7f386c146a20 2024-09-13T18:01:05.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.120+0000 7f3870889700 1 -- 172.21.15.40:0/3551067847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386c071960 con 0x7f386c146a20 2024-09-13T18:01:05.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.120+0000 7f3851ffb700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 0x7f3844004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:05.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.120+0000 7f3851ffb700 1 -- 172.21.15.40:0/3551067847 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3844007f40 con 0x7f3844002a40 2024-09-13T18:01:05.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.120+0000 7f386b7fe700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 0x7f3844004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:05.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.120+0000 7f386b7fe700 1 -- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 msgr2=0x7f3844004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-09-13T18:01:05.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.120+0000 7f386b7fe700 1 -- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 msgr2=0x7f3844004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:01:05.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.120+0000 7f386b7fe700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 0x7f3844004f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:01:05.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.120+0000 7f386b7fe700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 0x7f3844004f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:05.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.120+0000 7f3853fff700 1 -- 172.21.15.40:0/3551067847 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f385c024030 con 0x7f386c146a20 2024-09-13T18:01:05.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.120+0000 7f3853fff700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3858066760 0x7f3858068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:05.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.120+0000 7f3853fff700 1 -- 172.21.15.40:0/3551067847 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f385c0855c0 con 0x7f386c146a20 2024-09-13T18:01:05.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.120+0000 7f386bfff700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3858066760 0x7f3858068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:05.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.120+0000 7f386bfff700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3858066760 0x7f3858068c20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f386c0c70a0 tx=0x7f3864006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:05.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.324+0000 7f386b7fe700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 0x7f3844004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:05.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.324+0000 7f386b7fe700 1 -- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 msgr2=0x7f3844004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-13T18:01:05.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.324+0000 7f386b7fe700 1 -- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 msgr2=0x7f3844004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:01:05.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.324+0000 7f386b7fe700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 0x7f3844004f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:01:05.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.324+0000 7f386b7fe700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 0x7f3844004f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:01:05.728 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:01:05.730 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:01:05.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.728+0000 7f386b7fe700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 0x7f3844004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:05.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.728+0000 7f386b7fe700 1 -- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 msgr2=0x7f3844004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-13T18:01:05.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.728+0000 7f386b7fe700 1 -- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 msgr2=0x7f3844004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:01:05.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.728+0000 7f386b7fe700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 0x7f3844004f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:01:05.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:05.728+0000 7f386b7fe700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 0x7f3844004f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:01:06.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.532+0000 7f386b7fe700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 0x7f3844004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:06.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.532+0000 7f386b7fe700 1 -- 172.21.15.40:0/3551067847 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3844008100 con 0x7f3844002a40 2024-09-13T18:01:06.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.532+0000 7f386b7fe700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 0x7f3844004f00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3860007fd0 tx=0x7f3860017d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:06.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.536+0000 7f3853fff700 1 -- 172.21.15.40:0/3551067847 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38600052e0 con 0x7f3844002a40 2024-09-13T18:01:06.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.536+0000 7f3853fff700 1 -- 172.21.15.40:0/3551067847 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3860005440 con 0x7f3844002a40 2024-09-13T18:01:06.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.536+0000 7f3853fff700 1 -- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 msgr2=0x7f3844004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:06.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.536+0000 7f3853fff700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 0x7f3844004f00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3860007fd0 tx=0x7f3860017d00 comp rx=0 tx=0).stop 2024-09-13T18:01:06.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.536+0000 7f3853fff700 1 -- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 msgr2=0x7f3844004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:01:06.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.536+0000 7f3853fff700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844002a40 0x7f3844004f00 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3860007fd0 tx=0x7f3860017d00 comp rx=0 tx=0).stop 2024-09-13T18:01:06.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.548+0000 7f3851ffb700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844006f90 0x7f3844007400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:06.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.548+0000 7f3851ffb700 1 -- 172.21.15.40:0/3551067847 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384400a400 con 0x7f3844006f90 2024-09-13T18:01:06.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.552+0000 7f386b7fe700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844006f90 0x7f3844007400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:06.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.552+0000 7f386b7fe700 1 -- 172.21.15.40:0/3551067847 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f384400a5c0 con 0x7f3844006f90 2024-09-13T18:01:06.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.552+0000 7f386b7fe700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844006f90 0x7f3844007400 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3860007fd0 tx=0x7f38600012d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:06.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.552+0000 7f3853fff700 1 -- 172.21.15.40:0/3551067847 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860027070 con 0x7f3844006f90 2024-09-13T18:01:06.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.552+0000 7f3853fff700 1 -- 172.21.15.40:0/3551067847 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4449 (secure 0 0 0) 0x7f3860015aa0 con 0x7f3844006f90 2024-09-13T18:01:06.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.552+0000 7f3853fff700 1 -- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844006f90 msgr2=0x7f3844007400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:06.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.552+0000 7f3853fff700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844006f90 0x7f3844007400 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3860007fd0 tx=0x7f38600012d0 comp rx=0 tx=0).stop 2024-09-13T18:01:06.562 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:01:06.562 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:01:06.562 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:01:06.562 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:01:06.562 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 82, 2024-09-13T18:01:06.562 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:01:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:01:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:01:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:01:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:01:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:01:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:01:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:01:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:06.563 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:06.564 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:06.564 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:06.564 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:06.564 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:06.564 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:06.564 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:06.564 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:06.564 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:06.564 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:01:06.564 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:01:06.564 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:06.564 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:06.564 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:06.565 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:06.565 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:06.565 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:06.565 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:06.565 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:06.565 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:06.565 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:06.565 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:06.565 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:06.565 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:01:06.565 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:01:06.566 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:01:06.566 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:01:06.566 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:01:06.566 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:01:06.566 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:01:06.566 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:01:06.566 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:01:06.566 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:01:06.566 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:01:06.566 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:01:06.566 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:01:06.567 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:01:06.567 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:01:06.567 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:01:06.567 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:01:06.567 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:06.567 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:06.567 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:06.567 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:06.567 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:06.567 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:06.567 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:06.567 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:06.567 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:06.568 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:06.568 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:06.568 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:01:06.568 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:06.568 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:01:06.568 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:06.568 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:01:06.568 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:01:06.568 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:06.568 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:06.568 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:06.568 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:06.568 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:01:06.568 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:06.569 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:06.569 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:06.569 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:06.569 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:01:06.569 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:06.569 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:06.569 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:06.569 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:06.569 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:01:06.569 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:01:06.570 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:06.570 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:06.570 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:06.570 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:06.570 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:06.570 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:01:06.570 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:01:06.570 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:06.570 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:06.570 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:06.570 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:06.570 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:01:06.571 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:06.571 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:06.571 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:06.571 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:06.571 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:01:06.571 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:06.571 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:06.571 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:06.571 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:06.571 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:01:06.571 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:01:06.571 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:06.571 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:06.571 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:06.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:06.572 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:06.572 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:01:06.572 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:01:06.572 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:06.572 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:06.572 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:06.572 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:06.572 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:01:06.572 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:06.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:06.572 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:06.573 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:06.573 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:01:06.573 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:06.573 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:06.573 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:06.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:06.573 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:01:06.573 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:01:06.573 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:06.573 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:06.573 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:06.573 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:06.573 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:06.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:06.574 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:01:06.574 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:01:06.574 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:06.574 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:06.574 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:06.574 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:01:06.574 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:06.574 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:06.574 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:01:06.574 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:06.574 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:06.574 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:06.574 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:01:06.575 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:06.575 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:06.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:06.575 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:01:06.575 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:01:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.552+0000 7f3853fff700 1 -- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844006f90 msgr2=0x7f3844007400 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:01:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.552+0000 7f3853fff700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3844006f90 0x7f3844007400 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.552+0000 7f3851ffb700 1 -- 172.21.15.40:0/3551067847 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3858066760 msgr2=0x7f3858068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.552+0000 7f3851ffb700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3858066760 0x7f3858068c20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f386c0c70a0 tx=0x7f3864006040 comp rx=0 tx=0).stop 2024-09-13T18:01:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.552+0000 7f3851ffb700 1 -- 172.21.15.40:0/3551067847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f386c146a20 msgr2=0x7f386c145580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.552+0000 7f3851ffb700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f386c146a20 0x7f386c145580 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f385c014000 tx=0x7f385c0047e0 comp rx=0 tx=0).stop 2024-09-13T18:01:06.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.552+0000 7f386bfff700 1 -- 172.21.15.40:0/3551067847 reap_dead start 2024-09-13T18:01:06.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.552+0000 7f3851ffb700 1 -- 172.21.15.40:0/3551067847 shutdown_connections 2024-09-13T18:01:06.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.552+0000 7f3851ffb700 1 --2- 172.21.15.40:0/3551067847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f386c146a20 0x7f386c145580 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:06.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.552+0000 7f3851ffb700 1 -- 172.21.15.40:0/3551067847 >> 172.21.15.40:0/3551067847 conn(0x7f386c06cba0 msgr2=0x7f386c06a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:06.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.552+0000 7f3851ffb700 1 -- 172.21.15.40:0/3551067847 shutdown_connections 2024-09-13T18:01:06.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.552+0000 7f3851ffb700 1 -- 172.21.15.40:0/3551067847 wait complete. 2024-09-13T18:01:06.581 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:01:06.582 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:01:06.582 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:01:06.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.696+0000 7f7f2a4a4700 1 Processor -- start 2024-09-13T18:01:06.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.696+0000 7f7f2a4a4700 1 -- start start 2024-09-13T18:01:06.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.696+0000 7f7f2a4a4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7f24150690 0x7f7f240c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:06.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.696+0000 7f7f2a4a4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7f24070bc0 0x7f7f2406b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:06.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.696+0000 7f7f2a4a4700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f24159800 con 0x7f7f2406b9e0 2024-09-13T18:01:06.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.696+0000 7f7f2a4a4700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f240c8350 con 0x7f7f24070bc0 2024-09-13T18:01:06.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.696+0000 7f7f2a4a4700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f240c6d20 con 0x7f7f24150690 2024-09-13T18:01:06.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.696+0000 7f7f28ca1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7f24070bc0 0x7f7f2406b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:06.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.696+0000 7f7f28ca1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7f24070bc0 0x7f7f2406b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54232/0 (socket says 172.21.15.40:54232) 2024-09-13T18:01:06.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.696+0000 7f7f28ca1700 1 -- 172.21.15.40:0/666685485 learned_addr learned my addr 172.21.15.40:0/666685485 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:06.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.696+0000 7f7f294a2700 1 --2- 172.21.15.40:0/666685485 >> v2:172.21.15.43:3300/0 conn(0x7f7f24150690 0x7f7f240c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:06.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.696+0000 7f7f28ca1700 1 -- 172.21.15.40:0/666685485 >> v2:172.21.15.43:3300/0 conn(0x7f7f24150690 msgr2=0x7f7f240c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:01:06.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.696+0000 7f7f28ca1700 1 --2- 172.21.15.40:0/666685485 >> v2:172.21.15.43:3300/0 conn(0x7f7f24150690 0x7f7f240c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:06.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.696+0000 7f7f28ca1700 1 -- 172.21.15.40:0/666685485 >> v1:172.21.15.43:6789/0 conn(0x7f7f2406b9e0 legacy=0x7f7f241567e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:01:06.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.696+0000 7f7f28ca1700 1 -- 172.21.15.40:0/666685485 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f24150a70 con 0x7f7f24070bc0 2024-09-13T18:01:06.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.696+0000 7f7f28ca1700 1 --2- 172.21.15.40:0/666685485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f24070bc0 0x7f7f2406b4a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7f2406de80 tx=0x7f7f1400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e60baea079cca92 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:06.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.696+0000 7f7f1bfff700 1 -- 172.21.15.40:0/666685485 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1400dcf0 con 0x7f7f24070bc0 2024-09-13T18:01:06.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.696+0000 7f7f1bfff700 1 -- 172.21.15.40:0/666685485 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1400de50 con 0x7f7f24070bc0 2024-09-13T18:01:06.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.696+0000 7f7f1bfff700 1 -- 172.21.15.40:0/666685485 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f14004250 con 0x7f7f24070bc0 2024-09-13T18:01:06.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f2a4a4700 1 -- 172.21.15.40:0/666685485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f24070bc0 msgr2=0x7f7f2406b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f2a4a4700 1 --2- 172.21.15.40:0/666685485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f24070bc0 0x7f7f2406b4a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7f2406de80 tx=0x7f7f1400be90 comp rx=0 tx=0).stop 2024-09-13T18:01:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f2a4a4700 1 -- 172.21.15.40:0/666685485 shutdown_connections 2024-09-13T18:01:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f2a4a4700 1 --2- 172.21.15.40:0/666685485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f24070bc0 0x7f7f2406b4a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f2a4a4700 1 --2- 172.21.15.40:0/666685485 >> v2:172.21.15.43:3300/0 conn(0x7f7f24150690 0x7f7f240c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f2a4a4700 1 -- 172.21.15.40:0/666685485 >> 172.21.15.40:0/666685485 conn(0x7f7f240c88e0 msgr2=0x7f7f240c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f2a4a4700 1 -- 172.21.15.40:0/666685485 shutdown_connections 2024-09-13T18:01:06.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f2a4a4700 1 -- 172.21.15.40:0/666685485 wait complete. 2024-09-13T18:01:06.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f2a4a4700 1 Processor -- start 2024-09-13T18:01:06.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f2a4a4700 1 -- start start 2024-09-13T18:01:06.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f2a4a4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f2406b9e0 0x7f7f24159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:06.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f2a4a4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f24070bc0 0x7f7f241f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:06.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f2a4a4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f24150690 0x7f7f24155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:06.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f2a4a4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f24158e70 con 0x7f7f24070bc0 2024-09-13T18:01:06.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f2a4a4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f24155a90 con 0x7f7f24150690 2024-09-13T18:01:06.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f2a4a4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f24141ed0 con 0x7f7f2406b9e0 2024-09-13T18:01:06.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f28ca1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f24070bc0 0x7f7f241f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:06.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f28ca1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f24070bc0 0x7f7f241f2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54240/0 (socket says 172.21.15.40:54240) 2024-09-13T18:01:06.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f28ca1700 1 -- 172.21.15.40:0/922045783 learned_addr learned my addr 172.21.15.40:0/922045783 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:06.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f294a2700 1 --2- 172.21.15.40:0/922045783 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f2406b9e0 0x7f7f24159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:06.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.700+0000 7f7f29ca3700 1 --2- 172.21.15.40:0/922045783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f24150690 0x7f7f24155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:06.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.704+0000 7f7f28ca1700 1 -- 172.21.15.40:0/922045783 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f2406b9e0 msgr2=0x7f7f24159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:06.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.704+0000 7f7f28ca1700 1 --2- 172.21.15.40:0/922045783 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f2406b9e0 0x7f7f24159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:06.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.704+0000 7f7f28ca1700 1 -- 172.21.15.40:0/922045783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f24150690 msgr2=0x7f7f24155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:06.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.704+0000 7f7f28ca1700 1 --2- 172.21.15.40:0/922045783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f24150690 0x7f7f24155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:06.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.704+0000 7f7f28ca1700 1 -- 172.21.15.40:0/922045783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1400b810 con 0x7f7f24070bc0 2024-09-13T18:01:06.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.704+0000 7f7f28ca1700 1 --2- 172.21.15.40:0/922045783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f24070bc0 0x7f7f241f2ba0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400d5d0 tx=0x7f7f14004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:06.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.704+0000 7f7f19ffb700 1 -- 172.21.15.40:0/922045783 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f14004610 con 0x7f7f24070bc0 2024-09-13T18:01:06.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.704+0000 7f7f19ffb700 1 -- 172.21.15.40:0/922045783 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f14004230 con 0x7f7f24070bc0 2024-09-13T18:01:06.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.704+0000 7f7f19ffb700 1 -- 172.21.15.40:0/922045783 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f140106b0 con 0x7f7f24070bc0 2024-09-13T18:01:06.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.704+0000 7f7f2a4a4700 1 -- 172.21.15.40:0/922045783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f241587f0 con 0x7f7f24070bc0 2024-09-13T18:01:06.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.704+0000 7f7f2a4a4700 1 -- 172.21.15.40:0/922045783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f24140830 con 0x7f7f24070bc0 2024-09-13T18:01:06.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.704+0000 7f7f2a4a4700 1 -- 172.21.15.40:0/922045783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f2415a300 con 0x7f7f24070bc0 2024-09-13T18:01:06.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.704+0000 7f7f19ffb700 1 -- 172.21.15.40:0/922045783 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7f140193f0 con 0x7f7f24070bc0 2024-09-13T18:01:06.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.704+0000 7f7f19ffb700 1 --2- 172.21.15.40:0/922045783 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7f08066730 0x7f7f08068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:06.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.704+0000 7f7f19ffb700 1 -- 172.21.15.40:0/922045783 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7f14085a10 con 0x7f7f24070bc0 2024-09-13T18:01:06.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.704+0000 7f7f294a2700 1 --2- 172.21.15.40:0/922045783 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7f08066730 0x7f7f08068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:06.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.708+0000 7f7f294a2700 1 --2- 172.21.15.40:0/922045783 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7f08066730 0x7f7f08068bf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009b50 tx=0x7f7f20009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:06.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.712+0000 7f7f19ffb700 1 -- 172.21.15.40:0/922045783 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7f140529c0 con 0x7f7f24070bc0 2024-09-13T18:01:06.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.980+0000 7f7f2a4a4700 1 -- 172.21.15.40:0/922045783 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f7f24153cf0 con 0x7f7f24070bc0 2024-09-13T18:01:06.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.980+0000 7f7f19ffb700 1 -- 172.21.15.40:0/922045783 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f7f14055fe0 con 0x7f7f24070bc0 2024-09-13T18:01:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.984+0000 7f7f2a4a4700 1 -- 172.21.15.40:0/922045783 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7f08066730 msgr2=0x7f7f08068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.984+0000 7f7f2a4a4700 1 --2- 172.21.15.40:0/922045783 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7f08066730 0x7f7f08068bf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7f20009b50 tx=0x7f7f20009240 comp rx=0 tx=0).stop 2024-09-13T18:01:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.984+0000 7f7f2a4a4700 1 -- 172.21.15.40:0/922045783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f24070bc0 msgr2=0x7f7f241f2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.984+0000 7f7f2a4a4700 1 --2- 172.21.15.40:0/922045783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f24070bc0 0x7f7f241f2ba0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400d5d0 tx=0x7f7f14004770 comp rx=0 tx=0).stop 2024-09-13T18:01:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.984+0000 7f7f2a4a4700 1 -- 172.21.15.40:0/922045783 shutdown_connections 2024-09-13T18:01:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.984+0000 7f7f2a4a4700 1 --2- 172.21.15.40:0/922045783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f24150690 0x7f7f24155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.984+0000 7f7f2a4a4700 1 --2- 172.21.15.40:0/922045783 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f24070bc0 0x7f7f241f2ba0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.984+0000 7f7f2a4a4700 1 --2- 172.21.15.40:0/922045783 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7f08066730 0x7f7f08068bf0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:06.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.984+0000 7f7f2a4a4700 1 --2- 172.21.15.40:0/922045783 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f2406b9e0 0x7f7f24159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:06.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.984+0000 7f7f2a4a4700 1 -- 172.21.15.40:0/922045783 >> 172.21.15.40:0/922045783 conn(0x7f7f240c88e0 msgr2=0x7f7f2406ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:06.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.984+0000 7f7f2a4a4700 1 -- 172.21.15.40:0/922045783 shutdown_connections 2024-09-13T18:01:06.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:06.984+0000 7f7f2a4a4700 1 -- 172.21.15.40:0/922045783 wait complete. 2024-09-13T18:01:07.010 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:01:17.011 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:01:17.011 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:01:17.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.132+0000 7f15a883e700 1 Processor -- start 2024-09-13T18:01:17.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.132+0000 7f15a883e700 1 -- start start 2024-09-13T18:01:17.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a883e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15a40c69a0 0x7f15a4156f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a883e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f15a40c6d80 0x7f15a4155a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a883e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a4156650 con 0x7f15a40cb520 2024-09-13T18:01:17.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a883e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a41590e0 con 0x7f15a40c6d80 2024-09-13T18:01:17.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a883e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a41593c0 con 0x7f15a40c69a0 2024-09-13T18:01:17.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a1d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f15a40c6d80 0x7f15a4155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a259c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15a40c69a0 0x7f15a4156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a1d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f15a40c6d80 0x7f15a4155a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47364/0 (socket says 172.21.15.40:47364) 2024-09-13T18:01:17.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a1d9b700 1 -- 172.21.15.40:0/1642559915 learned_addr learned my addr 172.21.15.40:0/1642559915 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:17.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a1d9b700 1 -- 172.21.15.40:0/1642559915 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15a40c69a0 msgr2=0x7f15a4156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:01:17.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a1d9b700 1 --2- 172.21.15.40:0/1642559915 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15a40c69a0 0x7f15a4156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a1d9b700 1 -- 172.21.15.40:0/1642559915 >> v1:172.21.15.43:6789/0 conn(0x7f15a40cb520 legacy=0x7f15a414f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a1d9b700 1 -- 172.21.15.40:0/1642559915 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a41596a0 con 0x7f15a40c6d80 2024-09-13T18:01:17.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a1d9b700 1 --2- 172.21.15.40:0/1642559915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a40c6d80 0x7f15a4155a50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1598009910 tx=0x7f1598009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df3f6091f6d72a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:17.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a159a700 1 -- 172.21.15.40:0/1642559915 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598011830 con 0x7f15a40c6d80 2024-09-13T18:01:17.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a159a700 1 -- 172.21.15.40:0/1642559915 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1598011990 con 0x7f15a40c6d80 2024-09-13T18:01:17.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a159a700 1 -- 172.21.15.40:0/1642559915 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598011d40 con 0x7f15a40c6d80 2024-09-13T18:01:17.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a883e700 1 -- 172.21.15.40:0/1642559915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a40c6d80 msgr2=0x7f15a4155a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a883e700 1 --2- 172.21.15.40:0/1642559915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a40c6d80 0x7f15a4155a50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1598009910 tx=0x7f1598009c20 comp rx=0 tx=0).stop 2024-09-13T18:01:17.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a883e700 1 -- 172.21.15.40:0/1642559915 shutdown_connections 2024-09-13T18:01:17.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a883e700 1 --2- 172.21.15.40:0/1642559915 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a40c6d80 0x7f15a4155a50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a883e700 1 --2- 172.21.15.40:0/1642559915 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15a40c69a0 0x7f15a4156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.136+0000 7f15a883e700 1 -- 172.21.15.40:0/1642559915 >> 172.21.15.40:0/1642559915 conn(0x7f15a406a390 msgr2=0x7f15a406a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:17.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.140+0000 7f15a883e700 1 -- 172.21.15.40:0/1642559915 shutdown_connections 2024-09-13T18:01:17.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.140+0000 7f15a883e700 1 -- 172.21.15.40:0/1642559915 wait complete. 2024-09-13T18:01:17.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.140+0000 7f15a883e700 1 Processor -- start 2024-09-13T18:01:17.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.140+0000 7f15a883e700 1 -- start start 2024-09-13T18:01:17.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.140+0000 7f15a883e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a40c69a0 0x7f15a41547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.140+0000 7f15a883e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a40c6d80 0x7f15a41483a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.140+0000 7f15a883e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15a40cb520 0x7f15a4070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.140+0000 7f15a883e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a414fd10 con 0x7f15a40c6d80 2024-09-13T18:01:17.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.140+0000 7f15a883e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a414ade0 con 0x7f15a40c69a0 2024-09-13T18:01:17.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.140+0000 7f15a883e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a4154d10 con 0x7f15a40cb520 2024-09-13T18:01:17.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.144+0000 7f15a1d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a40c6d80 0x7f15a41483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.144+0000 7f15a1d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a40c6d80 0x7f15a41483a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47380/0 (socket says 172.21.15.40:47380) 2024-09-13T18:01:17.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.144+0000 7f15a1d9b700 1 -- 172.21.15.40:0/2619386486 learned_addr learned my addr 172.21.15.40:0/2619386486 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:17.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.144+0000 7f15a259c700 1 --2- 172.21.15.40:0/2619386486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a40c69a0 0x7f15a41547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.144+0000 7f15a2d9d700 1 --2- 172.21.15.40:0/2619386486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15a40cb520 0x7f15a4070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.144+0000 7f15a1d9b700 1 -- 172.21.15.40:0/2619386486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15a40cb520 msgr2=0x7f15a4070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.144+0000 7f15a1d9b700 1 --2- 172.21.15.40:0/2619386486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15a40cb520 0x7f15a4070fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.144+0000 7f15a1d9b700 1 -- 172.21.15.40:0/2619386486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a40c69a0 msgr2=0x7f15a41547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.144+0000 7f15a1d9b700 1 --2- 172.21.15.40:0/2619386486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a40c69a0 0x7f15a41547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.144+0000 7f15a1d9b700 1 -- 172.21.15.40:0/2619386486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1598009570 con 0x7f15a40c6d80 2024-09-13T18:01:17.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.144+0000 7f15a1d9b700 1 --2- 172.21.15.40:0/2619386486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a40c6d80 0x7f15a41483a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1598000c00 tx=0x7f1598004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:17.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.144+0000 7f1592ffd700 1 -- 172.21.15.40:0/2619386486 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598004590 con 0x7f15a40c6d80 2024-09-13T18:01:17.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.144+0000 7f1592ffd700 1 -- 172.21.15.40:0/2619386486 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15980036c0 con 0x7f15a40c6d80 2024-09-13T18:01:17.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.144+0000 7f15a883e700 1 -- 172.21.15.40:0/2619386486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a414bab0 con 0x7f15a40c6d80 2024-09-13T18:01:17.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.144+0000 7f1592ffd700 1 -- 172.21.15.40:0/2619386486 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598003a70 con 0x7f15a40c6d80 2024-09-13T18:01:17.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.144+0000 7f15a883e700 1 -- 172.21.15.40:0/2619386486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a414fef0 con 0x7f15a40c6d80 2024-09-13T18:01:17.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.144+0000 7f1592ffd700 1 -- 172.21.15.40:0/2619386486 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1598003bd0 con 0x7f15a40c6d80 2024-09-13T18:01:17.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.144+0000 7f15a883e700 1 -- 172.21.15.40:0/2619386486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a40c3520 con 0x7f15a40c6d80 2024-09-13T18:01:17.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.144+0000 7f1592ffd700 1 --2- 172.21.15.40:0/2619386486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1584066730 0x7f1584068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.148+0000 7f15a259c700 1 --2- 172.21.15.40:0/2619386486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1584066730 0x7f1584068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.148+0000 7f1592ffd700 1 -- 172.21.15.40:0/2619386486 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1598021030 con 0x7f15a40c6d80 2024-09-13T18:01:17.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.148+0000 7f15a259c700 1 --2- 172.21.15.40:0/2619386486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1584066730 0x7f1584068bf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1594009bf0 tx=0x7f1594009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:17.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.152+0000 7f1592ffd700 1 -- 172.21.15.40:0/2619386486 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1598052b70 con 0x7f15a40c6d80 2024-09-13T18:01:17.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.420+0000 7f15a883e700 1 -- 172.21.15.40:0/2619386486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f15a4156f40 con 0x7f15a40c6d80 2024-09-13T18:01:17.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.420+0000 7f1592ffd700 1 -- 172.21.15.40:0/2619386486 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f159802b080 con 0x7f15a40c6d80 2024-09-13T18:01:17.429 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":82,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:01:17.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.420+0000 7f15a883e700 1 -- 172.21.15.40:0/2619386486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1584066730 msgr2=0x7f1584068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.420+0000 7f15a883e700 1 --2- 172.21.15.40:0/2619386486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1584066730 0x7f1584068bf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1594009bf0 tx=0x7f1594009240 comp rx=0 tx=0).stop 2024-09-13T18:01:17.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.420+0000 7f15a883e700 1 -- 172.21.15.40:0/2619386486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a40c6d80 msgr2=0x7f15a41483a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.420+0000 7f15a883e700 1 --2- 172.21.15.40:0/2619386486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a40c6d80 0x7f15a41483a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1598000c00 tx=0x7f1598004770 comp rx=0 tx=0).stop 2024-09-13T18:01:17.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.420+0000 7f15a883e700 1 -- 172.21.15.40:0/2619386486 shutdown_connections 2024-09-13T18:01:17.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.420+0000 7f15a883e700 1 --2- 172.21.15.40:0/2619386486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1584066730 0x7f1584068bf0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.420+0000 7f15a883e700 1 --2- 172.21.15.40:0/2619386486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15a40cb520 0x7f15a4070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.420+0000 7f15a883e700 1 --2- 172.21.15.40:0/2619386486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15a40c6d80 0x7f15a41483a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.420+0000 7f15a883e700 1 --2- 172.21.15.40:0/2619386486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15a40c69a0 0x7f15a41547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.420+0000 7f15a883e700 1 -- 172.21.15.40:0/2619386486 >> 172.21.15.40:0/2619386486 conn(0x7f15a406a390 msgr2=0x7f15a414d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:17.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.420+0000 7f15a883e700 1 -- 172.21.15.40:0/2619386486 shutdown_connections 2024-09-13T18:01:17.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.420+0000 7f15a883e700 1 -- 172.21.15.40:0/2619386486 wait complete. 2024-09-13T18:01:17.443 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:01:17.443 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T18:01:17.443 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:01:17.443 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:01:17.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.556+0000 7fc96cf3c700 1 Processor -- start 2024-09-13T18:01:17.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.556+0000 7fc96cf3c700 1 -- start start 2024-09-13T18:01:17.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.556+0000 7fc96cf3c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc9681504c0 0x7fc96806b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.556+0000 7fc96cf3c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc96806b9e0 0x7fc96806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.556+0000 7fc96cf3c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96806ee10 con 0x7fc96806e980 2024-09-13T18:01:17.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.556+0000 7fc96cf3c700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9680c1a70 con 0x7fc9681504c0 2024-09-13T18:01:17.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.556+0000 7fc96cf3c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9681563b0 con 0x7fc96806b9e0 2024-09-13T18:01:17.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.556+0000 7fc967fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc96806e980 0x7fc9680c65c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33066/0 (socket says 172.21.15.40:33066) 2024-09-13T18:01:17.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.556+0000 7fc967fff700 1 -- 172.21.15.40:0/3385196154 learned_addr learned my addr 172.21.15.40:0/3385196154 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:01:17.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.556+0000 7fc95effd700 1 --2- 172.21.15.40:0/3385196154 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc96806b9e0 0x7fc96806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.556+0000 7fc9677fe700 1 --2- 172.21.15.40:0/3385196154 >> v2:172.21.15.43:3300/0 conn(0x7fc9681504c0 0x7fc96806b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.560+0000 7fc966ffd700 1 -- 172.21.15.40:0/3385196154 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1361092072 0 0) 0x7fc96806ee10 con 0x7fc96806e980 2024-09-13T18:01:17.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.560+0000 7fc966ffd700 1 -- 172.21.15.40:0/3385196154 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9540035d0 con 0x7fc96806e980 2024-09-13T18:01:17.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.560+0000 7fc95effd700 1 -- 172.21.15.40:0/3385196154 >> v2:172.21.15.43:3300/0 conn(0x7fc9681504c0 msgr2=0x7fc96806b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:01:17.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.560+0000 7fc95effd700 1 --2- 172.21.15.40:0/3385196154 >> v2:172.21.15.43:3300/0 conn(0x7fc9681504c0 0x7fc96806b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.560+0000 7fc95effd700 1 -- 172.21.15.40:0/3385196154 >> v1:172.21.15.40:6789/0 conn(0x7fc96806e980 legacy=0x7fc9680c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.560+0000 7fc95effd700 1 -- 172.21.15.40:0/3385196154 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9681571d0 con 0x7fc96806b9e0 2024-09-13T18:01:17.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.560+0000 7fc95effd700 1 --2- 172.21.15.40:0/3385196154 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc96806b9e0 0x7fc96806e440 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc95000b900 tx=0x7fc95000bc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3da8705d84311612 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:17.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.560+0000 7fc966ffd700 1 -- 172.21.15.40:0/3385196154 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95000ca30 con 0x7fc96806b9e0 2024-09-13T18:01:17.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.560+0000 7fc966ffd700 1 -- 172.21.15.40:0/3385196154 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95000cb90 con 0x7fc96806b9e0 2024-09-13T18:01:17.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.560+0000 7fc966ffd700 1 -- 172.21.15.40:0/3385196154 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95000f460 con 0x7fc96806b9e0 2024-09-13T18:01:17.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.560+0000 7fc96cf3c700 1 -- 172.21.15.40:0/3385196154 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc96806b9e0 msgr2=0x7fc96806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.560+0000 7fc96cf3c700 1 --2- 172.21.15.40:0/3385196154 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc96806b9e0 0x7fc96806e440 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc95000b900 tx=0x7fc95000bc10 comp rx=0 tx=0).stop 2024-09-13T18:01:17.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.560+0000 7fc96cf3c700 1 -- 172.21.15.40:0/3385196154 shutdown_connections 2024-09-13T18:01:17.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.560+0000 7fc96cf3c700 1 --2- 172.21.15.40:0/3385196154 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc96806b9e0 0x7fc96806e440 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc95000b900 tx=0x7fc95000bc10 comp rx=0 tx=0).stop 2024-09-13T18:01:17.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.560+0000 7fc96cf3c700 1 --2- 172.21.15.40:0/3385196154 >> v2:172.21.15.43:3300/0 conn(0x7fc9681504c0 0x7fc96806b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.560+0000 7fc96cf3c700 1 -- 172.21.15.40:0/3385196154 >> 172.21.15.40:0/3385196154 conn(0x7fc9680cbc40 msgr2=0x7fc9680cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:17.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.560+0000 7fc96cf3c700 1 -- 172.21.15.40:0/3385196154 shutdown_connections 2024-09-13T18:01:17.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.560+0000 7fc96cf3c700 1 -- 172.21.15.40:0/3385196154 wait complete. 2024-09-13T18:01:17.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc96cf3c700 1 Processor -- start 2024-09-13T18:01:17.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc96cf3c700 1 -- start start 2024-09-13T18:01:17.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc96cf3c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc96806e980 0x7fc9681eb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc96cf3c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9681504c0 0x7fc968154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc96cf3c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9681558d0 0x7fc9681546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc96cf3c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96814e8c0 con 0x7fc96806e980 2024-09-13T18:01:17.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc96cf3c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96814eb10 con 0x7fc9681558d0 2024-09-13T18:01:17.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc96cf3c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9681eaa70 con 0x7fc9681504c0 2024-09-13T18:01:17.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc9677fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc96806e980 0x7fc9681eb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc9677fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc96806e980 0x7fc9681eb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47396/0 (socket says 172.21.15.40:47396) 2024-09-13T18:01:17.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc9677fe700 1 -- 172.21.15.40:0/794013518 learned_addr learned my addr 172.21.15.40:0/794013518 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:17.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc967fff700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9681558d0 0x7fc9681546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc95effd700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9681504c0 0x7fc968154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc9677fe700 1 -- 172.21.15.40:0/794013518 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9681504c0 msgr2=0x7fc968154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc9677fe700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9681504c0 0x7fc968154170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc9677fe700 1 -- 172.21.15.40:0/794013518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9681558d0 msgr2=0x7fc9681546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc9677fe700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9681558d0 0x7fc9681546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc9677fe700 1 -- 172.21.15.40:0/794013518 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95000b560 con 0x7fc96806e980 2024-09-13T18:01:17.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc9677fe700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc96806e980 0x7fc9681eb1a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc958010340 tx=0x7fc958010650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:17.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc964ff9700 1 -- 172.21.15.40:0/794013518 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958012050 con 0x7fc96806e980 2024-09-13T18:01:17.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc964ff9700 1 -- 172.21.15.40:0/794013518 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc958013040 con 0x7fc96806e980 2024-09-13T18:01:17.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc96cf3c700 1 -- 172.21.15.40:0/794013518 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9681ead10 con 0x7fc96806e980 2024-09-13T18:01:17.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc964ff9700 1 -- 172.21.15.40:0/794013518 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95800d2d0 con 0x7fc96806e980 2024-09-13T18:01:17.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.564+0000 7fc96cf3c700 1 -- 172.21.15.40:0/794013518 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9680c1830 con 0x7fc96806e980 2024-09-13T18:01:17.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.568+0000 7fc96cf3c700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9681eb6e0 0x7fc96814e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.568+0000 7fc96cf3c700 1 -- 172.21.15.40:0/794013518 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc968067590 con 0x7fc9681eb6e0 2024-09-13T18:01:17.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.568+0000 7fc95effd700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9681eb6e0 0x7fc96814e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.568+0000 7fc95effd700 1 -- 172.21.15.40:0/794013518 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc968156c10 con 0x7fc9681eb6e0 2024-09-13T18:01:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.568+0000 7fc95effd700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9681eb6e0 0x7fc96814e300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc95000b900 tx=0x7fc950004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.568+0000 7fc964ff9700 1 -- 172.21.15.40:0/794013518 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc95800d430 con 0x7fc96806e980 2024-09-13T18:01:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.568+0000 7fc964ff9700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc948066760 0x7fc948068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.568+0000 7fc964ff9700 1 -- 172.21.15.40:0/794013518 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc958088870 con 0x7fc96806e980 2024-09-13T18:01:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.568+0000 7fc967fff700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc948066760 0x7fc948068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.568+0000 7fc967fff700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc948066760 0x7fc948068c20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc960007fd0 tx=0x7fc960007d60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.572+0000 7fc964ff9700 1 -- 172.21.15.40:0/794013518 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95001b040 con 0x7fc9681eb6e0 2024-09-13T18:01:17.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.572+0000 7fc964ff9700 1 -- 172.21.15.40:0/794013518 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc950004580 con 0x7fc9681eb6e0 2024-09-13T18:01:17.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.572+0000 7fc964ff9700 1 -- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9681eb6e0 msgr2=0x7fc96814e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.572+0000 7fc964ff9700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9681eb6e0 0x7fc96814e300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc95000b900 tx=0x7fc950004770 comp rx=0 tx=0).stop 2024-09-13T18:01:17.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.572+0000 7fc964ff9700 1 -- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9681eb6e0 msgr2=0x7fc96814e300 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:01:17.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.572+0000 7fc964ff9700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9681eb6e0 0x7fc96814e300 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.580+0000 7fc96cf3c700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc968070c00 0x7fc968140a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.580+0000 7fc96cf3c700 1 -- 172.21.15.40:0/794013518 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc968156930 con 0x7fc968070c00 2024-09-13T18:01:17.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.580+0000 7fc95effd700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc968070c00 0x7fc968140a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.580+0000 7fc95effd700 1 -- 172.21.15.40:0/794013518 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc968156ef0 con 0x7fc968070c00 2024-09-13T18:01:17.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.580+0000 7fc95effd700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc968070c00 0x7fc968140a70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc9680cc520 tx=0x7fc9500090e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:17.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.580+0000 7fc964ff9700 1 -- 172.21.15.40:0/794013518 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95002f040 con 0x7fc968070c00 2024-09-13T18:01:17.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.580+0000 7fc964ff9700 1 -- 172.21.15.40:0/794013518 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4784 (secure 0 0 0) 0x7fc950003ea0 con 0x7fc968070c00 2024-09-13T18:01:17.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.580+0000 7fc964ff9700 1 -- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc968070c00 msgr2=0x7fc968140a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.580+0000 7fc964ff9700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc968070c00 0x7fc968140a70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc9680cc520 tx=0x7fc9500090e0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.584+0000 7fc964ff9700 1 -- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc968070c00 msgr2=0x7fc968140a70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:01:17.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.584+0000 7fc964ff9700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc968070c00 0x7fc968140a70 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc9680cc520 tx=0x7fc9500090e0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.592 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:01:17.592 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:01:17.592 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:01:17.592 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:01:17.592 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 82, 2024-09-13T18:01:17.592 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:01:17.592 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:01:17.593 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:01:17.593 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:01:17.593 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:17.593 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 13, 2024-09-13T18:01:17.593 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:01:17.593 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:01:17.593 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:01:17.593 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:17.593 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:17.593 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:17.593 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:17.593 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:17.594 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:17.594 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:17.594 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:17.594 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:17.594 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:17.594 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:17.594 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:01:17.594 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:01:17.594 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:17.594 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:17.594 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:17.594 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:17.595 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:17.595 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:17.595 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:17.595 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:17.595 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:17.595 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:17.595 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:17.595 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.595 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:01:17.595 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:01:17.595 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:01:17.595 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:01:17.596 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:01:17.596 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:01:17.596 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:01:17.596 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:01:17.596 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:01:17.596 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:01:17.596 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:01:17.596 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:01:17.596 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:01:17.596 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:01:17.596 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:01:17.596 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:01:17.596 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:01:17.597 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:17.597 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:17.597 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:17.597 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:17.597 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:17.597 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:17.597 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:17.597 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:17.597 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:17.597 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:17.597 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:17.597 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:01:17.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.597 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:01:17.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.598 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:01:17.598 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:01:17.598 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:17.598 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:17.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.598 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:17.598 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:01:17.598 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:17.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.598 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:17.598 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:01:17.598 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:17.599 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.599 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:17.599 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.599 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:01:17.599 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:01:17.599 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:17.599 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:17.599 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:17.599 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.599 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.599 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:01:17.600 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:01:17.600 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:17.600 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:17.600 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.600 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:17.600 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:01:17.600 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:17.600 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.600 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.600 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:17.600 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:01:17.600 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:17.600 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.600 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:17.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.601 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:01:17.601 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:01:17.601 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:17.601 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:17.601 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:17.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.601 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.602 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:01:17.602 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:01:17.602 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:17.602 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:17.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.602 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:17.602 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:01:17.602 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:17.602 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.602 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.602 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:17.602 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:01:17.602 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:17.603 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.603 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:17.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.603 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:01:17.603 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:01:17.603 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:17.603 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:17.603 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:17.603 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.603 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:17.603 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.603 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:01:17.604 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:01:17.604 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.604 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:17.604 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:17.604 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:01:17.604 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.604 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:17.604 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:01:17.604 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.604 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:17.604 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:17.605 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:01:17.605 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.605 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:17.605 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:01:17.605 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.605 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:17.605 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:17.605 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:01:17.605 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.605 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:17.605 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:01:17.605 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.605 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:17.605 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:17.606 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:01:17.606 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.606 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:17.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.606 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:01:17.606 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:01:17.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.584+0000 7fc96cf3c700 1 -- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc948066760 msgr2=0x7fc948068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.584+0000 7fc96cf3c700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc948066760 0x7fc948068c20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc960007fd0 tx=0x7fc960007d60 comp rx=0 tx=0).stop 2024-09-13T18:01:17.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.584+0000 7fc967fff700 1 -- 172.21.15.40:0/794013518 reap_dead start 2024-09-13T18:01:17.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.584+0000 7fc96cf3c700 1 -- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc96806e980 msgr2=0x7fc9681eb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.584+0000 7fc96cf3c700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc96806e980 0x7fc9681eb1a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc958010340 tx=0x7fc958010650 comp rx=0 tx=0).stop 2024-09-13T18:01:17.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.584+0000 7fc96cf3c700 1 -- 172.21.15.40:0/794013518 shutdown_connections 2024-09-13T18:01:17.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.584+0000 7fc96cf3c700 1 --2- 172.21.15.40:0/794013518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc96806e980 0x7fc9681eb1a0 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc958010340 tx=0x7fc958010650 comp rx=0 tx=0).stop 2024-09-13T18:01:17.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.584+0000 7fc96cf3c700 1 -- 172.21.15.40:0/794013518 >> 172.21.15.40:0/794013518 conn(0x7fc9680cbc40 msgr2=0x7fc9681450f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:17.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.584+0000 7fc96cf3c700 1 -- 172.21.15.40:0/794013518 shutdown_connections 2024-09-13T18:01:17.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.584+0000 7fc96cf3c700 1 -- 172.21.15.40:0/794013518 wait complete. 2024-09-13T18:01:17.609 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:01:17.609 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:01:17.609 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:01:17.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.716+0000 7f724dbb9700 1 Processor -- start 2024-09-13T18:01:17.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.716+0000 7f724dbb9700 1 -- start start 2024-09-13T18:01:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.716+0000 7f724dbb9700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7248153a10 0x7f7248159540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.716+0000 7f724dbb9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f72480c4430 0x7f72480c5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.716+0000 7f724dbb9700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72480c4880 con 0x7f72480c4030 2024-09-13T18:01:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.716+0000 7f724dbb9700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f724815a490 con 0x7f72480c4430 2024-09-13T18:01:17.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.716+0000 7f724dbb9700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7248159bf0 con 0x7f7248153a10 2024-09-13T18:01:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.716+0000 7f724d3b8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f72480c4430 0x7f72480c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.716+0000 7f724d3b8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f72480c4430 0x7f72480c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47424/0 (socket says 172.21.15.40:47424) 2024-09-13T18:01:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.716+0000 7f724d3b8700 1 -- 172.21.15.40:0/160061905 learned_addr learned my addr 172.21.15.40:0/160061905 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.716+0000 7f724cbb7700 1 --2- 172.21.15.40:0/160061905 >> v2:172.21.15.43:3300/0 conn(0x7f7248153a10 0x7f7248159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.716+0000 7f724d3b8700 1 -- 172.21.15.40:0/160061905 >> v2:172.21.15.43:3300/0 conn(0x7f7248153a10 msgr2=0x7f7248159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:01:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.716+0000 7f724d3b8700 1 --2- 172.21.15.40:0/160061905 >> v2:172.21.15.43:3300/0 conn(0x7f7248153a10 0x7f7248159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.716+0000 7f724d3b8700 1 -- 172.21.15.40:0/160061905 >> v1:172.21.15.43:6789/0 conn(0x7f72480c4030 legacy=0x7f72480c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.716+0000 7f724d3b8700 1 -- 172.21.15.40:0/160061905 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7248159ed0 con 0x7f72480c4430 2024-09-13T18:01:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.716+0000 7f724d3b8700 1 --2- 172.21.15.40:0/160061905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72480c4430 0x7f72480c5400 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f724808c380 tx=0x7f724000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b9069643ddad78 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:17.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.716+0000 7f72477fe700 1 -- 172.21.15.40:0/160061905 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724000e070 con 0x7f72480c4430 2024-09-13T18:01:17.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.716+0000 7f72477fe700 1 -- 172.21.15.40:0/160061905 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724000b910 con 0x7f72480c4430 2024-09-13T18:01:17.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.716+0000 7f72477fe700 1 -- 172.21.15.40:0/160061905 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724000bcc0 con 0x7f72480c4430 2024-09-13T18:01:17.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f724dbb9700 1 -- 172.21.15.40:0/160061905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72480c4430 msgr2=0x7f72480c5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f724dbb9700 1 --2- 172.21.15.40:0/160061905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72480c4430 0x7f72480c5400 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f724808c380 tx=0x7f724000b330 comp rx=0 tx=0).stop 2024-09-13T18:01:17.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f724dbb9700 1 -- 172.21.15.40:0/160061905 shutdown_connections 2024-09-13T18:01:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f724dbb9700 1 --2- 172.21.15.40:0/160061905 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72480c4430 0x7f72480c5400 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f724dbb9700 1 --2- 172.21.15.40:0/160061905 >> v2:172.21.15.43:3300/0 conn(0x7f7248153a10 0x7f7248159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f724dbb9700 1 -- 172.21.15.40:0/160061905 >> 172.21.15.40:0/160061905 conn(0x7f7248066270 msgr2=0x7f7248066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f724dbb9700 1 -- 172.21.15.40:0/160061905 shutdown_connections 2024-09-13T18:01:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f724dbb9700 1 -- 172.21.15.40:0/160061905 wait complete. 2024-09-13T18:01:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f724dbb9700 1 Processor -- start 2024-09-13T18:01:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f724dbb9700 1 -- start start 2024-09-13T18:01:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f724dbb9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72480c4030 0x7f724814f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f724dbb9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72480c4430 0x7f724814f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f724dbb9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248153a10 0x7f72480c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f724dbb9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724814f590 con 0x7f72480c4430 2024-09-13T18:01:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f724dbb9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724815d8b0 con 0x7f7248153a10 2024-09-13T18:01:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f724dbb9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72480c8d60 con 0x7f72480c4030 2024-09-13T18:01:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f7247fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72480c4430 0x7f724814f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f7247fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72480c4430 0x7f724814f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47426/0 (socket says 172.21.15.40:47426) 2024-09-13T18:01:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f7247fff700 1 -- 172.21.15.40:0/2462893035 learned_addr learned my addr 172.21.15.40:0/2462893035 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f724d3b8700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248153a10 0x7f72480c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.720+0000 7f724cbb7700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72480c4030 0x7f724814f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f7247fff700 1 -- 172.21.15.40:0/2462893035 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72480c4030 msgr2=0x7f724814f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f7247fff700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72480c4030 0x7f724814f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f7247fff700 1 -- 172.21.15.40:0/2462893035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248153a10 msgr2=0x7f72480c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f7247fff700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248153a10 0x7f72480c8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f7247fff700 1 -- 172.21.15.40:0/2462893035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7240009b30 con 0x7f72480c4430 2024-09-13T18:01:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f7247fff700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72480c4430 0x7f724814f050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f723400c440 tx=0x7f723400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f72457fa700 1 -- 172.21.15.40:0/2462893035 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72340052d0 con 0x7f72480c4430 2024-09-13T18:01:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f72457fa700 1 -- 172.21.15.40:0/2462893035 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7234005430 con 0x7f72480c4430 2024-09-13T18:01:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f72457fa700 1 -- 172.21.15.40:0/2462893035 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72340055a0 con 0x7f72480c4430 2024-09-13T18:01:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f724dbb9700 1 -- 172.21.15.40:0/2462893035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72480c92e0 con 0x7f72480c4430 2024-09-13T18:01:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f724dbb9700 1 -- 172.21.15.40:0/2462893035 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72480c6370 con 0x7f72480c4430 2024-09-13T18:01:17.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f724dbb9700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72480c6530 0x7f72481e3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f724dbb9700 1 -- 172.21.15.40:0/2462893035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7248074830 con 0x7f72480c6530 2024-09-13T18:01:17.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f724cbb7700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72480c6530 0x7f72481e3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f72457fa700 1 -- 172.21.15.40:0/2462893035 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f72340036a0 con 0x7f72480c4430 2024-09-13T18:01:17.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f72457fa700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f722c066760 0x7f722c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f724cbb7700 1 -- 172.21.15.40:0/2462893035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f72480c4ee0 con 0x7f72480c6530 2024-09-13T18:01:17.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f724d3b8700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f722c066760 0x7f722c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f72457fa700 1 -- 172.21.15.40:0/2462893035 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f72340837b0 con 0x7f72480c4430 2024-09-13T18:01:17.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f724cbb7700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72480c6530 0x7f72481e3cd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f723c011830 tx=0x7f723c011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:17.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f724d3b8700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f722c066760 0x7f722c068c20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f724000b370 tx=0x7f7240009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:17.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.724+0000 7f72457fa700 1 -- 172.21.15.40:0/2462893035 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c004230 con 0x7f72480c6530 2024-09-13T18:01:17.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.728+0000 7f72457fa700 1 -- 172.21.15.40:0/2462893035 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f723c004390 con 0x7f72480c6530 2024-09-13T18:01:17.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.728+0000 7f72457fa700 1 -- 172.21.15.40:0/2462893035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72480c6530 msgr2=0x7f72481e3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.728+0000 7f72457fa700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72480c6530 0x7f72481e3cd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f723c011830 tx=0x7f723c011bf0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.728+0000 7f72457fa700 1 -- 172.21.15.40:0/2462893035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72480c6530 msgr2=0x7f72481e3cd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:01:17.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.728+0000 7f72457fa700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72480c6530 0x7f72481e3cd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.736+0000 7f724dbb9700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248149820 0x7f7248149c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.736+0000 7f724dbb9700 1 -- 172.21.15.40:0/2462893035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72480c4c00 con 0x7f7248149820 2024-09-13T18:01:17.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.736+0000 7f724cbb7700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248149820 0x7f7248149c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.736+0000 7f724cbb7700 1 -- 172.21.15.40:0/2462893035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f72480ca590 con 0x7f7248149820 2024-09-13T18:01:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.736+0000 7f724cbb7700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248149820 0x7f7248149c30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f723c011830 tx=0x7f723c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:17.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.736+0000 7f72457fa700 1 -- 172.21.15.40:0/2462893035 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c024070 con 0x7f7248149820 2024-09-13T18:01:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.736+0000 7f72457fa700 1 -- 172.21.15.40:0/2462893035 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7f723c015c60 con 0x7f7248149820 2024-09-13T18:01:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.736+0000 7f72457fa700 1 -- 172.21.15.40:0/2462893035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248149820 msgr2=0x7f7248149c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.736+0000 7f72457fa700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248149820 0x7f7248149c30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f723c011830 tx=0x7f723c004770 comp rx=0 tx=0).stop 2024-09-13T18:01:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.736+0000 7f72457fa700 1 -- 172.21.15.40:0/2462893035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248149820 msgr2=0x7f7248149c30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:01:17.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.736+0000 7f72457fa700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7248149820 0x7f7248149c30 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f723c011830 tx=0x7f723c004770 comp rx=0 tx=0).stop 2024-09-13T18:01:17.747 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:01:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:01:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:01:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:01:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 82, 2024-09-13T18:01:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:01:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:01:17.747 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:01:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:01:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 13, 2024-09-13T18:01:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:01:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:01:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:01:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:17.748 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:01:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:01:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:17.749 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:01:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:01:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:01:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:01:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:01:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:01:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:01:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:01:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:01:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:01:17.750 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:01:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:01:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:01:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:01:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:01:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:01:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:01:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:17.751 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:01:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:01:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:01:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:01:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:17.752 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:01:17.753 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:17.753 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.753 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.753 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:17.753 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:01:17.753 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:17.753 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.753 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:17.753 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.753 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:01:17.753 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:01:17.753 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:17.753 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:01:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:01:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:01:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:17.754 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:01:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:01:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:01:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.755 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:01:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:01:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:01:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:01:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:17.756 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.757 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:17.757 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.757 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:01:17.757 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:01:17.757 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:17.757 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:17.757 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:17.757 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.757 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:17.757 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.757 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:01:17.757 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:01:17.757 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.758 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:17.758 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:17.758 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:01:17.758 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.758 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:17.758 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:01:17.758 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.758 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:17.758 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:17.758 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:01:17.758 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.758 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:17.758 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:01:17.758 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.759 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:17.759 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:17.759 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:01:17.759 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.759 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:17.759 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:01:17.759 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.759 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:17.759 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:17.759 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:01:17.759 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.759 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:17.760 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:01:17.760 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.760 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:17.760 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:17.760 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:01:17.760 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.760 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:17.760 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.760 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:01:17.761 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:01:17.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.736+0000 7f724dbb9700 1 -- 172.21.15.40:0/2462893035 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f722c066760 msgr2=0x7f722c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.736+0000 7f724dbb9700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f722c066760 0x7f722c068c20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f724000b370 tx=0x7f7240009000 comp rx=0 tx=0).stop 2024-09-13T18:01:17.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.736+0000 7f724dbb9700 1 -- 172.21.15.40:0/2462893035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72480c4430 msgr2=0x7f724814f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.736+0000 7f724dbb9700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72480c4430 0x7f724814f050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f723400c440 tx=0x7f723400c800 comp rx=0 tx=0).stop 2024-09-13T18:01:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.736+0000 7f724d3b8700 1 -- 172.21.15.40:0/2462893035 reap_dead start 2024-09-13T18:01:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.736+0000 7f724dbb9700 1 -- 172.21.15.40:0/2462893035 shutdown_connections 2024-09-13T18:01:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.736+0000 7f724dbb9700 1 --2- 172.21.15.40:0/2462893035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72480c4430 0x7f724814f050 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.736+0000 7f724dbb9700 1 -- 172.21.15.40:0/2462893035 >> 172.21.15.40:0/2462893035 conn(0x7f7248066270 msgr2=0x7f72480cbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.740+0000 7f724dbb9700 1 -- 172.21.15.40:0/2462893035 shutdown_connections 2024-09-13T18:01:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.740+0000 7f724dbb9700 1 -- 172.21.15.40:0/2462893035 wait complete. 2024-09-13T18:01:17.763 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:01:17.763 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:01:17.763 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:01:17.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd073fff700 1 Processor -- start 2024-09-13T18:01:17.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd073fff700 1 -- start start 2024-09-13T18:01:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd073fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd0740c7f60 0x7fd074072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd073fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0740c5290 con 0x7fd0740c7320 2024-09-13T18:01:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd073fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0740727f0 con 0x7fd0740c8340 2024-09-13T18:01:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd073fff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd074071390 con 0x7fd0740c7f60 2024-09-13T18:01:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd06bfff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd0740c7320 0x7fd07415ab50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33070/0 (socket says 172.21.15.40:33070) 2024-09-13T18:01:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd06bfff700 1 -- 172.21.15.40:0/1436029277 learned_addr learned my addr 172.21.15.40:0/1436029277 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:01:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd072ffd700 1 --2- 172.21.15.40:0/1436029277 >> v2:172.21.15.43:3300/0 conn(0x7fd0740c7f60 0x7fd074072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd0727fc700 1 -- 172.21.15.40:0/1436029277 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305485005 0 0) 0x7fd0740c5290 con 0x7fd0740c7320 2024-09-13T18:01:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd0727fc700 1 -- 172.21.15.40:0/1436029277 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0600035d0 con 0x7fd0740c7320 2024-09-13T18:01:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd0727fc700 1 -- 172.21.15.40:0/1436029277 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2231839824 0 0) 0x7fd0600035d0 con 0x7fd0740c7320 2024-09-13T18:01:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd0727fc700 1 -- 172.21.15.40:0/1436029277 >> v2:172.21.15.43:3300/0 conn(0x7fd0740c7f60 msgr2=0x7fd074072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:01:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd0727fc700 1 --2- 172.21.15.40:0/1436029277 >> v2:172.21.15.43:3300/0 conn(0x7fd0740c7f60 0x7fd074072200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd0727fc700 1 -- 172.21.15.40:0/1436029277 >> v1:172.21.15.43:6789/0 conn(0x7fd0740c8340 legacy=0x7fd074151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd0727fc700 1 -- 172.21.15.40:0/1436029277 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0740c5870 con 0x7fd0740c7320 2024-09-13T18:01:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd0727fc700 1 -- 172.21.15.40:0/1436029277 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fd05c001670 con 0x7fd0740c7320 2024-09-13T18:01:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd0727fc700 1 -- 172.21.15.40:0/1436029277 >> v1:172.21.15.40:6789/0 conn(0x7fd0740c7320 legacy=0x7fd07415ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd0727fc700 1 --2- 172.21.15.40:0/1436029277 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd060003a20 0x7fd060003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd0727fc700 1 --2- 172.21.15.40:0/1436029277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd060004870 0x7fd060004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd0727fc700 1 --2- 172.21.15.40:0/1436029277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd060005230 0x7fd06000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd0727fc700 1 -- 172.21.15.40:0/1436029277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd074071390 con 0x7fd060004870 2024-09-13T18:01:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd0727fc700 1 -- 172.21.15.40:0/1436029277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06000c080 con 0x7fd060005230 2024-09-13T18:01:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd0727fc700 1 -- 172.21.15.40:0/1436029277 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06000c240 con 0x7fd060003a20 2024-09-13T18:01:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd06bfff700 1 --2- 172.21.15.40:0/1436029277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd060004870 0x7fd060004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd0737fe700 1 --2- 172.21.15.40:0/1436029277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd060005230 0x7fd06000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd072ffd700 1 --2- 172.21.15.40:0/1436029277 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd060003a20 0x7fd060003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.872+0000 7fd06bfff700 1 -- 172.21.15.40:0/1436029277 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd060003a20 msgr2=0x7fd060003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:01:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd06bfff700 1 --2- 172.21.15.40:0/1436029277 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd060003a20 0x7fd060003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd06bfff700 1 -- 172.21.15.40:0/1436029277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd060005230 msgr2=0x7fd06000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:01:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd06bfff700 1 --2- 172.21.15.40:0/1436029277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd060005230 0x7fd06000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd0737fe700 1 -- 172.21.15.40:0/1436029277 reap_dead start 2024-09-13T18:01:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd06bfff700 1 -- 172.21.15.40:0/1436029277 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0740c5870 con 0x7fd060004870 2024-09-13T18:01:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd06bfff700 1 --2- 172.21.15.40:0/1436029277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd060004870 0x7fd060004cf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00c340 tx=0x7fd0600058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3717b6f10f7c045 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd0727fc700 1 -- 172.21.15.40:0/1436029277 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c005980 con 0x7fd060004870 2024-09-13T18:01:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd0727fc700 1 -- 172.21.15.40:0/1436029277 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05c005ae0 con 0x7fd060004870 2024-09-13T18:01:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd0727fc700 1 -- 172.21.15.40:0/1436029277 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c005e90 con 0x7fd060004870 2024-09-13T18:01:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd073fff700 1 -- 172.21.15.40:0/1436029277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd060004870 msgr2=0x7fd060004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd073fff700 1 --2- 172.21.15.40:0/1436029277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd060004870 0x7fd060004cf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00c340 tx=0x7fd0600058a0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd073fff700 1 -- 172.21.15.40:0/1436029277 shutdown_connections 2024-09-13T18:01:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd073fff700 1 --2- 172.21.15.40:0/1436029277 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd060004870 0x7fd060004cf0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd073fff700 1 -- 172.21.15.40:0/1436029277 >> 172.21.15.40:0/1436029277 conn(0x7fd074066090 msgr2=0x7fd0740664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd073fff700 1 -- 172.21.15.40:0/1436029277 shutdown_connections 2024-09-13T18:01:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd073fff700 1 -- 172.21.15.40:0/1436029277 wait complete. 2024-09-13T18:01:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd073fff700 1 Processor -- start 2024-09-13T18:01:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd073fff700 1 -- start start 2024-09-13T18:01:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd073fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd060004870 0x7fd0740c04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd073fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0740c2650 0x7fd074146210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd073fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd074140790 0x7fd074140c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd073fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0740c4080 con 0x7fd060004870 2024-09-13T18:01:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd073fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0741f0210 con 0x7fd074140790 2024-09-13T18:01:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd073fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0741ef590 con 0x7fd0740c2650 2024-09-13T18:01:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd072ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd060004870 0x7fd0740c04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd072ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd060004870 0x7fd0740c04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47444/0 (socket says 172.21.15.40:47444) 2024-09-13T18:01:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd072ffd700 1 -- 172.21.15.40:0/2185869867 learned_addr learned my addr 172.21.15.40:0/2185869867 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd06bfff700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0740c2650 0x7fd074146210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.876+0000 7fd0737fe700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd074140790 0x7fd074140c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.880+0000 7fd072ffd700 1 -- 172.21.15.40:0/2185869867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0740c2650 msgr2=0x7fd074146210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.880+0000 7fd072ffd700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0740c2650 0x7fd074146210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.880+0000 7fd072ffd700 1 -- 172.21.15.40:0/2185869867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd074140790 msgr2=0x7fd074140c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.880+0000 7fd072ffd700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd074140790 0x7fd074140c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.880+0000 7fd072ffd700 1 -- 172.21.15.40:0/2185869867 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0741ef870 con 0x7fd060004870 2024-09-13T18:01:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.880+0000 7fd072ffd700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd060004870 0x7fd0740c04d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd064011ad0 tx=0x7fd064011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.880+0000 7fd06b7fe700 1 -- 172.21.15.40:0/2185869867 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06400fa60 con 0x7fd060004870 2024-09-13T18:01:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.880+0000 7fd06b7fe700 1 -- 172.21.15.40:0/2185869867 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06400fbc0 con 0x7fd060004870 2024-09-13T18:01:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.880+0000 7fd073fff700 1 -- 172.21.15.40:0/2185869867 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0741efb50 con 0x7fd060004870 2024-09-13T18:01:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.880+0000 7fd06b7fe700 1 -- 172.21.15.40:0/2185869867 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064014440 con 0x7fd060004870 2024-09-13T18:01:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.880+0000 7fd073fff700 1 -- 172.21.15.40:0/2185869867 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0741e3270 con 0x7fd060004870 2024-09-13T18:01:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.880+0000 7fd073fff700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0740c0ae0 0x7fd0740c0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.880+0000 7fd073fff700 1 -- 172.21.15.40:0/2185869867 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd074067720 con 0x7fd0740c0ae0 2024-09-13T18:01:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.880+0000 7fd06bfff700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0740c0ae0 0x7fd0740c0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.880+0000 7fd06bfff700 1 -- 172.21.15.40:0/2185869867 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd0740c6110 con 0x7fd0740c0ae0 2024-09-13T18:01:17.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.880+0000 7fd06bfff700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0740c0ae0 0x7fd0740c0f00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00c340 tx=0x7fd05c000fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:17.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.880+0000 7fd06b7fe700 1 -- 172.21.15.40:0/2185869867 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd0640145a0 con 0x7fd060004870 2024-09-13T18:01:17.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.880+0000 7fd06b7fe700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd054066760 0x7fd054068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.880+0000 7fd06b7fe700 1 -- 172.21.15.40:0/2185869867 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd064084d60 con 0x7fd060004870 2024-09-13T18:01:17.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.884+0000 7fd0737fe700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd054066760 0x7fd054068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.884+0000 7fd06b7fe700 1 -- 172.21.15.40:0/2185869867 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c0070d0 con 0x7fd0740c0ae0 2024-09-13T18:01:17.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.884+0000 7fd06b7fe700 1 -- 172.21.15.40:0/2185869867 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd05c007230 con 0x7fd0740c0ae0 2024-09-13T18:01:17.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.884+0000 7fd06b7fe700 1 -- 172.21.15.40:0/2185869867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0740c0ae0 msgr2=0x7fd0740c0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.884+0000 7fd06b7fe700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0740c0ae0 0x7fd0740c0f00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00c340 tx=0x7fd05c000fe0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.884+0000 7fd06b7fe700 1 -- 172.21.15.40:0/2185869867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0740c0ae0 msgr2=0x7fd0740c0f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:01:17.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.884+0000 7fd06b7fe700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0740c0ae0 0x7fd0740c0f00 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00c340 tx=0x7fd05c000fe0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.884+0000 7fd0737fe700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd054066760 0x7fd054068c20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd0740710e0 tx=0x7fd06c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:17.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.896+0000 7fd073fff700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd07414fc70 0x7fd0741500a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:17.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.896+0000 7fd073fff700 1 -- 172.21.15.40:0/2185869867 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0741efce0 con 0x7fd07414fc70 2024-09-13T18:01:17.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.896+0000 7fd06bfff700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd07414fc70 0x7fd0741500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:17.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.896+0000 7fd06bfff700 1 -- 172.21.15.40:0/2185869867 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd0740c5e30 con 0x7fd07414fc70 2024-09-13T18:01:17.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.896+0000 7fd06bfff700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd07414fc70 0x7fd0741500a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd07406e650 tx=0x7fd05c010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:17.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.896+0000 7fd06b7fe700 1 -- 172.21.15.40:0/2185869867 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c000ed0 con 0x7fd07414fc70 2024-09-13T18:01:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.896+0000 7fd06b7fe700 1 -- 172.21.15.40:0/2185869867 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4450 (secure 0 0 0) 0x7fd05c02e070 con 0x7fd07414fc70 2024-09-13T18:01:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.900+0000 7fd06b7fe700 1 -- 172.21.15.40:0/2185869867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd07414fc70 msgr2=0x7fd0741500a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.900+0000 7fd06b7fe700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd07414fc70 0x7fd0741500a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd07406e650 tx=0x7fd05c010000 comp rx=0 tx=0).stop 2024-09-13T18:01:17.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.900+0000 7fd06b7fe700 1 -- 172.21.15.40:0/2185869867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd07414fc70 msgr2=0x7fd0741500a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:01:17.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.900+0000 7fd06b7fe700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd07414fc70 0x7fd0741500a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.908 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:01:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:01:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:01:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:01:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 82, 2024-09-13T18:01:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:01:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:01:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:01:17.908 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:01:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 13, 2024-09-13T18:01:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:01:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:01:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:01:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:17.909 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:01:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:01:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:17.910 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:01:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:01:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:01:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:01:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:01:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:01:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:01:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:01:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:01:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:01:17.911 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:01:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:01:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:01:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:01:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:01:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:01:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:01:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:17.912 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:01:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:01:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:01:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:01:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.913 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:01:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:01:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.914 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:01:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:01:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:01:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:01:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:17.915 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:01:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:01:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:01:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:01:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:17.916 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:01:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:01:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:01:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.917 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:01:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:01:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:01:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.918 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:01:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:01:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:01:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:01:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:01:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:17.919 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:01:17.920 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:01:17.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.900+0000 7fd073fff700 1 -- 172.21.15.40:0/2185869867 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd054066760 msgr2=0x7fd054068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.900+0000 7fd073fff700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd054066760 0x7fd054068c20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd0740710e0 tx=0x7fd06c009040 comp rx=0 tx=0).stop 2024-09-13T18:01:17.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.900+0000 7fd073fff700 1 -- 172.21.15.40:0/2185869867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd060004870 msgr2=0x7fd0740c04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:17.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.900+0000 7fd073fff700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd060004870 0x7fd0740c04d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd064011ad0 tx=0x7fd064011e90 comp rx=0 tx=0).stop 2024-09-13T18:01:17.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.900+0000 7fd0737fe700 1 -- 172.21.15.40:0/2185869867 reap_dead start 2024-09-13T18:01:17.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.900+0000 7fd073fff700 1 -- 172.21.15.40:0/2185869867 shutdown_connections 2024-09-13T18:01:17.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.900+0000 7fd073fff700 1 --2- 172.21.15.40:0/2185869867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd060004870 0x7fd0740c04d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:17.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.900+0000 7fd073fff700 1 -- 172.21.15.40:0/2185869867 >> 172.21.15.40:0/2185869867 conn(0x7fd074066090 msgr2=0x7fd0741e28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:17.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.900+0000 7fd073fff700 1 -- 172.21.15.40:0/2185869867 shutdown_connections 2024-09-13T18:01:17.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:17.900+0000 7fd073fff700 1 -- 172.21.15.40:0/2185869867 wait complete. 2024-09-13T18:01:17.923 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:01:17.924 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-13T18:01:17.924 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T18:01:17.924 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-13T18:01:17.924 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-13T18:01:17.924 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T18:01:17.924 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:01:17.968 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T18:01:17.968 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T18:01:17.969 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:01:18.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a759e700 1 Processor -- start 2024-09-13T18:01:18.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a759e700 1 -- start start 2024-09-13T18:01:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a759e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f77a8153880 0x7f77a80c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a759e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f77a815cf60 0x7f77a815d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a759e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a815d910 con 0x7f77a80cb2f0 2024-09-13T18:01:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a759e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a8159770 con 0x7f77a815cf60 2024-09-13T18:01:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a759e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a8159a50 con 0x7f77a8153880 2024-09-13T18:01:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a659c700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7f77a8153880 msgr2=0x7f77a80c9ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:01:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a659c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f77a8153880 0x7f77a80c9ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:18.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a6d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f77a815cf60 0x7f77a815d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a6d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f77a815cf60 0x7f77a815d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47480/0 (socket says 172.21.15.40:47480) 2024-09-13T18:01:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a6d9d700 1 -- 172.21.15.40:0/3313759570 learned_addr learned my addr 172.21.15.40:0/3313759570 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a6d9d700 1 -- 172.21.15.40:0/3313759570 >> v2:172.21.15.43:3300/0 conn(0x7f77a8153880 msgr2=0x7f77a80c9ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:01:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a6d9d700 1 --2- 172.21.15.40:0/3313759570 >> v2:172.21.15.43:3300/0 conn(0x7f77a8153880 0x7f77a80c9ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a6d9d700 1 -- 172.21.15.40:0/3313759570 >> v1:172.21.15.40:6789/0 conn(0x7f77a80cb2f0 legacy=0x7f77a80cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a6d9d700 1 -- 172.21.15.40:0/3313759570 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a8159d30 con 0x7f77a815cf60 2024-09-13T18:01:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a6d9d700 1 --2- 172.21.15.40:0/3313759570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a815cf60 0x7f77a815d3d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f77a80ccac0 tx=0x7f77a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77565235ab072844 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a5d9b700 1 -- 172.21.15.40:0/3313759570 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a000e070 con 0x7f77a815cf60 2024-09-13T18:01:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a5d9b700 1 -- 172.21.15.40:0/3313759570 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a000b9c0 con 0x7f77a815cf60 2024-09-13T18:01:18.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a5d9b700 1 -- 172.21.15.40:0/3313759570 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a000bd70 con 0x7f77a815cf60 2024-09-13T18:01:18.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a759e700 1 -- 172.21.15.40:0/3313759570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a815cf60 msgr2=0x7f77a815d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:18.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a759e700 1 --2- 172.21.15.40:0/3313759570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a815cf60 0x7f77a815d3d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f77a80ccac0 tx=0x7f77a000b330 comp rx=0 tx=0).stop 2024-09-13T18:01:18.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a759e700 1 -- 172.21.15.40:0/3313759570 shutdown_connections 2024-09-13T18:01:18.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a759e700 1 --2- 172.21.15.40:0/3313759570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a815cf60 0x7f77a815d3d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:18.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a759e700 1 --2- 172.21.15.40:0/3313759570 >> v2:172.21.15.43:3300/0 conn(0x7f77a8153880 0x7f77a80c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:18.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a759e700 1 -- 172.21.15.40:0/3313759570 >> 172.21.15.40:0/3313759570 conn(0x7f77a806c5b0 msgr2=0x7f77a806c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:18.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a759e700 1 -- 172.21.15.40:0/3313759570 shutdown_connections 2024-09-13T18:01:18.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.072+0000 7f77a759e700 1 -- 172.21.15.40:0/3313759570 wait complete. 2024-09-13T18:01:18.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a759e700 1 Processor -- start 2024-09-13T18:01:18.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a759e700 1 -- start start 2024-09-13T18:01:18.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a80cb2f0 0x7f77a81f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:18.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a8153880 0x7f77a8158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:18.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f77a815cf60 0x7f77a8158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:18.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a80ce7c0 con 0x7f77a80cb2f0 2024-09-13T18:01:18.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a81f33b0 con 0x7f77a8153880 2024-09-13T18:01:18.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a81f3650 con 0x7f77a815cf60 2024-09-13T18:01:18.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f779ffff700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a8153880 msgr2=0x7f77a8158240 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:01:18.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a80cb2f0 0x7f77a81f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:18.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f779ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a8153880 0x7f77a8158240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:18.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a80cb2f0 0x7f77a81f2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47482/0 (socket says 172.21.15.40:47482) 2024-09-13T18:01:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a659c700 1 -- 172.21.15.40:0/547750537 learned_addr learned my addr 172.21.15.40:0/547750537 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a6d9d700 1 --2- 172.21.15.40:0/547750537 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f77a815cf60 0x7f77a8158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a659c700 1 -- 172.21.15.40:0/547750537 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f77a815cf60 msgr2=0x7f77a8158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a659c700 1 --2- 172.21.15.40:0/547750537 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f77a815cf60 0x7f77a8158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a659c700 1 -- 172.21.15.40:0/547750537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a8153880 msgr2=0x7f77a8158240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:01:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a659c700 1 --2- 172.21.15.40:0/547750537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a8153880 0x7f77a8158240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a659c700 1 -- 172.21.15.40:0/547750537 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a0009b30 con 0x7f77a80cb2f0 2024-09-13T18:01:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a6d9d700 1 --2- 172.21.15.40:0/547750537 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f77a815cf60 0x7f77a8158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:01:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a659c700 1 --2- 172.21.15.40:0/547750537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a80cb2f0 0x7f77a81f2a80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f779800b780 tx=0x7f779800bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f779effd700 1 -- 172.21.15.40:0/547750537 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798003e70 con 0x7f77a80cb2f0 2024-09-13T18:01:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f779effd700 1 -- 172.21.15.40:0/547750537 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7798005070 con 0x7f77a80cb2f0 2024-09-13T18:01:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a759e700 1 -- 172.21.15.40:0/547750537 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a80b17e0 con 0x7f77a80cb2f0 2024-09-13T18:01:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f779effd700 1 -- 172.21.15.40:0/547750537 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798005420 con 0x7f77a80cb2f0 2024-09-13T18:01:18.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.076+0000 7f77a759e700 1 -- 172.21.15.40:0/547750537 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a80ce4e0 con 0x7f77a80cb2f0 2024-09-13T18:01:18.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.080+0000 7f77a759e700 1 -- 172.21.15.40:0/547750537 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a80ca740 con 0x7f77a80cb2f0 2024-09-13T18:01:18.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.080+0000 7f779effd700 1 -- 172.21.15.40:0/547750537 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7798005580 con 0x7f77a80cb2f0 2024-09-13T18:01:18.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.080+0000 7f779effd700 1 --2- 172.21.15.40:0/547750537 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f77880666e0 0x7f7788068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:18.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.080+0000 7f779ffff700 1 --2- 172.21.15.40:0/547750537 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f77880666e0 0x7f7788068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:18.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.080+0000 7f779effd700 1 -- 172.21.15.40:0/547750537 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7798084130 con 0x7f77a80cb2f0 2024-09-13T18:01:18.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.080+0000 7f779ffff700 1 --2- 172.21.15.40:0/547750537 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f77880666e0 0x7f7788068ba0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7790001030 tx=0x7f7790007ae0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:18.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.084+0000 7f779effd700 1 -- 172.21.15.40:0/547750537 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f77980510e0 con 0x7f77a80cb2f0 2024-09-13T18:01:18.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.344+0000 7f77a759e700 1 -- 172.21.15.40:0/547750537 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f77a814d900 con 0x7f77a80cb2f0 2024-09-13T18:01:18.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.344+0000 7f779effd700 1 -- 172.21.15.40:0/547750537 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f7798005830 con 0x7f77a80cb2f0 2024-09-13T18:01:18.353 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":82,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:01:18.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.348+0000 7f77a759e700 1 -- 172.21.15.40:0/547750537 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f77880666e0 msgr2=0x7f7788068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:18.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.348+0000 7f77a759e700 1 --2- 172.21.15.40:0/547750537 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f77880666e0 0x7f7788068ba0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7790001030 tx=0x7f7790007ae0 comp rx=0 tx=0).stop 2024-09-13T18:01:18.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.348+0000 7f77a759e700 1 -- 172.21.15.40:0/547750537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a80cb2f0 msgr2=0x7f77a81f2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:18.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.348+0000 7f77a759e700 1 --2- 172.21.15.40:0/547750537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a80cb2f0 0x7f77a81f2a80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f779800b780 tx=0x7f779800bb40 comp rx=0 tx=0).stop 2024-09-13T18:01:18.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.348+0000 7f77a759e700 1 -- 172.21.15.40:0/547750537 shutdown_connections 2024-09-13T18:01:18.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.348+0000 7f77a759e700 1 --2- 172.21.15.40:0/547750537 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f77880666e0 0x7f7788068ba0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:18.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.348+0000 7f77a759e700 1 --2- 172.21.15.40:0/547750537 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f77a815cf60 0x7f77a8158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:18.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.348+0000 7f77a759e700 1 --2- 172.21.15.40:0/547750537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a8153880 0x7f77a8158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:18.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.348+0000 7f77a759e700 1 --2- 172.21.15.40:0/547750537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77a80cb2f0 0x7f77a81f2a80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:18.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.348+0000 7f77a759e700 1 -- 172.21.15.40:0/547750537 >> 172.21.15.40:0/547750537 conn(0x7f77a806c5b0 msgr2=0x7f77a80668b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:18.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.348+0000 7f77a759e700 1 -- 172.21.15.40:0/547750537 shutdown_connections 2024-09-13T18:01:18.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:18.348+0000 7f77a759e700 1 -- 172.21.15.40:0/547750537 wait complete. 2024-09-13T18:01:21.369 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:01:21.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.476+0000 7f2b76487700 1 Processor -- start 2024-09-13T18:01:21.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.476+0000 7f2b76487700 1 -- start start 2024-09-13T18:01:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b76487700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2b7014f9f0 0x7f2b70070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b76487700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2b7006ea60 0x7f2b70071620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b76487700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b7014fdd0 con 0x7f2b70070ae0 2024-09-13T18:01:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b76487700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b70070920 con 0x7f2b7006ea60 2024-09-13T18:01:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b76487700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b701564d0 con 0x7f2b7014f9f0 2024-09-13T18:01:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b75485700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7f2b7014f9f0 msgr2=0x7f2b70070290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:01:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b74c84700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2b7006ea60 0x7f2b70071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b75485700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2b7014f9f0 0x7f2b70070290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b74c84700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2b7006ea60 0x7f2b70071620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40310/0 (socket says 172.21.15.40:40310) 2024-09-13T18:01:21.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b74c84700 1 -- 172.21.15.40:0/384089899 learned_addr learned my addr 172.21.15.40:0/384089899 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:21.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b74c84700 1 -- 172.21.15.40:0/384089899 >> v2:172.21.15.43:3300/0 conn(0x7f2b7014f9f0 msgr2=0x7f2b70070290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:01:21.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b74c84700 1 --2- 172.21.15.40:0/384089899 >> v2:172.21.15.43:3300/0 conn(0x7f2b7014f9f0 0x7f2b70070290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:21.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b74c84700 1 -- 172.21.15.40:0/384089899 >> v1:172.21.15.40:6789/0 conn(0x7f2b70070ae0 legacy=0x7f2b7006d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:21.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b74c84700 1 -- 172.21.15.40:0/384089899 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b701558b0 con 0x7f2b7006ea60 2024-09-13T18:01:21.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b74c84700 1 --2- 172.21.15.40:0/384089899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b7006ea60 0x7f2b70071620 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2b7006beb0 tx=0x7f2b60009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19738dd87cfd25d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:21.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b67fff700 1 -- 172.21.15.40:0/384089899 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6000b8a0 con 0x7f2b7006ea60 2024-09-13T18:01:21.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b67fff700 1 -- 172.21.15.40:0/384089899 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6000ba00 con 0x7f2b7006ea60 2024-09-13T18:01:21.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b67fff700 1 -- 172.21.15.40:0/384089899 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6000bd90 con 0x7f2b7006ea60 2024-09-13T18:01:21.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b76487700 1 -- 172.21.15.40:0/384089899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b7006ea60 msgr2=0x7f2b70071620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:21.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b76487700 1 --2- 172.21.15.40:0/384089899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b7006ea60 0x7f2b70071620 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2b7006beb0 tx=0x7f2b60009bf0 comp rx=0 tx=0).stop 2024-09-13T18:01:21.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b76487700 1 -- 172.21.15.40:0/384089899 shutdown_connections 2024-09-13T18:01:21.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b76487700 1 --2- 172.21.15.40:0/384089899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b7006ea60 0x7f2b70071620 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:21.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b76487700 1 --2- 172.21.15.40:0/384089899 >> v2:172.21.15.43:3300/0 conn(0x7f2b7014f9f0 0x7f2b70070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:21.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b76487700 1 -- 172.21.15.40:0/384089899 >> 172.21.15.40:0/384089899 conn(0x7f2b7006a390 msgr2=0x7f2b7006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:21.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b76487700 1 -- 172.21.15.40:0/384089899 shutdown_connections 2024-09-13T18:01:21.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.480+0000 7f2b76487700 1 -- 172.21.15.40:0/384089899 wait complete. 2024-09-13T18:01:21.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b76487700 1 Processor -- start 2024-09-13T18:01:21.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b76487700 1 -- start start 2024-09-13T18:01:21.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b76487700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b7006ea60 0x7f2b70159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:21.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b76487700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b70070ae0 0x7f2b7014ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:21.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b76487700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2b7014f9f0 0x7f2b70158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:21.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b76487700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7006e390 con 0x7f2b7006ea60 2024-09-13T18:01:21.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b76487700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b700c4fa0 con 0x7f2b70070ae0 2024-09-13T18:01:21.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b76487700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7014abe0 con 0x7f2b7014f9f0 2024-09-13T18:01:21.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b75485700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b7006ea60 0x7f2b70159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b74c84700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b70070ae0 msgr2=0x7f2b7014ffb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:01:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b74c84700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b70070ae0 0x7f2b7014ffb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b75485700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b7006ea60 0x7f2b70159350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40326/0 (socket says 172.21.15.40:40326) 2024-09-13T18:01:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b75485700 1 -- 172.21.15.40:0/1329156672 learned_addr learned my addr 172.21.15.40:0/1329156672 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b75c86700 1 --2- 172.21.15.40:0/1329156672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2b7014f9f0 0x7f2b70158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b75485700 1 -- 172.21.15.40:0/1329156672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2b7014f9f0 msgr2=0x7f2b70158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b75485700 1 --2- 172.21.15.40:0/1329156672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2b7014f9f0 0x7f2b70158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b75485700 1 -- 172.21.15.40:0/1329156672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b70070ae0 msgr2=0x7f2b7014ffb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:01:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b75485700 1 --2- 172.21.15.40:0/1329156672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b70070ae0 0x7f2b7014ffb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b75485700 1 -- 172.21.15.40:0/1329156672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b60009570 con 0x7f2b7006ea60 2024-09-13T18:01:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b75c86700 1 --2- 172.21.15.40:0/1329156672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2b7014f9f0 0x7f2b70158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:01:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b75485700 1 --2- 172.21.15.40:0/1329156672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b7006ea60 0x7f2b70159350 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00b780 tx=0x7f2b6c00bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b65ffb700 1 -- 172.21.15.40:0/1329156672 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c003e70 con 0x7f2b7006ea60 2024-09-13T18:01:21.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b65ffb700 1 -- 172.21.15.40:0/1329156672 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6c005070 con 0x7f2b7006ea60 2024-09-13T18:01:21.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b76487700 1 -- 172.21.15.40:0/1329156672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b700c5180 con 0x7f2b7006ea60 2024-09-13T18:01:21.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b65ffb700 1 -- 172.21.15.40:0/1329156672 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c005420 con 0x7f2b7006ea60 2024-09-13T18:01:21.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b76487700 1 -- 172.21.15.40:0/1329156672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7014a9b0 con 0x7f2b7006ea60 2024-09-13T18:01:21.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.484+0000 7f2b76487700 1 -- 172.21.15.40:0/1329156672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b70070290 con 0x7f2b7006ea60 2024-09-13T18:01:21.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.488+0000 7f2b65ffb700 1 -- 172.21.15.40:0/1329156672 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2b6c005580 con 0x7f2b7006ea60 2024-09-13T18:01:21.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.488+0000 7f2b65ffb700 1 --2- 172.21.15.40:0/1329156672 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2b540666e0 0x7f2b54068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:21.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.488+0000 7f2b65ffb700 1 -- 172.21.15.40:0/1329156672 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2b6c083f00 con 0x7f2b7006ea60 2024-09-13T18:01:21.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.488+0000 7f2b74c84700 1 --2- 172.21.15.40:0/1329156672 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2b540666e0 0x7f2b54068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:21.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.488+0000 7f2b74c84700 1 --2- 172.21.15.40:0/1329156672 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2b540666e0 0x7f2b54068ba0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2b701f0020 tx=0x7f2b60008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:21.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.492+0000 7f2b65ffb700 1 -- 172.21.15.40:0/1329156672 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2b6c050f40 con 0x7f2b7006ea60 2024-09-13T18:01:21.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.756+0000 7f2b76487700 1 -- 172.21.15.40:0/1329156672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2b70147580 con 0x7f2b7006ea60 2024-09-13T18:01:21.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.760+0000 7f2b65ffb700 1 -- 172.21.15.40:0/1329156672 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f2b6c054560 con 0x7f2b7006ea60 2024-09-13T18:01:21.767 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":82,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:01:21.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.764+0000 7f2b76487700 1 -- 172.21.15.40:0/1329156672 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2b540666e0 msgr2=0x7f2b54068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:21.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.764+0000 7f2b76487700 1 --2- 172.21.15.40:0/1329156672 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2b540666e0 0x7f2b54068ba0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2b701f0020 tx=0x7f2b60008030 comp rx=0 tx=0).stop 2024-09-13T18:01:21.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.764+0000 7f2b76487700 1 -- 172.21.15.40:0/1329156672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b7006ea60 msgr2=0x7f2b70159350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:21.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.764+0000 7f2b76487700 1 --2- 172.21.15.40:0/1329156672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b7006ea60 0x7f2b70159350 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00b780 tx=0x7f2b6c00bb40 comp rx=0 tx=0).stop 2024-09-13T18:01:21.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.764+0000 7f2b76487700 1 -- 172.21.15.40:0/1329156672 shutdown_connections 2024-09-13T18:01:21.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.764+0000 7f2b76487700 1 --2- 172.21.15.40:0/1329156672 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2b540666e0 0x7f2b54068ba0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:21.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.764+0000 7f2b76487700 1 --2- 172.21.15.40:0/1329156672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2b7014f9f0 0x7f2b70158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:21.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.764+0000 7f2b76487700 1 --2- 172.21.15.40:0/1329156672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b70070ae0 0x7f2b7014ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:21.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.764+0000 7f2b76487700 1 --2- 172.21.15.40:0/1329156672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2b7006ea60 0x7f2b70159350 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:21.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.764+0000 7f2b76487700 1 -- 172.21.15.40:0/1329156672 >> 172.21.15.40:0/1329156672 conn(0x7f2b7006a390 msgr2=0x7f2b700c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:21.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.764+0000 7f2b76487700 1 -- 172.21.15.40:0/1329156672 shutdown_connections 2024-09-13T18:01:21.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:21.764+0000 7f2b76487700 1 -- 172.21.15.40:0/1329156672 wait complete. 2024-09-13T18:01:24.784 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:01:24.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7fb00a03b700 1 Processor -- start 2024-09-13T18:01:24.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7fb00a03b700 1 -- start start 2024-09-13T18:01:24.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7fb00a03b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb004153840 0x7fb0040c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:24.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7fb00a03b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb0040c9e20 0x7fb0040cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:24.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7fb00a03b700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0040cc060 con 0x7fb0040cc4f0 2024-09-13T18:01:24.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7fb00a03b700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0040b2840 con 0x7fb0040c9e20 2024-09-13T18:01:24.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7fb00a03b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0040ca870 con 0x7fb004153840 2024-09-13T18:01:24.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7fb00983a700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7fb0040c9e20 msgr2=0x7fb0040cf260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:01:24.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7fb008838700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7fb0040cc4f0 legacy=0x7fb00415d410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:01:24.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7fb00983a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb0040c9e20 0x7fb0040cf260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:24.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7fb009039700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb004153840 0x7fb0040c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:24.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7fb009039700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb004153840 0x7fb0040c3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:38440/0 (socket says 172.21.15.40:38440) 2024-09-13T18:01:24.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7fb009039700 1 -- 172.21.15.40:0/2044588291 learned_addr learned my addr 172.21.15.40:0/2044588291 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:24.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7fb009039700 1 -- 172.21.15.40:0/2044588291 >> v2:172.21.15.43:3300/0 conn(0x7fb0040c9e20 msgr2=0x7fb0040cf260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:01:24.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7fb009039700 1 --2- 172.21.15.40:0/2044588291 >> v2:172.21.15.43:3300/0 conn(0x7fb0040c9e20 0x7fb0040cf260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:24.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7fb009039700 1 -- 172.21.15.40:0/2044588291 >> v1:172.21.15.43:6789/0 conn(0x7fb0040cc4f0 legacy=0x7fb00415d410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:01:24.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7fb009039700 1 -- 172.21.15.40:0/2044588291 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb004153c20 con 0x7fb004153840 2024-09-13T18:01:24.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7fb009039700 1 --2- 172.21.15.40:0/2044588291 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb004153840 0x7fb0040c3eb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb000009910 tx=0x7fb000009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44511e78420b835c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:24.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7faffbfff700 1 -- 172.21.15.40:0/2044588291 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb000004230 con 0x7fb004153840 2024-09-13T18:01:24.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7faffbfff700 1 -- 172.21.15.40:0/2044588291 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb000004390 con 0x7fb004153840 2024-09-13T18:01:24.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.904+0000 7faffbfff700 1 -- 172.21.15.40:0/2044588291 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb000009e60 con 0x7fb004153840 2024-09-13T18:01:24.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb00a03b700 1 -- 172.21.15.40:0/2044588291 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb004153840 msgr2=0x7fb0040c3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:24.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb00a03b700 1 --2- 172.21.15.40:0/2044588291 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb004153840 0x7fb0040c3eb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb000009910 tx=0x7fb000009c20 comp rx=0 tx=0).stop 2024-09-13T18:01:24.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb00a03b700 1 -- 172.21.15.40:0/2044588291 shutdown_connections 2024-09-13T18:01:24.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb00a03b700 1 --2- 172.21.15.40:0/2044588291 >> v2:172.21.15.43:3300/0 conn(0x7fb0040c9e20 0x7fb0040cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:24.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb00a03b700 1 --2- 172.21.15.40:0/2044588291 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb004153840 0x7fb0040c3eb0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:24.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb00a03b700 1 -- 172.21.15.40:0/2044588291 >> 172.21.15.40:0/2044588291 conn(0x7fb00406c800 msgr2=0x7fb00406cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:24.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb00a03b700 1 -- 172.21.15.40:0/2044588291 shutdown_connections 2024-09-13T18:01:24.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb00a03b700 1 -- 172.21.15.40:0/2044588291 wait complete. 2024-09-13T18:01:24.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb00a03b700 1 Processor -- start 2024-09-13T18:01:24.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb00a03b700 1 -- start start 2024-09-13T18:01:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb00a03b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb0040c9e20 0x7fb004153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb00a03b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0040cc4f0 0x7fb0041efcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb00a03b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb004153840 0x7fb004156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb00a03b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0041f0230 con 0x7fb0040cc4f0 2024-09-13T18:01:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb00a03b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0041544e0 con 0x7fb004153840 2024-09-13T18:01:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb00a03b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0041547c0 con 0x7fb0040c9e20 2024-09-13T18:01:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb00983a700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb004153840 msgr2=0x7fb004156370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:01:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb00983a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb004153840 0x7fb004156370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:24.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb008838700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0040cc4f0 0x7fb0041efcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb009039700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb0040c9e20 0x7fb004153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb008838700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0040cc4f0 0x7fb0041efcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40328/0 (socket says 172.21.15.40:40328) 2024-09-13T18:01:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb008838700 1 -- 172.21.15.40:0/4007283335 learned_addr learned my addr 172.21.15.40:0/4007283335 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb008838700 1 -- 172.21.15.40:0/4007283335 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb0040c9e20 msgr2=0x7fb004153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb008838700 1 --2- 172.21.15.40:0/4007283335 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb0040c9e20 0x7fb004153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb008838700 1 -- 172.21.15.40:0/4007283335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb004153840 msgr2=0x7fb004156370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:01:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb008838700 1 --2- 172.21.15.40:0/4007283335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb004153840 0x7fb004156370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.908+0000 7fb008838700 1 -- 172.21.15.40:0/4007283335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb000009570 con 0x7fb0040cc4f0 2024-09-13T18:01:24.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.912+0000 7fb008838700 1 --2- 172.21.15.40:0/4007283335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0040cc4f0 0x7fb0041efcf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faff400b780 tx=0x7faff400ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.912+0000 7faff9ffb700 1 -- 172.21.15.40:0/4007283335 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff4003e60 con 0x7fb0040cc4f0 2024-09-13T18:01:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.912+0000 7fb00a03b700 1 -- 172.21.15.40:0/4007283335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0041e2710 con 0x7fb0040cc4f0 2024-09-13T18:01:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.912+0000 7faff9ffb700 1 -- 172.21.15.40:0/4007283335 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff4005070 con 0x7fb0040cc4f0 2024-09-13T18:01:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.912+0000 7fb00a03b700 1 -- 172.21.15.40:0/4007283335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0041e2cd0 con 0x7fb0040cc4f0 2024-09-13T18:01:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.912+0000 7faff9ffb700 1 -- 172.21.15.40:0/4007283335 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff4005440 con 0x7fb0040cc4f0 2024-09-13T18:01:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.912+0000 7fb00a03b700 1 -- 172.21.15.40:0/4007283335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe4005310 con 0x7fb0040cc4f0 2024-09-13T18:01:24.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.912+0000 7faff9ffb700 1 -- 172.21.15.40:0/4007283335 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faff40055a0 con 0x7fb0040cc4f0 2024-09-13T18:01:24.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.916+0000 7faff9ffb700 1 --2- 172.21.15.40:0/4007283335 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fafe80666e0 0x7fafe8068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:24.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.916+0000 7fb009039700 1 --2- 172.21.15.40:0/4007283335 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fafe80666e0 0x7fafe8068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:24.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.916+0000 7faff9ffb700 1 -- 172.21.15.40:0/4007283335 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7faff4084870 con 0x7fb0040cc4f0 2024-09-13T18:01:24.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.916+0000 7fb009039700 1 --2- 172.21.15.40:0/4007283335 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fafe80666e0 0x7fafe8068ba0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb000009910 tx=0x7fb000008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:24.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:24.920+0000 7faff9ffb700 1 -- 172.21.15.40:0/4007283335 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7faff4051990 con 0x7fb0040cc4f0 2024-09-13T18:01:25.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:25.180+0000 7fb00a03b700 1 -- 172.21.15.40:0/4007283335 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fafe4005890 con 0x7fb0040cc4f0 2024-09-13T18:01:25.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:25.180+0000 7faff9ffb700 1 -- 172.21.15.40:0/4007283335 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7faff40093a0 con 0x7fb0040cc4f0 2024-09-13T18:01:25.190 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":82,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:01:25.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:25.184+0000 7fb00a03b700 1 -- 172.21.15.40:0/4007283335 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fafe80666e0 msgr2=0x7fafe8068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:25.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:25.184+0000 7fb00a03b700 1 --2- 172.21.15.40:0/4007283335 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fafe80666e0 0x7fafe8068ba0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb000009910 tx=0x7fb000008030 comp rx=0 tx=0).stop 2024-09-13T18:01:25.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:25.184+0000 7fb00a03b700 1 -- 172.21.15.40:0/4007283335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0040cc4f0 msgr2=0x7fb0041efcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:25.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:25.184+0000 7fb00a03b700 1 --2- 172.21.15.40:0/4007283335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0040cc4f0 0x7fb0041efcf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faff400b780 tx=0x7faff400ba90 comp rx=0 tx=0).stop 2024-09-13T18:01:25.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:25.184+0000 7fb00a03b700 1 -- 172.21.15.40:0/4007283335 shutdown_connections 2024-09-13T18:01:25.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:25.184+0000 7fb00a03b700 1 --2- 172.21.15.40:0/4007283335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb004153840 0x7fb004156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:25.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:25.184+0000 7fb00a03b700 1 --2- 172.21.15.40:0/4007283335 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0040cc4f0 0x7fb0041efcf0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:25.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:25.184+0000 7fb00a03b700 1 --2- 172.21.15.40:0/4007283335 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fafe80666e0 0x7fafe8068ba0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:25.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:25.184+0000 7fb00a03b700 1 --2- 172.21.15.40:0/4007283335 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb0040c9e20 0x7fb004153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:25.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:25.184+0000 7fb00a03b700 1 -- 172.21.15.40:0/4007283335 >> 172.21.15.40:0/4007283335 conn(0x7fb00406c800 msgr2=0x7fb00414dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:25.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:25.184+0000 7fb00a03b700 1 -- 172.21.15.40:0/4007283335 shutdown_connections 2024-09-13T18:01:25.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:25.184+0000 7fb00a03b700 1 -- 172.21.15.40:0/4007283335 wait complete. 2024-09-13T18:01:28.208 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:01:28.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.315+0000 7fba0bfff700 1 Processor -- start 2024-09-13T18:01:28.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.315+0000 7fba0bfff700 1 -- start start 2024-09-13T18:01:28.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.315+0000 7fba0bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 0x7fba0c0c0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:28.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.315+0000 7fba0bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 0x7fba0c1565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:28.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.315+0000 7fba0bfff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba0c0b2210 con 0x7fba0c072240 2024-09-13T18:01:28.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.315+0000 7fba0bfff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba0c0c1780 con 0x7fba0c071e60 2024-09-13T18:01:28.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.315+0000 7fba0bfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c0c1a60 con 0x7fba0c0c28f0 2024-09-13T18:01:28.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.319+0000 7fba0affd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 0x7fba0c0c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:28.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.319+0000 7fba0affd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 0x7fba0c0c0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40338/0 (socket says 172.21.15.40:40338) 2024-09-13T18:01:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.319+0000 7fba0affd700 1 -- 172.21.15.40:0/2336806131 learned_addr learned my addr 172.21.15.40:0/2336806131 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.319+0000 7fba0b7fe700 1 --2- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 0x7fba0c1565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.319+0000 7fba0affd700 1 -- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 msgr2=0x7fba0c0c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:01:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.319+0000 7fba0affd700 1 -- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 msgr2=0x7fba0c0c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:01:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.319+0000 7fba0affd700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 0x7fba0c0c0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:01:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.319+0000 7fba0affd700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 0x7fba0c0c0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.319+0000 7fba0b7fe700 1 -- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 msgr2=0x7fba0c1565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:01:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.319+0000 7fba0b7fe700 1 -- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 msgr2=0x7fba0c1565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:01:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.319+0000 7fba0b7fe700 1 --2- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 0x7fba0c1565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:01:28.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.319+0000 7fba0b7fe700 1 --2- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 0x7fba0c1565c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:28.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.523+0000 7fba0affd700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 0x7fba0c0c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:28.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.523+0000 7fba0b7fe700 1 --2- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 0x7fba0c1565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:28.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.523+0000 7fba0affd700 1 -- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 msgr2=0x7fba0c0c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:01:28.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.523+0000 7fba0affd700 1 -- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 msgr2=0x7fba0c0c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:01:28.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.523+0000 7fba0affd700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 0x7fba0c0c0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:01:28.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.523+0000 7fba0affd700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 0x7fba0c0c0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:01:28.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.523+0000 7fba0b7fe700 1 -- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 msgr2=0x7fba0c1565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:01:28.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.523+0000 7fba0b7fe700 1 -- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 msgr2=0x7fba0c1565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:01:28.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.523+0000 7fba0b7fe700 1 --2- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 0x7fba0c1565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:01:28.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.523+0000 7fba0b7fe700 1 --2- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 0x7fba0c1565c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:01:28.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.927+0000 7fba0affd700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 0x7fba0c0c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:28.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.927+0000 7fba0b7fe700 1 --2- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 0x7fba0c1565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:28.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.927+0000 7fba0affd700 1 -- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 msgr2=0x7fba0c0c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:01:28.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.927+0000 7fba0affd700 1 -- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 msgr2=0x7fba0c0c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:01:28.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.927+0000 7fba0affd700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 0x7fba0c0c0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:01:28.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.927+0000 7fba0affd700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 0x7fba0c0c0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:01:28.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.927+0000 7fba0b7fe700 1 -- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 msgr2=0x7fba0c1565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:01:28.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.927+0000 7fba0b7fe700 1 -- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 msgr2=0x7fba0c1565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:01:28.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.927+0000 7fba0b7fe700 1 --2- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 0x7fba0c1565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:01:28.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:28.927+0000 7fba0b7fe700 1 --2- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 0x7fba0c1565c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:01:29.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:29.731+0000 7fba0affd700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 0x7fba0c0c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:29.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:29.731+0000 7fba0b7fe700 1 --2- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 0x7fba0c1565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:29.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:29.731+0000 7fba0affd700 1 -- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 msgr2=0x7fba0c0c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:01:29.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:29.731+0000 7fba0affd700 1 -- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 msgr2=0x7fba0c0c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:01:29.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:29.731+0000 7fba0affd700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 0x7fba0c0c0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:01:29.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:29.731+0000 7fba0b7fe700 1 -- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 msgr2=0x7fba0c1565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:01:29.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:29.731+0000 7fba0b7fe700 1 -- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 msgr2=0x7fba0c1565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:01:29.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:29.731+0000 7fba0b7fe700 1 --2- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 0x7fba0c1565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:01:29.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:29.731+0000 7fba0b7fe700 1 --2- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 0x7fba0c1565c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:01:29.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:29.731+0000 7fba0affd700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 0x7fba0c0c0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:01:31.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:31.319+0000 7fba08ff9700 1 -- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 msgr2=0x7fba0c1565c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:01:31.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:31.319+0000 7fba08ff9700 1 --2- 172.21.15.40:0/2336806131 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c0c28f0 0x7fba0c1565c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:31.319+0000 7fba08ff9700 1 -- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 msgr2=0x7fba0c0c0e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:01:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:31.319+0000 7fba08ff9700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 0x7fba0c0c0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:31.319+0000 7fba08ff9700 1 -- 172.21.15.40:0/2336806131 >> v1:172.21.15.40:6789/0 conn(0x7fba0c072240 legacy=0x7fba0c0c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:31.319+0000 7fba08ff9700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.43:3300/0 conn(0x7fb9ec007460 0x7fb9ec009920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:31.319+0000 7fba08ff9700 1 -- 172.21.15.40:0/2336806131 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba0c0c5d70 con 0x7fb9ec003740 2024-09-13T18:01:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:31.319+0000 7fba08ff9700 1 -- 172.21.15.40:0/2336806131 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba0c0c1780 con 0x7fb9ec007460 2024-09-13T18:01:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:31.319+0000 7fba0affd700 1 -- 172.21.15.40:0/2336806131 >> v1:172.21.15.43:6789/0 conn(0x7fb9ec003740 legacy=0x7fb9ec005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:01:31.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:31.319+0000 7fba0a7fc700 1 -- 172.21.15.40:0/2336806131 >> v2:172.21.15.43:3300/0 conn(0x7fb9ec007460 msgr2=0x7fb9ec009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:01:31.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:31.319+0000 7fba0a7fc700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.43:3300/0 conn(0x7fb9ec007460 0x7fb9ec009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:31.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:31.523+0000 7fba0affd700 1 -- 172.21.15.40:0/2336806131 >> v1:172.21.15.43:6789/0 conn(0x7fb9ec003740 legacy=0x7fb9ec005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:01:31.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:31.523+0000 7fba0a7fc700 1 -- 172.21.15.40:0/2336806131 >> v2:172.21.15.43:3300/0 conn(0x7fb9ec007460 msgr2=0x7fb9ec009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:01:31.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:31.523+0000 7fba0a7fc700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.43:3300/0 conn(0x7fb9ec007460 0x7fb9ec009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:01:31.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:31.927+0000 7fba0a7fc700 1 -- 172.21.15.40:0/2336806131 >> v2:172.21.15.43:3300/0 conn(0x7fb9ec007460 msgr2=0x7fb9ec009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:01:31.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:31.927+0000 7fba0affd700 1 -- 172.21.15.40:0/2336806131 >> v1:172.21.15.43:6789/0 conn(0x7fb9ec003740 legacy=0x7fb9ec005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:01:31.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:31.927+0000 7fba0a7fc700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.43:3300/0 conn(0x7fb9ec007460 0x7fb9ec009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:01:32.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:32.731+0000 7fba0affd700 1 -- 172.21.15.40:0/2336806131 >> v1:172.21.15.43:6789/0 conn(0x7fb9ec003740 legacy=0x7fb9ec005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:01:32.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:32.731+0000 7fba0a7fc700 1 -- 172.21.15.40:0/2336806131 >> v2:172.21.15.43:3300/0 conn(0x7fb9ec007460 msgr2=0x7fb9ec009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:01:32.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:32.731+0000 7fba0a7fc700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.43:3300/0 conn(0x7fb9ec007460 0x7fb9ec009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:01:34.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba08ff9700 1 -- 172.21.15.40:0/2336806131 >> v2:172.21.15.43:3300/0 conn(0x7fb9ec007460 msgr2=0x7fb9ec009920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:01:34.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba08ff9700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.43:3300/0 conn(0x7fb9ec007460 0x7fb9ec009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba08ff9700 1 -- 172.21.15.40:0/2336806131 >> v1:172.21.15.43:6789/0 conn(0x7fb9ec003740 legacy=0x7fb9ec005ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:01:34.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba08ff9700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.43:3300/0 conn(0x7fb9ec007460 0x7fb9ec009920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba08ff9700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 0x7fb9ec00a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba08ff9700 1 -- 172.21.15.40:0/2336806131 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ec00b020 con 0x7fba0c072240 2024-09-13T18:01:34.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba08ff9700 1 -- 172.21.15.40:0/2336806131 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba0c0c1780 con 0x7fba0c071e60 2024-09-13T18:01:34.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba08ff9700 1 -- 172.21.15.40:0/2336806131 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba0c0c1a60 con 0x7fb9ec007460 2024-09-13T18:01:34.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba0affd700 1 -- 172.21.15.40:0/2336806131 >> v2:172.21.15.43:3300/0 conn(0x7fb9ec007460 msgr2=0x7fb9ec009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:01:34.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba0b7fe700 1 -- 172.21.15.40:0/2336806131 reap_dead start 2024-09-13T18:01:34.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba0affd700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.43:3300/0 conn(0x7fb9ec007460 0x7fb9ec009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:34.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba0b7fe700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.40:3300/0 conn(0x7fba0c071e60 0x7fb9ec00a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:34.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba0b7fe700 1 -- 172.21.15.40:0/2336806131 >> v2:172.21.15.43:3300/0 conn(0x7fb9ec007460 msgr2=0x7fb9ec009920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:01:34.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba0b7fe700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.43:3300/0 conn(0x7fb9ec007460 0x7fb9ec009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba0b7fe700 1 -- 172.21.15.40:0/2336806131 >> v1:172.21.15.40:6789/0 conn(0x7fba0c072240 legacy=0x7fb9ec009fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:34.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba0b7fe700 1 -- 172.21.15.40:0/2336806131 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0c0c5d70 con 0x7fba0c071e60 2024-09-13T18:01:34.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba09ffb700 1 -- 172.21.15.40:0/2336806131 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2754029241 0 0) 0x7fb9ec00b020 con 0x7fba0c072240 2024-09-13T18:01:34.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba0b7fe700 1 --2- 172.21.15.40:0/2336806131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba0c071e60 0x7fb9ec00a800 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec003b20 tx=0x7fba04009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d222a2c47760a6f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:34.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba09ffb700 1 -- 172.21.15.40:0/2336806131 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04004750 con 0x7fba0c071e60 2024-09-13T18:01:34.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba09ffb700 1 -- 172.21.15.40:0/2336806131 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0400f040 con 0x7fba0c071e60 2024-09-13T18:01:34.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.319+0000 7fba09ffb700 1 -- 172.21.15.40:0/2336806131 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04016460 con 0x7fba0c071e60 2024-09-13T18:01:34.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.323+0000 7fba0bfff700 1 -- 172.21.15.40:0/2336806131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba0c071e60 msgr2=0x7fb9ec00a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:34.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.323+0000 7fba0bfff700 1 --2- 172.21.15.40:0/2336806131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba0c071e60 0x7fb9ec00a800 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec003b20 tx=0x7fba04009bc0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.327+0000 7fba0bfff700 1 -- 172.21.15.40:0/2336806131 shutdown_connections 2024-09-13T18:01:34.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.327+0000 7fba0bfff700 1 --2- 172.21.15.40:0/2336806131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba0c071e60 0x7fb9ec00a800 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.327+0000 7fba0bfff700 1 --2- 172.21.15.40:0/2336806131 >> v2:172.21.15.43:3300/0 conn(0x7fb9ec007460 0x7fb9ec009920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.327+0000 7fba0bfff700 1 -- 172.21.15.40:0/2336806131 >> 172.21.15.40:0/2336806131 conn(0x7fba0c06e4f0 msgr2=0x7fba0c06e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.327+0000 7fba0bfff700 1 -- 172.21.15.40:0/2336806131 shutdown_connections 2024-09-13T18:01:34.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.327+0000 7fba0bfff700 1 -- 172.21.15.40:0/2336806131 wait complete. 2024-09-13T18:01:34.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.331+0000 7fba0bfff700 1 Processor -- start 2024-09-13T18:01:34.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.331+0000 7fba0bfff700 1 -- start start 2024-09-13T18:01:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.331+0000 7fba0bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9ec007460 0x7fba0c151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.331+0000 7fba0bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c071e60 0x7fba0c0aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.331+0000 7fba0bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba0c072240 0x7fba0c0b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.331+0000 7fba0bfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c1ef530 con 0x7fba0c072240 2024-09-13T18:01:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.331+0000 7fba0bfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c1520c0 con 0x7fb9ec007460 2024-09-13T18:01:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.331+0000 7fba0bfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c0b04d0 con 0x7fba0c071e60 2024-09-13T18:01:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.331+0000 7fba0affd700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9ec007460 msgr2=0x7fba0c151b80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:01:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.331+0000 7fba0affd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9ec007460 0x7fba0c151b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.331+0000 7fba0b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba0c072240 0x7fba0c0b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.331+0000 7fba0b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba0c072240 0x7fba0c0b08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42398/0 (socket says 172.21.15.40:42398) 2024-09-13T18:01:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.331+0000 7fba0b7fe700 1 -- 172.21.15.40:0/2121610959 learned_addr learned my addr 172.21.15.40:0/2121610959 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.331+0000 7fba0a7fc700 1 --2- 172.21.15.40:0/2121610959 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c071e60 0x7fba0c0aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.331+0000 7fba0b7fe700 1 -- 172.21.15.40:0/2121610959 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c071e60 msgr2=0x7fba0c0aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.331+0000 7fba0b7fe700 1 --2- 172.21.15.40:0/2121610959 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c071e60 0x7fba0c0aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.331+0000 7fba0b7fe700 1 -- 172.21.15.40:0/2121610959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9ec007460 msgr2=0x7fba0c151b80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:01:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.331+0000 7fba0b7fe700 1 --2- 172.21.15.40:0/2121610959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9ec007460 0x7fba0c151b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.331+0000 7fba0b7fe700 1 -- 172.21.15.40:0/2121610959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba04009640 con 0x7fba0c072240 2024-09-13T18:01:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.331+0000 7fba0b7fe700 1 --2- 172.21.15.40:0/2121610959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba0c072240 0x7fba0c0b08f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0c2cd0 tx=0x7fba04004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.335+0000 7fba037fe700 1 -- 172.21.15.40:0/2121610959 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04016460 con 0x7fba0c072240 2024-09-13T18:01:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.335+0000 7fba0bfff700 1 -- 172.21.15.40:0/2121610959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba0c0adbd0 con 0x7fba0c072240 2024-09-13T18:01:34.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.335+0000 7fba0bfff700 1 -- 172.21.15.40:0/2121610959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba0c0ae490 con 0x7fba0c072240 2024-09-13T18:01:34.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.335+0000 7fba037fe700 1 -- 172.21.15.40:0/2121610959 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0402b660 con 0x7fba0c072240 2024-09-13T18:01:34.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.335+0000 7fba037fe700 1 -- 172.21.15.40:0/2121610959 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0402ba50 con 0x7fba0c072240 2024-09-13T18:01:34.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.339+0000 7fba0bfff700 1 -- 172.21.15.40:0/2121610959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f0005310 con 0x7fba0c072240 2024-09-13T18:01:34.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.339+0000 7fba037fe700 1 -- 172.21.15.40:0/2121610959 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fba04021440 con 0x7fba0c072240 2024-09-13T18:01:34.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.339+0000 7fba037fe700 1 --2- 172.21.15.40:0/2121610959 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb9ec067230 0x7fb9ec0696f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.339+0000 7fba037fe700 1 -- 172.21.15.40:0/2121610959 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fba040870c0 con 0x7fba0c072240 2024-09-13T18:01:34.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.339+0000 7fba0affd700 1 --2- 172.21.15.40:0/2121610959 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb9ec067230 0x7fb9ec0696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:34.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.339+0000 7fba0affd700 1 --2- 172.21.15.40:0/2121610959 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb9ec067230 0x7fb9ec0696f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0091a0 tx=0x7fb9fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:34.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.347+0000 7fba037fe700 1 -- 172.21.15.40:0/2121610959 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fba04054100 con 0x7fba0c072240 2024-09-13T18:01:34.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.599+0000 7fba0bfff700 1 -- 172.21.15.40:0/2121610959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb9f00055c0 con 0x7fba0c072240 2024-09-13T18:01:34.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.599+0000 7fba037fe700 1 -- 172.21.15.40:0/2121610959 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7fba04057720 con 0x7fba0c072240 2024-09-13T18:01:34.610 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":86,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:01:34.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.603+0000 7fba0bfff700 1 -- 172.21.15.40:0/2121610959 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb9ec067230 msgr2=0x7fb9ec0696f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:34.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.603+0000 7fba0bfff700 1 --2- 172.21.15.40:0/2121610959 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb9ec067230 0x7fb9ec0696f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0091a0 tx=0x7fb9fc008040 comp rx=0 tx=0).stop 2024-09-13T18:01:34.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.603+0000 7fba0bfff700 1 -- 172.21.15.40:0/2121610959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba0c072240 msgr2=0x7fba0c0b08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:34.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.603+0000 7fba0bfff700 1 --2- 172.21.15.40:0/2121610959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba0c072240 0x7fba0c0b08f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0c2cd0 tx=0x7fba04004af0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.603+0000 7fba0bfff700 1 -- 172.21.15.40:0/2121610959 shutdown_connections 2024-09-13T18:01:34.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.603+0000 7fba0bfff700 1 --2- 172.21.15.40:0/2121610959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba0c072240 0x7fba0c0b08f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.603+0000 7fba0bfff700 1 --2- 172.21.15.40:0/2121610959 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb9ec067230 0x7fb9ec0696f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.603+0000 7fba0bfff700 1 --2- 172.21.15.40:0/2121610959 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba0c071e60 0x7fba0c0aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.603+0000 7fba0bfff700 1 --2- 172.21.15.40:0/2121610959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9ec007460 0x7fba0c151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.603+0000 7fba0bfff700 1 -- 172.21.15.40:0/2121610959 >> 172.21.15.40:0/2121610959 conn(0x7fba0c06e4f0 msgr2=0x7fba0c06e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:34.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.603+0000 7fba0bfff700 1 -- 172.21.15.40:0/2121610959 shutdown_connections 2024-09-13T18:01:34.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.603+0000 7fba0bfff700 1 -- 172.21.15.40:0/2121610959 wait complete. 2024-09-13T18:01:34.628 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T18:01:34.629 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-13T18:01:34.629 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:01:34.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.739+0000 7f5f78a20700 1 Processor -- start 2024-09-13T18:01:34.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.739+0000 7f5f78a20700 1 -- start start 2024-09-13T18:01:34.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.739+0000 7f5f78a20700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5f6c0d57d0 0x7f5f6c0df360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.739+0000 7f5f78a20700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f6c007bf0 0x7f5f6c046da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.739+0000 7f5f78a20700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f6c046910 con 0x7f5f6c0459c0 2024-09-13T18:01:34.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.739+0000 7f5f78a20700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f6c0df8a0 con 0x7f5f6c0d57d0 2024-09-13T18:01:34.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.739+0000 7f5f78a20700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c0d5bb0 con 0x7f5f6c007bf0 2024-09-13T18:01:34.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.739+0000 7f5f737fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5f6c0d57d0 0x7f5f6c0df360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:34.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.739+0000 7f5f73fff700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7f5f6c0459c0 legacy=0x7f5f6c0db3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:01:34.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.739+0000 7f5f737fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5f6c0d57d0 0x7f5f6c0df360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42400/0 (socket says 172.21.15.40:42400) 2024-09-13T18:01:34.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.739+0000 7f5f737fe700 1 -- 172.21.15.40:0/999176599 learned_addr learned my addr 172.21.15.40:0/999176599 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:34.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.739+0000 7f5f72ffd700 1 --2- 172.21.15.40:0/999176599 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f6c007bf0 0x7f5f6c046da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:34.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.739+0000 7f5f737fe700 1 -- 172.21.15.40:0/999176599 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f6c007bf0 msgr2=0x7f5f6c046da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:01:34.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.739+0000 7f5f737fe700 1 --2- 172.21.15.40:0/999176599 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f6c007bf0 0x7f5f6c046da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.739+0000 7f5f737fe700 1 -- 172.21.15.40:0/999176599 >> v1:172.21.15.43:6789/0 conn(0x7f5f6c0459c0 legacy=0x7f5f6c0db3b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:01:34.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.739+0000 7f5f737fe700 1 -- 172.21.15.40:0/999176599 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c005e10 con 0x7f5f6c0d57d0 2024-09-13T18:01:34.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.743+0000 7f5f737fe700 1 --2- 172.21.15.40:0/999176599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0d57d0 0x7f5f6c0df360 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800bbb0 tx=0x7f5f6800bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca32558e3457c4f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:34.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.743+0000 7f5f727fc700 1 -- 172.21.15.40:0/999176599 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6800dcf0 con 0x7f5f6c0d57d0 2024-09-13T18:01:34.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.743+0000 7f5f727fc700 1 -- 172.21.15.40:0/999176599 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6800de50 con 0x7f5f6c0d57d0 2024-09-13T18:01:34.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.743+0000 7f5f727fc700 1 -- 172.21.15.40:0/999176599 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f68004250 con 0x7f5f6c0d57d0 2024-09-13T18:01:34.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.743+0000 7f5f78a20700 1 -- 172.21.15.40:0/999176599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0d57d0 msgr2=0x7f5f6c0df360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:34.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.743+0000 7f5f78a20700 1 --2- 172.21.15.40:0/999176599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0d57d0 0x7f5f6c0df360 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800bbb0 tx=0x7f5f6800bec0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.743+0000 7f5f78a20700 1 -- 172.21.15.40:0/999176599 shutdown_connections 2024-09-13T18:01:34.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.743+0000 7f5f78a20700 1 --2- 172.21.15.40:0/999176599 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f6c007bf0 0x7f5f6c046da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.743+0000 7f5f78a20700 1 --2- 172.21.15.40:0/999176599 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0d57d0 0x7f5f6c0df360 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.743+0000 7f5f78a20700 1 -- 172.21.15.40:0/999176599 >> 172.21.15.40:0/999176599 conn(0x7f5f6c050590 msgr2=0x7f5f6c0509a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:34.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.743+0000 7f5f78a20700 1 -- 172.21.15.40:0/999176599 shutdown_connections 2024-09-13T18:01:34.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.743+0000 7f5f78a20700 1 -- 172.21.15.40:0/999176599 wait complete. 2024-09-13T18:01:34.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.743+0000 7f5f78a20700 1 Processor -- start 2024-09-13T18:01:34.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.743+0000 7f5f78a20700 1 -- start start 2024-09-13T18:01:34.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f78a20700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f6c007bf0 0x7f5f6c0d7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f78a20700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0459c0 0x7f5f6c0dd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f78a20700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f6c0d57d0 0x7f5f6c0d5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f78a20700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c0d7130 con 0x7f5f6c0459c0 2024-09-13T18:01:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f78a20700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c0d81e0 con 0x7f5f6c007bf0 2024-09-13T18:01:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f78a20700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c177ae0 con 0x7f5f6c0d57d0 2024-09-13T18:01:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f737fe700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f6c007bf0 msgr2=0x7f5f6c0d7ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:01:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f737fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f6c007bf0 0x7f5f6c0d7ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f72ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0459c0 0x7f5f6c0dd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f72ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0459c0 0x7f5f6c0dd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42412/0 (socket says 172.21.15.40:42412) 2024-09-13T18:01:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f72ffd700 1 -- 172.21.15.40:0/417177329 learned_addr learned my addr 172.21.15.40:0/417177329 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f73fff700 1 --2- 172.21.15.40:0/417177329 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f6c0d57d0 0x7f5f6c0d5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f72ffd700 1 -- 172.21.15.40:0/417177329 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f6c0d57d0 msgr2=0x7f5f6c0d5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f72ffd700 1 --2- 172.21.15.40:0/417177329 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f6c0d57d0 0x7f5f6c0d5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f72ffd700 1 -- 172.21.15.40:0/417177329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f6c007bf0 msgr2=0x7f5f6c0d7ca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:01:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f72ffd700 1 --2- 172.21.15.40:0/417177329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f6c007bf0 0x7f5f6c0d7ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f72ffd700 1 -- 172.21.15.40:0/417177329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6800b810 con 0x7f5f6c0459c0 2024-09-13T18:01:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f72ffd700 1 --2- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0459c0 0x7f5f6c0dd810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000e9b0 tx=0x7f5f6000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:34.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f5ffff700 1 -- 172.21.15.40:0/417177329 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60009700 con 0x7f5f6c0459c0 2024-09-13T18:01:34.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f5ffff700 1 -- 172.21.15.40:0/417177329 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f60004020 con 0x7f5f6c0459c0 2024-09-13T18:01:34.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f78a20700 1 -- 172.21.15.40:0/417177329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6c032ca0 con 0x7f5f6c0459c0 2024-09-13T18:01:34.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f5ffff700 1 -- 172.21.15.40:0/417177329 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f600043b0 con 0x7f5f6c0459c0 2024-09-13T18:01:34.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f78a20700 1 -- 172.21.15.40:0/417177329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6c178010 con 0x7f5f6c0459c0 2024-09-13T18:01:34.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f78a20700 1 --2- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0077b0 0x7f5f6c0d6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f78a20700 1 -- 172.21.15.40:0/417177329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c008d40 con 0x7f5f6c0077b0 2024-09-13T18:01:34.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.747+0000 7f5f737fe700 1 --2- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0077b0 0x7f5f6c0d6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:34.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.751+0000 7f5f737fe700 1 -- 172.21.15.40:0/417177329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5f6c033f30 con 0x7f5f6c0077b0 2024-09-13T18:01:34.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.751+0000 7f5f737fe700 1 --2- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0077b0 0x7f5f6c0d6260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800bbb0 tx=0x7f5f68004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:34.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.751+0000 7f5f5ffff700 1 -- 172.21.15.40:0/417177329 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5f6000c850 con 0x7f5f6c0459c0 2024-09-13T18:01:34.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.751+0000 7f5f5ffff700 1 --2- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5f580666e0 0x7f5f58068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.751+0000 7f5f5ffff700 1 -- 172.21.15.40:0/417177329 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5f60084580 con 0x7f5f6c0459c0 2024-09-13T18:01:34.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.751+0000 7f5f73fff700 1 --2- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5f580666e0 0x7f5f58068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:34.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.751+0000 7f5f5ffff700 1 -- 172.21.15.40:0/417177329 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6801e070 con 0x7f5f6c0077b0 2024-09-13T18:01:34.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.751+0000 7f5f5ffff700 1 -- 172.21.15.40:0/417177329 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5f68003a30 con 0x7f5f6c0077b0 2024-09-13T18:01:34.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.751+0000 7f5f5ffff700 1 -- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0077b0 msgr2=0x7f5f6c0d6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:34.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.751+0000 7f5f5ffff700 1 --2- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0077b0 0x7f5f6c0d6260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800bbb0 tx=0x7f5f68004770 comp rx=0 tx=0).stop 2024-09-13T18:01:34.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.751+0000 7f5f5ffff700 1 -- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0077b0 msgr2=0x7f5f6c0d6260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:01:34.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.751+0000 7f5f5ffff700 1 --2- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0077b0 0x7f5f6c0d6260 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5f6800bbb0 tx=0x7f5f68004770 comp rx=0 tx=0).stop 2024-09-13T18:01:34.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.751+0000 7f5f73fff700 1 --2- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5f580666e0 0x7f5f58068ba0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5f7404f480 tx=0x7f5f74065ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:34.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.763+0000 7f5f78a20700 1 --2- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0deaa0 0x7f5f6c178350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.763+0000 7f5f78a20700 1 -- 172.21.15.40:0/417177329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c0df500 con 0x7f5f6c0deaa0 2024-09-13T18:01:34.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.763+0000 7f5f737fe700 1 --2- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0deaa0 0x7f5f6c178350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:34.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.763+0000 7f5f737fe700 1 -- 172.21.15.40:0/417177329 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5f6c033940 con 0x7f5f6c0deaa0 2024-09-13T18:01:34.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.763+0000 7f5f737fe700 1 --2- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0deaa0 0x7f5f6c178350 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5f68000c00 tx=0x7f5f6800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:34.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.763+0000 7f5f5ffff700 1 -- 172.21.15.40:0/417177329 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f68011d70 con 0x7f5f6c0deaa0 2024-09-13T18:01:34.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.763+0000 7f5f5ffff700 1 -- 172.21.15.40:0/417177329 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4938 (secure 0 0 0) 0x7f5f68027040 con 0x7f5f6c0deaa0 2024-09-13T18:01:34.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.763+0000 7f5f5ffff700 1 -- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0deaa0 msgr2=0x7f5f6c178350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:34.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.763+0000 7f5f5ffff700 1 --2- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0deaa0 0x7f5f6c178350 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5f68000c00 tx=0x7f5f6800b4f0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.763+0000 7f5f5ffff700 1 -- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0deaa0 msgr2=0x7f5f6c178350 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:01:34.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.763+0000 7f5f5ffff700 1 --2- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0deaa0 0x7f5f6c178350 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5f68000c00 tx=0x7f5f6800b4f0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.775 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:01:34.775 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:01:34.775 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:01:34.775 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:01:34.775 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 86, 2024-09-13T18:01:34.775 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:01:34.775 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:01:34.775 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:01:34.776 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:34.776 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:01:34.776 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:01:34.776 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:01:34.776 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:01:34.776 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:34.776 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:34.776 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:34.776 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:34.776 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:34.777 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:34.777 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:34.777 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:34.777 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:34.777 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:34.777 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:34.777 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:01:34.777 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:01:34.777 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:34.777 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:34.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:34.777 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:34.777 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:34.778 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:34.778 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:34.778 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:34.778 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:34.778 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:34.778 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:34.778 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.778 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:01:34.778 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:01:34.778 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:01:34.778 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:01:34.779 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:01:34.779 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:01:34.779 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:01:34.779 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:01:34.779 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:01:34.779 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:01:34.779 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:01:34.779 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:01:34.779 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:01:34.779 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:01:34.779 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:01:34.779 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:01:34.779 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:01:34.779 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:34.780 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:34.780 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:34.780 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:34.780 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:34.780 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:34.780 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:34.780 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:34.780 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:34.780 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:34.780 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:34.780 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:01:34.780 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.780 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:01:34.780 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.781 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:01:34.781 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:01:34.781 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:34.781 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:34.781 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.781 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:34.781 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:01:34.781 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:34.781 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.781 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.781 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:34.781 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:01:34.781 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:34.782 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:34.782 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:34.782 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.782 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:01:34.782 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:01:34.782 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:34.782 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:34.782 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:34.782 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.782 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.782 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:01:34.782 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:01:34.782 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:34.783 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:34.783 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.783 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:34.783 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:01:34.783 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:34.783 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.783 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.783 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:34.783 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:01:34.783 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:34.783 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:34.783 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:34.784 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.784 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:01:34.784 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:01:34.784 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:34.784 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:34.784 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:34.784 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.784 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.784 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:01:34.784 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:01:34.784 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:34.784 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:34.784 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.784 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:34.785 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:01:34.785 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:34.785 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.785 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.785 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:34.785 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:01:34.785 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:34.785 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:34.785 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:34.785 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.785 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:01:34.785 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:01:34.785 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:34.786 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:34.786 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:34.786 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:34.786 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:34.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.786 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:01:34.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:01:34.786 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.786 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:34.786 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:34.786 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:01:34.786 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:34.786 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:34.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:01:34.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.787 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:34.787 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:34.787 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:01:34.787 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:34.787 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:34.787 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:01:34.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.787 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:34.787 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:34.788 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 8 2024-09-13T18:01:34.788 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:34.788 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:34.788 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:01:34.788 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.788 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:34.788 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:34.788 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:01:34.788 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:34.788 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:34.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:01:34.788 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.788 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:34.788 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:34.789 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:01:34.789 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:34.789 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:34.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.789 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:01:34.789 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:01:34.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.763+0000 7f5f78a20700 1 -- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5f580666e0 msgr2=0x7f5f58068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:34.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.763+0000 7f5f78a20700 1 --2- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5f580666e0 0x7f5f58068ba0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5f7404f480 tx=0x7f5f74065ea0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.763+0000 7f5f78a20700 1 -- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0459c0 msgr2=0x7f5f6c0dd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:34.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.763+0000 7f5f73fff700 1 -- 172.21.15.40:0/417177329 reap_dead start 2024-09-13T18:01:34.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.763+0000 7f5f78a20700 1 --2- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0459c0 0x7f5f6c0dd810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000e9b0 tx=0x7f5f6000ecc0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.763+0000 7f5f78a20700 1 -- 172.21.15.40:0/417177329 shutdown_connections 2024-09-13T18:01:34.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.763+0000 7f5f78a20700 1 --2- 172.21.15.40:0/417177329 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f6c0459c0 0x7f5f6c0dd810 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.763+0000 7f5f78a20700 1 -- 172.21.15.40:0/417177329 >> 172.21.15.40:0/417177329 conn(0x7f5f6c050590 msgr2=0x7f5f6c04d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:34.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.767+0000 7f5f78a20700 1 -- 172.21.15.40:0/417177329 shutdown_connections 2024-09-13T18:01:34.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.767+0000 7f5f78a20700 1 -- 172.21.15.40:0/417177329 wait complete. 2024-09-13T18:01:34.791 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:01:34.791 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T18:01:34.791 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:01:34.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.899+0000 7fc0a4f44700 1 Processor -- start 2024-09-13T18:01:34.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.899+0000 7fc0a4f44700 1 -- start start 2024-09-13T18:01:34.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc0a4f44700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a0150240 0x7fc0a0066090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc0a4f44700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0a00c4660 0x7fc0a00c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc0a4f44700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a00c41d0 con 0x7fc0a01595f0 2024-09-13T18:01:34.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc0a4f44700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0a00665d0 con 0x7fc0a00c4660 2024-09-13T18:01:34.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc0a4f44700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a00c3ed0 con 0x7fc0a0150240 2024-09-13T18:01:34.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc09ffff700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7fc0a01595f0 legacy=0x7fc0a0159aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:01:34.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc097fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0a00c4660 0x7fc0a00c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:34.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc09f7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a0150240 0x7fc0a0066090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:34.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc097fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc0a00c4660 0x7fc0a00c4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42444/0 (socket says 172.21.15.40:42444) 2024-09-13T18:01:34.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc097fff700 1 -- 172.21.15.40:0/1077020985 learned_addr learned my addr 172.21.15.40:0/1077020985 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:34.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc097fff700 1 -- 172.21.15.40:0/1077020985 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a0150240 msgr2=0x7fc0a0066090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:01:34.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc097fff700 1 --2- 172.21.15.40:0/1077020985 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a0150240 0x7fc0a0066090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc097fff700 1 -- 172.21.15.40:0/1077020985 >> v1:172.21.15.43:6789/0 conn(0x7fc0a01595f0 legacy=0x7fc0a0159aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:01:34.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc097fff700 1 -- 172.21.15.40:0/1077020985 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a00c4fe0 con 0x7fc0a00c4660 2024-09-13T18:01:34.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc097fff700 1 --2- 172.21.15.40:0/1077020985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a00c4660 0x7fc0a00c4aa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc088009910 tx=0x7fc088009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55ac806c09ce2c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:34.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc09effd700 1 -- 172.21.15.40:0/1077020985 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc088004230 con 0x7fc0a00c4660 2024-09-13T18:01:34.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc09effd700 1 -- 172.21.15.40:0/1077020985 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc088004390 con 0x7fc0a00c4660 2024-09-13T18:01:34.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc09effd700 1 -- 172.21.15.40:0/1077020985 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0880036c0 con 0x7fc0a00c4660 2024-09-13T18:01:34.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc0a4f44700 1 -- 172.21.15.40:0/1077020985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a00c4660 msgr2=0x7fc0a00c4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:34.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc0a4f44700 1 --2- 172.21.15.40:0/1077020985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a00c4660 0x7fc0a00c4aa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc088009910 tx=0x7fc088009c20 comp rx=0 tx=0).stop 2024-09-13T18:01:34.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc0a4f44700 1 -- 172.21.15.40:0/1077020985 shutdown_connections 2024-09-13T18:01:34.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc0a4f44700 1 --2- 172.21.15.40:0/1077020985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a00c4660 0x7fc0a00c4aa0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc0a4f44700 1 --2- 172.21.15.40:0/1077020985 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a0150240 0x7fc0a0066090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.903+0000 7fc0a4f44700 1 -- 172.21.15.40:0/1077020985 >> 172.21.15.40:0/1077020985 conn(0x7fc0a006a390 msgr2=0x7fc0a006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:34.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.907+0000 7fc0a4f44700 1 -- 172.21.15.40:0/1077020985 shutdown_connections 2024-09-13T18:01:34.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.907+0000 7fc0a4f44700 1 -- 172.21.15.40:0/1077020985 wait complete. 2024-09-13T18:01:34.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.907+0000 7fc0a4f44700 1 Processor -- start 2024-09-13T18:01:34.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.907+0000 7fc0a4f44700 1 -- start start 2024-09-13T18:01:34.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.907+0000 7fc0a4f44700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a00c4660 0x7fc0a0153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.907+0000 7fc0a4f44700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a0150240 0x7fc0a0154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.907+0000 7fc0a4f44700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0a01595f0 0x7fc0a0152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.907+0000 7fc0a4f44700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a0153a30 con 0x7fc0a00c4660 2024-09-13T18:01:34.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.907+0000 7fc0a4f44700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a0159a80 con 0x7fc0a01595f0 2024-09-13T18:01:34.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.907+0000 7fc0a4f44700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a014c4f0 con 0x7fc0a0150240 2024-09-13T18:01:34.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc09f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a00c4660 0x7fc0a0153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:34.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc09f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a00c4660 0x7fc0a0153ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42454/0 (socket says 172.21.15.40:42454) 2024-09-13T18:01:34.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc09f7fe700 1 -- 172.21.15.40:0/2909071592 learned_addr learned my addr 172.21.15.40:0/2909071592 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:34.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc09ffff700 1 -- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0a01595f0 msgr2=0x7fc0a0152690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:01:34.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc09ffff700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0a01595f0 0x7fc0a0152690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:34.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc097fff700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a0150240 0x7fc0a0154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:34.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc09f7fe700 1 -- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a0150240 msgr2=0x7fc0a0154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:34.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc09f7fe700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a0150240 0x7fc0a0154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc09f7fe700 1 -- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0a01595f0 msgr2=0x7fc0a0152690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:01:34.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc09f7fe700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0a01595f0 0x7fc0a0152690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc09f7fe700 1 -- 172.21.15.40:0/2909071592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc088009570 con 0x7fc0a00c4660 2024-09-13T18:01:34.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc09f7fe700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a00c4660 0x7fc0a0153ed0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc09000e9d0 tx=0x7fc09000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:34.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc09cff9700 1 -- 172.21.15.40:0/2909071592 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc090009720 con 0x7fc0a00c4660 2024-09-13T18:01:34.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc09cff9700 1 -- 172.21.15.40:0/2909071592 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09000c850 con 0x7fc0a00c4660 2024-09-13T18:01:34.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc0a4f44700 1 -- 172.21.15.40:0/2909071592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a0151bb0 con 0x7fc0a00c4660 2024-09-13T18:01:34.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc09cff9700 1 -- 172.21.15.40:0/2909071592 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09000cbc0 con 0x7fc0a00c4660 2024-09-13T18:01:34.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc0a4f44700 1 -- 172.21.15.40:0/2909071592 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a0152170 con 0x7fc0a00c4660 2024-09-13T18:01:34.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc0a4f44700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a0152bd0 0x7fc0a01efdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc0a4f44700 1 -- 172.21.15.40:0/2909071592 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a0067600 con 0x7fc0a0152bd0 2024-09-13T18:01:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc097fff700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a0152bd0 0x7fc0a01efdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc09cff9700 1 -- 172.21.15.40:0/2909071592 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc09000cd20 con 0x7fc0a00c4660 2024-09-13T18:01:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc097fff700 1 -- 172.21.15.40:0/2909071592 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc0a0151a30 con 0x7fc0a0152bd0 2024-09-13T18:01:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc09cff9700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc080066710 0x7fc080068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc09cff9700 1 -- 172.21.15.40:0/2909071592 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc0900849f0 con 0x7fc0a00c4660 2024-09-13T18:01:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc09ffff700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc080066710 0x7fc080068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.911+0000 7fc097fff700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a0152bd0 0x7fc0a01efdf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc088004700 tx=0x7fc088004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.915+0000 7fc09cff9700 1 -- 172.21.15.40:0/2909071592 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc088011630 con 0x7fc0a0152bd0 2024-09-13T18:01:34.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.915+0000 7fc09ffff700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc080066710 0x7fc080068bd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc0a015b280 tx=0x7fc0980091f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:34.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.915+0000 7fc09cff9700 1 -- 172.21.15.40:0/2909071592 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc088011790 con 0x7fc0a0152bd0 2024-09-13T18:01:34.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.915+0000 7fc09cff9700 1 -- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a0152bd0 msgr2=0x7fc0a01efdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:34.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.915+0000 7fc09cff9700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a0152bd0 0x7fc0a01efdf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc088004700 tx=0x7fc088004770 comp rx=0 tx=0).stop 2024-09-13T18:01:34.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.915+0000 7fc09cff9700 1 -- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a0152bd0 msgr2=0x7fc0a01efdf0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:01:34.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.915+0000 7fc09cff9700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a0152bd0 0x7fc0a01efdf0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.927+0000 7fc0a4f44700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a01f0330 0x7fc0a01f0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:34.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.927+0000 7fc0a4f44700 1 -- 172.21.15.40:0/2909071592 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a0151d40 con 0x7fc0a01f0330 2024-09-13T18:01:34.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.927+0000 7fc097fff700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a01f0330 0x7fc0a01f0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:34.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.927+0000 7fc097fff700 1 -- 172.21.15.40:0/2909071592 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc0a0071300 con 0x7fc0a01f0330 2024-09-13T18:01:34.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.927+0000 7fc097fff700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a01f0330 0x7fc0a01f0740 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc088009910 tx=0x7fc0880092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:34.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.927+0000 7fc09cff9700 1 -- 172.21.15.40:0/2909071592 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08802a040 con 0x7fc0a01f0330 2024-09-13T18:01:34.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.927+0000 7fc09cff9700 1 -- 172.21.15.40:0/2909071592 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4604 (secure 0 0 0) 0x7fc08801b640 con 0x7fc0a01f0330 2024-09-13T18:01:34.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.927+0000 7fc09cff9700 1 -- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a01f0330 msgr2=0x7fc0a01f0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:34.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.927+0000 7fc09cff9700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a01f0330 0x7fc0a01f0740 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc088009910 tx=0x7fc0880092e0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.927+0000 7fc09cff9700 1 -- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a01f0330 msgr2=0x7fc0a01f0740 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:01:34.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.927+0000 7fc09cff9700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0a01f0330 0x7fc0a01f0740 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc088009910 tx=0x7fc0880092e0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.939 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:01:34.939 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:01:34.939 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:01:34.939 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:01:34.939 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 86, 2024-09-13T18:01:34.939 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:01:34.939 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:01:34.940 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:01:34.940 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:34.940 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:01:34.940 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:01:34.940 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:01:34.940 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:01:34.940 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:34.940 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:34.940 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:34.940 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:34.940 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:34.940 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:34.940 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:34.941 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:34.941 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:34.941 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:34.941 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:34.941 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:01:34.941 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:01:34.941 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:34.941 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:34.941 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:34.941 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:34.941 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:34.941 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:34.941 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:34.942 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:34.942 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:34.942 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:34.942 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:34.942 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.942 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:01:34.942 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:01:34.942 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:01:34.942 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:01:34.942 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:01:34.942 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:01:34.942 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:01:34.942 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:01:34.942 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:01:34.943 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:01:34.943 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:01:34.943 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:01:34.943 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:01:34.943 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:01:34.943 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:01:34.943 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:01:34.943 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:01:34.943 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:01:34.943 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:01:34.943 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:01:34.943 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:01:34.943 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:01:34.943 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:01:34.944 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:01:34.944 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:01:34.944 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:01:34.944 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:01:34.944 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:34.944 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:01:34.944 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.944 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:01:34.944 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.944 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:01:34.944 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:01:34.944 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:34.944 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:34.945 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.945 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:34.945 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:01:34.945 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:34.945 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.945 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.945 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:34.945 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:01:34.945 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:34.945 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:34.945 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:34.945 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.945 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:01:34.945 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:01:34.945 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:34.946 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:34.946 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:34.946 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.946 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.946 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:01:34.946 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:01:34.946 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:34.946 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:34.946 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.946 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:34.946 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:01:34.946 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:34.946 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.946 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.946 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:34.947 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:01:34.947 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:34.947 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:34.947 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:34.947 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.947 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:01:34.947 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:01:34.947 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:34.947 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:34.947 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:34.947 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.947 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.947 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:01:34.947 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:01:34.947 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:01:34.948 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:01:34.948 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.948 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:01:34.948 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:01:34.948 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:34.948 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.948 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.948 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:01:34.948 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:01:34.948 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:01:34.948 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:34.948 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:34.948 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.948 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:01:34.949 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:01:34.949 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:01:34.949 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:01:34.949 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:01:34.949 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:34.949 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:34.949 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.949 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:01:34.949 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:01:34.949 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.949 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:34.949 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:34.949 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:01:34.949 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:34.949 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:34.950 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:01:34.950 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.950 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:34.950 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:34.950 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:01:34.950 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:34.950 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:01:34.950 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:01:34.950 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:01:34.950 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:01:34.950 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:01:34.950 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:01:34.950 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:01:34.950 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:01:34.950 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:01:34.951 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:01:34.951 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:01:34.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.927+0000 7fc0a4f44700 1 -- 172.21.15.40:0/2909071592 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc080066710 msgr2=0x7fc080068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:34.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.927+0000 7fc0a4f44700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc080066710 0x7fc080068bd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc0a015b280 tx=0x7fc0980091f0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.931+0000 7fc0a4f44700 1 -- 172.21.15.40:0/2909071592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a00c4660 msgr2=0x7fc0a0153ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:34.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.931+0000 7fc0a4f44700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a00c4660 0x7fc0a0153ed0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc09000e9d0 tx=0x7fc09000ed90 comp rx=0 tx=0).stop 2024-09-13T18:01:34.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.931+0000 7fc09ffff700 1 -- 172.21.15.40:0/2909071592 reap_dead start 2024-09-13T18:01:34.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.931+0000 7fc0a4f44700 1 -- 172.21.15.40:0/2909071592 shutdown_connections 2024-09-13T18:01:34.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.931+0000 7fc0a4f44700 1 --2- 172.21.15.40:0/2909071592 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0a00c4660 0x7fc0a0153ed0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:34.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.931+0000 7fc0a4f44700 1 -- 172.21.15.40:0/2909071592 >> 172.21.15.40:0/2909071592 conn(0x7fc0a006a390 msgr2=0x7fc0a014f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:34.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.931+0000 7fc0a4f44700 1 -- 172.21.15.40:0/2909071592 shutdown_connections 2024-09-13T18:01:34.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:34.931+0000 7fc0a4f44700 1 -- 172.21.15.40:0/2909071592 wait complete. 2024-09-13T18:01:34.953 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:01:34.953 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:01:35.783 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:01:35.786 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:01:54.954 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-13T18:01:54.955 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-13T18:01:54.955 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T18:01:54.955 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T18:01:54.958 INFO:tasks.ceph.mon.b:Started 2024-09-13T18:01:54.959 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:01:54.959 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:01:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.071+0000 7f9cf5484700 1 Processor -- start 2024-09-13T18:01:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.071+0000 7f9cf5484700 1 -- start start 2024-09-13T18:01:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.071+0000 7f9cf5484700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9cf0151210 0x7f9cf006d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.071+0000 7f9cf5484700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf006ec70 con 0x7f9cf006d9d0 2024-09-13T18:01:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.071+0000 7f9cf5484700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf0158070 con 0x7f9cf01515f0 2024-09-13T18:01:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.071+0000 7f9cf5484700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf0157210 con 0x7f9cf0151210 2024-09-13T18:01:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.071+0000 7f9ceffff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9cf0151210 0x7f9cf006d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.071+0000 7f9cef7fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9cf01515f0 0x7f9cf00c4a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:58956/0 (socket says 172.21.15.40:58956) 2024-09-13T18:01:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.071+0000 7f9ceffff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9cf0151210 0x7f9cf006d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:52952/0 (socket says 172.21.15.40:52952) 2024-09-13T18:01:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.071+0000 7f9cef7fe700 1 -- 172.21.15.40:0/1411098511 learned_addr learned my addr 172.21.15.40:0/1411098511 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:01:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.071+0000 7f9ceffff700 1 -- 172.21.15.40:0/1411098511 >> v2:172.21.15.43:3300/0 conn(0x7f9cf0151210 msgr2=0x7f9cf006d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:01:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.071+0000 7f9ceffff700 1 -- 172.21.15.40:0/1411098511 >> v2:172.21.15.43:3300/0 conn(0x7f9cf0151210 msgr2=0x7f9cf006d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:01:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.071+0000 7f9ceffff700 1 --2- 172.21.15.40:0/1411098511 >> v2:172.21.15.43:3300/0 conn(0x7f9cf0151210 0x7f9cf006d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:01:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.071+0000 7f9ceffff700 1 --2- 172.21.15.40:0/1411098511 >> v2:172.21.15.43:3300/0 conn(0x7f9cf0151210 0x7f9cf006d490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.071+0000 7f9ceeffd700 1 -- 172.21.15.40:0/1411098511 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3680003294 0 0) 0x7f9cf006ec70 con 0x7f9cf006d9d0 2024-09-13T18:01:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.071+0000 7f9ceeffd700 1 -- 172.21.15.40:0/1411098511 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cd80035d0 con 0x7f9cf006d9d0 2024-09-13T18:01:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9ceeffd700 1 -- 172.21.15.40:0/1411098511 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 277726736 0 0) 0x7f9cd80035d0 con 0x7f9cf006d9d0 2024-09-13T18:01:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9ceeffd700 1 -- 172.21.15.40:0/1411098511 >> v2:172.21.15.43:3300/0 conn(0x7f9cf0151210 msgr2=0x7f9cf006d490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:01:55.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9ceeffd700 1 --2- 172.21.15.40:0/1411098511 >> v2:172.21.15.43:3300/0 conn(0x7f9cf0151210 0x7f9cf006d490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9ceeffd700 1 -- 172.21.15.40:0/1411098511 >> v1:172.21.15.43:6789/0 conn(0x7f9cf01515f0 legacy=0x7f9cf00c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9ceeffd700 1 -- 172.21.15.40:0/1411098511 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf01574f0 con 0x7f9cf006d9d0 2024-09-13T18:01:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9ceeffd700 1 -- 172.21.15.40:0/1411098511 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f9ce80019f0 con 0x7f9cf006d9d0 2024-09-13T18:01:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9ceeffd700 1 -- 172.21.15.40:0/1411098511 >> v1:172.21.15.40:6789/0 conn(0x7f9cf006d9d0 legacy=0x7f9cf0070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9ceeffd700 1 --2- 172.21.15.40:0/1411098511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cd8003a20 0x7f9cd8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9ceeffd700 1 --2- 172.21.15.40:0/1411098511 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9cd8004870 0x7f9cd8004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9ceeffd700 1 --2- 172.21.15.40:0/1411098511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8005230 0x7f9cd800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9ceeffd700 1 -- 172.21.15.40:0/1411098511 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf0157210 con 0x7f9cd8005230 2024-09-13T18:01:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9ceeffd700 1 -- 172.21.15.40:0/1411098511 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd8010570 con 0x7f9cd8003a20 2024-09-13T18:01:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9ceeffd700 1 -- 172.21.15.40:0/1411098511 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd8010730 con 0x7f9cd8004870 2024-09-13T18:01:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9cf4c83700 1 --2- 172.21.15.40:0/1411098511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8005230 0x7f9cd800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9cef7fe700 1 --2- 172.21.15.40:0/1411098511 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9cd8004870 0x7f9cd8004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:55.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9ceffff700 1 --2- 172.21.15.40:0/1411098511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cd8003a20 0x7f9cd8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9cf4c83700 1 -- 172.21.15.40:0/1411098511 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9cd8004870 msgr2=0x7f9cd8004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:01:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9cf4c83700 1 --2- 172.21.15.40:0/1411098511 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9cd8004870 0x7f9cd8004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9cf4c83700 1 -- 172.21.15.40:0/1411098511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cd8003a20 msgr2=0x7f9cd8003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:01:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9cf4c83700 1 --2- 172.21.15.40:0/1411098511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cd8003a20 0x7f9cd8003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9cf4c83700 1 -- 172.21.15.40:0/1411098511 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd80035d0 con 0x7f9cd8005230 2024-09-13T18:01:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9cf4c83700 1 -- 172.21.15.40:0/1411098511 reap_dead start 2024-09-13T18:01:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9cf4c83700 1 --2- 172.21.15.40:0/1411098511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8005230 0x7f9cd800ba30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800a830 tx=0x7f9cd80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996decd699faa26d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9ceeffd700 1 -- 172.21.15.40:0/1411098511 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce800a320 con 0x7f9cd8005230 2024-09-13T18:01:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9ceeffd700 1 -- 172.21.15.40:0/1411098511 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce8012660 con 0x7f9cd8005230 2024-09-13T18:01:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9ceeffd700 1 -- 172.21.15.40:0/1411098511 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8012a10 con 0x7f9cd8005230 2024-09-13T18:01:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9cf5484700 1 -- 172.21.15.40:0/1411098511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8005230 msgr2=0x7f9cd800ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9cf5484700 1 --2- 172.21.15.40:0/1411098511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8005230 0x7f9cd800ba30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800a830 tx=0x7f9cd80058a0 comp rx=0 tx=0).stop 2024-09-13T18:01:55.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9cf5484700 1 -- 172.21.15.40:0/1411098511 shutdown_connections 2024-09-13T18:01:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9cf5484700 1 --2- 172.21.15.40:0/1411098511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8005230 0x7f9cd800ba30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9cf5484700 1 -- 172.21.15.40:0/1411098511 >> 172.21.15.40:0/1411098511 conn(0x7f9cf006a390 msgr2=0x7f9cf006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9cf5484700 1 -- 172.21.15.40:0/1411098511 shutdown_connections 2024-09-13T18:01:55.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.075+0000 7f9cf5484700 1 -- 172.21.15.40:0/1411098511 wait complete. 2024-09-13T18:01:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9cf5484700 1 Processor -- start 2024-09-13T18:01:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9cf5484700 1 -- start start 2024-09-13T18:01:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9cf5484700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8005230 0x7f9cf01545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9cf5484700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9cf01582c0 0x7f9cf01ee4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9cf5484700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf01eacd0 0x7f9cf01eb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:55.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9cf5484700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf01edae0 con 0x7f9cd8005230 2024-09-13T18:01:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9cf5484700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf01ea4f0 con 0x7f9cf01eacd0 2024-09-13T18:01:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9cf5484700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf014daa0 con 0x7f9cf01582c0 2024-09-13T18:01:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9ceffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8005230 0x7f9cf01545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9ceffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8005230 0x7f9cf01545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51548/0 (socket says 172.21.15.40:51548) 2024-09-13T18:01:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9ceffff700 1 -- 172.21.15.40:0/3835987061 learned_addr learned my addr 172.21.15.40:0/3835987061 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9cf4c83700 1 --2- 172.21.15.40:0/3835987061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf01eacd0 0x7f9cf01eb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9cef7fe700 1 --2- 172.21.15.40:0/3835987061 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9cf01582c0 0x7f9cf01ee4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9ceffff700 1 -- 172.21.15.40:0/3835987061 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9cf01582c0 msgr2=0x7f9cf01ee4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9ceffff700 1 --2- 172.21.15.40:0/3835987061 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9cf01582c0 0x7f9cf01ee4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9ceffff700 1 -- 172.21.15.40:0/3835987061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf01eacd0 msgr2=0x7f9cf01eb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9cf4c83700 1 -- 172.21.15.40:0/3835987061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf01eacd0 msgr2=0x7f9cf01eb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-13T18:01:55.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9cf4c83700 1 -- 172.21.15.40:0/3835987061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf01eacd0 msgr2=0x7f9cf01eb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:01:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9cf4c83700 1 --2- 172.21.15.40:0/3835987061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf01eacd0 0x7f9cf01eb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:01:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9cf4c83700 1 --2- 172.21.15.40:0/3835987061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf01eacd0 0x7f9cf01eb130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9ceffff700 1 --2- 172.21.15.40:0/3835987061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf01eacd0 0x7f9cf01eb130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9ceffff700 1 -- 172.21.15.40:0/3835987061 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf014dd80 con 0x7f9cd8005230 2024-09-13T18:01:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9ceffff700 1 --2- 172.21.15.40:0/3835987061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8005230 0x7f9cf01545e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0011600 tx=0x7f9ce00119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9cecff9700 1 -- 172.21.15.40:0/3835987061 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce000c070 con 0x7f9cd8005230 2024-09-13T18:01:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9cecff9700 1 -- 172.21.15.40:0/3835987061 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce0015870 con 0x7f9cd8005230 2024-09-13T18:01:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9cecff9700 1 -- 172.21.15.40:0/3835987061 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0015c00 con 0x7f9cd8005230 2024-09-13T18:01:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9cf5484700 1 -- 172.21.15.40:0/3835987061 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf014e060 con 0x7f9cd8005230 2024-09-13T18:01:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9cf5484700 1 -- 172.21.15.40:0/3835987061 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf01ee9f0 con 0x7f9cd8005230 2024-09-13T18:01:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.079+0000 7f9cf5484700 1 -- 172.21.15.40:0/3835987061 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf014e220 con 0x7f9cd8005230 2024-09-13T18:01:55.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.083+0000 7f9cecff9700 1 -- 172.21.15.40:0/3835987061 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9ce0013400 con 0x7f9cd8005230 2024-09-13T18:01:55.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.083+0000 7f9cecff9700 1 --2- 172.21.15.40:0/3835987061 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9cdc066760 0x7f9cdc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:55.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.083+0000 7f9cef7fe700 1 --2- 172.21.15.40:0/3835987061 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9cdc066760 0x7f9cdc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:55.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.083+0000 7f9cecff9700 1 -- 172.21.15.40:0/3835987061 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9ce00846c0 con 0x7f9cd8005230 2024-09-13T18:01:55.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.083+0000 7f9cef7fe700 1 --2- 172.21.15.40:0/3835987061 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9cdc066760 0x7f9cdc068c20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9cf014b020 tx=0x7f9ce4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:55.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.087+0000 7f9cecff9700 1 -- 172.21.15.40:0/3835987061 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9ce0051700 con 0x7f9cd8005230 2024-09-13T18:01:55.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.339+0000 7f9cf5484700 1 -- 172.21.15.40:0/3835987061 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f9cf0066330 con 0x7f9cd8005230 2024-09-13T18:01:55.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.339+0000 7f9cecff9700 1 -- 172.21.15.40:0/3835987061 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f9ce0054d20 con 0x7f9cd8005230 2024-09-13T18:01:55.350 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":86,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:01:55.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.343+0000 7f9cf5484700 1 -- 172.21.15.40:0/3835987061 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9cdc066760 msgr2=0x7f9cdc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:55.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.343+0000 7f9cf5484700 1 --2- 172.21.15.40:0/3835987061 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9cdc066760 0x7f9cdc068c20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9cf014b020 tx=0x7f9ce4009040 comp rx=0 tx=0).stop 2024-09-13T18:01:55.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.343+0000 7f9cf5484700 1 -- 172.21.15.40:0/3835987061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8005230 msgr2=0x7f9cf01545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:55.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.343+0000 7f9cf5484700 1 --2- 172.21.15.40:0/3835987061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8005230 0x7f9cf01545e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0011600 tx=0x7f9ce00119c0 comp rx=0 tx=0).stop 2024-09-13T18:01:55.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.343+0000 7f9cf5484700 1 -- 172.21.15.40:0/3835987061 shutdown_connections 2024-09-13T18:01:55.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.343+0000 7f9cf5484700 1 --2- 172.21.15.40:0/3835987061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf01eacd0 0x7f9cf01eb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:55.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.343+0000 7f9cf5484700 1 --2- 172.21.15.40:0/3835987061 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9cdc066760 0x7f9cdc068c20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:55.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.343+0000 7f9cf5484700 1 --2- 172.21.15.40:0/3835987061 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9cf01582c0 0x7f9cf01ee4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:55.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.343+0000 7f9cf5484700 1 --2- 172.21.15.40:0/3835987061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9cd8005230 0x7f9cf01545e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:55.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.343+0000 7f9cf5484700 1 -- 172.21.15.40:0/3835987061 >> 172.21.15.40:0/3835987061 conn(0x7f9cf006a390 msgr2=0x7f9cf00c95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:55.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.343+0000 7f9cf5484700 1 -- 172.21.15.40:0/3835987061 shutdown_connections 2024-09-13T18:01:55.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:55.343+0000 7f9cf5484700 1 -- 172.21.15.40:0/3835987061 wait complete. 2024-09-13T18:01:58.364 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:01:58.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.471+0000 7fccccb93700 1 Processor -- start 2024-09-13T18:01:58.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.471+0000 7fccccb93700 1 -- start start 2024-09-13T18:01:58.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.471+0000 7fccccb93700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fccc815cf60 0x7fccc815d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:58.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.471+0000 7fccccb93700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccc815d910 con 0x7fccc8153880 2024-09-13T18:01:58.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.471+0000 7fccccb93700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccc8159770 con 0x7fccc80cb2f0 2024-09-13T18:01:58.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.471+0000 7fccccb93700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccc8159a50 con 0x7fccc815cf60 2024-09-13T18:01:58.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.471+0000 7fccc77fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fccc8153880 0x7fccc80c9ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40844/0 (socket says 172.21.15.40:40844) 2024-09-13T18:01:58.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.471+0000 7fccc77fe700 1 -- 172.21.15.40:0/4266577257 learned_addr learned my addr 172.21.15.40:0/4266577257 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:01:58.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.471+0000 7fccc7fff700 1 --2- 172.21.15.40:0/4266577257 >> v2:172.21.15.43:3300/0 conn(0x7fccc815cf60 0x7fccc815d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:58.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.471+0000 7fccc6ffd700 1 -- 172.21.15.40:0/4266577257 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725597156 0 0) 0x7fccc815d910 con 0x7fccc8153880 2024-09-13T18:01:58.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.471+0000 7fccc6ffd700 1 -- 172.21.15.40:0/4266577257 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccb40035d0 con 0x7fccc8153880 2024-09-13T18:01:58.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc7fff700 1 -- 172.21.15.40:0/4266577257 >> v2:172.21.15.43:3300/0 conn(0x7fccc815cf60 msgr2=0x7fccc815d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:01:58.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc7fff700 1 -- 172.21.15.40:0/4266577257 >> v2:172.21.15.43:3300/0 conn(0x7fccc815cf60 msgr2=0x7fccc815d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:01:58.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc7fff700 1 --2- 172.21.15.40:0/4266577257 >> v2:172.21.15.43:3300/0 conn(0x7fccc815cf60 0x7fccc815d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:01:58.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc7fff700 1 --2- 172.21.15.40:0/4266577257 >> v2:172.21.15.43:3300/0 conn(0x7fccc815cf60 0x7fccc815d3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:01:58.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc6ffd700 1 -- 172.21.15.40:0/4266577257 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1109474732 0 0) 0x7fccb40035d0 con 0x7fccc8153880 2024-09-13T18:01:58.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc6ffd700 1 -- 172.21.15.40:0/4266577257 >> v2:172.21.15.43:3300/0 conn(0x7fccc815cf60 msgr2=0x7fccc815d3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:01:58.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc6ffd700 1 --2- 172.21.15.40:0/4266577257 >> v2:172.21.15.43:3300/0 conn(0x7fccc815cf60 0x7fccc815d3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:58.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc6ffd700 1 -- 172.21.15.40:0/4266577257 >> v1:172.21.15.43:6789/0 conn(0x7fccc80cb2f0 legacy=0x7fccc80cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:58.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc6ffd700 1 -- 172.21.15.40:0/4266577257 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc8159d30 con 0x7fccc8153880 2024-09-13T18:01:58.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc6ffd700 1 -- 172.21.15.40:0/4266577257 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fccb8001670 con 0x7fccc8153880 2024-09-13T18:01:58.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc6ffd700 1 -- 172.21.15.40:0/4266577257 >> v1:172.21.15.40:6789/0 conn(0x7fccc8153880 legacy=0x7fccc80c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:58.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc6ffd700 1 --2- 172.21.15.40:0/4266577257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccb4003a20 0x7fccb4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:58.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc6ffd700 1 --2- 172.21.15.40:0/4266577257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fccb4004870 0x7fccb4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:58.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc6ffd700 1 --2- 172.21.15.40:0/4266577257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccb4005230 0x7fccb400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:58.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc6ffd700 1 -- 172.21.15.40:0/4266577257 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc8159a50 con 0x7fccb4005230 2024-09-13T18:01:58.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc6ffd700 1 -- 172.21.15.40:0/4266577257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb400c180 con 0x7fccb4003a20 2024-09-13T18:01:58.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc6ffd700 1 -- 172.21.15.40:0/4266577257 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb400c340 con 0x7fccb4004870 2024-09-13T18:01:58.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc7fff700 1 --2- 172.21.15.40:0/4266577257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccb4005230 0x7fccb400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:58.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccbeffd700 1 --2- 172.21.15.40:0/4266577257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fccb4004870 0x7fccb4004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:58.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc77fe700 1 --2- 172.21.15.40:0/4266577257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccb4003a20 0x7fccb4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:58.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc7fff700 1 -- 172.21.15.40:0/4266577257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fccb4004870 msgr2=0x7fccb4004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:01:58.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc7fff700 1 --2- 172.21.15.40:0/4266577257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fccb4004870 0x7fccb4004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:58.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc7fff700 1 -- 172.21.15.40:0/4266577257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccb4003a20 msgr2=0x7fccb4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:01:58.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc7fff700 1 --2- 172.21.15.40:0/4266577257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccb4003a20 0x7fccb4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:58.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc7fff700 1 -- 172.21.15.40:0/4266577257 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb40035d0 con 0x7fccb4005230 2024-09-13T18:01:58.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc7fff700 1 -- 172.21.15.40:0/4266577257 reap_dead start 2024-09-13T18:01:58.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc7fff700 1 --2- 172.21.15.40:0/4266577257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccb4005230 0x7fccb400ba30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009fd0 tx=0x7fccb40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e700ae361f6539b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:58.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc6ffd700 1 -- 172.21.15.40:0/4266577257 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0007e20 con 0x7fccb4005230 2024-09-13T18:01:58.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc6ffd700 1 -- 172.21.15.40:0/4266577257 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc0003a40 con 0x7fccb4005230 2024-09-13T18:01:58.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccc6ffd700 1 -- 172.21.15.40:0/4266577257 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0003df0 con 0x7fccb4005230 2024-09-13T18:01:58.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccccb93700 1 -- 172.21.15.40:0/4266577257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccb4005230 msgr2=0x7fccb400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:58.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccccb93700 1 --2- 172.21.15.40:0/4266577257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccb4005230 0x7fccb400ba30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009fd0 tx=0x7fccb40058a0 comp rx=0 tx=0).stop 2024-09-13T18:01:58.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccccb93700 1 -- 172.21.15.40:0/4266577257 shutdown_connections 2024-09-13T18:01:58.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccccb93700 1 --2- 172.21.15.40:0/4266577257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccb4005230 0x7fccb400ba30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:58.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccccb93700 1 -- 172.21.15.40:0/4266577257 >> 172.21.15.40:0/4266577257 conn(0x7fccc806c5b0 msgr2=0x7fccc806c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:58.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccccb93700 1 -- 172.21.15.40:0/4266577257 shutdown_connections 2024-09-13T18:01:58.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.475+0000 7fccccb93700 1 -- 172.21.15.40:0/4266577257 wait complete. 2024-09-13T18:01:58.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccccb93700 1 Processor -- start 2024-09-13T18:01:58.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccccb93700 1 -- start start 2024-09-13T18:01:58.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccccb93700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccb4005230 0x7fccc81f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:58.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccccb93700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fccc815c100 0x7fccc80c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:58.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccccb93700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccc814d7e0 0x7fccc80c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:58.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccccb93700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc80c6f70 con 0x7fccb4005230 2024-09-13T18:01:58.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccccb93700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc8158fa0 con 0x7fccc814d7e0 2024-09-13T18:01:58.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccccb93700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc80c8220 con 0x7fccc815c100 2024-09-13T18:01:58.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccc77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccb4005230 0x7fccc81f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:58.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccc77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccb4005230 0x7fccc81f2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:51566/0 (socket says 172.21.15.40:51566) 2024-09-13T18:01:58.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccc77fe700 1 -- 172.21.15.40:0/1409283636 learned_addr learned my addr 172.21.15.40:0/1409283636 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:01:58.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccbeffd700 1 --2- 172.21.15.40:0/1409283636 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fccc815c100 0x7fccc80c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:58.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccc77fe700 1 -- 172.21.15.40:0/1409283636 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fccc815c100 msgr2=0x7fccc80c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:58.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccc77fe700 1 --2- 172.21.15.40:0/1409283636 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fccc815c100 0x7fccc80c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:58.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccc77fe700 1 -- 172.21.15.40:0/1409283636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccc814d7e0 msgr2=0x7fccc80c51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:58.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccc77fe700 1 --2- 172.21.15.40:0/1409283636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccc814d7e0 0x7fccc80c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:58.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccc77fe700 1 -- 172.21.15.40:0/1409283636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc815c530 con 0x7fccb4005230 2024-09-13T18:01:58.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccc77fe700 1 --2- 172.21.15.40:0/1409283636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccb4005230 0x7fccc81f2590 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fccb800eb20 tx=0x7fccb800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:58.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccc4ff9700 1 -- 172.21.15.40:0/1409283636 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb8005e00 con 0x7fccb4005230 2024-09-13T18:01:58.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccc4ff9700 1 -- 172.21.15.40:0/1409283636 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb800c7e0 con 0x7fccb4005230 2024-09-13T18:01:58.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccccb93700 1 -- 172.21.15.40:0/1409283636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc80b2050 con 0x7fccb4005230 2024-09-13T18:01:58.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccc4ff9700 1 -- 172.21.15.40:0/1409283636 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb800cb90 con 0x7fccb4005230 2024-09-13T18:01:58.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.479+0000 7fccccb93700 1 -- 172.21.15.40:0/1409283636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc81591f0 con 0x7fccb4005230 2024-09-13T18:01:58.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.483+0000 7fccbdffb700 1 -- 172.21.15.40:0/1409283636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca0005310 con 0x7fccb4005230 2024-09-13T18:01:58.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.483+0000 7fccc4ff9700 1 -- 172.21.15.40:0/1409283636 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fccb800ccf0 con 0x7fccb4005230 2024-09-13T18:01:58.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.483+0000 7fccc4ff9700 1 --2- 172.21.15.40:0/1409283636 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcca8066760 0x7fcca8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:01:58.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.483+0000 7fccbeffd700 1 --2- 172.21.15.40:0/1409283636 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcca8066760 0x7fcca8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:01:58.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.483+0000 7fccc4ff9700 1 -- 172.21.15.40:0/1409283636 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fccb8084310 con 0x7fccb4005230 2024-09-13T18:01:58.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.483+0000 7fccbeffd700 1 --2- 172.21.15.40:0/1409283636 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcca8066760 0x7fcca8068c20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fccc80b4e50 tx=0x7fccb0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:01:58.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.487+0000 7fccc4ff9700 1 -- 172.21.15.40:0/1409283636 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fccb8051350 con 0x7fccb4005230 2024-09-13T18:01:58.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.735+0000 7fccbdffb700 1 -- 172.21.15.40:0/1409283636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fcca00055c0 con 0x7fccb4005230 2024-09-13T18:01:58.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.739+0000 7fccc4ff9700 1 -- 172.21.15.40:0/1409283636 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7fccb8054970 con 0x7fccb4005230 2024-09-13T18:01:58.747 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":86,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":26,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:01:58.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.739+0000 7fccbdffb700 1 -- 172.21.15.40:0/1409283636 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcca8066760 msgr2=0x7fcca8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:58.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.739+0000 7fccbdffb700 1 --2- 172.21.15.40:0/1409283636 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcca8066760 0x7fcca8068c20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fccc80b4e50 tx=0x7fccb0009040 comp rx=0 tx=0).stop 2024-09-13T18:01:58.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.739+0000 7fccbdffb700 1 -- 172.21.15.40:0/1409283636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccb4005230 msgr2=0x7fccc81f2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:01:58.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.739+0000 7fccbdffb700 1 --2- 172.21.15.40:0/1409283636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccb4005230 0x7fccc81f2590 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fccb800eb20 tx=0x7fccb800eee0 comp rx=0 tx=0).stop 2024-09-13T18:01:58.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.739+0000 7fccbdffb700 1 -- 172.21.15.40:0/1409283636 shutdown_connections 2024-09-13T18:01:58.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.739+0000 7fccbdffb700 1 --2- 172.21.15.40:0/1409283636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccc814d7e0 0x7fccc80c51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:58.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.739+0000 7fccbdffb700 1 --2- 172.21.15.40:0/1409283636 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcca8066760 0x7fcca8068c20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:58.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.739+0000 7fccbdffb700 1 --2- 172.21.15.40:0/1409283636 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fccc815c100 0x7fccc80c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:58.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.739+0000 7fccbdffb700 1 --2- 172.21.15.40:0/1409283636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccb4005230 0x7fccc81f2590 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:01:58.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.739+0000 7fccbdffb700 1 -- 172.21.15.40:0/1409283636 >> 172.21.15.40:0/1409283636 conn(0x7fccc806c5b0 msgr2=0x7fccc81514c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:01:58.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.739+0000 7fccbdffb700 1 -- 172.21.15.40:0/1409283636 shutdown_connections 2024-09-13T18:01:58.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:01:58.739+0000 7fccbdffb700 1 -- 172.21.15.40:0/1409283636 wait complete. 2024-09-13T18:02:01.765 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:02:01.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.879+0000 7fd9f6cd4700 1 Processor -- start 2024-09-13T18:02:01.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.879+0000 7fd9f6cd4700 1 -- start start 2024-09-13T18:02:01.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.879+0000 7fd9f6cd4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd9f006e960 0x7fd9f00c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:01.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.879+0000 7fd9f6cd4700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f00c6db0 con 0x7fd9f01534e0 2024-09-13T18:02:01.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.879+0000 7fd9f6cd4700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9f00c6920 con 0x7fd9f0070290 2024-09-13T18:02:01.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.879+0000 7fd9f6cd4700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9f00c5d50 con 0x7fd9f006e960 2024-09-13T18:02:01.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.879+0000 7fd9f64d3700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd9f006e960 0x7fd9f00c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:01.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.879+0000 7fd9f64d3700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd9f006e960 0x7fd9f00c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42602/0 (socket says 172.21.15.40:42602) 2024-09-13T18:02:01.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.879+0000 7fd9f64d3700 1 -- 172.21.15.40:0/4208825836 learned_addr learned my addr 172.21.15.40:0/4208825836 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:01.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.883+0000 7fd9f64d3700 1 -- 172.21.15.40:0/4208825836 >> v1:172.21.15.43:6789/0 conn(0x7fd9f0070290 legacy=0x7fd9f006d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:01.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.883+0000 7fd9f64d3700 1 -- 172.21.15.40:0/4208825836 >> v1:172.21.15.40:6789/0 conn(0x7fd9f01534e0 legacy=0x7fd9f0159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:01.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.883+0000 7fd9f64d3700 1 -- 172.21.15.40:0/4208825836 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f0070710 con 0x7fd9f006e960 2024-09-13T18:02:01.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.883+0000 7fd9f64d3700 1 --2- 172.21.15.40:0/4208825836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f006e960 0x7fd9f00c7d20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8009e50 tx=0x7fd9e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c4d742730e481c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:01.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.883+0000 7fd9f4cd0700 1 -- 172.21.15.40:0/4208825836 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e800e070 con 0x7fd9f006e960 2024-09-13T18:02:01.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.883+0000 7fd9f4cd0700 1 -- 172.21.15.40:0/4208825836 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e800b9c0 con 0x7fd9f006e960 2024-09-13T18:02:01.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.883+0000 7fd9f4cd0700 1 -- 172.21.15.40:0/4208825836 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e800bd70 con 0x7fd9f006e960 2024-09-13T18:02:01.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.883+0000 7fd9f6cd4700 1 -- 172.21.15.40:0/4208825836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f006e960 msgr2=0x7fd9f00c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.883+0000 7fd9f6cd4700 1 --2- 172.21.15.40:0/4208825836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f006e960 0x7fd9f00c7d20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8009e50 tx=0x7fd9e800b330 comp rx=0 tx=0).stop 2024-09-13T18:02:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.883+0000 7fd9f6cd4700 1 -- 172.21.15.40:0/4208825836 shutdown_connections 2024-09-13T18:02:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.883+0000 7fd9f6cd4700 1 --2- 172.21.15.40:0/4208825836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f006e960 0x7fd9f00c7d20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.883+0000 7fd9f6cd4700 1 -- 172.21.15.40:0/4208825836 >> 172.21.15.40:0/4208825836 conn(0x7fd9f006a390 msgr2=0x7fd9f006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.883+0000 7fd9f6cd4700 1 -- 172.21.15.40:0/4208825836 shutdown_connections 2024-09-13T18:02:01.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.883+0000 7fd9f6cd4700 1 -- 172.21.15.40:0/4208825836 wait complete. 2024-09-13T18:02:01.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f6cd4700 1 Processor -- start 2024-09-13T18:02:01.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f6cd4700 1 -- start start 2024-09-13T18:02:01.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f6cd4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f006e960 0x7fd9f014efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f6cd4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9f0070290 0x7fd9f0145d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f6cd4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9f01534e0 0x7fd9f01462a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f6cd4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f00cbd80 con 0x7fd9f006e960 2024-09-13T18:02:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f6cd4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f00c46a0 con 0x7fd9f0070290 2024-09-13T18:02:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f6cd4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f00c93d0 con 0x7fd9f01534e0 2024-09-13T18:02:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f5cd2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f006e960 0x7fd9f014efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f5cd2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f006e960 0x7fd9f014efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42614/0 (socket says 172.21.15.40:42614) 2024-09-13T18:02:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f5cd2700 1 -- 172.21.15.40:0/2150654944 learned_addr learned my addr 172.21.15.40:0/2150654944 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f54d1700 1 --2- 172.21.15.40:0/2150654944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9f0070290 0x7fd9f0145d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f64d3700 1 --2- 172.21.15.40:0/2150654944 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9f01534e0 0x7fd9f01462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f5cd2700 1 -- 172.21.15.40:0/2150654944 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9f01534e0 msgr2=0x7fd9f01462a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:01.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f5cd2700 1 --2- 172.21.15.40:0/2150654944 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9f01534e0 0x7fd9f01462a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:01.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f5cd2700 1 -- 172.21.15.40:0/2150654944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9f0070290 msgr2=0x7fd9f0145d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:01.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f5cd2700 1 --2- 172.21.15.40:0/2150654944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9f0070290 0x7fd9f0145d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:01.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f5cd2700 1 -- 172.21.15.40:0/2150654944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e8009b30 con 0x7fd9f006e960 2024-09-13T18:02:01.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f5cd2700 1 --2- 172.21.15.40:0/2150654944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f006e960 0x7fd9f014efd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00c500 tx=0x7fd9ec00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:01.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9e67fc700 1 -- 172.21.15.40:0/2150654944 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec0052d0 con 0x7fd9f006e960 2024-09-13T18:02:01.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9e67fc700 1 -- 172.21.15.40:0/2150654944 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ec005430 con 0x7fd9f006e960 2024-09-13T18:02:01.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f6cd4700 1 -- 172.21.15.40:0/2150654944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f00c96b0 con 0x7fd9f006e960 2024-09-13T18:02:01.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9e67fc700 1 -- 172.21.15.40:0/2150654944 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec0055a0 con 0x7fd9f006e960 2024-09-13T18:02:01.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.887+0000 7fd9f6cd4700 1 -- 172.21.15.40:0/2150654944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f00c9c70 con 0x7fd9f006e960 2024-09-13T18:02:01.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.891+0000 7fd9f6cd4700 1 -- 172.21.15.40:0/2150654944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9f006b0c0 con 0x7fd9f006e960 2024-09-13T18:02:01.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.891+0000 7fd9e67fc700 1 -- 172.21.15.40:0/2150654944 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd9ec0038e0 con 0x7fd9f006e960 2024-09-13T18:02:01.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.891+0000 7fd9e67fc700 1 --2- 172.21.15.40:0/2150654944 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd9d4066730 0x7fd9d4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:01.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.891+0000 7fd9e67fc700 1 -- 172.21.15.40:0/2150654944 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd9ec083830 con 0x7fd9f006e960 2024-09-13T18:02:01.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.891+0000 7fd9f54d1700 1 --2- 172.21.15.40:0/2150654944 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd9d4066730 0x7fd9d4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:01.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.891+0000 7fd9f54d1700 1 --2- 172.21.15.40:0/2150654944 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd9d4066730 0x7fd9d4068bf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00c4110 tx=0x7fd9e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:01.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:01.899+0000 7fd9e67fc700 1 -- 172.21.15.40:0/2150654944 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd9ec050870 con 0x7fd9f006e960 2024-09-13T18:02:02.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.167+0000 7fd9f6cd4700 1 -- 172.21.15.40:0/2150654944 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd9f00c7800 con 0x7fd9f006e960 2024-09-13T18:02:02.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.167+0000 7fd9e67fc700 1 -- 172.21.15.40:0/2150654944 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7fd9ec017620 con 0x7fd9f006e960 2024-09-13T18:02:02.179 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":88,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:02:02.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.171+0000 7fd9f6cd4700 1 -- 172.21.15.40:0/2150654944 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd9d4066730 msgr2=0x7fd9d4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.171+0000 7fd9f6cd4700 1 --2- 172.21.15.40:0/2150654944 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd9d4066730 0x7fd9d4068bf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00c4110 tx=0x7fd9e0009040 comp rx=0 tx=0).stop 2024-09-13T18:02:02.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.171+0000 7fd9f6cd4700 1 -- 172.21.15.40:0/2150654944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f006e960 msgr2=0x7fd9f014efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.171+0000 7fd9f6cd4700 1 --2- 172.21.15.40:0/2150654944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f006e960 0x7fd9f014efd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00c500 tx=0x7fd9ec00c810 comp rx=0 tx=0).stop 2024-09-13T18:02:02.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.171+0000 7fd9f6cd4700 1 -- 172.21.15.40:0/2150654944 shutdown_connections 2024-09-13T18:02:02.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.171+0000 7fd9f6cd4700 1 --2- 172.21.15.40:0/2150654944 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd9d4066730 0x7fd9d4068bf0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.171+0000 7fd9f6cd4700 1 --2- 172.21.15.40:0/2150654944 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9f01534e0 0x7fd9f01462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.171+0000 7fd9f6cd4700 1 --2- 172.21.15.40:0/2150654944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9f0070290 0x7fd9f0145d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.171+0000 7fd9f6cd4700 1 --2- 172.21.15.40:0/2150654944 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9f006e960 0x7fd9f014efd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.171+0000 7fd9f6cd4700 1 -- 172.21.15.40:0/2150654944 >> 172.21.15.40:0/2150654944 conn(0x7fd9f006a390 msgr2=0x7fd9f0157360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:02.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.171+0000 7fd9f6cd4700 1 -- 172.21.15.40:0/2150654944 shutdown_connections 2024-09-13T18:02:02.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.171+0000 7fd9f6cd4700 1 -- 172.21.15.40:0/2150654944 wait complete. 2024-09-13T18:02:02.196 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:02:02.196 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:02:02.197 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:02:02.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.291+0000 7f9c96db7700 1 Processor -- start 2024-09-13T18:02:02.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.291+0000 7f9c96db7700 1 -- start start 2024-09-13T18:02:02.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.291+0000 7f9c96db7700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9c88007b60 0x7f9c8804b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.291+0000 7f9c96db7700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c88047610 con 0x7f9c880d87a0 2024-09-13T18:02:02.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.291+0000 7f9c96db7700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c88006b50 con 0x7f9c880469c0 2024-09-13T18:02:02.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.291+0000 7f9c96db7700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c880076d0 con 0x7f9c88007b60 2024-09-13T18:02:02.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.291+0000 7f9c95db5700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9c880d87a0 0x7f9c880df110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54308/0 (socket says 172.21.15.40:54308) 2024-09-13T18:02:02.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.291+0000 7f9c95db5700 1 -- 172.21.15.40:0/240766660 learned_addr learned my addr 172.21.15.40:0/240766660 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:02:02.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.291+0000 7f9c955b4700 1 --2- 172.21.15.40:0/240766660 >> v2:172.21.15.43:3300/0 conn(0x7f9c88007b60 0x7f9c8804b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.291+0000 7f9c94db3700 1 -- 172.21.15.40:0/240766660 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 34951028 0 0) 0x7f9c88047610 con 0x7f9c880d87a0 2024-09-13T18:02:02.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.291+0000 7f9c94db3700 1 -- 172.21.15.40:0/240766660 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c7c0035d0 con 0x7f9c880d87a0 2024-09-13T18:02:02.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c94db3700 1 -- 172.21.15.40:0/240766660 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1735166499 0 0) 0x7f9c7c0035d0 con 0x7f9c880d87a0 2024-09-13T18:02:02.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c94db3700 1 -- 172.21.15.40:0/240766660 >> v2:172.21.15.43:3300/0 conn(0x7f9c88007b60 msgr2=0x7f9c8804b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:02.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c94db3700 1 --2- 172.21.15.40:0/240766660 >> v2:172.21.15.43:3300/0 conn(0x7f9c88007b60 0x7f9c8804b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c94db3700 1 -- 172.21.15.40:0/240766660 >> v1:172.21.15.43:6789/0 conn(0x7f9c880469c0 legacy=0x7f9c880e9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c94db3700 1 -- 172.21.15.40:0/240766660 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c880d8b80 con 0x7f9c880d87a0 2024-09-13T18:02:02.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c94db3700 1 -- 172.21.15.40:0/240766660 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f9c8c001670 con 0x7f9c880d87a0 2024-09-13T18:02:02.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c94db3700 1 -- 172.21.15.40:0/240766660 >> v1:172.21.15.40:6789/0 conn(0x7f9c880d87a0 legacy=0x7f9c880df110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c94db3700 1 --2- 172.21.15.40:0/240766660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c7c003a20 0x7f9c7c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c94db3700 1 --2- 172.21.15.40:0/240766660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c7c004870 0x7f9c7c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c94db3700 1 --2- 172.21.15.40:0/240766660 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c7c005230 0x7f9c7c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c94db3700 1 -- 172.21.15.40:0/240766660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880076d0 con 0x7f9c7c004870 2024-09-13T18:02:02.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c94db3700 1 -- 172.21.15.40:0/240766660 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c00c080 con 0x7f9c7c003a20 2024-09-13T18:02:02.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c94db3700 1 -- 172.21.15.40:0/240766660 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7c00c240 con 0x7f9c7c005230 2024-09-13T18:02:02.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c955b4700 1 --2- 172.21.15.40:0/240766660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c7c004870 0x7f9c7c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c95db5700 1 --2- 172.21.15.40:0/240766660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c7c003a20 0x7f9c7c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c965b6700 1 --2- 172.21.15.40:0/240766660 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c7c005230 0x7f9c7c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c955b4700 1 -- 172.21.15.40:0/240766660 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c7c005230 msgr2=0x7f9c7c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:02.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c955b4700 1 --2- 172.21.15.40:0/240766660 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c7c005230 0x7f9c7c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c955b4700 1 -- 172.21.15.40:0/240766660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c7c003a20 msgr2=0x7f9c7c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:02.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c955b4700 1 --2- 172.21.15.40:0/240766660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c7c003a20 0x7f9c7c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c955b4700 1 -- 172.21.15.40:0/240766660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c7c0035d0 con 0x7f9c7c004870 2024-09-13T18:02:02.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c965b6700 1 -- 172.21.15.40:0/240766660 reap_dead start 2024-09-13T18:02:02.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c955b4700 1 --2- 172.21.15.40:0/240766660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c7c004870 0x7f9c7c004cf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009b70 tx=0x7f9c7c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d589704910c3b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:02.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c94db3700 1 -- 172.21.15.40:0/240766660 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8000e6c0 con 0x7f9c7c004870 2024-09-13T18:02:02.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c94db3700 1 -- 172.21.15.40:0/240766660 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8000e820 con 0x7f9c7c004870 2024-09-13T18:02:02.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c94db3700 1 -- 172.21.15.40:0/240766660 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8000eb90 con 0x7f9c7c004870 2024-09-13T18:02:02.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c96db7700 1 -- 172.21.15.40:0/240766660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c7c004870 msgr2=0x7f9c7c004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c96db7700 1 --2- 172.21.15.40:0/240766660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c7c004870 0x7f9c7c004cf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9c80009b70 tx=0x7f9c7c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c96db7700 1 -- 172.21.15.40:0/240766660 shutdown_connections 2024-09-13T18:02:02.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c96db7700 1 --2- 172.21.15.40:0/240766660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c7c004870 0x7f9c7c004cf0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c96db7700 1 -- 172.21.15.40:0/240766660 >> 172.21.15.40:0/240766660 conn(0x7f9c88050e60 msgr2=0x7f9c88051270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:02.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c96db7700 1 -- 172.21.15.40:0/240766660 shutdown_connections 2024-09-13T18:02:02.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c96db7700 1 -- 172.21.15.40:0/240766660 wait complete. 2024-09-13T18:02:02.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c96db7700 1 Processor -- start 2024-09-13T18:02:02.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.295+0000 7f9c96db7700 1 -- start start 2024-09-13T18:02:02.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c96db7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c7c004870 0x7f9c88049df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c96db7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c880cfba0 0x7f9c880cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c96db7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c880d7010 0x7f9c880d7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c96db7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8804aef0 con 0x7f9c880cfba0 2024-09-13T18:02:02.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c96db7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880dd180 con 0x7f9c880d7010 2024-09-13T18:02:02.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c96db7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c88049310 con 0x7f9c7c004870 2024-09-13T18:02:02.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c955b4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c880cfba0 0x7f9c880cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c955b4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c880cfba0 0x7f9c880cae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42634/0 (socket says 172.21.15.40:42634) 2024-09-13T18:02:02.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c955b4700 1 -- 172.21.15.40:0/267678498 learned_addr learned my addr 172.21.15.40:0/267678498 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:02.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c95db5700 1 --2- 172.21.15.40:0/267678498 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c7c004870 0x7f9c88049df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c965b6700 1 --2- 172.21.15.40:0/267678498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c880d7010 0x7f9c880d7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c955b4700 1 -- 172.21.15.40:0/267678498 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c7c004870 msgr2=0x7f9c88049df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c955b4700 1 --2- 172.21.15.40:0/267678498 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c7c004870 0x7f9c88049df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c955b4700 1 -- 172.21.15.40:0/267678498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c880d7010 msgr2=0x7f9c880d7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c955b4700 1 --2- 172.21.15.40:0/267678498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c880d7010 0x7f9c880d7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c955b4700 1 -- 172.21.15.40:0/267678498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c880495f0 con 0x7f9c880cfba0 2024-09-13T18:02:02.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c955b4700 1 --2- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c880cfba0 0x7f9c880cae40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00c050 tx=0x7f9c800047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:02.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c867fc700 1 -- 172.21.15.40:0/267678498 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8000e6c0 con 0x7f9c880cfba0 2024-09-13T18:02:02.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c867fc700 1 -- 172.21.15.40:0/267678498 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c800038e0 con 0x7f9c880cfba0 2024-09-13T18:02:02.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c96db7700 1 -- 172.21.15.40:0/267678498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c880498d0 con 0x7f9c880cfba0 2024-09-13T18:02:02.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c867fc700 1 -- 172.21.15.40:0/267678498 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c80003c90 con 0x7f9c880cfba0 2024-09-13T18:02:02.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c96db7700 1 -- 172.21.15.40:0/267678498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c880de230 con 0x7f9c880cfba0 2024-09-13T18:02:02.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c96db7700 1 --2- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88046500 0x7f9c880d79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c96db7700 1 -- 172.21.15.40:0/267678498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8804aef0 con 0x7f9c88046500 2024-09-13T18:02:02.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.299+0000 7f9c95db5700 1 --2- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88046500 0x7f9c880d79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.303+0000 7f9c95db5700 1 -- 172.21.15.40:0/267678498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9c880d7f10 con 0x7f9c88046500 2024-09-13T18:02:02.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.303+0000 7f9c867fc700 1 -- 172.21.15.40:0/267678498 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9c80013470 con 0x7f9c880cfba0 2024-09-13T18:02:02.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.303+0000 7f9c95db5700 1 --2- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88046500 0x7f9c880d79d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c0017a0 tx=0x7f9c8c007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:02.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.303+0000 7f9c867fc700 1 --2- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9c74066730 0x7f9c74068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.303+0000 7f9c867fc700 1 -- 172.21.15.40:0/267678498 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9c80085970 con 0x7f9c880cfba0 2024-09-13T18:02:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.303+0000 7f9c965b6700 1 --2- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9c74066730 0x7f9c74068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.303+0000 7f9c867fc700 1 -- 172.21.15.40:0/267678498 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c0128c0 con 0x7f9c88046500 2024-09-13T18:02:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.303+0000 7f9c867fc700 1 -- 172.21.15.40:0/267678498 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9c8c012a20 con 0x7f9c88046500 2024-09-13T18:02:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.303+0000 7f9c867fc700 1 -- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88046500 msgr2=0x7f9c880d79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.303+0000 7f9c867fc700 1 --2- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88046500 0x7f9c880d79d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c0017a0 tx=0x7f9c8c007680 comp rx=0 tx=0).stop 2024-09-13T18:02:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.303+0000 7f9c867fc700 1 -- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88046500 msgr2=0x7f9c880d79d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.303+0000 7f9c867fc700 1 --2- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88046500 0x7f9c880d79d0 secure :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c0017a0 tx=0x7f9c8c007680 comp rx=0 tx=0).stop 2024-09-13T18:02:02.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.303+0000 7f9c965b6700 1 --2- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9c74066730 0x7f9c74068bf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9c90063cf0 tx=0x7f9c90065ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:02.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.311+0000 7f9c96db7700 1 --2- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88177b10 0x7f9c88177f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.311+0000 7f9c96db7700 1 -- 172.21.15.40:0/267678498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880095f0 con 0x7f9c88177b10 2024-09-13T18:02:02.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.311+0000 7f9c95db5700 1 --2- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88177b10 0x7f9c88177f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.311+0000 7f9c95db5700 1 -- 172.21.15.40:0/267678498 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9c88049190 con 0x7f9c88177b10 2024-09-13T18:02:02.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.311+0000 7f9c95db5700 1 --2- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88177b10 0x7f9c88177f20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c0017a0 tx=0x7f9c880091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:02.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.311+0000 7f9c867fc700 1 -- 172.21.15.40:0/267678498 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c024070 con 0x7f9c88177b10 2024-09-13T18:02:02.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.315+0000 7f9c867fc700 1 -- 172.21.15.40:0/267678498 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4949 (secure 0 0 0) 0x7f9c8c015650 con 0x7f9c88177b10 2024-09-13T18:02:02.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.315+0000 7f9c867fc700 1 -- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88177b10 msgr2=0x7f9c88177f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.315+0000 7f9c867fc700 1 --2- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88177b10 0x7f9c88177f20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c0017a0 tx=0x7f9c880091d0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.315+0000 7f9c867fc700 1 -- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88177b10 msgr2=0x7f9c88177f20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:02.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.315+0000 7f9c867fc700 1 --2- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c88177b10 0x7f9c88177f20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.325 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:02:02.325 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:02:02.325 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:02:02.325 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:02:02.325 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 88, 2024-09-13T18:02:02.325 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:02:02.325 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:02:02.325 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:02:02.325 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:02:02.325 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:02.325 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 3, 2024-09-13T18:02:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:02:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:02:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:02.326 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:02:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:02:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:02.327 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:02:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:02:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:02:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:02:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:02:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:02:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:02:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:02:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:02:02.328 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:02:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:02:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:02:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:02:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:02:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:02.329 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:02.330 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:02.330 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:02.330 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:02:02.330 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.330 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:02:02.330 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.330 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:02:02.330 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:02:02.330 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:02.330 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:02.330 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.330 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:02.330 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:02:02.331 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:02.331 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.331 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.331 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:02.331 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:02:02.331 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:02.331 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.331 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:02.331 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.331 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:02:02.331 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:02:02.331 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:02:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:02:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:02:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.332 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:02:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:02:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:02:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:02:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:02:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:02:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:02:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.334 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:02:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:02:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:02:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:02:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:02:02.335 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:02:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:02:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:02:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 8 2024-09-13T18:02:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.336 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:02:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:02:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:02:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:02:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.337 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.338 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:02:02.338 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:02:02.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.315+0000 7f9c73fff700 1 -- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9c74066730 msgr2=0x7f9c74068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.315+0000 7f9c73fff700 1 --2- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9c74066730 0x7f9c74068bf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9c90063cf0 tx=0x7f9c90065ea0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.315+0000 7f9c73fff700 1 -- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c880cfba0 msgr2=0x7f9c880cae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.315+0000 7f9c73fff700 1 --2- 172.21.15.40:0/267678498 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9c880cfba0 0x7f9c880cae40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00c050 tx=0x7f9c800047e0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.315+0000 7f9c965b6700 1 -- 172.21.15.40:0/267678498 reap_dead start 2024-09-13T18:02:02.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.315+0000 7f9c965b6700 1 -- 172.21.15.40:0/267678498 reap_dead start 2024-09-13T18:02:02.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.315+0000 7f9c73fff700 1 -- 172.21.15.40:0/267678498 shutdown_connections 2024-09-13T18:02:02.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.315+0000 7f9c73fff700 1 -- 172.21.15.40:0/267678498 >> 172.21.15.40:0/267678498 conn(0x7f9c88050e60 msgr2=0x7f9c8804d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:02.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.315+0000 7f9c73fff700 1 -- 172.21.15.40:0/267678498 shutdown_connections 2024-09-13T18:02:02.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.315+0000 7f9c73fff700 1 -- 172.21.15.40:0/267678498 wait complete. 2024-09-13T18:02:02.340 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:02:02.340 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:02:02.340 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:02:02.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.447+0000 7f523efb3700 1 Processor -- start 2024-09-13T18:02:02.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.447+0000 7f523efb3700 1 -- start start 2024-09-13T18:02:02.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523efb3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5238072380 0x7f523806a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523efb3700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5238072760 0x7f5238157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523efb3700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f523806bcb0 con 0x7f52380c6a00 2024-09-13T18:02:02.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523efb3700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52381574b0 con 0x7f5238072760 2024-09-13T18:02:02.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523efb3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52380c6f20 con 0x7f5238072380 2024-09-13T18:02:02.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523dfb1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5238072380 0x7f523806a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523d7b0700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5238072760 0x7f5238157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523dfb1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5238072380 0x7f523806a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:49478/0 (socket says 172.21.15.40:49478) 2024-09-13T18:02:02.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523d7b0700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5238072760 0x7f5238157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:46916/0 (socket says 172.21.15.40:46916) 2024-09-13T18:02:02.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523dfb1700 1 -- 172.21.15.40:0/2903606284 learned_addr learned my addr 172.21.15.40:0/2903606284 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:02.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523dfb1700 1 -- 172.21.15.40:0/2903606284 >> v2:172.21.15.43:3300/0 conn(0x7f5238072760 msgr2=0x7f5238157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:02.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523dfb1700 1 --2- 172.21.15.40:0/2903606284 >> v2:172.21.15.43:3300/0 conn(0x7f5238072760 0x7f5238157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523dfb1700 1 -- 172.21.15.40:0/2903606284 >> v1:172.21.15.43:6789/0 conn(0x7f52380c6a00 legacy=0x7f523806b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523dfb1700 1 -- 172.21.15.40:0/2903606284 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52380c7200 con 0x7f5238072380 2024-09-13T18:02:02.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523dfb1700 1 --2- 172.21.15.40:0/2903606284 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5238072380 0x7f523806a390 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5234009910 tx=0x7f5234009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1be7918b106ad31a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:02.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523cfaf700 1 -- 172.21.15.40:0/2903606284 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5234011830 con 0x7f5238072380 2024-09-13T18:02:02.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523cfaf700 1 -- 172.21.15.40:0/2903606284 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5234011990 con 0x7f5238072380 2024-09-13T18:02:02.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523cfaf700 1 -- 172.21.15.40:0/2903606284 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5234011d40 con 0x7f5238072380 2024-09-13T18:02:02.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523efb3700 1 -- 172.21.15.40:0/2903606284 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5238072380 msgr2=0x7f523806a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523efb3700 1 --2- 172.21.15.40:0/2903606284 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5238072380 0x7f523806a390 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5234009910 tx=0x7f5234009c20 comp rx=0 tx=0).stop 2024-09-13T18:02:02.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523efb3700 1 -- 172.21.15.40:0/2903606284 shutdown_connections 2024-09-13T18:02:02.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523efb3700 1 --2- 172.21.15.40:0/2903606284 >> v2:172.21.15.43:3300/0 conn(0x7f5238072760 0x7f5238157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523efb3700 1 --2- 172.21.15.40:0/2903606284 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5238072380 0x7f523806a390 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523efb3700 1 -- 172.21.15.40:0/2903606284 >> 172.21.15.40:0/2903606284 conn(0x7f52380c7ab0 msgr2=0x7f52380c7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:02.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523efb3700 1 -- 172.21.15.40:0/2903606284 shutdown_connections 2024-09-13T18:02:02.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.451+0000 7f523efb3700 1 -- 172.21.15.40:0/2903606284 wait complete. 2024-09-13T18:02:02.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523efb3700 1 Processor -- start 2024-09-13T18:02:02.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523efb3700 1 -- start start 2024-09-13T18:02:02.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523efb3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5238072380 0x7f52380be5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523efb3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5238072760 0x7f52380c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523efb3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52380c6a00 0x7f52381508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523efb3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52380711a0 con 0x7f5238072760 2024-09-13T18:02:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523efb3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5238143200 con 0x7f5238072380 2024-09-13T18:02:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523efb3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52381464b0 con 0x7f52380c6a00 2024-09-13T18:02:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523d7b0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5238072760 0x7f52380c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523d7b0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5238072760 0x7f52380c9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42668/0 (socket says 172.21.15.40:42668) 2024-09-13T18:02:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523d7b0700 1 -- 172.21.15.40:0/3152046198 learned_addr learned my addr 172.21.15.40:0/3152046198 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523e7b2700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52380c6a00 0x7f52381508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523dfb1700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5238072380 0x7f52380be5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523d7b0700 1 -- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52380c6a00 msgr2=0x7f52381508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523d7b0700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52380c6a00 0x7f52381508b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523d7b0700 1 -- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5238072380 msgr2=0x7f52380be5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523d7b0700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5238072380 0x7f52380be5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523d7b0700 1 -- 172.21.15.40:0/3152046198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5234009570 con 0x7f5238072760 2024-09-13T18:02:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523dfb1700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5238072380 0x7f52380be5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:02:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523d7b0700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5238072760 0x7f52380c9ad0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5228012b60 tx=0x7f5228012e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f522e7fc700 1 -- 172.21.15.40:0/3152046198 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522800da40 con 0x7f5238072760 2024-09-13T18:02:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523efb3700 1 -- 172.21.15.40:0/3152046198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5238070130 con 0x7f5238072760 2024-09-13T18:02:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f522e7fc700 1 -- 172.21.15.40:0/3152046198 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5228013040 con 0x7f5238072760 2024-09-13T18:02:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523efb3700 1 -- 172.21.15.40:0/3152046198 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52381469e0 con 0x7f5238072760 2024-09-13T18:02:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f522e7fc700 1 -- 172.21.15.40:0/3152046198 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228010a70 con 0x7f5238072760 2024-09-13T18:02:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523efb3700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5238150df0 0x7f5238146c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523efb3700 1 -- 172.21.15.40:0/3152046198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5238073eb0 con 0x7f5238150df0 2024-09-13T18:02:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.455+0000 7f523dfb1700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5238150df0 0x7f5238146c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.459+0000 7f523dfb1700 1 -- 172.21.15.40:0/3152046198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5238152020 con 0x7f5238150df0 2024-09-13T18:02:02.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.459+0000 7f523dfb1700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5238150df0 0x7f5238146c00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5234009910 tx=0x7f5234004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.459+0000 7f522e7fc700 1 -- 172.21.15.40:0/3152046198 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5228010bd0 con 0x7f5238072760 2024-09-13T18:02:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.459+0000 7f522e7fc700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f521c0666d0 0x7f521c068b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.459+0000 7f522e7fc700 1 -- 172.21.15.40:0/3152046198 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5228088780 con 0x7f5238072760 2024-09-13T18:02:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.459+0000 7f523e7b2700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f521c0666d0 0x7f521c068b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.459+0000 7f522e7fc700 1 -- 172.21.15.40:0/3152046198 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5234004550 con 0x7f5238150df0 2024-09-13T18:02:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.459+0000 7f522e7fc700 1 -- 172.21.15.40:0/3152046198 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f52340036c0 con 0x7f5238150df0 2024-09-13T18:02:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.459+0000 7f522e7fc700 1 -- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5238150df0 msgr2=0x7f5238146c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.459+0000 7f522e7fc700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5238150df0 0x7f5238146c00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5234009910 tx=0x7f5234004770 comp rx=0 tx=0).stop 2024-09-13T18:02:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.459+0000 7f522e7fc700 1 -- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5238150df0 msgr2=0x7f5238146c00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.459+0000 7f522e7fc700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5238150df0 0x7f5238146c00 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5234009910 tx=0x7f5234004770 comp rx=0 tx=0).stop 2024-09-13T18:02:02.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.459+0000 7f523e7b2700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f521c0666d0 0x7f521c068b90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f52300012e0 tx=0x7f5230009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:02.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f523efb3700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52381eecd0 0x7f523802b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f523efb3700 1 -- 172.21.15.40:0/3152046198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52381511d0 con 0x7f52381eecd0 2024-09-13T18:02:02.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f523dfb1700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52381eecd0 0x7f523802b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f523dfb1700 1 -- 172.21.15.40:0/3152046198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f52380c3e00 con 0x7f52381eecd0 2024-09-13T18:02:02.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f523dfb1700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52381eecd0 0x7f523802b6a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5234009150 tx=0x7f5234011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:02.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f522e7fc700 1 -- 172.21.15.40:0/3152046198 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523401ac00 con 0x7f52381eecd0 2024-09-13T18:02:02.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f522e7fc700 1 -- 172.21.15.40:0/3152046198 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4449 (secure 0 0 0) 0x7f523402c680 con 0x7f52381eecd0 2024-09-13T18:02:02.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f522e7fc700 1 -- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52381eecd0 msgr2=0x7f523802b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f522e7fc700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52381eecd0 0x7f523802b6a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5234009150 tx=0x7f5234011000 comp rx=0 tx=0).stop 2024-09-13T18:02:02.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f522e7fc700 1 -- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52381eecd0 msgr2=0x7f523802b6a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:02.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f522e7fc700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52381eecd0 0x7f523802b6a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.482 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:02:02.482 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:02:02.482 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:02:02.482 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:02:02.482 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 88, 2024-09-13T18:02:02.482 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:02:02.482 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:02:02.482 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:02:02.482 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:02:02.482 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:02.482 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 3, 2024-09-13T18:02:02.483 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:02.483 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:02:02.483 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:02:02.483 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:02.483 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:02.483 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:02.483 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:02.483 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:02.483 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:02.483 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:02.483 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:02.483 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:02.483 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:02.483 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:02.484 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:02:02.484 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:02:02.484 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:02.484 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:02.484 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:02.484 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:02.484 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:02.484 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:02.484 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:02.484 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:02.484 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:02.484 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:02.484 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:02.485 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.485 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:02:02.485 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:02:02.485 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:02:02.485 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:02:02.485 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:02:02.485 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:02:02.485 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:02.485 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:02.485 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:02:02.485 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:02:02.485 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:02:02.485 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:02:02.485 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:02:02.486 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:02:02.486 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:02:02.486 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:02.486 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:02:02.486 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:02.486 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:02.486 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:02.486 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:02.486 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:02.486 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:02.486 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:02.486 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:02.486 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:02.487 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:02.487 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:02.487 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:02:02.487 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.487 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:02:02.487 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.487 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:02:02.487 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:02:02.487 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:02.487 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:02.487 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.487 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:02.487 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:02:02.488 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:02.488 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.488 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.488 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:02.488 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:02:02.488 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:02.488 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.488 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:02.488 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.488 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:02:02.488 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:02:02.488 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:02.488 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:02.488 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:02.489 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.489 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.489 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:02:02.489 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:02:02.489 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:02.489 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:02.489 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.489 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:02.489 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:02:02.489 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:02.489 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.489 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.489 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:02.490 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:02:02.490 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:02.490 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.490 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:02.490 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.490 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:02:02.490 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:02:02.490 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:02.490 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:02.490 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:02.490 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.490 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.491 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:02:02.491 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:02:02.491 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:02.491 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:02.491 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.491 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:02.491 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:02:02.491 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:02.491 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.491 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.491 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:02.491 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:02:02.491 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:02.491 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.492 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:02.492 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.492 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:02:02.492 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:02:02.492 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:02.492 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:02.492 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:02.492 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.492 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:02.492 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.492 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:02:02.492 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:02:02.492 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.493 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:02.493 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:02.493 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:02:02.493 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.493 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:02.493 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:02:02.493 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.493 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:02.493 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:02.493 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:02:02.493 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.493 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:02.493 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.493 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:02:02.494 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:02:02.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f523efb3700 1 -- 172.21.15.40:0/3152046198 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f521c0666d0 msgr2=0x7f521c068b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f523efb3700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f521c0666d0 0x7f521c068b90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f52300012e0 tx=0x7f5230009040 comp rx=0 tx=0).stop 2024-09-13T18:02:02.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f523efb3700 1 -- 172.21.15.40:0/3152046198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5238072760 msgr2=0x7f52380c9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f523e7b2700 1 -- 172.21.15.40:0/3152046198 reap_dead start 2024-09-13T18:02:02.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f523efb3700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5238072760 0x7f52380c9ad0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5228012b60 tx=0x7f5228012e70 comp rx=0 tx=0).stop 2024-09-13T18:02:02.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f523efb3700 1 -- 172.21.15.40:0/3152046198 shutdown_connections 2024-09-13T18:02:02.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f523efb3700 1 --2- 172.21.15.40:0/3152046198 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5238072760 0x7f52380c9ad0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f523efb3700 1 -- 172.21.15.40:0/3152046198 >> 172.21.15.40:0/3152046198 conn(0x7f52380c7ab0 msgr2=0x7f52380c4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:02.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f523efb3700 1 -- 172.21.15.40:0/3152046198 shutdown_connections 2024-09-13T18:02:02.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.471+0000 7f523efb3700 1 -- 172.21.15.40:0/3152046198 wait complete. 2024-09-13T18:02:02.496 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:02:02.496 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:02:02.496 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:02:02.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.595+0000 7f5955c42700 1 Processor -- start 2024-09-13T18:02:02.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.595+0000 7f5955c42700 1 -- start start 2024-09-13T18:02:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.595+0000 7f5955c42700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f59500c6ef0 0x7f595006d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.595+0000 7f5955c42700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f595015d1f0 0x7f59501533f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.595+0000 7f5955c42700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5950071860 con 0x7f595015ce10 2024-09-13T18:02:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.595+0000 7f5955c42700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59500c7320 con 0x7f595015d1f0 2024-09-13T18:02:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.595+0000 7f5955c42700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59500c5e50 con 0x7f59500c6ef0 2024-09-13T18:02:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.595+0000 7f5955441700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f595015d1f0 0x7f59501533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.595+0000 7f5955441700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f595015d1f0 0x7f59501533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42682/0 (socket says 172.21.15.40:42682) 2024-09-13T18:02:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.595+0000 7f5955441700 1 -- 172.21.15.40:0/1572170352 learned_addr learned my addr 172.21.15.40:0/1572170352 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:02.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.595+0000 7f594ffff700 1 --2- 172.21.15.40:0/1572170352 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f59500c6ef0 0x7f595006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.599+0000 7f5955441700 1 -- 172.21.15.40:0/1572170352 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f59500c6ef0 msgr2=0x7f595006d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:02.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.599+0000 7f5955441700 1 --2- 172.21.15.40:0/1572170352 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f59500c6ef0 0x7f595006d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.599+0000 7f5955441700 1 -- 172.21.15.40:0/1572170352 >> v1:172.21.15.40:6789/0 conn(0x7f595015ce10 legacy=0x7f5950070fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.599+0000 7f5955441700 1 -- 172.21.15.40:0/1572170352 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59500c6130 con 0x7f595015d1f0 2024-09-13T18:02:02.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.599+0000 7f594ffff700 1 --2- 172.21.15.40:0/1572170352 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f59500c6ef0 0x7f595006d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:02:02.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.599+0000 7f5955441700 1 --2- 172.21.15.40:0/1572170352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f595015d1f0 0x7f59501533f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5948009e50 tx=0x7f594800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2634b1848212c84a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:02.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.599+0000 7f594f7fe700 1 -- 172.21.15.40:0/1572170352 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594800e070 con 0x7f595015d1f0 2024-09-13T18:02:02.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.599+0000 7f594f7fe700 1 -- 172.21.15.40:0/1572170352 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594800b9c0 con 0x7f595015d1f0 2024-09-13T18:02:02.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.599+0000 7f594f7fe700 1 -- 172.21.15.40:0/1572170352 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594800bd70 con 0x7f595015d1f0 2024-09-13T18:02:02.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.599+0000 7f5955c42700 1 -- 172.21.15.40:0/1572170352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f595015d1f0 msgr2=0x7f59501533f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.599+0000 7f5955c42700 1 --2- 172.21.15.40:0/1572170352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f595015d1f0 0x7f59501533f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5948009e50 tx=0x7f594800b330 comp rx=0 tx=0).stop 2024-09-13T18:02:02.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.599+0000 7f5955c42700 1 -- 172.21.15.40:0/1572170352 shutdown_connections 2024-09-13T18:02:02.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.599+0000 7f5955c42700 1 --2- 172.21.15.40:0/1572170352 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f595015d1f0 0x7f59501533f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.599+0000 7f5955c42700 1 --2- 172.21.15.40:0/1572170352 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f59500c6ef0 0x7f595006d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.599+0000 7f5955c42700 1 -- 172.21.15.40:0/1572170352 >> 172.21.15.40:0/1572170352 conn(0x7f59500cb450 msgr2=0x7f59500cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:02.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.599+0000 7f5955c42700 1 -- 172.21.15.40:0/1572170352 shutdown_connections 2024-09-13T18:02:02.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.599+0000 7f5955c42700 1 -- 172.21.15.40:0/1572170352 wait complete. 2024-09-13T18:02:02.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.599+0000 7f5955c42700 1 Processor -- start 2024-09-13T18:02:02.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f5955c42700 1 -- start start 2024-09-13T18:02:02.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f5955c42700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f59500c6ef0 0x7f59500c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f5955c42700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f595015ce10 0x7f59501ef780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f5955c42700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595015d1f0 0x7f5950148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f5955c42700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5950157060 con 0x7f595015ce10 2024-09-13T18:02:02.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f5955c42700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595015b760 con 0x7f595015d1f0 2024-09-13T18:02:02.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f5955c42700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59501eefa0 con 0x7f59500c6ef0 2024-09-13T18:02:02.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f594ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f595015ce10 0x7f59501ef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f594ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f595015ce10 0x7f59501ef780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42694/0 (socket says 172.21.15.40:42694) 2024-09-13T18:02:02.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f594ffff700 1 -- 172.21.15.40:0/3388849341 learned_addr learned my addr 172.21.15.40:0/3388849341 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:02.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f5954c40700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f59500c6ef0 0x7f59500c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f5955441700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595015d1f0 0x7f5950148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f594ffff700 1 -- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f59500c6ef0 msgr2=0x7f59500c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f594ffff700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f59500c6ef0 0x7f59500c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f594ffff700 1 -- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595015d1f0 msgr2=0x7f5950148c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f594ffff700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595015d1f0 0x7f5950148c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f594ffff700 1 -- 172.21.15.40:0/3388849341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5948009b30 con 0x7f595015ce10 2024-09-13T18:02:02.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f5954c40700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f59500c6ef0 0x7f59500c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:02:02.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f594ffff700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f595015ce10 0x7f59501ef780 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f593c00e810 tx=0x7f593c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:02.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f594d7fa700 1 -- 172.21.15.40:0/3388849341 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c004230 con 0x7f595015ce10 2024-09-13T18:02:02.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f594d7fa700 1 -- 172.21.15.40:0/3388849341 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593c004390 con 0x7f595015ce10 2024-09-13T18:02:02.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f5955c42700 1 -- 172.21.15.40:0/3388849341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59501f2a00 con 0x7f595015ce10 2024-09-13T18:02:02.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f5955c42700 1 -- 172.21.15.40:0/3388849341 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59501f2fc0 con 0x7f595015ce10 2024-09-13T18:02:02.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.603+0000 7f594d7fa700 1 -- 172.21.15.40:0/3388849341 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c0036c0 con 0x7f595015ce10 2024-09-13T18:02:02.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.607+0000 7f5955c42700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5930002a40 0x7f5930004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.607+0000 7f5955c42700 1 -- 172.21.15.40:0/3388849341 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5930007f40 con 0x7f5930002a40 2024-09-13T18:02:02.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.607+0000 7f594d7fa700 1 -- 172.21.15.40:0/3388849341 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f593c003880 con 0x7f595015ce10 2024-09-13T18:02:02.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.607+0000 7f5954c40700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5930002a40 0x7f5930004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.607+0000 7f594d7fa700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5934066800 0x7f5934068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.607+0000 7f5955441700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5934066800 0x7f5934068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.607+0000 7f594d7fa700 1 -- 172.21.15.40:0/3388849341 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f593c083fa0 con 0x7f595015ce10 2024-09-13T18:02:02.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.607+0000 7f5954c40700 1 -- 172.21.15.40:0/3388849341 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5930008100 con 0x7f5930002a40 2024-09-13T18:02:02.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.607+0000 7f5955441700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5934066800 0x7f5934068cc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f594800b370 tx=0x7f5948009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:02.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.607+0000 7f5954c40700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5930002a40 0x7f5930004f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f594400e700 tx=0x7f594400eac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:02.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.611+0000 7f594d7fa700 1 -- 172.21.15.40:0/3388849341 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59440052e0 con 0x7f5930002a40 2024-09-13T18:02:02.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.611+0000 7f594d7fa700 1 -- 172.21.15.40:0/3388849341 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5944005440 con 0x7f5930002a40 2024-09-13T18:02:02.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.611+0000 7f594d7fa700 1 -- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5930002a40 msgr2=0x7f5930004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.611+0000 7f594d7fa700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5930002a40 0x7f5930004f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f594400e700 tx=0x7f594400eac0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.611+0000 7f594d7fa700 1 -- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5930002a40 msgr2=0x7f5930004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:02.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.611+0000 7f594d7fa700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5930002a40 0x7f5930004f00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.619+0000 7f5955c42700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5930006fd0 0x7f5930007440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.619+0000 7f5955c42700 1 -- 172.21.15.40:0/3388849341 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593000a400 con 0x7f5930006fd0 2024-09-13T18:02:02.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.619+0000 7f5954c40700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5930006fd0 0x7f5930007440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.619+0000 7f5954c40700 1 -- 172.21.15.40:0/3388849341 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f593000a5c0 con 0x7f5930006fd0 2024-09-13T18:02:02.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.619+0000 7f5954c40700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5930006fd0 0x7f5930007440 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f594400e700 tx=0x7f5944000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:02.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.619+0000 7f594d7fa700 1 -- 172.21.15.40:0/3388849341 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5944021070 con 0x7f5930006fd0 2024-09-13T18:02:02.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.619+0000 7f594d7fa700 1 -- 172.21.15.40:0/3388849341 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4615 (secure 0 0 0) 0x7f5944006e80 con 0x7f5930006fd0 2024-09-13T18:02:02.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.619+0000 7f594d7fa700 1 -- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5930006fd0 msgr2=0x7f5930007440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.619+0000 7f594d7fa700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5930006fd0 0x7f5930007440 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f594400e700 tx=0x7f5944000ed0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.619+0000 7f594d7fa700 1 -- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5930006fd0 msgr2=0x7f5930007440 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:02.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.619+0000 7f594d7fa700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5930006fd0 0x7f5930007440 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.631 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:02:02.631 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:02:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:02:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:02:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 88, 2024-09-13T18:02:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:02:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:02:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:02:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:02:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 3, 2024-09-13T18:02:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:02:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:02:02.632 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:02:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:02:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:02.633 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:02:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:02:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:02:02.634 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:02:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:02:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:02:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:02:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:02:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:02:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:02:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:02:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:02:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:02:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:02:02.635 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:02:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.636 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:02:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:02:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:02:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:02:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:02:02.637 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:02:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:02:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:02:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:02:02.638 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:02:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:02:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:02.639 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:02:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:02:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:02:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:02:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:02.640 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:02:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:02:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:02:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:02:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:02.641 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:02:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:02:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:02:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:02:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.642 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:02:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:02:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:02:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:02.643 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:02:02.643 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:02:02.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.619+0000 7f5955c42700 1 -- 172.21.15.40:0/3388849341 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5934066800 msgr2=0x7f5934068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.619+0000 7f5955c42700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5934066800 0x7f5934068cc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f594800b370 tx=0x7f5948009000 comp rx=0 tx=0).stop 2024-09-13T18:02:02.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.619+0000 7f5955c42700 1 -- 172.21.15.40:0/3388849341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f595015ce10 msgr2=0x7f59501ef780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.619+0000 7f5955c42700 1 --2- 172.21.15.40:0/3388849341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f595015ce10 0x7f59501ef780 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f593c00e810 tx=0x7f593c00ebd0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.619+0000 7f5955441700 1 -- 172.21.15.40:0/3388849341 reap_dead start 2024-09-13T18:02:02.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.619+0000 7f5955441700 1 -- 172.21.15.40:0/3388849341 reap_dead start 2024-09-13T18:02:02.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.623+0000 7f5955c42700 1 -- 172.21.15.40:0/3388849341 shutdown_connections 2024-09-13T18:02:02.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.623+0000 7f5955c42700 1 -- 172.21.15.40:0/3388849341 >> 172.21.15.40:0/3388849341 conn(0x7f59500cb450 msgr2=0x7f5950071000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:02.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.623+0000 7f5955c42700 1 -- 172.21.15.40:0/3388849341 shutdown_connections 2024-09-13T18:02:02.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.623+0000 7f5955c42700 1 -- 172.21.15.40:0/3388849341 wait complete. 2024-09-13T18:02:02.646 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:02:02.646 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:02:02.646 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:02:02.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f4788f4b700 1 Processor -- start 2024-09-13T18:02:02.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f4788f4b700 1 -- start start 2024-09-13T18:02:02.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f4788f4b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f47840c69a0 0x7f4784156f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f4788f4b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47840c6d80 0x7f4784155a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f4788f4b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f47840cb520 0x7f4784155f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f4788f4b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4784156650 con 0x7f47840cb520 2024-09-13T18:02:02.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f4788f4b700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47841590e0 con 0x7f47840c69a0 2024-09-13T18:02:02.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f4788f4b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47841593c0 con 0x7f47840c6d80 2024-09-13T18:02:02.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f4782d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f47840cb520 0x7f4784155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f4782d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f47840cb520 0x7f4784155f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42700/0 (socket says 172.21.15.40:42700) 2024-09-13T18:02:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f4782d9d700 1 -- 172.21.15.40:0/1852962236 learned_addr learned my addr 172.21.15.40:0/1852962236 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f478259c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f47840c69a0 0x7f4784156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f4781d9b700 1 --2- 172.21.15.40:0/1852962236 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47840c6d80 0x7f4784155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f4782d9d700 1 -- 172.21.15.40:0/1852962236 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47840c6d80 msgr2=0x7f4784155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f4782d9d700 1 --2- 172.21.15.40:0/1852962236 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47840c6d80 0x7f4784155a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f4782d9d700 1 -- 172.21.15.40:0/1852962236 >> v2:172.21.15.43:3300/0 conn(0x7f47840c69a0 msgr2=0x7f4784156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f4782d9d700 1 --2- 172.21.15.40:0/1852962236 >> v2:172.21.15.43:3300/0 conn(0x7f47840c69a0 0x7f4784156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f4782d9d700 1 -- 172.21.15.40:0/1852962236 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47841596a0 con 0x7f47840cb520 2024-09-13T18:02:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f4782d9d700 1 --2- 172.21.15.40:0/1852962236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47840cb520 0x7f4784155f90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f478406ef00 tx=0x7f477c014b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd393010f6236733 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f478159a700 1 -- 172.21.15.40:0/1852962236 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c00b2e0 con 0x7f47840cb520 2024-09-13T18:02:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f478159a700 1 -- 172.21.15.40:0/1852962236 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477c00b440 con 0x7f47840cb520 2024-09-13T18:02:02.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.751+0000 7f478159a700 1 -- 172.21.15.40:0/1852962236 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c009840 con 0x7f47840cb520 2024-09-13T18:02:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.755+0000 7f4788f4b700 1 -- 172.21.15.40:0/1852962236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47840cb520 msgr2=0x7f4784155f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.755+0000 7f4788f4b700 1 --2- 172.21.15.40:0/1852962236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47840cb520 0x7f4784155f90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f478406ef00 tx=0x7f477c014b30 comp rx=0 tx=0).stop 2024-09-13T18:02:02.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.755+0000 7f4788f4b700 1 -- 172.21.15.40:0/1852962236 shutdown_connections 2024-09-13T18:02:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.755+0000 7f4788f4b700 1 --2- 172.21.15.40:0/1852962236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47840cb520 0x7f4784155f90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.755+0000 7f4788f4b700 1 --2- 172.21.15.40:0/1852962236 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47840c6d80 0x7f4784155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.755+0000 7f4788f4b700 1 --2- 172.21.15.40:0/1852962236 >> v2:172.21.15.43:3300/0 conn(0x7f47840c69a0 0x7f4784156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.755+0000 7f4788f4b700 1 -- 172.21.15.40:0/1852962236 >> 172.21.15.40:0/1852962236 conn(0x7f478406a390 msgr2=0x7f478406a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.755+0000 7f4788f4b700 1 -- 172.21.15.40:0/1852962236 shutdown_connections 2024-09-13T18:02:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.755+0000 7f4788f4b700 1 -- 172.21.15.40:0/1852962236 wait complete. 2024-09-13T18:02:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.755+0000 7f4788f4b700 1 Processor -- start 2024-09-13T18:02:02.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.755+0000 7f4788f4b700 1 -- start start 2024-09-13T18:02:02.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.755+0000 7f4788f4b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47840c69a0 0x7f478414de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.755+0000 7f4788f4b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47840c6d80 0x7f4784144250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.755+0000 7f4788f4b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47840cb520 0x7f4784141c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.755+0000 7f4788f4b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47840714a0 con 0x7f47840c6d80 2024-09-13T18:02:02.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.755+0000 7f4788f4b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47841ede60 con 0x7f47840cb520 2024-09-13T18:02:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.755+0000 7f4788f4b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47840c43e0 con 0x7f47840c69a0 2024-09-13T18:02:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.755+0000 7f4781d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47840c6d80 0x7f4784144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.755+0000 7f4781d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47840c6d80 0x7f4784144250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:42704/0 (socket says 172.21.15.40:42704) 2024-09-13T18:02:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.755+0000 7f4781d9b700 1 -- 172.21.15.40:0/1748861866 learned_addr learned my addr 172.21.15.40:0/1748861866 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.759+0000 7f478259c700 1 --2- 172.21.15.40:0/1748861866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47840c69a0 0x7f478414de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.759+0000 7f4782d9d700 1 --2- 172.21.15.40:0/1748861866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47840cb520 0x7f4784141c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.759+0000 7f4781d9b700 1 -- 172.21.15.40:0/1748861866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47840c69a0 msgr2=0x7f478414de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.759+0000 7f4781d9b700 1 --2- 172.21.15.40:0/1748861866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47840c69a0 0x7f478414de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.759+0000 7f4781d9b700 1 -- 172.21.15.40:0/1748861866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47840cb520 msgr2=0x7f4784141c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:02.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.759+0000 7f4781d9b700 1 --2- 172.21.15.40:0/1748861866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47840cb520 0x7f4784141c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:02.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.759+0000 7f4781d9b700 1 -- 172.21.15.40:0/1748861866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c00bb50 con 0x7f47840c6d80 2024-09-13T18:02:02.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.759+0000 7f4781d9b700 1 --2- 172.21.15.40:0/1748861866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47840c6d80 0x7f4784144250 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f477800ec10 tx=0x7f4778009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:02.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.759+0000 7f4772ffd700 1 -- 172.21.15.40:0/1748861866 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778009740 con 0x7f47840c6d80 2024-09-13T18:02:02.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.759+0000 7f4772ffd700 1 -- 172.21.15.40:0/1748861866 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477800c8e0 con 0x7f47840c6d80 2024-09-13T18:02:02.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.759+0000 7f4788f4b700 1 -- 172.21.15.40:0/1748861866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4784070310 con 0x7f47840c6d80 2024-09-13T18:02:02.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.759+0000 7f4772ffd700 1 -- 172.21.15.40:0/1748861866 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477800cc70 con 0x7f47840c6d80 2024-09-13T18:02:02.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.759+0000 7f4788f4b700 1 -- 172.21.15.40:0/1748861866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47840708d0 con 0x7f47840c6d80 2024-09-13T18:02:02.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.759+0000 7f4788f4b700 1 -- 172.21.15.40:0/1748861866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4784070af0 con 0x7f47840c6d80 2024-09-13T18:02:02.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.759+0000 7f4772ffd700 1 -- 172.21.15.40:0/1748861866 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4778013470 con 0x7f47840c6d80 2024-09-13T18:02:02.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.759+0000 7f4772ffd700 1 --2- 172.21.15.40:0/1748861866 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4764066730 0x7f4764068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:02.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.759+0000 7f478259c700 1 --2- 172.21.15.40:0/1748861866 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4764066730 0x7f4764068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:02.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.759+0000 7f4772ffd700 1 -- 172.21.15.40:0/1748861866 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4778084680 con 0x7f47840c6d80 2024-09-13T18:02:02.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.763+0000 7f478259c700 1 --2- 172.21.15.40:0/1748861866 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4764066730 0x7f4764068bf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f477400dd00 tx=0x7f477400f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:02.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:02.767+0000 7f4772ffd700 1 -- 172.21.15.40:0/1748861866 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4778051630 con 0x7f47840c6d80 2024-09-13T18:02:03.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:03.011+0000 7f4788f4b700 1 -- 172.21.15.40:0/1748861866 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f4784156f40 con 0x7f47840c6d80 2024-09-13T18:02:03.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:03.015+0000 7f4772ffd700 1 -- 172.21.15.40:0/1748861866 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f4778054c50 con 0x7f47840c6d80 2024-09-13T18:02:03.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:03.015+0000 7f4788f4b700 1 -- 172.21.15.40:0/1748861866 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4764066730 msgr2=0x7f4764068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:03.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:03.015+0000 7f4788f4b700 1 --2- 172.21.15.40:0/1748861866 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4764066730 0x7f4764068bf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f477400dd00 tx=0x7f477400f040 comp rx=0 tx=0).stop 2024-09-13T18:02:03.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:03.015+0000 7f4788f4b700 1 -- 172.21.15.40:0/1748861866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47840c6d80 msgr2=0x7f4784144250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:03.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:03.015+0000 7f4788f4b700 1 --2- 172.21.15.40:0/1748861866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47840c6d80 0x7f4784144250 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f477800ec10 tx=0x7f4778009e00 comp rx=0 tx=0).stop 2024-09-13T18:02:03.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:03.015+0000 7f4788f4b700 1 -- 172.21.15.40:0/1748861866 shutdown_connections 2024-09-13T18:02:03.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:03.015+0000 7f4788f4b700 1 --2- 172.21.15.40:0/1748861866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47840cb520 0x7f4784141c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:03.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:03.015+0000 7f4788f4b700 1 --2- 172.21.15.40:0/1748861866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47840c6d80 0x7f4784144250 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:03.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:03.015+0000 7f4788f4b700 1 --2- 172.21.15.40:0/1748861866 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4764066730 0x7f4764068bf0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:03.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:03.015+0000 7f4788f4b700 1 --2- 172.21.15.40:0/1748861866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f47840c69a0 0x7f478414de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:03.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:03.015+0000 7f4788f4b700 1 -- 172.21.15.40:0/1748861866 >> 172.21.15.40:0/1748861866 conn(0x7f478406a390 msgr2=0x7f478406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:03.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:03.015+0000 7f4788f4b700 1 -- 172.21.15.40:0/1748861866 shutdown_connections 2024-09-13T18:02:03.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:03.015+0000 7f4788f4b700 1 -- 172.21.15.40:0/1748861866 wait complete. 2024-09-13T18:02:03.038 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:02:05.839 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:02:05.842 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:02:13.039 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:02:13.039 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:02:13.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.151+0000 7fbacac52700 1 Processor -- start 2024-09-13T18:02:13.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.151+0000 7fbacac52700 1 -- start start 2024-09-13T18:02:13.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.151+0000 7fbacac52700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbac4153880 0x7fbac40c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.151+0000 7fbacac52700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbac40cb2f0 0x7fbac40cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.151+0000 7fbacac52700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac4159770 con 0x7fbac40cf3c0 2024-09-13T18:02:13.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.151+0000 7fbacac52700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbac4159a50 con 0x7fbac40cb2f0 2024-09-13T18:02:13.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.151+0000 7fbacac52700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac4159d30 con 0x7fbac4153880 2024-09-13T18:02:13.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.151+0000 7fbac944f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbac40cb2f0 0x7fbac40cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.151+0000 7fbac944f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbac40cb2f0 0x7fbac40cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52072/0 (socket says 172.21.15.40:52072) 2024-09-13T18:02:13.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.151+0000 7fbac944f700 1 -- 172.21.15.40:0/4138751071 learned_addr learned my addr 172.21.15.40:0/4138751071 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:13.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.151+0000 7fbac9c50700 1 --2- 172.21.15.40:0/4138751071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbac4153880 0x7fbac40c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.151+0000 7fbac944f700 1 -- 172.21.15.40:0/4138751071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbac4153880 msgr2=0x7fbac40c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:13.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.151+0000 7fbac944f700 1 --2- 172.21.15.40:0/4138751071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbac4153880 0x7fbac40c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.151+0000 7fbac944f700 1 -- 172.21.15.40:0/4138751071 >> v1:172.21.15.43:6789/0 conn(0x7fbac40cf3c0 legacy=0x7fbac415cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.151+0000 7fbac944f700 1 -- 172.21.15.40:0/4138751071 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac415a010 con 0x7fbac40cb2f0 2024-09-13T18:02:13.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.151+0000 7fbac9c50700 1 --2- 172.21.15.40:0/4138751071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbac4153880 0x7fbac40c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:02:13.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.155+0000 7fbac944f700 1 --2- 172.21.15.40:0/4138751071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac40cb2f0 0x7fbac40cee80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbab4009910 tx=0x7fbab4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de797ce07a09b76 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:13.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.155+0000 7fbac8c4e700 1 -- 172.21.15.40:0/4138751071 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab400ba60 con 0x7fbac40cb2f0 2024-09-13T18:02:13.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.155+0000 7fbac8c4e700 1 -- 172.21.15.40:0/4138751071 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab400bbc0 con 0x7fbac40cb2f0 2024-09-13T18:02:13.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.155+0000 7fbac8c4e700 1 -- 172.21.15.40:0/4138751071 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab4009e60 con 0x7fbac40cb2f0 2024-09-13T18:02:13.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.155+0000 7fbacac52700 1 -- 172.21.15.40:0/4138751071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac40cb2f0 msgr2=0x7fbac40cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.155+0000 7fbacac52700 1 --2- 172.21.15.40:0/4138751071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac40cb2f0 0x7fbac40cee80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbab4009910 tx=0x7fbab4009c20 comp rx=0 tx=0).stop 2024-09-13T18:02:13.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.155+0000 7fbacac52700 1 -- 172.21.15.40:0/4138751071 shutdown_connections 2024-09-13T18:02:13.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.155+0000 7fbacac52700 1 --2- 172.21.15.40:0/4138751071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac40cb2f0 0x7fbac40cee80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.155+0000 7fbacac52700 1 --2- 172.21.15.40:0/4138751071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbac4153880 0x7fbac40c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.155+0000 7fbacac52700 1 -- 172.21.15.40:0/4138751071 >> 172.21.15.40:0/4138751071 conn(0x7fbac406c5b0 msgr2=0x7fbac406c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:13.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.155+0000 7fbacac52700 1 -- 172.21.15.40:0/4138751071 shutdown_connections 2024-09-13T18:02:13.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.155+0000 7fbacac52700 1 -- 172.21.15.40:0/4138751071 wait complete. 2024-09-13T18:02:13.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.155+0000 7fbacac52700 1 Processor -- start 2024-09-13T18:02:13.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.155+0000 7fbacac52700 1 -- start start 2024-09-13T18:02:13.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbacac52700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbac40cb2f0 0x7fbac40c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbacac52700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbac40cf3c0 0x7fbac414bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbacac52700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac4153880 0x7fbac41f2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbacac52700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac40c80c0 con 0x7fbac4153880 2024-09-13T18:02:13.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbacac52700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac414a600 con 0x7fbac40cb2f0 2024-09-13T18:02:13.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbacac52700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac414c1c0 con 0x7fbac40cf3c0 2024-09-13T18:02:13.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbaca451700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac4153880 0x7fbac41f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbaca451700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac4153880 0x7fbac41f2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52074/0 (socket says 172.21.15.40:52074) 2024-09-13T18:02:13.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbaca451700 1 -- 172.21.15.40:0/1793202366 learned_addr learned my addr 172.21.15.40:0/1793202366 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:13.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbac9c50700 1 --2- 172.21.15.40:0/1793202366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbac40cb2f0 0x7fbac40c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbac944f700 1 --2- 172.21.15.40:0/1793202366 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbac40cf3c0 0x7fbac414bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbaca451700 1 -- 172.21.15.40:0/1793202366 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbac40cf3c0 msgr2=0x7fbac414bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbaca451700 1 --2- 172.21.15.40:0/1793202366 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbac40cf3c0 0x7fbac414bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbaca451700 1 -- 172.21.15.40:0/1793202366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbac40cb2f0 msgr2=0x7fbac40c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbaca451700 1 --2- 172.21.15.40:0/1793202366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbac40cb2f0 0x7fbac40c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbaca451700 1 -- 172.21.15.40:0/1793202366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab4009570 con 0x7fbac4153880 2024-09-13T18:02:13.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbaca451700 1 --2- 172.21.15.40:0/1793202366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac4153880 0x7fbac41f2340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00cc50 tx=0x7fbabc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:13.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbaba7fc700 1 -- 172.21.15.40:0/1793202366 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc003c20 con 0x7fbac4153880 2024-09-13T18:02:13.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbacac52700 1 -- 172.21.15.40:0/1793202366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac414b350 con 0x7fbac4153880 2024-09-13T18:02:13.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbaba7fc700 1 -- 172.21.15.40:0/1793202366 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbabc003d80 con 0x7fbac4153880 2024-09-13T18:02:13.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbacac52700 1 -- 172.21.15.40:0/1793202366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac414a870 con 0x7fbac4153880 2024-09-13T18:02:13.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbaba7fc700 1 -- 172.21.15.40:0/1793202366 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc00aa70 con 0x7fbac4153880 2024-09-13T18:02:13.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.159+0000 7fbacac52700 1 -- 172.21.15.40:0/1793202366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac40cee80 con 0x7fbac4153880 2024-09-13T18:02:13.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.163+0000 7fbaba7fc700 1 -- 172.21.15.40:0/1793202366 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbabc00abd0 con 0x7fbac4153880 2024-09-13T18:02:13.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.163+0000 7fbaba7fc700 1 --2- 172.21.15.40:0/1793202366 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbaa8066730 0x7fbaa8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.163+0000 7fbaba7fc700 1 -- 172.21.15.40:0/1793202366 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbabc0844d0 con 0x7fbac4153880 2024-09-13T18:02:13.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.163+0000 7fbac9c50700 1 --2- 172.21.15.40:0/1793202366 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbaa8066730 0x7fbaa8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.163+0000 7fbac9c50700 1 --2- 172.21.15.40:0/1793202366 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbaa8066730 0x7fbaa8068bf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbac000c960 tx=0x7fbac0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:13.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.167+0000 7fbaba7fc700 1 -- 172.21.15.40:0/1793202366 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbabc051490 con 0x7fbac4153880 2024-09-13T18:02:13.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.411+0000 7fbacac52700 1 -- 172.21.15.40:0/1793202366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbac40b1ad0 con 0x7fbac4153880 2024-09-13T18:02:13.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.411+0000 7fbaba7fc700 1 -- 172.21.15.40:0/1793202366 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fbabc054ab0 con 0x7fbac4153880 2024-09-13T18:02:13.423 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":88,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:02:13.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.415+0000 7fbacac52700 1 -- 172.21.15.40:0/1793202366 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbaa8066730 msgr2=0x7fbaa8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.415+0000 7fbacac52700 1 --2- 172.21.15.40:0/1793202366 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbaa8066730 0x7fbaa8068bf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbac000c960 tx=0x7fbac0008040 comp rx=0 tx=0).stop 2024-09-13T18:02:13.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.415+0000 7fbacac52700 1 -- 172.21.15.40:0/1793202366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac4153880 msgr2=0x7fbac41f2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.415+0000 7fbacac52700 1 --2- 172.21.15.40:0/1793202366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac4153880 0x7fbac41f2340 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00cc50 tx=0x7fbabc00cf60 comp rx=0 tx=0).stop 2024-09-13T18:02:13.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.415+0000 7fbacac52700 1 -- 172.21.15.40:0/1793202366 shutdown_connections 2024-09-13T18:02:13.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.415+0000 7fbacac52700 1 --2- 172.21.15.40:0/1793202366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbac4153880 0x7fbac41f2340 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.415+0000 7fbacac52700 1 --2- 172.21.15.40:0/1793202366 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbaa8066730 0x7fbaa8068bf0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.415+0000 7fbacac52700 1 --2- 172.21.15.40:0/1793202366 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbac40cf3c0 0x7fbac414bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.415+0000 7fbacac52700 1 --2- 172.21.15.40:0/1793202366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbac40cb2f0 0x7fbac40c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.415+0000 7fbacac52700 1 -- 172.21.15.40:0/1793202366 >> 172.21.15.40:0/1793202366 conn(0x7fbac406c5b0 msgr2=0x7fbac41529c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:13.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.415+0000 7fbacac52700 1 -- 172.21.15.40:0/1793202366 shutdown_connections 2024-09-13T18:02:13.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.415+0000 7fbacac52700 1 -- 172.21.15.40:0/1793202366 wait complete. 2024-09-13T18:02:13.438 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:02:13.438 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T18:02:13.438 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:02:13.438 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:02:13.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.555+0000 7fa42df10700 1 Processor -- start 2024-09-13T18:02:13.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.555+0000 7fa42df10700 1 -- start start 2024-09-13T18:02:13.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42df10700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa428153720 0x7fa4280ca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42df10700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa428161d40 0x7fa428164130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42df10700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa428159e50 con 0x7fa4280cf9b0 2024-09-13T18:02:13.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42df10700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4280c5ea0 con 0x7fa428161d40 2024-09-13T18:02:13.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42df10700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42815d390 con 0x7fa428153720 2024-09-13T18:02:13.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42d70f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa428161d40 0x7fa428164130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42d70f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa428161d40 0x7fa428164130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52088/0 (socket says 172.21.15.40:52088) 2024-09-13T18:02:13.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42d70f700 1 -- 172.21.15.40:0/1324022407 learned_addr learned my addr 172.21.15.40:0/1324022407 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:13.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42cf0e700 1 --2- 172.21.15.40:0/1324022407 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa428153720 0x7fa4280ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42d70f700 1 -- 172.21.15.40:0/1324022407 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa428153720 msgr2=0x7fa4280ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:13.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42d70f700 1 --2- 172.21.15.40:0/1324022407 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa428153720 0x7fa4280ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42d70f700 1 -- 172.21.15.40:0/1324022407 >> v1:172.21.15.43:6789/0 conn(0x7fa4280cf9b0 legacy=0x7fa42806c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42d70f700 1 -- 172.21.15.40:0/1324022407 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42815a080 con 0x7fa428161d40 2024-09-13T18:02:13.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42d70f700 1 --2- 172.21.15.40:0/1324022407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa428161d40 0x7fa428164130 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa420009e50 tx=0x7fa42000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9939b9aad5503e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:13.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa4277fe700 1 -- 172.21.15.40:0/1324022407 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42000e070 con 0x7fa428161d40 2024-09-13T18:02:13.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa4277fe700 1 -- 172.21.15.40:0/1324022407 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42000b910 con 0x7fa428161d40 2024-09-13T18:02:13.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa4277fe700 1 -- 172.21.15.40:0/1324022407 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42000bcc0 con 0x7fa428161d40 2024-09-13T18:02:13.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42df10700 1 -- 172.21.15.40:0/1324022407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa428161d40 msgr2=0x7fa428164130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42df10700 1 --2- 172.21.15.40:0/1324022407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa428161d40 0x7fa428164130 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa420009e50 tx=0x7fa42000b330 comp rx=0 tx=0).stop 2024-09-13T18:02:13.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42df10700 1 -- 172.21.15.40:0/1324022407 shutdown_connections 2024-09-13T18:02:13.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42df10700 1 --2- 172.21.15.40:0/1324022407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa428161d40 0x7fa428164130 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42df10700 1 --2- 172.21.15.40:0/1324022407 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa428153720 0x7fa4280ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42df10700 1 -- 172.21.15.40:0/1324022407 >> 172.21.15.40:0/1324022407 conn(0x7fa4280666c0 msgr2=0x7fa4280cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:13.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42df10700 1 -- 172.21.15.40:0/1324022407 shutdown_connections 2024-09-13T18:02:13.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.559+0000 7fa42df10700 1 -- 172.21.15.40:0/1324022407 wait complete. 2024-09-13T18:02:13.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa42df10700 1 Processor -- start 2024-09-13T18:02:13.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa42df10700 1 -- start start 2024-09-13T18:02:13.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa42df10700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4280cf9b0 0x7fa4280c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa42df10700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa428153720 0x7fa4280c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa42df10700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa428161d40 0x7fa428156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa42df10700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4280c69b0 con 0x7fa428161d40 2024-09-13T18:02:13.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa42df10700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4280c6790 con 0x7fa428153720 2024-09-13T18:02:13.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa42df10700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4281552d0 con 0x7fa4280cf9b0 2024-09-13T18:02:13.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa42d70f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa428161d40 0x7fa428156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa42d70f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa428161d40 0x7fa428156730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52092/0 (socket says 172.21.15.40:52092) 2024-09-13T18:02:13.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa42d70f700 1 -- 172.21.15.40:0/1549470383 learned_addr learned my addr 172.21.15.40:0/1549470383 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:13.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa42cf0e700 1 --2- 172.21.15.40:0/1549470383 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4280cf9b0 0x7fa4280c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa427fff700 1 --2- 172.21.15.40:0/1549470383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa428153720 0x7fa4280c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa42d70f700 1 -- 172.21.15.40:0/1549470383 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4280cf9b0 msgr2=0x7fa4280c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa42d70f700 1 --2- 172.21.15.40:0/1549470383 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4280cf9b0 0x7fa4280c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa42d70f700 1 -- 172.21.15.40:0/1549470383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa428153720 msgr2=0x7fa4280c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa42d70f700 1 --2- 172.21.15.40:0/1549470383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa428153720 0x7fa4280c6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa42d70f700 1 -- 172.21.15.40:0/1549470383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa420009b30 con 0x7fa428161d40 2024-09-13T18:02:13.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa42d70f700 1 --2- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa428161d40 0x7fa428156730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa42806bb00 tx=0x7fa420004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:13.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa4257fa700 1 -- 172.21.15.40:0/1549470383 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa420004800 con 0x7fa428161d40 2024-09-13T18:02:13.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa4257fa700 1 -- 172.21.15.40:0/1549470383 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4200099b0 con 0x7fa428161d40 2024-09-13T18:02:13.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa4257fa700 1 -- 172.21.15.40:0/1549470383 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42001c680 con 0x7fa428161d40 2024-09-13T18:02:13.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa42df10700 1 -- 172.21.15.40:0/1549470383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4280cb0f0 con 0x7fa428161d40 2024-09-13T18:02:13.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.563+0000 7fa42df10700 1 -- 172.21.15.40:0/1549470383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa428155800 con 0x7fa428161d40 2024-09-13T18:02:13.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.567+0000 7fa42df10700 1 --2- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4281ef0c0 0x7fa428155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.567+0000 7fa42df10700 1 -- 172.21.15.40:0/1549470383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa428155f60 con 0x7fa4281ef0c0 2024-09-13T18:02:13.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.567+0000 7fa42cf0e700 1 --2- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4281ef0c0 0x7fa428155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.567+0000 7fa42cf0e700 1 -- 172.21.15.40:0/1549470383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa42815ca60 con 0x7fa4281ef0c0 2024-09-13T18:02:13.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.567+0000 7fa42cf0e700 1 --2- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4281ef0c0 0x7fa428155a20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0119d0 tx=0x7fa41c011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:13.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.567+0000 7fa4257fa700 1 -- 172.21.15.40:0/1549470383 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c004240 con 0x7fa4281ef0c0 2024-09-13T18:02:13.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.567+0000 7fa4257fa700 1 -- 172.21.15.40:0/1549470383 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa42001c7e0 con 0x7fa428161d40 2024-09-13T18:02:13.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.567+0000 7fa4257fa700 1 --2- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa40c0667d0 0x7fa40c068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.567+0000 7fa4257fa700 1 -- 172.21.15.40:0/1549470383 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa420085da0 con 0x7fa428161d40 2024-09-13T18:02:13.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.567+0000 7fa427fff700 1 --2- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa40c0667d0 0x7fa40c068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.567+0000 7fa4257fa700 1 -- 172.21.15.40:0/1549470383 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa41c0043a0 con 0x7fa4281ef0c0 2024-09-13T18:02:13.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.567+0000 7fa4257fa700 1 -- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4281ef0c0 msgr2=0x7fa428155a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.567+0000 7fa4257fa700 1 --2- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4281ef0c0 0x7fa428155a20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0119d0 tx=0x7fa41c011d90 comp rx=0 tx=0).stop 2024-09-13T18:02:13.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.567+0000 7fa4257fa700 1 -- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4281ef0c0 msgr2=0x7fa428155a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:13.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.567+0000 7fa4257fa700 1 --2- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4281ef0c0 0x7fa428155a20 secure :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0119d0 tx=0x7fa41c011d90 comp rx=0 tx=0).stop 2024-09-13T18:02:13.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.567+0000 7fa427fff700 1 --2- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa40c0667d0 0x7fa40c068c90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa4280cad70 tx=0x7fa414009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:13.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.575+0000 7fa42df10700 1 --2- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4281ef650 0x7fa4281efa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.575+0000 7fa42df10700 1 -- 172.21.15.40:0/1549470383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa428074750 con 0x7fa4281ef650 2024-09-13T18:02:13.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.575+0000 7fa42cf0e700 1 --2- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4281ef650 0x7fa4281efa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.575+0000 7fa42cf0e700 1 -- 172.21.15.40:0/1549470383 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa428153b00 con 0x7fa4281ef650 2024-09-13T18:02:13.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.575+0000 7fa42cf0e700 1 --2- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4281ef650 0x7fa4281efa60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0119d0 tx=0x7fa41c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:13.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.575+0000 7fa4257fa700 1 -- 172.21.15.40:0/1549470383 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c00fd30 con 0x7fa4281ef650 2024-09-13T18:02:13.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.579+0000 7fa4257fa700 1 -- 172.21.15.40:0/1549470383 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4950 (secure 0 0 0) 0x7fa41c00fe90 con 0x7fa4281ef650 2024-09-13T18:02:13.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.579+0000 7fa4257fa700 1 -- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4281ef650 msgr2=0x7fa4281efa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.579+0000 7fa4257fa700 1 --2- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4281ef650 0x7fa4281efa60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0119d0 tx=0x7fa41c004770 comp rx=0 tx=0).stop 2024-09-13T18:02:13.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.579+0000 7fa4257fa700 1 -- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4281ef650 msgr2=0x7fa4281efa60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:13.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.579+0000 7fa4257fa700 1 --2- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4281ef650 0x7fa4281efa60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.588 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:02:13.588 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:02:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:02:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:02:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 88, 2024-09-13T18:02:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:02:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:02:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:02:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:02:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 14, 2024-09-13T18:02:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:02:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:02:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:13.589 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:02:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:02:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:13.590 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:02:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:02:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:02:13.591 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:02:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:02:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:02:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:02:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:02:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:02:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:02:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:02:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:02:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:02:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:02:13.592 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:02:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:02:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.593 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:02:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:02:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:02:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:02:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:13.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:02:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:02:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:02:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:02:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:13.595 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:02:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:02:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:02:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:02:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:13.596 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:02:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:02:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:02:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:13.597 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:02:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:02:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:02:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:02:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:02:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.598 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:02:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:02:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:02:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:02:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:13.599 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 8 2024-09-13T18:02:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:02:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:02:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:02:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:13.600 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:13.601 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:02:13.601 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.601 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:13.601 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.601 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:02:13.601 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:02:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.579+0000 7fa42df10700 1 -- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa40c0667d0 msgr2=0x7fa40c068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.579+0000 7fa42df10700 1 --2- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa40c0667d0 0x7fa40c068c90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa4280cad70 tx=0x7fa414009040 comp rx=0 tx=0).stop 2024-09-13T18:02:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.579+0000 7fa42df10700 1 -- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa428161d40 msgr2=0x7fa428156730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.579+0000 7fa42df10700 1 --2- 172.21.15.40:0/1549470383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa428161d40 0x7fa428156730 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa42806bb00 tx=0x7fa420004af0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.579+0000 7fa42d70f700 1 -- 172.21.15.40:0/1549470383 reap_dead start 2024-09-13T18:02:13.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.579+0000 7fa42d70f700 1 -- 172.21.15.40:0/1549470383 reap_dead start 2024-09-13T18:02:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.579+0000 7fa42df10700 1 -- 172.21.15.40:0/1549470383 shutdown_connections 2024-09-13T18:02:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.579+0000 7fa42df10700 1 -- 172.21.15.40:0/1549470383 >> 172.21.15.40:0/1549470383 conn(0x7fa4280666c0 msgr2=0x7fa4280b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.579+0000 7fa42df10700 1 -- 172.21.15.40:0/1549470383 shutdown_connections 2024-09-13T18:02:13.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.579+0000 7fa42df10700 1 -- 172.21.15.40:0/1549470383 wait complete. 2024-09-13T18:02:13.603 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:02:13.603 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:02:13.603 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:02:13.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3807fff700 1 Processor -- start 2024-09-13T18:02:13.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3807fff700 1 -- start start 2024-09-13T18:02:13.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3807fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3808153a90 0x7f38080c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3807fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38080704a0 0x7f38080c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3807fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f380815ce40 con 0x7f380815d2d0 2024-09-13T18:02:13.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3807fff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3808153e70 con 0x7f3808153a90 2024-09-13T18:02:13.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3807fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38080c88a0 con 0x7f38080704a0 2024-09-13T18:02:13.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3806ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3808153a90 0x7f38080c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3806ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3808153a90 0x7f38080c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52112/0 (socket says 172.21.15.40:52112) 2024-09-13T18:02:13.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3806ffd700 1 -- 172.21.15.40:0/2476269174 learned_addr learned my addr 172.21.15.40:0/2476269174 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:13.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f38077fe700 1 --1- 172.21.15.40:0/2476269174 >> v1:172.21.15.40:6789/0 conn(0x7f380815d2d0 0x7f380806f710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33350/0 (socket says 172.21.15.40:33350) 2024-09-13T18:02:13.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f38067fc700 1 --2- 172.21.15.40:0/2476269174 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38080704a0 0x7f38080c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3806ffd700 1 -- 172.21.15.40:0/2476269174 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38080704a0 msgr2=0x7f38080c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:13.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3806ffd700 1 --2- 172.21.15.40:0/2476269174 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38080704a0 0x7f38080c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3806ffd700 1 -- 172.21.15.40:0/2476269174 >> v1:172.21.15.40:6789/0 conn(0x7f380815d2d0 legacy=0x7f380806f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3806ffd700 1 -- 172.21.15.40:0/2476269174 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380815a280 con 0x7f3808153a90 2024-09-13T18:02:13.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3806ffd700 1 --2- 172.21.15.40:0/2476269174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3808153a90 0x7f38080c94a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009910 tx=0x7f37f8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91f2fe5b889f9b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:13.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3805ffb700 1 -- 172.21.15.40:0/2476269174 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f800ba60 con 0x7f3808153a90 2024-09-13T18:02:13.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3805ffb700 1 -- 172.21.15.40:0/2476269174 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f800bbc0 con 0x7f3808153a90 2024-09-13T18:02:13.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3805ffb700 1 -- 172.21.15.40:0/2476269174 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f8009e60 con 0x7f3808153a90 2024-09-13T18:02:13.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3807fff700 1 -- 172.21.15.40:0/2476269174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3808153a90 msgr2=0x7f38080c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3807fff700 1 --2- 172.21.15.40:0/2476269174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3808153a90 0x7f38080c94a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009910 tx=0x7f37f8009c20 comp rx=0 tx=0).stop 2024-09-13T18:02:13.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3807fff700 1 -- 172.21.15.40:0/2476269174 shutdown_connections 2024-09-13T18:02:13.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3807fff700 1 --2- 172.21.15.40:0/2476269174 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38080704a0 0x7f38080c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3807fff700 1 --2- 172.21.15.40:0/2476269174 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3808153a90 0x7f38080c94a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3807fff700 1 -- 172.21.15.40:0/2476269174 >> 172.21.15.40:0/2476269174 conn(0x7f38080ccef0 msgr2=0x7f38080cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:13.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3807fff700 1 -- 172.21.15.40:0/2476269174 shutdown_connections 2024-09-13T18:02:13.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.707+0000 7f3807fff700 1 -- 172.21.15.40:0/2476269174 wait complete. 2024-09-13T18:02:13.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f3807fff700 1 Processor -- start 2024-09-13T18:02:13.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f3807fff700 1 -- start start 2024-09-13T18:02:13.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f3807fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38080704a0 0x7f38080c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f3807fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3808153a90 0x7f38080c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f3807fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f380815d2d0 0x7f38081541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f3807fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3808157cd0 con 0x7f3808153a90 2024-09-13T18:02:13.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f3807fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38080b2040 con 0x7f38080704a0 2024-09-13T18:02:13.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f3807fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3808158b40 con 0x7f380815d2d0 2024-09-13T18:02:13.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f38067fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3808153a90 0x7f38080c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f38067fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3808153a90 0x7f38080c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52128/0 (socket says 172.21.15.40:52128) 2024-09-13T18:02:13.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f38067fc700 1 -- 172.21.15.40:0/992465701 learned_addr learned my addr 172.21.15.40:0/992465701 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:13.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f38077fe700 1 --2- 172.21.15.40:0/992465701 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f380815d2d0 0x7f38081541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f3806ffd700 1 --2- 172.21.15.40:0/992465701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38080704a0 0x7f38080c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f38067fc700 1 -- 172.21.15.40:0/992465701 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f380815d2d0 msgr2=0x7f38081541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f38067fc700 1 --2- 172.21.15.40:0/992465701 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f380815d2d0 0x7f38081541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f38067fc700 1 -- 172.21.15.40:0/992465701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38080704a0 msgr2=0x7f38080c5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f38067fc700 1 --2- 172.21.15.40:0/992465701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38080704a0 0x7f38080c5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f38067fc700 1 -- 172.21.15.40:0/992465701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f8009570 con 0x7f3808153a90 2024-09-13T18:02:13.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f38067fc700 1 --2- 172.21.15.40:0/992465701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3808153a90 0x7f38080c5860 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00d860 tx=0x7f37fc00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:13.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f37ef7fe700 1 -- 172.21.15.40:0/992465701 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc0125f0 con 0x7f3808153a90 2024-09-13T18:02:13.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.711+0000 7f37ef7fe700 1 -- 172.21.15.40:0/992465701 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37fc012750 con 0x7f3808153a90 2024-09-13T18:02:13.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.715+0000 7f3807fff700 1 -- 172.21.15.40:0/992465701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38081f42e0 con 0x7f3808153a90 2024-09-13T18:02:13.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.715+0000 7f37ef7fe700 1 -- 172.21.15.40:0/992465701 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc012ac0 con 0x7f3808153a90 2024-09-13T18:02:13.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.715+0000 7f3807fff700 1 -- 172.21.15.40:0/992465701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3808159090 con 0x7f3808153a90 2024-09-13T18:02:13.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.715+0000 7f3807fff700 1 --2- 172.21.15.40:0/992465701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38081546f0 0x7f38081f6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.715+0000 7f3807fff700 1 -- 172.21.15.40:0/992465701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3808074830 con 0x7f38081546f0 2024-09-13T18:02:13.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.715+0000 7f3806ffd700 1 --2- 172.21.15.40:0/992465701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38081546f0 0x7f38081f6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.715+0000 7f37ef7fe700 1 -- 172.21.15.40:0/992465701 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f37fc012c20 con 0x7f3808153a90 2024-09-13T18:02:13.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.715+0000 7f3806ffd700 1 -- 172.21.15.40:0/992465701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f380815a560 con 0x7f38081546f0 2024-09-13T18:02:13.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.715+0000 7f3806ffd700 1 --2- 172.21.15.40:0/992465701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38081546f0 0x7f38081f6a90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009910 tx=0x7f37f8004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:13.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.715+0000 7f37ef7fe700 1 --2- 172.21.15.40:0/992465701 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f37f4066730 0x7f37f4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.715+0000 7f37ef7fe700 1 -- 172.21.15.40:0/992465701 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f37fc0848e0 con 0x7f3808153a90 2024-09-13T18:02:13.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.715+0000 7f37ef7fe700 1 -- 172.21.15.40:0/992465701 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f80044d0 con 0x7f38081546f0 2024-09-13T18:02:13.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.715+0000 7f38077fe700 1 --2- 172.21.15.40:0/992465701 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f37f4066730 0x7f37f4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.715+0000 7f37ef7fe700 1 -- 172.21.15.40:0/992465701 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f37f80036c0 con 0x7f38081546f0 2024-09-13T18:02:13.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.715+0000 7f37ef7fe700 1 -- 172.21.15.40:0/992465701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38081546f0 msgr2=0x7f38081f6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.715+0000 7f37ef7fe700 1 --2- 172.21.15.40:0/992465701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38081546f0 0x7f38081f6a90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009910 tx=0x7f37f8004770 comp rx=0 tx=0).stop 2024-09-13T18:02:13.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.715+0000 7f37ef7fe700 1 -- 172.21.15.40:0/992465701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38081546f0 msgr2=0x7f38081f6a90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:13.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.715+0000 7f37ef7fe700 1 --2- 172.21.15.40:0/992465701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38081546f0 0x7f38081f6a90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.719+0000 7f38077fe700 1 --2- 172.21.15.40:0/992465701 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f37f4066730 0x7f37f4068bf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f38000012e0 tx=0x7f3800001a00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:13.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.727+0000 7f3807fff700 1 --2- 172.21.15.40:0/992465701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3808154ad0 0x7f38081f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.727+0000 7f3807fff700 1 -- 172.21.15.40:0/992465701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3808067520 con 0x7f3808154ad0 2024-09-13T18:02:13.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.727+0000 7f3806ffd700 1 --2- 172.21.15.40:0/992465701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3808154ad0 0x7f38081f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.727+0000 7f3806ffd700 1 -- 172.21.15.40:0/992465701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f38080c78f0 con 0x7f3808154ad0 2024-09-13T18:02:13.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.731+0000 7f3806ffd700 1 --2- 172.21.15.40:0/992465701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3808154ad0 0x7f38081f6fd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009c60 tx=0x7f37f8011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:13.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.731+0000 7f37ef7fe700 1 -- 172.21.15.40:0/992465701 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f8003e50 con 0x7f3808154ad0 2024-09-13T18:02:13.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.731+0000 7f37ef7fe700 1 -- 172.21.15.40:0/992465701 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4450 (secure 0 0 0) 0x7f37f8035440 con 0x7f3808154ad0 2024-09-13T18:02:13.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.731+0000 7f37ef7fe700 1 -- 172.21.15.40:0/992465701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3808154ad0 msgr2=0x7f38081f6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.731+0000 7f37ef7fe700 1 --2- 172.21.15.40:0/992465701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3808154ad0 0x7f38081f6fd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009c60 tx=0x7f37f8011000 comp rx=0 tx=0).stop 2024-09-13T18:02:13.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.731+0000 7f37ef7fe700 1 -- 172.21.15.40:0/992465701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3808154ad0 msgr2=0x7f38081f6fd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:13.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.731+0000 7f37ef7fe700 1 --2- 172.21.15.40:0/992465701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3808154ad0 0x7f38081f6fd0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.741 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:02:13.741 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:02:13.741 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:02:13.741 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:02:13.741 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 88, 2024-09-13T18:02:13.742 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:02:13.742 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:02:13.742 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:02:13.742 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:02:13.742 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:13.742 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 14, 2024-09-13T18:02:13.742 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:13.742 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:02:13.742 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:02:13.742 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:13.742 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:13.742 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:13.742 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:13.743 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:13.743 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:13.743 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:13.743 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:13.743 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:13.743 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:13.743 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:13.743 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:02:13.743 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:02:13.743 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:13.743 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:13.743 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:13.743 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:13.744 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:13.744 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:13.744 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:13.744 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:13.744 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:13.744 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:13.744 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:13.744 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.744 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:02:13.744 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:02:13.744 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:02:13.744 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:02:13.744 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:02:13.744 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:02:13.745 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:13.745 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:13.745 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:02:13.745 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:02:13.745 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:02:13.745 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:02:13.745 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:02:13.745 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:02:13.745 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:02:13.745 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:13.745 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:02:13.745 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:13.745 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:13.746 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:13.746 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:13.746 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:13.746 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:13.746 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:13.746 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:13.746 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:13.746 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:13.746 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:13.746 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:02:13.746 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.746 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:02:13.746 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.746 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:02:13.747 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:02:13.747 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:13.747 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:13.747 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.747 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:13.747 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:02:13.747 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:13.747 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.747 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.747 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:13.747 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:02:13.747 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:13.747 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.748 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:13.748 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.748 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:02:13.748 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:02:13.748 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:13.748 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:13.748 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:13.748 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.748 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.748 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:02:13.748 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:02:13.748 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:13.748 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:13.748 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.749 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:13.749 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:02:13.749 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:13.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.749 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.749 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:13.749 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:02:13.749 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:13.749 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.749 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:13.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.749 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:02:13.749 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:02:13.750 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:13.750 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:13.750 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:13.750 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.750 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.750 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:02:13.750 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:02:13.750 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:13.750 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:13.750 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.750 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:13.750 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:02:13.750 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:13.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.751 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.751 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:13.751 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:02:13.751 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:13.751 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.751 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:13.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.751 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:02:13.751 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:02:13.751 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:13.751 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:13.751 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:13.751 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.752 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:13.752 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.752 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:02:13.752 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:02:13.752 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.752 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:13.752 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:13.752 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:02:13.752 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.752 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:13.752 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:02:13.752 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.752 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:13.753 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:13.753 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:02:13.753 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.753 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:13.753 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.753 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:02:13.753 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:02:13.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.731+0000 7f3807fff700 1 -- 172.21.15.40:0/992465701 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f37f4066730 msgr2=0x7f37f4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.731+0000 7f3807fff700 1 --2- 172.21.15.40:0/992465701 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f37f4066730 0x7f37f4068bf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f38000012e0 tx=0x7f3800001a00 comp rx=0 tx=0).stop 2024-09-13T18:02:13.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.731+0000 7f3807fff700 1 -- 172.21.15.40:0/992465701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3808153a90 msgr2=0x7f38080c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.731+0000 7f3807fff700 1 --2- 172.21.15.40:0/992465701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3808153a90 0x7f38080c5860 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00d860 tx=0x7f37fc00db70 comp rx=0 tx=0).stop 2024-09-13T18:02:13.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.731+0000 7f38077fe700 1 -- 172.21.15.40:0/992465701 reap_dead start 2024-09-13T18:02:13.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.731+0000 7f38077fe700 1 -- 172.21.15.40:0/992465701 reap_dead start 2024-09-13T18:02:13.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.731+0000 7f3807fff700 1 -- 172.21.15.40:0/992465701 shutdown_connections 2024-09-13T18:02:13.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.731+0000 7f3807fff700 1 -- 172.21.15.40:0/992465701 >> 172.21.15.40:0/992465701 conn(0x7f38080ccef0 msgr2=0x7f3808073670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:13.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.731+0000 7f3807fff700 1 -- 172.21.15.40:0/992465701 shutdown_connections 2024-09-13T18:02:13.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.731+0000 7f3807fff700 1 -- 172.21.15.40:0/992465701 wait complete. 2024-09-13T18:02:13.756 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:02:13.756 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:02:13.756 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:02:13.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.883+0000 7fe9b1f82700 1 Processor -- start 2024-09-13T18:02:13.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.883+0000 7fe9b1f82700 1 -- start start 2024-09-13T18:02:13.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b1f82700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1503c0 0x7fe9ac0c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b1f82700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe9ac0c8500 0x7fe9ac0c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b1f82700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9ac0c8f80 con 0x7fe9ac0c71c0 2024-09-13T18:02:13.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b1f82700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9ac0c2a80 con 0x7fe9ac0c8500 2024-09-13T18:02:13.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b1f82700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac1507a0 con 0x7fe9ac1503c0 2024-09-13T18:02:13.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9abfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe9ac0c8500 0x7fe9ac0c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b0f80700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1503c0 0x7fe9ac0c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b0f80700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1503c0 0x7fe9ac0c79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:33222/0 (socket says 172.21.15.40:33222) 2024-09-13T18:02:13.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9abfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe9ac0c8500 0x7fe9ac0c6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:58956/0 (socket says 172.21.15.40:58956) 2024-09-13T18:02:13.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b0f80700 1 -- 172.21.15.40:0/3561111507 learned_addr learned my addr 172.21.15.40:0/3561111507 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:13.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b0f80700 1 -- 172.21.15.40:0/3561111507 >> v2:172.21.15.43:3300/0 conn(0x7fe9ac0c8500 msgr2=0x7fe9ac0c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:13.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b0f80700 1 --2- 172.21.15.40:0/3561111507 >> v2:172.21.15.43:3300/0 conn(0x7fe9ac0c8500 0x7fe9ac0c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b0f80700 1 -- 172.21.15.40:0/3561111507 >> v1:172.21.15.43:6789/0 conn(0x7fe9ac0c71c0 legacy=0x7fe9ac159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b0f80700 1 -- 172.21.15.40:0/3561111507 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac0c8a40 con 0x7fe9ac1503c0 2024-09-13T18:02:13.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9abfff700 1 --2- 172.21.15.40:0/3561111507 >> v2:172.21.15.43:3300/0 conn(0x7fe9ac0c8500 0x7fe9ac0c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:02:13.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b0f80700 1 --2- 172.21.15.40:0/3561111507 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1503c0 0x7fe9ac0c79c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe998009910 tx=0x7fe998009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ea6aeab2792ae5b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:13.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9ab7fe700 1 -- 172.21.15.40:0/3561111507 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe998011830 con 0x7fe9ac1503c0 2024-09-13T18:02:13.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9ab7fe700 1 -- 172.21.15.40:0/3561111507 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe998011990 con 0x7fe9ac1503c0 2024-09-13T18:02:13.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9ab7fe700 1 -- 172.21.15.40:0/3561111507 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe998011d40 con 0x7fe9ac1503c0 2024-09-13T18:02:13.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b1f82700 1 -- 172.21.15.40:0/3561111507 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1503c0 msgr2=0x7fe9ac0c79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b1f82700 1 --2- 172.21.15.40:0/3561111507 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1503c0 0x7fe9ac0c79c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe998009910 tx=0x7fe998009c20 comp rx=0 tx=0).stop 2024-09-13T18:02:13.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b1f82700 1 -- 172.21.15.40:0/3561111507 shutdown_connections 2024-09-13T18:02:13.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b1f82700 1 --2- 172.21.15.40:0/3561111507 >> v2:172.21.15.43:3300/0 conn(0x7fe9ac0c8500 0x7fe9ac0c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b1f82700 1 --2- 172.21.15.40:0/3561111507 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1503c0 0x7fe9ac0c79c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b1f82700 1 -- 172.21.15.40:0/3561111507 >> 172.21.15.40:0/3561111507 conn(0x7fe9ac06e6d0 msgr2=0x7fe9ac06eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:13.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b1f82700 1 -- 172.21.15.40:0/3561111507 shutdown_connections 2024-09-13T18:02:13.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.887+0000 7fe9b1f82700 1 -- 172.21.15.40:0/3561111507 wait complete. 2024-09-13T18:02:13.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.891+0000 7fe9b1f82700 1 Processor -- start 2024-09-13T18:02:13.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.891+0000 7fe9b1f82700 1 -- start start 2024-09-13T18:02:13.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.891+0000 7fe9b1f82700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9ac0c71c0 0x7fe9ac1e1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.891+0000 7fe9b1f82700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac0c8500 0x7fe9ac150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.891+0000 7fe9b1f82700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9ac1503c0 0x7fe9ac14b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.891+0000 7fe9b1f82700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac1e2be0 con 0x7fe9ac1503c0 2024-09-13T18:02:13.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.891+0000 7fe9b1f82700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac1e1cf0 con 0x7fe9ac0c71c0 2024-09-13T18:02:13.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.891+0000 7fe9b1f82700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac1e1fd0 con 0x7fe9ac0c8500 2024-09-13T18:02:13.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.891+0000 7fe9b1781700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9ac1503c0 0x7fe9ac14b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.891+0000 7fe9b1781700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9ac1503c0 0x7fe9ac14b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52138/0 (socket says 172.21.15.40:52138) 2024-09-13T18:02:13.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.891+0000 7fe9b1781700 1 -- 172.21.15.40:0/3380853841 learned_addr learned my addr 172.21.15.40:0/3380853841 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:13.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.891+0000 7fe9abfff700 1 --2- 172.21.15.40:0/3380853841 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac0c8500 0x7fe9ac150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.891+0000 7fe9b0f80700 1 --2- 172.21.15.40:0/3380853841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9ac0c71c0 0x7fe9ac1e1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.891+0000 7fe9b1781700 1 -- 172.21.15.40:0/3380853841 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac0c8500 msgr2=0x7fe9ac150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.895+0000 7fe9b1781700 1 --2- 172.21.15.40:0/3380853841 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac0c8500 0x7fe9ac150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.895+0000 7fe9b1781700 1 -- 172.21.15.40:0/3380853841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9ac0c71c0 msgr2=0x7fe9ac1e1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.895+0000 7fe9b1781700 1 --2- 172.21.15.40:0/3380853841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9ac0c71c0 0x7fe9ac1e1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.895+0000 7fe9b1781700 1 -- 172.21.15.40:0/3380853841 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a00094b0 con 0x7fe9ac1503c0 2024-09-13T18:02:13.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.895+0000 7fe9b1781700 1 --2- 172.21.15.40:0/3380853841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9ac1503c0 0x7fe9ac14b920 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400cc50 tx=0x7fe9a400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:13.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.895+0000 7fe9a97fa700 1 -- 172.21.15.40:0/3380853841 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4003c20 con 0x7fe9ac1503c0 2024-09-13T18:02:13.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.895+0000 7fe9a97fa700 1 -- 172.21.15.40:0/3380853841 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a4003d80 con 0x7fe9ac1503c0 2024-09-13T18:02:13.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.895+0000 7fe9b1f82700 1 -- 172.21.15.40:0/3380853841 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe998009570 con 0x7fe9ac1503c0 2024-09-13T18:02:13.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.895+0000 7fe9a97fa700 1 -- 172.21.15.40:0/3380853841 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a400aab0 con 0x7fe9ac1503c0 2024-09-13T18:02:13.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.895+0000 7fe9b1f82700 1 -- 172.21.15.40:0/3380853841 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ac1e2590 con 0x7fe9ac1503c0 2024-09-13T18:02:13.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.895+0000 7fe9b1f82700 1 --2- 172.21.15.40:0/3380853841 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1e27b0 0x7fe9ac1464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.895+0000 7fe9b1f82700 1 -- 172.21.15.40:0/3380853841 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac067600 con 0x7fe9ac1e27b0 2024-09-13T18:02:13.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.895+0000 7fe9b0f80700 1 --2- 172.21.15.40:0/3380853841 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1e27b0 0x7fe9ac1464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.895+0000 7fe9b0f80700 1 -- 172.21.15.40:0/3380853841 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe9ac0c2710 con 0x7fe9ac1e27b0 2024-09-13T18:02:13.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.895+0000 7fe9b0f80700 1 --2- 172.21.15.40:0/3380853841 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1e27b0 0x7fe9ac1464b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe998009910 tx=0x7fe998004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:13.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.895+0000 7fe9a97fa700 1 -- 172.21.15.40:0/3380853841 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe9a400ac10 con 0x7fe9ac1503c0 2024-09-13T18:02:13.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.899+0000 7fe9a97fa700 1 --2- 172.21.15.40:0/3380853841 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe990066760 0x7fe990068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.899+0000 7fe9abfff700 1 --2- 172.21.15.40:0/3380853841 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe990066760 0x7fe990068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.899+0000 7fe9a97fa700 1 -- 172.21.15.40:0/3380853841 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe9a40844d0 con 0x7fe9ac1503c0 2024-09-13T18:02:13.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.899+0000 7fe9a97fa700 1 -- 172.21.15.40:0/3380853841 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99801f070 con 0x7fe9ac1e27b0 2024-09-13T18:02:13.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.899+0000 7fe9a97fa700 1 -- 172.21.15.40:0/3380853841 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe998004100 con 0x7fe9ac1e27b0 2024-09-13T18:02:13.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.899+0000 7fe9a97fa700 1 -- 172.21.15.40:0/3380853841 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1e27b0 msgr2=0x7fe9ac1464b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.899+0000 7fe9a97fa700 1 --2- 172.21.15.40:0/3380853841 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1e27b0 0x7fe9ac1464b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe998009910 tx=0x7fe998004770 comp rx=0 tx=0).stop 2024-09-13T18:02:13.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.899+0000 7fe9a97fa700 1 -- 172.21.15.40:0/3380853841 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1e27b0 msgr2=0x7fe9ac1464b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:13.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.899+0000 7fe9a97fa700 1 --2- 172.21.15.40:0/3380853841 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1e27b0 0x7fe9ac1464b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.899+0000 7fe9abfff700 1 --2- 172.21.15.40:0/3380853841 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe990066760 0x7fe990068c20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac06afc0 tx=0x7fe9a0009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.907+0000 7fe9b1f82700 1 --2- 172.21.15.40:0/3380853841 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1e3050 0x7fe9ac1469f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.907+0000 7fe9b1f82700 1 -- 172.21.15.40:0/3380853841 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac0c8ce0 con 0x7fe9ac1e3050 2024-09-13T18:02:13.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.907+0000 7fe9b0f80700 1 --2- 172.21.15.40:0/3380853841 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1e3050 0x7fe9ac1469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.907+0000 7fe9b0f80700 1 -- 172.21.15.40:0/3380853841 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe9ac0b1e60 con 0x7fe9ac1e3050 2024-09-13T18:02:13.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.911+0000 7fe9b0f80700 1 --2- 172.21.15.40:0/3380853841 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1e3050 0x7fe9ac1469f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0ccbb0 tx=0x7fe998011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:13.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.911+0000 7fe9a97fa700 1 -- 172.21.15.40:0/3380853841 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99802a040 con 0x7fe9ac1e3050 2024-09-13T18:02:13.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.911+0000 7fe9a97fa700 1 -- 172.21.15.40:0/3380853841 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4616 (secure 0 0 0) 0x7fe99802b780 con 0x7fe9ac1e3050 2024-09-13T18:02:13.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.911+0000 7fe9a97fa700 1 -- 172.21.15.40:0/3380853841 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1e3050 msgr2=0x7fe9ac1469f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.911+0000 7fe9a97fa700 1 --2- 172.21.15.40:0/3380853841 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1e3050 0x7fe9ac1469f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0ccbb0 tx=0x7fe998011000 comp rx=0 tx=0).stop 2024-09-13T18:02:13.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.911+0000 7fe9a97fa700 1 -- 172.21.15.40:0/3380853841 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1e3050 msgr2=0x7fe9ac1469f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:13.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.911+0000 7fe9a97fa700 1 --2- 172.21.15.40:0/3380853841 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe9ac1e3050 0x7fe9ac1469f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:13.921 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:02:13.921 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:02:13.921 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:02:13.921 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:02:13.921 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 88, 2024-09-13T18:02:13.921 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:02:13.921 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:02:13.921 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:02:13.922 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:02:13.922 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:13.922 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 14, 2024-09-13T18:02:13.922 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:13.922 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:02:13.922 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:02:13.922 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:13.922 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:13.922 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:13.922 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:13.922 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:13.922 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:13.922 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:13.923 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:13.923 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:13.923 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:13.923 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:13.923 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:02:13.923 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:02:13.923 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:13.923 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:13.923 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:13.923 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:13.923 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:13.923 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:13.923 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:13.924 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:13.924 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:13.924 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:13.924 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:13.924 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.924 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:02:13.924 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:02:13.924 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:02:13.924 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:02:13.924 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:02:13.924 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:02:13.924 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:13.924 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:13.925 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:02:13.925 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:02:13.925 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:02:13.925 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:02:13.925 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:02:13.925 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:02:13.925 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:02:13.925 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:13.925 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:02:13.925 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:13.925 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:13.925 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:13.925 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:13.926 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:13.926 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:13.926 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:13.926 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:13.926 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:13.926 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:13.926 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:13.926 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:02:13.926 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.926 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:02:13.926 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.926 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:02:13.926 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:02:13.926 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:13.927 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:13.927 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.927 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:13.927 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:02:13.927 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:13.927 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.927 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.927 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:13.927 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:02:13.927 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:13.927 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.927 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:13.927 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.927 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:02:13.928 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:02:13.928 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:13.928 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:13.928 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:13.928 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.928 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.928 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:02:13.928 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:02:13.928 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:13.928 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:13.928 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.928 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:13.928 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:02:13.929 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:13.929 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.929 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.929 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:13.929 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:02:13.929 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:13.929 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.929 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:13.929 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.929 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:02:13.929 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:02:13.929 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:13.929 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:13.929 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:13.930 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.930 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.930 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:02:13.930 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:02:13.930 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:13.930 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:13.930 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.930 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:13.930 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:02:13.930 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:13.930 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.930 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.930 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:13.930 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:02:13.931 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:13.931 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.931 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:13.931 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.931 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:02:13.931 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:02:13.931 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:13.931 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:13.931 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:13.931 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.931 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:13.931 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.931 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:02:13.932 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:02:13.932 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.932 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:13.932 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:13.932 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:02:13.932 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.932 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:13.932 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:02:13.932 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.932 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:13.932 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:13.932 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:02:13.932 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.932 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:13.933 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:02:13.933 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:13.933 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:13.933 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:13.933 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:02:13.933 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:13.933 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:13.933 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:13.933 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:02:13.933 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:02:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.911+0000 7fe9b1f82700 1 -- 172.21.15.40:0/3380853841 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe990066760 msgr2=0x7fe990068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.911+0000 7fe9b1f82700 1 --2- 172.21.15.40:0/3380853841 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe990066760 0x7fe990068c20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac06afc0 tx=0x7fe9a0009240 comp rx=0 tx=0).stop 2024-09-13T18:02:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.911+0000 7fe9b1f82700 1 -- 172.21.15.40:0/3380853841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9ac1503c0 msgr2=0x7fe9ac14b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.911+0000 7fe9b1f82700 1 --2- 172.21.15.40:0/3380853841 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9ac1503c0 0x7fe9ac14b920 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe9a400cc50 tx=0x7fe9a400cf60 comp rx=0 tx=0).stop 2024-09-13T18:02:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.911+0000 7fe9b1781700 1 -- 172.21.15.40:0/3380853841 reap_dead start 2024-09-13T18:02:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.911+0000 7fe9b1781700 1 -- 172.21.15.40:0/3380853841 reap_dead start 2024-09-13T18:02:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.911+0000 7fe9b1f82700 1 -- 172.21.15.40:0/3380853841 shutdown_connections 2024-09-13T18:02:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.911+0000 7fe9b1f82700 1 -- 172.21.15.40:0/3380853841 >> 172.21.15.40:0/3380853841 conn(0x7fe9ac06e6d0 msgr2=0x7fe9ac071970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.911+0000 7fe9b1f82700 1 -- 172.21.15.40:0/3380853841 shutdown_connections 2024-09-13T18:02:13.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:13.911+0000 7fe9b1f82700 1 -- 172.21.15.40:0/3380853841 wait complete. 2024-09-13T18:02:13.935 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:02:13.936 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-13T18:02:13.936 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T18:02:13.936 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-13T18:02:13.936 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-13T18:02:13.936 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T18:02:13.936 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:02:13.981 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T18:02:13.981 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T18:02:13.981 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:02:14.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.095+0000 7ffb25519700 1 Processor -- start 2024-09-13T18:02:14.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.095+0000 7ffb25519700 1 -- start start 2024-09-13T18:02:14.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.095+0000 7ffb25519700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffb1804f220 0x7ffb180dedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:14.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.095+0000 7ffb25519700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb18050390 con 0x7ffb18007430 2024-09-13T18:02:14.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.095+0000 7ffb25519700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb18055030 con 0x7ffb180d8b40 2024-09-13T18:02:14.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.095+0000 7ffb25519700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb180508e0 con 0x7ffb1804f220 2024-09-13T18:02:14.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.095+0000 7ffb1f7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ffb18007430 0x7ffb180de6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33366/0 (socket says 172.21.15.40:33366) 2024-09-13T18:02:14.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.095+0000 7ffb1f7fe700 1 -- 172.21.15.40:0/2837500223 learned_addr learned my addr 172.21.15.40:0/2837500223 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:02:14.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.095+0000 7ffb24d18700 1 --2- 172.21.15.40:0/2837500223 >> v2:172.21.15.43:3300/0 conn(0x7ffb1804f220 0x7ffb180dedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1effd700 1 -- 172.21.15.40:0/2837500223 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1658068536 0 0) 0x7ffb18050390 con 0x7ffb18007430 2024-09-13T18:02:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1effd700 1 -- 172.21.15.40:0/2837500223 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb100035d0 con 0x7ffb18007430 2024-09-13T18:02:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1effd700 1 -- 172.21.15.40:0/2837500223 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3344777930 0 0) 0x7ffb18055030 con 0x7ffb180d8b40 2024-09-13T18:02:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1effd700 1 -- 172.21.15.40:0/2837500223 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb18050390 con 0x7ffb180d8b40 2024-09-13T18:02:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1effd700 1 -- 172.21.15.40:0/2837500223 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2695546328 0 0) 0x7ffb100035d0 con 0x7ffb18007430 2024-09-13T18:02:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1effd700 1 -- 172.21.15.40:0/2837500223 >> v2:172.21.15.43:3300/0 conn(0x7ffb1804f220 msgr2=0x7ffb180dedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1effd700 1 --2- 172.21.15.40:0/2837500223 >> v2:172.21.15.43:3300/0 conn(0x7ffb1804f220 0x7ffb180dedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1effd700 1 -- 172.21.15.40:0/2837500223 >> v1:172.21.15.43:6789/0 conn(0x7ffb180d8b40 legacy=0x7ffb18049090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1effd700 1 -- 172.21.15.40:0/2837500223 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb18050600 con 0x7ffb18007430 2024-09-13T18:02:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1effd700 1 -- 172.21.15.40:0/2837500223 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7ffb0c001670 con 0x7ffb18007430 2024-09-13T18:02:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1effd700 1 -- 172.21.15.40:0/2837500223 >> v1:172.21.15.40:6789/0 conn(0x7ffb18007430 legacy=0x7ffb180de6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1effd700 1 --2- 172.21.15.40:0/2837500223 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb10003c00 0x7ffb10004000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:14.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1effd700 1 --2- 172.21.15.40:0/2837500223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb10004870 0x7ffb10004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1effd700 1 --2- 172.21.15.40:0/2837500223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb100052a0 0x7ffb1000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1effd700 1 -- 172.21.15.40:0/2837500223 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb180508e0 con 0x7ffb10004870 2024-09-13T18:02:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1effd700 1 -- 172.21.15.40:0/2837500223 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1000c200 con 0x7ffb100052a0 2024-09-13T18:02:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1effd700 1 -- 172.21.15.40:0/2837500223 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1000c3c0 con 0x7ffb10003c00 2024-09-13T18:02:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb24d18700 1 --2- 172.21.15.40:0/2837500223 >> v2:172.21.15.43:3300/0 conn(0x7ffb1804f220 0x7ffb180dedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:02:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1ffff700 1 -- 172.21.15.40:0/2837500223 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb10003c00 msgr2=0x7ffb10004000 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:02:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1ffff700 1 --2- 172.21.15.40:0/2837500223 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb10003c00 0x7ffb10004000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:02:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1f7fe700 1 --2- 172.21.15.40:0/2837500223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb10004870 0x7ffb10004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb24d18700 1 --2- 172.21.15.40:0/2837500223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb100052a0 0x7ffb1000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1f7fe700 1 -- 172.21.15.40:0/2837500223 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb10003c00 msgr2=0x7ffb10004000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:02:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1f7fe700 1 --2- 172.21.15.40:0/2837500223 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb10003c00 0x7ffb10004000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1f7fe700 1 -- 172.21.15.40:0/2837500223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb100052a0 msgr2=0x7ffb1000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1f7fe700 1 --2- 172.21.15.40:0/2837500223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb100052a0 0x7ffb1000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:14.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1f7fe700 1 -- 172.21.15.40:0/2837500223 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb100035d0 con 0x7ffb10004870 2024-09-13T18:02:14.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb24d18700 1 -- 172.21.15.40:0/2837500223 reap_dead start 2024-09-13T18:02:14.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1f7fe700 1 --2- 172.21.15.40:0/2837500223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb10004870 0x7ffb10004cf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00c160 tx=0x7ffb100058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0490fb05829a321 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:14.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1effd700 1 -- 172.21.15.40:0/2837500223 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c00ccc0 con 0x7ffb10004870 2024-09-13T18:02:14.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1effd700 1 -- 172.21.15.40:0/2837500223 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0c00ce20 con 0x7ffb10004870 2024-09-13T18:02:14.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb1effd700 1 -- 172.21.15.40:0/2837500223 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c005ba0 con 0x7ffb10004870 2024-09-13T18:02:14.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb25519700 1 -- 172.21.15.40:0/2837500223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb10004870 msgr2=0x7ffb10004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:14.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb25519700 1 --2- 172.21.15.40:0/2837500223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb10004870 0x7ffb10004cf0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00c160 tx=0x7ffb100058a0 comp rx=0 tx=0).stop 2024-09-13T18:02:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb25519700 1 -- 172.21.15.40:0/2837500223 shutdown_connections 2024-09-13T18:02:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb25519700 1 --2- 172.21.15.40:0/2837500223 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb10004870 0x7ffb10004cf0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb25519700 1 -- 172.21.15.40:0/2837500223 >> 172.21.15.40:0/2837500223 conn(0x7ffb18054110 msgr2=0x7ffb18054520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb25519700 1 -- 172.21.15.40:0/2837500223 shutdown_connections 2024-09-13T18:02:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.099+0000 7ffb25519700 1 -- 172.21.15.40:0/2837500223 wait complete. 2024-09-13T18:02:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb25519700 1 Processor -- start 2024-09-13T18:02:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb25519700 1 -- start start 2024-09-13T18:02:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb25519700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb10004870 0x7ffb18053750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb25519700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb181660c0 0x7ffb18164910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb25519700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb180df580 0x7ffb18164e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb25519700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb18177d60 con 0x7ffb181660c0 2024-09-13T18:02:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb25519700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb18177f70 con 0x7ffb180df580 2024-09-13T18:02:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb25519700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb18178250 con 0x7ffb10004870 2024-09-13T18:02:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb1ffff700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb10004870 msgr2=0x7ffb18053750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:02:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb1ffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb10004870 0x7ffb18053750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:02:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb1f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb181660c0 0x7ffb18164910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb1f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb181660c0 0x7ffb18164910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52166/0 (socket says 172.21.15.40:52166) 2024-09-13T18:02:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb1f7fe700 1 -- 172.21.15.40:0/1443840046 learned_addr learned my addr 172.21.15.40:0/1443840046 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb24d18700 1 --2- 172.21.15.40:0/1443840046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb180df580 0x7ffb18164e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb1f7fe700 1 -- 172.21.15.40:0/1443840046 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb10004870 msgr2=0x7ffb18053750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:02:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb1f7fe700 1 --2- 172.21.15.40:0/1443840046 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb10004870 0x7ffb18053750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb1f7fe700 1 -- 172.21.15.40:0/1443840046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb180df580 msgr2=0x7ffb18164e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb1f7fe700 1 --2- 172.21.15.40:0/1443840046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb180df580 0x7ffb18164e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb1f7fe700 1 -- 172.21.15.40:0/1443840046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb18178530 con 0x7ffb181660c0 2024-09-13T18:02:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb24d18700 1 --2- 172.21.15.40:0/1443840046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb180df580 0x7ffb18164e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:02:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb1f7fe700 1 --2- 172.21.15.40:0/1443840046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb181660c0 0x7ffb18164910 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000c1d0 tx=0x7ffb0c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb1cff9700 1 -- 172.21.15.40:0/1443840046 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c004890 con 0x7ffb181660c0 2024-09-13T18:02:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb1cff9700 1 -- 172.21.15.40:0/1443840046 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0c004a10 con 0x7ffb181660c0 2024-09-13T18:02:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb25519700 1 -- 172.21.15.40:0/1443840046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb18178810 con 0x7ffb181660c0 2024-09-13T18:02:14.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb1cff9700 1 -- 172.21.15.40:0/1443840046 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c004dc0 con 0x7ffb181660c0 2024-09-13T18:02:14.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.103+0000 7ffb25519700 1 -- 172.21.15.40:0/1443840046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb180d56c0 con 0x7ffb181660c0 2024-09-13T18:02:14.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.107+0000 7ffb25519700 1 -- 172.21.15.40:0/1443840046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb00005310 con 0x7ffb181660c0 2024-09-13T18:02:14.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.107+0000 7ffb1cff9700 1 -- 172.21.15.40:0/1443840046 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffb0c019050 con 0x7ffb181660c0 2024-09-13T18:02:14.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.107+0000 7ffb1cff9700 1 --2- 172.21.15.40:0/1443840046 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffb040666e0 0x7ffb04068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:14.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.107+0000 7ffb1cff9700 1 -- 172.21.15.40:0/1443840046 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ffb0c085110 con 0x7ffb181660c0 2024-09-13T18:02:14.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.107+0000 7ffb1ffff700 1 --2- 172.21.15.40:0/1443840046 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffb040666e0 0x7ffb04068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:14.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.107+0000 7ffb1ffff700 1 --2- 172.21.15.40:0/1443840046 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffb040666e0 0x7ffb04068ba0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ffb14001300 tx=0x7ffb14007a80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:14.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.115+0000 7ffb1cff9700 1 -- 172.21.15.40:0/1443840046 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ffb0c052150 con 0x7ffb181660c0 2024-09-13T18:02:14.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.355+0000 7ffb25519700 1 -- 172.21.15.40:0/1443840046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ffb000055c0 con 0x7ffb181660c0 2024-09-13T18:02:14.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.355+0000 7ffb1cff9700 1 -- 172.21.15.40:0/1443840046 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7ffb0c055770 con 0x7ffb181660c0 2024-09-13T18:02:14.367 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":88,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:02:14.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.359+0000 7ffb25519700 1 -- 172.21.15.40:0/1443840046 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffb040666e0 msgr2=0x7ffb04068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:14.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.359+0000 7ffb25519700 1 --2- 172.21.15.40:0/1443840046 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffb040666e0 0x7ffb04068ba0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ffb14001300 tx=0x7ffb14007a80 comp rx=0 tx=0).stop 2024-09-13T18:02:14.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.359+0000 7ffb25519700 1 -- 172.21.15.40:0/1443840046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb181660c0 msgr2=0x7ffb18164910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:14.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.359+0000 7ffb25519700 1 --2- 172.21.15.40:0/1443840046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb181660c0 0x7ffb18164910 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000c1d0 tx=0x7ffb0c001930 comp rx=0 tx=0).stop 2024-09-13T18:02:14.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.359+0000 7ffb25519700 1 -- 172.21.15.40:0/1443840046 shutdown_connections 2024-09-13T18:02:14.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.359+0000 7ffb25519700 1 --2- 172.21.15.40:0/1443840046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb180df580 0x7ffb18164e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:14.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.359+0000 7ffb25519700 1 --2- 172.21.15.40:0/1443840046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb181660c0 0x7ffb18164910 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:14.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.359+0000 7ffb25519700 1 --2- 172.21.15.40:0/1443840046 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffb040666e0 0x7ffb04068ba0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:14.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.359+0000 7ffb25519700 1 --2- 172.21.15.40:0/1443840046 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb10004870 0x7ffb18053750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:14.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.359+0000 7ffb25519700 1 -- 172.21.15.40:0/1443840046 >> 172.21.15.40:0/1443840046 conn(0x7ffb18054110 msgr2=0x7ffb180382a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:14.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.359+0000 7ffb25519700 1 -- 172.21.15.40:0/1443840046 shutdown_connections 2024-09-13T18:02:14.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:14.359+0000 7ffb25519700 1 -- 172.21.15.40:0/1443840046 wait complete. 2024-09-13T18:02:17.384 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:02:17.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.499+0000 7f6ea8f5d700 1 Processor -- start 2024-09-13T18:02:17.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.499+0000 7f6ea8f5d700 1 -- start start 2024-09-13T18:02:17.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.499+0000 7f6ea8f5d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6ea4153840 0x7f6ea40c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:17.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.499+0000 7f6ea8f5d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6ea40cc4f0 0x7f6ea415d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:17.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.499+0000 7f6ea8f5d700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea40cc060 con 0x7f6ea40c9e20 2024-09-13T18:02:17.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.499+0000 7f6ea8f5d700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea40b2840 con 0x7f6ea40cc4f0 2024-09-13T18:02:17.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.499+0000 7f6ea8f5d700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea40ca870 con 0x7f6ea4153840 2024-09-13T18:02:17.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.499+0000 7f6ea3fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6ea40c9e20 0x7f6ea40cf260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33372/0 (socket says 172.21.15.40:33372) 2024-09-13T18:02:17.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.499+0000 7f6ea3fff700 1 -- 172.21.15.40:0/1823129669 learned_addr learned my addr 172.21.15.40:0/1823129669 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:02:17.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.499+0000 7f6ea2ffd700 1 --2- 172.21.15.40:0/1823129669 >> v2:172.21.15.40:3300/0 conn(0x7f6ea40cc4f0 0x7f6ea415d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:17.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.499+0000 7f6ea37fe700 1 --2- 172.21.15.40:0/1823129669 >> v2:172.21.15.43:3300/0 conn(0x7f6ea4153840 0x7f6ea40c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:17.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.499+0000 7f6ea2ffd700 1 -- 172.21.15.40:0/1823129669 >> v2:172.21.15.43:3300/0 conn(0x7f6ea4153840 msgr2=0x7f6ea40c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:17.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.499+0000 7f6ea2ffd700 1 --2- 172.21.15.40:0/1823129669 >> v2:172.21.15.43:3300/0 conn(0x7f6ea4153840 0x7f6ea40c3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:17.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.499+0000 7f6ea2ffd700 1 -- 172.21.15.40:0/1823129669 >> v1:172.21.15.40:6789/0 conn(0x7f6ea40c9e20 legacy=0x7f6ea40cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:17.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.499+0000 7f6ea27fc700 1 -- 172.21.15.40:0/1823129669 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 370930335 0 0) 0x7f6ea40cc060 con 0x7f6ea40c9e20 2024-09-13T18:02:17.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.499+0000 7f6ea2ffd700 1 -- 172.21.15.40:0/1823129669 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea4153c20 con 0x7f6ea40cc4f0 2024-09-13T18:02:17.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.499+0000 7f6ea2ffd700 1 --2- 172.21.15.40:0/1823129669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea40cc4f0 0x7f6ea415d410 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6ea408c410 tx=0x7f6e98009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86268904ff437f98 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:17.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.499+0000 7f6ea27fc700 1 -- 172.21.15.40:0/1823129669 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e98011830 con 0x7f6ea40cc4f0 2024-09-13T18:02:17.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.499+0000 7f6ea27fc700 1 -- 172.21.15.40:0/1823129669 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e98011990 con 0x7f6ea40cc4f0 2024-09-13T18:02:17.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.499+0000 7f6ea27fc700 1 -- 172.21.15.40:0/1823129669 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e98011d20 con 0x7f6ea40cc4f0 2024-09-13T18:02:17.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.503+0000 7f6ea8f5d700 1 -- 172.21.15.40:0/1823129669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea40cc4f0 msgr2=0x7f6ea415d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:17.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.503+0000 7f6ea8f5d700 1 --2- 172.21.15.40:0/1823129669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea40cc4f0 0x7f6ea415d410 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6ea408c410 tx=0x7f6e98009b30 comp rx=0 tx=0).stop 2024-09-13T18:02:17.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.503+0000 7f6ea8f5d700 1 -- 172.21.15.40:0/1823129669 shutdown_connections 2024-09-13T18:02:17.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.503+0000 7f6ea8f5d700 1 --2- 172.21.15.40:0/1823129669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea40cc4f0 0x7f6ea415d410 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:17.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.503+0000 7f6ea8f5d700 1 --2- 172.21.15.40:0/1823129669 >> v2:172.21.15.43:3300/0 conn(0x7f6ea4153840 0x7f6ea40c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:17.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.503+0000 7f6ea8f5d700 1 -- 172.21.15.40:0/1823129669 >> 172.21.15.40:0/1823129669 conn(0x7f6ea406c800 msgr2=0x7f6ea406cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:17.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.503+0000 7f6ea8f5d700 1 -- 172.21.15.40:0/1823129669 shutdown_connections 2024-09-13T18:02:17.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.503+0000 7f6ea8f5d700 1 -- 172.21.15.40:0/1823129669 wait complete. 2024-09-13T18:02:17.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.503+0000 7f6ea8f5d700 1 Processor -- start 2024-09-13T18:02:17.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.503+0000 7f6ea8f5d700 1 -- start start 2024-09-13T18:02:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.503+0000 7f6ea8f5d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ea40c9e20 0x7f6ea4145c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.503+0000 7f6ea8f5d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea40cc4f0 0x7f6ea41f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.503+0000 7f6ea8f5d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ea4153840 0x7f6ea41f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:17.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.503+0000 7f6ea8f5d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea41414c0 con 0x7f6ea40cc4f0 2024-09-13T18:02:17.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.503+0000 7f6ea8f5d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea41e3e90 con 0x7f6ea40c9e20 2024-09-13T18:02:17.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.503+0000 7f6ea8f5d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea41f2750 con 0x7f6ea4153840 2024-09-13T18:02:17.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.507+0000 7f6ea2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea40cc4f0 0x7f6ea41f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:17.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.507+0000 7f6ea3fff700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ea4153840 msgr2=0x7f6ea41f3120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:02:17.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.507+0000 7f6ea2ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea40cc4f0 0x7f6ea41f2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52180/0 (socket says 172.21.15.40:52180) 2024-09-13T18:02:17.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.507+0000 7f6ea2ffd700 1 -- 172.21.15.40:0/3085630435 learned_addr learned my addr 172.21.15.40:0/3085630435 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:17.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.507+0000 7f6ea3fff700 1 --2- 172.21.15.40:0/3085630435 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ea4153840 0x7f6ea41f3120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:02:17.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.507+0000 7f6ea37fe700 1 --2- 172.21.15.40:0/3085630435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ea40c9e20 0x7f6ea4145c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:17.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.507+0000 7f6ea2ffd700 1 -- 172.21.15.40:0/3085630435 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ea4153840 msgr2=0x7f6ea41f3120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:02:17.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.507+0000 7f6ea2ffd700 1 --2- 172.21.15.40:0/3085630435 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ea4153840 0x7f6ea41f3120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:17.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.507+0000 7f6ea2ffd700 1 -- 172.21.15.40:0/3085630435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ea40c9e20 msgr2=0x7f6ea4145c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:17.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.507+0000 7f6ea2ffd700 1 --2- 172.21.15.40:0/3085630435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ea40c9e20 0x7f6ea4145c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:17.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.507+0000 7f6ea2ffd700 1 -- 172.21.15.40:0/3085630435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e980094b0 con 0x7f6ea40cc4f0 2024-09-13T18:02:17.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.507+0000 7f6ea2ffd700 1 --2- 172.21.15.40:0/3085630435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea40cc4f0 0x7f6ea41f2be0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6e980090b0 tx=0x7f6e98004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:17.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.507+0000 7f6e8bfff700 1 -- 172.21.15.40:0/3085630435 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e98004570 con 0x7f6ea40cc4f0 2024-09-13T18:02:17.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.507+0000 7f6e8bfff700 1 -- 172.21.15.40:0/3085630435 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9801f070 con 0x7f6ea40cc4f0 2024-09-13T18:02:17.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.507+0000 7f6ea8f5d700 1 -- 172.21.15.40:0/3085630435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea41e4980 con 0x7f6ea40cc4f0 2024-09-13T18:02:17.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.507+0000 7f6e8bfff700 1 -- 172.21.15.40:0/3085630435 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e980038e0 con 0x7f6ea40cc4f0 2024-09-13T18:02:17.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.507+0000 7f6ea8f5d700 1 -- 172.21.15.40:0/3085630435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea41e43c0 con 0x7f6ea40cc4f0 2024-09-13T18:02:17.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.507+0000 7f6ea8f5d700 1 -- 172.21.15.40:0/3085630435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea40cba10 con 0x7f6ea40cc4f0 2024-09-13T18:02:17.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.507+0000 7f6e8bfff700 1 -- 172.21.15.40:0/3085630435 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6e98021030 con 0x7f6ea40cc4f0 2024-09-13T18:02:17.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.511+0000 7f6e8bfff700 1 --2- 172.21.15.40:0/3085630435 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e900666e0 0x7f6e90068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:17.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.511+0000 7f6ea37fe700 1 --2- 172.21.15.40:0/3085630435 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e900666e0 0x7f6e90068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:17.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.511+0000 7f6e8bfff700 1 -- 172.21.15.40:0/3085630435 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6e98085560 con 0x7f6ea40cc4f0 2024-09-13T18:02:17.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.511+0000 7f6ea37fe700 1 --2- 172.21.15.40:0/3085630435 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e900666e0 0x7f6e90068ba0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009ae0 tx=0x7f6e94008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:17.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.515+0000 7f6e8bfff700 1 -- 172.21.15.40:0/3085630435 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6e980523e0 con 0x7f6ea40cc4f0 2024-09-13T18:02:17.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.807+0000 7f6ea8f5d700 1 -- 172.21.15.40:0/3085630435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f6ea41f2890 con 0x7f6ea40cc4f0 2024-09-13T18:02:17.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.807+0000 7f6e8bfff700 1 -- 172.21.15.40:0/3085630435 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f6e98015080 con 0x7f6ea40cc4f0 2024-09-13T18:02:17.817 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":88,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:02:17.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.811+0000 7f6ea8f5d700 1 -- 172.21.15.40:0/3085630435 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e900666e0 msgr2=0x7f6e90068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:17.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.811+0000 7f6ea8f5d700 1 --2- 172.21.15.40:0/3085630435 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e900666e0 0x7f6e90068ba0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009ae0 tx=0x7f6e94008040 comp rx=0 tx=0).stop 2024-09-13T18:02:17.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.811+0000 7f6ea8f5d700 1 -- 172.21.15.40:0/3085630435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea40cc4f0 msgr2=0x7f6ea41f2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:17.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.811+0000 7f6ea8f5d700 1 --2- 172.21.15.40:0/3085630435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea40cc4f0 0x7f6ea41f2be0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6e980090b0 tx=0x7f6e98004770 comp rx=0 tx=0).stop 2024-09-13T18:02:17.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.811+0000 7f6ea8f5d700 1 -- 172.21.15.40:0/3085630435 shutdown_connections 2024-09-13T18:02:17.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.811+0000 7f6ea8f5d700 1 --2- 172.21.15.40:0/3085630435 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e900666e0 0x7f6e90068ba0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:17.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.811+0000 7f6ea8f5d700 1 --2- 172.21.15.40:0/3085630435 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ea4153840 0x7f6ea41f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:17.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.811+0000 7f6ea8f5d700 1 --2- 172.21.15.40:0/3085630435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ea40cc4f0 0x7f6ea41f2be0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:17.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.811+0000 7f6ea8f5d700 1 --2- 172.21.15.40:0/3085630435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ea40c9e20 0x7f6ea4145c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:17.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.811+0000 7f6ea8f5d700 1 -- 172.21.15.40:0/3085630435 >> 172.21.15.40:0/3085630435 conn(0x7f6ea406c800 msgr2=0x7f6ea40b3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:17.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.811+0000 7f6ea8f5d700 1 -- 172.21.15.40:0/3085630435 shutdown_connections 2024-09-13T18:02:17.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:17.811+0000 7f6ea8f5d700 1 -- 172.21.15.40:0/3085630435 wait complete. 2024-09-13T18:02:20.838 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:02:20.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.951+0000 7feea135d700 1 Processor -- start 2024-09-13T18:02:20.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.951+0000 7feea135d700 1 -- start start 2024-09-13T18:02:20.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.951+0000 7feea135d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fee940d8b50 0x7fee9404ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:20.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.951+0000 7feea135d700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee94050310 con 0x7fee94049000 2024-09-13T18:02:20.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.951+0000 7feea135d700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee940505d0 con 0x7fee94049420 2024-09-13T18:02:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.951+0000 7feea135d700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee940e2b80 con 0x7fee940d8b50 2024-09-13T18:02:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.951+0000 7fee9bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fee940d8b50 0x7fee9404ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.951+0000 7fee9bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fee940d8b50 0x7fee9404ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52192/0 (socket says 172.21.15.40:52192) 2024-09-13T18:02:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.951+0000 7fee9bfff700 1 -- 172.21.15.40:0/3604119402 learned_addr learned my addr 172.21.15.40:0/3604119402 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.951+0000 7fee9bfff700 1 -- 172.21.15.40:0/3604119402 >> v1:172.21.15.43:6789/0 conn(0x7fee94049420 legacy=0x7fee940e2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:20.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.951+0000 7fee9bfff700 1 -- 172.21.15.40:0/3604119402 >> v1:172.21.15.40:6789/0 conn(0x7fee94049000 legacy=0x7fee940e1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:20.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.951+0000 7fee9bfff700 1 -- 172.21.15.40:0/3604119402 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee940508b0 con 0x7fee940d8b50 2024-09-13T18:02:20.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.955+0000 7fee9bfff700 1 --2- 172.21.15.40:0/3604119402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee940d8b50 0x7fee9404ef10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fee90009910 tx=0x7fee90009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e620faafe5d4036b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:20.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.955+0000 7fee9affd700 1 -- 172.21.15.40:0/3604119402 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9000ba60 con 0x7fee940d8b50 2024-09-13T18:02:20.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.955+0000 7fee9affd700 1 -- 172.21.15.40:0/3604119402 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9000bbc0 con 0x7fee940d8b50 2024-09-13T18:02:20.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.955+0000 7fee9affd700 1 -- 172.21.15.40:0/3604119402 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee90009e60 con 0x7fee940d8b50 2024-09-13T18:02:20.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.955+0000 7feea135d700 1 -- 172.21.15.40:0/3604119402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee940d8b50 msgr2=0x7fee9404ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:20.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.955+0000 7feea135d700 1 --2- 172.21.15.40:0/3604119402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee940d8b50 0x7fee9404ef10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fee90009910 tx=0x7fee90009c20 comp rx=0 tx=0).stop 2024-09-13T18:02:20.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.955+0000 7feea135d700 1 -- 172.21.15.40:0/3604119402 shutdown_connections 2024-09-13T18:02:20.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.955+0000 7feea135d700 1 --2- 172.21.15.40:0/3604119402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee940d8b50 0x7fee9404ef10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:20.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.955+0000 7feea135d700 1 -- 172.21.15.40:0/3604119402 >> 172.21.15.40:0/3604119402 conn(0x7fee94054120 msgr2=0x7fee94054530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:20.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.955+0000 7feea135d700 1 -- 172.21.15.40:0/3604119402 shutdown_connections 2024-09-13T18:02:20.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.955+0000 7feea135d700 1 -- 172.21.15.40:0/3604119402 wait complete. 2024-09-13T18:02:20.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7feea135d700 1 Processor -- start 2024-09-13T18:02:20.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7feea135d700 1 -- start start 2024-09-13T18:02:20.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7feea135d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee94049000 0x7fee9404c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:20.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7feea135d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee94049420 0x7fee940e2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:20.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7feea135d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fee940d8b50 0x7fee9404d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:20.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7feea135d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee940e0ad0 con 0x7fee94049000 2024-09-13T18:02:20.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7feea135d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9404ccd0 con 0x7fee94049420 2024-09-13T18:02:20.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7feea135d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9404b5a0 con 0x7fee940d8b50 2024-09-13T18:02:20.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7feea0b5c700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fee940d8b50 msgr2=0x7fee9404d2b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:02:20.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7feea0b5c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fee940d8b50 0x7fee9404d2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:02:20.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7fee9bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee94049000 0x7fee9404c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:20.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7fee9bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee94049000 0x7fee9404c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52204/0 (socket says 172.21.15.40:52204) 2024-09-13T18:02:20.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7fee9bfff700 1 -- 172.21.15.40:0/4054224350 learned_addr learned my addr 172.21.15.40:0/4054224350 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:20.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7fee9b7fe700 1 --2- 172.21.15.40:0/4054224350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee94049420 0x7fee940e2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:20.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7fee9bfff700 1 -- 172.21.15.40:0/4054224350 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fee940d8b50 msgr2=0x7fee9404d2b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:02:20.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7fee9bfff700 1 --2- 172.21.15.40:0/4054224350 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fee940d8b50 0x7fee9404d2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:20.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7fee9bfff700 1 -- 172.21.15.40:0/4054224350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee94049420 msgr2=0x7fee940e2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:20.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7fee9bfff700 1 --2- 172.21.15.40:0/4054224350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee94049420 0x7fee940e2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:20.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7fee9bfff700 1 -- 172.21.15.40:0/4054224350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee90009570 con 0x7fee94049000 2024-09-13T18:02:20.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7fee9bfff700 1 --2- 172.21.15.40:0/4054224350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee94049000 0x7fee9404c560 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fee90006f80 tx=0x7fee90004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:20.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7fee98ff9700 1 -- 172.21.15.40:0/4054224350 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee90004550 con 0x7fee94049000 2024-09-13T18:02:20.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7fee98ff9700 1 -- 172.21.15.40:0/4054224350 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee900036c0 con 0x7fee94049000 2024-09-13T18:02:20.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7feea135d700 1 -- 172.21.15.40:0/4054224350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9404b840 con 0x7fee94049000 2024-09-13T18:02:20.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7fee98ff9700 1 -- 172.21.15.40:0/4054224350 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee90003a70 con 0x7fee94049000 2024-09-13T18:02:20.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.959+0000 7feea135d700 1 -- 172.21.15.40:0/4054224350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee940e1000 con 0x7fee94049000 2024-09-13T18:02:20.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.963+0000 7fee98ff9700 1 -- 172.21.15.40:0/4054224350 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fee90003bd0 con 0x7fee94049000 2024-09-13T18:02:20.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.963+0000 7feea135d700 1 -- 172.21.15.40:0/4054224350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee9404cfb0 con 0x7fee94049000 2024-09-13T18:02:20.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.963+0000 7fee98ff9700 1 --2- 172.21.15.40:0/4054224350 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fee800666e0 0x7fee80068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:20.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.963+0000 7fee98ff9700 1 -- 172.21.15.40:0/4054224350 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fee90021030 con 0x7fee94049000 2024-09-13T18:02:20.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.963+0000 7fee9b7fe700 1 --2- 172.21.15.40:0/4054224350 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fee800666e0 0x7fee80068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:20.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.963+0000 7fee9b7fe700 1 --2- 172.21.15.40:0/4054224350 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fee800666e0 0x7fee80068ba0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fee940e1720 tx=0x7fee88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:20.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:20.971+0000 7fee98ff9700 1 -- 172.21.15.40:0/4054224350 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fee90052d90 con 0x7fee94049000 2024-09-13T18:02:21.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:21.251+0000 7feea135d700 1 -- 172.21.15.40:0/4054224350 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fee94007680 con 0x7fee94049000 2024-09-13T18:02:21.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:21.251+0000 7fee98ff9700 1 -- 172.21.15.40:0/4054224350 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fee9002b620 con 0x7fee94049000 2024-09-13T18:02:21.262 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":88,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:02:21.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:21.255+0000 7feea135d700 1 -- 172.21.15.40:0/4054224350 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fee800666e0 msgr2=0x7fee80068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:21.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:21.255+0000 7feea135d700 1 --2- 172.21.15.40:0/4054224350 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fee800666e0 0x7fee80068ba0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fee940e1720 tx=0x7fee88009040 comp rx=0 tx=0).stop 2024-09-13T18:02:21.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:21.255+0000 7feea135d700 1 -- 172.21.15.40:0/4054224350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee94049000 msgr2=0x7fee9404c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:21.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:21.255+0000 7feea135d700 1 --2- 172.21.15.40:0/4054224350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee94049000 0x7fee9404c560 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fee90006f80 tx=0x7fee90004770 comp rx=0 tx=0).stop 2024-09-13T18:02:21.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:21.255+0000 7feea135d700 1 -- 172.21.15.40:0/4054224350 shutdown_connections 2024-09-13T18:02:21.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:21.255+0000 7feea135d700 1 --2- 172.21.15.40:0/4054224350 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fee800666e0 0x7fee80068ba0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:21.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:21.255+0000 7feea135d700 1 --2- 172.21.15.40:0/4054224350 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fee940d8b50 0x7fee9404d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:21.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:21.255+0000 7feea135d700 1 --2- 172.21.15.40:0/4054224350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee94049420 0x7fee940e2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:21.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:21.255+0000 7feea135d700 1 --2- 172.21.15.40:0/4054224350 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fee94049000 0x7fee9404c560 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:21.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:21.255+0000 7feea135d700 1 -- 172.21.15.40:0/4054224350 >> 172.21.15.40:0/4054224350 conn(0x7fee94054120 msgr2=0x7fee940d4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:21.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:21.255+0000 7feea135d700 1 -- 172.21.15.40:0/4054224350 shutdown_connections 2024-09-13T18:02:21.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:21.255+0000 7feea135d700 1 -- 172.21.15.40:0/4054224350 wait complete. 2024-09-13T18:02:24.285 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:02:24.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.403+0000 7ff6269eb700 1 Processor -- start 2024-09-13T18:02:24.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.403+0000 7ff6269eb700 1 -- start start 2024-09-13T18:02:24.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.407+0000 7ff6269eb700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 0x7ff6180df110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:24.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.407+0000 7ff6269eb700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff618007b60 0x7ff61804b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.407+0000 7ff6269eb700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 0x7ff618046e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.407+0000 7ff6269eb700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff618047610 con 0x7ff6180469c0 2024-09-13T18:02:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.407+0000 7ff6269eb700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff618006b50 con 0x7ff6180d87a0 2024-09-13T18:02:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.407+0000 7ff6269eb700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6180076d0 con 0x7ff618007b60 2024-09-13T18:02:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.407+0000 7ff6251e8700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff618007b60 msgr2=0x7ff61804b7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:02:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.407+0000 7ff6261ea700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 0x7ff618046e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.407+0000 7ff6251e8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff618007b60 0x7ff61804b7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:02:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.407+0000 7ff6261ea700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 0x7ff618046e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46384/0 (socket says 172.21.15.40:46384) 2024-09-13T18:02:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.407+0000 7ff6261ea700 1 -- 172.21.15.40:0/3804615641 learned_addr learned my addr 172.21.15.40:0/3804615641 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.407+0000 7ff6259e9700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 0x7ff6180df110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.407+0000 7ff6261ea700 1 -- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 msgr2=0x7ff618046e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:02:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.407+0000 7ff6261ea700 1 -- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 msgr2=0x7ff618046e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:02:24.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.407+0000 7ff6261ea700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 0x7ff618046e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:02:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.407+0000 7ff6261ea700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 0x7ff618046e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:02:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.407+0000 7ff6259e9700 1 -- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 msgr2=0x7ff6180df110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:02:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.407+0000 7ff6259e9700 1 -- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 msgr2=0x7ff6180df110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:02:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.407+0000 7ff6259e9700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 0x7ff6180df110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:02:24.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.407+0000 7ff6259e9700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 0x7ff6180df110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:02:24.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.611+0000 7ff6251e8700 1 -- 172.21.15.40:0/3804615641 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff618007b60 msgr2=0x7ff61804b7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:02:24.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.611+0000 7ff6251e8700 1 --2- 172.21.15.40:0/3804615641 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff618007b60 0x7ff61804b7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:02:24.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.611+0000 7ff6261ea700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 0x7ff618046e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:24.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.611+0000 7ff6259e9700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 0x7ff6180df110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:24.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.611+0000 7ff6259e9700 1 -- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 msgr2=0x7ff6180df110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:02:24.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.611+0000 7ff6259e9700 1 -- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 msgr2=0x7ff6180df110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:02:24.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.611+0000 7ff6259e9700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 0x7ff6180df110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:02:24.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.611+0000 7ff6261ea700 1 -- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 msgr2=0x7ff618046e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:02:24.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.611+0000 7ff6261ea700 1 -- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 msgr2=0x7ff618046e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:02:24.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.611+0000 7ff6261ea700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 0x7ff618046e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:02:24.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.611+0000 7ff6259e9700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 0x7ff6180df110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:02:24.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:24.611+0000 7ff6261ea700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 0x7ff618046e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:02:25.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.015+0000 7ff6251e8700 1 -- 172.21.15.40:0/3804615641 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff618007b60 msgr2=0x7ff61804b7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:02:25.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.015+0000 7ff6251e8700 1 --2- 172.21.15.40:0/3804615641 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff618007b60 0x7ff61804b7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:02:25.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.015+0000 7ff6261ea700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 0x7ff618046e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:25.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.015+0000 7ff6259e9700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 0x7ff6180df110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:25.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.015+0000 7ff6261ea700 1 -- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 msgr2=0x7ff618046e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:02:25.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.015+0000 7ff6261ea700 1 -- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 msgr2=0x7ff618046e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:02:25.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.015+0000 7ff6261ea700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 0x7ff618046e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:02:25.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.015+0000 7ff6261ea700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 0x7ff618046e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:02:25.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.015+0000 7ff6259e9700 1 -- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 msgr2=0x7ff6180df110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:02:25.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.015+0000 7ff6259e9700 1 -- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 msgr2=0x7ff6180df110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:02:25.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.015+0000 7ff6259e9700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 0x7ff6180df110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:02:25.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.015+0000 7ff6259e9700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 0x7ff6180df110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:02:25.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.818+0000 7ff6251e8700 1 -- 172.21.15.40:0/3804615641 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff618007b60 msgr2=0x7ff61804b7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:02:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.818+0000 7ff6251e8700 1 --2- 172.21.15.40:0/3804615641 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff618007b60 0x7ff61804b7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:02:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.818+0000 7ff6261ea700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 0x7ff618046e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.818+0000 7ff6259e9700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 0x7ff6180df110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.818+0000 7ff6261ea700 1 -- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 msgr2=0x7ff618046e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:02:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.818+0000 7ff6261ea700 1 -- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 msgr2=0x7ff618046e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:02:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.818+0000 7ff6261ea700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 0x7ff618046e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:02:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.818+0000 7ff6259e9700 1 -- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 msgr2=0x7ff6180df110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:02:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.818+0000 7ff6259e9700 1 -- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 msgr2=0x7ff6180df110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:02:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.818+0000 7ff6259e9700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 0x7ff6180df110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:02:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.818+0000 7ff6261ea700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 0x7ff618046e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:02:25.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:25.818+0000 7ff6259e9700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 0x7ff6180df110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:02:27.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:27.406+0000 7ff6177fe700 1 -- 172.21.15.40:0/3804615641 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff618007b60 msgr2=0x7ff61804b7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:02:27.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:27.406+0000 7ff6177fe700 1 --2- 172.21.15.40:0/3804615641 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff618007b60 0x7ff61804b7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:27.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:27.406+0000 7ff6177fe700 1 -- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 msgr2=0x7ff6180df110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:02:27.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:27.406+0000 7ff6177fe700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.43:3300/0 conn(0x7ff6180d87a0 0x7ff6180df110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:27.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:27.406+0000 7ff6177fe700 1 -- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 msgr2=0x7ff618046e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:02:27.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:27.406+0000 7ff6177fe700 1 --2- 172.21.15.40:0/3804615641 >> v2:172.21.15.40:3300/0 conn(0x7ff6180469c0 0x7ff618046e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:27.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:27.406+0000 7ff6177fe700 1 -- 172.21.15.40:0/3804615641 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6180d8b80 con 0x7ff604003710 2024-09-13T18:02:27.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:27.406+0000 7ff6177fe700 1 -- 172.21.15.40:0/3804615641 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60400c110 con 0x7ff6040073a0 2024-09-13T18:02:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.730+0000 7ff6249e7700 1 -- 172.21.15.40:0/3804615641 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723555040 0 0) 0x7ff60400c110 con 0x7ff6040073a0 2024-09-13T18:02:28.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.730+0000 7ff6249e7700 1 -- 172.21.15.40:0/3804615641 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff60c0035d0 con 0x7ff6040073a0 2024-09-13T18:02:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.734+0000 7ff6249e7700 1 -- 172.21.15.40:0/3804615641 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 138115992 0 0) 0x7ff6180d8b80 con 0x7ff604003710 2024-09-13T18:02:28.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.734+0000 7ff6249e7700 1 -- 172.21.15.40:0/3804615641 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff60400c110 con 0x7ff604003710 2024-09-13T18:02:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.734+0000 7ff6249e7700 1 -- 172.21.15.40:0/3804615641 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 895202131 0 0) 0x7ff60400c110 con 0x7ff604003710 2024-09-13T18:02:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.734+0000 7ff6249e7700 1 -- 172.21.15.40:0/3804615641 >> v1:172.21.15.43:6789/0 conn(0x7ff6040073a0 legacy=0x7ff604009860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.734+0000 7ff6249e7700 1 -- 172.21.15.40:0/3804615641 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60400d3d0 con 0x7ff604003710 2024-09-13T18:02:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.734+0000 7ff6249e7700 1 -- 172.21.15.40:0/3804615641 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7ff61c00c4f0 con 0x7ff604003710 2024-09-13T18:02:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.734+0000 7ff6249e7700 1 -- 172.21.15.40:0/3804615641 >> v1:172.21.15.40:6789/0 conn(0x7ff604003710 legacy=0x7ff604005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.734+0000 7ff6261ea700 1 -- 172.21.15.40:0/3804615641 reap_dead start 2024-09-13T18:02:28.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.734+0000 7ff6249e7700 1 --2- 172.21.15.40:0/3804615641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff60c003be0 0x7ff60c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:28.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.734+0000 7ff6249e7700 1 --2- 172.21.15.40:0/3804615641 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff60c004870 0x7ff60c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:28.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.734+0000 7ff6249e7700 1 --2- 172.21.15.40:0/3804615641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c005350 0x7ff60c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:28.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.734+0000 7ff6249e7700 1 -- 172.21.15.40:0/3804615641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c00c240 con 0x7ff60c005350 2024-09-13T18:02:28.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.734+0000 7ff6249e7700 1 -- 172.21.15.40:0/3804615641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c00c3b0 con 0x7ff60c003be0 2024-09-13T18:02:28.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.734+0000 7ff6249e7700 1 -- 172.21.15.40:0/3804615641 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c00c570 con 0x7ff60c004870 2024-09-13T18:02:28.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.738+0000 7ff6251e8700 1 -- 172.21.15.40:0/3804615641 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff60c004870 msgr2=0x7ff60c004cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:02:28.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.738+0000 7ff6251e8700 1 --2- 172.21.15.40:0/3804615641 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff60c004870 0x7ff60c004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:02:28.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.738+0000 7ff6261ea700 1 --2- 172.21.15.40:0/3804615641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c005350 0x7ff60c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:28.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.738+0000 7ff6259e9700 1 --2- 172.21.15.40:0/3804615641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff60c003be0 0x7ff60c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:28.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.738+0000 7ff6261ea700 1 -- 172.21.15.40:0/3804615641 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff60c004870 msgr2=0x7ff60c004cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:02:28.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.738+0000 7ff6261ea700 1 --2- 172.21.15.40:0/3804615641 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff60c004870 0x7ff60c004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:28.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.738+0000 7ff6261ea700 1 -- 172.21.15.40:0/3804615641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff60c003be0 msgr2=0x7ff60c003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:28.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.738+0000 7ff6261ea700 1 --2- 172.21.15.40:0/3804615641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff60c003be0 0x7ff60c003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:28.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.738+0000 7ff6261ea700 1 -- 172.21.15.40:0/3804615641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60400c110 con 0x7ff60c005350 2024-09-13T18:02:28.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.738+0000 7ff6261ea700 1 --2- 172.21.15.40:0/3804615641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c005350 0x7ff60c00ba30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff62004e9d0 tx=0x7ff60c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482ea13db9fe64ea server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:28.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.738+0000 7ff6249e7700 1 -- 172.21.15.40:0/3804615641 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62004f050 con 0x7ff60c005350 2024-09-13T18:02:28.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.738+0000 7ff6249e7700 1 -- 172.21.15.40:0/3804615641 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff620072040 con 0x7ff60c005350 2024-09-13T18:02:28.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.738+0000 7ff6249e7700 1 -- 172.21.15.40:0/3804615641 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff620067dd0 con 0x7ff60c005350 2024-09-13T18:02:28.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.742+0000 7ff6269eb700 1 -- 172.21.15.40:0/3804615641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c005350 msgr2=0x7ff60c00ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:28.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.742+0000 7ff6269eb700 1 --2- 172.21.15.40:0/3804615641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c005350 0x7ff60c00ba30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff62004e9d0 tx=0x7ff60c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:02:28.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.742+0000 7ff6269eb700 1 -- 172.21.15.40:0/3804615641 shutdown_connections 2024-09-13T18:02:28.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.742+0000 7ff6269eb700 1 --2- 172.21.15.40:0/3804615641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c005350 0x7ff60c00ba30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:28.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.742+0000 7ff6269eb700 1 --2- 172.21.15.40:0/3804615641 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff60c004870 0x7ff60c004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:28.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.742+0000 7ff6269eb700 1 --2- 172.21.15.40:0/3804615641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff60c003be0 0x7ff60c003fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:28.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.742+0000 7ff6269eb700 1 -- 172.21.15.40:0/3804615641 >> 172.21.15.40:0/3804615641 conn(0x7ff618050e60 msgr2=0x7ff618051270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:28.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.742+0000 7ff6269eb700 1 -- 172.21.15.40:0/3804615641 shutdown_connections 2024-09-13T18:02:28.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.742+0000 7ff6269eb700 1 -- 172.21.15.40:0/3804615641 wait complete. 2024-09-13T18:02:28.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.742+0000 7ff6269eb700 1 Processor -- start 2024-09-13T18:02:28.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.742+0000 7ff6269eb700 1 -- start start 2024-09-13T18:02:28.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6269eb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c003be0 0x7ff618048820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:28.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6269eb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff60c004870 0x7ff618048d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:28.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6269eb700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff60c005350 0x7ff61804a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:28.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6269eb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6180d0230 con 0x7ff60c003be0 2024-09-13T18:02:28.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6269eb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6180cfa20 con 0x7ff60c004870 2024-09-13T18:02:28.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6269eb700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff618048390 con 0x7ff60c005350 2024-09-13T18:02:28.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6261ea700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff60c005350 msgr2=0x7ff61804a5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:02:28.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6261ea700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff60c005350 0x7ff61804a5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:02:28.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6259e9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c003be0 0x7ff618048820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:28.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6259e9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c003be0 0x7ff618048820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46436/0 (socket says 172.21.15.40:46436) 2024-09-13T18:02:28.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6259e9700 1 -- 172.21.15.40:0/3581129469 learned_addr learned my addr 172.21.15.40:0/3581129469 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:28.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6251e8700 1 --2- 172.21.15.40:0/3581129469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff60c004870 0x7ff618048d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:28.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6259e9700 1 -- 172.21.15.40:0/3581129469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff60c005350 msgr2=0x7ff61804a5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:02:28.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6259e9700 1 --2- 172.21.15.40:0/3581129469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff60c005350 0x7ff61804a5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:28.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6259e9700 1 -- 172.21.15.40:0/3581129469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff60c004870 msgr2=0x7ff618048d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:28.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6259e9700 1 --2- 172.21.15.40:0/3581129469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff60c004870 0x7ff618048d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:28.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6259e9700 1 -- 172.21.15.40:0/3581129469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6180d2490 con 0x7ff60c003be0 2024-09-13T18:02:28.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6259e9700 1 --2- 172.21.15.40:0/3581129469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c003be0 0x7ff618048820 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff61c016a10 tx=0x7ff61c016d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:28.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6167fc700 1 -- 172.21.15.40:0/3581129469 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c014b90 con 0x7ff60c003be0 2024-09-13T18:02:28.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6167fc700 1 -- 172.21.15.40:0/3581129469 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff61c00c040 con 0x7ff60c003be0 2024-09-13T18:02:28.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6269eb700 1 -- 172.21.15.40:0/3581129469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6180d19c0 con 0x7ff60c003be0 2024-09-13T18:02:28.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6167fc700 1 -- 172.21.15.40:0/3581129469 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c018450 con 0x7ff60c003be0 2024-09-13T18:02:28.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.746+0000 7ff6269eb700 1 -- 172.21.15.40:0/3581129469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6180cff50 con 0x7ff60c003be0 2024-09-13T18:02:28.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.750+0000 7ff6269eb700 1 -- 172.21.15.40:0/3581129469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6180d3170 con 0x7ff60c003be0 2024-09-13T18:02:28.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.750+0000 7ff6167fc700 1 -- 172.21.15.40:0/3581129469 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff61c0185b0 con 0x7ff60c003be0 2024-09-13T18:02:28.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.750+0000 7ff6167fc700 1 --2- 172.21.15.40:0/3581129469 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff604066f80 0x7ff604069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:28.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.750+0000 7ff6167fc700 1 -- 172.21.15.40:0/3581129469 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff61c000e40 con 0x7ff60c003be0 2024-09-13T18:02:28.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.750+0000 7ff6251e8700 1 --2- 172.21.15.40:0/3581129469 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff604066f80 0x7ff604069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:28.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.750+0000 7ff6251e8700 1 --2- 172.21.15.40:0/3581129469 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff604066f80 0x7ff604069440 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff6180d45c0 tx=0x7ff6100091c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:28.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:28.758+0000 7ff6167fc700 1 -- 172.21.15.40:0/3581129469 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff61c0523a0 con 0x7ff60c003be0 2024-09-13T18:02:29.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.006+0000 7ff6269eb700 1 -- 172.21.15.40:0/3581129469 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff61804d310 con 0x7ff60c003be0 2024-09-13T18:02:29.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.010+0000 7ff6167fc700 1 -- 172.21.15.40:0/3581129469 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7ff61c0559c0 con 0x7ff60c003be0 2024-09-13T18:02:29.020 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":92,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:02:29.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.010+0000 7ff6269eb700 1 -- 172.21.15.40:0/3581129469 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff604066f80 msgr2=0x7ff604069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:29.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.010+0000 7ff6269eb700 1 --2- 172.21.15.40:0/3581129469 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff604066f80 0x7ff604069440 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff6180d45c0 tx=0x7ff6100091c0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.010+0000 7ff6269eb700 1 -- 172.21.15.40:0/3581129469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c003be0 msgr2=0x7ff618048820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:29.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.010+0000 7ff6269eb700 1 --2- 172.21.15.40:0/3581129469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c003be0 0x7ff618048820 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff61c016a10 tx=0x7ff61c016d20 comp rx=0 tx=0).stop 2024-09-13T18:02:29.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.010+0000 7ff6269eb700 1 -- 172.21.15.40:0/3581129469 shutdown_connections 2024-09-13T18:02:29.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.010+0000 7ff6269eb700 1 --2- 172.21.15.40:0/3581129469 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff604066f80 0x7ff604069440 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.010+0000 7ff6269eb700 1 --2- 172.21.15.40:0/3581129469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff60c005350 0x7ff61804a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.010+0000 7ff6269eb700 1 --2- 172.21.15.40:0/3581129469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff60c004870 0x7ff618048d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.010+0000 7ff6269eb700 1 --2- 172.21.15.40:0/3581129469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60c003be0 0x7ff618048820 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.010+0000 7ff6269eb700 1 -- 172.21.15.40:0/3581129469 >> 172.21.15.40:0/3581129469 conn(0x7ff618050e60 msgr2=0x7ff6180d5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:29.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.010+0000 7ff6269eb700 1 -- 172.21.15.40:0/3581129469 shutdown_connections 2024-09-13T18:02:29.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.010+0000 7ff6269eb700 1 -- 172.21.15.40:0/3581129469 wait complete. 2024-09-13T18:02:29.034 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T18:02:29.035 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-13T18:02:29.035 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:02:29.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.142+0000 7ff2cffff700 1 Processor -- start 2024-09-13T18:02:29.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.142+0000 7ff2cffff700 1 -- start start 2024-09-13T18:02:29.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2cffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff2d0153c30 0x7ff2d00b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:29.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2cffff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff2d00cac60 0x7ff2d00c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:29.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2cffff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d01596a0 con 0x7ff2d00ca440 2024-09-13T18:02:29.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2cffff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d00b20a0 con 0x7ff2d0153c30 2024-09-13T18:02:29.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2cffff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d0159960 con 0x7ff2d00cac60 2024-09-13T18:02:29.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2ceffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff2d0153c30 0x7ff2d00b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:29.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2ceffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff2d0153c30 0x7ff2d00b1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46452/0 (socket says 172.21.15.40:46452) 2024-09-13T18:02:29.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2ceffd700 1 -- 172.21.15.40:0/1125715139 learned_addr learned my addr 172.21.15.40:0/1125715139 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:29.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2cf7fe700 1 --1- 172.21.15.40:0/1125715139 >> v1:172.21.15.40:6789/0 conn(0x7ff2d00ca440 0x7ff2d00cb9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56696/0 (socket says 172.21.15.40:56696) 2024-09-13T18:02:29.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2ce7fc700 1 --2- 172.21.15.40:0/1125715139 >> v2:172.21.15.43:3300/0 conn(0x7ff2d00cac60 0x7ff2d00c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:29.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2ceffd700 1 -- 172.21.15.40:0/1125715139 >> v2:172.21.15.43:3300/0 conn(0x7ff2d00cac60 msgr2=0x7ff2d00c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:29.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2ceffd700 1 --2- 172.21.15.40:0/1125715139 >> v2:172.21.15.43:3300/0 conn(0x7ff2d00cac60 0x7ff2d00c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2ceffd700 1 -- 172.21.15.40:0/1125715139 >> v1:172.21.15.40:6789/0 conn(0x7ff2d00ca440 legacy=0x7ff2d00cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:29.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2ceffd700 1 -- 172.21.15.40:0/1125715139 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d0159c40 con 0x7ff2d0153c30 2024-09-13T18:02:29.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2ceffd700 1 --2- 172.21.15.40:0/1125715139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d0153c30 0x7ff2d00b1b60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00c9e60 tx=0x7ff2c0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2aae33797add900 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:29.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2cdffb700 1 -- 172.21.15.40:0/1125715139 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0011830 con 0x7ff2d0153c30 2024-09-13T18:02:29.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2cdffb700 1 -- 172.21.15.40:0/1125715139 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c0011990 con 0x7ff2d0153c30 2024-09-13T18:02:29.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2cdffb700 1 -- 172.21.15.40:0/1125715139 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0011d20 con 0x7ff2d0153c30 2024-09-13T18:02:29.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2cffff700 1 -- 172.21.15.40:0/1125715139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d0153c30 msgr2=0x7ff2d00b1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:29.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2cffff700 1 --2- 172.21.15.40:0/1125715139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d0153c30 0x7ff2d00b1b60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00c9e60 tx=0x7ff2c0009bf0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2cffff700 1 -- 172.21.15.40:0/1125715139 shutdown_connections 2024-09-13T18:02:29.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2cffff700 1 --2- 172.21.15.40:0/1125715139 >> v2:172.21.15.43:3300/0 conn(0x7ff2d00cac60 0x7ff2d00c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2cffff700 1 --2- 172.21.15.40:0/1125715139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d0153c30 0x7ff2d00b1b60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2cffff700 1 -- 172.21.15.40:0/1125715139 >> 172.21.15.40:0/1125715139 conn(0x7ff2d0066090 msgr2=0x7ff2d00664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:29.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2cffff700 1 -- 172.21.15.40:0/1125715139 shutdown_connections 2024-09-13T18:02:29.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.146+0000 7ff2cffff700 1 -- 172.21.15.40:0/1125715139 wait complete. 2024-09-13T18:02:29.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2cffff700 1 Processor -- start 2024-09-13T18:02:29.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2cffff700 1 -- start start 2024-09-13T18:02:29.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2cffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2d00ca440 0x7ff2d00ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:29.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2cffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d00cac60 0x7ff2d00c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:29.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2cffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff2d0153c30 0x7ff2d00c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:29.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2cffff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d01f36c0 con 0x7ff2d00cac60 2024-09-13T18:02:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2cffff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d00c5690 con 0x7ff2d00ca440 2024-09-13T18:02:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2cffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d00c4470 con 0x7ff2d0153c30 2024-09-13T18:02:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2cf7fe700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff2d0153c30 msgr2=0x7ff2d00c7820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:02:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2ce7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d00cac60 0x7ff2d00c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2cf7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff2d0153c30 0x7ff2d00c7820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:02:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2ce7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d00cac60 0x7ff2d00c3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46464/0 (socket says 172.21.15.40:46464) 2024-09-13T18:02:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2ce7fc700 1 -- 172.21.15.40:0/31534423 learned_addr learned my addr 172.21.15.40:0/31534423 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2ceffd700 1 --2- 172.21.15.40:0/31534423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2d00ca440 0x7ff2d00ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2ce7fc700 1 -- 172.21.15.40:0/31534423 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff2d0153c30 msgr2=0x7ff2d00c7820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:02:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2ce7fc700 1 --2- 172.21.15.40:0/31534423 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff2d0153c30 0x7ff2d00c7820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2ce7fc700 1 -- 172.21.15.40:0/31534423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2d00ca440 msgr2=0x7ff2d00ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2ce7fc700 1 --2- 172.21.15.40:0/31534423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2d00ca440 0x7ff2d00ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2ce7fc700 1 -- 172.21.15.40:0/31534423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c0009570 con 0x7ff2d00cac60 2024-09-13T18:02:29.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2ce7fc700 1 --2- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d00cac60 0x7ff2d00c3f30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400e9d0 tx=0x7ff2c400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:29.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2b77fe700 1 -- 172.21.15.40:0/31534423 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4009720 con 0x7ff2d00cac60 2024-09-13T18:02:29.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2b77fe700 1 -- 172.21.15.40:0/31534423 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c400c850 con 0x7ff2d00cac60 2024-09-13T18:02:29.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2cffff700 1 -- 172.21.15.40:0/31534423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d00c31b0 con 0x7ff2d00cac60 2024-09-13T18:02:29.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2b77fe700 1 -- 172.21.15.40:0/31534423 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c400cbc0 con 0x7ff2d00cac60 2024-09-13T18:02:29.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.150+0000 7ff2cffff700 1 -- 172.21.15.40:0/31534423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d01f3460 con 0x7ff2d00cac60 2024-09-13T18:02:29.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.154+0000 7ff2cffff700 1 --2- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d00c5950 0x7ff2d00c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:29.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.154+0000 7ff2cffff700 1 -- 172.21.15.40:0/31534423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d0074750 con 0x7ff2d00c5950 2024-09-13T18:02:29.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.154+0000 7ff2ceffd700 1 --2- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d00c5950 0x7ff2d00c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:29.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.154+0000 7ff2ceffd700 1 -- 172.21.15.40:0/31534423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff2d015a4e0 con 0x7ff2d00c5950 2024-09-13T18:02:29.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.154+0000 7ff2ceffd700 1 --2- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d00c5950 0x7ff2d00c37f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00c9e60 tx=0x7ff2c0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:29.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.154+0000 7ff2b77fe700 1 -- 172.21.15.40:0/31534423 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c001f070 con 0x7ff2d00c5950 2024-09-13T18:02:29.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.154+0000 7ff2b77fe700 1 -- 172.21.15.40:0/31534423 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff2c400cd20 con 0x7ff2d00cac60 2024-09-13T18:02:29.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.154+0000 7ff2b77fe700 1 --2- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff2bc066770 0x7ff2bc068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:29.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.154+0000 7ff2cf7fe700 1 --2- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff2bc066770 0x7ff2bc068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:29.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.154+0000 7ff2b77fe700 1 -- 172.21.15.40:0/31534423 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff2c4084660 con 0x7ff2d00cac60 2024-09-13T18:02:29.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.154+0000 7ff2b77fe700 1 -- 172.21.15.40:0/31534423 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff2c0004380 con 0x7ff2d00c5950 2024-09-13T18:02:29.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.154+0000 7ff2b77fe700 1 -- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d00c5950 msgr2=0x7ff2d00c37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:29.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.154+0000 7ff2b77fe700 1 --2- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d00c5950 0x7ff2d00c37f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00c9e60 tx=0x7ff2c0004770 comp rx=0 tx=0).stop 2024-09-13T18:02:29.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.154+0000 7ff2b77fe700 1 -- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d00c5950 msgr2=0x7ff2d00c37f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:29.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.154+0000 7ff2b77fe700 1 --2- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d00c5950 0x7ff2d00c37f0 secure :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00c9e60 tx=0x7ff2c0004770 comp rx=0 tx=0).stop 2024-09-13T18:02:29.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.154+0000 7ff2cf7fe700 1 --2- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff2bc066770 0x7ff2bc068c30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80014a0 tx=0x7ff2c80016c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:29.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2cffff700 1 --2- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d014ec90 0x7ff2d014f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:29.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2cffff700 1 -- 172.21.15.40:0/31534423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d00c5d30 con 0x7ff2d014ec90 2024-09-13T18:02:29.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2ceffd700 1 --2- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d014ec90 0x7ff2d014f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:29.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2ceffd700 1 -- 172.21.15.40:0/31534423 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff2d015a200 con 0x7ff2d014ec90 2024-09-13T18:02:29.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2ceffd700 1 --2- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d014ec90 0x7ff2d014f0a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009c30 tx=0x7ff2c0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:29.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2b77fe700 1 -- 172.21.15.40:0/31534423 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c00038d0 con 0x7ff2d014ec90 2024-09-13T18:02:29.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2b77fe700 1 -- 172.21.15.40:0/31534423 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4938 (secure 0 0 0) 0x7ff2c0003a30 con 0x7ff2d014ec90 2024-09-13T18:02:29.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2b77fe700 1 -- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d014ec90 msgr2=0x7ff2d014f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:29.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2b77fe700 1 --2- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d014ec90 0x7ff2d014f0a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009c30 tx=0x7ff2c0011000 comp rx=0 tx=0).stop 2024-09-13T18:02:29.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2b77fe700 1 -- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d014ec90 msgr2=0x7ff2d014f0a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:29.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2b77fe700 1 --2- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d014ec90 0x7ff2d014f0a0 secure :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009c30 tx=0x7ff2c0011000 comp rx=0 tx=0).stop 2024-09-13T18:02:29.179 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:02:29.179 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:02:29.179 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:02:29.179 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:02:29.179 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 92, 2024-09-13T18:02:29.179 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:02:29.179 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:02:29.179 INFO:teuthology.orchestra.run.smithi040.stdout: 1 2024-09-13T18:02:29.179 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:02:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:02:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:02:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:29.180 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:29.181 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:29.181 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:02:29.181 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:02:29.181 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:29.181 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:29.181 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:29.181 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:29.181 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:29.181 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:29.181 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:29.182 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:29.182 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:29.182 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:29.182 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:29.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.182 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:02:29.182 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:02:29.182 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:02:29.182 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:02:29.182 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:02:29.182 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:02:29.182 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:29.182 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:29.182 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:02:29.183 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:02:29.183 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:02:29.183 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:02:29.183 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:02:29.183 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:02:29.183 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:02:29.183 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:29.183 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:02:29.183 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:29.183 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:29.183 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:29.183 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:29.183 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:29.184 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:29.184 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:29.184 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:29.184 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:29.184 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:29.184 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:29.184 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:02:29.184 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.184 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:02:29.184 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.184 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:02:29.185 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:02:29.185 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:29.185 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:29.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.185 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:29.185 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:02:29.185 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:29.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.185 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:29.185 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:02:29.185 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:29.185 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:29.186 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:29.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.186 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:02:29.186 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:02:29.186 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:29.186 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:29.186 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:29.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.186 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:02:29.186 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:02:29.186 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:29.186 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:29.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.186 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:29.187 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:02:29.187 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:29.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.187 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.187 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:29.187 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:02:29.187 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:29.187 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:29.187 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:29.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.187 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:02:29.187 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:02:29.188 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:29.188 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:29.188 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:29.188 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.188 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.188 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:02:29.188 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:02:29.188 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:29.188 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:29.188 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.188 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:29.188 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:02:29.188 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:29.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.189 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:29.189 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:02:29.189 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:29.189 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:29.189 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:29.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.189 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:02:29.189 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:02:29.189 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:29.189 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:29.190 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:29.190 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:29.190 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:29.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.190 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:02:29.190 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:02:29.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.190 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:29.190 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:29.190 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:02:29.190 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:29.191 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:29.191 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:02:29.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.192 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:29.192 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:29.192 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:02:29.192 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:29.192 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:29.192 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:02:29.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.193 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:29.193 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:29.193 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 8 2024-09-13T18:02:29.193 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:29.193 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:29.193 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:02:29.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.193 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:29.193 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:29.193 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:02:29.193 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:29.193 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:29.193 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:02:29.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.194 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:29.194 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:29.194 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:02:29.194 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:29.194 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:29.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.194 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:02:29.194 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:02:29.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2cffff700 1 -- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff2bc066770 msgr2=0x7ff2bc068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:29.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2cffff700 1 --2- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff2bc066770 0x7ff2bc068c30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80014a0 tx=0x7ff2c80016c0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2cffff700 1 -- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d00cac60 msgr2=0x7ff2d00c3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2cffff700 1 --2- 172.21.15.40:0/31534423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff2d00cac60 0x7ff2d00c3f30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400e9d0 tx=0x7ff2c400ece0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2cf7fe700 1 -- 172.21.15.40:0/31534423 reap_dead start 2024-09-13T18:02:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2cf7fe700 1 -- 172.21.15.40:0/31534423 reap_dead start 2024-09-13T18:02:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2cffff700 1 -- 172.21.15.40:0/31534423 shutdown_connections 2024-09-13T18:02:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2cffff700 1 -- 172.21.15.40:0/31534423 >> 172.21.15.40:0/31534423 conn(0x7ff2d0066090 msgr2=0x7ff2d006c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2cffff700 1 -- 172.21.15.40:0/31534423 shutdown_connections 2024-09-13T18:02:29.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.166+0000 7ff2cffff700 1 -- 172.21.15.40:0/31534423 wait complete. 2024-09-13T18:02:29.196 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:02:29.196 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-13T18:02:29.197 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:02:29.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.306+0000 7fd4abfff700 1 Processor -- start 2024-09-13T18:02:29.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.306+0000 7fd4abfff700 1 -- start start 2024-09-13T18:02:29.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4abfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd4ac157d40 0x7fd4ac066220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:29.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4abfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4ac0cb2a0 0x7fd4ac0cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:29.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4abfff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ac1578b0 con 0x7fd4ac151050 2024-09-13T18:02:29.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4abfff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ac157090 con 0x7fd4ac0cb2a0 2024-09-13T18:02:29.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4abfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac0c6630 con 0x7fd4ac157d40 2024-09-13T18:02:29.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4aa7fc700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd4ac157d40 msgr2=0x7fd4ac066220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:02:29.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4aa7fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd4ac157d40 0x7fd4ac066220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:02:29.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4ab7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4ac0cb2a0 0x7fd4ac0cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:29.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4ab7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4ac0cb2a0 0x7fd4ac0cb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:47378/0 (socket says 172.21.15.40:47378) 2024-09-13T18:02:29.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4ab7fe700 1 -- 172.21.15.40:0/3759320631 learned_addr learned my addr 172.21.15.40:0/3759320631 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:29.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4aaffd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd4ac151050 0x7fd4ac0c2850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:57678/0 (socket says 172.21.15.40:57678) 2024-09-13T18:02:29.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4ab7fe700 1 -- 172.21.15.40:0/3759320631 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd4ac157d40 msgr2=0x7fd4ac066220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:02:29.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4ab7fe700 1 --2- 172.21.15.40:0/3759320631 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd4ac157d40 0x7fd4ac066220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4ab7fe700 1 -- 172.21.15.40:0/3759320631 >> v1:172.21.15.43:6789/0 conn(0x7fd4ac151050 legacy=0x7fd4ac0c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:29.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4ab7fe700 1 -- 172.21.15.40:0/3759320631 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac06b290 con 0x7fd4ac0cb2a0 2024-09-13T18:02:29.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4ab7fe700 1 --2- 172.21.15.40:0/3759320631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac0cb2a0 0x7fd4ac0cb6d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009e50 tx=0x7fd4a400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3658b63f9ae7e7e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:29.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4a9ffb700 1 -- 172.21.15.40:0/3759320631 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a400e070 con 0x7fd4ac0cb2a0 2024-09-13T18:02:29.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4a9ffb700 1 -- 172.21.15.40:0/3759320631 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a400b9c0 con 0x7fd4ac0cb2a0 2024-09-13T18:02:29.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4a9ffb700 1 -- 172.21.15.40:0/3759320631 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a400bd70 con 0x7fd4ac0cb2a0 2024-09-13T18:02:29.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4abfff700 1 -- 172.21.15.40:0/3759320631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac0cb2a0 msgr2=0x7fd4ac0cb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:29.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4abfff700 1 --2- 172.21.15.40:0/3759320631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac0cb2a0 0x7fd4ac0cb6d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009e50 tx=0x7fd4a400b330 comp rx=0 tx=0).stop 2024-09-13T18:02:29.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4abfff700 1 -- 172.21.15.40:0/3759320631 shutdown_connections 2024-09-13T18:02:29.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4abfff700 1 --2- 172.21.15.40:0/3759320631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac0cb2a0 0x7fd4ac0cb6d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4abfff700 1 --2- 172.21.15.40:0/3759320631 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd4ac157d40 0x7fd4ac066220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4abfff700 1 -- 172.21.15.40:0/3759320631 >> 172.21.15.40:0/3759320631 conn(0x7fd4ac06a390 msgr2=0x7fd4ac06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:29.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4abfff700 1 -- 172.21.15.40:0/3759320631 shutdown_connections 2024-09-13T18:02:29.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.310+0000 7fd4abfff700 1 -- 172.21.15.40:0/3759320631 wait complete. 2024-09-13T18:02:29.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4abfff700 1 Processor -- start 2024-09-13T18:02:29.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4abfff700 1 -- start start 2024-09-13T18:02:29.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4abfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac0cb2a0 0x7fd4ac1ed6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:29.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4abfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4ac151050 0x7fd4ac1540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:29.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4abfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd4ac157d40 0x7fd4ac1eaae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:29.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4abfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac1e06d0 con 0x7fd4ac151050 2024-09-13T18:02:29.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4abfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac1ed260 con 0x7fd4ac0cb2a0 2024-09-13T18:02:29.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4abfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac1e30a0 con 0x7fd4ac157d40 2024-09-13T18:02:29.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4ab7fe700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd4ac157d40 msgr2=0x7fd4ac1eaae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:02:29.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4ab7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd4ac157d40 0x7fd4ac1eaae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:02:29.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4aa7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4ac151050 0x7fd4ac1540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:29.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4aa7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4ac151050 0x7fd4ac1540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46494/0 (socket says 172.21.15.40:46494) 2024-09-13T18:02:29.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4aa7fc700 1 -- 172.21.15.40:0/1141399293 learned_addr learned my addr 172.21.15.40:0/1141399293 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:29.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4aaffd700 1 --2- 172.21.15.40:0/1141399293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac0cb2a0 0x7fd4ac1ed6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:29.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4aa7fc700 1 -- 172.21.15.40:0/1141399293 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd4ac157d40 msgr2=0x7fd4ac1eaae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:02:29.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4aa7fc700 1 --2- 172.21.15.40:0/1141399293 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd4ac157d40 0x7fd4ac1eaae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4aa7fc700 1 -- 172.21.15.40:0/1141399293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac0cb2a0 msgr2=0x7fd4ac1ed6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:29.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4aa7fc700 1 --2- 172.21.15.40:0/1141399293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac0cb2a0 0x7fd4ac1ed6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4aa7fc700 1 -- 172.21.15.40:0/1141399293 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a4009b30 con 0x7fd4ac151050 2024-09-13T18:02:29.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4aa7fc700 1 --2- 172.21.15.40:0/1141399293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4ac151050 0x7fd4ac1540d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000b7b0 tx=0x7fd4a000bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:29.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4937fe700 1 -- 172.21.15.40:0/1141399293 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0003e70 con 0x7fd4ac151050 2024-09-13T18:02:29.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4937fe700 1 -- 172.21.15.40:0/1141399293 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a0005070 con 0x7fd4ac151050 2024-09-13T18:02:29.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4abfff700 1 -- 172.21.15.40:0/1141399293 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ac1e3380 con 0x7fd4ac151050 2024-09-13T18:02:29.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4937fe700 1 -- 172.21.15.40:0/1141399293 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0005420 con 0x7fd4ac151050 2024-09-13T18:02:29.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4abfff700 1 -- 172.21.15.40:0/1141399293 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ac1ecef0 con 0x7fd4ac151050 2024-09-13T18:02:29.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4abfff700 1 --2- 172.21.15.40:0/1141399293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac1e1fd0 0x7fd4ac1e23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:29.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.314+0000 7fd4abfff700 1 -- 172.21.15.40:0/1141399293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac067600 con 0x7fd4ac1e1fd0 2024-09-13T18:02:29.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.318+0000 7fd4aaffd700 1 --2- 172.21.15.40:0/1141399293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac1e1fd0 0x7fd4ac1e23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:29.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.318+0000 7fd4aaffd700 1 -- 172.21.15.40:0/1141399293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd4ac1ecaa0 con 0x7fd4ac1e1fd0 2024-09-13T18:02:29.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.318+0000 7fd4aaffd700 1 --2- 172.21.15.40:0/1141399293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac1e1fd0 0x7fd4ac1e23b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd49c001580 tx=0x7fd49c001940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:29.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.318+0000 7fd4937fe700 1 -- 172.21.15.40:0/1141399293 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd4a0005580 con 0x7fd4ac151050 2024-09-13T18:02:29.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.318+0000 7fd4937fe700 1 --2- 172.21.15.40:0/1141399293 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd498066710 0x7fd498068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:29.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.318+0000 7fd4937fe700 1 -- 172.21.15.40:0/1141399293 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd4a0083e30 con 0x7fd4ac151050 2024-09-13T18:02:29.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.318+0000 7fd4ab7fe700 1 --2- 172.21.15.40:0/1141399293 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd498066710 0x7fd498068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:29.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.318+0000 7fd4937fe700 1 -- 172.21.15.40:0/1141399293 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c017070 con 0x7fd4ac1e1fd0 2024-09-13T18:02:29.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.318+0000 7fd4937fe700 1 -- 172.21.15.40:0/1141399293 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd49c012560 con 0x7fd4ac1e1fd0 2024-09-13T18:02:29.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.318+0000 7fd4937fe700 1 -- 172.21.15.40:0/1141399293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac1e1fd0 msgr2=0x7fd4ac1e23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:29.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.318+0000 7fd4937fe700 1 --2- 172.21.15.40:0/1141399293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac1e1fd0 0x7fd4ac1e23b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd49c001580 tx=0x7fd49c001940 comp rx=0 tx=0).stop 2024-09-13T18:02:29.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.318+0000 7fd4937fe700 1 -- 172.21.15.40:0/1141399293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac1e1fd0 msgr2=0x7fd4ac1e23b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:29.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.318+0000 7fd4937fe700 1 --2- 172.21.15.40:0/1141399293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac1e1fd0 0x7fd4ac1e23b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.318+0000 7fd4ab7fe700 1 --2- 172.21.15.40:0/1141399293 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd498066710 0x7fd498068bd0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400b370 tx=0x7fd4a4009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:29.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.330+0000 7fd4abfff700 1 --2- 172.21.15.40:0/1141399293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac1f0020 0x7fd4ac1f0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:29.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.330+0000 7fd4abfff700 1 -- 172.21.15.40:0/1141399293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac1ed0e0 con 0x7fd4ac1f0020 2024-09-13T18:02:29.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.330+0000 7fd4aaffd700 1 --2- 172.21.15.40:0/1141399293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac1f0020 0x7fd4ac1f0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:29.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.330+0000 7fd4aaffd700 1 -- 172.21.15.40:0/1141399293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd4ac1572a0 con 0x7fd4ac1f0020 2024-09-13T18:02:29.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.330+0000 7fd4aaffd700 1 --2- 172.21.15.40:0/1141399293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac1f0020 0x7fd4ac1f0400 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd49c015040 tx=0x7fd49c001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:29.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.330+0000 7fd4937fe700 1 -- 172.21.15.40:0/1141399293 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c023070 con 0x7fd4ac1f0020 2024-09-13T18:02:29.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.330+0000 7fd4937fe700 1 -- 172.21.15.40:0/1141399293 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4604 (secure 0 0 0) 0x7fd49c01e110 con 0x7fd4ac1f0020 2024-09-13T18:02:29.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.330+0000 7fd4937fe700 1 -- 172.21.15.40:0/1141399293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac1f0020 msgr2=0x7fd4ac1f0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:29.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.330+0000 7fd4937fe700 1 --2- 172.21.15.40:0/1141399293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac1f0020 0x7fd4ac1f0400 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd49c015040 tx=0x7fd49c001120 comp rx=0 tx=0).stop 2024-09-13T18:02:29.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.330+0000 7fd4937fe700 1 -- 172.21.15.40:0/1141399293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac1f0020 msgr2=0x7fd4ac1f0400 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:29.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.330+0000 7fd4937fe700 1 --2- 172.21.15.40:0/1141399293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ac1f0020 0x7fd4ac1f0400 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.344 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:02:29.344 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:02:29.344 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:02:29.344 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:02:29.344 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 92, 2024-09-13T18:02:29.344 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:02:29.344 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:02:29.344 INFO:teuthology.orchestra.run.smithi040.stdout: 1 2024-09-13T18:02:29.344 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:29.344 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:02:29.344 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:29.344 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:02:29.344 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:02:29.345 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:29.345 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:29.345 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:29.345 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:29.345 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:29.345 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:29.345 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:29.345 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:29.345 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:29.345 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:29.345 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:29.345 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:02:29.345 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:02:29.346 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:29.346 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:29.346 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:29.346 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:29.346 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:29.346 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:29.346 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:29.346 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:29.346 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:29.346 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:29.346 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:29.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.346 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:02:29.346 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:02:29.346 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:02:29.347 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:02:29.347 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:02:29.347 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:02:29.347 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:29.347 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:29.347 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:02:29.347 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:02:29.347 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:02:29.347 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:02:29.347 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:02:29.348 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:02:29.348 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:02:29.348 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:29.348 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:02:29.348 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:29.348 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:29.348 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:29.348 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:29.348 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:29.348 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:29.348 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:29.348 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:29.348 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:29.349 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:29.349 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:29.349 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:02:29.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.349 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:02:29.349 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.349 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:02:29.349 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:02:29.349 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:29.349 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:29.349 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.349 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:29.349 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:02:29.349 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:29.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.350 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.350 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:29.350 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:02:29.350 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:29.350 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:29.350 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:29.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.350 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:02:29.350 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:02:29.350 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:29.350 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:29.350 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:29.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.351 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:02:29.351 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:02:29.351 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:29.351 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:29.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.351 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:29.351 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:02:29.351 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:29.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.351 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:29.351 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:02:29.352 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:29.352 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:29.352 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:29.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.352 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:02:29.352 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:02:29.352 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:29.352 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:29.352 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:29.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.352 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:02:29.352 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:02:29.353 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:29.353 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:29.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.353 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:29.353 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:02:29.353 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:29.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.353 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:29.353 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:02:29.353 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:29.353 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:29.353 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:29.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.354 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:02:29.354 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:02:29.354 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:29.354 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:29.354 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:29.354 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:29.354 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:29.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.354 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:02:29.355 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:02:29.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.355 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:29.355 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:29.355 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:02:29.355 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:29.355 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:29.355 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:02:29.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.355 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:29.355 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:29.355 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:02:29.355 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:29.355 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:29.356 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:02:29.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:29.356 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:29.356 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:29.356 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:02:29.356 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:29.356 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:29.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:29.356 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:02:29.356 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:02:29.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.330+0000 7fd4abfff700 1 -- 172.21.15.40:0/1141399293 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd498066710 msgr2=0x7fd498068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:29.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.330+0000 7fd4abfff700 1 --2- 172.21.15.40:0/1141399293 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd498066710 0x7fd498068bd0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd4a400b370 tx=0x7fd4a4009000 comp rx=0 tx=0).stop 2024-09-13T18:02:29.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.330+0000 7fd4abfff700 1 -- 172.21.15.40:0/1141399293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4ac151050 msgr2=0x7fd4ac1540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:29.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.330+0000 7fd4abfff700 1 --2- 172.21.15.40:0/1141399293 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4ac151050 0x7fd4ac1540d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000b7b0 tx=0x7fd4a000bac0 comp rx=0 tx=0).stop 2024-09-13T18:02:29.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.334+0000 7fd4ab7fe700 1 -- 172.21.15.40:0/1141399293 reap_dead start 2024-09-13T18:02:29.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.334+0000 7fd4ab7fe700 1 -- 172.21.15.40:0/1141399293 reap_dead start 2024-09-13T18:02:29.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.334+0000 7fd4abfff700 1 -- 172.21.15.40:0/1141399293 shutdown_connections 2024-09-13T18:02:29.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.334+0000 7fd4abfff700 1 -- 172.21.15.40:0/1141399293 >> 172.21.15.40:0/1141399293 conn(0x7fd4ac06a390 msgr2=0x7fd4ac0c2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:29.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.334+0000 7fd4abfff700 1 -- 172.21.15.40:0/1141399293 shutdown_connections 2024-09-13T18:02:29.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:29.334+0000 7fd4abfff700 1 -- 172.21.15.40:0/1141399293 wait complete. 2024-09-13T18:02:29.361 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:02:29.361 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:02:35.894 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:02:35.897 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:02:49.362 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-13T18:02:49.362 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-13T18:02:49.363 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T18:02:49.363 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T18:02:49.365 INFO:tasks.ceph.mon.c:Started 2024-09-13T18:02:49.366 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:02:49.366 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:02:49.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.474+0000 7f55011bc700 1 Processor -- start 2024-09-13T18:02:49.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.474+0000 7f55011bc700 1 -- start start 2024-09-13T18:02:49.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f55011bc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f54fc14fb40 0x7f54fc0c6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:49.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f55011bc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f54fc0714d0 0x7f54fc0718e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:49.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f55011bc700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54fc071040 con 0x7f54fc06e130 2024-09-13T18:02:49.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f55011bc700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54fc070d40 con 0x7f54fc14fb40 2024-09-13T18:02:49.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f55011bc700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc14ff20 con 0x7f54fc0714d0 2024-09-13T18:02:49.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f54fbfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f54fc14fb40 0x7f54fc0c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:49.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f54fbfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f54fc14fb40 0x7f54fc0c6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:45526/0 (socket says 172.21.15.40:45526) 2024-09-13T18:02:49.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f54fbfff700 1 -- 172.21.15.40:0/3398450142 learned_addr learned my addr 172.21.15.40:0/3398450142 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:49.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f55009bb700 1 --2- 172.21.15.40:0/3398450142 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f54fc0714d0 0x7f54fc0718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:49.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f55009bb700 1 -- 172.21.15.40:0/3398450142 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f54fc0714d0 msgr2=0x7f54fc0718e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:02:49.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f55009bb700 1 -- 172.21.15.40:0/3398450142 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f54fc0714d0 msgr2=0x7f54fc0718e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:02:49.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f55009bb700 1 --2- 172.21.15.40:0/3398450142 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f54fc0714d0 0x7f54fc0718e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:02:49.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f55009bb700 1 --2- 172.21.15.40:0/3398450142 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f54fc0714d0 0x7f54fc0718e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:02:49.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f54fbfff700 1 -- 172.21.15.40:0/3398450142 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f54fc0714d0 msgr2=0x7f54fc0718e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:02:49.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f54fbfff700 1 --2- 172.21.15.40:0/3398450142 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f54fc0714d0 0x7f54fc0718e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:49.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f54fbfff700 1 -- 172.21.15.40:0/3398450142 >> v1:172.21.15.43:6789/0 conn(0x7f54fc06e130 legacy=0x7f54fc158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:49.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f54fbfff700 1 -- 172.21.15.40:0/3398450142 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc06e700 con 0x7f54fc14fb40 2024-09-13T18:02:49.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f54fbfff700 1 --2- 172.21.15.40:0/3398450142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54fc14fb40 0x7f54fc0c6410 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00aab0 tx=0x7f54ec00adc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96b999958b404bdd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:49.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f54faffd700 1 -- 172.21.15.40:0/3398450142 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00cb70 con 0x7f54fc14fb40 2024-09-13T18:02:49.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f54faffd700 1 -- 172.21.15.40:0/3398450142 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ec00ccd0 con 0x7f54fc14fb40 2024-09-13T18:02:49.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f54faffd700 1 -- 172.21.15.40:0/3398450142 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec018620 con 0x7f54fc14fb40 2024-09-13T18:02:49.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f55011bc700 1 -- 172.21.15.40:0/3398450142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54fc14fb40 msgr2=0x7f54fc0c6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:49.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f55011bc700 1 --2- 172.21.15.40:0/3398450142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54fc14fb40 0x7f54fc0c6410 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00aab0 tx=0x7f54ec00adc0 comp rx=0 tx=0).stop 2024-09-13T18:02:49.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f55011bc700 1 -- 172.21.15.40:0/3398450142 shutdown_connections 2024-09-13T18:02:49.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f55011bc700 1 --2- 172.21.15.40:0/3398450142 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f54fc0714d0 0x7f54fc0718e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:49.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f55011bc700 1 --2- 172.21.15.40:0/3398450142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54fc14fb40 0x7f54fc0c6410 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:49.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f55011bc700 1 -- 172.21.15.40:0/3398450142 >> 172.21.15.40:0/3398450142 conn(0x7f54fc0cb4c0 msgr2=0x7f54fc0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:49.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f55011bc700 1 -- 172.21.15.40:0/3398450142 shutdown_connections 2024-09-13T18:02:49.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.478+0000 7f55011bc700 1 -- 172.21.15.40:0/3398450142 wait complete. 2024-09-13T18:02:49.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f55011bc700 1 Processor -- start 2024-09-13T18:02:49.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f55011bc700 1 -- start start 2024-09-13T18:02:49.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f55011bc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54fc06e130 0x7f54fc151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:49.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f55011bc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f54fc0714d0 0x7f54fc156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:49.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f55011bc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54fc14fb40 0x7f54fc156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:49.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f55011bc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc1531d0 con 0x7f54fc14fb40 2024-09-13T18:02:49.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f55011bc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc0b7f60 con 0x7f54fc06e130 2024-09-13T18:02:49.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f55011bc700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc1558d0 con 0x7f54fc0714d0 2024-09-13T18:02:49.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f55009bb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54fc14fb40 0x7f54fc156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:49.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f55009bb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54fc14fb40 0x7f54fc156580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43894/0 (socket says 172.21.15.40:43894) 2024-09-13T18:02:49.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f55009bb700 1 -- 172.21.15.40:0/2620851693 learned_addr learned my addr 172.21.15.40:0/2620851693 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:49.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f54fbfff700 1 --2- 172.21.15.40:0/2620851693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54fc06e130 0x7f54fc151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:49.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f54fb7fe700 1 --2- 172.21.15.40:0/2620851693 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f54fc0714d0 0x7f54fc156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:49.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f54fb7fe700 1 -- 172.21.15.40:0/2620851693 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f54fc0714d0 msgr2=0x7f54fc156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:02:49.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f54fb7fe700 1 -- 172.21.15.40:0/2620851693 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f54fc0714d0 msgr2=0x7f54fc156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:02:49.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f54fb7fe700 1 --2- 172.21.15.40:0/2620851693 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f54fc0714d0 0x7f54fc156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:02:49.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f54fb7fe700 1 --2- 172.21.15.40:0/2620851693 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f54fc0714d0 0x7f54fc156040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:02:49.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f55009bb700 1 -- 172.21.15.40:0/2620851693 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f54fc0714d0 msgr2=0x7f54fc156040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:02:49.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f55009bb700 1 --2- 172.21.15.40:0/2620851693 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f54fc0714d0 0x7f54fc156040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:49.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f55009bb700 1 -- 172.21.15.40:0/2620851693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54fc06e130 msgr2=0x7f54fc151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:49.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f55009bb700 1 --2- 172.21.15.40:0/2620851693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54fc06e130 0x7f54fc151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:49.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f55009bb700 1 -- 172.21.15.40:0/2620851693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec00a710 con 0x7f54fc14fb40 2024-09-13T18:02:49.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f55009bb700 1 --2- 172.21.15.40:0/2620851693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54fc14fb40 0x7f54fc156580 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f54f4007e30 tx=0x7f54f400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:49.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f54f8ff9700 1 -- 172.21.15.40:0/2620851693 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f400ede0 con 0x7f54fc14fb40 2024-09-13T18:02:49.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f54f8ff9700 1 -- 172.21.15.40:0/2620851693 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f4014420 con 0x7f54fc14fb40 2024-09-13T18:02:49.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f54f8ff9700 1 -- 172.21.15.40:0/2620851693 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f40147d0 con 0x7f54fc14fb40 2024-09-13T18:02:49.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f55011bc700 1 -- 172.21.15.40:0/2620851693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54fc155bb0 con 0x7f54fc14fb40 2024-09-13T18:02:49.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f55011bc700 1 -- 172.21.15.40:0/2620851693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54fc152510 con 0x7f54fc14fb40 2024-09-13T18:02:49.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.482+0000 7f55011bc700 1 -- 172.21.15.40:0/2620851693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54fc159a30 con 0x7f54fc14fb40 2024-09-13T18:02:49.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.486+0000 7f54f8ff9700 1 -- 172.21.15.40:0/2620851693 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f54f4014930 con 0x7f54fc14fb40 2024-09-13T18:02:49.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.486+0000 7f54f8ff9700 1 --2- 172.21.15.40:0/2620851693 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f54e8066760 0x7f54e8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:49.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.486+0000 7f54f8ff9700 1 -- 172.21.15.40:0/2620851693 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f54f4084b90 con 0x7f54fc14fb40 2024-09-13T18:02:49.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.486+0000 7f54fbfff700 1 --2- 172.21.15.40:0/2620851693 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f54e8066760 0x7f54e8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:49.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.486+0000 7f54fbfff700 1 --2- 172.21.15.40:0/2620851693 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f54e8066760 0x7f54e8068c20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f54ec000c00 tx=0x7f54ec012000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:49.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.494+0000 7f54f8ff9700 1 -- 172.21.15.40:0/2620851693 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f54f4051b50 con 0x7f54fc14fb40 2024-09-13T18:02:49.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.762+0000 7f55011bc700 1 -- 172.21.15.40:0/2620851693 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f54fc0c6f90 con 0x7f54fc14fb40 2024-09-13T18:02:49.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.766+0000 7f54f8ff9700 1 -- 172.21.15.40:0/2620851693 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f54f4055170 con 0x7f54fc14fb40 2024-09-13T18:02:49.775 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":92,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:02:49.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.766+0000 7f55011bc700 1 -- 172.21.15.40:0/2620851693 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f54e8066760 msgr2=0x7f54e8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:49.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.766+0000 7f55011bc700 1 --2- 172.21.15.40:0/2620851693 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f54e8066760 0x7f54e8068c20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f54ec000c00 tx=0x7f54ec012000 comp rx=0 tx=0).stop 2024-09-13T18:02:49.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.766+0000 7f55011bc700 1 -- 172.21.15.40:0/2620851693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54fc14fb40 msgr2=0x7f54fc156580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:49.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.766+0000 7f55011bc700 1 --2- 172.21.15.40:0/2620851693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54fc14fb40 0x7f54fc156580 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f54f4007e30 tx=0x7f54f400e3c0 comp rx=0 tx=0).stop 2024-09-13T18:02:49.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.766+0000 7f55011bc700 1 -- 172.21.15.40:0/2620851693 shutdown_connections 2024-09-13T18:02:49.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.766+0000 7f55011bc700 1 --2- 172.21.15.40:0/2620851693 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54fc14fb40 0x7f54fc156580 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:49.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.766+0000 7f55011bc700 1 --2- 172.21.15.40:0/2620851693 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f54e8066760 0x7f54e8068c20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:49.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.766+0000 7f55011bc700 1 --2- 172.21.15.40:0/2620851693 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f54fc0714d0 0x7f54fc156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:49.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.766+0000 7f55011bc700 1 --2- 172.21.15.40:0/2620851693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54fc06e130 0x7f54fc151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:49.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.766+0000 7f55011bc700 1 -- 172.21.15.40:0/2620851693 >> 172.21.15.40:0/2620851693 conn(0x7f54fc0cb4c0 msgr2=0x7f54fc14e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:49.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.766+0000 7f55011bc700 1 -- 172.21.15.40:0/2620851693 shutdown_connections 2024-09-13T18:02:49.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:49.766+0000 7f55011bc700 1 -- 172.21.15.40:0/2620851693 wait complete. 2024-09-13T18:02:52.790 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:02:52.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:52.902+0000 7f28fffff700 1 Processor -- start 2024-09-13T18:02:52.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:52.902+0000 7f28fffff700 1 -- start start 2024-09-13T18:02:52.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:52.902+0000 7f28fffff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 0x7f2900071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:52.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:52.902+0000 7f28fffff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f290015a520 con 0x7f2900072130 2024-09-13T18:02:52.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:52.902+0000 7f28fffff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29001564b0 con 0x7f29001503d0 2024-09-13T18:02:52.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:52.902+0000 7f28fffff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2900072540 con 0x7f290006b4a0 2024-09-13T18:02:52.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:52.902+0000 7f28ff7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2900072130 0x7f2900160970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58266/0 (socket says 172.21.15.40:58266) 2024-09-13T18:02:52.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:52.902+0000 7f28ff7fe700 1 -- 172.21.15.40:0/3819930837 learned_addr learned my addr 172.21.15.40:0/3819930837 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:02:52.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:52.902+0000 7f28fe7fc700 1 --2- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 0x7f2900071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:52.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:52.902+0000 7f28fe7fc700 1 -- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 msgr2=0x7f2900071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:02:52.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:52.902+0000 7f28fe7fc700 1 -- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 msgr2=0x7f2900071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:02:52.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:52.902+0000 7f28fe7fc700 1 --2- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 0x7f2900071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:02:52.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:52.902+0000 7f28fe7fc700 1 --2- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 0x7f2900071bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:02:53.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:53.106+0000 7f28fe7fc700 1 --2- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 0x7f2900071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:53.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:53.106+0000 7f28fe7fc700 1 -- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 msgr2=0x7f2900071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:02:53.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:53.106+0000 7f28fe7fc700 1 -- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 msgr2=0x7f2900071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:02:53.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:53.106+0000 7f28fe7fc700 1 --2- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 0x7f2900071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:02:53.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:53.106+0000 7f28fe7fc700 1 --2- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 0x7f2900071bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:02:53.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:53.510+0000 7f28fe7fc700 1 --2- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 0x7f2900071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:53.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:53.510+0000 7f28fe7fc700 1 -- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 msgr2=0x7f2900071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:02:53.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:53.510+0000 7f28fe7fc700 1 -- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 msgr2=0x7f2900071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:02:53.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:53.510+0000 7f28fe7fc700 1 --2- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 0x7f2900071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:02:53.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:53.510+0000 7f28fe7fc700 1 --2- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 0x7f2900071bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:02:54.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:54.314+0000 7f28fe7fc700 1 --2- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 0x7f2900071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:54.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:54.314+0000 7f28fe7fc700 1 -- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 msgr2=0x7f2900071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:02:54.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:54.314+0000 7f28fe7fc700 1 -- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 msgr2=0x7f2900071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:02:54.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:54.314+0000 7f28fe7fc700 1 --2- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 0x7f2900071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:02:54.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:54.314+0000 7f28fe7fc700 1 --2- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 0x7f2900071bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:02:55.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fdffb700 1 -- 172.21.15.40:0/3819930837 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983989508 0 0) 0x7f29001564b0 con 0x7f29001503d0 2024-09-13T18:02:55.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fdffb700 1 -- 172.21.15.40:0/3819930837 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28e80035d0 con 0x7f29001503d0 2024-09-13T18:02:55.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fdffb700 1 -- 172.21.15.40:0/3819930837 <== mon.1 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3719387853 0 0) 0x7f28e80035d0 con 0x7f29001503d0 2024-09-13T18:02:55.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fdffb700 1 -- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 msgr2=0x7f2900071bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:02:55.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fdffb700 1 --2- 172.21.15.40:0/3819930837 >> v2:172.21.15.43:3300/0 conn(0x7f290006b4a0 0x7f2900071bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:55.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fdffb700 1 -- 172.21.15.40:0/3819930837 >> v1:172.21.15.40:6789/0 conn(0x7f2900072130 legacy=0x7f2900160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:55.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fdffb700 1 -- 172.21.15.40:0/3819930837 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290015a000 con 0x7f29001503d0 2024-09-13T18:02:55.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fdffb700 1 -- 172.21.15.40:0/3819930837 <== mon.1 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f28f80019f0 con 0x7f29001503d0 2024-09-13T18:02:55.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fdffb700 1 -- 172.21.15.40:0/3819930837 >> v1:172.21.15.43:6789/0 conn(0x7f29001503d0 legacy=0x7f2900156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:55.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fdffb700 1 --2- 172.21.15.40:0/3819930837 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28e8003a20 0x7f28e8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:55.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fdffb700 1 --2- 172.21.15.40:0/3819930837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28e8004870 0x7f28e8004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:55.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fdffb700 1 --2- 172.21.15.40:0/3819930837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e8005230 0x7f28e800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:55.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fdffb700 1 -- 172.21.15.40:0/3819930837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2900072540 con 0x7f28e8004870 2024-09-13T18:02:55.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fdffb700 1 -- 172.21.15.40:0/3819930837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e800c180 con 0x7f28e8005230 2024-09-13T18:02:55.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fdffb700 1 -- 172.21.15.40:0/3819930837 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e800c340 con 0x7f28e8003a20 2024-09-13T18:02:55.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fe7fc700 1 --2- 172.21.15.40:0/3819930837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28e8004870 0x7f28e8004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:55.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28feffd700 1 --2- 172.21.15.40:0/3819930837 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28e8003a20 0x7f28e8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:55.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28ff7fe700 1 --2- 172.21.15.40:0/3819930837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e8005230 0x7f28e800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:55.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fe7fc700 1 -- 172.21.15.40:0/3819930837 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28e8003a20 msgr2=0x7f28e8003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:55.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fe7fc700 1 --2- 172.21.15.40:0/3819930837 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28e8003a20 0x7f28e8003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:55.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fe7fc700 1 -- 172.21.15.40:0/3819930837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e8005230 msgr2=0x7f28e800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:55.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fe7fc700 1 --2- 172.21.15.40:0/3819930837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e8005230 0x7f28e800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:55.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fe7fc700 1 -- 172.21.15.40:0/3819930837 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e80035d0 con 0x7f28e8004870 2024-09-13T18:02:55.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28feffd700 1 --2- 172.21.15.40:0/3819930837 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28e8003a20 0x7f28e8003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:02:55.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28ff7fe700 1 --2- 172.21.15.40:0/3819930837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28e8005230 0x7f28e800ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:02:55.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28ff7fe700 1 -- 172.21.15.40:0/3819930837 reap_dead start 2024-09-13T18:02:55.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.546+0000 7f28fe7fc700 1 --2- 172.21.15.40:0/3819930837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28e8004870 0x7f28e8004cf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f28f40129d0 tx=0x7f28e80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2443c14b277fe92d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:55.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.554+0000 7f28fdffb700 1 -- 172.21.15.40:0/3819930837 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4012610 con 0x7f28e8004870 2024-09-13T18:02:55.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.554+0000 7f28fdffb700 1 -- 172.21.15.40:0/3819930837 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f400c040 con 0x7f28e8004870 2024-09-13T18:02:55.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.554+0000 7f28fdffb700 1 -- 172.21.15.40:0/3819930837 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4014a20 con 0x7f28e8004870 2024-09-13T18:02:55.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.554+0000 7f28fffff700 1 -- 172.21.15.40:0/3819930837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28e8004870 msgr2=0x7f28e8004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:55.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.554+0000 7f28fffff700 1 --2- 172.21.15.40:0/3819930837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28e8004870 0x7f28e8004cf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f28f40129d0 tx=0x7f28e80058a0 comp rx=0 tx=0).stop 2024-09-13T18:02:55.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.558+0000 7f28fffff700 1 -- 172.21.15.40:0/3819930837 shutdown_connections 2024-09-13T18:02:55.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.558+0000 7f28fffff700 1 --2- 172.21.15.40:0/3819930837 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28e8004870 0x7f28e8004cf0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:55.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.558+0000 7f28fffff700 1 -- 172.21.15.40:0/3819930837 >> 172.21.15.40:0/3819930837 conn(0x7f290006e4f0 msgr2=0x7f290006e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:55.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.558+0000 7f28fffff700 1 -- 172.21.15.40:0/3819930837 shutdown_connections 2024-09-13T18:02:55.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.558+0000 7f28fffff700 1 -- 172.21.15.40:0/3819930837 wait complete. 2024-09-13T18:02:55.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.558+0000 7f28fffff700 1 Processor -- start 2024-09-13T18:02:55.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.558+0000 7f28fffff700 1 -- start start 2024-09-13T18:02:55.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.558+0000 7f28fffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28e8004870 0x7f290006eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:55.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.558+0000 7f28fffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29000c5f50 0x7f2900148e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:55.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.558+0000 7f28fffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2900165dd0 0x7f2900149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:55.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.558+0000 7f28fffff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29000c43d0 con 0x7f29000c5f50 2024-09-13T18:02:55.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.558+0000 7f28fffff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29000c3d30 con 0x7f2900165dd0 2024-09-13T18:02:55.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.558+0000 7f28fffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29000c31b0 con 0x7f28e8004870 2024-09-13T18:02:55.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.558+0000 7f28fe7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29000c5f50 0x7f2900148e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:55.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.558+0000 7f28fe7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29000c5f50 0x7f2900148e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45876/0 (socket says 172.21.15.40:45876) 2024-09-13T18:02:55.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.558+0000 7f28ff7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2900165dd0 0x7f2900149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:55.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.558+0000 7f28fe7fc700 1 -- 172.21.15.40:0/1213093537 learned_addr learned my addr 172.21.15.40:0/1213093537 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:55.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.558+0000 7f28feffd700 1 --2- 172.21.15.40:0/1213093537 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28e8004870 0x7f290006eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:55.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.562+0000 7f28feffd700 1 -- 172.21.15.40:0/1213093537 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28e8004870 msgr2=0x7f290006eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:02:55.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.562+0000 7f28feffd700 1 -- 172.21.15.40:0/1213093537 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28e8004870 msgr2=0x7f290006eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:02:55.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.562+0000 7f28feffd700 1 --2- 172.21.15.40:0/1213093537 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28e8004870 0x7f290006eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:02:55.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.562+0000 7f28feffd700 1 --2- 172.21.15.40:0/1213093537 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28e8004870 0x7f290006eb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:02:55.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.562+0000 7f28ff7fe700 1 -- 172.21.15.40:0/1213093537 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28e8004870 msgr2=0x7f290006eb40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:02:55.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.562+0000 7f28ff7fe700 1 --2- 172.21.15.40:0/1213093537 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28e8004870 0x7f290006eb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:55.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.562+0000 7f28ff7fe700 1 -- 172.21.15.40:0/1213093537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29000c5f50 msgr2=0x7f2900148e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:55.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.562+0000 7f28ff7fe700 1 --2- 172.21.15.40:0/1213093537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29000c5f50 0x7f2900148e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:55.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.562+0000 7f28ff7fe700 1 -- 172.21.15.40:0/1213093537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2900154170 con 0x7f2900165dd0 2024-09-13T18:02:55.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.562+0000 7f28ff7fe700 1 --2- 172.21.15.40:0/1213093537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2900165dd0 0x7f2900149370 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f290006c150 tx=0x7f28f0007570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:55.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.562+0000 7f28e77fe700 1 -- 172.21.15.40:0/1213093537 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f00114c0 con 0x7f2900165dd0 2024-09-13T18:02:55.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.562+0000 7f28e77fe700 1 -- 172.21.15.40:0/1213093537 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f0011620 con 0x7f2900165dd0 2024-09-13T18:02:55.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.562+0000 7f28e77fe700 1 -- 172.21.15.40:0/1213093537 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0011990 con 0x7f2900165dd0 2024-09-13T18:02:55.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.562+0000 7f28fffff700 1 -- 172.21.15.40:0/1213093537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29000c3390 con 0x7f2900165dd0 2024-09-13T18:02:55.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.562+0000 7f28fffff700 1 -- 172.21.15.40:0/1213093537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29000c3950 con 0x7f2900165dd0 2024-09-13T18:02:55.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.562+0000 7f28fffff700 1 -- 172.21.15.40:0/1213093537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29001507b0 con 0x7f2900165dd0 2024-09-13T18:02:55.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.562+0000 7f28e77fe700 1 -- 172.21.15.40:0/1213093537 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f28f0011af0 con 0x7f2900165dd0 2024-09-13T18:02:55.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.562+0000 7f28e77fe700 1 --2- 172.21.15.40:0/1213093537 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f28ec066760 0x7f28ec068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:55.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.566+0000 7f28feffd700 1 --2- 172.21.15.40:0/1213093537 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f28ec066760 0x7f28ec068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:55.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.566+0000 7f28e77fe700 1 -- 172.21.15.40:0/1213093537 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f28f0083a60 con 0x7f2900165dd0 2024-09-13T18:02:55.576 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.566+0000 7f28feffd700 1 --2- 172.21.15.40:0/1213093537 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f28ec066760 0x7f28ec068c20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f28f8000f80 tx=0x7f28f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:55.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.574+0000 7f28e77fe700 1 -- 172.21.15.40:0/1213093537 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f28f0050ac0 con 0x7f2900165dd0 2024-09-13T18:02:55.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.838+0000 7f28fffff700 1 -- 172.21.15.40:0/1213093537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f290014d990 con 0x7f2900165dd0 2024-09-13T18:02:55.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.838+0000 7f28e77fe700 1 -- 172.21.15.40:0/1213093537 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1533 (secure 0 0 0) 0x7f28f0019070 con 0x7f2900165dd0 2024-09-13T18:02:55.849 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":96,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:02:55.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.842+0000 7f28fffff700 1 -- 172.21.15.40:0/1213093537 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f28ec066760 msgr2=0x7f28ec068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:55.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.842+0000 7f28fffff700 1 --2- 172.21.15.40:0/1213093537 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f28ec066760 0x7f28ec068c20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f28f8000f80 tx=0x7f28f8009040 comp rx=0 tx=0).stop 2024-09-13T18:02:55.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.842+0000 7f28fffff700 1 -- 172.21.15.40:0/1213093537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2900165dd0 msgr2=0x7f2900149370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:55.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.842+0000 7f28fffff700 1 --2- 172.21.15.40:0/1213093537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2900165dd0 0x7f2900149370 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f290006c150 tx=0x7f28f0007570 comp rx=0 tx=0).stop 2024-09-13T18:02:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.842+0000 7f28fffff700 1 -- 172.21.15.40:0/1213093537 shutdown_connections 2024-09-13T18:02:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.842+0000 7f28fffff700 1 --2- 172.21.15.40:0/1213093537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2900165dd0 0x7f2900149370 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.842+0000 7f28fffff700 1 --2- 172.21.15.40:0/1213093537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29000c5f50 0x7f2900148e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.842+0000 7f28fffff700 1 --2- 172.21.15.40:0/1213093537 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f28ec066760 0x7f28ec068c20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.842+0000 7f28fffff700 1 --2- 172.21.15.40:0/1213093537 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28e8004870 0x7f290006eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.842+0000 7f28fffff700 1 -- 172.21.15.40:0/1213093537 >> 172.21.15.40:0/1213093537 conn(0x7f290006e4f0 msgr2=0x7f29000c76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.842+0000 7f28fffff700 1 -- 172.21.15.40:0/1213093537 shutdown_connections 2024-09-13T18:02:55.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:55.842+0000 7f28fffff700 1 -- 172.21.15.40:0/1213093537 wait complete. 2024-09-13T18:02:58.867 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:02:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.974+0000 7f3902584700 1 Processor -- start 2024-09-13T18:02:58.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.974+0000 7f3902584700 1 -- start start 2024-09-13T18:02:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3902584700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f38fc070bc0 0x7f38fc0c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3902584700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f38fc1567e0 0x7f38fc156c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3902584700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38fc159800 con 0x7f38fc150690 2024-09-13T18:02:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3902584700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc0c8350 con 0x7f38fc070bc0 2024-09-13T18:02:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3902584700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc0c6d20 con 0x7f38fc1567e0 2024-09-13T18:02:58.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3900d81700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f38fc070bc0 0x7f38fc0c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3900d81700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f38fc070bc0 0x7f38fc0c67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45888/0 (socket says 172.21.15.40:45888) 2024-09-13T18:02:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3900d81700 1 -- 172.21.15.40:0/1117046358 learned_addr learned my addr 172.21.15.40:0/1117046358 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3901d83700 1 --2- 172.21.15.40:0/1117046358 >> v2:172.21.15.43:3300/0 conn(0x7f38fc1567e0 0x7f38fc156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3900d81700 1 -- 172.21.15.40:0/1117046358 >> v2:172.21.15.43:3300/0 conn(0x7f38fc1567e0 msgr2=0x7f38fc156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3900d81700 1 --2- 172.21.15.40:0/1117046358 >> v2:172.21.15.43:3300/0 conn(0x7f38fc1567e0 0x7f38fc156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3900d81700 1 -- 172.21.15.40:0/1117046358 >> v1:172.21.15.40:6789/0 conn(0x7f38fc150690 legacy=0x7f38fc06b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3900d81700 1 -- 172.21.15.40:0/1117046358 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc150a70 con 0x7f38fc070bc0 2024-09-13T18:02:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3900d81700 1 --2- 172.21.15.40:0/1117046358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc070bc0 0x7f38fc0c67e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0671f0 tx=0x7f38ec009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748a7f8c0f4f4241 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f38f3fff700 1 -- 172.21.15.40:0/1117046358 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec00b8a0 con 0x7f38fc070bc0 2024-09-13T18:02:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f38f3fff700 1 -- 172.21.15.40:0/1117046358 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ec00ba00 con 0x7f38fc070bc0 2024-09-13T18:02:58.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f38f3fff700 1 -- 172.21.15.40:0/1117046358 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec00bd90 con 0x7f38fc070bc0 2024-09-13T18:02:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3902584700 1 -- 172.21.15.40:0/1117046358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc070bc0 msgr2=0x7f38fc0c67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3902584700 1 --2- 172.21.15.40:0/1117046358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc070bc0 0x7f38fc0c67e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0671f0 tx=0x7f38ec009bf0 comp rx=0 tx=0).stop 2024-09-13T18:02:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3902584700 1 -- 172.21.15.40:0/1117046358 shutdown_connections 2024-09-13T18:02:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3902584700 1 --2- 172.21.15.40:0/1117046358 >> v2:172.21.15.43:3300/0 conn(0x7f38fc1567e0 0x7f38fc156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3902584700 1 --2- 172.21.15.40:0/1117046358 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc070bc0 0x7f38fc0c67e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:58.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3902584700 1 -- 172.21.15.40:0/1117046358 >> 172.21.15.40:0/1117046358 conn(0x7f38fc0c88e0 msgr2=0x7f38fc0c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3902584700 1 -- 172.21.15.40:0/1117046358 shutdown_connections 2024-09-13T18:02:58.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.978+0000 7f3902584700 1 -- 172.21.15.40:0/1117046358 wait complete. 2024-09-13T18:02:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3902584700 1 Processor -- start 2024-09-13T18:02:58.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3902584700 1 -- start start 2024-09-13T18:02:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3902584700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc070bc0 0x7f38fc148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3902584700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38fc150690 0x7f38fc142840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3902584700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc1567e0 0x7f38fc1f0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3902584700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc1eda30 con 0x7f38fc070bc0 2024-09-13T18:02:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3902584700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0ca900 con 0x7f38fc1567e0 2024-09-13T18:02:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3902584700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc147370 con 0x7f38fc150690 2024-09-13T18:02:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3901582700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc070bc0 0x7f38fc148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3901582700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc070bc0 0x7f38fc148250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45890/0 (socket says 172.21.15.40:45890) 2024-09-13T18:02:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3901582700 1 -- 172.21.15.40:0/1524169005 learned_addr learned my addr 172.21.15.40:0/1524169005 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3901d83700 1 --2- 172.21.15.40:0/1524169005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc1567e0 0x7f38fc1f0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:58.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3900d81700 1 --2- 172.21.15.40:0/1524169005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38fc150690 0x7f38fc142840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:58.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3901582700 1 -- 172.21.15.40:0/1524169005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38fc150690 msgr2=0x7f38fc142840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:58.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3901582700 1 --2- 172.21.15.40:0/1524169005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38fc150690 0x7f38fc142840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:58.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3901582700 1 -- 172.21.15.40:0/1524169005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc1567e0 msgr2=0x7f38fc1f0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:58.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3901582700 1 --2- 172.21.15.40:0/1524169005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc1567e0 0x7f38fc1f0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:58.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3901582700 1 -- 172.21.15.40:0/1524169005 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec009570 con 0x7f38fc070bc0 2024-09-13T18:02:58.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3900d81700 1 --2- 172.21.15.40:0/1524169005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38fc150690 0x7f38fc142840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:02:58.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3901582700 1 --2- 172.21.15.40:0/1524169005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc070bc0 0x7f38fc148250 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f38f800c500 tx=0x7f38f800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:58.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f38f1ffb700 1 -- 172.21.15.40:0/1524169005 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f80052d0 con 0x7f38fc070bc0 2024-09-13T18:02:58.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f38f1ffb700 1 -- 172.21.15.40:0/1524169005 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f8005430 con 0x7f38fc070bc0 2024-09-13T18:02:58.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3902584700 1 -- 172.21.15.40:0/1524169005 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38fc1eca10 con 0x7f38fc070bc0 2024-09-13T18:02:58.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f38f1ffb700 1 -- 172.21.15.40:0/1524169005 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f80055a0 con 0x7f38fc070bc0 2024-09-13T18:02:58.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3902584700 1 -- 172.21.15.40:0/1524169005 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38fc0ca340 con 0x7f38fc070bc0 2024-09-13T18:02:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.982+0000 7f3902584700 1 -- 172.21.15.40:0/1524169005 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38fc15a020 con 0x7f38fc070bc0 2024-09-13T18:02:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.986+0000 7f38f1ffb700 1 -- 172.21.15.40:0/1524169005 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f38f8011830 con 0x7f38fc070bc0 2024-09-13T18:02:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.986+0000 7f38f1ffb700 1 --2- 172.21.15.40:0/1524169005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f38e0066730 0x7f38e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.986+0000 7f38f1ffb700 1 -- 172.21.15.40:0/1524169005 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f38f8087950 con 0x7f38fc070bc0 2024-09-13T18:02:58.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.986+0000 7f3900d81700 1 --2- 172.21.15.40:0/1524169005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f38e0066730 0x7f38e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:58.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.986+0000 7f3900d81700 1 --2- 172.21.15.40:0/1524169005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f38e0066730 0x7f38e0068bf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f38fc06b430 tx=0x7f38ec008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:59.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:58.990+0000 7f38f1ffb700 1 -- 172.21.15.40:0/1524169005 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f38f8054900 con 0x7f38fc070bc0 2024-09-13T18:02:59.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.298+0000 7f3902584700 1 -- 172.21.15.40:0/1524169005 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f38fc0711a0 con 0x7f38fc070bc0 2024-09-13T18:02:59.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.298+0000 7f38f1ffb700 1 -- 172.21.15.40:0/1524169005 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1539 (secure 0 0 0) 0x7f38f801e620 con 0x7f38fc070bc0 2024-09-13T18:02:59.312 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":98,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:02:59.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.302+0000 7f3902584700 1 -- 172.21.15.40:0/1524169005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f38e0066730 msgr2=0x7f38e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.302+0000 7f3902584700 1 --2- 172.21.15.40:0/1524169005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f38e0066730 0x7f38e0068bf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f38fc06b430 tx=0x7f38ec008030 comp rx=0 tx=0).stop 2024-09-13T18:02:59.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.302+0000 7f3902584700 1 -- 172.21.15.40:0/1524169005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc070bc0 msgr2=0x7f38fc148250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.302+0000 7f3902584700 1 --2- 172.21.15.40:0/1524169005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc070bc0 0x7f38fc148250 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f38f800c500 tx=0x7f38f800c8c0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.302+0000 7f3902584700 1 -- 172.21.15.40:0/1524169005 shutdown_connections 2024-09-13T18:02:59.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.302+0000 7f3902584700 1 --2- 172.21.15.40:0/1524169005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38fc1567e0 0x7f38fc1f0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.302+0000 7f3902584700 1 --2- 172.21.15.40:0/1524169005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f38e0066730 0x7f38e0068bf0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.302+0000 7f3902584700 1 --2- 172.21.15.40:0/1524169005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f38fc150690 0x7f38fc142840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.302+0000 7f3902584700 1 --2- 172.21.15.40:0/1524169005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38fc070bc0 0x7f38fc148250 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.302+0000 7f3902584700 1 -- 172.21.15.40:0/1524169005 >> 172.21.15.40:0/1524169005 conn(0x7f38fc0c88e0 msgr2=0x7f38fc0c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:59.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.302+0000 7f3902584700 1 -- 172.21.15.40:0/1524169005 shutdown_connections 2024-09-13T18:02:59.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.302+0000 7f3902584700 1 -- 172.21.15.40:0/1524169005 wait complete. 2024-09-13T18:02:59.327 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:02:59.327 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:02:59.328 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:02:59.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b96863700 1 Processor -- start 2024-09-13T18:02:59.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b96863700 1 -- start start 2024-09-13T18:02:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b96863700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7b880d87a0 0x7f7b880df110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b96863700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7b880469c0 0x7f7b88046df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b96863700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b88047610 con 0x7f7b88007b60 2024-09-13T18:02:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b96863700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b8804bef0 con 0x7f7b880d87a0 2024-09-13T18:02:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b96863700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b88006b50 con 0x7f7b880469c0 2024-09-13T18:02:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b95861700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7b880d87a0 0x7f7b880df110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b95861700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7b880d87a0 0x7f7b880df110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45892/0 (socket says 172.21.15.40:45892) 2024-09-13T18:02:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b95861700 1 -- 172.21.15.40:0/3420089317 learned_addr learned my addr 172.21.15.40:0/3420089317 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b96062700 1 --2- 172.21.15.40:0/3420089317 >> v2:172.21.15.43:3300/0 conn(0x7f7b880469c0 0x7f7b88046df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b95861700 1 -- 172.21.15.40:0/3420089317 >> v2:172.21.15.43:3300/0 conn(0x7f7b880469c0 msgr2=0x7f7b88046df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:59.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b95861700 1 --2- 172.21.15.40:0/3420089317 >> v2:172.21.15.43:3300/0 conn(0x7f7b880469c0 0x7f7b88046df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b95861700 1 -- 172.21.15.40:0/3420089317 >> v1:172.21.15.43:6789/0 conn(0x7f7b88007b60 legacy=0x7f7b8804b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:59.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b95861700 1 -- 172.21.15.40:0/3420089317 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b880076d0 con 0x7f7b880d87a0 2024-09-13T18:02:59.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b95861700 1 --2- 172.21.15.40:0/3420089317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b880d87a0 0x7f7b880df110 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7b8804d890 tx=0x7f7b8c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50544b7a5afed9e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b9485f700 1 -- 172.21.15.40:0/3420089317 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c004230 con 0x7f7b880d87a0 2024-09-13T18:02:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b9485f700 1 -- 172.21.15.40:0/3420089317 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8c004390 con 0x7f7b880d87a0 2024-09-13T18:02:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b9485f700 1 -- 172.21.15.40:0/3420089317 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0036c0 con 0x7f7b880d87a0 2024-09-13T18:02:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b96863700 1 -- 172.21.15.40:0/3420089317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b880d87a0 msgr2=0x7f7b880df110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b96863700 1 --2- 172.21.15.40:0/3420089317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b880d87a0 0x7f7b880df110 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7b8804d890 tx=0x7f7b8c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b96863700 1 -- 172.21.15.40:0/3420089317 shutdown_connections 2024-09-13T18:02:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b96863700 1 --2- 172.21.15.40:0/3420089317 >> v2:172.21.15.43:3300/0 conn(0x7f7b880469c0 0x7f7b88046df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b96863700 1 --2- 172.21.15.40:0/3420089317 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b880d87a0 0x7f7b880df110 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b96863700 1 -- 172.21.15.40:0/3420089317 >> 172.21.15.40:0/3420089317 conn(0x7f7b88050e60 msgr2=0x7f7b88051270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b96863700 1 -- 172.21.15.40:0/3420089317 shutdown_connections 2024-09-13T18:02:59.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.426+0000 7f7b96863700 1 -- 172.21.15.40:0/3420089317 wait complete. 2024-09-13T18:02:59.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b96863700 1 Processor -- start 2024-09-13T18:02:59.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b96863700 1 -- start start 2024-09-13T18:02:59.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b96863700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b880469c0 0x7f7b880d0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b96863700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b880d87a0 0x7f7b880d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b96863700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b88048a00 0x7f7b88048e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b96863700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8804a620 con 0x7f7b880469c0 2024-09-13T18:02:59.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b96863700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b880493c0 con 0x7f7b88048a00 2024-09-13T18:02:59.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b96863700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b880df440 con 0x7f7b880d87a0 2024-09-13T18:02:59.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b95861700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b880469c0 0x7f7b880d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b95861700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b880469c0 0x7f7b880d0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45906/0 (socket says 172.21.15.40:45906) 2024-09-13T18:02:59.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b95861700 1 -- 172.21.15.40:0/2186866092 learned_addr learned my addr 172.21.15.40:0/2186866092 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:59.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b95060700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b880d87a0 0x7f7b880d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b96062700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b88048a00 0x7f7b88048e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b95060700 1 -- 172.21.15.40:0/2186866092 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b880d87a0 msgr2=0x7f7b880d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:02:59.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b95060700 1 -- 172.21.15.40:0/2186866092 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b880d87a0 msgr2=0x7f7b880d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:02:59.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b95060700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b880d87a0 0x7f7b880d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:02:59.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b95060700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b880d87a0 0x7f7b880d0dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:02:59.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b95861700 1 -- 172.21.15.40:0/2186866092 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b880d87a0 msgr2=0x7f7b880d0dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:02:59.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b95861700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b880d87a0 0x7f7b880d0dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b95861700 1 -- 172.21.15.40:0/2186866092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b88048a00 msgr2=0x7f7b88048e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b95861700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b88048a00 0x7f7b88048e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b95861700 1 -- 172.21.15.40:0/2186866092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8c009570 con 0x7f7b880469c0 2024-09-13T18:02:59.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.430+0000 7f7b95861700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b880469c0 0x7f7b880d0890 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800a8d0 tx=0x7f7b8c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:59.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b867fc700 1 -- 172.21.15.40:0/2186866092 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00b860 con 0x7f7b880469c0 2024-09-13T18:02:59.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b867fc700 1 -- 172.21.15.40:0/2186866092 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8c00b9c0 con 0x7f7b880469c0 2024-09-13T18:02:59.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b96863700 1 -- 172.21.15.40:0/2186866092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b880d2330 con 0x7f7b880469c0 2024-09-13T18:02:59.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b867fc700 1 -- 172.21.15.40:0/2186866092 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00bd50 con 0x7f7b880469c0 2024-09-13T18:02:59.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b96863700 1 -- 172.21.15.40:0/2186866092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8804ab80 con 0x7f7b880469c0 2024-09-13T18:02:59.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b96863700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b88007b60 0x7f7b88048160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b96863700 1 -- 172.21.15.40:0/2186866092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b880095f0 con 0x7f7b88007b60 2024-09-13T18:02:59.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b95060700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b88007b60 0x7f7b88048160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b95060700 1 -- 172.21.15.40:0/2186866092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b8804d310 con 0x7f7b88007b60 2024-09-13T18:02:59.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b95060700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b88007b60 0x7f7b88048160 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7b880df120 tx=0x7f7b8000e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:59.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b867fc700 1 -- 172.21.15.40:0/2186866092 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7b8c01b440 con 0x7f7b880469c0 2024-09-13T18:02:59.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b867fc700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7b74066760 0x7f7b74068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b867fc700 1 -- 172.21.15.40:0/2186866092 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7b8c085ae0 con 0x7f7b880469c0 2024-09-13T18:02:59.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b96062700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7b74066760 0x7f7b74068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b867fc700 1 -- 172.21.15.40:0/2186866092 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80006a90 con 0x7f7b88007b60 2024-09-13T18:02:59.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b867fc700 1 -- 172.21.15.40:0/2186866092 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7b80006bf0 con 0x7f7b88007b60 2024-09-13T18:02:59.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b867fc700 1 -- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b88007b60 msgr2=0x7f7b88048160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b867fc700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b88007b60 0x7f7b88048160 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7b880df120 tx=0x7f7b8000e860 comp rx=0 tx=0).stop 2024-09-13T18:02:59.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b867fc700 1 -- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b88007b60 msgr2=0x7f7b88048160 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:59.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b867fc700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b88007b60 0x7f7b88048160 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.434+0000 7f7b96062700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7b74066760 0x7f7b74068c20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7b900641b0 tx=0x7f7b9008e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:59.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b96863700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b88046500 0x7f7b88177da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b96863700 1 -- 172.21.15.40:0/2186866092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b880d8b80 con 0x7f7b88046500 2024-09-13T18:02:59.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b95060700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b88046500 0x7f7b88177da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b95060700 1 -- 172.21.15.40:0/2186866092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7b88006da0 con 0x7f7b88046500 2024-09-13T18:02:59.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b95060700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b88046500 0x7f7b88177da0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7b80017000 tx=0x7f7b80003810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:59.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b867fc700 1 -- 172.21.15.40:0/2186866092 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80022070 con 0x7f7b88046500 2024-09-13T18:02:59.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b867fc700 1 -- 172.21.15.40:0/2186866092 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4949 (secure 0 0 0) 0x7f7b8001d280 con 0x7f7b88046500 2024-09-13T18:02:59.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b867fc700 1 -- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b88046500 msgr2=0x7f7b88177da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b867fc700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b88046500 0x7f7b88177da0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7b80017000 tx=0x7f7b80003810 comp rx=0 tx=0).stop 2024-09-13T18:02:59.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b867fc700 1 -- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b88046500 msgr2=0x7f7b88177da0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:59.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b867fc700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b88046500 0x7f7b88177da0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.457 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:02:59.457 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:02:59.457 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:02:59.458 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:02:59.458 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 98, 2024-09-13T18:02:59.458 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:02:59.458 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:02:59.458 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:02:59.458 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:02:59.458 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:59.458 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:02:59.458 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:59.458 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:02:59.458 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:02:59.458 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:59.458 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:59.458 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:59.458 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:59.459 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:59.459 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:59.459 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:59.459 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:59.459 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:59.459 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:59.459 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:59.459 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:02:59.459 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:02:59.459 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:59.459 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:59.459 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:59.459 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:59.459 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:59.460 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:59.460 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:59.460 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:59.460 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:59.460 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:59.460 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:59.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.460 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:02:59.460 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:02:59.460 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:02:59.460 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:02:59.460 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:02:59.460 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:02:59.460 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:59.461 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:59.461 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:02:59.461 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:02:59.461 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:02:59.461 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:02:59.461 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:02:59.461 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:02:59.461 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:02:59.461 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:59.461 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:02:59.461 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:59.461 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:59.461 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:59.461 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:59.461 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:59.462 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:59.462 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:59.462 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:59.462 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:59.462 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:59.462 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:59.462 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:02:59.462 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.462 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:02:59.462 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.462 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:02:59.462 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:02:59.462 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:59.462 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:59.463 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.463 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:59.463 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:02:59.463 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:59.463 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.463 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.463 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:59.463 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:02:59.463 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:59.463 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.463 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:59.463 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.463 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:02:59.463 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:02:59.464 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:59.464 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:59.464 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:59.464 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.464 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.464 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:02:59.464 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:02:59.464 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:59.464 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:59.464 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.464 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:59.464 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:02:59.464 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:59.464 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.465 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.465 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:59.465 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:02:59.465 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:59.465 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.465 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:59.465 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.465 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:02:59.465 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:02:59.465 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:59.465 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:59.465 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:59.465 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.465 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.466 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:02:59.466 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:02:59.466 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:59.466 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:59.466 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.466 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:59.466 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:02:59.466 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:59.466 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.466 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.466 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:59.466 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:02:59.466 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:59.466 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.467 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:59.467 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.467 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:02:59.467 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:02:59.467 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:59.467 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:59.467 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:59.467 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.467 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:59.467 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.467 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:02:59.467 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:02:59.467 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.467 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:59.467 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:59.468 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:02:59.468 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.468 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:59.468 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:02:59.468 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.468 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:59.468 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:59.468 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:02:59.468 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.468 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:59.468 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:02:59.468 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.468 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:59.469 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:59.469 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 8 2024-09-13T18:02:59.469 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.469 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:59.469 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:02:59.469 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.469 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:59.469 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:59.469 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:02:59.469 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.469 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:59.469 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:02:59.469 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.469 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:59.469 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:59.470 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:02:59.470 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.470 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:59.470 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.470 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:02:59.470 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:02:59.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b96863700 1 -- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7b74066760 msgr2=0x7f7b74068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b96863700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7b74066760 0x7f7b74068c20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7b900641b0 tx=0x7f7b9008e040 comp rx=0 tx=0).stop 2024-09-13T18:02:59.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b96863700 1 -- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b880469c0 msgr2=0x7f7b880d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b96863700 1 --2- 172.21.15.40:0/2186866092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b880469c0 0x7f7b880d0890 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800a8d0 tx=0x7f7b8c004770 comp rx=0 tx=0).stop 2024-09-13T18:02:59.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b96062700 1 -- 172.21.15.40:0/2186866092 reap_dead start 2024-09-13T18:02:59.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b96062700 1 -- 172.21.15.40:0/2186866092 reap_dead start 2024-09-13T18:02:59.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b96863700 1 -- 172.21.15.40:0/2186866092 shutdown_connections 2024-09-13T18:02:59.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b96863700 1 -- 172.21.15.40:0/2186866092 >> 172.21.15.40:0/2186866092 conn(0x7f7b88050e60 msgr2=0x7f7b8804d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:59.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b96863700 1 -- 172.21.15.40:0/2186866092 shutdown_connections 2024-09-13T18:02:59.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.446+0000 7f7b96863700 1 -- 172.21.15.40:0/2186866092 wait complete. 2024-09-13T18:02:59.472 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:02:59.472 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:02:59.472 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:02:59.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.578+0000 7f6f7cb7a700 1 Processor -- start 2024-09-13T18:02:59.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.578+0000 7f6f7cb7a700 1 -- start start 2024-09-13T18:02:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.578+0000 7f6f7cb7a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6f781537e0 0x7f6f7815a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.578+0000 7f6f7cb7a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f78071b60 con 0x7f6f780726e0 2024-09-13T18:02:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.578+0000 7f6f7cb7a700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f780c7990 con 0x7f6f7806a390 2024-09-13T18:02:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.578+0000 7f6f7cb7a700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f780b2330 con 0x7f6f781537e0 2024-09-13T18:02:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.578+0000 7f6f75d9b700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6f780726e0 0x7f6f780c7280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58284/0 (socket says 172.21.15.40:58284) 2024-09-13T18:02:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.578+0000 7f6f76d9d700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6f7806a390 0x7f6f78163f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:56972/0 (socket says 172.21.15.40:56972) 2024-09-13T18:02:59.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.578+0000 7f6f75d9b700 1 -- 172.21.15.40:0/2741862217 learned_addr learned my addr 172.21.15.40:0/2741862217 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:02:59.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.578+0000 7f6f7659c700 1 --2- 172.21.15.40:0/2741862217 >> v2:172.21.15.43:3300/0 conn(0x7f6f781537e0 0x7f6f7815a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.578+0000 7f6f7559a700 1 -- 172.21.15.40:0/2741862217 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004678470 0 0) 0x7f6f78071b60 con 0x7f6f780726e0 2024-09-13T18:02:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.578+0000 7f6f7559a700 1 -- 172.21.15.40:0/2741862217 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f600035d0 con 0x7f6f780726e0 2024-09-13T18:02:59.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.578+0000 7f6f7559a700 1 -- 172.21.15.40:0/2741862217 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409169372 0 0) 0x7f6f780c7990 con 0x7f6f7806a390 2024-09-13T18:02:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.578+0000 7f6f7559a700 1 -- 172.21.15.40:0/2741862217 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f78071b60 con 0x7f6f7806a390 2024-09-13T18:02:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.578+0000 7f6f7559a700 1 -- 172.21.15.40:0/2741862217 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4104032954 0 0) 0x7f6f600035d0 con 0x7f6f780726e0 2024-09-13T18:02:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.578+0000 7f6f7659c700 1 -- 172.21.15.40:0/2741862217 >> v1:172.21.15.43:6789/0 conn(0x7f6f7806a390 legacy=0x7f6f78163f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.578+0000 7f6f7659c700 1 -- 172.21.15.40:0/2741862217 >> v1:172.21.15.40:6789/0 conn(0x7f6f780726e0 legacy=0x7f6f780c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.578+0000 7f6f7659c700 1 -- 172.21.15.40:0/2741862217 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f78071db0 con 0x7f6f781537e0 2024-09-13T18:02:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7659c700 1 --2- 172.21.15.40:0/2741862217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f781537e0 0x7f6f7815a0b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009850 tx=0x7f6f68009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8974d05402618cd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7559a700 1 -- 172.21.15.40:0/2741862217 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c001670 con 0x7f6f781537e0 2024-09-13T18:02:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7559a700 1 -- 172.21.15.40:0/2741862217 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f68004030 con 0x7f6f781537e0 2024-09-13T18:02:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7559a700 1 -- 172.21.15.40:0/2741862217 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f680043c0 con 0x7f6f781537e0 2024-09-13T18:02:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7cb7a700 1 -- 172.21.15.40:0/2741862217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f781537e0 msgr2=0x7f6f7815a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7cb7a700 1 --2- 172.21.15.40:0/2741862217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f781537e0 0x7f6f7815a0b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009850 tx=0x7f6f68009b60 comp rx=0 tx=0).stop 2024-09-13T18:02:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7cb7a700 1 -- 172.21.15.40:0/2741862217 shutdown_connections 2024-09-13T18:02:59.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7cb7a700 1 --2- 172.21.15.40:0/2741862217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f781537e0 0x7f6f7815a0b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7cb7a700 1 -- 172.21.15.40:0/2741862217 >> 172.21.15.40:0/2741862217 conn(0x7f6f7806e660 msgr2=0x7f6f7806ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:59.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7cb7a700 1 -- 172.21.15.40:0/2741862217 shutdown_connections 2024-09-13T18:02:59.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7cb7a700 1 -- 172.21.15.40:0/2741862217 wait complete. 2024-09-13T18:02:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7cb7a700 1 Processor -- start 2024-09-13T18:02:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7cb7a700 1 -- start start 2024-09-13T18:02:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7cb7a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f7806a390 0x7f6f781e39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7cb7a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f780726e0 0x7f6f780c3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7cb7a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f781537e0 0x7f6f78155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7cb7a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f781efc30 con 0x7f6f781537e0 2024-09-13T18:02:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7cb7a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f781f0160 con 0x7f6f7806a390 2024-09-13T18:02:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7cb7a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f781efe80 con 0x7f6f780726e0 2024-09-13T18:02:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f76d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f781537e0 0x7f6f78155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f7806a390 0x7f6f781e39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f7806a390 0x7f6f781e39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:34066/0 (socket says 172.21.15.40:34066) 2024-09-13T18:02:59.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f76d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f781537e0 0x7f6f78155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45926/0 (socket says 172.21.15.40:45926) 2024-09-13T18:02:59.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f7659c700 1 -- 172.21.15.40:0/2962373103 learned_addr learned my addr 172.21.15.40:0/2962373103 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:59.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f75d9b700 1 --2- 172.21.15.40:0/2962373103 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f780726e0 0x7f6f780c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f76d9d700 1 -- 172.21.15.40:0/2962373103 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f780726e0 msgr2=0x7f6f780c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.582+0000 7f6f76d9d700 1 --2- 172.21.15.40:0/2962373103 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f780726e0 0x7f6f780c3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f76d9d700 1 -- 172.21.15.40:0/2962373103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f7806a390 msgr2=0x7f6f781e39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f76d9d700 1 --2- 172.21.15.40:0/2962373103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f7806a390 0x7f6f781e39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f76d9d700 1 -- 172.21.15.40:0/2962373103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f680094b0 con 0x7f6f781537e0 2024-09-13T18:02:59.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f76d9d700 1 --2- 172.21.15.40:0/2962373103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f781537e0 0x7f6f78155e50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000cad0 tx=0x7f6f7000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:59.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f66ffd700 1 -- 172.21.15.40:0/2962373103 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7000a9a0 con 0x7f6f781537e0 2024-09-13T18:02:59.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f66ffd700 1 -- 172.21.15.40:0/2962373103 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7000ab00 con 0x7f6f781537e0 2024-09-13T18:02:59.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f7cb7a700 1 -- 172.21.15.40:0/2962373103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f78155090 con 0x7f6f781537e0 2024-09-13T18:02:59.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f66ffd700 1 -- 172.21.15.40:0/2962373103 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7000ae90 con 0x7f6f781537e0 2024-09-13T18:02:59.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f7cb7a700 1 -- 172.21.15.40:0/2962373103 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f78155650 con 0x7f6f781537e0 2024-09-13T18:02:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f7cb7a700 1 --2- 172.21.15.40:0/2962373103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f78155840 0x7f6f78156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f7cb7a700 1 -- 172.21.15.40:0/2962373103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f78067750 con 0x7f6f78155840 2024-09-13T18:02:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f7659c700 1 --2- 172.21.15.40:0/2962373103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f78155840 0x7f6f78156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f7659c700 1 -- 172.21.15.40:0/2962373103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f780c6d60 con 0x7f6f78155840 2024-09-13T18:02:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f66ffd700 1 -- 172.21.15.40:0/2962373103 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6f70007070 con 0x7f6f781537e0 2024-09-13T18:02:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f7659c700 1 --2- 172.21.15.40:0/2962373103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f78155840 0x7f6f78156390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6f68000c00 tx=0x7f6f68004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f66ffd700 1 --2- 172.21.15.40:0/2962373103 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6f58066760 0x7f6f58068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f66ffd700 1 -- 172.21.15.40:0/2962373103 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6f700841c0 con 0x7f6f781537e0 2024-09-13T18:02:59.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f75d9b700 1 --2- 172.21.15.40:0/2962373103 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6f58066760 0x7f6f58068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f66ffd700 1 -- 172.21.15.40:0/2962373103 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f68020070 con 0x7f6f78155840 2024-09-13T18:02:59.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f66ffd700 1 -- 172.21.15.40:0/2962373103 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6f68004570 con 0x7f6f78155840 2024-09-13T18:02:59.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f66ffd700 1 -- 172.21.15.40:0/2962373103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f78155840 msgr2=0x7f6f78156390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f66ffd700 1 --2- 172.21.15.40:0/2962373103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f78155840 0x7f6f78156390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6f68000c00 tx=0x7f6f68004770 comp rx=0 tx=0).stop 2024-09-13T18:02:59.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f66ffd700 1 -- 172.21.15.40:0/2962373103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f78155840 msgr2=0x7f6f78156390 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:59.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f66ffd700 1 --2- 172.21.15.40:0/2962373103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f78155840 0x7f6f78156390 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.586+0000 7f6f75d9b700 1 --2- 172.21.15.40:0/2962373103 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6f58066760 0x7f6f58068c20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6f7806db70 tx=0x7f6f6c001d60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:59.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.598+0000 7f6f7cb7a700 1 --2- 172.21.15.40:0/2962373103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f781e2790 0x7f6f781f2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.598+0000 7f6f7cb7a700 1 -- 172.21.15.40:0/2962373103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f780c6a80 con 0x7f6f781e2790 2024-09-13T18:02:59.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.598+0000 7f6f7659c700 1 --2- 172.21.15.40:0/2962373103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f781e2790 0x7f6f781f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.598+0000 7f6f7659c700 1 -- 172.21.15.40:0/2962373103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6f7806b220 con 0x7f6f781e2790 2024-09-13T18:02:59.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.598+0000 7f6f7659c700 1 --2- 172.21.15.40:0/2962373103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f781e2790 0x7f6f781f2720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009ba0 tx=0x7f6f68011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:59.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.598+0000 7f6f66ffd700 1 -- 172.21.15.40:0/2962373103 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6802cb30 con 0x7f6f781e2790 2024-09-13T18:02:59.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.598+0000 7f6f66ffd700 1 -- 172.21.15.40:0/2962373103 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4615 (secure 0 0 0) 0x7f6f6802cc90 con 0x7f6f781e2790 2024-09-13T18:02:59.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.598+0000 7f6f66ffd700 1 -- 172.21.15.40:0/2962373103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f781e2790 msgr2=0x7f6f781f2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.598+0000 7f6f66ffd700 1 --2- 172.21.15.40:0/2962373103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f781e2790 0x7f6f781f2720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009ba0 tx=0x7f6f68011000 comp rx=0 tx=0).stop 2024-09-13T18:02:59.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.598+0000 7f6f66ffd700 1 -- 172.21.15.40:0/2962373103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f781e2790 msgr2=0x7f6f781f2720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:59.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.598+0000 7f6f66ffd700 1 --2- 172.21.15.40:0/2962373103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f781e2790 0x7f6f781f2720 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.612 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:02:59.612 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:02:59.612 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:02:59.612 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:02:59.612 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 98, 2024-09-13T18:02:59.612 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:02:59.612 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:02:59.612 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:02:59.612 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:02:59.612 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:59.612 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 3, 2024-09-13T18:02:59.612 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:59.612 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:02:59.613 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:02:59.613 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:59.613 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:59.613 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:59.613 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:59.613 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:59.613 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:59.613 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:59.613 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:59.613 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:59.613 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:59.613 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:59.613 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:02:59.613 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:02:59.614 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:59.614 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:59.614 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:59.614 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:59.614 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:59.614 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:59.614 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:59.614 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:59.614 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:59.614 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:59.614 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:59.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.614 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:02:59.614 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:02:59.614 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:02:59.615 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:02:59.615 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:02:59.615 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:02:59.615 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:59.615 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:59.615 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:02:59.615 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:02:59.615 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:02:59.615 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:02:59.615 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:02:59.615 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:02:59.615 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:02:59.615 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:59.616 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:02:59.616 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:59.616 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:59.616 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:59.616 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:59.616 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:59.616 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:59.616 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:59.616 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:59.616 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:59.616 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:59.616 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:59.616 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:02:59.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.617 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:02:59.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.617 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:02:59.617 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:02:59.617 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:59.617 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:59.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.617 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:59.617 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:02:59.617 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:59.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.617 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:59.617 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:02:59.618 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:59.618 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.618 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:59.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.618 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:02:59.618 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:02:59.618 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:59.618 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:59.618 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:59.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.618 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:02:59.618 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:02:59.618 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:59.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:59.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.619 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:59.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:02:59.619 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:59.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.619 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:59.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:02:59.619 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:59.619 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.619 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:59.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:02:59.620 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:02:59.620 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:59.620 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:59.620 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:59.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.620 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:02:59.620 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:02:59.620 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:59.620 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:59.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.620 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:59.620 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:02:59.620 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:59.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.621 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:59.621 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:02:59.621 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:59.621 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.621 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:59.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.621 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:02:59.621 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:02:59.621 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:59.621 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:59.621 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:59.622 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.622 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:59.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.622 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:02:59.622 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:02:59.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.622 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:59.622 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:59.622 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:02:59.622 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.622 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:59.622 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:02:59.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.622 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:59.622 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:59.623 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:02:59.623 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.623 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:59.623 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:02:59.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.623 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:59.623 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:59.623 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:02:59.623 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.623 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:59.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.623 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:02:59.623 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:02:59.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.602+0000 7f6f7cb7a700 1 -- 172.21.15.40:0/2962373103 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6f58066760 msgr2=0x7f6f58068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.602+0000 7f6f7cb7a700 1 --2- 172.21.15.40:0/2962373103 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6f58066760 0x7f6f58068c20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6f7806db70 tx=0x7f6f6c001d60 comp rx=0 tx=0).stop 2024-09-13T18:02:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.602+0000 7f6f7cb7a700 1 -- 172.21.15.40:0/2962373103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f781537e0 msgr2=0x7f6f78155e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.602+0000 7f6f7cb7a700 1 --2- 172.21.15.40:0/2962373103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f781537e0 0x7f6f78155e50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000cad0 tx=0x7f6f7000cde0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.602+0000 7f6f76d9d700 1 -- 172.21.15.40:0/2962373103 reap_dead start 2024-09-13T18:02:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.602+0000 7f6f76d9d700 1 -- 172.21.15.40:0/2962373103 reap_dead start 2024-09-13T18:02:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.602+0000 7f6f7cb7a700 1 -- 172.21.15.40:0/2962373103 shutdown_connections 2024-09-13T18:02:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.602+0000 7f6f7cb7a700 1 -- 172.21.15.40:0/2962373103 >> 172.21.15.40:0/2962373103 conn(0x7f6f7806e660 msgr2=0x7f6f780c7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.602+0000 7f6f7cb7a700 1 -- 172.21.15.40:0/2962373103 shutdown_connections 2024-09-13T18:02:59.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.602+0000 7f6f7cb7a700 1 -- 172.21.15.40:0/2962373103 wait complete. 2024-09-13T18:02:59.626 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:02:59.626 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:02:59.626 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:02:59.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.742+0000 7f02a3fff700 1 Processor -- start 2024-09-13T18:02:59.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.742+0000 7f02a3fff700 1 -- start start 2024-09-13T18:02:59.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02a4153750 0x7f02a40c6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a3fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f02a415cfc0 0x7f02a415d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a3fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a415cb30 con 0x7f02a40c2980 2024-09-13T18:02:59.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a3fff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02a415d970 con 0x7f02a415cfc0 2024-09-13T18:02:59.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a3fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a41597c0 con 0x7f02a4153750 2024-09-13T18:02:59.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a27fc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f02a40c2980 0x7f02a40c8600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58290/0 (socket says 172.21.15.40:58290) 2024-09-13T18:02:59.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a27fc700 1 -- 172.21.15.40:0/3635957773 learned_addr learned my addr 172.21.15.40:0/3635957773 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:02:59.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a37fe700 1 --2- 172.21.15.40:0/3635957773 >> v2:172.21.15.43:3300/0 conn(0x7f02a415cfc0 0x7f02a415d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a2ffd700 1 --2- 172.21.15.40:0/3635957773 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02a4153750 0x7f02a40c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a1ffb700 1 -- 172.21.15.40:0/3635957773 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320597492 0 0) 0x7f02a415cb30 con 0x7f02a40c2980 2024-09-13T18:02:59.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a1ffb700 1 -- 172.21.15.40:0/3635957773 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f028c0035d0 con 0x7f02a40c2980 2024-09-13T18:02:59.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a1ffb700 1 -- 172.21.15.40:0/3635957773 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 557320691 0 0) 0x7f028c0035d0 con 0x7f02a40c2980 2024-09-13T18:02:59.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a1ffb700 1 -- 172.21.15.40:0/3635957773 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02a4153750 msgr2=0x7f02a40c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:59.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a1ffb700 1 --2- 172.21.15.40:0/3635957773 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02a4153750 0x7f02a40c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a1ffb700 1 -- 172.21.15.40:0/3635957773 >> v2:172.21.15.43:3300/0 conn(0x7f02a415cfc0 msgr2=0x7f02a415d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:59.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a1ffb700 1 --2- 172.21.15.40:0/3635957773 >> v2:172.21.15.43:3300/0 conn(0x7f02a415cfc0 0x7f02a415d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a1ffb700 1 -- 172.21.15.40:0/3635957773 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a4159aa0 con 0x7f02a40c2980 2024-09-13T18:02:59.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a1ffb700 1 -- 172.21.15.40:0/3635957773 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f0298001670 con 0x7f02a40c2980 2024-09-13T18:02:59.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a1ffb700 1 -- 172.21.15.40:0/3635957773 >> v1:172.21.15.40:6789/0 conn(0x7f02a40c2980 legacy=0x7f02a40c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a1ffb700 1 --2- 172.21.15.40:0/3635957773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f028c003a20 0x7f028c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a1ffb700 1 --2- 172.21.15.40:0/3635957773 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02a4153750 0x7f028c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a1ffb700 1 --2- 172.21.15.40:0/3635957773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f028c004db0 0x7f028c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a1ffb700 1 -- 172.21.15.40:0/3635957773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a415d970 con 0x7f028c003a20 2024-09-13T18:02:59.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a1ffb700 1 -- 172.21.15.40:0/3635957773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a41597c0 con 0x7f028c004db0 2024-09-13T18:02:59.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a1ffb700 1 -- 172.21.15.40:0/3635957773 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028c00ba30 con 0x7f02a4153750 2024-09-13T18:02:59.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a37fe700 1 --2- 172.21.15.40:0/3635957773 >> v2:172.21.15.43:3300/0 conn(0x7f02a415cfc0 0x7f02a415d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:02:59.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a2ffd700 1 --2- 172.21.15.40:0/3635957773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f028c003a20 0x7f028c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a27fc700 1 --2- 172.21.15.40:0/3635957773 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02a4153750 0x7f028c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a37fe700 1 --2- 172.21.15.40:0/3635957773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f028c004db0 0x7f028c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a2ffd700 1 -- 172.21.15.40:0/3635957773 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02a4153750 msgr2=0x7f028c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a2ffd700 1 --2- 172.21.15.40:0/3635957773 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02a4153750 0x7f028c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a2ffd700 1 -- 172.21.15.40:0/3635957773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f028c004db0 msgr2=0x7f028c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a2ffd700 1 --2- 172.21.15.40:0/3635957773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f028c004db0 0x7f028c005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.746+0000 7f02a2ffd700 1 -- 172.21.15.40:0/3635957773 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c0035d0 con 0x7f028c003a20 2024-09-13T18:02:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.750+0000 7f02a2ffd700 1 --2- 172.21.15.40:0/3635957773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f028c003a20 0x7f028c003e40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0294009b70 tx=0x7f028c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78590643d7f25f59 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:59.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.750+0000 7f02a1ffb700 1 -- 172.21.15.40:0/3635957773 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029400e6c0 con 0x7f028c003a20 2024-09-13T18:02:59.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.750+0000 7f02a1ffb700 1 -- 172.21.15.40:0/3635957773 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029400e820 con 0x7f028c003a20 2024-09-13T18:02:59.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.750+0000 7f02a1ffb700 1 -- 172.21.15.40:0/3635957773 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029400eb90 con 0x7f028c003a20 2024-09-13T18:02:59.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.750+0000 7f02a3fff700 1 -- 172.21.15.40:0/3635957773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f028c003a20 msgr2=0x7f028c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.750+0000 7f02a3fff700 1 --2- 172.21.15.40:0/3635957773 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f028c003a20 0x7f028c003e40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0294009b70 tx=0x7f028c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.750+0000 7f02a37fe700 1 -- 172.21.15.40:0/3635957773 reap_dead start 2024-09-13T18:02:59.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.750+0000 7f02a3fff700 1 -- 172.21.15.40:0/3635957773 shutdown_connections 2024-09-13T18:02:59.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.750+0000 7f02a3fff700 1 -- 172.21.15.40:0/3635957773 >> 172.21.15.40:0/3635957773 conn(0x7f02a406e620 msgr2=0x7f02a406ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:59.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.750+0000 7f02a3fff700 1 -- 172.21.15.40:0/3635957773 shutdown_connections 2024-09-13T18:02:59.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.750+0000 7f02a3fff700 1 -- 172.21.15.40:0/3635957773 wait complete. 2024-09-13T18:02:59.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.750+0000 7f02a3fff700 1 Processor -- start 2024-09-13T18:02:59.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.750+0000 7f02a3fff700 1 -- start start 2024-09-13T18:02:59.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a3fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f028c003a20 0x7f02a4154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02a40c2440 0x7f02a414e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02a4155de0 0x7f02a41f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a3fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a41ef400 con 0x7f02a4155de0 2024-09-13T18:02:59.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a3fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a41ef8f0 con 0x7f028c003a20 2024-09-13T18:02:59.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a3fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a4155620 con 0x7f02a40c2440 2024-09-13T18:02:59.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02a4155de0 0x7f02a41f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02a4155de0 0x7f02a41f1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45944/0 (socket says 172.21.15.40:45944) 2024-09-13T18:02:59.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a37fe700 1 -- 172.21.15.40:0/999686563 learned_addr learned my addr 172.21.15.40:0/999686563 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:59.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a2ffd700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f028c003a20 0x7f02a4154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a27fc700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02a40c2440 0x7f02a414e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a37fe700 1 -- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02a40c2440 msgr2=0x7f02a414e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a37fe700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02a40c2440 0x7f02a414e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a37fe700 1 -- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f028c003a20 msgr2=0x7f02a4154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a37fe700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f028c003a20 0x7f02a4154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a37fe700 1 -- 172.21.15.40:0/999686563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a40b1c60 con 0x7f02a4155de0 2024-09-13T18:02:59.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a27fc700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f02a40c2440 0x7f02a414e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:02:59.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a37fe700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02a4155de0 0x7f02a41f1ef0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f02a415d3a0 tx=0x7f029c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:59.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f028bfff700 1 -- 172.21.15.40:0/999686563 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c00e070 con 0x7f02a4155de0 2024-09-13T18:02:59.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f028bfff700 1 -- 172.21.15.40:0/999686563 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029c003c50 con 0x7f02a4155de0 2024-09-13T18:02:59.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a3fff700 1 -- 172.21.15.40:0/999686563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a41ef610 con 0x7f02a4155de0 2024-09-13T18:02:59.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f028bfff700 1 -- 172.21.15.40:0/999686563 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c003de0 con 0x7f02a4155de0 2024-09-13T18:02:59.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a3fff700 1 -- 172.21.15.40:0/999686563 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a4072800 con 0x7f02a4155de0 2024-09-13T18:02:59.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f0289ffb700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f027c002a40 0x7f027c004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f0289ffb700 1 -- 172.21.15.40:0/999686563 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027c007f40 con 0x7f027c002a40 2024-09-13T18:02:59.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.754+0000 7f02a2ffd700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f027c002a40 0x7f027c004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.758+0000 7f02a2ffd700 1 -- 172.21.15.40:0/999686563 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f027c008100 con 0x7f027c002a40 2024-09-13T18:02:59.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.758+0000 7f028bfff700 1 -- 172.21.15.40:0/999686563 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f029c011a60 con 0x7f02a4155de0 2024-09-13T18:02:59.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.758+0000 7f02a2ffd700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f027c002a40 0x7f027c004f00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0294009b70 tx=0x7f0294004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:59.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.758+0000 7f028bfff700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0290066760 0x7f0290068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.758+0000 7f02a27fc700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0290066760 0x7f0290068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.758+0000 7f028bfff700 1 -- 172.21.15.40:0/999686563 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f029c084ba0 con 0x7f02a4155de0 2024-09-13T18:02:59.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.758+0000 7f028bfff700 1 -- 172.21.15.40:0/999686563 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0294021040 con 0x7f027c002a40 2024-09-13T18:02:59.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.758+0000 7f028bfff700 1 -- 172.21.15.40:0/999686563 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0294013d10 con 0x7f027c002a40 2024-09-13T18:02:59.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.758+0000 7f028bfff700 1 -- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f027c002a40 msgr2=0x7f027c004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.758+0000 7f02a27fc700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0290066760 0x7f0290068c20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0298001eb0 tx=0x7f0298009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:59.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.758+0000 7f028bfff700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f027c002a40 0x7f027c004f00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0294009b70 tx=0x7f0294004770 comp rx=0 tx=0).stop 2024-09-13T18:02:59.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.758+0000 7f028bfff700 1 -- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f027c002a40 msgr2=0x7f027c004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:59.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.758+0000 7f028bfff700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f027c002a40 0x7f027c004f00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.766+0000 7f0289ffb700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f027c007020 0x7f027c0074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.766+0000 7f0289ffb700 1 -- 172.21.15.40:0/999686563 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027c00a400 con 0x7f027c007020 2024-09-13T18:02:59.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.770+0000 7f02a2ffd700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f027c007020 0x7f027c0074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.770+0000 7f02a2ffd700 1 -- 172.21.15.40:0/999686563 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f027c00a5c0 con 0x7f027c007020 2024-09-13T18:02:59.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.770+0000 7f02a2ffd700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f027c007020 0x7f027c0074b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f027c0080d0 tx=0x7f02940047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:59.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.770+0000 7f028bfff700 1 -- 172.21.15.40:0/999686563 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0294005e50 con 0x7f027c007020 2024-09-13T18:02:59.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.770+0000 7f028bfff700 1 -- 172.21.15.40:0/999686563 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4449 (secure 0 0 0) 0x7f029402f6d0 con 0x7f027c007020 2024-09-13T18:02:59.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.770+0000 7f028bfff700 1 -- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f027c007020 msgr2=0x7f027c0074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.770+0000 7f028bfff700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f027c007020 0x7f027c0074b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f027c0080d0 tx=0x7f02940047e0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.770+0000 7f028bfff700 1 -- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f027c007020 msgr2=0x7f027c0074b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:02:59.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.770+0000 7f028bfff700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f027c007020 0x7f027c0074b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.782 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:02:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:02:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:02:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:02:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 98, 2024-09-13T18:02:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:02:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:02:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:02:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:02:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 3, 2024-09-13T18:02:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:02:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:02:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:02:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:02:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:02:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:02:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:02:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:02:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:02:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:02:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:02:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:02:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:02:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:02:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:02:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:02:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:02:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:02:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:02:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:02:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:02:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:02:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:02:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:02:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:02:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:02:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:02:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:02:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:02:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:02:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:02:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:02:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:02:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:02:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:02:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:02:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:02:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:02:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:02:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:02:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:02:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.790 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:59.790 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:02:59.790 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:59.790 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.790 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:59.790 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.790 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:02:59.790 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:02:59.790 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:59.790 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:59.790 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:59.790 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.791 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.791 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:02:59.791 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:02:59.791 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:02:59.791 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:02:59.791 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.791 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:02:59.791 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:02:59.791 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:59.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.791 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.791 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:02:59.791 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:02:59.792 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:02:59.792 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.792 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:59.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.792 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:02:59.792 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:02:59.792 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:02:59.792 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:02:59.792 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:02:59.792 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.792 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:59.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.792 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:02:59.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:02:59.792 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.793 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:59.793 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:59.793 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:02:59.793 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.793 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:02:59.793 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:02:59.793 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:02:59.793 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:02:59.793 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:02:59.793 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:02:59.793 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:02:59.793 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:02:59.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:02:59.794 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:02:59.794 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:02:59.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.770+0000 7f0289ffb700 1 -- 172.21.15.40:0/999686563 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0290066760 msgr2=0x7f0290068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.770+0000 7f0289ffb700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0290066760 0x7f0290068c20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0298001eb0 tx=0x7f0298009040 comp rx=0 tx=0).stop 2024-09-13T18:02:59.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.770+0000 7f0289ffb700 1 -- 172.21.15.40:0/999686563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02a4155de0 msgr2=0x7f02a41f1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.770+0000 7f0289ffb700 1 --2- 172.21.15.40:0/999686563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f02a4155de0 0x7f02a41f1ef0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f02a415d3a0 tx=0x7f029c009f90 comp rx=0 tx=0).stop 2024-09-13T18:02:59.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.770+0000 7f02a37fe700 1 -- 172.21.15.40:0/999686563 reap_dead start 2024-09-13T18:02:59.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.770+0000 7f02a37fe700 1 -- 172.21.15.40:0/999686563 reap_dead start 2024-09-13T18:02:59.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.770+0000 7f0289ffb700 1 -- 172.21.15.40:0/999686563 shutdown_connections 2024-09-13T18:02:59.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.770+0000 7f0289ffb700 1 -- 172.21.15.40:0/999686563 >> 172.21.15.40:0/999686563 conn(0x7f02a406e620 msgr2=0x7f02a40c1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:59.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.770+0000 7f0289ffb700 1 -- 172.21.15.40:0/999686563 shutdown_connections 2024-09-13T18:02:59.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.770+0000 7f0289ffb700 1 -- 172.21.15.40:0/999686563 wait complete. 2024-09-13T18:02:59.796 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:02:59.796 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:02:59.796 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:02:59.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.898+0000 7f343ce1e700 1 Processor -- start 2024-09-13T18:02:59.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.898+0000 7f343ce1e700 1 -- start start 2024-09-13T18:02:59.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f343ce1e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f34381534e0 0x7f3438159d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f343ce1e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f343806e960 0x7f34380c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f343ce1e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34380c6db0 con 0x7f3438070290 2024-09-13T18:02:59.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f343ce1e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34380c6920 con 0x7f343806e960 2024-09-13T18:02:59.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f343ce1e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f343815a260 con 0x7f34381534e0 2024-09-13T18:02:59.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f3437fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f343806e960 0x7f34380c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f3437fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f343806e960 0x7f34380c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45954/0 (socket says 172.21.15.40:45954) 2024-09-13T18:02:59.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f3437fff700 1 -- 172.21.15.40:0/2274802095 learned_addr learned my addr 172.21.15.40:0/2274802095 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:59.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f34377fe700 1 --2- 172.21.15.40:0/2274802095 >> v2:172.21.15.43:3300/0 conn(0x7f34381534e0 0x7f3438159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f34367fc700 1 -- 172.21.15.40:0/2274802095 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106859403 0 0) 0x7f34380c6db0 con 0x7f3438070290 2024-09-13T18:02:59.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f34367fc700 1 -- 172.21.15.40:0/2274802095 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34200035d0 con 0x7f3438070290 2024-09-13T18:02:59.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f3437fff700 1 -- 172.21.15.40:0/2274802095 >> v2:172.21.15.43:3300/0 conn(0x7f34381534e0 msgr2=0x7f3438159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:02:59.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f3437fff700 1 --2- 172.21.15.40:0/2274802095 >> v2:172.21.15.43:3300/0 conn(0x7f34381534e0 0x7f3438159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f3437fff700 1 -- 172.21.15.40:0/2274802095 >> v1:172.21.15.40:6789/0 conn(0x7f3438070290 legacy=0x7f343806d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f3437fff700 1 -- 172.21.15.40:0/2274802095 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34380c5d50 con 0x7f343806e960 2024-09-13T18:02:59.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f3437fff700 1 --2- 172.21.15.40:0/2274802095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f343806e960 0x7f34380c7d20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3438074470 tx=0x7f343000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bf9c92a482ef26 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:59.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f34367fc700 1 -- 172.21.15.40:0/2274802095 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343000e070 con 0x7f343806e960 2024-09-13T18:02:59.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f34367fc700 1 -- 172.21.15.40:0/2274802095 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343000b910 con 0x7f343806e960 2024-09-13T18:02:59.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f34367fc700 1 -- 172.21.15.40:0/2274802095 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343000bcc0 con 0x7f343806e960 2024-09-13T18:02:59.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f343ce1e700 1 -- 172.21.15.40:0/2274802095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f343806e960 msgr2=0x7f34380c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f343ce1e700 1 --2- 172.21.15.40:0/2274802095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f343806e960 0x7f34380c7d20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3438074470 tx=0x7f343000b330 comp rx=0 tx=0).stop 2024-09-13T18:02:59.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f343ce1e700 1 -- 172.21.15.40:0/2274802095 shutdown_connections 2024-09-13T18:02:59.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f343ce1e700 1 --2- 172.21.15.40:0/2274802095 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f343806e960 0x7f34380c7d20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f343ce1e700 1 --2- 172.21.15.40:0/2274802095 >> v2:172.21.15.43:3300/0 conn(0x7f34381534e0 0x7f3438159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f343ce1e700 1 -- 172.21.15.40:0/2274802095 >> 172.21.15.40:0/2274802095 conn(0x7f343806a390 msgr2=0x7f343806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:02:59.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f343ce1e700 1 -- 172.21.15.40:0/2274802095 shutdown_connections 2024-09-13T18:02:59.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.902+0000 7f343ce1e700 1 -- 172.21.15.40:0/2274802095 wait complete. 2024-09-13T18:02:59.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f343ce1e700 1 Processor -- start 2024-09-13T18:02:59.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f343ce1e700 1 -- start start 2024-09-13T18:02:59.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f343ce1e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f343806e960 0x7f343815a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f343ce1e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3438070290 0x7f343814ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f343ce1e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f34381534e0 0x7f34381f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f343ce1e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343815a9a0 con 0x7f343806e960 2024-09-13T18:02:59.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f343ce1e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3438149020 con 0x7f3438070290 2024-09-13T18:02:59.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f343ce1e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343815abb0 con 0x7f34381534e0 2024-09-13T18:02:59.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f34377fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f343806e960 0x7f343815a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f34377fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f343806e960 0x7f343815a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45958/0 (socket says 172.21.15.40:45958) 2024-09-13T18:02:59.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f34377fe700 1 -- 172.21.15.40:0/3470250904 learned_addr learned my addr 172.21.15.40:0/3470250904 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:02:59.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f3436ffd700 1 --2- 172.21.15.40:0/3470250904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3438070290 0x7f343814ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f3437fff700 1 --2- 172.21.15.40:0/3470250904 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f34381534e0 0x7f34381f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f34377fe700 1 -- 172.21.15.40:0/3470250904 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f34381534e0 msgr2=0x7f34381f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f34377fe700 1 --2- 172.21.15.40:0/3470250904 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f34381534e0 0x7f34381f2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f34377fe700 1 -- 172.21.15.40:0/3470250904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3438070290 msgr2=0x7f343814ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:02:59.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f34377fe700 1 --2- 172.21.15.40:0/3470250904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3438070290 0x7f343814ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:02:59.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f34377fe700 1 -- 172.21.15.40:0/3470250904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3430009b30 con 0x7f343806e960 2024-09-13T18:02:59.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f3436ffd700 1 --2- 172.21.15.40:0/3470250904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3438070290 0x7f343814ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:02:59.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f3437fff700 1 --2- 172.21.15.40:0/3470250904 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f34381534e0 0x7f34381f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:02:59.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f34377fe700 1 --2- 172.21.15.40:0/3470250904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f343806e960 0x7f343815a460 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f342800e9d0 tx=0x7f342800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:59.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f341ffff700 1 -- 172.21.15.40:0/3470250904 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428009720 con 0x7f343806e960 2024-09-13T18:02:59.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f341ffff700 1 -- 172.21.15.40:0/3470250904 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342800c850 con 0x7f343806e960 2024-09-13T18:02:59.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f343ce1e700 1 -- 172.21.15.40:0/3470250904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343814e3f0 con 0x7f343806e960 2024-09-13T18:02:59.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f341ffff700 1 -- 172.21.15.40:0/3470250904 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342800cbc0 con 0x7f343806e960 2024-09-13T18:02:59.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.906+0000 7f343ce1e700 1 -- 172.21.15.40:0/3470250904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34381f2bb0 con 0x7f343806e960 2024-09-13T18:02:59.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.910+0000 7f343ce1e700 1 -- 172.21.15.40:0/3470250904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34380c8c20 con 0x7f343806e960 2024-09-13T18:02:59.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.910+0000 7f341ffff700 1 -- 172.21.15.40:0/3470250904 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f342800cd20 con 0x7f343806e960 2024-09-13T18:02:59.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.910+0000 7f341ffff700 1 --2- 172.21.15.40:0/3470250904 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3424066730 0x7f3424068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:02:59.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.910+0000 7f341ffff700 1 -- 172.21.15.40:0/3470250904 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3428084960 con 0x7f343806e960 2024-09-13T18:02:59.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.910+0000 7f3436ffd700 1 --2- 172.21.15.40:0/3470250904 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3424066730 0x7f3424068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:02:59.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.914+0000 7f3436ffd700 1 --2- 172.21.15.40:0/3470250904 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3424066730 0x7f3424068bf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f343806b470 tx=0x7f342c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:02:59.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:02:59.922+0000 7f341ffff700 1 -- 172.21.15.40:0/3470250904 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f34280519a0 con 0x7f343806e960 2024-09-13T18:03:00.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:00.158+0000 7f343ce1e700 1 -- 172.21.15.40:0/3470250904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f34381538c0 con 0x7f343806e960 2024-09-13T18:03:00.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:00.158+0000 7f341ffff700 1 -- 172.21.15.40:0/3470250904 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f3428054fc0 con 0x7f343806e960 2024-09-13T18:03:00.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:00.162+0000 7f343ce1e700 1 -- 172.21.15.40:0/3470250904 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3424066730 msgr2=0x7f3424068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:00.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:00.162+0000 7f343ce1e700 1 --2- 172.21.15.40:0/3470250904 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3424066730 0x7f3424068bf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f343806b470 tx=0x7f342c009040 comp rx=0 tx=0).stop 2024-09-13T18:03:00.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:00.162+0000 7f343ce1e700 1 -- 172.21.15.40:0/3470250904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f343806e960 msgr2=0x7f343815a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:00.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:00.162+0000 7f343ce1e700 1 --2- 172.21.15.40:0/3470250904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f343806e960 0x7f343815a460 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f342800e9d0 tx=0x7f342800ed90 comp rx=0 tx=0).stop 2024-09-13T18:03:00.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:00.162+0000 7f343ce1e700 1 -- 172.21.15.40:0/3470250904 shutdown_connections 2024-09-13T18:03:00.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:00.162+0000 7f343ce1e700 1 --2- 172.21.15.40:0/3470250904 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3424066730 0x7f3424068bf0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:00.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:00.162+0000 7f343ce1e700 1 --2- 172.21.15.40:0/3470250904 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f34381534e0 0x7f34381f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:00.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:00.162+0000 7f343ce1e700 1 --2- 172.21.15.40:0/3470250904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3438070290 0x7f343814ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:00.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:00.162+0000 7f343ce1e700 1 --2- 172.21.15.40:0/3470250904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f343806e960 0x7f343815a460 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:00.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:00.162+0000 7f343ce1e700 1 -- 172.21.15.40:0/3470250904 >> 172.21.15.40:0/3470250904 conn(0x7f343806a390 msgr2=0x7f3438151910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:00.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:00.162+0000 7f343ce1e700 1 -- 172.21.15.40:0/3470250904 shutdown_connections 2024-09-13T18:03:00.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:00.162+0000 7f343ce1e700 1 -- 172.21.15.40:0/3470250904 wait complete. 2024-09-13T18:03:00.188 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:03:05.949 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:03:05.952 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:03:10.189 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:03:10.189 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:03:10.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.310+0000 7fdf6ff55700 1 Processor -- start 2024-09-13T18:03:10.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.310+0000 7fdf6ff55700 1 -- start start 2024-09-13T18:03:10.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.310+0000 7fdf6ff55700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdf6814fb40 0x7fdf680c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.310+0000 7fdf6ff55700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdf680c7980 0x7fdf68156e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.310+0000 7fdf6ff55700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf6806e1d0 con 0x7fdf680c83e0 2024-09-13T18:03:10.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.310+0000 7fdf6ff55700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf68156390 con 0x7fdf6814fb40 2024-09-13T18:03:10.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.310+0000 7fdf6ff55700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf680c7dd0 con 0x7fdf680c7980 2024-09-13T18:03:10.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.310+0000 7fdf6dcf1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdf6814fb40 0x7fdf680c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.310+0000 7fdf6dcf1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdf6814fb40 0x7fdf680c6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55938/0 (socket says 172.21.15.40:55938) 2024-09-13T18:03:10.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.310+0000 7fdf6dcf1700 1 -- 172.21.15.40:0/1715764469 learned_addr learned my addr 172.21.15.40:0/1715764469 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:10.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.310+0000 7fdf6d4f0700 1 --2- 172.21.15.40:0/1715764469 >> v2:172.21.15.43:3300/0 conn(0x7fdf680c7980 0x7fdf68156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.314+0000 7fdf6dcf1700 1 -- 172.21.15.40:0/1715764469 >> v2:172.21.15.43:3300/0 conn(0x7fdf680c7980 msgr2=0x7fdf68156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:03:10.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.314+0000 7fdf6dcf1700 1 --2- 172.21.15.40:0/1715764469 >> v2:172.21.15.43:3300/0 conn(0x7fdf680c7980 0x7fdf68156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.314+0000 7fdf6dcf1700 1 -- 172.21.15.40:0/1715764469 >> v1:172.21.15.40:6789/0 conn(0x7fdf680c83e0 legacy=0x7fdf68158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.314+0000 7fdf6dcf1700 1 -- 172.21.15.40:0/1715764469 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf68156580 con 0x7fdf6814fb40 2024-09-13T18:03:10.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.314+0000 7fdf6dcf1700 1 --2- 172.21.15.40:0/1715764469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf6814fb40 0x7fdf680c6e60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdf680c5bf0 tx=0x7fdf6400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9196a9a9ae612860 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:10.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.314+0000 7fdf6ccef700 1 -- 172.21.15.40:0/1715764469 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6400dbf0 con 0x7fdf6814fb40 2024-09-13T18:03:10.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.314+0000 7fdf6ccef700 1 -- 172.21.15.40:0/1715764469 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6400dd50 con 0x7fdf6814fb40 2024-09-13T18:03:10.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.314+0000 7fdf6ccef700 1 -- 172.21.15.40:0/1715764469 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64004250 con 0x7fdf6814fb40 2024-09-13T18:03:10.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.314+0000 7fdf6ff55700 1 -- 172.21.15.40:0/1715764469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf6814fb40 msgr2=0x7fdf680c6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.314+0000 7fdf6ff55700 1 --2- 172.21.15.40:0/1715764469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf6814fb40 0x7fdf680c6e60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdf680c5bf0 tx=0x7fdf6400bea0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.314+0000 7fdf6ff55700 1 -- 172.21.15.40:0/1715764469 shutdown_connections 2024-09-13T18:03:10.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.314+0000 7fdf6ff55700 1 --2- 172.21.15.40:0/1715764469 >> v2:172.21.15.43:3300/0 conn(0x7fdf680c7980 0x7fdf68156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.314+0000 7fdf6ff55700 1 --2- 172.21.15.40:0/1715764469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf6814fb40 0x7fdf680c6e60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.314+0000 7fdf6ff55700 1 -- 172.21.15.40:0/1715764469 >> 172.21.15.40:0/1715764469 conn(0x7fdf6806a570 msgr2=0x7fdf6806a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:10.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.314+0000 7fdf6ff55700 1 -- 172.21.15.40:0/1715764469 shutdown_connections 2024-09-13T18:03:10.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.314+0000 7fdf6ff55700 1 -- 172.21.15.40:0/1715764469 wait complete. 2024-09-13T18:03:10.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.314+0000 7fdf6ff55700 1 Processor -- start 2024-09-13T18:03:10.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.314+0000 7fdf6ff55700 1 -- start start 2024-09-13T18:03:10.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6ff55700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf680c7980 0x7fdf68150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6ff55700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680c83e0 0x7fdf68157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6ff55700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf6814fb40 0x7fdf681526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6ff55700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf681575d0 con 0x7fdf680c83e0 2024-09-13T18:03:10.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6ff55700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf68155480 con 0x7fdf6814fb40 2024-09-13T18:03:10.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6ff55700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf68157820 con 0x7fdf680c7980 2024-09-13T18:03:10.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6d4f0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680c83e0 0x7fdf68157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6d4f0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680c83e0 0x7fdf68157b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55948/0 (socket says 172.21.15.40:55948) 2024-09-13T18:03:10.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6d4f0700 1 -- 172.21.15.40:0/1880987736 learned_addr learned my addr 172.21.15.40:0/1880987736 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:10.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6e4f2700 1 --2- 172.21.15.40:0/1880987736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf6814fb40 0x7fdf681526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6dcf1700 1 --2- 172.21.15.40:0/1880987736 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf680c7980 0x7fdf68150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6d4f0700 1 -- 172.21.15.40:0/1880987736 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf680c7980 msgr2=0x7fdf68150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6d4f0700 1 --2- 172.21.15.40:0/1880987736 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf680c7980 0x7fdf68150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6d4f0700 1 -- 172.21.15.40:0/1880987736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf6814fb40 msgr2=0x7fdf681526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6d4f0700 1 --2- 172.21.15.40:0/1880987736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf6814fb40 0x7fdf681526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6d4f0700 1 -- 172.21.15.40:0/1880987736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6400b820 con 0x7fdf680c83e0 2024-09-13T18:03:10.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6dcf1700 1 --2- 172.21.15.40:0/1880987736 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf680c7980 0x7fdf68150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:03:10.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6d4f0700 1 --2- 172.21.15.40:0/1880987736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680c83e0 0x7fdf68157b00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800e840 tx=0x7fdf5800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:10.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf5e7fc700 1 -- 172.21.15.40:0/1880987736 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58013680 con 0x7fdf680c83e0 2024-09-13T18:03:10.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6ff55700 1 -- 172.21.15.40:0/1880987736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf681ead00 con 0x7fdf680c83e0 2024-09-13T18:03:10.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6ff55700 1 -- 172.21.15.40:0/1880987736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf681eb2c0 con 0x7fdf680c83e0 2024-09-13T18:03:10.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf5e7fc700 1 -- 172.21.15.40:0/1880987736 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf580137e0 con 0x7fdf680c83e0 2024-09-13T18:03:10.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf5e7fc700 1 -- 172.21.15.40:0/1880987736 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58013b50 con 0x7fdf680c83e0 2024-09-13T18:03:10.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf6ff55700 1 -- 172.21.15.40:0/1880987736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf50005310 con 0x7fdf680c83e0 2024-09-13T18:03:10.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.318+0000 7fdf5e7fc700 1 -- 172.21.15.40:0/1880987736 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdf58013cb0 con 0x7fdf680c83e0 2024-09-13T18:03:10.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.322+0000 7fdf5e7fc700 1 --2- 172.21.15.40:0/1880987736 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf4c066730 0x7fdf4c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.322+0000 7fdf5e7fc700 1 -- 172.21.15.40:0/1880987736 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdf58084740 con 0x7fdf680c83e0 2024-09-13T18:03:10.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.322+0000 7fdf6dcf1700 1 --2- 172.21.15.40:0/1880987736 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf4c066730 0x7fdf4c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.322+0000 7fdf6dcf1700 1 --2- 172.21.15.40:0/1880987736 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf4c066730 0x7fdf4c068bf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdf64019040 tx=0x7fdf64014000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:10.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.326+0000 7fdf5e7fc700 1 -- 172.21.15.40:0/1880987736 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdf580516f0 con 0x7fdf680c83e0 2024-09-13T18:03:10.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.594+0000 7fdf6ff55700 1 -- 172.21.15.40:0/1880987736 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdf500055c0 con 0x7fdf680c83e0 2024-09-13T18:03:10.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.594+0000 7fdf5e7fc700 1 -- 172.21.15.40:0/1880987736 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fdf58017620 con 0x7fdf680c83e0 2024-09-13T18:03:10.607 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":98,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:03:10.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.598+0000 7fdf6ff55700 1 -- 172.21.15.40:0/1880987736 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf4c066730 msgr2=0x7fdf4c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.598+0000 7fdf6ff55700 1 --2- 172.21.15.40:0/1880987736 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf4c066730 0x7fdf4c068bf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdf64019040 tx=0x7fdf64014000 comp rx=0 tx=0).stop 2024-09-13T18:03:10.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.598+0000 7fdf6ff55700 1 -- 172.21.15.40:0/1880987736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680c83e0 msgr2=0x7fdf68157b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.598+0000 7fdf6ff55700 1 --2- 172.21.15.40:0/1880987736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680c83e0 0x7fdf68157b00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800e840 tx=0x7fdf5800eb50 comp rx=0 tx=0).stop 2024-09-13T18:03:10.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.598+0000 7fdf6ff55700 1 -- 172.21.15.40:0/1880987736 shutdown_connections 2024-09-13T18:03:10.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.598+0000 7fdf6ff55700 1 --2- 172.21.15.40:0/1880987736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf6814fb40 0x7fdf681526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.598+0000 7fdf6ff55700 1 --2- 172.21.15.40:0/1880987736 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680c83e0 0x7fdf68157b00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.598+0000 7fdf6ff55700 1 --2- 172.21.15.40:0/1880987736 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf4c066730 0x7fdf4c068bf0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.598+0000 7fdf6ff55700 1 --2- 172.21.15.40:0/1880987736 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf680c7980 0x7fdf68150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.598+0000 7fdf6ff55700 1 -- 172.21.15.40:0/1880987736 >> 172.21.15.40:0/1880987736 conn(0x7fdf6806a570 msgr2=0x7fdf6814c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:10.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.598+0000 7fdf6ff55700 1 -- 172.21.15.40:0/1880987736 shutdown_connections 2024-09-13T18:03:10.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.598+0000 7fdf6ff55700 1 -- 172.21.15.40:0/1880987736 wait complete. 2024-09-13T18:03:10.627 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:03:10.627 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T18:03:10.627 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:03:10.627 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:03:10.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.730+0000 7f7ae2ad5700 1 Processor -- start 2024-09-13T18:03:10.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.730+0000 7f7ae2ad5700 1 -- start start 2024-09-13T18:03:10.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.730+0000 7f7ae2ad5700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7adc0cf580 0x7f7adc15a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.730+0000 7f7ae2ad5700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7adc0cc250 0x7f7adc0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.730+0000 7f7ae2ad5700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7adc15a7a0 con 0x7f7adc0cf1a0 2024-09-13T18:03:10.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.730+0000 7f7ae2ad5700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7adc0cbdc0 con 0x7f7adc0cc250 2024-09-13T18:03:10.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.730+0000 7f7ae2ad5700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc153f80 con 0x7f7adc0cf580 2024-09-13T18:03:10.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.730+0000 7f7ae22d4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7adc0cc250 0x7f7adc0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.730+0000 7f7ae12d2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7adc0cf580 0x7f7adc15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.730+0000 7f7ae22d4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7adc0cc250 0x7f7adc0c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:58746/0 (socket says 172.21.15.40:58746) 2024-09-13T18:03:10.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.730+0000 7f7ae22d4700 1 -- 172.21.15.40:0/2790659232 learned_addr learned my addr 172.21.15.40:0/2790659232 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:10.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.730+0000 7f7ae22d4700 1 -- 172.21.15.40:0/2790659232 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7adc0cf580 msgr2=0x7f7adc15a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:03:10.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.730+0000 7f7ae22d4700 1 --2- 172.21.15.40:0/2790659232 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7adc0cf580 0x7f7adc15a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.730+0000 7f7ae22d4700 1 -- 172.21.15.40:0/2790659232 >> v1:172.21.15.43:6789/0 conn(0x7f7adc0cf1a0 legacy=0x7f7adc0c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.730+0000 7f7ae22d4700 1 -- 172.21.15.40:0/2790659232 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc0caed0 con 0x7f7adc0cc250 2024-09-13T18:03:10.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.730+0000 7f7ae22d4700 1 --2- 172.21.15.40:0/2790659232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7adc0cc250 0x7f7adc0c8aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009e50 tx=0x7f7ad400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7b1b66069b69e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:10.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.730+0000 7f7ae0ad1700 1 -- 172.21.15.40:0/2790659232 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad400e070 con 0x7f7adc0cc250 2024-09-13T18:03:10.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.730+0000 7f7ae0ad1700 1 -- 172.21.15.40:0/2790659232 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad400b910 con 0x7f7adc0cc250 2024-09-13T18:03:10.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae0ad1700 1 -- 172.21.15.40:0/2790659232 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad400bcc0 con 0x7f7adc0cc250 2024-09-13T18:03:10.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae2ad5700 1 -- 172.21.15.40:0/2790659232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7adc0cc250 msgr2=0x7f7adc0c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae2ad5700 1 --2- 172.21.15.40:0/2790659232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7adc0cc250 0x7f7adc0c8aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009e50 tx=0x7f7ad400b330 comp rx=0 tx=0).stop 2024-09-13T18:03:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae2ad5700 1 -- 172.21.15.40:0/2790659232 shutdown_connections 2024-09-13T18:03:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae2ad5700 1 --2- 172.21.15.40:0/2790659232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7adc0cc250 0x7f7adc0c8aa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae2ad5700 1 --2- 172.21.15.40:0/2790659232 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7adc0cf580 0x7f7adc15a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae2ad5700 1 -- 172.21.15.40:0/2790659232 >> 172.21.15.40:0/2790659232 conn(0x7f7adc066090 msgr2=0x7f7adc066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae2ad5700 1 -- 172.21.15.40:0/2790659232 shutdown_connections 2024-09-13T18:03:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae2ad5700 1 -- 172.21.15.40:0/2790659232 wait complete. 2024-09-13T18:03:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae2ad5700 1 Processor -- start 2024-09-13T18:03:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae2ad5700 1 -- start start 2024-09-13T18:03:10.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae2ad5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7adc0cc250 0x7f7adc1f2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae2ad5700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7adc0cf1a0 0x7f7adc1f0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae2ad5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7adc0cf580 0x7f7adc1f05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae2ad5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc1570a0 con 0x7f7adc0cf580 2024-09-13T18:03:10.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae2ad5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc14fc40 con 0x7f7adc0cc250 2024-09-13T18:03:10.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae2ad5700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc0c84e0 con 0x7f7adc0cf1a0 2024-09-13T18:03:10.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae22d4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7adc0cf580 0x7f7adc1f05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae1ad3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7adc0cc250 0x7f7adc1f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae12d2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7adc0cf1a0 0x7f7adc1f0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae1ad3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7adc0cc250 0x7f7adc1f2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:58750/0 (socket says 172.21.15.40:58750) 2024-09-13T18:03:10.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.734+0000 7f7ae1ad3700 1 -- 172.21.15.40:0/1713314312 learned_addr learned my addr 172.21.15.40:0/1713314312 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:10.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.738+0000 7f7ae22d4700 1 -- 172.21.15.40:0/1713314312 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7adc0cf1a0 msgr2=0x7f7adc1f0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.738+0000 7f7ae22d4700 1 --2- 172.21.15.40:0/1713314312 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7adc0cf1a0 0x7f7adc1f0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.738+0000 7f7ae22d4700 1 -- 172.21.15.40:0/1713314312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7adc0cc250 msgr2=0x7f7adc1f2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.738+0000 7f7ae22d4700 1 --2- 172.21.15.40:0/1713314312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7adc0cc250 0x7f7adc1f2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.738+0000 7f7ae22d4700 1 -- 172.21.15.40:0/1713314312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad4009b30 con 0x7f7adc0cf580 2024-09-13T18:03:10.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.738+0000 7f7ae1ad3700 1 --2- 172.21.15.40:0/1713314312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7adc0cc250 0x7f7adc1f2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:03:10.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.738+0000 7f7ae22d4700 1 --2- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7adc0cf580 0x7f7adc1f05b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4015000 tx=0x7f7ad4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:10.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.738+0000 7f7ad27fc700 1 -- 172.21.15.40:0/1713314312 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad40048c0 con 0x7f7adc0cf580 2024-09-13T18:03:10.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.738+0000 7f7ad27fc700 1 -- 172.21.15.40:0/1713314312 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad40099b0 con 0x7f7adc0cf580 2024-09-13T18:03:10.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.738+0000 7f7ae2ad5700 1 -- 172.21.15.40:0/1713314312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7adc14f7b0 con 0x7f7adc0cf580 2024-09-13T18:03:10.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.738+0000 7f7ae2ad5700 1 -- 172.21.15.40:0/1713314312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7adc1f2dd0 con 0x7f7adc0cf580 2024-09-13T18:03:10.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.738+0000 7f7ad27fc700 1 -- 172.21.15.40:0/1713314312 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad401c680 con 0x7f7adc0cf580 2024-09-13T18:03:10.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.738+0000 7f7ae2ad5700 1 --2- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac4002a40 0x7f7ac4004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.738+0000 7f7ae2ad5700 1 -- 172.21.15.40:0/1713314312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac4007f40 con 0x7f7ac4002a40 2024-09-13T18:03:10.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.738+0000 7f7ae1ad3700 1 --2- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac4002a40 0x7f7ac4004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.738+0000 7f7ae1ad3700 1 -- 172.21.15.40:0/1713314312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7ac4008100 con 0x7f7ac4002a40 2024-09-13T18:03:10.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.738+0000 7f7ae1ad3700 1 --2- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac4002a40 0x7f7ac4004f00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800fae0 tx=0x7f7ad800fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:10.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.738+0000 7f7ad27fc700 1 -- 172.21.15.40:0/1713314312 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7ad401c7e0 con 0x7f7adc0cf580 2024-09-13T18:03:10.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.738+0000 7f7ad27fc700 1 --2- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7ac0066760 0x7f7ac0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.742+0000 7f7ae12d2700 1 --2- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7ac0066760 0x7f7ac0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.742+0000 7f7ad27fc700 1 -- 172.21.15.40:0/1713314312 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8017070 con 0x7f7ac4002a40 2024-09-13T18:03:10.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.742+0000 7f7ad27fc700 1 -- 172.21.15.40:0/1713314312 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7ad40859b0 con 0x7f7adc0cf580 2024-09-13T18:03:10.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.742+0000 7f7ad27fc700 1 -- 172.21.15.40:0/1713314312 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7ad80052e0 con 0x7f7ac4002a40 2024-09-13T18:03:10.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.742+0000 7f7ad27fc700 1 -- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac4002a40 msgr2=0x7f7ac4004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.742+0000 7f7ad27fc700 1 --2- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac4002a40 0x7f7ac4004f00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800fae0 tx=0x7f7ad800fea0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.742+0000 7f7ad27fc700 1 -- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac4002a40 msgr2=0x7f7ac4004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:03:10.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.742+0000 7f7ad27fc700 1 --2- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac4002a40 0x7f7ac4004f00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.742+0000 7f7ae12d2700 1 --2- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7ac0066760 0x7f7ac0068c20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7adc15a770 tx=0x7f7acc009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:10.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.750+0000 7f7ae2ad5700 1 --2- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac4006fb0 0x7f7ac4007440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.750+0000 7f7ae2ad5700 1 -- 172.21.15.40:0/1713314312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac400a4a0 con 0x7f7ac4006fb0 2024-09-13T18:03:10.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.750+0000 7f7ae1ad3700 1 --2- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac4006fb0 0x7f7ac4007440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.750+0000 7f7ae1ad3700 1 -- 172.21.15.40:0/1713314312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7ac4007980 con 0x7f7ac4006fb0 2024-09-13T18:03:10.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.750+0000 7f7ae1ad3700 1 --2- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac4006fb0 0x7f7ac4007440 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8016000 tx=0x7f7ad8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:10.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.750+0000 7f7ad27fc700 1 -- 172.21.15.40:0/1713314312 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8023070 con 0x7f7ac4006fb0 2024-09-13T18:03:10.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.750+0000 7f7ad27fc700 1 -- 172.21.15.40:0/1713314312 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4950 (secure 0 0 0) 0x7f7ad801ee90 con 0x7f7ac4006fb0 2024-09-13T18:03:10.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.750+0000 7f7ad27fc700 1 -- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac4006fb0 msgr2=0x7f7ac4007440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.750+0000 7f7ad27fc700 1 --2- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac4006fb0 0x7f7ac4007440 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8016000 tx=0x7f7ad8001120 comp rx=0 tx=0).stop 2024-09-13T18:03:10.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.750+0000 7f7ad27fc700 1 -- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac4006fb0 msgr2=0x7f7ac4007440 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:03:10.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.750+0000 7f7ad27fc700 1 --2- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7ac4006fb0 0x7f7ac4007440 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.764 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:03:10.764 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:03:10.764 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:03:10.764 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:03:10.764 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 98, 2024-09-13T18:03:10.764 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:03:10.764 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:03:10.764 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:03:10.764 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:03:10.764 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:10.764 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 14, 2024-09-13T18:03:10.764 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:03:10.764 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:03:10.764 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:03:10.765 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:10.765 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:10.765 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:10.765 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:10.765 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:10.765 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:10.765 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:10.765 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:10.765 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:10.765 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:10.765 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:10.765 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:03:10.765 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:03:10.765 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:10.765 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:10.765 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:10.766 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:10.766 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:10.766 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:10.766 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:10.766 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:10.766 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:10.766 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:10.766 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:10.766 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.766 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:03:10.766 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:03:10.766 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:03:10.766 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:03:10.766 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:03:10.767 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:03:10.767 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:03:10.767 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:03:10.767 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:03:10.767 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:03:10.767 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:03:10.767 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:03:10.767 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:03:10.767 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:03:10.767 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:03:10.768 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:03:10.768 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:03:10.768 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:10.768 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:10.768 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:10.768 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:10.768 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:10.768 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:10.768 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:10.768 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:10.768 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:10.768 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:10.768 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:10.768 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:03:10.769 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:03:10.769 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.769 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:03:10.769 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:03:10.769 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:10.769 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:10.769 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.769 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:10.769 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:03:10.769 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:10.769 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.769 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.770 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:10.770 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:03:10.770 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:10.770 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:10.770 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:10.770 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.770 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:03:10.770 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:03:10.770 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:10.770 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:10.770 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:10.770 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.770 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.770 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:03:10.771 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:03:10.771 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:10.771 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:10.771 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.771 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:10.771 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:03:10.771 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:10.771 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.771 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.771 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:10.771 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:03:10.771 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:10.771 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:10.771 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:10.772 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.772 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:03:10.772 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:03:10.772 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:10.772 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:10.772 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:10.772 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.772 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.772 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:03:10.772 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:03:10.772 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:10.772 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:10.772 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.772 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:10.772 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:03:10.773 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:10.773 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.773 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.773 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:10.773 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:03:10.773 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:10.773 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:10.773 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:10.773 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.773 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:03:10.773 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:03:10.773 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:10.773 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:10.773 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:10.774 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:10.774 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:10.774 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.774 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:03:10.774 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:03:10.774 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.774 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:10.774 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:10.774 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:03:10.774 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:10.774 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:10.774 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:03:10.774 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.775 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:10.775 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:10.775 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:03:10.775 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:10.775 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:10.775 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:03:10.775 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.775 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:10.775 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:10.775 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 8 2024-09-13T18:03:10.775 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:10.775 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:10.775 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:03:10.775 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.775 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:10.775 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:10.776 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:03:10.776 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:10.776 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:10.776 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:03:10.776 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.776 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:10.776 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:10.776 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:03:10.776 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:10.776 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:10.776 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.776 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:03:10.776 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:03:10.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.754+0000 7f7ae2ad5700 1 -- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7ac0066760 msgr2=0x7f7ac0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.754+0000 7f7ae2ad5700 1 --2- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7ac0066760 0x7f7ac0068c20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7adc15a770 tx=0x7f7acc009240 comp rx=0 tx=0).stop 2024-09-13T18:03:10.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.754+0000 7f7ae2ad5700 1 -- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7adc0cf580 msgr2=0x7f7adc1f05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.754+0000 7f7ae2ad5700 1 --2- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7adc0cf580 0x7f7adc1f05b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4015000 tx=0x7f7ad4004af0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.754+0000 7f7ae22d4700 1 -- 172.21.15.40:0/1713314312 reap_dead start 2024-09-13T18:03:10.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.754+0000 7f7ae2ad5700 1 -- 172.21.15.40:0/1713314312 shutdown_connections 2024-09-13T18:03:10.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.754+0000 7f7ae2ad5700 1 --2- 172.21.15.40:0/1713314312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7adc0cf580 0x7f7adc1f05b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.754+0000 7f7ae2ad5700 1 -- 172.21.15.40:0/1713314312 >> 172.21.15.40:0/1713314312 conn(0x7f7adc066090 msgr2=0x7f7adc15b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:10.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.754+0000 7f7ae2ad5700 1 -- 172.21.15.40:0/1713314312 shutdown_connections 2024-09-13T18:03:10.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.754+0000 7f7ae2ad5700 1 -- 172.21.15.40:0/1713314312 wait complete. 2024-09-13T18:03:10.778 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:03:10.779 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:03:10.779 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:03:10.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.882+0000 7f759bfff700 1 Processor -- start 2024-09-13T18:03:10.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.882+0000 7f759bfff700 1 -- start start 2024-09-13T18:03:10.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.882+0000 7f759bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f75940d68c0 0x7f75940dce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.882+0000 7f759bfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7594007bf0 0x7f75940dfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.882+0000 7f759bfff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75940dd3b0 con 0x7f7594008010 2024-09-13T18:03:10.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.882+0000 7f759bfff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7594036bc0 con 0x7f7594007bf0 2024-09-13T18:03:10.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.882+0000 7f759bfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7594006b40 con 0x7f75940d68c0 2024-09-13T18:03:10.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.882+0000 7f759b7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7594008010 0x7f75940dff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54868/0 (socket says 172.21.15.40:54868) 2024-09-13T18:03:10.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.882+0000 7f759b7fe700 1 -- 172.21.15.40:0/4278920487 learned_addr learned my addr 172.21.15.40:0/4278920487 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:03:10.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f759a7fc700 1 --2- 172.21.15.40:0/4278920487 >> v2:172.21.15.43:3300/0 conn(0x7f7594007bf0 0x7f75940dfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f759affd700 1 --2- 172.21.15.40:0/4278920487 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f75940d68c0 0x7f75940dce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f7599ffb700 1 -- 172.21.15.40:0/4278920487 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187623109 0 0) 0x7f75940dd3b0 con 0x7f7594008010 2024-09-13T18:03:10.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f7599ffb700 1 -- 172.21.15.40:0/4278920487 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75840035d0 con 0x7f7594008010 2024-09-13T18:03:10.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f759affd700 1 -- 172.21.15.40:0/4278920487 >> v2:172.21.15.43:3300/0 conn(0x7f7594007bf0 msgr2=0x7f75940dfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:03:10.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f759affd700 1 --2- 172.21.15.40:0/4278920487 >> v2:172.21.15.43:3300/0 conn(0x7f7594007bf0 0x7f75940dfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f759affd700 1 -- 172.21.15.40:0/4278920487 >> v1:172.21.15.40:6789/0 conn(0x7f7594008010 legacy=0x7f75940dff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f759affd700 1 -- 172.21.15.40:0/4278920487 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7594036220 con 0x7f75940d68c0 2024-09-13T18:03:10.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f759a7fc700 1 --2- 172.21.15.40:0/4278920487 >> v2:172.21.15.43:3300/0 conn(0x7f7594007bf0 0x7f75940dfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:03:10.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f759affd700 1 --2- 172.21.15.40:0/4278920487 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f75940d68c0 0x7f75940dce70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f758c009850 tx=0x7f758c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3db01acef2d7d38 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:10.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f7599ffb700 1 -- 172.21.15.40:0/4278920487 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c011830 con 0x7f75940d68c0 2024-09-13T18:03:10.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f7599ffb700 1 -- 172.21.15.40:0/4278920487 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758c011990 con 0x7f75940d68c0 2024-09-13T18:03:10.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f7599ffb700 1 -- 172.21.15.40:0/4278920487 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c011d00 con 0x7f75940d68c0 2024-09-13T18:03:10.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f759bfff700 1 -- 172.21.15.40:0/4278920487 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f75940d68c0 msgr2=0x7f75940dce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f759bfff700 1 --2- 172.21.15.40:0/4278920487 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f75940d68c0 0x7f75940dce70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f758c009850 tx=0x7f758c009b60 comp rx=0 tx=0).stop 2024-09-13T18:03:10.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f759bfff700 1 -- 172.21.15.40:0/4278920487 shutdown_connections 2024-09-13T18:03:10.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f759bfff700 1 --2- 172.21.15.40:0/4278920487 >> v2:172.21.15.43:3300/0 conn(0x7f7594007bf0 0x7f75940dfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f759bfff700 1 --2- 172.21.15.40:0/4278920487 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f75940d68c0 0x7f75940dce70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f759bfff700 1 -- 172.21.15.40:0/4278920487 >> 172.21.15.40:0/4278920487 conn(0x7f7594050e60 msgr2=0x7f7594051270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:10.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f759bfff700 1 -- 172.21.15.40:0/4278920487 shutdown_connections 2024-09-13T18:03:10.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f759bfff700 1 -- 172.21.15.40:0/4278920487 wait complete. 2024-09-13T18:03:10.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f759bfff700 1 Processor -- start 2024-09-13T18:03:10.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.886+0000 7f759bfff700 1 -- start start 2024-09-13T18:03:10.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7594007bf0 0x7f75940df5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594008010 0x7f75940d4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75940d68c0 0x7f75941754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759bfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75940d04b0 con 0x7f75940d68c0 2024-09-13T18:03:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759bfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75940cf5e0 con 0x7f7594008010 2024-09-13T18:03:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759bfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75940d2b60 con 0x7f7594007bf0 2024-09-13T18:03:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75940d68c0 0x7f75941754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75940d68c0 0x7f75941754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55980/0 (socket says 172.21.15.40:55980) 2024-09-13T18:03:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759b7fe700 1 -- 172.21.15.40:0/4208525760 learned_addr learned my addr 172.21.15.40:0/4208525760 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759affd700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7594007bf0 0x7f75940df5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759a7fc700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594008010 0x7f75940d4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759b7fe700 1 -- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7594007bf0 msgr2=0x7f75940df5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759b7fe700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7594007bf0 0x7f75940df5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759b7fe700 1 -- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594008010 msgr2=0x7f75940d4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759b7fe700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594008010 0x7f75940d4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759b7fe700 1 -- 172.21.15.40:0/4208525760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75900094b0 con 0x7f75940d68c0 2024-09-13T18:03:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759a7fc700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594008010 0x7f75940d4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:03:10.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759affd700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7594007bf0 0x7f75940df5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:03:10.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759b7fe700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75940d68c0 0x7f75941754a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f759c0641b0 tx=0x7f759c0747c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:10.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f75837fe700 1 -- 172.21.15.40:0/4208525760 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c074db0 con 0x7f75940d68c0 2024-09-13T18:03:10.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f75837fe700 1 -- 172.21.15.40:0/4208525760 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759c0720c0 con 0x7f75940d68c0 2024-09-13T18:03:10.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759bfff700 1 -- 172.21.15.40:0/4208525760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758c0094b0 con 0x7f75940d68c0 2024-09-13T18:03:10.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f75837fe700 1 -- 172.21.15.40:0/4208525760 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c072450 con 0x7f75940d68c0 2024-09-13T18:03:10.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759bfff700 1 -- 172.21.15.40:0/4208525760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75940cfb10 con 0x7f75940d68c0 2024-09-13T18:03:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759bfff700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f759404b300 0x7f75940cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.890+0000 7f759bfff700 1 -- 172.21.15.40:0/4208525760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75940095f0 con 0x7f759404b300 2024-09-13T18:03:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.894+0000 7f759affd700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f759404b300 0x7f75940cfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.894+0000 7f75837fe700 1 -- 172.21.15.40:0/4208525760 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f759c0725b0 con 0x7f75940d68c0 2024-09-13T18:03:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.894+0000 7f759affd700 1 -- 172.21.15.40:0/4208525760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75940d0240 con 0x7f759404b300 2024-09-13T18:03:10.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.894+0000 7f759affd700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f759404b300 0x7f75940cfd00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f758c009850 tx=0x7f758c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:10.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.894+0000 7f75837fe700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7588066730 0x7f7588068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.894+0000 7f759a7fc700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7588066730 0x7f7588068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.894+0000 7f75837fe700 1 -- 172.21.15.40:0/4208525760 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f759c0fdf00 con 0x7f75940d68c0 2024-09-13T18:03:10.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.894+0000 7f75837fe700 1 -- 172.21.15.40:0/4208525760 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c020070 con 0x7f759404b300 2024-09-13T18:03:10.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.894+0000 7f75837fe700 1 -- 172.21.15.40:0/4208525760 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f758c0116e0 con 0x7f759404b300 2024-09-13T18:03:10.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.894+0000 7f75837fe700 1 -- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f759404b300 msgr2=0x7f75940cfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.894+0000 7f75837fe700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f759404b300 0x7f75940cfd00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f758c009850 tx=0x7f758c004770 comp rx=0 tx=0).stop 2024-09-13T18:03:10.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.894+0000 7f75837fe700 1 -- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f759404b300 msgr2=0x7f75940cfd00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:03:10.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.894+0000 7f75837fe700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f759404b300 0x7f75940cfd00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.894+0000 7f759a7fc700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7588066730 0x7f7588068bf0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f759416fb60 tx=0x7f7590009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:10.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.902+0000 7f759bfff700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594171f30 0x7f75941759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:10.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.902+0000 7f759bfff700 1 -- 172.21.15.40:0/4208525760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75940367e0 con 0x7f7594171f30 2024-09-13T18:03:10.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.902+0000 7f759affd700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594171f30 0x7f75941759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:10.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.906+0000 7f759affd700 1 -- 172.21.15.40:0/4208525760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7594006720 con 0x7f7594171f30 2024-09-13T18:03:10.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.906+0000 7f759affd700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594171f30 0x7f75941759e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f758c009ba0 tx=0x7f758c011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:10.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.906+0000 7f75837fe700 1 -- 172.21.15.40:0/4208525760 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c02eae0 con 0x7f7594171f30 2024-09-13T18:03:10.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.906+0000 7f75837fe700 1 -- 172.21.15.40:0/4208525760 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4616 (secure 0 0 0) 0x7f758c02ec40 con 0x7f7594171f30 2024-09-13T18:03:10.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.906+0000 7f75837fe700 1 -- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594171f30 msgr2=0x7f75941759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.906+0000 7f75837fe700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594171f30 0x7f75941759e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f758c009ba0 tx=0x7f758c011000 comp rx=0 tx=0).stop 2024-09-13T18:03:10.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.906+0000 7f75837fe700 1 -- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594171f30 msgr2=0x7f75941759e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:03:10.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.906+0000 7f75837fe700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7594171f30 0x7f75941759e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.917 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:03:10.918 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:03:10.918 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:03:10.918 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:03:10.918 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 98, 2024-09-13T18:03:10.918 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:03:10.918 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:03:10.918 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:03:10.918 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:03:10.918 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:10.918 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 14, 2024-09-13T18:03:10.918 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:03:10.918 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:03:10.918 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:03:10.919 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:10.919 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:10.919 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:10.919 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:10.919 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:10.919 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:10.919 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:10.919 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:10.919 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:10.919 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:10.919 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:10.919 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:03:10.919 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:03:10.919 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:10.920 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:10.920 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:10.920 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:10.920 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:10.920 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:10.920 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:10.920 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:10.920 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:10.920 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:10.920 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:10.920 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.920 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:03:10.920 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:03:10.920 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:03:10.921 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:03:10.921 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:03:10.921 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:03:10.921 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:03:10.921 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:03:10.921 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:03:10.921 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:03:10.921 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:03:10.921 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:03:10.922 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:03:10.922 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:03:10.922 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:03:10.922 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:03:10.922 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:03:10.922 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:10.922 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:10.922 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:10.922 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:10.922 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:10.922 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:10.922 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:10.922 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:10.923 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:10.923 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:10.923 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:10.923 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:03:10.923 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.923 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:03:10.923 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.923 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:03:10.923 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:03:10.923 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:10.923 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:10.923 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.923 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:10.923 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:03:10.923 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:10.924 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.924 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.924 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:10.924 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:03:10.924 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:10.924 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:10.924 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:10.924 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.924 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:03:10.924 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:03:10.924 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:10.924 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:10.924 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:10.924 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.925 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.925 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:03:10.925 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:03:10.925 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:10.925 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:10.925 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.925 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:10.925 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:03:10.925 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:10.925 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.925 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.925 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:10.925 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:03:10.925 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:10.926 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:10.926 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:10.926 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.926 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:03:10.926 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:03:10.926 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:10.926 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:10.926 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:10.926 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.926 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.926 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:03:10.926 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:03:10.926 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:10.926 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:10.927 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.927 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:10.927 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:03:10.927 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:10.927 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.927 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.927 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:10.927 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:03:10.927 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:10.927 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:10.928 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:10.928 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.928 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:03:10.928 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:03:10.928 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:10.928 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:10.928 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:10.928 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:10.928 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:10.928 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.928 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:03:10.928 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:03:10.929 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.929 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:10.929 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:10.929 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:03:10.929 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:10.929 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:10.929 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:03:10.929 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.929 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:10.929 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:10.929 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:03:10.929 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:10.929 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:10.930 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:03:10.930 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:10.930 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:10.930 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:10.930 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:03:10.930 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:10.930 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:10.930 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:10.930 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:03:10.930 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:03:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.906+0000 7f759bfff700 1 -- 172.21.15.40:0/4208525760 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7588066730 msgr2=0x7f7588068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.906+0000 7f759bfff700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7588066730 0x7f7588068bf0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f759416fb60 tx=0x7f7590009240 comp rx=0 tx=0).stop 2024-09-13T18:03:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.906+0000 7f759bfff700 1 -- 172.21.15.40:0/4208525760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75940d68c0 msgr2=0x7f75941754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.906+0000 7f759b7fe700 1 -- 172.21.15.40:0/4208525760 reap_dead start 2024-09-13T18:03:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.906+0000 7f759bfff700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75940d68c0 0x7f75941754a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f759c0641b0 tx=0x7f759c0747c0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.906+0000 7f759bfff700 1 -- 172.21.15.40:0/4208525760 shutdown_connections 2024-09-13T18:03:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.906+0000 7f759bfff700 1 --2- 172.21.15.40:0/4208525760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f75940d68c0 0x7f75941754a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:10.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.906+0000 7f759bfff700 1 -- 172.21.15.40:0/4208525760 >> 172.21.15.40:0/4208525760 conn(0x7f7594050e60 msgr2=0x7f7594034670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:10.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.906+0000 7f759bfff700 1 -- 172.21.15.40:0/4208525760 shutdown_connections 2024-09-13T18:03:10.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:10.906+0000 7f759bfff700 1 -- 172.21.15.40:0/4208525760 wait complete. 2024-09-13T18:03:10.933 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:03:10.933 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:03:10.933 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:03:11.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00ca3ab700 1 Processor -- start 2024-09-13T18:03:11.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00ca3ab700 1 -- start start 2024-09-13T18:03:11.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00ca3ab700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc007480 0x7f00bc04efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:11.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00ca3ab700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00bc054df0 con 0x7f00bc0d8920 2024-09-13T18:03:11.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00ca3ab700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00bc006ff0 con 0x7f00bc0e1c00 2024-09-13T18:03:11.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00ca3ab700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc050a10 con 0x7f00bc007480 2024-09-13T18:03:11.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00c93a9700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f00bc0d8920 0x7f00bc049090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54874/0 (socket says 172.21.15.40:54874) 2024-09-13T18:03:11.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00c93a9700 1 -- 172.21.15.40:0/2604976099 learned_addr learned my addr 172.21.15.40:0/2604976099 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:03:11.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00c8ba8700 1 --2- 172.21.15.40:0/2604976099 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc007480 0x7f00bc04efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:11.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00bbfff700 1 -- 172.21.15.40:0/2604976099 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405839347 0 0) 0x7f00bc054df0 con 0x7f00bc0d8920 2024-09-13T18:03:11.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00bbfff700 1 -- 172.21.15.40:0/2604976099 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00b00035d0 con 0x7f00bc0d8920 2024-09-13T18:03:11.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00bbfff700 1 -- 172.21.15.40:0/2604976099 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2553193442 0 0) 0x7f00bc006ff0 con 0x7f00bc0e1c00 2024-09-13T18:03:11.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00bbfff700 1 -- 172.21.15.40:0/2604976099 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00bc054df0 con 0x7f00bc0e1c00 2024-09-13T18:03:11.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00bbfff700 1 -- 172.21.15.40:0/2604976099 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3560839974 0 0) 0x7f00b00035d0 con 0x7f00bc0d8920 2024-09-13T18:03:11.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00bbfff700 1 -- 172.21.15.40:0/2604976099 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc007480 msgr2=0x7f00bc04efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:03:11.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00bbfff700 1 --2- 172.21.15.40:0/2604976099 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc007480 0x7f00bc04efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:11.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00bbfff700 1 -- 172.21.15.40:0/2604976099 >> v1:172.21.15.43:6789/0 conn(0x7f00bc0e1c00 legacy=0x7f00bc0e2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:11.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00bbfff700 1 -- 172.21.15.40:0/2604976099 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc0501b0 con 0x7f00bc0d8920 2024-09-13T18:03:11.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00bbfff700 1 -- 172.21.15.40:0/2604976099 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f00c0001670 con 0x7f00bc0d8920 2024-09-13T18:03:11.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00bbfff700 1 -- 172.21.15.40:0/2604976099 >> v1:172.21.15.40:6789/0 conn(0x7f00bc0d8920 legacy=0x7f00bc049090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:11.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00bbfff700 1 --2- 172.21.15.40:0/2604976099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00b0003be0 0x7f00b0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:11.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00bbfff700 1 --2- 172.21.15.40:0/2604976099 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc007480 0x7f00b0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00bbfff700 1 --2- 172.21.15.40:0/2604976099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b0004db0 0x7f00b0005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00bbfff700 1 -- 172.21.15.40:0/2604976099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc050a10 con 0x7f00b0003be0 2024-09-13T18:03:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00bbfff700 1 -- 172.21.15.40:0/2604976099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b000ba60 con 0x7f00b0004db0 2024-09-13T18:03:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00bbfff700 1 -- 172.21.15.40:0/2604976099 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b000bc20 con 0x7f00bc007480 2024-09-13T18:03:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00c93a9700 1 --2- 172.21.15.40:0/2604976099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00b0003be0 0x7f00b0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00c8ba8700 1 --2- 172.21.15.40:0/2604976099 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc007480 0x7f00b0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00c9baa700 1 --2- 172.21.15.40:0/2604976099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b0004db0 0x7f00b0005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00c93a9700 1 -- 172.21.15.40:0/2604976099 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc007480 msgr2=0x7f00b0004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:03:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00c93a9700 1 --2- 172.21.15.40:0/2604976099 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc007480 0x7f00b0004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00c93a9700 1 -- 172.21.15.40:0/2604976099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b0004db0 msgr2=0x7f00b0005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:03:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00c93a9700 1 --2- 172.21.15.40:0/2604976099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b0004db0 0x7f00b0005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.046+0000 7f00c93a9700 1 -- 172.21.15.40:0/2604976099 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b00035d0 con 0x7f00b0003be0 2024-09-13T18:03:11.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.050+0000 7f00c93a9700 1 --2- 172.21.15.40:0/2604976099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00b0003be0 0x7f00b0003fe0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f00c000c2a0 tx=0x7f00b00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d102b4dccd726aa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:11.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.050+0000 7f00bbfff700 1 -- 172.21.15.40:0/2604976099 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c0007c80 con 0x7f00b0003be0 2024-09-13T18:03:11.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.050+0000 7f00bbfff700 1 -- 172.21.15.40:0/2604976099 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c000ce60 con 0x7f00b0003be0 2024-09-13T18:03:11.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.050+0000 7f00bbfff700 1 -- 172.21.15.40:0/2604976099 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c0005bc0 con 0x7f00b0003be0 2024-09-13T18:03:11.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.050+0000 7f00ca3ab700 1 -- 172.21.15.40:0/2604976099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00b0003be0 msgr2=0x7f00b0003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:11.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.050+0000 7f00ca3ab700 1 --2- 172.21.15.40:0/2604976099 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00b0003be0 0x7f00b0003fe0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f00c000c2a0 tx=0x7f00b00058a0 comp rx=0 tx=0).stop 2024-09-13T18:03:11.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.050+0000 7f00c9baa700 1 -- 172.21.15.40:0/2604976099 reap_dead start 2024-09-13T18:03:11.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.050+0000 7f00ca3ab700 1 -- 172.21.15.40:0/2604976099 shutdown_connections 2024-09-13T18:03:11.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.050+0000 7f00ca3ab700 1 -- 172.21.15.40:0/2604976099 >> 172.21.15.40:0/2604976099 conn(0x7f00bc054110 msgr2=0x7f00bc054520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:11.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.050+0000 7f00ca3ab700 1 -- 172.21.15.40:0/2604976099 shutdown_connections 2024-09-13T18:03:11.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.050+0000 7f00ca3ab700 1 -- 172.21.15.40:0/2604976099 wait complete. 2024-09-13T18:03:11.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.050+0000 7f00ca3ab700 1 Processor -- start 2024-09-13T18:03:11.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.050+0000 7f00ca3ab700 1 -- start start 2024-09-13T18:03:11.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00ca3ab700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b0003be0 0x7f00bc039560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00ca3ab700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc0cf560 0x7f00bc0ce600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00ca3ab700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00bc0ceb40 0x7f00bc04d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00ca3ab700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc0ce1e0 con 0x7f00bc0ceb40 2024-09-13T18:03:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00ca3ab700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc174e30 con 0x7f00b0003be0 2024-09-13T18:03:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00ca3ab700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc175370 con 0x7f00bc0cf560 2024-09-13T18:03:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00c9baa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00bc0ceb40 0x7f00bc04d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00c9baa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00bc0ceb40 0x7f00bc04d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55990/0 (socket says 172.21.15.40:55990) 2024-09-13T18:03:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00c9baa700 1 -- 172.21.15.40:0/705319120 learned_addr learned my addr 172.21.15.40:0/705319120 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00c93a9700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b0003be0 0x7f00bc039560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00c8ba8700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc0cf560 0x7f00bc0ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00c9baa700 1 -- 172.21.15.40:0/705319120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc0cf560 msgr2=0x7f00bc0ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00c9baa700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc0cf560 0x7f00bc0ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00c9baa700 1 -- 172.21.15.40:0/705319120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b0003be0 msgr2=0x7f00bc039560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:11.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00c9baa700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00b0003be0 0x7f00bc039560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:11.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00c9baa700 1 -- 172.21.15.40:0/705319120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc0cdb60 con 0x7f00bc0ceb40 2024-09-13T18:03:11.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00c9baa700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00bc0ceb40 0x7f00bc04d610 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f00c4000b60 tx=0x7f00c40747c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:11.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00b9ffb700 1 -- 172.21.15.40:0/705319120 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c4090070 con 0x7f00bc0ceb40 2024-09-13T18:03:11.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00b9ffb700 1 -- 172.21.15.40:0/705319120 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c4067860 con 0x7f00bc0ceb40 2024-09-13T18:03:11.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00b9ffb700 1 -- 172.21.15.40:0/705319120 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c4072350 con 0x7f00bc0ceb40 2024-09-13T18:03:11.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00ca3ab700 1 -- 172.21.15.40:0/705319120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00bc175090 con 0x7f00bc0ceb40 2024-09-13T18:03:11.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00ca3ab700 1 -- 172.21.15.40:0/705319120 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00bc0d1850 con 0x7f00bc0ceb40 2024-09-13T18:03:11.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00ca3ab700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc0cf940 0x7f00bc04db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:11.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.054+0000 7f00ca3ab700 1 -- 172.21.15.40:0/705319120 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc0084b0 con 0x7f00bc0cf940 2024-09-13T18:03:11.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.058+0000 7f00c93a9700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc0cf940 0x7f00bc04db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:11.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.058+0000 7f00c93a9700 1 -- 172.21.15.40:0/705319120 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f00bc0dc250 con 0x7f00bc0cf940 2024-09-13T18:03:11.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.058+0000 7f00b9ffb700 1 -- 172.21.15.40:0/705319120 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f00c40724b0 con 0x7f00bc0ceb40 2024-09-13T18:03:11.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.058+0000 7f00c93a9700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc0cf940 0x7f00bc04db50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f00c000c2a0 tx=0x7f00bc036ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:11.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.058+0000 7f00b9ffb700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f00a8066760 0x7f00a8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:11.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.058+0000 7f00c8ba8700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f00a8066760 0x7f00a8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:11.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.058+0000 7f00b9ffb700 1 -- 172.21.15.40:0/705319120 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f00c40fe370 con 0x7f00bc0ceb40 2024-09-13T18:03:11.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.058+0000 7f00b9ffb700 1 -- 172.21.15.40:0/705319120 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c0001630 con 0x7f00bc0cf940 2024-09-13T18:03:11.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.058+0000 7f00b9ffb700 1 -- 172.21.15.40:0/705319120 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f00c00070d0 con 0x7f00bc0cf940 2024-09-13T18:03:11.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.058+0000 7f00b9ffb700 1 -- 172.21.15.40:0/705319120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc0cf940 msgr2=0x7f00bc04db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:11.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.058+0000 7f00b9ffb700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc0cf940 0x7f00bc04db50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f00c000c2a0 tx=0x7f00bc036ba0 comp rx=0 tx=0).stop 2024-09-13T18:03:11.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.058+0000 7f00b9ffb700 1 -- 172.21.15.40:0/705319120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc0cf940 msgr2=0x7f00bc04db50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:03:11.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.058+0000 7f00b9ffb700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc0cf940 0x7f00bc04db50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:11.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.058+0000 7f00c8ba8700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f00a8066760 0x7f00a8068c20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0d1be0 tx=0x7f00b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.070+0000 7f00ca3ab700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc177970 0x7f00bc177d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.070+0000 7f00ca3ab700 1 -- 172.21.15.40:0/705319120 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc0cf130 con 0x7f00bc177970 2024-09-13T18:03:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.070+0000 7f00c93a9700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc177970 0x7f00bc177d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:11.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.070+0000 7f00c93a9700 1 -- 172.21.15.40:0/705319120 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f00bc050450 con 0x7f00bc177970 2024-09-13T18:03:11.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.074+0000 7f00c93a9700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc177970 0x7f00bc177d80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f00c0001930 tx=0x7f00c0007000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:11.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.074+0000 7f00b9ffb700 1 -- 172.21.15.40:0/705319120 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c002c030 con 0x7f00bc177970 2024-09-13T18:03:11.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.074+0000 7f00b9ffb700 1 -- 172.21.15.40:0/705319120 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4450 (secure 0 0 0) 0x7f00c002d040 con 0x7f00bc177970 2024-09-13T18:03:11.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.074+0000 7f00b9ffb700 1 -- 172.21.15.40:0/705319120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc177970 msgr2=0x7f00bc177d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:11.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.074+0000 7f00b9ffb700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc177970 0x7f00bc177d80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f00c0001930 tx=0x7f00c0007000 comp rx=0 tx=0).stop 2024-09-13T18:03:11.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.074+0000 7f00b9ffb700 1 -- 172.21.15.40:0/705319120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc177970 msgr2=0x7f00bc177d80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:03:11.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.074+0000 7f00b9ffb700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00bc177970 0x7f00bc177d80 secure :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f00c0001930 tx=0x7f00c0007000 comp rx=0 tx=0).stop 2024-09-13T18:03:11.085 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:03:11.085 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:03:11.085 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:03:11.085 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:03:11.085 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 98, 2024-09-13T18:03:11.085 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:03:11.085 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:03:11.085 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:03:11.085 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:03:11.085 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:11.085 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 14, 2024-09-13T18:03:11.085 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:03:11.086 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:03:11.086 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:03:11.086 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:11.086 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:11.086 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:11.086 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:11.086 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:11.086 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:11.086 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:11.086 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:11.086 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:11.086 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:11.086 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:11.086 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:03:11.087 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:03:11.087 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:11.087 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:11.087 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:11.087 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:11.087 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:11.087 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:11.087 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:11.087 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:11.087 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:11.087 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:11.087 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:11.087 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:11.087 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:03:11.088 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:03:11.088 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:03:11.088 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:03:11.088 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:03:11.088 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:03:11.088 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:03:11.088 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:03:11.088 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:03:11.088 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:03:11.088 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:03:11.088 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:03:11.088 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:03:11.088 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:03:11.089 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:03:11.089 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:03:11.089 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:03:11.089 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:11.089 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:11.089 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:11.089 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:11.089 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:11.089 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:11.089 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:11.089 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:11.089 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:11.089 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:11.089 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:11.089 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:03:11.090 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:11.090 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:03:11.090 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:11.090 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:03:11.090 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:03:11.090 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:11.090 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:11.090 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:11.090 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:11.090 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:03:11.090 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:11.090 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:11.090 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:11.090 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:11.091 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:03:11.091 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:11.091 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:11.091 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:11.091 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:11.091 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:03:11.091 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:03:11.091 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:11.092 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:11.092 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:11.092 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:11.092 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:11.092 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:03:11.092 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:03:11.092 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:11.092 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:11.092 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:11.092 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:11.092 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:03:11.092 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:11.092 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:11.092 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:11.093 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:11.093 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:03:11.093 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:11.093 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:11.093 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:11.093 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:11.093 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:03:11.093 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:03:11.093 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:11.093 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:11.093 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:11.093 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:11.093 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:11.094 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:03:11.094 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:03:11.094 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:11.094 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:11.094 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:11.094 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:11.094 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:03:11.094 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:11.094 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:11.094 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:11.094 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:11.094 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:03:11.094 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:11.094 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:11.095 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:11.095 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:11.095 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:03:11.095 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:03:11.095 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:11.095 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:11.095 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:11.095 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:11.095 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:11.095 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:11.095 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:03:11.095 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:03:11.095 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:11.095 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:11.096 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:11.096 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:03:11.096 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:11.096 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:11.096 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:03:11.096 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:11.096 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:11.096 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:11.096 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:03:11.096 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:11.096 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:11.096 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:11.096 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:03:11.096 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:03:11.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.074+0000 7f00ca3ab700 1 -- 172.21.15.40:0/705319120 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f00a8066760 msgr2=0x7f00a8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:11.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.074+0000 7f00ca3ab700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f00a8066760 0x7f00a8068c20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0d1be0 tx=0x7f00b4008040 comp rx=0 tx=0).stop 2024-09-13T18:03:11.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.074+0000 7f00ca3ab700 1 -- 172.21.15.40:0/705319120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00bc0ceb40 msgr2=0x7f00bc04d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:11.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.074+0000 7f00c9baa700 1 -- 172.21.15.40:0/705319120 reap_dead start 2024-09-13T18:03:11.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.074+0000 7f00ca3ab700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00bc0ceb40 0x7f00bc04d610 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f00c4000b60 tx=0x7f00c40747c0 comp rx=0 tx=0).stop 2024-09-13T18:03:11.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.074+0000 7f00ca3ab700 1 -- 172.21.15.40:0/705319120 shutdown_connections 2024-09-13T18:03:11.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.074+0000 7f00ca3ab700 1 --2- 172.21.15.40:0/705319120 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00bc0ceb40 0x7f00bc04d610 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:11.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.074+0000 7f00ca3ab700 1 -- 172.21.15.40:0/705319120 >> 172.21.15.40:0/705319120 conn(0x7f00bc054110 msgr2=0x7f00bc0d3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:11.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.074+0000 7f00ca3ab700 1 -- 172.21.15.40:0/705319120 shutdown_connections 2024-09-13T18:03:11.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.074+0000 7f00ca3ab700 1 -- 172.21.15.40:0/705319120 wait complete. 2024-09-13T18:03:11.099 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:03:11.100 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-13T18:03:11.100 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T18:03:11.100 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-13T18:03:11.100 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-13T18:03:11.100 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T18:03:11.100 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:03:11.148 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T18:03:11.149 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T18:03:11.149 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:03:11.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.254+0000 7f7dfb59e700 1 Processor -- start 2024-09-13T18:03:11.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.254+0000 7f7dfb59e700 1 -- start start 2024-09-13T18:03:11.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.254+0000 7f7dfb59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7dfc1512c0 0x7f7dfc0c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:11.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.254+0000 7f7dfb59e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7dfc15b3e0 0x7f7dfc0c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:11.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.254+0000 7f7dfb59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dfc15a840 con 0x7f7dfc0b2d00 2024-09-13T18:03:11.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.254+0000 7f7dfb59e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dfc1516a0 con 0x7f7dfc15b3e0 2024-09-13T18:03:11.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.254+0000 7f7dfb59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc1580a0 con 0x7f7dfc1512c0 2024-09-13T18:03:11.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.254+0000 7f7df9d9b700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7dfc0b2d00 0x7f7dfc15acd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54884/0 (socket says 172.21.15.40:54884) 2024-09-13T18:03:11.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.254+0000 7f7df9d9b700 1 -- 172.21.15.40:0/2312062761 learned_addr learned my addr 172.21.15.40:0/2312062761 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:03:11.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.254+0000 7f7dfa59c700 1 -- 172.21.15.40:0/2312062761 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7dfc1512c0 msgr2=0x7f7dfc0c78f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:03:11.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.254+0000 7f7dfa59c700 1 --2- 172.21.15.40:0/2312062761 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7dfc1512c0 0x7f7dfc0c78f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:11.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.254+0000 7f7dfad9d700 1 --2- 172.21.15.40:0/2312062761 >> v2:172.21.15.43:3300/0 conn(0x7f7dfc15b3e0 0x7f7dfc0c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:11.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.254+0000 7f7df959a700 1 -- 172.21.15.40:0/2312062761 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 942330924 0 0) 0x7f7dfc15a840 con 0x7f7dfc0b2d00 2024-09-13T18:03:11.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.254+0000 7f7df959a700 1 -- 172.21.15.40:0/2312062761 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7de40035d0 con 0x7f7dfc0b2d00 2024-09-13T18:03:11.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.254+0000 7f7dfad9d700 1 -- 172.21.15.40:0/2312062761 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7dfc1512c0 msgr2=0x7f7dfc0c78f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:03:11.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.254+0000 7f7dfad9d700 1 --2- 172.21.15.40:0/2312062761 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7dfc1512c0 0x7f7dfc0c78f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:11.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.254+0000 7f7dfad9d700 1 -- 172.21.15.40:0/2312062761 >> v1:172.21.15.40:6789/0 conn(0x7f7dfc0b2d00 legacy=0x7f7dfc15acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:11.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.254+0000 7f7dfad9d700 1 -- 172.21.15.40:0/2312062761 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dfc157520 con 0x7f7dfc15b3e0 2024-09-13T18:03:11.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.258+0000 7f7dfad9d700 1 --2- 172.21.15.40:0/2312062761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dfc15b3e0 0x7f7dfc0c7e30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009d90 tx=0x7f7df400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f55c362a9c633599 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:11.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.258+0000 7f7df959a700 1 -- 172.21.15.40:0/2312062761 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df400e070 con 0x7f7dfc15b3e0 2024-09-13T18:03:11.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.258+0000 7f7df959a700 1 -- 172.21.15.40:0/2312062761 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df400bb20 con 0x7f7dfc15b3e0 2024-09-13T18:03:11.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.258+0000 7f7df959a700 1 -- 172.21.15.40:0/2312062761 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df4012430 con 0x7f7dfc15b3e0 2024-09-13T18:03:11.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.258+0000 7f7dfb59e700 1 -- 172.21.15.40:0/2312062761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dfc15b3e0 msgr2=0x7f7dfc0c7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:11.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.258+0000 7f7dfb59e700 1 --2- 172.21.15.40:0/2312062761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dfc15b3e0 0x7f7dfc0c7e30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009d90 tx=0x7f7df400b330 comp rx=0 tx=0).stop 2024-09-13T18:03:11.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.258+0000 7f7dfb59e700 1 -- 172.21.15.40:0/2312062761 shutdown_connections 2024-09-13T18:03:11.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.258+0000 7f7dfb59e700 1 --2- 172.21.15.40:0/2312062761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dfc15b3e0 0x7f7dfc0c7e30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:11.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.258+0000 7f7dfb59e700 1 --2- 172.21.15.40:0/2312062761 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7dfc1512c0 0x7f7dfc0c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:11.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.258+0000 7f7dfb59e700 1 -- 172.21.15.40:0/2312062761 >> 172.21.15.40:0/2312062761 conn(0x7f7dfc0ccef0 msgr2=0x7f7dfc0cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:11.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.258+0000 7f7dfb59e700 1 -- 172.21.15.40:0/2312062761 shutdown_connections 2024-09-13T18:03:11.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.258+0000 7f7dfb59e700 1 -- 172.21.15.40:0/2312062761 wait complete. 2024-09-13T18:03:11.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.258+0000 7f7dfb59e700 1 Processor -- start 2024-09-13T18:03:11.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.258+0000 7f7dfb59e700 1 -- start start 2024-09-13T18:03:11.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.258+0000 7f7dfb59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dfc0b2d00 0x7f7dfc06c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:11.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.258+0000 7f7dfb59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7dfc1512c0 0x7f7dfc14cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:11.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.258+0000 7f7dfb59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dfc15b3e0 0x7f7dfc14d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:11.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.258+0000 7f7dfb59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc154eb0 con 0x7f7dfc15b3e0 2024-09-13T18:03:11.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.258+0000 7f7dfb59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc06b970 con 0x7f7dfc0b2d00 2024-09-13T18:03:11.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.258+0000 7f7dfb59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc06d2a0 con 0x7f7dfc1512c0 2024-09-13T18:03:11.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.262+0000 7f7df9d9b700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7dfc1512c0 msgr2=0x7f7dfc14cda0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:03:11.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.262+0000 7f7df9d9b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7dfc1512c0 0x7f7dfc14cda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:11.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.262+0000 7f7dfad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dfc15b3e0 0x7f7dfc14d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:11.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.262+0000 7f7dfa59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dfc0b2d00 0x7f7dfc06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:11.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.262+0000 7f7dfad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dfc15b3e0 0x7f7dfc14d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56006/0 (socket says 172.21.15.40:56006) 2024-09-13T18:03:11.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.262+0000 7f7dfa59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dfc0b2d00 0x7f7dfc06c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:58816/0 (socket says 172.21.15.40:58816) 2024-09-13T18:03:11.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.262+0000 7f7dfad9d700 1 -- 172.21.15.40:0/1894227890 learned_addr learned my addr 172.21.15.40:0/1894227890 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:11.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.262+0000 7f7dfad9d700 1 -- 172.21.15.40:0/1894227890 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7dfc1512c0 msgr2=0x7f7dfc14cda0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:03:11.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.262+0000 7f7dfad9d700 1 --2- 172.21.15.40:0/1894227890 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7dfc1512c0 0x7f7dfc14cda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:11.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.262+0000 7f7dfad9d700 1 -- 172.21.15.40:0/1894227890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dfc0b2d00 msgr2=0x7f7dfc06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:11.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.262+0000 7f7dfad9d700 1 --2- 172.21.15.40:0/1894227890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dfc0b2d00 0x7f7dfc06c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:11.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.262+0000 7f7dfad9d700 1 -- 172.21.15.40:0/1894227890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df4009a70 con 0x7f7dfc15b3e0 2024-09-13T18:03:11.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.262+0000 7f7dfad9d700 1 --2- 172.21.15.40:0/1894227890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dfc15b3e0 0x7f7dfc14d2e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7df400b680 tx=0x7f7df4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:11.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.262+0000 7f7deaffd700 1 -- 172.21.15.40:0/1894227890 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df4012430 con 0x7f7dfc15b3e0 2024-09-13T18:03:11.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.262+0000 7f7deaffd700 1 -- 172.21.15.40:0/1894227890 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df4020070 con 0x7f7dfc15b3e0 2024-09-13T18:03:11.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.262+0000 7f7deaffd700 1 -- 172.21.15.40:0/1894227890 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df4012ea0 con 0x7f7dfc15b3e0 2024-09-13T18:03:11.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.262+0000 7f7dfb59e700 1 -- 172.21.15.40:0/1894227890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dfc06d000 con 0x7f7dfc15b3e0 2024-09-13T18:03:11.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.262+0000 7f7dfb59e700 1 -- 172.21.15.40:0/1894227890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dfc154bd0 con 0x7f7dfc15b3e0 2024-09-13T18:03:11.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.262+0000 7f7dfb59e700 1 -- 172.21.15.40:0/1894227890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dfc06b4a0 con 0x7f7dfc15b3e0 2024-09-13T18:03:11.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.262+0000 7f7deaffd700 1 -- 172.21.15.40:0/1894227890 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7df4004280 con 0x7f7dfc15b3e0 2024-09-13T18:03:11.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.266+0000 7f7deaffd700 1 --2- 172.21.15.40:0/1894227890 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7ddc0666e0 0x7f7ddc068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:11.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.266+0000 7f7dfa59c700 1 --2- 172.21.15.40:0/1894227890 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7ddc0666e0 0x7f7ddc068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:11.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.266+0000 7f7deaffd700 1 -- 172.21.15.40:0/1894227890 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7df4022070 con 0x7f7dfc15b3e0 2024-09-13T18:03:11.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.266+0000 7f7dfa59c700 1 --2- 172.21.15.40:0/1894227890 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7ddc0666e0 0x7f7ddc068ba0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7dec009470 tx=0x7f7dec009400 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:11.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.270+0000 7f7deaffd700 1 -- 172.21.15.40:0/1894227890 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7dfc06b4a0 con 0x7f7dfc15b3e0 2024-09-13T18:03:11.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.538+0000 7f7dfb59e700 1 -- 172.21.15.40:0/1894227890 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f7dfc14cac0 con 0x7f7dfc15b3e0 2024-09-13T18:03:11.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.542+0000 7f7deaffd700 1 -- 172.21.15.40:0/1894227890 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f7dfc14cac0 con 0x7f7dfc15b3e0 2024-09-13T18:03:11.552 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":98,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:03:11.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.542+0000 7f7dfb59e700 1 -- 172.21.15.40:0/1894227890 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7ddc0666e0 msgr2=0x7f7ddc068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:11.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.542+0000 7f7dfb59e700 1 --2- 172.21.15.40:0/1894227890 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7ddc0666e0 0x7f7ddc068ba0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7dec009470 tx=0x7f7dec009400 comp rx=0 tx=0).stop 2024-09-13T18:03:11.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.542+0000 7f7dfb59e700 1 -- 172.21.15.40:0/1894227890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dfc15b3e0 msgr2=0x7f7dfc14d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:11.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.542+0000 7f7dfb59e700 1 --2- 172.21.15.40:0/1894227890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dfc15b3e0 0x7f7dfc14d2e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7df400b680 tx=0x7f7df4004af0 comp rx=0 tx=0).stop 2024-09-13T18:03:11.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.542+0000 7f7dfb59e700 1 -- 172.21.15.40:0/1894227890 shutdown_connections 2024-09-13T18:03:11.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.542+0000 7f7dfb59e700 1 --2- 172.21.15.40:0/1894227890 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7dfc15b3e0 0x7f7dfc14d2e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:11.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.542+0000 7f7dfb59e700 1 --2- 172.21.15.40:0/1894227890 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7ddc0666e0 0x7f7ddc068ba0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:11.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.542+0000 7f7dfb59e700 1 --2- 172.21.15.40:0/1894227890 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7dfc1512c0 0x7f7dfc14cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:11.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.542+0000 7f7dfb59e700 1 --2- 172.21.15.40:0/1894227890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dfc0b2d00 0x7f7dfc06c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:11.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.542+0000 7f7dfb59e700 1 -- 172.21.15.40:0/1894227890 >> 172.21.15.40:0/1894227890 conn(0x7f7dfc0ccef0 msgr2=0x7f7dfc06e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:11.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.542+0000 7f7dfb59e700 1 -- 172.21.15.40:0/1894227890 shutdown_connections 2024-09-13T18:03:11.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:11.542+0000 7f7dfb59e700 1 -- 172.21.15.40:0/1894227890 wait complete. 2024-09-13T18:03:14.568 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:03:14.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.670+0000 7fb2ac9c9700 1 Processor -- start 2024-09-13T18:03:14.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.670+0000 7fb2ac9c9700 1 -- start start 2024-09-13T18:03:14.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.670+0000 7fb2ac9c9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb2a00d88f0 0x7fb2a004e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:14.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.670+0000 7fb2ac9c9700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb2a0046fa0 0x7fb2a004cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:14.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.670+0000 7fb2ac9c9700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a00474f0 con 0x7fb2a004d240 2024-09-13T18:03:14.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.670+0000 7fb2ac9c9700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a00de6b0 con 0x7fb2a0046fa0 2024-09-13T18:03:14.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.670+0000 7fb2ac9c9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a00de970 con 0x7fb2a00d88f0 2024-09-13T18:03:14.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.670+0000 7fb2a77fe700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb2a00d88f0 msgr2=0x7fb2a004e070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:03:14.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.670+0000 7fb2a77fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb2a00d88f0 0x7fb2a004e070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:14.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2a6ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb2a0046fa0 0x7fb2a004cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:14.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2a7fff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb2a004d240 0x7fb2a0047980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:43840/0 (socket says 172.21.15.40:43840) 2024-09-13T18:03:14.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2a7fff700 1 -- 172.21.15.40:0/2043109244 learned_addr learned my addr 172.21.15.40:0/2043109244 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:03:14.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2a6ffd700 1 -- 172.21.15.40:0/2043109244 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb2a00d88f0 msgr2=0x7fb2a004e070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:03:14.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2a6ffd700 1 --2- 172.21.15.40:0/2043109244 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb2a00d88f0 0x7fb2a004e070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:14.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2a6ffd700 1 -- 172.21.15.40:0/2043109244 >> v1:172.21.15.43:6789/0 conn(0x7fb2a004d240 legacy=0x7fb2a0047980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:14.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2a6ffd700 1 -- 172.21.15.40:0/2043109244 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a00dec50 con 0x7fb2a0046fa0 2024-09-13T18:03:14.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2a6ffd700 1 --2- 172.21.15.40:0/2043109244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2a0046fa0 0x7fb2a004cd00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb294009850 tx=0x7fb294009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2631fe049f9dc91 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:14.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2a67fc700 1 -- 172.21.15.40:0/2043109244 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294011830 con 0x7fb2a0046fa0 2024-09-13T18:03:14.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2a67fc700 1 -- 172.21.15.40:0/2043109244 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb294011990 con 0x7fb2a0046fa0 2024-09-13T18:03:14.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2a67fc700 1 -- 172.21.15.40:0/2043109244 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294011d20 con 0x7fb2a0046fa0 2024-09-13T18:03:14.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2ac9c9700 1 -- 172.21.15.40:0/2043109244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2a0046fa0 msgr2=0x7fb2a004cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:14.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2ac9c9700 1 --2- 172.21.15.40:0/2043109244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2a0046fa0 0x7fb2a004cd00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb294009850 tx=0x7fb294009b60 comp rx=0 tx=0).stop 2024-09-13T18:03:14.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2ac9c9700 1 -- 172.21.15.40:0/2043109244 shutdown_connections 2024-09-13T18:03:14.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2ac9c9700 1 --2- 172.21.15.40:0/2043109244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2a0046fa0 0x7fb2a004cd00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:14.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2ac9c9700 1 --2- 172.21.15.40:0/2043109244 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb2a00d88f0 0x7fb2a004e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:14.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2ac9c9700 1 -- 172.21.15.40:0/2043109244 >> 172.21.15.40:0/2043109244 conn(0x7fb2a0051f10 msgr2=0x7fb2a0052320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:14.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2ac9c9700 1 -- 172.21.15.40:0/2043109244 shutdown_connections 2024-09-13T18:03:14.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2ac9c9700 1 -- 172.21.15.40:0/2043109244 wait complete. 2024-09-13T18:03:14.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2ac9c9700 1 Processor -- start 2024-09-13T18:03:14.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.674+0000 7fb2ac9c9700 1 -- start start 2024-09-13T18:03:14.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2ac9c9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2a0046fa0 0x7fb2a00d4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:14.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2ac9c9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb2a004d240 0x7fb2a0175870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:14.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2ac9c9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a00d88f0 0x7fb2a004ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:14.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2ac9c9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a00d3e30 con 0x7fb2a00d88f0 2024-09-13T18:03:14.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2ac9c9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a0033480 con 0x7fb2a0046fa0 2024-09-13T18:03:14.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2ac9c9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a00d4340 con 0x7fb2a004d240 2024-09-13T18:03:14.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2a6ffd700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb2a004d240 msgr2=0x7fb2a0175870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:03:14.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2a6ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb2a004d240 0x7fb2a0175870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:14.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a00d88f0 0x7fb2a004ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:14.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a00d88f0 0x7fb2a004ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41344/0 (socket says 172.21.15.40:41344) 2024-09-13T18:03:14.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2a7fff700 1 -- 172.21.15.40:0/125925929 learned_addr learned my addr 172.21.15.40:0/125925929 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:14.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2a77fe700 1 --2- 172.21.15.40:0/125925929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2a0046fa0 0x7fb2a00d4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:14.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2a7fff700 1 -- 172.21.15.40:0/125925929 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb2a004d240 msgr2=0x7fb2a0175870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:03:14.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2a7fff700 1 --2- 172.21.15.40:0/125925929 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb2a004d240 0x7fb2a0175870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:14.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2a7fff700 1 -- 172.21.15.40:0/125925929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2a0046fa0 msgr2=0x7fb2a00d4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:14.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2a7fff700 1 --2- 172.21.15.40:0/125925929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2a0046fa0 0x7fb2a00d4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:14.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2a7fff700 1 -- 172.21.15.40:0/125925929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2940094b0 con 0x7fb2a00d88f0 2024-09-13T18:03:14.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2a7fff700 1 --2- 172.21.15.40:0/125925929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a00d88f0 0x7fb2a004ff60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8063990 tx=0x7fb2a80747c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:14.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb293fff700 1 -- 172.21.15.40:0/125925929 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8074db0 con 0x7fb2a00d88f0 2024-09-13T18:03:14.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb293fff700 1 -- 172.21.15.40:0/125925929 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a80720c0 con 0x7fb2a00d88f0 2024-09-13T18:03:14.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2ac9c9700 1 -- 172.21.15.40:0/125925929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a0031ab0 con 0x7fb2a00d88f0 2024-09-13T18:03:14.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb293fff700 1 -- 172.21.15.40:0/125925929 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8072430 con 0x7fb2a00d88f0 2024-09-13T18:03:14.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2ac9c9700 1 -- 172.21.15.40:0/125925929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a00d4060 con 0x7fb2a00d88f0 2024-09-13T18:03:14.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb293fff700 1 -- 172.21.15.40:0/125925929 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb2a8072590 con 0x7fb2a00d88f0 2024-09-13T18:03:14.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.678+0000 7fb2ac9c9700 1 -- 172.21.15.40:0/125925929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a0031c40 con 0x7fb2a00d88f0 2024-09-13T18:03:14.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.682+0000 7fb293fff700 1 --2- 172.21.15.40:0/125925929 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb28c0666e0 0x7fb28c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:14.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.682+0000 7fb293fff700 1 -- 172.21.15.40:0/125925929 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb2a80fe1e0 con 0x7fb2a00d88f0 2024-09-13T18:03:14.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.682+0000 7fb2a77fe700 1 --2- 172.21.15.40:0/125925929 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb28c0666e0 0x7fb28c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:14.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.682+0000 7fb2a77fe700 1 --2- 172.21.15.40:0/125925929 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb28c0666e0 0x7fb28c068ba0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009470 tx=0x7fb29c009400 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:14.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.690+0000 7fb293fff700 1 -- 172.21.15.40:0/125925929 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb2a80cb190 con 0x7fb2a00d88f0 2024-09-13T18:03:14.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.974+0000 7fb2ac9c9700 1 -- 172.21.15.40:0/125925929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb2a004cd00 con 0x7fb2a00d88f0 2024-09-13T18:03:14.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.978+0000 7fb293fff700 1 -- 172.21.15.40:0/125925929 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fb2a80ce7b0 con 0x7fb2a00d88f0 2024-09-13T18:03:14.988 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":98,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:03:14.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.982+0000 7fb2ac9c9700 1 -- 172.21.15.40:0/125925929 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb28c0666e0 msgr2=0x7fb28c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.982+0000 7fb2ac9c9700 1 --2- 172.21.15.40:0/125925929 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb28c0666e0 0x7fb28c068ba0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009470 tx=0x7fb29c009400 comp rx=0 tx=0).stop 2024-09-13T18:03:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.982+0000 7fb2ac9c9700 1 -- 172.21.15.40:0/125925929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a00d88f0 msgr2=0x7fb2a004ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.982+0000 7fb2ac9c9700 1 --2- 172.21.15.40:0/125925929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a00d88f0 0x7fb2a004ff60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8063990 tx=0x7fb2a80747c0 comp rx=0 tx=0).stop 2024-09-13T18:03:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.982+0000 7fb2ac9c9700 1 -- 172.21.15.40:0/125925929 shutdown_connections 2024-09-13T18:03:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.982+0000 7fb2ac9c9700 1 --2- 172.21.15.40:0/125925929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a00d88f0 0x7fb2a004ff60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.982+0000 7fb2ac9c9700 1 --2- 172.21.15.40:0/125925929 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb28c0666e0 0x7fb28c068ba0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.982+0000 7fb2ac9c9700 1 --2- 172.21.15.40:0/125925929 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb2a004d240 0x7fb2a0175870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.982+0000 7fb2ac9c9700 1 --2- 172.21.15.40:0/125925929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2a0046fa0 0x7fb2a00d4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.982+0000 7fb2ac9c9700 1 -- 172.21.15.40:0/125925929 >> 172.21.15.40:0/125925929 conn(0x7fb2a0051f10 msgr2=0x7fb2a0050d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.982+0000 7fb2ac9c9700 1 -- 172.21.15.40:0/125925929 shutdown_connections 2024-09-13T18:03:14.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:14.982+0000 7fb2ac9c9700 1 -- 172.21.15.40:0/125925929 wait complete. 2024-09-13T18:03:18.009 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:03:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.114+0000 7faa7088c700 1 Processor -- start 2024-09-13T18:03:18.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.114+0000 7faa7088c700 1 -- start start 2024-09-13T18:03:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.114+0000 7faa7088c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faa6c06dc80 0x7faa6c070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.114+0000 7faa7088c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6c14fdd0 con 0x7faa6c14f9f0 2024-09-13T18:03:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.114+0000 7faa7088c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6c1558b0 con 0x7faa6c06ea60 2024-09-13T18:03:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.114+0000 7faa7088c700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa6c070920 con 0x7faa6c06dc80 2024-09-13T18:03:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.114+0000 7faa6b7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7faa6c14f9f0 0x7faa6c070ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60126/0 (socket says 172.21.15.40:60126) 2024-09-13T18:03:18.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.114+0000 7faa6b7fe700 1 -- 172.21.15.40:0/1233516467 learned_addr learned my addr 172.21.15.40:0/1233516467 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:03:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.114+0000 7faa6bfff700 1 --2- 172.21.15.40:0/1233516467 >> v2:172.21.15.43:3300/0 conn(0x7faa6c06dc80 0x7faa6c070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.118+0000 7faa6bfff700 1 -- 172.21.15.40:0/1233516467 >> v1:172.21.15.43:6789/0 conn(0x7faa6c06ea60 legacy=0x7faa6c06d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.118+0000 7faa6bfff700 1 -- 172.21.15.40:0/1233516467 >> v1:172.21.15.40:6789/0 conn(0x7faa6c14f9f0 legacy=0x7faa6c070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:18.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.118+0000 7faa6bfff700 1 -- 172.21.15.40:0/1233516467 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6c155b50 con 0x7faa6c06dc80 2024-09-13T18:03:18.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.118+0000 7faa6bfff700 1 --2- 172.21.15.40:0/1233516467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c06dc80 0x7faa6c070290 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa6400aea0 tx=0x7faa6400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e1a52f5ac124ad4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:18.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.118+0000 7faa6a7fc700 1 -- 172.21.15.40:0/1233516467 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6400f070 con 0x7faa6c06dc80 2024-09-13T18:03:18.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.118+0000 7faa6a7fc700 1 -- 172.21.15.40:0/1233516467 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6400c9c0 con 0x7faa6c06dc80 2024-09-13T18:03:18.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.118+0000 7faa6a7fc700 1 -- 172.21.15.40:0/1233516467 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6400cd70 con 0x7faa6c06dc80 2024-09-13T18:03:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.118+0000 7faa7088c700 1 -- 172.21.15.40:0/1233516467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c06dc80 msgr2=0x7faa6c070290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.118+0000 7faa7088c700 1 --2- 172.21.15.40:0/1233516467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c06dc80 0x7faa6c070290 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa6400aea0 tx=0x7faa6400c330 comp rx=0 tx=0).stop 2024-09-13T18:03:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.118+0000 7faa7088c700 1 -- 172.21.15.40:0/1233516467 shutdown_connections 2024-09-13T18:03:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.118+0000 7faa7088c700 1 --2- 172.21.15.40:0/1233516467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c06dc80 0x7faa6c070290 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.118+0000 7faa7088c700 1 -- 172.21.15.40:0/1233516467 >> 172.21.15.40:0/1233516467 conn(0x7faa6c06a390 msgr2=0x7faa6c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.118+0000 7faa7088c700 1 -- 172.21.15.40:0/1233516467 shutdown_connections 2024-09-13T18:03:18.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.118+0000 7faa7088c700 1 -- 172.21.15.40:0/1233516467 wait complete. 2024-09-13T18:03:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.118+0000 7faa7088c700 1 Processor -- start 2024-09-13T18:03:18.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.118+0000 7faa7088c700 1 -- start start 2024-09-13T18:03:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa7088c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa6c06dc80 0x7faa6c14ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa7088c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c06ea60 0x7faa6c0ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa7088c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa6c14f9f0 0x7faa6c147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa7088c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c1564d0 con 0x7faa6c06dc80 2024-09-13T18:03:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa7088c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c1eee50 con 0x7faa6c06ea60 2024-09-13T18:03:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa7088c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c1ef6b0 con 0x7faa6c14f9f0 2024-09-13T18:03:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa6b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa6c06dc80 0x7faa6c14ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa6bfff700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa6c14f9f0 msgr2=0x7faa6c147be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:03:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa6bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa6c14f9f0 0x7faa6c147be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa6b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa6c06dc80 0x7faa6c14ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:41356/0 (socket says 172.21.15.40:41356) 2024-09-13T18:03:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa6b7fe700 1 -- 172.21.15.40:0/2558032737 learned_addr learned my addr 172.21.15.40:0/2558032737 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:18.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa6affd700 1 --2- 172.21.15.40:0/2558032737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c06ea60 0x7faa6c0ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:18.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa6b7fe700 1 -- 172.21.15.40:0/2558032737 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa6c14f9f0 msgr2=0x7faa6c147be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:03:18.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa6b7fe700 1 --2- 172.21.15.40:0/2558032737 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa6c14f9f0 0x7faa6c147be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:18.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa6b7fe700 1 -- 172.21.15.40:0/2558032737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c06ea60 msgr2=0x7faa6c0ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:18.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa6b7fe700 1 --2- 172.21.15.40:0/2558032737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c06ea60 0x7faa6c0ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:18.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa6b7fe700 1 -- 172.21.15.40:0/2558032737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6400ab80 con 0x7faa6c06dc80 2024-09-13T18:03:18.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa6b7fe700 1 --2- 172.21.15.40:0/2558032737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa6c06dc80 0x7faa6c14ffb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faa5c001eb0 tx=0x7faa5c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:18.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa53fff700 1 -- 172.21.15.40:0/2558032737 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c005d90 con 0x7faa6c06dc80 2024-09-13T18:03:18.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa53fff700 1 -- 172.21.15.40:0/2558032737 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5c0070d0 con 0x7faa6c06dc80 2024-09-13T18:03:18.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa7088c700 1 -- 172.21.15.40:0/2558032737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa6c153aa0 con 0x7faa6c06dc80 2024-09-13T18:03:18.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa53fff700 1 -- 172.21.15.40:0/2558032737 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c007440 con 0x7faa6c06dc80 2024-09-13T18:03:18.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa7088c700 1 -- 172.21.15.40:0/2558032737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa6c1ef060 con 0x7faa6c06dc80 2024-09-13T18:03:18.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.122+0000 7faa7088c700 1 -- 172.21.15.40:0/2558032737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa6c070290 con 0x7faa6c06dc80 2024-09-13T18:03:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.126+0000 7faa53fff700 1 -- 172.21.15.40:0/2558032737 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faa5c0075a0 con 0x7faa6c06dc80 2024-09-13T18:03:18.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.126+0000 7faa53fff700 1 --2- 172.21.15.40:0/2558032737 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faa58066710 0x7faa58068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:18.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.126+0000 7faa53fff700 1 -- 172.21.15.40:0/2558032737 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7faa5c083b80 con 0x7faa6c06dc80 2024-09-13T18:03:18.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.126+0000 7faa6affd700 1 --2- 172.21.15.40:0/2558032737 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faa58066710 0x7faa58068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:18.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.126+0000 7faa6affd700 1 --2- 172.21.15.40:0/2558032737 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faa58066710 0x7faa58068bd0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faa6c1556b0 tx=0x7faa60009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:18.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.130+0000 7faa53fff700 1 -- 172.21.15.40:0/2558032737 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7faa5c050d50 con 0x7faa6c06dc80 2024-09-13T18:03:18.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.434+0000 7faa7088c700 1 -- 172.21.15.40:0/2558032737 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7faa6c1504f0 con 0x7faa6c06dc80 2024-09-13T18:03:18.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.438+0000 7faa53fff700 1 -- 172.21.15.40:0/2558032737 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7faa5c021620 con 0x7faa6c06dc80 2024-09-13T18:03:18.448 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":98,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:03:18.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.442+0000 7faa7088c700 1 -- 172.21.15.40:0/2558032737 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faa58066710 msgr2=0x7faa58068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:18.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.442+0000 7faa7088c700 1 --2- 172.21.15.40:0/2558032737 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faa58066710 0x7faa58068bd0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faa6c1556b0 tx=0x7faa60009040 comp rx=0 tx=0).stop 2024-09-13T18:03:18.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.442+0000 7faa7088c700 1 -- 172.21.15.40:0/2558032737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa6c06dc80 msgr2=0x7faa6c14ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:18.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.442+0000 7faa7088c700 1 --2- 172.21.15.40:0/2558032737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa6c06dc80 0x7faa6c14ffb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faa5c001eb0 tx=0x7faa5c00c7b0 comp rx=0 tx=0).stop 2024-09-13T18:03:18.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.442+0000 7faa7088c700 1 -- 172.21.15.40:0/2558032737 shutdown_connections 2024-09-13T18:03:18.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.442+0000 7faa7088c700 1 --2- 172.21.15.40:0/2558032737 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faa58066710 0x7faa58068bd0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:18.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.442+0000 7faa7088c700 1 --2- 172.21.15.40:0/2558032737 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa6c14f9f0 0x7faa6c147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:18.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.442+0000 7faa7088c700 1 --2- 172.21.15.40:0/2558032737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c06ea60 0x7faa6c0ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:18.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.442+0000 7faa7088c700 1 --2- 172.21.15.40:0/2558032737 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa6c06dc80 0x7faa6c14ffb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:18.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.442+0000 7faa7088c700 1 -- 172.21.15.40:0/2558032737 >> 172.21.15.40:0/2558032737 conn(0x7faa6c06a390 msgr2=0x7faa6c0c3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:18.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.442+0000 7faa7088c700 1 -- 172.21.15.40:0/2558032737 shutdown_connections 2024-09-13T18:03:18.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:18.442+0000 7faa7088c700 1 -- 172.21.15.40:0/2558032737 wait complete. 2024-09-13T18:03:21.468 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:03:21.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.578+0000 7ff421ddf700 1 Processor -- start 2024-09-13T18:03:21.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.578+0000 7ff421ddf700 1 -- start start 2024-09-13T18:03:21.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.578+0000 7ff421ddf700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 0x7ff41c0c0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:21.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.578+0000 7ff421ddf700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff41c0c28f0 0x7ff41c1565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:21.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.578+0000 7ff421ddf700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff41c0b2210 con 0x7ff41c072240 2024-09-13T18:03:21.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.578+0000 7ff421ddf700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff41c0c1780 con 0x7ff41c071e60 2024-09-13T18:03:21.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.578+0000 7ff421ddf700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c0c1a60 con 0x7ff41c0c28f0 2024-09-13T18:03:21.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.578+0000 7ff4215de700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff41c0c28f0 msgr2=0x7ff41c1565c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:03:21.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.578+0000 7ff4215de700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff41c0c28f0 0x7ff41c1565c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:21.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.578+0000 7ff420ddd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 0x7ff41c0c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:21.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.578+0000 7ff420ddd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 0x7ff41c0c0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:50434/0 (socket says 172.21.15.40:50434) 2024-09-13T18:03:21.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.578+0000 7ff41bfff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff41c072240 0x7ff41c0c32e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:51304/0 (socket says 172.21.15.40:51304) 2024-09-13T18:03:21.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.578+0000 7ff41bfff700 1 -- 172.21.15.40:0/4252420405 learned_addr learned my addr 172.21.15.40:0/4252420405 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:03:21.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.582+0000 7ff420ddd700 1 -- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 msgr2=0x7ff41c0c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:03:21.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.582+0000 7ff420ddd700 1 -- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 msgr2=0x7ff41c0c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:03:21.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.582+0000 7ff420ddd700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 0x7ff41c0c0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:21.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.582+0000 7ff420ddd700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 0x7ff41c0c0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:21.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.782+0000 7ff4215de700 1 -- 172.21.15.40:0/4252420405 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff41c0c28f0 msgr2=0x7ff41c1565c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:03:21.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.782+0000 7ff4215de700 1 --2- 172.21.15.40:0/4252420405 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff41c0c28f0 0x7ff41c1565c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:03:21.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.786+0000 7ff420ddd700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 0x7ff41c0c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:21.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.786+0000 7ff420ddd700 1 -- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 msgr2=0x7ff41c0c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:03:21.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.786+0000 7ff420ddd700 1 -- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 msgr2=0x7ff41c0c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:03:21.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.786+0000 7ff420ddd700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 0x7ff41c0c0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:21.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:21.786+0000 7ff420ddd700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 0x7ff41c0c0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:03:22.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:22.186+0000 7ff4215de700 1 -- 172.21.15.40:0/4252420405 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff41c0c28f0 msgr2=0x7ff41c1565c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:03:22.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:22.186+0000 7ff4215de700 1 --2- 172.21.15.40:0/4252420405 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff41c0c28f0 0x7ff41c1565c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:03:22.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:22.190+0000 7ff420ddd700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 0x7ff41c0c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:22.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:22.190+0000 7ff420ddd700 1 -- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 msgr2=0x7ff41c0c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:03:22.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:22.190+0000 7ff420ddd700 1 -- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 msgr2=0x7ff41c0c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:03:22.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:22.190+0000 7ff420ddd700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 0x7ff41c0c0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:22.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:22.190+0000 7ff420ddd700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 0x7ff41c0c0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:03:22.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:22.990+0000 7ff4215de700 1 -- 172.21.15.40:0/4252420405 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff41c0c28f0 msgr2=0x7ff41c1565c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:03:23.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:22.990+0000 7ff4215de700 1 --2- 172.21.15.40:0/4252420405 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff41c0c28f0 0x7ff41c1565c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:03:23.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:22.994+0000 7ff420ddd700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 0x7ff41c0c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:23.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:22.994+0000 7ff420ddd700 1 -- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 msgr2=0x7ff41c0c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:03:23.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:22.994+0000 7ff420ddd700 1 -- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 msgr2=0x7ff41c0c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:03:23.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:22.994+0000 7ff420ddd700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 0x7ff41c0c0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:23.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:22.994+0000 7ff420ddd700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 0x7ff41c0c0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:03:24.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:24.578+0000 7ff41a7fc700 1 -- 172.21.15.40:0/4252420405 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff41c0c28f0 msgr2=0x7ff41c1565c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:03:24.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:24.578+0000 7ff41a7fc700 1 --2- 172.21.15.40:0/4252420405 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff41c0c28f0 0x7ff41c1565c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:24.578+0000 7ff41a7fc700 1 -- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 msgr2=0x7ff41c0c0e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:03:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:24.578+0000 7ff41a7fc700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.43:3300/0 conn(0x7ff41c071e60 0x7ff41c0c0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:24.578+0000 7ff41a7fc700 1 -- 172.21.15.40:0/4252420405 >> v1:172.21.15.43:6789/0 conn(0x7ff41c072240 legacy=0x7ff41c0c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:24.578+0000 7ff41a7fc700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.40:3300/0 conn(0x7ff400007480 0x7ff400009920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:24.578+0000 7ff41a7fc700 1 -- 172.21.15.40:0/4252420405 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff41c0c5d70 con 0x7ff400003740 2024-09-13T18:03:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:24.578+0000 7ff41a7fc700 1 -- 172.21.15.40:0/4252420405 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff41c0c1780 con 0x7ff400007480 2024-09-13T18:03:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:24.582+0000 7ff41bfff700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.40:3300/0 conn(0x7ff400007480 0x7ff400009920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:24.582+0000 7ff41bfff700 1 -- 172.21.15.40:0/4252420405 >> v2:172.21.15.40:3300/0 conn(0x7ff400007480 msgr2=0x7ff400009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:03:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:24.582+0000 7ff41bfff700 1 -- 172.21.15.40:0/4252420405 >> v2:172.21.15.40:3300/0 conn(0x7ff400007480 msgr2=0x7ff400009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:03:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:24.582+0000 7ff41bfff700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.40:3300/0 conn(0x7ff400007480 0x7ff400009920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:24.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:24.582+0000 7ff41bfff700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.40:3300/0 conn(0x7ff400007480 0x7ff400009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:24.786+0000 7ff41bfff700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.40:3300/0 conn(0x7ff400007480 0x7ff400009920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:24.786+0000 7ff41bfff700 1 -- 172.21.15.40:0/4252420405 >> v2:172.21.15.40:3300/0 conn(0x7ff400007480 msgr2=0x7ff400009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:03:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:24.786+0000 7ff41bfff700 1 -- 172.21.15.40:0/4252420405 >> v2:172.21.15.40:3300/0 conn(0x7ff400007480 msgr2=0x7ff400009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:03:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:24.786+0000 7ff41bfff700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.40:3300/0 conn(0x7ff400007480 0x7ff400009920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:24.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:24.786+0000 7ff41bfff700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.40:3300/0 conn(0x7ff400007480 0x7ff400009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:03:25.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.190+0000 7ff41bfff700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.40:3300/0 conn(0x7ff400007480 0x7ff400009920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:25.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.190+0000 7ff41bfff700 1 -- 172.21.15.40:0/4252420405 >> v2:172.21.15.40:3300/0 conn(0x7ff400007480 msgr2=0x7ff400009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:03:25.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.190+0000 7ff41bfff700 1 -- 172.21.15.40:0/4252420405 >> v2:172.21.15.40:3300/0 conn(0x7ff400007480 msgr2=0x7ff400009920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:03:25.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.190+0000 7ff41bfff700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.40:3300/0 conn(0x7ff400007480 0x7ff400009920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:25.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.190+0000 7ff41bfff700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.40:3300/0 conn(0x7ff400007480 0x7ff400009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:03:25.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.674+0000 7ff41b7fe700 1 -- 172.21.15.40:0/4252420405 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 899176265 0 0) 0x7ff41c0c5d70 con 0x7ff400003740 2024-09-13T18:03:25.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.674+0000 7ff41b7fe700 1 -- 172.21.15.40:0/4252420405 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff40c0035d0 con 0x7ff400003740 2024-09-13T18:03:25.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff41b7fe700 1 -- 172.21.15.40:0/4252420405 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3102438461 0 0) 0x7ff40c0035d0 con 0x7ff400003740 2024-09-13T18:03:25.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff41b7fe700 1 -- 172.21.15.40:0/4252420405 >> v2:172.21.15.40:3300/0 conn(0x7ff400007480 msgr2=0x7ff400009920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:03:25.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff41b7fe700 1 --2- 172.21.15.40:0/4252420405 >> v2:172.21.15.40:3300/0 conn(0x7ff400007480 0x7ff400009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:25.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff41b7fe700 1 -- 172.21.15.40:0/4252420405 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40000b020 con 0x7ff400003740 2024-09-13T18:03:25.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff41b7fe700 1 -- 172.21.15.40:0/4252420405 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7ff41000c570 con 0x7ff400003740 2024-09-13T18:03:25.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff41b7fe700 1 -- 172.21.15.40:0/4252420405 >> v1:172.21.15.40:6789/0 conn(0x7ff400003740 legacy=0x7ff400005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:25.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff4215de700 1 -- 172.21.15.40:0/4252420405 reap_dead start 2024-09-13T18:03:25.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff41b7fe700 1 --2- 172.21.15.40:0/4252420405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff40c003a20 0x7ff40c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:25.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff41b7fe700 1 --2- 172.21.15.40:0/4252420405 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff40c004870 0x7ff40c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:25.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff41b7fe700 1 --2- 172.21.15.40:0/4252420405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40c005280 0x7ff40c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:25.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff41b7fe700 1 -- 172.21.15.40:0/4252420405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c0c1780 con 0x7ff40c003a20 2024-09-13T18:03:25.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff41b7fe700 1 -- 172.21.15.40:0/4252420405 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c00c1b0 con 0x7ff40c005280 2024-09-13T18:03:25.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff41b7fe700 1 -- 172.21.15.40:0/4252420405 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c00c370 con 0x7ff40c004870 2024-09-13T18:03:25.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff41bfff700 1 -- 172.21.15.40:0/4252420405 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff40c004870 msgr2=0x7ff40c004cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:03:25.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff41bfff700 1 --2- 172.21.15.40:0/4252420405 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff40c004870 0x7ff40c004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:25.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff420ddd700 1 --2- 172.21.15.40:0/4252420405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff40c003a20 0x7ff40c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:25.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff4215de700 1 --2- 172.21.15.40:0/4252420405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40c005280 0x7ff40c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:25.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff420ddd700 1 -- 172.21.15.40:0/4252420405 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff40c004870 msgr2=0x7ff40c004cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:03:25.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff420ddd700 1 --2- 172.21.15.40:0/4252420405 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff40c004870 0x7ff40c004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:25.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff420ddd700 1 -- 172.21.15.40:0/4252420405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40c005280 msgr2=0x7ff40c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:03:25.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff420ddd700 1 --2- 172.21.15.40:0/4252420405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40c005280 0x7ff40c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:25.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff420ddd700 1 -- 172.21.15.40:0/4252420405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c0035d0 con 0x7ff40c003a20 2024-09-13T18:03:25.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff420ddd700 1 --2- 172.21.15.40:0/4252420405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff40c003a20 0x7ff40c003e20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff410009100 tx=0x7ff40c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afc70e53abed2a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:25.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff41b7fe700 1 -- 172.21.15.40:0/4252420405 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff410014ae0 con 0x7ff40c003a20 2024-09-13T18:03:25.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff41b7fe700 1 -- 172.21.15.40:0/4252420405 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41000c040 con 0x7ff40c003a20 2024-09-13T18:03:25.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff41b7fe700 1 -- 172.21.15.40:0/4252420405 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff410014e60 con 0x7ff40c003a20 2024-09-13T18:03:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff421ddf700 1 -- 172.21.15.40:0/4252420405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff40c003a20 msgr2=0x7ff40c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff421ddf700 1 --2- 172.21.15.40:0/4252420405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff40c003a20 0x7ff40c003e20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff410009100 tx=0x7ff40c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:03:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff421ddf700 1 -- 172.21.15.40:0/4252420405 shutdown_connections 2024-09-13T18:03:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff421ddf700 1 --2- 172.21.15.40:0/4252420405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40c005280 0x7ff40c00ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff421ddf700 1 --2- 172.21.15.40:0/4252420405 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff40c004870 0x7ff40c004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff421ddf700 1 --2- 172.21.15.40:0/4252420405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff40c003a20 0x7ff40c003e20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:25.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff421ddf700 1 -- 172.21.15.40:0/4252420405 >> 172.21.15.40:0/4252420405 conn(0x7ff41c06e4f0 msgr2=0x7ff41c06e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:25.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff421ddf700 1 -- 172.21.15.40:0/4252420405 shutdown_connections 2024-09-13T18:03:25.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.678+0000 7ff421ddf700 1 -- 172.21.15.40:0/4252420405 wait complete. 2024-09-13T18:03:25.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff421ddf700 1 Processor -- start 2024-09-13T18:03:25.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff421ddf700 1 -- start start 2024-09-13T18:03:25.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff421ddf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff40c003a20 0x7ff41c1ee710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:25.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff421ddf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40c004870 0x7ff41c0c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:25.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff421ddf700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff40c005280 0x7ff41c1516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:25.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff421ddf700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c06c480 con 0x7ff40c003a20 2024-09-13T18:03:25.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff421ddf700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c0b0ae0 con 0x7ff40c004870 2024-09-13T18:03:25.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff421ddf700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c0b1080 con 0x7ff40c005280 2024-09-13T18:03:25.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff420ddd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff40c003a20 0x7ff41c1ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:25.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff4215de700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff40c005280 msgr2=0x7ff41c1516e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:03:25.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff420ddd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff40c003a20 0x7ff41c1ee710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43370/0 (socket says 172.21.15.40:43370) 2024-09-13T18:03:25.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff420ddd700 1 -- 172.21.15.40:0/4165090559 learned_addr learned my addr 172.21.15.40:0/4165090559 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:25.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff4215de700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff40c005280 0x7ff41c1516e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:25.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff41bfff700 1 --2- 172.21.15.40:0/4165090559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40c004870 0x7ff41c0c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:25.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff420ddd700 1 -- 172.21.15.40:0/4165090559 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff40c005280 msgr2=0x7ff41c1516e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:03:25.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff420ddd700 1 --2- 172.21.15.40:0/4165090559 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff40c005280 0x7ff41c1516e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:25.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff420ddd700 1 -- 172.21.15.40:0/4165090559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40c004870 msgr2=0x7ff41c0c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:25.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff420ddd700 1 --2- 172.21.15.40:0/4165090559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40c004870 0x7ff41c0c7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:25.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff420ddd700 1 -- 172.21.15.40:0/4165090559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c06bbe0 con 0x7ff40c003a20 2024-09-13T18:03:25.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff420ddd700 1 --2- 172.21.15.40:0/4165090559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff40c003a20 0x7ff41c1ee710 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff4100148b0 tx=0x7ff410004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:25.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff4197fa700 1 -- 172.21.15.40:0/4165090559 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff410025070 con 0x7ff40c003a20 2024-09-13T18:03:25.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff4197fa700 1 -- 172.21.15.40:0/4165090559 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff410006c80 con 0x7ff40c003a20 2024-09-13T18:03:25.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff421ddf700 1 -- 172.21.15.40:0/4165090559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41c06bec0 con 0x7ff40c003a20 2024-09-13T18:03:25.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff4197fa700 1 -- 172.21.15.40:0/4165090559 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff410004230 con 0x7ff40c003a20 2024-09-13T18:03:25.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.682+0000 7ff421ddf700 1 -- 172.21.15.40:0/4165090559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41c0b0d10 con 0x7ff40c003a20 2024-09-13T18:03:25.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.686+0000 7ff421ddf700 1 -- 172.21.15.40:0/4165090559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff41c0c1a60 con 0x7ff40c003a20 2024-09-13T18:03:25.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.686+0000 7ff4197fa700 1 -- 172.21.15.40:0/4165090559 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff410004390 con 0x7ff40c003a20 2024-09-13T18:03:25.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.686+0000 7ff4197fa700 1 --2- 172.21.15.40:0/4165090559 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff400066fc0 0x7ff400069480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:25.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.686+0000 7ff4197fa700 1 -- 172.21.15.40:0/4165090559 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff410000e40 con 0x7ff40c003a20 2024-09-13T18:03:25.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.686+0000 7ff41bfff700 1 --2- 172.21.15.40:0/4165090559 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff400066fc0 0x7ff400069480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:25.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.686+0000 7ff41bfff700 1 --2- 172.21.15.40:0/4165090559 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff400066fc0 0x7ff400069480 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff41c1ef8a0 tx=0x7ff408009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:25.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.694+0000 7ff4197fa700 1 -- 172.21.15.40:0/4165090559 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff4100528e0 con 0x7ff40c003a20 2024-09-13T18:03:25.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.946+0000 7ff421ddf700 1 -- 172.21.15.40:0/4165090559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff41c1eec50 con 0x7ff40c003a20 2024-09-13T18:03:25.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.946+0000 7ff4197fa700 1 -- 172.21.15.40:0/4165090559 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7ff410055f00 con 0x7ff40c003a20 2024-09-13T18:03:25.957 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":102,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:03:25.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.950+0000 7ff421ddf700 1 -- 172.21.15.40:0/4165090559 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff400066fc0 msgr2=0x7ff400069480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:25.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.950+0000 7ff421ddf700 1 --2- 172.21.15.40:0/4165090559 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff400066fc0 0x7ff400069480 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff41c1ef8a0 tx=0x7ff408009040 comp rx=0 tx=0).stop 2024-09-13T18:03:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.950+0000 7ff421ddf700 1 -- 172.21.15.40:0/4165090559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff40c003a20 msgr2=0x7ff41c1ee710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.950+0000 7ff421ddf700 1 --2- 172.21.15.40:0/4165090559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff40c003a20 0x7ff41c1ee710 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff4100148b0 tx=0x7ff410004770 comp rx=0 tx=0).stop 2024-09-13T18:03:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.950+0000 7ff421ddf700 1 -- 172.21.15.40:0/4165090559 shutdown_connections 2024-09-13T18:03:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.950+0000 7ff421ddf700 1 --2- 172.21.15.40:0/4165090559 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff400066fc0 0x7ff400069480 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.950+0000 7ff421ddf700 1 --2- 172.21.15.40:0/4165090559 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff40c005280 0x7ff41c1516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.950+0000 7ff421ddf700 1 --2- 172.21.15.40:0/4165090559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff40c004870 0x7ff41c0c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.950+0000 7ff421ddf700 1 --2- 172.21.15.40:0/4165090559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff40c003a20 0x7ff41c1ee710 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.950+0000 7ff421ddf700 1 -- 172.21.15.40:0/4165090559 >> 172.21.15.40:0/4165090559 conn(0x7ff41c06e4f0 msgr2=0x7ff41c156210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.950+0000 7ff421ddf700 1 -- 172.21.15.40:0/4165090559 shutdown_connections 2024-09-13T18:03:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:25.950+0000 7ff421ddf700 1 -- 172.21.15.40:0/4165090559 wait complete. 2024-09-13T18:03:25.975 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T18:03:25.975 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-13T18:03:25.976 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:03:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.078+0000 7f4a449a6700 1 Processor -- start 2024-09-13T18:03:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.078+0000 7f4a449a6700 1 -- start start 2024-09-13T18:03:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.078+0000 7f4a449a6700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4a40071170 0x7f4a40070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.078+0000 7f4a449a6700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a400c4400 con 0x7f4a40150480 2024-09-13T18:03:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.078+0000 7f4a449a6700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a400c46e0 con 0x7f4a40150860 2024-09-13T18:03:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.078+0000 7f4a449a6700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a400709b0 con 0x7f4a40071170 2024-09-13T18:03:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.078+0000 7f4a3e59c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4a40150480 0x7f4a400c65f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47948/0 (socket says 172.21.15.40:47948) 2024-09-13T18:03:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.078+0000 7f4a3e59c700 1 -- 172.21.15.40:0/966331976 learned_addr learned my addr 172.21.15.40:0/966331976 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:03:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.078+0000 7f4a3ed9d700 1 --2- 172.21.15.40:0/966331976 >> v2:172.21.15.40:3300/0 conn(0x7f4a40071170 0x7f4a40070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:26.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.082+0000 7f4a3ed9d700 1 -- 172.21.15.40:0/966331976 >> v1:172.21.15.43:6789/0 conn(0x7f4a40150860 legacy=0x7f4a400c5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:26.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.082+0000 7f4a3ed9d700 1 -- 172.21.15.40:0/966331976 >> v1:172.21.15.40:6789/0 conn(0x7f4a40150480 legacy=0x7f4a400c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:26.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.082+0000 7f4a3d59a700 1 -- 172.21.15.40:0/966331976 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037171634 0 0) 0x7f4a400c4400 con 0x7f4a40150480 2024-09-13T18:03:26.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.082+0000 7f4a3ed9d700 1 -- 172.21.15.40:0/966331976 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a400c49c0 con 0x7f4a40071170 2024-09-13T18:03:26.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.082+0000 7f4a3ed9d700 1 --2- 172.21.15.40:0/966331976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a40071170 0x7f4a40070370 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009d90 tx=0x7f4a3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7f58ffdd5e8e1c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:26.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.082+0000 7f4a3d59a700 1 -- 172.21.15.40:0/966331976 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3800e070 con 0x7f4a40071170 2024-09-13T18:03:26.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.082+0000 7f4a3d59a700 1 -- 172.21.15.40:0/966331976 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3800bbd0 con 0x7f4a40071170 2024-09-13T18:03:26.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.082+0000 7f4a3d59a700 1 -- 172.21.15.40:0/966331976 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a38012430 con 0x7f4a40071170 2024-09-13T18:03:26.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.082+0000 7f4a449a6700 1 -- 172.21.15.40:0/966331976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a40071170 msgr2=0x7f4a40070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:26.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.082+0000 7f4a449a6700 1 --2- 172.21.15.40:0/966331976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a40071170 0x7f4a40070370 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009d90 tx=0x7f4a3800b330 comp rx=0 tx=0).stop 2024-09-13T18:03:26.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.082+0000 7f4a449a6700 1 -- 172.21.15.40:0/966331976 shutdown_connections 2024-09-13T18:03:26.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.082+0000 7f4a449a6700 1 --2- 172.21.15.40:0/966331976 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a40071170 0x7f4a40070370 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:26.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.082+0000 7f4a449a6700 1 -- 172.21.15.40:0/966331976 >> 172.21.15.40:0/966331976 conn(0x7f4a4006a390 msgr2=0x7f4a4006a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:26.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.082+0000 7f4a449a6700 1 -- 172.21.15.40:0/966331976 shutdown_connections 2024-09-13T18:03:26.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.082+0000 7f4a449a6700 1 -- 172.21.15.40:0/966331976 wait complete. 2024-09-13T18:03:26.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.082+0000 7f4a449a6700 1 Processor -- start 2024-09-13T18:03:26.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.082+0000 7f4a449a6700 1 -- start start 2024-09-13T18:03:26.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a449a6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a40071170 0x7f4a401ef460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:26.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a449a6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4a40150480 0x7f4a400baf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:26.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a449a6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a40150860 0x7f4a400bb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:26.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a449a6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a40153d00 con 0x7f4a40150860 2024-09-13T18:03:26.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a449a6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a40152280 con 0x7f4a40071170 2024-09-13T18:03:26.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a449a6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a401eded0 con 0x7f4a40150480 2024-09-13T18:03:26.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a3dd9b700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4a40150480 msgr2=0x7f4a400baf70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:03:26.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a3dd9b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4a40150480 0x7f4a400baf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:26.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a3ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a40150860 0x7f4a400bb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:26.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a3ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a40150860 0x7f4a400bb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43380/0 (socket says 172.21.15.40:43380) 2024-09-13T18:03:26.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a3ed9d700 1 -- 172.21.15.40:0/1206096550 learned_addr learned my addr 172.21.15.40:0/1206096550 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:26.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a3e59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a40071170 0x7f4a401ef460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:26.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a3ed9d700 1 -- 172.21.15.40:0/1206096550 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4a40150480 msgr2=0x7f4a400baf70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:03:26.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a3ed9d700 1 --2- 172.21.15.40:0/1206096550 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4a40150480 0x7f4a400baf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:26.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a3ed9d700 1 -- 172.21.15.40:0/1206096550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a40071170 msgr2=0x7f4a401ef460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:26.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a3ed9d700 1 --2- 172.21.15.40:0/1206096550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a40071170 0x7f4a401ef460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:26.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a3ed9d700 1 -- 172.21.15.40:0/1206096550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a38009a70 con 0x7f4a40150860 2024-09-13T18:03:26.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a3ed9d700 1 --2- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a40150860 0x7f4a400bb4b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4a400c41a0 tx=0x7f4a38004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:26.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a2effd700 1 -- 172.21.15.40:0/1206096550 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3801b810 con 0x7f4a40150860 2024-09-13T18:03:26.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a2effd700 1 -- 172.21.15.40:0/1206096550 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a38020070 con 0x7f4a40150860 2024-09-13T18:03:26.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a449a6700 1 -- 172.21.15.40:0/1206096550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a401524b0 con 0x7f4a40150860 2024-09-13T18:03:26.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a2effd700 1 -- 172.21.15.40:0/1206096550 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3801bbf0 con 0x7f4a40150860 2024-09-13T18:03:26.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a449a6700 1 -- 172.21.15.40:0/1206096550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a40153460 con 0x7f4a40150860 2024-09-13T18:03:26.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a449a6700 1 --2- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a40153740 0x7f4a401ee340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:26.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a449a6700 1 -- 172.21.15.40:0/1206096550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a40067720 con 0x7f4a40153740 2024-09-13T18:03:26.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a3e59c700 1 --2- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a40153740 0x7f4a401ee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:26.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a3e59c700 1 -- 172.21.15.40:0/1206096550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4a400c6d00 con 0x7f4a40153740 2024-09-13T18:03:26.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a2effd700 1 -- 172.21.15.40:0/1206096550 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4a3801bd50 con 0x7f4a40150860 2024-09-13T18:03:26.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.086+0000 7f4a3e59c700 1 --2- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a40153740 0x7f4a401ee340 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4a30000f60 tx=0x7f4a30001910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:26.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.090+0000 7f4a2effd700 1 --2- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4a20066730 0x7f4a20068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:26.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.090+0000 7f4a2effd700 1 -- 172.21.15.40:0/1206096550 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4a38022070 con 0x7f4a40150860 2024-09-13T18:03:26.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.090+0000 7f4a3dd9b700 1 --2- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4a20066730 0x7f4a20068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:26.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.090+0000 7f4a2effd700 1 -- 172.21.15.40:0/1206096550 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30017070 con 0x7f4a40153740 2024-09-13T18:03:26.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.090+0000 7f4a2effd700 1 -- 172.21.15.40:0/1206096550 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4a30012560 con 0x7f4a40153740 2024-09-13T18:03:26.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.090+0000 7f4a2effd700 1 -- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a40153740 msgr2=0x7f4a401ee340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:26.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.090+0000 7f4a2effd700 1 --2- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a40153740 0x7f4a401ee340 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4a30000f60 tx=0x7f4a30001910 comp rx=0 tx=0).stop 2024-09-13T18:03:26.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.090+0000 7f4a2effd700 1 -- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a40153740 msgr2=0x7f4a401ee340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:03:26.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.090+0000 7f4a2effd700 1 --2- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a40153740 0x7f4a401ee340 secure :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4a30000f60 tx=0x7f4a30001910 comp rx=0 tx=0).stop 2024-09-13T18:03:26.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.090+0000 7f4a3dd9b700 1 --2- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4a20066730 0x7f4a20068bf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4a34001030 tx=0x7f4a34007ae0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:26.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a449a6700 1 --2- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a401ee880 0x7f4a401540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:26.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a449a6700 1 -- 172.21.15.40:0/1206096550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a40156320 con 0x7f4a401ee880 2024-09-13T18:03:26.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a3e59c700 1 --2- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a401ee880 0x7f4a401540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:26.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a3e59c700 1 -- 172.21.15.40:0/1206096550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4a400c5720 con 0x7f4a401ee880 2024-09-13T18:03:26.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a3e59c700 1 --2- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a401ee880 0x7f4a401540b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4a30000f60 tx=0x7f4a30000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:26.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a2effd700 1 -- 172.21.15.40:0/1206096550 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30023070 con 0x7f4a401ee880 2024-09-13T18:03:26.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a2effd700 1 -- 172.21.15.40:0/1206096550 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4939 (secure 0 0 0) 0x7f4a30012d70 con 0x7f4a401ee880 2024-09-13T18:03:26.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a2effd700 1 -- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a401ee880 msgr2=0x7f4a401540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:26.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a2effd700 1 --2- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a401ee880 0x7f4a401540b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4a30000f60 tx=0x7f4a30000ed0 comp rx=0 tx=0).stop 2024-09-13T18:03:26.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a2effd700 1 -- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a401ee880 msgr2=0x7f4a401540b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:03:26.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a2effd700 1 --2- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a401ee880 0x7f4a401540b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:26.111 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:03:26.111 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:03:26.111 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:03:26.111 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:03:26.111 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 102, 2024-09-13T18:03:26.111 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:03:26.111 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:03:26.111 INFO:teuthology.orchestra.run.smithi040.stdout: 1 2024-09-13T18:03:26.111 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:26.111 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:03:26.111 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:03:26.112 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:03:26.112 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:03:26.112 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:26.112 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:26.112 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:26.112 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:26.112 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:26.112 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:26.112 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:26.112 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:26.112 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:26.113 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:26.113 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:26.113 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:03:26.113 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:03:26.113 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:26.113 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:26.113 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:26.113 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:26.113 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:26.114 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:26.114 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:26.114 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:26.114 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:26.114 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:26.114 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:26.114 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.114 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:03:26.114 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:03:26.114 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:03:26.115 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:03:26.115 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:03:26.115 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:03:26.115 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:03:26.115 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:03:26.115 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:03:26.115 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:03:26.115 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:03:26.115 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:03:26.115 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:03:26.115 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:03:26.115 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:03:26.115 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:03:26.116 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:03:26.116 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:26.116 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:26.116 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:26.116 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:26.116 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:26.116 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:26.116 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:26.116 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:26.116 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:26.116 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:26.116 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:26.117 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:03:26.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.117 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:03:26.117 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.117 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:03:26.117 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:03:26.117 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:26.117 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:26.117 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.118 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:26.118 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:03:26.118 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:26.118 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.118 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:26.118 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:03:26.118 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:26.118 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:26.118 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:26.118 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.119 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:03:26.119 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:03:26.119 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:26.119 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:26.119 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:26.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.119 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:03:26.119 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:03:26.119 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:26.119 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:26.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.119 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:26.120 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:03:26.120 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:26.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.120 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.120 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:26.120 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:03:26.120 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:26.121 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:26.121 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:26.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.121 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:03:26.121 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:03:26.121 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:26.122 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:26.122 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:26.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.122 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:03:26.122 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:03:26.122 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:26.122 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:26.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.122 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:26.122 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:03:26.122 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:26.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.123 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:26.123 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:03:26.123 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:26.123 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:26.123 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:26.123 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.123 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:03:26.123 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:03:26.123 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:26.123 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:26.123 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:26.123 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:26.123 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:26.123 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.124 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:03:26.124 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:03:26.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.124 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:26.124 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:26.124 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:03:26.124 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:26.124 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:26.124 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:03:26.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.124 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:26.125 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:26.125 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:03:26.125 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:26.125 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:26.125 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:03:26.125 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.125 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:26.125 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:26.125 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 8 2024-09-13T18:03:26.125 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:26.125 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:26.125 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:03:26.125 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.125 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:26.126 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:26.126 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:03:26.126 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:26.126 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:26.126 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:03:26.126 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.126 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:26.126 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:26.126 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:03:26.126 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:26.126 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:26.127 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.127 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:03:26.127 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:03:26.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a449a6700 1 -- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4a20066730 msgr2=0x7f4a20068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:26.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a449a6700 1 --2- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4a20066730 0x7f4a20068bf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4a34001030 tx=0x7f4a34007ae0 comp rx=0 tx=0).stop 2024-09-13T18:03:26.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a449a6700 1 -- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a40150860 msgr2=0x7f4a400bb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:26.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a3ed9d700 1 -- 172.21.15.40:0/1206096550 reap_dead start 2024-09-13T18:03:26.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a449a6700 1 --2- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a40150860 0x7f4a400bb4b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4a400c41a0 tx=0x7f4a38004af0 comp rx=0 tx=0).stop 2024-09-13T18:03:26.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a449a6700 1 -- 172.21.15.40:0/1206096550 shutdown_connections 2024-09-13T18:03:26.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a449a6700 1 --2- 172.21.15.40:0/1206096550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a40150860 0x7f4a400bb4b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:26.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a449a6700 1 -- 172.21.15.40:0/1206096550 >> 172.21.15.40:0/1206096550 conn(0x7f4a4006a390 msgr2=0x7f4a4014e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:26.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a449a6700 1 -- 172.21.15.40:0/1206096550 shutdown_connections 2024-09-13T18:03:26.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.098+0000 7f4a449a6700 1 -- 172.21.15.40:0/1206096550 wait complete. 2024-09-13T18:03:26.129 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:03:26.130 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-13T18:03:26.130 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:03:26.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a5359e700 1 Processor -- start 2024-09-13T18:03:26.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a5359e700 1 -- start start 2024-09-13T18:03:26.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a5359e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0a5414ff00 0x7f0a54070c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:26.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a5359e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0a54072510 0x7f0a541594d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:26.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a5359e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a54071880 con 0x7f0a54159a10 2024-09-13T18:03:26.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a5359e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a540667e0 con 0x7f0a5414ff00 2024-09-13T18:03:26.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a5359e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a540cbfe0 con 0x7f0a54072510 2024-09-13T18:03:26.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a5259c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0a5414ff00 0x7f0a54070c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:26.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a5259c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0a5414ff00 0x7f0a54070c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43418/0 (socket says 172.21.15.40:43418) 2024-09-13T18:03:26.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a5259c700 1 -- 172.21.15.40:0/4139739702 learned_addr learned my addr 172.21.15.40:0/4139739702 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:26.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a52d9d700 1 --1- 172.21.15.40:0/4139739702 >> v1:172.21.15.40:6789/0 conn(0x7f0a54159a10 0x7f0a540ca7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47956/0 (socket says 172.21.15.40:47956) 2024-09-13T18:03:26.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a49d9b700 1 --2- 172.21.15.40:0/4139739702 >> v2:172.21.15.43:3300/0 conn(0x7f0a54072510 0x7f0a541594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:26.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a5259c700 1 -- 172.21.15.40:0/4139739702 >> v2:172.21.15.43:3300/0 conn(0x7f0a54072510 msgr2=0x7f0a541594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:03:26.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a5259c700 1 --2- 172.21.15.40:0/4139739702 >> v2:172.21.15.43:3300/0 conn(0x7f0a54072510 0x7f0a541594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:26.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a5259c700 1 -- 172.21.15.40:0/4139739702 >> v1:172.21.15.40:6789/0 conn(0x7f0a54159a10 legacy=0x7f0a540ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:26.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a5259c700 1 -- 172.21.15.40:0/4139739702 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a540712c0 con 0x7f0a5414ff00 2024-09-13T18:03:26.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a5259c700 1 --2- 172.21.15.40:0/4139739702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a5414ff00 0x7f0a54070c10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0a54070850 tx=0x7f0a44009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ce3cd456c1beba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:26.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a51d9b700 1 -- 172.21.15.40:0/4139739702 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4400b8a0 con 0x7f0a5414ff00 2024-09-13T18:03:26.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a51d9b700 1 -- 172.21.15.40:0/4139739702 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4400ba00 con 0x7f0a5414ff00 2024-09-13T18:03:26.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a51d9b700 1 -- 172.21.15.40:0/4139739702 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4400bd90 con 0x7f0a5414ff00 2024-09-13T18:03:26.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a5359e700 1 -- 172.21.15.40:0/4139739702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a5414ff00 msgr2=0x7f0a54070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:26.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.242+0000 7f0a5359e700 1 --2- 172.21.15.40:0/4139739702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a5414ff00 0x7f0a54070c10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0a54070850 tx=0x7f0a44009bf0 comp rx=0 tx=0).stop 2024-09-13T18:03:26.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a5359e700 1 -- 172.21.15.40:0/4139739702 shutdown_connections 2024-09-13T18:03:26.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a5359e700 1 --2- 172.21.15.40:0/4139739702 >> v2:172.21.15.43:3300/0 conn(0x7f0a54072510 0x7f0a541594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:26.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a5359e700 1 --2- 172.21.15.40:0/4139739702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a5414ff00 0x7f0a54070c10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:26.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a5359e700 1 -- 172.21.15.40:0/4139739702 >> 172.21.15.40:0/4139739702 conn(0x7f0a5406a390 msgr2=0x7f0a5406a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:26.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a5359e700 1 -- 172.21.15.40:0/4139739702 shutdown_connections 2024-09-13T18:03:26.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a5359e700 1 -- 172.21.15.40:0/4139739702 wait complete. 2024-09-13T18:03:26.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a5359e700 1 Processor -- start 2024-09-13T18:03:26.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a5359e700 1 -- start start 2024-09-13T18:03:26.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a5359e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a54072510 0x7f0a54156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:26.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a5359e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a5414ff00 0x7f0a541eeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:26.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a5359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a54159a10 0x7f0a541551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:26.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a5359e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a541547d0 con 0x7f0a54159a10 2024-09-13T18:03:26.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a5359e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a54156a50 con 0x7f0a54072510 2024-09-13T18:03:26.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a5359e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a54156d20 con 0x7f0a5414ff00 2024-09-13T18:03:26.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a49d9b700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a5414ff00 msgr2=0x7f0a541eeaa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:03:26.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a49d9b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a5414ff00 0x7f0a541eeaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:26.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a52d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a54159a10 0x7f0a541551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:26.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a52d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a54159a10 0x7f0a541551b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43430/0 (socket says 172.21.15.40:43430) 2024-09-13T18:03:26.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a52d9d700 1 -- 172.21.15.40:0/94582879 learned_addr learned my addr 172.21.15.40:0/94582879 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:26.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a5259c700 1 --2- 172.21.15.40:0/94582879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a54072510 0x7f0a54156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:26.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a52d9d700 1 -- 172.21.15.40:0/94582879 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a5414ff00 msgr2=0x7f0a541eeaa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:03:26.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a52d9d700 1 --2- 172.21.15.40:0/94582879 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a5414ff00 0x7f0a541eeaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:26.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a52d9d700 1 -- 172.21.15.40:0/94582879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a54072510 msgr2=0x7f0a54156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:26.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a52d9d700 1 --2- 172.21.15.40:0/94582879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a54072510 0x7f0a54156510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:26.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a52d9d700 1 -- 172.21.15.40:0/94582879 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a44009570 con 0x7f0a54159a10 2024-09-13T18:03:26.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a52d9d700 1 --2- 172.21.15.40:0/94582879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a54159a10 0x7f0a541551b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00cbd0 tx=0x7f0a4c00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:26.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a4b7fe700 1 -- 172.21.15.40:0/94582879 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c00ab00 con 0x7f0a54159a10 2024-09-13T18:03:26.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a4b7fe700 1 -- 172.21.15.40:0/94582879 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4c00ac60 con 0x7f0a54159a10 2024-09-13T18:03:26.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a5359e700 1 -- 172.21.15.40:0/94582879 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a54154d20 con 0x7f0a54159a10 2024-09-13T18:03:26.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a4b7fe700 1 -- 172.21.15.40:0/94582879 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c007070 con 0x7f0a54159a10 2024-09-13T18:03:26.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.246+0000 7f0a5359e700 1 -- 172.21.15.40:0/94582879 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a541ef510 con 0x7f0a54159a10 2024-09-13T18:03:26.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.250+0000 7f0a5359e700 1 --2- 172.21.15.40:0/94582879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a541eefe0 0x7f0a541556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:26.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.250+0000 7f0a5359e700 1 -- 172.21.15.40:0/94582879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a54067670 con 0x7f0a541eefe0 2024-09-13T18:03:26.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.250+0000 7f0a5259c700 1 --2- 172.21.15.40:0/94582879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a541eefe0 0x7f0a541556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:26.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.250+0000 7f0a5259c700 1 -- 172.21.15.40:0/94582879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0a540cb7e0 con 0x7f0a541eefe0 2024-09-13T18:03:26.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.250+0000 7f0a5259c700 1 --2- 172.21.15.40:0/94582879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a541eefe0 0x7f0a541556f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0a54070850 tx=0x7f0a44004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:26.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.250+0000 7f0a4b7fe700 1 -- 172.21.15.40:0/94582879 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0a4c0071d0 con 0x7f0a54159a10 2024-09-13T18:03:26.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.250+0000 7f0a4b7fe700 1 --2- 172.21.15.40:0/94582879 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0a34066710 0x7f0a34068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:26.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.250+0000 7f0a4b7fe700 1 -- 172.21.15.40:0/94582879 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0a4c0840e0 con 0x7f0a54159a10 2024-09-13T18:03:26.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.250+0000 7f0a49d9b700 1 --2- 172.21.15.40:0/94582879 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0a34066710 0x7f0a34068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:26.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.250+0000 7f0a4b7fe700 1 -- 172.21.15.40:0/94582879 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a44020070 con 0x7f0a541eefe0 2024-09-13T18:03:26.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.250+0000 7f0a4b7fe700 1 -- 172.21.15.40:0/94582879 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0a440036c0 con 0x7f0a541eefe0 2024-09-13T18:03:26.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.250+0000 7f0a4b7fe700 1 -- 172.21.15.40:0/94582879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a541eefe0 msgr2=0x7f0a541556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:26.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.250+0000 7f0a4b7fe700 1 --2- 172.21.15.40:0/94582879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a541eefe0 0x7f0a541556f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0a54070850 tx=0x7f0a44004770 comp rx=0 tx=0).stop 2024-09-13T18:03:26.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.250+0000 7f0a4b7fe700 1 -- 172.21.15.40:0/94582879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a541eefe0 msgr2=0x7f0a541556f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:03:26.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.250+0000 7f0a4b7fe700 1 --2- 172.21.15.40:0/94582879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a541eefe0 0x7f0a541556f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:26.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.254+0000 7f0a49d9b700 1 --2- 172.21.15.40:0/94582879 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0a34066710 0x7f0a34068bd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c006f30 tx=0x7f0a3c006ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:26.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.258+0000 7f0a5359e700 1 --2- 172.21.15.40:0/94582879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a54154910 0x7f0a541edc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:26.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.258+0000 7f0a5359e700 1 -- 172.21.15.40:0/94582879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a540cb500 con 0x7f0a54154910 2024-09-13T18:03:26.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.262+0000 7f0a5259c700 1 --2- 172.21.15.40:0/94582879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a54154910 0x7f0a541edc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:26.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.262+0000 7f0a5259c700 1 -- 172.21.15.40:0/94582879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0a540cbac0 con 0x7f0a54154910 2024-09-13T18:03:26.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.262+0000 7f0a5259c700 1 --2- 172.21.15.40:0/94582879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a54154910 0x7f0a541edc00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0a5406b650 tx=0x7f0a44011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:26.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.262+0000 7f0a4b7fe700 1 -- 172.21.15.40:0/94582879 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a44030070 con 0x7f0a54154910 2024-09-13T18:03:26.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.262+0000 7f0a4b7fe700 1 -- 172.21.15.40:0/94582879 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4605 (secure 0 0 0) 0x7f0a44003e70 con 0x7f0a54154910 2024-09-13T18:03:26.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.262+0000 7f0a4b7fe700 1 -- 172.21.15.40:0/94582879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a54154910 msgr2=0x7f0a541edc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:26.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.262+0000 7f0a4b7fe700 1 --2- 172.21.15.40:0/94582879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a54154910 0x7f0a541edc00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0a5406b650 tx=0x7f0a44011000 comp rx=0 tx=0).stop 2024-09-13T18:03:26.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.262+0000 7f0a4b7fe700 1 -- 172.21.15.40:0/94582879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a54154910 msgr2=0x7f0a541edc00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:03:26.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.262+0000 7f0a4b7fe700 1 --2- 172.21.15.40:0/94582879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a54154910 0x7f0a541edc00 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0a5406b650 tx=0x7f0a44011000 comp rx=0 tx=0).stop 2024-09-13T18:03:26.275 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:03:26.275 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:03:26.275 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:03:26.275 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:03:26.275 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 102, 2024-09-13T18:03:26.275 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:03:26.276 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:03:26.276 INFO:teuthology.orchestra.run.smithi040.stdout: 1 2024-09-13T18:03:26.276 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:26.276 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:03:26.276 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:03:26.276 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:03:26.276 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:03:26.276 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:26.276 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:26.276 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:26.276 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:26.277 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:26.277 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:26.277 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:26.277 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:26.277 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:26.277 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:26.277 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:26.277 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:03:26.277 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:03:26.277 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:26.277 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:26.277 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:26.278 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:26.278 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:26.278 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:26.278 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:26.278 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:26.278 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:26.278 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:26.278 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:26.278 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.278 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:03:26.278 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:03:26.278 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:03:26.278 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:03:26.278 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:03:26.279 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:03:26.279 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:03:26.279 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:03:26.279 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:03:26.279 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:03:26.279 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:03:26.279 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:03:26.279 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:03:26.279 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:03:26.279 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:03:26.279 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:03:26.279 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:03:26.279 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:26.279 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:26.279 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:26.280 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:26.280 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:26.280 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:26.280 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:26.280 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:26.280 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:26.280 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:26.280 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:26.280 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:03:26.280 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.280 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:03:26.280 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.280 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:03:26.281 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:03:26.281 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:26.281 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:26.281 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.281 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:26.281 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:03:26.281 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:26.281 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.282 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.282 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:26.282 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:03:26.282 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:26.282 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:26.282 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:26.283 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.283 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:03:26.283 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:03:26.283 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:26.283 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:26.283 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:26.284 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.284 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.284 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:03:26.284 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:03:26.284 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:26.284 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:26.284 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.284 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:26.284 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:03:26.285 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:26.285 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.285 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.285 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:26.285 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:03:26.285 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:26.285 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:26.286 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:26.286 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.286 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:03:26.286 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:03:26.286 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:26.286 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:26.286 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:26.286 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.286 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.287 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:03:26.287 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:03:26.287 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:26.287 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:26.287 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.287 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:26.287 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:03:26.287 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:26.287 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.287 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.287 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:26.287 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:03:26.287 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:26.287 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:26.288 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:26.288 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.288 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:03:26.288 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:03:26.288 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:26.288 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:26.288 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:26.288 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:26.288 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:26.288 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.289 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:03:26.289 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:03:26.289 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.289 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:26.289 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:26.289 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:03:26.289 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:26.289 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:26.289 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:03:26.289 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.289 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:26.289 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:26.289 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:03:26.289 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:26.290 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:26.290 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:03:26.290 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:26.290 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:26.290 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:26.290 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:03:26.290 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:26.290 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:26.290 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:26.290 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:03:26.290 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:03:26.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.262+0000 7f0a5359e700 1 -- 172.21.15.40:0/94582879 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0a34066710 msgr2=0x7f0a34068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:26.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.262+0000 7f0a5359e700 1 --2- 172.21.15.40:0/94582879 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0a34066710 0x7f0a34068bd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c006f30 tx=0x7f0a3c006ec0 comp rx=0 tx=0).stop 2024-09-13T18:03:26.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.262+0000 7f0a5359e700 1 -- 172.21.15.40:0/94582879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a54159a10 msgr2=0x7f0a541551b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:26.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.262+0000 7f0a5359e700 1 --2- 172.21.15.40:0/94582879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a54159a10 0x7f0a541551b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00cbd0 tx=0x7f0a4c00cee0 comp rx=0 tx=0).stop 2024-09-13T18:03:26.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.262+0000 7f0a52d9d700 1 -- 172.21.15.40:0/94582879 reap_dead start 2024-09-13T18:03:26.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.262+0000 7f0a5359e700 1 -- 172.21.15.40:0/94582879 shutdown_connections 2024-09-13T18:03:26.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.262+0000 7f0a5359e700 1 --2- 172.21.15.40:0/94582879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a54159a10 0x7f0a541551b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:26.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.262+0000 7f0a5359e700 1 -- 172.21.15.40:0/94582879 >> 172.21.15.40:0/94582879 conn(0x7f0a5406a390 msgr2=0x7f0a540c9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:26.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.262+0000 7f0a5359e700 1 -- 172.21.15.40:0/94582879 shutdown_connections 2024-09-13T18:03:26.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:26.262+0000 7f0a5359e700 1 -- 172.21.15.40:0/94582879 wait complete. 2024-09-13T18:03:26.293 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:03:26.293 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:03:36.005 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:03:36.007 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:03:46.294 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-13T18:03:46.294 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-13T18:03:46.294 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T18:03:46.294 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T18:03:46.297 INFO:tasks.ceph.mon.c:Started 2024-09-13T18:03:46.297 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:03:46.297 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:03:46.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.406+0000 7f7125c73700 1 Processor -- start 2024-09-13T18:03:46.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.406+0000 7f7125c73700 1 -- start start 2024-09-13T18:03:46.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.406+0000 7f7125c73700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f71180d56f0 0x7f7118045cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:46.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.406+0000 7f7125c73700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7118008cc0 0x7f71180de920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:46.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.406+0000 7f7125c73700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711804bc00 con 0x7f71180dee60 2024-09-13T18:03:46.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.406+0000 7f7125c73700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71180d5ad0 con 0x7f71180d56f0 2024-09-13T18:03:46.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.406+0000 7f7125c73700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7118008830 con 0x7f7118008cc0 2024-09-13T18:03:46.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.406+0000 7f7125472700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f71180dee60 0x7f71180db330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36792/0 (socket says 172.21.15.40:36792) 2024-09-13T18:03:46.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7125472700 1 -- 172.21.15.40:0/2786829502 learned_addr learned my addr 172.21.15.40:0/2786829502 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:03:46.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f711ffff700 1 --2- 172.21.15.40:0/2786829502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7118008cc0 0x7f71180de920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:46.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7124c71700 1 --2- 172.21.15.40:0/2786829502 >> v2:172.21.15.43:3300/0 conn(0x7f71180d56f0 0x7f7118045cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:46.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f711f7fe700 1 -- 172.21.15.40:0/2786829502 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245820595 0 0) 0x7f711804bc00 con 0x7f71180dee60 2024-09-13T18:03:46.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f711f7fe700 1 -- 172.21.15.40:0/2786829502 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71100035d0 con 0x7f71180dee60 2024-09-13T18:03:46.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f711ffff700 1 -- 172.21.15.40:0/2786829502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7118008cc0 msgr2=0x7f71180de920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:03:46.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f711ffff700 1 -- 172.21.15.40:0/2786829502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7118008cc0 msgr2=0x7f71180de920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:03:46.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f711ffff700 1 --2- 172.21.15.40:0/2786829502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7118008cc0 0x7f71180de920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:46.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f711ffff700 1 --2- 172.21.15.40:0/2786829502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7118008cc0 0x7f71180de920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:46.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7124c71700 1 -- 172.21.15.40:0/2786829502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7118008cc0 msgr2=0x7f71180de920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:03:46.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7124c71700 1 --2- 172.21.15.40:0/2786829502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7118008cc0 0x7f71180de920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:46.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7124c71700 1 -- 172.21.15.40:0/2786829502 >> v1:172.21.15.40:6789/0 conn(0x7f71180dee60 legacy=0x7f71180db330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7124c71700 1 -- 172.21.15.40:0/2786829502 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711804c0d0 con 0x7f71180d56f0 2024-09-13T18:03:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7124c71700 1 --2- 172.21.15.40:0/2786829502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71180d56f0 0x7f7118045cf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7114009850 tx=0x7f7114009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faffcead74b85a40 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f711f7fe700 1 -- 172.21.15.40:0/2786829502 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7114004230 con 0x7f71180d56f0 2024-09-13T18:03:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f711f7fe700 1 -- 172.21.15.40:0/2786829502 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7114004390 con 0x7f71180d56f0 2024-09-13T18:03:46.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f711f7fe700 1 -- 172.21.15.40:0/2786829502 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71140036c0 con 0x7f71180d56f0 2024-09-13T18:03:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7125c73700 1 -- 172.21.15.40:0/2786829502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71180d56f0 msgr2=0x7f7118045cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7125c73700 1 --2- 172.21.15.40:0/2786829502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71180d56f0 0x7f7118045cf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7114009850 tx=0x7f7114009b60 comp rx=0 tx=0).stop 2024-09-13T18:03:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7125c73700 1 -- 172.21.15.40:0/2786829502 shutdown_connections 2024-09-13T18:03:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7125c73700 1 --2- 172.21.15.40:0/2786829502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7118008cc0 0x7f71180de920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7125c73700 1 --2- 172.21.15.40:0/2786829502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71180d56f0 0x7f7118045cf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7125c73700 1 -- 172.21.15.40:0/2786829502 >> 172.21.15.40:0/2786829502 conn(0x7f7118050e10 msgr2=0x7f7118051220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7125c73700 1 -- 172.21.15.40:0/2786829502 shutdown_connections 2024-09-13T18:03:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7125c73700 1 -- 172.21.15.40:0/2786829502 wait complete. 2024-09-13T18:03:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7125c73700 1 Processor -- start 2024-09-13T18:03:46.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7125c73700 1 -- start start 2024-09-13T18:03:46.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7125c73700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7118008cc0 0x7f71180d7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:46.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7125c73700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71180d56f0 0x7f7118177c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:46.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7125c73700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71180dee60 0x7f7118035650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:46.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7125c73700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7118178910 con 0x7f7118008cc0 2024-09-13T18:03:46.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7125c73700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7118178630 con 0x7f71180dee60 2024-09-13T18:03:46.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.410+0000 7f7125c73700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71180d6ce0 con 0x7f71180d56f0 2024-09-13T18:03:46.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f7124c71700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7118008cc0 0x7f71180d7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:46.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f7125472700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71180dee60 0x7f7118035650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:46.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f7124c71700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7118008cc0 0x7f71180d7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46228/0 (socket says 172.21.15.40:46228) 2024-09-13T18:03:46.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f7125472700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71180dee60 0x7f7118035650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:50402/0 (socket says 172.21.15.40:50402) 2024-09-13T18:03:46.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f7124c71700 1 -- 172.21.15.40:0/488986468 learned_addr learned my addr 172.21.15.40:0/488986468 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:46.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f711ffff700 1 --2- 172.21.15.40:0/488986468 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71180d56f0 0x7f7118177c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:46.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f711ffff700 1 -- 172.21.15.40:0/488986468 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71180d56f0 msgr2=0x7f7118177c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:03:46.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f711ffff700 1 -- 172.21.15.40:0/488986468 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71180d56f0 msgr2=0x7f7118177c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:03:46.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f711ffff700 1 --2- 172.21.15.40:0/488986468 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71180d56f0 0x7f7118177c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:46.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f711ffff700 1 --2- 172.21.15.40:0/488986468 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71180d56f0 0x7f7118177c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:46.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f7124c71700 1 -- 172.21.15.40:0/488986468 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71180d56f0 msgr2=0x7f7118177c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:03:46.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f7124c71700 1 --2- 172.21.15.40:0/488986468 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71180d56f0 0x7f7118177c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:46.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f7124c71700 1 -- 172.21.15.40:0/488986468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71180dee60 msgr2=0x7f7118035650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f7124c71700 1 --2- 172.21.15.40:0/488986468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71180dee60 0x7f7118035650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f7124c71700 1 -- 172.21.15.40:0/488986468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71140094b0 con 0x7f7118008cc0 2024-09-13T18:03:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f7125472700 1 --2- 172.21.15.40:0/488986468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71180dee60 0x7f7118035650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:03:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f7124c71700 1 --2- 172.21.15.40:0/488986468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7118008cc0 0x7f71180d7f80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f71140090b0 tx=0x7f7114004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f711d7fa700 1 -- 172.21.15.40:0/488986468 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7114006ea0 con 0x7f7118008cc0 2024-09-13T18:03:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f711d7fa700 1 -- 172.21.15.40:0/488986468 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711401f070 con 0x7f7118008cc0 2024-09-13T18:03:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f7125c73700 1 -- 172.21.15.40:0/488986468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71180d6fc0 con 0x7f7118008cc0 2024-09-13T18:03:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f7125c73700 1 -- 172.21.15.40:0/488986468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71180d7580 con 0x7f7118008cc0 2024-09-13T18:03:46.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f711d7fa700 1 -- 172.21.15.40:0/488986468 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7114011870 con 0x7f7118008cc0 2024-09-13T18:03:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f7125c73700 1 -- 172.21.15.40:0/488986468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7100005310 con 0x7f7118008cc0 2024-09-13T18:03:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.414+0000 7f711d7fa700 1 -- 172.21.15.40:0/488986468 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7114021030 con 0x7f7118008cc0 2024-09-13T18:03:46.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.418+0000 7f711d7fa700 1 --2- 172.21.15.40:0/488986468 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7104066760 0x7f7104068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:46.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.418+0000 7f711d7fa700 1 -- 172.21.15.40:0/488986468 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7114085390 con 0x7f7118008cc0 2024-09-13T18:03:46.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.418+0000 7f711ffff700 1 --2- 172.21.15.40:0/488986468 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7104066760 0x7f7104068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:46.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.418+0000 7f711ffff700 1 --2- 172.21.15.40:0/488986468 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7104066760 0x7f7104068c20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7118178ae0 tx=0x7f710c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:46.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.422+0000 7f711d7fa700 1 -- 172.21.15.40:0/488986468 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f71140522b0 con 0x7f7118008cc0 2024-09-13T18:03:46.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.674+0000 7f7125c73700 1 -- 172.21.15.40:0/488986468 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f7100005600 con 0x7f7118008cc0 2024-09-13T18:03:46.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.674+0000 7f711d7fa700 1 -- 172.21.15.40:0/488986468 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f71140558d0 con 0x7f7118008cc0 2024-09-13T18:03:46.685 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":102,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:03:46.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.678+0000 7f7125c73700 1 -- 172.21.15.40:0/488986468 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7104066760 msgr2=0x7f7104068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.678+0000 7f7125c73700 1 --2- 172.21.15.40:0/488986468 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7104066760 0x7f7104068c20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7118178ae0 tx=0x7f710c008040 comp rx=0 tx=0).stop 2024-09-13T18:03:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.678+0000 7f7125c73700 1 -- 172.21.15.40:0/488986468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7118008cc0 msgr2=0x7f71180d7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.678+0000 7f7125c73700 1 --2- 172.21.15.40:0/488986468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7118008cc0 0x7f71180d7f80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f71140090b0 tx=0x7f7114004770 comp rx=0 tx=0).stop 2024-09-13T18:03:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.678+0000 7f7125c73700 1 -- 172.21.15.40:0/488986468 shutdown_connections 2024-09-13T18:03:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.678+0000 7f7125c73700 1 --2- 172.21.15.40:0/488986468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71180dee60 0x7f7118035650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.678+0000 7f7125c73700 1 --2- 172.21.15.40:0/488986468 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7104066760 0x7f7104068c20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.678+0000 7f7125c73700 1 --2- 172.21.15.40:0/488986468 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71180d56f0 0x7f7118177c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.678+0000 7f7125c73700 1 --2- 172.21.15.40:0/488986468 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7118008cc0 0x7f71180d7f80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.678+0000 7f7125c73700 1 -- 172.21.15.40:0/488986468 >> 172.21.15.40:0/488986468 conn(0x7f7118050e10 msgr2=0x7f71180470a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.678+0000 7f7125c73700 1 -- 172.21.15.40:0/488986468 shutdown_connections 2024-09-13T18:03:46.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:46.678+0000 7f7125c73700 1 -- 172.21.15.40:0/488986468 wait complete. 2024-09-13T18:03:49.701 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:03:49.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:49.810+0000 7fdf56060700 1 Processor -- start 2024-09-13T18:03:49.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:49.810+0000 7fdf56060700 1 -- start start 2024-09-13T18:03:49.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:49.810+0000 7fdf56060700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 0x7fdf50071670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:49.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:49.810+0000 7fdf56060700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 0x7fdf50070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:49.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:49.810+0000 7fdf56060700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf500c8980 con 0x7fdf50070870 2024-09-13T18:03:49.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:49.810+0000 7fdf56060700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf500c63b0 con 0x7fdf50071bb0 2024-09-13T18:03:49.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:49.810+0000 7fdf56060700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf500c6650 con 0x7fdf5014f980 2024-09-13T18:03:49.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:49.810+0000 7fdf5485d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 0x7fdf50070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:49.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:49.810+0000 7fdf5485d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 0x7fdf50070330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46242/0 (socket says 172.21.15.40:46242) 2024-09-13T18:03:49.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:49.810+0000 7fdf5485d700 1 -- 172.21.15.40:0/2145063784 learned_addr learned my addr 172.21.15.40:0/2145063784 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:49.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:49.814+0000 7fdf5485d700 1 -- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 msgr2=0x7fdf50070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:03:49.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:49.814+0000 7fdf5485d700 1 -- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 msgr2=0x7fdf50070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:03:49.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:49.814+0000 7fdf5485d700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 0x7fdf50070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:49.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:49.814+0000 7fdf5505e700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 0x7fdf50071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:49.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:49.814+0000 7fdf5485d700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 0x7fdf50070330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:49.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:49.814+0000 7fdf5505e700 1 -- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 msgr2=0x7fdf50071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:03:49.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:49.814+0000 7fdf5505e700 1 -- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 msgr2=0x7fdf50071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:03:49.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:49.814+0000 7fdf5505e700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 0x7fdf50071670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:49.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:49.814+0000 7fdf5505e700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 0x7fdf50071670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:50.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:50.018+0000 7fdf5485d700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 0x7fdf50070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:50.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:50.018+0000 7fdf5505e700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 0x7fdf50071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:50.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:50.018+0000 7fdf5485d700 1 -- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 msgr2=0x7fdf50070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:03:50.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:50.018+0000 7fdf5485d700 1 -- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 msgr2=0x7fdf50070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:03:50.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:50.018+0000 7fdf5485d700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 0x7fdf50070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:50.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:50.018+0000 7fdf5485d700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 0x7fdf50070330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:03:50.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:50.018+0000 7fdf5505e700 1 -- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 msgr2=0x7fdf50071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:03:50.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:50.018+0000 7fdf5505e700 1 -- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 msgr2=0x7fdf50071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:03:50.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:50.018+0000 7fdf5505e700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 0x7fdf50071670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:50.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:50.018+0000 7fdf5505e700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 0x7fdf50071670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:03:50.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:50.422+0000 7fdf5485d700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 0x7fdf50070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:50.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:50.422+0000 7fdf5505e700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 0x7fdf50071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:50.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:50.422+0000 7fdf5485d700 1 -- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 msgr2=0x7fdf50070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:03:50.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:50.422+0000 7fdf5485d700 1 -- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 msgr2=0x7fdf50070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:03:50.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:50.422+0000 7fdf5485d700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 0x7fdf50070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:50.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:50.422+0000 7fdf5505e700 1 -- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 msgr2=0x7fdf50071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:03:50.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:50.422+0000 7fdf5505e700 1 -- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 msgr2=0x7fdf50071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:03:50.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:50.422+0000 7fdf5485d700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 0x7fdf50070330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:03:50.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:50.422+0000 7fdf5505e700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 0x7fdf50071670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:50.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:50.422+0000 7fdf5505e700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 0x7fdf50071670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:03:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:51.226+0000 7fdf5485d700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 0x7fdf50070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:51.226+0000 7fdf5485d700 1 -- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 msgr2=0x7fdf50070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:03:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:51.226+0000 7fdf5485d700 1 -- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 msgr2=0x7fdf50070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:03:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:51.226+0000 7fdf5485d700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 0x7fdf50070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:51.226+0000 7fdf5485d700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 0x7fdf50070330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:03:51.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:51.226+0000 7fdf5505e700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 0x7fdf50071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:51.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:51.226+0000 7fdf5505e700 1 -- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 msgr2=0x7fdf50071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:03:51.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:51.226+0000 7fdf5505e700 1 -- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 msgr2=0x7fdf50071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:03:51.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:51.226+0000 7fdf5505e700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 0x7fdf50071670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:51.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:51.226+0000 7fdf5505e700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 0x7fdf50071670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:03:52.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.042+0000 7fdf47fff700 1 -- 172.21.15.40:0/2145063784 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269693712 0 0) 0x7fdf500c8980 con 0x7fdf50070870 2024-09-13T18:03:52.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.042+0000 7fdf47fff700 1 -- 172.21.15.40:0/2145063784 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf3c0035d0 con 0x7fdf50070870 2024-09-13T18:03:52.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.042+0000 7fdf47fff700 1 -- 172.21.15.40:0/2145063784 <== mon.1 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3976308077 0 0) 0x7fdf3c0035d0 con 0x7fdf50070870 2024-09-13T18:03:52.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.042+0000 7fdf47fff700 1 -- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 msgr2=0x7fdf50071670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:03:52.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.042+0000 7fdf47fff700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.43:3300/0 conn(0x7fdf5014f980 0x7fdf50071670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:52.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.042+0000 7fdf47fff700 1 -- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 msgr2=0x7fdf50070330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:03:52.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.042+0000 7fdf47fff700 1 --2- 172.21.15.40:0/2145063784 >> v2:172.21.15.40:3300/0 conn(0x7fdf50071bb0 0x7fdf50070330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:52.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.042+0000 7fdf47fff700 1 -- 172.21.15.40:0/2145063784 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf500c60e0 con 0x7fdf50070870 2024-09-13T18:03:52.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.042+0000 7fdf47fff700 1 -- 172.21.15.40:0/2145063784 <== mon.1 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fdf480019f0 con 0x7fdf50070870 2024-09-13T18:03:52.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.042+0000 7fdf47fff700 1 -- 172.21.15.40:0/2145063784 >> v1:172.21.15.43:6789/0 conn(0x7fdf50070870 legacy=0x7fdf500c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:52.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.042+0000 7fdf47fff700 1 --2- 172.21.15.40:0/2145063784 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf3c003a20 0x7fdf3c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:52.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.042+0000 7fdf47fff700 1 --2- 172.21.15.40:0/2145063784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c004870 0x7fdf3c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:52.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.042+0000 7fdf47fff700 1 --2- 172.21.15.40:0/2145063784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf3c005230 0x7fdf3c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:52.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.042+0000 7fdf47fff700 1 -- 172.21.15.40:0/2145063784 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf500c63b0 con 0x7fdf3c005230 2024-09-13T18:03:52.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.042+0000 7fdf47fff700 1 -- 172.21.15.40:0/2145063784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf500c6650 con 0x7fdf3c004870 2024-09-13T18:03:52.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.042+0000 7fdf47fff700 1 -- 172.21.15.40:0/2145063784 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c00c130 con 0x7fdf3c003a20 2024-09-13T18:03:52.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.042+0000 7fdf5585f700 1 --2- 172.21.15.40:0/2145063784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf3c005230 0x7fdf3c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:52.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.046+0000 7fdf5485d700 1 --2- 172.21.15.40:0/2145063784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c004870 0x7fdf3c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:52.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.046+0000 7fdf5505e700 1 --2- 172.21.15.40:0/2145063784 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf3c003a20 0x7fdf3c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:52.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.046+0000 7fdf5505e700 1 -- 172.21.15.40:0/2145063784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c004870 msgr2=0x7fdf3c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:03:52.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.046+0000 7fdf5505e700 1 --2- 172.21.15.40:0/2145063784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c004870 0x7fdf3c004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:52.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.046+0000 7fdf5505e700 1 -- 172.21.15.40:0/2145063784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf3c005230 msgr2=0x7fdf3c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:03:52.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.046+0000 7fdf5505e700 1 --2- 172.21.15.40:0/2145063784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf3c005230 0x7fdf3c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:52.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.046+0000 7fdf5505e700 1 -- 172.21.15.40:0/2145063784 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c0035d0 con 0x7fdf3c003a20 2024-09-13T18:03:52.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.046+0000 7fdf5485d700 1 --2- 172.21.15.40:0/2145063784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf3c004870 0x7fdf3c004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:03:52.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.046+0000 7fdf5585f700 1 -- 172.21.15.40:0/2145063784 reap_dead start 2024-09-13T18:03:52.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.046+0000 7fdf5505e700 1 --2- 172.21.15.40:0/2145063784 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf3c003a20 0x7fdf3c003e40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c012730 tx=0x7fdf3c0058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a1efab6e74180f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.046+0000 7fdf47fff700 1 -- 172.21.15.40:0/2145063784 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c019ac0 con 0x7fdf3c003a20 2024-09-13T18:03:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.046+0000 7fdf47fff700 1 -- 172.21.15.40:0/2145063784 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4c01d070 con 0x7fdf3c003a20 2024-09-13T18:03:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.046+0000 7fdf47fff700 1 -- 172.21.15.40:0/2145063784 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c019e80 con 0x7fdf3c003a20 2024-09-13T18:03:52.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.046+0000 7fdf56060700 1 -- 172.21.15.40:0/2145063784 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf3c003a20 msgr2=0x7fdf3c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:52.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.046+0000 7fdf56060700 1 --2- 172.21.15.40:0/2145063784 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf3c003a20 0x7fdf3c003e40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c012730 tx=0x7fdf3c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:03:52.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.046+0000 7fdf56060700 1 -- 172.21.15.40:0/2145063784 shutdown_connections 2024-09-13T18:03:52.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.046+0000 7fdf56060700 1 --2- 172.21.15.40:0/2145063784 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf3c003a20 0x7fdf3c003e40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:52.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.046+0000 7fdf56060700 1 -- 172.21.15.40:0/2145063784 >> 172.21.15.40:0/2145063784 conn(0x7fdf500cb2e0 msgr2=0x7fdf500cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:52.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.046+0000 7fdf56060700 1 -- 172.21.15.40:0/2145063784 shutdown_connections 2024-09-13T18:03:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.046+0000 7fdf56060700 1 -- 172.21.15.40:0/2145063784 wait complete. 2024-09-13T18:03:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf56060700 1 Processor -- start 2024-09-13T18:03:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf56060700 1 -- start start 2024-09-13T18:03:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf56060700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf3c003a20 0x7fdf501eb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf56060700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf50157e00 0x7fdf50158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf56060700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf50156b80 0x7fdf50156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf56060700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf501548e0 con 0x7fdf50157e00 2024-09-13T18:03:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf56060700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf50157520 con 0x7fdf50156b80 2024-09-13T18:03:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf56060700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf50159580 con 0x7fdf3c003a20 2024-09-13T18:03:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf5485d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf50157e00 0x7fdf50158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf5485d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf50157e00 0x7fdf50158280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48444/0 (socket says 172.21.15.40:48444) 2024-09-13T18:03:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf5485d700 1 -- 172.21.15.40:0/2848762942 learned_addr learned my addr 172.21.15.40:0/2848762942 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf5505e700 1 --2- 172.21.15.40:0/2848762942 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf3c003a20 0x7fdf501eb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf5585f700 1 --2- 172.21.15.40:0/2848762942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf50156b80 0x7fdf50156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf5485d700 1 -- 172.21.15.40:0/2848762942 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf3c003a20 msgr2=0x7fdf501eb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf5485d700 1 --2- 172.21.15.40:0/2848762942 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf3c003a20 0x7fdf501eb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf5485d700 1 -- 172.21.15.40:0/2848762942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf50156b80 msgr2=0x7fdf50156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf5485d700 1 --2- 172.21.15.40:0/2848762942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf50156b80 0x7fdf50156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf5485d700 1 -- 172.21.15.40:0/2848762942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf50158ce0 con 0x7fdf50157e00 2024-09-13T18:03:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf5485d700 1 --2- 172.21.15.40:0/2848762942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf50157e00 0x7fdf50158280 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdf40004690 tx=0x7fdf400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf45ffb700 1 -- 172.21.15.40:0/2848762942 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf40019070 con 0x7fdf50157e00 2024-09-13T18:03:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf45ffb700 1 -- 172.21.15.40:0/2848762942 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf40014600 con 0x7fdf50157e00 2024-09-13T18:03:52.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf56060700 1 -- 172.21.15.40:0/2848762942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf50158fc0 con 0x7fdf50157e00 2024-09-13T18:03:52.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf56060700 1 -- 172.21.15.40:0/2848762942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf501589e0 con 0x7fdf50157e00 2024-09-13T18:03:52.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.050+0000 7fdf45ffb700 1 -- 172.21.15.40:0/2848762942 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf400149b0 con 0x7fdf50157e00 2024-09-13T18:03:52.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.054+0000 7fdf56060700 1 -- 172.21.15.40:0/2848762942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5014fd60 con 0x7fdf50157e00 2024-09-13T18:03:52.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.058+0000 7fdf45ffb700 1 -- 172.21.15.40:0/2848762942 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdf40010040 con 0x7fdf50157e00 2024-09-13T18:03:52.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.058+0000 7fdf45ffb700 1 --2- 172.21.15.40:0/2848762942 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf34066730 0x7fdf34068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:52.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.058+0000 7fdf45ffb700 1 -- 172.21.15.40:0/2848762942 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdf40084df0 con 0x7fdf50157e00 2024-09-13T18:03:52.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.058+0000 7fdf5505e700 1 --2- 172.21.15.40:0/2848762942 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf34066730 0x7fdf34068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:52.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.058+0000 7fdf5505e700 1 --2- 172.21.15.40:0/2848762942 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf34066730 0x7fdf34068bf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c000c00 tx=0x7fdf4c00e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:52.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.062+0000 7fdf45ffb700 1 -- 172.21.15.40:0/2848762942 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdf40051e10 con 0x7fdf50157e00 2024-09-13T18:03:52.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.314+0000 7fdf56060700 1 -- 172.21.15.40:0/2848762942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdf50070650 con 0x7fdf50157e00 2024-09-13T18:03:52.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.318+0000 7fdf45ffb700 1 -- 172.21.15.40:0/2848762942 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7fdf40055430 con 0x7fdf50157e00 2024-09-13T18:03:52.328 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":106,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:03:52.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.318+0000 7fdf56060700 1 -- 172.21.15.40:0/2848762942 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf34066730 msgr2=0x7fdf34068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:52.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.318+0000 7fdf56060700 1 --2- 172.21.15.40:0/2848762942 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf34066730 0x7fdf34068bf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c000c00 tx=0x7fdf4c00e000 comp rx=0 tx=0).stop 2024-09-13T18:03:52.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.318+0000 7fdf56060700 1 -- 172.21.15.40:0/2848762942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf50157e00 msgr2=0x7fdf50158280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:52.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.318+0000 7fdf56060700 1 --2- 172.21.15.40:0/2848762942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf50157e00 0x7fdf50158280 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdf40004690 tx=0x7fdf400046c0 comp rx=0 tx=0).stop 2024-09-13T18:03:52.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.318+0000 7fdf56060700 1 -- 172.21.15.40:0/2848762942 shutdown_connections 2024-09-13T18:03:52.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.318+0000 7fdf56060700 1 --2- 172.21.15.40:0/2848762942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf50156b80 0x7fdf50156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:52.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.318+0000 7fdf56060700 1 --2- 172.21.15.40:0/2848762942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf50157e00 0x7fdf50158280 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:52.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.318+0000 7fdf56060700 1 --2- 172.21.15.40:0/2848762942 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdf34066730 0x7fdf34068bf0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:52.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.318+0000 7fdf56060700 1 --2- 172.21.15.40:0/2848762942 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf3c003a20 0x7fdf501eb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:52.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.318+0000 7fdf56060700 1 -- 172.21.15.40:0/2848762942 >> 172.21.15.40:0/2848762942 conn(0x7fdf500cb2e0 msgr2=0x7fdf500c95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:52.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.318+0000 7fdf56060700 1 -- 172.21.15.40:0/2848762942 shutdown_connections 2024-09-13T18:03:52.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:52.318+0000 7fdf56060700 1 -- 172.21.15.40:0/2848762942 wait complete. 2024-09-13T18:03:55.344 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:03:55.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.458+0000 7f3e73fff700 1 Processor -- start 2024-09-13T18:03:55.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.458+0000 7f3e73fff700 1 -- start start 2024-09-13T18:03:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.458+0000 7f3e73fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3e740c9fd0 0x7f3e740c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.458+0000 7f3e73fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e7415a870 0x7f3e74150f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e73fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e74157280 con 0x7f3e740c9bf0 2024-09-13T18:03:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e73fff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e740c8f50 con 0x7f3e740c9fd0 2024-09-13T18:03:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e73fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e740c9230 con 0x7f3e7415a870 2024-09-13T18:03:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e72ffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3e740c9bf0 0x7f3e74157cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42530/0 (socket says 172.21.15.40:42530) 2024-09-13T18:03:55.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e72ffd700 1 -- 172.21.15.40:0/3808249068 learned_addr learned my addr 172.21.15.40:0/3808249068 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:03:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e737fe700 1 --2- 172.21.15.40:0/3808249068 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e7415a870 0x7f3e74150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e727fc700 1 --2- 172.21.15.40:0/3808249068 >> v2:172.21.15.43:3300/0 conn(0x7f3e740c9fd0 0x7f3e740c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e71ffb700 1 -- 172.21.15.40:0/3808249068 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 171452666 0 0) 0x7f3e74157280 con 0x7f3e740c9bf0 2024-09-13T18:03:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e71ffb700 1 -- 172.21.15.40:0/3808249068 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e5c0035d0 con 0x7f3e740c9bf0 2024-09-13T18:03:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e737fe700 1 -- 172.21.15.40:0/3808249068 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e7415a870 msgr2=0x7f3e74150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:03:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e737fe700 1 -- 172.21.15.40:0/3808249068 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e7415a870 msgr2=0x7f3e74150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:03:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e737fe700 1 --2- 172.21.15.40:0/3808249068 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e7415a870 0x7f3e74150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e737fe700 1 --2- 172.21.15.40:0/3808249068 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e7415a870 0x7f3e74150f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e71ffb700 1 -- 172.21.15.40:0/3808249068 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3684820495 0 0) 0x7f3e5c0035d0 con 0x7f3e740c9bf0 2024-09-13T18:03:55.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e71ffb700 1 -- 172.21.15.40:0/3808249068 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e7415a870 msgr2=0x7f3e74150f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:03:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e71ffb700 1 --2- 172.21.15.40:0/3808249068 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e7415a870 0x7f3e74150f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e71ffb700 1 -- 172.21.15.40:0/3808249068 >> v2:172.21.15.43:3300/0 conn(0x7f3e740c9fd0 msgr2=0x7f3e740c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:03:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e71ffb700 1 --2- 172.21.15.40:0/3808249068 >> v2:172.21.15.43:3300/0 conn(0x7f3e740c9fd0 0x7f3e740c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e71ffb700 1 -- 172.21.15.40:0/3808249068 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e740c9510 con 0x7f3e740c9bf0 2024-09-13T18:03:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e71ffb700 1 -- 172.21.15.40:0/3808249068 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f3e64001670 con 0x7f3e740c9bf0 2024-09-13T18:03:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e71ffb700 1 -- 172.21.15.40:0/3808249068 >> v1:172.21.15.40:6789/0 conn(0x7f3e740c9bf0 legacy=0x7f3e74157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e71ffb700 1 --2- 172.21.15.40:0/3808249068 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e7415a870 0x7f3e5c003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e71ffb700 1 --2- 172.21.15.40:0/3808249068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e5c003fa0 0x7f3e5c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e71ffb700 1 --2- 172.21.15.40:0/3808249068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e5c004db0 0x7f3e5c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e71ffb700 1 -- 172.21.15.40:0/3808249068 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e740c8f50 con 0x7f3e5c003fa0 2024-09-13T18:03:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e71ffb700 1 -- 172.21.15.40:0/3808249068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e740c9230 con 0x7f3e5c004db0 2024-09-13T18:03:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e71ffb700 1 -- 172.21.15.40:0/3808249068 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c00bb10 con 0x7f3e7415a870 2024-09-13T18:03:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e727fc700 1 --2- 172.21.15.40:0/3808249068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e5c003fa0 0x7f3e5c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e737fe700 1 --2- 172.21.15.40:0/3808249068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e5c004db0 0x7f3e5c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:55.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e72ffd700 1 --2- 172.21.15.40:0/3808249068 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e7415a870 0x7f3e5c003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e727fc700 1 -- 172.21.15.40:0/3808249068 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e7415a870 msgr2=0x7f3e5c003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:03:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e727fc700 1 --2- 172.21.15.40:0/3808249068 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e7415a870 0x7f3e5c003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e727fc700 1 -- 172.21.15.40:0/3808249068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e5c004db0 msgr2=0x7f3e5c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:03:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e727fc700 1 --2- 172.21.15.40:0/3808249068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e5c004db0 0x7f3e5c005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e727fc700 1 -- 172.21.15.40:0/3808249068 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5c0035d0 con 0x7f3e5c003fa0 2024-09-13T18:03:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e737fe700 1 --2- 172.21.15.40:0/3808249068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e5c004db0 0x7f3e5c005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:03:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e727fc700 1 --2- 172.21.15.40:0/3808249068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e5c003fa0 0x7f3e5c004870 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009b40 tx=0x7f3e5c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba92dff21710a02b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e71ffb700 1 -- 172.21.15.40:0/3808249068 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6800e6c0 con 0x7f3e5c003fa0 2024-09-13T18:03:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e71ffb700 1 -- 172.21.15.40:0/3808249068 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6800e820 con 0x7f3e5c003fa0 2024-09-13T18:03:55.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e71ffb700 1 -- 172.21.15.40:0/3808249068 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6800eb90 con 0x7f3e5c003fa0 2024-09-13T18:03:55.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e73fff700 1 -- 172.21.15.40:0/3808249068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e5c003fa0 msgr2=0x7f3e5c004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:55.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e73fff700 1 --2- 172.21.15.40:0/3808249068 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e5c003fa0 0x7f3e5c004870 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009b40 tx=0x7f3e5c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e737fe700 1 -- 172.21.15.40:0/3808249068 reap_dead start 2024-09-13T18:03:55.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e73fff700 1 -- 172.21.15.40:0/3808249068 shutdown_connections 2024-09-13T18:03:55.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e73fff700 1 -- 172.21.15.40:0/3808249068 >> 172.21.15.40:0/3808249068 conn(0x7f3e74066090 msgr2=0x7f3e74066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:55.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e73fff700 1 -- 172.21.15.40:0/3808249068 shutdown_connections 2024-09-13T18:03:55.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.462+0000 7f3e73fff700 1 -- 172.21.15.40:0/3808249068 wait complete. 2024-09-13T18:03:55.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e73fff700 1 Processor -- start 2024-09-13T18:03:55.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e73fff700 1 -- start start 2024-09-13T18:03:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e73fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e74148ea0 0x7f3e74151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e73fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e74159af0 0x7f3e74153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e73fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e74152fd0 0x7f3e7414fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e73fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e74155450 con 0x7f3e74148ea0 2024-09-13T18:03:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e73fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7415a6a0 con 0x7f3e74152fd0 2024-09-13T18:03:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e73fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e74154ed0 con 0x7f3e74159af0 2024-09-13T18:03:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e72ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e74148ea0 0x7f3e74151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e72ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e74148ea0 0x7f3e74151b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48452/0 (socket says 172.21.15.40:48452) 2024-09-13T18:03:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e72ffd700 1 -- 172.21.15.40:0/1483900831 learned_addr learned my addr 172.21.15.40:0/1483900831 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e727fc700 1 --2- 172.21.15.40:0/1483900831 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e74159af0 0x7f3e74153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e737fe700 1 --2- 172.21.15.40:0/1483900831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e74152fd0 0x7f3e7414fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e727fc700 1 -- 172.21.15.40:0/1483900831 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e74159af0 msgr2=0x7f3e74153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:03:55.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e727fc700 1 -- 172.21.15.40:0/1483900831 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e74159af0 msgr2=0x7f3e74153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:03:55.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e727fc700 1 --2- 172.21.15.40:0/1483900831 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e74159af0 0x7f3e74153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:55.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e727fc700 1 --2- 172.21.15.40:0/1483900831 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e74159af0 0x7f3e74153990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:55.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e72ffd700 1 -- 172.21.15.40:0/1483900831 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e74159af0 msgr2=0x7f3e74153990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:03:55.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e72ffd700 1 --2- 172.21.15.40:0/1483900831 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e74159af0 0x7f3e74153990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e72ffd700 1 -- 172.21.15.40:0/1483900831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e74152fd0 msgr2=0x7f3e7414fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:55.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e72ffd700 1 --2- 172.21.15.40:0/1483900831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e74152fd0 0x7f3e7414fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.466+0000 7f3e72ffd700 1 -- 172.21.15.40:0/1483900831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e74155170 con 0x7f3e74148ea0 2024-09-13T18:03:55.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.470+0000 7f3e72ffd700 1 --2- 172.21.15.40:0/1483900831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e74148ea0 0x7f3e74151b70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3e64001200 tx=0x7f3e6400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:55.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.470+0000 7f3e5b7fe700 1 -- 172.21.15.40:0/1483900831 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6400c7e0 con 0x7f3e74148ea0 2024-09-13T18:03:55.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.470+0000 7f3e5b7fe700 1 -- 172.21.15.40:0/1483900831 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6400c940 con 0x7f3e74148ea0 2024-09-13T18:03:55.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.470+0000 7f3e73fff700 1 -- 172.21.15.40:0/1483900831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e7415a050 con 0x7f3e74148ea0 2024-09-13T18:03:55.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.470+0000 7f3e5b7fe700 1 -- 172.21.15.40:0/1483900831 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6400ccf0 con 0x7f3e74148ea0 2024-09-13T18:03:55.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.470+0000 7f3e73fff700 1 -- 172.21.15.40:0/1483900831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e74149280 con 0x7f3e74148ea0 2024-09-13T18:03:55.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.470+0000 7f3e73fff700 1 -- 172.21.15.40:0/1483900831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e74157230 con 0x7f3e74148ea0 2024-09-13T18:03:55.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.470+0000 7f3e5b7fe700 1 -- 172.21.15.40:0/1483900831 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3e6401d460 con 0x7f3e74148ea0 2024-09-13T18:03:55.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.474+0000 7f3e5b7fe700 1 --2- 172.21.15.40:0/1483900831 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3e60066760 0x7f3e60068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:55.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.474+0000 7f3e5b7fe700 1 -- 172.21.15.40:0/1483900831 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3e64084cd0 con 0x7f3e74148ea0 2024-09-13T18:03:55.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.474+0000 7f3e727fc700 1 --2- 172.21.15.40:0/1483900831 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3e60066760 0x7f3e60068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:55.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.474+0000 7f3e727fc700 1 --2- 172.21.15.40:0/1483900831 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3e60066760 0x7f3e60068c20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3e74067bd0 tx=0x7f3e68008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:55.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.478+0000 7f3e5b7fe700 1 -- 172.21.15.40:0/1483900831 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3e64051ea0 con 0x7f3e74148ea0 2024-09-13T18:03:55.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.778+0000 7f3e73fff700 1 -- 172.21.15.40:0/1483900831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3e740c89a0 con 0x7f3e74148ea0 2024-09-13T18:03:55.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.778+0000 7f3e5b7fe700 1 -- 172.21.15.40:0/1483900831 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f3e640554c0 con 0x7f3e74148ea0 2024-09-13T18:03:55.788 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":108,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:03:55.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.778+0000 7f3e73fff700 1 -- 172.21.15.40:0/1483900831 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3e60066760 msgr2=0x7f3e60068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:55.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.778+0000 7f3e73fff700 1 --2- 172.21.15.40:0/1483900831 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3e60066760 0x7f3e60068c20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3e74067bd0 tx=0x7f3e68008030 comp rx=0 tx=0).stop 2024-09-13T18:03:55.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.778+0000 7f3e73fff700 1 -- 172.21.15.40:0/1483900831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e74148ea0 msgr2=0x7f3e74151b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.778+0000 7f3e73fff700 1 --2- 172.21.15.40:0/1483900831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e74148ea0 0x7f3e74151b70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3e64001200 tx=0x7f3e6400edd0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.778+0000 7f3e73fff700 1 -- 172.21.15.40:0/1483900831 shutdown_connections 2024-09-13T18:03:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.778+0000 7f3e73fff700 1 --2- 172.21.15.40:0/1483900831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e74152fd0 0x7f3e7414fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.778+0000 7f3e73fff700 1 --2- 172.21.15.40:0/1483900831 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3e60066760 0x7f3e60068c20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.778+0000 7f3e73fff700 1 --2- 172.21.15.40:0/1483900831 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e74159af0 0x7f3e74153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.778+0000 7f3e73fff700 1 --2- 172.21.15.40:0/1483900831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e74148ea0 0x7f3e74151b70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.778+0000 7f3e73fff700 1 -- 172.21.15.40:0/1483900831 >> 172.21.15.40:0/1483900831 conn(0x7f3e74066090 msgr2=0x7f3e7406f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.778+0000 7f3e73fff700 1 -- 172.21.15.40:0/1483900831 shutdown_connections 2024-09-13T18:03:55.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.778+0000 7f3e73fff700 1 -- 172.21.15.40:0/1483900831 wait complete. 2024-09-13T18:03:55.802 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:03:55.802 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:03:55.802 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:03:55.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.918+0000 7f0cfbfff700 1 Processor -- start 2024-09-13T18:03:55.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.918+0000 7f0cfbfff700 1 -- start start 2024-09-13T18:03:55.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfbfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cfc0724d0 0x7f0cfc070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:55.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfbfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0cfc0c6190 0x7f0cfc070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:55.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfbfff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cfc15aea0 con 0x7f0cfc153150 2024-09-13T18:03:55.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfbfff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cfc159e30 con 0x7f0cfc0c6190 2024-09-13T18:03:55.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfbfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc15a0d0 con 0x7f0cfc0724d0 2024-09-13T18:03:55.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfb7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0cfc0c6190 0x7f0cfc070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:55.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfa7fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cfc0724d0 0x7f0cfc070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:55.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfa7fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cfc0724d0 0x7f0cfc070aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:32842/0 (socket says 172.21.15.40:32842) 2024-09-13T18:03:55.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfa7fc700 1 -- 172.21.15.40:0/3406814875 learned_addr learned my addr 172.21.15.40:0/3406814875 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:55.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfa7fc700 1 -- 172.21.15.40:0/3406814875 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cfc0724d0 msgr2=0x7f0cfc070aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:03:55.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfa7fc700 1 -- 172.21.15.40:0/3406814875 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cfc0724d0 msgr2=0x7f0cfc070aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:03:55.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfa7fc700 1 --2- 172.21.15.40:0/3406814875 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cfc0724d0 0x7f0cfc070aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:55.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfa7fc700 1 --2- 172.21.15.40:0/3406814875 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cfc0724d0 0x7f0cfc070aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:55.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfb7fe700 1 -- 172.21.15.40:0/3406814875 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cfc0724d0 msgr2=0x7f0cfc070aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:03:55.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfb7fe700 1 --2- 172.21.15.40:0/3406814875 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cfc0724d0 0x7f0cfc070aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfb7fe700 1 -- 172.21.15.40:0/3406814875 >> v1:172.21.15.43:6789/0 conn(0x7f0cfc153150 legacy=0x7f0cfc15a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:55.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfb7fe700 1 -- 172.21.15.40:0/3406814875 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc0cb2a0 con 0x7f0cfc0c6190 2024-09-13T18:03:55.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfb7fe700 1 --2- 172.21.15.40:0/3406814875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cfc0c6190 0x7f0cfc070fe0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009d90 tx=0x7f0cf400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=376c85198d7b4205 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:55.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cf9ffb700 1 -- 172.21.15.40:0/3406814875 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf400e070 con 0x7f0cfc0c6190 2024-09-13T18:03:55.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cf9ffb700 1 -- 172.21.15.40:0/3406814875 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf400bbd0 con 0x7f0cfc0c6190 2024-09-13T18:03:55.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cf9ffb700 1 -- 172.21.15.40:0/3406814875 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4012430 con 0x7f0cfc0c6190 2024-09-13T18:03:55.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfbfff700 1 -- 172.21.15.40:0/3406814875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cfc0c6190 msgr2=0x7f0cfc070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:55.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfbfff700 1 --2- 172.21.15.40:0/3406814875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cfc0c6190 0x7f0cfc070fe0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009d90 tx=0x7f0cf400b330 comp rx=0 tx=0).stop 2024-09-13T18:03:55.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfbfff700 1 -- 172.21.15.40:0/3406814875 shutdown_connections 2024-09-13T18:03:55.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfbfff700 1 --2- 172.21.15.40:0/3406814875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cfc0c6190 0x7f0cfc070fe0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfbfff700 1 --2- 172.21.15.40:0/3406814875 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cfc0724d0 0x7f0cfc070aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.922+0000 7f0cfbfff700 1 -- 172.21.15.40:0/3406814875 >> 172.21.15.40:0/3406814875 conn(0x7f0cfc06a390 msgr2=0x7f0cfc06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:55.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.926+0000 7f0cfbfff700 1 -- 172.21.15.40:0/3406814875 shutdown_connections 2024-09-13T18:03:55.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.926+0000 7f0cfbfff700 1 -- 172.21.15.40:0/3406814875 wait complete. 2024-09-13T18:03:55.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.926+0000 7f0cfbfff700 1 Processor -- start 2024-09-13T18:03:55.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.926+0000 7f0cfbfff700 1 -- start start 2024-09-13T18:03:55.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.926+0000 7f0cfbfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cfc0724d0 0x7f0cfc157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:55.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.926+0000 7f0cfbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cfc0c6190 0x7f0cfc1587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:55.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.926+0000 7f0cfbfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cfc153150 0x7f0cfc1f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:55.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.926+0000 7f0cfbfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc0c42d0 con 0x7f0cfc0c6190 2024-09-13T18:03:55.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.926+0000 7f0cfbfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc158240 con 0x7f0cfc153150 2024-09-13T18:03:55.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.926+0000 7f0cfbfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc1f1d40 con 0x7f0cfc0724d0 2024-09-13T18:03:55.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfaffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cfc0724d0 0x7f0cfc157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:55.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfaffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cfc0724d0 0x7f0cfc157850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:32850/0 (socket says 172.21.15.40:32850) 2024-09-13T18:03:55.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfaffd700 1 -- 172.21.15.40:0/4233867757 learned_addr learned my addr 172.21.15.40:0/4233867757 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:55.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfb7fe700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cfc153150 0x7f0cfc1f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:55.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfa7fc700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cfc0c6190 0x7f0cfc1587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:55.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfaffd700 1 -- 172.21.15.40:0/4233867757 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cfc0724d0 msgr2=0x7f0cfc157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:03:55.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfaffd700 1 -- 172.21.15.40:0/4233867757 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cfc0724d0 msgr2=0x7f0cfc157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:03:55.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfaffd700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cfc0724d0 0x7f0cfc157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:55.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfaffd700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cfc0724d0 0x7f0cfc157850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:55.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfb7fe700 1 -- 172.21.15.40:0/4233867757 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cfc0724d0 msgr2=0x7f0cfc157850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:03:55.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfb7fe700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cfc0724d0 0x7f0cfc157850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfb7fe700 1 -- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cfc0c6190 msgr2=0x7f0cfc1587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:55.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfb7fe700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cfc0c6190 0x7f0cfc1587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfb7fe700 1 -- 172.21.15.40:0/4233867757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf4009a70 con 0x7f0cfc153150 2024-09-13T18:03:55.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfb7fe700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cfc153150 0x7f0cfc1f2500 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4016000 tx=0x7f0cf4004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:55.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cf3fff700 1 -- 172.21.15.40:0/4233867757 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4012430 con 0x7f0cfc153150 2024-09-13T18:03:55.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cf3fff700 1 -- 172.21.15.40:0/4233867757 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf4020070 con 0x7f0cfc153150 2024-09-13T18:03:55.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cf3fff700 1 -- 172.21.15.40:0/4233867757 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4004280 con 0x7f0cfc153150 2024-09-13T18:03:55.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfbfff700 1 -- 172.21.15.40:0/4233867757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cfc0c4010 con 0x7f0cfc153150 2024-09-13T18:03:55.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfbfff700 1 -- 172.21.15.40:0/4233867757 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cfc0c4b50 con 0x7f0cfc153150 2024-09-13T18:03:55.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfbfff700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cfc1f2a40 0x7f0cfc146030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:55.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfbfff700 1 -- 172.21.15.40:0/4233867757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc067600 con 0x7f0cfc1f2a40 2024-09-13T18:03:55.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfaffd700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cfc1f2a40 0x7f0cfc146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:55.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.930+0000 7f0cfaffd700 1 -- 172.21.15.40:0/4233867757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0cfc0cbb40 con 0x7f0cfc1f2a40 2024-09-13T18:03:55.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.934+0000 7f0cfaffd700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cfc1f2a40 0x7f0cfc146030 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00e700 tx=0x7f0cec00eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:55.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.934+0000 7f0cf3fff700 1 -- 172.21.15.40:0/4233867757 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec0052e0 con 0x7f0cfc1f2a40 2024-09-13T18:03:55.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.934+0000 7f0cf3fff700 1 -- 172.21.15.40:0/4233867757 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0cf40043e0 con 0x7f0cfc153150 2024-09-13T18:03:55.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.934+0000 7f0cf3fff700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0cdc0667d0 0x7f0cdc068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:55.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.934+0000 7f0cf3fff700 1 -- 172.21.15.40:0/4233867757 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0cf4022070 con 0x7f0cfc153150 2024-09-13T18:03:55.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.934+0000 7f0cfa7fc700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0cdc0667d0 0x7f0cdc068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:55.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.934+0000 7f0cf3fff700 1 -- 172.21.15.40:0/4233867757 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0cec005440 con 0x7f0cfc1f2a40 2024-09-13T18:03:55.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.934+0000 7f0cf3fff700 1 -- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cfc1f2a40 msgr2=0x7f0cfc146030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:55.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.934+0000 7f0cf3fff700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cfc1f2a40 0x7f0cfc146030 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00e700 tx=0x7f0cec00eac0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.934+0000 7f0cf3fff700 1 -- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cfc1f2a40 msgr2=0x7f0cfc146030 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:03:55.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.934+0000 7f0cf3fff700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cfc1f2a40 0x7f0cfc146030 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.934+0000 7f0cfa7fc700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0cdc0667d0 0x7f0cdc068c90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc159210 tx=0x7f0ce4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:55.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cfbfff700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cfc156de0 0x7f0cfc146570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:55.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cfbfff700 1 -- 172.21.15.40:0/4233867757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc0cb860 con 0x7f0cfc156de0 2024-09-13T18:03:55.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cfaffd700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cfc156de0 0x7f0cfc146570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:55.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cfaffd700 1 -- 172.21.15.40:0/4233867757 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0cfc0cc060 con 0x7f0cfc156de0 2024-09-13T18:03:55.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cfaffd700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cfc156de0 0x7f0cfc146570 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00e700 tx=0x7f0cec001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:55.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cf3fff700 1 -- 172.21.15.40:0/4233867757 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec021070 con 0x7f0cfc156de0 2024-09-13T18:03:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cf3fff700 1 -- 172.21.15.40:0/4233867757 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4950 (secure 0 0 0) 0x7f0cec006a90 con 0x7f0cfc156de0 2024-09-13T18:03:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cf3fff700 1 -- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cfc156de0 msgr2=0x7f0cfc146570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cf3fff700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cfc156de0 0x7f0cfc146570 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00e700 tx=0x7f0cec001080 comp rx=0 tx=0).stop 2024-09-13T18:03:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cf3fff700 1 -- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cfc156de0 msgr2=0x7f0cfc146570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:03:55.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cf3fff700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cfc156de0 0x7f0cfc146570 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.958 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:03:55.958 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:03:55.958 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:03:55.958 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:03:55.959 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 108, 2024-09-13T18:03:55.959 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:03:55.959 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:03:55.959 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:03:55.959 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:03:55.959 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:55.959 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:03:55.959 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:03:55.959 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:03:55.959 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:03:55.959 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:55.959 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:55.959 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:55.959 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:55.960 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:55.960 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:55.960 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:55.960 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:55.960 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:55.960 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:55.960 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:03:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:03:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:55.961 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:03:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:03:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:03:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:03:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:03:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:03:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:03:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:03:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:03:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:03:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:03:55.962 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:03:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:03:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:03:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:03:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:03:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:03:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:55.963 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:03:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:03:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:03:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:03:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:55.964 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:03:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:03:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:55.965 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:03:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:03:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:03:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:03:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:03:55.966 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:03:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:03:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:03:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:55.967 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:03:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:03:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:03:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:55.968 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:03:55.969 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:55.969 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:55.969 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:55.969 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:55.969 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:03:55.969 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:03:55.969 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:55.969 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:55.969 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:55.969 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:55.969 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:55.969 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:55.969 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:03:55.969 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:03:55.970 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:55.970 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:55.970 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:55.970 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:03:55.970 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:55.970 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:55.970 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:03:55.970 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:55.970 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:55.971 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:55.971 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:03:55.971 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:55.971 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:55.971 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:03:55.971 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:55.971 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:55.971 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:55.971 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 8 2024-09-13T18:03:55.971 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:55.971 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:55.971 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:03:55.971 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:55.971 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:55.972 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:55.972 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:03:55.972 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:55.972 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:55.972 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:03:55.972 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:55.972 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:55.972 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:55.972 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:03:55.972 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:55.972 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:55.972 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:55.972 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:03:55.972 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:03:55.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cfbfff700 1 -- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0cdc0667d0 msgr2=0x7f0cdc068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:55.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cfbfff700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0cdc0667d0 0x7f0cdc068c90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc159210 tx=0x7f0ce4008040 comp rx=0 tx=0).stop 2024-09-13T18:03:55.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cfbfff700 1 -- 172.21.15.40:0/4233867757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cfc153150 msgr2=0x7f0cfc1f2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:55.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cfb7fe700 1 -- 172.21.15.40:0/4233867757 reap_dead start 2024-09-13T18:03:55.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cfbfff700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cfc153150 0x7f0cfc1f2500 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4016000 tx=0x7f0cf4004af0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cfbfff700 1 -- 172.21.15.40:0/4233867757 shutdown_connections 2024-09-13T18:03:55.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cfbfff700 1 --2- 172.21.15.40:0/4233867757 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cfc153150 0x7f0cfc1f2500 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:55.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cfbfff700 1 -- 172.21.15.40:0/4233867757 >> 172.21.15.40:0/4233867757 conn(0x7f0cfc06a390 msgr2=0x7f0cfc1514a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:55.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cfbfff700 1 -- 172.21.15.40:0/4233867757 shutdown_connections 2024-09-13T18:03:55.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:55.946+0000 7f0cfbfff700 1 -- 172.21.15.40:0/4233867757 wait complete. 2024-09-13T18:03:55.975 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:03:55.975 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:03:55.975 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:03:56.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.090+0000 7f5eff59e700 1 Processor -- start 2024-09-13T18:03:56.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.090+0000 7f5eff59e700 1 -- start start 2024-09-13T18:03:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5eff59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f000c9050 0x7f5f000b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5eff59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5f000b2180 0x7f5f000cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5eff59e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5f000cbbf0 0x7f5f000cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5eff59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f00159620 con 0x7f5f000b2180 2024-09-13T18:03:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5eff59e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f0015a5d0 con 0x7f5f000cbbf0 2024-09-13T18:03:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5eff59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000b1f20 con 0x7f5f000c9050 2024-09-13T18:03:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5efdd9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5f000b2180 0x7f5f000cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5efdd9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5f000b2180 0x7f5f000cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48492/0 (socket says 172.21.15.40:48492) 2024-09-13T18:03:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5efdd9b700 1 -- 172.21.15.40:0/3526197799 learned_addr learned my addr 172.21.15.40:0/3526197799 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5efe59c700 1 --2- 172.21.15.40:0/3526197799 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f000c9050 0x7f5f000b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5efed9d700 1 --2- 172.21.15.40:0/3526197799 >> v2:172.21.15.43:3300/0 conn(0x7f5f000cbbf0 0x7f5f000cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5efdd9b700 1 -- 172.21.15.40:0/3526197799 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f000c9050 msgr2=0x7f5f000b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:03:56.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5efdd9b700 1 --2- 172.21.15.40:0/3526197799 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f000c9050 0x7f5f000b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5efdd9b700 1 -- 172.21.15.40:0/3526197799 >> v2:172.21.15.43:3300/0 conn(0x7f5f000cbbf0 msgr2=0x7f5f000cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:03:56.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5efdd9b700 1 --2- 172.21.15.40:0/3526197799 >> v2:172.21.15.43:3300/0 conn(0x7f5f000cbbf0 0x7f5f000cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5efdd9b700 1 -- 172.21.15.40:0/3526197799 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f000ca1e0 con 0x7f5f000b2180 2024-09-13T18:03:56.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5efdd9b700 1 --2- 172.21.15.40:0/3526197799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f000b2180 0x7f5f000cb6b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5f000cca50 tx=0x7f5ef4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef66f83ae4e4093 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:56.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5efd59a700 1 -- 172.21.15.40:0/3526197799 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef400ba20 con 0x7f5f000b2180 2024-09-13T18:03:56.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5efd59a700 1 -- 172.21.15.40:0/3526197799 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef400bb80 con 0x7f5f000b2180 2024-09-13T18:03:56.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5efd59a700 1 -- 172.21.15.40:0/3526197799 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4009e60 con 0x7f5f000b2180 2024-09-13T18:03:56.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5eff59e700 1 -- 172.21.15.40:0/3526197799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f000b2180 msgr2=0x7f5f000cb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5eff59e700 1 --2- 172.21.15.40:0/3526197799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f000b2180 0x7f5f000cb6b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5f000cca50 tx=0x7f5ef4009c20 comp rx=0 tx=0).stop 2024-09-13T18:03:56.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5eff59e700 1 -- 172.21.15.40:0/3526197799 shutdown_connections 2024-09-13T18:03:56.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5eff59e700 1 --2- 172.21.15.40:0/3526197799 >> v2:172.21.15.43:3300/0 conn(0x7f5f000cbbf0 0x7f5f000cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5eff59e700 1 --2- 172.21.15.40:0/3526197799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f000b2180 0x7f5f000cb6b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5eff59e700 1 --2- 172.21.15.40:0/3526197799 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f000c9050 0x7f5f000b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5eff59e700 1 -- 172.21.15.40:0/3526197799 >> 172.21.15.40:0/3526197799 conn(0x7f5f00066270 msgr2=0x7f5f00066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:56.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5eff59e700 1 -- 172.21.15.40:0/3526197799 shutdown_connections 2024-09-13T18:03:56.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.094+0000 7f5eff59e700 1 -- 172.21.15.40:0/3526197799 wait complete. 2024-09-13T18:03:56.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.098+0000 7f5eff59e700 1 Processor -- start 2024-09-13T18:03:56.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.098+0000 7f5eff59e700 1 -- start start 2024-09-13T18:03:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.098+0000 7f5eff59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f000b2180 0x7f5f001e4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.098+0000 7f5eff59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f000c9050 0x7f5f0014e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.098+0000 7f5eff59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f000cbbf0 0x7f5f000b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.098+0000 7f5eff59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0014f970 con 0x7f5f000c9050 2024-09-13T18:03:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.098+0000 7f5eff59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f001504b0 con 0x7f5f000b2180 2024-09-13T18:03:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.098+0000 7f5eff59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f001efc10 con 0x7f5f000cbbf0 2024-09-13T18:03:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.098+0000 7f5efdd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f000c9050 0x7f5f0014e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.098+0000 7f5efdd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f000c9050 0x7f5f0014e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48504/0 (socket says 172.21.15.40:48504) 2024-09-13T18:03:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.098+0000 7f5efdd9b700 1 -- 172.21.15.40:0/1509105882 learned_addr learned my addr 172.21.15.40:0/1509105882 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.098+0000 7f5efe59c700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f000b2180 0x7f5f001e4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.098+0000 7f5efed9d700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f000cbbf0 0x7f5f000b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.098+0000 7f5efdd9b700 1 -- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f000cbbf0 msgr2=0x7f5f000b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.098+0000 7f5efdd9b700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f000cbbf0 0x7f5f000b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.098+0000 7f5efdd9b700 1 -- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f000b2180 msgr2=0x7f5f001e4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.098+0000 7f5efdd9b700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f000b2180 0x7f5f001e4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.098+0000 7f5efdd9b700 1 -- 172.21.15.40:0/1509105882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef4009570 con 0x7f5f000c9050 2024-09-13T18:03:56.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.098+0000 7f5efe59c700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f000b2180 0x7f5f001e4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:03:56.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.102+0000 7f5efdd9b700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f000c9050 0x7f5f0014e650 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5f0008c780 tx=0x7f5ef4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:56.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.102+0000 7f5eeeffd700 1 -- 172.21.15.40:0/1509105882 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4004610 con 0x7f5f000c9050 2024-09-13T18:03:56.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.102+0000 7f5eeeffd700 1 -- 172.21.15.40:0/1509105882 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef40036c0 con 0x7f5f000c9050 2024-09-13T18:03:56.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.102+0000 7f5eff59e700 1 -- 172.21.15.40:0/1509105882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0014a250 con 0x7f5f000c9050 2024-09-13T18:03:56.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.102+0000 7f5eeeffd700 1 -- 172.21.15.40:0/1509105882 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4003a70 con 0x7f5f000c9050 2024-09-13T18:03:56.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.102+0000 7f5eff59e700 1 -- 172.21.15.40:0/1509105882 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0014fed0 con 0x7f5f000c9050 2024-09-13T18:03:56.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.102+0000 7f5eff59e700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f000ccce0 0x7f5f001e3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.102+0000 7f5eff59e700 1 -- 172.21.15.40:0/1509105882 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000746e0 con 0x7f5f000ccce0 2024-09-13T18:03:56.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.102+0000 7f5efe59c700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f000ccce0 0x7f5f001e3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.102+0000 7f5eeeffd700 1 -- 172.21.15.40:0/1509105882 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5ef4021070 con 0x7f5f000c9050 2024-09-13T18:03:56.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.102+0000 7f5eeeffd700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5ee0066760 0x7f5ee0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.102+0000 7f5efe59c700 1 -- 172.21.15.40:0/1509105882 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5f00159dd0 con 0x7f5f000ccce0 2024-09-13T18:03:56.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.102+0000 7f5eeeffd700 1 -- 172.21.15.40:0/1509105882 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5ef4085b10 con 0x7f5f000c9050 2024-09-13T18:03:56.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.102+0000 7f5efed9d700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5ee0066760 0x7f5ee0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.102+0000 7f5efe59c700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f000ccce0 0x7f5f001e3cd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00107c0 tx=0x7f5ef0010b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:56.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.102+0000 7f5eeeffd700 1 -- 172.21.15.40:0/1509105882 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0015880 con 0x7f5f000ccce0 2024-09-13T18:03:56.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.102+0000 7f5efed9d700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5ee0066760 0x7f5ee0068c20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009ee0 tx=0x7f5ef800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:56.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.106+0000 7f5eeeffd700 1 -- 172.21.15.40:0/1509105882 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5ef000c050 con 0x7f5f000ccce0 2024-09-13T18:03:56.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.106+0000 7f5eeeffd700 1 -- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f000ccce0 msgr2=0x7f5f001e3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.106+0000 7f5eeeffd700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f000ccce0 0x7f5f001e3cd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00107c0 tx=0x7f5ef0010b80 comp rx=0 tx=0).stop 2024-09-13T18:03:56.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.106+0000 7f5eeeffd700 1 -- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f000ccce0 msgr2=0x7f5f001e3cd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:03:56.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.106+0000 7f5eeeffd700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f000ccce0 0x7f5f001e3cd0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00107c0 tx=0x7f5ef0010b80 comp rx=0 tx=0).stop 2024-09-13T18:03:56.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.118+0000 7f5eff59e700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f001e4210 0x7f5f001f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.118+0000 7f5eff59e700 1 -- 172.21.15.40:0/1509105882 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f00159af0 con 0x7f5f001e4210 2024-09-13T18:03:56.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.118+0000 7f5efe59c700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f001e4210 0x7f5f001f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.118+0000 7f5efe59c700 1 -- 172.21.15.40:0/1509105882 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5f0015a0b0 con 0x7f5f001e4210 2024-09-13T18:03:56.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.118+0000 7f5efe59c700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f001e4210 0x7f5f001f2660 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00107c0 tx=0x7f5ef0004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:56.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.122+0000 7f5eeeffd700 1 -- 172.21.15.40:0/1509105882 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0021bb0 con 0x7f5f001e4210 2024-09-13T18:03:56.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.122+0000 7f5eeeffd700 1 -- 172.21.15.40:0/1509105882 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4616 (secure 0 0 0) 0x7f5ef0021d10 con 0x7f5f001e4210 2024-09-13T18:03:56.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.122+0000 7f5eeeffd700 1 -- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f001e4210 msgr2=0x7f5f001f2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.122+0000 7f5eeeffd700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f001e4210 0x7f5f001f2660 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00107c0 tx=0x7f5ef0004770 comp rx=0 tx=0).stop 2024-09-13T18:03:56.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.122+0000 7f5eeeffd700 1 -- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f001e4210 msgr2=0x7f5f001f2660 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:03:56.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.122+0000 7f5eeeffd700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f001e4210 0x7f5f001f2660 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.132 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:03:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:03:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:03:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:03:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 108, 2024-09-13T18:03:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:03:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:03:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:03:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:03:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 3, 2024-09-13T18:03:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:03:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:03:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:03:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:56.133 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:56.134 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:56.134 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:56.134 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:56.134 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:56.134 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:56.134 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:56.134 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:56.134 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:56.134 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:56.134 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:03:56.134 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:03:56.134 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:56.134 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:56.134 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:56.134 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:56.135 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:56.135 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:56.135 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:56.135 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:56.135 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:56.135 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:56.135 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:56.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.135 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:03:56.135 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:03:56.135 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:03:56.135 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:03:56.135 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:03:56.135 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:03:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:03:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:03:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:03:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:03:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:03:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:03:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:03:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:03:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:03:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:03:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:03:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:56.136 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:56.137 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:56.137 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:56.137 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:56.137 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:56.137 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:56.137 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:56.137 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:56.137 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:03:56.137 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.137 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:03:56.138 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.138 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:03:56.138 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:03:56.138 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:56.138 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:56.138 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.138 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:56.138 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:03:56.138 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:56.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.138 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.138 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:56.138 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:03:56.139 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:56.139 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:56.139 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:56.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.139 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:03:56.139 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:03:56.139 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:56.139 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:56.139 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:56.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.139 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.139 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:03:56.139 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:03:56.139 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:56.140 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:56.140 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.140 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:56.140 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:03:56.140 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:56.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.140 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.140 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:56.140 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:03:56.140 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:56.140 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:56.140 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:56.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.141 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:03:56.141 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:03:56.141 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:56.141 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:56.141 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:56.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.141 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:03:56.141 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:03:56.141 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:56.141 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:56.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.141 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:56.141 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:03:56.142 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:56.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.142 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:56.142 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:03:56.142 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:56.142 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:56.142 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:56.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.143 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:03:56.143 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:03:56.143 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:56.143 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:56.143 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:56.143 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:56.143 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:56.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.143 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:03:56.143 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:03:56.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.143 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:56.143 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:56.144 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:03:56.144 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:56.144 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:56.144 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:03:56.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.144 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:56.144 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:56.144 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:03:56.144 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:56.144 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:56.144 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:03:56.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.144 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:56.144 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:56.145 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:03:56.145 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:56.145 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:56.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.145 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:03:56.145 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:03:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.122+0000 7f5eff59e700 1 -- 172.21.15.40:0/1509105882 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5ee0066760 msgr2=0x7f5ee0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.122+0000 7f5eff59e700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5ee0066760 0x7f5ee0068c20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009ee0 tx=0x7f5ef800b040 comp rx=0 tx=0).stop 2024-09-13T18:03:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.122+0000 7f5eff59e700 1 -- 172.21.15.40:0/1509105882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f000c9050 msgr2=0x7f5f0014e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.122+0000 7f5eff59e700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f000c9050 0x7f5f0014e650 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5f0008c780 tx=0x7f5ef4004770 comp rx=0 tx=0).stop 2024-09-13T18:03:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.122+0000 7f5efed9d700 1 -- 172.21.15.40:0/1509105882 reap_dead start 2024-09-13T18:03:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.122+0000 7f5eff59e700 1 -- 172.21.15.40:0/1509105882 shutdown_connections 2024-09-13T18:03:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.122+0000 7f5eff59e700 1 --2- 172.21.15.40:0/1509105882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f000c9050 0x7f5f0014e650 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.122+0000 7f5eff59e700 1 -- 172.21.15.40:0/1509105882 >> 172.21.15.40:0/1509105882 conn(0x7f5f00066270 msgr2=0x7f5f00158530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.122+0000 7f5eff59e700 1 -- 172.21.15.40:0/1509105882 shutdown_connections 2024-09-13T18:03:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.122+0000 7f5eff59e700 1 -- 172.21.15.40:0/1509105882 wait complete. 2024-09-13T18:03:56.148 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:03:56.148 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:03:56.148 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:03:56.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d3a83c700 1 Processor -- start 2024-09-13T18:03:56.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d3a83c700 1 -- start start 2024-09-13T18:03:56.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d3a83c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c049000 0x7f5d2c0e1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d3a83c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2c050310 con 0x7f5d2c049420 2024-09-13T18:03:56.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d3a83c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2c0505d0 con 0x7f5d2c0d8b50 2024-09-13T18:03:56.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d3a83c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0508b0 con 0x7f5d2c049000 2024-09-13T18:03:56.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d3a03b700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5d2c049420 0x7f5d2c0e22a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42538/0 (socket says 172.21.15.40:42538) 2024-09-13T18:03:56.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d3a03b700 1 -- 172.21.15.40:0/742645946 learned_addr learned my addr 172.21.15.40:0/742645946 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:03:56.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d39039700 1 --2- 172.21.15.40:0/742645946 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c049000 0x7f5d2c0e1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d39039700 1 -- 172.21.15.40:0/742645946 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c049000 msgr2=0x7f5d2c0e1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:03:56.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d39039700 1 -- 172.21.15.40:0/742645946 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c049000 msgr2=0x7f5d2c0e1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:03:56.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d39039700 1 --2- 172.21.15.40:0/742645946 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c049000 0x7f5d2c0e1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:56.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d38838700 1 -- 172.21.15.40:0/742645946 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341053844 0 0) 0x7f5d2c050310 con 0x7f5d2c049420 2024-09-13T18:03:56.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d38838700 1 -- 172.21.15.40:0/742645946 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d200035d0 con 0x7f5d2c049420 2024-09-13T18:03:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d39039700 1 --2- 172.21.15.40:0/742645946 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c049000 0x7f5d2c0e1d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d38838700 1 -- 172.21.15.40:0/742645946 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618314838 0 0) 0x7f5d2c0505d0 con 0x7f5d2c0d8b50 2024-09-13T18:03:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d38838700 1 -- 172.21.15.40:0/742645946 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d2c050310 con 0x7f5d2c0d8b50 2024-09-13T18:03:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d38838700 1 -- 172.21.15.40:0/742645946 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2756270782 0 0) 0x7f5d200035d0 con 0x7f5d2c049420 2024-09-13T18:03:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d38838700 1 -- 172.21.15.40:0/742645946 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c049000 msgr2=0x7f5d2c0e1d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:03:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d38838700 1 --2- 172.21.15.40:0/742645946 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c049000 0x7f5d2c0e1d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d38838700 1 -- 172.21.15.40:0/742645946 >> v1:172.21.15.43:6789/0 conn(0x7f5d2c0d8b50 legacy=0x7f5d2c04ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d38838700 1 -- 172.21.15.40:0/742645946 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c0068c0 con 0x7f5d2c049420 2024-09-13T18:03:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.262+0000 7f5d38838700 1 -- 172.21.15.40:0/742645946 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f5d34067210 con 0x7f5d2c049420 2024-09-13T18:03:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d38838700 1 -- 172.21.15.40:0/742645946 >> v1:172.21.15.40:6789/0 conn(0x7f5d2c049420 legacy=0x7f5d2c0e22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d38838700 1 --2- 172.21.15.40:0/742645946 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c049000 0x7f5d20003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d38838700 1 --2- 172.21.15.40:0/742645946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d20004140 0x7f5d20004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d38838700 1 --2- 172.21.15.40:0/742645946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d20004db0 0x7f5d20005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d38838700 1 -- 172.21.15.40:0/742645946 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0508b0 con 0x7f5d20004140 2024-09-13T18:03:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d38838700 1 -- 172.21.15.40:0/742645946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2000bb10 con 0x7f5d20004db0 2024-09-13T18:03:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d38838700 1 -- 172.21.15.40:0/742645946 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2000bcd0 con 0x7f5d2c049000 2024-09-13T18:03:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d39039700 1 --2- 172.21.15.40:0/742645946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d20004140 0x7f5d20004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d3983a700 1 --2- 172.21.15.40:0/742645946 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c049000 0x7f5d20003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d3a03b700 1 --2- 172.21.15.40:0/742645946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d20004db0 0x7f5d20005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d39039700 1 -- 172.21.15.40:0/742645946 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c049000 msgr2=0x7f5d20003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:03:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d39039700 1 --2- 172.21.15.40:0/742645946 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c049000 0x7f5d20003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d39039700 1 -- 172.21.15.40:0/742645946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d20004db0 msgr2=0x7f5d20005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:03:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d39039700 1 --2- 172.21.15.40:0/742645946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d20004db0 0x7f5d20005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d39039700 1 -- 172.21.15.40:0/742645946 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d200035d0 con 0x7f5d20004140 2024-09-13T18:03:56.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d39039700 1 --2- 172.21.15.40:0/742645946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d20004140 0x7f5d20004870 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009870 tx=0x7f5d200058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aca5f3f4d3e6a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:56.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d38838700 1 -- 172.21.15.40:0/742645946 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2400c070 con 0x7f5d20004140 2024-09-13T18:03:56.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d38838700 1 -- 172.21.15.40:0/742645946 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d24013630 con 0x7f5d20004140 2024-09-13T18:03:56.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d38838700 1 -- 172.21.15.40:0/742645946 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d240139e0 con 0x7f5d20004140 2024-09-13T18:03:56.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d3a83c700 1 -- 172.21.15.40:0/742645946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d20004140 msgr2=0x7f5d20004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d3a83c700 1 --2- 172.21.15.40:0/742645946 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d20004140 0x7f5d20004870 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009870 tx=0x7f5d200058a0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d3a03b700 1 -- 172.21.15.40:0/742645946 reap_dead start 2024-09-13T18:03:56.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d3a83c700 1 -- 172.21.15.40:0/742645946 shutdown_connections 2024-09-13T18:03:56.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d3a83c700 1 -- 172.21.15.40:0/742645946 >> 172.21.15.40:0/742645946 conn(0x7f5d2c054120 msgr2=0x7f5d2c054530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:56.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d3a83c700 1 -- 172.21.15.40:0/742645946 shutdown_connections 2024-09-13T18:03:56.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.266+0000 7f5d3a83c700 1 -- 172.21.15.40:0/742645946 wait complete. 2024-09-13T18:03:56.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.270+0000 7f5d3a83c700 1 Processor -- start 2024-09-13T18:03:56.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.270+0000 7f5d3a83c700 1 -- start start 2024-09-13T18:03:56.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.270+0000 7f5d3a83c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d20004140 0x7f5d2c1742f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.270+0000 7f5d3a83c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0df2e0 0x7f5d2c04ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.270+0000 7f5d3a83c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d2c0cf870 0x7f5d2c053510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.270+0000 7f5d3a83c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0d0240 con 0x7f5d20004140 2024-09-13T18:03:56.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.270+0000 7f5d3a83c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c04e4f0 con 0x7f5d2c0cf870 2024-09-13T18:03:56.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.270+0000 7f5d3a83c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c04dc50 con 0x7f5d2c0df2e0 2024-09-13T18:03:56.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d3983a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d20004140 0x7f5d2c1742f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d3983a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d20004140 0x7f5d2c1742f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48516/0 (socket says 172.21.15.40:48516) 2024-09-13T18:03:56.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d3983a700 1 -- 172.21.15.40:0/1702857057 learned_addr learned my addr 172.21.15.40:0/1702857057 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:56.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d39039700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0df2e0 0x7f5d2c04ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d3a03b700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d2c0cf870 0x7f5d2c053510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d39039700 1 -- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0df2e0 msgr2=0x7f5d2c04ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:03:56.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d39039700 1 -- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0df2e0 msgr2=0x7f5d2c04ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:03:56.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d39039700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0df2e0 0x7f5d2c04ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:56.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d3983a700 1 -- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0df2e0 msgr2=0x7f5d2c04ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d39039700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0df2e0 0x7f5d2c04ce30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:56.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d3983a700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0df2e0 0x7f5d2c04ce30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d3983a700 1 -- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d2c0cf870 msgr2=0x7f5d2c053510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d3983a700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d2c0cf870 0x7f5d2c053510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d3983a700 1 -- 172.21.15.40:0/1702857057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c04df30 con 0x7f5d20004140 2024-09-13T18:03:56.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d3a03b700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d2c0cf870 0x7f5d2c053510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:03:56.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d3983a700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d20004140 0x7f5d2c1742f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000f840 tx=0x7f5d3000fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:56.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d2a7fc700 1 -- 172.21.15.40:0/1702857057 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d300052e0 con 0x7f5d20004140 2024-09-13T18:03:56.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d2a7fc700 1 -- 172.21.15.40:0/1702857057 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d30005440 con 0x7f5d20004140 2024-09-13T18:03:56.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d3a83c700 1 -- 172.21.15.40:0/1702857057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2c04e210 con 0x7f5d20004140 2024-09-13T18:03:56.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d2a7fc700 1 -- 172.21.15.40:0/1702857057 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d300038e0 con 0x7f5d20004140 2024-09-13T18:03:56.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d3a83c700 1 -- 172.21.15.40:0/1702857057 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c04c910 con 0x7f5d20004140 2024-09-13T18:03:56.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d3a83c700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0cfc50 0x7f5d2c0ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d3a83c700 1 -- 172.21.15.40:0/1702857057 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0084b0 con 0x7f5d2c0cfc50 2024-09-13T18:03:56.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d39039700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0cfc50 0x7f5d2c0ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d39039700 1 -- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0cfc50 msgr2=0x7f5d2c0ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-09-13T18:03:56.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d39039700 1 -- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0cfc50 msgr2=0x7f5d2c0ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:03:56.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d39039700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0cfc50 0x7f5d2c0ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:03:56.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d39039700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0cfc50 0x7f5d2c0ce8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:03:56.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.274+0000 7f5d2a7fc700 1 -- 172.21.15.40:0/1702857057 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5d30003a40 con 0x7f5d20004140 2024-09-13T18:03:56.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.278+0000 7f5d2a7fc700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5d18066730 0x7f5d18068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.278+0000 7f5d3a03b700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5d18066730 0x7f5d18068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.278+0000 7f5d2a7fc700 1 -- 172.21.15.40:0/1702857057 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5d300843f0 con 0x7f5d20004140 2024-09-13T18:03:56.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.278+0000 7f5d3a03b700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5d18066730 0x7f5d18068bf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5d34072e80 tx=0x7f5d34065ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:56.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.478+0000 7f5d39039700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0cfc50 0x7f5d2c0ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.478+0000 7f5d39039700 1 -- 172.21.15.40:0/1702857057 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d2c0d82c0 con 0x7f5d2c0cfc50 2024-09-13T18:03:56.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.478+0000 7f5d39039700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0cfc50 0x7f5d2c0ce8d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009870 tx=0x7f5d240047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:56.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.478+0000 7f5d2a7fc700 1 -- 172.21.15.40:0/1702857057 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d24023070 con 0x7f5d2c0cfc50 2024-09-13T18:03:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.478+0000 7f5d2a7fc700 1 -- 172.21.15.40:0/1702857057 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5d240137a0 con 0x7f5d2c0cfc50 2024-09-13T18:03:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.478+0000 7f5d2a7fc700 1 -- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0cfc50 msgr2=0x7f5d2c0ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.478+0000 7f5d2a7fc700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0cfc50 0x7f5d2c0ce8d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009870 tx=0x7f5d240047e0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.478+0000 7f5d2a7fc700 1 -- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0cfc50 msgr2=0x7f5d2c0ce8d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:03:56.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.478+0000 7f5d2a7fc700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0cfc50 0x7f5d2c0ce8d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.494+0000 7f5d3a83c700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0cee10 0x7f5d2c177860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.494+0000 7f5d3a83c700 1 -- 172.21.15.40:0/1702857057 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0d4480 con 0x7f5d2c0cee10 2024-09-13T18:03:56.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.494+0000 7f5d39039700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0cee10 0x7f5d2c177860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.494+0000 7f5d39039700 1 -- 172.21.15.40:0/1702857057 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5d2c007160 con 0x7f5d2c0cee10 2024-09-13T18:03:56.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.494+0000 7f5d39039700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0cee10 0x7f5d2c177860 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0551d0 tx=0x7f5d24008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:56.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.494+0000 7f5d2a7fc700 1 -- 172.21.15.40:0/1702857057 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2402d990 con 0x7f5d2c0cee10 2024-09-13T18:03:56.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.494+0000 7f5d2a7fc700 1 -- 172.21.15.40:0/1702857057 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4450 (secure 0 0 0) 0x7f5d24003e50 con 0x7f5d2c0cee10 2024-09-13T18:03:56.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.494+0000 7f5d2a7fc700 1 -- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0cee10 msgr2=0x7f5d2c177860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.494+0000 7f5d2a7fc700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0cee10 0x7f5d2c177860 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0551d0 tx=0x7f5d24008040 comp rx=0 tx=0).stop 2024-09-13T18:03:56.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.494+0000 7f5d2a7fc700 1 -- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0cee10 msgr2=0x7f5d2c177860 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:03:56.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.494+0000 7f5d2a7fc700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5d2c0cee10 0x7f5d2c177860 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.506 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:03:56.507 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:03:56.507 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:03:56.507 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:03:56.507 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 108, 2024-09-13T18:03:56.507 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:03:56.507 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:03:56.507 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:03:56.507 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:03:56.507 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:56.507 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 3, 2024-09-13T18:03:56.507 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:03:56.507 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:03:56.507 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:03:56.508 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:56.508 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:56.508 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:56.508 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:56.508 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:56.508 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:56.508 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:56.508 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:56.508 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:56.508 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:56.508 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:56.508 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:03:56.508 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:03:56.509 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:56.509 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:56.509 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:56.509 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:56.509 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:56.509 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:56.509 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:56.509 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:56.509 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:56.509 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:56.509 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:56.509 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.509 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:03:56.509 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:03:56.510 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:03:56.510 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:03:56.510 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:03:56.510 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:03:56.510 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:03:56.510 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:03:56.510 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:03:56.510 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:03:56.510 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:03:56.510 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:03:56.510 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:03:56.510 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:03:56.510 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:03:56.511 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:03:56.511 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:03:56.511 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:03:56.511 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:03:56.511 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:03:56.511 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:03:56.511 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:03:56.511 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:03:56.511 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:03:56.511 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:03:56.511 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:03:56.511 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:03:56.511 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:56.511 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:03:56.512 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.512 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:03:56.512 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.512 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:03:56.512 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:03:56.512 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:56.512 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:56.512 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.512 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:56.512 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:03:56.512 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:56.512 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.513 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.513 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:56.513 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:03:56.513 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:56.513 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:56.513 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:56.513 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.513 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:03:56.513 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:03:56.513 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:56.513 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:56.513 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:56.513 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.513 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.513 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:03:56.514 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:03:56.514 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:56.514 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:56.514 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.514 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:56.514 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:03:56.514 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:56.514 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.514 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.514 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:56.514 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:03:56.514 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:56.514 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:56.515 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:56.515 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.515 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:03:56.515 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:03:56.515 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:56.515 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:56.515 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:56.515 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.515 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.515 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:03:56.515 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:03:56.515 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:03:56.515 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:03:56.515 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.516 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:03:56.516 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:03:56.516 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:56.516 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.516 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.516 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:03:56.516 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:03:56.516 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:03:56.516 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:56.516 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:56.516 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.516 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:03:56.516 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:03:56.516 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:03:56.517 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:03:56.517 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:03:56.517 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:56.517 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:56.517 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.517 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:03:56.517 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:03:56.517 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.517 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:56.517 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:56.517 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:03:56.517 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:56.517 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:03:56.517 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:03:56.518 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:03:56.518 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:03:56.518 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:03:56.518 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:03:56.518 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:03:56.518 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:03:56.518 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:03:56.518 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:03:56.518 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:03:56.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.494+0000 7f5d3a83c700 1 -- 172.21.15.40:0/1702857057 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5d18066730 msgr2=0x7f5d18068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.494+0000 7f5d3a83c700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5d18066730 0x7f5d18068bf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5d34072e80 tx=0x7f5d34065ea0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.494+0000 7f5d3a83c700 1 -- 172.21.15.40:0/1702857057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d20004140 msgr2=0x7f5d2c1742f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.494+0000 7f5d3a83c700 1 --2- 172.21.15.40:0/1702857057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5d20004140 0x7f5d2c1742f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000f840 tx=0x7f5d3000fc00 comp rx=0 tx=0).stop 2024-09-13T18:03:56.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.494+0000 7f5d3a03b700 1 -- 172.21.15.40:0/1702857057 reap_dead start 2024-09-13T18:03:56.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.494+0000 7f5d3a03b700 1 -- 172.21.15.40:0/1702857057 reap_dead start 2024-09-13T18:03:56.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.494+0000 7f5d3a83c700 1 -- 172.21.15.40:0/1702857057 shutdown_connections 2024-09-13T18:03:56.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.494+0000 7f5d3a83c700 1 -- 172.21.15.40:0/1702857057 >> 172.21.15.40:0/1702857057 conn(0x7f5d2c054120 msgr2=0x7f5d2c0d2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:56.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.498+0000 7f5d3a83c700 1 -- 172.21.15.40:0/1702857057 shutdown_connections 2024-09-13T18:03:56.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.498+0000 7f5d3a83c700 1 -- 172.21.15.40:0/1702857057 wait complete. 2024-09-13T18:03:56.524 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:03:56.524 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:03:56.524 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:03:56.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.634+0000 7fad8094f700 1 Processor -- start 2024-09-13T18:03:56.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.634+0000 7fad8094f700 1 -- start start 2024-09-13T18:03:56.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.634+0000 7fad8094f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad7c153840 0x7fad7c0c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.634+0000 7fad8094f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fad7c0c9e20 0x7fad7c0cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.634+0000 7fad8094f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad7c0cc060 con 0x7fad7c0cc4f0 2024-09-13T18:03:56.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.634+0000 7fad8094f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad7c0b2840 con 0x7fad7c0c9e20 2024-09-13T18:03:56.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.634+0000 7fad8094f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c0ca870 con 0x7fad7c153840 2024-09-13T18:03:56.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.634+0000 7fad7b59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fad7c0c9e20 0x7fad7c0cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.634+0000 7fad7b59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fad7c0c9e20 0x7fad7c0cf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48526/0 (socket says 172.21.15.40:48526) 2024-09-13T18:03:56.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.634+0000 7fad7a59c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fad7c0cc4f0 0x7fad7c15d410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42548/0 (socket says 172.21.15.40:42548) 2024-09-13T18:03:56.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.634+0000 7fad7b59e700 1 -- 172.21.15.40:0/1153651446 learned_addr learned my addr 172.21.15.40:0/1153651446 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:56.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.634+0000 7fad7ad9d700 1 --2- 172.21.15.40:0/1153651446 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad7c153840 0x7fad7c0c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.634+0000 7fad7b59e700 1 -- 172.21.15.40:0/1153651446 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad7c153840 msgr2=0x7fad7c0c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:03:56.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.634+0000 7fad7b59e700 1 --2- 172.21.15.40:0/1153651446 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad7c153840 0x7fad7c0c3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.634+0000 7fad7b59e700 1 -- 172.21.15.40:0/1153651446 >> v1:172.21.15.40:6789/0 conn(0x7fad7c0cc4f0 legacy=0x7fad7c15d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.634+0000 7fad7b59e700 1 -- 172.21.15.40:0/1153651446 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c153c20 con 0x7fad7c0c9e20 2024-09-13T18:03:56.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.634+0000 7fad7b59e700 1 --2- 172.21.15.40:0/1153651446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad7c0c9e20 0x7fad7c0cf260 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fad74009e50 tx=0x7fad7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd12dfdad0049570 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:56.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.634+0000 7fad79d9b700 1 -- 172.21.15.40:0/1153651446 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7400e070 con 0x7fad7c0c9e20 2024-09-13T18:03:56.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.634+0000 7fad79d9b700 1 -- 172.21.15.40:0/1153651446 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7400b910 con 0x7fad7c0c9e20 2024-09-13T18:03:56.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.634+0000 7fad79d9b700 1 -- 172.21.15.40:0/1153651446 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7400bcc0 con 0x7fad7c0c9e20 2024-09-13T18:03:56.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad8094f700 1 -- 172.21.15.40:0/1153651446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad7c0c9e20 msgr2=0x7fad7c0cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad8094f700 1 --2- 172.21.15.40:0/1153651446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad7c0c9e20 0x7fad7c0cf260 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fad74009e50 tx=0x7fad7400b330 comp rx=0 tx=0).stop 2024-09-13T18:03:56.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad8094f700 1 -- 172.21.15.40:0/1153651446 shutdown_connections 2024-09-13T18:03:56.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad8094f700 1 --2- 172.21.15.40:0/1153651446 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad7c0c9e20 0x7fad7c0cf260 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad8094f700 1 --2- 172.21.15.40:0/1153651446 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad7c153840 0x7fad7c0c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad8094f700 1 -- 172.21.15.40:0/1153651446 >> 172.21.15.40:0/1153651446 conn(0x7fad7c06c800 msgr2=0x7fad7c06cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:56.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad8094f700 1 -- 172.21.15.40:0/1153651446 shutdown_connections 2024-09-13T18:03:56.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad8094f700 1 -- 172.21.15.40:0/1153651446 wait complete. 2024-09-13T18:03:56.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad8094f700 1 Processor -- start 2024-09-13T18:03:56.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad8094f700 1 -- start start 2024-09-13T18:03:56.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad8094f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad7c0c9e20 0x7fad7c1e3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad8094f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad7c0cc4f0 0x7fad7c155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad8094f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad7c153840 0x7fad7c156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad8094f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c14e5c0 con 0x7fad7c0cc4f0 2024-09-13T18:03:56.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad8094f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c14e180 con 0x7fad7c0c9e20 2024-09-13T18:03:56.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad8094f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c1f2a20 con 0x7fad7c153840 2024-09-13T18:03:56.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad7a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad7c0cc4f0 0x7fad7c155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad7a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad7c0cc4f0 0x7fad7c155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48528/0 (socket says 172.21.15.40:48528) 2024-09-13T18:03:56.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad7a59c700 1 -- 172.21.15.40:0/1783411649 learned_addr learned my addr 172.21.15.40:0/1783411649 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:03:56.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad7ad9d700 1 --2- 172.21.15.40:0/1783411649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad7c0c9e20 0x7fad7c1e3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad7b59e700 1 --2- 172.21.15.40:0/1783411649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad7c153840 0x7fad7c156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad7a59c700 1 -- 172.21.15.40:0/1783411649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad7c153840 msgr2=0x7fad7c156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad7a59c700 1 --2- 172.21.15.40:0/1783411649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad7c153840 0x7fad7c156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad7a59c700 1 -- 172.21.15.40:0/1783411649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad7c0c9e20 msgr2=0x7fad7c1e3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad7a59c700 1 --2- 172.21.15.40:0/1783411649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad7c0c9e20 0x7fad7c1e3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad7a59c700 1 -- 172.21.15.40:0/1783411649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad74009b30 con 0x7fad7c0cc4f0 2024-09-13T18:03:56.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad7b59e700 1 --2- 172.21.15.40:0/1783411649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad7c153840 0x7fad7c156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:03:56.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad7ad9d700 1 --2- 172.21.15.40:0/1783411649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad7c0c9e20 0x7fad7c1e3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:03:56.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.638+0000 7fad7a59c700 1 --2- 172.21.15.40:0/1783411649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad7c0cc4f0 0x7fad7c155c20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fad7000c6b0 tx=0x7fad7000ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:56.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.642+0000 7fad637fe700 1 -- 172.21.15.40:0/1783411649 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70005cd0 con 0x7fad7c0cc4f0 2024-09-13T18:03:56.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.642+0000 7fad637fe700 1 -- 172.21.15.40:0/1783411649 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad70005e30 con 0x7fad7c0cc4f0 2024-09-13T18:03:56.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.642+0000 7fad8094f700 1 -- 172.21.15.40:0/1783411649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7c1f2d00 con 0x7fad7c0cc4f0 2024-09-13T18:03:56.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.642+0000 7fad637fe700 1 -- 172.21.15.40:0/1783411649 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad700072d0 con 0x7fad7c0cc4f0 2024-09-13T18:03:56.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.642+0000 7fad8094f700 1 -- 172.21.15.40:0/1783411649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7c1f32c0 con 0x7fad7c0cc4f0 2024-09-13T18:03:56.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.642+0000 7fad8094f700 1 -- 172.21.15.40:0/1783411649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad7c0cbcf0 con 0x7fad7c0cc4f0 2024-09-13T18:03:56.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.642+0000 7fad637fe700 1 -- 172.21.15.40:0/1783411649 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fad70007430 con 0x7fad7c0cc4f0 2024-09-13T18:03:56.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.642+0000 7fad637fe700 1 --2- 172.21.15.40:0/1783411649 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad68066730 0x7fad68068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:03:56.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.642+0000 7fad637fe700 1 -- 172.21.15.40:0/1783411649 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fad70083900 con 0x7fad7c0cc4f0 2024-09-13T18:03:56.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.642+0000 7fad7ad9d700 1 --2- 172.21.15.40:0/1783411649 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad68066730 0x7fad68068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:03:56.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.642+0000 7fad7ad9d700 1 --2- 172.21.15.40:0/1783411649 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad68066730 0x7fad68068bf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009ba0 tx=0x7fad6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:03:56.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.650+0000 7fad637fe700 1 -- 172.21.15.40:0/1783411649 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fad70050960 con 0x7fad7c0cc4f0 2024-09-13T18:03:56.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.890+0000 7fad8094f700 1 -- 172.21.15.40:0/1783411649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fad7c0b4ee0 con 0x7fad7c0cc4f0 2024-09-13T18:03:56.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.894+0000 7fad637fe700 1 -- 172.21.15.40:0/1783411649 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fad70017080 con 0x7fad7c0cc4f0 2024-09-13T18:03:56.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.898+0000 7fad8094f700 1 -- 172.21.15.40:0/1783411649 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad68066730 msgr2=0x7fad68068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.898+0000 7fad8094f700 1 --2- 172.21.15.40:0/1783411649 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad68066730 0x7fad68068bf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009ba0 tx=0x7fad6c008040 comp rx=0 tx=0).stop 2024-09-13T18:03:56.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.898+0000 7fad8094f700 1 -- 172.21.15.40:0/1783411649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad7c0cc4f0 msgr2=0x7fad7c155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:03:56.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.898+0000 7fad8094f700 1 --2- 172.21.15.40:0/1783411649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad7c0cc4f0 0x7fad7c155c20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fad7000c6b0 tx=0x7fad7000ca70 comp rx=0 tx=0).stop 2024-09-13T18:03:56.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.898+0000 7fad8094f700 1 -- 172.21.15.40:0/1783411649 shutdown_connections 2024-09-13T18:03:56.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.898+0000 7fad8094f700 1 --2- 172.21.15.40:0/1783411649 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad68066730 0x7fad68068bf0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.898+0000 7fad8094f700 1 --2- 172.21.15.40:0/1783411649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad7c153840 0x7fad7c156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.898+0000 7fad8094f700 1 --2- 172.21.15.40:0/1783411649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad7c0cc4f0 0x7fad7c155c20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.898+0000 7fad8094f700 1 --2- 172.21.15.40:0/1783411649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad7c0c9e20 0x7fad7c1e3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:03:56.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.898+0000 7fad8094f700 1 -- 172.21.15.40:0/1783411649 >> 172.21.15.40:0/1783411649 conn(0x7fad7c06c800 msgr2=0x7fad7c152b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:03:56.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.898+0000 7fad8094f700 1 -- 172.21.15.40:0/1783411649 shutdown_connections 2024-09-13T18:03:56.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:03:56.898+0000 7fad8094f700 1 -- 172.21.15.40:0/1783411649 wait complete. 2024-09-13T18:03:56.922 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:04:06.078 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:04:06.081 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:04:06.923 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:04:06.923 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:04:07.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.038+0000 7ff169893700 1 Processor -- start 2024-09-13T18:04:07.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.038+0000 7ff169893700 1 -- start start 2024-09-13T18:04:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.038+0000 7ff169893700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff16406b4a0 0x7ff164071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.038+0000 7ff169893700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff164072130 0x7ff164160970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.038+0000 7ff169893700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff16415a520 con 0x7ff1641503d0 2024-09-13T18:04:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.038+0000 7ff169893700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff164157050 con 0x7ff164072130 2024-09-13T18:04:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.038+0000 7ff169893700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1641564b0 con 0x7ff16406b4a0 2024-09-13T18:04:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.038+0000 7ff169092700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff164072130 0x7ff164160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.038+0000 7ff168891700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff1641503d0 0x7ff164156940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33582/0 (socket says 172.21.15.40:33582) 2024-09-13T18:04:07.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.038+0000 7ff168891700 1 -- 172.21.15.40:0/4169938713 learned_addr learned my addr 172.21.15.40:0/4169938713 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:04:07.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.038+0000 7ff163fff700 1 --2- 172.21.15.40:0/4169938713 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff16406b4a0 0x7ff164071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.038+0000 7ff169092700 1 -- 172.21.15.40:0/4169938713 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff16406b4a0 msgr2=0x7ff164071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:04:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.038+0000 7ff169092700 1 --2- 172.21.15.40:0/4169938713 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff16406b4a0 0x7ff164071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.038+0000 7ff169092700 1 -- 172.21.15.40:0/4169938713 >> v1:172.21.15.40:6789/0 conn(0x7ff1641503d0 legacy=0x7ff164156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.038+0000 7ff1637fe700 1 -- 172.21.15.40:0/4169938713 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897056731 0 0) 0x7ff16415a520 con 0x7ff1641503d0 2024-09-13T18:04:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.038+0000 7ff169092700 1 -- 172.21.15.40:0/4169938713 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16415a000 con 0x7ff164072130 2024-09-13T18:04:07.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.042+0000 7ff169092700 1 --2- 172.21.15.40:0/4169938713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff164072130 0x7ff164160970 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009d90 tx=0x7ff15c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa0665146baab9e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.042+0000 7ff1637fe700 1 -- 172.21.15.40:0/4169938713 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c00e070 con 0x7ff164072130 2024-09-13T18:04:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.042+0000 7ff1637fe700 1 -- 172.21.15.40:0/4169938713 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15c00bb20 con 0x7ff164072130 2024-09-13T18:04:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.042+0000 7ff1637fe700 1 -- 172.21.15.40:0/4169938713 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c012430 con 0x7ff164072130 2024-09-13T18:04:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.042+0000 7ff169893700 1 -- 172.21.15.40:0/4169938713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff164072130 msgr2=0x7ff164160970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.042+0000 7ff169893700 1 --2- 172.21.15.40:0/4169938713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff164072130 0x7ff164160970 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009d90 tx=0x7ff15c00b330 comp rx=0 tx=0).stop 2024-09-13T18:04:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.042+0000 7ff169893700 1 -- 172.21.15.40:0/4169938713 shutdown_connections 2024-09-13T18:04:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.042+0000 7ff169893700 1 --2- 172.21.15.40:0/4169938713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff164072130 0x7ff164160970 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.042+0000 7ff169893700 1 --2- 172.21.15.40:0/4169938713 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff16406b4a0 0x7ff164071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.042+0000 7ff169893700 1 -- 172.21.15.40:0/4169938713 >> 172.21.15.40:0/4169938713 conn(0x7ff16406e4f0 msgr2=0x7ff16406e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.042+0000 7ff169893700 1 -- 172.21.15.40:0/4169938713 shutdown_connections 2024-09-13T18:04:07.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.042+0000 7ff169893700 1 -- 172.21.15.40:0/4169938713 wait complete. 2024-09-13T18:04:07.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.042+0000 7ff169893700 1 Processor -- start 2024-09-13T18:04:07.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.042+0000 7ff169893700 1 -- start start 2024-09-13T18:04:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff169893700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff16406b4a0 0x7ff164159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff169893700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff164072130 0x7ff164154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff169893700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1641503d0 0x7ff1641553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff169893700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1640c5820 con 0x7ff16406b4a0 2024-09-13T18:04:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff169893700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16406f3b0 con 0x7ff164072130 2024-09-13T18:04:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff169893700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1641549d0 con 0x7ff1641503d0 2024-09-13T18:04:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff168891700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff16406b4a0 0x7ff164159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff168891700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff16406b4a0 0x7ff164159030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56014/0 (socket says 172.21.15.40:56014) 2024-09-13T18:04:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff168891700 1 -- 172.21.15.40:0/1605213756 learned_addr learned my addr 172.21.15.40:0/1605213756 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:07.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff169092700 1 --2- 172.21.15.40:0/1605213756 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1641503d0 0x7ff1641553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff163fff700 1 --2- 172.21.15.40:0/1605213756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff164072130 0x7ff164154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff168891700 1 -- 172.21.15.40:0/1605213756 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1641503d0 msgr2=0x7ff1641553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff168891700 1 --2- 172.21.15.40:0/1605213756 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1641503d0 0x7ff1641553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff168891700 1 -- 172.21.15.40:0/1605213756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff164072130 msgr2=0x7ff164154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff168891700 1 --2- 172.21.15.40:0/1605213756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff164072130 0x7ff164154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff168891700 1 -- 172.21.15.40:0/1605213756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c009a70 con 0x7ff16406b4a0 2024-09-13T18:04:07.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff168891700 1 --2- 172.21.15.40:0/1605213756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff16406b4a0 0x7ff164159030 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff15800c510 tx=0x7ff15800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:07.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff1617fa700 1 -- 172.21.15.40:0/1605213756 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff158004ca0 con 0x7ff16406b4a0 2024-09-13T18:04:07.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff1617fa700 1 -- 172.21.15.40:0/1605213756 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff158004e20 con 0x7ff16406b4a0 2024-09-13T18:04:07.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff169893700 1 -- 172.21.15.40:0/1605213756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1641540e0 con 0x7ff16406b4a0 2024-09-13T18:04:07.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff1617fa700 1 -- 172.21.15.40:0/1605213756 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff158007310 con 0x7ff16406b4a0 2024-09-13T18:04:07.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff169893700 1 -- 172.21.15.40:0/1605213756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff164154380 con 0x7ff16406b4a0 2024-09-13T18:04:07.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.046+0000 7ff169893700 1 -- 172.21.15.40:0/1605213756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1641507b0 con 0x7ff16406b4a0 2024-09-13T18:04:07.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.050+0000 7ff1617fa700 1 -- 172.21.15.40:0/1605213756 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff158007470 con 0x7ff16406b4a0 2024-09-13T18:04:07.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.050+0000 7ff1617fa700 1 --2- 172.21.15.40:0/1605213756 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff150066730 0x7ff150068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.050+0000 7ff1617fa700 1 -- 172.21.15.40:0/1605213756 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff158084280 con 0x7ff16406b4a0 2024-09-13T18:04:07.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.050+0000 7ff163fff700 1 --2- 172.21.15.40:0/1605213756 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff150066730 0x7ff150068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.050+0000 7ff163fff700 1 --2- 172.21.15.40:0/1605213756 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff150066730 0x7ff150068bf0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff1640c6240 tx=0x7ff154008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:07.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.058+0000 7ff1617fa700 1 -- 172.21.15.40:0/1605213756 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff1580511a0 con 0x7ff16406b4a0 2024-09-13T18:04:07.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.326+0000 7ff169893700 1 -- 172.21.15.40:0/1605213756 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff16406b110 con 0x7ff16406b4a0 2024-09-13T18:04:07.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.326+0000 7ff1617fa700 1 -- 172.21.15.40:0/1605213756 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7ff158007720 con 0x7ff16406b4a0 2024-09-13T18:04:07.337 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":108,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:04:07.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.330+0000 7ff169893700 1 -- 172.21.15.40:0/1605213756 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff150066730 msgr2=0x7ff150068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.330+0000 7ff169893700 1 --2- 172.21.15.40:0/1605213756 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff150066730 0x7ff150068bf0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff1640c6240 tx=0x7ff154008040 comp rx=0 tx=0).stop 2024-09-13T18:04:07.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.330+0000 7ff169893700 1 -- 172.21.15.40:0/1605213756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff16406b4a0 msgr2=0x7ff164159030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.330+0000 7ff169893700 1 --2- 172.21.15.40:0/1605213756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff16406b4a0 0x7ff164159030 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff15800c510 tx=0x7ff15800c820 comp rx=0 tx=0).stop 2024-09-13T18:04:07.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.330+0000 7ff169893700 1 -- 172.21.15.40:0/1605213756 shutdown_connections 2024-09-13T18:04:07.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.330+0000 7ff169893700 1 --2- 172.21.15.40:0/1605213756 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff150066730 0x7ff150068bf0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.330+0000 7ff169893700 1 --2- 172.21.15.40:0/1605213756 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1641503d0 0x7ff1641553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.330+0000 7ff169893700 1 --2- 172.21.15.40:0/1605213756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff164072130 0x7ff164154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.330+0000 7ff169893700 1 --2- 172.21.15.40:0/1605213756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff16406b4a0 0x7ff164159030 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.330+0000 7ff169893700 1 -- 172.21.15.40:0/1605213756 >> 172.21.15.40:0/1605213756 conn(0x7ff16406e4f0 msgr2=0x7ff16406a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:07.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.330+0000 7ff169893700 1 -- 172.21.15.40:0/1605213756 shutdown_connections 2024-09-13T18:04:07.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.330+0000 7ff169893700 1 -- 172.21.15.40:0/1605213756 wait complete. 2024-09-13T18:04:07.352 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:04:07.352 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T18:04:07.352 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:04:07.353 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:04:07.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.458+0000 7f723cccc700 1 Processor -- start 2024-09-13T18:04:07.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.458+0000 7f723cccc700 1 -- start start 2024-09-13T18:04:07.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.458+0000 7f723cccc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f72300d69c0 0x7f7230007560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.458+0000 7f723cccc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f723004ca90 0x7f7230033e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.458+0000 7f723cccc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7230007ba0 0x7f72300343b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.458+0000 7f723cccc700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72300368b0 con 0x7f72300d69c0 2024-09-13T18:04:07.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.458+0000 7f723cccc700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72300dc510 con 0x7f723004ca90 2024-09-13T18:04:07.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.458+0000 7f723cccc700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7230005fa0 con 0x7f7230007ba0 2024-09-13T18:04:07.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.458+0000 7f72377fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f72300d69c0 0x7f7230007560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f72377fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f72300d69c0 0x7f7230007560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56016/0 (socket says 172.21.15.40:56016) 2024-09-13T18:04:07.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f72377fe700 1 -- 172.21.15.40:0/2446108250 learned_addr learned my addr 172.21.15.40:0/2446108250 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:07.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f7236ffd700 1 --2- 172.21.15.40:0/2446108250 >> v2:172.21.15.43:3300/0 conn(0x7f723004ca90 0x7f7230033e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f7237fff700 1 --2- 172.21.15.40:0/2446108250 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7230007ba0 0x7f72300343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f72377fe700 1 -- 172.21.15.40:0/2446108250 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7230007ba0 msgr2=0x7f72300343b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:04:07.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f72377fe700 1 --2- 172.21.15.40:0/2446108250 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7230007ba0 0x7f72300343b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f72377fe700 1 -- 172.21.15.40:0/2446108250 >> v2:172.21.15.43:3300/0 conn(0x7f723004ca90 msgr2=0x7f7230033e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:04:07.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f72377fe700 1 --2- 172.21.15.40:0/2446108250 >> v2:172.21.15.43:3300/0 conn(0x7f723004ca90 0x7f7230033e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f72377fe700 1 -- 172.21.15.40:0/2446108250 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7230036080 con 0x7f72300d69c0 2024-09-13T18:04:07.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f7236ffd700 1 --2- 172.21.15.40:0/2446108250 >> v2:172.21.15.43:3300/0 conn(0x7f723004ca90 0x7f7230033e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:04:07.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f72377fe700 1 --2- 172.21.15.40:0/2446108250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72300d69c0 0x7f7230007560 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7230035190 tx=0x7f722c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be51e41c6b695626 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:07.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f72367fc700 1 -- 172.21.15.40:0/2446108250 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722c005e50 con 0x7f72300d69c0 2024-09-13T18:04:07.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f72367fc700 1 -- 172.21.15.40:0/2446108250 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722c006030 con 0x7f72300d69c0 2024-09-13T18:04:07.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f72367fc700 1 -- 172.21.15.40:0/2446108250 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722c0063c0 con 0x7f72300d69c0 2024-09-13T18:04:07.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f723cccc700 1 -- 172.21.15.40:0/2446108250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72300d69c0 msgr2=0x7f7230007560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f723cccc700 1 --2- 172.21.15.40:0/2446108250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72300d69c0 0x7f7230007560 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7230035190 tx=0x7f722c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f723cccc700 1 -- 172.21.15.40:0/2446108250 shutdown_connections 2024-09-13T18:04:07.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f723cccc700 1 --2- 172.21.15.40:0/2446108250 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7230007ba0 0x7f72300343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f723cccc700 1 --2- 172.21.15.40:0/2446108250 >> v2:172.21.15.43:3300/0 conn(0x7f723004ca90 0x7f7230033e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f723cccc700 1 --2- 172.21.15.40:0/2446108250 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72300d69c0 0x7f7230007560 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f723cccc700 1 -- 172.21.15.40:0/2446108250 >> 172.21.15.40:0/2446108250 conn(0x7f72300505e0 msgr2=0x7f72300509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:07.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f723cccc700 1 -- 172.21.15.40:0/2446108250 shutdown_connections 2024-09-13T18:04:07.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f723cccc700 1 -- 172.21.15.40:0/2446108250 wait complete. 2024-09-13T18:04:07.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f723cccc700 1 Processor -- start 2024-09-13T18:04:07.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.462+0000 7f723cccc700 1 -- start start 2024-09-13T18:04:07.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f723cccc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7230007ba0 0x7f72300051a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f723cccc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723004ca90 0x7f72301724b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f723cccc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72300d69c0 0x7f72300cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f723cccc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72300d7ff0 con 0x7f723004ca90 2024-09-13T18:04:07.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f723cccc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7230171d10 con 0x7f7230007ba0 2024-09-13T18:04:07.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f723cccc700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7230049470 con 0x7f72300d69c0 2024-09-13T18:04:07.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f7236ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723004ca90 0x7f72301724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f7236ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723004ca90 0x7f72301724b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56028/0 (socket says 172.21.15.40:56028) 2024-09-13T18:04:07.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f7236ffd700 1 -- 172.21.15.40:0/2358716418 learned_addr learned my addr 172.21.15.40:0/2358716418 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:07.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f72377fe700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7230007ba0 0x7f72300051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f7237fff700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72300d69c0 0x7f72300cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f7236ffd700 1 -- 172.21.15.40:0/2358716418 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72300d69c0 msgr2=0x7f72300cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f7236ffd700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72300d69c0 0x7f72300cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f7236ffd700 1 -- 172.21.15.40:0/2358716418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7230007ba0 msgr2=0x7f72300051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f7236ffd700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7230007ba0 0x7f72300051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f7236ffd700 1 -- 172.21.15.40:0/2358716418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c009520 con 0x7f723004ca90 2024-09-13T18:04:07.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f7236ffd700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723004ca90 0x7f72301724b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f722400e760 tx=0x7f722400ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:07.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f7223fff700 1 -- 172.21.15.40:0/2358716418 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722400c070 con 0x7f723004ca90 2024-09-13T18:04:07.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f7223fff700 1 -- 172.21.15.40:0/2358716418 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7224004250 con 0x7f723004ca90 2024-09-13T18:04:07.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f723cccc700 1 -- 172.21.15.40:0/2358716418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7230049750 con 0x7f723004ca90 2024-09-13T18:04:07.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f7223fff700 1 -- 172.21.15.40:0/2358716418 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72240036c0 con 0x7f723004ca90 2024-09-13T18:04:07.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f723cccc700 1 -- 172.21.15.40:0/2358716418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7230049d10 con 0x7f723004ca90 2024-09-13T18:04:07.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f723cccc700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72301729f0 0x7f7230171ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f723cccc700 1 -- 172.21.15.40:0/2358716418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7230008d50 con 0x7f72301729f0 2024-09-13T18:04:07.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.466+0000 7f72377fe700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72301729f0 0x7f7230171ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.470+0000 7f72377fe700 1 -- 172.21.15.40:0/2358716418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f72300d0ac0 con 0x7f72301729f0 2024-09-13T18:04:07.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.470+0000 7f7223fff700 1 -- 172.21.15.40:0/2358716418 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7224003820 con 0x7f723004ca90 2024-09-13T18:04:07.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.470+0000 7f72377fe700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72301729f0 0x7f7230171ed0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7230035190 tx=0x7f722c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:07.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.470+0000 7f7223fff700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f721c066760 0x7f721c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.470+0000 7f7223fff700 1 -- 172.21.15.40:0/2358716418 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f72240842d0 con 0x7f723004ca90 2024-09-13T18:04:07.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.470+0000 7f7237fff700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f721c066760 0x7f721c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.470+0000 7f7223fff700 1 -- 172.21.15.40:0/2358716418 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722c022070 con 0x7f72301729f0 2024-09-13T18:04:07.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.470+0000 7f7223fff700 1 -- 172.21.15.40:0/2358716418 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f722c0039a0 con 0x7f72301729f0 2024-09-13T18:04:07.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.470+0000 7f7223fff700 1 -- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72301729f0 msgr2=0x7f7230171ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.470+0000 7f7223fff700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72301729f0 0x7f7230171ed0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7230035190 tx=0x7f722c004770 comp rx=0 tx=0).stop 2024-09-13T18:04:07.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.470+0000 7f7223fff700 1 -- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72301729f0 msgr2=0x7f7230171ed0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:04:07.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.470+0000 7f7223fff700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72301729f0 0x7f7230171ed0 secure :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7230035190 tx=0x7f722c004770 comp rx=0 tx=0).stop 2024-09-13T18:04:07.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.470+0000 7f7237fff700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f721c066760 0x7f721c068c20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f72380641b0 tx=0x7f723808e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:07.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.478+0000 7f723cccc700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723004a2d0 0x7f72300d0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.478+0000 7f723cccc700 1 -- 172.21.15.40:0/2358716418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72300362f0 con 0x7f723004a2d0 2024-09-13T18:04:07.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.478+0000 7f72377fe700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723004a2d0 0x7f72300d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.478+0000 7f72377fe700 1 -- 172.21.15.40:0/2358716418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7230034b20 con 0x7f723004a2d0 2024-09-13T18:04:07.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.482+0000 7f72377fe700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723004a2d0 0x7f72300d0320 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f722c013040 tx=0x7f722c008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:07.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.482+0000 7f7223fff700 1 -- 172.21.15.40:0/2358716418 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722c032070 con 0x7f723004a2d0 2024-09-13T18:04:07.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.482+0000 7f7223fff700 1 -- 172.21.15.40:0/2358716418 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4951 (secure 0 0 0) 0x7f722c02d2c0 con 0x7f723004a2d0 2024-09-13T18:04:07.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.482+0000 7f7223fff700 1 -- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723004a2d0 msgr2=0x7f72300d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.482+0000 7f7223fff700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723004a2d0 0x7f72300d0320 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f722c013040 tx=0x7f722c008000 comp rx=0 tx=0).stop 2024-09-13T18:04:07.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.482+0000 7f7223fff700 1 -- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723004a2d0 msgr2=0x7f72300d0320 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:04:07.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.482+0000 7f7223fff700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723004a2d0 0x7f72300d0320 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.492 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:04:07.493 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:04:07.493 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:04:07.493 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:04:07.493 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 108, 2024-09-13T18:04:07.493 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:04:07.493 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:04:07.493 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:04:07.493 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:04:07.493 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:07.493 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 14, 2024-09-13T18:04:07.493 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:04:07.493 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:04:07.493 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:04:07.494 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:07.494 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:07.494 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:07.494 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:07.494 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:07.494 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:07.494 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:07.494 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:07.494 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:07.494 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:07.494 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:07.494 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:04:07.495 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:04:07.495 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:07.495 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:07.495 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:07.495 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:07.495 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:07.495 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:07.495 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:07.495 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:07.495 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:07.495 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:07.495 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:07.495 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.495 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:04:07.496 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:04:07.496 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:04:07.496 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:04:07.496 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:04:07.496 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:04:07.496 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:04:07.496 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:04:07.496 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:04:07.496 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:04:07.496 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:04:07.496 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:04:07.496 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:04:07.496 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:04:07.496 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:04:07.496 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:04:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:04:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:04:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:04:07.497 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:04:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:04:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:04:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:04:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:07.498 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:04:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:04:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:04:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:04:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:07.499 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.500 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:07.500 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:04:07.500 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:07.500 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.500 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.500 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:07.500 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:04:07.500 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:07.500 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.500 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:07.500 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.500 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:04:07.500 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:04:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:04:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:04:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:04:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:07.501 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.502 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.502 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:07.502 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:04:07.502 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:07.502 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.502 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:07.502 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.502 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:04:07.502 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:04:07.502 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:07.502 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:07.502 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:07.502 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:04:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:04:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:04:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:04:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:07.503 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:04:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:04:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 8 2024-09-13T18:04:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:04:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.504 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:04:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:04:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:04:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.505 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:04:07.505 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:04:07.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.482+0000 7f723cccc700 1 -- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f721c066760 msgr2=0x7f721c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.482+0000 7f723cccc700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f721c066760 0x7f721c068c20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f72380641b0 tx=0x7f723808e040 comp rx=0 tx=0).stop 2024-09-13T18:04:07.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.482+0000 7f723cccc700 1 -- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723004ca90 msgr2=0x7f72301724b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.482+0000 7f7237fff700 1 -- 172.21.15.40:0/2358716418 reap_dead start 2024-09-13T18:04:07.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.482+0000 7f723cccc700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723004ca90 0x7f72301724b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f722400e760 tx=0x7f722400ea70 comp rx=0 tx=0).stop 2024-09-13T18:04:07.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.482+0000 7f723cccc700 1 -- 172.21.15.40:0/2358716418 shutdown_connections 2024-09-13T18:04:07.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.482+0000 7f723cccc700 1 --2- 172.21.15.40:0/2358716418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f723004ca90 0x7f72301724b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.482+0000 7f723cccc700 1 -- 172.21.15.40:0/2358716418 >> 172.21.15.40:0/2358716418 conn(0x7f72300505e0 msgr2=0x7f72300db0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:07.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.482+0000 7f723cccc700 1 -- 172.21.15.40:0/2358716418 shutdown_connections 2024-09-13T18:04:07.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.482+0000 7f723cccc700 1 -- 172.21.15.40:0/2358716418 wait complete. 2024-09-13T18:04:07.509 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:04:07.509 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:04:07.509 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:04:07.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.614+0000 7f746dd85700 1 Processor -- start 2024-09-13T18:04:07.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.614+0000 7f746dd85700 1 -- start start 2024-09-13T18:04:07.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f746dd85700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f746814e150 0x7f74680b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f746dd85700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f74680c52b0 0x7f74680c5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f746dd85700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74680cb810 con 0x7f74680ca6b0 2024-09-13T18:04:07.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f746dd85700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74680c1d50 con 0x7f74680c52b0 2024-09-13T18:04:07.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f746dd85700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746815aff0 con 0x7f746814e150 2024-09-13T18:04:07.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f746cd83700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f746814e150 0x7f74680b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f7467fff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f74680ca6b0 0x7f746815a8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:51326/0 (socket says 172.21.15.40:51326) 2024-09-13T18:04:07.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f7467fff700 1 -- 172.21.15.40:0/3691134786 learned_addr learned my addr 172.21.15.40:0/3691134786 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:04:07.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f746d584700 1 --2- 172.21.15.40:0/3691134786 >> v2:172.21.15.43:3300/0 conn(0x7f74680c52b0 0x7f74680c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f74677fe700 1 -- 172.21.15.40:0/3691134786 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 794517002 0 0) 0x7f74680cb810 con 0x7f74680ca6b0 2024-09-13T18:04:07.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f74677fe700 1 -- 172.21.15.40:0/3691134786 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74580035d0 con 0x7f74680ca6b0 2024-09-13T18:04:07.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f746d584700 1 -- 172.21.15.40:0/3691134786 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f746814e150 msgr2=0x7f74680b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:04:07.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f746d584700 1 --2- 172.21.15.40:0/3691134786 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f746814e150 0x7f74680b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f746d584700 1 -- 172.21.15.40:0/3691134786 >> v1:172.21.15.43:6789/0 conn(0x7f74680ca6b0 legacy=0x7f746815a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f746d584700 1 -- 172.21.15.40:0/3691134786 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746814e530 con 0x7f74680c52b0 2024-09-13T18:04:07.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f746d584700 1 --2- 172.21.15.40:0/3691134786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74680c52b0 0x7f74680c5700 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7460009dc0 tx=0x7f746000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b296b3b53d889758 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:07.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f74677fe700 1 -- 172.21.15.40:0/3691134786 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746000e070 con 0x7f74680c52b0 2024-09-13T18:04:07.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f74677fe700 1 -- 172.21.15.40:0/3691134786 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746000bae0 con 0x7f74680c52b0 2024-09-13T18:04:07.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f74677fe700 1 -- 172.21.15.40:0/3691134786 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746000be50 con 0x7f74680c52b0 2024-09-13T18:04:07.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f746dd85700 1 -- 172.21.15.40:0/3691134786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74680c52b0 msgr2=0x7f74680c5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f746dd85700 1 --2- 172.21.15.40:0/3691134786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74680c52b0 0x7f74680c5700 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7460009dc0 tx=0x7f746000b330 comp rx=0 tx=0).stop 2024-09-13T18:04:07.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f746dd85700 1 -- 172.21.15.40:0/3691134786 shutdown_connections 2024-09-13T18:04:07.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f746dd85700 1 --2- 172.21.15.40:0/3691134786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74680c52b0 0x7f74680c5700 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f746dd85700 1 --2- 172.21.15.40:0/3691134786 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f746814e150 0x7f74680b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f746dd85700 1 -- 172.21.15.40:0/3691134786 >> 172.21.15.40:0/3691134786 conn(0x7f746806e710 msgr2=0x7f746806eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:07.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f746dd85700 1 -- 172.21.15.40:0/3691134786 shutdown_connections 2024-09-13T18:04:07.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.618+0000 7f746dd85700 1 -- 172.21.15.40:0/3691134786 wait complete. 2024-09-13T18:04:07.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.622+0000 7f746dd85700 1 Processor -- start 2024-09-13T18:04:07.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.622+0000 7f746dd85700 1 -- start start 2024-09-13T18:04:07.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f746dd85700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f74680c52b0 0x7f746815a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f746dd85700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74680ca6b0 0x7f7468158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f746dd85700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f746814e150 0x7f746813e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f746dd85700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746806b850 con 0x7f74680ca6b0 2024-09-13T18:04:07.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f746dd85700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746815a550 con 0x7f746814e150 2024-09-13T18:04:07.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f746dd85700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746806ba80 con 0x7f74680c52b0 2024-09-13T18:04:07.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f7467fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74680ca6b0 0x7f7468158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f7467fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74680ca6b0 0x7f7468158930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56062/0 (socket says 172.21.15.40:56062) 2024-09-13T18:04:07.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f7467fff700 1 -- 172.21.15.40:0/372719752 learned_addr learned my addr 172.21.15.40:0/372719752 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:07.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f746d584700 1 --2- 172.21.15.40:0/372719752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f746814e150 0x7f746813e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f746cd83700 1 --2- 172.21.15.40:0/372719752 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f74680c52b0 0x7f746815a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f7467fff700 1 -- 172.21.15.40:0/372719752 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f74680c52b0 msgr2=0x7f746815a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f7467fff700 1 --2- 172.21.15.40:0/372719752 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f74680c52b0 0x7f746815a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f7467fff700 1 -- 172.21.15.40:0/372719752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f746814e150 msgr2=0x7f746813e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f7467fff700 1 --2- 172.21.15.40:0/372719752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f746814e150 0x7f746813e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f7467fff700 1 -- 172.21.15.40:0/372719752 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7460009a70 con 0x7f74680ca6b0 2024-09-13T18:04:07.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f7467fff700 1 --2- 172.21.15.40:0/372719752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74680ca6b0 0x7f7468158930 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f745400c700 tx=0x7f745400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:07.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f74657fa700 1 -- 172.21.15.40:0/372719752 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7454005d90 con 0x7f74680ca6b0 2024-09-13T18:04:07.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f74657fa700 1 -- 172.21.15.40:0/372719752 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74540070d0 con 0x7f74680ca6b0 2024-09-13T18:04:07.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f74657fa700 1 -- 172.21.15.40:0/372719752 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7454007480 con 0x7f74680ca6b0 2024-09-13T18:04:07.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f746dd85700 1 -- 172.21.15.40:0/372719752 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746806bd60 con 0x7f74680ca6b0 2024-09-13T18:04:07.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f746dd85700 1 -- 172.21.15.40:0/372719752 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7468159810 con 0x7f74680ca6b0 2024-09-13T18:04:07.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f746dd85700 1 --2- 172.21.15.40:0/372719752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7468159a30 0x7f746813e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f746dd85700 1 -- 172.21.15.40:0/372719752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7468067600 con 0x7f7468159a30 2024-09-13T18:04:07.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.626+0000 7f746cd83700 1 --2- 172.21.15.40:0/372719752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7468159a30 0x7f746813e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.630+0000 7f746cd83700 1 -- 172.21.15.40:0/372719752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74680cb530 con 0x7f7468159a30 2024-09-13T18:04:07.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.630+0000 7f746cd83700 1 --2- 172.21.15.40:0/372719752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7468159a30 0x7f746813e920 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f745c0119d0 tx=0x7f745c011d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:07.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.630+0000 7f74657fa700 1 -- 172.21.15.40:0/372719752 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f74540075e0 con 0x7f74680ca6b0 2024-09-13T18:04:07.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.630+0000 7f74657fa700 1 --2- 172.21.15.40:0/372719752 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f744c066760 0x7f744c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.630+0000 7f74657fa700 1 -- 172.21.15.40:0/372719752 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7454083970 con 0x7f74680ca6b0 2024-09-13T18:04:07.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.630+0000 7f74657fa700 1 -- 172.21.15.40:0/372719752 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c004240 con 0x7f7468159a30 2024-09-13T18:04:07.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.630+0000 7f746d584700 1 --2- 172.21.15.40:0/372719752 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f744c066760 0x7f744c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.630+0000 7f74657fa700 1 -- 172.21.15.40:0/372719752 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f745c0043a0 con 0x7f7468159a30 2024-09-13T18:04:07.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.630+0000 7f74657fa700 1 -- 172.21.15.40:0/372719752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7468159a30 msgr2=0x7f746813e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.630+0000 7f74657fa700 1 --2- 172.21.15.40:0/372719752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7468159a30 0x7f746813e920 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f745c0119d0 tx=0x7f745c011d90 comp rx=0 tx=0).stop 2024-09-13T18:04:07.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.630+0000 7f74657fa700 1 -- 172.21.15.40:0/372719752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7468159a30 msgr2=0x7f746813e920 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:04:07.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.630+0000 7f74657fa700 1 --2- 172.21.15.40:0/372719752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7468159a30 0x7f746813e920 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f745c0119d0 tx=0x7f745c011d90 comp rx=0 tx=0).stop 2024-09-13T18:04:07.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.630+0000 7f746d584700 1 --2- 172.21.15.40:0/372719752 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f744c066760 0x7f744c068c20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f746000b370 tx=0x7f746001d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:07.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.638+0000 7f746dd85700 1 --2- 172.21.15.40:0/372719752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74681ed0b0 0x7f74681f0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.638+0000 7f746dd85700 1 -- 172.21.15.40:0/372719752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74680cb290 con 0x7f74681ed0b0 2024-09-13T18:04:07.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.642+0000 7f746cd83700 1 --2- 172.21.15.40:0/372719752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74681ed0b0 0x7f74681f0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.642+0000 7f746cd83700 1 -- 172.21.15.40:0/372719752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f74680c2740 con 0x7f74681ed0b0 2024-09-13T18:04:07.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.642+0000 7f746cd83700 1 --2- 172.21.15.40:0/372719752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74681ed0b0 0x7f74681f0620 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f745c01b000 tx=0x7f745c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:07.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.642+0000 7f74657fa700 1 -- 172.21.15.40:0/372719752 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c00fcc0 con 0x7f74681ed0b0 2024-09-13T18:04:07.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.642+0000 7f74657fa700 1 -- 172.21.15.40:0/372719752 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4617 (secure 0 0 0) 0x7f745c00fe20 con 0x7f74681ed0b0 2024-09-13T18:04:07.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.642+0000 7f74657fa700 1 -- 172.21.15.40:0/372719752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74681ed0b0 msgr2=0x7f74681f0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.642+0000 7f74657fa700 1 --2- 172.21.15.40:0/372719752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74681ed0b0 0x7f74681f0620 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f745c01b000 tx=0x7f745c004770 comp rx=0 tx=0).stop 2024-09-13T18:04:07.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.642+0000 7f74657fa700 1 -- 172.21.15.40:0/372719752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74681ed0b0 msgr2=0x7f74681f0620 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:04:07.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.642+0000 7f74657fa700 1 --2- 172.21.15.40:0/372719752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74681ed0b0 0x7f74681f0620 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f745c01b000 tx=0x7f745c004770 comp rx=0 tx=0).stop 2024-09-13T18:04:07.654 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:04:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:04:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:04:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:04:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 108, 2024-09-13T18:04:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:04:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:04:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:04:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:04:07.654 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 14, 2024-09-13T18:04:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:04:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:04:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:04:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:07.655 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:04:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:04:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:07.656 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:04:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:04:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:04:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:04:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:04:07.657 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:04:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:04:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:04:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:04:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:04:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:04:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:04:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:04:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:04:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:04:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:04:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:04:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:07.658 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:04:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.659 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:04:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:04:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:04:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:04:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:04:07.660 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:04:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:04:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:04:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:04:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:07.661 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:04:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:04:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.662 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:04:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:04:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:04:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:04:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:04:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:07.663 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:04:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:04:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:04:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:07.664 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:04:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:04:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:04:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:04:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:07.665 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:04:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:04:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:04:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.666 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:04:07.666 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:04:07.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.642+0000 7f746dd85700 1 -- 172.21.15.40:0/372719752 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f744c066760 msgr2=0x7f744c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.642+0000 7f746dd85700 1 --2- 172.21.15.40:0/372719752 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f744c066760 0x7f744c068c20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f746000b370 tx=0x7f746001d040 comp rx=0 tx=0).stop 2024-09-13T18:04:07.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.642+0000 7f746dd85700 1 -- 172.21.15.40:0/372719752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74680ca6b0 msgr2=0x7f7468158930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.642+0000 7f746dd85700 1 --2- 172.21.15.40:0/372719752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74680ca6b0 0x7f7468158930 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f745400c700 tx=0x7f745400ca10 comp rx=0 tx=0).stop 2024-09-13T18:04:07.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.642+0000 7f746d584700 1 -- 172.21.15.40:0/372719752 reap_dead start 2024-09-13T18:04:07.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.642+0000 7f746d584700 1 -- 172.21.15.40:0/372719752 reap_dead start 2024-09-13T18:04:07.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.642+0000 7f746dd85700 1 -- 172.21.15.40:0/372719752 shutdown_connections 2024-09-13T18:04:07.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.642+0000 7f746dd85700 1 -- 172.21.15.40:0/372719752 >> 172.21.15.40:0/372719752 conn(0x7f746806e710 msgr2=0x7f746806eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:07.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.642+0000 7f746dd85700 1 -- 172.21.15.40:0/372719752 shutdown_connections 2024-09-13T18:04:07.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.642+0000 7f746dd85700 1 -- 172.21.15.40:0/372719752 wait complete. 2024-09-13T18:04:07.668 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:04:07.669 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:04:07.669 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:04:07.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.770+0000 7f962f59e700 1 Processor -- start 2024-09-13T18:04:07.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.770+0000 7f962f59e700 1 -- start start 2024-09-13T18:04:07.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.770+0000 7f962f59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96300cf1a0 0x7f963015a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.770+0000 7f962f59e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f96300cf580 0x7f96300c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.770+0000 7f962f59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f963015a7a0 con 0x7f96300cc250 2024-09-13T18:04:07.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.770+0000 7f962f59e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96300cbdc0 con 0x7f96300cf580 2024-09-13T18:04:07.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.770+0000 7f962f59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9630153f80 con 0x7f96300cf1a0 2024-09-13T18:04:07.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.770+0000 7f962ed9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f96300cc250 0x7f96300c8aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33598/0 (socket says 172.21.15.40:33598) 2024-09-13T18:04:07.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.770+0000 7f962ed9d700 1 -- 172.21.15.40:0/2727866550 learned_addr learned my addr 172.21.15.40:0/2727866550 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:04:07.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.770+0000 7f962dd9b700 1 --2- 172.21.15.40:0/2727866550 >> v2:172.21.15.43:3300/0 conn(0x7f96300cf580 0x7f96300c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.770+0000 7f962e59c700 1 --2- 172.21.15.40:0/2727866550 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96300cf1a0 0x7f963015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.770+0000 7f962d59a700 1 -- 172.21.15.40:0/2727866550 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2326169023 0 0) 0x7f9628001490 con 0x7f96300cc250 2024-09-13T18:04:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.770+0000 7f962d59a700 1 -- 172.21.15.40:0/2727866550 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96180035d0 con 0x7f96300cc250 2024-09-13T18:04:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.770+0000 7f962d59a700 1 -- 172.21.15.40:0/2727866550 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2520758963 0 0) 0x7f96180035d0 con 0x7f96300cc250 2024-09-13T18:04:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962d59a700 1 -- 172.21.15.40:0/2727866550 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96300cf1a0 msgr2=0x7f963015a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:04:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962d59a700 1 --2- 172.21.15.40:0/2727866550 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96300cf1a0 0x7f963015a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962d59a700 1 -- 172.21.15.40:0/2727866550 >> v2:172.21.15.43:3300/0 conn(0x7f96300cf580 msgr2=0x7f96300c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:04:07.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962d59a700 1 --2- 172.21.15.40:0/2727866550 >> v2:172.21.15.43:3300/0 conn(0x7f96300cf580 0x7f96300c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962d59a700 1 -- 172.21.15.40:0/2727866550 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96300caed0 con 0x7f96300cc250 2024-09-13T18:04:07.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962d59a700 1 -- 172.21.15.40:0/2727866550 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f9628001b80 con 0x7f96300cc250 2024-09-13T18:04:07.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962d59a700 1 -- 172.21.15.40:0/2727866550 >> v1:172.21.15.40:6789/0 conn(0x7f96300cc250 legacy=0x7f96300c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962d59a700 1 --2- 172.21.15.40:0/2727866550 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96300cf1a0 0x7f9618003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962d59a700 1 --2- 172.21.15.40:0/2727866550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9618003fa0 0x7f9618004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962d59a700 1 --2- 172.21.15.40:0/2727866550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9618004db0 0x7f9618005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962d59a700 1 -- 172.21.15.40:0/2727866550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96300cbdc0 con 0x7f9618004db0 2024-09-13T18:04:07.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962d59a700 1 -- 172.21.15.40:0/2727866550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9630153f80 con 0x7f9618003fa0 2024-09-13T18:04:07.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962d59a700 1 -- 172.21.15.40:0/2727866550 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961800ba30 con 0x7f96300cf1a0 2024-09-13T18:04:07.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962dd9b700 1 --2- 172.21.15.40:0/2727866550 >> v2:172.21.15.43:3300/0 conn(0x7f96300cf580 0x7f96300c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:04:07.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962ed9d700 1 --2- 172.21.15.40:0/2727866550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9618004db0 0x7f9618005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962e59c700 1 --2- 172.21.15.40:0/2727866550 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96300cf1a0 0x7f9618003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962dd9b700 1 --2- 172.21.15.40:0/2727866550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9618003fa0 0x7f9618004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962ed9d700 1 -- 172.21.15.40:0/2727866550 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96300cf1a0 msgr2=0x7f9618003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:04:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962ed9d700 1 --2- 172.21.15.40:0/2727866550 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96300cf1a0 0x7f9618003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962ed9d700 1 -- 172.21.15.40:0/2727866550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9618003fa0 msgr2=0x7f9618004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:04:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962ed9d700 1 --2- 172.21.15.40:0/2727866550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9618003fa0 0x7f9618004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962ed9d700 1 -- 172.21.15.40:0/2727866550 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96300caed0 con 0x7f9618004db0 2024-09-13T18:04:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962ed9d700 1 --2- 172.21.15.40:0/2727866550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9618004db0 0x7f9618005230 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f96280013d0 tx=0x7f96180058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955814693dea47bb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962d59a700 1 -- 172.21.15.40:0/2727866550 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9628012600 con 0x7f9618004db0 2024-09-13T18:04:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962d59a700 1 -- 172.21.15.40:0/2727866550 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9628012760 con 0x7f9618004db0 2024-09-13T18:04:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962d59a700 1 -- 172.21.15.40:0/2727866550 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9628012af0 con 0x7f9618004db0 2024-09-13T18:04:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962f59e700 1 -- 172.21.15.40:0/2727866550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9618004db0 msgr2=0x7f9618005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962f59e700 1 --2- 172.21.15.40:0/2727866550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9618004db0 0x7f9618005230 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f96280013d0 tx=0x7f96180058a0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962ed9d700 1 -- 172.21.15.40:0/2727866550 reap_dead start 2024-09-13T18:04:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962f59e700 1 -- 172.21.15.40:0/2727866550 shutdown_connections 2024-09-13T18:04:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962f59e700 1 -- 172.21.15.40:0/2727866550 >> 172.21.15.40:0/2727866550 conn(0x7f9630066090 msgr2=0x7f9630066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962f59e700 1 -- 172.21.15.40:0/2727866550 shutdown_connections 2024-09-13T18:04:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.774+0000 7f962f59e700 1 -- 172.21.15.40:0/2727866550 wait complete. 2024-09-13T18:04:07.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962f59e700 1 Processor -- start 2024-09-13T18:04:07.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962f59e700 1 -- start start 2024-09-13T18:04:07.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962f59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9618004db0 0x7f96301547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962f59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96300c2b10 0x7f96300c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962f59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96300c3f10 0x7f96301eefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962f59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96301eeb90 con 0x7f96300c3f10 2024-09-13T18:04:07.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962f59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9630155f50 con 0x7f9618004db0 2024-09-13T18:04:07.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962f59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96300c3070 con 0x7f96300c2b10 2024-09-13T18:04:07.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96300c3f10 0x7f96301eefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96300c3f10 0x7f96301eefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:56076/0 (socket says 172.21.15.40:56076) 2024-09-13T18:04:07.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962ed9d700 1 -- 172.21.15.40:0/179795685 learned_addr learned my addr 172.21.15.40:0/179795685 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:07.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962e59c700 1 --2- 172.21.15.40:0/179795685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9618004db0 0x7f96301547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962dd9b700 1 --2- 172.21.15.40:0/179795685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96300c2b10 0x7f96300c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962ed9d700 1 -- 172.21.15.40:0/179795685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96300c2b10 msgr2=0x7f96300c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962ed9d700 1 --2- 172.21.15.40:0/179795685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96300c2b10 0x7f96300c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962ed9d700 1 -- 172.21.15.40:0/179795685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9618004db0 msgr2=0x7f96301547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962ed9d700 1 --2- 172.21.15.40:0/179795685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9618004db0 0x7f96301547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962ed9d700 1 -- 172.21.15.40:0/179795685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96301553d0 con 0x7f96300c3f10 2024-09-13T18:04:07.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962ed9d700 1 --2- 172.21.15.40:0/179795685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96300c3f10 0x7f96301eefe0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9618004750 tx=0x7f9628001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f961effd700 1 -- 172.21.15.40:0/179795685 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9628023070 con 0x7f96300c3f10 2024-09-13T18:04:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f961effd700 1 -- 172.21.15.40:0/179795685 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9628001490 con 0x7f96300c3f10 2024-09-13T18:04:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962f59e700 1 -- 172.21.15.40:0/179795685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96301556b0 con 0x7f96300c3f10 2024-09-13T18:04:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f961effd700 1 -- 172.21.15.40:0/179795685 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962801e650 con 0x7f96300c3f10 2024-09-13T18:04:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962f59e700 1 -- 172.21.15.40:0/179795685 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9630155c70 con 0x7f96300c3f10 2024-09-13T18:04:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962f59e700 1 --2- 172.21.15.40:0/179795685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96301ef520 0x7f96301f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.778+0000 7f962f59e700 1 -- 172.21.15.40:0/179795685 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9630074750 con 0x7f96301ef520 2024-09-13T18:04:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.782+0000 7f962e59c700 1 --2- 172.21.15.40:0/179795685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96301ef520 0x7f96301f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.782+0000 7f961effd700 1 -- 172.21.15.40:0/179795685 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f962801e7b0 con 0x7f96300c3f10 2024-09-13T18:04:07.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.782+0000 7f962e59c700 1 -- 172.21.15.40:0/179795685 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f963015a380 con 0x7f96301ef520 2024-09-13T18:04:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.782+0000 7f961effd700 1 --2- 172.21.15.40:0/179795685 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9610066730 0x7f9610068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.782+0000 7f961effd700 1 -- 172.21.15.40:0/179795685 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9628085c20 con 0x7f96300c3f10 2024-09-13T18:04:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.782+0000 7f962e59c700 1 --2- 172.21.15.40:0/179795685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96301ef520 0x7f96301f2100 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9620014cf0 tx=0x7f9620014dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.782+0000 7f962dd9b700 1 --2- 172.21.15.40:0/179795685 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9610066730 0x7f9610068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.782+0000 7f961effd700 1 -- 172.21.15.40:0/179795685 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96200129b0 con 0x7f96301ef520 2024-09-13T18:04:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.782+0000 7f962dd9b700 1 --2- 172.21.15.40:0/179795685 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9610066730 0x7f9610068bf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f96300ce1c0 tx=0x7f9624008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.782+0000 7f961effd700 1 -- 172.21.15.40:0/179795685 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9620009ed0 con 0x7f96301ef520 2024-09-13T18:04:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.782+0000 7f961effd700 1 -- 172.21.15.40:0/179795685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96301ef520 msgr2=0x7f96301f2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.782+0000 7f961effd700 1 --2- 172.21.15.40:0/179795685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96301ef520 0x7f96301f2100 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9620014cf0 tx=0x7f9620014dd0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.782+0000 7f961effd700 1 -- 172.21.15.40:0/179795685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96301ef520 msgr2=0x7f96301f2100 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:04:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.782+0000 7f961effd700 1 --2- 172.21.15.40:0/179795685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96301ef520 0x7f96301f2100 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f962f59e700 1 --2- 172.21.15.40:0/179795685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96301f2640 0x7f96301f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:07.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f962f59e700 1 -- 172.21.15.40:0/179795685 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96300cb770 con 0x7f96301f2640 2024-09-13T18:04:07.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f962e59c700 1 --2- 172.21.15.40:0/179795685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96301f2640 0x7f96301f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:07.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f962e59c700 1 -- 172.21.15.40:0/179795685 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f96300cb490 con 0x7f96301f2640 2024-09-13T18:04:07.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f962e59c700 1 --2- 172.21.15.40:0/179795685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96301f2640 0x7f96301f2a50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9620015010 tx=0x7f96200047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:07.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f961effd700 1 -- 172.21.15.40:0/179795685 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9620006240 con 0x7f96301f2640 2024-09-13T18:04:07.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f961effd700 1 -- 172.21.15.40:0/179795685 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4451 (secure 0 0 0) 0x7f96200063a0 con 0x7f96301f2640 2024-09-13T18:04:07.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f961effd700 1 -- 172.21.15.40:0/179795685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96301f2640 msgr2=0x7f96301f2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f961effd700 1 --2- 172.21.15.40:0/179795685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96301f2640 0x7f96301f2a50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9620015010 tx=0x7f96200047e0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f961effd700 1 -- 172.21.15.40:0/179795685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96301f2640 msgr2=0x7f96301f2a50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:04:07.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f961effd700 1 --2- 172.21.15.40:0/179795685 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96301f2640 0x7f96301f2a50 secure :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9620015010 tx=0x7f96200047e0 comp rx=0 tx=0).stop 2024-09-13T18:04:07.806 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:04:07.806 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:04:07.806 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:04:07.806 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:04:07.806 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 108, 2024-09-13T18:04:07.806 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:04:07.806 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:04:07.807 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:04:07.807 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:04:07.807 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:07.807 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 14, 2024-09-13T18:04:07.807 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:04:07.807 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:04:07.807 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:04:07.807 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:07.807 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:07.807 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:07.807 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:07.807 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:07.807 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:07.808 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:07.808 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:07.808 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:07.808 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:07.808 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:07.808 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:04:07.808 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:04:07.808 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:07.808 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:07.808 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:07.808 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:07.808 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:07.809 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:07.809 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:07.809 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:07.809 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:07.809 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:07.809 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:07.809 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.809 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:04:07.809 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:04:07.809 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:04:07.809 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:04:07.809 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:04:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:04:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:04:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:04:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:04:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:04:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:04:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:04:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:04:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:04:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:04:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:04:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:04:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:07.810 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:07.811 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:07.811 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:07.811 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:07.811 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:07.811 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:07.811 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:07.811 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:07.811 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:07.812 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:07.812 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:04:07.812 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.812 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:04:07.812 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.812 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:04:07.812 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:04:07.812 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:07.812 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:07.812 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.812 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:07.812 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:04:07.813 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:07.813 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.813 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.813 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:07.813 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:04:07.813 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:07.813 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.813 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:07.813 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.813 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:04:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:04:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:04:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:04:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:04:07.814 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:04:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:04:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:04:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.815 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.816 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:04:07.816 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:04:07.816 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:07.816 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:07.816 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.816 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:07.816 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:04:07.816 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:07.816 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.816 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.816 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:07.816 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:04:07.816 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:04:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:04:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:04:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:04:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.817 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:07.818 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:07.818 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:04:07.818 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.818 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:07.818 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:04:07.818 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:07.818 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:07.818 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:07.818 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:04:07.818 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:07.818 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:07.818 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:07.818 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:04:07.818 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:04:07.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f962f59e700 1 -- 172.21.15.40:0/179795685 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9610066730 msgr2=0x7f9610068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f962f59e700 1 --2- 172.21.15.40:0/179795685 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9610066730 0x7f9610068bf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f96300ce1c0 tx=0x7f9624008040 comp rx=0 tx=0).stop 2024-09-13T18:04:07.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f962f59e700 1 -- 172.21.15.40:0/179795685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96300c3f10 msgr2=0x7f96301eefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:07.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f962f59e700 1 --2- 172.21.15.40:0/179795685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96300c3f10 0x7f96301eefe0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9618004750 tx=0x7f9628001360 comp rx=0 tx=0).stop 2024-09-13T18:04:07.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f962ed9d700 1 -- 172.21.15.40:0/179795685 reap_dead start 2024-09-13T18:04:07.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f962ed9d700 1 -- 172.21.15.40:0/179795685 reap_dead start 2024-09-13T18:04:07.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f962f59e700 1 -- 172.21.15.40:0/179795685 shutdown_connections 2024-09-13T18:04:07.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f962f59e700 1 -- 172.21.15.40:0/179795685 >> 172.21.15.40:0/179795685 conn(0x7f9630066090 msgr2=0x7f9630066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:07.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f962f59e700 1 -- 172.21.15.40:0/179795685 shutdown_connections 2024-09-13T18:04:07.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:07.794+0000 7f962f59e700 1 -- 172.21.15.40:0/179795685 wait complete. 2024-09-13T18:04:07.821 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:04:07.821 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-13T18:04:07.821 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T18:04:07.821 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-13T18:04:07.821 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T18:04:07.821 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T18:04:07.822 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:04:07.888 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T18:04:07.889 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T18:04:07.889 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:04:08.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.010+0000 7f82a305e700 1 Processor -- start 2024-09-13T18:04:08.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.010+0000 7f82a305e700 1 -- start start 2024-09-13T18:04:08.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.010+0000 7f82a305e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f82940477b0 0x7f8294047bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:08.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.010+0000 7f82a305e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82940088d0 0x7f8294008d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:08.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.010+0000 7f82a305e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82940e1cc0 con 0x7f82940d8ad0 2024-09-13T18:04:08.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.010+0000 7f82a305e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82940e1f80 con 0x7f82940477b0 2024-09-13T18:04:08.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.010+0000 7f82a305e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82940e2260 con 0x7f82940088d0 2024-09-13T18:04:08.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.010+0000 7f82a185b700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f82940477b0 msgr2=0x7f8294047bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:04:08.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.010+0000 7f82a185b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f82940477b0 0x7f8294047bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:08.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.010+0000 7f82a285d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82940088d0 0x7f8294008d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:08.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.010+0000 7f82a205c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f82940d8ad0 0x7f829404cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:51342/0 (socket says 172.21.15.40:51342) 2024-09-13T18:04:08.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.010+0000 7f82a205c700 1 -- 172.21.15.40:0/3827739531 learned_addr learned my addr 172.21.15.40:0/3827739531 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:04:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.010+0000 7f82a285d700 1 -- 172.21.15.40:0/3827739531 >> v2:172.21.15.40:3300/0 conn(0x7f82940477b0 msgr2=0x7f8294047bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:04:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.010+0000 7f82a285d700 1 --2- 172.21.15.40:0/3827739531 >> v2:172.21.15.40:3300/0 conn(0x7f82940477b0 0x7f8294047bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.010+0000 7f82a285d700 1 -- 172.21.15.40:0/3827739531 >> v1:172.21.15.43:6789/0 conn(0x7f82940d8ad0 legacy=0x7f829404cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.010+0000 7f82a105a700 1 -- 172.21.15.40:0/3827739531 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477977692 0 0) 0x7f82940e1cc0 con 0x7f82940d8ad0 2024-09-13T18:04:08.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.010+0000 7f82a285d700 1 -- 172.21.15.40:0/3827739531 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82940e2540 con 0x7f82940088d0 2024-09-13T18:04:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.010+0000 7f82a285d700 1 --2- 172.21.15.40:0/3827739531 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82940088d0 0x7f8294008d20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f829c069620 tx=0x7f829c069e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee6f378f559844b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a105a700 1 -- 172.21.15.40:0/3827739531 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c067fd0 con 0x7f82940088d0 2024-09-13T18:04:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a105a700 1 -- 172.21.15.40:0/3827739531 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829c068a70 con 0x7f82940088d0 2024-09-13T18:04:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a105a700 1 -- 172.21.15.40:0/3827739531 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c0746e0 con 0x7f82940088d0 2024-09-13T18:04:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a305e700 1 -- 172.21.15.40:0/3827739531 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82940088d0 msgr2=0x7f8294008d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a305e700 1 --2- 172.21.15.40:0/3827739531 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82940088d0 0x7f8294008d20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f829c069620 tx=0x7f829c069e70 comp rx=0 tx=0).stop 2024-09-13T18:04:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a305e700 1 -- 172.21.15.40:0/3827739531 shutdown_connections 2024-09-13T18:04:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a305e700 1 --2- 172.21.15.40:0/3827739531 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82940088d0 0x7f8294008d20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a305e700 1 --2- 172.21.15.40:0/3827739531 >> v2:172.21.15.40:3300/0 conn(0x7f82940477b0 0x7f8294047bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a305e700 1 -- 172.21.15.40:0/3827739531 >> 172.21.15.40:0/3827739531 conn(0x7f829400dc40 msgr2=0x7f829400e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a305e700 1 -- 172.21.15.40:0/3827739531 shutdown_connections 2024-09-13T18:04:08.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a305e700 1 -- 172.21.15.40:0/3827739531 wait complete. 2024-09-13T18:04:08.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a305e700 1 Processor -- start 2024-09-13T18:04:08.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a305e700 1 -- start start 2024-09-13T18:04:08.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a305e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82940088d0 0x7f8294177a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:08.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a305e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82940477b0 0x7f82940d0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:08.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a305e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82940d8ad0 0x7f82940d1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:08.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a305e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82940d0fd0 con 0x7f82940477b0 2024-09-13T18:04:08.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a305e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82940d0cd0 con 0x7f82940d8ad0 2024-09-13T18:04:08.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a305e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82941786e0 con 0x7f82940088d0 2024-09-13T18:04:08.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a185b700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82940477b0 msgr2=0x7f82940d0150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:04:08.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a185b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82940477b0 0x7f82940d0150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:08.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a205c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82940088d0 0x7f8294177a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:08.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a285d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82940d8ad0 0x7f82940d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:08.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a205c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82940088d0 0x7f8294177a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:36728/0 (socket says 172.21.15.40:36728) 2024-09-13T18:04:08.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a205c700 1 -- 172.21.15.40:0/2241687649 learned_addr learned my addr 172.21.15.40:0/2241687649 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:08.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a205c700 1 -- 172.21.15.40:0/2241687649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82940d8ad0 msgr2=0x7f82940d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:08.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a205c700 1 --2- 172.21.15.40:0/2241687649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82940d8ad0 0x7f82940d1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:08.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a205c700 1 -- 172.21.15.40:0/2241687649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82940477b0 msgr2=0x7f82940d0150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:04:08.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a205c700 1 --2- 172.21.15.40:0/2241687649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82940477b0 0x7f82940d0150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:08.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.014+0000 7f82a205c700 1 -- 172.21.15.40:0/2241687649 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829c04f5c0 con 0x7f82940088d0 2024-09-13T18:04:08.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.018+0000 7f82a205c700 1 --2- 172.21.15.40:0/2241687649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82940088d0 0x7f8294177a40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f829800c4d0 tx=0x7f829800c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:08.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.018+0000 7f8292ffd700 1 -- 172.21.15.40:0/2241687649 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8298004ca0 con 0x7f82940088d0 2024-09-13T18:04:08.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.018+0000 7f82a305e700 1 -- 172.21.15.40:0/2241687649 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8294178420 con 0x7f82940088d0 2024-09-13T18:04:08.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.018+0000 7f8292ffd700 1 -- 172.21.15.40:0/2241687649 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8298004e00 con 0x7f82940088d0 2024-09-13T18:04:08.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.018+0000 7f82a305e700 1 -- 172.21.15.40:0/2241687649 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8294049620 con 0x7f82940088d0 2024-09-13T18:04:08.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.018+0000 7f8292ffd700 1 -- 172.21.15.40:0/2241687649 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82980072d0 con 0x7f82940088d0 2024-09-13T18:04:08.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.018+0000 7f82a305e700 1 -- 172.21.15.40:0/2241687649 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8294047bb0 con 0x7f82940088d0 2024-09-13T18:04:08.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.018+0000 7f8292ffd700 1 -- 172.21.15.40:0/2241687649 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8298007430 con 0x7f82940088d0 2024-09-13T18:04:08.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.018+0000 7f8292ffd700 1 --2- 172.21.15.40:0/2241687649 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f82800666e0 0x7f8280068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:08.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.018+0000 7f8292ffd700 1 -- 172.21.15.40:0/2241687649 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8298084000 con 0x7f82940088d0 2024-09-13T18:04:08.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.018+0000 7f82a185b700 1 --2- 172.21.15.40:0/2241687649 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f82800666e0 0x7f8280068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:08.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.022+0000 7f82a185b700 1 --2- 172.21.15.40:0/2241687649 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f82800666e0 0x7f8280068ba0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8294175390 tx=0x7f828c006de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:08.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.030+0000 7f8292ffd700 1 -- 172.21.15.40:0/2241687649 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8298050f30 con 0x7f82940088d0 2024-09-13T18:04:08.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.302+0000 7f82a305e700 1 -- 172.21.15.40:0/2241687649 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f829404d840 con 0x7f82940088d0 2024-09-13T18:04:08.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.302+0000 7f8292ffd700 1 -- 172.21.15.40:0/2241687649 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f8298054550 con 0x7f82940088d0 2024-09-13T18:04:08.312 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":108,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:04:08.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.306+0000 7f82a305e700 1 -- 172.21.15.40:0/2241687649 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f82800666e0 msgr2=0x7f8280068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:08.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.306+0000 7f82a305e700 1 --2- 172.21.15.40:0/2241687649 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f82800666e0 0x7f8280068ba0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8294175390 tx=0x7f828c006de0 comp rx=0 tx=0).stop 2024-09-13T18:04:08.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.306+0000 7f82a305e700 1 -- 172.21.15.40:0/2241687649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82940088d0 msgr2=0x7f8294177a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:08.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.306+0000 7f82a305e700 1 --2- 172.21.15.40:0/2241687649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82940088d0 0x7f8294177a40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f829800c4d0 tx=0x7f829800c7e0 comp rx=0 tx=0).stop 2024-09-13T18:04:08.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.306+0000 7f82a305e700 1 -- 172.21.15.40:0/2241687649 shutdown_connections 2024-09-13T18:04:08.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.306+0000 7f82a305e700 1 --2- 172.21.15.40:0/2241687649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82940d8ad0 0x7f82940d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:08.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.306+0000 7f82a305e700 1 --2- 172.21.15.40:0/2241687649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82940477b0 0x7f82940d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:08.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.306+0000 7f82a305e700 1 --2- 172.21.15.40:0/2241687649 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f82800666e0 0x7f8280068ba0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:08.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.306+0000 7f82a305e700 1 --2- 172.21.15.40:0/2241687649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f82940088d0 0x7f8294177a40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:08.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.306+0000 7f82a305e700 1 -- 172.21.15.40:0/2241687649 >> 172.21.15.40:0/2241687649 conn(0x7f829400dc40 msgr2=0x7f82940d7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:08.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.306+0000 7f82a305e700 1 -- 172.21.15.40:0/2241687649 shutdown_connections 2024-09-13T18:04:08.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:08.306+0000 7f82a305e700 1 -- 172.21.15.40:0/2241687649 wait complete. 2024-09-13T18:04:11.329 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:04:11.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc6994f2700 1 Processor -- start 2024-09-13T18:04:11.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc6994f2700 1 -- start start 2024-09-13T18:04:11.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc6994f2700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc694071f70 0x7fc6940c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:11.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc6994f2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6940c7200 0x7fc694161ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:11.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc6994f2700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6940c55c0 con 0x7fc694072350 2024-09-13T18:04:11.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc6994f2700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69406a390 con 0x7fc694071f70 2024-09-13T18:04:11.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc6994f2700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc694157710 con 0x7fc6940c7200 2024-09-13T18:04:11.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc6937fe700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7fc694072350 legacy=0x7fc694150d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:04:11.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc693fff700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7fc694071f70 msgr2=0x7fc6940c5d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:04:11.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc693fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc694071f70 0x7fc6940c5d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:11.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc698cf1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6940c7200 0x7fc694161ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:11.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc698cf1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6940c7200 0x7fc694161ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:46118/0 (socket says 172.21.15.40:46118) 2024-09-13T18:04:11.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc698cf1700 1 -- 172.21.15.40:0/1044038824 learned_addr learned my addr 172.21.15.40:0/1044038824 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:11.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc698cf1700 1 -- 172.21.15.40:0/1044038824 >> v2:172.21.15.40:3300/0 conn(0x7fc694071f70 msgr2=0x7fc6940c5d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:04:11.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc698cf1700 1 --2- 172.21.15.40:0/1044038824 >> v2:172.21.15.40:3300/0 conn(0x7fc694071f70 0x7fc6940c5d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:11.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc698cf1700 1 -- 172.21.15.40:0/1044038824 >> v1:172.21.15.40:6789/0 conn(0x7fc694072350 legacy=0x7fc694150d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:04:11.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc698cf1700 1 -- 172.21.15.40:0/1044038824 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6941579d0 con 0x7fc6940c7200 2024-09-13T18:04:11.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc698cf1700 1 --2- 172.21.15.40:0/1044038824 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6940c7200 0x7fc694161ae0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009e50 tx=0x7fc68c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f73e70526ba6312 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:11.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc692ffd700 1 -- 172.21.15.40:0/1044038824 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c00e070 con 0x7fc6940c7200 2024-09-13T18:04:11.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc692ffd700 1 -- 172.21.15.40:0/1044038824 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c003a40 con 0x7fc6940c7200 2024-09-13T18:04:11.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.438+0000 7fc692ffd700 1 -- 172.21.15.40:0/1044038824 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c003df0 con 0x7fc6940c7200 2024-09-13T18:04:11.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.442+0000 7fc6994f2700 1 -- 172.21.15.40:0/1044038824 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6940c7200 msgr2=0x7fc694161ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:11.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.442+0000 7fc6994f2700 1 --2- 172.21.15.40:0/1044038824 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6940c7200 0x7fc694161ae0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009e50 tx=0x7fc68c00b330 comp rx=0 tx=0).stop 2024-09-13T18:04:11.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.442+0000 7fc6994f2700 1 -- 172.21.15.40:0/1044038824 shutdown_connections 2024-09-13T18:04:11.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.442+0000 7fc6994f2700 1 --2- 172.21.15.40:0/1044038824 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6940c7200 0x7fc694161ae0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:11.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.442+0000 7fc6994f2700 1 --2- 172.21.15.40:0/1044038824 >> v2:172.21.15.40:3300/0 conn(0x7fc694071f70 0x7fc6940c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:11.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.442+0000 7fc6994f2700 1 -- 172.21.15.40:0/1044038824 >> 172.21.15.40:0/1044038824 conn(0x7fc6940cabc0 msgr2=0x7fc6940cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:11.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.442+0000 7fc6994f2700 1 -- 172.21.15.40:0/1044038824 shutdown_connections 2024-09-13T18:04:11.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.442+0000 7fc6994f2700 1 -- 172.21.15.40:0/1044038824 wait complete. 2024-09-13T18:04:11.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.442+0000 7fc6994f2700 1 Processor -- start 2024-09-13T18:04:11.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.442+0000 7fc6994f2700 1 -- start start 2024-09-13T18:04:11.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.442+0000 7fc6994f2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc694071f70 0x7fc6941e2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:11.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.442+0000 7fc6994f2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc694072350 0x7fc694148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:11.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.442+0000 7fc6994f2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6940c7200 0x7fc6941e0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:11.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.442+0000 7fc6994f2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69414a020 con 0x7fc694072350 2024-09-13T18:04:11.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.442+0000 7fc6994f2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69414a440 con 0x7fc694071f70 2024-09-13T18:04:11.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.442+0000 7fc6994f2700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc694147cd0 con 0x7fc6940c7200 2024-09-13T18:04:11.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.442+0000 7fc6937fe700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc694072350 msgr2=0x7fc694148e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:04:11.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.442+0000 7fc6937fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc694072350 0x7fc694148e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:11.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.442+0000 7fc698cf1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6940c7200 0x7fc6941e0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:11.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.446+0000 7fc698cf1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6940c7200 0x7fc6941e0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:46130/0 (socket says 172.21.15.40:46130) 2024-09-13T18:04:11.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.446+0000 7fc698cf1700 1 -- 172.21.15.40:0/3181353026 learned_addr learned my addr 172.21.15.40:0/3181353026 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:11.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.446+0000 7fc693fff700 1 --2- 172.21.15.40:0/3181353026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc694071f70 0x7fc6941e2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:11.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.446+0000 7fc698cf1700 1 -- 172.21.15.40:0/3181353026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc694071f70 msgr2=0x7fc6941e2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:11.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.446+0000 7fc698cf1700 1 --2- 172.21.15.40:0/3181353026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc694071f70 0x7fc6941e2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:11.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.446+0000 7fc698cf1700 1 -- 172.21.15.40:0/3181353026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc694072350 msgr2=0x7fc694148e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:04:11.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.446+0000 7fc698cf1700 1 --2- 172.21.15.40:0/3181353026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc694072350 0x7fc694148e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:11.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.446+0000 7fc698cf1700 1 -- 172.21.15.40:0/3181353026 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68c009b30 con 0x7fc6940c7200 2024-09-13T18:04:11.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.446+0000 7fc698cf1700 1 --2- 172.21.15.40:0/3181353026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6940c7200 0x7fc6941e0470 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc694157850 tx=0x7fc68c004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:11.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.446+0000 7fc690ff9700 1 -- 172.21.15.40:0/3181353026 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c004240 con 0x7fc6940c7200 2024-09-13T18:04:11.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.446+0000 7fc690ff9700 1 -- 172.21.15.40:0/3181353026 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c0043c0 con 0x7fc6940c7200 2024-09-13T18:04:11.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.446+0000 7fc690ff9700 1 -- 172.21.15.40:0/3181353026 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c004770 con 0x7fc6940c7200 2024-09-13T18:04:11.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.446+0000 7fc6994f2700 1 -- 172.21.15.40:0/3181353026 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc694147f20 con 0x7fc6940c7200 2024-09-13T18:04:11.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.446+0000 7fc6994f2700 1 -- 172.21.15.40:0/3181353026 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc694149960 con 0x7fc6940c7200 2024-09-13T18:04:11.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.446+0000 7fc6994f2700 1 -- 172.21.15.40:0/3181353026 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6940cb6e0 con 0x7fc6940c7200 2024-09-13T18:04:11.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.450+0000 7fc690ff9700 1 -- 172.21.15.40:0/3181353026 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc68c019070 con 0x7fc6940c7200 2024-09-13T18:04:11.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.450+0000 7fc690ff9700 1 --2- 172.21.15.40:0/3181353026 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc6800666e0 0x7fc680068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:11.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.450+0000 7fc693fff700 1 --2- 172.21.15.40:0/3181353026 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc6800666e0 0x7fc680068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:11.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.450+0000 7fc690ff9700 1 -- 172.21.15.40:0/3181353026 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc68c0852f0 con 0x7fc6940c7200 2024-09-13T18:04:11.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.450+0000 7fc693fff700 1 --2- 172.21.15.40:0/3181353026 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc6800666e0 0x7fc680068ba0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc684009470 tx=0x7fc684009400 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:11.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.458+0000 7fc690ff9700 1 -- 172.21.15.40:0/3181353026 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc68c052330 con 0x7fc6940c7200 2024-09-13T18:04:11.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.746+0000 7fc6994f2700 1 -- 172.21.15.40:0/3181353026 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc6940cb280 con 0x7fc6940c7200 2024-09-13T18:04:11.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.746+0000 7fc690ff9700 1 -- 172.21.15.40:0/3181353026 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7fc68c017620 con 0x7fc6940c7200 2024-09-13T18:04:11.756 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":108,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:04:11.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.746+0000 7fc6994f2700 1 -- 172.21.15.40:0/3181353026 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc6800666e0 msgr2=0x7fc680068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:11.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.746+0000 7fc6994f2700 1 --2- 172.21.15.40:0/3181353026 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc6800666e0 0x7fc680068ba0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc684009470 tx=0x7fc684009400 comp rx=0 tx=0).stop 2024-09-13T18:04:11.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.746+0000 7fc6994f2700 1 -- 172.21.15.40:0/3181353026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6940c7200 msgr2=0x7fc6941e0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:11.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.746+0000 7fc6994f2700 1 --2- 172.21.15.40:0/3181353026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6940c7200 0x7fc6941e0470 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc694157850 tx=0x7fc68c004af0 comp rx=0 tx=0).stop 2024-09-13T18:04:11.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.750+0000 7fc6994f2700 1 -- 172.21.15.40:0/3181353026 shutdown_connections 2024-09-13T18:04:11.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.750+0000 7fc6994f2700 1 --2- 172.21.15.40:0/3181353026 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc6800666e0 0x7fc680068ba0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:11.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.750+0000 7fc6994f2700 1 --2- 172.21.15.40:0/3181353026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6940c7200 0x7fc6941e0470 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:11.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.750+0000 7fc6994f2700 1 --2- 172.21.15.40:0/3181353026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc694072350 0x7fc694148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:11.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.750+0000 7fc6994f2700 1 --2- 172.21.15.40:0/3181353026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc694071f70 0x7fc6941e2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:11.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.750+0000 7fc6994f2700 1 -- 172.21.15.40:0/3181353026 >> 172.21.15.40:0/3181353026 conn(0x7fc6940cabc0 msgr2=0x7fc69414d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:11.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.750+0000 7fc6994f2700 1 -- 172.21.15.40:0/3181353026 shutdown_connections 2024-09-13T18:04:11.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:11.750+0000 7fc6994f2700 1 -- 172.21.15.40:0/3181353026 wait complete. 2024-09-13T18:04:14.772 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:04:14.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.886+0000 7f71bf59e700 1 Processor -- start 2024-09-13T18:04:14.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.886+0000 7f71bf59e700 1 -- start start 2024-09-13T18:04:14.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.886+0000 7f71bf59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71c00c4210 0x7f71c00c4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:14.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.886+0000 7f71bf59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c00663d0 con 0x7f71c0159840 2024-09-13T18:04:14.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.886+0000 7f71bf59e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c0070d40 con 0x7f71c014fad0 2024-09-13T18:04:14.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.886+0000 7f71bf59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c015a330 con 0x7f71c00c4210 2024-09-13T18:04:14.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.886+0000 7f71be59c700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f71c0159840 legacy=0x7f71c0159c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:04:14.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.890+0000 7f71bdd9b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71c00c4210 0x7f71c00c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:14.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.890+0000 7f71bed9d700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f71c014fad0 0x7f71c014ff80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:43296/0 (socket says 172.21.15.40:43296) 2024-09-13T18:04:14.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.890+0000 7f71bed9d700 1 -- 172.21.15.40:0/378968373 learned_addr learned my addr 172.21.15.40:0/378968373 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:04:14.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.890+0000 7f71bdd9b700 1 -- 172.21.15.40:0/378968373 >> v1:172.21.15.43:6789/0 conn(0x7f71c014fad0 legacy=0x7f71c014ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:14.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.890+0000 7f71bdd9b700 1 -- 172.21.15.40:0/378968373 >> v1:172.21.15.40:6789/0 conn(0x7f71c0159840 legacy=0x7f71c0159c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:04:14.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.890+0000 7f71bdd9b700 1 -- 172.21.15.40:0/378968373 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c0070fe0 con 0x7f71c00c4210 2024-09-13T18:04:14.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.890+0000 7f71bdd9b700 1 --2- 172.21.15.40:0/378968373 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71c00c4210 0x7f71c00c4610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009880 tx=0x7f71b4009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=889088b76fbfc32b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:14.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.890+0000 7f71bd59a700 1 -- 172.21.15.40:0/378968373 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4004230 con 0x7f71c00c4210 2024-09-13T18:04:14.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.890+0000 7f71bd59a700 1 -- 172.21.15.40:0/378968373 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b4004390 con 0x7f71c00c4210 2024-09-13T18:04:14.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.890+0000 7f71bd59a700 1 -- 172.21.15.40:0/378968373 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b40036c0 con 0x7f71c00c4210 2024-09-13T18:04:14.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.890+0000 7f71bf59e700 1 -- 172.21.15.40:0/378968373 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71c00c4210 msgr2=0x7f71c00c4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:14.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.890+0000 7f71bf59e700 1 --2- 172.21.15.40:0/378968373 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71c00c4210 0x7f71c00c4610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009880 tx=0x7f71b4009b90 comp rx=0 tx=0).stop 2024-09-13T18:04:14.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.890+0000 7f71bf59e700 1 -- 172.21.15.40:0/378968373 shutdown_connections 2024-09-13T18:04:14.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.890+0000 7f71bf59e700 1 --2- 172.21.15.40:0/378968373 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71c00c4210 0x7f71c00c4610 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:14.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.890+0000 7f71bf59e700 1 -- 172.21.15.40:0/378968373 >> 172.21.15.40:0/378968373 conn(0x7f71c006a390 msgr2=0x7f71c006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:14.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.890+0000 7f71bf59e700 1 -- 172.21.15.40:0/378968373 shutdown_connections 2024-09-13T18:04:14.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.890+0000 7f71bf59e700 1 -- 172.21.15.40:0/378968373 wait complete. 2024-09-13T18:04:14.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bf59e700 1 Processor -- start 2024-09-13T18:04:14.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bf59e700 1 -- start start 2024-09-13T18:04:14.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bf59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c00c4210 0x7f71c01df7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:14.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bf59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71c014fad0 0x7f71c00c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:14.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bf59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71c0159840 0x7f71c014e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:14.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71be59c700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c00c4210 msgr2=0x7f71c01df7d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:04:14.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bf59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c0149a40 con 0x7f71c00c4210 2024-09-13T18:04:14.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bf59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c01f0c00 con 0x7f71c014fad0 2024-09-13T18:04:14.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bf59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c01f1130 con 0x7f71c0159840 2024-09-13T18:04:14.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71be59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c00c4210 0x7f71c01df7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:14.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bdd9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71c014fad0 0x7f71c00c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:14.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bed9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71c0159840 0x7f71c014e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:14.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bdd9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71c014fad0 0x7f71c00c5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:36726/0 (socket says 172.21.15.40:36726) 2024-09-13T18:04:14.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bed9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71c0159840 0x7f71c014e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:46140/0 (socket says 172.21.15.40:46140) 2024-09-13T18:04:14.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bdd9b700 1 -- 172.21.15.40:0/3070880458 learned_addr learned my addr 172.21.15.40:0/3070880458 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:14.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bed9d700 1 -- 172.21.15.40:0/3070880458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71c014fad0 msgr2=0x7f71c00c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:14.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bed9d700 1 --2- 172.21.15.40:0/3070880458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71c014fad0 0x7f71c00c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:14.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bed9d700 1 -- 172.21.15.40:0/3070880458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c00c4210 msgr2=0x7f71c01df7d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:04:14.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bed9d700 1 --2- 172.21.15.40:0/3070880458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c00c4210 0x7f71c01df7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:14.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bed9d700 1 -- 172.21.15.40:0/3070880458 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b40094b0 con 0x7f71c0159840 2024-09-13T18:04:14.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bed9d700 1 --2- 172.21.15.40:0/3070880458 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71c0159840 0x7f71c014e440 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f71b800caa0 tx=0x7f71b800cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:14.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71aeffd700 1 -- 172.21.15.40:0/3070880458 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8007480 con 0x7f71c0159840 2024-09-13T18:04:14.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bf59e700 1 -- 172.21.15.40:0/3070880458 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c01edbe0 con 0x7f71c0159840 2024-09-13T18:04:14.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71aeffd700 1 -- 172.21.15.40:0/3070880458 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b80075e0 con 0x7f71c0159840 2024-09-13T18:04:14.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71aeffd700 1 -- 172.21.15.40:0/3070880458 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b8007950 con 0x7f71c0159840 2024-09-13T18:04:14.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.894+0000 7f71bf59e700 1 -- 172.21.15.40:0/3070880458 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c01ece20 con 0x7f71c0159840 2024-09-13T18:04:14.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.898+0000 7f71bf59e700 1 -- 172.21.15.40:0/3070880458 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c00c3980 con 0x7f71c0159840 2024-09-13T18:04:14.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.898+0000 7f71aeffd700 1 -- 172.21.15.40:0/3070880458 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f71b801b3b0 con 0x7f71c0159840 2024-09-13T18:04:14.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.898+0000 7f71aeffd700 1 --2- 172.21.15.40:0/3070880458 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f71a00666e0 0x7f71a0068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:14.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.898+0000 7f71be59c700 1 --2- 172.21.15.40:0/3070880458 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f71a00666e0 0x7f71a0068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:14.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.898+0000 7f71aeffd700 1 -- 172.21.15.40:0/3070880458 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f71b8083dc0 con 0x7f71c0159840 2024-09-13T18:04:14.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.898+0000 7f71be59c700 1 --2- 172.21.15.40:0/3070880458 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f71a00666e0 0x7f71a0068ba0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f71b0003a30 tx=0x7f71b0009270 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:14.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:14.906+0000 7f71aeffd700 1 -- 172.21.15.40:0/3070880458 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f71c00c3980 con 0x7f71c0159840 2024-09-13T18:04:15.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:15.162+0000 7f71bf59e700 1 -- 172.21.15.40:0/3070880458 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f71c00c3980 con 0x7f71c0159840 2024-09-13T18:04:15.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:15.162+0000 7f71aeffd700 1 -- 172.21.15.40:0/3070880458 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f71c00c3980 con 0x7f71c0159840 2024-09-13T18:04:15.173 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":108,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:04:15.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:15.166+0000 7f71bf59e700 1 -- 172.21.15.40:0/3070880458 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f71a00666e0 msgr2=0x7f71a0068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:15.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:15.166+0000 7f71bf59e700 1 --2- 172.21.15.40:0/3070880458 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f71a00666e0 0x7f71a0068ba0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f71b0003a30 tx=0x7f71b0009270 comp rx=0 tx=0).stop 2024-09-13T18:04:15.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:15.166+0000 7f71bf59e700 1 -- 172.21.15.40:0/3070880458 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71c0159840 msgr2=0x7f71c014e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:15.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:15.166+0000 7f71bf59e700 1 --2- 172.21.15.40:0/3070880458 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71c0159840 0x7f71c014e440 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f71b800caa0 tx=0x7f71b800cad0 comp rx=0 tx=0).stop 2024-09-13T18:04:15.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:15.166+0000 7f71bf59e700 1 -- 172.21.15.40:0/3070880458 shutdown_connections 2024-09-13T18:04:15.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:15.166+0000 7f71bf59e700 1 --2- 172.21.15.40:0/3070880458 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f71a00666e0 0x7f71a0068ba0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:15.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:15.166+0000 7f71bf59e700 1 --2- 172.21.15.40:0/3070880458 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71c0159840 0x7f71c014e440 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:15.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:15.166+0000 7f71bf59e700 1 --2- 172.21.15.40:0/3070880458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71c014fad0 0x7f71c00c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:15.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:15.166+0000 7f71bf59e700 1 --2- 172.21.15.40:0/3070880458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c00c4210 0x7f71c01df7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:15.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:15.166+0000 7f71bf59e700 1 -- 172.21.15.40:0/3070880458 >> 172.21.15.40:0/3070880458 conn(0x7f71c006a390 msgr2=0x7f71c0155370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:15.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:15.166+0000 7f71bf59e700 1 -- 172.21.15.40:0/3070880458 shutdown_connections 2024-09-13T18:04:15.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:15.166+0000 7f71bf59e700 1 -- 172.21.15.40:0/3070880458 wait complete. 2024-09-13T18:04:18.190 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:04:18.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.302+0000 7f723f59e700 1 Processor -- start 2024-09-13T18:04:18.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.302+0000 7f723f59e700 1 -- start start 2024-09-13T18:04:18.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.302+0000 7f723f59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 0x7f7238046e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:18.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.302+0000 7f723f59e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 0x7f72380e04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:18.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.302+0000 7f723f59e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72380dfa70 con 0x7f723804de20 2024-09-13T18:04:18.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.302+0000 7f723f59e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72380dd330 con 0x7f72380dcf00 2024-09-13T18:04:18.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.302+0000 7f723f59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723804cf30 con 0x7f72380d6990 2024-09-13T18:04:18.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.302+0000 7f723e59c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 0x7f7238046e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:18.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.302+0000 7f723e59c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 0x7f7238046e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:46154/0 (socket says 172.21.15.40:46154) 2024-09-13T18:04:18.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.302+0000 7f723ed9d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 0x7f72380e04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:18.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.302+0000 7f723e59c700 1 -- 172.21.15.40:0/4225006515 learned_addr learned my addr 172.21.15.40:0/4225006515 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:18.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.302+0000 7f723e59c700 1 -- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 msgr2=0x7f7238046e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:04:18.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.302+0000 7f723e59c700 1 -- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 msgr2=0x7f7238046e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:18.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.302+0000 7f723e59c700 1 --2- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 0x7f7238046e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:18.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.302+0000 7f723e59c700 1 --2- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 0x7f7238046e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:18.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.302+0000 7f723ed9d700 1 -- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 msgr2=0x7f72380e04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:04:18.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.302+0000 7f723ed9d700 1 -- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 msgr2=0x7f72380e04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:18.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.302+0000 7f723ed9d700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 0x7f72380e04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:18.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.302+0000 7f723ed9d700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 0x7f72380e04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:18.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.506+0000 7f723e59c700 1 --2- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 0x7f7238046e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:18.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.506+0000 7f723ed9d700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 0x7f72380e04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:18.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.506+0000 7f723e59c700 1 -- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 msgr2=0x7f7238046e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:04:18.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.506+0000 7f723e59c700 1 -- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 msgr2=0x7f7238046e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:18.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.506+0000 7f723e59c700 1 --2- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 0x7f7238046e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:18.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.506+0000 7f723e59c700 1 --2- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 0x7f7238046e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:04:18.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.506+0000 7f723ed9d700 1 -- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 msgr2=0x7f72380e04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:04:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.506+0000 7f723ed9d700 1 -- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 msgr2=0x7f72380e04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.506+0000 7f723ed9d700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 0x7f72380e04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:18.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.506+0000 7f723ed9d700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 0x7f72380e04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:04:18.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.909+0000 7f723e59c700 1 --2- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 0x7f7238046e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:18.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.909+0000 7f723ed9d700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 0x7f72380e04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:18.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.909+0000 7f723ed9d700 1 -- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 msgr2=0x7f72380e04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:04:18.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.909+0000 7f723ed9d700 1 -- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 msgr2=0x7f72380e04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:18.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.909+0000 7f723ed9d700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 0x7f72380e04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:18.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.909+0000 7f723ed9d700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 0x7f72380e04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:04:18.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.909+0000 7f723e59c700 1 -- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 msgr2=0x7f7238046e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:04:18.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.909+0000 7f723e59c700 1 -- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 msgr2=0x7f7238046e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:18.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.909+0000 7f723e59c700 1 --2- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 0x7f7238046e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:18.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:18.909+0000 7f723e59c700 1 --2- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 0x7f7238046e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:04:19.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:19.713+0000 7f723ed9d700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 0x7f72380e04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:19.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:19.713+0000 7f723e59c700 1 --2- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 0x7f7238046e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:19.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:19.713+0000 7f723ed9d700 1 -- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 msgr2=0x7f72380e04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:04:19.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:19.713+0000 7f723ed9d700 1 -- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 msgr2=0x7f72380e04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:19.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:19.713+0000 7f723ed9d700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 0x7f72380e04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:19.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:19.713+0000 7f723e59c700 1 -- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 msgr2=0x7f7238046e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:04:19.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:19.713+0000 7f723e59c700 1 -- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 msgr2=0x7f7238046e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:19.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:19.713+0000 7f723ed9d700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 0x7f72380e04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:04:19.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:19.713+0000 7f723e59c700 1 --2- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 0x7f7238046e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:19.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:19.713+0000 7f723e59c700 1 --2- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 0x7f7238046e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:04:21.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:21.301+0000 7f722ffff700 1 -- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 msgr2=0x7f7238046e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:04:21.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:21.301+0000 7f722ffff700 1 --2- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72380d6990 0x7f7238046e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:21.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:21.301+0000 7f722ffff700 1 -- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 msgr2=0x7f72380e04c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:04:21.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:21.301+0000 7f722ffff700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f72380dcf00 0x7f72380e04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:21.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:21.301+0000 7f722ffff700 1 -- 172.21.15.40:0/4225006515 >> v1:172.21.15.43:6789/0 conn(0x7f723804de20 legacy=0x7f7238051ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:21.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:21.301+0000 7f722ffff700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.40:3300/0 conn(0x7f7220003720 0x7f7220005ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:21.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:21.301+0000 7f723e59c700 1 -- 172.21.15.40:0/4225006515 >> v2:172.21.15.40:3300/0 conn(0x7f7220003720 msgr2=0x7f7220005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:04:21.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:21.301+0000 7f722ffff700 1 -- 172.21.15.40:0/4225006515 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f723804d210 con 0x7f7220006280 2024-09-13T18:04:21.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:21.301+0000 7f722ffff700 1 -- 172.21.15.40:0/4225006515 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72380dd330 con 0x7f7220003720 2024-09-13T18:04:21.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:21.301+0000 7f723e59c700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.40:3300/0 conn(0x7f7220003720 0x7f7220005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:21.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:21.301+0000 7f723dd9b700 1 -- 172.21.15.40:0/4225006515 >> v1:172.21.15.40:6789/0 conn(0x7f7220006280 legacy=0x7f7220008720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:04:21.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:21.505+0000 7f723dd9b700 1 -- 172.21.15.40:0/4225006515 >> v1:172.21.15.40:6789/0 conn(0x7f7220006280 legacy=0x7f7220008720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:04:21.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:21.505+0000 7f723e59c700 1 -- 172.21.15.40:0/4225006515 >> v2:172.21.15.40:3300/0 conn(0x7f7220003720 msgr2=0x7f7220005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:04:21.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:21.505+0000 7f723e59c700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.40:3300/0 conn(0x7f7220003720 0x7f7220005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:04:21.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:21.909+0000 7f723dd9b700 1 -- 172.21.15.40:0/4225006515 >> v1:172.21.15.40:6789/0 conn(0x7f7220006280 legacy=0x7f7220008720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:04:21.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:21.909+0000 7f723e59c700 1 -- 172.21.15.40:0/4225006515 >> v2:172.21.15.40:3300/0 conn(0x7f7220003720 msgr2=0x7f7220005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:04:21.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:21.909+0000 7f723e59c700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.40:3300/0 conn(0x7f7220003720 0x7f7220005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:04:22.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:22.713+0000 7f723e59c700 1 -- 172.21.15.40:0/4225006515 >> v2:172.21.15.40:3300/0 conn(0x7f7220003720 msgr2=0x7f7220005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:04:22.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:22.713+0000 7f723dd9b700 1 -- 172.21.15.40:0/4225006515 >> v1:172.21.15.40:6789/0 conn(0x7f7220006280 legacy=0x7f7220008720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:04:22.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:22.713+0000 7f723e59c700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.40:3300/0 conn(0x7f7220003720 0x7f7220005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:04:24.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.301+0000 7f722ffff700 1 -- 172.21.15.40:0/4225006515 >> v2:172.21.15.40:3300/0 conn(0x7f7220003720 msgr2=0x7f7220005ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:04:24.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.301+0000 7f722ffff700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.40:3300/0 conn(0x7f7220003720 0x7f7220005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.301+0000 7f722ffff700 1 -- 172.21.15.40:0/4225006515 >> v1:172.21.15.40:6789/0 conn(0x7f7220006280 legacy=0x7f7220008720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:04:24.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.301+0000 7f723ed9d700 1 -- 172.21.15.40:0/4225006515 reap_dead start 2024-09-13T18:04:24.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.301+0000 7f722ffff700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.40:3300/0 conn(0x7f7220003720 0x7f7220005ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.301+0000 7f722ffff700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f7220008e90 0x7f7220009340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.301+0000 7f722ffff700 1 -- 172.21.15.40:0/4225006515 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f722000b000 con 0x7f7220006280 2024-09-13T18:04:24.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.301+0000 7f722ffff700 1 -- 172.21.15.40:0/4225006515 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72380dd330 con 0x7f7220003720 2024-09-13T18:04:24.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.301+0000 7f723dd9b700 1 -- 172.21.15.40:0/4225006515 >> v1:172.21.15.40:6789/0 conn(0x7f7220006280 legacy=0x7f7220008720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:04:24.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.301+0000 7f722ffff700 1 -- 172.21.15.40:0/4225006515 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f723804cf30 con 0x7f7220008e90 2024-09-13T18:04:24.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.301+0000 7f723e59c700 1 -- 172.21.15.40:0/4225006515 >> v2:172.21.15.40:3300/0 conn(0x7f7220003720 msgr2=0x7f7220005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:04:24.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.301+0000 7f723e59c700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.40:3300/0 conn(0x7f7220003720 0x7f7220005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:24.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.301+0000 7f723ed9d700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.43:3300/0 conn(0x7f7220008e90 0x7f7220009340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:24.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.301+0000 7f723ed9d700 1 -- 172.21.15.40:0/4225006515 >> v2:172.21.15.40:3300/0 conn(0x7f7220003720 msgr2=0x7f7220005ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:04:24.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.301+0000 7f723ed9d700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.40:3300/0 conn(0x7f7220003720 0x7f7220005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.301+0000 7f723ed9d700 1 -- 172.21.15.40:0/4225006515 >> v1:172.21.15.40:6789/0 conn(0x7f7220006280 legacy=0x7f7220008720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:04:24.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.301+0000 7f723ed9d700 1 -- 172.21.15.40:0/4225006515 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723804d210 con 0x7f7220008e90 2024-09-13T18:04:24.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.301+0000 7f723ed9d700 1 --2- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7220008e90 0x7f7220009340 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7240069620 tx=0x7f7240069e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea52289046852c1d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:24.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.305+0000 7f723d59a700 1 -- 172.21.15.40:0/4225006515 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7240067dd0 con 0x7f7220008e90 2024-09-13T18:04:24.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.305+0000 7f723d59a700 1 -- 172.21.15.40:0/4225006515 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7240072040 con 0x7f7220008e90 2024-09-13T18:04:24.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.305+0000 7f723d59a700 1 -- 172.21.15.40:0/4225006515 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72400744e0 con 0x7f7220008e90 2024-09-13T18:04:24.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.305+0000 7f723f59e700 1 -- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7220008e90 msgr2=0x7f7220009340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:24.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.305+0000 7f723f59e700 1 --2- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7220008e90 0x7f7220009340 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7240069620 tx=0x7f7240069e70 comp rx=0 tx=0).stop 2024-09-13T18:04:24.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.305+0000 7f723f59e700 1 -- 172.21.15.40:0/4225006515 shutdown_connections 2024-09-13T18:04:24.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.305+0000 7f723f59e700 1 --2- 172.21.15.40:0/4225006515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7220008e90 0x7f7220009340 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.305+0000 7f723f59e700 1 --2- 172.21.15.40:0/4225006515 >> v2:172.21.15.40:3300/0 conn(0x7f7220003720 0x7f7220005ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.305+0000 7f723f59e700 1 -- 172.21.15.40:0/4225006515 >> 172.21.15.40:0/4225006515 conn(0x7f7238009a40 msgr2=0x7f7238009e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:24.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.305+0000 7f723f59e700 1 -- 172.21.15.40:0/4225006515 shutdown_connections 2024-09-13T18:04:24.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.305+0000 7f723f59e700 1 -- 172.21.15.40:0/4225006515 wait complete. 2024-09-13T18:04:24.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723f59e700 1 Processor -- start 2024-09-13T18:04:24.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723f59e700 1 -- start start 2024-09-13T18:04:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723f59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7220003720 0x7f72380d2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723f59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7220006280 0x7f7238037620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723f59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7220008e90 0x7f7238037b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723f59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72380de890 con 0x7f7220006280 2024-09-13T18:04:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723f59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72380ddd80 con 0x7f7220003720 2024-09-13T18:04:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723f59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7238009020 con 0x7f7220008e90 2024-09-13T18:04:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723dd9b700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7220006280 msgr2=0x7f7238037620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:04:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723dd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7220006280 0x7f7238037620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723e59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7220003720 0x7f72380d2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723ed9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7220008e90 0x7f7238037b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723e59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7220003720 0x7f72380d2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:37916/0 (socket says 172.21.15.40:37916) 2024-09-13T18:04:24.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723ed9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7220008e90 0x7f7238037b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:36804/0 (socket says 172.21.15.40:36804) 2024-09-13T18:04:24.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723e59c700 1 -- 172.21.15.40:0/1569876997 learned_addr learned my addr 172.21.15.40:0/1569876997 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:24.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723ed9d700 1 -- 172.21.15.40:0/1569876997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7220003720 msgr2=0x7f72380d2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723ed9d700 1 --2- 172.21.15.40:0/1569876997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7220003720 0x7f72380d2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723ed9d700 1 -- 172.21.15.40:0/1569876997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7220006280 msgr2=0x7f7238037620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:04:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723ed9d700 1 --2- 172.21.15.40:0/1569876997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7220006280 0x7f7238037620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723ed9d700 1 -- 172.21.15.40:0/1569876997 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724004f5c0 con 0x7f7220008e90 2024-09-13T18:04:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723ed9d700 1 --2- 172.21.15.40:0/1569876997 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7220008e90 0x7f7238037b60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7240000b60 tx=0x7f7240068470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f722effd700 1 -- 172.21.15.40:0/1569876997 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724009d070 con 0x7f7220008e90 2024-09-13T18:04:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f722effd700 1 -- 172.21.15.40:0/1569876997 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7240074800 con 0x7f7220008e90 2024-09-13T18:04:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723f59e700 1 -- 172.21.15.40:0/1569876997 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72380097b0 con 0x7f7220008e90 2024-09-13T18:04:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f722effd700 1 -- 172.21.15.40:0/1569876997 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72400721b0 con 0x7f7220008e90 2024-09-13T18:04:24.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.309+0000 7f723f59e700 1 -- 172.21.15.40:0/1569876997 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72380094d0 con 0x7f7220008e90 2024-09-13T18:04:24.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.313+0000 7f722effd700 1 -- 172.21.15.40:0/1569876997 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7240072330 con 0x7f7220008e90 2024-09-13T18:04:24.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.313+0000 7f723f59e700 1 -- 172.21.15.40:0/1569876997 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72380dfca0 con 0x7f7220008e90 2024-09-13T18:04:24.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.313+0000 7f722effd700 1 --2- 172.21.15.40:0/1569876997 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f72200677c0 0x7f7220069c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.313+0000 7f722effd700 1 -- 172.21.15.40:0/1569876997 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f72400ff320 con 0x7f7220008e90 2024-09-13T18:04:24.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.313+0000 7f723e59c700 1 --2- 172.21.15.40:0/1569876997 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f72200677c0 0x7f7220069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:24.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.313+0000 7f723e59c700 1 --2- 172.21.15.40:0/1569876997 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f72200677c0 0x7f7220069c80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7230006eb0 tx=0x7f7230008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:24.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.325+0000 7f722effd700 1 -- 172.21.15.40:0/1569876997 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f72380dfca0 con 0x7f7220008e90 2024-09-13T18:04:24.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.577+0000 7f723f59e700 1 -- 172.21.15.40:0/1569876997 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f72380dfca0 con 0x7f7220008e90 2024-09-13T18:04:24.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.581+0000 7f722effd700 1 -- 172.21.15.40:0/1569876997 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f72380dfca0 con 0x7f7220008e90 2024-09-13T18:04:24.591 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":110,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:04:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.581+0000 7f723f59e700 1 -- 172.21.15.40:0/1569876997 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f72200677c0 msgr2=0x7f7220069c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.581+0000 7f723f59e700 1 --2- 172.21.15.40:0/1569876997 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f72200677c0 0x7f7220069c80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7230006eb0 tx=0x7f7230008040 comp rx=0 tx=0).stop 2024-09-13T18:04:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.581+0000 7f723f59e700 1 -- 172.21.15.40:0/1569876997 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7220008e90 msgr2=0x7f7238037b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.581+0000 7f723f59e700 1 --2- 172.21.15.40:0/1569876997 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7220008e90 0x7f7238037b60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7240000b60 tx=0x7f7240068470 comp rx=0 tx=0).stop 2024-09-13T18:04:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.581+0000 7f723f59e700 1 -- 172.21.15.40:0/1569876997 shutdown_connections 2024-09-13T18:04:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.581+0000 7f723f59e700 1 --2- 172.21.15.40:0/1569876997 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f72200677c0 0x7f7220069c80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.581+0000 7f723f59e700 1 --2- 172.21.15.40:0/1569876997 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7220008e90 0x7f7238037b60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.581+0000 7f723f59e700 1 --2- 172.21.15.40:0/1569876997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7220006280 0x7f7238037620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.581+0000 7f723f59e700 1 --2- 172.21.15.40:0/1569876997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7220003720 0x7f72380d2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.581+0000 7f723f59e700 1 -- 172.21.15.40:0/1569876997 >> 172.21.15.40:0/1569876997 conn(0x7f7238009a40 msgr2=0x7f723800e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.581+0000 7f723f59e700 1 -- 172.21.15.40:0/1569876997 shutdown_connections 2024-09-13T18:04:24.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.581+0000 7f723f59e700 1 -- 172.21.15.40:0/1569876997 wait complete. 2024-09-13T18:04:24.609 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T18:04:24.610 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-13T18:04:24.610 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:04:24.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.725+0000 7f6d51b4d700 1 Processor -- start 2024-09-13T18:04:24.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.725+0000 7f6d51b4d700 1 -- start start 2024-09-13T18:04:24.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.725+0000 7f6d51b4d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d4c0c3de0 0x7f6d4c06e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.725+0000 7f6d51b4d700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d4c06e580 con 0x7f6d4c06dde0 2024-09-13T18:04:24.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.725+0000 7f6d4bfff700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f6d4c06dde0 legacy=0x7f6d4c0cd3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:04:24.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.725+0000 7f6d51b4d700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d4c151980 con 0x7f6d4c1515a0 2024-09-13T18:04:24.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.725+0000 7f6d51b4d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c0c3930 con 0x7f6d4c0c3de0 2024-09-13T18:04:24.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.729+0000 7f6d5134c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d4c0c3de0 0x7f6d4c06e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:24.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.729+0000 7f6d5134c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d4c0c3de0 0x7f6d4c06e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:36806/0 (socket says 172.21.15.40:36806) 2024-09-13T18:04:24.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.729+0000 7f6d5134c700 1 -- 172.21.15.40:0/4205206648 learned_addr learned my addr 172.21.15.40:0/4205206648 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:24.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.729+0000 7f6d50b4b700 1 --1- 172.21.15.40:0/4205206648 >> v1:172.21.15.43:6789/0 conn(0x7f6d4c1515a0 0x7f6d4c15a810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:36406/0 (socket says 172.21.15.40:36406) 2024-09-13T18:04:24.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.729+0000 7f6d4b7fe700 1 -- 172.21.15.40:0/4205206648 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2031289680 0 0) 0x7f6d4c151980 con 0x7f6d4c1515a0 2024-09-13T18:04:24.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.729+0000 7f6d4b7fe700 1 -- 172.21.15.40:0/4205206648 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d3c0035d0 con 0x7f6d4c1515a0 2024-09-13T18:04:24.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.729+0000 7f6d5134c700 1 -- 172.21.15.40:0/4205206648 >> v1:172.21.15.43:6789/0 conn(0x7f6d4c1515a0 legacy=0x7f6d4c15a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:24.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.729+0000 7f6d5134c700 1 -- 172.21.15.40:0/4205206648 >> v1:172.21.15.40:6789/0 conn(0x7f6d4c06dde0 legacy=0x7f6d4c0cd3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:04:24.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.729+0000 7f6d5134c700 1 -- 172.21.15.40:0/4205206648 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4c0c2020 con 0x7f6d4c0c3de0 2024-09-13T18:04:24.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.729+0000 7f6d5134c700 1 --2- 172.21.15.40:0/4205206648 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d4c0c3de0 0x7f6d4c06e960 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0c2210 tx=0x7f6d4400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c25ef352c5ff4e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:24.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.729+0000 7f6d4b7fe700 1 -- 172.21.15.40:0/4205206648 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4400e070 con 0x7f6d4c0c3de0 2024-09-13T18:04:24.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.729+0000 7f6d4b7fe700 1 -- 172.21.15.40:0/4205206648 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4400b8d0 con 0x7f6d4c0c3de0 2024-09-13T18:04:24.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.729+0000 7f6d4b7fe700 1 -- 172.21.15.40:0/4205206648 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4400bc40 con 0x7f6d4c0c3de0 2024-09-13T18:04:24.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.729+0000 7f6d51b4d700 1 -- 172.21.15.40:0/4205206648 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d4c0c3de0 msgr2=0x7f6d4c06e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:24.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.729+0000 7f6d51b4d700 1 --2- 172.21.15.40:0/4205206648 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d4c0c3de0 0x7f6d4c06e960 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0c2210 tx=0x7f6d4400b330 comp rx=0 tx=0).stop 2024-09-13T18:04:24.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.729+0000 7f6d51b4d700 1 -- 172.21.15.40:0/4205206648 shutdown_connections 2024-09-13T18:04:24.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.729+0000 7f6d51b4d700 1 --2- 172.21.15.40:0/4205206648 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d4c0c3de0 0x7f6d4c06e960 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.729+0000 7f6d51b4d700 1 -- 172.21.15.40:0/4205206648 >> 172.21.15.40:0/4205206648 conn(0x7f6d4c072070 msgr2=0x7f6d4c072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:24.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.729+0000 7f6d51b4d700 1 -- 172.21.15.40:0/4205206648 shutdown_connections 2024-09-13T18:04:24.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.729+0000 7f6d51b4d700 1 -- 172.21.15.40:0/4205206648 wait complete. 2024-09-13T18:04:24.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.733+0000 7f6d51b4d700 1 Processor -- start 2024-09-13T18:04:24.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.733+0000 7f6d51b4d700 1 -- start start 2024-09-13T18:04:24.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.733+0000 7f6d51b4d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d4c06dde0 0x7f6d4c151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.733+0000 7f6d51b4d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c0c3de0 0x7f6d4c143d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.733+0000 7f6d51b4d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d4c1515a0 0x7f6d4c152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.733+0000 7f6d50b4b700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d4c06dde0 msgr2=0x7f6d4c151b50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:04:24.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.733+0000 7f6d51b4d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c144290 con 0x7f6d4c06dde0 2024-09-13T18:04:24.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.733+0000 7f6d50b4b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d4c06dde0 0x7f6d4c151b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:24.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.733+0000 7f6d51b4d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c1e18c0 con 0x7f6d4c0c3de0 2024-09-13T18:04:24.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.733+0000 7f6d51b4d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c0cb420 con 0x7f6d4c1515a0 2024-09-13T18:04:24.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.733+0000 7f6d4bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c0c3de0 0x7f6d4c143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:24.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.733+0000 7f6d5134c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d4c1515a0 0x7f6d4c152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:24.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.733+0000 7f6d4bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c0c3de0 0x7f6d4c143d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:37918/0 (socket says 172.21.15.40:37918) 2024-09-13T18:04:24.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.733+0000 7f6d4bfff700 1 -- 172.21.15.40:0/730381581 learned_addr learned my addr 172.21.15.40:0/730381581 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:24.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.733+0000 7f6d4bfff700 1 -- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d4c1515a0 msgr2=0x7f6d4c152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:24.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.733+0000 7f6d4bfff700 1 --2- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d4c1515a0 0x7f6d4c152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.733+0000 7f6d4bfff700 1 -- 172.21.15.40:0/730381581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d4c06dde0 msgr2=0x7f6d4c151b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:04:24.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.733+0000 7f6d4bfff700 1 --2- 172.21.15.40:0/730381581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d4c06dde0 0x7f6d4c151b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.733+0000 7f6d4bfff700 1 -- 172.21.15.40:0/730381581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d44009b30 con 0x7f6d4c0c3de0 2024-09-13T18:04:24.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.733+0000 7f6d5134c700 1 --2- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6d4c1515a0 0x7f6d4c152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:04:24.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.737+0000 7f6d4bfff700 1 --2- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c0c3de0 0x7f6d4c143d50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800b780 tx=0x7f6d3800ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:24.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.737+0000 7f6d497fa700 1 -- 172.21.15.40:0/730381581 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38003e70 con 0x7f6d4c0c3de0 2024-09-13T18:04:24.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.737+0000 7f6d51b4d700 1 -- 172.21.15.40:0/730381581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d4c1e1b60 con 0x7f6d4c0c3de0 2024-09-13T18:04:24.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.737+0000 7f6d497fa700 1 -- 172.21.15.40:0/730381581 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d38005070 con 0x7f6d4c0c3de0 2024-09-13T18:04:24.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.737+0000 7f6d497fa700 1 -- 172.21.15.40:0/730381581 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38005400 con 0x7f6d4c0c3de0 2024-09-13T18:04:24.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.737+0000 7f6d51b4d700 1 -- 172.21.15.40:0/730381581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4c0cb700 con 0x7f6d4c0c3de0 2024-09-13T18:04:24.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.737+0000 7f6d51b4d700 1 --2- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c152b50 0x7f6d4c1f0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.737+0000 7f6d51b4d700 1 -- 172.21.15.40:0/730381581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c074750 con 0x7f6d4c152b50 2024-09-13T18:04:24.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.737+0000 7f6d50b4b700 1 --2- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c152b50 0x7f6d4c1f0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:24.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.737+0000 7f6d50b4b700 1 -- 172.21.15.40:0/730381581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6d4c158020 con 0x7f6d4c152b50 2024-09-13T18:04:24.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.737+0000 7f6d50b4b700 1 --2- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c152b50 0x7f6d4c1f0400 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000c470 tx=0x7f6d4000c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:24.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.737+0000 7f6d497fa700 1 -- 172.21.15.40:0/730381581 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6d38005560 con 0x7f6d4c0c3de0 2024-09-13T18:04:24.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.741+0000 7f6d497fa700 1 --2- 172.21.15.40:0/730381581 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6d300666e0 0x7f6d30068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.741+0000 7f6d497fa700 1 -- 172.21.15.40:0/730381581 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6d380837d0 con 0x7f6d4c0c3de0 2024-09-13T18:04:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.741+0000 7f6d5134c700 1 --2- 172.21.15.40:0/730381581 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6d300666e0 0x7f6d30068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.741+0000 7f6d497fa700 1 -- 172.21.15.40:0/730381581 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d40005cd0 con 0x7f6d4c152b50 2024-09-13T18:04:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.741+0000 7f6d497fa700 1 -- 172.21.15.40:0/730381581 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6d40005e30 con 0x7f6d4c152b50 2024-09-13T18:04:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.741+0000 7f6d497fa700 1 -- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c152b50 msgr2=0x7f6d4c1f0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.741+0000 7f6d497fa700 1 --2- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c152b50 0x7f6d4c1f0400 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000c470 tx=0x7f6d4000c830 comp rx=0 tx=0).stop 2024-09-13T18:04:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.741+0000 7f6d497fa700 1 -- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c152b50 msgr2=0x7f6d4c1f0400 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:04:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.741+0000 7f6d497fa700 1 --2- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c152b50 0x7f6d4c1f0400 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.745+0000 7f6d5134c700 1 --2- 172.21.15.40:0/730381581 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6d300666e0 0x7f6d30068ba0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6d44009e50 tx=0x7f6d44009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:24.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.749+0000 7f6d51b4d700 1 --2- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c1f0940 0x7f6d4c1f0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.749+0000 7f6d51b4d700 1 -- 172.21.15.40:0/730381581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c157d40 con 0x7f6d4c1f0940 2024-09-13T18:04:24.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.749+0000 7f6d50b4b700 1 --2- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c1f0940 0x7f6d4c1f0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:24.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.749+0000 7f6d50b4b700 1 -- 172.21.15.40:0/730381581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6d4c0cdae0 con 0x7f6d4c1f0940 2024-09-13T18:04:24.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.753+0000 7f6d50b4b700 1 --2- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c1f0940 0x7f6d4c1f0d50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6d40001080 tx=0x7f6d40000fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:24.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.753+0000 7f6d497fa700 1 -- 172.21.15.40:0/730381581 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d40007710 con 0x7f6d4c1f0940 2024-09-13T18:04:24.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.753+0000 7f6d497fa700 1 -- 172.21.15.40:0/730381581 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4939 (secure 0 0 0) 0x7f6d40023410 con 0x7f6d4c1f0940 2024-09-13T18:04:24.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.753+0000 7f6d497fa700 1 -- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c1f0940 msgr2=0x7f6d4c1f0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:24.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.753+0000 7f6d497fa700 1 --2- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c1f0940 0x7f6d4c1f0d50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6d40001080 tx=0x7f6d40000fe0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.753+0000 7f6d497fa700 1 -- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c1f0940 msgr2=0x7f6d4c1f0d50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:04:24.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.753+0000 7f6d497fa700 1 --2- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c1f0940 0x7f6d4c1f0d50 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6d40001080 tx=0x7f6d40000fe0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.764 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:04:24.765 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:04:24.765 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:04:24.765 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:04:24.765 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 110, 2024-09-13T18:04:24.765 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:04:24.765 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:04:24.765 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:04:24.765 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:24.765 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:04:24.765 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:04:24.765 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:04:24.765 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:04:24.765 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:24.766 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:24.766 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:24.766 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:24.766 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:24.766 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:24.766 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:24.766 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:24.766 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:24.766 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:24.766 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:24.766 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:04:24.766 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:04:24.766 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:24.766 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:24.767 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:24.767 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:24.767 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:24.767 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:24.767 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:24.767 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:24.767 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:24.767 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:24.767 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:24.767 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.767 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:04:24.767 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:04:24.767 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:04:24.767 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:04:24.768 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:04:24.768 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:04:24.768 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:04:24.768 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:04:24.768 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:04:24.768 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:04:24.768 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:04:24.768 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:04:24.768 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:04:24.768 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:04:24.768 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:04:24.768 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:04:24.768 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:04:24.769 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:24.769 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:24.769 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:24.769 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:24.769 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:24.769 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:24.769 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:24.769 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:24.769 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:24.769 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:24.769 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:24.769 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:04:24.769 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.770 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:04:24.770 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.770 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:04:24.770 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:04:24.770 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:24.770 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:24.770 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.770 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:24.770 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:04:24.770 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:24.770 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.770 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.770 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:24.771 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:04:24.771 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:24.771 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:24.771 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:24.771 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.771 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:04:24.771 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:04:24.771 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:24.771 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:24.771 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:24.771 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.771 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.771 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:04:24.771 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:04:24.771 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:24.772 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:24.772 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.772 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:24.772 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:04:24.772 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:24.772 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.772 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.772 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:24.772 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:04:24.772 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:24.772 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:24.772 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:24.772 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.772 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:04:24.773 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:04:24.773 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:24.773 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:24.773 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:24.773 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.773 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.773 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:04:24.773 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:04:24.773 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:24.773 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:24.773 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.773 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:24.773 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:04:24.774 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:24.774 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.774 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.774 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:24.774 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:04:24.774 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:24.774 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:24.774 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:24.774 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.774 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:04:24.774 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:04:24.774 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:24.774 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:24.774 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:24.775 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:24.775 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:24.775 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.775 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:04:24.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:04:24.775 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.775 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:24.775 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:24.775 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:04:24.775 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:24.775 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:24.775 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:04:24.775 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.775 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:24.776 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:24.776 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:04:24.776 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:24.776 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:24.776 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:04:24.776 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.776 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:24.776 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:24.776 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:04:24.776 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:24.776 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:24.776 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:04:24.776 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.777 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:24.777 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:24.777 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:04:24.777 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:24.777 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:24.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:04:24.777 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.777 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:24.777 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:24.777 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:04:24.777 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:24.777 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:24.777 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.777 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:04:24.778 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:04:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.753+0000 7f6d51b4d700 1 -- 172.21.15.40:0/730381581 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6d300666e0 msgr2=0x7f6d30068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.753+0000 7f6d51b4d700 1 --2- 172.21.15.40:0/730381581 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6d300666e0 0x7f6d30068ba0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6d44009e50 tx=0x7f6d44009000 comp rx=0 tx=0).stop 2024-09-13T18:04:24.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.753+0000 7f6d51b4d700 1 -- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c0c3de0 msgr2=0x7f6d4c143d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:24.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.753+0000 7f6d5134c700 1 -- 172.21.15.40:0/730381581 reap_dead start 2024-09-13T18:04:24.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.753+0000 7f6d51b4d700 1 --2- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c0c3de0 0x7f6d4c143d50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6d3800b780 tx=0x7f6d3800ba90 comp rx=0 tx=0).stop 2024-09-13T18:04:24.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.753+0000 7f6d51b4d700 1 -- 172.21.15.40:0/730381581 shutdown_connections 2024-09-13T18:04:24.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.753+0000 7f6d51b4d700 1 --2- 172.21.15.40:0/730381581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d4c0c3de0 0x7f6d4c143d50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.753+0000 7f6d51b4d700 1 -- 172.21.15.40:0/730381581 >> 172.21.15.40:0/730381581 conn(0x7f6d4c072070 msgr2=0x7f6d4c14a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:24.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.753+0000 7f6d51b4d700 1 -- 172.21.15.40:0/730381581 shutdown_connections 2024-09-13T18:04:24.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.753+0000 7f6d51b4d700 1 -- 172.21.15.40:0/730381581 wait complete. 2024-09-13T18:04:24.782 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:04:24.782 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T18:04:24.782 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:04:24.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.885+0000 7f8fb9025700 1 Processor -- start 2024-09-13T18:04:24.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.885+0000 7f8fb9025700 1 -- start start 2024-09-13T18:04:24.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.885+0000 7f8fb9025700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac045a40 0x7f8fac045e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.885+0000 7f8fb9025700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8fac007c70 0x7f8fac0df190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.885+0000 7f8fb9025700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fac0334f0 con 0x7f8fac0df6d0 2024-09-13T18:04:24.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.885+0000 7f8fb9025700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fac033a00 con 0x7f8fac007c70 2024-09-13T18:04:24.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.885+0000 7f8fb9025700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac046360 con 0x7f8fac045a40 2024-09-13T18:04:24.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.885+0000 7f8fb37fe700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f8fac007c70 msgr2=0x7f8fac0df190 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:04:24.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.885+0000 7f8fb8824700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f8fac0df6d0 legacy=0x7f8fac0db400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:04:24.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.885+0000 7f8fb37fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8fac007c70 0x7f8fac0df190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:24.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.885+0000 7f8fb3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac045a40 0x7f8fac045e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:24.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.885+0000 7f8fb3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac045a40 0x7f8fac045e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:36824/0 (socket says 172.21.15.40:36824) 2024-09-13T18:04:24.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.885+0000 7f8fb3fff700 1 -- 172.21.15.40:0/3297168783 learned_addr learned my addr 172.21.15.40:0/3297168783 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:24.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.889+0000 7f8fb3fff700 1 -- 172.21.15.40:0/3297168783 >> v2:172.21.15.40:3300/0 conn(0x7f8fac007c70 msgr2=0x7f8fac0df190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:04:24.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.889+0000 7f8fb3fff700 1 --2- 172.21.15.40:0/3297168783 >> v2:172.21.15.40:3300/0 conn(0x7f8fac007c70 0x7f8fac0df190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.889+0000 7f8fb3fff700 1 -- 172.21.15.40:0/3297168783 >> v1:172.21.15.40:6789/0 conn(0x7f8fac0df6d0 legacy=0x7f8fac0db400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:04:24.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.889+0000 7f8fb3fff700 1 -- 172.21.15.40:0/3297168783 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac033720 con 0x7f8fac045a40 2024-09-13T18:04:24.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.889+0000 7f8fb3fff700 1 --2- 172.21.15.40:0/3297168783 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac045a40 0x7f8fac045e20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009910 tx=0x7f8fa8009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d8604429cebe44e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:24.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.889+0000 7f8fb2ffd700 1 -- 172.21.15.40:0/3297168783 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8005e50 con 0x7f8fac045a40 2024-09-13T18:04:24.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.889+0000 7f8fb2ffd700 1 -- 172.21.15.40:0/3297168783 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa8009e60 con 0x7f8fac045a40 2024-09-13T18:04:24.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.889+0000 7f8fb2ffd700 1 -- 172.21.15.40:0/3297168783 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8004250 con 0x7f8fac045a40 2024-09-13T18:04:24.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.889+0000 7f8fb9025700 1 -- 172.21.15.40:0/3297168783 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac045a40 msgr2=0x7f8fac045e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:24.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.889+0000 7f8fb9025700 1 --2- 172.21.15.40:0/3297168783 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac045a40 0x7f8fac045e20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009910 tx=0x7f8fa8009c20 comp rx=0 tx=0).stop 2024-09-13T18:04:24.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.889+0000 7f8fb9025700 1 -- 172.21.15.40:0/3297168783 shutdown_connections 2024-09-13T18:04:24.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.889+0000 7f8fb9025700 1 --2- 172.21.15.40:0/3297168783 >> v2:172.21.15.40:3300/0 conn(0x7f8fac007c70 0x7f8fac0df190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.889+0000 7f8fb9025700 1 --2- 172.21.15.40:0/3297168783 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac045a40 0x7f8fac045e20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.889+0000 7f8fb9025700 1 -- 172.21.15.40:0/3297168783 >> 172.21.15.40:0/3297168783 conn(0x7f8fac0505e0 msgr2=0x7f8fac0509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:24.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.889+0000 7f8fb9025700 1 -- 172.21.15.40:0/3297168783 shutdown_connections 2024-09-13T18:04:24.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.889+0000 7f8fb9025700 1 -- 172.21.15.40:0/3297168783 wait complete. 2024-09-13T18:04:24.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.889+0000 7f8fb9025700 1 Processor -- start 2024-09-13T18:04:24.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.889+0000 7f8fb9025700 1 -- start start 2024-09-13T18:04:24.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb9025700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac007c70 0x7f8fac0ddef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb9025700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fac045a40 0x7f8fac0e8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb9025700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fac0df6d0 0x7f8fac0e8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb9025700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac0e9800 con 0x7f8fac0df6d0 2024-09-13T18:04:24.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb9025700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac0e83c0 con 0x7f8fac045a40 2024-09-13T18:04:24.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb9025700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac0db430 con 0x7f8fac007c70 2024-09-13T18:04:24.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb8824700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fac0df6d0 msgr2=0x7f8fac0e8d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:04:24.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb8824700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fac0df6d0 0x7f8fac0e8d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:24.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac007c70 0x7f8fac0ddef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:24.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fac045a40 0x7f8fac0e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:24.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac007c70 0x7f8fac0ddef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:36832/0 (socket says 172.21.15.40:36832) 2024-09-13T18:04:24.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb3fff700 1 -- 172.21.15.40:0/3361408948 learned_addr learned my addr 172.21.15.40:0/3361408948 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:24.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb3fff700 1 -- 172.21.15.40:0/3361408948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fac045a40 msgr2=0x7f8fac0e8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:24.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb3fff700 1 --2- 172.21.15.40:0/3361408948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fac045a40 0x7f8fac0e8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb3fff700 1 -- 172.21.15.40:0/3361408948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fac0df6d0 msgr2=0x7f8fac0e8d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:04:24.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb3fff700 1 --2- 172.21.15.40:0/3361408948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fac0df6d0 0x7f8fac0e8d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb3fff700 1 -- 172.21.15.40:0/3361408948 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa8009520 con 0x7f8fac007c70 2024-09-13T18:04:24.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb3fff700 1 --2- 172.21.15.40:0/3361408948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac007c70 0x7f8fac0ddef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8000c00 tx=0x7f8fa8004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:24.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb0ff9700 1 -- 172.21.15.40:0/3361408948 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8003d00 con 0x7f8fac007c70 2024-09-13T18:04:24.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb0ff9700 1 -- 172.21.15.40:0/3361408948 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa8003e60 con 0x7f8fac007c70 2024-09-13T18:04:24.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb9025700 1 -- 172.21.15.40:0/3361408948 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fac0e7de0 con 0x7f8fac007c70 2024-09-13T18:04:24.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb0ff9700 1 -- 172.21.15.40:0/3361408948 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa800b850 con 0x7f8fac007c70 2024-09-13T18:04:24.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb9025700 1 -- 172.21.15.40:0/3361408948 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fac0e8050 con 0x7f8fac007c70 2024-09-13T18:04:24.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb9025700 1 --2- 172.21.15.40:0/3361408948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac0d9f50 0x7f8fac0c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.893+0000 7f8fb9025700 1 -- 172.21.15.40:0/3361408948 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac008d50 con 0x7f8fac0d9f50 2024-09-13T18:04:24.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.897+0000 7f8fb37fe700 1 --2- 172.21.15.40:0/3361408948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac0d9f50 0x7f8fac0c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:24.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.897+0000 7f8fb0ff9700 1 -- 172.21.15.40:0/3361408948 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8fa800b9b0 con 0x7f8fac007c70 2024-09-13T18:04:24.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.897+0000 7f8fb37fe700 1 -- 172.21.15.40:0/3361408948 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8fac0de990 con 0x7f8fac0d9f50 2024-09-13T18:04:24.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.897+0000 7f8fb0ff9700 1 --2- 172.21.15.40:0/3361408948 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8f98066710 0x7f8f98068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.897+0000 7f8fb8824700 1 --2- 172.21.15.40:0/3361408948 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8f98066710 0x7f8f98068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:24.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.897+0000 7f8fb0ff9700 1 -- 172.21.15.40:0/3361408948 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8fa8085ae0 con 0x7f8fac007c70 2024-09-13T18:04:24.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.897+0000 7f8fb37fe700 1 --2- 172.21.15.40:0/3361408948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac0d9f50 0x7f8fac0c5e40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0de8b0 tx=0x7f8fa0009a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:24.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.897+0000 7f8fb0ff9700 1 -- 172.21.15.40:0/3361408948 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0005ac0 con 0x7f8fac0d9f50 2024-09-13T18:04:24.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.897+0000 7f8fb8824700 1 --2- 172.21.15.40:0/3361408948 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8f98066710 0x7f8f98068bd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8fb404f480 tx=0x7f8fb4065ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:24.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.897+0000 7f8fb0ff9700 1 -- 172.21.15.40:0/3361408948 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8fa0005c20 con 0x7f8fac0d9f50 2024-09-13T18:04:24.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.897+0000 7f8fb0ff9700 1 -- 172.21.15.40:0/3361408948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac0d9f50 msgr2=0x7f8fac0c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:24.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.897+0000 7f8fb0ff9700 1 --2- 172.21.15.40:0/3361408948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac0d9f50 0x7f8fac0c5e40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0de8b0 tx=0x7f8fa0009a30 comp rx=0 tx=0).stop 2024-09-13T18:04:24.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.897+0000 7f8fb0ff9700 1 -- 172.21.15.40:0/3361408948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac0d9f50 msgr2=0x7f8fac0c5e40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:04:24.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.897+0000 7f8fb0ff9700 1 --2- 172.21.15.40:0/3361408948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac0d9f50 0x7f8fac0c5e40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.909+0000 7f8fb9025700 1 --2- 172.21.15.40:0/3361408948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac04ae40 0x7f8fac0c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.909+0000 7f8fb9025700 1 -- 172.21.15.40:0/3361408948 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac0c68c0 con 0x7f8fac04ae40 2024-09-13T18:04:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.909+0000 7f8fb37fe700 1 --2- 172.21.15.40:0/3361408948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac04ae40 0x7f8fac0c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:24.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.913+0000 7f8fb37fe700 1 -- 172.21.15.40:0/3361408948 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8fac0062a0 con 0x7f8fac04ae40 2024-09-13T18:04:24.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.913+0000 7f8fb37fe700 1 --2- 172.21.15.40:0/3361408948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac04ae40 0x7f8fac0c6380 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00b380 tx=0x7f8fa0005a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:24.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.913+0000 7f8fb0ff9700 1 -- 172.21.15.40:0/3361408948 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0020070 con 0x7f8fac04ae40 2024-09-13T18:04:24.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.913+0000 7f8fb0ff9700 1 -- 172.21.15.40:0/3361408948 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4937 (secure 0 0 0) 0x7f8fa0005fc0 con 0x7f8fac04ae40 2024-09-13T18:04:24.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.913+0000 7f8fb0ff9700 1 -- 172.21.15.40:0/3361408948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac04ae40 msgr2=0x7f8fac0c6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:24.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.913+0000 7f8fb0ff9700 1 --2- 172.21.15.40:0/3361408948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac04ae40 0x7f8fac0c6380 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00b380 tx=0x7f8fa0005a50 comp rx=0 tx=0).stop 2024-09-13T18:04:24.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.913+0000 7f8fb0ff9700 1 -- 172.21.15.40:0/3361408948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac04ae40 msgr2=0x7f8fac0c6380 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:04:24.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.913+0000 7f8fb0ff9700 1 --2- 172.21.15.40:0/3361408948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac04ae40 0x7f8fac0c6380 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00b380 tx=0x7f8fa0005a50 comp rx=0 tx=0).stop 2024-09-13T18:04:24.925 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:04:24.925 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:04:24.925 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:04:24.925 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:04:24.926 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 110, 2024-09-13T18:04:24.926 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:04:24.926 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:04:24.926 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:04:24.926 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:24.926 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:04:24.926 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:04:24.926 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:04:24.926 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:04:24.926 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:24.926 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:24.926 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:24.926 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:24.926 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:24.927 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:24.927 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:24.927 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:24.927 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:24.927 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:24.927 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:24.927 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:04:24.927 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:04:24.927 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:24.927 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:24.927 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:24.927 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:24.927 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:24.927 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:24.928 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:24.928 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:24.928 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:24.928 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:24.928 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:24.928 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.928 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:04:24.928 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:04:24.928 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:04:24.928 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:04:24.928 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:04:24.928 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:04:24.928 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:04:24.929 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:04:24.929 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:04:24.929 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:04:24.929 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:04:24.929 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:04:24.929 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:04:24.929 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:04:24.929 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:04:24.929 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:04:24.929 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:04:24.929 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:24.929 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:24.929 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:24.929 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:24.930 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:24.930 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:24.930 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:24.930 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:24.930 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:24.930 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:24.930 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:24.930 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:04:24.930 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.930 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:04:24.930 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.930 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:04:24.930 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:04:24.931 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:24.931 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:24.931 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.931 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:24.931 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:04:24.931 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:24.931 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.931 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.931 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:24.931 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:04:24.931 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:24.931 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:24.931 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:24.931 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.932 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:04:24.932 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:04:24.932 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:24.932 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:24.932 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:24.932 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.932 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.932 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:04:24.932 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:04:24.932 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:24.932 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:24.932 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.932 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:24.932 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:04:24.932 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:24.933 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.933 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.933 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:24.933 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:04:24.933 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:24.933 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:24.933 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:24.933 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.933 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:04:24.933 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:04:24.933 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:24.933 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:24.933 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:24.934 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.934 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.934 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:04:24.934 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:04:24.934 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:24.934 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:24.934 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.934 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:24.934 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:04:24.934 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:24.934 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.934 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.934 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:24.934 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:04:24.934 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:24.935 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:24.935 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:24.935 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.935 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:04:24.935 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:04:24.935 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:24.935 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:24.935 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:24.935 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:24.935 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:24.935 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.935 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:04:24.935 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:04:24.935 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.935 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:24.935 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:24.936 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:04:24.936 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:24.936 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:24.936 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:04:24.936 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.936 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:24.936 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:24.936 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:04:24.936 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:24.936 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:24.936 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:04:24.936 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.936 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:24.936 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:24.937 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:04:24.937 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:24.937 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:24.937 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:04:24.937 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.937 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:24.937 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:24.937 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:04:24.937 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:24.937 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:24.937 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:04:24.937 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:24.937 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:24.937 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:24.938 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:04:24.938 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:24.938 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:24.938 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:24.938 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:04:24.938 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:04:24.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.913+0000 7f8fb9025700 1 -- 172.21.15.40:0/3361408948 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8f98066710 msgr2=0x7f8f98068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:24.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.913+0000 7f8fb9025700 1 --2- 172.21.15.40:0/3361408948 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8f98066710 0x7f8f98068bd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8fb404f480 tx=0x7f8fb4065ea0 comp rx=0 tx=0).stop 2024-09-13T18:04:24.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.913+0000 7f8fb9025700 1 -- 172.21.15.40:0/3361408948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac007c70 msgr2=0x7f8fac0ddef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:24.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.913+0000 7f8fb9025700 1 --2- 172.21.15.40:0/3361408948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fac007c70 0x7f8fac0ddef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8000c00 tx=0x7f8fa8004770 comp rx=0 tx=0).stop 2024-09-13T18:04:24.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.913+0000 7f8fb8824700 1 -- 172.21.15.40:0/3361408948 reap_dead start 2024-09-13T18:04:24.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.913+0000 7f8fb8824700 1 -- 172.21.15.40:0/3361408948 reap_dead start 2024-09-13T18:04:24.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.913+0000 7f8fb9025700 1 -- 172.21.15.40:0/3361408948 shutdown_connections 2024-09-13T18:04:24.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.913+0000 7f8fb9025700 1 -- 172.21.15.40:0/3361408948 >> 172.21.15.40:0/3361408948 conn(0x7f8fac0505e0 msgr2=0x7f8fac04cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:24.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.913+0000 7f8fb9025700 1 -- 172.21.15.40:0/3361408948 shutdown_connections 2024-09-13T18:04:24.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:24.913+0000 7f8fb9025700 1 -- 172.21.15.40:0/3361408948 wait complete. 2024-09-13T18:04:24.941 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:04:24.941 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:04:36.133 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:04:36.135 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:04:44.942 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T18:04:44.942 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-13T18:04:44.943 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T18:04:44.943 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T18:04:44.945 INFO:tasks.ceph.mon.a:Started 2024-09-13T18:04:44.945 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:04:44.946 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:04:45.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8712705700 1 Processor -- start 2024-09-13T18:04:45.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8712705700 1 -- start start 2024-09-13T18:04:45.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8712705700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f870c1515f0 0x7f870c0c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:45.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8712705700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f870c0663b0 0x7f870c0c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:45.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8712705700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f870c06ec70 con 0x7f870c151210 2024-09-13T18:04:45.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8712705700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f870c158070 con 0x7f870c0663b0 2024-09-13T18:04:45.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8712705700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870c157210 con 0x7f870c1515f0 2024-09-13T18:04:45.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8711f04700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f870c0663b0 0x7f870c0c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:45.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8711703700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f870c151210 0x7f870c06d490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46972/0 (socket says 172.21.15.40:46972) 2024-09-13T18:04:45.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8711f04700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f870c0663b0 0x7f870c0c4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45062/0 (socket says 172.21.15.40:45062) 2024-09-13T18:04:45.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8711703700 1 -- 172.21.15.40:0/4089430513 learned_addr learned my addr 172.21.15.40:0/4089430513 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:04:45.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8710f02700 1 --2- 172.21.15.40:0/4089430513 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f870c1515f0 0x7f870c0c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:45.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8711f04700 1 -- 172.21.15.40:0/4089430513 >> v2:172.21.15.40:3300/0 conn(0x7f870c0663b0 msgr2=0x7f870c0c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:04:45.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8711f04700 1 -- 172.21.15.40:0/4089430513 >> v2:172.21.15.40:3300/0 conn(0x7f870c0663b0 msgr2=0x7f870c0c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:45.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8711f04700 1 --2- 172.21.15.40:0/4089430513 >> v2:172.21.15.40:3300/0 conn(0x7f870c0663b0 0x7f870c0c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:45.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8711f04700 1 --2- 172.21.15.40:0/4089430513 >> v2:172.21.15.40:3300/0 conn(0x7f870c0663b0 0x7f870c0c4fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:45.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8710f02700 1 -- 172.21.15.40:0/4089430513 >> v2:172.21.15.40:3300/0 conn(0x7f870c0663b0 msgr2=0x7f870c0c4fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:04:45.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8710f02700 1 --2- 172.21.15.40:0/4089430513 >> v2:172.21.15.40:3300/0 conn(0x7f870c0663b0 0x7f870c0c4fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:45.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8710f02700 1 -- 172.21.15.40:0/4089430513 >> v1:172.21.15.40:6789/0 conn(0x7f870c151210 legacy=0x7f870c06d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:45.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8710f02700 1 -- 172.21.15.40:0/4089430513 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870c1574f0 con 0x7f870c1515f0 2024-09-13T18:04:45.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8710f02700 1 --2- 172.21.15.40:0/4089430513 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f870c1515f0 0x7f870c0c4a60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009850 tx=0x7f86fc009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec6130448fcec5e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:45.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8703fff700 1 -- 172.21.15.40:0/4089430513 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc011830 con 0x7f870c1515f0 2024-09-13T18:04:45.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8703fff700 1 -- 172.21.15.40:0/4089430513 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86fc011990 con 0x7f870c1515f0 2024-09-13T18:04:45.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8703fff700 1 -- 172.21.15.40:0/4089430513 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc011d20 con 0x7f870c1515f0 2024-09-13T18:04:45.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8712705700 1 -- 172.21.15.40:0/4089430513 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f870c1515f0 msgr2=0x7f870c0c4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:45.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.045+0000 7f8712705700 1 --2- 172.21.15.40:0/4089430513 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f870c1515f0 0x7f870c0c4a60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009850 tx=0x7f86fc009b60 comp rx=0 tx=0).stop 2024-09-13T18:04:45.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8712705700 1 -- 172.21.15.40:0/4089430513 shutdown_connections 2024-09-13T18:04:45.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8712705700 1 --2- 172.21.15.40:0/4089430513 >> v2:172.21.15.40:3300/0 conn(0x7f870c0663b0 0x7f870c0c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:45.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8712705700 1 --2- 172.21.15.40:0/4089430513 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f870c1515f0 0x7f870c0c4a60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8712705700 1 -- 172.21.15.40:0/4089430513 >> 172.21.15.40:0/4089430513 conn(0x7f870c06a390 msgr2=0x7f870c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8712705700 1 -- 172.21.15.40:0/4089430513 shutdown_connections 2024-09-13T18:04:45.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8712705700 1 -- 172.21.15.40:0/4089430513 wait complete. 2024-09-13T18:04:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8712705700 1 Processor -- start 2024-09-13T18:04:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8712705700 1 -- start start 2024-09-13T18:04:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8712705700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f870c0663b0 0x7f870c1f3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8712705700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f870c151210 0x7f870c146e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8712705700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f870c1515f0 0x7f870c0c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8712705700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c0c9600 con 0x7f870c0663b0 2024-09-13T18:04:45.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8712705700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c0c98e0 con 0x7f870c151210 2024-09-13T18:04:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8712705700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870c0c9bc0 con 0x7f870c1515f0 2024-09-13T18:04:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8711703700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f870c0663b0 0x7f870c1f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8711703700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f870c0663b0 0x7f870c1f3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45066/0 (socket says 172.21.15.40:45066) 2024-09-13T18:04:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8711703700 1 -- 172.21.15.40:0/2319986207 learned_addr learned my addr 172.21.15.40:0/2319986207 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8711703700 1 -- 172.21.15.40:0/2319986207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f870c0663b0 msgr2=0x7f870c1f3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:04:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8711703700 1 -- 172.21.15.40:0/2319986207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f870c0663b0 msgr2=0x7f870c1f3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:04:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8711703700 1 --2- 172.21.15.40:0/2319986207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f870c0663b0 0x7f870c1f3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8711f04700 1 --2- 172.21.15.40:0/2319986207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f870c1515f0 0x7f870c0c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8711703700 1 --2- 172.21.15.40:0/2319986207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f870c0663b0 0x7f870c1f3930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:45.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.049+0000 7f8710f02700 1 --2- 172.21.15.40:0/2319986207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f870c151210 0x7f870c146e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:45.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.053+0000 7f8710f02700 1 -- 172.21.15.40:0/2319986207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f870c1515f0 msgr2=0x7f870c0c3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:45.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.053+0000 7f8710f02700 1 --2- 172.21.15.40:0/2319986207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f870c1515f0 0x7f870c0c3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:45.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.053+0000 7f8710f02700 1 -- 172.21.15.40:0/2319986207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f870c0663b0 msgr2=0x7f870c1f3930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:04:45.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.053+0000 7f8710f02700 1 --2- 172.21.15.40:0/2319986207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f870c0663b0 0x7f870c1f3930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:45.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.053+0000 7f8710f02700 1 -- 172.21.15.40:0/2319986207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86fc0094b0 con 0x7f870c151210 2024-09-13T18:04:45.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.053+0000 7f8711f04700 1 --2- 172.21.15.40:0/2319986207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f870c1515f0 0x7f870c0c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:04:45.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.053+0000 7f8710f02700 1 --2- 172.21.15.40:0/2319986207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f870c151210 0x7f870c146e80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f86fc016000 tx=0x7f86fc004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:45.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.053+0000 7f8701ffb700 1 -- 172.21.15.40:0/2319986207 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc00b9b0 con 0x7f870c151210 2024-09-13T18:04:45.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.053+0000 7f8701ffb700 1 -- 172.21.15.40:0/2319986207 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86fc01f070 con 0x7f870c151210 2024-09-13T18:04:45.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.053+0000 7f8712705700 1 -- 172.21.15.40:0/2319986207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f870c0c9ea0 con 0x7f870c151210 2024-09-13T18:04:45.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.053+0000 7f8701ffb700 1 -- 172.21.15.40:0/2319986207 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc00bd70 con 0x7f870c151210 2024-09-13T18:04:45.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.053+0000 7f8712705700 1 -- 172.21.15.40:0/2319986207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870c0cb450 con 0x7f870c151210 2024-09-13T18:04:45.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.053+0000 7f8712705700 1 -- 172.21.15.40:0/2319986207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870c0c5fa0 con 0x7f870c151210 2024-09-13T18:04:45.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.053+0000 7f8701ffb700 1 -- 172.21.15.40:0/2319986207 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f86fc01a430 con 0x7f870c151210 2024-09-13T18:04:45.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.053+0000 7f8701ffb700 1 --2- 172.21.15.40:0/2319986207 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86f0066730 0x7f86f0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:45.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.057+0000 7f8711703700 1 --2- 172.21.15.40:0/2319986207 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86f0066730 0x7f86f0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:45.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.057+0000 7f8701ffb700 1 -- 172.21.15.40:0/2319986207 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f86fc021070 con 0x7f870c151210 2024-09-13T18:04:45.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.057+0000 7f8711703700 1 --2- 172.21.15.40:0/2319986207 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86f0066730 0x7f86f0068bf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8708007de0 tx=0x7f8708009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:45.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.065+0000 7f8701ffb700 1 -- 172.21.15.40:0/2319986207 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f86fc053270 con 0x7f870c151210 2024-09-13T18:04:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.317+0000 7f8712705700 1 -- 172.21.15.40:0/2319986207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f870c1f3560 con 0x7f870c151210 2024-09-13T18:04:45.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.317+0000 7f8701ffb700 1 -- 172.21.15.40:0/2319986207 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f86fc056890 con 0x7f870c151210 2024-09-13T18:04:45.330 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":110,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":22,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:04:45.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.321+0000 7f8712705700 1 -- 172.21.15.40:0/2319986207 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86f0066730 msgr2=0x7f86f0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:45.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.321+0000 7f8712705700 1 --2- 172.21.15.40:0/2319986207 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86f0066730 0x7f86f0068bf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8708007de0 tx=0x7f8708009040 comp rx=0 tx=0).stop 2024-09-13T18:04:45.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.321+0000 7f8712705700 1 -- 172.21.15.40:0/2319986207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f870c151210 msgr2=0x7f870c146e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.321+0000 7f8712705700 1 --2- 172.21.15.40:0/2319986207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f870c151210 0x7f870c146e80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f86fc016000 tx=0x7f86fc004770 comp rx=0 tx=0).stop 2024-09-13T18:04:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.321+0000 7f8712705700 1 -- 172.21.15.40:0/2319986207 shutdown_connections 2024-09-13T18:04:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.321+0000 7f8712705700 1 --2- 172.21.15.40:0/2319986207 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f86f0066730 0x7f86f0068bf0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.321+0000 7f8712705700 1 --2- 172.21.15.40:0/2319986207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f870c1515f0 0x7f870c0c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.321+0000 7f8712705700 1 --2- 172.21.15.40:0/2319986207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f870c151210 0x7f870c146e80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.321+0000 7f8712705700 1 --2- 172.21.15.40:0/2319986207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f870c0663b0 0x7f870c1f3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.321+0000 7f8712705700 1 -- 172.21.15.40:0/2319986207 >> 172.21.15.40:0/2319986207 conn(0x7f870c06a390 msgr2=0x7f870c14d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.321+0000 7f8712705700 1 -- 172.21.15.40:0/2319986207 shutdown_connections 2024-09-13T18:04:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:45.321+0000 7f8712705700 1 -- 172.21.15.40:0/2319986207 wait complete. 2024-09-13T18:04:48.349 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:04:48.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2359e700 1 Processor -- start 2024-09-13T18:04:48.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2359e700 1 -- start start 2024-09-13T18:04:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2359e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efe1c046f90 0x7efe1c04af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2359e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe1c007640 con 0x7efe1c0d8a20 2024-09-13T18:04:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2359e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe1c045470 con 0x7efe1c04ca40 2024-09-13T18:04:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2359e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe1c0473a0 con 0x7efe1c046f90 2024-09-13T18:04:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2259c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efe1c0d8a20 0x7efe1c006920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46974/0 (socket says 172.21.15.40:46974) 2024-09-13T18:04:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2259c700 1 -- 172.21.15.40:0/3154310674 learned_addr learned my addr 172.21.15.40:0/3154310674 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:04:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe22d9d700 1 --2- 172.21.15.40:0/3154310674 >> v2:172.21.15.40:3300/0 conn(0x7efe1c046f90 0x7efe1c04af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe22d9d700 1 -- 172.21.15.40:0/3154310674 >> v2:172.21.15.40:3300/0 conn(0x7efe1c046f90 msgr2=0x7efe1c04af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:04:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe22d9d700 1 -- 172.21.15.40:0/3154310674 >> v2:172.21.15.40:3300/0 conn(0x7efe1c046f90 msgr2=0x7efe1c04af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe22d9d700 1 --2- 172.21.15.40:0/3154310674 >> v2:172.21.15.40:3300/0 conn(0x7efe1c046f90 0x7efe1c04af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:48.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe22d9d700 1 --2- 172.21.15.40:0/3154310674 >> v2:172.21.15.40:3300/0 conn(0x7efe1c046f90 0x7efe1c04af10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2159a700 1 -- 172.21.15.40:0/3154310674 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947390366 0 0) 0x7efe1c045470 con 0x7efe1c04ca40 2024-09-13T18:04:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2159a700 1 -- 172.21.15.40:0/3154310674 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe0c0035d0 con 0x7efe1c04ca40 2024-09-13T18:04:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2159a700 1 -- 172.21.15.40:0/3154310674 <== mon.1 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 111944955 0 0) 0x7efe0c0035d0 con 0x7efe1c04ca40 2024-09-13T18:04:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2159a700 1 -- 172.21.15.40:0/3154310674 >> v2:172.21.15.40:3300/0 conn(0x7efe1c046f90 msgr2=0x7efe1c04af10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:04:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2159a700 1 --2- 172.21.15.40:0/3154310674 >> v2:172.21.15.40:3300/0 conn(0x7efe1c046f90 0x7efe1c04af10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:48.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2159a700 1 -- 172.21.15.40:0/3154310674 >> v1:172.21.15.40:6789/0 conn(0x7efe1c0d8a20 legacy=0x7efe1c006920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2159a700 1 -- 172.21.15.40:0/3154310674 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c046b00 con 0x7efe1c04ca40 2024-09-13T18:04:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2159a700 1 -- 172.21.15.40:0/3154310674 <== mon.1 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7efe18001670 con 0x7efe1c04ca40 2024-09-13T18:04:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2159a700 1 -- 172.21.15.40:0/3154310674 >> v1:172.21.15.43:6789/0 conn(0x7efe1c04ca40 legacy=0x7efe1c0e6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2159a700 1 --2- 172.21.15.40:0/3154310674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe0c003a20 0x7efe0c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2159a700 1 --2- 172.21.15.40:0/3154310674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe0c004870 0x7efe0c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2159a700 1 --2- 172.21.15.40:0/3154310674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe0c005350 0x7efe0c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2159a700 1 -- 172.21.15.40:0/3154310674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c0473a0 con 0x7efe0c004870 2024-09-13T18:04:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2159a700 1 -- 172.21.15.40:0/3154310674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c00c250 con 0x7efe0c003a20 2024-09-13T18:04:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2159a700 1 -- 172.21.15.40:0/3154310674 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c00c410 con 0x7efe0c005350 2024-09-13T18:04:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe21d9b700 1 --2- 172.21.15.40:0/3154310674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe0c004870 0x7efe0c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2259c700 1 --2- 172.21.15.40:0/3154310674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe0c003a20 0x7efe0c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:48.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe22d9d700 1 --2- 172.21.15.40:0/3154310674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe0c005350 0x7efe0c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe21d9b700 1 -- 172.21.15.40:0/3154310674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe0c005350 msgr2=0x7efe0c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:04:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe21d9b700 1 --2- 172.21.15.40:0/3154310674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe0c005350 0x7efe0c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe21d9b700 1 -- 172.21.15.40:0/3154310674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe0c003a20 msgr2=0x7efe0c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:04:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe21d9b700 1 --2- 172.21.15.40:0/3154310674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe0c003a20 0x7efe0c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe21d9b700 1 -- 172.21.15.40:0/3154310674 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0c0035d0 con 0x7efe0c004870 2024-09-13T18:04:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe22d9d700 1 -- 172.21.15.40:0/3154310674 reap_dead start 2024-09-13T18:04:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.477+0000 7efe2259c700 1 --2- 172.21.15.40:0/3154310674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe0c003a20 0x7efe0c003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:04:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.481+0000 7efe21d9b700 1 --2- 172.21.15.40:0/3154310674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe0c004870 0x7efe0c004cf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efe1800c160 tx=0x7efe0c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95553962c702a457 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.481+0000 7efe2159a700 1 -- 172.21.15.40:0/3154310674 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18007b50 con 0x7efe0c004870 2024-09-13T18:04:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.481+0000 7efe2159a700 1 -- 172.21.15.40:0/3154310674 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1800ccc0 con 0x7efe0c004870 2024-09-13T18:04:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.481+0000 7efe2159a700 1 -- 172.21.15.40:0/3154310674 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18012660 con 0x7efe0c004870 2024-09-13T18:04:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.481+0000 7efe2359e700 1 -- 172.21.15.40:0/3154310674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe0c004870 msgr2=0x7efe0c004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.481+0000 7efe2359e700 1 --2- 172.21.15.40:0/3154310674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe0c004870 0x7efe0c004cf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efe1800c160 tx=0x7efe0c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:04:48.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.481+0000 7efe2359e700 1 -- 172.21.15.40:0/3154310674 shutdown_connections 2024-09-13T18:04:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.481+0000 7efe2359e700 1 --2- 172.21.15.40:0/3154310674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe0c004870 0x7efe0c004cf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.481+0000 7efe2359e700 1 -- 172.21.15.40:0/3154310674 >> 172.21.15.40:0/3154310674 conn(0x7efe1c050590 msgr2=0x7efe1c0509a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.481+0000 7efe2359e700 1 -- 172.21.15.40:0/3154310674 shutdown_connections 2024-09-13T18:04:48.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.481+0000 7efe2359e700 1 -- 172.21.15.40:0/3154310674 wait complete. 2024-09-13T18:04:48.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.481+0000 7efe2359e700 1 Processor -- start 2024-09-13T18:04:48.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.481+0000 7efe2359e700 1 -- start start 2024-09-13T18:04:48.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.481+0000 7efe2359e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe0c004870 0x7efe1c0df7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:48.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.481+0000 7efe2359e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe1c048d20 0x7efe1c035120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:48.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.481+0000 7efe2359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe1c0c8db0 0x7efe1c0c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:48.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.481+0000 7efe2359e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c047900 con 0x7efe1c0c8db0 2024-09-13T18:04:48.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.481+0000 7efe2359e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c0c9770 con 0x7efe0c004870 2024-09-13T18:04:48.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.481+0000 7efe2359e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c047af0 con 0x7efe1c048d20 2024-09-13T18:04:48.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe22d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe1c0c8db0 0x7efe1c0c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:48.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe22d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe1c0c8db0 0x7efe1c0c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45080/0 (socket says 172.21.15.40:45080) 2024-09-13T18:04:48.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe2259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe0c004870 0x7efe1c0df7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:48.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe22d9d700 1 -- 172.21.15.40:0/4003309935 learned_addr learned my addr 172.21.15.40:0/4003309935 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:48.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe21d9b700 1 --2- 172.21.15.40:0/4003309935 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe1c048d20 0x7efe1c035120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:48.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe22d9d700 1 -- 172.21.15.40:0/4003309935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe1c0c8db0 msgr2=0x7efe1c0c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:04:48.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe22d9d700 1 -- 172.21.15.40:0/4003309935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe1c0c8db0 msgr2=0x7efe1c0c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:04:48.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe22d9d700 1 --2- 172.21.15.40:0/4003309935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe1c0c8db0 0x7efe1c0c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:48.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe22d9d700 1 --2- 172.21.15.40:0/4003309935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe1c0c8db0 0x7efe1c0c9230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:48.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe21d9b700 1 -- 172.21.15.40:0/4003309935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe0c004870 msgr2=0x7efe1c0df7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:48.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe21d9b700 1 --2- 172.21.15.40:0/4003309935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe0c004870 0x7efe1c0df7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:48.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe21d9b700 1 -- 172.21.15.40:0/4003309935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe1c0c8db0 msgr2=0x7efe1c0c9230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:04:48.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe21d9b700 1 --2- 172.21.15.40:0/4003309935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe1c0c8db0 0x7efe1c0c9230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:48.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe21d9b700 1 -- 172.21.15.40:0/4003309935 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c036350 con 0x7efe1c048d20 2024-09-13T18:04:48.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe2259c700 1 --2- 172.21.15.40:0/4003309935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe0c004870 0x7efe1c0df7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:04:48.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe21d9b700 1 --2- 172.21.15.40:0/4003309935 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe1c048d20 0x7efe1c035120 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efe18001050 tx=0x7efe1c007bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:48.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe12ffd700 1 -- 172.21.15.40:0/4003309935 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18005270 con 0x7efe1c048d20 2024-09-13T18:04:48.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe12ffd700 1 -- 172.21.15.40:0/4003309935 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe180053f0 con 0x7efe1c048d20 2024-09-13T18:04:48.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe2359e700 1 -- 172.21.15.40:0/4003309935 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe1c0cecc0 con 0x7efe1c048d20 2024-09-13T18:04:48.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe12ffd700 1 -- 172.21.15.40:0/4003309935 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe180038e0 con 0x7efe1c048d20 2024-09-13T18:04:48.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe2359e700 1 -- 172.21.15.40:0/4003309935 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe1c048240 con 0x7efe1c048d20 2024-09-13T18:04:48.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.485+0000 7efe2359e700 1 -- 172.21.15.40:0/4003309935 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe1c0459a0 con 0x7efe1c048d20 2024-09-13T18:04:48.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.489+0000 7efe12ffd700 1 -- 172.21.15.40:0/4003309935 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efe18022070 con 0x7efe1c048d20 2024-09-13T18:04:48.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.489+0000 7efe12ffd700 1 --2- 172.21.15.40:0/4003309935 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7efe04066760 0x7efe04068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:48.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.489+0000 7efe12ffd700 1 -- 172.21.15.40:0/4003309935 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7efe18085620 con 0x7efe1c048d20 2024-09-13T18:04:48.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.489+0000 7efe2259c700 1 --2- 172.21.15.40:0/4003309935 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7efe04066760 0x7efe04068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:48.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.489+0000 7efe2259c700 1 --2- 172.21.15.40:0/4003309935 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7efe04066760 0x7efe04068c20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efe1c003eb0 tx=0x7efe14001190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:48.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.497+0000 7efe12ffd700 1 -- 172.21.15.40:0/4003309935 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efe18056070 con 0x7efe1c048d20 2024-09-13T18:04:48.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.757+0000 7efe2359e700 1 -- 172.21.15.40:0/4003309935 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7efe1c0064f0 con 0x7efe1c048d20 2024-09-13T18:04:48.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.757+0000 7efe12ffd700 1 -- 172.21.15.40:0/4003309935 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7efe18072e60 con 0x7efe1c048d20 2024-09-13T18:04:48.764 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":110,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":26,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:04:48.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.757+0000 7efe2359e700 1 -- 172.21.15.40:0/4003309935 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7efe04066760 msgr2=0x7efe04068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:48.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.757+0000 7efe2359e700 1 --2- 172.21.15.40:0/4003309935 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7efe04066760 0x7efe04068c20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efe1c003eb0 tx=0x7efe14001190 comp rx=0 tx=0).stop 2024-09-13T18:04:48.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.757+0000 7efe2359e700 1 -- 172.21.15.40:0/4003309935 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe1c048d20 msgr2=0x7efe1c035120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:48.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.757+0000 7efe2359e700 1 --2- 172.21.15.40:0/4003309935 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe1c048d20 0x7efe1c035120 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efe18001050 tx=0x7efe1c007bd0 comp rx=0 tx=0).stop 2024-09-13T18:04:48.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.757+0000 7efe2359e700 1 -- 172.21.15.40:0/4003309935 shutdown_connections 2024-09-13T18:04:48.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.757+0000 7efe2359e700 1 --2- 172.21.15.40:0/4003309935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe1c0c8db0 0x7efe1c0c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:48.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.757+0000 7efe2359e700 1 --2- 172.21.15.40:0/4003309935 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7efe04066760 0x7efe04068c20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:48.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.757+0000 7efe2359e700 1 --2- 172.21.15.40:0/4003309935 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7efe1c048d20 0x7efe1c035120 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:48.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.757+0000 7efe2359e700 1 --2- 172.21.15.40:0/4003309935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe0c004870 0x7efe1c0df7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:48.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.757+0000 7efe2359e700 1 -- 172.21.15.40:0/4003309935 >> 172.21.15.40:0/4003309935 conn(0x7efe1c050590 msgr2=0x7efe1c04d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:48.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.757+0000 7efe2359e700 1 -- 172.21.15.40:0/4003309935 shutdown_connections 2024-09-13T18:04:48.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:48.757+0000 7efe2359e700 1 -- 172.21.15.40:0/4003309935 wait complete. 2024-09-13T18:04:51.785 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:04:51.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:51.889+0000 7fe5593ee700 1 Processor -- start 2024-09-13T18:04:51.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:51.889+0000 7fe5593ee700 1 -- start start 2024-09-13T18:04:51.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:51.889+0000 7fe5593ee700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 0x7fe54c00b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:51.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:51.889+0000 7fe5593ee700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 0x7fe54c04d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:51.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:51.889+0000 7fe5593ee700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe54c0d5c10 con 0x7fe54c008860 2024-09-13T18:04:51.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:51.889+0000 7fe5593ee700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe54c00ba60 con 0x7fe54c0d5830 2024-09-13T18:04:51.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:51.889+0000 7fe5593ee700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c0dc430 con 0x7fe54c04cf20 2024-09-13T18:04:51.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:51.889+0000 7fe5537fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe54c008860 0x7fe54c0ded90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48150/0 (socket says 172.21.15.40:48150) 2024-09-13T18:04:51.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:51.889+0000 7fe5537fe700 1 -- 172.21.15.40:0/2156368452 learned_addr learned my addr 172.21.15.40:0/2156368452 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:04:51.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:51.889+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 0x7fe54c00b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:51.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:51.889+0000 7fe552ffd700 1 -- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 msgr2=0x7fe54c00b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:04:51.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:51.889+0000 7fe552ffd700 1 -- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 msgr2=0x7fe54c00b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:51.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:51.889+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 0x7fe54c00b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:51.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:51.889+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 0x7fe54c04d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:51.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:51.889+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 0x7fe54c00b520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:51.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:51.889+0000 7fe5527fc700 1 -- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 msgr2=0x7fe54c04d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:04:51.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:51.889+0000 7fe5527fc700 1 -- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 msgr2=0x7fe54c04d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:51.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:51.889+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 0x7fe54c04d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:51.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:51.889+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 0x7fe54c04d320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:52.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:52.093+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 0x7fe54c00b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:52.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:52.093+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 0x7fe54c04d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:52.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:52.093+0000 7fe552ffd700 1 -- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 msgr2=0x7fe54c00b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:04:52.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:52.093+0000 7fe552ffd700 1 -- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 msgr2=0x7fe54c00b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:52.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:52.093+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 0x7fe54c00b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:52.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:52.093+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 0x7fe54c00b520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:04:52.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:52.093+0000 7fe5527fc700 1 -- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 msgr2=0x7fe54c04d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:04:52.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:52.093+0000 7fe5527fc700 1 -- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 msgr2=0x7fe54c04d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:52.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:52.093+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 0x7fe54c04d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:52.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:52.093+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 0x7fe54c04d320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:04:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:52.497+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 0x7fe54c00b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:52.497+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 0x7fe54c04d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:52.497+0000 7fe552ffd700 1 -- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 msgr2=0x7fe54c00b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:04:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:52.497+0000 7fe552ffd700 1 -- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 msgr2=0x7fe54c00b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:52.497+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 0x7fe54c00b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:52.497+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 0x7fe54c00b520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:04:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:52.497+0000 7fe5527fc700 1 -- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 msgr2=0x7fe54c04d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:04:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:52.497+0000 7fe5527fc700 1 -- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 msgr2=0x7fe54c04d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:52.497+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 0x7fe54c04d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:52.497+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 0x7fe54c04d320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:04:53.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:53.301+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 0x7fe54c00b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:53.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:53.301+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 0x7fe54c04d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:53.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:53.301+0000 7fe552ffd700 1 -- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 msgr2=0x7fe54c00b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:04:53.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:53.301+0000 7fe552ffd700 1 -- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 msgr2=0x7fe54c00b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:53.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:53.301+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 0x7fe54c00b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:53.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:53.301+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 0x7fe54c00b520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:04:53.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:53.301+0000 7fe5527fc700 1 -- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 msgr2=0x7fe54c04d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:04:53.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:53.301+0000 7fe5527fc700 1 -- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 msgr2=0x7fe54c04d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:53.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:53.301+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 0x7fe54c04d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:53.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:53.301+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 0x7fe54c04d320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:04:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:54.889+0000 7fe550ff9700 1 -- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 msgr2=0x7fe54c04d320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:04:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:54.889+0000 7fe550ff9700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe54c04cf20 0x7fe54c04d320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:54.889+0000 7fe550ff9700 1 -- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 msgr2=0x7fe54c00b520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:04:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:54.889+0000 7fe550ff9700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.40:3300/0 conn(0x7fe54c0d5830 0x7fe54c00b520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:54.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:54.889+0000 7fe550ff9700 1 -- 172.21.15.40:0/2156368452 >> v1:172.21.15.40:6789/0 conn(0x7fe54c008860 legacy=0x7fe54c0ded90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:54.889+0000 7fe550ff9700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.43:3300/0 conn(0x7fe540003720 0x7fe540005ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:54.889+0000 7fe550ff9700 1 -- 172.21.15.40:0/2156368452 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe54c0db5d0 con 0x7fe540006280 2024-09-13T18:04:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:54.889+0000 7fe550ff9700 1 -- 172.21.15.40:0/2156368452 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe54c00ba60 con 0x7fe540003720 2024-09-13T18:04:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:54.889+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.43:3300/0 conn(0x7fe540003720 0x7fe540005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:54.889+0000 7fe552ffd700 1 -- 172.21.15.40:0/2156368452 >> v2:172.21.15.43:3300/0 conn(0x7fe540003720 msgr2=0x7fe540005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:04:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:54.889+0000 7fe552ffd700 1 -- 172.21.15.40:0/2156368452 >> v2:172.21.15.43:3300/0 conn(0x7fe540003720 msgr2=0x7fe540005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:54.889+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.43:3300/0 conn(0x7fe540003720 0x7fe540005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:54.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:54.889+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.43:3300/0 conn(0x7fe540003720 0x7fe540005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:55.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:55.093+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.43:3300/0 conn(0x7fe540003720 0x7fe540005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:55.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:55.093+0000 7fe552ffd700 1 -- 172.21.15.40:0/2156368452 >> v2:172.21.15.43:3300/0 conn(0x7fe540003720 msgr2=0x7fe540005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:04:55.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:55.093+0000 7fe552ffd700 1 -- 172.21.15.40:0/2156368452 >> v2:172.21.15.43:3300/0 conn(0x7fe540003720 msgr2=0x7fe540005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:55.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:55.093+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.43:3300/0 conn(0x7fe540003720 0x7fe540005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:55.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:55.093+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.43:3300/0 conn(0x7fe540003720 0x7fe540005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:04:55.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:55.497+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.43:3300/0 conn(0x7fe540003720 0x7fe540005ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:55.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:55.497+0000 7fe552ffd700 1 -- 172.21.15.40:0/2156368452 >> v2:172.21.15.43:3300/0 conn(0x7fe540003720 msgr2=0x7fe540005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:04:55.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:55.497+0000 7fe552ffd700 1 -- 172.21.15.40:0/2156368452 >> v2:172.21.15.43:3300/0 conn(0x7fe540003720 msgr2=0x7fe540005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:04:55.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:55.497+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.43:3300/0 conn(0x7fe540003720 0x7fe540005ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:55.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:55.497+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.43:3300/0 conn(0x7fe540003720 0x7fe540005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:04:56.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.285+0000 7fe551ffb700 1 -- 172.21.15.40:0/2156368452 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 836466406 0 0) 0x7fe54c0db5d0 con 0x7fe540006280 2024-09-13T18:04:56.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.285+0000 7fe551ffb700 1 -- 172.21.15.40:0/2156368452 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe53c0035d0 con 0x7fe540006280 2024-09-13T18:04:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.293+0000 7fe551ffb700 1 -- 172.21.15.40:0/2156368452 <== mon.1 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4030816371 0 0) 0x7fe53c0035d0 con 0x7fe540006280 2024-09-13T18:04:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.293+0000 7fe551ffb700 1 -- 172.21.15.40:0/2156368452 >> v2:172.21.15.43:3300/0 conn(0x7fe540003720 msgr2=0x7fe540005ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:04:56.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.293+0000 7fe551ffb700 1 --2- 172.21.15.40:0/2156368452 >> v2:172.21.15.43:3300/0 conn(0x7fe540003720 0x7fe540005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:56.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.293+0000 7fe551ffb700 1 -- 172.21.15.40:0/2156368452 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54000b020 con 0x7fe540006280 2024-09-13T18:04:56.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.293+0000 7fe551ffb700 1 -- 172.21.15.40:0/2156368452 <== mon.1 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fe54800c4f0 con 0x7fe540006280 2024-09-13T18:04:56.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.293+0000 7fe551ffb700 1 -- 172.21.15.40:0/2156368452 >> v1:172.21.15.43:6789/0 conn(0x7fe540006280 legacy=0x7fe540008740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:56.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.293+0000 7fe5537fe700 1 -- 172.21.15.40:0/2156368452 reap_dead start 2024-09-13T18:04:56.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.293+0000 7fe551ffb700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c003a20 0x7fe53c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:56.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.293+0000 7fe551ffb700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c004870 0x7fe53c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:56.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.293+0000 7fe551ffb700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 0x7fe53c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:56.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.293+0000 7fe551ffb700 1 -- 172.21.15.40:0/2156368452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c00ba60 con 0x7fe53c005300 2024-09-13T18:04:56.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.293+0000 7fe551ffb700 1 -- 172.21.15.40:0/2156368452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c00c250 con 0x7fe53c004870 2024-09-13T18:04:56.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.293+0000 7fe551ffb700 1 -- 172.21.15.40:0/2156368452 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53c00c410 con 0x7fe53c003a20 2024-09-13T18:04:56.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.293+0000 7fe5537fe700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 0x7fe53c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:56.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.293+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c003a20 0x7fe53c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:56.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.293+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c004870 0x7fe53c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:56.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.297+0000 7fe5537fe700 1 -- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c003a20 msgr2=0x7fe53c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:04:56.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.297+0000 7fe5537fe700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c003a20 0x7fe53c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:56.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.297+0000 7fe5537fe700 1 -- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c004870 msgr2=0x7fe53c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:04:56.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.297+0000 7fe5537fe700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c004870 0x7fe53c004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:56.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.297+0000 7fe5537fe700 1 -- 172.21.15.40:0/2156368452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53c0035d0 con 0x7fe53c005300 2024-09-13T18:04:56.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.297+0000 7fe5537fe700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 0x7fe53c00ba30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe55402ce60 tx=0x7fe53c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76195c179d7666ad server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:56.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.297+0000 7fe551ffb700 1 -- 172.21.15.40:0/2156368452 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55408f070 con 0x7fe53c005300 2024-09-13T18:04:56.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.297+0000 7fe551ffb700 1 -- 172.21.15.40:0/2156368452 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55404ece0 con 0x7fe53c005300 2024-09-13T18:04:56.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.297+0000 7fe551ffb700 1 -- 172.21.15.40:0/2156368452 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554067ff0 con 0x7fe53c005300 2024-09-13T18:04:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.297+0000 7fe5593ee700 1 -- 172.21.15.40:0/2156368452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 msgr2=0x7fe53c00ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.297+0000 7fe5593ee700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 0x7fe53c00ba30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe55402ce60 tx=0x7fe53c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:04:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.297+0000 7fe5593ee700 1 -- 172.21.15.40:0/2156368452 shutdown_connections 2024-09-13T18:04:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.297+0000 7fe5593ee700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 0x7fe53c00ba30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.297+0000 7fe5593ee700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c004870 0x7fe53c004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.297+0000 7fe5593ee700 1 --2- 172.21.15.40:0/2156368452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c003a20 0x7fe53c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.297+0000 7fe5593ee700 1 -- 172.21.15.40:0/2156368452 >> 172.21.15.40:0/2156368452 conn(0x7fe54c00a780 msgr2=0x7fe54c00ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.297+0000 7fe5593ee700 1 -- 172.21.15.40:0/2156368452 shutdown_connections 2024-09-13T18:04:56.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.297+0000 7fe5593ee700 1 -- 172.21.15.40:0/2156368452 wait complete. 2024-09-13T18:04:56.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe5593ee700 1 Processor -- start 2024-09-13T18:04:56.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe5593ee700 1 -- start start 2024-09-13T18:04:56.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe5593ee700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 0x7fe54c0503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:56.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe5593ee700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c004870 0x7fe54c0dd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:56.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe5593ee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 0x7fe54c0dd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:56.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe5593ee700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c0d9d90 con 0x7fe53c005300 2024-09-13T18:04:56.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe5593ee700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c0d6720 con 0x7fe53c003a20 2024-09-13T18:04:56.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe5593ee700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c0d9f80 con 0x7fe53c004870 2024-09-13T18:04:56.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe5537fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 0x7fe54c0dd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:56.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe552ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 0x7fe54c0503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:56.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe552ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 0x7fe54c0503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:42386/0 (socket says 172.21.15.40:42386) 2024-09-13T18:04:56.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe552ffd700 1 -- 172.21.15.40:0/2982021179 learned_addr learned my addr 172.21.15.40:0/2982021179 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:56.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c004870 0x7fe54c0dd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:56.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe5537fe700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 msgr2=0x7fe54c0dd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-13T18:04:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe5537fe700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 msgr2=0x7fe54c0dd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:04:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe552ffd700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 msgr2=0x7fe54c0503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:04:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe5537fe700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 0x7fe54c0dd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe552ffd700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 msgr2=0x7fe54c0503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:04:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 0x7fe54c0503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe5537fe700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 0x7fe54c0dd770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 0x7fe54c0503d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe5527fc700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c004870 msgr2=0x7fe54c0dd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:04:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe5527fc700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c004870 msgr2=0x7fe54c0dd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:04:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c004870 0x7fe54c0dd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:56.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.301+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c004870 0x7fe54c0dd230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:04:56.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.505+0000 7fe5537fe700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 0x7fe54c0dd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:56.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.505+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 0x7fe54c0503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:56.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.505+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c004870 0x7fe54c0dd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:56.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.505+0000 7fe5537fe700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 msgr2=0x7fe54c0dd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:04:56.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.505+0000 7fe5537fe700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 msgr2=0x7fe54c0dd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:04:56.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.505+0000 7fe5537fe700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 0x7fe54c0dd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:56.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.505+0000 7fe552ffd700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 msgr2=0x7fe54c0503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:04:56.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.505+0000 7fe552ffd700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 msgr2=0x7fe54c0503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:04:56.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.505+0000 7fe5527fc700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c004870 msgr2=0x7fe54c0dd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-13T18:04:56.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.505+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 0x7fe54c0503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:56.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.505+0000 7fe5527fc700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c004870 msgr2=0x7fe54c0dd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:04:56.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.505+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c004870 0x7fe54c0dd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:56.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.505+0000 7fe5537fe700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 0x7fe54c0dd770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:04:56.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.505+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 0x7fe54c0503d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:04:56.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.505+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c004870 0x7fe54c0dd230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:04:56.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.909+0000 7fe5537fe700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 0x7fe54c0dd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:56.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.909+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c004870 0x7fe54c0dd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:56.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.909+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 0x7fe54c0503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:56.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.909+0000 7fe5537fe700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 msgr2=0x7fe54c0dd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-13T18:04:56.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.909+0000 7fe5537fe700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 msgr2=0x7fe54c0dd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:04:56.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.909+0000 7fe5537fe700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 0x7fe54c0dd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:56.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.909+0000 7fe552ffd700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 msgr2=0x7fe54c0503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:04:56.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.909+0000 7fe552ffd700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 msgr2=0x7fe54c0503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:04:56.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.909+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 0x7fe54c0503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:56.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.909+0000 7fe5537fe700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 0x7fe54c0dd770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:04:56.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.909+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 0x7fe54c0503d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:04:56.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.909+0000 7fe5527fc700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c004870 msgr2=0x7fe54c0dd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:04:56.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.909+0000 7fe5527fc700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c004870 msgr2=0x7fe54c0dd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:04:56.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.909+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c004870 0x7fe54c0dd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:04:56.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:56.909+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c004870 0x7fe54c0dd230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:04:57.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.713+0000 7fe5537fe700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 0x7fe54c0dd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:57.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.713+0000 7fe5527fc700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c004870 0x7fe54c0dd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:57.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.713+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 0x7fe54c0503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:57.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.713+0000 7fe5537fe700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c004870 msgr2=0x7fe54c0dd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.713+0000 7fe5537fe700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c004870 0x7fe54c0dd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.713+0000 7fe5537fe700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 msgr2=0x7fe54c0503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.713+0000 7fe5537fe700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 0x7fe54c0503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.713+0000 7fe5537fe700 1 -- 172.21.15.40:0/2982021179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c0da220 con 0x7fe53c005300 2024-09-13T18:04:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.713+0000 7fe5537fe700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 0x7fe54c0dd770 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe554073070 tx=0x7fe554069e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.713+0000 7fe53b7fe700 1 -- 172.21.15.40:0/2982021179 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55409c070 con 0x7fe53c005300 2024-09-13T18:04:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.713+0000 7fe53b7fe700 1 -- 172.21.15.40:0/2982021179 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe554067730 con 0x7fe53c005300 2024-09-13T18:04:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.713+0000 7fe5593ee700 1 -- 172.21.15.40:0/2982021179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe54c0dae60 con 0x7fe53c005300 2024-09-13T18:04:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.713+0000 7fe53b7fe700 1 -- 172.21.15.40:0/2982021179 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55408f040 con 0x7fe53c005300 2024-09-13T18:04:57.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.713+0000 7fe5593ee700 1 -- 172.21.15.40:0/2982021179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54c0da7a0 con 0x7fe53c005300 2024-09-13T18:04:57.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.717+0000 7fe5593ee700 1 -- 172.21.15.40:0/2982021179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe54c0dc130 con 0x7fe53c005300 2024-09-13T18:04:57.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.717+0000 7fe53b7fe700 1 -- 172.21.15.40:0/2982021179 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe5540678e0 con 0x7fe53c005300 2024-09-13T18:04:57.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.717+0000 7fe53b7fe700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe540067030 0x7fe5400694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:57.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.717+0000 7fe53b7fe700 1 -- 172.21.15.40:0/2982021179 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe5540ff660 con 0x7fe53c005300 2024-09-13T18:04:57.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.717+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe540067030 0x7fe5400694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:57.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.717+0000 7fe552ffd700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe540067030 0x7fe5400694f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe544006df0 tx=0x7fe54401a040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:57.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:57.725+0000 7fe53b7fe700 1 -- 172.21.15.40:0/2982021179 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe54c0dc130 con 0x7fe53c005300 2024-09-13T18:04:58.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.009+0000 7fe5593ee700 1 -- 172.21.15.40:0/2982021179 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe54c0dc130 con 0x7fe53c005300 2024-09-13T18:04:58.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.009+0000 7fe53b7fe700 1 -- 172.21.15.40:0/2982021179 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fe54c0dc130 con 0x7fe53c005300 2024-09-13T18:04:58.015 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":118,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:04:58.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.013+0000 7fe5593ee700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe540067030 msgr2=0x7fe5400694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.013+0000 7fe5593ee700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe540067030 0x7fe5400694f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe544006df0 tx=0x7fe54401a040 comp rx=0 tx=0).stop 2024-09-13T18:04:58.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.013+0000 7fe5593ee700 1 -- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 msgr2=0x7fe54c0dd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.013+0000 7fe5593ee700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 0x7fe54c0dd770 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe554073070 tx=0x7fe554069e70 comp rx=0 tx=0).stop 2024-09-13T18:04:58.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.013+0000 7fe5593ee700 1 -- 172.21.15.40:0/2982021179 shutdown_connections 2024-09-13T18:04:58.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.013+0000 7fe5593ee700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe53c005300 0x7fe54c0dd770 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.013+0000 7fe5593ee700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe540067030 0x7fe5400694f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.013+0000 7fe5593ee700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe53c004870 0x7fe54c0dd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.013+0000 7fe5593ee700 1 --2- 172.21.15.40:0/2982021179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe53c003a20 0x7fe54c0503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.013+0000 7fe5593ee700 1 -- 172.21.15.40:0/2982021179 >> 172.21.15.40:0/2982021179 conn(0x7fe54c00a780 msgr2=0x7fe54c047130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:58.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.013+0000 7fe5593ee700 1 -- 172.21.15.40:0/2982021179 shutdown_connections 2024-09-13T18:04:58.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.013+0000 7fe5593ee700 1 -- 172.21.15.40:0/2982021179 wait complete. 2024-09-13T18:04:58.033 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:04:58.034 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:04:58.034 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:04:58.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.145+0000 7f8f70a11700 1 Processor -- start 2024-09-13T18:04:58.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.145+0000 7f8f70a11700 1 -- start start 2024-09-13T18:04:58.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.145+0000 7f8f70a11700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f640d87a0 0x7f8f640df110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.145+0000 7f8f70a11700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8f640469c0 0x7f8f64046e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.145+0000 7f8f70a11700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f64047610 con 0x7f8f64007b60 2024-09-13T18:04:58.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.145+0000 7f8f70a11700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f6404bef0 con 0x7f8f640469c0 2024-09-13T18:04:58.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.145+0000 7f8f70a11700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f64006b50 con 0x7f8f640d87a0 2024-09-13T18:04:58.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.145+0000 7f8f6affd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8f64007b60 0x7f8f6404b7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48156/0 (socket says 172.21.15.40:48156) 2024-09-13T18:04:58.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.145+0000 7f8f6affd700 1 -- 172.21.15.40:0/4169567826 learned_addr learned my addr 172.21.15.40:0/4169567826 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:04:58.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.145+0000 7f8f6bfff700 1 --2- 172.21.15.40:0/4169567826 >> v2:172.21.15.43:3300/0 conn(0x7f8f640469c0 0x7f8f64046e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.145+0000 7f8f6b7fe700 1 --2- 172.21.15.40:0/4169567826 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f640d87a0 0x7f8f640df110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.149+0000 7f8f6a7fc700 1 -- 172.21.15.40:0/4169567826 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048533684 0 0) 0x7f8f64047610 con 0x7f8f64007b60 2024-09-13T18:04:58.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.149+0000 7f8f6a7fc700 1 -- 172.21.15.40:0/4169567826 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f5c0035d0 con 0x7f8f64007b60 2024-09-13T18:04:58.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.149+0000 7f8f6bfff700 1 -- 172.21.15.40:0/4169567826 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f640d87a0 msgr2=0x7f8f640df110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:04:58.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.149+0000 7f8f6bfff700 1 --2- 172.21.15.40:0/4169567826 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f640d87a0 0x7f8f640df110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.149+0000 7f8f6bfff700 1 -- 172.21.15.40:0/4169567826 >> v1:172.21.15.40:6789/0 conn(0x7f8f64007b60 legacy=0x7f8f6404b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.149+0000 7f8f6bfff700 1 -- 172.21.15.40:0/4169567826 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f640076d0 con 0x7f8f640469c0 2024-09-13T18:04:58.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.149+0000 7f8f6bfff700 1 --2- 172.21.15.40:0/4169567826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f640469c0 0x7f8f64046e10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c069620 tx=0x7f8f6c069e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6a2be561274d5fb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:58.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.149+0000 7f8f6a7fc700 1 -- 172.21.15.40:0/4169567826 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c067dd0 con 0x7f8f640469c0 2024-09-13T18:04:58.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.149+0000 7f8f6a7fc700 1 -- 172.21.15.40:0/4169567826 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c067f30 con 0x7f8f640469c0 2024-09-13T18:04:58.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.149+0000 7f8f6a7fc700 1 -- 172.21.15.40:0/4169567826 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0746e0 con 0x7f8f640469c0 2024-09-13T18:04:58.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.149+0000 7f8f70a11700 1 -- 172.21.15.40:0/4169567826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f640469c0 msgr2=0x7f8f64046e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.149+0000 7f8f70a11700 1 --2- 172.21.15.40:0/4169567826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f640469c0 0x7f8f64046e10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c069620 tx=0x7f8f6c069e70 comp rx=0 tx=0).stop 2024-09-13T18:04:58.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.149+0000 7f8f70a11700 1 -- 172.21.15.40:0/4169567826 shutdown_connections 2024-09-13T18:04:58.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.149+0000 7f8f70a11700 1 --2- 172.21.15.40:0/4169567826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f640469c0 0x7f8f64046e10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.149+0000 7f8f70a11700 1 --2- 172.21.15.40:0/4169567826 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f640d87a0 0x7f8f640df110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.149+0000 7f8f70a11700 1 -- 172.21.15.40:0/4169567826 >> 172.21.15.40:0/4169567826 conn(0x7f8f64050e60 msgr2=0x7f8f64051270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:58.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.149+0000 7f8f70a11700 1 -- 172.21.15.40:0/4169567826 shutdown_connections 2024-09-13T18:04:58.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.149+0000 7f8f70a11700 1 -- 172.21.15.40:0/4169567826 wait complete. 2024-09-13T18:04:58.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.149+0000 7f8f70a11700 1 Processor -- start 2024-09-13T18:04:58.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.149+0000 7f8f70a11700 1 -- start start 2024-09-13T18:04:58.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f70a11700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f640469c0 0x7f8f64174320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f70a11700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f640d87a0 0x7f8f640db5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f70a11700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f640dbb30 0x7f8f64049a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f70a11700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f640495b0 con 0x7f8f640d87a0 2024-09-13T18:04:58.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f70a11700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f64048370 con 0x7f8f640469c0 2024-09-13T18:04:58.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f70a11700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f640dbfe0 con 0x7f8f640dbb30 2024-09-13T18:04:58.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f6affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f640d87a0 0x7f8f640db5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f6affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f640d87a0 0x7f8f640db5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34932/0 (socket says 172.21.15.40:34932) 2024-09-13T18:04:58.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f6affd700 1 -- 172.21.15.40:0/621775956 learned_addr learned my addr 172.21.15.40:0/621775956 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:58.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f6b7fe700 1 --2- 172.21.15.40:0/621775956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f640469c0 0x7f8f64174320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f6bfff700 1 --2- 172.21.15.40:0/621775956 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f640dbb30 0x7f8f64049a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f6affd700 1 -- 172.21.15.40:0/621775956 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f640dbb30 msgr2=0x7f8f64049a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f6affd700 1 --2- 172.21.15.40:0/621775956 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8f640dbb30 0x7f8f64049a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f6affd700 1 -- 172.21.15.40:0/621775956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f640469c0 msgr2=0x7f8f64174320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f6affd700 1 --2- 172.21.15.40:0/621775956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f640469c0 0x7f8f64174320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f6affd700 1 -- 172.21.15.40:0/621775956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c04f5c0 con 0x7f8f640d87a0 2024-09-13T18:04:58.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f6affd700 1 --2- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f640d87a0 0x7f8f640db5f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8f580010e0 tx=0x7f8f5800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:58.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f57fff700 1 -- 172.21.15.40:0/621775956 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f58005d90 con 0x7f8f640d87a0 2024-09-13T18:04:58.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f57fff700 1 -- 172.21.15.40:0/621775956 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5800ce70 con 0x7f8f640d87a0 2024-09-13T18:04:58.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f70a11700 1 -- 172.21.15.40:0/621775956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f640485a0 con 0x7f8f640d87a0 2024-09-13T18:04:58.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f57fff700 1 -- 172.21.15.40:0/621775956 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f580072d0 con 0x7f8f640d87a0 2024-09-13T18:04:58.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.153+0000 7f8f70a11700 1 -- 172.21.15.40:0/621775956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f64048b60 con 0x7f8f640d87a0 2024-09-13T18:04:58.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.157+0000 7f8f70a11700 1 --2- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f64007b60 0x7f8f64049f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.157+0000 7f8f70a11700 1 -- 172.21.15.40:0/621775956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f640095f0 con 0x7f8f64007b60 2024-09-13T18:04:58.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.157+0000 7f8f6b7fe700 1 --2- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f64007b60 0x7f8f64049f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.157+0000 7f8f6b7fe700 1 -- 172.21.15.40:0/621775956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f640df650 con 0x7f8f64007b60 2024-09-13T18:04:58.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.157+0000 7f8f6b7fe700 1 --2- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f64007b60 0x7f8f64049f40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8f600119d0 tx=0x7f8f60011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:58.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.157+0000 7f8f57fff700 1 -- 172.21.15.40:0/621775956 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f60004240 con 0x7f8f64007b60 2024-09-13T18:04:58.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.157+0000 7f8f57fff700 1 -- 172.21.15.40:0/621775956 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8f58007430 con 0x7f8f640d87a0 2024-09-13T18:04:58.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.157+0000 7f8f57fff700 1 --2- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8f50066800 0x7f8f50068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.157+0000 7f8f57fff700 1 -- 172.21.15.40:0/621775956 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8f58083980 con 0x7f8f640d87a0 2024-09-13T18:04:58.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.157+0000 7f8f6bfff700 1 --2- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8f50066800 0x7f8f50068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.157+0000 7f8f57fff700 1 -- 172.21.15.40:0/621775956 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8f600043a0 con 0x7f8f64007b60 2024-09-13T18:04:58.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.157+0000 7f8f57fff700 1 -- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f64007b60 msgr2=0x7f8f64049f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.157+0000 7f8f57fff700 1 --2- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f64007b60 0x7f8f64049f40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8f600119d0 tx=0x7f8f60011d90 comp rx=0 tx=0).stop 2024-09-13T18:04:58.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.157+0000 7f8f57fff700 1 -- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f64007b60 msgr2=0x7f8f64049f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:04:58.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.157+0000 7f8f57fff700 1 --2- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f64007b60 0x7f8f64049f40 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8f600119d0 tx=0x7f8f60011d90 comp rx=0 tx=0).stop 2024-09-13T18:04:58.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.157+0000 7f8f6bfff700 1 --2- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8f50066800 0x7f8f50068cc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8f6400a8d0 tx=0x7f8f6c096040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:58.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.169+0000 7f8f70a11700 1 --2- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f64046500 0x7f8f64048d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.169+0000 7f8f70a11700 1 -- 172.21.15.40:0/621775956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f640492c0 con 0x7f8f64046500 2024-09-13T18:04:58.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.169+0000 7f8f6b7fe700 1 --2- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f64046500 0x7f8f64048d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.169+0000 7f8f6b7fe700 1 -- 172.21.15.40:0/621775956 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8f640d8b80 con 0x7f8f64046500 2024-09-13T18:04:58.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.169+0000 7f8f6b7fe700 1 --2- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f64046500 0x7f8f64048d80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8f6001b000 tx=0x7f8f60004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:58.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.169+0000 7f8f57fff700 1 -- 172.21.15.40:0/621775956 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6000fdc0 con 0x7f8f64046500 2024-09-13T18:04:58.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.169+0000 7f8f57fff700 1 -- 172.21.15.40:0/621775956 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4452 (secure 0 0 0) 0x7f8f600283a0 con 0x7f8f64046500 2024-09-13T18:04:58.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.169+0000 7f8f57fff700 1 -- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f64046500 msgr2=0x7f8f64048d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.169+0000 7f8f57fff700 1 --2- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f64046500 0x7f8f64048d80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8f6001b000 tx=0x7f8f60004770 comp rx=0 tx=0).stop 2024-09-13T18:04:58.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.169+0000 7f8f57fff700 1 -- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f64046500 msgr2=0x7f8f64048d80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:04:58.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.169+0000 7f8f57fff700 1 --2- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f64046500 0x7f8f64048d80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.179 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:04:58.179 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:04:58.179 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:04:58.179 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:04:58.179 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 118, 2024-09-13T18:04:58.179 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:04:58.180 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:04:58.180 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:04:58.180 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:04:58.180 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:58.180 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:04:58.180 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:04:58.180 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:04:58.180 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:04:58.180 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:58.180 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:58.180 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:58.180 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:58.180 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:58.181 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:58.181 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:58.181 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:58.181 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:58.181 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:58.181 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:58.181 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:04:58.181 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:04:58.181 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:58.181 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:58.181 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:58.181 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:58.181 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:58.182 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:58.182 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:58.182 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:58.182 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:58.182 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:58.182 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:58.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.182 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:04:58.182 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:04:58.182 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:04:58.182 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:04:58.182 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:04:58.182 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:04:58.183 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:04:58.183 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:04:58.183 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:04:58.183 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:04:58.183 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:04:58.183 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:04:58.183 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:04:58.183 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:04:58.183 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:04:58.183 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:04:58.183 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:04:58.183 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:58.183 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:58.183 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:58.184 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:58.184 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:58.184 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:58.184 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:58.184 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:58.184 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:58.184 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:58.184 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:58.184 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:04:58.184 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.184 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:04:58.184 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.185 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:04:58.185 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:04:58.185 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:58.188 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:58.188 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.188 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:58.188 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:04:58.188 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:58.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.189 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:58.189 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:04:58.189 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:58.189 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.189 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:58.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.189 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:04:58.189 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:04:58.189 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:58.189 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:58.189 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:58.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.190 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:04:58.190 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:04:58.190 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:58.190 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:58.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.190 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:58.190 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:04:58.190 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:58.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.190 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:58.190 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:04:58.191 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:58.191 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.191 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:58.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.191 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:04:58.191 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:04:58.191 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:58.191 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:58.191 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:58.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.191 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:04:58.191 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:04:58.191 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:58.192 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:58.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.192 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:58.192 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:04:58.192 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:58.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.192 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:58.192 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:04:58.192 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:58.192 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.192 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:58.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.192 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:04:58.193 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:04:58.193 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:58.193 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:58.193 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:58.193 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.193 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:58.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.193 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:04:58.193 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:04:58.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.193 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:58.193 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:58.193 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:04:58.193 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.194 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:58.194 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:04:58.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.194 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:58.194 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:58.194 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:04:58.194 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.194 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:58.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.194 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:04:58.194 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:04:58.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.169+0000 7f8f70a11700 1 -- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8f50066800 msgr2=0x7f8f50068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.169+0000 7f8f70a11700 1 --2- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8f50066800 0x7f8f50068cc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8f6400a8d0 tx=0x7f8f6c096040 comp rx=0 tx=0).stop 2024-09-13T18:04:58.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.169+0000 7f8f70a11700 1 -- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f640d87a0 msgr2=0x7f8f640db5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.169+0000 7f8f70a11700 1 --2- 172.21.15.40:0/621775956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f640d87a0 0x7f8f640db5f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8f580010e0 tx=0x7f8f5800c7d0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.169+0000 7f8f6bfff700 1 -- 172.21.15.40:0/621775956 reap_dead start 2024-09-13T18:04:58.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.169+0000 7f8f6bfff700 1 -- 172.21.15.40:0/621775956 reap_dead start 2024-09-13T18:04:58.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.169+0000 7f8f70a11700 1 -- 172.21.15.40:0/621775956 shutdown_connections 2024-09-13T18:04:58.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.169+0000 7f8f70a11700 1 -- 172.21.15.40:0/621775956 >> 172.21.15.40:0/621775956 conn(0x7f8f64050e60 msgr2=0x7f8f640d6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:58.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.173+0000 7f8f70a11700 1 -- 172.21.15.40:0/621775956 shutdown_connections 2024-09-13T18:04:58.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.173+0000 7f8f70a11700 1 -- 172.21.15.40:0/621775956 wait complete. 2024-09-13T18:04:58.196 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:04:58.197 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:04:58.197 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:04:58.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.301+0000 7f0ccf59e700 1 Processor -- start 2024-09-13T18:04:58.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.301+0000 7f0ccf59e700 1 -- start start 2024-09-13T18:04:58.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccf59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0cc8045240 0x7f0cc80dba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccf59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc80d5b90 con 0x7f0cc80dbfd0 2024-09-13T18:04:58.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccf59e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc8045670 con 0x7f0cc80d57b0 2024-09-13T18:04:58.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccf59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc8050f20 con 0x7f0cc8045240 2024-09-13T18:04:58.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccdd9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0cc8045240 0x7f0cc80dba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccdd9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0cc8045240 0x7f0cc80dba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34956/0 (socket says 172.21.15.40:34956) 2024-09-13T18:04:58.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccdd9b700 1 -- 172.21.15.40:0/1637863538 learned_addr learned my addr 172.21.15.40:0/1637863538 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:58.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0cced9d700 1 --1- 172.21.15.40:0/1637863538 >> v1:172.21.15.40:6789/0 conn(0x7f0cc80dbfd0 0x7f0cc80463f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48172/0 (socket says 172.21.15.40:48172) 2024-09-13T18:04:58.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccdd9b700 1 -- 172.21.15.40:0/1637863538 >> v1:172.21.15.43:6789/0 conn(0x7f0cc80d57b0 legacy=0x7f0cc8005f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:04:58.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccdd9b700 1 -- 172.21.15.40:0/1637863538 >> v1:172.21.15.40:6789/0 conn(0x7f0cc80dbfd0 legacy=0x7f0cc80463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccdd9b700 1 -- 172.21.15.40:0/1637863538 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc8032e80 con 0x7f0cc8045240 2024-09-13T18:04:58.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccdd9b700 1 --2- 172.21.15.40:0/1637863538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cc8045240 0x7f0cc80dba90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009910 tx=0x7f0cc4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96855da1c1abe19 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:58.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccd59a700 1 -- 172.21.15.40:0/1637863538 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4004230 con 0x7f0cc8045240 2024-09-13T18:04:58.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccd59a700 1 -- 172.21.15.40:0/1637863538 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc4004390 con 0x7f0cc8045240 2024-09-13T18:04:58.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccd59a700 1 -- 172.21.15.40:0/1637863538 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4009e60 con 0x7f0cc8045240 2024-09-13T18:04:58.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccf59e700 1 -- 172.21.15.40:0/1637863538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cc8045240 msgr2=0x7f0cc80dba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccf59e700 1 --2- 172.21.15.40:0/1637863538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cc8045240 0x7f0cc80dba90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009910 tx=0x7f0cc4009c20 comp rx=0 tx=0).stop 2024-09-13T18:04:58.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccf59e700 1 -- 172.21.15.40:0/1637863538 shutdown_connections 2024-09-13T18:04:58.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccf59e700 1 --2- 172.21.15.40:0/1637863538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cc8045240 0x7f0cc80dba90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccf59e700 1 -- 172.21.15.40:0/1637863538 >> 172.21.15.40:0/1637863538 conn(0x7f0cc8050150 msgr2=0x7f0cc8050560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:58.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccf59e700 1 -- 172.21.15.40:0/1637863538 shutdown_connections 2024-09-13T18:04:58.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.305+0000 7f0ccf59e700 1 -- 172.21.15.40:0/1637863538 wait complete. 2024-09-13T18:04:58.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0ccf59e700 1 Processor -- start 2024-09-13T18:04:58.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0ccf59e700 1 -- start start 2024-09-13T18:04:58.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0ccf59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cc8045240 0x7f0cc80d1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0ccf59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cc80d57b0 0x7f0cc80c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0ccf59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cc80dbfd0 0x7f0cc80d0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0ccf59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc80d27c0 con 0x7f0cc80d57b0 2024-09-13T18:04:58.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0ccf59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc80db2e0 con 0x7f0cc80dbfd0 2024-09-13T18:04:58.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0ccf59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc80c8110 con 0x7f0cc8045240 2024-09-13T18:04:58.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0ccdd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cc80d57b0 0x7f0cc80c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0ccdd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cc80d57b0 0x7f0cc80c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34958/0 (socket says 172.21.15.40:34958) 2024-09-13T18:04:58.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0ccdd9b700 1 -- 172.21.15.40:0/2260762973 learned_addr learned my addr 172.21.15.40:0/2260762973 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0cced9d700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cc80dbfd0 0x7f0cc80d0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0cce59c700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cc8045240 0x7f0cc80d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0ccdd9b700 1 -- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cc8045240 msgr2=0x7f0cc80d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0ccdd9b700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cc8045240 0x7f0cc80d1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0ccdd9b700 1 -- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cc80dbfd0 msgr2=0x7f0cc80d0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0ccdd9b700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cc80dbfd0 0x7f0cc80d0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0ccdd9b700 1 -- 172.21.15.40:0/2260762973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc4009570 con 0x7f0cc80d57b0 2024-09-13T18:04:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0cce59c700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cc8045240 0x7f0cc80d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:04:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0cced9d700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cc80dbfd0 0x7f0cc80d0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:04:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0ccdd9b700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cc80d57b0 0x7f0cc80c8b40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009c60 tx=0x7f0cc4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0cbeffd700 1 -- 172.21.15.40:0/2260762973 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc400bdb0 con 0x7f0cc80d57b0 2024-09-13T18:04:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0ccf59e700 1 -- 172.21.15.40:0/2260762973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc80d2a40 con 0x7f0cc80d57b0 2024-09-13T18:04:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0cbeffd700 1 -- 172.21.15.40:0/2260762973 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc401f070 con 0x7f0cc80d57b0 2024-09-13T18:04:58.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0cbeffd700 1 -- 172.21.15.40:0/2260762973 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4003900 con 0x7f0cc80d57b0 2024-09-13T18:04:58.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.309+0000 7f0ccf59e700 1 -- 172.21.15.40:0/2260762973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc80db000 con 0x7f0cc80d57b0 2024-09-13T18:04:58.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.313+0000 7f0ccf59e700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cc80c7960 0x7f0cc80d9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.313+0000 7f0ccf59e700 1 -- 172.21.15.40:0/2260762973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc8008900 con 0x7f0cc80c7960 2024-09-13T18:04:58.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.313+0000 7f0cce59c700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cc80c7960 0x7f0cc80d9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.313+0000 7f0cce59c700 1 -- 172.21.15.40:0/2260762973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0cc8033720 con 0x7f0cc80c7960 2024-09-13T18:04:58.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.313+0000 7f0cbeffd700 1 -- 172.21.15.40:0/2260762973 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0cc4021030 con 0x7f0cc80d57b0 2024-09-13T18:04:58.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.313+0000 7f0cce59c700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cc80c7960 0x7f0cc80d9030 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000e730 tx=0x7f0cc000eaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:58.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.313+0000 7f0cbeffd700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0cb0066730 0x7f0cb0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.313+0000 7f0cbeffd700 1 -- 172.21.15.40:0/2260762973 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0cc4085cb0 con 0x7f0cc80d57b0 2024-09-13T18:04:58.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.313+0000 7f0cced9d700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0cb0066730 0x7f0cb0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.313+0000 7f0cbeffd700 1 -- 172.21.15.40:0/2260762973 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0005020 con 0x7f0cc80c7960 2024-09-13T18:04:58.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.313+0000 7f0cbeffd700 1 -- 172.21.15.40:0/2260762973 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0cc0005180 con 0x7f0cc80c7960 2024-09-13T18:04:58.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.313+0000 7f0cbeffd700 1 -- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cc80c7960 msgr2=0x7f0cc80d9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.313+0000 7f0cbeffd700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cc80c7960 0x7f0cc80d9030 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000e730 tx=0x7f0cc000eaf0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.313+0000 7f0cbeffd700 1 -- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cc80c7960 msgr2=0x7f0cc80d9030 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:04:58.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.313+0000 7f0cbeffd700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cc80c7960 0x7f0cc80d9030 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000e730 tx=0x7f0cc000eaf0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.313+0000 7f0cced9d700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0cb0066730 0x7f0cb0068bf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0cd004ffd0 tx=0x7f0cd0065ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:58.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.321+0000 7f0ccf59e700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cc80d0a90 0x7f0cc80d9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.321+0000 7f0ccf59e700 1 -- 172.21.15.40:0/2260762973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc8033440 con 0x7f0cc80d0a90 2024-09-13T18:04:58.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.321+0000 7f0cce59c700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cc80d0a90 0x7f0cc80d9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.325+0000 7f0cce59c700 1 -- 172.21.15.40:0/2260762973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0cc8033a00 con 0x7f0cc80d0a90 2024-09-13T18:04:58.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.325+0000 7f0cce59c700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cc80d0a90 0x7f0cc80d9570 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000e730 tx=0x7f0cc0003810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:58.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.325+0000 7f0cbeffd700 1 -- 172.21.15.40:0/2260762973 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0021070 con 0x7f0cc80d0a90 2024-09-13T18:04:58.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.325+0000 7f0cbeffd700 1 -- 172.21.15.40:0/2260762973 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7f0cc00073f0 con 0x7f0cc80d0a90 2024-09-13T18:04:58.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.325+0000 7f0cbeffd700 1 -- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cc80d0a90 msgr2=0x7f0cc80d9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.325+0000 7f0cbeffd700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cc80d0a90 0x7f0cc80d9570 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000e730 tx=0x7f0cc0003810 comp rx=0 tx=0).stop 2024-09-13T18:04:58.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.325+0000 7f0cbeffd700 1 -- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cc80d0a90 msgr2=0x7f0cc80d9570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:04:58.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.325+0000 7f0cbeffd700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cc80d0a90 0x7f0cc80d9570 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.333 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:04:58.333 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:04:58.333 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:04:58.333 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:04:58.333 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 118, 2024-09-13T18:04:58.333 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:04:58.334 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:04:58.334 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:04:58.334 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:04:58.334 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:58.334 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:04:58.334 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:04:58.334 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:04:58.334 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:04:58.334 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:58.334 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:58.334 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:58.334 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:58.334 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:58.334 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:58.335 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:58.335 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:58.335 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:58.335 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:58.335 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:58.335 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:04:58.335 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:04:58.335 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:58.335 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:58.335 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:58.335 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:58.335 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:58.335 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:58.335 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:58.335 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:58.336 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:58.336 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:58.336 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:58.336 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.336 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:04:58.336 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:04:58.336 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:04:58.336 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:04:58.336 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:04:58.336 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:04:58.336 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:04:58.336 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:04:58.336 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:04:58.337 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:04:58.337 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:04:58.337 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:04:58.337 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:04:58.337 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:04:58.337 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:04:58.337 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:04:58.337 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:04:58.337 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:58.337 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:58.337 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:58.337 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:58.337 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:58.337 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:58.338 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:58.338 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:58.338 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:58.338 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:58.338 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:58.338 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:04:58.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.338 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:04:58.338 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.338 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:04:58.338 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:04:58.338 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:58.338 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:58.338 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.339 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:58.339 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:04:58.339 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:58.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.339 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.339 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:58.339 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:04:58.339 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:58.339 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.339 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:58.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.339 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:04:58.339 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:04:58.339 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:58.339 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:58.340 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:58.340 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.340 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:04:58.340 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:04:58.340 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:58.340 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:58.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.340 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:58.340 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:04:58.340 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:58.340 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.341 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:58.341 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:04:58.341 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:58.341 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.341 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:58.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.341 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:04:58.341 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:04:58.341 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:58.341 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:58.341 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:58.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.341 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.342 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:04:58.342 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:04:58.342 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:58.342 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:58.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.342 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:58.342 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:04:58.342 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:58.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.342 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:58.342 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:04:58.342 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:58.343 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.343 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:58.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.343 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:04:58.343 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:04:58.343 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:58.343 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:58.343 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:58.343 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.343 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:58.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.343 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:04:58.343 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:04:58.344 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.344 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:58.344 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:58.344 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:04:58.344 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.344 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:58.344 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:04:58.344 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.344 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:58.344 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:58.344 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:04:58.344 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.344 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:58.344 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:04:58.345 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.345 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:58.345 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:58.345 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:04:58.345 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.345 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:58.345 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:04:58.345 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.345 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:58.345 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:58.345 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:04:58.345 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.345 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:58.346 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:04:58.346 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.346 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:58.346 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:58.346 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:04:58.346 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.346 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:58.346 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.346 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:04:58.346 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:04:58.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.325+0000 7f0ccf59e700 1 -- 172.21.15.40:0/2260762973 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0cb0066730 msgr2=0x7f0cb0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.325+0000 7f0ccf59e700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0cb0066730 0x7f0cb0068bf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0cd004ffd0 tx=0x7f0cd0065ea0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.325+0000 7f0ccf59e700 1 -- 172.21.15.40:0/2260762973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cc80d57b0 msgr2=0x7f0cc80c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.325+0000 7f0ccf59e700 1 --2- 172.21.15.40:0/2260762973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cc80d57b0 0x7f0cc80c8b40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009c60 tx=0x7f0cc4004770 comp rx=0 tx=0).stop 2024-09-13T18:04:58.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.325+0000 7f0cced9d700 1 -- 172.21.15.40:0/2260762973 reap_dead start 2024-09-13T18:04:58.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.325+0000 7f0cced9d700 1 -- 172.21.15.40:0/2260762973 reap_dead start 2024-09-13T18:04:58.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.325+0000 7f0ccf59e700 1 -- 172.21.15.40:0/2260762973 shutdown_connections 2024-09-13T18:04:58.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.325+0000 7f0ccf59e700 1 -- 172.21.15.40:0/2260762973 >> 172.21.15.40:0/2260762973 conn(0x7f0cc8050150 msgr2=0x7f0cc8006d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:58.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.325+0000 7f0ccf59e700 1 -- 172.21.15.40:0/2260762973 shutdown_connections 2024-09-13T18:04:58.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.325+0000 7f0ccf59e700 1 -- 172.21.15.40:0/2260762973 wait complete. 2024-09-13T18:04:58.348 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:04:58.349 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:04:58.349 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:04:58.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.453+0000 7fa4cbfff700 1 Processor -- start 2024-09-13T18:04:58.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.453+0000 7fa4cbfff700 1 -- start start 2024-09-13T18:04:58.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.453+0000 7fa4cbfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4cc0c54e0 0x7fa4cc071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.453+0000 7fa4cbfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa4cc0c4f90 0x7fa4cc15a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.453+0000 7fa4cbfff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4cc0706f0 con 0x7fa4cc150240 2024-09-13T18:04:58.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.453+0000 7fa4cbfff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4cc0c28c0 con 0x7fa4cc0c4f90 2024-09-13T18:04:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.453+0000 7fa4cbfff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4cc070450 con 0x7fa4cc0c54e0 2024-09-13T18:04:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.453+0000 7fa4cb7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa4cc0c4f90 0x7fa4cc15a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.453+0000 7fa4caffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa4cc150240 0x7fa4cc070ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48188/0 (socket says 172.21.15.40:48188) 2024-09-13T18:04:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.453+0000 7fa4caffd700 1 -- 172.21.15.40:0/2912514851 learned_addr learned my addr 172.21.15.40:0/2912514851 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:04:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.453+0000 7fa4c3fff700 1 --2- 172.21.15.40:0/2912514851 >> v2:172.21.15.43:3300/0 conn(0x7fa4cc0c54e0 0x7fa4cc071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.453+0000 7fa4cb7fe700 1 -- 172.21.15.40:0/2912514851 >> v2:172.21.15.43:3300/0 conn(0x7fa4cc0c54e0 msgr2=0x7fa4cc071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:04:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.453+0000 7fa4cb7fe700 1 --2- 172.21.15.40:0/2912514851 >> v2:172.21.15.43:3300/0 conn(0x7fa4cc0c54e0 0x7fa4cc071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.453+0000 7fa4cb7fe700 1 -- 172.21.15.40:0/2912514851 >> v1:172.21.15.40:6789/0 conn(0x7fa4cc150240 legacy=0x7fa4cc070ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.453+0000 7fa4cb7fe700 1 -- 172.21.15.40:0/2912514851 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4cc156c00 con 0x7fa4cc0c4f90 2024-09-13T18:04:58.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.457+0000 7fa4cb7fe700 1 --2- 172.21.15.40:0/2912514851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4cc0c4f90 0x7fa4cc15a0e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0c9200 tx=0x7fa4c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29e07cf1fb086f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:58.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.457+0000 7fa4ca7fc700 1 -- 172.21.15.40:0/2912514851 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c400e070 con 0x7fa4cc0c4f90 2024-09-13T18:04:58.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.457+0000 7fa4ca7fc700 1 -- 172.21.15.40:0/2912514851 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c400bad0 con 0x7fa4cc0c4f90 2024-09-13T18:04:58.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.457+0000 7fa4ca7fc700 1 -- 172.21.15.40:0/2912514851 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c400be80 con 0x7fa4cc0c4f90 2024-09-13T18:04:58.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.457+0000 7fa4cbfff700 1 -- 172.21.15.40:0/2912514851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4cc0c4f90 msgr2=0x7fa4cc15a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.457+0000 7fa4cbfff700 1 --2- 172.21.15.40:0/2912514851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4cc0c4f90 0x7fa4cc15a0e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0c9200 tx=0x7fa4c400b330 comp rx=0 tx=0).stop 2024-09-13T18:04:58.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.457+0000 7fa4cbfff700 1 -- 172.21.15.40:0/2912514851 shutdown_connections 2024-09-13T18:04:58.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.457+0000 7fa4cbfff700 1 --2- 172.21.15.40:0/2912514851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4cc0c4f90 0x7fa4cc15a0e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.457+0000 7fa4cbfff700 1 --2- 172.21.15.40:0/2912514851 >> v2:172.21.15.43:3300/0 conn(0x7fa4cc0c54e0 0x7fa4cc071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.457+0000 7fa4cbfff700 1 -- 172.21.15.40:0/2912514851 >> 172.21.15.40:0/2912514851 conn(0x7fa4cc06a390 msgr2=0x7fa4cc06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:58.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.457+0000 7fa4cbfff700 1 -- 172.21.15.40:0/2912514851 shutdown_connections 2024-09-13T18:04:58.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.457+0000 7fa4cbfff700 1 -- 172.21.15.40:0/2912514851 wait complete. 2024-09-13T18:04:58.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.457+0000 7fa4cbfff700 1 Processor -- start 2024-09-13T18:04:58.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.457+0000 7fa4cbfff700 1 -- start start 2024-09-13T18:04:58.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4cbfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4cc0c4f90 0x7fa4cc1f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4cbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4cc0c54e0 0x7fa4cc0c0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4cbfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4cc150240 0x7fa4cc1efcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4cbfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc0bc670 con 0x7fa4cc0c54e0 2024-09-13T18:04:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4cbfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc155c90 con 0x7fa4cc150240 2024-09-13T18:04:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4cbfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc0ca6a0 con 0x7fa4cc0c4f90 2024-09-13T18:04:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4c3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4cc0c54e0 0x7fa4cc0c0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4c3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4cc0c54e0 0x7fa4cc0c0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34978/0 (socket says 172.21.15.40:34978) 2024-09-13T18:04:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4c3fff700 1 -- 172.21.15.40:0/1776071555 learned_addr learned my addr 172.21.15.40:0/1776071555 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4caffd700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4cc0c4f90 0x7fa4cc1f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4cb7fe700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4cc150240 0x7fa4cc1efcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4c3fff700 1 -- 172.21.15.40:0/1776071555 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4cc0c4f90 msgr2=0x7fa4cc1f1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4c3fff700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4cc0c4f90 0x7fa4cc1f1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4c3fff700 1 -- 172.21.15.40:0/1776071555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4cc150240 msgr2=0x7fa4cc1efcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4c3fff700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4cc150240 0x7fa4cc1efcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4c3fff700 1 -- 172.21.15.40:0/1776071555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c4009a70 con 0x7fa4cc0c54e0 2024-09-13T18:04:58.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4c3fff700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4cc0c54e0 0x7fa4cc0c0ac0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400e840 tx=0x7fa4b400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:58.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4c8ff9700 1 -- 172.21.15.40:0/1776071555 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4013610 con 0x7fa4cc0c54e0 2024-09-13T18:04:58.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4c8ff9700 1 -- 172.21.15.40:0/1776071555 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b4013770 con 0x7fa4cc0c54e0 2024-09-13T18:04:58.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4cbfff700 1 -- 172.21.15.40:0/1776071555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4cc154da0 con 0x7fa4cc0c54e0 2024-09-13T18:04:58.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4c8ff9700 1 -- 172.21.15.40:0/1776071555 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4013b20 con 0x7fa4cc0c54e0 2024-09-13T18:04:58.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.461+0000 7fa4cbfff700 1 -- 172.21.15.40:0/1776071555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4cc155360 con 0x7fa4cc0c54e0 2024-09-13T18:04:58.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.465+0000 7fa4cbfff700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4cc155520 0x7fa4cc14cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.465+0000 7fa4cbfff700 1 -- 172.21.15.40:0/1776071555 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc067600 con 0x7fa4cc155520 2024-09-13T18:04:58.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.465+0000 7fa4c8ff9700 1 -- 172.21.15.40:0/1776071555 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa4b4013c80 con 0x7fa4cc0c54e0 2024-09-13T18:04:58.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.465+0000 7fa4caffd700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4cc155520 0x7fa4cc14cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.465+0000 7fa4c8ff9700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4ac066730 0x7fa4ac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.465+0000 7fa4cb7fe700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4ac066730 0x7fa4ac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.465+0000 7fa4c8ff9700 1 -- 172.21.15.40:0/1776071555 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa4b4084150 con 0x7fa4cc0c54e0 2024-09-13T18:04:58.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.465+0000 7fa4caffd700 1 -- 172.21.15.40:0/1776071555 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa4cc0664b0 con 0x7fa4cc155520 2024-09-13T18:04:58.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.465+0000 7fa4caffd700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4cc155520 0x7fa4cc14cc50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc000f60 tx=0x7fa4bc0018b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:58.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.465+0000 7fa4cb7fe700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4ac066730 0x7fa4ac068bf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400b370 tx=0x7fa4c4014040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:58.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.465+0000 7fa4c8ff9700 1 -- 172.21.15.40:0/1776071555 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc010510 con 0x7fa4cc155520 2024-09-13T18:04:58.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.465+0000 7fa4c8ff9700 1 -- 172.21.15.40:0/1776071555 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa4bc010670 con 0x7fa4cc155520 2024-09-13T18:04:58.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.465+0000 7fa4c8ff9700 1 -- 172.21.15.40:0/1776071555 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4cc155520 msgr2=0x7fa4cc14cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.465+0000 7fa4c8ff9700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4cc155520 0x7fa4cc14cc50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc000f60 tx=0x7fa4bc0018b0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.465+0000 7fa4c8ff9700 1 -- 172.21.15.40:0/1776071555 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4cc155520 msgr2=0x7fa4cc14cc50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:04:58.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.465+0000 7fa4c8ff9700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4cc155520 0x7fa4cc14cc50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.477+0000 7fa4cbfff700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4cc14d190 0x7fa4cc0c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.477+0000 7fa4cbfff700 1 -- 172.21.15.40:0/1776071555 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc1598e0 con 0x7fa4cc14d190 2024-09-13T18:04:58.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.477+0000 7fa4caffd700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4cc14d190 0x7fa4cc0c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.477+0000 7fa4caffd700 1 -- 172.21.15.40:0/1776071555 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa4cc159600 con 0x7fa4cc14d190 2024-09-13T18:04:58.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.477+0000 7fa4caffd700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4cc14d190 0x7fa4cc0c70d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc016000 tx=0x7fa4bc000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:58.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.477+0000 7fa4c8ff9700 1 -- 172.21.15.40:0/1776071555 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc021070 con 0x7fa4cc14d190 2024-09-13T18:04:58.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.477+0000 7fa4c8ff9700 1 -- 172.21.15.40:0/1776071555 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7fa4bc01cbd0 con 0x7fa4cc14d190 2024-09-13T18:04:58.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.477+0000 7fa4c8ff9700 1 -- 172.21.15.40:0/1776071555 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4cc14d190 msgr2=0x7fa4cc0c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.477+0000 7fa4c8ff9700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4cc14d190 0x7fa4cc0c70d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc016000 tx=0x7fa4bc000ed0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.477+0000 7fa4c8ff9700 1 -- 172.21.15.40:0/1776071555 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4cc14d190 msgr2=0x7fa4cc0c70d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:04:58.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.477+0000 7fa4c8ff9700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4cc14d190 0x7fa4cc0c70d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.487 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:04:58.487 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:04:58.487 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:04:58.487 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:04:58.487 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 118, 2024-09-13T18:04:58.487 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:04:58.487 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:04:58.487 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:04:58.487 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:04:58.487 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:58.487 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:04:58.487 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:04:58.487 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:04:58.488 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:04:58.488 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:58.488 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:58.488 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:58.488 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:58.488 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:58.488 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:58.488 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:58.488 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:58.488 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:58.488 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:58.488 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:58.488 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:04:58.489 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:04:58.489 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:58.489 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:58.489 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:58.489 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:58.489 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:58.489 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:58.489 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:58.489 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:58.489 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:58.489 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:58.489 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:58.489 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.490 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:04:58.490 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:04:58.490 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:04:58.490 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:04:58.490 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:04:58.490 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:04:58.490 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:04:58.490 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:04:58.490 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:04:58.490 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:04:58.490 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:04:58.490 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:04:58.490 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:04:58.490 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:04:58.491 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:04:58.491 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:04:58.491 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:04:58.491 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:04:58.491 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:04:58.491 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:04:58.491 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:04:58.491 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:04:58.491 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:04:58.491 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:04:58.491 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:04:58.491 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:04:58.491 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:04:58.492 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:58.492 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:04:58.492 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.492 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:04:58.492 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.492 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:04:58.492 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:04:58.492 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:58.492 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:58.492 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.492 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:58.492 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:04:58.492 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:58.493 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.493 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.493 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:58.493 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:04:58.493 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:58.493 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.493 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:58.493 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.493 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:04:58.493 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:04:58.493 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:58.493 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:58.493 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:58.494 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.494 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.494 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:04:58.494 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:04:58.494 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:58.494 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:58.494 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.494 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:58.494 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:04:58.494 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:58.494 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.494 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.494 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:58.495 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:04:58.495 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:58.495 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.495 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:58.495 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.495 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:04:58.495 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:04:58.495 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:58.495 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:58.495 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:58.495 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.495 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.495 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:04:58.496 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:04:58.496 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:04:58.496 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:04:58.496 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.496 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:04:58.496 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:04:58.496 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:58.496 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.496 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.496 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:04:58.496 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:04:58.496 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:04:58.496 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.496 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:58.497 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.497 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:04:58.497 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:04:58.497 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:04:58.497 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:04:58.497 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:04:58.497 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.497 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:58.497 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.497 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:04:58.497 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:04:58.497 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.498 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:58.498 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:58.498 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:04:58.498 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.498 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:58.498 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:04:58.498 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.498 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:58.498 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:58.498 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:04:58.498 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.498 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:58.499 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:04:58.499 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.499 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:58.499 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:58.499 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:04:58.499 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.499 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:58.499 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:04:58.499 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.499 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:58.499 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:58.500 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:04:58.500 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.500 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:04:58.500 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:04:58.500 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:04:58.500 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:04:58.500 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:04:58.500 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:04:58.500 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:04:58.500 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:04:58.500 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:04:58.500 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:04:58.500 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:04:58.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.477+0000 7fa4cbfff700 1 -- 172.21.15.40:0/1776071555 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4ac066730 msgr2=0x7fa4ac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.477+0000 7fa4cbfff700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa4ac066730 0x7fa4ac068bf0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400b370 tx=0x7fa4c4014040 comp rx=0 tx=0).stop 2024-09-13T18:04:58.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.477+0000 7fa4cbfff700 1 -- 172.21.15.40:0/1776071555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4cc0c54e0 msgr2=0x7fa4cc0c0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.477+0000 7fa4cbfff700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4cc0c54e0 0x7fa4cc0c0ac0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400e840 tx=0x7fa4b400ec00 comp rx=0 tx=0).stop 2024-09-13T18:04:58.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.477+0000 7fa4cb7fe700 1 -- 172.21.15.40:0/1776071555 reap_dead start 2024-09-13T18:04:58.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.481+0000 7fa4cbfff700 1 -- 172.21.15.40:0/1776071555 shutdown_connections 2024-09-13T18:04:58.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.481+0000 7fa4cbfff700 1 --2- 172.21.15.40:0/1776071555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4cc0c54e0 0x7fa4cc0c0ac0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.481+0000 7fa4cbfff700 1 -- 172.21.15.40:0/1776071555 >> 172.21.15.40:0/1776071555 conn(0x7fa4cc06a390 msgr2=0x7fa4cc06e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:58.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.481+0000 7fa4cbfff700 1 -- 172.21.15.40:0/1776071555 shutdown_connections 2024-09-13T18:04:58.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.481+0000 7fa4cbfff700 1 -- 172.21.15.40:0/1776071555 wait complete. 2024-09-13T18:04:58.503 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:04:58.503 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:04:58.503 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:04:58.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.609+0000 7f1d37fff700 1 Processor -- start 2024-09-13T18:04:58.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.609+0000 7f1d37fff700 1 -- start start 2024-09-13T18:04:58.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.609+0000 7f1d37fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d38156ef0 0x7f1d380c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.609+0000 7f1d37fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d38159800 con 0x7f1d38150690 2024-09-13T18:04:58.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.609+0000 7f1d37fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d38150a70 con 0x7f1d38070bc0 2024-09-13T18:04:58.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.609+0000 7f1d37fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d380c8350 con 0x7f1d38156ef0 2024-09-13T18:04:58.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.609+0000 7f1d36ffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1d38150690 0x7f1d3806b4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48198/0 (socket says 172.21.15.40:48198) 2024-09-13T18:04:58.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.609+0000 7f1d36ffd700 1 -- 172.21.15.40:0/1153495400 learned_addr learned my addr 172.21.15.40:0/1153495400 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:04:58.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.609+0000 7f1d377fe700 1 --2- 172.21.15.40:0/1153495400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d38156ef0 0x7f1d380c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.609+0000 7f1d35ffb700 1 -- 172.21.15.40:0/1153495400 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923980859 0 0) 0x7f1d38159800 con 0x7f1d38150690 2024-09-13T18:04:58.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.609+0000 7f1d35ffb700 1 -- 172.21.15.40:0/1153495400 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d200035d0 con 0x7f1d38150690 2024-09-13T18:04:58.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.609+0000 7f1d377fe700 1 -- 172.21.15.40:0/1153495400 >> v1:172.21.15.43:6789/0 conn(0x7f1d38070bc0 legacy=0x7f1d381567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.609+0000 7f1d35ffb700 1 -- 172.21.15.40:0/1153495400 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812469806 0 0) 0x7f1d38150a70 con 0x7f1d38070bc0 2024-09-13T18:04:58.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.609+0000 7f1d377fe700 1 -- 172.21.15.40:0/1153495400 >> v1:172.21.15.40:6789/0 conn(0x7f1d38150690 legacy=0x7f1d3806b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.609+0000 7f1d377fe700 1 -- 172.21.15.40:0/1153495400 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d381562e0 con 0x7f1d38156ef0 2024-09-13T18:04:58.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.613+0000 7f1d377fe700 1 --2- 172.21.15.40:0/1153495400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d38156ef0 0x7f1d380c67e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009dc0 tx=0x7f1d3000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e70ed2215fd945a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:58.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.613+0000 7f1d35ffb700 1 -- 172.21.15.40:0/1153495400 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3000e070 con 0x7f1d38156ef0 2024-09-13T18:04:58.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.613+0000 7f1d35ffb700 1 -- 172.21.15.40:0/1153495400 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d30003a40 con 0x7f1d38156ef0 2024-09-13T18:04:58.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.613+0000 7f1d35ffb700 1 -- 172.21.15.40:0/1153495400 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30003db0 con 0x7f1d38156ef0 2024-09-13T18:04:58.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.613+0000 7f1d37fff700 1 -- 172.21.15.40:0/1153495400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d38156ef0 msgr2=0x7f1d380c67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.613+0000 7f1d37fff700 1 --2- 172.21.15.40:0/1153495400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d38156ef0 0x7f1d380c67e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009dc0 tx=0x7f1d3000b330 comp rx=0 tx=0).stop 2024-09-13T18:04:58.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.613+0000 7f1d37fff700 1 -- 172.21.15.40:0/1153495400 shutdown_connections 2024-09-13T18:04:58.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.613+0000 7f1d37fff700 1 --2- 172.21.15.40:0/1153495400 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d38156ef0 0x7f1d380c67e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.613+0000 7f1d37fff700 1 -- 172.21.15.40:0/1153495400 >> 172.21.15.40:0/1153495400 conn(0x7f1d380c88e0 msgr2=0x7f1d380c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:58.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.613+0000 7f1d37fff700 1 -- 172.21.15.40:0/1153495400 shutdown_connections 2024-09-13T18:04:58.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.613+0000 7f1d37fff700 1 -- 172.21.15.40:0/1153495400 wait complete. 2024-09-13T18:04:58.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.613+0000 7f1d37fff700 1 Processor -- start 2024-09-13T18:04:58.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d37fff700 1 -- start start 2024-09-13T18:04:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d37fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d38070bc0 0x7f1d381f2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d37fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d38150690 0x7f1d38148ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d37fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d38156ef0 0x7f1d38071f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d37fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d381f2dd0 con 0x7f1d38156ef0 2024-09-13T18:04:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d37fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3814b5f0 con 0x7f1d38070bc0 2024-09-13T18:04:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d37fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3814b8d0 con 0x7f1d38150690 2024-09-13T18:04:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d377fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d38156ef0 0x7f1d38071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d377fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d38156ef0 0x7f1d38071f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34990/0 (socket says 172.21.15.40:34990) 2024-09-13T18:04:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d377fe700 1 -- 172.21.15.40:0/1694395137 learned_addr learned my addr 172.21.15.40:0/1694395137 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:04:58.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d367fc700 1 --2- 172.21.15.40:0/1694395137 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d38150690 0x7f1d38148ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d36ffd700 1 --2- 172.21.15.40:0/1694395137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d38070bc0 0x7f1d381f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d377fe700 1 -- 172.21.15.40:0/1694395137 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d38150690 msgr2=0x7f1d38148ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d377fe700 1 --2- 172.21.15.40:0/1694395137 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d38150690 0x7f1d38148ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d377fe700 1 -- 172.21.15.40:0/1694395137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d38070bc0 msgr2=0x7f1d381f2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d377fe700 1 --2- 172.21.15.40:0/1694395137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d38070bc0 0x7f1d381f2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d377fe700 1 -- 172.21.15.40:0/1694395137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d30009a70 con 0x7f1d38156ef0 2024-09-13T18:04:58.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d36ffd700 1 --2- 172.21.15.40:0/1694395137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d38070bc0 0x7f1d381f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:04:58.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d377fe700 1 --2- 172.21.15.40:0/1694395137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d38156ef0 0x7f1d38071f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1d30004000 tx=0x7f1d30004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:58.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d1f7fe700 1 -- 172.21.15.40:0/1694395137 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30009410 con 0x7f1d38156ef0 2024-09-13T18:04:58.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d37fff700 1 -- 172.21.15.40:0/1694395137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d3814bbb0 con 0x7f1d38156ef0 2024-09-13T18:04:58.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d1f7fe700 1 -- 172.21.15.40:0/1694395137 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d30004760 con 0x7f1d38156ef0 2024-09-13T18:04:58.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d1f7fe700 1 -- 172.21.15.40:0/1694395137 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3001c460 con 0x7f1d38156ef0 2024-09-13T18:04:58.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.617+0000 7f1d37fff700 1 -- 172.21.15.40:0/1694395137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d381f2370 con 0x7f1d38156ef0 2024-09-13T18:04:58.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.621+0000 7f1d37fff700 1 -- 172.21.15.40:0/1694395137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d381f2550 con 0x7f1d38156ef0 2024-09-13T18:04:58.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.621+0000 7f1d1f7fe700 1 -- 172.21.15.40:0/1694395137 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1d3001c5c0 con 0x7f1d38156ef0 2024-09-13T18:04:58.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.621+0000 7f1d1f7fe700 1 --2- 172.21.15.40:0/1694395137 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d24066730 0x7f1d24068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:04:58.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.621+0000 7f1d1f7fe700 1 -- 172.21.15.40:0/1694395137 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1d30085230 con 0x7f1d38156ef0 2024-09-13T18:04:58.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.621+0000 7f1d36ffd700 1 --2- 172.21.15.40:0/1694395137 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d24066730 0x7f1d24068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:04:58.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.621+0000 7f1d36ffd700 1 --2- 172.21.15.40:0/1694395137 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d24066730 0x7f1d24068bf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1d28005930 tx=0x7f1d28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:04:58.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.625+0000 7f1d1f7fe700 1 -- 172.21.15.40:0/1694395137 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1d3008a060 con 0x7f1d38156ef0 2024-09-13T18:04:58.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.889+0000 7f1d37fff700 1 -- 172.21.15.40:0/1694395137 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f1d3815a5e0 con 0x7f1d38156ef0 2024-09-13T18:04:58.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.893+0000 7f1d1f7fe700 1 -- 172.21.15.40:0/1694395137 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f1d300521e0 con 0x7f1d38156ef0 2024-09-13T18:04:58.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.893+0000 7f1d37fff700 1 -- 172.21.15.40:0/1694395137 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d24066730 msgr2=0x7f1d24068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.893+0000 7f1d37fff700 1 --2- 172.21.15.40:0/1694395137 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d24066730 0x7f1d24068bf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1d28005930 tx=0x7f1d28009040 comp rx=0 tx=0).stop 2024-09-13T18:04:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.893+0000 7f1d37fff700 1 -- 172.21.15.40:0/1694395137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d38156ef0 msgr2=0x7f1d38071f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:04:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.893+0000 7f1d37fff700 1 --2- 172.21.15.40:0/1694395137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d38156ef0 0x7f1d38071f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1d30004000 tx=0x7f1d30004af0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.893+0000 7f1d37fff700 1 -- 172.21.15.40:0/1694395137 shutdown_connections 2024-09-13T18:04:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.893+0000 7f1d37fff700 1 --2- 172.21.15.40:0/1694395137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d38156ef0 0x7f1d38071f20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.893+0000 7f1d37fff700 1 --2- 172.21.15.40:0/1694395137 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d24066730 0x7f1d24068bf0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.893+0000 7f1d37fff700 1 --2- 172.21.15.40:0/1694395137 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d38150690 0x7f1d38148ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.893+0000 7f1d37fff700 1 --2- 172.21.15.40:0/1694395137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d38070bc0 0x7f1d381f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:04:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.893+0000 7f1d37fff700 1 -- 172.21.15.40:0/1694395137 >> 172.21.15.40:0/1694395137 conn(0x7f1d380c88e0 msgr2=0x7f1d380c6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:04:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.893+0000 7f1d37fff700 1 -- 172.21.15.40:0/1694395137 shutdown_connections 2024-09-13T18:04:58.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:04:58.893+0000 7f1d37fff700 1 -- 172.21.15.40:0/1694395137 wait complete. 2024-09-13T18:04:58.915 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:05:06.188 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:05:06.191 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:05:08.916 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:05:08.917 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:05:09.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.029+0000 7f2be1656700 1 Processor -- start 2024-09-13T18:05:09.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.029+0000 7f2be1656700 1 -- start start 2024-09-13T18:05:09.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.029+0000 7f2be1656700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2bdc153880 0x7f2bdc0c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.029+0000 7f2be1656700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bdc159770 con 0x7f2bdc15cf60 2024-09-13T18:05:09.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.029+0000 7f2be1656700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bdc159a50 con 0x7f2bdc0cb2f0 2024-09-13T18:05:09.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.029+0000 7f2be1656700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc159d30 con 0x7f2bdc153880 2024-09-13T18:05:09.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.029+0000 7f2be0e55700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2bdc15cf60 0x7f2bdc163ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44542/0 (socket says 172.21.15.40:44542) 2024-09-13T18:05:09.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.029+0000 7f2be0e55700 1 -- 172.21.15.40:0/3797360319 learned_addr learned my addr 172.21.15.40:0/3797360319 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:05:09.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.029+0000 7f2bdbfff700 1 --2- 172.21.15.40:0/3797360319 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2bdc153880 0x7f2bdc0c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.029+0000 7f2bdaffd700 1 -- 172.21.15.40:0/3797360319 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492190778 0 0) 0x7f2bdc159770 con 0x7f2bdc15cf60 2024-09-13T18:05:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.029+0000 7f2bdaffd700 1 -- 172.21.15.40:0/3797360319 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bc80035d0 con 0x7f2bdc15cf60 2024-09-13T18:05:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.029+0000 7f2bdaffd700 1 -- 172.21.15.40:0/3797360319 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 818142345 0 0) 0x7f2bdc159a50 con 0x7f2bdc0cb2f0 2024-09-13T18:05:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.029+0000 7f2bdaffd700 1 -- 172.21.15.40:0/3797360319 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bdc159770 con 0x7f2bdc0cb2f0 2024-09-13T18:05:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.029+0000 7f2bdbfff700 1 -- 172.21.15.40:0/3797360319 >> v1:172.21.15.43:6789/0 conn(0x7f2bdc0cb2f0 legacy=0x7f2bdc0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.029+0000 7f2bdaffd700 1 -- 172.21.15.40:0/3797360319 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3062876357 0 0) 0x7f2bc80035d0 con 0x7f2bdc15cf60 2024-09-13T18:05:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.029+0000 7f2bdbfff700 1 -- 172.21.15.40:0/3797360319 >> v1:172.21.15.40:6789/0 conn(0x7f2bdc15cf60 legacy=0x7f2bdc163ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.029+0000 7f2bdbfff700 1 -- 172.21.15.40:0/3797360319 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bdc15a010 con 0x7f2bdc153880 2024-09-13T18:05:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.029+0000 7f2bdbfff700 1 --2- 172.21.15.40:0/3797360319 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2bdc153880 0x7f2bdc0c9ef0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009880 tx=0x7f2bc4009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=decdb898acf11c42 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.029+0000 7f2bdaffd700 1 -- 172.21.15.40:0/3797360319 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4001970 con 0x7f2bdc153880 2024-09-13T18:05:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.033+0000 7f2bdaffd700 1 -- 172.21.15.40:0/3797360319 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc4004030 con 0x7f2bdc153880 2024-09-13T18:05:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.033+0000 7f2bdaffd700 1 -- 172.21.15.40:0/3797360319 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc40043c0 con 0x7f2bdc153880 2024-09-13T18:05:09.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.033+0000 7f2be1656700 1 -- 172.21.15.40:0/3797360319 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2bdc153880 msgr2=0x7f2bdc0c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.033+0000 7f2be1656700 1 --2- 172.21.15.40:0/3797360319 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2bdc153880 0x7f2bdc0c9ef0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009880 tx=0x7f2bc4009b90 comp rx=0 tx=0).stop 2024-09-13T18:05:09.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.033+0000 7f2be1656700 1 -- 172.21.15.40:0/3797360319 shutdown_connections 2024-09-13T18:05:09.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.033+0000 7f2be1656700 1 --2- 172.21.15.40:0/3797360319 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2bdc153880 0x7f2bdc0c9ef0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.033+0000 7f2be1656700 1 -- 172.21.15.40:0/3797360319 >> 172.21.15.40:0/3797360319 conn(0x7f2bdc06c5b0 msgr2=0x7f2bdc06c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:09.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.033+0000 7f2be1656700 1 -- 172.21.15.40:0/3797360319 shutdown_connections 2024-09-13T18:05:09.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.033+0000 7f2be1656700 1 -- 172.21.15.40:0/3797360319 wait complete. 2024-09-13T18:05:09.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.033+0000 7f2be1656700 1 Processor -- start 2024-09-13T18:05:09.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.033+0000 7f2be1656700 1 -- start start 2024-09-13T18:05:09.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.033+0000 7f2be1656700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bdc0cb2f0 0x7f2bdc0c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.033+0000 7f2be1656700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bdc153880 0x7f2bdc158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.033+0000 7f2be1656700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2bdc15cf60 0x7f2bdc15bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.033+0000 7f2be1656700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc0c6890 con 0x7f2bdc153880 2024-09-13T18:05:09.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.033+0000 7f2be1656700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc158500 con 0x7f2bdc0cb2f0 2024-09-13T18:05:09.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.033+0000 7f2be1656700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc158ed0 con 0x7f2bdc15cf60 2024-09-13T18:05:09.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.033+0000 7f2bdb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bdc153880 0x7f2bdc158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.033+0000 7f2bdb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bdc153880 0x7f2bdc158990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38858/0 (socket says 172.21.15.40:38858) 2024-09-13T18:05:09.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.033+0000 7f2bdb7fe700 1 -- 172.21.15.40:0/1707243545 learned_addr learned my addr 172.21.15.40:0/1707243545 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:09.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.037+0000 7f2bdbfff700 1 --2- 172.21.15.40:0/1707243545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bdc0cb2f0 0x7f2bdc0c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.037+0000 7f2be0e55700 1 --2- 172.21.15.40:0/1707243545 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2bdc15cf60 0x7f2bdc15bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.037+0000 7f2bdb7fe700 1 -- 172.21.15.40:0/1707243545 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2bdc15cf60 msgr2=0x7f2bdc15bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.037+0000 7f2bdb7fe700 1 --2- 172.21.15.40:0/1707243545 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2bdc15cf60 0x7f2bdc15bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.037+0000 7f2bdb7fe700 1 -- 172.21.15.40:0/1707243545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bdc0cb2f0 msgr2=0x7f2bdc0c6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.037+0000 7f2bdb7fe700 1 --2- 172.21.15.40:0/1707243545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bdc0cb2f0 0x7f2bdc0c6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.037+0000 7f2bdb7fe700 1 -- 172.21.15.40:0/1707243545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc40094b0 con 0x7f2bdc153880 2024-09-13T18:05:09.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.037+0000 7f2bdb7fe700 1 --2- 172.21.15.40:0/1707243545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bdc153880 0x7f2bdc158990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00c5b0 tx=0x7f2bcc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.037+0000 7f2bd8ff9700 1 -- 172.21.15.40:0/1707243545 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc0052d0 con 0x7f2bdc153880 2024-09-13T18:05:09.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.037+0000 7f2bd8ff9700 1 -- 172.21.15.40:0/1707243545 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bcc005450 con 0x7f2bdc153880 2024-09-13T18:05:09.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.037+0000 7f2be1656700 1 -- 172.21.15.40:0/1707243545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bdc154100 con 0x7f2bdc153880 2024-09-13T18:05:09.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.037+0000 7f2bd8ff9700 1 -- 172.21.15.40:0/1707243545 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc0038e0 con 0x7f2bdc153880 2024-09-13T18:05:09.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.037+0000 7f2be1656700 1 -- 172.21.15.40:0/1707243545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bdc1593b0 con 0x7f2bdc153880 2024-09-13T18:05:09.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.037+0000 7f2be1656700 1 -- 172.21.15.40:0/1707243545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bdc1586b0 con 0x7f2bdc153880 2024-09-13T18:05:09.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.037+0000 7f2bd8ff9700 1 -- 172.21.15.40:0/1707243545 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2bcc003a40 con 0x7f2bdc153880 2024-09-13T18:05:09.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.041+0000 7f2bd8ff9700 1 --2- 172.21.15.40:0/1707243545 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2bbc066760 0x7f2bbc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.041+0000 7f2bd8ff9700 1 -- 172.21.15.40:0/1707243545 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2bcc083880 con 0x7f2bdc153880 2024-09-13T18:05:09.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.041+0000 7f2bdbfff700 1 --2- 172.21.15.40:0/1707243545 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2bbc066760 0x7f2bbc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.041+0000 7f2bdbfff700 1 --2- 172.21.15.40:0/1707243545 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2bbc066760 0x7f2bbc068c20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2bc40090b0 tx=0x7f2bc4010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.045+0000 7f2bd8ff9700 1 -- 172.21.15.40:0/1707243545 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2bdc1586b0 con 0x7f2bdc153880 2024-09-13T18:05:09.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.305+0000 7f2be1656700 1 -- 172.21.15.40:0/1707243545 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2bdc1586b0 con 0x7f2bdc153880 2024-09-13T18:05:09.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.305+0000 7f2bd8ff9700 1 -- 172.21.15.40:0/1707243545 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f2bcc017620 con 0x7f2bdc153880 2024-09-13T18:05:09.314 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":118,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:05:09.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.309+0000 7f2be1656700 1 -- 172.21.15.40:0/1707243545 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2bbc066760 msgr2=0x7f2bbc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.309+0000 7f2be1656700 1 --2- 172.21.15.40:0/1707243545 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2bbc066760 0x7f2bbc068c20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2bc40090b0 tx=0x7f2bc4010000 comp rx=0 tx=0).stop 2024-09-13T18:05:09.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.309+0000 7f2be1656700 1 -- 172.21.15.40:0/1707243545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bdc153880 msgr2=0x7f2bdc158990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.309+0000 7f2be1656700 1 --2- 172.21.15.40:0/1707243545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bdc153880 0x7f2bdc158990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00c5b0 tx=0x7f2bcc00c8c0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.309+0000 7f2be1656700 1 -- 172.21.15.40:0/1707243545 shutdown_connections 2024-09-13T18:05:09.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.309+0000 7f2be1656700 1 --2- 172.21.15.40:0/1707243545 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2bbc066760 0x7f2bbc068c20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.309+0000 7f2be1656700 1 --2- 172.21.15.40:0/1707243545 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2bdc15cf60 0x7f2bdc15bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.309+0000 7f2be1656700 1 --2- 172.21.15.40:0/1707243545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2bdc153880 0x7f2bdc158990 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.309+0000 7f2be1656700 1 --2- 172.21.15.40:0/1707243545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bdc0cb2f0 0x7f2bdc0c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.309+0000 7f2be1656700 1 -- 172.21.15.40:0/1707243545 >> 172.21.15.40:0/1707243545 conn(0x7f2bdc06c5b0 msgr2=0x7f2bdc0b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:09.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.309+0000 7f2be1656700 1 -- 172.21.15.40:0/1707243545 shutdown_connections 2024-09-13T18:05:09.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.309+0000 7f2be1656700 1 -- 172.21.15.40:0/1707243545 wait complete. 2024-09-13T18:05:09.333 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:05:09.333 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T18:05:09.333 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:05:09.334 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:05:09.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faadb045700 1 Processor -- start 2024-09-13T18:05:09.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faadb045700 1 -- start start 2024-09-13T18:05:09.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faadb045700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faad40cf1a0 0x7faad415a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faadb045700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad40cc250 0x7faad40c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faadb045700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad415a7a0 con 0x7faad40cf580 2024-09-13T18:05:09.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faadb045700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faad40cbdc0 con 0x7faad40cf1a0 2024-09-13T18:05:09.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faadb045700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad4153f80 con 0x7faad40cc250 2024-09-13T18:05:09.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faad9842700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7faad40cf580 0x7faad40c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44546/0 (socket says 172.21.15.40:44546) 2024-09-13T18:05:09.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faad9842700 1 -- 172.21.15.40:0/1589111330 learned_addr learned my addr 172.21.15.40:0/1589111330 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:05:09.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faada844700 1 --2- 172.21.15.40:0/1589111330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad40cc250 0x7faad40c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faada043700 1 --2- 172.21.15.40:0/1589111330 >> v2:172.21.15.43:3300/0 conn(0x7faad40cf1a0 0x7faad415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faad9041700 1 -- 172.21.15.40:0/1589111330 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2628447736 0 0) 0x7faac4001110 con 0x7faad40cf580 2024-09-13T18:05:09.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faad9041700 1 -- 172.21.15.40:0/1589111330 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faac00035d0 con 0x7faad40cf580 2024-09-13T18:05:09.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faad9041700 1 -- 172.21.15.40:0/1589111330 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2809280484 0 0) 0x7faac00035d0 con 0x7faad40cf580 2024-09-13T18:05:09.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faad9041700 1 -- 172.21.15.40:0/1589111330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad40cc250 msgr2=0x7faad40c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:05:09.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faad9041700 1 --2- 172.21.15.40:0/1589111330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad40cc250 0x7faad40c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faad9041700 1 -- 172.21.15.40:0/1589111330 >> v2:172.21.15.43:3300/0 conn(0x7faad40cf1a0 msgr2=0x7faad415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:05:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faad9041700 1 --2- 172.21.15.40:0/1589111330 >> v2:172.21.15.43:3300/0 conn(0x7faad40cf1a0 0x7faad415a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faad9041700 1 -- 172.21.15.40:0/1589111330 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad40caed0 con 0x7faad40cf580 2024-09-13T18:05:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faad9041700 1 -- 172.21.15.40:0/1589111330 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7faac4001800 con 0x7faad40cf580 2024-09-13T18:05:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faad9041700 1 -- 172.21.15.40:0/1589111330 >> v1:172.21.15.40:6789/0 conn(0x7faad40cf580 legacy=0x7faad40c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faad9041700 1 --2- 172.21.15.40:0/1589111330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faac0003a20 0x7faac0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faad9041700 1 --2- 172.21.15.40:0/1589111330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad40cc250 0x7faac0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faad9041700 1 --2- 172.21.15.40:0/1589111330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faac0004db0 0x7faac0005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faad9041700 1 -- 172.21.15.40:0/1589111330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad40cbdc0 con 0x7faac0003a20 2024-09-13T18:05:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faad9041700 1 -- 172.21.15.40:0/1589111330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad4153f80 con 0x7faac0004db0 2024-09-13T18:05:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faad9041700 1 -- 172.21.15.40:0/1589111330 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac000ba30 con 0x7faad40cc250 2024-09-13T18:05:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faada043700 1 --2- 172.21.15.40:0/1589111330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faac0003a20 0x7faac0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faada844700 1 --2- 172.21.15.40:0/1589111330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faac0004db0 0x7faac0005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faad9842700 1 --2- 172.21.15.40:0/1589111330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad40cc250 0x7faac0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faada043700 1 -- 172.21.15.40:0/1589111330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad40cc250 msgr2=0x7faac0004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:05:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faada043700 1 --2- 172.21.15.40:0/1589111330 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad40cc250 0x7faac0004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faada043700 1 -- 172.21.15.40:0/1589111330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faac0004db0 msgr2=0x7faac0005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:05:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faada043700 1 --2- 172.21.15.40:0/1589111330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faac0004db0 0x7faac0005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faada043700 1 -- 172.21.15.40:0/1589111330 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac00035d0 con 0x7faac0003a20 2024-09-13T18:05:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faada844700 1 --2- 172.21.15.40:0/1589111330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faac0004db0 0x7faac0005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:05:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.441+0000 7faada043700 1 --2- 172.21.15.40:0/1589111330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faac0003a20 0x7faac0003e40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faad0009b70 tx=0x7faac00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35625398bf687fa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.445+0000 7faad9041700 1 -- 172.21.15.40:0/1589111330 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad000e6c0 con 0x7faac0003a20 2024-09-13T18:05:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.445+0000 7faad9041700 1 -- 172.21.15.40:0/1589111330 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad000e820 con 0x7faac0003a20 2024-09-13T18:05:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.445+0000 7faad9041700 1 -- 172.21.15.40:0/1589111330 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad000eb90 con 0x7faac0003a20 2024-09-13T18:05:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.445+0000 7faadb045700 1 -- 172.21.15.40:0/1589111330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faac0003a20 msgr2=0x7faac0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.445+0000 7faadb045700 1 --2- 172.21.15.40:0/1589111330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faac0003a20 0x7faac0003e40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faad0009b70 tx=0x7faac00058a0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.445+0000 7faada844700 1 -- 172.21.15.40:0/1589111330 reap_dead start 2024-09-13T18:05:09.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.445+0000 7faadb045700 1 -- 172.21.15.40:0/1589111330 shutdown_connections 2024-09-13T18:05:09.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.445+0000 7faadb045700 1 -- 172.21.15.40:0/1589111330 >> 172.21.15.40:0/1589111330 conn(0x7faad4066090 msgr2=0x7faad4066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:09.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.445+0000 7faadb045700 1 -- 172.21.15.40:0/1589111330 shutdown_connections 2024-09-13T18:05:09.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.445+0000 7faadb045700 1 -- 172.21.15.40:0/1589111330 wait complete. 2024-09-13T18:05:09.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.445+0000 7faadb045700 1 Processor -- start 2024-09-13T18:05:09.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.445+0000 7faadb045700 1 -- start start 2024-09-13T18:05:09.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.445+0000 7faadb045700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faac0003a20 0x7faad4155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.445+0000 7faadb045700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faad4156f90 0x7faad4154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.445+0000 7faadb045700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad40c5c00 0x7faad414e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faadb045700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad414ae60 con 0x7faac0003a20 2024-09-13T18:05:09.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faadb045700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad414b090 con 0x7faad4156f90 2024-09-13T18:05:09.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faadb045700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad4154830 con 0x7faad40c5c00 2024-09-13T18:05:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faada043700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faac0003a20 0x7faad4155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faada043700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faac0003a20 0x7faad4155310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38880/0 (socket says 172.21.15.40:38880) 2024-09-13T18:05:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faada043700 1 -- 172.21.15.40:0/3791890616 learned_addr learned my addr 172.21.15.40:0/3791890616 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faada844700 1 --2- 172.21.15.40:0/3791890616 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad40c5c00 0x7faad414e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faad9842700 1 --2- 172.21.15.40:0/3791890616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faad4156f90 0x7faad4154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faada043700 1 -- 172.21.15.40:0/3791890616 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad40c5c00 msgr2=0x7faad414e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faada043700 1 --2- 172.21.15.40:0/3791890616 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad40c5c00 0x7faad414e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faada043700 1 -- 172.21.15.40:0/3791890616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faad4156f90 msgr2=0x7faad4154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faada043700 1 --2- 172.21.15.40:0/3791890616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faad4156f90 0x7faad4154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faada043700 1 -- 172.21.15.40:0/3791890616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad4154b10 con 0x7faac0003a20 2024-09-13T18:05:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faada043700 1 --2- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faac0003a20 0x7faad4155310 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faad00090b0 tx=0x7faad00047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faacaffd700 1 -- 172.21.15.40:0/3791890616 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad000e6c0 con 0x7faac0003a20 2024-09-13T18:05:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faacaffd700 1 -- 172.21.15.40:0/3791890616 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad00036c0 con 0x7faac0003a20 2024-09-13T18:05:09.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faadb045700 1 -- 172.21.15.40:0/3791890616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad4154df0 con 0x7faac0003a20 2024-09-13T18:05:09.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faacaffd700 1 -- 172.21.15.40:0/3791890616 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad0003a70 con 0x7faac0003a20 2024-09-13T18:05:09.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faadb045700 1 -- 172.21.15.40:0/3791890616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad4150030 con 0x7faac0003a20 2024-09-13T18:05:09.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faadb045700 1 --2- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faad40c7ae0 0x7faad414a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faadb045700 1 -- 172.21.15.40:0/3791890616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad4074750 con 0x7faad40c7ae0 2024-09-13T18:05:09.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faad9842700 1 --2- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faad40c7ae0 0x7faad414a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faad9842700 1 -- 172.21.15.40:0/3791890616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faad40cb770 con 0x7faad40c7ae0 2024-09-13T18:05:09.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faad9842700 1 --2- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faad40c7ae0 0x7faad414a080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faad40c6630 tx=0x7faac4007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faacaffd700 1 -- 172.21.15.40:0/3791890616 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac4012890 con 0x7faad40c7ae0 2024-09-13T18:05:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.449+0000 7faacaffd700 1 -- 172.21.15.40:0/3791890616 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faad0035030 con 0x7faac0003a20 2024-09-13T18:05:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.453+0000 7faacaffd700 1 --2- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faab80667d0 0x7faab8068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.453+0000 7faacaffd700 1 -- 172.21.15.40:0/3791890616 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7faad0096cc0 con 0x7faac0003a20 2024-09-13T18:05:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.453+0000 7faada844700 1 --2- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faab80667d0 0x7faab8068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.453+0000 7faacaffd700 1 -- 172.21.15.40:0/3791890616 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7faac40129f0 con 0x7faad40c7ae0 2024-09-13T18:05:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.453+0000 7faacaffd700 1 -- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faad40c7ae0 msgr2=0x7faad414a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.453+0000 7faacaffd700 1 --2- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faad40c7ae0 0x7faad414a080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faad40c6630 tx=0x7faac4007680 comp rx=0 tx=0).stop 2024-09-13T18:05:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.453+0000 7faacaffd700 1 -- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faad40c7ae0 msgr2=0x7faad414a080 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:05:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.453+0000 7faacaffd700 1 --2- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faad40c7ae0 0x7faad414a080 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.453+0000 7faada844700 1 --2- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faab80667d0 0x7faab8068c90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7faad40ccb60 tx=0x7faacc00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faadb045700 1 --2- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faad40c7ec0 0x7faad414a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faadb045700 1 -- 172.21.15.40:0/3791890616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad4154fe0 con 0x7faad40c7ec0 2024-09-13T18:05:09.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faad9842700 1 --2- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faad40c7ec0 0x7faad414a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faad9842700 1 -- 172.21.15.40:0/3791890616 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7faad40cb490 con 0x7faad40c7ec0 2024-09-13T18:05:09.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faad9842700 1 --2- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faad40c7ec0 0x7faad414a5c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faad40c7160 tx=0x7faac4001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faacaffd700 1 -- 172.21.15.40:0/3791890616 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac4024070 con 0x7faad40c7ec0 2024-09-13T18:05:09.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faacaffd700 1 -- 172.21.15.40:0/3791890616 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4453 (secure 0 0 0) 0x7faac4015900 con 0x7faad40c7ec0 2024-09-13T18:05:09.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faacaffd700 1 -- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faad40c7ec0 msgr2=0x7faad414a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faacaffd700 1 --2- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faad40c7ec0 0x7faad414a5c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faad40c7160 tx=0x7faac4001b90 comp rx=0 tx=0).stop 2024-09-13T18:05:09.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faacaffd700 1 -- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faad40c7ec0 msgr2=0x7faad414a5c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:05:09.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faacaffd700 1 --2- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faad40c7ec0 0x7faad414a5c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.469 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:05:09.469 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:05:09.469 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:05:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:05:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 118, 2024-09-13T18:05:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:05:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:05:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:05:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:05:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 11, 2024-09-13T18:05:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:05:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:05:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:05:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:09.470 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:05:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:05:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:09.471 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:05:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:05:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:05:09.472 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:05:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:05:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:05:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:05:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:05:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:05:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:05:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:05:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:05:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:05:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:05:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:05:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:05:09.473 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:05:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:05:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.474 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:05:09.475 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.475 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:05:09.475 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:05:09.475 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:09.475 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:09.475 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.475 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:09.475 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:05:09.475 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:09.476 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.476 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.476 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:09.476 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:05:09.476 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:09.476 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.476 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:09.476 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.476 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:05:09.476 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:05:09.476 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:09.476 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:09.477 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:09.477 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.477 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.477 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:05:09.477 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:05:09.477 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:09.477 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:09.477 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.477 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:09.477 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:05:09.477 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:09.478 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.478 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.478 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:09.478 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:05:09.478 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:09.478 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.478 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:09.478 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.478 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:05:09.478 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:05:09.478 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:09.478 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:05:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:05:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:05:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.479 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:05:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:05:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:05:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:09.480 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:05:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:05:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:05:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:05:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:09.481 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:05:09.482 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.482 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:09.482 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.482 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:05:09.482 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:05:09.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faadb045700 1 -- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faab80667d0 msgr2=0x7faab8068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faadb045700 1 --2- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faab80667d0 0x7faab8068c90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7faad40ccb60 tx=0x7faacc00b040 comp rx=0 tx=0).stop 2024-09-13T18:05:09.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faadb045700 1 -- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faac0003a20 msgr2=0x7faad4155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faadb045700 1 --2- 172.21.15.40:0/3791890616 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faac0003a20 0x7faad4155310 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faad00090b0 tx=0x7faad00047e0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faada844700 1 -- 172.21.15.40:0/3791890616 reap_dead start 2024-09-13T18:05:09.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faada844700 1 -- 172.21.15.40:0/3791890616 reap_dead start 2024-09-13T18:05:09.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faadb045700 1 -- 172.21.15.40:0/3791890616 shutdown_connections 2024-09-13T18:05:09.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faadb045700 1 -- 172.21.15.40:0/3791890616 >> 172.21.15.40:0/3791890616 conn(0x7faad4066090 msgr2=0x7faad40c66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:09.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faadb045700 1 -- 172.21.15.40:0/3791890616 shutdown_connections 2024-09-13T18:05:09.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.461+0000 7faadb045700 1 -- 172.21.15.40:0/3791890616 wait complete. 2024-09-13T18:05:09.484 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:05:09.484 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:05:09.485 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:05:09.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21b0d07700 1 Processor -- start 2024-09-13T18:05:09.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21b0d07700 1 -- start start 2024-09-13T18:05:09.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21b0d07700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21a4005b00 0x7f21a4044be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21b0d07700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f21a4045120 0x7f21a404aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21b0d07700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21a4006f90 con 0x7f21a40d4950 2024-09-13T18:05:09.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21b0d07700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a4005ee0 con 0x7f21a4005b00 2024-09-13T18:05:09.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21b0d07700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a4007420 con 0x7f21a4045120 2024-09-13T18:05:09.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21ab7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f21a40d4950 0x7f21a40db120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44562/0 (socket says 172.21.15.40:44562) 2024-09-13T18:05:09.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21ab7fe700 1 -- 172.21.15.40:0/1402598993 learned_addr learned my addr 172.21.15.40:0/1402598993 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:05:09.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21aaffd700 1 --2- 172.21.15.40:0/1402598993 >> v2:172.21.15.40:3300/0 conn(0x7f21a4005b00 0x7f21a4044be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21abfff700 1 --2- 172.21.15.40:0/1402598993 >> v2:172.21.15.43:3300/0 conn(0x7f21a4045120 0x7f21a404aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21aa7fc700 1 -- 172.21.15.40:0/1402598993 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455610842 0 0) 0x7f21a4006f90 con 0x7f21a40d4950 2024-09-13T18:05:09.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21aa7fc700 1 -- 172.21.15.40:0/1402598993 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f219c0035d0 con 0x7f21a40d4950 2024-09-13T18:05:09.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21aaffd700 1 -- 172.21.15.40:0/1402598993 >> v2:172.21.15.43:3300/0 conn(0x7f21a4045120 msgr2=0x7f21a404aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:05:09.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21aaffd700 1 --2- 172.21.15.40:0/1402598993 >> v2:172.21.15.43:3300/0 conn(0x7f21a4045120 0x7f21a404aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21aaffd700 1 -- 172.21.15.40:0/1402598993 >> v1:172.21.15.40:6789/0 conn(0x7f21a40d4950 legacy=0x7f21a40db120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21aaffd700 1 -- 172.21.15.40:0/1402598993 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a40060e0 con 0x7f21a4005b00 2024-09-13T18:05:09.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21aaffd700 1 --2- 172.21.15.40:0/1402598993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a4005b00 0x7f21a4044be0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f21a40dedd0 tx=0x7f219800ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8698f45d2d448f7d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21aa7fc700 1 -- 172.21.15.40:0/1402598993 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219800cb30 con 0x7f21a4005b00 2024-09-13T18:05:09.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21aa7fc700 1 -- 172.21.15.40:0/1402598993 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219800ae90 con 0x7f21a4005b00 2024-09-13T18:05:09.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21aa7fc700 1 -- 172.21.15.40:0/1402598993 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198018400 con 0x7f21a4005b00 2024-09-13T18:05:09.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21b0d07700 1 -- 172.21.15.40:0/1402598993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a4005b00 msgr2=0x7f21a4044be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21b0d07700 1 --2- 172.21.15.40:0/1402598993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a4005b00 0x7f21a4044be0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f21a40dedd0 tx=0x7f219800ad00 comp rx=0 tx=0).stop 2024-09-13T18:05:09.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21b0d07700 1 -- 172.21.15.40:0/1402598993 shutdown_connections 2024-09-13T18:05:09.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21b0d07700 1 --2- 172.21.15.40:0/1402598993 >> v2:172.21.15.43:3300/0 conn(0x7f21a4045120 0x7f21a404aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21b0d07700 1 --2- 172.21.15.40:0/1402598993 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a4005b00 0x7f21a4044be0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21b0d07700 1 -- 172.21.15.40:0/1402598993 >> 172.21.15.40:0/1402598993 conn(0x7f21a404fd00 msgr2=0x7f21a4050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:09.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21b0d07700 1 -- 172.21.15.40:0/1402598993 shutdown_connections 2024-09-13T18:05:09.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.597+0000 7f21b0d07700 1 -- 172.21.15.40:0/1402598993 wait complete. 2024-09-13T18:05:09.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21b0d07700 1 Processor -- start 2024-09-13T18:05:09.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21b0d07700 1 -- start start 2024-09-13T18:05:09.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21b0d07700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21a4005b00 0x7f21a40ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21b0d07700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a4045120 0x7f21a4177db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21b0d07700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21a40d4950 0x7f21a41782f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21b0d07700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a4177920 con 0x7f21a4045120 2024-09-13T18:05:09.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21b0d07700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a40dc740 con 0x7f21a4005b00 2024-09-13T18:05:09.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21b0d07700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a40dc9e0 con 0x7f21a40d4950 2024-09-13T18:05:09.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21aaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a4045120 0x7f21a4177db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21aaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a4045120 0x7f21a4177db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38928/0 (socket says 172.21.15.40:38928) 2024-09-13T18:05:09.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21aaffd700 1 -- 172.21.15.40:0/1036294488 learned_addr learned my addr 172.21.15.40:0/1036294488 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:09.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21abfff700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21a40d4950 0x7f21a41782f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21ab7fe700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21a4005b00 0x7f21a40ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21aaffd700 1 -- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21a40d4950 msgr2=0x7f21a41782f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21aaffd700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21a40d4950 0x7f21a41782f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21aaffd700 1 -- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21a4005b00 msgr2=0x7f21a40ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21aaffd700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21a4005b00 0x7f21a40ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21aaffd700 1 -- 172.21.15.40:0/1036294488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219800a650 con 0x7f21a4045120 2024-09-13T18:05:09.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21abfff700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21a40d4950 0x7f21a41782f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:05:09.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21ab7fe700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21a4005b00 0x7f21a40ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:05:09.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21aaffd700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a4045120 0x7f21a4177db0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f219800a1c0 tx=0x7f2198004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f2197fff700 1 -- 172.21.15.40:0/1036294488 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198003ad0 con 0x7f21a4045120 2024-09-13T18:05:09.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f2197fff700 1 -- 172.21.15.40:0/1036294488 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2198003c50 con 0x7f21a4045120 2024-09-13T18:05:09.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.601+0000 7f21b0d07700 1 -- 172.21.15.40:0/1036294488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a40dccc0 con 0x7f21a4045120 2024-09-13T18:05:09.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.605+0000 7f2197fff700 1 -- 172.21.15.40:0/1036294488 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21980045f0 con 0x7f21a4045120 2024-09-13T18:05:09.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.605+0000 7f21b0d07700 1 -- 172.21.15.40:0/1036294488 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a404eb30 con 0x7f21a4045120 2024-09-13T18:05:09.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.605+0000 7f21b0d07700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21a404f890 0x7f21a40c7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.605+0000 7f21b0d07700 1 -- 172.21.15.40:0/1036294488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a40084b0 con 0x7f21a404f890 2024-09-13T18:05:09.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.605+0000 7f21ab7fe700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21a404f890 0x7f21a40c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.605+0000 7f2197fff700 1 -- 172.21.15.40:0/1036294488 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f219802c450 con 0x7f21a4045120 2024-09-13T18:05:09.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.605+0000 7f21ab7fe700 1 -- 172.21.15.40:0/1036294488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f21a4006940 con 0x7f21a404f890 2024-09-13T18:05:09.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.605+0000 7f2197fff700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2190066730 0x7f2190068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.605+0000 7f2197fff700 1 -- 172.21.15.40:0/1036294488 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2198085b20 con 0x7f21a4045120 2024-09-13T18:05:09.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.605+0000 7f21abfff700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2190066730 0x7f2190068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.605+0000 7f21ab7fe700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21a404f890 0x7f21a40c7410 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f21a0000f60 tx=0x7f21a000ea90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.605+0000 7f2197fff700 1 -- 172.21.15.40:0/1036294488 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0005b70 con 0x7f21a404f890 2024-09-13T18:05:09.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.605+0000 7f21abfff700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2190066730 0x7f2190068bf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0641b0 tx=0x7f21ac06e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.605+0000 7f2197fff700 1 -- 172.21.15.40:0/1036294488 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f21a0005cd0 con 0x7f21a404f890 2024-09-13T18:05:09.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.605+0000 7f2197fff700 1 -- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21a404f890 msgr2=0x7f21a40c7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.605+0000 7f2197fff700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21a404f890 0x7f21a40c7410 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f21a0000f60 tx=0x7f21a000ea90 comp rx=0 tx=0).stop 2024-09-13T18:05:09.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.605+0000 7f2197fff700 1 -- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21a404f890 msgr2=0x7f21a40c7410 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:05:09.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.605+0000 7f2197fff700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21a404f890 0x7f21a40c7410 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.617+0000 7f21b0d07700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21a4049570 0x7f21a40c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.617+0000 7f21b0d07700 1 -- 172.21.15.40:0/1036294488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a4006660 con 0x7f21a4049570 2024-09-13T18:05:09.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.617+0000 7f21ab7fe700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21a4049570 0x7f21a40c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.617+0000 7f21ab7fe700 1 -- 172.21.15.40:0/1036294488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f21a4006c20 con 0x7f21a4049570 2024-09-13T18:05:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.617+0000 7f21ab7fe700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21a4049570 0x7f21a40c7950 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f21a0000f60 tx=0x7f21a0001150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.621+0000 7f2197fff700 1 -- 172.21.15.40:0/1036294488 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a0021070 con 0x7f21a4049570 2024-09-13T18:05:09.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.621+0000 7f2197fff700 1 -- 172.21.15.40:0/1036294488 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4949 (secure 0 0 0) 0x7f21a001c8e0 con 0x7f21a4049570 2024-09-13T18:05:09.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.621+0000 7f2197fff700 1 -- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21a4049570 msgr2=0x7f21a40c7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.621+0000 7f2197fff700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21a4049570 0x7f21a40c7950 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f21a0000f60 tx=0x7f21a0001150 comp rx=0 tx=0).stop 2024-09-13T18:05:09.628 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:05:09.628 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:05:09.628 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:05:09.628 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:05:09.628 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 118, 2024-09-13T18:05:09.628 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:05:09.628 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:05:09.628 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:05:09.629 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:05:09.629 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:09.629 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 11, 2024-09-13T18:05:09.629 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:05:09.629 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:05:09.629 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:05:09.629 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:09.629 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:09.629 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:09.629 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:09.629 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:09.629 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:09.629 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:09.629 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:09.630 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:09.630 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:09.630 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:09.630 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:05:09.630 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:05:09.630 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:09.630 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:09.630 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:09.630 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:09.630 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:09.631 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:09.631 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:09.631 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:09.631 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:09.631 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:09.631 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:09.631 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.631 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:05:09.631 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:05:09.631 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:05:09.631 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:05:09.631 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:05:09.631 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:05:09.631 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:05:09.632 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:05:09.632 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:05:09.632 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:05:09.632 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:05:09.632 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:05:09.632 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:05:09.632 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:05:09.632 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:05:09.632 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:05:09.632 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:05:09.632 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:09.632 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:09.632 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:09.632 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:09.633 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:09.633 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:09.633 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:09.633 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:09.633 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:09.633 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:09.633 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:09.633 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:05:09.633 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.633 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:05:09.633 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.633 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:05:09.633 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:05:09.633 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:09.634 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:09.634 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.634 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:09.634 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:05:09.634 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:09.634 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.634 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.634 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:09.634 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:05:09.634 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:09.634 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.634 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:09.634 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.634 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:05:09.635 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:05:09.635 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:09.635 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:09.635 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:09.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.635 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.635 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:05:09.635 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:05:09.635 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:09.635 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:09.635 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.635 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:09.635 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:05:09.636 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:09.636 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.636 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.636 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:09.636 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:05:09.636 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:09.636 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.636 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:09.636 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.636 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:05:09.636 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:05:09.636 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:09.636 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:09.636 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:09.637 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.637 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:05:09.637 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:05:09.637 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:09.637 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:09.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.637 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:09.637 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:05:09.637 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:09.637 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.637 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:09.637 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:05:09.637 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:09.638 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.638 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:09.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.638 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:05:09.638 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:05:09.638 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:09.638 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:09.638 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:09.638 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.638 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:09.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.638 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:05:09.638 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:05:09.638 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.638 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:09.639 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:09.639 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:05:09.639 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.639 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:09.639 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:05:09.639 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.639 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:09.639 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:09.639 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:05:09.639 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.639 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:09.639 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:05:09.639 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.639 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:09.640 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:09.640 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:05:09.640 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.640 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:09.640 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:05:09.640 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.640 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:09.640 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:09.640 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:05:09.640 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.640 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:09.640 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:05:09.640 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.641 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:09.641 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:09.641 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:05:09.641 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.641 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:09.641 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.641 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:05:09.641 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:05:09.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.621+0000 7f2197fff700 1 -- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21a4049570 msgr2=0x7f21a40c7950 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:05:09.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.621+0000 7f2197fff700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21a4049570 0x7f21a40c7950 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.621+0000 7f21b0d07700 1 -- 172.21.15.40:0/1036294488 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2190066730 msgr2=0x7f2190068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.621+0000 7f21b0d07700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2190066730 0x7f2190068bf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0641b0 tx=0x7f21ac06e040 comp rx=0 tx=0).stop 2024-09-13T18:05:09.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.621+0000 7f21b0d07700 1 -- 172.21.15.40:0/1036294488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a4045120 msgr2=0x7f21a4177db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.621+0000 7f21b0d07700 1 --2- 172.21.15.40:0/1036294488 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21a4045120 0x7f21a4177db0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f219800a1c0 tx=0x7f2198004770 comp rx=0 tx=0).stop 2024-09-13T18:05:09.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.621+0000 7f21abfff700 1 -- 172.21.15.40:0/1036294488 reap_dead start 2024-09-13T18:05:09.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.621+0000 7f21abfff700 1 -- 172.21.15.40:0/1036294488 reap_dead start 2024-09-13T18:05:09.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.621+0000 7f21b0d07700 1 -- 172.21.15.40:0/1036294488 shutdown_connections 2024-09-13T18:05:09.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.621+0000 7f21b0d07700 1 -- 172.21.15.40:0/1036294488 >> 172.21.15.40:0/1036294488 conn(0x7f21a404fd00 msgr2=0x7f21a40500e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:09.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.621+0000 7f21b0d07700 1 -- 172.21.15.40:0/1036294488 shutdown_connections 2024-09-13T18:05:09.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.621+0000 7f21b0d07700 1 -- 172.21.15.40:0/1036294488 wait complete. 2024-09-13T18:05:09.643 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:05:09.644 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:05:09.644 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:05:09.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.749+0000 7f8ff2dcc700 1 Processor -- start 2024-09-13T18:05:09.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.749+0000 7f8ff2dcc700 1 -- start start 2024-09-13T18:05:09.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.749+0000 7f8ff2dcc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8fec0cf580 0x7f8fec15a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.749+0000 7f8ff2dcc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8fec0cc250 0x7f8fec0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.749+0000 7f8ff2dcc700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fec15a7a0 con 0x7f8fec0cf1a0 2024-09-13T18:05:09.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.749+0000 7f8ff2dcc700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fec0cbdc0 con 0x7f8fec0cc250 2024-09-13T18:05:09.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.749+0000 7f8ff2dcc700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fec153f80 con 0x7f8fec0cf580 2024-09-13T18:05:09.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.749+0000 7f8ff25cb700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8fec0cc250 0x7f8fec0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.749+0000 7f8ff25cb700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8fec0cc250 0x7f8fec0c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38944/0 (socket says 172.21.15.40:38944) 2024-09-13T18:05:09.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.749+0000 7f8ff15c9700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8fec0cf580 0x7f8fec15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.749+0000 7f8ff25cb700 1 -- 172.21.15.40:0/1155993545 learned_addr learned my addr 172.21.15.40:0/1155993545 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:09.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.749+0000 7f8ff25cb700 1 -- 172.21.15.40:0/1155993545 >> v2:172.21.15.43:3300/0 conn(0x7f8fec0cf580 msgr2=0x7f8fec15a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:05:09.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.749+0000 7f8ff25cb700 1 --2- 172.21.15.40:0/1155993545 >> v2:172.21.15.43:3300/0 conn(0x7f8fec0cf580 0x7f8fec15a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.749+0000 7f8ff25cb700 1 -- 172.21.15.40:0/1155993545 >> v1:172.21.15.43:6789/0 conn(0x7f8fec0cf1a0 legacy=0x7f8fec0c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.749+0000 7f8ff25cb700 1 -- 172.21.15.40:0/1155993545 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec0caed0 con 0x7f8fec0cc250 2024-09-13T18:05:09.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.749+0000 7f8ff25cb700 1 --2- 172.21.15.40:0/1155993545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fec0cc250 0x7f8fec0c8aa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8fec08d080 tx=0x7f8fe400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e517cdd96ab877 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.753+0000 7f8ff0dc8700 1 -- 172.21.15.40:0/1155993545 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400e070 con 0x7f8fec0cc250 2024-09-13T18:05:09.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.753+0000 7f8ff0dc8700 1 -- 172.21.15.40:0/1155993545 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe400b910 con 0x7f8fec0cc250 2024-09-13T18:05:09.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.753+0000 7f8ff0dc8700 1 -- 172.21.15.40:0/1155993545 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400bcc0 con 0x7f8fec0cc250 2024-09-13T18:05:09.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.753+0000 7f8ff2dcc700 1 -- 172.21.15.40:0/1155993545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fec0cc250 msgr2=0x7f8fec0c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.753+0000 7f8ff2dcc700 1 --2- 172.21.15.40:0/1155993545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fec0cc250 0x7f8fec0c8aa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8fec08d080 tx=0x7f8fe400b330 comp rx=0 tx=0).stop 2024-09-13T18:05:09.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.753+0000 7f8ff2dcc700 1 -- 172.21.15.40:0/1155993545 shutdown_connections 2024-09-13T18:05:09.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.753+0000 7f8ff2dcc700 1 --2- 172.21.15.40:0/1155993545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fec0cc250 0x7f8fec0c8aa0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.753+0000 7f8ff2dcc700 1 --2- 172.21.15.40:0/1155993545 >> v2:172.21.15.43:3300/0 conn(0x7f8fec0cf580 0x7f8fec15a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.753+0000 7f8ff2dcc700 1 -- 172.21.15.40:0/1155993545 >> 172.21.15.40:0/1155993545 conn(0x7f8fec066090 msgr2=0x7f8fec066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:09.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.753+0000 7f8ff2dcc700 1 -- 172.21.15.40:0/1155993545 shutdown_connections 2024-09-13T18:05:09.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.753+0000 7f8ff2dcc700 1 -- 172.21.15.40:0/1155993545 wait complete. 2024-09-13T18:05:09.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.753+0000 7f8ff2dcc700 1 Processor -- start 2024-09-13T18:05:09.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.753+0000 7f8ff2dcc700 1 -- start start 2024-09-13T18:05:09.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8ff2dcc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fec0cc250 0x7f8fec158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8ff2dcc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fec0cf1a0 0x7f8fec1543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8ff2dcc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fec0cf580 0x7f8fec155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8ff2dcc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec1552d0 con 0x7f8fec0cf580 2024-09-13T18:05:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8ff2dcc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec157a70 con 0x7f8fec0cf1a0 2024-09-13T18:05:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8ff2dcc700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec158ac0 con 0x7f8fec0cc250 2024-09-13T18:05:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8ff25cb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fec0cf580 0x7f8fec155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8ff25cb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fec0cf580 0x7f8fec155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38946/0 (socket says 172.21.15.40:38946) 2024-09-13T18:05:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8ff25cb700 1 -- 172.21.15.40:0/1103576813 learned_addr learned my addr 172.21.15.40:0/1103576813 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8ff15c9700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fec0cf1a0 0x7f8fec1543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8ff1dca700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fec0cc250 0x7f8fec158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8ff25cb700 1 -- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fec0cc250 msgr2=0x7f8fec158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8ff25cb700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fec0cc250 0x7f8fec158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8ff25cb700 1 -- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fec0cf1a0 msgr2=0x7f8fec1543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8ff25cb700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fec0cf1a0 0x7f8fec1543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8ff25cb700 1 -- 172.21.15.40:0/1103576813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe4009b30 con 0x7f8fec0cf580 2024-09-13T18:05:09.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8ff1dca700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fec0cc250 0x7f8fec158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:05:09.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8ff25cb700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fec0cf580 0x7f8fec155720 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8fec08c270 tx=0x7f8fe4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8fe27fc700 1 -- 172.21.15.40:0/1103576813 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4004930 con 0x7f8fec0cf580 2024-09-13T18:05:09.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8fe27fc700 1 -- 172.21.15.40:0/1103576813 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe40092e0 con 0x7f8fec0cf580 2024-09-13T18:05:09.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8ff2dcc700 1 -- 172.21.15.40:0/1103576813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fec157090 con 0x7f8fec0cf580 2024-09-13T18:05:09.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8fe27fc700 1 -- 172.21.15.40:0/1103576813 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe401c660 con 0x7f8fec0cf580 2024-09-13T18:05:09.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.757+0000 7f8ff2dcc700 1 -- 172.21.15.40:0/1103576813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fec1567f0 con 0x7f8fec0cf580 2024-09-13T18:05:09.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.761+0000 7f8ff2dcc700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fec157440 0x7f8fec156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.761+0000 7f8ff2dcc700 1 -- 172.21.15.40:0/1103576813 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec074750 con 0x7f8fec157440 2024-09-13T18:05:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.761+0000 7f8ff1dca700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fec157440 0x7f8fec156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.761+0000 7f8fe27fc700 1 -- 172.21.15.40:0/1103576813 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8fe401c7c0 con 0x7f8fec0cf580 2024-09-13T18:05:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.761+0000 7f8ff1dca700 1 -- 172.21.15.40:0/1103576813 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8fec0cb770 con 0x7f8fec157440 2024-09-13T18:05:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.761+0000 7f8ff1dca700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fec157440 0x7f8fec156ad0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc000f60 tx=0x7f8fdc00ea90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.761+0000 7f8fe27fc700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8fd0066730 0x7f8fd0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.761+0000 7f8ff15c9700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8fd0066730 0x7f8fd0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.761+0000 7f8fe27fc700 1 -- 172.21.15.40:0/1103576813 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8fe4085e80 con 0x7f8fec0cf580 2024-09-13T18:05:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.761+0000 7f8fe27fc700 1 -- 172.21.15.40:0/1103576813 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc0052e0 con 0x7f8fec157440 2024-09-13T18:05:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.761+0000 7f8fe27fc700 1 -- 172.21.15.40:0/1103576813 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8fdc005440 con 0x7f8fec157440 2024-09-13T18:05:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.761+0000 7f8fe27fc700 1 -- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fec157440 msgr2=0x7f8fec156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.761+0000 7f8fe27fc700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fec157440 0x7f8fec156ad0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc000f60 tx=0x7f8fdc00ea90 comp rx=0 tx=0).stop 2024-09-13T18:05:09.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.761+0000 7f8fe27fc700 1 -- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fec157440 msgr2=0x7f8fec156ad0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:05:09.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.761+0000 7f8fe27fc700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fec157440 0x7f8fec156ad0 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc000f60 tx=0x7f8fdc00ea90 comp rx=0 tx=0).stop 2024-09-13T18:05:09.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.761+0000 7f8ff15c9700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8fd0066730 0x7f8fd0068bf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0ce480 tx=0x7f8fe8009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.773+0000 7f8ff2dcc700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fec14d300 0x7f8fec14d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.773+0000 7f8ff2dcc700 1 -- 172.21.15.40:0/1103576813 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0cb490 con 0x7f8fec14d300 2024-09-13T18:05:09.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.773+0000 7f8ff1dca700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fec14d300 0x7f8fec14d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.773+0000 7f8ff1dca700 1 -- 172.21.15.40:0/1103576813 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8fec0cba50 con 0x7f8fec14d300 2024-09-13T18:05:09.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.773+0000 7f8ff1dca700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fec14d300 0x7f8fec14d710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc000f60 tx=0x7f8fdc001120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.777+0000 7f8fe27fc700 1 -- 172.21.15.40:0/1103576813 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc021070 con 0x7f8fec14d300 2024-09-13T18:05:09.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.777+0000 7f8fe27fc700 1 -- 172.21.15.40:0/1103576813 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4949 (secure 0 0 0) 0x7f8fdc006e40 con 0x7f8fec14d300 2024-09-13T18:05:09.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.777+0000 7f8fe27fc700 1 -- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fec14d300 msgr2=0x7f8fec14d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.777+0000 7f8fe27fc700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fec14d300 0x7f8fec14d710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc000f60 tx=0x7f8fdc001120 comp rx=0 tx=0).stop 2024-09-13T18:05:09.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.777+0000 7f8fe27fc700 1 -- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fec14d300 msgr2=0x7f8fec14d710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:05:09.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.777+0000 7f8fe27fc700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fec14d300 0x7f8fec14d710 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc000f60 tx=0x7f8fdc001120 comp rx=0 tx=0).stop 2024-09-13T18:05:09.784 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:05:09.784 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:05:09.784 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:05:09.784 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:05:09.784 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 118, 2024-09-13T18:05:09.784 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:05:09.784 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:05:09.785 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:05:09.785 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:05:09.785 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:09.785 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 12, 2024-09-13T18:05:09.785 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:05:09.785 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:05:09.785 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:05:09.785 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:09.785 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:09.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:09.785 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:09.785 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:09.785 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:09.786 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:09.786 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:09.786 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:09.786 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:09.786 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:09.786 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:05:09.786 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:05:09.786 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:09.786 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:09.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:09.786 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:09.786 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:09.786 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:09.787 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:09.787 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:09.787 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:09.787 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:09.787 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:09.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.787 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:05:09.787 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:05:09.787 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:05:09.787 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:05:09.787 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:05:09.787 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:05:09.788 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:05:09.788 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:05:09.788 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:05:09.788 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:05:09.788 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:05:09.788 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:05:09.788 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:05:09.788 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:05:09.788 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:05:09.788 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:05:09.788 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:05:09.788 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:09.788 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:09.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:09.789 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:09.789 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:09.789 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:09.789 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:09.789 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:09.789 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:09.789 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:09.789 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:09.789 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:05:09.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:05:09.789 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.790 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:05:09.790 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:05:09.790 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:09.790 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:09.790 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.790 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:09.790 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:05:09.790 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:09.790 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.790 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.790 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:09.790 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:05:09.790 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:09.790 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.791 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:09.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.791 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:05:09.791 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:05:09.791 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:09.791 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:09.791 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:09.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.791 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.792 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:05:09.792 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:05:09.792 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:09.792 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:09.792 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.792 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:09.792 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:05:09.792 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:09.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.792 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.793 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:09.793 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:05:09.793 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:09.793 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.793 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:09.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.793 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:05:09.793 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:05:09.793 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:09.793 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:09.793 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:09.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.793 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.794 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:05:09.794 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:05:09.794 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:09.794 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:09.794 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.794 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:09.794 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:05:09.794 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:09.794 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.794 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.794 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:09.794 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:05:09.794 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:09.794 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.795 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:09.795 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.795 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:05:09.795 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:05:09.795 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:09.795 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:09.795 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:09.795 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.795 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:09.795 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.795 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:05:09.795 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:05:09.795 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.795 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:09.796 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:09.796 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:05:09.796 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.796 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:09.796 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:05:09.796 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.796 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:09.796 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:09.796 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:05:09.796 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.796 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:09.796 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:05:09.796 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.797 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:09.797 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:09.797 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:05:09.797 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.797 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:09.797 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:05:09.797 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.797 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:09.797 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:09.797 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:05:09.797 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.797 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:09.797 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:05:09.798 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:09.798 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:09.798 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:09.798 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:05:09.798 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:09.798 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:09.798 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:09.798 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:05:09.798 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:05:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.777+0000 7f8ff2dcc700 1 -- 172.21.15.40:0/1103576813 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8fd0066730 msgr2=0x7f8fd0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.777+0000 7f8ff2dcc700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8fd0066730 0x7f8fd0068bf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0ce480 tx=0x7f8fe8009240 comp rx=0 tx=0).stop 2024-09-13T18:05:09.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.777+0000 7f8ff25cb700 1 -- 172.21.15.40:0/1103576813 reap_dead start 2024-09-13T18:05:09.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.777+0000 7f8ff2dcc700 1 -- 172.21.15.40:0/1103576813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fec0cf580 msgr2=0x7f8fec155720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.777+0000 7f8ff2dcc700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fec0cf580 0x7f8fec155720 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8fec08c270 tx=0x7f8fe4004af0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.777+0000 7f8ff2dcc700 1 -- 172.21.15.40:0/1103576813 shutdown_connections 2024-09-13T18:05:09.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.777+0000 7f8ff2dcc700 1 --2- 172.21.15.40:0/1103576813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8fec0cf580 0x7f8fec155720 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.777+0000 7f8ff2dcc700 1 -- 172.21.15.40:0/1103576813 >> 172.21.15.40:0/1103576813 conn(0x7f8fec066090 msgr2=0x7f8fec0c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:09.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.777+0000 7f8ff2dcc700 1 -- 172.21.15.40:0/1103576813 shutdown_connections 2024-09-13T18:05:09.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.777+0000 7f8ff2dcc700 1 -- 172.21.15.40:0/1103576813 wait complete. 2024-09-13T18:05:09.800 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:05:09.801 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-13T18:05:09.801 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T18:05:09.801 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-13T18:05:09.801 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-13T18:05:09.801 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T18:05:09.801 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:05:09.849 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T18:05:09.849 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T18:05:09.849 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:05:09.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.953+0000 7fe6b759e700 1 Processor -- start 2024-09-13T18:05:09.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.953+0000 7fe6b759e700 1 -- start start 2024-09-13T18:05:09.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.953+0000 7fe6b759e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe6b004ca90 0x7fe6b0007560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.953+0000 7fe6b759e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b0005fa0 con 0x7fe6b00d69c0 2024-09-13T18:05:09.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.953+0000 7fe6b759e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b0036080 con 0x7fe6b0007ba0 2024-09-13T18:05:09.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.953+0000 7fe6b759e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b00368b0 con 0x7fe6b004ca90 2024-09-13T18:05:09.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.953+0000 7fe6b5d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe6b004ca90 0x7fe6b0007560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.953+0000 7fe6b659c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe6b00d69c0 0x7fe6b0033e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44574/0 (socket says 172.21.15.40:44574) 2024-09-13T18:05:09.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.953+0000 7fe6b659c700 1 -- 172.21.15.40:0/63695948 learned_addr learned my addr 172.21.15.40:0/63695948 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:05:09.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.953+0000 7fe6b6d9d700 1 -- 172.21.15.40:0/63695948 >> v1:172.21.15.43:6789/0 conn(0x7fe6b0007ba0 legacy=0x7fe6b00dc960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:05:09.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.953+0000 7fe6b5d9b700 1 -- 172.21.15.40:0/63695948 >> v1:172.21.15.43:6789/0 conn(0x7fe6b0007ba0 legacy=0x7fe6b00dc960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:05:09.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.953+0000 7fe6b5d9b700 1 -- 172.21.15.40:0/63695948 >> v1:172.21.15.40:6789/0 conn(0x7fe6b00d69c0 legacy=0x7fe6b0033e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.953+0000 7fe6b5d9b700 1 -- 172.21.15.40:0/63695948 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b0034b20 con 0x7fe6b004ca90 2024-09-13T18:05:09.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.953+0000 7fe6b559a700 1 -- 172.21.15.40:0/63695948 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 91622587 0 0) 0x7fe6b0005fa0 con 0x7fe6b00d69c0 2024-09-13T18:05:09.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.957+0000 7fe6b5d9b700 1 --2- 172.21.15.40:0/63695948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6b004ca90 0x7fe6b0007560 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009850 tx=0x7fe6ac009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae8d922ee6c0a5f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.957+0000 7fe6b559a700 1 -- 172.21.15.40:0/63695948 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac004230 con 0x7fe6b004ca90 2024-09-13T18:05:09.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.957+0000 7fe6b559a700 1 -- 172.21.15.40:0/63695948 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ac004390 con 0x7fe6b004ca90 2024-09-13T18:05:09.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.957+0000 7fe6b559a700 1 -- 172.21.15.40:0/63695948 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac0036c0 con 0x7fe6b004ca90 2024-09-13T18:05:09.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.957+0000 7fe6b759e700 1 -- 172.21.15.40:0/63695948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6b004ca90 msgr2=0x7fe6b0007560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.957+0000 7fe6b759e700 1 --2- 172.21.15.40:0/63695948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6b004ca90 0x7fe6b0007560 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009850 tx=0x7fe6ac009b60 comp rx=0 tx=0).stop 2024-09-13T18:05:09.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.957+0000 7fe6b759e700 1 -- 172.21.15.40:0/63695948 shutdown_connections 2024-09-13T18:05:09.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.957+0000 7fe6b759e700 1 --2- 172.21.15.40:0/63695948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6b004ca90 0x7fe6b0007560 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.957+0000 7fe6b759e700 1 -- 172.21.15.40:0/63695948 >> 172.21.15.40:0/63695948 conn(0x7fe6b00505e0 msgr2=0x7fe6b00509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:09.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.957+0000 7fe6b759e700 1 -- 172.21.15.40:0/63695948 shutdown_connections 2024-09-13T18:05:09.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.957+0000 7fe6b759e700 1 -- 172.21.15.40:0/63695948 wait complete. 2024-09-13T18:05:09.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b759e700 1 Processor -- start 2024-09-13T18:05:09.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b759e700 1 -- start start 2024-09-13T18:05:09.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe6b0007ba0 0x7fe6b00d2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b004ca90 0x7fe6b004ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6b00d69c0 0x7fe6b004efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b00d30f0 con 0x7fe6b00d69c0 2024-09-13T18:05:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b0047b90 con 0x7fe6b004ca90 2024-09-13T18:05:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b004a5f0 con 0x7fe6b0007ba0 2024-09-13T18:05:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6b00d69c0 0x7fe6b004efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b5d9b700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b004ca90 msgr2=0x7fe6b004ea60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:05:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b5d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b004ca90 0x7fe6b004ea60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6b00d69c0 0x7fe6b004efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39046/0 (socket says 172.21.15.40:39046) 2024-09-13T18:05:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b6d9d700 1 -- 172.21.15.40:0/2864718782 learned_addr learned my addr 172.21.15.40:0/2864718782 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b659c700 1 --2- 172.21.15.40:0/2864718782 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe6b0007ba0 0x7fe6b00d2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b6d9d700 1 -- 172.21.15.40:0/2864718782 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe6b0007ba0 msgr2=0x7fe6b00d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:09.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b6d9d700 1 --2- 172.21.15.40:0/2864718782 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe6b0007ba0 0x7fe6b00d2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b6d9d700 1 -- 172.21.15.40:0/2864718782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b004ca90 msgr2=0x7fe6b004ea60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:05:09.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b6d9d700 1 --2- 172.21.15.40:0/2864718782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b004ca90 0x7fe6b004ea60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:09.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b6d9d700 1 -- 172.21.15.40:0/2864718782 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ac0094b0 con 0x7fe6b00d69c0 2024-09-13T18:05:09.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b6d9d700 1 --2- 172.21.15.40:0/2864718782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6b00d69c0 0x7fe6b004efa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80640f0 tx=0x7fe6b80680b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6a6ffd700 1 -- 172.21.15.40:0/2864718782 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b80677f0 con 0x7fe6b00d69c0 2024-09-13T18:05:09.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6a6ffd700 1 -- 172.21.15.40:0/2864718782 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b80720c0 con 0x7fe6b00d69c0 2024-09-13T18:05:09.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b759e700 1 -- 172.21.15.40:0/2864718782 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b0049ec0 con 0x7fe6b00d69c0 2024-09-13T18:05:09.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6a6ffd700 1 -- 172.21.15.40:0/2864718782 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b80724b0 con 0x7fe6b00d69c0 2024-09-13T18:05:09.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.961+0000 7fe6b759e700 1 -- 172.21.15.40:0/2864718782 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b0047de0 con 0x7fe6b00d69c0 2024-09-13T18:05:09.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.965+0000 7fe6b759e700 1 -- 172.21.15.40:0/2864718782 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b0007560 con 0x7fe6b00d69c0 2024-09-13T18:05:09.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.965+0000 7fe6a6ffd700 1 -- 172.21.15.40:0/2864718782 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe6b8072610 con 0x7fe6b00d69c0 2024-09-13T18:05:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.965+0000 7fe6a6ffd700 1 --2- 172.21.15.40:0/2864718782 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe6980666e0 0x7fe698068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.965+0000 7fe6a6ffd700 1 -- 172.21.15.40:0/2864718782 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe6b80fe1a0 con 0x7fe6b00d69c0 2024-09-13T18:05:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.965+0000 7fe6b659c700 1 --2- 172.21.15.40:0/2864718782 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe6980666e0 0x7fe698068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:09.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.965+0000 7fe6b659c700 1 --2- 172.21.15.40:0/2864718782 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe6980666e0 0x7fe698068ba0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80010e0 tx=0x7fe6a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:09.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:09.973+0000 7fe6a6ffd700 1 -- 172.21.15.40:0/2864718782 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe6b80cb0c0 con 0x7fe6b00d69c0 2024-09-13T18:05:10.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:10.221+0000 7fe6b759e700 1 -- 172.21.15.40:0/2864718782 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe6b0000dd0 con 0x7fe6b00d69c0 2024-09-13T18:05:10.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:10.221+0000 7fe6a6ffd700 1 -- 172.21.15.40:0/2864718782 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7fe6b80ce6e0 con 0x7fe6b00d69c0 2024-09-13T18:05:10.228 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":118,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:05:10.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:10.221+0000 7fe6b759e700 1 -- 172.21.15.40:0/2864718782 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe6980666e0 msgr2=0x7fe698068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:10.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:10.221+0000 7fe6b759e700 1 --2- 172.21.15.40:0/2864718782 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe6980666e0 0x7fe698068ba0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80010e0 tx=0x7fe6a8009040 comp rx=0 tx=0).stop 2024-09-13T18:05:10.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:10.221+0000 7fe6b759e700 1 -- 172.21.15.40:0/2864718782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6b00d69c0 msgr2=0x7fe6b004efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:10.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:10.221+0000 7fe6b759e700 1 --2- 172.21.15.40:0/2864718782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6b00d69c0 0x7fe6b004efa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80640f0 tx=0x7fe6b80680b0 comp rx=0 tx=0).stop 2024-09-13T18:05:10.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:10.221+0000 7fe6b759e700 1 -- 172.21.15.40:0/2864718782 shutdown_connections 2024-09-13T18:05:10.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:10.221+0000 7fe6b759e700 1 --2- 172.21.15.40:0/2864718782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe6b00d69c0 0x7fe6b004efa0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:10.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:10.221+0000 7fe6b759e700 1 --2- 172.21.15.40:0/2864718782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b004ca90 0x7fe6b004ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:10.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:10.221+0000 7fe6b759e700 1 --2- 172.21.15.40:0/2864718782 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe6980666e0 0x7fe698068ba0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:10.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:10.221+0000 7fe6b759e700 1 --2- 172.21.15.40:0/2864718782 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe6b0007ba0 0x7fe6b00d2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:10.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:10.221+0000 7fe6b759e700 1 -- 172.21.15.40:0/2864718782 >> 172.21.15.40:0/2864718782 conn(0x7fe6b00505e0 msgr2=0x7fe6b00d54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:10.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:10.225+0000 7fe6b759e700 1 -- 172.21.15.40:0/2864718782 shutdown_connections 2024-09-13T18:05:10.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:10.225+0000 7fe6b759e700 1 -- 172.21.15.40:0/2864718782 wait complete. 2024-09-13T18:05:13.244 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:05:13.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.357+0000 7f8a75736700 1 Processor -- start 2024-09-13T18:05:13.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a75736700 1 -- start start 2024-09-13T18:05:13.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a75736700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8a680d89f0 0x7f8a680e2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:13.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a75736700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a680481a0 0x7f8a680485d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:13.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a75736700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a68007890 con 0x7f8a680085e0 2024-09-13T18:05:13.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a75736700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a680478e0 con 0x7f8a680481a0 2024-09-13T18:05:13.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a75736700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a680e2c70 con 0x7f8a680d89f0 2024-09-13T18:05:13.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a6f7fe700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7f8a680085e0 legacy=0x7f8a68008a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:05:13.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a6ffff700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7f8a680d89f0 msgr2=0x7f8a680e2730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:05:13.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a74f35700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a680481a0 0x7f8a680485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:13.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a6ffff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8a680d89f0 0x7f8a680e2730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:13.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a74f35700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8a680481a0 0x7f8a680485d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54562/0 (socket says 172.21.15.40:54562) 2024-09-13T18:05:13.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a74f35700 1 -- 172.21.15.40:0/1057249063 learned_addr learned my addr 172.21.15.40:0/1057249063 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:13.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a74f35700 1 -- 172.21.15.40:0/1057249063 >> v2:172.21.15.43:3300/0 conn(0x7f8a680d89f0 msgr2=0x7f8a680e2730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:05:13.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a74f35700 1 --2- 172.21.15.40:0/1057249063 >> v2:172.21.15.43:3300/0 conn(0x7f8a680d89f0 0x7f8a680e2730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:13.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a74f35700 1 -- 172.21.15.40:0/1057249063 >> v1:172.21.15.43:6789/0 conn(0x7f8a680085e0 legacy=0x7f8a68008a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:05:13.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a74f35700 1 -- 172.21.15.40:0/1057249063 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6804d030 con 0x7f8a680481a0 2024-09-13T18:05:13.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a74f35700 1 --2- 172.21.15.40:0/1057249063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a680481a0 0x7f8a680485d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8a70063990 tx=0x7f8a70069e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf531f325700ad8a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:13.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a6effd700 1 -- 172.21.15.40:0/1057249063 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a70068a10 con 0x7f8a680481a0 2024-09-13T18:05:13.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a6effd700 1 -- 172.21.15.40:0/1057249063 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a70067dd0 con 0x7f8a680481a0 2024-09-13T18:05:13.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a6effd700 1 -- 172.21.15.40:0/1057249063 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a700744e0 con 0x7f8a680481a0 2024-09-13T18:05:13.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a75736700 1 -- 172.21.15.40:0/1057249063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a680481a0 msgr2=0x7f8a680485d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:13.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a75736700 1 --2- 172.21.15.40:0/1057249063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a680481a0 0x7f8a680485d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8a70063990 tx=0x7f8a70069e70 comp rx=0 tx=0).stop 2024-09-13T18:05:13.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a75736700 1 -- 172.21.15.40:0/1057249063 shutdown_connections 2024-09-13T18:05:13.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a75736700 1 --2- 172.21.15.40:0/1057249063 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a680481a0 0x7f8a680485d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:13.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a75736700 1 --2- 172.21.15.40:0/1057249063 >> v2:172.21.15.43:3300/0 conn(0x7f8a680d89f0 0x7f8a680e2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:13.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a75736700 1 -- 172.21.15.40:0/1057249063 >> 172.21.15.40:0/1057249063 conn(0x7f8a6804e9b0 msgr2=0x7f8a6804edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:13.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a75736700 1 -- 172.21.15.40:0/1057249063 shutdown_connections 2024-09-13T18:05:13.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.361+0000 7f8a75736700 1 -- 172.21.15.40:0/1057249063 wait complete. 2024-09-13T18:05:13.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.365+0000 7f8a75736700 1 Processor -- start 2024-09-13T18:05:13.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.365+0000 7f8a75736700 1 -- start start 2024-09-13T18:05:13.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.365+0000 7f8a75736700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8a680085e0 0x7f8a681782a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:13.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.365+0000 7f8a75736700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a680481a0 0x7f8a68169610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:13.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.365+0000 7f8a75736700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a680d89f0 0x7f8a68031dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:13.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.365+0000 7f8a75736700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a68031930 con 0x7f8a680d89f0 2024-09-13T18:05:13.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.365+0000 7f8a75736700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a68035180 con 0x7f8a680481a0 2024-09-13T18:05:13.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.365+0000 7f8a75736700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a680353d0 con 0x7f8a680085e0 2024-09-13T18:05:13.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.365+0000 7f8a6f7fe700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a680481a0 msgr2=0x7f8a68169610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:05:13.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.365+0000 7f8a6f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a680481a0 0x7f8a68169610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:13.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.365+0000 7f8a74f35700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a680d89f0 0x7f8a68031dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:13.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.365+0000 7f8a74f35700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a680d89f0 0x7f8a68031dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54572/0 (socket says 172.21.15.40:54572) 2024-09-13T18:05:13.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.365+0000 7f8a74f35700 1 -- 172.21.15.40:0/2813284059 learned_addr learned my addr 172.21.15.40:0/2813284059 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:13.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.365+0000 7f8a6ffff700 1 --2- 172.21.15.40:0/2813284059 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8a680085e0 0x7f8a681782a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:13.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.365+0000 7f8a74f35700 1 -- 172.21.15.40:0/2813284059 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8a680085e0 msgr2=0x7f8a681782a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:13.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.365+0000 7f8a74f35700 1 --2- 172.21.15.40:0/2813284059 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8a680085e0 0x7f8a681782a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:13.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.365+0000 7f8a74f35700 1 -- 172.21.15.40:0/2813284059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a680481a0 msgr2=0x7f8a68169610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:05:13.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.365+0000 7f8a74f35700 1 --2- 172.21.15.40:0/2813284059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a680481a0 0x7f8a68169610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:13.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.365+0000 7f8a74f35700 1 -- 172.21.15.40:0/2813284059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7004f5c0 con 0x7f8a680d89f0 2024-09-13T18:05:13.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.365+0000 7f8a6ffff700 1 --2- 172.21.15.40:0/2813284059 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8a680085e0 0x7f8a681782a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:05:13.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.369+0000 7f8a74f35700 1 --2- 172.21.15.40:0/2813284059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a680d89f0 0x7f8a68031dc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8a700641b0 tx=0x7f8a70068470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:13.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.369+0000 7f8a6cff9700 1 -- 172.21.15.40:0/2813284059 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a70067890 con 0x7f8a680d89f0 2024-09-13T18:05:13.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.369+0000 7f8a6cff9700 1 -- 172.21.15.40:0/2813284059 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a70072150 con 0x7f8a680d89f0 2024-09-13T18:05:13.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.369+0000 7f8a6cff9700 1 -- 172.21.15.40:0/2813284059 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a700724e0 con 0x7f8a680d89f0 2024-09-13T18:05:13.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.369+0000 7f8a75736700 1 -- 172.21.15.40:0/2813284059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a68037e20 con 0x7f8a680d89f0 2024-09-13T18:05:13.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.369+0000 7f8a75736700 1 -- 172.21.15.40:0/2813284059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a68051160 con 0x7f8a680d89f0 2024-09-13T18:05:13.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.369+0000 7f8a75736700 1 -- 172.21.15.40:0/2813284059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a680e27c0 con 0x7f8a680d89f0 2024-09-13T18:05:13.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.369+0000 7f8a6cff9700 1 -- 172.21.15.40:0/2813284059 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8a70072640 con 0x7f8a680d89f0 2024-09-13T18:05:13.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.369+0000 7f8a6cff9700 1 --2- 172.21.15.40:0/2813284059 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8a540666e0 0x7f8a54068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:13.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.369+0000 7f8a6cff9700 1 -- 172.21.15.40:0/2813284059 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8a700ffee0 con 0x7f8a680d89f0 2024-09-13T18:05:13.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.369+0000 7f8a6ffff700 1 --2- 172.21.15.40:0/2813284059 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8a540666e0 0x7f8a54068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:13.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.373+0000 7f8a6ffff700 1 --2- 172.21.15.40:0/2813284059 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8a540666e0 0x7f8a54068ba0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8a64003960 tx=0x7f8a64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:13.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.377+0000 7f8a6cff9700 1 -- 172.21.15.40:0/2813284059 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8a700cce90 con 0x7f8a680d89f0 2024-09-13T18:05:13.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.657+0000 7f8a75736700 1 -- 172.21.15.40:0/2813284059 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8a68031ae0 con 0x7f8a680d89f0 2024-09-13T18:05:13.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.661+0000 7f8a6cff9700 1 -- 172.21.15.40:0/2813284059 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f8a68031ae0 con 0x7f8a680d89f0 2024-09-13T18:05:13.667 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":118,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:05:13.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.661+0000 7f8a75736700 1 -- 172.21.15.40:0/2813284059 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8a540666e0 msgr2=0x7f8a54068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:13.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.661+0000 7f8a75736700 1 --2- 172.21.15.40:0/2813284059 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8a540666e0 0x7f8a54068ba0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8a64003960 tx=0x7f8a64008040 comp rx=0 tx=0).stop 2024-09-13T18:05:13.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.661+0000 7f8a75736700 1 -- 172.21.15.40:0/2813284059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a680d89f0 msgr2=0x7f8a68031dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:13.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.661+0000 7f8a75736700 1 --2- 172.21.15.40:0/2813284059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a680d89f0 0x7f8a68031dc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8a700641b0 tx=0x7f8a70068470 comp rx=0 tx=0).stop 2024-09-13T18:05:13.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.661+0000 7f8a75736700 1 -- 172.21.15.40:0/2813284059 shutdown_connections 2024-09-13T18:05:13.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.661+0000 7f8a75736700 1 --2- 172.21.15.40:0/2813284059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8a680d89f0 0x7f8a68031dc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:13.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.661+0000 7f8a75736700 1 --2- 172.21.15.40:0/2813284059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a680481a0 0x7f8a68169610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:13.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.661+0000 7f8a75736700 1 --2- 172.21.15.40:0/2813284059 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8a540666e0 0x7f8a54068ba0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:13.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.661+0000 7f8a75736700 1 --2- 172.21.15.40:0/2813284059 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8a680085e0 0x7f8a681782a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:13.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.661+0000 7f8a75736700 1 -- 172.21.15.40:0/2813284059 >> 172.21.15.40:0/2813284059 conn(0x7f8a6804e9b0 msgr2=0x7f8a680513a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:13.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.661+0000 7f8a75736700 1 -- 172.21.15.40:0/2813284059 shutdown_connections 2024-09-13T18:05:13.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:13.661+0000 7f8a75736700 1 -- 172.21.15.40:0/2813284059 wait complete. 2024-09-13T18:05:16.686 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:05:16.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.817+0000 7fc4eca45700 1 Processor -- start 2024-09-13T18:05:16.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.817+0000 7fc4eca45700 1 -- start start 2024-09-13T18:05:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4eca45700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc4e00d69c0 0x7fc4e0007560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4eca45700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc4e0007ba0 0x7fc4e00dc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4eca45700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e0005fa0 con 0x7fc4e004ca90 2024-09-13T18:05:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4eca45700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e00368b0 con 0x7fc4e0007ba0 2024-09-13T18:05:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4eca45700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e00dc510 con 0x7fc4e00d69c0 2024-09-13T18:05:16.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4e759e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc4e0007ba0 0x7fc4e00dc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:16.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4e659c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc4e004ca90 0x7fc4e0033e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56764/0 (socket says 172.21.15.40:56764) 2024-09-13T18:05:16.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4e659c700 1 -- 172.21.15.40:0/1153474226 learned_addr learned my addr 172.21.15.40:0/1153474226 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:05:16.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4e6d9d700 1 -- 172.21.15.40:0/1153474226 >> v2:172.21.15.43:3300/0 conn(0x7fc4e00d69c0 msgr2=0x7fc4e0007560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:05:16.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4e6d9d700 1 --2- 172.21.15.40:0/1153474226 >> v2:172.21.15.43:3300/0 conn(0x7fc4e00d69c0 0x7fc4e0007560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4e759e700 1 -- 172.21.15.40:0/1153474226 >> v2:172.21.15.43:3300/0 conn(0x7fc4e00d69c0 msgr2=0x7fc4e0007560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:05:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4e759e700 1 --2- 172.21.15.40:0/1153474226 >> v2:172.21.15.43:3300/0 conn(0x7fc4e00d69c0 0x7fc4e0007560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4e759e700 1 -- 172.21.15.40:0/1153474226 >> v1:172.21.15.40:6789/0 conn(0x7fc4e004ca90 legacy=0x7fc4e0033e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4e759e700 1 -- 172.21.15.40:0/1153474226 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e0036080 con 0x7fc4e0007ba0 2024-09-13T18:05:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4e759e700 1 --2- 172.21.15.40:0/1153474226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e0007ba0 0x7fc4e00dc960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8069620 tx=0x7fc4e8069e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11dfe1223aec0f76 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4e5d9b700 1 -- 172.21.15.40:0/1153474226 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8067fd0 con 0x7fc4e0007ba0 2024-09-13T18:05:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4e5d9b700 1 -- 172.21.15.40:0/1153474226 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e8068a70 con 0x7fc4e0007ba0 2024-09-13T18:05:16.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4e5d9b700 1 -- 172.21.15.40:0/1153474226 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e80746e0 con 0x7fc4e0007ba0 2024-09-13T18:05:16.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4eca45700 1 -- 172.21.15.40:0/1153474226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e0007ba0 msgr2=0x7fc4e00dc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:16.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4eca45700 1 --2- 172.21.15.40:0/1153474226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e0007ba0 0x7fc4e00dc960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8069620 tx=0x7fc4e8069e70 comp rx=0 tx=0).stop 2024-09-13T18:05:16.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4eca45700 1 -- 172.21.15.40:0/1153474226 shutdown_connections 2024-09-13T18:05:16.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4eca45700 1 --2- 172.21.15.40:0/1153474226 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e0007ba0 0x7fc4e00dc960 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4eca45700 1 --2- 172.21.15.40:0/1153474226 >> v2:172.21.15.43:3300/0 conn(0x7fc4e00d69c0 0x7fc4e0007560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4eca45700 1 -- 172.21.15.40:0/1153474226 >> 172.21.15.40:0/1153474226 conn(0x7fc4e00505e0 msgr2=0x7fc4e00509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4eca45700 1 -- 172.21.15.40:0/1153474226 shutdown_connections 2024-09-13T18:05:16.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.821+0000 7fc4eca45700 1 -- 172.21.15.40:0/1153474226 wait complete. 2024-09-13T18:05:16.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.825+0000 7fc4eca45700 1 Processor -- start 2024-09-13T18:05:16.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.825+0000 7fc4eca45700 1 -- start start 2024-09-13T18:05:16.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.825+0000 7fc4eca45700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e0007ba0 0x7fc4e00d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:16.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.825+0000 7fc4eca45700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc4e004ca90 0x7fc4e00489c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:16.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.825+0000 7fc4eca45700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e00d69c0 0x7fc4e0174a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:16.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.825+0000 7fc4eca45700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e00485a0 con 0x7fc4e00d69c0 2024-09-13T18:05:16.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.825+0000 7fc4eca45700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e0047fb0 con 0x7fc4e0007ba0 2024-09-13T18:05:16.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.825+0000 7fc4eca45700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e0173240 con 0x7fc4e004ca90 2024-09-13T18:05:16.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.825+0000 7fc4e759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e00d69c0 0x7fc4e0174a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:16.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.825+0000 7fc4e759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e00d69c0 0x7fc4e0174a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54590/0 (socket says 172.21.15.40:54590) 2024-09-13T18:05:16.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.825+0000 7fc4e759e700 1 -- 172.21.15.40:0/1817451366 learned_addr learned my addr 172.21.15.40:0/1817451366 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:16.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.825+0000 7fc4e659c700 1 --2- 172.21.15.40:0/1817451366 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc4e004ca90 0x7fc4e00489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:16.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.825+0000 7fc4e759e700 1 -- 172.21.15.40:0/1817451366 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc4e004ca90 msgr2=0x7fc4e00489c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:16.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.825+0000 7fc4e759e700 1 --2- 172.21.15.40:0/1817451366 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc4e004ca90 0x7fc4e00489c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:16.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.825+0000 7fc4e759e700 1 -- 172.21.15.40:0/1817451366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e0007ba0 msgr2=0x7fc4e00d1f20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:05:16.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.825+0000 7fc4e759e700 1 --2- 172.21.15.40:0/1817451366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e0007ba0 0x7fc4e00d1f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:16.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.825+0000 7fc4e759e700 1 -- 172.21.15.40:0/1817451366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e804f5c0 con 0x7fc4e00d69c0 2024-09-13T18:05:16.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.825+0000 7fc4e659c700 1 --2- 172.21.15.40:0/1817451366 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc4e004ca90 0x7fc4e00489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:05:16.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.825+0000 7fc4e759e700 1 --2- 172.21.15.40:0/1817451366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e00d69c0 0x7fc4e0174a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8067bd0 tx=0x7fc4e8068470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:16.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.829+0000 7fc4cf7fe700 1 -- 172.21.15.40:0/1817451366 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8072560 con 0x7fc4e00d69c0 2024-09-13T18:05:16.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.829+0000 7fc4cf7fe700 1 -- 172.21.15.40:0/1817451366 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e80726c0 con 0x7fc4e00d69c0 2024-09-13T18:05:16.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.829+0000 7fc4cf7fe700 1 -- 172.21.15.40:0/1817451366 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8072a30 con 0x7fc4e00d69c0 2024-09-13T18:05:16.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.829+0000 7fc4eca45700 1 -- 172.21.15.40:0/1817451366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e004a270 con 0x7fc4e00d69c0 2024-09-13T18:05:16.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.829+0000 7fc4eca45700 1 -- 172.21.15.40:0/1817451366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e00e05f0 con 0x7fc4e00d69c0 2024-09-13T18:05:16.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.829+0000 7fc4eca45700 1 -- 172.21.15.40:0/1817451366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e0007560 con 0x7fc4e00d69c0 2024-09-13T18:05:16.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.829+0000 7fc4cf7fe700 1 -- 172.21.15.40:0/1817451366 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc4e8091020 con 0x7fc4e00d69c0 2024-09-13T18:05:16.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.829+0000 7fc4cf7fe700 1 --2- 172.21.15.40:0/1817451366 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc4d4066730 0x7fc4d4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:16.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.829+0000 7fc4cf7fe700 1 -- 172.21.15.40:0/1817451366 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc4e81001a0 con 0x7fc4e00d69c0 2024-09-13T18:05:16.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.833+0000 7fc4e6d9d700 1 --2- 172.21.15.40:0/1817451366 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc4d4066730 0x7fc4d4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:16.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.833+0000 7fc4e6d9d700 1 --2- 172.21.15.40:0/1817451366 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc4d4066730 0x7fc4d4068bf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8003a30 tx=0x7fc4d8006d20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:16.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:16.837+0000 7fc4cf7fe700 1 -- 172.21.15.40:0/1817451366 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc4e80cd2c0 con 0x7fc4e00d69c0 2024-09-13T18:05:17.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:17.089+0000 7fc4eca45700 1 -- 172.21.15.40:0/1817451366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc4e0006930 con 0x7fc4e00d69c0 2024-09-13T18:05:17.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:17.093+0000 7fc4cf7fe700 1 -- 172.21.15.40:0/1817451366 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7fc4e80d08e0 con 0x7fc4e00d69c0 2024-09-13T18:05:17.099 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":118,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:05:17.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:17.097+0000 7fc4eca45700 1 -- 172.21.15.40:0/1817451366 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc4d4066730 msgr2=0x7fc4d4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:17.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:17.097+0000 7fc4eca45700 1 --2- 172.21.15.40:0/1817451366 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc4d4066730 0x7fc4d4068bf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8003a30 tx=0x7fc4d8006d20 comp rx=0 tx=0).stop 2024-09-13T18:05:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:17.097+0000 7fc4eca45700 1 -- 172.21.15.40:0/1817451366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e00d69c0 msgr2=0x7fc4e0174a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:17.097+0000 7fc4eca45700 1 --2- 172.21.15.40:0/1817451366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e00d69c0 0x7fc4e0174a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8067bd0 tx=0x7fc4e8068470 comp rx=0 tx=0).stop 2024-09-13T18:05:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:17.097+0000 7fc4eca45700 1 -- 172.21.15.40:0/1817451366 shutdown_connections 2024-09-13T18:05:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:17.097+0000 7fc4eca45700 1 --2- 172.21.15.40:0/1817451366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4e00d69c0 0x7fc4e0174a10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:17.097+0000 7fc4eca45700 1 --2- 172.21.15.40:0/1817451366 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc4d4066730 0x7fc4d4068bf0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:17.097+0000 7fc4eca45700 1 --2- 172.21.15.40:0/1817451366 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc4e004ca90 0x7fc4e00489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:17.097+0000 7fc4eca45700 1 --2- 172.21.15.40:0/1817451366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4e0007ba0 0x7fc4e00d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:17.097+0000 7fc4eca45700 1 -- 172.21.15.40:0/1817451366 >> 172.21.15.40:0/1817451366 conn(0x7fc4e00505e0 msgr2=0x7fc4e00dc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:17.097+0000 7fc4eca45700 1 -- 172.21.15.40:0/1817451366 shutdown_connections 2024-09-13T18:05:17.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:17.097+0000 7fc4eca45700 1 -- 172.21.15.40:0/1817451366 wait complete. 2024-09-13T18:05:20.119 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:05:20.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.229+0000 7f2367fff700 1 Processor -- start 2024-09-13T18:05:20.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.229+0000 7f2367fff700 1 -- start start 2024-09-13T18:05:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.229+0000 7f2367fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 0x7f2368155810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.229+0000 7f2367fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f23680c6d30 0x7f23680c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.229+0000 7f2367fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2368159cd0 con 0x7f236814fbe0 2024-09-13T18:05:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.229+0000 7f2367fff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236814ffc0 con 0x7f23680c6d30 2024-09-13T18:05:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.229+0000 7f2367fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23680c8ec0 con 0x7f23680c74b0 2024-09-13T18:05:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.229+0000 7f23677fe700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7f23680c6d30 msgr2=0x7f23680c7ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:05:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.229+0000 7f2366ffd700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7f236814fbe0 legacy=0x7f23681595c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:05:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.229+0000 7f23677fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f23680c6d30 0x7f23680c7ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.229+0000 7f23667fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 0x7f2368155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.229+0000 7f23667fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 0x7f2368155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:41050/0 (socket says 172.21.15.40:41050) 2024-09-13T18:05:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.229+0000 7f23667fc700 1 -- 172.21.15.40:0/401576231 learned_addr learned my addr 172.21.15.40:0/401576231 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.229+0000 7f23667fc700 1 -- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 msgr2=0x7f2368155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:05:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.229+0000 7f23667fc700 1 -- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 msgr2=0x7f2368155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:05:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.229+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 0x7f2368155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:05:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.229+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 0x7f2368155810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:20.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.433+0000 7f23677fe700 1 -- 172.21.15.40:0/401576231 >> v2:172.21.15.43:3300/0 conn(0x7f23680c6d30 msgr2=0x7f23680c7ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:05:20.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.433+0000 7f2366ffd700 1 -- 172.21.15.40:0/401576231 >> v1:172.21.15.43:6789/0 conn(0x7f236814fbe0 legacy=0x7f23681595c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:05:20.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.433+0000 7f23677fe700 1 --2- 172.21.15.40:0/401576231 >> v2:172.21.15.43:3300/0 conn(0x7f23680c6d30 0x7f23680c7ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:05:20.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.433+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 0x7f2368155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:20.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.433+0000 7f23667fc700 1 -- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 msgr2=0x7f2368155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:05:20.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.433+0000 7f23667fc700 1 -- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 msgr2=0x7f2368155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:05:20.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.433+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 0x7f2368155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:05:20.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.433+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 0x7f2368155810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:05:20.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.837+0000 7f23677fe700 1 -- 172.21.15.40:0/401576231 >> v2:172.21.15.43:3300/0 conn(0x7f23680c6d30 msgr2=0x7f23680c7ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:05:20.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.837+0000 7f2366ffd700 1 -- 172.21.15.40:0/401576231 >> v1:172.21.15.43:6789/0 conn(0x7f236814fbe0 legacy=0x7f23681595c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:05:20.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.837+0000 7f23677fe700 1 --2- 172.21.15.40:0/401576231 >> v2:172.21.15.43:3300/0 conn(0x7f23680c6d30 0x7f23680c7ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:05:20.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.837+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 0x7f2368155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:20.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.837+0000 7f23667fc700 1 -- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 msgr2=0x7f2368155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:05:20.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.837+0000 7f23667fc700 1 -- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 msgr2=0x7f2368155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:05:20.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.837+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 0x7f2368155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:05:20.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:20.837+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 0x7f2368155810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:05:21.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:21.641+0000 7f2366ffd700 1 -- 172.21.15.40:0/401576231 >> v1:172.21.15.43:6789/0 conn(0x7f236814fbe0 legacy=0x7f23681595c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:05:21.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:21.641+0000 7f23677fe700 1 -- 172.21.15.40:0/401576231 >> v2:172.21.15.43:3300/0 conn(0x7f23680c6d30 msgr2=0x7f23680c7ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:05:21.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:21.641+0000 7f23677fe700 1 --2- 172.21.15.40:0/401576231 >> v2:172.21.15.43:3300/0 conn(0x7f23680c6d30 0x7f23680c7ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:05:21.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:21.641+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 0x7f2368155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:21.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:21.641+0000 7f23667fc700 1 -- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 msgr2=0x7f2368155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:05:21.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:21.641+0000 7f23667fc700 1 -- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 msgr2=0x7f2368155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:05:21.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:21.641+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 0x7f2368155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:05:21.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:21.641+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 0x7f2368155810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:05:23.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.229+0000 7f2364ff9700 1 -- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 msgr2=0x7f2368155810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:05:23.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.229+0000 7f2364ff9700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f23680c74b0 0x7f2368155810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:23.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.229+0000 7f2364ff9700 1 -- 172.21.15.40:0/401576231 >> v2:172.21.15.43:3300/0 conn(0x7f23680c6d30 msgr2=0x7f23680c7ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:05:23.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.229+0000 7f2364ff9700 1 --2- 172.21.15.40:0/401576231 >> v2:172.21.15.43:3300/0 conn(0x7f23680c6d30 0x7f23680c7ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:23.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.229+0000 7f2364ff9700 1 -- 172.21.15.40:0/401576231 >> v1:172.21.15.43:6789/0 conn(0x7f236814fbe0 legacy=0x7f23681595c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:05:23.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.229+0000 7f2364ff9700 1 --2- 172.21.15.40:0/401576231 >> v2:172.21.15.40:3300/0 conn(0x7f2354007490 0x7f2354009950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:23.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.229+0000 7f2364ff9700 1 -- 172.21.15.40:0/401576231 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2368158e00 con 0x7f2354003750 2024-09-13T18:05:23.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.229+0000 7f2364ff9700 1 -- 172.21.15.40:0/401576231 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236814ffc0 con 0x7f2354007490 2024-09-13T18:05:23.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.229+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> v2:172.21.15.40:3300/0 conn(0x7f2354007490 0x7f2354009950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:23.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.229+0000 7f23667fc700 1 -- 172.21.15.40:0/401576231 >> v2:172.21.15.40:3300/0 conn(0x7f2354007490 msgr2=0x7f2354009950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:05:23.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.229+0000 7f23667fc700 1 -- 172.21.15.40:0/401576231 >> v2:172.21.15.40:3300/0 conn(0x7f2354007490 msgr2=0x7f2354009950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:05:23.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.229+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> v2:172.21.15.40:3300/0 conn(0x7f2354007490 0x7f2354009950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:05:23.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.229+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> v2:172.21.15.40:3300/0 conn(0x7f2354007490 0x7f2354009950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:23.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.433+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> v2:172.21.15.40:3300/0 conn(0x7f2354007490 0x7f2354009950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:23.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.433+0000 7f23667fc700 1 -- 172.21.15.40:0/401576231 >> v2:172.21.15.40:3300/0 conn(0x7f2354007490 msgr2=0x7f2354009950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:05:23.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.433+0000 7f23667fc700 1 -- 172.21.15.40:0/401576231 >> v2:172.21.15.40:3300/0 conn(0x7f2354007490 msgr2=0x7f2354009950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:05:23.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.433+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> v2:172.21.15.40:3300/0 conn(0x7f2354007490 0x7f2354009950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:05:23.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.433+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> v2:172.21.15.40:3300/0 conn(0x7f2354007490 0x7f2354009950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:05:23.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.837+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> v2:172.21.15.40:3300/0 conn(0x7f2354007490 0x7f2354009950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:23.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.837+0000 7f23667fc700 1 -- 172.21.15.40:0/401576231 >> v2:172.21.15.40:3300/0 conn(0x7f2354007490 msgr2=0x7f2354009950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:05:23.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.837+0000 7f23667fc700 1 -- 172.21.15.40:0/401576231 >> v2:172.21.15.40:3300/0 conn(0x7f2354007490 msgr2=0x7f2354009950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:05:23.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.837+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> v2:172.21.15.40:3300/0 conn(0x7f2354007490 0x7f2354009950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:05:23.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:23.837+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> v2:172.21.15.40:3300/0 conn(0x7f2354007490 0x7f2354009950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:05:24.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.457+0000 7f2365ffb700 1 -- 172.21.15.40:0/401576231 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223475697 0 0) 0x7f2368158e00 con 0x7f2354003750 2024-09-13T18:05:24.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.457+0000 7f2365ffb700 1 -- 172.21.15.40:0/401576231 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23500035d0 con 0x7f2354003750 2024-09-13T18:05:24.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2365ffb700 1 -- 172.21.15.40:0/401576231 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1177225179 0 0) 0x7f23500035d0 con 0x7f2354003750 2024-09-13T18:05:24.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2365ffb700 1 -- 172.21.15.40:0/401576231 >> v2:172.21.15.40:3300/0 conn(0x7f2354007490 msgr2=0x7f2354009950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:05:24.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2365ffb700 1 --2- 172.21.15.40:0/401576231 >> v2:172.21.15.40:3300/0 conn(0x7f2354007490 0x7f2354009950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2365ffb700 1 -- 172.21.15.40:0/401576231 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2368159cd0 con 0x7f2354003750 2024-09-13T18:05:24.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2365ffb700 1 -- 172.21.15.40:0/401576231 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f23600045a0 con 0x7f2354003750 2024-09-13T18:05:24.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2365ffb700 1 -- 172.21.15.40:0/401576231 >> v1:172.21.15.40:6789/0 conn(0x7f2354003750 legacy=0x7f2354005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:24.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f23677fe700 1 -- 172.21.15.40:0/401576231 reap_dead start 2024-09-13T18:05:24.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2365ffb700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2350003a20 0x7f2350003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:24.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2365ffb700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2350004870 0x7f2350004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:24.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2365ffb700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2350005280 0x7f235000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:24.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2365ffb700 1 -- 172.21.15.40:0/401576231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236814ffc0 con 0x7f2350005280 2024-09-13T18:05:24.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2365ffb700 1 -- 172.21.15.40:0/401576231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235000c1b0 con 0x7f2350004870 2024-09-13T18:05:24.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2365ffb700 1 -- 172.21.15.40:0/401576231 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235000c370 con 0x7f2350003a20 2024-09-13T18:05:24.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f23677fe700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2350005280 0x7f235000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:24.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f23667fc700 1 -- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2350004870 msgr2=0x7f2350004cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:05:24.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f23667fc700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2350004870 0x7f2350004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:24.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2366ffd700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2350003a20 0x7f2350003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:24.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f23677fe700 1 -- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2350003a20 msgr2=0x7f2350003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:05:24.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f23677fe700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2350003a20 0x7f2350003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f23677fe700 1 -- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2350004870 msgr2=0x7f2350004cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:05:24.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f23677fe700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2350004870 0x7f2350004cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f23677fe700 1 -- 172.21.15.40:0/401576231 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23500035d0 con 0x7f2350005280 2024-09-13T18:05:24.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f23677fe700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2350005280 0x7f235000ba30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2368072580 tx=0x7f23500058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91862aeb38c109be server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:24.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2365ffb700 1 -- 172.21.15.40:0/401576231 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c0097b0 con 0x7f2350005280 2024-09-13T18:05:24.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2365ffb700 1 -- 172.21.15.40:0/401576231 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c00ba90 con 0x7f2350005280 2024-09-13T18:05:24.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2365ffb700 1 -- 172.21.15.40:0/401576231 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c00be40 con 0x7f2350005280 2024-09-13T18:05:24.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2367fff700 1 -- 172.21.15.40:0/401576231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2350005280 msgr2=0x7f235000ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:24.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2367fff700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2350005280 0x7f235000ba30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2368072580 tx=0x7f23500058a0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2367fff700 1 -- 172.21.15.40:0/401576231 shutdown_connections 2024-09-13T18:05:24.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2367fff700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2350005280 0x7f235000ba30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2367fff700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2350004870 0x7f2350004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2367fff700 1 --2- 172.21.15.40:0/401576231 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2350003a20 0x7f2350003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2367fff700 1 -- 172.21.15.40:0/401576231 >> 172.21.15.40:0/401576231 conn(0x7f2368066f50 msgr2=0x7f236806a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:24.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2367fff700 1 -- 172.21.15.40:0/401576231 shutdown_connections 2024-09-13T18:05:24.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.461+0000 7f2367fff700 1 -- 172.21.15.40:0/401576231 wait complete. 2024-09-13T18:05:24.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.465+0000 7f2367fff700 1 Processor -- start 2024-09-13T18:05:24.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.465+0000 7f2367fff700 1 -- start start 2024-09-13T18:05:24.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.465+0000 7f2367fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2350003a20 0x7f2368153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:24.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.465+0000 7f2367fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2350004870 0x7f236806e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:24.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.465+0000 7f2367fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2350005280 0x7f23680c3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:24.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.465+0000 7f2367fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236806a860 con 0x7f2350003a20 2024-09-13T18:05:24.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.465+0000 7f2367fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23681575a0 con 0x7f2350004870 2024-09-13T18:05:24.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.465+0000 7f2367fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2368157880 con 0x7f2350005280 2024-09-13T18:05:24.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.465+0000 7f23667fc700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2350004870 msgr2=0x7f236806e420 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:05:24.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.465+0000 7f2366ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2350003a20 0x7f2368153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:24.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.465+0000 7f2366ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2350003a20 0x7f2368153c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37242/0 (socket says 172.21.15.40:37242) 2024-09-13T18:05:24.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.465+0000 7f2366ffd700 1 -- 172.21.15.40:0/1262189755 learned_addr learned my addr 172.21.15.40:0/1262189755 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:24.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.465+0000 7f23677fe700 1 --2- 172.21.15.40:0/1262189755 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2350005280 0x7f23680c3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:24.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.465+0000 7f23667fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2350004870 0x7f236806e420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:24.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.465+0000 7f2366ffd700 1 -- 172.21.15.40:0/1262189755 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2350005280 msgr2=0x7f23680c3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:24.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.465+0000 7f2366ffd700 1 --2- 172.21.15.40:0/1262189755 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2350005280 0x7f23680c3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.465+0000 7f2366ffd700 1 -- 172.21.15.40:0/1262189755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2350004870 msgr2=0x7f236806e420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:05:24.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.465+0000 7f2366ffd700 1 --2- 172.21.15.40:0/1262189755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2350004870 0x7f236806e420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.465+0000 7f2366ffd700 1 -- 172.21.15.40:0/1262189755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2368157b60 con 0x7f2350003a20 2024-09-13T18:05:24.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.469+0000 7f2366ffd700 1 --2- 172.21.15.40:0/1262189755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2350003a20 0x7f2368153c90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f236000ead0 tx=0x7f236000ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:24.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.469+0000 7f234ffff700 1 -- 172.21.15.40:0/1262189755 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23600062e0 con 0x7f2350003a20 2024-09-13T18:05:24.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.469+0000 7f234ffff700 1 -- 172.21.15.40:0/1262189755 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2360006480 con 0x7f2350003a20 2024-09-13T18:05:24.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.469+0000 7f234ffff700 1 -- 172.21.15.40:0/1262189755 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23600049d0 con 0x7f2350003a20 2024-09-13T18:05:24.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.469+0000 7f2367fff700 1 -- 172.21.15.40:0/1262189755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2368157e40 con 0x7f2350003a20 2024-09-13T18:05:24.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.469+0000 7f2367fff700 1 -- 172.21.15.40:0/1262189755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2368156f80 con 0x7f2350003a20 2024-09-13T18:05:24.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.469+0000 7f2367fff700 1 -- 172.21.15.40:0/1262189755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2368156690 con 0x7f2350003a20 2024-09-13T18:05:24.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.469+0000 7f234ffff700 1 -- 172.21.15.40:0/1262189755 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2360004b30 con 0x7f2350003a20 2024-09-13T18:05:24.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.469+0000 7f234ffff700 1 --2- 172.21.15.40:0/1262189755 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2354066d50 0x7f2354069210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:24.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.469+0000 7f234ffff700 1 -- 172.21.15.40:0/1262189755 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2360084560 con 0x7f2350003a20 2024-09-13T18:05:24.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.469+0000 7f23667fc700 1 --2- 172.21.15.40:0/1262189755 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2354066d50 0x7f2354069210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:24.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.473+0000 7f23667fc700 1 --2- 172.21.15.40:0/1262189755 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2354066d50 0x7f2354069210 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f235800b340 tx=0x7f235800f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:24.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.477+0000 7f234ffff700 1 -- 172.21.15.40:0/1262189755 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f23600515a0 con 0x7f2350003a20 2024-09-13T18:05:24.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.725+0000 7f2367fff700 1 -- 172.21.15.40:0/1262189755 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f2368158c10 con 0x7f2350003a20 2024-09-13T18:05:24.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.729+0000 7f234ffff700 1 -- 172.21.15.40:0/1262189755 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f2360054bc0 con 0x7f2350003a20 2024-09-13T18:05:24.735 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":122,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:05:24.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.729+0000 7f2367fff700 1 -- 172.21.15.40:0/1262189755 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2354066d50 msgr2=0x7f2354069210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:24.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.729+0000 7f2367fff700 1 --2- 172.21.15.40:0/1262189755 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2354066d50 0x7f2354069210 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f235800b340 tx=0x7f235800f040 comp rx=0 tx=0).stop 2024-09-13T18:05:24.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.729+0000 7f2367fff700 1 -- 172.21.15.40:0/1262189755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2350003a20 msgr2=0x7f2368153c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:24.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.729+0000 7f2367fff700 1 --2- 172.21.15.40:0/1262189755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2350003a20 0x7f2368153c90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f236000ead0 tx=0x7f236000ede0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.729+0000 7f2367fff700 1 -- 172.21.15.40:0/1262189755 shutdown_connections 2024-09-13T18:05:24.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.729+0000 7f2367fff700 1 --2- 172.21.15.40:0/1262189755 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2354066d50 0x7f2354069210 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.729+0000 7f2367fff700 1 --2- 172.21.15.40:0/1262189755 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2350005280 0x7f23680c3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.729+0000 7f2367fff700 1 --2- 172.21.15.40:0/1262189755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2350004870 0x7f236806e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.729+0000 7f2367fff700 1 --2- 172.21.15.40:0/1262189755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2350003a20 0x7f2368153c90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.729+0000 7f2367fff700 1 -- 172.21.15.40:0/1262189755 >> 172.21.15.40:0/1262189755 conn(0x7f2368066f50 msgr2=0x7f23680c60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:24.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.729+0000 7f2367fff700 1 -- 172.21.15.40:0/1262189755 shutdown_connections 2024-09-13T18:05:24.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.729+0000 7f2367fff700 1 -- 172.21.15.40:0/1262189755 wait complete. 2024-09-13T18:05:24.755 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T18:05:24.755 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-13T18:05:24.756 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:05:24.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.865+0000 7f9db114c700 1 Processor -- start 2024-09-13T18:05:24.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.869+0000 7f9db114c700 1 -- start start 2024-09-13T18:05:24.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.869+0000 7f9db114c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9dac0c8500 0x7f9dac0c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:24.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.869+0000 7f9db114c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9dac0c71c0 0x7f9dac159630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:24.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.869+0000 7f9db114c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dac0c8f80 con 0x7f9dac1503c0 2024-09-13T18:05:24.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.869+0000 7f9db114c700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dac0c2a80 con 0x7f9dac0c71c0 2024-09-13T18:05:24.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.869+0000 7f9db114c700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dac0c80d0 con 0x7f9dac0c8500 2024-09-13T18:05:24.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.869+0000 7f9dabfff700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7f9dac1503c0 legacy=0x7f9dac0c79c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:05:24.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.869+0000 7f9dab7fe700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7f9dac0c8500 msgr2=0x7f9dac0c6c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:05:24.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.869+0000 7f9dab7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9dac0c8500 0x7f9dac0c6c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:24.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.869+0000 7f9db094b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9dac0c71c0 0x7f9dac159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:24.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.869+0000 7f9db094b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9dac0c71c0 0x7f9dac159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37258/0 (socket says 172.21.15.40:37258) 2024-09-13T18:05:24.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.869+0000 7f9db094b700 1 -- 172.21.15.40:0/317198702 learned_addr learned my addr 172.21.15.40:0/317198702 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:24.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.869+0000 7f9db094b700 1 -- 172.21.15.40:0/317198702 >> v2:172.21.15.43:3300/0 conn(0x7f9dac0c8500 msgr2=0x7f9dac0c6c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:05:24.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.869+0000 7f9db094b700 1 --2- 172.21.15.40:0/317198702 >> v2:172.21.15.43:3300/0 conn(0x7f9dac0c8500 0x7f9dac0c6c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.869+0000 7f9db094b700 1 -- 172.21.15.40:0/317198702 >> v1:172.21.15.43:6789/0 conn(0x7f9dac1503c0 legacy=0x7f9dac0c79c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:05:24.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.869+0000 7f9db094b700 1 -- 172.21.15.40:0/317198702 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dac1507a0 con 0x7f9dac0c71c0 2024-09-13T18:05:24.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.869+0000 7f9db094b700 1 --2- 172.21.15.40:0/317198702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac0c71c0 0x7f9dac159630 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9dac072b70 tx=0x7f9da400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1539f02fc8a8554d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:24.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.869+0000 7f9daaffd700 1 -- 172.21.15.40:0/317198702 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da400e070 con 0x7f9dac0c71c0 2024-09-13T18:05:24.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.869+0000 7f9daaffd700 1 -- 172.21.15.40:0/317198702 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da4003a40 con 0x7f9dac0c71c0 2024-09-13T18:05:24.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.869+0000 7f9daaffd700 1 -- 172.21.15.40:0/317198702 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4003df0 con 0x7f9dac0c71c0 2024-09-13T18:05:24.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.873+0000 7f9db114c700 1 -- 172.21.15.40:0/317198702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac0c71c0 msgr2=0x7f9dac159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:24.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.873+0000 7f9db114c700 1 --2- 172.21.15.40:0/317198702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac0c71c0 0x7f9dac159630 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9dac072b70 tx=0x7f9da400b330 comp rx=0 tx=0).stop 2024-09-13T18:05:24.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.873+0000 7f9db114c700 1 -- 172.21.15.40:0/317198702 shutdown_connections 2024-09-13T18:05:24.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.873+0000 7f9db114c700 1 --2- 172.21.15.40:0/317198702 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac0c71c0 0x7f9dac159630 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.873+0000 7f9db114c700 1 --2- 172.21.15.40:0/317198702 >> v2:172.21.15.43:3300/0 conn(0x7f9dac0c8500 0x7f9dac0c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.873+0000 7f9db114c700 1 -- 172.21.15.40:0/317198702 >> 172.21.15.40:0/317198702 conn(0x7f9dac06e6d0 msgr2=0x7f9dac06eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:24.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.873+0000 7f9db114c700 1 -- 172.21.15.40:0/317198702 shutdown_connections 2024-09-13T18:05:24.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.873+0000 7f9db114c700 1 -- 172.21.15.40:0/317198702 wait complete. 2024-09-13T18:05:24.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.873+0000 7f9db114c700 1 Processor -- start 2024-09-13T18:05:24.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.873+0000 7f9db114c700 1 -- start start 2024-09-13T18:05:24.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9db114c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac0c71c0 0x7f9dac1ef030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9db114c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9dac0c8500 0x7f9dac06b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9db114c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dac1503c0 0x7f9dac1ebcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9db114c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac0c4970 con 0x7f9dac0c71c0 2024-09-13T18:05:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9db114c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac148180 con 0x7f9dac1503c0 2024-09-13T18:05:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9db114c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac0c4b80 con 0x7f9dac0c8500 2024-09-13T18:05:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9db094b700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dac1503c0 msgr2=0x7f9dac1ebcf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:05:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9db094b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dac1503c0 0x7f9dac1ebcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9dab7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9dac0c8500 0x7f9dac06b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9dabfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac0c71c0 0x7f9dac1ef030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9dab7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9dac0c8500 0x7f9dac06b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:39212/0 (socket says 172.21.15.40:39212) 2024-09-13T18:05:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9dabfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac0c71c0 0x7f9dac1ef030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37274/0 (socket says 172.21.15.40:37274) 2024-09-13T18:05:24.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9dab7fe700 1 -- 172.21.15.40:0/4012579881 learned_addr learned my addr 172.21.15.40:0/4012579881 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:24.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9dabfff700 1 -- 172.21.15.40:0/4012579881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9dac0c8500 msgr2=0x7f9dac06b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:24.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9dabfff700 1 --2- 172.21.15.40:0/4012579881 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9dac0c8500 0x7f9dac06b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9dabfff700 1 -- 172.21.15.40:0/4012579881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dac1503c0 msgr2=0x7f9dac1ebcf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:05:24.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9dabfff700 1 --2- 172.21.15.40:0/4012579881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dac1503c0 0x7f9dac1ebcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9dabfff700 1 -- 172.21.15.40:0/4012579881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da4009b30 con 0x7f9dac0c71c0 2024-09-13T18:05:24.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9dabfff700 1 --2- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac0c71c0 0x7f9dac1ef030 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00b500 tx=0x7f9d9c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:24.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9da8ff9700 1 -- 172.21.15.40:0/4012579881 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c003e70 con 0x7f9dac0c71c0 2024-09-13T18:05:24.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9da8ff9700 1 -- 172.21.15.40:0/4012579881 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9c005070 con 0x7f9dac0c71c0 2024-09-13T18:05:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9da8ff9700 1 -- 172.21.15.40:0/4012579881 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c005440 con 0x7f9dac0c71c0 2024-09-13T18:05:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9db114c700 1 -- 172.21.15.40:0/4012579881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dac146c10 con 0x7f9dac0c71c0 2024-09-13T18:05:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9db114c700 1 -- 172.21.15.40:0/4012579881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dac148530 con 0x7f9dac0c71c0 2024-09-13T18:05:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9db114c700 1 --2- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac1412e0 0x7f9dac1ec230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9db114c700 1 -- 172.21.15.40:0/4012579881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac146f40 con 0x7f9dac1412e0 2024-09-13T18:05:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9dab7fe700 1 --2- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac1412e0 0x7f9dac1ec230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9da8ff9700 1 -- 172.21.15.40:0/4012579881 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9d9c0055a0 con 0x7f9dac0c71c0 2024-09-13T18:05:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9dab7fe700 1 -- 172.21.15.40:0/4012579881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9dac0675e0 con 0x7f9dac1412e0 2024-09-13T18:05:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.877+0000 7f9dab7fe700 1 --2- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac1412e0 0x7f9dac1ec230 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9dac06c350 tx=0x7f9da0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:24.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.881+0000 7f9da8ff9700 1 --2- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9d980666e0 0x7f9d98068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:24.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.881+0000 7f9db094b700 1 --2- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9d980666e0 0x7f9d98068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:24.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.881+0000 7f9da8ff9700 1 -- 172.21.15.40:0/4012579881 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9d9c083d60 con 0x7f9dac0c71c0 2024-09-13T18:05:24.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.881+0000 7f9da8ff9700 1 -- 172.21.15.40:0/4012579881 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da0012600 con 0x7f9dac1412e0 2024-09-13T18:05:24.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.881+0000 7f9da8ff9700 1 -- 172.21.15.40:0/4012579881 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9da0012760 con 0x7f9dac1412e0 2024-09-13T18:05:24.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.881+0000 7f9da8ff9700 1 -- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac1412e0 msgr2=0x7f9dac1ec230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:24.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.881+0000 7f9da8ff9700 1 --2- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac1412e0 0x7f9dac1ec230 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9dac06c350 tx=0x7f9da0009b30 comp rx=0 tx=0).stop 2024-09-13T18:05:24.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.881+0000 7f9da8ff9700 1 -- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac1412e0 msgr2=0x7f9dac1ec230 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:05:24.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.881+0000 7f9da8ff9700 1 --2- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac1412e0 0x7f9dac1ec230 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.881+0000 7f9db094b700 1 --2- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9d980666e0 0x7f9d98068ba0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0b17f0 tx=0x7f9da4009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:24.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9db114c700 1 --2- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac146570 0x7f9dac1ec770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:24.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9db114c700 1 -- 172.21.15.40:0/4012579881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac0c8ce0 con 0x7f9dac146570 2024-09-13T18:05:24.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9dab7fe700 1 --2- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac146570 0x7f9dac1ec770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:24.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9dab7fe700 1 -- 172.21.15.40:0/4012579881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9dac0c8a40 con 0x7f9dac146570 2024-09-13T18:05:24.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9dab7fe700 1 --2- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac146570 0x7f9dac1ec770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9da0005a50 tx=0x7f9da0005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:24.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9da8ff9700 1 -- 172.21.15.40:0/4012579881 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da0023070 con 0x7f9dac146570 2024-09-13T18:05:24.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9da8ff9700 1 -- 172.21.15.40:0/4012579881 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4773 (secure 0 0 0) 0x7f9da0012e70 con 0x7f9dac146570 2024-09-13T18:05:24.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9da8ff9700 1 -- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac146570 msgr2=0x7f9dac1ec770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:24.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9da8ff9700 1 --2- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac146570 0x7f9dac1ec770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9da0005a50 tx=0x7f9da0005a80 comp rx=0 tx=0).stop 2024-09-13T18:05:24.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9da8ff9700 1 -- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac146570 msgr2=0x7f9dac1ec770 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:05:24.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9da8ff9700 1 --2- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac146570 0x7f9dac1ec770 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:24.897 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:05:24.897 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:05:24.898 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:05:24.898 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:05:24.898 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 122, 2024-09-13T18:05:24.898 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:05:24.898 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:05:24.898 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:05:24.898 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:24.898 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:05:24.898 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:05:24.898 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:05:24.898 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:05:24.898 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:24.898 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:24.898 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:24.899 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:24.899 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:24.899 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:24.899 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:24.899 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:24.899 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:24.899 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:24.899 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:24.899 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:05:24.899 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:05:24.899 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:24.899 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:24.899 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:24.899 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:24.899 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:24.900 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:24.900 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:24.900 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:24.900 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:24.900 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:24.900 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:24.900 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:24.900 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:05:24.900 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:05:24.900 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:05:24.900 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:05:24.900 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:05:24.900 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:05:24.900 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:05:24.901 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:05:24.901 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:05:24.901 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:05:24.901 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:05:24.901 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:05:24.901 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:05:24.901 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:05:24.901 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:05:24.901 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:05:24.901 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:05:24.901 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:24.901 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:24.901 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:24.901 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:24.902 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:24.902 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:24.902 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:24.902 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:24.902 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:24.902 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:24.902 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:24.902 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:05:24.902 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:24.902 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:05:24.902 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:24.902 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:05:24.902 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:05:24.902 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:24.902 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:24.903 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:24.903 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:24.903 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:05:24.903 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:24.903 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:24.903 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:24.903 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:24.903 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:05:24.903 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:24.903 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:24.903 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:24.903 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:24.903 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:05:24.903 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:05:24.904 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:24.904 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:24.904 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:24.904 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:24.904 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:24.904 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:05:24.904 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:05:24.904 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:24.904 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:24.904 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:24.904 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:24.904 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:05:24.904 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:24.904 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:24.904 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:24.905 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:24.905 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:05:24.905 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:24.905 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:24.905 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:24.905 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:24.905 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:05:24.905 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:05:24.905 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:24.905 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:24.905 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:24.905 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:24.905 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:24.905 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:05:24.906 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:05:24.906 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:24.906 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:24.906 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:24.906 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:24.906 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:05:24.906 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:24.906 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:24.906 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:24.906 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:24.906 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:05:24.906 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:24.906 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:24.906 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:24.906 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:24.907 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:05:24.907 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:05:24.907 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:24.907 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:24.907 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:24.907 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:24.907 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:24.907 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:24.907 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:05:24.907 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:05:24.907 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:24.907 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:24.907 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:24.907 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:05:24.908 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:24.908 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:24.908 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:05:24.908 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:24.908 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:24.908 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:24.908 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:05:24.908 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:24.908 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:24.908 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:05:24.908 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:24.908 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:24.908 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:24.908 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:05:24.909 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:24.909 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:24.909 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:05:24.909 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:24.909 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:24.909 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:24.909 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:05:24.909 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:24.909 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:24.909 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:24.909 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:05:24.909 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:05:24.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9db114c700 1 -- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9d980666e0 msgr2=0x7f9d98068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:24.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9db114c700 1 --2- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9d980666e0 0x7f9d98068ba0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0b17f0 tx=0x7f9da4009000 comp rx=0 tx=0).stop 2024-09-13T18:05:24.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9db114c700 1 -- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac0c71c0 msgr2=0x7f9dac1ef030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:24.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9db114c700 1 --2- 172.21.15.40:0/4012579881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9dac0c71c0 0x7f9dac1ef030 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00b500 tx=0x7f9d9c00b810 comp rx=0 tx=0).stop 2024-09-13T18:05:24.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9db094b700 1 -- 172.21.15.40:0/4012579881 reap_dead start 2024-09-13T18:05:24.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9db094b700 1 -- 172.21.15.40:0/4012579881 reap_dead start 2024-09-13T18:05:24.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9db114c700 1 -- 172.21.15.40:0/4012579881 shutdown_connections 2024-09-13T18:05:24.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9db114c700 1 -- 172.21.15.40:0/4012579881 >> 172.21.15.40:0/4012579881 conn(0x7f9dac06e6d0 msgr2=0x7f9dac14b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:24.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9db114c700 1 -- 172.21.15.40:0/4012579881 shutdown_connections 2024-09-13T18:05:24.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:24.889+0000 7f9db114c700 1 -- 172.21.15.40:0/4012579881 wait complete. 2024-09-13T18:05:24.912 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:05:24.912 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T18:05:24.912 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:05:25.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.029+0000 7f421b59e700 1 Processor -- start 2024-09-13T18:05:25.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.029+0000 7f421b59e700 1 -- start start 2024-09-13T18:05:25.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421b59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c151580 0x7f421c0c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:25.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421b59e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f421c0b1fb0 0x7f421c0c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:25.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421b59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f421c071260 0x7f421c157800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421b59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f421c070ae0 con 0x7f421c071260 2024-09-13T18:05:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421b59e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f421c070d30 con 0x7f421c0b1fb0 2024-09-13T18:05:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421b59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421c0c29c0 con 0x7f421c151580 2024-09-13T18:05:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f4219d9b700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7f421c0b1fb0 msgr2=0x7f421c0c57f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:05:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421ad9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f421c071260 0x7f421c157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f4219d9b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f421c0b1fb0 0x7f421c0c57f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421ad9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f421c071260 0x7f421c157800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37294/0 (socket says 172.21.15.40:37294) 2024-09-13T18:05:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421ad9d700 1 -- 172.21.15.40:0/2099866152 learned_addr learned my addr 172.21.15.40:0/2099866152 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421a59c700 1 --2- 172.21.15.40:0/2099866152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c151580 0x7f421c0c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:25.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421ad9d700 1 -- 172.21.15.40:0/2099866152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c151580 msgr2=0x7f421c0c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:05:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421ad9d700 1 --2- 172.21.15.40:0/2099866152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c151580 0x7f421c0c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421ad9d700 1 -- 172.21.15.40:0/2099866152 >> v2:172.21.15.43:3300/0 conn(0x7f421c0b1fb0 msgr2=0x7f421c0c57f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:05:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421ad9d700 1 --2- 172.21.15.40:0/2099866152 >> v2:172.21.15.43:3300/0 conn(0x7f421c0b1fb0 0x7f421c0c57f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421ad9d700 1 -- 172.21.15.40:0/2099866152 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421c0c2ca0 con 0x7f421c071260 2024-09-13T18:05:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421ad9d700 1 --2- 172.21.15.40:0/2099866152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c071260 0x7f421c157800 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f421c0c5f70 tx=0x7f421400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a288d58b00d68f17 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421959a700 1 -- 172.21.15.40:0/2099866152 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421400e070 con 0x7f421c071260 2024-09-13T18:05:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421959a700 1 -- 172.21.15.40:0/2099866152 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421400b910 con 0x7f421c071260 2024-09-13T18:05:25.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421959a700 1 -- 172.21.15.40:0/2099866152 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421400bcc0 con 0x7f421c071260 2024-09-13T18:05:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421b59e700 1 -- 172.21.15.40:0/2099866152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c071260 msgr2=0x7f421c157800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421b59e700 1 --2- 172.21.15.40:0/2099866152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c071260 0x7f421c157800 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f421c0c5f70 tx=0x7f421400b330 comp rx=0 tx=0).stop 2024-09-13T18:05:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421b59e700 1 -- 172.21.15.40:0/2099866152 shutdown_connections 2024-09-13T18:05:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421b59e700 1 --2- 172.21.15.40:0/2099866152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c071260 0x7f421c157800 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421b59e700 1 --2- 172.21.15.40:0/2099866152 >> v2:172.21.15.43:3300/0 conn(0x7f421c0b1fb0 0x7f421c0c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421b59e700 1 --2- 172.21.15.40:0/2099866152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c151580 0x7f421c0c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421b59e700 1 -- 172.21.15.40:0/2099866152 >> 172.21.15.40:0/2099866152 conn(0x7f421c06e6d0 msgr2=0x7f421c06eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421b59e700 1 -- 172.21.15.40:0/2099866152 shutdown_connections 2024-09-13T18:05:25.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.033+0000 7f421b59e700 1 -- 172.21.15.40:0/2099866152 wait complete. 2024-09-13T18:05:25.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.037+0000 7f421b59e700 1 Processor -- start 2024-09-13T18:05:25.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.037+0000 7f421b59e700 1 -- start start 2024-09-13T18:05:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.037+0000 7f421b59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c071260 0x7f421c0cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.037+0000 7f421b59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f421c0b1fb0 0x7f421c0cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.037+0000 7f421b59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c151580 0x7f421c1f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.037+0000 7f421b59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421c0cb260 con 0x7f421c071260 2024-09-13T18:05:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.037+0000 7f421b59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421c148680 con 0x7f421c0b1fb0 2024-09-13T18:05:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.037+0000 7f421b59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421c15a460 con 0x7f421c151580 2024-09-13T18:05:25.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.037+0000 7f4219d9b700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f421c0b1fb0 msgr2=0x7f421c0cbbf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:05:25.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.037+0000 7f421a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c071260 0x7f421c0cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:25.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.037+0000 7f4219d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f421c0b1fb0 0x7f421c0cbbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:25.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.037+0000 7f421a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c071260 0x7f421c0cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37300/0 (socket says 172.21.15.40:37300) 2024-09-13T18:05:25.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.037+0000 7f421a59c700 1 -- 172.21.15.40:0/3897194664 learned_addr learned my addr 172.21.15.40:0/3897194664 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:25.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.037+0000 7f421ad9d700 1 --2- 172.21.15.40:0/3897194664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c151580 0x7f421c1f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:25.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.037+0000 7f421a59c700 1 -- 172.21.15.40:0/3897194664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c151580 msgr2=0x7f421c1f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:25.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.037+0000 7f421a59c700 1 --2- 172.21.15.40:0/3897194664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c151580 0x7f421c1f21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:25.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.037+0000 7f421a59c700 1 -- 172.21.15.40:0/3897194664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f421c0b1fb0 msgr2=0x7f421c0cbbf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:05:25.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.037+0000 7f421a59c700 1 --2- 172.21.15.40:0/3897194664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f421c0b1fb0 0x7f421c0cbbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:25.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.037+0000 7f421a59c700 1 -- 172.21.15.40:0/3897194664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4214009b30 con 0x7f421c071260 2024-09-13T18:05:25.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f421a59c700 1 --2- 172.21.15.40:0/3897194664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c071260 0x7f421c0cb6b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f420c00ebe0 tx=0x7f420c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:25.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f420affd700 1 -- 172.21.15.40:0/3897194664 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c009740 con 0x7f421c071260 2024-09-13T18:05:25.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f420affd700 1 -- 172.21.15.40:0/3897194664 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420c00c850 con 0x7f421c071260 2024-09-13T18:05:25.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f421b59e700 1 -- 172.21.15.40:0/3897194664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f421c159ec0 con 0x7f421c071260 2024-09-13T18:05:25.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f420affd700 1 -- 172.21.15.40:0/3897194664 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c00cbe0 con 0x7f421c071260 2024-09-13T18:05:25.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f421b59e700 1 -- 172.21.15.40:0/3897194664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f421c0b1cb0 con 0x7f421c071260 2024-09-13T18:05:25.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f421b59e700 1 --2- 172.21.15.40:0/3897194664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c1f2710 0x7f421c147760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:25.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f421b59e700 1 -- 172.21.15.40:0/3897194664 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421c067600 con 0x7f421c1f2710 2024-09-13T18:05:25.049 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f4219d9b700 1 --2- 172.21.15.40:0/3897194664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c1f2710 0x7f421c147760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:25.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f4219d9b700 1 -- 172.21.15.40:0/3897194664 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f421c1570b0 con 0x7f421c1f2710 2024-09-13T18:05:25.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f4219d9b700 1 --2- 172.21.15.40:0/3897194664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c1f2710 0x7f421c147760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f421c157950 tx=0x7f4210009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:25.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f420affd700 1 -- 172.21.15.40:0/3897194664 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421000ca80 con 0x7f421c1f2710 2024-09-13T18:05:25.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f420affd700 1 -- 172.21.15.40:0/3897194664 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f420c00cd40 con 0x7f421c071260 2024-09-13T18:05:25.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f420affd700 1 --2- 172.21.15.40:0/3897194664 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f41fc066800 0x7f41fc068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:25.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f420affd700 1 -- 172.21.15.40:0/3897194664 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f420c0845a0 con 0x7f421c071260 2024-09-13T18:05:25.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f421ad9d700 1 --2- 172.21.15.40:0/3897194664 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f41fc066800 0x7f41fc068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:25.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f420affd700 1 -- 172.21.15.40:0/3897194664 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f421000cbe0 con 0x7f421c1f2710 2024-09-13T18:05:25.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f420affd700 1 -- 172.21.15.40:0/3897194664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c1f2710 msgr2=0x7f421c147760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:25.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f420affd700 1 --2- 172.21.15.40:0/3897194664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c1f2710 0x7f421c147760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f421c157950 tx=0x7f4210009f10 comp rx=0 tx=0).stop 2024-09-13T18:05:25.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f420affd700 1 -- 172.21.15.40:0/3897194664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c1f2710 msgr2=0x7f421c147760 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:05:25.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.041+0000 7f420affd700 1 --2- 172.21.15.40:0/3897194664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c1f2710 0x7f421c147760 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:25.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.045+0000 7f421ad9d700 1 --2- 172.21.15.40:0/3897194664 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f41fc066800 0x7f41fc068cc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f421c0c5f70 tx=0x7f4214009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:25.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.053+0000 7f421b59e700 1 --2- 172.21.15.40:0/3897194664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c1ee9f0 0x7f421c1eee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:25.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.053+0000 7f421b59e700 1 -- 172.21.15.40:0/3897194664 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421c0cb3a0 con 0x7f421c1ee9f0 2024-09-13T18:05:25.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.053+0000 7f4219d9b700 1 --2- 172.21.15.40:0/3897194664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c1ee9f0 0x7f421c1eee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:25.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.053+0000 7f4219d9b700 1 -- 172.21.15.40:0/3897194664 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f421c151960 con 0x7f421c1ee9f0 2024-09-13T18:05:25.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.053+0000 7f4219d9b700 1 --2- 172.21.15.40:0/3897194664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c1ee9f0 0x7f421c1eee00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f421c157950 tx=0x7f4210005ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:25.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.053+0000 7f420affd700 1 -- 172.21.15.40:0/3897194664 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421001aba0 con 0x7f421c1ee9f0 2024-09-13T18:05:25.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.053+0000 7f420affd700 1 -- 172.21.15.40:0/3897194664 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4937 (secure 0 0 0) 0x7f4210015470 con 0x7f421c1ee9f0 2024-09-13T18:05:25.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.053+0000 7f420affd700 1 -- 172.21.15.40:0/3897194664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c1ee9f0 msgr2=0x7f421c1eee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:25.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.053+0000 7f420affd700 1 --2- 172.21.15.40:0/3897194664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c1ee9f0 0x7f421c1eee00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f421c157950 tx=0x7f4210005ac0 comp rx=0 tx=0).stop 2024-09-13T18:05:25.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.053+0000 7f420affd700 1 -- 172.21.15.40:0/3897194664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c1ee9f0 msgr2=0x7f421c1eee00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:05:25.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.053+0000 7f420affd700 1 --2- 172.21.15.40:0/3897194664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f421c1ee9f0 0x7f421c1eee00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:25.063 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:05:25.063 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:05:25.063 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:05:25.063 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:05:25.064 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 122, 2024-09-13T18:05:25.064 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:05:25.064 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:05:25.064 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:05:25.064 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:25.064 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:05:25.064 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:05:25.064 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:05:25.064 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:05:25.064 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:25.064 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:25.064 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:25.064 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:25.064 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:25.065 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:25.065 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:25.065 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:25.065 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:25.065 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:25.065 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:25.065 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:05:25.065 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:05:25.065 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:25.065 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:25.065 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:25.065 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:25.065 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:25.066 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:25.066 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:25.066 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:25.066 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:25.066 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:25.066 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:25.066 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:25.066 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:05:25.066 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:05:25.066 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:05:25.066 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:05:25.066 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:05:25.066 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:05:25.066 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:05:25.067 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:05:25.067 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:05:25.067 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:05:25.067 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:05:25.067 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:05:25.067 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:05:25.067 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:05:25.067 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:05:25.067 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:05:25.067 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:05:25.067 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:25.067 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:25.067 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:25.067 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:25.068 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:25.068 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:25.068 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:25.068 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:25.068 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:25.068 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:25.068 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:25.068 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:05:25.068 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:25.068 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:05:25.068 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:25.068 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:05:25.068 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:05:25.069 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:25.069 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:25.069 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:25.069 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:25.069 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:05:25.069 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:25.069 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:25.069 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:25.069 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:25.069 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:05:25.069 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:25.069 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:25.069 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:25.069 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:25.070 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:05:25.070 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:05:25.070 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:25.070 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:25.070 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:25.070 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:25.070 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:25.070 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:05:25.070 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:05:25.070 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:25.070 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:25.070 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:25.070 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:25.071 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:05:25.071 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:25.071 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:25.071 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:25.071 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:25.071 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:05:25.071 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:25.071 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:25.071 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:25.071 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:25.071 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:05:25.071 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:05:25.071 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:25.071 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:25.072 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:25.072 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:25.072 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:25.072 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:05:25.072 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:05:25.072 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:25.072 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:25.072 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:25.072 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:25.072 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:05:25.072 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:25.072 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:25.072 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:25.072 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:25.073 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:05:25.073 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:25.073 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:25.073 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:25.073 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:25.073 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:05:25.073 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:05:25.073 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:25.073 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:25.073 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:25.073 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:25.073 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:25.073 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:25.073 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:05:25.074 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:05:25.074 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:25.074 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:25.074 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:25.074 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:05:25.074 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:25.074 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:25.074 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:05:25.074 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:25.074 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:25.074 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:25.074 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:05:25.074 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:25.074 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:25.075 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:05:25.075 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:25.075 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:25.075 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:25.075 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:05:25.075 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:25.075 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:25.076 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:05:25.076 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:25.076 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:25.076 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:25.076 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:05:25.076 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:25.076 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:25.076 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:05:25.076 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:25.076 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:25.076 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:25.076 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:05:25.076 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:25.076 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:25.077 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:25.077 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:05:25.077 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:05:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.057+0000 7f421b59e700 1 -- 172.21.15.40:0/3897194664 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f41fc066800 msgr2=0x7f41fc068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.057+0000 7f421b59e700 1 --2- 172.21.15.40:0/3897194664 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f41fc066800 0x7f41fc068cc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f421c0c5f70 tx=0x7f4214009000 comp rx=0 tx=0).stop 2024-09-13T18:05:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.057+0000 7f421b59e700 1 -- 172.21.15.40:0/3897194664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c071260 msgr2=0x7f421c0cb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.057+0000 7f421b59e700 1 --2- 172.21.15.40:0/3897194664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f421c071260 0x7f421c0cb6b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f420c00ebe0 tx=0x7f420c00eef0 comp rx=0 tx=0).stop 2024-09-13T18:05:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.057+0000 7f421ad9d700 1 -- 172.21.15.40:0/3897194664 reap_dead start 2024-09-13T18:05:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.057+0000 7f421ad9d700 1 -- 172.21.15.40:0/3897194664 reap_dead start 2024-09-13T18:05:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.057+0000 7f421b59e700 1 -- 172.21.15.40:0/3897194664 shutdown_connections 2024-09-13T18:05:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.057+0000 7f421b59e700 1 -- 172.21.15.40:0/3897194664 >> 172.21.15.40:0/3897194664 conn(0x7f421c06e6d0 msgr2=0x7f421c14eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.057+0000 7f421b59e700 1 -- 172.21.15.40:0/3897194664 shutdown_connections 2024-09-13T18:05:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:25.057+0000 7f421b59e700 1 -- 172.21.15.40:0/3897194664 wait complete. 2024-09-13T18:05:25.079 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:05:25.079 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:05:36.593 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:05:36.596 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:05:45.080 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-13T18:05:45.080 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-13T18:05:45.080 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T18:05:45.080 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T18:05:45.083 INFO:tasks.ceph.mon.b:Started 2024-09-13T18:05:45.083 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:05:45.083 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:05:45.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.200+0000 7fd07759e700 1 Processor -- start 2024-09-13T18:05:45.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.200+0000 7fd07759e700 1 -- start start 2024-09-13T18:05:45.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.200+0000 7fd07759e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd078151210 0x7fd07806d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:45.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07759e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd07806ec70 con 0x7fd07806d9d0 2024-09-13T18:05:45.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07759e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd078158070 con 0x7fd0781515f0 2024-09-13T18:05:45.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07759e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd078157210 con 0x7fd078151210 2024-09-13T18:05:45.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd076d9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd07806d9d0 0x7fd078070cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45868/0 (socket says 172.21.15.40:45868) 2024-09-13T18:05:45.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd076d9d700 1 -- 172.21.15.40:0/3697676649 learned_addr learned my addr 172.21.15.40:0/3697676649 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:05:45.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07659c700 1 --2- 172.21.15.40:0/3697676649 >> v2:172.21.15.43:3300/0 conn(0x7fd078151210 0x7fd07806d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07659c700 1 -- 172.21.15.40:0/3697676649 >> v2:172.21.15.43:3300/0 conn(0x7fd078151210 msgr2=0x7fd07806d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:05:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07659c700 1 -- 172.21.15.40:0/3697676649 >> v2:172.21.15.43:3300/0 conn(0x7fd078151210 msgr2=0x7fd07806d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:05:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07659c700 1 --2- 172.21.15.40:0/3697676649 >> v2:172.21.15.43:3300/0 conn(0x7fd078151210 0x7fd07806d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:05:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07659c700 1 --2- 172.21.15.40:0/3697676649 >> v2:172.21.15.43:3300/0 conn(0x7fd078151210 0x7fd07806d490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07559a700 1 -- 172.21.15.40:0/3697676649 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223066614 0 0) 0x7fd07806ec70 con 0x7fd07806d9d0 2024-09-13T18:05:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07559a700 1 -- 172.21.15.40:0/3697676649 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0600035d0 con 0x7fd07806d9d0 2024-09-13T18:05:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07559a700 1 -- 172.21.15.40:0/3697676649 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4147739955 0 0) 0x7fd0600035d0 con 0x7fd07806d9d0 2024-09-13T18:05:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07559a700 1 -- 172.21.15.40:0/3697676649 >> v2:172.21.15.43:3300/0 conn(0x7fd078151210 msgr2=0x7fd07806d490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:05:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07559a700 1 --2- 172.21.15.40:0/3697676649 >> v2:172.21.15.43:3300/0 conn(0x7fd078151210 0x7fd07806d490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07559a700 1 -- 172.21.15.40:0/3697676649 >> v1:172.21.15.43:6789/0 conn(0x7fd0781515f0 legacy=0x7fd0780c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07559a700 1 -- 172.21.15.40:0/3697676649 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0781574f0 con 0x7fd07806d9d0 2024-09-13T18:05:45.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07559a700 1 -- 172.21.15.40:0/3697676649 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fd0700019f0 con 0x7fd07806d9d0 2024-09-13T18:05:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07559a700 1 -- 172.21.15.40:0/3697676649 >> v1:172.21.15.40:6789/0 conn(0x7fd07806d9d0 legacy=0x7fd078070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07559a700 1 --2- 172.21.15.40:0/3697676649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd060003a20 0x7fd060003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07559a700 1 --2- 172.21.15.40:0/3697676649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd060004870 0x7fd060004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07559a700 1 --2- 172.21.15.40:0/3697676649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd060005230 0x7fd06000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07559a700 1 -- 172.21.15.40:0/3697676649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd078157210 con 0x7fd060003a20 2024-09-13T18:05:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07559a700 1 -- 172.21.15.40:0/3697676649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06000c160 con 0x7fd060005230 2024-09-13T18:05:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07559a700 1 -- 172.21.15.40:0/3697676649 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06000c320 con 0x7fd060004870 2024-09-13T18:05:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07659c700 1 --2- 172.21.15.40:0/3697676649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd060003a20 0x7fd060003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd075d9b700 1 --2- 172.21.15.40:0/3697676649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd060004870 0x7fd060004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd076d9d700 1 --2- 172.21.15.40:0/3697676649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd060005230 0x7fd06000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07659c700 1 -- 172.21.15.40:0/3697676649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd060004870 msgr2=0x7fd060004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:05:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07659c700 1 --2- 172.21.15.40:0/3697676649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd060004870 0x7fd060004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:45.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07659c700 1 -- 172.21.15.40:0/3697676649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd060005230 msgr2=0x7fd06000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:05:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07659c700 1 --2- 172.21.15.40:0/3697676649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd060005230 0x7fd06000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07659c700 1 -- 172.21.15.40:0/3697676649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0600035d0 con 0x7fd060003a20 2024-09-13T18:05:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd076d9d700 1 --2- 172.21.15.40:0/3697676649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd060005230 0x7fd06000ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:05:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd076d9d700 1 -- 172.21.15.40:0/3697676649 reap_dead start 2024-09-13T18:05:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07659c700 1 --2- 172.21.15.40:0/3697676649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd060003a20 0x7fd060003e40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd068009870 tx=0x7fd0600058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e701457e6cf19128 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07559a700 1 -- 172.21.15.40:0/3697676649 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06800c070 con 0x7fd060003a20 2024-09-13T18:05:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07559a700 1 -- 172.21.15.40:0/3697676649 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd068013630 con 0x7fd060003a20 2024-09-13T18:05:45.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07559a700 1 -- 172.21.15.40:0/3697676649 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0680139e0 con 0x7fd060003a20 2024-09-13T18:05:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07759e700 1 -- 172.21.15.40:0/3697676649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd060003a20 msgr2=0x7fd060003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07759e700 1 --2- 172.21.15.40:0/3697676649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd060003a20 0x7fd060003e40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd068009870 tx=0x7fd0600058a0 comp rx=0 tx=0).stop 2024-09-13T18:05:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07759e700 1 -- 172.21.15.40:0/3697676649 shutdown_connections 2024-09-13T18:05:45.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07759e700 1 --2- 172.21.15.40:0/3697676649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd060003a20 0x7fd060003e40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:45.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07759e700 1 -- 172.21.15.40:0/3697676649 >> 172.21.15.40:0/3697676649 conn(0x7fd07806a390 msgr2=0x7fd07806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:45.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07759e700 1 -- 172.21.15.40:0/3697676649 shutdown_connections 2024-09-13T18:05:45.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.204+0000 7fd07759e700 1 -- 172.21.15.40:0/3697676649 wait complete. 2024-09-13T18:05:45.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd07759e700 1 Processor -- start 2024-09-13T18:05:45.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd07759e700 1 -- start start 2024-09-13T18:05:45.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd07759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd060003a20 0x7fd07814a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:45.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd07759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0781edb10 0x7fd07814fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:45.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd07759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd078148f60 0x7fd0781493e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:45.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd07759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0781486e0 con 0x7fd0781edb10 2024-09-13T18:05:45.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd07759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07815ada0 con 0x7fd060003a20 2024-09-13T18:05:45.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd07759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd078149920 con 0x7fd078148f60 2024-09-13T18:05:45.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd075d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0781edb10 0x7fd07814fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd075d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0781edb10 0x7fd07814fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46644/0 (socket says 172.21.15.40:46644) 2024-09-13T18:05:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd075d9b700 1 -- 172.21.15.40:0/3116983474 learned_addr learned my addr 172.21.15.40:0/3116983474 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd07659c700 1 --2- 172.21.15.40:0/3116983474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd060003a20 0x7fd07814a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd076d9d700 1 --2- 172.21.15.40:0/3116983474 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd078148f60 0x7fd0781493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd07659c700 1 -- 172.21.15.40:0/3116983474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd060003a20 msgr2=0x7fd07814a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:05:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd07659c700 1 -- 172.21.15.40:0/3116983474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd060003a20 msgr2=0x7fd07814a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:05:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd07659c700 1 --2- 172.21.15.40:0/3116983474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd060003a20 0x7fd07814a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:05:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd07659c700 1 --2- 172.21.15.40:0/3116983474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd060003a20 0x7fd07814a330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd075d9b700 1 -- 172.21.15.40:0/3116983474 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd078148f60 msgr2=0x7fd0781493e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd075d9b700 1 --2- 172.21.15.40:0/3116983474 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd078148f60 0x7fd0781493e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd075d9b700 1 -- 172.21.15.40:0/3116983474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd060003a20 msgr2=0x7fd07814a330 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:05:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd075d9b700 1 --2- 172.21.15.40:0/3116983474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd060003a20 0x7fd07814a330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:45.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.208+0000 7fd075d9b700 1 -- 172.21.15.40:0/3116983474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd078147ad0 con 0x7fd0781edb10 2024-09-13T18:05:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.212+0000 7fd075d9b700 1 --2- 172.21.15.40:0/3116983474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0781edb10 0x7fd07814fcf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00f7f0 tx=0x7fd06c00fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.212+0000 7fd066ffd700 1 -- 172.21.15.40:0/3116983474 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c0013c0 con 0x7fd0781edb10 2024-09-13T18:05:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.212+0000 7fd066ffd700 1 -- 172.21.15.40:0/3116983474 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06c005320 con 0x7fd0781edb10 2024-09-13T18:05:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.212+0000 7fd07759e700 1 -- 172.21.15.40:0/3116983474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd078147db0 con 0x7fd0781edb10 2024-09-13T18:05:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.212+0000 7fd066ffd700 1 -- 172.21.15.40:0/3116983474 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c006aa0 con 0x7fd0781edb10 2024-09-13T18:05:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.212+0000 7fd07759e700 1 -- 172.21.15.40:0/3116983474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd078148370 con 0x7fd0781edb10 2024-09-13T18:05:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.212+0000 7fd07759e700 1 -- 172.21.15.40:0/3116983474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07806d490 con 0x7fd0781edb10 2024-09-13T18:05:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.212+0000 7fd066ffd700 1 -- 172.21.15.40:0/3116983474 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd06c006c00 con 0x7fd0781edb10 2024-09-13T18:05:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.212+0000 7fd066ffd700 1 --2- 172.21.15.40:0/3116983474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd058066730 0x7fd058068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.212+0000 7fd066ffd700 1 -- 172.21.15.40:0/3116983474 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd06c083cb0 con 0x7fd0781edb10 2024-09-13T18:05:45.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.212+0000 7fd07659c700 1 --2- 172.21.15.40:0/3116983474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd058066730 0x7fd058068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:45.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.216+0000 7fd07659c700 1 --2- 172.21.15.40:0/3116983474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd058066730 0x7fd058068bf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd068016040 tx=0x7fd068013000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:45.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.220+0000 7fd066ffd700 1 -- 172.21.15.40:0/3116983474 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd06c050c60 con 0x7fd0781edb10 2024-09-13T18:05:45.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.476+0000 7fd07759e700 1 -- 172.21.15.40:0/3116983474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fd0780c7fe0 con 0x7fd0781edb10 2024-09-13T18:05:45.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.480+0000 7fd066ffd700 1 -- 172.21.15.40:0/3116983474 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7fd06c01a080 con 0x7fd0781edb10 2024-09-13T18:05:45.487 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":122,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:05:45.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.480+0000 7fd07759e700 1 -- 172.21.15.40:0/3116983474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd058066730 msgr2=0x7fd058068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:45.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.480+0000 7fd07759e700 1 --2- 172.21.15.40:0/3116983474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd058066730 0x7fd058068bf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd068016040 tx=0x7fd068013000 comp rx=0 tx=0).stop 2024-09-13T18:05:45.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.480+0000 7fd07759e700 1 -- 172.21.15.40:0/3116983474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0781edb10 msgr2=0x7fd07814fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:45.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.480+0000 7fd07759e700 1 --2- 172.21.15.40:0/3116983474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0781edb10 0x7fd07814fcf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00f7f0 tx=0x7fd06c00fb00 comp rx=0 tx=0).stop 2024-09-13T18:05:45.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.480+0000 7fd07759e700 1 -- 172.21.15.40:0/3116983474 shutdown_connections 2024-09-13T18:05:45.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.480+0000 7fd07759e700 1 --2- 172.21.15.40:0/3116983474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd058066730 0x7fd058068bf0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:45.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.480+0000 7fd07759e700 1 --2- 172.21.15.40:0/3116983474 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd078148f60 0x7fd0781493e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:45.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.480+0000 7fd07759e700 1 --2- 172.21.15.40:0/3116983474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd0781edb10 0x7fd07814fcf0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:45.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.480+0000 7fd07759e700 1 --2- 172.21.15.40:0/3116983474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd060003a20 0x7fd07814a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:45.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.480+0000 7fd07759e700 1 -- 172.21.15.40:0/3116983474 >> 172.21.15.40:0/3116983474 conn(0x7fd07806a390 msgr2=0x7fd0780c8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:45.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.480+0000 7fd07759e700 1 -- 172.21.15.40:0/3116983474 shutdown_connections 2024-09-13T18:05:45.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:45.480+0000 7fd07759e700 1 -- 172.21.15.40:0/3116983474 wait complete. 2024-09-13T18:05:48.504 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:05:48.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.616+0000 7f756759e700 1 Processor -- start 2024-09-13T18:05:48.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.616+0000 7f756759e700 1 -- start start 2024-09-13T18:05:48.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7560036150 0x7f7560007840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:48.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756759e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7560033260 0x7f75600e5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:48.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756759e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75600db8a0 con 0x7f7560051190 2024-09-13T18:05:48.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756759e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75600067c0 con 0x7f7560033260 2024-09-13T18:05:48.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7560006aa0 con 0x7f7560036150 2024-09-13T18:05:48.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756659c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7560036150 0x7f7560007840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:48.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f7566d9d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7560033260 0x7f75600e5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:48.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f7565d9b700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7560051190 0x7f756004c3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:59700/0 (socket says 172.21.15.40:59700) 2024-09-13T18:05:48.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f7565d9b700 1 -- 172.21.15.40:0/624577643 learned_addr learned my addr 172.21.15.40:0/624577643 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:05:48.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f7566d9d700 1 -- 172.21.15.40:0/624577643 >> v2:172.21.15.43:3300/0 conn(0x7f7560033260 msgr2=0x7f75600e5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:05:48.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f7566d9d700 1 -- 172.21.15.40:0/624577643 >> v2:172.21.15.43:3300/0 conn(0x7f7560033260 msgr2=0x7f75600e5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:05:48.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f7566d9d700 1 --2- 172.21.15.40:0/624577643 >> v2:172.21.15.43:3300/0 conn(0x7f7560033260 0x7f75600e5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:05:48.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f7566d9d700 1 --2- 172.21.15.40:0/624577643 >> v2:172.21.15.43:3300/0 conn(0x7f7560033260 0x7f75600e5380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:48.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756659c700 1 -- 172.21.15.40:0/624577643 >> v2:172.21.15.43:3300/0 conn(0x7f7560033260 msgr2=0x7f75600e5380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:05:48.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756659c700 1 --2- 172.21.15.40:0/624577643 >> v2:172.21.15.43:3300/0 conn(0x7f7560033260 0x7f75600e5380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:48.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756659c700 1 -- 172.21.15.40:0/624577643 >> v1:172.21.15.43:6789/0 conn(0x7f7560051190 legacy=0x7f756004c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:48.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756659c700 1 -- 172.21.15.40:0/624577643 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7560006d80 con 0x7f7560036150 2024-09-13T18:05:48.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756659c700 1 --2- 172.21.15.40:0/624577643 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7560036150 0x7f7560007840 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f755800a9e0 tx=0x7f755800acf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cf4fdb0bca34785 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:48.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756559a700 1 -- 172.21.15.40:0/624577643 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558005e50 con 0x7f7560036150 2024-09-13T18:05:48.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756559a700 1 -- 172.21.15.40:0/624577643 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755800c760 con 0x7f7560036150 2024-09-13T18:05:48.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756559a700 1 -- 172.21.15.40:0/624577643 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755800caf0 con 0x7f7560036150 2024-09-13T18:05:48.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756759e700 1 -- 172.21.15.40:0/624577643 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7560036150 msgr2=0x7f7560007840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:48.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756759e700 1 --2- 172.21.15.40:0/624577643 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7560036150 0x7f7560007840 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f755800a9e0 tx=0x7f755800acf0 comp rx=0 tx=0).stop 2024-09-13T18:05:48.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756759e700 1 -- 172.21.15.40:0/624577643 shutdown_connections 2024-09-13T18:05:48.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756759e700 1 --2- 172.21.15.40:0/624577643 >> v2:172.21.15.43:3300/0 conn(0x7f7560033260 0x7f75600e5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:48.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756759e700 1 --2- 172.21.15.40:0/624577643 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7560036150 0x7f7560007840 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:48.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756759e700 1 -- 172.21.15.40:0/624577643 >> 172.21.15.40:0/624577643 conn(0x7f75600505e0 msgr2=0x7f75600509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:48.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.620+0000 7f756759e700 1 -- 172.21.15.40:0/624577643 shutdown_connections 2024-09-13T18:05:48.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f756759e700 1 -- 172.21.15.40:0/624577643 wait complete. 2024-09-13T18:05:48.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f756759e700 1 Processor -- start 2024-09-13T18:05:48.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f756759e700 1 -- start start 2024-09-13T18:05:48.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f756759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7560033260 0x7f75600dfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:48.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f756759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7560036150 0x7f7560047b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:48.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f756759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7560051190 0x7f7560049be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:48.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f756759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75600497c0 con 0x7f7560033260 2024-09-13T18:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f756759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75600d79a0 con 0x7f7560036150 2024-09-13T18:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f756759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75600d7c80 con 0x7f7560051190 2024-09-13T18:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f756659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7560033260 0x7f75600dfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f756659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7560033260 0x7f75600dfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46652/0 (socket says 172.21.15.40:46652) 2024-09-13T18:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f756659c700 1 -- 172.21.15.40:0/1376961162 learned_addr learned my addr 172.21.15.40:0/1376961162 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f7566d9d700 1 --2- 172.21.15.40:0/1376961162 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7560051190 0x7f7560049be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f7565d9b700 1 --2- 172.21.15.40:0/1376961162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7560036150 0x7f7560047b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f7565d9b700 1 -- 172.21.15.40:0/1376961162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7560036150 msgr2=0x7f7560047b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f7565d9b700 1 -- 172.21.15.40:0/1376961162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7560036150 msgr2=0x7f7560047b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f7565d9b700 1 --2- 172.21.15.40:0/1376961162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7560036150 0x7f7560047b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f7565d9b700 1 --2- 172.21.15.40:0/1376961162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7560036150 0x7f7560047b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:48.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f756659c700 1 -- 172.21.15.40:0/1376961162 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7560051190 msgr2=0x7f7560049be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:48.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f756659c700 1 --2- 172.21.15.40:0/1376961162 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7560051190 0x7f7560049be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:48.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f756659c700 1 -- 172.21.15.40:0/1376961162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7560036150 msgr2=0x7f7560047b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:05:48.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f756659c700 1 --2- 172.21.15.40:0/1376961162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7560036150 0x7f7560047b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:48.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f756659c700 1 -- 172.21.15.40:0/1376961162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755800a5f0 con 0x7f7560033260 2024-09-13T18:05:48.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f756659c700 1 --2- 172.21.15.40:0/1376961162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7560033260 0x7f75600dfb80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7558000c00 tx=0x7f7558004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:48.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f7556ffd700 1 -- 172.21.15.40:0/1376961162 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755801f070 con 0x7f7560033260 2024-09-13T18:05:48.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f756759e700 1 -- 172.21.15.40:0/1376961162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75600d7f60 con 0x7f7560033260 2024-09-13T18:05:48.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f756759e700 1 -- 172.21.15.40:0/1376961162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75600493e0 con 0x7f7560033260 2024-09-13T18:05:48.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f7556ffd700 1 -- 172.21.15.40:0/1376961162 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75580064c0 con 0x7f7560033260 2024-09-13T18:05:48.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.624+0000 7f7556ffd700 1 -- 172.21.15.40:0/1376961162 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558006870 con 0x7f7560033260 2024-09-13T18:05:48.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.628+0000 7f756759e700 1 -- 172.21.15.40:0/1376961162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f754c005310 con 0x7f7560033260 2024-09-13T18:05:48.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.628+0000 7f7556ffd700 1 -- 172.21.15.40:0/1376961162 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7558019450 con 0x7f7560033260 2024-09-13T18:05:48.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.628+0000 7f7556ffd700 1 --2- 172.21.15.40:0/1376961162 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7548066730 0x7f7548068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:48.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.628+0000 7f7556ffd700 1 -- 172.21.15.40:0/1376961162 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7558085980 con 0x7f7560033260 2024-09-13T18:05:48.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.628+0000 7f7565d9b700 1 --2- 172.21.15.40:0/1376961162 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7548066730 0x7f7548068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:48.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.632+0000 7f7565d9b700 1 --2- 172.21.15.40:0/1376961162 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7548066730 0x7f7548068bf0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f75600d99c0 tx=0x7f755c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:48.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.636+0000 7f7556ffd700 1 -- 172.21.15.40:0/1376961162 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f75580528b0 con 0x7f7560033260 2024-09-13T18:05:48.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.900+0000 7f756759e700 1 -- 172.21.15.40:0/1376961162 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f754c0055c0 con 0x7f7560033260 2024-09-13T18:05:48.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.900+0000 7f7556ffd700 1 -- 172.21.15.40:0/1376961162 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f7558055ed0 con 0x7f7560033260 2024-09-13T18:05:48.910 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":122,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:05:48.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.904+0000 7f756759e700 1 -- 172.21.15.40:0/1376961162 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7548066730 msgr2=0x7f7548068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:48.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.904+0000 7f756759e700 1 --2- 172.21.15.40:0/1376961162 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7548066730 0x7f7548068bf0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f75600d99c0 tx=0x7f755c009040 comp rx=0 tx=0).stop 2024-09-13T18:05:48.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.904+0000 7f756759e700 1 -- 172.21.15.40:0/1376961162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7560033260 msgr2=0x7f75600dfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:48.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.904+0000 7f756759e700 1 --2- 172.21.15.40:0/1376961162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7560033260 0x7f75600dfb80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7558000c00 tx=0x7f7558004770 comp rx=0 tx=0).stop 2024-09-13T18:05:48.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.904+0000 7f756759e700 1 -- 172.21.15.40:0/1376961162 shutdown_connections 2024-09-13T18:05:48.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.904+0000 7f756759e700 1 --2- 172.21.15.40:0/1376961162 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7548066730 0x7f7548068bf0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:48.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.904+0000 7f756759e700 1 --2- 172.21.15.40:0/1376961162 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7560051190 0x7f7560049be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:48.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.904+0000 7f756759e700 1 --2- 172.21.15.40:0/1376961162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7560036150 0x7f7560047b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:48.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.904+0000 7f756759e700 1 --2- 172.21.15.40:0/1376961162 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7560033260 0x7f75600dfb80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:48.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.904+0000 7f756759e700 1 -- 172.21.15.40:0/1376961162 >> 172.21.15.40:0/1376961162 conn(0x7f75600505e0 msgr2=0x7f756004c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:48.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.904+0000 7f756759e700 1 -- 172.21.15.40:0/1376961162 shutdown_connections 2024-09-13T18:05:48.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:48.904+0000 7f756759e700 1 -- 172.21.15.40:0/1376961162 wait complete. 2024-09-13T18:05:51.925 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:05:52.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549316c700 1 Processor -- start 2024-09-13T18:05:52.051 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549316c700 1 -- start start 2024-09-13T18:05:52.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549316c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f548c1534a0 0x7f548c070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549316c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f548c0c6880 0x7f548c0c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549316c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548c159490 con 0x7f548c0c7f70 2024-09-13T18:05:52.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549316c700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f548c06c670 con 0x7f548c0c6880 2024-09-13T18:05:52.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549316c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548c0c71f0 con 0x7f548c1534a0 2024-09-13T18:05:52.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f5491969700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f548c0c7f70 0x7f548c06e3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47974/0 (socket says 172.21.15.40:47974) 2024-09-13T18:05:52.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f5491969700 1 -- 172.21.15.40:0/3795710234 learned_addr learned my addr 172.21.15.40:0/3795710234 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:05:52.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549216a700 1 --2- 172.21.15.40:0/3795710234 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f548c1534a0 0x7f548c070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549296b700 1 --2- 172.21.15.40:0/3795710234 >> v2:172.21.15.43:3300/0 conn(0x7f548c0c6880 0x7f548c0c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.057 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f5491168700 1 -- 172.21.15.40:0/3795710234 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3085782181 0 0) 0x7f548c159490 con 0x7f548c0c7f70 2024-09-13T18:05:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f5491168700 1 -- 172.21.15.40:0/3795710234 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54780035d0 con 0x7f548c0c7f70 2024-09-13T18:05:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549296b700 1 -- 172.21.15.40:0/3795710234 >> v2:172.21.15.43:3300/0 conn(0x7f548c0c6880 msgr2=0x7f548c0c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:05:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549296b700 1 -- 172.21.15.40:0/3795710234 >> v2:172.21.15.43:3300/0 conn(0x7f548c0c6880 msgr2=0x7f548c0c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:05:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549296b700 1 --2- 172.21.15.40:0/3795710234 >> v2:172.21.15.43:3300/0 conn(0x7f548c0c6880 0x7f548c0c6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:05:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549296b700 1 --2- 172.21.15.40:0/3795710234 >> v2:172.21.15.43:3300/0 conn(0x7f548c0c6880 0x7f548c0c6cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f5491168700 1 -- 172.21.15.40:0/3795710234 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3981465652 0 0) 0x7f54780035d0 con 0x7f548c0c7f70 2024-09-13T18:05:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f5491168700 1 -- 172.21.15.40:0/3795710234 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f548c1534a0 msgr2=0x7f548c070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:05:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f5491168700 1 --2- 172.21.15.40:0/3795710234 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f548c1534a0 0x7f548c070ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f5491168700 1 -- 172.21.15.40:0/3795710234 >> v2:172.21.15.43:3300/0 conn(0x7f548c0c6880 msgr2=0x7f548c0c6cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:05:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f5491168700 1 --2- 172.21.15.40:0/3795710234 >> v2:172.21.15.43:3300/0 conn(0x7f548c0c6880 0x7f548c0c6cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f5491168700 1 -- 172.21.15.40:0/3795710234 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c159710 con 0x7f548c0c7f70 2024-09-13T18:05:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f5491168700 1 -- 172.21.15.40:0/3795710234 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f547c001670 con 0x7f548c0c7f70 2024-09-13T18:05:52.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f5491168700 1 -- 172.21.15.40:0/3795710234 >> v1:172.21.15.40:6789/0 conn(0x7f548c0c7f70 legacy=0x7f548c06e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f5491168700 1 --2- 172.21.15.40:0/3795710234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5478003a20 0x7f5478003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f5491168700 1 --2- 172.21.15.40:0/3795710234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5478004870 0x7f5478004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f5491168700 1 --2- 172.21.15.40:0/3795710234 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f548c1534a0 0x7f5478005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f5491168700 1 -- 172.21.15.40:0/3795710234 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c06c670 con 0x7f5478003a20 2024-09-13T18:05:52.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f5491168700 1 -- 172.21.15.40:0/3795710234 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c0c71f0 con 0x7f5478004870 2024-09-13T18:05:52.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f5491168700 1 -- 172.21.15.40:0/3795710234 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547800bae0 con 0x7f548c1534a0 2024-09-13T18:05:52.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549216a700 1 --2- 172.21.15.40:0/3795710234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5478003a20 0x7f5478003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549296b700 1 --2- 172.21.15.40:0/3795710234 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f548c1534a0 0x7f5478005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f5491969700 1 --2- 172.21.15.40:0/3795710234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5478004870 0x7f5478004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549216a700 1 -- 172.21.15.40:0/3795710234 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f548c1534a0 msgr2=0x7f5478005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:05:52.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549216a700 1 --2- 172.21.15.40:0/3795710234 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f548c1534a0 0x7f5478005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549216a700 1 -- 172.21.15.40:0/3795710234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5478004870 msgr2=0x7f5478004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:05:52.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549216a700 1 --2- 172.21.15.40:0/3795710234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5478004870 0x7f5478004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549216a700 1 -- 172.21.15.40:0/3795710234 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54780035d0 con 0x7f5478003a20 2024-09-13T18:05:52.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f5491969700 1 --2- 172.21.15.40:0/3795710234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5478004870 0x7f5478004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:05:52.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.044+0000 7f549216a700 1 --2- 172.21.15.40:0/3795710234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5478003a20 0x7f5478003e40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5488009b70 tx=0x7f54780058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3610fec05346dfe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.048+0000 7f5491168700 1 -- 172.21.15.40:0/3795710234 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548800e6c0 con 0x7f5478003a20 2024-09-13T18:05:52.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.048+0000 7f5491168700 1 -- 172.21.15.40:0/3795710234 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548800e820 con 0x7f5478003a20 2024-09-13T18:05:52.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.048+0000 7f5491168700 1 -- 172.21.15.40:0/3795710234 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548800eb90 con 0x7f5478003a20 2024-09-13T18:05:52.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.048+0000 7f549316c700 1 -- 172.21.15.40:0/3795710234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5478003a20 msgr2=0x7f5478003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.048+0000 7f549316c700 1 --2- 172.21.15.40:0/3795710234 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5478003a20 0x7f5478003e40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5488009b70 tx=0x7f54780058a0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.048+0000 7f549296b700 1 -- 172.21.15.40:0/3795710234 reap_dead start 2024-09-13T18:05:52.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.048+0000 7f549316c700 1 -- 172.21.15.40:0/3795710234 shutdown_connections 2024-09-13T18:05:52.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.048+0000 7f549316c700 1 -- 172.21.15.40:0/3795710234 >> 172.21.15.40:0/3795710234 conn(0x7f548c0cbd80 msgr2=0x7f548c0cc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:52.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.048+0000 7f549316c700 1 -- 172.21.15.40:0/3795710234 shutdown_connections 2024-09-13T18:05:52.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.048+0000 7f549316c700 1 -- 172.21.15.40:0/3795710234 wait complete. 2024-09-13T18:05:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.048+0000 7f549316c700 1 Processor -- start 2024-09-13T18:05:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.048+0000 7f549316c700 1 -- start start 2024-09-13T18:05:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549316c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5478003a20 0x7f548c1e23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549316c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f548c1e2930 0x7f548c14e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549316c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548c1e3c80 0x7f548c14e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549316c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c14fd20 con 0x7f5478003a20 2024-09-13T18:05:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549316c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c1dff20 con 0x7f548c1e3c80 2024-09-13T18:05:52.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549316c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548c158100 con 0x7f548c1e2930 2024-09-13T18:05:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549216a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5478003a20 0x7f548c1e23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549216a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5478003a20 0x7f548c1e23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47670/0 (socket says 172.21.15.40:47670) 2024-09-13T18:05:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549216a700 1 -- 172.21.15.40:0/2693394262 learned_addr learned my addr 172.21.15.40:0/2693394262 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549296b700 1 --2- 172.21.15.40:0/2693394262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548c1e3c80 0x7f548c14e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f5491969700 1 --2- 172.21.15.40:0/2693394262 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f548c1e2930 0x7f548c14e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549296b700 1 -- 172.21.15.40:0/2693394262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548c1e3c80 msgr2=0x7f548c14e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:05:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549296b700 1 -- 172.21.15.40:0/2693394262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548c1e3c80 msgr2=0x7f548c14e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:05:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549296b700 1 --2- 172.21.15.40:0/2693394262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548c1e3c80 0x7f548c14e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:05:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549216a700 1 -- 172.21.15.40:0/2693394262 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f548c1e2930 msgr2=0x7f548c14e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549216a700 1 --2- 172.21.15.40:0/2693394262 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f548c1e2930 0x7f548c14e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549216a700 1 -- 172.21.15.40:0/2693394262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548c1e3c80 msgr2=0x7f548c14e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549296b700 1 --2- 172.21.15.40:0/2693394262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548c1e3c80 0x7f548c14e9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549216a700 1 --2- 172.21.15.40:0/2693394262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548c1e3c80 0x7f548c14e9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549216a700 1 -- 172.21.15.40:0/2693394262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c1e45c0 con 0x7f5478003a20 2024-09-13T18:05:52.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549216a700 1 --2- 172.21.15.40:0/2693394262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5478003a20 0x7f548c1e23f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f547800ba30 tx=0x7f54880047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f5482ffd700 1 -- 172.21.15.40:0/2693394262 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548800e6c0 con 0x7f5478003a20 2024-09-13T18:05:52.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f5482ffd700 1 -- 172.21.15.40:0/2693394262 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5488013e70 con 0x7f5478003a20 2024-09-13T18:05:52.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549316c700 1 -- 172.21.15.40:0/2693394262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548c1e1040 con 0x7f5478003a20 2024-09-13T18:05:52.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f5482ffd700 1 -- 172.21.15.40:0/2693394262 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488004250 con 0x7f5478003a20 2024-09-13T18:05:52.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549316c700 1 -- 172.21.15.40:0/2693394262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548c1e1600 con 0x7f5478003a20 2024-09-13T18:05:52.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.052+0000 7f549316c700 1 -- 172.21.15.40:0/2693394262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f548c1e1820 con 0x7f5478003a20 2024-09-13T18:05:52.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.056+0000 7f5482ffd700 1 -- 172.21.15.40:0/2693394262 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f54880043b0 con 0x7f5478003a20 2024-09-13T18:05:52.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.056+0000 7f5482ffd700 1 --2- 172.21.15.40:0/2693394262 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5470066730 0x7f5470068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.056+0000 7f5482ffd700 1 -- 172.21.15.40:0/2693394262 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5488086430 con 0x7f5478003a20 2024-09-13T18:05:52.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.056+0000 7f5491969700 1 --2- 172.21.15.40:0/2693394262 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5470066730 0x7f5470068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.056+0000 7f5491969700 1 --2- 172.21.15.40:0/2693394262 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5470066730 0x7f5470068bf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f547c000c00 tx=0x7f547c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.060+0000 7f5482ffd700 1 -- 172.21.15.40:0/2693394262 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5488053670 con 0x7f5478003a20 2024-09-13T18:05:52.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.336+0000 7f549316c700 1 -- 172.21.15.40:0/2693394262 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f548c072830 con 0x7f5478003a20 2024-09-13T18:05:52.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.336+0000 7f5482ffd700 1 -- 172.21.15.40:0/2693394262 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f5488017620 con 0x7f5478003a20 2024-09-13T18:05:52.344 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":124,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:05:52.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.340+0000 7f549316c700 1 -- 172.21.15.40:0/2693394262 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5470066730 msgr2=0x7f5470068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.340+0000 7f549316c700 1 --2- 172.21.15.40:0/2693394262 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5470066730 0x7f5470068bf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f547c000c00 tx=0x7f547c009040 comp rx=0 tx=0).stop 2024-09-13T18:05:52.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.340+0000 7f549316c700 1 -- 172.21.15.40:0/2693394262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5478003a20 msgr2=0x7f548c1e23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.340+0000 7f549316c700 1 --2- 172.21.15.40:0/2693394262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5478003a20 0x7f548c1e23f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f547800ba30 tx=0x7f54880047e0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.340+0000 7f549316c700 1 -- 172.21.15.40:0/2693394262 shutdown_connections 2024-09-13T18:05:52.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.340+0000 7f549316c700 1 --2- 172.21.15.40:0/2693394262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f548c1e3c80 0x7f548c14e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.340+0000 7f549316c700 1 --2- 172.21.15.40:0/2693394262 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5470066730 0x7f5470068bf0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.340+0000 7f549316c700 1 --2- 172.21.15.40:0/2693394262 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f548c1e2930 0x7f548c14e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.340+0000 7f549316c700 1 --2- 172.21.15.40:0/2693394262 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5478003a20 0x7f548c1e23f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.340+0000 7f549316c700 1 -- 172.21.15.40:0/2693394262 >> 172.21.15.40:0/2693394262 conn(0x7f548c0cbd80 msgr2=0x7f548c156d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:52.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.340+0000 7f549316c700 1 -- 172.21.15.40:0/2693394262 shutdown_connections 2024-09-13T18:05:52.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.340+0000 7f549316c700 1 -- 172.21.15.40:0/2693394262 wait complete. 2024-09-13T18:05:52.365 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:05:52.365 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:05:52.365 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:05:52.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.464+0000 7f32b1747700 1 Processor -- start 2024-09-13T18:05:52.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.464+0000 7f32b1747700 1 -- start start 2024-09-13T18:05:52.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32b1747700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f32ac153a90 0x7f32ac0c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32b1747700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f32ac0704a0 0x7f32ac0c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32b1747700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f32ac15d2d0 0x7f32ac15d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32b1747700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32ac153e70 con 0x7f32ac153a90 2024-09-13T18:05:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32b1747700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32ac15ce40 con 0x7f32ac15d2d0 2024-09-13T18:05:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32b1747700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac0c88a0 con 0x7f32ac0704a0 2024-09-13T18:05:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32abfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f32ac153a90 0x7f32ac0c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32abfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f32ac153a90 0x7f32ac0c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47680/0 (socket says 172.21.15.40:47680) 2024-09-13T18:05:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32abfff700 1 -- 172.21.15.40:0/3328482701 learned_addr learned my addr 172.21.15.40:0/3328482701 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32b0f46700 1 --2- 172.21.15.40:0/3328482701 >> v2:172.21.15.43:3300/0 conn(0x7f32ac15d2d0 0x7f32ac15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32a3fff700 1 --2- 172.21.15.40:0/3328482701 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f32ac0704a0 0x7f32ac0c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32abfff700 1 -- 172.21.15.40:0/3328482701 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f32ac0704a0 msgr2=0x7f32ac0c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:05:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32abfff700 1 --2- 172.21.15.40:0/3328482701 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f32ac0704a0 0x7f32ac0c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32b0f46700 1 -- 172.21.15.40:0/3328482701 >> v2:172.21.15.43:3300/0 conn(0x7f32ac15d2d0 msgr2=0x7f32ac15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:05:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32abfff700 1 -- 172.21.15.40:0/3328482701 >> v2:172.21.15.43:3300/0 conn(0x7f32ac15d2d0 msgr2=0x7f32ac15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:05:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32b0f46700 1 -- 172.21.15.40:0/3328482701 >> v2:172.21.15.43:3300/0 conn(0x7f32ac15d2d0 msgr2=0x7f32ac15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:05:52.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32b0f46700 1 --2- 172.21.15.40:0/3328482701 >> v2:172.21.15.43:3300/0 conn(0x7f32ac15d2d0 0x7f32ac15d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:05:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32b0f46700 1 --2- 172.21.15.40:0/3328482701 >> v2:172.21.15.43:3300/0 conn(0x7f32ac15d2d0 0x7f32ac15d700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32abfff700 1 --2- 172.21.15.40:0/3328482701 >> v2:172.21.15.43:3300/0 conn(0x7f32ac15d2d0 0x7f32ac15d700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32abfff700 1 -- 172.21.15.40:0/3328482701 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac15a280 con 0x7f32ac153a90 2024-09-13T18:05:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32abfff700 1 --2- 172.21.15.40:0/3328482701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac153a90 0x7f32ac0c94a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f32ac06f370 tx=0x7f329c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa7b6a366f1c8f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32ab7fe700 1 -- 172.21.15.40:0/3328482701 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c00b8a0 con 0x7f32ac153a90 2024-09-13T18:05:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32ab7fe700 1 -- 172.21.15.40:0/3328482701 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329c00ba00 con 0x7f32ac153a90 2024-09-13T18:05:52.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32ab7fe700 1 -- 172.21.15.40:0/3328482701 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c00bd90 con 0x7f32ac153a90 2024-09-13T18:05:52.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32b1747700 1 -- 172.21.15.40:0/3328482701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac153a90 msgr2=0x7f32ac0c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32b1747700 1 --2- 172.21.15.40:0/3328482701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac153a90 0x7f32ac0c94a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f32ac06f370 tx=0x7f329c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32b1747700 1 -- 172.21.15.40:0/3328482701 shutdown_connections 2024-09-13T18:05:52.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32b1747700 1 --2- 172.21.15.40:0/3328482701 >> v2:172.21.15.43:3300/0 conn(0x7f32ac15d2d0 0x7f32ac15d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32b1747700 1 --2- 172.21.15.40:0/3328482701 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f32ac0704a0 0x7f32ac0c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32b1747700 1 --2- 172.21.15.40:0/3328482701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac153a90 0x7f32ac0c94a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32b1747700 1 -- 172.21.15.40:0/3328482701 >> 172.21.15.40:0/3328482701 conn(0x7f32ac0ccef0 msgr2=0x7f32ac0cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:52.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32b1747700 1 -- 172.21.15.40:0/3328482701 shutdown_connections 2024-09-13T18:05:52.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.468+0000 7f32b1747700 1 -- 172.21.15.40:0/3328482701 wait complete. 2024-09-13T18:05:52.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32b1747700 1 Processor -- start 2024-09-13T18:05:52.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32b1747700 1 -- start start 2024-09-13T18:05:52.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32b1747700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac0704a0 0x7f32ac1e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32b1747700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32ac153a90 0x7f32ac1ef080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32b1747700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f32ac15d2d0 0x7f32ac1e2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32b1747700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac1f2ff0 con 0x7f32ac0704a0 2024-09-13T18:05:52.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32b1747700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac15c180 con 0x7f32ac153a90 2024-09-13T18:05:52.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32b1747700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac15a9f0 con 0x7f32ac15d2d0 2024-09-13T18:05:52.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32abfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac0704a0 0x7f32ac1e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32abfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac0704a0 0x7f32ac1e5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47688/0 (socket says 172.21.15.40:47688) 2024-09-13T18:05:52.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32abfff700 1 -- 172.21.15.40:0/3997947680 learned_addr learned my addr 172.21.15.40:0/3997947680 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:52.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32a3fff700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32ac153a90 0x7f32ac1ef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32b0f46700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f32ac15d2d0 0x7f32ac1e2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32a3fff700 1 -- 172.21.15.40:0/3997947680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32ac153a90 msgr2=0x7f32ac1ef080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:05:52.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32a3fff700 1 -- 172.21.15.40:0/3997947680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32ac153a90 msgr2=0x7f32ac1ef080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:05:52.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32a3fff700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32ac153a90 0x7f32ac1ef080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:05:52.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32a3fff700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32ac153a90 0x7f32ac1ef080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:52.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32abfff700 1 -- 172.21.15.40:0/3997947680 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f32ac15d2d0 msgr2=0x7f32ac1e2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32abfff700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f32ac15d2d0 0x7f32ac1e2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32abfff700 1 -- 172.21.15.40:0/3997947680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32ac153a90 msgr2=0x7f32ac1ef080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:05:52.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32abfff700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32ac153a90 0x7f32ac1ef080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32abfff700 1 -- 172.21.15.40:0/3997947680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329c009570 con 0x7f32ac0704a0 2024-09-13T18:05:52.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32abfff700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac0704a0 0x7f32ac1e5b80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f32ac074750 tx=0x7f329c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32a97fa700 1 -- 172.21.15.40:0/3997947680 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c0036c0 con 0x7f32ac0704a0 2024-09-13T18:05:52.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32a97fa700 1 -- 172.21.15.40:0/3997947680 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329c01f070 con 0x7f32ac0704a0 2024-09-13T18:05:52.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32a97fa700 1 -- 172.21.15.40:0/3997947680 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c003a20 con 0x7f32ac0704a0 2024-09-13T18:05:52.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32b1747700 1 -- 172.21.15.40:0/3997947680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ac1f2690 con 0x7f32ac0704a0 2024-09-13T18:05:52.484 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.472+0000 7f32b1747700 1 -- 172.21.15.40:0/3997947680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ac1f2930 con 0x7f32ac0704a0 2024-09-13T18:05:52.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.476+0000 7f32b1747700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac15bc00 0x7f32ac1e2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.476+0000 7f32b1747700 1 -- 172.21.15.40:0/3997947680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac074830 con 0x7f32ac15bc00 2024-09-13T18:05:52.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.476+0000 7f32a3fff700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac15bc00 0x7f32ac1e2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.476+0000 7f32a3fff700 1 -- 172.21.15.40:0/3997947680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f32ac0c3c80 con 0x7f32ac15bc00 2024-09-13T18:05:52.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.476+0000 7f32a3fff700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac15bc00 0x7f32ac1e2950 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f32ac1e34c0 tx=0x7f3294010b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.476+0000 7f32a97fa700 1 -- 172.21.15.40:0/3997947680 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3294015880 con 0x7f32ac15bc00 2024-09-13T18:05:52.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.476+0000 7f32a97fa700 1 -- 172.21.15.40:0/3997947680 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f329c021030 con 0x7f32ac0704a0 2024-09-13T18:05:52.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.476+0000 7f32a97fa700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f328c0667a0 0x7f328c068c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.476+0000 7f32a97fa700 1 -- 172.21.15.40:0/3997947680 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f329c085290 con 0x7f32ac0704a0 2024-09-13T18:05:52.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.476+0000 7f32b0f46700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f328c0667a0 0x7f328c068c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.476+0000 7f32a97fa700 1 -- 172.21.15.40:0/3997947680 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f329400c050 con 0x7f32ac15bc00 2024-09-13T18:05:52.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.476+0000 7f32a97fa700 1 -- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac15bc00 msgr2=0x7f32ac1e2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.476+0000 7f32a97fa700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac15bc00 0x7f32ac1e2950 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f32ac1e34c0 tx=0x7f3294010b20 comp rx=0 tx=0).stop 2024-09-13T18:05:52.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.476+0000 7f32a97fa700 1 -- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac15bc00 msgr2=0x7f32ac1e2950 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:05:52.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.476+0000 7f32a97fa700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac15bc00 0x7f32ac1e2950 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f32ac1e34c0 tx=0x7f3294010b20 comp rx=0 tx=0).stop 2024-09-13T18:05:52.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.476+0000 7f32b0f46700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f328c0667a0 0x7f328c068c60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009610 tx=0x7f32a40097d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32b1747700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac156dc0 0x7f32ac1e2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32b1747700 1 -- 172.21.15.40:0/3997947680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac15a560 con 0x7f32ac156dc0 2024-09-13T18:05:52.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32a3fff700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac156dc0 0x7f32ac1e2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32a3fff700 1 -- 172.21.15.40:0/3997947680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f32ac151d30 con 0x7f32ac156dc0 2024-09-13T18:05:52.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32a3fff700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac156dc0 0x7f32ac1e2e90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f329401a000 tx=0x7f3294004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32a97fa700 1 -- 172.21.15.40:0/3997947680 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329402a620 con 0x7f32ac156dc0 2024-09-13T18:05:52.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32a97fa700 1 -- 172.21.15.40:0/3997947680 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4784 (secure 0 0 0) 0x7f329401e420 con 0x7f32ac156dc0 2024-09-13T18:05:52.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32a97fa700 1 -- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac156dc0 msgr2=0x7f32ac1e2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32a97fa700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac156dc0 0x7f32ac1e2e90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f329401a000 tx=0x7f3294004770 comp rx=0 tx=0).stop 2024-09-13T18:05:52.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32a97fa700 1 -- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac156dc0 msgr2=0x7f32ac1e2e90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:05:52.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32a97fa700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac156dc0 0x7f32ac1e2e90 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f329401a000 tx=0x7f3294004770 comp rx=0 tx=0).stop 2024-09-13T18:05:52.498 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:05:52.498 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:05:52.498 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:05:52.498 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:05:52.498 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 124, 2024-09-13T18:05:52.498 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:05:52.498 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:05:52.498 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:05:52.498 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:05:52.498 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:52.498 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:05:52.499 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:05:52.499 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:05:52.499 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:05:52.499 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:52.499 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:52.499 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:52.499 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:52.499 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:52.499 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:52.499 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:52.499 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:52.499 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:52.499 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:52.499 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:52.499 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:05:52.500 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:05:52.500 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:52.500 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:52.500 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:52.500 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:52.500 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:52.500 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:52.500 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:52.500 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:52.500 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:52.500 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:52.500 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:52.500 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.501 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:05:52.501 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:05:52.501 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:05:52.501 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:05:52.501 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:05:52.501 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:05:52.501 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:05:52.501 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:05:52.501 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:05:52.501 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:05:52.501 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:05:52.501 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:05:52.501 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:05:52.501 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:05:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:05:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:05:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:05:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:52.502 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:05:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:05:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:05:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:05:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:05:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.503 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:05:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:05:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:05:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:05:52.504 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:05:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:05:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:05:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.505 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:05:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:05:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:05:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:05:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.506 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:05:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:05:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:05:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:05:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:52.507 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:05:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:05:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:05:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:05:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.508 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:05:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:05:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:05:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:05:52.509 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.510 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:52.510 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:52.510 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:05:52.510 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.510 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:52.510 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.510 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:05:52.510 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:05:52.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32b1747700 1 -- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f328c0667a0 msgr2=0x7f328c068c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32b1747700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f328c0667a0 0x7f328c068c60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009610 tx=0x7f32a40097d0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32b1747700 1 -- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac0704a0 msgr2=0x7f32ac1e5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32b1747700 1 --2- 172.21.15.40:0/3997947680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32ac0704a0 0x7f32ac1e5b80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f32ac074750 tx=0x7f329c004770 comp rx=0 tx=0).stop 2024-09-13T18:05:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32b0f46700 1 -- 172.21.15.40:0/3997947680 reap_dead start 2024-09-13T18:05:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32b0f46700 1 -- 172.21.15.40:0/3997947680 reap_dead start 2024-09-13T18:05:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32b1747700 1 -- 172.21.15.40:0/3997947680 shutdown_connections 2024-09-13T18:05:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32b1747700 1 -- 172.21.15.40:0/3997947680 >> 172.21.15.40:0/3997947680 conn(0x7f32ac0ccef0 msgr2=0x7f32ac0c96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32b1747700 1 -- 172.21.15.40:0/3997947680 shutdown_connections 2024-09-13T18:05:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.488+0000 7f32b1747700 1 -- 172.21.15.40:0/3997947680 wait complete. 2024-09-13T18:05:52.512 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:05:52.512 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:05:52.513 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:05:52.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.624+0000 7ff29bfff700 1 Processor -- start 2024-09-13T18:05:52.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.624+0000 7ff29bfff700 1 -- start start 2024-09-13T18:05:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.628+0000 7ff29bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c0724d0 0x7ff29c070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.628+0000 7ff29bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff29c0c6190 0x7ff29c070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.628+0000 7ff29bfff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29c15aea0 con 0x7ff29c153150 2024-09-13T18:05:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.628+0000 7ff29bfff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff29c159e30 con 0x7ff29c0c6190 2024-09-13T18:05:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.628+0000 7ff29bfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c15a0d0 con 0x7ff29c0724d0 2024-09-13T18:05:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.628+0000 7ff29b7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff29c0c6190 0x7ff29c070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.628+0000 7ff29b7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff29c0c6190 0x7ff29c070fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47706/0 (socket says 172.21.15.40:47706) 2024-09-13T18:05:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.628+0000 7ff29b7fe700 1 -- 172.21.15.40:0/230001674 learned_addr learned my addr 172.21.15.40:0/230001674 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.628+0000 7ff29a7fc700 1 --2- 172.21.15.40:0/230001674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c0724d0 0x7ff29c070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.628+0000 7ff29b7fe700 1 -- 172.21.15.40:0/230001674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c0724d0 msgr2=0x7ff29c070aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:05:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.628+0000 7ff29b7fe700 1 --2- 172.21.15.40:0/230001674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c0724d0 0x7ff29c070aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.628+0000 7ff29b7fe700 1 -- 172.21.15.40:0/230001674 >> v1:172.21.15.43:6789/0 conn(0x7ff29c153150 legacy=0x7ff29c15a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.628+0000 7ff29b7fe700 1 -- 172.21.15.40:0/230001674 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c0cb2a0 con 0x7ff29c0c6190 2024-09-13T18:05:52.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.628+0000 7ff29a7fc700 1 --2- 172.21.15.40:0/230001674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c0724d0 0x7ff29c070aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:05:52.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.628+0000 7ff29b7fe700 1 --2- 172.21.15.40:0/230001674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0c6190 0x7ff29c070fe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff294009e50 tx=0x7ff29400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327ca2f6df6c377c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.628+0000 7ff299ffb700 1 -- 172.21.15.40:0/230001674 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29400e070 con 0x7ff29c0c6190 2024-09-13T18:05:52.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.628+0000 7ff299ffb700 1 -- 172.21.15.40:0/230001674 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29400b9c0 con 0x7ff29c0c6190 2024-09-13T18:05:52.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.628+0000 7ff299ffb700 1 -- 172.21.15.40:0/230001674 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29400bd70 con 0x7ff29c0c6190 2024-09-13T18:05:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.632+0000 7ff29bfff700 1 -- 172.21.15.40:0/230001674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0c6190 msgr2=0x7ff29c070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.632+0000 7ff29bfff700 1 --2- 172.21.15.40:0/230001674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0c6190 0x7ff29c070fe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff294009e50 tx=0x7ff29400b330 comp rx=0 tx=0).stop 2024-09-13T18:05:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.632+0000 7ff29bfff700 1 -- 172.21.15.40:0/230001674 shutdown_connections 2024-09-13T18:05:52.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.632+0000 7ff29bfff700 1 --2- 172.21.15.40:0/230001674 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0c6190 0x7ff29c070fe0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.632+0000 7ff29bfff700 1 --2- 172.21.15.40:0/230001674 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c0724d0 0x7ff29c070aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.632+0000 7ff29bfff700 1 -- 172.21.15.40:0/230001674 >> 172.21.15.40:0/230001674 conn(0x7ff29c06a390 msgr2=0x7ff29c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:52.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.632+0000 7ff29bfff700 1 -- 172.21.15.40:0/230001674 shutdown_connections 2024-09-13T18:05:52.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.632+0000 7ff29bfff700 1 -- 172.21.15.40:0/230001674 wait complete. 2024-09-13T18:05:52.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.632+0000 7ff29bfff700 1 Processor -- start 2024-09-13T18:05:52.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.632+0000 7ff29bfff700 1 -- start start 2024-09-13T18:05:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.632+0000 7ff29bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0724d0 0x7ff29c14b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.632+0000 7ff29bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c6190 0x7ff29c0c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.632+0000 7ff29bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c153150 0x7ff29c149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.632+0000 7ff29bfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c0c7c90 con 0x7ff29c0724d0 2024-09-13T18:05:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.632+0000 7ff29bfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c1eefd0 con 0x7ff29c0c6190 2024-09-13T18:05:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.632+0000 7ff29bfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c0c6f60 con 0x7ff29c153150 2024-09-13T18:05:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.632+0000 7ff29affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0724d0 0x7ff29c14b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0724d0 0x7ff29c14b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47712/0 (socket says 172.21.15.40:47712) 2024-09-13T18:05:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29affd700 1 -- 172.21.15.40:0/4291559306 learned_addr learned my addr 172.21.15.40:0/4291559306 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29a7fc700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c6190 0x7ff29c0c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29b7fe700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c153150 0x7ff29c149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29a7fc700 1 -- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c6190 msgr2=0x7ff29c0c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:05:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29a7fc700 1 -- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c6190 msgr2=0x7ff29c0c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:05:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29a7fc700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c6190 0x7ff29c0c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:05:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29a7fc700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c6190 0x7ff29c0c8120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:05:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29affd700 1 -- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c153150 msgr2=0x7ff29c149b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29affd700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c153150 0x7ff29c149b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29affd700 1 -- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c6190 msgr2=0x7ff29c0c8120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:05:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29affd700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c6190 0x7ff29c0c8120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29affd700 1 -- 172.21.15.40:0/4291559306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff294009b30 con 0x7ff29c0724d0 2024-09-13T18:05:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29affd700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0724d0 0x7ff29c14b760 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00c500 tx=0x7ff28c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff283fff700 1 -- 172.21.15.40:0/4291559306 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c0052d0 con 0x7ff29c0724d0 2024-09-13T18:05:52.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff283fff700 1 -- 172.21.15.40:0/4291559306 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28c005430 con 0x7ff29c0724d0 2024-09-13T18:05:52.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff283fff700 1 -- 172.21.15.40:0/4291559306 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c0055a0 con 0x7ff29c0724d0 2024-09-13T18:05:52.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29bfff700 1 -- 172.21.15.40:0/4291559306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff29c14e300 con 0x7ff29c0724d0 2024-09-13T18:05:52.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29bfff700 1 -- 172.21.15.40:0/4291559306 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff29c1ef250 con 0x7ff29c0724d0 2024-09-13T18:05:52.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29bfff700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c1590e0 0x7ff29c0c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29bfff700 1 -- 172.21.15.40:0/4291559306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c067600 con 0x7ff29c1590e0 2024-09-13T18:05:52.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29a7fc700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c1590e0 0x7ff29c0c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff283fff700 1 -- 172.21.15.40:0/4291559306 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff28c0038e0 con 0x7ff29c0724d0 2024-09-13T18:05:52.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29a7fc700 1 -- 172.21.15.40:0/4291559306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff29c0665e0 con 0x7ff29c1590e0 2024-09-13T18:05:52.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.636+0000 7ff29a7fc700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c1590e0 0x7ff29c0c7560 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff29c14b210 tx=0x7ff290009820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.640+0000 7ff283fff700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff288066730 0x7ff288068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.640+0000 7ff29b7fe700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff288066730 0x7ff288068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.640+0000 7ff283fff700 1 -- 172.21.15.40:0/4291559306 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff28c083840 con 0x7ff29c0724d0 2024-09-13T18:05:52.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.640+0000 7ff283fff700 1 -- 172.21.15.40:0/4291559306 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29000c070 con 0x7ff29c1590e0 2024-09-13T18:05:52.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.640+0000 7ff29b7fe700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff288066730 0x7ff288068bf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff29400b370 tx=0x7ff294009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.640+0000 7ff283fff700 1 -- 172.21.15.40:0/4291559306 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff290009e40 con 0x7ff29c1590e0 2024-09-13T18:05:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.640+0000 7ff283fff700 1 -- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c1590e0 msgr2=0x7ff29c0c7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.640+0000 7ff283fff700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c1590e0 0x7ff29c0c7560 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff29c14b210 tx=0x7ff290009820 comp rx=0 tx=0).stop 2024-09-13T18:05:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.640+0000 7ff283fff700 1 -- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c1590e0 msgr2=0x7ff29c0c7560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:05:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.640+0000 7ff283fff700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c1590e0 0x7ff29c0c7560 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.648+0000 7ff29bfff700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c8660 0x7ff29c14a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.648+0000 7ff29bfff700 1 -- 172.21.15.40:0/4291559306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c0c7aa0 con 0x7ff29c0c8660 2024-09-13T18:05:52.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.648+0000 7ff29a7fc700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c8660 0x7ff29c14a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.652+0000 7ff29a7fc700 1 -- 172.21.15.40:0/4291559306 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff29c070320 con 0x7ff29c0c8660 2024-09-13T18:05:52.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.652+0000 7ff29a7fc700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c8660 0x7ff29c14a060 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff290004700 tx=0x7ff290004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.652+0000 7ff283fff700 1 -- 172.21.15.40:0/4291559306 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff290023070 con 0x7ff29c0c8660 2024-09-13T18:05:52.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.652+0000 7ff283fff700 1 -- 172.21.15.40:0/4291559306 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4450 (secure 0 0 0) 0x7ff290013e30 con 0x7ff29c0c8660 2024-09-13T18:05:52.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.652+0000 7ff283fff700 1 -- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c8660 msgr2=0x7ff29c14a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.652+0000 7ff283fff700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c8660 0x7ff29c14a060 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff290004700 tx=0x7ff290004770 comp rx=0 tx=0).stop 2024-09-13T18:05:52.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.652+0000 7ff283fff700 1 -- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c8660 msgr2=0x7ff29c14a060 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:05:52.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.652+0000 7ff283fff700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c8660 0x7ff29c14a060 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.661 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:05:52.661 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:05:52.661 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:05:52.661 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:05:52.661 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 124, 2024-09-13T18:05:52.661 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:05:52.662 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:05:52.662 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:05:52.662 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:05:52.662 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:52.662 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:05:52.662 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:05:52.662 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:05:52.662 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:05:52.662 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:52.662 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:52.662 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:52.662 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:52.662 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:52.663 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:52.663 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:52.663 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:52.663 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:52.663 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:52.663 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:52.663 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:05:52.663 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:05:52.663 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:52.663 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:52.664 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:52.664 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:52.664 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:52.664 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:52.664 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:52.664 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:52.664 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:52.664 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:52.664 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:52.664 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:05:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:05:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:05:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:05:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:05:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:05:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:05:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:05:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:05:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:05:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:05:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:05:52.665 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:05:52.666 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:05:52.666 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:05:52.666 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:05:52.666 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:05:52.666 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:52.666 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:52.666 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:52.666 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:52.666 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:52.666 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:52.666 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:52.666 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:52.667 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:52.667 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:52.667 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:52.667 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:05:52.667 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.668 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:05:52.668 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.668 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:05:52.668 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:05:52.668 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:52.668 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:52.669 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.669 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:52.672 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:05:52.672 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:52.673 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.673 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.673 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:52.673 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:05:52.673 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:52.673 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.673 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:52.673 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.673 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:05:52.673 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:05:52.674 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:52.674 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:52.674 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:52.674 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.675 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.675 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:05:52.675 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:05:52.675 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:52.675 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:52.676 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.676 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:52.676 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:05:52.676 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:52.676 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.676 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.676 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:52.676 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:05:52.676 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:52.676 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.676 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:52.676 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.677 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:05:52.677 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:05:52.677 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:52.677 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:52.677 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:52.677 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.677 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.677 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:05:52.677 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:05:52.677 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:52.677 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:52.677 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.678 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:52.678 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:05:52.678 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:52.678 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.678 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.678 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:52.678 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:05:52.678 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:52.678 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.678 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:52.678 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.678 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:05:52.679 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:05:52.679 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:52.679 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:52.679 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:52.679 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.679 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:52.680 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.680 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:05:52.680 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:05:52.680 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.680 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:52.680 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:52.680 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:05:52.680 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.681 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:52.681 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:05:52.681 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.681 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:52.681 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:52.681 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:05:52.681 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.681 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:52.681 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.681 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:05:52.682 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:05:52.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.652+0000 7ff29bfff700 1 -- 172.21.15.40:0/4291559306 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff288066730 msgr2=0x7ff288068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.652+0000 7ff29bfff700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff288066730 0x7ff288068bf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff29400b370 tx=0x7ff294009000 comp rx=0 tx=0).stop 2024-09-13T18:05:52.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.652+0000 7ff29bfff700 1 -- 172.21.15.40:0/4291559306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0724d0 msgr2=0x7ff29c14b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.652+0000 7ff29b7fe700 1 -- 172.21.15.40:0/4291559306 reap_dead start 2024-09-13T18:05:52.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.652+0000 7ff29bfff700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0724d0 0x7ff29c14b760 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00c500 tx=0x7ff28c00c810 comp rx=0 tx=0).stop 2024-09-13T18:05:52.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.652+0000 7ff29bfff700 1 -- 172.21.15.40:0/4291559306 shutdown_connections 2024-09-13T18:05:52.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.652+0000 7ff29bfff700 1 --2- 172.21.15.40:0/4291559306 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0724d0 0x7ff29c14b760 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.652+0000 7ff29bfff700 1 -- 172.21.15.40:0/4291559306 >> 172.21.15.40:0/4291559306 conn(0x7ff29c06a390 msgr2=0x7ff29c0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:52.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.652+0000 7ff29bfff700 1 -- 172.21.15.40:0/4291559306 shutdown_connections 2024-09-13T18:05:52.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.652+0000 7ff29bfff700 1 -- 172.21.15.40:0/4291559306 wait complete. 2024-09-13T18:05:52.684 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:05:52.684 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:05:52.685 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:05:52.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.796+0000 7f1e2487a700 1 Processor -- start 2024-09-13T18:05:52.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.796+0000 7f1e2487a700 1 -- start start 2024-09-13T18:05:52.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.796+0000 7f1e2487a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1e200ca9d0 0x7f1e20157800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.796+0000 7f1e2487a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1e20157d40 0x7f1e200c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.796+0000 7f1e2487a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e20157370 con 0x7f1e20151440 2024-09-13T18:05:52.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.796+0000 7f1e2487a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e200cb1f0 con 0x7f1e200ca9d0 2024-09-13T18:05:52.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.796+0000 7f1e2487a700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e200caf10 con 0x7f1e20157d40 2024-09-13T18:05:52.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.796+0000 7f1e17fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1e200ca9d0 0x7f1e20157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.796+0000 7f1e1f7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1e20151440 0x7f1e200661b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47982/0 (socket says 172.21.15.40:47982) 2024-09-13T18:05:52.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.796+0000 7f1e1f7fe700 1 -- 172.21.15.40:0/1290010492 learned_addr learned my addr 172.21.15.40:0/1290010492 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:05:52.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.796+0000 7f1e1ffff700 1 --2- 172.21.15.40:0/1290010492 >> v2:172.21.15.43:3300/0 conn(0x7f1e20157d40 0x7f1e200c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.796+0000 7f1e17fff700 1 -- 172.21.15.40:0/1290010492 >> v2:172.21.15.43:3300/0 conn(0x7f1e20157d40 msgr2=0x7f1e200c4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:05:52.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.796+0000 7f1e17fff700 1 --2- 172.21.15.40:0/1290010492 >> v2:172.21.15.43:3300/0 conn(0x7f1e20157d40 0x7f1e200c4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.796+0000 7f1e17fff700 1 -- 172.21.15.40:0/1290010492 >> v1:172.21.15.40:6789/0 conn(0x7f1e20151440 legacy=0x7f1e200661b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.796+0000 7f1e17fff700 1 -- 172.21.15.40:0/1290010492 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e200c9710 con 0x7f1e200ca9d0 2024-09-13T18:05:52.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.796+0000 7f1e1effd700 1 -- 172.21.15.40:0/1290010492 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234392715 0 0) 0x7f1e20157370 con 0x7f1e20151440 2024-09-13T18:05:52.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.796+0000 7f1e17fff700 1 --2- 172.21.15.40:0/1290010492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e200ca9d0 0x7f1e20157800 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1e200c5c70 tx=0x7f1e08009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1979d426b87aac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.796+0000 7f1e1effd700 1 -- 172.21.15.40:0/1290010492 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08011830 con 0x7f1e200ca9d0 2024-09-13T18:05:52.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.796+0000 7f1e1effd700 1 -- 172.21.15.40:0/1290010492 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e08011990 con 0x7f1e200ca9d0 2024-09-13T18:05:52.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.796+0000 7f1e1effd700 1 -- 172.21.15.40:0/1290010492 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08011d40 con 0x7f1e200ca9d0 2024-09-13T18:05:52.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e2487a700 1 -- 172.21.15.40:0/1290010492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e200ca9d0 msgr2=0x7f1e20157800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e2487a700 1 --2- 172.21.15.40:0/1290010492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e200ca9d0 0x7f1e20157800 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1e200c5c70 tx=0x7f1e08009b30 comp rx=0 tx=0).stop 2024-09-13T18:05:52.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e2487a700 1 -- 172.21.15.40:0/1290010492 shutdown_connections 2024-09-13T18:05:52.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e2487a700 1 --2- 172.21.15.40:0/1290010492 >> v2:172.21.15.43:3300/0 conn(0x7f1e20157d40 0x7f1e200c4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e2487a700 1 --2- 172.21.15.40:0/1290010492 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e200ca9d0 0x7f1e20157800 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e2487a700 1 -- 172.21.15.40:0/1290010492 >> 172.21.15.40:0/1290010492 conn(0x7f1e2006a390 msgr2=0x7f1e2006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:52.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e2487a700 1 -- 172.21.15.40:0/1290010492 shutdown_connections 2024-09-13T18:05:52.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e2487a700 1 -- 172.21.15.40:0/1290010492 wait complete. 2024-09-13T18:05:52.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e2487a700 1 Processor -- start 2024-09-13T18:05:52.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e2487a700 1 -- start start 2024-09-13T18:05:52.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e2487a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1e200ca9d0 0x7f1e201ee450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e2487a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e20151440 0x7f1e201edaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e2487a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e20157d40 0x7f1e201f0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e2487a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e201ed610 con 0x7f1e20151440 2024-09-13T18:05:52.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e2487a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e201f0a90 con 0x7f1e20157d40 2024-09-13T18:05:52.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e2487a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e201ecfe0 con 0x7f1e200ca9d0 2024-09-13T18:05:52.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e17fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e20151440 0x7f1e201edaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e17fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e20151440 0x7f1e201edaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47724/0 (socket says 172.21.15.40:47724) 2024-09-13T18:05:52.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e17fff700 1 -- 172.21.15.40:0/2428749867 learned_addr learned my addr 172.21.15.40:0/2428749867 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:52.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e1f7fe700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1e200ca9d0 0x7f1e201ee450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e1ffff700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e20157d40 0x7f1e201f0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e17fff700 1 -- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1e200ca9d0 msgr2=0x7f1e201ee450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e17fff700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1e200ca9d0 0x7f1e201ee450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e17fff700 1 -- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e20157d40 msgr2=0x7f1e201f0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e17fff700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e20157d40 0x7f1e201f0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e17fff700 1 -- 172.21.15.40:0/2428749867 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e080094b0 con 0x7f1e20151440 2024-09-13T18:05:52.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e1ffff700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e20157d40 0x7f1e201f0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:05:52.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e17fff700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e20151440 0x7f1e201edaa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1e08009b70 tx=0x7f1e08004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e1cff9700 1 -- 172.21.15.40:0/2428749867 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e080036c0 con 0x7f1e20151440 2024-09-13T18:05:52.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e1cff9700 1 -- 172.21.15.40:0/2428749867 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0801e070 con 0x7f1e20151440 2024-09-13T18:05:52.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e2487a700 1 -- 172.21.15.40:0/2428749867 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e201f0d20 con 0x7f1e20151440 2024-09-13T18:05:52.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e1cff9700 1 -- 172.21.15.40:0/2428749867 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08003a80 con 0x7f1e20151440 2024-09-13T18:05:52.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.800+0000 7f1e2487a700 1 -- 172.21.15.40:0/2428749867 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e201f12e0 con 0x7f1e20151440 2024-09-13T18:05:52.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.804+0000 7f1e2487a700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1e20154920 0x7f1e201e3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.804+0000 7f1e2487a700 1 -- 172.21.15.40:0/2428749867 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e20067600 con 0x7f1e20154920 2024-09-13T18:05:52.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.804+0000 7f1e1f7fe700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1e20154920 0x7f1e201e3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.804+0000 7f1e1f7fe700 1 -- 172.21.15.40:0/2428749867 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e201f14d0 con 0x7f1e20154920 2024-09-13T18:05:52.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.804+0000 7f1e1cff9700 1 -- 172.21.15.40:0/2428749867 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1e08003be0 con 0x7f1e20151440 2024-09-13T18:05:52.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.804+0000 7f1e1f7fe700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1e20154920 0x7f1e201e3ea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e10000f60 tx=0x7f1e10001710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.804+0000 7f1e1cff9700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1e00066730 0x7f1e00068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.804+0000 7f1e1cff9700 1 -- 172.21.15.40:0/2428749867 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1e080855c0 con 0x7f1e20151440 2024-09-13T18:05:52.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.804+0000 7f1e1ffff700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1e00066730 0x7f1e00068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.804+0000 7f1e1cff9700 1 -- 172.21.15.40:0/2428749867 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e100114f0 con 0x7f1e20154920 2024-09-13T18:05:52.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.804+0000 7f1e1cff9700 1 -- 172.21.15.40:0/2428749867 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f1e10011650 con 0x7f1e20154920 2024-09-13T18:05:52.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.804+0000 7f1e1cff9700 1 -- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1e20154920 msgr2=0x7f1e201e3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.804+0000 7f1e1cff9700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1e20154920 0x7f1e201e3ea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e10000f60 tx=0x7f1e10001710 comp rx=0 tx=0).stop 2024-09-13T18:05:52.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.804+0000 7f1e1cff9700 1 -- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1e20154920 msgr2=0x7f1e201e3ea0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:05:52.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.804+0000 7f1e1cff9700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1e20154920 0x7f1e201e3ea0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.808+0000 7f1e1ffff700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1e00066730 0x7f1e00068bf0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009f70 tx=0x7f1e1800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.816+0000 7f1e2487a700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1e201ed120 0x7f1e20146c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.816+0000 7f1e2487a700 1 -- 172.21.15.40:0/2428749867 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e200c9cd0 con 0x7f1e201ed120 2024-09-13T18:05:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.820+0000 7f1e1f7fe700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1e201ed120 0x7f1e20146c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.820+0000 7f1e1f7fe700 1 -- 172.21.15.40:0/2428749867 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1e200c9140 con 0x7f1e201ed120 2024-09-13T18:05:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.820+0000 7f1e1f7fe700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1e201ed120 0x7f1e20146c00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1e10016000 tx=0x7f1e10001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.820+0000 7f1e1cff9700 1 -- 172.21.15.40:0/2428749867 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10021070 con 0x7f1e201ed120 2024-09-13T18:05:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.820+0000 7f1e1cff9700 1 -- 172.21.15.40:0/2428749867 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7f1e1001c8e0 con 0x7f1e201ed120 2024-09-13T18:05:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.820+0000 7f1e1cff9700 1 -- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1e201ed120 msgr2=0x7f1e20146c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.820+0000 7f1e1cff9700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1e201ed120 0x7f1e20146c00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1e10016000 tx=0x7f1e10001080 comp rx=0 tx=0).stop 2024-09-13T18:05:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.820+0000 7f1e1cff9700 1 -- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1e201ed120 msgr2=0x7f1e20146c00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:05:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.820+0000 7f1e1cff9700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1e201ed120 0x7f1e20146c00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.829 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:05:52.829 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:05:52.830 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:05:52.830 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:05:52.830 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 124, 2024-09-13T18:05:52.830 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:05:52.830 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:05:52.830 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:05:52.830 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:05:52.830 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:52.830 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:05:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:05:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:05:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:05:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:52.831 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:05:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:05:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:52.832 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:05:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:05:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:05:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:05:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:05:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:05:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:05:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:05:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:05:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:05:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:05:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:05:52.833 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:05:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:05:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:05:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:05:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:05:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:05:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:05:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:05:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:05:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:05:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:05:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:05:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:05:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:05:52.834 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:05:52.835 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:52.835 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:05:52.835 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.835 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:05:52.835 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.835 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:05:52.835 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:05:52.835 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:52.835 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:52.835 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.835 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:52.836 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:05:52.836 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:52.836 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.836 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.836 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:52.836 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:05:52.836 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:52.836 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.836 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:52.836 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:05:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:05:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:05:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:05:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.837 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:05:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:05:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:05:52.838 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:05:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:05:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:05:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:05:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:05:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:05:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:05:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:52.839 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:05:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:05:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:05:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:05:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:05:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:05:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:05:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:05:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.840 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:05:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:05:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:05:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:05:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:52.841 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:52.842 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:05:52.842 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.842 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:52.842 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:05:52.842 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.842 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:52.842 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:05:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:05:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:05:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:05:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:05:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:05:52.843 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:05:52.844 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:05:52.844 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:05:52.844 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:05:52.844 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:05:52.844 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:05:52.844 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:05:52.844 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:05:52.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.820+0000 7f1e2487a700 1 -- 172.21.15.40:0/2428749867 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1e00066730 msgr2=0x7f1e00068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.820+0000 7f1e2487a700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1e00066730 0x7f1e00068bf0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009f70 tx=0x7f1e1800b040 comp rx=0 tx=0).stop 2024-09-13T18:05:52.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.820+0000 7f1e2487a700 1 -- 172.21.15.40:0/2428749867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e20151440 msgr2=0x7f1e201edaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.820+0000 7f1e2487a700 1 --2- 172.21.15.40:0/2428749867 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1e20151440 0x7f1e201edaa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1e08009b70 tx=0x7f1e08004770 comp rx=0 tx=0).stop 2024-09-13T18:05:52.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.820+0000 7f1e1ffff700 1 -- 172.21.15.40:0/2428749867 reap_dead start 2024-09-13T18:05:52.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.820+0000 7f1e1ffff700 1 -- 172.21.15.40:0/2428749867 reap_dead start 2024-09-13T18:05:52.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.820+0000 7f1e2487a700 1 -- 172.21.15.40:0/2428749867 shutdown_connections 2024-09-13T18:05:52.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.820+0000 7f1e2487a700 1 -- 172.21.15.40:0/2428749867 >> 172.21.15.40:0/2428749867 conn(0x7f1e2006a390 msgr2=0x7f1e200c7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:52.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.820+0000 7f1e2487a700 1 -- 172.21.15.40:0/2428749867 shutdown_connections 2024-09-13T18:05:52.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.820+0000 7f1e2487a700 1 -- 172.21.15.40:0/2428749867 wait complete. 2024-09-13T18:05:52.848 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:05:52.848 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:05:52.848 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:05:52.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.948+0000 7f477ca45700 1 Processor -- start 2024-09-13T18:05:52.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.948+0000 7f477ca45700 1 -- start start 2024-09-13T18:05:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.948+0000 7f477ca45700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4770045d00 0x7f47700dbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.948+0000 7f477ca45700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4770007830 con 0x7f4770051a10 2024-09-13T18:05:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.948+0000 7f477ca45700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4770007a80 con 0x7f477004cc20 2024-09-13T18:05:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.948+0000 7f477ca45700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4770007d60 con 0x7f4770045d00 2024-09-13T18:05:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.948+0000 7f477659c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4770051a10 0x7f47700e3bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47994/0 (socket says 172.21.15.40:47994) 2024-09-13T18:05:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.948+0000 7f477659c700 1 -- 172.21.15.40:0/2092152430 learned_addr learned my addr 172.21.15.40:0/2092152430 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:05:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.948+0000 7f477759e700 1 --2- 172.21.15.40:0/2092152430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4770045d00 0x7f47700dbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.948+0000 7f4775d9b700 1 -- 172.21.15.40:0/2092152430 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 115294670 0 0) 0x7f4770007830 con 0x7f4770051a10 2024-09-13T18:05:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.948+0000 7f4775d9b700 1 -- 172.21.15.40:0/2092152430 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47600035d0 con 0x7f4770051a10 2024-09-13T18:05:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.948+0000 7f4775d9b700 1 -- 172.21.15.40:0/2092152430 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155400423 0 0) 0x7f4770007a80 con 0x7f477004cc20 2024-09-13T18:05:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.948+0000 7f4775d9b700 1 -- 172.21.15.40:0/2092152430 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4770007830 con 0x7f477004cc20 2024-09-13T18:05:52.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.948+0000 7f477759e700 1 -- 172.21.15.40:0/2092152430 >> v1:172.21.15.43:6789/0 conn(0x7f477004cc20 legacy=0x7f4770008b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.948+0000 7f477759e700 1 -- 172.21.15.40:0/2092152430 >> v1:172.21.15.40:6789/0 conn(0x7f4770051a10 legacy=0x7f47700e3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.948+0000 7f477759e700 1 -- 172.21.15.40:0/2092152430 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4770008040 con 0x7f4770045d00 2024-09-13T18:05:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.948+0000 7f477759e700 1 --2- 172.21.15.40:0/2092152430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4770045d00 0x7f47700dbcb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4778069620 tx=0x7f4778069e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71f866db0e72d1c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.948+0000 7f4775d9b700 1 -- 172.21.15.40:0/2092152430 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778067dd0 con 0x7f4770045d00 2024-09-13T18:05:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.952+0000 7f4775d9b700 1 -- 172.21.15.40:0/2092152430 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4778067f30 con 0x7f4770045d00 2024-09-13T18:05:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.952+0000 7f4775d9b700 1 -- 172.21.15.40:0/2092152430 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47780746e0 con 0x7f4770045d00 2024-09-13T18:05:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.952+0000 7f477ca45700 1 -- 172.21.15.40:0/2092152430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4770045d00 msgr2=0x7f47700dbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.952+0000 7f477ca45700 1 --2- 172.21.15.40:0/2092152430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4770045d00 0x7f47700dbcb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4778069620 tx=0x7f4778069e70 comp rx=0 tx=0).stop 2024-09-13T18:05:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.952+0000 7f477ca45700 1 -- 172.21.15.40:0/2092152430 shutdown_connections 2024-09-13T18:05:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.952+0000 7f477ca45700 1 --2- 172.21.15.40:0/2092152430 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4770045d00 0x7f47700dbcb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.952+0000 7f477ca45700 1 -- 172.21.15.40:0/2092152430 >> 172.21.15.40:0/2092152430 conn(0x7f4770050e60 msgr2=0x7f4770051270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:52.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.952+0000 7f477ca45700 1 -- 172.21.15.40:0/2092152430 shutdown_connections 2024-09-13T18:05:52.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.952+0000 7f477ca45700 1 -- 172.21.15.40:0/2092152430 wait complete. 2024-09-13T18:05:52.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.952+0000 7f477ca45700 1 Processor -- start 2024-09-13T18:05:52.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.952+0000 7f477ca45700 1 -- start start 2024-09-13T18:05:52.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f477ca45700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4770045d00 0x7f4770173100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f477ca45700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477004cc20 0x7f47701665b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f477ca45700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4770051a10 0x7f47701674d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f477ca45700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4770166190 con 0x7f4770045d00 2024-09-13T18:05:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f477ca45700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47700d62a0 con 0x7f477004cc20 2024-09-13T18:05:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f477ca45700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4770173640 con 0x7f4770051a10 2024-09-13T18:05:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f4776d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4770045d00 0x7f4770173100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f4776d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4770045d00 0x7f4770173100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47734/0 (socket says 172.21.15.40:47734) 2024-09-13T18:05:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f4776d9d700 1 -- 172.21.15.40:0/3092507579 learned_addr learned my addr 172.21.15.40:0/3092507579 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:05:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f477659c700 1 --2- 172.21.15.40:0/3092507579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477004cc20 0x7f47701665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f477759e700 1 --2- 172.21.15.40:0/3092507579 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4770051a10 0x7f47701674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f4776d9d700 1 -- 172.21.15.40:0/3092507579 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4770051a10 msgr2=0x7f47701674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f4776d9d700 1 --2- 172.21.15.40:0/3092507579 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4770051a10 0x7f47701674d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f4776d9d700 1 -- 172.21.15.40:0/3092507579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477004cc20 msgr2=0x7f47701665b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:52.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f4776d9d700 1 --2- 172.21.15.40:0/3092507579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477004cc20 0x7f47701665b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f4776d9d700 1 -- 172.21.15.40:0/3092507579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477804f5c0 con 0x7f4770045d00 2024-09-13T18:05:52.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f4776d9d700 1 --2- 172.21.15.40:0/3092507579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4770045d00 0x7f4770173100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f476800c790 tx=0x7f476800caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f475f7fe700 1 -- 172.21.15.40:0/3092507579 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47680052d0 con 0x7f4770045d00 2024-09-13T18:05:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f475f7fe700 1 -- 172.21.15.40:0/3092507579 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4768005450 con 0x7f4770045d00 2024-09-13T18:05:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f477ca45700 1 -- 172.21.15.40:0/3092507579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4770167ff0 con 0x7f4770045d00 2024-09-13T18:05:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f475f7fe700 1 -- 172.21.15.40:0/3092507579 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47680038e0 con 0x7f4770045d00 2024-09-13T18:05:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f477ca45700 1 -- 172.21.15.40:0/3092507579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47701657f0 con 0x7f4770045d00 2024-09-13T18:05:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.956+0000 7f477ca45700 1 -- 172.21.15.40:0/3092507579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4770037080 con 0x7f4770045d00 2024-09-13T18:05:52.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.960+0000 7f475f7fe700 1 -- 172.21.15.40:0/3092507579 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4768003a40 con 0x7f4770045d00 2024-09-13T18:05:52.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.960+0000 7f475f7fe700 1 --2- 172.21.15.40:0/3092507579 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4764066730 0x7f4764068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:05:52.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.960+0000 7f475f7fe700 1 -- 172.21.15.40:0/3092507579 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4768083940 con 0x7f4770045d00 2024-09-13T18:05:52.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.960+0000 7f477659c700 1 --2- 172.21.15.40:0/3092507579 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4764066730 0x7f4764068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:05:52.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.960+0000 7f477659c700 1 --2- 172.21.15.40:0/3092507579 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4764066730 0x7f4764068bf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4770046ba0 tx=0x7f476c007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:05:52.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:52.964+0000 7f475f7fe700 1 -- 172.21.15.40:0/3092507579 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f47680508f0 con 0x7f4770045d00 2024-09-13T18:05:53.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:53.240+0000 7f477ca45700 1 -- 172.21.15.40:0/3092507579 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f4770007a80 con 0x7f4770045d00 2024-09-13T18:05:53.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:53.240+0000 7f475f7fe700 1 -- 172.21.15.40:0/3092507579 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f4768053f10 con 0x7f4770045d00 2024-09-13T18:05:53.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:53.244+0000 7f477ca45700 1 -- 172.21.15.40:0/3092507579 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4764066730 msgr2=0x7f4764068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:53.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:53.244+0000 7f477ca45700 1 --2- 172.21.15.40:0/3092507579 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4764066730 0x7f4764068bf0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4770046ba0 tx=0x7f476c007de0 comp rx=0 tx=0).stop 2024-09-13T18:05:53.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:53.244+0000 7f477ca45700 1 -- 172.21.15.40:0/3092507579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4770045d00 msgr2=0x7f4770173100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:05:53.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:53.244+0000 7f477ca45700 1 --2- 172.21.15.40:0/3092507579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4770045d00 0x7f4770173100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f476800c790 tx=0x7f476800caa0 comp rx=0 tx=0).stop 2024-09-13T18:05:53.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:53.244+0000 7f477ca45700 1 -- 172.21.15.40:0/3092507579 shutdown_connections 2024-09-13T18:05:53.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:53.244+0000 7f477ca45700 1 --2- 172.21.15.40:0/3092507579 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4764066730 0x7f4764068bf0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:53.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:53.244+0000 7f477ca45700 1 --2- 172.21.15.40:0/3092507579 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4770051a10 0x7f47701674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:53.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:53.244+0000 7f477ca45700 1 --2- 172.21.15.40:0/3092507579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477004cc20 0x7f47701665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:53.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:53.244+0000 7f477ca45700 1 --2- 172.21.15.40:0/3092507579 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4770045d00 0x7f4770173100 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:05:53.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:53.244+0000 7f477ca45700 1 -- 172.21.15.40:0/3092507579 >> 172.21.15.40:0/3092507579 conn(0x7f4770050e60 msgr2=0x7f4770046bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:05:53.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:53.244+0000 7f477ca45700 1 -- 172.21.15.40:0/3092507579 shutdown_connections 2024-09-13T18:05:53.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:05:53.244+0000 7f477ca45700 1 -- 172.21.15.40:0/3092507579 wait complete. 2024-09-13T18:05:53.268 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:06:03.269 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:06:03.269 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:06:03.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.392+0000 7f4f1759e700 1 Processor -- start 2024-09-13T18:06:03.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.392+0000 7f4f1759e700 1 -- start start 2024-09-13T18:06:03.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.392+0000 7f4f1759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f181512c0 0x7f4f18157b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.392+0000 7f4f1759e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4f180b2b70 0x7f4f1806f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.392+0000 7f4f1759e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f18070390 con 0x7f4f1806fee0 2024-09-13T18:06:03.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.392+0000 7f4f1759e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f1806e780 con 0x7f4f180b2b70 2024-09-13T18:06:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.392+0000 7f4f1759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f180c3a80 con 0x7f4f181512c0 2024-09-13T18:06:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f15d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4f180b2b70 0x7f4f1806f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f16d9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4f1806fee0 0x7f4f180c1f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36744/0 (socket says 172.21.15.40:36744) 2024-09-13T18:06:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f15d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4f180b2b70 0x7f4f1806f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53498/0 (socket says 172.21.15.40:53498) 2024-09-13T18:06:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f16d9d700 1 -- 172.21.15.40:0/1629563312 learned_addr learned my addr 172.21.15.40:0/1629563312 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:06:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f1659c700 1 --2- 172.21.15.40:0/1629563312 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f181512c0 0x7f4f18157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f15d9b700 1 -- 172.21.15.40:0/1629563312 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f181512c0 msgr2=0x7f4f18157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:06:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f15d9b700 1 --2- 172.21.15.40:0/1629563312 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f181512c0 0x7f4f18157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f15d9b700 1 -- 172.21.15.40:0/1629563312 >> v1:172.21.15.40:6789/0 conn(0x7f4f1806fee0 legacy=0x7f4f180c1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f1559a700 1 -- 172.21.15.40:0/1629563312 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3702477298 0 0) 0x7f4f18070390 con 0x7f4f1806fee0 2024-09-13T18:06:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f15d9b700 1 -- 172.21.15.40:0/1629563312 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f18158070 con 0x7f4f180b2b70 2024-09-13T18:06:03.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f15d9b700 1 --2- 172.21.15.40:0/1629563312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f180b2b70 0x7f4f1806f9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009840 tx=0x7f4f0c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574c9ab7ef524155 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:03.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f1559a700 1 -- 172.21.15.40:0/1629563312 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c005e50 con 0x7f4f180b2b70 2024-09-13T18:06:03.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f1559a700 1 -- 172.21.15.40:0/1629563312 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0c006030 con 0x7f4f180b2b70 2024-09-13T18:06:03.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f1559a700 1 -- 172.21.15.40:0/1629563312 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c0063c0 con 0x7f4f180b2b70 2024-09-13T18:06:03.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f1759e700 1 -- 172.21.15.40:0/1629563312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f180b2b70 msgr2=0x7f4f1806f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:03.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f1759e700 1 --2- 172.21.15.40:0/1629563312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f180b2b70 0x7f4f1806f9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009840 tx=0x7f4f0c009b50 comp rx=0 tx=0).stop 2024-09-13T18:06:03.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f1759e700 1 -- 172.21.15.40:0/1629563312 shutdown_connections 2024-09-13T18:06:03.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f1759e700 1 --2- 172.21.15.40:0/1629563312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f180b2b70 0x7f4f1806f9a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f1759e700 1 --2- 172.21.15.40:0/1629563312 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f181512c0 0x7f4f18157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f1759e700 1 -- 172.21.15.40:0/1629563312 >> 172.21.15.40:0/1629563312 conn(0x7f4f18066090 msgr2=0x7f4f18066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:03.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f1759e700 1 -- 172.21.15.40:0/1629563312 shutdown_connections 2024-09-13T18:06:03.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f1759e700 1 -- 172.21.15.40:0/1629563312 wait complete. 2024-09-13T18:06:03.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f1759e700 1 Processor -- start 2024-09-13T18:06:03.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.396+0000 7f4f1759e700 1 -- start start 2024-09-13T18:06:03.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f1759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1806fee0 0x7f4f181496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f1759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f181512c0 0x7f4f18149c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f1759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f18146a00 0x7f4f18144810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f1759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f18156e30 con 0x7f4f181512c0 2024-09-13T18:06:03.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f1759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f180c4c50 con 0x7f4f1806fee0 2024-09-13T18:06:03.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f1759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f18148ca0 con 0x7f4f18146a00 2024-09-13T18:06:03.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f15d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f181512c0 0x7f4f18149c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f15d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f181512c0 0x7f4f18149c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53510/0 (socket says 172.21.15.40:53510) 2024-09-13T18:06:03.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f15d9b700 1 -- 172.21.15.40:0/191785146 learned_addr learned my addr 172.21.15.40:0/191785146 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:03.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f1659c700 1 --2- 172.21.15.40:0/191785146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1806fee0 0x7f4f181496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f16d9d700 1 --2- 172.21.15.40:0/191785146 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f18146a00 0x7f4f18144810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f15d9b700 1 -- 172.21.15.40:0/191785146 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f18146a00 msgr2=0x7f4f18144810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:03.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f15d9b700 1 --2- 172.21.15.40:0/191785146 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f18146a00 0x7f4f18144810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f15d9b700 1 -- 172.21.15.40:0/191785146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1806fee0 msgr2=0x7f4f181496f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:03.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f15d9b700 1 --2- 172.21.15.40:0/191785146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1806fee0 0x7f4f181496f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f15d9b700 1 -- 172.21.15.40:0/191785146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0c009450 con 0x7f4f181512c0 2024-09-13T18:06:03.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f15d9b700 1 --2- 172.21.15.40:0/191785146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f181512c0 0x7f4f18149c30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009b90 tx=0x7f4f0c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:03.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f06ffd700 1 -- 172.21.15.40:0/191785146 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c003ad0 con 0x7f4f181512c0 2024-09-13T18:06:03.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f06ffd700 1 -- 172.21.15.40:0/191785146 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0c003c30 con 0x7f4f181512c0 2024-09-13T18:06:03.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f06ffd700 1 -- 172.21.15.40:0/191785146 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c004580 con 0x7f4f181512c0 2024-09-13T18:06:03.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f1759e700 1 -- 172.21.15.40:0/191785146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f18155d00 con 0x7f4f181512c0 2024-09-13T18:06:03.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.400+0000 7f4f1759e700 1 -- 172.21.15.40:0/191785146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f181562a0 con 0x7f4f181512c0 2024-09-13T18:06:03.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.404+0000 7f4f1759e700 1 -- 172.21.15.40:0/191785146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f180c7de0 con 0x7f4f181512c0 2024-09-13T18:06:03.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.404+0000 7f4f06ffd700 1 -- 172.21.15.40:0/191785146 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4f0c022070 con 0x7f4f181512c0 2024-09-13T18:06:03.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.404+0000 7f4f06ffd700 1 --2- 172.21.15.40:0/191785146 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4ef8066760 0x7f4ef8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.404+0000 7f4f06ffd700 1 -- 172.21.15.40:0/191785146 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4f0c0854e0 con 0x7f4f181512c0 2024-09-13T18:06:03.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.404+0000 7f4f1659c700 1 --2- 172.21.15.40:0/191785146 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4ef8066760 0x7f4ef8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.404+0000 7f4f1659c700 1 --2- 172.21.15.40:0/191785146 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4ef8066760 0x7f4ef8068c20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009b50 tx=0x7f4f08009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:03.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.408+0000 7f4f06ffd700 1 -- 172.21.15.40:0/191785146 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4f0c052490 con 0x7f4f181512c0 2024-09-13T18:06:03.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.652+0000 7f4f1759e700 1 -- 172.21.15.40:0/191785146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4f180c9480 con 0x7f4f181512c0 2024-09-13T18:06:03.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.652+0000 7f4f06ffd700 1 -- 172.21.15.40:0/191785146 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f4f0c055ab0 con 0x7f4f181512c0 2024-09-13T18:06:03.663 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":124,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:06:03.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.656+0000 7f4f1759e700 1 -- 172.21.15.40:0/191785146 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4ef8066760 msgr2=0x7f4ef8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:03.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.656+0000 7f4f1759e700 1 --2- 172.21.15.40:0/191785146 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4ef8066760 0x7f4ef8068c20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009b50 tx=0x7f4f08009240 comp rx=0 tx=0).stop 2024-09-13T18:06:03.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.656+0000 7f4f1759e700 1 -- 172.21.15.40:0/191785146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f181512c0 msgr2=0x7f4f18149c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:03.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.656+0000 7f4f1759e700 1 --2- 172.21.15.40:0/191785146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f181512c0 0x7f4f18149c30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009b90 tx=0x7f4f0c004770 comp rx=0 tx=0).stop 2024-09-13T18:06:03.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.656+0000 7f4f1759e700 1 -- 172.21.15.40:0/191785146 shutdown_connections 2024-09-13T18:06:03.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.656+0000 7f4f1759e700 1 --2- 172.21.15.40:0/191785146 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4ef8066760 0x7f4ef8068c20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.656+0000 7f4f1759e700 1 --2- 172.21.15.40:0/191785146 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4f18146a00 0x7f4f18144810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.656+0000 7f4f1759e700 1 --2- 172.21.15.40:0/191785146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4f181512c0 0x7f4f18149c30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.656+0000 7f4f1759e700 1 --2- 172.21.15.40:0/191785146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f1806fee0 0x7f4f181496f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.656+0000 7f4f1759e700 1 -- 172.21.15.40:0/191785146 >> 172.21.15.40:0/191785146 conn(0x7f4f18066090 msgr2=0x7f4f1814f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:03.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.656+0000 7f4f1759e700 1 -- 172.21.15.40:0/191785146 shutdown_connections 2024-09-13T18:06:03.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.656+0000 7f4f1759e700 1 -- 172.21.15.40:0/191785146 wait complete. 2024-09-13T18:06:03.681 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:06:03.681 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T18:06:03.682 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:06:03.682 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:06:03.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.800+0000 7f667f0db700 1 Processor -- start 2024-09-13T18:06:03.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.800+0000 7f667f0db700 1 -- start start 2024-09-13T18:06:03.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.800+0000 7f667f0db700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f66700d6670 0x7f66700e0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.800+0000 7f667f0db700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6670046df0 0x7f667004de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.800+0000 7f667f0db700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66700dd3c0 con 0x7f667004d0f0 2024-09-13T18:06:03.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.800+0000 7f667f0db700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66700e0180 con 0x7f66700d6670 2024-09-13T18:06:03.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.800+0000 7f667f0db700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66700dcb80 con 0x7f6670046df0 2024-09-13T18:06:03.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.800+0000 7f667e0d9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f66700d6670 0x7f66700e0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.800+0000 7f667e0d9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f66700d6670 0x7f66700e0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53522/0 (socket says 172.21.15.40:53522) 2024-09-13T18:06:03.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.800+0000 7f667e0d9700 1 -- 172.21.15.40:0/2105807403 learned_addr learned my addr 172.21.15.40:0/2105807403 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:03.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.800+0000 7f667e8da700 1 --2- 172.21.15.40:0/2105807403 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6670046df0 0x7f667004de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.800+0000 7f667e0d9700 1 -- 172.21.15.40:0/2105807403 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6670046df0 msgr2=0x7f667004de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:06:03.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.800+0000 7f667e0d9700 1 --2- 172.21.15.40:0/2105807403 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6670046df0 0x7f667004de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.800+0000 7f667e0d9700 1 -- 172.21.15.40:0/2105807403 >> v1:172.21.15.43:6789/0 conn(0x7f667004d0f0 legacy=0x7f66700dc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:03.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.800+0000 7f667e0d9700 1 -- 172.21.15.40:0/2105807403 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6670047260 con 0x7f66700d6670 2024-09-13T18:06:03.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.800+0000 7f667e0d9700 1 --2- 172.21.15.40:0/2105807403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d6670 0x7f66700e0610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6674009910 tx=0x7f6674009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8471cc58854e0a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:03.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.800+0000 7f667d0d7700 1 -- 172.21.15.40:0/2105807403 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667400ba60 con 0x7f66700d6670 2024-09-13T18:06:03.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.800+0000 7f667d0d7700 1 -- 172.21.15.40:0/2105807403 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667400bbc0 con 0x7f66700d6670 2024-09-13T18:06:03.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.800+0000 7f667d0d7700 1 -- 172.21.15.40:0/2105807403 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674009e60 con 0x7f66700d6670 2024-09-13T18:06:03.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.804+0000 7f667f0db700 1 -- 172.21.15.40:0/2105807403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d6670 msgr2=0x7f66700e0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:03.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.804+0000 7f667f0db700 1 --2- 172.21.15.40:0/2105807403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d6670 0x7f66700e0610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6674009910 tx=0x7f6674009c20 comp rx=0 tx=0).stop 2024-09-13T18:06:03.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.804+0000 7f667f0db700 1 -- 172.21.15.40:0/2105807403 shutdown_connections 2024-09-13T18:06:03.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.804+0000 7f667f0db700 1 --2- 172.21.15.40:0/2105807403 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6670046df0 0x7f667004de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.804+0000 7f667f0db700 1 --2- 172.21.15.40:0/2105807403 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d6670 0x7f66700e0610 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.804+0000 7f667f0db700 1 -- 172.21.15.40:0/2105807403 >> 172.21.15.40:0/2105807403 conn(0x7f667000dc40 msgr2=0x7f667000e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:03.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.804+0000 7f667f0db700 1 -- 172.21.15.40:0/2105807403 shutdown_connections 2024-09-13T18:06:03.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.804+0000 7f667f0db700 1 -- 172.21.15.40:0/2105807403 wait complete. 2024-09-13T18:06:03.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667f0db700 1 Processor -- start 2024-09-13T18:06:03.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667f0db700 1 -- start start 2024-09-13T18:06:03.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667f0db700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6670046df0 0x7f66700d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667f0db700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f667004d0f0 0x7f66700c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667f0db700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66700d6670 0x7f66700c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667f0db700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66700deb60 con 0x7f667004d0f0 2024-09-13T18:06:03.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667f0db700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66700dc570 con 0x7f6670046df0 2024-09-13T18:06:03.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667f0db700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66700ddfb0 con 0x7f66700d6670 2024-09-13T18:06:03.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667d8d8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f667004d0f0 0x7f66700c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667d8d8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f667004d0f0 0x7f66700c98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53524/0 (socket says 172.21.15.40:53524) 2024-09-13T18:06:03.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667d8d8700 1 -- 172.21.15.40:0/3631114064 learned_addr learned my addr 172.21.15.40:0/3631114064 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:03.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667e0d9700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6670046df0 0x7f66700d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667e8da700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66700d6670 0x7f66700c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667e0d9700 1 -- 172.21.15.40:0/3631114064 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66700d6670 msgr2=0x7f66700c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:03.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667e0d9700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f66700d6670 0x7f66700c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667e0d9700 1 -- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f667004d0f0 msgr2=0x7f66700c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:03.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667e0d9700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f667004d0f0 0x7f66700c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667e0d9700 1 -- 172.21.15.40:0/3631114064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6674009570 con 0x7f6670046df0 2024-09-13T18:06:03.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667d8d8700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f667004d0f0 0x7f66700c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:06:03.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667e0d9700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6670046df0 0x7f66700d15f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6674016000 tx=0x7f6674004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:03.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f666effd700 1 -- 172.21.15.40:0/3631114064 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674004550 con 0x7f6670046df0 2024-09-13T18:06:03.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f666effd700 1 -- 172.21.15.40:0/3631114064 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667401f070 con 0x7f6670046df0 2024-09-13T18:06:03.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667f0db700 1 -- 172.21.15.40:0/3631114064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66700deff0 con 0x7f6670046df0 2024-09-13T18:06:03.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f666effd700 1 -- 172.21.15.40:0/3631114064 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674003900 con 0x7f6670046df0 2024-09-13T18:06:03.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.808+0000 7f667f0db700 1 -- 172.21.15.40:0/3631114064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66700de230 con 0x7f6670046df0 2024-09-13T18:06:03.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.812+0000 7f667f0db700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d1b30 0x7f66700de3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.812+0000 7f667f0db700 1 -- 172.21.15.40:0/3631114064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66700074a0 con 0x7f66700d1b30 2024-09-13T18:06:03.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.812+0000 7f667d8d8700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d1b30 0x7f66700de3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.812+0000 7f667d8d8700 1 -- 172.21.15.40:0/3631114064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f66700dfb30 con 0x7f66700d1b30 2024-09-13T18:06:03.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.812+0000 7f667d8d8700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d1b30 0x7f66700de3c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f667004f4d0 tx=0x7f666800fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:03.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.812+0000 7f666effd700 1 -- 172.21.15.40:0/3631114064 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6668017070 con 0x7f66700d1b30 2024-09-13T18:06:03.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.812+0000 7f666effd700 1 -- 172.21.15.40:0/3631114064 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6674003a60 con 0x7f6670046df0 2024-09-13T18:06:03.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.812+0000 7f666effd700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f665c0667d0 0x7f665c068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.812+0000 7f666effd700 1 -- 172.21.15.40:0/3631114064 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6674021070 con 0x7f6670046df0 2024-09-13T18:06:03.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.812+0000 7f667e8da700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f665c0667d0 0x7f665c068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.812+0000 7f666effd700 1 -- 172.21.15.40:0/3631114064 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f66680052e0 con 0x7f66700d1b30 2024-09-13T18:06:03.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.812+0000 7f666effd700 1 -- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d1b30 msgr2=0x7f66700de3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:03.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.812+0000 7f666effd700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d1b30 0x7f66700de3c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f667004f4d0 tx=0x7f666800fe70 comp rx=0 tx=0).stop 2024-09-13T18:06:03.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.812+0000 7f666effd700 1 -- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d1b30 msgr2=0x7f66700de3c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:06:03.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.812+0000 7f666effd700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66700d1b30 0x7f66700de3c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.812+0000 7f667e8da700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f665c0667d0 0x7f665c068c90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f66780641b0 tx=0x7f667808e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:03.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f667f0db700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6670177680 0x7f6670177a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f667f0db700 1 -- 172.21.15.40:0/3631114064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66700decd0 con 0x7f6670177680 2024-09-13T18:06:03.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f667d8d8700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6670177680 0x7f6670177a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f667d8d8700 1 -- 172.21.15.40:0/3631114064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f66700dd0e0 con 0x7f6670177680 2024-09-13T18:06:03.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f667d8d8700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6670177680 0x7f6670177a60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6670171ad0 tx=0x7f6668001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:03.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f666effd700 1 -- 172.21.15.40:0/3631114064 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6668022070 con 0x7f6670177680 2024-09-13T18:06:03.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f666effd700 1 -- 172.21.15.40:0/3631114064 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4785 (secure 0 0 0) 0x7f666801de20 con 0x7f6670177680 2024-09-13T18:06:03.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f666effd700 1 -- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6670177680 msgr2=0x7f6670177a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:03.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f666effd700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6670177680 0x7f6670177a60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6670171ad0 tx=0x7f6668001120 comp rx=0 tx=0).stop 2024-09-13T18:06:03.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f666effd700 1 -- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6670177680 msgr2=0x7f6670177a60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:06:03.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f666effd700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6670177680 0x7f6670177a60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.834 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:06:03.834 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:06:03.834 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:06:03.834 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:06:03.834 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 124, 2024-09-13T18:06:03.834 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:06:03.834 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:06:03.834 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:06:03.834 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:06:03.835 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:03.835 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 12, 2024-09-13T18:06:03.835 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:06:03.835 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:06:03.835 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:06:03.835 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:06:03.835 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:06:03.835 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:06:03.835 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:06:03.835 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:06:03.835 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:06:03.835 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:06:03.835 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:06:03.835 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:06:03.836 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:06:03.836 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:03.836 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:06:03.836 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:06:03.836 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:06:03.836 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:06:03.836 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:06:03.836 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:06:03.836 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:06:03.836 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:06:03.836 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:06:03.836 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:06:03.836 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:06:03.837 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:06:03.837 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:03.837 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:03.837 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:06:03.837 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:06:03.837 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:06:03.837 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:06:03.837 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:06:03.837 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:06:03.837 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:06:03.837 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:06:03.837 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:06:03.837 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:06:03.838 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:06:03.838 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:06:03.838 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:06:03.838 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:06:03.838 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:06:03.838 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:06:03.838 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:06:03.838 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:06:03.838 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:06:03.838 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:06:03.838 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:06:03.838 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:06:03.838 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:06:03.839 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:06:03.839 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:06:03.839 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:06:03.839 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:06:03.839 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:03.839 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:06:03.839 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:03.839 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:06:03.839 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:03.839 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:06:03.839 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:06:03.839 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:06:03.839 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:06:03.839 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:03.840 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:06:03.840 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:06:03.840 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:03.840 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:03.840 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:03.840 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:06:03.840 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:06:03.840 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:03.840 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:03.840 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:03.840 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:03.840 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:06:03.840 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:06:03.841 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:06:03.841 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:06:03.841 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:06:03.841 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:03.841 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:03.841 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:06:03.841 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:06:03.841 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:06:03.841 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:06:03.841 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:03.841 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:06:03.841 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:06:03.841 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:03.841 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:03.842 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:03.842 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:06:03.842 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:06:03.842 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:03.842 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:03.842 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:03.842 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:03.842 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:06:03.842 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:06:03.842 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:06:03.842 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:06:03.842 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:06:03.842 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:03.842 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:03.842 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:06:03.843 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:06:03.843 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:06:03.843 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:06:03.843 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:03.843 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:06:03.843 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:06:03.843 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:03.843 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:03.843 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:03.843 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:06:03.843 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:06:03.843 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:03.843 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:03.843 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:03.844 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:03.844 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:06:03.844 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:06:03.844 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:06:03.844 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:06:03.844 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:06:03.844 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:03.844 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:03.844 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:03.844 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:06:03.844 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:06:03.844 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:03.844 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:06:03.845 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:06:03.845 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:06:03.845 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:03.845 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:03.845 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:06:03.845 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:03.845 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:06:03.845 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:06:03.845 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:06:03.845 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:03.845 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:03.845 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:06:03.845 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:03.845 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:06:03.846 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:06:03.846 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:06:03.846 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:03.846 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:03.846 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:06:03.846 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:03.846 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:06:03.846 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:06:03.846 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:06:03.846 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:03.846 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:03.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:03.846 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:06:03.846 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:06:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f667f0db700 1 -- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f665c0667d0 msgr2=0x7f665c068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f667f0db700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f665c0667d0 0x7f665c068c90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f66780641b0 tx=0x7f667808e040 comp rx=0 tx=0).stop 2024-09-13T18:06:03.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f667f0db700 1 -- 172.21.15.40:0/3631114064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6670046df0 msgr2=0x7f66700d15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:03.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f667f0db700 1 --2- 172.21.15.40:0/3631114064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6670046df0 0x7f66700d15f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6674016000 tx=0x7f6674004770 comp rx=0 tx=0).stop 2024-09-13T18:06:03.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f667e8da700 1 -- 172.21.15.40:0/3631114064 reap_dead start 2024-09-13T18:06:03.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f667e8da700 1 -- 172.21.15.40:0/3631114064 reap_dead start 2024-09-13T18:06:03.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f667f0db700 1 -- 172.21.15.40:0/3631114064 shutdown_connections 2024-09-13T18:06:03.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f667f0db700 1 -- 172.21.15.40:0/3631114064 >> 172.21.15.40:0/3631114064 conn(0x7f667000dc40 msgr2=0x7f66700d4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:03.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f667f0db700 1 -- 172.21.15.40:0/3631114064 shutdown_connections 2024-09-13T18:06:03.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.824+0000 7f667f0db700 1 -- 172.21.15.40:0/3631114064 wait complete. 2024-09-13T18:06:03.849 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:06:03.849 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:06:03.849 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:06:03.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.964+0000 7f1d8db70700 1 Processor -- start 2024-09-13T18:06:03.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.964+0000 7f1d8db70700 1 -- start start 2024-09-13T18:06:03.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.964+0000 7f1d8db70700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d881503c0 0x7f1d880c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.964+0000 7f1d8db70700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1d88159630 0x7f1d88159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.964+0000 7f1d8db70700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d880c8f80 con 0x7f1d880c8500 2024-09-13T18:06:03.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.964+0000 7f1d8db70700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d880c2a80 con 0x7f1d88159630 2024-09-13T18:06:03.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.964+0000 7f1d8db70700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8815a020 con 0x7f1d881503c0 2024-09-13T18:06:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.964+0000 7f1d8cb6e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d881503c0 0x7f1d880c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.964+0000 7f1d8d36f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1d88159630 0x7f1d88159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.964+0000 7f1d8cb6e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d881503c0 0x7f1d880c79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:44008/0 (socket says 172.21.15.40:44008) 2024-09-13T18:06:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.964+0000 7f1d8cb6e700 1 -- 172.21.15.40:0/488419662 learned_addr learned my addr 172.21.15.40:0/488419662 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.968+0000 7f1d8cb6e700 1 -- 172.21.15.40:0/488419662 >> v2:172.21.15.43:3300/0 conn(0x7f1d88159630 msgr2=0x7f1d88159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:06:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.968+0000 7f1d8cb6e700 1 --2- 172.21.15.40:0/488419662 >> v2:172.21.15.43:3300/0 conn(0x7f1d88159630 0x7f1d88159ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.968+0000 7f1d8cb6e700 1 -- 172.21.15.40:0/488419662 >> v1:172.21.15.43:6789/0 conn(0x7f1d880c8500 legacy=0x7f1d880c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:03.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.968+0000 7f1d8cb6e700 1 -- 172.21.15.40:0/488419662 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d881507a0 con 0x7f1d881503c0 2024-09-13T18:06:03.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.968+0000 7f1d8cb6e700 1 --2- 172.21.15.40:0/488419662 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d881503c0 0x7f1d880c79c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009e50 tx=0x7f1d8000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee202df74cd6b670 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:03.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.968+0000 7f1d877fe700 1 -- 172.21.15.40:0/488419662 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8000e070 con 0x7f1d881503c0 2024-09-13T18:06:03.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.968+0000 7f1d877fe700 1 -- 172.21.15.40:0/488419662 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8000b910 con 0x7f1d881503c0 2024-09-13T18:06:03.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.968+0000 7f1d877fe700 1 -- 172.21.15.40:0/488419662 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8000bcc0 con 0x7f1d881503c0 2024-09-13T18:06:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.968+0000 7f1d8db70700 1 -- 172.21.15.40:0/488419662 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d881503c0 msgr2=0x7f1d880c79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.968+0000 7f1d8db70700 1 --2- 172.21.15.40:0/488419662 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d881503c0 0x7f1d880c79c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009e50 tx=0x7f1d8000b330 comp rx=0 tx=0).stop 2024-09-13T18:06:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.968+0000 7f1d8db70700 1 -- 172.21.15.40:0/488419662 shutdown_connections 2024-09-13T18:06:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.968+0000 7f1d8db70700 1 --2- 172.21.15.40:0/488419662 >> v2:172.21.15.43:3300/0 conn(0x7f1d88159630 0x7f1d88159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.968+0000 7f1d8db70700 1 --2- 172.21.15.40:0/488419662 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d881503c0 0x7f1d880c79c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.968+0000 7f1d8db70700 1 -- 172.21.15.40:0/488419662 >> 172.21.15.40:0/488419662 conn(0x7f1d8806e6d0 msgr2=0x7f1d8806eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:03.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.968+0000 7f1d8db70700 1 -- 172.21.15.40:0/488419662 shutdown_connections 2024-09-13T18:06:03.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.968+0000 7f1d8db70700 1 -- 172.21.15.40:0/488419662 wait complete. 2024-09-13T18:06:03.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.968+0000 7f1d8db70700 1 Processor -- start 2024-09-13T18:06:03.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.968+0000 7f1d8db70700 1 -- start start 2024-09-13T18:06:03.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d8db70700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d880c8500 0x7f1d88149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d8db70700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d881503c0 0x7f1d880c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d8db70700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d88159630 0x7f1d880cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d8db70700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d881ecd60 con 0x7f1d881503c0 2024-09-13T18:06:03.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d8db70700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d881ecf50 con 0x7f1d88159630 2024-09-13T18:06:03.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d8db70700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d881ed230 con 0x7f1d880c8500 2024-09-13T18:06:03.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d87fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d881503c0 0x7f1d880c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d87fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d881503c0 0x7f1d880c58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53538/0 (socket says 172.21.15.40:53538) 2024-09-13T18:06:03.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d87fff700 1 -- 172.21.15.40:0/3207105164 learned_addr learned my addr 172.21.15.40:0/3207105164 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:03.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d8d36f700 1 --2- 172.21.15.40:0/3207105164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d88159630 0x7f1d880cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d8cb6e700 1 --2- 172.21.15.40:0/3207105164 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d880c8500 0x7f1d88149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d87fff700 1 -- 172.21.15.40:0/3207105164 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d880c8500 msgr2=0x7f1d88149e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:03.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d87fff700 1 --2- 172.21.15.40:0/3207105164 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d880c8500 0x7f1d88149e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d87fff700 1 -- 172.21.15.40:0/3207105164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d88159630 msgr2=0x7f1d880cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d87fff700 1 --2- 172.21.15.40:0/3207105164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d88159630 0x7f1d880cbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d87fff700 1 -- 172.21.15.40:0/3207105164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d80009b30 con 0x7f1d881503c0 2024-09-13T18:06:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d87fff700 1 --2- 172.21.15.40:0/3207105164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d881503c0 0x7f1d880c58e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800c500 tx=0x7f1d7800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d857fa700 1 -- 172.21.15.40:0/3207105164 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d780052d0 con 0x7f1d881503c0 2024-09-13T18:06:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d857fa700 1 -- 172.21.15.40:0/3207105164 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d78005430 con 0x7f1d881503c0 2024-09-13T18:06:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d857fa700 1 -- 172.21.15.40:0/3207105164 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d780055a0 con 0x7f1d881503c0 2024-09-13T18:06:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d8db70700 1 -- 172.21.15.40:0/3207105164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d881ed510 con 0x7f1d881503c0 2024-09-13T18:06:03.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d8db70700 1 -- 172.21.15.40:0/3207105164 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d881edad0 con 0x7f1d881503c0 2024-09-13T18:06:03.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d8db70700 1 --2- 172.21.15.40:0/3207105164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d68002a40 0x7f1d68004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d8db70700 1 -- 172.21.15.40:0/3207105164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d68007f40 con 0x7f1d68002a40 2024-09-13T18:06:03.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.972+0000 7f1d8cb6e700 1 --2- 172.21.15.40:0/3207105164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d68002a40 0x7f1d68004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.976+0000 7f1d8cb6e700 1 -- 172.21.15.40:0/3207105164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d68008100 con 0x7f1d68002a40 2024-09-13T18:06:03.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.976+0000 7f1d857fa700 1 -- 172.21.15.40:0/3207105164 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1d780038e0 con 0x7f1d881503c0 2024-09-13T18:06:03.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.976+0000 7f1d8cb6e700 1 --2- 172.21.15.40:0/3207105164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d68002a40 0x7f1d68004f00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009e50 tx=0x7f1d80004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:03.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.976+0000 7f1d857fa700 1 --2- 172.21.15.40:0/3207105164 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d74066730 0x7f1d74068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.976+0000 7f1d857fa700 1 -- 172.21.15.40:0/3207105164 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1d780838f0 con 0x7f1d881503c0 2024-09-13T18:06:03.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.976+0000 7f1d8d36f700 1 --2- 172.21.15.40:0/3207105164 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d74066730 0x7f1d74068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.976+0000 7f1d857fa700 1 -- 172.21.15.40:0/3207105164 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d800048e0 con 0x7f1d68002a40 2024-09-13T18:06:03.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.976+0000 7f1d857fa700 1 -- 172.21.15.40:0/3207105164 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f1d80026470 con 0x7f1d68002a40 2024-09-13T18:06:03.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.976+0000 7f1d857fa700 1 -- 172.21.15.40:0/3207105164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d68002a40 msgr2=0x7f1d68004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:03.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.976+0000 7f1d857fa700 1 --2- 172.21.15.40:0/3207105164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d68002a40 0x7f1d68004f00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009e50 tx=0x7f1d80004af0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.976+0000 7f1d857fa700 1 -- 172.21.15.40:0/3207105164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d68002a40 msgr2=0x7f1d68004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:06:03.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.976+0000 7f1d857fa700 1 --2- 172.21.15.40:0/3207105164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d68002a40 0x7f1d68004f00 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009e50 tx=0x7f1d80004af0 comp rx=0 tx=0).stop 2024-09-13T18:06:03.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.976+0000 7f1d8d36f700 1 --2- 172.21.15.40:0/3207105164 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d74066730 0x7f1d74068bf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c009b50 tx=0x7f1d7c009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:03.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.988+0000 7f1d8db70700 1 --2- 172.21.15.40:0/3207105164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d68007020 0x7f1d68007490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:03.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.988+0000 7f1d8db70700 1 -- 172.21.15.40:0/3207105164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6800a400 con 0x7f1d68007020 2024-09-13T18:06:03.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.988+0000 7f1d8cb6e700 1 --2- 172.21.15.40:0/3207105164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d68007020 0x7f1d68007490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:03.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.988+0000 7f1d8cb6e700 1 -- 172.21.15.40:0/3207105164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1d6800a5c0 con 0x7f1d68007020 2024-09-13T18:06:03.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.988+0000 7f1d8cb6e700 1 --2- 172.21.15.40:0/3207105164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d68007020 0x7f1d68007490 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1d680080d0 tx=0x7f1d80011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:03.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.988+0000 7f1d857fa700 1 -- 172.21.15.40:0/3207105164 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8002b780 con 0x7f1d68007020 2024-09-13T18:06:03.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.992+0000 7f1d857fa700 1 -- 172.21.15.40:0/3207105164 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4451 (secure 0 0 0) 0x7f1d8002b8e0 con 0x7f1d68007020 2024-09-13T18:06:03.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.992+0000 7f1d857fa700 1 -- 172.21.15.40:0/3207105164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d68007020 msgr2=0x7f1d68007490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:03.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.992+0000 7f1d857fa700 1 --2- 172.21.15.40:0/3207105164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d68007020 0x7f1d68007490 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1d680080d0 tx=0x7f1d80011000 comp rx=0 tx=0).stop 2024-09-13T18:06:03.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.992+0000 7f1d857fa700 1 -- 172.21.15.40:0/3207105164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d68007020 msgr2=0x7f1d68007490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:06:03.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.992+0000 7f1d857fa700 1 --2- 172.21.15.40:0/3207105164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d68007020 0x7f1d68007490 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1d680080d0 tx=0x7f1d80011000 comp rx=0 tx=0).stop 2024-09-13T18:06:04.000 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:06:04.000 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:06:04.000 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:06:04.000 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:06:04.000 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 124, 2024-09-13T18:06:04.001 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:06:04.001 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:06:04.001 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:06:04.001 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:06:04.001 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:04.001 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 13, 2024-09-13T18:06:04.001 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:06:04.001 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:06:04.001 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:06:04.001 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:06:04.001 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:06:04.001 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:06:04.001 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:06:04.001 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:06:04.002 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:06:04.002 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:06:04.002 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:06:04.002 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:06:04.002 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:06:04.002 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:04.002 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:06:04.002 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:06:04.002 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:06:04.002 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:06:04.002 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:06:04.002 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:06:04.002 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:06:04.002 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:06:04.003 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:06:04.003 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:06:04.003 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:06:04.003 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:06:04.003 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:04.003 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.003 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:06:04.003 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:06:04.003 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:06:04.003 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:06:04.003 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:06:04.003 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:06:04.003 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:06:04.003 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:06:04.003 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:06:04.004 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:06:04.004 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:06:04.004 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:06:04.004 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:06:04.004 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:06:04.004 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:06:04.004 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:06:04.004 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:06:04.004 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:06:04.004 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:06:04.004 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:06:04.004 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:06:04.004 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:06:04.004 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:06:04.005 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:06:04.005 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:06:04.005 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:06:04.005 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:06:04.005 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:04.005 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:06:04.005 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.005 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:06:04.005 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.005 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:06:04.005 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:06:04.005 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:06:04.005 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:06:04.005 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.006 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:06:04.006 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:06:04.006 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:04.006 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.006 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.006 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:06:04.006 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:06:04.006 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:04.006 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:04.006 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:04.006 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.006 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:06:04.006 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:06:04.006 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:06:04.007 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:06:04.007 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:06:04.007 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.007 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.007 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:06:04.007 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:06:04.007 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:06:04.007 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:06:04.007 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.007 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:06:04.007 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:06:04.007 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:04.007 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.007 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.007 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:06:04.008 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:06:04.008 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:04.008 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:04.008 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:04.008 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.008 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:06:04.008 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:06:04.008 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:06:04.008 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:06:04.008 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:06:04.008 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.008 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.008 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:06:04.008 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:06:04.009 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:06:04.009 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:06:04.009 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.009 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:06:04.009 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:06:04.009 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:04.009 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.009 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.009 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:06:04.009 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:06:04.009 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:04.009 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:04.009 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:04.009 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:06:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:06:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:06:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:06:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:06:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:06:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:06:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:06:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:06:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:06:04.010 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:06:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:06:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:06:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:06:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.011 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:06:04.011 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:06:04.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.992+0000 7f1d8db70700 1 -- 172.21.15.40:0/3207105164 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d74066730 msgr2=0x7f1d74068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:04.012 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.992+0000 7f1d8db70700 1 --2- 172.21.15.40:0/3207105164 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d74066730 0x7f1d74068bf0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c009b50 tx=0x7f1d7c009240 comp rx=0 tx=0).stop 2024-09-13T18:06:04.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.992+0000 7f1d8db70700 1 -- 172.21.15.40:0/3207105164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d881503c0 msgr2=0x7f1d880c58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:04.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.992+0000 7f1d8db70700 1 --2- 172.21.15.40:0/3207105164 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d881503c0 0x7f1d880c58e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800c500 tx=0x7f1d7800c810 comp rx=0 tx=0).stop 2024-09-13T18:06:04.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.992+0000 7f1d8d36f700 1 -- 172.21.15.40:0/3207105164 reap_dead start 2024-09-13T18:06:04.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.992+0000 7f1d8d36f700 1 -- 172.21.15.40:0/3207105164 reap_dead start 2024-09-13T18:06:04.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.992+0000 7f1d8db70700 1 -- 172.21.15.40:0/3207105164 shutdown_connections 2024-09-13T18:06:04.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.992+0000 7f1d8db70700 1 -- 172.21.15.40:0/3207105164 >> 172.21.15.40:0/3207105164 conn(0x7f1d8806e6d0 msgr2=0x7f1d8814d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:04.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.992+0000 7f1d8db70700 1 -- 172.21.15.40:0/3207105164 shutdown_connections 2024-09-13T18:06:04.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:03.992+0000 7f1d8db70700 1 -- 172.21.15.40:0/3207105164 wait complete. 2024-09-13T18:06:04.014 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:06:04.014 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:06:04.015 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:06:04.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.120+0000 7f414759e700 1 Processor -- start 2024-09-13T18:06:04.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.120+0000 7f414759e700 1 -- start start 2024-09-13T18:06:04.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.124+0000 7f414759e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4148156b80 0x7f414806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:04.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.124+0000 7f414759e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41481599e0 con 0x7f41481595b0 2024-09-13T18:06:04.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.124+0000 7f414759e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f414815d8a0 con 0x7f41480b21a0 2024-09-13T18:06:04.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.124+0000 7f414759e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f414806bde0 con 0x7f4148156b80 2024-09-13T18:06:04.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.124+0000 7f4146d9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f41481595b0 0x7f4148163e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36756/0 (socket says 172.21.15.40:36756) 2024-09-13T18:06:04.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.124+0000 7f4146d9d700 1 -- 172.21.15.40:0/3244685897 learned_addr learned my addr 172.21.15.40:0/3244685897 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:06:04.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.124+0000 7f414659c700 1 --2- 172.21.15.40:0/3244685897 >> v2:172.21.15.43:3300/0 conn(0x7f4148156b80 0x7f414806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:04.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.124+0000 7f4145d9b700 1 -- 172.21.15.40:0/3244685897 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1433423316 0 0) 0x7f4140001490 con 0x7f41481595b0 2024-09-13T18:06:04.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.124+0000 7f4145d9b700 1 -- 172.21.15.40:0/3244685897 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41340035d0 con 0x7f41481595b0 2024-09-13T18:06:04.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.124+0000 7f414659c700 1 -- 172.21.15.40:0/3244685897 >> v1:172.21.15.43:6789/0 conn(0x7f41480b21a0 legacy=0x7f41480cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:04.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.124+0000 7f414659c700 1 -- 172.21.15.40:0/3244685897 >> v1:172.21.15.40:6789/0 conn(0x7f41481595b0 legacy=0x7f4148163e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:04.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.124+0000 7f414659c700 1 -- 172.21.15.40:0/3244685897 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414806e1d0 con 0x7f4148156b80 2024-09-13T18:06:04.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.124+0000 7f414659c700 1 --2- 172.21.15.40:0/3244685897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4148156b80 0x7f414806e9a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4138009850 tx=0x7f4138009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=476758d10653ef41 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:04.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.124+0000 7f4145d9b700 1 -- 172.21.15.40:0/3244685897 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4138011830 con 0x7f4148156b80 2024-09-13T18:06:04.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.124+0000 7f4145d9b700 1 -- 172.21.15.40:0/3244685897 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4138011990 con 0x7f4148156b80 2024-09-13T18:06:04.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.124+0000 7f4145d9b700 1 -- 172.21.15.40:0/3244685897 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4138011d00 con 0x7f4148156b80 2024-09-13T18:06:04.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.124+0000 7f414759e700 1 -- 172.21.15.40:0/3244685897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4148156b80 msgr2=0x7f414806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:04.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.124+0000 7f414759e700 1 --2- 172.21.15.40:0/3244685897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4148156b80 0x7f414806e9a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4138009850 tx=0x7f4138009b60 comp rx=0 tx=0).stop 2024-09-13T18:06:04.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.128+0000 7f414759e700 1 -- 172.21.15.40:0/3244685897 shutdown_connections 2024-09-13T18:06:04.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.128+0000 7f414759e700 1 --2- 172.21.15.40:0/3244685897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4148156b80 0x7f414806e9a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:04.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.128+0000 7f414759e700 1 -- 172.21.15.40:0/3244685897 >> 172.21.15.40:0/3244685897 conn(0x7f4148072070 msgr2=0x7f4148072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:04.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.128+0000 7f414759e700 1 -- 172.21.15.40:0/3244685897 shutdown_connections 2024-09-13T18:06:04.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.128+0000 7f414759e700 1 -- 172.21.15.40:0/3244685897 wait complete. 2024-09-13T18:06:04.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.128+0000 7f414759e700 1 Processor -- start 2024-09-13T18:06:04.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.128+0000 7f414759e700 1 -- start start 2024-09-13T18:06:04.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.128+0000 7f414759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41480b21a0 0x7f414815b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:04.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.128+0000 7f414759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4148156b80 0x7f41481e38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:04.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.128+0000 7f414759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41481595b0 0x7f41480cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:04.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.128+0000 7f414759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41480b1ae0 con 0x7f41480b21a0 2024-09-13T18:06:04.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.128+0000 7f414759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41481e29c0 con 0x7f4148156b80 2024-09-13T18:06:04.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.128+0000 7f414759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414815c2d0 con 0x7f41481595b0 2024-09-13T18:06:04.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.128+0000 7f414659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41480b21a0 0x7f414815b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:04.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.128+0000 7f414659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41480b21a0 0x7f414815b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53554/0 (socket says 172.21.15.40:53554) 2024-09-13T18:06:04.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.128+0000 7f414659c700 1 -- 172.21.15.40:0/623383501 learned_addr learned my addr 172.21.15.40:0/623383501 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:04.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.128+0000 7f413ffff700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4148156b80 0x7f41481e38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:04.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.128+0000 7f4146d9d700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41481595b0 0x7f41480cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:04.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f414659c700 1 -- 172.21.15.40:0/623383501 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41481595b0 msgr2=0x7f41480cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:04.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f414659c700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41481595b0 0x7f41480cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:04.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f414659c700 1 -- 172.21.15.40:0/623383501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4148156b80 msgr2=0x7f41481e38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:04.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f414659c700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4148156b80 0x7f41481e38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:04.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f414659c700 1 -- 172.21.15.40:0/623383501 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41380094b0 con 0x7f41480b21a0 2024-09-13T18:06:04.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f414659c700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41480b21a0 0x7f414815b400 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4138009ba0 tx=0x7f4138004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:04.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f413effd700 1 -- 172.21.15.40:0/623383501 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4138004550 con 0x7f41480b21a0 2024-09-13T18:06:04.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f413effd700 1 -- 172.21.15.40:0/623383501 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413801f070 con 0x7f41480b21a0 2024-09-13T18:06:04.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f414759e700 1 -- 172.21.15.40:0/623383501 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f414815c5b0 con 0x7f41480b21a0 2024-09-13T18:06:04.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f413effd700 1 -- 172.21.15.40:0/623383501 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4138003900 con 0x7f41480b21a0 2024-09-13T18:06:04.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f414759e700 1 -- 172.21.15.40:0/623383501 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f414815ba60 con 0x7f41480b21a0 2024-09-13T18:06:04.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f414759e700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4148155bc0 0x7f41481f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:04.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f414759e700 1 -- 172.21.15.40:0/623383501 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4148074750 con 0x7f4148155bc0 2024-09-13T18:06:04.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f413ffff700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4148155bc0 0x7f41481f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:04.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f413effd700 1 -- 172.21.15.40:0/623383501 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4138003a60 con 0x7f41480b21a0 2024-09-13T18:06:04.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f413ffff700 1 -- 172.21.15.40:0/623383501 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f414806b820 con 0x7f4148155bc0 2024-09-13T18:06:04.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f413ffff700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4148155bc0 0x7f41481f23e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4148159500 tx=0x7f413000fb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:04.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f413effd700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4128066760 0x7f4128068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:04.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f413effd700 1 -- 172.21.15.40:0/623383501 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4138021070 con 0x7f41480b21a0 2024-09-13T18:06:04.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f4146d9d700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4128066760 0x7f4128068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:04.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.132+0000 7f413effd700 1 -- 172.21.15.40:0/623383501 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4130005320 con 0x7f4148155bc0 2024-09-13T18:06:04.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.136+0000 7f4146d9d700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4128066760 0x7f4128068c20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f41400015f0 tx=0x7f4140001980 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:04.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.136+0000 7f413effd700 1 -- 172.21.15.40:0/623383501 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4130005480 con 0x7f4148155bc0 2024-09-13T18:06:04.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.136+0000 7f413effd700 1 -- 172.21.15.40:0/623383501 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4148155bc0 msgr2=0x7f41481f23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:04.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.136+0000 7f413effd700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4148155bc0 0x7f41481f23e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4148159500 tx=0x7f413000fb00 comp rx=0 tx=0).stop 2024-09-13T18:06:04.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.136+0000 7f413effd700 1 -- 172.21.15.40:0/623383501 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4148155bc0 msgr2=0x7f41481f23e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:06:04.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.136+0000 7f413effd700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4148155bc0 0x7f41481f23e0 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4148159500 tx=0x7f413000fb00 comp rx=0 tx=0).stop 2024-09-13T18:06:04.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.144+0000 7f414759e700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41481f2920 0x7f41481f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:04.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.144+0000 7f414759e700 1 -- 172.21.15.40:0/623383501 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414815c7a0 con 0x7f41481f2920 2024-09-13T18:06:04.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.144+0000 7f413ffff700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41481f2920 0x7f41481f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:04.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.144+0000 7f413ffff700 1 -- 172.21.15.40:0/623383501 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f414806b580 con 0x7f41481f2920 2024-09-13T18:06:04.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.144+0000 7f413ffff700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41481f2920 0x7f41481f2d30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4148159500 tx=0x7f4130007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:04.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.144+0000 7f413effd700 1 -- 172.21.15.40:0/623383501 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4130023070 con 0x7f41481f2920 2024-09-13T18:06:04.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.144+0000 7f413effd700 1 -- 172.21.15.40:0/623383501 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4949 (secure 0 0 0) 0x7f413001c2a0 con 0x7f41481f2920 2024-09-13T18:06:04.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.144+0000 7f413effd700 1 -- 172.21.15.40:0/623383501 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41481f2920 msgr2=0x7f41481f2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:04.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.144+0000 7f413effd700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41481f2920 0x7f41481f2d30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4148159500 tx=0x7f4130007f90 comp rx=0 tx=0).stop 2024-09-13T18:06:04.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.144+0000 7f413effd700 1 -- 172.21.15.40:0/623383501 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41481f2920 msgr2=0x7f41481f2d30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:06:04.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.144+0000 7f413effd700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41481f2920 0x7f41481f2d30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:04.155 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:06:04.155 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:06:04.155 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:06:04.155 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:06:04.155 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 124, 2024-09-13T18:06:04.155 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:06:04.155 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:06:04.156 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:06:04.156 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:06:04.156 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:04.156 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 13, 2024-09-13T18:06:04.156 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:06:04.156 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:06:04.156 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:06:04.156 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:06:04.156 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:06:04.156 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:06:04.156 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:06:04.156 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:06:04.156 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:06:04.157 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:06:04.157 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:06:04.157 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:06:04.157 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:06:04.157 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:04.157 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:06:04.157 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:06:04.157 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:06:04.157 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:06:04.157 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:06:04.158 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:06:04.158 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:06:04.158 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:06:04.158 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:06:04.158 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:06:04.158 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:06:04.159 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:06:04.159 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:04.159 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.159 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:06:04.159 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:06:04.159 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:06:04.159 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:06:04.159 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:06:04.159 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:06:04.159 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:06:04.159 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:06:04.159 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:06:04.159 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:06:04.159 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:06:04.160 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:06:04.160 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:06:04.160 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:06:04.160 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:06:04.160 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:06:04.160 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:06:04.160 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:06:04.160 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:06:04.160 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:06:04.160 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:06:04.160 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:06:04.160 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:06:04.160 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:06:04.160 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:06:04.161 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:06:04.161 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:06:04.161 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:04.161 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:06:04.161 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.161 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:06:04.161 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.161 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:06:04.161 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:06:04.161 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:06:04.161 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:06:04.161 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.161 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:06:04.161 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:06:04.162 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:04.162 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.162 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.162 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:06:04.162 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:06:04.162 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:04.162 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:04.162 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:04.162 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.162 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:06:04.162 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:06:04.162 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:06:04.162 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:06:04.162 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:06:04.162 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.163 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.163 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:06:04.163 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:06:04.163 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:06:04.163 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:06:04.163 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.163 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:06:04.163 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:06:04.163 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:04.163 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.163 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.163 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:06:04.163 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:06:04.163 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:04.163 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:04.164 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:04.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.164 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:06:04.164 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:06:04.164 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:06:04.164 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:06:04.164 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:06:04.164 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.164 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.164 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:06:04.164 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:06:04.164 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:06:04.164 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:06:04.164 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.165 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:06:04.165 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:06:04.165 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:04.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.165 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.165 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:06:04.165 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:06:04.165 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:04.165 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:04.165 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:04.165 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.165 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:06:04.165 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:06:04.165 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:06:04.166 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:06:04.166 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:06:04.166 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:04.166 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:04.166 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.166 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:06:04.166 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:06:04.166 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.166 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:06:04.166 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:06:04.166 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:06:04.166 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:04.166 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:04.166 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:06:04.166 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.167 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:06:04.167 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:06:04.167 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:06:04.167 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:04.167 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:04.167 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:06:04.167 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.167 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:06:04.167 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:06:04.167 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:06:04.167 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:04.167 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:04.167 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:06:04.167 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.168 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:06:04.168 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:06:04.168 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:06:04.168 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:04.168 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:04.168 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:06:04.168 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:04.168 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:06:04.168 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:06:04.168 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:06:04.168 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:04.168 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:04.168 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:04.168 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:06:04.169 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:06:04.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.148+0000 7f414759e700 1 -- 172.21.15.40:0/623383501 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4128066760 msgr2=0x7f4128068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:04.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.148+0000 7f414759e700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4128066760 0x7f4128068c20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f41400015f0 tx=0x7f4140001980 comp rx=0 tx=0).stop 2024-09-13T18:06:04.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.148+0000 7f414759e700 1 -- 172.21.15.40:0/623383501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41480b21a0 msgr2=0x7f414815b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:04.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.148+0000 7f414759e700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41480b21a0 0x7f414815b400 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4138009ba0 tx=0x7f4138004770 comp rx=0 tx=0).stop 2024-09-13T18:06:04.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.148+0000 7f4146d9d700 1 -- 172.21.15.40:0/623383501 reap_dead start 2024-09-13T18:06:04.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.148+0000 7f414759e700 1 -- 172.21.15.40:0/623383501 shutdown_connections 2024-09-13T18:06:04.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.148+0000 7f414759e700 1 --2- 172.21.15.40:0/623383501 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41480b21a0 0x7f414815b400 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:04.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.148+0000 7f414759e700 1 -- 172.21.15.40:0/623383501 >> 172.21.15.40:0/623383501 conn(0x7f4148072070 msgr2=0x7f41480c1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:04.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.148+0000 7f414759e700 1 -- 172.21.15.40:0/623383501 shutdown_connections 2024-09-13T18:06:04.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.148+0000 7f414759e700 1 -- 172.21.15.40:0/623383501 wait complete. 2024-09-13T18:06:04.171 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:06:04.171 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-13T18:06:04.171 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T18:06:04.171 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-13T18:06:04.172 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T18:06:04.172 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T18:06:04.172 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:06:04.213 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T18:06:04.213 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T18:06:04.213 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:06:04.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.332+0000 7febc3191700 1 Processor -- start 2024-09-13T18:06:04.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.332+0000 7febc3191700 1 -- start start 2024-09-13T18:06:04.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.332+0000 7febc3191700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7febbc15cf60 0x7febbc15d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:04.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.332+0000 7febc218f700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7febbc153880 legacy=0x7febbc0c9ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:06:04.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.332+0000 7febc3191700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febbc15d910 con 0x7febbc153880 2024-09-13T18:06:04.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.332+0000 7febc3191700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febbc159770 con 0x7febbc0cb2f0 2024-09-13T18:06:04.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.332+0000 7febc3191700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febbc159a50 con 0x7febbc15cf60 2024-09-13T18:06:04.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.336+0000 7febc2990700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7febbc15cf60 0x7febbc15d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:04.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.336+0000 7febc198e700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7febbc0cb2f0 0x7febbc0cee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:46372/0 (socket says 172.21.15.40:46372) 2024-09-13T18:06:04.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.336+0000 7febc198e700 1 -- 172.21.15.40:0/3580931554 learned_addr learned my addr 172.21.15.40:0/3580931554 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:06:04.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.336+0000 7febc2990700 1 -- 172.21.15.40:0/3580931554 >> v1:172.21.15.43:6789/0 conn(0x7febbc0cb2f0 legacy=0x7febbc0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:04.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.336+0000 7febc2990700 1 -- 172.21.15.40:0/3580931554 >> v1:172.21.15.40:6789/0 conn(0x7febbc153880 legacy=0x7febbc0c9ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:06:04.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.336+0000 7febc2990700 1 -- 172.21.15.40:0/3580931554 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc159d30 con 0x7febbc15cf60 2024-09-13T18:06:04.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.336+0000 7febc2990700 1 --2- 172.21.15.40:0/3580931554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febbc15cf60 0x7febbc15d3d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7febb4009d90 tx=0x7febb400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50b44c639743f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:04.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.336+0000 7febc118d700 1 -- 172.21.15.40:0/3580931554 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb400e070 con 0x7febbc15cf60 2024-09-13T18:06:04.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.336+0000 7febc118d700 1 -- 172.21.15.40:0/3580931554 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb400bb90 con 0x7febbc15cf60 2024-09-13T18:06:04.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.336+0000 7febc118d700 1 -- 172.21.15.40:0/3580931554 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb4003a40 con 0x7febbc15cf60 2024-09-13T18:06:04.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.336+0000 7febc3191700 1 -- 172.21.15.40:0/3580931554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febbc15cf60 msgr2=0x7febbc15d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:04.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.336+0000 7febc3191700 1 --2- 172.21.15.40:0/3580931554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febbc15cf60 0x7febbc15d3d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7febb4009d90 tx=0x7febb400b330 comp rx=0 tx=0).stop 2024-09-13T18:06:04.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.336+0000 7febc3191700 1 -- 172.21.15.40:0/3580931554 shutdown_connections 2024-09-13T18:06:04.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.336+0000 7febc3191700 1 --2- 172.21.15.40:0/3580931554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febbc15cf60 0x7febbc15d3d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:04.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.336+0000 7febc3191700 1 -- 172.21.15.40:0/3580931554 >> 172.21.15.40:0/3580931554 conn(0x7febbc06c5b0 msgr2=0x7febbc06c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:04.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.336+0000 7febc3191700 1 -- 172.21.15.40:0/3580931554 shutdown_connections 2024-09-13T18:06:04.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.336+0000 7febc3191700 1 -- 172.21.15.40:0/3580931554 wait complete. 2024-09-13T18:06:04.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.336+0000 7febc3191700 1 Processor -- start 2024-09-13T18:06:04.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.336+0000 7febc3191700 1 -- start start 2024-09-13T18:06:04.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febc3191700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7febbc0cb2f0 0x7febbc158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:04.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febc3191700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febbc153880 0x7febbc0cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:04.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febc3191700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febbc15cf60 0x7febbc15b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:04.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febc3191700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc1efa90 con 0x7febbc153880 2024-09-13T18:06:04.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febc3191700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc158d50 con 0x7febbc15cf60 2024-09-13T18:06:04.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febc3191700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febbc159520 con 0x7febbc0cb2f0 2024-09-13T18:06:04.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febc198e700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febbc153880 msgr2=0x7febbc0cdda0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:06:04.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febc198e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febbc153880 0x7febbc0cdda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:04.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febc218f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7febbc0cb2f0 0x7febbc158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:04.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febc2990700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febbc15cf60 0x7febbc15b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:04.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febc218f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7febbc0cb2f0 0x7febbc158810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:44084/0 (socket says 172.21.15.40:44084) 2024-09-13T18:06:04.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febc218f700 1 -- 172.21.15.40:0/86020719 learned_addr learned my addr 172.21.15.40:0/86020719 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:04.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febc218f700 1 -- 172.21.15.40:0/86020719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febbc15cf60 msgr2=0x7febbc15b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:04.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febc218f700 1 --2- 172.21.15.40:0/86020719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febbc15cf60 0x7febbc15b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:04.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febc218f700 1 -- 172.21.15.40:0/86020719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febbc153880 msgr2=0x7febbc0cdda0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:06:04.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febc218f700 1 --2- 172.21.15.40:0/86020719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febbc153880 0x7febbc0cdda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:04.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febc218f700 1 -- 172.21.15.40:0/86020719 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb4009a70 con 0x7febbc0cb2f0 2024-09-13T18:06:04.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febc218f700 1 --2- 172.21.15.40:0/86020719 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7febbc0cb2f0 0x7febbc158810 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7febb800b780 tx=0x7febb800ba90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:04.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febb2ffd700 1 -- 172.21.15.40:0/86020719 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb8003e70 con 0x7febbc0cb2f0 2024-09-13T18:06:04.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febb2ffd700 1 -- 172.21.15.40:0/86020719 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb8005070 con 0x7febbc0cb2f0 2024-09-13T18:06:04.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febc3191700 1 -- 172.21.15.40:0/86020719 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febbc0ce2e0 con 0x7febbc0cb2f0 2024-09-13T18:06:04.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febb2ffd700 1 -- 172.21.15.40:0/86020719 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb8005440 con 0x7febbc0cb2f0 2024-09-13T18:06:04.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.340+0000 7febc3191700 1 -- 172.21.15.40:0/86020719 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febbc159240 con 0x7febbc0cb2f0 2024-09-13T18:06:04.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.344+0000 7febc3191700 1 -- 172.21.15.40:0/86020719 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febbc15d3d0 con 0x7febbc0cb2f0 2024-09-13T18:06:04.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.344+0000 7febb2ffd700 1 -- 172.21.15.40:0/86020719 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7febb80055a0 con 0x7febbc0cb2f0 2024-09-13T18:06:04.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.344+0000 7febb2ffd700 1 --2- 172.21.15.40:0/86020719 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7feba00666e0 0x7feba0068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:04.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.344+0000 7febb2ffd700 1 -- 172.21.15.40:0/86020719 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7febb8083fa0 con 0x7febbc0cb2f0 2024-09-13T18:06:04.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.344+0000 7febc198e700 1 --2- 172.21.15.40:0/86020719 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7feba00666e0 0x7feba0068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:04.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.344+0000 7febc198e700 1 --2- 172.21.15.40:0/86020719 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7feba00666e0 0x7feba0068ba0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7febac005930 tx=0x7febac007b50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:04.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.352+0000 7febb2ffd700 1 -- 172.21.15.40:0/86020719 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7febb80b5db0 con 0x7febbc0cb2f0 2024-09-13T18:06:04.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.628+0000 7febc3191700 1 -- 172.21.15.40:0/86020719 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7febbc156e50 con 0x7febbc0cb2f0 2024-09-13T18:06:04.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.628+0000 7febb2ffd700 1 -- 172.21.15.40:0/86020719 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7febb8050f60 con 0x7febbc0cb2f0 2024-09-13T18:06:04.638 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":124,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:06:04.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.632+0000 7febc3191700 1 -- 172.21.15.40:0/86020719 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7feba00666e0 msgr2=0x7feba0068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:04.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.632+0000 7febc3191700 1 --2- 172.21.15.40:0/86020719 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7feba00666e0 0x7feba0068ba0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7febac005930 tx=0x7febac007b50 comp rx=0 tx=0).stop 2024-09-13T18:06:04.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.632+0000 7febc3191700 1 -- 172.21.15.40:0/86020719 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7febbc0cb2f0 msgr2=0x7febbc158810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:04.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.632+0000 7febc3191700 1 --2- 172.21.15.40:0/86020719 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7febbc0cb2f0 0x7febbc158810 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7febb800b780 tx=0x7febb800ba90 comp rx=0 tx=0).stop 2024-09-13T18:06:04.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.632+0000 7febc3191700 1 -- 172.21.15.40:0/86020719 shutdown_connections 2024-09-13T18:06:04.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.632+0000 7febc3191700 1 --2- 172.21.15.40:0/86020719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febbc15cf60 0x7febbc15b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:04.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.632+0000 7febc3191700 1 --2- 172.21.15.40:0/86020719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febbc153880 0x7febbc0cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:04.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.632+0000 7febc3191700 1 --2- 172.21.15.40:0/86020719 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7feba00666e0 0x7feba0068ba0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:04.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.632+0000 7febc3191700 1 --2- 172.21.15.40:0/86020719 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7febbc0cb2f0 0x7febbc158810 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:04.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.632+0000 7febc3191700 1 -- 172.21.15.40:0/86020719 >> 172.21.15.40:0/86020719 conn(0x7febbc06c5b0 msgr2=0x7febbc066090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:04.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.632+0000 7febc3191700 1 -- 172.21.15.40:0/86020719 shutdown_connections 2024-09-13T18:06:04.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:04.632+0000 7febc3191700 1 -- 172.21.15.40:0/86020719 wait complete. 2024-09-13T18:06:06.649 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:06:06.652 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:06:07.659 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:06:07.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.776+0000 7f3e79777700 1 Processor -- start 2024-09-13T18:06:07.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.776+0000 7f3e79777700 1 -- start start 2024-09-13T18:06:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.776+0000 7f3e79777700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3e740cb2f0 0x7f3e740cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.776+0000 7f3e79777700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e740cf3c0 0x7f3e7415cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.776+0000 7f3e79777700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e74159770 con 0x7f3e74153880 2024-09-13T18:06:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.776+0000 7f3e79777700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e74159a50 con 0x7f3e740cb2f0 2024-09-13T18:06:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.776+0000 7f3e79777700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e74159d30 con 0x7f3e740cf3c0 2024-09-13T18:06:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.776+0000 7f3e737fe700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f3e740cb2f0 msgr2=0x7f3e740cee80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:06:07.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.776+0000 7f3e73fff700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f3e74153880 legacy=0x7f3e740c9ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:06:07.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.776+0000 7f3e737fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3e740cb2f0 0x7f3e740cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:07.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.776+0000 7f3e78f76700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e740cf3c0 0x7f3e7415cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:07.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.776+0000 7f3e78f76700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e740cf3c0 0x7f3e7415cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:44090/0 (socket says 172.21.15.40:44090) 2024-09-13T18:06:07.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.776+0000 7f3e78f76700 1 -- 172.21.15.40:0/966033383 learned_addr learned my addr 172.21.15.40:0/966033383 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:07.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.780+0000 7f3e78f76700 1 -- 172.21.15.40:0/966033383 >> v2:172.21.15.40:3300/0 conn(0x7f3e740cb2f0 msgr2=0x7f3e740cee80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:06:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.780+0000 7f3e78f76700 1 --2- 172.21.15.40:0/966033383 >> v2:172.21.15.40:3300/0 conn(0x7f3e740cb2f0 0x7f3e740cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.780+0000 7f3e78f76700 1 -- 172.21.15.40:0/966033383 >> v1:172.21.15.40:6789/0 conn(0x7f3e74153880 legacy=0x7f3e740c9ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:06:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.780+0000 7f3e78f76700 1 -- 172.21.15.40:0/966033383 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7415a010 con 0x7f3e740cf3c0 2024-09-13T18:06:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.780+0000 7f3e78f76700 1 --2- 172.21.15.40:0/966033383 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e740cf3c0 0x7f3e7415cf60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009e50 tx=0x7f3e6c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a7d604390811d4e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.780+0000 7f3e72ffd700 1 -- 172.21.15.40:0/966033383 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c00e070 con 0x7f3e740cf3c0 2024-09-13T18:06:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.780+0000 7f3e72ffd700 1 -- 172.21.15.40:0/966033383 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c003a40 con 0x7f3e740cf3c0 2024-09-13T18:06:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.780+0000 7f3e72ffd700 1 -- 172.21.15.40:0/966033383 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c003df0 con 0x7f3e740cf3c0 2024-09-13T18:06:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.780+0000 7f3e79777700 1 -- 172.21.15.40:0/966033383 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e740cf3c0 msgr2=0x7f3e7415cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.780+0000 7f3e79777700 1 --2- 172.21.15.40:0/966033383 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e740cf3c0 0x7f3e7415cf60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009e50 tx=0x7f3e6c00b330 comp rx=0 tx=0).stop 2024-09-13T18:06:07.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.780+0000 7f3e79777700 1 -- 172.21.15.40:0/966033383 shutdown_connections 2024-09-13T18:06:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.780+0000 7f3e79777700 1 --2- 172.21.15.40:0/966033383 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e740cf3c0 0x7f3e7415cf60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.780+0000 7f3e79777700 1 --2- 172.21.15.40:0/966033383 >> v2:172.21.15.40:3300/0 conn(0x7f3e740cb2f0 0x7f3e740cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.780+0000 7f3e79777700 1 -- 172.21.15.40:0/966033383 >> 172.21.15.40:0/966033383 conn(0x7f3e7406c5b0 msgr2=0x7f3e7406c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.780+0000 7f3e79777700 1 -- 172.21.15.40:0/966033383 shutdown_connections 2024-09-13T18:06:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.780+0000 7f3e79777700 1 -- 172.21.15.40:0/966033383 wait complete. 2024-09-13T18:06:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.780+0000 7f3e79777700 1 Processor -- start 2024-09-13T18:06:07.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.780+0000 7f3e79777700 1 -- start start 2024-09-13T18:06:07.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e79777700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e740cb2f0 0x7f3e7415c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:07.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e79777700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e740cf3c0 0x7f3e74153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:07.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e79777700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e74153880 0x7f3e7415acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:07.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e79777700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7415a860 con 0x7f3e740cf3c0 2024-09-13T18:06:07.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e79777700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e741590b0 con 0x7f3e740cb2f0 2024-09-13T18:06:07.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e79777700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e74158530 con 0x7f3e74153880 2024-09-13T18:06:07.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e737fe700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e740cf3c0 msgr2=0x7f3e74153fd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:06:07.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e737fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e740cf3c0 0x7f3e74153fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:07.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e73fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e740cb2f0 0x7f3e7415c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:07.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e73fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e740cb2f0 0x7f3e7415c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:39374/0 (socket says 172.21.15.40:39374) 2024-09-13T18:06:07.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e78f76700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e74153880 0x7f3e7415acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:07.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e73fff700 1 -- 172.21.15.40:0/2890635341 learned_addr learned my addr 172.21.15.40:0/2890635341 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:07.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e73fff700 1 -- 172.21.15.40:0/2890635341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e74153880 msgr2=0x7f3e7415acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:07.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e73fff700 1 --2- 172.21.15.40:0/2890635341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e74153880 0x7f3e7415acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:07.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e73fff700 1 -- 172.21.15.40:0/2890635341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e740cf3c0 msgr2=0x7f3e74153fd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:06:07.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e73fff700 1 --2- 172.21.15.40:0/2890635341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e740cf3c0 0x7f3e74153fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:07.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e73fff700 1 -- 172.21.15.40:0/2890635341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c009b30 con 0x7f3e740cb2f0 2024-09-13T18:06:07.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e78f76700 1 --2- 172.21.15.40:0/2890635341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e74153880 0x7f3e7415acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:06:07.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e73fff700 1 --2- 172.21.15.40:0/2890635341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e740cb2f0 0x7f3e7415c530 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400b780 tx=0x7f3e6400ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:07.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e70ff9700 1 -- 172.21.15.40:0/2890635341 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64003e70 con 0x7f3e740cb2f0 2024-09-13T18:06:07.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e70ff9700 1 -- 172.21.15.40:0/2890635341 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e64005070 con 0x7f3e740cb2f0 2024-09-13T18:06:07.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e79777700 1 -- 172.21.15.40:0/2890635341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e74158810 con 0x7f3e740cb2f0 2024-09-13T18:06:07.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e70ff9700 1 -- 172.21.15.40:0/2890635341 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64005420 con 0x7f3e740cb2f0 2024-09-13T18:06:07.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.784+0000 7f3e79777700 1 -- 172.21.15.40:0/2890635341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e74158dd0 con 0x7f3e740cb2f0 2024-09-13T18:06:07.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.788+0000 7f3e79777700 1 -- 172.21.15.40:0/2890635341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e74066f70 con 0x7f3e740cb2f0 2024-09-13T18:06:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.788+0000 7f3e70ff9700 1 -- 172.21.15.40:0/2890635341 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3e64005580 con 0x7f3e740cb2f0 2024-09-13T18:06:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.788+0000 7f3e70ff9700 1 --2- 172.21.15.40:0/2890635341 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3e600666e0 0x7f3e60068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.788+0000 7f3e70ff9700 1 -- 172.21.15.40:0/2890635341 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3e64084980 con 0x7f3e740cb2f0 2024-09-13T18:06:07.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.788+0000 7f3e737fe700 1 --2- 172.21.15.40:0/2890635341 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3e600666e0 0x7f3e60068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:07.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.788+0000 7f3e737fe700 1 --2- 172.21.15.40:0/2890635341 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3e600666e0 0x7f3e60068ba0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3e74151eb0 tx=0x7f3e68009270 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:07.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:07.800+0000 7f3e70ff9700 1 -- 172.21.15.40:0/2890635341 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3e640519c0 con 0x7f3e740cb2f0 2024-09-13T18:06:08.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:08.052+0000 7f3e79777700 1 -- 172.21.15.40:0/2890635341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3e7415d8b0 con 0x7f3e740cb2f0 2024-09-13T18:06:08.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:08.052+0000 7f3e70ff9700 1 -- 172.21.15.40:0/2890635341 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f3e64005830 con 0x7f3e740cb2f0 2024-09-13T18:06:08.061 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":124,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:06:08.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:08.056+0000 7f3e79777700 1 -- 172.21.15.40:0/2890635341 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3e600666e0 msgr2=0x7f3e60068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:08.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:08.056+0000 7f3e79777700 1 --2- 172.21.15.40:0/2890635341 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3e600666e0 0x7f3e60068ba0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3e74151eb0 tx=0x7f3e68009270 comp rx=0 tx=0).stop 2024-09-13T18:06:08.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:08.056+0000 7f3e79777700 1 -- 172.21.15.40:0/2890635341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e740cb2f0 msgr2=0x7f3e7415c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:08.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:08.056+0000 7f3e79777700 1 --2- 172.21.15.40:0/2890635341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e740cb2f0 0x7f3e7415c530 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400b780 tx=0x7f3e6400ba90 comp rx=0 tx=0).stop 2024-09-13T18:06:08.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:08.056+0000 7f3e79777700 1 -- 172.21.15.40:0/2890635341 shutdown_connections 2024-09-13T18:06:08.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:08.056+0000 7f3e79777700 1 --2- 172.21.15.40:0/2890635341 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3e600666e0 0x7f3e60068ba0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:08.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:08.056+0000 7f3e79777700 1 --2- 172.21.15.40:0/2890635341 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3e74153880 0x7f3e7415acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:08.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:08.056+0000 7f3e79777700 1 --2- 172.21.15.40:0/2890635341 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3e740cf3c0 0x7f3e74153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:08.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:08.056+0000 7f3e79777700 1 --2- 172.21.15.40:0/2890635341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e740cb2f0 0x7f3e7415c530 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:08.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:08.056+0000 7f3e79777700 1 -- 172.21.15.40:0/2890635341 >> 172.21.15.40:0/2890635341 conn(0x7f3e7406c5b0 msgr2=0x7f3e74066090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:08.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:08.056+0000 7f3e79777700 1 -- 172.21.15.40:0/2890635341 shutdown_connections 2024-09-13T18:06:08.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:08.056+0000 7f3e79777700 1 -- 172.21.15.40:0/2890635341 wait complete. 2024-09-13T18:06:11.083 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:06:11.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.188+0000 7fc6fc2d3700 1 Processor -- start 2024-09-13T18:06:11.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.188+0000 7fc6fc2d3700 1 -- start start 2024-09-13T18:06:11.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.188+0000 7fc6fc2d3700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc6f40c4dd0 0x7fc6f4159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:11.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.188+0000 7fc6fc2d3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6f40c41b0 0x7fc6f40c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:11.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.188+0000 7fc6fc2d3700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f4159a30 con 0x7fc6f40c4700 2024-09-13T18:06:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.188+0000 7fc6fc2d3700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f406ae40 con 0x7fc6f40c4dd0 2024-09-13T18:06:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.188+0000 7fc6fc2d3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f40cb500 con 0x7fc6f40c41b0 2024-09-13T18:06:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6f986e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6f40c41b0 0x7fc6f40c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6fa06f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc6f40c4dd0 0x7fc6f4159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6f986e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6f40c41b0 0x7fc6f40c51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:44102/0 (socket says 172.21.15.40:44102) 2024-09-13T18:06:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6f986e700 1 -- 172.21.15.40:0/3764512900 learned_addr learned my addr 172.21.15.40:0/3764512900 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6fa870700 1 --1- 172.21.15.40:0/3764512900 >> v1:172.21.15.43:6789/0 conn(0x7fc6f40c4700 0x7fc6f415cba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:46388/0 (socket says 172.21.15.40:46388) 2024-09-13T18:06:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6f986e700 1 -- 172.21.15.40:0/3764512900 >> v2:172.21.15.43:3300/0 conn(0x7fc6f40c4dd0 msgr2=0x7fc6f4159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:06:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6f986e700 1 --2- 172.21.15.40:0/3764512900 >> v2:172.21.15.43:3300/0 conn(0x7fc6f40c4dd0 0x7fc6f4159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6f986e700 1 -- 172.21.15.40:0/3764512900 >> v1:172.21.15.43:6789/0 conn(0x7fc6f40c4700 legacy=0x7fc6f415cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6f986e700 1 -- 172.21.15.40:0/3764512900 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f40cb7e0 con 0x7fc6f40c41b0 2024-09-13T18:06:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6fa06f700 1 --2- 172.21.15.40:0/3764512900 >> v2:172.21.15.43:3300/0 conn(0x7fc6f40c4dd0 0x7fc6f4159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:06:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6f986e700 1 --2- 172.21.15.40:0/3764512900 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6f40c41b0 0x7fc6f40c51b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009910 tx=0x7fc6e4009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29ad620fc550cdd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6f906d700 1 -- 172.21.15.40:0/3764512900 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4011830 con 0x7fc6f40c41b0 2024-09-13T18:06:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6f906d700 1 -- 172.21.15.40:0/3764512900 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e4011990 con 0x7fc6f40c41b0 2024-09-13T18:06:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6f906d700 1 -- 172.21.15.40:0/3764512900 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4011d40 con 0x7fc6f40c41b0 2024-09-13T18:06:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6fc2d3700 1 -- 172.21.15.40:0/3764512900 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6f40c41b0 msgr2=0x7fc6f40c51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6fc2d3700 1 --2- 172.21.15.40:0/3764512900 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6f40c41b0 0x7fc6f40c51b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009910 tx=0x7fc6e4009c20 comp rx=0 tx=0).stop 2024-09-13T18:06:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6fc2d3700 1 -- 172.21.15.40:0/3764512900 shutdown_connections 2024-09-13T18:06:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6fc2d3700 1 --2- 172.21.15.40:0/3764512900 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6f40c41b0 0x7fc6f40c51b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6fc2d3700 1 --2- 172.21.15.40:0/3764512900 >> v2:172.21.15.43:3300/0 conn(0x7fc6f40c4dd0 0x7fc6f4159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6fc2d3700 1 -- 172.21.15.40:0/3764512900 >> 172.21.15.40:0/3764512900 conn(0x7fc6f40cefd0 msgr2=0x7fc6f40cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6fc2d3700 1 -- 172.21.15.40:0/3764512900 shutdown_connections 2024-09-13T18:06:11.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.192+0000 7fc6fc2d3700 1 -- 172.21.15.40:0/3764512900 wait complete. 2024-09-13T18:06:11.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6fc2d3700 1 Processor -- start 2024-09-13T18:06:11.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6fc2d3700 1 -- start start 2024-09-13T18:06:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6fc2d3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f40c41b0 0x7fc6f406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6fc2d3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6f40c4700 0x7fc6f41e40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6fc2d3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f40c4dd0 0x7fc6f41e4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6fc2d3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f40b4a10 con 0x7fc6f40c4dd0 2024-09-13T18:06:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6fc2d3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f41590e0 con 0x7fc6f40c41b0 2024-09-13T18:06:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6fc2d3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f40b5310 con 0x7fc6f40c4700 2024-09-13T18:06:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6fa870700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f40c4dd0 msgr2=0x7fc6f41e4620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:06:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6fa870700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f40c4dd0 0x7fc6f41e4620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6fa06f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f40c41b0 0x7fc6f406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:11.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6fa06f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f40c41b0 0x7fc6f406c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:39390/0 (socket says 172.21.15.40:39390) 2024-09-13T18:06:11.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6f986e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6f40c4700 0x7fc6f41e40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:11.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6fa06f700 1 -- 172.21.15.40:0/2958758018 learned_addr learned my addr 172.21.15.40:0/2958758018 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:11.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6fa06f700 1 -- 172.21.15.40:0/2958758018 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6f40c4700 msgr2=0x7fc6f41e40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:11.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6fa06f700 1 --2- 172.21.15.40:0/2958758018 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6f40c4700 0x7fc6f41e40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:11.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6fa06f700 1 -- 172.21.15.40:0/2958758018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f40c4dd0 msgr2=0x7fc6f41e4620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:06:11.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6fa06f700 1 --2- 172.21.15.40:0/2958758018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f40c4dd0 0x7fc6f41e4620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:11.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6fa06f700 1 -- 172.21.15.40:0/2958758018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f00094b0 con 0x7fc6f40c41b0 2024-09-13T18:06:11.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6f986e700 1 --2- 172.21.15.40:0/2958758018 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6f40c4700 0x7fc6f41e40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:06:11.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6fa06f700 1 --2- 172.21.15.40:0/2958758018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f40c41b0 0x7fc6f406c920 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000ebe0 tx=0x7fc6f000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:11.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6eaffd700 1 -- 172.21.15.40:0/2958758018 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f000c850 con 0x7fc6f40c41b0 2024-09-13T18:06:11.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6eaffd700 1 -- 172.21.15.40:0/2958758018 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f000c9b0 con 0x7fc6f40c41b0 2024-09-13T18:06:11.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6eaffd700 1 -- 172.21.15.40:0/2958758018 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f000cd40 con 0x7fc6f40c41b0 2024-09-13T18:06:11.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.196+0000 7fc6fc2d3700 1 -- 172.21.15.40:0/2958758018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e4009570 con 0x7fc6f40c41b0 2024-09-13T18:06:11.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.200+0000 7fc6fc2d3700 1 -- 172.21.15.40:0/2958758018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f40b4730 con 0x7fc6f40c41b0 2024-09-13T18:06:11.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.200+0000 7fc6fc2d3700 1 -- 172.21.15.40:0/2958758018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f4153b70 con 0x7fc6f40c41b0 2024-09-13T18:06:11.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.200+0000 7fc6eaffd700 1 -- 172.21.15.40:0/2958758018 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc6f0013470 con 0x7fc6f40c41b0 2024-09-13T18:06:11.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.200+0000 7fc6eaffd700 1 --2- 172.21.15.40:0/2958758018 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc6d80666e0 0x7fc6d8068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:11.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.200+0000 7fc6eaffd700 1 -- 172.21.15.40:0/2958758018 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc6f0084980 con 0x7fc6f40c41b0 2024-09-13T18:06:11.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.200+0000 7fc6f986e700 1 --2- 172.21.15.40:0/2958758018 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc6d80666e0 0x7fc6d8068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:11.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.204+0000 7fc6f986e700 1 --2- 172.21.15.40:0/2958758018 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc6d80666e0 0x7fc6d8068ba0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009910 tx=0x7fc6e4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:11.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.216+0000 7fc6eaffd700 1 -- 172.21.15.40:0/2958758018 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc6f00518b0 con 0x7fc6f40c41b0 2024-09-13T18:06:11.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.472+0000 7fc6fc2d3700 1 -- 172.21.15.40:0/2958758018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc6f40b3ad0 con 0x7fc6f40c41b0 2024-09-13T18:06:11.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.472+0000 7fc6eaffd700 1 -- 172.21.15.40:0/2958758018 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7fc6f000cea0 con 0x7fc6f40c41b0 2024-09-13T18:06:11.481 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":124,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:06:11.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.476+0000 7fc6fc2d3700 1 -- 172.21.15.40:0/2958758018 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc6d80666e0 msgr2=0x7fc6d8068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:11.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.476+0000 7fc6fc2d3700 1 --2- 172.21.15.40:0/2958758018 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc6d80666e0 0x7fc6d8068ba0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009910 tx=0x7fc6e4008030 comp rx=0 tx=0).stop 2024-09-13T18:06:11.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.476+0000 7fc6fc2d3700 1 -- 172.21.15.40:0/2958758018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f40c41b0 msgr2=0x7fc6f406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:11.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.476+0000 7fc6fc2d3700 1 --2- 172.21.15.40:0/2958758018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f40c41b0 0x7fc6f406c920 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000ebe0 tx=0x7fc6f000eef0 comp rx=0 tx=0).stop 2024-09-13T18:06:11.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.476+0000 7fc6fc2d3700 1 -- 172.21.15.40:0/2958758018 shutdown_connections 2024-09-13T18:06:11.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.476+0000 7fc6fc2d3700 1 --2- 172.21.15.40:0/2958758018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f40c4dd0 0x7fc6f41e4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:11.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.476+0000 7fc6fc2d3700 1 --2- 172.21.15.40:0/2958758018 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc6d80666e0 0x7fc6d8068ba0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:11.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.476+0000 7fc6fc2d3700 1 --2- 172.21.15.40:0/2958758018 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6f40c4700 0x7fc6f41e40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:11.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.476+0000 7fc6fc2d3700 1 --2- 172.21.15.40:0/2958758018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f40c41b0 0x7fc6f406c920 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:11.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.476+0000 7fc6fc2d3700 1 -- 172.21.15.40:0/2958758018 >> 172.21.15.40:0/2958758018 conn(0x7fc6f40cefd0 msgr2=0x7fc6f4151390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:11.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.476+0000 7fc6fc2d3700 1 -- 172.21.15.40:0/2958758018 shutdown_connections 2024-09-13T18:06:11.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:11.476+0000 7fc6fc2d3700 1 -- 172.21.15.40:0/2958758018 wait complete. 2024-09-13T18:06:14.503 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:06:14.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.612+0000 7fc8fbfff700 1 Processor -- start 2024-09-13T18:06:14.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.612+0000 7fc8fbfff700 1 -- start start 2024-09-13T18:06:14.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.612+0000 7fc8fbfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 0x7fc8fc0c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:14.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.612+0000 7fc8fbfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc8fc070500 0x7fc8fc0c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:14.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.612+0000 7fc8fbfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 0x7fc8fc071f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:14.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.612+0000 7fc8fbfff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8fc0cba20 con 0x7fc8fc070500 2024-09-13T18:06:14.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.612+0000 7fc8fbfff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8fc1560e0 con 0x7fc8fc1500c0 2024-09-13T18:06:14.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.612+0000 7fc8fbfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc1563c0 con 0x7fc8fc071b40 2024-09-13T18:06:14.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.612+0000 7fc8fa7fc700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7fc8fc070500 msgr2=0x7fc8fc0c6c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:06:14.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.612+0000 7fc8fa7fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc8fc070500 0x7fc8fc0c6c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:14.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.612+0000 7fc8fb7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 0x7fc8fc071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:14.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.612+0000 7fc8faffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 0x7fc8fc0c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:14.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.612+0000 7fc8fb7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 0x7fc8fc071f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:43196/0 (socket says 172.21.15.40:43196) 2024-09-13T18:06:14.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.612+0000 7fc8fb7fe700 1 -- 172.21.15.40:0/1120988657 learned_addr learned my addr 172.21.15.40:0/1120988657 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:14.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.616+0000 7fc8fb7fe700 1 -- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 msgr2=0x7fc8fc071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:06:14.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.616+0000 7fc8fb7fe700 1 -- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 msgr2=0x7fc8fc071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:14.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.616+0000 7fc8fb7fe700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 0x7fc8fc071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:14.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.616+0000 7fc8fb7fe700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 0x7fc8fc071f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:14.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.616+0000 7fc8faffd700 1 -- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 msgr2=0x7fc8fc0c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:06:14.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.616+0000 7fc8faffd700 1 -- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 msgr2=0x7fc8fc0c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:14.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.616+0000 7fc8faffd700 1 --2- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 0x7fc8fc0c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:14.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.616+0000 7fc8faffd700 1 --2- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 0x7fc8fc0c66d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:14.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.816+0000 7fc8fa7fc700 1 -- 172.21.15.40:0/1120988657 >> v2:172.21.15.40:3300/0 conn(0x7fc8fc070500 msgr2=0x7fc8fc0c6c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:06:14.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.816+0000 7fc8fa7fc700 1 --2- 172.21.15.40:0/1120988657 >> v2:172.21.15.40:3300/0 conn(0x7fc8fc070500 0x7fc8fc0c6c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:06:14.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.820+0000 7fc8fb7fe700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 0x7fc8fc071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:14.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.820+0000 7fc8faffd700 1 --2- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 0x7fc8fc0c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:14.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.820+0000 7fc8fb7fe700 1 -- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 msgr2=0x7fc8fc071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:06:14.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.820+0000 7fc8fb7fe700 1 -- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 msgr2=0x7fc8fc071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:14.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.820+0000 7fc8fb7fe700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 0x7fc8fc071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:14.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.820+0000 7fc8faffd700 1 -- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 msgr2=0x7fc8fc0c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:06:14.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.820+0000 7fc8fb7fe700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 0x7fc8fc071f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:06:14.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.820+0000 7fc8faffd700 1 -- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 msgr2=0x7fc8fc0c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:14.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.820+0000 7fc8faffd700 1 --2- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 0x7fc8fc0c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:14.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:14.820+0000 7fc8faffd700 1 --2- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 0x7fc8fc0c66d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:06:15.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:15.220+0000 7fc8fa7fc700 1 -- 172.21.15.40:0/1120988657 >> v2:172.21.15.40:3300/0 conn(0x7fc8fc070500 msgr2=0x7fc8fc0c6c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:06:15.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:15.220+0000 7fc8fa7fc700 1 --2- 172.21.15.40:0/1120988657 >> v2:172.21.15.40:3300/0 conn(0x7fc8fc070500 0x7fc8fc0c6c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:06:15.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:15.224+0000 7fc8fb7fe700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 0x7fc8fc071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:15.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:15.224+0000 7fc8faffd700 1 --2- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 0x7fc8fc0c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:15.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:15.224+0000 7fc8fb7fe700 1 -- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 msgr2=0x7fc8fc071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:06:15.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:15.224+0000 7fc8fb7fe700 1 -- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 msgr2=0x7fc8fc071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:15.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:15.224+0000 7fc8fb7fe700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 0x7fc8fc071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:15.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:15.224+0000 7fc8fb7fe700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 0x7fc8fc071f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:06:15.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:15.224+0000 7fc8faffd700 1 -- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 msgr2=0x7fc8fc0c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:06:15.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:15.224+0000 7fc8faffd700 1 -- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 msgr2=0x7fc8fc0c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:15.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:15.224+0000 7fc8faffd700 1 --2- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 0x7fc8fc0c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:15.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:15.224+0000 7fc8faffd700 1 --2- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 0x7fc8fc0c66d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:06:16.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:16.024+0000 7fc8fa7fc700 1 -- 172.21.15.40:0/1120988657 >> v2:172.21.15.40:3300/0 conn(0x7fc8fc070500 msgr2=0x7fc8fc0c6c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:06:16.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:16.024+0000 7fc8fa7fc700 1 --2- 172.21.15.40:0/1120988657 >> v2:172.21.15.40:3300/0 conn(0x7fc8fc070500 0x7fc8fc0c6c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:06:16.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:16.028+0000 7fc8fb7fe700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 0x7fc8fc071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:16.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:16.028+0000 7fc8faffd700 1 --2- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 0x7fc8fc0c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:16.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:16.028+0000 7fc8fb7fe700 1 -- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 msgr2=0x7fc8fc071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:06:16.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:16.028+0000 7fc8fb7fe700 1 -- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 msgr2=0x7fc8fc071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:16.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:16.028+0000 7fc8fb7fe700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 0x7fc8fc071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:16.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:16.028+0000 7fc8fb7fe700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 0x7fc8fc071f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:06:16.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:16.028+0000 7fc8faffd700 1 -- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 msgr2=0x7fc8fc0c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:06:16.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:16.028+0000 7fc8faffd700 1 -- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 msgr2=0x7fc8fc0c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:16.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:16.028+0000 7fc8faffd700 1 --2- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 0x7fc8fc0c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:16.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:16.028+0000 7fc8faffd700 1 --2- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 0x7fc8fc0c66d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:06:17.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:17.612+0000 7fc8f8ff9700 1 -- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 msgr2=0x7fc8fc071f60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:06:17.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:17.612+0000 7fc8f8ff9700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8fc071b40 0x7fc8fc071f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:17.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:17.612+0000 7fc8f8ff9700 1 -- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 msgr2=0x7fc8fc0c66d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:06:17.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:17.612+0000 7fc8f8ff9700 1 --2- 172.21.15.40:0/1120988657 >> v2:172.21.15.43:3300/0 conn(0x7fc8fc1500c0 0x7fc8fc0c66d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:17.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:17.612+0000 7fc8f8ff9700 1 -- 172.21.15.40:0/1120988657 >> v2:172.21.15.40:3300/0 conn(0x7fc8fc070500 msgr2=0x7fc8fc0c6c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:06:17.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:17.612+0000 7fc8f8ff9700 1 --2- 172.21.15.40:0/1120988657 >> v2:172.21.15.40:3300/0 conn(0x7fc8fc070500 0x7fc8fc0c6c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:17.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:17.612+0000 7fc8f8ff9700 1 -- 172.21.15.40:0/1120988657 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8fc1566a0 con 0x7fc8e8007430 2024-09-13T18:06:17.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:17.612+0000 7fc8f8ff9700 1 -- 172.21.15.40:0/1120988657 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e800c1a0 con 0x7fc8e80036f0 2024-09-13T18:06:17.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:17.612+0000 7fc8fa7fc700 1 -- 172.21.15.40:0/1120988657 >> v1:172.21.15.40:6789/0 conn(0x7fc8e8007430 legacy=0x7fc8e80098f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:06:17.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:17.816+0000 7fc8fa7fc700 1 -- 172.21.15.40:0/1120988657 >> v1:172.21.15.40:6789/0 conn(0x7fc8e8007430 legacy=0x7fc8e80098f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:06:18.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.220+0000 7fc8fa7fc700 1 -- 172.21.15.40:0/1120988657 >> v1:172.21.15.40:6789/0 conn(0x7fc8e8007430 legacy=0x7fc8e80098f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:06:18.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.684+0000 7fc8f9ffb700 1 -- 172.21.15.40:0/1120988657 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 364196592 0 0) 0x7fc8e800c1a0 con 0x7fc8e80036f0 2024-09-13T18:06:18.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.684+0000 7fc8f9ffb700 1 -- 172.21.15.40:0/1120988657 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8e40035d0 con 0x7fc8e80036f0 2024-09-13T18:06:18.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.688+0000 7fc8f9ffb700 1 -- 172.21.15.40:0/1120988657 <== mon.1 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3808963782 0 0) 0x7fc8e40035d0 con 0x7fc8e80036f0 2024-09-13T18:06:18.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.688+0000 7fc8f9ffb700 1 -- 172.21.15.40:0/1120988657 >> v1:172.21.15.40:6789/0 conn(0x7fc8e8007430 legacy=0x7fc8e80098f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:06:18.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.688+0000 7fc8f9ffb700 1 -- 172.21.15.40:0/1120988657 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e800d3a0 con 0x7fc8e80036f0 2024-09-13T18:06:18.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.688+0000 7fc8f9ffb700 1 -- 172.21.15.40:0/1120988657 <== mon.1 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fc8ec00c4f0 con 0x7fc8e80036f0 2024-09-13T18:06:18.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.688+0000 7fc8f9ffb700 1 -- 172.21.15.40:0/1120988657 >> v1:172.21.15.43:6789/0 conn(0x7fc8e80036f0 legacy=0x7fc8e8005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:18.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.688+0000 7fc8fb7fe700 1 -- 172.21.15.40:0/1120988657 reap_dead start 2024-09-13T18:06:18.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.688+0000 7fc8f9ffb700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e4003a20 0x7fc8e4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:18.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.688+0000 7fc8f9ffb700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8e4004870 0x7fc8e4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.688+0000 7fc8f9ffb700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e4005350 0x7fc8e400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.688+0000 7fc8faffd700 1 -- 172.21.15.40:0/1120988657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e4003a20 msgr2=0x7fc8e4003e00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:06:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.688+0000 7fc8f9ffb700 1 -- 172.21.15.40:0/1120988657 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e400c240 con 0x7fc8e4003a20 2024-09-13T18:06:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.688+0000 7fc8faffd700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e4003a20 0x7fc8e4003e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.688+0000 7fc8f9ffb700 1 -- 172.21.15.40:0/1120988657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e400c3b0 con 0x7fc8e4005350 2024-09-13T18:06:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.688+0000 7fc8f9ffb700 1 -- 172.21.15.40:0/1120988657 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e400c570 con 0x7fc8e4004870 2024-09-13T18:06:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.688+0000 7fc8fb7fe700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e4005350 0x7fc8e400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:18.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.688+0000 7fc8fa7fc700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8e4004870 0x7fc8e4004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:18.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.692+0000 7fc8fb7fe700 1 -- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8e4004870 msgr2=0x7fc8e4004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:06:18.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.692+0000 7fc8fb7fe700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8e4004870 0x7fc8e4004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:18.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.692+0000 7fc8fb7fe700 1 -- 172.21.15.40:0/1120988657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e4003a20 msgr2=0x7fc8e4003e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:06:18.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.692+0000 7fc8fb7fe700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e4003a20 0x7fc8e4003e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:18.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.692+0000 7fc8fb7fe700 1 -- 172.21.15.40:0/1120988657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e40035d0 con 0x7fc8e4005350 2024-09-13T18:06:18.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.692+0000 7fc8fb7fe700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e4005350 0x7fc8e400ba30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc06b980 tx=0x7fc8e40058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b82de1593796ba7c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.692+0000 7fc8f9ffb700 1 -- 172.21.15.40:0/1120988657 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4015be0 con 0x7fc8e4005350 2024-09-13T18:06:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.692+0000 7fc8f9ffb700 1 -- 172.21.15.40:0/1120988657 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f4015d40 con 0x7fc8e4005350 2024-09-13T18:06:18.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.692+0000 7fc8f9ffb700 1 -- 172.21.15.40:0/1120988657 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f401e680 con 0x7fc8e4005350 2024-09-13T18:06:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.696+0000 7fc8fbfff700 1 -- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e4005350 msgr2=0x7fc8e400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.696+0000 7fc8fbfff700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e4005350 0x7fc8e400ba30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc06b980 tx=0x7fc8e40058a0 comp rx=0 tx=0).stop 2024-09-13T18:06:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.696+0000 7fc8fbfff700 1 -- 172.21.15.40:0/1120988657 shutdown_connections 2024-09-13T18:06:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.696+0000 7fc8fbfff700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e4005350 0x7fc8e400ba30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.696+0000 7fc8fbfff700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8e4004870 0x7fc8e4004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.696+0000 7fc8fbfff700 1 --2- 172.21.15.40:0/1120988657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e4003a20 0x7fc8e4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.696+0000 7fc8fbfff700 1 -- 172.21.15.40:0/1120988657 >> 172.21.15.40:0/1120988657 conn(0x7fc8fc0cabc0 msgr2=0x7fc8fc0cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.696+0000 7fc8fbfff700 1 -- 172.21.15.40:0/1120988657 shutdown_connections 2024-09-13T18:06:18.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.696+0000 7fc8fbfff700 1 -- 172.21.15.40:0/1120988657 wait complete. 2024-09-13T18:06:18.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.696+0000 7fc8fbfff700 1 Processor -- start 2024-09-13T18:06:18.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.696+0000 7fc8fbfff700 1 -- start start 2024-09-13T18:06:18.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8fbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e4003a20 0x7fc8fc14baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:18.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8fbfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8e4004870 0x7fc8fc14bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8fbfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e4005350 0x7fc8fc1eab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8fbfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc1ede10 con 0x7fc8e4003a20 2024-09-13T18:06:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8fbfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc1eec90 con 0x7fc8e4005350 2024-09-13T18:06:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8fbfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc1ee080 con 0x7fc8e4004870 2024-09-13T18:06:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8faffd700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e4003a20 msgr2=0x7fc8fc14baa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:06:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8faffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e4003a20 0x7fc8fc14baa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8fb7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e4005350 0x7fc8fc1eab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8fa7fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8e4004870 0x7fc8fc14bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8fb7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e4005350 0x7fc8fc1eab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:47646/0 (socket says 172.21.15.40:47646) 2024-09-13T18:06:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8fb7fe700 1 -- 172.21.15.40:0/2324965920 learned_addr learned my addr 172.21.15.40:0/2324965920 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8fb7fe700 1 -- 172.21.15.40:0/2324965920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8e4004870 msgr2=0x7fc8fc14bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8fb7fe700 1 --2- 172.21.15.40:0/2324965920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8e4004870 0x7fc8fc14bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8fb7fe700 1 -- 172.21.15.40:0/2324965920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e4003a20 msgr2=0x7fc8fc14baa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:06:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8fb7fe700 1 --2- 172.21.15.40:0/2324965920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e4003a20 0x7fc8fc14baa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8fb7fe700 1 -- 172.21.15.40:0/2324965920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8fc1ee360 con 0x7fc8e4005350 2024-09-13T18:06:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8fb7fe700 1 --2- 172.21.15.40:0/2324965920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e4005350 0x7fc8fc1eab10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8007310 tx=0x7fc8f4004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8e37fe700 1 -- 172.21.15.40:0/2324965920 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4015be0 con 0x7fc8e4005350 2024-09-13T18:06:18.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8e37fe700 1 -- 172.21.15.40:0/2324965920 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f401ee30 con 0x7fc8e4005350 2024-09-13T18:06:18.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8e37fe700 1 -- 172.21.15.40:0/2324965920 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4003c80 con 0x7fc8e4005350 2024-09-13T18:06:18.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8fbfff700 1 -- 172.21.15.40:0/2324965920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8fc1ee640 con 0x7fc8e4005350 2024-09-13T18:06:18.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.700+0000 7fc8fbfff700 1 -- 172.21.15.40:0/2324965920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8fc14a700 con 0x7fc8e4005350 2024-09-13T18:06:18.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.704+0000 7fc8fbfff700 1 -- 172.21.15.40:0/2324965920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8fc06b650 con 0x7fc8e4005350 2024-09-13T18:06:18.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.704+0000 7fc8e37fe700 1 -- 172.21.15.40:0/2324965920 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc8f4004030 con 0x7fc8e4005350 2024-09-13T18:06:18.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.704+0000 7fc8e37fe700 1 --2- 172.21.15.40:0/2324965920 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8e8066f10 0x7fc8e80693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:18.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.704+0000 7fc8e37fe700 1 -- 172.21.15.40:0/2324965920 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc8f4087900 con 0x7fc8e4005350 2024-09-13T18:06:18.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.704+0000 7fc8faffd700 1 --2- 172.21.15.40:0/2324965920 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8e8066f10 0x7fc8e80693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:18.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.704+0000 7fc8faffd700 1 --2- 172.21.15.40:0/2324965920 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8e8066f10 0x7fc8e80693d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec00c620 tx=0x7fc8ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:18.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.712+0000 7fc8e37fe700 1 -- 172.21.15.40:0/2324965920 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc8f4058480 con 0x7fc8e4005350 2024-09-13T18:06:18.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.968+0000 7fc8fbfff700 1 -- 172.21.15.40:0/2324965920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc8fc0cb950 con 0x7fc8e4005350 2024-09-13T18:06:18.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.968+0000 7fc8e37fe700 1 -- 172.21.15.40:0/2324965920 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7fc8f4058010 con 0x7fc8e4005350 2024-09-13T18:06:18.978 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":126,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:06:18.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.972+0000 7fc8fbfff700 1 -- 172.21.15.40:0/2324965920 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8e8066f10 msgr2=0x7fc8e80693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:18.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.972+0000 7fc8fbfff700 1 --2- 172.21.15.40:0/2324965920 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8e8066f10 0x7fc8e80693d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec00c620 tx=0x7fc8ec008040 comp rx=0 tx=0).stop 2024-09-13T18:06:18.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.972+0000 7fc8fbfff700 1 -- 172.21.15.40:0/2324965920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e4005350 msgr2=0x7fc8fc1eab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.972+0000 7fc8fbfff700 1 --2- 172.21.15.40:0/2324965920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e4005350 0x7fc8fc1eab10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8007310 tx=0x7fc8f4004af0 comp rx=0 tx=0).stop 2024-09-13T18:06:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.972+0000 7fc8fbfff700 1 -- 172.21.15.40:0/2324965920 shutdown_connections 2024-09-13T18:06:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.972+0000 7fc8fbfff700 1 --2- 172.21.15.40:0/2324965920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8e4005350 0x7fc8fc1eab10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.972+0000 7fc8fbfff700 1 --2- 172.21.15.40:0/2324965920 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8e8066f10 0x7fc8e80693d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.972+0000 7fc8fbfff700 1 --2- 172.21.15.40:0/2324965920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8e4004870 0x7fc8fc14bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.972+0000 7fc8fbfff700 1 --2- 172.21.15.40:0/2324965920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8e4003a20 0x7fc8fc14baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.972+0000 7fc8fbfff700 1 -- 172.21.15.40:0/2324965920 >> 172.21.15.40:0/2324965920 conn(0x7fc8fc0cabc0 msgr2=0x7fc8fc0c7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.972+0000 7fc8fbfff700 1 -- 172.21.15.40:0/2324965920 shutdown_connections 2024-09-13T18:06:18.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:18.972+0000 7fc8fbfff700 1 -- 172.21.15.40:0/2324965920 wait complete. 2024-09-13T18:06:18.997 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T18:06:18.997 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-13T18:06:18.997 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:06:19.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.096+0000 7fe1533d7700 1 Processor -- start 2024-09-13T18:06:19.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.096+0000 7fe1533d7700 1 -- start start 2024-09-13T18:06:19.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe1533d7700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe14c0b0d70 0x7fe14c157f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:19.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe1533d7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe14c06a950 0x7fe14c06ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:19.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe1533d7700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe14c158440 con 0x7fe14c151750 2024-09-13T18:06:19.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe1533d7700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe14c06a4c0 con 0x7fe14c0b0d70 2024-09-13T18:06:19.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe1533d7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c06b320 con 0x7fe14c06a950 2024-09-13T18:06:19.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe151974700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe14c06a950 0x7fe14c06ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:19.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe150972700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe14c0b0d70 0x7fe14c157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:19.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe151974700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe14c06a950 0x7fe14c06ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:43248/0 (socket says 172.21.15.40:43248) 2024-09-13T18:06:19.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe151173700 1 --1- 172.21.15.40:0/3684646830 >> v1:172.21.15.43:6789/0 conn(0x7fe14c151750 0x7fe14c06fbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:47506/0 (socket says 172.21.15.40:47506) 2024-09-13T18:06:19.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe151974700 1 -- 172.21.15.40:0/3684646830 learned_addr learned my addr 172.21.15.40:0/3684646830 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:19.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe151974700 1 -- 172.21.15.40:0/3684646830 >> v2:172.21.15.43:3300/0 conn(0x7fe14c0b0d70 msgr2=0x7fe14c157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:06:19.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe151974700 1 --2- 172.21.15.40:0/3684646830 >> v2:172.21.15.43:3300/0 conn(0x7fe14c0b0d70 0x7fe14c157f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:19.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe151974700 1 -- 172.21.15.40:0/3684646830 >> v1:172.21.15.43:6789/0 conn(0x7fe14c151750 legacy=0x7fe14c06fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:19.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe151974700 1 -- 172.21.15.40:0/3684646830 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14c06cf60 con 0x7fe14c06a950 2024-09-13T18:06:19.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe150972700 1 --2- 172.21.15.40:0/3684646830 >> v2:172.21.15.43:3300/0 conn(0x7fe14c0b0d70 0x7fe14c157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:06:19.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe151974700 1 --2- 172.21.15.40:0/3684646830 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe14c06a950 0x7fe14c06ade0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009910 tx=0x7fe13c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abd9d7cb2667ff25 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:19.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe143fff700 1 -- 172.21.15.40:0/3684646830 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c011830 con 0x7fe14c06a950 2024-09-13T18:06:19.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe143fff700 1 -- 172.21.15.40:0/3684646830 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c011990 con 0x7fe14c06a950 2024-09-13T18:06:19.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe143fff700 1 -- 172.21.15.40:0/3684646830 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c011d40 con 0x7fe14c06a950 2024-09-13T18:06:19.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe1533d7700 1 -- 172.21.15.40:0/3684646830 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe14c06a950 msgr2=0x7fe14c06ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:19.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe1533d7700 1 --2- 172.21.15.40:0/3684646830 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe14c06a950 0x7fe14c06ade0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009910 tx=0x7fe13c009c20 comp rx=0 tx=0).stop 2024-09-13T18:06:19.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe1533d7700 1 -- 172.21.15.40:0/3684646830 shutdown_connections 2024-09-13T18:06:19.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe1533d7700 1 --2- 172.21.15.40:0/3684646830 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe14c06a950 0x7fe14c06ade0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:19.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe1533d7700 1 --2- 172.21.15.40:0/3684646830 >> v2:172.21.15.43:3300/0 conn(0x7fe14c0b0d70 0x7fe14c157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:19.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe1533d7700 1 -- 172.21.15.40:0/3684646830 >> 172.21.15.40:0/3684646830 conn(0x7fe14c06d5b0 msgr2=0x7fe14c06d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:19.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe1533d7700 1 -- 172.21.15.40:0/3684646830 shutdown_connections 2024-09-13T18:06:19.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.100+0000 7fe1533d7700 1 -- 172.21.15.40:0/3684646830 wait complete. 2024-09-13T18:06:19.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.104+0000 7fe1533d7700 1 Processor -- start 2024-09-13T18:06:19.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.104+0000 7fe1533d7700 1 -- start start 2024-09-13T18:06:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.104+0000 7fe1533d7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe14c06a950 0x7fe14c1550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.104+0000 7fe1533d7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c0b0d70 0x7fe14c150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.104+0000 7fe1533d7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe14c151750 0x7fe14c14cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.104+0000 7fe1533d7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c1571e0 con 0x7fe14c151750 2024-09-13T18:06:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.104+0000 7fe1533d7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c155620 con 0x7fe14c0b0d70 2024-09-13T18:06:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.104+0000 7fe1533d7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c157570 con 0x7fe14c06a950 2024-09-13T18:06:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.104+0000 7fe151974700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe14c151750 msgr2=0x7fe14c14cb40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:06:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.104+0000 7fe151974700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe14c151750 0x7fe14c14cb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.104+0000 7fe150972700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c0b0d70 0x7fe14c150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.104+0000 7fe150972700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c0b0d70 0x7fe14c150420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:47664/0 (socket says 172.21.15.40:47664) 2024-09-13T18:06:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.104+0000 7fe150972700 1 -- 172.21.15.40:0/2292165363 learned_addr learned my addr 172.21.15.40:0/2292165363 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:19.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.104+0000 7fe151173700 1 --2- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe14c06a950 0x7fe14c1550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.108+0000 7fe150972700 1 -- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe14c06a950 msgr2=0x7fe14c1550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.108+0000 7fe150972700 1 --2- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe14c06a950 0x7fe14c1550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.108+0000 7fe150972700 1 -- 172.21.15.40:0/2292165363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe14c151750 msgr2=0x7fe14c14cb40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:06:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.108+0000 7fe150972700 1 --2- 172.21.15.40:0/2292165363 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe14c151750 0x7fe14c14cb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.108+0000 7fe150972700 1 -- 172.21.15.40:0/2292165363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1480094b0 con 0x7fe14c0b0d70 2024-09-13T18:06:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.108+0000 7fe151173700 1 --2- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe14c06a950 0x7fe14c1550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:06:19.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.108+0000 7fe150972700 1 --2- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c0b0d70 0x7fe14c150420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe14800ebe0 tx=0x7fe14800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:19.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.108+0000 7fe141ffb700 1 -- 172.21.15.40:0/2292165363 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14800c850 con 0x7fe14c0b0d70 2024-09-13T18:06:19.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.108+0000 7fe141ffb700 1 -- 172.21.15.40:0/2292165363 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14800c9b0 con 0x7fe14c0b0d70 2024-09-13T18:06:19.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.108+0000 7fe1533d7700 1 -- 172.21.15.40:0/2292165363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13c009570 con 0x7fe14c0b0d70 2024-09-13T18:06:19.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.108+0000 7fe141ffb700 1 -- 172.21.15.40:0/2292165363 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14800cd40 con 0x7fe14c0b0d70 2024-09-13T18:06:19.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.108+0000 7fe1533d7700 1 -- 172.21.15.40:0/2292165363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14c155b70 con 0x7fe14c0b0d70 2024-09-13T18:06:19.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.108+0000 7fe1533d7700 1 --2- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c1548c0 0x7fe14c155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:19.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.108+0000 7fe1533d7700 1 -- 172.21.15.40:0/2292165363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c067840 con 0x7fe14c1548c0 2024-09-13T18:06:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.108+0000 7fe151173700 1 --2- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c1548c0 0x7fe14c155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.108+0000 7fe151173700 1 -- 172.21.15.40:0/2292165363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe14c071240 con 0x7fe14c1548c0 2024-09-13T18:06:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.108+0000 7fe151173700 1 --2- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c1548c0 0x7fe14c155e50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe1440012e0 tx=0x7fe14400fb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.108+0000 7fe141ffb700 1 -- 172.21.15.40:0/2292165363 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe148013470 con 0x7fe14c0b0d70 2024-09-13T18:06:19.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.112+0000 7fe141ffb700 1 --2- 172.21.15.40:0/2292165363 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe1300666b0 0x7fe130068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:19.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.112+0000 7fe141ffb700 1 -- 172.21.15.40:0/2292165363 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe144011580 con 0x7fe14c1548c0 2024-09-13T18:06:19.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.112+0000 7fe141ffb700 1 -- 172.21.15.40:0/2292165363 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe148084840 con 0x7fe14c0b0d70 2024-09-13T18:06:19.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.112+0000 7fe151974700 1 --2- 172.21.15.40:0/2292165363 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe1300666b0 0x7fe130068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:19.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.112+0000 7fe141ffb700 1 -- 172.21.15.40:0/2292165363 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe1440116e0 con 0x7fe14c1548c0 2024-09-13T18:06:19.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.112+0000 7fe141ffb700 1 -- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c1548c0 msgr2=0x7fe14c155e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:19.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.112+0000 7fe141ffb700 1 --2- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c1548c0 0x7fe14c155e50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe1440012e0 tx=0x7fe14400fb00 comp rx=0 tx=0).stop 2024-09-13T18:06:19.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.112+0000 7fe141ffb700 1 -- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c1548c0 msgr2=0x7fe14c155e50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:06:19.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.112+0000 7fe141ffb700 1 --2- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c1548c0 0x7fe14c155e50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:19.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.112+0000 7fe151974700 1 --2- 172.21.15.40:0/2292165363 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe1300666b0 0x7fe130068b70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009c60 tx=0x7fe13c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:19.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.120+0000 7fe1533d7700 1 --2- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c14d080 0x7fe14c1f02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:19.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.120+0000 7fe1533d7700 1 -- 172.21.15.40:0/2292165363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c070f60 con 0x7fe14c14d080 2024-09-13T18:06:19.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.120+0000 7fe151173700 1 --2- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c14d080 0x7fe14c1f02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:19.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.120+0000 7fe151173700 1 -- 172.21.15.40:0/2292165363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe14c071520 con 0x7fe14c14d080 2024-09-13T18:06:19.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.120+0000 7fe151173700 1 --2- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c14d080 0x7fe14c1f02b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe1440012e0 tx=0x7fe144001400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:19.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.120+0000 7fe141ffb700 1 -- 172.21.15.40:0/2292165363 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe144023070 con 0x7fe14c14d080 2024-09-13T18:06:19.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.120+0000 7fe141ffb700 1 -- 172.21.15.40:0/2292165363 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4773 (secure 0 0 0) 0x7fe144013600 con 0x7fe14c14d080 2024-09-13T18:06:19.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.120+0000 7fe141ffb700 1 -- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c14d080 msgr2=0x7fe14c1f02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:19.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.120+0000 7fe141ffb700 1 --2- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c14d080 0x7fe14c1f02b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe1440012e0 tx=0x7fe144001400 comp rx=0 tx=0).stop 2024-09-13T18:06:19.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.120+0000 7fe141ffb700 1 -- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c14d080 msgr2=0x7fe14c1f02b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:06:19.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.120+0000 7fe141ffb700 1 --2- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c14d080 0x7fe14c1f02b0 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe1440012e0 tx=0x7fe144001400 comp rx=0 tx=0).stop 2024-09-13T18:06:19.131 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:06:19.131 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:06:19.131 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:06:19.131 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:06:19.132 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 126, 2024-09-13T18:06:19.132 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:06:19.132 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:06:19.132 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:06:19.132 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:19.132 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:06:19.132 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:06:19.132 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:06:19.132 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:06:19.132 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:06:19.132 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:06:19.132 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:06:19.132 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:06:19.132 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:06:19.133 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:06:19.133 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:06:19.133 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:06:19.133 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:06:19.133 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:06:19.133 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:19.133 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:06:19.133 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:06:19.133 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:06:19.133 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:06:19.133 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:06:19.133 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:06:19.133 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:06:19.133 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:06:19.134 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:06:19.134 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:06:19.134 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:06:19.134 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:06:19.134 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:19.134 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.134 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:06:19.134 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:06:19.134 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:06:19.135 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:06:19.135 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:06:19.135 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:06:19.135 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:06:19.135 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:06:19.135 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:06:19.135 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:06:19.135 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:06:19.135 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:06:19.135 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:06:19.135 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:06:19.135 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:06:19.135 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:06:19.135 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:06:19.136 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:06:19.136 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:06:19.136 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:06:19.136 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:06:19.136 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:06:19.136 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:06:19.136 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:06:19.136 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:06:19.136 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:06:19.136 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:06:19.136 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:19.136 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:06:19.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.136 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:06:19.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.137 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:06:19.137 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:06:19.137 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:06:19.137 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:06:19.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.137 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:06:19.137 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:06:19.137 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:19.137 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.137 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:06:19.137 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:06:19.138 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:19.138 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:19.138 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:19.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.138 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:06:19.138 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:06:19.138 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:06:19.138 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:06:19.138 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:06:19.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.138 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.138 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:06:19.138 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:06:19.138 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:06:19.138 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:06:19.139 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.139 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:06:19.139 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:06:19.139 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:19.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.139 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.139 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:06:19.139 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:06:19.139 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:19.139 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:19.139 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:19.139 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.139 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:06:19.139 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:06:19.140 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:06:19.140 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:06:19.140 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:06:19.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.140 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.140 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:06:19.140 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:06:19.140 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:06:19.140 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:06:19.140 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.140 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:06:19.140 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:06:19.140 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:19.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.141 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:06:19.141 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:06:19.141 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:19.141 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:19.141 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:19.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.141 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:06:19.141 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:06:19.141 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:06:19.141 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:06:19.141 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:06:19.141 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:19.142 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:19.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.142 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:06:19.142 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:06:19.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.142 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:06:19.142 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:06:19.142 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:06:19.142 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:19.142 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:19.142 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:06:19.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.142 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:06:19.142 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:06:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:06:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:06:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:06:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:06:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:06:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:06:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:06:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:06:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:06:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:06:19.144 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:06:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.124+0000 7fe1533d7700 1 -- 172.21.15.40:0/2292165363 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe1300666b0 msgr2=0x7fe130068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:19.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.124+0000 7fe1533d7700 1 --2- 172.21.15.40:0/2292165363 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe1300666b0 0x7fe130068b70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009c60 tx=0x7fe13c008030 comp rx=0 tx=0).stop 2024-09-13T18:06:19.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.124+0000 7fe1533d7700 1 -- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c0b0d70 msgr2=0x7fe14c150420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:19.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.124+0000 7fe1533d7700 1 --2- 172.21.15.40:0/2292165363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe14c0b0d70 0x7fe14c150420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe14800ebe0 tx=0x7fe14800eef0 comp rx=0 tx=0).stop 2024-09-13T18:06:19.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.124+0000 7fe151974700 1 -- 172.21.15.40:0/2292165363 reap_dead start 2024-09-13T18:06:19.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.124+0000 7fe151974700 1 -- 172.21.15.40:0/2292165363 reap_dead start 2024-09-13T18:06:19.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.124+0000 7fe1533d7700 1 -- 172.21.15.40:0/2292165363 shutdown_connections 2024-09-13T18:06:19.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.124+0000 7fe1533d7700 1 -- 172.21.15.40:0/2292165363 >> 172.21.15.40:0/2292165363 conn(0x7fe14c06d5b0 msgr2=0x7fe14c0c9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:19.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.124+0000 7fe1533d7700 1 -- 172.21.15.40:0/2292165363 shutdown_connections 2024-09-13T18:06:19.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.124+0000 7fe1533d7700 1 -- 172.21.15.40:0/2292165363 wait complete. 2024-09-13T18:06:19.146 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:06:19.147 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T18:06:19.147 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:06:19.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52b3fff700 1 Processor -- start 2024-09-13T18:06:19.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52b3fff700 1 -- start start 2024-09-13T18:06:19.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b406e410 0x7f52b40c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:19.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52b3fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f52b4153330 0x7f52b4153780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:19.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52b3fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b415d9a0 con 0x7f52b406e7f0 2024-09-13T18:06:19.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52b3fff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b4153cc0 con 0x7f52b4153330 2024-09-13T18:06:19.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52b3fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b4072240 con 0x7f52b406e410 2024-09-13T18:06:19.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52abfff700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f52b406e7f0 legacy=0x7f52b4159f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:06:19.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52b37fe700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f52b4153330 msgr2=0x7f52b4153780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:06:19.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52b37fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f52b4153330 0x7f52b4153780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:19.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52b2ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b406e410 0x7f52b40c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:19.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52b2ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b406e410 0x7f52b40c65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:43260/0 (socket says 172.21.15.40:43260) 2024-09-13T18:06:19.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52b2ffd700 1 -- 172.21.15.40:0/4112710409 learned_addr learned my addr 172.21.15.40:0/4112710409 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:19.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52b2ffd700 1 -- 172.21.15.40:0/4112710409 >> v2:172.21.15.40:3300/0 conn(0x7f52b4153330 msgr2=0x7f52b4153780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:06:19.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52b2ffd700 1 --2- 172.21.15.40:0/4112710409 >> v2:172.21.15.40:3300/0 conn(0x7f52b4153330 0x7f52b4153780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:19.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52b2ffd700 1 -- 172.21.15.40:0/4112710409 >> v1:172.21.15.40:6789/0 conn(0x7f52b406e7f0 legacy=0x7f52b4159f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:06:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52b2ffd700 1 -- 172.21.15.40:0/4112710409 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b40c6b00 con 0x7f52b406e410 2024-09-13T18:06:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52b2ffd700 1 --2- 172.21.15.40:0/4112710409 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b406e410 0x7f52b40c65c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009910 tx=0x7f52a4009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99edd861347cda1f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52b27fc700 1 -- 172.21.15.40:0/4112710409 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a4004230 con 0x7f52b406e410 2024-09-13T18:06:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52b27fc700 1 -- 172.21.15.40:0/4112710409 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a4004390 con 0x7f52b406e410 2024-09-13T18:06:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.252+0000 7f52b27fc700 1 -- 172.21.15.40:0/4112710409 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a4009e60 con 0x7f52b406e410 2024-09-13T18:06:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b3fff700 1 -- 172.21.15.40:0/4112710409 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b406e410 msgr2=0x7f52b40c65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b3fff700 1 --2- 172.21.15.40:0/4112710409 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b406e410 0x7f52b40c65c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009910 tx=0x7f52a4009c20 comp rx=0 tx=0).stop 2024-09-13T18:06:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b3fff700 1 -- 172.21.15.40:0/4112710409 shutdown_connections 2024-09-13T18:06:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b3fff700 1 --2- 172.21.15.40:0/4112710409 >> v2:172.21.15.40:3300/0 conn(0x7f52b4153330 0x7f52b4153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b3fff700 1 --2- 172.21.15.40:0/4112710409 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b406e410 0x7f52b40c65c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b3fff700 1 -- 172.21.15.40:0/4112710409 >> 172.21.15.40:0/4112710409 conn(0x7f52b40cbc40 msgr2=0x7f52b40cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b3fff700 1 -- 172.21.15.40:0/4112710409 shutdown_connections 2024-09-13T18:06:19.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b3fff700 1 -- 172.21.15.40:0/4112710409 wait complete. 2024-09-13T18:06:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b3fff700 1 Processor -- start 2024-09-13T18:06:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b3fff700 1 -- start start 2024-09-13T18:06:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b406e410 0x7f52b41ef5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52b406e7f0 0x7f52b40b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52b4153330 0x7f52b40ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b3fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b40c2fa0 con 0x7f52b4153330 2024-09-13T18:06:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b3fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b40b1c00 con 0x7f52b406e7f0 2024-09-13T18:06:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b3fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b4154ec0 con 0x7f52b406e410 2024-09-13T18:06:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b37fe700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52b4153330 msgr2=0x7f52b40ae580 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:06:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52b4153330 0x7f52b40ae580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b2ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b406e410 0x7f52b41ef5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:19.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b2ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b406e410 0x7f52b41ef5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:43274/0 (socket says 172.21.15.40:43274) 2024-09-13T18:06:19.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b2ffd700 1 -- 172.21.15.40:0/3050491763 learned_addr learned my addr 172.21.15.40:0/3050491763 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:19.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52abfff700 1 --2- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52b406e7f0 0x7f52b40b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b2ffd700 1 -- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52b406e7f0 msgr2=0x7f52b40b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b2ffd700 1 --2- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52b406e7f0 0x7f52b40b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b2ffd700 1 -- 172.21.15.40:0/3050491763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52b4153330 msgr2=0x7f52b40ae580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:06:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b2ffd700 1 --2- 172.21.15.40:0/3050491763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f52b4153330 0x7f52b40ae580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.256+0000 7f52b2ffd700 1 -- 172.21.15.40:0/3050491763 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a4009570 con 0x7f52b406e410 2024-09-13T18:06:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.260+0000 7f52b2ffd700 1 --2- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b406e410 0x7f52b41ef5f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009c60 tx=0x7f52a4004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:19.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.260+0000 7f52ab7fe700 1 -- 172.21.15.40:0/3050491763 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a400bc80 con 0x7f52b406e410 2024-09-13T18:06:19.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.260+0000 7f52ab7fe700 1 -- 172.21.15.40:0/3050491763 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a401f070 con 0x7f52b406e410 2024-09-13T18:06:19.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.260+0000 7f52b3fff700 1 -- 172.21.15.40:0/3050491763 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b40aeac0 con 0x7f52b406e410 2024-09-13T18:06:19.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.260+0000 7f52ab7fe700 1 -- 172.21.15.40:0/3050491763 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a40036c0 con 0x7f52b406e410 2024-09-13T18:06:19.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.260+0000 7f52b3fff700 1 -- 172.21.15.40:0/3050491763 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b40c3490 con 0x7f52b406e410 2024-09-13T18:06:19.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.260+0000 7f52b3fff700 1 --2- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b4156a90 0x7f52b40c4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:19.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.260+0000 7f52b3fff700 1 -- 172.21.15.40:0/3050491763 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b4067590 con 0x7f52b4156a90 2024-09-13T18:06:19.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.260+0000 7f52abfff700 1 --2- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b4156a90 0x7f52b40c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:19.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.260+0000 7f52abfff700 1 -- 172.21.15.40:0/3050491763 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52b415d100 con 0x7f52b4156a90 2024-09-13T18:06:19.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.260+0000 7f52ab7fe700 1 -- 172.21.15.40:0/3050491763 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f52a4021030 con 0x7f52b406e410 2024-09-13T18:06:19.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.260+0000 7f52abfff700 1 --2- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b4156a90 0x7f52b40c4240 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f529c003a30 tx=0x7f529c009aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:19.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.260+0000 7f52ab7fe700 1 --2- 172.21.15.40:0/3050491763 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5294066710 0x7f5294068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:19.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.260+0000 7f52ab7fe700 1 -- 172.21.15.40:0/3050491763 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f52a4085a10 con 0x7f52b406e410 2024-09-13T18:06:19.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.260+0000 7f52b37fe700 1 --2- 172.21.15.40:0/3050491763 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5294066710 0x7f5294068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:19.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.260+0000 7f52ab7fe700 1 -- 172.21.15.40:0/3050491763 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c012600 con 0x7f52b4156a90 2024-09-13T18:06:19.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.264+0000 7f52ab7fe700 1 -- 172.21.15.40:0/3050491763 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f529c012760 con 0x7f52b4156a90 2024-09-13T18:06:19.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.264+0000 7f52ab7fe700 1 -- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b4156a90 msgr2=0x7f52b40c4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:19.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.264+0000 7f52ab7fe700 1 --2- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b4156a90 0x7f52b40c4240 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f529c003a30 tx=0x7f529c009aa0 comp rx=0 tx=0).stop 2024-09-13T18:06:19.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.264+0000 7f52ab7fe700 1 -- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b4156a90 msgr2=0x7f52b40c4240 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:06:19.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.264+0000 7f52ab7fe700 1 --2- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b4156a90 0x7f52b40c4240 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f529c003a30 tx=0x7f529c009aa0 comp rx=0 tx=0).stop 2024-09-13T18:06:19.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.264+0000 7f52b37fe700 1 --2- 172.21.15.40:0/3050491763 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5294066710 0x7f5294068bd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f52b41567f0 tx=0x7f52ac009270 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.272+0000 7f52b3fff700 1 --2- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b4155910 0x7f52b4155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.272+0000 7f52b3fff700 1 -- 172.21.15.40:0/3050491763 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b415cb80 con 0x7f52b4155910 2024-09-13T18:06:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.272+0000 7f52abfff700 1 --2- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b4155910 0x7f52b4155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.272+0000 7f52abfff700 1 -- 172.21.15.40:0/3050491763 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f52b415d3e0 con 0x7f52b4155910 2024-09-13T18:06:19.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.272+0000 7f52abfff700 1 --2- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b4155910 0x7f52b4155000 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f529c003a30 tx=0x7f529c005a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:19.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.272+0000 7f52ab7fe700 1 -- 172.21.15.40:0/3050491763 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c023070 con 0x7f52b4155910 2024-09-13T18:06:19.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.272+0000 7f52ab7fe700 1 -- 172.21.15.40:0/3050491763 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4937 (secure 0 0 0) 0x7f529c005d00 con 0x7f52b4155910 2024-09-13T18:06:19.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.272+0000 7f52ab7fe700 1 -- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b4155910 msgr2=0x7f52b4155000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:19.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.272+0000 7f52ab7fe700 1 --2- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b4155910 0x7f52b4155000 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f529c003a30 tx=0x7f529c005a80 comp rx=0 tx=0).stop 2024-09-13T18:06:19.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.272+0000 7f52ab7fe700 1 -- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b4155910 msgr2=0x7f52b4155000 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:06:19.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.272+0000 7f52ab7fe700 1 --2- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b4155910 0x7f52b4155000 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:19.283 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:06:19.283 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:06:19.283 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:06:19.283 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:06:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 126, 2024-09-13T18:06:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:06:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:06:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:06:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:06:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:06:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:06:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:06:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:06:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:06:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:06:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:06:19.284 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:06:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:06:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:06:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:06:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:06:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:06:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:06:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:06:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:06:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:06:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:06:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:06:19.285 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:06:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:06:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:06:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:06:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:06:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:06:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:06:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:06:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:06:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:06:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:06:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:06:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:06:19.286 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:06:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:06:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:06:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:06:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:06:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:06:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:06:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:06:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:06:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:06:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:06:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:06:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:06:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:06:19.287 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:06:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:06:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:06:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:06:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:06:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:06:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:06:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:06:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:06:19.288 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:06:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:06:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:06:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:06:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:06:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:06:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:06:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:19.289 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:06:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:06:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:06:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:06:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:06:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:06:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:06:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:06:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:06:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.290 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:06:19.291 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:06:19.291 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:19.291 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.291 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.291 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:06:19.291 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:06:19.291 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:19.291 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:19.291 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:19.291 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.291 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:06:19.291 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:06:19.291 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:06:19.292 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:06:19.292 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:06:19.292 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.292 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.292 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:06:19.292 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:06:19.292 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:06:19.292 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:06:19.292 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.292 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:06:19.292 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:06:19.292 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:19.292 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.292 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.293 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:06:19.293 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:06:19.293 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:06:19.293 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:19.293 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:19.293 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.293 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:06:19.293 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:06:19.293 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:06:19.293 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:06:19.293 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:06:19.293 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:19.293 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:19.294 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.294 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:06:19.294 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:06:19.294 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.294 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:06:19.294 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:06:19.294 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:06:19.294 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:19.294 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:19.294 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:06:19.294 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.294 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:06:19.294 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:06:19.294 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:06:19.295 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:19.295 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:19.295 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:06:19.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.295 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:06:19.295 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:06:19.295 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:06:19.295 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:19.295 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:19.295 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:06:19.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.295 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:06:19.295 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:06:19.295 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:06:19.296 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:19.296 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:06:19.296 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:06:19.296 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:06:19.296 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:06:19.296 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:06:19.296 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:06:19.296 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:06:19.296 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:06:19.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:06:19.296 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:06:19.296 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:06:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.276+0000 7f52b3fff700 1 -- 172.21.15.40:0/3050491763 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5294066710 msgr2=0x7f5294068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.276+0000 7f52b3fff700 1 --2- 172.21.15.40:0/3050491763 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5294066710 0x7f5294068bd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f52b41567f0 tx=0x7f52ac009270 comp rx=0 tx=0).stop 2024-09-13T18:06:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.276+0000 7f52b3fff700 1 -- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b406e410 msgr2=0x7f52b41ef5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.276+0000 7f52b3fff700 1 --2- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b406e410 0x7f52b41ef5f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009c60 tx=0x7f52a4004770 comp rx=0 tx=0).stop 2024-09-13T18:06:19.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.276+0000 7f52b37fe700 1 -- 172.21.15.40:0/3050491763 reap_dead start 2024-09-13T18:06:19.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.276+0000 7f52b3fff700 1 -- 172.21.15.40:0/3050491763 shutdown_connections 2024-09-13T18:06:19.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.276+0000 7f52b3fff700 1 --2- 172.21.15.40:0/3050491763 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f52b406e410 0x7f52b41ef5f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:19.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.276+0000 7f52b3fff700 1 -- 172.21.15.40:0/3050491763 >> 172.21.15.40:0/3050491763 conn(0x7f52b40cbc40 msgr2=0x7f52b40cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:19.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.276+0000 7f52b3fff700 1 -- 172.21.15.40:0/3050491763 shutdown_connections 2024-09-13T18:06:19.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:19.276+0000 7f52b3fff700 1 -- 172.21.15.40:0/3050491763 wait complete. 2024-09-13T18:06:19.304 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:06:19.304 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:06:36.705 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:06:36.707 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:06:39.305 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T18:06:39.305 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-13T18:06:39.305 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T18:06:39.305 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T18:06:39.308 INFO:tasks.ceph.mon.a:Started 2024-09-13T18:06:39.308 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:06:39.308 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:06:39.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.424+0000 7f8c58a11700 1 Processor -- start 2024-09-13T18:06:39.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.424+0000 7f8c58a11700 1 -- start start 2024-09-13T18:06:39.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.424+0000 7f8c58a11700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8c54070120 0x7f8c54164dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:39.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.424+0000 7f8c58a11700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c5415a130 con 0x7f8c54156f70 2024-09-13T18:06:39.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.424+0000 7f8c58a11700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c5415a3d0 con 0x7f8c540c8c10 2024-09-13T18:06:39.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.424+0000 7f8c58a11700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c540c4e20 con 0x7f8c54070120 2024-09-13T18:06:39.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.424+0000 7f8c5359e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8c54070120 0x7f8c54164dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:39.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.424+0000 7f8c5359e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8c54070120 0x7f8c54164dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32822/0 (socket says 172.21.15.40:32822) 2024-09-13T18:06:39.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.424+0000 7f8c5359e700 1 -- 172.21.15.40:0/1505722282 learned_addr learned my addr 172.21.15.40:0/1505722282 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:39.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.424+0000 7f8c52d9d700 1 --1- 172.21.15.40:0/1505722282 >> v1:172.21.15.40:6789/0 conn(0x7f8c54156f70 0x7f8c540c9980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:48564/0 (socket says 172.21.15.40:48564) 2024-09-13T18:06:39.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.424+0000 7f8c5359e700 1 -- 172.21.15.40:0/1505722282 >> v2:172.21.15.40:3300/0 conn(0x7f8c54070120 msgr2=0x7f8c54164dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:06:39.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.424+0000 7f8c5359e700 1 -- 172.21.15.40:0/1505722282 >> v2:172.21.15.40:3300/0 conn(0x7f8c54070120 msgr2=0x7f8c54164dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:39.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.424+0000 7f8c5359e700 1 --2- 172.21.15.40:0/1505722282 >> v2:172.21.15.40:3300/0 conn(0x7f8c54070120 0x7f8c54164dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:39.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c5359e700 1 --2- 172.21.15.40:0/1505722282 >> v2:172.21.15.40:3300/0 conn(0x7f8c54070120 0x7f8c54164dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:39.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c51d9b700 1 -- 172.21.15.40:0/1505722282 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4046647043 0 0) 0x7f8c5415a3d0 con 0x7f8c540c8c10 2024-09-13T18:06:39.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c51d9b700 1 -- 172.21.15.40:0/1505722282 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c3c0035d0 con 0x7f8c540c8c10 2024-09-13T18:06:39.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c51d9b700 1 -- 172.21.15.40:0/1505722282 <== mon.1 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1376808802 0 0) 0x7f8c3c0035d0 con 0x7f8c540c8c10 2024-09-13T18:06:39.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c51d9b700 1 -- 172.21.15.40:0/1505722282 >> v2:172.21.15.40:3300/0 conn(0x7f8c54070120 msgr2=0x7f8c54164dc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:06:39.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c51d9b700 1 --2- 172.21.15.40:0/1505722282 >> v2:172.21.15.40:3300/0 conn(0x7f8c54070120 0x7f8c54164dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:39.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c51d9b700 1 -- 172.21.15.40:0/1505722282 >> v1:172.21.15.40:6789/0 conn(0x7f8c54156f70 legacy=0x7f8c540c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:39.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c51d9b700 1 -- 172.21.15.40:0/1505722282 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c540c7f50 con 0x7f8c540c8c10 2024-09-13T18:06:39.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c51d9b700 1 -- 172.21.15.40:0/1505722282 <== mon.1 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f8c48001670 con 0x7f8c540c8c10 2024-09-13T18:06:39.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c51d9b700 1 -- 172.21.15.40:0/1505722282 >> v1:172.21.15.43:6789/0 conn(0x7f8c540c8c10 legacy=0x7f8c5406fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:39.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c51d9b700 1 --2- 172.21.15.40:0/1505722282 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8c3c003a20 0x7f8c3c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:39.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c51d9b700 1 --2- 172.21.15.40:0/1505722282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c3c004870 0x7f8c3c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:39.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c51d9b700 1 --2- 172.21.15.40:0/1505722282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c3c005230 0x7f8c3c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:39.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c51d9b700 1 -- 172.21.15.40:0/1505722282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c00c150 con 0x7f8c3c004870 2024-09-13T18:06:39.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c51d9b700 1 -- 172.21.15.40:0/1505722282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c00c2c0 con 0x7f8c3c005230 2024-09-13T18:06:39.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c51d9b700 1 -- 172.21.15.40:0/1505722282 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c00c480 con 0x7f8c3c003a20 2024-09-13T18:06:39.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c5259c700 1 --2- 172.21.15.40:0/1505722282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c3c004870 0x7f8c3c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:39.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c52d9d700 1 --2- 172.21.15.40:0/1505722282 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8c3c003a20 0x7f8c3c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:39.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c5359e700 1 --2- 172.21.15.40:0/1505722282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c3c005230 0x7f8c3c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:39.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c5259c700 1 -- 172.21.15.40:0/1505722282 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8c3c003a20 msgr2=0x7f8c3c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:06:39.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c5259c700 1 --2- 172.21.15.40:0/1505722282 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8c3c003a20 0x7f8c3c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:39.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c5259c700 1 -- 172.21.15.40:0/1505722282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c3c005230 msgr2=0x7f8c3c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:06:39.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c5259c700 1 --2- 172.21.15.40:0/1505722282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c3c005230 0x7f8c3c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:39.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c5359e700 1 -- 172.21.15.40:0/1505722282 reap_dead start 2024-09-13T18:06:39.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c5259c700 1 -- 172.21.15.40:0/1505722282 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c0035d0 con 0x7f8c3c004870 2024-09-13T18:06:39.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c5259c700 1 --2- 172.21.15.40:0/1505722282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c3c004870 0x7f8c3c004cf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800c160 tx=0x7f8c3c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351b10ad335c546d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:39.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c51d9b700 1 -- 172.21.15.40:0/1505722282 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4800ccc0 con 0x7f8c3c004870 2024-09-13T18:06:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c51d9b700 1 -- 172.21.15.40:0/1505722282 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4800ce20 con 0x7f8c3c004870 2024-09-13T18:06:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c51d9b700 1 -- 172.21.15.40:0/1505722282 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c48012660 con 0x7f8c3c004870 2024-09-13T18:06:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c58a11700 1 -- 172.21.15.40:0/1505722282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c3c004870 msgr2=0x7f8c3c004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c58a11700 1 --2- 172.21.15.40:0/1505722282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c3c004870 0x7f8c3c004cf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800c160 tx=0x7f8c3c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:06:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c58a11700 1 -- 172.21.15.40:0/1505722282 shutdown_connections 2024-09-13T18:06:39.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c58a11700 1 --2- 172.21.15.40:0/1505722282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c3c004870 0x7f8c3c004cf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:39.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c58a11700 1 -- 172.21.15.40:0/1505722282 >> 172.21.15.40:0/1505722282 conn(0x7f8c54066090 msgr2=0x7f8c54066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:39.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c58a11700 1 -- 172.21.15.40:0/1505722282 shutdown_connections 2024-09-13T18:06:39.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.428+0000 7f8c58a11700 1 -- 172.21.15.40:0/1505722282 wait complete. 2024-09-13T18:06:39.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c58a11700 1 Processor -- start 2024-09-13T18:06:39.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c58a11700 1 -- start start 2024-09-13T18:06:39.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c58a11700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c3c004870 0x7f8c540c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:39.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c58a11700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8c54159770 0x7f8c54151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:39.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c58a11700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c54151a50 0x7f8c5414e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:39.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c58a11700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c54150c40 con 0x7f8c3c004870 2024-09-13T18:06:39.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c58a11700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c54151ed0 con 0x7f8c54151a50 2024-09-13T18:06:39.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c58a11700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c54151030 con 0x7f8c54159770 2024-09-13T18:06:39.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c52d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c3c004870 0x7f8c540c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:39.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c52d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c3c004870 0x7f8c540c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:32842/0 (socket says 172.21.15.40:32842) 2024-09-13T18:06:39.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c52d9d700 1 -- 172.21.15.40:0/2585530570 learned_addr learned my addr 172.21.15.40:0/2585530570 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:39.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c5259c700 1 --2- 172.21.15.40:0/2585530570 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8c54159770 0x7f8c54151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:39.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c5359e700 1 --2- 172.21.15.40:0/2585530570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c54151a50 0x7f8c5414e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:39.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c52d9d700 1 -- 172.21.15.40:0/2585530570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c3c004870 msgr2=0x7f8c540c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:06:39.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c52d9d700 1 -- 172.21.15.40:0/2585530570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c3c004870 msgr2=0x7f8c540c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:06:39.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c52d9d700 1 --2- 172.21.15.40:0/2585530570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c3c004870 0x7f8c540c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:39.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c52d9d700 1 --2- 172.21.15.40:0/2585530570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c3c004870 0x7f8c540c6c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:39.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c5259c700 1 -- 172.21.15.40:0/2585530570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c54151a50 msgr2=0x7f8c5414e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:39.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c5259c700 1 --2- 172.21.15.40:0/2585530570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c54151a50 0x7f8c5414e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:39.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c5259c700 1 -- 172.21.15.40:0/2585530570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c3c004870 msgr2=0x7f8c540c6c10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:06:39.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c5259c700 1 --2- 172.21.15.40:0/2585530570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c3c004870 0x7f8c540c6c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:39.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.432+0000 7f8c5259c700 1 -- 172.21.15.40:0/2585530570 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c541f2690 con 0x7f8c54159770 2024-09-13T18:06:39.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.436+0000 7f8c5259c700 1 --2- 172.21.15.40:0/2585530570 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8c54159770 0x7f8c54151510 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8c48016000 tx=0x7f8c48000fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:39.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.436+0000 7f8c3b7fe700 1 -- 172.21.15.40:0/2585530570 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c48005270 con 0x7f8c54159770 2024-09-13T18:06:39.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.436+0000 7f8c3b7fe700 1 -- 172.21.15.40:0/2585530570 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c48005430 con 0x7f8c54159770 2024-09-13T18:06:39.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.436+0000 7f8c3b7fe700 1 -- 172.21.15.40:0/2585530570 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c480055a0 con 0x7f8c54159770 2024-09-13T18:06:39.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.436+0000 7f8c58a11700 1 -- 172.21.15.40:0/2585530570 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c541f2970 con 0x7f8c54159770 2024-09-13T18:06:39.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.436+0000 7f8c58a11700 1 -- 172.21.15.40:0/2585530570 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c541f2f30 con 0x7f8c54159770 2024-09-13T18:06:39.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.436+0000 7f8c58a11700 1 -- 172.21.15.40:0/2585530570 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c541511a0 con 0x7f8c54159770 2024-09-13T18:06:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.436+0000 7f8c3b7fe700 1 -- 172.21.15.40:0/2585530570 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8c48022070 con 0x7f8c54159770 2024-09-13T18:06:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.440+0000 7f8c3b7fe700 1 --2- 172.21.15.40:0/2585530570 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8c40066730 0x7f8c40068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.440+0000 7f8c3b7fe700 1 -- 172.21.15.40:0/2585530570 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8c480855c0 con 0x7f8c54159770 2024-09-13T18:06:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.440+0000 7f8c52d9d700 1 --2- 172.21.15.40:0/2585530570 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8c40066730 0x7f8c40068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.440+0000 7f8c52d9d700 1 --2- 172.21.15.40:0/2585530570 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8c40066730 0x7f8c40068bf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8c440048f0 tx=0x7f8c44009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:39.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.448+0000 7f8c3b7fe700 1 -- 172.21.15.40:0/2585530570 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8c541511a0 con 0x7f8c54159770 2024-09-13T18:06:39.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.708+0000 7f8c58a11700 1 -- 172.21.15.40:0/2585530570 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8c54151170 con 0x7f8c54159770 2024-09-13T18:06:39.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.708+0000 7f8c3b7fe700 1 -- 172.21.15.40:0/2585530570 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f8c48052600 con 0x7f8c54159770 2024-09-13T18:06:39.719 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":126,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:06:39.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.712+0000 7f8c58a11700 1 -- 172.21.15.40:0/2585530570 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8c40066730 msgr2=0x7f8c40068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:39.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.712+0000 7f8c58a11700 1 --2- 172.21.15.40:0/2585530570 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8c40066730 0x7f8c40068bf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8c440048f0 tx=0x7f8c44009040 comp rx=0 tx=0).stop 2024-09-13T18:06:39.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.712+0000 7f8c58a11700 1 -- 172.21.15.40:0/2585530570 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8c54159770 msgr2=0x7f8c54151510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:39.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.712+0000 7f8c58a11700 1 --2- 172.21.15.40:0/2585530570 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8c54159770 0x7f8c54151510 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8c48016000 tx=0x7f8c48000fe0 comp rx=0 tx=0).stop 2024-09-13T18:06:39.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.712+0000 7f8c58a11700 1 -- 172.21.15.40:0/2585530570 shutdown_connections 2024-09-13T18:06:39.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.712+0000 7f8c58a11700 1 --2- 172.21.15.40:0/2585530570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c54151a50 0x7f8c5414e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:39.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.712+0000 7f8c58a11700 1 --2- 172.21.15.40:0/2585530570 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8c40066730 0x7f8c40068bf0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:39.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.712+0000 7f8c58a11700 1 --2- 172.21.15.40:0/2585530570 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8c54159770 0x7f8c54151510 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:39.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.712+0000 7f8c58a11700 1 --2- 172.21.15.40:0/2585530570 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c3c004870 0x7f8c540c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:39.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.712+0000 7f8c58a11700 1 -- 172.21.15.40:0/2585530570 >> 172.21.15.40:0/2585530570 conn(0x7f8c54066090 msgr2=0x7f8c54153320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:39.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.712+0000 7f8c58a11700 1 -- 172.21.15.40:0/2585530570 shutdown_connections 2024-09-13T18:06:39.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:39.712+0000 7f8c58a11700 1 -- 172.21.15.40:0/2585530570 wait complete. 2024-09-13T18:06:42.735 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:06:42.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74b4f15700 1 Processor -- start 2024-09-13T18:06:42.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74b4f15700 1 -- start start 2024-09-13T18:06:42.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74b4f15700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f74a804cc20 0x7f74a80dbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:42.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74b4f15700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f74a8045d00 0x7f74a804d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:42.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74b4f15700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a8007830 con 0x7f74a8051a10 2024-09-13T18:06:42.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74b4f15700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74a804d540 con 0x7f74a8045d00 2024-09-13T18:06:42.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74b4f15700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74a8007a80 con 0x7f74a804cc20 2024-09-13T18:06:42.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74affff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f74a8045d00 0x7f74a804d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:42.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74affff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f74a8045d00 0x7f74a804d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57982/0 (socket says 172.21.15.40:57982) 2024-09-13T18:06:42.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74affff700 1 -- 172.21.15.40:0/23988931 learned_addr learned my addr 172.21.15.40:0/23988931 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:42.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74af7fe700 1 --2- 172.21.15.40:0/23988931 >> v2:172.21.15.43:3300/0 conn(0x7f74a804cc20 0x7f74a80dbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:42.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74affff700 1 -- 172.21.15.40:0/23988931 >> v2:172.21.15.40:3300/0 conn(0x7f74a8045d00 msgr2=0x7f74a804d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:06:42.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74affff700 1 -- 172.21.15.40:0/23988931 >> v2:172.21.15.40:3300/0 conn(0x7f74a8045d00 msgr2=0x7f74a804d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:42.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74affff700 1 --2- 172.21.15.40:0/23988931 >> v2:172.21.15.40:3300/0 conn(0x7f74a8045d00 0x7f74a804d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:42.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74affff700 1 --2- 172.21.15.40:0/23988931 >> v2:172.21.15.40:3300/0 conn(0x7f74a8045d00 0x7f74a804d000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:42.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74af7fe700 1 -- 172.21.15.40:0/23988931 >> v2:172.21.15.40:3300/0 conn(0x7f74a8045d00 msgr2=0x7f74a804d000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:06:42.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74af7fe700 1 --2- 172.21.15.40:0/23988931 >> v2:172.21.15.40:3300/0 conn(0x7f74a8045d00 0x7f74a804d000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:42.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74af7fe700 1 -- 172.21.15.40:0/23988931 >> v1:172.21.15.43:6789/0 conn(0x7f74a8051a10 legacy=0x7f74a8008b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:42.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74af7fe700 1 -- 172.21.15.40:0/23988931 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a8007d60 con 0x7f74a804cc20 2024-09-13T18:06:42.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74af7fe700 1 --2- 172.21.15.40:0/23988931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a804cc20 0x7f74a80dbcb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f74a800a880 tx=0x7f74a400bdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58dc5c1273089a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:42.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74ae7fc700 1 -- 172.21.15.40:0/23988931 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a400dd30 con 0x7f74a804cc20 2024-09-13T18:06:42.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74ae7fc700 1 -- 172.21.15.40:0/23988931 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a400de90 con 0x7f74a804cc20 2024-09-13T18:06:42.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.852+0000 7f74ae7fc700 1 -- 172.21.15.40:0/23988931 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a4010690 con 0x7f74a804cc20 2024-09-13T18:06:42.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74b4f15700 1 -- 172.21.15.40:0/23988931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a804cc20 msgr2=0x7f74a80dbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:42.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74b4f15700 1 --2- 172.21.15.40:0/23988931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a804cc20 0x7f74a80dbcb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f74a800a880 tx=0x7f74a400bdd0 comp rx=0 tx=0).stop 2024-09-13T18:06:42.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74b4f15700 1 -- 172.21.15.40:0/23988931 shutdown_connections 2024-09-13T18:06:42.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74b4f15700 1 --2- 172.21.15.40:0/23988931 >> v2:172.21.15.40:3300/0 conn(0x7f74a8045d00 0x7f74a804d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:42.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74b4f15700 1 --2- 172.21.15.40:0/23988931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a804cc20 0x7f74a80dbcb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:42.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74b4f15700 1 -- 172.21.15.40:0/23988931 >> 172.21.15.40:0/23988931 conn(0x7f74a8050e60 msgr2=0x7f74a8051270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:42.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74b4f15700 1 -- 172.21.15.40:0/23988931 shutdown_connections 2024-09-13T18:06:42.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74b4f15700 1 -- 172.21.15.40:0/23988931 wait complete. 2024-09-13T18:06:42.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74b4f15700 1 Processor -- start 2024-09-13T18:06:42.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74b4f15700 1 -- start start 2024-09-13T18:06:42.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74b4f15700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f74a8045d00 0x7f74a804ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:42.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74b4f15700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a804cc20 0x7f74a80da250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:42.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74b4f15700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a8051a10 0x7f74a8166ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:42.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74b4f15700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a804b280 con 0x7f74a8051a10 2024-09-13T18:06:42.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74b4f15700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80d2fa0 con 0x7f74a804cc20 2024-09-13T18:06:42.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74b4f15700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80d2d00 con 0x7f74a8045d00 2024-09-13T18:06:42.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74affff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a8051a10 0x7f74a8166ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:42.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74affff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a8051a10 0x7f74a8166ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57996/0 (socket says 172.21.15.40:57996) 2024-09-13T18:06:42.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74affff700 1 -- 172.21.15.40:0/82892497 learned_addr learned my addr 172.21.15.40:0/82892497 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:42.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74af7fe700 1 --2- 172.21.15.40:0/82892497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f74a8045d00 0x7f74a804ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:42.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74aeffd700 1 --2- 172.21.15.40:0/82892497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a804cc20 0x7f74a80da250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:42.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74affff700 1 -- 172.21.15.40:0/82892497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a8051a10 msgr2=0x7f74a8166ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-13T18:06:42.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74affff700 1 -- 172.21.15.40:0/82892497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a8051a10 msgr2=0x7f74a8166ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:06:42.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74affff700 1 --2- 172.21.15.40:0/82892497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a8051a10 0x7f74a8166ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:42.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.856+0000 7f74affff700 1 --2- 172.21.15.40:0/82892497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a8051a10 0x7f74a8166ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:42.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.860+0000 7f74af7fe700 1 -- 172.21.15.40:0/82892497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a804cc20 msgr2=0x7f74a80da250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:42.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.860+0000 7f74af7fe700 1 --2- 172.21.15.40:0/82892497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a804cc20 0x7f74a80da250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:42.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.860+0000 7f74af7fe700 1 -- 172.21.15.40:0/82892497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a8051a10 msgr2=0x7f74a8166ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:06:42.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.860+0000 7f74af7fe700 1 --2- 172.21.15.40:0/82892497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a8051a10 0x7f74a8166ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:42.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.860+0000 7f74af7fe700 1 -- 172.21.15.40:0/82892497 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a400b750 con 0x7f74a8045d00 2024-09-13T18:06:42.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.860+0000 7f74af7fe700 1 --2- 172.21.15.40:0/82892497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f74a8045d00 0x7f74a804ad40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f74a4018000 tx=0x7f74a4004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:42.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.860+0000 7f749bfff700 1 -- 172.21.15.40:0/82892497 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a401d070 con 0x7f74a8045d00 2024-09-13T18:06:42.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.860+0000 7f749bfff700 1 -- 172.21.15.40:0/82892497 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a4010620 con 0x7f74a8045d00 2024-09-13T18:06:42.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.860+0000 7f74b4f15700 1 -- 172.21.15.40:0/82892497 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a804a840 con 0x7f74a8045d00 2024-09-13T18:06:42.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.860+0000 7f749bfff700 1 -- 172.21.15.40:0/82892497 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a4010c70 con 0x7f74a8045d00 2024-09-13T18:06:42.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.860+0000 7f74b4f15700 1 -- 172.21.15.40:0/82892497 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a80499d0 con 0x7f74a8045d00 2024-09-13T18:06:42.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.860+0000 7f74b4f15700 1 -- 172.21.15.40:0/82892497 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a8049bc0 con 0x7f74a8045d00 2024-09-13T18:06:42.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.860+0000 7f749bfff700 1 -- 172.21.15.40:0/82892497 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f74a40036c0 con 0x7f74a8045d00 2024-09-13T18:06:42.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.860+0000 7f749bfff700 1 --2- 172.21.15.40:0/82892497 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7494066760 0x7f7494068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:42.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.864+0000 7f74aeffd700 1 --2- 172.21.15.40:0/82892497 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7494066760 0x7f7494068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:42.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.864+0000 7f749bfff700 1 -- 172.21.15.40:0/82892497 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f74a4085e50 con 0x7f74a8045d00 2024-09-13T18:06:42.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.864+0000 7f74aeffd700 1 --2- 172.21.15.40:0/82892497 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7494066760 0x7f7494068c20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f749c0018a0 tx=0x7f749c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:42.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:42.872+0000 7f749bfff700 1 -- 172.21.15.40:0/82892497 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f74a4053090 con 0x7f74a8045d00 2024-09-13T18:06:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:43.116+0000 7f74b4f15700 1 -- 172.21.15.40:0/82892497 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f74a804bbe0 con 0x7f74a8045d00 2024-09-13T18:06:43.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:43.116+0000 7f749bfff700 1 -- 172.21.15.40:0/82892497 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f74a4027610 con 0x7f74a8045d00 2024-09-13T18:06:43.125 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":126,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":24,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:06:43.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:43.120+0000 7f74b4f15700 1 -- 172.21.15.40:0/82892497 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7494066760 msgr2=0x7f7494068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:43.120+0000 7f74b4f15700 1 --2- 172.21.15.40:0/82892497 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7494066760 0x7f7494068c20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f749c0018a0 tx=0x7f749c009040 comp rx=0 tx=0).stop 2024-09-13T18:06:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:43.120+0000 7f74b4f15700 1 -- 172.21.15.40:0/82892497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f74a8045d00 msgr2=0x7f74a804ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:43.120+0000 7f74b4f15700 1 --2- 172.21.15.40:0/82892497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f74a8045d00 0x7f74a804ad40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f74a4018000 tx=0x7f74a4004770 comp rx=0 tx=0).stop 2024-09-13T18:06:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:43.120+0000 7f74b4f15700 1 -- 172.21.15.40:0/82892497 shutdown_connections 2024-09-13T18:06:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:43.120+0000 7f74b4f15700 1 --2- 172.21.15.40:0/82892497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74a8051a10 0x7f74a8166ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:43.120+0000 7f74b4f15700 1 --2- 172.21.15.40:0/82892497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74a804cc20 0x7f74a80da250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:43.120+0000 7f74b4f15700 1 --2- 172.21.15.40:0/82892497 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7494066760 0x7f7494068c20 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:43.120+0000 7f74b4f15700 1 --2- 172.21.15.40:0/82892497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f74a8045d00 0x7f74a804ad40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:43.120+0000 7f74b4f15700 1 -- 172.21.15.40:0/82892497 >> 172.21.15.40:0/82892497 conn(0x7f74a8050e60 msgr2=0x7f74a8050440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:43.120+0000 7f74b4f15700 1 -- 172.21.15.40:0/82892497 shutdown_connections 2024-09-13T18:06:43.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:43.120+0000 7f74b4f15700 1 -- 172.21.15.40:0/82892497 wait complete. 2024-09-13T18:06:46.141 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:06:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.248+0000 7f46259ec700 1 Processor -- start 2024-09-13T18:06:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.248+0000 7f46259ec700 1 -- start start 2024-09-13T18:06:46.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.248+0000 7f46259ec700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4620151470 0x7f46200c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:46.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.248+0000 7f46259ec700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4620071bf0 0x7f4620072020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:46.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.248+0000 7f46259ec700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46200c4520 con 0x7f46200b1bd0 2024-09-13T18:06:46.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.248+0000 7f46259ec700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46200c4dc0 con 0x7f4620151470 2024-09-13T18:06:46.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.248+0000 7f46259ec700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4620157cf0 con 0x7f4620071bf0 2024-09-13T18:06:46.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.248+0000 7f46249ea700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4620151470 0x7f46200c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:46.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.248+0000 7f461ffff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f46200b1bd0 0x7f46201571d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39774/0 (socket says 172.21.15.40:39774) 2024-09-13T18:06:46.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.248+0000 7f461ffff700 1 -- 172.21.15.40:0/3489250175 learned_addr learned my addr 172.21.15.40:0/3489250175 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:06:46.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.248+0000 7f46249ea700 1 -- 172.21.15.40:0/3489250175 >> v2:172.21.15.40:3300/0 conn(0x7f4620151470 msgr2=0x7f46200c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:06:46.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.248+0000 7f46249ea700 1 -- 172.21.15.40:0/3489250175 >> v2:172.21.15.40:3300/0 conn(0x7f4620151470 msgr2=0x7f46200c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:46.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.248+0000 7f46249ea700 1 --2- 172.21.15.40:0/3489250175 >> v2:172.21.15.40:3300/0 conn(0x7f4620151470 0x7f46200c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:46.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.248+0000 7f46249ea700 1 --2- 172.21.15.40:0/3489250175 >> v2:172.21.15.40:3300/0 conn(0x7f4620151470 0x7f46200c5250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:46.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.248+0000 7f46251eb700 1 --2- 172.21.15.40:0/3489250175 >> v2:172.21.15.43:3300/0 conn(0x7f4620071bf0 0x7f4620072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:46.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.248+0000 7f46251eb700 1 -- 172.21.15.40:0/3489250175 >> v2:172.21.15.40:3300/0 conn(0x7f4620151470 msgr2=0x7f46200c5250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:06:46.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.248+0000 7f46251eb700 1 --2- 172.21.15.40:0/3489250175 >> v2:172.21.15.40:3300/0 conn(0x7f4620151470 0x7f46200c5250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:46.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.248+0000 7f46251eb700 1 -- 172.21.15.40:0/3489250175 >> v1:172.21.15.40:6789/0 conn(0x7f46200b1bd0 legacy=0x7f46201571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:46.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.248+0000 7f46251eb700 1 -- 172.21.15.40:0/3489250175 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4620157fd0 con 0x7f4620071bf0 2024-09-13T18:06:46.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.248+0000 7f46251eb700 1 --2- 172.21.15.40:0/3489250175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4620071bf0 0x7f4620072020 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4620070b70 tx=0x7f461800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96f1054577607cf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.252+0000 7f461f7fe700 1 -- 172.21.15.40:0/3489250175 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461800e070 con 0x7f4620071bf0 2024-09-13T18:06:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.252+0000 7f461f7fe700 1 -- 172.21.15.40:0/3489250175 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461800bbd0 con 0x7f4620071bf0 2024-09-13T18:06:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.252+0000 7f461f7fe700 1 -- 172.21.15.40:0/3489250175 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4618012430 con 0x7f4620071bf0 2024-09-13T18:06:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.252+0000 7f46259ec700 1 -- 172.21.15.40:0/3489250175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4620071bf0 msgr2=0x7f4620072020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.252+0000 7f46259ec700 1 --2- 172.21.15.40:0/3489250175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4620071bf0 0x7f4620072020 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4620070b70 tx=0x7f461800b330 comp rx=0 tx=0).stop 2024-09-13T18:06:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.252+0000 7f46259ec700 1 -- 172.21.15.40:0/3489250175 shutdown_connections 2024-09-13T18:06:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.252+0000 7f46259ec700 1 --2- 172.21.15.40:0/3489250175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4620071bf0 0x7f4620072020 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.252+0000 7f46259ec700 1 --2- 172.21.15.40:0/3489250175 >> v2:172.21.15.40:3300/0 conn(0x7f4620151470 0x7f46200c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.252+0000 7f46259ec700 1 -- 172.21.15.40:0/3489250175 >> 172.21.15.40:0/3489250175 conn(0x7f462006e4f0 msgr2=0x7f462006e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.252+0000 7f46259ec700 1 -- 172.21.15.40:0/3489250175 shutdown_connections 2024-09-13T18:06:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.252+0000 7f46259ec700 1 -- 172.21.15.40:0/3489250175 wait complete. 2024-09-13T18:06:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.252+0000 7f46259ec700 1 Processor -- start 2024-09-13T18:06:46.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.252+0000 7f46259ec700 1 -- start start 2024-09-13T18:06:46.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f46259ec700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4620071bf0 0x7f462006c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:46.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f46259ec700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46200b1bd0 0x7f46201f04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:46.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f46259ec700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4620151470 0x7f46201563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:46.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f46259ec700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462006d440 con 0x7f4620071bf0 2024-09-13T18:06:46.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f46259ec700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462006c110 con 0x7f46200b1bd0 2024-09-13T18:06:46.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f46259ec700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462006cad0 con 0x7f4620151470 2024-09-13T18:06:46.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f46249ea700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4620071bf0 0x7f462006c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:46.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f46249ea700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4620071bf0 0x7f462006c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58010/0 (socket says 172.21.15.40:58010) 2024-09-13T18:06:46.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f46249ea700 1 -- 172.21.15.40:0/1454984141 learned_addr learned my addr 172.21.15.40:0/1454984141 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:46.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f461ffff700 1 --2- 172.21.15.40:0/1454984141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46200b1bd0 0x7f46201f04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:46.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f46249ea700 1 -- 172.21.15.40:0/1454984141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4620071bf0 msgr2=0x7f462006c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:06:46.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f46249ea700 1 -- 172.21.15.40:0/1454984141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4620071bf0 msgr2=0x7f462006c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:06:46.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f46249ea700 1 --2- 172.21.15.40:0/1454984141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4620071bf0 0x7f462006c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:46.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f46251eb700 1 --2- 172.21.15.40:0/1454984141 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4620151470 0x7f46201563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:46.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f46249ea700 1 --2- 172.21.15.40:0/1454984141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4620071bf0 0x7f462006c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:46.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f461ffff700 1 -- 172.21.15.40:0/1454984141 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4620151470 msgr2=0x7f46201563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:46.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f461ffff700 1 --2- 172.21.15.40:0/1454984141 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4620151470 0x7f46201563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:46.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f461ffff700 1 -- 172.21.15.40:0/1454984141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4620071bf0 msgr2=0x7f462006c530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:06:46.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f461ffff700 1 --2- 172.21.15.40:0/1454984141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4620071bf0 0x7f462006c530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:46.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f461ffff700 1 -- 172.21.15.40:0/1454984141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4618009a70 con 0x7f46200b1bd0 2024-09-13T18:06:46.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f461ffff700 1 --2- 172.21.15.40:0/1454984141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46200b1bd0 0x7f46201f04a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f460c00c390 tx=0x7f460c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:46.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f461d7fa700 1 -- 172.21.15.40:0/1454984141 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c005d90 con 0x7f46200b1bd0 2024-09-13T18:06:46.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f461d7fa700 1 -- 172.21.15.40:0/1454984141 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460c00ce10 con 0x7f46200b1bd0 2024-09-13T18:06:46.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f46259ec700 1 -- 172.21.15.40:0/1454984141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462006cf80 con 0x7f46200b1bd0 2024-09-13T18:06:46.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f46259ec700 1 -- 172.21.15.40:0/1454984141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462006ba50 con 0x7f46200b1bd0 2024-09-13T18:06:46.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f461d7fa700 1 -- 172.21.15.40:0/1454984141 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c0072f0 con 0x7f46200b1bd0 2024-09-13T18:06:46.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.256+0000 7f46259ec700 1 -- 172.21.15.40:0/1454984141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4600005310 con 0x7f46200b1bd0 2024-09-13T18:06:46.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.260+0000 7f461d7fa700 1 -- 172.21.15.40:0/1454984141 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f460c007450 con 0x7f46200b1bd0 2024-09-13T18:06:46.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.260+0000 7f461d7fa700 1 --2- 172.21.15.40:0/1454984141 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4604066730 0x7f4604068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:46.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.260+0000 7f461d7fa700 1 -- 172.21.15.40:0/1454984141 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f460c083730 con 0x7f46200b1bd0 2024-09-13T18:06:46.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.260+0000 7f46249ea700 1 --2- 172.21.15.40:0/1454984141 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4604066730 0x7f4604068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:46.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.260+0000 7f46249ea700 1 --2- 172.21.15.40:0/1454984141 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4604066730 0x7f4604068bf0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4614006f30 tx=0x7f4614008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:46.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.268+0000 7f461d7fa700 1 -- 172.21.15.40:0/1454984141 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f460c050770 con 0x7f46200b1bd0 2024-09-13T18:06:46.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.560+0000 7f46259ec700 1 -- 172.21.15.40:0/1454984141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f46000055c0 con 0x7f46200b1bd0 2024-09-13T18:06:46.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.564+0000 7f461d7fa700 1 -- 172.21.15.40:0/1454984141 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f460c017610 con 0x7f46200b1bd0 2024-09-13T18:06:46.571 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":126,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":27,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:06:46.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.564+0000 7f46259ec700 1 -- 172.21.15.40:0/1454984141 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4604066730 msgr2=0x7f4604068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:46.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.564+0000 7f46259ec700 1 --2- 172.21.15.40:0/1454984141 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4604066730 0x7f4604068bf0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4614006f30 tx=0x7f4614008040 comp rx=0 tx=0).stop 2024-09-13T18:06:46.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.564+0000 7f46259ec700 1 -- 172.21.15.40:0/1454984141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46200b1bd0 msgr2=0x7f46201f04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:46.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.564+0000 7f46259ec700 1 --2- 172.21.15.40:0/1454984141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46200b1bd0 0x7f46201f04a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f460c00c390 tx=0x7f460c00c6a0 comp rx=0 tx=0).stop 2024-09-13T18:06:46.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.564+0000 7f46259ec700 1 -- 172.21.15.40:0/1454984141 shutdown_connections 2024-09-13T18:06:46.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.564+0000 7f46259ec700 1 --2- 172.21.15.40:0/1454984141 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4604066730 0x7f4604068bf0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:46.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.564+0000 7f46259ec700 1 --2- 172.21.15.40:0/1454984141 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4620151470 0x7f46201563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:46.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.564+0000 7f46259ec700 1 --2- 172.21.15.40:0/1454984141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46200b1bd0 0x7f46201f04a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:46.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.564+0000 7f46259ec700 1 --2- 172.21.15.40:0/1454984141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4620071bf0 0x7f462006c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:46.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.564+0000 7f46259ec700 1 -- 172.21.15.40:0/1454984141 >> 172.21.15.40:0/1454984141 conn(0x7f462006e4f0 msgr2=0x7f46200c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:46.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.564+0000 7f46259ec700 1 -- 172.21.15.40:0/1454984141 shutdown_connections 2024-09-13T18:06:46.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:46.564+0000 7f46259ec700 1 -- 172.21.15.40:0/1454984141 wait complete. 2024-09-13T18:06:49.591 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:06:49.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba77fff700 1 Processor -- start 2024-09-13T18:06:49.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba77fff700 1 -- start start 2024-09-13T18:06:49.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba77fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba70046df0 0x7fba70051e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:49.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba77fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fba70047480 0x7fba700523b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:49.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba77fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba700e26a0 con 0x7fba700d8b10 2024-09-13T18:06:49.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba77fff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7004d3b0 con 0x7fba70047480 2024-09-13T18:06:49.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba77fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba700e2b30 con 0x7fba70046df0 2024-09-13T18:06:49.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba777fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fba70047480 0x7fba700523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:49.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba777fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fba70047480 0x7fba700523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58012/0 (socket says 172.21.15.40:58012) 2024-09-13T18:06:49.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba777fe700 1 -- 172.21.15.40:0/882481648 learned_addr learned my addr 172.21.15.40:0/882481648 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:49.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba767fc700 1 --2- 172.21.15.40:0/882481648 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba70046df0 0x7fba70051e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:49.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba777fe700 1 -- 172.21.15.40:0/882481648 >> v2:172.21.15.40:3300/0 conn(0x7fba70047480 msgr2=0x7fba700523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:06:49.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba777fe700 1 -- 172.21.15.40:0/882481648 >> v2:172.21.15.40:3300/0 conn(0x7fba70047480 msgr2=0x7fba700523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:49.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba777fe700 1 --2- 172.21.15.40:0/882481648 >> v2:172.21.15.40:3300/0 conn(0x7fba70047480 0x7fba700523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:49.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba777fe700 1 --2- 172.21.15.40:0/882481648 >> v2:172.21.15.40:3300/0 conn(0x7fba70047480 0x7fba700523b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:49.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba767fc700 1 -- 172.21.15.40:0/882481648 >> v2:172.21.15.40:3300/0 conn(0x7fba70047480 msgr2=0x7fba700523b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:06:49.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba767fc700 1 --2- 172.21.15.40:0/882481648 >> v2:172.21.15.40:3300/0 conn(0x7fba70047480 0x7fba700523b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:49.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba767fc700 1 -- 172.21.15.40:0/882481648 >> v1:172.21.15.43:6789/0 conn(0x7fba700d8b10 legacy=0x7fba70047a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:49.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba767fc700 1 -- 172.21.15.40:0/882481648 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7004d0d0 con 0x7fba70046df0 2024-09-13T18:06:49.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba767fc700 1 --2- 172.21.15.40:0/882481648 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba70046df0 0x7fba70051e70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009850 tx=0x7fba6c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2001878d88cf3b88 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:49.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba75ffb700 1 -- 172.21.15.40:0/882481648 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c011830 con 0x7fba70046df0 2024-09-13T18:06:49.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba75ffb700 1 -- 172.21.15.40:0/882481648 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6c011990 con 0x7fba70046df0 2024-09-13T18:06:49.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.708+0000 7fba75ffb700 1 -- 172.21.15.40:0/882481648 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c011d20 con 0x7fba70046df0 2024-09-13T18:06:49.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.712+0000 7fba77fff700 1 -- 172.21.15.40:0/882481648 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba70046df0 msgr2=0x7fba70051e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:49.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.712+0000 7fba77fff700 1 --2- 172.21.15.40:0/882481648 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba70046df0 0x7fba70051e70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009850 tx=0x7fba6c009b60 comp rx=0 tx=0).stop 2024-09-13T18:06:49.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.712+0000 7fba77fff700 1 -- 172.21.15.40:0/882481648 shutdown_connections 2024-09-13T18:06:49.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.712+0000 7fba77fff700 1 --2- 172.21.15.40:0/882481648 >> v2:172.21.15.40:3300/0 conn(0x7fba70047480 0x7fba700523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:49.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.712+0000 7fba77fff700 1 --2- 172.21.15.40:0/882481648 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba70046df0 0x7fba70051e70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:49.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.712+0000 7fba77fff700 1 -- 172.21.15.40:0/882481648 >> 172.21.15.40:0/882481648 conn(0x7fba7000dc40 msgr2=0x7fba7000e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:49.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.712+0000 7fba77fff700 1 -- 172.21.15.40:0/882481648 shutdown_connections 2024-09-13T18:06:49.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.712+0000 7fba77fff700 1 -- 172.21.15.40:0/882481648 wait complete. 2024-09-13T18:06:49.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.712+0000 7fba77fff700 1 Processor -- start 2024-09-13T18:06:49.721 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.712+0000 7fba77fff700 1 -- start start 2024-09-13T18:06:49.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.712+0000 7fba77fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba70046df0 0x7fba701780c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:49.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.712+0000 7fba77fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba70047480 0x7fba7016a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:49.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba77fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba700d8b10 0x7fba700dba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:49.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba77fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7004b9a0 con 0x7fba70047480 2024-09-13T18:06:49.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba77fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba70178600 con 0x7fba70046df0 2024-09-13T18:06:49.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba77fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7004b100 con 0x7fba700d8b10 2024-09-13T18:06:49.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba767fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba70047480 0x7fba7016a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:49.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba767fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba70047480 0x7fba7016a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58026/0 (socket says 172.21.15.40:58026) 2024-09-13T18:06:49.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba767fc700 1 -- 172.21.15.40:0/885172751 learned_addr learned my addr 172.21.15.40:0/885172751 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:06:49.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba76ffd700 1 --2- 172.21.15.40:0/885172751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba70046df0 0x7fba701780c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:49.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba767fc700 1 -- 172.21.15.40:0/885172751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba70047480 msgr2=0x7fba7016a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:06:49.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba767fc700 1 -- 172.21.15.40:0/885172751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba70047480 msgr2=0x7fba7016a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:06:49.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba767fc700 1 --2- 172.21.15.40:0/885172751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba70047480 0x7fba7016a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:49.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba777fe700 1 --2- 172.21.15.40:0/885172751 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba700d8b10 0x7fba700dba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:49.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba767fc700 1 --2- 172.21.15.40:0/885172751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba70047480 0x7fba7016a000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:49.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba777fe700 1 -- 172.21.15.40:0/885172751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba70046df0 msgr2=0x7fba701780c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:49.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba777fe700 1 --2- 172.21.15.40:0/885172751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba70046df0 0x7fba701780c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:49.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba777fe700 1 -- 172.21.15.40:0/885172751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba70047480 msgr2=0x7fba7016a000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:06:49.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba777fe700 1 --2- 172.21.15.40:0/885172751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba70047480 0x7fba7016a000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:49.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba777fe700 1 -- 172.21.15.40:0/885172751 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6c0094b0 con 0x7fba700d8b10 2024-09-13T18:06:49.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba777fe700 1 --2- 172.21.15.40:0/885172751 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba700d8b10 0x7fba700dba60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba7804e9d0 tx=0x7fba78074f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:49.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba5f7fe700 1 -- 172.21.15.40:0/885172751 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78067720 con 0x7fba700d8b10 2024-09-13T18:06:49.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba5f7fe700 1 -- 172.21.15.40:0/885172751 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba78072050 con 0x7fba700d8b10 2024-09-13T18:06:49.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba77fff700 1 -- 172.21.15.40:0/885172751 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7004b3e0 con 0x7fba700d8b10 2024-09-13T18:06:49.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba5f7fe700 1 -- 172.21.15.40:0/885172751 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba780721c0 con 0x7fba700d8b10 2024-09-13T18:06:49.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba77fff700 1 -- 172.21.15.40:0/885172751 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba701778c0 con 0x7fba700d8b10 2024-09-13T18:06:49.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.716+0000 7fba77fff700 1 -- 172.21.15.40:0/885172751 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7000a620 con 0x7fba700d8b10 2024-09-13T18:06:49.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.720+0000 7fba5f7fe700 1 -- 172.21.15.40:0/885172751 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fba78072340 con 0x7fba700d8b10 2024-09-13T18:06:49.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.720+0000 7fba5f7fe700 1 --2- 172.21.15.40:0/885172751 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fba64066730 0x7fba64068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:49.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.720+0000 7fba5f7fe700 1 -- 172.21.15.40:0/885172751 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fba780fed70 con 0x7fba700d8b10 2024-09-13T18:06:49.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.720+0000 7fba76ffd700 1 --2- 172.21.15.40:0/885172751 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fba64066730 0x7fba64068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:49.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.720+0000 7fba76ffd700 1 --2- 172.21.15.40:0/885172751 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fba64066730 0x7fba64068bf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fba68000f60 tx=0x7fba68009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:06:49.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:49.728+0000 7fba5f7fe700 1 -- 172.21.15.40:0/885172751 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fba780cbdb0 con 0x7fba700d8b10 2024-09-13T18:06:50.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:50.012+0000 7fba77fff700 1 -- 172.21.15.40:0/885172751 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fba7004cd00 con 0x7fba700d8b10 2024-09-13T18:06:50.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:50.012+0000 7fba5f7fe700 1 -- 172.21.15.40:0/885172751 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7fba780cf3d0 con 0x7fba700d8b10 2024-09-13T18:06:50.020 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":126,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":31,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:06:50.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:50.012+0000 7fba77fff700 1 -- 172.21.15.40:0/885172751 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fba64066730 msgr2=0x7fba64068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:50.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:50.012+0000 7fba77fff700 1 --2- 172.21.15.40:0/885172751 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fba64066730 0x7fba64068bf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fba68000f60 tx=0x7fba68009040 comp rx=0 tx=0).stop 2024-09-13T18:06:50.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:50.012+0000 7fba77fff700 1 -- 172.21.15.40:0/885172751 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba700d8b10 msgr2=0x7fba700dba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:50.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:50.012+0000 7fba77fff700 1 --2- 172.21.15.40:0/885172751 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba700d8b10 0x7fba700dba60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba7804e9d0 tx=0x7fba78074f00 comp rx=0 tx=0).stop 2024-09-13T18:06:50.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:50.016+0000 7fba77fff700 1 -- 172.21.15.40:0/885172751 shutdown_connections 2024-09-13T18:06:50.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:50.016+0000 7fba77fff700 1 --2- 172.21.15.40:0/885172751 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fba64066730 0x7fba64068bf0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:50.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:50.016+0000 7fba77fff700 1 --2- 172.21.15.40:0/885172751 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba700d8b10 0x7fba700dba60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:50.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:50.016+0000 7fba77fff700 1 --2- 172.21.15.40:0/885172751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba70047480 0x7fba7016a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:50.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:50.016+0000 7fba77fff700 1 --2- 172.21.15.40:0/885172751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba70046df0 0x7fba701780c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:50.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:50.016+0000 7fba77fff700 1 -- 172.21.15.40:0/885172751 >> 172.21.15.40:0/885172751 conn(0x7fba7000dc40 msgr2=0x7fba700e0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:06:50.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:50.016+0000 7fba77fff700 1 -- 172.21.15.40:0/885172751 shutdown_connections 2024-09-13T18:06:50.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:50.016+0000 7fba77fff700 1 -- 172.21.15.40:0/885172751 wait complete. 2024-09-13T18:06:53.039 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:06:53.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.136+0000 7f816e38e700 1 Processor -- start 2024-09-13T18:06:53.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.136+0000 7f816e38e700 1 -- start start 2024-09-13T18:06:53.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.136+0000 7f816e38e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 0x7f8168071b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:53.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.136+0000 7f816e38e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81680cba20 con 0x7f81681500c0 2024-09-13T18:06:53.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.136+0000 7f816e38e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81680724f0 con 0x7f8168072080 2024-09-13T18:06:53.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.136+0000 7f816e38e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81681560e0 con 0x7f8168070500 2024-09-13T18:06:53.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.136+0000 7f816d38c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f81681500c0 0x7f81680c66d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46454/0 (socket says 172.21.15.40:46454) 2024-09-13T18:06:53.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.136+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 learned_addr learned my addr 172.21.15.40:0/2436105594 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:06:53.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.136+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 0x7f8168071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:53.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.136+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 msgr2=0x7f8168071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:06:53.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.136+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 msgr2=0x7f8168071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:53.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.136+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 0x7f8168071b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:53.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.136+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 0x7f8168071b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:53.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.340+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 0x7f8168071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:53.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.340+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 msgr2=0x7f8168071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:06:53.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.340+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 msgr2=0x7f8168071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:53.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.340+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 0x7f8168071b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:53.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.340+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 0x7f8168071b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:06:53.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.743+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 0x7f8168071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:53.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.743+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 msgr2=0x7f8168071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:06:53.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.743+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 msgr2=0x7f8168071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:53.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.743+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 0x7f8168071b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:53.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:53.743+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 0x7f8168071b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:06:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:54.411+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 >> v1:172.21.15.40:6789/0 conn(0x7f81681500c0 legacy=0x7f81680c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-13T18:06:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:54.411+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 >> v1:172.21.15.40:6789/0 conn(0x7f81681500c0 legacy=0x7f81680c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:06:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:54.411+0000 7f816d38c700 1 --1- 172.21.15.40:0/2436105594 >> v1:172.21.15.40:6789/0 conn(0x7f81681500c0 0x7f81680c66d0 :-1 s=OPENED pgs=31 cs=1 l=1).handle_message read tag failed 2024-09-13T18:06:54.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:54.411+0000 7f816d38c700 1 --1- 172.21.15.40:0/2436105594 >> v1:172.21.15.40:6789/0 conn(0x7f81681500c0 0x7f81680c66d0 :-1 s=OPENED pgs=31 cs=1 l=1).fault on lossy channel, failing 2024-09-13T18:06:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:54.547+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 0x7f8168071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:54.547+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 msgr2=0x7f8168071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:06:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:54.547+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 msgr2=0x7f8168071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:54.547+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 0x7f8168071b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:54.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:54.547+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 0x7f8168071b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:06:56.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.135+0000 7f815effd700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 msgr2=0x7f8168071b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:06:56.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.135+0000 7f815effd700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f8168070500 0x7f8168071b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.135+0000 7f815effd700 1 -- 172.21.15.40:0/2436105594 >> v1:172.21.15.43:6789/0 conn(0x7f8168072080 legacy=0x7f81681608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:06:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.135+0000 7f815effd700 1 -- 172.21.15.40:0/2436105594 >> v1:172.21.15.40:6789/0 conn(0x7f81681500c0 legacy=0x7f81680c66d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:06:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.135+0000 7f815effd700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c005b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.135+0000 7f815effd700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c0086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.135+0000 7f815effd700 1 -- 172.21.15.40:0/2436105594 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81681560e0 con 0x7f814c006230 2024-09-13T18:06:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.135+0000 7f815effd700 1 -- 172.21.15.40:0/2436105594 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814c008d80 con 0x7f814c0036c0 2024-09-13T18:06:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.135+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c0086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.135+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.135+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 msgr2=0x7f814c0086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:06:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.135+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 msgr2=0x7f814c0086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.135+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c0086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.135+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c0086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.135+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 msgr2=0x7f814c005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:06:56.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.135+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 msgr2=0x7f814c005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:56.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.135+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:56.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.135+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:56.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.339+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c0086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:56.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.339+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:56.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.339+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 msgr2=0x7f814c0086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:06:56.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.339+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 msgr2=0x7f814c0086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:56.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.339+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c0086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:56.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.339+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c0086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:06:56.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.339+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 msgr2=0x7f814c005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:06:56.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.339+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 msgr2=0x7f814c005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:56.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.339+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:56.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.339+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:06:56.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.743+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c0086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:56.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.743+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:56.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.743+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 msgr2=0x7f814c0086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:06:56.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.743+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 msgr2=0x7f814c0086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:56.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.743+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c0086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:56.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.743+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c0086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:06:56.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.743+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 msgr2=0x7f814c005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:06:56.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.743+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 msgr2=0x7f814c005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:56.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.743+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:56.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:56.743+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:06:57.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:57.547+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c0086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:57.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:57.547+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:57.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:57.547+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 msgr2=0x7f814c0086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:06:57.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:57.547+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 msgr2=0x7f814c0086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:57.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:57.547+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c0086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:57.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:57.547+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c0086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:06:57.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:57.547+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 msgr2=0x7f814c005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:06:57.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:57.547+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 msgr2=0x7f814c005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:57.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:57.547+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:57.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:57.547+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:06:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.135+0000 7f815effd700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 msgr2=0x7f814c005b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:06:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.135+0000 7f815effd700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.135+0000 7f815effd700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 msgr2=0x7f814c0086f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:06:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.135+0000 7f815effd700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c0086f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:06:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.135+0000 7f815effd700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c008670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.135+0000 7f816db8d700 1 -- 172.21.15.40:0/2436105594 reap_dead start 2024-09-13T18:06:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.135+0000 7f815effd700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c00c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:06:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.135+0000 7f815effd700 1 -- 172.21.15.40:0/2436105594 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81681563c0 con 0x7f81681500c0 2024-09-13T18:06:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.135+0000 7f815effd700 1 -- 172.21.15.40:0/2436105594 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81681560e0 con 0x7f814c006230 2024-09-13T18:06:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.135+0000 7f815effd700 1 -- 172.21.15.40:0/2436105594 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814c008d80 con 0x7f814c0036c0 2024-09-13T18:06:59.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.135+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c008670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:59.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.135+0000 7f816db8d700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c00c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:59.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.135+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 msgr2=0x7f814c008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:06:59.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.135+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 msgr2=0x7f814c008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:59.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.135+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c008670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:59.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.135+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:59.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.135+0000 7f816db8d700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 msgr2=0x7f814c00c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:06:59.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.135+0000 7f816db8d700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 msgr2=0x7f814c00c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:59.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.135+0000 7f816db8d700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c00c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:59.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.135+0000 7f816db8d700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c00c8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:06:59.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.339+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c008670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:59.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.339+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 msgr2=0x7f814c008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:06:59.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.339+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 msgr2=0x7f814c008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:59.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.339+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c008670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:59.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.339+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:06:59.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.339+0000 7f816db8d700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c00c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:59.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.339+0000 7f816db8d700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 msgr2=0x7f814c00c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:06:59.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.339+0000 7f816db8d700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 msgr2=0x7f814c00c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:59.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.339+0000 7f816db8d700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c00c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:59.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.339+0000 7f816db8d700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c00c8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:06:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.743+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c008670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.743+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 msgr2=0x7f814c008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:06:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.743+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 msgr2=0x7f814c008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.743+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c008670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.743+0000 7f816db8d700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c00c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:06:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.743+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:06:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.743+0000 7f816db8d700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 msgr2=0x7f814c00c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:06:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.743+0000 7f816db8d700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 msgr2=0x7f814c00c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:06:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.743+0000 7f816db8d700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c00c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:06:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:06:59.743+0000 7f816db8d700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c00c8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:07:00.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:00.547+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c008670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:00.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:00.547+0000 7f816db8d700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c00c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:00.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:00.547+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 msgr2=0x7f814c008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:07:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:00.547+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 msgr2=0x7f814c008670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:07:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:00.547+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c008670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:07:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:00.547+0000 7f816db8d700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 msgr2=0x7f814c00c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:07:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:00.547+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:07:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:00.547+0000 7f816db8d700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 msgr2=0x7f814c00c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:07:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:00.547+0000 7f816db8d700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c00c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:07:00.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:00.547+0000 7f816db8d700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c00c8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:07:02.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.135+0000 7f815effd700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 msgr2=0x7f814c00c8b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:07:02.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.135+0000 7f815effd700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f814c0036c0 0x7f814c00c8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:02.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.135+0000 7f815effd700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 msgr2=0x7f814c008670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:07:02.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.135+0000 7f815effd700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.40:3300/0 conn(0x7f814c006230 0x7f814c008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:02.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.135+0000 7f815effd700 1 -- 172.21.15.40:0/2436105594 >> v1:172.21.15.40:6789/0 conn(0x7f81681500c0 legacy=0x7f814c00b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:02.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.135+0000 7f815effd700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f814c010900 0x7f814c012cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:02.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.135+0000 7f815effd700 1 -- 172.21.15.40:0/2436105594 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814c005c50 con 0x7f814c00cdf0 2024-09-13T18:07:02.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.135+0000 7f815effd700 1 -- 172.21.15.40:0/2436105594 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81681560e0 con 0x7f814c010900 2024-09-13T18:07:02.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.135+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f814c010900 0x7f814c012cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:02.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.135+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f814c010900 msgr2=0x7f814c012cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:07:02.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.135+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f814c010900 msgr2=0x7f814c012cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:07:02.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.135+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f814c010900 0x7f814c012cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:07:02.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.135+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f814c010900 0x7f814c012cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:07:02.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.339+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f814c010900 0x7f814c012cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:02.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.339+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f814c010900 msgr2=0x7f814c012cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:07:02.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.339+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f814c010900 msgr2=0x7f814c012cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:07:02.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.339+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f814c010900 0x7f814c012cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:07:02.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.339+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f814c010900 0x7f814c012cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:07:02.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.743+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f814c010900 0x7f814c012cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:02.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.743+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f814c010900 msgr2=0x7f814c012cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:07:02.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.743+0000 7f816cb8b700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f814c010900 msgr2=0x7f814c012cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:07:02.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.743+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f814c010900 0x7f814c012cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:07:02.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:02.743+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f814c010900 0x7f814c012cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:07:03.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f815ffff700 1 -- 172.21.15.40:0/2436105594 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 91963558 0 0) 0x7f814c005c50 con 0x7f814c00cdf0 2024-09-13T18:07:03.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f815ffff700 1 -- 172.21.15.40:0/2436105594 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81540035d0 con 0x7f814c00cdf0 2024-09-13T18:07:03.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f815ffff700 1 -- 172.21.15.40:0/2436105594 <== mon.1 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1498368896 0 0) 0x7f81540035d0 con 0x7f814c00cdf0 2024-09-13T18:07:03.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f815ffff700 1 -- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f814c010900 msgr2=0x7f814c012cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:07:03.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f815ffff700 1 --2- 172.21.15.40:0/2436105594 >> v2:172.21.15.43:3300/0 conn(0x7f814c010900 0x7f814c012cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:03.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f815ffff700 1 -- 172.21.15.40:0/2436105594 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c0143b0 con 0x7f814c00cdf0 2024-09-13T18:07:03.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f815ffff700 1 -- 172.21.15.40:0/2436105594 <== mon.1 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f81640014d0 con 0x7f814c00cdf0 2024-09-13T18:07:03.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f815ffff700 1 -- 172.21.15.40:0/2436105594 >> v1:172.21.15.43:6789/0 conn(0x7f814c00cdf0 legacy=0x7f814c00f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:03.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f816db8d700 1 -- 172.21.15.40:0/2436105594 reap_dead start 2024-09-13T18:07:03.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f815ffff700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8154003a20 0x7f8154003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:03.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f815ffff700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8154004870 0x7f8154004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:03.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f815ffff700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8154005280 0x7f815400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:03.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f815ffff700 1 -- 172.21.15.40:0/2436105594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81681560e0 con 0x7f8154003a20 2024-09-13T18:07:03.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f815ffff700 1 -- 172.21.15.40:0/2436105594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815400c1b0 con 0x7f8154005280 2024-09-13T18:07:03.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f815ffff700 1 -- 172.21.15.40:0/2436105594 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815400c370 con 0x7f8154004870 2024-09-13T18:07:03.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8154003a20 0x7f8154003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:03.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f816cb8b700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8154004870 0x7f8154004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:03.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f816db8d700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8154005280 0x7f815400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:03.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8154004870 msgr2=0x7f8154004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:03.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8154004870 0x7f8154004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:03.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.095+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8154005280 msgr2=0x7f815400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:03.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.099+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8154005280 0x7f815400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:03.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.099+0000 7f816d38c700 1 -- 172.21.15.40:0/2436105594 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81540035d0 con 0x7f8154003a20 2024-09-13T18:07:03.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.099+0000 7f816d38c700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8154003a20 0x7f8154003e20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8164009010 tx=0x7f81540058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3fa433916e8e3c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:03.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.099+0000 7f815ffff700 1 -- 172.21.15.40:0/2436105594 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816401e070 con 0x7f8154003a20 2024-09-13T18:07:03.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.099+0000 7f815ffff700 1 -- 172.21.15.40:0/2436105594 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8164007530 con 0x7f8154003a20 2024-09-13T18:07:03.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.099+0000 7f815ffff700 1 -- 172.21.15.40:0/2436105594 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81640078a0 con 0x7f8154003a20 2024-09-13T18:07:03.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.099+0000 7f816e38e700 1 -- 172.21.15.40:0/2436105594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8154003a20 msgr2=0x7f8154003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:03.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.099+0000 7f816e38e700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8154003a20 0x7f8154003e20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8164009010 tx=0x7f81540058a0 comp rx=0 tx=0).stop 2024-09-13T18:07:03.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.099+0000 7f816e38e700 1 -- 172.21.15.40:0/2436105594 shutdown_connections 2024-09-13T18:07:03.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.099+0000 7f816e38e700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8154005280 0x7f815400ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:03.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.099+0000 7f816e38e700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8154004870 0x7f8154004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:03.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.099+0000 7f816e38e700 1 --2- 172.21.15.40:0/2436105594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8154003a20 0x7f8154003e20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:03.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.099+0000 7f816e38e700 1 -- 172.21.15.40:0/2436105594 >> 172.21.15.40:0/2436105594 conn(0x7f81680cabc0 msgr2=0x7f81680cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:03.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.099+0000 7f816e38e700 1 -- 172.21.15.40:0/2436105594 shutdown_connections 2024-09-13T18:07:03.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.099+0000 7f816e38e700 1 -- 172.21.15.40:0/2436105594 wait complete. 2024-09-13T18:07:03.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816e38e700 1 Processor -- start 2024-09-13T18:07:03.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816e38e700 1 -- start start 2024-09-13T18:07:03.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816e38e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8154003a20 0x7f8168154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:03.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816e38e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8154004870 0x7f8168154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:03.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816e38e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8154005280 0x7f8168149810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:03.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816e38e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81681ef770 con 0x7f8154004870 2024-09-13T18:07:03.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816e38e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81681eee60 con 0x7f8154005280 2024-09-13T18:07:03.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816e38e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81681f2e80 con 0x7f8154003a20 2024-09-13T18:07:03.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816cb8b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8154004870 0x7f8168154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:03.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816d38c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8154003a20 0x7f8168154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:03.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816db8d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8154005280 0x7f8168149810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:03.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816cb8b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8154004870 0x7f8168154d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36462/0 (socket says 172.21.15.40:36462) 2024-09-13T18:07:03.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816cb8b700 1 -- 172.21.15.40:0/4270650998 learned_addr learned my addr 172.21.15.40:0/4270650998 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:03.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816cb8b700 1 -- 172.21.15.40:0/4270650998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8154004870 msgr2=0x7f8168154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:07:03.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816cb8b700 1 -- 172.21.15.40:0/4270650998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8154004870 msgr2=0x7f8168154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:07:03.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816cb8b700 1 --2- 172.21.15.40:0/4270650998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8154004870 0x7f8168154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:07:03.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816cb8b700 1 --2- 172.21.15.40:0/4270650998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8154004870 0x7f8168154d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:07:03.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816d38c700 1 -- 172.21.15.40:0/4270650998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8154005280 msgr2=0x7f8168149810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:03.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816d38c700 1 --2- 172.21.15.40:0/4270650998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8154005280 0x7f8168149810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:03.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816d38c700 1 -- 172.21.15.40:0/4270650998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8154004870 msgr2=0x7f8168154d10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:07:03.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816d38c700 1 --2- 172.21.15.40:0/4270650998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8154004870 0x7f8168154d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:03.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.103+0000 7f816d38c700 1 -- 172.21.15.40:0/4270650998 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81681f3160 con 0x7f8154003a20 2024-09-13T18:07:03.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.107+0000 7f816d38c700 1 --2- 172.21.15.40:0/4270650998 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8154003a20 0x7f8168154140 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8154005850 tx=0x7f81640011d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:03.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.107+0000 7f815dffb700 1 -- 172.21.15.40:0/4270650998 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81640078b0 con 0x7f8154003a20 2024-09-13T18:07:03.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.107+0000 7f815dffb700 1 -- 172.21.15.40:0/4270650998 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81640014d0 con 0x7f8154003a20 2024-09-13T18:07:03.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.107+0000 7f816e38e700 1 -- 172.21.15.40:0/4270650998 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81681ef120 con 0x7f8154003a20 2024-09-13T18:07:03.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.107+0000 7f815dffb700 1 -- 172.21.15.40:0/4270650998 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8164007b30 con 0x7f8154003a20 2024-09-13T18:07:03.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.107+0000 7f816e38e700 1 -- 172.21.15.40:0/4270650998 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81680c28e0 con 0x7f8154003a20 2024-09-13T18:07:03.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.107+0000 7f815dffb700 1 -- 172.21.15.40:0/4270650998 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8164004fa0 con 0x7f8154003a20 2024-09-13T18:07:03.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.107+0000 7f816e38e700 1 -- 172.21.15.40:0/4270650998 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81680cb6e0 con 0x7f8154003a20 2024-09-13T18:07:03.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.107+0000 7f815dffb700 1 --2- 172.21.15.40:0/4270650998 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f814c0679c0 0x7f814c069e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:03.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.107+0000 7f815dffb700 1 -- 172.21.15.40:0/4270650998 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8164086aa0 con 0x7f8154003a20 2024-09-13T18:07:03.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.107+0000 7f816cb8b700 1 --2- 172.21.15.40:0/4270650998 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f814c0679c0 0x7f814c069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:03.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.111+0000 7f816cb8b700 1 --2- 172.21.15.40:0/4270650998 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f814c0679c0 0x7f814c069e80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f81681537f0 tx=0x7f815800e710 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:03.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.115+0000 7f815dffb700 1 -- 172.21.15.40:0/4270650998 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f81640574f0 con 0x7f8154003a20 2024-09-13T18:07:03.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:03.379+0000 7f816e38e700 1 -- 172.21.15.40:0/4270650998 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f81680cb6e0 con 0x7f8154003a20 2024-09-13T18:07:04.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.847+0000 7f815dffb700 1 -- 172.21.15.40:0/4270650998 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f8164057080 con 0x7f8154003a20 2024-09-13T18:07:04.857 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":140,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:07:04.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.851+0000 7f816e38e700 1 -- 172.21.15.40:0/4270650998 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f814c0679c0 msgr2=0x7f814c069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:04.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.851+0000 7f816e38e700 1 --2- 172.21.15.40:0/4270650998 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f814c0679c0 0x7f814c069e80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f81681537f0 tx=0x7f815800e710 comp rx=0 tx=0).stop 2024-09-13T18:07:04.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.851+0000 7f816e38e700 1 -- 172.21.15.40:0/4270650998 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8154003a20 msgr2=0x7f8168154140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:04.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.851+0000 7f816e38e700 1 --2- 172.21.15.40:0/4270650998 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8154003a20 0x7f8168154140 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8154005850 tx=0x7f81640011d0 comp rx=0 tx=0).stop 2024-09-13T18:07:04.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.851+0000 7f816e38e700 1 -- 172.21.15.40:0/4270650998 shutdown_connections 2024-09-13T18:07:04.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.851+0000 7f816e38e700 1 --2- 172.21.15.40:0/4270650998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8154005280 0x7f8168149810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:04.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.851+0000 7f816e38e700 1 --2- 172.21.15.40:0/4270650998 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8154004870 0x7f8168154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:04.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.851+0000 7f816e38e700 1 --2- 172.21.15.40:0/4270650998 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f814c0679c0 0x7f814c069e80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:04.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.851+0000 7f816e38e700 1 --2- 172.21.15.40:0/4270650998 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8154003a20 0x7f8168154140 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:04.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.851+0000 7f816e38e700 1 -- 172.21.15.40:0/4270650998 >> 172.21.15.40:0/4270650998 conn(0x7f81680cabc0 msgr2=0x7f81680c6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:04.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.851+0000 7f816e38e700 1 -- 172.21.15.40:0/4270650998 shutdown_connections 2024-09-13T18:07:04.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.851+0000 7f816e38e700 1 -- 172.21.15.40:0/4270650998 wait complete. 2024-09-13T18:07:04.877 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:07:04.877 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:07:04.877 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:07:04.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.983+0000 7f1b67fff700 1 Processor -- start 2024-09-13T18:07:04.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.983+0000 7f1b67fff700 1 -- start start 2024-09-13T18:07:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b67fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1b6814fe10 0x7f1b680c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b67fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b681501f0 0x7f1b680c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b67fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b680c75b0 con 0x7f1b680663b0 2024-09-13T18:07:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b67fff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b68159fe0 con 0x7f1b6814fe10 2024-09-13T18:07:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b67fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b680c7ae0 con 0x7f1b681501f0 2024-09-13T18:07:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b66ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1b6814fe10 0x7f1b680c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b677fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1b680663b0 0x7f1b6815d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56780/0 (socket says 172.21.15.40:56780) 2024-09-13T18:07:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b677fe700 1 -- 172.21.15.40:0/1180544943 learned_addr learned my addr 172.21.15.40:0/1180544943 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:07:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b667fc700 1 --2- 172.21.15.40:0/1180544943 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b681501f0 0x7f1b680c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b65ffb700 1 -- 172.21.15.40:0/1180544943 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1853240222 0 0) 0x7f1b680c75b0 con 0x7f1b680663b0 2024-09-13T18:07:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b65ffb700 1 -- 172.21.15.40:0/1180544943 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b500035d0 con 0x7f1b680663b0 2024-09-13T18:07:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b66ffd700 1 -- 172.21.15.40:0/1180544943 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b681501f0 msgr2=0x7f1b680c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:04.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b66ffd700 1 --2- 172.21.15.40:0/1180544943 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b681501f0 0x7f1b680c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b66ffd700 1 -- 172.21.15.40:0/1180544943 >> v1:172.21.15.40:6789/0 conn(0x7f1b680663b0 legacy=0x7f1b6815d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b66ffd700 1 -- 172.21.15.40:0/1180544943 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6815a280 con 0x7f1b6814fe10 2024-09-13T18:07:04.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b667fc700 1 --2- 172.21.15.40:0/1180544943 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b681501f0 0x7f1b680c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:07:04.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b66ffd700 1 --2- 172.21.15.40:0/1180544943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b6814fe10 0x7f1b680c64d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009850 tx=0x7f1b58009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e999432af5e938 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:04.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b65ffb700 1 -- 172.21.15.40:0/1180544943 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b58011830 con 0x7f1b6814fe10 2024-09-13T18:07:04.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b65ffb700 1 -- 172.21.15.40:0/1180544943 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b58011990 con 0x7f1b6814fe10 2024-09-13T18:07:04.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b65ffb700 1 -- 172.21.15.40:0/1180544943 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b58011d00 con 0x7f1b6814fe10 2024-09-13T18:07:04.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b67fff700 1 -- 172.21.15.40:0/1180544943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b6814fe10 msgr2=0x7f1b680c64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:04.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b67fff700 1 --2- 172.21.15.40:0/1180544943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b6814fe10 0x7f1b680c64d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009850 tx=0x7f1b58009b60 comp rx=0 tx=0).stop 2024-09-13T18:07:04.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b67fff700 1 -- 172.21.15.40:0/1180544943 shutdown_connections 2024-09-13T18:07:04.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b67fff700 1 --2- 172.21.15.40:0/1180544943 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b681501f0 0x7f1b680c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:04.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b67fff700 1 --2- 172.21.15.40:0/1180544943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b6814fe10 0x7f1b680c64d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:04.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b67fff700 1 -- 172.21.15.40:0/1180544943 >> 172.21.15.40:0/1180544943 conn(0x7f1b6806a390 msgr2=0x7f1b6806a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:04.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b67fff700 1 -- 172.21.15.40:0/1180544943 shutdown_connections 2024-09-13T18:07:04.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.987+0000 7f1b67fff700 1 -- 172.21.15.40:0/1180544943 wait complete. 2024-09-13T18:07:04.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b67fff700 1 Processor -- start 2024-09-13T18:07:04.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b67fff700 1 -- start start 2024-09-13T18:07:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b67fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b680663b0 0x7f1b68156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b67fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b6814fe10 0x7f1b681f2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b67fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b6815b940 0x7f1b681ef6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b67fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b681ef2a0 con 0x7f1b6815b940 2024-09-13T18:07:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b67fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b681eef00 con 0x7f1b680663b0 2024-09-13T18:07:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b67fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b681f3250 con 0x7f1b6814fe10 2024-09-13T18:07:05.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b677fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b6815b940 0x7f1b681ef6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b677fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b6815b940 0x7f1b681ef6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36484/0 (socket says 172.21.15.40:36484) 2024-09-13T18:07:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b677fe700 1 -- 172.21.15.40:0/998964113 learned_addr learned my addr 172.21.15.40:0/998964113 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b667fc700 1 --2- 172.21.15.40:0/998964113 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b6814fe10 0x7f1b681f2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b66ffd700 1 --2- 172.21.15.40:0/998964113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b680663b0 0x7f1b68156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b677fe700 1 -- 172.21.15.40:0/998964113 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b6814fe10 msgr2=0x7f1b681f2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b677fe700 1 --2- 172.21.15.40:0/998964113 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b6814fe10 0x7f1b681f2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b677fe700 1 -- 172.21.15.40:0/998964113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b680663b0 msgr2=0x7f1b68156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b677fe700 1 --2- 172.21.15.40:0/998964113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b680663b0 0x7f1b68156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b677fe700 1 -- 172.21.15.40:0/998964113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b580094b0 con 0x7f1b6815b940 2024-09-13T18:07:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b677fe700 1 --2- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b6815b940 0x7f1b681ef6c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000caa0 tx=0x7f1b6000cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b4ffff700 1 -- 172.21.15.40:0/998964113 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b60007480 con 0x7f1b6815b940 2024-09-13T18:07:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b4ffff700 1 -- 172.21.15.40:0/998964113 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b600075e0 con 0x7f1b6815b940 2024-09-13T18:07:05.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b67fff700 1 -- 172.21.15.40:0/998964113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b68151e60 con 0x7f1b6815b940 2024-09-13T18:07:05.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b4ffff700 1 -- 172.21.15.40:0/998964113 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b60007970 con 0x7f1b6815b940 2024-09-13T18:07:05.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b67fff700 1 -- 172.21.15.40:0/998964113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b68152410 con 0x7f1b6815b940 2024-09-13T18:07:05.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b67fff700 1 --2- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b681526f0 0x7f1b6814e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b67fff700 1 -- 172.21.15.40:0/998964113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b68067640 con 0x7f1b681526f0 2024-09-13T18:07:05.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.991+0000 7f1b66ffd700 1 --2- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b681526f0 0x7f1b6814e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.995+0000 7f1b66ffd700 1 -- 172.21.15.40:0/998964113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b6815cb50 con 0x7f1b681526f0 2024-09-13T18:07:05.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.995+0000 7f1b66ffd700 1 --2- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b681526f0 0x7f1b6814e790 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009850 tx=0x7f1b58004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:05.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.995+0000 7f1b4ffff700 1 -- 172.21.15.40:0/998964113 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1b600113a0 con 0x7f1b6815b940 2024-09-13T18:07:05.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.995+0000 7f1b4ffff700 1 --2- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b54066700 0x7f1b54068bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.995+0000 7f1b4ffff700 1 -- 172.21.15.40:0/998964113 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1b60083d90 con 0x7f1b6815b940 2024-09-13T18:07:05.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.995+0000 7f1b667fc700 1 --2- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b54066700 0x7f1b54068bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.995+0000 7f1b4ffff700 1 -- 172.21.15.40:0/998964113 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5801f070 con 0x7f1b681526f0 2024-09-13T18:07:05.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.995+0000 7f1b4ffff700 1 -- 172.21.15.40:0/998964113 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f1b58011cc0 con 0x7f1b681526f0 2024-09-13T18:07:05.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.995+0000 7f1b4ffff700 1 -- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b681526f0 msgr2=0x7f1b6814e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.995+0000 7f1b4ffff700 1 --2- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b681526f0 0x7f1b6814e790 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009850 tx=0x7f1b58004770 comp rx=0 tx=0).stop 2024-09-13T18:07:05.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.995+0000 7f1b4ffff700 1 -- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b681526f0 msgr2=0x7f1b6814e790 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:05.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.995+0000 7f1b4ffff700 1 --2- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b681526f0 0x7f1b6814e790 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:04.995+0000 7f1b667fc700 1 --2- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b54066700 0x7f1b54068bc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1b68151df0 tx=0x7f1b5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:05.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b67fff700 1 --2- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b6814ecd0 0x7f1b680c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b67fff700 1 -- 172.21.15.40:0/998964113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b681ef0c0 con 0x7f1b6814ecd0 2024-09-13T18:07:05.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b66ffd700 1 --2- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b6814ecd0 0x7f1b680c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b66ffd700 1 -- 172.21.15.40:0/998964113 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f1b68152ad0 con 0x7f1b6814ecd0 2024-09-13T18:07:05.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b66ffd700 1 --2- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b6814ecd0 0x7f1b680c39f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1b58000c00 tx=0x7f1b58011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:05.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b4ffff700 1 -- 172.21.15.40:0/998964113 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5802b2b0 con 0x7f1b6814ecd0 2024-09-13T18:07:05.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b4ffff700 1 -- 172.21.15.40:0/998964113 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4452 (secure 0 0 0) 0x7f1b5802b620 con 0x7f1b6814ecd0 2024-09-13T18:07:05.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b4ffff700 1 -- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b6814ecd0 msgr2=0x7f1b680c39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b4ffff700 1 --2- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b6814ecd0 0x7f1b680c39f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1b58000c00 tx=0x7f1b58011000 comp rx=0 tx=0).stop 2024-09-13T18:07:05.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b4ffff700 1 -- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b6814ecd0 msgr2=0x7f1b680c39f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:05.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b4ffff700 1 --2- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b6814ecd0 0x7f1b680c39f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.017 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:07:05.017 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:07:05.017 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:07:05.017 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:07:05.017 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 140, 2024-09-13T18:07:05.018 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:07:05.018 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:07:05.018 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:07:05.018 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:07:05.018 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:05.018 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:07:05.018 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:05.018 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:07:05.018 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:07:05.018 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:05.018 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:05.018 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:05.018 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:05.019 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:05.019 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:05.019 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:05.019 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:05.019 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:05.019 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:05.019 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:05.019 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:07:05.019 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:07:05.019 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:05.019 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:05.019 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:05.019 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:05.019 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:05.020 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:05.020 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:05.020 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:05.020 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:05.020 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:05.020 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:05.020 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.020 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:07:05.020 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:07:05.020 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:07:05.020 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:07:05.020 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:07:05.020 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:07:05.020 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:05.021 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:05.021 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:07:05.021 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:07:05.021 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:07:05.021 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:07:05.021 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:07:05.021 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:07:05.021 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:07:05.021 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:05.021 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:07:05.021 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:05.021 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:05.022 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:05.022 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:05.022 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:05.022 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:05.022 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:05.022 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:05.022 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:05.022 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:05.022 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:05.022 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:07:05.022 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.022 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:07:05.022 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.022 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:07:05.023 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:07:05.023 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:05.023 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:05.023 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.023 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:05.023 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:07:05.023 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:05.023 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.023 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.023 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:05.023 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:07:05.023 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:05.023 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.023 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:05.024 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.024 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:07:05.024 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:07:05.024 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:05.024 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:05.024 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:05.024 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.024 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.024 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:07:05.024 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:07:05.024 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:05.024 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:05.024 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.025 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:05.025 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:07:05.025 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:05.025 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.025 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.025 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:05.025 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:07:05.025 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:05.025 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.025 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:05.025 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.025 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:07:05.025 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:07:05.026 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:05.026 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:05.026 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:05.026 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.026 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.026 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:07:05.026 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:07:05.026 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:05.026 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:05.026 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.026 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:05.026 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:07:05.027 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:05.027 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.027 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.027 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:05.027 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:07:05.027 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:05.027 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.027 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:05.027 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.027 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:07:05.027 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:07:05.027 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:05.027 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:05.028 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:05.028 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.028 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:05.028 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.028 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:07:05.028 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:07:05.028 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.028 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:05.028 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:05.028 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:07:05.028 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.028 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:05.028 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:07:05.029 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.029 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:05.029 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:05.029 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:07:05.029 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.029 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:05.029 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.029 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:07:05.029 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:07:05.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b67fff700 1 -- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b54066700 msgr2=0x7f1b54068bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b67fff700 1 --2- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b54066700 0x7f1b54068bc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1b68151df0 tx=0x7f1b5c008040 comp rx=0 tx=0).stop 2024-09-13T18:07:05.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b67fff700 1 -- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b6815b940 msgr2=0x7f1b681ef6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b67fff700 1 --2- 172.21.15.40:0/998964113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b6815b940 0x7f1b681ef6c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000caa0 tx=0x7f1b6000cad0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b677fe700 1 -- 172.21.15.40:0/998964113 reap_dead start 2024-09-13T18:07:05.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b677fe700 1 -- 172.21.15.40:0/998964113 reap_dead start 2024-09-13T18:07:05.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b67fff700 1 -- 172.21.15.40:0/998964113 shutdown_connections 2024-09-13T18:07:05.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b67fff700 1 -- 172.21.15.40:0/998964113 >> 172.21.15.40:0/998964113 conn(0x7f1b6806a390 msgr2=0x7f1b68071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:05.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b67fff700 1 -- 172.21.15.40:0/998964113 shutdown_connections 2024-09-13T18:07:05.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.007+0000 7f1b67fff700 1 -- 172.21.15.40:0/998964113 wait complete. 2024-09-13T18:07:05.031 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:07:05.032 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:07:05.032 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:07:05.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f1791a10700 1 Processor -- start 2024-09-13T18:07:05.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f1791a10700 1 -- start start 2024-09-13T18:07:05.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f1791a10700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f17840d58d0 0x7f17840077d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f1791a10700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f17840461f0 0x7f1784046740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f1791a10700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1784033140 0x7f17840335d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f1791a10700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1784006490 con 0x7f17840d58d0 2024-09-13T18:07:05.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f1791a10700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1784036020 con 0x7f17840461f0 2024-09-13T18:07:05.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f1791a10700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17840459e0 con 0x7f1784033140 2024-09-13T18:07:05.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f1790a0e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f17840d58d0 0x7f17840077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f1790a0e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f17840d58d0 0x7f17840077d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36506/0 (socket says 172.21.15.40:36506) 2024-09-13T18:07:05.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f1790a0e700 1 -- 172.21.15.40:0/3782832035 learned_addr learned my addr 172.21.15.40:0/3782832035 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:05.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f178bfff700 1 --2- 172.21.15.40:0/3782832035 >> v2:172.21.15.43:3300/0 conn(0x7f17840461f0 0x7f1784046740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f179120f700 1 --2- 172.21.15.40:0/3782832035 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1784033140 0x7f17840335d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f1790a0e700 1 -- 172.21.15.40:0/3782832035 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1784033140 msgr2=0x7f17840335d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:05.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f1790a0e700 1 --2- 172.21.15.40:0/3782832035 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1784033140 0x7f17840335d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f1790a0e700 1 -- 172.21.15.40:0/3782832035 >> v2:172.21.15.43:3300/0 conn(0x7f17840461f0 msgr2=0x7f1784046740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:05.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f1790a0e700 1 --2- 172.21.15.40:0/3782832035 >> v2:172.21.15.43:3300/0 conn(0x7f17840461f0 0x7f1784046740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f1790a0e700 1 -- 172.21.15.40:0/3782832035 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1784045f80 con 0x7f17840d58d0 2024-09-13T18:07:05.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f1790a0e700 1 --2- 172.21.15.40:0/3782832035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17840d58d0 0x7f17840077d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f178404cea0 tx=0x7f1778009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b69e9d11f48434 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:05.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f178b7fe700 1 -- 172.21.15.40:0/3782832035 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177800b8a0 con 0x7f17840d58d0 2024-09-13T18:07:05.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f178b7fe700 1 -- 172.21.15.40:0/3782832035 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177800ba00 con 0x7f17840d58d0 2024-09-13T18:07:05.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.143+0000 7f178b7fe700 1 -- 172.21.15.40:0/3782832035 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177800bd90 con 0x7f17840d58d0 2024-09-13T18:07:05.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.147+0000 7f1791a10700 1 -- 172.21.15.40:0/3782832035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17840d58d0 msgr2=0x7f17840077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.147+0000 7f1791a10700 1 --2- 172.21.15.40:0/3782832035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17840d58d0 0x7f17840077d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f178404cea0 tx=0x7f1778009bf0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.147+0000 7f1791a10700 1 -- 172.21.15.40:0/3782832035 shutdown_connections 2024-09-13T18:07:05.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.147+0000 7f1791a10700 1 --2- 172.21.15.40:0/3782832035 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1784033140 0x7f17840335d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.147+0000 7f1791a10700 1 --2- 172.21.15.40:0/3782832035 >> v2:172.21.15.43:3300/0 conn(0x7f17840461f0 0x7f1784046740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.147+0000 7f1791a10700 1 --2- 172.21.15.40:0/3782832035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17840d58d0 0x7f17840077d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.147+0000 7f1791a10700 1 -- 172.21.15.40:0/3782832035 >> 172.21.15.40:0/3782832035 conn(0x7f1784050590 msgr2=0x7f17840509a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:05.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.147+0000 7f1791a10700 1 -- 172.21.15.40:0/3782832035 shutdown_connections 2024-09-13T18:07:05.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.147+0000 7f1791a10700 1 -- 172.21.15.40:0/3782832035 wait complete. 2024-09-13T18:07:05.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.147+0000 7f1791a10700 1 Processor -- start 2024-09-13T18:07:05.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.147+0000 7f1791a10700 1 -- start start 2024-09-13T18:07:05.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.147+0000 7f1791a10700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1784033140 0x7f17840d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.147+0000 7f1791a10700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17840461f0 0x7f17840c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.147+0000 7f1791a10700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f17840d58d0 0x7f17840d3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.147+0000 7f1791a10700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1784049500 con 0x7f1784033140 2024-09-13T18:07:05.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.147+0000 7f1791a10700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17840485b0 con 0x7f17840461f0 2024-09-13T18:07:05.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.147+0000 7f1791a10700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17840c7800 con 0x7f17840d58d0 2024-09-13T18:07:05.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f1790a0e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1784033140 0x7f17840d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f1790a0e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1784033140 0x7f17840d4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36514/0 (socket says 172.21.15.40:36514) 2024-09-13T18:07:05.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f1790a0e700 1 -- 172.21.15.40:0/749583989 learned_addr learned my addr 172.21.15.40:0/749583989 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:05.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f178bfff700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17840461f0 0x7f17840c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f179120f700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f17840d58d0 0x7f17840d3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f1790a0e700 1 -- 172.21.15.40:0/749583989 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f17840d58d0 msgr2=0x7f17840d3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f1790a0e700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f17840d58d0 0x7f17840d3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f1790a0e700 1 -- 172.21.15.40:0/749583989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17840461f0 msgr2=0x7f17840c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f1790a0e700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17840461f0 0x7f17840c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f1790a0e700 1 -- 172.21.15.40:0/749583989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1778009570 con 0x7f1784033140 2024-09-13T18:07:05.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f1790a0e700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1784033140 0x7f17840d4cc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1778000c00 tx=0x7f1778004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:05.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f17897fa700 1 -- 172.21.15.40:0/749583989 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17780038c0 con 0x7f1784033140 2024-09-13T18:07:05.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f17897fa700 1 -- 172.21.15.40:0/749583989 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177801e070 con 0x7f1784033140 2024-09-13T18:07:05.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f17897fa700 1 -- 172.21.15.40:0/749583989 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1778003c80 con 0x7f1784033140 2024-09-13T18:07:05.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f1791a10700 1 -- 172.21.15.40:0/749583989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1784174240 con 0x7f1784033140 2024-09-13T18:07:05.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f1791a10700 1 -- 172.21.15.40:0/749583989 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1784048ae0 con 0x7f1784033140 2024-09-13T18:07:05.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f1791a10700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f178404a370 0x7f17840d39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f1791a10700 1 -- 172.21.15.40:0/749583989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1784008d40 con 0x7f178404a370 2024-09-13T18:07:05.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f178bfff700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f178404a370 0x7f17840d39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f17897fa700 1 -- 172.21.15.40:0/749583989 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1778011420 con 0x7f1784033140 2024-09-13T18:07:05.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f178bfff700 1 -- 172.21.15.40:0/749583989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1784005ed0 con 0x7f178404a370 2024-09-13T18:07:05.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f17897fa700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1770066730 0x7f1770068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.151+0000 7f178bfff700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f178404a370 0x7f17840d39a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f178404c6a0 tx=0x7f1780011960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:05.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.155+0000 7f17897fa700 1 -- 172.21.15.40:0/749583989 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1778020070 con 0x7f1784033140 2024-09-13T18:07:05.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.155+0000 7f179120f700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1770066730 0x7f1770068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.155+0000 7f17897fa700 1 -- 172.21.15.40:0/749583989 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17800045a0 con 0x7f178404a370 2024-09-13T18:07:05.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.155+0000 7f179120f700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1770066730 0x7f1770068bf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f178c0641b0 tx=0x7f178c08e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:05.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.155+0000 7f17897fa700 1 -- 172.21.15.40:0/749583989 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f17800036c0 con 0x7f178404a370 2024-09-13T18:07:05.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.155+0000 7f17897fa700 1 -- 172.21.15.40:0/749583989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f178404a370 msgr2=0x7f17840d39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.155+0000 7f17897fa700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f178404a370 0x7f17840d39a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f178404c6a0 tx=0x7f1780011960 comp rx=0 tx=0).stop 2024-09-13T18:07:05.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.155+0000 7f17897fa700 1 -- 172.21.15.40:0/749583989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f178404a370 msgr2=0x7f17840d39a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:05.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.155+0000 7f17897fa700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f178404a370 0x7f17840d39a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.167+0000 7f1791a10700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17841734f0 0x7f1784173900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.167+0000 7f1791a10700 1 -- 172.21.15.40:0/749583989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1784045ca0 con 0x7f17841734f0 2024-09-13T18:07:05.176 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.167+0000 7f178bfff700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17841734f0 0x7f1784173900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.167+0000 7f178bfff700 1 -- 172.21.15.40:0/749583989 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f17840061b0 con 0x7f17841734f0 2024-09-13T18:07:05.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.167+0000 7f178bfff700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17841734f0 0x7f1784173900 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1780004700 tx=0x7f17800047e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:05.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.167+0000 7f17897fa700 1 -- 172.21.15.40:0/749583989 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1780015780 con 0x7f17841734f0 2024-09-13T18:07:05.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.171+0000 7f17897fa700 1 -- 172.21.15.40:0/749583989 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4782 (secure 0 0 0) 0x7f17800158e0 con 0x7f17841734f0 2024-09-13T18:07:05.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.171+0000 7f17897fa700 1 -- 172.21.15.40:0/749583989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17841734f0 msgr2=0x7f1784173900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.171+0000 7f17897fa700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17841734f0 0x7f1784173900 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1780004700 tx=0x7f17800047e0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.171+0000 7f17897fa700 1 -- 172.21.15.40:0/749583989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17841734f0 msgr2=0x7f1784173900 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:05.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.171+0000 7f17897fa700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17841734f0 0x7f1784173900 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.180 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:07:05.180 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:07:05.180 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:07:05.180 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:07:05.180 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 140, 2024-09-13T18:07:05.180 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:07:05.180 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:07:05.180 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:07:05.180 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:07:05.180 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:05.180 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:07:05.180 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:05.180 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:07:05.180 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:07:05.181 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:05.181 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:05.181 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:05.181 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:05.181 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:05.181 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:05.181 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:05.181 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:05.181 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:05.181 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:05.181 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:05.181 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:07:05.181 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:07:05.182 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:05.182 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:05.182 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:05.182 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:05.182 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:05.182 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:05.182 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:05.182 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:05.182 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:05.182 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:05.182 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:05.182 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.182 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:07:05.182 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:07:05.183 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:07:05.183 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:07:05.183 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:07:05.183 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:07:05.183 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:05.183 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:05.183 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:07:05.183 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:07:05.183 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:07:05.183 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:07:05.183 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:07:05.183 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:07:05.183 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:07:05.183 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:05.183 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:07:05.184 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:05.184 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:05.184 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:05.184 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:05.184 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:05.184 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:05.184 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:05.184 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:05.184 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:05.184 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:05.184 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:05.184 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:07:05.184 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.185 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:07:05.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.185 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:07:05.185 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:07:05.185 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:05.185 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:05.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.185 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:05.185 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:07:05.185 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:05.185 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.185 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.185 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:05.185 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:07:05.185 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:05.186 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.186 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:05.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.186 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:07:05.186 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:07:05.186 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:05.186 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:05.186 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:05.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.186 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.186 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:07:05.186 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:07:05.186 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:05.187 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:05.187 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.187 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:05.187 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:07:05.187 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:05.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.187 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.187 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:05.187 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:07:05.187 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:05.187 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.187 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:05.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.187 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:07:05.187 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:07:05.188 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:05.188 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:05.188 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:05.188 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.188 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.188 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:07:05.188 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:07:05.188 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:05.188 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:05.188 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.188 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:05.188 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:07:05.188 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:05.188 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.189 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:05.189 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:07:05.189 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:05.189 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.189 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:05.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.189 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:07:05.189 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:07:05.189 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:05.189 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:05.189 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:05.190 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.190 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:05.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.190 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:07:05.190 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:07:05.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.190 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:05.190 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:05.190 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:07:05.190 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.190 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:05.190 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:07:05.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.190 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:05.191 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:05.191 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:07:05.191 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.191 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:05.191 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:07:05.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.191 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:05.191 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:05.191 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:07:05.191 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.191 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:05.191 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:07:05.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.191 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:05.191 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:05.192 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:07:05.192 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.192 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:05.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.192 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:07:05.192 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:07:05.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.171+0000 7f1791a10700 1 -- 172.21.15.40:0/749583989 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1770066730 msgr2=0x7f1770068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.171+0000 7f1791a10700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1770066730 0x7f1770068bf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f178c0641b0 tx=0x7f178c08e040 comp rx=0 tx=0).stop 2024-09-13T18:07:05.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.171+0000 7f1791a10700 1 -- 172.21.15.40:0/749583989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1784033140 msgr2=0x7f17840d4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.171+0000 7f1791a10700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1784033140 0x7f17840d4cc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1778000c00 tx=0x7f1778004770 comp rx=0 tx=0).stop 2024-09-13T18:07:05.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.171+0000 7f179120f700 1 -- 172.21.15.40:0/749583989 reap_dead start 2024-09-13T18:07:05.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.171+0000 7f1791a10700 1 -- 172.21.15.40:0/749583989 shutdown_connections 2024-09-13T18:07:05.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.171+0000 7f1791a10700 1 --2- 172.21.15.40:0/749583989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1784033140 0x7f17840d4cc0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.171+0000 7f1791a10700 1 -- 172.21.15.40:0/749583989 >> 172.21.15.40:0/749583989 conn(0x7f1784050590 msgr2=0x7f1784004e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:05.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.171+0000 7f1791a10700 1 -- 172.21.15.40:0/749583989 shutdown_connections 2024-09-13T18:07:05.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.171+0000 7f1791a10700 1 -- 172.21.15.40:0/749583989 wait complete. 2024-09-13T18:07:05.194 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:07:05.194 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:07:05.195 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:07:05.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.299+0000 7f2a1e319700 1 Processor -- start 2024-09-13T18:07:05.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.299+0000 7f2a1e319700 1 -- start start 2024-09-13T18:07:05.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.299+0000 7f2a1e319700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a1815ce10 0x7f2a180c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.299+0000 7f2a1e319700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a1806d940 con 0x7f2a1806d540 2024-09-13T18:07:05.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.299+0000 7f2a1e319700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a18071860 con 0x7f2a1815d1f0 2024-09-13T18:07:05.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.299+0000 7f2a1e319700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a180c5e50 con 0x7f2a1815ce10 2024-09-13T18:07:05.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.299+0000 7f2a1cb16700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2a1806d540 0x7f2a18070fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56784/0 (socket says 172.21.15.40:56784) 2024-09-13T18:07:05.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.299+0000 7f2a1cb16700 1 -- 172.21.15.40:0/4131199970 learned_addr learned my addr 172.21.15.40:0/4131199970 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:07:05.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.299+0000 7f2a1d317700 1 --2- 172.21.15.40:0/4131199970 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a1815ce10 0x7f2a180c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.299+0000 7f2a0ffff700 1 -- 172.21.15.40:0/4131199970 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1273106804 0 0) 0x7f2a1806d940 con 0x7f2a1806d540 2024-09-13T18:07:05.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.299+0000 7f2a0ffff700 1 -- 172.21.15.40:0/4131199970 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a040035d0 con 0x7f2a1806d540 2024-09-13T18:07:05.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.299+0000 7f2a1d317700 1 -- 172.21.15.40:0/4131199970 >> v1:172.21.15.43:6789/0 conn(0x7f2a1815d1f0 legacy=0x7f2a181533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.299+0000 7f2a0ffff700 1 -- 172.21.15.40:0/4131199970 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 317796815 0 0) 0x7f2a18071860 con 0x7f2a1815d1f0 2024-09-13T18:07:05.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.299+0000 7f2a1d317700 1 -- 172.21.15.40:0/4131199970 >> v1:172.21.15.40:6789/0 conn(0x7f2a1806d540 legacy=0x7f2a18070fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.299+0000 7f2a1d317700 1 -- 172.21.15.40:0/4131199970 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a180c6130 con 0x7f2a1815ce10 2024-09-13T18:07:05.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.299+0000 7f2a1d317700 1 --2- 172.21.15.40:0/4131199970 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a1815ce10 0x7f2a180c6ef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a14009880 tx=0x7f2a14009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b82fded5bce0b5fb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:05.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.299+0000 7f2a0ffff700 1 -- 172.21.15.40:0/4131199970 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a14011830 con 0x7f2a1815ce10 2024-09-13T18:07:05.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.299+0000 7f2a0ffff700 1 -- 172.21.15.40:0/4131199970 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a14011990 con 0x7f2a1815ce10 2024-09-13T18:07:05.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.299+0000 7f2a0ffff700 1 -- 172.21.15.40:0/4131199970 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a14011d20 con 0x7f2a1815ce10 2024-09-13T18:07:05.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.303+0000 7f2a1e319700 1 -- 172.21.15.40:0/4131199970 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a1815ce10 msgr2=0x7f2a180c6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.303+0000 7f2a1e319700 1 --2- 172.21.15.40:0/4131199970 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a1815ce10 0x7f2a180c6ef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a14009880 tx=0x7f2a14009b90 comp rx=0 tx=0).stop 2024-09-13T18:07:05.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.303+0000 7f2a1e319700 1 -- 172.21.15.40:0/4131199970 shutdown_connections 2024-09-13T18:07:05.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.303+0000 7f2a1e319700 1 --2- 172.21.15.40:0/4131199970 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a1815ce10 0x7f2a180c6ef0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.303+0000 7f2a1e319700 1 -- 172.21.15.40:0/4131199970 >> 172.21.15.40:0/4131199970 conn(0x7f2a180cb450 msgr2=0x7f2a180cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:05.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.303+0000 7f2a1e319700 1 -- 172.21.15.40:0/4131199970 shutdown_connections 2024-09-13T18:07:05.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.303+0000 7f2a1e319700 1 -- 172.21.15.40:0/4131199970 wait complete. 2024-09-13T18:07:05.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.303+0000 7f2a1e319700 1 Processor -- start 2024-09-13T18:07:05.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.303+0000 7f2a1e319700 1 -- start start 2024-09-13T18:07:05.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.303+0000 7f2a1e319700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a1806d540 0x7f2a18153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.303+0000 7f2a1e319700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a1815ce10 0x7f2a181efcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.303+0000 7f2a1e319700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a1815d1f0 0x7f2a180c4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.303+0000 7f2a1e319700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a18156620 con 0x7f2a1815ce10 2024-09-13T18:07:05.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.303+0000 7f2a1e319700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a181f0200 con 0x7f2a1815d1f0 2024-09-13T18:07:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.303+0000 7f2a1e319700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a18155470 con 0x7f2a1806d540 2024-09-13T18:07:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1cb16700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a1815ce10 0x7f2a181efcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1cb16700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a1815ce10 0x7f2a181efcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36524/0 (socket says 172.21.15.40:36524) 2024-09-13T18:07:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1cb16700 1 -- 172.21.15.40:0/2875736217 learned_addr learned my addr 172.21.15.40:0/2875736217 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1d317700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a1806d540 0x7f2a18153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1db18700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a1815d1f0 0x7f2a180c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1cb16700 1 -- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a1806d540 msgr2=0x7f2a18153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1cb16700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a1806d540 0x7f2a18153f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1cb16700 1 -- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a1815d1f0 msgr2=0x7f2a180c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1cb16700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a1815d1f0 0x7f2a180c4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1cb16700 1 -- 172.21.15.40:0/2875736217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a140094b0 con 0x7f2a1815ce10 2024-09-13T18:07:05.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1db18700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a1815d1f0 0x7f2a180c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:07:05.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1d317700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a1806d540 0x7f2a18153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:07:05.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1cb16700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a1815ce10 0x7f2a181efcc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800c500 tx=0x7f2a0800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:05.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a0dffb700 1 -- 172.21.15.40:0/2875736217 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a08005d90 con 0x7f2a1815ce10 2024-09-13T18:07:05.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a0dffb700 1 -- 172.21.15.40:0/2875736217 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a080070d0 con 0x7f2a1815ce10 2024-09-13T18:07:05.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a0dffb700 1 -- 172.21.15.40:0/2875736217 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a08007440 con 0x7f2a1815ce10 2024-09-13T18:07:05.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1e319700 1 -- 172.21.15.40:0/2875736217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a18154e60 con 0x7f2a1815ce10 2024-09-13T18:07:05.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1e319700 1 -- 172.21.15.40:0/2875736217 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a18154840 con 0x7f2a1815ce10 2024-09-13T18:07:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1e319700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a18149be0 0x7f2a181558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1e319700 1 -- 172.21.15.40:0/2875736217 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a180675d0 con 0x7f2a18149be0 2024-09-13T18:07:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1d317700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a18149be0 0x7f2a181558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a0dffb700 1 -- 172.21.15.40:0/2875736217 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2a080075a0 con 0x7f2a1815ce10 2024-09-13T18:07:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a0dffb700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f29fc066730 0x7f29fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a0dffb700 1 -- 172.21.15.40:0/2875736217 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2a08083830 con 0x7f2a1815ce10 2024-09-13T18:07:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1d317700 1 -- 172.21.15.40:0/2875736217 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2a1806ec30 con 0x7f2a18149be0 2024-09-13T18:07:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1db18700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f29fc066730 0x7f29fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.307+0000 7f2a1d317700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a18149be0 0x7f2a181558e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2a14009880 tx=0x7f2a140047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.311+0000 7f2a0dffb700 1 -- 172.21.15.40:0/2875736217 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1401a7b0 con 0x7f2a18149be0 2024-09-13T18:07:05.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.311+0000 7f2a1db18700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f29fc066730 0x7f29fc068bf0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2a100012e0 tx=0x7f2a10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:05.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.311+0000 7f2a0dffb700 1 -- 172.21.15.40:0/2875736217 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f2a1401a910 con 0x7f2a18149be0 2024-09-13T18:07:05.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.311+0000 7f2a0dffb700 1 -- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a18149be0 msgr2=0x7f2a181558e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.311+0000 7f2a0dffb700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a18149be0 0x7f2a181558e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2a14009880 tx=0x7f2a140047e0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.311+0000 7f2a0dffb700 1 -- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a18149be0 msgr2=0x7f2a181558e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:05.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.311+0000 7f2a0dffb700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a18149be0 0x7f2a181558e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.323+0000 7f2a1e319700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a18154ff0 0x7f2a18148b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.323+0000 7f2a1e319700 1 -- 172.21.15.40:0/2875736217 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1806c000 con 0x7f2a18154ff0 2024-09-13T18:07:05.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.323+0000 7f2a1d317700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a18154ff0 0x7f2a18148b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.323+0000 7f2a1d317700 1 -- 172.21.15.40:0/2875736217 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2a18153d90 con 0x7f2a18154ff0 2024-09-13T18:07:05.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.323+0000 7f2a1d317700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a18154ff0 0x7f2a18148b60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2a140097d0 tx=0x7f2a14004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:05.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.323+0000 7f2a0dffb700 1 -- 172.21.15.40:0/2875736217 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1402cae0 con 0x7f2a18154ff0 2024-09-13T18:07:05.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.323+0000 7f2a0dffb700 1 -- 172.21.15.40:0/2875736217 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7f2a1402c300 con 0x7f2a18154ff0 2024-09-13T18:07:05.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.323+0000 7f2a0dffb700 1 -- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a18154ff0 msgr2=0x7f2a18148b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.327+0000 7f2a0dffb700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a18154ff0 0x7f2a18148b60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2a140097d0 tx=0x7f2a14004770 comp rx=0 tx=0).stop 2024-09-13T18:07:05.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.327+0000 7f2a0dffb700 1 -- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a18154ff0 msgr2=0x7f2a18148b60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:05.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.327+0000 7f2a0dffb700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a18154ff0 0x7f2a18148b60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.335 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:07:05.335 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:07:05.335 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:07:05.335 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:07:05.336 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 140, 2024-09-13T18:07:05.336 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:07:05.336 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:07:05.336 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:07:05.336 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:07:05.336 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:05.336 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:07:05.336 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:05.336 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:07:05.336 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:07:05.336 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:05.336 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:05.336 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:05.336 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:05.337 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:05.337 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:05.337 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:05.337 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:05.337 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:05.337 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:05.337 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:05.337 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:07:05.337 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:07:05.337 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:05.337 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:05.337 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:05.337 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:05.337 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:05.338 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:05.338 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:05.338 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:05.338 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:05.338 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:05.338 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:05.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.338 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:07:05.338 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:07:05.338 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:07:05.338 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:07:05.338 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:07:05.338 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:07:05.338 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:05.339 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:05.339 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:07:05.339 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:07:05.339 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:07:05.339 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:07:05.339 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:07:05.339 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:07:05.339 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:07:05.339 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:05.339 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:07:05.339 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:05.339 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:05.339 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:05.339 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:05.340 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:05.340 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:05.340 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:05.340 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:05.340 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:05.340 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:05.340 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:05.340 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:07:05.340 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.340 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:07:05.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.340 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:07:05.340 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:07:05.341 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:05.341 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:05.341 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.341 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:05.341 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:07:05.341 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:05.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.341 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.341 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:05.341 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:07:05.341 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:05.341 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.341 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:05.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.342 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:07:05.342 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:07:05.342 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:05.342 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:05.342 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:05.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.342 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:07:05.342 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:07:05.342 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:05.342 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:05.342 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.342 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:05.343 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:07:05.343 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:05.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.343 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.343 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:05.343 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:07:05.343 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:05.343 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.343 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:05.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.343 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:07:05.343 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:07:05.343 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:05.343 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:05.344 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:05.344 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.344 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.344 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:07:05.344 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:07:05.344 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:05.344 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:05.344 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.344 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:05.344 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:07:05.344 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:05.344 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.344 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.344 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:05.344 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:07:05.345 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:05.345 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.345 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:05.345 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.345 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:07:05.345 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:07:05.345 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:05.345 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:05.345 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:05.345 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.345 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:05.345 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.345 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:07:05.345 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:07:05.346 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.346 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:05.346 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:05.346 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:07:05.346 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.346 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:05.346 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:07:05.346 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.346 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:05.346 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:05.346 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:07:05.346 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.346 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:05.346 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:07:05.346 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.347 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:05.347 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:05.347 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:07:05.347 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.347 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:05.347 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:07:05.347 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.347 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:05.347 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:05.347 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:07:05.347 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.347 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:05.347 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:07:05.347 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:05.347 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:05.348 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:05.348 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:07:05.348 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:05.348 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:05.348 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:05.348 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:07:05.348 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:07:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.327+0000 7f2a1e319700 1 -- 172.21.15.40:0/2875736217 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f29fc066730 msgr2=0x7f29fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.327+0000 7f2a1e319700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f29fc066730 0x7f29fc068bf0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2a100012e0 tx=0x7f2a10009040 comp rx=0 tx=0).stop 2024-09-13T18:07:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.327+0000 7f2a1e319700 1 -- 172.21.15.40:0/2875736217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a1815ce10 msgr2=0x7f2a181efcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.327+0000 7f2a1e319700 1 --2- 172.21.15.40:0/2875736217 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a1815ce10 0x7f2a181efcc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800c500 tx=0x7f2a0800c8c0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.327+0000 7f2a1db18700 1 -- 172.21.15.40:0/2875736217 reap_dead start 2024-09-13T18:07:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.327+0000 7f2a1db18700 1 -- 172.21.15.40:0/2875736217 reap_dead start 2024-09-13T18:07:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.327+0000 7f2a1e319700 1 -- 172.21.15.40:0/2875736217 shutdown_connections 2024-09-13T18:07:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.327+0000 7f2a1e319700 1 -- 172.21.15.40:0/2875736217 >> 172.21.15.40:0/2875736217 conn(0x7f2a180cb450 msgr2=0x7f2a1806db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.327+0000 7f2a1e319700 1 -- 172.21.15.40:0/2875736217 shutdown_connections 2024-09-13T18:07:05.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.327+0000 7f2a1e319700 1 -- 172.21.15.40:0/2875736217 wait complete. 2024-09-13T18:07:05.351 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:07:05.351 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:07:05.352 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:07:05.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.455+0000 7f97cf4f7700 1 Processor -- start 2024-09-13T18:07:05.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.455+0000 7f97cf4f7700 1 -- start start 2024-09-13T18:07:05.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.455+0000 7f97cf4f7700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f97c004d0f0 0x7f97c00e0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.455+0000 7f97cf4f7700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c00dd3c0 con 0x7f97c0046df0 2024-09-13T18:07:05.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.455+0000 7f97cf4f7700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c00e0180 con 0x7f97c00d6670 2024-09-13T18:07:05.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.455+0000 7f97cf4f7700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c00dcb80 con 0x7f97c004d0f0 2024-09-13T18:07:05.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.455+0000 7f97cecf6700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f97c0046df0 0x7f97c004de20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:56788/0 (socket says 172.21.15.40:56788) 2024-09-13T18:07:05.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.455+0000 7f97cecf6700 1 -- 172.21.15.40:0/241029856 learned_addr learned my addr 172.21.15.40:0/241029856 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:07:05.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.459+0000 7f97cdcf4700 1 --2- 172.21.15.40:0/241029856 >> v2:172.21.15.43:3300/0 conn(0x7f97c004d0f0 0x7f97c00e0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.459+0000 7f97cd4f3700 1 -- 172.21.15.40:0/241029856 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 162416889 0 0) 0x7f97c00dd3c0 con 0x7f97c0046df0 2024-09-13T18:07:05.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.459+0000 7f97cd4f3700 1 -- 172.21.15.40:0/241029856 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97b40035d0 con 0x7f97c0046df0 2024-09-13T18:07:05.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.459+0000 7f97cdcf4700 1 -- 172.21.15.40:0/241029856 >> v1:172.21.15.43:6789/0 conn(0x7f97c00d6670 legacy=0x7f97c00dc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.459+0000 7f97cdcf4700 1 -- 172.21.15.40:0/241029856 >> v1:172.21.15.40:6789/0 conn(0x7f97c0046df0 legacy=0x7f97c004de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.459+0000 7f97cdcf4700 1 -- 172.21.15.40:0/241029856 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c0047220 con 0x7f97c004d0f0 2024-09-13T18:07:05.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.459+0000 7f97cdcf4700 1 --2- 172.21.15.40:0/241029856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c004d0f0 0x7f97c00e0610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009850 tx=0x7f97b8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78567ba50a46f060 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:05.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.459+0000 7f97cd4f3700 1 -- 172.21.15.40:0/241029856 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8011830 con 0x7f97c004d0f0 2024-09-13T18:07:05.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.459+0000 7f97cd4f3700 1 -- 172.21.15.40:0/241029856 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b8011990 con 0x7f97c004d0f0 2024-09-13T18:07:05.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.459+0000 7f97cd4f3700 1 -- 172.21.15.40:0/241029856 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8011d00 con 0x7f97c004d0f0 2024-09-13T18:07:05.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.459+0000 7f97cf4f7700 1 -- 172.21.15.40:0/241029856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c004d0f0 msgr2=0x7f97c00e0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.459+0000 7f97cf4f7700 1 --2- 172.21.15.40:0/241029856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c004d0f0 0x7f97c00e0610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009850 tx=0x7f97b8009b60 comp rx=0 tx=0).stop 2024-09-13T18:07:05.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.459+0000 7f97cf4f7700 1 -- 172.21.15.40:0/241029856 shutdown_connections 2024-09-13T18:07:05.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.459+0000 7f97cf4f7700 1 --2- 172.21.15.40:0/241029856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c004d0f0 0x7f97c00e0610 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.459+0000 7f97cf4f7700 1 -- 172.21.15.40:0/241029856 >> 172.21.15.40:0/241029856 conn(0x7f97c000dc40 msgr2=0x7f97c000e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:05.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.459+0000 7f97cf4f7700 1 -- 172.21.15.40:0/241029856 shutdown_connections 2024-09-13T18:07:05.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.459+0000 7f97cf4f7700 1 -- 172.21.15.40:0/241029856 wait complete. 2024-09-13T18:07:05.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97cf4f7700 1 Processor -- start 2024-09-13T18:07:05.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97cf4f7700 1 -- start start 2024-09-13T18:07:05.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97cf4f7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f97c0046df0 0x7f97c00d8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97cf4f7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97c004d0f0 0x7f97c00d9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97cf4f7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c00d6670 0x7f97c00dafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97cf4f7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00d6f90 con 0x7f97c004d0f0 2024-09-13T18:07:05.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97cf4f7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00d6d20 con 0x7f97c00d6670 2024-09-13T18:07:05.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97cf4f7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00da200 con 0x7f97c0046df0 2024-09-13T18:07:05.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97cdcf4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97c004d0f0 0x7f97c00d9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97cdcf4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97c004d0f0 0x7f97c00d9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36532/0 (socket says 172.21.15.40:36532) 2024-09-13T18:07:05.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97cdcf4700 1 -- 172.21.15.40:0/1827802312 learned_addr learned my addr 172.21.15.40:0/1827802312 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:05.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97ce4f5700 1 --2- 172.21.15.40:0/1827802312 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f97c0046df0 0x7f97c00d8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97cecf6700 1 --2- 172.21.15.40:0/1827802312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c00d6670 0x7f97c00dafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97cdcf4700 1 -- 172.21.15.40:0/1827802312 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f97c0046df0 msgr2=0x7f97c00d8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97cdcf4700 1 --2- 172.21.15.40:0/1827802312 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f97c0046df0 0x7f97c00d8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97cdcf4700 1 -- 172.21.15.40:0/1827802312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c00d6670 msgr2=0x7f97c00dafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97cdcf4700 1 --2- 172.21.15.40:0/1827802312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c00d6670 0x7f97c00dafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97cdcf4700 1 -- 172.21.15.40:0/1827802312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b80094b0 con 0x7f97c004d0f0 2024-09-13T18:07:05.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97cdcf4700 1 --2- 172.21.15.40:0/1827802312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97c004d0f0 0x7f97c00d9600 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009ba0 tx=0x7f97b8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:05.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97beffd700 1 -- 172.21.15.40:0/1827802312 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8004550 con 0x7f97c004d0f0 2024-09-13T18:07:05.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97beffd700 1 -- 172.21.15.40:0/1827802312 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b801f070 con 0x7f97c004d0f0 2024-09-13T18:07:05.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97cf4f7700 1 -- 172.21.15.40:0/1827802312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c00da4e0 con 0x7f97c004d0f0 2024-09-13T18:07:05.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97beffd700 1 -- 172.21.15.40:0/1827802312 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8003900 con 0x7f97c004d0f0 2024-09-13T18:07:05.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.463+0000 7f97cf4f7700 1 -- 172.21.15.40:0/1827802312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c00daaa0 con 0x7f97c004d0f0 2024-09-13T18:07:05.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.467+0000 7f97cf4f7700 1 -- 172.21.15.40:0/1827802312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c004ce50 con 0x7f97c004d0f0 2024-09-13T18:07:05.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.467+0000 7f97beffd700 1 -- 172.21.15.40:0/1827802312 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f97b8003a60 con 0x7f97c004d0f0 2024-09-13T18:07:05.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.467+0000 7f97beffd700 1 --2- 172.21.15.40:0/1827802312 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f97ac066730 0x7f97ac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:05.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.467+0000 7f97beffd700 1 -- 172.21.15.40:0/1827802312 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f97b8021070 con 0x7f97c004d0f0 2024-09-13T18:07:05.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.467+0000 7f97ce4f5700 1 --2- 172.21.15.40:0/1827802312 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f97ac066730 0x7f97ac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:05.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.467+0000 7f97ce4f5700 1 --2- 172.21.15.40:0/1827802312 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f97ac066730 0x7f97ac068bf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f97c40018a0 tx=0x7f97c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:05.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.475+0000 7f97beffd700 1 -- 172.21.15.40:0/1827802312 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f97b8052ca0 con 0x7f97c004d0f0 2024-09-13T18:07:05.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.747+0000 7f97cf4f7700 1 -- 172.21.15.40:0/1827802312 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f97c0048d70 con 0x7f97c004d0f0 2024-09-13T18:07:05.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.751+0000 7f97beffd700 1 -- 172.21.15.40:0/1827802312 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f97b8015020 con 0x7f97c004d0f0 2024-09-13T18:07:05.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.751+0000 7f97cf4f7700 1 -- 172.21.15.40:0/1827802312 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f97ac066730 msgr2=0x7f97ac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.751+0000 7f97cf4f7700 1 --2- 172.21.15.40:0/1827802312 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f97ac066730 0x7f97ac068bf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f97c40018a0 tx=0x7f97c4009040 comp rx=0 tx=0).stop 2024-09-13T18:07:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.751+0000 7f97cf4f7700 1 -- 172.21.15.40:0/1827802312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97c004d0f0 msgr2=0x7f97c00d9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.751+0000 7f97cf4f7700 1 --2- 172.21.15.40:0/1827802312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97c004d0f0 0x7f97c00d9600 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009ba0 tx=0x7f97b8004770 comp rx=0 tx=0).stop 2024-09-13T18:07:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.751+0000 7f97cf4f7700 1 -- 172.21.15.40:0/1827802312 shutdown_connections 2024-09-13T18:07:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.751+0000 7f97cf4f7700 1 --2- 172.21.15.40:0/1827802312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c00d6670 0x7f97c00dafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.751+0000 7f97cf4f7700 1 --2- 172.21.15.40:0/1827802312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97c004d0f0 0x7f97c00d9600 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.751+0000 7f97cf4f7700 1 --2- 172.21.15.40:0/1827802312 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f97ac066730 0x7f97ac068bf0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.751+0000 7f97cf4f7700 1 --2- 172.21.15.40:0/1827802312 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f97c0046df0 0x7f97c00d8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.751+0000 7f97cf4f7700 1 -- 172.21.15.40:0/1827802312 >> 172.21.15.40:0/1827802312 conn(0x7f97c000dc40 msgr2=0x7f97c00cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.751+0000 7f97cf4f7700 1 -- 172.21.15.40:0/1827802312 shutdown_connections 2024-09-13T18:07:05.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:05.751+0000 7f97cf4f7700 1 -- 172.21.15.40:0/1827802312 wait complete. 2024-09-13T18:07:05.773 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:07:06.760 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:07:06.763 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:07:15.775 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:07:15.775 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:07:15.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.887+0000 7f8928b09700 1 Processor -- start 2024-09-13T18:07:15.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.887+0000 7f8928b09700 1 -- start start 2024-09-13T18:07:15.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.887+0000 7f8928b09700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f891c04dff0 0x7f891c0482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:15.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.887+0000 7f8928b09700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f891c04daa0 0x7f891c0e9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:15.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.887+0000 7f8928b09700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f891c0e2800 con 0x7f891c047ed0 2024-09-13T18:07:15.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.887+0000 7f8928b09700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f891c0096b0 con 0x7f891c04daa0 2024-09-13T18:07:15.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.887+0000 7f8928b09700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f891c0e1cc0 con 0x7f891c04dff0 2024-09-13T18:07:15.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.887+0000 7f89237fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f891c047ed0 0x7f891c0de6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49920/0 (socket says 172.21.15.40:49920) 2024-09-13T18:07:15.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.887+0000 7f89237fe700 1 -- 172.21.15.40:0/3817226193 learned_addr learned my addr 172.21.15.40:0/3817226193 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:07:15.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.887+0000 7f8923fff700 1 --2- 172.21.15.40:0/3817226193 >> v2:172.21.15.40:3300/0 conn(0x7f891c04daa0 0x7f891c0e9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:15.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.887+0000 7f8922ffd700 1 --2- 172.21.15.40:0/3817226193 >> v2:172.21.15.43:3300/0 conn(0x7f891c04dff0 0x7f891c0482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:15.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.887+0000 7f8923fff700 1 -- 172.21.15.40:0/3817226193 >> v2:172.21.15.43:3300/0 conn(0x7f891c04dff0 msgr2=0x7f891c0482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:15.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.887+0000 7f8923fff700 1 --2- 172.21.15.40:0/3817226193 >> v2:172.21.15.43:3300/0 conn(0x7f891c04dff0 0x7f891c0482b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:15.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.887+0000 7f8923fff700 1 -- 172.21.15.40:0/3817226193 >> v1:172.21.15.40:6789/0 conn(0x7f891c047ed0 legacy=0x7f891c0de6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:15.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.887+0000 7f8923fff700 1 -- 172.21.15.40:0/3817226193 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891c04d090 con 0x7f891c04daa0 2024-09-13T18:07:15.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.887+0000 7f8923fff700 1 --2- 172.21.15.40:0/3817226193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c04daa0 0x7f891c0e9050 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8924069620 tx=0x7f8924069e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46245d5f624be901 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:15.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.887+0000 7f89227fc700 1 -- 172.21.15.40:0/3817226193 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924067dd0 con 0x7f891c04daa0 2024-09-13T18:07:15.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.887+0000 7f89227fc700 1 -- 172.21.15.40:0/3817226193 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8924067f30 con 0x7f891c04daa0 2024-09-13T18:07:15.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.887+0000 7f89227fc700 1 -- 172.21.15.40:0/3817226193 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924074700 con 0x7f891c04daa0 2024-09-13T18:07:15.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.891+0000 7f8928b09700 1 -- 172.21.15.40:0/3817226193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c04daa0 msgr2=0x7f891c0e9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:15.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.891+0000 7f8928b09700 1 --2- 172.21.15.40:0/3817226193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c04daa0 0x7f891c0e9050 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8924069620 tx=0x7f8924069e70 comp rx=0 tx=0).stop 2024-09-13T18:07:15.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.891+0000 7f8928b09700 1 -- 172.21.15.40:0/3817226193 shutdown_connections 2024-09-13T18:07:15.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.891+0000 7f8928b09700 1 --2- 172.21.15.40:0/3817226193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c04daa0 0x7f891c0e9050 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:15.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.891+0000 7f8928b09700 1 --2- 172.21.15.40:0/3817226193 >> v2:172.21.15.43:3300/0 conn(0x7f891c04dff0 0x7f891c0482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:15.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.891+0000 7f8928b09700 1 -- 172.21.15.40:0/3817226193 >> 172.21.15.40:0/3817226193 conn(0x7f891c009a40 msgr2=0x7f891c009e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:15.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.891+0000 7f8928b09700 1 -- 172.21.15.40:0/3817226193 shutdown_connections 2024-09-13T18:07:15.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.891+0000 7f8928b09700 1 -- 172.21.15.40:0/3817226193 wait complete. 2024-09-13T18:07:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.891+0000 7f8928b09700 1 Processor -- start 2024-09-13T18:07:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.891+0000 7f8928b09700 1 -- start start 2024-09-13T18:07:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.891+0000 7f8928b09700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f891c047ed0 0x7f891c175210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.891+0000 7f8928b09700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f891c04daa0 0x7f891c033470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.891+0000 7f8928b09700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c04dff0 0x7f891c0d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.891+0000 7f8928b09700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c175750 con 0x7f891c04dff0 2024-09-13T18:07:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.891+0000 7f8928b09700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0df890 con 0x7f891c04daa0 2024-09-13T18:07:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.891+0000 7f8928b09700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0e1340 con 0x7f891c047ed0 2024-09-13T18:07:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.891+0000 7f8923fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c04dff0 0x7f891c0d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.891+0000 7f8923fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c04dff0 0x7f891c0d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57276/0 (socket says 172.21.15.40:57276) 2024-09-13T18:07:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.891+0000 7f8923fff700 1 -- 172.21.15.40:0/255561848 learned_addr learned my addr 172.21.15.40:0/255561848 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.891+0000 7f8922ffd700 1 --2- 172.21.15.40:0/255561848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f891c04daa0 0x7f891c033470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.895+0000 7f89237fe700 1 --2- 172.21.15.40:0/255561848 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f891c047ed0 0x7f891c175210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.895+0000 7f8923fff700 1 -- 172.21.15.40:0/255561848 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f891c047ed0 msgr2=0x7f891c175210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.895+0000 7f8923fff700 1 --2- 172.21.15.40:0/255561848 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f891c047ed0 0x7f891c175210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.895+0000 7f8923fff700 1 -- 172.21.15.40:0/255561848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f891c04daa0 msgr2=0x7f891c033470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.895+0000 7f8923fff700 1 --2- 172.21.15.40:0/255561848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f891c04daa0 0x7f891c033470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.895+0000 7f8923fff700 1 -- 172.21.15.40:0/255561848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892404f5c0 con 0x7f891c04dff0 2024-09-13T18:07:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.895+0000 7f8923fff700 1 --2- 172.21.15.40:0/255561848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c04dff0 0x7f891c0d1c20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8924067bd0 tx=0x7f8924068470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.895+0000 7f890ffff700 1 -- 172.21.15.40:0/255561848 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924067930 con 0x7f891c04dff0 2024-09-13T18:07:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.895+0000 7f890ffff700 1 -- 172.21.15.40:0/255561848 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8924072150 con 0x7f891c04dff0 2024-09-13T18:07:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.895+0000 7f8928b09700 1 -- 172.21.15.40:0/255561848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891c0e15e0 con 0x7f891c04dff0 2024-09-13T18:07:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.895+0000 7f890ffff700 1 -- 172.21.15.40:0/255561848 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89240724c0 con 0x7f891c04dff0 2024-09-13T18:07:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.895+0000 7f8928b09700 1 -- 172.21.15.40:0/255561848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891c0dfae0 con 0x7f891c04dff0 2024-09-13T18:07:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.895+0000 7f8928b09700 1 -- 172.21.15.40:0/255561848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891c04e5c0 con 0x7f891c04dff0 2024-09-13T18:07:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.895+0000 7f890ffff700 1 -- 172.21.15.40:0/255561848 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8924091020 con 0x7f891c04dff0 2024-09-13T18:07:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.899+0000 7f890ffff700 1 --2- 172.21.15.40:0/255561848 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8908066760 0x7f8908068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:15.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.899+0000 7f890ffff700 1 -- 172.21.15.40:0/255561848 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f89241000f0 con 0x7f891c04dff0 2024-09-13T18:07:15.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.899+0000 7f89237fe700 1 --2- 172.21.15.40:0/255561848 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8908066760 0x7f8908068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:15.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.899+0000 7f89237fe700 1 --2- 172.21.15.40:0/255561848 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8908066760 0x7f8908068c20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8918005930 tx=0x7f8918009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:15.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:15.903+0000 7f890ffff700 1 -- 172.21.15.40:0/255561848 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f89240cd130 con 0x7f891c04dff0 2024-09-13T18:07:16.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.163+0000 7f8928b09700 1 -- 172.21.15.40:0/255561848 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f891c0dc0f0 con 0x7f891c04dff0 2024-09-13T18:07:16.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.167+0000 7f890ffff700 1 -- 172.21.15.40:0/255561848 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f89240d0750 con 0x7f891c04dff0 2024-09-13T18:07:16.175 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":140,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:07:16.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.167+0000 7f8928b09700 1 -- 172.21.15.40:0/255561848 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8908066760 msgr2=0x7f8908068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.167+0000 7f8928b09700 1 --2- 172.21.15.40:0/255561848 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8908066760 0x7f8908068c20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8918005930 tx=0x7f8918009040 comp rx=0 tx=0).stop 2024-09-13T18:07:16.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.167+0000 7f8928b09700 1 -- 172.21.15.40:0/255561848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c04dff0 msgr2=0x7f891c0d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.167+0000 7f8928b09700 1 --2- 172.21.15.40:0/255561848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c04dff0 0x7f891c0d1c20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8924067bd0 tx=0x7f8924068470 comp rx=0 tx=0).stop 2024-09-13T18:07:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.167+0000 7f8928b09700 1 -- 172.21.15.40:0/255561848 shutdown_connections 2024-09-13T18:07:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.167+0000 7f8928b09700 1 --2- 172.21.15.40:0/255561848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f891c04dff0 0x7f891c0d1c20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.167+0000 7f8928b09700 1 --2- 172.21.15.40:0/255561848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f891c04daa0 0x7f891c033470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.167+0000 7f8928b09700 1 --2- 172.21.15.40:0/255561848 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8908066760 0x7f8908068c20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.167+0000 7f8928b09700 1 --2- 172.21.15.40:0/255561848 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f891c047ed0 0x7f891c175210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.167+0000 7f8928b09700 1 -- 172.21.15.40:0/255561848 >> 172.21.15.40:0/255561848 conn(0x7f891c009a40 msgr2=0x7f891c00e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.167+0000 7f8928b09700 1 -- 172.21.15.40:0/255561848 shutdown_connections 2024-09-13T18:07:16.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.167+0000 7f8928b09700 1 -- 172.21.15.40:0/255561848 wait complete. 2024-09-13T18:07:16.193 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:07:16.193 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T18:07:16.194 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:07:16.194 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:07:16.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.291+0000 7ff881786700 1 Processor -- start 2024-09-13T18:07:16.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.291+0000 7ff881786700 1 -- start start 2024-09-13T18:07:16.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.291+0000 7ff881786700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff87c14fa30 0x7ff87c158e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.291+0000 7ff881786700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff87c14fe10 0x7ff87c0c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.291+0000 7ff881786700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87c159ca0 con 0x7ff87c1561a0 2024-09-13T18:07:16.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.291+0000 7ff881786700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff87c072100 con 0x7ff87c14fa30 2024-09-13T18:07:16.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.291+0000 7ff881786700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c159400 con 0x7ff87c14fe10 2024-09-13T18:07:16.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.291+0000 7ff87bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff87c14fa30 0x7ff87c158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.291+0000 7ff87bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff87c14fa30 0x7ff87c158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57278/0 (socket says 172.21.15.40:57278) 2024-09-13T18:07:16.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.291+0000 7ff87bfff700 1 -- 172.21.15.40:0/326797093 learned_addr learned my addr 172.21.15.40:0/326797093 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:16.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.291+0000 7ff87b7fe700 1 --2- 172.21.15.40:0/326797093 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff87c14fe10 0x7ff87c0c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.295+0000 7ff87bfff700 1 -- 172.21.15.40:0/326797093 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff87c14fe10 msgr2=0x7ff87c0c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:16.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.295+0000 7ff87bfff700 1 --2- 172.21.15.40:0/326797093 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff87c14fe10 0x7ff87c0c6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.295+0000 7ff87bfff700 1 -- 172.21.15.40:0/326797093 >> v1:172.21.15.43:6789/0 conn(0x7ff87c1561a0 legacy=0x7ff87c0c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.295+0000 7ff87bfff700 1 -- 172.21.15.40:0/326797093 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c1596e0 con 0x7ff87c14fa30 2024-09-13T18:07:16.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.295+0000 7ff87bfff700 1 --2- 172.21.15.40:0/326797093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c14fa30 0x7ff87c158e00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00aab0 tx=0x7ff86c00adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e70ba00dbffd49 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:16.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.295+0000 7ff87affd700 1 -- 172.21.15.40:0/326797093 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c00cb70 con 0x7ff87c14fa30 2024-09-13T18:07:16.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.295+0000 7ff87affd700 1 -- 172.21.15.40:0/326797093 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86c00ccd0 con 0x7ff87c14fa30 2024-09-13T18:07:16.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.295+0000 7ff87affd700 1 -- 172.21.15.40:0/326797093 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c018620 con 0x7ff87c14fa30 2024-09-13T18:07:16.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.295+0000 7ff881786700 1 -- 172.21.15.40:0/326797093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c14fa30 msgr2=0x7ff87c158e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.295+0000 7ff881786700 1 --2- 172.21.15.40:0/326797093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c14fa30 0x7ff87c158e00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00aab0 tx=0x7ff86c00adc0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.295+0000 7ff881786700 1 -- 172.21.15.40:0/326797093 shutdown_connections 2024-09-13T18:07:16.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.295+0000 7ff881786700 1 --2- 172.21.15.40:0/326797093 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff87c14fe10 0x7ff87c0c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.295+0000 7ff881786700 1 --2- 172.21.15.40:0/326797093 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c14fa30 0x7ff87c158e00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.295+0000 7ff881786700 1 -- 172.21.15.40:0/326797093 >> 172.21.15.40:0/326797093 conn(0x7ff87c06a390 msgr2=0x7ff87c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:16.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.295+0000 7ff881786700 1 -- 172.21.15.40:0/326797093 shutdown_connections 2024-09-13T18:07:16.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.295+0000 7ff881786700 1 -- 172.21.15.40:0/326797093 wait complete. 2024-09-13T18:07:16.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.299+0000 7ff881786700 1 Processor -- start 2024-09-13T18:07:16.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.299+0000 7ff881786700 1 -- start start 2024-09-13T18:07:16.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.299+0000 7ff881786700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff87c14fa30 0x7ff87c14bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.299+0000 7ff881786700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c1561a0 0x7ff87c1eb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.299+0000 7ff881786700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff87c0cade0 0x7ff87c0c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.299+0000 7ff881786700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c1ea500 con 0x7ff87c1561a0 2024-09-13T18:07:16.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.299+0000 7ff881786700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c0c1fb0 con 0x7ff87c14fa30 2024-09-13T18:07:16.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.299+0000 7ff881786700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c0c9f30 con 0x7ff87c0cade0 2024-09-13T18:07:16.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.299+0000 7ff87b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c1561a0 0x7ff87c1eb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.299+0000 7ff87b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c1561a0 0x7ff87c1eb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57282/0 (socket says 172.21.15.40:57282) 2024-09-13T18:07:16.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.299+0000 7ff87b7fe700 1 -- 172.21.15.40:0/2084430580 learned_addr learned my addr 172.21.15.40:0/2084430580 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:16.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.299+0000 7ff87bfff700 1 --2- 172.21.15.40:0/2084430580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff87c14fa30 0x7ff87c14bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.299+0000 7ff880f85700 1 --2- 172.21.15.40:0/2084430580 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff87c0cade0 0x7ff87c0c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.299+0000 7ff87b7fe700 1 -- 172.21.15.40:0/2084430580 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff87c0cade0 msgr2=0x7ff87c0c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.299+0000 7ff87b7fe700 1 --2- 172.21.15.40:0/2084430580 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff87c0cade0 0x7ff87c0c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.299+0000 7ff87b7fe700 1 -- 172.21.15.40:0/2084430580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff87c14fa30 msgr2=0x7ff87c14bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.299+0000 7ff87b7fe700 1 --2- 172.21.15.40:0/2084430580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff87c14fa30 0x7ff87c14bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.299+0000 7ff87b7fe700 1 -- 172.21.15.40:0/2084430580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86c00a710 con 0x7ff87c1561a0 2024-09-13T18:07:16.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.299+0000 7ff87b7fe700 1 --2- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c1561a0 0x7ff87c1eb580 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff870010bb0 tx=0x7ff870010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:16.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff878ff9700 1 -- 172.21.15.40:0/2084430580 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870009720 con 0x7ff87c1561a0 2024-09-13T18:07:16.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff878ff9700 1 -- 172.21.15.40:0/2084430580 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87000ed20 con 0x7ff87c1561a0 2024-09-13T18:07:16.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff881786700 1 -- 172.21.15.40:0/2084430580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87c1ea220 con 0x7ff87c1561a0 2024-09-13T18:07:16.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff878ff9700 1 -- 172.21.15.40:0/2084430580 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870011690 con 0x7ff87c1561a0 2024-09-13T18:07:16.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff881786700 1 -- 172.21.15.40:0/2084430580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff87c0ca5e0 con 0x7ff87c1561a0 2024-09-13T18:07:16.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff881786700 1 --2- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c14fe10 0x7ff87c0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff881786700 1 -- 172.21.15.40:0/2084430580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c067600 con 0x7ff87c14fe10 2024-09-13T18:07:16.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff87bfff700 1 --2- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c14fe10 0x7ff87c0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff878ff9700 1 -- 172.21.15.40:0/2084430580 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff8700117f0 con 0x7ff87c1561a0 2024-09-13T18:07:16.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff87bfff700 1 -- 172.21.15.40:0/2084430580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff87c155c80 con 0x7ff87c14fe10 2024-09-13T18:07:16.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff87bfff700 1 --2- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c14fe10 0x7ff87c0c9740 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00aab0 tx=0x7ff86c0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:16.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff878ff9700 1 --2- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff868066730 0x7ff868068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff880f85700 1 --2- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff868066730 0x7ff868068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff878ff9700 1 -- 172.21.15.40:0/2084430580 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff870086750 con 0x7ff87c1561a0 2024-09-13T18:07:16.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff878ff9700 1 -- 172.21.15.40:0/2084430580 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c027070 con 0x7ff87c14fe10 2024-09-13T18:07:16.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff878ff9700 1 -- 172.21.15.40:0/2084430580 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff86c01d040 con 0x7ff87c14fe10 2024-09-13T18:07:16.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff878ff9700 1 -- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c14fe10 msgr2=0x7ff87c0c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff878ff9700 1 --2- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c14fe10 0x7ff87c0c9740 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00aab0 tx=0x7ff86c0047e0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff878ff9700 1 -- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c14fe10 msgr2=0x7ff87c0c9740 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:16.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff878ff9700 1 --2- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c14fe10 0x7ff87c0c9740 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00aab0 tx=0x7ff86c0047e0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.303+0000 7ff880f85700 1 --2- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff868066730 0x7ff868068bf0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff8740012e0 tx=0x7ff874009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:16.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff881786700 1 --2- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c0ca800 0x7ff87c1ed680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff881786700 1 -- 172.21.15.40:0/2084430580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c0c2120 con 0x7ff87c0ca800 2024-09-13T18:07:16.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff87bfff700 1 --2- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c0ca800 0x7ff87c1ed680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff87bfff700 1 -- 172.21.15.40:0/2084430580 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff87c156610 con 0x7ff87c0ca800 2024-09-13T18:07:16.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff87bfff700 1 --2- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c0ca800 0x7ff87c1ed680 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00afd0 tx=0x7ff86c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:16.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff878ff9700 1 -- 172.21.15.40:0/2084430580 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c01c050 con 0x7ff87c0ca800 2024-09-13T18:07:16.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff878ff9700 1 -- 172.21.15.40:0/2084430580 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4453 (secure 0 0 0) 0x7ff86c003e50 con 0x7ff87c0ca800 2024-09-13T18:07:16.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff878ff9700 1 -- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c0ca800 msgr2=0x7ff87c1ed680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff878ff9700 1 --2- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c0ca800 0x7ff87c1ed680 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00afd0 tx=0x7ff86c004770 comp rx=0 tx=0).stop 2024-09-13T18:07:16.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff878ff9700 1 -- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c0ca800 msgr2=0x7ff87c1ed680 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:16.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff878ff9700 1 --2- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c0ca800 0x7ff87c1ed680 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00afd0 tx=0x7ff86c004770 comp rx=0 tx=0).stop 2024-09-13T18:07:16.326 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:07:16.326 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:07:16.326 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:07:16.326 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:07:16.326 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 140, 2024-09-13T18:07:16.326 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:07:16.326 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:07:16.326 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:07:16.326 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:07:16.326 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:16.326 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 11, 2024-09-13T18:07:16.327 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:16.327 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:07:16.327 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:07:16.327 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:16.327 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:16.327 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:16.327 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:16.327 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:16.327 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:16.327 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:16.327 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:16.327 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:16.327 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:16.327 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:16.328 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:07:16.328 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:07:16.328 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:16.328 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:16.328 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:16.328 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:16.328 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:16.328 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:16.328 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:16.328 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:16.328 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:16.328 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:16.328 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:16.329 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.329 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:07:16.329 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:07:16.329 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:07:16.329 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:07:16.329 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:07:16.329 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:07:16.329 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:16.329 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:16.329 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:07:16.329 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:07:16.329 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:07:16.329 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:07:16.330 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:07:16.330 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:07:16.330 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:07:16.330 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:16.330 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:07:16.330 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:16.330 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:16.330 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:16.330 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:16.330 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:16.330 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:16.331 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:16.331 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:16.331 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:16.331 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:16.331 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:16.331 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:07:16.331 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.331 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:07:16.331 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.331 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:07:16.331 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:07:16.331 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:16.332 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:16.332 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.332 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:16.332 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:07:16.332 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:16.332 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.332 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.332 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:16.332 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:07:16.332 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:16.332 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.332 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:16.332 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.333 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:07:16.333 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:07:16.333 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:16.333 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:16.333 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:16.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.333 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:07:16.333 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:07:16.333 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:16.333 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:16.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.333 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:16.333 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:07:16.334 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:16.334 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.334 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.334 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:16.334 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:07:16.334 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:16.334 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.334 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:16.334 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.334 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:07:16.334 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:07:16.334 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:16.334 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:16.334 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:16.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.335 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.335 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:07:16.335 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:07:16.335 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:16.335 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:16.335 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.335 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:16.335 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:07:16.335 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:16.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.335 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.335 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:16.335 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:07:16.336 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:16.336 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.336 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:16.336 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.336 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:07:16.336 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:07:16.336 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:16.336 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:16.336 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:16.336 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.336 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:16.336 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.336 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:07:16.336 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:07:16.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.337 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:16.337 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:16.337 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:07:16.337 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.337 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:16.337 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:07:16.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.337 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:16.337 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:16.337 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:07:16.337 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.337 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:16.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.338 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:07:16.338 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:07:16.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff881786700 1 -- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff868066730 msgr2=0x7ff868068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff881786700 1 --2- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff868066730 0x7ff868068bf0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff8740012e0 tx=0x7ff874009040 comp rx=0 tx=0).stop 2024-09-13T18:07:16.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff881786700 1 -- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c1561a0 msgr2=0x7ff87c1eb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff881786700 1 --2- 172.21.15.40:0/2084430580 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff87c1561a0 0x7ff87c1eb580 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff870010bb0 tx=0x7ff870010ec0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff880f85700 1 -- 172.21.15.40:0/2084430580 reap_dead start 2024-09-13T18:07:16.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff880f85700 1 -- 172.21.15.40:0/2084430580 reap_dead start 2024-09-13T18:07:16.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff881786700 1 -- 172.21.15.40:0/2084430580 shutdown_connections 2024-09-13T18:07:16.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff881786700 1 -- 172.21.15.40:0/2084430580 >> 172.21.15.40:0/2084430580 conn(0x7ff87c06a390 msgr2=0x7ff87c14d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:16.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff881786700 1 -- 172.21.15.40:0/2084430580 shutdown_connections 2024-09-13T18:07:16.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.315+0000 7ff881786700 1 -- 172.21.15.40:0/2084430580 wait complete. 2024-09-13T18:07:16.340 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:07:16.340 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:07:16.341 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:07:16.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da7fff700 1 Processor -- start 2024-09-13T18:07:16.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da7fff700 1 -- start start 2024-09-13T18:07:16.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da7fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6da8151850 0x7f6da80c2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da7fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6da80c78f0 0x7f6da81572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6da80c3ee0 0x7f6da80c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da7fff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da8157820 con 0x7f6da80c3ee0 2024-09-13T18:07:16.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da7fff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da80c8d50 con 0x7f6da8151850 2024-09-13T18:07:16.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da7fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da80c7400 con 0x7f6da80c78f0 2024-09-13T18:07:16.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da77fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6da80c3ee0 0x7f6da80c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da77fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6da80c3ee0 0x7f6da80c8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57304/0 (socket says 172.21.15.40:57304) 2024-09-13T18:07:16.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da77fe700 1 -- 172.21.15.40:0/3302937720 learned_addr learned my addr 172.21.15.40:0/3302937720 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:16.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da6ffd700 1 --2- 172.21.15.40:0/3302937720 >> v2:172.21.15.43:3300/0 conn(0x7f6da8151850 0x7f6da80c2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da67fc700 1 --2- 172.21.15.40:0/3302937720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6da80c78f0 0x7f6da81572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da77fe700 1 -- 172.21.15.40:0/3302937720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6da80c78f0 msgr2=0x7f6da81572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:16.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da77fe700 1 --2- 172.21.15.40:0/3302937720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6da80c78f0 0x7f6da81572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da77fe700 1 -- 172.21.15.40:0/3302937720 >> v2:172.21.15.43:3300/0 conn(0x7f6da8151850 msgr2=0x7f6da80c2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:16.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da77fe700 1 --2- 172.21.15.40:0/3302937720 >> v2:172.21.15.43:3300/0 conn(0x7f6da8151850 0x7f6da80c2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da77fe700 1 -- 172.21.15.40:0/3302937720 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da80c20d0 con 0x7f6da80c3ee0 2024-09-13T18:07:16.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da77fe700 1 --2- 172.21.15.40:0/3302937720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da80c3ee0 0x7f6da80c8f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6da804e7b0 tx=0x7f6da000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39eec199ec5493 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:16.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da5ffb700 1 -- 172.21.15.40:0/3302937720 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da000e070 con 0x7f6da80c3ee0 2024-09-13T18:07:16.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da5ffb700 1 -- 172.21.15.40:0/3302937720 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da000b910 con 0x7f6da80c3ee0 2024-09-13T18:07:16.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da5ffb700 1 -- 172.21.15.40:0/3302937720 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da000bcc0 con 0x7f6da80c3ee0 2024-09-13T18:07:16.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da7fff700 1 -- 172.21.15.40:0/3302937720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da80c3ee0 msgr2=0x7f6da80c8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da7fff700 1 --2- 172.21.15.40:0/3302937720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da80c3ee0 0x7f6da80c8f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6da804e7b0 tx=0x7f6da000b330 comp rx=0 tx=0).stop 2024-09-13T18:07:16.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da7fff700 1 -- 172.21.15.40:0/3302937720 shutdown_connections 2024-09-13T18:07:16.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da7fff700 1 --2- 172.21.15.40:0/3302937720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da80c3ee0 0x7f6da80c8f20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da7fff700 1 --2- 172.21.15.40:0/3302937720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6da80c78f0 0x7f6da81572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da7fff700 1 --2- 172.21.15.40:0/3302937720 >> v2:172.21.15.43:3300/0 conn(0x7f6da8151850 0x7f6da80c2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.447+0000 7f6da7fff700 1 -- 172.21.15.40:0/3302937720 >> 172.21.15.40:0/3302937720 conn(0x7f6da80ccef0 msgr2=0x7f6da80cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:16.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.451+0000 7f6da7fff700 1 -- 172.21.15.40:0/3302937720 shutdown_connections 2024-09-13T18:07:16.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.451+0000 7f6da7fff700 1 -- 172.21.15.40:0/3302937720 wait complete. 2024-09-13T18:07:16.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.451+0000 7f6da7fff700 1 Processor -- start 2024-09-13T18:07:16.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.451+0000 7f6da7fff700 1 -- start start 2024-09-13T18:07:16.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.451+0000 7f6da7fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6da80c3ee0 0x7f6da80ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.451+0000 7f6da7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6da80c78f0 0x7f6da814a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.451+0000 7f6da7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da8151850 0x7f6da814a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.451+0000 7f6da7fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da814c0c0 con 0x7f6da8151850 2024-09-13T18:07:16.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.451+0000 7f6da7fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da814d5c0 con 0x7f6da80c78f0 2024-09-13T18:07:16.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.451+0000 7f6da7fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da814cd20 con 0x7f6da80c3ee0 2024-09-13T18:07:16.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6da77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da8151850 0x7f6da814a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6da77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da8151850 0x7f6da814a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57320/0 (socket says 172.21.15.40:57320) 2024-09-13T18:07:16.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6da77fe700 1 -- 172.21.15.40:0/141488819 learned_addr learned my addr 172.21.15.40:0/141488819 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:16.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6da67fc700 1 --2- 172.21.15.40:0/141488819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6da80c78f0 0x7f6da814a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6da6ffd700 1 --2- 172.21.15.40:0/141488819 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6da80c3ee0 0x7f6da80ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6da77fe700 1 -- 172.21.15.40:0/141488819 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6da80c3ee0 msgr2=0x7f6da80ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6da77fe700 1 --2- 172.21.15.40:0/141488819 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6da80c3ee0 0x7f6da80ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6da77fe700 1 -- 172.21.15.40:0/141488819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6da80c78f0 msgr2=0x7f6da814a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6da77fe700 1 --2- 172.21.15.40:0/141488819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6da80c78f0 0x7f6da814a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6da77fe700 1 -- 172.21.15.40:0/141488819 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da0009b30 con 0x7f6da8151850 2024-09-13T18:07:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6da77fe700 1 --2- 172.21.15.40:0/141488819 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da8151850 0x7f6da814a790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6da0012000 tx=0x7f6da0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6d8f7fe700 1 -- 172.21.15.40:0/141488819 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da0004900 con 0x7f6da8151850 2024-09-13T18:07:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6d8f7fe700 1 -- 172.21.15.40:0/141488819 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da00092e0 con 0x7f6da8151850 2024-09-13T18:07:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6da7fff700 1 -- 172.21.15.40:0/141488819 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da814d000 con 0x7f6da8151850 2024-09-13T18:07:16.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6d8f7fe700 1 -- 172.21.15.40:0/141488819 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da001b650 con 0x7f6da8151850 2024-09-13T18:07:16.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6da7fff700 1 -- 172.21.15.40:0/141488819 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da8149d30 con 0x7f6da8151850 2024-09-13T18:07:16.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6da7fff700 1 --2- 172.21.15.40:0/141488819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6da8146a80 0x7f6da8146e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6da7fff700 1 -- 172.21.15.40:0/141488819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da8074830 con 0x7f6da8146a80 2024-09-13T18:07:16.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6da6ffd700 1 --2- 172.21.15.40:0/141488819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6da8146a80 0x7f6da8146e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6da6ffd700 1 -- 172.21.15.40:0/141488819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6da80c8740 con 0x7f6da8146a80 2024-09-13T18:07:16.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6da6ffd700 1 --2- 172.21.15.40:0/141488819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6da8146a80 0x7f6da8146e60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0119d0 tx=0x7f6d9c011d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:16.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.455+0000 7f6d8f7fe700 1 -- 172.21.15.40:0/141488819 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c004240 con 0x7f6da8146a80 2024-09-13T18:07:16.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.459+0000 7f6d8f7fe700 1 -- 172.21.15.40:0/141488819 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6da001b7b0 con 0x7f6da8151850 2024-09-13T18:07:16.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.459+0000 7f6d8f7fe700 1 --2- 172.21.15.40:0/141488819 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6d940667d0 0x7f6d94068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.459+0000 7f6d8f7fe700 1 -- 172.21.15.40:0/141488819 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6da0085850 con 0x7f6da8151850 2024-09-13T18:07:16.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.459+0000 7f6da67fc700 1 --2- 172.21.15.40:0/141488819 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6d940667d0 0x7f6d94068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.459+0000 7f6d8f7fe700 1 -- 172.21.15.40:0/141488819 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6d9c0043a0 con 0x7f6da8146a80 2024-09-13T18:07:16.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.459+0000 7f6d8f7fe700 1 -- 172.21.15.40:0/141488819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6da8146a80 msgr2=0x7f6da8146e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.459+0000 7f6d8f7fe700 1 --2- 172.21.15.40:0/141488819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6da8146a80 0x7f6da8146e60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0119d0 tx=0x7f6d9c011d90 comp rx=0 tx=0).stop 2024-09-13T18:07:16.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.459+0000 7f6d8f7fe700 1 -- 172.21.15.40:0/141488819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6da8146a80 msgr2=0x7f6da8146e60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:16.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.459+0000 7f6d8f7fe700 1 --2- 172.21.15.40:0/141488819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6da8146a80 0x7f6da8146e60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.459+0000 7f6da67fc700 1 --2- 172.21.15.40:0/141488819 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6d940667d0 0x7f6d94068c90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6da814ebb0 tx=0x7f6d98009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.467+0000 7f6da7fff700 1 --2- 172.21.15.40:0/141488819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6da81473a0 0x7f6da8148c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.467+0000 7f6da7fff700 1 -- 172.21.15.40:0/141488819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da806ebc0 con 0x7f6da81473a0 2024-09-13T18:07:16.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.467+0000 7f6da6ffd700 1 --2- 172.21.15.40:0/141488819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6da81473a0 0x7f6da8148c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.467+0000 7f6da6ffd700 1 -- 172.21.15.40:0/141488819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6da80c8a70 con 0x7f6da81473a0 2024-09-13T18:07:16.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.471+0000 7f6da6ffd700 1 --2- 172.21.15.40:0/141488819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6da81473a0 0x7f6da8148c40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0119d0 tx=0x7f6d9c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:16.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.471+0000 7f6d8f7fe700 1 -- 172.21.15.40:0/141488819 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c01e450 con 0x7f6da81473a0 2024-09-13T18:07:16.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.471+0000 7f6d8f7fe700 1 -- 172.21.15.40:0/141488819 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4783 (secure 0 0 0) 0x7f6d9c0283a0 con 0x7f6da81473a0 2024-09-13T18:07:16.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.471+0000 7f6d8f7fe700 1 -- 172.21.15.40:0/141488819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6da81473a0 msgr2=0x7f6da8148c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.471+0000 7f6d8f7fe700 1 --2- 172.21.15.40:0/141488819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6da81473a0 0x7f6da8148c40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0119d0 tx=0x7f6d9c004770 comp rx=0 tx=0).stop 2024-09-13T18:07:16.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.471+0000 7f6d8f7fe700 1 -- 172.21.15.40:0/141488819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6da81473a0 msgr2=0x7f6da8148c40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:16.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.471+0000 7f6d8f7fe700 1 --2- 172.21.15.40:0/141488819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6da81473a0 0x7f6da8148c40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.481 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:07:16.481 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:07:16.482 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:07:16.482 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:07:16.482 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 140, 2024-09-13T18:07:16.482 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:07:16.482 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:07:16.482 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:07:16.482 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:07:16.483 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:16.483 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 11, 2024-09-13T18:07:16.483 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:16.483 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:07:16.483 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:07:16.483 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:16.483 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:16.484 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:16.484 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:16.484 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:16.484 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:16.484 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:16.484 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:16.484 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:16.485 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:16.485 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:16.485 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:07:16.485 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:07:16.485 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:16.485 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:16.485 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:16.486 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:16.486 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:16.486 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:16.486 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:16.486 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:16.486 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:16.486 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:16.487 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:16.487 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.487 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:07:16.487 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:07:16.487 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:07:16.487 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:07:16.487 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:07:16.488 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:07:16.488 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:16.488 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:16.488 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:07:16.488 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:07:16.488 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:07:16.488 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:07:16.489 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:07:16.489 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:07:16.489 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:07:16.489 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:16.489 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:07:16.489 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:16.490 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:16.490 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:16.490 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:16.490 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:16.490 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:16.490 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:16.490 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:16.491 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:16.491 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:16.491 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:16.491 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:07:16.491 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.491 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:07:16.491 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.492 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:07:16.492 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:07:16.492 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:16.492 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:16.492 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.492 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:16.492 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:07:16.493 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:16.493 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.493 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.493 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:16.493 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:07:16.493 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:16.493 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.494 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:16.494 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.494 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:07:16.494 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:07:16.494 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:16.494 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:16.494 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:16.495 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.495 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.495 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:07:16.495 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:07:16.495 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:16.495 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:16.495 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.495 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:16.496 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:07:16.496 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:16.496 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.496 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.496 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:16.496 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:07:16.497 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:16.497 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.497 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:16.497 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.497 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:07:16.497 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:07:16.497 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:16.498 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:16.498 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:16.498 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.498 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.498 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:07:16.498 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:07:16.498 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:16.499 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:16.499 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.499 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:16.499 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:07:16.499 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:16.499 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.500 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.500 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:16.500 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:07:16.500 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:16.500 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.500 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:16.500 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.501 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:07:16.501 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:07:16.501 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:16.501 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:16.501 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:16.501 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.502 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:16.502 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.502 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:07:16.502 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:07:16.502 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.502 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:16.502 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:16.503 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:07:16.503 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.503 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:16.503 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:07:16.503 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.503 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:16.504 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:16.504 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:07:16.504 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.504 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:16.504 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:07:16.504 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.505 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:16.505 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:16.505 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:07:16.505 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.505 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:16.505 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:07:16.505 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.506 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:16.506 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:16.506 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:07:16.506 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.506 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:16.506 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.507 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:07:16.507 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:07:16.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.471+0000 7f6da7fff700 1 -- 172.21.15.40:0/141488819 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6d940667d0 msgr2=0x7f6d94068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.471+0000 7f6da7fff700 1 --2- 172.21.15.40:0/141488819 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6d940667d0 0x7f6d94068c90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6da814ebb0 tx=0x7f6d98009240 comp rx=0 tx=0).stop 2024-09-13T18:07:16.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.471+0000 7f6da7fff700 1 -- 172.21.15.40:0/141488819 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da8151850 msgr2=0x7f6da814a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.471+0000 7f6da7fff700 1 --2- 172.21.15.40:0/141488819 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da8151850 0x7f6da814a790 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6da0012000 tx=0x7f6da0004af0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.471+0000 7f6da77fe700 1 -- 172.21.15.40:0/141488819 reap_dead start 2024-09-13T18:07:16.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.471+0000 7f6da77fe700 1 -- 172.21.15.40:0/141488819 reap_dead start 2024-09-13T18:07:16.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.471+0000 7f6da7fff700 1 -- 172.21.15.40:0/141488819 shutdown_connections 2024-09-13T18:07:16.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.471+0000 7f6da7fff700 1 -- 172.21.15.40:0/141488819 >> 172.21.15.40:0/141488819 conn(0x7f6da80ccef0 msgr2=0x7f6da81e2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:16.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.471+0000 7f6da7fff700 1 -- 172.21.15.40:0/141488819 shutdown_connections 2024-09-13T18:07:16.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.471+0000 7f6da7fff700 1 -- 172.21.15.40:0/141488819 wait complete. 2024-09-13T18:07:16.510 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:07:16.510 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:07:16.511 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:07:16.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.623+0000 7f46cdf34700 1 Processor -- start 2024-09-13T18:07:16.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.623+0000 7f46cdf34700 1 -- start start 2024-09-13T18:07:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.623+0000 7f46cdf34700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46c80c4f90 0x7f46c8071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.623+0000 7f46cdf34700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c80706f0 con 0x7f46c8150240 2024-09-13T18:07:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.623+0000 7f46cdf34700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c80c28c0 con 0x7f46c80c54e0 2024-09-13T18:07:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.623+0000 7f46cdf34700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c8070450 con 0x7f46c80c4f90 2024-09-13T18:07:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.623+0000 7f46ccf32700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f46c8150240 0x7f46c8070ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49922/0 (socket says 172.21.15.40:49922) 2024-09-13T18:07:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.623+0000 7f46ccf32700 1 -- 172.21.15.40:0/529232433 learned_addr learned my addr 172.21.15.40:0/529232433 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:07:16.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.623+0000 7f46cd733700 1 --2- 172.21.15.40:0/529232433 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46c80c4f90 0x7f46c8071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.623+0000 7f46c77fe700 1 -- 172.21.15.40:0/529232433 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2576129827 0 0) 0x7f46c80706f0 con 0x7f46c8150240 2024-09-13T18:07:16.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.623+0000 7f46c77fe700 1 -- 172.21.15.40:0/529232433 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46b80035d0 con 0x7f46c8150240 2024-09-13T18:07:16.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.623+0000 7f46c77fe700 1 -- 172.21.15.40:0/529232433 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 180805233 0 0) 0x7f46c80c28c0 con 0x7f46c80c54e0 2024-09-13T18:07:16.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.627+0000 7f46c77fe700 1 -- 172.21.15.40:0/529232433 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46c80706f0 con 0x7f46c80c54e0 2024-09-13T18:07:16.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.627+0000 7f46c77fe700 1 -- 172.21.15.40:0/529232433 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2903287756 0 0) 0x7f46b80035d0 con 0x7f46c8150240 2024-09-13T18:07:16.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.627+0000 7f46cd733700 1 -- 172.21.15.40:0/529232433 >> v1:172.21.15.43:6789/0 conn(0x7f46c80c54e0 legacy=0x7f46c815e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.627+0000 7f46cd733700 1 -- 172.21.15.40:0/529232433 >> v1:172.21.15.40:6789/0 conn(0x7f46c8150240 legacy=0x7f46c8070ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.627+0000 7f46cd733700 1 -- 172.21.15.40:0/529232433 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c8156c00 con 0x7f46c80c4f90 2024-09-13T18:07:16.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.627+0000 7f46cd733700 1 --2- 172.21.15.40:0/529232433 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46c80c4f90 0x7f46c8071660 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009dc0 tx=0x7f46c000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85208bde2300b1c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:16.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.627+0000 7f46c77fe700 1 -- 172.21.15.40:0/529232433 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c000e070 con 0x7f46c80c4f90 2024-09-13T18:07:16.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.627+0000 7f46c77fe700 1 -- 172.21.15.40:0/529232433 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c0003a40 con 0x7f46c80c4f90 2024-09-13T18:07:16.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.627+0000 7f46c77fe700 1 -- 172.21.15.40:0/529232433 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc001670 con 0x7f46c80c4f90 2024-09-13T18:07:16.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.627+0000 7f46cdf34700 1 -- 172.21.15.40:0/529232433 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46c80c4f90 msgr2=0x7f46c8071660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.627+0000 7f46cdf34700 1 --2- 172.21.15.40:0/529232433 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46c80c4f90 0x7f46c8071660 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009dc0 tx=0x7f46c000b330 comp rx=0 tx=0).stop 2024-09-13T18:07:16.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.627+0000 7f46cdf34700 1 -- 172.21.15.40:0/529232433 shutdown_connections 2024-09-13T18:07:16.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.627+0000 7f46cdf34700 1 --2- 172.21.15.40:0/529232433 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46c80c4f90 0x7f46c8071660 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.627+0000 7f46cdf34700 1 -- 172.21.15.40:0/529232433 >> 172.21.15.40:0/529232433 conn(0x7f46c806a390 msgr2=0x7f46c806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:16.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.627+0000 7f46cdf34700 1 -- 172.21.15.40:0/529232433 shutdown_connections 2024-09-13T18:07:16.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.627+0000 7f46cdf34700 1 -- 172.21.15.40:0/529232433 wait complete. 2024-09-13T18:07:16.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46cdf34700 1 Processor -- start 2024-09-13T18:07:16.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46cdf34700 1 -- start start 2024-09-13T18:07:16.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46cdf34700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46c80c4f90 0x7f46c80c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46cdf34700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46c80c54e0 0x7f46c814cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46cdf34700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46c8150240 0x7f46c814d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46cdf34700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c80b9160 con 0x7f46c80c54e0 2024-09-13T18:07:16.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46cdf34700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c81536a0 con 0x7f46c80c4f90 2024-09-13T18:07:16.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46cdf34700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c8153960 con 0x7f46c8150240 2024-09-13T18:07:16.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46c7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46c80c54e0 0x7f46c814cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46c7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46c80c54e0 0x7f46c814cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57322/0 (socket says 172.21.15.40:57322) 2024-09-13T18:07:16.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46c7fff700 1 -- 172.21.15.40:0/3546996877 learned_addr learned my addr 172.21.15.40:0/3546996877 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:16.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46ccf32700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46c80c4f90 0x7f46c80c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46cd733700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46c8150240 0x7f46c814d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46c7fff700 1 -- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46c8150240 msgr2=0x7f46c814d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46c7fff700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46c8150240 0x7f46c814d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46c7fff700 1 -- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46c80c4f90 msgr2=0x7f46c80c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46c7fff700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46c80c4f90 0x7f46c80c72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46c7fff700 1 -- 172.21.15.40:0/3546996877 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c0009a70 con 0x7f46c80c54e0 2024-09-13T18:07:16.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46cd733700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46c8150240 0x7f46c814d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:07:16.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46c7fff700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46c80c54e0 0x7f46c814cc70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f46b400c730 tx=0x7f46b400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:16.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46c57fa700 1 -- 172.21.15.40:0/3546996877 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b40052d0 con 0x7f46c80c54e0 2024-09-13T18:07:16.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46cdf34700 1 -- 172.21.15.40:0/3546996877 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c80b85e0 con 0x7f46c80c54e0 2024-09-13T18:07:16.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46cdf34700 1 -- 172.21.15.40:0/3546996877 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c80b8ba0 con 0x7f46c80c54e0 2024-09-13T18:07:16.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46c57fa700 1 -- 172.21.15.40:0/3546996877 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b4005450 con 0x7f46c80c54e0 2024-09-13T18:07:16.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.631+0000 7f46c57fa700 1 -- 172.21.15.40:0/3546996877 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b40038e0 con 0x7f46c80c54e0 2024-09-13T18:07:16.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.635+0000 7f46cdf34700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46a8002a40 0x7f46a8004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.635+0000 7f46cdf34700 1 -- 172.21.15.40:0/3546996877 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a8007f40 con 0x7f46a8002a40 2024-09-13T18:07:16.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.635+0000 7f46ccf32700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46a8002a40 0x7f46a8004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.635+0000 7f46c57fa700 1 -- 172.21.15.40:0/3546996877 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f46b4003a40 con 0x7f46c80c54e0 2024-09-13T18:07:16.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.635+0000 7f46ccf32700 1 -- 172.21.15.40:0/3546996877 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f46a8008100 con 0x7f46a8002a40 2024-09-13T18:07:16.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.635+0000 7f46ccf32700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46a8002a40 0x7f46a8004f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00f770 tx=0x7f46bc00fb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:16.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.635+0000 7f46c57fa700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f46ac066730 0x7f46ac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.635+0000 7f46cd733700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f46ac066730 0x7f46ac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.635+0000 7f46c57fa700 1 -- 172.21.15.40:0/3546996877 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f46b4083a30 con 0x7f46c80c54e0 2024-09-13T18:07:16.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.635+0000 7f46c57fa700 1 -- 172.21.15.40:0/3546996877 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc011580 con 0x7f46a8002a40 2024-09-13T18:07:16.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.635+0000 7f46cd733700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f46ac066730 0x7f46ac068bf0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009dc0 tx=0x7f46c0011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:16.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.635+0000 7f46c57fa700 1 -- 172.21.15.40:0/3546996877 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f46bc0116e0 con 0x7f46a8002a40 2024-09-13T18:07:16.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.635+0000 7f46c57fa700 1 -- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46a8002a40 msgr2=0x7f46a8004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.635+0000 7f46c57fa700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46a8002a40 0x7f46a8004f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00f770 tx=0x7f46bc00fb30 comp rx=0 tx=0).stop 2024-09-13T18:07:16.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.635+0000 7f46c57fa700 1 -- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46a8002a40 msgr2=0x7f46a8004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:16.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.635+0000 7f46c57fa700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46a8002a40 0x7f46a8004f00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.647+0000 7f46cdf34700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46a8006f30 0x7f46a8007380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.647+0000 7f46cdf34700 1 -- 172.21.15.40:0/3546996877 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a800a4a0 con 0x7f46a8006f30 2024-09-13T18:07:16.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.647+0000 7f46ccf32700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46a8006f30 0x7f46a8007380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.647+0000 7f46ccf32700 1 -- 172.21.15.40:0/3546996877 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f46a80078c0 con 0x7f46a8006f30 2024-09-13T18:07:16.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.647+0000 7f46ccf32700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46a8006f30 0x7f46a8007380 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f46bc016000 tx=0x7f46bc000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:16.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.647+0000 7f46c57fa700 1 -- 172.21.15.40:0/3546996877 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc021070 con 0x7f46a8006f30 2024-09-13T18:07:16.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.647+0000 7f46c57fa700 1 -- 172.21.15.40:0/3546996877 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4949 (secure 0 0 0) 0x7f46bc01cdf0 con 0x7f46a8006f30 2024-09-13T18:07:16.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.647+0000 7f46c57fa700 1 -- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46a8006f30 msgr2=0x7f46a8007380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.647+0000 7f46c57fa700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46a8006f30 0x7f46a8007380 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f46bc016000 tx=0x7f46bc000ed0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.647+0000 7f46c57fa700 1 -- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46a8006f30 msgr2=0x7f46a8007380 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:16.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.647+0000 7f46c57fa700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46a8006f30 0x7f46a8007380 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.659 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:07:16.659 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:07:16.659 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:07:16.659 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:07:16.659 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 140, 2024-09-13T18:07:16.659 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:07:16.660 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:07:16.660 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:07:16.660 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:07:16.660 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:16.660 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 11, 2024-09-13T18:07:16.660 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:16.660 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:07:16.660 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:07:16.660 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:16.660 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:16.660 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:16.660 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:16.660 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:16.661 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:16.661 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:16.661 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:16.661 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:16.661 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:16.661 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:16.661 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:07:16.661 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:07:16.661 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:16.661 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:16.661 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:16.661 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:16.661 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:16.661 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:16.662 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:16.662 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:16.662 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:16.662 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:16.662 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:16.662 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.662 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:07:16.662 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:07:16.662 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:07:16.662 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:07:16.662 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:07:16.662 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:07:16.662 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:16.662 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:16.663 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:07:16.663 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:07:16.663 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:07:16.663 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:07:16.663 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:07:16.663 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:07:16.663 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:07:16.663 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:16.663 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:07:16.663 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:16.663 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:16.663 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:16.663 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:16.664 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:16.664 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:16.664 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:16.664 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:16.664 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:16.664 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:16.664 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:16.664 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:07:16.664 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.664 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:07:16.664 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.664 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:07:16.664 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:07:16.665 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:16.665 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:16.665 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.665 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:16.665 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:07:16.665 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:16.665 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.665 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.665 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:16.665 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:07:16.665 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:16.665 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.665 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:16.665 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.665 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:07:16.666 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:07:16.666 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:16.666 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:16.666 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:16.666 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.666 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.666 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:07:16.666 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:07:16.666 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:16.666 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:16.666 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.666 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:16.666 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:07:16.667 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:16.667 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.667 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.667 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:16.667 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:07:16.667 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:16.667 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.667 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:16.667 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.667 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:07:16.667 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:07:16.667 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:16.667 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:16.667 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:16.668 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.668 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.668 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:07:16.668 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:07:16.668 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:16.668 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:16.668 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.668 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:16.668 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:07:16.668 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:16.668 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.668 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.668 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:16.668 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:07:16.669 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:16.669 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.669 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:16.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.669 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:07:16.669 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:07:16.669 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:16.669 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:16.669 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:16.669 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.669 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:16.669 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.669 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:07:16.669 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:07:16.670 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.670 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:16.670 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:16.670 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:07:16.670 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.670 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:16.670 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:07:16.670 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.670 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:16.670 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:16.670 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:07:16.670 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.670 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:16.670 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:07:16.671 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.671 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:16.671 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:16.671 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:07:16.671 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.671 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:16.671 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:07:16.671 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.671 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:16.671 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:16.671 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:07:16.671 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.671 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:16.671 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:07:16.672 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:16.672 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:16.672 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:16.672 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:07:16.672 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:16.672 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:16.672 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:16.672 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:07:16.672 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:07:16.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.651+0000 7f46cdf34700 1 -- 172.21.15.40:0/3546996877 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f46ac066730 msgr2=0x7f46ac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.651+0000 7f46cdf34700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f46ac066730 0x7f46ac068bf0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009dc0 tx=0x7f46c0011000 comp rx=0 tx=0).stop 2024-09-13T18:07:16.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.651+0000 7f46cdf34700 1 -- 172.21.15.40:0/3546996877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46c80c54e0 msgr2=0x7f46c814cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.651+0000 7f46cdf34700 1 --2- 172.21.15.40:0/3546996877 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46c80c54e0 0x7f46c814cc70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f46b400c730 tx=0x7f46b400ca40 comp rx=0 tx=0).stop 2024-09-13T18:07:16.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.651+0000 7f46cd733700 1 -- 172.21.15.40:0/3546996877 reap_dead start 2024-09-13T18:07:16.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.651+0000 7f46cd733700 1 -- 172.21.15.40:0/3546996877 reap_dead start 2024-09-13T18:07:16.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.651+0000 7f46cdf34700 1 -- 172.21.15.40:0/3546996877 shutdown_connections 2024-09-13T18:07:16.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.651+0000 7f46cdf34700 1 -- 172.21.15.40:0/3546996877 >> 172.21.15.40:0/3546996877 conn(0x7f46c806a390 msgr2=0x7f46c81562c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:16.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.651+0000 7f46cdf34700 1 -- 172.21.15.40:0/3546996877 shutdown_connections 2024-09-13T18:07:16.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.651+0000 7f46cdf34700 1 -- 172.21.15.40:0/3546996877 wait complete. 2024-09-13T18:07:16.675 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:07:16.675 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-13T18:07:16.675 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T18:07:16.675 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-13T18:07:16.675 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-13T18:07:16.675 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T18:07:16.675 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:07:16.715 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T18:07:16.715 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T18:07:16.715 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:07:16.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.819+0000 7fb3c90a9700 1 Processor -- start 2024-09-13T18:07:16.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.819+0000 7fb3c90a9700 1 -- start start 2024-09-13T18:07:16.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.819+0000 7fb3c90a9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3bc0d68c0 0x7fb3bc0dce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.819+0000 7fb3c90a9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3bc007bf0 0x7fb3bc0dfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.819+0000 7fb3c90a9700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3bc0dd3b0 con 0x7fb3bc008010 2024-09-13T18:07:16.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.819+0000 7fb3c90a9700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3bc036bc0 con 0x7fb3bc007bf0 2024-09-13T18:07:16.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.819+0000 7fb3c90a9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc006b40 con 0x7fb3bc0d68c0 2024-09-13T18:07:16.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.819+0000 7fb3c37fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3bc007bf0 0x7fb3bc0dfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.819+0000 7fb3c37fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3bc007bf0 0x7fb3bc0dfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57354/0 (socket says 172.21.15.40:57354) 2024-09-13T18:07:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.819+0000 7fb3c37fe700 1 -- 172.21.15.40:0/48284621 learned_addr learned my addr 172.21.15.40:0/48284621 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.819+0000 7fb3c88a8700 1 -- 172.21.15.40:0/48284621 >> v1:172.21.15.43:6789/0 conn(0x7fb3bc008010 legacy=0x7fb3bc0dff50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:07:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.819+0000 7fb3c3fff700 1 --2- 172.21.15.40:0/48284621 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3bc0d68c0 0x7fb3bc0dce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.819+0000 7fb3c37fe700 1 -- 172.21.15.40:0/48284621 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3bc0d68c0 msgr2=0x7fb3bc0dce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.819+0000 7fb3c37fe700 1 --2- 172.21.15.40:0/48284621 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3bc0d68c0 0x7fb3bc0dce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.819+0000 7fb3c37fe700 1 -- 172.21.15.40:0/48284621 >> v1:172.21.15.43:6789/0 conn(0x7fb3bc008010 legacy=0x7fb3bc0dff50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:07:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.819+0000 7fb3c37fe700 1 -- 172.21.15.40:0/48284621 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3bc036220 con 0x7fb3bc007bf0 2024-09-13T18:07:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.819+0000 7fb3c3fff700 1 --2- 172.21.15.40:0/48284621 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3bc0d68c0 0x7fb3bc0dce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:07:16.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.819+0000 7fb3c37fe700 1 --2- 172.21.15.40:0/48284621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3bc007bf0 0x7fb3bc0dfa10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009910 tx=0x7fb3b0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e97c5d9e23460 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:16.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.819+0000 7fb3c2ffd700 1 -- 172.21.15.40:0/48284621 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0004230 con 0x7fb3bc007bf0 2024-09-13T18:07:16.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.819+0000 7fb3c2ffd700 1 -- 172.21.15.40:0/48284621 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b0004390 con 0x7fb3bc007bf0 2024-09-13T18:07:16.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.823+0000 7fb3c2ffd700 1 -- 172.21.15.40:0/48284621 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0009e60 con 0x7fb3bc007bf0 2024-09-13T18:07:16.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.823+0000 7fb3c90a9700 1 -- 172.21.15.40:0/48284621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3bc007bf0 msgr2=0x7fb3bc0dfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.823+0000 7fb3c90a9700 1 --2- 172.21.15.40:0/48284621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3bc007bf0 0x7fb3bc0dfa10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009910 tx=0x7fb3b0009c20 comp rx=0 tx=0).stop 2024-09-13T18:07:16.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.823+0000 7fb3c90a9700 1 -- 172.21.15.40:0/48284621 shutdown_connections 2024-09-13T18:07:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.823+0000 7fb3c90a9700 1 --2- 172.21.15.40:0/48284621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3bc007bf0 0x7fb3bc0dfa10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.823+0000 7fb3c90a9700 1 --2- 172.21.15.40:0/48284621 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3bc0d68c0 0x7fb3bc0dce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.823+0000 7fb3c90a9700 1 -- 172.21.15.40:0/48284621 >> 172.21.15.40:0/48284621 conn(0x7fb3bc050e60 msgr2=0x7fb3bc051270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.823+0000 7fb3c90a9700 1 -- 172.21.15.40:0/48284621 shutdown_connections 2024-09-13T18:07:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.823+0000 7fb3c90a9700 1 -- 172.21.15.40:0/48284621 wait complete. 2024-09-13T18:07:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.823+0000 7fb3c90a9700 1 Processor -- start 2024-09-13T18:07:16.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.823+0000 7fb3c90a9700 1 -- start start 2024-09-13T18:07:16.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.823+0000 7fb3c90a9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3bc007bf0 0x7fb3bc0d8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.823+0000 7fb3c90a9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3bc008010 0x7fb3bc0dae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.823+0000 7fb3c90a9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3bc0d68c0 0x7fb3bc04ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.823+0000 7fb3c90a9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc0d3af0 con 0x7fb3bc007bf0 2024-09-13T18:07:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.823+0000 7fb3c90a9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc172790 con 0x7fb3bc008010 2024-09-13T18:07:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.823+0000 7fb3c90a9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc0d2e70 con 0x7fb3bc0d68c0 2024-09-13T18:07:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.823+0000 7fb3c37fe700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3bc008010 msgr2=0x7fb3bc0dae70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:07:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.823+0000 7fb3c37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3bc008010 0x7fb3bc0dae70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:07:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.823+0000 7fb3c3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3bc007bf0 0x7fb3bc0d8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.827+0000 7fb3c3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3bc007bf0 0x7fb3bc0d8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57360/0 (socket says 172.21.15.40:57360) 2024-09-13T18:07:16.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.827+0000 7fb3c3fff700 1 -- 172.21.15.40:0/2306694092 learned_addr learned my addr 172.21.15.40:0/2306694092 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:16.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.827+0000 7fb3c88a8700 1 --2- 172.21.15.40:0/2306694092 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3bc0d68c0 0x7fb3bc04ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.827+0000 7fb3c3fff700 1 -- 172.21.15.40:0/2306694092 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3bc0d68c0 msgr2=0x7fb3bc04ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:16.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.827+0000 7fb3c3fff700 1 --2- 172.21.15.40:0/2306694092 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3bc0d68c0 0x7fb3bc04ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.827+0000 7fb3c3fff700 1 -- 172.21.15.40:0/2306694092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3bc008010 msgr2=0x7fb3bc0dae70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:07:16.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.827+0000 7fb3c3fff700 1 --2- 172.21.15.40:0/2306694092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3bc008010 0x7fb3bc0dae70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:16.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.827+0000 7fb3c3fff700 1 -- 172.21.15.40:0/2306694092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b0009570 con 0x7fb3bc007bf0 2024-09-13T18:07:16.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.827+0000 7fb3c88a8700 1 --2- 172.21.15.40:0/2306694092 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3bc0d68c0 0x7fb3bc04ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:07:16.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.827+0000 7fb3c3fff700 1 --2- 172.21.15.40:0/2306694092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3bc007bf0 0x7fb3bc0d8e50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb3b800e810 tx=0x7fb3b800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:16.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.827+0000 7fb3c0ff9700 1 -- 172.21.15.40:0/2306694092 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8004230 con 0x7fb3bc007bf0 2024-09-13T18:07:16.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.827+0000 7fb3c0ff9700 1 -- 172.21.15.40:0/2306694092 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b8004390 con 0x7fb3bc007bf0 2024-09-13T18:07:16.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.827+0000 7fb3c90a9700 1 -- 172.21.15.40:0/2306694092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3bc0d3150 con 0x7fb3bc007bf0 2024-09-13T18:07:16.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.827+0000 7fb3c0ff9700 1 -- 172.21.15.40:0/2306694092 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80036c0 con 0x7fb3bc007bf0 2024-09-13T18:07:16.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.827+0000 7fb3c90a9700 1 -- 172.21.15.40:0/2306694092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3bc0d3710 con 0x7fb3bc007bf0 2024-09-13T18:07:16.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.827+0000 7fb3c90a9700 1 -- 172.21.15.40:0/2306694092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3bc0dce70 con 0x7fb3bc007bf0 2024-09-13T18:07:16.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.827+0000 7fb3c0ff9700 1 -- 172.21.15.40:0/2306694092 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb3b8003840 con 0x7fb3bc007bf0 2024-09-13T18:07:16.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.831+0000 7fb3c0ff9700 1 --2- 172.21.15.40:0/2306694092 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb3a80666e0 0x7fb3a8068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:16.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.831+0000 7fb3c0ff9700 1 -- 172.21.15.40:0/2306694092 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb3b8083f70 con 0x7fb3bc007bf0 2024-09-13T18:07:16.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.831+0000 7fb3c37fe700 1 --2- 172.21.15.40:0/2306694092 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb3a80666e0 0x7fb3a8068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:16.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.831+0000 7fb3c37fe700 1 --2- 172.21.15.40:0/2306694092 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb3a80666e0 0x7fb3a8068ba0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009910 tx=0x7fb3b0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:16.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:16.835+0000 7fb3c0ff9700 1 -- 172.21.15.40:0/2306694092 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb3b8050f20 con 0x7fb3bc007bf0 2024-09-13T18:07:17.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:17.111+0000 7fb3c90a9700 1 -- 172.21.15.40:0/2306694092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb3bc0d9d20 con 0x7fb3bc007bf0 2024-09-13T18:07:17.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:17.111+0000 7fb3c0ff9700 1 -- 172.21.15.40:0/2306694092 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7fb3b8020080 con 0x7fb3bc007bf0 2024-09-13T18:07:17.120 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":140,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:07:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:17.111+0000 7fb3c90a9700 1 -- 172.21.15.40:0/2306694092 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb3a80666e0 msgr2=0x7fb3a8068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:17.111+0000 7fb3c90a9700 1 --2- 172.21.15.40:0/2306694092 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb3a80666e0 0x7fb3a8068ba0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009910 tx=0x7fb3b0008030 comp rx=0 tx=0).stop 2024-09-13T18:07:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:17.111+0000 7fb3c90a9700 1 -- 172.21.15.40:0/2306694092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3bc007bf0 msgr2=0x7fb3bc0d8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:17.111+0000 7fb3c90a9700 1 --2- 172.21.15.40:0/2306694092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3bc007bf0 0x7fb3bc0d8e50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb3b800e810 tx=0x7fb3b800ebd0 comp rx=0 tx=0).stop 2024-09-13T18:07:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:17.111+0000 7fb3c90a9700 1 -- 172.21.15.40:0/2306694092 shutdown_connections 2024-09-13T18:07:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:17.111+0000 7fb3c90a9700 1 --2- 172.21.15.40:0/2306694092 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb3a80666e0 0x7fb3a8068ba0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:17.111+0000 7fb3c90a9700 1 --2- 172.21.15.40:0/2306694092 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3bc0d68c0 0x7fb3bc04ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:17.111+0000 7fb3c90a9700 1 --2- 172.21.15.40:0/2306694092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3bc008010 0x7fb3bc0dae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:17.111+0000 7fb3c90a9700 1 --2- 172.21.15.40:0/2306694092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3bc007bf0 0x7fb3bc0d8e50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:17.111+0000 7fb3c90a9700 1 -- 172.21.15.40:0/2306694092 >> 172.21.15.40:0/2306694092 conn(0x7fb3bc050e60 msgr2=0x7fb3bc0d4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:17.111+0000 7fb3c90a9700 1 -- 172.21.15.40:0/2306694092 shutdown_connections 2024-09-13T18:07:17.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:17.115+0000 7fb3c90a9700 1 -- 172.21.15.40:0/2306694092 wait complete. 2024-09-13T18:07:20.140 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:07:20.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.255+0000 7f4405663700 1 Processor -- start 2024-09-13T18:07:20.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.255+0000 7f4405663700 1 -- start start 2024-09-13T18:07:20.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.255+0000 7f4405663700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f44001503d0 0x7f4400156940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:20.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.255+0000 7f4405663700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f440006b4a0 0x7f4400071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:20.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.255+0000 7f4405663700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440015a520 con 0x7f4400072130 2024-09-13T18:07:20.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.255+0000 7f4405663700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44001564b0 con 0x7f44001503d0 2024-09-13T18:07:20.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.255+0000 7f4405663700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440015a000 con 0x7f440006b4a0 2024-09-13T18:07:20.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.255+0000 7f43fffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f44001503d0 0x7f4400156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:20.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.255+0000 7f4404e62700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7f4400072130 legacy=0x7f440006cfc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:07:20.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.255+0000 7f43fffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f44001503d0 0x7f4400156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57362/0 (socket says 172.21.15.40:57362) 2024-09-13T18:07:20.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.255+0000 7f43fffff700 1 -- 172.21.15.40:0/3941139657 learned_addr learned my addr 172.21.15.40:0/3941139657 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:20.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.259+0000 7f43f7fff700 1 --2- 172.21.15.40:0/3941139657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f440006b4a0 0x7f4400071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:20.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.259+0000 7f43fffff700 1 -- 172.21.15.40:0/3941139657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f440006b4a0 msgr2=0x7f4400071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:20.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.259+0000 7f43fffff700 1 --2- 172.21.15.40:0/3941139657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f440006b4a0 0x7f4400071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:20.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.259+0000 7f43fffff700 1 -- 172.21.15.40:0/3941139657 >> v1:172.21.15.43:6789/0 conn(0x7f4400072130 legacy=0x7f440006cfc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:07:20.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.259+0000 7f43fffff700 1 -- 172.21.15.40:0/3941139657 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440015a280 con 0x7f44001503d0 2024-09-13T18:07:20.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.259+0000 7f43fffff700 1 --2- 172.21.15.40:0/3941139657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44001503d0 0x7f4400156940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f43f000bbb0 tx=0x7f43f000bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34286025bd1cfe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:20.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.259+0000 7f43ff7fe700 1 -- 172.21.15.40:0/3941139657 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f000dd50 con 0x7f44001503d0 2024-09-13T18:07:20.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.259+0000 7f43ff7fe700 1 -- 172.21.15.40:0/3941139657 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f0004030 con 0x7f44001503d0 2024-09-13T18:07:20.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.259+0000 7f43ff7fe700 1 -- 172.21.15.40:0/3941139657 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f00043e0 con 0x7f44001503d0 2024-09-13T18:07:20.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.259+0000 7f4405663700 1 -- 172.21.15.40:0/3941139657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44001503d0 msgr2=0x7f4400156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:20.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.259+0000 7f4405663700 1 --2- 172.21.15.40:0/3941139657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44001503d0 0x7f4400156940 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f43f000bbb0 tx=0x7f43f000bec0 comp rx=0 tx=0).stop 2024-09-13T18:07:20.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.259+0000 7f4405663700 1 -- 172.21.15.40:0/3941139657 shutdown_connections 2024-09-13T18:07:20.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.259+0000 7f4405663700 1 --2- 172.21.15.40:0/3941139657 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f440006b4a0 0x7f4400071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:20.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.259+0000 7f4405663700 1 --2- 172.21.15.40:0/3941139657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44001503d0 0x7f4400156940 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:20.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.259+0000 7f4405663700 1 -- 172.21.15.40:0/3941139657 >> 172.21.15.40:0/3941139657 conn(0x7f440006e4f0 msgr2=0x7f440006e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:20.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.259+0000 7f4405663700 1 -- 172.21.15.40:0/3941139657 shutdown_connections 2024-09-13T18:07:20.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.259+0000 7f4405663700 1 -- 172.21.15.40:0/3941139657 wait complete. 2024-09-13T18:07:20.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.259+0000 7f4405663700 1 Processor -- start 2024-09-13T18:07:20.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.259+0000 7f4405663700 1 -- start start 2024-09-13T18:07:20.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f4405663700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f440006b4a0 0x7f4400155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:20.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f4405663700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4400072130 0x7f440014eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:20.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f4405663700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44001503d0 0x7f44001ef800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:20.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f4405663700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44000c5c90 con 0x7f4400072130 2024-09-13T18:07:20.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f4405663700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44001ef3b0 con 0x7f44001503d0 2024-09-13T18:07:20.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f4405663700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44000c4c30 con 0x7f440006b4a0 2024-09-13T18:07:20.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f4404e62700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44001503d0 msgr2=0x7f44001ef800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:07:20.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f43f7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4400072130 0x7f440014eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f4404e62700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44001503d0 0x7f44001ef800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:07:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f43f7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4400072130 0x7f440014eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57364/0 (socket says 172.21.15.40:57364) 2024-09-13T18:07:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f43f7fff700 1 -- 172.21.15.40:0/3789521014 learned_addr learned my addr 172.21.15.40:0/3789521014 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f43fffff700 1 --2- 172.21.15.40:0/3789521014 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f440006b4a0 0x7f4400155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f43f7fff700 1 -- 172.21.15.40:0/3789521014 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f440006b4a0 msgr2=0x7f4400155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f43f7fff700 1 --2- 172.21.15.40:0/3789521014 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f440006b4a0 0x7f4400155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f43f7fff700 1 -- 172.21.15.40:0/3789521014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44001503d0 msgr2=0x7f44001ef800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:07:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f43f7fff700 1 --2- 172.21.15.40:0/3789521014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44001503d0 0x7f44001ef800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f43f7fff700 1 -- 172.21.15.40:0/3789521014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f000b810 con 0x7f4400072130 2024-09-13T18:07:20.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f43f7fff700 1 --2- 172.21.15.40:0/3789521014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4400072130 0x7f440014eeb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f43e800e760 tx=0x7f43e800ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:20.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f43fd7fa700 1 -- 172.21.15.40:0/3789521014 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8013610 con 0x7f4400072130 2024-09-13T18:07:20.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f43fd7fa700 1 -- 172.21.15.40:0/3789521014 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e8013770 con 0x7f4400072130 2024-09-13T18:07:20.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f4405663700 1 -- 172.21.15.40:0/3789521014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4400155e60 con 0x7f4400072130 2024-09-13T18:07:20.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f43fd7fa700 1 -- 172.21.15.40:0/3789521014 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8013ae0 con 0x7f4400072130 2024-09-13T18:07:20.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f4405663700 1 -- 172.21.15.40:0/3789521014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44000c53f0 con 0x7f4400072130 2024-09-13T18:07:20.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f4405663700 1 -- 172.21.15.40:0/3789521014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44001560d0 con 0x7f4400072130 2024-09-13T18:07:20.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.263+0000 7f43fd7fa700 1 -- 172.21.15.40:0/3789521014 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f43e8013c40 con 0x7f4400072130 2024-09-13T18:07:20.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.267+0000 7f43fd7fa700 1 --2- 172.21.15.40:0/3789521014 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f43e00666e0 0x7f43e0068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:20.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.267+0000 7f43fd7fa700 1 -- 172.21.15.40:0/3789521014 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f43e8084100 con 0x7f4400072130 2024-09-13T18:07:20.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.267+0000 7f43fffff700 1 --2- 172.21.15.40:0/3789521014 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f43e00666e0 0x7f43e0068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:20.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.267+0000 7f43fffff700 1 --2- 172.21.15.40:0/3789521014 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f43e00666e0 0x7f43e0068ba0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f43f0000c00 tx=0x7f43f001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:20.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.275+0000 7f43fd7fa700 1 -- 172.21.15.40:0/3789521014 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f43e80510b0 con 0x7f4400072130 2024-09-13T18:07:20.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.551+0000 7f4405663700 1 -- 172.21.15.40:0/3789521014 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f44000b18b0 con 0x7f4400072130 2024-09-13T18:07:20.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.551+0000 7f43fd7fa700 1 -- 172.21.15.40:0/3789521014 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f43e8021620 con 0x7f4400072130 2024-09-13T18:07:20.561 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":140,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:07:20.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.555+0000 7f4405663700 1 -- 172.21.15.40:0/3789521014 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f43e00666e0 msgr2=0x7f43e0068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:20.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.555+0000 7f4405663700 1 --2- 172.21.15.40:0/3789521014 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f43e00666e0 0x7f43e0068ba0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f43f0000c00 tx=0x7f43f001c040 comp rx=0 tx=0).stop 2024-09-13T18:07:20.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.555+0000 7f4405663700 1 -- 172.21.15.40:0/3789521014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4400072130 msgr2=0x7f440014eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:20.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.555+0000 7f4405663700 1 --2- 172.21.15.40:0/3789521014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4400072130 0x7f440014eeb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f43e800e760 tx=0x7f43e800ea70 comp rx=0 tx=0).stop 2024-09-13T18:07:20.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.555+0000 7f4405663700 1 -- 172.21.15.40:0/3789521014 shutdown_connections 2024-09-13T18:07:20.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.555+0000 7f4405663700 1 --2- 172.21.15.40:0/3789521014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44001503d0 0x7f44001ef800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:20.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.555+0000 7f4405663700 1 --2- 172.21.15.40:0/3789521014 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4400072130 0x7f440014eeb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:20.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.555+0000 7f4405663700 1 --2- 172.21.15.40:0/3789521014 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f43e00666e0 0x7f43e0068ba0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:20.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.555+0000 7f4405663700 1 --2- 172.21.15.40:0/3789521014 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f440006b4a0 0x7f4400155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:20.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.555+0000 7f4405663700 1 -- 172.21.15.40:0/3789521014 >> 172.21.15.40:0/3789521014 conn(0x7f440006e4f0 msgr2=0x7f44000c2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:20.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.555+0000 7f4405663700 1 -- 172.21.15.40:0/3789521014 shutdown_connections 2024-09-13T18:07:20.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:20.555+0000 7f4405663700 1 -- 172.21.15.40:0/3789521014 wait complete. 2024-09-13T18:07:23.579 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:07:23.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.683+0000 7f158c928700 1 Processor -- start 2024-09-13T18:07:23.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.683+0000 7f158c928700 1 -- start start 2024-09-13T18:07:23.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f158c928700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1588153810 0x7f15880ca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:23.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f158c928700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f15880c3da0 0x7f1588159d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:23.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f158c928700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158806d400 con 0x7f15880cafd0 2024-09-13T18:07:23.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f158c928700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158815a260 con 0x7f15880c3da0 2024-09-13T18:07:23.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f158c928700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158806ce40 con 0x7f1588153810 2024-09-13T18:07:23.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f157ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f15880c3da0 0x7f1588159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:23.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f1587fff700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7f15880cafd0 legacy=0x7f158806a390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:07:23.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f15877fe700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7f1588153810 msgr2=0x7f15880ca390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:07:23.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f157ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f15880c3da0 0x7f1588159d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34442/0 (socket says 172.21.15.40:34442) 2024-09-13T18:07:23.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f157ffff700 1 -- 172.21.15.40:0/3359092682 learned_addr learned my addr 172.21.15.40:0/3359092682 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:23.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f15877fe700 1 --2- 172.21.15.40:0/3359092682 >> v2:172.21.15.43:3300/0 conn(0x7f1588153810 0x7f15880ca390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:07:23.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f157ffff700 1 -- 172.21.15.40:0/3359092682 >> v2:172.21.15.43:3300/0 conn(0x7f1588153810 msgr2=0x7f15880ca390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:07:23.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f157ffff700 1 --2- 172.21.15.40:0/3359092682 >> v2:172.21.15.43:3300/0 conn(0x7f1588153810 0x7f15880ca390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:23.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f157ffff700 1 -- 172.21.15.40:0/3359092682 >> v1:172.21.15.43:6789/0 conn(0x7f15880cafd0 legacy=0x7f158806a390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:07:23.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f157ffff700 1 -- 172.21.15.40:0/3359092682 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158806d120 con 0x7f15880c3da0 2024-09-13T18:07:23.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f157ffff700 1 --2- 172.21.15.40:0/3359092682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15880c3da0 0x7f1588159d20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1588066f90 tx=0x7f1570009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7764387e4be2d2c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:23.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f1586ffd700 1 -- 172.21.15.40:0/3359092682 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1570004230 con 0x7f15880c3da0 2024-09-13T18:07:23.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f1586ffd700 1 -- 172.21.15.40:0/3359092682 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1570004390 con 0x7f15880c3da0 2024-09-13T18:07:23.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f1586ffd700 1 -- 172.21.15.40:0/3359092682 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15700036c0 con 0x7f15880c3da0 2024-09-13T18:07:23.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f158c928700 1 -- 172.21.15.40:0/3359092682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15880c3da0 msgr2=0x7f1588159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:23.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f158c928700 1 --2- 172.21.15.40:0/3359092682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15880c3da0 0x7f1588159d20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1588066f90 tx=0x7f1570009bf0 comp rx=0 tx=0).stop 2024-09-13T18:07:23.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f158c928700 1 -- 172.21.15.40:0/3359092682 shutdown_connections 2024-09-13T18:07:23.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f158c928700 1 --2- 172.21.15.40:0/3359092682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15880c3da0 0x7f1588159d20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:23.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f158c928700 1 --2- 172.21.15.40:0/3359092682 >> v2:172.21.15.43:3300/0 conn(0x7f1588153810 0x7f15880ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:23.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f158c928700 1 -- 172.21.15.40:0/3359092682 >> 172.21.15.40:0/3359092682 conn(0x7f15880cbca0 msgr2=0x7f15880cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:23.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f158c928700 1 -- 172.21.15.40:0/3359092682 shutdown_connections 2024-09-13T18:07:23.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.687+0000 7f158c928700 1 -- 172.21.15.40:0/3359092682 wait complete. 2024-09-13T18:07:23.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f158c928700 1 Processor -- start 2024-09-13T18:07:23.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f158c928700 1 -- start start 2024-09-13T18:07:23.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f158c928700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15880c3da0 0x7f15881f2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:23.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f158c928700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15880cafd0 0x7f15881f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:23.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f158c928700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1588153810 0x7f15880c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:23.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f158c928700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15881f3450 con 0x7f15880cafd0 2024-09-13T18:07:23.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f158c928700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15880b1790 con 0x7f1588153810 2024-09-13T18:07:23.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f158c928700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15881e4990 con 0x7f15880c3da0 2024-09-13T18:07:23.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f1587fff700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1588153810 msgr2=0x7f15880c7b40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:07:23.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f1587fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1588153810 0x7f15880c7b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:07:23.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f157ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15880cafd0 0x7f15881f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:23.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f15877fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15880c3da0 0x7f15881f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:23.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f157ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15880cafd0 0x7f15881f2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34454/0 (socket says 172.21.15.40:34454) 2024-09-13T18:07:23.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f157ffff700 1 -- 172.21.15.40:0/3588882377 learned_addr learned my addr 172.21.15.40:0/3588882377 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:23.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f157ffff700 1 -- 172.21.15.40:0/3588882377 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15880c3da0 msgr2=0x7f15881f2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:23.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f157ffff700 1 --2- 172.21.15.40:0/3588882377 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15880c3da0 0x7f15881f2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:23.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f157ffff700 1 -- 172.21.15.40:0/3588882377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1588153810 msgr2=0x7f15880c7b40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:07:23.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f157ffff700 1 --2- 172.21.15.40:0/3588882377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1588153810 0x7f15880c7b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:23.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f157ffff700 1 -- 172.21.15.40:0/3588882377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1570009570 con 0x7f15880cafd0 2024-09-13T18:07:23.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f157ffff700 1 --2- 172.21.15.40:0/3588882377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15880cafd0 0x7f15881f2da0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1570009c30 tx=0x7f1570004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:23.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f1584ff9700 1 -- 172.21.15.40:0/3588882377 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157000b860 con 0x7f15880cafd0 2024-09-13T18:07:23.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f1584ff9700 1 -- 172.21.15.40:0/3588882377 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157000b9c0 con 0x7f15880cafd0 2024-09-13T18:07:23.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f158c928700 1 -- 172.21.15.40:0/3588882377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15881e4c70 con 0x7f15880cafd0 2024-09-13T18:07:23.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f1584ff9700 1 -- 172.21.15.40:0/3588882377 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157000bd30 con 0x7f15880cafd0 2024-09-13T18:07:23.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.691+0000 7f158c928700 1 -- 172.21.15.40:0/3588882377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15881e5380 con 0x7f15880cafd0 2024-09-13T18:07:23.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.695+0000 7f1584ff9700 1 -- 172.21.15.40:0/3588882377 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f157001b440 con 0x7f15880cafd0 2024-09-13T18:07:23.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.695+0000 7f1584ff9700 1 --2- 172.21.15.40:0/3588882377 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1568066710 0x7f1568068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:23.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.695+0000 7f158c928700 1 -- 172.21.15.40:0/3588882377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15880cb400 con 0x7f15880cafd0 2024-09-13T18:07:23.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.695+0000 7f1584ff9700 1 -- 172.21.15.40:0/3588882377 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1570085da0 con 0x7f15880cafd0 2024-09-13T18:07:23.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.695+0000 7f15877fe700 1 --2- 172.21.15.40:0/3588882377 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1568066710 0x7f1568068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:23.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.695+0000 7f15877fe700 1 --2- 172.21.15.40:0/3588882377 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1568066710 0x7f1568068bd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1578009470 tx=0x7f1578009400 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:23.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.703+0000 7f1584ff9700 1 -- 172.21.15.40:0/3588882377 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1570052de0 con 0x7f15880cafd0 2024-09-13T18:07:23.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.951+0000 7f158c928700 1 -- 172.21.15.40:0/3588882377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f15880c5bd0 con 0x7f15880cafd0 2024-09-13T18:07:23.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.951+0000 7f1584ff9700 1 -- 172.21.15.40:0/3588882377 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f157001f070 con 0x7f15880cafd0 2024-09-13T18:07:23.961 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":140,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:07:23.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.955+0000 7f158c928700 1 -- 172.21.15.40:0/3588882377 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1568066710 msgr2=0x7f1568068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:23.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.955+0000 7f158c928700 1 --2- 172.21.15.40:0/3588882377 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1568066710 0x7f1568068bd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1578009470 tx=0x7f1578009400 comp rx=0 tx=0).stop 2024-09-13T18:07:23.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.955+0000 7f158c928700 1 -- 172.21.15.40:0/3588882377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15880cafd0 msgr2=0x7f15881f2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:23.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.955+0000 7f158c928700 1 --2- 172.21.15.40:0/3588882377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15880cafd0 0x7f15881f2da0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1570009c30 tx=0x7f1570004770 comp rx=0 tx=0).stop 2024-09-13T18:07:23.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.955+0000 7f158c928700 1 -- 172.21.15.40:0/3588882377 shutdown_connections 2024-09-13T18:07:23.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.955+0000 7f158c928700 1 --2- 172.21.15.40:0/3588882377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1588153810 0x7f15880c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:23.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.955+0000 7f158c928700 1 --2- 172.21.15.40:0/3588882377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15880cafd0 0x7f15881f2da0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:23.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.955+0000 7f158c928700 1 --2- 172.21.15.40:0/3588882377 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1568066710 0x7f1568068bd0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:23.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.955+0000 7f158c928700 1 --2- 172.21.15.40:0/3588882377 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15880c3da0 0x7f15881f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:23.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.955+0000 7f158c928700 1 -- 172.21.15.40:0/3588882377 >> 172.21.15.40:0/3588882377 conn(0x7f15880cbca0 msgr2=0x7f15880ccce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:23.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.955+0000 7f158c928700 1 -- 172.21.15.40:0/3588882377 shutdown_connections 2024-09-13T18:07:23.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:23.955+0000 7f158c928700 1 -- 172.21.15.40:0/3588882377 wait complete. 2024-09-13T18:07:26.977 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:07:27.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.087+0000 7f559217d700 1 Processor -- start 2024-09-13T18:07:27.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.087+0000 7f559217d700 1 -- start start 2024-09-13T18:07:27.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.087+0000 7f559217d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f558c070d60 0x7f558c066290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:27.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.087+0000 7f559217d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 0x7f558c1532b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:27.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.087+0000 7f559217d700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f558c0c76b0 con 0x7f558c070980 2024-09-13T18:07:27.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.087+0000 7f559217d700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f558c06dbf0 con 0x7f558c06e010 2024-09-13T18:07:27.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.087+0000 7f559217d700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f558c0667d0 con 0x7f558c070d60 2024-09-13T18:07:27.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.087+0000 7f559197c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 0x7f558c1532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:27.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.087+0000 7f559097a700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7f558c070d60 msgr2=0x7f558c066290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:07:27.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.087+0000 7f559117b700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f558c070980 0x7f558c15d3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40256/0 (socket says 172.21.15.40:40256) 2024-09-13T18:07:27.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.087+0000 7f559097a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f558c070d60 0x7f558c066290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:07:27.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.087+0000 7f559117b700 1 -- 172.21.15.40:0/1348858650 learned_addr learned my addr 172.21.15.40:0/1348858650 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:07:27.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.087+0000 7f559197c700 1 -- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 msgr2=0x7f558c1532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:07:27.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.087+0000 7f559197c700 1 -- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 msgr2=0x7f558c1532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:07:27.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.087+0000 7f559197c700 1 --2- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 0x7f558c1532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:07:27.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.087+0000 7f559197c700 1 --2- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 0x7f558c1532b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:07:27.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.291+0000 7f559097a700 1 -- 172.21.15.40:0/1348858650 >> v2:172.21.15.43:3300/0 conn(0x7f558c070d60 msgr2=0x7f558c066290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:07:27.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.291+0000 7f559097a700 1 --2- 172.21.15.40:0/1348858650 >> v2:172.21.15.43:3300/0 conn(0x7f558c070d60 0x7f558c066290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:07:27.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.291+0000 7f559197c700 1 --2- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 0x7f558c1532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:27.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.291+0000 7f559197c700 1 -- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 msgr2=0x7f558c1532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:07:27.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.291+0000 7f559197c700 1 -- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 msgr2=0x7f558c1532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:07:27.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.291+0000 7f559197c700 1 --2- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 0x7f558c1532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:07:27.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.291+0000 7f559197c700 1 --2- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 0x7f558c1532b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:07:27.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.695+0000 7f559097a700 1 -- 172.21.15.40:0/1348858650 >> v2:172.21.15.43:3300/0 conn(0x7f558c070d60 msgr2=0x7f558c066290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:07:27.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.695+0000 7f559197c700 1 --2- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 0x7f558c1532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:27.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.695+0000 7f559097a700 1 --2- 172.21.15.40:0/1348858650 >> v2:172.21.15.43:3300/0 conn(0x7f558c070d60 0x7f558c066290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:07:27.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.695+0000 7f559197c700 1 -- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 msgr2=0x7f558c1532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:07:27.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.695+0000 7f559197c700 1 -- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 msgr2=0x7f558c1532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:07:27.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.695+0000 7f559197c700 1 --2- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 0x7f558c1532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:07:27.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:27.695+0000 7f559197c700 1 --2- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 0x7f558c1532b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:07:28.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:28.499+0000 7f559197c700 1 --2- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 0x7f558c1532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:28.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:28.499+0000 7f559097a700 1 -- 172.21.15.40:0/1348858650 >> v2:172.21.15.43:3300/0 conn(0x7f558c070d60 msgr2=0x7f558c066290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:07:28.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:28.499+0000 7f559097a700 1 --2- 172.21.15.40:0/1348858650 >> v2:172.21.15.43:3300/0 conn(0x7f558c070d60 0x7f558c066290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:07:28.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:28.499+0000 7f559197c700 1 -- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 msgr2=0x7f558c1532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:07:28.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:28.499+0000 7f559197c700 1 -- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 msgr2=0x7f558c1532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:07:28.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:28.499+0000 7f559197c700 1 --2- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 0x7f558c1532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:07:28.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:28.499+0000 7f559197c700 1 --2- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 0x7f558c1532b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:07:30.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.087+0000 7f5582ffd700 1 -- 172.21.15.40:0/1348858650 >> v2:172.21.15.43:3300/0 conn(0x7f558c070d60 msgr2=0x7f558c066290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:07:30.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.087+0000 7f5582ffd700 1 --2- 172.21.15.40:0/1348858650 >> v2:172.21.15.43:3300/0 conn(0x7f558c070d60 0x7f558c066290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:30.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.087+0000 7f5582ffd700 1 -- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 msgr2=0x7f558c1532b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:07:30.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.087+0000 7f5582ffd700 1 --2- 172.21.15.40:0/1348858650 >> v2:172.21.15.40:3300/0 conn(0x7f558c06e010 0x7f558c1532b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:30.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.087+0000 7f5582ffd700 1 -- 172.21.15.40:0/1348858650 >> v1:172.21.15.40:6789/0 conn(0x7f558c070980 legacy=0x7f558c15d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:30.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.087+0000 7f5582ffd700 1 --2- 172.21.15.40:0/1348858650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5570003710 0x7f5570005b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:30.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.087+0000 7f5582ffd700 1 -- 172.21.15.40:0/1348858650 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f558c0c7990 con 0x7f5570006280 2024-09-13T18:07:30.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.087+0000 7f5582ffd700 1 -- 172.21.15.40:0/1348858650 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558c06dbf0 con 0x7f5570003710 2024-09-13T18:07:30.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.087+0000 7f559097a700 1 -- 172.21.15.40:0/1348858650 >> v1:172.21.15.43:6789/0 conn(0x7f5570006280 legacy=0x7f5570008720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:07:30.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.087+0000 7f559117b700 1 --2- 172.21.15.40:0/1348858650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5570003710 0x7f5570005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:30.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.087+0000 7f559117b700 1 -- 172.21.15.40:0/1348858650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5570003710 msgr2=0x7f5570005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:07:30.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.087+0000 7f559117b700 1 -- 172.21.15.40:0/1348858650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5570003710 msgr2=0x7f5570005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:07:30.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.087+0000 7f559117b700 1 --2- 172.21.15.40:0/1348858650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5570003710 0x7f5570005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:07:30.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.087+0000 7f559117b700 1 --2- 172.21.15.40:0/1348858650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5570003710 0x7f5570005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:07:30.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.291+0000 7f559097a700 1 -- 172.21.15.40:0/1348858650 >> v1:172.21.15.43:6789/0 conn(0x7f5570006280 legacy=0x7f5570008720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:07:30.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.291+0000 7f559117b700 1 --2- 172.21.15.40:0/1348858650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5570003710 0x7f5570005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:30.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.291+0000 7f559117b700 1 -- 172.21.15.40:0/1348858650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5570003710 msgr2=0x7f5570005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:07:30.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.291+0000 7f559117b700 1 -- 172.21.15.40:0/1348858650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5570003710 msgr2=0x7f5570005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:07:30.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.291+0000 7f559117b700 1 --2- 172.21.15.40:0/1348858650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5570003710 0x7f5570005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:07:30.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.291+0000 7f559117b700 1 --2- 172.21.15.40:0/1348858650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5570003710 0x7f5570005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:07:30.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.695+0000 7f559097a700 1 -- 172.21.15.40:0/1348858650 >> v1:172.21.15.43:6789/0 conn(0x7f5570006280 legacy=0x7f5570008720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:07:30.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.695+0000 7f559117b700 1 --2- 172.21.15.40:0/1348858650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5570003710 0x7f5570005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:30.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.695+0000 7f559117b700 1 -- 172.21.15.40:0/1348858650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5570003710 msgr2=0x7f5570005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:07:30.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.695+0000 7f559117b700 1 -- 172.21.15.40:0/1348858650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5570003710 msgr2=0x7f5570005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:07:30.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.695+0000 7f559117b700 1 --2- 172.21.15.40:0/1348858650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5570003710 0x7f5570005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:07:30.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:30.695+0000 7f559117b700 1 --2- 172.21.15.40:0/1348858650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5570003710 0x7f5570005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:07:31.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.499+0000 7f559097a700 1 -- 172.21.15.40:0/1348858650 >> v1:172.21.15.43:6789/0 conn(0x7f5570006280 legacy=0x7f5570008720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:07:31.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.499+0000 7f559117b700 1 --2- 172.21.15.40:0/1348858650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5570003710 0x7f5570005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:31.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.499+0000 7f559117b700 1 -- 172.21.15.40:0/1348858650 >> v1:172.21.15.43:6789/0 conn(0x7f5570006280 legacy=0x7f5570008720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:07:31.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.499+0000 7f559117b700 1 -- 172.21.15.40:0/1348858650 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557000b0b0 con 0x7f5570003710 2024-09-13T18:07:31.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.499+0000 7f559117b700 1 --2- 172.21.15.40:0/1348858650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5570003710 0x7f5570005b60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5588005d90 tx=0x7f5588007330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a815546e00616b66 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:31.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.503+0000 7f5583fff700 1 -- 172.21.15.40:0/1348858650 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588005e80 con 0x7f5570003710 2024-09-13T18:07:31.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.503+0000 7f5583fff700 1 -- 172.21.15.40:0/1348858650 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5588016480 con 0x7f5570003710 2024-09-13T18:07:31.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.503+0000 7f5583fff700 1 -- 172.21.15.40:0/1348858650 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55880167f0 con 0x7f5570003710 2024-09-13T18:07:31.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559217d700 1 -- 172.21.15.40:0/1348858650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5570003710 msgr2=0x7f5570005b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:31.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559217d700 1 --2- 172.21.15.40:0/1348858650 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5570003710 0x7f5570005b60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5588005d90 tx=0x7f5588007330 comp rx=0 tx=0).stop 2024-09-13T18:07:31.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559197c700 1 -- 172.21.15.40:0/1348858650 reap_dead start 2024-09-13T18:07:31.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559217d700 1 -- 172.21.15.40:0/1348858650 shutdown_connections 2024-09-13T18:07:31.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559217d700 1 -- 172.21.15.40:0/1348858650 >> 172.21.15.40:0/1348858650 conn(0x7f558c0c7d70 msgr2=0x7f558c0c8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:31.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559217d700 1 -- 172.21.15.40:0/1348858650 shutdown_connections 2024-09-13T18:07:31.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559217d700 1 -- 172.21.15.40:0/1348858650 wait complete. 2024-09-13T18:07:31.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559217d700 1 Processor -- start 2024-09-13T18:07:31.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559217d700 1 -- start start 2024-09-13T18:07:31.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559217d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f558c148f50 0x7f558c1ee6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:31.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559217d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f558c149330 0x7f558c1eebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:31.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559217d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f558c14c6a0 0x7f558c14a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:31.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559217d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558c14a4e0 con 0x7f558c149330 2024-09-13T18:07:31.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559217d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558c0c5360 con 0x7f558c148f50 2024-09-13T18:07:31.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559217d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558c1ef2c0 con 0x7f558c14c6a0 2024-09-13T18:07:31.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559117b700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f558c148f50 msgr2=0x7f558c1ee6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:07:31.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559097a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f558c149330 0x7f558c1eebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:31.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559117b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f558c148f50 0x7f558c1ee6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:07:31.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559097a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f558c149330 0x7f558c1eebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33298/0 (socket says 172.21.15.40:33298) 2024-09-13T18:07:31.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559097a700 1 -- 172.21.15.40:0/1062419709 learned_addr learned my addr 172.21.15.40:0/1062419709 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:31.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.507+0000 7f559197c700 1 --2- 172.21.15.40:0/1062419709 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f558c14c6a0 0x7f558c14a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:31.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.511+0000 7f559097a700 1 -- 172.21.15.40:0/1062419709 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f558c14c6a0 msgr2=0x7f558c14a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:31.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.511+0000 7f559097a700 1 --2- 172.21.15.40:0/1062419709 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f558c14c6a0 0x7f558c14a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:31.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.511+0000 7f559097a700 1 -- 172.21.15.40:0/1062419709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f558c148f50 msgr2=0x7f558c1ee6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:07:31.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.511+0000 7f559097a700 1 --2- 172.21.15.40:0/1062419709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f558c148f50 0x7f558c1ee6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:31.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.511+0000 7f559097a700 1 -- 172.21.15.40:0/1062419709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f558800f040 con 0x7f558c149330 2024-09-13T18:07:31.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.511+0000 7f559097a700 1 --2- 172.21.15.40:0/1062419709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f558c149330 0x7f558c1eebf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f557c003e20 tx=0x7f557c00bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:31.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.511+0000 7f5581ffb700 1 -- 172.21.15.40:0/1062419709 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c00c980 con 0x7f558c149330 2024-09-13T18:07:31.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.511+0000 7f559217d700 1 -- 172.21.15.40:0/1062419709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f558c1ef5a0 con 0x7f558c149330 2024-09-13T18:07:31.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.511+0000 7f559217d700 1 -- 172.21.15.40:0/1062419709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f558c149e20 con 0x7f558c149330 2024-09-13T18:07:31.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.511+0000 7f5581ffb700 1 -- 172.21.15.40:0/1062419709 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557c00a040 con 0x7f558c149330 2024-09-13T18:07:31.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.511+0000 7f5581ffb700 1 -- 172.21.15.40:0/1062419709 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c00cce0 con 0x7f558c149330 2024-09-13T18:07:31.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.515+0000 7f5581ffb700 1 -- 172.21.15.40:0/1062419709 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f557c012030 con 0x7f558c149330 2024-09-13T18:07:31.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.515+0000 7f559217d700 1 -- 172.21.15.40:0/1062419709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556c005310 con 0x7f558c149330 2024-09-13T18:07:31.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.515+0000 7f5581ffb700 1 --2- 172.21.15.40:0/1062419709 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5570067550 0x7f5570069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:31.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.515+0000 7f559117b700 1 --2- 172.21.15.40:0/1062419709 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5570067550 0x7f5570069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:31.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.515+0000 7f5581ffb700 1 -- 172.21.15.40:0/1062419709 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f557c0839c0 con 0x7f558c149330 2024-09-13T18:07:31.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.515+0000 7f559117b700 1 --2- 172.21.15.40:0/1062419709 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5570067550 0x7f5570069a10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f558c0cc320 tx=0x7f5588010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:31.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.523+0000 7f5581ffb700 1 -- 172.21.15.40:0/1062419709 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f557c050850 con 0x7f558c149330 2024-09-13T18:07:31.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.763+0000 7f559217d700 1 -- 172.21.15.40:0/1062419709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f556c005b40 con 0x7f558c149330 2024-09-13T18:07:31.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.763+0000 7f5581ffb700 1 -- 172.21.15.40:0/1062419709 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f557c01a080 con 0x7f558c149330 2024-09-13T18:07:31.773 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":144,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:07:31.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.767+0000 7f559217d700 1 -- 172.21.15.40:0/1062419709 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5570067550 msgr2=0x7f5570069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:31.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.767+0000 7f559217d700 1 --2- 172.21.15.40:0/1062419709 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5570067550 0x7f5570069a10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f558c0cc320 tx=0x7f5588010000 comp rx=0 tx=0).stop 2024-09-13T18:07:31.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.767+0000 7f559217d700 1 -- 172.21.15.40:0/1062419709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f558c149330 msgr2=0x7f558c1eebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:31.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.767+0000 7f559217d700 1 --2- 172.21.15.40:0/1062419709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f558c149330 0x7f558c1eebf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f557c003e20 tx=0x7f557c00bc00 comp rx=0 tx=0).stop 2024-09-13T18:07:31.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.767+0000 7f559217d700 1 -- 172.21.15.40:0/1062419709 shutdown_connections 2024-09-13T18:07:31.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.767+0000 7f559217d700 1 --2- 172.21.15.40:0/1062419709 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5570067550 0x7f5570069a10 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:31.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.767+0000 7f559217d700 1 --2- 172.21.15.40:0/1062419709 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f558c14c6a0 0x7f558c14a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:31.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.767+0000 7f559217d700 1 --2- 172.21.15.40:0/1062419709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f558c149330 0x7f558c1eebf0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:31.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.767+0000 7f559217d700 1 --2- 172.21.15.40:0/1062419709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f558c148f50 0x7f558c1ee6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:31.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.767+0000 7f559217d700 1 -- 172.21.15.40:0/1062419709 >> 172.21.15.40:0/1062419709 conn(0x7f558c0c7d70 msgr2=0x7f558c152360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:31.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.767+0000 7f559217d700 1 -- 172.21.15.40:0/1062419709 shutdown_connections 2024-09-13T18:07:31.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.767+0000 7f559217d700 1 -- 172.21.15.40:0/1062419709 wait complete. 2024-09-13T18:07:31.787 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T18:07:31.788 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-13T18:07:31.788 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:07:31.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.895+0000 7fea5920d700 1 Processor -- start 2024-09-13T18:07:31.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.895+0000 7fea5920d700 1 -- start start 2024-09-13T18:07:31.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea5920d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fea54153730 0x7fea54159f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:31.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea5920d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fea540c7fb0 0x7fea540c8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:31.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea5920d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fea540c8b70 0x7fea5415cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:31.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea5920d700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea5415a450 con 0x7fea54153730 2024-09-13T18:07:31.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea5920d700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea54070ae0 con 0x7fea540c8b70 2024-09-13T18:07:31.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea5920d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea54071010 con 0x7fea540c7fb0 2024-09-13T18:07:31.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea58a0c700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7fea540c8b70 msgr2=0x7fea5415cfe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:07:31.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea58a0c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fea540c8b70 0x7fea5415cfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:07:31.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea53fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fea54153730 0x7fea54159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:31.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea53fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fea54153730 0x7fea54159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33306/0 (socket says 172.21.15.40:33306) 2024-09-13T18:07:31.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea53fff700 1 -- 172.21.15.40:0/1464452043 learned_addr learned my addr 172.21.15.40:0/1464452043 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:31.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea537fe700 1 --2- 172.21.15.40:0/1464452043 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fea540c7fb0 0x7fea540c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:31.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea53fff700 1 -- 172.21.15.40:0/1464452043 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fea540c7fb0 msgr2=0x7fea540c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:31.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea53fff700 1 --2- 172.21.15.40:0/1464452043 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fea540c7fb0 0x7fea540c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:31.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea53fff700 1 -- 172.21.15.40:0/1464452043 >> v2:172.21.15.43:3300/0 conn(0x7fea540c8b70 msgr2=0x7fea5415cfe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:07:31.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea53fff700 1 --2- 172.21.15.40:0/1464452043 >> v2:172.21.15.43:3300/0 conn(0x7fea540c8b70 0x7fea5415cfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:31.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea53fff700 1 -- 172.21.15.40:0/1464452043 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea54072420 con 0x7fea54153730 2024-09-13T18:07:31.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea53fff700 1 --2- 172.21.15.40:0/1464452043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea54153730 0x7fea54159f10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fea540c96e0 tx=0x7fea44009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1adb173de37b7f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:31.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea52ffd700 1 -- 172.21.15.40:0/1464452043 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44011830 con 0x7fea54153730 2024-09-13T18:07:31.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea52ffd700 1 -- 172.21.15.40:0/1464452043 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea44011990 con 0x7fea54153730 2024-09-13T18:07:31.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea52ffd700 1 -- 172.21.15.40:0/1464452043 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44011d20 con 0x7fea54153730 2024-09-13T18:07:31.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea5920d700 1 -- 172.21.15.40:0/1464452043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea54153730 msgr2=0x7fea54159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:31.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea5920d700 1 --2- 172.21.15.40:0/1464452043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea54153730 0x7fea54159f10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fea540c96e0 tx=0x7fea44009bf0 comp rx=0 tx=0).stop 2024-09-13T18:07:31.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea5920d700 1 -- 172.21.15.40:0/1464452043 shutdown_connections 2024-09-13T18:07:31.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea5920d700 1 --2- 172.21.15.40:0/1464452043 >> v2:172.21.15.43:3300/0 conn(0x7fea540c8b70 0x7fea5415cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:31.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea5920d700 1 --2- 172.21.15.40:0/1464452043 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fea540c7fb0 0x7fea540c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:31.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea5920d700 1 --2- 172.21.15.40:0/1464452043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea54153730 0x7fea54159f10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:31.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea5920d700 1 -- 172.21.15.40:0/1464452043 >> 172.21.15.40:0/1464452043 conn(0x7fea5406e6d0 msgr2=0x7fea5406eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:31.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea5920d700 1 -- 172.21.15.40:0/1464452043 shutdown_connections 2024-09-13T18:07:31.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.899+0000 7fea5920d700 1 -- 172.21.15.40:0/1464452043 wait complete. 2024-09-13T18:07:31.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea5920d700 1 Processor -- start 2024-09-13T18:07:31.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea5920d700 1 -- start start 2024-09-13T18:07:31.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea5920d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fea540c7fb0 0x7fea541562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:31.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea5920d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea540c8b70 0x7fea541f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:31.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea5920d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea54153730 0x7fea54150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:31.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea5920d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea540c6040 con 0x7fea54153730 2024-09-13T18:07:31.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea5920d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea54155570 con 0x7fea540c8b70 2024-09-13T18:07:31.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea5920d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea540c6d00 con 0x7fea540c7fb0 2024-09-13T18:07:31.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea537fe700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea540c8b70 msgr2=0x7fea541f03f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:07:31.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea537fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea540c8b70 0x7fea541f03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:07:31.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea58a0c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea54153730 0x7fea54150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:31.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea58a0c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea54153730 0x7fea54150390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33316/0 (socket says 172.21.15.40:33316) 2024-09-13T18:07:31.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea58a0c700 1 -- 172.21.15.40:0/3126972040 learned_addr learned my addr 172.21.15.40:0/3126972040 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:31.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea53fff700 1 --2- 172.21.15.40:0/3126972040 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fea540c7fb0 0x7fea541562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:31.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea58a0c700 1 -- 172.21.15.40:0/3126972040 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fea540c7fb0 msgr2=0x7fea541562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:31.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea58a0c700 1 --2- 172.21.15.40:0/3126972040 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fea540c7fb0 0x7fea541562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:31.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea58a0c700 1 -- 172.21.15.40:0/3126972040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea540c8b70 msgr2=0x7fea541f03f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:07:31.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea58a0c700 1 --2- 172.21.15.40:0/3126972040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea540c8b70 0x7fea541f03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:31.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea58a0c700 1 -- 172.21.15.40:0/3126972040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea44009570 con 0x7fea54153730 2024-09-13T18:07:31.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea58a0c700 1 --2- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea54153730 0x7fea54150390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00b9c0 tx=0x7fea4c00bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:31.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea50ff9700 1 -- 172.21.15.40:0/3126972040 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c00d8e0 con 0x7fea54153730 2024-09-13T18:07:31.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea50ff9700 1 -- 172.21.15.40:0/3126972040 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4c006e90 con 0x7fea54153730 2024-09-13T18:07:31.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea5920d700 1 -- 172.21.15.40:0/3126972040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea541569f0 con 0x7fea54153730 2024-09-13T18:07:31.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea50ff9700 1 -- 172.21.15.40:0/3126972040 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c00dc80 con 0x7fea54153730 2024-09-13T18:07:31.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.903+0000 7fea5920d700 1 -- 172.21.15.40:0/3126972040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea541557c0 con 0x7fea54153730 2024-09-13T18:07:31.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.907+0000 7fea5920d700 1 --2- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea541508d0 0x7fea541559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:31.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.907+0000 7fea5920d700 1 -- 172.21.15.40:0/3126972040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea54067600 con 0x7fea541508d0 2024-09-13T18:07:31.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.907+0000 7fea53fff700 1 --2- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea541508d0 0x7fea541559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:31.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.907+0000 7fea50ff9700 1 -- 172.21.15.40:0/3126972040 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fea4c015400 con 0x7fea54153730 2024-09-13T18:07:31.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.907+0000 7fea53fff700 1 -- 172.21.15.40:0/3126972040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea54152aa0 con 0x7fea541508d0 2024-09-13T18:07:31.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.907+0000 7fea53fff700 1 --2- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea541508d0 0x7fea541559e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fea540c96e0 tx=0x7fea44004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:31.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.907+0000 7fea50ff9700 1 --2- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fea380666e0 0x7fea38068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:31.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.907+0000 7fea537fe700 1 --2- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fea380666e0 0x7fea38068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:31.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.907+0000 7fea50ff9700 1 -- 172.21.15.40:0/3126972040 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fea4c0843e0 con 0x7fea54153730 2024-09-13T18:07:31.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.907+0000 7fea50ff9700 1 -- 172.21.15.40:0/3126972040 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44020070 con 0x7fea541508d0 2024-09-13T18:07:31.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.907+0000 7fea50ff9700 1 -- 172.21.15.40:0/3126972040 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fea44004310 con 0x7fea541508d0 2024-09-13T18:07:31.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.907+0000 7fea50ff9700 1 -- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea541508d0 msgr2=0x7fea541559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:31.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.907+0000 7fea50ff9700 1 --2- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea541508d0 0x7fea541559e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fea540c96e0 tx=0x7fea44004770 comp rx=0 tx=0).stop 2024-09-13T18:07:31.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.907+0000 7fea50ff9700 1 -- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea541508d0 msgr2=0x7fea541559e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:31.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.907+0000 7fea50ff9700 1 --2- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea541508d0 0x7fea541559e0 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fea540c96e0 tx=0x7fea44004770 comp rx=0 tx=0).stop 2024-09-13T18:07:31.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.907+0000 7fea537fe700 1 --2- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fea380666e0 0x7fea38068ba0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fea540c33a0 tx=0x7fea3c009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:31.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea5920d700 1 --2- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea541f2760 0x7fea541f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:31.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea5920d700 1 -- 172.21.15.40:0/3126972040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea54072140 con 0x7fea541f2760 2024-09-13T18:07:31.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea53fff700 1 --2- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea541f2760 0x7fea541f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:31.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea53fff700 1 -- 172.21.15.40:0/3126972040 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fea54071e60 con 0x7fea541f2760 2024-09-13T18:07:31.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea53fff700 1 --2- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea541f2760 0x7fea541f2b70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fea44009150 tx=0x7fea44011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:31.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea50ff9700 1 -- 172.21.15.40:0/3126972040 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4402dae0 con 0x7fea541f2760 2024-09-13T18:07:31.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea50ff9700 1 -- 172.21.15.40:0/3126972040 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4773 (secure 0 0 0) 0x7fea4402d300 con 0x7fea541f2760 2024-09-13T18:07:31.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea50ff9700 1 -- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea541f2760 msgr2=0x7fea541f2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:31.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea50ff9700 1 --2- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea541f2760 0x7fea541f2b70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fea44009150 tx=0x7fea44011000 comp rx=0 tx=0).stop 2024-09-13T18:07:31.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea50ff9700 1 -- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea541f2760 msgr2=0x7fea541f2b70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:31.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea50ff9700 1 --2- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea541f2760 0x7fea541f2b70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:31.930 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:07:31.930 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:07:31.930 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:07:31.930 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:07:31.930 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 144, 2024-09-13T18:07:31.930 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:07:31.930 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:07:31.930 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:07:31.930 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:31.931 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:07:31.931 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:31.931 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:07:31.931 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:07:31.931 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:31.931 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:31.931 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:31.931 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:31.931 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:31.931 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:31.931 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:31.931 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:31.931 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:31.931 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:31.932 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:31.932 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:07:31.932 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:07:31.932 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:31.932 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:31.932 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:31.932 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:31.932 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:31.932 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:31.932 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:31.932 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:31.932 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:31.932 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:31.932 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:31.933 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:31.933 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:07:31.933 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:07:31.933 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:07:31.933 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:07:31.933 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:07:31.933 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:07:31.933 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:31.933 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:31.933 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:07:31.933 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:07:31.933 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:07:31.933 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:07:31.934 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:07:31.934 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:07:31.934 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:07:31.934 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:31.934 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:07:31.934 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:31.934 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:31.934 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:31.934 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:31.934 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:31.934 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:31.934 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:31.934 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:31.934 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:31.935 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:31.935 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:31.935 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:07:31.935 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:31.935 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:07:31.935 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:31.935 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:07:31.935 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:07:31.935 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:31.935 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:31.935 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:31.935 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:31.935 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:07:31.935 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:31.935 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:31.936 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:31.936 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:31.936 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:07:31.936 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:31.936 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:31.936 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:31.936 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:31.936 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:07:31.936 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:07:31.936 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:31.936 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:31.936 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:31.936 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:31.937 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:31.937 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:07:31.937 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:07:31.937 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:31.937 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:31.937 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:31.937 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:31.937 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:07:31.937 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:31.937 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:31.937 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:31.937 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:31.937 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:07:31.937 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:31.938 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:31.938 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:31.938 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:31.938 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:07:31.938 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:07:31.938 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:31.938 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:31.938 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:31.938 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:31.938 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:31.938 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:07:31.938 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:07:31.938 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:31.938 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:31.939 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:31.939 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:31.939 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:07:31.939 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:31.939 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:31.939 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:31.939 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:31.939 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:07:31.939 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:31.939 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:31.939 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:31.939 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:31.939 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:07:31.939 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:07:31.940 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:31.940 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:31.940 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:31.940 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:31.940 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:31.940 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:31.940 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:07:31.940 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:07:31.940 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:31.940 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:31.940 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:31.940 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:07:31.940 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:31.940 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:31.941 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:07:31.941 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:31.941 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:31.941 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:31.941 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:07:31.941 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:31.941 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:31.941 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:07:31.941 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:31.941 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:31.941 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:31.941 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:07:31.941 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:31.942 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:31.942 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:07:31.942 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:31.942 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:31.942 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:31.942 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:07:31.942 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:31.942 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:31.942 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:31.942 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:07:31.942 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:07:31.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea5920d700 1 -- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fea380666e0 msgr2=0x7fea38068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:31.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea5920d700 1 --2- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fea380666e0 0x7fea38068ba0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fea540c33a0 tx=0x7fea3c009240 comp rx=0 tx=0).stop 2024-09-13T18:07:31.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea5920d700 1 -- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea54153730 msgr2=0x7fea54150390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:31.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea5920d700 1 --2- 172.21.15.40:0/3126972040 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea54153730 0x7fea54150390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00b9c0 tx=0x7fea4c00bcd0 comp rx=0 tx=0).stop 2024-09-13T18:07:31.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea58a0c700 1 -- 172.21.15.40:0/3126972040 reap_dead start 2024-09-13T18:07:31.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea58a0c700 1 -- 172.21.15.40:0/3126972040 reap_dead start 2024-09-13T18:07:31.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea5920d700 1 -- 172.21.15.40:0/3126972040 shutdown_connections 2024-09-13T18:07:31.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea5920d700 1 -- 172.21.15.40:0/3126972040 >> 172.21.15.40:0/3126972040 conn(0x7fea5406e6d0 msgr2=0x7fea54151f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:31.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea5920d700 1 -- 172.21.15.40:0/3126972040 shutdown_connections 2024-09-13T18:07:31.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:31.919+0000 7fea5920d700 1 -- 172.21.15.40:0/3126972040 wait complete. 2024-09-13T18:07:31.945 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:07:31.945 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T18:07:31.945 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:07:32.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.051+0000 7fcf7f59e700 1 Processor -- start 2024-09-13T18:07:32.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.051+0000 7fcf7f59e700 1 -- start start 2024-09-13T18:07:32.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.051+0000 7fcf7f59e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcf8006dcb0 0x7fcf80159e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:32.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.051+0000 7fcf7f59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf8015a340 0x7fcf80163ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:32.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.051+0000 7fcf7f59e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf80159970 con 0x7fcf8006e090 2024-09-13T18:07:32.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.051+0000 7fcf7f59e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf8006d2d0 con 0x7fcf8006dcb0 2024-09-13T18:07:32.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.051+0000 7fcf7f59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf80159620 con 0x7fcf8015a340 2024-09-13T18:07:32.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.051+0000 7fcf7dd9b700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7fcf8006e090 legacy=0x7fcf800c7cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:07:32.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.051+0000 7fcf7e59c700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7fcf8006dcb0 msgr2=0x7fcf80159e00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:07:32.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.051+0000 7fcf7e59c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcf8006dcb0 0x7fcf80159e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:07:32.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.051+0000 7fcf7ed9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf8015a340 0x7fcf80163ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:32.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.051+0000 7fcf7ed9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf8015a340 0x7fcf80163ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:41052/0 (socket says 172.21.15.40:41052) 2024-09-13T18:07:32.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.051+0000 7fcf7ed9d700 1 -- 172.21.15.40:0/547193671 learned_addr learned my addr 172.21.15.40:0/547193671 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:32.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.055+0000 7fcf7ed9d700 1 -- 172.21.15.40:0/547193671 >> v2:172.21.15.43:3300/0 conn(0x7fcf8006dcb0 msgr2=0x7fcf80159e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:07:32.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.055+0000 7fcf7ed9d700 1 --2- 172.21.15.40:0/547193671 >> v2:172.21.15.43:3300/0 conn(0x7fcf8006dcb0 0x7fcf80159e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:32.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.055+0000 7fcf7ed9d700 1 -- 172.21.15.40:0/547193671 >> v1:172.21.15.43:6789/0 conn(0x7fcf8006e090 legacy=0x7fcf800c7cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:07:32.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.055+0000 7fcf7ed9d700 1 -- 172.21.15.40:0/547193671 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf800c6c90 con 0x7fcf8015a340 2024-09-13T18:07:32.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.055+0000 7fcf7ed9d700 1 --2- 172.21.15.40:0/547193671 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf8015a340 0x7fcf80163ec0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009e50 tx=0x7fcf7800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=500ceb77403b8fc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:32.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.055+0000 7fcf7d59a700 1 -- 172.21.15.40:0/547193671 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7800e070 con 0x7fcf8015a340 2024-09-13T18:07:32.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.055+0000 7fcf7d59a700 1 -- 172.21.15.40:0/547193671 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf78003a40 con 0x7fcf8015a340 2024-09-13T18:07:32.065 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.055+0000 7fcf7d59a700 1 -- 172.21.15.40:0/547193671 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf78003df0 con 0x7fcf8015a340 2024-09-13T18:07:32.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.055+0000 7fcf7f59e700 1 -- 172.21.15.40:0/547193671 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf8015a340 msgr2=0x7fcf80163ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:32.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.055+0000 7fcf7f59e700 1 --2- 172.21.15.40:0/547193671 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf8015a340 0x7fcf80163ec0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009e50 tx=0x7fcf7800b330 comp rx=0 tx=0).stop 2024-09-13T18:07:32.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.055+0000 7fcf7f59e700 1 -- 172.21.15.40:0/547193671 shutdown_connections 2024-09-13T18:07:32.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.055+0000 7fcf7f59e700 1 --2- 172.21.15.40:0/547193671 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf8015a340 0x7fcf80163ec0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:32.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.055+0000 7fcf7f59e700 1 --2- 172.21.15.40:0/547193671 >> v2:172.21.15.43:3300/0 conn(0x7fcf8006dcb0 0x7fcf80159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:32.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.055+0000 7fcf7f59e700 1 -- 172.21.15.40:0/547193671 >> 172.21.15.40:0/547193671 conn(0x7fcf80066090 msgr2=0x7fcf800664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:32.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.055+0000 7fcf7f59e700 1 -- 172.21.15.40:0/547193671 shutdown_connections 2024-09-13T18:07:32.066 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.055+0000 7fcf7f59e700 1 -- 172.21.15.40:0/547193671 wait complete. 2024-09-13T18:07:32.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7f59e700 1 Processor -- start 2024-09-13T18:07:32.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7f59e700 1 -- start start 2024-09-13T18:07:32.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7f59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf8006dcb0 0x7fcf80149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:32.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7f59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf8006e090 0x7fcf8014ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:32.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7f59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf8015a340 0x7fcf8014f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:32.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7f59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf800713d0 con 0x7fcf8006dcb0 2024-09-13T18:07:32.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7f59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf800720a0 con 0x7fcf8015a340 2024-09-13T18:07:32.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7f59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf80070e50 con 0x7fcf8006e090 2024-09-13T18:07:32.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7ed9d700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf8015a340 msgr2=0x7fcf8014f3d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:07:32.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7ed9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf8015a340 0x7fcf8014f3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:07:32.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf8006dcb0 0x7fcf80149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:32.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf8006dcb0 0x7fcf80149d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33334/0 (socket says 172.21.15.40:33334) 2024-09-13T18:07:32.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7e59c700 1 -- 172.21.15.40:0/3843567862 learned_addr learned my addr 172.21.15.40:0/3843567862 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:32.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7dd9b700 1 --2- 172.21.15.40:0/3843567862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf8006e090 0x7fcf8014ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:32.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7e59c700 1 -- 172.21.15.40:0/3843567862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf8006e090 msgr2=0x7fcf8014ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:32.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7e59c700 1 --2- 172.21.15.40:0/3843567862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf8006e090 0x7fcf8014ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:32.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7e59c700 1 -- 172.21.15.40:0/3843567862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf8015a340 msgr2=0x7fcf8014f3d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:07:32.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7e59c700 1 --2- 172.21.15.40:0/3843567862 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf8015a340 0x7fcf8014f3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:32.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7e59c700 1 -- 172.21.15.40:0/3843567862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf78009b30 con 0x7fcf8006dcb0 2024-09-13T18:07:32.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7e59c700 1 --2- 172.21.15.40:0/3843567862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf8006dcb0 0x7fcf80149d20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800b780 tx=0x7fcf6800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:32.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf76ffd700 1 -- 172.21.15.40:0/3843567862 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68003e70 con 0x7fcf8006dcb0 2024-09-13T18:07:32.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf76ffd700 1 -- 172.21.15.40:0/3843567862 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf68005070 con 0x7fcf8006dcb0 2024-09-13T18:07:32.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7f59e700 1 -- 172.21.15.40:0/3843567862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8014a570 con 0x7fcf8006dcb0 2024-09-13T18:07:32.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf76ffd700 1 -- 172.21.15.40:0/3843567862 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68005420 con 0x7fcf8006dcb0 2024-09-13T18:07:32.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.059+0000 7fcf7f59e700 1 -- 172.21.15.40:0/3843567862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf801580d0 con 0x7fcf8006dcb0 2024-09-13T18:07:32.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.063+0000 7fcf7f59e700 1 --2- 172.21.15.40:0/3843567862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf800715e0 0x7fcf80153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:32.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.063+0000 7fcf7f59e700 1 -- 172.21.15.40:0/3843567862 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf80067720 con 0x7fcf800715e0 2024-09-13T18:07:32.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.063+0000 7fcf7dd9b700 1 --2- 172.21.15.40:0/3843567862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf800715e0 0x7fcf80153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:32.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.063+0000 7fcf76ffd700 1 -- 172.21.15.40:0/3843567862 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcf68005580 con 0x7fcf8006dcb0 2024-09-13T18:07:32.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.063+0000 7fcf7dd9b700 1 -- 172.21.15.40:0/3843567862 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf800c7530 con 0x7fcf800715e0 2024-09-13T18:07:32.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.063+0000 7fcf76ffd700 1 --2- 172.21.15.40:0/3843567862 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcf600666e0 0x7fcf60068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:32.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.063+0000 7fcf76ffd700 1 -- 172.21.15.40:0/3843567862 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcf68083e70 con 0x7fcf8006dcb0 2024-09-13T18:07:32.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.063+0000 7fcf7dd9b700 1 --2- 172.21.15.40:0/3843567862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf800715e0 0x7fcf80153b50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcf80073880 tx=0x7fcf70009ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:32.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.063+0000 7fcf7ed9d700 1 --2- 172.21.15.40:0/3843567862 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcf600666e0 0x7fcf60068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:32.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.063+0000 7fcf76ffd700 1 -- 172.21.15.40:0/3843567862 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70012600 con 0x7fcf800715e0 2024-09-13T18:07:32.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.063+0000 7fcf7ed9d700 1 --2- 172.21.15.40:0/3843567862 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcf600666e0 0x7fcf60068ba0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcf8015a750 tx=0x7fcf78009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:32.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.063+0000 7fcf76ffd700 1 -- 172.21.15.40:0/3843567862 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fcf70012760 con 0x7fcf800715e0 2024-09-13T18:07:32.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.063+0000 7fcf76ffd700 1 -- 172.21.15.40:0/3843567862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf800715e0 msgr2=0x7fcf80153b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:32.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.063+0000 7fcf76ffd700 1 --2- 172.21.15.40:0/3843567862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf800715e0 0x7fcf80153b50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcf80073880 tx=0x7fcf70009ab0 comp rx=0 tx=0).stop 2024-09-13T18:07:32.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.063+0000 7fcf76ffd700 1 -- 172.21.15.40:0/3843567862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf800715e0 msgr2=0x7fcf80153b50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:32.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.063+0000 7fcf76ffd700 1 --2- 172.21.15.40:0/3843567862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf800715e0 0x7fcf80153b50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:32.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.075+0000 7fcf7f59e700 1 --2- 172.21.15.40:0/3843567862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf80154090 0x7fcf80155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:32.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.075+0000 7fcf7f59e700 1 -- 172.21.15.40:0/3843567862 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf800c7250 con 0x7fcf80154090 2024-09-13T18:07:32.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.075+0000 7fcf7dd9b700 1 --2- 172.21.15.40:0/3843567862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf80154090 0x7fcf80155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:32.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.075+0000 7fcf7dd9b700 1 -- 172.21.15.40:0/3843567862 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcf800c6590 con 0x7fcf80154090 2024-09-13T18:07:32.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.079+0000 7fcf7dd9b700 1 --2- 172.21.15.40:0/3843567862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf80154090 0x7fcf80155bb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcf70014000 tx=0x7fcf70005a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:32.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.079+0000 7fcf76ffd700 1 -- 172.21.15.40:0/3843567862 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70023070 con 0x7fcf80154090 2024-09-13T18:07:32.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.079+0000 7fcf76ffd700 1 -- 172.21.15.40:0/3843567862 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4937 (secure 0 0 0) 0x7fcf700128d0 con 0x7fcf80154090 2024-09-13T18:07:32.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.079+0000 7fcf76ffd700 1 -- 172.21.15.40:0/3843567862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf80154090 msgr2=0x7fcf80155bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:32.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.079+0000 7fcf76ffd700 1 --2- 172.21.15.40:0/3843567862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf80154090 0x7fcf80155bb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcf70014000 tx=0x7fcf70005a80 comp rx=0 tx=0).stop 2024-09-13T18:07:32.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.079+0000 7fcf76ffd700 1 -- 172.21.15.40:0/3843567862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf80154090 msgr2=0x7fcf80155bb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:32.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.079+0000 7fcf76ffd700 1 --2- 172.21.15.40:0/3843567862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf80154090 0x7fcf80155bb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:32.089 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:07:32.089 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:07:32.089 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:07:32.089 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:07:32.089 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 144, 2024-09-13T18:07:32.089 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:07:32.089 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:07:32.089 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:07:32.089 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:32.089 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:07:32.089 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:32.090 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:07:32.090 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:07:32.090 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:32.090 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:32.090 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:32.090 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:32.090 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:32.090 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:32.090 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:32.090 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:32.090 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:32.090 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:32.090 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:32.091 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:07:32.091 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:07:32.091 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:32.091 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:32.091 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:32.091 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:32.091 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:32.091 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:32.091 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:32.091 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:32.091 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:32.091 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:32.091 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:32.091 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:32.092 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:07:32.092 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:07:32.092 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:07:32.092 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:07:32.092 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:07:32.092 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:07:32.092 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:32.092 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:32.092 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:07:32.092 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:07:32.092 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:07:32.092 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:07:32.092 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:07:32.093 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:07:32.093 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:07:32.093 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:32.093 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:07:32.093 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:32.093 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:32.093 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:32.093 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:32.093 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:32.093 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:32.093 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:32.093 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:32.093 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:32.094 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:32.094 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:32.094 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:07:32.094 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:32.094 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:07:32.094 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:32.094 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:07:32.094 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:07:32.094 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:32.094 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:32.094 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:32.094 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:32.094 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:07:32.095 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:32.095 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:32.095 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:32.095 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:32.095 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:07:32.095 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:32.095 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:32.095 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:32.095 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:32.095 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:07:32.095 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:07:32.095 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:32.095 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:32.095 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:32.096 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:32.096 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:32.096 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:07:32.096 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:07:32.096 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:32.096 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:32.096 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:32.096 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:32.096 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:07:32.096 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:32.096 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:32.096 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:32.096 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:32.097 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:07:32.097 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:32.097 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:32.097 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:32.097 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:32.097 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:07:32.097 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:07:32.097 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:32.097 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:32.097 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:32.097 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:32.097 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:32.097 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:07:32.098 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:07:32.098 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:32.098 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:32.098 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:32.098 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:32.098 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:07:32.098 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:32.098 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:32.098 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:32.098 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:32.098 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:07:32.098 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:32.098 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:32.098 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:32.099 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:32.099 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:07:32.099 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:07:32.099 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:32.099 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:32.099 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:32.099 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:32.099 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:32.099 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:32.099 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:07:32.099 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:07:32.099 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:32.099 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:32.100 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:32.100 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:07:32.100 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:32.100 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:32.100 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:07:32.100 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:32.100 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:32.100 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:32.100 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:07:32.100 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:32.100 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:32.100 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:07:32.100 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:32.100 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:32.101 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:32.101 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:07:32.101 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:32.101 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:32.101 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:07:32.101 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:32.101 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:32.101 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:32.101 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:07:32.101 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:32.101 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:32.101 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:07:32.101 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:32.102 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:32.102 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:32.102 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:07:32.102 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:32.102 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:32.102 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:32.102 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:07:32.102 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:07:32.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.079+0000 7fcf7f59e700 1 -- 172.21.15.40:0/3843567862 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcf600666e0 msgr2=0x7fcf60068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:32.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.079+0000 7fcf7f59e700 1 --2- 172.21.15.40:0/3843567862 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcf600666e0 0x7fcf60068ba0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcf8015a750 tx=0x7fcf78009000 comp rx=0 tx=0).stop 2024-09-13T18:07:32.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.079+0000 7fcf7f59e700 1 -- 172.21.15.40:0/3843567862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf8006dcb0 msgr2=0x7fcf80149d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:32.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.079+0000 7fcf7f59e700 1 --2- 172.21.15.40:0/3843567862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf8006dcb0 0x7fcf80149d20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800b780 tx=0x7fcf6800ba90 comp rx=0 tx=0).stop 2024-09-13T18:07:32.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.079+0000 7fcf7ed9d700 1 -- 172.21.15.40:0/3843567862 reap_dead start 2024-09-13T18:07:32.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.079+0000 7fcf7ed9d700 1 -- 172.21.15.40:0/3843567862 reap_dead start 2024-09-13T18:07:32.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.079+0000 7fcf7f59e700 1 -- 172.21.15.40:0/3843567862 shutdown_connections 2024-09-13T18:07:32.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.079+0000 7fcf7f59e700 1 -- 172.21.15.40:0/3843567862 >> 172.21.15.40:0/3843567862 conn(0x7fcf80066090 msgr2=0x7fcf8006a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:32.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.079+0000 7fcf7f59e700 1 -- 172.21.15.40:0/3843567862 shutdown_connections 2024-09-13T18:07:32.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:32.079+0000 7fcf7f59e700 1 -- 172.21.15.40:0/3843567862 wait complete. 2024-09-13T18:07:32.104 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:07:32.104 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:07:36.841 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:07:36.844 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:07:52.105 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-13T18:07:52.105 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-13T18:07:52.105 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T18:07:52.106 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T18:07:52.108 INFO:tasks.ceph.mon.b:Started 2024-09-13T18:07:52.108 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:07:52.109 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:07:52.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.219+0000 7f4e90fc7700 1 Processor -- start 2024-09-13T18:07:52.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.219+0000 7f4e90fc7700 1 -- start start 2024-09-13T18:07:52.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e90fc7700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4e8c1503d0 0x7f4e8c156940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:52.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e90fc7700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e8c15a520 con 0x7f4e8c15e960 2024-09-13T18:07:52.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e90fc7700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e8c1564b0 con 0x7f4e8c06b4a0 2024-09-13T18:07:52.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e90fc7700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8c15a000 con 0x7f4e8c1503d0 2024-09-13T18:07:52.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8bfff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4e8c15e960 0x7f4e8c160d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40260/0 (socket says 172.21.15.40:40260) 2024-09-13T18:07:52.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8bfff700 1 -- 172.21.15.40:0/1192907869 learned_addr learned my addr 172.21.15.40:0/1192907869 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:07:52.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8b7fe700 1 --2- 172.21.15.40:0/1192907869 >> v2:172.21.15.43:3300/0 conn(0x7f4e8c1503d0 0x7f4e8c156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:52.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8b7fe700 1 -- 172.21.15.40:0/1192907869 >> v2:172.21.15.43:3300/0 conn(0x7f4e8c1503d0 msgr2=0x7f4e8c156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:07:52.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8b7fe700 1 -- 172.21.15.40:0/1192907869 >> v2:172.21.15.43:3300/0 conn(0x7f4e8c1503d0 msgr2=0x7f4e8c156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:07:52.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8b7fe700 1 --2- 172.21.15.40:0/1192907869 >> v2:172.21.15.43:3300/0 conn(0x7f4e8c1503d0 0x7f4e8c156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:07:52.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8b7fe700 1 --2- 172.21.15.40:0/1192907869 >> v2:172.21.15.43:3300/0 conn(0x7f4e8c1503d0 0x7f4e8c156940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:07:52.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8a7fc700 1 -- 172.21.15.40:0/1192907869 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508933063 0 0) 0x7f4e8c15a520 con 0x7f4e8c15e960 2024-09-13T18:07:52.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8a7fc700 1 -- 172.21.15.40:0/1192907869 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e740035d0 con 0x7f4e8c15e960 2024-09-13T18:07:52.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8a7fc700 1 -- 172.21.15.40:0/1192907869 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3610340815 0 0) 0x7f4e740035d0 con 0x7f4e8c15e960 2024-09-13T18:07:52.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8a7fc700 1 -- 172.21.15.40:0/1192907869 >> v2:172.21.15.43:3300/0 conn(0x7f4e8c1503d0 msgr2=0x7f4e8c156940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:07:52.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8a7fc700 1 --2- 172.21.15.40:0/1192907869 >> v2:172.21.15.43:3300/0 conn(0x7f4e8c1503d0 0x7f4e8c156940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:52.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8a7fc700 1 -- 172.21.15.40:0/1192907869 >> v1:172.21.15.43:6789/0 conn(0x7f4e8c06b4a0 legacy=0x7f4e8c071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:52.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8a7fc700 1 -- 172.21.15.40:0/1192907869 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c15a280 con 0x7f4e8c15e960 2024-09-13T18:07:52.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8a7fc700 1 -- 172.21.15.40:0/1192907869 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f4e840019f0 con 0x7f4e8c15e960 2024-09-13T18:07:52.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8a7fc700 1 -- 172.21.15.40:0/1192907869 >> v1:172.21.15.40:6789/0 conn(0x7f4e8c15e960 legacy=0x7f4e8c160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:52.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8a7fc700 1 --2- 172.21.15.40:0/1192907869 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e74003a20 0x7f4e74003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:52.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8a7fc700 1 --2- 172.21.15.40:0/1192907869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e74004870 0x7f4e74004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:52.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8a7fc700 1 --2- 172.21.15.40:0/1192907869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e74005230 0x7f4e7400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:52.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8a7fc700 1 -- 172.21.15.40:0/1192907869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c15a000 con 0x7f4e74004870 2024-09-13T18:07:52.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8a7fc700 1 -- 172.21.15.40:0/1192907869 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7400c180 con 0x7f4e74005230 2024-09-13T18:07:52.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8a7fc700 1 -- 172.21.15.40:0/1192907869 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7400c340 con 0x7f4e74003a20 2024-09-13T18:07:52.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8affd700 1 --2- 172.21.15.40:0/1192907869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e74004870 0x7f4e74004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:52.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8b7fe700 1 --2- 172.21.15.40:0/1192907869 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e74003a20 0x7f4e74003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:52.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8bfff700 1 --2- 172.21.15.40:0/1192907869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e74005230 0x7f4e7400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:52.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8affd700 1 -- 172.21.15.40:0/1192907869 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e74003a20 msgr2=0x7f4e74003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:52.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8affd700 1 --2- 172.21.15.40:0/1192907869 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e74003a20 0x7f4e74003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:52.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8affd700 1 -- 172.21.15.40:0/1192907869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e74005230 msgr2=0x7f4e7400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:52.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8affd700 1 --2- 172.21.15.40:0/1192907869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e74005230 0x7f4e7400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:52.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8affd700 1 -- 172.21.15.40:0/1192907869 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e740035d0 con 0x7f4e74004870 2024-09-13T18:07:52.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8bfff700 1 --2- 172.21.15.40:0/1192907869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e74005230 0x7f4e7400ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:07:52.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8bfff700 1 -- 172.21.15.40:0/1192907869 reap_dead start 2024-09-13T18:07:52.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8affd700 1 --2- 172.21.15.40:0/1192907869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e74004870 0x7f4e74004cf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4e800017e0 tx=0x7f4e740058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716143028b6962c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:52.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8a7fc700 1 -- 172.21.15.40:0/1192907869 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e800052d0 con 0x7f4e74004870 2024-09-13T18:07:52.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8a7fc700 1 -- 172.21.15.40:0/1192907869 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e80005450 con 0x7f4e74004870 2024-09-13T18:07:52.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.223+0000 7f4e8a7fc700 1 -- 172.21.15.40:0/1192907869 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e800038e0 con 0x7f4e74004870 2024-09-13T18:07:52.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.227+0000 7f4e90fc7700 1 -- 172.21.15.40:0/1192907869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e74004870 msgr2=0x7f4e74004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:52.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.227+0000 7f4e90fc7700 1 --2- 172.21.15.40:0/1192907869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e74004870 0x7f4e74004cf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4e800017e0 tx=0x7f4e740058a0 comp rx=0 tx=0).stop 2024-09-13T18:07:52.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.227+0000 7f4e90fc7700 1 -- 172.21.15.40:0/1192907869 shutdown_connections 2024-09-13T18:07:52.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.227+0000 7f4e90fc7700 1 --2- 172.21.15.40:0/1192907869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e74004870 0x7f4e74004cf0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:52.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.227+0000 7f4e90fc7700 1 -- 172.21.15.40:0/1192907869 >> 172.21.15.40:0/1192907869 conn(0x7f4e8c06e4f0 msgr2=0x7f4e8c06e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:52.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.227+0000 7f4e90fc7700 1 -- 172.21.15.40:0/1192907869 shutdown_connections 2024-09-13T18:07:52.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.227+0000 7f4e90fc7700 1 -- 172.21.15.40:0/1192907869 wait complete. 2024-09-13T18:07:52.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.227+0000 7f4e90fc7700 1 Processor -- start 2024-09-13T18:07:52.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.227+0000 7f4e90fc7700 1 -- start start 2024-09-13T18:07:52.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e90fc7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e74004870 0x7f4e8c0ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:52.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e90fc7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e8c14c000 0x7f4e8c14c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:52.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e90fc7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e8c148e60 0x7f4e8c1492e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:52.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e90fc7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c14bb70 con 0x7f4e74004870 2024-09-13T18:07:52.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e90fc7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c14b870 con 0x7f4e8c148e60 2024-09-13T18:07:52.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e90fc7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c14c9a0 con 0x7f4e8c14c000 2024-09-13T18:07:52.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e8b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e74004870 0x7f4e8c0ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:52.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e8b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e74004870 0x7f4e8c0ca5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58428/0 (socket says 172.21.15.40:58428) 2024-09-13T18:07:52.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e8b7fe700 1 -- 172.21.15.40:0/795976966 learned_addr learned my addr 172.21.15.40:0/795976966 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:52.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e8affd700 1 --2- 172.21.15.40:0/795976966 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e8c14c000 0x7f4e8c14c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:52.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e8bfff700 1 --2- 172.21.15.40:0/795976966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e8c148e60 0x7f4e8c1492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:52.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e8b7fe700 1 -- 172.21.15.40:0/795976966 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e8c14c000 msgr2=0x7f4e8c14c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:52.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e8b7fe700 1 --2- 172.21.15.40:0/795976966 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e8c14c000 0x7f4e8c14c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:52.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e8b7fe700 1 -- 172.21.15.40:0/795976966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e8c148e60 msgr2=0x7f4e8c1492e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:52.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e8b7fe700 1 --2- 172.21.15.40:0/795976966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e8c148e60 0x7f4e8c1492e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:52.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e8b7fe700 1 -- 172.21.15.40:0/795976966 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c158ab0 con 0x7f4e74004870 2024-09-13T18:07:52.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e8affd700 1 --2- 172.21.15.40:0/795976966 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e8c14c000 0x7f4e8c14c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:07:52.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e8b7fe700 1 --2- 172.21.15.40:0/795976966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e74004870 0x7f4e8c0ca5c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c011930 tx=0x7f4e7c011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:52.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e73fff700 1 -- 172.21.15.40:0/795976966 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c00c070 con 0x7f4e74004870 2024-09-13T18:07:52.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e73fff700 1 -- 172.21.15.40:0/795976966 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c004020 con 0x7f4e74004870 2024-09-13T18:07:52.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e90fc7700 1 -- 172.21.15.40:0/795976966 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e8c158d70 con 0x7f4e74004870 2024-09-13T18:07:52.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e73fff700 1 -- 172.21.15.40:0/795976966 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0043d0 con 0x7f4e74004870 2024-09-13T18:07:52.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e90fc7700 1 -- 172.21.15.40:0/795976966 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e8c1eec70 con 0x7f4e74004870 2024-09-13T18:07:52.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.231+0000 7f4e90fc7700 1 -- 172.21.15.40:0/795976966 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e8c1560d0 con 0x7f4e74004870 2024-09-13T18:07:52.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.235+0000 7f4e73fff700 1 -- 172.21.15.40:0/795976966 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4e7c00f8e0 con 0x7f4e74004870 2024-09-13T18:07:52.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.235+0000 7f4e73fff700 1 --2- 172.21.15.40:0/795976966 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4e78066720 0x7f4e78068be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:52.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.235+0000 7f4e8affd700 1 --2- 172.21.15.40:0/795976966 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4e78066720 0x7f4e78068be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:52.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.235+0000 7f4e73fff700 1 -- 172.21.15.40:0/795976966 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4e7c0845c0 con 0x7f4e74004870 2024-09-13T18:07:52.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.235+0000 7f4e8affd700 1 --2- 172.21.15.40:0/795976966 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4e78066720 0x7f4e78068be0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4e80005040 tx=0x7f4e80009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:52.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.243+0000 7f4e73fff700 1 -- 172.21.15.40:0/795976966 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4e7c051460 con 0x7f4e74004870 2024-09-13T18:07:52.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.511+0000 7f4e90fc7700 1 -- 172.21.15.40:0/795976966 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4e8c0c2850 con 0x7f4e74004870 2024-09-13T18:07:52.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.511+0000 7f4e73fff700 1 -- 172.21.15.40:0/795976966 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f4e7c00f040 con 0x7f4e74004870 2024-09-13T18:07:52.522 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":144,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:07:52.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.515+0000 7f4e90fc7700 1 -- 172.21.15.40:0/795976966 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4e78066720 msgr2=0x7f4e78068be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:52.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.515+0000 7f4e90fc7700 1 --2- 172.21.15.40:0/795976966 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4e78066720 0x7f4e78068be0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4e80005040 tx=0x7f4e80009030 comp rx=0 tx=0).stop 2024-09-13T18:07:52.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.515+0000 7f4e90fc7700 1 -- 172.21.15.40:0/795976966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e74004870 msgr2=0x7f4e8c0ca5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:52.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.515+0000 7f4e90fc7700 1 --2- 172.21.15.40:0/795976966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e74004870 0x7f4e8c0ca5c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c011930 tx=0x7f4e7c011cf0 comp rx=0 tx=0).stop 2024-09-13T18:07:52.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.515+0000 7f4e90fc7700 1 -- 172.21.15.40:0/795976966 shutdown_connections 2024-09-13T18:07:52.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.515+0000 7f4e90fc7700 1 --2- 172.21.15.40:0/795976966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e8c148e60 0x7f4e8c1492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:52.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.515+0000 7f4e90fc7700 1 --2- 172.21.15.40:0/795976966 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4e78066720 0x7f4e78068be0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:52.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.515+0000 7f4e90fc7700 1 --2- 172.21.15.40:0/795976966 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e8c14c000 0x7f4e8c14c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:52.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.515+0000 7f4e90fc7700 1 --2- 172.21.15.40:0/795976966 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e74004870 0x7f4e8c0ca5c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:52.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.515+0000 7f4e90fc7700 1 -- 172.21.15.40:0/795976966 >> 172.21.15.40:0/795976966 conn(0x7f4e8c06e4f0 msgr2=0x7f4e8c06cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:52.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.515+0000 7f4e90fc7700 1 -- 172.21.15.40:0/795976966 shutdown_connections 2024-09-13T18:07:52.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:52.515+0000 7f4e90fc7700 1 -- 172.21.15.40:0/795976966 wait complete. 2024-09-13T18:07:55.542 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:07:55.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.651+0000 7f7f18e1e700 1 Processor -- start 2024-09-13T18:07:55.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.651+0000 7f7f18e1e700 1 -- start start 2024-09-13T18:07:55.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.651+0000 7f7f18e1e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f14070bc0 0x7f7f140c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:55.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.651+0000 7f7f18e1e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7f141567e0 0x7f7f14156c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:55.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f18e1e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f141571d0 con 0x7f7f14150690 2024-09-13T18:07:55.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f18e1e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f140c8350 con 0x7f7f141567e0 2024-09-13T18:07:55.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f18e1e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f140c6d20 con 0x7f7f14070bc0 2024-09-13T18:07:55.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f13fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7f141567e0 0x7f7f14156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:55.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f13fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7f141567e0 0x7f7f14156c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58436/0 (socket says 172.21.15.40:58436) 2024-09-13T18:07:55.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f12ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f14070bc0 0x7f7f140c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:55.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f137fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7f14150690 0x7f7f1406b4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:37750/0 (socket says 172.21.15.40:37750) 2024-09-13T18:07:55.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f13fff700 1 -- 172.21.15.40:0/3855168396 learned_addr learned my addr 172.21.15.40:0/3855168396 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:55.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f13fff700 1 -- 172.21.15.40:0/3855168396 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f14070bc0 msgr2=0x7f7f140c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:55.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f13fff700 1 --2- 172.21.15.40:0/3855168396 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f14070bc0 0x7f7f140c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:55.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f13fff700 1 -- 172.21.15.40:0/3855168396 >> v1:172.21.15.43:6789/0 conn(0x7f7f14150690 legacy=0x7f7f1406b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:55.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f13fff700 1 -- 172.21.15.40:0/3855168396 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f14159800 con 0x7f7f141567e0 2024-09-13T18:07:55.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f13fff700 1 --2- 172.21.15.40:0/3855168396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f141567e0 0x7f7f14156c90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009e50 tx=0x7f7f0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae681721adef9445 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:55.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f127fc700 1 -- 172.21.15.40:0/3855168396 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c00e070 con 0x7f7f141567e0 2024-09-13T18:07:55.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f127fc700 1 -- 172.21.15.40:0/3855168396 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c00b910 con 0x7f7f141567e0 2024-09-13T18:07:55.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f127fc700 1 -- 172.21.15.40:0/3855168396 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c00bcc0 con 0x7f7f141567e0 2024-09-13T18:07:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f18e1e700 1 -- 172.21.15.40:0/3855168396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f141567e0 msgr2=0x7f7f14156c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f18e1e700 1 --2- 172.21.15.40:0/3855168396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f141567e0 0x7f7f14156c90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009e50 tx=0x7f7f0c00b330 comp rx=0 tx=0).stop 2024-09-13T18:07:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f18e1e700 1 -- 172.21.15.40:0/3855168396 shutdown_connections 2024-09-13T18:07:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f18e1e700 1 --2- 172.21.15.40:0/3855168396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f141567e0 0x7f7f14156c90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f18e1e700 1 --2- 172.21.15.40:0/3855168396 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f14070bc0 0x7f7f140c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f18e1e700 1 -- 172.21.15.40:0/3855168396 >> 172.21.15.40:0/3855168396 conn(0x7f7f140c88e0 msgr2=0x7f7f140c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f18e1e700 1 -- 172.21.15.40:0/3855168396 shutdown_connections 2024-09-13T18:07:55.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f18e1e700 1 -- 172.21.15.40:0/3855168396 wait complete. 2024-09-13T18:07:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f18e1e700 1 Processor -- start 2024-09-13T18:07:55.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.655+0000 7f7f18e1e700 1 -- start start 2024-09-13T18:07:55.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f18e1e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f14070bc0 0x7f7f14154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:55.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f18e1e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f14150690 0x7f7f14157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:55.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f18e1e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f141567e0 0x7f7f14146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:55.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f18e1e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14158700 con 0x7f7f141567e0 2024-09-13T18:07:55.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f18e1e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1406bd40 con 0x7f7f14070bc0 2024-09-13T18:07:55.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f18e1e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f141485b0 con 0x7f7f14150690 2024-09-13T18:07:55.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f13fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f141567e0 0x7f7f14146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:55.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f13fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f141567e0 0x7f7f14146530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58442/0 (socket says 172.21.15.40:58442) 2024-09-13T18:07:55.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f13fff700 1 -- 172.21.15.40:0/245984146 learned_addr learned my addr 172.21.15.40:0/245984146 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:55.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f12ffd700 1 --2- 172.21.15.40:0/245984146 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f14150690 0x7f7f14157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:55.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f137fe700 1 --2- 172.21.15.40:0/245984146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f14070bc0 0x7f7f14154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:55.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f13fff700 1 -- 172.21.15.40:0/245984146 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f14150690 msgr2=0x7f7f14157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:55.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f13fff700 1 --2- 172.21.15.40:0/245984146 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f14150690 0x7f7f14157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:55.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f137fe700 1 -- 172.21.15.40:0/245984146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f14070bc0 msgr2=0x7f7f14154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:07:55.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f137fe700 1 -- 172.21.15.40:0/245984146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f14070bc0 msgr2=0x7f7f14154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:07:55.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f13fff700 1 -- 172.21.15.40:0/245984146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f14070bc0 msgr2=0x7f7f14154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:55.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f137fe700 1 --2- 172.21.15.40:0/245984146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f14070bc0 0x7f7f14154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:07:55.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f137fe700 1 --2- 172.21.15.40:0/245984146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f14070bc0 0x7f7f14154d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:07:55.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f13fff700 1 --2- 172.21.15.40:0/245984146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f14070bc0 0x7f7f14154d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:55.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f13fff700 1 -- 172.21.15.40:0/245984146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0c009b30 con 0x7f7f141567e0 2024-09-13T18:07:55.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f13fff700 1 --2- 172.21.15.40:0/245984146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f141567e0 0x7f7f14146530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00b680 tx=0x7f7f0c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:55.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7efbfff700 1 -- 172.21.15.40:0/245984146 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0048e0 con 0x7f7f141567e0 2024-09-13T18:07:55.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7efbfff700 1 -- 172.21.15.40:0/245984146 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c0092e0 con 0x7f7f141567e0 2024-09-13T18:07:55.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7efbfff700 1 -- 172.21.15.40:0/245984146 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c01c660 con 0x7f7f141567e0 2024-09-13T18:07:55.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f18e1e700 1 -- 172.21.15.40:0/245984146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f14148da0 con 0x7f7f141567e0 2024-09-13T18:07:55.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f18e1e700 1 -- 172.21.15.40:0/245984146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f14158140 con 0x7f7f141567e0 2024-09-13T18:07:55.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.659+0000 7f7f18e1e700 1 -- 172.21.15.40:0/245984146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f14071430 con 0x7f7f141567e0 2024-09-13T18:07:55.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.663+0000 7f7efbfff700 1 -- 172.21.15.40:0/245984146 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7f0c01c7c0 con 0x7f7f141567e0 2024-09-13T18:07:55.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.663+0000 7f7efbfff700 1 --2- 172.21.15.40:0/245984146 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7f00066730 0x7f7f00068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:55.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.663+0000 7f7efbfff700 1 -- 172.21.15.40:0/245984146 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7f0c085da0 con 0x7f7f141567e0 2024-09-13T18:07:55.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.663+0000 7f7f137fe700 1 --2- 172.21.15.40:0/245984146 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7f00066730 0x7f7f00068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:55.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.663+0000 7f7f137fe700 1 --2- 172.21.15.40:0/245984146 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7f00066730 0x7f7f00068bf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7f04000f60 tx=0x7f7f04007d10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:55.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.667+0000 7f7efbfff700 1 -- 172.21.15.40:0/245984146 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7f0c052d50 con 0x7f7f141567e0 2024-09-13T18:07:55.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.923+0000 7f7f18e1e700 1 -- 172.21.15.40:0/245984146 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f7f140cb770 con 0x7f7f141567e0 2024-09-13T18:07:55.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.923+0000 7f7efbfff700 1 -- 172.21.15.40:0/245984146 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f7f0c056370 con 0x7f7f141567e0 2024-09-13T18:07:55.933 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":144,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:07:55.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.927+0000 7f7f18e1e700 1 -- 172.21.15.40:0/245984146 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7f00066730 msgr2=0x7f7f00068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:55.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.927+0000 7f7f18e1e700 1 --2- 172.21.15.40:0/245984146 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7f00066730 0x7f7f00068bf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7f04000f60 tx=0x7f7f04007d10 comp rx=0 tx=0).stop 2024-09-13T18:07:55.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.927+0000 7f7f18e1e700 1 -- 172.21.15.40:0/245984146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f141567e0 msgr2=0x7f7f14146530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:55.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.927+0000 7f7f18e1e700 1 --2- 172.21.15.40:0/245984146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f141567e0 0x7f7f14146530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00b680 tx=0x7f7f0c004af0 comp rx=0 tx=0).stop 2024-09-13T18:07:55.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.927+0000 7f7f18e1e700 1 -- 172.21.15.40:0/245984146 shutdown_connections 2024-09-13T18:07:55.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.927+0000 7f7f18e1e700 1 --2- 172.21.15.40:0/245984146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7f141567e0 0x7f7f14146530 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:55.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.927+0000 7f7f18e1e700 1 --2- 172.21.15.40:0/245984146 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7f00066730 0x7f7f00068bf0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:55.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.927+0000 7f7f18e1e700 1 --2- 172.21.15.40:0/245984146 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f14150690 0x7f7f14157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:55.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.927+0000 7f7f18e1e700 1 --2- 172.21.15.40:0/245984146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f14070bc0 0x7f7f14154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:55.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.927+0000 7f7f18e1e700 1 -- 172.21.15.40:0/245984146 >> 172.21.15.40:0/245984146 conn(0x7f7f140c88e0 msgr2=0x7f7f1414d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:55.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.927+0000 7f7f18e1e700 1 -- 172.21.15.40:0/245984146 shutdown_connections 2024-09-13T18:07:55.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:55.927+0000 7f7f18e1e700 1 -- 172.21.15.40:0/245984146 wait complete. 2024-09-13T18:07:58.949 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:07:59.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0dc2470700 1 Processor -- start 2024-09-13T18:07:59.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0dc2470700 1 -- start start 2024-09-13T18:07:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0dc2470700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0dbc06e9e0 0x7f0dbc070d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0dc2470700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0dbc070290 0x7f0dbc071250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0dc2470700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dbc157af0 con 0x7f0dbc151180 2024-09-13T18:07:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0dc2470700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dbc158020 con 0x7f0dbc06e9e0 2024-09-13T18:07:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0dc2470700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc157d40 con 0x7f0dbc070290 2024-09-13T18:07:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0dc146e700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0dbc151180 0x7f0dbc0c4a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40268/0 (socket says 172.21.15.40:40268) 2024-09-13T18:07:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0dc146e700 1 -- 172.21.15.40:0/3538815570 learned_addr learned my addr 172.21.15.40:0/3538815570 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:07:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0dc0c6d700 1 --2- 172.21.15.40:0/3538815570 >> v2:172.21.15.43:3300/0 conn(0x7f0dbc06e9e0 0x7f0dbc070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0dc1c6f700 1 --2- 172.21.15.40:0/3538815570 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0dbc070290 0x7f0dbc071250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0db3fff700 1 -- 172.21.15.40:0/3538815570 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581454318 0 0) 0x7f0dbc157af0 con 0x7f0dbc151180 2024-09-13T18:07:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0dc0c6d700 1 -- 172.21.15.40:0/3538815570 >> v2:172.21.15.43:3300/0 conn(0x7f0dbc06e9e0 msgr2=0x7f0dbc070d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:07:59.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0dc0c6d700 1 -- 172.21.15.40:0/3538815570 >> v2:172.21.15.43:3300/0 conn(0x7f0dbc06e9e0 msgr2=0x7f0dbc070d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:07:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0dc0c6d700 1 --2- 172.21.15.40:0/3538815570 >> v2:172.21.15.43:3300/0 conn(0x7f0dbc06e9e0 0x7f0dbc070d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:07:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0db3fff700 1 -- 172.21.15.40:0/3538815570 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0da80035d0 con 0x7f0dbc151180 2024-09-13T18:07:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0dc0c6d700 1 --2- 172.21.15.40:0/3538815570 >> v2:172.21.15.43:3300/0 conn(0x7f0dbc06e9e0 0x7f0dbc070d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:07:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0db3fff700 1 -- 172.21.15.40:0/3538815570 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 173896028 0 0) 0x7f0da80035d0 con 0x7f0dbc151180 2024-09-13T18:07:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0dc1c6f700 1 -- 172.21.15.40:0/3538815570 >> v2:172.21.15.43:3300/0 conn(0x7f0dbc06e9e0 msgr2=0x7f0dbc070d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:07:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0dc1c6f700 1 --2- 172.21.15.40:0/3538815570 >> v2:172.21.15.43:3300/0 conn(0x7f0dbc06e9e0 0x7f0dbc070d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0dc1c6f700 1 -- 172.21.15.40:0/3538815570 >> v1:172.21.15.40:6789/0 conn(0x7f0dbc151180 legacy=0x7f0dbc0c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0dc1c6f700 1 -- 172.21.15.40:0/3538815570 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc06d630 con 0x7f0dbc070290 2024-09-13T18:07:59.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0dc1c6f700 1 --2- 172.21.15.40:0/3538815570 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0dbc070290 0x7f0dbc071250 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0db400f370 tx=0x7f0db400f680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f20821389f5d4e3b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:59.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0db3fff700 1 -- 172.21.15.40:0/3538815570 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db4012070 con 0x7f0dbc070290 2024-09-13T18:07:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0db3fff700 1 -- 172.21.15.40:0/3538815570 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db400fd70 con 0x7f0dbc070290 2024-09-13T18:07:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.066+0000 7f0db3fff700 1 -- 172.21.15.40:0/3538815570 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db8001670 con 0x7f0dbc070290 2024-09-13T18:07:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.070+0000 7f0dc2470700 1 -- 172.21.15.40:0/3538815570 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0dbc070290 msgr2=0x7f0dbc071250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.070+0000 7f0dc2470700 1 --2- 172.21.15.40:0/3538815570 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0dbc070290 0x7f0dbc071250 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0db400f370 tx=0x7f0db400f680 comp rx=0 tx=0).stop 2024-09-13T18:07:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.070+0000 7f0dc2470700 1 -- 172.21.15.40:0/3538815570 shutdown_connections 2024-09-13T18:07:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.070+0000 7f0dc2470700 1 --2- 172.21.15.40:0/3538815570 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0dbc070290 0x7f0dbc071250 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.070+0000 7f0dc2470700 1 --2- 172.21.15.40:0/3538815570 >> v2:172.21.15.43:3300/0 conn(0x7f0dbc06e9e0 0x7f0dbc070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.070+0000 7f0dc2470700 1 -- 172.21.15.40:0/3538815570 >> 172.21.15.40:0/3538815570 conn(0x7f0dbc06a390 msgr2=0x7f0dbc06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.070+0000 7f0dc2470700 1 -- 172.21.15.40:0/3538815570 shutdown_connections 2024-09-13T18:07:59.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.070+0000 7f0dc2470700 1 -- 172.21.15.40:0/3538815570 wait complete. 2024-09-13T18:07:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.070+0000 7f0dc2470700 1 Processor -- start 2024-09-13T18:07:59.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.070+0000 7f0dc2470700 1 -- start start 2024-09-13T18:07:59.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc2470700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dbc06e9e0 0x7f0dbc0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc2470700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc070290 0x7f0dbc147700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc2470700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0dbc151180 0x7f0dbc147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc2470700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc14ecf0 con 0x7f0dbc06e9e0 2024-09-13T18:07:59.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc2470700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc1eb3b0 con 0x7f0dbc070290 2024-09-13T18:07:59.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc2470700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc14e9a0 con 0x7f0dbc151180 2024-09-13T18:07:59.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc146e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dbc06e9e0 0x7f0dbc0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc146e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dbc06e9e0 0x7f0dbc0cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58456/0 (socket says 172.21.15.40:58456) 2024-09-13T18:07:59.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc146e700 1 -- 172.21.15.40:0/4158019844 learned_addr learned my addr 172.21.15.40:0/4158019844 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:59.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc0c6d700 1 --2- 172.21.15.40:0/4158019844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc070290 0x7f0dbc147700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc1c6f700 1 --2- 172.21.15.40:0/4158019844 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0dbc151180 0x7f0dbc147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc0c6d700 1 -- 172.21.15.40:0/4158019844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc070290 msgr2=0x7f0dbc147700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:07:59.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc0c6d700 1 -- 172.21.15.40:0/4158019844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc070290 msgr2=0x7f0dbc147700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:07:59.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc0c6d700 1 --2- 172.21.15.40:0/4158019844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc070290 0x7f0dbc147700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:07:59.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc0c6d700 1 --2- 172.21.15.40:0/4158019844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc070290 0x7f0dbc147700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:07:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc146e700 1 -- 172.21.15.40:0/4158019844 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0dbc151180 msgr2=0x7f0dbc147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc146e700 1 --2- 172.21.15.40:0/4158019844 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0dbc151180 0x7f0dbc147c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc146e700 1 -- 172.21.15.40:0/4158019844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc070290 msgr2=0x7f0dbc147700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:07:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc146e700 1 --2- 172.21.15.40:0/4158019844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc070290 0x7f0dbc147700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc146e700 1 -- 172.21.15.40:0/4158019844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db400f050 con 0x7f0dbc06e9e0 2024-09-13T18:07:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc146e700 1 --2- 172.21.15.40:0/4158019844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dbc06e9e0 0x7f0dbc0cb590 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0db800c770 tx=0x7f0db800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0db1ffb700 1 -- 172.21.15.40:0/4158019844 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db8004ca0 con 0x7f0dbc06e9e0 2024-09-13T18:07:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0db1ffb700 1 -- 172.21.15.40:0/4158019844 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db8004e00 con 0x7f0dbc06e9e0 2024-09-13T18:07:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc2470700 1 -- 172.21.15.40:0/4158019844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dbc1eb8a0 con 0x7f0dbc06e9e0 2024-09-13T18:07:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0db1ffb700 1 -- 172.21.15.40:0/4158019844 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db80072f0 con 0x7f0dbc06e9e0 2024-09-13T18:07:59.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc2470700 1 -- 172.21.15.40:0/4158019844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dbc1eb5e0 con 0x7f0dbc06e9e0 2024-09-13T18:07:59.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0dc2470700 1 -- 172.21.15.40:0/4158019844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dbc1549f0 con 0x7f0dbc06e9e0 2024-09-13T18:07:59.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.074+0000 7f0db1ffb700 1 -- 172.21.15.40:0/4158019844 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0db8007450 con 0x7f0dbc06e9e0 2024-09-13T18:07:59.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.078+0000 7f0db1ffb700 1 --2- 172.21.15.40:0/4158019844 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0da0066730 0x7f0da0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.078+0000 7f0db1ffb700 1 -- 172.21.15.40:0/4158019844 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0db8083630 con 0x7f0dbc06e9e0 2024-09-13T18:07:59.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.078+0000 7f0dc0c6d700 1 --2- 172.21.15.40:0/4158019844 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0da0066730 0x7f0da0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.078+0000 7f0dc0c6d700 1 --2- 172.21.15.40:0/4158019844 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0da0066730 0x7f0da0068bf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc148730 tx=0x7f0dac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:59.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.082+0000 7f0db1ffb700 1 -- 172.21.15.40:0/4158019844 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0db80505f0 con 0x7f0dbc06e9e0 2024-09-13T18:07:59.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.346+0000 7f0dc2470700 1 -- 172.21.15.40:0/4158019844 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0dbc06e4a0 con 0x7f0dbc06e9e0 2024-09-13T18:07:59.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.346+0000 7f0db1ffb700 1 -- 172.21.15.40:0/4158019844 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f0db8053c10 con 0x7f0dbc06e9e0 2024-09-13T18:07:59.359 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":146,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:07:59.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.350+0000 7f0dc2470700 1 -- 172.21.15.40:0/4158019844 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0da0066730 msgr2=0x7f0da0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.350+0000 7f0dc2470700 1 --2- 172.21.15.40:0/4158019844 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0da0066730 0x7f0da0068bf0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc148730 tx=0x7f0dac008040 comp rx=0 tx=0).stop 2024-09-13T18:07:59.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.350+0000 7f0dc2470700 1 -- 172.21.15.40:0/4158019844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dbc06e9e0 msgr2=0x7f0dbc0cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.350+0000 7f0dc2470700 1 --2- 172.21.15.40:0/4158019844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dbc06e9e0 0x7f0dbc0cb590 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0db800c770 tx=0x7f0db800cb30 comp rx=0 tx=0).stop 2024-09-13T18:07:59.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.350+0000 7f0dc2470700 1 -- 172.21.15.40:0/4158019844 shutdown_connections 2024-09-13T18:07:59.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.350+0000 7f0dc2470700 1 --2- 172.21.15.40:0/4158019844 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0da0066730 0x7f0da0068bf0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.350+0000 7f0dc2470700 1 --2- 172.21.15.40:0/4158019844 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0dbc151180 0x7f0dbc147c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.350+0000 7f0dc2470700 1 --2- 172.21.15.40:0/4158019844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc070290 0x7f0dbc147700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.350+0000 7f0dc2470700 1 --2- 172.21.15.40:0/4158019844 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0dbc06e9e0 0x7f0dbc0cb590 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.350+0000 7f0dc2470700 1 -- 172.21.15.40:0/4158019844 >> 172.21.15.40:0/4158019844 conn(0x7f0dbc06a390 msgr2=0x7f0dbc156ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:59.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.350+0000 7f0dc2470700 1 -- 172.21.15.40:0/4158019844 shutdown_connections 2024-09-13T18:07:59.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.350+0000 7f0dc2470700 1 -- 172.21.15.40:0/4158019844 wait complete. 2024-09-13T18:07:59.374 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:07:59.374 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:07:59.374 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:07:59.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.474+0000 7fc91a8b1700 1 Processor -- start 2024-09-13T18:07:59.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.474+0000 7fc91a8b1700 1 -- start start 2024-09-13T18:07:59.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc91a8b1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc9141577d0 0x7fc9140c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc91a8b1700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc914070ae0 con 0x7fc91415d8a0 2024-09-13T18:07:59.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc91a8b1700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc914070d80 con 0x7fc9140b1c60 2024-09-13T18:07:59.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc91a8b1700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc914071020 con 0x7fc9141577d0 2024-09-13T18:07:59.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9190ae700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc91415d8a0 0x7fc9141616b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40276/0 (socket says 172.21.15.40:40276) 2024-09-13T18:07:59.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9190ae700 1 -- 172.21.15.40:0/2806004005 learned_addr learned my addr 172.21.15.40:0/2806004005 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:07:59.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9198af700 1 --2- 172.21.15.40:0/2806004005 >> v2:172.21.15.40:3300/0 conn(0x7fc9141577d0 0x7fc9140c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9188ad700 1 -- 172.21.15.40:0/2806004005 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 430154068 0 0) 0x7fc914070ae0 con 0x7fc91415d8a0 2024-09-13T18:07:59.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9188ad700 1 -- 172.21.15.40:0/2806004005 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9000035d0 con 0x7fc91415d8a0 2024-09-13T18:07:59.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9188ad700 1 -- 172.21.15.40:0/2806004005 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 867687551 0 0) 0x7fc9000035d0 con 0x7fc91415d8a0 2024-09-13T18:07:59.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9188ad700 1 -- 172.21.15.40:0/2806004005 >> v2:172.21.15.40:3300/0 conn(0x7fc9141577d0 msgr2=0x7fc9140c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:59.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9188ad700 1 --2- 172.21.15.40:0/2806004005 >> v2:172.21.15.40:3300/0 conn(0x7fc9141577d0 0x7fc9140c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9188ad700 1 -- 172.21.15.40:0/2806004005 >> v1:172.21.15.43:6789/0 conn(0x7fc9140b1c60 legacy=0x7fc9140c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9188ad700 1 -- 172.21.15.40:0/2806004005 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9140712c0 con 0x7fc91415d8a0 2024-09-13T18:07:59.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9188ad700 1 -- 172.21.15.40:0/2806004005 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fc910001670 con 0x7fc91415d8a0 2024-09-13T18:07:59.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9188ad700 1 -- 172.21.15.40:0/2806004005 >> v1:172.21.15.40:6789/0 conn(0x7fc91415d8a0 legacy=0x7fc9141616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9188ad700 1 --2- 172.21.15.40:0/2806004005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc900003a20 0x7fc900003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9188ad700 1 --2- 172.21.15.40:0/2806004005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc900004870 0x7fc900004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9188ad700 1 --2- 172.21.15.40:0/2806004005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc900005230 0x7fc90000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9188ad700 1 -- 172.21.15.40:0/2806004005 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc914071020 con 0x7fc900004870 2024-09-13T18:07:59.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9188ad700 1 -- 172.21.15.40:0/2806004005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90000c0e0 con 0x7fc900005230 2024-09-13T18:07:59.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9188ad700 1 -- 172.21.15.40:0/2806004005 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90000c2a0 con 0x7fc900003a20 2024-09-13T18:07:59.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9190ae700 1 --2- 172.21.15.40:0/2806004005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc900004870 0x7fc900004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc91a0b0700 1 --2- 172.21.15.40:0/2806004005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc900005230 0x7fc90000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9198af700 1 --2- 172.21.15.40:0/2806004005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc900003a20 0x7fc900003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9190ae700 1 -- 172.21.15.40:0/2806004005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc900003a20 msgr2=0x7fc900003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:59.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9190ae700 1 --2- 172.21.15.40:0/2806004005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc900003a20 0x7fc900003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9190ae700 1 -- 172.21.15.40:0/2806004005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc900005230 msgr2=0x7fc90000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:59.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9190ae700 1 --2- 172.21.15.40:0/2806004005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc900005230 0x7fc90000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc91a0b0700 1 -- 172.21.15.40:0/2806004005 reap_dead start 2024-09-13T18:07:59.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9190ae700 1 -- 172.21.15.40:0/2806004005 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9000035d0 con 0x7fc900004870 2024-09-13T18:07:59.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9190ae700 1 --2- 172.21.15.40:0/2806004005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc900004870 0x7fc900004cf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc91000c2f0 tx=0x7fc9000058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3473a09e59175 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:59.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9188ad700 1 -- 172.21.15.40:0/2806004005 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc910007c30 con 0x7fc900004870 2024-09-13T18:07:59.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9188ad700 1 -- 172.21.15.40:0/2806004005 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91000ce30 con 0x7fc900004870 2024-09-13T18:07:59.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.478+0000 7fc9188ad700 1 -- 172.21.15.40:0/2806004005 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc910005ba0 con 0x7fc900004870 2024-09-13T18:07:59.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.482+0000 7fc91a8b1700 1 -- 172.21.15.40:0/2806004005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc900004870 msgr2=0x7fc900004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.482+0000 7fc91a8b1700 1 --2- 172.21.15.40:0/2806004005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc900004870 0x7fc900004cf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc91000c2f0 tx=0x7fc9000058a0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.482+0000 7fc91a8b1700 1 -- 172.21.15.40:0/2806004005 shutdown_connections 2024-09-13T18:07:59.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.482+0000 7fc91a8b1700 1 --2- 172.21.15.40:0/2806004005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc900004870 0x7fc900004cf0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.482+0000 7fc91a8b1700 1 -- 172.21.15.40:0/2806004005 >> 172.21.15.40:0/2806004005 conn(0x7fc91406e6d0 msgr2=0x7fc91406eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:59.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.482+0000 7fc91a8b1700 1 -- 172.21.15.40:0/2806004005 shutdown_connections 2024-09-13T18:07:59.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.482+0000 7fc91a8b1700 1 -- 172.21.15.40:0/2806004005 wait complete. 2024-09-13T18:07:59.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.482+0000 7fc91a8b1700 1 Processor -- start 2024-09-13T18:07:59.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.482+0000 7fc91a8b1700 1 -- start start 2024-09-13T18:07:59.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.482+0000 7fc91a8b1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc900004870 0x7fc9140c1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.482+0000 7fc91a8b1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc914156bf0 0x7fc9141f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.482+0000 7fc91a8b1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9141f3e80 0x7fc9140c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.482+0000 7fc91a8b1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9141f7660 con 0x7fc9141f3e80 2024-09-13T18:07:59.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.482+0000 7fc91a8b1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9141f4610 con 0x7fc914156bf0 2024-09-13T18:07:59.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.482+0000 7fc91a8b1700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9141f31a0 con 0x7fc900004870 2024-09-13T18:07:59.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.482+0000 7fc91a0b0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9141f3e80 0x7fc9140c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.482+0000 7fc91a0b0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9141f3e80 0x7fc9140c5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58492/0 (socket says 172.21.15.40:58492) 2024-09-13T18:07:59.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.482+0000 7fc91a0b0700 1 -- 172.21.15.40:0/3795393847 learned_addr learned my addr 172.21.15.40:0/3795393847 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:59.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.482+0000 7fc9190ae700 1 --2- 172.21.15.40:0/3795393847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc914156bf0 0x7fc9141f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.486+0000 7fc9198af700 1 --2- 172.21.15.40:0/3795393847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc900004870 0x7fc9140c1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.486+0000 7fc91a0b0700 1 -- 172.21.15.40:0/3795393847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc900004870 msgr2=0x7fc9140c1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.486+0000 7fc91a0b0700 1 --2- 172.21.15.40:0/3795393847 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc900004870 0x7fc9140c1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.486+0000 7fc91a0b0700 1 -- 172.21.15.40:0/3795393847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc914156bf0 msgr2=0x7fc9141f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.486+0000 7fc91a0b0700 1 --2- 172.21.15.40:0/3795393847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc914156bf0 0x7fc9141f3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.486+0000 7fc91a0b0700 1 -- 172.21.15.40:0/3795393847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91415fbf0 con 0x7fc9141f3e80 2024-09-13T18:07:59.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.486+0000 7fc91a0b0700 1 --2- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9141f3e80 0x7fc9140c5660 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc91415b4d0 tx=0x7fc90400f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:59.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.486+0000 7fc90a7fc700 1 -- 172.21.15.40:0/3795393847 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9040050d0 con 0x7fc9141f3e80 2024-09-13T18:07:59.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.486+0000 7fc91a8b1700 1 -- 172.21.15.40:0/3795393847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc91415fed0 con 0x7fc9141f3e80 2024-09-13T18:07:59.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.486+0000 7fc90a7fc700 1 -- 172.21.15.40:0/3795393847 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc904005250 con 0x7fc9141f3e80 2024-09-13T18:07:59.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.486+0000 7fc90a7fc700 1 -- 172.21.15.40:0/3795393847 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9040036a0 con 0x7fc9141f3e80 2024-09-13T18:07:59.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.486+0000 7fc91a8b1700 1 -- 172.21.15.40:0/3795393847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9141f3420 con 0x7fc9141f3e80 2024-09-13T18:07:59.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.486+0000 7fc91a8b1700 1 --2- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9141f7120 0x7fc914152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.486+0000 7fc91a8b1700 1 -- 172.21.15.40:0/3795393847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc914067600 con 0x7fc9141f7120 2024-09-13T18:07:59.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.486+0000 7fc9198af700 1 --2- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9141f7120 0x7fc914152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.486+0000 7fc9198af700 1 -- 172.21.15.40:0/3795393847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc914160eb0 con 0x7fc9141f7120 2024-09-13T18:07:59.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.486+0000 7fc9198af700 1 --2- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9141f7120 0x7fc914152210 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc90c004980 tx=0x7fc90c007610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:59.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.486+0000 7fc90a7fc700 1 -- 172.21.15.40:0/3795393847 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c00e030 con 0x7fc9141f7120 2024-09-13T18:07:59.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.486+0000 7fc90a7fc700 1 -- 172.21.15.40:0/3795393847 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc904003800 con 0x7fc9141f3e80 2024-09-13T18:07:59.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.486+0000 7fc90a7fc700 1 --2- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8f80667d0 0x7fc8f8068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.490+0000 7fc90a7fc700 1 -- 172.21.15.40:0/3795393847 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc904083e30 con 0x7fc9141f3e80 2024-09-13T18:07:59.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.490+0000 7fc9190ae700 1 --2- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8f80667d0 0x7fc8f8068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.490+0000 7fc90a7fc700 1 -- 172.21.15.40:0/3795393847 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc90c00f040 con 0x7fc9141f7120 2024-09-13T18:07:59.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.490+0000 7fc90a7fc700 1 -- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9141f7120 msgr2=0x7fc914152210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.490+0000 7fc90a7fc700 1 --2- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9141f7120 0x7fc914152210 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc90c004980 tx=0x7fc90c007610 comp rx=0 tx=0).stop 2024-09-13T18:07:59.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.490+0000 7fc90a7fc700 1 -- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9141f7120 msgr2=0x7fc914152210 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:59.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.490+0000 7fc90a7fc700 1 --2- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9141f7120 0x7fc914152210 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.490+0000 7fc9190ae700 1 --2- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8f80667d0 0x7fc8f8068c90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc910001930 tx=0x7fc91000a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:59.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc91a8b1700 1 --2- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9140c5ba0 0x7fc914152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc91a8b1700 1 -- 172.21.15.40:0/3795393847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc914071840 con 0x7fc9140c5ba0 2024-09-13T18:07:59.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc9198af700 1 --2- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9140c5ba0 0x7fc914152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc9198af700 1 -- 172.21.15.40:0/3795393847 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc914161190 con 0x7fc9140c5ba0 2024-09-13T18:07:59.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc9198af700 1 --2- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9140c5ba0 0x7fc914152750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc90c004980 tx=0x7fc90c004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:59.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc90a7fc700 1 -- 172.21.15.40:0/3795393847 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c02b430 con 0x7fc9140c5ba0 2024-09-13T18:07:59.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc90a7fc700 1 -- 172.21.15.40:0/3795393847 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4784 (secure 0 0 0) 0x7fc90c02b590 con 0x7fc9140c5ba0 2024-09-13T18:07:59.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc90a7fc700 1 -- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9140c5ba0 msgr2=0x7fc914152750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc90a7fc700 1 --2- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9140c5ba0 0x7fc914152750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc90c004980 tx=0x7fc90c004b60 comp rx=0 tx=0).stop 2024-09-13T18:07:59.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc90a7fc700 1 -- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9140c5ba0 msgr2=0x7fc914152750 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:59.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc90a7fc700 1 --2- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9140c5ba0 0x7fc914152750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.511 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:07:59.511 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:07:59.512 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:07:59.512 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:07:59.512 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 146, 2024-09-13T18:07:59.512 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:07:59.512 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:07:59.512 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:07:59.513 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:07:59.513 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:59.513 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:07:59.513 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:59.513 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:07:59.514 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:07:59.514 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:59.514 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:59.514 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:59.514 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:59.514 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:59.515 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:59.515 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:59.515 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:59.515 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:59.515 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:59.515 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:59.516 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:07:59.516 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:07:59.516 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:59.516 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:59.516 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:59.517 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:59.517 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:59.517 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:59.517 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:59.517 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:59.517 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:59.518 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:59.518 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:59.518 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.518 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:07:59.518 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:07:59.518 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:07:59.519 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:07:59.519 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:07:59.519 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:07:59.519 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:59.519 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:59.519 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:07:59.520 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:07:59.520 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:07:59.520 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:07:59.520 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:07:59.520 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:07:59.521 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:07:59.521 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:59.521 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:07:59.521 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:59.521 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:59.521 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:59.522 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:59.522 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:59.522 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:59.522 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:59.522 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:59.522 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:59.523 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:59.523 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:59.523 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:07:59.523 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.523 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:07:59.523 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.524 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:07:59.524 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:07:59.524 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:59.524 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:59.524 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.524 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:59.525 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:07:59.525 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:59.525 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.525 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.525 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:59.525 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:07:59.525 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:59.526 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.526 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:59.526 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.526 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:07:59.526 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:07:59.526 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:59.527 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:59.527 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:59.527 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.527 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.527 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:07:59.527 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:07:59.527 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:59.527 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:59.527 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.527 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:59.528 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:07:59.528 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:59.528 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.528 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.528 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:59.528 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:07:59.528 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:59.528 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.528 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:59.528 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.528 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:07:59.528 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:07:59.528 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:59.528 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:59.529 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:59.529 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.529 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.529 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:07:59.529 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:07:59.529 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:59.529 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:59.529 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.529 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:59.529 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:07:59.529 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:59.529 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.529 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.530 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:59.530 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:07:59.530 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:59.530 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.530 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:59.530 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.530 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:07:59.530 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:07:59.530 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:59.530 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:59.530 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:59.530 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.531 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:59.531 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.531 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:07:59.531 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:07:59.531 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.531 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:59.531 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:59.531 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:07:59.531 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.531 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:59.532 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:07:59.532 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.532 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:59.532 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:59.532 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:07:59.532 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.532 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:59.532 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:07:59.533 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.533 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:59.533 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:59.533 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:07:59.536 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.536 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:59.536 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:07:59.536 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.536 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:59.537 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:59.537 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:07:59.537 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.537 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:59.537 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.537 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:07:59.537 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:07:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc91a8b1700 1 -- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8f80667d0 msgr2=0x7fc8f8068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc91a8b1700 1 --2- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc8f80667d0 0x7fc8f8068c90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc910001930 tx=0x7fc91000a000 comp rx=0 tx=0).stop 2024-09-13T18:07:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc91a8b1700 1 -- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9141f3e80 msgr2=0x7fc9140c5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc91a8b1700 1 --2- 172.21.15.40:0/3795393847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9141f3e80 0x7fc9140c5660 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc91415b4d0 tx=0x7fc90400f820 comp rx=0 tx=0).stop 2024-09-13T18:07:59.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc91a0b0700 1 -- 172.21.15.40:0/3795393847 reap_dead start 2024-09-13T18:07:59.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc91a0b0700 1 -- 172.21.15.40:0/3795393847 reap_dead start 2024-09-13T18:07:59.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc91a8b1700 1 -- 172.21.15.40:0/3795393847 shutdown_connections 2024-09-13T18:07:59.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc91a8b1700 1 -- 172.21.15.40:0/3795393847 >> 172.21.15.40:0/3795393847 conn(0x7fc91406e6d0 msgr2=0x7fc9140c6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:59.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc91a8b1700 1 -- 172.21.15.40:0/3795393847 shutdown_connections 2024-09-13T18:07:59.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.498+0000 7fc91a8b1700 1 -- 172.21.15.40:0/3795393847 wait complete. 2024-09-13T18:07:59.542 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:07:59.542 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:07:59.543 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:07:59.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.658+0000 7f5a9b0c0700 1 Processor -- start 2024-09-13T18:07:59.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.658+0000 7f5a9b0c0700 1 -- start start 2024-09-13T18:07:59.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9b0c0700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a940cf580 0x7f5a9415a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9b0c0700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5a940cc250 0x7f5a940c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9b0c0700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a9415a7a0 con 0x7f5a940cf1a0 2024-09-13T18:07:59.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9b0c0700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a940cbdc0 con 0x7f5a940cc250 2024-09-13T18:07:59.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9b0c0700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a94153f80 con 0x7f5a940cf580 2024-09-13T18:07:59.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9a0be700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5a940cf1a0 0x7f5a940c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40282/0 (socket says 172.21.15.40:40282) 2024-09-13T18:07:59.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9a0be700 1 -- 172.21.15.40:0/2324905101 learned_addr learned my addr 172.21.15.40:0/2324905101 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:07:59.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9a8bf700 1 --2- 172.21.15.40:0/2324905101 >> v2:172.21.15.40:3300/0 conn(0x7f5a940cc250 0x7f5a940c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a998bd700 1 --2- 172.21.15.40:0/2324905101 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a940cf580 0x7f5a9415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a990bc700 1 -- 172.21.15.40:0/2324905101 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1208608634 0 0) 0x7f5a90001110 con 0x7f5a940cf1a0 2024-09-13T18:07:59.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a990bc700 1 -- 172.21.15.40:0/2324905101 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a800035d0 con 0x7f5a940cf1a0 2024-09-13T18:07:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9a8bf700 1 -- 172.21.15.40:0/2324905101 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a940cf580 msgr2=0x7f5a9415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9a8bf700 1 --2- 172.21.15.40:0/2324905101 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a940cf580 0x7f5a9415a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9a8bf700 1 -- 172.21.15.40:0/2324905101 >> v1:172.21.15.40:6789/0 conn(0x7f5a940cf1a0 legacy=0x7f5a940c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a990bc700 1 -- 172.21.15.40:0/2324905101 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3235430621 0 0) 0x7f5a800035d0 con 0x7f5a940cf1a0 2024-09-13T18:07:59.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9a8bf700 1 -- 172.21.15.40:0/2324905101 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a940caed0 con 0x7f5a940cc250 2024-09-13T18:07:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9a8bf700 1 --2- 172.21.15.40:0/2324905101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a940cc250 0x7f5a940c8aa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009d90 tx=0x7f5a8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5cb7eb3ceabd5c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a990bc700 1 -- 172.21.15.40:0/2324905101 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c00e070 con 0x7f5a940cc250 2024-09-13T18:07:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a990bc700 1 -- 172.21.15.40:0/2324905101 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8c003a40 con 0x7f5a940cc250 2024-09-13T18:07:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a990bc700 1 -- 172.21.15.40:0/2324905101 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a90001800 con 0x7f5a940cc250 2024-09-13T18:07:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9b0c0700 1 -- 172.21.15.40:0/2324905101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a940cc250 msgr2=0x7f5a940c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9b0c0700 1 --2- 172.21.15.40:0/2324905101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a940cc250 0x7f5a940c8aa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009d90 tx=0x7f5a8c00b330 comp rx=0 tx=0).stop 2024-09-13T18:07:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9b0c0700 1 -- 172.21.15.40:0/2324905101 shutdown_connections 2024-09-13T18:07:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9b0c0700 1 --2- 172.21.15.40:0/2324905101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a940cc250 0x7f5a940c8aa0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9b0c0700 1 --2- 172.21.15.40:0/2324905101 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a940cf580 0x7f5a9415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9b0c0700 1 -- 172.21.15.40:0/2324905101 >> 172.21.15.40:0/2324905101 conn(0x7f5a94066090 msgr2=0x7f5a94066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:59.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9b0c0700 1 -- 172.21.15.40:0/2324905101 shutdown_connections 2024-09-13T18:07:59.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.662+0000 7f5a9b0c0700 1 -- 172.21.15.40:0/2324905101 wait complete. 2024-09-13T18:07:59.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9b0c0700 1 Processor -- start 2024-09-13T18:07:59.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9b0c0700 1 -- start start 2024-09-13T18:07:59.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9b0c0700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a940cc250 0x7f5a940b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9b0c0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a940cf1a0 0x7f5a9414fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9b0c0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a940cf580 0x7f5a94150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9b0c0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a941ef020 con 0x7f5a940cf580 2024-09-13T18:07:59.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9b0c0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a94156dc0 con 0x7f5a940cf1a0 2024-09-13T18:07:59.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9b0c0700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a94157010 con 0x7f5a940cc250 2024-09-13T18:07:59.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9a8bf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a940cf580 0x7f5a94150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9a8bf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a940cf580 0x7f5a94150420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58530/0 (socket says 172.21.15.40:58530) 2024-09-13T18:07:59.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9a8bf700 1 -- 172.21.15.40:0/3392812500 learned_addr learned my addr 172.21.15.40:0/3392812500 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:59.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9a0be700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a940cc250 0x7f5a940b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a998bd700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a940cf1a0 0x7f5a9414fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9a8bf700 1 -- 172.21.15.40:0/3392812500 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a940cc250 msgr2=0x7f5a940b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9a8bf700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a940cc250 0x7f5a940b2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9a8bf700 1 -- 172.21.15.40:0/3392812500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a940cf1a0 msgr2=0x7f5a9414fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9a8bf700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a940cf1a0 0x7f5a9414fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9a8bf700 1 -- 172.21.15.40:0/3392812500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c009a70 con 0x7f5a940cf580 2024-09-13T18:07:59.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9a8bf700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a940cf580 0x7f5a94150420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5a9408c270 tx=0x7f5a8c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:59.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a8affd700 1 -- 172.21.15.40:0/3392812500 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c0045c0 con 0x7f5a940cf580 2024-09-13T18:07:59.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a8affd700 1 -- 172.21.15.40:0/3392812500 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8c004760 con 0x7f5a940cf580 2024-09-13T18:07:59.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9b0c0700 1 -- 172.21.15.40:0/3392812500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a941eed40 con 0x7f5a940cf580 2024-09-13T18:07:59.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a8affd700 1 -- 172.21.15.40:0/3392812500 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c012430 con 0x7f5a940cf580 2024-09-13T18:07:59.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9b0c0700 1 -- 172.21.15.40:0/3392812500 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9414e210 con 0x7f5a940cf580 2024-09-13T18:07:59.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9b0c0700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a940cd290 0x7f5a9414e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.666+0000 7f5a9b0c0700 1 -- 172.21.15.40:0/3392812500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a94074750 con 0x7f5a940cd290 2024-09-13T18:07:59.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.670+0000 7f5a9a0be700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a940cd290 0x7f5a9414e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.670+0000 7f5a9a0be700 1 -- 172.21.15.40:0/3392812500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a940b40a0 con 0x7f5a940cd290 2024-09-13T18:07:59.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.670+0000 7f5a8affd700 1 -- 172.21.15.40:0/3392812500 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5a8c012590 con 0x7f5a940cf580 2024-09-13T18:07:59.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.670+0000 7f5a9a0be700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a940cd290 0x7f5a9414e730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5a90000c00 tx=0x7f5a9000ea90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:59.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.670+0000 7f5a8affd700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5a78066730 0x7f5a78068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.670+0000 7f5a8affd700 1 -- 172.21.15.40:0/3392812500 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5a8c0865e0 con 0x7f5a940cf580 2024-09-13T18:07:59.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.670+0000 7f5a998bd700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5a78066730 0x7f5a78068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.670+0000 7f5a8affd700 1 -- 172.21.15.40:0/3392812500 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a90014480 con 0x7f5a940cd290 2024-09-13T18:07:59.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.670+0000 7f5a8affd700 1 -- 172.21.15.40:0/3392812500 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5a900145e0 con 0x7f5a940cd290 2024-09-13T18:07:59.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.670+0000 7f5a8affd700 1 -- 172.21.15.40:0/3392812500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a940cd290 msgr2=0x7f5a9414e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.670+0000 7f5a8affd700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a940cd290 0x7f5a9414e730 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5a90000c00 tx=0x7f5a9000ea90 comp rx=0 tx=0).stop 2024-09-13T18:07:59.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.670+0000 7f5a8affd700 1 -- 172.21.15.40:0/3392812500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a940cd290 msgr2=0x7f5a9414e730 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:59.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.670+0000 7f5a8affd700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a940cd290 0x7f5a9414e730 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.670+0000 7f5a998bd700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5a78066730 0x7f5a78068bf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5a940b3e10 tx=0x7f5a84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:59.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.678+0000 7f5a9b0c0700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a9414ec70 0x7f5a941480d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.678+0000 7f5a9b0c0700 1 -- 172.21.15.40:0/3392812500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9414fab0 con 0x7f5a9414ec70 2024-09-13T18:07:59.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.678+0000 7f5a9a0be700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a9414ec70 0x7f5a941480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.678+0000 7f5a9a0be700 1 -- 172.21.15.40:0/3392812500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5a940cb770 con 0x7f5a9414ec70 2024-09-13T18:07:59.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.678+0000 7f5a9a0be700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a9414ec70 0x7f5a941480d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5a90016000 tx=0x7f5a90001430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:59.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.682+0000 7f5a8affd700 1 -- 172.21.15.40:0/3392812500 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9001dad0 con 0x7f5a9414ec70 2024-09-13T18:07:59.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.682+0000 7f5a8affd700 1 -- 172.21.15.40:0/3392812500 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4450 (secure 0 0 0) 0x7f5a9001dc30 con 0x7f5a9414ec70 2024-09-13T18:07:59.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.682+0000 7f5a8affd700 1 -- 172.21.15.40:0/3392812500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a9414ec70 msgr2=0x7f5a941480d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.682+0000 7f5a8affd700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a9414ec70 0x7f5a941480d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5a90016000 tx=0x7f5a90001430 comp rx=0 tx=0).stop 2024-09-13T18:07:59.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.682+0000 7f5a8affd700 1 -- 172.21.15.40:0/3392812500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a9414ec70 msgr2=0x7f5a941480d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:59.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.682+0000 7f5a8affd700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a9414ec70 0x7f5a941480d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.692 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:07:59.692 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:07:59.692 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:07:59.692 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:07:59.692 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 146, 2024-09-13T18:07:59.692 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:07:59.692 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:07:59.692 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:07:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:07:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:07:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:07:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:07:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:07:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:07:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:59.695 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:59.695 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:59.695 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.695 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:07:59.695 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:07:59.695 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:07:59.695 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:07:59.695 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:07:59.695 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:07:59.695 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:59.695 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:59.696 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:07:59.696 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:07:59.696 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:07:59.696 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:07:59.696 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:07:59.696 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:07:59.696 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:07:59.696 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:59.696 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:07:59.696 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:59.696 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:59.696 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:59.696 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:59.697 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:59.697 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:59.697 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:59.697 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:59.697 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:59.697 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:59.697 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:59.697 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:07:59.697 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.697 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:07:59.697 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.697 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:07:59.698 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:07:59.698 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:59.698 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:59.698 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.698 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:59.698 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:07:59.698 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:59.698 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.698 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.698 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:59.698 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:07:59.698 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:59.699 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.699 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:59.699 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.699 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:07:59.699 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:07:59.699 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:59.699 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:59.699 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:59.699 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.699 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.699 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:07:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:07:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:07:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:07:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:07:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:07:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:07:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:07:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:07:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:59.703 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:07:59.703 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:59.703 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.703 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:59.703 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.703 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:07:59.703 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:07:59.703 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:59.703 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:59.703 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:59.703 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.703 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:59.704 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.704 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:07:59.704 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:07:59.704 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.704 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:59.704 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:59.704 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:07:59.704 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.704 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:59.704 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:07:59.705 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.705 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:59.705 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:59.705 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:07:59.705 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.705 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:59.705 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.705 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:07:59.705 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:07:59.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.682+0000 7f5a9b0c0700 1 -- 172.21.15.40:0/3392812500 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5a78066730 msgr2=0x7f5a78068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.682+0000 7f5a9b0c0700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5a78066730 0x7f5a78068bf0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5a940b3e10 tx=0x7f5a84008040 comp rx=0 tx=0).stop 2024-09-13T18:07:59.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.682+0000 7f5a9b0c0700 1 -- 172.21.15.40:0/3392812500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a940cf580 msgr2=0x7f5a94150420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.682+0000 7f5a9b0c0700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a940cf580 0x7f5a94150420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5a9408c270 tx=0x7f5a8c004af0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.682+0000 7f5a9a8bf700 1 -- 172.21.15.40:0/3392812500 reap_dead start 2024-09-13T18:07:59.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.682+0000 7f5a9b0c0700 1 -- 172.21.15.40:0/3392812500 shutdown_connections 2024-09-13T18:07:59.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.682+0000 7f5a9b0c0700 1 --2- 172.21.15.40:0/3392812500 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a940cf580 0x7f5a94150420 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.682+0000 7f5a9b0c0700 1 -- 172.21.15.40:0/3392812500 >> 172.21.15.40:0/3392812500 conn(0x7f5a94066090 msgr2=0x7f5a940c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:59.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.682+0000 7f5a9b0c0700 1 -- 172.21.15.40:0/3392812500 shutdown_connections 2024-09-13T18:07:59.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.682+0000 7f5a9b0c0700 1 -- 172.21.15.40:0/3392812500 wait complete. 2024-09-13T18:07:59.708 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:07:59.708 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:07:59.709 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:07:59.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f794a0d0700 1 Processor -- start 2024-09-13T18:07:59.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f794a0d0700 1 -- start start 2024-09-13T18:07:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f794a0d0700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79440b1fb0 0x7f7944157800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f794a0d0700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7944070ae0 con 0x7f7944071260 2024-09-13T18:07:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f794a0d0700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7944070d30 con 0x7f7944151580 2024-09-13T18:07:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f794a0d0700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79440c29c0 con 0x7f79440b1fb0 2024-09-13T18:07:59.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f79498cf700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7944071260 0x7f7944161b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40298/0 (socket says 172.21.15.40:40298) 2024-09-13T18:07:59.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f79498cf700 1 -- 172.21.15.40:0/3017300795 learned_addr learned my addr 172.21.15.40:0/3017300795 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:07:59.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f79490ce700 1 --1- 172.21.15.40:0/3017300795 >> v1:172.21.15.43:6789/0 conn(0x7f7944151580 0x7f79440c52b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:37758/0 (socket says 172.21.15.40:37758) 2024-09-13T18:07:59.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f79488cd700 1 --2- 172.21.15.40:0/3017300795 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79440b1fb0 0x7f7944157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f793bfff700 1 -- 172.21.15.40:0/3017300795 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3381791152 0 0) 0x7f7944070ae0 con 0x7f7944071260 2024-09-13T18:07:59.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f793bfff700 1 -- 172.21.15.40:0/3017300795 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79300035d0 con 0x7f7944071260 2024-09-13T18:07:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f793bfff700 1 -- 172.21.15.40:0/3017300795 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229082356 0 0) 0x7f7944070d30 con 0x7f7944151580 2024-09-13T18:07:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f793bfff700 1 -- 172.21.15.40:0/3017300795 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7944070ae0 con 0x7f7944151580 2024-09-13T18:07:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f79488cd700 1 -- 172.21.15.40:0/3017300795 >> v1:172.21.15.43:6789/0 conn(0x7f7944151580 legacy=0x7f79440c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f79488cd700 1 -- 172.21.15.40:0/3017300795 >> v1:172.21.15.40:6789/0 conn(0x7f7944071260 legacy=0x7f7944161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f79488cd700 1 -- 172.21.15.40:0/3017300795 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79440c2ca0 con 0x7f79440b1fb0 2024-09-13T18:07:59.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f79488cd700 1 --2- 172.21.15.40:0/3017300795 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79440b1fb0 0x7f7944157800 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7934009880 tx=0x7f7934009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba6d2e6bc2537bb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:59.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f793bfff700 1 -- 172.21.15.40:0/3017300795 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934011830 con 0x7f79440b1fb0 2024-09-13T18:07:59.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f793bfff700 1 -- 172.21.15.40:0/3017300795 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7934011990 con 0x7f79440b1fb0 2024-09-13T18:07:59.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.810+0000 7f793bfff700 1 -- 172.21.15.40:0/3017300795 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934011d00 con 0x7f79440b1fb0 2024-09-13T18:07:59.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.814+0000 7f794a0d0700 1 -- 172.21.15.40:0/3017300795 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79440b1fb0 msgr2=0x7f7944157800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.814+0000 7f794a0d0700 1 --2- 172.21.15.40:0/3017300795 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79440b1fb0 0x7f7944157800 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7934009880 tx=0x7f7934009b90 comp rx=0 tx=0).stop 2024-09-13T18:07:59.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.814+0000 7f794a0d0700 1 -- 172.21.15.40:0/3017300795 shutdown_connections 2024-09-13T18:07:59.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.814+0000 7f794a0d0700 1 --2- 172.21.15.40:0/3017300795 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79440b1fb0 0x7f7944157800 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.814+0000 7f794a0d0700 1 -- 172.21.15.40:0/3017300795 >> 172.21.15.40:0/3017300795 conn(0x7f794406e6d0 msgr2=0x7f794406eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:59.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.814+0000 7f794a0d0700 1 -- 172.21.15.40:0/3017300795 shutdown_connections 2024-09-13T18:07:59.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.814+0000 7f794a0d0700 1 -- 172.21.15.40:0/3017300795 wait complete. 2024-09-13T18:07:59.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.814+0000 7f794a0d0700 1 Processor -- start 2024-09-13T18:07:59.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.814+0000 7f794a0d0700 1 -- start start 2024-09-13T18:07:59.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f794a0d0700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7944071260 0x7f79441547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f794a0d0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79440b1fb0 0x7f79440ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f794a0d0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7944151580 0x7f79440ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f794a0d0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7944154d30 con 0x7f7944151580 2024-09-13T18:07:59.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f794a0d0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79441f0520 con 0x7f79440b1fb0 2024-09-13T18:07:59.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f794a0d0700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79440cb860 con 0x7f7944071260 2024-09-13T18:07:59.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f79498cf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7944151580 0x7f79440ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f79498cf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7944151580 0x7f79440ca950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58540/0 (socket says 172.21.15.40:58540) 2024-09-13T18:07:59.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f79498cf700 1 -- 172.21.15.40:0/2787521911 learned_addr learned my addr 172.21.15.40:0/2787521911 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:59.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f79490ce700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7944071260 0x7f79441547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f79488cd700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79440b1fb0 0x7f79440ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f79498cf700 1 -- 172.21.15.40:0/2787521911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7944071260 msgr2=0x7f79441547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f79498cf700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7944071260 0x7f79441547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f79498cf700 1 -- 172.21.15.40:0/2787521911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79440b1fb0 msgr2=0x7f79440ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f79498cf700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79440b1fb0 0x7f79440ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f79498cf700 1 -- 172.21.15.40:0/2787521911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79340094b0 con 0x7f7944151580 2024-09-13T18:07:59.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f79498cf700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7944151580 0x7f79440ca950 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f793c00cb40 tx=0x7f793c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:59.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f7939ffb700 1 -- 172.21.15.40:0/2787521911 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c007500 con 0x7f7944151580 2024-09-13T18:07:59.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f7939ffb700 1 -- 172.21.15.40:0/2787521911 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793c007660 con 0x7f7944151580 2024-09-13T18:07:59.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f794a0d0700 1 -- 172.21.15.40:0/2787521911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79441f0a10 con 0x7f7944151580 2024-09-13T18:07:59.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f7939ffb700 1 -- 172.21.15.40:0/2787521911 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c0079d0 con 0x7f7944151580 2024-09-13T18:07:59.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.818+0000 7f794a0d0700 1 -- 172.21.15.40:0/2787521911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79441f0730 con 0x7f7944151580 2024-09-13T18:07:59.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.822+0000 7f794a0d0700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79440c9a80 0x7f79440caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.822+0000 7f794a0d0700 1 -- 172.21.15.40:0/2787521911 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7944067600 con 0x7f79440c9a80 2024-09-13T18:07:59.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.822+0000 7f79490ce700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79440c9a80 0x7f79440caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.822+0000 7f79490ce700 1 -- 172.21.15.40:0/2787521911 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7944151960 con 0x7f79440c9a80 2024-09-13T18:07:59.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.822+0000 7f79490ce700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79440c9a80 0x7f79440caf40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f79400011e0 tx=0x7f79400018b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:59.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.822+0000 7f7939ffb700 1 -- 172.21.15.40:0/2787521911 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f793c01b3b0 con 0x7f7944151580 2024-09-13T18:07:59.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.822+0000 7f7939ffb700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7928066780 0x7f7928068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.822+0000 7f7939ffb700 1 -- 172.21.15.40:0/2787521911 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f793c083d00 con 0x7f7944151580 2024-09-13T18:07:59.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.822+0000 7f79488cd700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7928066780 0x7f7928068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.822+0000 7f7939ffb700 1 -- 172.21.15.40:0/2787521911 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7940017070 con 0x7f79440c9a80 2024-09-13T18:07:59.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.822+0000 7f7939ffb700 1 -- 172.21.15.40:0/2787521911 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7940012670 con 0x7f79440c9a80 2024-09-13T18:07:59.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.822+0000 7f7939ffb700 1 -- 172.21.15.40:0/2787521911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79440c9a80 msgr2=0x7f79440caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.822+0000 7f7939ffb700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79440c9a80 0x7f79440caf40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f79400011e0 tx=0x7f79400018b0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.822+0000 7f7939ffb700 1 -- 172.21.15.40:0/2787521911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79440c9a80 msgr2=0x7f79440caf40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:59.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.822+0000 7f7939ffb700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79440c9a80 0x7f79440caf40 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f79400011e0 tx=0x7f79400018b0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.822+0000 7f79488cd700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7928066780 0x7f7928068c40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f79340090b0 tx=0x7f7934010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:59.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.834+0000 7f794a0d0700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79441598d0 0x7f79440cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.834+0000 7f794a0d0700 1 -- 172.21.15.40:0/2787521911 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7944157290 con 0x7f79441598d0 2024-09-13T18:07:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.834+0000 7f79490ce700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79441598d0 0x7f79440cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.846 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.834+0000 7f79490ce700 1 -- 172.21.15.40:0/2787521911 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7944071840 con 0x7f79441598d0 2024-09-13T18:07:59.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.838+0000 7f79490ce700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79441598d0 0x7f79440cbcd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f79400011e0 tx=0x7f7940001150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:59.847 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.838+0000 7f7939ffb700 1 -- 172.21.15.40:0/2787521911 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7940023070 con 0x7f79441598d0 2024-09-13T18:07:59.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.838+0000 7f7939ffb700 1 -- 172.21.15.40:0/2787521911 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7f794001ee90 con 0x7f79441598d0 2024-09-13T18:07:59.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.838+0000 7f7939ffb700 1 -- 172.21.15.40:0/2787521911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79441598d0 msgr2=0x7f79440cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.838+0000 7f7939ffb700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79441598d0 0x7f79440cbcd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f79400011e0 tx=0x7f7940001150 comp rx=0 tx=0).stop 2024-09-13T18:07:59.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.838+0000 7f7939ffb700 1 -- 172.21.15.40:0/2787521911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79441598d0 msgr2=0x7f79440cbcd0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:07:59.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.838+0000 7f7939ffb700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f79441598d0 0x7f79440cbcd0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.849 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:07:59.849 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:07:59.849 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:07:59.849 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:07:59.850 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 146, 2024-09-13T18:07:59.850 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:07:59.850 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:07:59.850 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:07:59.850 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:07:59.850 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:59.850 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:07:59.850 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:59.850 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:07:59.850 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:07:59.851 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:59.851 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:59.851 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:59.851 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:59.851 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:59.851 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:59.851 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:59.851 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:59.851 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:59.851 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:59.851 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:59.852 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:07:59.852 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:07:59.852 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:59.852 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:59.852 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:59.852 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:59.852 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:59.852 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:59.852 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:59.852 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:59.852 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:59.852 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:59.853 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:59.853 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.853 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:07:59.853 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:07:59.853 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:07:59.853 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:07:59.853 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:07:59.853 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:07:59.853 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:59.853 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:07:59.853 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:07:59.853 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:07:59.853 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:07:59.854 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:07:59.854 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:07:59.854 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:07:59.854 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:07:59.854 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:07:59.854 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:07:59.854 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:07:59.854 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:07:59.854 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:07:59.855 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:07:59.855 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:07:59.855 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:07:59.855 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:07:59.855 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:07:59.855 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:07:59.855 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:07:59.855 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:59.855 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:07:59.855 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.855 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:07:59.855 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.856 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:07:59.856 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:07:59.856 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:59.856 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:59.856 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.856 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:59.856 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:07:59.856 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:59.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.856 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.856 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:59.856 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:07:59.856 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:59.857 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.857 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:59.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.857 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:07:59.857 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:07:59.857 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:59.857 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:59.857 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:59.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.857 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.857 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:07:59.857 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:07:59.857 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:59.858 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:59.858 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.858 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:59.858 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:07:59.858 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:59.858 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.858 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.858 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:59.858 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:07:59.858 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:59.858 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.858 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:59.859 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.859 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:07:59.859 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:07:59.859 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:59.859 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:59.860 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:59.860 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.860 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.860 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:07:59.860 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:07:59.860 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:07:59.860 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:07:59.860 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.860 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:07:59.860 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:07:59.861 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:59.861 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.861 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.861 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:07:59.861 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:07:59.861 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:07:59.861 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.861 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:59.861 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.861 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:07:59.861 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:07:59.861 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:07:59.862 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:07:59.862 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:07:59.862 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.862 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:59.862 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.862 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:07:59.862 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:07:59.862 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.862 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:59.862 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:59.862 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:07:59.862 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.862 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:59.862 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:07:59.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.863 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:59.863 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:59.863 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:07:59.863 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.863 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:59.863 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:07:59.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.863 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:59.863 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:59.863 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:07:59.863 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.863 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:59.864 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:07:59.864 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.864 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:59.864 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:59.864 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:07:59.864 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.864 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:07:59.864 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:07:59.864 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:07:59.864 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:07:59.864 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:07:59.865 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:07:59.865 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:07:59.865 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:07:59.865 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:07:59.865 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:07:59.865 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:07:59.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.838+0000 7f794a0d0700 1 -- 172.21.15.40:0/2787521911 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7928066780 msgr2=0x7f7928068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.838+0000 7f794a0d0700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7928066780 0x7f7928068c40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f79340090b0 tx=0x7f7934010000 comp rx=0 tx=0).stop 2024-09-13T18:07:59.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.838+0000 7f794a0d0700 1 -- 172.21.15.40:0/2787521911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7944151580 msgr2=0x7f79440ca950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.838+0000 7f79498cf700 1 -- 172.21.15.40:0/2787521911 reap_dead start 2024-09-13T18:07:59.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.838+0000 7f794a0d0700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7944151580 0x7f79440ca950 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f793c00cb40 tx=0x7f793c00cb70 comp rx=0 tx=0).stop 2024-09-13T18:07:59.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.838+0000 7f794a0d0700 1 -- 172.21.15.40:0/2787521911 shutdown_connections 2024-09-13T18:07:59.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.838+0000 7f794a0d0700 1 --2- 172.21.15.40:0/2787521911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7944151580 0x7f79440ca950 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.838+0000 7f794a0d0700 1 -- 172.21.15.40:0/2787521911 >> 172.21.15.40:0/2787521911 conn(0x7f794406e6d0 msgr2=0x7f794414fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:59.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.838+0000 7f794a0d0700 1 -- 172.21.15.40:0/2787521911 shutdown_connections 2024-09-13T18:07:59.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.838+0000 7f794a0d0700 1 -- 172.21.15.40:0/2787521911 wait complete. 2024-09-13T18:07:59.867 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:07:59.867 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:07:59.867 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:07:59.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.978+0000 7fdde5310700 1 Processor -- start 2024-09-13T18:07:59.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.978+0000 7fdde5310700 1 -- start start 2024-09-13T18:07:59.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdde5310700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fddd8045a40 0x7fddd8045e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdde5310700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fddd8007c70 0x7fddd80df190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdde5310700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fddd80df6d0 0x7fddd80db400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdde5310700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd8033a00 con 0x7fddd8045a40 2024-09-13T18:07:59.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdde5310700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd8046360 con 0x7fddd80df6d0 2024-09-13T18:07:59.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdde5310700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd80334f0 con 0x7fddd8007c70 2024-09-13T18:07:59.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdddffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fddd8045a40 0x7fddd8045e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdddffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fddd8045a40 0x7fddd8045e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58550/0 (socket says 172.21.15.40:58550) 2024-09-13T18:07:59.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdddffff700 1 -- 172.21.15.40:0/2301039676 learned_addr learned my addr 172.21.15.40:0/2301039676 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:59.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdde4b0f700 1 --2- 172.21.15.40:0/2301039676 >> v2:172.21.15.43:3300/0 conn(0x7fddd80df6d0 0x7fddd80db400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdddffff700 1 -- 172.21.15.40:0/2301039676 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fddd8007c70 msgr2=0x7fddd80df190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:59.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdddffff700 1 --2- 172.21.15.40:0/2301039676 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fddd8007c70 0x7fddd80df190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdddffff700 1 -- 172.21.15.40:0/2301039676 >> v2:172.21.15.43:3300/0 conn(0x7fddd80df6d0 msgr2=0x7fddd80db400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:07:59.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdddffff700 1 --2- 172.21.15.40:0/2301039676 >> v2:172.21.15.43:3300/0 conn(0x7fddd80df6d0 0x7fddd80db400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdddffff700 1 -- 172.21.15.40:0/2301039676 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd8033720 con 0x7fddd8045a40 2024-09-13T18:07:59.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdddffff700 1 --2- 172.21.15.40:0/2301039676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd8045a40 0x7fddd8045e20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009910 tx=0x7fddd4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820a9907036121f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:59.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdddeffd700 1 -- 172.21.15.40:0/2301039676 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4005e50 con 0x7fddd8045a40 2024-09-13T18:07:59.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdddeffd700 1 -- 172.21.15.40:0/2301039676 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd4006030 con 0x7fddd8045a40 2024-09-13T18:07:59.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdddeffd700 1 -- 172.21.15.40:0/2301039676 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd40063c0 con 0x7fddd8045a40 2024-09-13T18:07:59.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdde5310700 1 -- 172.21.15.40:0/2301039676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd8045a40 msgr2=0x7fddd8045e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdde5310700 1 --2- 172.21.15.40:0/2301039676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd8045a40 0x7fddd8045e20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009910 tx=0x7fddd4009bf0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdde5310700 1 -- 172.21.15.40:0/2301039676 shutdown_connections 2024-09-13T18:07:59.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdde5310700 1 --2- 172.21.15.40:0/2301039676 >> v2:172.21.15.43:3300/0 conn(0x7fddd80df6d0 0x7fddd80db400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdde5310700 1 --2- 172.21.15.40:0/2301039676 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fddd8007c70 0x7fddd80df190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdde5310700 1 --2- 172.21.15.40:0/2301039676 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd8045a40 0x7fddd8045e20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdde5310700 1 -- 172.21.15.40:0/2301039676 >> 172.21.15.40:0/2301039676 conn(0x7fddd80505e0 msgr2=0x7fddd80509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:07:59.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdde5310700 1 -- 172.21.15.40:0/2301039676 shutdown_connections 2024-09-13T18:07:59.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdde5310700 1 -- 172.21.15.40:0/2301039676 wait complete. 2024-09-13T18:07:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdde5310700 1 Processor -- start 2024-09-13T18:07:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.982+0000 7fdde5310700 1 -- start start 2024-09-13T18:07:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdde5310700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fddd8007c70 0x7fddd8173070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdde5310700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd8045a40 0x7fddd80d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdde5310700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd80df6d0 0x7fddd80d3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdde5310700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd80d37b0 con 0x7fddd8045a40 2024-09-13T18:07:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdde5310700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd80df270 con 0x7fddd80df6d0 2024-09-13T18:07:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdde5310700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd81735b0 con 0x7fddd8007c70 2024-09-13T18:07:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdddf7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd8045a40 0x7fddd80d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdddf7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd8045a40 0x7fddd80d08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58558/0 (socket says 172.21.15.40:58558) 2024-09-13T18:07:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdddf7fe700 1 -- 172.21.15.40:0/3126470603 learned_addr learned my addr 172.21.15.40:0/3126470603 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:07:59.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdddffff700 1 --2- 172.21.15.40:0/3126470603 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fddd8007c70 0x7fddd8173070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdde4b0f700 1 --2- 172.21.15.40:0/3126470603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd80df6d0 0x7fddd80d3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:07:59.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdddf7fe700 1 -- 172.21.15.40:0/3126470603 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fddd8007c70 msgr2=0x7fddd8173070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdddf7fe700 1 --2- 172.21.15.40:0/3126470603 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fddd8007c70 0x7fddd8173070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdddf7fe700 1 -- 172.21.15.40:0/3126470603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd80df6d0 msgr2=0x7fddd80d3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:07:59.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdddf7fe700 1 --2- 172.21.15.40:0/3126470603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd80df6d0 0x7fddd80d3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:07:59.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdddf7fe700 1 -- 172.21.15.40:0/3126470603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd4009520 con 0x7fddd8045a40 2024-09-13T18:07:59.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdde4b0f700 1 --2- 172.21.15.40:0/3126470603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd80df6d0 0x7fddd80d3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:07:59.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdddf7fe700 1 --2- 172.21.15.40:0/3126470603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd8045a40 0x7fddd80d08d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00b550 tx=0x7fddcc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:07:59.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdddcff9700 1 -- 172.21.15.40:0/3126470603 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc011830 con 0x7fddd8045a40 2024-09-13T18:07:59.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdddcff9700 1 -- 172.21.15.40:0/3126470603 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc011990 con 0x7fddd8045a40 2024-09-13T18:07:59.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdde5310700 1 -- 172.21.15.40:0/3126470603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd80d0480 con 0x7fddd8045a40 2024-09-13T18:07:59.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdddcff9700 1 -- 172.21.15.40:0/3126470603 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc011d40 con 0x7fddd8045a40 2024-09-13T18:07:59.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdde5310700 1 -- 172.21.15.40:0/3126470603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd80d01a0 con 0x7fddd8045a40 2024-09-13T18:07:59.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdddcff9700 1 -- 172.21.15.40:0/3126470603 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fddcc01a400 con 0x7fddd8045a40 2024-09-13T18:07:59.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.986+0000 7fdde5310700 1 -- 172.21.15.40:0/3126470603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd8036140 con 0x7fddd8045a40 2024-09-13T18:07:59.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.990+0000 7fdddcff9700 1 --2- 172.21.15.40:0/3126470603 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fddc4066730 0x7fddc4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:07:59.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.990+0000 7fdddcff9700 1 -- 172.21.15.40:0/3126470603 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fddcc084610 con 0x7fddd8045a40 2024-09-13T18:07:59.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.990+0000 7fdddffff700 1 --2- 172.21.15.40:0/3126470603 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fddc4066730 0x7fddc4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:00.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.990+0000 7fdddffff700 1 --2- 172.21.15.40:0/3126470603 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fddc4066730 0x7fddc4068bf0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009c30 tx=0x7fddd4008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:00.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:07:59.998+0000 7fdddcff9700 1 -- 172.21.15.40:0/3126470603 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fddcc0515c0 con 0x7fddd8045a40 2024-09-13T18:08:00.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:00.246+0000 7fdde5310700 1 -- 172.21.15.40:0/3126470603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fddd80d8e10 con 0x7fddd8045a40 2024-09-13T18:08:00.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:00.246+0000 7fdddcff9700 1 -- 172.21.15.40:0/3126470603 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fddcc054be0 con 0x7fddd8045a40 2024-09-13T18:08:00.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:00.250+0000 7fdde5310700 1 -- 172.21.15.40:0/3126470603 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fddc4066730 msgr2=0x7fddc4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:00.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:00.250+0000 7fdde5310700 1 --2- 172.21.15.40:0/3126470603 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fddc4066730 0x7fddc4068bf0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009c30 tx=0x7fddd4008070 comp rx=0 tx=0).stop 2024-09-13T18:08:00.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:00.250+0000 7fdde5310700 1 -- 172.21.15.40:0/3126470603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd8045a40 msgr2=0x7fddd80d08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:00.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:00.250+0000 7fdde5310700 1 --2- 172.21.15.40:0/3126470603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd8045a40 0x7fddd80d08d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fddcc00b550 tx=0x7fddcc00b910 comp rx=0 tx=0).stop 2024-09-13T18:08:00.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:00.250+0000 7fdde5310700 1 -- 172.21.15.40:0/3126470603 shutdown_connections 2024-09-13T18:08:00.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:00.250+0000 7fdde5310700 1 --2- 172.21.15.40:0/3126470603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd80df6d0 0x7fddd80d3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:00.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:00.250+0000 7fdde5310700 1 --2- 172.21.15.40:0/3126470603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fddd8045a40 0x7fddd80d08d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:00.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:00.250+0000 7fdde5310700 1 --2- 172.21.15.40:0/3126470603 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fddc4066730 0x7fddc4068bf0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:00.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:00.250+0000 7fdde5310700 1 --2- 172.21.15.40:0/3126470603 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fddd8007c70 0x7fddd8173070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:00.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:00.250+0000 7fdde5310700 1 -- 172.21.15.40:0/3126470603 >> 172.21.15.40:0/3126470603 conn(0x7fddd80505e0 msgr2=0x7fddd80d2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:08:00.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:00.250+0000 7fdde5310700 1 -- 172.21.15.40:0/3126470603 shutdown_connections 2024-09-13T18:08:00.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:00.250+0000 7fdde5310700 1 -- 172.21.15.40:0/3126470603 wait complete. 2024-09-13T18:08:00.274 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:08:06.912 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:08:06.915 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:08:10.276 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:08:10.276 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:08:10.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.382+0000 7fc826ca1700 1 Processor -- start 2024-09-13T18:08:10.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.382+0000 7fc826ca1700 1 -- start start 2024-09-13T18:08:10.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.386+0000 7fc826ca1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8180077d0 0x7fc8180e6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.386+0000 7fc826ca1700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc818006490 con 0x7fc8180461f0 2024-09-13T18:08:10.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.386+0000 7fc826ca1700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc818036020 con 0x7fc8180d58d0 2024-09-13T18:08:10.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.386+0000 7fc826ca1700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc818046e50 con 0x7fc8180077d0 2024-09-13T18:08:10.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.386+0000 7fc82549e700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc8180461f0 0x7fc818033140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52902/0 (socket says 172.21.15.40:52902) 2024-09-13T18:08:10.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.386+0000 7fc82549e700 1 -- 172.21.15.40:0/1805552049 learned_addr learned my addr 172.21.15.40:0/1805552049 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:08:10.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.386+0000 7fc8264a0700 1 --2- 172.21.15.40:0/1805552049 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8180077d0 0x7fc8180e6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.386+0000 7fc824c9d700 1 -- 172.21.15.40:0/1805552049 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1433275578 0 0) 0x7fc818006490 con 0x7fc8180461f0 2024-09-13T18:08:10.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.386+0000 7fc824c9d700 1 -- 172.21.15.40:0/1805552049 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc80c0035d0 con 0x7fc8180461f0 2024-09-13T18:08:10.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.386+0000 7fc824c9d700 1 -- 172.21.15.40:0/1805552049 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3169929931 0 0) 0x7fc818036020 con 0x7fc8180d58d0 2024-09-13T18:08:10.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.386+0000 7fc8264a0700 1 -- 172.21.15.40:0/1805552049 >> v1:172.21.15.43:6789/0 conn(0x7fc8180d58d0 legacy=0x7fc818046740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.386+0000 7fc8264a0700 1 -- 172.21.15.40:0/1805552049 >> v1:172.21.15.40:6789/0 conn(0x7fc8180461f0 legacy=0x7fc818033140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.386+0000 7fc8264a0700 1 -- 172.21.15.40:0/1805552049 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818007c60 con 0x7fc8180077d0 2024-09-13T18:08:10.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.386+0000 7fc8264a0700 1 --2- 172.21.15.40:0/1805552049 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8180077d0 0x7fc8180e6e40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc820069620 tx=0x7fc820069e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24893af648d795e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:10.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.386+0000 7fc824c9d700 1 -- 172.21.15.40:0/1805552049 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc820067dd0 con 0x7fc8180077d0 2024-09-13T18:08:10.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.386+0000 7fc824c9d700 1 -- 172.21.15.40:0/1805552049 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc820067f30 con 0x7fc8180077d0 2024-09-13T18:08:10.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.386+0000 7fc824c9d700 1 -- 172.21.15.40:0/1805552049 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8200746e0 con 0x7fc8180077d0 2024-09-13T18:08:10.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.390+0000 7fc826ca1700 1 -- 172.21.15.40:0/1805552049 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8180077d0 msgr2=0x7fc8180e6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.390+0000 7fc826ca1700 1 --2- 172.21.15.40:0/1805552049 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8180077d0 0x7fc8180e6e40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc820069620 tx=0x7fc820069e70 comp rx=0 tx=0).stop 2024-09-13T18:08:10.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.390+0000 7fc826ca1700 1 -- 172.21.15.40:0/1805552049 shutdown_connections 2024-09-13T18:08:10.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.390+0000 7fc826ca1700 1 --2- 172.21.15.40:0/1805552049 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8180077d0 0x7fc8180e6e40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.390+0000 7fc826ca1700 1 -- 172.21.15.40:0/1805552049 >> 172.21.15.40:0/1805552049 conn(0x7fc818050590 msgr2=0x7fc8180509a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:08:10.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.390+0000 7fc826ca1700 1 -- 172.21.15.40:0/1805552049 shutdown_connections 2024-09-13T18:08:10.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.390+0000 7fc826ca1700 1 -- 172.21.15.40:0/1805552049 wait complete. 2024-09-13T18:08:10.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.390+0000 7fc826ca1700 1 Processor -- start 2024-09-13T18:08:10.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.390+0000 7fc826ca1700 1 -- start start 2024-09-13T18:08:10.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.390+0000 7fc826ca1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8180077d0 0x7fc81804a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.390+0000 7fc826ca1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8180461f0 0x7fc8180de240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.390+0000 7fc826ca1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8180d58d0 0x7fc8180de780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.390+0000 7fc826ca1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180dc6f0 con 0x7fc8180d58d0 2024-09-13T18:08:10.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.390+0000 7fc826ca1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81804a850 con 0x7fc8180461f0 2024-09-13T18:08:10.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.390+0000 7fc826ca1700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc818172e20 con 0x7fc8180077d0 2024-09-13T18:08:10.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc8264a0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8180d58d0 0x7fc8180de780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.405 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc8264a0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8180d58d0 0x7fc8180de780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49396/0 (socket says 172.21.15.40:49396) 2024-09-13T18:08:10.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc8264a0700 1 -- 172.21.15.40:0/180554243 learned_addr learned my addr 172.21.15.40:0/180554243 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:08:10.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc825c9f700 1 --2- 172.21.15.40:0/180554243 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8180077d0 0x7fc81804a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc82549e700 1 --2- 172.21.15.40:0/180554243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8180461f0 0x7fc8180de240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc8264a0700 1 -- 172.21.15.40:0/180554243 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8180077d0 msgr2=0x7fc81804a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc8264a0700 1 --2- 172.21.15.40:0/180554243 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8180077d0 0x7fc81804a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc8264a0700 1 -- 172.21.15.40:0/180554243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8180461f0 msgr2=0x7fc8180de240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc8264a0700 1 --2- 172.21.15.40:0/180554243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8180461f0 0x7fc8180de240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc8264a0700 1 -- 172.21.15.40:0/180554243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82004f5c0 con 0x7fc8180d58d0 2024-09-13T18:08:10.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc8264a0700 1 --2- 172.21.15.40:0/180554243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8180d58d0 0x7fc8180de780 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc820069620 tx=0x7fc820068470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:10.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc8167fc700 1 -- 172.21.15.40:0/180554243 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc820067930 con 0x7fc8180d58d0 2024-09-13T18:08:10.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc8167fc700 1 -- 172.21.15.40:0/180554243 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc820072150 con 0x7fc8180d58d0 2024-09-13T18:08:10.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc826ca1700 1 -- 172.21.15.40:0/180554243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8181730e0 con 0x7fc8180d58d0 2024-09-13T18:08:10.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc8167fc700 1 -- 172.21.15.40:0/180554243 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc820072520 con 0x7fc8180d58d0 2024-09-13T18:08:10.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc826ca1700 1 -- 172.21.15.40:0/180554243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8180d8920 con 0x7fc8180d58d0 2024-09-13T18:08:10.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc826ca1700 1 -- 172.21.15.40:0/180554243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8180dc860 con 0x7fc8180d58d0 2024-09-13T18:08:10.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc8167fc700 1 -- 172.21.15.40:0/180554243 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc820091020 con 0x7fc8180d58d0 2024-09-13T18:08:10.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc8167fc700 1 --2- 172.21.15.40:0/180554243 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc804066730 0x7fc804068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc8167fc700 1 -- 172.21.15.40:0/180554243 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc8200ffca0 con 0x7fc8180d58d0 2024-09-13T18:08:10.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.394+0000 7fc825c9f700 1 --2- 172.21.15.40:0/180554243 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc804066730 0x7fc804068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.398+0000 7fc825c9f700 1 --2- 172.21.15.40:0/180554243 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc804066730 0x7fc804068bf0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc81c007fd0 tx=0x7fc81c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:10.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.402+0000 7fc8167fc700 1 -- 172.21.15.40:0/180554243 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc8200ccc60 con 0x7fc8180d58d0 2024-09-13T18:08:10.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.674+0000 7fc826ca1700 1 -- 172.21.15.40:0/180554243 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc818005ed0 con 0x7fc8180d58d0 2024-09-13T18:08:10.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.674+0000 7fc8167fc700 1 -- 172.21.15.40:0/180554243 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7fc8200a2620 con 0x7fc8180d58d0 2024-09-13T18:08:10.685 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":146,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:08:10.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.678+0000 7fc826ca1700 1 -- 172.21.15.40:0/180554243 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc804066730 msgr2=0x7fc804068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.678+0000 7fc826ca1700 1 --2- 172.21.15.40:0/180554243 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc804066730 0x7fc804068bf0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc81c007fd0 tx=0x7fc81c009040 comp rx=0 tx=0).stop 2024-09-13T18:08:10.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.678+0000 7fc826ca1700 1 -- 172.21.15.40:0/180554243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8180d58d0 msgr2=0x7fc8180de780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.678+0000 7fc826ca1700 1 --2- 172.21.15.40:0/180554243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8180d58d0 0x7fc8180de780 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc820069620 tx=0x7fc820068470 comp rx=0 tx=0).stop 2024-09-13T18:08:10.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.678+0000 7fc826ca1700 1 -- 172.21.15.40:0/180554243 shutdown_connections 2024-09-13T18:08:10.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.678+0000 7fc826ca1700 1 --2- 172.21.15.40:0/180554243 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8180d58d0 0x7fc8180de780 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.678+0000 7fc826ca1700 1 --2- 172.21.15.40:0/180554243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8180461f0 0x7fc8180de240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.678+0000 7fc826ca1700 1 --2- 172.21.15.40:0/180554243 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc804066730 0x7fc804068bf0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.678+0000 7fc826ca1700 1 --2- 172.21.15.40:0/180554243 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8180077d0 0x7fc81804a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.678+0000 7fc826ca1700 1 -- 172.21.15.40:0/180554243 >> 172.21.15.40:0/180554243 conn(0x7fc818050590 msgr2=0x7fc8180067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:08:10.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.678+0000 7fc826ca1700 1 -- 172.21.15.40:0/180554243 shutdown_connections 2024-09-13T18:08:10.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.678+0000 7fc826ca1700 1 -- 172.21.15.40:0/180554243 wait complete. 2024-09-13T18:08:10.700 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:08:10.700 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T18:08:10.700 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:08:10.701 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:08:10.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.798+0000 7fa4162f4700 1 Processor -- start 2024-09-13T18:08:10.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.798+0000 7fa4162f4700 1 -- start start 2024-09-13T18:08:10.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa4162f4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4100cc290 0x7fa410159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa4162f4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa41015cea0 0x7fa41015a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa4162f4700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa41015d370 con 0x7fa4100cc670 2024-09-13T18:08:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa4162f4700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4101598b0 con 0x7fa41015cea0 2024-09-13T18:08:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa4162f4700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa41015d650 con 0x7fa4100cc290 2024-09-13T18:08:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa40f7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa41015cea0 0x7fa41015a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa414891700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa4100cc670 0x7fa4101500d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52918/0 (socket says 172.21.15.40:52918) 2024-09-13T18:08:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa40f7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa41015cea0 0x7fa41015a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49402/0 (socket says 172.21.15.40:49402) 2024-09-13T18:08:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa414891700 1 -- 172.21.15.40:0/3223261959 learned_addr learned my addr 172.21.15.40:0/3223261959 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:08:10.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa40ffff700 1 --2- 172.21.15.40:0/3223261959 >> v2:172.21.15.43:3300/0 conn(0x7fa4100cc290 0x7fa410159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa40f7fe700 1 -- 172.21.15.40:0/3223261959 >> v2:172.21.15.43:3300/0 conn(0x7fa4100cc290 msgr2=0x7fa410159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:08:10.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa40f7fe700 1 --2- 172.21.15.40:0/3223261959 >> v2:172.21.15.43:3300/0 conn(0x7fa4100cc290 0x7fa410159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa40f7fe700 1 -- 172.21.15.40:0/3223261959 >> v1:172.21.15.40:6789/0 conn(0x7fa4100cc670 legacy=0x7fa4101500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa40f7fe700 1 -- 172.21.15.40:0/3223261959 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41015d930 con 0x7fa41015cea0 2024-09-13T18:08:10.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa40ffff700 1 --2- 172.21.15.40:0/3223261959 >> v2:172.21.15.43:3300/0 conn(0x7fa4100cc290 0x7fa410159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:08:10.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa40f7fe700 1 --2- 172.21.15.40:0/3223261959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41015cea0 0x7fa41015a520 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa4100ccbd0 tx=0x7fa404009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd42c1ad1e6b381a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:10.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa40effd700 1 -- 172.21.15.40:0/3223261959 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404011830 con 0x7fa41015cea0 2024-09-13T18:08:10.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa40effd700 1 -- 172.21.15.40:0/3223261959 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa404011990 con 0x7fa41015cea0 2024-09-13T18:08:10.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa40effd700 1 -- 172.21.15.40:0/3223261959 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404011d40 con 0x7fa41015cea0 2024-09-13T18:08:10.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa4162f4700 1 -- 172.21.15.40:0/3223261959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41015cea0 msgr2=0x7fa41015a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa4162f4700 1 --2- 172.21.15.40:0/3223261959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41015cea0 0x7fa41015a520 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa4100ccbd0 tx=0x7fa404009b30 comp rx=0 tx=0).stop 2024-09-13T18:08:10.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa4162f4700 1 -- 172.21.15.40:0/3223261959 shutdown_connections 2024-09-13T18:08:10.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa4162f4700 1 --2- 172.21.15.40:0/3223261959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41015cea0 0x7fa41015a520 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa4162f4700 1 --2- 172.21.15.40:0/3223261959 >> v2:172.21.15.43:3300/0 conn(0x7fa4100cc290 0x7fa410159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa4162f4700 1 -- 172.21.15.40:0/3223261959 >> 172.21.15.40:0/3223261959 conn(0x7fa410066480 msgr2=0x7fa410066890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:08:10.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa4162f4700 1 -- 172.21.15.40:0/3223261959 shutdown_connections 2024-09-13T18:08:10.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.802+0000 7fa4162f4700 1 -- 172.21.15.40:0/3223261959 wait complete. 2024-09-13T18:08:10.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.806+0000 7fa4162f4700 1 Processor -- start 2024-09-13T18:08:10.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.806+0000 7fa4162f4700 1 -- start start 2024-09-13T18:08:10.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.806+0000 7fa4162f4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4100cc290 0x7fa4100ac550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.806+0000 7fa4162f4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4100cc670 0x7fa4100c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.806+0000 7fa4162f4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41015cea0 0x7fa4101ef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.806+0000 7fa4162f4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4100c9400 con 0x7fa41015cea0 2024-09-13T18:08:10.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.806+0000 7fa4162f4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41015c610 con 0x7fa4100cc670 2024-09-13T18:08:10.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.806+0000 7fa4162f4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa41015c8f0 con 0x7fa4100cc290 2024-09-13T18:08:10.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.806+0000 7fa414891700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41015cea0 0x7fa4101ef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.806+0000 7fa414891700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41015cea0 0x7fa4101ef1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49404/0 (socket says 172.21.15.40:49404) 2024-09-13T18:08:10.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.806+0000 7fa414891700 1 -- 172.21.15.40:0/1583834564 learned_addr learned my addr 172.21.15.40:0/1583834564 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:08:10.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.806+0000 7fa40f7fe700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4100cc670 0x7fa4100c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.806+0000 7fa40ffff700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4100cc290 0x7fa4100ac550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.806+0000 7fa414891700 1 -- 172.21.15.40:0/1583834564 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4100cc290 msgr2=0x7fa4100ac550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.806+0000 7fa414891700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa4100cc290 0x7fa4100ac550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.806+0000 7fa414891700 1 -- 172.21.15.40:0/1583834564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4100cc670 msgr2=0x7fa4100c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.806+0000 7fa414891700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4100cc670 0x7fa4100c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.806+0000 7fa414891700 1 -- 172.21.15.40:0/1583834564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4040094b0 con 0x7fa41015cea0 2024-09-13T18:08:10.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.806+0000 7fa40f7fe700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4100cc670 0x7fa4100c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:08:10.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.810+0000 7fa414891700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41015cea0 0x7fa4101ef1f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa40800caf0 tx=0x7fa40800cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:10.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.810+0000 7fa40cff9700 1 -- 172.21.15.40:0/1583834564 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa408007480 con 0x7fa41015cea0 2024-09-13T18:08:10.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.810+0000 7fa40cff9700 1 -- 172.21.15.40:0/1583834564 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4080075e0 con 0x7fa41015cea0 2024-09-13T18:08:10.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.810+0000 7fa4162f4700 1 -- 172.21.15.40:0/1583834564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa41015cbd0 con 0x7fa41015cea0 2024-09-13T18:08:10.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.810+0000 7fa4162f4700 1 -- 172.21.15.40:0/1583834564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4100c9090 con 0x7fa41015cea0 2024-09-13T18:08:10.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.810+0000 7fa40cff9700 1 -- 172.21.15.40:0/1583834564 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa408007950 con 0x7fa41015cea0 2024-09-13T18:08:10.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.810+0000 7fa4162f4700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41014c560 0x7fa4101ef730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.810+0000 7fa4162f4700 1 -- 172.21.15.40:0/1583834564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa410067520 con 0x7fa41014c560 2024-09-13T18:08:10.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.810+0000 7fa40ffff700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41014c560 0x7fa4101ef730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.810+0000 7fa40ffff700 1 -- 172.21.15.40:0/1583834564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa4100cb7b0 con 0x7fa41014c560 2024-09-13T18:08:10.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.810+0000 7fa40ffff700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41014c560 0x7fa4101ef730 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa400011830 tx=0x7fa400011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:10.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.810+0000 7fa40cff9700 1 -- 172.21.15.40:0/1583834564 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4000045a0 con 0x7fa41014c560 2024-09-13T18:08:10.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.810+0000 7fa40cff9700 1 -- 172.21.15.40:0/1583834564 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa40801b3b0 con 0x7fa41015cea0 2024-09-13T18:08:10.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.810+0000 7fa40cff9700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa3fc0667d0 0x7fa3fc068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.814+0000 7fa40cff9700 1 -- 172.21.15.40:0/1583834564 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa408083e20 con 0x7fa41015cea0 2024-09-13T18:08:10.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.814+0000 7fa40f7fe700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa3fc0667d0 0x7fa3fc068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.814+0000 7fa40cff9700 1 -- 172.21.15.40:0/1583834564 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa4000036c0 con 0x7fa41014c560 2024-09-13T18:08:10.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.814+0000 7fa40cff9700 1 -- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41014c560 msgr2=0x7fa4101ef730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.814+0000 7fa40cff9700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41014c560 0x7fa4101ef730 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa400011830 tx=0x7fa400011bf0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.814+0000 7fa40cff9700 1 -- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41014c560 msgr2=0x7fa4101ef730 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:08:10.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.814+0000 7fa40cff9700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41014c560 0x7fa4101ef730 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa400011830 tx=0x7fa400011bf0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.814+0000 7fa40f7fe700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa3fc0667d0 0x7fa3fc068c90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa4040090b0 tx=0x7fa40401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:10.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.822+0000 7fa4162f4700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4100ab880 0x7fa4101f2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.822+0000 7fa4162f4700 1 -- 172.21.15.40:0/1583834564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41015cd60 con 0x7fa4100ab880 2024-09-13T18:08:10.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.822+0000 7fa40ffff700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4100ab880 0x7fa4101f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.826+0000 7fa40ffff700 1 -- 172.21.15.40:0/1583834564 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa410159ac0 con 0x7fa4100ab880 2024-09-13T18:08:10.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.826+0000 7fa40ffff700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4100ab880 0x7fa4101f2540 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa400011830 tx=0x7fa400004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.826+0000 7fa40cff9700 1 -- 172.21.15.40:0/1583834564 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa400003e40 con 0x7fa4100ab880 2024-09-13T18:08:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.826+0000 7fa40cff9700 1 -- 172.21.15.40:0/1583834564 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4785 (secure 0 0 0) 0x7fa4000159d0 con 0x7fa4100ab880 2024-09-13T18:08:10.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.826+0000 7fa40cff9700 1 -- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4100ab880 msgr2=0x7fa4101f2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.826+0000 7fa40cff9700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4100ab880 0x7fa4101f2540 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa400011830 tx=0x7fa400004770 comp rx=0 tx=0).stop 2024-09-13T18:08:10.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.826+0000 7fa40cff9700 1 -- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4100ab880 msgr2=0x7fa4101f2540 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:08:10.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.826+0000 7fa40cff9700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4100ab880 0x7fa4101f2540 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.837 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:08:10.837 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:08:10.837 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:08:10.837 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:08:10.837 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 146, 2024-09-13T18:08:10.837 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:08:10.837 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:08:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:08:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:08:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 13, 2024-09-13T18:08:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:08:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:08:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:08:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:08:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:08:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:08:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:08:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:08:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:08:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:08:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:08:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:08:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:08:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:08:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:08:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:08:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:08:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:08:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:08:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:08:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:08:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:08:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:08:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:08:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:08:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:08:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:08:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:08:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:08:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:08:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:08:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:08:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:08:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:08:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:08:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:08:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:08:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:08:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:08:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:08:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:08:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:08:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:08:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:08:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:08:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:08:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:08:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:08:10.842 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:08:10.842 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:08:10.842 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:08:10.842 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:08:10.842 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:10.842 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:08:10.842 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:10.842 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:08:10.842 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:10.842 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:08:10.842 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:08:10.842 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:08:10.842 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:08:10.842 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:10.843 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:08:10.843 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:08:10.843 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:10.843 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:10.843 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:10.843 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:08:10.843 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:08:10.843 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:10.843 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:10.843 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:10.843 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:10.843 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:08:10.843 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:08:10.843 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:08:10.844 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:08:10.844 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:08:10.844 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:10.844 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:10.844 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:08:10.844 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:08:10.844 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:08:10.844 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:08:10.844 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:10.844 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:08:10.844 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:08:10.844 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:10.844 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:10.844 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:10.845 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:08:10.845 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:08:10.845 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:10.845 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:10.845 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:10.845 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:10.845 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:08:10.845 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:08:10.845 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:08:10.845 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:08:10.845 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:08:10.845 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:10.845 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:10.845 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:08:10.846 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:08:10.846 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:08:10.846 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:08:10.846 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:10.846 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:08:10.846 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:08:10.846 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:10.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:10.846 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:10.846 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:08:10.846 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:08:10.846 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:10.846 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:10.846 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:08:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:08:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:08:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:08:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:08:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:08:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:08:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:10.847 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:08:10.848 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:10.848 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:10.848 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:08:10.848 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:10.848 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:10.848 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:10.848 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:08:10.848 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:10.848 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:10.848 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:08:10.848 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:10.848 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:10.848 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:10.848 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:08:10.849 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:10.849 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:10.849 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:08:10.849 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:10.849 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:10.849 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:10.849 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:08:10.849 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:10.849 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:10.849 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:10.849 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:08:10.849 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:08:10.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.826+0000 7fa4162f4700 1 -- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa3fc0667d0 msgr2=0x7fa3fc068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.826+0000 7fa4162f4700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa3fc0667d0 0x7fa3fc068c90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa4040090b0 tx=0x7fa40401c040 comp rx=0 tx=0).stop 2024-09-13T18:08:10.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.826+0000 7fa4162f4700 1 -- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41015cea0 msgr2=0x7fa4101ef1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.826+0000 7fa414891700 1 -- 172.21.15.40:0/1583834564 reap_dead start 2024-09-13T18:08:10.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.826+0000 7fa4162f4700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41015cea0 0x7fa4101ef1f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa40800caf0 tx=0x7fa40800cb20 comp rx=0 tx=0).stop 2024-09-13T18:08:10.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.826+0000 7fa4162f4700 1 -- 172.21.15.40:0/1583834564 shutdown_connections 2024-09-13T18:08:10.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.826+0000 7fa4162f4700 1 --2- 172.21.15.40:0/1583834564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41015cea0 0x7fa4101ef1f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.826+0000 7fa4162f4700 1 -- 172.21.15.40:0/1583834564 >> 172.21.15.40:0/1583834564 conn(0x7fa410066480 msgr2=0x7fa41014f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:08:10.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.826+0000 7fa4162f4700 1 -- 172.21.15.40:0/1583834564 shutdown_connections 2024-09-13T18:08:10.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.826+0000 7fa4162f4700 1 -- 172.21.15.40:0/1583834564 wait complete. 2024-09-13T18:08:10.852 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:08:10.852 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:08:10.852 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:08:10.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.962+0000 7faa25584700 1 Processor -- start 2024-09-13T18:08:10.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.962+0000 7faa25584700 1 -- start start 2024-09-13T18:08:10.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.962+0000 7faa25584700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa18050820 0x7faa18035240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.962+0000 7faa25584700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa180d5a20 con 0x7faa18045e10 2024-09-13T18:08:10.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.962+0000 7faa25584700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa180dbe40 con 0x7faa180d5640 2024-09-13T18:08:10.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.962+0000 7faa25584700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa180dc120 con 0x7faa18050820 2024-09-13T18:08:10.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.962+0000 7faa1f7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7faa18045e10 0x7faa18034b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52932/0 (socket says 172.21.15.40:52932) 2024-09-13T18:08:10.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.962+0000 7faa1f7fe700 1 -- 172.21.15.40:0/3548854360 learned_addr learned my addr 172.21.15.40:0/3548854360 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:08:10.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.962+0000 7faa24d83700 1 --2- 172.21.15.40:0/3548854360 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa18050820 0x7faa18035240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.962+0000 7faa1effd700 1 -- 172.21.15.40:0/3548854360 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365062077 0 0) 0x7faa180d5a20 con 0x7faa18045e10 2024-09-13T18:08:10.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.962+0000 7faa1effd700 1 -- 172.21.15.40:0/3548854360 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa100057e0 con 0x7faa18045e10 2024-09-13T18:08:10.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1effd700 1 -- 172.21.15.40:0/3548854360 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 805911416 0 0) 0x7faa100057e0 con 0x7faa18045e10 2024-09-13T18:08:10.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1effd700 1 -- 172.21.15.40:0/3548854360 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa18050820 msgr2=0x7faa18035240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:08:10.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1effd700 1 --2- 172.21.15.40:0/3548854360 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa18050820 0x7faa18035240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1effd700 1 -- 172.21.15.40:0/3548854360 >> v1:172.21.15.43:6789/0 conn(0x7faa180d5640 legacy=0x7faa18045700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1effd700 1 -- 172.21.15.40:0/3548854360 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa18005b00 con 0x7faa18045e10 2024-09-13T18:08:10.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1effd700 1 -- 172.21.15.40:0/3548854360 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7faa0c001670 con 0x7faa18045e10 2024-09-13T18:08:10.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1effd700 1 -- 172.21.15.40:0/3548854360 >> v1:172.21.15.40:6789/0 conn(0x7faa18045e10 legacy=0x7faa18034b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1effd700 1 --2- 172.21.15.40:0/3548854360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa10005c60 0x7faa10006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1effd700 1 --2- 172.21.15.40:0/3548854360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa10006a80 0x7faa10006f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1effd700 1 --2- 172.21.15.40:0/3548854360 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa18050820 0x7faa10007440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1effd700 1 -- 172.21.15.40:0/3548854360 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa180dc120 con 0x7faa10006a80 2024-09-13T18:08:10.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1effd700 1 -- 172.21.15.40:0/3548854360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1000dd20 con 0x7faa10005c60 2024-09-13T18:08:10.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1effd700 1 -- 172.21.15.40:0/3548854360 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1000dee0 con 0x7faa18050820 2024-09-13T18:08:10.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1f7fe700 1 --2- 172.21.15.40:0/3548854360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa10006a80 0x7faa10006f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1ffff700 1 --2- 172.21.15.40:0/3548854360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa10005c60 0x7faa10006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa24d83700 1 --2- 172.21.15.40:0/3548854360 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa18050820 0x7faa10007440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1f7fe700 1 -- 172.21.15.40:0/3548854360 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa18050820 msgr2=0x7faa10007440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:08:10.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1f7fe700 1 --2- 172.21.15.40:0/3548854360 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa18050820 0x7faa10007440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1f7fe700 1 -- 172.21.15.40:0/3548854360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa10005c60 msgr2=0x7faa10006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:08:10.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1f7fe700 1 --2- 172.21.15.40:0/3548854360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa10005c60 0x7faa10006040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1f7fe700 1 -- 172.21.15.40:0/3548854360 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa100057e0 con 0x7faa10006a80 2024-09-13T18:08:10.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1f7fe700 1 --2- 172.21.15.40:0/3548854360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa10006a80 0x7faa10006f00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00c210 tx=0x7faa10007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2746c41091c55b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:10.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1effd700 1 -- 172.21.15.40:0/3548854360 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c00cdb0 con 0x7faa10006a80 2024-09-13T18:08:10.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1effd700 1 -- 172.21.15.40:0/3548854360 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c005980 con 0x7faa10006a80 2024-09-13T18:08:10.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa1effd700 1 -- 172.21.15.40:0/3548854360 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c005d30 con 0x7faa10006a80 2024-09-13T18:08:10.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa25584700 1 -- 172.21.15.40:0/3548854360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa10006a80 msgr2=0x7faa10006f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa25584700 1 --2- 172.21.15.40:0/3548854360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa10006a80 0x7faa10006f00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00c210 tx=0x7faa10007ab0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa24d83700 1 -- 172.21.15.40:0/3548854360 reap_dead start 2024-09-13T18:08:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa25584700 1 -- 172.21.15.40:0/3548854360 shutdown_connections 2024-09-13T18:08:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa25584700 1 -- 172.21.15.40:0/3548854360 >> 172.21.15.40:0/3548854360 conn(0x7faa1804fd00 msgr2=0x7faa18050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:08:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa25584700 1 -- 172.21.15.40:0/3548854360 shutdown_connections 2024-09-13T18:08:10.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.966+0000 7faa25584700 1 -- 172.21.15.40:0/3548854360 wait complete. 2024-09-13T18:08:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa25584700 1 Processor -- start 2024-09-13T18:08:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa25584700 1 -- start start 2024-09-13T18:08:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa25584700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa10006a80 0x7faa180da260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa25584700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa18049890 0x7faa18173f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa25584700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa18049cf0 0x7faa181744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa25584700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa180d01d0 con 0x7faa18049890 2024-09-13T18:08:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa25584700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa180ce0d0 con 0x7faa18049cf0 2024-09-13T18:08:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa25584700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa180ce3b0 con 0x7faa10006a80 2024-09-13T18:08:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa1f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa18049890 0x7faa18173f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa1f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa18049890 0x7faa18173f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49424/0 (socket says 172.21.15.40:49424) 2024-09-13T18:08:10.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa1f7fe700 1 -- 172.21.15.40:0/2629084581 learned_addr learned my addr 172.21.15.40:0/2629084581 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:08:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa24d83700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa18049cf0 0x7faa181744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa1ffff700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa10006a80 0x7faa180da260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa1f7fe700 1 -- 172.21.15.40:0/2629084581 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa10006a80 msgr2=0x7faa180da260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa1f7fe700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa10006a80 0x7faa180da260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa1f7fe700 1 -- 172.21.15.40:0/2629084581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa18049cf0 msgr2=0x7faa181744d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa1f7fe700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa18049cf0 0x7faa181744d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa1f7fe700 1 -- 172.21.15.40:0/2629084581 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa180ce690 con 0x7faa18049890 2024-09-13T18:08:10.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa1f7fe700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa18049890 0x7faa18173f90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faa1000dcf0 tx=0x7faa0c001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:10.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa1cff9700 1 -- 172.21.15.40:0/2629084581 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c005e20 con 0x7faa18049890 2024-09-13T18:08:10.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa1cff9700 1 -- 172.21.15.40:0/2629084581 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c004aa0 con 0x7faa18049890 2024-09-13T18:08:10.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa25584700 1 -- 172.21.15.40:0/2629084581 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa180ce970 con 0x7faa18049890 2024-09-13T18:08:10.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa1cff9700 1 -- 172.21.15.40:0/2629084581 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c004e30 con 0x7faa18049890 2024-09-13T18:08:10.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.970+0000 7faa25584700 1 -- 172.21.15.40:0/2629084581 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa18044d00 con 0x7faa18049890 2024-09-13T18:08:10.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.974+0000 7faa25584700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa180ceb30 0x7faa180cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.974+0000 7faa25584700 1 -- 172.21.15.40:0/2629084581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1804a0d0 con 0x7faa180ceb30 2024-09-13T18:08:10.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.974+0000 7faa1ffff700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa180ceb30 0x7faa180cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.974+0000 7faa1ffff700 1 -- 172.21.15.40:0/2629084581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa1804c3b0 con 0x7faa180ceb30 2024-09-13T18:08:10.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.974+0000 7faa1cff9700 1 -- 172.21.15.40:0/2629084581 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faa0c007070 con 0x7faa18049890 2024-09-13T18:08:10.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.974+0000 7faa1ffff700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa180ceb30 0x7faa180cd4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7faa14007fd0 tx=0x7faa14012bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:10.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.974+0000 7faa1cff9700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faa04066730 0x7faa04068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:10.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.974+0000 7faa1cff9700 1 -- 172.21.15.40:0/2629084581 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7faa0c0851a0 con 0x7faa18049890 2024-09-13T18:08:10.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.974+0000 7faa24d83700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faa04066730 0x7faa04068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:10.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.974+0000 7faa1cff9700 1 -- 172.21.15.40:0/2629084581 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14005480 con 0x7faa180ceb30 2024-09-13T18:08:10.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.974+0000 7faa1cff9700 1 -- 172.21.15.40:0/2629084581 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7faa14006880 con 0x7faa180ceb30 2024-09-13T18:08:10.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.974+0000 7faa1cff9700 1 -- 172.21.15.40:0/2629084581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa180ceb30 msgr2=0x7faa180cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:10.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.974+0000 7faa1cff9700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa180ceb30 0x7faa180cd4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7faa14007fd0 tx=0x7faa14012bf0 comp rx=0 tx=0).stop 2024-09-13T18:08:10.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.974+0000 7faa1cff9700 1 -- 172.21.15.40:0/2629084581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa180ceb30 msgr2=0x7faa180cd4e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:08:11.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.974+0000 7faa1cff9700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa180ceb30 0x7faa180cd4e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:11.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.974+0000 7faa24d83700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faa04066730 0x7faa04068bf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7faa200641b0 tx=0x7faa2008d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:11.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.986+0000 7faa25584700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa180cda20 0x7faa18176f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:11.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.986+0000 7faa25584700 1 -- 172.21.15.40:0/2629084581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa180ce820 con 0x7faa180cda20 2024-09-13T18:08:11.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.986+0000 7faa1ffff700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa180cda20 0x7faa18176f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:11.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.986+0000 7faa1ffff700 1 -- 172.21.15.40:0/2629084581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7faa180451e0 con 0x7faa180cda20 2024-09-13T18:08:11.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.986+0000 7faa1ffff700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa180cda20 0x7faa18176f50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7faa14007fd0 tx=0x7faa140012b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:11.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.986+0000 7faa1cff9700 1 -- 172.21.15.40:0/2629084581 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14025070 con 0x7faa180cda20 2024-09-13T18:08:11.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.986+0000 7faa1cff9700 1 -- 172.21.15.40:0/2629084581 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4451 (secure 0 0 0) 0x7faa140075e0 con 0x7faa180cda20 2024-09-13T18:08:11.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.986+0000 7faa1cff9700 1 -- 172.21.15.40:0/2629084581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa180cda20 msgr2=0x7faa18176f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:11.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.986+0000 7faa1cff9700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa180cda20 0x7faa18176f50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7faa14007fd0 tx=0x7faa140012b0 comp rx=0 tx=0).stop 2024-09-13T18:08:11.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.986+0000 7faa1cff9700 1 -- 172.21.15.40:0/2629084581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa180cda20 msgr2=0x7faa18176f50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:08:11.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.986+0000 7faa1cff9700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa180cda20 0x7faa18176f50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:11.004 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:08:11.005 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:08:11.005 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:08:11.005 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:08:11.005 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 146, 2024-09-13T18:08:11.005 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:08:11.005 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:08:11.005 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:08:11.005 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:08:11.005 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:11.005 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 13, 2024-09-13T18:08:11.005 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:08:11.006 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:08:11.006 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:08:11.006 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:08:11.006 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:08:11.006 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:08:11.006 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:08:11.006 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:08:11.006 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:08:11.006 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:08:11.006 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:08:11.006 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:08:11.006 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:08:11.006 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:11.006 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:08:11.007 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:08:11.007 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:08:11.007 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:08:11.007 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:08:11.007 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:08:11.007 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:08:11.007 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:08:11.007 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:08:11.007 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:08:11.007 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:08:11.007 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:08:11.007 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:11.007 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.007 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:08:11.008 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:08:11.008 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:08:11.008 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:08:11.008 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:08:11.008 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:08:11.008 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:08:11.008 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:08:11.008 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:08:11.008 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:08:11.008 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:08:11.008 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:08:11.008 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:08:11.008 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:08:11.008 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:08:11.009 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:08:11.009 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:08:11.009 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:08:11.009 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:08:11.009 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:08:11.009 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:08:11.009 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:08:11.009 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:08:11.009 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:08:11.009 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:08:11.009 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:08:11.009 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:08:11.009 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:11.009 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:08:11.010 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.010 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:08:11.010 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.010 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:08:11.010 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:08:11.010 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:08:11.010 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:08:11.010 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.010 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:08:11.010 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:08:11.010 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:11.010 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.010 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.010 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:08:11.011 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:08:11.011 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:11.011 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:11.011 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:11.011 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.011 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:08:11.011 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:08:11.011 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:08:11.011 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:08:11.011 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:08:11.011 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.011 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.011 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:08:11.011 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:08:11.012 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:08:11.012 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:08:11.012 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.012 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:08:11.012 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:08:11.012 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:11.012 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.012 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.012 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:08:11.012 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:08:11.012 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:11.012 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:11.012 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:11.012 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.013 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:08:11.013 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:08:11.013 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:08:11.013 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:08:11.013 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:08:11.013 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.013 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.013 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:08:11.013 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:08:11.013 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:08:11.013 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:08:11.013 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.013 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:08:11.013 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:08:11.014 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:11.014 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.014 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.014 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:08:11.014 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:08:11.014 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:11.014 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:11.014 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:11.014 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.014 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:08:11.014 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:08:11.014 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:08:11.014 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:08:11.014 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:08:11.015 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:11.015 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:11.015 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.015 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:08:11.015 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:08:11.015 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.015 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:11.015 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:11.015 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:08:11.015 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:11.015 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:11.015 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:08:11.015 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.015 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:11.016 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:11.016 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:08:11.016 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:11.016 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:11.016 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.016 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:08:11.016 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:08:11.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.986+0000 7faa25584700 1 -- 172.21.15.40:0/2629084581 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faa04066730 msgr2=0x7faa04068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:11.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.986+0000 7faa25584700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faa04066730 0x7faa04068bf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7faa200641b0 tx=0x7faa2008d040 comp rx=0 tx=0).stop 2024-09-13T18:08:11.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.986+0000 7faa25584700 1 -- 172.21.15.40:0/2629084581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa18049890 msgr2=0x7faa18173f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:11.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.986+0000 7faa25584700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa18049890 0x7faa18173f90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faa1000dcf0 tx=0x7faa0c001980 comp rx=0 tx=0).stop 2024-09-13T18:08:11.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.986+0000 7faa24d83700 1 -- 172.21.15.40:0/2629084581 reap_dead start 2024-09-13T18:08:11.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.990+0000 7faa25584700 1 -- 172.21.15.40:0/2629084581 shutdown_connections 2024-09-13T18:08:11.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.990+0000 7faa25584700 1 --2- 172.21.15.40:0/2629084581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa18049890 0x7faa18173f90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:11.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.990+0000 7faa25584700 1 -- 172.21.15.40:0/2629084581 >> 172.21.15.40:0/2629084581 conn(0x7faa1804fd00 msgr2=0x7faa1804a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:08:11.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.990+0000 7faa25584700 1 -- 172.21.15.40:0/2629084581 shutdown_connections 2024-09-13T18:08:11.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:10.990+0000 7faa25584700 1 -- 172.21.15.40:0/2629084581 wait complete. 2024-09-13T18:08:11.018 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:08:11.018 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:08:11.018 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:08:11.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.142+0000 7f423138e700 1 Processor -- start 2024-09-13T18:08:11.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.142+0000 7f423138e700 1 -- start start 2024-09-13T18:08:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.142+0000 7f423138e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f422c0c7fb0 0x7f422c159f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.142+0000 7f423138e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f422c15cfe0 0x7f422c15d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.142+0000 7f423138e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422c15a450 con 0x7f422c153730 2024-09-13T18:08:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.142+0000 7f423138e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f422c070ae0 con 0x7f422c15cfe0 2024-09-13T18:08:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.142+0000 7f423138e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f422c071010 con 0x7f422c0c7fb0 2024-09-13T18:08:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.142+0000 7f4230b8d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f422c15cfe0 0x7f422c15d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.142+0000 7f4230b8d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f422c15cfe0 0x7f422c15d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49440/0 (socket says 172.21.15.40:49440) 2024-09-13T18:08:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.142+0000 7f4230b8d700 1 -- 172.21.15.40:0/1375597691 learned_addr learned my addr 172.21.15.40:0/1375597691 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:08:11.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.142+0000 7f422b7fe700 1 --2- 172.21.15.40:0/1375597691 >> v2:172.21.15.43:3300/0 conn(0x7f422c0c7fb0 0x7f422c159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:11.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.142+0000 7f4230b8d700 1 -- 172.21.15.40:0/1375597691 >> v2:172.21.15.43:3300/0 conn(0x7f422c0c7fb0 msgr2=0x7f422c159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:08:11.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.142+0000 7f4230b8d700 1 --2- 172.21.15.40:0/1375597691 >> v2:172.21.15.43:3300/0 conn(0x7f422c0c7fb0 0x7f422c159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:11.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.142+0000 7f4230b8d700 1 -- 172.21.15.40:0/1375597691 >> v1:172.21.15.43:6789/0 conn(0x7f422c153730 legacy=0x7f422c0c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:11.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.142+0000 7f4230b8d700 1 -- 172.21.15.40:0/1375597691 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c15d990 con 0x7f422c15cfe0 2024-09-13T18:08:11.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.142+0000 7f4230b8d700 1 --2- 172.21.15.40:0/1375597691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c15cfe0 0x7f422c15d450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f422c1570c0 tx=0x7f422400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3e0c7c085415a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:11.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.142+0000 7f422affd700 1 -- 172.21.15.40:0/1375597691 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422400e070 con 0x7f422c15cfe0 2024-09-13T18:08:11.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f422affd700 1 -- 172.21.15.40:0/1375597691 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422400b910 con 0x7f422c15cfe0 2024-09-13T18:08:11.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f422affd700 1 -- 172.21.15.40:0/1375597691 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422400bcc0 con 0x7f422c15cfe0 2024-09-13T18:08:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f423138e700 1 -- 172.21.15.40:0/1375597691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c15cfe0 msgr2=0x7f422c15d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f423138e700 1 --2- 172.21.15.40:0/1375597691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c15cfe0 0x7f422c15d450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f422c1570c0 tx=0x7f422400b330 comp rx=0 tx=0).stop 2024-09-13T18:08:11.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f423138e700 1 -- 172.21.15.40:0/1375597691 shutdown_connections 2024-09-13T18:08:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f423138e700 1 --2- 172.21.15.40:0/1375597691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c15cfe0 0x7f422c15d450 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f423138e700 1 --2- 172.21.15.40:0/1375597691 >> v2:172.21.15.43:3300/0 conn(0x7f422c0c7fb0 0x7f422c159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f423138e700 1 -- 172.21.15.40:0/1375597691 >> 172.21.15.40:0/1375597691 conn(0x7f422c06e6d0 msgr2=0x7f422c06eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:08:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f423138e700 1 -- 172.21.15.40:0/1375597691 shutdown_connections 2024-09-13T18:08:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f423138e700 1 -- 172.21.15.40:0/1375597691 wait complete. 2024-09-13T18:08:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f423138e700 1 Processor -- start 2024-09-13T18:08:11.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f423138e700 1 -- start start 2024-09-13T18:08:11.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f423138e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f422c0c7fb0 0x7f422c14fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f423138e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c153730 0x7f422c155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f423138e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c15cfe0 0x7f422c155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f423138e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c1501a0 con 0x7f422c15cfe0 2024-09-13T18:08:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f423138e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c153c90 con 0x7f422c153730 2024-09-13T18:08:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f423138e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c153f50 con 0x7f422c0c7fb0 2024-09-13T18:08:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f4230b8d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c15cfe0 0x7f422c155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f4230b8d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c15cfe0 0x7f422c155e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49456/0 (socket says 172.21.15.40:49456) 2024-09-13T18:08:11.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f4230b8d700 1 -- 172.21.15.40:0/2688612583 learned_addr learned my addr 172.21.15.40:0/2688612583 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:08:11.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f422bfff700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f422c0c7fb0 0x7f422c14fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:11.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f422b7fe700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c153730 0x7f422c155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:11.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f4230b8d700 1 -- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f422c0c7fb0 msgr2=0x7f422c14fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:11.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f4230b8d700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f422c0c7fb0 0x7f422c14fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:11.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f4230b8d700 1 -- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c153730 msgr2=0x7f422c155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:11.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f4230b8d700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c153730 0x7f422c155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:11.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f4230b8d700 1 -- 172.21.15.40:0/2688612583 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4224009b30 con 0x7f422c15cfe0 2024-09-13T18:08:11.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f422bfff700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f422c0c7fb0 0x7f422c14fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:08:11.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f422b7fe700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f422c153730 0x7f422c155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:08:11.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.146+0000 7f4230b8d700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c15cfe0 0x7f422c155e80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f422c0c8600 tx=0x7f4224004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:11.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.150+0000 7f4228ff9700 1 -- 172.21.15.40:0/2688612583 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42240092e0 con 0x7f422c15cfe0 2024-09-13T18:08:11.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.150+0000 7f4228ff9700 1 -- 172.21.15.40:0/2688612583 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422401b430 con 0x7f422c15cfe0 2024-09-13T18:08:11.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.150+0000 7f423138e700 1 -- 172.21.15.40:0/2688612583 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422c154230 con 0x7f422c15cfe0 2024-09-13T18:08:11.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.150+0000 7f4228ff9700 1 -- 172.21.15.40:0/2688612583 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422401b7e0 con 0x7f422c15cfe0 2024-09-13T18:08:11.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.150+0000 7f423138e700 1 -- 172.21.15.40:0/2688612583 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422c1547f0 con 0x7f422c15cfe0 2024-09-13T18:08:11.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.150+0000 7f423138e700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f420c002a40 0x7f420c004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:11.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.150+0000 7f423138e700 1 -- 172.21.15.40:0/2688612583 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420c007f40 con 0x7f420c002a40 2024-09-13T18:08:11.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.150+0000 7f422bfff700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f420c002a40 0x7f420c004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:11.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.150+0000 7f4228ff9700 1 -- 172.21.15.40:0/2688612583 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f422401b940 con 0x7f422c15cfe0 2024-09-13T18:08:11.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.150+0000 7f4228ff9700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4218066760 0x7f4218068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:11.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.150+0000 7f422bfff700 1 -- 172.21.15.40:0/2688612583 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f420c008100 con 0x7f420c002a40 2024-09-13T18:08:11.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.150+0000 7f4228ff9700 1 -- 172.21.15.40:0/2688612583 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4224085ae0 con 0x7f422c15cfe0 2024-09-13T18:08:11.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.150+0000 7f422b7fe700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4218066760 0x7f4218068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:11.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.150+0000 7f422bfff700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f420c002a40 0x7f420c004f00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f421c000f60 tx=0x7f421c00ea90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:11.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.150+0000 7f4228ff9700 1 -- 172.21.15.40:0/2688612583 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c0052e0 con 0x7f420c002a40 2024-09-13T18:08:11.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.150+0000 7f422b7fe700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4218066760 0x7f4218068c20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f422c0cba50 tx=0x7f4220008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:11.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.154+0000 7f4228ff9700 1 -- 172.21.15.40:0/2688612583 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f421c005440 con 0x7f420c002a40 2024-09-13T18:08:11.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.154+0000 7f4228ff9700 1 -- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f420c002a40 msgr2=0x7f420c004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:11.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.154+0000 7f4228ff9700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f420c002a40 0x7f420c004f00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f421c000f60 tx=0x7f421c00ea90 comp rx=0 tx=0).stop 2024-09-13T18:08:11.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.154+0000 7f4228ff9700 1 -- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f420c002a40 msgr2=0x7f420c004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:08:11.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.154+0000 7f4228ff9700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f420c002a40 0x7f420c004f00 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f421c000f60 tx=0x7f421c00ea90 comp rx=0 tx=0).stop 2024-09-13T18:08:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.170+0000 7f423138e700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f420c006f90 0x7f420c007420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.170+0000 7f423138e700 1 -- 172.21.15.40:0/2688612583 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f420c00a400 con 0x7f420c006f90 2024-09-13T18:08:11.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.170+0000 7f422bfff700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f420c006f90 0x7f420c007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:11.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.170+0000 7f422bfff700 1 -- 172.21.15.40:0/2688612583 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f420c00a5c0 con 0x7f420c006f90 2024-09-13T18:08:11.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.174+0000 7f422bfff700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f420c006f90 0x7f420c007420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f421c000f60 tx=0x7f421c001120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:11.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.174+0000 7f4228ff9700 1 -- 172.21.15.40:0/2688612583 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c021070 con 0x7f420c006f90 2024-09-13T18:08:11.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.174+0000 7f4228ff9700 1 -- 172.21.15.40:0/2688612583 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4949 (secure 0 0 0) 0x7f421c007230 con 0x7f420c006f90 2024-09-13T18:08:11.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.174+0000 7f4228ff9700 1 -- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f420c006f90 msgr2=0x7f420c007420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:11.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.174+0000 7f4228ff9700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f420c006f90 0x7f420c007420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f421c000f60 tx=0x7f421c001120 comp rx=0 tx=0).stop 2024-09-13T18:08:11.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.174+0000 7f4228ff9700 1 -- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f420c006f90 msgr2=0x7f420c007420 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:08:11.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.174+0000 7f4228ff9700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f420c006f90 0x7f420c007420 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:11.185 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:08:11.185 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:08:11.185 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:08:11.185 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:08:11.185 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 146, 2024-09-13T18:08:11.185 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:08:11.185 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:08:11.185 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:08:11.185 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:08:11.186 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:11.186 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 13, 2024-09-13T18:08:11.186 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:08:11.186 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:08:11.186 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:08:11.186 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:08:11.186 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:08:11.186 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:08:11.186 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:08:11.186 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:08:11.186 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:08:11.186 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:08:11.186 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:08:11.187 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:08:11.187 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:08:11.187 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:11.187 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:08:11.187 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:08:11.187 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:08:11.187 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:08:11.187 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:08:11.187 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:08:11.187 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:08:11.187 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:08:11.187 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:08:11.187 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:08:11.187 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:08:11.188 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:08:11.188 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:11.188 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.188 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:08:11.188 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:08:11.188 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:08:11.188 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:08:11.188 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:08:11.188 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:08:11.188 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:08:11.188 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:08:11.188 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:08:11.188 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:08:11.188 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:08:11.189 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:08:11.189 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:08:11.189 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:08:11.189 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:08:11.189 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:08:11.189 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:08:11.189 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:08:11.189 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:08:11.189 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:08:11.189 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:08:11.189 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:08:11.189 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:08:11.189 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:08:11.190 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:08:11.190 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:08:11.190 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:08:11.190 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:11.190 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:08:11.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.190 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:08:11.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.190 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:08:11.190 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:08:11.190 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:08:11.190 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:08:11.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.190 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:08:11.191 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:08:11.191 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:11.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.191 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:08:11.191 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:08:11.191 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:11.191 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:11.191 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:11.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.191 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:08:11.191 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:08:11.191 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:08:11.191 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:08:11.192 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:08:11.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.192 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:08:11.192 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:08:11.192 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:08:11.192 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:08:11.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.192 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:08:11.192 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:08:11.192 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:11.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.192 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:08:11.193 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:08:11.193 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:11.193 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:11.193 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:11.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.193 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:08:11.193 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:08:11.193 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:08:11.193 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:08:11.193 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:08:11.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.193 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:08:11.194 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:08:11.194 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:08:11.194 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:08:11.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.194 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:08:11.194 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:08:11.194 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:11.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.194 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:08:11.194 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:08:11.194 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:11.194 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:11.194 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:11.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.195 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:08:11.195 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:08:11.195 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:08:11.195 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:08:11.195 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:08:11.195 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:11.195 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:11.195 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.195 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:08:11.195 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:08:11.195 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.195 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:11.195 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:11.195 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:08:11.196 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:11.196 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:11.196 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:08:11.196 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.196 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:11.196 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:11.196 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:08:11.196 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:11.196 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:11.196 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:08:11.196 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.196 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:11.197 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:11.197 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:08:11.197 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:11.197 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:11.197 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:08:11.197 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.197 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:11.197 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:11.197 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:08:11.197 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:11.197 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:11.197 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:08:11.197 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:11.198 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:11.198 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:11.198 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:08:11.198 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:11.198 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:11.198 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:11.198 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:08:11.198 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:08:11.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.174+0000 7f423138e700 1 -- 172.21.15.40:0/2688612583 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4218066760 msgr2=0x7f4218068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.174+0000 7f423138e700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4218066760 0x7f4218068c20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f422c0cba50 tx=0x7f4220008040 comp rx=0 tx=0).stop 2024-09-13T18:08:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.174+0000 7f423138e700 1 -- 172.21.15.40:0/2688612583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c15cfe0 msgr2=0x7f422c155e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.174+0000 7f4230b8d700 1 -- 172.21.15.40:0/2688612583 reap_dead start 2024-09-13T18:08:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.174+0000 7f423138e700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c15cfe0 0x7f422c155e80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f422c0c8600 tx=0x7f4224004af0 comp rx=0 tx=0).stop 2024-09-13T18:08:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.174+0000 7f423138e700 1 -- 172.21.15.40:0/2688612583 shutdown_connections 2024-09-13T18:08:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.174+0000 7f423138e700 1 --2- 172.21.15.40:0/2688612583 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f422c15cfe0 0x7f422c155e80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.174+0000 7f423138e700 1 -- 172.21.15.40:0/2688612583 >> 172.21.15.40:0/2688612583 conn(0x7f422c06e6d0 msgr2=0x7f422c151170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:08:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.174+0000 7f423138e700 1 -- 172.21.15.40:0/2688612583 shutdown_connections 2024-09-13T18:08:11.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:11.174+0000 7f423138e700 1 -- 172.21.15.40:0/2688612583 wait complete. 2024-09-13T18:08:11.200 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:08:11.201 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-13T18:08:11.201 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['a'] 2024-09-13T18:08:11.201 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-13T18:08:11.201 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-13T18:08:11.201 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T18:08:11.201 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:08:11.253 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T18:08:11.253 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-09-13T18:08:11.253 INFO:tasks.ceph.mon.a:Sent signal 19 2024-09-13T18:08:11.253 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-13T18:08:26.255 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-09-13T18:08:26.255 INFO:tasks.ceph.mon.a:Sent signal 18 2024-09-13T18:08:26.255 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T18:08:26.256 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:08:26.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.354+0000 7f29f1aab700 1 Processor -- start 2024-09-13T18:08:26.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.354+0000 7f29f1aab700 1 -- start start 2024-09-13T18:08:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.358+0000 7f29f1aab700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 0x7f29ec071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.358+0000 7f29f1aab700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29ec15a520 con 0x7f29ec072130 2024-09-13T18:08:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.358+0000 7f29f1aab700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29ec1564b0 con 0x7f29ec1503d0 2024-09-13T18:08:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.358+0000 7f29f1aab700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29ec072540 con 0x7f29ec06b4a0 2024-09-13T18:08:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.358+0000 7f29f12aa700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f29ec072130 0x7f29ec160970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55066/0 (socket says 172.21.15.40:55066) 2024-09-13T18:08:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.358+0000 7f29f12aa700 1 -- 172.21.15.40:0/3706147582 learned_addr learned my addr 172.21.15.40:0/3706147582 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:08:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.358+0000 7f29ebfff700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 0x7f29ec071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.358+0000 7f29ebfff700 1 -- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 msgr2=0x7f29ec071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:08:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.358+0000 7f29ebfff700 1 -- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 msgr2=0x7f29ec071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:08:26.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.358+0000 7f29ebfff700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 0x7f29ec071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:08:26.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.358+0000 7f29ebfff700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 0x7f29ec071bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:08:26.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.562+0000 7f29ebfff700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 0x7f29ec071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:26.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.562+0000 7f29ebfff700 1 -- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 msgr2=0x7f29ec071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:08:26.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.562+0000 7f29ebfff700 1 -- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 msgr2=0x7f29ec071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:08:26.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.562+0000 7f29ebfff700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 0x7f29ec071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:08:26.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.562+0000 7f29ebfff700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 0x7f29ec071bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:08:26.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.966+0000 7f29ebfff700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 0x7f29ec071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:26.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.966+0000 7f29ebfff700 1 -- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 msgr2=0x7f29ec071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:08:26.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.966+0000 7f29ebfff700 1 -- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 msgr2=0x7f29ec071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:08:26.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.966+0000 7f29ebfff700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 0x7f29ec071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:08:26.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:26.966+0000 7f29ebfff700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 0x7f29ec071bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:08:27.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:27.770+0000 7f29ebfff700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 0x7f29ec071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:27.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:27.770+0000 7f29ebfff700 1 -- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 msgr2=0x7f29ec071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:08:27.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:27.770+0000 7f29ebfff700 1 -- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 msgr2=0x7f29ec071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:08:27.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:27.770+0000 7f29ebfff700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 0x7f29ec071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:08:27.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:27.770+0000 7f29ebfff700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 0x7f29ec071bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:08:29.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.358+0000 7f29ea7fc700 1 -- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 msgr2=0x7f29ec071bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:08:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.358+0000 7f29ea7fc700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29ec06b4a0 0x7f29ec071bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.358+0000 7f29ea7fc700 1 -- 172.21.15.40:0/3706147582 >> v1:172.21.15.43:6789/0 conn(0x7f29ec1503d0 legacy=0x7f29ec156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.358+0000 7f29ea7fc700 1 -- 172.21.15.40:0/3706147582 >> v1:172.21.15.40:6789/0 conn(0x7f29ec072130 legacy=0x7f29ec160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.358+0000 7f29ea7fc700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 0x7f29d0005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.358+0000 7f29ea7fc700 1 --2- 172.21.15.40:0/3706147582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29d0006200 0x7f29d0008680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.358+0000 7f29ea7fc700 1 -- 172.21.15.40:0/3706147582 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29ec072540 con 0x7f29d0003740 2024-09-13T18:08:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.358+0000 7f29ea7fc700 1 -- 172.21.15.40:0/3706147582 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d0008da0 con 0x7f29d0006200 2024-09-13T18:08:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.358+0000 7f29ebfff700 1 -- 172.21.15.40:0/3706147582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29d0006200 msgr2=0x7f29d0008680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:08:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.358+0000 7f29ebfff700 1 --2- 172.21.15.40:0/3706147582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29d0006200 0x7f29d0008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:08:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.358+0000 7f29f0aa9700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 0x7f29d0005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.358+0000 7f29f0aa9700 1 -- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 msgr2=0x7f29d0005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:08:29.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.358+0000 7f29f0aa9700 1 -- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 msgr2=0x7f29d0005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:08:29.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.358+0000 7f29f0aa9700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 0x7f29d0005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:08:29.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.358+0000 7f29f0aa9700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 0x7f29d0005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:08:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.562+0000 7f29ebfff700 1 -- 172.21.15.40:0/3706147582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29d0006200 msgr2=0x7f29d0008680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:08:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.562+0000 7f29ebfff700 1 --2- 172.21.15.40:0/3706147582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29d0006200 0x7f29d0008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:08:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.562+0000 7f29f0aa9700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 0x7f29d0005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.562+0000 7f29f0aa9700 1 -- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 msgr2=0x7f29d0005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:08:29.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.562+0000 7f29f0aa9700 1 -- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 msgr2=0x7f29d0005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:08:29.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.562+0000 7f29f0aa9700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 0x7f29d0005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:08:29.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.562+0000 7f29f0aa9700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 0x7f29d0005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:08:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.966+0000 7f29ebfff700 1 -- 172.21.15.40:0/3706147582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29d0006200 msgr2=0x7f29d0008680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:08:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.966+0000 7f29ebfff700 1 --2- 172.21.15.40:0/3706147582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29d0006200 0x7f29d0008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:08:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.966+0000 7f29f0aa9700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 0x7f29d0005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.966+0000 7f29f0aa9700 1 -- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 msgr2=0x7f29d0005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:08:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.966+0000 7f29f0aa9700 1 -- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 msgr2=0x7f29d0005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:08:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.966+0000 7f29f0aa9700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 0x7f29d0005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:08:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:29.966+0000 7f29f0aa9700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 0x7f29d0005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:08:30.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:30.770+0000 7f29ebfff700 1 -- 172.21.15.40:0/3706147582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29d0006200 msgr2=0x7f29d0008680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:08:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:30.770+0000 7f29ebfff700 1 --2- 172.21.15.40:0/3706147582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29d0006200 0x7f29d0008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:08:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:30.770+0000 7f29f0aa9700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 0x7f29d0005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:30.770+0000 7f29f0aa9700 1 -- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 msgr2=0x7f29d0005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:08:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:30.770+0000 7f29f0aa9700 1 -- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 msgr2=0x7f29d0005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:08:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:30.770+0000 7f29f0aa9700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 0x7f29d0005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:08:30.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:30.770+0000 7f29f0aa9700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 0x7f29d0005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:08:32.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.358+0000 7f29ea7fc700 1 -- 172.21.15.40:0/3706147582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29d0006200 msgr2=0x7f29d0008680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:08:32.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.358+0000 7f29ea7fc700 1 --2- 172.21.15.40:0/3706147582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29d0006200 0x7f29d0008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.358+0000 7f29ea7fc700 1 -- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 msgr2=0x7f29d0005bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:08:32.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.358+0000 7f29ea7fc700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.40:3300/0 conn(0x7f29d0003740 0x7f29d0005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.358+0000 7f29f12aa700 1 -- 172.21.15.40:0/3706147582 reap_dead start 2024-09-13T18:08:32.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.358+0000 7f29ea7fc700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29d000c220 0x7f29d000e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.358+0000 7f29ea7fc700 1 -- 172.21.15.40:0/3706147582 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29ec15a000 con 0x7f29d0007d90 2024-09-13T18:08:32.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.358+0000 7f29ea7fc700 1 -- 172.21.15.40:0/3706147582 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d000fc30 con 0x7f29d0003bb0 2024-09-13T18:08:32.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.358+0000 7f29ea7fc700 1 -- 172.21.15.40:0/3706147582 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29ec072540 con 0x7f29d000c220 2024-09-13T18:08:32.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.358+0000 7f29f12aa700 1 --2- 172.21.15.40:0/3706147582 >> v2:172.21.15.43:3300/0 conn(0x7f29d000c220 0x7f29d000e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:32.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.358+0000 7f29eb7fe700 1 -- 172.21.15.40:0/3706147582 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2560608246 0 0) 0x7f29ec15a000 con 0x7f29d0007d90 2024-09-13T18:08:32.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.358+0000 7f29eb7fe700 1 -- 172.21.15.40:0/3706147582 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29dc0035d0 con 0x7f29d0007d90 2024-09-13T18:08:32.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.358+0000 7f29f12aa700 1 -- 172.21.15.40:0/3706147582 >> v1:172.21.15.43:6789/0 conn(0x7f29d0003bb0 legacy=0x7f29d0006670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:32.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.358+0000 7f29f12aa700 1 -- 172.21.15.40:0/3706147582 >> v1:172.21.15.40:6789/0 conn(0x7f29d0007d90 legacy=0x7f29d00081b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:32.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.358+0000 7f29f12aa700 1 -- 172.21.15.40:0/3706147582 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d0010e10 con 0x7f29d000c220 2024-09-13T18:08:32.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.358+0000 7f29f12aa700 1 --2- 172.21.15.40:0/3706147582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29d000c220 0x7f29d000e610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f29ec067090 tx=0x7f29e4001f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57220c519c26a408 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:32.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.358+0000 7f29eb7fe700 1 -- 172.21.15.40:0/3706147582 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e400c9b0 con 0x7f29d000c220 2024-09-13T18:08:32.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.358+0000 7f29eb7fe700 1 -- 172.21.15.40:0/3706147582 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e40020f0 con 0x7f29d000c220 2024-09-13T18:08:32.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.358+0000 7f29eb7fe700 1 -- 172.21.15.40:0/3706147582 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e400cd50 con 0x7f29d000c220 2024-09-13T18:08:32.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.362+0000 7f29f1aab700 1 -- 172.21.15.40:0/3706147582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29d000c220 msgr2=0x7f29d000e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:32.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.362+0000 7f29f1aab700 1 --2- 172.21.15.40:0/3706147582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29d000c220 0x7f29d000e610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f29ec067090 tx=0x7f29e4001f60 comp rx=0 tx=0).stop 2024-09-13T18:08:32.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.362+0000 7f29f1aab700 1 -- 172.21.15.40:0/3706147582 shutdown_connections 2024-09-13T18:08:32.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.362+0000 7f29f1aab700 1 --2- 172.21.15.40:0/3706147582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29d000c220 0x7f29d000e610 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.362+0000 7f29f1aab700 1 -- 172.21.15.40:0/3706147582 >> 172.21.15.40:0/3706147582 conn(0x7f29ec06e4f0 msgr2=0x7f29ec06e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:08:32.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.362+0000 7f29f1aab700 1 -- 172.21.15.40:0/3706147582 shutdown_connections 2024-09-13T18:08:32.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.362+0000 7f29f1aab700 1 -- 172.21.15.40:0/3706147582 wait complete. 2024-09-13T18:08:32.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.366+0000 7f29f1aab700 1 Processor -- start 2024-09-13T18:08:32.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.366+0000 7f29f1aab700 1 -- start start 2024-09-13T18:08:32.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.366+0000 7f29f1aab700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d0003bb0 0x7f29ec151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.366+0000 7f29f1aab700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29d0007d90 0x7f29ec158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.366+0000 7f29f1aab700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29d000c220 0x7f29ec0c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.366+0000 7f29f1aab700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec1512c0 con 0x7f29d0003bb0 2024-09-13T18:08:32.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.366+0000 7f29f1aab700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec14a4b0 con 0x7f29d0007d90 2024-09-13T18:08:32.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.366+0000 7f29f1aab700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec150d60 con 0x7f29d000c220 2024-09-13T18:08:32.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.366+0000 7f29f12aa700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29d000c220 msgr2=0x7f29ec0c45d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:08:32.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.366+0000 7f29f0aa9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d0003bb0 0x7f29ec151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:32.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.366+0000 7f29f12aa700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29d000c220 0x7f29ec0c45d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:08:32.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.366+0000 7f29f0aa9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d0003bb0 0x7f29ec151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46502/0 (socket says 172.21.15.40:46502) 2024-09-13T18:08:32.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.366+0000 7f29f0aa9700 1 -- 172.21.15.40:0/3146865627 learned_addr learned my addr 172.21.15.40:0/3146865627 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:08:32.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.366+0000 7f29ebfff700 1 --2- 172.21.15.40:0/3146865627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29d0007d90 0x7f29ec158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:32.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.366+0000 7f29f0aa9700 1 -- 172.21.15.40:0/3146865627 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29d000c220 msgr2=0x7f29ec0c45d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:08:32.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.366+0000 7f29f0aa9700 1 --2- 172.21.15.40:0/3146865627 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29d000c220 0x7f29ec0c45d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.366+0000 7f29f0aa9700 1 -- 172.21.15.40:0/3146865627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29d0007d90 msgr2=0x7f29ec158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:32.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.366+0000 7f29f0aa9700 1 --2- 172.21.15.40:0/3146865627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29d0007d90 0x7f29ec158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.366+0000 7f29f0aa9700 1 -- 172.21.15.40:0/3146865627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec15a520 con 0x7f29d0003bb0 2024-09-13T18:08:32.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.370+0000 7f29f0aa9700 1 --2- 172.21.15.40:0/3146865627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d0003bb0 0x7f29ec151500 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f29e00055f0 tx=0x7f29e0001570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:32.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.370+0000 7f29e97fa700 1 -- 172.21.15.40:0/3146865627 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e0016450 con 0x7f29d0003bb0 2024-09-13T18:08:32.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.370+0000 7f29e97fa700 1 -- 172.21.15.40:0/3146865627 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e00165d0 con 0x7f29d0003bb0 2024-09-13T18:08:32.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.370+0000 7f29f1aab700 1 -- 172.21.15.40:0/3146865627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ec150fe0 con 0x7f29d0003bb0 2024-09-13T18:08:32.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.370+0000 7f29e97fa700 1 -- 172.21.15.40:0/3146865627 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e00169c0 con 0x7f29d0003bb0 2024-09-13T18:08:32.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.370+0000 7f29f1aab700 1 -- 172.21.15.40:0/3146865627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ec0c54c0 con 0x7f29d0003bb0 2024-09-13T18:08:32.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.370+0000 7f29f1aab700 1 -- 172.21.15.40:0/3146865627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ec0c7eb0 con 0x7f29d0003bb0 2024-09-13T18:08:32.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.374+0000 7f29e97fa700 1 -- 172.21.15.40:0/3146865627 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f29e0016b20 con 0x7f29d0003bb0 2024-09-13T18:08:32.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.374+0000 7f29e97fa700 1 --2- 172.21.15.40:0/3146865627 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f29d0068380 0x7f29d006a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.374+0000 7f29ebfff700 1 --2- 172.21.15.40:0/3146865627 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f29d0068380 0x7f29d006a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:32.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.374+0000 7f29e97fa700 1 -- 172.21.15.40:0/3146865627 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f29e0084a30 con 0x7f29d0003bb0 2024-09-13T18:08:32.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.374+0000 7f29ebfff700 1 --2- 172.21.15.40:0/3146865627 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f29d0068380 0x7f29d006a840 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f29ec14cb90 tx=0x7f29d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:32.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.382+0000 7f29e97fa700 1 -- 172.21.15.40:0/3146865627 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f29ec0c7eb0 con 0x7f29d0003bb0 2024-09-13T18:08:32.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.622+0000 7f29f1aab700 1 -- 172.21.15.40:0/3146865627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f29ec0c7e80 con 0x7f29d0003bb0 2024-09-13T18:08:32.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.622+0000 7f29e97fa700 1 -- 172.21.15.40:0/3146865627 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f29e0051940 con 0x7f29d0003bb0 2024-09-13T18:08:32.634 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":150,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:08:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.626+0000 7f29f1aab700 1 -- 172.21.15.40:0/3146865627 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f29d0068380 msgr2=0x7f29d006a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.626+0000 7f29f1aab700 1 --2- 172.21.15.40:0/3146865627 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f29d0068380 0x7f29d006a840 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f29ec14cb90 tx=0x7f29d8008040 comp rx=0 tx=0).stop 2024-09-13T18:08:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.626+0000 7f29f1aab700 1 -- 172.21.15.40:0/3146865627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d0003bb0 msgr2=0x7f29ec151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.626+0000 7f29f1aab700 1 --2- 172.21.15.40:0/3146865627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d0003bb0 0x7f29ec151500 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f29e00055f0 tx=0x7f29e0001570 comp rx=0 tx=0).stop 2024-09-13T18:08:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.626+0000 7f29f1aab700 1 -- 172.21.15.40:0/3146865627 shutdown_connections 2024-09-13T18:08:32.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.626+0000 7f29f1aab700 1 --2- 172.21.15.40:0/3146865627 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f29d0068380 0x7f29d006a840 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.626+0000 7f29f1aab700 1 --2- 172.21.15.40:0/3146865627 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f29d000c220 0x7f29ec0c45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.626+0000 7f29f1aab700 1 --2- 172.21.15.40:0/3146865627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29d0007d90 0x7f29ec158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.626+0000 7f29f1aab700 1 --2- 172.21.15.40:0/3146865627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29d0003bb0 0x7f29ec151500 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.626+0000 7f29f1aab700 1 -- 172.21.15.40:0/3146865627 >> 172.21.15.40:0/3146865627 conn(0x7f29ec06e4f0 msgr2=0x7f29ec0c2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:08:32.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.626+0000 7f29f1aab700 1 -- 172.21.15.40:0/3146865627 shutdown_connections 2024-09-13T18:08:32.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.626+0000 7f29f1aab700 1 -- 172.21.15.40:0/3146865627 wait complete. 2024-09-13T18:08:32.649 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T18:08:32.649 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-13T18:08:32.649 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:08:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.758+0000 7fc32bfff700 1 Processor -- start 2024-09-13T18:08:32.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.758+0000 7fc32bfff700 1 -- start start 2024-09-13T18:08:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.758+0000 7fc32bfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc32c0c8050 0x7fc32c1562e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.758+0000 7fc32bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc32c0c8430 0x7fc32c0c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.758+0000 7fc32bfff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc32c159f40 con 0x7fc32c070a60 2024-09-13T18:08:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.758+0000 7fc32bfff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc32c156820 con 0x7fc32c0c8430 2024-09-13T18:08:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.758+0000 7fc32bfff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc32c0c6ee0 con 0x7fc32c0c8050 2024-09-13T18:08:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.758+0000 7fc32a7fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc32c0c8430 0x7fc32c0c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.758+0000 7fc32a7fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc32c0c8430 0x7fc32c0c66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46506/0 (socket says 172.21.15.40:46506) 2024-09-13T18:08:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.758+0000 7fc32a7fc700 1 -- 172.21.15.40:0/924373747 learned_addr learned my addr 172.21.15.40:0/924373747 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:08:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.758+0000 7fc32b7fe700 1 --1- 172.21.15.40:0/924373747 >> v1:172.21.15.40:6789/0 conn(0x7fc32c070a60 0x7fc32c150160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53522/0 (socket says 172.21.15.40:53522) 2024-09-13T18:08:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.758+0000 7fc32affd700 1 --2- 172.21.15.40:0/924373747 >> v2:172.21.15.43:3300/0 conn(0x7fc32c0c8050 0x7fc32c1562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.758+0000 7fc32a7fc700 1 -- 172.21.15.40:0/924373747 >> v2:172.21.15.43:3300/0 conn(0x7fc32c0c8050 msgr2=0x7fc32c1562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:08:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.758+0000 7fc32a7fc700 1 --2- 172.21.15.40:0/924373747 >> v2:172.21.15.43:3300/0 conn(0x7fc32c0c8050 0x7fc32c1562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.758+0000 7fc32a7fc700 1 -- 172.21.15.40:0/924373747 >> v1:172.21.15.40:6789/0 conn(0x7fc32c070a60 legacy=0x7fc32c150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:32.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32a7fc700 1 -- 172.21.15.40:0/924373747 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32c0c71c0 con 0x7fc32c0c8430 2024-09-13T18:08:32.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32a7fc700 1 --2- 172.21.15.40:0/924373747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0c8430 0x7fc32c0c66a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0cbe70 tx=0x7fc31c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0e0436de36b07c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:32.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc329ffb700 1 -- 172.21.15.40:0/924373747 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c00b8a0 con 0x7fc32c0c8430 2024-09-13T18:08:32.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc329ffb700 1 -- 172.21.15.40:0/924373747 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31c00ba00 con 0x7fc32c0c8430 2024-09-13T18:08:32.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc329ffb700 1 -- 172.21.15.40:0/924373747 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c00bd90 con 0x7fc32c0c8430 2024-09-13T18:08:32.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32bfff700 1 -- 172.21.15.40:0/924373747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0c8430 msgr2=0x7fc32c0c66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:32.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32bfff700 1 --2- 172.21.15.40:0/924373747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0c8430 0x7fc32c0c66a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0cbe70 tx=0x7fc31c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32bfff700 1 -- 172.21.15.40:0/924373747 shutdown_connections 2024-09-13T18:08:32.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32bfff700 1 --2- 172.21.15.40:0/924373747 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0c8430 0x7fc32c0c66a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32bfff700 1 --2- 172.21.15.40:0/924373747 >> v2:172.21.15.43:3300/0 conn(0x7fc32c0c8050 0x7fc32c1562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32bfff700 1 -- 172.21.15.40:0/924373747 >> 172.21.15.40:0/924373747 conn(0x7fc32c066090 msgr2=0x7fc32c0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:08:32.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32bfff700 1 -- 172.21.15.40:0/924373747 shutdown_connections 2024-09-13T18:08:32.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32bfff700 1 -- 172.21.15.40:0/924373747 wait complete. 2024-09-13T18:08:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32bfff700 1 Processor -- start 2024-09-13T18:08:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32bfff700 1 -- start start 2024-09-13T18:08:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc32c070a60 0x7fc32c13ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc32c0c8050 0x7fc32c0c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0c8430 0x7fc32c0c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32bfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c1ed8f0 con 0x7fc32c0c8430 2024-09-13T18:08:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32bfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c13ef90 con 0x7fc32c0c8050 2024-09-13T18:08:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32bfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c1edde0 con 0x7fc32c070a60 2024-09-13T18:08:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32affd700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc32c070a60 msgr2=0x7fc32c13ea50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:08:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32affd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc32c070a60 0x7fc32c13ea50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:08:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0c8430 0x7fc32c0c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0c8430 0x7fc32c0c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46514/0 (socket says 172.21.15.40:46514) 2024-09-13T18:08:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32b7fe700 1 -- 172.21.15.40:0/1425390485 learned_addr learned my addr 172.21.15.40:0/1425390485 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:08:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32a7fc700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc32c0c8050 0x7fc32c0c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32b7fe700 1 -- 172.21.15.40:0/1425390485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc32c070a60 msgr2=0x7fc32c13ea50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:08:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32b7fe700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc32c070a60 0x7fc32c13ea50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.762+0000 7fc32b7fe700 1 -- 172.21.15.40:0/1425390485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc32c0c8050 msgr2=0x7fc32c0c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc32b7fe700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc32c0c8050 0x7fc32c0c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc32b7fe700 1 -- 172.21.15.40:0/1425390485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31c009570 con 0x7fc32c0c8430 2024-09-13T18:08:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc32a7fc700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc32c0c8050 0x7fc32c0c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:08:32.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc32b7fe700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0c8430 0x7fc32c0c9110 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc32400c4d0 tx=0x7fc32400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc3237fe700 1 -- 172.21.15.40:0/1425390485 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc324005cd0 con 0x7fc32c0c8430 2024-09-13T18:08:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc3237fe700 1 -- 172.21.15.40:0/1425390485 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc324005e30 con 0x7fc32c0c8430 2024-09-13T18:08:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc32bfff700 1 -- 172.21.15.40:0/1425390485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc32c144010 con 0x7fc32c0c8430 2024-09-13T18:08:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc32bfff700 1 -- 172.21.15.40:0/1425390485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc32c1edb00 con 0x7fc32c0c8430 2024-09-13T18:08:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc3237fe700 1 -- 172.21.15.40:0/1425390485 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3240072d0 con 0x7fc32c0c8430 2024-09-13T18:08:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc32bfff700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c144210 0x7fc32c0cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc32bfff700 1 -- 172.21.15.40:0/1425390485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c067720 con 0x7fc32c144210 2024-09-13T18:08:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc32affd700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c144210 0x7fc32c0cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc32affd700 1 -- 172.21.15.40:0/1425390485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc32c0c6be0 con 0x7fc32c144210 2024-09-13T18:08:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc3237fe700 1 -- 172.21.15.40:0/1425390485 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc324007430 con 0x7fc32c0c8430 2024-09-13T18:08:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc32affd700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c144210 0x7fc32c0cae90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc314007df0 tx=0x7fc31400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc3237fe700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc30c0666e0 0x7fc30c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc3237fe700 1 -- 172.21.15.40:0/1425390485 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc3240834f0 con 0x7fc32c0c8430 2024-09-13T18:08:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc32a7fc700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc30c0666e0 0x7fc30c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc3237fe700 1 -- 172.21.15.40:0/1425390485 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31400eea0 con 0x7fc32c144210 2024-09-13T18:08:32.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc3237fe700 1 -- 172.21.15.40:0/1425390485 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc314003a40 con 0x7fc32c144210 2024-09-13T18:08:32.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc3237fe700 1 -- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c144210 msgr2=0x7fc32c0cae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:32.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc3237fe700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c144210 0x7fc32c0cae90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc314007df0 tx=0x7fc31400e3c0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc3237fe700 1 -- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c144210 msgr2=0x7fc32c0cae90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:08:32.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.766+0000 7fc3237fe700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c144210 0x7fc32c0cae90 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc314007df0 tx=0x7fc31400e3c0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.770+0000 7fc32a7fc700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc30c0666e0 0x7fc30c068ba0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc31c000c00 tx=0x7fc31c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:32.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.774+0000 7fc32bfff700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c1412d0 0x7fc32c0cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.774+0000 7fc32bfff700 1 -- 172.21.15.40:0/1425390485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c0cb910 con 0x7fc32c1412d0 2024-09-13T18:08:32.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.774+0000 7fc32affd700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c1412d0 0x7fc32c0cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:32.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.774+0000 7fc32affd700 1 -- 172.21.15.40:0/1425390485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc32c159bd0 con 0x7fc32c1412d0 2024-09-13T18:08:32.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.774+0000 7fc32affd700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c1412d0 0x7fc32c0cb3d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc314016000 tx=0x7fc314004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.778+0000 7fc3237fe700 1 -- 172.21.15.40:0/1425390485 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc314004230 con 0x7fc32c1412d0 2024-09-13T18:08:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.778+0000 7fc3237fe700 1 -- 172.21.15.40:0/1425390485 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4939 (secure 0 0 0) 0x7fc314004390 con 0x7fc32c1412d0 2024-09-13T18:08:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.778+0000 7fc3237fe700 1 -- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c1412d0 msgr2=0x7fc32c0cb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:32.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.778+0000 7fc3237fe700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c1412d0 0x7fc32c0cb3d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc314016000 tx=0x7fc314004b60 comp rx=0 tx=0).stop 2024-09-13T18:08:32.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.778+0000 7fc3237fe700 1 -- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c1412d0 msgr2=0x7fc32c0cb3d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:08:32.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.778+0000 7fc3237fe700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c1412d0 0x7fc32c0cb3d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.788 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:08:32.788 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:08:32.788 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:08:32.788 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:08:32.788 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 150, 2024-09-13T18:08:32.788 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:08:32.788 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:08:32.789 INFO:teuthology.orchestra.run.smithi040.stdout: 1 2024-09-13T18:08:32.789 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:32.789 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:08:32.789 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:08:32.789 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:08:32.789 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:08:32.789 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:08:32.789 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:08:32.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:08:32.789 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:08:32.789 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:08:32.789 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:08:32.789 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:08:32.790 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:08:32.790 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:08:32.790 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:08:32.790 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:32.790 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:08:32.790 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:08:32.790 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:08:32.790 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:08:32.790 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:08:32.790 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:08:32.790 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:08:32.790 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:08:32.790 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:08:32.790 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:08:32.791 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:08:32.791 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:08:32.791 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:32.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.791 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:08:32.791 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:08:32.791 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:08:32.791 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:08:32.791 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:08:32.791 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:08:32.791 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:08:32.791 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:08:32.792 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:08:32.792 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:08:32.792 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:08:32.792 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:08:32.792 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:08:32.792 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:08:32.792 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:08:32.792 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:08:32.792 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:08:32.792 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:08:32.792 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:08:32.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:08:32.792 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:08:32.792 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:08:32.793 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:08:32.793 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:08:32.793 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:08:32.793 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:08:32.793 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:08:32.793 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:32.793 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:08:32.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.793 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:08:32.793 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.793 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:08:32.793 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:08:32.793 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:08:32.793 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:08:32.794 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.794 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:08:32.794 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:08:32.794 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:32.794 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.794 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.794 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:08:32.794 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:08:32.794 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:32.794 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:32.794 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:32.794 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.794 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:08:32.795 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:08:32.795 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:08:32.795 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:08:32.795 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:08:32.795 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.795 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.795 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:08:32.795 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:08:32.795 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:08:32.795 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:08:32.795 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.795 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:08:32.795 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:08:32.795 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:32.795 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.796 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.796 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:08:32.796 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:08:32.796 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:32.796 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:32.796 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:32.796 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.796 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:08:32.796 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:08:32.796 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:08:32.796 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:08:32.796 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:08:32.796 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.797 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.797 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:08:32.797 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:08:32.797 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:08:32.797 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:08:32.797 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.797 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:08:32.797 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:08:32.797 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:32.797 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.797 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.797 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:08:32.797 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:08:32.798 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:32.798 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:32.798 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:32.798 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.798 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:08:32.798 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:08:32.798 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:08:32.798 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:08:32.798 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:08:32.798 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:08:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:08:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:08:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:08:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:08:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:08:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:08:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:08:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:08:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:08:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:08:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.802 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:08:32.802 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:08:32.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.778+0000 7fc32bfff700 1 -- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc30c0666e0 msgr2=0x7fc30c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:32.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.778+0000 7fc32bfff700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc30c0666e0 0x7fc30c068ba0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc31c000c00 tx=0x7fc31c008030 comp rx=0 tx=0).stop 2024-09-13T18:08:32.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.778+0000 7fc32b7fe700 1 -- 172.21.15.40:0/1425390485 reap_dead start 2024-09-13T18:08:32.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.778+0000 7fc32bfff700 1 -- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0c8430 msgr2=0x7fc32c0c9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:32.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.778+0000 7fc32bfff700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0c8430 0x7fc32c0c9110 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc32400c4d0 tx=0x7fc32400c890 comp rx=0 tx=0).stop 2024-09-13T18:08:32.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.778+0000 7fc32bfff700 1 -- 172.21.15.40:0/1425390485 shutdown_connections 2024-09-13T18:08:32.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.778+0000 7fc32bfff700 1 --2- 172.21.15.40:0/1425390485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc32c0c8430 0x7fc32c0c9110 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.778+0000 7fc32bfff700 1 -- 172.21.15.40:0/1425390485 >> 172.21.15.40:0/1425390485 conn(0x7fc32c066090 msgr2=0x7fc32c0710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:08:32.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.778+0000 7fc32bfff700 1 -- 172.21.15.40:0/1425390485 shutdown_connections 2024-09-13T18:08:32.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.778+0000 7fc32bfff700 1 -- 172.21.15.40:0/1425390485 wait complete. 2024-09-13T18:08:32.804 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:08:32.804 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-13T18:08:32.805 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:08:32.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43dcb3e700 1 Processor -- start 2024-09-13T18:08:32.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43dcb3e700 1 -- start start 2024-09-13T18:08:32.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43dcb3e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43d814fab0 0x7f43d80c3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43dcb3e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f43d80c9790 0x7f43d81599a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43dcb3e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d81561c0 con 0x7f43d814fe90 2024-09-13T18:08:32.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43dcb3e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d81564a0 con 0x7f43d80c9790 2024-09-13T18:08:32.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43dcb3e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d8156780 con 0x7f43d814fab0 2024-09-13T18:08:32.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43d77fe700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43d814fab0 msgr2=0x7f43d80c3650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:08:32.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43d77fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43d814fab0 0x7f43d80c3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:08:32.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43cffff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f43d814fe90 0x7f43d8155910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53538/0 (socket says 172.21.15.40:53538) 2024-09-13T18:08:32.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43cffff700 1 -- 172.21.15.40:0/640484207 learned_addr learned my addr 172.21.15.40:0/640484207 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:08:32.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43d7fff700 1 --2- 172.21.15.40:0/640484207 >> v2:172.21.15.43:3300/0 conn(0x7f43d80c9790 0x7f43d81599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:32.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43d6ffd700 1 -- 172.21.15.40:0/640484207 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339919309 0 0) 0x7f43d81561c0 con 0x7f43d814fe90 2024-09-13T18:08:32.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43d6ffd700 1 -- 172.21.15.40:0/640484207 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43c40035d0 con 0x7f43d814fe90 2024-09-13T18:08:32.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43d7fff700 1 -- 172.21.15.40:0/640484207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43d814fab0 msgr2=0x7f43d80c3650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:08:32.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43d7fff700 1 --2- 172.21.15.40:0/640484207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43d814fab0 0x7f43d80c3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43d7fff700 1 -- 172.21.15.40:0/640484207 >> v1:172.21.15.40:6789/0 conn(0x7f43d814fe90 legacy=0x7f43d8155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:32.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43d6ffd700 1 -- 172.21.15.40:0/640484207 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1940203478 0 0) 0x7f43c40035d0 con 0x7f43d814fe90 2024-09-13T18:08:32.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43d7fff700 1 -- 172.21.15.40:0/640484207 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d8158ec0 con 0x7f43d80c9790 2024-09-13T18:08:32.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43d7fff700 1 --2- 172.21.15.40:0/640484207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d80c9790 0x7f43d81599a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009d90 tx=0x7f43d000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64bc4c02f9553171 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:32.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43d6ffd700 1 -- 172.21.15.40:0/640484207 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d000e070 con 0x7f43d80c9790 2024-09-13T18:08:32.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43d6ffd700 1 -- 172.21.15.40:0/640484207 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d0003a40 con 0x7f43d80c9790 2024-09-13T18:08:32.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.910+0000 7f43d6ffd700 1 -- 172.21.15.40:0/640484207 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0001670 con 0x7f43d80c9790 2024-09-13T18:08:32.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43dcb3e700 1 -- 172.21.15.40:0/640484207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d80c9790 msgr2=0x7f43d81599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:32.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43dcb3e700 1 --2- 172.21.15.40:0/640484207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d80c9790 0x7f43d81599a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009d90 tx=0x7f43d000b330 comp rx=0 tx=0).stop 2024-09-13T18:08:32.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43dcb3e700 1 -- 172.21.15.40:0/640484207 shutdown_connections 2024-09-13T18:08:32.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43dcb3e700 1 --2- 172.21.15.40:0/640484207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d80c9790 0x7f43d81599a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43dcb3e700 1 --2- 172.21.15.40:0/640484207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43d814fab0 0x7f43d80c3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43dcb3e700 1 -- 172.21.15.40:0/640484207 >> 172.21.15.40:0/640484207 conn(0x7f43d806a390 msgr2=0x7f43d806a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:08:32.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43dcb3e700 1 -- 172.21.15.40:0/640484207 shutdown_connections 2024-09-13T18:08:32.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43dcb3e700 1 -- 172.21.15.40:0/640484207 wait complete. 2024-09-13T18:08:32.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43dcb3e700 1 Processor -- start 2024-09-13T18:08:32.925 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43dcb3e700 1 -- start start 2024-09-13T18:08:32.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43dcb3e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d80c9790 0x7f43d81521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43dcb3e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43d814fab0 0x7f43d814b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43dcb3e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43d80c5560 0x7f43d814b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43dcb3e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d814acf0 con 0x7f43d80c5560 2024-09-13T18:08:32.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43dcb3e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d8152d30 con 0x7f43d80c9790 2024-09-13T18:08:32.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43dcb3e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d8152fa0 con 0x7f43d814fab0 2024-09-13T18:08:32.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43cffff700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43d814fab0 msgr2=0x7f43d814b180 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:08:32.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43cffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43d814fab0 0x7f43d814b180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:08:32.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43d7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43d80c5560 0x7f43d814b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:32.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43d7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43d80c5560 0x7f43d814b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46554/0 (socket says 172.21.15.40:46554) 2024-09-13T18:08:32.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43d7fff700 1 -- 172.21.15.40:0/2318970345 learned_addr learned my addr 172.21.15.40:0/2318970345 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:08:32.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.914+0000 7f43d77fe700 1 --2- 172.21.15.40:0/2318970345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d80c9790 0x7f43d81521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:32.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.918+0000 7f43d7fff700 1 -- 172.21.15.40:0/2318970345 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43d814fab0 msgr2=0x7f43d814b180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:08:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.918+0000 7f43d7fff700 1 --2- 172.21.15.40:0/2318970345 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43d814fab0 0x7f43d814b180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.918+0000 7f43d7fff700 1 -- 172.21.15.40:0/2318970345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d80c9790 msgr2=0x7f43d81521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.918+0000 7f43d7fff700 1 --2- 172.21.15.40:0/2318970345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d80c9790 0x7f43d81521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.918+0000 7f43d7fff700 1 -- 172.21.15.40:0/2318970345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d0009a70 con 0x7f43d80c5560 2024-09-13T18:08:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.918+0000 7f43d7fff700 1 --2- 172.21.15.40:0/2318970345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43d80c5560 0x7f43d814b6c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f43d000b680 tx=0x7f43d0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.918+0000 7f43d4ff9700 1 -- 172.21.15.40:0/2318970345 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d00045f0 con 0x7f43d80c5560 2024-09-13T18:08:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.918+0000 7f43d4ff9700 1 -- 172.21.15.40:0/2318970345 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d0004750 con 0x7f43d80c5560 2024-09-13T18:08:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.918+0000 7f43dcb3e700 1 -- 172.21.15.40:0/2318970345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d8153280 con 0x7f43d80c5560 2024-09-13T18:08:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.918+0000 7f43d4ff9700 1 -- 172.21.15.40:0/2318970345 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d001e430 con 0x7f43d80c5560 2024-09-13T18:08:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.918+0000 7f43dcb3e700 1 -- 172.21.15.40:0/2318970345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d81f2de0 con 0x7f43d80c5560 2024-09-13T18:08:32.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.918+0000 7f43dcb3e700 1 --2- 172.21.15.40:0/2318970345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d814fe90 0x7f43d81f2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.918+0000 7f43dcb3e700 1 -- 172.21.15.40:0/2318970345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d8067670 con 0x7f43d814fe90 2024-09-13T18:08:32.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.918+0000 7f43d77fe700 1 --2- 172.21.15.40:0/2318970345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d814fe90 0x7f43d81f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:32.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.918+0000 7f43d77fe700 1 -- 172.21.15.40:0/2318970345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f43d8159480 con 0x7f43d814fe90 2024-09-13T18:08:32.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.918+0000 7f43d77fe700 1 --2- 172.21.15.40:0/2318970345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d814fe90 0x7f43d81f2210 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f43c8009710 tx=0x7f43c8009ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:32.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.918+0000 7f43d4ff9700 1 -- 172.21.15.40:0/2318970345 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c8012600 con 0x7f43d814fe90 2024-09-13T18:08:32.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.922+0000 7f43d4ff9700 1 -- 172.21.15.40:0/2318970345 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f43d001e590 con 0x7f43d80c5560 2024-09-13T18:08:32.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.922+0000 7f43d4ff9700 1 --2- 172.21.15.40:0/2318970345 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f43b8066800 0x7f43b8068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.922+0000 7f43d4ff9700 1 -- 172.21.15.40:0/2318970345 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f43d0085aa0 con 0x7f43d80c5560 2024-09-13T18:08:32.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.922+0000 7f43cffff700 1 --2- 172.21.15.40:0/2318970345 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f43b8066800 0x7f43b8068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:32.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.922+0000 7f43d4ff9700 1 -- 172.21.15.40:0/2318970345 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f43c8012760 con 0x7f43d814fe90 2024-09-13T18:08:32.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.922+0000 7f43d4ff9700 1 -- 172.21.15.40:0/2318970345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d814fe90 msgr2=0x7f43d81f2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:32.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.922+0000 7f43d4ff9700 1 --2- 172.21.15.40:0/2318970345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d814fe90 0x7f43d81f2210 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f43c8009710 tx=0x7f43c8009ad0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.922+0000 7f43d4ff9700 1 -- 172.21.15.40:0/2318970345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d814fe90 msgr2=0x7f43d81f2210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:08:32.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.922+0000 7f43d4ff9700 1 --2- 172.21.15.40:0/2318970345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d814fe90 0x7f43d81f2210 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.922+0000 7f43cffff700 1 --2- 172.21.15.40:0/2318970345 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f43b8066800 0x7f43b8068cc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f43c0001340 tx=0x7f43c00012b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:32.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.930+0000 7f43dcb3e700 1 --2- 172.21.15.40:0/2318970345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d814c8f0 0x7f43d81f2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:08:32.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.930+0000 7f43dcb3e700 1 -- 172.21.15.40:0/2318970345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d80c3650 con 0x7f43d814c8f0 2024-09-13T18:08:32.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.930+0000 7f43d77fe700 1 --2- 172.21.15.40:0/2318970345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d814c8f0 0x7f43d81f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:08:32.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.930+0000 7f43d77fe700 1 -- 172.21.15.40:0/2318970345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f43d814ae30 con 0x7f43d814c8f0 2024-09-13T18:08:32.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.934+0000 7f43d77fe700 1 --2- 172.21.15.40:0/2318970345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d814c8f0 0x7f43d81f2750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f43c8009710 tx=0x7f43c8005a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:08:32.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.934+0000 7f43d4ff9700 1 -- 172.21.15.40:0/2318970345 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c8023070 con 0x7f43d814c8f0 2024-09-13T18:08:32.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.934+0000 7f43d4ff9700 1 -- 172.21.15.40:0/2318970345 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4937 (secure 0 0 0) 0x7f43c801ee80 con 0x7f43d814c8f0 2024-09-13T18:08:32.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.934+0000 7f43d4ff9700 1 -- 172.21.15.40:0/2318970345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d814c8f0 msgr2=0x7f43d81f2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:32.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.934+0000 7f43d4ff9700 1 --2- 172.21.15.40:0/2318970345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d814c8f0 0x7f43d81f2750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f43c8009710 tx=0x7f43c8005a80 comp rx=0 tx=0).stop 2024-09-13T18:08:32.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.934+0000 7f43d4ff9700 1 -- 172.21.15.40:0/2318970345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d814c8f0 msgr2=0x7f43d81f2750 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:08:32.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.934+0000 7f43d4ff9700 1 --2- 172.21.15.40:0/2318970345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43d814c8f0 0x7f43d81f2750 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.945 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:08:32.945 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:08:32.945 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:08:32.945 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:08:32.945 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 150, 2024-09-13T18:08:32.945 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:08:32.945 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:08:32.945 INFO:teuthology.orchestra.run.smithi040.stdout: 1 2024-09-13T18:08:32.945 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:32.945 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:08:32.946 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:08:32.946 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:08:32.946 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:08:32.946 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:08:32.946 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:08:32.946 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:08:32.946 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:08:32.946 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:08:32.946 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:08:32.946 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:08:32.946 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:08:32.946 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:08:32.946 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:08:32.946 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:32.947 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:08:32.947 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:08:32.947 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:08:32.947 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:08:32.947 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:08:32.947 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:08:32.947 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:08:32.947 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:08:32.947 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:08:32.947 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:08:32.947 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:08:32.947 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:08:32.947 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:32.947 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.948 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:08:32.948 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:08:32.948 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:08:32.948 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:08:32.948 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:08:32.948 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:08:32.948 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:08:32.948 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:08:32.948 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:08:32.948 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:08:32.948 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:08:32.948 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:08:32.948 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:08:32.948 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:08:32.949 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:08:32.949 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:08:32.949 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:08:32.949 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:08:32.949 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:08:32.949 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:08:32.949 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:08:32.949 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:08:32.949 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:08:32.949 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:08:32.949 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:08:32.949 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:08:32.949 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:08:32.949 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:32.950 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:08:32.950 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.950 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:08:32.950 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.950 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:08:32.950 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:08:32.950 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:08:32.950 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:08:32.950 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.950 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:08:32.950 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:08:32.950 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:32.950 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.950 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.951 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:08:32.951 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:08:32.951 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:32.951 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:32.951 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:32.951 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.951 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:08:32.951 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:08:32.951 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:08:32.951 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:08:32.951 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:08:32.951 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.951 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.951 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:08:32.952 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:08:32.952 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:08:32.952 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:08:32.952 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.952 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:08:32.952 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:08:32.952 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:32.952 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.952 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.953 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:08:32.953 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:08:32.953 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:32.953 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:32.953 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:32.953 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.953 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:08:32.953 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:08:32.953 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:08:32.953 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:08:32.953 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:08:32.953 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.953 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.953 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:08:32.954 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:08:32.954 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:08:32.954 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:08:32.954 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.954 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:08:32.954 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:08:32.954 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:32.954 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.954 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.954 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:08:32.954 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:08:32.954 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:08:32.954 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:32.954 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:32.955 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.955 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:08:32.955 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:08:32.955 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:08:32.955 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:08:32.955 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:08:32.955 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:32.955 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:32.955 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.955 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:08:32.955 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:08:32.955 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.955 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:32.955 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:32.955 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:08:32.956 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:32.956 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:32.956 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:08:32.956 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.956 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:32.956 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:32.956 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:08:32.956 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:32.956 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:32.956 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:08:32.956 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.956 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:32.956 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:32.956 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:08:32.957 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:32.957 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:32.957 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:08:32.957 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.957 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:32.957 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:32.957 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:08:32.957 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:32.957 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:08:32.957 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:08:32.957 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:08:32.957 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:08:32.957 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:08:32.957 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:08:32.958 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:08:32.958 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:08:32.958 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:08:32.958 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:08:32.958 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:08:32.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.934+0000 7f43dcb3e700 1 -- 172.21.15.40:0/2318970345 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f43b8066800 msgr2=0x7f43b8068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:32.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.934+0000 7f43dcb3e700 1 --2- 172.21.15.40:0/2318970345 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f43b8066800 0x7f43b8068cc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f43c0001340 tx=0x7f43c00012b0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.934+0000 7f43dcb3e700 1 -- 172.21.15.40:0/2318970345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43d80c5560 msgr2=0x7f43d814b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:08:32.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.934+0000 7f43d7fff700 1 -- 172.21.15.40:0/2318970345 reap_dead start 2024-09-13T18:08:32.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.934+0000 7f43dcb3e700 1 --2- 172.21.15.40:0/2318970345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43d80c5560 0x7f43d814b6c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f43d000b680 tx=0x7f43d0004af0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.934+0000 7f43dcb3e700 1 -- 172.21.15.40:0/2318970345 shutdown_connections 2024-09-13T18:08:32.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.934+0000 7f43dcb3e700 1 --2- 172.21.15.40:0/2318970345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43d80c5560 0x7f43d814b6c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:08:32.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.934+0000 7f43dcb3e700 1 -- 172.21.15.40:0/2318970345 >> 172.21.15.40:0/2318970345 conn(0x7f43d806a390 msgr2=0x7f43d80cb360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:08:32.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.934+0000 7f43dcb3e700 1 -- 172.21.15.40:0/2318970345 shutdown_connections 2024-09-13T18:08:32.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:08:32.934+0000 7f43dcb3e700 1 -- 172.21.15.40:0/2318970345 wait complete. 2024-09-13T18:08:32.961 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:08:32.961 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:08:36.969 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:08:36.972 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:08:52.963 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-13T18:08:52.963 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-13T18:08:52.963 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T18:08:52.964 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T18:08:52.967 INFO:tasks.ceph.mon.c:Started 2024-09-13T18:08:52.967 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-09-13T18:08:52.967 INFO:tasks.ceph.mon.a:Sent signal 19 2024-09-13T18:08:52.967 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-13T18:09:07.024 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:09:07.027 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:09:07.968 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-09-13T18:09:07.969 INFO:tasks.ceph.mon.a:Sent signal 18 2024-09-13T18:09:07.969 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:09:07.969 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:09:08.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.073+0000 7f7a99a09700 1 Processor -- start 2024-09-13T18:09:08.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.073+0000 7f7a99a09700 1 -- start start 2024-09-13T18:09:08.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.073+0000 7f7a99a09700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7a9414ffd0 0x7f7a940724a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.073+0000 7f7a99a09700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7a940c7f70 0x7f7a9415cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.073+0000 7f7a99a09700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a94159360 con 0x7f7a941503b0 2024-09-13T18:09:08.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.073+0000 7f7a99a09700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9415a2a0 con 0x7f7a940c7f70 2024-09-13T18:09:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.073+0000 7f7a99a09700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a94070fe0 con 0x7f7a9414ffd0 2024-09-13T18:09:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.073+0000 7f7a99208700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7a940c7f70 0x7f7a9415cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.073+0000 7f7a99208700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7a940c7f70 0x7f7a9415cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59392/0 (socket says 172.21.15.40:59392) 2024-09-13T18:09:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.073+0000 7f7a99208700 1 -- 172.21.15.40:0/834420053 learned_addr learned my addr 172.21.15.40:0/834420053 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.073+0000 7f7a98a07700 1 --2- 172.21.15.40:0/834420053 >> v2:172.21.15.43:3300/0 conn(0x7f7a9414ffd0 0x7f7a940724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.073+0000 7f7a99208700 1 -- 172.21.15.40:0/834420053 >> v2:172.21.15.43:3300/0 conn(0x7f7a9414ffd0 msgr2=0x7f7a940724a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:09:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.073+0000 7f7a99208700 1 --2- 172.21.15.40:0/834420053 >> v2:172.21.15.43:3300/0 conn(0x7f7a9414ffd0 0x7f7a940724a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.073+0000 7f7a99208700 1 -- 172.21.15.40:0/834420053 >> v1:172.21.15.43:6789/0 conn(0x7f7a941503b0 legacy=0x7f7a9406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.073+0000 7f7a99208700 1 -- 172.21.15.40:0/834420053 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a94159620 con 0x7f7a940c7f70 2024-09-13T18:09:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.073+0000 7f7a99208700 1 --2- 172.21.15.40:0/834420053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a940c7f70 0x7f7a9415cb00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7a9406ef00 tx=0x7f7a8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e12f1ff636c483 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.073+0000 7f7a937fe700 1 -- 172.21.15.40:0/834420053 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c00e070 con 0x7f7a940c7f70 2024-09-13T18:09:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.073+0000 7f7a937fe700 1 -- 172.21.15.40:0/834420053 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8c00b910 con 0x7f7a940c7f70 2024-09-13T18:09:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.073+0000 7f7a937fe700 1 -- 172.21.15.40:0/834420053 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c00bcc0 con 0x7f7a940c7f70 2024-09-13T18:09:08.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99a09700 1 -- 172.21.15.40:0/834420053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a940c7f70 msgr2=0x7f7a9415cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99a09700 1 --2- 172.21.15.40:0/834420053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a940c7f70 0x7f7a9415cb00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7a9406ef00 tx=0x7f7a8c00b330 comp rx=0 tx=0).stop 2024-09-13T18:09:08.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99a09700 1 -- 172.21.15.40:0/834420053 shutdown_connections 2024-09-13T18:09:08.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99a09700 1 --2- 172.21.15.40:0/834420053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a940c7f70 0x7f7a9415cb00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99a09700 1 --2- 172.21.15.40:0/834420053 >> v2:172.21.15.43:3300/0 conn(0x7f7a9414ffd0 0x7f7a940724a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99a09700 1 -- 172.21.15.40:0/834420053 >> 172.21.15.40:0/834420053 conn(0x7f7a940cbd80 msgr2=0x7f7a940cc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:08.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99a09700 1 -- 172.21.15.40:0/834420053 shutdown_connections 2024-09-13T18:09:08.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99a09700 1 -- 172.21.15.40:0/834420053 wait complete. 2024-09-13T18:09:08.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99a09700 1 Processor -- start 2024-09-13T18:09:08.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99a09700 1 -- start start 2024-09-13T18:09:08.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99a09700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a940c7f70 0x7f7a94149590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99a09700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7a9414ffd0 0x7f7a9414b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99a09700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a941503b0 0x7f7a9414bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99a09700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a94156de0 con 0x7f7a941503b0 2024-09-13T18:09:08.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99a09700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9414ab80 con 0x7f7a940c7f70 2024-09-13T18:09:08.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99a09700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a94156b00 con 0x7f7a9414ffd0 2024-09-13T18:09:08.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99208700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a941503b0 0x7f7a9414bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99208700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a941503b0 0x7f7a9414bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59398/0 (socket says 172.21.15.40:59398) 2024-09-13T18:09:08.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99208700 1 -- 172.21.15.40:0/2657417351 learned_addr learned my addr 172.21.15.40:0/2657417351 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:08.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a98a07700 1 --2- 172.21.15.40:0/2657417351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a940c7f70 0x7f7a94149590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a93fff700 1 --2- 172.21.15.40:0/2657417351 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7a9414ffd0 0x7f7a9414b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99208700 1 -- 172.21.15.40:0/2657417351 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7a9414ffd0 msgr2=0x7f7a9414b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99208700 1 --2- 172.21.15.40:0/2657417351 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7a9414ffd0 0x7f7a9414b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99208700 1 -- 172.21.15.40:0/2657417351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a940c7f70 msgr2=0x7f7a94149590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99208700 1 --2- 172.21.15.40:0/2657417351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a940c7f70 0x7f7a94149590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99208700 1 -- 172.21.15.40:0/2657417351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8c009b30 con 0x7f7a941503b0 2024-09-13T18:09:08.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a93fff700 1 --2- 172.21.15.40:0/2657417351 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7a9414ffd0 0x7f7a9414b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:09:08.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.077+0000 7f7a99208700 1 --2- 172.21.15.40:0/2657417351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a941503b0 0x7f7a9414bd40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7a940c2f70 tx=0x7f7a8c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:08.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.081+0000 7f7a917fa700 1 -- 172.21.15.40:0/2657417351 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c004930 con 0x7f7a941503b0 2024-09-13T18:09:08.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.081+0000 7f7a917fa700 1 -- 172.21.15.40:0/2657417351 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8c0092e0 con 0x7f7a941503b0 2024-09-13T18:09:08.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.081+0000 7f7a99a09700 1 -- 172.21.15.40:0/2657417351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a9414a230 con 0x7f7a941503b0 2024-09-13T18:09:08.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.081+0000 7f7a917fa700 1 -- 172.21.15.40:0/2657417351 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c01c660 con 0x7f7a941503b0 2024-09-13T18:09:08.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.081+0000 7f7a99a09700 1 -- 172.21.15.40:0/2657417351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a9414a810 con 0x7f7a941503b0 2024-09-13T18:09:08.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.081+0000 7f7a99a09700 1 -- 172.21.15.40:0/2657417351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a9414acf0 con 0x7f7a941503b0 2024-09-13T18:09:08.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.081+0000 7f7a917fa700 1 -- 172.21.15.40:0/2657417351 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7a8c01c7c0 con 0x7f7a941503b0 2024-09-13T18:09:08.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.081+0000 7f7a917fa700 1 --2- 172.21.15.40:0/2657417351 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7a78066760 0x7f7a78068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.081+0000 7f7a917fa700 1 -- 172.21.15.40:0/2657417351 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7a8c085ea0 con 0x7f7a941503b0 2024-09-13T18:09:08.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.081+0000 7f7a98a07700 1 --2- 172.21.15.40:0/2657417351 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7a78066760 0x7f7a78068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.081+0000 7f7a98a07700 1 --2- 172.21.15.40:0/2657417351 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7a78066760 0x7f7a78068c20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009bf0 tx=0x7f7a88009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:08.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.089+0000 7f7a917fa700 1 -- 172.21.15.40:0/2657417351 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7a8c052ee0 con 0x7f7a941503b0 2024-09-13T18:09:08.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.377+0000 7f7a99a09700 1 -- 172.21.15.40:0/2657417351 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f7a940724a0 con 0x7f7a941503b0 2024-09-13T18:09:08.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.377+0000 7f7a917fa700 1 -- 172.21.15.40:0/2657417351 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f7a8c056500 con 0x7f7a941503b0 2024-09-13T18:09:08.389 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":152,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:09:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.377+0000 7f7a99a09700 1 -- 172.21.15.40:0/2657417351 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7a78066760 msgr2=0x7f7a78068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.377+0000 7f7a99a09700 1 --2- 172.21.15.40:0/2657417351 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7a78066760 0x7f7a78068c20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009bf0 tx=0x7f7a88009240 comp rx=0 tx=0).stop 2024-09-13T18:09:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.377+0000 7f7a99a09700 1 -- 172.21.15.40:0/2657417351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a941503b0 msgr2=0x7f7a9414bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.377+0000 7f7a99a09700 1 --2- 172.21.15.40:0/2657417351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a941503b0 0x7f7a9414bd40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7a940c2f70 tx=0x7f7a8c004af0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.377+0000 7f7a99a09700 1 -- 172.21.15.40:0/2657417351 shutdown_connections 2024-09-13T18:09:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.377+0000 7f7a99a09700 1 --2- 172.21.15.40:0/2657417351 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a941503b0 0x7f7a9414bd40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.377+0000 7f7a99a09700 1 --2- 172.21.15.40:0/2657417351 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7a78066760 0x7f7a78068c20 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.377+0000 7f7a99a09700 1 --2- 172.21.15.40:0/2657417351 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7a9414ffd0 0x7f7a9414b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.377+0000 7f7a99a09700 1 --2- 172.21.15.40:0/2657417351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a940c7f70 0x7f7a94149590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.377+0000 7f7a99a09700 1 -- 172.21.15.40:0/2657417351 >> 172.21.15.40:0/2657417351 conn(0x7f7a940cbd80 msgr2=0x7f7a940cc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:08.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.377+0000 7f7a99a09700 1 -- 172.21.15.40:0/2657417351 shutdown_connections 2024-09-13T18:09:08.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.377+0000 7f7a99a09700 1 -- 172.21.15.40:0/2657417351 wait complete. 2024-09-13T18:09:08.402 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:09:08.403 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:09:08.403 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:09:08.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.505+0000 7fcd04957700 1 Processor -- start 2024-09-13T18:09:08.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.505+0000 7fcd04957700 1 -- start start 2024-09-13T18:09:08.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.509+0000 7fcd04957700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcd00156a90 0x7fcd0006e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.509+0000 7fcd04957700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd00066460 con 0x7fcd00150900 2024-09-13T18:09:08.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.509+0000 7fcd04957700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd00066130 con 0x7fcd00150520 2024-09-13T18:09:08.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.509+0000 7fcd04957700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd000c3830 con 0x7fcd00156a90 2024-09-13T18:09:08.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.509+0000 7fccfffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcd00156a90 0x7fcd0006e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.509+0000 7fccfffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcd00156a90 0x7fcd0006e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59412/0 (socket says 172.21.15.40:59412) 2024-09-13T18:09:08.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.509+0000 7fccff7fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcd00150520 0x7fcd000c4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:34798/0 (socket says 172.21.15.40:34798) 2024-09-13T18:09:08.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.509+0000 7fccfffff700 1 -- 172.21.15.40:0/3324885026 learned_addr learned my addr 172.21.15.40:0/3324885026 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:08.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.509+0000 7fccfffff700 1 -- 172.21.15.40:0/3324885026 >> v1:172.21.15.43:6789/0 conn(0x7fcd00150520 legacy=0x7fcd000c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.509+0000 7fccfffff700 1 -- 172.21.15.40:0/3324885026 >> v1:172.21.15.40:6789/0 conn(0x7fcd00150900 legacy=0x7fcd00156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.509+0000 7fccfffff700 1 -- 172.21.15.40:0/3324885026 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd00070e80 con 0x7fcd00156a90 2024-09-13T18:09:08.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.509+0000 7fccfffff700 1 --2- 172.21.15.40:0/3324885026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00156a90 0x7fcd0006e860 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fccf400bbb0 tx=0x7fccf400bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a88544c4e8d9b9c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:08.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.509+0000 7fccfe7fc700 1 -- 172.21.15.40:0/3324885026 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf400dd90 con 0x7fcd00156a90 2024-09-13T18:09:08.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.509+0000 7fccfe7fc700 1 -- 172.21.15.40:0/3324885026 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf4010470 con 0x7fcd00156a90 2024-09-13T18:09:08.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.509+0000 7fccfe7fc700 1 -- 172.21.15.40:0/3324885026 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf4010820 con 0x7fcd00156a90 2024-09-13T18:09:08.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.509+0000 7fcd04957700 1 -- 172.21.15.40:0/3324885026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00156a90 msgr2=0x7fcd0006e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.509+0000 7fcd04957700 1 --2- 172.21.15.40:0/3324885026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00156a90 0x7fcd0006e860 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fccf400bbb0 tx=0x7fccf400bec0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.513+0000 7fcd04957700 1 -- 172.21.15.40:0/3324885026 shutdown_connections 2024-09-13T18:09:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.513+0000 7fcd04957700 1 --2- 172.21.15.40:0/3324885026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00156a90 0x7fcd0006e860 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.513+0000 7fcd04957700 1 -- 172.21.15.40:0/3324885026 >> 172.21.15.40:0/3324885026 conn(0x7fcd000cb450 msgr2=0x7fcd000cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.513+0000 7fcd04957700 1 -- 172.21.15.40:0/3324885026 shutdown_connections 2024-09-13T18:09:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.513+0000 7fcd04957700 1 -- 172.21.15.40:0/3324885026 wait complete. 2024-09-13T18:09:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.513+0000 7fcd04957700 1 Processor -- start 2024-09-13T18:09:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.513+0000 7fcd04957700 1 -- start start 2024-09-13T18:09:08.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.513+0000 7fcd04957700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcd00150520 0x7fcd001ef500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.513+0000 7fcd04957700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00156a90 0x7fcd00140fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.513+0000 7fcd04957700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd00150cf0 0x7fcd00140790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.513+0000 7fcd04957700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd00145790 con 0x7fcd00156a90 2024-09-13T18:09:08.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.513+0000 7fcd04957700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd001eeaf0 con 0x7fcd00150cf0 2024-09-13T18:09:08.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.513+0000 7fcd04957700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd00154cc0 con 0x7fcd00150520 2024-09-13T18:09:08.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.513+0000 7fccfeffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00156a90 0x7fcd00140fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.513+0000 7fccfffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd00150cf0 0x7fcd00140790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.513+0000 7fccfeffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00156a90 0x7fcd00140fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59420/0 (socket says 172.21.15.40:59420) 2024-09-13T18:09:08.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.513+0000 7fccfeffd700 1 -- 172.21.15.40:0/1666701168 learned_addr learned my addr 172.21.15.40:0/1666701168 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:08.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fccff7fe700 1 --2- 172.21.15.40:0/1666701168 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcd00150520 0x7fcd001ef500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fccfeffd700 1 -- 172.21.15.40:0/1666701168 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcd00150520 msgr2=0x7fcd001ef500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fccfeffd700 1 --2- 172.21.15.40:0/1666701168 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcd00150520 0x7fcd001ef500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fccfeffd700 1 -- 172.21.15.40:0/1666701168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd00150cf0 msgr2=0x7fcd00140790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fccfeffd700 1 --2- 172.21.15.40:0/1666701168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd00150cf0 0x7fcd00140790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fccfeffd700 1 -- 172.21.15.40:0/1666701168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf400b810 con 0x7fcd00156a90 2024-09-13T18:09:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fccff7fe700 1 --2- 172.21.15.40:0/1666701168 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcd00150520 0x7fcd001ef500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:09:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fccfeffd700 1 --2- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00156a90 0x7fcd00140fb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fccf000c500 tx=0x7fccf000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fcce7fff700 1 -- 172.21.15.40:0/1666701168 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf00052d0 con 0x7fcd00156a90 2024-09-13T18:09:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fcce7fff700 1 -- 172.21.15.40:0/1666701168 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf0005430 con 0x7fcd00156a90 2024-09-13T18:09:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fcce7fff700 1 -- 172.21.15.40:0/1666701168 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf00055a0 con 0x7fcd00156a90 2024-09-13T18:09:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fcd04957700 1 -- 172.21.15.40:0/1666701168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd001efa40 con 0x7fcd00156a90 2024-09-13T18:09:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fcd04957700 1 -- 172.21.15.40:0/1666701168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd00154420 con 0x7fcd00156a90 2024-09-13T18:09:08.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fcd04957700 1 --2- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00150900 0x7fcd001eec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fcd04957700 1 -- 172.21.15.40:0/1666701168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd000675d0 con 0x7fcd00150900 2024-09-13T18:09:08.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fccff7fe700 1 --2- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00150900 0x7fcd001eec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fccff7fe700 1 -- 172.21.15.40:0/1666701168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcd000723a0 con 0x7fcd00150900 2024-09-13T18:09:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fcce7fff700 1 -- 172.21.15.40:0/1666701168 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fccf00038e0 con 0x7fcd00156a90 2024-09-13T18:09:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fccff7fe700 1 --2- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00150900 0x7fcd001eec60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fccf800e390 tx=0x7fccf800e750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fcce7fff700 1 --2- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fccec066730 0x7fccec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fcce7fff700 1 -- 172.21.15.40:0/1666701168 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fccf0083970 con 0x7fcd00156a90 2024-09-13T18:09:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fccfffff700 1 --2- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fccec066730 0x7fccec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.517+0000 7fcce7fff700 1 -- 172.21.15.40:0/1666701168 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf8013420 con 0x7fcd00150900 2024-09-13T18:09:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.521+0000 7fcce7fff700 1 -- 172.21.15.40:0/1666701168 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fccf8013580 con 0x7fcd00150900 2024-09-13T18:09:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.521+0000 7fcce7fff700 1 -- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00150900 msgr2=0x7fcd001eec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.521+0000 7fcce7fff700 1 --2- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00150900 0x7fcd001eec60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fccf800e390 tx=0x7fccf800e750 comp rx=0 tx=0).stop 2024-09-13T18:09:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.521+0000 7fcce7fff700 1 -- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00150900 msgr2=0x7fcd001eec60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:09:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.521+0000 7fcce7fff700 1 --2- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00150900 0x7fcd001eec60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.521+0000 7fccfffff700 1 --2- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fccec066730 0x7fccec068bf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fccf400bf00 tx=0x7fccf401b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:08.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.529+0000 7fcd04957700 1 --2- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00153f60 0x7fcd000ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.529+0000 7fcd04957700 1 -- 172.21.15.40:0/1666701168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd00152830 con 0x7fcd00153f60 2024-09-13T18:09:08.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.529+0000 7fccff7fe700 1 --2- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00153f60 0x7fcd000ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.533+0000 7fccff7fe700 1 -- 172.21.15.40:0/1666701168 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcd0006ed30 con 0x7fcd00153f60 2024-09-13T18:09:08.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.533+0000 7fccff7fe700 1 --2- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00153f60 0x7fcd000ca4b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fccf800e390 tx=0x7fccf80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:08.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.533+0000 7fcce7fff700 1 -- 172.21.15.40:0/1666701168 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf8013db0 con 0x7fcd00153f60 2024-09-13T18:09:08.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.533+0000 7fcce7fff700 1 -- 172.21.15.40:0/1666701168 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4950 (secure 0 0 0) 0x7fccf8028400 con 0x7fcd00153f60 2024-09-13T18:09:08.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.533+0000 7fcce7fff700 1 -- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00153f60 msgr2=0x7fcd000ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.533+0000 7fcce7fff700 1 --2- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00153f60 0x7fcd000ca4b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fccf800e390 tx=0x7fccf80014a0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.533+0000 7fcce7fff700 1 -- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00153f60 msgr2=0x7fcd000ca4b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:09:08.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.533+0000 7fcce7fff700 1 --2- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00153f60 0x7fcd000ca4b0 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fccf800e390 tx=0x7fccf80014a0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.545 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:09:08.545 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:09:08.545 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:09:08.545 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:09:08.545 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 152, 2024-09-13T18:09:08.545 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:09:08.545 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:09:08.546 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:09:08.546 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:09:08.546 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:08.546 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:09:08.546 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:09:08.546 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:09:08.546 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:09:08.546 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:08.546 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:08.546 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:08.547 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:08.547 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:08.547 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:08.547 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:08.547 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:08.547 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:08.547 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:08.547 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:08.547 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:09:08.547 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:09:08.547 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:08.548 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:08.548 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:08.548 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:08.548 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:08.548 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:08.548 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:08.548 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:08.548 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:08.548 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:08.548 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:08.548 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.548 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:09:08.548 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:09:08.549 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:09:08.549 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:09:08.549 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:09:08.549 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:09:08.549 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:09:08.549 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:09:08.549 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:09:08.549 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:09:08.549 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:09:08.549 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:09:08.550 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:09:08.550 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:09:08.550 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:09:08.550 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:09:08.550 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:09:08.550 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:08.550 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:08.550 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:08.550 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:08.550 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:08.550 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:08.550 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:08.550 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:08.550 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:08.551 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:08.551 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:08.551 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:09:08.551 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.551 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:09:08.551 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.551 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:09:08.551 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:09:08.551 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:08.551 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:08.551 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.551 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:08.551 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:09:08.552 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:08.552 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.552 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.552 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:08.552 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:09:08.552 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:08.552 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.552 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:08.552 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.552 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:09:08.553 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:09:08.553 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:08.553 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:08.553 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:08.553 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.553 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.553 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:09:08.553 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:09:08.553 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:08.553 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:08.553 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.554 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:08.554 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:09:08.554 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:08.554 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.554 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.554 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:08.554 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:09:08.554 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:08.554 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.554 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:08.554 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.554 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:09:08.554 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:09:08.554 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:08.555 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:08.555 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:08.555 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.555 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.555 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:09:08.555 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:09:08.555 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:08.555 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:08.555 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.556 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:08.556 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:09:08.556 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:08.556 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.556 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.556 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:08.557 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:09:08.557 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:08.557 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.557 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:08.557 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.557 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:09:08.557 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:09:08.557 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:09:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:09:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:09:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:09:08.558 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.559 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:08.559 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:08.559 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:09:08.559 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.559 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:08.559 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:09:08.559 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.559 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:08.559 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:08.559 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:09:08.559 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.559 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:08.560 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:09:08.560 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.560 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:08.560 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:08.560 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:09:08.560 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.560 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:08.560 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:09:08.560 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.560 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:08.560 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:08.560 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:09:08.560 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.560 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:08.561 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.561 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:09:08.561 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:09:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.533+0000 7fcd04957700 1 -- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fccec066730 msgr2=0x7fccec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.533+0000 7fcd04957700 1 --2- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fccec066730 0x7fccec068bf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fccf400bf00 tx=0x7fccf401b040 comp rx=0 tx=0).stop 2024-09-13T18:09:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.533+0000 7fcd04957700 1 -- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00156a90 msgr2=0x7fcd00140fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.533+0000 7fcd04957700 1 --2- 172.21.15.40:0/1666701168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd00156a90 0x7fcd00140fb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fccf000c500 tx=0x7fccf000c810 comp rx=0 tx=0).stop 2024-09-13T18:09:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.533+0000 7fccfffff700 1 -- 172.21.15.40:0/1666701168 reap_dead start 2024-09-13T18:09:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.533+0000 7fccfffff700 1 -- 172.21.15.40:0/1666701168 reap_dead start 2024-09-13T18:09:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.533+0000 7fcd04957700 1 -- 172.21.15.40:0/1666701168 shutdown_connections 2024-09-13T18:09:08.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.533+0000 7fcd04957700 1 -- 172.21.15.40:0/1666701168 >> 172.21.15.40:0/1666701168 conn(0x7fcd000cb450 msgr2=0x7fcd000c5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:08.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.533+0000 7fcd04957700 1 -- 172.21.15.40:0/1666701168 shutdown_connections 2024-09-13T18:09:08.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.533+0000 7fcd04957700 1 -- 172.21.15.40:0/1666701168 wait complete. 2024-09-13T18:09:08.563 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:09:08.563 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:09:08.563 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:09:08.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.669+0000 7f3ac55ec700 1 Processor -- start 2024-09-13T18:09:08.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.669+0000 7f3ac55ec700 1 -- start start 2024-09-13T18:09:08.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.669+0000 7f3ac55ec700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3ab80d8b30 0x7f3ab80df0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.669+0000 7f3ac55ec700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3ab8045cf0 0x7f3ab804b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.669+0000 7f3ac55ec700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab80df620 con 0x7f3ab80460f0 2024-09-13T18:09:08.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.669+0000 7f3ac55ec700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab80087d0 con 0x7f3ab80d8b30 2024-09-13T18:09:08.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.669+0000 7f3ac55ec700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab8007860 con 0x7f3ab8045cf0 2024-09-13T18:09:08.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.669+0000 7f3abffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3ab80d8b30 0x7f3ab80df0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.669+0000 7f3abffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3ab80d8b30 0x7f3ab80df0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59454/0 (socket says 172.21.15.40:59454) 2024-09-13T18:09:08.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.669+0000 7f3abffff700 1 -- 172.21.15.40:0/3920201057 learned_addr learned my addr 172.21.15.40:0/3920201057 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:08.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.669+0000 7f3abf7fe700 1 --2- 172.21.15.40:0/3920201057 >> v2:172.21.15.43:3300/0 conn(0x7f3ab8045cf0 0x7f3ab804b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.669+0000 7f3abffff700 1 -- 172.21.15.40:0/3920201057 >> v2:172.21.15.43:3300/0 conn(0x7f3ab8045cf0 msgr2=0x7f3ab804b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:09:08.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.669+0000 7f3abffff700 1 --2- 172.21.15.40:0/3920201057 >> v2:172.21.15.43:3300/0 conn(0x7f3ab8045cf0 0x7f3ab804b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.669+0000 7f3abffff700 1 -- 172.21.15.40:0/3920201057 >> v1:172.21.15.40:6789/0 conn(0x7f3ab80460f0 legacy=0x7f3ab8046990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.669+0000 7f3abffff700 1 -- 172.21.15.40:0/3920201057 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab80465a0 con 0x7f3ab80d8b30 2024-09-13T18:09:08.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.669+0000 7f3abffff700 1 --2- 172.21.15.40:0/3920201057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab80d8b30 0x7f3ab80df0e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800e4c0 tx=0x7f3ab4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ffe99bca845a11c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:08.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.669+0000 7f3abeffd700 1 -- 172.21.15.40:0/3920201057 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4011830 con 0x7f3ab80d8b30 2024-09-13T18:09:08.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.669+0000 7f3abeffd700 1 -- 172.21.15.40:0/3920201057 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab4011990 con 0x7f3ab80d8b30 2024-09-13T18:09:08.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.669+0000 7f3abeffd700 1 -- 172.21.15.40:0/3920201057 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4011d20 con 0x7f3ab80d8b30 2024-09-13T18:09:08.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.673+0000 7f3ac55ec700 1 -- 172.21.15.40:0/3920201057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab80d8b30 msgr2=0x7f3ab80df0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.673+0000 7f3ac55ec700 1 --2- 172.21.15.40:0/3920201057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab80d8b30 0x7f3ab80df0e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800e4c0 tx=0x7f3ab4009bf0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.673+0000 7f3ac55ec700 1 -- 172.21.15.40:0/3920201057 shutdown_connections 2024-09-13T18:09:08.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.673+0000 7f3ac55ec700 1 --2- 172.21.15.40:0/3920201057 >> v2:172.21.15.43:3300/0 conn(0x7f3ab8045cf0 0x7f3ab804b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.673+0000 7f3ac55ec700 1 --2- 172.21.15.40:0/3920201057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab80d8b30 0x7f3ab80df0e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.673+0000 7f3ac55ec700 1 -- 172.21.15.40:0/3920201057 >> 172.21.15.40:0/3920201057 conn(0x7f3ab8050e10 msgr2=0x7f3ab8051220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:08.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.673+0000 7f3ac55ec700 1 -- 172.21.15.40:0/3920201057 shutdown_connections 2024-09-13T18:09:08.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.673+0000 7f3ac55ec700 1 -- 172.21.15.40:0/3920201057 wait complete. 2024-09-13T18:09:08.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.673+0000 7f3ac55ec700 1 Processor -- start 2024-09-13T18:09:08.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.673+0000 7f3ac55ec700 1 -- start start 2024-09-13T18:09:08.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3ac55ec700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ab8045cf0 0x7f3ab80da180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3ac55ec700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab80460f0 0x7f3ab80da6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3ac55ec700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab80dd3a0 0x7f3ab80d3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3ac55ec700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80cebf0 con 0x7f3ab80460f0 2024-09-13T18:09:08.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3ac55ec700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80db860 con 0x7f3ab80dd3a0 2024-09-13T18:09:08.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3ac55ec700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80dbb40 con 0x7f3ab8045cf0 2024-09-13T18:09:08.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3abf7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab80460f0 0x7f3ab80da6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3abf7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab80460f0 0x7f3ab80da6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59470/0 (socket says 172.21.15.40:59470) 2024-09-13T18:09:08.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3abf7fe700 1 -- 172.21.15.40:0/229967104 learned_addr learned my addr 172.21.15.40:0/229967104 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:08.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3abffff700 1 --2- 172.21.15.40:0/229967104 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ab8045cf0 0x7f3ab80da180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3ac4deb700 1 --2- 172.21.15.40:0/229967104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab80dd3a0 0x7f3ab80d3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3abf7fe700 1 -- 172.21.15.40:0/229967104 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ab8045cf0 msgr2=0x7f3ab80da180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3abf7fe700 1 --2- 172.21.15.40:0/229967104 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ab8045cf0 0x7f3ab80da180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3abf7fe700 1 -- 172.21.15.40:0/229967104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab80dd3a0 msgr2=0x7f3ab80d3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3abf7fe700 1 --2- 172.21.15.40:0/229967104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab80dd3a0 0x7f3ab80d3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3abf7fe700 1 -- 172.21.15.40:0/229967104 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab4009570 con 0x7f3ab80460f0 2024-09-13T18:09:08.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3abf7fe700 1 --2- 172.21.15.40:0/229967104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab80460f0 0x7f3ab80da6c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00e9d0 tx=0x7f3aac00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:08.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3abcff9700 1 -- 172.21.15.40:0/229967104 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac009720 con 0x7f3ab80460f0 2024-09-13T18:09:08.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3abcff9700 1 -- 172.21.15.40:0/229967104 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aac00ee70 con 0x7f3ab80460f0 2024-09-13T18:09:08.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3ac55ec700 1 -- 172.21.15.40:0/229967104 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab80dbe20 con 0x7f3ab80460f0 2024-09-13T18:09:08.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3abcff9700 1 -- 172.21.15.40:0/229967104 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac00ca50 con 0x7f3ab80460f0 2024-09-13T18:09:08.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3ac55ec700 1 -- 172.21.15.40:0/229967104 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab80dc1b0 con 0x7f3ab80460f0 2024-09-13T18:09:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3ac55ec700 1 --2- 172.21.15.40:0/229967104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab80d4320 0x7f3ab817c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3ac55ec700 1 -- 172.21.15.40:0/229967104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80095e0 con 0x7f3ab80d4320 2024-09-13T18:09:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3abffff700 1 --2- 172.21.15.40:0/229967104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab80d4320 0x7f3ab817c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.677+0000 7f3abffff700 1 -- 172.21.15.40:0/229967104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3ab80cf240 con 0x7f3ab80d4320 2024-09-13T18:09:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.681+0000 7f3abffff700 1 --2- 172.21.15.40:0/229967104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab80d4320 0x7f3ab817c6e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800e4c0 tx=0x7f3ab4004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.681+0000 7f3abcff9700 1 -- 172.21.15.40:0/229967104 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3aac00cbb0 con 0x7f3ab80460f0 2024-09-13T18:09:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.681+0000 7f3abcff9700 1 --2- 172.21.15.40:0/229967104 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3aa4066760 0x7f3aa4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.681+0000 7f3abcff9700 1 -- 172.21.15.40:0/229967104 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3aac0844c0 con 0x7f3ab80460f0 2024-09-13T18:09:08.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.681+0000 7f3ac4deb700 1 --2- 172.21.15.40:0/229967104 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3aa4066760 0x7f3aa4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.681+0000 7f3abcff9700 1 -- 172.21.15.40:0/229967104 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4020070 con 0x7f3ab80d4320 2024-09-13T18:09:08.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.681+0000 7f3abcff9700 1 -- 172.21.15.40:0/229967104 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3ab4004380 con 0x7f3ab80d4320 2024-09-13T18:09:08.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.681+0000 7f3abcff9700 1 -- 172.21.15.40:0/229967104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab80d4320 msgr2=0x7f3ab817c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.681+0000 7f3abcff9700 1 --2- 172.21.15.40:0/229967104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab80d4320 0x7f3ab817c6e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800e4c0 tx=0x7f3ab4004770 comp rx=0 tx=0).stop 2024-09-13T18:09:08.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.681+0000 7f3abcff9700 1 -- 172.21.15.40:0/229967104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab80d4320 msgr2=0x7f3ab817c6e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:09:08.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.681+0000 7f3abcff9700 1 --2- 172.21.15.40:0/229967104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab80d4320 0x7f3ab817c6e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.681+0000 7f3ac4deb700 1 --2- 172.21.15.40:0/229967104 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3aa4066760 0x7f3aa4068c20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0063cf0 tx=0x7f3ac0065ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:08.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.693+0000 7f3ac55ec700 1 --2- 172.21.15.40:0/229967104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab817cc20 0x7f3ab8178f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.693+0000 7f3ac55ec700 1 -- 172.21.15.40:0/229967104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab8007210 con 0x7f3ab817cc20 2024-09-13T18:09:08.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.693+0000 7f3abffff700 1 --2- 172.21.15.40:0/229967104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab817cc20 0x7f3ab8178f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.693+0000 7f3abffff700 1 -- 172.21.15.40:0/229967104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3ab8006f30 con 0x7f3ab817cc20 2024-09-13T18:09:08.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.693+0000 7f3abffff700 1 --2- 172.21.15.40:0/229967104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab817cc20 0x7f3ab8178f00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009c30 tx=0x7f3ab4011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:08.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.693+0000 7f3abcff9700 1 -- 172.21.15.40:0/229967104 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab402a830 con 0x7f3ab817cc20 2024-09-13T18:09:08.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.693+0000 7f3abcff9700 1 -- 172.21.15.40:0/229967104 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7f3ab402a990 con 0x7f3ab817cc20 2024-09-13T18:09:08.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.693+0000 7f3abcff9700 1 -- 172.21.15.40:0/229967104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab817cc20 msgr2=0x7f3ab8178f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.693+0000 7f3abcff9700 1 --2- 172.21.15.40:0/229967104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab817cc20 0x7f3ab8178f00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009c30 tx=0x7f3ab4011000 comp rx=0 tx=0).stop 2024-09-13T18:09:08.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.693+0000 7f3abcff9700 1 -- 172.21.15.40:0/229967104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab817cc20 msgr2=0x7f3ab8178f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:09:08.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.693+0000 7f3abcff9700 1 --2- 172.21.15.40:0/229967104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab817cc20 0x7f3ab8178f00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.708 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:09:08.709 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:09:08.709 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:09:08.709 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:09:08.709 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 152, 2024-09-13T18:09:08.709 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:09:08.710 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:09:08.710 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:09:08.710 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:09:08.710 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:08.710 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:09:08.711 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:09:08.711 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:09:08.711 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:09:08.711 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:08.711 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:08.711 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:08.712 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:08.712 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:08.712 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:08.712 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:08.712 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:08.713 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:08.713 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:08.713 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:08.713 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:09:08.713 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:09:08.714 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:08.714 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:08.714 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:08.714 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:08.714 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:08.714 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:08.715 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:08.715 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:08.715 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:08.715 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:08.715 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:08.716 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.716 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:09:08.716 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:09:08.716 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:09:08.716 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:09:08.716 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:09:08.717 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:09:08.717 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:09:08.717 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:09:08.717 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:09:08.717 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:09:08.718 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:09:08.718 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:09:08.718 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:09:08.718 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:09:08.718 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:09:08.719 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:09:08.719 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:09:08.719 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:08.719 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:08.719 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:08.719 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:08.720 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:08.720 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:08.720 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:08.720 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:08.720 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:08.721 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:08.721 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:08.721 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:09:08.721 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.721 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:09:08.722 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.722 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:09:08.722 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:09:08.722 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:08.722 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:08.723 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.723 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:08.723 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:09:08.723 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:08.723 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.723 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.724 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:08.724 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:09:08.724 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:08.724 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.724 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:08.724 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.725 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:09:08.725 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:09:08.725 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:08.725 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:08.725 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:08.725 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.725 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.725 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:09:08.725 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:09:08.725 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:08.725 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:08.725 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.725 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:08.726 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:09:08.726 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:08.726 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.726 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.726 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:08.726 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:09:08.726 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:08.726 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.726 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:08.726 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.726 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:09:08.726 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:09:08.727 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:08.727 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:08.727 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:08.727 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.727 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.727 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:09:08.727 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:09:08.727 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:08.727 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:08.727 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.727 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:08.727 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:09:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:09:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:09:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:09:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:08.728 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:08.729 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:08.729 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.729 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:08.729 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.729 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:09:08.729 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:09:08.729 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.729 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:08.729 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:08.729 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:09:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:09:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:09:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:09:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:09:08.730 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:09:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:09:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:09:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:09:08.731 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.732 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:08.732 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.732 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:09:08.732 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:09:08.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.693+0000 7f3ac55ec700 1 -- 172.21.15.40:0/229967104 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3aa4066760 msgr2=0x7f3aa4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.693+0000 7f3ac55ec700 1 --2- 172.21.15.40:0/229967104 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3aa4066760 0x7f3aa4068c20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0063cf0 tx=0x7f3ac0065ea0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.693+0000 7f3ac55ec700 1 -- 172.21.15.40:0/229967104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab80460f0 msgr2=0x7f3ab80da6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.693+0000 7f3ac55ec700 1 --2- 172.21.15.40:0/229967104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ab80460f0 0x7f3ab80da6c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00e9d0 tx=0x7f3aac00ece0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.697+0000 7f3ac4deb700 1 -- 172.21.15.40:0/229967104 reap_dead start 2024-09-13T18:09:08.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.697+0000 7f3ac4deb700 1 -- 172.21.15.40:0/229967104 reap_dead start 2024-09-13T18:09:08.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.697+0000 7f3ac55ec700 1 -- 172.21.15.40:0/229967104 shutdown_connections 2024-09-13T18:09:08.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.697+0000 7f3ac55ec700 1 -- 172.21.15.40:0/229967104 >> 172.21.15.40:0/229967104 conn(0x7f3ab8050e10 msgr2=0x7f3ab8004f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:08.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.697+0000 7f3ac55ec700 1 -- 172.21.15.40:0/229967104 shutdown_connections 2024-09-13T18:09:08.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.697+0000 7f3ac55ec700 1 -- 172.21.15.40:0/229967104 wait complete. 2024-09-13T18:09:08.734 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:09:08.735 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:09:08.735 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:09:08.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.857+0000 7fd93aeea700 1 Processor -- start 2024-09-13T18:09:08.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.857+0000 7fd93aeea700 1 -- start start 2024-09-13T18:09:08.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.857+0000 7fd93aeea700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd934151730 0x7fd9340c3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.857+0000 7fd93aeea700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9340c3e70 0x7fd9341579f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.857+0000 7fd93aeea700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9340c3580 con 0x7fd93406e070 2024-09-13T18:09:08.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.857+0000 7fd93aeea700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd934151b10 con 0x7fd934151730 2024-09-13T18:09:08.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.857+0000 7fd93aeea700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd934157f30 con 0x7fd9340c3e70 2024-09-13T18:09:08.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.857+0000 7fd9396e7700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd93406e070 0x7fd9341572e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43630/0 (socket says 172.21.15.40:43630) 2024-09-13T18:09:08.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.857+0000 7fd9396e7700 1 -- 172.21.15.40:0/846561096 learned_addr learned my addr 172.21.15.40:0/846561096 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:09:08.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.857+0000 7fd93a6e9700 1 --2- 172.21.15.40:0/846561096 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9340c3e70 0x7fd9341579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.857+0000 7fd939ee8700 1 --2- 172.21.15.40:0/846561096 >> v2:172.21.15.43:3300/0 conn(0x7fd934151730 0x7fd9340c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.857+0000 7fd938ee6700 1 -- 172.21.15.40:0/846561096 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679753768 0 0) 0x7fd9340c3580 con 0x7fd93406e070 2024-09-13T18:09:08.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.857+0000 7fd938ee6700 1 -- 172.21.15.40:0/846561096 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9200035d0 con 0x7fd93406e070 2024-09-13T18:09:08.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.857+0000 7fd939ee8700 1 -- 172.21.15.40:0/846561096 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9340c3e70 msgr2=0x7fd9341579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:09:08.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.857+0000 7fd939ee8700 1 --2- 172.21.15.40:0/846561096 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9340c3e70 0x7fd9341579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.857+0000 7fd939ee8700 1 -- 172.21.15.40:0/846561096 >> v1:172.21.15.40:6789/0 conn(0x7fd93406e070 legacy=0x7fd9341572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.857+0000 7fd939ee8700 1 -- 172.21.15.40:0/846561096 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9340700d0 con 0x7fd934151730 2024-09-13T18:09:08.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.857+0000 7fd93a6e9700 1 --2- 172.21.15.40:0/846561096 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9340c3e70 0x7fd9341579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:09:08.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.861+0000 7fd939ee8700 1 --2- 172.21.15.40:0/846561096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd934151730 0x7fd9340c3930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd930009850 tx=0x7fd930009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88471acd4746fd96 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:08.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.861+0000 7fd938ee6700 1 -- 172.21.15.40:0/846561096 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930011830 con 0x7fd934151730 2024-09-13T18:09:08.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.861+0000 7fd938ee6700 1 -- 172.21.15.40:0/846561096 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd930011990 con 0x7fd934151730 2024-09-13T18:09:08.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.861+0000 7fd938ee6700 1 -- 172.21.15.40:0/846561096 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930011d40 con 0x7fd934151730 2024-09-13T18:09:08.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.861+0000 7fd93aeea700 1 -- 172.21.15.40:0/846561096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd934151730 msgr2=0x7fd9340c3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.861+0000 7fd93aeea700 1 --2- 172.21.15.40:0/846561096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd934151730 0x7fd9340c3930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd930009850 tx=0x7fd930009b60 comp rx=0 tx=0).stop 2024-09-13T18:09:08.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.861+0000 7fd93aeea700 1 -- 172.21.15.40:0/846561096 shutdown_connections 2024-09-13T18:09:08.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.861+0000 7fd93aeea700 1 --2- 172.21.15.40:0/846561096 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9340c3e70 0x7fd9341579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.861+0000 7fd93aeea700 1 --2- 172.21.15.40:0/846561096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd934151730 0x7fd9340c3930 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.861+0000 7fd93aeea700 1 -- 172.21.15.40:0/846561096 >> 172.21.15.40:0/846561096 conn(0x7fd934066090 msgr2=0x7fd934066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:08.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.861+0000 7fd93aeea700 1 -- 172.21.15.40:0/846561096 shutdown_connections 2024-09-13T18:09:08.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.861+0000 7fd93aeea700 1 -- 172.21.15.40:0/846561096 wait complete. 2024-09-13T18:09:08.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.861+0000 7fd93aeea700 1 Processor -- start 2024-09-13T18:09:08.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.861+0000 7fd93aeea700 1 -- start start 2024-09-13T18:09:08.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.861+0000 7fd93aeea700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd93406e070 0x7fd9340c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.861+0000 7fd93aeea700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9340c3e70 0x7fd9341f0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.861+0000 7fd93aeea700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd934151730 0x7fd934147900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.861+0000 7fd93aeea700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9340cd980 con 0x7fd934151730 2024-09-13T18:09:08.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.861+0000 7fd93aeea700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9340c6040 con 0x7fd93406e070 2024-09-13T18:09:08.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.861+0000 7fd93aeea700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9340c5780 con 0x7fd9340c3e70 2024-09-13T18:09:08.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd93a6e9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd934151730 0x7fd934147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd93a6e9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd934151730 0x7fd934147900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59482/0 (socket says 172.21.15.40:59482) 2024-09-13T18:09:08.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd93a6e9700 1 -- 172.21.15.40:0/407269872 learned_addr learned my addr 172.21.15.40:0/407269872 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:08.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd939ee8700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd93406e070 0x7fd9340c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd9396e7700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9340c3e70 0x7fd9341f0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd93a6e9700 1 -- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9340c3e70 msgr2=0x7fd9341f0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd93a6e9700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9340c3e70 0x7fd9341f0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd93a6e9700 1 -- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd93406e070 msgr2=0x7fd9340c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd93a6e9700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd93406e070 0x7fd9340c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd93a6e9700 1 -- 172.21.15.40:0/407269872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c009a70 con 0x7fd934151730 2024-09-13T18:09:08.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd9396e7700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9340c3e70 0x7fd9341f0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:09:08.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd93a6e9700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd934151730 0x7fd934147900 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd92c00efc0 tx=0x7fd92c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:08.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd92a7fc700 1 -- 172.21.15.40:0/407269872 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c00cd40 con 0x7fd934151730 2024-09-13T18:09:08.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd92a7fc700 1 -- 172.21.15.40:0/407269872 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92c00f040 con 0x7fd934151730 2024-09-13T18:09:08.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd93aeea700 1 -- 172.21.15.40:0/407269872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9300094b0 con 0x7fd934151730 2024-09-13T18:09:08.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd92a7fc700 1 -- 172.21.15.40:0/407269872 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c0146c0 con 0x7fd934151730 2024-09-13T18:09:08.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd93aeea700 1 -- 172.21.15.40:0/407269872 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9340c6990 con 0x7fd934151730 2024-09-13T18:09:08.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd93aeea700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9340c7190 0x7fd934147e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd93aeea700 1 -- 172.21.15.40:0/407269872 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9340cd740 con 0x7fd9340c7190 2024-09-13T18:09:08.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd939ee8700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9340c7190 0x7fd934147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd92a7fc700 1 -- 172.21.15.40:0/407269872 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd92c014820 con 0x7fd934151730 2024-09-13T18:09:08.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd939ee8700 1 -- 172.21.15.40:0/407269872 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd934158120 con 0x7fd9340c7190 2024-09-13T18:09:08.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd939ee8700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9340c7190 0x7fd934147e40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd930009850 tx=0x7fd9300047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:08.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.865+0000 7fd92a7fc700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd918066730 0x7fd918068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.869+0000 7fd92a7fc700 1 -- 172.21.15.40:0/407269872 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd92c084ff0 con 0x7fd934151730 2024-09-13T18:09:08.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.869+0000 7fd9396e7700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd918066730 0x7fd918068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.869+0000 7fd92a7fc700 1 -- 172.21.15.40:0/407269872 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930004450 con 0x7fd9340c7190 2024-09-13T18:09:08.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.869+0000 7fd9396e7700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd918066730 0x7fd918068bf0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd9340c8880 tx=0x7fd924009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:08.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.869+0000 7fd92a7fc700 1 -- 172.21.15.40:0/407269872 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd9300045b0 con 0x7fd9340c7190 2024-09-13T18:09:08.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.869+0000 7fd92a7fc700 1 -- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9340c7190 msgr2=0x7fd934147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.869+0000 7fd92a7fc700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9340c7190 0x7fd934147e40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd930009850 tx=0x7fd9300047e0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.869+0000 7fd92a7fc700 1 -- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9340c7190 msgr2=0x7fd934147e40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:09:08.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.869+0000 7fd92a7fc700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9340c7190 0x7fd934147e40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.881+0000 7fd93aeea700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9341f0940 0x7fd9340c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:08.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.881+0000 7fd93aeea700 1 -- 172.21.15.40:0/407269872 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd934074830 con 0x7fd9341f0940 2024-09-13T18:09:08.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.881+0000 7fd939ee8700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9341f0940 0x7fd9340c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:08.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.881+0000 7fd939ee8700 1 -- 172.21.15.40:0/407269872 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd93406eb20 con 0x7fd9341f0940 2024-09-13T18:09:08.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.881+0000 7fd939ee8700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9341f0940 0x7fd9340c42f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd930009ba0 tx=0x7fd930004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:08.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.881+0000 7fd92a7fc700 1 -- 172.21.15.40:0/407269872 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930003e60 con 0x7fd9341f0940 2024-09-13T18:09:08.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.885+0000 7fd92a7fc700 1 -- 172.21.15.40:0/407269872 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4450 (secure 0 0 0) 0x7fd93002c720 con 0x7fd9341f0940 2024-09-13T18:09:08.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.885+0000 7fd92a7fc700 1 -- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9341f0940 msgr2=0x7fd9340c42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.885+0000 7fd92a7fc700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9341f0940 0x7fd9340c42f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd930009ba0 tx=0x7fd930004770 comp rx=0 tx=0).stop 2024-09-13T18:09:08.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.885+0000 7fd92a7fc700 1 -- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9341f0940 msgr2=0x7fd9340c42f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:09:08.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.885+0000 7fd92a7fc700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd9341f0940 0x7fd9340c42f0 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd930009ba0 tx=0x7fd930004770 comp rx=0 tx=0).stop 2024-09-13T18:09:08.896 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:09:08.896 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:09:08.896 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:09:08.896 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:09:08.897 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 152, 2024-09-13T18:09:08.897 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:09:08.897 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:09:08.897 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:09:08.897 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:09:08.897 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:08.897 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:09:08.897 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:09:08.897 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:09:08.897 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:09:08.898 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:08.898 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:08.898 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:08.898 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:08.898 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:08.898 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:08.898 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:08.898 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:08.898 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:08.899 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:08.899 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:08.899 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:09:08.899 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:09:08.899 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:08.899 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:08.899 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:08.899 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:08.899 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:08.899 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:08.900 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:08.900 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:08.900 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:08.900 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:08.900 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:08.900 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.900 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:09:08.900 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:09:08.900 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:09:08.900 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:09:08.900 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:09:08.900 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:09:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:09:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:09:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:09:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:09:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:09:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:09:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:09:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:09:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:09:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:09:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:09:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:08.901 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:08.902 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:08.902 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:08.902 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:08.902 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:08.902 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:08.902 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:08.902 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:08.902 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:08.902 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:09:08.902 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.902 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:09:08.902 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.902 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:09:08.903 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:09:08.903 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:08.903 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:08.903 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.903 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:08.903 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:09:08.903 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:08.903 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.903 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.903 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:08.903 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:09:08.903 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:08.903 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.903 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:08.904 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.904 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:09:08.904 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:09:08.904 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:08.904 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:08.904 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:08.904 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.904 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.904 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:09:08.904 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:09:08.904 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:08.904 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:08.904 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.905 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:08.905 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:09:08.905 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:08.905 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.905 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.905 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:08.905 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:09:08.905 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:08.905 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.905 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:08.905 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.905 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:09:08.905 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:09:08.906 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:08.906 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:08.906 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:08.906 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.906 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.906 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:09:08.906 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:09:08.906 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:08.906 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:08.906 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.906 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:08.906 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:09:08.906 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:08.907 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.907 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.907 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:08.907 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:09:08.907 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:08.907 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.907 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:08.907 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.907 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:09:08.907 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:09:08.907 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:08.907 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:08.907 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:08.907 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.908 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:08.908 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.908 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:09:08.908 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:09:08.908 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.908 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:08.908 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:08.908 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:09:08.908 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.908 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:08.908 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:09:08.908 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:08.908 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:08.909 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:08.909 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:09:08.909 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:08.909 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:08.909 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:08.909 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:09:08.909 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:09:08.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.885+0000 7fd93aeea700 1 -- 172.21.15.40:0/407269872 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd918066730 msgr2=0x7fd918068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.885+0000 7fd93aeea700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd918066730 0x7fd918068bf0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd9340c8880 tx=0x7fd924009040 comp rx=0 tx=0).stop 2024-09-13T18:09:08.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.885+0000 7fd93aeea700 1 -- 172.21.15.40:0/407269872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd934151730 msgr2=0x7fd934147900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:08.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.885+0000 7fd93aeea700 1 --2- 172.21.15.40:0/407269872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd934151730 0x7fd934147900 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd92c00efc0 tx=0x7fd92c00c3a0 comp rx=0 tx=0).stop 2024-09-13T18:09:08.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.885+0000 7fd93a6e9700 1 -- 172.21.15.40:0/407269872 reap_dead start 2024-09-13T18:09:08.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.885+0000 7fd93a6e9700 1 -- 172.21.15.40:0/407269872 reap_dead start 2024-09-13T18:09:08.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.885+0000 7fd93aeea700 1 -- 172.21.15.40:0/407269872 shutdown_connections 2024-09-13T18:09:08.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.885+0000 7fd93aeea700 1 -- 172.21.15.40:0/407269872 >> 172.21.15.40:0/407269872 conn(0x7fd934066090 msgr2=0x7fd93414ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:08.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.885+0000 7fd93aeea700 1 -- 172.21.15.40:0/407269872 shutdown_connections 2024-09-13T18:09:08.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:08.885+0000 7fd93aeea700 1 -- 172.21.15.40:0/407269872 wait complete. 2024-09-13T18:09:08.911 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:09:08.911 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:09:08.911 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:09:09.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.025+0000 7f593ae0e700 1 Processor -- start 2024-09-13T18:09:09.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.025+0000 7f593ae0e700 1 -- start start 2024-09-13T18:09:09.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.025+0000 7f593ae0e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f592c04ca90 0x7f592c0df120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:09.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.025+0000 7f593ae0e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592c045480 con 0x7f592c046f00 2024-09-13T18:09:09.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.025+0000 7f593ae0e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592c0456b0 con 0x7f592c0d8b00 2024-09-13T18:09:09.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.025+0000 7f593ae0e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f592c045950 con 0x7f592c04ca90 2024-09-13T18:09:09.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.025+0000 7f593a60d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f592c046f00 0x7f592c006880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:43640/0 (socket says 172.21.15.40:43640) 2024-09-13T18:09:09.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.025+0000 7f593a60d700 1 -- 172.21.15.40:0/202566989 learned_addr learned my addr 172.21.15.40:0/202566989 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:09:09.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.025+0000 7f593960b700 1 --2- 172.21.15.40:0/202566989 >> v2:172.21.15.43:3300/0 conn(0x7f592c04ca90 0x7f592c0df120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:09.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.025+0000 7f5938e0a700 1 -- 172.21.15.40:0/202566989 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3826210670 0 0) 0x7f592c045480 con 0x7f592c046f00 2024-09-13T18:09:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.025+0000 7f5938e0a700 1 -- 172.21.15.40:0/202566989 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59200035d0 con 0x7f592c046f00 2024-09-13T18:09:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.025+0000 7f5938e0a700 1 -- 172.21.15.40:0/202566989 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1555421237 0 0) 0x7f592c0456b0 con 0x7f592c0d8b00 2024-09-13T18:09:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.025+0000 7f5938e0a700 1 -- 172.21.15.40:0/202566989 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f592c045480 con 0x7f592c0d8b00 2024-09-13T18:09:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.025+0000 7f593960b700 1 -- 172.21.15.40:0/202566989 >> v1:172.21.15.43:6789/0 conn(0x7f592c0d8b00 legacy=0x7f592c04af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.025+0000 7f593960b700 1 -- 172.21.15.40:0/202566989 >> v1:172.21.15.40:6789/0 conn(0x7f592c046f00 legacy=0x7f592c006880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.025+0000 7f593960b700 1 -- 172.21.15.40:0/202566989 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592c046140 con 0x7f592c04ca90 2024-09-13T18:09:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.029+0000 7f593960b700 1 --2- 172.21.15.40:0/202566989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c04ca90 0x7f592c0df120 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5924009850 tx=0x7f5924009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cae21645d6c3a2d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.029+0000 7f5938e0a700 1 -- 172.21.15.40:0/202566989 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5924011830 con 0x7f592c04ca90 2024-09-13T18:09:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.029+0000 7f5938e0a700 1 -- 172.21.15.40:0/202566989 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5924011990 con 0x7f592c04ca90 2024-09-13T18:09:09.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.029+0000 7f5938e0a700 1 -- 172.21.15.40:0/202566989 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5924011d40 con 0x7f592c04ca90 2024-09-13T18:09:09.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.029+0000 7f593ae0e700 1 -- 172.21.15.40:0/202566989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c04ca90 msgr2=0x7f592c0df120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:09.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.029+0000 7f593ae0e700 1 --2- 172.21.15.40:0/202566989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c04ca90 0x7f592c0df120 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5924009850 tx=0x7f5924009b60 comp rx=0 tx=0).stop 2024-09-13T18:09:09.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.029+0000 7f593ae0e700 1 -- 172.21.15.40:0/202566989 shutdown_connections 2024-09-13T18:09:09.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.029+0000 7f593ae0e700 1 --2- 172.21.15.40:0/202566989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c04ca90 0x7f592c0df120 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:09.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.029+0000 7f593ae0e700 1 -- 172.21.15.40:0/202566989 >> 172.21.15.40:0/202566989 conn(0x7f592c0505e0 msgr2=0x7f592c0509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:09.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.029+0000 7f593ae0e700 1 -- 172.21.15.40:0/202566989 shutdown_connections 2024-09-13T18:09:09.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.029+0000 7f593ae0e700 1 -- 172.21.15.40:0/202566989 wait complete. 2024-09-13T18:09:09.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.029+0000 7f593ae0e700 1 Processor -- start 2024-09-13T18:09:09.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.029+0000 7f593ae0e700 1 -- start start 2024-09-13T18:09:09.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f593ae0e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c046f00 0x7f592c0d7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:09.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f593ae0e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f592c04ca90 0x7f592c0cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:09.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f593ae0e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f592c0d8b00 0x7f592c0ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:09.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f593ae0e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c0de4f0 con 0x7f592c04ca90 2024-09-13T18:09:09.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f593ae0e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c0d7b70 con 0x7f592c046f00 2024-09-13T18:09:09.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f593ae0e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592c178140 con 0x7f592c0d8b00 2024-09-13T18:09:09.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f593960b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f592c04ca90 0x7f592c0cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:09.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f593960b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f592c04ca90 0x7f592c0cdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59488/0 (socket says 172.21.15.40:59488) 2024-09-13T18:09:09.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f593960b700 1 -- 172.21.15.40:0/405820406 learned_addr learned my addr 172.21.15.40:0/405820406 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:09.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f5939e0c700 1 --2- 172.21.15.40:0/405820406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c046f00 0x7f592c0d7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:09.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f593a60d700 1 --2- 172.21.15.40:0/405820406 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f592c0d8b00 0x7f592c0ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:09.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f593960b700 1 -- 172.21.15.40:0/405820406 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f592c0d8b00 msgr2=0x7f592c0ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:09.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f593960b700 1 --2- 172.21.15.40:0/405820406 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f592c0d8b00 0x7f592c0ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:09.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f593960b700 1 -- 172.21.15.40:0/405820406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c046f00 msgr2=0x7f592c0d7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:09.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f593960b700 1 --2- 172.21.15.40:0/405820406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c046f00 0x7f592c0d7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:09.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f593960b700 1 -- 172.21.15.40:0/405820406 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59240094b0 con 0x7f592c04ca90 2024-09-13T18:09:09.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f593960b700 1 --2- 172.21.15.40:0/405820406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f592c04ca90 0x7f592c0cdcb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f59240090b0 tx=0x7f5924004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:09.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f592a7fc700 1 -- 172.21.15.40:0/405820406 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5924004380 con 0x7f592c04ca90 2024-09-13T18:09:09.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f592a7fc700 1 -- 172.21.15.40:0/405820406 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592401f070 con 0x7f592c04ca90 2024-09-13T18:09:09.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f593ae0e700 1 -- 172.21.15.40:0/405820406 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c178420 con 0x7f592c04ca90 2024-09-13T18:09:09.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f592a7fc700 1 -- 172.21.15.40:0/405820406 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59240036c0 con 0x7f592c04ca90 2024-09-13T18:09:09.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f593ae0e700 1 -- 172.21.15.40:0/405820406 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c1789e0 con 0x7f592c04ca90 2024-09-13T18:09:09.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f593ae0e700 1 -- 172.21.15.40:0/405820406 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592c007a50 con 0x7f592c04ca90 2024-09-13T18:09:09.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.033+0000 7f592a7fc700 1 -- 172.21.15.40:0/405820406 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5924021030 con 0x7f592c04ca90 2024-09-13T18:09:09.047 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.037+0000 7f592a7fc700 1 --2- 172.21.15.40:0/405820406 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5918066760 0x7f5918068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:09.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.037+0000 7f592a7fc700 1 -- 172.21.15.40:0/405820406 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5924085590 con 0x7f592c04ca90 2024-09-13T18:09:09.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.037+0000 7f5939e0c700 1 --2- 172.21.15.40:0/405820406 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5918066760 0x7f5918068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:09.048 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.037+0000 7f5939e0c700 1 --2- 172.21.15.40:0/405820406 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5918066760 0x7f5918068c20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5930001840 tx=0x7f5930009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:09.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.041+0000 7f592a7fc700 1 -- 172.21.15.40:0/405820406 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5924052540 con 0x7f592c04ca90 2024-09-13T18:09:09.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.273+0000 7f593ae0e700 1 -- 172.21.15.40:0/405820406 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f592c036600 con 0x7f592c04ca90 2024-09-13T18:09:09.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.277+0000 7f592a7fc700 1 -- 172.21.15.40:0/405820406 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f5924006ce0 con 0x7f592c04ca90 2024-09-13T18:09:09.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.277+0000 7f593ae0e700 1 -- 172.21.15.40:0/405820406 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5918066760 msgr2=0x7f5918068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:09.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.277+0000 7f593ae0e700 1 --2- 172.21.15.40:0/405820406 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5918066760 0x7f5918068c20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5930001840 tx=0x7f5930009040 comp rx=0 tx=0).stop 2024-09-13T18:09:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.277+0000 7f593ae0e700 1 -- 172.21.15.40:0/405820406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f592c04ca90 msgr2=0x7f592c0cdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.277+0000 7f593ae0e700 1 --2- 172.21.15.40:0/405820406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f592c04ca90 0x7f592c0cdcb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f59240090b0 tx=0x7f5924004770 comp rx=0 tx=0).stop 2024-09-13T18:09:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.277+0000 7f593ae0e700 1 -- 172.21.15.40:0/405820406 shutdown_connections 2024-09-13T18:09:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.277+0000 7f593ae0e700 1 --2- 172.21.15.40:0/405820406 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5918066760 0x7f5918068c20 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.277+0000 7f593ae0e700 1 --2- 172.21.15.40:0/405820406 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f592c0d8b00 0x7f592c0ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.277+0000 7f593ae0e700 1 --2- 172.21.15.40:0/405820406 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f592c04ca90 0x7f592c0cdcb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.277+0000 7f593ae0e700 1 --2- 172.21.15.40:0/405820406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c046f00 0x7f592c0d7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.277+0000 7f593ae0e700 1 -- 172.21.15.40:0/405820406 >> 172.21.15.40:0/405820406 conn(0x7f592c0505e0 msgr2=0x7f592c0cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.277+0000 7f593ae0e700 1 -- 172.21.15.40:0/405820406 shutdown_connections 2024-09-13T18:09:09.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:09.277+0000 7f593ae0e700 1 -- 172.21.15.40:0/405820406 wait complete. 2024-09-13T18:09:09.304 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:09:19.305 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:09:19.306 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:09:19.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.421+0000 7fa8972c0700 1 Processor -- start 2024-09-13T18:09:19.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.421+0000 7fa8972c0700 1 -- start start 2024-09-13T18:09:19.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa8972c0700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa890159dd0 0x7fa89006d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:19.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa8972c0700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa890070830 con 0x7fa890153470 2024-09-13T18:09:19.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa8972c0700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa89015a1d0 con 0x7fa89006d9d0 2024-09-13T18:09:19.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa8972c0700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa89006ed10 con 0x7fa890159dd0 2024-09-13T18:09:19.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa895abd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa890159dd0 0x7fa89006d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:19.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa895abd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa890159dd0 0x7fa89006d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33592/0 (socket says 172.21.15.40:33592) 2024-09-13T18:09:19.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa8962be700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa890153470 0x7fa8900661e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47106/0 (socket says 172.21.15.40:47106) 2024-09-13T18:09:19.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa895abd700 1 -- 172.21.15.40:0/3163295672 learned_addr learned my addr 172.21.15.40:0/3163295672 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:19.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa895abd700 1 -- 172.21.15.40:0/3163295672 >> v1:172.21.15.43:6789/0 conn(0x7fa89006d9d0 legacy=0x7fa8900c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:19.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa895abd700 1 -- 172.21.15.40:0/3163295672 >> v1:172.21.15.40:6789/0 conn(0x7fa890153470 legacy=0x7fa8900661e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:19.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa895abd700 1 -- 172.21.15.40:0/3163295672 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa890070290 con 0x7fa890159dd0 2024-09-13T18:09:19.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa895abd700 1 --2- 172.21.15.40:0/3163295672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890159dd0 0x7fa89006d490 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa880009910 tx=0x7fa880009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bec56288271ffd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:19.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa8952bc700 1 -- 172.21.15.40:0/3163295672 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88000ba60 con 0x7fa890159dd0 2024-09-13T18:09:19.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa8952bc700 1 -- 172.21.15.40:0/3163295672 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88000bbc0 con 0x7fa890159dd0 2024-09-13T18:09:19.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa8952bc700 1 -- 172.21.15.40:0/3163295672 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880009e60 con 0x7fa890159dd0 2024-09-13T18:09:19.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa8972c0700 1 -- 172.21.15.40:0/3163295672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890159dd0 msgr2=0x7fa89006d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:19.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa8972c0700 1 --2- 172.21.15.40:0/3163295672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890159dd0 0x7fa89006d490 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa880009910 tx=0x7fa880009c20 comp rx=0 tx=0).stop 2024-09-13T18:09:19.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa8972c0700 1 -- 172.21.15.40:0/3163295672 shutdown_connections 2024-09-13T18:09:19.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa8972c0700 1 --2- 172.21.15.40:0/3163295672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890159dd0 0x7fa89006d490 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:19.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa8972c0700 1 -- 172.21.15.40:0/3163295672 >> 172.21.15.40:0/3163295672 conn(0x7fa89006a390 msgr2=0x7fa89006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:19.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa8972c0700 1 -- 172.21.15.40:0/3163295672 shutdown_connections 2024-09-13T18:09:19.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.425+0000 7fa8972c0700 1 -- 172.21.15.40:0/3163295672 wait complete. 2024-09-13T18:09:19.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa8972c0700 1 Processor -- start 2024-09-13T18:09:19.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa8972c0700 1 -- start start 2024-09-13T18:09:19.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa8972c0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa89006d9d0 0x7fa89014ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:19.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa8972c0700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa890153470 0x7fa8901ef020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:19.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa8972c0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890159dd0 0x7fa89014d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:19.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa8972c0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa890159500 con 0x7fa890159dd0 2024-09-13T18:09:19.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa8972c0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8901ef560 con 0x7fa89006d9d0 2024-09-13T18:09:19.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa8972c0700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89014e910 con 0x7fa890153470 2024-09-13T18:09:19.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa896abf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890159dd0 0x7fa89014d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:19.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa896abf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890159dd0 0x7fa89014d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33596/0 (socket says 172.21.15.40:33596) 2024-09-13T18:09:19.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa896abf700 1 -- 172.21.15.40:0/274997328 learned_addr learned my addr 172.21.15.40:0/274997328 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:19.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa8962be700 1 --2- 172.21.15.40:0/274997328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa89006d9d0 0x7fa89014ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:19.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa895abd700 1 --2- 172.21.15.40:0/274997328 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa890153470 0x7fa8901ef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:19.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa896abf700 1 -- 172.21.15.40:0/274997328 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa890153470 msgr2=0x7fa8901ef020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:19.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa896abf700 1 --2- 172.21.15.40:0/274997328 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa890153470 0x7fa8901ef020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:19.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa896abf700 1 -- 172.21.15.40:0/274997328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa89006d9d0 msgr2=0x7fa89014ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:19.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa896abf700 1 --2- 172.21.15.40:0/274997328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa89006d9d0 0x7fa89014ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:19.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa896abf700 1 -- 172.21.15.40:0/274997328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa880009570 con 0x7fa890159dd0 2024-09-13T18:09:19.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa896abf700 1 --2- 172.21.15.40:0/274997328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890159dd0 0x7fa89014d7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa88800cc50 tx=0x7fa88800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:19.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa886ffd700 1 -- 172.21.15.40:0/274997328 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888003c20 con 0x7fa890159dd0 2024-09-13T18:09:19.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa886ffd700 1 -- 172.21.15.40:0/274997328 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa888003d80 con 0x7fa890159dd0 2024-09-13T18:09:19.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa8972c0700 1 -- 172.21.15.40:0/274997328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8901e3c10 con 0x7fa890159dd0 2024-09-13T18:09:19.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa886ffd700 1 -- 172.21.15.40:0/274997328 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88800aa50 con 0x7fa890159dd0 2024-09-13T18:09:19.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.429+0000 7fa8972c0700 1 -- 172.21.15.40:0/274997328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa890159220 con 0x7fa890159dd0 2024-09-13T18:09:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.433+0000 7fa8972c0700 1 -- 172.21.15.40:0/274997328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8900c8730 con 0x7fa890159dd0 2024-09-13T18:09:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.433+0000 7fa886ffd700 1 -- 172.21.15.40:0/274997328 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa88800abb0 con 0x7fa890159dd0 2024-09-13T18:09:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.433+0000 7fa886ffd700 1 --2- 172.21.15.40:0/274997328 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa874066730 0x7fa874068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.433+0000 7fa886ffd700 1 -- 172.21.15.40:0/274997328 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa8880844c0 con 0x7fa890159dd0 2024-09-13T18:09:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.433+0000 7fa8962be700 1 --2- 172.21.15.40:0/274997328 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa874066730 0x7fa874068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.433+0000 7fa8962be700 1 --2- 172.21.15.40:0/274997328 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa874066730 0x7fa874068bf0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa88c004e50 tx=0x7fa88c007ef0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:19.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.437+0000 7fa886ffd700 1 -- 172.21.15.40:0/274997328 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa888051450 con 0x7fa890159dd0 2024-09-13T18:09:19.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.685+0000 7fa8972c0700 1 -- 172.21.15.40:0/274997328 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fa89014ea50 con 0x7fa890159dd0 2024-09-13T18:09:19.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.685+0000 7fa886ffd700 1 -- 172.21.15.40:0/274997328 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7fa888059e60 con 0x7fa890159dd0 2024-09-13T18:09:19.692 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":152,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:09:19.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.685+0000 7fa8972c0700 1 -- 172.21.15.40:0/274997328 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa874066730 msgr2=0x7fa874068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:19.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.685+0000 7fa8972c0700 1 --2- 172.21.15.40:0/274997328 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa874066730 0x7fa874068bf0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa88c004e50 tx=0x7fa88c007ef0 comp rx=0 tx=0).stop 2024-09-13T18:09:19.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.689+0000 7fa8972c0700 1 -- 172.21.15.40:0/274997328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890159dd0 msgr2=0x7fa89014d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:19.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.689+0000 7fa8972c0700 1 --2- 172.21.15.40:0/274997328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890159dd0 0x7fa89014d7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa88800cc50 tx=0x7fa88800cf60 comp rx=0 tx=0).stop 2024-09-13T18:09:19.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.689+0000 7fa8972c0700 1 -- 172.21.15.40:0/274997328 shutdown_connections 2024-09-13T18:09:19.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.689+0000 7fa8972c0700 1 --2- 172.21.15.40:0/274997328 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa890159dd0 0x7fa89014d7a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:19.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.689+0000 7fa8972c0700 1 --2- 172.21.15.40:0/274997328 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa874066730 0x7fa874068bf0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:19.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.689+0000 7fa8972c0700 1 --2- 172.21.15.40:0/274997328 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa890153470 0x7fa8901ef020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:19.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.689+0000 7fa8972c0700 1 --2- 172.21.15.40:0/274997328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa89006d9d0 0x7fa89014ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:19.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.689+0000 7fa8972c0700 1 -- 172.21.15.40:0/274997328 >> 172.21.15.40:0/274997328 conn(0x7fa89006a390 msgr2=0x7fa8900661e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:19.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.689+0000 7fa8972c0700 1 -- 172.21.15.40:0/274997328 shutdown_connections 2024-09-13T18:09:19.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.689+0000 7fa8972c0700 1 -- 172.21.15.40:0/274997328 wait complete. 2024-09-13T18:09:19.708 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:09:19.708 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T18:09:19.708 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:09:19.708 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:09:19.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.817+0000 7f6f36763700 1 Processor -- start 2024-09-13T18:09:19.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.817+0000 7f6f36763700 1 -- start start 2024-09-13T18:09:19.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.817+0000 7f6f36763700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f30151440 0x7f6f300661b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:19.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.817+0000 7f6f36763700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f30157370 con 0x7f6f300c4f40 2024-09-13T18:09:19.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.817+0000 7f6f36763700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f300666f0 con 0x7f6f300ca9d0 2024-09-13T18:09:19.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.817+0000 7f6f36763700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f300cb1f0 con 0x7f6f30151440 2024-09-13T18:09:19.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.817+0000 7f6f35f62700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6f300c4f40 0x7f6f30161b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47118/0 (socket says 172.21.15.40:47118) 2024-09-13T18:09:19.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.817+0000 7f6f35f62700 1 -- 172.21.15.40:0/3322837822 learned_addr learned my addr 172.21.15.40:0/3322837822 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:09:19.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.817+0000 7f6f35761700 1 --2- 172.21.15.40:0/3322837822 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f30151440 0x7f6f300661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:19.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.817+0000 7f6f27fff700 1 -- 172.21.15.40:0/3322837822 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261281085 0 0) 0x7f6f30157370 con 0x7f6f300c4f40 2024-09-13T18:09:19.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.817+0000 7f6f27fff700 1 -- 172.21.15.40:0/3322837822 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f1c0035d0 con 0x7f6f300c4f40 2024-09-13T18:09:19.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.817+0000 7f6f27fff700 1 -- 172.21.15.40:0/3322837822 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258247554 0 0) 0x7f6f300666f0 con 0x7f6f300ca9d0 2024-09-13T18:09:19.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.817+0000 7f6f27fff700 1 -- 172.21.15.40:0/3322837822 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f30157370 con 0x7f6f300ca9d0 2024-09-13T18:09:19.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.817+0000 7f6f35761700 1 -- 172.21.15.40:0/3322837822 >> v1:172.21.15.43:6789/0 conn(0x7f6f300ca9d0 legacy=0x7f6f30157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:19.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.817+0000 7f6f35761700 1 -- 172.21.15.40:0/3322837822 >> v1:172.21.15.40:6789/0 conn(0x7f6f300c4f40 legacy=0x7f6f30161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:19.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.817+0000 7f6f35761700 1 -- 172.21.15.40:0/3322837822 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f300caf10 con 0x7f6f30151440 2024-09-13T18:09:19.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.821+0000 7f6f35761700 1 --2- 172.21.15.40:0/3322837822 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f30151440 0x7f6f300661b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009880 tx=0x7f6f20009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44105b248b5f5277 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:19.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.821+0000 7f6f27fff700 1 -- 172.21.15.40:0/3322837822 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f20011830 con 0x7f6f30151440 2024-09-13T18:09:19.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.821+0000 7f6f27fff700 1 -- 172.21.15.40:0/3322837822 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f20011990 con 0x7f6f30151440 2024-09-13T18:09:19.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.821+0000 7f6f27fff700 1 -- 172.21.15.40:0/3322837822 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f20011d00 con 0x7f6f30151440 2024-09-13T18:09:19.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.821+0000 7f6f36763700 1 -- 172.21.15.40:0/3322837822 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f30151440 msgr2=0x7f6f300661b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:19.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.821+0000 7f6f36763700 1 --2- 172.21.15.40:0/3322837822 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f30151440 0x7f6f300661b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009880 tx=0x7f6f20009b90 comp rx=0 tx=0).stop 2024-09-13T18:09:19.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.821+0000 7f6f36763700 1 -- 172.21.15.40:0/3322837822 shutdown_connections 2024-09-13T18:09:19.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.821+0000 7f6f36763700 1 --2- 172.21.15.40:0/3322837822 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f30151440 0x7f6f300661b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:19.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.821+0000 7f6f36763700 1 -- 172.21.15.40:0/3322837822 >> 172.21.15.40:0/3322837822 conn(0x7f6f3006a390 msgr2=0x7f6f3006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:19.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.821+0000 7f6f36763700 1 -- 172.21.15.40:0/3322837822 shutdown_connections 2024-09-13T18:09:19.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.821+0000 7f6f36763700 1 -- 172.21.15.40:0/3322837822 wait complete. 2024-09-13T18:09:19.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.825+0000 7f6f36763700 1 Processor -- start 2024-09-13T18:09:19.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.825+0000 7f6f36763700 1 -- start start 2024-09-13T18:09:19.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.825+0000 7f6f36763700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f300c4f40 0x7f6f301f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:19.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.825+0000 7f6f36763700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f300ca9d0 0x7f6f3014c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:19.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.825+0000 7f6f36763700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f30151440 0x7f6f3014c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:19.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.825+0000 7f6f36763700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f30157070 con 0x7f6f300ca9d0 2024-09-13T18:09:19.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.825+0000 7f6f36763700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f30156470 con 0x7f6f30151440 2024-09-13T18:09:19.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.825+0000 7f6f36763700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f30156e00 con 0x7f6f300c4f40 2024-09-13T18:09:19.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.825+0000 7f6f34f60700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f300ca9d0 0x7f6f3014c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:19.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.825+0000 7f6f34f60700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f300ca9d0 0x7f6f3014c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33612/0 (socket says 172.21.15.40:33612) 2024-09-13T18:09:19.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.825+0000 7f6f34f60700 1 -- 172.21.15.40:0/1107409990 learned_addr learned my addr 172.21.15.40:0/1107409990 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:19.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.825+0000 7f6f35761700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f300c4f40 0x7f6f301f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:19.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.825+0000 7f6f35f62700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f30151440 0x7f6f3014c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:19.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.825+0000 7f6f34f60700 1 -- 172.21.15.40:0/1107409990 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f300c4f40 msgr2=0x7f6f301f2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:19.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.825+0000 7f6f34f60700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6f300c4f40 0x7f6f301f2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:19.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.825+0000 7f6f34f60700 1 -- 172.21.15.40:0/1107409990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f30151440 msgr2=0x7f6f3014c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:19.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.825+0000 7f6f34f60700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f30151440 0x7f6f3014c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:19.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.825+0000 7f6f34f60700 1 -- 172.21.15.40:0/1107409990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f200094b0 con 0x7f6f300ca9d0 2024-09-13T18:09:19.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.825+0000 7f6f34f60700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f300ca9d0 0x7f6f3014c0f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00c440 tx=0x7f6f2c00c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:19.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.825+0000 7f6f25ffb700 1 -- 172.21.15.40:0/1107409990 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0052d0 con 0x7f6f300ca9d0 2024-09-13T18:09:19.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.829+0000 7f6f36763700 1 -- 172.21.15.40:0/1107409990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f301f3340 con 0x7f6f300ca9d0 2024-09-13T18:09:19.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.829+0000 7f6f36763700 1 -- 172.21.15.40:0/1107409990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f30156100 con 0x7f6f300ca9d0 2024-09-13T18:09:19.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.829+0000 7f6f25ffb700 1 -- 172.21.15.40:0/1107409990 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2c005430 con 0x7f6f300ca9d0 2024-09-13T18:09:19.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.829+0000 7f6f25ffb700 1 -- 172.21.15.40:0/1107409990 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0055a0 con 0x7f6f300ca9d0 2024-09-13T18:09:19.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.829+0000 7f6f36763700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f10002a40 0x7f6f10004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:19.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.829+0000 7f6f36763700 1 -- 172.21.15.40:0/1107409990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f10007f40 con 0x7f6f10002a40 2024-09-13T18:09:19.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.829+0000 7f6f35761700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f10002a40 0x7f6f10004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:19.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.829+0000 7f6f35761700 1 -- 172.21.15.40:0/1107409990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f10008100 con 0x7f6f10002a40 2024-09-13T18:09:19.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.829+0000 7f6f35761700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f10002a40 0x7f6f10004f00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009880 tx=0x7f6f20004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:19.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.829+0000 7f6f25ffb700 1 -- 172.21.15.40:0/1107409990 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6f2c0038a0 con 0x7f6f300ca9d0 2024-09-13T18:09:19.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.829+0000 7f6f25ffb700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6f14066700 0x7f6f14068bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:19.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.829+0000 7f6f25ffb700 1 -- 172.21.15.40:0/1107409990 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6f2c0837d0 con 0x7f6f300ca9d0 2024-09-13T18:09:19.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.829+0000 7f6f35f62700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6f14066700 0x7f6f14068bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:19.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.829+0000 7f6f25ffb700 1 -- 172.21.15.40:0/1107409990 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f20006ea0 con 0x7f6f10002a40 2024-09-13T18:09:19.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.829+0000 7f6f25ffb700 1 -- 172.21.15.40:0/1107409990 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6f20004300 con 0x7f6f10002a40 2024-09-13T18:09:19.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.829+0000 7f6f25ffb700 1 -- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f10002a40 msgr2=0x7f6f10004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:19.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.829+0000 7f6f25ffb700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f10002a40 0x7f6f10004f00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009880 tx=0x7f6f20004770 comp rx=0 tx=0).stop 2024-09-13T18:09:19.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.829+0000 7f6f25ffb700 1 -- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f10002a40 msgr2=0x7f6f10004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:09:19.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.829+0000 7f6f25ffb700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f10002a40 0x7f6f10004f00 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009880 tx=0x7f6f20004770 comp rx=0 tx=0).stop 2024-09-13T18:09:19.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.829+0000 7f6f35f62700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6f14066700 0x7f6f14068bc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6f28005cb0 tx=0x7f6f28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:19.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f36763700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f10006fe0 0x7f6f10007470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:19.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f36763700 1 -- 172.21.15.40:0/1107409990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1000a400 con 0x7f6f10006fe0 2024-09-13T18:09:19.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f35761700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f10006fe0 0x7f6f10007470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:19.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f35761700 1 -- 172.21.15.40:0/1107409990 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6f1000a5c0 con 0x7f6f10006fe0 2024-09-13T18:09:19.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f35761700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f10006fe0 0x7f6f10007470 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6f200097d0 tx=0x7f6f20008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:19.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f25ffb700 1 -- 172.21.15.40:0/1107409990 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2002a830 con 0x7f6f10006fe0 2024-09-13T18:09:19.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f25ffb700 1 -- 172.21.15.40:0/1107409990 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4951 (secure 0 0 0) 0x7f6f20024410 con 0x7f6f10006fe0 2024-09-13T18:09:19.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f25ffb700 1 -- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f10006fe0 msgr2=0x7f6f10007470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:19.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f25ffb700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f10006fe0 0x7f6f10007470 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6f200097d0 tx=0x7f6f20008030 comp rx=0 tx=0).stop 2024-09-13T18:09:19.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f25ffb700 1 -- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f10006fe0 msgr2=0x7f6f10007470 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:09:19.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f25ffb700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f10006fe0 0x7f6f10007470 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:19.850 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:09:19.850 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:09:19.850 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:09:19.850 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:09:19.851 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 152, 2024-09-13T18:09:19.851 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:09:19.851 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:09:19.851 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:09:19.851 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:09:19.851 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:19.851 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 11, 2024-09-13T18:09:19.851 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:09:19.851 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:09:19.851 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:09:19.851 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:19.851 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:19.851 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:19.852 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:19.852 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:19.852 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:19.852 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:19.852 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:19.852 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:19.852 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:19.852 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:19.852 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:09:19.852 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:09:19.852 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:19.852 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:19.852 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:19.853 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:19.853 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:19.853 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:19.853 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:19.853 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:19.853 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:19.853 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:19.853 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:19.853 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:19.853 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:09:19.853 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:09:19.853 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:09:19.853 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:09:19.854 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:09:19.854 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:09:19.854 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:09:19.854 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:09:19.854 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:09:19.854 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:09:19.854 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:09:19.854 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:09:19.854 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:09:19.854 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:09:19.854 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:09:19.854 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:09:19.854 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:09:19.854 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:19.855 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:19.855 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:19.855 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:19.855 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:19.855 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:19.855 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:19.855 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:19.855 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:19.855 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:19.855 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:19.855 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:09:19.855 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:19.855 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:09:19.856 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:19.856 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:09:19.856 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:09:19.856 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:19.856 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:19.856 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:19.856 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:19.856 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:09:19.856 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:19.856 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:19.856 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:19.856 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:19.856 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:09:19.856 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:19.857 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:19.857 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:19.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:19.857 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:09:19.857 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:09:19.857 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:19.857 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:19.857 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:19.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:19.857 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:19.857 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:09:19.857 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:09:19.857 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:19.857 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:19.857 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:19.858 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:19.858 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:09:19.858 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:19.858 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:19.858 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:19.858 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:19.858 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:09:19.858 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:19.858 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:19.858 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:19.858 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:19.858 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:09:19.858 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:09:19.858 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:19.858 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:19.859 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:19.859 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:19.859 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:19.859 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:09:19.859 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:09:19.859 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:19.859 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:19.859 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:19.859 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:19.859 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:09:19.859 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:19.859 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:19.859 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:19.859 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:19.860 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:09:19.860 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:19.860 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:19.860 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:19.860 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:19.860 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:09:19.860 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:09:19.860 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:19.860 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:19.860 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:19.860 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:19.860 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:19.860 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:19.860 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:09:19.861 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:09:19.861 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:19.861 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:19.861 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:19.861 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:09:19.861 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:19.861 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:19.861 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:09:19.861 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:19.861 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:19.861 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:19.861 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:09:19.861 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:19.861 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:09:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:09:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:09:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:09:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:19.862 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:09:19.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:19.863 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:19.863 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:19.863 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:09:19.863 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:19.863 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:19.863 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:19.863 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:09:19.863 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:09:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f36763700 1 -- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6f14066700 msgr2=0x7f6f14068bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f36763700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6f14066700 0x7f6f14068bc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6f28005cb0 tx=0x7f6f28009040 comp rx=0 tx=0).stop 2024-09-13T18:09:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f36763700 1 -- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f300ca9d0 msgr2=0x7f6f3014c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f35f62700 1 -- 172.21.15.40:0/1107409990 reap_dead start 2024-09-13T18:09:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f36763700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f300ca9d0 0x7f6f3014c0f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00c440 tx=0x7f6f2c00c750 comp rx=0 tx=0).stop 2024-09-13T18:09:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f36763700 1 -- 172.21.15.40:0/1107409990 shutdown_connections 2024-09-13T18:09:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f36763700 1 --2- 172.21.15.40:0/1107409990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f300ca9d0 0x7f6f3014c0f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f36763700 1 -- 172.21.15.40:0/1107409990 >> 172.21.15.40:0/1107409990 conn(0x7f6f3006a390 msgr2=0x7f6f300c26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f36763700 1 -- 172.21.15.40:0/1107409990 shutdown_connections 2024-09-13T18:09:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.841+0000 7f6f36763700 1 -- 172.21.15.40:0/1107409990 wait complete. 2024-09-13T18:09:19.866 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:09:19.866 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:09:19.866 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:09:19.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.977+0000 7fa7288b3700 1 Processor -- start 2024-09-13T18:09:19.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.977+0000 7fa7288b3700 1 -- start start 2024-09-13T18:09:19.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.977+0000 7fa7288b3700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa7240c8500 0x7fa7240c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:19.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa7288b3700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa7240c71c0 0x7fa724159630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:19.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa7288b3700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7240c8f80 con 0x7fa7241503c0 2024-09-13T18:09:19.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa7288b3700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7240c2a80 con 0x7fa7240c8500 2024-09-13T18:09:19.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa7288b3700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7240c80d0 con 0x7fa7240c71c0 2024-09-13T18:09:19.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa722ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa7240c8500 0x7fa7240c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:19.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa722ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa7240c8500 0x7fa7240c6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33644/0 (socket says 172.21.15.40:33644) 2024-09-13T18:09:19.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa722ffd700 1 -- 172.21.15.40:0/1068726912 learned_addr learned my addr 172.21.15.40:0/1068726912 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:19.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa722ffd700 1 -- 172.21.15.40:0/1068726912 >> v2:172.21.15.43:3300/0 conn(0x7fa7240c71c0 msgr2=0x7fa724159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:09:19.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa722ffd700 1 --2- 172.21.15.40:0/1068726912 >> v2:172.21.15.43:3300/0 conn(0x7fa7240c71c0 0x7fa724159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:19.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa722ffd700 1 -- 172.21.15.40:0/1068726912 >> v1:172.21.15.43:6789/0 conn(0x7fa7241503c0 legacy=0x7fa7240c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:09:19.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa722ffd700 1 -- 172.21.15.40:0/1068726912 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7241507a0 con 0x7fa7240c8500 2024-09-13T18:09:19.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa722ffd700 1 --2- 172.21.15.40:0/1068726912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7240c8500 0x7fa7240c6c80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa72406c640 tx=0x7fa714009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3fb4170d9588bb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:19.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa7227fc700 1 -- 172.21.15.40:0/1068726912 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa714004230 con 0x7fa7240c8500 2024-09-13T18:09:19.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa7227fc700 1 -- 172.21.15.40:0/1068726912 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa714004390 con 0x7fa7240c8500 2024-09-13T18:09:19.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa7227fc700 1 -- 172.21.15.40:0/1068726912 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7140036c0 con 0x7fa7240c8500 2024-09-13T18:09:19.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa7288b3700 1 -- 172.21.15.40:0/1068726912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7240c8500 msgr2=0x7fa7240c6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:19.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa7288b3700 1 --2- 172.21.15.40:0/1068726912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7240c8500 0x7fa7240c6c80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa72406c640 tx=0x7fa714009bf0 comp rx=0 tx=0).stop 2024-09-13T18:09:19.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa7288b3700 1 -- 172.21.15.40:0/1068726912 shutdown_connections 2024-09-13T18:09:19.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa7288b3700 1 --2- 172.21.15.40:0/1068726912 >> v2:172.21.15.43:3300/0 conn(0x7fa7240c71c0 0x7fa724159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:19.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa7288b3700 1 --2- 172.21.15.40:0/1068726912 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7240c8500 0x7fa7240c6c80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:19.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa7288b3700 1 -- 172.21.15.40:0/1068726912 >> 172.21.15.40:0/1068726912 conn(0x7fa72406e6d0 msgr2=0x7fa72406eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:19.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa7288b3700 1 -- 172.21.15.40:0/1068726912 shutdown_connections 2024-09-13T18:09:19.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.981+0000 7fa7288b3700 1 -- 172.21.15.40:0/1068726912 wait complete. 2024-09-13T18:09:19.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.985+0000 7fa7288b3700 1 Processor -- start 2024-09-13T18:09:19.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.985+0000 7fa7288b3700 1 -- start start 2024-09-13T18:09:19.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.985+0000 7fa7288b3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7240c71c0 0x7fa7241ecd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:19.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.985+0000 7fa7288b3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7240c8500 0x7fa72414bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:19.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.985+0000 7fa7288b3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7241503c0 0x7fa724146380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:19.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.985+0000 7fa7288b3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7241f0a90 con 0x7fa7240c71c0 2024-09-13T18:09:19.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.985+0000 7fa7288b3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7241e19b0 con 0x7fa7241503c0 2024-09-13T18:09:19.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.985+0000 7fa7288b3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7241e13f0 con 0x7fa7240c8500 2024-09-13T18:09:19.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.985+0000 7fa7237fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7240c71c0 0x7fa7241ecd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:19.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.985+0000 7fa7237fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7240c71c0 0x7fa7241ecd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33660/0 (socket says 172.21.15.40:33660) 2024-09-13T18:09:19.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.985+0000 7fa7237fe700 1 -- 172.21.15.40:0/1415993478 learned_addr learned my addr 172.21.15.40:0/1415993478 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:19.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.985+0000 7fa722ffd700 1 --2- 172.21.15.40:0/1415993478 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7240c8500 0x7fa72414bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:19.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.985+0000 7fa723fff700 1 --2- 172.21.15.40:0/1415993478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7241503c0 0x7fa724146380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:19.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.985+0000 7fa7237fe700 1 -- 172.21.15.40:0/1415993478 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7240c8500 msgr2=0x7fa72414bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:19.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.985+0000 7fa7237fe700 1 --2- 172.21.15.40:0/1415993478 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7240c8500 0x7fa72414bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:19.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.985+0000 7fa7237fe700 1 -- 172.21.15.40:0/1415993478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7241503c0 msgr2=0x7fa724146380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:19.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.985+0000 7fa7237fe700 1 --2- 172.21.15.40:0/1415993478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7241503c0 0x7fa724146380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:19.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.985+0000 7fa7237fe700 1 -- 172.21.15.40:0/1415993478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa714009570 con 0x7fa7240c71c0 2024-09-13T18:09:19.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.985+0000 7fa7237fe700 1 --2- 172.21.15.40:0/1415993478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7240c71c0 0x7fa7241ecd30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00ba50 tx=0x7fa71c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:19.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa713fff700 1 -- 172.21.15.40:0/1415993478 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c00c930 con 0x7fa7240c71c0 2024-09-13T18:09:19.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa713fff700 1 -- 172.21.15.40:0/1415993478 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c00ca90 con 0x7fa7240c71c0 2024-09-13T18:09:19.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa7288b3700 1 -- 172.21.15.40:0/1415993478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7241e16d0 con 0x7fa7240c71c0 2024-09-13T18:09:19.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa713fff700 1 -- 172.21.15.40:0/1415993478 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c00ce40 con 0x7fa7240c71c0 2024-09-13T18:09:19.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa7288b3700 1 -- 172.21.15.40:0/1415993478 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7241f06b0 con 0x7fa7240c71c0 2024-09-13T18:09:19.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa7288b3700 1 --2- 172.21.15.40:0/1415993478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7241f0eb0 0x7fa7241468c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:19.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa7288b3700 1 -- 172.21.15.40:0/1415993478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7240675e0 con 0x7fa7241f0eb0 2024-09-13T18:09:19.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa722ffd700 1 --2- 172.21.15.40:0/1415993478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7241f0eb0 0x7fa7241468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:20.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa722ffd700 1 -- 172.21.15.40:0/1415993478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa7240c8ce0 con 0x7fa7241f0eb0 2024-09-13T18:09:20.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa713fff700 1 -- 172.21.15.40:0/1415993478 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa71c007370 con 0x7fa7240c71c0 2024-09-13T18:09:20.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa722ffd700 1 --2- 172.21.15.40:0/1415993478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7241f0eb0 0x7fa7241468c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa714004700 tx=0x7fa714004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:20.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa713fff700 1 --2- 172.21.15.40:0/1415993478 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa708066760 0x7fa708068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:20.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa713fff700 1 -- 172.21.15.40:0/1415993478 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa71c083c80 con 0x7fa7240c71c0 2024-09-13T18:09:20.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa723fff700 1 --2- 172.21.15.40:0/1415993478 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa708066760 0x7fa708068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:20.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa713fff700 1 -- 172.21.15.40:0/1415993478 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa714020070 con 0x7fa7241f0eb0 2024-09-13T18:09:20.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa713fff700 1 -- 172.21.15.40:0/1415993478 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa71400b660 con 0x7fa7241f0eb0 2024-09-13T18:09:20.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa713fff700 1 -- 172.21.15.40:0/1415993478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7241f0eb0 msgr2=0x7fa7241468c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:20.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa713fff700 1 --2- 172.21.15.40:0/1415993478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7241f0eb0 0x7fa7241468c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa714004700 tx=0x7fa714004770 comp rx=0 tx=0).stop 2024-09-13T18:09:20.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa713fff700 1 -- 172.21.15.40:0/1415993478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7241f0eb0 msgr2=0x7fa7241468c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:09:20.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa713fff700 1 --2- 172.21.15.40:0/1415993478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7241f0eb0 0x7fa7241468c0 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa714004700 tx=0x7fa714004770 comp rx=0 tx=0).stop 2024-09-13T18:09:20.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.989+0000 7fa723fff700 1 --2- 172.21.15.40:0/1415993478 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa708066760 0x7fa708068c20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa7241478b0 tx=0x7fa718009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:20.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.997+0000 7fa7288b3700 1 --2- 172.21.15.40:0/1415993478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7241e0e90 0x7fa72406a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:20.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:19.997+0000 7fa7288b3700 1 -- 172.21.15.40:0/1415993478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7240c8a40 con 0x7fa7241e0e90 2024-09-13T18:09:20.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.001+0000 7fa722ffd700 1 --2- 172.21.15.40:0/1415993478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7241e0e90 0x7fa72406a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:20.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.001+0000 7fa722ffd700 1 -- 172.21.15.40:0/1415993478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa7240c2710 con 0x7fa7241e0e90 2024-09-13T18:09:20.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.001+0000 7fa722ffd700 1 --2- 172.21.15.40:0/1415993478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7241e0e90 0x7fa72406a390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa714009150 tx=0x7fa714011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:20.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.001+0000 7fa713fff700 1 -- 172.21.15.40:0/1415993478 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa714030070 con 0x7fa7241e0e90 2024-09-13T18:09:20.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.001+0000 7fa713fff700 1 -- 172.21.15.40:0/1415993478 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4949 (secure 0 0 0) 0x7fa71402b9f0 con 0x7fa7241e0e90 2024-09-13T18:09:20.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.001+0000 7fa713fff700 1 -- 172.21.15.40:0/1415993478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7241e0e90 msgr2=0x7fa72406a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:20.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.001+0000 7fa713fff700 1 --2- 172.21.15.40:0/1415993478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7241e0e90 0x7fa72406a390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa714009150 tx=0x7fa714011000 comp rx=0 tx=0).stop 2024-09-13T18:09:20.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.001+0000 7fa713fff700 1 -- 172.21.15.40:0/1415993478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7241e0e90 msgr2=0x7fa72406a390 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:09:20.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.001+0000 7fa713fff700 1 --2- 172.21.15.40:0/1415993478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7241e0e90 0x7fa72406a390 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa714009150 tx=0x7fa714011000 comp rx=0 tx=0).stop 2024-09-13T18:09:20.009 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:09:20.010 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:09:20.010 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:09:20.010 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:09:20.010 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 152, 2024-09-13T18:09:20.010 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:09:20.010 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:09:20.010 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:09:20.010 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:09:20.010 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:20.011 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 12, 2024-09-13T18:09:20.011 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:09:20.011 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:09:20.011 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:09:20.011 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:20.011 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:20.011 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:20.011 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:20.011 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:20.011 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:20.011 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:20.011 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:20.011 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:20.012 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:20.012 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:20.012 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:09:20.012 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:09:20.012 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:20.012 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:20.012 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:20.012 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:20.012 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:20.012 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:20.012 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:20.012 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:20.012 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:20.012 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:20.013 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:20.013 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.013 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:09:20.013 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:09:20.013 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:09:20.013 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:09:20.013 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:09:20.013 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:09:20.013 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:09:20.013 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:09:20.013 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:09:20.014 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:09:20.014 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:09:20.014 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:09:20.014 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:09:20.014 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:09:20.014 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:09:20.015 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:09:20.015 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:09:20.015 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:20.015 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:20.015 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:20.015 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:20.015 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:20.016 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:20.016 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:20.016 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:20.016 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:20.016 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:20.016 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:20.017 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:09:20.017 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.017 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:09:20.017 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.017 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:09:20.017 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:09:20.018 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:20.018 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:20.018 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.018 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:20.018 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:09:20.018 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:20.019 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.019 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.019 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:20.019 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:09:20.019 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:20.019 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:20.020 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:20.020 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.020 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:09:20.020 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:09:20.020 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:20.020 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:20.021 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:20.021 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.021 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.021 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:09:20.021 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:09:20.021 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:20.022 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:20.022 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.022 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:20.022 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:09:20.022 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:20.022 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.023 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.023 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:20.023 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:09:20.023 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:20.023 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:20.024 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:20.024 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.024 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:09:20.024 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:09:20.024 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:20.024 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:20.025 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:20.025 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.025 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.025 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:09:20.026 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:09:20.026 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:20.026 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:20.026 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.026 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:20.026 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:09:20.027 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:20.027 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.027 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.027 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:20.027 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:09:20.027 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:20.028 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:20.028 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:20.028 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.028 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:09:20.028 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:09:20.028 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:20.029 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:20.029 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:20.029 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:20.029 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:20.029 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.029 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:09:20.030 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:09:20.030 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.030 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:20.030 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:20.030 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:09:20.030 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:20.031 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:20.031 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:09:20.031 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.031 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:20.031 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:20.031 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:09:20.032 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:20.032 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:20.032 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:09:20.032 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.032 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:20.033 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:20.033 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:09:20.033 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:20.033 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:20.033 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:09:20.033 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.034 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:20.034 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:20.034 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:09:20.034 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:20.034 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:20.034 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:09:20.035 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.035 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:20.035 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:20.035 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:09:20.035 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:20.035 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:20.036 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.036 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:09:20.036 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:09:20.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.001+0000 7fa7288b3700 1 -- 172.21.15.40:0/1415993478 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa708066760 msgr2=0x7fa708068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:20.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.001+0000 7fa7288b3700 1 --2- 172.21.15.40:0/1415993478 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa708066760 0x7fa708068c20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa7241478b0 tx=0x7fa718009190 comp rx=0 tx=0).stop 2024-09-13T18:09:20.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.001+0000 7fa7288b3700 1 -- 172.21.15.40:0/1415993478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7240c71c0 msgr2=0x7fa7241ecd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:20.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.001+0000 7fa7288b3700 1 --2- 172.21.15.40:0/1415993478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7240c71c0 0x7fa7241ecd30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00ba50 tx=0x7fa71c00bd60 comp rx=0 tx=0).stop 2024-09-13T18:09:20.039 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.001+0000 7fa723fff700 1 -- 172.21.15.40:0/1415993478 reap_dead start 2024-09-13T18:09:20.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.001+0000 7fa723fff700 1 -- 172.21.15.40:0/1415993478 reap_dead start 2024-09-13T18:09:20.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.001+0000 7fa7288b3700 1 -- 172.21.15.40:0/1415993478 shutdown_connections 2024-09-13T18:09:20.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.001+0000 7fa7288b3700 1 -- 172.21.15.40:0/1415993478 >> 172.21.15.40:0/1415993478 conn(0x7fa72406e6d0 msgr2=0x7fa72414dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:20.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.001+0000 7fa7288b3700 1 -- 172.21.15.40:0/1415993478 shutdown_connections 2024-09-13T18:09:20.040 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.001+0000 7fa7288b3700 1 -- 172.21.15.40:0/1415993478 wait complete. 2024-09-13T18:09:20.040 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:09:20.040 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:09:20.040 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:09:20.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.153+0000 7f0b772f9700 1 Processor -- start 2024-09-13T18:09:20.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.153+0000 7f0b772f9700 1 -- start start 2024-09-13T18:09:20.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.153+0000 7f0b772f9700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0b700cf9b0 0x7f0b7006c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:20.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.153+0000 7f0b772f9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0b7006c5b0 0x7f0b70163d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:20.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.153+0000 7f0b772f9700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b70159e50 con 0x7f0b70153720 2024-09-13T18:09:20.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.153+0000 7f0b772f9700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b700c5ea0 con 0x7f0b7006c5b0 2024-09-13T18:09:20.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.153+0000 7f0b772f9700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b7015d390 con 0x7f0b700cf9b0 2024-09-13T18:09:20.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.153+0000 7f0b76af8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0b7006c5b0 0x7f0b70163d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:20.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.153+0000 7f0b76af8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0b7006c5b0 0x7f0b70163d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33668/0 (socket says 172.21.15.40:33668) 2024-09-13T18:09:20.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.153+0000 7f0b76af8700 1 -- 172.21.15.40:0/656583079 learned_addr learned my addr 172.21.15.40:0/656583079 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:20.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.153+0000 7f0b75af6700 1 --2- 172.21.15.40:0/656583079 >> v2:172.21.15.43:3300/0 conn(0x7f0b700cf9b0 0x7f0b7006c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:20.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.153+0000 7f0b76af8700 1 -- 172.21.15.40:0/656583079 >> v2:172.21.15.43:3300/0 conn(0x7f0b700cf9b0 msgr2=0x7f0b7006c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:09:20.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.153+0000 7f0b76af8700 1 --2- 172.21.15.40:0/656583079 >> v2:172.21.15.43:3300/0 conn(0x7f0b700cf9b0 0x7f0b7006c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:20.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.153+0000 7f0b76af8700 1 -- 172.21.15.40:0/656583079 >> v1:172.21.15.43:6789/0 conn(0x7f0b70153720 legacy=0x7f0b700ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:09:20.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.153+0000 7f0b76af8700 1 -- 172.21.15.40:0/656583079 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b700cac20 con 0x7f0b7006c5b0 2024-09-13T18:09:20.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.153+0000 7f0b75af6700 1 --2- 172.21.15.40:0/656583079 >> v2:172.21.15.43:3300/0 conn(0x7f0b700cf9b0 0x7f0b7006c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:09:20.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.153+0000 7f0b76af8700 1 --2- 172.21.15.40:0/656583079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7006c5b0 0x7f0b70163d50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0b700b1b20 tx=0x7f0b6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792fe6bf8aee7668 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:20.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b752f5700 1 -- 172.21.15.40:0/656583079 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6800e070 con 0x7f0b7006c5b0 2024-09-13T18:09:20.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b752f5700 1 -- 172.21.15.40:0/656583079 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b68003a40 con 0x7f0b7006c5b0 2024-09-13T18:09:20.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b752f5700 1 -- 172.21.15.40:0/656583079 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68003df0 con 0x7f0b7006c5b0 2024-09-13T18:09:20.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b772f9700 1 -- 172.21.15.40:0/656583079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7006c5b0 msgr2=0x7f0b70163d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:20.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b772f9700 1 --2- 172.21.15.40:0/656583079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7006c5b0 0x7f0b70163d50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0b700b1b20 tx=0x7f0b6800b330 comp rx=0 tx=0).stop 2024-09-13T18:09:20.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b772f9700 1 -- 172.21.15.40:0/656583079 shutdown_connections 2024-09-13T18:09:20.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b772f9700 1 --2- 172.21.15.40:0/656583079 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7006c5b0 0x7f0b70163d50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:20.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b772f9700 1 --2- 172.21.15.40:0/656583079 >> v2:172.21.15.43:3300/0 conn(0x7f0b700cf9b0 0x7f0b7006c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:20.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b772f9700 1 -- 172.21.15.40:0/656583079 >> 172.21.15.40:0/656583079 conn(0x7f0b700666c0 msgr2=0x7f0b700cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:20.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b772f9700 1 -- 172.21.15.40:0/656583079 shutdown_connections 2024-09-13T18:09:20.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b772f9700 1 -- 172.21.15.40:0/656583079 wait complete. 2024-09-13T18:09:20.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b772f9700 1 Processor -- start 2024-09-13T18:09:20.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b772f9700 1 -- start start 2024-09-13T18:09:20.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b772f9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7006c5b0 0x7f0b70158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:20.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b772f9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b700cf9b0 0x7f0b70153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:20.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b772f9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b70153720 0x7f0b70157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:20.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b772f9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b701eeee0 con 0x7f0b7006c5b0 2024-09-13T18:09:20.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b772f9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b701ef0d0 con 0x7f0b700cf9b0 2024-09-13T18:09:20.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b772f9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b701ef3b0 con 0x7f0b70153720 2024-09-13T18:09:20.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b762f7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7006c5b0 0x7f0b70158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:20.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b762f7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7006c5b0 0x7f0b70158d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33672/0 (socket says 172.21.15.40:33672) 2024-09-13T18:09:20.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.157+0000 7f0b762f7700 1 -- 172.21.15.40:0/2286794327 learned_addr learned my addr 172.21.15.40:0/2286794327 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:20.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b75af6700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b700cf9b0 0x7f0b70153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:20.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b76af8700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b70153720 0x7f0b70157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:20.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b762f7700 1 -- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b70153720 msgr2=0x7f0b70157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:20.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b762f7700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b70153720 0x7f0b70157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:20.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b762f7700 1 -- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b700cf9b0 msgr2=0x7f0b70153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:20.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b762f7700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b700cf9b0 0x7f0b70153fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:20.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b762f7700 1 -- 172.21.15.40:0/2286794327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b68009b30 con 0x7f0b7006c5b0 2024-09-13T18:09:20.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b76af8700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b70153720 0x7f0b70157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:09:20.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b762f7700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7006c5b0 0x7f0b70158d20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000b4a0 tx=0x7f0b6000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:20.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b66ffd700 1 -- 172.21.15.40:0/2286794327 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60011830 con 0x7f0b7006c5b0 2024-09-13T18:09:20.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b66ffd700 1 -- 172.21.15.40:0/2286794327 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b60011990 con 0x7f0b7006c5b0 2024-09-13T18:09:20.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b772f9700 1 -- 172.21.15.40:0/2286794327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b701ef690 con 0x7f0b7006c5b0 2024-09-13T18:09:20.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b66ffd700 1 -- 172.21.15.40:0/2286794327 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60011d40 con 0x7f0b7006c5b0 2024-09-13T18:09:20.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b772f9700 1 -- 172.21.15.40:0/2286794327 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b701efc50 con 0x7f0b7006c5b0 2024-09-13T18:09:20.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b772f9700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b701553b0 0x7f0b70157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:20.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b772f9700 1 -- 172.21.15.40:0/2286794327 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b70074750 con 0x7f0b701553b0 2024-09-13T18:09:20.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b75af6700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b701553b0 0x7f0b70157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:20.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b75af6700 1 -- 172.21.15.40:0/2286794327 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b70159c40 con 0x7f0b701553b0 2024-09-13T18:09:20.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b66ffd700 1 -- 172.21.15.40:0/2286794327 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0b6001a400 con 0x7f0b7006c5b0 2024-09-13T18:09:20.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b75af6700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b701553b0 0x7f0b70157950 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0b700c4330 tx=0x7f0b6c00eb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:20.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.161+0000 7f0b66ffd700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0b54066730 0x7f0b54068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:20.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.165+0000 7f0b76af8700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0b54066730 0x7f0b54068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:20.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.165+0000 7f0b66ffd700 1 -- 172.21.15.40:0/2286794327 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0b60084370 con 0x7f0b7006c5b0 2024-09-13T18:09:20.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.165+0000 7f0b66ffd700 1 -- 172.21.15.40:0/2286794327 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c004020 con 0x7f0b701553b0 2024-09-13T18:09:20.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.165+0000 7f0b66ffd700 1 -- 172.21.15.40:0/2286794327 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0b6c004180 con 0x7f0b701553b0 2024-09-13T18:09:20.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.165+0000 7f0b66ffd700 1 -- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b701553b0 msgr2=0x7f0b70157950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:20.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.165+0000 7f0b66ffd700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b701553b0 0x7f0b70157950 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0b700c4330 tx=0x7f0b6c00eb00 comp rx=0 tx=0).stop 2024-09-13T18:09:20.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.165+0000 7f0b66ffd700 1 -- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b701553b0 msgr2=0x7f0b70157950 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:09:20.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.165+0000 7f0b66ffd700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b701553b0 0x7f0b70157950 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:20.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.165+0000 7f0b76af8700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0b54066730 0x7f0b54068bf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009fd0 tx=0x7f0b68009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:20.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.173+0000 7f0b772f9700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b70151fd0 0x7f0b7014c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:20.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.173+0000 7f0b772f9700 1 -- 172.21.15.40:0/2286794327 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b70152d40 con 0x7f0b70151fd0 2024-09-13T18:09:20.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.173+0000 7f0b75af6700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b70151fd0 0x7f0b7014c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:20.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.173+0000 7f0b75af6700 1 -- 172.21.15.40:0/2286794327 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0b700c4f40 con 0x7f0b70151fd0 2024-09-13T18:09:20.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.173+0000 7f0b75af6700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b70151fd0 0x7f0b7014c9b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0b700cc5d0 tx=0x7f0b6c004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:20.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.173+0000 7f0b66ffd700 1 -- 172.21.15.40:0/2286794327 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c01f020 con 0x7f0b70151fd0 2024-09-13T18:09:20.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.177+0000 7f0b66ffd700 1 -- 172.21.15.40:0/2286794327 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4451 (secure 0 0 0) 0x7f0b6c01f180 con 0x7f0b70151fd0 2024-09-13T18:09:20.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.177+0000 7f0b66ffd700 1 -- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b70151fd0 msgr2=0x7f0b7014c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:20.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.177+0000 7f0b66ffd700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b70151fd0 0x7f0b7014c9b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0b700cc5d0 tx=0x7f0b6c004770 comp rx=0 tx=0).stop 2024-09-13T18:09:20.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.177+0000 7f0b66ffd700 1 -- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b70151fd0 msgr2=0x7f0b7014c9b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:09:20.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.177+0000 7f0b66ffd700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0b70151fd0 0x7f0b7014c9b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:20.184 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:09:20.184 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:09:20.184 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:09:20.184 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:09:20.184 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 152, 2024-09-13T18:09:20.184 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:09:20.184 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:09:20.184 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:09:20.184 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:09:20.184 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:20.185 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 12, 2024-09-13T18:09:20.185 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:09:20.185 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:09:20.185 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:09:20.185 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:20.185 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:20.185 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:20.185 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:20.185 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:20.185 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:20.185 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:20.185 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:20.185 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:20.186 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:20.186 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:20.186 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:09:20.186 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:09:20.186 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:20.186 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:20.186 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:20.186 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:20.186 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:20.186 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:20.186 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:20.186 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:20.186 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:20.186 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:20.187 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:20.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.187 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:09:20.187 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:09:20.187 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:09:20.187 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:09:20.187 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:09:20.187 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:09:20.187 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:09:20.187 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:09:20.187 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:09:20.187 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:09:20.187 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:09:20.187 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:09:20.188 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:09:20.188 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:09:20.188 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:09:20.188 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:09:20.188 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:09:20.188 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:20.188 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:20.188 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:20.188 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:20.188 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:20.188 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:20.188 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:20.188 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:20.188 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:20.189 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:20.189 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:20.189 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:09:20.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.189 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:09:20.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.189 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:09:20.189 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:09:20.189 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:20.189 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:20.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.190 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:20.190 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:09:20.190 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:20.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.190 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:20.190 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:09:20.190 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:20.190 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:20.190 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:20.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.190 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:09:20.191 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:09:20.191 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:20.191 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:20.191 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:20.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.191 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:09:20.191 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:09:20.191 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:20.191 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:20.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.192 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:20.192 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:09:20.192 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:20.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.192 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:20.192 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:09:20.192 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:20.192 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:20.192 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:20.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.192 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:09:20.193 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:09:20.193 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:20.193 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:20.193 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:20.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.193 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:09:20.193 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:09:20.193 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:20.193 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:20.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.193 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:20.194 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:09:20.194 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:20.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.194 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:20.194 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:09:20.194 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:20.194 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:20.194 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:20.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.195 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:09:20.195 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:09:20.195 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:20.195 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:20.195 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:20.195 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:20.195 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:20.195 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.195 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:09:20.195 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:09:20.195 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.196 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:20.196 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:20.196 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:09:20.196 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:20.196 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:20.196 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:09:20.196 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:20.196 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:20.196 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:20.197 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:09:20.197 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:20.197 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:20.197 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:20.197 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:09:20.197 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:09:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.177+0000 7f0b772f9700 1 -- 172.21.15.40:0/2286794327 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0b54066730 msgr2=0x7f0b54068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.177+0000 7f0b772f9700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0b54066730 0x7f0b54068bf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009fd0 tx=0x7f0b68009000 comp rx=0 tx=0).stop 2024-09-13T18:09:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.177+0000 7f0b772f9700 1 -- 172.21.15.40:0/2286794327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7006c5b0 msgr2=0x7f0b70158d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.177+0000 7f0b772f9700 1 --2- 172.21.15.40:0/2286794327 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b7006c5b0 0x7f0b70158d20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000b4a0 tx=0x7f0b6000b860 comp rx=0 tx=0).stop 2024-09-13T18:09:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.177+0000 7f0b76af8700 1 -- 172.21.15.40:0/2286794327 reap_dead start 2024-09-13T18:09:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.177+0000 7f0b76af8700 1 -- 172.21.15.40:0/2286794327 reap_dead start 2024-09-13T18:09:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.177+0000 7f0b772f9700 1 -- 172.21.15.40:0/2286794327 shutdown_connections 2024-09-13T18:09:20.198 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.177+0000 7f0b772f9700 1 -- 172.21.15.40:0/2286794327 >> 172.21.15.40:0/2286794327 conn(0x7f0b700666c0 msgr2=0x7f0b700cba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:20.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.177+0000 7f0b772f9700 1 -- 172.21.15.40:0/2286794327 shutdown_connections 2024-09-13T18:09:20.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.177+0000 7f0b772f9700 1 -- 172.21.15.40:0/2286794327 wait complete. 2024-09-13T18:09:20.199 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:09:20.200 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-13T18:09:20.200 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T18:09:20.200 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-13T18:09:20.200 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T18:09:20.200 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T18:09:20.200 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:09:20.242 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T18:09:20.242 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T18:09:20.242 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:09:20.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b23fff700 1 Processor -- start 2024-09-13T18:09:20.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b23fff700 1 -- start start 2024-09-13T18:09:20.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b23fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b2406afd0 0x7f1b24071f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:20.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b23fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1b240c2e60 0x7f1b240706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:20.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b23fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b240717d0 con 0x7f1b2414fb00 2024-09-13T18:09:20.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b23fff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b24071b20 con 0x7f1b240c2e60 2024-09-13T18:09:20.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b23fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2406add0 con 0x7f1b2406afd0 2024-09-13T18:09:20.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b237fe700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f1b240c2e60 msgr2=0x7f1b240706e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:09:20.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b237fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1b240c2e60 0x7f1b240706e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:09:20.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b1bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b2406afd0 0x7f1b24071f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:20.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b1bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b2406afd0 0x7f1b24071f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:54694/0 (socket says 172.21.15.40:54694) 2024-09-13T18:09:20.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b1bfff700 1 -- 172.21.15.40:0/1544152866 learned_addr learned my addr 172.21.15.40:0/1544152866 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:20.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b1bfff700 1 -- 172.21.15.40:0/1544152866 >> v2:172.21.15.40:3300/0 conn(0x7f1b240c2e60 msgr2=0x7f1b240706e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:09:20.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b1bfff700 1 --2- 172.21.15.40:0/1544152866 >> v2:172.21.15.40:3300/0 conn(0x7f1b240c2e60 0x7f1b240706e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:20.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b1bfff700 1 -- 172.21.15.40:0/1544152866 >> v1:172.21.15.43:6789/0 conn(0x7f1b2414fb00 legacy=0x7f1b24158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:20.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b227fc700 1 -- 172.21.15.40:0/1544152866 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 998836771 0 0) 0x7f1b240717d0 con 0x7f1b2414fb00 2024-09-13T18:09:20.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b1bfff700 1 -- 172.21.15.40:0/1544152866 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b240c32f0 con 0x7f1b2406afd0 2024-09-13T18:09:20.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b1bfff700 1 --2- 172.21.15.40:0/1544152866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b2406afd0 0x7f1b24071f40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c009910 tx=0x7f1b0c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80e5ba62d0ca97bb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b227fc700 1 -- 172.21.15.40:0/1544152866 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c00ba60 con 0x7f1b2406afd0 2024-09-13T18:09:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b227fc700 1 -- 172.21.15.40:0/1544152866 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0c00bbc0 con 0x7f1b2406afd0 2024-09-13T18:09:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.353+0000 7f1b227fc700 1 -- 172.21.15.40:0/1544152866 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c009e60 con 0x7f1b2406afd0 2024-09-13T18:09:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b23fff700 1 -- 172.21.15.40:0/1544152866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b2406afd0 msgr2=0x7f1b24071f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b23fff700 1 --2- 172.21.15.40:0/1544152866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b2406afd0 0x7f1b24071f40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c009910 tx=0x7f1b0c009c20 comp rx=0 tx=0).stop 2024-09-13T18:09:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b23fff700 1 -- 172.21.15.40:0/1544152866 shutdown_connections 2024-09-13T18:09:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b23fff700 1 --2- 172.21.15.40:0/1544152866 >> v2:172.21.15.40:3300/0 conn(0x7f1b240c2e60 0x7f1b240706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:20.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b23fff700 1 --2- 172.21.15.40:0/1544152866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b2406afd0 0x7f1b24071f40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:20.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b23fff700 1 -- 172.21.15.40:0/1544152866 >> 172.21.15.40:0/1544152866 conn(0x7f1b240cabc0 msgr2=0x7f1b240cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:20.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b23fff700 1 -- 172.21.15.40:0/1544152866 shutdown_connections 2024-09-13T18:09:20.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b23fff700 1 -- 172.21.15.40:0/1544152866 wait complete. 2024-09-13T18:09:20.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b23fff700 1 Processor -- start 2024-09-13T18:09:20.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b23fff700 1 -- start start 2024-09-13T18:09:20.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b23fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b2406afd0 0x7f1b2414e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:20.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b23fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b240c2e60 0x7f1b241f2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:20.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b23fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b2414fb00 0x7f1b241f2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:20.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b23fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b24156820 con 0x7f1b240c2e60 2024-09-13T18:09:20.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b23fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b24159710 con 0x7f1b2406afd0 2024-09-13T18:09:20.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b23fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b241486c0 con 0x7f1b2414fb00 2024-09-13T18:09:20.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b1bfff700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b240c2e60 msgr2=0x7f1b241f2940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:09:20.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b1bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b240c2e60 0x7f1b241f2940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:09:20.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b237fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b2414fb00 0x7f1b241f2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:20.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b22ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b2406afd0 0x7f1b2414e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b237fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b2414fb00 0x7f1b241f2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:54696/0 (socket says 172.21.15.40:54696) 2024-09-13T18:09:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b237fe700 1 -- 172.21.15.40:0/545116087 learned_addr learned my addr 172.21.15.40:0/545116087 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b22ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b2406afd0 0x7f1b2414e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:34202/0 (socket says 172.21.15.40:34202) 2024-09-13T18:09:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b237fe700 1 -- 172.21.15.40:0/545116087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b2406afd0 msgr2=0x7f1b2414e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b237fe700 1 --2- 172.21.15.40:0/545116087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b2406afd0 0x7f1b2414e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b237fe700 1 -- 172.21.15.40:0/545116087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b240c2e60 msgr2=0x7f1b241f2940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:09:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b237fe700 1 --2- 172.21.15.40:0/545116087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b240c2e60 0x7f1b241f2940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b237fe700 1 -- 172.21.15.40:0/545116087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c009570 con 0x7f1b2414fb00 2024-09-13T18:09:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.357+0000 7f1b22ffd700 1 --2- 172.21.15.40:0/545116087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b2406afd0 0x7f1b2414e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:09:20.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.361+0000 7f1b237fe700 1 --2- 172.21.15.40:0/545116087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b2414fb00 0x7f1b241f2e80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c00be90 tx=0x7f1b1c00bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:20.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.361+0000 7f1b20ff9700 1 -- 172.21.15.40:0/545116087 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c00cc10 con 0x7f1b2414fb00 2024-09-13T18:09:20.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.361+0000 7f1b23fff700 1 -- 172.21.15.40:0/545116087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b24159980 con 0x7f1b2414fb00 2024-09-13T18:09:20.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.361+0000 7f1b20ff9700 1 -- 172.21.15.40:0/545116087 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1c00cd90 con 0x7f1b2414fb00 2024-09-13T18:09:20.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.361+0000 7f1b23fff700 1 -- 172.21.15.40:0/545116087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b24156540 con 0x7f1b2414fb00 2024-09-13T18:09:20.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.361+0000 7f1b20ff9700 1 -- 172.21.15.40:0/545116087 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c016670 con 0x7f1b2414fb00 2024-09-13T18:09:20.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.361+0000 7f1b23fff700 1 -- 172.21.15.40:0/545116087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b24159dc0 con 0x7f1b2414fb00 2024-09-13T18:09:20.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.361+0000 7f1b20ff9700 1 -- 172.21.15.40:0/545116087 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1b1c012020 con 0x7f1b2414fb00 2024-09-13T18:09:20.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.365+0000 7f1b20ff9700 1 --2- 172.21.15.40:0/545116087 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b040666e0 0x7f1b04068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:20.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.365+0000 7f1b20ff9700 1 -- 172.21.15.40:0/545116087 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1b1c0844f0 con 0x7f1b2414fb00 2024-09-13T18:09:20.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.365+0000 7f1b22ffd700 1 --2- 172.21.15.40:0/545116087 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b040666e0 0x7f1b04068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:20.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.365+0000 7f1b22ffd700 1 --2- 172.21.15.40:0/545116087 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b040666e0 0x7f1b04068ba0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1b14007fd0 tx=0x7f1b14009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:20.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.373+0000 7f1b20ff9700 1 -- 172.21.15.40:0/545116087 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1b1c051420 con 0x7f1b2414fb00 2024-09-13T18:09:20.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.653+0000 7f1b23fff700 1 -- 172.21.15.40:0/545116087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1b24148cc0 con 0x7f1b2414fb00 2024-09-13T18:09:20.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.657+0000 7f1b20ff9700 1 -- 172.21.15.40:0/545116087 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f1b1c054a40 con 0x7f1b2414fb00 2024-09-13T18:09:20.664 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":152,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:09:20.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.661+0000 7f1b23fff700 1 -- 172.21.15.40:0/545116087 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b040666e0 msgr2=0x7f1b04068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:20.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.661+0000 7f1b23fff700 1 --2- 172.21.15.40:0/545116087 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b040666e0 0x7f1b04068ba0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1b14007fd0 tx=0x7f1b14009040 comp rx=0 tx=0).stop 2024-09-13T18:09:20.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.661+0000 7f1b23fff700 1 -- 172.21.15.40:0/545116087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b2414fb00 msgr2=0x7f1b241f2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:20.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.661+0000 7f1b23fff700 1 --2- 172.21.15.40:0/545116087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b2414fb00 0x7f1b241f2e80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c00be90 tx=0x7f1b1c00bec0 comp rx=0 tx=0).stop 2024-09-13T18:09:20.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.661+0000 7f1b23fff700 1 -- 172.21.15.40:0/545116087 shutdown_connections 2024-09-13T18:09:20.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.661+0000 7f1b23fff700 1 --2- 172.21.15.40:0/545116087 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1b040666e0 0x7f1b04068ba0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:20.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.661+0000 7f1b23fff700 1 --2- 172.21.15.40:0/545116087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1b2414fb00 0x7f1b241f2e80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:20.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.661+0000 7f1b23fff700 1 --2- 172.21.15.40:0/545116087 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b240c2e60 0x7f1b241f2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:20.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.661+0000 7f1b23fff700 1 --2- 172.21.15.40:0/545116087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b2406afd0 0x7f1b2414e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:20.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.661+0000 7f1b23fff700 1 -- 172.21.15.40:0/545116087 >> 172.21.15.40:0/545116087 conn(0x7f1b240cabc0 msgr2=0x7f1b2414c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:20.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.661+0000 7f1b23fff700 1 -- 172.21.15.40:0/545116087 shutdown_connections 2024-09-13T18:09:20.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:20.661+0000 7f1b23fff700 1 -- 172.21.15.40:0/545116087 wait complete. 2024-09-13T18:09:23.683 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:09:23.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.809+0000 7f21e7213700 1 Processor -- start 2024-09-13T18:09:23.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.809+0000 7f21e7213700 1 -- start start 2024-09-13T18:09:23.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.809+0000 7f21e7213700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21d80d58d0 0x7f21d8046720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:23.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.809+0000 7f21e7213700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f21d80461d0 0x7f21d8033120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:23.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.809+0000 7f21e6211700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f21d80d58d0 msgr2=0x7f21d8046720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:09:23.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.809+0000 7f21e7213700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d8045fc0 con 0x7f21d8045470 2024-09-13T18:09:23.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.809+0000 7f21e7213700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d8004090 con 0x7f21d80d58d0 2024-09-13T18:09:23.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.809+0000 7f21e6211700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f21d80d58d0 0x7f21d8046720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:09:23.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.809+0000 7f21e6a12700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f21d8045470 legacy=0x7f21d80deb00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:09:23.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.809+0000 7f21e7213700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d8004330 con 0x7f21d80461d0 2024-09-13T18:09:23.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.809+0000 7f21e5a10700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f21d80461d0 0x7f21d8033120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:23.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.809+0000 7f21e5a10700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f21d80461d0 0x7f21d8033120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:34466/0 (socket says 172.21.15.40:34466) 2024-09-13T18:09:23.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.809+0000 7f21e5a10700 1 -- 172.21.15.40:0/1585769642 learned_addr learned my addr 172.21.15.40:0/1585769642 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:23.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.813+0000 7f21e5a10700 1 -- 172.21.15.40:0/1585769642 >> v2:172.21.15.40:3300/0 conn(0x7f21d80d58d0 msgr2=0x7f21d8046720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:09:23.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.813+0000 7f21e5a10700 1 --2- 172.21.15.40:0/1585769642 >> v2:172.21.15.40:3300/0 conn(0x7f21d80d58d0 0x7f21d8046720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:23.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.813+0000 7f21e5a10700 1 -- 172.21.15.40:0/1585769642 >> v1:172.21.15.40:6789/0 conn(0x7f21d8045470 legacy=0x7f21d80deb00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:09:23.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.813+0000 7f21e5a10700 1 -- 172.21.15.40:0/1585769642 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d8007980 con 0x7f21d80461d0 2024-09-13T18:09:23.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.813+0000 7f21e5a10700 1 --2- 172.21.15.40:0/1585769642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d80461d0 0x7f21d8033120 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f21d800d2b0 tx=0x7f21d000be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4ead324f5f0c50e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:23.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.813+0000 7f21e520f700 1 -- 172.21.15.40:0/1585769642 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d000dd50 con 0x7f21d80461d0 2024-09-13T18:09:23.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.813+0000 7f21e520f700 1 -- 172.21.15.40:0/1585769642 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d0004030 con 0x7f21d80461d0 2024-09-13T18:09:23.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.813+0000 7f21e520f700 1 -- 172.21.15.40:0/1585769642 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d00043c0 con 0x7f21d80461d0 2024-09-13T18:09:23.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.813+0000 7f21e7213700 1 -- 172.21.15.40:0/1585769642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d80461d0 msgr2=0x7f21d8033120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:23.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.813+0000 7f21e7213700 1 --2- 172.21.15.40:0/1585769642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d80461d0 0x7f21d8033120 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f21d800d2b0 tx=0x7f21d000be90 comp rx=0 tx=0).stop 2024-09-13T18:09:23.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.813+0000 7f21e7213700 1 -- 172.21.15.40:0/1585769642 shutdown_connections 2024-09-13T18:09:23.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.813+0000 7f21e7213700 1 --2- 172.21.15.40:0/1585769642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d80461d0 0x7f21d8033120 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:23.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.813+0000 7f21e7213700 1 --2- 172.21.15.40:0/1585769642 >> v2:172.21.15.40:3300/0 conn(0x7f21d80d58d0 0x7f21d8046720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:23.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.813+0000 7f21e7213700 1 -- 172.21.15.40:0/1585769642 >> 172.21.15.40:0/1585769642 conn(0x7f21d8050590 msgr2=0x7f21d80509a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:23.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.813+0000 7f21e7213700 1 -- 172.21.15.40:0/1585769642 shutdown_connections 2024-09-13T18:09:23.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.813+0000 7f21e7213700 1 -- 172.21.15.40:0/1585769642 wait complete. 2024-09-13T18:09:23.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.817+0000 7f21e7213700 1 Processor -- start 2024-09-13T18:09:23.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.817+0000 7f21e7213700 1 -- start start 2024-09-13T18:09:23.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.817+0000 7f21e7213700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d8045470 0x7f21d80c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:23.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.817+0000 7f21e7213700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21d80461d0 0x7f21d80c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:23.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.817+0000 7f21e7213700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21d80d58d0 0x7f21d80d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:23.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.817+0000 7f21e7213700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80073b0 con 0x7f21d80461d0 2024-09-13T18:09:23.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.817+0000 7f21e7213700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80cfb70 con 0x7f21d8045470 2024-09-13T18:09:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.817+0000 7f21e7213700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80c5660 con 0x7f21d80d58d0 2024-09-13T18:09:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.817+0000 7f21e5a10700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21d80461d0 msgr2=0x7f21d80c4bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:09:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.817+0000 7f21e5a10700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21d80461d0 0x7f21d80c4bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:09:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.817+0000 7f21e6211700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d8045470 0x7f21d80c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.817+0000 7f21e6211700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d8045470 0x7f21d80c5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:34472/0 (socket says 172.21.15.40:34472) 2024-09-13T18:09:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.817+0000 7f21e6211700 1 -- 172.21.15.40:0/1630588308 learned_addr learned my addr 172.21.15.40:0/1630588308 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.817+0000 7f21e6a12700 1 --2- 172.21.15.40:0/1630588308 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21d80d58d0 0x7f21d80d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:23.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.817+0000 7f21e6211700 1 -- 172.21.15.40:0/1630588308 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21d80d58d0 msgr2=0x7f21d80d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:23.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.817+0000 7f21e6211700 1 --2- 172.21.15.40:0/1630588308 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21d80d58d0 0x7f21d80d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:23.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.817+0000 7f21e6211700 1 -- 172.21.15.40:0/1630588308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21d80461d0 msgr2=0x7f21d80c4bf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:09:23.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.817+0000 7f21e6211700 1 --2- 172.21.15.40:0/1630588308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21d80461d0 0x7f21d80c4bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:23.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.817+0000 7f21e6211700 1 -- 172.21.15.40:0/1630588308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d000b810 con 0x7f21d8045470 2024-09-13T18:09:23.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.821+0000 7f21e6211700 1 --2- 172.21.15.40:0/1630588308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d8045470 0x7f21d80c5bb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00b570 tx=0x7f21dc00b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:23.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.821+0000 7f21d6ffd700 1 -- 172.21.15.40:0/1630588308 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc003e70 con 0x7f21d8045470 2024-09-13T18:09:23.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.821+0000 7f21d6ffd700 1 -- 172.21.15.40:0/1630588308 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21dc005070 con 0x7f21d8045470 2024-09-13T18:09:23.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.821+0000 7f21e7213700 1 -- 172.21.15.40:0/1630588308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d80d1960 con 0x7f21d8045470 2024-09-13T18:09:23.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.821+0000 7f21d6ffd700 1 -- 172.21.15.40:0/1630588308 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc005440 con 0x7f21d8045470 2024-09-13T18:09:23.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.821+0000 7f21e7213700 1 -- 172.21.15.40:0/1630588308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d80070d0 con 0x7f21d8045470 2024-09-13T18:09:23.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.821+0000 7f21e7213700 1 -- 172.21.15.40:0/1630588308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d8045bf0 con 0x7f21d8045470 2024-09-13T18:09:23.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.821+0000 7f21d6ffd700 1 -- 172.21.15.40:0/1630588308 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f21dc0055a0 con 0x7f21d8045470 2024-09-13T18:09:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.821+0000 7f21d6ffd700 1 --2- 172.21.15.40:0/1630588308 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f21c40666e0 0x7f21c4068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.825+0000 7f21e5a10700 1 --2- 172.21.15.40:0/1630588308 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f21c40666e0 0x7f21c4068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.825+0000 7f21d6ffd700 1 -- 172.21.15.40:0/1630588308 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f21dc083930 con 0x7f21d8045470 2024-09-13T18:09:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.825+0000 7f21e5a10700 1 --2- 172.21.15.40:0/1630588308 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f21c40666e0 0x7f21c4068ba0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f21d80c6720 tx=0x7f21d0013000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:23.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:23.837+0000 7f21d6ffd700 1 -- 172.21.15.40:0/1630588308 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f21dc050840 con 0x7f21d8045470 2024-09-13T18:09:24.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:24.081+0000 7f21e7213700 1 -- 172.21.15.40:0/1630588308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f21d8004540 con 0x7f21d8045470 2024-09-13T18:09:24.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:24.085+0000 7f21d6ffd700 1 -- 172.21.15.40:0/1630588308 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f21dc016080 con 0x7f21d8045470 2024-09-13T18:09:24.092 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":152,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:09:24.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:24.085+0000 7f21e7213700 1 -- 172.21.15.40:0/1630588308 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f21c40666e0 msgr2=0x7f21c4068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:24.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:24.085+0000 7f21e7213700 1 --2- 172.21.15.40:0/1630588308 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f21c40666e0 0x7f21c4068ba0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f21d80c6720 tx=0x7f21d0013000 comp rx=0 tx=0).stop 2024-09-13T18:09:24.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:24.085+0000 7f21e7213700 1 -- 172.21.15.40:0/1630588308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d8045470 msgr2=0x7f21d80c5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:24.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:24.085+0000 7f21e7213700 1 --2- 172.21.15.40:0/1630588308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d8045470 0x7f21d80c5bb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00b570 tx=0x7f21dc00b880 comp rx=0 tx=0).stop 2024-09-13T18:09:24.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:24.085+0000 7f21e7213700 1 -- 172.21.15.40:0/1630588308 shutdown_connections 2024-09-13T18:09:24.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:24.085+0000 7f21e7213700 1 --2- 172.21.15.40:0/1630588308 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f21c40666e0 0x7f21c4068ba0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:24.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:24.085+0000 7f21e7213700 1 --2- 172.21.15.40:0/1630588308 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f21d80d58d0 0x7f21d80d1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:24.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:24.085+0000 7f21e7213700 1 --2- 172.21.15.40:0/1630588308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f21d80461d0 0x7f21d80c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:24.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:24.085+0000 7f21e7213700 1 --2- 172.21.15.40:0/1630588308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d8045470 0x7f21d80c5bb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:24.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:24.085+0000 7f21e7213700 1 -- 172.21.15.40:0/1630588308 >> 172.21.15.40:0/1630588308 conn(0x7f21d8050590 msgr2=0x7f21d80d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:24.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:24.085+0000 7f21e7213700 1 -- 172.21.15.40:0/1630588308 shutdown_connections 2024-09-13T18:09:24.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:24.085+0000 7f21e7213700 1 -- 172.21.15.40:0/1630588308 wait complete. 2024-09-13T18:09:27.108 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:09:27.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.233+0000 7fc874fcd700 1 Processor -- start 2024-09-13T18:09:27.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.233+0000 7fc874fcd700 1 -- start start 2024-09-13T18:09:27.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.233+0000 7fc874fcd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8700714d0 0x7fc8700718e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:27.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.233+0000 7fc874fcd700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc870071040 con 0x7fc87014fb40 2024-09-13T18:09:27.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.233+0000 7fc86f7fe700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7fc87014fb40 legacy=0x7fc8700c6410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:09:27.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.233+0000 7fc874fcd700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc870070d40 con 0x7fc87006e130 2024-09-13T18:09:27.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.233+0000 7fc874fcd700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87014ff20 con 0x7fc8700714d0 2024-09-13T18:09:27.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.233+0000 7fc86effd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc87006e130 0x7fc870158de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:48298/0 (socket says 172.21.15.40:48298) 2024-09-13T18:09:27.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.233+0000 7fc86effd700 1 -- 172.21.15.40:0/4148643212 learned_addr learned my addr 172.21.15.40:0/4148643212 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:09:27.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.233+0000 7fc86ffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8700714d0 0x7fc8700718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:27.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.237+0000 7fc86ffff700 1 -- 172.21.15.40:0/4148643212 >> v1:172.21.15.43:6789/0 conn(0x7fc87006e130 legacy=0x7fc870158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:27.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.237+0000 7fc86ffff700 1 -- 172.21.15.40:0/4148643212 >> v1:172.21.15.40:6789/0 conn(0x7fc87014fb40 legacy=0x7fc8700c6410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:09:27.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.237+0000 7fc86ffff700 1 -- 172.21.15.40:0/4148643212 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87006e700 con 0x7fc8700714d0 2024-09-13T18:09:27.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.237+0000 7fc86ffff700 1 --2- 172.21.15.40:0/4148643212 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8700714d0 0x7fc8700718e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc868009dc0 tx=0x7fc86800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baaf3b0521ddacc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:27.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.237+0000 7fc86e7fc700 1 -- 172.21.15.40:0/4148643212 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86800e070 con 0x7fc8700714d0 2024-09-13T18:09:27.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.237+0000 7fc86e7fc700 1 -- 172.21.15.40:0/4148643212 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86800bb90 con 0x7fc8700714d0 2024-09-13T18:09:27.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.237+0000 7fc86e7fc700 1 -- 172.21.15.40:0/4148643212 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868003a40 con 0x7fc8700714d0 2024-09-13T18:09:27.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.237+0000 7fc874fcd700 1 -- 172.21.15.40:0/4148643212 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8700714d0 msgr2=0x7fc8700718e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:27.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.237+0000 7fc874fcd700 1 --2- 172.21.15.40:0/4148643212 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8700714d0 0x7fc8700718e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc868009dc0 tx=0x7fc86800b330 comp rx=0 tx=0).stop 2024-09-13T18:09:27.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.237+0000 7fc874fcd700 1 -- 172.21.15.40:0/4148643212 shutdown_connections 2024-09-13T18:09:27.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.237+0000 7fc874fcd700 1 --2- 172.21.15.40:0/4148643212 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8700714d0 0x7fc8700718e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:27.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.237+0000 7fc874fcd700 1 -- 172.21.15.40:0/4148643212 >> 172.21.15.40:0/4148643212 conn(0x7fc8700cb4c0 msgr2=0x7fc8700cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:27.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.237+0000 7fc874fcd700 1 -- 172.21.15.40:0/4148643212 shutdown_connections 2024-09-13T18:09:27.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.237+0000 7fc874fcd700 1 -- 172.21.15.40:0/4148643212 wait complete. 2024-09-13T18:09:27.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc874fcd700 1 Processor -- start 2024-09-13T18:09:27.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc874fcd700 1 -- start start 2024-09-13T18:09:27.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc874fcd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc87006e130 0x7fc8701557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:27.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc874fcd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8700714d0 0x7fc8701f2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:27.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc874fcd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc87014fb40 0x7fc8701f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:27.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc874fcd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8700c5720 con 0x7fc87014fb40 2024-09-13T18:09:27.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc874fcd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8701efec0 con 0x7fc87006e130 2024-09-13T18:09:27.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc874fcd700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8701ef030 con 0x7fc8700714d0 2024-09-13T18:09:27.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc86ffff700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc87014fb40 msgr2=0x7fc8701f2c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:09:27.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc86ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc87014fb40 0x7fc8701f2c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:09:27.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc86effd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8700714d0 0x7fc8701f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:27.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc86f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc87006e130 0x7fc8701557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:27.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc86f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc87006e130 0x7fc8701557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:34488/0 (socket says 172.21.15.40:34488) 2024-09-13T18:09:27.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc86effd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8700714d0 0x7fc8701f2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:47682/0 (socket says 172.21.15.40:47682) 2024-09-13T18:09:27.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc86f7fe700 1 -- 172.21.15.40:0/3835002891 learned_addr learned my addr 172.21.15.40:0/3835002891 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:27.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc86effd700 1 -- 172.21.15.40:0/3835002891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc87006e130 msgr2=0x7fc8701557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:27.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc86effd700 1 --2- 172.21.15.40:0/3835002891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc87006e130 0x7fc8701557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:27.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc86effd700 1 -- 172.21.15.40:0/3835002891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc87014fb40 msgr2=0x7fc8701f2c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:09:27.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc86effd700 1 --2- 172.21.15.40:0/3835002891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc87014fb40 0x7fc8701f2c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:27.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc86effd700 1 -- 172.21.15.40:0/3835002891 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc868009a70 con 0x7fc8700714d0 2024-09-13T18:09:27.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc86f7fe700 1 --2- 172.21.15.40:0/3835002891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc87006e130 0x7fc8701557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:09:27.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc86effd700 1 --2- 172.21.15.40:0/3835002891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8700714d0 0x7fc8701f2750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc86400c480 tx=0x7fc86400c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:27.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc857fff700 1 -- 172.21.15.40:0/3835002891 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864005d90 con 0x7fc8700714d0 2024-09-13T18:09:27.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc857fff700 1 -- 172.21.15.40:0/3835002891 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8640070d0 con 0x7fc8700714d0 2024-09-13T18:09:27.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc874fcd700 1 -- 172.21.15.40:0/3835002891 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8700bb450 con 0x7fc8700714d0 2024-09-13T18:09:27.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc857fff700 1 -- 172.21.15.40:0/3835002891 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864007440 con 0x7fc8700714d0 2024-09-13T18:09:27.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.241+0000 7fc874fcd700 1 -- 172.21.15.40:0/3835002891 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8701ef520 con 0x7fc8700714d0 2024-09-13T18:09:27.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.245+0000 7fc874fcd700 1 -- 172.21.15.40:0/3835002891 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8701599b0 con 0x7fc8700714d0 2024-09-13T18:09:27.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.245+0000 7fc857fff700 1 -- 172.21.15.40:0/3835002891 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc8640075a0 con 0x7fc8700714d0 2024-09-13T18:09:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.245+0000 7fc857fff700 1 --2- 172.21.15.40:0/3835002891 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc85c0666e0 0x7fc85c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.245+0000 7fc857fff700 1 -- 172.21.15.40:0/3835002891 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc864083b00 con 0x7fc8700714d0 2024-09-13T18:09:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.245+0000 7fc86f7fe700 1 --2- 172.21.15.40:0/3835002891 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc85c0666e0 0x7fc85c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:27.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.245+0000 7fc86f7fe700 1 --2- 172.21.15.40:0/3835002891 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc85c0666e0 0x7fc85c068ba0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc860006fd0 tx=0x7fc860006f00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:27.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.253+0000 7fc857fff700 1 -- 172.21.15.40:0/3835002891 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc864050d90 con 0x7fc8700714d0 2024-09-13T18:09:27.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.505+0000 7fc874fcd700 1 -- 172.21.15.40:0/3835002891 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc8700c6410 con 0x7fc8700714d0 2024-09-13T18:09:27.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.509+0000 7fc857fff700 1 -- 172.21.15.40:0/3835002891 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7fc864021620 con 0x7fc8700714d0 2024-09-13T18:09:27.516 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":152,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:09:27.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.509+0000 7fc874fcd700 1 -- 172.21.15.40:0/3835002891 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc85c0666e0 msgr2=0x7fc85c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:27.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.509+0000 7fc874fcd700 1 --2- 172.21.15.40:0/3835002891 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc85c0666e0 0x7fc85c068ba0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc860006fd0 tx=0x7fc860006f00 comp rx=0 tx=0).stop 2024-09-13T18:09:27.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.509+0000 7fc874fcd700 1 -- 172.21.15.40:0/3835002891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8700714d0 msgr2=0x7fc8701f2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:27.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.509+0000 7fc874fcd700 1 --2- 172.21.15.40:0/3835002891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8700714d0 0x7fc8701f2750 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc86400c480 tx=0x7fc86400c790 comp rx=0 tx=0).stop 2024-09-13T18:09:27.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.509+0000 7fc874fcd700 1 -- 172.21.15.40:0/3835002891 shutdown_connections 2024-09-13T18:09:27.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.509+0000 7fc874fcd700 1 --2- 172.21.15.40:0/3835002891 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc87014fb40 0x7fc8701f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:27.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.509+0000 7fc874fcd700 1 --2- 172.21.15.40:0/3835002891 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc85c0666e0 0x7fc85c068ba0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:27.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.509+0000 7fc874fcd700 1 --2- 172.21.15.40:0/3835002891 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc8700714d0 0x7fc8701f2750 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:27.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.509+0000 7fc874fcd700 1 --2- 172.21.15.40:0/3835002891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc87006e130 0x7fc8701557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:27.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.509+0000 7fc874fcd700 1 -- 172.21.15.40:0/3835002891 >> 172.21.15.40:0/3835002891 conn(0x7fc8700cb4c0 msgr2=0x7fc8700c71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:27.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.509+0000 7fc874fcd700 1 -- 172.21.15.40:0/3835002891 shutdown_connections 2024-09-13T18:09:27.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:27.509+0000 7fc874fcd700 1 -- 172.21.15.40:0/3835002891 wait complete. 2024-09-13T18:09:30.532 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:09:30.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:30.657+0000 7f859bfff700 1 Processor -- start 2024-09-13T18:09:30.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:30.657+0000 7f859bfff700 1 -- start start 2024-09-13T18:09:30.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:30.657+0000 7f859bfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 0x7f859c070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:30.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:30.657+0000 7f859bfff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f859c14fdd0 con 0x7f859c06ea60 2024-09-13T18:09:30.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:30.657+0000 7f859bfff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f859c1558b0 con 0x7f859c071620 2024-09-13T18:09:30.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:30.657+0000 7f859bfff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f859c070920 con 0x7f859c14f9f0 2024-09-13T18:09:30.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:30.657+0000 7f8593fff700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f859c06ea60 legacy=0x7f859c070ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:09:30.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:30.657+0000 7f859affd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 0x7f859c070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:30.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:30.657+0000 7f859b7fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f859c071620 0x7f859c06d570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:48306/0 (socket says 172.21.15.40:48306) 2024-09-13T18:09:30.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:30.657+0000 7f859b7fe700 1 -- 172.21.15.40:0/2780585490 learned_addr learned my addr 172.21.15.40:0/2780585490 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:09:30.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:30.657+0000 7f859affd700 1 -- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 msgr2=0x7f859c070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:09:30.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:30.657+0000 7f859affd700 1 -- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 msgr2=0x7f859c070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:09:30.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:30.657+0000 7f859affd700 1 --2- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 0x7f859c070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:09:30.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:30.657+0000 7f859affd700 1 --2- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 0x7f859c070290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:09:30.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:30.861+0000 7f8593fff700 1 -- 172.21.15.40:0/2780585490 >> v1:172.21.15.40:6789/0 conn(0x7f859c06ea60 legacy=0x7f859c070ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:09:30.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:30.861+0000 7f859affd700 1 --2- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 0x7f859c070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:30.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:30.861+0000 7f859affd700 1 -- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 msgr2=0x7f859c070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:09:30.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:30.861+0000 7f859affd700 1 -- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 msgr2=0x7f859c070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:09:30.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:30.861+0000 7f859affd700 1 --2- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 0x7f859c070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:09:30.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:30.861+0000 7f859affd700 1 --2- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 0x7f859c070290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:09:31.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:31.265+0000 7f8593fff700 1 -- 172.21.15.40:0/2780585490 >> v1:172.21.15.40:6789/0 conn(0x7f859c06ea60 legacy=0x7f859c070ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:09:31.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:31.265+0000 7f859affd700 1 --2- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 0x7f859c070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:31.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:31.265+0000 7f859affd700 1 -- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 msgr2=0x7f859c070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:09:31.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:31.265+0000 7f859affd700 1 -- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 msgr2=0x7f859c070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:09:31.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:31.265+0000 7f859affd700 1 --2- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 0x7f859c070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:09:31.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:31.265+0000 7f859affd700 1 --2- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 0x7f859c070290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:09:32.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:32.069+0000 7f8593fff700 1 -- 172.21.15.40:0/2780585490 >> v1:172.21.15.40:6789/0 conn(0x7f859c06ea60 legacy=0x7f859c070ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:09:32.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:32.069+0000 7f859affd700 1 --2- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 0x7f859c070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:32.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:32.069+0000 7f859affd700 1 -- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 msgr2=0x7f859c070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:09:32.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:32.069+0000 7f859affd700 1 -- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 msgr2=0x7f859c070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:09:32.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:32.069+0000 7f859affd700 1 --2- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 0x7f859c070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:09:32.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:32.069+0000 7f859affd700 1 --2- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 0x7f859c070290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:09:33.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.657+0000 7f85997fa700 1 -- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 msgr2=0x7f859c070290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:09:33.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.657+0000 7f85997fa700 1 --2- 172.21.15.40:0/2780585490 >> v2:172.21.15.43:3300/0 conn(0x7f859c14f9f0 0x7f859c070290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:33.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.657+0000 7f85997fa700 1 -- 172.21.15.40:0/2780585490 >> v1:172.21.15.43:6789/0 conn(0x7f859c071620 legacy=0x7f859c06d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:33.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.657+0000 7f85997fa700 1 -- 172.21.15.40:0/2780585490 >> v1:172.21.15.40:6789/0 conn(0x7f859c06ea60 legacy=0x7f859c070ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:09:33.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.657+0000 7f85997fa700 1 --2- 172.21.15.40:0/2780585490 >> v2:172.21.15.40:3300/0 conn(0x7f857c003720 0x7f857c005ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:33.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.657+0000 7f85997fa700 1 --2- 172.21.15.40:0/2780585490 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f857c006190 0x7f857c008650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:33.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.657+0000 7f85997fa700 1 -- 172.21.15.40:0/2780585490 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f859c070920 con 0x7f857c003720 2024-09-13T18:09:33.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.657+0000 7f85997fa700 1 -- 172.21.15.40:0/2780585490 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f857c008d70 con 0x7f857c006190 2024-09-13T18:09:33.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.657+0000 7f859affd700 1 -- 172.21.15.40:0/2780585490 >> v2:172.21.15.40:3300/0 conn(0x7f857c003720 msgr2=0x7f857c005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:09:33.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.657+0000 7f859affd700 1 --2- 172.21.15.40:0/2780585490 >> v2:172.21.15.40:3300/0 conn(0x7f857c003720 0x7f857c005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:09:33.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.657+0000 7f8593fff700 1 --2- 172.21.15.40:0/2780585490 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f857c006190 0x7f857c008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:33.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.661+0000 7f8593fff700 1 -- 172.21.15.40:0/2780585490 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f857c006190 msgr2=0x7f857c008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:09:33.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.661+0000 7f8593fff700 1 -- 172.21.15.40:0/2780585490 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f857c006190 msgr2=0x7f857c008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:09:33.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.661+0000 7f8593fff700 1 --2- 172.21.15.40:0/2780585490 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f857c006190 0x7f857c008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:09:33.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.661+0000 7f8593fff700 1 --2- 172.21.15.40:0/2780585490 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f857c006190 0x7f857c008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:09:33.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.861+0000 7f859affd700 1 -- 172.21.15.40:0/2780585490 >> v2:172.21.15.40:3300/0 conn(0x7f857c003720 msgr2=0x7f857c005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:09:33.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.861+0000 7f859affd700 1 --2- 172.21.15.40:0/2780585490 >> v2:172.21.15.40:3300/0 conn(0x7f857c003720 0x7f857c005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:09:33.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.865+0000 7f8593fff700 1 --2- 172.21.15.40:0/2780585490 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f857c006190 0x7f857c008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:33.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.865+0000 7f8593fff700 1 -- 172.21.15.40:0/2780585490 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f857c006190 msgr2=0x7f857c008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:09:33.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.865+0000 7f8593fff700 1 -- 172.21.15.40:0/2780585490 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f857c006190 msgr2=0x7f857c008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:09:33.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.865+0000 7f8593fff700 1 --2- 172.21.15.40:0/2780585490 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f857c006190 0x7f857c008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:09:33.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:33.865+0000 7f8593fff700 1 --2- 172.21.15.40:0/2780585490 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f857c006190 0x7f857c008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:09:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:34.265+0000 7f859affd700 1 -- 172.21.15.40:0/2780585490 >> v2:172.21.15.40:3300/0 conn(0x7f857c003720 msgr2=0x7f857c005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:09:34.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:34.265+0000 7f859affd700 1 --2- 172.21.15.40:0/2780585490 >> v2:172.21.15.40:3300/0 conn(0x7f857c003720 0x7f857c005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:09:34.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:34.269+0000 7f8593fff700 1 --2- 172.21.15.40:0/2780585490 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f857c006190 0x7f857c008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:34.269+0000 7f8593fff700 1 -- 172.21.15.40:0/2780585490 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f857c006190 msgr2=0x7f857c008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:09:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:34.269+0000 7f8593fff700 1 -- 172.21.15.40:0/2780585490 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f857c006190 msgr2=0x7f857c008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:09:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:34.269+0000 7f8593fff700 1 --2- 172.21.15.40:0/2780585490 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f857c006190 0x7f857c008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:09:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:34.269+0000 7f8593fff700 1 --2- 172.21.15.40:0/2780585490 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f857c006190 0x7f857c008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:09:35.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.069+0000 7f859affd700 1 -- 172.21.15.40:0/2780585490 >> v2:172.21.15.40:3300/0 conn(0x7f857c003720 msgr2=0x7f857c005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:09:35.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.069+0000 7f859affd700 1 --2- 172.21.15.40:0/2780585490 >> v2:172.21.15.40:3300/0 conn(0x7f857c003720 0x7f857c005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:09:35.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.073+0000 7f8593fff700 1 --2- 172.21.15.40:0/2780585490 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f857c006190 0x7f857c008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:35.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.073+0000 7f8593fff700 1 -- 172.21.15.40:0/2780585490 >> v2:172.21.15.40:3300/0 conn(0x7f857c003720 msgr2=0x7f857c005ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:09:35.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.073+0000 7f8593fff700 1 --2- 172.21.15.40:0/2780585490 >> v2:172.21.15.40:3300/0 conn(0x7f857c003720 0x7f857c005ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.073+0000 7f8593fff700 1 -- 172.21.15.40:0/2780585490 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859c155b50 con 0x7f857c006190 2024-09-13T18:09:35.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.073+0000 7f8593fff700 1 --2- 172.21.15.40:0/2780585490 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f857c006190 0x7f857c008650 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8584003a30 tx=0x7f858400b6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f9ad0a65b87f2e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:35.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.073+0000 7f859a7fc700 1 -- 172.21.15.40:0/2780585490 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8584016610 con 0x7f857c006190 2024-09-13T18:09:35.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.073+0000 7f859a7fc700 1 -- 172.21.15.40:0/2780585490 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8584011050 con 0x7f857c006190 2024-09-13T18:09:35.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.073+0000 7f859a7fc700 1 -- 172.21.15.40:0/2780585490 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8584016990 con 0x7f857c006190 2024-09-13T18:09:35.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.073+0000 7f859bfff700 1 -- 172.21.15.40:0/2780585490 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f857c006190 msgr2=0x7f857c008650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:35.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.073+0000 7f859bfff700 1 --2- 172.21.15.40:0/2780585490 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f857c006190 0x7f857c008650 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8584003a30 tx=0x7f858400b6d0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.073+0000 7f859b7fe700 1 -- 172.21.15.40:0/2780585490 reap_dead start 2024-09-13T18:09:35.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.073+0000 7f859bfff700 1 -- 172.21.15.40:0/2780585490 shutdown_connections 2024-09-13T18:09:35.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.073+0000 7f859bfff700 1 -- 172.21.15.40:0/2780585490 >> 172.21.15.40:0/2780585490 conn(0x7f859c06a390 msgr2=0x7f859c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:35.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.073+0000 7f859bfff700 1 -- 172.21.15.40:0/2780585490 shutdown_connections 2024-09-13T18:09:35.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.073+0000 7f859bfff700 1 -- 172.21.15.40:0/2780585490 wait complete. 2024-09-13T18:09:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.077+0000 7f859bfff700 1 Processor -- start 2024-09-13T18:09:35.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.077+0000 7f859bfff700 1 -- start start 2024-09-13T18:09:35.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.077+0000 7f859bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f857c006190 0x7f859c1f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.077+0000 7f859bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f859c0c1c40 0x7f859c1f1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f859bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859c06e0f0 0x7f859c1f1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f859bfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859c1564d0 con 0x7f859c06e0f0 2024-09-13T18:09:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f859bfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859c1594e0 con 0x7f857c006190 2024-09-13T18:09:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f859bfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f859c0ca860 con 0x7f859c0c1c40 2024-09-13T18:09:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f859b7fe700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859c06e0f0 msgr2=0x7f859c1f1b60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:09:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f859b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859c06e0f0 0x7f859c1f1b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:09:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f8593fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f859c0c1c40 0x7f859c1f1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f859affd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f857c006190 0x7f859c1f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f8593fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f859c0c1c40 0x7f859c1f1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:51632/0 (socket says 172.21.15.40:51632) 2024-09-13T18:09:35.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f8593fff700 1 -- 172.21.15.40:0/1135272404 learned_addr learned my addr 172.21.15.40:0/1135272404 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f8593fff700 1 -- 172.21.15.40:0/1135272404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f857c006190 msgr2=0x7f859c1f23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f8593fff700 1 --2- 172.21.15.40:0/1135272404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f857c006190 0x7f859c1f23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f8593fff700 1 -- 172.21.15.40:0/1135272404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859c06e0f0 msgr2=0x7f859c1f1b60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:09:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f8593fff700 1 --2- 172.21.15.40:0/1135272404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859c06e0f0 0x7f859c1f1b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f8593fff700 1 -- 172.21.15.40:0/1135272404 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8584009a20 con 0x7f859c0c1c40 2024-09-13T18:09:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f8593fff700 1 --2- 172.21.15.40:0/1135272404 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f859c0c1c40 0x7f859c1f1620 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f857c008f00 tx=0x7f8584005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f85937fe700 1 -- 172.21.15.40:0/1135272404 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8584020620 con 0x7f859c0c1c40 2024-09-13T18:09:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f85937fe700 1 -- 172.21.15.40:0/1135272404 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85840207a0 con 0x7f859c0c1c40 2024-09-13T18:09:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f859bfff700 1 -- 172.21.15.40:0/1135272404 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f859c0bc950 con 0x7f859c0c1c40 2024-09-13T18:09:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f85937fe700 1 -- 172.21.15.40:0/1135272404 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8584020b50 con 0x7f859c0c1c40 2024-09-13T18:09:35.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.081+0000 7f859bfff700 1 -- 172.21.15.40:0/1135272404 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859c1f2c70 con 0x7f859c0c1c40 2024-09-13T18:09:35.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.085+0000 7f859bfff700 1 -- 172.21.15.40:0/1135272404 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f859c070290 con 0x7f859c0c1c40 2024-09-13T18:09:35.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.085+0000 7f85937fe700 1 -- 172.21.15.40:0/1135272404 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f858401d050 con 0x7f859c0c1c40 2024-09-13T18:09:35.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.085+0000 7f85937fe700 1 --2- 172.21.15.40:0/1135272404 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f857c067480 0x7f857c069940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:35.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.085+0000 7f85937fe700 1 -- 172.21.15.40:0/1135272404 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8584085de0 con 0x7f859c0c1c40 2024-09-13T18:09:35.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.085+0000 7f859affd700 1 --2- 172.21.15.40:0/1135272404 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f857c067480 0x7f857c069940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:35.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.085+0000 7f859affd700 1 --2- 172.21.15.40:0/1135272404 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f857c067480 0x7f857c069940 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f858c006df0 tx=0x7f858c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:35.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.097+0000 7f85937fe700 1 -- 172.21.15.40:0/1135272404 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8584052e20 con 0x7f859c0c1c40 2024-09-13T18:09:35.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.361+0000 7f859bfff700 1 -- 172.21.15.40:0/1135272404 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f859c0bc950 con 0x7f859c0c1c40 2024-09-13T18:09:35.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.361+0000 7f85937fe700 1 -- 172.21.15.40:0/1135272404 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f859c0bc950 con 0x7f859c0c1c40 2024-09-13T18:09:35.371 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":154,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:09:35.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.365+0000 7f859bfff700 1 -- 172.21.15.40:0/1135272404 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f857c067480 msgr2=0x7f857c069940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:35.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.365+0000 7f859bfff700 1 --2- 172.21.15.40:0/1135272404 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f857c067480 0x7f857c069940 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f858c006df0 tx=0x7f858c008040 comp rx=0 tx=0).stop 2024-09-13T18:09:35.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.365+0000 7f859bfff700 1 -- 172.21.15.40:0/1135272404 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f859c0c1c40 msgr2=0x7f859c1f1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:35.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.365+0000 7f859bfff700 1 --2- 172.21.15.40:0/1135272404 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f859c0c1c40 0x7f859c1f1620 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f857c008f00 tx=0x7f8584005e60 comp rx=0 tx=0).stop 2024-09-13T18:09:35.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.365+0000 7f859bfff700 1 -- 172.21.15.40:0/1135272404 shutdown_connections 2024-09-13T18:09:35.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.365+0000 7f859bfff700 1 --2- 172.21.15.40:0/1135272404 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f859c06e0f0 0x7f859c1f1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.365+0000 7f859bfff700 1 --2- 172.21.15.40:0/1135272404 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f857c067480 0x7f857c069940 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.365+0000 7f859bfff700 1 --2- 172.21.15.40:0/1135272404 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f859c0c1c40 0x7f859c1f1620 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.365+0000 7f859bfff700 1 --2- 172.21.15.40:0/1135272404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f857c006190 0x7f859c1f23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.365+0000 7f859bfff700 1 -- 172.21.15.40:0/1135272404 >> 172.21.15.40:0/1135272404 conn(0x7f859c06a390 msgr2=0x7f859c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:35.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.365+0000 7f859bfff700 1 -- 172.21.15.40:0/1135272404 shutdown_connections 2024-09-13T18:09:35.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.365+0000 7f859bfff700 1 -- 172.21.15.40:0/1135272404 wait complete. 2024-09-13T18:09:35.387 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T18:09:35.387 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-13T18:09:35.387 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:09:35.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.489+0000 7fab4e20d700 1 Processor -- start 2024-09-13T18:09:35.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.489+0000 7fab4e20d700 1 -- start start 2024-09-13T18:09:35.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.489+0000 7fab4e20d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fab48153150 0x7fab4815a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:35.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.489+0000 7fab4e20d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fab480724d0 0x7fab48070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:35.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.489+0000 7fab4e20d700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab48159e30 con 0x7fab480c6190 2024-09-13T18:09:35.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.489+0000 7fab4e20d700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab4815a0d0 con 0x7fab48153150 2024-09-13T18:09:35.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.489+0000 7fab4e20d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab480cb2a0 con 0x7fab480724d0 2024-09-13T18:09:35.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.489+0000 7fab4d20b700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7fab48153150 msgr2=0x7fab4815a790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:09:35.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.489+0000 7fab4d20b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fab48153150 0x7fab4815a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:09:35.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.493+0000 7fab4ca0a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fab480724d0 0x7fab48070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:35.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.493+0000 7fab4da0c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fab480c6190 0x7fab48070fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:45868/0 (socket says 172.21.15.40:45868) 2024-09-13T18:09:35.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.493+0000 7fab4da0c700 1 -- 172.21.15.40:0/1347714502 learned_addr learned my addr 172.21.15.40:0/1347714502 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:09:35.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.493+0000 7fab4ca0a700 1 -- 172.21.15.40:0/1347714502 >> v2:172.21.15.40:3300/0 conn(0x7fab48153150 msgr2=0x7fab4815a790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:09:35.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.493+0000 7fab4ca0a700 1 --2- 172.21.15.40:0/1347714502 >> v2:172.21.15.40:3300/0 conn(0x7fab48153150 0x7fab4815a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.493+0000 7fab3ffff700 1 -- 172.21.15.40:0/1347714502 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3870945400 0 0) 0x7fab48159e30 con 0x7fab480c6190 2024-09-13T18:09:35.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.493+0000 7fab4ca0a700 1 -- 172.21.15.40:0/1347714502 >> v1:172.21.15.43:6789/0 conn(0x7fab480c6190 legacy=0x7fab48070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:35.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.493+0000 7fab4ca0a700 1 -- 172.21.15.40:0/1347714502 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab480cb580 con 0x7fab480724d0 2024-09-13T18:09:35.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.493+0000 7fab4ca0a700 1 --2- 172.21.15.40:0/1347714502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fab480724d0 0x7fab48070aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fab38009850 tx=0x7fab38009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37d87df2ed087dd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:35.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.493+0000 7fab3ffff700 1 -- 172.21.15.40:0/1347714502 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38011830 con 0x7fab480724d0 2024-09-13T18:09:35.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.493+0000 7fab3ffff700 1 -- 172.21.15.40:0/1347714502 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab38011990 con 0x7fab480724d0 2024-09-13T18:09:35.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.493+0000 7fab3ffff700 1 -- 172.21.15.40:0/1347714502 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38011d20 con 0x7fab480724d0 2024-09-13T18:09:35.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.493+0000 7fab4e20d700 1 -- 172.21.15.40:0/1347714502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fab480724d0 msgr2=0x7fab48070aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:35.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.493+0000 7fab4e20d700 1 --2- 172.21.15.40:0/1347714502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fab480724d0 0x7fab48070aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fab38009850 tx=0x7fab38009b60 comp rx=0 tx=0).stop 2024-09-13T18:09:35.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.493+0000 7fab4e20d700 1 -- 172.21.15.40:0/1347714502 shutdown_connections 2024-09-13T18:09:35.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.493+0000 7fab4e20d700 1 --2- 172.21.15.40:0/1347714502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fab480724d0 0x7fab48070aa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.493+0000 7fab4e20d700 1 --2- 172.21.15.40:0/1347714502 >> v2:172.21.15.40:3300/0 conn(0x7fab48153150 0x7fab4815a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.493+0000 7fab4e20d700 1 -- 172.21.15.40:0/1347714502 >> 172.21.15.40:0/1347714502 conn(0x7fab4806a390 msgr2=0x7fab4806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:35.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.493+0000 7fab4e20d700 1 -- 172.21.15.40:0/1347714502 shutdown_connections 2024-09-13T18:09:35.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.493+0000 7fab4e20d700 1 -- 172.21.15.40:0/1347714502 wait complete. 2024-09-13T18:09:35.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4e20d700 1 Processor -- start 2024-09-13T18:09:35.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4e20d700 1 -- start start 2024-09-13T18:09:35.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4e20d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fab480724d0 0x7fab480c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:35.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4e20d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab480c6190 0x7fab481eea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:35.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4e20d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab48153150 0x7fab481eefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:35.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4e20d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab480c3800 con 0x7fab480c6190 2024-09-13T18:09:35.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4e20d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab480c3090 con 0x7fab48153150 2024-09-13T18:09:35.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4e20d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab480c3370 con 0x7fab480724d0 2024-09-13T18:09:35.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4ca0a700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab480c6190 msgr2=0x7fab481eea90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:09:35.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4ca0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab480c6190 0x7fab481eea90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:09:35.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4d20b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fab480724d0 0x7fab480c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:35.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4d20b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fab480724d0 0x7fab480c4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:51650/0 (socket says 172.21.15.40:51650) 2024-09-13T18:09:35.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4d20b700 1 -- 172.21.15.40:0/379135928 learned_addr learned my addr 172.21.15.40:0/379135928 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:35.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4da0c700 1 --2- 172.21.15.40:0/379135928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab48153150 0x7fab481eefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:35.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4d20b700 1 -- 172.21.15.40:0/379135928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab48153150 msgr2=0x7fab481eefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:35.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4d20b700 1 --2- 172.21.15.40:0/379135928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab48153150 0x7fab481eefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4d20b700 1 -- 172.21.15.40:0/379135928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab480c6190 msgr2=0x7fab481eea90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:09:35.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4d20b700 1 --2- 172.21.15.40:0/379135928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab480c6190 0x7fab481eea90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4d20b700 1 -- 172.21.15.40:0/379135928 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab380094b0 con 0x7fab480724d0 2024-09-13T18:09:35.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4d20b700 1 --2- 172.21.15.40:0/379135928 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fab480724d0 0x7fab480c4460 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fab4400b570 tx=0x7fab4400b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:35.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab3dffb700 1 -- 172.21.15.40:0/379135928 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44003e70 con 0x7fab480724d0 2024-09-13T18:09:35.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab3dffb700 1 -- 172.21.15.40:0/379135928 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab44005070 con 0x7fab480724d0 2024-09-13T18:09:35.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4e20d700 1 -- 172.21.15.40:0/379135928 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab481e18b0 con 0x7fab480724d0 2024-09-13T18:09:35.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab3dffb700 1 -- 172.21.15.40:0/379135928 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44005440 con 0x7fab480724d0 2024-09-13T18:09:35.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.497+0000 7fab4e20d700 1 -- 172.21.15.40:0/379135928 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab481e1680 con 0x7fab480724d0 2024-09-13T18:09:35.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.501+0000 7fab4e20d700 1 --2- 172.21.15.40:0/379135928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab48158df0 0x7fab480c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:35.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.501+0000 7fab4e20d700 1 -- 172.21.15.40:0/379135928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab48067600 con 0x7fab48158df0 2024-09-13T18:09:35.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.501+0000 7fab4ca0a700 1 --2- 172.21.15.40:0/379135928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab48158df0 0x7fab480c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:35.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.501+0000 7fab4ca0a700 1 -- 172.21.15.40:0/379135928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab4806cfb0 con 0x7fab48158df0 2024-09-13T18:09:35.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.501+0000 7fab4ca0a700 1 --2- 172.21.15.40:0/379135928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab48158df0 0x7fab480c49a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fab38009850 tx=0x7fab38004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:35.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.501+0000 7fab3dffb700 1 -- 172.21.15.40:0/379135928 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab380036c0 con 0x7fab48158df0 2024-09-13T18:09:35.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.501+0000 7fab3dffb700 1 -- 172.21.15.40:0/379135928 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fab440055a0 con 0x7fab480724d0 2024-09-13T18:09:35.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.501+0000 7fab3dffb700 1 --2- 172.21.15.40:0/379135928 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fab2c0667d0 0x7fab2c068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:35.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.501+0000 7fab3dffb700 1 -- 172.21.15.40:0/379135928 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fab440838e0 con 0x7fab480724d0 2024-09-13T18:09:35.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.501+0000 7fab4da0c700 1 --2- 172.21.15.40:0/379135928 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fab2c0667d0 0x7fab2c068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:35.510 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.501+0000 7fab3dffb700 1 -- 172.21.15.40:0/379135928 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fab3801f070 con 0x7fab48158df0 2024-09-13T18:09:35.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.501+0000 7fab3dffb700 1 -- 172.21.15.40:0/379135928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab48158df0 msgr2=0x7fab480c49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:35.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.501+0000 7fab3dffb700 1 --2- 172.21.15.40:0/379135928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab48158df0 0x7fab480c49a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fab38009850 tx=0x7fab38004770 comp rx=0 tx=0).stop 2024-09-13T18:09:35.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.501+0000 7fab3dffb700 1 -- 172.21.15.40:0/379135928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab48158df0 msgr2=0x7fab480c49a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:09:35.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.501+0000 7fab3dffb700 1 --2- 172.21.15.40:0/379135928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab48158df0 0x7fab480c49a0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fab38009850 tx=0x7fab38004770 comp rx=0 tx=0).stop 2024-09-13T18:09:35.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.501+0000 7fab4da0c700 1 --2- 172.21.15.40:0/379135928 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fab2c0667d0 0x7fab2c068c90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fab40001d10 tx=0x7fab40001ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:35.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.513+0000 7fab4e20d700 1 --2- 172.21.15.40:0/379135928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab48149760 0x7fab48149b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:35.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.513+0000 7fab4e20d700 1 -- 172.21.15.40:0/379135928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab48070320 con 0x7fab48149760 2024-09-13T18:09:35.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.517+0000 7fab4ca0a700 1 --2- 172.21.15.40:0/379135928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab48149760 0x7fab48149b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:35.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.517+0000 7fab4ca0a700 1 -- 172.21.15.40:0/379135928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fab480cbb40 con 0x7fab48149760 2024-09-13T18:09:35.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.517+0000 7fab4ca0a700 1 --2- 172.21.15.40:0/379135928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab48149760 0x7fab48149b70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fab380090b0 tx=0x7fab38009240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:35.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.517+0000 7fab3dffb700 1 -- 172.21.15.40:0/379135928 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3802a040 con 0x7fab48149760 2024-09-13T18:09:35.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.517+0000 7fab3dffb700 1 -- 172.21.15.40:0/379135928 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4939 (secure 0 0 0) 0x7fab3802b320 con 0x7fab48149760 2024-09-13T18:09:35.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.517+0000 7fab3dffb700 1 -- 172.21.15.40:0/379135928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab48149760 msgr2=0x7fab48149b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:35.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.517+0000 7fab3dffb700 1 --2- 172.21.15.40:0/379135928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab48149760 0x7fab48149b70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fab380090b0 tx=0x7fab38009240 comp rx=0 tx=0).stop 2024-09-13T18:09:35.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.517+0000 7fab3dffb700 1 -- 172.21.15.40:0/379135928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab48149760 msgr2=0x7fab48149b70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:09:35.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.517+0000 7fab3dffb700 1 --2- 172.21.15.40:0/379135928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab48149760 0x7fab48149b70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.526 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:09:35.526 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:09:35.526 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:09:35.526 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:09:35.526 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 154, 2024-09-13T18:09:35.526 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:09:35.526 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:09:35.526 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:09:35.526 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:35.526 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:09:35.526 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:09:35.526 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:09:35.527 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:09:35.527 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:35.527 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:35.527 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:35.527 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:35.527 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:35.527 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:35.527 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:35.527 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:35.527 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:35.527 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:35.527 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:35.527 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:09:35.528 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:09:35.528 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:35.528 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:35.528 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:35.528 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:35.528 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:35.528 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:35.528 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:35.528 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:35.528 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:35.528 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:35.528 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:35.528 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.528 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:09:35.529 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:09:35.529 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:09:35.529 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:09:35.529 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:09:35.529 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:09:35.529 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:09:35.529 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:09:35.529 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:09:35.529 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:09:35.529 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:09:35.529 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:09:35.529 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:09:35.529 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:09:35.530 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:09:35.530 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:09:35.530 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:09:35.530 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:35.530 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:35.530 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:35.530 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:35.530 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:35.530 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:35.530 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:35.530 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:35.530 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:35.530 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:35.531 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:35.531 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:09:35.531 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.531 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:09:35.531 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.531 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:09:35.531 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:09:35.531 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:35.532 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:35.532 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.532 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:35.532 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:09:35.532 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:35.532 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.532 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.532 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:35.532 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:09:35.532 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:35.533 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:35.533 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:35.533 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.533 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:09:35.533 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:09:35.533 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:35.533 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:35.534 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:35.534 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.534 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.534 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:09:35.534 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:09:35.534 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:35.534 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:35.534 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.534 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:35.534 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:09:35.534 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:35.534 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.534 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.535 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:35.536 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:09:35.536 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:35.536 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:35.536 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:35.536 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.536 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:09:35.536 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:09:35.536 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:35.536 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:35.537 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:35.537 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.537 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.537 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:09:35.537 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:09:35.537 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:35.537 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:35.537 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.537 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:35.537 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:09:35.537 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:35.537 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.537 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.538 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:35.538 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:09:35.538 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:35.538 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:35.538 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:35.538 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.538 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:09:35.538 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:09:35.538 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:35.538 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:35.538 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:35.538 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:35.538 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:35.539 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.539 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:09:35.539 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:09:35.539 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.539 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:35.539 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:35.539 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:09:35.539 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:35.539 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:35.539 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:09:35.539 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.539 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:35.539 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:35.540 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:09:35.540 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:35.540 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:35.540 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:09:35.540 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.540 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:35.540 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:35.540 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 8 2024-09-13T18:09:35.540 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:35.540 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:35.540 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:09:35.540 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.540 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:35.541 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:35.541 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:09:35.541 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:35.541 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:35.541 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:09:35.541 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.541 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:35.541 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:35.541 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:09:35.541 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:35.541 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:35.541 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.542 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:09:35.542 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:09:35.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.517+0000 7fab4e20d700 1 -- 172.21.15.40:0/379135928 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fab2c0667d0 msgr2=0x7fab2c068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:35.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.517+0000 7fab4e20d700 1 --2- 172.21.15.40:0/379135928 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fab2c0667d0 0x7fab2c068c90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fab40001d10 tx=0x7fab40001ca0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.517+0000 7fab4e20d700 1 -- 172.21.15.40:0/379135928 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fab480724d0 msgr2=0x7fab480c4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:35.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.517+0000 7fab4e20d700 1 --2- 172.21.15.40:0/379135928 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fab480724d0 0x7fab480c4460 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fab4400b570 tx=0x7fab4400b880 comp rx=0 tx=0).stop 2024-09-13T18:09:35.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.517+0000 7fab4da0c700 1 -- 172.21.15.40:0/379135928 reap_dead start 2024-09-13T18:09:35.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.517+0000 7fab4da0c700 1 -- 172.21.15.40:0/379135928 reap_dead start 2024-09-13T18:09:35.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.517+0000 7fab4e20d700 1 -- 172.21.15.40:0/379135928 shutdown_connections 2024-09-13T18:09:35.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.517+0000 7fab4e20d700 1 -- 172.21.15.40:0/379135928 >> 172.21.15.40:0/379135928 conn(0x7fab4806a390 msgr2=0x7fab480c6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:35.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.517+0000 7fab4e20d700 1 -- 172.21.15.40:0/379135928 shutdown_connections 2024-09-13T18:09:35.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.517+0000 7fab4e20d700 1 -- 172.21.15.40:0/379135928 wait complete. 2024-09-13T18:09:35.544 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:09:35.544 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T18:09:35.544 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:09:35.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285ffff700 1 Processor -- start 2024-09-13T18:09:35.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285ffff700 1 -- start start 2024-09-13T18:09:35.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285ffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28580d87a0 0x7f28580df110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:35.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2858007b60 0x7f285804b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:35.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285ffff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2858047610 con 0x7f28580469c0 2024-09-13T18:09:35.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285ffff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2858006b50 con 0x7f2858007b60 2024-09-13T18:09:35.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285ffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28580076d0 con 0x7f28580d87a0 2024-09-13T18:09:35.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285e7fc700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f2858007b60 msgr2=0x7f285804b7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:09:35.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285e7fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2858007b60 0x7f285804b7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:09:35.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285f7fe700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f28580469c0 legacy=0x7f28580e1c80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:09:35.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285effd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28580d87a0 0x7f28580df110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:35.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285effd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28580d87a0 0x7f28580df110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:51654/0 (socket says 172.21.15.40:51654) 2024-09-13T18:09:35.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285effd700 1 -- 172.21.15.40:0/3585543289 learned_addr learned my addr 172.21.15.40:0/3585543289 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:35.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285effd700 1 -- 172.21.15.40:0/3585543289 >> v2:172.21.15.40:3300/0 conn(0x7f2858007b60 msgr2=0x7f285804b7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:09:35.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285effd700 1 --2- 172.21.15.40:0/3585543289 >> v2:172.21.15.40:3300/0 conn(0x7f2858007b60 0x7f285804b7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285effd700 1 -- 172.21.15.40:0/3585543289 >> v1:172.21.15.40:6789/0 conn(0x7f28580469c0 legacy=0x7f28580e1c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:09:35.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285effd700 1 -- 172.21.15.40:0/3585543289 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28580d8b80 con 0x7f28580d87a0 2024-09-13T18:09:35.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285effd700 1 --2- 172.21.15.40:0/3585543289 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28580d87a0 0x7f28580df110 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2854009910 tx=0x7f2854009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=477d0567a3e8d658 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:35.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285dffb700 1 -- 172.21.15.40:0/3585543289 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854004230 con 0x7f28580d87a0 2024-09-13T18:09:35.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285dffb700 1 -- 172.21.15.40:0/3585543289 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2854004390 con 0x7f28580d87a0 2024-09-13T18:09:35.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.665+0000 7f285dffb700 1 -- 172.21.15.40:0/3585543289 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854009e60 con 0x7f28580d87a0 2024-09-13T18:09:35.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.669+0000 7f285ffff700 1 -- 172.21.15.40:0/3585543289 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28580d87a0 msgr2=0x7f28580df110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:35.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.669+0000 7f285ffff700 1 --2- 172.21.15.40:0/3585543289 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28580d87a0 0x7f28580df110 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2854009910 tx=0x7f2854009c20 comp rx=0 tx=0).stop 2024-09-13T18:09:35.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.669+0000 7f285ffff700 1 -- 172.21.15.40:0/3585543289 shutdown_connections 2024-09-13T18:09:35.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.669+0000 7f285ffff700 1 --2- 172.21.15.40:0/3585543289 >> v2:172.21.15.40:3300/0 conn(0x7f2858007b60 0x7f285804b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.669+0000 7f285ffff700 1 --2- 172.21.15.40:0/3585543289 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28580d87a0 0x7f28580df110 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.669+0000 7f285ffff700 1 -- 172.21.15.40:0/3585543289 >> 172.21.15.40:0/3585543289 conn(0x7f2858050e60 msgr2=0x7f2858051270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:35.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.669+0000 7f285ffff700 1 -- 172.21.15.40:0/3585543289 shutdown_connections 2024-09-13T18:09:35.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.669+0000 7f285ffff700 1 -- 172.21.15.40:0/3585543289 wait complete. 2024-09-13T18:09:35.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.669+0000 7f285ffff700 1 Processor -- start 2024-09-13T18:09:35.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.669+0000 7f285ffff700 1 -- start start 2024-09-13T18:09:35.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.669+0000 7f285ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28580469c0 0x7f285804aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:35.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.669+0000 7f285ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28580d87a0 0x7f28580d2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:35.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.669+0000 7f285ffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28580cf480 0x7f2858175190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:35.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.669+0000 7f285ffff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285804a040 con 0x7f28580d87a0 2024-09-13T18:09:35.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.669+0000 7f285ffff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28580d1680 con 0x7f28580469c0 2024-09-13T18:09:35.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.669+0000 7f285ffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28580d1960 con 0x7f28580cf480 2024-09-13T18:09:35.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.669+0000 7f285e7fc700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28580d87a0 msgr2=0x7f28580d2290 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:09:35.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f285e7fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28580d87a0 0x7f28580d2290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:09:35.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f285f7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28580cf480 0x7f2858175190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:35.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f285effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28580469c0 0x7f285804aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:35.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f285f7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28580cf480 0x7f2858175190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:51668/0 (socket says 172.21.15.40:51668) 2024-09-13T18:09:35.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f285f7fe700 1 -- 172.21.15.40:0/1851706809 learned_addr learned my addr 172.21.15.40:0/1851706809 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:35.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f285f7fe700 1 -- 172.21.15.40:0/1851706809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28580469c0 msgr2=0x7f285804aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:35.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f285f7fe700 1 --2- 172.21.15.40:0/1851706809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28580469c0 0x7f285804aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f285f7fe700 1 -- 172.21.15.40:0/1851706809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28580d87a0 msgr2=0x7f28580d2290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:09:35.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f285f7fe700 1 --2- 172.21.15.40:0/1851706809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28580d87a0 0x7f28580d2290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f285f7fe700 1 -- 172.21.15.40:0/1851706809 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2854009570 con 0x7f28580cf480 2024-09-13T18:09:35.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f285f7fe700 1 --2- 172.21.15.40:0/1851706809 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28580cf480 0x7f2858175190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2860000b60 tx=0x7f28600680b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f2847fff700 1 -- 172.21.15.40:0/1851706809 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28600720c0 con 0x7f28580cf480 2024-09-13T18:09:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f285ffff700 1 -- 172.21.15.40:0/1851706809 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28580d1c40 con 0x7f28580cf480 2024-09-13T18:09:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f285ffff700 1 -- 172.21.15.40:0/1851706809 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2858174c70 con 0x7f28580cf480 2024-09-13T18:09:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f2847fff700 1 -- 172.21.15.40:0/1851706809 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2860072240 con 0x7f28580cf480 2024-09-13T18:09:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f2847fff700 1 -- 172.21.15.40:0/1851706809 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2860072630 con 0x7f28580cf480 2024-09-13T18:09:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f285ffff700 1 --2- 172.21.15.40:0/1851706809 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2840002a40 0x7f2840004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f285ffff700 1 -- 172.21.15.40:0/1851706809 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2840007f40 con 0x7f2840002a40 2024-09-13T18:09:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f285effd700 1 --2- 172.21.15.40:0/1851706809 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2840002a40 0x7f2840004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f285effd700 1 -- 172.21.15.40:0/1851706809 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2840008100 con 0x7f2840002a40 2024-09-13T18:09:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f2847fff700 1 -- 172.21.15.40:0/1851706809 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2860091020 con 0x7f28580cf480 2024-09-13T18:09:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.673+0000 7f285effd700 1 --2- 172.21.15.40:0/1851706809 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2840002a40 0x7f2840004f00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2854004700 tx=0x7f2854004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:35.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.677+0000 7f2847fff700 1 --2- 172.21.15.40:0/1851706809 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f284c0666e0 0x7f284c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.677+0000 7f2847fff700 1 -- 172.21.15.40:0/1851706809 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f28600fe830 con 0x7f28580cf480 2024-09-13T18:09:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.677+0000 7f285e7fc700 1 --2- 172.21.15.40:0/1851706809 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f284c0666e0 0x7f284c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.677+0000 7f2847fff700 1 -- 172.21.15.40:0/1851706809 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28540038c0 con 0x7f2840002a40 2024-09-13T18:09:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.677+0000 7f2847fff700 1 -- 172.21.15.40:0/1851706809 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f285400bec0 con 0x7f2840002a40 2024-09-13T18:09:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.677+0000 7f2847fff700 1 -- 172.21.15.40:0/1851706809 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2840002a40 msgr2=0x7f2840004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.677+0000 7f2847fff700 1 --2- 172.21.15.40:0/1851706809 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2840002a40 0x7f2840004f00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2854004700 tx=0x7f2854004770 comp rx=0 tx=0).stop 2024-09-13T18:09:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.677+0000 7f2847fff700 1 -- 172.21.15.40:0/1851706809 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2840002a40 msgr2=0x7f2840004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:09:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.677+0000 7f2847fff700 1 --2- 172.21.15.40:0/1851706809 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2840002a40 0x7f2840004f00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.677+0000 7f285e7fc700 1 --2- 172.21.15.40:0/1851706809 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f284c0666e0 0x7f284c068ba0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2850006fd0 tx=0x7f2850006de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:35.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.685+0000 7f285ffff700 1 --2- 172.21.15.40:0/1851706809 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2840007020 0x7f28400074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:35.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.685+0000 7f285ffff700 1 -- 172.21.15.40:0/1851706809 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284000a400 con 0x7f2840007020 2024-09-13T18:09:35.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.685+0000 7f285effd700 1 --2- 172.21.15.40:0/1851706809 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2840007020 0x7f28400074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:35.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.689+0000 7f285effd700 1 -- 172.21.15.40:0/1851706809 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f284000a5c0 con 0x7f2840007020 2024-09-13T18:09:35.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.689+0000 7f285effd700 1 --2- 172.21.15.40:0/1851706809 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2840007020 0x7f28400074b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2854008000 tx=0x7f2854011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:35.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.689+0000 7f2847fff700 1 -- 172.21.15.40:0/1851706809 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854029070 con 0x7f2840007020 2024-09-13T18:09:35.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.689+0000 7f2847fff700 1 -- 172.21.15.40:0/1851706809 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4605 (secure 0 0 0) 0x7f2854011420 con 0x7f2840007020 2024-09-13T18:09:35.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.689+0000 7f2847fff700 1 -- 172.21.15.40:0/1851706809 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2840007020 msgr2=0x7f28400074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:35.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.689+0000 7f2847fff700 1 --2- 172.21.15.40:0/1851706809 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2840007020 0x7f28400074b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2854008000 tx=0x7f2854011000 comp rx=0 tx=0).stop 2024-09-13T18:09:35.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.689+0000 7f2847fff700 1 -- 172.21.15.40:0/1851706809 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2840007020 msgr2=0x7f28400074b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:09:35.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.689+0000 7f2847fff700 1 --2- 172.21.15.40:0/1851706809 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2840007020 0x7f28400074b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.697 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:09:35.697 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:09:35.697 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:09:35.697 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:09:35.697 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 154, 2024-09-13T18:09:35.697 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:09:35.697 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:09:35.697 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:09:35.697 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:35.698 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:09:35.698 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:09:35.698 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:09:35.698 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:09:35.698 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:35.698 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:35.698 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:35.698 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:35.698 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:35.698 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:35.698 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:35.699 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:35.699 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:35.699 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:35.699 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:35.699 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:09:35.699 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:09:35.699 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:35.699 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:35.699 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:35.699 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:35.699 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:35.699 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:35.699 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:35.700 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:35.700 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:35.700 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:35.700 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:35.700 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.700 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:09:35.700 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:09:35.700 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:09:35.700 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:09:35.700 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:09:35.700 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:09:35.700 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:09:35.701 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:09:35.701 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:09:35.701 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:09:35.701 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:09:35.701 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:09:35.701 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:09:35.701 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:09:35.701 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:09:35.701 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:09:35.701 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:09:35.701 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:09:35.701 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:09:35.701 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:09:35.702 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:09:35.702 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:09:35.702 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:09:35.702 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:09:35.702 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:09:35.702 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:09:35.702 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:09:35.702 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:35.702 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:09:35.702 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.702 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:09:35.702 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.702 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:09:35.702 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:09:35.702 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:35.703 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:35.703 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.703 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:35.703 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:09:35.703 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:35.703 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.703 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.703 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:35.703 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:09:35.703 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:35.703 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:35.704 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:35.704 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.704 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:09:35.704 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:09:35.704 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:35.704 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:35.704 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:35.704 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.704 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.704 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:09:35.704 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:09:35.704 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:35.704 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:35.704 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.705 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:35.705 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:09:35.705 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:35.705 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.705 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.705 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:35.705 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:09:35.705 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:35.705 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:35.705 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:35.705 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.705 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:09:35.705 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:09:35.705 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:35.706 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:35.706 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:35.706 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.706 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.706 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:09:35.706 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:09:35.706 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:09:35.706 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:09:35.706 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.706 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:09:35.706 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:09:35.706 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:35.706 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.706 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.707 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:09:35.707 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:09:35.707 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:09:35.707 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:35.707 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:35.707 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.707 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:09:35.707 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:09:35.707 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:09:35.707 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:09:35.707 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:09:35.707 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:35.707 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:35.707 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.708 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:09:35.708 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:09:35.708 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.708 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:35.708 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:35.708 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:09:35.708 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:35.708 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:35.708 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:09:35.708 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.708 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:35.708 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:35.708 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:09:35.709 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:35.709 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:09:35.709 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:09:35.709 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:09:35.709 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:09:35.709 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:09:35.709 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:09:35.709 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:09:35.709 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:09:35.709 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:09:35.709 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:09:35.709 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:09:35.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.689+0000 7f285ffff700 1 -- 172.21.15.40:0/1851706809 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f284c0666e0 msgr2=0x7f284c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:35.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.689+0000 7f285ffff700 1 --2- 172.21.15.40:0/1851706809 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f284c0666e0 0x7f284c068ba0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2850006fd0 tx=0x7f2850006de0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.689+0000 7f285f7fe700 1 -- 172.21.15.40:0/1851706809 reap_dead start 2024-09-13T18:09:35.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.689+0000 7f285ffff700 1 -- 172.21.15.40:0/1851706809 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28580cf480 msgr2=0x7f2858175190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:35.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.689+0000 7f285ffff700 1 --2- 172.21.15.40:0/1851706809 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28580cf480 0x7f2858175190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2860000b60 tx=0x7f28600680b0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.689+0000 7f285ffff700 1 -- 172.21.15.40:0/1851706809 shutdown_connections 2024-09-13T18:09:35.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.689+0000 7f285ffff700 1 --2- 172.21.15.40:0/1851706809 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f28580cf480 0x7f2858175190 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:35.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.689+0000 7f285ffff700 1 -- 172.21.15.40:0/1851706809 >> 172.21.15.40:0/1851706809 conn(0x7f2858050e60 msgr2=0x7f285804c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:35.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.689+0000 7f285ffff700 1 -- 172.21.15.40:0/1851706809 shutdown_connections 2024-09-13T18:09:35.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:35.689+0000 7f285ffff700 1 -- 172.21.15.40:0/1851706809 wait complete. 2024-09-13T18:09:35.712 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:09:35.712 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:09:37.080 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:09:37.082 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:09:55.713 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T18:09:55.713 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-13T18:09:55.713 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T18:09:55.714 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T18:09:55.717 INFO:tasks.ceph.mon.a:Started 2024-09-13T18:09:55.717 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:09:55.717 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:09:55.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.821+0000 7f30b6649700 1 Processor -- start 2024-09-13T18:09:55.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.821+0000 7f30b6649700 1 -- start start 2024-09-13T18:09:55.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b6649700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f30b006ea60 0x7f30b0070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:55.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b6649700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f30b0071620 0x7f30b006d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:55.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b6649700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b014fdd0 con 0x7f30b014f9f0 2024-09-13T18:09:55.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b6649700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b0070920 con 0x7f30b006ea60 2024-09-13T18:09:55.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b6649700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b01564d0 con 0x7f30b0071620 2024-09-13T18:09:55.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b4e46700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f30b006ea60 0x7f30b0070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:55.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b4e46700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f30b006ea60 0x7f30b0070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45872/0 (socket says 172.21.15.40:45872) 2024-09-13T18:09:55.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b4e46700 1 -- 172.21.15.40:0/3704068261 learned_addr learned my addr 172.21.15.40:0/3704068261 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:55.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b5e48700 1 --2- 172.21.15.40:0/3704068261 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f30b0071620 0x7f30b006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:55.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b4e46700 1 -- 172.21.15.40:0/3704068261 >> v2:172.21.15.40:3300/0 conn(0x7f30b006ea60 msgr2=0x7f30b0070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:09:55.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b4e46700 1 -- 172.21.15.40:0/3704068261 >> v2:172.21.15.40:3300/0 conn(0x7f30b006ea60 msgr2=0x7f30b0070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:09:55.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b4e46700 1 --2- 172.21.15.40:0/3704068261 >> v2:172.21.15.40:3300/0 conn(0x7f30b006ea60 0x7f30b0070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:09:55.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b4e46700 1 --2- 172.21.15.40:0/3704068261 >> v2:172.21.15.40:3300/0 conn(0x7f30b006ea60 0x7f30b0070290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:09:55.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30a7fff700 1 -- 172.21.15.40:0/3704068261 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784567358 0 0) 0x7f30b014fdd0 con 0x7f30b014f9f0 2024-09-13T18:09:55.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b5e48700 1 -- 172.21.15.40:0/3704068261 >> v2:172.21.15.40:3300/0 conn(0x7f30b006ea60 msgr2=0x7f30b0070290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:09:55.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b5e48700 1 --2- 172.21.15.40:0/3704068261 >> v2:172.21.15.40:3300/0 conn(0x7f30b006ea60 0x7f30b0070290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:55.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b5e48700 1 -- 172.21.15.40:0/3704068261 >> v1:172.21.15.43:6789/0 conn(0x7f30b014f9f0 legacy=0x7f30b0070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:55.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b5e48700 1 -- 172.21.15.40:0/3704068261 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b01558b0 con 0x7f30b0071620 2024-09-13T18:09:55.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b5e48700 1 --2- 172.21.15.40:0/3704068261 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f30b0071620 0x7f30b006d570 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009d90 tx=0x7f30a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a61ee37553555668 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:55.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30a7fff700 1 -- 172.21.15.40:0/3704068261 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a800e070 con 0x7f30b0071620 2024-09-13T18:09:55.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30a7fff700 1 -- 172.21.15.40:0/3704068261 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a800bbd0 con 0x7f30b0071620 2024-09-13T18:09:55.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30a7fff700 1 -- 172.21.15.40:0/3704068261 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a8012430 con 0x7f30b0071620 2024-09-13T18:09:55.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b6649700 1 -- 172.21.15.40:0/3704068261 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f30b0071620 msgr2=0x7f30b006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:55.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b6649700 1 --2- 172.21.15.40:0/3704068261 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f30b0071620 0x7f30b006d570 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009d90 tx=0x7f30a800b330 comp rx=0 tx=0).stop 2024-09-13T18:09:55.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b6649700 1 -- 172.21.15.40:0/3704068261 shutdown_connections 2024-09-13T18:09:55.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b6649700 1 --2- 172.21.15.40:0/3704068261 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f30b0071620 0x7f30b006d570 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:55.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b6649700 1 --2- 172.21.15.40:0/3704068261 >> v2:172.21.15.40:3300/0 conn(0x7f30b006ea60 0x7f30b0070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:55.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b6649700 1 -- 172.21.15.40:0/3704068261 >> 172.21.15.40:0/3704068261 conn(0x7f30b006a390 msgr2=0x7f30b006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:55.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b6649700 1 -- 172.21.15.40:0/3704068261 shutdown_connections 2024-09-13T18:09:55.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.825+0000 7f30b6649700 1 -- 172.21.15.40:0/3704068261 wait complete. 2024-09-13T18:09:55.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b6649700 1 Processor -- start 2024-09-13T18:09:55.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b6649700 1 -- start start 2024-09-13T18:09:55.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b6649700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f30b006ea60 0x7f30b0150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:55.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b6649700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30b0071620 0x7f30b00ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:55.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b6649700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30b014f9f0 0x7f30b01f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:55.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b6649700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b01e29a0 con 0x7f30b014f9f0 2024-09-13T18:09:55.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b6649700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b00cade0 con 0x7f30b0071620 2024-09-13T18:09:55.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b6649700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b01e20e0 con 0x7f30b006ea60 2024-09-13T18:09:55.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b5e48700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30b014f9f0 0x7f30b01f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:55.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b5e48700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30b014f9f0 0x7f30b01f1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45876/0 (socket says 172.21.15.40:45876) 2024-09-13T18:09:55.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b5e48700 1 -- 172.21.15.40:0/3164624003 learned_addr learned my addr 172.21.15.40:0/3164624003 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:55.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b5e48700 1 -- 172.21.15.40:0/3164624003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30b014f9f0 msgr2=0x7f30b01f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-13T18:09:55.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b5e48700 1 -- 172.21.15.40:0/3164624003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30b014f9f0 msgr2=0x7f30b01f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:09:55.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b5e48700 1 --2- 172.21.15.40:0/3164624003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30b014f9f0 0x7f30b01f1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:09:55.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b5647700 1 --2- 172.21.15.40:0/3164624003 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f30b006ea60 0x7f30b0150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:55.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b4e46700 1 --2- 172.21.15.40:0/3164624003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30b0071620 0x7f30b00ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:55.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b5e48700 1 --2- 172.21.15.40:0/3164624003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30b014f9f0 0x7f30b01f1fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:09:55.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b5647700 1 -- 172.21.15.40:0/3164624003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30b0071620 msgr2=0x7f30b00ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:55.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b5647700 1 --2- 172.21.15.40:0/3164624003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30b0071620 0x7f30b00ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:55.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b5647700 1 -- 172.21.15.40:0/3164624003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30b014f9f0 msgr2=0x7f30b01f1fd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:09:55.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b5647700 1 --2- 172.21.15.40:0/3164624003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30b014f9f0 0x7f30b01f1fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:55.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.829+0000 7f30b5647700 1 -- 172.21.15.40:0/3164624003 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a8009a70 con 0x7f30b006ea60 2024-09-13T18:09:55.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.833+0000 7f30b5647700 1 --2- 172.21.15.40:0/3164624003 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f30b006ea60 0x7f30b0150340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00c410 tx=0x7f30ac00c720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:55.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.833+0000 7f30a5ffb700 1 -- 172.21.15.40:0/3164624003 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0052d0 con 0x7f30b006ea60 2024-09-13T18:09:55.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.833+0000 7f30a5ffb700 1 -- 172.21.15.40:0/3164624003 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ac00ce90 con 0x7f30b006ea60 2024-09-13T18:09:55.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.833+0000 7f30b6649700 1 -- 172.21.15.40:0/3164624003 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b0150880 con 0x7f30b006ea60 2024-09-13T18:09:55.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.833+0000 7f30a5ffb700 1 -- 172.21.15.40:0/3164624003 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0036c0 con 0x7f30b006ea60 2024-09-13T18:09:55.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.833+0000 7f30b6649700 1 -- 172.21.15.40:0/3164624003 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b01e2350 con 0x7f30b006ea60 2024-09-13T18:09:55.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.833+0000 7f30b6649700 1 -- 172.21.15.40:0/3164624003 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b0067fc0 con 0x7f30b006ea60 2024-09-13T18:09:55.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.833+0000 7f30a5ffb700 1 -- 172.21.15.40:0/3164624003 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f30ac003820 con 0x7f30b006ea60 2024-09-13T18:09:55.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.833+0000 7f30a5ffb700 1 --2- 172.21.15.40:0/3164624003 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3094066760 0x7f3094068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:55.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.833+0000 7f30a5ffb700 1 -- 172.21.15.40:0/3164624003 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f30ac083b50 con 0x7f30b006ea60 2024-09-13T18:09:55.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.833+0000 7f30b4e46700 1 --2- 172.21.15.40:0/3164624003 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3094066760 0x7f3094068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:55.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.837+0000 7f30b4e46700 1 --2- 172.21.15.40:0/3164624003 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3094066760 0x7f3094068c20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f30b0159c30 tx=0x7f30a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:55.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:55.845+0000 7f30a5ffb700 1 -- 172.21.15.40:0/3164624003 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f30ac083e60 con 0x7f30b006ea60 2024-09-13T18:09:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:56.113+0000 7f30b6649700 1 -- 172.21.15.40:0/3164624003 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f30b0070290 con 0x7f30b006ea60 2024-09-13T18:09:56.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:56.113+0000 7f30a5ffb700 1 -- 172.21.15.40:0/3164624003 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f30ac017610 con 0x7f30b006ea60 2024-09-13T18:09:56.121 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":154,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:09:56.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:56.117+0000 7f30b6649700 1 -- 172.21.15.40:0/3164624003 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3094066760 msgr2=0x7f3094068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:56.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:56.117+0000 7f30b6649700 1 --2- 172.21.15.40:0/3164624003 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3094066760 0x7f3094068c20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f30b0159c30 tx=0x7f30a0008040 comp rx=0 tx=0).stop 2024-09-13T18:09:56.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:56.117+0000 7f30b6649700 1 -- 172.21.15.40:0/3164624003 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f30b006ea60 msgr2=0x7f30b0150340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:56.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:56.117+0000 7f30b6649700 1 --2- 172.21.15.40:0/3164624003 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f30b006ea60 0x7f30b0150340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00c410 tx=0x7f30ac00c720 comp rx=0 tx=0).stop 2024-09-13T18:09:56.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:56.117+0000 7f30b6649700 1 -- 172.21.15.40:0/3164624003 shutdown_connections 2024-09-13T18:09:56.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:56.117+0000 7f30b6649700 1 --2- 172.21.15.40:0/3164624003 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30b014f9f0 0x7f30b01f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:56.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:56.117+0000 7f30b6649700 1 --2- 172.21.15.40:0/3164624003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30b0071620 0x7f30b00ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:56.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:56.117+0000 7f30b6649700 1 --2- 172.21.15.40:0/3164624003 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3094066760 0x7f3094068c20 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:56.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:56.117+0000 7f30b6649700 1 --2- 172.21.15.40:0/3164624003 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f30b006ea60 0x7f30b0150340 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:56.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:56.117+0000 7f30b6649700 1 -- 172.21.15.40:0/3164624003 >> 172.21.15.40:0/3164624003 conn(0x7f30b006a390 msgr2=0x7f30b01e18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:56.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:56.117+0000 7f30b6649700 1 -- 172.21.15.40:0/3164624003 shutdown_connections 2024-09-13T18:09:56.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:56.117+0000 7f30b6649700 1 -- 172.21.15.40:0/3164624003 wait complete. 2024-09-13T18:09:59.141 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:09:59.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.261+0000 7f807759e700 1 Processor -- start 2024-09-13T18:09:59.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.261+0000 7f807759e700 1 -- start start 2024-09-13T18:09:59.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.261+0000 7f807759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80780b2b70 0x7f807806f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:59.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.261+0000 7f807759e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f807806fee0 0x7f80780c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:59.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.261+0000 7f807759e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8078070390 con 0x7f80781512c0 2024-09-13T18:09:59.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.261+0000 7f807759e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807806e780 con 0x7f807806fee0 2024-09-13T18:09:59.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.261+0000 7f807759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80780c3a80 con 0x7f80780b2b70 2024-09-13T18:09:59.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.261+0000 7f807659c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f80781512c0 0x7f8078157b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49042/0 (socket says 172.21.15.40:49042) 2024-09-13T18:09:59.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.261+0000 7f807659c700 1 -- 172.21.15.40:0/865359957 learned_addr learned my addr 172.21.15.40:0/865359957 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:09:59.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.261+0000 7f8076d9d700 1 --2- 172.21.15.40:0/865359957 >> v2:172.21.15.43:3300/0 conn(0x7f807806fee0 0x7f80780c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:59.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.261+0000 7f806ffff700 1 --2- 172.21.15.40:0/865359957 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80780b2b70 0x7f807806f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:59.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.261+0000 7f8076d9d700 1 -- 172.21.15.40:0/865359957 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80780b2b70 msgr2=0x7f807806f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:09:59.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.261+0000 7f8076d9d700 1 --2- 172.21.15.40:0/865359957 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80780b2b70 0x7f807806f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:59.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.261+0000 7f8076d9d700 1 -- 172.21.15.40:0/865359957 >> v1:172.21.15.40:6789/0 conn(0x7f80781512c0 legacy=0x7f8078157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:59.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.261+0000 7f8076d9d700 1 -- 172.21.15.40:0/865359957 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80780c80c0 con 0x7f807806fee0 2024-09-13T18:09:59.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.261+0000 7f806ffff700 1 --2- 172.21.15.40:0/865359957 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80780b2b70 0x7f807806f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:09:59.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.261+0000 7f8076d9d700 1 --2- 172.21.15.40:0/865359957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f807806fee0 0x7f80780c1f80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8070009d90 tx=0x7f807000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6312afa6334a65da server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:59.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.261+0000 7f8075d9b700 1 -- 172.21.15.40:0/865359957 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807000e070 con 0x7f807806fee0 2024-09-13T18:09:59.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.261+0000 7f8075d9b700 1 -- 172.21.15.40:0/865359957 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807000bb20 con 0x7f807806fee0 2024-09-13T18:09:59.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.261+0000 7f8075d9b700 1 -- 172.21.15.40:0/865359957 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070012430 con 0x7f807806fee0 2024-09-13T18:09:59.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807759e700 1 -- 172.21.15.40:0/865359957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f807806fee0 msgr2=0x7f80780c1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:59.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807759e700 1 --2- 172.21.15.40:0/865359957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f807806fee0 0x7f80780c1f80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8070009d90 tx=0x7f807000b330 comp rx=0 tx=0).stop 2024-09-13T18:09:59.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807759e700 1 -- 172.21.15.40:0/865359957 shutdown_connections 2024-09-13T18:09:59.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807759e700 1 --2- 172.21.15.40:0/865359957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f807806fee0 0x7f80780c1f80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:59.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807759e700 1 --2- 172.21.15.40:0/865359957 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80780b2b70 0x7f807806f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:59.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807759e700 1 -- 172.21.15.40:0/865359957 >> 172.21.15.40:0/865359957 conn(0x7f8078066090 msgr2=0x7f8078066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:59.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807759e700 1 -- 172.21.15.40:0/865359957 shutdown_connections 2024-09-13T18:09:59.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807759e700 1 -- 172.21.15.40:0/865359957 wait complete. 2024-09-13T18:09:59.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807759e700 1 Processor -- start 2024-09-13T18:09:59.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807759e700 1 -- start start 2024-09-13T18:09:59.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f807806fee0 0x7f80781ef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:59.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80780b2b70 0x7f807806c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:59.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80781512c0 0x7f8078152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:59.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80781efe00 con 0x7f807806fee0 2024-09-13T18:09:59.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80780c4490 con 0x7f80781512c0 2024-09-13T18:09:59.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80780c4730 con 0x7f80780b2b70 2024-09-13T18:09:59.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f807806fee0 0x7f80781ef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:59.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f807806fee0 0x7f80781ef8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45882/0 (socket says 172.21.15.40:45882) 2024-09-13T18:09:59.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807659c700 1 -- 172.21.15.40:0/1791920155 learned_addr learned my addr 172.21.15.40:0/1791920155 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:09:59.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f806ffff700 1 --2- 172.21.15.40:0/1791920155 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80780b2b70 0x7f807806c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:59.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807659c700 1 -- 172.21.15.40:0/1791920155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f807806fee0 msgr2=0x7f80781ef8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:09:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807659c700 1 -- 172.21.15.40:0/1791920155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f807806fee0 msgr2=0x7f80781ef8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:09:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807659c700 1 --2- 172.21.15.40:0/1791920155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f807806fee0 0x7f80781ef8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:09:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f8076d9d700 1 --2- 172.21.15.40:0/1791920155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80781512c0 0x7f8078152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f807659c700 1 --2- 172.21.15.40:0/1791920155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f807806fee0 0x7f80781ef8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:09:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f806ffff700 1 -- 172.21.15.40:0/1791920155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80781512c0 msgr2=0x7f8078152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f806ffff700 1 --2- 172.21.15.40:0/1791920155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80781512c0 0x7f8078152270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f806ffff700 1 -- 172.21.15.40:0/1791920155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f807806fee0 msgr2=0x7f80781ef8c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:09:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f806ffff700 1 --2- 172.21.15.40:0/1791920155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f807806fee0 0x7f80781ef8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f806ffff700 1 -- 172.21.15.40:0/1791920155 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8060009450 con 0x7f80780b2b70 2024-09-13T18:09:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.265+0000 7f8076d9d700 1 --2- 172.21.15.40:0/1791920155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80781512c0 0x7f8078152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:09:59.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.269+0000 7f806ffff700 1 --2- 172.21.15.40:0/1791920155 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80780b2b70 0x7f807806c760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f806000e980 tx=0x7f806000ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:59.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.269+0000 7f806effd700 1 -- 172.21.15.40:0/1791920155 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806000eed0 con 0x7f80780b2b70 2024-09-13T18:09:59.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.269+0000 7f807759e700 1 -- 172.21.15.40:0/1791920155 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8070009a70 con 0x7f80780b2b70 2024-09-13T18:09:59.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.269+0000 7f806effd700 1 -- 172.21.15.40:0/1791920155 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8060005a40 con 0x7f80780b2b70 2024-09-13T18:09:59.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.269+0000 7f806effd700 1 -- 172.21.15.40:0/1791920155 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8060005dd0 con 0x7f80780b2b70 2024-09-13T18:09:59.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.269+0000 7f807759e700 1 -- 172.21.15.40:0/1791920155 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80780c4cf0 con 0x7f80780b2b70 2024-09-13T18:09:59.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.269+0000 7f807759e700 1 -- 172.21.15.40:0/1791920155 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80780667a0 con 0x7f80780b2b70 2024-09-13T18:09:59.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.269+0000 7f806effd700 1 -- 172.21.15.40:0/1791920155 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f806000c850 con 0x7f80780b2b70 2024-09-13T18:09:59.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.269+0000 7f806effd700 1 --2- 172.21.15.40:0/1791920155 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8058066730 0x7f8058068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:09:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.269+0000 7f806effd700 1 -- 172.21.15.40:0/1791920155 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8060084b60 con 0x7f80780b2b70 2024-09-13T18:09:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.269+0000 7f807659c700 1 --2- 172.21.15.40:0/1791920155 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8058066730 0x7f8058068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:09:59.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.273+0000 7f807659c700 1 --2- 172.21.15.40:0/1791920155 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8058066730 0x7f8058068bf0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8068007fd0 tx=0x7f8068009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:09:59.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.277+0000 7f806effd700 1 -- 172.21.15.40:0/1791920155 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8060051ba0 con 0x7f80780b2b70 2024-09-13T18:09:59.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.525+0000 7f807759e700 1 -- 172.21.15.40:0/1791920155 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f80781516a0 con 0x7f80780b2b70 2024-09-13T18:09:59.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.529+0000 7f806effd700 1 -- 172.21.15.40:0/1791920155 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f8060020020 con 0x7f80780b2b70 2024-09-13T18:09:59.535 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":154,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":24,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:09:59.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.529+0000 7f807759e700 1 -- 172.21.15.40:0/1791920155 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8058066730 msgr2=0x7f8058068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:59.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.529+0000 7f807759e700 1 --2- 172.21.15.40:0/1791920155 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8058066730 0x7f8058068bf0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8068007fd0 tx=0x7f8068009040 comp rx=0 tx=0).stop 2024-09-13T18:09:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.529+0000 7f807759e700 1 -- 172.21.15.40:0/1791920155 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80780b2b70 msgr2=0x7f807806c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:09:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.529+0000 7f807759e700 1 --2- 172.21.15.40:0/1791920155 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80780b2b70 0x7f807806c760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f806000e980 tx=0x7f806000ec90 comp rx=0 tx=0).stop 2024-09-13T18:09:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.529+0000 7f807759e700 1 -- 172.21.15.40:0/1791920155 shutdown_connections 2024-09-13T18:09:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.529+0000 7f807759e700 1 --2- 172.21.15.40:0/1791920155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80781512c0 0x7f8078152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.529+0000 7f807759e700 1 --2- 172.21.15.40:0/1791920155 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8058066730 0x7f8058068bf0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.529+0000 7f807759e700 1 --2- 172.21.15.40:0/1791920155 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f80780b2b70 0x7f807806c760 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.529+0000 7f807759e700 1 --2- 172.21.15.40:0/1791920155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f807806fee0 0x7f80781ef8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:09:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.529+0000 7f807759e700 1 -- 172.21.15.40:0/1791920155 >> 172.21.15.40:0/1791920155 conn(0x7f8078066090 msgr2=0x7f807814eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:09:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.529+0000 7f807759e700 1 -- 172.21.15.40:0/1791920155 shutdown_connections 2024-09-13T18:09:59.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:09:59.529+0000 7f807759e700 1 -- 172.21.15.40:0/1791920155 wait complete. 2024-09-13T18:10:02.551 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:10:02.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.653+0000 7f22eaf70700 1 Processor -- start 2024-09-13T18:10:02.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.653+0000 7f22eaf70700 1 -- start start 2024-09-13T18:10:02.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22eaf70700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f22dc0d54f0 0x7f22dc008a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:02.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22eaf70700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22dc04ccb0 0x7f22dc0de980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:02.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22eaf70700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22dc008600 con 0x7f22dc0deec0 2024-09-13T18:10:02.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22eaf70700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22dc0d58d0 con 0x7f22dc0d54f0 2024-09-13T18:10:02.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22eaf70700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22dc04d210 con 0x7f22dc04ccb0 2024-09-13T18:10:02.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22e9f6e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f22dc0d54f0 0x7f22dc008a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:02.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22e9f6e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f22dc0d54f0 0x7f22dc008a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49234/0 (socket says 172.21.15.40:49234) 2024-09-13T18:10:02.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22e9f6e700 1 -- 172.21.15.40:0/897721848 learned_addr learned my addr 172.21.15.40:0/897721848 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:10:02.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22ea76f700 1 --1- 172.21.15.40:0/897721848 >> v1:172.21.15.40:6789/0 conn(0x7f22dc0deec0 0x7f22dc0db390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55054/0 (socket says 172.21.15.40:55054) 2024-09-13T18:10:02.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22e976d700 1 --2- 172.21.15.40:0/897721848 >> v2:172.21.15.43:3300/0 conn(0x7f22dc04ccb0 0x7f22dc0de980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:02.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22e9f6e700 1 -- 172.21.15.40:0/897721848 >> v2:172.21.15.43:3300/0 conn(0x7f22dc04ccb0 msgr2=0x7f22dc0de980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:10:02.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22e9f6e700 1 --2- 172.21.15.40:0/897721848 >> v2:172.21.15.43:3300/0 conn(0x7f22dc04ccb0 0x7f22dc0de980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:02.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22e9f6e700 1 -- 172.21.15.40:0/897721848 >> v1:172.21.15.40:6789/0 conn(0x7f22dc0deec0 legacy=0x7f22dc0db390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:02.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22e9f6e700 1 -- 172.21.15.40:0/897721848 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc007a30 con 0x7f22dc0d54f0 2024-09-13T18:10:02.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22e8f6c700 1 -- 172.21.15.40:0/897721848 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1886678873 0 0) 0x7f22dc008600 con 0x7f22dc0deec0 2024-09-13T18:10:02.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22e9f6e700 1 --2- 172.21.15.40:0/897721848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc0d54f0 0x7f22dc008a90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0dfaa0 tx=0x7f22e0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f2df8380973b74 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:02.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22e8f6c700 1 -- 172.21.15.40:0/897721848 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e000ba20 con 0x7f22dc0d54f0 2024-09-13T18:10:02.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22e8f6c700 1 -- 172.21.15.40:0/897721848 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e000bb80 con 0x7f22dc0d54f0 2024-09-13T18:10:02.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22e8f6c700 1 -- 172.21.15.40:0/897721848 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0009e60 con 0x7f22dc0d54f0 2024-09-13T18:10:02.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22eaf70700 1 -- 172.21.15.40:0/897721848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc0d54f0 msgr2=0x7f22dc008a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:02.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22eaf70700 1 --2- 172.21.15.40:0/897721848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc0d54f0 0x7f22dc008a90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0dfaa0 tx=0x7f22e0009c20 comp rx=0 tx=0).stop 2024-09-13T18:10:02.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22eaf70700 1 -- 172.21.15.40:0/897721848 shutdown_connections 2024-09-13T18:10:02.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22eaf70700 1 --2- 172.21.15.40:0/897721848 >> v2:172.21.15.43:3300/0 conn(0x7f22dc04ccb0 0x7f22dc0de980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:02.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22eaf70700 1 --2- 172.21.15.40:0/897721848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc0d54f0 0x7f22dc008a90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:02.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22eaf70700 1 -- 172.21.15.40:0/897721848 >> 172.21.15.40:0/897721848 conn(0x7f22dc050e10 msgr2=0x7f22dc051220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:02.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22eaf70700 1 -- 172.21.15.40:0/897721848 shutdown_connections 2024-09-13T18:10:02.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.657+0000 7f22eaf70700 1 -- 172.21.15.40:0/897721848 wait complete. 2024-09-13T18:10:02.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22eaf70700 1 Processor -- start 2024-09-13T18:10:02.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22eaf70700 1 -- start start 2024-09-13T18:10:02.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22eaf70700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22dc04ccb0 0x7f22dc0d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:02.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22eaf70700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc0d54f0 0x7f22dc173ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:02.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22eaf70700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f22dc0deec0 0x7f22dc0d6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:02.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22eaf70700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc0d6790 con 0x7f22dc0d54f0 2024-09-13T18:10:02.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22eaf70700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc0cfb20 con 0x7f22dc04ccb0 2024-09-13T18:10:02.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22eaf70700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc0cfe00 con 0x7f22dc0deec0 2024-09-13T18:10:02.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22e976d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc0d54f0 0x7f22dc173ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:02.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22e976d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc0d54f0 0x7f22dc173ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49238/0 (socket says 172.21.15.40:49238) 2024-09-13T18:10:02.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22e976d700 1 -- 172.21.15.40:0/1842482602 learned_addr learned my addr 172.21.15.40:0/1842482602 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:10:02.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22e9f6e700 1 --2- 172.21.15.40:0/1842482602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22dc04ccb0 0x7f22dc0d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:02.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22ea76f700 1 --2- 172.21.15.40:0/1842482602 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f22dc0deec0 0x7f22dc0d6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:02.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22e976d700 1 -- 172.21.15.40:0/1842482602 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f22dc0deec0 msgr2=0x7f22dc0d6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:02.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22e976d700 1 --2- 172.21.15.40:0/1842482602 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f22dc0deec0 0x7f22dc0d6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:02.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22e976d700 1 -- 172.21.15.40:0/1842482602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22dc04ccb0 msgr2=0x7f22dc0d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:02.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22e976d700 1 --2- 172.21.15.40:0/1842482602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22dc04ccb0 0x7f22dc0d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:02.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22e976d700 1 -- 172.21.15.40:0/1842482602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e0009570 con 0x7f22dc0d54f0 2024-09-13T18:10:02.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22e976d700 1 --2- 172.21.15.40:0/1842482602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc0d54f0 0x7f22dc173ff0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f22d400e760 tx=0x7f22d400ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:02.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22da7fc700 1 -- 172.21.15.40:0/1842482602 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4013610 con 0x7f22dc0d54f0 2024-09-13T18:10:02.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22eaf70700 1 -- 172.21.15.40:0/1842482602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22dc0d00e0 con 0x7f22dc0d54f0 2024-09-13T18:10:02.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22da7fc700 1 -- 172.21.15.40:0/1842482602 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d4013770 con 0x7f22dc0d54f0 2024-09-13T18:10:02.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22da7fc700 1 -- 172.21.15.40:0/1842482602 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4013ae0 con 0x7f22dc0d54f0 2024-09-13T18:10:02.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.661+0000 7f22eaf70700 1 -- 172.21.15.40:0/1842482602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22dc04b850 con 0x7f22dc0d54f0 2024-09-13T18:10:02.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.665+0000 7f22da7fc700 1 -- 172.21.15.40:0/1842482602 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f22d4013c40 con 0x7f22dc0d54f0 2024-09-13T18:10:02.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.665+0000 7f22eaf70700 1 -- 172.21.15.40:0/1842482602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22dc0d6280 con 0x7f22dc0d54f0 2024-09-13T18:10:02.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.665+0000 7f22da7fc700 1 --2- 172.21.15.40:0/1842482602 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f22c8066760 0x7f22c8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:02.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.665+0000 7f22da7fc700 1 -- 172.21.15.40:0/1842482602 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f22d40841a0 con 0x7f22dc0d54f0 2024-09-13T18:10:02.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.665+0000 7f22e9f6e700 1 --2- 172.21.15.40:0/1842482602 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f22c8066760 0x7f22c8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:02.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.665+0000 7f22e9f6e700 1 --2- 172.21.15.40:0/1842482602 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f22c8066760 0x7f22c8068c20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f22e0000c00 tx=0x7f22e0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:02.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.673+0000 7f22da7fc700 1 -- 172.21.15.40:0/1842482602 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f22dc0d6280 con 0x7f22dc0d54f0 2024-09-13T18:10:02.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.949+0000 7f22eaf70700 1 -- 172.21.15.40:0/1842482602 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f22dc0d6280 con 0x7f22dc0d54f0 2024-09-13T18:10:02.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.953+0000 7f22da7fc700 1 -- 172.21.15.40:0/1842482602 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f22dc0d6280 con 0x7f22dc0d54f0 2024-09-13T18:10:02.959 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":158,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:10:02.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.953+0000 7f22eaf70700 1 -- 172.21.15.40:0/1842482602 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f22c8066760 msgr2=0x7f22c8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:02.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.953+0000 7f22eaf70700 1 --2- 172.21.15.40:0/1842482602 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f22c8066760 0x7f22c8068c20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f22e0000c00 tx=0x7f22e0008030 comp rx=0 tx=0).stop 2024-09-13T18:10:02.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.953+0000 7f22eaf70700 1 -- 172.21.15.40:0/1842482602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc0d54f0 msgr2=0x7f22dc173ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:02.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.953+0000 7f22eaf70700 1 --2- 172.21.15.40:0/1842482602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc0d54f0 0x7f22dc173ff0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f22d400e760 tx=0x7f22d400ea70 comp rx=0 tx=0).stop 2024-09-13T18:10:02.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.953+0000 7f22eaf70700 1 -- 172.21.15.40:0/1842482602 shutdown_connections 2024-09-13T18:10:02.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.953+0000 7f22eaf70700 1 --2- 172.21.15.40:0/1842482602 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f22c8066760 0x7f22c8068c20 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:02.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.953+0000 7f22eaf70700 1 --2- 172.21.15.40:0/1842482602 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f22dc0deec0 0x7f22dc0d6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:02.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.953+0000 7f22eaf70700 1 --2- 172.21.15.40:0/1842482602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22dc0d54f0 0x7f22dc173ff0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:02.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.953+0000 7f22eaf70700 1 --2- 172.21.15.40:0/1842482602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22dc04ccb0 0x7f22dc0d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:02.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.953+0000 7f22eaf70700 1 -- 172.21.15.40:0/1842482602 >> 172.21.15.40:0/1842482602 conn(0x7f22dc050e10 msgr2=0x7f22dc035180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:02.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.953+0000 7f22eaf70700 1 -- 172.21.15.40:0/1842482602 shutdown_connections 2024-09-13T18:10:02.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:02.953+0000 7f22eaf70700 1 -- 172.21.15.40:0/1842482602 wait complete. 2024-09-13T18:10:02.981 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:10:02.983 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:10:02.984 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:10:03.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.089+0000 7fbaf3fff700 1 Processor -- start 2024-09-13T18:10:03.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.089+0000 7fbaf3fff700 1 -- start start 2024-09-13T18:10:03.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.089+0000 7fbaf3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbaf4163d00 0x7fbaf40c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.089+0000 7fbaf3fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbaf406b080 0x7fbaf40c6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.089+0000 7fbaf3fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaf4167290 con 0x7fbaf40c7ed0 2024-09-13T18:10:03.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.089+0000 7fbaf3fff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaf40c1630 con 0x7fbaf406b080 2024-09-13T18:10:03.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.089+0000 7fbaf3fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf4166680 con 0x7fbaf4163d00 2024-09-13T18:10:03.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.089+0000 7fbaf37fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbaf406b080 0x7fbaf40c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.089+0000 7fbaebfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbaf4163d00 0x7fbaf40c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.089+0000 7fbaf37fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbaf406b080 0x7fbaf40c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:59782/0 (socket says 172.21.15.40:59782) 2024-09-13T18:10:03.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.089+0000 7fbaf37fe700 1 -- 172.21.15.40:0/2883100553 learned_addr learned my addr 172.21.15.40:0/2883100553 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:10:03.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.093+0000 7fbaebfff700 1 -- 172.21.15.40:0/2883100553 >> v2:172.21.15.43:3300/0 conn(0x7fbaf406b080 msgr2=0x7fbaf40c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:10:03.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.093+0000 7fbaebfff700 1 --2- 172.21.15.40:0/2883100553 >> v2:172.21.15.43:3300/0 conn(0x7fbaf406b080 0x7fbaf40c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.093+0000 7fbaebfff700 1 -- 172.21.15.40:0/2883100553 >> v1:172.21.15.43:6789/0 conn(0x7fbaf40c7ed0 legacy=0x7fbaf41635f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:10:03.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.093+0000 7fbaebfff700 1 -- 172.21.15.40:0/2883100553 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf4166960 con 0x7fbaf4163d00 2024-09-13T18:10:03.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.093+0000 7fbaf37fe700 1 --2- 172.21.15.40:0/2883100553 >> v2:172.21.15.43:3300/0 conn(0x7fbaf406b080 0x7fbaf40c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:10:03.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.093+0000 7fbaebfff700 1 --2- 172.21.15.40:0/2883100553 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbaf4163d00 0x7fbaf40c7250 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009850 tx=0x7fbadc009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2523854fe9a7b43 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.093+0000 7fbaf27fc700 1 -- 172.21.15.40:0/2883100553 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc004230 con 0x7fbaf4163d00 2024-09-13T18:10:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.093+0000 7fbaf27fc700 1 -- 172.21.15.40:0/2883100553 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbadc004390 con 0x7fbaf4163d00 2024-09-13T18:10:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.093+0000 7fbaf27fc700 1 -- 172.21.15.40:0/2883100553 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc0036c0 con 0x7fbaf4163d00 2024-09-13T18:10:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.093+0000 7fbaf3fff700 1 -- 172.21.15.40:0/2883100553 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbaf4163d00 msgr2=0x7fbaf40c7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.093+0000 7fbaf3fff700 1 --2- 172.21.15.40:0/2883100553 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbaf4163d00 0x7fbaf40c7250 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009850 tx=0x7fbadc009b60 comp rx=0 tx=0).stop 2024-09-13T18:10:03.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.093+0000 7fbaf3fff700 1 -- 172.21.15.40:0/2883100553 shutdown_connections 2024-09-13T18:10:03.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.093+0000 7fbaf3fff700 1 --2- 172.21.15.40:0/2883100553 >> v2:172.21.15.43:3300/0 conn(0x7fbaf406b080 0x7fbaf40c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.093+0000 7fbaf3fff700 1 --2- 172.21.15.40:0/2883100553 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbaf4163d00 0x7fbaf40c7250 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.093+0000 7fbaf3fff700 1 -- 172.21.15.40:0/2883100553 >> 172.21.15.40:0/2883100553 conn(0x7fbaf4066090 msgr2=0x7fbaf40664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:03.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.097+0000 7fbaf3fff700 1 -- 172.21.15.40:0/2883100553 shutdown_connections 2024-09-13T18:10:03.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.097+0000 7fbaf3fff700 1 -- 172.21.15.40:0/2883100553 wait complete. 2024-09-13T18:10:03.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.097+0000 7fbaf3fff700 1 Processor -- start 2024-09-13T18:10:03.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.097+0000 7fbaf3fff700 1 -- start start 2024-09-13T18:10:03.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbaf406b080 0x7fbaf40c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf3fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbaf4163d00 0x7fbaf4160650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf40c21d0 0x7fbaf4160b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf3fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf40ad190 con 0x7fbaf40c21d0 2024-09-13T18:10:03.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf3fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf40c3cf0 con 0x7fbaf4163d00 2024-09-13T18:10:03.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf3fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf40c3730 con 0x7fbaf406b080 2024-09-13T18:10:03.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf40c21d0 0x7fbaf4160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf40c21d0 0x7fbaf4160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49248/0 (socket says 172.21.15.40:49248) 2024-09-13T18:10:03.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf37fe700 1 -- 172.21.15.40:0/2587729948 learned_addr learned my addr 172.21.15.40:0/2587729948 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:10:03.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaebfff700 1 --2- 172.21.15.40:0/2587729948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbaf4163d00 0x7fbaf4160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf2ffd700 1 --2- 172.21.15.40:0/2587729948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbaf406b080 0x7fbaf40c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf37fe700 1 -- 172.21.15.40:0/2587729948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbaf406b080 msgr2=0x7fbaf40c30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf37fe700 1 --2- 172.21.15.40:0/2587729948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbaf406b080 0x7fbaf40c30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf37fe700 1 -- 172.21.15.40:0/2587729948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbaf4163d00 msgr2=0x7fbaf4160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf37fe700 1 --2- 172.21.15.40:0/2587729948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbaf4163d00 0x7fbaf4160650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf37fe700 1 -- 172.21.15.40:0/2587729948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaec009b30 con 0x7fbaf40c21d0 2024-09-13T18:10:03.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf37fe700 1 --2- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf40c21d0 0x7fbaf4160b90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00efc0 tx=0x7fbaec009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:03.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaeb7fe700 1 -- 172.21.15.40:0/2587729948 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec018070 con 0x7fbaf40c21d0 2024-09-13T18:10:03.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaeb7fe700 1 -- 172.21.15.40:0/2587729948 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaec00f040 con 0x7fbaf40c21d0 2024-09-13T18:10:03.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaeb7fe700 1 -- 172.21.15.40:0/2587729948 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec003c60 con 0x7fbaf40c21d0 2024-09-13T18:10:03.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf3fff700 1 -- 172.21.15.40:0/2587729948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbadc0094b0 con 0x7fbaf40c21d0 2024-09-13T18:10:03.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf3fff700 1 -- 172.21.15.40:0/2587729948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf41fcdf0 con 0x7fbaf40c21d0 2024-09-13T18:10:03.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf3fff700 1 --2- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf41fcfe0 0x7fbaf40c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf3fff700 1 -- 172.21.15.40:0/2587729948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf4067720 con 0x7fbaf41fcfe0 2024-09-13T18:10:03.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.101+0000 7fbaf2ffd700 1 --2- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf41fcfe0 0x7fbaf40c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.105+0000 7fbaf2ffd700 1 -- 172.21.15.40:0/2587729948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbaf4166370 con 0x7fbaf41fcfe0 2024-09-13T18:10:03.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.105+0000 7fbaf2ffd700 1 --2- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf41fcfe0 0x7fbaf40c4ad0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbaf406e660 tx=0x7fbae400d880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:03.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.105+0000 7fbaeb7fe700 1 -- 172.21.15.40:0/2587729948 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae40125f0 con 0x7fbaf41fcfe0 2024-09-13T18:10:03.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.105+0000 7fbaeb7fe700 1 -- 172.21.15.40:0/2587729948 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fbae4012750 con 0x7fbaf41fcfe0 2024-09-13T18:10:03.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.105+0000 7fbaeb7fe700 1 -- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf41fcfe0 msgr2=0x7fbaf40c4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.105+0000 7fbaeb7fe700 1 --2- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf41fcfe0 0x7fbaf40c4ad0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbaf406e660 tx=0x7fbae400d880 comp rx=0 tx=0).stop 2024-09-13T18:10:03.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.105+0000 7fbaeb7fe700 1 -- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf41fcfe0 msgr2=0x7fbaf40c4ad0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:10:03.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.105+0000 7fbaeb7fe700 1 --2- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf41fcfe0 0x7fbaf40c4ad0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.105+0000 7fbaeb7fe700 1 -- 172.21.15.40:0/2587729948 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbaec00c9e0 con 0x7fbaf40c21d0 2024-09-13T18:10:03.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.105+0000 7fbaeb7fe700 1 --2- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbad4066470 0x7fbad4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.105+0000 7fbaeb7fe700 1 -- 172.21.15.40:0/2587729948 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbaec084760 con 0x7fbaf40c21d0 2024-09-13T18:10:03.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.105+0000 7fbaf2ffd700 1 --2- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbad4066470 0x7fbad4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.105+0000 7fbaf2ffd700 1 --2- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbad4066470 0x7fbad4068930 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbaf406e660 tx=0x7fbae4012000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:03.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaf3fff700 1 --2- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf41fd3c0 0x7fbaf40c5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaf3fff700 1 -- 172.21.15.40:0/2587729948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf4166f20 con 0x7fbaf41fd3c0 2024-09-13T18:10:03.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaebfff700 1 --2- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf41fd3c0 0x7fbaf40c5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaebfff700 1 -- 172.21.15.40:0/2587729948 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbaf40c82b0 con 0x7fbaf41fd3c0 2024-09-13T18:10:03.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaebfff700 1 --2- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf41fd3c0 0x7fbaf40c5010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbadc004700 tx=0x7fbadc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:03.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaeb7fe700 1 -- 172.21.15.40:0/2587729948 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc006ea0 con 0x7fbaf41fd3c0 2024-09-13T18:10:03.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaeb7fe700 1 -- 172.21.15.40:0/2587729948 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4452 (secure 0 0 0) 0x7fbadc020070 con 0x7fbaf41fd3c0 2024-09-13T18:10:03.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaeb7fe700 1 -- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf41fd3c0 msgr2=0x7fbaf40c5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaeb7fe700 1 --2- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf41fd3c0 0x7fbaf40c5010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbadc004700 tx=0x7fbadc004770 comp rx=0 tx=0).stop 2024-09-13T18:10:03.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaeb7fe700 1 -- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf41fd3c0 msgr2=0x7fbaf40c5010 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:10:03.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaeb7fe700 1 --2- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf41fd3c0 0x7fbaf40c5010 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.123 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:10:03.123 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:10:03.123 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:10:03.123 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:10:03.123 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 158, 2024-09-13T18:10:03.123 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:10:03.123 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:10:03.123 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:10:03.123 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:10:03.123 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:03.123 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:10:03.123 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:10:03.123 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:10:03.123 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:10:03.124 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:03.124 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:03.124 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:03.124 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:03.124 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:03.124 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:03.124 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:03.124 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:03.124 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:03.124 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:03.124 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:03.124 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:10:03.124 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:10:03.125 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:03.125 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:03.125 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:03.125 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:03.125 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:03.125 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:03.125 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:03.125 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:03.125 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:03.125 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:03.125 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:03.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.125 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:10:03.125 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:10:03.126 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:10:03.126 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:10:03.126 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:10:03.126 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:10:03.126 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:10:03.126 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:10:03.127 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:10:03.127 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:10:03.127 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:10:03.127 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:10:03.127 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:10:03.127 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:10:03.127 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:10:03.128 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:10:03.128 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:10:03.128 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:03.128 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:03.128 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:03.129 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:03.129 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:03.129 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:03.129 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:03.129 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:03.129 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:03.129 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:03.129 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:03.130 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:10:03.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.130 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:10:03.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.130 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:10:03.130 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:10:03.130 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:03.131 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:03.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.131 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:03.131 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:10:03.131 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:03.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.131 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:03.131 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:10:03.131 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:03.132 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.132 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:03.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.132 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:10:03.133 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:10:03.133 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:03.133 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:03.133 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:03.133 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.134 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.134 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:10:03.134 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:10:03.134 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:03.134 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:03.135 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.135 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:03.135 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:10:03.135 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:03.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.136 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.136 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:03.136 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:10:03.136 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:03.137 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.137 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:03.137 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.137 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:10:03.137 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:10:03.138 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:03.138 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:03.138 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:03.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.138 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.138 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:10:03.139 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:10:03.139 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:03.139 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:03.139 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.139 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:03.140 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:10:03.140 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:03.140 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.140 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.140 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:03.141 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:10:03.141 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:03.141 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.141 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:03.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.141 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:10:03.142 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:10:03.142 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:03.142 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:03.142 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:03.143 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.143 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:03.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.143 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:10:03.143 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:10:03.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.144 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:03.144 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:03.144 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:10:03.144 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.144 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:03.145 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:10:03.145 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.145 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:03.145 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:03.145 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:10:03.146 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.146 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:03.146 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.146 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:10:03.146 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:10:03.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaf3fff700 1 -- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbad4066470 msgr2=0x7fbad4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaf3fff700 1 --2- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbad4066470 0x7fbad4068930 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbaf406e660 tx=0x7fbae4012000 comp rx=0 tx=0).stop 2024-09-13T18:10:03.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaf3fff700 1 -- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf40c21d0 msgr2=0x7fbaf4160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaf3fff700 1 --2- 172.21.15.40:0/2587729948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbaf40c21d0 0x7fbaf4160b90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00efc0 tx=0x7fbaec009f90 comp rx=0 tx=0).stop 2024-09-13T18:10:03.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaf37fe700 1 -- 172.21.15.40:0/2587729948 reap_dead start 2024-09-13T18:10:03.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaf37fe700 1 -- 172.21.15.40:0/2587729948 reap_dead start 2024-09-13T18:10:03.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaf3fff700 1 -- 172.21.15.40:0/2587729948 shutdown_connections 2024-09-13T18:10:03.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaf3fff700 1 -- 172.21.15.40:0/2587729948 >> 172.21.15.40:0/2587729948 conn(0x7fbaf4066090 msgr2=0x7fbaf406ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:03.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaf3fff700 1 -- 172.21.15.40:0/2587729948 shutdown_connections 2024-09-13T18:10:03.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.113+0000 7fbaf3fff700 1 -- 172.21.15.40:0/2587729948 wait complete. 2024-09-13T18:10:03.152 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:10:03.152 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:10:03.152 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:10:03.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.257+0000 7ff34359e700 1 Processor -- start 2024-09-13T18:10:03.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.257+0000 7ff34359e700 1 -- start start 2024-09-13T18:10:03.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.257+0000 7ff34359e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff33c0d5050 0x7ff33c045470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34359e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff33c007e30 0x7ff33c0de1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34359e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33c04c840 con 0x7ff33c04b5a0 2024-09-13T18:10:03.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34359e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff33c04cde0 con 0x7ff33c0d5050 2024-09-13T18:10:03.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34359e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c0dbc00 con 0x7ff33c007e30 2024-09-13T18:10:03.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34259c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff33c0d5050 0x7ff33c045470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff342d9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff33c04b5a0 0x7ff33c0de6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55068/0 (socket says 172.21.15.40:55068) 2024-09-13T18:10:03.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff342d9d700 1 -- 172.21.15.40:0/3872065519 learned_addr learned my addr 172.21.15.40:0/3872065519 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:10:03.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff341d9b700 1 --2- 172.21.15.40:0/3872065519 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff33c007e30 0x7ff33c0de1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34159a700 1 -- 172.21.15.40:0/3872065519 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455003479 0 0) 0x7ff33c04c840 con 0x7ff33c04b5a0 2024-09-13T18:10:03.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34259c700 1 -- 172.21.15.40:0/3872065519 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff33c007e30 msgr2=0x7ff33c0de1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:10:03.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34259c700 1 --2- 172.21.15.40:0/3872065519 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff33c007e30 0x7ff33c0de1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34259c700 1 -- 172.21.15.40:0/3872065519 >> v1:172.21.15.40:6789/0 conn(0x7ff33c04b5a0 legacy=0x7ff33c0de6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34259c700 1 -- 172.21.15.40:0/3872065519 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c04cb00 con 0x7ff33c0d5050 2024-09-13T18:10:03.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34259c700 1 --2- 172.21.15.40:0/3872065519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33c0d5050 0x7ff33c045470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff334009850 tx=0x7ff334009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9703532cda887874 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:03.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34159a700 1 -- 172.21.15.40:0/3872065519 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334011830 con 0x7ff33c0d5050 2024-09-13T18:10:03.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34159a700 1 -- 172.21.15.40:0/3872065519 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff334011990 con 0x7ff33c0d5050 2024-09-13T18:10:03.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34159a700 1 -- 172.21.15.40:0/3872065519 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334011d40 con 0x7ff33c0d5050 2024-09-13T18:10:03.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34359e700 1 -- 172.21.15.40:0/3872065519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33c0d5050 msgr2=0x7ff33c045470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34359e700 1 --2- 172.21.15.40:0/3872065519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33c0d5050 0x7ff33c045470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff334009850 tx=0x7ff334009b60 comp rx=0 tx=0).stop 2024-09-13T18:10:03.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34359e700 1 -- 172.21.15.40:0/3872065519 shutdown_connections 2024-09-13T18:10:03.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34359e700 1 --2- 172.21.15.40:0/3872065519 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff33c007e30 0x7ff33c0de1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34359e700 1 --2- 172.21.15.40:0/3872065519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33c0d5050 0x7ff33c045470 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34359e700 1 -- 172.21.15.40:0/3872065519 >> 172.21.15.40:0/3872065519 conn(0x7ff33c050590 msgr2=0x7ff33c0509a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:03.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34359e700 1 -- 172.21.15.40:0/3872065519 shutdown_connections 2024-09-13T18:10:03.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.261+0000 7ff34359e700 1 -- 172.21.15.40:0/3872065519 wait complete. 2024-09-13T18:10:03.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff34359e700 1 Processor -- start 2024-09-13T18:10:03.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff34359e700 1 -- start start 2024-09-13T18:10:03.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff34359e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c007e30 0x7ff33c034860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff34359e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff33c04b5a0 0x7ff33c162740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff34359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33c0d5050 0x7ff33c0c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff34359e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c005b80 con 0x7ff33c0d5050 2024-09-13T18:10:03.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff34359e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c1733d0 con 0x7ff33c007e30 2024-09-13T18:10:03.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff34359e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c034da0 con 0x7ff33c04b5a0 2024-09-13T18:10:03.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff342d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33c0d5050 0x7ff33c0c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff342d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33c0d5050 0x7ff33c0c9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49290/0 (socket says 172.21.15.40:49290) 2024-09-13T18:10:03.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff342d9d700 1 -- 172.21.15.40:0/3292227528 learned_addr learned my addr 172.21.15.40:0/3292227528 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:10:03.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff341d9b700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff33c04b5a0 0x7ff33c162740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff34259c700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c007e30 0x7ff33c034860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff342d9d700 1 -- 172.21.15.40:0/3292227528 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff33c04b5a0 msgr2=0x7ff33c162740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff342d9d700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff33c04b5a0 0x7ff33c162740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff342d9d700 1 -- 172.21.15.40:0/3292227528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c007e30 msgr2=0x7ff33c034860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff342d9d700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c007e30 0x7ff33c034860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff342d9d700 1 -- 172.21.15.40:0/3292227528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3340094b0 con 0x7ff33c0d5050 2024-09-13T18:10:03.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff342d9d700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33c0d5050 0x7ff33c0c9ba0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff34404ffd0 tx=0x7ff3440747c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:03.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff332ffd700 1 -- 172.21.15.40:0/3292227528 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff344091070 con 0x7ff33c0d5050 2024-09-13T18:10:03.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff332ffd700 1 -- 172.21.15.40:0/3292227528 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff344074db0 con 0x7ff33c0d5050 2024-09-13T18:10:03.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff34359e700 1 -- 172.21.15.40:0/3292227528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33c034fd0 con 0x7ff33c0d5050 2024-09-13T18:10:03.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff332ffd700 1 -- 172.21.15.40:0/3292227528 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34406e9c0 con 0x7ff33c0d5050 2024-09-13T18:10:03.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.265+0000 7ff34359e700 1 -- 172.21.15.40:0/3292227528 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33c0052e0 con 0x7ff33c0d5050 2024-09-13T18:10:03.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.269+0000 7ff34359e700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c0ca0e0 0x7ff33c005470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.269+0000 7ff34359e700 1 -- 172.21.15.40:0/3292227528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c008d60 con 0x7ff33c0ca0e0 2024-09-13T18:10:03.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.269+0000 7ff34259c700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c0ca0e0 0x7ff33c005470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.269+0000 7ff332ffd700 1 -- 172.21.15.40:0/3292227528 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff34406eb20 con 0x7ff33c0d5050 2024-09-13T18:10:03.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.269+0000 7ff34259c700 1 -- 172.21.15.40:0/3292227528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff33c007910 con 0x7ff33c0ca0e0 2024-09-13T18:10:03.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.269+0000 7ff332ffd700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff324066730 0x7ff324068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.269+0000 7ff332ffd700 1 -- 172.21.15.40:0/3292227528 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff3441026f0 con 0x7ff33c0d5050 2024-09-13T18:10:03.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.269+0000 7ff341d9b700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff324066730 0x7ff324068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.269+0000 7ff34259c700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c0ca0e0 0x7ff33c005470 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff334009850 tx=0x7ff3340047e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:03.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.269+0000 7ff332ffd700 1 -- 172.21.15.40:0/3292227528 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3340036c0 con 0x7ff33c0ca0e0 2024-09-13T18:10:03.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.269+0000 7ff341d9b700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff324066730 0x7ff324068bf0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff33c167640 tx=0x7ff338009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:03.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.269+0000 7ff332ffd700 1 -- 172.21.15.40:0/3292227528 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff334003820 con 0x7ff33c0ca0e0 2024-09-13T18:10:03.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.269+0000 7ff332ffd700 1 -- 172.21.15.40:0/3292227528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c0ca0e0 msgr2=0x7ff33c005470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.269+0000 7ff332ffd700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c0ca0e0 0x7ff33c005470 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff334009850 tx=0x7ff3340047e0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.269+0000 7ff332ffd700 1 -- 172.21.15.40:0/3292227528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c0ca0e0 msgr2=0x7ff33c005470 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:10:03.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.269+0000 7ff332ffd700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c0ca0e0 0x7ff33c005470 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff334009850 tx=0x7ff3340047e0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff34359e700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c173820 0x7ff33c173c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff34359e700 1 -- 172.21.15.40:0/3292227528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c007630 con 0x7ff33c173820 2024-09-13T18:10:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff34259c700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c173820 0x7ff33c173c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff34259c700 1 -- 172.21.15.40:0/3292227528 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff33c0459b0 con 0x7ff33c173820 2024-09-13T18:10:03.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff34259c700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c173820 0x7ff33c173c30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff334009ba0 tx=0x7ff334004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:03.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff332ffd700 1 -- 172.21.15.40:0/3292227528 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334003ee0 con 0x7ff33c173820 2024-09-13T18:10:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff332ffd700 1 -- 172.21.15.40:0/3292227528 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7ff33402c720 con 0x7ff33c173820 2024-09-13T18:10:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff332ffd700 1 -- 172.21.15.40:0/3292227528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c173820 msgr2=0x7ff33c173c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff332ffd700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c173820 0x7ff33c173c30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff334009ba0 tx=0x7ff334004770 comp rx=0 tx=0).stop 2024-09-13T18:10:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff332ffd700 1 -- 172.21.15.40:0/3292227528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c173820 msgr2=0x7ff33c173c30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:10:03.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff332ffd700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff33c173820 0x7ff33c173c30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.290 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:10:03.290 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:10:03.290 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:10:03.290 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:10:03.291 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 158, 2024-09-13T18:10:03.291 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:10:03.291 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:10:03.291 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:10:03.291 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:10:03.291 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:03.291 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:10:03.291 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:10:03.291 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:10:03.291 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:10:03.291 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:03.291 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:03.291 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:03.291 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:03.292 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:03.292 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:03.292 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:03.292 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:03.292 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:03.292 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:03.292 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:03.292 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:10:03.292 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:10:03.292 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:03.292 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:03.293 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:03.293 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:03.293 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:03.293 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:03.293 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:03.293 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:03.293 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:03.293 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:03.293 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:03.293 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.293 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:10:03.294 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:10:03.294 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:10:03.294 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:10:03.294 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:10:03.294 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:10:03.294 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:10:03.294 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:10:03.294 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:10:03.294 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:10:03.294 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:10:03.294 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:10:03.294 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:10:03.295 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:10:03.295 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:10:03.295 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:10:03.295 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:10:03.295 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:03.295 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:03.295 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:03.295 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:03.295 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:03.295 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:03.295 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:03.295 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:03.295 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:03.295 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:03.296 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:03.296 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:10:03.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.296 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:10:03.296 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.296 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:10:03.296 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:10:03.296 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:03.296 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:03.296 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.296 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:03.296 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:10:03.296 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:03.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.297 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:03.297 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:10:03.297 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:03.297 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.297 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:03.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.297 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:10:03.297 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:10:03.297 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:03.297 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:03.297 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:03.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.298 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.298 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:10:03.298 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:10:03.298 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:03.298 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:03.298 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.298 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:03.298 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:10:03.298 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:03.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.298 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.298 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:03.298 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:10:03.298 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:03.299 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.299 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:03.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.299 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:10:03.299 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:10:03.299 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:03.299 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:03.299 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:03.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.299 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:10:03.299 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:10:03.300 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:03.300 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:03.300 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.300 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:03.300 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:10:03.300 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:03.300 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.300 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.300 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:03.300 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:10:03.300 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:03.300 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.301 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:03.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.301 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:10:03.301 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:10:03.301 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:03.301 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:03.301 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:03.301 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.301 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:03.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.301 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:10:03.301 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:10:03.301 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.302 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:03.302 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:03.302 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:10:03.302 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.302 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:03.302 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:10:03.302 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.302 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:03.302 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:03.302 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:10:03.302 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.303 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:03.303 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:10:03.303 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.303 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:03.303 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:03.303 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 8 2024-09-13T18:10:03.304 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.304 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:03.304 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:10:03.304 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.304 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:03.304 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:03.304 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:10:03.305 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.305 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:03.305 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:10:03.305 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.305 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:03.305 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:03.306 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:10:03.306 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.306 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:03.306 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.306 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:10:03.306 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:10:03.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff34359e700 1 -- 172.21.15.40:0/3292227528 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff324066730 msgr2=0x7ff324068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff34359e700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff324066730 0x7ff324068bf0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff33c167640 tx=0x7ff338009240 comp rx=0 tx=0).stop 2024-09-13T18:10:03.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff34359e700 1 -- 172.21.15.40:0/3292227528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33c0d5050 msgr2=0x7ff33c0c9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff342d9d700 1 -- 172.21.15.40:0/3292227528 reap_dead start 2024-09-13T18:10:03.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff34359e700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33c0d5050 0x7ff33c0c9ba0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff34404ffd0 tx=0x7ff3440747c0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff34359e700 1 -- 172.21.15.40:0/3292227528 shutdown_connections 2024-09-13T18:10:03.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff34359e700 1 --2- 172.21.15.40:0/3292227528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff33c0d5050 0x7ff33c0c9ba0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff34359e700 1 -- 172.21.15.40:0/3292227528 >> 172.21.15.40:0/3292227528 conn(0x7ff33c050590 msgr2=0x7ff33c0d0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:03.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff34359e700 1 -- 172.21.15.40:0/3292227528 shutdown_connections 2024-09-13T18:10:03.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.281+0000 7ff34359e700 1 -- 172.21.15.40:0/3292227528 wait complete. 2024-09-13T18:10:03.311 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:10:03.311 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:10:03.311 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:10:03.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2ad339700 1 Processor -- start 2024-09-13T18:10:03.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2ad339700 1 -- start start 2024-09-13T18:10:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2ad339700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe2a80724d0 0x7fe2a8070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2ad339700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2a815aea0 con 0x7fe2a80c6190 2024-09-13T18:10:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2ad339700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2a8159e30 con 0x7fe2a8153150 2024-09-13T18:10:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2ad339700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2a815a0d0 con 0x7fe2a80724d0 2024-09-13T18:10:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2acb38700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe2a80c6190 0x7fe2a8070fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55080/0 (socket says 172.21.15.40:55080) 2024-09-13T18:10:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2acb38700 1 -- 172.21.15.40:0/2032865792 learned_addr learned my addr 172.21.15.40:0/2032865792 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:10:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2a77fe700 1 --2- 172.21.15.40:0/2032865792 >> v2:172.21.15.43:3300/0 conn(0x7fe2a80724d0 0x7fe2a8070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2032865792 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2877328437 0 0) 0x7fe2a815aea0 con 0x7fe2a80c6190 2024-09-13T18:10:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2032865792 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2900035d0 con 0x7fe2a80c6190 2024-09-13T18:10:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2032865792 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1978775814 0 0) 0x7fe2900035d0 con 0x7fe2a80c6190 2024-09-13T18:10:03.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2032865792 >> v2:172.21.15.43:3300/0 conn(0x7fe2a80724d0 msgr2=0x7fe2a8070aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:10:03.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2a6ffd700 1 --2- 172.21.15.40:0/2032865792 >> v2:172.21.15.43:3300/0 conn(0x7fe2a80724d0 0x7fe2a8070aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2032865792 >> v1:172.21.15.43:6789/0 conn(0x7fe2a8153150 legacy=0x7fe2a815a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2032865792 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a80cb2a0 con 0x7fe2a80c6190 2024-09-13T18:10:03.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2032865792 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fe2a00019f0 con 0x7fe2a80c6190 2024-09-13T18:10:03.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2032865792 >> v1:172.21.15.40:6789/0 conn(0x7fe2a80c6190 legacy=0x7fe2a8070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2a6ffd700 1 --2- 172.21.15.40:0/2032865792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe290003a20 0x7fe290003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2a6ffd700 1 --2- 172.21.15.40:0/2032865792 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe290004870 0x7fe290004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2a6ffd700 1 --2- 172.21.15.40:0/2032865792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe290005230 0x7fe29000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2032865792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a815a0d0 con 0x7fe290003a20 2024-09-13T18:10:03.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2032865792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29000c0e0 con 0x7fe290005230 2024-09-13T18:10:03.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.417+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2032865792 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29000c2a0 con 0x7fe290004870 2024-09-13T18:10:03.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2a7fff700 1 --2- 172.21.15.40:0/2032865792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe290003a20 0x7fe290003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2a77fe700 1 --2- 172.21.15.40:0/2032865792 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe290004870 0x7fe290004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2acb38700 1 --2- 172.21.15.40:0/2032865792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe290005230 0x7fe29000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2a7fff700 1 -- 172.21.15.40:0/2032865792 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe290004870 msgr2=0x7fe290004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:10:03.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2a7fff700 1 --2- 172.21.15.40:0/2032865792 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe290004870 0x7fe290004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2a7fff700 1 -- 172.21.15.40:0/2032865792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe290005230 msgr2=0x7fe29000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:10:03.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2a7fff700 1 --2- 172.21.15.40:0/2032865792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe290005230 0x7fe29000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2a7fff700 1 -- 172.21.15.40:0/2032865792 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2900035d0 con 0x7fe290003a20 2024-09-13T18:10:03.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2acb38700 1 -- 172.21.15.40:0/2032865792 reap_dead start 2024-09-13T18:10:03.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2a7fff700 1 --2- 172.21.15.40:0/2032865792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe290003a20 0x7fe290003e40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe2980017e0 tx=0x7fe2900058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ef5183b5e5e17a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:03.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2032865792 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2980052d0 con 0x7fe290003a20 2024-09-13T18:10:03.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2032865792 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe298005450 con 0x7fe290003a20 2024-09-13T18:10:03.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2032865792 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2980038e0 con 0x7fe290003a20 2024-09-13T18:10:03.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2ad339700 1 -- 172.21.15.40:0/2032865792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe290003a20 msgr2=0x7fe290003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2ad339700 1 --2- 172.21.15.40:0/2032865792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe290003a20 0x7fe290003e40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe2980017e0 tx=0x7fe2900058a0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2ad339700 1 -- 172.21.15.40:0/2032865792 shutdown_connections 2024-09-13T18:10:03.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2ad339700 1 --2- 172.21.15.40:0/2032865792 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe290003a20 0x7fe290003e40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2ad339700 1 -- 172.21.15.40:0/2032865792 >> 172.21.15.40:0/2032865792 conn(0x7fe2a806a390 msgr2=0x7fe2a806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:03.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2ad339700 1 -- 172.21.15.40:0/2032865792 shutdown_connections 2024-09-13T18:10:03.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2ad339700 1 -- 172.21.15.40:0/2032865792 wait complete. 2024-09-13T18:10:03.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2ad339700 1 Processor -- start 2024-09-13T18:10:03.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.421+0000 7fe2ad339700 1 -- start start 2024-09-13T18:10:03.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2ad339700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe290003a20 0x7fe2a81e3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2ad339700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2a814fdd0 0x7fe2a8150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2ad339700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2a81f2480 0x7fe2a81f2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2ad339700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a8151370 con 0x7fe290003a20 2024-09-13T18:10:03.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2ad339700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a81f1ff0 con 0x7fe2a81f2480 2024-09-13T18:10:03.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2ad339700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a8150790 con 0x7fe2a814fdd0 2024-09-13T18:10:03.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe290003a20 0x7fe2a81e3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe290003a20 0x7fe2a81e3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49300/0 (socket says 172.21.15.40:49300) 2024-09-13T18:10:03.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2a7fff700 1 -- 172.21.15.40:0/3122836672 learned_addr learned my addr 172.21.15.40:0/3122836672 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:10:03.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2a77fe700 1 --2- 172.21.15.40:0/3122836672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2a814fdd0 0x7fe2a8150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2acb38700 1 --2- 172.21.15.40:0/3122836672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2a81f2480 0x7fe2a81f2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2a7fff700 1 -- 172.21.15.40:0/3122836672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2a814fdd0 msgr2=0x7fe2a8150250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2a7fff700 1 --2- 172.21.15.40:0/3122836672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2a814fdd0 0x7fe2a8150250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2a7fff700 1 -- 172.21.15.40:0/3122836672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2a81f2480 msgr2=0x7fe2a81f2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2a7fff700 1 --2- 172.21.15.40:0/3122836672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2a81f2480 0x7fe2a81f2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2a7fff700 1 -- 172.21.15.40:0/3122836672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a81f2e40 con 0x7fe290003a20 2024-09-13T18:10:03.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2a7fff700 1 --2- 172.21.15.40:0/3122836672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe290003a20 0x7fe2a81e3550 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe298005040 tx=0x7fe298001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:03.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2a4ff9700 1 -- 172.21.15.40:0/3122836672 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe298001290 con 0x7fe290003a20 2024-09-13T18:10:03.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2a4ff9700 1 -- 172.21.15.40:0/3122836672 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe298007040 con 0x7fe290003a20 2024-09-13T18:10:03.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2a4ff9700 1 -- 172.21.15.40:0/3122836672 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2980073f0 con 0x7fe290003a20 2024-09-13T18:10:03.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2ad339700 1 -- 172.21.15.40:0/3122836672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a81ef4a0 con 0x7fe290003a20 2024-09-13T18:10:03.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2ad339700 1 -- 172.21.15.40:0/3122836672 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a81ef740 con 0x7fe290003a20 2024-09-13T18:10:03.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2ad339700 1 --2- 172.21.15.40:0/3122836672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2a81515c0 0x7fe2a81e1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2ad339700 1 -- 172.21.15.40:0/3122836672 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a8067600 con 0x7fe2a81515c0 2024-09-13T18:10:03.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2a77fe700 1 --2- 172.21.15.40:0/3122836672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2a81515c0 0x7fe2a81e1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2a77fe700 1 -- 172.21.15.40:0/3122836672 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe2a8067dd0 con 0x7fe2a81515c0 2024-09-13T18:10:03.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2a4ff9700 1 -- 172.21.15.40:0/3122836672 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe298007550 con 0x7fe290003a20 2024-09-13T18:10:03.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.425+0000 7fe2a77fe700 1 --2- 172.21.15.40:0/3122836672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2a81515c0 0x7fe2a81e1530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0146a0 tx=0x7fe29c014a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:03.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.429+0000 7fe2a4ff9700 1 --2- 172.21.15.40:0/3122836672 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe294066730 0x7fe294068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.429+0000 7fe2a4ff9700 1 -- 172.21.15.40:0/3122836672 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe298084dc0 con 0x7fe290003a20 2024-09-13T18:10:03.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.429+0000 7fe2acb38700 1 --2- 172.21.15.40:0/3122836672 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe294066730 0x7fe294068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.429+0000 7fe2a4ff9700 1 -- 172.21.15.40:0/3122836672 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c0044a0 con 0x7fe2a81515c0 2024-09-13T18:10:03.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.429+0000 7fe2a4ff9700 1 -- 172.21.15.40:0/3122836672 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe29c0036c0 con 0x7fe2a81515c0 2024-09-13T18:10:03.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.429+0000 7fe2a4ff9700 1 -- 172.21.15.40:0/3122836672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2a81515c0 msgr2=0x7fe2a81e1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.429+0000 7fe2a4ff9700 1 --2- 172.21.15.40:0/3122836672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2a81515c0 0x7fe2a81e1530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0146a0 tx=0x7fe29c014a60 comp rx=0 tx=0).stop 2024-09-13T18:10:03.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.429+0000 7fe2acb38700 1 --2- 172.21.15.40:0/3122836672 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe294066730 0x7fe294068bf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80c6570 tx=0x7fe2a00021d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:03.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.429+0000 7fe2a4ff9700 1 -- 172.21.15.40:0/3122836672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2a81515c0 msgr2=0x7fe2a81e1530 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:10:03.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.429+0000 7fe2a4ff9700 1 --2- 172.21.15.40:0/3122836672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2a81515c0 0x7fe2a81e1530 secure :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0146a0 tx=0x7fe29c014a60 comp rx=0 tx=0).stop 2024-09-13T18:10:03.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.437+0000 7fe2ad339700 1 --2- 172.21.15.40:0/3122836672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2a81519a0 0x7fe2a81e1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.437+0000 7fe2ad339700 1 -- 172.21.15.40:0/3122836672 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a80cbb40 con 0x7fe2a81519a0 2024-09-13T18:10:03.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.437+0000 7fe2a77fe700 1 --2- 172.21.15.40:0/3122836672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2a81519a0 0x7fe2a81e1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.437+0000 7fe2a77fe700 1 -- 172.21.15.40:0/3122836672 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe2a80cb860 con 0x7fe2a81519a0 2024-09-13T18:10:03.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.437+0000 7fe2a77fe700 1 --2- 172.21.15.40:0/3122836672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2a81519a0 0x7fe2a81e1a70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0146a0 tx=0x7fe29c0047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:03.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.437+0000 7fe2a4ff9700 1 -- 172.21.15.40:0/3122836672 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c02d070 con 0x7fe2a81519a0 2024-09-13T18:10:03.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.441+0000 7fe2a4ff9700 1 -- 172.21.15.40:0/3122836672 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4616 (secure 0 0 0) 0x7fe29c025040 con 0x7fe2a81519a0 2024-09-13T18:10:03.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.441+0000 7fe2a4ff9700 1 -- 172.21.15.40:0/3122836672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2a81519a0 msgr2=0x7fe2a81e1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.441+0000 7fe2a4ff9700 1 --2- 172.21.15.40:0/3122836672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2a81519a0 0x7fe2a81e1a70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0146a0 tx=0x7fe29c0047e0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.441+0000 7fe2a4ff9700 1 -- 172.21.15.40:0/3122836672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2a81519a0 msgr2=0x7fe2a81e1a70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:10:03.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.441+0000 7fe2a4ff9700 1 --2- 172.21.15.40:0/3122836672 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2a81519a0 0x7fe2a81e1a70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.448 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:10:03.448 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:10:03.448 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:10:03.448 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:10:03.448 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 158, 2024-09-13T18:10:03.448 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:10:03.448 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:10:03.448 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:10:03.449 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:10:03.449 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:03.449 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:10:03.449 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:10:03.449 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:10:03.449 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:10:03.449 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:03.449 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:03.449 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:03.449 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:03.449 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:03.449 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:03.449 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:03.449 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:03.450 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:03.450 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:03.450 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:03.450 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:10:03.450 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:10:03.450 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:03.450 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:03.450 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:03.450 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:03.450 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:03.450 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:03.450 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:03.450 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:03.451 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:03.451 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:03.451 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:03.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.451 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:10:03.451 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:10:03.451 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:10:03.451 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:10:03.451 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:10:03.451 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:10:03.451 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:10:03.451 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:10:03.451 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:10:03.451 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:10:03.452 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:10:03.452 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:10:03.452 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:10:03.452 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:10:03.452 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:10:03.452 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:10:03.452 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:10:03.452 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:03.452 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:03.452 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:03.452 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:03.452 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:03.452 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:03.452 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:03.453 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:03.453 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:03.453 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:03.453 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:03.453 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:10:03.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.453 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:10:03.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.453 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:10:03.453 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:10:03.453 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:03.453 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:03.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.454 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:03.454 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:10:03.454 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:03.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.454 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.454 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:03.454 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:10:03.454 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:03.454 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.454 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:03.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.454 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:10:03.454 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:10:03.455 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:03.455 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:03.455 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:03.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.455 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.455 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:10:03.455 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:10:03.455 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:03.455 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:03.455 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.455 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:03.455 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:10:03.455 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:03.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.456 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:03.456 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:10:03.456 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:03.456 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.456 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:03.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.456 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:10:03.456 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:10:03.456 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:03.456 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:03.456 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:03.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.457 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:10:03.457 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:10:03.457 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:03.457 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:03.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.457 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:03.457 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:10:03.457 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:03.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.457 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:03.457 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:10:03.458 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:03.458 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.458 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:03.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.458 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:10:03.458 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:10:03.458 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:03.458 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:03.458 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:03.458 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.458 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:03.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.458 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:10:03.459 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:10:03.459 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.459 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:03.459 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:03.459 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:10:03.459 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.459 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:03.459 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:10:03.459 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.459 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:03.459 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:03.459 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:10:03.459 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.459 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:03.460 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:10:03.460 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:03.460 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:03.460 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:03.460 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:10:03.460 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:03.460 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:03.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:03.460 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:10:03.460 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:10:03.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.441+0000 7fe2ad339700 1 -- 172.21.15.40:0/3122836672 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe294066730 msgr2=0x7fe294068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.441+0000 7fe2ad339700 1 --2- 172.21.15.40:0/3122836672 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe294066730 0x7fe294068bf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80c6570 tx=0x7fe2a00021d0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.441+0000 7fe2ad339700 1 -- 172.21.15.40:0/3122836672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe290003a20 msgr2=0x7fe2a81e3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.441+0000 7fe2ad339700 1 --2- 172.21.15.40:0/3122836672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe290003a20 0x7fe2a81e3550 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe298005040 tx=0x7fe298001120 comp rx=0 tx=0).stop 2024-09-13T18:10:03.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.441+0000 7fe2acb38700 1 -- 172.21.15.40:0/3122836672 reap_dead start 2024-09-13T18:10:03.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.441+0000 7fe2acb38700 1 -- 172.21.15.40:0/3122836672 reap_dead start 2024-09-13T18:10:03.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.441+0000 7fe2ad339700 1 -- 172.21.15.40:0/3122836672 shutdown_connections 2024-09-13T18:10:03.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.441+0000 7fe2ad339700 1 -- 172.21.15.40:0/3122836672 >> 172.21.15.40:0/3122836672 conn(0x7fe2a806a390 msgr2=0x7fe2a815b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:03.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.441+0000 7fe2ad339700 1 -- 172.21.15.40:0/3122836672 shutdown_connections 2024-09-13T18:10:03.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.441+0000 7fe2ad339700 1 -- 172.21.15.40:0/3122836672 wait complete. 2024-09-13T18:10:03.464 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:10:03.465 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:10:03.465 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:10:03.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2f367700 1 Processor -- start 2024-09-13T18:10:03.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2f367700 1 -- start start 2024-09-13T18:10:03.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2f367700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6c200d8920 0x7f6c20049090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2f367700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6c20007480 0x7f6c2004efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2f367700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c20054df0 con 0x7f6c200e1c00 2024-09-13T18:10:03.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2f367700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c20050a10 con 0x7f6c200d8920 2024-09-13T18:10:03.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2f367700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c20006ff0 con 0x7f6c20007480 2024-09-13T18:10:03.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2e365700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6c200d8920 0x7f6c20049090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2e365700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6c200d8920 0x7f6c20049090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49306/0 (socket says 172.21.15.40:49306) 2024-09-13T18:10:03.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2e365700 1 -- 172.21.15.40:0/2268094075 learned_addr learned my addr 172.21.15.40:0/2268094075 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:10:03.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2db64700 1 --2- 172.21.15.40:0/2268094075 >> v2:172.21.15.43:3300/0 conn(0x7f6c20007480 0x7f6c2004efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2e365700 1 -- 172.21.15.40:0/2268094075 >> v2:172.21.15.43:3300/0 conn(0x7f6c20007480 msgr2=0x7f6c2004efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:10:03.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2e365700 1 --2- 172.21.15.40:0/2268094075 >> v2:172.21.15.43:3300/0 conn(0x7f6c20007480 0x7f6c2004efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2e365700 1 -- 172.21.15.40:0/2268094075 >> v1:172.21.15.40:6789/0 conn(0x7f6c200e1c00 legacy=0x7f6c200e2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2d363700 1 -- 172.21.15.40:0/2268094075 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090911979 0 0) 0x7f6c20054df0 con 0x7f6c200e1c00 2024-09-13T18:10:03.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2e365700 1 -- 172.21.15.40:0/2268094075 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c200495d0 con 0x7f6c200d8920 2024-09-13T18:10:03.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2e365700 1 --2- 172.21.15.40:0/2268094075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c200d8920 0x7f6c20049090 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6c20011b40 tx=0x7f6c24009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39714f77966bdd43 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:03.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2d363700 1 -- 172.21.15.40:0/2268094075 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2400b8a0 con 0x7f6c200d8920 2024-09-13T18:10:03.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2d363700 1 -- 172.21.15.40:0/2268094075 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2400ba00 con 0x7f6c200d8920 2024-09-13T18:10:03.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2d363700 1 -- 172.21.15.40:0/2268094075 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2400bd90 con 0x7f6c200d8920 2024-09-13T18:10:03.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2f367700 1 -- 172.21.15.40:0/2268094075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c200d8920 msgr2=0x7f6c20049090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2f367700 1 --2- 172.21.15.40:0/2268094075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c200d8920 0x7f6c20049090 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6c20011b40 tx=0x7f6c24009bf0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2f367700 1 -- 172.21.15.40:0/2268094075 shutdown_connections 2024-09-13T18:10:03.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2f367700 1 --2- 172.21.15.40:0/2268094075 >> v2:172.21.15.43:3300/0 conn(0x7f6c20007480 0x7f6c2004efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2f367700 1 --2- 172.21.15.40:0/2268094075 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c200d8920 0x7f6c20049090 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.581+0000 7f6c2f367700 1 -- 172.21.15.40:0/2268094075 >> 172.21.15.40:0/2268094075 conn(0x7f6c20054110 msgr2=0x7f6c20054520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:03.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2f367700 1 -- 172.21.15.40:0/2268094075 shutdown_connections 2024-09-13T18:10:03.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2f367700 1 -- 172.21.15.40:0/2268094075 wait complete. 2024-09-13T18:10:03.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2f367700 1 Processor -- start 2024-09-13T18:10:03.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2f367700 1 -- start start 2024-09-13T18:10:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2f367700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6c20007480 0x7f6c200da020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2f367700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c200d8920 0x7f6c200da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2f367700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c200e1c00 0x7f6c200396c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2f367700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2004c920 con 0x7f6c200e1c00 2024-09-13T18:10:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2f367700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2004ddd0 con 0x7f6c200d8920 2024-09-13T18:10:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2f367700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2004ae70 con 0x7f6c20007480 2024-09-13T18:10:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2eb66700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c200e1c00 0x7f6c200396c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2eb66700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c200e1c00 0x7f6c200396c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49314/0 (socket says 172.21.15.40:49314) 2024-09-13T18:10:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2eb66700 1 -- 172.21.15.40:0/3764469342 learned_addr learned my addr 172.21.15.40:0/3764469342 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:10:03.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2e365700 1 --2- 172.21.15.40:0/3764469342 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6c20007480 0x7f6c200da020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2db64700 1 --2- 172.21.15.40:0/3764469342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c200d8920 0x7f6c200da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2eb66700 1 -- 172.21.15.40:0/3764469342 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6c20007480 msgr2=0x7f6c200da020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2eb66700 1 --2- 172.21.15.40:0/3764469342 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6c20007480 0x7f6c200da020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2eb66700 1 -- 172.21.15.40:0/3764469342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c200d8920 msgr2=0x7f6c200da560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2eb66700 1 --2- 172.21.15.40:0/3764469342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c200d8920 0x7f6c200da560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2eb66700 1 -- 172.21.15.40:0/3764469342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c24009570 con 0x7f6c200e1c00 2024-09-13T18:10:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2e365700 1 --2- 172.21.15.40:0/3764469342 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6c20007480 0x7f6c200da020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:10:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2eb66700 1 --2- 172.21.15.40:0/3764469342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c200e1c00 0x7f6c200396c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6c28000b60 tx=0x7f6c28074f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c1effd700 1 -- 172.21.15.40:0/3764469342 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28067730 con 0x7f6c200e1c00 2024-09-13T18:10:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c1effd700 1 -- 172.21.15.40:0/3764469342 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c28067890 con 0x7f6c200e1c00 2024-09-13T18:10:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2f367700 1 -- 172.21.15.40:0/3764469342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2004c3c0 con 0x7f6c200e1c00 2024-09-13T18:10:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c1effd700 1 -- 172.21.15.40:0/3764469342 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28072370 con 0x7f6c200e1c00 2024-09-13T18:10:03.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.585+0000 7f6c2f367700 1 -- 172.21.15.40:0/3764469342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2004e000 con 0x7f6c200e1c00 2024-09-13T18:10:03.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.589+0000 7f6c2f367700 1 -- 172.21.15.40:0/3764469342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c20050730 con 0x7f6c200e1c00 2024-09-13T18:10:03.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.589+0000 7f6c1effd700 1 -- 172.21.15.40:0/3764469342 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6c280724d0 con 0x7f6c200e1c00 2024-09-13T18:10:03.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.589+0000 7f6c1effd700 1 --2- 172.21.15.40:0/3764469342 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6c0c066760 0x7f6c0c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:03.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.589+0000 7f6c1effd700 1 -- 172.21.15.40:0/3764469342 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6c280fea50 con 0x7f6c200e1c00 2024-09-13T18:10:03.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.589+0000 7f6c2e365700 1 --2- 172.21.15.40:0/3764469342 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6c0c066760 0x7f6c0c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:03.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.589+0000 7f6c2e365700 1 --2- 172.21.15.40:0/3764469342 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6c0c066760 0x7f6c0c068c20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6c24000c00 tx=0x7f6c24008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:03.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.597+0000 7f6c1effd700 1 -- 172.21.15.40:0/3764469342 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6c280cba00 con 0x7f6c200e1c00 2024-09-13T18:10:03.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.849+0000 7f6c2f367700 1 -- 172.21.15.40:0/3764469342 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f6c20049c00 con 0x7f6c200e1c00 2024-09-13T18:10:03.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.849+0000 7f6c1effd700 1 -- 172.21.15.40:0/3764469342 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f6c280cf020 con 0x7f6c200e1c00 2024-09-13T18:10:03.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.853+0000 7f6c2f367700 1 -- 172.21.15.40:0/3764469342 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6c0c066760 msgr2=0x7f6c0c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.853+0000 7f6c2f367700 1 --2- 172.21.15.40:0/3764469342 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6c0c066760 0x7f6c0c068c20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6c24000c00 tx=0x7f6c24008030 comp rx=0 tx=0).stop 2024-09-13T18:10:03.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.853+0000 7f6c2f367700 1 -- 172.21.15.40:0/3764469342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c200e1c00 msgr2=0x7f6c200396c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:03.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.853+0000 7f6c2f367700 1 --2- 172.21.15.40:0/3764469342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c200e1c00 0x7f6c200396c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6c28000b60 tx=0x7f6c28074f20 comp rx=0 tx=0).stop 2024-09-13T18:10:03.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.853+0000 7f6c2f367700 1 -- 172.21.15.40:0/3764469342 shutdown_connections 2024-09-13T18:10:03.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.853+0000 7f6c2f367700 1 --2- 172.21.15.40:0/3764469342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c200e1c00 0x7f6c200396c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.853+0000 7f6c2f367700 1 --2- 172.21.15.40:0/3764469342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c200d8920 0x7f6c200da560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.853+0000 7f6c2f367700 1 --2- 172.21.15.40:0/3764469342 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6c0c066760 0x7f6c0c068c20 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.853+0000 7f6c2f367700 1 --2- 172.21.15.40:0/3764469342 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6c20007480 0x7f6c200da020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:03.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.853+0000 7f6c2f367700 1 -- 172.21.15.40:0/3764469342 >> 172.21.15.40:0/3764469342 conn(0x7f6c20054110 msgr2=0x7f6c200050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:03.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.853+0000 7f6c2f367700 1 -- 172.21.15.40:0/3764469342 shutdown_connections 2024-09-13T18:10:03.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:03.853+0000 7f6c2f367700 1 -- 172.21.15.40:0/3764469342 wait complete. 2024-09-13T18:10:03.877 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:10:07.134 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:10:07.136 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:10:13.878 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:10:13.878 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:10:13.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.989+0000 7f19fb181700 1 Processor -- start 2024-09-13T18:10:13.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.989+0000 7f19fb181700 1 -- start start 2024-09-13T18:10:13.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fb181700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19f4151470 0x7f19f40c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:13.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fb181700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f19f40b1bd0 0x7f19f41571d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:13.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fb181700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f19f4157710 0x7f19f4071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:13.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fb181700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f40c4520 con 0x7f19f4157710 2024-09-13T18:10:13.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fb181700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f40c4dc0 con 0x7f19f40b1bd0 2024-09-13T18:10:13.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fb181700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f4157cf0 con 0x7f19f4151470 2024-09-13T18:10:13.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fa980700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f19f4157710 0x7f19f4071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fa980700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f19f4157710 0x7f19f4071bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40826/0 (socket says 172.21.15.40:40826) 2024-09-13T18:10:14.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fa980700 1 -- 172.21.15.40:0/1930890988 learned_addr learned my addr 172.21.15.40:0/1930890988 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:10:14.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fa17f700 1 --2- 172.21.15.40:0/1930890988 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19f4151470 0x7f19f40c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19f997e700 1 --2- 172.21.15.40:0/1930890988 >> v2:172.21.15.43:3300/0 conn(0x7f19f40b1bd0 0x7f19f41571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fa980700 1 -- 172.21.15.40:0/1930890988 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19f4151470 msgr2=0x7f19f40c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:10:14.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fa980700 1 --2- 172.21.15.40:0/1930890988 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19f4151470 0x7f19f40c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fa980700 1 -- 172.21.15.40:0/1930890988 >> v2:172.21.15.43:3300/0 conn(0x7f19f40b1bd0 msgr2=0x7f19f41571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:10:14.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fa980700 1 --2- 172.21.15.40:0/1930890988 >> v2:172.21.15.43:3300/0 conn(0x7f19f40b1bd0 0x7f19f41571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fa980700 1 -- 172.21.15.40:0/1930890988 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f4157fd0 con 0x7f19f4157710 2024-09-13T18:10:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fa980700 1 --2- 172.21.15.40:0/1930890988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19f4157710 0x7f19f4071bf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f19f4066f50 tx=0x7f19ec009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a586f40bea3223 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19f917d700 1 -- 172.21.15.40:0/1930890988 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec011830 con 0x7f19f4157710 2024-09-13T18:10:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19f917d700 1 -- 172.21.15.40:0/1930890988 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ec011990 con 0x7f19f4157710 2024-09-13T18:10:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19f917d700 1 -- 172.21.15.40:0/1930890988 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec011d20 con 0x7f19f4157710 2024-09-13T18:10:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fb181700 1 -- 172.21.15.40:0/1930890988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19f4157710 msgr2=0x7f19f4071bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fb181700 1 --2- 172.21.15.40:0/1930890988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19f4157710 0x7f19f4071bf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f19f4066f50 tx=0x7f19ec009bf0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fb181700 1 -- 172.21.15.40:0/1930890988 shutdown_connections 2024-09-13T18:10:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fb181700 1 --2- 172.21.15.40:0/1930890988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19f4157710 0x7f19f4071bf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fb181700 1 --2- 172.21.15.40:0/1930890988 >> v2:172.21.15.43:3300/0 conn(0x7f19f40b1bd0 0x7f19f41571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fb181700 1 --2- 172.21.15.40:0/1930890988 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19f4151470 0x7f19f40c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fb181700 1 -- 172.21.15.40:0/1930890988 >> 172.21.15.40:0/1930890988 conn(0x7f19f406e4f0 msgr2=0x7f19f406e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fb181700 1 -- 172.21.15.40:0/1930890988 shutdown_connections 2024-09-13T18:10:14.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.993+0000 7f19fb181700 1 -- 172.21.15.40:0/1930890988 wait complete. 2024-09-13T18:10:14.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19fb181700 1 Processor -- start 2024-09-13T18:10:14.003 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19fb181700 1 -- start start 2024-09-13T18:10:14.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19fb181700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19f40b1bd0 0x7f19f406db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19fb181700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19f4151470 0x7f19f41409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19fb181700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19f4157710 0x7f19f41400e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19fb181700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f4140620 con 0x7f19f4151470 2024-09-13T18:10:14.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19fb181700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f406b5c0 con 0x7f19f4157710 2024-09-13T18:10:14.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19fb181700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f406b320 con 0x7f19f40b1bd0 2024-09-13T18:10:14.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19f997e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19f4151470 0x7f19f41409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19f997e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19f4151470 0x7f19f41409a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40834/0 (socket says 172.21.15.40:40834) 2024-09-13T18:10:14.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19f997e700 1 -- 172.21.15.40:0/1717058125 learned_addr learned my addr 172.21.15.40:0/1717058125 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:10:14.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19fa980700 1 --2- 172.21.15.40:0/1717058125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19f4157710 0x7f19f41400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19fa17f700 1 --2- 172.21.15.40:0/1717058125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19f40b1bd0 0x7f19f406db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19f997e700 1 -- 172.21.15.40:0/1717058125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19f40b1bd0 msgr2=0x7f19f406db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19f997e700 1 --2- 172.21.15.40:0/1717058125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19f40b1bd0 0x7f19f406db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19f997e700 1 -- 172.21.15.40:0/1717058125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19f4157710 msgr2=0x7f19f41400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19f997e700 1 --2- 172.21.15.40:0/1717058125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19f4157710 0x7f19f41400e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19f997e700 1 -- 172.21.15.40:0/1717058125 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec009570 con 0x7f19f4151470 2024-09-13T18:10:14.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19fa17f700 1 --2- 172.21.15.40:0/1717058125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19f40b1bd0 0x7f19f406db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:10:14.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19f997e700 1 --2- 172.21.15.40:0/1717058125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19f4151470 0x7f19f41409a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f19e400e7c0 tx=0x7f19e400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:14.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19eaffd700 1 -- 172.21.15.40:0/1717058125 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4013680 con 0x7f19f4151470 2024-09-13T18:10:14.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19eaffd700 1 -- 172.21.15.40:0/1717058125 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e40137e0 con 0x7f19f4151470 2024-09-13T18:10:14.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19fb181700 1 -- 172.21.15.40:0/1717058125 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f406ce20 con 0x7f19f4151470 2024-09-13T18:10:14.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19eaffd700 1 -- 172.21.15.40:0/1717058125 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4013bb0 con 0x7f19f4151470 2024-09-13T18:10:14.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:13.997+0000 7f19fb181700 1 -- 172.21.15.40:0/1717058125 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f406d620 con 0x7f19f4151470 2024-09-13T18:10:14.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.001+0000 7f19fb181700 1 -- 172.21.15.40:0/1717058125 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f41571d0 con 0x7f19f4151470 2024-09-13T18:10:14.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.001+0000 7f19eaffd700 1 -- 172.21.15.40:0/1717058125 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f19e4013d10 con 0x7f19f4151470 2024-09-13T18:10:14.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.001+0000 7f19eaffd700 1 --2- 172.21.15.40:0/1717058125 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f19d8066760 0x7f19d8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.001+0000 7f19eaffd700 1 -- 172.21.15.40:0/1717058125 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f19e4083f40 con 0x7f19f4151470 2024-09-13T18:10:14.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.001+0000 7f19fa17f700 1 --2- 172.21.15.40:0/1717058125 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f19d8066760 0x7f19d8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.001+0000 7f19fa17f700 1 --2- 172.21.15.40:0/1717058125 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f19d8066760 0x7f19d8068c20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f19f0007440 tx=0x7f19f000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:14.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.009+0000 7f19eaffd700 1 -- 172.21.15.40:0/1717058125 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f19e4050e70 con 0x7f19f4151470 2024-09-13T18:10:14.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.285+0000 7f19fb181700 1 -- 172.21.15.40:0/1717058125 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f19f406f5a0 con 0x7f19f4151470 2024-09-13T18:10:14.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.285+0000 7f19eaffd700 1 -- 172.21.15.40:0/1717058125 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f19e4017620 con 0x7f19f4151470 2024-09-13T18:10:14.294 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":158,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:10:14.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.289+0000 7f19fb181700 1 -- 172.21.15.40:0/1717058125 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f19d8066760 msgr2=0x7f19d8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.289+0000 7f19fb181700 1 --2- 172.21.15.40:0/1717058125 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f19d8066760 0x7f19d8068c20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f19f0007440 tx=0x7f19f000b040 comp rx=0 tx=0).stop 2024-09-13T18:10:14.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.289+0000 7f19fb181700 1 -- 172.21.15.40:0/1717058125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19f4151470 msgr2=0x7f19f41409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.289+0000 7f19fb181700 1 --2- 172.21.15.40:0/1717058125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19f4151470 0x7f19f41409a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f19e400e7c0 tx=0x7f19e400eb80 comp rx=0 tx=0).stop 2024-09-13T18:10:14.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.289+0000 7f19fb181700 1 -- 172.21.15.40:0/1717058125 shutdown_connections 2024-09-13T18:10:14.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.289+0000 7f19fb181700 1 --2- 172.21.15.40:0/1717058125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19f4157710 0x7f19f41400e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.289+0000 7f19fb181700 1 --2- 172.21.15.40:0/1717058125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19f4151470 0x7f19f41409a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.289+0000 7f19fb181700 1 --2- 172.21.15.40:0/1717058125 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f19d8066760 0x7f19d8068c20 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.289+0000 7f19fb181700 1 --2- 172.21.15.40:0/1717058125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f19f40b1bd0 0x7f19f406db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.289+0000 7f19fb181700 1 -- 172.21.15.40:0/1717058125 >> 172.21.15.40:0/1717058125 conn(0x7f19f406e4f0 msgr2=0x7f19f40c3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:14.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.289+0000 7f19fb181700 1 -- 172.21.15.40:0/1717058125 shutdown_connections 2024-09-13T18:10:14.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.289+0000 7f19fb181700 1 -- 172.21.15.40:0/1717058125 wait complete. 2024-09-13T18:10:14.309 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:10:14.309 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T18:10:14.309 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:10:14.309 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:10:14.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.413+0000 7fc6f49db700 1 Processor -- start 2024-09-13T18:10:14.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.413+0000 7fc6f49db700 1 -- start start 2024-09-13T18:10:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6f49db700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc6f00c9980 0x7fc6f00c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6f49db700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f015a440 con 0x7fc6f00c8c10 2024-09-13T18:10:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6f49db700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f006f760 con 0x7fc6f0070300 2024-09-13T18:10:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6f49db700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f00ca2a0 con 0x7fc6f00c9980 2024-09-13T18:10:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6eeffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc6f00c8c10 0x7fc6f006fbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55348/0 (socket says 172.21.15.40:55348) 2024-09-13T18:10:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6eeffd700 1 -- 172.21.15.40:0/2744754388 learned_addr learned my addr 172.21.15.40:0/2744754388 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:10:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ef7fe700 1 --2- 172.21.15.40:0/2744754388 >> v2:172.21.15.43:3300/0 conn(0x7fc6f00c9980 0x7fc6f00c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ee7fc700 1 -- 172.21.15.40:0/2744754388 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028281319 0 0) 0x7fc6f015a440 con 0x7fc6f00c8c10 2024-09-13T18:10:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ee7fc700 1 -- 172.21.15.40:0/2744754388 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6d80035d0 con 0x7fc6f00c8c10 2024-09-13T18:10:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ee7fc700 1 -- 172.21.15.40:0/2744754388 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529414310 0 0) 0x7fc6f006f760 con 0x7fc6f0070300 2024-09-13T18:10:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ee7fc700 1 -- 172.21.15.40:0/2744754388 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6f015a440 con 0x7fc6f0070300 2024-09-13T18:10:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ee7fc700 1 -- 172.21.15.40:0/2744754388 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 800265972 0 0) 0x7fc6d80035d0 con 0x7fc6f00c8c10 2024-09-13T18:10:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ee7fc700 1 -- 172.21.15.40:0/2744754388 >> v2:172.21.15.43:3300/0 conn(0x7fc6f00c9980 msgr2=0x7fc6f00c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:10:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ee7fc700 1 --2- 172.21.15.40:0/2744754388 >> v2:172.21.15.43:3300/0 conn(0x7fc6f00c9980 0x7fc6f00c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ee7fc700 1 -- 172.21.15.40:0/2744754388 >> v1:172.21.15.43:6789/0 conn(0x7fc6f0070300 legacy=0x7fc6f0163db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ee7fc700 1 -- 172.21.15.40:0/2744754388 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f01598c0 con 0x7fc6f00c8c10 2024-09-13T18:10:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ee7fc700 1 -- 172.21.15.40:0/2744754388 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fc6e4001670 con 0x7fc6f00c8c10 2024-09-13T18:10:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ee7fc700 1 -- 172.21.15.40:0/2744754388 >> v1:172.21.15.40:6789/0 conn(0x7fc6f00c8c10 legacy=0x7fc6f006fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ee7fc700 1 --2- 172.21.15.40:0/2744754388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6d8003bb0 0x7fc6d8003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ee7fc700 1 --2- 172.21.15.40:0/2744754388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6d8004870 0x7fc6d8004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ee7fc700 1 --2- 172.21.15.40:0/2744754388 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6d8005230 0x7fc6d800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ee7fc700 1 -- 172.21.15.40:0/2744754388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f00ca2a0 con 0x7fc6d8003bb0 2024-09-13T18:10:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ee7fc700 1 -- 172.21.15.40:0/2744754388 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d800c0b0 con 0x7fc6d8004870 2024-09-13T18:10:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ee7fc700 1 -- 172.21.15.40:0/2744754388 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d800c270 con 0x7fc6d8005230 2024-09-13T18:10:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ef7fe700 1 --2- 172.21.15.40:0/2744754388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6d8003bb0 0x7fc6d8003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6eeffd700 1 --2- 172.21.15.40:0/2744754388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6d8004870 0x7fc6d8004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6effff700 1 --2- 172.21.15.40:0/2744754388 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6d8005230 0x7fc6d800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ef7fe700 1 -- 172.21.15.40:0/2744754388 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6d8005230 msgr2=0x7fc6d800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:10:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ef7fe700 1 --2- 172.21.15.40:0/2744754388 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6d8005230 0x7fc6d800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ef7fe700 1 -- 172.21.15.40:0/2744754388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6d8004870 msgr2=0x7fc6d8004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:10:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ef7fe700 1 --2- 172.21.15.40:0/2744754388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6d8004870 0x7fc6d8004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ef7fe700 1 -- 172.21.15.40:0/2744754388 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d80035d0 con 0x7fc6d8003bb0 2024-09-13T18:10:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6effff700 1 -- 172.21.15.40:0/2744754388 reap_dead start 2024-09-13T18:10:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ef7fe700 1 --2- 172.21.15.40:0/2744754388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6d8003bb0 0x7fc6d8003fb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009b70 tx=0x7fc6d80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9039498d05355 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ee7fc700 1 -- 172.21.15.40:0/2744754388 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e000e6c0 con 0x7fc6d8003bb0 2024-09-13T18:10:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ee7fc700 1 -- 172.21.15.40:0/2744754388 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e000e820 con 0x7fc6d8003bb0 2024-09-13T18:10:14.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.417+0000 7fc6ee7fc700 1 -- 172.21.15.40:0/2744754388 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e000ebb0 con 0x7fc6d8003bb0 2024-09-13T18:10:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.421+0000 7fc6f49db700 1 -- 172.21.15.40:0/2744754388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6d8003bb0 msgr2=0x7fc6d8003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.421+0000 7fc6f49db700 1 --2- 172.21.15.40:0/2744754388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6d8003bb0 0x7fc6d8003fb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009b70 tx=0x7fc6d80058a0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.421+0000 7fc6f49db700 1 -- 172.21.15.40:0/2744754388 shutdown_connections 2024-09-13T18:10:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.421+0000 7fc6f49db700 1 --2- 172.21.15.40:0/2744754388 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6d8003bb0 0x7fc6d8003fb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.421+0000 7fc6f49db700 1 -- 172.21.15.40:0/2744754388 >> 172.21.15.40:0/2744754388 conn(0x7fc6f0066090 msgr2=0x7fc6f0066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.421+0000 7fc6f49db700 1 -- 172.21.15.40:0/2744754388 shutdown_connections 2024-09-13T18:10:14.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.421+0000 7fc6f49db700 1 -- 172.21.15.40:0/2744754388 wait complete. 2024-09-13T18:10:14.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.421+0000 7fc6f49db700 1 Processor -- start 2024-09-13T18:10:14.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.421+0000 7fc6f49db700 1 -- start start 2024-09-13T18:10:14.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6f49db700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6d8003bb0 0x7fc6f006b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6f49db700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f00c5de0 0x7fc6f0158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6f49db700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f01ef500 0x7fc6f01ef980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6f49db700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f01efec0 con 0x7fc6f01ef500 2024-09-13T18:10:14.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6f49db700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f01f2a70 con 0x7fc6f00c5de0 2024-09-13T18:10:14.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6f49db700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f00c5880 con 0x7fc6d8003bb0 2024-09-13T18:10:14.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6effff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f01ef500 0x7fc6f01ef980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6effff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f01ef500 0x7fc6f01ef980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40858/0 (socket says 172.21.15.40:40858) 2024-09-13T18:10:14.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6effff700 1 -- 172.21.15.40:0/3583183743 learned_addr learned my addr 172.21.15.40:0/3583183743 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:10:14.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6eeffd700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f00c5de0 0x7fc6f0158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6ef7fe700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6d8003bb0 0x7fc6f006b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6effff700 1 -- 172.21.15.40:0/3583183743 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6d8003bb0 msgr2=0x7fc6f006b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6effff700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6d8003bb0 0x7fc6f006b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6effff700 1 -- 172.21.15.40:0/3583183743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f00c5de0 msgr2=0x7fc6f0158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6effff700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f00c5de0 0x7fc6f0158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6effff700 1 -- 172.21.15.40:0/3583183743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f01f2cc0 con 0x7fc6f01ef500 2024-09-13T18:10:14.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6ef7fe700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6d8003bb0 0x7fc6f006b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:10:14.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6eeffd700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f00c5de0 0x7fc6f0158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:10:14.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6effff700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f01ef500 0x7fc6f01ef980 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00cada0 tx=0x7fc6e800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:14.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6d7fff700 1 -- 172.21.15.40:0/3583183743 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8003cf0 con 0x7fc6f01ef500 2024-09-13T18:10:14.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6d7fff700 1 -- 172.21.15.40:0/3583183743 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e8003e50 con 0x7fc6f01ef500 2024-09-13T18:10:14.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6f49db700 1 -- 172.21.15.40:0/3583183743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f01f2fa0 con 0x7fc6f01ef500 2024-09-13T18:10:14.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6d7fff700 1 -- 172.21.15.40:0/3583183743 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e800dbd0 con 0x7fc6f01ef500 2024-09-13T18:10:14.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6f49db700 1 -- 172.21.15.40:0/3583183743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f01f3560 con 0x7fc6f01ef500 2024-09-13T18:10:14.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6f49db700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f01495f0 0x7fc6f01499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6f49db700 1 -- 172.21.15.40:0/3583183743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f0074830 con 0x7fc6f01495f0 2024-09-13T18:10:14.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.425+0000 7fc6ef7fe700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f01495f0 0x7fc6f01499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.429+0000 7fc6d7fff700 1 -- 172.21.15.40:0/3583183743 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc6e800dd30 con 0x7fc6f01ef500 2024-09-13T18:10:14.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.429+0000 7fc6ef7fe700 1 -- 172.21.15.40:0/3583183743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6f006f6d0 con 0x7fc6f01495f0 2024-09-13T18:10:14.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.429+0000 7fc6ef7fe700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f01495f0 0x7fc6f01499d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009b70 tx=0x7fc6e00047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:14.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.429+0000 7fc6d7fff700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc6dc066730 0x7fc6dc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.429+0000 7fc6eeffd700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc6dc066730 0x7fc6dc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.429+0000 7fc6d7fff700 1 -- 172.21.15.40:0/3583183743 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc6e80847c0 con 0x7fc6f01ef500 2024-09-13T18:10:14.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.429+0000 7fc6d7fff700 1 -- 172.21.15.40:0/3583183743 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0021070 con 0x7fc6f01495f0 2024-09-13T18:10:14.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.429+0000 7fc6d7fff700 1 -- 172.21.15.40:0/3583183743 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc6e00036c0 con 0x7fc6f01495f0 2024-09-13T18:10:14.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.429+0000 7fc6d7fff700 1 -- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f01495f0 msgr2=0x7fc6f01499d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.429+0000 7fc6d7fff700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f01495f0 0x7fc6f01499d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009b70 tx=0x7fc6e00047e0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.429+0000 7fc6d7fff700 1 -- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f01495f0 msgr2=0x7fc6f01499d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:10:14.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.429+0000 7fc6d7fff700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f01495f0 0x7fc6f01499d0 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009b70 tx=0x7fc6e00047e0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.429+0000 7fc6eeffd700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc6dc066730 0x7fc6dc068bf0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4000fd0 tx=0x7fc6e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:14.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6f49db700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f00cbe20 0x7fc6f00cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6f49db700 1 -- 172.21.15.40:0/3583183743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f00c3fb0 con 0x7fc6f00cbe20 2024-09-13T18:10:14.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6ef7fe700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f00cbe20 0x7fc6f00cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6ef7fe700 1 -- 172.21.15.40:0/3583183743 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc6f015a160 con 0x7fc6f00cbe20 2024-09-13T18:10:14.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6ef7fe700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f00cbe20 0x7fc6f00cc230 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0000c00 tx=0x7fc6e0009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:14.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6d7fff700 1 -- 172.21.15.40:0/3583183743 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e000e6c0 con 0x7fc6f00cbe20 2024-09-13T18:10:14.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6d7fff700 1 -- 172.21.15.40:0/3583183743 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4453 (secure 0 0 0) 0x7fc6e002d300 con 0x7fc6f00cbe20 2024-09-13T18:10:14.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6d7fff700 1 -- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f00cbe20 msgr2=0x7fc6f00cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6d7fff700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f00cbe20 0x7fc6f00cc230 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0000c00 tx=0x7fc6e0009240 comp rx=0 tx=0).stop 2024-09-13T18:10:14.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6d7fff700 1 -- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f00cbe20 msgr2=0x7fc6f00cc230 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:10:14.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6d7fff700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f00cbe20 0x7fc6f00cc230 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.450 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:10:14.450 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:10:14.450 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:10:14.451 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:10:14.451 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 158, 2024-09-13T18:10:14.451 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:10:14.451 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:10:14.451 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:10:14.451 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:10:14.451 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:14.451 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 12, 2024-09-13T18:10:14.451 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:10:14.451 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:10:14.451 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:10:14.452 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:14.452 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:14.452 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:14.452 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:14.452 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:14.452 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:14.452 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:14.452 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:14.452 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:14.452 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:14.452 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:14.452 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:10:14.452 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:10:14.452 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:10:14.453 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:10:14.454 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:10:14.454 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:10:14.454 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:10:14.454 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:10:14.454 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:10:14.454 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:10:14.454 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:10:14.454 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:10:14.454 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:10:14.454 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:10:14.454 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:10:14.454 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:10:14.454 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:10:14.454 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:10:14.455 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:10:14.455 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:14.455 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:14.455 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:14.455 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:14.455 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:14.455 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:14.455 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:14.455 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:14.455 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:14.455 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:14.455 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:14.455 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:10:14.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.456 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:10:14.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.456 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:10:14.456 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:10:14.456 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:14.456 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:14.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.456 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:14.456 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:10:14.456 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:14.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.456 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:14.457 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:10:14.457 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:14.457 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.457 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:14.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.457 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:10:14.457 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:10:14.457 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:14.457 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:14.457 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:14.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.458 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:10:14.458 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:10:14.458 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:14.458 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:14.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.458 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:14.458 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:10:14.458 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:14.458 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.458 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:14.458 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:10:14.458 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:14.459 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.459 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:14.459 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.459 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:10:14.459 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:10:14.459 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:14.459 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:14.459 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:14.459 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.459 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.459 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:10:14.459 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:10:14.459 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:14.459 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:14.460 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.460 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:14.460 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:10:14.460 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:14.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.460 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.460 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:14.460 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:10:14.460 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:14.460 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.460 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:14.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.460 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:10:14.461 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:10:14.461 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:14.461 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:14.461 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:14.461 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.461 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:14.461 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.461 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:10:14.461 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:10:14.461 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.461 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:14.461 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:14.461 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:10:14.461 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.462 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:14.462 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:10:14.462 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.462 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:14.462 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:14.462 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:10:14.462 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.462 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:14.462 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.462 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:10:14.462 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:10:14.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6f49db700 1 -- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc6dc066730 msgr2=0x7fc6dc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6f49db700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc6dc066730 0x7fc6dc068bf0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4000fd0 tx=0x7fc6e4009040 comp rx=0 tx=0).stop 2024-09-13T18:10:14.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6f49db700 1 -- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f01ef500 msgr2=0x7fc6f01ef980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6f49db700 1 --2- 172.21.15.40:0/3583183743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6f01ef500 0x7fc6f01ef980 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00cada0 tx=0x7fc6e800d430 comp rx=0 tx=0).stop 2024-09-13T18:10:14.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6effff700 1 -- 172.21.15.40:0/3583183743 reap_dead start 2024-09-13T18:10:14.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6effff700 1 -- 172.21.15.40:0/3583183743 reap_dead start 2024-09-13T18:10:14.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6f49db700 1 -- 172.21.15.40:0/3583183743 shutdown_connections 2024-09-13T18:10:14.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6f49db700 1 -- 172.21.15.40:0/3583183743 >> 172.21.15.40:0/3583183743 conn(0x7fc6f0066090 msgr2=0x7fc6f00c2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:14.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6f49db700 1 -- 172.21.15.40:0/3583183743 shutdown_connections 2024-09-13T18:10:14.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.441+0000 7fc6f49db700 1 -- 172.21.15.40:0/3583183743 wait complete. 2024-09-13T18:10:14.466 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:10:14.466 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:10:14.466 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:10:14.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093b59e700 1 Processor -- start 2024-09-13T18:10:14.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093b59e700 1 -- start start 2024-09-13T18:10:14.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093b59e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f093c0cf1a0 0x7f093c15a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093b59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f093c0cf580 0x7f093c0c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093b59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f093c0cc250 0x7f093c0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093b59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093c15a7a0 con 0x7f093c0cf580 2024-09-13T18:10:14.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093b59e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093c0cbdc0 con 0x7f093c0cf1a0 2024-09-13T18:10:14.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093b59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093c153f80 con 0x7f093c0cc250 2024-09-13T18:10:14.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f0939d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f093c0cf580 0x7f093c0c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f0939d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f093c0cf580 0x7f093c0c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40882/0 (socket says 172.21.15.40:40882) 2024-09-13T18:10:14.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f0939d9b700 1 -- 172.21.15.40:0/457935751 learned_addr learned my addr 172.21.15.40:0/457935751 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:10:14.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093ad9d700 1 --2- 172.21.15.40:0/457935751 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f093c0cc250 0x7f093c0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093a59c700 1 --2- 172.21.15.40:0/457935751 >> v2:172.21.15.43:3300/0 conn(0x7f093c0cf1a0 0x7f093c15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f0939d9b700 1 -- 172.21.15.40:0/457935751 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f093c0cc250 msgr2=0x7f093c0c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:10:14.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f0939d9b700 1 --2- 172.21.15.40:0/457935751 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f093c0cc250 0x7f093c0c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f0939d9b700 1 -- 172.21.15.40:0/457935751 >> v2:172.21.15.43:3300/0 conn(0x7f093c0cf1a0 msgr2=0x7f093c15a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:10:14.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f0939d9b700 1 --2- 172.21.15.40:0/457935751 >> v2:172.21.15.43:3300/0 conn(0x7f093c0cf1a0 0x7f093c15a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f0939d9b700 1 -- 172.21.15.40:0/457935751 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c0caed0 con 0x7f093c0cf580 2024-09-13T18:10:14.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f0939d9b700 1 --2- 172.21.15.40:0/457935751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f093c0cf580 0x7f093c0c9fd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f093c067020 tx=0x7f0930009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcae23a0e6952264 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:14.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093959a700 1 -- 172.21.15.40:0/457935751 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093000b8a0 con 0x7f093c0cf580 2024-09-13T18:10:14.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093959a700 1 -- 172.21.15.40:0/457935751 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093000ba00 con 0x7f093c0cf580 2024-09-13T18:10:14.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093959a700 1 -- 172.21.15.40:0/457935751 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093000bd90 con 0x7f093c0cf580 2024-09-13T18:10:14.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093b59e700 1 -- 172.21.15.40:0/457935751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f093c0cf580 msgr2=0x7f093c0c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093b59e700 1 --2- 172.21.15.40:0/457935751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f093c0cf580 0x7f093c0c9fd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f093c067020 tx=0x7f0930009bf0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093b59e700 1 -- 172.21.15.40:0/457935751 shutdown_connections 2024-09-13T18:10:14.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093b59e700 1 --2- 172.21.15.40:0/457935751 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f093c0cc250 0x7f093c0c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093b59e700 1 --2- 172.21.15.40:0/457935751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f093c0cf580 0x7f093c0c9fd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093b59e700 1 --2- 172.21.15.40:0/457935751 >> v2:172.21.15.43:3300/0 conn(0x7f093c0cf1a0 0x7f093c15a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093b59e700 1 -- 172.21.15.40:0/457935751 >> 172.21.15.40:0/457935751 conn(0x7f093c066090 msgr2=0x7f093c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:14.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093b59e700 1 -- 172.21.15.40:0/457935751 shutdown_connections 2024-09-13T18:10:14.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.577+0000 7f093b59e700 1 -- 172.21.15.40:0/457935751 wait complete. 2024-09-13T18:10:14.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f093b59e700 1 Processor -- start 2024-09-13T18:10:14.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f093b59e700 1 -- start start 2024-09-13T18:10:14.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f093b59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f093c0cc250 0x7f093c0b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f093b59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f093c0cf1a0 0x7f093c0c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f093b59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f093c0cf580 0x7f093c0c2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f093b59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c154ea0 con 0x7f093c0cc250 2024-09-13T18:10:14.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f093b59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0c7d70 con 0x7f093c0cf1a0 2024-09-13T18:10:14.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f093b59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0c8030 con 0x7f093c0cf580 2024-09-13T18:10:14.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f093a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f093c0cc250 0x7f093c0b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f093a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f093c0cc250 0x7f093c0b3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40892/0 (socket says 172.21.15.40:40892) 2024-09-13T18:10:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f093a59c700 1 -- 172.21.15.40:0/312131549 learned_addr learned my addr 172.21.15.40:0/312131549 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:10:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f0939d9b700 1 --2- 172.21.15.40:0/312131549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f093c0cf1a0 0x7f093c0c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f093ad9d700 1 --2- 172.21.15.40:0/312131549 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f093c0cf580 0x7f093c0c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f093a59c700 1 -- 172.21.15.40:0/312131549 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f093c0cf580 msgr2=0x7f093c0c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f093a59c700 1 --2- 172.21.15.40:0/312131549 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f093c0cf580 0x7f093c0c2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f093a59c700 1 -- 172.21.15.40:0/312131549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f093c0cf1a0 msgr2=0x7f093c0c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f093a59c700 1 --2- 172.21.15.40:0/312131549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f093c0cf1a0 0x7f093c0c51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f093a59c700 1 -- 172.21.15.40:0/312131549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0930009570 con 0x7f093c0cc250 2024-09-13T18:10:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f093a59c700 1 --2- 172.21.15.40:0/312131549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f093c0cc250 0x7f093c0b3540 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f092c00e760 tx=0x7f092c00ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:14.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f092affd700 1 -- 172.21.15.40:0/312131549 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c00c070 con 0x7f093c0cc250 2024-09-13T18:10:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f092affd700 1 -- 172.21.15.40:0/312131549 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092c013610 con 0x7f093c0cc250 2024-09-13T18:10:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f092affd700 1 -- 172.21.15.40:0/312131549 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c013980 con 0x7f093c0cc250 2024-09-13T18:10:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f093b59e700 1 -- 172.21.15.40:0/312131549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093c154150 con 0x7f093c0cc250 2024-09-13T18:10:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.581+0000 7f093b59e700 1 -- 172.21.15.40:0/312131549 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093c0c41a0 con 0x7f093c0cc250 2024-09-13T18:10:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.585+0000 7f0928ff9700 1 --2- 172.21.15.40:0/312131549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0914002a40 0x7f0914004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.585+0000 7f0928ff9700 1 -- 172.21.15.40:0/312131549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0914007f40 con 0x7f0914002a40 2024-09-13T18:10:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.585+0000 7f0939d9b700 1 --2- 172.21.15.40:0/312131549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0914002a40 0x7f0914004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.585+0000 7f092affd700 1 -- 172.21.15.40:0/312131549 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f092c013ae0 con 0x7f093c0cc250 2024-09-13T18:10:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.585+0000 7f0939d9b700 1 -- 172.21.15.40:0/312131549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0914008100 con 0x7f0914002a40 2024-09-13T18:10:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.585+0000 7f092affd700 1 --2- 172.21.15.40:0/312131549 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f091c066760 0x7f091c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.585+0000 7f092affd700 1 -- 172.21.15.40:0/312131549 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f092c084330 con 0x7f093c0cc250 2024-09-13T18:10:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.585+0000 7f093ad9d700 1 --2- 172.21.15.40:0/312131549 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f091c066760 0x7f091c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.585+0000 7f0939d9b700 1 --2- 172.21.15.40:0/312131549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0914002a40 0x7f0914004f00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f093c067020 tx=0x7f0930004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:14.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.585+0000 7f092affd700 1 -- 172.21.15.40:0/312131549 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930020070 con 0x7f0914002a40 2024-09-13T18:10:14.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.585+0000 7f093ad9d700 1 --2- 172.21.15.40:0/312131549 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f091c066760 0x7f091c068c20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0934009a10 tx=0x7f09340096f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:14.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.585+0000 7f092affd700 1 -- 172.21.15.40:0/312131549 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f093001b9a0 con 0x7f0914002a40 2024-09-13T18:10:14.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.585+0000 7f092affd700 1 -- 172.21.15.40:0/312131549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0914002a40 msgr2=0x7f0914004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.585+0000 7f092affd700 1 --2- 172.21.15.40:0/312131549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0914002a40 0x7f0914004f00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f093c067020 tx=0x7f0930004770 comp rx=0 tx=0).stop 2024-09-13T18:10:14.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.585+0000 7f092affd700 1 -- 172.21.15.40:0/312131549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0914002a40 msgr2=0x7f0914004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:10:14.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.585+0000 7f092affd700 1 --2- 172.21.15.40:0/312131549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0914002a40 0x7f0914004f00 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f093c067020 tx=0x7f0930004770 comp rx=0 tx=0).stop 2024-09-13T18:10:14.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.597+0000 7f0928ff9700 1 --2- 172.21.15.40:0/312131549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0914006f90 0x7f0914007420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.597+0000 7f0928ff9700 1 -- 172.21.15.40:0/312131549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091400a400 con 0x7f0914006f90 2024-09-13T18:10:14.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.597+0000 7f0939d9b700 1 --2- 172.21.15.40:0/312131549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0914006f90 0x7f0914007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.597+0000 7f0939d9b700 1 -- 172.21.15.40:0/312131549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f091400a5c0 con 0x7f0914006f90 2024-09-13T18:10:14.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.597+0000 7f0939d9b700 1 --2- 172.21.15.40:0/312131549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0914006f90 0x7f0914007420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0930000c00 tx=0x7f0930011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:14.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.597+0000 7f092affd700 1 -- 172.21.15.40:0/312131549 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930035440 con 0x7f0914006f90 2024-09-13T18:10:14.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.601+0000 7f092affd700 1 -- 172.21.15.40:0/312131549 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4949 (secure 0 0 0) 0x7f09300355a0 con 0x7f0914006f90 2024-09-13T18:10:14.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.601+0000 7f092affd700 1 -- 172.21.15.40:0/312131549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0914006f90 msgr2=0x7f0914007420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.601+0000 7f092affd700 1 --2- 172.21.15.40:0/312131549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0914006f90 0x7f0914007420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0930000c00 tx=0x7f0930011000 comp rx=0 tx=0).stop 2024-09-13T18:10:14.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.601+0000 7f092affd700 1 -- 172.21.15.40:0/312131549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0914006f90 msgr2=0x7f0914007420 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:10:14.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.601+0000 7f092affd700 1 --2- 172.21.15.40:0/312131549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0914006f90 0x7f0914007420 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.608 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:10:14.608 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:10:14.608 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:10:14.608 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:10:14.608 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 158, 2024-09-13T18:10:14.608 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:10:14.608 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:10:14.608 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:10:14.609 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:10:14.609 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:14.609 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 12, 2024-09-13T18:10:14.609 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:10:14.609 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:10:14.609 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:10:14.609 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:14.610 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:14.610 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:14.610 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:14.610 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:14.610 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:14.610 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:14.611 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:14.611 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:14.611 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:14.611 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:14.611 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:10:14.611 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:10:14.612 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:14.612 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:14.612 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:14.612 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:14.612 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:14.612 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:14.612 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:14.613 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:14.613 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:14.613 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:14.613 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:14.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.613 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:10:14.614 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:10:14.614 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:10:14.614 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:10:14.614 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:10:14.614 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:10:14.614 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:10:14.614 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:10:14.614 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:10:14.615 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:10:14.615 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:10:14.615 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:10:14.615 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:10:14.615 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:10:14.615 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:10:14.616 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:10:14.616 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:10:14.616 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:14.616 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:14.616 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:14.616 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:14.617 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:14.617 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:14.617 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:14.617 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:14.617 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:14.617 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:14.618 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:14.618 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:10:14.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.618 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:10:14.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.618 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:10:14.618 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:10:14.618 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:14.618 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:14.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.619 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:14.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:10:14.619 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:14.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.619 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:14.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:10:14.619 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:14.619 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.619 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:14.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:10:14.619 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:10:14.620 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:14.620 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:14.620 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:14.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.620 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:10:14.620 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:10:14.620 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:14.620 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:14.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.620 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:14.620 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:10:14.620 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:14.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.621 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:14.621 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:10:14.621 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:14.621 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.621 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:14.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.621 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:10:14.621 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:10:14.621 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:14.621 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:14.621 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:14.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.622 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:10:14.622 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:10:14.622 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:14.622 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:14.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.622 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:14.622 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:10:14.622 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:14.622 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.622 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:14.622 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:10:14.623 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:14.623 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.623 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:14.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.623 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:10:14.623 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:10:14.623 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:14.623 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:14.623 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:14.623 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.623 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:14.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.623 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:10:14.623 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:10:14.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.624 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:14.624 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:14.624 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:10:14.624 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.624 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:14.624 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:10:14.624 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.624 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:14.624 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:14.625 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:10:14.625 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.625 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:14.625 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:10:14.625 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.625 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:14.625 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:14.625 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 8 2024-09-13T18:10:14.626 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.626 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:14.626 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:10:14.626 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.626 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:14.626 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:14.626 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:10:14.626 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.627 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:14.627 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:10:14.627 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.627 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:14.627 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:14.627 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:10:14.627 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.627 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:14.627 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.628 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:10:14.628 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:10:14.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.601+0000 7f0928ff9700 1 -- 172.21.15.40:0/312131549 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f091c066760 msgr2=0x7f091c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.601+0000 7f0928ff9700 1 --2- 172.21.15.40:0/312131549 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f091c066760 0x7f091c068c20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0934009a10 tx=0x7f09340096f0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.601+0000 7f0928ff9700 1 -- 172.21.15.40:0/312131549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f093c0cc250 msgr2=0x7f093c0b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.601+0000 7f0928ff9700 1 --2- 172.21.15.40:0/312131549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f093c0cc250 0x7f093c0b3540 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f092c00e760 tx=0x7f092c00ea70 comp rx=0 tx=0).stop 2024-09-13T18:10:14.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.601+0000 7f093ad9d700 1 -- 172.21.15.40:0/312131549 reap_dead start 2024-09-13T18:10:14.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.601+0000 7f093ad9d700 1 -- 172.21.15.40:0/312131549 reap_dead start 2024-09-13T18:10:14.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.601+0000 7f0928ff9700 1 -- 172.21.15.40:0/312131549 shutdown_connections 2024-09-13T18:10:14.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.601+0000 7f0928ff9700 1 -- 172.21.15.40:0/312131549 >> 172.21.15.40:0/312131549 conn(0x7f093c066090 msgr2=0x7f093c15bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:14.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.601+0000 7f0928ff9700 1 -- 172.21.15.40:0/312131549 shutdown_connections 2024-09-13T18:10:14.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.601+0000 7f0928ff9700 1 -- 172.21.15.40:0/312131549 wait complete. 2024-09-13T18:10:14.631 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:10:14.631 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:10:14.632 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:10:14.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a72b6700 1 Processor -- start 2024-09-13T18:10:14.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a72b6700 1 -- start start 2024-09-13T18:10:14.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a72b6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0071580 0x7ff3a00719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a72b6700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff3a0161f90 0x7ff3a0164380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a72b6700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a00c16f0 con 0x7ff3a0156e10 2024-09-13T18:10:14.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a72b6700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a0159700 con 0x7ff3a0161f90 2024-09-13T18:10:14.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a72b6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a00c3060 con 0x7ff3a0071580 2024-09-13T18:10:14.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a62b4700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff3a0156e10 0x7ff3a006c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55350/0 (socket says 172.21.15.40:55350) 2024-09-13T18:10:14.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a62b4700 1 -- 172.21.15.40:0/80820646 learned_addr learned my addr 172.21.15.40:0/80820646 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:10:14.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a5ab3700 1 --2- 172.21.15.40:0/80820646 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0071580 0x7ff3a00719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a6ab5700 1 --2- 172.21.15.40:0/80820646 >> v2:172.21.15.43:3300/0 conn(0x7ff3a0161f90 0x7ff3a0164380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a52b2700 1 -- 172.21.15.40:0/80820646 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354379836 0 0) 0x7ff3a00c16f0 con 0x7ff3a0156e10 2024-09-13T18:10:14.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a52b2700 1 -- 172.21.15.40:0/80820646 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff38c0035d0 con 0x7ff3a0156e10 2024-09-13T18:10:14.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a52b2700 1 -- 172.21.15.40:0/80820646 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 675458593 0 0) 0x7ff38c0035d0 con 0x7ff3a0156e10 2024-09-13T18:10:14.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a52b2700 1 -- 172.21.15.40:0/80820646 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0071580 msgr2=0x7ff3a00719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:10:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a52b2700 1 --2- 172.21.15.40:0/80820646 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0071580 0x7ff3a00719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a52b2700 1 -- 172.21.15.40:0/80820646 >> v2:172.21.15.43:3300/0 conn(0x7ff3a0161f90 msgr2=0x7ff3a0164380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:10:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a52b2700 1 --2- 172.21.15.40:0/80820646 >> v2:172.21.15.43:3300/0 conn(0x7ff3a0161f90 0x7ff3a0164380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a52b2700 1 -- 172.21.15.40:0/80820646 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a00c2d80 con 0x7ff3a0156e10 2024-09-13T18:10:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a52b2700 1 -- 172.21.15.40:0/80820646 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7ff39c001670 con 0x7ff3a0156e10 2024-09-13T18:10:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a52b2700 1 -- 172.21.15.40:0/80820646 >> v1:172.21.15.40:6789/0 conn(0x7ff3a0156e10 legacy=0x7ff3a006c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a52b2700 1 --2- 172.21.15.40:0/80820646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff38c003a20 0x7ff38c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a52b2700 1 --2- 172.21.15.40:0/80820646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff38c004870 0x7ff38c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a52b2700 1 --2- 172.21.15.40:0/80820646 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff38c005230 0x7ff38c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a52b2700 1 -- 172.21.15.40:0/80820646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c00bf70 con 0x7ff38c004870 2024-09-13T18:10:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a52b2700 1 -- 172.21.15.40:0/80820646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c00c0b0 con 0x7ff38c003a20 2024-09-13T18:10:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a52b2700 1 -- 172.21.15.40:0/80820646 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c00c220 con 0x7ff38c005230 2024-09-13T18:10:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a5ab3700 1 --2- 172.21.15.40:0/80820646 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0071580 0x7ff3a00719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:10:14.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a5ab3700 1 --2- 172.21.15.40:0/80820646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff38c004870 0x7ff38c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a62b4700 1 --2- 172.21.15.40:0/80820646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff38c003a20 0x7ff38c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a6ab5700 1 --2- 172.21.15.40:0/80820646 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff38c005230 0x7ff38c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a5ab3700 1 -- 172.21.15.40:0/80820646 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff38c005230 msgr2=0x7ff38c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:10:14.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a5ab3700 1 --2- 172.21.15.40:0/80820646 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff38c005230 0x7ff38c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a5ab3700 1 -- 172.21.15.40:0/80820646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff38c003a20 msgr2=0x7ff38c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:10:14.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a5ab3700 1 --2- 172.21.15.40:0/80820646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff38c003a20 0x7ff38c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a5ab3700 1 -- 172.21.15.40:0/80820646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c0035d0 con 0x7ff38c004870 2024-09-13T18:10:14.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.741+0000 7ff3a6ab5700 1 --2- 172.21.15.40:0/80820646 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff38c005230 0x7ff38c00ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:10:14.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.745+0000 7ff3a5ab3700 1 --2- 172.21.15.40:0/80820646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff38c004870 0x7ff38c004cf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00c64a0 tx=0x7ff38c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc02c94cc5d46be5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:14.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.745+0000 7ff3a52b2700 1 -- 172.21.15.40:0/80820646 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3900042f0 con 0x7ff38c004870 2024-09-13T18:10:14.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.745+0000 7ff3a52b2700 1 -- 172.21.15.40:0/80820646 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff390004450 con 0x7ff38c004870 2024-09-13T18:10:14.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.745+0000 7ff3a52b2700 1 -- 172.21.15.40:0/80820646 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3900038e0 con 0x7ff38c004870 2024-09-13T18:10:14.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.745+0000 7ff3a72b6700 1 -- 172.21.15.40:0/80820646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff38c004870 msgr2=0x7ff38c004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.745+0000 7ff3a72b6700 1 --2- 172.21.15.40:0/80820646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff38c004870 0x7ff38c004cf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00c64a0 tx=0x7ff38c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.745+0000 7ff3a6ab5700 1 -- 172.21.15.40:0/80820646 reap_dead start 2024-09-13T18:10:14.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.745+0000 7ff3a72b6700 1 -- 172.21.15.40:0/80820646 shutdown_connections 2024-09-13T18:10:14.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.745+0000 7ff3a72b6700 1 -- 172.21.15.40:0/80820646 >> 172.21.15.40:0/80820646 conn(0x7ff3a00cc7f0 msgr2=0x7ff3a00c88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:14.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.745+0000 7ff3a72b6700 1 -- 172.21.15.40:0/80820646 shutdown_connections 2024-09-13T18:10:14.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.745+0000 7ff3a72b6700 1 -- 172.21.15.40:0/80820646 wait complete. 2024-09-13T18:10:14.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.745+0000 7ff3a72b6700 1 Processor -- start 2024-09-13T18:10:14.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.745+0000 7ff3a72b6700 1 -- start start 2024-09-13T18:10:14.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a72b6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3a00c6740 0x7ff3a0149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a72b6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3a00c6b20 0x7ff3a01efce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a72b6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0143fc0 0x7ff3a0144470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a72b6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a01449b0 con 0x7ff3a00c6740 2024-09-13T18:10:14.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a72b6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a00ca4c0 con 0x7ff3a00c6b20 2024-09-13T18:10:14.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a72b6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a01ef520 con 0x7ff3a0143fc0 2024-09-13T18:10:14.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a62b4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3a00c6740 0x7ff3a0149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a62b4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3a00c6740 0x7ff3a0149d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40920/0 (socket says 172.21.15.40:40920) 2024-09-13T18:10:14.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a62b4700 1 -- 172.21.15.40:0/1465370290 learned_addr learned my addr 172.21.15.40:0/1465370290 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:10:14.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a6ab5700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0143fc0 0x7ff3a0144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a5ab3700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3a00c6b20 0x7ff3a01efce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a62b4700 1 -- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0143fc0 msgr2=0x7ff3a0144470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a62b4700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0143fc0 0x7ff3a0144470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a62b4700 1 -- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3a00c6b20 msgr2=0x7ff3a01efce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a62b4700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3a00c6b20 0x7ff3a01efce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a62b4700 1 -- 172.21.15.40:0/1465370290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a00cadb0 con 0x7ff3a00c6740 2024-09-13T18:10:14.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a5ab3700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3a00c6b20 0x7ff3a01efce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:10:14.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a62b4700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3a00c6740 0x7ff3a0149d50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0048f0 tx=0x7ff39c007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:14.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff396ffd700 1 -- 172.21.15.40:0/1465370290 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c00f690 con 0x7ff3a00c6740 2024-09-13T18:10:14.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff396ffd700 1 -- 172.21.15.40:0/1465370290 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39c00f810 con 0x7ff3a00c6740 2024-09-13T18:10:14.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a72b6700 1 -- 172.21.15.40:0/1465370290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a01f3510 con 0x7ff3a00c6740 2024-09-13T18:10:14.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff396ffd700 1 -- 172.21.15.40:0/1465370290 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39c00fbc0 con 0x7ff3a00c6740 2024-09-13T18:10:14.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a72b6700 1 -- 172.21.15.40:0/1465370290 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a01f37f0 con 0x7ff3a00c6740 2024-09-13T18:10:14.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a72b6700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0155340 0x7ff3a0155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a72b6700 1 -- 172.21.15.40:0/1465370290 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a0067560 con 0x7ff3a0155340 2024-09-13T18:10:14.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a5ab3700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0155340 0x7ff3a0155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a5ab3700 1 -- 172.21.15.40:0/1465370290 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff3a00c1310 con 0x7ff3a0155340 2024-09-13T18:10:14.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff396ffd700 1 -- 172.21.15.40:0/1465370290 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff39c00fd20 con 0x7ff3a00c6740 2024-09-13T18:10:14.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.749+0000 7ff3a5ab3700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0155340 0x7ff3a0155740 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00c64a0 tx=0x7ff390004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:14.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.753+0000 7ff396ffd700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff384066760 0x7ff384068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.753+0000 7ff396ffd700 1 -- 172.21.15.40:0/1465370290 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff39c084520 con 0x7ff3a00c6740 2024-09-13T18:10:14.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.753+0000 7ff3a6ab5700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff384066760 0x7ff384068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.753+0000 7ff396ffd700 1 -- 172.21.15.40:0/1465370290 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390006cc0 con 0x7ff3a0155340 2024-09-13T18:10:14.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.753+0000 7ff396ffd700 1 -- 172.21.15.40:0/1465370290 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff3900094b0 con 0x7ff3a0155340 2024-09-13T18:10:14.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.753+0000 7ff396ffd700 1 -- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0155340 msgr2=0x7ff3a0155740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.753+0000 7ff396ffd700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0155340 0x7ff3a0155740 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00c64a0 tx=0x7ff390004770 comp rx=0 tx=0).stop 2024-09-13T18:10:14.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.753+0000 7ff396ffd700 1 -- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0155340 msgr2=0x7ff3a0155740 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:10:14.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.753+0000 7ff396ffd700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0155340 0x7ff3a0155740 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00c64a0 tx=0x7ff390004770 comp rx=0 tx=0).stop 2024-09-13T18:10:14.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.753+0000 7ff3a6ab5700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff384066760 0x7ff384068c20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff3a015dd30 tx=0x7ff39800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:14.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.761+0000 7ff3a72b6700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0155c80 0x7ff3a0152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:14.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.761+0000 7ff3a72b6700 1 -- 172.21.15.40:0/1465370290 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a00b12b0 con 0x7ff3a0155c80 2024-09-13T18:10:14.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.761+0000 7ff3a5ab3700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0155c80 0x7ff3a0152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:14.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.761+0000 7ff3a5ab3700 1 -- 172.21.15.40:0/1465370290 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff3a01efa00 con 0x7ff3a0155c80 2024-09-13T18:10:14.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.761+0000 7ff3a5ab3700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0155c80 0x7ff3a0152f40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0072b40 tx=0x7ff3900047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:14.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.761+0000 7ff396ffd700 1 -- 172.21.15.40:0/1465370290 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39002b780 con 0x7ff3a0155c80 2024-09-13T18:10:14.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.761+0000 7ff396ffd700 1 -- 172.21.15.40:0/1465370290 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4617 (secure 0 0 0) 0x7ff39002b8e0 con 0x7ff3a0155c80 2024-09-13T18:10:14.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.761+0000 7ff396ffd700 1 -- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0155c80 msgr2=0x7ff3a0152f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.761+0000 7ff396ffd700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0155c80 0x7ff3a0152f40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0072b40 tx=0x7ff3900047e0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.761+0000 7ff396ffd700 1 -- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0155c80 msgr2=0x7ff3a0152f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:10:14.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.761+0000 7ff396ffd700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3a0155c80 0x7ff3a0152f40 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0072b40 tx=0x7ff3900047e0 comp rx=0 tx=0).stop 2024-09-13T18:10:14.771 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:10:14.771 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:10:14.771 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:10:14.771 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:10:14.772 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 158, 2024-09-13T18:10:14.772 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:10:14.772 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:10:14.772 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:10:14.772 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:10:14.772 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:14.772 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 12, 2024-09-13T18:10:14.772 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:10:14.772 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:10:14.772 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:10:14.772 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:14.773 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:14.773 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:14.773 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:14.773 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:14.773 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:14.773 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:14.773 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:14.773 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:14.773 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:14.773 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:14.773 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:10:14.773 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:10:14.773 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:14.774 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:14.774 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:14.774 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:14.774 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:14.774 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:14.774 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:14.774 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:14.774 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:14.774 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:14.774 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:14.774 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.774 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:10:14.774 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:10:14.775 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:10:14.775 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:10:14.775 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:10:14.775 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:10:14.775 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:10:14.775 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:10:14.775 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:10:14.775 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:10:14.775 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:10:14.775 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:10:14.775 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:10:14.775 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:10:14.776 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:10:14.776 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:10:14.776 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:10:14.776 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:14.776 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:14.776 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:14.776 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:14.776 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:14.776 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:14.776 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:14.777 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:14.777 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:14.777 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:14.777 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:14.777 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:10:14.777 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.777 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:10:14.777 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.777 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:10:14.777 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:10:14.778 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:14.778 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:14.778 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.778 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:14.778 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:10:14.778 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:14.778 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.778 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.778 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:14.778 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:10:14.778 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:14.778 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.778 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:14.778 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.779 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:10:14.779 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:10:14.779 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:14.779 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:14.779 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:14.779 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.779 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.779 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:10:14.779 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:10:14.779 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:14.779 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:14.779 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.779 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:14.780 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:10:14.780 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:14.780 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.780 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.780 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:14.780 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:10:14.780 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:14.781 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.781 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:14.781 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.781 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:10:14.781 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:10:14.781 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:14.781 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:14.781 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:14.781 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.781 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.781 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:10:14.781 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:10:14.782 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:14.782 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:14.782 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.782 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:14.782 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:10:14.782 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:14.782 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.782 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.782 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:14.782 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:10:14.783 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:14.783 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.783 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:14.783 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.783 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:10:14.783 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:10:14.783 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:14.783 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:14.783 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:14.783 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.783 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:14.783 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.784 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:10:14.784 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:10:14.784 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.784 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:14.784 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:14.784 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:10:14.784 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.785 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:14.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:10:14.785 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.785 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:14.785 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:14.785 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:10:14.785 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.785 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:14.785 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:10:14.785 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:14.785 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:14.785 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:14.786 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:10:14.786 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:14.786 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:14.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:14.786 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:10:14.786 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:10:14.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.765+0000 7ff3a72b6700 1 -- 172.21.15.40:0/1465370290 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff384066760 msgr2=0x7ff384068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.765+0000 7ff3a72b6700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff384066760 0x7ff384068c20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff3a015dd30 tx=0x7ff39800b040 comp rx=0 tx=0).stop 2024-09-13T18:10:14.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.765+0000 7ff3a72b6700 1 -- 172.21.15.40:0/1465370290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3a00c6740 msgr2=0x7ff3a0149d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:14.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.765+0000 7ff3a72b6700 1 --2- 172.21.15.40:0/1465370290 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3a00c6740 0x7ff3a0149d50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0048f0 tx=0x7ff39c007620 comp rx=0 tx=0).stop 2024-09-13T18:10:14.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.765+0000 7ff3a6ab5700 1 -- 172.21.15.40:0/1465370290 reap_dead start 2024-09-13T18:10:14.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.765+0000 7ff3a6ab5700 1 -- 172.21.15.40:0/1465370290 reap_dead start 2024-09-13T18:10:14.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.765+0000 7ff3a72b6700 1 -- 172.21.15.40:0/1465370290 shutdown_connections 2024-09-13T18:10:14.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.765+0000 7ff3a72b6700 1 -- 172.21.15.40:0/1465370290 >> 172.21.15.40:0/1465370290 conn(0x7ff3a00cc7f0 msgr2=0x7ff3a01547c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:14.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.765+0000 7ff3a72b6700 1 -- 172.21.15.40:0/1465370290 shutdown_connections 2024-09-13T18:10:14.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:14.765+0000 7ff3a72b6700 1 -- 172.21.15.40:0/1465370290 wait complete. 2024-09-13T18:10:14.789 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:10:14.789 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-13T18:10:14.789 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['c'] 2024-09-13T18:10:14.789 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-13T18:10:14.790 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-13T18:10:14.790 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T18:10:14.790 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:10:14.849 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T18:10:14.849 INFO:tasks.mon_thrash:Sending STOP to mon c 2024-09-13T18:10:14.849 INFO:tasks.ceph.mon.c:Sent signal 19 2024-09-13T18:10:14.850 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-13T18:10:29.850 INFO:tasks.mon_thrash:Sending CONT to mon c 2024-09-13T18:10:29.851 INFO:tasks.ceph.mon.c:Sent signal 18 2024-09-13T18:10:29.851 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T18:10:29.852 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:10:29.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:29.969+0000 7f711e5b1700 1 Processor -- start 2024-09-13T18:10:29.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:29.969+0000 7f711e5b1700 1 -- start start 2024-09-13T18:10:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:29.973+0000 7f711e5b1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 0x7f71180c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:29.973+0000 7f711e5b1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 0x7f71180c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:29.973+0000 7f711e5b1700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711815a840 con 0x7f7118070120 2024-09-13T18:10:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:29.973+0000 7f711e5b1700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71181516a0 con 0x7f71180b2d00 2024-09-13T18:10:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:29.973+0000 7f711e5b1700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71181580a0 con 0x7f71181512c0 2024-09-13T18:10:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:29.973+0000 7f711ddb0700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7118070120 0x7f711815acd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45994/0 (socket says 172.21.15.40:45994) 2024-09-13T18:10:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:29.973+0000 7f711ddb0700 1 -- 172.21.15.40:0/4189575107 learned_addr learned my addr 172.21.15.40:0/4189575107 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:10:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:29.973+0000 7f711cdae700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 0x7f71180c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:29.973+0000 7f711cdae700 1 -- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 msgr2=0x7f71180c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:10:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:29.973+0000 7f711cdae700 1 -- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 msgr2=0x7f71180c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:10:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:29.973+0000 7f711cdae700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 0x7f71180c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:10:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:29.973+0000 7f711cdae700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 0x7f71180c7e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:10:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:29.973+0000 7f711d5af700 1 --2- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 0x7f71180c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:29.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:29.973+0000 7f711d5af700 1 -- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 msgr2=0x7f71180c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:10:29.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:29.973+0000 7f711d5af700 1 -- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 msgr2=0x7f71180c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:10:29.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:29.973+0000 7f711d5af700 1 --2- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 0x7f71180c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:10:29.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:29.973+0000 7f711d5af700 1 --2- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 0x7f71180c78f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:10:30.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:30.177+0000 7f711cdae700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 0x7f71180c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:30.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:30.177+0000 7f711cdae700 1 -- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 msgr2=0x7f71180c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:10:30.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:30.177+0000 7f711cdae700 1 -- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 msgr2=0x7f71180c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:10:30.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:30.177+0000 7f711cdae700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 0x7f71180c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:10:30.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:30.177+0000 7f711cdae700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 0x7f71180c7e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:10:30.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:30.177+0000 7f711d5af700 1 --2- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 0x7f71180c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:30.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:30.177+0000 7f711d5af700 1 -- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 msgr2=0x7f71180c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:10:30.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:30.177+0000 7f711d5af700 1 -- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 msgr2=0x7f71180c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:10:30.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:30.177+0000 7f711d5af700 1 --2- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 0x7f71180c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:10:30.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:30.177+0000 7f711d5af700 1 --2- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 0x7f71180c78f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:10:30.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:30.581+0000 7f711cdae700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 0x7f71180c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:30.581+0000 7f711d5af700 1 --2- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 0x7f71180c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:30.581+0000 7f711cdae700 1 -- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 msgr2=0x7f71180c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:10:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:30.581+0000 7f711cdae700 1 -- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 msgr2=0x7f71180c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:10:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:30.581+0000 7f711cdae700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 0x7f71180c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:10:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:30.581+0000 7f711cdae700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 0x7f71180c7e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:10:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:30.581+0000 7f711d5af700 1 -- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 msgr2=0x7f71180c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:10:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:30.581+0000 7f711d5af700 1 -- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 msgr2=0x7f71180c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:10:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:30.581+0000 7f711d5af700 1 --2- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 0x7f71180c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:10:30.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:30.581+0000 7f711d5af700 1 --2- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 0x7f71180c78f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:10:31.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:31.385+0000 7f711cdae700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 0x7f71180c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:31.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:31.385+0000 7f711d5af700 1 --2- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 0x7f71180c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:31.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:31.385+0000 7f711cdae700 1 -- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 msgr2=0x7f71180c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:10:31.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:31.385+0000 7f711cdae700 1 -- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 msgr2=0x7f71180c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:10:31.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:31.385+0000 7f711cdae700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 0x7f71180c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:10:31.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:31.385+0000 7f711cdae700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 0x7f71180c7e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:10:31.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:31.385+0000 7f711d5af700 1 -- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 msgr2=0x7f71180c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:10:31.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:31.385+0000 7f711d5af700 1 -- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 msgr2=0x7f71180c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:10:31.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:31.385+0000 7f711d5af700 1 --2- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 0x7f71180c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:10:31.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:31.385+0000 7f711d5af700 1 --2- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 0x7f71180c78f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:10:32.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:32.973+0000 7f710effd700 1 -- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 msgr2=0x7f71180c78f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:10:32.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:32.973+0000 7f710effd700 1 --2- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71181512c0 0x7f71180c78f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:32.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:32.973+0000 7f710effd700 1 -- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 msgr2=0x7f71180c7e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:10:32.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:32.973+0000 7f710effd700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.40:3300/0 conn(0x7f71180b2d00 0x7f71180c7e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:32.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:32.973+0000 7f710effd700 1 -- 172.21.15.40:0/4189575107 >> v1:172.21.15.40:6789/0 conn(0x7f7118070120 legacy=0x7f711815acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:32.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:32.973+0000 7f710effd700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.43:3300/0 conn(0x7f70fc007460 0x7f70fc009900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:32.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:32.973+0000 7f710effd700 1 -- 172.21.15.40:0/4189575107 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7118157520 con 0x7f70fc003720 2024-09-13T18:10:32.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:32.973+0000 7f710effd700 1 -- 172.21.15.40:0/4189575107 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71181516a0 con 0x7f70fc007460 2024-09-13T18:10:32.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:32.973+0000 7f711d5af700 1 -- 172.21.15.40:0/4189575107 >> v1:172.21.15.43:6789/0 conn(0x7f70fc003720 legacy=0x7f70fc005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:10:32.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:32.973+0000 7f711cdae700 1 -- 172.21.15.40:0/4189575107 >> v2:172.21.15.43:3300/0 conn(0x7f70fc007460 msgr2=0x7f70fc009900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:10:32.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:32.973+0000 7f711cdae700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.43:3300/0 conn(0x7f70fc007460 0x7f70fc009900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:10:33.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:33.177+0000 7f711d5af700 1 -- 172.21.15.40:0/4189575107 >> v1:172.21.15.43:6789/0 conn(0x7f70fc003720 legacy=0x7f70fc005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:10:33.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:33.177+0000 7f711cdae700 1 -- 172.21.15.40:0/4189575107 >> v2:172.21.15.43:3300/0 conn(0x7f70fc007460 msgr2=0x7f70fc009900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:10:33.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:33.177+0000 7f711cdae700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.43:3300/0 conn(0x7f70fc007460 0x7f70fc009900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:10:33.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:33.581+0000 7f711cdae700 1 -- 172.21.15.40:0/4189575107 >> v2:172.21.15.43:3300/0 conn(0x7f70fc007460 msgr2=0x7f70fc009900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:10:33.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:33.581+0000 7f711d5af700 1 -- 172.21.15.40:0/4189575107 >> v1:172.21.15.43:6789/0 conn(0x7f70fc003720 legacy=0x7f70fc005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:10:33.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:33.581+0000 7f711cdae700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.43:3300/0 conn(0x7f70fc007460 0x7f70fc009900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:10:34.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:34.385+0000 7f711cdae700 1 -- 172.21.15.40:0/4189575107 >> v2:172.21.15.43:3300/0 conn(0x7f70fc007460 msgr2=0x7f70fc009900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:10:34.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:34.385+0000 7f711d5af700 1 -- 172.21.15.40:0/4189575107 >> v1:172.21.15.43:6789/0 conn(0x7f70fc003720 legacy=0x7f70fc005ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:10:34.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:34.385+0000 7f711cdae700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.43:3300/0 conn(0x7f70fc007460 0x7f70fc009900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:10:35.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f710effd700 1 -- 172.21.15.40:0/4189575107 >> v2:172.21.15.43:3300/0 conn(0x7f70fc007460 msgr2=0x7f70fc009900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:10:35.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f710effd700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.43:3300/0 conn(0x7f70fc007460 0x7f70fc009900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:35.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f710effd700 1 -- 172.21.15.40:0/4189575107 >> v1:172.21.15.43:6789/0 conn(0x7f70fc003720 legacy=0x7f70fc005ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:10:35.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f711ddb0700 1 -- 172.21.15.40:0/4189575107 reap_dead start 2024-09-13T18:10:35.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f710effd700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.43:3300/0 conn(0x7f70fc007460 0x7f70fc009900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:35.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f710effd700 1 --2- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70fc0062d0 0x7f70fc005b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:35.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f710effd700 1 -- 172.21.15.40:0/4189575107 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70fc00b000 con 0x7f70fc006750 2024-09-13T18:10:35.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f710effd700 1 -- 172.21.15.40:0/4189575107 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71181516a0 con 0x7f70fc007460 2024-09-13T18:10:35.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f710effd700 1 -- 172.21.15.40:0/4189575107 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71181580a0 con 0x7f70fc0062d0 2024-09-13T18:10:35.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f711d5af700 1 -- 172.21.15.40:0/4189575107 >> v2:172.21.15.43:3300/0 conn(0x7f70fc007460 msgr2=0x7f70fc009900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:10:35.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f711d5af700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.43:3300/0 conn(0x7f70fc007460 0x7f70fc009900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:10:35.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f711ddb0700 1 -- 172.21.15.40:0/4189575107 >> v1:172.21.15.43:6789/0 conn(0x7f70fc006750 legacy=0x7f70fc006c00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:10:35.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f711cdae700 1 --2- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70fc0062d0 0x7f70fc005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:35.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f711cdae700 1 -- 172.21.15.40:0/4189575107 >> v2:172.21.15.43:3300/0 conn(0x7f70fc007460 msgr2=0x7f70fc009900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:10:35.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f711cdae700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.43:3300/0 conn(0x7f70fc007460 0x7f70fc009900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:35.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f711cdae700 1 -- 172.21.15.40:0/4189575107 >> v1:172.21.15.43:6789/0 conn(0x7f70fc006750 legacy=0x7f70fc006c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:10:35.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f711cdae700 1 -- 172.21.15.40:0/4189575107 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7118157520 con 0x7f70fc0062d0 2024-09-13T18:10:35.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f711cdae700 1 --2- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70fc0062d0 0x7f70fc005b10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7108006b40 tx=0x7f710800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b4ea69f0401e166 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:35.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f710ffff700 1 -- 172.21.15.40:0/4189575107 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108004020 con 0x7f70fc0062d0 2024-09-13T18:10:35.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f710ffff700 1 -- 172.21.15.40:0/4189575107 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710800c040 con 0x7f70fc0062d0 2024-09-13T18:10:35.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.973+0000 7f710ffff700 1 -- 172.21.15.40:0/4189575107 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71080043a0 con 0x7f70fc0062d0 2024-09-13T18:10:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.977+0000 7f711e5b1700 1 -- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70fc0062d0 msgr2=0x7f70fc005b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.977+0000 7f711e5b1700 1 --2- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70fc0062d0 0x7f70fc005b10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7108006b40 tx=0x7f710800ce30 comp rx=0 tx=0).stop 2024-09-13T18:10:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.977+0000 7f711e5b1700 1 -- 172.21.15.40:0/4189575107 shutdown_connections 2024-09-13T18:10:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.977+0000 7f711e5b1700 1 --2- 172.21.15.40:0/4189575107 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70fc0062d0 0x7f70fc005b10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.977+0000 7f711e5b1700 1 --2- 172.21.15.40:0/4189575107 >> v2:172.21.15.43:3300/0 conn(0x7f70fc007460 0x7f70fc009900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:35.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.977+0000 7f711e5b1700 1 -- 172.21.15.40:0/4189575107 >> 172.21.15.40:0/4189575107 conn(0x7f71180ccef0 msgr2=0x7f71180cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:35.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.977+0000 7f711e5b1700 1 -- 172.21.15.40:0/4189575107 shutdown_connections 2024-09-13T18:10:35.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.977+0000 7f711e5b1700 1 -- 172.21.15.40:0/4189575107 wait complete. 2024-09-13T18:10:35.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.977+0000 7f711e5b1700 1 Processor -- start 2024-09-13T18:10:35.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.977+0000 7f711e5b1700 1 -- start start 2024-09-13T18:10:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f711e5b1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70fc0062d0 0x7f71181f04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f711e5b1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70fc006750 0x7f71180c5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f711e5b1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70fc007460 0x7f711814be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f711e5b1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71181f13b0 con 0x7f70fc007460 2024-09-13T18:10:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f711e5b1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71181f0b10 con 0x7f70fc006750 2024-09-13T18:10:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f711e5b1700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71181f0df0 con 0x7f70fc0062d0 2024-09-13T18:10:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f711cdae700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70fc006750 msgr2=0x7f71180c5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:10:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f711cdae700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70fc006750 0x7f71180c5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:10:35.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f711ddb0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70fc007460 0x7f711814be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f711d5af700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70fc0062d0 0x7f71181f04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f711ddb0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70fc007460 0x7f711814be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55360/0 (socket says 172.21.15.40:55360) 2024-09-13T18:10:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f711ddb0700 1 -- 172.21.15.40:0/587051742 learned_addr learned my addr 172.21.15.40:0/587051742 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:10:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f711d5af700 1 -- 172.21.15.40:0/587051742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70fc006750 msgr2=0x7f71180c5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:10:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f711d5af700 1 --2- 172.21.15.40:0/587051742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70fc006750 0x7f71180c5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f711d5af700 1 -- 172.21.15.40:0/587051742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70fc007460 msgr2=0x7f711814be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f711d5af700 1 --2- 172.21.15.40:0/587051742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70fc007460 0x7f711814be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f711d5af700 1 -- 172.21.15.40:0/587051742 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710800c7b0 con 0x7f70fc0062d0 2024-09-13T18:10:35.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f711d5af700 1 --2- 172.21.15.40:0/587051742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70fc0062d0 0x7f71181f04d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7114004740 tx=0x7f711400c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:35.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f710dffb700 1 -- 172.21.15.40:0/587051742 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7114016660 con 0x7f70fc0062d0 2024-09-13T18:10:35.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f711e5b1700 1 -- 172.21.15.40:0/587051742 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71181f10d0 con 0x7f70fc0062d0 2024-09-13T18:10:35.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f711e5b1700 1 -- 172.21.15.40:0/587051742 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711814b8e0 con 0x7f70fc0062d0 2024-09-13T18:10:35.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f710dffb700 1 -- 172.21.15.40:0/587051742 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711400c040 con 0x7f70fc0062d0 2024-09-13T18:10:35.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.981+0000 7f710dffb700 1 -- 172.21.15.40:0/587051742 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7114016a20 con 0x7f70fc0062d0 2024-09-13T18:10:35.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.985+0000 7f711e5b1700 1 -- 172.21.15.40:0/587051742 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f8005310 con 0x7f70fc0062d0 2024-09-13T18:10:35.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.985+0000 7f710dffb700 1 -- 172.21.15.40:0/587051742 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7114016b80 con 0x7f70fc0062d0 2024-09-13T18:10:35.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.985+0000 7f710dffb700 1 --2- 172.21.15.40:0/587051742 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70fc067e40 0x7f70fc06a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:35.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.985+0000 7f711cdae700 1 --2- 172.21.15.40:0/587051742 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70fc067e40 0x7f70fc06a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:35.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.985+0000 7f710dffb700 1 -- 172.21.15.40:0/587051742 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7114085000 con 0x7f70fc0062d0 2024-09-13T18:10:35.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.985+0000 7f711cdae700 1 --2- 172.21.15.40:0/587051742 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70fc067e40 0x7f70fc06a300 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7108006b40 tx=0x7f7108019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:36.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:35.997+0000 7f710dffb700 1 -- 172.21.15.40:0/587051742 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7114051e80 con 0x7f70fc0062d0 2024-09-13T18:10:36.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.273+0000 7f711e5b1700 1 -- 172.21.15.40:0/587051742 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f70f8005890 con 0x7f70fc0062d0 2024-09-13T18:10:36.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.277+0000 7f710dffb700 1 -- 172.21.15.40:0/587051742 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f7114024610 con 0x7f70fc0062d0 2024-09-13T18:10:36.283 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":160,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:10:36.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.277+0000 7f711e5b1700 1 -- 172.21.15.40:0/587051742 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70fc067e40 msgr2=0x7f70fc06a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:36.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.277+0000 7f711e5b1700 1 --2- 172.21.15.40:0/587051742 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70fc067e40 0x7f70fc06a300 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7108006b40 tx=0x7f7108019000 comp rx=0 tx=0).stop 2024-09-13T18:10:36.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.277+0000 7f711e5b1700 1 -- 172.21.15.40:0/587051742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70fc0062d0 msgr2=0x7f71181f04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:36.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.277+0000 7f711e5b1700 1 --2- 172.21.15.40:0/587051742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70fc0062d0 0x7f71181f04d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7114004740 tx=0x7f711400c8b0 comp rx=0 tx=0).stop 2024-09-13T18:10:36.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.277+0000 7f711e5b1700 1 -- 172.21.15.40:0/587051742 shutdown_connections 2024-09-13T18:10:36.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.277+0000 7f711e5b1700 1 --2- 172.21.15.40:0/587051742 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70fc007460 0x7f711814be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:36.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.277+0000 7f711e5b1700 1 --2- 172.21.15.40:0/587051742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70fc006750 0x7f71180c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:36.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.277+0000 7f711e5b1700 1 --2- 172.21.15.40:0/587051742 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70fc067e40 0x7f70fc06a300 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:36.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.277+0000 7f711e5b1700 1 --2- 172.21.15.40:0/587051742 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70fc0062d0 0x7f71181f04d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:36.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.277+0000 7f711e5b1700 1 -- 172.21.15.40:0/587051742 >> 172.21.15.40:0/587051742 conn(0x7f71180ccef0 msgr2=0x7f711806dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:36.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.277+0000 7f711e5b1700 1 -- 172.21.15.40:0/587051742 shutdown_connections 2024-09-13T18:10:36.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.277+0000 7f711e5b1700 1 -- 172.21.15.40:0/587051742 wait complete. 2024-09-13T18:10:36.299 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T18:10:36.300 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-13T18:10:36.300 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:10:36.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.421+0000 7f57621c7700 1 Processor -- start 2024-09-13T18:10:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.421+0000 7f57621c7700 1 -- start start 2024-09-13T18:10:36.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.421+0000 7f57621c7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f575c0c54e0 0x7f575c071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f57621c7700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f575c0706f0 con 0x7f575c150240 2024-09-13T18:10:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f57621c7700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f575c0c28c0 con 0x7f575c0c4f90 2024-09-13T18:10:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f57621c7700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f575c070450 con 0x7f575c0c54e0 2024-09-13T18:10:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f57619c6700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7f575c0c4f90 legacy=0x7f575c160990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:10:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f57609c4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f575c0c54e0 0x7f575c071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f57611c5700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f575c150240 0x7f575c070ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:41682/0 (socket says 172.21.15.40:41682) 2024-09-13T18:10:36.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f57611c5700 1 -- 172.21.15.40:0/886265640 learned_addr learned my addr 172.21.15.40:0/886265640 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:10:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f57609c4700 1 -- 172.21.15.40:0/886265640 >> v1:172.21.15.43:6789/0 conn(0x7f575c0c4f90 legacy=0x7f575c160990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:10:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f57609c4700 1 -- 172.21.15.40:0/886265640 >> v1:172.21.15.40:6789/0 conn(0x7f575c150240 legacy=0x7f575c070ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f5753fff700 1 -- 172.21.15.40:0/886265640 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469386128 0 0) 0x7f575c0706f0 con 0x7f575c150240 2024-09-13T18:10:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f57609c4700 1 -- 172.21.15.40:0/886265640 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575c156c00 con 0x7f575c0c54e0 2024-09-13T18:10:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f57609c4700 1 --2- 172.21.15.40:0/886265640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c0c54e0 0x7f575c071660 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5758009850 tx=0x7f5758009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6baaffe3a0cedb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f5753fff700 1 -- 172.21.15.40:0/886265640 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5758004230 con 0x7f575c0c54e0 2024-09-13T18:10:36.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f5753fff700 1 -- 172.21.15.40:0/886265640 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5758004390 con 0x7f575c0c54e0 2024-09-13T18:10:36.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f5753fff700 1 -- 172.21.15.40:0/886265640 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57580036c0 con 0x7f575c0c54e0 2024-09-13T18:10:36.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f57621c7700 1 -- 172.21.15.40:0/886265640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c0c54e0 msgr2=0x7f575c071660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:36.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f57621c7700 1 --2- 172.21.15.40:0/886265640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c0c54e0 0x7f575c071660 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5758009850 tx=0x7f5758009b60 comp rx=0 tx=0).stop 2024-09-13T18:10:36.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f57621c7700 1 -- 172.21.15.40:0/886265640 shutdown_connections 2024-09-13T18:10:36.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f57621c7700 1 --2- 172.21.15.40:0/886265640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c0c54e0 0x7f575c071660 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:36.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f57621c7700 1 -- 172.21.15.40:0/886265640 >> 172.21.15.40:0/886265640 conn(0x7f575c06a390 msgr2=0x7f575c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:36.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f57621c7700 1 -- 172.21.15.40:0/886265640 shutdown_connections 2024-09-13T18:10:36.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.425+0000 7f57621c7700 1 -- 172.21.15.40:0/886265640 wait complete. 2024-09-13T18:10:36.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57621c7700 1 Processor -- start 2024-09-13T18:10:36.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57621c7700 1 -- start start 2024-09-13T18:10:36.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57621c7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f575c0c4f90 0x7f575c155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:36.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57621c7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f575c0c54e0 0x7f575c1eebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:36.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57621c7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c150240 0x7f575c14ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:36.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57621c7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575c1ef120 con 0x7f575c150240 2024-09-13T18:10:36.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57621c7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575c154bb0 con 0x7f575c0c54e0 2024-09-13T18:10:36.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57621c7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f575c0c6030 con 0x7f575c0c4f90 2024-09-13T18:10:36.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57609c4700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f575c0c54e0 msgr2=0x7f575c1eebe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:10:36.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57609c4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f575c0c54e0 0x7f575c1eebe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:10:36.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57619c6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c150240 0x7f575c14ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:36.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57619c6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c150240 0x7f575c14ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55378/0 (socket says 172.21.15.40:55378) 2024-09-13T18:10:36.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57619c6700 1 -- 172.21.15.40:0/2043065876 learned_addr learned my addr 172.21.15.40:0/2043065876 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:10:36.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57611c5700 1 --2- 172.21.15.40:0/2043065876 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f575c0c4f90 0x7f575c155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:36.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57619c6700 1 -- 172.21.15.40:0/2043065876 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f575c0c4f90 msgr2=0x7f575c155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:36.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57619c6700 1 --2- 172.21.15.40:0/2043065876 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f575c0c4f90 0x7f575c155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:36.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57619c6700 1 -- 172.21.15.40:0/2043065876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f575c0c54e0 msgr2=0x7f575c1eebe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:10:36.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57619c6700 1 --2- 172.21.15.40:0/2043065876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f575c0c54e0 0x7f575c1eebe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:36.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57619c6700 1 -- 172.21.15.40:0/2043065876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57580094b0 con 0x7f575c150240 2024-09-13T18:10:36.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57619c6700 1 --2- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c150240 0x7f575c14ca50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f574c00b500 tx=0x7f574c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:36.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f5751ffb700 1 -- 172.21.15.40:0/2043065876 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c003e20 con 0x7f575c150240 2024-09-13T18:10:36.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f5751ffb700 1 -- 172.21.15.40:0/2043065876 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574c005070 con 0x7f575c150240 2024-09-13T18:10:36.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57621c7700 1 -- 172.21.15.40:0/2043065876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f575c0c5d50 con 0x7f575c150240 2024-09-13T18:10:36.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f5751ffb700 1 -- 172.21.15.40:0/2043065876 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c005420 con 0x7f575c150240 2024-09-13T18:10:36.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.429+0000 7f57621c7700 1 -- 172.21.15.40:0/2043065876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f575c1f2540 con 0x7f575c150240 2024-09-13T18:10:36.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.433+0000 7f57621c7700 1 --2- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c1546e0 0x7f575c1f2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:36.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.433+0000 7f57621c7700 1 -- 172.21.15.40:0/2043065876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575c067600 con 0x7f575c1546e0 2024-09-13T18:10:36.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.433+0000 7f57611c5700 1 --2- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c1546e0 0x7f575c1f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:36.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.433+0000 7f57611c5700 1 -- 172.21.15.40:0/2043065876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f575c06cfb0 con 0x7f575c1546e0 2024-09-13T18:10:36.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.433+0000 7f5751ffb700 1 -- 172.21.15.40:0/2043065876 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f574c005580 con 0x7f575c150240 2024-09-13T18:10:36.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.433+0000 7f57611c5700 1 --2- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c1546e0 0x7f575c1f2700 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5754000f80 tx=0x7f5754007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:36.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.433+0000 7f5751ffb700 1 --2- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f57400666e0 0x7f5740068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:36.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.433+0000 7f5751ffb700 1 -- 172.21.15.40:0/2043065876 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f574c0838a0 con 0x7f575c150240 2024-09-13T18:10:36.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.433+0000 7f57609c4700 1 --2- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f57400666e0 0x7f5740068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.433+0000 7f5751ffb700 1 -- 172.21.15.40:0/2043065876 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57540124e0 con 0x7f575c1546e0 2024-09-13T18:10:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.433+0000 7f5751ffb700 1 -- 172.21.15.40:0/2043065876 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5754018040 con 0x7f575c1546e0 2024-09-13T18:10:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.433+0000 7f5751ffb700 1 -- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c1546e0 msgr2=0x7f575c1f2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.433+0000 7f5751ffb700 1 --2- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c1546e0 0x7f575c1f2700 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5754000f80 tx=0x7f5754007a00 comp rx=0 tx=0).stop 2024-09-13T18:10:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.433+0000 7f5751ffb700 1 -- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c1546e0 msgr2=0x7f575c1f2700 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:10:36.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.433+0000 7f5751ffb700 1 --2- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c1546e0 0x7f575c1f2700 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:36.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.433+0000 7f57609c4700 1 --2- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f57400666e0 0x7f5740068ba0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5758009850 tx=0x7f5758008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:36.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.445+0000 7f57621c7700 1 --2- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c1f2d40 0x7f575c14cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:36.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.445+0000 7f57621c7700 1 -- 172.21.15.40:0/2043065876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575c14d4d0 con 0x7f575c1f2d40 2024-09-13T18:10:36.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.445+0000 7f57611c5700 1 --2- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c1f2d40 0x7f575c14cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:36.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.445+0000 7f57611c5700 1 -- 172.21.15.40:0/2043065876 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f575c158b70 con 0x7f575c1f2d40 2024-09-13T18:10:36.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.445+0000 7f57611c5700 1 --2- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c1f2d40 0x7f575c14cf90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5754001970 tx=0x7f5754001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:36.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.445+0000 7f5751ffb700 1 -- 172.21.15.40:0/2043065876 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575402a070 con 0x7f575c1f2d40 2024-09-13T18:10:36.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.445+0000 7f5751ffb700 1 -- 172.21.15.40:0/2043065876 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4773 (secure 0 0 0) 0x7f575401d460 con 0x7f575c1f2d40 2024-09-13T18:10:36.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.445+0000 7f5751ffb700 1 -- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c1f2d40 msgr2=0x7f575c14cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:36.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.445+0000 7f5751ffb700 1 --2- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c1f2d40 0x7f575c14cf90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5754001970 tx=0x7f5754001250 comp rx=0 tx=0).stop 2024-09-13T18:10:36.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.445+0000 7f5751ffb700 1 -- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c1f2d40 msgr2=0x7f575c14cf90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:10:36.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.445+0000 7f5751ffb700 1 --2- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c1f2d40 0x7f575c14cf90 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5754001970 tx=0x7f5754001250 comp rx=0 tx=0).stop 2024-09-13T18:10:36.455 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:10:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:10:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:10:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:10:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 160, 2024-09-13T18:10:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:10:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:10:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:10:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:10:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:10:36.455 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:10:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:10:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:36.456 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:10:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:10:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:36.457 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.458 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:10:36.458 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:10:36.458 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:10:36.458 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:10:36.458 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:10:36.458 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:10:36.458 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:10:36.458 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:10:36.458 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:10:36.458 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:10:36.458 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:10:36.458 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:10:36.458 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:10:36.458 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:10:36.459 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:10:36.459 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:10:36.459 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:10:36.459 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:36.459 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:36.459 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:36.459 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:36.459 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:36.459 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:36.459 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:36.459 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:36.459 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:36.459 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:36.459 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:36.460 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:10:36.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.460 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:10:36.460 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.460 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:10:36.460 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:10:36.460 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:36.460 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:36.460 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.460 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:36.460 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:10:36.460 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:36.460 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.461 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.461 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:36.461 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:10:36.461 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:36.461 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:36.461 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:36.461 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.461 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:10:36.461 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:10:36.461 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:36.461 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:36.461 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:36.461 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.461 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.462 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:10:36.462 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:10:36.462 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:36.462 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:36.462 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.462 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:36.462 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:10:36.462 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:36.462 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.462 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.462 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:36.462 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:10:36.463 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:36.463 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:36.463 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:36.463 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.463 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:10:36.463 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:10:36.463 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:36.463 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:36.463 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:36.463 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.463 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.464 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:10:36.464 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:10:36.464 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:36.464 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:36.464 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.464 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:36.464 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:10:36.464 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:36.464 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.464 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.465 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:36.465 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:10:36.465 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:36.465 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:36.465 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:36.465 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.465 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:10:36.465 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:10:36.465 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:36.465 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:36.465 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:36.465 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:36.465 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:36.465 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.466 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:10:36.466 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:10:36.466 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.466 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:36.466 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:36.466 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:10:36.466 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:36.466 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:36.466 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:10:36.466 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.466 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:36.466 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:36.466 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:10:36.467 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:36.467 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:36.467 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:10:36.467 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.467 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:36.467 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:36.467 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:10:36.467 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:36.467 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:36.467 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:10:36.467 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.467 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:36.467 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:36.467 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:10:36.467 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:36.468 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:36.468 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.468 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:10:36.468 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:10:36.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.445+0000 7f57621c7700 1 -- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f57400666e0 msgr2=0x7f5740068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:36.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.445+0000 7f57621c7700 1 --2- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f57400666e0 0x7f5740068ba0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5758009850 tx=0x7f5758008030 comp rx=0 tx=0).stop 2024-09-13T18:10:36.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.445+0000 7f57621c7700 1 -- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c150240 msgr2=0x7f575c14ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:36.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.445+0000 7f57621c7700 1 --2- 172.21.15.40:0/2043065876 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f575c150240 0x7f575c14ca50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f574c00b500 tx=0x7f574c00b810 comp rx=0 tx=0).stop 2024-09-13T18:10:36.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.445+0000 7f57619c6700 1 -- 172.21.15.40:0/2043065876 reap_dead start 2024-09-13T18:10:36.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.449+0000 7f57619c6700 1 -- 172.21.15.40:0/2043065876 reap_dead start 2024-09-13T18:10:36.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.449+0000 7f57621c7700 1 -- 172.21.15.40:0/2043065876 shutdown_connections 2024-09-13T18:10:36.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.449+0000 7f57621c7700 1 -- 172.21.15.40:0/2043065876 >> 172.21.15.40:0/2043065876 conn(0x7f575c06a390 msgr2=0x7f575c14eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:36.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.449+0000 7f57621c7700 1 -- 172.21.15.40:0/2043065876 shutdown_connections 2024-09-13T18:10:36.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.449+0000 7f57621c7700 1 -- 172.21.15.40:0/2043065876 wait complete. 2024-09-13T18:10:36.470 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:10:36.470 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T18:10:36.470 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:10:36.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.577+0000 7f7658825700 1 Processor -- start 2024-09-13T18:10:36.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.577+0000 7f7658825700 1 -- start start 2024-09-13T18:10:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.577+0000 7f7658825700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7654153720 0x7f76540ca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.577+0000 7f7658825700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f76540cf9b0 0x7f765406c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.577+0000 7f7658825700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7654159e50 con 0x7f765406c5b0 2024-09-13T18:10:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.577+0000 7f7658825700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76540c5ea0 con 0x7f7654153720 2024-09-13T18:10:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.577+0000 7f7658825700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f765415d390 con 0x7f76540cf9b0 2024-09-13T18:10:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.577+0000 7f76537fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7654153720 0x7f76540ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.577+0000 7f7652ffd700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7f76540cf9b0 msgr2=0x7f765406c070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:10:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.577+0000 7f7653fff700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7f765406c5b0 legacy=0x7f765406aac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:10:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.577+0000 7f76537fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7654153720 0x7f76540ca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55412/0 (socket says 172.21.15.40:55412) 2024-09-13T18:10:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.577+0000 7f76537fe700 1 -- 172.21.15.40:0/3775192397 learned_addr learned my addr 172.21.15.40:0/3775192397 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:10:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.577+0000 7f7652ffd700 1 --2- 172.21.15.40:0/3775192397 >> v2:172.21.15.43:3300/0 conn(0x7f76540cf9b0 0x7f765406c070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:10:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.581+0000 7f76537fe700 1 -- 172.21.15.40:0/3775192397 >> v2:172.21.15.43:3300/0 conn(0x7f76540cf9b0 msgr2=0x7f765406c070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:10:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.581+0000 7f76537fe700 1 --2- 172.21.15.40:0/3775192397 >> v2:172.21.15.43:3300/0 conn(0x7f76540cf9b0 0x7f765406c070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.581+0000 7f76537fe700 1 -- 172.21.15.40:0/3775192397 >> v1:172.21.15.43:6789/0 conn(0x7f765406c5b0 legacy=0x7f765406aac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:10:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.581+0000 7f76537fe700 1 -- 172.21.15.40:0/3775192397 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765415a080 con 0x7f7654153720 2024-09-13T18:10:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.581+0000 7f76537fe700 1 --2- 172.21.15.40:0/3775192397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7654153720 0x7f76540ca510 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f76540b1b20 tx=0x7f764c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1390fdccddb613 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.581+0000 7f76527fc700 1 -- 172.21.15.40:0/3775192397 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c00e070 con 0x7f7654153720 2024-09-13T18:10:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.581+0000 7f76527fc700 1 -- 172.21.15.40:0/3775192397 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c003a40 con 0x7f7654153720 2024-09-13T18:10:36.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.581+0000 7f76527fc700 1 -- 172.21.15.40:0/3775192397 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c003df0 con 0x7f7654153720 2024-09-13T18:10:36.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.581+0000 7f7658825700 1 -- 172.21.15.40:0/3775192397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7654153720 msgr2=0x7f76540ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:36.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.581+0000 7f7658825700 1 --2- 172.21.15.40:0/3775192397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7654153720 0x7f76540ca510 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f76540b1b20 tx=0x7f764c00b330 comp rx=0 tx=0).stop 2024-09-13T18:10:36.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.581+0000 7f7658825700 1 -- 172.21.15.40:0/3775192397 shutdown_connections 2024-09-13T18:10:36.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.581+0000 7f7658825700 1 --2- 172.21.15.40:0/3775192397 >> v2:172.21.15.43:3300/0 conn(0x7f76540cf9b0 0x7f765406c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:36.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.581+0000 7f7658825700 1 --2- 172.21.15.40:0/3775192397 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7654153720 0x7f76540ca510 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:36.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.581+0000 7f7658825700 1 -- 172.21.15.40:0/3775192397 >> 172.21.15.40:0/3775192397 conn(0x7f76540666c0 msgr2=0x7f76540cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:36.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.581+0000 7f7658825700 1 -- 172.21.15.40:0/3775192397 shutdown_connections 2024-09-13T18:10:36.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.581+0000 7f7658825700 1 -- 172.21.15.40:0/3775192397 wait complete. 2024-09-13T18:10:36.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.581+0000 7f7658825700 1 Processor -- start 2024-09-13T18:10:36.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.581+0000 7f7658825700 1 -- start start 2024-09-13T18:10:36.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f7658825700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f765406c5b0 0x7f7654157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f7658825700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76540cf9b0 0x7f7654158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f7658825700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7654153720 0x7f765414ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f7658825700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7654157570 con 0x7f765406c5b0 2024-09-13T18:10:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f7658825700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76541f2f20 con 0x7f76540cf9b0 2024-09-13T18:10:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f7658825700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765414cac0 con 0x7f7654153720 2024-09-13T18:10:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f76537fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f765406c5b0 0x7f7654157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f7652ffd700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76540cf9b0 msgr2=0x7f7654158370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:10:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f76537fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f765406c5b0 0x7f7654157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55428/0 (socket says 172.21.15.40:55428) 2024-09-13T18:10:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f76537fe700 1 -- 172.21.15.40:0/368865207 learned_addr learned my addr 172.21.15.40:0/368865207 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:10:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f7652ffd700 1 --2- 172.21.15.40:0/368865207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76540cf9b0 0x7f7654158370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:10:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f7653fff700 1 --2- 172.21.15.40:0/368865207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7654153720 0x7f765414ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:36.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f76537fe700 1 -- 172.21.15.40:0/368865207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7654153720 msgr2=0x7f765414ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:36.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f76537fe700 1 --2- 172.21.15.40:0/368865207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7654153720 0x7f765414ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:36.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f76537fe700 1 -- 172.21.15.40:0/368865207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76540cf9b0 msgr2=0x7f7654158370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:10:36.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f76537fe700 1 --2- 172.21.15.40:0/368865207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76540cf9b0 0x7f7654158370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:36.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f76537fe700 1 -- 172.21.15.40:0/368865207 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c009b30 con 0x7f765406c5b0 2024-09-13T18:10:36.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f76537fe700 1 --2- 172.21.15.40:0/368865207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f765406c5b0 0x7f7654157030 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f764c000f80 tx=0x7f764c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:36.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f763bfff700 1 -- 172.21.15.40:0/368865207 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c0042c0 con 0x7f765406c5b0 2024-09-13T18:10:36.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f763bfff700 1 -- 172.21.15.40:0/368865207 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c004440 con 0x7f765406c5b0 2024-09-13T18:10:36.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f763bfff700 1 -- 172.21.15.40:0/368865207 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c0047f0 con 0x7f765406c5b0 2024-09-13T18:10:36.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f7658825700 1 -- 172.21.15.40:0/368865207 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76541f31c0 con 0x7f765406c5b0 2024-09-13T18:10:36.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f7658825700 1 -- 172.21.15.40:0/368865207 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76540b3c70 con 0x7f765406c5b0 2024-09-13T18:10:36.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f7658825700 1 --2- 172.21.15.40:0/368865207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76541590d0 0x7f765414bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:36.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.585+0000 7f7658825700 1 -- 172.21.15.40:0/368865207 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7654074750 con 0x7f76541590d0 2024-09-13T18:10:36.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.589+0000 7f7652ffd700 1 --2- 172.21.15.40:0/368865207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76541590d0 0x7f765414bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:36.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.589+0000 7f763bfff700 1 -- 172.21.15.40:0/368865207 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f764c019070 con 0x7f765406c5b0 2024-09-13T18:10:36.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.589+0000 7f7652ffd700 1 -- 172.21.15.40:0/368865207 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f765415ca60 con 0x7f76541590d0 2024-09-13T18:10:36.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.589+0000 7f763bfff700 1 --2- 172.21.15.40:0/368865207 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f76400666e0 0x7f7640068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:36.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.589+0000 7f763bfff700 1 -- 172.21.15.40:0/368865207 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f764c0855e0 con 0x7f765406c5b0 2024-09-13T18:10:36.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.589+0000 7f7652ffd700 1 --2- 172.21.15.40:0/368865207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76541590d0 0x7f765414bb10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f765415c990 tx=0x7f7648009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:36.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.589+0000 7f7653fff700 1 --2- 172.21.15.40:0/368865207 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f76400666e0 0x7f7640068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:36.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.589+0000 7f763bfff700 1 -- 172.21.15.40:0/368865207 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648003ab0 con 0x7f76541590d0 2024-09-13T18:10:36.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.589+0000 7f7653fff700 1 --2- 172.21.15.40:0/368865207 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f76400666e0 0x7f7640068ba0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f76440038f0 tx=0x7f7644008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:36.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.589+0000 7f763bfff700 1 -- 172.21.15.40:0/368865207 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7648003c10 con 0x7f76541590d0 2024-09-13T18:10:36.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.589+0000 7f763bfff700 1 -- 172.21.15.40:0/368865207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76541590d0 msgr2=0x7f765414bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:36.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.589+0000 7f763bfff700 1 --2- 172.21.15.40:0/368865207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76541590d0 0x7f765414bb10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f765415c990 tx=0x7f7648009f30 comp rx=0 tx=0).stop 2024-09-13T18:10:36.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.589+0000 7f763bfff700 1 -- 172.21.15.40:0/368865207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76541590d0 msgr2=0x7f765414bb10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:10:36.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.589+0000 7f763bfff700 1 --2- 172.21.15.40:0/368865207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76541590d0 0x7f765414bb10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:36.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.601+0000 7f7658825700 1 --2- 172.21.15.40:0/368865207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76540b41a0 0x7f765414c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:10:36.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.601+0000 7f7658825700 1 -- 172.21.15.40:0/368865207 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7654153b00 con 0x7f76540b41a0 2024-09-13T18:10:36.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.601+0000 7f7652ffd700 1 --2- 172.21.15.40:0/368865207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76540b41a0 0x7f765414c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:10:36.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.601+0000 7f7652ffd700 1 -- 172.21.15.40:0/368865207 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f765415cd40 con 0x7f76540b41a0 2024-09-13T18:10:36.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.601+0000 7f7652ffd700 1 --2- 172.21.15.40:0/368865207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76540b41a0 0x7f765414c050 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f76540b3920 tx=0x7f7648005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:10:36.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.601+0000 7f763bfff700 1 -- 172.21.15.40:0/368865207 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764801a990 con 0x7f76540b41a0 2024-09-13T18:10:36.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.601+0000 7f763bfff700 1 -- 172.21.15.40:0/368865207 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4937 (secure 0 0 0) 0x7f764801aaf0 con 0x7f76540b41a0 2024-09-13T18:10:36.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.601+0000 7f763bfff700 1 -- 172.21.15.40:0/368865207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76540b41a0 msgr2=0x7f765414c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:36.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.601+0000 7f763bfff700 1 --2- 172.21.15.40:0/368865207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76540b41a0 0x7f765414c050 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f76540b3920 tx=0x7f7648005ad0 comp rx=0 tx=0).stop 2024-09-13T18:10:36.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.601+0000 7f763bfff700 1 -- 172.21.15.40:0/368865207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76540b41a0 msgr2=0x7f765414c050 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:10:36.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.601+0000 7f763bfff700 1 --2- 172.21.15.40:0/368865207 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f76540b41a0 0x7f765414c050 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:10:36.611 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:10:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:10:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:10:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:10:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 160, 2024-09-13T18:10:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:10:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:10:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:10:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:10:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:10:36.611 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:10:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:10:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:10:36.612 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:10:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:10:36.613 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:10:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:10:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:10:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:10:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:10:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:10:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:10:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:10:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:10:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:10:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:10:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:10:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:10:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:10:36.614 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:10:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:10:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:10:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:10:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:10:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:10:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:10:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:10:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:10:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:10:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:10:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:10:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:10:36.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:10:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:10:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:10:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:10:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:36.616 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:10:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:10:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:10:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:10:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:10:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:36.617 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:36.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.618 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:36.618 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:10:36.618 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:36.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.618 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:36.618 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:10:36.618 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:36.618 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:36.618 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:36.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.618 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:10:36.618 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:10:36.619 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:36.619 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:36.619 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:36.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.619 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:10:36.619 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:10:36.619 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:10:36.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:10:36.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.619 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:10:36.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:10:36.619 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:36.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.619 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.620 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:10:36.620 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:10:36.620 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:10:36.620 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:36.620 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:36.620 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.620 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:10:36.620 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:10:36.620 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:10:36.620 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:10:36.620 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:10:36.620 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:36.620 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:36.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.621 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:10:36.621 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:10:36.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.621 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:36.621 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:36.621 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:10:36.621 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:36.621 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:36.621 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:10:36.621 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.621 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:36.621 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:36.621 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:10:36.622 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:36.622 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:36.622 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:10:36.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.622 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:36.622 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:36.622 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:10:36.622 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:36.622 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:36.622 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:10:36.622 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.622 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:36.622 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:36.622 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:10:36.622 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:36.623 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:10:36.623 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:10:36.623 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:10:36.623 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:10:36.623 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:10:36.623 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:10:36.623 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:10:36.623 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:10:36.623 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:10:36.623 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:10:36.623 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:10:36.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.605+0000 7f7658825700 1 -- 172.21.15.40:0/368865207 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f76400666e0 msgr2=0x7f7640068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:36.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.605+0000 7f7658825700 1 --2- 172.21.15.40:0/368865207 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f76400666e0 0x7f7640068ba0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f76440038f0 tx=0x7f7644008040 comp rx=0 tx=0).stop 2024-09-13T18:10:36.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.605+0000 7f7658825700 1 -- 172.21.15.40:0/368865207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f765406c5b0 msgr2=0x7f7654157030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:10:36.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.605+0000 7f7658825700 1 --2- 172.21.15.40:0/368865207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f765406c5b0 0x7f7654157030 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f764c000f80 tx=0x7f764c004af0 comp rx=0 tx=0).stop 2024-09-13T18:10:36.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.605+0000 7f7653fff700 1 -- 172.21.15.40:0/368865207 reap_dead start 2024-09-13T18:10:36.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.605+0000 7f7653fff700 1 -- 172.21.15.40:0/368865207 reap_dead start 2024-09-13T18:10:36.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.605+0000 7f7658825700 1 -- 172.21.15.40:0/368865207 shutdown_connections 2024-09-13T18:10:36.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.605+0000 7f7658825700 1 -- 172.21.15.40:0/368865207 >> 172.21.15.40:0/368865207 conn(0x7f76540666c0 msgr2=0x7f76540c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:10:36.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.605+0000 7f7658825700 1 -- 172.21.15.40:0/368865207 shutdown_connections 2024-09-13T18:10:36.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:10:36.605+0000 7f7658825700 1 -- 172.21.15.40:0/368865207 wait complete. 2024-09-13T18:10:36.627 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:10:36.627 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:10:37.447 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:10:37.449 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:10:56.628 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-13T18:10:56.628 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-13T18:10:56.628 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T18:10:56.628 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T18:10:56.631 INFO:tasks.ceph.mon.b:Started 2024-09-13T18:10:56.631 INFO:tasks.mon_thrash:Sending STOP to mon c 2024-09-13T18:10:56.632 INFO:tasks.ceph.mon.c:Sent signal 19 2024-09-13T18:10:56.632 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-13T18:11:07.497 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:11:07.499 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:11:11.633 INFO:tasks.mon_thrash:Sending CONT to mon c 2024-09-13T18:11:11.633 INFO:tasks.ceph.mon.c:Sent signal 18 2024-09-13T18:11:11.634 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:11:11.634 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:11:11.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:11.740+0000 7f727ffff700 1 Processor -- start 2024-09-13T18:11:11.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:11.740+0000 7f727ffff700 1 -- start start 2024-09-13T18:11:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:11.740+0000 7f727ffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 0x7f7280070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:11.744+0000 7f727ffff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f728014fdd0 con 0x7f728014f9f0 2024-09-13T18:11:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:11.744+0000 7f727ffff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72801558b0 con 0x7f7280071620 2024-09-13T18:11:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:11.744+0000 7f727ffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7280070920 con 0x7f728006ea60 2024-09-13T18:11:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:11.744+0000 7f727effd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f728014f9f0 0x7f7280070ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49356/0 (socket says 172.21.15.40:49356) 2024-09-13T18:11:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:11.744+0000 7f727effd700 1 -- 172.21.15.40:0/1213283344 learned_addr learned my addr 172.21.15.40:0/1213283344 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:11:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:11.744+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 0x7f7280070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:11.744+0000 7f727e7fc700 1 -- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 msgr2=0x7f7280070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:11:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:11.744+0000 7f727e7fc700 1 -- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 msgr2=0x7f7280070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:11:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:11.744+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 0x7f7280070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:11:11.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:11.744+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 0x7f7280070290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:11:11.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:11.948+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 0x7f7280070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:11.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:11.948+0000 7f727e7fc700 1 -- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 msgr2=0x7f7280070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:11:11.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:11.948+0000 7f727e7fc700 1 -- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 msgr2=0x7f7280070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:11:11.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:11.948+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 0x7f7280070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:11:11.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:11.948+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 0x7f7280070290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:11:12.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:12.352+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 0x7f7280070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:12.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:12.352+0000 7f727e7fc700 1 -- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 msgr2=0x7f7280070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:11:12.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:12.352+0000 7f727e7fc700 1 -- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 msgr2=0x7f7280070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:11:12.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:12.352+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 0x7f7280070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:11:12.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:12.352+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 0x7f7280070290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:11:13.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:13.156+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 0x7f7280070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:13.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:13.156+0000 7f727e7fc700 1 -- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 msgr2=0x7f7280070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:11:13.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:13.156+0000 7f727e7fc700 1 -- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 msgr2=0x7f7280070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:11:13.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:13.156+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 0x7f7280070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:11:13.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:13.156+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 0x7f7280070290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:11:14.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.744+0000 7f727cff9700 1 -- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 msgr2=0x7f7280070290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:11:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.744+0000 7f727cff9700 1 --2- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f728006ea60 0x7f7280070290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.744+0000 7f727cff9700 1 -- 172.21.15.40:0/1213283344 >> v1:172.21.15.43:6789/0 conn(0x7f7280071620 legacy=0x7f728006d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.744+0000 7f727cff9700 1 -- 172.21.15.40:0/1213283344 >> v1:172.21.15.40:6789/0 conn(0x7f728014f9f0 legacy=0x7f7280070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.744+0000 7f727cff9700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 0x7f726c005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.744+0000 7f727cff9700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 0x7f726c0086c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.744+0000 7f727cff9700 1 -- 172.21.15.40:0/1213283344 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7280070920 con 0x7f726c006220 2024-09-13T18:11:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.744+0000 7f727cff9700 1 -- 172.21.15.40:0/1213283344 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f726c008d70 con 0x7f726c003770 2024-09-13T18:11:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.744+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 0x7f726c0086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.744+0000 7f727effd700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 0x7f726c005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.744+0000 7f727e7fc700 1 -- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 msgr2=0x7f726c0086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:11:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.744+0000 7f727e7fc700 1 -- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 msgr2=0x7f726c0086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:11:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.744+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 0x7f726c0086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:11:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.744+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 0x7f726c0086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:11:14.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.744+0000 7f727effd700 1 -- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 msgr2=0x7f726c005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:11:14.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.744+0000 7f727effd700 1 -- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 msgr2=0x7f726c005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:11:14.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.744+0000 7f727effd700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 0x7f726c005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:11:14.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.744+0000 7f727effd700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 0x7f726c005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:11:14.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.948+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 0x7f726c0086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:14.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.948+0000 7f727effd700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 0x7f726c005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.948+0000 7f727e7fc700 1 -- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 msgr2=0x7f726c0086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:11:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.948+0000 7f727e7fc700 1 -- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 msgr2=0x7f726c0086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:11:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.948+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 0x7f726c0086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:11:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.948+0000 7f727effd700 1 -- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 msgr2=0x7f726c005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:11:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.948+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 0x7f726c0086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:11:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.948+0000 7f727effd700 1 -- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 msgr2=0x7f726c005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:11:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.948+0000 7f727effd700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 0x7f726c005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:11:14.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:14.948+0000 7f727effd700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 0x7f726c005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:11:15.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:15.352+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 0x7f726c0086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:15.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:15.352+0000 7f727effd700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 0x7f726c005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:15.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:15.352+0000 7f727e7fc700 1 -- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 msgr2=0x7f726c0086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:11:15.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:15.352+0000 7f727e7fc700 1 -- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 msgr2=0x7f726c0086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:11:15.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:15.352+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 0x7f726c0086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:11:15.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:15.352+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 0x7f726c0086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:11:15.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:15.352+0000 7f727effd700 1 -- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 msgr2=0x7f726c005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:11:15.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:15.352+0000 7f727effd700 1 -- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 msgr2=0x7f726c005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:11:15.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:15.352+0000 7f727effd700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 0x7f726c005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:11:15.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:15.352+0000 7f727effd700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 0x7f726c005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:11:16.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:16.156+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 0x7f726c0086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:16.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:16.156+0000 7f727effd700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 0x7f726c005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:16.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:16.156+0000 7f727e7fc700 1 -- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 msgr2=0x7f726c0086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:11:16.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:16.156+0000 7f727e7fc700 1 -- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 msgr2=0x7f726c0086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:11:16.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:16.156+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 0x7f726c0086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:11:16.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:16.156+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 0x7f726c0086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:11:16.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:16.156+0000 7f727effd700 1 -- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 msgr2=0x7f726c005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:11:16.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:16.156+0000 7f727effd700 1 -- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 msgr2=0x7f726c005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:11:16.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:16.156+0000 7f727effd700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 0x7f726c005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:11:16.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:16.156+0000 7f727effd700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 0x7f726c005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:11:17.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727cff9700 1 -- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 msgr2=0x7f726c005bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:11:17.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727cff9700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003770 0x7f726c005bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:17.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727cff9700 1 -- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 msgr2=0x7f726c0086c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:11:17.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727cff9700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c006220 0x7f726c0086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:17.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727f7fe700 1 -- 172.21.15.40:0/1213283344 reap_dead start 2024-09-13T18:11:17.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727cff9700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003be0 0x7f726c006600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:17.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727cff9700 1 --2- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f726c006b40 0x7f726c006fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:17.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727cff9700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c007530 0x7f726c0079e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:17.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727cff9700 1 -- 172.21.15.40:0/1213283344 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7280070920 con 0x7f726c007530 2024-09-13T18:11:17.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727cff9700 1 -- 172.21.15.40:0/1213283344 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f726c008d70 con 0x7f726c003be0 2024-09-13T18:11:17.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727cff9700 1 -- 172.21.15.40:0/1213283344 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726c0081c0 con 0x7f726c006b40 2024-09-13T18:11:17.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f726c006b40 0x7f726c006fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:17.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727effd700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003be0 0x7f726c006600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:17.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727f7fe700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c007530 0x7f726c0079e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:17.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727e7fc700 1 -- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003be0 msgr2=0x7f726c006600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:11:17.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003be0 0x7f726c006600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:17.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727e7fc700 1 -- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c007530 msgr2=0x7f726c0079e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:11:17.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c007530 0x7f726c0079e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:17.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727e7fc700 1 -- 172.21.15.40:0/1213283344 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7280155b50 con 0x7f726c006b40 2024-09-13T18:11:17.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727f7fe700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c007530 0x7f726c0079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:11:17.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727e7fc700 1 --2- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f726c006b40 0x7f726c006fc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f728006e7f0 tx=0x7f7274014f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d21c07ed942c3b29 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:17.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727dffb700 1 -- 172.21.15.40:0/1213283344 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274013b60 con 0x7f726c006b40 2024-09-13T18:11:17.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727dffb700 1 -- 172.21.15.40:0/1213283344 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7274013ce0 con 0x7f726c006b40 2024-09-13T18:11:17.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.744+0000 7f727dffb700 1 -- 172.21.15.40:0/1213283344 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727401f690 con 0x7f726c006b40 2024-09-13T18:11:17.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.748+0000 7f727ffff700 1 -- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f726c006b40 msgr2=0x7f726c006fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:17.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.748+0000 7f727ffff700 1 --2- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f726c006b40 0x7f726c006fc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f728006e7f0 tx=0x7f7274014f30 comp rx=0 tx=0).stop 2024-09-13T18:11:17.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.748+0000 7f727ffff700 1 -- 172.21.15.40:0/1213283344 shutdown_connections 2024-09-13T18:11:17.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.748+0000 7f727ffff700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.40:3300/0 conn(0x7f726c007530 0x7f726c0079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:17.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.748+0000 7f727ffff700 1 --2- 172.21.15.40:0/1213283344 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f726c006b40 0x7f726c006fc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:17.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.748+0000 7f727ffff700 1 --2- 172.21.15.40:0/1213283344 >> v2:172.21.15.43:3300/0 conn(0x7f726c003be0 0x7f726c006600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:17.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.748+0000 7f727ffff700 1 -- 172.21.15.40:0/1213283344 >> 172.21.15.40:0/1213283344 conn(0x7f728006a390 msgr2=0x7f728006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:17.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.748+0000 7f727ffff700 1 -- 172.21.15.40:0/1213283344 shutdown_connections 2024-09-13T18:11:17.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.748+0000 7f727ffff700 1 -- 172.21.15.40:0/1213283344 wait complete. 2024-09-13T18:11:17.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f727ffff700 1 Processor -- start 2024-09-13T18:11:17.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f727ffff700 1 -- start start 2024-09-13T18:11:17.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f727ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f726c003be0 0x7f72800c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:17.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f727ffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f726c006b40 0x7f7280152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:17.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f727ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f726c007530 0x7f72801f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:17.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f727ffff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72801564d0 con 0x7f726c007530 2024-09-13T18:11:17.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f727ffff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7280150640 con 0x7f726c003be0 2024-09-13T18:11:17.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f727ffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7280150360 con 0x7f726c006b40 2024-09-13T18:11:17.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f727f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f726c007530 0x7f72801f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:17.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f727effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f726c003be0 0x7f72800c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f727e7fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f726c006b40 0x7f7280152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f727e7fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f726c006b40 0x7f7280152690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:38036/0 (socket says 172.21.15.40:38036) 2024-09-13T18:11:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f727e7fc700 1 -- 172.21.15.40:0/977329464 learned_addr learned my addr 172.21.15.40:0/977329464 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f727e7fc700 1 -- 172.21.15.40:0/977329464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f726c003be0 msgr2=0x7f72800c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f727e7fc700 1 --2- 172.21.15.40:0/977329464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f726c003be0 0x7f72800c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f727e7fc700 1 -- 172.21.15.40:0/977329464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f726c007530 msgr2=0x7f72801f2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f727e7fc700 1 --2- 172.21.15.40:0/977329464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f726c007530 0x7f72801f2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f727e7fc700 1 -- 172.21.15.40:0/977329464 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72740148b0 con 0x7f726c006b40 2024-09-13T18:11:17.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f727e7fc700 1 --2- 172.21.15.40:0/977329464 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f726c006b40 0x7f7280152690 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7274022040 tx=0x7f7274004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:17.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f72677fe700 1 -- 172.21.15.40:0/977329464 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274006cc0 con 0x7f726c006b40 2024-09-13T18:11:17.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f72677fe700 1 -- 172.21.15.40:0/977329464 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727401fd20 con 0x7f726c006b40 2024-09-13T18:11:17.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.752+0000 7f727ffff700 1 -- 172.21.15.40:0/977329464 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72801eeec0 con 0x7f726c006b40 2024-09-13T18:11:17.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.756+0000 7f72677fe700 1 -- 172.21.15.40:0/977329464 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72740336a0 con 0x7f726c006b40 2024-09-13T18:11:17.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.756+0000 7f727ffff700 1 -- 172.21.15.40:0/977329464 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72801ef480 con 0x7f726c006b40 2024-09-13T18:11:17.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.756+0000 7f727ffff700 1 -- 172.21.15.40:0/977329464 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7280066090 con 0x7f726c006b40 2024-09-13T18:11:17.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.756+0000 7f72677fe700 1 -- 172.21.15.40:0/977329464 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7274026070 con 0x7f726c006b40 2024-09-13T18:11:17.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.756+0000 7f72677fe700 1 --2- 172.21.15.40:0/977329464 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f726c0684f0 0x7f726c06a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:17.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.756+0000 7f72677fe700 1 -- 172.21.15.40:0/977329464 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7274086ed0 con 0x7f726c006b40 2024-09-13T18:11:17.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.756+0000 7f727effd700 1 --2- 172.21.15.40:0/977329464 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f726c0684f0 0x7f726c06a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:17.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.756+0000 7f727effd700 1 --2- 172.21.15.40:0/977329464 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f726c0684f0 0x7f726c06a9b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7270007dd0 tx=0x7f7270009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:17.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:17.768+0000 7f72677fe700 1 -- 172.21.15.40:0/977329464 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f72740579a0 con 0x7f726c006b40 2024-09-13T18:11:18.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.024+0000 7f727ffff700 1 -- 172.21.15.40:0/977329464 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f72800c7930 con 0x7f726c006b40 2024-09-13T18:11:18.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.024+0000 7f72677fe700 1 -- 172.21.15.40:0/977329464 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f7274057530 con 0x7f726c006b40 2024-09-13T18:11:18.031 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":166,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:11:18.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.024+0000 7f727ffff700 1 -- 172.21.15.40:0/977329464 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f726c0684f0 msgr2=0x7f726c06a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.024+0000 7f727ffff700 1 --2- 172.21.15.40:0/977329464 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f726c0684f0 0x7f726c06a9b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7270007dd0 tx=0x7f7270009040 comp rx=0 tx=0).stop 2024-09-13T18:11:18.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.024+0000 7f727ffff700 1 -- 172.21.15.40:0/977329464 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f726c006b40 msgr2=0x7f7280152690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.024+0000 7f727ffff700 1 --2- 172.21.15.40:0/977329464 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f726c006b40 0x7f7280152690 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7274022040 tx=0x7f7274004770 comp rx=0 tx=0).stop 2024-09-13T18:11:18.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.028+0000 7f727ffff700 1 -- 172.21.15.40:0/977329464 shutdown_connections 2024-09-13T18:11:18.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.028+0000 7f727ffff700 1 --2- 172.21.15.40:0/977329464 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f726c007530 0x7f72801f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.028+0000 7f727ffff700 1 --2- 172.21.15.40:0/977329464 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f726c0684f0 0x7f726c06a9b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.028+0000 7f727ffff700 1 --2- 172.21.15.40:0/977329464 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f726c006b40 0x7f7280152690 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.028+0000 7f727ffff700 1 --2- 172.21.15.40:0/977329464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f726c003be0 0x7f72800c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.028+0000 7f727ffff700 1 -- 172.21.15.40:0/977329464 >> 172.21.15.40:0/977329464 conn(0x7f728006a390 msgr2=0x7f728006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:18.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.028+0000 7f727ffff700 1 -- 172.21.15.40:0/977329464 shutdown_connections 2024-09-13T18:11:18.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.028+0000 7f727ffff700 1 -- 172.21.15.40:0/977329464 wait complete. 2024-09-13T18:11:18.047 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:11:18.047 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:11:18.047 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:11:18.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.148+0000 7fa1ee3ee700 1 Processor -- start 2024-09-13T18:11:18.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.148+0000 7fa1ee3ee700 1 -- start start 2024-09-13T18:11:18.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1ee3ee700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa1e814fa30 0x7fa1e8158e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1ee3ee700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e8159ca0 con 0x7fa1e80c6c40 2024-09-13T18:11:18.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1ee3ee700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e8159400 con 0x7fa1e814fe10 2024-09-13T18:11:18.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1ee3ee700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e80c7090 con 0x7fa1e814fa30 2024-09-13T18:11:18.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1edbed700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa1e80c6c40 0x7fa1e80c7bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49372/0 (socket says 172.21.15.40:49372) 2024-09-13T18:11:18.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1edbed700 1 -- 172.21.15.40:0/1090880683 learned_addr learned my addr 172.21.15.40:0/1090880683 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:11:18.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1ed3ec700 1 --2- 172.21.15.40:0/1090880683 >> v2:172.21.15.43:3300/0 conn(0x7fa1e814fa30 0x7fa1e8158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1dffff700 1 -- 172.21.15.40:0/1090880683 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3346539136 0 0) 0x7fa1e8159ca0 con 0x7fa1e80c6c40 2024-09-13T18:11:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1dffff700 1 -- 172.21.15.40:0/1090880683 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1d40046e0 con 0x7fa1e80c6c40 2024-09-13T18:11:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1dffff700 1 -- 172.21.15.40:0/1090880683 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 829844118 0 0) 0x7fa1d40046e0 con 0x7fa1e80c6c40 2024-09-13T18:11:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1dffff700 1 -- 172.21.15.40:0/1090880683 >> v2:172.21.15.43:3300/0 conn(0x7fa1e814fa30 msgr2=0x7fa1e8158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:11:18.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1dffff700 1 --2- 172.21.15.40:0/1090880683 >> v2:172.21.15.43:3300/0 conn(0x7fa1e814fa30 0x7fa1e8158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1dffff700 1 -- 172.21.15.40:0/1090880683 >> v1:172.21.15.43:6789/0 conn(0x7fa1e814fe10 legacy=0x7fa1e81561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1dffff700 1 -- 172.21.15.40:0/1090880683 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e81596e0 con 0x7fa1e80c6c40 2024-09-13T18:11:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1dffff700 1 -- 172.21.15.40:0/1090880683 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fa1e00019f0 con 0x7fa1e80c6c40 2024-09-13T18:11:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1dffff700 1 -- 172.21.15.40:0/1090880683 >> v1:172.21.15.40:6789/0 conn(0x7fa1e80c6c40 legacy=0x7fa1e80c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1dffff700 1 --2- 172.21.15.40:0/1090880683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1d4004b30 0x7fa1d4004f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1dffff700 1 --2- 172.21.15.40:0/1090880683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1d4005490 0x7fa1d4005980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1dffff700 1 --2- 172.21.15.40:0/1090880683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1d40060c0 0x7fa1d40035d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1dffff700 1 -- 172.21.15.40:0/1090880683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e80c7090 con 0x7fa1d40060c0 2024-09-13T18:11:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1dffff700 1 -- 172.21.15.40:0/1090880683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d4003b10 con 0x7fa1d4004b30 2024-09-13T18:11:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1dffff700 1 -- 172.21.15.40:0/1090880683 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d4003cd0 con 0x7fa1d4005490 2024-09-13T18:11:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1edbed700 1 --2- 172.21.15.40:0/1090880683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1d40060c0 0x7fa1d40035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1ed3ec700 1 --2- 172.21.15.40:0/1090880683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1d4004b30 0x7fa1d4004f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1ecbeb700 1 --2- 172.21.15.40:0/1090880683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1d4005490 0x7fa1d4005980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1edbed700 1 -- 172.21.15.40:0/1090880683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1d4005490 msgr2=0x7fa1d4005980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:11:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1edbed700 1 --2- 172.21.15.40:0/1090880683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1d4005490 0x7fa1d4005980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1edbed700 1 -- 172.21.15.40:0/1090880683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1d4004b30 msgr2=0x7fa1d4004f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:11:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1edbed700 1 --2- 172.21.15.40:0/1090880683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1d4004b30 0x7fa1d4004f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1edbed700 1 -- 172.21.15.40:0/1090880683 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d40046e0 con 0x7fa1d40060c0 2024-09-13T18:11:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1ed3ec700 1 --2- 172.21.15.40:0/1090880683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1d4004b30 0x7fa1d4004f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:11:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1edbed700 1 -- 172.21.15.40:0/1090880683 reap_dead start 2024-09-13T18:11:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.152+0000 7fa1edbed700 1 --2- 172.21.15.40:0/1090880683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1d40060c0 0x7fa1d40035d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000a830 tx=0x7fa1d40069b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb456abf27c7a7c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.156+0000 7fa1dffff700 1 -- 172.21.15.40:0/1090880683 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e000a2f0 con 0x7fa1d40060c0 2024-09-13T18:11:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.156+0000 7fa1dffff700 1 -- 172.21.15.40:0/1090880683 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e00072f0 con 0x7fa1d40060c0 2024-09-13T18:11:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.156+0000 7fa1dffff700 1 -- 172.21.15.40:0/1090880683 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e00076a0 con 0x7fa1d40060c0 2024-09-13T18:11:18.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.156+0000 7fa1ee3ee700 1 -- 172.21.15.40:0/1090880683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1d40060c0 msgr2=0x7fa1d40035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.156+0000 7fa1ee3ee700 1 --2- 172.21.15.40:0/1090880683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1d40060c0 0x7fa1d40035d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000a830 tx=0x7fa1d40069b0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.156+0000 7fa1ee3ee700 1 -- 172.21.15.40:0/1090880683 shutdown_connections 2024-09-13T18:11:18.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.156+0000 7fa1ee3ee700 1 --2- 172.21.15.40:0/1090880683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1d40060c0 0x7fa1d40035d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.156+0000 7fa1ee3ee700 1 -- 172.21.15.40:0/1090880683 >> 172.21.15.40:0/1090880683 conn(0x7fa1e806a390 msgr2=0x7fa1e806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:18.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.156+0000 7fa1ee3ee700 1 -- 172.21.15.40:0/1090880683 shutdown_connections 2024-09-13T18:11:18.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.156+0000 7fa1ee3ee700 1 -- 172.21.15.40:0/1090880683 wait complete. 2024-09-13T18:11:18.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.156+0000 7fa1ee3ee700 1 Processor -- start 2024-09-13T18:11:18.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.156+0000 7fa1ee3ee700 1 -- start start 2024-09-13T18:11:18.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1ee3ee700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1d40060c0 0x7fa1e8151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1ee3ee700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e81eda70 0x7fa1e8150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1ee3ee700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8152f90 0x7fa1e80c4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1ee3ee700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e81506d0 con 0x7fa1e8152f90 2024-09-13T18:11:18.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1ee3ee700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8152280 con 0x7fa1e81eda70 2024-09-13T18:11:18.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1ee3ee700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8151030 con 0x7fa1d40060c0 2024-09-13T18:11:18.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1edbed700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8152f90 0x7fa1e80c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1edbed700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8152f90 0x7fa1e80c4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35560/0 (socket says 172.21.15.40:35560) 2024-09-13T18:11:18.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1edbed700 1 -- 172.21.15.40:0/330937982 learned_addr learned my addr 172.21.15.40:0/330937982 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:18.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1ed3ec700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1d40060c0 0x7fa1e8151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1ecbeb700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e81eda70 0x7fa1e8150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1edbed700 1 -- 172.21.15.40:0/330937982 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1d40060c0 msgr2=0x7fa1e8151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1edbed700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1d40060c0 0x7fa1e8151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1edbed700 1 -- 172.21.15.40:0/330937982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e81eda70 msgr2=0x7fa1e8150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1edbed700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e81eda70 0x7fa1e8150af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1edbed700 1 -- 172.21.15.40:0/330937982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e80ca470 con 0x7fa1e8152f90 2024-09-13T18:11:18.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1ecbeb700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1e81eda70 0x7fa1e8150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:11:18.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1edbed700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8152f90 0x7fa1e80c4260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8073e90 tx=0x7fa1e0001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:18.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1ddffb700 1 -- 172.21.15.40:0/330937982 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0001250 con 0x7fa1e8152f90 2024-09-13T18:11:18.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1ddffb700 1 -- 172.21.15.40:0/330937982 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e0007670 con 0x7fa1e8152f90 2024-09-13T18:11:18.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1ee3ee700 1 -- 172.21.15.40:0/330937982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e80c9c10 con 0x7fa1e8152f90 2024-09-13T18:11:18.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1ddffb700 1 -- 172.21.15.40:0/330937982 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0007a20 con 0x7fa1e8152f90 2024-09-13T18:11:18.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1ee3ee700 1 -- 172.21.15.40:0/330937982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e80c9eb0 con 0x7fa1e8152f90 2024-09-13T18:11:18.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1ee3ee700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e814cf70 0x7fa1e814d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1ee3ee700 1 -- 172.21.15.40:0/330937982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8067600 con 0x7fa1e814cf70 2024-09-13T18:11:18.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.160+0000 7fa1ed3ec700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e814cf70 0x7fa1e814d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.164+0000 7fa1ed3ec700 1 -- 172.21.15.40:0/330937982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa1e81559a0 con 0x7fa1e814cf70 2024-09-13T18:11:18.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.164+0000 7fa1ddffb700 1 -- 172.21.15.40:0/330937982 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa1e001a070 con 0x7fa1e8152f90 2024-09-13T18:11:18.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.164+0000 7fa1ed3ec700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e814cf70 0x7fa1e814d350 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4013910 tx=0x7fa1e4013cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:18.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.164+0000 7fa1ddffb700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1cc066730 0x7fa1cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.164+0000 7fa1ddffb700 1 -- 172.21.15.40:0/330937982 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa1e0085b70 con 0x7fa1e8152f90 2024-09-13T18:11:18.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.164+0000 7fa1ecbeb700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1cc066730 0x7fa1cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.164+0000 7fa1ddffb700 1 -- 172.21.15.40:0/330937982 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4011940 con 0x7fa1e814cf70 2024-09-13T18:11:18.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.164+0000 7fa1ddffb700 1 -- 172.21.15.40:0/330937982 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa1e4011aa0 con 0x7fa1e814cf70 2024-09-13T18:11:18.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.164+0000 7fa1ddffb700 1 -- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e814cf70 msgr2=0x7fa1e814d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.164+0000 7fa1ddffb700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e814cf70 0x7fa1e814d350 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4013910 tx=0x7fa1e4013cd0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.164+0000 7fa1ddffb700 1 -- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e814cf70 msgr2=0x7fa1e814d350 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:11:18.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.164+0000 7fa1ddffb700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e814cf70 0x7fa1e814d350 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.164+0000 7fa1ecbeb700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1cc066730 0x7fa1cc068bf0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa1e81589a0 tx=0x7fa1d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:18.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.172+0000 7fa1ee3ee700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e814d890 0x7fa1e8157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.172+0000 7fa1ee3ee700 1 -- 172.21.15.40:0/330937982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8072100 con 0x7fa1e814d890 2024-09-13T18:11:18.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.172+0000 7fa1ed3ec700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e814d890 0x7fa1e8157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.172+0000 7fa1ed3ec700 1 -- 172.21.15.40:0/330937982 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa1e8155c80 con 0x7fa1e814d890 2024-09-13T18:11:18.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.172+0000 7fa1ed3ec700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e814d890 0x7fa1e8157160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4013910 tx=0x7fa1e40047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:18.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.172+0000 7fa1ddffb700 1 -- 172.21.15.40:0/330937982 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40247a0 con 0x7fa1e814d890 2024-09-13T18:11:18.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.172+0000 7fa1ddffb700 1 -- 172.21.15.40:0/330937982 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4784 (secure 0 0 0) 0x7fa1e4024900 con 0x7fa1e814d890 2024-09-13T18:11:18.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.172+0000 7fa1ddffb700 1 -- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e814d890 msgr2=0x7fa1e8157160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.172+0000 7fa1ddffb700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e814d890 0x7fa1e8157160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4013910 tx=0x7fa1e40047e0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.172+0000 7fa1ddffb700 1 -- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e814d890 msgr2=0x7fa1e8157160 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:11:18.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.172+0000 7fa1ddffb700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e814d890 0x7fa1e8157160 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.183 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:11:18.183 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:11:18.183 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:11:18.183 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:11:18.183 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 166, 2024-09-13T18:11:18.183 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:11:18.183 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:11:18.183 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:11:18.184 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:11:18.184 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:18.184 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:11:18.184 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:11:18.184 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:11:18.184 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:11:18.184 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:18.184 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:18.184 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:18.184 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:18.184 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:18.184 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:18.184 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:18.185 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:18.185 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:18.185 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:18.185 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:18.185 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:11:18.185 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:11:18.185 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:18.185 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:18.185 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:18.185 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:18.185 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:18.186 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:18.186 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:18.186 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:18.186 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:18.186 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:18.186 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:18.186 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.186 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:11:18.187 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:11:18.187 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:11:18.187 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:11:18.187 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:11:18.187 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:11:18.187 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:11:18.187 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:11:18.187 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:11:18.187 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:11:18.187 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:11:18.187 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:11:18.188 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:11:18.188 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:11:18.188 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:11:18.188 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:11:18.188 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:11:18.188 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:18.188 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:18.188 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:18.189 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:18.189 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:18.189 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:18.189 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:18.189 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:18.189 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:18.189 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:18.189 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:18.189 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:11:18.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.189 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:11:18.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.190 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:11:18.190 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:11:18.190 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:18.190 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:18.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.190 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:18.190 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:11:18.190 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:18.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.190 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:18.190 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:11:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:11:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:11:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:11:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:11:18.191 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:18.192 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:18.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.192 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:18.192 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:11:18.192 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:18.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.192 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.192 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:18.192 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:11:18.192 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:18.192 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.193 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:18.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.193 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:11:18.193 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:11:18.193 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:18.193 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:18.193 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:18.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.193 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:11:18.193 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:11:18.193 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:18.193 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:18.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.194 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:18.194 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:11:18.194 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:18.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.194 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:18.194 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:11:18.194 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:18.194 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.195 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:18.195 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.195 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:11:18.195 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:11:18.195 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:18.195 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:18.195 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:18.195 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.195 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:18.195 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.195 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:11:18.195 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:11:18.195 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.196 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:18.196 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:18.196 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:11:18.196 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.196 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:18.196 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:11:18.196 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.196 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:18.196 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:18.196 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:11:18.196 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.196 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:18.196 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:11:18.197 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.197 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:18.197 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:18.197 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:11:18.197 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.197 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:18.197 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:11:18.197 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.197 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:18.197 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:18.197 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:11:18.197 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.198 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:18.198 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.198 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:11:18.198 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:11:18.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.176+0000 7fa1ee3ee700 1 -- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1cc066730 msgr2=0x7fa1cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.176+0000 7fa1ee3ee700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1cc066730 0x7fa1cc068bf0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa1e81589a0 tx=0x7fa1d8009040 comp rx=0 tx=0).stop 2024-09-13T18:11:18.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.176+0000 7fa1ee3ee700 1 -- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8152f90 msgr2=0x7fa1e80c4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.176+0000 7fa1edbed700 1 -- 172.21.15.40:0/330937982 reap_dead start 2024-09-13T18:11:18.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.176+0000 7fa1ee3ee700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8152f90 0x7fa1e80c4260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8073e90 tx=0x7fa1e0001d80 comp rx=0 tx=0).stop 2024-09-13T18:11:18.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.176+0000 7fa1ee3ee700 1 -- 172.21.15.40:0/330937982 shutdown_connections 2024-09-13T18:11:18.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.176+0000 7fa1ee3ee700 1 --2- 172.21.15.40:0/330937982 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8152f90 0x7fa1e80c4260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.176+0000 7fa1ee3ee700 1 -- 172.21.15.40:0/330937982 >> 172.21.15.40:0/330937982 conn(0x7fa1e806a390 msgr2=0x7fa1e8073700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:18.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.176+0000 7fa1ee3ee700 1 -- 172.21.15.40:0/330937982 shutdown_connections 2024-09-13T18:11:18.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.176+0000 7fa1ee3ee700 1 -- 172.21.15.40:0/330937982 wait complete. 2024-09-13T18:11:18.200 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:11:18.200 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:11:18.200 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:11:18.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.316+0000 7fa64ffff700 1 Processor -- start 2024-09-13T18:11:18.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.316+0000 7fa64ffff700 1 -- start start 2024-09-13T18:11:18.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.316+0000 7fa64ffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa650153910 0x7fa6500c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.316+0000 7fa64ffff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65015c9c0 con 0x7fa650159ff0 2024-09-13T18:11:18.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.316+0000 7fa64ffff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65015d7e0 con 0x7fa650153530 2024-09-13T18:11:18.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.316+0000 7fa64ffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65015cc60 con 0x7fa650153910 2024-09-13T18:11:18.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.316+0000 7fa64f7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa650159ff0 0x7fa65006d8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49380/0 (socket says 172.21.15.40:49380) 2024-09-13T18:11:18.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.316+0000 7fa64f7fe700 1 -- 172.21.15.40:0/3026709523 learned_addr learned my addr 172.21.15.40:0/3026709523 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:11:18.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.316+0000 7fa647fff700 1 --2- 172.21.15.40:0/3026709523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa650153910 0x7fa6500c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.316+0000 7fa64e7fc700 1 -- 172.21.15.40:0/3026709523 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1896051904 0 0) 0x7fa65015c9c0 con 0x7fa650159ff0 2024-09-13T18:11:18.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.316+0000 7fa64e7fc700 1 -- 172.21.15.40:0/3026709523 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa63c0035d0 con 0x7fa650159ff0 2024-09-13T18:11:18.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.316+0000 7fa64e7fc700 1 -- 172.21.15.40:0/3026709523 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1226403759 0 0) 0x7fa65015d7e0 con 0x7fa650153530 2024-09-13T18:11:18.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.316+0000 7fa647fff700 1 -- 172.21.15.40:0/3026709523 >> v1:172.21.15.43:6789/0 conn(0x7fa650153530 legacy=0x7fa6500c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.316+0000 7fa647fff700 1 -- 172.21.15.40:0/3026709523 >> v1:172.21.15.40:6789/0 conn(0x7fa650159ff0 legacy=0x7fa65006d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.316+0000 7fa647fff700 1 -- 172.21.15.40:0/3026709523 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65015cf40 con 0x7fa650153910 2024-09-13T18:11:18.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa647fff700 1 --2- 172.21.15.40:0/3026709523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa650153910 0x7fa6500c5e50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa638009880 tx=0x7fa638009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37adc6098492d7a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:18.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa64e7fc700 1 -- 172.21.15.40:0/3026709523 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa648001970 con 0x7fa650153910 2024-09-13T18:11:18.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa64e7fc700 1 -- 172.21.15.40:0/3026709523 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa638010590 con 0x7fa650153910 2024-09-13T18:11:18.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa64e7fc700 1 -- 172.21.15.40:0/3026709523 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638010900 con 0x7fa650153910 2024-09-13T18:11:18.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa64ffff700 1 -- 172.21.15.40:0/3026709523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa650153910 msgr2=0x7fa6500c5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa64ffff700 1 --2- 172.21.15.40:0/3026709523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa650153910 0x7fa6500c5e50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa638009880 tx=0x7fa638009b90 comp rx=0 tx=0).stop 2024-09-13T18:11:18.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa64ffff700 1 -- 172.21.15.40:0/3026709523 shutdown_connections 2024-09-13T18:11:18.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa64ffff700 1 --2- 172.21.15.40:0/3026709523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa650153910 0x7fa6500c5e50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa64ffff700 1 -- 172.21.15.40:0/3026709523 >> 172.21.15.40:0/3026709523 conn(0x7fa65006a570 msgr2=0x7fa65006a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:18.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa64ffff700 1 -- 172.21.15.40:0/3026709523 shutdown_connections 2024-09-13T18:11:18.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa64ffff700 1 -- 172.21.15.40:0/3026709523 wait complete. 2024-09-13T18:11:18.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa64ffff700 1 Processor -- start 2024-09-13T18:11:18.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa64ffff700 1 -- start start 2024-09-13T18:11:18.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa64ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa650153530 0x7fa6500cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa64ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa650153910 0x7fa65015bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa64ffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa650159ff0 0x7fa6501ef7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa64ffff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa650156ef0 con 0x7fa650153910 2024-09-13T18:11:18.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa64ffff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500ca760 con 0x7fa650153530 2024-09-13T18:11:18.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa64ffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500caa40 con 0x7fa650159ff0 2024-09-13T18:11:18.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa647fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa650153910 0x7fa65015bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa647fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa650153910 0x7fa65015bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35588/0 (socket says 172.21.15.40:35588) 2024-09-13T18:11:18.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa647fff700 1 -- 172.21.15.40:0/1730828210 learned_addr learned my addr 172.21.15.40:0/1730828210 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:18.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa64effd700 1 --2- 172.21.15.40:0/1730828210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa650153530 0x7fa6500cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.320+0000 7fa64f7fe700 1 --2- 172.21.15.40:0/1730828210 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa650159ff0 0x7fa6501ef7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.324+0000 7fa647fff700 1 -- 172.21.15.40:0/1730828210 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa650159ff0 msgr2=0x7fa6501ef7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.324+0000 7fa647fff700 1 --2- 172.21.15.40:0/1730828210 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa650159ff0 0x7fa6501ef7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.324+0000 7fa647fff700 1 -- 172.21.15.40:0/1730828210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa650153530 msgr2=0x7fa6500cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.324+0000 7fa647fff700 1 --2- 172.21.15.40:0/1730828210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa650153530 0x7fa6500cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.324+0000 7fa647fff700 1 -- 172.21.15.40:0/1730828210 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6380094b0 con 0x7fa650153910 2024-09-13T18:11:18.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.324+0000 7fa647fff700 1 --2- 172.21.15.40:0/1730828210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa650153910 0x7fa65015bce0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa638013040 tx=0x7fa638004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:18.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.324+0000 7fa6477fe700 1 -- 172.21.15.40:0/1730828210 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638004230 con 0x7fa650153910 2024-09-13T18:11:18.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.324+0000 7fa6477fe700 1 -- 172.21.15.40:0/1730828210 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63801e070 con 0x7fa650153910 2024-09-13T18:11:18.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.324+0000 7fa64ffff700 1 -- 172.21.15.40:0/1730828210 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6500cad20 con 0x7fa650153910 2024-09-13T18:11:18.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.324+0000 7fa6477fe700 1 -- 172.21.15.40:0/1730828210 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6380045d0 con 0x7fa650153910 2024-09-13T18:11:18.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.324+0000 7fa64ffff700 1 -- 172.21.15.40:0/1730828210 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6501580b0 con 0x7fa650153910 2024-09-13T18:11:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.324+0000 7fa64ffff700 1 --2- 172.21.15.40:0/1730828210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa650156920 0x7fa65014de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.324+0000 7fa64ffff700 1 -- 172.21.15.40:0/1730828210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500675d0 con 0x7fa650156920 2024-09-13T18:11:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.324+0000 7fa64effd700 1 --2- 172.21.15.40:0/1730828210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa650156920 0x7fa65014de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.324+0000 7fa64effd700 1 -- 172.21.15.40:0/1730828210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa650155cf0 con 0x7fa650156920 2024-09-13T18:11:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.324+0000 7fa6477fe700 1 -- 172.21.15.40:0/1730828210 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa6380036c0 con 0x7fa650153910 2024-09-13T18:11:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.324+0000 7fa64effd700 1 --2- 172.21.15.40:0/1730828210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa650156920 0x7fa65014de50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa640001120 tx=0x7fa64000fb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.324+0000 7fa6477fe700 1 --2- 172.21.15.40:0/1730828210 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa630066730 0x7fa630068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.324+0000 7fa6477fe700 1 -- 172.21.15.40:0/1730828210 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa638085520 con 0x7fa650153910 2024-09-13T18:11:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.324+0000 7fa64f7fe700 1 --2- 172.21.15.40:0/1730828210 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa630066730 0x7fa630068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.328+0000 7fa6477fe700 1 -- 172.21.15.40:0/1730828210 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6400052e0 con 0x7fa650156920 2024-09-13T18:11:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.328+0000 7fa6477fe700 1 -- 172.21.15.40:0/1730828210 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa640005440 con 0x7fa650156920 2024-09-13T18:11:18.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.328+0000 7fa6477fe700 1 -- 172.21.15.40:0/1730828210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa650156920 msgr2=0x7fa65014de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.328+0000 7fa6477fe700 1 --2- 172.21.15.40:0/1730828210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa650156920 0x7fa65014de50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa640001120 tx=0x7fa64000fb00 comp rx=0 tx=0).stop 2024-09-13T18:11:18.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.328+0000 7fa6477fe700 1 -- 172.21.15.40:0/1730828210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa650156920 msgr2=0x7fa65014de50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:11:18.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.328+0000 7fa64f7fe700 1 --2- 172.21.15.40:0/1730828210 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa630066730 0x7fa630068bf0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa648001560 tx=0x7fa648001b90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:18.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.328+0000 7fa6477fe700 1 --2- 172.21.15.40:0/1730828210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa650156920 0x7fa65014de50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.336+0000 7fa64ffff700 1 --2- 172.21.15.40:0/1730828210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa65014e390 0x7fa6501f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.336+0000 7fa64ffff700 1 -- 172.21.15.40:0/1730828210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500caeb0 con 0x7fa65014e390 2024-09-13T18:11:18.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.336+0000 7fa64effd700 1 --2- 172.21.15.40:0/1730828210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa65014e390 0x7fa6501f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.336+0000 7fa64effd700 1 -- 172.21.15.40:0/1730828210 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa65015d500 con 0x7fa65014e390 2024-09-13T18:11:18.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.336+0000 7fa64effd700 1 --2- 172.21.15.40:0/1730828210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa65014e390 0x7fa6501f24c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa640017000 tx=0x7fa640007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:18.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.336+0000 7fa6477fe700 1 -- 172.21.15.40:0/1730828210 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640023070 con 0x7fa65014e390 2024-09-13T18:11:18.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.336+0000 7fa6477fe700 1 -- 172.21.15.40:0/1730828210 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4450 (secure 0 0 0) 0x7fa6400133f0 con 0x7fa65014e390 2024-09-13T18:11:18.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.340+0000 7fa6477fe700 1 -- 172.21.15.40:0/1730828210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa65014e390 msgr2=0x7fa6501f24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.340+0000 7fa6477fe700 1 --2- 172.21.15.40:0/1730828210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa65014e390 0x7fa6501f24c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa640017000 tx=0x7fa640007f90 comp rx=0 tx=0).stop 2024-09-13T18:11:18.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.340+0000 7fa6477fe700 1 -- 172.21.15.40:0/1730828210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa65014e390 msgr2=0x7fa6501f24c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:11:18.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.340+0000 7fa6477fe700 1 --2- 172.21.15.40:0/1730828210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa65014e390 0x7fa6501f24c0 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa640017000 tx=0x7fa640007f90 comp rx=0 tx=0).stop 2024-09-13T18:11:18.347 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:11:18.347 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:11:18.347 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:11:18.347 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:11:18.347 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 166, 2024-09-13T18:11:18.347 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:11:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:11:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:11:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:11:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:11:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:11:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:11:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:11:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:18.348 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:11:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:11:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:18.349 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:11:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:11:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:11:18.350 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:11:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:11:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:11:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:11:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:11:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:11:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:11:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:11:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:11:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:11:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:11:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:11:18.351 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:11:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:11:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:18.352 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:11:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:11:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:11:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:11:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:11:18.353 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:11:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:11:18.354 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:11:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:11:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:11:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:11:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:18.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:11:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:11:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:11:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:18.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:11:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:11:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:11:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:11:18.357 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:11:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:11:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:11:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:11:18.358 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:11:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:11:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:11:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.359 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:11:18.360 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:11:18.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.340+0000 7fa64ffff700 1 -- 172.21.15.40:0/1730828210 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa630066730 msgr2=0x7fa630068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.340+0000 7fa64ffff700 1 --2- 172.21.15.40:0/1730828210 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa630066730 0x7fa630068bf0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa648001560 tx=0x7fa648001b90 comp rx=0 tx=0).stop 2024-09-13T18:11:18.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.340+0000 7fa64ffff700 1 -- 172.21.15.40:0/1730828210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa650153910 msgr2=0x7fa65015bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.340+0000 7fa64ffff700 1 --2- 172.21.15.40:0/1730828210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa650153910 0x7fa65015bce0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa638013040 tx=0x7fa638004770 comp rx=0 tx=0).stop 2024-09-13T18:11:18.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.340+0000 7fa64f7fe700 1 -- 172.21.15.40:0/1730828210 reap_dead start 2024-09-13T18:11:18.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.340+0000 7fa64f7fe700 1 -- 172.21.15.40:0/1730828210 reap_dead start 2024-09-13T18:11:18.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.340+0000 7fa64ffff700 1 -- 172.21.15.40:0/1730828210 shutdown_connections 2024-09-13T18:11:18.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.340+0000 7fa64ffff700 1 -- 172.21.15.40:0/1730828210 >> 172.21.15.40:0/1730828210 conn(0x7fa65006a570 msgr2=0x7fa65006a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:18.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.340+0000 7fa64ffff700 1 -- 172.21.15.40:0/1730828210 shutdown_connections 2024-09-13T18:11:18.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.340+0000 7fa64ffff700 1 -- 172.21.15.40:0/1730828210 wait complete. 2024-09-13T18:11:18.362 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:11:18.362 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:11:18.362 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:11:18.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.476+0000 7fe2b759e700 1 Processor -- start 2024-09-13T18:11:18.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.476+0000 7fe2b759e700 1 -- start start 2024-09-13T18:11:18.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.476+0000 7fe2b759e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe2b0036150 0x7fe2b0007840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.476+0000 7fe2b759e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe2b0033260 0x7fe2b00e5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.476+0000 7fe2b759e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b00db8a0 con 0x7fe2b0051190 2024-09-13T18:11:18.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.476+0000 7fe2b759e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b00067c0 con 0x7fe2b0033260 2024-09-13T18:11:18.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.476+0000 7fe2b759e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b0006aa0 con 0x7fe2b0036150 2024-09-13T18:11:18.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.476+0000 7fe2b6d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe2b0033260 0x7fe2b00e5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.476+0000 7fe2b6d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe2b0033260 0x7fe2b00e5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35600/0 (socket says 172.21.15.40:35600) 2024-09-13T18:11:18.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.476+0000 7fe2b6d9d700 1 -- 172.21.15.40:0/813088734 learned_addr learned my addr 172.21.15.40:0/813088734 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:18.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.476+0000 7fe2b659c700 1 --2- 172.21.15.40:0/813088734 >> v2:172.21.15.43:3300/0 conn(0x7fe2b0036150 0x7fe2b0007840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.476+0000 7fe2b6d9d700 1 -- 172.21.15.40:0/813088734 >> v2:172.21.15.43:3300/0 conn(0x7fe2b0036150 msgr2=0x7fe2b0007840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:11:18.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.476+0000 7fe2b6d9d700 1 --2- 172.21.15.40:0/813088734 >> v2:172.21.15.43:3300/0 conn(0x7fe2b0036150 0x7fe2b0007840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.476+0000 7fe2b6d9d700 1 -- 172.21.15.40:0/813088734 >> v1:172.21.15.43:6789/0 conn(0x7fe2b0051190 legacy=0x7fe2b004c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.476+0000 7fe2b6d9d700 1 -- 172.21.15.40:0/813088734 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b0006d80 con 0x7fe2b0033260 2024-09-13T18:11:18.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.476+0000 7fe2b659c700 1 --2- 172.21.15.40:0/813088734 >> v2:172.21.15.43:3300/0 conn(0x7fe2b0036150 0x7fe2b0007840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:11:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.476+0000 7fe2b6d9d700 1 --2- 172.21.15.40:0/813088734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2b0033260 0x7fe2b00e5380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8063990 tx=0x7fe2b8069e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd815fb17b604d62 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.476+0000 7fe2b559a700 1 -- 172.21.15.40:0/813088734 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8068a70 con 0x7fe2b0033260 2024-09-13T18:11:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.476+0000 7fe2b559a700 1 -- 172.21.15.40:0/813088734 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b8067dd0 con 0x7fe2b0033260 2024-09-13T18:11:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.476+0000 7fe2b559a700 1 -- 172.21.15.40:0/813088734 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b80744e0 con 0x7fe2b0033260 2024-09-13T18:11:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b759e700 1 -- 172.21.15.40:0/813088734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2b0033260 msgr2=0x7fe2b00e5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b759e700 1 --2- 172.21.15.40:0/813088734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2b0033260 0x7fe2b00e5380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8063990 tx=0x7fe2b8069e70 comp rx=0 tx=0).stop 2024-09-13T18:11:18.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b759e700 1 -- 172.21.15.40:0/813088734 shutdown_connections 2024-09-13T18:11:18.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b759e700 1 --2- 172.21.15.40:0/813088734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2b0033260 0x7fe2b00e5380 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b759e700 1 --2- 172.21.15.40:0/813088734 >> v2:172.21.15.43:3300/0 conn(0x7fe2b0036150 0x7fe2b0007840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b759e700 1 -- 172.21.15.40:0/813088734 >> 172.21.15.40:0/813088734 conn(0x7fe2b00505e0 msgr2=0x7fe2b00509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:18.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b759e700 1 -- 172.21.15.40:0/813088734 shutdown_connections 2024-09-13T18:11:18.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b759e700 1 -- 172.21.15.40:0/813088734 wait complete. 2024-09-13T18:11:18.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b759e700 1 Processor -- start 2024-09-13T18:11:18.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b759e700 1 -- start start 2024-09-13T18:11:18.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2b0033260 0x7fe2b004c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2b0036150 0x7fe2b0170320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2b0051190 0x7fe2b0170860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b0174440 con 0x7fe2b0051190 2024-09-13T18:11:18.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b00dbbc0 con 0x7fe2b0036150 2024-09-13T18:11:18.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b00dbe60 con 0x7fe2b0033260 2024-09-13T18:11:18.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2b0051190 0x7fe2b0170860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2b0051190 0x7fe2b0170860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35608/0 (socket says 172.21.15.40:35608) 2024-09-13T18:11:18.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b6d9d700 1 -- 172.21.15.40:0/2301278589 learned_addr learned my addr 172.21.15.40:0/2301278589 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:18.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b659c700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2b0033260 0x7fe2b004c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b5d9b700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2b0036150 0x7fe2b0170320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b6d9d700 1 -- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2b0033260 msgr2=0x7fe2b004c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b6d9d700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2b0033260 0x7fe2b004c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b6d9d700 1 -- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2b0036150 msgr2=0x7fe2b0170320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b6d9d700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2b0036150 0x7fe2b0170320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b6d9d700 1 -- 172.21.15.40:0/2301278589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b804f5c0 con 0x7fe2b0051190 2024-09-13T18:11:18.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b659c700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2b0033260 0x7fe2b004c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:11:18.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.480+0000 7fe2b6d9d700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2b0051190 0x7fe2b0170860 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8069620 tx=0x7fe2b8068470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:18.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.484+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2301278589 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b80956c0 con 0x7fe2b0051190 2024-09-13T18:11:18.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.484+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2301278589 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b8095820 con 0x7fe2b0051190 2024-09-13T18:11:18.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.484+0000 7fe2b759e700 1 -- 172.21.15.40:0/2301278589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b0173830 con 0x7fe2b0051190 2024-09-13T18:11:18.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.484+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2301278589 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8095bb0 con 0x7fe2b0051190 2024-09-13T18:11:18.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.484+0000 7fe2b759e700 1 -- 172.21.15.40:0/2301278589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b0173df0 con 0x7fe2b0051190 2024-09-13T18:11:18.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.484+0000 7fe2b759e700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2b0173f80 0x7fe2b00dc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.484+0000 7fe2b759e700 1 -- 172.21.15.40:0/2301278589 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b0008d50 con 0x7fe2b0173f80 2024-09-13T18:11:18.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.484+0000 7fe2b659c700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2b0173f80 0x7fe2b00dc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.484+0000 7fe2b659c700 1 -- 172.21.15.40:0/2301278589 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe2b0045480 con 0x7fe2b0173f80 2024-09-13T18:11:18.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.484+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2301278589 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe2b8095d10 con 0x7fe2b0051190 2024-09-13T18:11:18.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.484+0000 7fe2b659c700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2b0173f80 0x7fe2b00dc540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80105e0 tx=0x7fe2a80109a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:18.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.484+0000 7fe2a6ffd700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe298066760 0x7fe298068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.484+0000 7fe2b5d9b700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe298066760 0x7fe298068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.484+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2301278589 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe2b81039b0 con 0x7fe2b0051190 2024-09-13T18:11:18.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.484+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2301278589 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80118c0 con 0x7fe2b0173f80 2024-09-13T18:11:18.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.488+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2301278589 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe2a800c030 con 0x7fe2b0173f80 2024-09-13T18:11:18.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.488+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2b0173f80 msgr2=0x7fe2b00dc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.488+0000 7fe2a6ffd700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2b0173f80 0x7fe2b00dc540 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80105e0 tx=0x7fe2a80109a0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.488+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2b0173f80 msgr2=0x7fe2b00dc540 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:11:18.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.488+0000 7fe2a6ffd700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2b0173f80 0x7fe2b00dc540 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.488+0000 7fe2b5d9b700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe298066760 0x7fe298068c20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe2b004ce50 tx=0x7fe2ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:18.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2b759e700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2b004bcb0 0x7fe2b00dca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2b759e700 1 -- 172.21.15.40:0/2301278589 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b0007340 con 0x7fe2b004bcb0 2024-09-13T18:11:18.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2b659c700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2b004bcb0 0x7fe2b00dca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2b659c700 1 -- 172.21.15.40:0/2301278589 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe2b00db480 con 0x7fe2b004bcb0 2024-09-13T18:11:18.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2b659c700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2b004bcb0 0x7fe2b00dca80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80105e0 tx=0x7fe2a8004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:18.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2301278589 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8020880 con 0x7fe2b004bcb0 2024-09-13T18:11:18.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2301278589 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7fe2a8024070 con 0x7fe2b004bcb0 2024-09-13T18:11:18.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2b004bcb0 msgr2=0x7fe2b00dca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2a6ffd700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2b004bcb0 0x7fe2b00dca80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80105e0 tx=0x7fe2a8004770 comp rx=0 tx=0).stop 2024-09-13T18:11:18.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2a6ffd700 1 -- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2b004bcb0 msgr2=0x7fe2b00dca80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:11:18.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2a6ffd700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe2b004bcb0 0x7fe2b00dca80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.509 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:11:18.509 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:11:18.509 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:11:18.509 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:11:18.509 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 166, 2024-09-13T18:11:18.510 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:11:18.510 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:11:18.510 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:11:18.510 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:11:18.510 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:18.510 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:11:18.510 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:11:18.510 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:11:18.510 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:11:18.510 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:18.510 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:18.510 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:18.510 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:18.510 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:18.511 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:18.511 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:18.511 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:18.511 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:18.511 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:18.511 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:18.511 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:11:18.511 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:11:18.511 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:18.511 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:18.511 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:18.511 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:18.511 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:18.511 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:18.512 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:18.512 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:18.512 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:18.512 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:18.512 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:18.512 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.512 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:11:18.512 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:11:18.512 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:11:18.512 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:11:18.512 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:11:18.512 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:11:18.512 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:11:18.513 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:11:18.513 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:11:18.513 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:11:18.513 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:11:18.513 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:11:18.513 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:11:18.513 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:11:18.513 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:11:18.513 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:11:18.513 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:11:18.513 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:18.513 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:18.513 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:18.514 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:18.514 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:18.514 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:18.514 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:18.514 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:18.514 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:18.514 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:18.514 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:18.514 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:11:18.514 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.514 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:11:18.514 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.514 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:11:18.514 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:11:18.515 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:18.515 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:18.515 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.515 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:18.515 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:11:18.515 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:18.515 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.515 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.515 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:18.515 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:11:18.515 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:18.515 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.515 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:18.516 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.516 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:11:18.516 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:11:18.516 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:18.516 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:18.516 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:18.516 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.516 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.516 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:11:18.516 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:11:18.516 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:18.516 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:18.516 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.517 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:18.517 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:11:18.517 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:18.517 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.517 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.517 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:18.517 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:11:18.517 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:18.517 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.517 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:18.517 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.517 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:11:18.517 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:11:18.518 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:18.518 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:18.518 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:18.518 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.518 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.518 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:11:18.518 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:11:18.518 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:18.518 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:18.518 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.518 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:18.518 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:11:18.518 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:18.518 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.519 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.519 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:18.519 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:11:18.519 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:18.519 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.519 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:18.519 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.519 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:11:18.519 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:11:18.519 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:18.520 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:18.520 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:18.520 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.520 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:18.520 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.520 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:11:18.520 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:11:18.520 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.520 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:18.520 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:18.520 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:11:18.520 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.520 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:18.521 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:11:18.521 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.521 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:18.521 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:18.521 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:11:18.521 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.521 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:18.521 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:11:18.521 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.521 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:18.521 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:18.521 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:11:18.521 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.522 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:18.522 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:11:18.522 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.522 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:18.522 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:18.522 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:11:18.522 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.522 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:18.522 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:11:18.522 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:18.522 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:18.522 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:18.522 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:11:18.522 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:18.523 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:18.523 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:18.523 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:11:18.523 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:11:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2b759e700 1 -- 172.21.15.40:0/2301278589 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe298066760 msgr2=0x7fe298068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2b759e700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe298066760 0x7fe298068c20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe2b004ce50 tx=0x7fe2ac009040 comp rx=0 tx=0).stop 2024-09-13T18:11:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2b759e700 1 -- 172.21.15.40:0/2301278589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2b0051190 msgr2=0x7fe2b0170860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2b759e700 1 --2- 172.21.15.40:0/2301278589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2b0051190 0x7fe2b0170860 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8069620 tx=0x7fe2b8068470 comp rx=0 tx=0).stop 2024-09-13T18:11:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2b6d9d700 1 -- 172.21.15.40:0/2301278589 reap_dead start 2024-09-13T18:11:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2b6d9d700 1 -- 172.21.15.40:0/2301278589 reap_dead start 2024-09-13T18:11:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2b759e700 1 -- 172.21.15.40:0/2301278589 shutdown_connections 2024-09-13T18:11:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2b759e700 1 -- 172.21.15.40:0/2301278589 >> 172.21.15.40:0/2301278589 conn(0x7fe2b00505e0 msgr2=0x7fe2b0045f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2b759e700 1 -- 172.21.15.40:0/2301278589 shutdown_connections 2024-09-13T18:11:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.500+0000 7fe2b759e700 1 -- 172.21.15.40:0/2301278589 wait complete. 2024-09-13T18:11:18.526 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:11:18.526 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:11:18.526 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:11:18.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae4c806700 1 Processor -- start 2024-09-13T18:11:18.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae4c806700 1 -- start start 2024-09-13T18:11:18.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae4c806700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae480cb2f0 0x7fae480cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae4c806700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae48159770 con 0x7fae480cf3c0 2024-09-13T18:11:18.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae4c806700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae48159a50 con 0x7fae48153880 2024-09-13T18:11:18.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae4c806700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae48159d30 con 0x7fae480cb2f0 2024-09-13T18:11:18.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae47fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fae480cf3c0 0x7fae4815cf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49384/0 (socket says 172.21.15.40:49384) 2024-09-13T18:11:18.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae47fff700 1 -- 172.21.15.40:0/3386399540 learned_addr learned my addr 172.21.15.40:0/3386399540 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:11:18.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae46ffd700 1 --2- 172.21.15.40:0/3386399540 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae480cb2f0 0x7fae480cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae467fc700 1 -- 172.21.15.40:0/3386399540 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714557842 0 0) 0x7fae48159770 con 0x7fae480cf3c0 2024-09-13T18:11:18.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae467fc700 1 -- 172.21.15.40:0/3386399540 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae300035d0 con 0x7fae480cf3c0 2024-09-13T18:11:18.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae467fc700 1 -- 172.21.15.40:0/3386399540 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 727467798 0 0) 0x7fae48159a50 con 0x7fae48153880 2024-09-13T18:11:18.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae467fc700 1 -- 172.21.15.40:0/3386399540 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae48159770 con 0x7fae48153880 2024-09-13T18:11:18.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae46ffd700 1 -- 172.21.15.40:0/3386399540 >> v1:172.21.15.43:6789/0 conn(0x7fae48153880 legacy=0x7fae480c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae467fc700 1 -- 172.21.15.40:0/3386399540 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3897057933 0 0) 0x7fae300035d0 con 0x7fae480cf3c0 2024-09-13T18:11:18.641 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae46ffd700 1 -- 172.21.15.40:0/3386399540 >> v1:172.21.15.40:6789/0 conn(0x7fae480cf3c0 legacy=0x7fae4815cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae46ffd700 1 -- 172.21.15.40:0/3386399540 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4815a010 con 0x7fae480cb2f0 2024-09-13T18:11:18.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae46ffd700 1 --2- 172.21.15.40:0/3386399540 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae480cb2f0 0x7fae480cee80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009880 tx=0x7fae3c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5258454ef927629 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:18.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae467fc700 1 -- 172.21.15.40:0/3386399540 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae400019f0 con 0x7fae480cb2f0 2024-09-13T18:11:18.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae467fc700 1 -- 172.21.15.40:0/3386399540 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3c004030 con 0x7fae480cb2f0 2024-09-13T18:11:18.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae467fc700 1 -- 172.21.15.40:0/3386399540 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c0043a0 con 0x7fae480cb2f0 2024-09-13T18:11:18.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae4c806700 1 -- 172.21.15.40:0/3386399540 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae480cb2f0 msgr2=0x7fae480cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae4c806700 1 --2- 172.21.15.40:0/3386399540 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae480cb2f0 0x7fae480cee80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009880 tx=0x7fae3c009b90 comp rx=0 tx=0).stop 2024-09-13T18:11:18.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae4c806700 1 -- 172.21.15.40:0/3386399540 shutdown_connections 2024-09-13T18:11:18.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae4c806700 1 --2- 172.21.15.40:0/3386399540 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae480cb2f0 0x7fae480cee80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae4c806700 1 -- 172.21.15.40:0/3386399540 >> 172.21.15.40:0/3386399540 conn(0x7fae4806c5b0 msgr2=0x7fae4806c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:18.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae4c806700 1 -- 172.21.15.40:0/3386399540 shutdown_connections 2024-09-13T18:11:18.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.632+0000 7fae4c806700 1 -- 172.21.15.40:0/3386399540 wait complete. 2024-09-13T18:11:18.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae4c806700 1 Processor -- start 2024-09-13T18:11:18.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae4c806700 1 -- start start 2024-09-13T18:11:18.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae4c806700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae480cb2f0 0x7fae481e65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae4c806700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae480cf3c0 0x7fae4815b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae4c806700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae48153880 0x7fae481f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae4c806700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae481f0060 con 0x7fae480cf3c0 2024-09-13T18:11:18.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae4c806700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae481f2730 con 0x7fae48153880 2024-09-13T18:11:18.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae4c806700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae481ef2a0 con 0x7fae480cb2f0 2024-09-13T18:11:18.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae46ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae480cf3c0 0x7fae4815b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae46ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae480cf3c0 0x7fae4815b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35616/0 (socket says 172.21.15.40:35616) 2024-09-13T18:11:18.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae46ffd700 1 -- 172.21.15.40:0/2557983249 learned_addr learned my addr 172.21.15.40:0/2557983249 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:18.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae477fe700 1 --2- 172.21.15.40:0/2557983249 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae480cb2f0 0x7fae481e65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae47fff700 1 --2- 172.21.15.40:0/2557983249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae48153880 0x7fae481f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae46ffd700 1 -- 172.21.15.40:0/2557983249 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae480cb2f0 msgr2=0x7fae481e65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae46ffd700 1 --2- 172.21.15.40:0/2557983249 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae480cb2f0 0x7fae481e65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae46ffd700 1 -- 172.21.15.40:0/2557983249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae48153880 msgr2=0x7fae481f2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae46ffd700 1 --2- 172.21.15.40:0/2557983249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae48153880 0x7fae481f2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae46ffd700 1 -- 172.21.15.40:0/2557983249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c0094b0 con 0x7fae480cf3c0 2024-09-13T18:11:18.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae46ffd700 1 --2- 172.21.15.40:0/2557983249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae480cf3c0 0x7fae4815b490 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fae3c000c00 tx=0x7fae3c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:18.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae2ffff700 1 -- 172.21.15.40:0/2557983249 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c004590 con 0x7fae480cf3c0 2024-09-13T18:11:18.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.636+0000 7fae2ffff700 1 -- 172.21.15.40:0/2557983249 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3c003d00 con 0x7fae480cf3c0 2024-09-13T18:11:18.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.640+0000 7fae2ffff700 1 -- 172.21.15.40:0/2557983249 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c011870 con 0x7fae480cf3c0 2024-09-13T18:11:18.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.640+0000 7fae4c806700 1 -- 172.21.15.40:0/2557983249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae481ef580 con 0x7fae480cf3c0 2024-09-13T18:11:18.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.640+0000 7fae4c806700 1 -- 172.21.15.40:0/2557983249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae481efb40 con 0x7fae480cf3c0 2024-09-13T18:11:18.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.640+0000 7fae4c806700 1 -- 172.21.15.40:0/2557983249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae48159a50 con 0x7fae480cf3c0 2024-09-13T18:11:18.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.640+0000 7fae2ffff700 1 -- 172.21.15.40:0/2557983249 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fae3c0119d0 con 0x7fae480cf3c0 2024-09-13T18:11:18.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.640+0000 7fae2ffff700 1 --2- 172.21.15.40:0/2557983249 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fae34066730 0x7fae34068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:18.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.640+0000 7fae2ffff700 1 -- 172.21.15.40:0/2557983249 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fae3c085310 con 0x7fae480cf3c0 2024-09-13T18:11:18.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.640+0000 7fae477fe700 1 --2- 172.21.15.40:0/2557983249 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fae34066730 0x7fae34068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:18.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.640+0000 7fae477fe700 1 --2- 172.21.15.40:0/2557983249 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fae34066730 0x7fae34068bf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fae380011e0 tx=0x7fae38009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:18.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.648+0000 7fae2ffff700 1 -- 172.21.15.40:0/2557983249 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fae3c0523e0 con 0x7fae480cf3c0 2024-09-13T18:11:18.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.936+0000 7fae4c806700 1 -- 172.21.15.40:0/2557983249 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fae480b2bb0 con 0x7fae480cf3c0 2024-09-13T18:11:18.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.936+0000 7fae2ffff700 1 -- 172.21.15.40:0/2557983249 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fae3c055a00 con 0x7fae480cf3c0 2024-09-13T18:11:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.940+0000 7fae4c806700 1 -- 172.21.15.40:0/2557983249 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fae34066730 msgr2=0x7fae34068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.940+0000 7fae4c806700 1 --2- 172.21.15.40:0/2557983249 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fae34066730 0x7fae34068bf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fae380011e0 tx=0x7fae38009040 comp rx=0 tx=0).stop 2024-09-13T18:11:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.940+0000 7fae4c806700 1 -- 172.21.15.40:0/2557983249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae480cf3c0 msgr2=0x7fae4815b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.940+0000 7fae4c806700 1 --2- 172.21.15.40:0/2557983249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae480cf3c0 0x7fae4815b490 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fae3c000c00 tx=0x7fae3c004770 comp rx=0 tx=0).stop 2024-09-13T18:11:18.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.940+0000 7fae4c806700 1 -- 172.21.15.40:0/2557983249 shutdown_connections 2024-09-13T18:11:18.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.940+0000 7fae4c806700 1 --2- 172.21.15.40:0/2557983249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae48153880 0x7fae481f2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.940+0000 7fae4c806700 1 --2- 172.21.15.40:0/2557983249 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fae480cf3c0 0x7fae4815b490 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.940+0000 7fae4c806700 1 --2- 172.21.15.40:0/2557983249 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fae34066730 0x7fae34068bf0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.940+0000 7fae4c806700 1 --2- 172.21.15.40:0/2557983249 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fae480cb2f0 0x7fae481e65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:18.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.940+0000 7fae4c806700 1 -- 172.21.15.40:0/2557983249 >> 172.21.15.40:0/2557983249 conn(0x7fae4806c5b0 msgr2=0x7fae48066090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:18.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.940+0000 7fae4c806700 1 -- 172.21.15.40:0/2557983249 shutdown_connections 2024-09-13T18:11:18.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:18.940+0000 7fae4c806700 1 -- 172.21.15.40:0/2557983249 wait complete. 2024-09-13T18:11:18.967 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:11:28.968 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:11:28.969 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:11:29.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.088+0000 7f35ea088700 1 Processor -- start 2024-09-13T18:11:29.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.088+0000 7f35ea088700 1 -- start start 2024-09-13T18:11:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.088+0000 7f35ea088700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f35e406ea10 0x7f35e40663b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.088+0000 7f35ea088700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f35e40c50b0 0x7f35e41570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.088+0000 7f35ea088700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e40c76c0 con 0x7f35e40c4cd0 2024-09-13T18:11:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.088+0000 7f35ea088700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e4157740 con 0x7f35e406ea10 2024-09-13T18:11:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.088+0000 7f35ea088700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e41579e0 con 0x7f35e40c50b0 2024-09-13T18:11:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.088+0000 7f35e8885700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f35e406ea10 0x7f35e40663b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.088+0000 7f35e9086700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f35e40c4cd0 0x7f35e40c7b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58558/0 (socket says 172.21.15.40:58558) 2024-09-13T18:11:29.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.088+0000 7f35e9086700 1 -- 172.21.15.40:0/29685763 learned_addr learned my addr 172.21.15.40:0/29685763 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:11:29.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.088+0000 7f35e9887700 1 --2- 172.21.15.40:0/29685763 >> v2:172.21.15.43:3300/0 conn(0x7f35e40c50b0 0x7f35e41570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.088+0000 7f35dbfff700 1 -- 172.21.15.40:0/29685763 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1176186836 0 0) 0x7f35e40c76c0 con 0x7f35e40c4cd0 2024-09-13T18:11:29.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.088+0000 7f35dbfff700 1 -- 172.21.15.40:0/29685763 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35d00035d0 con 0x7f35e40c4cd0 2024-09-13T18:11:29.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.088+0000 7f35e8885700 1 -- 172.21.15.40:0/29685763 >> v2:172.21.15.43:3300/0 conn(0x7f35e40c50b0 msgr2=0x7f35e41570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:11:29.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.088+0000 7f35e8885700 1 --2- 172.21.15.40:0/29685763 >> v2:172.21.15.43:3300/0 conn(0x7f35e40c50b0 0x7f35e41570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.088+0000 7f35e8885700 1 -- 172.21.15.40:0/29685763 >> v1:172.21.15.40:6789/0 conn(0x7f35e40c4cd0 legacy=0x7f35e40c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.088+0000 7f35e8885700 1 -- 172.21.15.40:0/29685763 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e4157ca0 con 0x7f35e406ea10 2024-09-13T18:11:29.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.088+0000 7f35e8885700 1 --2- 172.21.15.40:0/29685763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e406ea10 0x7f35e40663b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f35e40c5760 tx=0x7f35d4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aaa930c3e8fad2f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:29.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.088+0000 7f35dbfff700 1 -- 172.21.15.40:0/29685763 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d4011830 con 0x7f35e406ea10 2024-09-13T18:11:29.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.092+0000 7f35dbfff700 1 -- 172.21.15.40:0/29685763 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d4011990 con 0x7f35e406ea10 2024-09-13T18:11:29.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.092+0000 7f35dbfff700 1 -- 172.21.15.40:0/29685763 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d4011d20 con 0x7f35e406ea10 2024-09-13T18:11:29.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.092+0000 7f35ea088700 1 -- 172.21.15.40:0/29685763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e406ea10 msgr2=0x7f35e40663b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.092+0000 7f35ea088700 1 --2- 172.21.15.40:0/29685763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e406ea10 0x7f35e40663b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f35e40c5760 tx=0x7f35d4009b30 comp rx=0 tx=0).stop 2024-09-13T18:11:29.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.092+0000 7f35ea088700 1 -- 172.21.15.40:0/29685763 shutdown_connections 2024-09-13T18:11:29.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.092+0000 7f35ea088700 1 --2- 172.21.15.40:0/29685763 >> v2:172.21.15.43:3300/0 conn(0x7f35e40c50b0 0x7f35e41570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.092+0000 7f35ea088700 1 --2- 172.21.15.40:0/29685763 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e406ea10 0x7f35e40663b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.092+0000 7f35ea088700 1 -- 172.21.15.40:0/29685763 >> 172.21.15.40:0/29685763 conn(0x7f35e406a390 msgr2=0x7f35e406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:29.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.092+0000 7f35ea088700 1 -- 172.21.15.40:0/29685763 shutdown_connections 2024-09-13T18:11:29.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.092+0000 7f35ea088700 1 -- 172.21.15.40:0/29685763 wait complete. 2024-09-13T18:11:29.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.092+0000 7f35ea088700 1 Processor -- start 2024-09-13T18:11:29.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.092+0000 7f35ea088700 1 -- start start 2024-09-13T18:11:29.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35ea088700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35e406ea10 0x7f35e4152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35ea088700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e40c4cd0 0x7f35e41ecbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35ea088700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35e40c50b0 0x7f35e414bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35ea088700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e4149240 con 0x7f35e40c4cd0 2024-09-13T18:11:29.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35ea088700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e4157390 con 0x7f35e40c50b0 2024-09-13T18:11:29.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35ea088700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e4148c80 con 0x7f35e406ea10 2024-09-13T18:11:29.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35e8885700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e40c4cd0 0x7f35e41ecbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35e8885700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e40c4cd0 0x7f35e41ecbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36914/0 (socket says 172.21.15.40:36914) 2024-09-13T18:11:29.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35e8885700 1 -- 172.21.15.40:0/3636444039 learned_addr learned my addr 172.21.15.40:0/3636444039 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:29.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35e9086700 1 --2- 172.21.15.40:0/3636444039 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35e406ea10 0x7f35e4152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35e9887700 1 --2- 172.21.15.40:0/3636444039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35e40c50b0 0x7f35e414bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35e8885700 1 -- 172.21.15.40:0/3636444039 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35e406ea10 msgr2=0x7f35e4152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35e8885700 1 --2- 172.21.15.40:0/3636444039 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35e406ea10 0x7f35e4152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35e8885700 1 -- 172.21.15.40:0/3636444039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35e40c50b0 msgr2=0x7f35e414bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35e8885700 1 --2- 172.21.15.40:0/3636444039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35e40c50b0 0x7f35e414bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35e8885700 1 -- 172.21.15.40:0/3636444039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d40094b0 con 0x7f35e40c4cd0 2024-09-13T18:11:29.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35e8885700 1 --2- 172.21.15.40:0/3636444039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e40c4cd0 0x7f35e41ecbd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f35d40090b0 tx=0x7f35d4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:29.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35d9ffb700 1 -- 172.21.15.40:0/3636444039 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d4004480 con 0x7f35e40c4cd0 2024-09-13T18:11:29.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35d9ffb700 1 -- 172.21.15.40:0/3636444039 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d401f070 con 0x7f35e40c4cd0 2024-09-13T18:11:29.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35ea088700 1 -- 172.21.15.40:0/3636444039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e4148f60 con 0x7f35e40c4cd0 2024-09-13T18:11:29.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35d9ffb700 1 -- 172.21.15.40:0/3636444039 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d40038c0 con 0x7f35e40c4cd0 2024-09-13T18:11:29.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35ea088700 1 -- 172.21.15.40:0/3636444039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e4155970 con 0x7f35e40c4cd0 2024-09-13T18:11:29.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35ea088700 1 -- 172.21.15.40:0/3636444039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e40663b0 con 0x7f35e40c4cd0 2024-09-13T18:11:29.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.096+0000 7f35d9ffb700 1 -- 172.21.15.40:0/3636444039 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f35d4021030 con 0x7f35e40c4cd0 2024-09-13T18:11:29.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.100+0000 7f35d9ffb700 1 --2- 172.21.15.40:0/3636444039 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f35c8066730 0x7f35c8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.100+0000 7f35d9ffb700 1 -- 172.21.15.40:0/3636444039 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f35d4085500 con 0x7f35e40c4cd0 2024-09-13T18:11:29.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.100+0000 7f35e9086700 1 --2- 172.21.15.40:0/3636444039 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f35c8066730 0x7f35c8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.100+0000 7f35e9086700 1 --2- 172.21.15.40:0/3636444039 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f35c8066730 0x7f35c8068bf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f35e00011e0 tx=0x7f35e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:29.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.104+0000 7f35d9ffb700 1 -- 172.21.15.40:0/3636444039 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f35d40524b0 con 0x7f35e40c4cd0 2024-09-13T18:11:29.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.420+0000 7f35ea088700 1 -- 172.21.15.40:0/3636444039 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f35e40c7ed0 con 0x7f35e40c4cd0 2024-09-13T18:11:29.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.424+0000 7f35d9ffb700 1 -- 172.21.15.40:0/3636444039 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f35d4015620 con 0x7f35e40c4cd0 2024-09-13T18:11:29.431 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":166,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:11:29.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.428+0000 7f35ea088700 1 -- 172.21.15.40:0/3636444039 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f35c8066730 msgr2=0x7f35c8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.428+0000 7f35ea088700 1 --2- 172.21.15.40:0/3636444039 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f35c8066730 0x7f35c8068bf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f35e00011e0 tx=0x7f35e0009040 comp rx=0 tx=0).stop 2024-09-13T18:11:29.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.428+0000 7f35ea088700 1 -- 172.21.15.40:0/3636444039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e40c4cd0 msgr2=0x7f35e41ecbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.428+0000 7f35ea088700 1 --2- 172.21.15.40:0/3636444039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e40c4cd0 0x7f35e41ecbd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f35d40090b0 tx=0x7f35d4004770 comp rx=0 tx=0).stop 2024-09-13T18:11:29.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.428+0000 7f35ea088700 1 -- 172.21.15.40:0/3636444039 shutdown_connections 2024-09-13T18:11:29.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.428+0000 7f35ea088700 1 --2- 172.21.15.40:0/3636444039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35e40c50b0 0x7f35e414bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.428+0000 7f35ea088700 1 --2- 172.21.15.40:0/3636444039 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35e40c4cd0 0x7f35e41ecbd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.428+0000 7f35ea088700 1 --2- 172.21.15.40:0/3636444039 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f35c8066730 0x7f35c8068bf0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.428+0000 7f35ea088700 1 --2- 172.21.15.40:0/3636444039 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35e406ea10 0x7f35e4152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.428+0000 7f35ea088700 1 -- 172.21.15.40:0/3636444039 >> 172.21.15.40:0/3636444039 conn(0x7f35e406a390 msgr2=0x7f35e406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:29.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.428+0000 7f35ea088700 1 -- 172.21.15.40:0/3636444039 shutdown_connections 2024-09-13T18:11:29.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.428+0000 7f35ea088700 1 -- 172.21.15.40:0/3636444039 wait complete. 2024-09-13T18:11:29.451 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:11:29.451 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T18:11:29.451 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:11:29.451 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:11:29.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4faf319700 1 Processor -- start 2024-09-13T18:11:29.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4faf319700 1 -- start start 2024-09-13T18:11:29.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4faf319700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4fa8159f90 0x7f4fa8164dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4faf319700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa80c7ab0 con 0x7f4fa8156bf0 2024-09-13T18:11:29.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4faf319700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa806ff60 con 0x7f4fa80c2010 2024-09-13T18:11:29.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4faf319700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa80c33f0 con 0x7f4fa8159f90 2024-09-13T18:11:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4faeb18700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4fa8159f90 0x7f4fa8164dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4faeb18700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4fa8159f90 0x7f4fa8164dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36920/0 (socket says 172.21.15.40:36920) 2024-09-13T18:11:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4faeb18700 1 -- 172.21.15.40:0/1083453642 learned_addr learned my addr 172.21.15.40:0/1083453642 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4fae317700 1 --1- 172.21.15.40:0/1083453642 >> v1:172.21.15.40:6789/0 conn(0x7f4fa8156bf0 0x7f4fa80c2ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58562/0 (socket says 172.21.15.40:58562) 2024-09-13T18:11:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4faeb18700 1 -- 172.21.15.40:0/1083453642 >> v1:172.21.15.43:6789/0 conn(0x7f4fa80c2010 legacy=0x7f4fa80c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:11:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4faeb18700 1 -- 172.21.15.40:0/1083453642 >> v1:172.21.15.40:6789/0 conn(0x7f4fa8156bf0 legacy=0x7f4fa80c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4faeb18700 1 -- 172.21.15.40:0/1083453642 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa806ecf0 con 0x7f4fa8159f90 2024-09-13T18:11:29.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4faeb18700 1 --2- 172.21.15.40:0/1083453642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa8159f90 0x7f4fa8164dc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0009e50 tx=0x7f4fa000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fb31f2a72e3bf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:29.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4fad315700 1 -- 172.21.15.40:0/1083453642 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa000e070 con 0x7f4fa8159f90 2024-09-13T18:11:29.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4fad315700 1 -- 172.21.15.40:0/1083453642 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa000b9c0 con 0x7f4fa8159f90 2024-09-13T18:11:29.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4fad315700 1 -- 172.21.15.40:0/1083453642 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa000bd70 con 0x7f4fa8159f90 2024-09-13T18:11:29.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4faf319700 1 -- 172.21.15.40:0/1083453642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa8159f90 msgr2=0x7f4fa8164dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4faf319700 1 --2- 172.21.15.40:0/1083453642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa8159f90 0x7f4fa8164dc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0009e50 tx=0x7f4fa000b330 comp rx=0 tx=0).stop 2024-09-13T18:11:29.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4faf319700 1 -- 172.21.15.40:0/1083453642 shutdown_connections 2024-09-13T18:11:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4faf319700 1 --2- 172.21.15.40:0/1083453642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa8159f90 0x7f4fa8164dc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4faf319700 1 -- 172.21.15.40:0/1083453642 >> 172.21.15.40:0/1083453642 conn(0x7f4fa8066090 msgr2=0x7f4fa8066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4faf319700 1 -- 172.21.15.40:0/1083453642 shutdown_connections 2024-09-13T18:11:29.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.556+0000 7f4faf319700 1 -- 172.21.15.40:0/1083453642 wait complete. 2024-09-13T18:11:29.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4faf319700 1 Processor -- start 2024-09-13T18:11:29.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4faf319700 1 -- start start 2024-09-13T18:11:29.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4faf319700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4fa80c2010 0x7f4fa81ee950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4faf319700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa8156bf0 0x7f4fa81e3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4faf319700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fa8159f90 0x7f4fa806c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4faf319700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa81e4a60 con 0x7f4fa8156bf0 2024-09-13T18:11:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4faf319700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa81e44c0 con 0x7f4fa8159f90 2024-09-13T18:11:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4faf319700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa81e4780 con 0x7f4fa80c2010 2024-09-13T18:11:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4fadb16700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa8156bf0 0x7f4fa81e3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4fae317700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4fa80c2010 0x7f4fa81ee950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4faeb18700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fa8159f90 0x7f4fa806c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4fadb16700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa8156bf0 0x7f4fa81e3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36928/0 (socket says 172.21.15.40:36928) 2024-09-13T18:11:29.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4fadb16700 1 -- 172.21.15.40:0/2392708431 learned_addr learned my addr 172.21.15.40:0/2392708431 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4fadb16700 1 -- 172.21.15.40:0/2392708431 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4fa80c2010 msgr2=0x7f4fa81ee950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4fadb16700 1 --2- 172.21.15.40:0/2392708431 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4fa80c2010 0x7f4fa81ee950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4fadb16700 1 -- 172.21.15.40:0/2392708431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fa8159f90 msgr2=0x7f4fa806c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4fadb16700 1 --2- 172.21.15.40:0/2392708431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fa8159f90 0x7f4fa806c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4fadb16700 1 -- 172.21.15.40:0/2392708431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa0009b30 con 0x7f4fa8156bf0 2024-09-13T18:11:29.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4fadb16700 1 --2- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa8156bf0 0x7f4fa81e3b10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800c500 tx=0x7f4f9800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4f9effd700 1 -- 172.21.15.40:0/2392708431 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f980052d0 con 0x7f4fa8156bf0 2024-09-13T18:11:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4f9effd700 1 -- 172.21.15.40:0/2392708431 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f98005430 con 0x7f4fa8156bf0 2024-09-13T18:11:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4faf319700 1 -- 172.21.15.40:0/2392708431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa81f1ff0 con 0x7f4fa8156bf0 2024-09-13T18:11:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4f9effd700 1 -- 172.21.15.40:0/2392708431 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f980055a0 con 0x7f4fa8156bf0 2024-09-13T18:11:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.560+0000 7f4faf319700 1 -- 172.21.15.40:0/2392708431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa81f25b0 con 0x7f4fa8156bf0 2024-09-13T18:11:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.564+0000 7f4faf319700 1 --2- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa8154320 0x7f4fa806c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.564+0000 7f4faf319700 1 -- 172.21.15.40:0/2392708431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa8074830 con 0x7f4fa8154320 2024-09-13T18:11:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.564+0000 7f4fae317700 1 --2- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa8154320 0x7f4fa806c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.564+0000 7f4fae317700 1 -- 172.21.15.40:0/2392708431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fa806f910 con 0x7f4fa8154320 2024-09-13T18:11:29.572 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.564+0000 7f4fae317700 1 --2- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa8154320 0x7f4fa806c9c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4001580 tx=0x7f4fa4001940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.564+0000 7f4f9effd700 1 -- 172.21.15.40:0/2392708431 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4017070 con 0x7f4fa8154320 2024-09-13T18:11:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.564+0000 7f4f9effd700 1 -- 172.21.15.40:0/2392708431 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4f980038e0 con 0x7f4fa8156bf0 2024-09-13T18:11:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.564+0000 7f4f9effd700 1 --2- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4f8c0667d0 0x7f4f8c068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.564+0000 7f4f9effd700 1 -- 172.21.15.40:0/2392708431 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4f980837a0 con 0x7f4fa8156bf0 2024-09-13T18:11:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.564+0000 7f4faeb18700 1 --2- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4f8c0667d0 0x7f4f8c068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.564+0000 7f4f9effd700 1 -- 172.21.15.40:0/2392708431 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4fa4012560 con 0x7f4fa8154320 2024-09-13T18:11:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.564+0000 7f4f9effd700 1 -- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa8154320 msgr2=0x7f4fa806c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.564+0000 7f4f9effd700 1 --2- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa8154320 0x7f4fa806c9c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4001580 tx=0x7f4fa4001940 comp rx=0 tx=0).stop 2024-09-13T18:11:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.564+0000 7f4f9effd700 1 -- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa8154320 msgr2=0x7f4fa806c9c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:11:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.564+0000 7f4f9effd700 1 --2- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa8154320 0x7f4fa806c9c0 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4001580 tx=0x7f4fa4001940 comp rx=0 tx=0).stop 2024-09-13T18:11:29.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.564+0000 7f4faeb18700 1 --2- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4f8c0667d0 0x7f4f8c068c90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4fa000b370 tx=0x7f4fa0009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:29.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4faf319700 1 --2- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa806e710 0x7f4fa806d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4faf319700 1 -- 172.21.15.40:0/2392708431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa806fbf0 con 0x7f4fa806e710 2024-09-13T18:11:29.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4fae317700 1 --2- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa806e710 0x7f4fa806d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4fae317700 1 -- 172.21.15.40:0/2392708431 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4fa806f610 con 0x7f4fa806e710 2024-09-13T18:11:29.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4fae317700 1 --2- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa806e710 0x7f4fa806d490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4015040 tx=0x7f4fa4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:29.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4f9effd700 1 -- 172.21.15.40:0/2392708431 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4023070 con 0x7f4fa806e710 2024-09-13T18:11:29.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4f9effd700 1 -- 172.21.15.40:0/2392708431 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4785 (secure 0 0 0) 0x7f4fa4012cf0 con 0x7f4fa806e710 2024-09-13T18:11:29.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4f9effd700 1 -- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa806e710 msgr2=0x7f4fa806d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4f9effd700 1 --2- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa806e710 0x7f4fa806d490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4015040 tx=0x7f4fa4001120 comp rx=0 tx=0).stop 2024-09-13T18:11:29.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4f9effd700 1 -- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa806e710 msgr2=0x7f4fa806d490 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:11:29.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4f9effd700 1 --2- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa806e710 0x7f4fa806d490 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.585 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:11:29.585 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:11:29.585 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:11:29.585 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:11:29.585 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 166, 2024-09-13T18:11:29.585 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:11:29.586 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:11:29.586 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:11:29.586 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:11:29.586 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:29.586 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 12, 2024-09-13T18:11:29.586 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:11:29.586 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:11:29.586 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:11:29.587 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:29.587 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:29.587 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:29.587 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:29.587 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:29.587 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:29.587 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:29.587 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:29.587 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:29.587 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:29.587 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:29.587 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:11:29.587 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:11:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:29.588 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:11:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:11:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:11:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:11:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:11:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:11:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:11:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:11:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:11:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:11:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:11:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:11:29.589 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:11:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:11:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:11:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:11:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:11:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:29.590 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:11:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:11:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:11:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:11:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:11:29.591 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:11:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.592 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:11:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:11:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:11:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:11:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:29.593 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:11:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:11:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:11:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:11:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:29.594 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:11:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:11:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:11:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.595 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:11:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:11:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:11:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:11:29.596 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:11:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:11:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:11:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:29.597 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:11:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:11:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:11:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:11:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.598 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:29.599 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:29.599 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:11:29.599 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.599 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:29.599 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.599 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:11:29.599 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:11:29.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4faf319700 1 -- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4f8c0667d0 msgr2=0x7f4f8c068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4faf319700 1 --2- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4f8c0667d0 0x7f4f8c068c90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4fa000b370 tx=0x7f4fa0009000 comp rx=0 tx=0).stop 2024-09-13T18:11:29.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4faf319700 1 -- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa8156bf0 msgr2=0x7f4fa81e3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4faf319700 1 --2- 172.21.15.40:0/2392708431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4fa8156bf0 0x7f4fa81e3b10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800c500 tx=0x7f4f9800c810 comp rx=0 tx=0).stop 2024-09-13T18:11:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4faeb18700 1 -- 172.21.15.40:0/2392708431 reap_dead start 2024-09-13T18:11:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4faeb18700 1 -- 172.21.15.40:0/2392708431 reap_dead start 2024-09-13T18:11:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4faf319700 1 -- 172.21.15.40:0/2392708431 shutdown_connections 2024-09-13T18:11:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4faf319700 1 -- 172.21.15.40:0/2392708431 >> 172.21.15.40:0/2392708431 conn(0x7f4fa8066090 msgr2=0x7f4fa80c7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4faf319700 1 -- 172.21.15.40:0/2392708431 shutdown_connections 2024-09-13T18:11:29.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.576+0000 7f4faf319700 1 -- 172.21.15.40:0/2392708431 wait complete. 2024-09-13T18:11:29.601 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:11:29.601 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:11:29.602 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:11:29.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.716+0000 7fcefea4b700 1 Processor -- start 2024-09-13T18:11:29.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.716+0000 7fcefea4b700 1 -- start start 2024-09-13T18:11:29.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.716+0000 7fcefea4b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcef8070a70 0x7fcef8071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.716+0000 7fcefea4b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcef8071970 0x7fcef8156710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.716+0000 7fcefea4b700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef8159770 con 0x7fcef81503f0 2024-09-13T18:11:29.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.716+0000 7fcefea4b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef8159a30 con 0x7fcef8071970 2024-09-13T18:11:29.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.716+0000 7fcefea4b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8159d10 con 0x7fcef8070a70 2024-09-13T18:11:29.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.716+0000 7fcefe24a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcef8071970 0x7fcef8156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.716+0000 7fcefe24a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcef8071970 0x7fcef8156710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36956/0 (socket says 172.21.15.40:36956) 2024-09-13T18:11:29.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.716+0000 7fcefe24a700 1 -- 172.21.15.40:0/2906055916 learned_addr learned my addr 172.21.15.40:0/2906055916 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.716+0000 7fcefd248700 1 --2- 172.21.15.40:0/2906055916 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcef8070a70 0x7fcef8071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.716+0000 7fcefe24a700 1 -- 172.21.15.40:0/2906055916 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcef8070a70 msgr2=0x7fcef8071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:11:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.716+0000 7fcefe24a700 1 --2- 172.21.15.40:0/2906055916 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcef8070a70 0x7fcef8071430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.716+0000 7fcefe24a700 1 -- 172.21.15.40:0/2906055916 >> v1:172.21.15.43:6789/0 conn(0x7fcef81503f0 legacy=0x7fcef80c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.716+0000 7fcefe24a700 1 -- 172.21.15.40:0/2906055916 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef8159ff0 con 0x7fcef8071970 2024-09-13T18:11:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.716+0000 7fcefe24a700 1 --2- 172.21.15.40:0/2906055916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcef8071970 0x7fcef8156710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009e50 tx=0x7fcef000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5f4b6bcd279324 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.716+0000 7fcefca47700 1 -- 172.21.15.40:0/2906055916 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef000bbc0 con 0x7fcef8071970 2024-09-13T18:11:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.716+0000 7fcefca47700 1 -- 172.21.15.40:0/2906055916 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef000bd20 con 0x7fcef8071970 2024-09-13T18:11:29.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.716+0000 7fcefca47700 1 -- 172.21.15.40:0/2906055916 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0010710 con 0x7fcef8071970 2024-09-13T18:11:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefea4b700 1 -- 172.21.15.40:0/2906055916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcef8071970 msgr2=0x7fcef8156710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefea4b700 1 --2- 172.21.15.40:0/2906055916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcef8071970 0x7fcef8156710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009e50 tx=0x7fcef000b330 comp rx=0 tx=0).stop 2024-09-13T18:11:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefea4b700 1 -- 172.21.15.40:0/2906055916 shutdown_connections 2024-09-13T18:11:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefea4b700 1 --2- 172.21.15.40:0/2906055916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcef8071970 0x7fcef8156710 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefea4b700 1 --2- 172.21.15.40:0/2906055916 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcef8070a70 0x7fcef8071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefea4b700 1 -- 172.21.15.40:0/2906055916 >> 172.21.15.40:0/2906055916 conn(0x7fcef80cc7f0 msgr2=0x7fcef80c88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefea4b700 1 -- 172.21.15.40:0/2906055916 shutdown_connections 2024-09-13T18:11:29.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefea4b700 1 -- 172.21.15.40:0/2906055916 wait complete. 2024-09-13T18:11:29.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefea4b700 1 Processor -- start 2024-09-13T18:11:29.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefea4b700 1 -- start start 2024-09-13T18:11:29.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefea4b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcef8070a70 0x7fcef814e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefea4b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef8071970 0x7fcef81447a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefea4b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcef81503f0 0x7fcef8150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefea4b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef81431d0 con 0x7fcef81503f0 2024-09-13T18:11:29.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefea4b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef81ec2b0 con 0x7fcef8071970 2024-09-13T18:11:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefea4b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef81ec590 con 0x7fcef8070a70 2024-09-13T18:11:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefe24a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcef81503f0 0x7fcef8150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefe24a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcef81503f0 0x7fcef8150c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36970/0 (socket says 172.21.15.40:36970) 2024-09-13T18:11:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefe24a700 1 -- 172.21.15.40:0/2560510227 learned_addr learned my addr 172.21.15.40:0/2560510227 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefd248700 1 --2- 172.21.15.40:0/2560510227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef8071970 0x7fcef81447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.720+0000 7fcefda49700 1 --2- 172.21.15.40:0/2560510227 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcef8070a70 0x7fcef814e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.724+0000 7fcefe24a700 1 -- 172.21.15.40:0/2560510227 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcef8070a70 msgr2=0x7fcef814e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.724+0000 7fcefe24a700 1 --2- 172.21.15.40:0/2560510227 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcef8070a70 0x7fcef814e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.724+0000 7fcefe24a700 1 -- 172.21.15.40:0/2560510227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef8071970 msgr2=0x7fcef81447a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.724+0000 7fcefe24a700 1 --2- 172.21.15.40:0/2560510227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef8071970 0x7fcef81447a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.724+0000 7fcefe24a700 1 -- 172.21.15.40:0/2560510227 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef0009b30 con 0x7fcef81503f0 2024-09-13T18:11:29.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.724+0000 7fcefe24a700 1 --2- 172.21.15.40:0/2560510227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcef81503f0 0x7fcef8150c60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcef000b370 tx=0x7fcef0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:29.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.724+0000 7fceee7fc700 1 -- 172.21.15.40:0/2560510227 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef00099b0 con 0x7fcef81503f0 2024-09-13T18:11:29.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.724+0000 7fceee7fc700 1 -- 172.21.15.40:0/2560510227 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef00106e0 con 0x7fcef81503f0 2024-09-13T18:11:29.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.724+0000 7fcefea4b700 1 -- 172.21.15.40:0/2560510227 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef8143cc0 con 0x7fcef81503f0 2024-09-13T18:11:29.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.724+0000 7fceee7fc700 1 -- 172.21.15.40:0/2560510227 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0010ac0 con 0x7fcef81503f0 2024-09-13T18:11:29.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.724+0000 7fcefea4b700 1 -- 172.21.15.40:0/2560510227 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef8157590 con 0x7fcef81503f0 2024-09-13T18:11:29.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.724+0000 7fcefea4b700 1 --2- 172.21.15.40:0/2560510227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef8158480 0x7fcef8158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.724+0000 7fcefea4b700 1 -- 172.21.15.40:0/2560510227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8067560 con 0x7fcef8158480 2024-09-13T18:11:29.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.724+0000 7fcefda49700 1 --2- 172.21.15.40:0/2560510227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef8158480 0x7fcef8158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.724+0000 7fcefda49700 1 -- 172.21.15.40:0/2560510227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcef8156180 con 0x7fcef8158480 2024-09-13T18:11:29.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.724+0000 7fcefda49700 1 --2- 172.21.15.40:0/2560510227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef8158480 0x7fcef8158860 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcef400f770 tx=0x7fcef400fb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:29.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.724+0000 7fceee7fc700 1 -- 172.21.15.40:0/2560510227 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcef0010c20 con 0x7fcef81503f0 2024-09-13T18:11:29.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.724+0000 7fceee7fc700 1 --2- 172.21.15.40:0/2560510227 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcedc066730 0x7fcedc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.724+0000 7fcefd248700 1 --2- 172.21.15.40:0/2560510227 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcedc066730 0x7fcedc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.724+0000 7fceee7fc700 1 -- 172.21.15.40:0/2560510227 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcef008a110 con 0x7fcef81503f0 2024-09-13T18:11:29.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.728+0000 7fceee7fc700 1 -- 172.21.15.40:0/2560510227 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef40052e0 con 0x7fcef8158480 2024-09-13T18:11:29.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.728+0000 7fceee7fc700 1 -- 172.21.15.40:0/2560510227 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fcef4005440 con 0x7fcef8158480 2024-09-13T18:11:29.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.728+0000 7fceee7fc700 1 -- 172.21.15.40:0/2560510227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef8158480 msgr2=0x7fcef8158860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.728+0000 7fceee7fc700 1 --2- 172.21.15.40:0/2560510227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef8158480 0x7fcef8158860 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcef400f770 tx=0x7fcef400fb30 comp rx=0 tx=0).stop 2024-09-13T18:11:29.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.728+0000 7fceee7fc700 1 -- 172.21.15.40:0/2560510227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef8158480 msgr2=0x7fcef8158860 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:11:29.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.728+0000 7fceee7fc700 1 --2- 172.21.15.40:0/2560510227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef8158480 0x7fcef8158860 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.728+0000 7fcefd248700 1 --2- 172.21.15.40:0/2560510227 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcedc066730 0x7fcedc068bf0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcef80c96c0 tx=0x7fcee8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:29.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.736+0000 7fcefea4b700 1 --2- 172.21.15.40:0/2560510227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef8158da0 0x7fcef81eec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.736+0000 7fcefea4b700 1 -- 172.21.15.40:0/2560510227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8143e50 con 0x7fcef8158da0 2024-09-13T18:11:29.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.736+0000 7fcefda49700 1 --2- 172.21.15.40:0/2560510227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef8158da0 0x7fcef81eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.736+0000 7fcefda49700 1 -- 172.21.15.40:0/2560510227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcef815a5b0 con 0x7fcef8158da0 2024-09-13T18:11:29.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.736+0000 7fcefda49700 1 --2- 172.21.15.40:0/2560510227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef8158da0 0x7fcef81eec70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcef400f770 tx=0x7fcef4001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:29.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.736+0000 7fceee7fc700 1 -- 172.21.15.40:0/2560510227 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef4023070 con 0x7fcef8158da0 2024-09-13T18:11:29.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.736+0000 7fceee7fc700 1 -- 172.21.15.40:0/2560510227 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4451 (secure 0 0 0) 0x7fcef4027410 con 0x7fcef8158da0 2024-09-13T18:11:29.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.736+0000 7fceee7fc700 1 -- 172.21.15.40:0/2560510227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef8158da0 msgr2=0x7fcef81eec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.736+0000 7fceee7fc700 1 --2- 172.21.15.40:0/2560510227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef8158da0 0x7fcef81eec70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcef400f770 tx=0x7fcef4001120 comp rx=0 tx=0).stop 2024-09-13T18:11:29.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.736+0000 7fceee7fc700 1 -- 172.21.15.40:0/2560510227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef8158da0 msgr2=0x7fcef81eec70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:11:29.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.736+0000 7fceee7fc700 1 --2- 172.21.15.40:0/2560510227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef8158da0 0x7fcef81eec70 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.746 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:11:29.746 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:11:29.746 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:11:29.746 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:11:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 166, 2024-09-13T18:11:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:11:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:11:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:11:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:11:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 13, 2024-09-13T18:11:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:11:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:11:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:11:29.747 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:11:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:11:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:29.748 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:11:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:11:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:11:29.749 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:11:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:11:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:11:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:11:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:11:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:11:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:11:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:11:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:11:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:11:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:11:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:11:29.750 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:11:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:11:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:29.751 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:11:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:11:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:11:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:11:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:11:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:29.752 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:11:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:11:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:11:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:29.753 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:11:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:11:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:11:29.754 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:29.755 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.755 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.755 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:29.755 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:11:29.755 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:29.755 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.755 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:29.755 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.755 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:11:29.755 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:11:29.755 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:29.755 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:29.756 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:29.756 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.756 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.756 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:11:29.756 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:11:29.756 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:29.756 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:29.756 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.756 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:29.756 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:11:29.756 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:29.756 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.756 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.756 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:29.757 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:11:29.757 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:29.757 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.757 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:29.757 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.757 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:11:29.757 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:11:29.757 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:29.758 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:29.758 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:29.758 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.758 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:29.758 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.758 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:11:29.758 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:11:29.758 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.758 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:29.758 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:29.758 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:11:29.758 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.758 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:29.758 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:11:29.758 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.759 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:29.759 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:29.759 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:11:29.759 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.759 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:29.759 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.759 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:11:29.759 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:11:29.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.736+0000 7fcefea4b700 1 -- 172.21.15.40:0/2560510227 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcedc066730 msgr2=0x7fcedc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.736+0000 7fcefea4b700 1 --2- 172.21.15.40:0/2560510227 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcedc066730 0x7fcedc068bf0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcef80c96c0 tx=0x7fcee8008040 comp rx=0 tx=0).stop 2024-09-13T18:11:29.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.736+0000 7fcefea4b700 1 -- 172.21.15.40:0/2560510227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcef81503f0 msgr2=0x7fcef8150c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.736+0000 7fcefea4b700 1 --2- 172.21.15.40:0/2560510227 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcef81503f0 0x7fcef8150c60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcef000b370 tx=0x7fcef0004af0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.736+0000 7fcefe24a700 1 -- 172.21.15.40:0/2560510227 reap_dead start 2024-09-13T18:11:29.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.736+0000 7fcefe24a700 1 -- 172.21.15.40:0/2560510227 reap_dead start 2024-09-13T18:11:29.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.740+0000 7fcefea4b700 1 -- 172.21.15.40:0/2560510227 shutdown_connections 2024-09-13T18:11:29.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.740+0000 7fcefea4b700 1 -- 172.21.15.40:0/2560510227 >> 172.21.15.40:0/2560510227 conn(0x7fcef80cc7f0 msgr2=0x7fcef8071050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:29.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.740+0000 7fcefea4b700 1 -- 172.21.15.40:0/2560510227 shutdown_connections 2024-09-13T18:11:29.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.740+0000 7fcefea4b700 1 -- 172.21.15.40:0/2560510227 wait complete. 2024-09-13T18:11:29.763 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:11:29.763 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:11:29.763 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:11:29.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.868+0000 7f447eb36700 1 Processor -- start 2024-09-13T18:11:29.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.868+0000 7f447eb36700 1 -- start start 2024-09-13T18:11:29.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.868+0000 7f447eb36700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4478151050 0x7f4478066220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.868+0000 7f447eb36700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4478157d40 0x7f44780c2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.868+0000 7f447eb36700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44781578b0 con 0x7f44780cb2a0 2024-09-13T18:11:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.868+0000 7f447eb36700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4478157090 con 0x7f4478157d40 2024-09-13T18:11:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.868+0000 7f447eb36700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44780c6630 con 0x7f4478151050 2024-09-13T18:11:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.868+0000 7f447d333700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4478157d40 0x7f44780c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.868+0000 7f447d333700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4478157d40 0x7f44780c2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36976/0 (socket says 172.21.15.40:36976) 2024-09-13T18:11:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.868+0000 7f447d333700 1 -- 172.21.15.40:0/3190341682 learned_addr learned my addr 172.21.15.40:0/3190341682 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.868+0000 7f447db34700 1 --2- 172.21.15.40:0/3190341682 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4478151050 0x7f4478066220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.868+0000 7f447d333700 1 -- 172.21.15.40:0/3190341682 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4478151050 msgr2=0x7f4478066220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:11:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.868+0000 7f447d333700 1 --2- 172.21.15.40:0/3190341682 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4478151050 0x7f4478066220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.868+0000 7f447d333700 1 -- 172.21.15.40:0/3190341682 >> v1:172.21.15.43:6789/0 conn(0x7f44780cb2a0 legacy=0x7f44780cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.868+0000 7f447d333700 1 -- 172.21.15.40:0/3190341682 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447806b290 con 0x7f4478157d40 2024-09-13T18:11:29.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.868+0000 7f447d333700 1 --2- 172.21.15.40:0/3190341682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4478157d40 0x7f44780c2850 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4468009910 tx=0x7f4468009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5395edb061f962db server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:29.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.868+0000 7f447cb32700 1 -- 172.21.15.40:0/3190341682 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446800ba60 con 0x7f4478157d40 2024-09-13T18:11:29.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.868+0000 7f447cb32700 1 -- 172.21.15.40:0/3190341682 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446800bbc0 con 0x7f4478157d40 2024-09-13T18:11:29.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.868+0000 7f447cb32700 1 -- 172.21.15.40:0/3190341682 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468009e60 con 0x7f4478157d40 2024-09-13T18:11:29.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447eb36700 1 -- 172.21.15.40:0/3190341682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4478157d40 msgr2=0x7f44780c2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447eb36700 1 --2- 172.21.15.40:0/3190341682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4478157d40 0x7f44780c2850 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4468009910 tx=0x7f4468009c20 comp rx=0 tx=0).stop 2024-09-13T18:11:29.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447eb36700 1 -- 172.21.15.40:0/3190341682 shutdown_connections 2024-09-13T18:11:29.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447eb36700 1 --2- 172.21.15.40:0/3190341682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4478157d40 0x7f44780c2850 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447eb36700 1 --2- 172.21.15.40:0/3190341682 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4478151050 0x7f4478066220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447eb36700 1 -- 172.21.15.40:0/3190341682 >> 172.21.15.40:0/3190341682 conn(0x7f447806a390 msgr2=0x7f447806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:29.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447eb36700 1 -- 172.21.15.40:0/3190341682 shutdown_connections 2024-09-13T18:11:29.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447eb36700 1 -- 172.21.15.40:0/3190341682 wait complete. 2024-09-13T18:11:29.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447eb36700 1 Processor -- start 2024-09-13T18:11:29.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447eb36700 1 -- start start 2024-09-13T18:11:29.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447eb36700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44780cb2a0 0x7f4478155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447eb36700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4478151050 0x7f44780ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447eb36700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4478157d40 0x7f44781464f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447eb36700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44781edd90 con 0x7f4478151050 2024-09-13T18:11:29.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447eb36700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4478154cd0 con 0x7f4478157d40 2024-09-13T18:11:29.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447eb36700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4478156560 con 0x7f44780cb2a0 2024-09-13T18:11:29.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447d333700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4478151050 0x7f44780ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447d333700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4478151050 0x7f44780ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36978/0 (socket says 172.21.15.40:36978) 2024-09-13T18:11:29.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447d333700 1 -- 172.21.15.40:0/1990344607 learned_addr learned my addr 172.21.15.40:0/1990344607 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:29.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447db34700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44780cb2a0 0x7f4478155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447e335700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4478157d40 0x7f44781464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447d333700 1 -- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44780cb2a0 msgr2=0x7f4478155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447d333700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44780cb2a0 0x7f4478155720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447d333700 1 -- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4478157d40 msgr2=0x7f44781464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447d333700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4478157d40 0x7f44781464f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447d333700 1 -- 172.21.15.40:0/1990344607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4468009570 con 0x7f4478151050 2024-09-13T18:11:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447db34700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44780cb2a0 0x7f4478155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:11:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447d333700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4478151050 0x7f44780ca410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4468000c00 tx=0x7f4468004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f446e7fc700 1 -- 172.21.15.40:0/1990344607 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468004610 con 0x7f4478151050 2024-09-13T18:11:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f446e7fc700 1 -- 172.21.15.40:0/1990344607 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446801e070 con 0x7f4478151050 2024-09-13T18:11:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f446e7fc700 1 -- 172.21.15.40:0/1990344607 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468003900 con 0x7f4478151050 2024-09-13T18:11:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.872+0000 7f447eb36700 1 -- 172.21.15.40:0/1990344607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4478156840 con 0x7f4478151050 2024-09-13T18:11:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.876+0000 7f447eb36700 1 -- 172.21.15.40:0/1990344607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4478155200 con 0x7f4478151050 2024-09-13T18:11:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.876+0000 7f447eb36700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4478148b50 0x7f4478146a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.876+0000 7f447eb36700 1 -- 172.21.15.40:0/1990344607 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4478067600 con 0x7f4478148b50 2024-09-13T18:11:29.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.876+0000 7f447db34700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4478148b50 0x7f4478146a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.876+0000 7f446e7fc700 1 -- 172.21.15.40:0/1990344607 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4468003a60 con 0x7f4478151050 2024-09-13T18:11:29.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.876+0000 7f447db34700 1 -- 172.21.15.40:0/1990344607 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44780c2d90 con 0x7f4478148b50 2024-09-13T18:11:29.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.876+0000 7f446e7fc700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f445c066760 0x7f445c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.876+0000 7f446e7fc700 1 -- 172.21.15.40:0/1990344607 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4468020070 con 0x7f4478151050 2024-09-13T18:11:29.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.876+0000 7f447e335700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f445c066760 0x7f445c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.876+0000 7f447db34700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4478148b50 0x7f4478146a30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4474010830 tx=0x7f4474010bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:29.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.876+0000 7f446e7fc700 1 -- 172.21.15.40:0/1990344607 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474004020 con 0x7f4478148b50 2024-09-13T18:11:29.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.876+0000 7f447e335700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f445c066760 0x7f445c068c20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4478149140 tx=0x7f4470009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:29.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.876+0000 7f446e7fc700 1 -- 172.21.15.40:0/1990344607 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f447400c050 con 0x7f4478148b50 2024-09-13T18:11:29.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.876+0000 7f446e7fc700 1 -- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4478148b50 msgr2=0x7f4478146a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.876+0000 7f446e7fc700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4478148b50 0x7f4478146a30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4474010830 tx=0x7f4474010bf0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.876+0000 7f446e7fc700 1 -- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4478148b50 msgr2=0x7f4478146a30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:11:29.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.876+0000 7f446e7fc700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4478148b50 0x7f4478146a30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.888+0000 7f447eb36700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4478156c10 0x7f44781f0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:29.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.888+0000 7f447eb36700 1 -- 172.21.15.40:0/1990344607 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4478146f70 con 0x7f4478156c10 2024-09-13T18:11:29.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.888+0000 7f447db34700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4478156c10 0x7f44781f0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:29.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.892+0000 7f447db34700 1 -- 172.21.15.40:0/1990344607 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4478066760 con 0x7f4478156c10 2024-09-13T18:11:29.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.892+0000 7f447db34700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4478156c10 0x7f44781f0340 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f447401a000 tx=0x7f4474004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:29.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.892+0000 7f446e7fc700 1 -- 172.21.15.40:0/1990344607 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474020220 con 0x7f4478156c10 2024-09-13T18:11:29.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.892+0000 7f446e7fc700 1 -- 172.21.15.40:0/1990344607 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4949 (secure 0 0 0) 0x7f4474003b20 con 0x7f4478156c10 2024-09-13T18:11:29.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.892+0000 7f446e7fc700 1 -- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4478156c10 msgr2=0x7f44781f0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.892+0000 7f446e7fc700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4478156c10 0x7f44781f0340 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f447401a000 tx=0x7f4474004770 comp rx=0 tx=0).stop 2024-09-13T18:11:29.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.892+0000 7f446e7fc700 1 -- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4478156c10 msgr2=0x7f44781f0340 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:11:29.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.892+0000 7f446e7fc700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4478156c10 0x7f44781f0340 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:29.900 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:11:29.900 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:11:29.900 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:11:29.901 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:11:29.901 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 166, 2024-09-13T18:11:29.901 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:11:29.901 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:11:29.901 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:11:29.901 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:11:29.901 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:29.901 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 13, 2024-09-13T18:11:29.901 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:11:29.901 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:11:29.901 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:11:29.902 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:29.902 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:29.902 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:29.902 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:29.902 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:29.902 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:29.902 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:29.902 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:29.902 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:29.902 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:29.903 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:29.903 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:11:29.903 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:11:29.903 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:29.903 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:29.903 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:29.903 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:29.903 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:29.903 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:29.903 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:29.903 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:29.903 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:29.903 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:29.904 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:29.904 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.904 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:11:29.904 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:11:29.904 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:11:29.904 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:11:29.904 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:11:29.904 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:11:29.904 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:11:29.904 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:11:29.904 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:11:29.904 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:11:29.905 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:11:29.905 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:11:29.905 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:11:29.905 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:11:29.905 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:11:29.905 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:11:29.905 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:11:29.905 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:29.905 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:29.905 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:29.905 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:29.906 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:29.906 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:29.906 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:29.906 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:29.906 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:29.906 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:29.906 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:29.906 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:11:29.906 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.906 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:11:29.906 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.906 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:11:29.907 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:11:29.907 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:29.907 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:29.907 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.907 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:29.907 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:11:29.907 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:29.907 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.907 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.907 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:29.907 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:11:29.907 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:29.907 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.907 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:29.908 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.908 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:11:29.908 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:11:29.908 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:29.908 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:29.908 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:29.908 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.908 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.908 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:11:29.908 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:11:29.908 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:29.909 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:29.909 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.909 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:29.909 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:11:29.909 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:29.909 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.909 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.909 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:29.909 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:11:29.909 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:29.909 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.909 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:29.909 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.910 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:11:29.910 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:11:29.910 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:29.910 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:29.910 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:29.910 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.910 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.910 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:11:29.910 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:11:29.910 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:29.910 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:29.910 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.910 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:29.910 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:11:29.910 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:29.911 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.911 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.911 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:29.911 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:11:29.911 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:29.911 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.911 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:29.911 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.911 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:11:29.911 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:11:29.911 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:29.911 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:29.912 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:29.912 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.912 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:29.912 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.912 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:11:29.912 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:11:29.912 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.912 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:29.912 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:29.912 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:11:29.912 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.912 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:29.912 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:11:29.913 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.913 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:29.913 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:29.913 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:11:29.913 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.913 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:29.913 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:11:29.913 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.913 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:29.913 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:29.913 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:11:29.913 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.913 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:29.914 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:11:29.914 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.914 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:29.914 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:29.914 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:11:29.914 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.914 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:29.914 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:11:29.914 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:29.914 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:29.915 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:29.915 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:11:29.915 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:29.915 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:29.915 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:29.915 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:11:29.915 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:11:29.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.892+0000 7f447eb36700 1 -- 172.21.15.40:0/1990344607 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f445c066760 msgr2=0x7f445c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.892+0000 7f447eb36700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f445c066760 0x7f445c068c20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4478149140 tx=0x7f4470009040 comp rx=0 tx=0).stop 2024-09-13T18:11:29.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.892+0000 7f447eb36700 1 -- 172.21.15.40:0/1990344607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4478151050 msgr2=0x7f44780ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:29.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.892+0000 7f447eb36700 1 --2- 172.21.15.40:0/1990344607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4478151050 0x7f44780ca410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4468000c00 tx=0x7f4468004770 comp rx=0 tx=0).stop 2024-09-13T18:11:29.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.892+0000 7f447e335700 1 -- 172.21.15.40:0/1990344607 reap_dead start 2024-09-13T18:11:29.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.892+0000 7f447e335700 1 -- 172.21.15.40:0/1990344607 reap_dead start 2024-09-13T18:11:29.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.892+0000 7f447eb36700 1 -- 172.21.15.40:0/1990344607 shutdown_connections 2024-09-13T18:11:29.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.892+0000 7f447eb36700 1 -- 172.21.15.40:0/1990344607 >> 172.21.15.40:0/1990344607 conn(0x7f447806a390 msgr2=0x7f447806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:29.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.892+0000 7f447eb36700 1 -- 172.21.15.40:0/1990344607 shutdown_connections 2024-09-13T18:11:29.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:29.892+0000 7f447eb36700 1 -- 172.21.15.40:0/1990344607 wait complete. 2024-09-13T18:11:29.917 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:11:29.918 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-13T18:11:29.918 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T18:11:29.918 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-13T18:11:29.918 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T18:11:29.918 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T18:11:29.918 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:11:29.957 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T18:11:29.957 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T18:11:29.958 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:11:30.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.068+0000 7f422555e700 1 Processor -- start 2024-09-13T18:11:30.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.068+0000 7f422555e700 1 -- start start 2024-09-13T18:11:30.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f422555e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4220151470 0x7f42200c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:30.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f422555e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f42200b1bd0 0x7f42201571d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:30.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f422555e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4220157710 0x7f4220071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:30.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f422555e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42200c4520 con 0x7f4220151470 2024-09-13T18:11:30.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f422555e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42200c4dc0 con 0x7f42200b1bd0 2024-09-13T18:11:30.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f422555e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4220157cf0 con 0x7f4220157710 2024-09-13T18:11:30.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f421ffff700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f4220151470 msgr2=0x7f42200c5250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:11:30.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f421ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4220151470 0x7f42200c5250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:11:30.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f4224d5d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4220157710 0x7f4220071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:30.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f421f7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f42200b1bd0 0x7f42201571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:30.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f421f7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f42200b1bd0 0x7f42201571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:43346/0 (socket says 172.21.15.40:43346) 2024-09-13T18:11:30.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f421f7fe700 1 -- 172.21.15.40:0/1153501835 learned_addr learned my addr 172.21.15.40:0/1153501835 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:30.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f4224d5d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4220157710 0x7f4220071bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:56568/0 (socket says 172.21.15.40:56568) 2024-09-13T18:11:30.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f421f7fe700 1 -- 172.21.15.40:0/1153501835 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4220157710 msgr2=0x7f4220071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:11:30.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f421f7fe700 1 --2- 172.21.15.40:0/1153501835 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4220157710 0x7f4220071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:30.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f421f7fe700 1 -- 172.21.15.40:0/1153501835 >> v2:172.21.15.40:3300/0 conn(0x7f4220151470 msgr2=0x7f42200c5250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:11:30.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f421f7fe700 1 --2- 172.21.15.40:0/1153501835 >> v2:172.21.15.40:3300/0 conn(0x7f4220151470 0x7f42200c5250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:30.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f421f7fe700 1 -- 172.21.15.40:0/1153501835 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4220157fd0 con 0x7f42200b1bd0 2024-09-13T18:11:30.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f421f7fe700 1 --2- 172.21.15.40:0/1153501835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42200b1bd0 0x7f42201571d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f42200c3a30 tx=0x7f4214009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5042ca64e0e1891f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:30.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f421effd700 1 -- 172.21.15.40:0/1153501835 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4214011830 con 0x7f42200b1bd0 2024-09-13T18:11:30.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f421effd700 1 -- 172.21.15.40:0/1153501835 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4214011990 con 0x7f42200b1bd0 2024-09-13T18:11:30.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f421effd700 1 -- 172.21.15.40:0/1153501835 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4214011d20 con 0x7f42200b1bd0 2024-09-13T18:11:30.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f422555e700 1 -- 172.21.15.40:0/1153501835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42200b1bd0 msgr2=0x7f42201571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:30.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f422555e700 1 --2- 172.21.15.40:0/1153501835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42200b1bd0 0x7f42201571d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f42200c3a30 tx=0x7f4214009bf0 comp rx=0 tx=0).stop 2024-09-13T18:11:30.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f422555e700 1 -- 172.21.15.40:0/1153501835 shutdown_connections 2024-09-13T18:11:30.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f422555e700 1 --2- 172.21.15.40:0/1153501835 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4220157710 0x7f4220071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:30.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f422555e700 1 --2- 172.21.15.40:0/1153501835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42200b1bd0 0x7f42201571d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:30.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f422555e700 1 --2- 172.21.15.40:0/1153501835 >> v2:172.21.15.40:3300/0 conn(0x7f4220151470 0x7f42200c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:30.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f422555e700 1 -- 172.21.15.40:0/1153501835 >> 172.21.15.40:0/1153501835 conn(0x7f422006e4f0 msgr2=0x7f422006e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:30.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f422555e700 1 -- 172.21.15.40:0/1153501835 shutdown_connections 2024-09-13T18:11:30.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.072+0000 7f422555e700 1 -- 172.21.15.40:0/1153501835 wait complete. 2024-09-13T18:11:30.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.076+0000 7f422555e700 1 Processor -- start 2024-09-13T18:11:30.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.076+0000 7f422555e700 1 -- start start 2024-09-13T18:11:30.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.076+0000 7f422555e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42200b1bd0 0x7f42200c0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:30.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.076+0000 7f422555e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4220151470 0x7f42200cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:30.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.076+0000 7f421ffff700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42200b1bd0 msgr2=0x7f42200c0e90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:11:30.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.076+0000 7f422555e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4220157710 0x7f42201e1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:30.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.076+0000 7f421ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42200b1bd0 0x7f42200c0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:11:30.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.076+0000 7f422555e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42201e2550 con 0x7f42200b1bd0 2024-09-13T18:11:30.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.076+0000 7f422555e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42201e2810 con 0x7f4220151470 2024-09-13T18:11:30.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.076+0000 7f422555e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4220154180 con 0x7f4220157710 2024-09-13T18:11:30.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.076+0000 7f421f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4220151470 0x7f42200cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:30.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.076+0000 7f421f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4220151470 0x7f42200cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:43354/0 (socket says 172.21.15.40:43354) 2024-09-13T18:11:30.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.076+0000 7f421f7fe700 1 -- 172.21.15.40:0/2430272532 learned_addr learned my addr 172.21.15.40:0/2430272532 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:30.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.076+0000 7f4224d5d700 1 --2- 172.21.15.40:0/2430272532 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4220157710 0x7f42201e1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:30.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.080+0000 7f421f7fe700 1 -- 172.21.15.40:0/2430272532 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4220157710 msgr2=0x7f42201e1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:30.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.080+0000 7f421f7fe700 1 --2- 172.21.15.40:0/2430272532 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4220157710 0x7f42201e1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:30.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.080+0000 7f421f7fe700 1 -- 172.21.15.40:0/2430272532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42200b1bd0 msgr2=0x7f42200c0e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:11:30.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.080+0000 7f421f7fe700 1 --2- 172.21.15.40:0/2430272532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42200b1bd0 0x7f42200c0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:30.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.080+0000 7f421f7fe700 1 -- 172.21.15.40:0/2430272532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4214009570 con 0x7f4220151470 2024-09-13T18:11:30.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.080+0000 7f421f7fe700 1 --2- 172.21.15.40:0/2430272532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4220151470 0x7f42200cb0d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f42200c4f00 tx=0x7f4214004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:30.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.080+0000 7f421cff9700 1 -- 172.21.15.40:0/2430272532 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4214004510 con 0x7f4220151470 2024-09-13T18:11:30.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.080+0000 7f422555e700 1 -- 172.21.15.40:0/2430272532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42200c13d0 con 0x7f4220151470 2024-09-13T18:11:30.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.080+0000 7f421cff9700 1 -- 172.21.15.40:0/2430272532 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421401e070 con 0x7f4220151470 2024-09-13T18:11:30.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.080+0000 7f421cff9700 1 -- 172.21.15.40:0/2430272532 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4214003900 con 0x7f4220151470 2024-09-13T18:11:30.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.080+0000 7f422555e700 1 -- 172.21.15.40:0/2430272532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42200c2960 con 0x7f4220151470 2024-09-13T18:11:30.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.080+0000 7f422555e700 1 -- 172.21.15.40:0/2430272532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f422006a430 con 0x7f4220151470 2024-09-13T18:11:30.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.084+0000 7f421cff9700 1 -- 172.21.15.40:0/2430272532 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4214003a60 con 0x7f4220151470 2024-09-13T18:11:30.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.084+0000 7f421cff9700 1 --2- 172.21.15.40:0/2430272532 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f420c0666e0 0x7f420c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:30.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.084+0000 7f421ffff700 1 --2- 172.21.15.40:0/2430272532 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f420c0666e0 0x7f420c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:30.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.084+0000 7f421cff9700 1 -- 172.21.15.40:0/2430272532 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4214020070 con 0x7f4220151470 2024-09-13T18:11:30.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.084+0000 7f421ffff700 1 --2- 172.21.15.40:0/2430272532 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f420c0666e0 0x7f420c068ba0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4210003db0 tx=0x7f4210009270 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:30.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.092+0000 7f421cff9700 1 -- 172.21.15.40:0/2430272532 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4214052870 con 0x7f4220151470 2024-09-13T18:11:30.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.376+0000 7f422555e700 1 -- 172.21.15.40:0/2430272532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f42201571d0 con 0x7f4220151470 2024-09-13T18:11:30.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.376+0000 7f421cff9700 1 -- 172.21.15.40:0/2430272532 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f4214055e90 con 0x7f4220151470 2024-09-13T18:11:30.384 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":166,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:11:30.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.380+0000 7f422555e700 1 -- 172.21.15.40:0/2430272532 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f420c0666e0 msgr2=0x7f420c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:30.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.380+0000 7f422555e700 1 --2- 172.21.15.40:0/2430272532 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f420c0666e0 0x7f420c068ba0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4210003db0 tx=0x7f4210009270 comp rx=0 tx=0).stop 2024-09-13T18:11:30.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.380+0000 7f422555e700 1 -- 172.21.15.40:0/2430272532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4220151470 msgr2=0x7f42200cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:30.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.380+0000 7f422555e700 1 --2- 172.21.15.40:0/2430272532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4220151470 0x7f42200cb0d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f42200c4f00 tx=0x7f4214004770 comp rx=0 tx=0).stop 2024-09-13T18:11:30.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.380+0000 7f422555e700 1 -- 172.21.15.40:0/2430272532 shutdown_connections 2024-09-13T18:11:30.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.380+0000 7f422555e700 1 --2- 172.21.15.40:0/2430272532 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f420c0666e0 0x7f420c068ba0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.380+0000 7f422555e700 1 --2- 172.21.15.40:0/2430272532 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4220157710 0x7f42201e1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.380+0000 7f422555e700 1 --2- 172.21.15.40:0/2430272532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4220151470 0x7f42200cb0d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.380+0000 7f422555e700 1 --2- 172.21.15.40:0/2430272532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42200b1bd0 0x7f42200c0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.380+0000 7f422555e700 1 -- 172.21.15.40:0/2430272532 >> 172.21.15.40:0/2430272532 conn(0x7f422006e4f0 msgr2=0x7f42201565d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.380+0000 7f422555e700 1 -- 172.21.15.40:0/2430272532 shutdown_connections 2024-09-13T18:11:30.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:30.380+0000 7f422555e700 1 -- 172.21.15.40:0/2430272532 wait complete. 2024-09-13T18:11:33.401 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:11:33.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.520+0000 7fb56a85d700 1 Processor -- start 2024-09-13T18:11:33.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.520+0000 7fb56a85d700 1 -- start start 2024-09-13T18:11:33.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.520+0000 7fb56a85d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb55c04ca40 0x7fb55c046f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.520+0000 7fb56a85d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb55c04af10 0x7fb55c0df0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.520+0000 7fb56a85d700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55c0075c0 con 0x7fb55c0d8a20 2024-09-13T18:11:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.520+0000 7fb56a85d700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb55c04b360 con 0x7fb55c04af10 2024-09-13T18:11:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.520+0000 7fb56a85d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c045470 con 0x7fb55c04ca40 2024-09-13T18:11:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.520+0000 7fb56a05c700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7fb55c04af10 msgr2=0x7fb55c0df0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:11:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.520+0000 7fb56a05c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb55c04af10 0x7fb55c0df0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:11:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.520+0000 7fb56905a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb55c04ca40 0x7fb55c046f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.520+0000 7fb56985b700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb55c0d8a20 0x7fb55c0068a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:45736/0 (socket says 172.21.15.40:45736) 2024-09-13T18:11:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.520+0000 7fb56985b700 1 -- 172.21.15.40:0/434419461 learned_addr learned my addr 172.21.15.40:0/434419461 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:11:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.520+0000 7fb56905a700 1 -- 172.21.15.40:0/434419461 >> v2:172.21.15.40:3300/0 conn(0x7fb55c04af10 msgr2=0x7fb55c0df0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:11:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.520+0000 7fb56905a700 1 --2- 172.21.15.40:0/434419461 >> v2:172.21.15.40:3300/0 conn(0x7fb55c04af10 0x7fb55c0df0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.520+0000 7fb56905a700 1 -- 172.21.15.40:0/434419461 >> v1:172.21.15.43:6789/0 conn(0x7fb55c0d8a20 legacy=0x7fb55c0068a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:33.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.520+0000 7fb56905a700 1 -- 172.21.15.40:0/434419461 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c046b00 con 0x7fb55c04ca40 2024-09-13T18:11:33.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.524+0000 7fb56905a700 1 --2- 172.21.15.40:0/434419461 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb55c04ca40 0x7fb55c046f90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb554009850 tx=0x7fb554009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=123eb018da3c0698 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:33.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.524+0000 7fb568859700 1 -- 172.21.15.40:0/434419461 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb554011830 con 0x7fb55c04ca40 2024-09-13T18:11:33.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.524+0000 7fb568859700 1 -- 172.21.15.40:0/434419461 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb554011990 con 0x7fb55c04ca40 2024-09-13T18:11:33.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.524+0000 7fb568859700 1 -- 172.21.15.40:0/434419461 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb554011d20 con 0x7fb55c04ca40 2024-09-13T18:11:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.524+0000 7fb56a85d700 1 -- 172.21.15.40:0/434419461 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb55c04ca40 msgr2=0x7fb55c046f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.524+0000 7fb56a85d700 1 --2- 172.21.15.40:0/434419461 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb55c04ca40 0x7fb55c046f90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb554009850 tx=0x7fb554009b60 comp rx=0 tx=0).stop 2024-09-13T18:11:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.524+0000 7fb56a85d700 1 -- 172.21.15.40:0/434419461 shutdown_connections 2024-09-13T18:11:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.524+0000 7fb56a85d700 1 --2- 172.21.15.40:0/434419461 >> v2:172.21.15.40:3300/0 conn(0x7fb55c04af10 0x7fb55c0df0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.524+0000 7fb56a85d700 1 --2- 172.21.15.40:0/434419461 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb55c04ca40 0x7fb55c046f90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.524+0000 7fb56a85d700 1 -- 172.21.15.40:0/434419461 >> 172.21.15.40:0/434419461 conn(0x7fb55c050590 msgr2=0x7fb55c0509a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.524+0000 7fb56a85d700 1 -- 172.21.15.40:0/434419461 shutdown_connections 2024-09-13T18:11:33.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.524+0000 7fb56a85d700 1 -- 172.21.15.40:0/434419461 wait complete. 2024-09-13T18:11:33.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.524+0000 7fb56a85d700 1 Processor -- start 2024-09-13T18:11:33.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.524+0000 7fb56a85d700 1 -- start start 2024-09-13T18:11:33.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56a85d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c04af10 0x7fb55c0d8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:33.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56a85d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb55c04ca40 0x7fb55c0d6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:33.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56a85d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb55c0d8a20 0x7fb55c174270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:33.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56905a700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb55c04ca40 msgr2=0x7fb55c0d6b90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:11:33.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56a85d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c0492a0 con 0x7fb55c04ca40 2024-09-13T18:11:33.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56905a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb55c04ca40 0x7fb55c0d6b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:11:33.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56a85d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c047a50 con 0x7fb55c04af10 2024-09-13T18:11:33.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56a85d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c047d30 con 0x7fb55c0d8a20 2024-09-13T18:11:33.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56985b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c04af10 0x7fb55c0d8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:33.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56985b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c04af10 0x7fb55c0d8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:46486/0 (socket says 172.21.15.40:46486) 2024-09-13T18:11:33.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56a05c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb55c0d8a20 0x7fb55c174270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:33.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56985b700 1 -- 172.21.15.40:0/40095799 learned_addr learned my addr 172.21.15.40:0/40095799 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:33.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56985b700 1 -- 172.21.15.40:0/40095799 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb55c0d8a20 msgr2=0x7fb55c174270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:33.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56985b700 1 --2- 172.21.15.40:0/40095799 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb55c0d8a20 0x7fb55c174270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:33.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56985b700 1 -- 172.21.15.40:0/40095799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb55c04ca40 msgr2=0x7fb55c0d6b90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:11:33.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56985b700 1 --2- 172.21.15.40:0/40095799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb55c04ca40 0x7fb55c0d6b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:33.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56985b700 1 -- 172.21.15.40:0/40095799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5540094b0 con 0x7fb55c04af10 2024-09-13T18:11:33.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56a05c700 1 --2- 172.21.15.40:0/40095799 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb55c0d8a20 0x7fb55c174270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:11:33.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56985b700 1 --2- 172.21.15.40:0/40095799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c04af10 0x7fb55c0d8fd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb560001eb0 tx=0x7fb56000c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:33.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb55a7fc700 1 -- 172.21.15.40:0/40095799 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb560005d90 con 0x7fb55c04af10 2024-09-13T18:11:33.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb55a7fc700 1 -- 172.21.15.40:0/40095799 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5600070d0 con 0x7fb55c04af10 2024-09-13T18:11:33.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56a85d700 1 -- 172.21.15.40:0/40095799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55c048c50 con 0x7fb55c04af10 2024-09-13T18:11:33.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb55a7fc700 1 -- 172.21.15.40:0/40095799 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb560007440 con 0x7fb55c04af10 2024-09-13T18:11:33.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.528+0000 7fb56a85d700 1 -- 172.21.15.40:0/40095799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55c048010 con 0x7fb55c04af10 2024-09-13T18:11:33.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.532+0000 7fb56a85d700 1 -- 172.21.15.40:0/40095799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb55c046790 con 0x7fb55c04af10 2024-09-13T18:11:33.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.532+0000 7fb55a7fc700 1 -- 172.21.15.40:0/40095799 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb5600075a0 con 0x7fb55c04af10 2024-09-13T18:11:33.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.532+0000 7fb55a7fc700 1 --2- 172.21.15.40:0/40095799 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb548066710 0x7fb548068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:33.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.532+0000 7fb55a7fc700 1 -- 172.21.15.40:0/40095799 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb5600836f0 con 0x7fb55c04af10 2024-09-13T18:11:33.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.532+0000 7fb56905a700 1 --2- 172.21.15.40:0/40095799 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb548066710 0x7fb548068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:33.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.532+0000 7fb56905a700 1 --2- 172.21.15.40:0/40095799 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb548066710 0x7fb548068bd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0060d0 tx=0x7fb554008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:33.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.540+0000 7fb55a7fc700 1 -- 172.21.15.40:0/40095799 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb5600507e0 con 0x7fb55c04af10 2024-09-13T18:11:33.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.792+0000 7fb56a85d700 1 -- 172.21.15.40:0/40095799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fb55c0dbf70 con 0x7fb55c04af10 2024-09-13T18:11:33.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.796+0000 7fb55a7fc700 1 -- 172.21.15.40:0/40095799 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7fb560053e00 con 0x7fb55c04af10 2024-09-13T18:11:33.803 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":166,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:11:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.796+0000 7fb56a85d700 1 -- 172.21.15.40:0/40095799 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb548066710 msgr2=0x7fb548068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.796+0000 7fb56a85d700 1 --2- 172.21.15.40:0/40095799 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb548066710 0x7fb548068bd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0060d0 tx=0x7fb554008030 comp rx=0 tx=0).stop 2024-09-13T18:11:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.796+0000 7fb56a85d700 1 -- 172.21.15.40:0/40095799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c04af10 msgr2=0x7fb55c0d8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:33.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.796+0000 7fb56a85d700 1 --2- 172.21.15.40:0/40095799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c04af10 0x7fb55c0d8fd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb560001eb0 tx=0x7fb56000c7b0 comp rx=0 tx=0).stop 2024-09-13T18:11:33.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.796+0000 7fb56a85d700 1 -- 172.21.15.40:0/40095799 shutdown_connections 2024-09-13T18:11:33.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.796+0000 7fb56a85d700 1 --2- 172.21.15.40:0/40095799 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb548066710 0x7fb548068bd0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:33.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.796+0000 7fb56a85d700 1 --2- 172.21.15.40:0/40095799 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb55c0d8a20 0x7fb55c174270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:33.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.796+0000 7fb56a85d700 1 --2- 172.21.15.40:0/40095799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb55c04ca40 0x7fb55c0d6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:33.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.796+0000 7fb56a85d700 1 --2- 172.21.15.40:0/40095799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb55c04af10 0x7fb55c0d8fd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:33.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.796+0000 7fb56a85d700 1 -- 172.21.15.40:0/40095799 >> 172.21.15.40:0/40095799 conn(0x7fb55c050590 msgr2=0x7fb55c0d77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:33.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.796+0000 7fb56a85d700 1 -- 172.21.15.40:0/40095799 shutdown_connections 2024-09-13T18:11:33.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:33.796+0000 7fb56a85d700 1 -- 172.21.15.40:0/40095799 wait complete. 2024-09-13T18:11:36.817 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:11:36.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.940+0000 7fe7ceb61700 1 Processor -- start 2024-09-13T18:11:36.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.940+0000 7fe7ceb61700 1 -- start start 2024-09-13T18:11:36.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.940+0000 7fe7ceb61700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe7c80c55c0 0x7fe7c80c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:36.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.940+0000 7fe7ceb61700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe7c806a390 0x7fe7c806a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:36.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.940+0000 7fe7ceb61700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c8155f30 con 0x7fe7c814f5b0 2024-09-13T18:11:36.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.940+0000 7fe7ceb61700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c81553f0 con 0x7fe7c80c55c0 2024-09-13T18:11:36.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.940+0000 7fe7ceb61700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c80c7110 con 0x7fe7c806a390 2024-09-13T18:11:36.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.940+0000 7fe7cdb5f700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7fe7c814f5b0 legacy=0x7fe7c80c6960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:11:36.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.940+0000 7fe7cd35e700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7fe7c80c55c0 msgr2=0x7fe7c80c5a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:11:36.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.940+0000 7fe7cd35e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe7c80c55c0 0x7fe7c80c5a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:11:36.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.940+0000 7fe7ce360700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe7c806a390 0x7fe7c806a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:36.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.940+0000 7fe7ce360700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe7c806a390 0x7fe7c806a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:52762/0 (socket says 172.21.15.40:52762) 2024-09-13T18:11:36.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.940+0000 7fe7ce360700 1 -- 172.21.15.40:0/1697046321 learned_addr learned my addr 172.21.15.40:0/1697046321 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:36.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.940+0000 7fe7ce360700 1 -- 172.21.15.40:0/1697046321 >> v2:172.21.15.40:3300/0 conn(0x7fe7c80c55c0 msgr2=0x7fe7c80c5a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:11:36.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.940+0000 7fe7ce360700 1 --2- 172.21.15.40:0/1697046321 >> v2:172.21.15.40:3300/0 conn(0x7fe7c80c55c0 0x7fe7c80c5a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:36.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.940+0000 7fe7ce360700 1 -- 172.21.15.40:0/1697046321 >> v1:172.21.15.40:6789/0 conn(0x7fe7c814f5b0 legacy=0x7fe7c80c6960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:11:36.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.940+0000 7fe7ce360700 1 -- 172.21.15.40:0/1697046321 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c8155600 con 0x7fe7c806a390 2024-09-13T18:11:36.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.940+0000 7fe7ce360700 1 --2- 172.21.15.40:0/1697046321 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe7c806a390 0x7fe7c806a820 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009910 tx=0x7fe7b8009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cecba5087c547291 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:36.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.944+0000 7fe7ccb5d700 1 -- 172.21.15.40:0/1697046321 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8004230 con 0x7fe7c806a390 2024-09-13T18:11:36.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.944+0000 7fe7ccb5d700 1 -- 172.21.15.40:0/1697046321 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b8004390 con 0x7fe7c806a390 2024-09-13T18:11:36.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.944+0000 7fe7ccb5d700 1 -- 172.21.15.40:0/1697046321 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8009e60 con 0x7fe7c806a390 2024-09-13T18:11:36.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.944+0000 7fe7ceb61700 1 -- 172.21.15.40:0/1697046321 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe7c806a390 msgr2=0x7fe7c806a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:36.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.944+0000 7fe7ceb61700 1 --2- 172.21.15.40:0/1697046321 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe7c806a390 0x7fe7c806a820 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009910 tx=0x7fe7b8009c20 comp rx=0 tx=0).stop 2024-09-13T18:11:36.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.944+0000 7fe7ceb61700 1 -- 172.21.15.40:0/1697046321 shutdown_connections 2024-09-13T18:11:36.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.944+0000 7fe7ceb61700 1 --2- 172.21.15.40:0/1697046321 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe7c806a390 0x7fe7c806a820 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:36.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.944+0000 7fe7ceb61700 1 --2- 172.21.15.40:0/1697046321 >> v2:172.21.15.40:3300/0 conn(0x7fe7c80c55c0 0x7fe7c80c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:36.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.944+0000 7fe7ceb61700 1 -- 172.21.15.40:0/1697046321 >> 172.21.15.40:0/1697046321 conn(0x7fe7c80cabc0 msgr2=0x7fe7c80cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:36.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.944+0000 7fe7ceb61700 1 -- 172.21.15.40:0/1697046321 shutdown_connections 2024-09-13T18:11:36.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.944+0000 7fe7ceb61700 1 -- 172.21.15.40:0/1697046321 wait complete. 2024-09-13T18:11:36.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.944+0000 7fe7ceb61700 1 Processor -- start 2024-09-13T18:11:36.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.944+0000 7fe7ceb61700 1 -- start start 2024-09-13T18:11:36.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7ceb61700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7c806a390 0x7fe7c81e49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:36.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7ceb61700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe7c80c55c0 0x7fe7c80c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:36.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7ceb61700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7c814f5b0 0x7fe7c80c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:36.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7ceb61700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c81ede10 con 0x7fe7c814f5b0 2024-09-13T18:11:36.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7ceb61700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c81e4730 con 0x7fe7c806a390 2024-09-13T18:11:36.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7ceb61700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c81e4f30 con 0x7fe7c80c55c0 2024-09-13T18:11:36.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7ce360700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7c814f5b0 msgr2=0x7fe7c80c9960 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:11:36.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7ce360700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7c814f5b0 0x7fe7c80c9960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:11:36.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7cdb5f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7c806a390 0x7fe7c81e49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:36.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7cdb5f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7c806a390 0x7fe7c81e49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:46490/0 (socket says 172.21.15.40:46490) 2024-09-13T18:11:36.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7cdb5f700 1 -- 172.21.15.40:0/1771719005 learned_addr learned my addr 172.21.15.40:0/1771719005 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:36.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7cd35e700 1 --2- 172.21.15.40:0/1771719005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe7c80c55c0 0x7fe7c80c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:36.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7cdb5f700 1 -- 172.21.15.40:0/1771719005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe7c80c55c0 msgr2=0x7fe7c80c2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:36.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7cdb5f700 1 --2- 172.21.15.40:0/1771719005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe7c80c55c0 0x7fe7c80c2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:36.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7cdb5f700 1 -- 172.21.15.40:0/1771719005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7c814f5b0 msgr2=0x7fe7c80c9960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:11:36.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7cdb5f700 1 --2- 172.21.15.40:0/1771719005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7c814f5b0 0x7fe7c80c9960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:36.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7cdb5f700 1 -- 172.21.15.40:0/1771719005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b8009570 con 0x7fe7c806a390 2024-09-13T18:11:36.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7cdb5f700 1 --2- 172.21.15.40:0/1771719005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7c806a390 0x7fe7c81e49f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000ba10 tx=0x7fe7c000bd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:36.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7be7fc700 1 -- 172.21.15.40:0/1771719005 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c000c950 con 0x7fe7c806a390 2024-09-13T18:11:36.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7be7fc700 1 -- 172.21.15.40:0/1771719005 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c000cad0 con 0x7fe7c806a390 2024-09-13T18:11:36.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7be7fc700 1 -- 172.21.15.40:0/1771719005 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c000ce80 con 0x7fe7c806a390 2024-09-13T18:11:36.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7ceb61700 1 -- 172.21.15.40:0/1771719005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c81ece00 con 0x7fe7c806a390 2024-09-13T18:11:36.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.948+0000 7fe7ceb61700 1 -- 172.21.15.40:0/1771719005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c81e5170 con 0x7fe7c806a390 2024-09-13T18:11:36.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.952+0000 7fe7ceb61700 1 -- 172.21.15.40:0/1771719005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c8072830 con 0x7fe7c806a390 2024-09-13T18:11:36.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.952+0000 7fe7be7fc700 1 -- 172.21.15.40:0/1771719005 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe7c0012070 con 0x7fe7c806a390 2024-09-13T18:11:36.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.952+0000 7fe7be7fc700 1 --2- 172.21.15.40:0/1771719005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe7ac0666e0 0x7fe7ac068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:36.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.952+0000 7fe7be7fc700 1 -- 172.21.15.40:0/1771719005 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe7c0084260 con 0x7fe7c806a390 2024-09-13T18:11:36.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.952+0000 7fe7cd35e700 1 --2- 172.21.15.40:0/1771719005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe7ac0666e0 0x7fe7ac068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:36.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.952+0000 7fe7cd35e700 1 --2- 172.21.15.40:0/1771719005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe7ac0666e0 0x7fe7ac068ba0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80c96e0 tx=0x7fe7c4003fd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:36.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:36.960+0000 7fe7be7fc700 1 -- 172.21.15.40:0/1771719005 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe7c00512a0 con 0x7fe7c806a390 2024-09-13T18:11:37.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:37.212+0000 7fe7ceb61700 1 -- 172.21.15.40:0/1771719005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe7c8066fb0 con 0x7fe7c806a390 2024-09-13T18:11:37.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:37.212+0000 7fe7be7fc700 1 -- 172.21.15.40:0/1771719005 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7fe7c8066fb0 con 0x7fe7c806a390 2024-09-13T18:11:37.222 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":166,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:11:37.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:37.216+0000 7fe7ceb61700 1 -- 172.21.15.40:0/1771719005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe7ac0666e0 msgr2=0x7fe7ac068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:37.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:37.216+0000 7fe7ceb61700 1 --2- 172.21.15.40:0/1771719005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe7ac0666e0 0x7fe7ac068ba0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80c96e0 tx=0x7fe7c4003fd0 comp rx=0 tx=0).stop 2024-09-13T18:11:37.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:37.216+0000 7fe7ceb61700 1 -- 172.21.15.40:0/1771719005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7c806a390 msgr2=0x7fe7c81e49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:37.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:37.216+0000 7fe7ceb61700 1 --2- 172.21.15.40:0/1771719005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7c806a390 0x7fe7c81e49f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000ba10 tx=0x7fe7c000bd20 comp rx=0 tx=0).stop 2024-09-13T18:11:37.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:37.216+0000 7fe7ceb61700 1 -- 172.21.15.40:0/1771719005 shutdown_connections 2024-09-13T18:11:37.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:37.216+0000 7fe7ceb61700 1 --2- 172.21.15.40:0/1771719005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7c814f5b0 0x7fe7c80c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:37.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:37.216+0000 7fe7ceb61700 1 --2- 172.21.15.40:0/1771719005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe7ac0666e0 0x7fe7ac068ba0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:37.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:37.216+0000 7fe7ceb61700 1 --2- 172.21.15.40:0/1771719005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe7c80c55c0 0x7fe7c80c2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:37.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:37.216+0000 7fe7ceb61700 1 --2- 172.21.15.40:0/1771719005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7c806a390 0x7fe7c81e49f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:37.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:37.216+0000 7fe7ceb61700 1 -- 172.21.15.40:0/1771719005 >> 172.21.15.40:0/1771719005 conn(0x7fe7c80cabc0 msgr2=0x7fe7c80cafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:37.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:37.216+0000 7fe7ceb61700 1 -- 172.21.15.40:0/1771719005 shutdown_connections 2024-09-13T18:11:37.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:37.216+0000 7fe7ceb61700 1 -- 172.21.15.40:0/1771719005 wait complete. 2024-09-13T18:11:37.542 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:11:37.545 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:11:40.237 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:11:40.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.348+0000 7fc9abfff700 1 Processor -- start 2024-09-13T18:11:40.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.348+0000 7fc9abfff700 1 -- start start 2024-09-13T18:11:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.348+0000 7fc9abfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 0x7fc9ac066290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.348+0000 7fc9abfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc9ac070d60 0x7fc9ac15d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.352+0000 7fc9abfff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac15d8f0 con 0x7fc9ac06e010 2024-09-13T18:11:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.352+0000 7fc9abfff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ac06dbf0 con 0x7fc9ac070d60 2024-09-13T18:11:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.352+0000 7fc9abfff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ac0667d0 con 0x7fc9ac070980 2024-09-13T18:11:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.352+0000 7fc9aa7fc700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7fc9ac070d60 msgr2=0x7fc9ac15d3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:11:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.352+0000 7fc9aa7fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc9ac070d60 0x7fc9ac15d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:11:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.352+0000 7fc9aaffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 0x7fc9ac066290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.352+0000 7fc9ab7fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc9ac06e010 0x7fc9ac1532b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:45742/0 (socket says 172.21.15.40:45742) 2024-09-13T18:11:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.352+0000 7fc9ab7fe700 1 -- 172.21.15.40:0/3002670622 learned_addr learned my addr 172.21.15.40:0/3002670622 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:11:40.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.352+0000 7fc9aaffd700 1 -- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 msgr2=0x7fc9ac066290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:11:40.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.352+0000 7fc9aaffd700 1 -- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 msgr2=0x7fc9ac066290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:11:40.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.352+0000 7fc9aaffd700 1 --2- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 0x7fc9ac066290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:11:40.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.352+0000 7fc9aaffd700 1 --2- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 0x7fc9ac066290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:11:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.556+0000 7fc9aa7fc700 1 -- 172.21.15.40:0/3002670622 >> v2:172.21.15.40:3300/0 conn(0x7fc9ac070d60 msgr2=0x7fc9ac15d3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:11:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.556+0000 7fc9aa7fc700 1 --2- 172.21.15.40:0/3002670622 >> v2:172.21.15.40:3300/0 conn(0x7fc9ac070d60 0x7fc9ac15d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:11:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.556+0000 7fc9aaffd700 1 --2- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 0x7fc9ac066290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.556+0000 7fc9aaffd700 1 -- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 msgr2=0x7fc9ac066290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:11:40.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.556+0000 7fc9aaffd700 1 -- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 msgr2=0x7fc9ac066290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:11:40.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.556+0000 7fc9aaffd700 1 --2- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 0x7fc9ac066290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:11:40.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.556+0000 7fc9aaffd700 1 --2- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 0x7fc9ac066290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:11:40.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.960+0000 7fc9aa7fc700 1 -- 172.21.15.40:0/3002670622 >> v2:172.21.15.40:3300/0 conn(0x7fc9ac070d60 msgr2=0x7fc9ac15d3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:11:40.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.960+0000 7fc9aa7fc700 1 --2- 172.21.15.40:0/3002670622 >> v2:172.21.15.40:3300/0 conn(0x7fc9ac070d60 0x7fc9ac15d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:11:40.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.960+0000 7fc9aaffd700 1 --2- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 0x7fc9ac066290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:40.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.960+0000 7fc9aaffd700 1 -- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 msgr2=0x7fc9ac066290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:11:40.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.960+0000 7fc9aaffd700 1 -- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 msgr2=0x7fc9ac066290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:11:40.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.960+0000 7fc9aaffd700 1 --2- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 0x7fc9ac066290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:11:40.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:40.960+0000 7fc9aaffd700 1 --2- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 0x7fc9ac066290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:11:41.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:41.764+0000 7fc9aa7fc700 1 -- 172.21.15.40:0/3002670622 >> v2:172.21.15.40:3300/0 conn(0x7fc9ac070d60 msgr2=0x7fc9ac15d3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:11:41.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:41.764+0000 7fc9aa7fc700 1 --2- 172.21.15.40:0/3002670622 >> v2:172.21.15.40:3300/0 conn(0x7fc9ac070d60 0x7fc9ac15d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:11:41.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:41.764+0000 7fc9aaffd700 1 --2- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 0x7fc9ac066290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:41.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:41.764+0000 7fc9aaffd700 1 -- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 msgr2=0x7fc9ac066290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:11:41.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:41.764+0000 7fc9aaffd700 1 -- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 msgr2=0x7fc9ac066290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:11:41.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:41.764+0000 7fc9aaffd700 1 --2- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 0x7fc9ac066290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:11:41.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:41.764+0000 7fc9aaffd700 1 --2- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 0x7fc9ac066290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:11:43.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.348+0000 7fc9a8ff9700 1 -- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 msgr2=0x7fc9ac066290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:11:43.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.348+0000 7fc9a8ff9700 1 --2- 172.21.15.40:0/3002670622 >> v2:172.21.15.43:3300/0 conn(0x7fc9ac070980 0x7fc9ac066290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:43.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.352+0000 7fc9a8ff9700 1 -- 172.21.15.40:0/3002670622 >> v2:172.21.15.40:3300/0 conn(0x7fc9ac070d60 msgr2=0x7fc9ac15d3b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:11:43.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.352+0000 7fc9a8ff9700 1 --2- 172.21.15.40:0/3002670622 >> v2:172.21.15.40:3300/0 conn(0x7fc9ac070d60 0x7fc9ac15d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.352+0000 7fc9a8ff9700 1 -- 172.21.15.40:0/3002670622 >> v1:172.21.15.43:6789/0 conn(0x7fc9ac06e010 legacy=0x7fc9ac1532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.352+0000 7fc9a8ff9700 1 --2- 172.21.15.40:0/3002670622 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc998007420 0x7fc9980098c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.352+0000 7fc9a8ff9700 1 -- 172.21.15.40:0/3002670622 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac0c76b0 con 0x7fc998003740 2024-09-13T18:11:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.352+0000 7fc9a8ff9700 1 -- 172.21.15.40:0/3002670622 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac06dbf0 con 0x7fc998007420 2024-09-13T18:11:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.352+0000 7fc9aaffd700 1 -- 172.21.15.40:0/3002670622 >> v1:172.21.15.40:6789/0 conn(0x7fc998003740 legacy=0x7fc998005bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:11:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.352+0000 7fc9aa7fc700 1 --2- 172.21.15.40:0/3002670622 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc998007420 0x7fc9980098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.352+0000 7fc9aa7fc700 1 -- 172.21.15.40:0/3002670622 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc998007420 msgr2=0x7fc9980098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:11:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.352+0000 7fc9aa7fc700 1 -- 172.21.15.40:0/3002670622 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc998007420 msgr2=0x7fc9980098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:11:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.352+0000 7fc9aa7fc700 1 --2- 172.21.15.40:0/3002670622 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc998007420 0x7fc9980098c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:11:43.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.352+0000 7fc9aa7fc700 1 --2- 172.21.15.40:0/3002670622 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc998007420 0x7fc9980098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:11:43.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.556+0000 7fc9aaffd700 1 -- 172.21.15.40:0/3002670622 >> v1:172.21.15.40:6789/0 conn(0x7fc998003740 legacy=0x7fc998005bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:11:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.556+0000 7fc9aa7fc700 1 --2- 172.21.15.40:0/3002670622 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc998007420 0x7fc9980098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.556+0000 7fc9aa7fc700 1 -- 172.21.15.40:0/3002670622 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc998007420 msgr2=0x7fc9980098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:11:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.556+0000 7fc9aa7fc700 1 -- 172.21.15.40:0/3002670622 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc998007420 msgr2=0x7fc9980098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:11:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.556+0000 7fc9aa7fc700 1 --2- 172.21.15.40:0/3002670622 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc998007420 0x7fc9980098c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:11:43.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.556+0000 7fc9aa7fc700 1 --2- 172.21.15.40:0/3002670622 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc998007420 0x7fc9980098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:11:43.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.960+0000 7fc9aaffd700 1 -- 172.21.15.40:0/3002670622 >> v1:172.21.15.40:6789/0 conn(0x7fc998003740 legacy=0x7fc998005bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:11:43.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.960+0000 7fc9aa7fc700 1 --2- 172.21.15.40:0/3002670622 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc998007420 0x7fc9980098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:43.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.960+0000 7fc9aa7fc700 1 -- 172.21.15.40:0/3002670622 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc998007420 msgr2=0x7fc9980098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:11:43.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.960+0000 7fc9aa7fc700 1 -- 172.21.15.40:0/3002670622 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc998007420 msgr2=0x7fc9980098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:11:43.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.960+0000 7fc9aa7fc700 1 --2- 172.21.15.40:0/3002670622 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc998007420 0x7fc9980098c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:11:43.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:43.960+0000 7fc9aa7fc700 1 --2- 172.21.15.40:0/3002670622 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc998007420 0x7fc9980098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:11:44.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.764+0000 7fc9aaffd700 1 -- 172.21.15.40:0/3002670622 >> v1:172.21.15.40:6789/0 conn(0x7fc998003740 legacy=0x7fc998005bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:11:44.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.764+0000 7fc9aa7fc700 1 --2- 172.21.15.40:0/3002670622 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc998007420 0x7fc9980098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:44.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.764+0000 7fc9aa7fc700 1 -- 172.21.15.40:0/3002670622 >> v1:172.21.15.40:6789/0 conn(0x7fc998003740 legacy=0x7fc998005bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:11:44.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.764+0000 7fc9aa7fc700 1 -- 172.21.15.40:0/3002670622 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99800b0a0 con 0x7fc998007420 2024-09-13T18:11:44.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.764+0000 7fc9aa7fc700 1 --2- 172.21.15.40:0/3002670622 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc998007420 0x7fc9980098c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009d40 tx=0x7fc9a000b6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a177729ccc4b59f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:44.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.764+0000 7fc9a9ffb700 1 -- 172.21.15.40:0/3002670622 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0016610 con 0x7fc998007420 2024-09-13T18:11:44.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.764+0000 7fc9a9ffb700 1 -- 172.21.15.40:0/3002670622 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a0011050 con 0x7fc998007420 2024-09-13T18:11:44.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.764+0000 7fc9a9ffb700 1 -- 172.21.15.40:0/3002670622 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0016990 con 0x7fc998007420 2024-09-13T18:11:44.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.764+0000 7fc9abfff700 1 -- 172.21.15.40:0/3002670622 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc998007420 msgr2=0x7fc9980098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:44.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.764+0000 7fc9abfff700 1 --2- 172.21.15.40:0/3002670622 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc998007420 0x7fc9980098c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009d40 tx=0x7fc9a000b6d0 comp rx=0 tx=0).stop 2024-09-13T18:11:44.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.764+0000 7fc9ab7fe700 1 -- 172.21.15.40:0/3002670622 reap_dead start 2024-09-13T18:11:44.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.764+0000 7fc9abfff700 1 -- 172.21.15.40:0/3002670622 shutdown_connections 2024-09-13T18:11:44.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.764+0000 7fc9abfff700 1 -- 172.21.15.40:0/3002670622 >> 172.21.15.40:0/3002670622 conn(0x7fc9ac0c7d70 msgr2=0x7fc9ac0c8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:44.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.764+0000 7fc9abfff700 1 -- 172.21.15.40:0/3002670622 shutdown_connections 2024-09-13T18:11:44.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.764+0000 7fc9abfff700 1 -- 172.21.15.40:0/3002670622 wait complete. 2024-09-13T18:11:44.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.768+0000 7fc9abfff700 1 Processor -- start 2024-09-13T18:11:44.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.768+0000 7fc9abfff700 1 -- start start 2024-09-13T18:11:44.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9abfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9ac0c4080 0x7fc9ac14bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:44.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9abfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9ac0c4460 0x7fc9ac149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:44.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9abfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9ac0bc420 0x7fc9ac14a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:44.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9abfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac14cf20 con 0x7fc9ac0c4080 2024-09-13T18:11:44.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9abfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac14b960 con 0x7fc9ac0c4460 2024-09-13T18:11:44.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9aaffd700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9ac0c4080 msgr2=0x7fc9ac14bbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:11:44.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9abfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac14ca90 con 0x7fc9ac0bc420 2024-09-13T18:11:44.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9aaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9ac0c4080 0x7fc9ac14bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:11:44.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9aa7fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9ac0c4460 0x7fc9ac149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:44.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9aa7fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9ac0c4460 0x7fc9ac149d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:55874/0 (socket says 172.21.15.40:55874) 2024-09-13T18:11:44.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9aa7fc700 1 -- 172.21.15.40:0/1630905474 learned_addr learned my addr 172.21.15.40:0/1630905474 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:44.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9ab7fe700 1 --2- 172.21.15.40:0/1630905474 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9ac0bc420 0x7fc9ac14a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:44.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9aa7fc700 1 -- 172.21.15.40:0/1630905474 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9ac0bc420 msgr2=0x7fc9ac14a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:44.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9aa7fc700 1 --2- 172.21.15.40:0/1630905474 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9ac0bc420 0x7fc9ac14a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:44.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9aa7fc700 1 -- 172.21.15.40:0/1630905474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9ac0c4080 msgr2=0x7fc9ac14bbe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:11:44.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9aa7fc700 1 --2- 172.21.15.40:0/1630905474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9ac0c4080 0x7fc9ac14bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:44.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9aa7fc700 1 -- 172.21.15.40:0/1630905474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a0009a20 con 0x7fc9ac0c4460 2024-09-13T18:11:44.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9aa7fc700 1 --2- 172.21.15.40:0/1630905474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9ac0c4460 0x7fc9ac149d60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0003940 tx=0x7fc9a0005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:44.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9937fe700 1 -- 172.21.15.40:0/1630905474 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0003a10 con 0x7fc9ac0c4460 2024-09-13T18:11:44.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9937fe700 1 -- 172.21.15.40:0/1630905474 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a0020620 con 0x7fc9ac0c4460 2024-09-13T18:11:44.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9937fe700 1 -- 172.21.15.40:0/1630905474 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a00209d0 con 0x7fc9ac0c4460 2024-09-13T18:11:44.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9abfff700 1 -- 172.21.15.40:0/1630905474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ac14b400 con 0x7fc9ac0c4460 2024-09-13T18:11:44.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.772+0000 7fc9abfff700 1 -- 172.21.15.40:0/1630905474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ac14c740 con 0x7fc9ac0c4460 2024-09-13T18:11:44.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.776+0000 7fc9abfff700 1 -- 172.21.15.40:0/1630905474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ac06e4c0 con 0x7fc9ac0c4460 2024-09-13T18:11:44.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.776+0000 7fc9937fe700 1 -- 172.21.15.40:0/1630905474 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc9a001d050 con 0x7fc9ac0c4460 2024-09-13T18:11:44.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.780+0000 7fc9937fe700 1 --2- 172.21.15.40:0/1630905474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc998067510 0x7fc9980699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:44.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.780+0000 7fc9937fe700 1 -- 172.21.15.40:0/1630905474 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc9a0085f40 con 0x7fc9ac0c4460 2024-09-13T18:11:44.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.780+0000 7fc9aaffd700 1 --2- 172.21.15.40:0/1630905474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc998067510 0x7fc9980699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:44.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.780+0000 7fc9aaffd700 1 --2- 172.21.15.40:0/1630905474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc998067510 0x7fc9980699d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009100 tx=0x7fc99c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:44.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:44.788+0000 7fc9937fe700 1 -- 172.21.15.40:0/1630905474 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc9a0053000 con 0x7fc9ac0c4460 2024-09-13T18:11:45.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.024+0000 7fc9abfff700 1 -- 172.21.15.40:0/1630905474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc9ac14b680 con 0x7fc9ac0c4460 2024-09-13T18:11:45.032 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.024+0000 7fc9937fe700 1 -- 172.21.15.40:0/1630905474 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7fc9a0056620 con 0x7fc9ac0c4460 2024-09-13T18:11:45.032 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":168,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:11:45.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.028+0000 7fc9abfff700 1 -- 172.21.15.40:0/1630905474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc998067510 msgr2=0x7fc9980699d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:45.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.028+0000 7fc9abfff700 1 --2- 172.21.15.40:0/1630905474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc998067510 0x7fc9980699d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009100 tx=0x7fc99c008040 comp rx=0 tx=0).stop 2024-09-13T18:11:45.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.028+0000 7fc9abfff700 1 -- 172.21.15.40:0/1630905474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9ac0c4460 msgr2=0x7fc9ac149d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:45.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.028+0000 7fc9abfff700 1 --2- 172.21.15.40:0/1630905474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9ac0c4460 0x7fc9ac149d60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0003940 tx=0x7fc9a0005e60 comp rx=0 tx=0).stop 2024-09-13T18:11:45.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.028+0000 7fc9abfff700 1 -- 172.21.15.40:0/1630905474 shutdown_connections 2024-09-13T18:11:45.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.028+0000 7fc9abfff700 1 --2- 172.21.15.40:0/1630905474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc998067510 0x7fc9980699d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.028+0000 7fc9abfff700 1 --2- 172.21.15.40:0/1630905474 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9ac0bc420 0x7fc9ac14a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.028+0000 7fc9abfff700 1 --2- 172.21.15.40:0/1630905474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9ac0c4460 0x7fc9ac149d60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.028+0000 7fc9abfff700 1 --2- 172.21.15.40:0/1630905474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9ac0c4080 0x7fc9ac14bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.028+0000 7fc9abfff700 1 -- 172.21.15.40:0/1630905474 >> 172.21.15.40:0/1630905474 conn(0x7fc9ac0c7d70 msgr2=0x7fc9ac15d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:45.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.028+0000 7fc9abfff700 1 -- 172.21.15.40:0/1630905474 shutdown_connections 2024-09-13T18:11:45.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.028+0000 7fc9abfff700 1 -- 172.21.15.40:0/1630905474 wait complete. 2024-09-13T18:11:45.046 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T18:11:45.047 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-13T18:11:45.047 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:11:45.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.148+0000 7fd552893700 1 Processor -- start 2024-09-13T18:11:45.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.148+0000 7fd552893700 1 -- start start 2024-09-13T18:11:45.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.148+0000 7fd552893700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd54c153150 0x7fd54c15a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:45.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.148+0000 7fd552893700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd54c0724d0 0x7fd54c070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:45.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.148+0000 7fd552893700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd54c0c6190 0x7fd54c070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:45.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.148+0000 7fd551090700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7fd54c0724d0 msgr2=0x7fd54c070aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:11:45.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.148+0000 7fd552893700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54c159e30 con 0x7fd54c0724d0 2024-09-13T18:11:45.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.148+0000 7fd551891700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd54c153150 0x7fd54c15a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:45.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.148+0000 7fd551090700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd54c0724d0 0x7fd54c070aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:11:45.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.148+0000 7fd552893700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54c15a0d0 con 0x7fd54c153150 2024-09-13T18:11:45.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.148+0000 7fd552893700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c0cb2a0 con 0x7fd54c0c6190 2024-09-13T18:11:45.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.148+0000 7fd551891700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd54c153150 0x7fd54c15a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:55876/0 (socket says 172.21.15.40:55876) 2024-09-13T18:11:45.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.148+0000 7fd551891700 1 -- 172.21.15.40:0/3380138582 learned_addr learned my addr 172.21.15.40:0/3380138582 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:45.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.148+0000 7fd552092700 1 --2- 172.21.15.40:0/3380138582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd54c0c6190 0x7fd54c070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:45.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.148+0000 7fd551891700 1 -- 172.21.15.40:0/3380138582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd54c0c6190 msgr2=0x7fd54c070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:11:45.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.148+0000 7fd551891700 1 --2- 172.21.15.40:0/3380138582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd54c0c6190 0x7fd54c070fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.148+0000 7fd551891700 1 -- 172.21.15.40:0/3380138582 >> v2:172.21.15.40:3300/0 conn(0x7fd54c0724d0 msgr2=0x7fd54c070aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:11:45.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.148+0000 7fd551891700 1 --2- 172.21.15.40:0/3380138582 >> v2:172.21.15.40:3300/0 conn(0x7fd54c0724d0 0x7fd54c070aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.148+0000 7fd551891700 1 -- 172.21.15.40:0/3380138582 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c0cb580 con 0x7fd54c153150 2024-09-13T18:11:45.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.148+0000 7fd552092700 1 --2- 172.21.15.40:0/3380138582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd54c0c6190 0x7fd54c070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:11:45.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.152+0000 7fd551891700 1 --2- 172.21.15.40:0/3380138582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c153150 0x7fd54c15a790 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0cc390 tx=0x7fd548009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a89086f2aed898b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:45.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.152+0000 7fd55088f700 1 -- 172.21.15.40:0/3380138582 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548011830 con 0x7fd54c153150 2024-09-13T18:11:45.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.152+0000 7fd55088f700 1 -- 172.21.15.40:0/3380138582 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd548011990 con 0x7fd54c153150 2024-09-13T18:11:45.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.152+0000 7fd55088f700 1 -- 172.21.15.40:0/3380138582 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548011d20 con 0x7fd54c153150 2024-09-13T18:11:45.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.152+0000 7fd552893700 1 -- 172.21.15.40:0/3380138582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c153150 msgr2=0x7fd54c15a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:45.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.152+0000 7fd552893700 1 --2- 172.21.15.40:0/3380138582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c153150 0x7fd54c15a790 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0cc390 tx=0x7fd548009bf0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.152+0000 7fd552893700 1 -- 172.21.15.40:0/3380138582 shutdown_connections 2024-09-13T18:11:45.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.152+0000 7fd552893700 1 --2- 172.21.15.40:0/3380138582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd54c0c6190 0x7fd54c070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.152+0000 7fd552893700 1 --2- 172.21.15.40:0/3380138582 >> v2:172.21.15.40:3300/0 conn(0x7fd54c0724d0 0x7fd54c070aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.152+0000 7fd552893700 1 --2- 172.21.15.40:0/3380138582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c153150 0x7fd54c15a790 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.152+0000 7fd552893700 1 -- 172.21.15.40:0/3380138582 >> 172.21.15.40:0/3380138582 conn(0x7fd54c06a390 msgr2=0x7fd54c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:45.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.152+0000 7fd552893700 1 -- 172.21.15.40:0/3380138582 shutdown_connections 2024-09-13T18:11:45.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.152+0000 7fd552893700 1 -- 172.21.15.40:0/3380138582 wait complete. 2024-09-13T18:11:45.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.152+0000 7fd552893700 1 Processor -- start 2024-09-13T18:11:45.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.152+0000 7fd552893700 1 -- start start 2024-09-13T18:11:45.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd552893700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c0724d0 0x7fd54c151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:45.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd552893700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd54c0c6190 0x7fd54c1e17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:45.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd552893700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd54c153150 0x7fd54c0c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:45.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd552893700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c158cb0 con 0x7fd54c0c6190 2024-09-13T18:11:45.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd552893700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c153cc0 con 0x7fd54c0724d0 2024-09-13T18:11:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd552893700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c153fa0 con 0x7fd54c153150 2024-09-13T18:11:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd551090700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd54c0c6190 msgr2=0x7fd54c1e17e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:11:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd551090700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd54c0c6190 0x7fd54c1e17e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:11:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd551891700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c0724d0 0x7fd54c151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd551891700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c0724d0 0x7fd54c151510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:55880/0 (socket says 172.21.15.40:55880) 2024-09-13T18:11:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd551891700 1 -- 172.21.15.40:0/2589486453 learned_addr learned my addr 172.21.15.40:0/2589486453 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:45.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd552092700 1 --2- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd54c153150 0x7fd54c0c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:45.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd551891700 1 -- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd54c153150 msgr2=0x7fd54c0c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:45.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd551891700 1 --2- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd54c153150 0x7fd54c0c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd551891700 1 -- 172.21.15.40:0/2589486453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd54c0c6190 msgr2=0x7fd54c1e17e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:11:45.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd551891700 1 --2- 172.21.15.40:0/2589486453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd54c0c6190 0x7fd54c1e17e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd551891700 1 -- 172.21.15.40:0/2589486453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd544009a70 con 0x7fd54c0724d0 2024-09-13T18:11:45.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd551891700 1 --2- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c0724d0 0x7fd54c151510 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd548014040 tx=0x7fd548004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:45.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd5427fc700 1 -- 172.21.15.40:0/2589486453 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548004550 con 0x7fd54c0724d0 2024-09-13T18:11:45.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd552893700 1 -- 172.21.15.40:0/2589486453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd548009570 con 0x7fd54c0724d0 2024-09-13T18:11:45.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd5427fc700 1 -- 172.21.15.40:0/2589486453 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54801e070 con 0x7fd54c0724d0 2024-09-13T18:11:45.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd552893700 1 -- 172.21.15.40:0/2589486453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54c154560 con 0x7fd54c0724d0 2024-09-13T18:11:45.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd5427fc700 1 -- 172.21.15.40:0/2589486453 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5480038c0 con 0x7fd54c0724d0 2024-09-13T18:11:45.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd552893700 1 --2- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c1e1d20 0x7fd54c0c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:45.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.156+0000 7fd552893700 1 -- 172.21.15.40:0/2589486453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c067600 con 0x7fd54c1e1d20 2024-09-13T18:11:45.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.160+0000 7fd551090700 1 --2- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c1e1d20 0x7fd54c0c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:45.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.160+0000 7fd551090700 1 -- 172.21.15.40:0/2589486453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd54c0cbb40 con 0x7fd54c1e1d20 2024-09-13T18:11:45.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.160+0000 7fd551090700 1 --2- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c1e1d20 0x7fd54c0c8070 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd54c154bc0 tx=0x7fd53c009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:45.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.160+0000 7fd5427fc700 1 -- 172.21.15.40:0/2589486453 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd548003a20 con 0x7fd54c0724d0 2024-09-13T18:11:45.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.160+0000 7fd5427fc700 1 --2- 172.21.15.40:0/2589486453 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd5300666e0 0x7fd530068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:45.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.160+0000 7fd5427fc700 1 -- 172.21.15.40:0/2589486453 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd548020070 con 0x7fd54c0724d0 2024-09-13T18:11:45.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.160+0000 7fd552092700 1 --2- 172.21.15.40:0/2589486453 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd5300666e0 0x7fd530068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:45.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.160+0000 7fd5427fc700 1 -- 172.21.15.40:0/2589486453 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c003ab0 con 0x7fd54c1e1d20 2024-09-13T18:11:45.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.160+0000 7fd5427fc700 1 -- 172.21.15.40:0/2589486453 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd53c003c10 con 0x7fd54c1e1d20 2024-09-13T18:11:45.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.160+0000 7fd5427fc700 1 -- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c1e1d20 msgr2=0x7fd54c0c8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:45.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.160+0000 7fd5427fc700 1 --2- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c1e1d20 0x7fd54c0c8070 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd54c154bc0 tx=0x7fd53c009f30 comp rx=0 tx=0).stop 2024-09-13T18:11:45.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.160+0000 7fd5427fc700 1 -- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c1e1d20 msgr2=0x7fd54c0c8070 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:11:45.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.160+0000 7fd5427fc700 1 --2- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c1e1d20 0x7fd54c0c8070 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd54c154bc0 tx=0x7fd53c009f30 comp rx=0 tx=0).stop 2024-09-13T18:11:45.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.160+0000 7fd552092700 1 --2- 172.21.15.40:0/2589486453 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd5300666e0 0x7fd530068ba0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd544011580 tx=0x7fd54400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:45.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd552893700 1 --2- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c1fe9a0 0x7fd54c1fedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:45.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd552893700 1 -- 172.21.15.40:0/2589486453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c1547a0 con 0x7fd54c1fe9a0 2024-09-13T18:11:45.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd551090700 1 --2- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c1fe9a0 0x7fd54c1fedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:45.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd551090700 1 -- 172.21.15.40:0/2589486453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd54c0c7770 con 0x7fd54c1fe9a0 2024-09-13T18:11:45.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd551090700 1 --2- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c1fe9a0 0x7fd54c1fedb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd54c154bc0 tx=0x7fd53c005ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:45.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd5427fc700 1 -- 172.21.15.40:0/2589486453 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c01a780 con 0x7fd54c1fe9a0 2024-09-13T18:11:45.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd5427fc700 1 -- 172.21.15.40:0/2589486453 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4773 (secure 0 0 0) 0x7fd53c019070 con 0x7fd54c1fe9a0 2024-09-13T18:11:45.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd5427fc700 1 -- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c1fe9a0 msgr2=0x7fd54c1fedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:45.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd5427fc700 1 --2- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c1fe9a0 0x7fd54c1fedb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd54c154bc0 tx=0x7fd53c005ad0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd5427fc700 1 -- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c1fe9a0 msgr2=0x7fd54c1fedb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:11:45.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd5427fc700 1 --2- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c1fe9a0 0x7fd54c1fedb0 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd54c154bc0 tx=0x7fd53c005ad0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.182 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:11:45.182 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:11:45.182 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:11:45.182 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:11:45.182 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 168, 2024-09-13T18:11:45.182 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:11:45.182 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:11:45.182 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:11:45.182 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:45.182 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:11:45.183 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:11:45.183 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:11:45.183 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:11:45.183 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:45.183 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:45.183 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:45.183 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:45.183 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:45.183 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:45.183 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:45.183 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:45.183 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:45.183 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:45.183 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:45.184 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:11:45.184 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:11:45.184 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:45.184 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:45.184 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:45.184 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:45.184 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:45.184 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:45.184 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:45.184 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:45.184 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:45.184 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:45.184 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:45.184 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.185 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:11:45.185 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:11:45.185 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:11:45.185 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:11:45.185 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:11:45.185 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:11:45.186 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:11:45.186 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:11:45.186 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:11:45.186 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:11:45.186 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:11:45.186 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:11:45.186 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:11:45.186 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:11:45.186 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:11:45.186 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:11:45.186 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:11:45.186 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:45.187 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:45.187 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:45.187 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:45.187 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:45.187 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:45.187 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:45.187 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:45.187 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:45.187 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:45.187 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:45.187 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:11:45.187 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.187 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:11:45.187 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.188 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:11:45.188 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:11:45.188 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:45.188 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:45.188 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.188 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:45.188 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:11:45.188 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:45.188 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.188 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.188 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:45.188 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:11:45.188 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:45.188 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:45.189 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:45.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.189 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:11:45.189 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:11:45.189 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:45.189 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:45.189 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:45.189 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.189 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.189 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:11:45.189 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:11:45.189 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:45.189 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:45.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.190 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:45.190 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:11:45.190 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:45.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.190 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.190 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:45.190 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:11:45.190 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:45.190 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:45.190 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:45.190 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.190 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:11:45.190 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:11:45.191 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:45.191 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:45.191 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:45.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.191 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:11:45.191 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:11:45.191 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:45.191 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:45.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.191 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:45.191 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:11:45.191 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:45.191 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.191 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.192 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:45.192 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:11:45.192 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:45.192 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:45.192 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:45.192 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.192 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:11:45.192 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:11:45.192 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:45.192 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:45.192 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:45.192 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:45.192 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:45.193 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.193 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:11:45.193 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:11:45.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.193 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:45.193 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:45.193 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:11:45.193 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:45.193 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:45.193 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:11:45.193 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.193 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:45.193 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:45.193 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:11:45.194 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:45.194 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:45.194 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:11:45.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.194 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:45.194 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:45.194 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:11:45.194 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:45.194 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:45.194 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:11:45.194 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.194 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:45.194 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:45.195 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:11:45.195 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:45.195 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:45.195 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.195 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:11:45.195 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:11:45.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd552893700 1 -- 172.21.15.40:0/2589486453 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd5300666e0 msgr2=0x7fd530068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:45.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd552893700 1 --2- 172.21.15.40:0/2589486453 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd5300666e0 0x7fd530068ba0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd544011580 tx=0x7fd54400b040 comp rx=0 tx=0).stop 2024-09-13T18:11:45.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd552893700 1 -- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c0724d0 msgr2=0x7fd54c151510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd552893700 1 --2- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c0724d0 0x7fd54c151510 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd548014040 tx=0x7fd548004770 comp rx=0 tx=0).stop 2024-09-13T18:11:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd552092700 1 -- 172.21.15.40:0/2589486453 reap_dead start 2024-09-13T18:11:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd552893700 1 -- 172.21.15.40:0/2589486453 shutdown_connections 2024-09-13T18:11:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd552893700 1 --2- 172.21.15.40:0/2589486453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd54c0724d0 0x7fd54c151510 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd552893700 1 -- 172.21.15.40:0/2589486453 >> 172.21.15.40:0/2589486453 conn(0x7fd54c06a390 msgr2=0x7fd54c0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd552893700 1 -- 172.21.15.40:0/2589486453 shutdown_connections 2024-09-13T18:11:45.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.172+0000 7fd552893700 1 -- 172.21.15.40:0/2589486453 wait complete. 2024-09-13T18:11:45.197 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:11:45.197 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T18:11:45.198 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:11:45.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.312+0000 7fc043fff700 1 Processor -- start 2024-09-13T18:11:45.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.312+0000 7fc043fff700 1 -- start start 2024-09-13T18:11:45.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc043fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc03c0d6990 0x7fc03c046e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:45.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc043fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc03c04de20 0x7fc03c0dcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:45.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc043fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc03c0dfa70 con 0x7fc03c0e04c0 2024-09-13T18:11:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc043fff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03c04cf30 con 0x7fc03c0d6990 2024-09-13T18:11:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc043fff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03c04d210 con 0x7fc03c04de20 2024-09-13T18:11:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc042ffd700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7fc03c0d6990 msgr2=0x7fc03c046e00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:11:45.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc042ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc03c0d6990 0x7fc03c046e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:11:45.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc0427fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc03c04de20 0x7fc03c0dcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:45.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc0427fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc03c04de20 0x7fc03c0dcf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:55904/0 (socket says 172.21.15.40:55904) 2024-09-13T18:11:45.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc0427fc700 1 -- 172.21.15.40:0/3406218851 learned_addr learned my addr 172.21.15.40:0/3406218851 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:45.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc0437fe700 1 --1- 172.21.15.40:0/3406218851 >> v1:172.21.15.43:6789/0 conn(0x7fc03c0e04c0 0x7fc03c051ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:53160/0 (socket says 172.21.15.40:53160) 2024-09-13T18:11:45.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc0427fc700 1 -- 172.21.15.40:0/3406218851 >> v2:172.21.15.40:3300/0 conn(0x7fc03c0d6990 msgr2=0x7fc03c046e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:11:45.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc0427fc700 1 --2- 172.21.15.40:0/3406218851 >> v2:172.21.15.40:3300/0 conn(0x7fc03c0d6990 0x7fc03c046e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc0427fc700 1 -- 172.21.15.40:0/3406218851 >> v1:172.21.15.43:6789/0 conn(0x7fc03c0e04c0 legacy=0x7fc03c051ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:45.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc0427fc700 1 -- 172.21.15.40:0/3406218851 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03c04d4f0 con 0x7fc03c04de20 2024-09-13T18:11:45.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc0427fc700 1 --2- 172.21.15.40:0/3406218851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03c04de20 0x7fc03c0dcf00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc03c04fe10 tx=0x7fc034009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60e57f437277be00 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:45.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc041ffb700 1 -- 172.21.15.40:0/3406218851 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03400b8a0 con 0x7fc03c04de20 2024-09-13T18:11:45.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc041ffb700 1 -- 172.21.15.40:0/3406218851 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03400ba00 con 0x7fc03c04de20 2024-09-13T18:11:45.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc041ffb700 1 -- 172.21.15.40:0/3406218851 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03400bd90 con 0x7fc03c04de20 2024-09-13T18:11:45.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc043fff700 1 -- 172.21.15.40:0/3406218851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03c04de20 msgr2=0x7fc03c0dcf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:45.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc043fff700 1 --2- 172.21.15.40:0/3406218851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03c04de20 0x7fc03c0dcf00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc03c04fe10 tx=0x7fc034009bf0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc043fff700 1 -- 172.21.15.40:0/3406218851 shutdown_connections 2024-09-13T18:11:45.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc043fff700 1 --2- 172.21.15.40:0/3406218851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03c04de20 0x7fc03c0dcf00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc043fff700 1 --2- 172.21.15.40:0/3406218851 >> v2:172.21.15.40:3300/0 conn(0x7fc03c0d6990 0x7fc03c046e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc043fff700 1 -- 172.21.15.40:0/3406218851 >> 172.21.15.40:0/3406218851 conn(0x7fc03c009a40 msgr2=0x7fc03c009e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:45.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc043fff700 1 -- 172.21.15.40:0/3406218851 shutdown_connections 2024-09-13T18:11:45.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.316+0000 7fc043fff700 1 -- 172.21.15.40:0/3406218851 wait complete. 2024-09-13T18:11:45.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc043fff700 1 Processor -- start 2024-09-13T18:11:45.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc043fff700 1 -- start start 2024-09-13T18:11:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc043fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03c04de20 0x7fc03c04b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc043fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc03c0d6990 0x7fc03c0da960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc043fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc03c0e04c0 0x7fc03c0db550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc043fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c04a660 con 0x7fc03c0e04c0 2024-09-13T18:11:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc043fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c04cc90 con 0x7fc03c04de20 2024-09-13T18:11:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc043fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c1787a0 con 0x7fc03c0d6990 2024-09-13T18:11:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc0437fe700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc03c0e04c0 msgr2=0x7fc03c0db550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:11:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc0437fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc03c0e04c0 0x7fc03c0db550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:11:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc042ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03c04de20 0x7fc03c04b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc042ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03c04de20 0x7fc03c04b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:55914/0 (socket says 172.21.15.40:55914) 2024-09-13T18:11:45.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc042ffd700 1 -- 172.21.15.40:0/154446667 learned_addr learned my addr 172.21.15.40:0/154446667 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:11:45.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc0427fc700 1 --2- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc03c0d6990 0x7fc03c0da960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:45.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc0427fc700 1 -- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03c04de20 msgr2=0x7fc03c04b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:45.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc0427fc700 1 --2- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03c04de20 0x7fc03c04b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc0427fc700 1 -- 172.21.15.40:0/154446667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc03c0e04c0 msgr2=0x7fc03c0db550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:11:45.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc0427fc700 1 --2- 172.21.15.40:0/154446667 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc03c0e04c0 0x7fc03c0db550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc0427fc700 1 -- 172.21.15.40:0/154446667 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc034009570 con 0x7fc03c0d6990 2024-09-13T18:11:45.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc042ffd700 1 --2- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03c04de20 0x7fc03c04b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:11:45.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc0427fc700 1 --2- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc03c0d6990 0x7fc03c0da960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc034016000 tx=0x7fc034004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:45.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc02b7fe700 1 -- 172.21.15.40:0/154446667 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034003ad0 con 0x7fc03c0d6990 2024-09-13T18:11:45.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc02b7fe700 1 -- 172.21.15.40:0/154446667 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03401e070 con 0x7fc03c0d6990 2024-09-13T18:11:45.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.320+0000 7fc043fff700 1 -- 172.21.15.40:0/154446667 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc03c04a040 con 0x7fc03c0d6990 2024-09-13T18:11:45.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.324+0000 7fc02b7fe700 1 -- 172.21.15.40:0/154446667 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034003e90 con 0x7fc03c0d6990 2024-09-13T18:11:45.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.324+0000 7fc043fff700 1 -- 172.21.15.40:0/154446667 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03c04ba60 con 0x7fc03c0d6990 2024-09-13T18:11:45.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.324+0000 7fc043fff700 1 --2- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc03c0da2f0 0x7fc03c177d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:45.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.324+0000 7fc043fff700 1 -- 172.21.15.40:0/154446667 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c0074b0 con 0x7fc03c0da2f0 2024-09-13T18:11:45.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.324+0000 7fc042ffd700 1 --2- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc03c0da2f0 0x7fc03c177d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:45.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.324+0000 7fc02b7fe700 1 -- 172.21.15.40:0/154446667 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc034011420 con 0x7fc03c0d6990 2024-09-13T18:11:45.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.324+0000 7fc042ffd700 1 -- 172.21.15.40:0/154446667 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc03c0e08d0 con 0x7fc03c0da2f0 2024-09-13T18:11:45.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.324+0000 7fc042ffd700 1 --2- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc03c0da2f0 0x7fc03c177d30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc0380055f0 tx=0x7fc0380059b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.324+0000 7fc02b7fe700 1 --2- 172.21.15.40:0/154446667 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc0300666e0 0x7fc030068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.324+0000 7fc0437fe700 1 --2- 172.21.15.40:0/154446667 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc0300666e0 0x7fc030068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.324+0000 7fc02b7fe700 1 -- 172.21.15.40:0/154446667 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc034020070 con 0x7fc03c0d6990 2024-09-13T18:11:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.324+0000 7fc02b7fe700 1 -- 172.21.15.40:0/154446667 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03800e8b0 con 0x7fc03c0da2f0 2024-09-13T18:11:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.324+0000 7fc02b7fe700 1 -- 172.21.15.40:0/154446667 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc03800ea10 con 0x7fc03c0da2f0 2024-09-13T18:11:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.324+0000 7fc02b7fe700 1 -- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc03c0da2f0 msgr2=0x7fc03c177d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.324+0000 7fc02b7fe700 1 --2- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc03c0da2f0 0x7fc03c177d30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc0380055f0 tx=0x7fc0380059b0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.324+0000 7fc02b7fe700 1 -- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc03c0da2f0 msgr2=0x7fc03c177d30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:11:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.324+0000 7fc02b7fe700 1 --2- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc03c0da2f0 0x7fc03c177d30 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc0380055f0 tx=0x7fc0380059b0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.324+0000 7fc0437fe700 1 --2- 172.21.15.40:0/154446667 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc0300666e0 0x7fc030068ba0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc04404ffd0 tx=0x7fc044065ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:45.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.332+0000 7fc043fff700 1 --2- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc03c178270 0x7fc03c008a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:11:45.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.332+0000 7fc043fff700 1 -- 172.21.15.40:0/154446667 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c0dffa0 con 0x7fc03c178270 2024-09-13T18:11:45.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.336+0000 7fc042ffd700 1 --2- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc03c178270 0x7fc03c008a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:11:45.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.336+0000 7fc042ffd700 1 -- 172.21.15.40:0/154446667 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc03c049ea0 con 0x7fc03c178270 2024-09-13T18:11:45.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.336+0000 7fc042ffd700 1 --2- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc03c178270 0x7fc03c008a70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc038018000 tx=0x7fc038005a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:11:45.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.336+0000 7fc02b7fe700 1 -- 172.21.15.40:0/154446667 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc038005cf0 con 0x7fc03c178270 2024-09-13T18:11:45.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.336+0000 7fc02b7fe700 1 -- 172.21.15.40:0/154446667 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4937 (secure 0 0 0) 0x7fc038005e50 con 0x7fc03c178270 2024-09-13T18:11:45.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.336+0000 7fc02b7fe700 1 -- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc03c178270 msgr2=0x7fc03c008a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:45.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.336+0000 7fc02b7fe700 1 --2- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc03c178270 0x7fc03c008a70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc038018000 tx=0x7fc038005a80 comp rx=0 tx=0).stop 2024-09-13T18:11:45.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.336+0000 7fc02b7fe700 1 -- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc03c178270 msgr2=0x7fc03c008a70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:11:45.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.336+0000 7fc02b7fe700 1 --2- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc03c178270 0x7fc03c008a70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.345 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:11:45.345 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:11:45.345 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:11:45.345 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:11:45.345 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 168, 2024-09-13T18:11:45.345 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:11:45.345 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:11:45.345 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:11:45.346 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:45.346 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:11:45.346 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:11:45.346 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:11:45.346 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:11:45.346 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:45.346 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:45.346 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:45.346 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:45.347 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:45.347 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:45.347 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:45.347 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:45.347 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:45.347 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:45.347 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:45.347 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:11:45.347 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:11:45.347 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:45.347 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:45.347 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:45.347 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:45.347 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:45.348 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:45.348 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:45.348 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:45.348 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:45.348 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:45.348 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:45.348 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.348 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:11:45.348 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:11:45.348 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:11:45.348 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:11:45.348 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:11:45.348 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:11:45.349 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:11:45.349 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:11:45.349 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:11:45.349 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:11:45.349 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:11:45.349 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:11:45.349 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:11:45.349 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:11:45.349 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:11:45.349 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:11:45.349 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:11:45.349 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:11:45.349 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:11:45.349 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:11:45.350 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:11:45.350 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:11:45.350 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:11:45.350 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:11:45.350 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:11:45.350 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:11:45.350 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:11:45.350 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:45.350 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:11:45.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.350 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:11:45.350 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.350 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:11:45.350 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:11:45.351 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:45.351 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:45.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.351 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:45.351 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:11:45.351 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:45.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.351 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:45.351 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:11:45.351 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:45.351 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:45.351 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:45.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.352 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:11:45.352 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:11:45.352 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:45.352 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:45.352 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:45.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.352 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:11:45.352 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:11:45.352 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:45.352 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:45.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.352 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:45.353 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:11:45.353 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:45.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.353 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:45.353 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:11:45.353 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:45.353 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:45.353 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:45.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.353 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:11:45.353 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:11:45.353 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:45.353 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:45.353 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:45.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.354 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:11:45.354 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:11:45.354 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:11:45.354 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:11:45.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.354 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:11:45.354 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:11:45.354 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:45.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.354 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:11:45.354 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:11:45.355 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:11:45.355 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:45.355 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:45.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.355 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:11:45.355 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:11:45.355 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:11:45.355 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:11:45.355 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:11:45.355 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:45.355 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:45.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.355 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:11:45.355 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:11:45.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.356 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:45.356 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:45.356 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:11:45.356 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:45.356 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:45.356 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:11:45.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.356 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:45.356 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:45.356 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:11:45.356 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:45.356 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:45.356 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:11:45.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.357 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:45.357 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:45.357 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:11:45.357 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:45.357 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:45.357 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:11:45.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.357 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:45.357 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:45.357 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:11:45.357 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:45.357 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:11:45.357 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:11:45.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:11:45.358 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:11:45.358 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:11:45.358 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:11:45.358 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:11:45.358 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:11:45.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:11:45.358 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:11:45.358 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:11:45.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.336+0000 7fc043fff700 1 -- 172.21.15.40:0/154446667 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc0300666e0 msgr2=0x7fc030068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:45.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.336+0000 7fc043fff700 1 --2- 172.21.15.40:0/154446667 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc0300666e0 0x7fc030068ba0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc04404ffd0 tx=0x7fc044065ea0 comp rx=0 tx=0).stop 2024-09-13T18:11:45.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.336+0000 7fc043fff700 1 -- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc03c0d6990 msgr2=0x7fc03c0da960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:11:45.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.336+0000 7fc043fff700 1 --2- 172.21.15.40:0/154446667 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc03c0d6990 0x7fc03c0da960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc034016000 tx=0x7fc034004770 comp rx=0 tx=0).stop 2024-09-13T18:11:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.336+0000 7fc0437fe700 1 -- 172.21.15.40:0/154446667 reap_dead start 2024-09-13T18:11:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.336+0000 7fc0437fe700 1 -- 172.21.15.40:0/154446667 reap_dead start 2024-09-13T18:11:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.336+0000 7fc043fff700 1 -- 172.21.15.40:0/154446667 shutdown_connections 2024-09-13T18:11:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.336+0000 7fc043fff700 1 -- 172.21.15.40:0/154446667 >> 172.21.15.40:0/154446667 conn(0x7fc03c009a40 msgr2=0x7fc03c04e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:11:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.336+0000 7fc043fff700 1 -- 172.21.15.40:0/154446667 shutdown_connections 2024-09-13T18:11:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:11:45.336+0000 7fc043fff700 1 -- 172.21.15.40:0/154446667 wait complete. 2024-09-13T18:11:45.360 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:11:45.360 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:12:05.362 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T18:12:05.362 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-13T18:12:05.362 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T18:12:05.362 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T18:12:05.365 INFO:tasks.ceph.mon.a:Started 2024-09-13T18:12:05.365 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:12:05.365 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:12:06.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.264+0000 7f1c67e4c700 1 Processor -- start 2024-09-13T18:12:06.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.268+0000 7f1c67e4c700 1 -- start start 2024-09-13T18:12:06.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.268+0000 7f1c67e4c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1c6014ffb0 0x7f1c600c0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:06.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.268+0000 7f1c67e4c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1c600c0090 0x7f1c60156740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:06.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.268+0000 7f1c67e4c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c60159cf0 con 0x7f1c6006dc90 2024-09-13T18:12:06.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.268+0000 7f1c67e4c700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c6006e1f0 con 0x7f1c6014ffb0 2024-09-13T18:12:06.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.268+0000 7f1c67e4c700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c600cbd00 con 0x7f1c600c0090 2024-09-13T18:12:06.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.268+0000 7f1c65be8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1c6014ffb0 0x7f1c600c0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:06.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.268+0000 7f1c65be8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1c6014ffb0 0x7f1c600c0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40756/0 (socket says 172.21.15.40:40756) 2024-09-13T18:12:06.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.268+0000 7f1c65be8700 1 -- 172.21.15.40:0/2356864420 learned_addr learned my addr 172.21.15.40:0/2356864420 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:06.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.268+0000 7f1c65be8700 1 -- 172.21.15.40:0/2356864420 >> v2:172.21.15.40:3300/0 conn(0x7f1c6014ffb0 msgr2=0x7f1c600c0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:12:06.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.268+0000 7f1c65be8700 1 -- 172.21.15.40:0/2356864420 >> v2:172.21.15.40:3300/0 conn(0x7f1c6014ffb0 msgr2=0x7f1c600c0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:12:06.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.268+0000 7f1c65be8700 1 --2- 172.21.15.40:0/2356864420 >> v2:172.21.15.40:3300/0 conn(0x7f1c6014ffb0 0x7f1c600c0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:06.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.268+0000 7f1c663e9700 1 --2- 172.21.15.40:0/2356864420 >> v2:172.21.15.43:3300/0 conn(0x7f1c600c0090 0x7f1c60156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:06.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.268+0000 7f1c65be8700 1 --2- 172.21.15.40:0/2356864420 >> v2:172.21.15.40:3300/0 conn(0x7f1c6014ffb0 0x7f1c600c0670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:12:06.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.268+0000 7f1c663e9700 1 -- 172.21.15.40:0/2356864420 >> v2:172.21.15.40:3300/0 conn(0x7f1c6014ffb0 msgr2=0x7f1c600c0670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:12:06.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.268+0000 7f1c663e9700 1 --2- 172.21.15.40:0/2356864420 >> v2:172.21.15.40:3300/0 conn(0x7f1c6014ffb0 0x7f1c600c0670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:06.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.268+0000 7f1c663e9700 1 -- 172.21.15.40:0/2356864420 >> v1:172.21.15.43:6789/0 conn(0x7f1c6006dc90 legacy=0x7f1c6006a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:06.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.268+0000 7f1c663e9700 1 -- 172.21.15.40:0/2356864420 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c60155c60 con 0x7f1c600c0090 2024-09-13T18:12:06.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.272+0000 7f1c663e9700 1 --2- 172.21.15.40:0/2356864420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c600c0090 0x7f1c60156740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1c60066bd0 tx=0x7f1c5800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa6dcaac7275975f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:06.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.272+0000 7f1c64be6700 1 -- 172.21.15.40:0/2356864420 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5800e070 con 0x7f1c600c0090 2024-09-13T18:12:06.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.272+0000 7f1c64be6700 1 -- 172.21.15.40:0/2356864420 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5800bbd0 con 0x7f1c600c0090 2024-09-13T18:12:06.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.272+0000 7f1c64be6700 1 -- 172.21.15.40:0/2356864420 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c58012430 con 0x7f1c600c0090 2024-09-13T18:12:06.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.272+0000 7f1c67e4c700 1 -- 172.21.15.40:0/2356864420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c600c0090 msgr2=0x7f1c60156740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:06.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.272+0000 7f1c67e4c700 1 --2- 172.21.15.40:0/2356864420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c600c0090 0x7f1c60156740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1c60066bd0 tx=0x7f1c5800b330 comp rx=0 tx=0).stop 2024-09-13T18:12:06.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.272+0000 7f1c67e4c700 1 -- 172.21.15.40:0/2356864420 shutdown_connections 2024-09-13T18:12:06.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.272+0000 7f1c67e4c700 1 --2- 172.21.15.40:0/2356864420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c600c0090 0x7f1c60156740 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:06.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.272+0000 7f1c67e4c700 1 --2- 172.21.15.40:0/2356864420 >> v2:172.21.15.40:3300/0 conn(0x7f1c6014ffb0 0x7f1c600c0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:06.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.272+0000 7f1c67e4c700 1 -- 172.21.15.40:0/2356864420 >> 172.21.15.40:0/2356864420 conn(0x7f1c600cadd0 msgr2=0x7f1c600cb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:06.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.272+0000 7f1c67e4c700 1 -- 172.21.15.40:0/2356864420 shutdown_connections 2024-09-13T18:12:06.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.272+0000 7f1c67e4c700 1 -- 172.21.15.40:0/2356864420 wait complete. 2024-09-13T18:12:06.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.272+0000 7f1c67e4c700 1 Processor -- start 2024-09-13T18:12:06.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.272+0000 7f1c67e4c700 1 -- start start 2024-09-13T18:12:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c67e4c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c6006dc90 0x7f1c60157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c67e4c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c600c0090 0x7f1c60158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c67e4c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c6014ffb0 0x7f1c6014b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c67e4c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6014b0b0 con 0x7f1c6006dc90 2024-09-13T18:12:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c67e4c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c60151340 con 0x7f1c600c0090 2024-09-13T18:12:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c67e4c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6014b370 con 0x7f1c6014ffb0 2024-09-13T18:12:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c65be8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c6006dc90 0x7f1c60157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c65be8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c6006dc90 0x7f1c60157f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40772/0 (socket says 172.21.15.40:40772) 2024-09-13T18:12:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c65be8700 1 -- 172.21.15.40:0/809271168 learned_addr learned my addr 172.21.15.40:0/809271168 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c663e9700 1 --2- 172.21.15.40:0/809271168 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c6014ffb0 0x7f1c6014b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c653e7700 1 --2- 172.21.15.40:0/809271168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c600c0090 0x7f1c60158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:06.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c65be8700 1 -- 172.21.15.40:0/809271168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c6006dc90 msgr2=0x7f1c60157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:12:06.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c65be8700 1 -- 172.21.15.40:0/809271168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c6006dc90 msgr2=0x7f1c60157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:12:06.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c65be8700 1 --2- 172.21.15.40:0/809271168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c6006dc90 0x7f1c60157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:06.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c65be8700 1 --2- 172.21.15.40:0/809271168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c6006dc90 0x7f1c60157f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:12:06.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c653e7700 1 -- 172.21.15.40:0/809271168 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c6014ffb0 msgr2=0x7f1c6014b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:06.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c653e7700 1 --2- 172.21.15.40:0/809271168 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c6014ffb0 0x7f1c6014b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:06.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c653e7700 1 -- 172.21.15.40:0/809271168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c6006dc90 msgr2=0x7f1c60157f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:12:06.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c653e7700 1 --2- 172.21.15.40:0/809271168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c6006dc90 0x7f1c60157f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:06.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c653e7700 1 -- 172.21.15.40:0/809271168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c58009a70 con 0x7f1c600c0090 2024-09-13T18:12:06.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c653e7700 1 --2- 172.21.15.40:0/809271168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c600c0090 0x7f1c60158490 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000c430 tx=0x7f1c5000c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:06.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c567fc700 1 -- 172.21.15.40:0/809271168 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c500052d0 con 0x7f1c600c0090 2024-09-13T18:12:06.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c567fc700 1 -- 172.21.15.40:0/809271168 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c50005450 con 0x7f1c600c0090 2024-09-13T18:12:06.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c67e4c700 1 -- 172.21.15.40:0/809271168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c601504b0 con 0x7f1c600c0090 2024-09-13T18:12:06.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c567fc700 1 -- 172.21.15.40:0/809271168 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50006ac0 con 0x7f1c600c0090 2024-09-13T18:12:06.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.276+0000 7f1c67e4c700 1 -- 172.21.15.40:0/809271168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c60151890 con 0x7f1c600c0090 2024-09-13T18:12:06.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.280+0000 7f1c67e4c700 1 -- 172.21.15.40:0/809271168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c6014b4b0 con 0x7f1c600c0090 2024-09-13T18:12:06.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.280+0000 7f1c567fc700 1 -- 172.21.15.40:0/809271168 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1c500036a0 con 0x7f1c600c0090 2024-09-13T18:12:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.280+0000 7f1c567fc700 1 --2- 172.21.15.40:0/809271168 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1c44066730 0x7f1c44068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.280+0000 7f1c65be8700 1 --2- 172.21.15.40:0/809271168 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1c44066730 0x7f1c44068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:06.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.280+0000 7f1c567fc700 1 -- 172.21.15.40:0/809271168 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1c50083740 con 0x7f1c600c0090 2024-09-13T18:12:06.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.280+0000 7f1c65be8700 1 --2- 172.21.15.40:0/809271168 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1c44066730 0x7f1c44068bf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1c600c9f70 tx=0x7f1c5c00b420 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:06.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.288+0000 7f1c567fc700 1 -- 172.21.15.40:0/809271168 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1c6014b4b0 con 0x7f1c600c0090 2024-09-13T18:12:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.560+0000 7f1c67e4c700 1 -- 172.21.15.40:0/809271168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1c6014b4b0 con 0x7f1c600c0090 2024-09-13T18:12:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.564+0000 7f1c567fc700 1 -- 172.21.15.40:0/809271168 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f1c6014b4b0 con 0x7f1c600c0090 2024-09-13T18:12:06.570 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":168,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:12:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.564+0000 7f1c67e4c700 1 -- 172.21.15.40:0/809271168 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1c44066730 msgr2=0x7f1c44068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.564+0000 7f1c67e4c700 1 --2- 172.21.15.40:0/809271168 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1c44066730 0x7f1c44068bf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1c600c9f70 tx=0x7f1c5c00b420 comp rx=0 tx=0).stop 2024-09-13T18:12:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.564+0000 7f1c67e4c700 1 -- 172.21.15.40:0/809271168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c600c0090 msgr2=0x7f1c60158490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.564+0000 7f1c67e4c700 1 --2- 172.21.15.40:0/809271168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c600c0090 0x7f1c60158490 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000c430 tx=0x7f1c5000c740 comp rx=0 tx=0).stop 2024-09-13T18:12:06.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.564+0000 7f1c67e4c700 1 -- 172.21.15.40:0/809271168 shutdown_connections 2024-09-13T18:12:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.564+0000 7f1c67e4c700 1 --2- 172.21.15.40:0/809271168 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1c44066730 0x7f1c44068bf0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.564+0000 7f1c67e4c700 1 --2- 172.21.15.40:0/809271168 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1c6014ffb0 0x7f1c6014b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.564+0000 7f1c67e4c700 1 --2- 172.21.15.40:0/809271168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c600c0090 0x7f1c60158490 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.564+0000 7f1c67e4c700 1 --2- 172.21.15.40:0/809271168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c6006dc90 0x7f1c60157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.564+0000 7f1c67e4c700 1 -- 172.21.15.40:0/809271168 >> 172.21.15.40:0/809271168 conn(0x7f1c600cadd0 msgr2=0x7f1c600cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.564+0000 7f1c67e4c700 1 -- 172.21.15.40:0/809271168 shutdown_connections 2024-09-13T18:12:06.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:06.564+0000 7f1c67e4c700 1 -- 172.21.15.40:0/809271168 wait complete. 2024-09-13T18:12:07.590 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:12:07.592 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:12:09.658 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:12:09.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.848+0000 7f93ecb25700 1 Processor -- start 2024-09-13T18:12:09.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.848+0000 7f93ecb25700 1 -- start start 2024-09-13T18:12:09.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.848+0000 7f93ecb25700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f93e8071e60 0x7f93e80c0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:09.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.848+0000 7f93ecb25700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f93e8072240 0x7f93e80c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:09.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.848+0000 7f93ecb25700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e80b2210 con 0x7f93e80c28f0 2024-09-13T18:12:09.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.848+0000 7f93ecb25700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e80c3820 con 0x7f93e8071e60 2024-09-13T18:12:09.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.848+0000 7f93ecb25700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e80c1780 con 0x7f93e8072240 2024-09-13T18:12:09.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.848+0000 7f93e77fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f93e8071e60 0x7f93e80c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:09.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.848+0000 7f93e7fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f93e80c28f0 0x7f93e81565c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46294/0 (socket says 172.21.15.40:46294) 2024-09-13T18:12:09.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.848+0000 7f93e7fff700 1 -- 172.21.15.40:0/2677607859 learned_addr learned my addr 172.21.15.40:0/2677607859 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:12:09.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.848+0000 7f93dffff700 1 --2- 172.21.15.40:0/2677607859 >> v2:172.21.15.43:3300/0 conn(0x7f93e8072240 0x7f93e80c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:09.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.848+0000 7f93e77fe700 1 -- 172.21.15.40:0/2677607859 >> v2:172.21.15.40:3300/0 conn(0x7f93e8071e60 msgr2=0x7f93e80c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:12:09.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.848+0000 7f93e77fe700 1 -- 172.21.15.40:0/2677607859 >> v2:172.21.15.40:3300/0 conn(0x7f93e8071e60 msgr2=0x7f93e80c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:12:09.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.848+0000 7f93e77fe700 1 --2- 172.21.15.40:0/2677607859 >> v2:172.21.15.40:3300/0 conn(0x7f93e8071e60 0x7f93e80c0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:09.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.852+0000 7f93e77fe700 1 --2- 172.21.15.40:0/2677607859 >> v2:172.21.15.40:3300/0 conn(0x7f93e8071e60 0x7f93e80c0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:12:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.852+0000 7f93dffff700 1 -- 172.21.15.40:0/2677607859 >> v2:172.21.15.40:3300/0 conn(0x7f93e8071e60 msgr2=0x7f93e80c0e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:12:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.852+0000 7f93dffff700 1 --2- 172.21.15.40:0/2677607859 >> v2:172.21.15.40:3300/0 conn(0x7f93e8071e60 0x7f93e80c0e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.852+0000 7f93dffff700 1 -- 172.21.15.40:0/2677607859 >> v1:172.21.15.40:6789/0 conn(0x7f93e80c28f0 legacy=0x7f93e81565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.852+0000 7f93dffff700 1 -- 172.21.15.40:0/2677607859 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e80c1a60 con 0x7f93e8072240 2024-09-13T18:12:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.852+0000 7f93dffff700 1 --2- 172.21.15.40:0/2677607859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93e8072240 0x7f93e80c32e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f93e80b1b60 tx=0x7f93d0009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=563893e5e00ad186 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.852+0000 7f93e6ffd700 1 -- 172.21.15.40:0/2677607859 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d0011830 con 0x7f93e8072240 2024-09-13T18:12:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.852+0000 7f93e6ffd700 1 -- 172.21.15.40:0/2677607859 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d0011990 con 0x7f93e8072240 2024-09-13T18:12:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.852+0000 7f93e6ffd700 1 -- 172.21.15.40:0/2677607859 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d0011d40 con 0x7f93e8072240 2024-09-13T18:12:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.852+0000 7f93ecb25700 1 -- 172.21.15.40:0/2677607859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93e8072240 msgr2=0x7f93e80c32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.852+0000 7f93ecb25700 1 --2- 172.21.15.40:0/2677607859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93e8072240 0x7f93e80c32e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f93e80b1b60 tx=0x7f93d0009b30 comp rx=0 tx=0).stop 2024-09-13T18:12:09.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.852+0000 7f93ecb25700 1 -- 172.21.15.40:0/2677607859 shutdown_connections 2024-09-13T18:12:09.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.852+0000 7f93ecb25700 1 --2- 172.21.15.40:0/2677607859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93e8072240 0x7f93e80c32e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:09.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.852+0000 7f93ecb25700 1 --2- 172.21.15.40:0/2677607859 >> v2:172.21.15.40:3300/0 conn(0x7f93e8071e60 0x7f93e80c0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:09.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.852+0000 7f93ecb25700 1 -- 172.21.15.40:0/2677607859 >> 172.21.15.40:0/2677607859 conn(0x7f93e806e4f0 msgr2=0x7f93e806e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:09.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.852+0000 7f93ecb25700 1 -- 172.21.15.40:0/2677607859 shutdown_connections 2024-09-13T18:12:09.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.852+0000 7f93ecb25700 1 -- 172.21.15.40:0/2677607859 wait complete. 2024-09-13T18:12:09.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93ecb25700 1 Processor -- start 2024-09-13T18:12:09.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93ecb25700 1 -- start start 2024-09-13T18:12:09.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93ecb25700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93e8071e60 0x7f93e814bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:09.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93ecb25700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93e8072240 0x7f93e81f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:09.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93ecb25700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f93e80c28f0 0x7f93e81f0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:09.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93ecb25700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e814e740 con 0x7f93e8072240 2024-09-13T18:12:09.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93ecb25700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e80c40f0 con 0x7f93e8071e60 2024-09-13T18:12:09.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93ecb25700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e8152990 con 0x7f93e80c28f0 2024-09-13T18:12:09.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93dffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93e8072240 0x7f93e81f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:09.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93dffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93e8072240 0x7f93e81f1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:40790/0 (socket says 172.21.15.40:40790) 2024-09-13T18:12:09.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93dffff700 1 -- 172.21.15.40:0/3125866204 learned_addr learned my addr 172.21.15.40:0/3125866204 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:09.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93e77fe700 1 --2- 172.21.15.40:0/3125866204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93e8071e60 0x7f93e814bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:09.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93e7fff700 1 --2- 172.21.15.40:0/3125866204 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f93e80c28f0 0x7f93e81f0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:09.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93dffff700 1 -- 172.21.15.40:0/3125866204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93e8072240 msgr2=0x7f93e81f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:12:09.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93dffff700 1 -- 172.21.15.40:0/3125866204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93e8072240 msgr2=0x7f93e81f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:12:09.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93dffff700 1 --2- 172.21.15.40:0/3125866204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93e8072240 0x7f93e81f1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:09.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93dffff700 1 --2- 172.21.15.40:0/3125866204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93e8072240 0x7f93e81f1fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:12:09.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93e77fe700 1 -- 172.21.15.40:0/3125866204 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f93e80c28f0 msgr2=0x7f93e81f0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:09.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93e77fe700 1 --2- 172.21.15.40:0/3125866204 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f93e80c28f0 0x7f93e81f0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:09.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93e77fe700 1 -- 172.21.15.40:0/3125866204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93e8072240 msgr2=0x7f93e81f1fd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:12:09.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93e77fe700 1 --2- 172.21.15.40:0/3125866204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93e8072240 0x7f93e81f1fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:09.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93e77fe700 1 -- 172.21.15.40:0/3125866204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d00094b0 con 0x7f93e8071e60 2024-09-13T18:12:09.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93e77fe700 1 --2- 172.21.15.40:0/3125866204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93e8071e60 0x7f93e814bf40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f93d800e7e0 tx=0x7f93d800eaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:09.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.856+0000 7f93e4ff9700 1 -- 172.21.15.40:0/3125866204 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d800c070 con 0x7f93e8071e60 2024-09-13T18:12:09.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.860+0000 7f93e4ff9700 1 -- 172.21.15.40:0/3125866204 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d8004020 con 0x7f93e8071e60 2024-09-13T18:12:09.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.860+0000 7f93ecb25700 1 -- 172.21.15.40:0/3125866204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e80c4350 con 0x7f93e8071e60 2024-09-13T18:12:09.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.860+0000 7f93e4ff9700 1 -- 172.21.15.40:0/3125866204 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d8004410 con 0x7f93e8071e60 2024-09-13T18:12:09.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.860+0000 7f93ecb25700 1 -- 172.21.15.40:0/3125866204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e814e460 con 0x7f93e8071e60 2024-09-13T18:12:09.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.864+0000 7f93e4ff9700 1 -- 172.21.15.40:0/3125866204 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f93d80036c0 con 0x7f93e8071e60 2024-09-13T18:12:09.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.864+0000 7f93ecb25700 1 -- 172.21.15.40:0/3125866204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e80c2d40 con 0x7f93e8071e60 2024-09-13T18:12:09.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.864+0000 7f93e4ff9700 1 --2- 172.21.15.40:0/3125866204 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f93c8066760 0x7f93c8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:09.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.864+0000 7f93e4ff9700 1 -- 172.21.15.40:0/3125866204 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f93d8083f90 con 0x7f93e8071e60 2024-09-13T18:12:09.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.864+0000 7f93dffff700 1 --2- 172.21.15.40:0/3125866204 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f93c8066760 0x7f93c8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:09.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.864+0000 7f93dffff700 1 --2- 172.21.15.40:0/3125866204 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f93c8066760 0x7f93c8068c20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f93e806dc60 tx=0x7f93d0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:09.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:09.876+0000 7f93e4ff9700 1 -- 172.21.15.40:0/3125866204 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f93d8050f30 con 0x7f93e8071e60 2024-09-13T18:12:10.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:10.152+0000 7f93ecb25700 1 -- 172.21.15.40:0/3125866204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f93e806d8a0 con 0x7f93e8071e60 2024-09-13T18:12:10.162 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:10.152+0000 7f93e4ff9700 1 -- 172.21.15.40:0/3125866204 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f93d8017610 con 0x7f93e8071e60 2024-09-13T18:12:10.162 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":168,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:12:10.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:10.156+0000 7f93ecb25700 1 -- 172.21.15.40:0/3125866204 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f93c8066760 msgr2=0x7f93c8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:10.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:10.156+0000 7f93ecb25700 1 --2- 172.21.15.40:0/3125866204 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f93c8066760 0x7f93c8068c20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f93e806dc60 tx=0x7f93d0008030 comp rx=0 tx=0).stop 2024-09-13T18:12:10.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:10.156+0000 7f93ecb25700 1 -- 172.21.15.40:0/3125866204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93e8071e60 msgr2=0x7f93e814bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:10.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:10.156+0000 7f93ecb25700 1 --2- 172.21.15.40:0/3125866204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93e8071e60 0x7f93e814bf40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f93d800e7e0 tx=0x7f93d800eaf0 comp rx=0 tx=0).stop 2024-09-13T18:12:10.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:10.156+0000 7f93ecb25700 1 -- 172.21.15.40:0/3125866204 shutdown_connections 2024-09-13T18:12:10.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:10.156+0000 7f93ecb25700 1 --2- 172.21.15.40:0/3125866204 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f93c8066760 0x7f93c8068c20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:10.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:10.156+0000 7f93ecb25700 1 --2- 172.21.15.40:0/3125866204 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f93e80c28f0 0x7f93e81f0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:10.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:10.156+0000 7f93ecb25700 1 --2- 172.21.15.40:0/3125866204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f93e8072240 0x7f93e81f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:10.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:10.156+0000 7f93ecb25700 1 --2- 172.21.15.40:0/3125866204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93e8071e60 0x7f93e814bf40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:10.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:10.156+0000 7f93ecb25700 1 -- 172.21.15.40:0/3125866204 >> 172.21.15.40:0/3125866204 conn(0x7f93e806e4f0 msgr2=0x7f93e8158a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:10.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:10.156+0000 7f93ecb25700 1 -- 172.21.15.40:0/3125866204 shutdown_connections 2024-09-13T18:12:10.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:10.156+0000 7f93ecb25700 1 -- 172.21.15.40:0/3125866204 wait complete. 2024-09-13T18:12:13.181 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:12:13.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.356+0000 7f877b59e700 1 Processor -- start 2024-09-13T18:12:13.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.356+0000 7f877b59e700 1 -- start start 2024-09-13T18:12:13.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.356+0000 7f877b59e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 0x7f877c159d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:13.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.356+0000 7f877b59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 0x7f877c0c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:13.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.356+0000 7f877b59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f877c0c6db0 con 0x7f877c070290 2024-09-13T18:12:13.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.356+0000 7f877b59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f877c0c6920 con 0x7f877c06e960 2024-09-13T18:12:13.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.356+0000 7f877b59e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f877c15a260 con 0x7f877c1534e0 2024-09-13T18:12:13.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.356+0000 7f8773fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f877c070290 0x7f877c06d490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33690/0 (socket says 172.21.15.40:33690) 2024-09-13T18:12:13.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.356+0000 7f8773fff700 1 -- 172.21.15.40:0/4110760541 learned_addr learned my addr 172.21.15.40:0/4110760541 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:12:13.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.356+0000 7f877ad9d700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 0x7f877c0c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:13.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.356+0000 7f877a59c700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 0x7f877c159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:13.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.356+0000 7f877ad9d700 1 -- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 msgr2=0x7f877c0c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:12:13.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.356+0000 7f877ad9d700 1 -- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 msgr2=0x7f877c0c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:12:13.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.356+0000 7f877ad9d700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 0x7f877c0c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:13.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.356+0000 7f877ad9d700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 0x7f877c0c7d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:12:13.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.356+0000 7f877a59c700 1 -- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 msgr2=0x7f877c159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:12:13.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.356+0000 7f877a59c700 1 -- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 msgr2=0x7f877c159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:12:13.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.356+0000 7f877a59c700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 0x7f877c159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:13.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.356+0000 7f877a59c700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 0x7f877c159d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:12:13.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.560+0000 7f877ad9d700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 0x7f877c0c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:13.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.560+0000 7f877a59c700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 0x7f877c159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:13.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.560+0000 7f877ad9d700 1 -- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 msgr2=0x7f877c0c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:12:13.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.560+0000 7f877ad9d700 1 -- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 msgr2=0x7f877c0c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:12:13.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.560+0000 7f877ad9d700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 0x7f877c0c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:13.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.560+0000 7f877ad9d700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 0x7f877c0c7d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:12:13.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.560+0000 7f877a59c700 1 -- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 msgr2=0x7f877c159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:12:13.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.560+0000 7f877a59c700 1 -- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 msgr2=0x7f877c159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:12:13.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.560+0000 7f877a59c700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 0x7f877c159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:13.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.560+0000 7f877a59c700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 0x7f877c159d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:12:13.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.964+0000 7f877ad9d700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 0x7f877c0c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:13.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.964+0000 7f877ad9d700 1 -- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 msgr2=0x7f877c0c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:12:13.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.964+0000 7f877ad9d700 1 -- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 msgr2=0x7f877c0c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:12:13.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.964+0000 7f877a59c700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 0x7f877c159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:13.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.964+0000 7f877ad9d700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 0x7f877c0c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:13.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.964+0000 7f877ad9d700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 0x7f877c0c7d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:12:13.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.964+0000 7f877a59c700 1 -- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 msgr2=0x7f877c159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:12:13.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.964+0000 7f877a59c700 1 -- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 msgr2=0x7f877c159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:12:13.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.964+0000 7f877a59c700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 0x7f877c159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:13.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:13.964+0000 7f877a59c700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 0x7f877c159d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:12:14.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:14.768+0000 7f877ad9d700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 0x7f877c0c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:14.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:14.768+0000 7f877a59c700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 0x7f877c159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:14.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:14.768+0000 7f877ad9d700 1 -- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 msgr2=0x7f877c0c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:12:14.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:14.768+0000 7f877ad9d700 1 -- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 msgr2=0x7f877c0c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:12:14.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:14.768+0000 7f877ad9d700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 0x7f877c0c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:14.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:14.768+0000 7f877ad9d700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 0x7f877c0c7d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:12:14.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:14.768+0000 7f877a59c700 1 -- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 msgr2=0x7f877c159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:12:14.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:14.768+0000 7f877a59c700 1 -- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 msgr2=0x7f877c159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:12:14.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:14.768+0000 7f877a59c700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 0x7f877c159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:14.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:14.768+0000 7f877a59c700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 0x7f877c159d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:12:15.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:15.716+0000 7f8773fff700 1 -- 172.21.15.40:0/4110760541 >> v1:172.21.15.40:6789/0 conn(0x7f877c070290 legacy=0x7f877c06d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-13T18:12:15.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:15.716+0000 7f8773fff700 1 -- 172.21.15.40:0/4110760541 >> v1:172.21.15.40:6789/0 conn(0x7f877c070290 legacy=0x7f877c06d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:12:15.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:15.716+0000 7f8773fff700 1 --1- 172.21.15.40:0/4110760541 >> v1:172.21.15.40:6789/0 conn(0x7f877c070290 0x7f877c06d490 :-1 s=OPENED pgs=25 cs=1 l=1).handle_message read tag failed 2024-09-13T18:12:15.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:15.716+0000 7f8773fff700 1 --1- 172.21.15.40:0/4110760541 >> v1:172.21.15.40:6789/0 conn(0x7f877c070290 0x7f877c06d490 :-1 s=OPENED pgs=25 cs=1 l=1).fault on lossy channel, failing 2024-09-13T18:12:16.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.356+0000 7f8778d99700 1 -- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 msgr2=0x7f877c159d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:12:16.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.356+0000 7f8778d99700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.43:3300/0 conn(0x7f877c1534e0 0x7f877c159d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:16.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.356+0000 7f8778d99700 1 -- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 msgr2=0x7f877c0c7d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:12:16.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.356+0000 7f8778d99700 1 --2- 172.21.15.40:0/4110760541 >> v2:172.21.15.40:3300/0 conn(0x7f877c06e960 0x7f877c0c7d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:16.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.356+0000 7f8778d99700 1 -- 172.21.15.40:0/4110760541 >> v1:172.21.15.40:6789/0 conn(0x7f877c070290 legacy=0x7f877c06d490 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:12:16.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.356+0000 7f8778d99700 1 --2- 172.21.15.40:0/4110760541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f875c003650 0x7f875c005aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:16.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.356+0000 7f8778d99700 1 -- 172.21.15.40:0/4110760541 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f877c0c5d50 con 0x7f875c006190 2024-09-13T18:12:16.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.356+0000 7f8778d99700 1 -- 172.21.15.40:0/4110760541 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f877c0c6920 con 0x7f875c003650 2024-09-13T18:12:16.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.356+0000 7f877a59c700 1 --2- 172.21.15.40:0/4110760541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f875c003650 0x7f875c005aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:16.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.360+0000 7f877a59c700 1 -- 172.21.15.40:0/4110760541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f875c003650 msgr2=0x7f875c005aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:12:16.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.360+0000 7f877a59c700 1 -- 172.21.15.40:0/4110760541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f875c003650 msgr2=0x7f875c005aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:12:16.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.360+0000 7f877a59c700 1 --2- 172.21.15.40:0/4110760541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f875c003650 0x7f875c005aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:16.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.360+0000 7f877a59c700 1 --2- 172.21.15.40:0/4110760541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f875c003650 0x7f875c005aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:12:16.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.564+0000 7f877a59c700 1 --2- 172.21.15.40:0/4110760541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f875c003650 0x7f875c005aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:16.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.564+0000 7f877a59c700 1 -- 172.21.15.40:0/4110760541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f875c003650 msgr2=0x7f875c005aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:12:16.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.564+0000 7f877a59c700 1 -- 172.21.15.40:0/4110760541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f875c003650 msgr2=0x7f875c005aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:12:16.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.564+0000 7f877a59c700 1 --2- 172.21.15.40:0/4110760541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f875c003650 0x7f875c005aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:16.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.564+0000 7f877a59c700 1 --2- 172.21.15.40:0/4110760541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f875c003650 0x7f875c005aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:12:16.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.964+0000 7f8779d9b700 1 -- 172.21.15.40:0/4110760541 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38150254 0 0) 0x7f877c0c5d50 con 0x7f875c006190 2024-09-13T18:12:16.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.964+0000 7f8779d9b700 1 -- 172.21.15.40:0/4110760541 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87680035d0 con 0x7f875c006190 2024-09-13T18:12:16.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.968+0000 7f877a59c700 1 --2- 172.21.15.40:0/4110760541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f875c003650 0x7f875c005aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:16.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.968+0000 7f8779d9b700 1 -- 172.21.15.40:0/4110760541 <== mon.1 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3997832404 0 0) 0x7f87680035d0 con 0x7f875c006190 2024-09-13T18:12:16.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.968+0000 7f877a59c700 1 -- 172.21.15.40:0/4110760541 >> v1:172.21.15.43:6789/0 conn(0x7f875c006190 legacy=0x7f875c008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:16.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.968+0000 7f877a59c700 1 -- 172.21.15.40:0/4110760541 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c00af80 con 0x7f875c003650 2024-09-13T18:12:16.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.968+0000 7f877a59c700 1 --2- 172.21.15.40:0/4110760541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f875c003650 0x7f875c005aa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f876c0046f0 tx=0x7f876c0039a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=986f2216ee94078d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:16.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.984+0000 7f8779d9b700 1 -- 172.21.15.40:0/4110760541 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87640018b0 con 0x7f875c003650 2024-09-13T18:12:16.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.984+0000 7f8779d9b700 1 -- 172.21.15.40:0/4110760541 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876c00c040 con 0x7f875c003650 2024-09-13T18:12:16.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.984+0000 7f8779d9b700 1 -- 172.21.15.40:0/4110760541 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c0186c0 con 0x7f875c003650 2024-09-13T18:12:16.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.984+0000 7f877b59e700 1 -- 172.21.15.40:0/4110760541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f875c003650 msgr2=0x7f875c005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:16.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.984+0000 7f877b59e700 1 --2- 172.21.15.40:0/4110760541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f875c003650 0x7f875c005aa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f876c0046f0 tx=0x7f876c0039a0 comp rx=0 tx=0).stop 2024-09-13T18:12:16.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.984+0000 7f877ad9d700 1 -- 172.21.15.40:0/4110760541 reap_dead start 2024-09-13T18:12:16.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.984+0000 7f877b59e700 1 -- 172.21.15.40:0/4110760541 shutdown_connections 2024-09-13T18:12:16.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.984+0000 7f877b59e700 1 -- 172.21.15.40:0/4110760541 >> 172.21.15.40:0/4110760541 conn(0x7f877c06a390 msgr2=0x7f877c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:16.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.984+0000 7f877b59e700 1 -- 172.21.15.40:0/4110760541 shutdown_connections 2024-09-13T18:12:16.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.984+0000 7f877b59e700 1 -- 172.21.15.40:0/4110760541 wait complete. 2024-09-13T18:12:16.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f877b59e700 1 Processor -- start 2024-09-13T18:12:16.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f877b59e700 1 -- start start 2024-09-13T18:12:16.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f877b59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 0x7f877c1541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:16.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f877b59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 0x7f877c154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:16.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f877b59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 0x7f877c148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:16.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f877b59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c1567e0 con 0x7f877c14ad70 2024-09-13T18:12:16.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f877b59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c14b1d0 con 0x7f877c1527b0 2024-09-13T18:12:16.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f877b59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f877c14b490 con 0x7f877c14e060 2024-09-13T18:12:16.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f877a59c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 0x7f877c1541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:16.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f877ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 0x7f877c148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:16.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f8773fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 0x7f877c154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:16.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f877a59c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 0x7f877c1541f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:36952/0 (socket says 172.21.15.40:36952) 2024-09-13T18:12:16.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f877a59c700 1 -- 172.21.15.40:0/8786471 learned_addr learned my addr 172.21.15.40:0/8786471 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:16.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f877a59c700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 msgr2=0x7f877c1541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:12:16.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f877a59c700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 msgr2=0x7f877c1541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:12:16.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f877a59c700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 0x7f877c1541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:16.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f8773fff700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 msgr2=0x7f877c154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:12:16.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f8773fff700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 msgr2=0x7f877c154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:12:16.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f877a59c700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 0x7f877c1541f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:12:16.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f8773fff700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 0x7f877c154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:16.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f877ad9d700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 msgr2=0x7f877c148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-13T18:12:16.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f877ad9d700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 msgr2=0x7f877c148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:12:16.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f877ad9d700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 0x7f877c148af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:17.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f8773fff700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 0x7f877c154730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:12:17.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:16.988+0000 7f877ad9d700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 0x7f877c148af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:12:17.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.192+0000 7f877ad9d700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 0x7f877c148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:17.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.192+0000 7f877a59c700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 0x7f877c1541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:17.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.192+0000 7f8773fff700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 0x7f877c154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:17.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.192+0000 7f877ad9d700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 msgr2=0x7f877c148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:12:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.192+0000 7f877ad9d700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 msgr2=0x7f877c148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:12:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.192+0000 7f877ad9d700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 0x7f877c148af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.192+0000 7f877a59c700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 msgr2=0x7f877c1541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-13T18:12:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.192+0000 7f877a59c700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 msgr2=0x7f877c1541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:12:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.192+0000 7f877a59c700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 0x7f877c1541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.192+0000 7f877ad9d700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 0x7f877c148af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:12:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.192+0000 7f8773fff700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 msgr2=0x7f877c154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:12:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.192+0000 7f8773fff700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 msgr2=0x7f877c154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:12:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.192+0000 7f877a59c700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 0x7f877c1541f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:12:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.192+0000 7f8773fff700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 0x7f877c154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:17.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.192+0000 7f8773fff700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 0x7f877c154730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:12:17.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.596+0000 7f877ad9d700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 0x7f877c148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:17.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.596+0000 7f877a59c700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 0x7f877c1541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:17.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.596+0000 7f877ad9d700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 msgr2=0x7f877c148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-13T18:12:17.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.596+0000 7f877ad9d700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 msgr2=0x7f877c148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:12:17.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.596+0000 7f877ad9d700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 0x7f877c148af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:17.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.596+0000 7f8773fff700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 0x7f877c154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:17.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.596+0000 7f877ad9d700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 0x7f877c148af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:12:17.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.596+0000 7f877a59c700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 msgr2=0x7f877c1541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:12:17.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.596+0000 7f877a59c700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 msgr2=0x7f877c1541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:12:17.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.596+0000 7f877a59c700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 0x7f877c1541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:17.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.596+0000 7f877a59c700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 0x7f877c1541f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:12:17.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.596+0000 7f8773fff700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 msgr2=0x7f877c154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:12:17.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.596+0000 7f8773fff700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 msgr2=0x7f877c154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:12:17.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.596+0000 7f8773fff700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 0x7f877c154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:17.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:17.596+0000 7f8773fff700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 0x7f877c154730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:12:18.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.400+0000 7f877ad9d700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 0x7f877c148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:18.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.400+0000 7f8773fff700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 0x7f877c154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:18.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.400+0000 7f877a59c700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 0x7f877c1541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:18.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.400+0000 7f877ad9d700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 msgr2=0x7f877c1541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:18.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.400+0000 7f877ad9d700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 0x7f877c1541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:18.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.400+0000 7f877ad9d700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 msgr2=0x7f877c154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:18.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.400+0000 7f877ad9d700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 0x7f877c154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:18.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.400+0000 7f877ad9d700 1 -- 172.21.15.40:0/8786471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876c00e040 con 0x7f877c14ad70 2024-09-13T18:12:18.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.400+0000 7f877a59c700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 0x7f877c1541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:12:18.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.400+0000 7f8773fff700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 0x7f877c154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:12:18.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.400+0000 7f877ad9d700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 0x7f877c148af0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f877c0c7c70 tx=0x7f8774017d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:18.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.400+0000 7f8772ffd700 1 -- 172.21.15.40:0/8786471 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8774017e70 con 0x7f877c14ad70 2024-09-13T18:12:18.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.400+0000 7f8772ffd700 1 -- 172.21.15.40:0/8786471 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877401c070 con 0x7f877c14ad70 2024-09-13T18:12:18.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.400+0000 7f877b59e700 1 -- 172.21.15.40:0/8786471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f877c14b770 con 0x7f877c14ad70 2024-09-13T18:12:18.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.400+0000 7f8772ffd700 1 -- 172.21.15.40:0/8786471 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8774003c80 con 0x7f877c14ad70 2024-09-13T18:12:18.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.400+0000 7f877b59e700 1 -- 172.21.15.40:0/8786471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f877c14bd30 con 0x7f877c14ad70 2024-09-13T18:12:18.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.404+0000 7f877b59e700 1 -- 172.21.15.40:0/8786471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f877c06ba20 con 0x7f877c14ad70 2024-09-13T18:12:18.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.404+0000 7f8772ffd700 1 -- 172.21.15.40:0/8786471 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8774015a00 con 0x7f877c14ad70 2024-09-13T18:12:18.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.404+0000 7f8772ffd700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f875c067570 0x7f875c069a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:18.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.404+0000 7f877a59c700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f875c067570 0x7f875c069a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:18.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.404+0000 7f8772ffd700 1 -- 172.21.15.40:0/8786471 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f877401e070 con 0x7f877c14ad70 2024-09-13T18:12:18.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.404+0000 7f877a59c700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f875c067570 0x7f875c069a30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f876c0039e0 tx=0x7f876c024040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:18.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.412+0000 7f8772ffd700 1 -- 172.21.15.40:0/8786471 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f87740524c0 con 0x7f877c14ad70 2024-09-13T18:12:18.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.700+0000 7f877b59e700 1 -- 172.21.15.40:0/8786471 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f877c1538c0 con 0x7f877c14ad70 2024-09-13T18:12:18.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.700+0000 7f8772ffd700 1 -- 172.21.15.40:0/8786471 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f8774055ae0 con 0x7f877c14ad70 2024-09-13T18:12:18.708 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":176,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:12:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.700+0000 7f877b59e700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f875c067570 msgr2=0x7f875c069a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.700+0000 7f877b59e700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f875c067570 0x7f875c069a30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f876c0039e0 tx=0x7f876c024040 comp rx=0 tx=0).stop 2024-09-13T18:12:18.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.700+0000 7f877b59e700 1 -- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 msgr2=0x7f877c148af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.700+0000 7f877b59e700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 0x7f877c148af0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f877c0c7c70 tx=0x7f8774017d40 comp rx=0 tx=0).stop 2024-09-13T18:12:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.700+0000 7f877b59e700 1 -- 172.21.15.40:0/8786471 shutdown_connections 2024-09-13T18:12:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.700+0000 7f877b59e700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f877c14ad70 0x7f877c148af0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.700+0000 7f877b59e700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f877c1527b0 0x7f877c154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.700+0000 7f877b59e700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f875c067570 0x7f875c069a30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.700+0000 7f877b59e700 1 --2- 172.21.15.40:0/8786471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f877c14e060 0x7f877c1541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.700+0000 7f877b59e700 1 -- 172.21.15.40:0/8786471 >> 172.21.15.40:0/8786471 conn(0x7f877c06a390 msgr2=0x7f877c06dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.700+0000 7f877b59e700 1 -- 172.21.15.40:0/8786471 shutdown_connections 2024-09-13T18:12:18.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.700+0000 7f877b59e700 1 -- 172.21.15.40:0/8786471 wait complete. 2024-09-13T18:12:18.788 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:12:18.789 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:12:18.789 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:12:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f5705b700 1 Processor -- start 2024-09-13T18:12:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f5705b700 1 -- start start 2024-09-13T18:12:18.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f5705b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f480d8b30 0x7f2f480df0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f5705b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2f480460f0 0x7f2f48046990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f5705b700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f480df620 con 0x7f2f48045cf0 2024-09-13T18:12:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f5705b700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f480087d0 con 0x7f2f480460f0 2024-09-13T18:12:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f5705b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f48007860 con 0x7f2f480d8b30 2024-09-13T18:12:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f55858700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2f48045cf0 0x7f2f4804b790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33704/0 (socket says 172.21.15.40:33704) 2024-09-13T18:12:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f55858700 1 -- 172.21.15.40:0/2414102196 learned_addr learned my addr 172.21.15.40:0/2414102196 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:12:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f5685a700 1 --2- 172.21.15.40:0/2414102196 >> v2:172.21.15.43:3300/0 conn(0x7f2f480460f0 0x7f2f48046990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f56059700 1 --2- 172.21.15.40:0/2414102196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f480d8b30 0x7f2f480df0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f55057700 1 -- 172.21.15.40:0/2414102196 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578591157 0 0) 0x7f2f40001110 con 0x7f2f48045cf0 2024-09-13T18:12:18.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f55057700 1 -- 172.21.15.40:0/2414102196 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f3c0035d0 con 0x7f2f48045cf0 2024-09-13T18:12:18.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f55057700 1 -- 172.21.15.40:0/2414102196 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2263237870 0 0) 0x7f2f3c0035d0 con 0x7f2f48045cf0 2024-09-13T18:12:18.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f55057700 1 -- 172.21.15.40:0/2414102196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f480d8b30 msgr2=0x7f2f480df0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:12:18.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f55057700 1 --2- 172.21.15.40:0/2414102196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f480d8b30 0x7f2f480df0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:18.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f55057700 1 -- 172.21.15.40:0/2414102196 >> v2:172.21.15.43:3300/0 conn(0x7f2f480460f0 msgr2=0x7f2f48046990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:12:18.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f55057700 1 --2- 172.21.15.40:0/2414102196 >> v2:172.21.15.43:3300/0 conn(0x7f2f480460f0 0x7f2f48046990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:18.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f55057700 1 -- 172.21.15.40:0/2414102196 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f480465a0 con 0x7f2f48045cf0 2024-09-13T18:12:18.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f55057700 1 -- 172.21.15.40:0/2414102196 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f2f40001800 con 0x7f2f48045cf0 2024-09-13T18:12:18.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f55057700 1 -- 172.21.15.40:0/2414102196 >> v1:172.21.15.40:6789/0 conn(0x7f2f48045cf0 legacy=0x7f2f4804b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:18.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f55057700 1 --2- 172.21.15.40:0/2414102196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f480d8b30 0x7f2f3c003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:18.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f55057700 1 --2- 172.21.15.40:0/2414102196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c003fa0 0x7f2f3c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:18.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f55057700 1 --2- 172.21.15.40:0/2414102196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f3c004db0 0x7f2f3c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:18.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f55057700 1 -- 172.21.15.40:0/2414102196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f48007860 con 0x7f2f3c004db0 2024-09-13T18:12:18.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f55057700 1 -- 172.21.15.40:0/2414102196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c00bac0 con 0x7f2f3c003fa0 2024-09-13T18:12:18.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f55057700 1 -- 172.21.15.40:0/2414102196 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c00bc80 con 0x7f2f480d8b30 2024-09-13T18:12:18.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f5685a700 1 --2- 172.21.15.40:0/2414102196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f3c004db0 0x7f2f3c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:18.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f56059700 1 --2- 172.21.15.40:0/2414102196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f480d8b30 0x7f2f3c003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:18.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f55858700 1 --2- 172.21.15.40:0/2414102196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c003fa0 0x7f2f3c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:18.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f5685a700 1 -- 172.21.15.40:0/2414102196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f480d8b30 msgr2=0x7f2f3c003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:12:18.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f5685a700 1 --2- 172.21.15.40:0/2414102196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f480d8b30 0x7f2f3c003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:18.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f5685a700 1 -- 172.21.15.40:0/2414102196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c003fa0 msgr2=0x7f2f3c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:12:18.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f5685a700 1 --2- 172.21.15.40:0/2414102196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c003fa0 0x7f2f3c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:18.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f5685a700 1 -- 172.21.15.40:0/2414102196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c0035d0 con 0x7f2f3c004db0 2024-09-13T18:12:18.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.952+0000 7f2f56059700 1 --2- 172.21.15.40:0/2414102196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f480d8b30 0x7f2f3c003a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:12:18.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f55858700 1 --2- 172.21.15.40:0/2414102196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f3c003fa0 0x7f2f3c004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:12:18.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f5685a700 1 --2- 172.21.15.40:0/2414102196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f3c004db0 0x7f2f3c005230 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2f5002ce60 tx=0x7f2f3c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db19c24c83e2c0d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:18.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f55057700 1 -- 172.21.15.40:0/2414102196 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f50090070 con 0x7f2f3c004db0 2024-09-13T18:12:18.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f55057700 1 -- 172.21.15.40:0/2414102196 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f50068860 con 0x7f2f3c004db0 2024-09-13T18:12:18.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f55057700 1 -- 172.21.15.40:0/2414102196 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f50067dd0 con 0x7f2f3c004db0 2024-09-13T18:12:18.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f5705b700 1 -- 172.21.15.40:0/2414102196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f3c004db0 msgr2=0x7f2f3c005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:18.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f5705b700 1 --2- 172.21.15.40:0/2414102196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f3c004db0 0x7f2f3c005230 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2f5002ce60 tx=0x7f2f3c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:12:18.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f5685a700 1 -- 172.21.15.40:0/2414102196 reap_dead start 2024-09-13T18:12:18.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f5705b700 1 -- 172.21.15.40:0/2414102196 shutdown_connections 2024-09-13T18:12:18.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f5705b700 1 -- 172.21.15.40:0/2414102196 >> 172.21.15.40:0/2414102196 conn(0x7f2f48050e10 msgr2=0x7f2f48051220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:18.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f5705b700 1 -- 172.21.15.40:0/2414102196 shutdown_connections 2024-09-13T18:12:18.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f5705b700 1 -- 172.21.15.40:0/2414102196 wait complete. 2024-09-13T18:12:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f5705b700 1 Processor -- start 2024-09-13T18:12:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f5705b700 1 -- start start 2024-09-13T18:12:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f5705b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f480dac00 0x7f2f4804ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f5705b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f480dbd60 0x7f2f480cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f5705b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f480da040 0x7f2f480dafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f5705b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480d2a40 con 0x7f2f480dac00 2024-09-13T18:12:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f5705b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480cddc0 con 0x7f2f480dbd60 2024-09-13T18:12:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f5705b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480d2270 con 0x7f2f480da040 2024-09-13T18:12:18.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f56059700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f480dac00 0x7f2f4804ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.956+0000 7f2f56059700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f480dac00 0x7f2f4804ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60156/0 (socket says 172.21.15.40:60156) 2024-09-13T18:12:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f56059700 1 -- 172.21.15.40:0/2906621896 learned_addr learned my addr 172.21.15.40:0/2906621896 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f55858700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f480dbd60 0x7f2f480cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f5685a700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f480da040 0x7f2f480dafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f56059700 1 -- 172.21.15.40:0/2906621896 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f480da040 msgr2=0x7f2f480dafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f56059700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f480da040 0x7f2f480dafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f56059700 1 -- 172.21.15.40:0/2906621896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f480dbd60 msgr2=0x7f2f480cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f56059700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f480dbd60 0x7f2f480cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f56059700 1 -- 172.21.15.40:0/2906621896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f480cefb0 con 0x7f2f480dac00 2024-09-13T18:12:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f5685a700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f480da040 0x7f2f480dafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:12:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f56059700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f480dac00 0x7f2f4804ee30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c011900 tx=0x7f2f4c011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f46ffd700 1 -- 172.21.15.40:0/2906621896 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c004220 con 0x7f2f480dac00 2024-09-13T18:12:18.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f46ffd700 1 -- 172.21.15.40:0/2906621896 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4c004380 con 0x7f2f480dac00 2024-09-13T18:12:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f5705b700 1 -- 172.21.15.40:0/2906621896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f480d2480 con 0x7f2f480dac00 2024-09-13T18:12:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f46ffd700 1 -- 172.21.15.40:0/2906621896 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c0036c0 con 0x7f2f480dac00 2024-09-13T18:12:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f5705b700 1 -- 172.21.15.40:0/2906621896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f480cd080 con 0x7f2f480dac00 2024-09-13T18:12:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f5705b700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f480cd2a0 0x7f2f480cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f5705b700 1 -- 172.21.15.40:0/2906621896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480095e0 con 0x7f2f480cd2a0 2024-09-13T18:12:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f55858700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f480cd2a0 0x7f2f480cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f55858700 1 -- 172.21.15.40:0/2906621896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f480df830 con 0x7f2f480cd2a0 2024-09-13T18:12:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f46ffd700 1 -- 172.21.15.40:0/2906621896 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2f4c003820 con 0x7f2f480dac00 2024-09-13T18:12:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f55858700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f480cd2a0 0x7f2f480cb650 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2f40000c00 tx=0x7f2f40010b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.960+0000 7f2f46ffd700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f34066730 0x7f2f34068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.964+0000 7f2f5685a700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f34066730 0x7f2f34068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:18.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.964+0000 7f2f46ffd700 1 -- 172.21.15.40:0/2906621896 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2f4c0844e0 con 0x7f2f480dac00 2024-09-13T18:12:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.964+0000 7f2f46ffd700 1 -- 172.21.15.40:0/2906621896 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40012790 con 0x7f2f480cd2a0 2024-09-13T18:12:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.964+0000 7f2f46ffd700 1 -- 172.21.15.40:0/2906621896 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f2f400128f0 con 0x7f2f480cd2a0 2024-09-13T18:12:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.964+0000 7f2f46ffd700 1 -- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f480cd2a0 msgr2=0x7f2f480cb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.964+0000 7f2f46ffd700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f480cd2a0 0x7f2f480cb650 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2f40000c00 tx=0x7f2f40010b10 comp rx=0 tx=0).stop 2024-09-13T18:12:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.964+0000 7f2f46ffd700 1 -- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f480cd2a0 msgr2=0x7f2f480cb650 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:12:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.964+0000 7f2f46ffd700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f480cd2a0 0x7f2f480cb650 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:18.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.964+0000 7f2f5685a700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f34066730 0x7f2f34068bf0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0047f0 tx=0x7f2f50097040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:18.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f5705b700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f480da420 0x7f2f480cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:18.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f5705b700 1 -- 172.21.15.40:0/2906621896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f48007210 con 0x7f2f480da420 2024-09-13T18:12:18.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f55858700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f480da420 0x7f2f480cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:18.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f55858700 1 -- 172.21.15.40:0/2906621896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2f48006f30 con 0x7f2f480da420 2024-09-13T18:12:18.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f55858700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f480da420 0x7f2f480cbb90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2f40000fb0 tx=0x7f2f48047900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:18.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f46ffd700 1 -- 172.21.15.40:0/2906621896 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f400146c0 con 0x7f2f480da420 2024-09-13T18:12:18.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f46ffd700 1 -- 172.21.15.40:0/2906621896 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4452 (secure 0 0 0) 0x7f2f40014820 con 0x7f2f480da420 2024-09-13T18:12:18.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f46ffd700 1 -- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f480da420 msgr2=0x7f2f480cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:18.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f46ffd700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f480da420 0x7f2f480cbb90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2f40000fb0 tx=0x7f2f48047900 comp rx=0 tx=0).stop 2024-09-13T18:12:18.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f46ffd700 1 -- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f480da420 msgr2=0x7f2f480cbb90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:12:18.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f46ffd700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f480da420 0x7f2f480cbb90 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:18.981 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:12:18.981 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:12:18.982 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:12:18.982 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:12:18.982 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 176, 2024-09-13T18:12:18.982 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:12:18.982 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:12:18.982 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:12:18.982 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:12:18.982 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:18.982 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:12:18.982 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:12:18.982 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:12:18.982 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:12:18.982 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:18.982 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:18.983 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:18.983 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:18.983 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:18.983 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:18.983 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:18.983 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:18.983 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:18.983 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:18.983 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:18.983 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:12:18.983 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:12:18.983 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:18.983 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:18.984 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:18.984 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:18.984 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:18.984 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:18.984 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:18.984 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:18.984 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:18.984 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:18.984 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:18.984 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:18.984 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:12:18.984 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:12:18.984 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:12:18.985 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:12:18.985 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:12:18.985 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:12:18.985 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:12:18.985 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:12:18.985 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:12:18.985 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:12:18.985 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:12:18.985 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:12:18.985 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:12:18.985 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:12:18.985 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:12:18.985 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:12:18.986 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:12:18.986 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:18.986 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:18.986 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:18.986 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:18.986 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:18.986 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:18.986 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:18.986 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:18.986 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:18.986 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:18.986 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:18.986 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:12:18.986 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:18.987 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:12:18.987 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:18.987 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:12:18.987 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:12:18.987 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:18.987 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:18.987 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:18.987 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:18.987 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:12:18.987 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:18.987 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:18.987 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:18.987 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:18.987 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:12:18.988 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:18.988 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:18.988 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:18.988 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:18.988 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:12:18.988 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:12:18.988 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:18.988 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:18.988 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:18.988 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:18.988 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:18.988 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:12:18.988 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:12:18.989 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:18.989 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:18.989 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:18.989 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:18.989 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:12:18.989 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:18.989 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:18.989 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:18.989 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:18.989 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:12:18.989 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:18.989 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:18.989 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:18.990 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:18.990 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:12:18.990 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:12:18.990 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:18.990 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:18.990 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:18.990 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:18.990 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:18.990 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:12:18.990 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:12:18.990 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:18.990 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:18.990 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:18.990 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:18.991 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:12:18.991 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:18.991 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:18.991 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:18.991 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:18.991 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:12:18.991 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:18.991 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:18.991 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:18.991 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:18.991 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:12:18.991 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:12:18.991 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:18.991 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:18.992 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:18.992 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:18.992 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:18.992 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:18.992 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:12:18.992 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:12:18.992 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:18.992 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:18.992 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:18.992 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:12:18.992 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:18.992 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:18.992 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:12:18.992 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:18.993 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:18.993 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:18.993 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:12:18.993 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:18.993 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:18.993 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:18.993 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:12:18.993 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:12:18.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f5705b700 1 -- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f34066730 msgr2=0x7f2f34068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:18.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f5705b700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f34066730 0x7f2f34068bf0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0047f0 tx=0x7f2f50097040 comp rx=0 tx=0).stop 2024-09-13T18:12:18.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f5705b700 1 -- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f480dac00 msgr2=0x7f2f4804ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:18.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f5705b700 1 --2- 172.21.15.40:0/2906621896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f480dac00 0x7f2f4804ee30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c011900 tx=0x7f2f4c011cc0 comp rx=0 tx=0).stop 2024-09-13T18:12:18.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f5685a700 1 -- 172.21.15.40:0/2906621896 reap_dead start 2024-09-13T18:12:18.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f5685a700 1 -- 172.21.15.40:0/2906621896 reap_dead start 2024-09-13T18:12:18.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f5705b700 1 -- 172.21.15.40:0/2906621896 shutdown_connections 2024-09-13T18:12:18.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f5705b700 1 -- 172.21.15.40:0/2906621896 >> 172.21.15.40:0/2906621896 conn(0x7f2f48050e10 msgr2=0x7f2f4804cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:18.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f5705b700 1 -- 172.21.15.40:0/2906621896 shutdown_connections 2024-09-13T18:12:18.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:18.972+0000 7f2f5705b700 1 -- 172.21.15.40:0/2906621896 wait complete. 2024-09-13T18:12:18.996 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:12:18.996 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:12:18.996 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:12:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.108+0000 7fac30bcf700 1 Processor -- start 2024-09-13T18:12:19.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac30bcf700 1 -- start start 2024-09-13T18:12:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac30bcf700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fac2c0c7540 0x7fac2c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac30bcf700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fac2c0c30f0 0x7fac2c06da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac30bcf700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac2c0c7a30 con 0x7fac2c1515a0 2024-09-13T18:12:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac30bcf700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2c151980 con 0x7fac2c0c7540 2024-09-13T18:12:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac30bcf700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2c157430 con 0x7fac2c0c30f0 2024-09-13T18:12:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac2b7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fac2c1515a0 0x7fac2c0c29e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33712/0 (socket says 172.21.15.40:33712) 2024-09-13T18:12:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac2b7fe700 1 -- 172.21.15.40:0/12587325 learned_addr learned my addr 172.21.15.40:0/12587325 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:12:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac2affd700 1 --2- 172.21.15.40:0/12587325 >> v2:172.21.15.40:3300/0 conn(0x7fac2c0c7540 0x7fac2c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac2bfff700 1 --2- 172.21.15.40:0/12587325 >> v2:172.21.15.43:3300/0 conn(0x7fac2c0c30f0 0x7fac2c06da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac2a7fc700 1 -- 172.21.15.40:0/12587325 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1883592005 0 0) 0x7fac2c0c7a30 con 0x7fac2c1515a0 2024-09-13T18:12:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac2affd700 1 -- 172.21.15.40:0/12587325 >> v2:172.21.15.43:3300/0 conn(0x7fac2c0c30f0 msgr2=0x7fac2c06da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:12:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac2affd700 1 --2- 172.21.15.40:0/12587325 >> v2:172.21.15.43:3300/0 conn(0x7fac2c0c30f0 0x7fac2c06da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac2affd700 1 -- 172.21.15.40:0/12587325 >> v1:172.21.15.40:6789/0 conn(0x7fac2c1515a0 legacy=0x7fac2c0c29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac2affd700 1 -- 172.21.15.40:0/12587325 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2c157710 con 0x7fac2c0c7540 2024-09-13T18:12:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac2bfff700 1 --2- 172.21.15.40:0/12587325 >> v2:172.21.15.43:3300/0 conn(0x7fac2c0c30f0 0x7fac2c06da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:12:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac2affd700 1 --2- 172.21.15.40:0/12587325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac2c0c7540 0x7fac2c06d540 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fac2c067a40 tx=0x7fac20009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfb9434c7910b43 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac2a7fc700 1 -- 172.21.15.40:0/12587325 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac20011830 con 0x7fac2c0c7540 2024-09-13T18:12:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac2a7fc700 1 -- 172.21.15.40:0/12587325 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac20011990 con 0x7fac2c0c7540 2024-09-13T18:12:19.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac2a7fc700 1 -- 172.21.15.40:0/12587325 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac20011d40 con 0x7fac2c0c7540 2024-09-13T18:12:19.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac30bcf700 1 -- 172.21.15.40:0/12587325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac2c0c7540 msgr2=0x7fac2c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac30bcf700 1 --2- 172.21.15.40:0/12587325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac2c0c7540 0x7fac2c06d540 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fac2c067a40 tx=0x7fac20009b30 comp rx=0 tx=0).stop 2024-09-13T18:12:19.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac30bcf700 1 -- 172.21.15.40:0/12587325 shutdown_connections 2024-09-13T18:12:19.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac30bcf700 1 --2- 172.21.15.40:0/12587325 >> v2:172.21.15.43:3300/0 conn(0x7fac2c0c30f0 0x7fac2c06da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac30bcf700 1 --2- 172.21.15.40:0/12587325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac2c0c7540 0x7fac2c06d540 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac30bcf700 1 -- 172.21.15.40:0/12587325 >> 172.21.15.40:0/12587325 conn(0x7fac2c0cb450 msgr2=0x7fac2c0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:19.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac30bcf700 1 -- 172.21.15.40:0/12587325 shutdown_connections 2024-09-13T18:12:19.123 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.112+0000 7fac30bcf700 1 -- 172.21.15.40:0/12587325 wait complete. 2024-09-13T18:12:19.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac30bcf700 1 Processor -- start 2024-09-13T18:12:19.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac30bcf700 1 -- start start 2024-09-13T18:12:19.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac30bcf700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac2c0c30f0 0x7fac2c156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac30bcf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac2c0c7540 0x7fac2c1e1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac30bcf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac2c1515a0 0x7fac2c1e20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac30bcf700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c156710 con 0x7fac2c1515a0 2024-09-13T18:12:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac30bcf700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c1563c0 con 0x7fac2c0c7540 2024-09-13T18:12:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac30bcf700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c071440 con 0x7fac2c0c30f0 2024-09-13T18:12:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac2bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac2c1515a0 0x7fac2c1e20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac2bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac2c1515a0 0x7fac2c1e20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60186/0 (socket says 172.21.15.40:60186) 2024-09-13T18:12:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac2bfff700 1 -- 172.21.15.40:0/1509300789 learned_addr learned my addr 172.21.15.40:0/1509300789 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac2affd700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac2c0c7540 0x7fac2c1e1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac2b7fe700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac2c0c30f0 0x7fac2c156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac2bfff700 1 -- 172.21.15.40:0/1509300789 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac2c0c30f0 msgr2=0x7fac2c156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac2bfff700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fac2c0c30f0 0x7fac2c156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac2bfff700 1 -- 172.21.15.40:0/1509300789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac2c0c7540 msgr2=0x7fac2c1e1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac2bfff700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac2c0c7540 0x7fac2c1e1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac2bfff700 1 -- 172.21.15.40:0/1509300789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac200094b0 con 0x7fac2c1515a0 2024-09-13T18:12:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac2bfff700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac2c1515a0 0x7fac2c1e20f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fac24007e20 tx=0x7fac2400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac13fff700 1 -- 172.21.15.40:0/1509300789 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2400de90 con 0x7fac2c1515a0 2024-09-13T18:12:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac13fff700 1 -- 172.21.15.40:0/1509300789 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2400f040 con 0x7fac2c1515a0 2024-09-13T18:12:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac30bcf700 1 -- 172.21.15.40:0/1509300789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2c0716e0 con 0x7fac2c1515a0 2024-09-13T18:12:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac13fff700 1 -- 172.21.15.40:0/1509300789 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac24003c80 con 0x7fac2c1515a0 2024-09-13T18:12:19.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.116+0000 7fac30bcf700 1 -- 172.21.15.40:0/1509300789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2c071f30 con 0x7fac2c1515a0 2024-09-13T18:12:19.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.120+0000 7fac30bcf700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac0c002a40 0x7fac0c004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.120+0000 7fac30bcf700 1 -- 172.21.15.40:0/1509300789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c007f40 con 0x7fac0c002a40 2024-09-13T18:12:19.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.120+0000 7fac13fff700 1 -- 172.21.15.40:0/1509300789 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fac24004030 con 0x7fac2c1515a0 2024-09-13T18:12:19.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.120+0000 7fac13fff700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fac18066760 0x7fac18068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.128 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.120+0000 7fac2b7fe700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac0c002a40 0x7fac0c004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.120+0000 7fac13fff700 1 -- 172.21.15.40:0/1509300789 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fac24084690 con 0x7fac2c1515a0 2024-09-13T18:12:19.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.120+0000 7fac2affd700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fac18066760 0x7fac18068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.120+0000 7fac2b7fe700 1 -- 172.21.15.40:0/1509300789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fac0c008100 con 0x7fac0c002a40 2024-09-13T18:12:19.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.120+0000 7fac2affd700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fac18066760 0x7fac18068c20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fac200090b0 tx=0x7fac2001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:19.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.120+0000 7fac2b7fe700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac0c002a40 0x7fac0c004f00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fac1c000f60 tx=0x7fac1c001710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:19.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.120+0000 7fac13fff700 1 -- 172.21.15.40:0/1509300789 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0114f0 con 0x7fac0c002a40 2024-09-13T18:12:19.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.124+0000 7fac13fff700 1 -- 172.21.15.40:0/1509300789 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fac1c011650 con 0x7fac0c002a40 2024-09-13T18:12:19.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.124+0000 7fac13fff700 1 -- 172.21.15.40:0/1509300789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac0c002a40 msgr2=0x7fac0c004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.124+0000 7fac13fff700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac0c002a40 0x7fac0c004f00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fac1c000f60 tx=0x7fac1c001710 comp rx=0 tx=0).stop 2024-09-13T18:12:19.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.124+0000 7fac13fff700 1 -- 172.21.15.40:0/1509300789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac0c002a40 msgr2=0x7fac0c004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:12:19.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.124+0000 7fac13fff700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac0c002a40 0x7fac0c004f00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.132+0000 7fac30bcf700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac0c007020 0x7fac0c007490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.132+0000 7fac30bcf700 1 -- 172.21.15.40:0/1509300789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c00a400 con 0x7fac0c007020 2024-09-13T18:12:19.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.132+0000 7fac2b7fe700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac0c007020 0x7fac0c007490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.132+0000 7fac2b7fe700 1 -- 172.21.15.40:0/1509300789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fac0c00a5c0 con 0x7fac0c007020 2024-09-13T18:12:19.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.132+0000 7fac2b7fe700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac0c007020 0x7fac0c007490 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fac1c016000 tx=0x7fac1c001080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:19.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.132+0000 7fac13fff700 1 -- 172.21.15.40:0/1509300789 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c021070 con 0x7fac0c007020 2024-09-13T18:12:19.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.132+0000 7fac13fff700 1 -- 172.21.15.40:0/1509300789 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4782 (secure 0 0 0) 0x7fac1c01c9c0 con 0x7fac0c007020 2024-09-13T18:12:19.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.132+0000 7fac13fff700 1 -- 172.21.15.40:0/1509300789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac0c007020 msgr2=0x7fac0c007490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.132+0000 7fac13fff700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac0c007020 0x7fac0c007490 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fac1c016000 tx=0x7fac1c001080 comp rx=0 tx=0).stop 2024-09-13T18:12:19.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.132+0000 7fac13fff700 1 -- 172.21.15.40:0/1509300789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac0c007020 msgr2=0x7fac0c007490 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:12:19.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.132+0000 7fac13fff700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac0c007020 0x7fac0c007490 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.143 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:12:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:12:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:12:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:12:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 176, 2024-09-13T18:12:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:12:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:12:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:12:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:12:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:12:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:12:19.143 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:12:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:12:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:12:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:12:19.144 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:19.145 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:19.145 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:19.145 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:19.145 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:19.145 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:19.145 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:19.145 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:19.145 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:19.145 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:19.145 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:19.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.145 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:12:19.145 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:12:19.145 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:12:19.145 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:12:19.146 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:12:19.146 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:12:19.146 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:12:19.146 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:12:19.146 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:12:19.146 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:12:19.146 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:12:19.146 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:12:19.146 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:12:19.146 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:12:19.146 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:12:19.146 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:12:19.146 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:12:19.147 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:19.147 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:19.147 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:19.147 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:19.147 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:19.147 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:19.147 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:19.147 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:19.147 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:19.147 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:19.147 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:19.147 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:12:19.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.147 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:12:19.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.148 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:12:19.148 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:12:19.148 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:19.148 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:19.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.148 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:19.148 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:12:19.148 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:19.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.148 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:19.148 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:12:19.149 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:19.149 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:19.149 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:19.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.149 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:12:19.149 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:12:19.149 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:19.149 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:19.149 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:19.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.149 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.149 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:12:19.149 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:12:19.150 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:19.150 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:19.150 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.150 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:19.150 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:12:19.150 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:19.150 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.150 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.150 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:19.150 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:12:19.150 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:19.150 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:19.150 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:19.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.151 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:12:19.151 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:12:19.151 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:19.151 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:19.151 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:19.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.151 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.151 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:12:19.151 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:12:19.151 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:19.151 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:19.151 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.151 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:19.152 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:12:19.152 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:19.152 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.152 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.152 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:19.152 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:12:19.152 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:19.152 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:19.152 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:19.152 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.152 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:12:19.152 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:12:19.152 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:19.152 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:19.152 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:19.153 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:19.153 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:19.153 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.153 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:12:19.153 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:12:19.153 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.153 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:19.153 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:19.153 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:12:19.153 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:19.153 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:19.153 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:12:19.153 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.154 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:19.154 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:19.154 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:12:19.154 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:19.154 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:19.154 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:12:19.154 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.154 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:19.154 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:19.154 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:12:19.154 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:19.154 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:19.154 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:12:19.154 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.155 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:19.155 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:19.155 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:12:19.155 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:19.155 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:19.155 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.155 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:12:19.155 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:12:19.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.136+0000 7fac30bcf700 1 -- 172.21.15.40:0/1509300789 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fac18066760 msgr2=0x7fac18068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.136+0000 7fac30bcf700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fac18066760 0x7fac18068c20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fac200090b0 tx=0x7fac2001c040 comp rx=0 tx=0).stop 2024-09-13T18:12:19.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.136+0000 7fac30bcf700 1 -- 172.21.15.40:0/1509300789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac2c1515a0 msgr2=0x7fac2c1e20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.136+0000 7fac2bfff700 1 -- 172.21.15.40:0/1509300789 reap_dead start 2024-09-13T18:12:19.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.136+0000 7fac30bcf700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac2c1515a0 0x7fac2c1e20f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fac24007e20 tx=0x7fac2400d3b0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.136+0000 7fac30bcf700 1 -- 172.21.15.40:0/1509300789 shutdown_connections 2024-09-13T18:12:19.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.136+0000 7fac30bcf700 1 --2- 172.21.15.40:0/1509300789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac2c1515a0 0x7fac2c1e20f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.136+0000 7fac30bcf700 1 -- 172.21.15.40:0/1509300789 >> 172.21.15.40:0/1509300789 conn(0x7fac2c0cb450 msgr2=0x7fac2c06d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:19.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.136+0000 7fac30bcf700 1 -- 172.21.15.40:0/1509300789 shutdown_connections 2024-09-13T18:12:19.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.136+0000 7fac30bcf700 1 -- 172.21.15.40:0/1509300789 wait complete. 2024-09-13T18:12:19.160 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:12:19.160 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:12:19.161 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:12:19.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.272+0000 7f6de359e700 1 Processor -- start 2024-09-13T18:12:19.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.272+0000 7f6de359e700 1 -- start start 2024-09-13T18:12:19.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.272+0000 7f6de359e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6de4150520 0x7f6de406e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de359e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de4066460 con 0x7f6de4150900 2024-09-13T18:12:19.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de359e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de4066130 con 0x7f6de4071180 2024-09-13T18:12:19.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de359e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de40c3830 con 0x7f6de4150520 2024-09-13T18:12:19.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de259c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6de4150520 0x7f6de406e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de1d9b700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6de4150900 0x7f6de40c4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33720/0 (socket says 172.21.15.40:33720) 2024-09-13T18:12:19.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de1d9b700 1 -- 172.21.15.40:0/1656160880 learned_addr learned my addr 172.21.15.40:0/1656160880 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:12:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de259c700 1 -- 172.21.15.40:0/1656160880 >> v1:172.21.15.43:6789/0 conn(0x7f6de4071180 legacy=0x7f6de4156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de259c700 1 -- 172.21.15.40:0/1656160880 >> v1:172.21.15.40:6789/0 conn(0x7f6de4150900 legacy=0x7f6de40c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de259c700 1 -- 172.21.15.40:0/1656160880 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de4070e80 con 0x7f6de4150520 2024-09-13T18:12:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de159a700 1 -- 172.21.15.40:0/1656160880 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257819262 0 0) 0x7f6de4066460 con 0x7f6de4150900 2024-09-13T18:12:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de259c700 1 --2- 172.21.15.40:0/1656160880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6de4150520 0x7f6de406e860 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009850 tx=0x7f6dd4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f512448f30836a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de159a700 1 -- 172.21.15.40:0/1656160880 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4011830 con 0x7f6de4150520 2024-09-13T18:12:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de159a700 1 -- 172.21.15.40:0/1656160880 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd4011990 con 0x7f6de4150520 2024-09-13T18:12:19.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de159a700 1 -- 172.21.15.40:0/1656160880 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4011d20 con 0x7f6de4150520 2024-09-13T18:12:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de359e700 1 -- 172.21.15.40:0/1656160880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6de4150520 msgr2=0x7f6de406e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de359e700 1 --2- 172.21.15.40:0/1656160880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6de4150520 0x7f6de406e860 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009850 tx=0x7f6dd4009b60 comp rx=0 tx=0).stop 2024-09-13T18:12:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de359e700 1 -- 172.21.15.40:0/1656160880 shutdown_connections 2024-09-13T18:12:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de359e700 1 --2- 172.21.15.40:0/1656160880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6de4150520 0x7f6de406e860 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de359e700 1 -- 172.21.15.40:0/1656160880 >> 172.21.15.40:0/1656160880 conn(0x7f6de40cb450 msgr2=0x7f6de40cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de359e700 1 -- 172.21.15.40:0/1656160880 shutdown_connections 2024-09-13T18:12:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.276+0000 7f6de359e700 1 -- 172.21.15.40:0/1656160880 wait complete. 2024-09-13T18:12:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de359e700 1 Processor -- start 2024-09-13T18:12:19.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de359e700 1 -- start start 2024-09-13T18:12:19.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de359e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6de4071180 0x7f6de41f0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de359e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de4150520 0x7f6de4071900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6de4153b40 0x7f6de413bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de359e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de41efeb0 con 0x7f6de4153b40 2024-09-13T18:12:19.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de359e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de41537c0 con 0x7f6de4150520 2024-09-13T18:12:19.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de359e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de41f0dd0 con 0x7f6de4071180 2024-09-13T18:12:19.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de2d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6de4153b40 0x7f6de413bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de2d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6de4153b40 0x7f6de413bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60206/0 (socket says 172.21.15.40:60206) 2024-09-13T18:12:19.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de2d9d700 1 -- 172.21.15.40:0/59165793 learned_addr learned my addr 172.21.15.40:0/59165793 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:19.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de1d9b700 1 --2- 172.21.15.40:0/59165793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de4150520 0x7f6de4071900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de259c700 1 --2- 172.21.15.40:0/59165793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6de4071180 0x7f6de41f0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de2d9d700 1 -- 172.21.15.40:0/59165793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6de4071180 msgr2=0x7f6de41f0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de2d9d700 1 --2- 172.21.15.40:0/59165793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6de4071180 0x7f6de41f0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de2d9d700 1 -- 172.21.15.40:0/59165793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de4150520 msgr2=0x7f6de4071900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de2d9d700 1 --2- 172.21.15.40:0/59165793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6de4150520 0x7f6de4071900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de2d9d700 1 -- 172.21.15.40:0/59165793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd40094b0 con 0x7f6de4153b40 2024-09-13T18:12:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de2d9d700 1 --2- 172.21.15.40:0/59165793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6de4153b40 0x7f6de413bb70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00cc80 tx=0x7f6ddc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:19.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6dd2ffd700 1 -- 172.21.15.40:0/59165793 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc003c20 con 0x7f6de4153b40 2024-09-13T18:12:19.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6dd2ffd700 1 -- 172.21.15.40:0/59165793 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc003d80 con 0x7f6de4153b40 2024-09-13T18:12:19.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de359e700 1 -- 172.21.15.40:0/59165793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de41f0400 con 0x7f6de4153b40 2024-09-13T18:12:19.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6dd2ffd700 1 -- 172.21.15.40:0/59165793 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc00aaa0 con 0x7f6de4153b40 2024-09-13T18:12:19.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.280+0000 7f6de359e700 1 -- 172.21.15.40:0/59165793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de414f8a0 con 0x7f6de4153b40 2024-09-13T18:12:19.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.284+0000 7f6de359e700 1 --2- 172.21.15.40:0/59165793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6de414fb80 0x7f6de413c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.284+0000 7f6de359e700 1 -- 172.21.15.40:0/59165793 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de40675d0 con 0x7f6de414fb80 2024-09-13T18:12:19.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.284+0000 7f6de259c700 1 --2- 172.21.15.40:0/59165793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6de414fb80 0x7f6de413c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.284+0000 7f6de259c700 1 -- 172.21.15.40:0/59165793 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6de40c2ae0 con 0x7f6de414fb80 2024-09-13T18:12:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.284+0000 7f6de259c700 1 --2- 172.21.15.40:0/59165793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6de414fb80 0x7f6de413c0b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009850 tx=0x7f6dd4004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.284+0000 7f6dd2ffd700 1 -- 172.21.15.40:0/59165793 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4020070 con 0x7f6de414fb80 2024-09-13T18:12:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.284+0000 7f6dd2ffd700 1 -- 172.21.15.40:0/59165793 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6ddc00ac00 con 0x7f6de4153b40 2024-09-13T18:12:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.284+0000 7f6dd2ffd700 1 --2- 172.21.15.40:0/59165793 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6dc40667d0 0x7f6dc4068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.284+0000 7f6dd2ffd700 1 -- 172.21.15.40:0/59165793 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6ddc084620 con 0x7f6de4153b40 2024-09-13T18:12:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.284+0000 7f6de1d9b700 1 --2- 172.21.15.40:0/59165793 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6dc40667d0 0x7f6dc4068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.284+0000 7f6dd2ffd700 1 -- 172.21.15.40:0/59165793 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6dd401b9d0 con 0x7f6de414fb80 2024-09-13T18:12:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.284+0000 7f6dd2ffd700 1 -- 172.21.15.40:0/59165793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6de414fb80 msgr2=0x7f6de413c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.284+0000 7f6dd2ffd700 1 --2- 172.21.15.40:0/59165793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6de414fb80 0x7f6de413c0b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009850 tx=0x7f6dd4004770 comp rx=0 tx=0).stop 2024-09-13T18:12:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.284+0000 7f6dd2ffd700 1 -- 172.21.15.40:0/59165793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6de414fb80 msgr2=0x7f6de413c0b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:12:19.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.284+0000 7f6dd2ffd700 1 --2- 172.21.15.40:0/59165793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6de414fb80 0x7f6de413c0b0 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009850 tx=0x7f6dd4004770 comp rx=0 tx=0).stop 2024-09-13T18:12:19.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.288+0000 7f6de1d9b700 1 --2- 172.21.15.40:0/59165793 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6dc40667d0 0x7f6dc4068c90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6de4072390 tx=0x7f6dd8004a70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.296+0000 7f6de359e700 1 --2- 172.21.15.40:0/59165793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6de4150900 0x7f6de41f91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.296+0000 7f6de359e700 1 -- 172.21.15.40:0/59165793 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de4070490 con 0x7f6de4150900 2024-09-13T18:12:19.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.296+0000 7f6de259c700 1 --2- 172.21.15.40:0/59165793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6de4150900 0x7f6de41f91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.296+0000 7f6de259c700 1 -- 172.21.15.40:0/59165793 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6de40c56f0 con 0x7f6de4150900 2024-09-13T18:12:19.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.296+0000 7f6de259c700 1 --2- 172.21.15.40:0/59165793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6de4150900 0x7f6de41f91f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009fd0 tx=0x7f6dd4011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:19.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.296+0000 7f6dd2ffd700 1 -- 172.21.15.40:0/59165793 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4024700 con 0x7f6de4150900 2024-09-13T18:12:19.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.296+0000 7f6dd2ffd700 1 -- 172.21.15.40:0/59165793 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7f6dd4024860 con 0x7f6de4150900 2024-09-13T18:12:19.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.296+0000 7f6dd2ffd700 1 -- 172.21.15.40:0/59165793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6de4150900 msgr2=0x7f6de41f91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.296+0000 7f6dd2ffd700 1 --2- 172.21.15.40:0/59165793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6de4150900 0x7f6de41f91f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009fd0 tx=0x7f6dd4011000 comp rx=0 tx=0).stop 2024-09-13T18:12:19.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.296+0000 7f6dd2ffd700 1 -- 172.21.15.40:0/59165793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6de4150900 msgr2=0x7f6de41f91f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:12:19.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.296+0000 7f6dd2ffd700 1 --2- 172.21.15.40:0/59165793 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6de4150900 0x7f6de41f91f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.306 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:12:19.306 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:12:19.306 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:12:19.306 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:12:19.307 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 176, 2024-09-13T18:12:19.307 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:12:19.307 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:12:19.307 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:12:19.307 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:12:19.307 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:19.307 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:12:19.307 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:12:19.307 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:12:19.307 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:12:19.307 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:19.307 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:19.308 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:19.308 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:19.308 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:19.308 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:19.308 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:19.308 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:19.308 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:19.308 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:19.308 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:19.308 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:12:19.309 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:12:19.309 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:19.309 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:19.309 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:19.309 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:19.309 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:19.309 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:19.309 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:19.309 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:19.309 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:19.309 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:19.310 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:19.310 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.310 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:12:19.310 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:12:19.310 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:12:19.310 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:12:19.310 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:12:19.310 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:12:19.310 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:12:19.310 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:12:19.310 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:12:19.311 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:12:19.311 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:12:19.311 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:12:19.311 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:12:19.311 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:12:19.311 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:12:19.311 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:12:19.311 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:12:19.311 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:19.311 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:19.311 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:19.311 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:19.312 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:19.312 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:19.312 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:19.312 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:19.312 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:19.312 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:19.312 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:19.312 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:12:19.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.312 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:12:19.313 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.313 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:12:19.313 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:12:19.313 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:19.313 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:19.313 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.313 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:19.313 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:12:19.313 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:19.313 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.313 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.313 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:19.314 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:12:19.314 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:19.314 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:19.314 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:19.314 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.314 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:12:19.314 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:12:19.314 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:19.314 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:19.314 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:19.314 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.314 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.315 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:12:19.315 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:12:19.315 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:19.315 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:19.315 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.315 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:19.315 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:12:19.315 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:19.315 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.315 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.315 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:19.316 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:12:19.316 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:19.316 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:19.316 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:19.316 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.316 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:12:19.316 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:12:19.316 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:19.316 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:19.316 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:19.316 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.316 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.317 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:12:19.317 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:12:19.317 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:19.317 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:19.317 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.317 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:19.317 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:12:19.317 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:19.317 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.317 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.317 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:19.318 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:12:19.318 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:19.318 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:19.318 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:19.318 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.318 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:12:19.318 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:12:19.318 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:19.318 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:19.318 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:19.318 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:19.319 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:19.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.319 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:12:19.319 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:12:19.319 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.319 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:19.319 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:19.319 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:12:19.319 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:19.319 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:19.319 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:12:19.320 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.320 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:19.320 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:19.320 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:12:19.320 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:19.320 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:19.320 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:12:19.320 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.320 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:19.320 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:19.320 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:12:19.320 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:19.321 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:19.321 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:12:19.321 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.321 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:19.321 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:19.321 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:12:19.321 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:19.321 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:19.321 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:12:19.321 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:19.321 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:19.322 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:19.322 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:12:19.322 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:19.322 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:19.322 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:19.322 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:12:19.322 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:12:19.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.296+0000 7f6de359e700 1 -- 172.21.15.40:0/59165793 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6dc40667d0 msgr2=0x7f6dc4068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.296+0000 7f6de359e700 1 --2- 172.21.15.40:0/59165793 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6dc40667d0 0x7f6dc4068c90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6de4072390 tx=0x7f6dd8004a70 comp rx=0 tx=0).stop 2024-09-13T18:12:19.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.296+0000 7f6de359e700 1 -- 172.21.15.40:0/59165793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6de4153b40 msgr2=0x7f6de413bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.296+0000 7f6de359e700 1 --2- 172.21.15.40:0/59165793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6de4153b40 0x7f6de413bb70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00cc80 tx=0x7f6ddc00cf90 comp rx=0 tx=0).stop 2024-09-13T18:12:19.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.300+0000 7f6de2d9d700 1 -- 172.21.15.40:0/59165793 reap_dead start 2024-09-13T18:12:19.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.300+0000 7f6de2d9d700 1 -- 172.21.15.40:0/59165793 reap_dead start 2024-09-13T18:12:19.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.300+0000 7f6de359e700 1 -- 172.21.15.40:0/59165793 shutdown_connections 2024-09-13T18:12:19.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.300+0000 7f6de359e700 1 -- 172.21.15.40:0/59165793 >> 172.21.15.40:0/59165793 conn(0x7f6de40cb450 msgr2=0x7f6de40c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:19.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.300+0000 7f6de359e700 1 -- 172.21.15.40:0/59165793 shutdown_connections 2024-09-13T18:12:19.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.300+0000 7f6de359e700 1 -- 172.21.15.40:0/59165793 wait complete. 2024-09-13T18:12:19.324 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:12:19.324 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:12:19.325 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:12:19.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.436+0000 7f1b00ef5700 1 Processor -- start 2024-09-13T18:12:19.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.436+0000 7f1b00ef5700 1 -- start start 2024-09-13T18:12:19.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.436+0000 7f1b00ef5700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1af4051a10 0x7f1af40dbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.436+0000 7f1b00ef5700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1af4045d00 0x7f1af404d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.436+0000 7f1b00ef5700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af4007830 con 0x7f1af404cc20 2024-09-13T18:12:19.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.436+0000 7f1b00ef5700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af404d540 con 0x7f1af4051a10 2024-09-13T18:12:19.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.436+0000 7f1b00ef5700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af4007a80 con 0x7f1af4045d00 2024-09-13T18:12:19.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.436+0000 7f1afaffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1af4051a10 0x7f1af40dbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.436+0000 7f1afb7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1af404cc20 0x7f1af4008b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33736/0 (socket says 172.21.15.40:33736) 2024-09-13T18:12:19.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.436+0000 7f1afb7fe700 1 -- 172.21.15.40:0/3674060860 learned_addr learned my addr 172.21.15.40:0/3674060860 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:12:19.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.436+0000 7f1afbfff700 1 --2- 172.21.15.40:0/3674060860 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1af4045d00 0x7f1af404d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.436+0000 7f1afaffd700 1 -- 172.21.15.40:0/3674060860 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1af4045d00 msgr2=0x7f1af404d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:12:19.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.436+0000 7f1afaffd700 1 --2- 172.21.15.40:0/3674060860 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1af4045d00 0x7f1af404d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.436+0000 7f1afaffd700 1 -- 172.21.15.40:0/3674060860 >> v1:172.21.15.40:6789/0 conn(0x7f1af404cc20 legacy=0x7f1af4008b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.436+0000 7f1afa7fc700 1 -- 172.21.15.40:0/3674060860 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922870352 0 0) 0x7f1af4007830 con 0x7f1af404cc20 2024-09-13T18:12:19.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.436+0000 7f1afaffd700 1 -- 172.21.15.40:0/3674060860 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af4007d60 con 0x7f1af4051a10 2024-09-13T18:12:19.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.436+0000 7f1afaffd700 1 --2- 172.21.15.40:0/3674060860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1af4051a10 0x7f1af40dbcb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009850 tx=0x7f1ae8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1ca088f070ec2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:19.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.436+0000 7f1afa7fc700 1 -- 172.21.15.40:0/3674060860 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae800f8a0 con 0x7f1af4051a10 2024-09-13T18:12:19.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.436+0000 7f1afa7fc700 1 -- 172.21.15.40:0/3674060860 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae800fa00 con 0x7f1af4051a10 2024-09-13T18:12:19.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.436+0000 7f1afa7fc700 1 -- 172.21.15.40:0/3674060860 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae800fd90 con 0x7f1af4051a10 2024-09-13T18:12:19.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1b00ef5700 1 -- 172.21.15.40:0/3674060860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1af4051a10 msgr2=0x7f1af40dbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1b00ef5700 1 --2- 172.21.15.40:0/3674060860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1af4051a10 0x7f1af40dbcb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009850 tx=0x7f1ae8009b60 comp rx=0 tx=0).stop 2024-09-13T18:12:19.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1b00ef5700 1 -- 172.21.15.40:0/3674060860 shutdown_connections 2024-09-13T18:12:19.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1b00ef5700 1 --2- 172.21.15.40:0/3674060860 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1af4045d00 0x7f1af404d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1b00ef5700 1 --2- 172.21.15.40:0/3674060860 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1af4051a10 0x7f1af40dbcb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1b00ef5700 1 -- 172.21.15.40:0/3674060860 >> 172.21.15.40:0/3674060860 conn(0x7f1af4050e60 msgr2=0x7f1af4051270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:19.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1b00ef5700 1 -- 172.21.15.40:0/3674060860 shutdown_connections 2024-09-13T18:12:19.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1b00ef5700 1 -- 172.21.15.40:0/3674060860 wait complete. 2024-09-13T18:12:19.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1b00ef5700 1 Processor -- start 2024-09-13T18:12:19.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1b00ef5700 1 -- start start 2024-09-13T18:12:19.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1b00ef5700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1af4045d00 0x7f1af40da290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1b00ef5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1af404cc20 0x7f1af40d2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1b00ef5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1af4051a10 0x7f1af40d8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1b00ef5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af40d5fd0 con 0x7f1af4051a10 2024-09-13T18:12:19.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1b00ef5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af40d3160 con 0x7f1af404cc20 2024-09-13T18:12:19.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1b00ef5700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af4173770 con 0x7f1af4045d00 2024-09-13T18:12:19.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1afbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1af4051a10 0x7f1af40d8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1afbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1af4051a10 0x7f1af40d8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60228/0 (socket says 172.21.15.40:60228) 2024-09-13T18:12:19.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1afbfff700 1 -- 172.21.15.40:0/2804584345 learned_addr learned my addr 172.21.15.40:0/2804584345 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:19.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1afb7fe700 1 --2- 172.21.15.40:0/2804584345 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1af4045d00 0x7f1af40da290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1afaffd700 1 --2- 172.21.15.40:0/2804584345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1af404cc20 0x7f1af40d2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1afbfff700 1 -- 172.21.15.40:0/2804584345 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1af4045d00 msgr2=0x7f1af40da290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1afbfff700 1 --2- 172.21.15.40:0/2804584345 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1af4045d00 0x7f1af40da290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1afbfff700 1 -- 172.21.15.40:0/2804584345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1af404cc20 msgr2=0x7f1af40d2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1afbfff700 1 --2- 172.21.15.40:0/2804584345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1af404cc20 0x7f1af40d2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1afbfff700 1 -- 172.21.15.40:0/2804584345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae80094b0 con 0x7f1af4051a10 2024-09-13T18:12:19.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1afbfff700 1 --2- 172.21.15.40:0/2804584345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1af4051a10 0x7f1af40d8660 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1afc04f940 tx=0x7f1afc0680b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:19.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1ae7fff700 1 -- 172.21.15.40:0/2804584345 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc067930 con 0x7f1af4051a10 2024-09-13T18:12:19.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1b00ef5700 1 -- 172.21.15.40:0/2804584345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af40da920 con 0x7f1af4051a10 2024-09-13T18:12:19.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.440+0000 7f1b00ef5700 1 -- 172.21.15.40:0/2804584345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af40d6500 con 0x7f1af4051a10 2024-09-13T18:12:19.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.444+0000 7f1ae7fff700 1 -- 172.21.15.40:0/2804584345 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1afc072070 con 0x7f1af4051a10 2024-09-13T18:12:19.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.444+0000 7f1ae7fff700 1 -- 172.21.15.40:0/2804584345 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc0723e0 con 0x7f1af4051a10 2024-09-13T18:12:19.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.444+0000 7f1b00ef5700 1 -- 172.21.15.40:0/2804584345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af404bbe0 con 0x7f1af4051a10 2024-09-13T18:12:19.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.444+0000 7f1ae7fff700 1 -- 172.21.15.40:0/2804584345 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1afc072540 con 0x7f1af4051a10 2024-09-13T18:12:19.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.444+0000 7f1ae7fff700 1 --2- 172.21.15.40:0/2804584345 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1ae0066730 0x7f1ae0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:19.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.444+0000 7f1ae7fff700 1 -- 172.21.15.40:0/2804584345 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1afc0fea30 con 0x7f1af4051a10 2024-09-13T18:12:19.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.444+0000 7f1afb7fe700 1 --2- 172.21.15.40:0/2804584345 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1ae0066730 0x7f1ae0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:19.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.444+0000 7f1afb7fe700 1 --2- 172.21.15.40:0/2804584345 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1ae0066730 0x7f1ae0068bf0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1af0000f60 tx=0x7f1af0007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:19.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.452+0000 7f1ae7fff700 1 -- 172.21.15.40:0/2804584345 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1afc0cba70 con 0x7f1af4051a10 2024-09-13T18:12:19.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.711+0000 7f1b00ef5700 1 -- 172.21.15.40:0/2804584345 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f1af4000d90 con 0x7f1af4051a10 2024-09-13T18:12:19.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.711+0000 7f1ae7fff700 1 -- 172.21.15.40:0/2804584345 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f1afc09a020 con 0x7f1af4051a10 2024-09-13T18:12:19.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.715+0000 7f1b00ef5700 1 -- 172.21.15.40:0/2804584345 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1ae0066730 msgr2=0x7f1ae0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.715+0000 7f1b00ef5700 1 --2- 172.21.15.40:0/2804584345 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1ae0066730 0x7f1ae0068bf0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1af0000f60 tx=0x7f1af0007c00 comp rx=0 tx=0).stop 2024-09-13T18:12:19.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.715+0000 7f1b00ef5700 1 -- 172.21.15.40:0/2804584345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1af4051a10 msgr2=0x7f1af40d8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:19.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.715+0000 7f1b00ef5700 1 --2- 172.21.15.40:0/2804584345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1af4051a10 0x7f1af40d8660 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1afc04f940 tx=0x7f1afc0680b0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.715+0000 7f1b00ef5700 1 -- 172.21.15.40:0/2804584345 shutdown_connections 2024-09-13T18:12:19.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.715+0000 7f1b00ef5700 1 --2- 172.21.15.40:0/2804584345 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1af4051a10 0x7f1af40d8660 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.715+0000 7f1b00ef5700 1 --2- 172.21.15.40:0/2804584345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1af404cc20 0x7f1af40d2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.715+0000 7f1b00ef5700 1 --2- 172.21.15.40:0/2804584345 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1ae0066730 0x7f1ae0068bf0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.715+0000 7f1b00ef5700 1 --2- 172.21.15.40:0/2804584345 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1af4045d00 0x7f1af40da290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:19.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.715+0000 7f1b00ef5700 1 -- 172.21.15.40:0/2804584345 >> 172.21.15.40:0/2804584345 conn(0x7f1af4050e60 msgr2=0x7f1af40d3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:19.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.715+0000 7f1b00ef5700 1 -- 172.21.15.40:0/2804584345 shutdown_connections 2024-09-13T18:12:19.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:19.715+0000 7f1b00ef5700 1 -- 172.21.15.40:0/2804584345 wait complete. 2024-09-13T18:12:19.738 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:12:29.739 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:12:29.740 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:12:29.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.851+0000 7f179cd19700 1 Processor -- start 2024-09-13T18:12:29.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.851+0000 7f179cd19700 1 -- start start 2024-09-13T18:12:29.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.851+0000 7f179cd19700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1798072240 0x7f17980c0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:29.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.851+0000 7f179cd19700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17980b2210 con 0x7f1798071e60 2024-09-13T18:12:29.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.851+0000 7f179cd19700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17980c1780 con 0x7f17980c28f0 2024-09-13T18:12:29.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.851+0000 7f179cd19700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17980c1a60 con 0x7f1798072240 2024-09-13T18:12:29.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.851+0000 7f17977fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1798071e60 0x7f17980c32e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34946/0 (socket says 172.21.15.40:34946) 2024-09-13T18:12:29.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.851+0000 7f17977fe700 1 -- 172.21.15.40:0/122316007 learned_addr learned my addr 172.21.15.40:0/122316007 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:12:29.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.851+0000 7f178ffff700 1 --2- 172.21.15.40:0/122316007 >> v2:172.21.15.43:3300/0 conn(0x7f1798072240 0x7f17980c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:29.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.851+0000 7f1796ffd700 1 -- 172.21.15.40:0/122316007 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321046385 0 0) 0x7f17980b2210 con 0x7f1798071e60 2024-09-13T18:12:29.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.851+0000 7f1796ffd700 1 -- 172.21.15.40:0/122316007 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17840035d0 con 0x7f1798071e60 2024-09-13T18:12:29.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.851+0000 7f178ffff700 1 -- 172.21.15.40:0/122316007 >> v1:172.21.15.43:6789/0 conn(0x7f17980c28f0 legacy=0x7f17981565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:29.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.851+0000 7f178ffff700 1 -- 172.21.15.40:0/122316007 >> v1:172.21.15.40:6789/0 conn(0x7f1798071e60 legacy=0x7f17980c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:29.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.851+0000 7f178ffff700 1 -- 172.21.15.40:0/122316007 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17980c5d70 con 0x7f1798072240 2024-09-13T18:12:29.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.851+0000 7f178ffff700 1 --2- 172.21.15.40:0/122316007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1798072240 0x7f17980c0e90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1780009850 tx=0x7f1780009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a13965a3f99cbe53 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:29.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.851+0000 7f1796ffd700 1 -- 172.21.15.40:0/122316007 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1780011830 con 0x7f1798072240 2024-09-13T18:12:29.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.851+0000 7f1796ffd700 1 -- 172.21.15.40:0/122316007 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1780011990 con 0x7f1798072240 2024-09-13T18:12:29.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.851+0000 7f1796ffd700 1 -- 172.21.15.40:0/122316007 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1780011d40 con 0x7f1798072240 2024-09-13T18:12:29.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f179cd19700 1 -- 172.21.15.40:0/122316007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1798072240 msgr2=0x7f17980c0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:29.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f179cd19700 1 --2- 172.21.15.40:0/122316007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1798072240 0x7f17980c0e90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1780009850 tx=0x7f1780009b60 comp rx=0 tx=0).stop 2024-09-13T18:12:29.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f179cd19700 1 -- 172.21.15.40:0/122316007 shutdown_connections 2024-09-13T18:12:29.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f179cd19700 1 --2- 172.21.15.40:0/122316007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1798072240 0x7f17980c0e90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:29.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f179cd19700 1 -- 172.21.15.40:0/122316007 >> 172.21.15.40:0/122316007 conn(0x7f179806e4f0 msgr2=0x7f179806e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:29.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f179cd19700 1 -- 172.21.15.40:0/122316007 shutdown_connections 2024-09-13T18:12:29.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f179cd19700 1 -- 172.21.15.40:0/122316007 wait complete. 2024-09-13T18:12:29.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f179cd19700 1 Processor -- start 2024-09-13T18:12:29.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f179cd19700 1 -- start start 2024-09-13T18:12:29.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f179cd19700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1798071e60 0x7f17980c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:29.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f179cd19700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1798072240 0x7f17980cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:29.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f179cd19700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17980c28f0 0x7f17980cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:29.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f179cd19700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179814e850 con 0x7f1798071e60 2024-09-13T18:12:29.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f179cd19700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17981ef330 con 0x7f17980c28f0 2024-09-13T18:12:29.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f179cd19700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1798152ae0 con 0x7f1798072240 2024-09-13T18:12:29.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f17977fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1798071e60 0x7f17980c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:29.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f17977fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1798071e60 0x7f17980c7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47060/0 (socket says 172.21.15.40:47060) 2024-09-13T18:12:29.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f17977fe700 1 -- 172.21.15.40:0/2608555833 learned_addr learned my addr 172.21.15.40:0/2608555833 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:29.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f178ffff700 1 --2- 172.21.15.40:0/2608555833 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1798072240 0x7f17980cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:29.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f1797fff700 1 --2- 172.21.15.40:0/2608555833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17980c28f0 0x7f17980cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:29.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f17977fe700 1 -- 172.21.15.40:0/2608555833 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1798072240 msgr2=0x7f17980cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:29.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f17977fe700 1 --2- 172.21.15.40:0/2608555833 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1798072240 0x7f17980cad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:29.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f17977fe700 1 -- 172.21.15.40:0/2608555833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17980c28f0 msgr2=0x7f17980cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:29.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f17977fe700 1 --2- 172.21.15.40:0/2608555833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17980c28f0 0x7f17980cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:29.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f17977fe700 1 -- 172.21.15.40:0/2608555833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17800094b0 con 0x7f1798071e60 2024-09-13T18:12:29.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f178ffff700 1 --2- 172.21.15.40:0/2608555833 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1798072240 0x7f17980cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:12:29.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.855+0000 7f17977fe700 1 --2- 172.21.15.40:0/2608555833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1798071e60 0x7f17980c7ce0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f178800c700 tx=0x7f178800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:29.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.859+0000 7f1794ff9700 1 -- 172.21.15.40:0/2608555833 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788005d90 con 0x7f1798071e60 2024-09-13T18:12:29.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.859+0000 7f1794ff9700 1 -- 172.21.15.40:0/2608555833 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17880070d0 con 0x7f1798071e60 2024-09-13T18:12:29.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.859+0000 7f1794ff9700 1 -- 172.21.15.40:0/2608555833 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788007480 con 0x7f1798071e60 2024-09-13T18:12:29.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.859+0000 7f179cd19700 1 -- 172.21.15.40:0/2608555833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1798152dc0 con 0x7f1798071e60 2024-09-13T18:12:29.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.859+0000 7f179cd19700 1 -- 172.21.15.40:0/2608555833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1798153380 con 0x7f1798071e60 2024-09-13T18:12:29.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.859+0000 7f179cd19700 1 -- 172.21.15.40:0/2608555833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17980c0e90 con 0x7f1798071e60 2024-09-13T18:12:29.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.859+0000 7f1794ff9700 1 -- 172.21.15.40:0/2608555833 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f17880075e0 con 0x7f1798071e60 2024-09-13T18:12:29.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.859+0000 7f1794ff9700 1 --2- 172.21.15.40:0/2608555833 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1778066730 0x7f1778068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:29.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.859+0000 7f1794ff9700 1 -- 172.21.15.40:0/2608555833 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1788083b60 con 0x7f1798071e60 2024-09-13T18:12:29.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.859+0000 7f178ffff700 1 --2- 172.21.15.40:0/2608555833 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1778066730 0x7f1778068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:29.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.859+0000 7f178ffff700 1 --2- 172.21.15.40:0/2608555833 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1778066730 0x7f1778068bf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f17800090b0 tx=0x7f1780008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:29.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:29.867+0000 7f1794ff9700 1 -- 172.21.15.40:0/2608555833 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1788050b20 con 0x7f1798071e60 2024-09-13T18:12:30.136 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.127+0000 7f179cd19700 1 -- 172.21.15.40:0/2608555833 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f17981598e0 con 0x7f1798071e60 2024-09-13T18:12:30.137 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.127+0000 7f1794ff9700 1 -- 172.21.15.40:0/2608555833 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f1788054140 con 0x7f1798071e60 2024-09-13T18:12:30.137 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":176,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:12:30.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.131+0000 7f179cd19700 1 -- 172.21.15.40:0/2608555833 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1778066730 msgr2=0x7f1778068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.131+0000 7f179cd19700 1 --2- 172.21.15.40:0/2608555833 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1778066730 0x7f1778068bf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f17800090b0 tx=0x7f1780008030 comp rx=0 tx=0).stop 2024-09-13T18:12:30.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.131+0000 7f179cd19700 1 -- 172.21.15.40:0/2608555833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1798071e60 msgr2=0x7f17980c7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.131+0000 7f179cd19700 1 --2- 172.21.15.40:0/2608555833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1798071e60 0x7f17980c7ce0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f178800c700 tx=0x7f178800cac0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.131+0000 7f179cd19700 1 -- 172.21.15.40:0/2608555833 shutdown_connections 2024-09-13T18:12:30.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.131+0000 7f179cd19700 1 --2- 172.21.15.40:0/2608555833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17980c28f0 0x7f17980cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.131+0000 7f179cd19700 1 --2- 172.21.15.40:0/2608555833 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1778066730 0x7f1778068bf0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.131+0000 7f179cd19700 1 --2- 172.21.15.40:0/2608555833 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1798072240 0x7f17980cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.131+0000 7f179cd19700 1 --2- 172.21.15.40:0/2608555833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1798071e60 0x7f17980c7ce0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.131+0000 7f179cd19700 1 -- 172.21.15.40:0/2608555833 >> 172.21.15.40:0/2608555833 conn(0x7f179806e4f0 msgr2=0x7f179806eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:30.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.131+0000 7f179cd19700 1 -- 172.21.15.40:0/2608555833 shutdown_connections 2024-09-13T18:12:30.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.131+0000 7f179cd19700 1 -- 172.21.15.40:0/2608555833 wait complete. 2024-09-13T18:12:30.151 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:12:30.151 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T18:12:30.151 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:12:30.152 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:12:30.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.259+0000 7fa39bfff700 1 Processor -- start 2024-09-13T18:12:30.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.259+0000 7fa39bfff700 1 -- start start 2024-09-13T18:12:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa39c1503c0 0x7fa39c0c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39bfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa39c0c8500 0x7fa39c0c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39bfff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa39c0c8f80 con 0x7fa39c0c71c0 2024-09-13T18:12:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39bfff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa39c0c2a80 con 0x7fa39c0c8500 2024-09-13T18:12:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39bfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c1507a0 con 0x7fa39c1503c0 2024-09-13T18:12:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39a7fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa39c0c8500 0x7fa39c0c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39a7fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa39c0c8500 0x7fa39c0c6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47072/0 (socket says 172.21.15.40:47072) 2024-09-13T18:12:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39a7fc700 1 -- 172.21.15.40:0/3599705158 learned_addr learned my addr 172.21.15.40:0/3599705158 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39b7fe700 1 --1- 172.21.15.40:0/3599705158 >> v1:172.21.15.40:6789/0 conn(0x7fa39c0c71c0 0x7fa39c159630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34948/0 (socket says 172.21.15.40:34948) 2024-09-13T18:12:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39affd700 1 --2- 172.21.15.40:0/3599705158 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa39c1503c0 0x7fa39c0c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39a7fc700 1 -- 172.21.15.40:0/3599705158 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa39c1503c0 msgr2=0x7fa39c0c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:12:30.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39a7fc700 1 --2- 172.21.15.40:0/3599705158 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa39c1503c0 0x7fa39c0c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39a7fc700 1 -- 172.21.15.40:0/3599705158 >> v1:172.21.15.40:6789/0 conn(0x7fa39c0c71c0 legacy=0x7fa39c159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa399ffb700 1 -- 172.21.15.40:0/3599705158 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2221686017 0 0) 0x7fa39c0c8f80 con 0x7fa39c0c71c0 2024-09-13T18:12:30.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39a7fc700 1 -- 172.21.15.40:0/3599705158 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c0c8a40 con 0x7fa39c0c8500 2024-09-13T18:12:30.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39a7fc700 1 --2- 172.21.15.40:0/3599705158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c0c8500 0x7fa39c0c6c80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa39000be50 tx=0x7fa39000d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daef3cb5353bde07 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:30.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa399ffb700 1 -- 172.21.15.40:0/3599705158 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa390010070 con 0x7fa39c0c8500 2024-09-13T18:12:30.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa399ffb700 1 -- 172.21.15.40:0/3599705158 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39000d9c0 con 0x7fa39c0c8500 2024-09-13T18:12:30.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa399ffb700 1 -- 172.21.15.40:0/3599705158 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39000dd70 con 0x7fa39c0c8500 2024-09-13T18:12:30.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39bfff700 1 -- 172.21.15.40:0/3599705158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c0c8500 msgr2=0x7fa39c0c6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39bfff700 1 --2- 172.21.15.40:0/3599705158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c0c8500 0x7fa39c0c6c80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa39000be50 tx=0x7fa39000d330 comp rx=0 tx=0).stop 2024-09-13T18:12:30.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39bfff700 1 -- 172.21.15.40:0/3599705158 shutdown_connections 2024-09-13T18:12:30.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39bfff700 1 --2- 172.21.15.40:0/3599705158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c0c8500 0x7fa39c0c6c80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39bfff700 1 --2- 172.21.15.40:0/3599705158 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa39c1503c0 0x7fa39c0c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39bfff700 1 -- 172.21.15.40:0/3599705158 >> 172.21.15.40:0/3599705158 conn(0x7fa39c06e6d0 msgr2=0x7fa39c06eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:30.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39bfff700 1 -- 172.21.15.40:0/3599705158 shutdown_connections 2024-09-13T18:12:30.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.263+0000 7fa39bfff700 1 -- 172.21.15.40:0/3599705158 wait complete. 2024-09-13T18:12:30.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.267+0000 7fa39bfff700 1 Processor -- start 2024-09-13T18:12:30.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.267+0000 7fa39bfff700 1 -- start start 2024-09-13T18:12:30.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.267+0000 7fa39bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c0c71c0 0x7fa39c1574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.267+0000 7fa39bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa39c0c8500 0x7fa39c158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.267+0000 7fa39bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa39c1503c0 0x7fa39c158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.267+0000 7fa39bfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c1eb6e0 con 0x7fa39c0c71c0 2024-09-13T18:12:30.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.267+0000 7fa39bfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c1537c0 con 0x7fa39c0c8500 2024-09-13T18:12:30.278 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.267+0000 7fa39bfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c06aff0 con 0x7fa39c1503c0 2024-09-13T18:12:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.267+0000 7fa39affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c0c71c0 0x7fa39c1574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.267+0000 7fa39affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c0c71c0 0x7fa39c1574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47078/0 (socket says 172.21.15.40:47078) 2024-09-13T18:12:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.267+0000 7fa39affd700 1 -- 172.21.15.40:0/2151178001 learned_addr learned my addr 172.21.15.40:0/2151178001 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.267+0000 7fa39b7fe700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa39c1503c0 0x7fa39c158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.267+0000 7fa39a7fc700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa39c0c8500 0x7fa39c158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.267+0000 7fa39affd700 1 -- 172.21.15.40:0/2151178001 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa39c1503c0 msgr2=0x7fa39c158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.267+0000 7fa39affd700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa39c1503c0 0x7fa39c158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.267+0000 7fa39affd700 1 -- 172.21.15.40:0/2151178001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa39c0c8500 msgr2=0x7fa39c158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.267+0000 7fa39affd700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa39c0c8500 0x7fa39c158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.267+0000 7fa39affd700 1 -- 172.21.15.40:0/2151178001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39000bb30 con 0x7fa39c0c71c0 2024-09-13T18:12:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.267+0000 7fa39a7fc700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa39c0c8500 0x7fa39c158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:12:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.267+0000 7fa39affd700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c0c71c0 0x7fa39c1574e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00d800 tx=0x7fa38c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:30.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa3837fe700 1 -- 172.21.15.40:0/2151178001 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c0125f0 con 0x7fa39c0c71c0 2024-09-13T18:12:30.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa3837fe700 1 -- 172.21.15.40:0/2151178001 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38c012750 con 0x7fa39c0c71c0 2024-09-13T18:12:30.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa39bfff700 1 -- 172.21.15.40:0/2151178001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39c06b590 con 0x7fa39c0c71c0 2024-09-13T18:12:30.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa3837fe700 1 -- 172.21.15.40:0/2151178001 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c012ac0 con 0x7fa39c0c71c0 2024-09-13T18:12:30.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa39bfff700 1 -- 172.21.15.40:0/2151178001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39c06b2b0 con 0x7fa39c0c71c0 2024-09-13T18:12:30.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa39bfff700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c158430 0x7fa39c144360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa39bfff700 1 -- 172.21.15.40:0/2151178001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c06b160 con 0x7fa39c158430 2024-09-13T18:12:30.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa39a7fc700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c158430 0x7fa39c144360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa39a7fc700 1 -- 172.21.15.40:0/2151178001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa39c0c6730 con 0x7fa39c158430 2024-09-13T18:12:30.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa3837fe700 1 -- 172.21.15.40:0/2151178001 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa38c012c20 con 0x7fa39c0c71c0 2024-09-13T18:12:30.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa39a7fc700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c158430 0x7fa39c144360 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa39000be50 tx=0x7fa390004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:30.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa3837fe700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa388066730 0x7fa388068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa3837fe700 1 -- 172.21.15.40:0/2151178001 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa38c0845e0 con 0x7fa39c0c71c0 2024-09-13T18:12:30.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa39b7fe700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa388066730 0x7fa388068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa3837fe700 1 -- 172.21.15.40:0/2151178001 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39000b2e0 con 0x7fa39c158430 2024-09-13T18:12:30.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa3837fe700 1 -- 172.21.15.40:0/2151178001 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa3900045d0 con 0x7fa39c158430 2024-09-13T18:12:30.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa3837fe700 1 -- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c158430 msgr2=0x7fa39c144360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa3837fe700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c158430 0x7fa39c144360 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa39000be50 tx=0x7fa390004770 comp rx=0 tx=0).stop 2024-09-13T18:12:30.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa3837fe700 1 -- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c158430 msgr2=0x7fa39c144360 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:12:30.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa3837fe700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c158430 0x7fa39c144360 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.271+0000 7fa39b7fe700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa388066730 0x7fa388068bf0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa394001d10 tx=0x7fa394001ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:30.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa39bfff700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c157e50 0x7fa39c1448a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa39bfff700 1 -- 172.21.15.40:0/2151178001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c067600 con 0x7fa39c157e50 2024-09-13T18:12:30.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa39a7fc700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c157e50 0x7fa39c1448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa39a7fc700 1 -- 172.21.15.40:0/2151178001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa39c0c2710 con 0x7fa39c157e50 2024-09-13T18:12:30.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa39a7fc700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c157e50 0x7fa39c1448a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa39c06d6a0 tx=0x7fa390013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:30.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa3837fe700 1 -- 172.21.15.40:0/2151178001 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa390026ba0 con 0x7fa39c157e50 2024-09-13T18:12:30.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa3837fe700 1 -- 172.21.15.40:0/2151178001 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4453 (secure 0 0 0) 0x7fa39002c040 con 0x7fa39c157e50 2024-09-13T18:12:30.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa3837fe700 1 -- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c157e50 msgr2=0x7fa39c1448a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa3837fe700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c157e50 0x7fa39c1448a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa39c06d6a0 tx=0x7fa390013000 comp rx=0 tx=0).stop 2024-09-13T18:12:30.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa3837fe700 1 -- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c157e50 msgr2=0x7fa39c1448a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:12:30.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa3837fe700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c157e50 0x7fa39c1448a0 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa39c06d6a0 tx=0x7fa390013000 comp rx=0 tx=0).stop 2024-09-13T18:12:30.293 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:12:30.293 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:12:30.293 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:12:30.293 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:12:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 176, 2024-09-13T18:12:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:12:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:12:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:12:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:12:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 11, 2024-09-13T18:12:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:12:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:12:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:12:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:12:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:12:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:12:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:12:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:12:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:12:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:12:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:12:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:12:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:12:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:12:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:12:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:12:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:12:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:12:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:12:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:12:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:12:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:12:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:12:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:12:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:12:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:12:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:12:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:12:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:12:30.300 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:12:30.300 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:30.300 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:30.300 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:30.300 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.300 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.300 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:12:30.300 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:12:30.300 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:30.300 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:30.300 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.300 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:30.301 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:12:30.301 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:30.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.301 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.301 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:30.301 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:12:30.301 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:30.301 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.301 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:30.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.301 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:12:30.301 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:12:30.301 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:30.302 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:30.302 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:30.302 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.302 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.302 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:12:30.302 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:12:30.302 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:30.302 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:30.302 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.302 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:30.302 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:12:30.302 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:30.302 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.302 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.303 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:30.303 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:12:30.303 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:30.303 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.303 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:30.303 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.303 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:12:30.303 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:12:30.303 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:30.303 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:30.303 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:30.303 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.303 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:30.303 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.304 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:12:30.304 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:12:30.304 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.304 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:30.304 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:30.304 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:12:30.304 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.304 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:30.304 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:12:30.304 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.304 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:30.304 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:30.304 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:12:30.304 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.305 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:30.305 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.305 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:12:30.305 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:12:30.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa39bfff700 1 -- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa388066730 msgr2=0x7fa388068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa39bfff700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa388066730 0x7fa388068bf0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa394001d10 tx=0x7fa394001ca0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa39bfff700 1 -- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c0c71c0 msgr2=0x7fa39c1574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa39bfff700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c0c71c0 0x7fa39c1574e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00d800 tx=0x7fa38c00db10 comp rx=0 tx=0).stop 2024-09-13T18:12:30.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa39b7fe700 1 -- 172.21.15.40:0/2151178001 reap_dead start 2024-09-13T18:12:30.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa39bfff700 1 -- 172.21.15.40:0/2151178001 shutdown_connections 2024-09-13T18:12:30.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa39bfff700 1 --2- 172.21.15.40:0/2151178001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa39c0c71c0 0x7fa39c1574e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa39bfff700 1 -- 172.21.15.40:0/2151178001 >> 172.21.15.40:0/2151178001 conn(0x7fa39c06e6d0 msgr2=0x7fa39c1562a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:30.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa39bfff700 1 -- 172.21.15.40:0/2151178001 shutdown_connections 2024-09-13T18:12:30.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.283+0000 7fa39bfff700 1 -- 172.21.15.40:0/2151178001 wait complete. 2024-09-13T18:12:30.307 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:12:30.307 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:12:30.308 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:12:30.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.423+0000 7fada1a6f700 1 Processor -- start 2024-09-13T18:12:30.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.423+0000 7fada1a6f700 1 -- start start 2024-09-13T18:12:30.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.423+0000 7fada1a6f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fad9404cc20 0x7fad940dbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.423+0000 7fada1a6f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad94007830 con 0x7fad94051a10 2024-09-13T18:12:30.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.423+0000 7fada1a6f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad94007a80 con 0x7fad94045d00 2024-09-13T18:12:30.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.423+0000 7fada1a6f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad94007d60 con 0x7fad9404cc20 2024-09-13T18:12:30.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.423+0000 7fad9bfff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fad94051a10 0x7fad94008b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34958/0 (socket says 172.21.15.40:34958) 2024-09-13T18:12:30.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.423+0000 7fad9bfff700 1 -- 172.21.15.40:0/1856599236 learned_addr learned my addr 172.21.15.40:0/1856599236 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:12:30.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.423+0000 7fada0a6d700 1 --2- 172.21.15.40:0/1856599236 >> v2:172.21.15.43:3300/0 conn(0x7fad9404cc20 0x7fad940dbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.427+0000 7fad9b7fe700 1 -- 172.21.15.40:0/1856599236 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4137448191 0 0) 0x7fad94007830 con 0x7fad94051a10 2024-09-13T18:12:30.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.427+0000 7fad9b7fe700 1 -- 172.21.15.40:0/1856599236 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad8c0035d0 con 0x7fad94051a10 2024-09-13T18:12:30.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.427+0000 7fada0a6d700 1 -- 172.21.15.40:0/1856599236 >> v1:172.21.15.43:6789/0 conn(0x7fad94045d00 legacy=0x7fad940e5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.427+0000 7fada0a6d700 1 -- 172.21.15.40:0/1856599236 >> v1:172.21.15.40:6789/0 conn(0x7fad94051a10 legacy=0x7fad94008b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.427+0000 7fada0a6d700 1 -- 172.21.15.40:0/1856599236 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad94008040 con 0x7fad9404cc20 2024-09-13T18:12:30.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.427+0000 7fada0a6d700 1 --2- 172.21.15.40:0/1856599236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9404cc20 0x7fad940dbcb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad90009850 tx=0x7fad90009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6a7c21084d04900 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:30.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.427+0000 7fad9b7fe700 1 -- 172.21.15.40:0/1856599236 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90011830 con 0x7fad9404cc20 2024-09-13T18:12:30.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.427+0000 7fad9b7fe700 1 -- 172.21.15.40:0/1856599236 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad90011990 con 0x7fad9404cc20 2024-09-13T18:12:30.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.427+0000 7fad9b7fe700 1 -- 172.21.15.40:0/1856599236 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90011d40 con 0x7fad9404cc20 2024-09-13T18:12:30.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.427+0000 7fada1a6f700 1 -- 172.21.15.40:0/1856599236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9404cc20 msgr2=0x7fad940dbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.427+0000 7fada1a6f700 1 --2- 172.21.15.40:0/1856599236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9404cc20 0x7fad940dbcb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad90009850 tx=0x7fad90009b60 comp rx=0 tx=0).stop 2024-09-13T18:12:30.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.427+0000 7fada1a6f700 1 -- 172.21.15.40:0/1856599236 shutdown_connections 2024-09-13T18:12:30.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.427+0000 7fada1a6f700 1 --2- 172.21.15.40:0/1856599236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9404cc20 0x7fad940dbcb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.427+0000 7fada1a6f700 1 -- 172.21.15.40:0/1856599236 >> 172.21.15.40:0/1856599236 conn(0x7fad94050e60 msgr2=0x7fad94051270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:30.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.427+0000 7fada1a6f700 1 -- 172.21.15.40:0/1856599236 shutdown_connections 2024-09-13T18:12:30.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.427+0000 7fada1a6f700 1 -- 172.21.15.40:0/1856599236 wait complete. 2024-09-13T18:12:30.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fada1a6f700 1 Processor -- start 2024-09-13T18:12:30.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fada1a6f700 1 -- start start 2024-09-13T18:12:30.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fada1a6f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad94045d00 0x7fad940ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fada1a6f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad9404cc20 0x7fad940465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fada1a6f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad94051a10 0x7fad94165520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fada1a6f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad94164720 con 0x7fad9404cc20 2024-09-13T18:12:30.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fada1a6f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad94164ef0 con 0x7fad94051a10 2024-09-13T18:12:30.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fada1a6f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad940c5a70 con 0x7fad94045d00 2024-09-13T18:12:30.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fad9bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad9404cc20 0x7fad940465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fad9bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad9404cc20 0x7fad940465e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47092/0 (socket says 172.21.15.40:47092) 2024-09-13T18:12:30.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fad9bfff700 1 -- 172.21.15.40:0/320463000 learned_addr learned my addr 172.21.15.40:0/320463000 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:30.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fada0a6d700 1 --2- 172.21.15.40:0/320463000 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad94045d00 0x7fad940ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fada126e700 1 --2- 172.21.15.40:0/320463000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad94051a10 0x7fad94165520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fad9bfff700 1 -- 172.21.15.40:0/320463000 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad94045d00 msgr2=0x7fad940ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fad9bfff700 1 --2- 172.21.15.40:0/320463000 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad94045d00 0x7fad940ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fad9bfff700 1 -- 172.21.15.40:0/320463000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad94051a10 msgr2=0x7fad94165520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fad9bfff700 1 --2- 172.21.15.40:0/320463000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad94051a10 0x7fad94165520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fad9bfff700 1 -- 172.21.15.40:0/320463000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad900094b0 con 0x7fad9404cc20 2024-09-13T18:12:30.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fad9bfff700 1 --2- 172.21.15.40:0/320463000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad9404cc20 0x7fad940465e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fad8800e6e0 tx=0x7fad8800e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:30.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fad997fa700 1 -- 172.21.15.40:0/320463000 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad88005d90 con 0x7fad9404cc20 2024-09-13T18:12:30.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fad997fa700 1 -- 172.21.15.40:0/320463000 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad880070d0 con 0x7fad9404cc20 2024-09-13T18:12:30.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fada1a6f700 1 -- 172.21.15.40:0/320463000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad94166100 con 0x7fad9404cc20 2024-09-13T18:12:30.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fad997fa700 1 -- 172.21.15.40:0/320463000 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad88007460 con 0x7fad9404cc20 2024-09-13T18:12:30.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.431+0000 7fada1a6f700 1 -- 172.21.15.40:0/320463000 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad94164c10 con 0x7fad9404cc20 2024-09-13T18:12:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.435+0000 7fada1a6f700 1 --2- 172.21.15.40:0/320463000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad94165a60 0x7fad940c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.435+0000 7fada1a6f700 1 -- 172.21.15.40:0/320463000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad940095f0 con 0x7fad94165a60 2024-09-13T18:12:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.435+0000 7fada0a6d700 1 --2- 172.21.15.40:0/320463000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad94165a60 0x7fad940c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.435+0000 7fada0a6d700 1 -- 172.21.15.40:0/320463000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad94005680 con 0x7fad94165a60 2024-09-13T18:12:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.435+0000 7fada0a6d700 1 --2- 172.21.15.40:0/320463000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad94165a60 0x7fad940c5be0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fad90009850 tx=0x7fad90004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.435+0000 7fad997fa700 1 -- 172.21.15.40:0/320463000 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fad880075c0 con 0x7fad9404cc20 2024-09-13T18:12:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.435+0000 7fad997fa700 1 --2- 172.21.15.40:0/320463000 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad800666d0 0x7fad80068b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.435+0000 7fada126e700 1 --2- 172.21.15.40:0/320463000 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad800666d0 0x7fad80068b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.435+0000 7fad997fa700 1 -- 172.21.15.40:0/320463000 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fad88086750 con 0x7fad9404cc20 2024-09-13T18:12:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.435+0000 7fad997fa700 1 -- 172.21.15.40:0/320463000 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90020070 con 0x7fad94165a60 2024-09-13T18:12:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.435+0000 7fad997fa700 1 -- 172.21.15.40:0/320463000 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fad900117f0 con 0x7fad94165a60 2024-09-13T18:12:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.435+0000 7fad997fa700 1 -- 172.21.15.40:0/320463000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad94165a60 msgr2=0x7fad940c5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.435+0000 7fad997fa700 1 --2- 172.21.15.40:0/320463000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad94165a60 0x7fad940c5be0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fad90009850 tx=0x7fad90004770 comp rx=0 tx=0).stop 2024-09-13T18:12:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.435+0000 7fad997fa700 1 -- 172.21.15.40:0/320463000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad94165a60 msgr2=0x7fad940c5be0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:12:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.435+0000 7fad997fa700 1 --2- 172.21.15.40:0/320463000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad94165a60 0x7fad940c5be0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.435+0000 7fada126e700 1 --2- 172.21.15.40:0/320463000 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad800666d0 0x7fad80068b90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fad9c063cf0 tx=0x7fad9c065ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:30.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.447+0000 7fada1a6f700 1 --2- 172.21.15.40:0/320463000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9404c530 0x7fad940c1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.447+0000 7fada1a6f700 1 -- 172.21.15.40:0/320463000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad940c6120 con 0x7fad9404c530 2024-09-13T18:12:30.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.447+0000 7fada0a6d700 1 --2- 172.21.15.40:0/320463000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9404c530 0x7fad940c1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.447+0000 7fada0a6d700 1 -- 172.21.15.40:0/320463000 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fad9404f460 con 0x7fad9404c530 2024-09-13T18:12:30.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.447+0000 7fada0a6d700 1 --2- 172.21.15.40:0/320463000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9404c530 0x7fad940c1930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fad90000c00 tx=0x7fad90011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:30.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.447+0000 7fad997fa700 1 -- 172.21.15.40:0/320463000 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90003b40 con 0x7fad9404c530 2024-09-13T18:12:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.447+0000 7fad997fa700 1 -- 172.21.15.40:0/320463000 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4783 (secure 0 0 0) 0x7fad90003ca0 con 0x7fad9404c530 2024-09-13T18:12:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.447+0000 7fad997fa700 1 -- 172.21.15.40:0/320463000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9404c530 msgr2=0x7fad940c1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.447+0000 7fad997fa700 1 --2- 172.21.15.40:0/320463000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9404c530 0x7fad940c1930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fad90000c00 tx=0x7fad90011000 comp rx=0 tx=0).stop 2024-09-13T18:12:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.447+0000 7fad997fa700 1 -- 172.21.15.40:0/320463000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9404c530 msgr2=0x7fad940c1930 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:12:30.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.447+0000 7fad997fa700 1 --2- 172.21.15.40:0/320463000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad9404c530 0x7fad940c1930 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.458 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:12:30.458 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:12:30.458 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:12:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:12:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 176, 2024-09-13T18:12:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:12:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:12:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:12:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:12:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 12, 2024-09-13T18:12:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:12:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:12:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:12:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:30.460 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:30.460 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:30.460 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:30.460 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:30.460 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:30.460 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:30.460 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:30.460 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:30.460 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:12:30.460 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:12:30.460 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:30.460 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:30.460 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:30.461 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:30.461 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:30.461 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:30.461 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:30.461 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:30.461 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:30.461 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:30.461 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:30.461 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.464 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:12:30.464 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:12:30.464 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:12:30.464 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:12:30.465 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:12:30.465 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:12:30.465 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:12:30.465 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:12:30.465 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:12:30.465 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:12:30.465 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:12:30.465 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:12:30.465 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:12:30.465 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:12:30.465 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:12:30.465 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:12:30.465 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:12:30.465 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:30.465 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:30.466 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:30.466 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:30.466 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:30.466 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:30.466 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:30.466 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:30.466 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:30.466 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:30.466 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:30.466 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:12:30.466 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.466 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:12:30.466 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.466 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:12:30.467 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:12:30.467 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:30.467 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:30.467 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.467 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:30.467 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:12:30.467 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:30.467 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.467 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.467 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:30.467 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:12:30.467 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:30.467 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.467 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:30.468 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.468 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:12:30.468 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:12:30.468 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:30.468 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:30.468 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:30.468 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.468 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.468 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:12:30.468 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:12:30.468 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:30.468 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:30.468 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.468 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:30.469 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:12:30.469 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:30.469 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.469 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.469 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:30.469 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:12:30.469 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:30.469 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.469 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:30.469 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.469 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:12:30.469 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:12:30.469 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:30.469 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:30.470 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:30.470 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.470 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.470 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:12:30.470 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:12:30.470 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:30.470 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:30.470 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.470 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:30.470 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:12:30.470 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:30.470 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.470 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.470 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:30.471 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:12:30.471 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:30.471 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.471 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:30.471 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.471 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:12:30.471 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:12:30.471 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:30.471 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:30.471 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:30.471 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.471 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:30.471 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.471 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:12:30.472 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:12:30.472 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.472 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:30.472 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:30.472 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:12:30.472 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.472 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:30.472 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:12:30.472 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.472 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:30.472 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:30.472 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:12:30.472 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.472 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:30.473 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:12:30.473 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.473 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:30.473 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:30.473 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:12:30.473 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.473 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:30.473 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:12:30.473 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.473 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:30.473 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:30.473 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:12:30.473 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.473 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:30.474 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.474 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:12:30.474 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:12:30.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.451+0000 7fada1a6f700 1 -- 172.21.15.40:0/320463000 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad800666d0 msgr2=0x7fad80068b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.451+0000 7fada1a6f700 1 --2- 172.21.15.40:0/320463000 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad800666d0 0x7fad80068b90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fad9c063cf0 tx=0x7fad9c065ea0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.451+0000 7fada1a6f700 1 -- 172.21.15.40:0/320463000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad9404cc20 msgr2=0x7fad940465e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.451+0000 7fada1a6f700 1 --2- 172.21.15.40:0/320463000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad9404cc20 0x7fad940465e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fad8800e6e0 tx=0x7fad8800e9f0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.451+0000 7fada126e700 1 -- 172.21.15.40:0/320463000 reap_dead start 2024-09-13T18:12:30.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.451+0000 7fada126e700 1 -- 172.21.15.40:0/320463000 reap_dead start 2024-09-13T18:12:30.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.451+0000 7fada1a6f700 1 -- 172.21.15.40:0/320463000 shutdown_connections 2024-09-13T18:12:30.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.451+0000 7fada1a6f700 1 -- 172.21.15.40:0/320463000 >> 172.21.15.40:0/320463000 conn(0x7fad94050e60 msgr2=0x7fad94035e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:30.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.451+0000 7fada1a6f700 1 -- 172.21.15.40:0/320463000 shutdown_connections 2024-09-13T18:12:30.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.451+0000 7fada1a6f700 1 -- 172.21.15.40:0/320463000 wait complete. 2024-09-13T18:12:30.476 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:12:30.476 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:12:30.476 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:12:30.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.587+0000 7f2fb359e700 1 Processor -- start 2024-09-13T18:12:30.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.587+0000 7f2fb359e700 1 -- start start 2024-09-13T18:12:30.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.587+0000 7f2fb359e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2fb4153720 0x7f2fb40ca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.587+0000 7f2fb359e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb40cf9b0 0x7f2fb406c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.587+0000 7f2fb359e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb4159e50 con 0x7f2fb406c5b0 2024-09-13T18:12:30.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.587+0000 7f2fb359e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb40c5ea0 con 0x7f2fb4153720 2024-09-13T18:12:30.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.587+0000 7f2fb359e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb415d390 con 0x7f2fb40cf9b0 2024-09-13T18:12:30.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.587+0000 7f2fb259c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2fb4153720 0x7f2fb40ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.587+0000 7f2fb259c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2fb4153720 0x7f2fb40ca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47098/0 (socket says 172.21.15.40:47098) 2024-09-13T18:12:30.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.587+0000 7f2fb259c700 1 -- 172.21.15.40:0/2045420010 learned_addr learned my addr 172.21.15.40:0/2045420010 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:30.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.587+0000 7f2fabfff700 1 --2- 172.21.15.40:0/2045420010 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb40cf9b0 0x7f2fb406c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.591+0000 7f2fb259c700 1 -- 172.21.15.40:0/2045420010 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb40cf9b0 msgr2=0x7f2fb406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:12:30.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.591+0000 7f2fb259c700 1 --2- 172.21.15.40:0/2045420010 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb40cf9b0 0x7f2fb406c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.591+0000 7f2fb259c700 1 -- 172.21.15.40:0/2045420010 >> v1:172.21.15.43:6789/0 conn(0x7f2fb406c5b0 legacy=0x7f2fb406aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.591+0000 7f2fb259c700 1 -- 172.21.15.40:0/2045420010 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb415a080 con 0x7f2fb4153720 2024-09-13T18:12:30.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.591+0000 7f2fb259c700 1 --2- 172.21.15.40:0/2045420010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fb4153720 0x7f2fb40ca510 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009910 tx=0x7f2fa4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e9f347e31f98ec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.591+0000 7f2fb1d9b700 1 -- 172.21.15.40:0/2045420010 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4011830 con 0x7f2fb4153720 2024-09-13T18:12:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.591+0000 7f2fb1d9b700 1 -- 172.21.15.40:0/2045420010 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa4011990 con 0x7f2fb4153720 2024-09-13T18:12:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.591+0000 7f2fb1d9b700 1 -- 172.21.15.40:0/2045420010 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4011d40 con 0x7f2fb4153720 2024-09-13T18:12:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.591+0000 7f2fb359e700 1 -- 172.21.15.40:0/2045420010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fb4153720 msgr2=0x7f2fb40ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.591+0000 7f2fb359e700 1 --2- 172.21.15.40:0/2045420010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fb4153720 0x7f2fb40ca510 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009910 tx=0x7f2fa4009c20 comp rx=0 tx=0).stop 2024-09-13T18:12:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.591+0000 7f2fb359e700 1 -- 172.21.15.40:0/2045420010 shutdown_connections 2024-09-13T18:12:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.591+0000 7f2fb359e700 1 --2- 172.21.15.40:0/2045420010 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb40cf9b0 0x7f2fb406c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.591+0000 7f2fb359e700 1 --2- 172.21.15.40:0/2045420010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fb4153720 0x7f2fb40ca510 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.591+0000 7f2fb359e700 1 -- 172.21.15.40:0/2045420010 >> 172.21.15.40:0/2045420010 conn(0x7f2fb40666c0 msgr2=0x7f2fb40cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.591+0000 7f2fb359e700 1 -- 172.21.15.40:0/2045420010 shutdown_connections 2024-09-13T18:12:30.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.591+0000 7f2fb359e700 1 -- 172.21.15.40:0/2045420010 wait complete. 2024-09-13T18:12:30.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.595+0000 7f2fb359e700 1 Processor -- start 2024-09-13T18:12:30.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.595+0000 7f2fb359e700 1 -- start start 2024-09-13T18:12:30.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.595+0000 7f2fb359e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb406c5b0 0x7f2fb40ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.595+0000 7f2fb359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fb40cf9b0 0x7f2fb41ef1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.595+0000 7f2fb359e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fb4153720 0x7f2fb41ef6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.595+0000 7f2fb359e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb4154dc0 con 0x7f2fb40cf9b0 2024-09-13T18:12:30.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.595+0000 7f2fb359e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb41f33f0 con 0x7f2fb4153720 2024-09-13T18:12:30.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.595+0000 7f2fb359e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb41efc20 con 0x7f2fb406c5b0 2024-09-13T18:12:30.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.595+0000 7f2fb259c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb406c5b0 0x7f2fb40ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.595+0000 7f2fb259c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb406c5b0 0x7f2fb40ca070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:39386/0 (socket says 172.21.15.40:39386) 2024-09-13T18:12:30.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.595+0000 7f2fb259c700 1 -- 172.21.15.40:0/4074708948 learned_addr learned my addr 172.21.15.40:0/4074708948 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:30.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.595+0000 7f2fb2d9d700 1 --2- 172.21.15.40:0/4074708948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fb4153720 0x7f2fb41ef6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.595+0000 7f2fabfff700 1 --2- 172.21.15.40:0/4074708948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fb40cf9b0 0x7f2fb41ef1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.595+0000 7f2fb259c700 1 -- 172.21.15.40:0/4074708948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fb4153720 msgr2=0x7f2fb41ef6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.595+0000 7f2fb259c700 1 --2- 172.21.15.40:0/4074708948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fb4153720 0x7f2fb41ef6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.595+0000 7f2fb259c700 1 -- 172.21.15.40:0/4074708948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fb40cf9b0 msgr2=0x7f2fb41ef1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.595+0000 7f2fb259c700 1 --2- 172.21.15.40:0/4074708948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fb40cf9b0 0x7f2fb41ef1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.595+0000 7f2fb259c700 1 -- 172.21.15.40:0/4074708948 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa4009570 con 0x7f2fb406c5b0 2024-09-13T18:12:30.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.595+0000 7f2fb259c700 1 --2- 172.21.15.40:0/4074708948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb406c5b0 0x7f2fb40ca070 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4013000 tx=0x7f2fa4004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:30.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.599+0000 7f2faaffd700 1 -- 172.21.15.40:0/4074708948 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4004510 con 0x7f2fb406c5b0 2024-09-13T18:12:30.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.599+0000 7f2faaffd700 1 -- 172.21.15.40:0/4074708948 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa401e070 con 0x7f2fb406c5b0 2024-09-13T18:12:30.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.599+0000 7f2fb359e700 1 -- 172.21.15.40:0/4074708948 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb41f27b0 con 0x7f2fb406c5b0 2024-09-13T18:12:30.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.599+0000 7f2faaffd700 1 -- 172.21.15.40:0/4074708948 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4003900 con 0x7f2fb406c5b0 2024-09-13T18:12:30.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.599+0000 7f2fb359e700 1 -- 172.21.15.40:0/4074708948 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb41f2d30 con 0x7f2fb406c5b0 2024-09-13T18:12:30.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.603+0000 7f2fb359e700 1 --2- 172.21.15.40:0/4074708948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb41f2ec0 0x7f2fb414c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.603+0000 7f2fb359e700 1 -- 172.21.15.40:0/4074708948 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb4074750 con 0x7f2fb41f2ec0 2024-09-13T18:12:30.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.603+0000 7f2fabfff700 1 --2- 172.21.15.40:0/4074708948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb41f2ec0 0x7f2fb414c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.603+0000 7f2fabfff700 1 -- 172.21.15.40:0/4074708948 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2fb415ca60 con 0x7f2fb41f2ec0 2024-09-13T18:12:30.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.603+0000 7f2faaffd700 1 -- 172.21.15.40:0/4074708948 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2fa4003a60 con 0x7f2fb406c5b0 2024-09-13T18:12:30.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.603+0000 7f2fabfff700 1 --2- 172.21.15.40:0/4074708948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb41f2ec0 0x7f2fb414c760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2fb414d950 tx=0x7f2f9c011d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:30.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.603+0000 7f2faaffd700 1 --2- 172.21.15.40:0/4074708948 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f94066760 0x7f2f94068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.603+0000 7f2fb2d9d700 1 --2- 172.21.15.40:0/4074708948 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f94066760 0x7f2f94068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.603+0000 7f2faaffd700 1 -- 172.21.15.40:0/4074708948 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2fa4085440 con 0x7f2fb406c5b0 2024-09-13T18:12:30.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.603+0000 7f2faaffd700 1 -- 172.21.15.40:0/4074708948 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c004030 con 0x7f2fb41f2ec0 2024-09-13T18:12:30.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.603+0000 7f2faaffd700 1 -- 172.21.15.40:0/4074708948 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f2f9c004190 con 0x7f2fb41f2ec0 2024-09-13T18:12:30.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.603+0000 7f2faaffd700 1 -- 172.21.15.40:0/4074708948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb41f2ec0 msgr2=0x7f2fb414c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.603+0000 7f2faaffd700 1 --2- 172.21.15.40:0/4074708948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb41f2ec0 0x7f2fb414c760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2fb414d950 tx=0x7f2f9c011d30 comp rx=0 tx=0).stop 2024-09-13T18:12:30.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.603+0000 7f2faaffd700 1 -- 172.21.15.40:0/4074708948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb41f2ec0 msgr2=0x7f2fb414c760 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:12:30.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.603+0000 7f2faaffd700 1 --2- 172.21.15.40:0/4074708948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb41f2ec0 0x7f2fb414c760 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.603+0000 7f2fb2d9d700 1 --2- 172.21.15.40:0/4074708948 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f94066760 0x7f2f94068c20 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0012e0 tx=0x7f2fac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:30.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.611+0000 7f2fb359e700 1 --2- 172.21.15.40:0/4074708948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb4154750 0x7f2fb414cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.611+0000 7f2fb359e700 1 -- 172.21.15.40:0/4074708948 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb415cd40 con 0x7f2fb4154750 2024-09-13T18:12:30.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.611+0000 7f2fabfff700 1 --2- 172.21.15.40:0/4074708948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb4154750 0x7f2fb414cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.611+0000 7f2fabfff700 1 -- 172.21.15.40:0/4074708948 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2fb4067a40 con 0x7f2fb4154750 2024-09-13T18:12:30.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.615+0000 7f2fabfff700 1 --2- 172.21.15.40:0/4074708948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb4154750 0x7f2fb414cca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4151810 tx=0x7f2f9c004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:30.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.615+0000 7f2faaffd700 1 -- 172.21.15.40:0/4074708948 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c014440 con 0x7f2fb4154750 2024-09-13T18:12:30.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.615+0000 7f2faaffd700 1 -- 172.21.15.40:0/4074708948 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4949 (secure 0 0 0) 0x7f2f9c0273a0 con 0x7f2fb4154750 2024-09-13T18:12:30.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.615+0000 7f2faaffd700 1 -- 172.21.15.40:0/4074708948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb4154750 msgr2=0x7f2fb414cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.615+0000 7f2faaffd700 1 --2- 172.21.15.40:0/4074708948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb4154750 0x7f2fb414cca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4151810 tx=0x7f2f9c004770 comp rx=0 tx=0).stop 2024-09-13T18:12:30.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.615+0000 7f2faaffd700 1 -- 172.21.15.40:0/4074708948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb4154750 msgr2=0x7f2fb414cca0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:12:30.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.615+0000 7f2faaffd700 1 --2- 172.21.15.40:0/4074708948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb4154750 0x7f2fb414cca0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.624 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:12:30.624 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:12:30.624 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:12:30.624 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:12:30.624 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 176, 2024-09-13T18:12:30.625 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:12:30.625 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:12:30.625 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:12:30.625 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:12:30.625 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:30.625 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 12, 2024-09-13T18:12:30.625 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:12:30.625 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:12:30.625 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:12:30.625 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:30.625 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:30.625 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:30.626 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:30.626 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:30.626 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:30.626 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:30.626 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:30.626 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:30.626 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:30.626 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:30.626 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:12:30.626 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:12:30.626 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:30.626 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:30.626 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:30.627 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:30.627 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:30.627 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:30.627 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:30.627 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:30.627 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:30.627 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:30.627 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:30.627 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.627 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:12:30.627 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:12:30.627 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:12:30.627 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:12:30.627 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:12:30.628 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:12:30.628 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:12:30.628 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:12:30.628 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:12:30.628 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:12:30.628 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:12:30.628 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:12:30.628 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:12:30.628 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:12:30.628 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:12:30.628 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:12:30.628 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:12:30.628 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:30.629 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:30.629 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:30.629 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:30.629 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:30.629 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:30.629 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:30.629 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:30.629 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:30.629 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:30.629 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:30.629 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:12:30.629 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.630 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:12:30.630 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.630 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:12:30.630 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:12:30.630 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:30.630 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:30.630 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.630 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:30.630 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:12:30.630 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:30.630 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.631 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.631 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:30.631 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:12:30.631 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:30.631 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.631 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:30.631 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.631 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:12:30.631 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:12:30.631 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:30.631 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:30.631 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:30.631 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.632 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.632 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:12:30.632 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:12:30.632 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:30.632 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:30.632 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.632 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:30.632 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:12:30.632 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:30.632 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.633 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.633 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:30.633 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:12:30.633 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:30.633 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.633 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:30.633 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.633 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:12:30.633 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:12:30.633 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:30.633 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:30.633 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:30.633 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.634 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.634 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:12:30.634 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:12:30.634 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:30.634 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:30.634 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.634 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:30.634 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:12:30.634 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:30.634 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.634 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.634 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:30.634 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:12:30.634 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:30.634 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.635 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:30.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.635 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:12:30.635 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:12:30.635 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:30.635 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:30.635 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:30.635 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.635 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:30.635 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.635 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:12:30.635 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:12:30.635 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.635 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:30.636 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:30.636 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:12:30.636 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.636 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:30.636 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:12:30.636 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.636 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:30.636 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:30.636 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:12:30.636 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.636 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:30.636 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:12:30.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.637 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:30.637 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:30.637 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:12:30.637 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.637 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:30.637 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:12:30.637 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.637 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:30.637 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:30.637 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:12:30.637 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.637 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:30.638 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:12:30.638 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:30.638 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:30.638 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:30.638 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:12:30.638 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:30.638 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:30.638 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:30.638 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:12:30.638 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:12:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.615+0000 7f2fb359e700 1 -- 172.21.15.40:0/4074708948 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f94066760 msgr2=0x7f2f94068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.615+0000 7f2fb359e700 1 --2- 172.21.15.40:0/4074708948 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f94066760 0x7f2f94068c20 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0012e0 tx=0x7f2fac009040 comp rx=0 tx=0).stop 2024-09-13T18:12:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.615+0000 7f2fb359e700 1 -- 172.21.15.40:0/4074708948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb406c5b0 msgr2=0x7f2fb40ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.615+0000 7f2fb359e700 1 --2- 172.21.15.40:0/4074708948 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fb406c5b0 0x7f2fb40ca070 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4013000 tx=0x7f2fa4004770 comp rx=0 tx=0).stop 2024-09-13T18:12:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.615+0000 7f2fb2d9d700 1 -- 172.21.15.40:0/4074708948 reap_dead start 2024-09-13T18:12:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.615+0000 7f2fb2d9d700 1 -- 172.21.15.40:0/4074708948 reap_dead start 2024-09-13T18:12:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.615+0000 7f2fb359e700 1 -- 172.21.15.40:0/4074708948 shutdown_connections 2024-09-13T18:12:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.615+0000 7f2fb359e700 1 -- 172.21.15.40:0/4074708948 >> 172.21.15.40:0/4074708948 conn(0x7f2fb40666c0 msgr2=0x7f2fb4150f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.615+0000 7f2fb359e700 1 -- 172.21.15.40:0/4074708948 shutdown_connections 2024-09-13T18:12:30.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.615+0000 7f2fb359e700 1 -- 172.21.15.40:0/4074708948 wait complete. 2024-09-13T18:12:30.642 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:12:30.642 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-13T18:12:30.642 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T18:12:30.642 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-13T18:12:30.642 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T18:12:30.642 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T18:12:30.643 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:12:30.684 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T18:12:30.684 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T18:12:30.684 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:12:30.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.787+0000 7feff542c700 1 Processor -- start 2024-09-13T18:12:30.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.787+0000 7feff542c700 1 -- start start 2024-09-13T18:12:30.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.787+0000 7feff542c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feff01503d0 0x7feff0156940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.787+0000 7feff542c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feff015e960 0x7feff0160d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.787+0000 7feff542c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff015a520 con 0x7feff006b4a0 2024-09-13T18:12:30.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.787+0000 7feff542c700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff01564b0 con 0x7feff015e960 2024-09-13T18:12:30.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.787+0000 7feff542c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff015a000 con 0x7feff01503d0 2024-09-13T18:12:30.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7fefeffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feff01503d0 0x7feff0156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7fefef7fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7feff006b4a0 0x7feff0071bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:48774/0 (socket says 172.21.15.40:48774) 2024-09-13T18:12:30.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7fefef7fe700 1 -- 172.21.15.40:0/785479094 learned_addr learned my addr 172.21.15.40:0/785479094 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:12:30.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7feff4c2b700 1 --2- 172.21.15.40:0/785479094 >> v2:172.21.15.43:3300/0 conn(0x7feff015e960 0x7feff0160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7fefeeffd700 1 -- 172.21.15.40:0/785479094 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299218849 0 0) 0x7feff015a520 con 0x7feff006b4a0 2024-09-13T18:12:30.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7fefeeffd700 1 -- 172.21.15.40:0/785479094 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefd80035d0 con 0x7feff006b4a0 2024-09-13T18:12:30.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7fefeffff700 1 -- 172.21.15.40:0/785479094 >> v2:172.21.15.43:3300/0 conn(0x7feff015e960 msgr2=0x7feff0160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:12:30.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7fefeffff700 1 --2- 172.21.15.40:0/785479094 >> v2:172.21.15.43:3300/0 conn(0x7feff015e960 0x7feff0160d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7fefeffff700 1 -- 172.21.15.40:0/785479094 >> v1:172.21.15.43:6789/0 conn(0x7feff006b4a0 legacy=0x7feff0071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7fefeffff700 1 -- 172.21.15.40:0/785479094 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff015a280 con 0x7feff01503d0 2024-09-13T18:12:30.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7fefeffff700 1 --2- 172.21.15.40:0/785479094 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feff01503d0 0x7feff0156940 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009850 tx=0x7fefe0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a35c4d707451a8c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:30.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7fefeeffd700 1 -- 172.21.15.40:0/785479094 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe000f8a0 con 0x7feff01503d0 2024-09-13T18:12:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7fefeeffd700 1 -- 172.21.15.40:0/785479094 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe000fa00 con 0x7feff01503d0 2024-09-13T18:12:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7fefeeffd700 1 -- 172.21.15.40:0/785479094 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe000fd70 con 0x7feff01503d0 2024-09-13T18:12:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7feff542c700 1 -- 172.21.15.40:0/785479094 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feff01503d0 msgr2=0x7feff0156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7feff542c700 1 --2- 172.21.15.40:0/785479094 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feff01503d0 0x7feff0156940 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009850 tx=0x7fefe0009b60 comp rx=0 tx=0).stop 2024-09-13T18:12:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7feff542c700 1 -- 172.21.15.40:0/785479094 shutdown_connections 2024-09-13T18:12:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7feff542c700 1 --2- 172.21.15.40:0/785479094 >> v2:172.21.15.43:3300/0 conn(0x7feff015e960 0x7feff0160d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7feff542c700 1 --2- 172.21.15.40:0/785479094 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feff01503d0 0x7feff0156940 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7feff542c700 1 -- 172.21.15.40:0/785479094 >> 172.21.15.40:0/785479094 conn(0x7feff006e4f0 msgr2=0x7feff006e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7feff542c700 1 -- 172.21.15.40:0/785479094 shutdown_connections 2024-09-13T18:12:30.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.791+0000 7feff542c700 1 -- 172.21.15.40:0/785479094 wait complete. 2024-09-13T18:12:30.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7feff542c700 1 Processor -- start 2024-09-13T18:12:30.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7feff542c700 1 -- start start 2024-09-13T18:12:30.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7feff542c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff006b4a0 0x7feff0070e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7feff542c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feff01503d0 0x7feff01ee340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7feff542c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff015e960 0x7feff01ee880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7feff542c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff00cb030 con 0x7feff015e960 2024-09-13T18:12:30.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7feff542c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff00cb2d0 con 0x7feff006b4a0 2024-09-13T18:12:30.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7feff542c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff00cb5b0 con 0x7feff01503d0 2024-09-13T18:12:30.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7feff4c2b700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff015e960 msgr2=0x7feff01ee880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:12:30.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7feff4c2b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff015e960 0x7feff01ee880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:12:30.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7fefef7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feff01503d0 0x7feff01ee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7fefeffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff006b4a0 0x7feff0070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7fefef7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feff01503d0 0x7feff01ee340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:39428/0 (socket says 172.21.15.40:39428) 2024-09-13T18:12:30.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7fefef7fe700 1 -- 172.21.15.40:0/1358319550 learned_addr learned my addr 172.21.15.40:0/1358319550 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:30.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7fefeffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff006b4a0 0x7feff0070e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:37252/0 (socket says 172.21.15.40:37252) 2024-09-13T18:12:30.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7fefef7fe700 1 -- 172.21.15.40:0/1358319550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff006b4a0 msgr2=0x7feff0070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:30.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7fefef7fe700 1 --2- 172.21.15.40:0/1358319550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff006b4a0 0x7feff0070e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7fefef7fe700 1 -- 172.21.15.40:0/1358319550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff015e960 msgr2=0x7feff01ee880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:12:30.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7fefef7fe700 1 --2- 172.21.15.40:0/1358319550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff015e960 0x7feff01ee880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:30.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7fefef7fe700 1 -- 172.21.15.40:0/1358319550 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe00094b0 con 0x7feff01503d0 2024-09-13T18:12:30.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7fefeffff700 1 --2- 172.21.15.40:0/1358319550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff006b4a0 0x7feff0070e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:12:30.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7fefef7fe700 1 --2- 172.21.15.40:0/1358319550 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feff01503d0 0x7feff01ee340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fefe400c700 tx=0x7fefe400ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:30.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7fefecff9700 1 -- 172.21.15.40:0/1358319550 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe4005d90 con 0x7feff01503d0 2024-09-13T18:12:30.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7fefecff9700 1 -- 172.21.15.40:0/1358319550 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe40070d0 con 0x7feff01503d0 2024-09-13T18:12:30.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7feff542c700 1 -- 172.21.15.40:0/1358319550 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff0145a40 con 0x7feff01503d0 2024-09-13T18:12:30.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.795+0000 7fefecff9700 1 -- 172.21.15.40:0/1358319550 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe4007480 con 0x7feff01503d0 2024-09-13T18:12:30.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.799+0000 7feff542c700 1 -- 172.21.15.40:0/1358319550 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff0146000 con 0x7feff01503d0 2024-09-13T18:12:30.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.799+0000 7feff542c700 1 -- 172.21.15.40:0/1358319550 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff014d7a0 con 0x7feff01503d0 2024-09-13T18:12:30.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.799+0000 7fefecff9700 1 -- 172.21.15.40:0/1358319550 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fefe40075e0 con 0x7feff01503d0 2024-09-13T18:12:30.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.799+0000 7fefecff9700 1 --2- 172.21.15.40:0/1358319550 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fefdc066710 0x7fefdc068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:30.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.799+0000 7fefecff9700 1 -- 172.21.15.40:0/1358319550 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fefe4083ad0 con 0x7feff01503d0 2024-09-13T18:12:30.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.799+0000 7fefeffff700 1 --2- 172.21.15.40:0/1358319550 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fefdc066710 0x7fefdc068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:30.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.803+0000 7fefeffff700 1 --2- 172.21.15.40:0/1358319550 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fefdc066710 0x7fefdc068bd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fefe00090b0 tx=0x7fefe0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:30.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:30.811+0000 7fefecff9700 1 -- 172.21.15.40:0/1358319550 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fefe40509f0 con 0x7feff01503d0 2024-09-13T18:12:31.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:31.063+0000 7feff542c700 1 -- 172.21.15.40:0/1358319550 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7feff00c25b0 con 0x7feff01503d0 2024-09-13T18:12:31.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:31.067+0000 7fefecff9700 1 -- 172.21.15.40:0/1358319550 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7fefe4017020 con 0x7feff01503d0 2024-09-13T18:12:31.074 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":176,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:12:31.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:31.067+0000 7feff542c700 1 -- 172.21.15.40:0/1358319550 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fefdc066710 msgr2=0x7fefdc068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:31.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:31.067+0000 7feff542c700 1 --2- 172.21.15.40:0/1358319550 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fefdc066710 0x7fefdc068bd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fefe00090b0 tx=0x7fefe0008030 comp rx=0 tx=0).stop 2024-09-13T18:12:31.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:31.067+0000 7feff542c700 1 -- 172.21.15.40:0/1358319550 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feff01503d0 msgr2=0x7feff01ee340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:31.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:31.067+0000 7feff542c700 1 --2- 172.21.15.40:0/1358319550 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feff01503d0 0x7feff01ee340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fefe400c700 tx=0x7fefe400ca10 comp rx=0 tx=0).stop 2024-09-13T18:12:31.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:31.067+0000 7feff542c700 1 -- 172.21.15.40:0/1358319550 shutdown_connections 2024-09-13T18:12:31.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:31.067+0000 7feff542c700 1 --2- 172.21.15.40:0/1358319550 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feff015e960 0x7feff01ee880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:31.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:31.067+0000 7feff542c700 1 --2- 172.21.15.40:0/1358319550 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fefdc066710 0x7fefdc068bd0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:31.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:31.067+0000 7feff542c700 1 --2- 172.21.15.40:0/1358319550 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feff01503d0 0x7feff01ee340 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:31.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:31.067+0000 7feff542c700 1 --2- 172.21.15.40:0/1358319550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff006b4a0 0x7feff0070e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:31.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:31.067+0000 7feff542c700 1 -- 172.21.15.40:0/1358319550 >> 172.21.15.40:0/1358319550 conn(0x7feff006e4f0 msgr2=0x7feff006d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:31.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:31.067+0000 7feff542c700 1 -- 172.21.15.40:0/1358319550 shutdown_connections 2024-09-13T18:12:31.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:31.067+0000 7feff542c700 1 -- 172.21.15.40:0/1358319550 wait complete. 2024-09-13T18:12:34.090 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:12:34.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.203+0000 7f0a3542d700 1 Processor -- start 2024-09-13T18:12:34.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.203+0000 7f0a3542d700 1 -- start start 2024-09-13T18:12:34.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.203+0000 7f0a3542d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0a30150f30 0x7f0a300c7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:34.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.203+0000 7f0a3542d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a30151310 0x7f0a300c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:34.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.203+0000 7f0a3542d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0a300661b0 0x7f0a300c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:34.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.203+0000 7f0a3542d700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a300c7f10 con 0x7f0a30150f30 2024-09-13T18:12:34.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.203+0000 7f0a3542d700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a301572b0 con 0x7f0a300661b0 2024-09-13T18:12:34.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.203+0000 7f0a3542d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a300c8120 con 0x7f0a30151310 2024-09-13T18:12:34.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.203+0000 7f0a2ffff700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f0a30150f30 msgr2=0x7f0a300c7320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:12:34.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.203+0000 7f0a2ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0a30150f30 0x7f0a300c7320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:12:34.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.203+0000 7f0a2f7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a30151310 0x7f0a300c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:34.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.203+0000 7f0a34c2c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0a300661b0 0x7f0a300c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:34.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.203+0000 7f0a2f7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a30151310 0x7f0a300c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:41864/0 (socket says 172.21.15.40:41864) 2024-09-13T18:12:34.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.203+0000 7f0a2f7fe700 1 -- 172.21.15.40:0/1075575832 learned_addr learned my addr 172.21.15.40:0/1075575832 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:34.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.203+0000 7f0a34c2c700 1 -- 172.21.15.40:0/1075575832 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a30151310 msgr2=0x7f0a300c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:12:34.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.203+0000 7f0a34c2c700 1 --2- 172.21.15.40:0/1075575832 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a30151310 0x7f0a300c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:34.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.203+0000 7f0a34c2c700 1 -- 172.21.15.40:0/1075575832 >> v2:172.21.15.40:3300/0 conn(0x7f0a30150f30 msgr2=0x7f0a300c7320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:12:34.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.203+0000 7f0a34c2c700 1 --2- 172.21.15.40:0/1075575832 >> v2:172.21.15.40:3300/0 conn(0x7f0a30150f30 0x7f0a300c7320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:34.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.203+0000 7f0a34c2c700 1 -- 172.21.15.40:0/1075575832 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a300c8400 con 0x7f0a300661b0 2024-09-13T18:12:34.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.203+0000 7f0a2f7fe700 1 --2- 172.21.15.40:0/1075575832 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a30151310 0x7f0a300c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:12:34.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.207+0000 7f0a34c2c700 1 --2- 172.21.15.40:0/1075575832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a300661b0 0x7f0a300c50d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a300c4a60 tx=0x7f0a2800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a9285a328bfb5dc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:34.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.207+0000 7f0a2effd700 1 -- 172.21.15.40:0/1075575832 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2800e070 con 0x7f0a300661b0 2024-09-13T18:12:34.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.207+0000 7f0a2effd700 1 -- 172.21.15.40:0/1075575832 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2800bb20 con 0x7f0a300661b0 2024-09-13T18:12:34.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.207+0000 7f0a2effd700 1 -- 172.21.15.40:0/1075575832 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a28012430 con 0x7f0a300661b0 2024-09-13T18:12:34.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.207+0000 7f0a3542d700 1 -- 172.21.15.40:0/1075575832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a300661b0 msgr2=0x7f0a300c50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:34.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.207+0000 7f0a3542d700 1 --2- 172.21.15.40:0/1075575832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a300661b0 0x7f0a300c50d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a300c4a60 tx=0x7f0a2800b330 comp rx=0 tx=0).stop 2024-09-13T18:12:34.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.207+0000 7f0a3542d700 1 -- 172.21.15.40:0/1075575832 shutdown_connections 2024-09-13T18:12:34.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.207+0000 7f0a3542d700 1 --2- 172.21.15.40:0/1075575832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a300661b0 0x7f0a300c50d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.207+0000 7f0a3542d700 1 --2- 172.21.15.40:0/1075575832 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a30151310 0x7f0a300c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.207+0000 7f0a3542d700 1 --2- 172.21.15.40:0/1075575832 >> v2:172.21.15.40:3300/0 conn(0x7f0a30150f30 0x7f0a300c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.207+0000 7f0a3542d700 1 -- 172.21.15.40:0/1075575832 >> 172.21.15.40:0/1075575832 conn(0x7f0a300cb4c0 msgr2=0x7f0a300cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.207+0000 7f0a3542d700 1 -- 172.21.15.40:0/1075575832 shutdown_connections 2024-09-13T18:12:34.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.207+0000 7f0a3542d700 1 -- 172.21.15.40:0/1075575832 wait complete. 2024-09-13T18:12:34.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.207+0000 7f0a3542d700 1 Processor -- start 2024-09-13T18:12:34.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.207+0000 7f0a3542d700 1 -- start start 2024-09-13T18:12:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a3542d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a300661b0 0x7f0a30145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a3542d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a30150f30 0x7f0a30141890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a3542d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a30151310 0x7f0a3013f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a2ffff700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a300661b0 msgr2=0x7f0a30145750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:12:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a3542d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a30140770 con 0x7f0a300661b0 2024-09-13T18:12:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a2ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a300661b0 0x7f0a30145750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:12:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a3542d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a30152e40 con 0x7f0a30151310 2024-09-13T18:12:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a3542d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a301562e0 con 0x7f0a30150f30 2024-09-13T18:12:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a2f7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a30150f30 0x7f0a30141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a2f7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a30150f30 0x7f0a30141890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:41876/0 (socket says 172.21.15.40:41876) 2024-09-13T18:12:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a2f7fe700 1 -- 172.21.15.40:0/1180942059 learned_addr learned my addr 172.21.15.40:0/1180942059 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:34.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a34c2c700 1 --2- 172.21.15.40:0/1180942059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a30151310 0x7f0a3013f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:34.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a2f7fe700 1 -- 172.21.15.40:0/1180942059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a30151310 msgr2=0x7f0a3013f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:34.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a2f7fe700 1 --2- 172.21.15.40:0/1180942059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a30151310 0x7f0a3013f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:34.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a2f7fe700 1 -- 172.21.15.40:0/1180942059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a300661b0 msgr2=0x7f0a30145750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:12:34.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a2f7fe700 1 --2- 172.21.15.40:0/1180942059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a300661b0 0x7f0a30145750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:34.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a2f7fe700 1 -- 172.21.15.40:0/1180942059 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a24009570 con 0x7f0a30150f30 2024-09-13T18:12:34.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a2f7fe700 1 --2- 172.21.15.40:0/1180942059 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a30150f30 0x7f0a30141890 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400ebb0 tx=0x7f0a2400eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:34.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a2cff9700 1 -- 172.21.15.40:0/1180942059 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2400c850 con 0x7f0a30150f30 2024-09-13T18:12:34.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a2cff9700 1 -- 172.21.15.40:0/1180942059 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2400c9b0 con 0x7f0a30150f30 2024-09-13T18:12:34.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a3542d700 1 -- 172.21.15.40:0/1180942059 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a28009a70 con 0x7f0a30150f30 2024-09-13T18:12:34.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a2cff9700 1 -- 172.21.15.40:0/1180942059 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2400cd60 con 0x7f0a30150f30 2024-09-13T18:12:34.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.211+0000 7f0a3542d700 1 -- 172.21.15.40:0/1180942059 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a30153380 con 0x7f0a30150f30 2024-09-13T18:12:34.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.215+0000 7f0a3542d700 1 -- 172.21.15.40:0/1180942059 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a300c50d0 con 0x7f0a30150f30 2024-09-13T18:12:34.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.215+0000 7f0a2cff9700 1 -- 172.21.15.40:0/1180942059 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0a24013470 con 0x7f0a30150f30 2024-09-13T18:12:34.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.215+0000 7f0a2cff9700 1 --2- 172.21.15.40:0/1180942059 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0a1c0666e0 0x7f0a1c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:34.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.215+0000 7f0a2ffff700 1 --2- 172.21.15.40:0/1180942059 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0a1c0666e0 0x7f0a1c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:34.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.215+0000 7f0a2cff9700 1 -- 172.21.15.40:0/1180942059 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0a24084660 con 0x7f0a30150f30 2024-09-13T18:12:34.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.215+0000 7f0a2ffff700 1 --2- 172.21.15.40:0/1180942059 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0a1c0666e0 0x7f0a1c068ba0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0a20006fd0 tx=0x7f0a20006de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:34.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.227+0000 7f0a2cff9700 1 -- 172.21.15.40:0/1180942059 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0a24051580 con 0x7f0a30150f30 2024-09-13T18:12:34.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.507+0000 7f0a3542d700 1 -- 172.21.15.40:0/1180942059 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0a3006e2a0 con 0x7f0a30150f30 2024-09-13T18:12:34.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.507+0000 7f0a2cff9700 1 -- 172.21.15.40:0/1180942059 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f0a24017620 con 0x7f0a30150f30 2024-09-13T18:12:34.516 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":176,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:12:34.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.511+0000 7f0a3542d700 1 -- 172.21.15.40:0/1180942059 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0a1c0666e0 msgr2=0x7f0a1c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.511+0000 7f0a3542d700 1 --2- 172.21.15.40:0/1180942059 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0a1c0666e0 0x7f0a1c068ba0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0a20006fd0 tx=0x7f0a20006de0 comp rx=0 tx=0).stop 2024-09-13T18:12:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.511+0000 7f0a3542d700 1 -- 172.21.15.40:0/1180942059 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a30150f30 msgr2=0x7f0a30141890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.511+0000 7f0a3542d700 1 --2- 172.21.15.40:0/1180942059 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a30150f30 0x7f0a30141890 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400ebb0 tx=0x7f0a2400eec0 comp rx=0 tx=0).stop 2024-09-13T18:12:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.511+0000 7f0a3542d700 1 -- 172.21.15.40:0/1180942059 shutdown_connections 2024-09-13T18:12:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.511+0000 7f0a3542d700 1 --2- 172.21.15.40:0/1180942059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a30151310 0x7f0a3013f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.511+0000 7f0a3542d700 1 --2- 172.21.15.40:0/1180942059 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0a1c0666e0 0x7f0a1c068ba0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.511+0000 7f0a3542d700 1 --2- 172.21.15.40:0/1180942059 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a30150f30 0x7f0a30141890 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.511+0000 7f0a3542d700 1 --2- 172.21.15.40:0/1180942059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0a300661b0 0x7f0a30145750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.511+0000 7f0a3542d700 1 -- 172.21.15.40:0/1180942059 >> 172.21.15.40:0/1180942059 conn(0x7f0a300cb4c0 msgr2=0x7f0a3014c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.511+0000 7f0a3542d700 1 -- 172.21.15.40:0/1180942059 shutdown_connections 2024-09-13T18:12:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:34.511+0000 7f0a3542d700 1 -- 172.21.15.40:0/1180942059 wait complete. 2024-09-13T18:12:37.531 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:12:37.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.643+0000 7f41eb0d6700 1 Processor -- start 2024-09-13T18:12:37.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.643+0000 7f41eb0d6700 1 -- start start 2024-09-13T18:12:37.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.643+0000 7f41eb0d6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41dc0d87a0 0x7f41dc0df110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:37.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41eb0d6700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41dc047610 con 0x7f41dc007b60 2024-09-13T18:12:37.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41eb0d6700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41dc006b50 con 0x7f41dc0469c0 2024-09-13T18:12:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41eb0d6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc04bef0 con 0x7f41dc0d87a0 2024-09-13T18:12:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41e98d3700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f41dc007b60 legacy=0x7f41dc04b7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:12:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41ea0d4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41dc0d87a0 0x7f41dc0df110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41ea8d5700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f41dc0469c0 0x7f41dc0e9010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:51010/0 (socket says 172.21.15.40:51010) 2024-09-13T18:12:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41ea0d4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41dc0d87a0 0x7f41dc0df110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:41890/0 (socket says 172.21.15.40:41890) 2024-09-13T18:12:37.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41ea8d5700 1 -- 172.21.15.40:0/1038786449 learned_addr learned my addr 172.21.15.40:0/1038786449 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:12:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41ea0d4700 1 -- 172.21.15.40:0/1038786449 >> v1:172.21.15.43:6789/0 conn(0x7f41dc0469c0 legacy=0x7f41dc0e9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41ea0d4700 1 -- 172.21.15.40:0/1038786449 >> v1:172.21.15.40:6789/0 conn(0x7f41dc007b60 legacy=0x7f41dc04b7e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:12:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41ea0d4700 1 -- 172.21.15.40:0/1038786449 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc0076d0 con 0x7f41dc0d87a0 2024-09-13T18:12:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41ea0d4700 1 --2- 172.21.15.40:0/1038786449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41dc0d87a0 0x7f41dc0df110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f41e4069620 tx=0x7f41e4069e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e51739599b131d7f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41e90d2700 1 -- 172.21.15.40:0/1038786449 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4067fd0 con 0x7f41dc0d87a0 2024-09-13T18:12:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41e90d2700 1 -- 172.21.15.40:0/1038786449 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e4068a70 con 0x7f41dc0d87a0 2024-09-13T18:12:37.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41e90d2700 1 -- 172.21.15.40:0/1038786449 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4074720 con 0x7f41dc0d87a0 2024-09-13T18:12:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41eb0d6700 1 -- 172.21.15.40:0/1038786449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41dc0d87a0 msgr2=0x7f41dc0df110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41eb0d6700 1 --2- 172.21.15.40:0/1038786449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41dc0d87a0 0x7f41dc0df110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f41e4069620 tx=0x7f41e4069e70 comp rx=0 tx=0).stop 2024-09-13T18:12:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41eb0d6700 1 -- 172.21.15.40:0/1038786449 shutdown_connections 2024-09-13T18:12:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41eb0d6700 1 --2- 172.21.15.40:0/1038786449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41dc0d87a0 0x7f41dc0df110 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:37.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41eb0d6700 1 -- 172.21.15.40:0/1038786449 >> 172.21.15.40:0/1038786449 conn(0x7f41dc050e60 msgr2=0x7f41dc051270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:37.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41eb0d6700 1 -- 172.21.15.40:0/1038786449 shutdown_connections 2024-09-13T18:12:37.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.647+0000 7f41eb0d6700 1 -- 172.21.15.40:0/1038786449 wait complete. 2024-09-13T18:12:37.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41eb0d6700 1 Processor -- start 2024-09-13T18:12:37.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41eb0d6700 1 -- start start 2024-09-13T18:12:37.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41eb0d6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41dc0469c0 0x7f41dc16b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:37.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41eb0d6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41dc0d87a0 0x7f41dc0dc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:37.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41eb0d6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41dc04a870 0x7f41dc0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:37.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41eb0d6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc1780a0 con 0x7f41dc04a870 2024-09-13T18:12:37.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41eb0d6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc04b000 con 0x7f41dc0469c0 2024-09-13T18:12:37.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41eb0d6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc049360 con 0x7f41dc0d87a0 2024-09-13T18:12:37.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41ea8d5700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41dc04a870 msgr2=0x7f41dc0ce6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:12:37.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41ea8d5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41dc04a870 0x7f41dc0ce6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:12:37.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41ea0d4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41dc0469c0 0x7f41dc16b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:37.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41e98d3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41dc0d87a0 0x7f41dc0dc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:37.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41ea0d4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41dc0469c0 0x7f41dc16b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:49372/0 (socket says 172.21.15.40:49372) 2024-09-13T18:12:37.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41ea0d4700 1 -- 172.21.15.40:0/1429387157 learned_addr learned my addr 172.21.15.40:0/1429387157 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:37.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41e98d3700 1 -- 172.21.15.40:0/1429387157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41dc0469c0 msgr2=0x7f41dc16b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:37.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41e98d3700 1 --2- 172.21.15.40:0/1429387157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41dc0469c0 0x7f41dc16b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:37.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41e98d3700 1 -- 172.21.15.40:0/1429387157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41dc04a870 msgr2=0x7f41dc0ce6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:12:37.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41e98d3700 1 --2- 172.21.15.40:0/1429387157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41dc04a870 0x7f41dc0ce6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:37.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41e98d3700 1 -- 172.21.15.40:0/1429387157 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e404f5c0 con 0x7f41dc0d87a0 2024-09-13T18:12:37.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41ea0d4700 1 --2- 172.21.15.40:0/1429387157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41dc0469c0 0x7f41dc16b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:12:37.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41e98d3700 1 --2- 172.21.15.40:0/1429387157 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41dc0d87a0 0x7f41dc0dc500 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f41e000b540 tx=0x7f41e000b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:37.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41daffd700 1 -- 172.21.15.40:0/1429387157 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0003e70 con 0x7f41dc0d87a0 2024-09-13T18:12:37.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41eb0d6700 1 -- 172.21.15.40:0/1429387157 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41dc049640 con 0x7f41dc0d87a0 2024-09-13T18:12:37.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41daffd700 1 -- 172.21.15.40:0/1429387157 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e0005070 con 0x7f41dc0d87a0 2024-09-13T18:12:37.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41eb0d6700 1 -- 172.21.15.40:0/1429387157 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41dc049c00 con 0x7f41dc0d87a0 2024-09-13T18:12:37.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.651+0000 7f41daffd700 1 -- 172.21.15.40:0/1429387157 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e00053e0 con 0x7f41dc0d87a0 2024-09-13T18:12:37.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.655+0000 7f41eb0d6700 1 -- 172.21.15.40:0/1429387157 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41dc007080 con 0x7f41dc0d87a0 2024-09-13T18:12:37.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.655+0000 7f41daffd700 1 -- 172.21.15.40:0/1429387157 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f41e0005540 con 0x7f41dc0d87a0 2024-09-13T18:12:37.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.655+0000 7f41daffd700 1 --2- 172.21.15.40:0/1429387157 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f41c80666e0 0x7f41c8068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:37.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.655+0000 7f41daffd700 1 -- 172.21.15.40:0/1429387157 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f41e0083e70 con 0x7f41dc0d87a0 2024-09-13T18:12:37.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.655+0000 7f41ea0d4700 1 --2- 172.21.15.40:0/1429387157 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f41c80666e0 0x7f41c8068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:37.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.655+0000 7f41ea0d4700 1 --2- 172.21.15.40:0/1429387157 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f41c80666e0 0x7f41c8068ba0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f41e40641b0 tx=0x7f41e406d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:37.671 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:12:37.673 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:12:37.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.667+0000 7f41daffd700 1 -- 172.21.15.40:0/1429387157 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f41e0050e30 con 0x7f41dc0d87a0 2024-09-13T18:12:37.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.931+0000 7f41eb0d6700 1 -- 172.21.15.40:0/1429387157 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f41dc0d5d60 con 0x7f41dc0d87a0 2024-09-13T18:12:37.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.931+0000 7f41daffd700 1 -- 172.21.15.40:0/1429387157 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f41e0054450 con 0x7f41dc0d87a0 2024-09-13T18:12:37.940 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":176,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:12:37.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.935+0000 7f41eb0d6700 1 -- 172.21.15.40:0/1429387157 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f41c80666e0 msgr2=0x7f41c8068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:37.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.935+0000 7f41eb0d6700 1 --2- 172.21.15.40:0/1429387157 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f41c80666e0 0x7f41c8068ba0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f41e40641b0 tx=0x7f41e406d040 comp rx=0 tx=0).stop 2024-09-13T18:12:37.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.935+0000 7f41eb0d6700 1 -- 172.21.15.40:0/1429387157 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41dc0d87a0 msgr2=0x7f41dc0dc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:37.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.935+0000 7f41eb0d6700 1 --2- 172.21.15.40:0/1429387157 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41dc0d87a0 0x7f41dc0dc500 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f41e000b540 tx=0x7f41e000b850 comp rx=0 tx=0).stop 2024-09-13T18:12:37.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.935+0000 7f41eb0d6700 1 -- 172.21.15.40:0/1429387157 shutdown_connections 2024-09-13T18:12:37.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.935+0000 7f41eb0d6700 1 --2- 172.21.15.40:0/1429387157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41dc04a870 0x7f41dc0ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:37.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.935+0000 7f41eb0d6700 1 --2- 172.21.15.40:0/1429387157 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f41c80666e0 0x7f41c8068ba0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:37.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.935+0000 7f41eb0d6700 1 --2- 172.21.15.40:0/1429387157 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f41dc0d87a0 0x7f41dc0dc500 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:37.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.935+0000 7f41eb0d6700 1 --2- 172.21.15.40:0/1429387157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41dc0469c0 0x7f41dc16b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:37.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.935+0000 7f41eb0d6700 1 -- 172.21.15.40:0/1429387157 >> 172.21.15.40:0/1429387157 conn(0x7f41dc050e60 msgr2=0x7f41dc167cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:37.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.935+0000 7f41eb0d6700 1 -- 172.21.15.40:0/1429387157 shutdown_connections 2024-09-13T18:12:37.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:37.935+0000 7f41eb0d6700 1 -- 172.21.15.40:0/1429387157 wait complete. 2024-09-13T18:12:40.961 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:12:41.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.083+0000 7ff1f58ef700 1 Processor -- start 2024-09-13T18:12:41.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.083+0000 7ff1f58ef700 1 -- start start 2024-09-13T18:12:41.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.087+0000 7ff1f58ef700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 0x7ff1f00cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:41.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.087+0000 7ff1f48ed700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7ff1f0153880 legacy=0x7ff1f00c9ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:12:41.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.087+0000 7ff1f58ef700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f0159770 con 0x7ff1f0153880 2024-09-13T18:12:41.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.087+0000 7ff1f58ef700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f0159a50 con 0x7ff1f00cf3c0 2024-09-13T18:12:41.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.087+0000 7ff1f58ef700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f0159d30 con 0x7ff1f00cb2f0 2024-09-13T18:12:41.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.087+0000 7ff1effff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 0x7ff1f00cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:41.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.087+0000 7ff1f50ee700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff1f00cf3c0 0x7ff1f015cf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:51016/0 (socket says 172.21.15.40:51016) 2024-09-13T18:12:41.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.087+0000 7ff1f50ee700 1 -- 172.21.15.40:0/3501640454 learned_addr learned my addr 172.21.15.40:0/3501640454 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:12:41.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.087+0000 7ff1effff700 1 -- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 msgr2=0x7ff1f00cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:12:41.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.087+0000 7ff1effff700 1 -- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 msgr2=0x7ff1f00cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:12:41.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.087+0000 7ff1effff700 1 --2- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 0x7ff1f00cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:41.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.087+0000 7ff1effff700 1 --2- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 0x7ff1f00cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:12:41.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.291+0000 7ff1f48ed700 1 -- 172.21.15.40:0/3501640454 >> v1:172.21.15.40:6789/0 conn(0x7ff1f0153880 legacy=0x7ff1f00c9ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:12:41.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.291+0000 7ff1effff700 1 --2- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 0x7ff1f00cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:41.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.291+0000 7ff1effff700 1 -- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 msgr2=0x7ff1f00cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:12:41.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.291+0000 7ff1effff700 1 -- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 msgr2=0x7ff1f00cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:12:41.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.291+0000 7ff1effff700 1 --2- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 0x7ff1f00cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:41.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.291+0000 7ff1effff700 1 --2- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 0x7ff1f00cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:12:41.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.695+0000 7ff1f48ed700 1 -- 172.21.15.40:0/3501640454 >> v1:172.21.15.40:6789/0 conn(0x7ff1f0153880 legacy=0x7ff1f00c9ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:12:41.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.695+0000 7ff1effff700 1 --2- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 0x7ff1f00cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:41.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.695+0000 7ff1effff700 1 -- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 msgr2=0x7ff1f00cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:12:41.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.695+0000 7ff1effff700 1 -- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 msgr2=0x7ff1f00cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:12:41.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.695+0000 7ff1effff700 1 --2- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 0x7ff1f00cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:41.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:41.695+0000 7ff1effff700 1 --2- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 0x7ff1f00cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:12:42.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:42.499+0000 7ff1f48ed700 1 -- 172.21.15.40:0/3501640454 >> v1:172.21.15.40:6789/0 conn(0x7ff1f0153880 legacy=0x7ff1f00c9ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:12:42.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:42.499+0000 7ff1effff700 1 --2- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 0x7ff1f00cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:42.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:42.499+0000 7ff1effff700 1 -- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 msgr2=0x7ff1f00cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:12:42.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:42.499+0000 7ff1effff700 1 -- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 msgr2=0x7ff1f00cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:12:42.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:42.499+0000 7ff1effff700 1 --2- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 0x7ff1f00cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:42.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:42.499+0000 7ff1effff700 1 --2- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 0x7ff1f00cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:12:44.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.087+0000 7ff1ee7fc700 1 -- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 msgr2=0x7ff1f00cee80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:12:44.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.087+0000 7ff1ee7fc700 1 --2- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1f00cb2f0 0x7ff1f00cee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:44.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.087+0000 7ff1ee7fc700 1 -- 172.21.15.40:0/3501640454 >> v1:172.21.15.43:6789/0 conn(0x7ff1f00cf3c0 legacy=0x7ff1f015cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:44.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.087+0000 7ff1ee7fc700 1 -- 172.21.15.40:0/3501640454 >> v1:172.21.15.40:6789/0 conn(0x7ff1f0153880 legacy=0x7ff1f00c9ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:12:44.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.087+0000 7ff1ee7fc700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 0x7ff1d4005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:44.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.087+0000 7ff1ee7fc700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.40:3300/0 conn(0x7ff1d4006220 0x7ff1d40086c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:44.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.087+0000 7ff1ee7fc700 1 -- 172.21.15.40:0/3501640454 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f0159d30 con 0x7ff1d4006220 2024-09-13T18:12:44.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.087+0000 7ff1ee7fc700 1 -- 172.21.15.40:0/3501640454 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d4008d70 con 0x7ff1d4003770 2024-09-13T18:12:44.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.087+0000 7ff1effff700 1 -- 172.21.15.40:0/3501640454 >> v2:172.21.15.40:3300/0 conn(0x7ff1d4006220 msgr2=0x7ff1d40086c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:12:44.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.087+0000 7ff1effff700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.40:3300/0 conn(0x7ff1d4006220 0x7ff1d40086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:12:44.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.087+0000 7ff1f48ed700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 0x7ff1d4005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:44.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.087+0000 7ff1f48ed700 1 -- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 msgr2=0x7ff1d4005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:12:44.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.087+0000 7ff1f48ed700 1 -- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 msgr2=0x7ff1d4005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:12:44.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.087+0000 7ff1f48ed700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 0x7ff1d4005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:44.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.087+0000 7ff1f48ed700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 0x7ff1d4005bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:12:44.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.291+0000 7ff1effff700 1 -- 172.21.15.40:0/3501640454 >> v2:172.21.15.40:3300/0 conn(0x7ff1d4006220 msgr2=0x7ff1d40086c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:12:44.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.291+0000 7ff1effff700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.40:3300/0 conn(0x7ff1d4006220 0x7ff1d40086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:12:44.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.291+0000 7ff1f48ed700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 0x7ff1d4005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:44.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.291+0000 7ff1f48ed700 1 -- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 msgr2=0x7ff1d4005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:12:44.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.291+0000 7ff1f48ed700 1 -- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 msgr2=0x7ff1d4005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:12:44.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.291+0000 7ff1f48ed700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 0x7ff1d4005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:44.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.291+0000 7ff1f48ed700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 0x7ff1d4005bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:12:44.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.695+0000 7ff1effff700 1 -- 172.21.15.40:0/3501640454 >> v2:172.21.15.40:3300/0 conn(0x7ff1d4006220 msgr2=0x7ff1d40086c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:12:44.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.695+0000 7ff1effff700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.40:3300/0 conn(0x7ff1d4006220 0x7ff1d40086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:12:44.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.695+0000 7ff1f48ed700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 0x7ff1d4005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:44.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.695+0000 7ff1f48ed700 1 -- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 msgr2=0x7ff1d4005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:12:44.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.695+0000 7ff1f48ed700 1 -- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 msgr2=0x7ff1d4005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:12:44.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.695+0000 7ff1f48ed700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 0x7ff1d4005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:44.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:44.695+0000 7ff1f48ed700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 0x7ff1d4005bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:12:45.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:45.499+0000 7ff1effff700 1 -- 172.21.15.40:0/3501640454 >> v2:172.21.15.40:3300/0 conn(0x7ff1d4006220 msgr2=0x7ff1d40086c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:12:45.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:45.499+0000 7ff1effff700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.40:3300/0 conn(0x7ff1d4006220 0x7ff1d40086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:12:45.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:45.499+0000 7ff1f48ed700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 0x7ff1d4005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:45.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:45.499+0000 7ff1f48ed700 1 -- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 msgr2=0x7ff1d4005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:12:45.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:45.499+0000 7ff1f48ed700 1 -- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 msgr2=0x7ff1d4005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:12:45.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:45.499+0000 7ff1f48ed700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 0x7ff1d4005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:12:45.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:45.499+0000 7ff1f48ed700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 0x7ff1d4005bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:12:47.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1ee7fc700 1 -- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 msgr2=0x7ff1d4005bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:12:47.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1ee7fc700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003770 0x7ff1d4005bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1ee7fc700 1 -- 172.21.15.40:0/3501640454 >> v2:172.21.15.40:3300/0 conn(0x7ff1d4006220 msgr2=0x7ff1d40086c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:12:47.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1ee7fc700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.40:3300/0 conn(0x7ff1d4006220 0x7ff1d40086c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1f50ee700 1 -- 172.21.15.40:0/3501640454 reap_dead start 2024-09-13T18:12:47.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1ee7fc700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003bc0 0x7ff1d40066b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1ee7fc700 1 --2- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d4006bf0 0x7ff1d4006fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1ee7fc700 1 -- 172.21.15.40:0/3501640454 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f015a010 con 0x7ff1d4007510 2024-09-13T18:12:47.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1ee7fc700 1 -- 172.21.15.40:0/3501640454 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f0159d30 con 0x7ff1d4003bc0 2024-09-13T18:12:47.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1ee7fc700 1 -- 172.21.15.40:0/3501640454 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d4008d70 con 0x7ff1d4006bf0 2024-09-13T18:12:47.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1f50ee700 1 -- 172.21.15.40:0/3501640454 >> v1:172.21.15.40:6789/0 conn(0x7ff1d4007510 legacy=0x7ff1d4007970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:12:47.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1effff700 1 --2- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d4006bf0 0x7ff1d4006fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:47.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1f48ed700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003bc0 0x7ff1d40066b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:47.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1effff700 1 -- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003bc0 msgr2=0x7ff1d40066b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:12:47.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1effff700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003bc0 0x7ff1d40066b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1effff700 1 -- 172.21.15.40:0/3501640454 >> v1:172.21.15.40:6789/0 conn(0x7ff1d4007510 legacy=0x7ff1d4007970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:12:47.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1effff700 1 -- 172.21.15.40:0/3501640454 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f0159770 con 0x7ff1d4006bf0 2024-09-13T18:12:47.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1f48ed700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003bc0 0x7ff1d40066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:12:47.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1effff700 1 --2- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d4006bf0 0x7ff1d4006fd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff1f00cb0b0 tx=0x7ff1e400cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2681ff3da76e351 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:47.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1ef7fe700 1 -- 172.21.15.40:0/3501640454 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4004230 con 0x7ff1d4006bf0 2024-09-13T18:12:47.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.087+0000 7ff1ef7fe700 1 -- 172.21.15.40:0/3501640454 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e400c040 con 0x7ff1d4006bf0 2024-09-13T18:12:47.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.091+0000 7ff1ef7fe700 1 -- 172.21.15.40:0/3501640454 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e40045d0 con 0x7ff1d4006bf0 2024-09-13T18:12:47.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.091+0000 7ff1f58ef700 1 -- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d4006bf0 msgr2=0x7ff1d4006fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:47.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.091+0000 7ff1f58ef700 1 --2- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d4006bf0 0x7ff1d4006fd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff1f00cb0b0 tx=0x7ff1e400cd80 comp rx=0 tx=0).stop 2024-09-13T18:12:47.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.091+0000 7ff1f58ef700 1 -- 172.21.15.40:0/3501640454 shutdown_connections 2024-09-13T18:12:47.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.091+0000 7ff1f58ef700 1 --2- 172.21.15.40:0/3501640454 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d4006bf0 0x7ff1d4006fd0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.091+0000 7ff1f58ef700 1 --2- 172.21.15.40:0/3501640454 >> v2:172.21.15.43:3300/0 conn(0x7ff1d4003bc0 0x7ff1d40066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.091+0000 7ff1f58ef700 1 -- 172.21.15.40:0/3501640454 >> 172.21.15.40:0/3501640454 conn(0x7ff1f006c5b0 msgr2=0x7ff1f006c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:47.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.091+0000 7ff1f58ef700 1 -- 172.21.15.40:0/3501640454 shutdown_connections 2024-09-13T18:12:47.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.091+0000 7ff1f58ef700 1 -- 172.21.15.40:0/3501640454 wait complete. 2024-09-13T18:12:47.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.095+0000 7ff1f58ef700 1 Processor -- start 2024-09-13T18:12:47.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.095+0000 7ff1f58ef700 1 -- start start 2024-09-13T18:12:47.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.095+0000 7ff1f58ef700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1d4003bc0 0x7ff1f01e1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.095+0000 7ff1f58ef700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d4006bf0 0x7ff1f01e6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.095+0000 7ff1f58ef700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d4007510 0x7ff1f01f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.095+0000 7ff1f58ef700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f00c8a00 con 0x7ff1d4003bc0 2024-09-13T18:12:47.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.095+0000 7ff1f58ef700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f01e2800 con 0x7ff1d4007510 2024-09-13T18:12:47.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.095+0000 7ff1f58ef700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f00c7420 con 0x7ff1d4006bf0 2024-09-13T18:12:47.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.095+0000 7ff1f48ed700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1d4003bc0 msgr2=0x7ff1f01e1ee0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:12:47.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.095+0000 7ff1f48ed700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1d4003bc0 0x7ff1f01e1ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:12:47.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.095+0000 7ff1effff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d4006bf0 0x7ff1f01e6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:47.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.095+0000 7ff1f50ee700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d4007510 0x7ff1f01f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:47.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.095+0000 7ff1effff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d4006bf0 0x7ff1f01e6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:37222/0 (socket says 172.21.15.40:37222) 2024-09-13T18:12:47.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.095+0000 7ff1f50ee700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d4007510 0x7ff1f01f03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:34482/0 (socket says 172.21.15.40:34482) 2024-09-13T18:12:47.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.095+0000 7ff1effff700 1 -- 172.21.15.40:0/4289128865 learned_addr learned my addr 172.21.15.40:0/4289128865 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:47.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.099+0000 7ff1effff700 1 -- 172.21.15.40:0/4289128865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d4007510 msgr2=0x7ff1f01f03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:47.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.099+0000 7ff1effff700 1 --2- 172.21.15.40:0/4289128865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d4007510 0x7ff1f01f03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.099+0000 7ff1effff700 1 -- 172.21.15.40:0/4289128865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1d4003bc0 msgr2=0x7ff1f01e1ee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:12:47.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.099+0000 7ff1effff700 1 --2- 172.21.15.40:0/4289128865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1d4003bc0 0x7ff1f01e1ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.099+0000 7ff1effff700 1 -- 172.21.15.40:0/4289128865 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc009ad0 con 0x7ff1d4006bf0 2024-09-13T18:12:47.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.099+0000 7ff1f50ee700 1 --2- 172.21.15.40:0/4289128865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d4007510 0x7ff1f01f03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:12:47.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.099+0000 7ff1effff700 1 --2- 172.21.15.40:0/4289128865 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d4006bf0 0x7ff1f01e6020 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400d010 tx=0x7ff1e4004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:47.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.099+0000 7ff1ed7fa700 1 -- 172.21.15.40:0/4289128865 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4026070 con 0x7ff1d4006bf0 2024-09-13T18:12:47.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.099+0000 7ff1ed7fa700 1 -- 172.21.15.40:0/4289128865 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e4014e50 con 0x7ff1d4006bf0 2024-09-13T18:12:47.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.099+0000 7ff1ed7fa700 1 -- 172.21.15.40:0/4289128865 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e40036c0 con 0x7ff1d4006bf0 2024-09-13T18:12:47.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.099+0000 7ff1f58ef700 1 -- 172.21.15.40:0/4289128865 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e400c700 con 0x7ff1d4006bf0 2024-09-13T18:12:47.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.099+0000 7ff1f58ef700 1 -- 172.21.15.40:0/4289128865 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f00c8690 con 0x7ff1d4006bf0 2024-09-13T18:12:47.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.099+0000 7ff1f58ef700 1 -- 172.21.15.40:0/4289128865 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f00c8820 con 0x7ff1d4006bf0 2024-09-13T18:12:47.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.103+0000 7ff1ed7fa700 1 -- 172.21.15.40:0/4289128865 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff1e4003820 con 0x7ff1d4006bf0 2024-09-13T18:12:47.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.103+0000 7ff1ed7fa700 1 --2- 172.21.15.40:0/4289128865 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff1d4068290 0x7ff1d406a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.103+0000 7ff1ed7fa700 1 -- 172.21.15.40:0/4289128865 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff1e4085d50 con 0x7ff1d4006bf0 2024-09-13T18:12:47.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.103+0000 7ff1f48ed700 1 --2- 172.21.15.40:0/4289128865 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff1d4068290 0x7ff1d406a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:47.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.103+0000 7ff1f48ed700 1 --2- 172.21.15.40:0/4289128865 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff1d4068290 0x7ff1d406a750 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc003a30 tx=0x7ff1dc00d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:47.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.111+0000 7ff1ed7fa700 1 -- 172.21.15.40:0/4289128865 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff1e4089030 con 0x7ff1d4006bf0 2024-09-13T18:12:47.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.351+0000 7ff1f58ef700 1 -- 172.21.15.40:0/4289128865 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff1f00c9ef0 con 0x7ff1d4006bf0 2024-09-13T18:12:47.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.351+0000 7ff1ed7fa700 1 -- 172.21.15.40:0/4289128865 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7ff1e401b610 con 0x7ff1d4006bf0 2024-09-13T18:12:47.360 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":178,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:12:47.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.355+0000 7ff1f58ef700 1 -- 172.21.15.40:0/4289128865 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff1d4068290 msgr2=0x7ff1d406a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:47.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.355+0000 7ff1f58ef700 1 --2- 172.21.15.40:0/4289128865 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff1d4068290 0x7ff1d406a750 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc003a30 tx=0x7ff1dc00d040 comp rx=0 tx=0).stop 2024-09-13T18:12:47.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.355+0000 7ff1f58ef700 1 -- 172.21.15.40:0/4289128865 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d4006bf0 msgr2=0x7ff1f01e6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:47.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.355+0000 7ff1f58ef700 1 --2- 172.21.15.40:0/4289128865 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d4006bf0 0x7ff1f01e6020 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400d010 tx=0x7ff1e4004770 comp rx=0 tx=0).stop 2024-09-13T18:12:47.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.355+0000 7ff1f58ef700 1 -- 172.21.15.40:0/4289128865 shutdown_connections 2024-09-13T18:12:47.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.355+0000 7ff1f58ef700 1 --2- 172.21.15.40:0/4289128865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d4007510 0x7ff1f01f03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.355+0000 7ff1f58ef700 1 --2- 172.21.15.40:0/4289128865 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff1d4068290 0x7ff1d406a750 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.355+0000 7ff1f58ef700 1 --2- 172.21.15.40:0/4289128865 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d4006bf0 0x7ff1f01e6020 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.355+0000 7ff1f58ef700 1 --2- 172.21.15.40:0/4289128865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1d4003bc0 0x7ff1f01e1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.355+0000 7ff1f58ef700 1 -- 172.21.15.40:0/4289128865 >> 172.21.15.40:0/4289128865 conn(0x7ff1f006c5b0 msgr2=0x7ff1f00cdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:47.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.355+0000 7ff1f58ef700 1 -- 172.21.15.40:0/4289128865 shutdown_connections 2024-09-13T18:12:47.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.355+0000 7ff1f58ef700 1 -- 172.21.15.40:0/4289128865 wait complete. 2024-09-13T18:12:47.375 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T18:12:47.375 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-13T18:12:47.375 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:12:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.487+0000 7fc3ba3d8700 1 Processor -- start 2024-09-13T18:12:47.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.487+0000 7fc3ba3d8700 1 -- start start 2024-09-13T18:12:47.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.487+0000 7fc3ba3d8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3b40c7f60 0x7fc3b40c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.487+0000 7fc3ba3d8700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc3b40c7250 0x7fc3b41577f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.487+0000 7fc3ba3d8700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b406b400 con 0x7fc3b40c8340 2024-09-13T18:12:47.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.487+0000 7fc3ba3d8700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b415e920 con 0x7fc3b40c7250 2024-09-13T18:12:47.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.487+0000 7fc3ba3d8700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40c76c0 con 0x7fc3b40c7f60 2024-09-13T18:12:47.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.487+0000 7fc3b93d6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3b40c7f60 0x7fc3b40c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:47.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.487+0000 7fc3b9bd7700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc3b40c7250 0x7fc3b41577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:47.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.487+0000 7fc3b93d6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3b40c7f60 0x7fc3b40c61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:37234/0 (socket says 172.21.15.40:37234) 2024-09-13T18:12:47.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.487+0000 7fc3b93d6700 1 -- 172.21.15.40:0/1917131060 learned_addr learned my addr 172.21.15.40:0/1917131060 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:47.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.487+0000 7fc3b93d6700 1 -- 172.21.15.40:0/1917131060 >> v2:172.21.15.43:3300/0 conn(0x7fc3b40c7250 msgr2=0x7fc3b41577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:12:47.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.487+0000 7fc3b93d6700 1 --2- 172.21.15.40:0/1917131060 >> v2:172.21.15.43:3300/0 conn(0x7fc3b40c7250 0x7fc3b41577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.487+0000 7fc3b93d6700 1 -- 172.21.15.40:0/1917131060 >> v1:172.21.15.43:6789/0 conn(0x7fc3b40c8340 legacy=0x7fc3b406d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:47.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.487+0000 7fc3b93d6700 1 -- 172.21.15.40:0/1917131060 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b415dda0 con 0x7fc3b40c7f60 2024-09-13T18:12:47.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.487+0000 7fc3b93d6700 1 --2- 172.21.15.40:0/1917131060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3b40c7f60 0x7fc3b40c61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009910 tx=0x7fc39c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b45dfb123abe208 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:47.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.487+0000 7fc3b3fff700 1 -- 172.21.15.40:0/1917131060 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c011830 con 0x7fc3b40c7f60 2024-09-13T18:12:47.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.487+0000 7fc3b3fff700 1 -- 172.21.15.40:0/1917131060 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39c011990 con 0x7fc3b40c7f60 2024-09-13T18:12:47.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.487+0000 7fc3b3fff700 1 -- 172.21.15.40:0/1917131060 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c011d40 con 0x7fc3b40c7f60 2024-09-13T18:12:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3ba3d8700 1 -- 172.21.15.40:0/1917131060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3b40c7f60 msgr2=0x7fc3b40c61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3ba3d8700 1 --2- 172.21.15.40:0/1917131060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3b40c7f60 0x7fc3b40c61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009910 tx=0x7fc39c009c20 comp rx=0 tx=0).stop 2024-09-13T18:12:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3ba3d8700 1 -- 172.21.15.40:0/1917131060 shutdown_connections 2024-09-13T18:12:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3ba3d8700 1 --2- 172.21.15.40:0/1917131060 >> v2:172.21.15.43:3300/0 conn(0x7fc3b40c7250 0x7fc3b41577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3ba3d8700 1 --2- 172.21.15.40:0/1917131060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3b40c7f60 0x7fc3b40c61e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3ba3d8700 1 -- 172.21.15.40:0/1917131060 >> 172.21.15.40:0/1917131060 conn(0x7fc3b4066090 msgr2=0x7fc3b40664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:47.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3ba3d8700 1 -- 172.21.15.40:0/1917131060 shutdown_connections 2024-09-13T18:12:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3ba3d8700 1 -- 172.21.15.40:0/1917131060 wait complete. 2024-09-13T18:12:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3ba3d8700 1 Processor -- start 2024-09-13T18:12:47.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3ba3d8700 1 -- start start 2024-09-13T18:12:47.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3ba3d8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b40c7250 0x7fc3b41f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3ba3d8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3b40c7f60 0x7fc3b40c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3ba3d8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b40c8340 0x7fc3b415cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3ba3d8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b415b100 con 0x7fc3b40c8340 2024-09-13T18:12:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3ba3d8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b41f6600 con 0x7fc3b40c7250 2024-09-13T18:12:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3ba3d8700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b41f7590 con 0x7fc3b40c7f60 2024-09-13T18:12:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3b9bd7700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b40c8340 msgr2=0x7fc3b415cbc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:12:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3b9bd7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b40c8340 0x7fc3b415cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:12:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3b8bd5700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3b40c7f60 0x7fc3b40c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3b93d6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b40c7250 0x7fc3b41f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3b8bd5700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3b40c7f60 0x7fc3b40c8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:37248/0 (socket says 172.21.15.40:37248) 2024-09-13T18:12:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.491+0000 7fc3b8bd5700 1 -- 172.21.15.40:0/4002679683 learned_addr learned my addr 172.21.15.40:0/4002679683 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.495+0000 7fc3b8bd5700 1 -- 172.21.15.40:0/4002679683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b40c7250 msgr2=0x7fc3b41f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.495+0000 7fc3b8bd5700 1 --2- 172.21.15.40:0/4002679683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b40c7250 0x7fc3b41f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.495+0000 7fc3b8bd5700 1 -- 172.21.15.40:0/4002679683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b40c8340 msgr2=0x7fc3b415cbc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:12:47.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.495+0000 7fc3b8bd5700 1 --2- 172.21.15.40:0/4002679683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3b40c8340 0x7fc3b415cbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.495+0000 7fc3b8bd5700 1 -- 172.21.15.40:0/4002679683 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39c009570 con 0x7fc3b40c7f60 2024-09-13T18:12:47.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.495+0000 7fc3b8bd5700 1 --2- 172.21.15.40:0/4002679683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3b40c7f60 0x7fc3b40c8cb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc3a400cba0 tx=0x7fc3a400ceb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.495+0000 7fc3b1ffb700 1 -- 172.21.15.40:0/4002679683 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4003c20 con 0x7fc3b40c7f60 2024-09-13T18:12:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.495+0000 7fc3b1ffb700 1 -- 172.21.15.40:0/4002679683 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a4003d80 con 0x7fc3b40c7f60 2024-09-13T18:12:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.495+0000 7fc3b1ffb700 1 -- 172.21.15.40:0/4002679683 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a400ab10 con 0x7fc3b40c7f60 2024-09-13T18:12:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.495+0000 7fc3ba3d8700 1 -- 172.21.15.40:0/4002679683 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b41f3b30 con 0x7fc3b40c7f60 2024-09-13T18:12:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.495+0000 7fc3ba3d8700 1 -- 172.21.15.40:0/4002679683 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b41f6850 con 0x7fc3b40c7f60 2024-09-13T18:12:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.495+0000 7fc3ba3d8700 1 --2- 172.21.15.40:0/4002679683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b41f6a70 0x7fc3b4151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.495+0000 7fc3ba3d8700 1 -- 172.21.15.40:0/4002679683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b4067720 con 0x7fc3b41f6a70 2024-09-13T18:12:47.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.495+0000 7fc3b93d6700 1 --2- 172.21.15.40:0/4002679683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b41f6a70 0x7fc3b4151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:47.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.495+0000 7fc3b93d6700 1 -- 172.21.15.40:0/4002679683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc3b415e640 con 0x7fc3b41f6a70 2024-09-13T18:12:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.495+0000 7fc3b93d6700 1 --2- 172.21.15.40:0/4002679683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b41f6a70 0x7fc3b4151020 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009910 tx=0x7fc39c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.495+0000 7fc3b1ffb700 1 -- 172.21.15.40:0/4002679683 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc3a400ac70 con 0x7fc3b40c7f60 2024-09-13T18:12:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.499+0000 7fc3b1ffb700 1 --2- 172.21.15.40:0/4002679683 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc394066710 0x7fc394068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.499+0000 7fc3b1ffb700 1 -- 172.21.15.40:0/4002679683 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc3a4084520 con 0x7fc3b40c7f60 2024-09-13T18:12:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.499+0000 7fc3b9bd7700 1 --2- 172.21.15.40:0/4002679683 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc394066710 0x7fc394068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.499+0000 7fc3b1ffb700 1 -- 172.21.15.40:0/4002679683 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c020070 con 0x7fc3b41f6a70 2024-09-13T18:12:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.499+0000 7fc3b1ffb700 1 -- 172.21.15.40:0/4002679683 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc39c0044f0 con 0x7fc3b41f6a70 2024-09-13T18:12:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.499+0000 7fc3b1ffb700 1 -- 172.21.15.40:0/4002679683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b41f6a70 msgr2=0x7fc3b4151020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.499+0000 7fc3b1ffb700 1 --2- 172.21.15.40:0/4002679683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b41f6a70 0x7fc3b4151020 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009910 tx=0x7fc39c004770 comp rx=0 tx=0).stop 2024-09-13T18:12:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.499+0000 7fc3b1ffb700 1 -- 172.21.15.40:0/4002679683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b41f6a70 msgr2=0x7fc3b4151020 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:12:47.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.499+0000 7fc3b1ffb700 1 --2- 172.21.15.40:0/4002679683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b41f6a70 0x7fc3b4151020 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.499+0000 7fc3b9bd7700 1 --2- 172.21.15.40:0/4002679683 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc394066710 0x7fc394068bd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0095a0 tx=0x7fc3ac009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:47.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.507+0000 7fc3ba3d8700 1 --2- 172.21.15.40:0/4002679683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b40c3760 0x7fc3b4151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.507+0000 7fc3ba3d8700 1 -- 172.21.15.40:0/4002679683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b41f6e50 con 0x7fc3b40c3760 2024-09-13T18:12:47.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.507+0000 7fc3b93d6700 1 --2- 172.21.15.40:0/4002679683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b40c3760 0x7fc3b4151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:47.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.507+0000 7fc3b93d6700 1 -- 172.21.15.40:0/4002679683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc3b415e360 con 0x7fc3b40c3760 2024-09-13T18:12:47.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.507+0000 7fc3b93d6700 1 --2- 172.21.15.40:0/4002679683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b40c3760 0x7fc3b4151560 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009150 tx=0x7fc39c011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:47.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.507+0000 7fc3b1ffb700 1 -- 172.21.15.40:0/4002679683 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c01ca50 con 0x7fc3b40c3760 2024-09-13T18:12:47.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.507+0000 7fc3b1ffb700 1 -- 172.21.15.40:0/4002679683 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4773 (secure 0 0 0) 0x7fc39c02f070 con 0x7fc3b40c3760 2024-09-13T18:12:47.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.507+0000 7fc3b1ffb700 1 -- 172.21.15.40:0/4002679683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b40c3760 msgr2=0x7fc3b4151560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:47.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.507+0000 7fc3b1ffb700 1 --2- 172.21.15.40:0/4002679683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b40c3760 0x7fc3b4151560 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009150 tx=0x7fc39c011000 comp rx=0 tx=0).stop 2024-09-13T18:12:47.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.507+0000 7fc3b1ffb700 1 -- 172.21.15.40:0/4002679683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b40c3760 msgr2=0x7fc3b4151560 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:12:47.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.507+0000 7fc3b1ffb700 1 --2- 172.21.15.40:0/4002679683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b40c3760 0x7fc3b4151560 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.518 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:12:47.518 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:12:47.518 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:12:47.518 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:12:47.518 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 178, 2024-09-13T18:12:47.518 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:12:47.518 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:12:47.518 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:12:47.519 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:47.519 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:12:47.519 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:12:47.519 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:12:47.519 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:12:47.519 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:47.519 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:47.519 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:47.519 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:47.519 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:47.519 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:47.519 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:47.519 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:47.520 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:47.520 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:47.520 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:47.520 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:12:47.520 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:12:47.520 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:47.520 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:47.520 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:47.520 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:47.520 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:47.520 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:47.520 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:47.520 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:47.520 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:47.521 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:47.521 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:47.521 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.521 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:12:47.521 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:12:47.521 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:12:47.521 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:12:47.521 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:12:47.521 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:12:47.521 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:12:47.521 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:12:47.521 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:12:47.522 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:12:47.522 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:12:47.522 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:12:47.522 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:12:47.522 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:12:47.522 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:12:47.522 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:12:47.522 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:12:47.522 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:47.522 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:47.522 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:47.522 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:47.522 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:47.523 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:47.523 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:47.523 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:47.523 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:47.523 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:47.523 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:47.523 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:12:47.523 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.523 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:12:47.523 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.523 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:12:47.523 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:12:47.523 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:47.524 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:47.524 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.524 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:47.524 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:12:47.524 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:47.524 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.524 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.524 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:47.524 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:12:47.524 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:47.524 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:47.524 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:47.524 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.524 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:12:47.525 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:12:47.525 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:47.525 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:47.525 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:47.525 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.525 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.525 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:12:47.525 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:12:47.525 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:47.525 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:47.525 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.525 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:47.525 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:12:47.526 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:47.526 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.526 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.526 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:47.526 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:12:47.526 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:47.526 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:47.526 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:47.526 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.526 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:12:47.526 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:12:47.526 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:47.526 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:47.526 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:47.527 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.527 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.527 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:12:47.527 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:12:47.527 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:47.527 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:47.527 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.527 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:47.527 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:12:47.527 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:47.527 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.527 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.527 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:47.527 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:12:47.528 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:47.528 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:47.528 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:47.528 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.528 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:12:47.528 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:12:47.528 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:47.528 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:47.528 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:47.528 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:47.528 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:47.528 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.528 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:12:47.528 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:12:47.528 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.529 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:47.529 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:47.529 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:12:47.529 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:47.529 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:47.529 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:12:47.529 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.529 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:47.529 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:47.529 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:12:47.529 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:47.529 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:47.530 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:12:47.530 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.530 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:47.530 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:47.530 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:12:47.530 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:47.530 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:47.530 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:12:47.530 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.530 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:47.530 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:47.530 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:12:47.530 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:47.531 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:47.531 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.531 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:12:47.531 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:12:47.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.507+0000 7fc3ba3d8700 1 -- 172.21.15.40:0/4002679683 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc394066710 msgr2=0x7fc394068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:47.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.507+0000 7fc3ba3d8700 1 --2- 172.21.15.40:0/4002679683 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc394066710 0x7fc394068bd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0095a0 tx=0x7fc3ac009240 comp rx=0 tx=0).stop 2024-09-13T18:12:47.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.507+0000 7fc3ba3d8700 1 -- 172.21.15.40:0/4002679683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3b40c7f60 msgr2=0x7fc3b40c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:47.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.507+0000 7fc3ba3d8700 1 --2- 172.21.15.40:0/4002679683 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3b40c7f60 0x7fc3b40c8cb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc3a400cba0 tx=0x7fc3a400ceb0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.507+0000 7fc3b9bd7700 1 -- 172.21.15.40:0/4002679683 reap_dead start 2024-09-13T18:12:47.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.507+0000 7fc3b9bd7700 1 -- 172.21.15.40:0/4002679683 reap_dead start 2024-09-13T18:12:47.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.507+0000 7fc3ba3d8700 1 -- 172.21.15.40:0/4002679683 shutdown_connections 2024-09-13T18:12:47.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.507+0000 7fc3ba3d8700 1 -- 172.21.15.40:0/4002679683 >> 172.21.15.40:0/4002679683 conn(0x7fc3b4066090 msgr2=0x7fc3b40735e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:47.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.511+0000 7fc3ba3d8700 1 -- 172.21.15.40:0/4002679683 shutdown_connections 2024-09-13T18:12:47.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.511+0000 7fc3ba3d8700 1 -- 172.21.15.40:0/4002679683 wait complete. 2024-09-13T18:12:47.533 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:12:47.533 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T18:12:47.533 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:12:47.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724e87b700 1 Processor -- start 2024-09-13T18:12:47.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724e87b700 1 -- start start 2024-09-13T18:12:47.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724e87b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f724814fdd0 0x7f72480c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724e87b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f72480c0a10 0x7f72480c0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724e87b700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7248159eb0 con 0x7f72480c72e0 2024-09-13T18:12:47.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724e87b700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72480c76e0 con 0x7f72480c0a10 2024-09-13T18:12:47.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724e87b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72480c13e0 con 0x7f724814fdd0 2024-09-13T18:12:47.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724d879700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f724814fdd0 0x7f72480c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:47.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724e07a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f72480c0a10 0x7f72480c0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:47.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724d879700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f724814fdd0 0x7f72480c6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:37252/0 (socket says 172.21.15.40:37252) 2024-09-13T18:12:47.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724d879700 1 -- 172.21.15.40:0/1025574862 learned_addr learned my addr 172.21.15.40:0/1025574862 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:47.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724d879700 1 -- 172.21.15.40:0/1025574862 >> v2:172.21.15.43:3300/0 conn(0x7f72480c0a10 msgr2=0x7f72480c0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:12:47.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724d879700 1 --2- 172.21.15.40:0/1025574862 >> v2:172.21.15.43:3300/0 conn(0x7f72480c0a10 0x7f72480c0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724d879700 1 -- 172.21.15.40:0/1025574862 >> v1:172.21.15.43:6789/0 conn(0x7f72480c72e0 legacy=0x7f72480c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:47.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724d879700 1 -- 172.21.15.40:0/1025574862 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7248158fc0 con 0x7f724814fdd0 2024-09-13T18:12:47.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724c877700 1 -- 172.21.15.40:0/1025574862 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749579587 0 0) 0x7f7248159eb0 con 0x7f72480c72e0 2024-09-13T18:12:47.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724d879700 1 --2- 172.21.15.40:0/1025574862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f724814fdd0 0x7f72480c6da0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7244009910 tx=0x7f7244009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35eb534181a1500a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:47.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724c877700 1 -- 172.21.15.40:0/1025574862 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7244011830 con 0x7f724814fdd0 2024-09-13T18:12:47.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724c877700 1 -- 172.21.15.40:0/1025574862 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7244011990 con 0x7f724814fdd0 2024-09-13T18:12:47.654 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724c877700 1 -- 172.21.15.40:0/1025574862 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7244011d40 con 0x7f724814fdd0 2024-09-13T18:12:47.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724e87b700 1 -- 172.21.15.40:0/1025574862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f724814fdd0 msgr2=0x7f72480c6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:47.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724e87b700 1 --2- 172.21.15.40:0/1025574862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f724814fdd0 0x7f72480c6da0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7244009910 tx=0x7f7244009c20 comp rx=0 tx=0).stop 2024-09-13T18:12:47.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724e87b700 1 -- 172.21.15.40:0/1025574862 shutdown_connections 2024-09-13T18:12:47.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724e87b700 1 --2- 172.21.15.40:0/1025574862 >> v2:172.21.15.43:3300/0 conn(0x7f72480c0a10 0x7f72480c0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724e87b700 1 --2- 172.21.15.40:0/1025574862 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f724814fdd0 0x7f72480c6da0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.643+0000 7f724e87b700 1 -- 172.21.15.40:0/1025574862 >> 172.21.15.40:0/1025574862 conn(0x7f7248066090 msgr2=0x7f72480664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:47.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724e87b700 1 -- 172.21.15.40:0/1025574862 shutdown_connections 2024-09-13T18:12:47.655 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724e87b700 1 -- 172.21.15.40:0/1025574862 wait complete. 2024-09-13T18:12:47.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724e87b700 1 Processor -- start 2024-09-13T18:12:47.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724e87b700 1 -- start start 2024-09-13T18:12:47.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724e87b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72480c0a10 0x7f7248071b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724e87b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72480c72e0 0x7f72480c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724e87b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f724814fdd0 0x7f72480c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724e87b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72480c2410 con 0x7f72480c0a10 2024-09-13T18:12:47.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724e87b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72481403e0 con 0x7f72480c72e0 2024-09-13T18:12:47.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724e87b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72480cbc90 con 0x7f724814fdd0 2024-09-13T18:12:47.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724d879700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72480c0a10 msgr2=0x7f7248071b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:12:47.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724d879700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72480c0a10 0x7f7248071b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:12:47.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724e07a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f724814fdd0 0x7f72480c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:47.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724d078700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72480c72e0 0x7f72480c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:47.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724e07a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f724814fdd0 0x7f72480c9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:37258/0 (socket says 172.21.15.40:37258) 2024-09-13T18:12:47.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724e07a700 1 -- 172.21.15.40:0/3227678257 learned_addr learned my addr 172.21.15.40:0/3227678257 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:12:47.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724e07a700 1 -- 172.21.15.40:0/3227678257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72480c72e0 msgr2=0x7f72480c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:47.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724e07a700 1 --2- 172.21.15.40:0/3227678257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72480c72e0 0x7f72480c91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724e07a700 1 -- 172.21.15.40:0/3227678257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72480c0a10 msgr2=0x7f7248071b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:12:47.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724e07a700 1 --2- 172.21.15.40:0/3227678257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f72480c0a10 0x7f7248071b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.647+0000 7f724e07a700 1 -- 172.21.15.40:0/3227678257 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7244009570 con 0x7f724814fdd0 2024-09-13T18:12:47.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.651+0000 7f724e07a700 1 --2- 172.21.15.40:0/3227678257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f724814fdd0 0x7f72480c9710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f724000efc0 tx=0x7f724000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:47.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.651+0000 7f723e7fc700 1 -- 172.21.15.40:0/3227678257 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7240009ea0 con 0x7f724814fdd0 2024-09-13T18:12:47.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.651+0000 7f723e7fc700 1 -- 172.21.15.40:0/3227678257 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724000f040 con 0x7f724814fdd0 2024-09-13T18:12:47.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.651+0000 7f723e7fc700 1 -- 172.21.15.40:0/3227678257 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7240014480 con 0x7f724814fdd0 2024-09-13T18:12:47.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.651+0000 7f724e87b700 1 -- 172.21.15.40:0/3227678257 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72480ca990 con 0x7f724814fdd0 2024-09-13T18:12:47.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.651+0000 7f724e87b700 1 -- 172.21.15.40:0/3227678257 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7248140670 con 0x7f724814fdd0 2024-09-13T18:12:47.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.651+0000 7f724e87b700 1 --2- 172.21.15.40:0/3227678257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72480c8cb0 0x7f72481f07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.651+0000 7f724e87b700 1 -- 172.21.15.40:0/3227678257 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7248067720 con 0x7f72480c8cb0 2024-09-13T18:12:47.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.651+0000 7f724d879700 1 --2- 172.21.15.40:0/3227678257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72480c8cb0 0x7f72481f07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:47.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.651+0000 7f724d879700 1 -- 172.21.15.40:0/3227678257 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7248159860 con 0x7f72480c8cb0 2024-09-13T18:12:47.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.651+0000 7f724d879700 1 --2- 172.21.15.40:0/3227678257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72480c8cb0 0x7f72481f07f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7244009c60 tx=0x7f7244004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:47.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.651+0000 7f723e7fc700 1 -- 172.21.15.40:0/3227678257 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f72400145e0 con 0x7f724814fdd0 2024-09-13T18:12:47.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.655+0000 7f723e7fc700 1 --2- 172.21.15.40:0/3227678257 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f722c0666e0 0x7f722c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.655+0000 7f723e7fc700 1 -- 172.21.15.40:0/3227678257 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7240085090 con 0x7f724814fdd0 2024-09-13T18:12:47.663 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.655+0000 7f724d078700 1 --2- 172.21.15.40:0/3227678257 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f722c0666e0 0x7f722c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:47.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.655+0000 7f723e7fc700 1 -- 172.21.15.40:0/3227678257 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724401a830 con 0x7f72480c8cb0 2024-09-13T18:12:47.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.655+0000 7f723e7fc700 1 -- 172.21.15.40:0/3227678257 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f724401a990 con 0x7f72480c8cb0 2024-09-13T18:12:47.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.655+0000 7f723e7fc700 1 -- 172.21.15.40:0/3227678257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72480c8cb0 msgr2=0x7f72481f07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:47.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.655+0000 7f723e7fc700 1 --2- 172.21.15.40:0/3227678257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72480c8cb0 0x7f72481f07f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7244009c60 tx=0x7f7244004770 comp rx=0 tx=0).stop 2024-09-13T18:12:47.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.655+0000 7f723e7fc700 1 -- 172.21.15.40:0/3227678257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72480c8cb0 msgr2=0x7f72481f07f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:12:47.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.655+0000 7f723e7fc700 1 --2- 172.21.15.40:0/3227678257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72480c8cb0 0x7f72481f07f0 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7244009c60 tx=0x7f7244004770 comp rx=0 tx=0).stop 2024-09-13T18:12:47.664 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.655+0000 7f724d078700 1 --2- 172.21.15.40:0/3227678257 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f722c0666e0 0x7f722c068ba0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7248158eb0 tx=0x7f7238009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:47.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.667+0000 7f724e87b700 1 --2- 172.21.15.40:0/3227678257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72480cb0c0 0x7f72481f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:12:47.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.667+0000 7f724e87b700 1 -- 172.21.15.40:0/3227678257 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7248140520 con 0x7f72480cb0c0 2024-09-13T18:12:47.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.667+0000 7f724d879700 1 --2- 172.21.15.40:0/3227678257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72480cb0c0 0x7f72481f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:12:47.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.667+0000 7f724d879700 1 -- 172.21.15.40:0/3227678257 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7248159580 con 0x7f72480cb0c0 2024-09-13T18:12:47.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.667+0000 7f724d879700 1 --2- 172.21.15.40:0/3227678257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72480cb0c0 0x7f72481f0d30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7244009150 tx=0x7f72440092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:12:47.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.667+0000 7f723e7fc700 1 -- 172.21.15.40:0/3227678257 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724402a580 con 0x7f72480cb0c0 2024-09-13T18:12:47.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.671+0000 7f723e7fc700 1 -- 172.21.15.40:0/3227678257 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4937 (secure 0 0 0) 0x7f724402a6e0 con 0x7f72480cb0c0 2024-09-13T18:12:47.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.671+0000 7f723e7fc700 1 -- 172.21.15.40:0/3227678257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72480cb0c0 msgr2=0x7f72481f0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:47.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.671+0000 7f723e7fc700 1 --2- 172.21.15.40:0/3227678257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72480cb0c0 0x7f72481f0d30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7244009150 tx=0x7f72440092e0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.671+0000 7f723e7fc700 1 -- 172.21.15.40:0/3227678257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72480cb0c0 msgr2=0x7f72481f0d30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:12:47.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.671+0000 7f723e7fc700 1 --2- 172.21.15.40:0/3227678257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f72480cb0c0 0x7f72481f0d30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.679 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:12:47.679 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:12:47.679 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:12:47.679 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:12:47.679 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 178, 2024-09-13T18:12:47.679 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:12:47.679 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:12:47.679 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:12:47.679 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:47.680 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:12:47.680 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:12:47.680 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:12:47.680 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:12:47.680 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:47.680 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:47.680 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:47.680 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:47.680 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:47.680 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:47.680 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:47.680 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:47.680 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:47.681 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:47.681 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:47.681 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:12:47.681 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:12:47.681 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:47.681 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:47.681 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:47.681 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:47.681 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:47.681 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:47.681 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:47.681 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:47.681 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:47.681 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:47.682 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:47.682 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.682 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:12:47.682 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:12:47.682 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:12:47.682 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:12:47.682 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:12:47.682 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:12:47.682 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:12:47.682 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:12:47.682 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:12:47.682 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:12:47.682 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:12:47.682 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:12:47.683 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:12:47.683 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:12:47.683 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:12:47.683 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:12:47.683 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:12:47.683 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:12:47.683 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:12:47.683 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:12:47.683 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:12:47.683 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:12:47.683 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:12:47.683 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:12:47.683 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:12:47.684 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:12:47.684 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:12:47.684 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:47.684 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:12:47.684 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.684 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:12:47.684 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.684 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:12:47.684 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:12:47.684 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:47.684 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:47.684 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.684 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:47.684 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:12:47.685 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:47.685 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.685 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.685 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:47.685 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:12:47.685 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:47.685 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:47.685 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:47.685 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.686 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:12:47.686 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:12:47.686 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:47.686 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:47.686 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:47.686 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.686 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.686 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:12:47.686 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:12:47.686 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:47.686 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:47.686 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.686 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:47.687 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:12:47.687 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:47.687 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.687 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.687 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:47.687 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:12:47.687 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:47.687 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:47.687 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:47.687 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.687 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:12:47.687 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:12:47.687 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:47.688 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:47.688 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:47.688 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.688 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.688 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:12:47.688 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:12:47.688 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:12:47.688 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:12:47.688 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.688 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:12:47.688 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:12:47.688 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:47.688 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.688 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.689 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:12:47.689 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:12:47.689 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:12:47.689 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:47.689 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:47.689 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.689 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:12:47.689 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:12:47.689 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:12:47.689 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:12:47.689 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:12:47.689 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:47.689 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:47.690 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.690 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:12:47.690 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:12:47.690 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.690 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:47.690 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:47.691 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:12:47.691 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:47.691 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:47.691 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:12:47.691 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.691 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:47.691 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:47.691 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:12:47.691 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:47.691 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:47.692 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:12:47.692 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.692 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:47.692 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:47.692 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:12:47.692 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:47.692 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:47.692 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:12:47.692 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.692 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:47.692 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:47.692 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:12:47.692 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:47.693 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:12:47.693 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:12:47.693 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:12:47.693 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:12:47.693 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:12:47.693 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:12:47.693 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:12:47.693 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:12:47.693 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:12:47.693 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:12:47.693 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:12:47.694 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.671+0000 7f724e87b700 1 -- 172.21.15.40:0/3227678257 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f722c0666e0 msgr2=0x7f722c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:47.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.671+0000 7f724e87b700 1 --2- 172.21.15.40:0/3227678257 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f722c0666e0 0x7f722c068ba0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7248158eb0 tx=0x7f7238009040 comp rx=0 tx=0).stop 2024-09-13T18:12:47.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.671+0000 7f724e87b700 1 -- 172.21.15.40:0/3227678257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f724814fdd0 msgr2=0x7f72480c9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:12:47.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.671+0000 7f724e07a700 1 -- 172.21.15.40:0/3227678257 reap_dead start 2024-09-13T18:12:47.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.671+0000 7f724e87b700 1 --2- 172.21.15.40:0/3227678257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f724814fdd0 0x7f72480c9710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f724000efc0 tx=0x7f724000c3a0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.671+0000 7f724e87b700 1 -- 172.21.15.40:0/3227678257 shutdown_connections 2024-09-13T18:12:47.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.671+0000 7f724e87b700 1 --2- 172.21.15.40:0/3227678257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f724814fdd0 0x7f72480c9710 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:12:47.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.671+0000 7f724e87b700 1 -- 172.21.15.40:0/3227678257 >> 172.21.15.40:0/3227678257 conn(0x7f7248066090 msgr2=0x7f724814ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:12:47.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.671+0000 7f724e87b700 1 -- 172.21.15.40:0/3227678257 shutdown_connections 2024-09-13T18:12:47.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:12:47.671+0000 7f724e87b700 1 -- 172.21.15.40:0/3227678257 wait complete. 2024-09-13T18:12:47.695 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:12:47.696 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:13:07.697 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T18:13:07.697 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-13T18:13:07.697 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T18:13:07.697 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T18:13:07.700 INFO:tasks.ceph.mon.a:Started 2024-09-13T18:13:07.700 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:13:07.700 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:13:07.720 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:13:07.747 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:13:07.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.887+0000 7f60f5c62700 1 Processor -- start 2024-09-13T18:13:07.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.887+0000 7f60f5c62700 1 -- start start 2024-09-13T18:13:07.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60f5c62700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60f0163b20 0x7f60f00c07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:07.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60f5c62700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f60f00c57d0 0x7f60f00c0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:07.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60f5c62700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60f00c5d90 con 0x7f60f0166890 2024-09-13T18:13:07.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60f5c62700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f00c1aa0 con 0x7f60f00c57d0 2024-09-13T18:13:07.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60f5c62700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f006d640 con 0x7f60f0163b20 2024-09-13T18:13:07.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60eeffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f60f00c57d0 0x7f60f00c0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:07.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60eeffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f60f00c57d0 0x7f60f00c0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:42022/0 (socket says 172.21.15.40:42022) 2024-09-13T18:13:07.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60eeffd700 1 -- 172.21.15.40:0/841692446 learned_addr learned my addr 172.21.15.40:0/841692446 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:07.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60ef7fe700 1 --2- 172.21.15.40:0/841692446 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60f0163b20 0x7f60f00c07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:07.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60eeffd700 1 -- 172.21.15.40:0/841692446 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60f0163b20 msgr2=0x7f60f00c07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:13:07.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60eeffd700 1 --2- 172.21.15.40:0/841692446 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60f0163b20 0x7f60f00c07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:07.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60eeffd700 1 -- 172.21.15.40:0/841692446 >> v1:172.21.15.43:6789/0 conn(0x7f60f0166890 legacy=0x7f60f006a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:07.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60eeffd700 1 -- 172.21.15.40:0/841692446 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f0163f00 con 0x7f60f00c57d0 2024-09-13T18:13:07.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60ee7fc700 1 -- 172.21.15.40:0/841692446 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 31995258 0 0) 0x7f60f00c5d90 con 0x7f60f0166890 2024-09-13T18:13:07.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60eeffd700 1 --2- 172.21.15.40:0/841692446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60f00c57d0 0x7f60f00c0cf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f60d8009910 tx=0x7f60d8009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c644dd508d3e3f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:07.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60ee7fc700 1 -- 172.21.15.40:0/841692446 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d8011830 con 0x7f60f00c57d0 2024-09-13T18:13:07.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60ee7fc700 1 -- 172.21.15.40:0/841692446 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d8011990 con 0x7f60f00c57d0 2024-09-13T18:13:07.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60ee7fc700 1 -- 172.21.15.40:0/841692446 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d8011d40 con 0x7f60f00c57d0 2024-09-13T18:13:07.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60f5c62700 1 -- 172.21.15.40:0/841692446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60f00c57d0 msgr2=0x7f60f00c0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:07.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60f5c62700 1 --2- 172.21.15.40:0/841692446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60f00c57d0 0x7f60f00c0cf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f60d8009910 tx=0x7f60d8009c20 comp rx=0 tx=0).stop 2024-09-13T18:13:07.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60f5c62700 1 -- 172.21.15.40:0/841692446 shutdown_connections 2024-09-13T18:13:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60f5c62700 1 --2- 172.21.15.40:0/841692446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60f00c57d0 0x7f60f00c0cf0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60f5c62700 1 --2- 172.21.15.40:0/841692446 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60f0163b20 0x7f60f00c07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60f5c62700 1 -- 172.21.15.40:0/841692446 >> 172.21.15.40:0/841692446 conn(0x7f60f00cadd0 msgr2=0x7f60f00cb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60f5c62700 1 -- 172.21.15.40:0/841692446 shutdown_connections 2024-09-13T18:13:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.891+0000 7f60f5c62700 1 -- 172.21.15.40:0/841692446 wait complete. 2024-09-13T18:13:07.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.895+0000 7f60f5c62700 1 Processor -- start 2024-09-13T18:13:07.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.895+0000 7f60f5c62700 1 -- start start 2024-09-13T18:13:07.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.895+0000 7f60f5c62700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60f00c57d0 0x7f60f00704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:07.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.895+0000 7f60f5c62700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60f0163b20 0x7f60f00710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:07.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.895+0000 7f60f5c62700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60f0166890 0x7f60f0071630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:07.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.895+0000 7f60f5c62700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f0072150 con 0x7f60f00c57d0 2024-09-13T18:13:07.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.895+0000 7f60f5c62700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f00723b0 con 0x7f60f0166890 2024-09-13T18:13:07.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.895+0000 7f60f5c62700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f0203c30 con 0x7f60f0163b20 2024-09-13T18:13:07.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.895+0000 7f60ef7fe700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60f00c57d0 msgr2=0x7f60f00704d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:13:07.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.895+0000 7f60ef7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60f00c57d0 0x7f60f00704d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:07.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.895+0000 7f60eeffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60f0163b20 0x7f60f00710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:07.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.895+0000 7f60effff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60f0166890 0x7f60f0071630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:07.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.895+0000 7f60eeffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60f0163b20 0x7f60f00710f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:48182/0 (socket says 172.21.15.40:48182) 2024-09-13T18:13:07.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.895+0000 7f60eeffd700 1 -- 172.21.15.40:0/104169508 learned_addr learned my addr 172.21.15.40:0/104169508 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:07.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.899+0000 7f60eeffd700 1 -- 172.21.15.40:0/104169508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60f0166890 msgr2=0x7f60f0071630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:07.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.899+0000 7f60eeffd700 1 --2- 172.21.15.40:0/104169508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60f0166890 0x7f60f0071630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:07.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.899+0000 7f60eeffd700 1 -- 172.21.15.40:0/104169508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60f00c57d0 msgr2=0x7f60f00704d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:13:07.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.899+0000 7f60eeffd700 1 --2- 172.21.15.40:0/104169508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60f00c57d0 0x7f60f00704d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:07.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.899+0000 7f60eeffd700 1 -- 172.21.15.40:0/104169508 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d8009570 con 0x7f60f0163b20 2024-09-13T18:13:07.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.899+0000 7f60eeffd700 1 --2- 172.21.15.40:0/104169508 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60f0163b20 0x7f60f00710f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f60d8013000 tx=0x7f60d8004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:07.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.899+0000 7f60e7fff700 1 -- 172.21.15.40:0/104169508 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d8004590 con 0x7f60f0163b20 2024-09-13T18:13:07.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.899+0000 7f60e7fff700 1 -- 172.21.15.40:0/104169508 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d801e070 con 0x7f60f0163b20 2024-09-13T18:13:07.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.899+0000 7f60e7fff700 1 -- 172.21.15.40:0/104169508 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d8003900 con 0x7f60f0163b20 2024-09-13T18:13:07.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.899+0000 7f60f5c62700 1 -- 172.21.15.40:0/104169508 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f0203f10 con 0x7f60f0163b20 2024-09-13T18:13:07.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.899+0000 7f60f5c62700 1 -- 172.21.15.40:0/104169508 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f00caab0 con 0x7f60f0163b20 2024-09-13T18:13:07.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.899+0000 7f60f5c62700 1 -- 172.21.15.40:0/104169508 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f0070290 con 0x7f60f0163b20 2024-09-13T18:13:07.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.899+0000 7f60e7fff700 1 -- 172.21.15.40:0/104169508 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f60d8003a60 con 0x7f60f0163b20 2024-09-13T18:13:07.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.903+0000 7f60e7fff700 1 --2- 172.21.15.40:0/104169508 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f60d00666e0 0x7f60d0068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:07.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.903+0000 7f60ef7fe700 1 --2- 172.21.15.40:0/104169508 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f60d00666e0 0x7f60d0068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:07.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.903+0000 7f60e7fff700 1 -- 172.21.15.40:0/104169508 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f60d8020070 con 0x7f60f0163b20 2024-09-13T18:13:07.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.903+0000 7f60ef7fe700 1 --2- 172.21.15.40:0/104169508 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f60d00666e0 0x7f60d0068ba0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009500 tx=0x7f60e0009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:07.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:07.911+0000 7f60e7fff700 1 -- 172.21.15.40:0/104169508 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f60d8052b00 con 0x7f60f0163b20 2024-09-13T18:13:08.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:08.159+0000 7f60f5c62700 1 -- 172.21.15.40:0/104169508 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f60f016a4d0 con 0x7f60f0163b20 2024-09-13T18:13:08.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:08.159+0000 7f60e7fff700 1 -- 172.21.15.40:0/104169508 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f60d8056120 con 0x7f60f0163b20 2024-09-13T18:13:08.168 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":178,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":22,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:13:08.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:08.163+0000 7f60f5c62700 1 -- 172.21.15.40:0/104169508 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f60d00666e0 msgr2=0x7f60d0068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:08.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:08.163+0000 7f60f5c62700 1 --2- 172.21.15.40:0/104169508 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f60d00666e0 0x7f60d0068ba0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009500 tx=0x7f60e0009240 comp rx=0 tx=0).stop 2024-09-13T18:13:08.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:08.163+0000 7f60f5c62700 1 -- 172.21.15.40:0/104169508 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60f0163b20 msgr2=0x7f60f00710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:08.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:08.163+0000 7f60f5c62700 1 --2- 172.21.15.40:0/104169508 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60f0163b20 0x7f60f00710f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f60d8013000 tx=0x7f60d8004770 comp rx=0 tx=0).stop 2024-09-13T18:13:08.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:08.163+0000 7f60f5c62700 1 -- 172.21.15.40:0/104169508 shutdown_connections 2024-09-13T18:13:08.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:08.163+0000 7f60f5c62700 1 --2- 172.21.15.40:0/104169508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60f0166890 0x7f60f0071630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:08.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:08.163+0000 7f60f5c62700 1 --2- 172.21.15.40:0/104169508 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f60d00666e0 0x7f60d0068ba0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:08.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:08.163+0000 7f60f5c62700 1 --2- 172.21.15.40:0/104169508 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60f0163b20 0x7f60f00710f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:08.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:08.163+0000 7f60f5c62700 1 --2- 172.21.15.40:0/104169508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60f00c57d0 0x7f60f00704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:08.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:08.163+0000 7f60f5c62700 1 -- 172.21.15.40:0/104169508 >> 172.21.15.40:0/104169508 conn(0x7f60f00cadd0 msgr2=0x7f60f01652c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:08.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:08.163+0000 7f60f5c62700 1 -- 172.21.15.40:0/104169508 shutdown_connections 2024-09-13T18:13:08.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:08.163+0000 7f60f5c62700 1 -- 172.21.15.40:0/104169508 wait complete. 2024-09-13T18:13:11.185 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:13:11.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa8759e700 1 Processor -- start 2024-09-13T18:13:11.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa8759e700 1 -- start start 2024-09-13T18:13:11.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa8759e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faa8806ea60 0x7faa88070290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:11.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa8759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa88071620 0x7faa8806d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:11.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa8759e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa8814fdd0 con 0x7faa8814f9f0 2024-09-13T18:13:11.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa8759e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa88070920 con 0x7faa8806ea60 2024-09-13T18:13:11.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa8759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa881564d0 con 0x7faa88071620 2024-09-13T18:13:11.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa7dd9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faa8806ea60 0x7faa88070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:11.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa8659c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7faa8814f9f0 0x7faa88070ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58922/0 (socket says 172.21.15.40:58922) 2024-09-13T18:13:11.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa7dd9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faa8806ea60 0x7faa88070290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48964/0 (socket says 172.21.15.40:48964) 2024-09-13T18:13:11.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa8659c700 1 -- 172.21.15.40:0/842431966 learned_addr learned my addr 172.21.15.40:0/842431966 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:13:11.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa7dd9b700 1 -- 172.21.15.40:0/842431966 >> v2:172.21.15.40:3300/0 conn(0x7faa8806ea60 msgr2=0x7faa88070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:13:11.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa7dd9b700 1 -- 172.21.15.40:0/842431966 >> v2:172.21.15.40:3300/0 conn(0x7faa8806ea60 msgr2=0x7faa88070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:13:11.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa7dd9b700 1 --2- 172.21.15.40:0/842431966 >> v2:172.21.15.40:3300/0 conn(0x7faa8806ea60 0x7faa88070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:13:11.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa86d9d700 1 --2- 172.21.15.40:0/842431966 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa88071620 0x7faa8806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:11.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa7dd9b700 1 --2- 172.21.15.40:0/842431966 >> v2:172.21.15.40:3300/0 conn(0x7faa8806ea60 0x7faa88070290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:11.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa86d9d700 1 -- 172.21.15.40:0/842431966 >> v2:172.21.15.40:3300/0 conn(0x7faa8806ea60 msgr2=0x7faa88070290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:13:11.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa86d9d700 1 --2- 172.21.15.40:0/842431966 >> v2:172.21.15.40:3300/0 conn(0x7faa8806ea60 0x7faa88070290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:11.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa86d9d700 1 -- 172.21.15.40:0/842431966 >> v1:172.21.15.40:6789/0 conn(0x7faa8814f9f0 legacy=0x7faa88070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:11.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa86d9d700 1 -- 172.21.15.40:0/842431966 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa881558b0 con 0x7faa88071620 2024-09-13T18:13:11.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.319+0000 7faa86d9d700 1 --2- 172.21.15.40:0/842431966 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa88071620 0x7faa8806d570 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faa80009d90 tx=0x7faa8000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73d4316910a0c268 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:11.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.323+0000 7faa85d9b700 1 -- 172.21.15.40:0/842431966 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8000e070 con 0x7faa88071620 2024-09-13T18:13:11.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.323+0000 7faa85d9b700 1 -- 172.21.15.40:0/842431966 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8000bbd0 con 0x7faa88071620 2024-09-13T18:13:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.323+0000 7faa85d9b700 1 -- 172.21.15.40:0/842431966 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80012430 con 0x7faa88071620 2024-09-13T18:13:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.323+0000 7faa8759e700 1 -- 172.21.15.40:0/842431966 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa88071620 msgr2=0x7faa8806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.323+0000 7faa8759e700 1 --2- 172.21.15.40:0/842431966 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa88071620 0x7faa8806d570 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faa80009d90 tx=0x7faa8000b330 comp rx=0 tx=0).stop 2024-09-13T18:13:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.323+0000 7faa8759e700 1 -- 172.21.15.40:0/842431966 shutdown_connections 2024-09-13T18:13:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.323+0000 7faa8759e700 1 --2- 172.21.15.40:0/842431966 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa88071620 0x7faa8806d570 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.323+0000 7faa8759e700 1 --2- 172.21.15.40:0/842431966 >> v2:172.21.15.40:3300/0 conn(0x7faa8806ea60 0x7faa88070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.323+0000 7faa8759e700 1 -- 172.21.15.40:0/842431966 >> 172.21.15.40:0/842431966 conn(0x7faa8806a390 msgr2=0x7faa8806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.323+0000 7faa8759e700 1 -- 172.21.15.40:0/842431966 shutdown_connections 2024-09-13T18:13:11.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.323+0000 7faa8759e700 1 -- 172.21.15.40:0/842431966 wait complete. 2024-09-13T18:13:11.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.323+0000 7faa8759e700 1 Processor -- start 2024-09-13T18:13:11.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.323+0000 7faa8759e700 1 -- start start 2024-09-13T18:13:11.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa8759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa8806ea60 0x7faa88152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:11.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa8759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa88071620 0x7faa8814cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:11.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa8759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa8814f9f0 0x7faa88153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:11.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa8759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa88152680 con 0x7faa8806ea60 2024-09-13T18:13:11.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa8759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa881536f0 con 0x7faa88071620 2024-09-13T18:13:11.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa8759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa88152a00 con 0x7faa8814f9f0 2024-09-13T18:13:11.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa8659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa8806ea60 0x7faa88152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:11.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa8659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa8806ea60 0x7faa88152140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48970/0 (socket says 172.21.15.40:48970) 2024-09-13T18:13:11.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa8659c700 1 -- 172.21.15.40:0/588316225 learned_addr learned my addr 172.21.15.40:0/588316225 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:11.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa86d9d700 1 --2- 172.21.15.40:0/588316225 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa8814f9f0 0x7faa88153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:11.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa8659c700 1 -- 172.21.15.40:0/588316225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa8806ea60 msgr2=0x7faa88152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:13:11.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa8659c700 1 -- 172.21.15.40:0/588316225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa8806ea60 msgr2=0x7faa88152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:13:11.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa8659c700 1 --2- 172.21.15.40:0/588316225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa8806ea60 0x7faa88152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:13:11.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa8659c700 1 --2- 172.21.15.40:0/588316225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa8806ea60 0x7faa88152140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:11.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa7dd9b700 1 --2- 172.21.15.40:0/588316225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa88071620 0x7faa8814cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:11.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa86d9d700 1 -- 172.21.15.40:0/588316225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa88071620 msgr2=0x7faa8814cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:11.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa86d9d700 1 --2- 172.21.15.40:0/588316225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa88071620 0x7faa8814cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:11.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa86d9d700 1 -- 172.21.15.40:0/588316225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa8806ea60 msgr2=0x7faa88152140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:13:11.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa86d9d700 1 --2- 172.21.15.40:0/588316225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa8806ea60 0x7faa88152140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:11.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa86d9d700 1 -- 172.21.15.40:0/588316225 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa80009a70 con 0x7faa8814f9f0 2024-09-13T18:13:11.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa86d9d700 1 --2- 172.21.15.40:0/588316225 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa8814f9f0 0x7faa88153b80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faa80016000 tx=0x7faa80004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:11.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa7f7fe700 1 -- 172.21.15.40:0/588316225 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80012430 con 0x7faa8814f9f0 2024-09-13T18:13:11.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa8759e700 1 -- 172.21.15.40:0/588316225 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa88151780 con 0x7faa8814f9f0 2024-09-13T18:13:11.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa7f7fe700 1 -- 172.21.15.40:0/588316225 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa80020070 con 0x7faa8814f9f0 2024-09-13T18:13:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa7f7fe700 1 -- 172.21.15.40:0/588316225 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80004280 con 0x7faa8814f9f0 2024-09-13T18:13:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.327+0000 7faa8759e700 1 -- 172.21.15.40:0/588316225 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa88151270 con 0x7faa8814f9f0 2024-09-13T18:13:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.331+0000 7faa8759e700 1 -- 172.21.15.40:0/588316225 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa6c005310 con 0x7faa8814f9f0 2024-09-13T18:13:11.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.331+0000 7faa7f7fe700 1 -- 172.21.15.40:0/588316225 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faa800043e0 con 0x7faa8814f9f0 2024-09-13T18:13:11.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.331+0000 7faa7f7fe700 1 --2- 172.21.15.40:0/588316225 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faa68066730 0x7faa68068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:11.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.331+0000 7faa7f7fe700 1 -- 172.21.15.40:0/588316225 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7faa80022070 con 0x7faa8814f9f0 2024-09-13T18:13:11.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.331+0000 7faa8659c700 1 --2- 172.21.15.40:0/588316225 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faa68066730 0x7faa68068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:11.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.331+0000 7faa8659c700 1 --2- 172.21.15.40:0/588316225 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faa68066730 0x7faa68068bf0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faa78000f60 tx=0x7faa78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:11.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.339+0000 7faa7f7fe700 1 -- 172.21.15.40:0/588316225 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7faa800532e0 con 0x7faa8814f9f0 2024-09-13T18:13:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.575+0000 7faa8759e700 1 -- 172.21.15.40:0/588316225 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7faa6c0055c0 con 0x7faa8814f9f0 2024-09-13T18:13:11.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.575+0000 7faa7f7fe700 1 -- 172.21.15.40:0/588316225 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7faa80056900 con 0x7faa8814f9f0 2024-09-13T18:13:11.586 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":178,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:13:11.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.579+0000 7faa8759e700 1 -- 172.21.15.40:0/588316225 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faa68066730 msgr2=0x7faa68068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:11.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.579+0000 7faa8759e700 1 --2- 172.21.15.40:0/588316225 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faa68066730 0x7faa68068bf0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faa78000f60 tx=0x7faa78009040 comp rx=0 tx=0).stop 2024-09-13T18:13:11.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.579+0000 7faa8759e700 1 -- 172.21.15.40:0/588316225 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa8814f9f0 msgr2=0x7faa88153b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:11.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.579+0000 7faa8759e700 1 --2- 172.21.15.40:0/588316225 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa8814f9f0 0x7faa88153b80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faa80016000 tx=0x7faa80004af0 comp rx=0 tx=0).stop 2024-09-13T18:13:11.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.579+0000 7faa8759e700 1 -- 172.21.15.40:0/588316225 shutdown_connections 2024-09-13T18:13:11.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.579+0000 7faa8759e700 1 --2- 172.21.15.40:0/588316225 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faa68066730 0x7faa68068bf0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:11.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.579+0000 7faa8759e700 1 --2- 172.21.15.40:0/588316225 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faa8814f9f0 0x7faa88153b80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:11.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.579+0000 7faa8759e700 1 --2- 172.21.15.40:0/588316225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa88071620 0x7faa8814cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:11.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.579+0000 7faa8759e700 1 --2- 172.21.15.40:0/588316225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa8806ea60 0x7faa88152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:11.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.579+0000 7faa8759e700 1 -- 172.21.15.40:0/588316225 >> 172.21.15.40:0/588316225 conn(0x7faa8806a390 msgr2=0x7faa8814e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:11.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.579+0000 7faa8759e700 1 -- 172.21.15.40:0/588316225 shutdown_connections 2024-09-13T18:13:11.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:11.579+0000 7faa8759e700 1 -- 172.21.15.40:0/588316225 wait complete. 2024-09-13T18:13:14.602 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:13:14.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.719+0000 7fc2c9d05700 1 Processor -- start 2024-09-13T18:13:14.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.719+0000 7fc2c9d05700 1 -- start start 2024-09-13T18:13:14.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.719+0000 7fc2c9d05700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 0x7fc2c4066220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:14.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.719+0000 7fc2c9d05700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 0x7fc2c406d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:14.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.719+0000 7fc2c9d05700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c4066760 con 0x7fc2c40c74b0 2024-09-13T18:13:14.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.719+0000 7fc2c9d05700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c40c5950 con 0x7fc2c41513f0 2024-09-13T18:13:14.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.719+0000 7fc2c9d05700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c41571a0 con 0x7fc2c4157e90 2024-09-13T18:13:14.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.719+0000 7fc2c8d03700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 0x7fc2c4066220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:14.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.719+0000 7fc2c8d03700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 0x7fc2c4066220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37922/0 (socket says 172.21.15.40:37922) 2024-09-13T18:13:14.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.719+0000 7fc2c8d03700 1 -- 172.21.15.40:0/930431034 learned_addr learned my addr 172.21.15.40:0/930431034 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:14.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.719+0000 7fc2c8d03700 1 -- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 msgr2=0x7fc2c4066220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:13:14.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.719+0000 7fc2c8d03700 1 -- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 msgr2=0x7fc2c4066220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:13:14.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.719+0000 7fc2c8d03700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 0x7fc2c4066220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:13:14.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.719+0000 7fc2c8d03700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 0x7fc2c4066220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:14.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.719+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 0x7fc2c406d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:14.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.719+0000 7fc2c3fff700 1 -- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 msgr2=0x7fc2c406d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:13:14.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.719+0000 7fc2c3fff700 1 -- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 msgr2=0x7fc2c406d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:13:14.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.719+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 0x7fc2c406d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:13:14.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.719+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 0x7fc2c406d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:14.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.923+0000 7fc2c8d03700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 0x7fc2c4066220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:14.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.923+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 0x7fc2c406d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:14.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.923+0000 7fc2c8d03700 1 -- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 msgr2=0x7fc2c4066220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:13:14.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.923+0000 7fc2c8d03700 1 -- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 msgr2=0x7fc2c4066220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:13:14.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.923+0000 7fc2c3fff700 1 -- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 msgr2=0x7fc2c406d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:13:14.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.923+0000 7fc2c8d03700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 0x7fc2c4066220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:13:14.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.923+0000 7fc2c3fff700 1 -- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 msgr2=0x7fc2c406d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:13:14.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.923+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 0x7fc2c406d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:13:14.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.923+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 0x7fc2c406d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:13:14.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:14.923+0000 7fc2c8d03700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 0x7fc2c4066220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:13:15.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:15.327+0000 7fc2c8d03700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 0x7fc2c4066220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:15.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:15.327+0000 7fc2c8d03700 1 -- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 msgr2=0x7fc2c4066220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:13:15.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:15.327+0000 7fc2c8d03700 1 -- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 msgr2=0x7fc2c4066220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:13:15.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:15.327+0000 7fc2c8d03700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 0x7fc2c4066220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:13:15.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:15.327+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 0x7fc2c406d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:15.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:15.327+0000 7fc2c8d03700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 0x7fc2c4066220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:13:15.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:15.327+0000 7fc2c3fff700 1 -- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 msgr2=0x7fc2c406d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:13:15.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:15.327+0000 7fc2c3fff700 1 -- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 msgr2=0x7fc2c406d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:13:15.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:15.327+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 0x7fc2c406d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:13:15.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:15.327+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 0x7fc2c406d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:13:16.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:16.131+0000 7fc2c8d03700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 0x7fc2c4066220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:16.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:16.131+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 0x7fc2c406d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:16.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:16.131+0000 7fc2c8d03700 1 -- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 msgr2=0x7fc2c4066220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:13:16.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:16.131+0000 7fc2c8d03700 1 -- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 msgr2=0x7fc2c4066220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:13:16.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:16.131+0000 7fc2c8d03700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 0x7fc2c4066220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:13:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:16.131+0000 7fc2c8d03700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 0x7fc2c4066220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:13:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:16.131+0000 7fc2c3fff700 1 -- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 msgr2=0x7fc2c406d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:13:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:16.131+0000 7fc2c3fff700 1 -- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 msgr2=0x7fc2c406d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:13:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:16.131+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 0x7fc2c406d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:13:16.140 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:16.131+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 0x7fc2c406d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:13:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.719+0000 7fc2c27fc700 1 -- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 msgr2=0x7fc2c406d5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:13:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.719+0000 7fc2c27fc700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.43:3300/0 conn(0x7fc2c4157e90 0x7fc2c406d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.719+0000 7fc2c27fc700 1 -- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 msgr2=0x7fc2c4066220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:13:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.719+0000 7fc2c27fc700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2c41513f0 0x7fc2c4066220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.719+0000 7fc2c27fc700 1 -- 172.21.15.40:0/930431034 >> v1:172.21.15.43:6789/0 conn(0x7fc2c40c74b0 legacy=0x7fc2c40c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.719+0000 7fc2c27fc700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 0x7fc2a8009800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.719+0000 7fc2c27fc700 1 -- 172.21.15.40:0/930431034 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c406db30 con 0x7fc2a8003710 2024-09-13T18:13:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.719+0000 7fc2c27fc700 1 -- 172.21.15.40:0/930431034 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c40c5950 con 0x7fc2a8007340 2024-09-13T18:13:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.719+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 0x7fc2a8009800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.719+0000 7fc2c3fff700 1 -- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 msgr2=0x7fc2a8009800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:13:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.719+0000 7fc2c3fff700 1 -- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 msgr2=0x7fc2a8009800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:13:17.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.719+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 0x7fc2a8009800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:13:17.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.719+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 0x7fc2a8009800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:17.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.899+0000 7fc2c8d03700 1 -- 172.21.15.40:0/930431034 >> v1:172.21.15.40:6789/0 conn(0x7fc2a8003710 legacy=0x7fc2a8005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-13T18:13:17.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.899+0000 7fc2c8d03700 1 -- 172.21.15.40:0/930431034 >> v1:172.21.15.40:6789/0 conn(0x7fc2a8003710 legacy=0x7fc2a8005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:13:17.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.899+0000 7fc2c8d03700 1 --1- 172.21.15.40:0/930431034 >> v1:172.21.15.40:6789/0 conn(0x7fc2a8003710 0x7fc2a8005b70 :-1 s=OPENED pgs=25 cs=1 l=1).handle_message read tag failed 2024-09-13T18:13:17.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.899+0000 7fc2c8d03700 1 --1- 172.21.15.40:0/930431034 >> v1:172.21.15.40:6789/0 conn(0x7fc2a8003710 0x7fc2a8005b70 :-1 s=OPENED pgs=25 cs=1 l=1).fault on lossy channel, failing 2024-09-13T18:13:17.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.923+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 0x7fc2a8009800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:17.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.923+0000 7fc2c3fff700 1 -- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 msgr2=0x7fc2a8009800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:13:17.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.923+0000 7fc2c3fff700 1 -- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 msgr2=0x7fc2a8009800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:13:17.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.923+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 0x7fc2a8009800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:13:17.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:17.923+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 0x7fc2a8009800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:13:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:18.327+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 0x7fc2a8009800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:18.327+0000 7fc2c3fff700 1 -- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 msgr2=0x7fc2a8009800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:13:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:18.327+0000 7fc2c3fff700 1 -- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 msgr2=0x7fc2a8009800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:13:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:18.327+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 0x7fc2a8009800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:13:18.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:18.327+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 0x7fc2a8009800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:13:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:19.131+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 0x7fc2a8009800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:19.131+0000 7fc2c3fff700 1 -- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 msgr2=0x7fc2a8009800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:13:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:19.131+0000 7fc2c3fff700 1 -- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 msgr2=0x7fc2a8009800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:13:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:19.131+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 0x7fc2a8009800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:13:19.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:19.131+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 0x7fc2a8009800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:13:20.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.719+0000 7fc2c27fc700 1 -- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 msgr2=0x7fc2a8009800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:13:20.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.719+0000 7fc2c27fc700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8007340 0x7fc2a8009800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:20.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.719+0000 7fc2c27fc700 1 -- 172.21.15.40:0/930431034 >> v1:172.21.15.40:6789/0 conn(0x7fc2a8003710 legacy=0x7fc2a8005b70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:13:20.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.719+0000 7fc2c9504700 1 -- 172.21.15.40:0/930431034 reap_dead start 2024-09-13T18:13:20.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.719+0000 7fc2c27fc700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a800ab00 0x7fc2a8009800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:20.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.719+0000 7fc2c27fc700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2a8005cf0 0x7fc2a800d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:20.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.719+0000 7fc2c27fc700 1 -- 172.21.15.40:0/930431034 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a800afc0 con 0x7fc2a8009f70 2024-09-13T18:13:20.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.719+0000 7fc2c27fc700 1 -- 172.21.15.40:0/930431034 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c40c5950 con 0x7fc2a8005cf0 2024-09-13T18:13:20.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.719+0000 7fc2c27fc700 1 -- 172.21.15.40:0/930431034 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c41571a0 con 0x7fc2a800ab00 2024-09-13T18:13:20.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.719+0000 7fc2c9504700 1 --2- 172.21.15.40:0/930431034 >> v2:172.21.15.40:3300/0 conn(0x7fc2a8005cf0 0x7fc2a800d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:20.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.719+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a800ab00 0x7fc2a8009800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:20.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.723+0000 7fc2c9504700 1 -- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a800ab00 msgr2=0x7fc2a8009800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:13:20.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.723+0000 7fc2c9504700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a800ab00 0x7fc2a8009800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:20.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.723+0000 7fc2c9504700 1 -- 172.21.15.40:0/930431034 >> v1:172.21.15.40:6789/0 conn(0x7fc2a8009f70 legacy=0x7fc2a800a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:20.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.723+0000 7fc2c9504700 1 -- 172.21.15.40:0/930431034 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a800ea60 con 0x7fc2a8005cf0 2024-09-13T18:13:20.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.723+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a800ab00 0x7fc2a8009800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:13:20.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.723+0000 7fc2c9504700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2a8005cf0 0x7fc2a800d2a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40c6c40 tx=0x7fc2bc0021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8bff9cee491098 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:20.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.723+0000 7fc2c37fe700 1 -- 172.21.15.40:0/930431034 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc00cad0 con 0x7fc2a8005cf0 2024-09-13T18:13:20.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.723+0000 7fc2c37fe700 1 -- 172.21.15.40:0/930431034 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2bc00cc30 con 0x7fc2a8005cf0 2024-09-13T18:13:20.732 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.723+0000 7fc2c37fe700 1 -- 172.21.15.40:0/930431034 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0070d0 con 0x7fc2a8005cf0 2024-09-13T18:13:20.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.723+0000 7fc2c9d05700 1 -- 172.21.15.40:0/930431034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2a8005cf0 msgr2=0x7fc2a800d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:20.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.723+0000 7fc2c9d05700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2a8005cf0 0x7fc2a800d2a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40c6c40 tx=0x7fc2bc0021f0 comp rx=0 tx=0).stop 2024-09-13T18:13:20.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.723+0000 7fc2c9d05700 1 -- 172.21.15.40:0/930431034 shutdown_connections 2024-09-13T18:13:20.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.723+0000 7fc2c9d05700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2a8005cf0 0x7fc2a800d2a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:20.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.723+0000 7fc2c9d05700 1 --2- 172.21.15.40:0/930431034 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a800ab00 0x7fc2a8009800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:20.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.723+0000 7fc2c9d05700 1 -- 172.21.15.40:0/930431034 >> 172.21.15.40:0/930431034 conn(0x7fc2c406a570 msgr2=0x7fc2c406a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:20.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.723+0000 7fc2c9d05700 1 -- 172.21.15.40:0/930431034 shutdown_connections 2024-09-13T18:13:20.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.723+0000 7fc2c9d05700 1 -- 172.21.15.40:0/930431034 wait complete. 2024-09-13T18:13:20.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.727+0000 7fc2c9d05700 1 Processor -- start 2024-09-13T18:13:20.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.727+0000 7fc2c9d05700 1 -- start start 2024-09-13T18:13:20.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.727+0000 7fc2c9d05700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8005cf0 0x7fc2c40cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:20.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.727+0000 7fc2c9d05700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a8009f70 0x7fc2c414b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:20.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.727+0000 7fc2c9d05700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2a800ab00 0x7fc2c414bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:20.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.727+0000 7fc2c9d05700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c414d240 con 0x7fc2a800ab00 2024-09-13T18:13:20.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.727+0000 7fc2c9d05700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c4144030 con 0x7fc2a8009f70 2024-09-13T18:13:20.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.727+0000 7fc2c9d05700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c41450e0 con 0x7fc2a8005cf0 2024-09-13T18:13:20.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.727+0000 7fc2c9504700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2a800ab00 0x7fc2c414bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:20.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.727+0000 7fc2c9504700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2a800ab00 0x7fc2c414bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37968/0 (socket says 172.21.15.40:37968) 2024-09-13T18:13:20.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.727+0000 7fc2c9504700 1 -- 172.21.15.40:0/3682112552 learned_addr learned my addr 172.21.15.40:0/3682112552 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:20.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.727+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/3682112552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a8009f70 0x7fc2c414b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:20.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.727+0000 7fc2c8d03700 1 --2- 172.21.15.40:0/3682112552 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8005cf0 0x7fc2c40cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:20.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.731+0000 7fc2c9504700 1 -- 172.21.15.40:0/3682112552 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8005cf0 msgr2=0x7fc2c40cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:20.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.731+0000 7fc2c9504700 1 --2- 172.21.15.40:0/3682112552 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8005cf0 0x7fc2c40cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:20.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.731+0000 7fc2c9504700 1 -- 172.21.15.40:0/3682112552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a8009f70 msgr2=0x7fc2c414b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:20.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.731+0000 7fc2c9504700 1 --2- 172.21.15.40:0/3682112552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a8009f70 0x7fc2c414b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:20.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.731+0000 7fc2c9504700 1 -- 172.21.15.40:0/3682112552 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b000e040 con 0x7fc2a800ab00 2024-09-13T18:13:20.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.731+0000 7fc2c3fff700 1 --2- 172.21.15.40:0/3682112552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a8009f70 0x7fc2c414b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:13:20.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.731+0000 7fc2c9504700 1 --2- 172.21.15.40:0/3682112552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2a800ab00 0x7fc2c414bd80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40661b0 tx=0x7fc2bc0014d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:20.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.731+0000 7fc2c17fa700 1 -- 172.21.15.40:0/3682112552 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc024070 con 0x7fc2a800ab00 2024-09-13T18:13:20.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.731+0000 7fc2c17fa700 1 -- 172.21.15.40:0/3682112552 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2bc001bb0 con 0x7fc2a800ab00 2024-09-13T18:13:20.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.731+0000 7fc2c17fa700 1 -- 172.21.15.40:0/3682112552 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc003a20 con 0x7fc2a800ab00 2024-09-13T18:13:20.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.731+0000 7fc2c9d05700 1 -- 172.21.15.40:0/3682112552 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c4066760 con 0x7fc2a800ab00 2024-09-13T18:13:20.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.731+0000 7fc2c9d05700 1 -- 172.21.15.40:0/3682112552 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c4144690 con 0x7fc2a800ab00 2024-09-13T18:13:20.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.731+0000 7fc2c9d05700 1 -- 172.21.15.40:0/3682112552 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c41573b0 con 0x7fc2a800ab00 2024-09-13T18:13:20.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.735+0000 7fc2c17fa700 1 -- 172.21.15.40:0/3682112552 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc2bc019070 con 0x7fc2a800ab00 2024-09-13T18:13:20.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.735+0000 7fc2c17fa700 1 --2- 172.21.15.40:0/3682112552 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc2a8068480 0x7fc2a806a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:20.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.735+0000 7fc2c17fa700 1 -- 172.21.15.40:0/3682112552 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc2bc0855e0 con 0x7fc2a800ab00 2024-09-13T18:13:20.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.735+0000 7fc2c8d03700 1 --2- 172.21.15.40:0/3682112552 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc2a8068480 0x7fc2a806a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:20.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.735+0000 7fc2c8d03700 1 --2- 172.21.15.40:0/3682112552 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc2a8068480 0x7fc2a806a940 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8006bc0 tx=0x7fc2b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:20.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:20.743+0000 7fc2c17fa700 1 -- 172.21.15.40:0/3682112552 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc2bc052740 con 0x7fc2a800ab00 2024-09-13T18:13:21.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.011+0000 7fc2c9d05700 1 -- 172.21.15.40:0/3682112552 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc2c40661e0 con 0x7fc2a800ab00 2024-09-13T18:13:21.021 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.011+0000 7fc2c17fa700 1 -- 172.21.15.40:0/3682112552 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fc2bc055d60 con 0x7fc2a800ab00 2024-09-13T18:13:21.021 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":186,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:13:21.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.015+0000 7fc2c9d05700 1 -- 172.21.15.40:0/3682112552 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc2a8068480 msgr2=0x7fc2a806a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.015+0000 7fc2c9d05700 1 --2- 172.21.15.40:0/3682112552 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc2a8068480 0x7fc2a806a940 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8006bc0 tx=0x7fc2b8008040 comp rx=0 tx=0).stop 2024-09-13T18:13:21.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.015+0000 7fc2c9d05700 1 -- 172.21.15.40:0/3682112552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2a800ab00 msgr2=0x7fc2c414bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.015+0000 7fc2c9d05700 1 --2- 172.21.15.40:0/3682112552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2a800ab00 0x7fc2c414bd80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40661b0 tx=0x7fc2bc0014d0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.015+0000 7fc2c9d05700 1 -- 172.21.15.40:0/3682112552 shutdown_connections 2024-09-13T18:13:21.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.015+0000 7fc2c9d05700 1 --2- 172.21.15.40:0/3682112552 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc2a800ab00 0x7fc2c414bd80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.015+0000 7fc2c9d05700 1 --2- 172.21.15.40:0/3682112552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2a8009f70 0x7fc2c414b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.015+0000 7fc2c9d05700 1 --2- 172.21.15.40:0/3682112552 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc2a8068480 0x7fc2a806a940 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.015+0000 7fc2c9d05700 1 --2- 172.21.15.40:0/3682112552 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc2a8005cf0 0x7fc2c40cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.015+0000 7fc2c9d05700 1 -- 172.21.15.40:0/3682112552 >> 172.21.15.40:0/3682112552 conn(0x7fc2c406a570 msgr2=0x7fc2c41467d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:21.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.015+0000 7fc2c9d05700 1 -- 172.21.15.40:0/3682112552 shutdown_connections 2024-09-13T18:13:21.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.015+0000 7fc2c9d05700 1 -- 172.21.15.40:0/3682112552 wait complete. 2024-09-13T18:13:21.042 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:13:21.042 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:13:21.043 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:13:21.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.155+0000 7f2728ea7700 1 Processor -- start 2024-09-13T18:13:21.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.155+0000 7f2728ea7700 1 -- start start 2024-09-13T18:13:21.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.155+0000 7f2728ea7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f271c04ca90 0x7f271c007560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.155+0000 7f2728ea7700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f271c005fa0 con 0x7f271c0d69c0 2024-09-13T18:13:21.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.155+0000 7f2728ea7700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f271c036080 con 0x7f271c007ba0 2024-09-13T18:13:21.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.155+0000 7f2728ea7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0368b0 con 0x7f271c04ca90 2024-09-13T18:13:21.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.155+0000 7f27237fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f271c0d69c0 0x7f271c033e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46950/0 (socket says 172.21.15.40:46950) 2024-09-13T18:13:21.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.155+0000 7f27237fe700 1 -- 172.21.15.40:0/3171261744 learned_addr learned my addr 172.21.15.40:0/3171261744 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:13:21.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.155+0000 7f2722ffd700 1 --2- 172.21.15.40:0/3171261744 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f271c04ca90 0x7f271c007560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.155+0000 7f27227fc700 1 -- 172.21.15.40:0/3171261744 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901787410 0 0) 0x7f271c005fa0 con 0x7f271c0d69c0 2024-09-13T18:13:21.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.155+0000 7f27227fc700 1 -- 172.21.15.40:0/3171261744 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27140035d0 con 0x7f271c0d69c0 2024-09-13T18:13:21.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.159+0000 7f27227fc700 1 -- 172.21.15.40:0/3171261744 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2104263910 0 0) 0x7f271c036080 con 0x7f271c007ba0 2024-09-13T18:13:21.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.159+0000 7f27227fc700 1 -- 172.21.15.40:0/3171261744 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f271c005fa0 con 0x7f271c007ba0 2024-09-13T18:13:21.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.159+0000 7f2722ffd700 1 -- 172.21.15.40:0/3171261744 >> v1:172.21.15.43:6789/0 conn(0x7f271c007ba0 legacy=0x7f271c0dc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.159+0000 7f2722ffd700 1 -- 172.21.15.40:0/3171261744 >> v1:172.21.15.40:6789/0 conn(0x7f271c0d69c0 legacy=0x7f271c033e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.159+0000 7f2722ffd700 1 -- 172.21.15.40:0/3171261744 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c034b20 con 0x7f271c04ca90 2024-09-13T18:13:21.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.159+0000 7f2722ffd700 1 --2- 172.21.15.40:0/3171261744 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f271c04ca90 0x7f271c007560 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2710009880 tx=0x7f2710009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=465f30ff8d983723 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.159+0000 7f27227fc700 1 -- 172.21.15.40:0/3171261744 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2710004230 con 0x7f271c04ca90 2024-09-13T18:13:21.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.159+0000 7f27227fc700 1 -- 172.21.15.40:0/3171261744 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2710004390 con 0x7f271c04ca90 2024-09-13T18:13:21.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.159+0000 7f27227fc700 1 -- 172.21.15.40:0/3171261744 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27100036c0 con 0x7f271c04ca90 2024-09-13T18:13:21.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.159+0000 7f2728ea7700 1 -- 172.21.15.40:0/3171261744 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f271c04ca90 msgr2=0x7f271c007560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.159+0000 7f2728ea7700 1 --2- 172.21.15.40:0/3171261744 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f271c04ca90 0x7f271c007560 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2710009880 tx=0x7f2710009b90 comp rx=0 tx=0).stop 2024-09-13T18:13:21.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.159+0000 7f2728ea7700 1 -- 172.21.15.40:0/3171261744 shutdown_connections 2024-09-13T18:13:21.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.159+0000 7f2728ea7700 1 --2- 172.21.15.40:0/3171261744 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f271c04ca90 0x7f271c007560 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.159+0000 7f2728ea7700 1 -- 172.21.15.40:0/3171261744 >> 172.21.15.40:0/3171261744 conn(0x7f271c0505e0 msgr2=0x7f271c0509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:21.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.159+0000 7f2728ea7700 1 -- 172.21.15.40:0/3171261744 shutdown_connections 2024-09-13T18:13:21.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.159+0000 7f2728ea7700 1 -- 172.21.15.40:0/3171261744 wait complete. 2024-09-13T18:13:21.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.159+0000 7f2728ea7700 1 Processor -- start 2024-09-13T18:13:21.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2728ea7700 1 -- start start 2024-09-13T18:13:21.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2728ea7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f271c007ba0 0x7f271c04f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.173 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2728ea7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f271c04ca90 0x7f271c173290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2728ea7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f271c0d69c0 0x7f271c1737d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2728ea7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0d3870 con 0x7f271c0d69c0 2024-09-13T18:13:21.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2728ea7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c049540 con 0x7f271c04ca90 2024-09-13T18:13:21.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2728ea7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0d3aa0 con 0x7f271c007ba0 2024-09-13T18:13:21.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2723fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f271c0d69c0 0x7f271c1737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2723fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f271c0d69c0 0x7f271c1737d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37970/0 (socket says 172.21.15.40:37970) 2024-09-13T18:13:21.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2723fff700 1 -- 172.21.15.40:0/3378178649 learned_addr learned my addr 172.21.15.40:0/3378178649 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:21.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2722ffd700 1 --2- 172.21.15.40:0/3378178649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f271c04ca90 0x7f271c173290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f27237fe700 1 --2- 172.21.15.40:0/3378178649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f271c007ba0 0x7f271c04f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2723fff700 1 -- 172.21.15.40:0/3378178649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f271c007ba0 msgr2=0x7f271c04f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2723fff700 1 --2- 172.21.15.40:0/3378178649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f271c007ba0 0x7f271c04f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2723fff700 1 -- 172.21.15.40:0/3378178649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f271c04ca90 msgr2=0x7f271c173290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2723fff700 1 --2- 172.21.15.40:0/3378178649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f271c04ca90 0x7f271c173290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2723fff700 1 -- 172.21.15.40:0/3378178649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27100094b0 con 0x7f271c0d69c0 2024-09-13T18:13:21.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2723fff700 1 --2- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f271c0d69c0 0x7f271c1737d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2724000b60 tx=0x7f27240680b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f270ffff700 1 -- 172.21.15.40:0/3378178649 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2724067730 con 0x7f271c0d69c0 2024-09-13T18:13:21.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2728ea7700 1 -- 172.21.15.40:0/3378178649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f271c048930 con 0x7f271c0d69c0 2024-09-13T18:13:21.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2728ea7700 1 -- 172.21.15.40:0/3378178649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f271c048ef0 con 0x7f271c0d69c0 2024-09-13T18:13:21.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f270ffff700 1 -- 172.21.15.40:0/3378178649 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27240678b0 con 0x7f271c0d69c0 2024-09-13T18:13:21.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f270ffff700 1 -- 172.21.15.40:0/3378178649 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2724072390 con 0x7f271c0d69c0 2024-09-13T18:13:21.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2728ea7700 1 --2- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f271c0490e0 0x7f271c0cf440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f2728ea7700 1 -- 172.21.15.40:0/3378178649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c008d50 con 0x7f271c0490e0 2024-09-13T18:13:21.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.163+0000 7f27237fe700 1 --2- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f271c0490e0 0x7f271c0cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.167+0000 7f27237fe700 1 -- 172.21.15.40:0/3378178649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f271c0365d0 con 0x7f271c0490e0 2024-09-13T18:13:21.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.167+0000 7f27237fe700 1 --2- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f271c0490e0 0x7f271c0cf440 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f271800f770 tx=0x7f271800fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.167+0000 7f270ffff700 1 -- 172.21.15.40:0/3378178649 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27180072e0 con 0x7f271c0490e0 2024-09-13T18:13:21.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.167+0000 7f270ffff700 1 -- 172.21.15.40:0/3378178649 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f27240724f0 con 0x7f271c0d69c0 2024-09-13T18:13:21.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.167+0000 7f270ffff700 1 --2- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f27080667d0 0x7f2708068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.167+0000 7f2722ffd700 1 --2- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f27080667d0 0x7f2708068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.167+0000 7f270ffff700 1 -- 172.21.15.40:0/3378178649 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f27240fe820 con 0x7f271c0d69c0 2024-09-13T18:13:21.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.167+0000 7f270ffff700 1 -- 172.21.15.40:0/3378178649 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f2718007440 con 0x7f271c0490e0 2024-09-13T18:13:21.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.167+0000 7f270ffff700 1 -- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f271c0490e0 msgr2=0x7f271c0cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.167+0000 7f270ffff700 1 --2- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f271c0490e0 0x7f271c0cf440 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f271800f770 tx=0x7f271800fb30 comp rx=0 tx=0).stop 2024-09-13T18:13:21.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.167+0000 7f270ffff700 1 -- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f271c0490e0 msgr2=0x7f271c0cf440 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:13:21.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.167+0000 7f270ffff700 1 --2- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f271c0490e0 0x7f271c0cf440 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.167+0000 7f2722ffd700 1 --2- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f27080667d0 0x7f2708068c90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2710009880 tx=0x7f2710010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.175+0000 7f2728ea7700 1 --2- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f271c0d9c70 0x7f271c0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.175+0000 7f2728ea7700 1 -- 172.21.15.40:0/3378178649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0cfec0 con 0x7f271c0d9c70 2024-09-13T18:13:21.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.175+0000 7f27237fe700 1 --2- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f271c0d9c70 0x7f271c0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.185 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.175+0000 7f27237fe700 1 -- 172.21.15.40:0/3378178649 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f271c0362f0 con 0x7f271c0d9c70 2024-09-13T18:13:21.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.175+0000 7f27237fe700 1 --2- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f271c0d9c70 0x7f271c0cf980 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f271800f770 tx=0x7f2718000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.179+0000 7f270ffff700 1 -- 172.21.15.40:0/3378178649 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2718022070 con 0x7f271c0d9c70 2024-09-13T18:13:21.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.183+0000 7f270ffff700 1 -- 172.21.15.40:0/3378178649 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4452 (secure 0 0 0) 0x7f271801db60 con 0x7f271c0d9c70 2024-09-13T18:13:21.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.183+0000 7f270ffff700 1 -- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f271c0d9c70 msgr2=0x7f271c0cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.183+0000 7f270ffff700 1 --2- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f271c0d9c70 0x7f271c0cf980 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f271800f770 tx=0x7f2718000bc0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.183+0000 7f270ffff700 1 -- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f271c0d9c70 msgr2=0x7f271c0cf980 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:13:21.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.183+0000 7f270ffff700 1 --2- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f271c0d9c70 0x7f271c0cf980 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.191 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:13:21.191 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:13:21.191 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:13:21.191 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:13:21.191 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 186, 2024-09-13T18:13:21.191 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:13:21.191 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:13:21.192 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:13:21.192 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:13:21.192 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:21.192 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:13:21.192 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:13:21.192 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:13:21.192 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:13:21.192 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:21.192 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:21.192 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:21.192 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:21.192 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:21.192 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:21.193 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:21.193 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:21.193 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:21.193 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:21.193 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:21.193 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:13:21.193 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:13:21.193 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:21.193 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:21.193 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:21.193 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:21.193 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:21.194 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:21.194 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:21.194 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:21.194 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:21.194 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:21.194 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:21.194 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.194 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:13:21.194 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:13:21.194 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:13:21.194 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:13:21.194 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:13:21.195 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:13:21.195 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:13:21.195 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:13:21.195 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:13:21.195 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:13:21.195 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:13:21.195 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:13:21.195 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:13:21.195 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:13:21.195 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:13:21.195 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:13:21.195 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:13:21.195 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:21.195 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:21.196 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:21.196 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:21.196 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:21.196 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:21.196 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:21.196 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:21.196 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:21.196 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:21.196 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:21.196 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:13:21.196 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.196 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:13:21.196 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.196 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:13:21.197 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:13:21.197 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:21.197 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:21.197 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.197 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:21.197 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:13:21.197 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:21.197 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.197 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.197 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:21.197 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:13:21.198 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:21.198 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.198 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:21.198 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.198 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:13:21.198 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:13:21.198 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:21.198 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:21.198 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:21.198 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.198 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.198 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:13:21.198 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:13:21.199 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:21.199 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:21.199 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.199 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:21.199 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:13:21.199 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:21.199 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.199 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.199 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:21.199 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:13:21.199 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:21.199 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.199 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:21.200 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.200 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:13:21.200 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:13:21.200 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:21.200 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:21.200 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:21.201 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.201 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.201 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:13:21.201 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:13:21.201 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:21.201 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:21.201 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.201 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:21.201 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:13:21.201 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:21.201 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.202 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.202 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:21.202 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:13:21.202 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:21.202 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.202 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:21.202 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.202 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:13:21.202 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:13:21.202 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:21.202 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:21.202 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:21.202 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.202 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:21.203 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.203 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:13:21.203 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:13:21.203 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.203 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:21.203 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:21.203 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:13:21.203 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.203 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:21.203 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:13:21.203 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.203 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:21.203 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:21.203 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:13:21.203 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.203 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:21.204 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.204 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:13:21.204 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:13:21.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.183+0000 7f2728ea7700 1 -- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f27080667d0 msgr2=0x7f2708068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.183+0000 7f2728ea7700 1 --2- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f27080667d0 0x7f2708068c90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f2710009880 tx=0x7f2710010000 comp rx=0 tx=0).stop 2024-09-13T18:13:21.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.183+0000 7f2728ea7700 1 -- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f271c0d69c0 msgr2=0x7f271c1737d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.183+0000 7f2728ea7700 1 --2- 172.21.15.40:0/3378178649 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f271c0d69c0 0x7f271c1737d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2724000b60 tx=0x7f27240680b0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.183+0000 7f2723fff700 1 -- 172.21.15.40:0/3378178649 reap_dead start 2024-09-13T18:13:21.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.183+0000 7f2723fff700 1 -- 172.21.15.40:0/3378178649 reap_dead start 2024-09-13T18:13:21.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.183+0000 7f2728ea7700 1 -- 172.21.15.40:0/3378178649 shutdown_connections 2024-09-13T18:13:21.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.183+0000 7f2728ea7700 1 -- 172.21.15.40:0/3378178649 >> 172.21.15.40:0/3378178649 conn(0x7f271c0505e0 msgr2=0x7f271c0d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:21.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.183+0000 7f2728ea7700 1 -- 172.21.15.40:0/3378178649 shutdown_connections 2024-09-13T18:13:21.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.183+0000 7f2728ea7700 1 -- 172.21.15.40:0/3378178649 wait complete. 2024-09-13T18:13:21.206 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:13:21.206 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:13:21.206 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:13:21.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.307+0000 7f34465d4700 1 Processor -- start 2024-09-13T18:13:21.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.307+0000 7f34465d4700 1 -- start start 2024-09-13T18:13:21.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34465d4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3440151580 0x7f34400c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34465d4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f34400b1fb0 0x7f34400c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34465d4700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3440070ae0 con 0x7f3440071260 2024-09-13T18:13:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34465d4700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3440070d30 con 0x7f34400b1fb0 2024-09-13T18:13:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34465d4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34400c29c0 con 0x7f3440151580 2024-09-13T18:13:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34455d2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3440151580 0x7f34400c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f3444dd1700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f34400b1fb0 0x7f34400c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34455d2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3440151580 0x7f34400c52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:58354/0 (socket says 172.21.15.40:58354) 2024-09-13T18:13:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34455d2700 1 -- 172.21.15.40:0/1693594854 learned_addr learned my addr 172.21.15.40:0/1693594854 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34455d2700 1 -- 172.21.15.40:0/1693594854 >> v2:172.21.15.43:3300/0 conn(0x7f34400b1fb0 msgr2=0x7f34400c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:13:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34455d2700 1 --2- 172.21.15.40:0/1693594854 >> v2:172.21.15.43:3300/0 conn(0x7f34400b1fb0 0x7f34400c57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34455d2700 1 -- 172.21.15.40:0/1693594854 >> v1:172.21.15.43:6789/0 conn(0x7f3440071260 legacy=0x7f3440157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34455d2700 1 -- 172.21.15.40:0/1693594854 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34400c2ca0 con 0x7f3440151580 2024-09-13T18:13:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34455d2700 1 --2- 172.21.15.40:0/1693594854 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3440151580 0x7f34400c52b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f343c009910 tx=0x7f343c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1e5fb267c03067b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f3437fff700 1 -- 172.21.15.40:0/1693594854 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c011830 con 0x7f3440151580 2024-09-13T18:13:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f3437fff700 1 -- 172.21.15.40:0/1693594854 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343c011990 con 0x7f3440151580 2024-09-13T18:13:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f3437fff700 1 -- 172.21.15.40:0/1693594854 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c011d40 con 0x7f3440151580 2024-09-13T18:13:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34465d4700 1 -- 172.21.15.40:0/1693594854 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3440151580 msgr2=0x7f34400c52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34465d4700 1 --2- 172.21.15.40:0/1693594854 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3440151580 0x7f34400c52b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f343c009910 tx=0x7f343c009c20 comp rx=0 tx=0).stop 2024-09-13T18:13:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34465d4700 1 -- 172.21.15.40:0/1693594854 shutdown_connections 2024-09-13T18:13:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34465d4700 1 --2- 172.21.15.40:0/1693594854 >> v2:172.21.15.43:3300/0 conn(0x7f34400b1fb0 0x7f34400c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34465d4700 1 --2- 172.21.15.40:0/1693594854 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3440151580 0x7f34400c52b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34465d4700 1 -- 172.21.15.40:0/1693594854 >> 172.21.15.40:0/1693594854 conn(0x7f344006e6d0 msgr2=0x7f344006eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34465d4700 1 -- 172.21.15.40:0/1693594854 shutdown_connections 2024-09-13T18:13:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.311+0000 7f34465d4700 1 -- 172.21.15.40:0/1693594854 wait complete. 2024-09-13T18:13:21.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f34465d4700 1 Processor -- start 2024-09-13T18:13:21.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f34465d4700 1 -- start start 2024-09-13T18:13:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f34465d4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3440071260 0x7f344006bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f34465d4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f34400b1fb0 0x7f34400c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f34465d4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3440151580 0x7f34400c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f34465d4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34400cafb0 con 0x7f3440151580 2024-09-13T18:13:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f34465d4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34400cc940 con 0x7f3440071260 2024-09-13T18:13:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f34465d4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34401f2a00 con 0x7f34400b1fb0 2024-09-13T18:13:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f3445dd3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3440151580 0x7f34400c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f34455d2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3440071260 0x7f344006bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f34455d2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3440071260 0x7f344006bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:37790/0 (socket says 172.21.15.40:37790) 2024-09-13T18:13:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f34455d2700 1 -- 172.21.15.40:0/2223253325 learned_addr learned my addr 172.21.15.40:0/2223253325 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f3445dd3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3440151580 0x7f34400c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37996/0 (socket says 172.21.15.40:37996) 2024-09-13T18:13:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f3444dd1700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f34400b1fb0 0x7f34400c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f34455d2700 1 -- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f34400b1fb0 msgr2=0x7f34400c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f34455d2700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f34400b1fb0 0x7f34400c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f34455d2700 1 -- 172.21.15.40:0/2223253325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3440151580 msgr2=0x7f34400c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f34455d2700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3440151580 0x7f34400c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f34455d2700 1 -- 172.21.15.40:0/2223253325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c009570 con 0x7f3440071260 2024-09-13T18:13:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f3445dd3700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3440151580 0x7f34400c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:13:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f3444dd1700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f34400b1fb0 0x7f34400c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:13:21.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f34455d2700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3440071260 0x7f344006bd40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f343c009c60 tx=0x7f343c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f3435ffb700 1 -- 172.21.15.40:0/2223253325 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c004520 con 0x7f3440071260 2024-09-13T18:13:21.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f3435ffb700 1 -- 172.21.15.40:0/2223253325 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343c01e070 con 0x7f3440071260 2024-09-13T18:13:21.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f34465d4700 1 -- 172.21.15.40:0/2223253325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34400ca3a0 con 0x7f3440071260 2024-09-13T18:13:21.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f3435ffb700 1 -- 172.21.15.40:0/2223253325 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c0038c0 con 0x7f3440071260 2024-09-13T18:13:21.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.315+0000 7f34465d4700 1 -- 172.21.15.40:0/2223253325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34400ca960 con 0x7f3440071260 2024-09-13T18:13:21.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.319+0000 7f34465d4700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34400cab20 0x7f344014a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.319+0000 7f34465d4700 1 -- 172.21.15.40:0/2223253325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34400cbfe0 con 0x7f34400cab20 2024-09-13T18:13:21.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.319+0000 7f3444dd1700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34400cab20 0x7f344014a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.319+0000 7f3444dd1700 1 -- 172.21.15.40:0/2223253325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3440151960 con 0x7f34400cab20 2024-09-13T18:13:21.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.319+0000 7f3444dd1700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34400cab20 0x7f344014a2c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f344014f620 tx=0x7f3430011b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.319+0000 7f3435ffb700 1 -- 172.21.15.40:0/2223253325 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f343c003a20 con 0x7f3440071260 2024-09-13T18:13:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.319+0000 7f3435ffb700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3424066760 0x7f3424068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.319+0000 7f3435ffb700 1 -- 172.21.15.40:0/2223253325 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f343c020070 con 0x7f3440071260 2024-09-13T18:13:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.319+0000 7f3445dd3700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3424066760 0x7f3424068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.319+0000 7f3435ffb700 1 -- 172.21.15.40:0/2223253325 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430015870 con 0x7f34400cab20 2024-09-13T18:13:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.319+0000 7f3435ffb700 1 -- 172.21.15.40:0/2223253325 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f34300159d0 con 0x7f34400cab20 2024-09-13T18:13:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.319+0000 7f3435ffb700 1 -- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34400cab20 msgr2=0x7f344014a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.319+0000 7f3435ffb700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34400cab20 0x7f344014a2c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f344014f620 tx=0x7f3430011b90 comp rx=0 tx=0).stop 2024-09-13T18:13:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.319+0000 7f3435ffb700 1 -- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34400cab20 msgr2=0x7f344014a2c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:13:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.319+0000 7f3435ffb700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34400cab20 0x7f344014a2c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.319+0000 7f3445dd3700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3424066760 0x7f3424068c20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f34380012e0 tx=0x7f3438009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.331+0000 7f34465d4700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f344014aee0 0x7f344014a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.331+0000 7f34465d4700 1 -- 172.21.15.40:0/2223253325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3440067600 con 0x7f344014aee0 2024-09-13T18:13:21.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.331+0000 7f3444dd1700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f344014aee0 0x7f344014a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.335+0000 7f3444dd1700 1 -- 172.21.15.40:0/2223253325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3440157290 con 0x7f344014aee0 2024-09-13T18:13:21.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.335+0000 7f3444dd1700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f344014aee0 0x7f344014a800 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f343001b000 tx=0x7f3430004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.335+0000 7f3435ffb700 1 -- 172.21.15.40:0/2223253325 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430024070 con 0x7f344014aee0 2024-09-13T18:13:21.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.335+0000 7f3435ffb700 1 -- 172.21.15.40:0/2223253325 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4782 (secure 0 0 0) 0x7f34300068e0 con 0x7f344014aee0 2024-09-13T18:13:21.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.335+0000 7f3435ffb700 1 -- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f344014aee0 msgr2=0x7f344014a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.335+0000 7f3435ffb700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f344014aee0 0x7f344014a800 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f343001b000 tx=0x7f3430004770 comp rx=0 tx=0).stop 2024-09-13T18:13:21.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.335+0000 7f3435ffb700 1 -- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f344014aee0 msgr2=0x7f344014a800 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:13:21.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.335+0000 7f3435ffb700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f344014aee0 0x7f344014a800 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.344 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:13:21.344 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:13:21.344 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:13:21.344 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:13:21.344 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 186, 2024-09-13T18:13:21.344 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:13:21.344 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:13:21.344 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:13:21.344 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:13:21.345 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:21.345 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:13:21.345 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:13:21.345 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:13:21.345 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:13:21.345 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:21.345 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:21.345 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:21.345 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:21.345 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:21.345 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:21.345 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:21.345 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:21.346 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:21.346 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:21.346 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:21.346 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:13:21.346 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:13:21.346 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:21.346 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:21.346 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:21.346 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:21.346 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:21.346 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:21.346 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:21.346 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:21.347 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:21.347 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:21.347 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:21.347 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.347 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:13:21.347 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:13:21.347 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:13:21.347 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:13:21.347 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:13:21.347 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:13:21.347 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:13:21.347 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:13:21.347 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:13:21.347 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:13:21.348 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:13:21.348 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:13:21.348 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:13:21.348 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:13:21.348 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:13:21.348 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:13:21.348 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:13:21.348 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:21.348 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:21.348 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:21.348 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:21.348 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:21.348 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:21.349 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:21.349 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:21.349 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:21.349 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:21.349 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:21.349 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:13:21.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.349 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:13:21.349 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.349 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:13:21.349 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:13:21.349 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:21.349 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:21.349 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.350 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:21.350 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:13:21.350 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:21.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.350 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.350 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:21.350 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:13:21.350 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:21.350 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.350 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:21.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.350 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:13:21.350 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:13:21.351 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:21.351 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:21.351 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:21.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.351 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:13:21.351 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:13:21.351 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:21.351 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:21.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.351 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:21.351 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:13:21.351 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:21.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.352 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:21.352 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:13:21.352 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:21.352 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.352 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:21.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.352 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:13:21.352 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:13:21.352 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:21.352 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:21.352 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:21.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.353 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:13:21.353 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:13:21.353 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:21.353 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:21.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.353 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:21.353 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:13:21.353 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:21.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.353 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:21.353 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:13:21.353 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:21.354 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.354 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:21.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.354 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:13:21.354 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:13:21.354 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:21.354 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:21.354 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:21.354 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.354 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:21.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.354 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:13:21.354 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:13:21.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.355 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:21.355 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:21.355 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:13:21.355 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.355 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:21.355 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:13:21.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.355 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:21.355 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:21.355 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:13:21.355 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.355 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:21.355 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:13:21.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.356 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:21.356 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:21.356 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:13:21.356 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.356 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:21.356 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:13:21.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.356 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:21.356 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:21.356 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:13:21.356 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.356 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:21.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.357 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:13:21.357 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:13:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.335+0000 7f34465d4700 1 -- 172.21.15.40:0/2223253325 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3424066760 msgr2=0x7f3424068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.335+0000 7f34465d4700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3424066760 0x7f3424068c20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f34380012e0 tx=0x7f3438009040 comp rx=0 tx=0).stop 2024-09-13T18:13:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.335+0000 7f34465d4700 1 -- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3440071260 msgr2=0x7f344006bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.335+0000 7f3445dd3700 1 -- 172.21.15.40:0/2223253325 reap_dead start 2024-09-13T18:13:21.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.335+0000 7f34465d4700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3440071260 0x7f344006bd40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f343c009c60 tx=0x7f343c004770 comp rx=0 tx=0).stop 2024-09-13T18:13:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.335+0000 7f34465d4700 1 -- 172.21.15.40:0/2223253325 shutdown_connections 2024-09-13T18:13:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.335+0000 7f34465d4700 1 --2- 172.21.15.40:0/2223253325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3440071260 0x7f344006bd40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.335+0000 7f34465d4700 1 -- 172.21.15.40:0/2223253325 >> 172.21.15.40:0/2223253325 conn(0x7f344006e6d0 msgr2=0x7f344006eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.335+0000 7f34465d4700 1 -- 172.21.15.40:0/2223253325 shutdown_connections 2024-09-13T18:13:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.335+0000 7f34465d4700 1 -- 172.21.15.40:0/2223253325 wait complete. 2024-09-13T18:13:21.360 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:13:21.360 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:13:21.360 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:13:21.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.459+0000 7f2463fff700 1 Processor -- start 2024-09-13T18:13:21.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.459+0000 7f2463fff700 1 -- start start 2024-09-13T18:13:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.459+0000 7f2463fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2464153790 0x7f24640713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.459+0000 7f2463fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f246415a1f0 con 0x7f24640cb6b0 2024-09-13T18:13:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.459+0000 7f2463fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24640cbcb0 con 0x7f24640cb290 2024-09-13T18:13:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.459+0000 7f2463fff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24641595e0 con 0x7f2464153790 2024-09-13T18:13:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.459+0000 7f24637fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f24640cb6b0 0x7f24640ca1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59166/0 (socket says 172.21.15.40:59166) 2024-09-13T18:13:21.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.459+0000 7f24637fe700 1 -- 172.21.15.40:0/517207616 learned_addr learned my addr 172.21.15.40:0/517207616 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:13:21.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.463+0000 7f2462ffd700 1 --2- 172.21.15.40:0/517207616 >> v2:172.21.15.43:3300/0 conn(0x7f2464153790 0x7f24640713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.463+0000 7f2461ffb700 1 -- 172.21.15.40:0/517207616 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 304309536 0 0) 0x7f246415a1f0 con 0x7f24640cb6b0 2024-09-13T18:13:21.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.463+0000 7f2461ffb700 1 -- 172.21.15.40:0/517207616 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f244c0035d0 con 0x7f24640cb6b0 2024-09-13T18:13:21.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.463+0000 7f2462ffd700 1 -- 172.21.15.40:0/517207616 >> v1:172.21.15.43:6789/0 conn(0x7f24640cb290 legacy=0x7f24640c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.463+0000 7f2461ffb700 1 -- 172.21.15.40:0/517207616 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 937098967 0 0) 0x7f24640cbcb0 con 0x7f24640cb290 2024-09-13T18:13:21.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.463+0000 7f2462ffd700 1 -- 172.21.15.40:0/517207616 >> v1:172.21.15.40:6789/0 conn(0x7f24640cb6b0 legacy=0x7f24640ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.463+0000 7f2462ffd700 1 -- 172.21.15.40:0/517207616 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24641598c0 con 0x7f2464153790 2024-09-13T18:13:21.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.463+0000 7f2462ffd700 1 --2- 172.21.15.40:0/517207616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2464153790 0x7f24640713e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2458009850 tx=0x7f2458009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15c16f904605206 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.463+0000 7f2461ffb700 1 -- 172.21.15.40:0/517207616 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2458011830 con 0x7f2464153790 2024-09-13T18:13:21.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.463+0000 7f2461ffb700 1 -- 172.21.15.40:0/517207616 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2458011990 con 0x7f2464153790 2024-09-13T18:13:21.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.463+0000 7f2461ffb700 1 -- 172.21.15.40:0/517207616 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2458011d40 con 0x7f2464153790 2024-09-13T18:13:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.463+0000 7f2463fff700 1 -- 172.21.15.40:0/517207616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2464153790 msgr2=0x7f24640713e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.463+0000 7f2463fff700 1 --2- 172.21.15.40:0/517207616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2464153790 0x7f24640713e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2458009850 tx=0x7f2458009b60 comp rx=0 tx=0).stop 2024-09-13T18:13:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.463+0000 7f2463fff700 1 -- 172.21.15.40:0/517207616 shutdown_connections 2024-09-13T18:13:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.463+0000 7f2463fff700 1 --2- 172.21.15.40:0/517207616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2464153790 0x7f24640713e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.463+0000 7f2463fff700 1 -- 172.21.15.40:0/517207616 >> 172.21.15.40:0/517207616 conn(0x7f246406dcf0 msgr2=0x7f246406e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:21.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.463+0000 7f2463fff700 1 -- 172.21.15.40:0/517207616 shutdown_connections 2024-09-13T18:13:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.463+0000 7f2463fff700 1 -- 172.21.15.40:0/517207616 wait complete. 2024-09-13T18:13:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f2463fff700 1 Processor -- start 2024-09-13T18:13:21.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f2463fff700 1 -- start start 2024-09-13T18:13:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f2463fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24640cb290 0x7f24641588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f2463fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24640cb6b0 0x7f24640c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f2463fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2464153790 0x7f24640c4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f2463fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24641f00a0 con 0x7f24640cb290 2024-09-13T18:13:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f2463fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24640c3790 con 0x7f2464153790 2024-09-13T18:13:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f2463fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24641efb20 con 0x7f24640cb6b0 2024-09-13T18:13:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f2462ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24640cb290 0x7f24641588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.477 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f2462ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24640cb290 0x7f24641588f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54350/0 (socket says 172.21.15.40:54350) 2024-09-13T18:13:21.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f2462ffd700 1 -- 172.21.15.40:0/3235261070 learned_addr learned my addr 172.21.15.40:0/3235261070 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:21.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f24637fe700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2464153790 0x7f24640c4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f24627fc700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24640cb6b0 0x7f24640c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f2462ffd700 1 -- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24640cb6b0 msgr2=0x7f24640c3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f2462ffd700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24640cb6b0 0x7f24640c3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f2462ffd700 1 -- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2464153790 msgr2=0x7f24640c4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f2462ffd700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2464153790 0x7f24640c4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f2462ffd700 1 -- 172.21.15.40:0/3235261070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24580094b0 con 0x7f24640cb290 2024-09-13T18:13:21.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f24637fe700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2464153790 0x7f24640c4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:13:21.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f2462ffd700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24640cb290 0x7f24641588f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f24580090b0 tx=0x7f2458004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f244bfff700 1 -- 172.21.15.40:0/3235261070 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2458019b60 con 0x7f24640cb290 2024-09-13T18:13:21.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f244bfff700 1 -- 172.21.15.40:0/3235261070 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245801e070 con 0x7f24640cb290 2024-09-13T18:13:21.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f2463fff700 1 -- 172.21.15.40:0/3235261070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24641efdc0 con 0x7f24640cb290 2024-09-13T18:13:21.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f244bfff700 1 -- 172.21.15.40:0/3235261070 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245801a420 con 0x7f24640cb290 2024-09-13T18:13:21.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.467+0000 7f2463fff700 1 -- 172.21.15.40:0/3235261070 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24640c3420 con 0x7f24640cb290 2024-09-13T18:13:21.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.471+0000 7f2463fff700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24640c4f20 0x7f24641f2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.471+0000 7f2463fff700 1 -- 172.21.15.40:0/3235261070 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2464067590 con 0x7f24640c4f20 2024-09-13T18:13:21.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.471+0000 7f24627fc700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24640c4f20 0x7f24641f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.471+0000 7f24627fc700 1 -- 172.21.15.40:0/3235261070 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2464153b70 con 0x7f24640c4f20 2024-09-13T18:13:21.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.471+0000 7f244bfff700 1 -- 172.21.15.40:0/3235261070 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f245801a580 con 0x7f24640cb290 2024-09-13T18:13:21.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.471+0000 7f24627fc700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24640c4f20 0x7f24641f2610 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f24640c29b0 tx=0x7f245400ea90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.471+0000 7f244bfff700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f24500667b0 0x7f2450068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.471+0000 7f244bfff700 1 -- 172.21.15.40:0/3235261070 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2458085460 con 0x7f24640cb290 2024-09-13T18:13:21.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.471+0000 7f24637fe700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f24500667b0 0x7f2450068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.471+0000 7f244bfff700 1 -- 172.21.15.40:0/3235261070 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24540052e0 con 0x7f24640c4f20 2024-09-13T18:13:21.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.471+0000 7f24637fe700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f24500667b0 0x7f2450068c70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f245c001560 tx=0x7f245c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.471+0000 7f244bfff700 1 -- 172.21.15.40:0/3235261070 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f2454005440 con 0x7f24640c4f20 2024-09-13T18:13:21.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.471+0000 7f244bfff700 1 -- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24640c4f20 msgr2=0x7f24641f2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.471+0000 7f244bfff700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24640c4f20 0x7f24641f2610 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f24640c29b0 tx=0x7f245400ea90 comp rx=0 tx=0).stop 2024-09-13T18:13:21.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.471+0000 7f244bfff700 1 -- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24640c4f20 msgr2=0x7f24641f2610 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:13:21.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.471+0000 7f244bfff700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24640c4f20 0x7f24641f2610 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.479+0000 7f2463fff700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24641e2390 0x7f24641f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.479+0000 7f2463fff700 1 -- 172.21.15.40:0/3235261070 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24640c3900 con 0x7f24641e2390 2024-09-13T18:13:21.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.483+0000 7f24627fc700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24641e2390 0x7f24641f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.483+0000 7f24627fc700 1 -- 172.21.15.40:0/3235261070 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2464159e80 con 0x7f24641e2390 2024-09-13T18:13:21.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.483+0000 7f24627fc700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24641e2390 0x7f24641f2b50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2454013000 tx=0x7f2454001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.483+0000 7f244bfff700 1 -- 172.21.15.40:0/3235261070 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2454003ab0 con 0x7f24641e2390 2024-09-13T18:13:21.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.483+0000 7f244bfff700 1 -- 172.21.15.40:0/3235261070 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7f2454003c10 con 0x7f24641e2390 2024-09-13T18:13:21.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.483+0000 7f244bfff700 1 -- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24641e2390 msgr2=0x7f24641f2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.483+0000 7f244bfff700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24641e2390 0x7f24641f2b50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2454013000 tx=0x7f2454001080 comp rx=0 tx=0).stop 2024-09-13T18:13:21.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.483+0000 7f244bfff700 1 -- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24641e2390 msgr2=0x7f24641f2b50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:13:21.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.483+0000 7f244bfff700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24641e2390 0x7f24641f2b50 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2454013000 tx=0x7f2454001080 comp rx=0 tx=0).stop 2024-09-13T18:13:21.493 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:13:21.493 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:13:21.493 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:13:21.493 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:13:21.493 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 186, 2024-09-13T18:13:21.493 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:13:21.493 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:13:21.493 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:13:21.493 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:13:21.493 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:21.493 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:13:21.493 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:13:21.493 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:13:21.494 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:13:21.494 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:21.494 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:21.494 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:21.494 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:21.494 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:21.494 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:21.494 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:21.494 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:21.494 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:21.494 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:21.494 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:21.494 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:13:21.495 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:13:21.495 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:21.495 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:21.495 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:21.495 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:21.495 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:21.495 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:21.495 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:21.495 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:21.495 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:21.495 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:21.495 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:21.495 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.496 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:13:21.496 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:13:21.496 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:13:21.496 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:13:21.496 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:13:21.496 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:13:21.496 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:13:21.496 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:13:21.496 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:13:21.496 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:13:21.496 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:13:21.496 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:13:21.497 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:13:21.497 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:13:21.497 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:13:21.497 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:13:21.497 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:13:21.497 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:21.497 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:21.497 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:21.497 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:21.497 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:21.497 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:21.497 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:21.497 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:21.497 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:21.498 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:21.498 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:21.498 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:13:21.498 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.498 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:13:21.498 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.498 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:13:21.498 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:13:21.498 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:21.498 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:21.498 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.498 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:21.498 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:13:21.498 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:21.498 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.499 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.499 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:21.499 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:13:21.499 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:21.499 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.499 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:21.499 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.499 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:13:21.499 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:13:21.499 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:21.499 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:21.499 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:21.499 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.500 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.500 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:13:21.500 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:13:21.500 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:21.500 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:21.500 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.500 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:21.500 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:13:21.500 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:21.500 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.500 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.500 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:21.501 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:13:21.501 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:21.501 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.501 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:21.501 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.501 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:13:21.501 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:13:21.501 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:21.501 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:21.501 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:21.501 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.501 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.501 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:13:21.501 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:13:21.501 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:21.502 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:21.502 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.502 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:21.502 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:13:21.502 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:21.502 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.502 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.502 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:21.502 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:13:21.502 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:21.502 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.502 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:21.502 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.502 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:13:21.502 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:13:21.503 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:21.503 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:21.503 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:21.503 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.503 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:21.503 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.503 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:13:21.503 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:13:21.503 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.503 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:21.503 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:21.503 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:13:21.503 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.503 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:21.503 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:13:21.504 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.504 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:21.504 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:21.504 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:13:21.504 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.504 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:21.504 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:13:21.504 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.504 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:21.504 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:21.504 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:13:21.504 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.505 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:21.505 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:13:21.505 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.505 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:21.505 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:21.505 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:13:21.505 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.505 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:21.505 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:13:21.505 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:21.505 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:21.505 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:21.505 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:13:21.505 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:21.506 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:21.506 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:21.506 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:13:21.506 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:13:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.483+0000 7f2463fff700 1 -- 172.21.15.40:0/3235261070 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f24500667b0 msgr2=0x7f2450068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.483+0000 7f2463fff700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f24500667b0 0x7f2450068c70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f245c001560 tx=0x7f245c009040 comp rx=0 tx=0).stop 2024-09-13T18:13:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.483+0000 7f2463fff700 1 -- 172.21.15.40:0/3235261070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24640cb290 msgr2=0x7f24641588f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.483+0000 7f2463fff700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24640cb290 0x7f24641588f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f24580090b0 tx=0x7f2458004770 comp rx=0 tx=0).stop 2024-09-13T18:13:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.483+0000 7f24637fe700 1 -- 172.21.15.40:0/3235261070 reap_dead start 2024-09-13T18:13:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.483+0000 7f2463fff700 1 -- 172.21.15.40:0/3235261070 shutdown_connections 2024-09-13T18:13:21.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.483+0000 7f2463fff700 1 --2- 172.21.15.40:0/3235261070 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24640cb290 0x7f24641588f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.483+0000 7f2463fff700 1 -- 172.21.15.40:0/3235261070 >> 172.21.15.40:0/3235261070 conn(0x7f246406dcf0 msgr2=0x7f2464152010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:21.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.483+0000 7f2463fff700 1 -- 172.21.15.40:0/3235261070 shutdown_connections 2024-09-13T18:13:21.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.483+0000 7f2463fff700 1 -- 172.21.15.40:0/3235261070 wait complete. 2024-09-13T18:13:21.508 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:13:21.508 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:13:21.508 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:13:21.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.611+0000 7fc13ffff700 1 Processor -- start 2024-09-13T18:13:21.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.611+0000 7fc13ffff700 1 -- start start 2024-09-13T18:13:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.615+0000 7fc13ffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc14006afd0 0x7fc140071f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.615+0000 7fc13ffff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc1400c2e60 0x7fc1400706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.615+0000 7fc13ffff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1400717d0 con 0x7fc14014fb00 2024-09-13T18:13:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.615+0000 7fc13ffff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc140071b20 con 0x7fc1400c2e60 2024-09-13T18:13:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.615+0000 7fc13ffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14006add0 con 0x7fc14006afd0 2024-09-13T18:13:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.615+0000 7fc13e7fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc14006afd0 0x7fc140071f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.615+0000 7fc13f7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc1400c2e60 0x7fc1400706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.615+0000 7fc13e7fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc14006afd0 0x7fc140071f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:42362/0 (socket says 172.21.15.40:42362) 2024-09-13T18:13:21.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.615+0000 7fc13effd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc14014fb00 0x7fc140158ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:49352/0 (socket says 172.21.15.40:49352) 2024-09-13T18:13:21.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.615+0000 7fc13e7fc700 1 -- 172.21.15.40:0/386190164 learned_addr learned my addr 172.21.15.40:0/386190164 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.615+0000 7fc13e7fc700 1 -- 172.21.15.40:0/386190164 >> v2:172.21.15.43:3300/0 conn(0x7fc1400c2e60 msgr2=0x7fc1400706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:13:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.615+0000 7fc13e7fc700 1 --2- 172.21.15.40:0/386190164 >> v2:172.21.15.43:3300/0 conn(0x7fc1400c2e60 0x7fc1400706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.615+0000 7fc13e7fc700 1 -- 172.21.15.40:0/386190164 >> v1:172.21.15.43:6789/0 conn(0x7fc14014fb00 legacy=0x7fc140158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.615+0000 7fc13e7fc700 1 -- 172.21.15.40:0/386190164 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1400c32f0 con 0x7fc14006afd0 2024-09-13T18:13:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.615+0000 7fc13f7fe700 1 --2- 172.21.15.40:0/386190164 >> v2:172.21.15.43:3300/0 conn(0x7fc1400c2e60 0x7fc1400706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:13:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.615+0000 7fc13e7fc700 1 --2- 172.21.15.40:0/386190164 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc14006afd0 0x7fc140071f40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc134009910 tx=0x7fc134009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce6dfc74f3fe3d0e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.615+0000 7fc13dffb700 1 -- 172.21.15.40:0/386190164 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc134011830 con 0x7fc14006afd0 2024-09-13T18:13:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.615+0000 7fc13dffb700 1 -- 172.21.15.40:0/386190164 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc134011990 con 0x7fc14006afd0 2024-09-13T18:13:21.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.615+0000 7fc13dffb700 1 -- 172.21.15.40:0/386190164 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc134011d40 con 0x7fc14006afd0 2024-09-13T18:13:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.619+0000 7fc13ffff700 1 -- 172.21.15.40:0/386190164 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc14006afd0 msgr2=0x7fc140071f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.619+0000 7fc13ffff700 1 --2- 172.21.15.40:0/386190164 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc14006afd0 0x7fc140071f40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc134009910 tx=0x7fc134009c20 comp rx=0 tx=0).stop 2024-09-13T18:13:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.619+0000 7fc13ffff700 1 -- 172.21.15.40:0/386190164 shutdown_connections 2024-09-13T18:13:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.619+0000 7fc13ffff700 1 --2- 172.21.15.40:0/386190164 >> v2:172.21.15.43:3300/0 conn(0x7fc1400c2e60 0x7fc1400706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.619+0000 7fc13ffff700 1 --2- 172.21.15.40:0/386190164 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc14006afd0 0x7fc140071f40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.619+0000 7fc13ffff700 1 -- 172.21.15.40:0/386190164 >> 172.21.15.40:0/386190164 conn(0x7fc1400cabc0 msgr2=0x7fc1400cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.619+0000 7fc13ffff700 1 -- 172.21.15.40:0/386190164 shutdown_connections 2024-09-13T18:13:21.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.619+0000 7fc13ffff700 1 -- 172.21.15.40:0/386190164 wait complete. 2024-09-13T18:13:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.619+0000 7fc13ffff700 1 Processor -- start 2024-09-13T18:13:21.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.619+0000 7fc13ffff700 1 -- start start 2024-09-13T18:13:21.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc13ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc14006afd0 0x7fc1401ed650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc13ffff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc1400c2e60 0x7fc1401f09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc13ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc14014fb00 0x7fc1401516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc13ffff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc140151c10 con 0x7fc14006afd0 2024-09-13T18:13:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc13ffff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1401593f0 con 0x7fc14014fb00 2024-09-13T18:13:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc13ffff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1401596d0 con 0x7fc1400c2e60 2024-09-13T18:13:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc13effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc14006afd0 0x7fc1401ed650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc13effd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc14006afd0 0x7fc1401ed650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54352/0 (socket says 172.21.15.40:54352) 2024-09-13T18:13:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc13effd700 1 -- 172.21.15.40:0/1544681152 learned_addr learned my addr 172.21.15.40:0/1544681152 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc13e7fc700 1 --2- 172.21.15.40:0/1544681152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc1400c2e60 0x7fc1401f09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc13f7fe700 1 --2- 172.21.15.40:0/1544681152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc14014fb00 0x7fc1401516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc13effd700 1 -- 172.21.15.40:0/1544681152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc1400c2e60 msgr2=0x7fc1401f09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc13effd700 1 --2- 172.21.15.40:0/1544681152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc1400c2e60 0x7fc1401f09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc13effd700 1 -- 172.21.15.40:0/1544681152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc14014fb00 msgr2=0x7fc1401516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc13effd700 1 --2- 172.21.15.40:0/1544681152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc14014fb00 0x7fc1401516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc13effd700 1 -- 172.21.15.40:0/1544681152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc138009a70 con 0x7fc14006afd0 2024-09-13T18:13:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc13f7fe700 1 --2- 172.21.15.40:0/1544681152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc14014fb00 0x7fc1401516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:13:21.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc13effd700 1 --2- 172.21.15.40:0/1544681152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc14006afd0 0x7fc1401ed650 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc13000c700 tx=0x7fc13000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc1277fe700 1 -- 172.21.15.40:0/1544681152 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130005cd0 con 0x7fc14006afd0 2024-09-13T18:13:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc1277fe700 1 -- 172.21.15.40:0/1544681152 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc130005e30 con 0x7fc14006afd0 2024-09-13T18:13:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc13ffff700 1 -- 172.21.15.40:0/1544681152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc134009570 con 0x7fc14006afd0 2024-09-13T18:13:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc1277fe700 1 -- 172.21.15.40:0/1544681152 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1300072d0 con 0x7fc14006afd0 2024-09-13T18:13:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.623+0000 7fc13ffff700 1 -- 172.21.15.40:0/1544681152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1401ecb70 con 0x7fc14006afd0 2024-09-13T18:13:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.627+0000 7fc1277fe700 1 -- 172.21.15.40:0/1544681152 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc130007430 con 0x7fc14006afd0 2024-09-13T18:13:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.627+0000 7fc13ffff700 1 -- 172.21.15.40:0/1544681152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc140159dc0 con 0x7fc14006afd0 2024-09-13T18:13:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.627+0000 7fc1277fe700 1 --2- 172.21.15.40:0/1544681152 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc12c066730 0x7fc12c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.627+0000 7fc13e7fc700 1 --2- 172.21.15.40:0/1544681152 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc12c066730 0x7fc12c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.627+0000 7fc1277fe700 1 -- 172.21.15.40:0/1544681152 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc130083860 con 0x7fc14006afd0 2024-09-13T18:13:21.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.627+0000 7fc13e7fc700 1 --2- 172.21.15.40:0/1544681152 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc12c066730 0x7fc12c068bf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc134009150 tx=0x7fc134008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:21.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.639+0000 7fc1277fe700 1 -- 172.21.15.40:0/1544681152 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc1300508c0 con 0x7fc14006afd0 2024-09-13T18:13:21.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.887+0000 7fc13ffff700 1 -- 172.21.15.40:0/1544681152 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fc140153320 con 0x7fc14006afd0 2024-09-13T18:13:21.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.887+0000 7fc1277fe700 1 -- 172.21.15.40:0/1544681152 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fc130017080 con 0x7fc14006afd0 2024-09-13T18:13:21.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.887+0000 7fc13ffff700 1 -- 172.21.15.40:0/1544681152 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc12c066730 msgr2=0x7fc12c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.887+0000 7fc13ffff700 1 --2- 172.21.15.40:0/1544681152 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc12c066730 0x7fc12c068bf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc134009150 tx=0x7fc134008030 comp rx=0 tx=0).stop 2024-09-13T18:13:21.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.887+0000 7fc13ffff700 1 -- 172.21.15.40:0/1544681152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc14006afd0 msgr2=0x7fc1401ed650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:21.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.887+0000 7fc13ffff700 1 --2- 172.21.15.40:0/1544681152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc14006afd0 0x7fc1401ed650 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc13000c700 tx=0x7fc13000ca10 comp rx=0 tx=0).stop 2024-09-13T18:13:21.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.891+0000 7fc13ffff700 1 -- 172.21.15.40:0/1544681152 shutdown_connections 2024-09-13T18:13:21.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.891+0000 7fc13ffff700 1 --2- 172.21.15.40:0/1544681152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc14014fb00 0x7fc1401516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.891+0000 7fc13ffff700 1 --2- 172.21.15.40:0/1544681152 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc12c066730 0x7fc12c068bf0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.891+0000 7fc13ffff700 1 --2- 172.21.15.40:0/1544681152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc1400c2e60 0x7fc1401f09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.891+0000 7fc13ffff700 1 --2- 172.21.15.40:0/1544681152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc14006afd0 0x7fc1401ed650 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:21.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.891+0000 7fc13ffff700 1 -- 172.21.15.40:0/1544681152 >> 172.21.15.40:0/1544681152 conn(0x7fc1400cabc0 msgr2=0x7fc14014cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:21.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.891+0000 7fc13ffff700 1 -- 172.21.15.40:0/1544681152 shutdown_connections 2024-09-13T18:13:21.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:21.891+0000 7fc13ffff700 1 -- 172.21.15.40:0/1544681152 wait complete. 2024-09-13T18:13:21.911 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:13:31.912 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:13:31.913 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:13:32.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.039+0000 7f4c4ef2e700 1 Processor -- start 2024-09-13T18:13:32.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.039+0000 7f4c4ef2e700 1 -- start start 2024-09-13T18:13:32.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4ef2e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4c400dea90 0x7f4c40006390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4ef2e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c400071b0 0x7f4c400068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4ef2e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c40046730 con 0x7f4c400d4ba0 2024-09-13T18:13:32.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4ef2e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c40007020 con 0x7f4c400dea90 2024-09-13T18:13:32.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4ef2e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c40046bc0 con 0x7f4c400071b0 2024-09-13T18:13:32.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4d72b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4c400dea90 0x7f4c40006390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4df2c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4c400d4ba0 0x7f4c4004a680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39592/0 (socket says 172.21.15.40:39592) 2024-09-13T18:13:32.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4d72b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4c400dea90 0x7f4c40006390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47310/0 (socket says 172.21.15.40:47310) 2024-09-13T18:13:32.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4df2c700 1 -- 172.21.15.40:0/3525025238 learned_addr learned my addr 172.21.15.40:0/3525025238 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:13:32.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4e72d700 1 --2- 172.21.15.40:0/3525025238 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c400071b0 0x7f4c400068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4d72b700 1 -- 172.21.15.40:0/3525025238 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c400071b0 msgr2=0x7f4c400068d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:13:32.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4d72b700 1 --2- 172.21.15.40:0/3525025238 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c400071b0 0x7f4c400068d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4d72b700 1 -- 172.21.15.40:0/3525025238 >> v1:172.21.15.40:6789/0 conn(0x7f4c400d4ba0 legacy=0x7f4c4004a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4cf2a700 1 -- 172.21.15.40:0/3525025238 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055721607 0 0) 0x7f4c40046730 con 0x7f4c400d4ba0 2024-09-13T18:13:32.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4d72b700 1 -- 172.21.15.40:0/3525025238 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c400defd0 con 0x7f4c400dea90 2024-09-13T18:13:32.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4e72d700 1 --2- 172.21.15.40:0/3525025238 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c400071b0 0x7f4c400068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:13:32.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4d72b700 1 --2- 172.21.15.40:0/3525025238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c400dea90 0x7f4c40006390 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009850 tx=0x7f4c38009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9304234d9d44bc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4cf2a700 1 -- 172.21.15.40:0/3525025238 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38011830 con 0x7f4c400dea90 2024-09-13T18:13:32.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4cf2a700 1 -- 172.21.15.40:0/3525025238 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c38011990 con 0x7f4c400dea90 2024-09-13T18:13:32.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4cf2a700 1 -- 172.21.15.40:0/3525025238 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38011d20 con 0x7f4c400dea90 2024-09-13T18:13:32.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4ef2e700 1 -- 172.21.15.40:0/3525025238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c400dea90 msgr2=0x7f4c40006390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4ef2e700 1 --2- 172.21.15.40:0/3525025238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c400dea90 0x7f4c40006390 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009850 tx=0x7f4c38009b60 comp rx=0 tx=0).stop 2024-09-13T18:13:32.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4ef2e700 1 -- 172.21.15.40:0/3525025238 shutdown_connections 2024-09-13T18:13:32.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4ef2e700 1 --2- 172.21.15.40:0/3525025238 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c400071b0 0x7f4c400068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4ef2e700 1 --2- 172.21.15.40:0/3525025238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c400dea90 0x7f4c40006390 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.054 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4ef2e700 1 -- 172.21.15.40:0/3525025238 >> 172.21.15.40:0/3525025238 conn(0x7f4c4004fd00 msgr2=0x7f4c40050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:32.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4ef2e700 1 -- 172.21.15.40:0/3525025238 shutdown_connections 2024-09-13T18:13:32.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.043+0000 7f4c4ef2e700 1 -- 172.21.15.40:0/3525025238 wait complete. 2024-09-13T18:13:32.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4ef2e700 1 Processor -- start 2024-09-13T18:13:32.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4ef2e700 1 -- start start 2024-09-13T18:13:32.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4ef2e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c400071b0 0x7f4c40049f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4ef2e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c400d4ba0 0x7f4c400dcf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4ef2e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c400dea90 0x7f4c40048e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4ef2e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4004a6a0 con 0x7f4c400071b0 2024-09-13T18:13:32.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4ef2e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c40170d50 con 0x7f4c400dea90 2024-09-13T18:13:32.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4ef2e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c40170ff0 con 0x7f4c400d4ba0 2024-09-13T18:13:32.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4df2c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c400071b0 0x7f4c40049f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4df2c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c400071b0 0x7f4c40049f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47314/0 (socket says 172.21.15.40:47314) 2024-09-13T18:13:32.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4df2c700 1 -- 172.21.15.40:0/2991348395 learned_addr learned my addr 172.21.15.40:0/2991348395 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:32.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4d72b700 1 --2- 172.21.15.40:0/2991348395 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c400d4ba0 0x7f4c400dcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4e72d700 1 --2- 172.21.15.40:0/2991348395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c400dea90 0x7f4c40048e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4df2c700 1 -- 172.21.15.40:0/2991348395 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c400d4ba0 msgr2=0x7f4c400dcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4df2c700 1 --2- 172.21.15.40:0/2991348395 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c400d4ba0 0x7f4c400dcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4df2c700 1 -- 172.21.15.40:0/2991348395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c400dea90 msgr2=0x7f4c40048e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4df2c700 1 --2- 172.21.15.40:0/2991348395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c400dea90 0x7f4c40048e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4df2c700 1 -- 172.21.15.40:0/2991348395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c380094b0 con 0x7f4c400071b0 2024-09-13T18:13:32.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4df2c700 1 --2- 172.21.15.40:0/2991348395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c400071b0 0x7f4c40049f90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400c580 tx=0x7f4c4400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c3e7fc700 1 -- 172.21.15.40:0/2991348395 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c44004ca0 con 0x7f4c400071b0 2024-09-13T18:13:32.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c3e7fc700 1 -- 172.21.15.40:0/2991348395 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c44004e20 con 0x7f4c400071b0 2024-09-13T18:13:32.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4ef2e700 1 -- 172.21.15.40:0/2991348395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c40048050 con 0x7f4c400071b0 2024-09-13T18:13:32.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c4ef2e700 1 -- 172.21.15.40:0/2991348395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c40048610 con 0x7f4c400071b0 2024-09-13T18:13:32.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.047+0000 7f4c3e7fc700 1 -- 172.21.15.40:0/2991348395 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c44007310 con 0x7f4c400071b0 2024-09-13T18:13:32.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.051+0000 7f4c4ef2e700 1 -- 172.21.15.40:0/2991348395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c30005310 con 0x7f4c400071b0 2024-09-13T18:13:32.060 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.051+0000 7f4c3e7fc700 1 -- 172.21.15.40:0/2991348395 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4c44007470 con 0x7f4c400071b0 2024-09-13T18:13:32.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.051+0000 7f4c3e7fc700 1 --2- 172.21.15.40:0/2991348395 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4c2c066730 0x7f4c2c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.051+0000 7f4c3e7fc700 1 -- 172.21.15.40:0/2991348395 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4c44084260 con 0x7f4c400071b0 2024-09-13T18:13:32.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.051+0000 7f4c4d72b700 1 --2- 172.21.15.40:0/2991348395 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4c2c066730 0x7f4c2c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.055+0000 7f4c4d72b700 1 --2- 172.21.15.40:0/2991348395 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4c2c066730 0x7f4c2c068bf0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4c38014040 tx=0x7f4c38008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.059+0000 7f4c3e7fc700 1 -- 172.21.15.40:0/2991348395 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4c44051180 con 0x7f4c400071b0 2024-09-13T18:13:32.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.315+0000 7f4c4ef2e700 1 -- 172.21.15.40:0/2991348395 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4c300055c0 con 0x7f4c400071b0 2024-09-13T18:13:32.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.315+0000 7f4c3e7fc700 1 -- 172.21.15.40:0/2991348395 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f4c44007720 con 0x7f4c400071b0 2024-09-13T18:13:32.323 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":186,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:13:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.319+0000 7f4c4ef2e700 1 -- 172.21.15.40:0/2991348395 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4c2c066730 msgr2=0x7f4c2c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.319+0000 7f4c4ef2e700 1 --2- 172.21.15.40:0/2991348395 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4c2c066730 0x7f4c2c068bf0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4c38014040 tx=0x7f4c38008030 comp rx=0 tx=0).stop 2024-09-13T18:13:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.319+0000 7f4c4ef2e700 1 -- 172.21.15.40:0/2991348395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c400071b0 msgr2=0x7f4c40049f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.319+0000 7f4c4ef2e700 1 --2- 172.21.15.40:0/2991348395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c400071b0 0x7f4c40049f90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400c580 tx=0x7f4c4400c890 comp rx=0 tx=0).stop 2024-09-13T18:13:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.319+0000 7f4c4ef2e700 1 -- 172.21.15.40:0/2991348395 shutdown_connections 2024-09-13T18:13:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.319+0000 7f4c4ef2e700 1 --2- 172.21.15.40:0/2991348395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c400dea90 0x7f4c40048e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.319+0000 7f4c4ef2e700 1 --2- 172.21.15.40:0/2991348395 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4c2c066730 0x7f4c2c068bf0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.319+0000 7f4c4ef2e700 1 --2- 172.21.15.40:0/2991348395 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4c400d4ba0 0x7f4c400dcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.319+0000 7f4c4ef2e700 1 --2- 172.21.15.40:0/2991348395 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4c400071b0 0x7f4c40049f90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.319+0000 7f4c4ef2e700 1 -- 172.21.15.40:0/2991348395 >> 172.21.15.40:0/2991348395 conn(0x7f4c4004fd00 msgr2=0x7f4c4004e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.319+0000 7f4c4ef2e700 1 -- 172.21.15.40:0/2991348395 shutdown_connections 2024-09-13T18:13:32.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.319+0000 7f4c4ef2e700 1 -- 172.21.15.40:0/2991348395 wait complete. 2024-09-13T18:13:32.342 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:13:32.342 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T18:13:32.342 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:13:32.342 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:13:32.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.447+0000 7f5a9cd57700 1 Processor -- start 2024-09-13T18:13:32.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.447+0000 7f5a9cd57700 1 -- start start 2024-09-13T18:13:32.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a9cd57700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5a98153720 0x7f5a980ca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a9cd57700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a980cf9b0 0x7f5a9806c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a9cd57700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5a9806c5b0 0x7f5a9806aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a9cd57700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a980c5ea0 con 0x7f5a98153720 2024-09-13T18:13:32.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a9cd57700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a98159e50 con 0x7f5a9806c5b0 2024-09-13T18:13:32.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a9cd57700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9815d390 con 0x7f5a980cf9b0 2024-09-13T18:13:32.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a977fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5a98153720 0x7f5a980ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a977fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5a98153720 0x7f5a980ca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47320/0 (socket says 172.21.15.40:47320) 2024-09-13T18:13:32.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a977fe700 1 -- 172.21.15.40:0/840974521 learned_addr learned my addr 172.21.15.40:0/840974521 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:32.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a96ffd700 1 --2- 172.21.15.40:0/840974521 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a980cf9b0 0x7f5a9806c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a97fff700 1 --2- 172.21.15.40:0/840974521 >> v2:172.21.15.43:3300/0 conn(0x7f5a9806c5b0 0x7f5a9806aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a977fe700 1 -- 172.21.15.40:0/840974521 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a980cf9b0 msgr2=0x7f5a9806c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:13:32.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a977fe700 1 --2- 172.21.15.40:0/840974521 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a980cf9b0 0x7f5a9806c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a977fe700 1 -- 172.21.15.40:0/840974521 >> v2:172.21.15.43:3300/0 conn(0x7f5a9806c5b0 msgr2=0x7f5a9806aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:13:32.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a977fe700 1 --2- 172.21.15.40:0/840974521 >> v2:172.21.15.43:3300/0 conn(0x7f5a9806c5b0 0x7f5a9806aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a977fe700 1 -- 172.21.15.40:0/840974521 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9815a080 con 0x7f5a98153720 2024-09-13T18:13:32.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a96ffd700 1 --2- 172.21.15.40:0/840974521 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a980cf9b0 0x7f5a9806c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:13:32.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a977fe700 1 --2- 172.21.15.40:0/840974521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a98153720 0x7f5a980ca510 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5a980b46a0 tx=0x7f5a88009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb14a492246ee29 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a967fc700 1 -- 172.21.15.40:0/840974521 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8800b8a0 con 0x7f5a98153720 2024-09-13T18:13:32.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a967fc700 1 -- 172.21.15.40:0/840974521 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8800ba00 con 0x7f5a98153720 2024-09-13T18:13:32.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a967fc700 1 -- 172.21.15.40:0/840974521 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8800bd90 con 0x7f5a98153720 2024-09-13T18:13:32.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a9cd57700 1 -- 172.21.15.40:0/840974521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a98153720 msgr2=0x7f5a980ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a9cd57700 1 --2- 172.21.15.40:0/840974521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a98153720 0x7f5a980ca510 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5a980b46a0 tx=0x7f5a88009bf0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a9cd57700 1 -- 172.21.15.40:0/840974521 shutdown_connections 2024-09-13T18:13:32.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a9cd57700 1 --2- 172.21.15.40:0/840974521 >> v2:172.21.15.43:3300/0 conn(0x7f5a9806c5b0 0x7f5a9806aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a9cd57700 1 --2- 172.21.15.40:0/840974521 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a980cf9b0 0x7f5a9806c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a9cd57700 1 --2- 172.21.15.40:0/840974521 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a98153720 0x7f5a980ca510 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a9cd57700 1 -- 172.21.15.40:0/840974521 >> 172.21.15.40:0/840974521 conn(0x7f5a980666c0 msgr2=0x7f5a980cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:32.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a9cd57700 1 -- 172.21.15.40:0/840974521 shutdown_connections 2024-09-13T18:13:32.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a9cd57700 1 -- 172.21.15.40:0/840974521 wait complete. 2024-09-13T18:13:32.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a9cd57700 1 Processor -- start 2024-09-13T18:13:32.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.451+0000 7f5a9cd57700 1 -- start start 2024-09-13T18:13:32.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a9cd57700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a9806c5b0 0x7f5a98153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a9cd57700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a980cf9b0 0x7f5a980ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a9cd57700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a98153720 0x7f5a981f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a9cd57700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a981efa80 con 0x7f5a98153720 2024-09-13T18:13:32.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a9cd57700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9815c370 con 0x7f5a9806c5b0 2024-09-13T18:13:32.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a9cd57700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a98157ea0 con 0x7f5a980cf9b0 2024-09-13T18:13:32.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a97fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a98153720 0x7f5a981f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a97fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a98153720 0x7f5a981f28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47334/0 (socket says 172.21.15.40:47334) 2024-09-13T18:13:32.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a977fe700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a9806c5b0 0x7f5a98153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a97fff700 1 -- 172.21.15.40:0/2357015427 learned_addr learned my addr 172.21.15.40:0/2357015427 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:32.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a96ffd700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a980cf9b0 0x7f5a980ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a97fff700 1 -- 172.21.15.40:0/2357015427 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a980cf9b0 msgr2=0x7f5a980ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a97fff700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5a980cf9b0 0x7f5a980ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a97fff700 1 -- 172.21.15.40:0/2357015427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a9806c5b0 msgr2=0x7f5a98153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a97fff700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a9806c5b0 0x7f5a98153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a97fff700 1 -- 172.21.15.40:0/2357015427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a88009570 con 0x7f5a98153720 2024-09-13T18:13:32.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a97fff700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a98153720 0x7f5a981f28e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5a90007e90 tx=0x7f5a9000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a7ffff700 1 -- 172.21.15.40:0/2357015427 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9000ed30 con 0x7f5a98153720 2024-09-13T18:13:32.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a7ffff700 1 -- 172.21.15.40:0/2357015427 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9000eeb0 con 0x7f5a98153720 2024-09-13T18:13:32.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a9cd57700 1 -- 172.21.15.40:0/2357015427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9815c0b0 con 0x7f5a98153720 2024-09-13T18:13:32.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a7ffff700 1 -- 172.21.15.40:0/2357015427 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a90014640 con 0x7f5a98153720 2024-09-13T18:13:32.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.455+0000 7f5a9cd57700 1 -- 172.21.15.40:0/2357015427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a980cec70 con 0x7f5a98153720 2024-09-13T18:13:32.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.459+0000 7f5a9cd57700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a98157030 0x7f5a9815c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.459+0000 7f5a9cd57700 1 -- 172.21.15.40:0/2357015427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a98074750 con 0x7f5a98157030 2024-09-13T18:13:32.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.459+0000 7f5a7ffff700 1 -- 172.21.15.40:0/2357015427 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5a900147a0 con 0x7f5a98153720 2024-09-13T18:13:32.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.459+0000 7f5a977fe700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a98157030 0x7f5a9815c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.459+0000 7f5a977fe700 1 -- 172.21.15.40:0/2357015427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a98153b00 con 0x7f5a98157030 2024-09-13T18:13:32.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.459+0000 7f5a7ffff700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5a84066760 0x7f5a84068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.459+0000 7f5a7ffff700 1 -- 172.21.15.40:0/2357015427 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5a90084b30 con 0x7f5a98153720 2024-09-13T18:13:32.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.459+0000 7f5a96ffd700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5a84066760 0x7f5a84068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.459+0000 7f5a977fe700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a98157030 0x7f5a9815c5f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5a980b46a0 tx=0x7f5a88004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.459+0000 7f5a7ffff700 1 -- 172.21.15.40:0/2357015427 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8801f070 con 0x7f5a98157030 2024-09-13T18:13:32.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.459+0000 7f5a7ffff700 1 -- 172.21.15.40:0/2357015427 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5a88004320 con 0x7f5a98157030 2024-09-13T18:13:32.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.459+0000 7f5a96ffd700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5a84066760 0x7f5a84068c20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5a980cf150 tx=0x7f5a8c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.459+0000 7f5a7ffff700 1 -- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a98157030 msgr2=0x7f5a9815c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.459+0000 7f5a7ffff700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a98157030 0x7f5a9815c5f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5a980b46a0 tx=0x7f5a88004770 comp rx=0 tx=0).stop 2024-09-13T18:13:32.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.459+0000 7f5a7ffff700 1 -- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a98157030 msgr2=0x7f5a9815c5f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:13:32.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.459+0000 7f5a7ffff700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a98157030 0x7f5a9815c5f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.471+0000 7f5a9cd57700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a981f2e20 0x7f5a9815a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.471+0000 7f5a9cd57700 1 -- 172.21.15.40:0/2357015427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9815cd40 con 0x7f5a981f2e20 2024-09-13T18:13:32.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.471+0000 7f5a977fe700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a981f2e20 0x7f5a9815a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.471+0000 7f5a977fe700 1 -- 172.21.15.40:0/2357015427 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5a98067a40 con 0x7f5a981f2e20 2024-09-13T18:13:32.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.471+0000 7f5a977fe700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a981f2e20 0x7f5a9815a590 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009150 tx=0x7f5a88011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.471+0000 7f5a7ffff700 1 -- 172.21.15.40:0/2357015427 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8802c850 con 0x7f5a981f2e20 2024-09-13T18:13:32.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.471+0000 7f5a7ffff700 1 -- 172.21.15.40:0/2357015427 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4453 (secure 0 0 0) 0x7f5a88030070 con 0x7f5a981f2e20 2024-09-13T18:13:32.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.471+0000 7f5a7ffff700 1 -- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a981f2e20 msgr2=0x7f5a9815a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.471+0000 7f5a7ffff700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a981f2e20 0x7f5a9815a590 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009150 tx=0x7f5a88011000 comp rx=0 tx=0).stop 2024-09-13T18:13:32.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.471+0000 7f5a7ffff700 1 -- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a981f2e20 msgr2=0x7f5a9815a590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:13:32.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.471+0000 7f5a7ffff700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a981f2e20 0x7f5a9815a590 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.482 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:13:32.482 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:13:32.482 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:13:32.482 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:13:32.483 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 186, 2024-09-13T18:13:32.483 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:13:32.483 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:13:32.483 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:13:32.483 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:13:32.483 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:32.483 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 11, 2024-09-13T18:13:32.483 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:13:32.483 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:13:32.483 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:13:32.483 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:32.483 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:32.483 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:32.484 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:32.484 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:32.484 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:32.484 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:32.484 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:32.484 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:32.484 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:32.484 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:32.484 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:13:32.484 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:13:32.484 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:32.484 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:32.484 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:32.484 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:32.485 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:32.485 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:32.485 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:32.485 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:32.485 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:32.485 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:32.485 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:32.485 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.485 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:13:32.485 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:13:32.485 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:13:32.485 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:13:32.485 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:13:32.486 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:13:32.486 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:13:32.486 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:13:32.486 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:13:32.486 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:13:32.486 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:13:32.486 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:13:32.486 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:13:32.486 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:13:32.486 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:13:32.486 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:13:32.486 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:13:32.486 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:32.486 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:13:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:13:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.487 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:13:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:13:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:13:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:13:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:32.488 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:13:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:13:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:13:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:13:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:32.489 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.490 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:32.490 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:13:32.490 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:32.490 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.490 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.490 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:32.490 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:13:32.490 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:32.490 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.490 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:32.490 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.490 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:13:32.490 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:13:32.490 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:32.491 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:32.491 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:32.491 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.491 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.491 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:13:32.491 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:13:32.491 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:32.491 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:32.491 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.491 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:32.491 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:13:32.491 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:32.491 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.491 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.492 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:32.492 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:13:32.492 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:32.492 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.492 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:32.492 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.492 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:13:32.492 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:13:32.492 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:32.492 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:32.492 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:32.492 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.492 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:32.492 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.493 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:13:32.493 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:13:32.493 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.493 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:32.493 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:32.493 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:13:32.493 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.493 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:32.493 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:13:32.493 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.493 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:32.493 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:32.493 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:13:32.493 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.493 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:32.494 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.494 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:13:32.494 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:13:32.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.471+0000 7f5a9cd57700 1 -- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5a84066760 msgr2=0x7f5a84068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.471+0000 7f5a9cd57700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5a84066760 0x7f5a84068c20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5a980cf150 tx=0x7f5a8c008040 comp rx=0 tx=0).stop 2024-09-13T18:13:32.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.471+0000 7f5a9cd57700 1 -- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a98153720 msgr2=0x7f5a981f28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.471+0000 7f5a9cd57700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a98153720 0x7f5a981f28e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5a90007e90 tx=0x7f5a9000e3c0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.471+0000 7f5a97fff700 1 -- 172.21.15.40:0/2357015427 reap_dead start 2024-09-13T18:13:32.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.471+0000 7f5a9cd57700 1 -- 172.21.15.40:0/2357015427 shutdown_connections 2024-09-13T18:13:32.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.471+0000 7f5a9cd57700 1 --2- 172.21.15.40:0/2357015427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a98153720 0x7f5a981f28e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.471+0000 7f5a9cd57700 1 -- 172.21.15.40:0/2357015427 >> 172.21.15.40:0/2357015427 conn(0x7f5a980666c0 msgr2=0x7f5a98151d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:32.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.475+0000 7f5a9cd57700 1 -- 172.21.15.40:0/2357015427 shutdown_connections 2024-09-13T18:13:32.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.475+0000 7f5a9cd57700 1 -- 172.21.15.40:0/2357015427 wait complete. 2024-09-13T18:13:32.498 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:13:32.498 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:13:32.498 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:13:32.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f735e2a9700 1 Processor -- start 2024-09-13T18:13:32.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f735e2a9700 1 -- start start 2024-09-13T18:13:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f735e2a9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7358153730 0x7f7358159f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f735e2a9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f735815cfe0 0x7f735815d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f735e2a9700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735815a450 con 0x7f73580c7fb0 2024-09-13T18:13:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f735e2a9700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f735815d9d0 con 0x7f735815cfe0 2024-09-13T18:13:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f735e2a9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7358070ae0 con 0x7f7358153730 2024-09-13T18:13:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f735daa8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f735815cfe0 0x7f735815d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f735caa6700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f73580c7fb0 0x7f73580c8630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39608/0 (socket says 172.21.15.40:39608) 2024-09-13T18:13:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f735caa6700 1 -- 172.21.15.40:0/3022783172 learned_addr learned my addr 172.21.15.40:0/3022783172 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:13:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f735d2a7700 1 --2- 172.21.15.40:0/3022783172 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7358153730 0x7f7358159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f734ffff700 1 -- 172.21.15.40:0/3022783172 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 678049747 0 0) 0x7f7348001110 con 0x7f73580c7fb0 2024-09-13T18:13:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f734ffff700 1 -- 172.21.15.40:0/3022783172 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73440035d0 con 0x7f73580c7fb0 2024-09-13T18:13:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f735d2a7700 1 -- 172.21.15.40:0/3022783172 >> v2:172.21.15.40:3300/0 conn(0x7f735815cfe0 msgr2=0x7f735815d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:13:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f735d2a7700 1 --2- 172.21.15.40:0/3022783172 >> v2:172.21.15.40:3300/0 conn(0x7f735815cfe0 0x7f735815d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f735d2a7700 1 -- 172.21.15.40:0/3022783172 >> v1:172.21.15.40:6789/0 conn(0x7f73580c7fb0 legacy=0x7f73580c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f735d2a7700 1 -- 172.21.15.40:0/3022783172 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7358071010 con 0x7f7358153730 2024-09-13T18:13:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f735d2a7700 1 --2- 172.21.15.40:0/3022783172 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7358153730 0x7f7358159f10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7354009850 tx=0x7f7354009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4335f71c0b0e0da2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f734ffff700 1 -- 172.21.15.40:0/3022783172 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354011830 con 0x7f7358153730 2024-09-13T18:13:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f734ffff700 1 -- 172.21.15.40:0/3022783172 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7354011990 con 0x7f7358153730 2024-09-13T18:13:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.607+0000 7f734ffff700 1 -- 172.21.15.40:0/3022783172 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354011d00 con 0x7f7358153730 2024-09-13T18:13:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.611+0000 7f735e2a9700 1 -- 172.21.15.40:0/3022783172 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7358153730 msgr2=0x7f7358159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.611+0000 7f735e2a9700 1 --2- 172.21.15.40:0/3022783172 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7358153730 0x7f7358159f10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7354009850 tx=0x7f7354009b60 comp rx=0 tx=0).stop 2024-09-13T18:13:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.611+0000 7f735e2a9700 1 -- 172.21.15.40:0/3022783172 shutdown_connections 2024-09-13T18:13:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.611+0000 7f735e2a9700 1 --2- 172.21.15.40:0/3022783172 >> v2:172.21.15.40:3300/0 conn(0x7f735815cfe0 0x7f735815d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.611+0000 7f735e2a9700 1 --2- 172.21.15.40:0/3022783172 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7358153730 0x7f7358159f10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.611+0000 7f735e2a9700 1 -- 172.21.15.40:0/3022783172 >> 172.21.15.40:0/3022783172 conn(0x7f735806e6d0 msgr2=0x7f735806eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.611+0000 7f735e2a9700 1 -- 172.21.15.40:0/3022783172 shutdown_connections 2024-09-13T18:13:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.611+0000 7f735e2a9700 1 -- 172.21.15.40:0/3022783172 wait complete. 2024-09-13T18:13:32.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.611+0000 7f735e2a9700 1 Processor -- start 2024-09-13T18:13:32.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.611+0000 7f735e2a9700 1 -- start start 2024-09-13T18:13:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735e2a9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73580c7fb0 0x7f73581f3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735e2a9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7358153730 0x7f73581efad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735e2a9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f735815cfe0 0x7f735814ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735e2a9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73580c5ff0 con 0x7f73580c7fb0 2024-09-13T18:13:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735e2a9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73581ef0c0 con 0x7f735815cfe0 2024-09-13T18:13:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735e2a9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73581ef640 con 0x7f7358153730 2024-09-13T18:13:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735d2a7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73580c7fb0 0x7f73581f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735d2a7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73580c7fb0 0x7f73581f3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47380/0 (socket says 172.21.15.40:47380) 2024-09-13T18:13:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735d2a7700 1 -- 172.21.15.40:0/2252133368 learned_addr learned my addr 172.21.15.40:0/2252133368 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735caa6700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7358153730 0x7f73581efad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735daa8700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f735815cfe0 0x7f735814ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735d2a7700 1 -- 172.21.15.40:0/2252133368 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7358153730 msgr2=0x7f73581efad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735d2a7700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7358153730 0x7f73581efad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735d2a7700 1 -- 172.21.15.40:0/2252133368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f735815cfe0 msgr2=0x7f735814ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735d2a7700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f735815cfe0 0x7f735814ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735d2a7700 1 -- 172.21.15.40:0/2252133368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73540094b0 con 0x7f73580c7fb0 2024-09-13T18:13:32.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735d2a7700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73580c7fb0 0x7f73581f3040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f73540090b0 tx=0x7f7354004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f734dffb700 1 -- 172.21.15.40:0/2252133368 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73540044d0 con 0x7f73580c7fb0 2024-09-13T18:13:32.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735e2a9700 1 -- 172.21.15.40:0/2252133368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73580c41f0 con 0x7f73580c7fb0 2024-09-13T18:13:32.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735e2a9700 1 -- 172.21.15.40:0/2252133368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73580c5d10 con 0x7f73580c7fb0 2024-09-13T18:13:32.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f734dffb700 1 -- 172.21.15.40:0/2252133368 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735401f070 con 0x7f73580c7fb0 2024-09-13T18:13:32.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f734dffb700 1 -- 172.21.15.40:0/2252133368 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354003900 con 0x7f73580c7fb0 2024-09-13T18:13:32.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735e2a9700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73580c2ef0 0x7f7358153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735e2a9700 1 -- 172.21.15.40:0/2252133368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73580675e0 con 0x7f73580c2ef0 2024-09-13T18:13:32.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.615+0000 7f735caa6700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73580c2ef0 0x7f7358153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.619+0000 7f735caa6700 1 -- 172.21.15.40:0/2252133368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7358071e60 con 0x7f73580c2ef0 2024-09-13T18:13:32.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.619+0000 7f735caa6700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73580c2ef0 0x7f7358153b10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7358153140 tx=0x7f734800fb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.619+0000 7f734dffb700 1 -- 172.21.15.40:0/2252133368 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7354003a60 con 0x7f73580c7fb0 2024-09-13T18:13:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.619+0000 7f734dffb700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f733c066730 0x7f733c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.619+0000 7f735daa8700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f733c066730 0x7f733c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.619+0000 7f734dffb700 1 -- 172.21.15.40:0/2252133368 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7354021070 con 0x7f73580c7fb0 2024-09-13T18:13:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.619+0000 7f734dffb700 1 -- 172.21.15.40:0/2252133368 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7348017070 con 0x7f73580c2ef0 2024-09-13T18:13:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.619+0000 7f734dffb700 1 -- 172.21.15.40:0/2252133368 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7348005b70 con 0x7f73580c2ef0 2024-09-13T18:13:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.619+0000 7f734dffb700 1 -- 172.21.15.40:0/2252133368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73580c2ef0 msgr2=0x7f7358153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.619+0000 7f734dffb700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73580c2ef0 0x7f7358153b10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7358153140 tx=0x7f734800fb00 comp rx=0 tx=0).stop 2024-09-13T18:13:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.619+0000 7f734dffb700 1 -- 172.21.15.40:0/2252133368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73580c2ef0 msgr2=0x7f7358153b10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:13:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.619+0000 7f734dffb700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73580c2ef0 0x7f7358153b10 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7358153140 tx=0x7f734800fb00 comp rx=0 tx=0).stop 2024-09-13T18:13:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.619+0000 7f735daa8700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f733c066730 0x7f733c068bf0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7350011340 tx=0x7f735000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.627+0000 7f735e2a9700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7358150510 0x7f7358154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.627+0000 7f735e2a9700 1 -- 172.21.15.40:0/2252133368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7358070d30 con 0x7f7358150510 2024-09-13T18:13:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.627+0000 7f735caa6700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7358150510 0x7f7358154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.627+0000 7f735caa6700 1 -- 172.21.15.40:0/2252133368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7358072140 con 0x7f7358150510 2024-09-13T18:13:32.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.627+0000 7f735caa6700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7358150510 0x7f7358154050 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7348007fd0 tx=0x7f7348000ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.627+0000 7f734dffb700 1 -- 172.21.15.40:0/2252133368 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7348022070 con 0x7f7358150510 2024-09-13T18:13:32.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.627+0000 7f734dffb700 1 -- 172.21.15.40:0/2252133368 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4783 (secure 0 0 0) 0x7f73480073f0 con 0x7f7358150510 2024-09-13T18:13:32.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.631+0000 7f734dffb700 1 -- 172.21.15.40:0/2252133368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7358150510 msgr2=0x7f7358154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.631+0000 7f734dffb700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7358150510 0x7f7358154050 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7348007fd0 tx=0x7f7348000ed0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.631+0000 7f734dffb700 1 -- 172.21.15.40:0/2252133368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7358150510 msgr2=0x7f7358154050 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:13:32.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.631+0000 7f734dffb700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7358150510 0x7f7358154050 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7348007fd0 tx=0x7f7348000ed0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.639 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:13:32.639 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:13:32.639 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:13:32.639 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:13:32.639 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 186, 2024-09-13T18:13:32.639 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:13:32.639 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:13:32.639 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:13:32.639 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:13:32.639 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:32.640 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 12, 2024-09-13T18:13:32.640 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:13:32.640 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:13:32.640 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:13:32.640 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:32.640 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:32.640 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:32.640 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:32.640 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:32.640 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:32.640 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:32.640 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:32.640 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:32.641 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:32.641 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:32.641 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:13:32.641 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:13:32.641 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:32.641 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:32.641 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:32.641 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:32.641 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:32.641 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:32.641 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:32.641 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:32.641 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:32.641 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:32.642 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:32.642 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.642 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:13:32.642 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:13:32.642 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:13:32.642 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:13:32.642 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:13:32.642 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:13:32.642 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:13:32.642 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:13:32.642 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:13:32.642 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:13:32.642 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:13:32.642 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:13:32.643 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:13:32.643 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:13:32.643 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:13:32.643 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:13:32.643 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:13:32.643 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:32.643 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:32.643 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:32.643 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:32.643 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:32.643 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:32.643 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:32.643 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:32.643 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:32.643 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:32.644 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:32.644 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:13:32.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.644 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:13:32.644 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.644 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:13:32.644 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:13:32.644 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:32.644 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:32.644 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.644 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:32.644 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:13:32.644 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:32.644 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.644 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.645 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:32.645 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:13:32.645 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:32.645 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.645 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:32.645 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.645 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:13:32.645 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:13:32.645 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:32.645 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:32.645 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:32.645 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.645 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.645 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:13:32.646 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:13:32.646 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:32.646 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:32.646 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.646 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:32.646 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:13:32.646 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:32.646 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.646 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.646 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:32.646 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:13:32.646 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:32.646 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.646 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:32.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.647 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:13:32.647 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:13:32.647 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:32.647 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:32.647 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:32.647 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.647 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.647 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:13:32.647 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:13:32.647 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:32.647 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:32.647 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.647 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:32.647 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:13:32.648 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:32.648 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.648 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.648 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:32.648 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:13:32.648 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:32.648 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.648 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:32.648 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.648 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:13:32.648 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:13:32.648 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:32.648 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:32.648 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:32.649 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.649 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:32.649 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.649 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:13:32.649 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:13:32.649 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.649 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:32.649 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:32.649 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:13:32.649 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.649 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:32.649 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:13:32.649 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.650 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:32.650 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:32.650 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:13:32.650 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.650 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:32.650 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:13:32.650 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.650 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:32.650 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:32.650 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:13:32.650 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.650 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:32.650 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:13:32.650 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.651 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:32.651 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:32.651 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:13:32.651 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.651 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:32.651 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.651 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:13:32.651 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:13:32.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.631+0000 7f735e2a9700 1 -- 172.21.15.40:0/2252133368 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f733c066730 msgr2=0x7f733c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.631+0000 7f735e2a9700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f733c066730 0x7f733c068bf0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7350011340 tx=0x7f735000b040 comp rx=0 tx=0).stop 2024-09-13T18:13:32.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.631+0000 7f735e2a9700 1 -- 172.21.15.40:0/2252133368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73580c7fb0 msgr2=0x7f73581f3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.631+0000 7f735e2a9700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73580c7fb0 0x7f73581f3040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f73540090b0 tx=0x7f7354004770 comp rx=0 tx=0).stop 2024-09-13T18:13:32.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.631+0000 7f735daa8700 1 -- 172.21.15.40:0/2252133368 reap_dead start 2024-09-13T18:13:32.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.631+0000 7f735e2a9700 1 -- 172.21.15.40:0/2252133368 shutdown_connections 2024-09-13T18:13:32.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.631+0000 7f735e2a9700 1 --2- 172.21.15.40:0/2252133368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73580c7fb0 0x7f73581f3040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.631+0000 7f735e2a9700 1 -- 172.21.15.40:0/2252133368 >> 172.21.15.40:0/2252133368 conn(0x7f735806e6d0 msgr2=0x7f73581525e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:32.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.631+0000 7f735e2a9700 1 -- 172.21.15.40:0/2252133368 shutdown_connections 2024-09-13T18:13:32.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.631+0000 7f735e2a9700 1 -- 172.21.15.40:0/2252133368 wait complete. 2024-09-13T18:13:32.659 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:13:32.659 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:13:32.659 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:13:32.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.763+0000 7f97a7fff700 1 Processor -- start 2024-09-13T18:13:32.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.763+0000 7f97a7fff700 1 -- start start 2024-09-13T18:13:32.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a7fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f97a00d8ad0 0x7f97a004cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a7fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a00e1cc0 con 0x7f97a004d240 2024-09-13T18:13:32.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a7fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a00e1f80 con 0x7f97a0047ec0 2024-09-13T18:13:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a7fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a004d640 con 0x7f97a00d8ad0 2024-09-13T18:13:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a67fc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f97a004d240 0x7f97a00477b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39616/0 (socket says 172.21.15.40:39616) 2024-09-13T18:13:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a67fc700 1 -- 172.21.15.40:0/4142152649 learned_addr learned my addr 172.21.15.40:0/4142152649 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:13:32.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a6ffd700 1 --2- 172.21.15.40:0/4142152649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f97a00d8ad0 0x7f97a004cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a5ffb700 1 -- 172.21.15.40:0/4142152649 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856509397 0 0) 0x7f97a00e1cc0 con 0x7f97a004d240 2024-09-13T18:13:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a5ffb700 1 -- 172.21.15.40:0/4142152649 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97900035d0 con 0x7f97a004d240 2024-09-13T18:13:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a5ffb700 1 -- 172.21.15.40:0/4142152649 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2644853215 0 0) 0x7f97a00e1f80 con 0x7f97a0047ec0 2024-09-13T18:13:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a5ffb700 1 -- 172.21.15.40:0/4142152649 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97a00e1cc0 con 0x7f97a0047ec0 2024-09-13T18:13:32.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a6ffd700 1 -- 172.21.15.40:0/4142152649 >> v1:172.21.15.43:6789/0 conn(0x7f97a0047ec0 legacy=0x7f97a00088d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a6ffd700 1 -- 172.21.15.40:0/4142152649 >> v1:172.21.15.40:6789/0 conn(0x7f97a004d240 legacy=0x7f97a00477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a6ffd700 1 -- 172.21.15.40:0/4142152649 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a00e2260 con 0x7f97a00d8ad0 2024-09-13T18:13:32.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a6ffd700 1 --2- 172.21.15.40:0/4142152649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f97a00d8ad0 0x7f97a004cd00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f979c009880 tx=0x7f979c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61d6e695c01ed451 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a5ffb700 1 -- 172.21.15.40:0/4142152649 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c011830 con 0x7f97a00d8ad0 2024-09-13T18:13:32.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a5ffb700 1 -- 172.21.15.40:0/4142152649 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c011990 con 0x7f97a00d8ad0 2024-09-13T18:13:32.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a5ffb700 1 -- 172.21.15.40:0/4142152649 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c011d20 con 0x7f97a00d8ad0 2024-09-13T18:13:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a7fff700 1 -- 172.21.15.40:0/4142152649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f97a00d8ad0 msgr2=0x7f97a004cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a7fff700 1 --2- 172.21.15.40:0/4142152649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f97a00d8ad0 0x7f97a004cd00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f979c009880 tx=0x7f979c009b90 comp rx=0 tx=0).stop 2024-09-13T18:13:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a7fff700 1 -- 172.21.15.40:0/4142152649 shutdown_connections 2024-09-13T18:13:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a7fff700 1 --2- 172.21.15.40:0/4142152649 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f97a00d8ad0 0x7f97a004cd00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a7fff700 1 -- 172.21.15.40:0/4142152649 >> 172.21.15.40:0/4142152649 conn(0x7f97a000dc40 msgr2=0x7f97a000e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a7fff700 1 -- 172.21.15.40:0/4142152649 shutdown_connections 2024-09-13T18:13:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.767+0000 7f97a7fff700 1 -- 172.21.15.40:0/4142152649 wait complete. 2024-09-13T18:13:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f97a7fff700 1 Processor -- start 2024-09-13T18:13:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f97a7fff700 1 -- start start 2024-09-13T18:13:32.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f97a7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97a0047ec0 0x7f97a004b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f97a7fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f97a004d240 0x7f97a0050eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f97a7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97a00d8ad0 0x7f97a00513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f97a7fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a0174e90 con 0x7f97a0047ec0 2024-09-13T18:13:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f97a7fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a01776e0 con 0x7f97a00d8ad0 2024-09-13T18:13:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f97a7fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a0177980 con 0x7f97a004d240 2024-09-13T18:13:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f97a6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97a0047ec0 0x7f97a004b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f97a6ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97a0047ec0 0x7f97a004b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47384/0 (socket says 172.21.15.40:47384) 2024-09-13T18:13:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f97a6ffd700 1 -- 172.21.15.40:0/1849824196 learned_addr learned my addr 172.21.15.40:0/1849824196 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f97a67fc700 1 --2- 172.21.15.40:0/1849824196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f97a004d240 0x7f97a0050eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f97a77fe700 1 --2- 172.21.15.40:0/1849824196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97a00d8ad0 0x7f97a00513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f97a6ffd700 1 -- 172.21.15.40:0/1849824196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f97a004d240 msgr2=0x7f97a0050eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f97a6ffd700 1 --2- 172.21.15.40:0/1849824196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f97a004d240 0x7f97a0050eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f97a6ffd700 1 -- 172.21.15.40:0/1849824196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97a00d8ad0 msgr2=0x7f97a00513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f97a6ffd700 1 --2- 172.21.15.40:0/1849824196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97a00d8ad0 0x7f97a00513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f97a6ffd700 1 -- 172.21.15.40:0/1849824196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c0094b0 con 0x7f97a0047ec0 2024-09-13T18:13:32.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f97a6ffd700 1 --2- 172.21.15.40:0/1849824196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97a0047ec0 0x7f97a004b650 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f979c013040 tx=0x7f979c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f978f7fe700 1 -- 172.21.15.40:0/1849824196 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c004510 con 0x7f97a0047ec0 2024-09-13T18:13:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.771+0000 7f978f7fe700 1 -- 172.21.15.40:0/1849824196 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c0036c0 con 0x7f97a0047ec0 2024-09-13T18:13:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.775+0000 7f97a7fff700 1 -- 172.21.15.40:0/1849824196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a0174280 con 0x7f97a0047ec0 2024-09-13T18:13:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.775+0000 7f97a7fff700 1 -- 172.21.15.40:0/1849824196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a0174840 con 0x7f97a0047ec0 2024-09-13T18:13:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.775+0000 7f978f7fe700 1 -- 172.21.15.40:0/1849824196 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c003a70 con 0x7f97a0047ec0 2024-09-13T18:13:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.775+0000 7f97a7fff700 1 --2- 172.21.15.40:0/1849824196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9788002a40 0x7f9788004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.775+0000 7f97a7fff700 1 -- 172.21.15.40:0/1849824196 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9788007f40 con 0x7f9788002a40 2024-09-13T18:13:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.775+0000 7f97a67fc700 1 --2- 172.21.15.40:0/1849824196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9788002a40 0x7f9788004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.775+0000 7f97a67fc700 1 -- 172.21.15.40:0/1849824196 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9788008100 con 0x7f9788002a40 2024-09-13T18:13:32.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.775+0000 7f97a67fc700 1 --2- 172.21.15.40:0/1849824196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9788002a40 0x7f9788004f00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9798007fd0 tx=0x7f979800fb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.775+0000 7f978f7fe700 1 -- 172.21.15.40:0/1849824196 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f979c003bd0 con 0x7f97a0047ec0 2024-09-13T18:13:32.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.775+0000 7f978f7fe700 1 --2- 172.21.15.40:0/1849824196 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9794066730 0x7f9794068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.775+0000 7f97a77fe700 1 --2- 172.21.15.40:0/1849824196 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9794066730 0x7f9794068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.775+0000 7f978f7fe700 1 -- 172.21.15.40:0/1849824196 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f979c085b30 con 0x7f97a0047ec0 2024-09-13T18:13:32.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.775+0000 7f978f7fe700 1 -- 172.21.15.40:0/1849824196 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798005b70 con 0x7f9788002a40 2024-09-13T18:13:32.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.775+0000 7f978f7fe700 1 -- 172.21.15.40:0/1849824196 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9798005cd0 con 0x7f9788002a40 2024-09-13T18:13:32.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.775+0000 7f978f7fe700 1 -- 172.21.15.40:0/1849824196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9788002a40 msgr2=0x7f9788004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.775+0000 7f978f7fe700 1 --2- 172.21.15.40:0/1849824196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9788002a40 0x7f9788004f00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9798007fd0 tx=0x7f979800fb00 comp rx=0 tx=0).stop 2024-09-13T18:13:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.775+0000 7f978f7fe700 1 -- 172.21.15.40:0/1849824196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9788002a40 msgr2=0x7f9788004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:13:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.775+0000 7f978f7fe700 1 --2- 172.21.15.40:0/1849824196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9788002a40 0x7f9788004f00 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9798007fd0 tx=0x7f979800fb00 comp rx=0 tx=0).stop 2024-09-13T18:13:32.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.779+0000 7f97a77fe700 1 --2- 172.21.15.40:0/1849824196 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9794066730 0x7f9794068bf0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f97a8063cf0 tx=0x7f97a8065ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.787+0000 7f97a7fff700 1 --2- 172.21.15.40:0/1849824196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9788006ef0 0x7f9788007360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.787+0000 7f97a7fff700 1 -- 172.21.15.40:0/1849824196 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978800a4a0 con 0x7f9788006ef0 2024-09-13T18:13:32.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.787+0000 7f97a67fc700 1 --2- 172.21.15.40:0/1849824196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9788006ef0 0x7f9788007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.787+0000 7f97a67fc700 1 -- 172.21.15.40:0/1849824196 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f97880078a0 con 0x7f9788006ef0 2024-09-13T18:13:32.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.787+0000 7f97a67fc700 1 --2- 172.21.15.40:0/1849824196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9788006ef0 0x7f9788007360 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f97880080d0 tx=0x7f9798000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.787+0000 7f978f7fe700 1 -- 172.21.15.40:0/1849824196 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9798022070 con 0x7f9788006ef0 2024-09-13T18:13:32.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.787+0000 7f978f7fe700 1 -- 172.21.15.40:0/1849824196 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4949 (secure 0 0 0) 0x7f9798005e40 con 0x7f9788006ef0 2024-09-13T18:13:32.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.787+0000 7f978f7fe700 1 -- 172.21.15.40:0/1849824196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9788006ef0 msgr2=0x7f9788007360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.787+0000 7f978f7fe700 1 --2- 172.21.15.40:0/1849824196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9788006ef0 0x7f9788007360 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f97880080d0 tx=0x7f9798000ed0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.787+0000 7f978f7fe700 1 -- 172.21.15.40:0/1849824196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9788006ef0 msgr2=0x7f9788007360 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:13:32.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.787+0000 7f978f7fe700 1 --2- 172.21.15.40:0/1849824196 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9788006ef0 0x7f9788007360 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f97880080d0 tx=0x7f9798000ed0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.798 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:13:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:13:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:13:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:13:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 186, 2024-09-13T18:13:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:13:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:13:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:13:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:13:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 12, 2024-09-13T18:13:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:13:32.799 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:13:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:13:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:32.800 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:13:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:13:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:32.801 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:32.802 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.802 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:13:32.802 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:13:32.802 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:13:32.802 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:13:32.802 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:13:32.802 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:13:32.802 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:13:32.802 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:13:32.802 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:13:32.802 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:13:32.802 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:13:32.802 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:13:32.802 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:13:32.802 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:13:32.803 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:13:32.803 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:13:32.803 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:13:32.803 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:32.803 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:32.803 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:32.803 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:32.803 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:32.803 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:32.803 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:32.803 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:32.803 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:32.803 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:32.803 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:32.804 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:13:32.804 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.804 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:13:32.804 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.804 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:13:32.804 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:13:32.804 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:32.804 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:32.804 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.804 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:32.804 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:13:32.804 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:32.804 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.804 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.805 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:32.805 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:13:32.805 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:32.805 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.805 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:32.805 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.805 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:13:32.805 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:13:32.805 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:32.805 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:32.805 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:32.805 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.805 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.805 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:13:32.806 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:13:32.806 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:32.806 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:32.806 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.806 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:32.806 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:13:32.806 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:32.806 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.806 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.806 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:32.806 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:13:32.806 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:32.806 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.806 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:32.807 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.807 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:13:32.807 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:13:32.807 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:32.807 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:32.807 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:32.807 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.807 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.807 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:13:32.807 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:13:32.807 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:32.807 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:32.807 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.808 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:32.808 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:13:32.808 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:32.808 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.808 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.808 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:32.808 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:13:32.808 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:32.808 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.808 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:32.808 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.808 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:13:32.808 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:13:32.808 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:32.809 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:32.809 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:32.809 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.809 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:32.809 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.809 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:13:32.809 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:13:32.809 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.809 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:32.809 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:32.809 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:13:32.809 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.810 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:32.810 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:13:32.810 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.810 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:32.810 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:32.810 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:13:32.810 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.810 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:32.810 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:13:32.810 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.810 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:32.810 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:32.810 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:13:32.810 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.811 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:32.811 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:13:32.811 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.811 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:32.811 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:32.811 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:13:32.811 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.811 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:32.811 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:13:32.811 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:32.811 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:32.811 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:32.811 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:13:32.812 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:32.812 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:32.812 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:32.812 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:13:32.812 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:13:32.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.791+0000 7f97a7fff700 1 -- 172.21.15.40:0/1849824196 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9794066730 msgr2=0x7f9794068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.791+0000 7f97a7fff700 1 --2- 172.21.15.40:0/1849824196 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9794066730 0x7f9794068bf0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f97a8063cf0 tx=0x7f97a8065ea0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.791+0000 7f97a7fff700 1 -- 172.21.15.40:0/1849824196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97a0047ec0 msgr2=0x7f97a004b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.791+0000 7f97a7fff700 1 --2- 172.21.15.40:0/1849824196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f97a0047ec0 0x7f97a004b650 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f979c013040 tx=0x7f979c004770 comp rx=0 tx=0).stop 2024-09-13T18:13:32.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.791+0000 7f97a77fe700 1 -- 172.21.15.40:0/1849824196 reap_dead start 2024-09-13T18:13:32.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.791+0000 7f97a77fe700 1 -- 172.21.15.40:0/1849824196 reap_dead start 2024-09-13T18:13:32.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.791+0000 7f97a7fff700 1 -- 172.21.15.40:0/1849824196 shutdown_connections 2024-09-13T18:13:32.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.791+0000 7f97a7fff700 1 -- 172.21.15.40:0/1849824196 >> 172.21.15.40:0/1849824196 conn(0x7f97a000dc40 msgr2=0x7f97a00d76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:32.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.791+0000 7f97a7fff700 1 -- 172.21.15.40:0/1849824196 shutdown_connections 2024-09-13T18:13:32.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.791+0000 7f97a7fff700 1 -- 172.21.15.40:0/1849824196 wait complete. 2024-09-13T18:13:32.814 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:13:32.814 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-13T18:13:32.814 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T18:13:32.814 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-13T18:13:32.814 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-13T18:13:32.815 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T18:13:32.815 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:13:32.865 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T18:13:32.865 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T18:13:32.866 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:13:32.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f18e0a700 1 Processor -- start 2024-09-13T18:13:32.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f18e0a700 1 -- start start 2024-09-13T18:13:32.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f18e0a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f0c0d57d0 0x7f1f0c0df360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f18e0a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f0c007c90 0x7f1f0c046da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f18e0a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0c046910 con 0x7f1f0c0459c0 2024-09-13T18:13:32.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f18e0a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0c0df8a0 con 0x7f1f0c007c90 2024-09-13T18:13:32.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f18e0a700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0c0d5bb0 con 0x7f1f0c0d57d0 2024-09-13T18:13:32.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f137fe700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7f1f0c0d57d0 msgr2=0x7f1f0c0df360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:13:32.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f12ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f0c007c90 0x7f1f0c046da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f137fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f0c0d57d0 0x7f1f0c0df360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:32.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f12ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1f0c007c90 0x7f1f0c046da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47400/0 (socket says 172.21.15.40:47400) 2024-09-13T18:13:32.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f12ffd700 1 -- 172.21.15.40:0/2422009859 learned_addr learned my addr 172.21.15.40:0/2422009859 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:32.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f127fc700 1 -- 172.21.15.40:0/2422009859 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2134004119 0 0) 0x7f1f0c046910 con 0x7f1f0c0459c0 2024-09-13T18:13:32.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f127fc700 1 -- 172.21.15.40:0/2422009859 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f040035d0 con 0x7f1f0c0459c0 2024-09-13T18:13:32.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f12ffd700 1 -- 172.21.15.40:0/2422009859 >> v2:172.21.15.43:3300/0 conn(0x7f1f0c0d57d0 msgr2=0x7f1f0c0df360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:13:32.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f12ffd700 1 --2- 172.21.15.40:0/2422009859 >> v2:172.21.15.43:3300/0 conn(0x7f1f0c0d57d0 0x7f1f0c0df360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f12ffd700 1 -- 172.21.15.40:0/2422009859 >> v1:172.21.15.40:6789/0 conn(0x7f1f0c0459c0 legacy=0x7f1f0c0db3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f12ffd700 1 -- 172.21.15.40:0/2422009859 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c005eb0 con 0x7f1f0c007c90 2024-09-13T18:13:32.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f12ffd700 1 --2- 172.21.15.40:0/2422009859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f0c007c90 0x7f1f0c046da0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009ff0 tx=0x7f1f00009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7e7b703fbe371 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f127fc700 1 -- 172.21.15.40:0/2422009859 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00011830 con 0x7f1f0c007c90 2024-09-13T18:13:32.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f127fc700 1 -- 172.21.15.40:0/2422009859 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f00011990 con 0x7f1f0c007c90 2024-09-13T18:13:32.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.975+0000 7f1f127fc700 1 -- 172.21.15.40:0/2422009859 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00011d20 con 0x7f1f0c007c90 2024-09-13T18:13:32.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f18e0a700 1 -- 172.21.15.40:0/2422009859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f0c007c90 msgr2=0x7f1f0c046da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f18e0a700 1 --2- 172.21.15.40:0/2422009859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f0c007c90 0x7f1f0c046da0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009ff0 tx=0x7f1f00009bf0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f18e0a700 1 -- 172.21.15.40:0/2422009859 shutdown_connections 2024-09-13T18:13:32.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f18e0a700 1 --2- 172.21.15.40:0/2422009859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f0c007c90 0x7f1f0c046da0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f18e0a700 1 --2- 172.21.15.40:0/2422009859 >> v2:172.21.15.43:3300/0 conn(0x7f1f0c0d57d0 0x7f1f0c0df360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f18e0a700 1 -- 172.21.15.40:0/2422009859 >> 172.21.15.40:0/2422009859 conn(0x7f1f0c050590 msgr2=0x7f1f0c0509a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:32.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f18e0a700 1 -- 172.21.15.40:0/2422009859 shutdown_connections 2024-09-13T18:13:32.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f18e0a700 1 -- 172.21.15.40:0/2422009859 wait complete. 2024-09-13T18:13:32.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f18e0a700 1 Processor -- start 2024-09-13T18:13:32.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f18e0a700 1 -- start start 2024-09-13T18:13:32.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f18e0a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f0c007c90 0x7f1f0c0cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f18e0a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f0c0459c0 0x7f1f0c0c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f18e0a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f0c0d57d0 0x7f1f0c0c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f18e0a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c177400 con 0x7f1f0c007c90 2024-09-13T18:13:32.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f18e0a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c1776e0 con 0x7f1f0c0459c0 2024-09-13T18:13:32.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f18e0a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c1779c0 con 0x7f1f0c0d57d0 2024-09-13T18:13:32.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f137fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f0c007c90 0x7f1f0c0cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f12ffd700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f0c0459c0 msgr2=0x7f1f0c0c8e20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:13:32.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f12ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f0c0459c0 0x7f1f0c0c8e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:32.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f137fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f0c007c90 0x7f1f0c0cc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47402/0 (socket says 172.21.15.40:47402) 2024-09-13T18:13:32.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.979+0000 7f1f137fe700 1 -- 172.21.15.40:0/1197401362 learned_addr learned my addr 172.21.15.40:0/1197401362 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:32.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.983+0000 7f1f13fff700 1 --2- 172.21.15.40:0/1197401362 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f0c0d57d0 0x7f1f0c0c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.983+0000 7f1f137fe700 1 -- 172.21.15.40:0/1197401362 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f0c0d57d0 msgr2=0x7f1f0c0c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.983+0000 7f1f137fe700 1 --2- 172.21.15.40:0/1197401362 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f0c0d57d0 0x7f1f0c0c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.983+0000 7f1f137fe700 1 -- 172.21.15.40:0/1197401362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f0c0459c0 msgr2=0x7f1f0c0c8e20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:13:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.983+0000 7f1f137fe700 1 --2- 172.21.15.40:0/1197401362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f0c0459c0 0x7f1f0c0c8e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.983+0000 7f1f137fe700 1 -- 172.21.15.40:0/1197401362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f00009570 con 0x7f1f0c007c90 2024-09-13T18:13:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.983+0000 7f1f137fe700 1 --2- 172.21.15.40:0/1197401362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f0c007c90 0x7f1f0c0cc420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800b780 tx=0x7f1f0800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:32.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.983+0000 7f1efffff700 1 -- 172.21.15.40:0/1197401362 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08003e70 con 0x7f1f0c007c90 2024-09-13T18:13:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.983+0000 7f1efffff700 1 -- 172.21.15.40:0/1197401362 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f08005070 con 0x7f1f0c007c90 2024-09-13T18:13:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.983+0000 7f1f18e0a700 1 -- 172.21.15.40:0/1197401362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0c177ca0 con 0x7f1f0c007c90 2024-09-13T18:13:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.983+0000 7f1efffff700 1 -- 172.21.15.40:0/1197401362 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08005440 con 0x7f1f0c007c90 2024-09-13T18:13:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.983+0000 7f1f18e0a700 1 -- 172.21.15.40:0/1197401362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0c178260 con 0x7f1f0c007c90 2024-09-13T18:13:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.983+0000 7f1f18e0a700 1 -- 172.21.15.40:0/1197401362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0c0ca0b0 con 0x7f1f0c007c90 2024-09-13T18:13:32.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.983+0000 7f1efffff700 1 -- 172.21.15.40:0/1197401362 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1f080055a0 con 0x7f1f0c007c90 2024-09-13T18:13:32.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.983+0000 7f1efffff700 1 --2- 172.21.15.40:0/1197401362 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1ef80666e0 0x7f1ef8068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:32.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.987+0000 7f1f12ffd700 1 --2- 172.21.15.40:0/1197401362 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1ef80666e0 0x7f1ef8068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:32.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.987+0000 7f1efffff700 1 -- 172.21.15.40:0/1197401362 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1f08083f70 con 0x7f1f0c007c90 2024-09-13T18:13:32.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.987+0000 7f1f12ffd700 1 --2- 172.21.15.40:0/1197401362 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1ef80666e0 0x7f1ef8068ba0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0d5100 tx=0x7f1f00008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:33.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:32.991+0000 7f1efffff700 1 -- 172.21.15.40:0/1197401362 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1f080b5db0 con 0x7f1f0c007c90 2024-09-13T18:13:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:33.287+0000 7f1f18e0a700 1 -- 172.21.15.40:0/1197401362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1f0c0455c0 con 0x7f1f0c007c90 2024-09-13T18:13:33.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:33.287+0000 7f1efffff700 1 -- 172.21.15.40:0/1197401362 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f1f08050e90 con 0x7f1f0c007c90 2024-09-13T18:13:33.297 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":186,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:13:33.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:33.291+0000 7f1f18e0a700 1 -- 172.21.15.40:0/1197401362 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1ef80666e0 msgr2=0x7f1ef8068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:33.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:33.291+0000 7f1f18e0a700 1 --2- 172.21.15.40:0/1197401362 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1ef80666e0 0x7f1ef8068ba0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0d5100 tx=0x7f1f00008030 comp rx=0 tx=0).stop 2024-09-13T18:13:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:33.291+0000 7f1f18e0a700 1 -- 172.21.15.40:0/1197401362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f0c007c90 msgr2=0x7f1f0c0cc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:33.291+0000 7f1f18e0a700 1 --2- 172.21.15.40:0/1197401362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f0c007c90 0x7f1f0c0cc420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800b780 tx=0x7f1f0800ba90 comp rx=0 tx=0).stop 2024-09-13T18:13:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:33.291+0000 7f1f18e0a700 1 -- 172.21.15.40:0/1197401362 shutdown_connections 2024-09-13T18:13:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:33.291+0000 7f1f18e0a700 1 --2- 172.21.15.40:0/1197401362 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1ef80666e0 0x7f1ef8068ba0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:33.291+0000 7f1f18e0a700 1 --2- 172.21.15.40:0/1197401362 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f0c0d57d0 0x7f1f0c0c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:33.291+0000 7f1f18e0a700 1 --2- 172.21.15.40:0/1197401362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f0c0459c0 0x7f1f0c0c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:33.291+0000 7f1f18e0a700 1 --2- 172.21.15.40:0/1197401362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f0c007c90 0x7f1f0c0cc420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:33.291+0000 7f1f18e0a700 1 -- 172.21.15.40:0/1197401362 >> 172.21.15.40:0/1197401362 conn(0x7f1f0c050590 msgr2=0x7f1f0c045df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:33.291+0000 7f1f18e0a700 1 -- 172.21.15.40:0/1197401362 shutdown_connections 2024-09-13T18:13:33.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:33.291+0000 7f1f18e0a700 1 -- 172.21.15.40:0/1197401362 wait complete. 2024-09-13T18:13:36.313 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:13:36.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39ebfff700 1 Processor -- start 2024-09-13T18:13:36.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39ebfff700 1 -- start start 2024-09-13T18:13:36.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39ebfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f39ec0c6880 0x7f39ec0c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:36.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39ebfff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39ec159490 con 0x7f39ec1534a0 2024-09-13T18:13:36.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39ebfff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39ec0c7230 con 0x7f39ec0c7f70 2024-09-13T18:13:36.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39ebfff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39ec06c670 con 0x7f39ec0c6880 2024-09-13T18:13:36.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39eb7fe700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7f39ec0c6880 msgr2=0x7f39ec0c6cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:13:36.452 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39ea7fc700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7f39ec0c7f70 legacy=0x7f39ec06e3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:13:36.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39eaffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f39ec1534a0 0x7f39ec070ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39632/0 (socket says 172.21.15.40:39632) 2024-09-13T18:13:36.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39eaffd700 1 -- 172.21.15.40:0/1808637581 learned_addr learned my addr 172.21.15.40:0/1808637581 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:13:36.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39eb7fe700 1 --2- 172.21.15.40:0/1808637581 >> v2:172.21.15.43:3300/0 conn(0x7f39ec0c6880 0x7f39ec0c6cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:36.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39e9ffb700 1 -- 172.21.15.40:0/1808637581 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176811760 0 0) 0x7f39ec159490 con 0x7f39ec1534a0 2024-09-13T18:13:36.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39e9ffb700 1 -- 172.21.15.40:0/1808637581 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39d40035d0 con 0x7f39ec1534a0 2024-09-13T18:13:36.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39e9ffb700 1 -- 172.21.15.40:0/1808637581 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2847995922 0 0) 0x7f39d40035d0 con 0x7f39ec1534a0 2024-09-13T18:13:36.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39e9ffb700 1 -- 172.21.15.40:0/1808637581 >> v2:172.21.15.43:3300/0 conn(0x7f39ec0c6880 msgr2=0x7f39ec0c6cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:13:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39e9ffb700 1 --2- 172.21.15.40:0/1808637581 >> v2:172.21.15.43:3300/0 conn(0x7f39ec0c6880 0x7f39ec0c6cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39e9ffb700 1 -- 172.21.15.40:0/1808637581 >> v1:172.21.15.43:6789/0 conn(0x7f39ec0c7f70 legacy=0x7f39ec06e3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:13:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39e9ffb700 1 -- 172.21.15.40:0/1808637581 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ec159710 con 0x7f39ec1534a0 2024-09-13T18:13:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39e9ffb700 1 -- 172.21.15.40:0/1808637581 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f39dc001670 con 0x7f39ec1534a0 2024-09-13T18:13:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39e9ffb700 1 -- 172.21.15.40:0/1808637581 >> v1:172.21.15.40:6789/0 conn(0x7f39ec1534a0 legacy=0x7f39ec070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39e9ffb700 1 --2- 172.21.15.40:0/1808637581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d4003a20 0x7f39d4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39e9ffb700 1 --2- 172.21.15.40:0/1808637581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d4004870 0x7f39d4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39e9ffb700 1 --2- 172.21.15.40:0/1808637581 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39d40052a0 0x7f39d400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39e9ffb700 1 -- 172.21.15.40:0/1808637581 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec06c670 con 0x7f39d4004870 2024-09-13T18:13:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39e9ffb700 1 -- 172.21.15.40:0/1808637581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d400c200 con 0x7f39d4003a20 2024-09-13T18:13:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39e9ffb700 1 -- 172.21.15.40:0/1808637581 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d400c3c0 con 0x7f39d40052a0 2024-09-13T18:13:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39eaffd700 1 -- 172.21.15.40:0/1808637581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d4003a20 msgr2=0x7f39d4003e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:13:36.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39eaffd700 1 --2- 172.21.15.40:0/1808637581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d4003a20 0x7f39d4003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:36.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39ea7fc700 1 --2- 172.21.15.40:0/1808637581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d4004870 0x7f39d4004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:36.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39eb7fe700 1 --2- 172.21.15.40:0/1808637581 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39d40052a0 0x7f39d400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:36.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39ea7fc700 1 -- 172.21.15.40:0/1808637581 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39d40052a0 msgr2=0x7f39d400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:13:36.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39ea7fc700 1 --2- 172.21.15.40:0/1808637581 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39d40052a0 0x7f39d400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:36.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39ea7fc700 1 -- 172.21.15.40:0/1808637581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d4003a20 msgr2=0x7f39d4003e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:13:36.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39ea7fc700 1 --2- 172.21.15.40:0/1808637581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39d4003a20 0x7f39d4003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:36.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39ea7fc700 1 -- 172.21.15.40:0/1808637581 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d40035d0 con 0x7f39d4004870 2024-09-13T18:13:36.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39eb7fe700 1 --2- 172.21.15.40:0/1808637581 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39d40052a0 0x7f39d400ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:13:36.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39eb7fe700 1 -- 172.21.15.40:0/1808637581 reap_dead start 2024-09-13T18:13:36.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.442+0000 7f39ea7fc700 1 --2- 172.21.15.40:0/1808637581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d4004870 0x7f39d4004cf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f39e00098e0 tx=0x7f39d40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae4b2b3ed91871c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:36.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.446+0000 7f39e9ffb700 1 -- 172.21.15.40:0/1808637581 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0009380 con 0x7f39d4004870 2024-09-13T18:13:36.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.446+0000 7f39e9ffb700 1 -- 172.21.15.40:0/1808637581 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e0003b90 con 0x7f39d4004870 2024-09-13T18:13:36.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.446+0000 7f39e9ffb700 1 -- 172.21.15.40:0/1808637581 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0005070 con 0x7f39d4004870 2024-09-13T18:13:36.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.446+0000 7f39ebfff700 1 -- 172.21.15.40:0/1808637581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d4004870 msgr2=0x7f39d4004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:36.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.446+0000 7f39ebfff700 1 --2- 172.21.15.40:0/1808637581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d4004870 0x7f39d4004cf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f39e00098e0 tx=0x7f39d40058a0 comp rx=0 tx=0).stop 2024-09-13T18:13:36.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.446+0000 7f39ebfff700 1 -- 172.21.15.40:0/1808637581 shutdown_connections 2024-09-13T18:13:36.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.446+0000 7f39ebfff700 1 --2- 172.21.15.40:0/1808637581 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d4004870 0x7f39d4004cf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:36.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.446+0000 7f39ebfff700 1 -- 172.21.15.40:0/1808637581 >> 172.21.15.40:0/1808637581 conn(0x7f39ec0cbd80 msgr2=0x7f39ec0cc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:36.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.446+0000 7f39ebfff700 1 -- 172.21.15.40:0/1808637581 shutdown_connections 2024-09-13T18:13:36.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.446+0000 7f39ebfff700 1 -- 172.21.15.40:0/1808637581 wait complete. 2024-09-13T18:13:36.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.446+0000 7f39ebfff700 1 Processor -- start 2024-09-13T18:13:36.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.446+0000 7f39ebfff700 1 -- start start 2024-09-13T18:13:36.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39ebfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d4004870 0x7f39ec14d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:36.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39ebfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ec157260 0x7f39ec0c4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:36.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39ebfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39ec150490 0x7f39ec0c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:36.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39ebfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec0c4b60 con 0x7f39d4004870 2024-09-13T18:13:36.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39ebfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec1ebc30 con 0x7f39ec157260 2024-09-13T18:13:36.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39ebfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec1ebf10 con 0x7f39ec150490 2024-09-13T18:13:36.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39ea7fc700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ec157260 msgr2=0x7f39ec0c4f80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:13:36.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39eaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d4004870 0x7f39ec14d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:36.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39ea7fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ec157260 0x7f39ec0c4f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:36.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39eaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d4004870 0x7f39ec14d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47416/0 (socket says 172.21.15.40:47416) 2024-09-13T18:13:36.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39eaffd700 1 -- 172.21.15.40:0/1271135470 learned_addr learned my addr 172.21.15.40:0/1271135470 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:36.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39eb7fe700 1 --2- 172.21.15.40:0/1271135470 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39ec150490 0x7f39ec0c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:36.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39eaffd700 1 -- 172.21.15.40:0/1271135470 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39ec150490 msgr2=0x7f39ec0c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:36.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39eaffd700 1 --2- 172.21.15.40:0/1271135470 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39ec150490 0x7f39ec0c54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:36.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39eaffd700 1 -- 172.21.15.40:0/1271135470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ec157260 msgr2=0x7f39ec0c4f80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:13:36.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39eaffd700 1 --2- 172.21.15.40:0/1271135470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ec157260 0x7f39ec0c4f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:36.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39eaffd700 1 -- 172.21.15.40:0/1271135470 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ec0c4780 con 0x7f39d4004870 2024-09-13T18:13:36.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39eb7fe700 1 --2- 172.21.15.40:0/1271135470 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39ec150490 0x7f39ec0c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:13:36.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39eaffd700 1 --2- 172.21.15.40:0/1271135470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d4004870 0x7f39ec14d560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00caf0 tx=0x7f39dc00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:36.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39d37fe700 1 -- 172.21.15.40:0/1271135470 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc004a90 con 0x7f39d4004870 2024-09-13T18:13:36.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39d37fe700 1 -- 172.21.15.40:0/1271135470 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39dc004bf0 con 0x7f39d4004870 2024-09-13T18:13:36.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39ebfff700 1 -- 172.21.15.40:0/1271135470 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ec14dfd0 con 0x7f39d4004870 2024-09-13T18:13:36.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39d37fe700 1 -- 172.21.15.40:0/1271135470 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc0070d0 con 0x7f39d4004870 2024-09-13T18:13:36.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39ebfff700 1 -- 172.21.15.40:0/1271135470 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ec14e400 con 0x7f39d4004870 2024-09-13T18:13:36.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39d37fe700 1 -- 172.21.15.40:0/1271135470 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f39dc007230 con 0x7f39d4004870 2024-09-13T18:13:36.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.450+0000 7f39ebfff700 1 -- 172.21.15.40:0/1271135470 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39ec0c45c0 con 0x7f39d4004870 2024-09-13T18:13:36.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.454+0000 7f39d37fe700 1 --2- 172.21.15.40:0/1271135470 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f39d80666e0 0x7f39d8068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:36.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.454+0000 7f39d37fe700 1 -- 172.21.15.40:0/1271135470 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f39dc083870 con 0x7f39d4004870 2024-09-13T18:13:36.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.454+0000 7f39ea7fc700 1 --2- 172.21.15.40:0/1271135470 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f39d80666e0 0x7f39d8068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:36.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.454+0000 7f39ea7fc700 1 --2- 172.21.15.40:0/1271135470 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f39d80666e0 0x7f39d8068ba0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f39ec158730 tx=0x7f39e0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:36.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.458+0000 7f39d37fe700 1 -- 172.21.15.40:0/1271135470 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f39dc0508f0 con 0x7f39d4004870 2024-09-13T18:13:36.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.726+0000 7f39ebfff700 1 -- 172.21.15.40:0/1271135470 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f39ec0c4ca0 con 0x7f39d4004870 2024-09-13T18:13:36.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.726+0000 7f39d37fe700 1 -- 172.21.15.40:0/1271135470 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f39dc018080 con 0x7f39d4004870 2024-09-13T18:13:36.737 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":186,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:13:36.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.730+0000 7f39ebfff700 1 -- 172.21.15.40:0/1271135470 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f39d80666e0 msgr2=0x7f39d8068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:36.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.730+0000 7f39ebfff700 1 --2- 172.21.15.40:0/1271135470 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f39d80666e0 0x7f39d8068ba0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f39ec158730 tx=0x7f39e0008030 comp rx=0 tx=0).stop 2024-09-13T18:13:36.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.730+0000 7f39ebfff700 1 -- 172.21.15.40:0/1271135470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d4004870 msgr2=0x7f39ec14d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:36.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.730+0000 7f39ebfff700 1 --2- 172.21.15.40:0/1271135470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d4004870 0x7f39ec14d560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00caf0 tx=0x7f39dc00ceb0 comp rx=0 tx=0).stop 2024-09-13T18:13:36.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.730+0000 7f39ebfff700 1 -- 172.21.15.40:0/1271135470 shutdown_connections 2024-09-13T18:13:36.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.730+0000 7f39ebfff700 1 --2- 172.21.15.40:0/1271135470 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f39d80666e0 0x7f39d8068ba0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:36.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.730+0000 7f39ebfff700 1 --2- 172.21.15.40:0/1271135470 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39ec150490 0x7f39ec0c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:36.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.730+0000 7f39ebfff700 1 --2- 172.21.15.40:0/1271135470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39ec157260 0x7f39ec0c4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:36.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.730+0000 7f39ebfff700 1 --2- 172.21.15.40:0/1271135470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f39d4004870 0x7f39ec14d560 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:36.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.730+0000 7f39ebfff700 1 -- 172.21.15.40:0/1271135470 >> 172.21.15.40:0/1271135470 conn(0x7f39ec0cbd80 msgr2=0x7f39ec06e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:36.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.730+0000 7f39ebfff700 1 -- 172.21.15.40:0/1271135470 shutdown_connections 2024-09-13T18:13:36.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:36.730+0000 7f39ebfff700 1 -- 172.21.15.40:0/1271135470 wait complete. 2024-09-13T18:13:37.810 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:13:37.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:13:39.753 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:13:39.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc75ab6d700 1 Processor -- start 2024-09-13T18:13:39.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc75ab6d700 1 -- start start 2024-09-13T18:13:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc75ab6d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc74c0d87f0 0x7fc74c04c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc75ab6d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc74c0d8bd0 0x7fc74c008860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc75ab6d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc74c04c9a0 0x7fc74c008da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc75ab6d700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc74c04cef0 con 0x7fc74c0d8bd0 2024-09-13T18:13:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc75ab6d700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc74c00b310 con 0x7fc74c04c9a0 2024-09-13T18:13:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc75ab6d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c04c790 con 0x7fc74c0d87f0 2024-09-13T18:13:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc75a36c700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7fc74c04c9a0 msgr2=0x7fc74c008da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:13:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc75a36c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc74c04c9a0 0x7fc74c008da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc75936a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc74c0d8bd0 0x7fc74c008860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc75936a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc74c0d8bd0 0x7fc74c008860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47432/0 (socket says 172.21.15.40:47432) 2024-09-13T18:13:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc75936a700 1 -- 172.21.15.40:0/1727195545 learned_addr learned my addr 172.21.15.40:0/1727195545 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:39.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc759b6b700 1 --2- 172.21.15.40:0/1727195545 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc74c0d87f0 0x7fc74c04c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:39.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc75936a700 1 -- 172.21.15.40:0/1727195545 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc74c0d87f0 msgr2=0x7fc74c04c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:13:39.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc75936a700 1 --2- 172.21.15.40:0/1727195545 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc74c0d87f0 0x7fc74c04c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:39.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc75936a700 1 -- 172.21.15.40:0/1727195545 >> v2:172.21.15.43:3300/0 conn(0x7fc74c04c9a0 msgr2=0x7fc74c008da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:13:39.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc75936a700 1 --2- 172.21.15.40:0/1727195545 >> v2:172.21.15.43:3300/0 conn(0x7fc74c04c9a0 0x7fc74c008da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:39.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc75936a700 1 -- 172.21.15.40:0/1727195545 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74c00b580 con 0x7fc74c0d8bd0 2024-09-13T18:13:39.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc75936a700 1 --2- 172.21.15.40:0/1727195545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c0d8bd0 0x7fc74c008860 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00a1d0 tx=0x7fc744009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29f06890252c303 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:39.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc758b69700 1 -- 172.21.15.40:0/1727195545 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744011830 con 0x7fc74c0d8bd0 2024-09-13T18:13:39.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc758b69700 1 -- 172.21.15.40:0/1727195545 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc744011990 con 0x7fc74c0d8bd0 2024-09-13T18:13:39.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.870+0000 7fc758b69700 1 -- 172.21.15.40:0/1727195545 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744011d20 con 0x7fc74c0d8bd0 2024-09-13T18:13:39.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.874+0000 7fc75ab6d700 1 -- 172.21.15.40:0/1727195545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c0d8bd0 msgr2=0x7fc74c008860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:39.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.874+0000 7fc75ab6d700 1 --2- 172.21.15.40:0/1727195545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c0d8bd0 0x7fc74c008860 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc74c00a1d0 tx=0x7fc744009bf0 comp rx=0 tx=0).stop 2024-09-13T18:13:39.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.874+0000 7fc75ab6d700 1 -- 172.21.15.40:0/1727195545 shutdown_connections 2024-09-13T18:13:39.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.874+0000 7fc75ab6d700 1 --2- 172.21.15.40:0/1727195545 >> v2:172.21.15.43:3300/0 conn(0x7fc74c04c9a0 0x7fc74c008da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:39.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.874+0000 7fc75ab6d700 1 --2- 172.21.15.40:0/1727195545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c0d8bd0 0x7fc74c008860 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:39.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.874+0000 7fc75ab6d700 1 --2- 172.21.15.40:0/1727195545 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc74c0d87f0 0x7fc74c04c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:39.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.874+0000 7fc75ab6d700 1 -- 172.21.15.40:0/1727195545 >> 172.21.15.40:0/1727195545 conn(0x7fc74c00a780 msgr2=0x7fc74c00ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:39.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.874+0000 7fc75ab6d700 1 -- 172.21.15.40:0/1727195545 shutdown_connections 2024-09-13T18:13:39.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.874+0000 7fc75ab6d700 1 -- 172.21.15.40:0/1727195545 wait complete. 2024-09-13T18:13:39.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.874+0000 7fc75ab6d700 1 Processor -- start 2024-09-13T18:13:39.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.874+0000 7fc75ab6d700 1 -- start start 2024-09-13T18:13:39.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc75ab6d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c04c9a0 0x7fc74c04a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:39.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc75ab6d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc74c0d87f0 0x7fc74c175ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:39.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc75ab6d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc74c0d8bd0 0x7fc74c1761e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc75ab6d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c0dbc50 con 0x7fc74c04c9a0 2024-09-13T18:13:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc75ab6d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c16a460 con 0x7fc74c0d87f0 2024-09-13T18:13:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc75ab6d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c16a6e0 con 0x7fc74c0d8bd0 2024-09-13T18:13:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc759b6b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c04c9a0 0x7fc74c04a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc75936a700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc74c0d87f0 msgr2=0x7fc74c175ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:13:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc75936a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc74c0d87f0 0x7fc74c175ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc759b6b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c04c9a0 0x7fc74c04a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47434/0 (socket says 172.21.15.40:47434) 2024-09-13T18:13:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc759b6b700 1 -- 172.21.15.40:0/3784597125 learned_addr learned my addr 172.21.15.40:0/3784597125 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc75a36c700 1 --2- 172.21.15.40:0/3784597125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc74c0d8bd0 0x7fc74c1761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc759b6b700 1 -- 172.21.15.40:0/3784597125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc74c0d8bd0 msgr2=0x7fc74c1761e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc759b6b700 1 --2- 172.21.15.40:0/3784597125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc74c0d8bd0 0x7fc74c1761e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc759b6b700 1 -- 172.21.15.40:0/3784597125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc74c0d87f0 msgr2=0x7fc74c175ca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:13:39.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc759b6b700 1 --2- 172.21.15.40:0/3784597125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc74c0d87f0 0x7fc74c175ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc759b6b700 1 -- 172.21.15.40:0/3784597125 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc744009570 con 0x7fc74c04c9a0 2024-09-13T18:13:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc75a36c700 1 --2- 172.21.15.40:0/3784597125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc74c0d8bd0 0x7fc74c1761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:13:39.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc759b6b700 1 --2- 172.21.15.40:0/3784597125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c04c9a0 0x7fc74c04a560 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc75000e840 tx=0x7fc75000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc74a7fc700 1 -- 172.21.15.40:0/3784597125 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7500095a0 con 0x7fc74c04c9a0 2024-09-13T18:13:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc74a7fc700 1 -- 172.21.15.40:0/3784597125 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75000c690 con 0x7fc74c04c9a0 2024-09-13T18:13:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc75ab6d700 1 -- 172.21.15.40:0/3784597125 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc74c0d9fe0 con 0x7fc74c04c9a0 2024-09-13T18:13:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc74a7fc700 1 -- 172.21.15.40:0/3784597125 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75000ca00 con 0x7fc74c04c9a0 2024-09-13T18:13:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc75ab6d700 1 -- 172.21.15.40:0/3784597125 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc74c0da5a0 con 0x7fc74c04c9a0 2024-09-13T18:13:39.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.878+0000 7fc75ab6d700 1 -- 172.21.15.40:0/3784597125 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc74c04c250 con 0x7fc74c04c9a0 2024-09-13T18:13:39.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.882+0000 7fc74a7fc700 1 -- 172.21.15.40:0/3784597125 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc75000cb60 con 0x7fc74c04c9a0 2024-09-13T18:13:39.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.882+0000 7fc74a7fc700 1 --2- 172.21.15.40:0/3784597125 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc7380666e0 0x7fc738068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:39.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.882+0000 7fc75936a700 1 --2- 172.21.15.40:0/3784597125 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc7380666e0 0x7fc738068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:39.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.882+0000 7fc74a7fc700 1 -- 172.21.15.40:0/3784597125 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc7500844a0 con 0x7fc74c04c9a0 2024-09-13T18:13:39.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.882+0000 7fc75936a700 1 --2- 172.21.15.40:0/3784597125 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc7380666e0 0x7fc738068ba0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0d8fd0 tx=0x7fc744008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:39.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:39.886+0000 7fc74a7fc700 1 -- 172.21.15.40:0/3784597125 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc750051450 con 0x7fc74c04c9a0 2024-09-13T18:13:40.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:40.170+0000 7fc75ab6d700 1 -- 172.21.15.40:0/3784597125 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc74c0d59a0 con 0x7fc74c04c9a0 2024-09-13T18:13:40.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:40.170+0000 7fc74a7fc700 1 -- 172.21.15.40:0/3784597125 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7fc750059e60 con 0x7fc74c04c9a0 2024-09-13T18:13:40.180 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":186,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:13:40.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:40.170+0000 7fc75ab6d700 1 -- 172.21.15.40:0/3784597125 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc7380666e0 msgr2=0x7fc738068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:40.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:40.170+0000 7fc75ab6d700 1 --2- 172.21.15.40:0/3784597125 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc7380666e0 0x7fc738068ba0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0d8fd0 tx=0x7fc744008030 comp rx=0 tx=0).stop 2024-09-13T18:13:40.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:40.170+0000 7fc75ab6d700 1 -- 172.21.15.40:0/3784597125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c04c9a0 msgr2=0x7fc74c04a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:40.170+0000 7fc75ab6d700 1 --2- 172.21.15.40:0/3784597125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c04c9a0 0x7fc74c04a560 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc75000e840 tx=0x7fc75000ec00 comp rx=0 tx=0).stop 2024-09-13T18:13:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:40.170+0000 7fc75ab6d700 1 -- 172.21.15.40:0/3784597125 shutdown_connections 2024-09-13T18:13:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:40.170+0000 7fc75ab6d700 1 --2- 172.21.15.40:0/3784597125 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc7380666e0 0x7fc738068ba0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:40.170+0000 7fc75ab6d700 1 --2- 172.21.15.40:0/3784597125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc74c0d8bd0 0x7fc74c1761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:40.170+0000 7fc75ab6d700 1 --2- 172.21.15.40:0/3784597125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc74c0d87f0 0x7fc74c175ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:40.170+0000 7fc75ab6d700 1 --2- 172.21.15.40:0/3784597125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc74c04c9a0 0x7fc74c04a560 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:40.174+0000 7fc75ab6d700 1 -- 172.21.15.40:0/3784597125 >> 172.21.15.40:0/3784597125 conn(0x7fc74c00a780 msgr2=0x7fc74c0d2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:40.174+0000 7fc75ab6d700 1 -- 172.21.15.40:0/3784597125 shutdown_connections 2024-09-13T18:13:40.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:40.174+0000 7fc75ab6d700 1 -- 172.21.15.40:0/3784597125 wait complete. 2024-09-13T18:13:43.196 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:13:43.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:43.318+0000 7fecc32a0700 1 Processor -- start 2024-09-13T18:13:43.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:43.318+0000 7fecc32a0700 1 -- start start 2024-09-13T18:13:43.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:43.322+0000 7fecc32a0700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fecbc0c7f20 0x7fecbc0c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:43.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:43.322+0000 7fecc32a0700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecbc0c8890 con 0x7fecbc153900 2024-09-13T18:13:43.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:43.322+0000 7fecc32a0700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecbc153ce0 con 0x7fecbc159ba0 2024-09-13T18:13:43.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:43.322+0000 7fecc32a0700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecbc1596c0 con 0x7fecbc0c7f20 2024-09-13T18:13:43.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:43.322+0000 7fecc083b700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7fecbc0c7f20 msgr2=0x7fecbc0c8350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:13:43.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:43.322+0000 7fecc183d700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7fecbc159ba0 legacy=0x7fecbc159fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:13:43.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:43.322+0000 7fecc083b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fecbc0c7f20 0x7fecbc0c8350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:43.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:43.322+0000 7fecc103c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fecbc153900 0x7fecbc0c5f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59024/0 (socket says 172.21.15.40:59024) 2024-09-13T18:13:43.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:43.322+0000 7fecc103c700 1 -- 172.21.15.40:0/1609381642 learned_addr learned my addr 172.21.15.40:0/1609381642 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:13:43.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:43.526+0000 7fecc183d700 1 -- 172.21.15.40:0/1609381642 >> v1:172.21.15.43:6789/0 conn(0x7fecbc159ba0 legacy=0x7fecbc159fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:13:43.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:43.526+0000 7fecc083b700 1 -- 172.21.15.40:0/1609381642 >> v2:172.21.15.43:3300/0 conn(0x7fecbc0c7f20 msgr2=0x7fecbc0c8350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:13:43.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:43.526+0000 7fecc083b700 1 --2- 172.21.15.40:0/1609381642 >> v2:172.21.15.43:3300/0 conn(0x7fecbc0c7f20 0x7fecbc0c8350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:13:43.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:43.930+0000 7fecc083b700 1 -- 172.21.15.40:0/1609381642 >> v2:172.21.15.43:3300/0 conn(0x7fecbc0c7f20 msgr2=0x7fecbc0c8350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:13:43.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:43.930+0000 7fecc183d700 1 -- 172.21.15.40:0/1609381642 >> v1:172.21.15.43:6789/0 conn(0x7fecbc159ba0 legacy=0x7fecbc159fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:13:43.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:43.930+0000 7fecc083b700 1 --2- 172.21.15.40:0/1609381642 >> v2:172.21.15.43:3300/0 conn(0x7fecbc0c7f20 0x7fecbc0c8350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:13:44.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:44.734+0000 7fecc083b700 1 -- 172.21.15.40:0/1609381642 >> v2:172.21.15.43:3300/0 conn(0x7fecbc0c7f20 msgr2=0x7fecbc0c8350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:13:44.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:44.734+0000 7fecc183d700 1 -- 172.21.15.40:0/1609381642 >> v1:172.21.15.43:6789/0 conn(0x7fecbc159ba0 legacy=0x7fecbc159fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:13:44.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:44.734+0000 7fecc083b700 1 --2- 172.21.15.40:0/1609381642 >> v2:172.21.15.43:3300/0 conn(0x7fecbc0c7f20 0x7fecbc0c8350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:13:46.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.322+0000 7fecb2ffd700 1 -- 172.21.15.40:0/1609381642 >> v2:172.21.15.43:3300/0 conn(0x7fecbc0c7f20 msgr2=0x7fecbc0c8350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:13:46.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.322+0000 7fecb2ffd700 1 --2- 172.21.15.40:0/1609381642 >> v2:172.21.15.43:3300/0 conn(0x7fecbc0c7f20 0x7fecbc0c8350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:46.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.322+0000 7fecb2ffd700 1 -- 172.21.15.40:0/1609381642 >> v1:172.21.15.43:6789/0 conn(0x7fecbc159ba0 legacy=0x7fecbc159fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:13:46.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.322+0000 7fecb2ffd700 1 -- 172.21.15.40:0/1609381642 >> v1:172.21.15.40:6789/0 conn(0x7fecbc153900 legacy=0x7fecbc0c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:46.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.322+0000 7fecb2ffd700 1 --2- 172.21.15.40:0/1609381642 >> v2:172.21.15.40:3300/0 conn(0x7feca0003730 0x7feca0005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:46.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.322+0000 7fecb2ffd700 1 --2- 172.21.15.40:0/1609381642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca00061a0 0x7feca0008660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:46.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.322+0000 7fecb2ffd700 1 -- 172.21.15.40:0/1609381642 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecbc1596c0 con 0x7feca0003730 2024-09-13T18:13:46.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.322+0000 7fecb2ffd700 1 -- 172.21.15.40:0/1609381642 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca0008d80 con 0x7feca00061a0 2024-09-13T18:13:46.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.322+0000 7fecc103c700 1 --2- 172.21.15.40:0/1609381642 >> v2:172.21.15.40:3300/0 conn(0x7feca0003730 0x7feca0005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:46.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.322+0000 7fecc083b700 1 --2- 172.21.15.40:0/1609381642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca00061a0 0x7feca0008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:46.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.322+0000 7fecc103c700 1 -- 172.21.15.40:0/1609381642 >> v2:172.21.15.40:3300/0 conn(0x7feca0003730 msgr2=0x7feca0005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:13:46.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.322+0000 7fecc103c700 1 -- 172.21.15.40:0/1609381642 >> v2:172.21.15.40:3300/0 conn(0x7feca0003730 msgr2=0x7feca0005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:13:46.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.322+0000 7fecc103c700 1 --2- 172.21.15.40:0/1609381642 >> v2:172.21.15.40:3300/0 conn(0x7feca0003730 0x7feca0005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:13:46.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.322+0000 7fecc103c700 1 --2- 172.21.15.40:0/1609381642 >> v2:172.21.15.40:3300/0 conn(0x7feca0003730 0x7feca0005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:46.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.322+0000 7fecc083b700 1 -- 172.21.15.40:0/1609381642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca00061a0 msgr2=0x7feca0008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:13:46.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.322+0000 7fecc083b700 1 -- 172.21.15.40:0/1609381642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca00061a0 msgr2=0x7feca0008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:13:46.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.322+0000 7fecc083b700 1 --2- 172.21.15.40:0/1609381642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca00061a0 0x7feca0008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:13:46.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.322+0000 7fecc083b700 1 --2- 172.21.15.40:0/1609381642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca00061a0 0x7feca0008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:46.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.526+0000 7fecc103c700 1 --2- 172.21.15.40:0/1609381642 >> v2:172.21.15.40:3300/0 conn(0x7feca0003730 0x7feca0005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:46.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.526+0000 7fecc083b700 1 --2- 172.21.15.40:0/1609381642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca00061a0 0x7feca0008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:46.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.526+0000 7fecc103c700 1 -- 172.21.15.40:0/1609381642 >> v2:172.21.15.40:3300/0 conn(0x7feca0003730 msgr2=0x7feca0005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:13:46.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.526+0000 7fecc103c700 1 -- 172.21.15.40:0/1609381642 >> v2:172.21.15.40:3300/0 conn(0x7feca0003730 msgr2=0x7feca0005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:13:46.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.526+0000 7fecc103c700 1 --2- 172.21.15.40:0/1609381642 >> v2:172.21.15.40:3300/0 conn(0x7feca0003730 0x7feca0005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:13:46.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.526+0000 7fecc103c700 1 --2- 172.21.15.40:0/1609381642 >> v2:172.21.15.40:3300/0 conn(0x7feca0003730 0x7feca0005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:13:46.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.526+0000 7fecc083b700 1 -- 172.21.15.40:0/1609381642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca00061a0 msgr2=0x7feca0008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:13:46.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.526+0000 7fecc083b700 1 -- 172.21.15.40:0/1609381642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca00061a0 msgr2=0x7feca0008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:13:46.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.526+0000 7fecc083b700 1 --2- 172.21.15.40:0/1609381642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca00061a0 0x7feca0008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:13:46.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.526+0000 7fecc083b700 1 --2- 172.21.15.40:0/1609381642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca00061a0 0x7feca0008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:13:46.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.930+0000 7fecc103c700 1 --2- 172.21.15.40:0/1609381642 >> v2:172.21.15.40:3300/0 conn(0x7feca0003730 0x7feca0005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:46.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.930+0000 7fecc083b700 1 --2- 172.21.15.40:0/1609381642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca00061a0 0x7feca0008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:46.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.930+0000 7fecc103c700 1 -- 172.21.15.40:0/1609381642 >> v2:172.21.15.40:3300/0 conn(0x7feca0003730 msgr2=0x7feca0005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:13:46.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.930+0000 7fecc103c700 1 -- 172.21.15.40:0/1609381642 >> v2:172.21.15.40:3300/0 conn(0x7feca0003730 msgr2=0x7feca0005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:13:46.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.930+0000 7fecc103c700 1 --2- 172.21.15.40:0/1609381642 >> v2:172.21.15.40:3300/0 conn(0x7feca0003730 0x7feca0005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:13:46.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.930+0000 7fecc103c700 1 --2- 172.21.15.40:0/1609381642 >> v2:172.21.15.40:3300/0 conn(0x7feca0003730 0x7feca0005bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:13:46.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.930+0000 7fecc083b700 1 -- 172.21.15.40:0/1609381642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca00061a0 msgr2=0x7feca0008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:13:46.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.930+0000 7fecc083b700 1 -- 172.21.15.40:0/1609381642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca00061a0 msgr2=0x7feca0008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:13:46.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.930+0000 7fecc083b700 1 --2- 172.21.15.40:0/1609381642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca00061a0 0x7feca0008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:13:46.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:46.930+0000 7fecc083b700 1 --2- 172.21.15.40:0/1609381642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca00061a0 0x7feca0008660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:13:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.734+0000 7fecc103c700 1 --2- 172.21.15.40:0/1609381642 >> v2:172.21.15.40:3300/0 conn(0x7feca0003730 0x7feca0005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:47.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.734+0000 7fecc083b700 1 --2- 172.21.15.40:0/1609381642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca00061a0 0x7feca0008660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.734+0000 7fecc103c700 1 -- 172.21.15.40:0/1609381642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca00061a0 msgr2=0x7feca0008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:13:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.734+0000 7fecc103c700 1 --2- 172.21.15.40:0/1609381642 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca00061a0 0x7feca0008660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.734+0000 7fecc103c700 1 -- 172.21.15.40:0/1609381642 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecbc15cf10 con 0x7feca0003730 2024-09-13T18:13:47.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.734+0000 7fecc103c700 1 --2- 172.21.15.40:0/1609381642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feca0003730 0x7feca0005bb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fecb8007dd0 tx=0x7fecb8007330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afe34a4e26c0409 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:47.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.738+0000 7fecb3fff700 1 -- 172.21.15.40:0/1609381642 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb8005d90 con 0x7feca0003730 2024-09-13T18:13:47.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.738+0000 7fecb3fff700 1 -- 172.21.15.40:0/1609381642 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb8016480 con 0x7feca0003730 2024-09-13T18:13:47.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.738+0000 7fecb3fff700 1 -- 172.21.15.40:0/1609381642 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb8016830 con 0x7feca0003730 2024-09-13T18:13:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.738+0000 7fecc32a0700 1 -- 172.21.15.40:0/1609381642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feca0003730 msgr2=0x7feca0005bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.738+0000 7fecc32a0700 1 --2- 172.21.15.40:0/1609381642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feca0003730 0x7feca0005bb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fecb8007dd0 tx=0x7fecb8007330 comp rx=0 tx=0).stop 2024-09-13T18:13:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.738+0000 7fecc183d700 1 -- 172.21.15.40:0/1609381642 reap_dead start 2024-09-13T18:13:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.738+0000 7fecc32a0700 1 -- 172.21.15.40:0/1609381642 shutdown_connections 2024-09-13T18:13:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.738+0000 7fecc32a0700 1 --2- 172.21.15.40:0/1609381642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feca0003730 0x7feca0005bb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:47.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.738+0000 7fecc32a0700 1 -- 172.21.15.40:0/1609381642 >> 172.21.15.40:0/1609381642 conn(0x7fecbc06a390 msgr2=0x7fecbc06a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.738+0000 7fecc32a0700 1 -- 172.21.15.40:0/1609381642 shutdown_connections 2024-09-13T18:13:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.738+0000 7fecc32a0700 1 -- 172.21.15.40:0/1609381642 wait complete. 2024-09-13T18:13:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc32a0700 1 Processor -- start 2024-09-13T18:13:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc32a0700 1 -- start start 2024-09-13T18:13:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc32a0700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca0003730 0x7fecbc1efc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc32a0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecbc1584b0 0x7fecbc15b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc32a0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecbc154730 0x7fecbc154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc32a0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc15c610 con 0x7fecbc154730 2024-09-13T18:13:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc32a0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc1f0170 con 0x7fecbc1584b0 2024-09-13T18:13:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc32a0700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc15ac80 con 0x7feca0003730 2024-09-13T18:13:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc083b700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecbc1584b0 msgr2=0x7fecbc15b110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:13:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc083b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecbc1584b0 0x7fecbc15b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:47.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc183d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecbc154730 0x7fecbc154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc103c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca0003730 0x7fecbc1efc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc183d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecbc154730 0x7fecbc154b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38168/0 (socket says 172.21.15.40:38168) 2024-09-13T18:13:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc183d700 1 -- 172.21.15.40:0/1346409930 learned_addr learned my addr 172.21.15.40:0/1346409930 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc183d700 1 -- 172.21.15.40:0/1346409930 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca0003730 msgr2=0x7fecbc1efc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc183d700 1 --2- 172.21.15.40:0/1346409930 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca0003730 0x7fecbc1efc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc183d700 1 -- 172.21.15.40:0/1346409930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecbc1584b0 msgr2=0x7fecbc15b110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:13:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc183d700 1 --2- 172.21.15.40:0/1346409930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecbc1584b0 0x7fecbc15b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc183d700 1 -- 172.21.15.40:0/1346409930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb800f040 con 0x7fecbc154730 2024-09-13T18:13:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc183d700 1 --2- 172.21.15.40:0/1346409930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecbc154730 0x7fecbc154b90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fecb400bd20 tx=0x7fecb4009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:47.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecb1ffb700 1 -- 172.21.15.40:0/1346409930 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb400cdd0 con 0x7fecbc154730 2024-09-13T18:13:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecb1ffb700 1 -- 172.21.15.40:0/1346409930 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb4010070 con 0x7fecbc154730 2024-09-13T18:13:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc32a0700 1 -- 172.21.15.40:0/1346409930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecbc15bab0 con 0x7fecbc154730 2024-09-13T18:13:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecb1ffb700 1 -- 172.21.15.40:0/1346409930 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb40155d0 con 0x7fecbc154730 2024-09-13T18:13:47.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.742+0000 7fecc32a0700 1 -- 172.21.15.40:0/1346409930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecbc15c050 con 0x7fecbc154730 2024-09-13T18:13:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.746+0000 7fecc32a0700 1 -- 172.21.15.40:0/1346409930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecbc0c8df0 con 0x7fecbc154730 2024-09-13T18:13:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.746+0000 7fecb1ffb700 1 -- 172.21.15.40:0/1346409930 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fecb4015730 con 0x7fecbc154730 2024-09-13T18:13:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.746+0000 7fecb1ffb700 1 --2- 172.21.15.40:0/1346409930 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7feca0067380 0x7feca0069840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.746+0000 7fecb1ffb700 1 -- 172.21.15.40:0/1346409930 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fecb4084000 con 0x7fecbc154730 2024-09-13T18:13:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.746+0000 7fecc103c700 1 --2- 172.21.15.40:0/1346409930 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7feca0067380 0x7feca0069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.746+0000 7fecc103c700 1 --2- 172.21.15.40:0/1346409930 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7feca0067380 0x7feca0069840 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fecb8007370 tx=0x7fecb8024040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:47.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:47.750+0000 7fecb1ffb700 1 -- 172.21.15.40:0/1346409930 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fecb4050fb0 con 0x7fecbc154730 2024-09-13T18:13:48.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.034+0000 7fecc32a0700 1 -- 172.21.15.40:0/1346409930 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fecbc15adc0 con 0x7fecbc154730 2024-09-13T18:13:48.045 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.034+0000 7fecb1ffb700 1 -- 172.21.15.40:0/1346409930 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7fecb401b020 con 0x7fecbc154730 2024-09-13T18:13:48.045 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":190,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:13:48.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.042+0000 7fecc32a0700 1 -- 172.21.15.40:0/1346409930 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7feca0067380 msgr2=0x7feca0069840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:48.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.042+0000 7fecc32a0700 1 --2- 172.21.15.40:0/1346409930 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7feca0067380 0x7feca0069840 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fecb8007370 tx=0x7fecb8024040 comp rx=0 tx=0).stop 2024-09-13T18:13:48.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.042+0000 7fecc32a0700 1 -- 172.21.15.40:0/1346409930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecbc154730 msgr2=0x7fecbc154b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:48.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.042+0000 7fecc32a0700 1 --2- 172.21.15.40:0/1346409930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecbc154730 0x7fecbc154b90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fecb400bd20 tx=0x7fecb4009e10 comp rx=0 tx=0).stop 2024-09-13T18:13:48.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.042+0000 7fecc32a0700 1 -- 172.21.15.40:0/1346409930 shutdown_connections 2024-09-13T18:13:48.052 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.042+0000 7fecc32a0700 1 --2- 172.21.15.40:0/1346409930 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fecbc154730 0x7fecbc154b90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.042+0000 7fecc32a0700 1 --2- 172.21.15.40:0/1346409930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecbc1584b0 0x7fecbc15b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.042+0000 7fecc32a0700 1 --2- 172.21.15.40:0/1346409930 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7feca0067380 0x7feca0069840 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.042+0000 7fecc32a0700 1 --2- 172.21.15.40:0/1346409930 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feca0003730 0x7fecbc1efc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.042+0000 7fecc32a0700 1 -- 172.21.15.40:0/1346409930 >> 172.21.15.40:0/1346409930 conn(0x7fecbc06a390 msgr2=0x7fecbc070370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:48.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.042+0000 7fecc32a0700 1 -- 172.21.15.40:0/1346409930 shutdown_connections 2024-09-13T18:13:48.053 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.042+0000 7fecc32a0700 1 -- 172.21.15.40:0/1346409930 wait complete. 2024-09-13T18:13:48.066 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T18:13:48.066 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-13T18:13:48.066 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:13:48.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6ec11b4700 1 Processor -- start 2024-09-13T18:13:48.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6ec11b4700 1 -- start start 2024-09-13T18:13:48.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6ec11b4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6ebc153720 0x7f6ebc0ca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6ec11b4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6ebc0cf9b0 0x7f6ebc06c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6ec11b4700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ebc159e50 con 0x7f6ebc06c5b0 2024-09-13T18:13:48.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6ec11b4700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ebc0c5ea0 con 0x7f6ebc153720 2024-09-13T18:13:48.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6ec11b4700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ebc15d390 con 0x7f6ebc0cf9b0 2024-09-13T18:13:48.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6ebbfff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6ebc153720 0x7f6ebc0ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:48.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6eb37fe700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7f6ebc0cf9b0 msgr2=0x7f6ebc06c070 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:13:48.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6ec09b3700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6ebc06c5b0 0x7f6ebc06aac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59026/0 (socket says 172.21.15.40:59026) 2024-09-13T18:13:48.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6ec09b3700 1 -- 172.21.15.40:0/2252942603 learned_addr learned my addr 172.21.15.40:0/2252942603 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:13:48.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6eb37fe700 1 --2- 172.21.15.40:0/2252942603 >> v2:172.21.15.43:3300/0 conn(0x7f6ebc0cf9b0 0x7f6ebc06c070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:48.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6ebbfff700 1 -- 172.21.15.40:0/2252942603 >> v2:172.21.15.43:3300/0 conn(0x7f6ebc0cf9b0 msgr2=0x7f6ebc06c070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:13:48.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6ebbfff700 1 --2- 172.21.15.40:0/2252942603 >> v2:172.21.15.43:3300/0 conn(0x7f6ebc0cf9b0 0x7f6ebc06c070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6ebbfff700 1 -- 172.21.15.40:0/2252942603 >> v1:172.21.15.40:6789/0 conn(0x7f6ebc06c5b0 legacy=0x7f6ebc06aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:48.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6ebb7fe700 1 -- 172.21.15.40:0/2252942603 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2979656763 0 0) 0x7f6ebc159e50 con 0x7f6ebc06c5b0 2024-09-13T18:13:48.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6ebbfff700 1 -- 172.21.15.40:0/2252942603 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc15a080 con 0x7f6ebc153720 2024-09-13T18:13:48.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6ebbfff700 1 --2- 172.21.15.40:0/2252942603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc153720 0x7f6ebc0ca510 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0b1b20 tx=0x7f6eac009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a979ed6961be5807 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:48.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6ebb7fe700 1 -- 172.21.15.40:0/2252942603 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac011830 con 0x7f6ebc153720 2024-09-13T18:13:48.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6ebb7fe700 1 -- 172.21.15.40:0/2252942603 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac011990 con 0x7f6ebc153720 2024-09-13T18:13:48.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.182+0000 7f6ebb7fe700 1 -- 172.21.15.40:0/2252942603 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac011d40 con 0x7f6ebc153720 2024-09-13T18:13:48.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.186+0000 7f6ec11b4700 1 -- 172.21.15.40:0/2252942603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc153720 msgr2=0x7f6ebc0ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:48.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.186+0000 7f6ec11b4700 1 --2- 172.21.15.40:0/2252942603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc153720 0x7f6ebc0ca510 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0b1b20 tx=0x7f6eac009b30 comp rx=0 tx=0).stop 2024-09-13T18:13:48.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.186+0000 7f6ec11b4700 1 -- 172.21.15.40:0/2252942603 shutdown_connections 2024-09-13T18:13:48.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.186+0000 7f6ec11b4700 1 --2- 172.21.15.40:0/2252942603 >> v2:172.21.15.43:3300/0 conn(0x7f6ebc0cf9b0 0x7f6ebc06c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.186+0000 7f6ec11b4700 1 --2- 172.21.15.40:0/2252942603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc153720 0x7f6ebc0ca510 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.186+0000 7f6ec11b4700 1 -- 172.21.15.40:0/2252942603 >> 172.21.15.40:0/2252942603 conn(0x7f6ebc0666c0 msgr2=0x7f6ebc0cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:48.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.186+0000 7f6ec11b4700 1 -- 172.21.15.40:0/2252942603 shutdown_connections 2024-09-13T18:13:48.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.186+0000 7f6ec11b4700 1 -- 172.21.15.40:0/2252942603 wait complete. 2024-09-13T18:13:48.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.186+0000 7f6ec11b4700 1 Processor -- start 2024-09-13T18:13:48.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.186+0000 7f6ec11b4700 1 -- start start 2024-09-13T18:13:48.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ec11b4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc06c5b0 0x7f6ebc155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ec11b4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ebc0cf9b0 0x7f6ebc15be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ec11b4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ebc153720 0x7f6ebc1eef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ec11b4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc151f10 con 0x7f6ebc06c5b0 2024-09-13T18:13:48.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ec11b4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc0ca000 con 0x7f6ebc153720 2024-09-13T18:13:48.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ec11b4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1567a0 con 0x7f6ebc0cf9b0 2024-09-13T18:13:48.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ebbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc06c5b0 0x7f6ebc155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:48.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ebbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc06c5b0 0x7f6ebc155e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38174/0 (socket says 172.21.15.40:38174) 2024-09-13T18:13:48.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ebbfff700 1 -- 172.21.15.40:0/1917917443 learned_addr learned my addr 172.21.15.40:0/1917917443 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:48.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ec09b3700 1 -- 172.21.15.40:0/1917917443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ebc153720 msgr2=0x7f6ebc1eef90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:13:48.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ec09b3700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ebc153720 0x7f6ebc1eef90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:48.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6eb37fe700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ebc0cf9b0 0x7f6ebc15be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:48.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ebbfff700 1 -- 172.21.15.40:0/1917917443 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ebc0cf9b0 msgr2=0x7f6ebc15be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:48.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ebbfff700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ebc0cf9b0 0x7f6ebc15be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ebbfff700 1 -- 172.21.15.40:0/1917917443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ebc153720 msgr2=0x7f6ebc1eef90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:13:48.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ebbfff700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ebc153720 0x7f6ebc1eef90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ebbfff700 1 -- 172.21.15.40:0/1917917443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac0094b0 con 0x7f6ebc06c5b0 2024-09-13T18:13:48.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6eb37fe700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ebc0cf9b0 0x7f6ebc15be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:13:48.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ebbfff700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc06c5b0 0x7f6ebc155e10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0090b0 tx=0x7f6eac004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:48.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6eb97fa700 1 -- 172.21.15.40:0/1917917443 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac0036c0 con 0x7f6ebc06c5b0 2024-09-13T18:13:48.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6eb97fa700 1 -- 172.21.15.40:0/1917917443 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac01e070 con 0x7f6ebc06c5b0 2024-09-13T18:13:48.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6eb97fa700 1 -- 172.21.15.40:0/1917917443 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac003a60 con 0x7f6ebc06c5b0 2024-09-13T18:13:48.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ec11b4700 1 -- 172.21.15.40:0/1917917443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc1564c0 con 0x7f6ebc06c5b0 2024-09-13T18:13:48.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ec11b4700 1 -- 172.21.15.40:0/1917917443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ebc1f2880 con 0x7f6ebc06c5b0 2024-09-13T18:13:48.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ec11b4700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc1ef4d0 0x7f6ebc1f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6ec11b4700 1 -- 172.21.15.40:0/1917917443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc074750 con 0x7f6ebc1ef4d0 2024-09-13T18:13:48.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6eb37fe700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc1ef4d0 0x7f6ebc1f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:48.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6eb37fe700 1 -- 172.21.15.40:0/1917917443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ebc15ca60 con 0x7f6ebc1ef4d0 2024-09-13T18:13:48.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6eb37fe700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc1ef4d0 0x7f6ebc1f2aa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0cf980 tx=0x7f6ea400c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:48.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6eb97fa700 1 -- 172.21.15.40:0/1917917443 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4003e70 con 0x7f6ebc1ef4d0 2024-09-13T18:13:48.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6eb97fa700 1 -- 172.21.15.40:0/1917917443 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6ea4005070 con 0x7f6ebc1ef4d0 2024-09-13T18:13:48.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6eb97fa700 1 -- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc1ef4d0 msgr2=0x7f6ebc1f2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:48.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6eb97fa700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc1ef4d0 0x7f6ebc1f2aa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0cf980 tx=0x7f6ea400c9d0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6eb97fa700 1 -- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc1ef4d0 msgr2=0x7f6ebc1f2aa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:13:48.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.190+0000 7f6eb97fa700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc1ef4d0 0x7f6ebc1f2aa0 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0cf980 tx=0x7f6ea400c9d0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.194+0000 7f6eb97fa700 1 -- 172.21.15.40:0/1917917443 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6eac003bc0 con 0x7f6ebc06c5b0 2024-09-13T18:13:48.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.194+0000 7f6eb97fa700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e9c0664b0 0x7f6e9c068970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.194+0000 7f6eb97fa700 1 -- 172.21.15.40:0/1917917443 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6eac020070 con 0x7f6ebc06c5b0 2024-09-13T18:13:48.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.194+0000 7f6eb37fe700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e9c0664b0 0x7f6e9c068970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:48.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.194+0000 7f6eb37fe700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e9c0664b0 0x7f6e9c068970 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0cf980 tx=0x7f6ea4005000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.210+0000 7f6ec11b4700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc1553e0 0x7f6ebc14c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.210+0000 7f6ec11b4700 1 -- 172.21.15.40:0/1917917443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc153b00 con 0x7f6ebc1553e0 2024-09-13T18:13:48.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.210+0000 7f6ec09b3700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc1553e0 0x7f6ebc14c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:48.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.210+0000 7f6ec09b3700 1 -- 172.21.15.40:0/1917917443 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6ebc15cd40 con 0x7f6ebc1553e0 2024-09-13T18:13:48.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.210+0000 7f6ec09b3700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc1553e0 0x7f6ebc14c860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4004c40 tx=0x7f6eb400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:48.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.210+0000 7f6eb97fa700 1 -- 172.21.15.40:0/1917917443 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb400aa00 con 0x7f6ebc1553e0 2024-09-13T18:13:48.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.210+0000 7f6eb97fa700 1 -- 172.21.15.40:0/1917917443 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4773 (secure 0 0 0) 0x7f6eb400ab60 con 0x7f6ebc1553e0 2024-09-13T18:13:48.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.210+0000 7f6eb97fa700 1 -- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc1553e0 msgr2=0x7f6ebc14c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:48.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.210+0000 7f6eb97fa700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc1553e0 0x7f6ebc14c860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4004c40 tx=0x7f6eb400cf50 comp rx=0 tx=0).stop 2024-09-13T18:13:48.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.210+0000 7f6eb97fa700 1 -- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc1553e0 msgr2=0x7f6ebc14c860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:13:48.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.210+0000 7f6eb97fa700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc1553e0 0x7f6ebc14c860 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.221 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:13:48.221 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:13:48.221 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:13:48.221 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:13:48.222 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 190, 2024-09-13T18:13:48.222 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:13:48.222 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:13:48.222 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:13:48.222 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:48.222 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:13:48.222 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:13:48.222 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:13:48.222 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:13:48.223 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:48.223 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:48.223 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:48.223 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:48.223 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:48.223 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:48.223 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:48.223 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:48.223 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:48.223 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:48.223 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:48.223 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:13:48.224 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:13:48.224 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:48.224 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:48.224 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:48.224 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:48.224 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:48.224 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:48.224 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:48.224 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:48.224 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:48.224 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:48.224 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:48.224 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.224 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:13:48.225 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:13:48.225 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:13:48.225 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:13:48.225 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:13:48.225 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:13:48.225 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:13:48.225 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:13:48.225 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:13:48.225 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:13:48.225 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:13:48.225 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:13:48.225 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:13:48.225 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:13:48.226 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:13:48.226 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:13:48.226 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:13:48.226 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:48.226 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:48.226 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:48.226 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:48.226 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:48.226 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:48.226 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:48.226 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:48.226 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:48.226 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:48.227 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:48.227 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:13:48.227 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.227 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:13:48.227 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.227 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:13:48.227 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:13:48.227 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:48.227 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:48.228 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.228 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:48.228 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:13:48.228 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:48.228 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.228 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.228 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:48.228 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:13:48.228 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:48.228 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:48.228 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:48.228 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.228 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:13:48.229 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:13:48.229 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:48.229 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:48.229 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:48.229 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.229 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.229 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:13:48.229 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:13:48.229 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:48.229 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:48.229 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.229 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:48.229 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:13:48.229 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:48.230 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.230 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.230 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:48.230 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:13:48.230 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:48.230 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:48.230 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:48.230 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.230 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:13:48.230 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:13:48.230 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:48.230 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:48.230 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:48.230 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.231 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.231 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:13:48.231 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:13:48.231 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:48.231 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:48.231 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.231 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:48.231 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:13:48.231 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:48.231 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.231 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.231 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:48.231 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:13:48.231 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:48.232 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:48.232 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:48.232 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.232 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:13:48.232 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:13:48.232 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:48.232 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:48.232 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:48.232 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:48.232 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:48.232 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.233 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:13:48.233 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:13:48.233 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.233 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:48.233 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:48.233 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:13:48.233 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:48.233 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:48.234 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:13:48.234 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.234 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:48.234 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:48.234 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:13:48.234 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:48.234 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:48.234 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:13:48.234 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.234 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:48.234 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:48.234 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:13:48.234 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:48.234 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:48.235 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:13:48.235 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.235 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:48.235 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:48.235 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:13:48.235 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:48.235 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:48.235 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.235 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:13:48.235 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:13:48.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.214+0000 7f6ec11b4700 1 -- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e9c0664b0 msgr2=0x7f6e9c068970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:48.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.214+0000 7f6ec11b4700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e9c0664b0 0x7f6e9c068970 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0cf980 tx=0x7f6ea4005000 comp rx=0 tx=0).stop 2024-09-13T18:13:48.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.214+0000 7f6ec11b4700 1 -- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc06c5b0 msgr2=0x7f6ebc155e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:48.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.214+0000 7f6ec11b4700 1 --2- 172.21.15.40:0/1917917443 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6ebc06c5b0 0x7f6ebc155e10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0090b0 tx=0x7f6eac004770 comp rx=0 tx=0).stop 2024-09-13T18:13:48.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.214+0000 7f6ec09b3700 1 -- 172.21.15.40:0/1917917443 reap_dead start 2024-09-13T18:13:48.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.214+0000 7f6ec11b4700 1 -- 172.21.15.40:0/1917917443 shutdown_connections 2024-09-13T18:13:48.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.214+0000 7f6ec11b4700 1 -- 172.21.15.40:0/1917917443 >> 172.21.15.40:0/1917917443 conn(0x7f6ebc0666c0 msgr2=0x7f6ebc150f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:48.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.214+0000 7f6ec11b4700 1 -- 172.21.15.40:0/1917917443 shutdown_connections 2024-09-13T18:13:48.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.214+0000 7f6ec11b4700 1 -- 172.21.15.40:0/1917917443 wait complete. 2024-09-13T18:13:48.237 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:13:48.238 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T18:13:48.238 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:13:48.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.346+0000 7fa15ca02700 1 Processor -- start 2024-09-13T18:13:48.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.346+0000 7fa15ca02700 1 -- start start 2024-09-13T18:13:48.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.346+0000 7fa15ca02700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa158150240 0x7fa158071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.346+0000 7fa15ca02700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1580c54e0 0x7fa15815a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.346+0000 7fa15ca02700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1580706f0 con 0x7fa1580c4f90 2024-09-13T18:13:48.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.346+0000 7fa15ca02700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1580c28c0 con 0x7fa158150240 2024-09-13T18:13:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.346+0000 7fa15ca02700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa158070450 con 0x7fa1580c54e0 2024-09-13T18:13:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.346+0000 7fa156d9d700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7fa158150240 msgr2=0x7fa158071660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:13:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.346+0000 7fa156d9d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa158150240 0x7fa158071660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.346+0000 7fa15759e700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa1580c4f90 0x7fa158070ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59030/0 (socket says 172.21.15.40:59030) 2024-09-13T18:13:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.346+0000 7fa15759e700 1 -- 172.21.15.40:0/947289125 learned_addr learned my addr 172.21.15.40:0/947289125 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:13:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.346+0000 7fa15659c700 1 --2- 172.21.15.40:0/947289125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1580c54e0 0x7fa15815a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.346+0000 7fa155d9b700 1 -- 172.21.15.40:0/947289125 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624705435 0 0) 0x7fa1580706f0 con 0x7fa1580c4f90 2024-09-13T18:13:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.346+0000 7fa155d9b700 1 -- 172.21.15.40:0/947289125 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1400035d0 con 0x7fa1580c4f90 2024-09-13T18:13:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa155d9b700 1 -- 172.21.15.40:0/947289125 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2460535881 0 0) 0x7fa1400035d0 con 0x7fa1580c4f90 2024-09-13T18:13:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa155d9b700 1 -- 172.21.15.40:0/947289125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1580c54e0 msgr2=0x7fa15815a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:13:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa155d9b700 1 --2- 172.21.15.40:0/947289125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1580c54e0 0x7fa15815a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa155d9b700 1 -- 172.21.15.40:0/947289125 >> v2:172.21.15.43:3300/0 conn(0x7fa158150240 msgr2=0x7fa158071660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:13:48.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa155d9b700 1 --2- 172.21.15.40:0/947289125 >> v2:172.21.15.43:3300/0 conn(0x7fa158150240 0x7fa158071660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa155d9b700 1 -- 172.21.15.40:0/947289125 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa158156c00 con 0x7fa1580c4f90 2024-09-13T18:13:48.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa155d9b700 1 -- 172.21.15.40:0/947289125 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fa1500019f0 con 0x7fa1580c4f90 2024-09-13T18:13:48.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa15659c700 1 --2- 172.21.15.40:0/947289125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1580c54e0 0x7fa15815a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:13:48.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa155d9b700 1 -- 172.21.15.40:0/947289125 >> v1:172.21.15.40:6789/0 conn(0x7fa1580c4f90 legacy=0x7fa158070ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:48.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa155d9b700 1 --2- 172.21.15.40:0/947289125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1580c54e0 0x7fa140003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa155d9b700 1 --2- 172.21.15.40:0/947289125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa140003f60 0x7fa140004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa155d9b700 1 --2- 172.21.15.40:0/947289125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa140004db0 0x7fa140005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa155d9b700 1 -- 172.21.15.40:0/947289125 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580c28c0 con 0x7fa140003f60 2024-09-13T18:13:48.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa155d9b700 1 -- 172.21.15.40:0/947289125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa158070450 con 0x7fa140004db0 2024-09-13T18:13:48.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa155d9b700 1 -- 172.21.15.40:0/947289125 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14000bb90 con 0x7fa1580c54e0 2024-09-13T18:13:48.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa15659c700 1 --2- 172.21.15.40:0/947289125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa140003f60 0x7fa140004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:48.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa15759e700 1 -- 172.21.15.40:0/947289125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa140004db0 msgr2=0x7fa140005230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:13:48.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa156d9d700 1 --2- 172.21.15.40:0/947289125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1580c54e0 0x7fa140003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:48.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa15759e700 1 --2- 172.21.15.40:0/947289125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa140004db0 0x7fa140005230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:48.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa15659c700 1 -- 172.21.15.40:0/947289125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1580c54e0 msgr2=0x7fa140003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:13:48.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa15659c700 1 --2- 172.21.15.40:0/947289125 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1580c54e0 0x7fa140003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa15659c700 1 -- 172.21.15.40:0/947289125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa140004db0 msgr2=0x7fa140005230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:13:48.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa15659c700 1 --2- 172.21.15.40:0/947289125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa140004db0 0x7fa140005230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa15659c700 1 -- 172.21.15.40:0/947289125 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1480094b0 con 0x7fa140003f60 2024-09-13T18:13:48.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa15659c700 1 --2- 172.21.15.40:0/947289125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa140003f60 0x7fa140004870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa148009d40 tx=0x7fa1400058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a51b01508b95a5f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:48.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa155d9b700 1 -- 172.21.15.40:0/947289125 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148009e90 con 0x7fa140003f60 2024-09-13T18:13:48.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa155d9b700 1 -- 172.21.15.40:0/947289125 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14800e9a0 con 0x7fa140003f60 2024-09-13T18:13:48.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa155d9b700 1 -- 172.21.15.40:0/947289125 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14800ed10 con 0x7fa140003f60 2024-09-13T18:13:48.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa15ca02700 1 -- 172.21.15.40:0/947289125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa140003f60 msgr2=0x7fa140004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:48.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa15ca02700 1 --2- 172.21.15.40:0/947289125 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa140003f60 0x7fa140004870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa148009d40 tx=0x7fa1400058a0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa15759e700 1 -- 172.21.15.40:0/947289125 reap_dead start 2024-09-13T18:13:48.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa15ca02700 1 -- 172.21.15.40:0/947289125 shutdown_connections 2024-09-13T18:13:48.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa15ca02700 1 -- 172.21.15.40:0/947289125 >> 172.21.15.40:0/947289125 conn(0x7fa15806a390 msgr2=0x7fa15806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:48.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa15ca02700 1 -- 172.21.15.40:0/947289125 shutdown_connections 2024-09-13T18:13:48.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.350+0000 7fa15ca02700 1 -- 172.21.15.40:0/947289125 wait complete. 2024-09-13T18:13:48.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa15ca02700 1 Processor -- start 2024-09-13T18:13:48.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa15ca02700 1 -- start start 2024-09-13T18:13:48.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa15ca02700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa140003f60 0x7fa158071660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa15ca02700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa15815b100 0x7fa158158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa15ca02700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1581f04d0 0x7fa1581f0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa15ca02700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1581efda0 con 0x7fa140003f60 2024-09-13T18:13:48.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa15ca02700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa158155300 con 0x7fa1581f04d0 2024-09-13T18:13:48.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa15ca02700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1581f0e90 con 0x7fa15815b100 2024-09-13T18:13:48.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa156d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa140003f60 0x7fa158071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:48.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa15759e700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1581f04d0 msgr2=0x7fa1581f0950 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:13:48.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa156d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa140003f60 0x7fa158071660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38208/0 (socket says 172.21.15.40:38208) 2024-09-13T18:13:48.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa156d9d700 1 -- 172.21.15.40:0/1340188303 learned_addr learned my addr 172.21.15.40:0/1340188303 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:13:48.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa15759e700 1 --2- 172.21.15.40:0/1340188303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1581f04d0 0x7fa1581f0950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:13:48.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa15659c700 1 --2- 172.21.15.40:0/1340188303 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa15815b100 0x7fa158158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:48.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa156d9d700 1 -- 172.21.15.40:0/1340188303 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa15815b100 msgr2=0x7fa158158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:48.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa156d9d700 1 --2- 172.21.15.40:0/1340188303 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa15815b100 0x7fa158158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa156d9d700 1 -- 172.21.15.40:0/1340188303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1581f04d0 msgr2=0x7fa1581f0950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:13:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa156d9d700 1 --2- 172.21.15.40:0/1340188303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1581f04d0 0x7fa1581f0950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa156d9d700 1 -- 172.21.15.40:0/1340188303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1581ef800 con 0x7fa140003f60 2024-09-13T18:13:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa156d9d700 1 --2- 172.21.15.40:0/1340188303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa140003f60 0x7fa158071660 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa14c00ebb0 tx=0x7fa14c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa13f7fe700 1 -- 172.21.15.40:0/1340188303 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c003e70 con 0x7fa140003f60 2024-09-13T18:13:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa13f7fe700 1 -- 172.21.15.40:0/1340188303 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14c00c930 con 0x7fa140003f60 2024-09-13T18:13:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa15ca02700 1 -- 172.21.15.40:0/1340188303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa15814ebe0 con 0x7fa140003f60 2024-09-13T18:13:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa13f7fe700 1 -- 172.21.15.40:0/1340188303 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c00cd20 con 0x7fa140003f60 2024-09-13T18:13:48.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.354+0000 7fa15ca02700 1 -- 172.21.15.40:0/1340188303 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1581efac0 con 0x7fa140003f60 2024-09-13T18:13:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.358+0000 7fa15ca02700 1 --2- 172.21.15.40:0/1340188303 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa158154df0 0x7fa158162ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.358+0000 7fa15ca02700 1 -- 172.21.15.40:0/1340188303 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1581efc50 con 0x7fa158154df0 2024-09-13T18:13:48.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.358+0000 7fa15659c700 1 --2- 172.21.15.40:0/1340188303 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa158154df0 0x7fa158162ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.358+0000 7fa13f7fe700 1 -- 172.21.15.40:0/1340188303 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa14c005070 con 0x7fa140003f60 2024-09-13T18:13:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.358+0000 7fa15659c700 1 -- 172.21.15.40:0/1340188303 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa1581598e0 con 0x7fa158154df0 2024-09-13T18:13:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.358+0000 7fa15659c700 1 --2- 172.21.15.40:0/1340188303 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa158154df0 0x7fa158162ca0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa148009d40 tx=0x7fa148004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.358+0000 7fa13f7fe700 1 --2- 172.21.15.40:0/1340188303 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1440666e0 0x7fa144068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.358+0000 7fa13f7fe700 1 -- 172.21.15.40:0/1340188303 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa14c084be0 con 0x7fa140003f60 2024-09-13T18:13:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.358+0000 7fa15759e700 1 --2- 172.21.15.40:0/1340188303 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1440666e0 0x7fa144068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.358+0000 7fa13f7fe700 1 -- 172.21.15.40:0/1340188303 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148021040 con 0x7fa158154df0 2024-09-13T18:13:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.358+0000 7fa15759e700 1 --2- 172.21.15.40:0/1340188303 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1440666e0 0x7fa144068ba0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa15806c0b0 tx=0x7fa150001850 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.358+0000 7fa13f7fe700 1 -- 172.21.15.40:0/1340188303 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa148009e90 con 0x7fa158154df0 2024-09-13T18:13:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.358+0000 7fa13f7fe700 1 -- 172.21.15.40:0/1340188303 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa158154df0 msgr2=0x7fa158162ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.358+0000 7fa13f7fe700 1 --2- 172.21.15.40:0/1340188303 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa158154df0 0x7fa158162ca0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa148009d40 tx=0x7fa148004770 comp rx=0 tx=0).stop 2024-09-13T18:13:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.358+0000 7fa13f7fe700 1 -- 172.21.15.40:0/1340188303 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa158154df0 msgr2=0x7fa158162ca0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:13:48.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.358+0000 7fa13f7fe700 1 --2- 172.21.15.40:0/1340188303 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa158154df0 0x7fa158162ca0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.366+0000 7fa15ca02700 1 --2- 172.21.15.40:0/1340188303 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1581631e0 0x7fa1581f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:13:48.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.366+0000 7fa15ca02700 1 -- 172.21.15.40:0/1340188303 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa158067600 con 0x7fa1581631e0 2024-09-13T18:13:48.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.366+0000 7fa15659c700 1 --2- 172.21.15.40:0/1340188303 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1581631e0 0x7fa1581f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:13:48.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.370+0000 7fa15659c700 1 -- 172.21.15.40:0/1340188303 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa158159600 con 0x7fa1581631e0 2024-09-13T18:13:48.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.370+0000 7fa15659c700 1 --2- 172.21.15.40:0/1340188303 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1581631e0 0x7fa1581f2290 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa1580c22a0 tx=0x7fa1480047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:13:48.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.370+0000 7fa13f7fe700 1 -- 172.21.15.40:0/1340188303 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1480068e0 con 0x7fa1581631e0 2024-09-13T18:13:48.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.370+0000 7fa13f7fe700 1 -- 172.21.15.40:0/1340188303 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4937 (secure 0 0 0) 0x7fa14802e300 con 0x7fa1581631e0 2024-09-13T18:13:48.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.370+0000 7fa13f7fe700 1 -- 172.21.15.40:0/1340188303 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1581631e0 msgr2=0x7fa1581f2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:48.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.370+0000 7fa13f7fe700 1 --2- 172.21.15.40:0/1340188303 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1581631e0 0x7fa1581f2290 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa1580c22a0 tx=0x7fa1480047e0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.370+0000 7fa13f7fe700 1 -- 172.21.15.40:0/1340188303 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1581631e0 msgr2=0x7fa1581f2290 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:13:48.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.370+0000 7fa13f7fe700 1 --2- 172.21.15.40:0/1340188303 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa1581631e0 0x7fa1581f2290 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.381 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:13:48.381 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:13:48.381 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:13:48.381 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:13:48.381 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 190, 2024-09-13T18:13:48.381 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:13:48.381 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:13:48.381 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:13:48.381 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:48.381 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:13:48.381 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:13:48.381 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:13:48.382 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:13:48.382 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:48.382 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:48.382 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:48.382 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:48.382 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:48.382 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:48.382 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:48.382 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:48.382 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:48.383 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:48.383 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:48.383 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:13:48.383 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:13:48.383 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:48.383 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:48.383 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:48.384 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:48.384 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:48.384 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:48.384 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:48.384 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:48.384 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:48.385 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:48.385 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:48.385 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.385 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:13:48.385 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:13:48.385 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:13:48.386 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:13:48.386 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:13:48.386 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:13:48.386 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:13:48.386 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:13:48.386 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:13:48.387 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:13:48.387 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:13:48.387 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:13:48.387 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:13:48.387 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:13:48.387 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:13:48.388 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:13:48.388 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:13:48.388 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:13:48.388 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:13:48.388 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:13:48.388 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:13:48.388 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:13:48.388 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:13:48.389 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:13:48.389 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:13:48.389 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:13:48.389 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:13:48.389 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:48.389 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:13:48.389 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.389 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:13:48.389 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.389 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:13:48.389 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:13:48.389 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:48.389 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:48.389 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.390 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:48.390 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:13:48.390 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:48.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.390 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.390 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:48.390 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:13:48.390 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:48.390 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:48.390 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:48.390 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.391 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:13:48.391 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:13:48.391 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:48.391 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:48.391 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:48.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.391 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.391 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:13:48.391 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:13:48.391 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:48.391 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:48.391 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.392 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:48.392 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:13:48.392 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:48.392 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.392 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.392 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:48.392 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:13:48.392 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:48.392 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:48.392 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:48.392 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.392 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:13:48.392 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:13:48.392 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:48.392 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:48.393 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:48.393 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.393 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.393 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:13:48.393 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:13:48.393 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:13:48.393 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:13:48.393 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.393 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:13:48.393 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:13:48.393 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:48.393 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.393 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.393 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:13:48.393 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:13:48.394 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:13:48.394 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:48.394 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:48.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.394 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:13:48.394 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:13:48.394 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:13:48.394 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:13:48.394 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:13:48.394 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:48.394 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:48.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.394 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:13:48.394 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:13:48.394 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.394 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:48.395 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:48.395 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:13:48.395 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:48.395 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:48.395 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:13:48.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.395 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:48.395 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:48.395 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:13:48.395 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:48.395 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:48.395 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:13:48.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.395 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:48.395 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:48.395 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:13:48.396 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:48.396 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:48.396 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:13:48.396 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.396 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:48.396 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:48.396 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:13:48.396 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:48.396 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:13:48.396 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:13:48.396 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:13:48.396 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:13:48.396 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:13:48.396 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:13:48.397 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:13:48.397 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:13:48.397 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:13:48.397 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:13:48.397 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:13:48.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.370+0000 7fa15ca02700 1 -- 172.21.15.40:0/1340188303 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1440666e0 msgr2=0x7fa144068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:48.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.370+0000 7fa15ca02700 1 --2- 172.21.15.40:0/1340188303 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1440666e0 0x7fa144068ba0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa15806c0b0 tx=0x7fa150001850 comp rx=0 tx=0).stop 2024-09-13T18:13:48.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.370+0000 7fa15ca02700 1 -- 172.21.15.40:0/1340188303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa140003f60 msgr2=0x7fa158071660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:13:48.398 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.370+0000 7fa15ca02700 1 --2- 172.21.15.40:0/1340188303 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa140003f60 0x7fa158071660 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa14c00ebb0 tx=0x7fa14c00eec0 comp rx=0 tx=0).stop 2024-09-13T18:13:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.370+0000 7fa15759e700 1 -- 172.21.15.40:0/1340188303 reap_dead start 2024-09-13T18:13:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.370+0000 7fa15759e700 1 -- 172.21.15.40:0/1340188303 reap_dead start 2024-09-13T18:13:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.370+0000 7fa15ca02700 1 -- 172.21.15.40:0/1340188303 shutdown_connections 2024-09-13T18:13:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.370+0000 7fa15ca02700 1 -- 172.21.15.40:0/1340188303 >> 172.21.15.40:0/1340188303 conn(0x7fa15806a390 msgr2=0x7fa1580cb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:13:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.370+0000 7fa15ca02700 1 -- 172.21.15.40:0/1340188303 shutdown_connections 2024-09-13T18:13:48.399 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:13:48.370+0000 7fa15ca02700 1 -- 172.21.15.40:0/1340188303 wait complete. 2024-09-13T18:13:48.399 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:13:48.399 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:14:07.861 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:14:07.863 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:14:08.400 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-13T18:14:08.400 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-13T18:14:08.401 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T18:14:08.401 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T18:14:08.403 INFO:tasks.ceph.mon.b:Started 2024-09-13T18:14:08.403 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:14:08.404 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:14:08.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.514+0000 7ff678932700 1 Processor -- start 2024-09-13T18:14:08.525 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.514+0000 7ff678932700 1 -- start start 2024-09-13T18:14:08.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff678932700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff674151470 0x7ff6740c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:08.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff678932700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff6740b1bd0 0x7ff6741571d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:08.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff678932700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff674157710 0x7ff674071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:08.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff678932700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6740c4520 con 0x7ff6740b1bd0 2024-09-13T18:14:08.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff678932700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6740c4dc0 con 0x7ff674157710 2024-09-13T18:14:08.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff678932700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff674157cf0 con 0x7ff674151470 2024-09-13T18:14:08.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff67259c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff6740b1bd0 0x7ff6741571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:08.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff67259c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff6740b1bd0 0x7ff6741571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43680/0 (socket says 172.21.15.40:43680) 2024-09-13T18:14:08.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff67259c700 1 -- 172.21.15.40:0/1052558013 learned_addr learned my addr 172.21.15.40:0/1052558013 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff67359e700 1 --2- 172.21.15.40:0/1052558013 >> v2:172.21.15.43:3300/0 conn(0x7ff674157710 0x7ff674071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff672d9d700 1 --2- 172.21.15.40:0/1052558013 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff674151470 0x7ff6740c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff67359e700 1 -- 172.21.15.40:0/1052558013 >> v2:172.21.15.43:3300/0 conn(0x7ff674157710 msgr2=0x7ff674071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:14:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff67359e700 1 -- 172.21.15.40:0/1052558013 >> v2:172.21.15.43:3300/0 conn(0x7ff674157710 msgr2=0x7ff674071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:14:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff67259c700 1 -- 172.21.15.40:0/1052558013 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff674151470 msgr2=0x7ff6740c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:14:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff67259c700 1 --2- 172.21.15.40:0/1052558013 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff674151470 0x7ff6740c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff67359e700 1 --2- 172.21.15.40:0/1052558013 >> v2:172.21.15.43:3300/0 conn(0x7ff674157710 0x7ff674071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:14:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff67259c700 1 -- 172.21.15.40:0/1052558013 >> v2:172.21.15.43:3300/0 conn(0x7ff674157710 msgr2=0x7ff674071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:14:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff67359e700 1 --2- 172.21.15.40:0/1052558013 >> v2:172.21.15.43:3300/0 conn(0x7ff674157710 0x7ff674071bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:14:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff67259c700 1 --2- 172.21.15.40:0/1052558013 >> v2:172.21.15.43:3300/0 conn(0x7ff674157710 0x7ff674071bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff67259c700 1 -- 172.21.15.40:0/1052558013 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff674157fd0 con 0x7ff6740b1bd0 2024-09-13T18:14:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff67259c700 1 --2- 172.21.15.40:0/1052558013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6740b1bd0 0x7ff6741571d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff6740b2140 tx=0x7ff668009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ed6f78b8afe8b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff671d9b700 1 -- 172.21.15.40:0/1052558013 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff668011830 con 0x7ff6740b1bd0 2024-09-13T18:14:08.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff671d9b700 1 -- 172.21.15.40:0/1052558013 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff668011990 con 0x7ff6740b1bd0 2024-09-13T18:14:08.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff671d9b700 1 -- 172.21.15.40:0/1052558013 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff668011d20 con 0x7ff6740b1bd0 2024-09-13T18:14:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff678932700 1 -- 172.21.15.40:0/1052558013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6740b1bd0 msgr2=0x7ff6741571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff678932700 1 --2- 172.21.15.40:0/1052558013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6740b1bd0 0x7ff6741571d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff6740b2140 tx=0x7ff668009bf0 comp rx=0 tx=0).stop 2024-09-13T18:14:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff678932700 1 -- 172.21.15.40:0/1052558013 shutdown_connections 2024-09-13T18:14:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff678932700 1 --2- 172.21.15.40:0/1052558013 >> v2:172.21.15.43:3300/0 conn(0x7ff674157710 0x7ff674071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff678932700 1 --2- 172.21.15.40:0/1052558013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6740b1bd0 0x7ff6741571d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff678932700 1 --2- 172.21.15.40:0/1052558013 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff674151470 0x7ff6740c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff678932700 1 -- 172.21.15.40:0/1052558013 >> 172.21.15.40:0/1052558013 conn(0x7ff67406e4f0 msgr2=0x7ff67406e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff678932700 1 -- 172.21.15.40:0/1052558013 shutdown_connections 2024-09-13T18:14:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.518+0000 7ff678932700 1 -- 172.21.15.40:0/1052558013 wait complete. 2024-09-13T18:14:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff678932700 1 Processor -- start 2024-09-13T18:14:08.531 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff678932700 1 -- start start 2024-09-13T18:14:08.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff678932700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6740b1bd0 0x7ff6741ece10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff678932700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff674151470 0x7ff67406c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff678932700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff674157710 0x7ff6741e2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff678932700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67406af10 con 0x7ff6740b1bd0 2024-09-13T18:14:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff678932700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67406cc30 con 0x7ff674151470 2024-09-13T18:14:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff678932700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67406cf10 con 0x7ff674157710 2024-09-13T18:14:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff672d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6740b1bd0 0x7ff6741ece10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff672d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6740b1bd0 0x7ff6741ece10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:43686/0 (socket says 172.21.15.40:43686) 2024-09-13T18:14:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff672d9d700 1 -- 172.21.15.40:0/17603008 learned_addr learned my addr 172.21.15.40:0/17603008 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff67259c700 1 --2- 172.21.15.40:0/17603008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff674151470 0x7ff67406c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff67359e700 1 --2- 172.21.15.40:0/17603008 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff674157710 0x7ff6741e2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff67259c700 1 -- 172.21.15.40:0/17603008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff674151470 msgr2=0x7ff67406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:14:08.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff67259c700 1 -- 172.21.15.40:0/17603008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff674151470 msgr2=0x7ff67406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:14:08.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff67259c700 1 --2- 172.21.15.40:0/17603008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff674151470 0x7ff67406c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:14:08.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff67259c700 1 --2- 172.21.15.40:0/17603008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff674151470 0x7ff67406c610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:14:08.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff672d9d700 1 -- 172.21.15.40:0/17603008 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff674157710 msgr2=0x7ff6741e2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:08.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff672d9d700 1 --2- 172.21.15.40:0/17603008 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff674157710 0x7ff6741e2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:08.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff672d9d700 1 -- 172.21.15.40:0/17603008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff674151470 msgr2=0x7ff67406c610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:14:08.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff672d9d700 1 --2- 172.21.15.40:0/17603008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff674151470 0x7ff67406c610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:08.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff672d9d700 1 -- 172.21.15.40:0/17603008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff668009570 con 0x7ff6740b1bd0 2024-09-13T18:14:08.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff672d9d700 1 --2- 172.21.15.40:0/17603008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6740b1bd0 0x7ff6741ece10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff66400e870 tx=0x7ff66400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:08.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff65b7fe700 1 -- 172.21.15.40:0/17603008 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff664009540 con 0x7ff6740b1bd0 2024-09-13T18:14:08.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff65b7fe700 1 -- 172.21.15.40:0/17603008 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66400c640 con 0x7ff6740b1bd0 2024-09-13T18:14:08.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff678932700 1 -- 172.21.15.40:0/17603008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67406d1f0 con 0x7ff6740b1bd0 2024-09-13T18:14:08.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff65b7fe700 1 -- 172.21.15.40:0/17603008 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66400c9b0 con 0x7ff6740b1bd0 2024-09-13T18:14:08.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.522+0000 7ff678932700 1 -- 172.21.15.40:0/17603008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67406d7b0 con 0x7ff6740b1bd0 2024-09-13T18:14:08.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.526+0000 7ff678932700 1 -- 172.21.15.40:0/17603008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6741571d0 con 0x7ff6740b1bd0 2024-09-13T18:14:08.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.526+0000 7ff65b7fe700 1 -- 172.21.15.40:0/17603008 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff66400cb10 con 0x7ff6740b1bd0 2024-09-13T18:14:08.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.526+0000 7ff65b7fe700 1 --2- 172.21.15.40:0/17603008 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff660066730 0x7ff660068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:08.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.526+0000 7ff67259c700 1 --2- 172.21.15.40:0/17603008 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff660066730 0x7ff660068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:08.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.526+0000 7ff65b7fe700 1 -- 172.21.15.40:0/17603008 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff664084620 con 0x7ff6740b1bd0 2024-09-13T18:14:08.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.526+0000 7ff67259c700 1 --2- 172.21.15.40:0/17603008 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff660066730 0x7ff660068bf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff6740b2140 tx=0x7ff668008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:08.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.534+0000 7ff65b7fe700 1 -- 172.21.15.40:0/17603008 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff6640514a0 con 0x7ff6740b1bd0 2024-09-13T18:14:08.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.794+0000 7ff678932700 1 -- 172.21.15.40:0/17603008 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff67406d9a0 con 0x7ff6740b1bd0 2024-09-13T18:14:08.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.798+0000 7ff65b7fe700 1 -- 172.21.15.40:0/17603008 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7ff664017610 con 0x7ff6740b1bd0 2024-09-13T18:14:08.807 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":190,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:14:08.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.798+0000 7ff678932700 1 -- 172.21.15.40:0/17603008 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff660066730 msgr2=0x7ff660068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:08.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.798+0000 7ff678932700 1 --2- 172.21.15.40:0/17603008 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff660066730 0x7ff660068bf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff6740b2140 tx=0x7ff668008030 comp rx=0 tx=0).stop 2024-09-13T18:14:08.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.798+0000 7ff678932700 1 -- 172.21.15.40:0/17603008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6740b1bd0 msgr2=0x7ff6741ece10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:08.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.798+0000 7ff678932700 1 --2- 172.21.15.40:0/17603008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6740b1bd0 0x7ff6741ece10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff66400e870 tx=0x7ff66400ec30 comp rx=0 tx=0).stop 2024-09-13T18:14:08.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.798+0000 7ff678932700 1 -- 172.21.15.40:0/17603008 shutdown_connections 2024-09-13T18:14:08.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.798+0000 7ff678932700 1 --2- 172.21.15.40:0/17603008 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff660066730 0x7ff660068bf0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:08.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.798+0000 7ff678932700 1 --2- 172.21.15.40:0/17603008 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff674157710 0x7ff6741e2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:08.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.798+0000 7ff678932700 1 --2- 172.21.15.40:0/17603008 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff674151470 0x7ff67406c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:08.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.798+0000 7ff678932700 1 --2- 172.21.15.40:0/17603008 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6740b1bd0 0x7ff6741ece10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:08.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.798+0000 7ff678932700 1 -- 172.21.15.40:0/17603008 >> 172.21.15.40:0/17603008 conn(0x7ff67406e4f0 msgr2=0x7ff674155300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:08.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.798+0000 7ff678932700 1 -- 172.21.15.40:0/17603008 shutdown_connections 2024-09-13T18:14:08.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:08.798+0000 7ff678932700 1 -- 172.21.15.40:0/17603008 wait complete. 2024-09-13T18:14:11.823 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:14:11.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.930+0000 7fce349e4700 1 Processor -- start 2024-09-13T18:14:11.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.930+0000 7fce349e4700 1 -- start start 2024-09-13T18:14:11.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.930+0000 7fce349e4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fce30153840 0x7fce300c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:11.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.930+0000 7fce349e4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce300c9e20 0x7fce300cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:11.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.930+0000 7fce349e4700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce300cc060 con 0x7fce300cc4f0 2024-09-13T18:14:11.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.930+0000 7fce349e4700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce300b2840 con 0x7fce30153840 2024-09-13T18:14:11.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.930+0000 7fce349e4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce300ca870 con 0x7fce300c9e20 2024-09-13T18:14:11.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.930+0000 7fce2f7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fce30153840 0x7fce300c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:11.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.930+0000 7fce2f7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fce30153840 0x7fce300c3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38294/0 (socket says 172.21.15.40:38294) 2024-09-13T18:14:11.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.930+0000 7fce2f7fe700 1 -- 172.21.15.40:0/2974329520 learned_addr learned my addr 172.21.15.40:0/2974329520 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:11.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.930+0000 7fce2ffff700 1 --2- 172.21.15.40:0/2974329520 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce300c9e20 0x7fce300cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:11.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.930+0000 7fce2f7fe700 1 -- 172.21.15.40:0/2974329520 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce300c9e20 msgr2=0x7fce300cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:14:11.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.930+0000 7fce2f7fe700 1 --2- 172.21.15.40:0/2974329520 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce300c9e20 0x7fce300cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:11.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.930+0000 7fce2f7fe700 1 -- 172.21.15.40:0/2974329520 >> v1:172.21.15.43:6789/0 conn(0x7fce300cc4f0 legacy=0x7fce3015d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:11.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.930+0000 7fce2f7fe700 1 -- 172.21.15.40:0/2974329520 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce30153c20 con 0x7fce30153840 2024-09-13T18:14:11.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.930+0000 7fce2f7fe700 1 --2- 172.21.15.40:0/2974329520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce30153840 0x7fce300c3eb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fce20009910 tx=0x7fce20009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6d68027ddabea7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:11.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.930+0000 7fce2effd700 1 -- 172.21.15.40:0/2974329520 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20011830 con 0x7fce30153840 2024-09-13T18:14:11.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.930+0000 7fce2effd700 1 -- 172.21.15.40:0/2974329520 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce20011990 con 0x7fce30153840 2024-09-13T18:14:11.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.930+0000 7fce2effd700 1 -- 172.21.15.40:0/2974329520 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20011d40 con 0x7fce30153840 2024-09-13T18:14:11.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.934+0000 7fce349e4700 1 -- 172.21.15.40:0/2974329520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce30153840 msgr2=0x7fce300c3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:11.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.934+0000 7fce349e4700 1 --2- 172.21.15.40:0/2974329520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce30153840 0x7fce300c3eb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fce20009910 tx=0x7fce20009c20 comp rx=0 tx=0).stop 2024-09-13T18:14:11.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.934+0000 7fce349e4700 1 -- 172.21.15.40:0/2974329520 shutdown_connections 2024-09-13T18:14:11.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.934+0000 7fce349e4700 1 --2- 172.21.15.40:0/2974329520 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce300c9e20 0x7fce300cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:11.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.934+0000 7fce349e4700 1 --2- 172.21.15.40:0/2974329520 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce30153840 0x7fce300c3eb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:11.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.934+0000 7fce349e4700 1 -- 172.21.15.40:0/2974329520 >> 172.21.15.40:0/2974329520 conn(0x7fce3006c800 msgr2=0x7fce3006cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:11.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.934+0000 7fce349e4700 1 -- 172.21.15.40:0/2974329520 shutdown_connections 2024-09-13T18:14:11.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.934+0000 7fce349e4700 1 -- 172.21.15.40:0/2974329520 wait complete. 2024-09-13T18:14:11.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.934+0000 7fce349e4700 1 Processor -- start 2024-09-13T18:14:11.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.934+0000 7fce349e4700 1 -- start start 2024-09-13T18:14:11.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce349e4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce300c9e20 0x7fce301460a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:11.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce349e4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce300cc4f0 0x7fce30140780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:11.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce349e4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce30153840 0x7fce300b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:11.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce349e4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce301e4840 con 0x7fce300c9e20 2024-09-13T18:14:11.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce349e4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce301466e0 con 0x7fce300cc4f0 2024-09-13T18:14:11.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce349e4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce300ccd10 con 0x7fce30153840 2024-09-13T18:14:11.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce2f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce300c9e20 0x7fce301460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:11.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce2f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce300c9e20 0x7fce301460a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38310/0 (socket says 172.21.15.40:38310) 2024-09-13T18:14:11.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce2f7fe700 1 -- 172.21.15.40:0/2162478588 learned_addr learned my addr 172.21.15.40:0/2162478588 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:11.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce27fff700 1 --2- 172.21.15.40:0/2162478588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce300cc4f0 0x7fce30140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:11.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce2ffff700 1 --2- 172.21.15.40:0/2162478588 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce30153840 0x7fce300b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:11.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce2f7fe700 1 -- 172.21.15.40:0/2162478588 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce30153840 msgr2=0x7fce300b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:11.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce2f7fe700 1 --2- 172.21.15.40:0/2162478588 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce30153840 0x7fce300b4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:11.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce2f7fe700 1 -- 172.21.15.40:0/2162478588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce300cc4f0 msgr2=0x7fce30140780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:11.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce2f7fe700 1 --2- 172.21.15.40:0/2162478588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce300cc4f0 0x7fce30140780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:11.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce2f7fe700 1 -- 172.21.15.40:0/2162478588 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce20009570 con 0x7fce300c9e20 2024-09-13T18:14:11.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce2ffff700 1 --2- 172.21.15.40:0/2162478588 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce30153840 0x7fce300b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:14:11.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce2f7fe700 1 --2- 172.21.15.40:0/2162478588 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce300c9e20 0x7fce301460a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fce3008c6a0 tx=0x7fce20004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:11.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce2cff9700 1 -- 172.21.15.40:0/2162478588 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20004590 con 0x7fce300c9e20 2024-09-13T18:14:11.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce2cff9700 1 -- 172.21.15.40:0/2162478588 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2001e070 con 0x7fce300c9e20 2024-09-13T18:14:11.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce349e4700 1 -- 172.21.15.40:0/2162478588 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce300ccff0 con 0x7fce300c9e20 2024-09-13T18:14:11.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce2cff9700 1 -- 172.21.15.40:0/2162478588 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20003900 con 0x7fce300c9e20 2024-09-13T18:14:11.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce349e4700 1 -- 172.21.15.40:0/2162478588 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce300cd5b0 con 0x7fce300c9e20 2024-09-13T18:14:11.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.938+0000 7fce349e4700 1 -- 172.21.15.40:0/2162478588 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce301e28e0 con 0x7fce300c9e20 2024-09-13T18:14:11.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.942+0000 7fce2cff9700 1 -- 172.21.15.40:0/2162478588 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fce20003a60 con 0x7fce300c9e20 2024-09-13T18:14:11.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.942+0000 7fce2cff9700 1 --2- 172.21.15.40:0/2162478588 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fce10066760 0x7fce10068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:11.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.942+0000 7fce2cff9700 1 -- 172.21.15.40:0/2162478588 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fce20085450 con 0x7fce300c9e20 2024-09-13T18:14:11.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.942+0000 7fce27fff700 1 --2- 172.21.15.40:0/2162478588 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fce10066760 0x7fce10068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:11.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.942+0000 7fce27fff700 1 --2- 172.21.15.40:0/2162478588 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fce10066760 0x7fce10068c20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fce301e57a0 tx=0x7fce18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:11.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:11.946+0000 7fce2cff9700 1 -- 172.21.15.40:0/2162478588 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fce20052400 con 0x7fce300c9e20 2024-09-13T18:14:12.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:12.202+0000 7fce349e4700 1 -- 172.21.15.40:0/2162478588 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fce300cbcf0 con 0x7fce300c9e20 2024-09-13T18:14:12.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:12.202+0000 7fce2cff9700 1 -- 172.21.15.40:0/2162478588 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7fce20055a20 con 0x7fce300c9e20 2024-09-13T18:14:12.213 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":190,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:14:12.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:12.206+0000 7fce349e4700 1 -- 172.21.15.40:0/2162478588 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fce10066760 msgr2=0x7fce10068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:12.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:12.206+0000 7fce349e4700 1 --2- 172.21.15.40:0/2162478588 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fce10066760 0x7fce10068c20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fce301e57a0 tx=0x7fce18009040 comp rx=0 tx=0).stop 2024-09-13T18:14:12.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:12.206+0000 7fce349e4700 1 -- 172.21.15.40:0/2162478588 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce300c9e20 msgr2=0x7fce301460a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:12.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:12.206+0000 7fce349e4700 1 --2- 172.21.15.40:0/2162478588 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce300c9e20 0x7fce301460a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fce3008c6a0 tx=0x7fce20004770 comp rx=0 tx=0).stop 2024-09-13T18:14:12.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:12.206+0000 7fce349e4700 1 -- 172.21.15.40:0/2162478588 shutdown_connections 2024-09-13T18:14:12.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:12.206+0000 7fce349e4700 1 --2- 172.21.15.40:0/2162478588 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fce10066760 0x7fce10068c20 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:12.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:12.206+0000 7fce349e4700 1 --2- 172.21.15.40:0/2162478588 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce30153840 0x7fce300b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:12.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:12.206+0000 7fce349e4700 1 --2- 172.21.15.40:0/2162478588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce300cc4f0 0x7fce30140780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:12.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:12.206+0000 7fce349e4700 1 --2- 172.21.15.40:0/2162478588 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce300c9e20 0x7fce301460a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:12.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:12.206+0000 7fce349e4700 1 -- 172.21.15.40:0/2162478588 >> 172.21.15.40:0/2162478588 conn(0x7fce3006c800 msgr2=0x7fce3014e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:12.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:12.206+0000 7fce349e4700 1 -- 172.21.15.40:0/2162478588 shutdown_connections 2024-09-13T18:14:12.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:12.206+0000 7fce349e4700 1 -- 172.21.15.40:0/2162478588 wait complete. 2024-09-13T18:14:15.230 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:14:15.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.350+0000 7f46d988f700 1 Processor -- start 2024-09-13T18:14:15.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.350+0000 7f46d988f700 1 -- start start 2024-09-13T18:14:15.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.350+0000 7f46d988f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f46d406e5d0 0x7f46d4153120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.350+0000 7f46d988f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d40c7bb0 con 0x7f46d415cbc0 2024-09-13T18:14:15.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.350+0000 7f46d988f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d4153a60 con 0x7f46d4153660 2024-09-13T18:14:15.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.350+0000 7f46d988f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d406b4c0 con 0x7f46d406e5d0 2024-09-13T18:14:15.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.350+0000 7f46d3fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f46d406e5d0 0x7f46d4153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.354+0000 7f46d3fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f46d406e5d0 0x7f46d4153120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38322/0 (socket says 172.21.15.40:38322) 2024-09-13T18:14:15.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.354+0000 7f46d3fff700 1 -- 172.21.15.40:0/4276072707 learned_addr learned my addr 172.21.15.40:0/4276072707 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:15.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.354+0000 7f46d3fff700 1 -- 172.21.15.40:0/4276072707 >> v1:172.21.15.43:6789/0 conn(0x7f46d4153660 legacy=0x7f46d406f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:14:15.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.354+0000 7f46d3fff700 1 -- 172.21.15.40:0/4276072707 >> v1:172.21.15.40:6789/0 conn(0x7f46d415cbc0 legacy=0x7f46d415cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:14:15.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.354+0000 7f46d3fff700 1 -- 172.21.15.40:0/4276072707 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d406b7a0 con 0x7f46d406e5d0 2024-09-13T18:14:15.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.354+0000 7f46d3fff700 1 --2- 172.21.15.40:0/4276072707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46d406e5d0 0x7f46d4153120 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009910 tx=0x7f46c0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4924ef9b0ce2a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:15.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.354+0000 7f46d37fe700 1 -- 172.21.15.40:0/4276072707 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0004230 con 0x7f46d406e5d0 2024-09-13T18:14:15.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.354+0000 7f46d37fe700 1 -- 172.21.15.40:0/4276072707 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c0004390 con 0x7f46d406e5d0 2024-09-13T18:14:15.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.354+0000 7f46d37fe700 1 -- 172.21.15.40:0/4276072707 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c0009e60 con 0x7f46d406e5d0 2024-09-13T18:14:15.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.354+0000 7f46d988f700 1 -- 172.21.15.40:0/4276072707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46d406e5d0 msgr2=0x7f46d4153120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:15.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.354+0000 7f46d988f700 1 --2- 172.21.15.40:0/4276072707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46d406e5d0 0x7f46d4153120 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009910 tx=0x7f46c0009c20 comp rx=0 tx=0).stop 2024-09-13T18:14:15.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.354+0000 7f46d988f700 1 -- 172.21.15.40:0/4276072707 shutdown_connections 2024-09-13T18:14:15.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.354+0000 7f46d988f700 1 --2- 172.21.15.40:0/4276072707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46d406e5d0 0x7f46d4153120 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.354+0000 7f46d988f700 1 -- 172.21.15.40:0/4276072707 >> 172.21.15.40:0/4276072707 conn(0x7f46d40ccef0 msgr2=0x7f46d40cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:15.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.354+0000 7f46d988f700 1 -- 172.21.15.40:0/4276072707 shutdown_connections 2024-09-13T18:14:15.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.354+0000 7f46d988f700 1 -- 172.21.15.40:0/4276072707 wait complete. 2024-09-13T18:14:15.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.354+0000 7f46d988f700 1 Processor -- start 2024-09-13T18:14:15.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.354+0000 7f46d988f700 1 -- start start 2024-09-13T18:14:15.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d988f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46d406e5d0 0x7f46d40cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d988f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46d4153660 0x7f46d41ef7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d988f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46d415cbc0 0x7f46d41efce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d988f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d41ef310 con 0x7f46d406e5d0 2024-09-13T18:14:15.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d988f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d41f26e0 con 0x7f46d415cbc0 2024-09-13T18:14:15.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d988f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d41f29b0 con 0x7f46d4153660 2024-09-13T18:14:15.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d888d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46d406e5d0 0x7f46d40cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d888d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46d406e5d0 0x7f46d40cbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38326/0 (socket says 172.21.15.40:38326) 2024-09-13T18:14:15.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d888d700 1 -- 172.21.15.40:0/2042036117 learned_addr learned my addr 172.21.15.40:0/2042036117 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d3fff700 1 --2- 172.21.15.40:0/2042036117 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46d4153660 0x7f46d41ef7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d908e700 1 --2- 172.21.15.40:0/2042036117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46d415cbc0 0x7f46d41efce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d888d700 1 -- 172.21.15.40:0/2042036117 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46d4153660 msgr2=0x7f46d41ef7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d888d700 1 --2- 172.21.15.40:0/2042036117 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46d4153660 0x7f46d41ef7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d888d700 1 -- 172.21.15.40:0/2042036117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46d415cbc0 msgr2=0x7f46d41efce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d888d700 1 --2- 172.21.15.40:0/2042036117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46d415cbc0 0x7f46d41efce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d888d700 1 -- 172.21.15.40:0/2042036117 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c0009570 con 0x7f46d406e5d0 2024-09-13T18:14:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d908e700 1 --2- 172.21.15.40:0/2042036117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46d415cbc0 0x7f46d41efce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:14:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d888d700 1 --2- 172.21.15.40:0/2042036117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46d406e5d0 0x7f46d40cbfb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f46c800c500 tx=0x7f46c800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d17fa700 1 -- 172.21.15.40:0/2042036117 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c80052d0 con 0x7f46d406e5d0 2024-09-13T18:14:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d17fa700 1 -- 172.21.15.40:0/2042036117 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c8005430 con 0x7f46d406e5d0 2024-09-13T18:14:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d988f700 1 -- 172.21.15.40:0/2042036117 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d41f2c90 con 0x7f46d406e5d0 2024-09-13T18:14:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d17fa700 1 -- 172.21.15.40:0/2042036117 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c80055a0 con 0x7f46d406e5d0 2024-09-13T18:14:15.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d988f700 1 -- 172.21.15.40:0/2042036117 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d41f3250 con 0x7f46d406e5d0 2024-09-13T18:14:15.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d988f700 1 -- 172.21.15.40:0/2042036117 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d40c7450 con 0x7f46d406e5d0 2024-09-13T18:14:15.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.358+0000 7f46d17fa700 1 -- 172.21.15.40:0/2042036117 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f46c80036a0 con 0x7f46d406e5d0 2024-09-13T18:14:15.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.362+0000 7f46d17fa700 1 --2- 172.21.15.40:0/2042036117 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f46b8066760 0x7f46b8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.362+0000 7f46d3fff700 1 --2- 172.21.15.40:0/2042036117 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f46b8066760 0x7f46b8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.362+0000 7f46d17fa700 1 -- 172.21.15.40:0/2042036117 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f46c8083920 con 0x7f46d406e5d0 2024-09-13T18:14:15.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.362+0000 7f46d3fff700 1 --2- 172.21.15.40:0/2042036117 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f46b8066760 0x7f46b8068c20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009910 tx=0x7f46c0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:15.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.366+0000 7f46d17fa700 1 -- 172.21.15.40:0/2042036117 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f46c8050830 con 0x7f46d406e5d0 2024-09-13T18:14:15.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.610+0000 7f46d988f700 1 -- 172.21.15.40:0/2042036117 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f46d406d180 con 0x7f46d406e5d0 2024-09-13T18:14:15.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.610+0000 7f46d17fa700 1 -- 172.21.15.40:0/2042036117 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f46c8053e50 con 0x7f46d406e5d0 2024-09-13T18:14:15.621 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":192,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:14:15.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.614+0000 7f46d988f700 1 -- 172.21.15.40:0/2042036117 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f46b8066760 msgr2=0x7f46b8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:15.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.614+0000 7f46d988f700 1 --2- 172.21.15.40:0/2042036117 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f46b8066760 0x7f46b8068c20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009910 tx=0x7f46c0008030 comp rx=0 tx=0).stop 2024-09-13T18:14:15.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.614+0000 7f46d988f700 1 -- 172.21.15.40:0/2042036117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46d406e5d0 msgr2=0x7f46d40cbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:15.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.614+0000 7f46d988f700 1 --2- 172.21.15.40:0/2042036117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46d406e5d0 0x7f46d40cbfb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f46c800c500 tx=0x7f46c800c8c0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.614+0000 7f46d988f700 1 -- 172.21.15.40:0/2042036117 shutdown_connections 2024-09-13T18:14:15.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.614+0000 7f46d988f700 1 --2- 172.21.15.40:0/2042036117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46d415cbc0 0x7f46d41efce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.614+0000 7f46d988f700 1 --2- 172.21.15.40:0/2042036117 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f46b8066760 0x7f46b8068c20 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.614+0000 7f46d988f700 1 --2- 172.21.15.40:0/2042036117 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46d4153660 0x7f46d41ef7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.614+0000 7f46d988f700 1 --2- 172.21.15.40:0/2042036117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46d406e5d0 0x7f46d40cbfb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.614+0000 7f46d988f700 1 -- 172.21.15.40:0/2042036117 >> 172.21.15.40:0/2042036117 conn(0x7f46d40ccef0 msgr2=0x7f46d414cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:15.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.614+0000 7f46d988f700 1 -- 172.21.15.40:0/2042036117 shutdown_connections 2024-09-13T18:14:15.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.614+0000 7f46d988f700 1 -- 172.21.15.40:0/2042036117 wait complete. 2024-09-13T18:14:15.637 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:14:15.637 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:14:15.637 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:14:15.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.746+0000 7fcabf59e700 1 Processor -- start 2024-09-13T18:14:15.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.746+0000 7fcabf59e700 1 -- start start 2024-09-13T18:14:15.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabf59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcab80d66d0 0x7fcab804a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabf59e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcab8045470 0x7fcab8035810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabf59e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab80dd390 con 0x7fcab804a010 2024-09-13T18:14:15.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabf59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab80065c0 con 0x7fcab80d66d0 2024-09-13T18:14:15.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabf59e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab804af10 con 0x7fcab8045470 2024-09-13T18:14:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabe59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcab80d66d0 0x7fcab804a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabe59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcab80d66d0 0x7fcab804a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38332/0 (socket says 172.21.15.40:38332) 2024-09-13T18:14:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabe59c700 1 -- 172.21.15.40:0/3263196090 learned_addr learned my addr 172.21.15.40:0/3263196090 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabed9d700 1 --2- 172.21.15.40:0/3263196090 >> v2:172.21.15.43:3300/0 conn(0x7fcab8045470 0x7fcab8035810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabe59c700 1 -- 172.21.15.40:0/3263196090 >> v2:172.21.15.43:3300/0 conn(0x7fcab8045470 msgr2=0x7fcab8035810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:14:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabe59c700 1 --2- 172.21.15.40:0/3263196090 >> v2:172.21.15.43:3300/0 conn(0x7fcab8045470 0x7fcab8035810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabe59c700 1 -- 172.21.15.40:0/3263196090 >> v1:172.21.15.43:6789/0 conn(0x7fcab804a010 legacy=0x7fcab8005eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:14:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabe59c700 1 -- 172.21.15.40:0/3263196090 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab80dd0b0 con 0x7fcab80d66d0 2024-09-13T18:14:15.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabe59c700 1 --2- 172.21.15.40:0/3263196090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab80d66d0 0x7fcab804a660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009910 tx=0x7fcab0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15710a120ad476d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:15.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabd59a700 1 -- 172.21.15.40:0/3263196090 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0004230 con 0x7fcab80d66d0 2024-09-13T18:14:15.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabd59a700 1 -- 172.21.15.40:0/3263196090 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab0004390 con 0x7fcab80d66d0 2024-09-13T18:14:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabd59a700 1 -- 172.21.15.40:0/3263196090 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0009e60 con 0x7fcab80d66d0 2024-09-13T18:14:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabf59e700 1 -- 172.21.15.40:0/3263196090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab80d66d0 msgr2=0x7fcab804a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabf59e700 1 --2- 172.21.15.40:0/3263196090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab80d66d0 0x7fcab804a660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009910 tx=0x7fcab0009c20 comp rx=0 tx=0).stop 2024-09-13T18:14:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabf59e700 1 -- 172.21.15.40:0/3263196090 shutdown_connections 2024-09-13T18:14:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabf59e700 1 --2- 172.21.15.40:0/3263196090 >> v2:172.21.15.43:3300/0 conn(0x7fcab8045470 0x7fcab8035810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabf59e700 1 --2- 172.21.15.40:0/3263196090 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab80d66d0 0x7fcab804a660 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabf59e700 1 -- 172.21.15.40:0/3263196090 >> 172.21.15.40:0/3263196090 conn(0x7fcab8050590 msgr2=0x7fcab80509a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabf59e700 1 -- 172.21.15.40:0/3263196090 shutdown_connections 2024-09-13T18:14:15.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.750+0000 7fcabf59e700 1 -- 172.21.15.40:0/3263196090 wait complete. 2024-09-13T18:14:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabf59e700 1 Processor -- start 2024-09-13T18:14:15.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabf59e700 1 -- start start 2024-09-13T18:14:15.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabf59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab8045470 0x7fcab8048b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabf59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab804a010 0x7fcab8049090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabf59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcab80d66d0 0x7fcab8167060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabf59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab81658a0 con 0x7fcab804a010 2024-09-13T18:14:15.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabf59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab80ccbd0 con 0x7fcab8045470 2024-09-13T18:14:15.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabf59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab8165af0 con 0x7fcab80d66d0 2024-09-13T18:14:15.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabdd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab804a010 0x7fcab8049090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabdd9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab804a010 0x7fcab8049090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38348/0 (socket says 172.21.15.40:38348) 2024-09-13T18:14:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabdd9b700 1 -- 172.21.15.40:0/1461975499 learned_addr learned my addr 172.21.15.40:0/1461975499 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabe59c700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab8045470 0x7fcab8048b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabed9d700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcab80d66d0 0x7fcab8167060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabe59c700 1 -- 172.21.15.40:0/1461975499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab8045470 msgr2=0x7fcab8048b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:14:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabe59c700 1 -- 172.21.15.40:0/1461975499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab8045470 msgr2=0x7fcab8048b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:14:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabe59c700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab8045470 0x7fcab8048b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:14:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabe59c700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab8045470 0x7fcab8048b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:14:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabdd9b700 1 -- 172.21.15.40:0/1461975499 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcab80d66d0 msgr2=0x7fcab8167060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabdd9b700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcab80d66d0 0x7fcab8167060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabdd9b700 1 -- 172.21.15.40:0/1461975499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab8045470 msgr2=0x7fcab8048b50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:14:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabdd9b700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab8045470 0x7fcab8048b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabdd9b700 1 -- 172.21.15.40:0/1461975499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab0009570 con 0x7fcab804a010 2024-09-13T18:14:15.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabdd9b700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab804a010 0x7fcab8049090 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcab400c550 tx=0x7fcab400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:15.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcaaeffd700 1 -- 172.21.15.40:0/1461975499 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4011790 con 0x7fcab804a010 2024-09-13T18:14:15.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcaaeffd700 1 -- 172.21.15.40:0/1461975499 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab40118f0 con 0x7fcab804a010 2024-09-13T18:14:15.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcabf59e700 1 -- 172.21.15.40:0/1461975499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab8166860 con 0x7fcab804a010 2024-09-13T18:14:15.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.754+0000 7fcaaeffd700 1 -- 172.21.15.40:0/1461975499 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4011ca0 con 0x7fcab804a010 2024-09-13T18:14:15.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.758+0000 7fcabf59e700 1 -- 172.21.15.40:0/1461975499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab80c9d30 con 0x7fcab804a010 2024-09-13T18:14:15.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.758+0000 7fcabf59e700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab80cc310 0x7fcab81675a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.758+0000 7fcabf59e700 1 -- 172.21.15.40:0/1461975499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab8008d40 con 0x7fcab80cc310 2024-09-13T18:14:15.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.758+0000 7fcabe59c700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab80cc310 0x7fcab81675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.758+0000 7fcaaeffd700 1 -- 172.21.15.40:0/1461975499 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcab401c460 con 0x7fcab804a010 2024-09-13T18:14:15.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.758+0000 7fcabe59c700 1 -- 172.21.15.40:0/1461975499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcab804cc00 con 0x7fcab80cc310 2024-09-13T18:14:15.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.758+0000 7fcabe59c700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab80cc310 0x7fcab81675a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcab0004700 tx=0x7fcab0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:15.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.758+0000 7fcaaeffd700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcaa0066730 0x7fcaa0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.758+0000 7fcabed9d700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcaa0066730 0x7fcaa0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.758+0000 7fcaaeffd700 1 -- 172.21.15.40:0/1461975499 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcab4084310 con 0x7fcab804a010 2024-09-13T18:14:15.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.758+0000 7fcaaeffd700 1 -- 172.21.15.40:0/1461975499 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab00038c0 con 0x7fcab80cc310 2024-09-13T18:14:15.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.758+0000 7fcaaeffd700 1 -- 172.21.15.40:0/1461975499 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fcab000be50 con 0x7fcab80cc310 2024-09-13T18:14:15.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.758+0000 7fcaaeffd700 1 -- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab80cc310 msgr2=0x7fcab81675a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:15.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.758+0000 7fcaaeffd700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab80cc310 0x7fcab81675a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcab0004700 tx=0x7fcab0004770 comp rx=0 tx=0).stop 2024-09-13T18:14:15.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.758+0000 7fcabed9d700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcaa0066730 0x7fcaa0068bf0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fcac00641b0 tx=0x7fcac008e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:15.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.758+0000 7fcaaeffd700 1 -- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab80cc310 msgr2=0x7fcab81675a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:14:15.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.758+0000 7fcaaeffd700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab80cc310 0x7fcab81675a0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcab0004700 tx=0x7fcab0004770 comp rx=0 tx=0).stop 2024-09-13T18:14:15.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.766+0000 7fcabf59e700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab80c8740 0x7fcab80c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.766+0000 7fcabf59e700 1 -- 172.21.15.40:0/1461975499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab80d6ab0 con 0x7fcab80c8740 2024-09-13T18:14:15.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.766+0000 7fcabe59c700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab80c8740 0x7fcab80c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.766+0000 7fcabe59c700 1 -- 172.21.15.40:0/1461975499 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcab8007f20 con 0x7fcab80c8740 2024-09-13T18:14:15.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.766+0000 7fcabe59c700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab80c8740 0x7fcab80c8b50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcab0000c00 tx=0x7fcab0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:15.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.770+0000 7fcaaeffd700 1 -- 172.21.15.40:0/1461975499 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0029040 con 0x7fcab80c8740 2024-09-13T18:14:15.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.770+0000 7fcaaeffd700 1 -- 172.21.15.40:0/1461975499 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4784 (secure 0 0 0) 0x7fcab002ab00 con 0x7fcab80c8740 2024-09-13T18:14:15.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.770+0000 7fcaaeffd700 1 -- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab80c8740 msgr2=0x7fcab80c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:15.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.770+0000 7fcaaeffd700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab80c8740 0x7fcab80c8b50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcab0000c00 tx=0x7fcab0011000 comp rx=0 tx=0).stop 2024-09-13T18:14:15.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.770+0000 7fcaaeffd700 1 -- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab80c8740 msgr2=0x7fcab80c8b50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:14:15.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.770+0000 7fcaaeffd700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab80c8740 0x7fcab80c8b50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.781 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:14:15.781 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:14:15.781 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:14:15.781 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:14:15.781 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 192, 2024-09-13T18:14:15.781 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:14:15.782 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:14:15.782 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:14:15.782 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:14:15.782 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:15.782 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:14:15.782 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:14:15.782 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:14:15.782 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:14:15.782 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:15.782 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:15.782 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:15.782 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:15.782 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:15.782 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:15.783 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:15.783 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:15.783 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:15.783 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:15.783 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:15.783 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:14:15.783 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:14:15.783 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:15.783 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:15.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:15.783 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:15.783 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:15.783 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:15.784 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:15.784 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:15.784 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:15.784 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:15.784 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:15.784 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:15.784 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:14:15.784 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:14:15.784 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:14:15.784 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:14:15.784 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:14:15.784 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:14:15.784 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:14:15.784 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:14:15.785 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:14:15.785 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:14:15.785 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:14:15.785 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:14:15.785 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:14:15.785 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:14:15.785 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:14:15.785 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:14:15.785 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:14:15.785 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:15.785 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:15.785 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:15.785 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:15.785 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:15.786 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:15.786 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:15.786 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:15.786 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:15.786 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:15.786 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:15.786 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:14:15.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:15.786 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:14:15.786 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:15.786 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:14:15.786 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:14:15.786 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:15.786 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:15.786 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:15.787 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:15.787 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:14:15.787 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:15.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:15.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:15.787 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:15.787 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:14:15.787 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:15.787 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:15.787 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:15.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:15.787 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:14:15.787 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:14:15.788 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:15.788 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:15.788 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:15.788 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:15.788 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:15.788 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:14:15.788 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:14:15.788 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:15.788 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:15.788 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:15.788 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:15.788 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:14:15.788 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:15.788 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:15.788 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:15.789 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:15.789 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:14:15.789 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:15.789 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:15.789 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:15.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:15.789 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:14:15.789 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:14:15.789 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:15.789 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:15.789 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:15.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:15.789 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:15.789 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:14:15.790 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:14:15.790 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:15.790 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:15.790 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:15.790 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:15.790 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:14:15.790 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:15.790 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:15.790 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:15.790 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:15.790 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:14:15.790 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:15.790 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:15.791 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:15.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:15.791 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:14:15.791 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:14:15.791 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:15.791 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:15.791 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:15.791 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:15.791 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:15.791 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:15.791 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:14:15.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:14:15.791 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:15.791 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:15.792 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:15.792 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:14:15.792 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:15.792 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:15.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:14:15.792 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:15.792 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:15.792 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:15.792 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:14:15.792 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:15.792 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:15.792 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:14:15.792 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:15.793 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:15.793 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:15.793 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:14:15.793 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:15.793 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:15.793 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:14:15.793 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:15.793 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:15.793 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:15.793 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:14:15.793 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:15.793 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:15.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:15.793 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:14:15.793 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:14:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.770+0000 7fcabf59e700 1 -- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcaa0066730 msgr2=0x7fcaa0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.770+0000 7fcabf59e700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcaa0066730 0x7fcaa0068bf0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fcac00641b0 tx=0x7fcac008e040 comp rx=0 tx=0).stop 2024-09-13T18:14:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.770+0000 7fcabf59e700 1 -- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab804a010 msgr2=0x7fcab8049090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.770+0000 7fcabf59e700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab804a010 0x7fcab8049090 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcab400c550 tx=0x7fcab400c860 comp rx=0 tx=0).stop 2024-09-13T18:14:15.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.770+0000 7fcabed9d700 1 -- 172.21.15.40:0/1461975499 reap_dead start 2024-09-13T18:14:15.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.770+0000 7fcabf59e700 1 -- 172.21.15.40:0/1461975499 shutdown_connections 2024-09-13T18:14:15.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.770+0000 7fcabf59e700 1 --2- 172.21.15.40:0/1461975499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab804a010 0x7fcab8049090 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.770+0000 7fcabf59e700 1 -- 172.21.15.40:0/1461975499 >> 172.21.15.40:0/1461975499 conn(0x7fcab8050590 msgr2=0x7fcab80cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:15.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.770+0000 7fcabf59e700 1 -- 172.21.15.40:0/1461975499 shutdown_connections 2024-09-13T18:14:15.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.770+0000 7fcabf59e700 1 -- 172.21.15.40:0/1461975499 wait complete. 2024-09-13T18:14:15.796 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:14:15.796 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:14:15.796 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:14:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.890+0000 7f7119c48700 1 Processor -- start 2024-09-13T18:14:15.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.890+0000 7f7119c48700 1 -- start start 2024-09-13T18:14:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7119c48700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7114153af0 0x7f71140c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7119c48700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71140cb420 0x7f71140cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7119c48700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71140cf4d0 0x7f711415d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7119c48700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f711415cc30 con 0x7f71140cb420 2024-09-13T18:14:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7119c48700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71140ca9b0 con 0x7f7114153af0 2024-09-13T18:14:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7119c48700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7114159930 con 0x7f71140cf4d0 2024-09-13T18:14:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7112ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71140cb420 0x7f71140cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7112ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f71140cb420 0x7f71140cef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38376/0 (socket says 172.21.15.40:38376) 2024-09-13T18:14:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7112ffd700 1 -- 172.21.15.40:0/3011479434 learned_addr learned my addr 172.21.15.40:0/3011479434 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f71137fe700 1 --2- 172.21.15.40:0/3011479434 >> v2:172.21.15.43:3300/0 conn(0x7f7114153af0 0x7f71140c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7113fff700 1 --2- 172.21.15.40:0/3011479434 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71140cf4d0 0x7f711415d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f71137fe700 1 -- 172.21.15.40:0/3011479434 >> v2:172.21.15.43:3300/0 conn(0x7f7114153af0 msgr2=0x7f71140c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:14:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f71137fe700 1 -- 172.21.15.40:0/3011479434 >> v2:172.21.15.43:3300/0 conn(0x7f7114153af0 msgr2=0x7f71140c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:14:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f71137fe700 1 --2- 172.21.15.40:0/3011479434 >> v2:172.21.15.43:3300/0 conn(0x7f7114153af0 0x7f71140c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:14:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f71137fe700 1 --2- 172.21.15.40:0/3011479434 >> v2:172.21.15.43:3300/0 conn(0x7f7114153af0 0x7f71140c9fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:14:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7112ffd700 1 -- 172.21.15.40:0/3011479434 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71140cf4d0 msgr2=0x7f711415d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:14:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7112ffd700 1 --2- 172.21.15.40:0/3011479434 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71140cf4d0 0x7f711415d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7112ffd700 1 -- 172.21.15.40:0/3011479434 >> v2:172.21.15.43:3300/0 conn(0x7f7114153af0 msgr2=0x7f71140c9fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:14:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7112ffd700 1 --2- 172.21.15.40:0/3011479434 >> v2:172.21.15.43:3300/0 conn(0x7f7114153af0 0x7f71140c9fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7112ffd700 1 -- 172.21.15.40:0/3011479434 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7114159c10 con 0x7f71140cb420 2024-09-13T18:14:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7112ffd700 1 --2- 172.21.15.40:0/3011479434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71140cb420 0x7f71140cef90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f711408c630 tx=0x7f7104009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4c4eaa0e50bffd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f71127fc700 1 -- 172.21.15.40:0/3011479434 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104011830 con 0x7f71140cb420 2024-09-13T18:14:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f71127fc700 1 -- 172.21.15.40:0/3011479434 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7104011990 con 0x7f71140cb420 2024-09-13T18:14:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f71127fc700 1 -- 172.21.15.40:0/3011479434 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104011d20 con 0x7f71140cb420 2024-09-13T18:14:15.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7119c48700 1 -- 172.21.15.40:0/3011479434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71140cb420 msgr2=0x7f71140cef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:15.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7119c48700 1 --2- 172.21.15.40:0/3011479434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71140cb420 0x7f71140cef90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f711408c630 tx=0x7f7104009bf0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7119c48700 1 -- 172.21.15.40:0/3011479434 shutdown_connections 2024-09-13T18:14:15.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7119c48700 1 --2- 172.21.15.40:0/3011479434 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71140cf4d0 0x7f711415d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7119c48700 1 --2- 172.21.15.40:0/3011479434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71140cb420 0x7f71140cef90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7119c48700 1 --2- 172.21.15.40:0/3011479434 >> v2:172.21.15.43:3300/0 conn(0x7f7114153af0 0x7f71140c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7119c48700 1 -- 172.21.15.40:0/3011479434 >> 172.21.15.40:0/3011479434 conn(0x7f7114066090 msgr2=0x7f71140664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:15.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7119c48700 1 -- 172.21.15.40:0/3011479434 shutdown_connections 2024-09-13T18:14:15.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.894+0000 7f7119c48700 1 -- 172.21.15.40:0/3011479434 wait complete. 2024-09-13T18:14:15.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7119c48700 1 Processor -- start 2024-09-13T18:14:15.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7119c48700 1 -- start start 2024-09-13T18:14:15.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7119c48700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71140cb420 0x7f7114153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7119c48700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71140cf4d0 0x7f7114154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7119c48700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7114153af0 0x7f7114155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7119c48700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71140cedd0 con 0x7f7114153af0 2024-09-13T18:14:15.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7119c48700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711415c870 con 0x7f71140cb420 2024-09-13T18:14:15.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7119c48700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711415c240 con 0x7f71140cf4d0 2024-09-13T18:14:15.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7113fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7114153af0 0x7f7114155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7113fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7114153af0 0x7f7114155070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38382/0 (socket says 172.21.15.40:38382) 2024-09-13T18:14:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7113fff700 1 -- 172.21.15.40:0/1550588526 learned_addr learned my addr 172.21.15.40:0/1550588526 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f71137fe700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71140cb420 0x7f7114153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7112ffd700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71140cf4d0 0x7f7114154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f71137fe700 1 -- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71140cb420 msgr2=0x7f7114153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:14:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f71137fe700 1 -- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71140cb420 msgr2=0x7f7114153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:14:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f71137fe700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71140cb420 0x7f7114153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:14:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f71137fe700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71140cb420 0x7f7114153f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:14:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7113fff700 1 -- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71140cf4d0 msgr2=0x7f7114154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7113fff700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71140cf4d0 0x7f7114154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7113fff700 1 -- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71140cb420 msgr2=0x7f7114153f60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:14:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7113fff700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71140cb420 0x7f7114153f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7113fff700 1 -- 172.21.15.40:0/1550588526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7104009570 con 0x7f7114153af0 2024-09-13T18:14:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7112ffd700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f71140cf4d0 0x7f7114154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:14:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7113fff700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7114153af0 0x7f7114155070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f710c00eed0 tx=0x7f710c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:15.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f710bfff700 1 -- 172.21.15.40:0/1550588526 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c010040 con 0x7f7114153af0 2024-09-13T18:14:15.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f710bfff700 1 -- 172.21.15.40:0/1550588526 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710c00bcd0 con 0x7f7114153af0 2024-09-13T18:14:15.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7119c48700 1 -- 172.21.15.40:0/1550588526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71140ceaf0 con 0x7f7114153af0 2024-09-13T18:14:15.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f710bfff700 1 -- 172.21.15.40:0/1550588526 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c009ea0 con 0x7f7114153af0 2024-09-13T18:14:15.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.898+0000 7f7119c48700 1 -- 172.21.15.40:0/1550588526 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711415c490 con 0x7f7114153af0 2024-09-13T18:14:15.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.902+0000 7f7119c48700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 0x7f70f8004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.902+0000 7f7119c48700 1 -- 172.21.15.40:0/1550588526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f8007f40 con 0x7f70f8002a40 2024-09-13T18:14:15.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.902+0000 7f71137fe700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 0x7f70f8004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.902+0000 7f71137fe700 1 -- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 msgr2=0x7f70f8004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-09-13T18:14:15.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.902+0000 7f71137fe700 1 -- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 msgr2=0x7f70f8004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:14:15.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.902+0000 7f71137fe700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 0x7f70f8004f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:14:15.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.902+0000 7f71137fe700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 0x7f70f8004f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:14:15.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.902+0000 7f710bfff700 1 -- 172.21.15.40:0/1550588526 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f710c014640 con 0x7f7114153af0 2024-09-13T18:14:15.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.902+0000 7f710bfff700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70f4066730 0x7f70f4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:15.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.902+0000 7f710bfff700 1 -- 172.21.15.40:0/1550588526 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f710c084b80 con 0x7f7114153af0 2024-09-13T18:14:15.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.902+0000 7f7112ffd700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70f4066730 0x7f70f4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:15.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:15.902+0000 7f7112ffd700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70f4066730 0x7f70f4068bf0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7104000c00 tx=0x7f7104008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:16.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:16.106+0000 7f71137fe700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 0x7f70f8004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:16.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:16.106+0000 7f71137fe700 1 -- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 msgr2=0x7f70f8004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-13T18:14:16.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:16.106+0000 7f71137fe700 1 -- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 msgr2=0x7f70f8004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:14:16.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:16.106+0000 7f71137fe700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 0x7f70f8004f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:14:16.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:16.106+0000 7f71137fe700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 0x7f70f8004f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:14:16.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:16.510+0000 7f71137fe700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 0x7f70f8004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:16.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:16.510+0000 7f71137fe700 1 -- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 msgr2=0x7f70f8004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-13T18:14:16.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:16.510+0000 7f71137fe700 1 -- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 msgr2=0x7f70f8004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:14:16.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:16.510+0000 7f71137fe700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 0x7f70f8004f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:14:16.521 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:16.510+0000 7f71137fe700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 0x7f70f8004f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:14:17.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:17.314+0000 7f71137fe700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 0x7f70f8004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:17.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:17.314+0000 7f71137fe700 1 -- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 msgr2=0x7f70f8004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-13T18:14:17.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:17.314+0000 7f71137fe700 1 -- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 msgr2=0x7f70f8004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:14:17.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:17.314+0000 7f71137fe700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 0x7f70f8004f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:14:17.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:17.314+0000 7f71137fe700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 0x7f70f8004f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:14:18.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.918+0000 7f71137fe700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 0x7f70f8004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:18.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.918+0000 7f71137fe700 1 -- 172.21.15.40:0/1550588526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f70f8008100 con 0x7f70f8002a40 2024-09-13T18:14:18.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.918+0000 7f71137fe700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 0x7f70f8004f00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f70fc018990 tx=0x7f70fc018d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:18.928 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.918+0000 7f710bfff700 1 -- 172.21.15.40:0/1550588526 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc014bd0 con 0x7f70f8002a40 2024-09-13T18:14:18.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.918+0000 7f710bfff700 1 -- 172.21.15.40:0/1550588526 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f70fc00c040 con 0x7f70f8002a40 2024-09-13T18:14:18.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.918+0000 7f710bfff700 1 -- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 msgr2=0x7f70f8004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:18.929 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.918+0000 7f710bfff700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 0x7f70f8004f00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f70fc018990 tx=0x7f70fc018d50 comp rx=0 tx=0).stop 2024-09-13T18:14:18.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.918+0000 7f710bfff700 1 -- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 msgr2=0x7f70f8004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:14:18.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.918+0000 7f710bfff700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8002a40 0x7f70f8004f00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:18.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f7119c48700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8007020 0x7f70f8007470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:18.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f7119c48700 1 -- 172.21.15.40:0/1550588526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f800a400 con 0x7f70f8007020 2024-09-13T18:14:18.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f71137fe700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8007020 0x7f70f8007470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:18.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f71137fe700 1 -- 172.21.15.40:0/1550588526 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f70f800a5c0 con 0x7f70f8007020 2024-09-13T18:14:18.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f71137fe700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8007020 0x7f70f8007470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f70fc006b40 tx=0x7f70fc004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:18.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f710bfff700 1 -- 172.21.15.40:0/1550588526 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc004250 con 0x7f70f8007020 2024-09-13T18:14:18.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f710bfff700 1 -- 172.21.15.40:0/1550588526 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4450 (secure 0 0 0) 0x7f70fc0043b0 con 0x7f70f8007020 2024-09-13T18:14:18.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f710bfff700 1 -- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8007020 msgr2=0x7f70f8007470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:18.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f710bfff700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8007020 0x7f70f8007470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f70fc006b40 tx=0x7f70fc004770 comp rx=0 tx=0).stop 2024-09-13T18:14:18.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f710bfff700 1 -- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8007020 msgr2=0x7f70f8007470 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:14:18.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f710bfff700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f8007020 0x7f70f8007470 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:18.946 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:14:18.946 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:14:18.946 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:14:18.946 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:14:18.946 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 192, 2024-09-13T18:14:18.946 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:14:18.946 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:14:18.946 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:14:18.946 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:14:18.946 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:18.947 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 4, 2024-09-13T18:14:18.947 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:14:18.947 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:14:18.947 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:14:18.947 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:18.947 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:18.947 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:18.947 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:18.947 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:18.947 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:18.947 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:18.947 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:18.947 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:18.947 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:18.948 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:18.948 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:14:18.948 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:14:18.948 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:18.948 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:18.948 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:18.948 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:18.948 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:18.948 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:18.948 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:18.948 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:14:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:14:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:14:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:14:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:14:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:14:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:14:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:14:18.949 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:14:18.950 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:14:18.950 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:14:18.950 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:14:18.950 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:14:18.950 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:14:18.950 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:14:18.950 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:14:18.950 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:14:18.950 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:18.950 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:18.950 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:18.950 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:18.950 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:18.951 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:18.951 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:18.951 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:18.951 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:18.951 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:18.951 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:18.951 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:14:18.951 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:18.951 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:14:18.951 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:18.951 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:14:18.951 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:14:18.951 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:18.951 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:18.952 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:18.952 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:18.952 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:14:18.952 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:18.952 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:18.952 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:18.952 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:18.952 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:14:18.952 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:18.952 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:18.952 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:18.952 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:18.952 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:14:18.952 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:14:18.953 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:18.953 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:18.953 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:18.953 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:18.953 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:18.953 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:14:18.953 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:14:18.953 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:18.953 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:18.953 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:18.953 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:18.953 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:14:18.953 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:18.954 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:18.954 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:18.954 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:18.954 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:14:18.954 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:18.954 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:18.954 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:18.954 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:18.954 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:14:18.954 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:14:18.954 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:18.954 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:18.954 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:18.954 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:18.955 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:18.955 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:14:18.955 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:14:18.955 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:18.955 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:18.955 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:18.955 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:18.955 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:14:18.955 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:18.955 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:18.955 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:18.955 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:18.955 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:14:18.956 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:18.956 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:18.956 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:18.956 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:18.956 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:14:18.956 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:14:18.956 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:18.956 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:18.956 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:18.956 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:18.957 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:18.957 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:18.957 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:14:18.957 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:14:18.957 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:18.957 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:18.957 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:18.957 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:14:18.957 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:18.957 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:18.957 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:14:18.957 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:18.957 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:18.958 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:18.958 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:14:18.958 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:18.958 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:18.958 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:18.958 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:14:18.958 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:14:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f7119c48700 1 -- 172.21.15.40:0/1550588526 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70f4066730 msgr2=0x7f70f4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f7119c48700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70f4066730 0x7f70f4068bf0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7104000c00 tx=0x7f7104008030 comp rx=0 tx=0).stop 2024-09-13T18:14:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f7113fff700 1 -- 172.21.15.40:0/1550588526 reap_dead start 2024-09-13T18:14:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f7119c48700 1 -- 172.21.15.40:0/1550588526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7114153af0 msgr2=0x7f7114155070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f7119c48700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7114153af0 0x7f7114155070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f710c00eed0 tx=0x7f710c00b390 comp rx=0 tx=0).stop 2024-09-13T18:14:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f7119c48700 1 -- 172.21.15.40:0/1550588526 shutdown_connections 2024-09-13T18:14:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f7119c48700 1 --2- 172.21.15.40:0/1550588526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7114153af0 0x7f7114155070 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f7119c48700 1 -- 172.21.15.40:0/1550588526 >> 172.21.15.40:0/1550588526 conn(0x7f7114066090 msgr2=0x7f71140b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f7119c48700 1 -- 172.21.15.40:0/1550588526 shutdown_connections 2024-09-13T18:14:18.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:18.934+0000 7f7119c48700 1 -- 172.21.15.40:0/1550588526 wait complete. 2024-09-13T18:14:18.960 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:14:18.961 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:14:18.961 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:14:19.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.078+0000 7fe3ccf25700 1 Processor -- start 2024-09-13T18:14:19.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.078+0000 7fe3ccf25700 1 -- start start 2024-09-13T18:14:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.078+0000 7fe3ccf25700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c80cf1a0 0x7fe3c815a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.078+0000 7fe3ccf25700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe3c80cc250 0x7fe3c80c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.078+0000 7fe3ccf25700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c815a7a0 con 0x7fe3c80cf580 2024-09-13T18:14:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.078+0000 7fe3ccf25700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c80cbdc0 con 0x7fe3c80cc250 2024-09-13T18:14:19.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.078+0000 7fe3ccf25700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c8153f80 con 0x7fe3c80cf1a0 2024-09-13T18:14:19.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.078+0000 7fe3c7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe3c80cc250 0x7fe3c80c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:19.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.078+0000 7fe3c7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe3c80cc250 0x7fe3c80c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38396/0 (socket says 172.21.15.40:38396) 2024-09-13T18:14:19.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.078+0000 7fe3c7fff700 1 -- 172.21.15.40:0/3859535906 learned_addr learned my addr 172.21.15.40:0/3859535906 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:19.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.078+0000 7fe3c77fe700 1 --2- 172.21.15.40:0/3859535906 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c80cf1a0 0x7fe3c815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:19.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.082+0000 7fe3c7fff700 1 -- 172.21.15.40:0/3859535906 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c80cf1a0 msgr2=0x7fe3c815a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:14:19.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.082+0000 7fe3c7fff700 1 --2- 172.21.15.40:0/3859535906 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c80cf1a0 0x7fe3c815a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:19.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.082+0000 7fe3c7fff700 1 -- 172.21.15.40:0/3859535906 >> v1:172.21.15.40:6789/0 conn(0x7fe3c80cf580 legacy=0x7fe3c80c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:14:19.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.082+0000 7fe3c7fff700 1 -- 172.21.15.40:0/3859535906 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c80caed0 con 0x7fe3c80cc250 2024-09-13T18:14:19.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.082+0000 7fe3c7fff700 1 --2- 172.21.15.40:0/3859535906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3c80cc250 0x7fe3c80c8aa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009e50 tx=0x7fe3c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb64aa768e57ad58 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:19.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.082+0000 7fe3c6ffd700 1 -- 172.21.15.40:0/3859535906 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c000e070 con 0x7fe3c80cc250 2024-09-13T18:14:19.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.082+0000 7fe3c6ffd700 1 -- 172.21.15.40:0/3859535906 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c000b910 con 0x7fe3c80cc250 2024-09-13T18:14:19.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.082+0000 7fe3c6ffd700 1 -- 172.21.15.40:0/3859535906 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c000bcc0 con 0x7fe3c80cc250 2024-09-13T18:14:19.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.082+0000 7fe3ccf25700 1 -- 172.21.15.40:0/3859535906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3c80cc250 msgr2=0x7fe3c80c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:19.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.082+0000 7fe3ccf25700 1 --2- 172.21.15.40:0/3859535906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3c80cc250 0x7fe3c80c8aa0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009e50 tx=0x7fe3c000b330 comp rx=0 tx=0).stop 2024-09-13T18:14:19.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.082+0000 7fe3ccf25700 1 -- 172.21.15.40:0/3859535906 shutdown_connections 2024-09-13T18:14:19.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.082+0000 7fe3ccf25700 1 --2- 172.21.15.40:0/3859535906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3c80cc250 0x7fe3c80c8aa0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:19.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.082+0000 7fe3ccf25700 1 --2- 172.21.15.40:0/3859535906 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c80cf1a0 0x7fe3c815a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:19.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.082+0000 7fe3ccf25700 1 -- 172.21.15.40:0/3859535906 >> 172.21.15.40:0/3859535906 conn(0x7fe3c8066090 msgr2=0x7fe3c8066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:19.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.082+0000 7fe3ccf25700 1 -- 172.21.15.40:0/3859535906 shutdown_connections 2024-09-13T18:14:19.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.082+0000 7fe3ccf25700 1 -- 172.21.15.40:0/3859535906 wait complete. 2024-09-13T18:14:19.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3ccf25700 1 Processor -- start 2024-09-13T18:14:19.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3ccf25700 1 -- start start 2024-09-13T18:14:19.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3ccf25700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3c80cc250 0x7fe3c8159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3ccf25700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c80cf1a0 0x7fe3c81eee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3ccf25700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3c80cf580 0x7fe3c81ef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3ccf25700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c81ee9e0 con 0x7fe3c80cf580 2024-09-13T18:14:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3ccf25700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c815a190 con 0x7fe3c80cc250 2024-09-13T18:14:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3ccf25700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c80b2bb0 con 0x7fe3c80cf1a0 2024-09-13T18:14:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3c7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3c80cf580 0x7fe3c81ef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3c7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3c80cf580 0x7fe3c81ef3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38406/0 (socket says 172.21.15.40:38406) 2024-09-13T18:14:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3c7fff700 1 -- 172.21.15.40:0/292251497 learned_addr learned my addr 172.21.15.40:0/292251497 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3c77fe700 1 --2- 172.21.15.40:0/292251497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3c80cc250 0x7fe3c8159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3bffff700 1 --2- 172.21.15.40:0/292251497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c80cf1a0 0x7fe3c81eee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3c7fff700 1 -- 172.21.15.40:0/292251497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c80cf1a0 msgr2=0x7fe3c81eee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:19.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3c7fff700 1 --2- 172.21.15.40:0/292251497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c80cf1a0 0x7fe3c81eee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:19.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3c7fff700 1 -- 172.21.15.40:0/292251497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3c80cc250 msgr2=0x7fe3c8159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:19.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3c7fff700 1 --2- 172.21.15.40:0/292251497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3c80cc250 0x7fe3c8159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:19.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3c7fff700 1 -- 172.21.15.40:0/292251497 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c0009b30 con 0x7fe3c80cf580 2024-09-13T18:14:19.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3c7fff700 1 --2- 172.21.15.40:0/292251497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3c80cf580 0x7fe3c81ef3b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe3c808c270 tx=0x7fe3c0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:19.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3c4ff9700 1 -- 172.21.15.40:0/292251497 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0004800 con 0x7fe3c80cf580 2024-09-13T18:14:19.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3c4ff9700 1 -- 172.21.15.40:0/292251497 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c00099b0 con 0x7fe3c80cf580 2024-09-13T18:14:19.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3ccf25700 1 -- 172.21.15.40:0/292251497 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c80b2e90 con 0x7fe3c80cf580 2024-09-13T18:14:19.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3c4ff9700 1 -- 172.21.15.40:0/292251497 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c001c680 con 0x7fe3c80cf580 2024-09-13T18:14:19.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.086+0000 7fe3ccf25700 1 -- 172.21.15.40:0/292251497 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c815a3e0 con 0x7fe3c80cf580 2024-09-13T18:14:19.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.090+0000 7fe3ccf25700 1 --2- 172.21.15.40:0/292251497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c80b3ac0 0x7fe3c81f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:19.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.090+0000 7fe3ccf25700 1 -- 172.21.15.40:0/292251497 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c8074750 con 0x7fe3c80b3ac0 2024-09-13T18:14:19.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.090+0000 7fe3c77fe700 1 --2- 172.21.15.40:0/292251497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c80b3ac0 0x7fe3c81f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:19.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.090+0000 7fe3c77fe700 1 -- 172.21.15.40:0/292251497 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe3c80cb770 con 0x7fe3c80b3ac0 2024-09-13T18:14:19.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.090+0000 7fe3c77fe700 1 --2- 172.21.15.40:0/292251497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c80b3ac0 0x7fe3c81f1db0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00119d0 tx=0x7fe3b0011d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:19.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.090+0000 7fe3c4ff9700 1 -- 172.21.15.40:0/292251497 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe3c001c7e0 con 0x7fe3c80cf580 2024-09-13T18:14:19.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.090+0000 7fe3c4ff9700 1 --2- 172.21.15.40:0/292251497 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe3a8066730 0x7fe3a8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:19.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.090+0000 7fe3c4ff9700 1 -- 172.21.15.40:0/292251497 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0004240 con 0x7fe3c80b3ac0 2024-09-13T18:14:19.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.090+0000 7fe3c4ff9700 1 -- 172.21.15.40:0/292251497 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe3c0086020 con 0x7fe3c80cf580 2024-09-13T18:14:19.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.090+0000 7fe3bffff700 1 --2- 172.21.15.40:0/292251497 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe3a8066730 0x7fe3a8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:19.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.090+0000 7fe3c4ff9700 1 -- 172.21.15.40:0/292251497 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe3b00043a0 con 0x7fe3c80b3ac0 2024-09-13T18:14:19.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.090+0000 7fe3c4ff9700 1 -- 172.21.15.40:0/292251497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c80b3ac0 msgr2=0x7fe3c81f1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:19.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.090+0000 7fe3c4ff9700 1 --2- 172.21.15.40:0/292251497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c80b3ac0 0x7fe3c81f1db0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00119d0 tx=0x7fe3b0011d90 comp rx=0 tx=0).stop 2024-09-13T18:14:19.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.090+0000 7fe3c4ff9700 1 -- 172.21.15.40:0/292251497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c80b3ac0 msgr2=0x7fe3c81f1db0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:14:19.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.090+0000 7fe3c4ff9700 1 --2- 172.21.15.40:0/292251497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c80b3ac0 0x7fe3c81f1db0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:19.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.090+0000 7fe3bffff700 1 --2- 172.21.15.40:0/292251497 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe3a8066730 0x7fe3a8068bf0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80ca0b0 tx=0x7fe3b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:19.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.098+0000 7fe3ccf25700 1 --2- 172.21.15.40:0/292251497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c81f22f0 0x7fe3c81f2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:19.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.098+0000 7fe3ccf25700 1 -- 172.21.15.40:0/292251497 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c80c48e0 con 0x7fe3c81f22f0 2024-09-13T18:14:19.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.098+0000 7fe3c77fe700 1 --2- 172.21.15.40:0/292251497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c81f22f0 0x7fe3c81f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:19.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.102+0000 7fe3c77fe700 1 -- 172.21.15.40:0/292251497 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe3c80cb490 con 0x7fe3c81f22f0 2024-09-13T18:14:19.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.102+0000 7fe3c77fe700 1 --2- 172.21.15.40:0/292251497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c81f22f0 0x7fe3c81f2700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe3b001b000 tx=0x7fe3b0004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:19.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.102+0000 7fe3c4ff9700 1 -- 172.21.15.40:0/292251497 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b001e650 con 0x7fe3c81f22f0 2024-09-13T18:14:19.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.102+0000 7fe3c4ff9700 1 -- 172.21.15.40:0/292251497 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7fe3b000fe60 con 0x7fe3c81f22f0 2024-09-13T18:14:19.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.102+0000 7fe3c4ff9700 1 -- 172.21.15.40:0/292251497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c81f22f0 msgr2=0x7fe3c81f2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:19.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.102+0000 7fe3c4ff9700 1 --2- 172.21.15.40:0/292251497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c81f22f0 0x7fe3c81f2700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe3b001b000 tx=0x7fe3b0004770 comp rx=0 tx=0).stop 2024-09-13T18:14:19.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.102+0000 7fe3c4ff9700 1 -- 172.21.15.40:0/292251497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c81f22f0 msgr2=0x7fe3c81f2700 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:14:19.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.102+0000 7fe3c4ff9700 1 --2- 172.21.15.40:0/292251497 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3c81f22f0 0x7fe3c81f2700 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe3b001b000 tx=0x7fe3b0004770 comp rx=0 tx=0).stop 2024-09-13T18:14:19.112 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:14:19.112 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:14:19.112 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:14:19.112 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:14:19.112 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 192, 2024-09-13T18:14:19.112 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:14:19.112 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:14:19.112 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:14:19.112 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:14:19.112 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:19.113 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 5, 2024-09-13T18:14:19.113 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:14:19.113 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:14:19.113 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:14:19.113 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:19.113 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:19.113 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:19.113 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:19.113 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:19.113 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:19.113 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:19.113 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:19.113 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:19.113 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:19.114 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:19.114 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:14:19.114 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:14:19.114 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:19.114 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:19.114 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:19.114 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:19.114 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:19.114 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:19.114 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:19.114 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:19.114 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:19.114 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:19.114 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:19.115 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:19.115 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:14:19.115 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:14:19.115 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:14:19.115 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:14:19.115 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:14:19.115 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:14:19.115 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:14:19.115 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:14:19.115 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:14:19.115 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:14:19.115 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:14:19.115 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:14:19.116 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:14:19.116 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:14:19.116 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:14:19.116 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:14:19.116 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:14:19.116 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:19.116 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:19.116 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:19.116 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:19.116 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:19.116 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:19.116 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:19.116 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:19.116 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:19.117 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:19.117 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:19.117 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:14:19.117 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:19.117 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:14:19.117 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:19.117 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:14:19.117 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:14:19.117 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:19.117 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:19.117 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:19.117 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:19.117 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:14:19.118 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:19.118 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:19.118 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:19.118 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:19.118 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:14:19.118 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:19.118 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:19.118 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:19.118 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:19.118 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:14:19.118 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:14:19.118 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:19.118 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:19.118 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:19.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:19.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:19.119 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:14:19.119 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:14:19.119 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:19.119 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:19.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:19.119 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:19.119 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:14:19.119 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:19.119 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:19.119 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:19.119 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:19.120 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:14:19.120 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:19.120 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:19.120 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:19.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:19.120 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:14:19.120 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:14:19.120 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:19.120 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:19.120 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:19.120 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:19.120 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:19.120 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:14:19.120 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:14:19.121 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:19.121 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:19.121 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:19.121 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:19.121 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:14:19.121 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:19.121 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:19.121 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:19.121 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:19.121 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:14:19.121 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:19.121 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:19.121 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:19.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:19.122 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:14:19.122 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:14:19.122 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:19.122 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:19.122 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:19.122 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:19.122 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:19.122 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:19.122 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:14:19.122 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:14:19.122 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:19.122 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:19.122 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:19.123 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:14:19.123 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:19.123 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:19.123 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:14:19.123 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:19.123 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:19.123 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:19.123 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:14:19.123 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:19.123 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:19.123 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:14:19.123 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:19.123 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:19.123 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:19.124 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:14:19.124 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:19.124 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:19.124 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:14:19.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:19.124 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:19.124 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:19.124 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:14:19.124 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:19.124 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:19.124 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:14:19.124 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:19.124 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:19.124 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:19.125 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:14:19.125 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:19.125 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:19.125 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:19.125 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:14:19.125 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:14:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.102+0000 7fe3ccf25700 1 -- 172.21.15.40:0/292251497 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe3a8066730 msgr2=0x7fe3a8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.102+0000 7fe3ccf25700 1 --2- 172.21.15.40:0/292251497 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe3a8066730 0x7fe3a8068bf0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80ca0b0 tx=0x7fe3b8009040 comp rx=0 tx=0).stop 2024-09-13T18:14:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.102+0000 7fe3ccf25700 1 -- 172.21.15.40:0/292251497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3c80cf580 msgr2=0x7fe3c81ef3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.102+0000 7fe3ccf25700 1 --2- 172.21.15.40:0/292251497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe3c80cf580 0x7fe3c81ef3b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe3c808c270 tx=0x7fe3c0004af0 comp rx=0 tx=0).stop 2024-09-13T18:14:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.102+0000 7fe3c7fff700 1 -- 172.21.15.40:0/292251497 reap_dead start 2024-09-13T18:14:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.102+0000 7fe3c7fff700 1 -- 172.21.15.40:0/292251497 reap_dead start 2024-09-13T18:14:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.102+0000 7fe3ccf25700 1 -- 172.21.15.40:0/292251497 shutdown_connections 2024-09-13T18:14:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.102+0000 7fe3ccf25700 1 -- 172.21.15.40:0/292251497 >> 172.21.15.40:0/292251497 conn(0x7fe3c8066090 msgr2=0x7fe3c80c7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.102+0000 7fe3ccf25700 1 -- 172.21.15.40:0/292251497 shutdown_connections 2024-09-13T18:14:19.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.102+0000 7fe3ccf25700 1 -- 172.21.15.40:0/292251497 wait complete. 2024-09-13T18:14:19.129 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:14:19.129 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:14:19.130 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:14:19.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848f106700 1 Processor -- start 2024-09-13T18:14:19.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848f106700 1 -- start start 2024-09-13T18:14:19.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848f106700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8480051a10 0x7f84800dbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:19.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848f106700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8480045d00 0x7f848004d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:19.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848f106700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8480007830 con 0x7f848004cc20 2024-09-13T18:14:19.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848f106700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f848004d540 con 0x7f8480051a10 2024-09-13T18:14:19.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848f106700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8480007a80 con 0x7f8480045d00 2024-09-13T18:14:19.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848e104700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f848004cc20 0x7f8480008b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36566/0 (socket says 172.21.15.40:36566) 2024-09-13T18:14:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848e104700 1 -- 172.21.15.40:0/2217433063 learned_addr learned my addr 172.21.15.40:0/2217433063 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:14:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848e905700 1 --2- 172.21.15.40:0/2217433063 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8480045d00 0x7f848004d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848d903700 1 --2- 172.21.15.40:0/2217433063 >> v2:172.21.15.43:3300/0 conn(0x7f8480051a10 0x7f84800dbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848d102700 1 -- 172.21.15.40:0/2217433063 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723556685 0 0) 0x7f8480007830 con 0x7f848004cc20 2024-09-13T18:14:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848d102700 1 -- 172.21.15.40:0/2217433063 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84740035d0 con 0x7f848004cc20 2024-09-13T18:14:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848d102700 1 -- 172.21.15.40:0/2217433063 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3243563753 0 0) 0x7f84740035d0 con 0x7f848004cc20 2024-09-13T18:14:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848e905700 1 -- 172.21.15.40:0/2217433063 >> v2:172.21.15.43:3300/0 conn(0x7f8480051a10 msgr2=0x7f84800dbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:14:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848e905700 1 --2- 172.21.15.40:0/2217433063 >> v2:172.21.15.43:3300/0 conn(0x7f8480051a10 0x7f84800dbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848e905700 1 -- 172.21.15.40:0/2217433063 >> v1:172.21.15.40:6789/0 conn(0x7f848004cc20 legacy=0x7f8480008b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848e905700 1 -- 172.21.15.40:0/2217433063 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8480007d60 con 0x7f8480045d00 2024-09-13T18:14:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848e905700 1 --2- 172.21.15.40:0/2217433063 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8480045d00 0x7f848004d000 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8488069620 tx=0x7f8488069e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4981d786b10a94d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:19.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848d102700 1 -- 172.21.15.40:0/2217433063 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8484001670 con 0x7f8480045d00 2024-09-13T18:14:19.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848d102700 1 -- 172.21.15.40:0/2217433063 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84880689b0 con 0x7f8480045d00 2024-09-13T18:14:19.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848d102700 1 -- 172.21.15.40:0/2217433063 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488067fd0 con 0x7f8480045d00 2024-09-13T18:14:19.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848f106700 1 -- 172.21.15.40:0/2217433063 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8480045d00 msgr2=0x7f848004d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:19.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848f106700 1 --2- 172.21.15.40:0/2217433063 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8480045d00 0x7f848004d000 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8488069620 tx=0x7f8488069e70 comp rx=0 tx=0).stop 2024-09-13T18:14:19.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848f106700 1 -- 172.21.15.40:0/2217433063 shutdown_connections 2024-09-13T18:14:19.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848f106700 1 --2- 172.21.15.40:0/2217433063 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8480045d00 0x7f848004d000 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:19.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848f106700 1 --2- 172.21.15.40:0/2217433063 >> v2:172.21.15.43:3300/0 conn(0x7f8480051a10 0x7f84800dbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:19.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848f106700 1 -- 172.21.15.40:0/2217433063 >> 172.21.15.40:0/2217433063 conn(0x7f8480050e60 msgr2=0x7f8480051270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:19.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.238+0000 7f848f106700 1 -- 172.21.15.40:0/2217433063 shutdown_connections 2024-09-13T18:14:19.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.242+0000 7f848f106700 1 -- 172.21.15.40:0/2217433063 wait complete. 2024-09-13T18:14:19.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.242+0000 7f848f106700 1 Processor -- start 2024-09-13T18:14:19.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.242+0000 7f848f106700 1 -- start start 2024-09-13T18:14:19.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.242+0000 7f848f106700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8480045d00 0x7f84800def10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:19.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.242+0000 7f848f106700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f848004cc20 0x7f84800da1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:19.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.242+0000 7f848f106700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8480051a10 0x7f8480049970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:19.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.242+0000 7f848f106700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8480174f90 con 0x7f8480045d00 2024-09-13T18:14:19.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.242+0000 7f848f106700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84800df450 con 0x7f848004cc20 2024-09-13T18:14:19.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.242+0000 7f848f106700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84800491b0 con 0x7f8480051a10 2024-09-13T18:14:19.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.242+0000 7f848e104700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8480045d00 0x7f84800def10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:19.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.242+0000 7f848e104700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8480045d00 0x7f84800def10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:38408/0 (socket says 172.21.15.40:38408) 2024-09-13T18:14:19.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.242+0000 7f848e104700 1 -- 172.21.15.40:0/935357776 learned_addr learned my addr 172.21.15.40:0/935357776 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:19.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.242+0000 7f848d903700 1 --2- 172.21.15.40:0/935357776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f848004cc20 0x7f84800da1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:19.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.242+0000 7f848e905700 1 --2- 172.21.15.40:0/935357776 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8480051a10 0x7f8480049970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:19.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.246+0000 7f848e104700 1 -- 172.21.15.40:0/935357776 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8480051a10 msgr2=0x7f8480049970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:19.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.246+0000 7f848e104700 1 --2- 172.21.15.40:0/935357776 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8480051a10 0x7f8480049970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:19.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.246+0000 7f848e104700 1 -- 172.21.15.40:0/935357776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f848004cc20 msgr2=0x7f84800da1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:19.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.246+0000 7f848e104700 1 --2- 172.21.15.40:0/935357776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f848004cc20 0x7f84800da1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:19.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.246+0000 7f848e104700 1 -- 172.21.15.40:0/935357776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848804f5c0 con 0x7f8480045d00 2024-09-13T18:14:19.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.246+0000 7f848e104700 1 --2- 172.21.15.40:0/935357776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8480045d00 0x7f84800def10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f84840011e0 tx=0x7f848400ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:19.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.246+0000 7f847effd700 1 -- 172.21.15.40:0/935357776 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8484004ca0 con 0x7f8480045d00 2024-09-13T18:14:19.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.246+0000 7f847effd700 1 -- 172.21.15.40:0/935357776 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8484004e00 con 0x7f8480045d00 2024-09-13T18:14:19.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.246+0000 7f847effd700 1 -- 172.21.15.40:0/935357776 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84840072f0 con 0x7f8480045d00 2024-09-13T18:14:19.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.246+0000 7f848f106700 1 -- 172.21.15.40:0/935357776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84800d3690 con 0x7f8480045d00 2024-09-13T18:14:19.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.246+0000 7f848f106700 1 -- 172.21.15.40:0/935357776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84800d3c50 con 0x7f8480045d00 2024-09-13T18:14:19.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.246+0000 7f848f106700 1 -- 172.21.15.40:0/935357776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848004bbe0 con 0x7f8480045d00 2024-09-13T18:14:19.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.246+0000 7f847effd700 1 -- 172.21.15.40:0/935357776 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8484007450 con 0x7f8480045d00 2024-09-13T18:14:19.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.246+0000 7f847effd700 1 --2- 172.21.15.40:0/935357776 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f846c066730 0x7f846c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:19.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.246+0000 7f847effd700 1 -- 172.21.15.40:0/935357776 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f84840842c0 con 0x7f8480045d00 2024-09-13T18:14:19.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.246+0000 7f848d903700 1 --2- 172.21.15.40:0/935357776 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f846c066730 0x7f846c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:19.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.250+0000 7f848d903700 1 --2- 172.21.15.40:0/935357776 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f846c066730 0x7f846c068bf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f84800d4190 tx=0x7f8478009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:19.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.258+0000 7f847effd700 1 -- 172.21.15.40:0/935357776 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8484051270 con 0x7f8480045d00 2024-09-13T18:14:19.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.522+0000 7f848f106700 1 -- 172.21.15.40:0/935357776 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f848004c230 con 0x7f8480045d00 2024-09-13T18:14:19.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.526+0000 7f847effd700 1 -- 172.21.15.40:0/935357776 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f8484054890 con 0x7f8480045d00 2024-09-13T18:14:19.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.530+0000 7f848f106700 1 -- 172.21.15.40:0/935357776 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f846c066730 msgr2=0x7f846c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:19.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.530+0000 7f848f106700 1 --2- 172.21.15.40:0/935357776 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f846c066730 0x7f846c068bf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f84800d4190 tx=0x7f8478009240 comp rx=0 tx=0).stop 2024-09-13T18:14:19.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.530+0000 7f848f106700 1 -- 172.21.15.40:0/935357776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8480045d00 msgr2=0x7f84800def10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:19.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.530+0000 7f848f106700 1 --2- 172.21.15.40:0/935357776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8480045d00 0x7f84800def10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f84840011e0 tx=0x7f848400ca50 comp rx=0 tx=0).stop 2024-09-13T18:14:19.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.530+0000 7f848f106700 1 -- 172.21.15.40:0/935357776 shutdown_connections 2024-09-13T18:14:19.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.530+0000 7f848f106700 1 --2- 172.21.15.40:0/935357776 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f846c066730 0x7f846c068bf0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:19.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.530+0000 7f848f106700 1 --2- 172.21.15.40:0/935357776 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8480051a10 0x7f8480049970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:19.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.530+0000 7f848f106700 1 --2- 172.21.15.40:0/935357776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f848004cc20 0x7f84800da1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:19.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.530+0000 7f848f106700 1 --2- 172.21.15.40:0/935357776 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8480045d00 0x7f84800def10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:19.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.530+0000 7f848f106700 1 -- 172.21.15.40:0/935357776 >> 172.21.15.40:0/935357776 conn(0x7f8480050e60 msgr2=0x7f84800d47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:19.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.530+0000 7f848f106700 1 -- 172.21.15.40:0/935357776 shutdown_connections 2024-09-13T18:14:19.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:19.530+0000 7f848f106700 1 -- 172.21.15.40:0/935357776 wait complete. 2024-09-13T18:14:19.556 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:14:29.558 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:14:29.558 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:14:29.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.694+0000 7f15d7fff700 1 Processor -- start 2024-09-13T18:14:29.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.694+0000 7f15d7fff700 1 -- start start 2024-09-13T18:14:29.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.694+0000 7f15d7fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f15d8156e90 0x7f15d8159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:29.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.694+0000 7f15d7fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f15d80c6d70 0x7f15d806e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:29.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.694+0000 7f15d7fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d80c7920 con 0x7f15d8150570 2024-09-13T18:14:29.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.694+0000 7f15d7fff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d80c8110 con 0x7f15d80c6d70 2024-09-13T18:14:29.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.694+0000 7f15d7fff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d81561a0 con 0x7f15d8156e90 2024-09-13T18:14:29.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.694+0000 7f15d77fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f15d80c6d70 0x7f15d806e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:29.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.694+0000 7f15d77fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f15d80c6d70 0x7f15d806e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39554/0 (socket says 172.21.15.40:39554) 2024-09-13T18:14:29.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.694+0000 7f15d77fe700 1 -- 172.21.15.40:0/3382243758 learned_addr learned my addr 172.21.15.40:0/3382243758 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:29.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.694+0000 7f15d67fc700 1 --2- 172.21.15.40:0/3382243758 >> v2:172.21.15.43:3300/0 conn(0x7f15d8156e90 0x7f15d8159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:29.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.698+0000 7f15d77fe700 1 -- 172.21.15.40:0/3382243758 >> v2:172.21.15.43:3300/0 conn(0x7f15d8156e90 msgr2=0x7f15d8159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:14:29.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.698+0000 7f15d77fe700 1 --2- 172.21.15.40:0/3382243758 >> v2:172.21.15.43:3300/0 conn(0x7f15d8156e90 0x7f15d8159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:29.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.698+0000 7f15d77fe700 1 -- 172.21.15.40:0/3382243758 >> v1:172.21.15.43:6789/0 conn(0x7f15d8150570 legacy=0x7f15d80b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:29.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.698+0000 7f15d77fe700 1 -- 172.21.15.40:0/3382243758 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d8150950 con 0x7f15d80c6d70 2024-09-13T18:14:29.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.698+0000 7f15d77fe700 1 --2- 172.21.15.40:0/3382243758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15d80c6d70 0x7f15d806e3d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f15d806c750 tx=0x7f15d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e819897e28ca75a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:29.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.698+0000 7f15d5ffb700 1 -- 172.21.15.40:0/3382243758 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d000e070 con 0x7f15d80c6d70 2024-09-13T18:14:29.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.698+0000 7f15d5ffb700 1 -- 172.21.15.40:0/3382243758 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d000b910 con 0x7f15d80c6d70 2024-09-13T18:14:29.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.698+0000 7f15d5ffb700 1 -- 172.21.15.40:0/3382243758 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d000bcc0 con 0x7f15d80c6d70 2024-09-13T18:14:29.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.698+0000 7f15d7fff700 1 -- 172.21.15.40:0/3382243758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15d80c6d70 msgr2=0x7f15d806e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:29.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.698+0000 7f15d7fff700 1 --2- 172.21.15.40:0/3382243758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15d80c6d70 0x7f15d806e3d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f15d806c750 tx=0x7f15d000b330 comp rx=0 tx=0).stop 2024-09-13T18:14:29.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.698+0000 7f15d7fff700 1 -- 172.21.15.40:0/3382243758 shutdown_connections 2024-09-13T18:14:29.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.698+0000 7f15d7fff700 1 --2- 172.21.15.40:0/3382243758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15d80c6d70 0x7f15d806e3d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:29.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.698+0000 7f15d7fff700 1 --2- 172.21.15.40:0/3382243758 >> v2:172.21.15.43:3300/0 conn(0x7f15d8156e90 0x7f15d8159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:29.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.698+0000 7f15d7fff700 1 -- 172.21.15.40:0/3382243758 >> 172.21.15.40:0/3382243758 conn(0x7f15d80cbd80 msgr2=0x7f15d80cc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:29.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.698+0000 7f15d7fff700 1 -- 172.21.15.40:0/3382243758 shutdown_connections 2024-09-13T18:14:29.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.698+0000 7f15d7fff700 1 -- 172.21.15.40:0/3382243758 wait complete. 2024-09-13T18:14:29.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.698+0000 7f15d7fff700 1 Processor -- start 2024-09-13T18:14:29.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.698+0000 7f15d7fff700 1 -- start start 2024-09-13T18:14:29.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15d7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15d80c6d70 0x7f15d814a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:29.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15d7fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15d8150570 0x7f15d80c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:29.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15d7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15d8156e90 0x7f15d814b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:29.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15d7fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d81edb00 con 0x7f15d8156e90 2024-09-13T18:14:29.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15d7fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d81edf20 con 0x7f15d80c6d70 2024-09-13T18:14:29.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15d7fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d80c5780 con 0x7f15d8150570 2024-09-13T18:14:29.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15d77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15d8156e90 0x7f15d814b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:29.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15d77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15d8156e90 0x7f15d814b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39570/0 (socket says 172.21.15.40:39570) 2024-09-13T18:14:29.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15d77fe700 1 -- 172.21.15.40:0/2885732292 learned_addr learned my addr 172.21.15.40:0/2885732292 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:29.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15d6ffd700 1 --2- 172.21.15.40:0/2885732292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15d80c6d70 0x7f15d814a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:29.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15d67fc700 1 --2- 172.21.15.40:0/2885732292 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15d8150570 0x7f15d80c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:29.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15d77fe700 1 -- 172.21.15.40:0/2885732292 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15d8150570 msgr2=0x7f15d80c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:29.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15d77fe700 1 --2- 172.21.15.40:0/2885732292 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15d8150570 0x7f15d80c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:29.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15d77fe700 1 -- 172.21.15.40:0/2885732292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15d80c6d70 msgr2=0x7f15d814a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:29.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15d77fe700 1 --2- 172.21.15.40:0/2885732292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15d80c6d70 0x7f15d814a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:29.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15d77fe700 1 -- 172.21.15.40:0/2885732292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d0009b30 con 0x7f15d8156e90 2024-09-13T18:14:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15d77fe700 1 --2- 172.21.15.40:0/2885732292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15d8156e90 0x7f15d814b3b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f15d0003f70 tx=0x7f15d0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15bf7fe700 1 -- 172.21.15.40:0/2885732292 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d00099b0 con 0x7f15d8156e90 2024-09-13T18:14:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15bf7fe700 1 -- 172.21.15.40:0/2885732292 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d0004250 con 0x7f15d8156e90 2024-09-13T18:14:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15d7fff700 1 -- 172.21.15.40:0/2885732292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d814b8f0 con 0x7f15d8156e90 2024-09-13T18:14:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15d7fff700 1 -- 172.21.15.40:0/2885732292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d814adf0 con 0x7f15d8156e90 2024-09-13T18:14:29.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.702+0000 7f15bf7fe700 1 -- 172.21.15.40:0/2885732292 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d00045e0 con 0x7f15d8156e90 2024-09-13T18:14:29.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.706+0000 7f15bf7fe700 1 -- 172.21.15.40:0/2885732292 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f15d0004740 con 0x7f15d8156e90 2024-09-13T18:14:29.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.706+0000 7f15bf7fe700 1 --2- 172.21.15.40:0/2885732292 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f15c4066760 0x7f15c4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:29.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.706+0000 7f15bf7fe700 1 -- 172.21.15.40:0/2885732292 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f15d0085340 con 0x7f15d8156e90 2024-09-13T18:14:29.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.706+0000 7f15d6ffd700 1 --2- 172.21.15.40:0/2885732292 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f15c4066760 0x7f15c4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:29.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.706+0000 7f15d7fff700 1 -- 172.21.15.40:0/2885732292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d814b8f0 con 0x7f15d8156e90 2024-09-13T18:14:29.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.706+0000 7f15d6ffd700 1 --2- 172.21.15.40:0/2885732292 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f15c4066760 0x7f15c4068c20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f15c8000f60 tx=0x7f15c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:29.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.710+0000 7f15bf7fe700 1 -- 172.21.15.40:0/2885732292 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f15d814b8f0 con 0x7f15d8156e90 2024-09-13T18:14:29.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.966+0000 7f15d7fff700 1 -- 172.21.15.40:0/2885732292 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f15d81edc40 con 0x7f15d8156e90 2024-09-13T18:14:29.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.966+0000 7f15bf7fe700 1 -- 172.21.15.40:0/2885732292 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f15d81edc40 con 0x7f15d8156e90 2024-09-13T18:14:29.977 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":192,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:14:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.970+0000 7f15d7fff700 1 -- 172.21.15.40:0/2885732292 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f15c4066760 msgr2=0x7f15c4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:29.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.970+0000 7f15d7fff700 1 --2- 172.21.15.40:0/2885732292 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f15c4066760 0x7f15c4068c20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f15c8000f60 tx=0x7f15c8009040 comp rx=0 tx=0).stop 2024-09-13T18:14:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.970+0000 7f15d7fff700 1 -- 172.21.15.40:0/2885732292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15d8156e90 msgr2=0x7f15d814b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.970+0000 7f15d7fff700 1 --2- 172.21.15.40:0/2885732292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15d8156e90 0x7f15d814b3b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f15d0003f70 tx=0x7f15d0004af0 comp rx=0 tx=0).stop 2024-09-13T18:14:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.970+0000 7f15d7fff700 1 -- 172.21.15.40:0/2885732292 shutdown_connections 2024-09-13T18:14:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.970+0000 7f15d7fff700 1 --2- 172.21.15.40:0/2885732292 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15d8156e90 0x7f15d814b3b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.970+0000 7f15d7fff700 1 --2- 172.21.15.40:0/2885732292 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f15c4066760 0x7f15c4068c20 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.970+0000 7f15d7fff700 1 --2- 172.21.15.40:0/2885732292 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f15d8150570 0x7f15d80c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.970+0000 7f15d7fff700 1 --2- 172.21.15.40:0/2885732292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15d80c6d70 0x7f15d814a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:29.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.970+0000 7f15d7fff700 1 -- 172.21.15.40:0/2885732292 >> 172.21.15.40:0/2885732292 conn(0x7f15d80cbd80 msgr2=0x7f15d806e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:29.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.970+0000 7f15d7fff700 1 -- 172.21.15.40:0/2885732292 shutdown_connections 2024-09-13T18:14:29.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:29.970+0000 7f15d7fff700 1 -- 172.21.15.40:0/2885732292 wait complete. 2024-09-13T18:14:29.991 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:14:29.992 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T18:14:29.992 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:14:29.992 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:14:30.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.098+0000 7ff2a1283700 1 Processor -- start 2024-09-13T18:14:30.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.098+0000 7ff2a1283700 1 -- start start 2024-09-13T18:14:30.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2a1283700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c0c8500 0x7ff29c0c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2a1283700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff29c0c71c0 0x7ff29c159630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2a1283700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29c0c8f80 con 0x7ff29c1503c0 2024-09-13T18:14:30.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2a1283700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff29c0c2a80 con 0x7ff29c0c71c0 2024-09-13T18:14:30.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2a1283700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c0c80d0 con 0x7ff29c0c8500 2024-09-13T18:14:30.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2a0a82700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff29c0c71c0 0x7ff29c159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2937fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c0c8500 0x7ff29c0c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff29bfff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff29c1503c0 0x7ff29c0c79c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:40330/0 (socket says 172.21.15.40:40330) 2024-09-13T18:14:30.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2a0a82700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff29c0c71c0 0x7ff29c159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:53646/0 (socket says 172.21.15.40:53646) 2024-09-13T18:14:30.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff29bfff700 1 -- 172.21.15.40:0/865867169 learned_addr learned my addr 172.21.15.40:0/865867169 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:14:30.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2a0a82700 1 -- 172.21.15.40:0/865867169 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c0c8500 msgr2=0x7ff29c0c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:14:30.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2a0a82700 1 --2- 172.21.15.40:0/865867169 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c0c8500 0x7ff29c0c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2a0a82700 1 -- 172.21.15.40:0/865867169 >> v1:172.21.15.43:6789/0 conn(0x7ff29c1503c0 legacy=0x7ff29c0c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff29b7fe700 1 -- 172.21.15.40:0/865867169 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082852950 0 0) 0x7ff29c0c8f80 con 0x7ff29c1503c0 2024-09-13T18:14:30.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2a0a82700 1 -- 172.21.15.40:0/865867169 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c1507a0 con 0x7ff29c0c71c0 2024-09-13T18:14:30.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2a0a82700 1 --2- 172.21.15.40:0/865867169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c71c0 0x7ff29c159630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff294009d90 tx=0x7ff29400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b139a5603a4cb0ba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:30.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff29b7fe700 1 -- 172.21.15.40:0/865867169 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29400bd20 con 0x7ff29c0c71c0 2024-09-13T18:14:30.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff29b7fe700 1 -- 172.21.15.40:0/865867169 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29400be80 con 0x7ff29c0c71c0 2024-09-13T18:14:30.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff29b7fe700 1 -- 172.21.15.40:0/865867169 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2940116a0 con 0x7ff29c0c71c0 2024-09-13T18:14:30.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2a1283700 1 -- 172.21.15.40:0/865867169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c71c0 msgr2=0x7ff29c159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2a1283700 1 --2- 172.21.15.40:0/865867169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c71c0 0x7ff29c159630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff294009d90 tx=0x7ff29400b330 comp rx=0 tx=0).stop 2024-09-13T18:14:30.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2a1283700 1 -- 172.21.15.40:0/865867169 shutdown_connections 2024-09-13T18:14:30.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2a1283700 1 --2- 172.21.15.40:0/865867169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c71c0 0x7ff29c159630 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2a1283700 1 --2- 172.21.15.40:0/865867169 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c0c8500 0x7ff29c0c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2a1283700 1 -- 172.21.15.40:0/865867169 >> 172.21.15.40:0/865867169 conn(0x7ff29c06e6d0 msgr2=0x7ff29c06eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:30.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2a1283700 1 -- 172.21.15.40:0/865867169 shutdown_connections 2024-09-13T18:14:30.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.102+0000 7ff2a1283700 1 -- 172.21.15.40:0/865867169 wait complete. 2024-09-13T18:14:30.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.106+0000 7ff2a1283700 1 Processor -- start 2024-09-13T18:14:30.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.106+0000 7ff2a1283700 1 -- start start 2024-09-13T18:14:30.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2a1283700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c71c0 0x7ff29c1473c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2a1283700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0c8500 0x7ff29c148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2a1283700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c1503c0 0x7ff29c149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2a1283700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c06b730 con 0x7ff29c0c8500 2024-09-13T18:14:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2a1283700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c142eb0 con 0x7ff29c0c71c0 2024-09-13T18:14:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2a1283700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c06b980 con 0x7ff29c1503c0 2024-09-13T18:14:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2937fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0c8500 0x7ff29c148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2937fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0c8500 0x7ff29c148ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39576/0 (socket says 172.21.15.40:39576) 2024-09-13T18:14:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2937fe700 1 -- 172.21.15.40:0/2138301307 learned_addr learned my addr 172.21.15.40:0/2138301307 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff29bfff700 1 --2- 172.21.15.40:0/2138301307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c71c0 0x7ff29c1473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2a0a82700 1 --2- 172.21.15.40:0/2138301307 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c1503c0 0x7ff29c149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2937fe700 1 -- 172.21.15.40:0/2138301307 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c1503c0 msgr2=0x7ff29c149530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2937fe700 1 --2- 172.21.15.40:0/2138301307 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c1503c0 0x7ff29c149530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2937fe700 1 -- 172.21.15.40:0/2138301307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c71c0 msgr2=0x7ff29c1473c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2937fe700 1 --2- 172.21.15.40:0/2138301307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c0c71c0 0x7ff29c1473c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2937fe700 1 -- 172.21.15.40:0/2138301307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff294009a70 con 0x7ff29c0c8500 2024-09-13T18:14:30.120 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2937fe700 1 --2- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0c8500 0x7ff29c148ff0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff28400e9d0 tx=0x7ff28400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2997fa700 1 -- 172.21.15.40:0/2138301307 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284009720 con 0x7ff29c0c8500 2024-09-13T18:14:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2997fa700 1 -- 172.21.15.40:0/2138301307 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28400ee70 con 0x7ff29c0c8500 2024-09-13T18:14:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2a1283700 1 -- 172.21.15.40:0/2138301307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff29c06bc60 con 0x7ff29c0c8500 2024-09-13T18:14:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2997fa700 1 -- 172.21.15.40:0/2138301307 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28400ca50 con 0x7ff29c0c8500 2024-09-13T18:14:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2a1283700 1 -- 172.21.15.40:0/2138301307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff29c06c220 con 0x7ff29c0c8500 2024-09-13T18:14:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2a1283700 1 --2- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c1f23b0 0x7ff29c1f2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff2a1283700 1 -- 172.21.15.40:0/2138301307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c067600 con 0x7ff29c1f23b0 2024-09-13T18:14:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff29bfff700 1 --2- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c1f23b0 0x7ff29c1f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff29bfff700 1 -- 172.21.15.40:0/2138301307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff29c0c8ce0 con 0x7ff29c1f23b0 2024-09-13T18:14:30.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.110+0000 7ff29bfff700 1 --2- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c1f23b0 0x7ff29c1f2970 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff28c000c00 tx=0x7ff28c001860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:30.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.114+0000 7ff2997fa700 1 -- 172.21.15.40:0/2138301307 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c017070 con 0x7ff29c1f23b0 2024-09-13T18:14:30.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.114+0000 7ff2997fa700 1 -- 172.21.15.40:0/2138301307 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff28400cbb0 con 0x7ff29c0c8500 2024-09-13T18:14:30.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.114+0000 7ff2997fa700 1 --2- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff27c066800 0x7ff27c068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.114+0000 7ff2997fa700 1 -- 172.21.15.40:0/2138301307 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff284084490 con 0x7ff29c0c8500 2024-09-13T18:14:30.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.114+0000 7ff2a0a82700 1 --2- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff27c066800 0x7ff27c068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.114+0000 7ff2997fa700 1 -- 172.21.15.40:0/2138301307 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff28c012560 con 0x7ff29c1f23b0 2024-09-13T18:14:30.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.114+0000 7ff2997fa700 1 -- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c1f23b0 msgr2=0x7ff29c1f2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.114+0000 7ff2997fa700 1 --2- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c1f23b0 0x7ff29c1f2970 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff28c000c00 tx=0x7ff28c001860 comp rx=0 tx=0).stop 2024-09-13T18:14:30.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.114+0000 7ff2997fa700 1 -- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c1f23b0 msgr2=0x7ff29c1f2970 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:14:30.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.114+0000 7ff2997fa700 1 --2- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c1f23b0 0x7ff29c1f2970 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.114+0000 7ff2a0a82700 1 --2- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff27c066800 0x7ff27c068cc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff29400b370 tx=0x7ff294013000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:30.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.122+0000 7ff2a1283700 1 --2- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0c77a0 0x7ff29c1f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.122+0000 7ff2a1283700 1 -- 172.21.15.40:0/2138301307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c06bdf0 con 0x7ff29c0c77a0 2024-09-13T18:14:30.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.122+0000 7ff29bfff700 1 --2- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0c77a0 0x7ff29c1f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.122+0000 7ff29bfff700 1 -- 172.21.15.40:0/2138301307 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff29c0c8a40 con 0x7ff29c0c77a0 2024-09-13T18:14:30.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.122+0000 7ff29bfff700 1 --2- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0c77a0 0x7ff29c1f2eb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff28c015040 tx=0x7ff28c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:30.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.122+0000 7ff2997fa700 1 -- 172.21.15.40:0/2138301307 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c023070 con 0x7ff29c0c77a0 2024-09-13T18:14:30.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.126+0000 7ff2997fa700 1 -- 172.21.15.40:0/2138301307 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4785 (secure 0 0 0) 0x7ff28c01ee10 con 0x7ff29c0c77a0 2024-09-13T18:14:30.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.126+0000 7ff2997fa700 1 -- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0c77a0 msgr2=0x7ff29c1f2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.126+0000 7ff2997fa700 1 --2- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0c77a0 0x7ff29c1f2eb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff28c015040 tx=0x7ff28c000ed0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.126+0000 7ff2997fa700 1 -- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0c77a0 msgr2=0x7ff29c1f2eb0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:14:30.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.126+0000 7ff2997fa700 1 --2- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0c77a0 0x7ff29c1f2eb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.135 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:14:30.135 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:14:30.135 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:14:30.135 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:14:30.135 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 192, 2024-09-13T18:14:30.135 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:14:30.136 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:14:30.136 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:14:30.136 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:14:30.136 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:30.136 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 16, 2024-09-13T18:14:30.136 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:14:30.136 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:14:30.136 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:14:30.136 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:30.136 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:30.136 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:30.137 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:30.137 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:30.137 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:30.137 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:30.137 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:30.137 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:30.137 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:30.137 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:30.137 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:14:30.137 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:14:30.137 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:30.137 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:30.138 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:30.138 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:30.138 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:30.138 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:30.138 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:30.138 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:30.138 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:30.138 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:30.138 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:30.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.138 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:14:30.138 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:14:30.138 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:14:30.139 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:14:30.139 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:14:30.139 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:14:30.139 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:14:30.139 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:14:30.139 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:14:30.139 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:14:30.139 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:14:30.139 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:14:30.139 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:14:30.139 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:14:30.139 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:14:30.140 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:14:30.140 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:14:30.140 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:30.140 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:30.140 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:30.140 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:30.140 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:30.140 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:30.140 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:30.140 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:30.140 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:30.140 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:30.140 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:30.141 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:14:30.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.141 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:14:30.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.141 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:14:30.141 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:14:30.141 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:30.141 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:30.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.141 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:30.141 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:14:30.141 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:30.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.142 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:30.142 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:14:30.142 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:30.142 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.142 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:30.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.142 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:14:30.142 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:14:30.143 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:30.143 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:30.143 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:30.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.143 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:14:30.143 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:14:30.143 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:30.143 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:30.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.143 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:30.143 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:14:30.143 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:30.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.144 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:30.144 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:14:30.144 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:30.144 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.144 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:30.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.144 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:14:30.144 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:14:30.144 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:30.145 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:30.145 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:30.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.145 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.145 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:14:30.145 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:14:30.145 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:30.145 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:30.145 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.145 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:30.145 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:14:30.145 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:30.146 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.146 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.146 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:30.146 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:14:30.146 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:30.146 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.146 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:30.146 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.146 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:14:30.146 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:14:30.146 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:30.147 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:30.147 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:30.147 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.147 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:30.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.147 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:14:30.147 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:14:30.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.147 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:30.147 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:30.147 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:14:30.147 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.147 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:30.148 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:14:30.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.148 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:30.148 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:30.148 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:14:30.148 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.148 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:30.148 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:14:30.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.148 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:30.148 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:30.148 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:14:30.149 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.149 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:30.149 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:14:30.149 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.149 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:30.149 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:30.149 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:14:30.149 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.149 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:30.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.149 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:14:30.149 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:14:30.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.126+0000 7ff2a1283700 1 -- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff27c066800 msgr2=0x7ff27c068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.126+0000 7ff2a1283700 1 --2- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff27c066800 0x7ff27c068cc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff29400b370 tx=0x7ff294013000 comp rx=0 tx=0).stop 2024-09-13T18:14:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.126+0000 7ff2a1283700 1 -- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0c8500 msgr2=0x7ff29c148ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.126+0000 7ff2a1283700 1 --2- 172.21.15.40:0/2138301307 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff29c0c8500 0x7ff29c148ff0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff28400e9d0 tx=0x7ff28400ece0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.126+0000 7ff2a0a82700 1 -- 172.21.15.40:0/2138301307 reap_dead start 2024-09-13T18:14:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.126+0000 7ff2a0a82700 1 -- 172.21.15.40:0/2138301307 reap_dead start 2024-09-13T18:14:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.126+0000 7ff2a1283700 1 -- 172.21.15.40:0/2138301307 shutdown_connections 2024-09-13T18:14:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.126+0000 7ff2a1283700 1 -- 172.21.15.40:0/2138301307 >> 172.21.15.40:0/2138301307 conn(0x7ff29c06e6d0 msgr2=0x7ff29c14b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.126+0000 7ff2a1283700 1 -- 172.21.15.40:0/2138301307 shutdown_connections 2024-09-13T18:14:30.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.126+0000 7ff2a1283700 1 -- 172.21.15.40:0/2138301307 wait complete. 2024-09-13T18:14:30.152 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:14:30.152 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:14:30.152 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:14:30.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f477a344700 1 Processor -- start 2024-09-13T18:14:30.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f477a344700 1 -- start start 2024-09-13T18:14:30.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f477a344700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4774153720 0x7f47740ca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f477a344700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4774161d40 0x7f4774164130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f477a344700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4774159e50 con 0x7f47740cf9b0 2024-09-13T18:14:30.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f477a344700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47740c5ea0 con 0x7f4774161d40 2024-09-13T18:14:30.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f477a344700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477415d390 con 0x7f4774153720 2024-09-13T18:14:30.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f4779b43700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4774161d40 0x7f4774164130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f4779b43700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4774161d40 0x7f4774164130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39598/0 (socket says 172.21.15.40:39598) 2024-09-13T18:14:30.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f4779b43700 1 -- 172.21.15.40:0/807605848 learned_addr learned my addr 172.21.15.40:0/807605848 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:30.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f4779342700 1 --2- 172.21.15.40:0/807605848 >> v2:172.21.15.43:3300/0 conn(0x7f4774153720 0x7f47740ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f4779b43700 1 -- 172.21.15.40:0/807605848 >> v2:172.21.15.43:3300/0 conn(0x7f4774153720 msgr2=0x7f47740ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:14:30.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f4779b43700 1 --2- 172.21.15.40:0/807605848 >> v2:172.21.15.43:3300/0 conn(0x7f4774153720 0x7f47740ca510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f4779b43700 1 -- 172.21.15.40:0/807605848 >> v1:172.21.15.43:6789/0 conn(0x7f47740cf9b0 legacy=0x7f477406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f4779b43700 1 -- 172.21.15.40:0/807605848 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477415a080 con 0x7f4774161d40 2024-09-13T18:14:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f4779b43700 1 --2- 172.21.15.40:0/807605848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4774161d40 0x7f4774164130 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f47740cf940 tx=0x7f476c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce81ea2aac64d419 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:30.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f476bfff700 1 -- 172.21.15.40:0/807605848 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c00e070 con 0x7f4774161d40 2024-09-13T18:14:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f476bfff700 1 -- 172.21.15.40:0/807605848 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476c00b910 con 0x7f4774161d40 2024-09-13T18:14:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f476bfff700 1 -- 172.21.15.40:0/807605848 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c00bcc0 con 0x7f4774161d40 2024-09-13T18:14:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f477a344700 1 -- 172.21.15.40:0/807605848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4774161d40 msgr2=0x7f4774164130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f477a344700 1 --2- 172.21.15.40:0/807605848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4774161d40 0x7f4774164130 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f47740cf940 tx=0x7f476c00b330 comp rx=0 tx=0).stop 2024-09-13T18:14:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f477a344700 1 -- 172.21.15.40:0/807605848 shutdown_connections 2024-09-13T18:14:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f477a344700 1 --2- 172.21.15.40:0/807605848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4774161d40 0x7f4774164130 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f477a344700 1 --2- 172.21.15.40:0/807605848 >> v2:172.21.15.43:3300/0 conn(0x7f4774153720 0x7f47740ca510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f477a344700 1 -- 172.21.15.40:0/807605848 >> 172.21.15.40:0/807605848 conn(0x7f47740666c0 msgr2=0x7f47740cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f477a344700 1 -- 172.21.15.40:0/807605848 shutdown_connections 2024-09-13T18:14:30.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.258+0000 7f477a344700 1 -- 172.21.15.40:0/807605848 wait complete. 2024-09-13T18:14:30.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.262+0000 7f477a344700 1 Processor -- start 2024-09-13T18:14:30.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.262+0000 7f477a344700 1 -- start start 2024-09-13T18:14:30.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.262+0000 7f477a344700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47740cf9b0 0x7f47740ce380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.262+0000 7f477a344700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4774153720 0x7f47741effc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.262+0000 7f477a344700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4774161d40 0x7f47740c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.262+0000 7f477a344700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47740c8b80 con 0x7f47740cf9b0 2024-09-13T18:14:30.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.262+0000 7f477a344700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47740c9310 con 0x7f4774153720 2024-09-13T18:14:30.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.262+0000 7f477a344700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477415a6c0 con 0x7f4774161d40 2024-09-13T18:14:30.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.262+0000 7f4779342700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47740cf9b0 0x7f47740ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.262+0000 7f4779342700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47740cf9b0 0x7f47740ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39612/0 (socket says 172.21.15.40:39612) 2024-09-13T18:14:30.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.262+0000 7f4779342700 1 -- 172.21.15.40:0/1917780354 learned_addr learned my addr 172.21.15.40:0/1917780354 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:30.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.262+0000 7f4779b43700 1 --2- 172.21.15.40:0/1917780354 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4774161d40 0x7f47740c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.262+0000 7f4778b41700 1 --2- 172.21.15.40:0/1917780354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4774153720 0x7f47741effc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.262+0000 7f4779342700 1 -- 172.21.15.40:0/1917780354 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4774161d40 msgr2=0x7f47740c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.262+0000 7f4779342700 1 --2- 172.21.15.40:0/1917780354 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4774161d40 0x7f47740c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.262+0000 7f4779342700 1 -- 172.21.15.40:0/1917780354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4774153720 msgr2=0x7f47741effc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.262+0000 7f4779342700 1 --2- 172.21.15.40:0/1917780354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4774153720 0x7f47741effc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.262+0000 7f4779342700 1 -- 172.21.15.40:0/1917780354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c009b30 con 0x7f47740cf9b0 2024-09-13T18:14:30.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.266+0000 7f4779342700 1 --2- 172.21.15.40:0/1917780354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47740cf9b0 0x7f47740ce380 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f477000ebe0 tx=0x7f477000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:30.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.266+0000 7f4769ffb700 1 -- 172.21.15.40:0/1917780354 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4770009740 con 0x7f47740cf9b0 2024-09-13T18:14:30.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.266+0000 7f4769ffb700 1 -- 172.21.15.40:0/1917780354 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477000c850 con 0x7f47740cf9b0 2024-09-13T18:14:30.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.266+0000 7f477a344700 1 -- 172.21.15.40:0/1917780354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47741f3fe0 con 0x7f47740cf9b0 2024-09-13T18:14:30.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.266+0000 7f4769ffb700 1 -- 172.21.15.40:0/1917780354 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477000cbe0 con 0x7f47740cf9b0 2024-09-13T18:14:30.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.266+0000 7f477a344700 1 -- 172.21.15.40:0/1917780354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47740c8df0 con 0x7f47740cf9b0 2024-09-13T18:14:30.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.266+0000 7f475f7fe700 1 --2- 172.21.15.40:0/1917780354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477414c950 0x7f47740b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.266+0000 7f475f7fe700 1 -- 172.21.15.40:0/1917780354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4774074750 con 0x7f477414c950 2024-09-13T18:14:30.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.266+0000 7f4778b41700 1 --2- 172.21.15.40:0/1917780354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477414c950 0x7f47740b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.266+0000 7f4769ffb700 1 -- 172.21.15.40:0/1917780354 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f477000cd40 con 0x7f47740cf9b0 2024-09-13T18:14:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.266+0000 7f4778b41700 1 -- 172.21.15.40:0/1917780354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4774068810 con 0x7f477414c950 2024-09-13T18:14:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.266+0000 7f4769ffb700 1 --2- 172.21.15.40:0/1917780354 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4758066730 0x7f4758068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.266+0000 7f4778b41700 1 --2- 172.21.15.40:0/1917780354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477414c950 0x7f47740b25f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f47740b3cb0 tx=0x7f476400fb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.266+0000 7f4769ffb700 1 -- 172.21.15.40:0/1917780354 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4770084490 con 0x7f47740cf9b0 2024-09-13T18:14:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.266+0000 7f4779b43700 1 --2- 172.21.15.40:0/1917780354 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4758066730 0x7f4758068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.266+0000 7f4769ffb700 1 -- 172.21.15.40:0/1917780354 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764017070 con 0x7f477414c950 2024-09-13T18:14:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.266+0000 7f4779b43700 1 --2- 172.21.15.40:0/1917780354 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4758066730 0x7f4758068bf0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f47740b46c0 tx=0x7f476c009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.270+0000 7f4769ffb700 1 -- 172.21.15.40:0/1917780354 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f47640052e0 con 0x7f477414c950 2024-09-13T18:14:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.270+0000 7f4769ffb700 1 -- 172.21.15.40:0/1917780354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477414c950 msgr2=0x7f47740b25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.270+0000 7f4769ffb700 1 --2- 172.21.15.40:0/1917780354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477414c950 0x7f47740b25f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f47740b3cb0 tx=0x7f476400fb00 comp rx=0 tx=0).stop 2024-09-13T18:14:30.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.270+0000 7f4769ffb700 1 -- 172.21.15.40:0/1917780354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477414c950 msgr2=0x7f47740b25f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:14:30.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.270+0000 7f4769ffb700 1 --2- 172.21.15.40:0/1917780354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477414c950 0x7f47740b25f0 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f47740b3cb0 tx=0x7f476400fb00 comp rx=0 tx=0).stop 2024-09-13T18:14:30.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.274+0000 7f475f7fe700 1 --2- 172.21.15.40:0/1917780354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47741f0500 0x7f47740b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.285 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.274+0000 7f475f7fe700 1 -- 172.21.15.40:0/1917780354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4774151e00 con 0x7f47741f0500 2024-09-13T18:14:30.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.274+0000 7f4778b41700 1 --2- 172.21.15.40:0/1917780354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47741f0500 0x7f47740b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.278+0000 7f4778b41700 1 -- 172.21.15.40:0/1917780354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f477414f890 con 0x7f47741f0500 2024-09-13T18:14:30.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.278+0000 7f4778b41700 1 --2- 172.21.15.40:0/1917780354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47741f0500 0x7f47740b2b30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4774067890 tx=0x7f4764001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:30.287 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.278+0000 7f4769ffb700 1 -- 172.21.15.40:0/1917780354 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764023070 con 0x7f47741f0500 2024-09-13T18:14:30.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.278+0000 7f4769ffb700 1 -- 172.21.15.40:0/1917780354 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4451 (secure 0 0 0) 0x7f476401ed20 con 0x7f47741f0500 2024-09-13T18:14:30.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.278+0000 7f4769ffb700 1 -- 172.21.15.40:0/1917780354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47741f0500 msgr2=0x7f47740b2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.278+0000 7f4769ffb700 1 --2- 172.21.15.40:0/1917780354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47741f0500 0x7f47740b2b30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4774067890 tx=0x7f4764001120 comp rx=0 tx=0).stop 2024-09-13T18:14:30.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.278+0000 7f4769ffb700 1 -- 172.21.15.40:0/1917780354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47741f0500 msgr2=0x7f47740b2b30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:14:30.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.278+0000 7f4769ffb700 1 --2- 172.21.15.40:0/1917780354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47741f0500 0x7f47740b2b30 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.288 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:14:30.288 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:14:30.288 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:14:30.288 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:14:30.289 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 192, 2024-09-13T18:14:30.289 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:14:30.289 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:14:30.289 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:14:30.289 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:14:30.289 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:30.289 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 16, 2024-09-13T18:14:30.289 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:14:30.289 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:14:30.289 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:14:30.289 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:30.289 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:30.289 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:30.290 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:30.290 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:30.290 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:30.290 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:30.290 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:30.290 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:30.290 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:30.290 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:30.290 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:14:30.290 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:14:30.290 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:30.290 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:30.290 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:30.290 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:30.291 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:30.291 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:30.291 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:30.291 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:30.291 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:30.291 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:30.291 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:30.291 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.291 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:14:30.291 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:14:30.291 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:14:30.291 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:14:30.291 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:14:30.291 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:14:30.291 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:14:30.292 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:14:30.292 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:14:30.292 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:14:30.292 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:14:30.292 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:14:30.292 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:14:30.292 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:14:30.292 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:14:30.292 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:14:30.292 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:14:30.292 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:30.292 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:30.292 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:30.293 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:30.293 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:30.293 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:30.293 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:30.293 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:30.293 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:30.293 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:30.293 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:30.293 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:14:30.293 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.293 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:14:30.293 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.293 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:14:30.293 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:14:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:14:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:14:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:30.294 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:14:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:14:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:14:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:14:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:14:30.295 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:14:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:14:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:14:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:30.296 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:14:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:14:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:14:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:14:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:14:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:14:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:14:30.298 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:14:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:14:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:14:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:14:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.299 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:14:30.299 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:14:30.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.278+0000 7f475f7fe700 1 -- 172.21.15.40:0/1917780354 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4758066730 msgr2=0x7f4758068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.278+0000 7f475f7fe700 1 --2- 172.21.15.40:0/1917780354 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4758066730 0x7f4758068bf0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f47740b46c0 tx=0x7f476c009000 comp rx=0 tx=0).stop 2024-09-13T18:14:30.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.278+0000 7f475f7fe700 1 -- 172.21.15.40:0/1917780354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47740cf9b0 msgr2=0x7f47740ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.278+0000 7f475f7fe700 1 --2- 172.21.15.40:0/1917780354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47740cf9b0 0x7f47740ce380 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f477000ebe0 tx=0x7f477000eef0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.278+0000 7f4779b43700 1 -- 172.21.15.40:0/1917780354 reap_dead start 2024-09-13T18:14:30.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.278+0000 7f4779b43700 1 -- 172.21.15.40:0/1917780354 reap_dead start 2024-09-13T18:14:30.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.278+0000 7f475f7fe700 1 -- 172.21.15.40:0/1917780354 shutdown_connections 2024-09-13T18:14:30.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.278+0000 7f475f7fe700 1 -- 172.21.15.40:0/1917780354 >> 172.21.15.40:0/1917780354 conn(0x7f47740666c0 msgr2=0x7f4774150f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:30.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.278+0000 7f475f7fe700 1 -- 172.21.15.40:0/1917780354 shutdown_connections 2024-09-13T18:14:30.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.278+0000 7f475f7fe700 1 -- 172.21.15.40:0/1917780354 wait complete. 2024-09-13T18:14:30.302 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:14:30.303 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:14:30.303 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:14:30.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.406+0000 7f5f95bd8700 1 Processor -- start 2024-09-13T18:14:30.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.406+0000 7f5f95bd8700 1 -- start start 2024-09-13T18:14:30.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f95bd8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5f900cf1a0 0x7f5f9015a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f95bd8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f900cc250 0x7f5f900c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f95bd8700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f9015a7a0 con 0x7f5f900cf580 2024-09-13T18:14:30.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f95bd8700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f900cbdc0 con 0x7f5f900cf1a0 2024-09-13T18:14:30.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f95bd8700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f90153f80 con 0x7f5f900cc250 2024-09-13T18:14:30.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f94bd6700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5f900cf1a0 0x7f5f9015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f8ffff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f5f900cf580 0x7f5f900c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:44876/0 (socket says 172.21.15.40:44876) 2024-09-13T18:14:30.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f94bd6700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5f900cf1a0 0x7f5f9015a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39628/0 (socket says 172.21.15.40:39628) 2024-09-13T18:14:30.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f8ffff700 1 -- 172.21.15.40:0/1068562342 learned_addr learned my addr 172.21.15.40:0/1068562342 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:14:30.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f953d7700 1 --2- 172.21.15.40:0/1068562342 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f900cc250 0x7f5f900c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f94bd6700 1 -- 172.21.15.40:0/1068562342 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f900cc250 msgr2=0x7f5f900c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:14:30.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f94bd6700 1 --2- 172.21.15.40:0/1068562342 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f900cc250 0x7f5f900c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f94bd6700 1 -- 172.21.15.40:0/1068562342 >> v1:172.21.15.40:6789/0 conn(0x7f5f900cf580 legacy=0x7f5f900c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f8f7fe700 1 -- 172.21.15.40:0/1068562342 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3531004485 0 0) 0x7f5f84001110 con 0x7f5f900cf580 2024-09-13T18:14:30.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f94bd6700 1 -- 172.21.15.40:0/1068562342 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f900caed0 con 0x7f5f900cf1a0 2024-09-13T18:14:30.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f953d7700 1 --2- 172.21.15.40:0/1068562342 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f900cc250 0x7f5f900c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:14:30.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f94bd6700 1 --2- 172.21.15.40:0/1068562342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f900cf1a0 0x7f5f9015a260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009850 tx=0x7f5f7c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e4b647e766519b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:30.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f8f7fe700 1 -- 172.21.15.40:0/1068562342 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c011830 con 0x7f5f900cf1a0 2024-09-13T18:14:30.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f8f7fe700 1 -- 172.21.15.40:0/1068562342 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7c011990 con 0x7f5f900cf1a0 2024-09-13T18:14:30.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f8f7fe700 1 -- 172.21.15.40:0/1068562342 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c011d20 con 0x7f5f900cf1a0 2024-09-13T18:14:30.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f95bd8700 1 -- 172.21.15.40:0/1068562342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f900cf1a0 msgr2=0x7f5f9015a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f95bd8700 1 --2- 172.21.15.40:0/1068562342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f900cf1a0 0x7f5f9015a260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009850 tx=0x7f5f7c009b60 comp rx=0 tx=0).stop 2024-09-13T18:14:30.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f95bd8700 1 -- 172.21.15.40:0/1068562342 shutdown_connections 2024-09-13T18:14:30.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f95bd8700 1 --2- 172.21.15.40:0/1068562342 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f900cc250 0x7f5f900c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f95bd8700 1 --2- 172.21.15.40:0/1068562342 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f900cf1a0 0x7f5f9015a260 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f95bd8700 1 -- 172.21.15.40:0/1068562342 >> 172.21.15.40:0/1068562342 conn(0x7f5f90066090 msgr2=0x7f5f90066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:30.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f95bd8700 1 -- 172.21.15.40:0/1068562342 shutdown_connections 2024-09-13T18:14:30.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f95bd8700 1 -- 172.21.15.40:0/1068562342 wait complete. 2024-09-13T18:14:30.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f95bd8700 1 Processor -- start 2024-09-13T18:14:30.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.410+0000 7f5f95bd8700 1 -- start start 2024-09-13T18:14:30.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f95bd8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f900cc250 0x7f5f900c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f95bd8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f900cf1a0 0x7f5f901f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f95bd8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f900cf580 0x7f5f90150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f95bd8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f90158270 con 0x7f5f900cc250 2024-09-13T18:14:30.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f95bd8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f90158d30 con 0x7f5f900cf1a0 2024-09-13T18:14:30.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f95bd8700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9015a4a0 con 0x7f5f900cf580 2024-09-13T18:14:30.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f94bd6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f900cc250 0x7f5f900c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f94bd6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f900cc250 0x7f5f900c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39644/0 (socket says 172.21.15.40:39644) 2024-09-13T18:14:30.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f94bd6700 1 -- 172.21.15.40:0/2959897026 learned_addr learned my addr 172.21.15.40:0/2959897026 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:30.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f953d7700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f900cf580 0x7f5f90150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f8ffff700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f900cf1a0 0x7f5f901f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f94bd6700 1 -- 172.21.15.40:0/2959897026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f900cf580 msgr2=0x7f5f90150170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f94bd6700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f900cf580 0x7f5f90150170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f94bd6700 1 -- 172.21.15.40:0/2959897026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f900cf1a0 msgr2=0x7f5f901f2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f94bd6700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f900cf1a0 0x7f5f901f2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f94bd6700 1 -- 172.21.15.40:0/2959897026 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c0094b0 con 0x7f5f900cc250 2024-09-13T18:14:30.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f94bd6700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f900cc250 0x7f5f900c9bd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5f9008c270 tx=0x7f5f7c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:30.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f8d7fa700 1 -- 172.21.15.40:0/2959897026 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c0038c0 con 0x7f5f900cc250 2024-09-13T18:14:30.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f8d7fa700 1 -- 172.21.15.40:0/2959897026 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7c003a20 con 0x7f5f900cc250 2024-09-13T18:14:30.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f95bd8700 1 -- 172.21.15.40:0/2959897026 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f90157550 con 0x7f5f900cc250 2024-09-13T18:14:30.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f8d7fa700 1 -- 172.21.15.40:0/2959897026 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c003db0 con 0x7f5f900cc250 2024-09-13T18:14:30.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f95bd8700 1 -- 172.21.15.40:0/2959897026 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f90159230 con 0x7f5f900cc250 2024-09-13T18:14:30.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f95bd8700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f901f2a40 0x7f5f901506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f95bd8700 1 -- 172.21.15.40:0/2959897026 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f90074750 con 0x7f5f901f2a40 2024-09-13T18:14:30.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.414+0000 7f5f8ffff700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f901f2a40 0x7f5f901506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.418+0000 7f5f8d7fa700 1 -- 172.21.15.40:0/2959897026 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5f7c01a430 con 0x7f5f900cc250 2024-09-13T18:14:30.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.418+0000 7f5f8d7fa700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5f74066730 0x7f5f74068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.418+0000 7f5f8ffff700 1 -- 172.21.15.40:0/2959897026 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5f90153ad0 con 0x7f5f901f2a40 2024-09-13T18:14:30.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.418+0000 7f5f8d7fa700 1 -- 172.21.15.40:0/2959897026 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5f7c021030 con 0x7f5f900cc250 2024-09-13T18:14:30.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.418+0000 7f5f953d7700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5f74066730 0x7f5f74068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.418+0000 7f5f8ffff700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f901f2a40 0x7f5f901506b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5f900c9810 tx=0x7f5f84001710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:30.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.418+0000 7f5f8d7fa700 1 -- 172.21.15.40:0/2959897026 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f840114f0 con 0x7f5f901f2a40 2024-09-13T18:14:30.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.418+0000 7f5f953d7700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5f74066730 0x7f5f74068bf0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009aa0 tx=0x7f5f88003aa0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:30.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.418+0000 7f5f8d7fa700 1 -- 172.21.15.40:0/2959897026 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5f84011650 con 0x7f5f901f2a40 2024-09-13T18:14:30.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.418+0000 7f5f8d7fa700 1 -- 172.21.15.40:0/2959897026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f901f2a40 msgr2=0x7f5f901506b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.418+0000 7f5f8d7fa700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f901f2a40 0x7f5f901506b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5f900c9810 tx=0x7f5f84001710 comp rx=0 tx=0).stop 2024-09-13T18:14:30.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.418+0000 7f5f8d7fa700 1 -- 172.21.15.40:0/2959897026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f901f2a40 msgr2=0x7f5f901506b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:14:30.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.418+0000 7f5f8d7fa700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f901f2a40 0x7f5f901506b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.430+0000 7f5f95bd8700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f90159570 0x7f5f900c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.430+0000 7f5f95bd8700 1 -- 172.21.15.40:0/2959897026 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900cb770 con 0x7f5f90159570 2024-09-13T18:14:30.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.430+0000 7f5f8ffff700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f90159570 0x7f5f900c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.434+0000 7f5f8ffff700 1 -- 172.21.15.40:0/2959897026 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f5f900cb490 con 0x7f5f90159570 2024-09-13T18:14:30.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.434+0000 7f5f8ffff700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f90159570 0x7f5f900c3b20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5f84013000 tx=0x7f5f84001080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.434+0000 7f5f8d7fa700 1 -- 172.21.15.40:0/2959897026 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8401db50 con 0x7f5f90159570 2024-09-13T18:14:30.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.434+0000 7f5f8d7fa700 1 -- 172.21.15.40:0/2959897026 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4949 (secure 0 0 0) 0x7f5f8401dcb0 con 0x7f5f90159570 2024-09-13T18:14:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.434+0000 7f5f8d7fa700 1 -- 172.21.15.40:0/2959897026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f90159570 msgr2=0x7f5f900c3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.434+0000 7f5f8d7fa700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f90159570 0x7f5f900c3b20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5f84013000 tx=0x7f5f84001080 comp rx=0 tx=0).stop 2024-09-13T18:14:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.434+0000 7f5f8d7fa700 1 -- 172.21.15.40:0/2959897026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f90159570 msgr2=0x7f5f900c3b20 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:14:30.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.434+0000 7f5f8d7fa700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5f90159570 0x7f5f900c3b20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.444 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:14:30.444 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:14:30.444 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:14:30.444 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:14:30.444 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 192, 2024-09-13T18:14:30.444 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:14:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:14:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:14:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:14:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 16, 2024-09-13T18:14:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:14:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:14:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:14:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:30.445 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:14:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:14:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:30.446 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:30.447 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:30.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.447 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:14:30.447 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:14:30.447 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:14:30.447 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:14:30.447 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:14:30.447 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:14:30.447 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:14:30.447 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:14:30.447 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:14:30.448 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:14:30.448 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:14:30.448 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:14:30.448 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:14:30.448 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:14:30.448 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:14:30.448 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:14:30.448 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:14:30.448 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:30.448 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:30.448 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:30.448 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:30.449 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:30.449 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:30.449 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:30.449 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:30.449 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:30.449 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:30.449 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:30.449 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:14:30.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.449 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:14:30.449 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.449 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:14:30.449 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:14:30.450 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:30.450 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:30.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.450 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:30.450 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:14:30.450 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:30.450 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.450 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:30.450 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:14:30.450 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:30.451 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.451 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:30.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.451 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:14:30.451 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:14:30.451 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:30.451 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:30.451 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:30.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.451 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:14:30.451 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:14:30.451 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:30.452 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:30.452 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.452 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:30.452 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:14:30.452 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:30.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.452 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.452 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:30.453 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:14:30.453 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:30.453 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.453 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:30.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.453 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:14:30.453 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:14:30.453 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:30.453 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:30.453 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:30.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.453 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.453 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:14:30.453 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:14:30.454 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:30.454 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:30.454 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.454 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:30.454 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:14:30.454 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:30.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.454 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.455 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:30.455 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:14:30.455 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:30.455 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.455 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:30.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.455 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:14:30.455 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:14:30.455 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:30.455 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:30.455 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:30.455 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.456 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:30.456 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.456 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:14:30.456 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:14:30.456 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.456 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:30.456 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:30.456 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:14:30.456 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.456 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:30.456 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:14:30.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.457 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:30.457 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:30.457 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:14:30.457 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.457 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:30.457 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:14:30.457 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.457 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:30.457 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:30.457 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:14:30.458 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.458 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:30.458 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:14:30.458 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.458 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:30.458 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:30.458 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:14:30.458 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.458 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:30.458 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:14:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:14:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:30.459 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:14:30.459 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:14:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.434+0000 7f5f95bd8700 1 -- 172.21.15.40:0/2959897026 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5f74066730 msgr2=0x7f5f74068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.434+0000 7f5f95bd8700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5f74066730 0x7f5f74068bf0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009aa0 tx=0x7f5f88003aa0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.434+0000 7f5f953d7700 1 -- 172.21.15.40:0/2959897026 reap_dead start 2024-09-13T18:14:30.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.434+0000 7f5f95bd8700 1 -- 172.21.15.40:0/2959897026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f900cc250 msgr2=0x7f5f900c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.434+0000 7f5f95bd8700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f900cc250 0x7f5f900c9bd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5f9008c270 tx=0x7f5f7c004770 comp rx=0 tx=0).stop 2024-09-13T18:14:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.434+0000 7f5f95bd8700 1 -- 172.21.15.40:0/2959897026 shutdown_connections 2024-09-13T18:14:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.434+0000 7f5f95bd8700 1 --2- 172.21.15.40:0/2959897026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f900cc250 0x7f5f900c9bd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.434+0000 7f5f95bd8700 1 -- 172.21.15.40:0/2959897026 >> 172.21.15.40:0/2959897026 conn(0x7f5f90066090 msgr2=0x7f5f9015c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.434+0000 7f5f95bd8700 1 -- 172.21.15.40:0/2959897026 shutdown_connections 2024-09-13T18:14:30.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.434+0000 7f5f95bd8700 1 -- 172.21.15.40:0/2959897026 wait complete. 2024-09-13T18:14:30.462 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:14:30.462 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-13T18:14:30.462 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T18:14:30.462 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-13T18:14:30.462 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T18:14:30.463 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T18:14:30.463 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:14:30.512 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T18:14:30.513 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T18:14:30.513 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:14:30.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.614+0000 7fcab6340700 1 Processor -- start 2024-09-13T18:14:30.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.614+0000 7fcab6340700 1 -- start start 2024-09-13T18:14:30.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.614+0000 7fcab6340700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcab0153880 0x7fcab00c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.614+0000 7fcab6340700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcab00cb2f0 0x7fcab00cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.614+0000 7fcab6340700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab0159770 con 0x7fcab00cf3c0 2024-09-13T18:14:30.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.614+0000 7fcab6340700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab0159a50 con 0x7fcab0153880 2024-09-13T18:14:30.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.614+0000 7fcab6340700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab0159d30 con 0x7fcab00cb2f0 2024-09-13T18:14:30.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.614+0000 7fcab533e700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7fcab0153880 msgr2=0x7fcab00c9ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:14:30.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.614+0000 7fcab533e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcab0153880 0x7fcab00c9ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:14:30.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.618+0000 7fcab5b3f700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7fcab00cf3c0 legacy=0x7fcab015cf60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:14:30.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.618+0000 7fcab4b3d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcab00cb2f0 0x7fcab00cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.618+0000 7fcab4b3d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcab00cb2f0 0x7fcab00cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:53750/0 (socket says 172.21.15.40:53750) 2024-09-13T18:14:30.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.618+0000 7fcab4b3d700 1 -- 172.21.15.40:0/2214873358 learned_addr learned my addr 172.21.15.40:0/2214873358 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:30.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.618+0000 7fcab4b3d700 1 -- 172.21.15.40:0/2214873358 >> v2:172.21.15.40:3300/0 conn(0x7fcab0153880 msgr2=0x7fcab00c9ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:14:30.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.618+0000 7fcab4b3d700 1 --2- 172.21.15.40:0/2214873358 >> v2:172.21.15.40:3300/0 conn(0x7fcab0153880 0x7fcab00c9ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.618+0000 7fcab4b3d700 1 -- 172.21.15.40:0/2214873358 >> v1:172.21.15.40:6789/0 conn(0x7fcab00cf3c0 legacy=0x7fcab015cf60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:14:30.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.618+0000 7fcab4b3d700 1 -- 172.21.15.40:0/2214873358 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab015a010 con 0x7fcab00cb2f0 2024-09-13T18:14:30.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.618+0000 7fcab4b3d700 1 --2- 172.21.15.40:0/2214873358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab00cb2f0 0x7fcab00cee80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcab00cb0b0 tx=0x7fcaa0009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=361b9317c0421ee server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:30.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.618+0000 7fcaa7fff700 1 -- 172.21.15.40:0/2214873358 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0004230 con 0x7fcab00cb2f0 2024-09-13T18:14:30.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.618+0000 7fcaa7fff700 1 -- 172.21.15.40:0/2214873358 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa0004390 con 0x7fcab00cb2f0 2024-09-13T18:14:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.618+0000 7fcaa7fff700 1 -- 172.21.15.40:0/2214873358 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00036c0 con 0x7fcab00cb2f0 2024-09-13T18:14:30.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.618+0000 7fcab6340700 1 -- 172.21.15.40:0/2214873358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab00cb2f0 msgr2=0x7fcab00cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.618+0000 7fcab6340700 1 --2- 172.21.15.40:0/2214873358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab00cb2f0 0x7fcab00cee80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcab00cb0b0 tx=0x7fcaa0009bf0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.618+0000 7fcab6340700 1 -- 172.21.15.40:0/2214873358 shutdown_connections 2024-09-13T18:14:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.618+0000 7fcab6340700 1 --2- 172.21.15.40:0/2214873358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab00cb2f0 0x7fcab00cee80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.618+0000 7fcab6340700 1 --2- 172.21.15.40:0/2214873358 >> v2:172.21.15.40:3300/0 conn(0x7fcab0153880 0x7fcab00c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.618+0000 7fcab6340700 1 -- 172.21.15.40:0/2214873358 >> 172.21.15.40:0/2214873358 conn(0x7fcab006c5b0 msgr2=0x7fcab006c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.618+0000 7fcab6340700 1 -- 172.21.15.40:0/2214873358 shutdown_connections 2024-09-13T18:14:30.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.618+0000 7fcab6340700 1 -- 172.21.15.40:0/2214873358 wait complete. 2024-09-13T18:14:30.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.622+0000 7fcab6340700 1 Processor -- start 2024-09-13T18:14:30.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.622+0000 7fcab6340700 1 -- start start 2024-09-13T18:14:30.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.622+0000 7fcab6340700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcab00cb2f0 0x7fcab015b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.622+0000 7fcab6340700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab00cf3c0 0x7fcab014fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.622+0000 7fcab6340700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab0153880 0x7fcab0150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.622+0000 7fcab6340700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab0156e00 con 0x7fcab00cf3c0 2024-09-13T18:14:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.622+0000 7fcab6340700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab0154920 con 0x7fcab0153880 2024-09-13T18:14:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.622+0000 7fcab6340700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab014db20 con 0x7fcab00cb2f0 2024-09-13T18:14:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.622+0000 7fcab4b3d700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab00cf3c0 msgr2=0x7fcab014fdd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:14:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.622+0000 7fcab4b3d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab00cf3c0 0x7fcab014fdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:14:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.622+0000 7fcab5b3f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab0153880 0x7fcab0150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.622+0000 7fcab533e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcab00cb2f0 0x7fcab015b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.622+0000 7fcab5b3f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab0153880 0x7fcab0150310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:53766/0 (socket says 172.21.15.40:53766) 2024-09-13T18:14:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.622+0000 7fcab5b3f700 1 -- 172.21.15.40:0/3729603471 learned_addr learned my addr 172.21.15.40:0/3729603471 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:30.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.626+0000 7fcab5b3f700 1 -- 172.21.15.40:0/3729603471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcab00cb2f0 msgr2=0x7fcab015b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.626+0000 7fcab5b3f700 1 --2- 172.21.15.40:0/3729603471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcab00cb2f0 0x7fcab015b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.626+0000 7fcab5b3f700 1 -- 172.21.15.40:0/3729603471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab00cf3c0 msgr2=0x7fcab014fdd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:14:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.626+0000 7fcab5b3f700 1 --2- 172.21.15.40:0/3729603471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab00cf3c0 0x7fcab014fdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.626+0000 7fcab5b3f700 1 -- 172.21.15.40:0/3729603471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa0009570 con 0x7fcab0153880 2024-09-13T18:14:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.626+0000 7fcab5b3f700 1 --2- 172.21.15.40:0/3729603471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab0153880 0x7fcab0150310 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800be60 tx=0x7fcaa8009f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.626+0000 7fcaa5ffb700 1 -- 172.21.15.40:0/3729603471 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa800cd10 con 0x7fcab0153880 2024-09-13T18:14:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.626+0000 7fcab6340700 1 -- 172.21.15.40:0/3729603471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab00c9660 con 0x7fcab0153880 2024-09-13T18:14:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.626+0000 7fcaa5ffb700 1 -- 172.21.15.40:0/3729603471 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa8016430 con 0x7fcab0153880 2024-09-13T18:14:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.626+0000 7fcaa5ffb700 1 -- 172.21.15.40:0/3729603471 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa80167e0 con 0x7fcab0153880 2024-09-13T18:14:30.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.626+0000 7fcab6340700 1 -- 172.21.15.40:0/3729603471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab0154e80 con 0x7fcab0153880 2024-09-13T18:14:30.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.626+0000 7fcab6340700 1 -- 172.21.15.40:0/3729603471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab014dd40 con 0x7fcab0153880 2024-09-13T18:14:30.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.626+0000 7fcaa5ffb700 1 -- 172.21.15.40:0/3729603471 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcaa8012020 con 0x7fcab0153880 2024-09-13T18:14:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.630+0000 7fcaa5ffb700 1 --2- 172.21.15.40:0/3729603471 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fca940666e0 0x7fca94068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.630+0000 7fcaa5ffb700 1 -- 172.21.15.40:0/3729603471 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcaa8084420 con 0x7fcab0153880 2024-09-13T18:14:30.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.630+0000 7fcab533e700 1 --2- 172.21.15.40:0/3729603471 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fca940666e0 0x7fca94068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:30.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.630+0000 7fcab533e700 1 --2- 172.21.15.40:0/3729603471 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fca940666e0 0x7fca94068ba0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009470 tx=0x7fcaac009400 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:30.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.638+0000 7fcaa5ffb700 1 -- 172.21.15.40:0/3729603471 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcab014dd40 con 0x7fcab0153880 2024-09-13T18:14:30.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.910+0000 7fcab6340700 1 -- 172.21.15.40:0/3729603471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fcab014dce0 con 0x7fcab0153880 2024-09-13T18:14:30.920 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.910+0000 7fcaa5ffb700 1 -- 172.21.15.40:0/3729603471 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7fcaa801b620 con 0x7fcab0153880 2024-09-13T18:14:30.921 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":192,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:14:30.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.914+0000 7fcab6340700 1 -- 172.21.15.40:0/3729603471 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fca940666e0 msgr2=0x7fca94068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.914+0000 7fcab6340700 1 --2- 172.21.15.40:0/3729603471 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fca940666e0 0x7fca94068ba0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009470 tx=0x7fcaac009400 comp rx=0 tx=0).stop 2024-09-13T18:14:30.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.914+0000 7fcab6340700 1 -- 172.21.15.40:0/3729603471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab0153880 msgr2=0x7fcab0150310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:30.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.914+0000 7fcab6340700 1 --2- 172.21.15.40:0/3729603471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab0153880 0x7fcab0150310 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800be60 tx=0x7fcaa8009f30 comp rx=0 tx=0).stop 2024-09-13T18:14:30.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.914+0000 7fcab6340700 1 -- 172.21.15.40:0/3729603471 shutdown_connections 2024-09-13T18:14:30.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.914+0000 7fcab6340700 1 --2- 172.21.15.40:0/3729603471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab0153880 0x7fcab0150310 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.914+0000 7fcab6340700 1 --2- 172.21.15.40:0/3729603471 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcab00cf3c0 0x7fcab014fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.914+0000 7fcab6340700 1 --2- 172.21.15.40:0/3729603471 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fca940666e0 0x7fca94068ba0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.914+0000 7fcab6340700 1 --2- 172.21.15.40:0/3729603471 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcab00cb2f0 0x7fcab015b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:30.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.914+0000 7fcab6340700 1 -- 172.21.15.40:0/3729603471 >> 172.21.15.40:0/3729603471 conn(0x7fcab006c5b0 msgr2=0x7fcab00ccd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:30.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.914+0000 7fcab6340700 1 -- 172.21.15.40:0/3729603471 shutdown_connections 2024-09-13T18:14:30.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:30.914+0000 7fcab6340700 1 -- 172.21.15.40:0/3729603471 wait complete. 2024-09-13T18:14:33.936 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:14:34.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.058+0000 7f1418d3d700 1 Processor -- start 2024-09-13T18:14:34.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.058+0000 7f1418d3d700 1 -- start start 2024-09-13T18:14:34.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.058+0000 7f1418d3d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f14140c5190 0x7f141415aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:34.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.058+0000 7f1418d3d700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14140c4870 con 0x7f14140c5570 2024-09-13T18:14:34.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.058+0000 7f1418d3d700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14140c7d10 con 0x7f14140c7260 2024-09-13T18:14:34.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.058+0000 7f1418d3d700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14140c3da0 con 0x7f14140c5190 2024-09-13T18:14:34.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.058+0000 7f1412ffd700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f14140c5570 legacy=0x7f1414150d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:14:34.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.058+0000 7f14137fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f14140c5190 0x7f141415aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:34.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.058+0000 7f14137fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f14140c5190 0x7f141415aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:51898/0 (socket says 172.21.15.40:51898) 2024-09-13T18:14:34.069 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.058+0000 7f14137fe700 1 -- 172.21.15.40:0/2438270321 learned_addr learned my addr 172.21.15.40:0/2438270321 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:34.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.058+0000 7f14137fe700 1 -- 172.21.15.40:0/2438270321 >> v1:172.21.15.43:6789/0 conn(0x7f14140c7260 legacy=0x7f14141619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:34.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.058+0000 7f14137fe700 1 -- 172.21.15.40:0/2438270321 >> v1:172.21.15.40:6789/0 conn(0x7f14140c5570 legacy=0x7f1414150d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:14:34.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.058+0000 7f14137fe700 1 -- 172.21.15.40:0/2438270321 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14140c7ff0 con 0x7f14140c5190 2024-09-13T18:14:34.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.058+0000 7f14137fe700 1 --2- 172.21.15.40:0/2438270321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14140c5190 0x7f141415aee0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f140c009e50 tx=0x7f140c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d898d90d53441511 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:34.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.058+0000 7f14127fc700 1 -- 172.21.15.40:0/2438270321 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c00e070 con 0x7f14140c5190 2024-09-13T18:14:34.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.058+0000 7f14127fc700 1 -- 172.21.15.40:0/2438270321 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c003a40 con 0x7f14140c5190 2024-09-13T18:14:34.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.058+0000 7f14127fc700 1 -- 172.21.15.40:0/2438270321 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c003df0 con 0x7f14140c5190 2024-09-13T18:14:34.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.062+0000 7f1418d3d700 1 -- 172.21.15.40:0/2438270321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14140c5190 msgr2=0x7f141415aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:34.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.062+0000 7f1418d3d700 1 --2- 172.21.15.40:0/2438270321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14140c5190 0x7f141415aee0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f140c009e50 tx=0x7f140c00b330 comp rx=0 tx=0).stop 2024-09-13T18:14:34.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.062+0000 7f1418d3d700 1 -- 172.21.15.40:0/2438270321 shutdown_connections 2024-09-13T18:14:34.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.062+0000 7f1418d3d700 1 --2- 172.21.15.40:0/2438270321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14140c5190 0x7f141415aee0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:34.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.062+0000 7f1418d3d700 1 -- 172.21.15.40:0/2438270321 >> 172.21.15.40:0/2438270321 conn(0x7f14140cb2e0 msgr2=0x7f14140cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:34.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.062+0000 7f1418d3d700 1 -- 172.21.15.40:0/2438270321 shutdown_connections 2024-09-13T18:14:34.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.062+0000 7f1418d3d700 1 -- 172.21.15.40:0/2438270321 wait complete. 2024-09-13T18:14:34.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.062+0000 7f1418d3d700 1 Processor -- start 2024-09-13T18:14:34.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.062+0000 7f1418d3d700 1 -- start start 2024-09-13T18:14:34.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f1418d3d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14140c5190 0x7f1414151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:34.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f1418d3d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14140c5570 0x7f1414156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:34.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f1418d3d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f14140c7260 0x7f1414145bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:34.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f1418d3d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14141ecc20 con 0x7f14140c5570 2024-09-13T18:14:34.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f1418d3d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14140c3780 con 0x7f14140c5190 2024-09-13T18:14:34.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f1418d3d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1414151c90 con 0x7f14140c7260 2024-09-13T18:14:34.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f1412ffd700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14140c5570 msgr2=0x7f1414156180 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:14:34.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f1412ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14140c5570 0x7f1414156180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:14:34.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f14137fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14140c5190 0x7f1414151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:34.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f14137fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14140c5190 0x7f1414151750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:51908/0 (socket says 172.21.15.40:51908) 2024-09-13T18:14:34.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f14137fe700 1 -- 172.21.15.40:0/219601605 learned_addr learned my addr 172.21.15.40:0/219601605 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:34.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f1413fff700 1 --2- 172.21.15.40:0/219601605 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f14140c7260 0x7f1414145bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:34.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f14137fe700 1 -- 172.21.15.40:0/219601605 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f14140c7260 msgr2=0x7f1414145bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:34.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f14137fe700 1 --2- 172.21.15.40:0/219601605 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f14140c7260 0x7f1414145bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:34.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f14137fe700 1 -- 172.21.15.40:0/219601605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14140c5570 msgr2=0x7f1414156180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:14:34.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f14137fe700 1 --2- 172.21.15.40:0/219601605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14140c5570 0x7f1414156180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:34.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f14137fe700 1 -- 172.21.15.40:0/219601605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140c009b30 con 0x7f14140c5190 2024-09-13T18:14:34.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f14137fe700 1 --2- 172.21.15.40:0/219601605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14140c5190 0x7f1414151750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f140c015040 tx=0x7f140c004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:34.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f140bfff700 1 -- 172.21.15.40:0/219601605 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c004440 con 0x7f14140c5190 2024-09-13T18:14:34.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f140bfff700 1 -- 172.21.15.40:0/219601605 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c0045c0 con 0x7f14140c5190 2024-09-13T18:14:34.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f1418d3d700 1 -- 172.21.15.40:0/219601605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14141498b0 con 0x7f14140c5190 2024-09-13T18:14:34.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f140bfff700 1 -- 172.21.15.40:0/219601605 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c004970 con 0x7f14140c5190 2024-09-13T18:14:34.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.066+0000 7f1418d3d700 1 -- 172.21.15.40:0/219601605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14141ed0f0 con 0x7f14140c5190 2024-09-13T18:14:34.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.070+0000 7f1418d3d700 1 -- 172.21.15.40:0/219601605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14141f01f0 con 0x7f14140c5190 2024-09-13T18:14:34.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.070+0000 7f140bfff700 1 -- 172.21.15.40:0/219601605 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f140c012430 con 0x7f14140c5190 2024-09-13T18:14:34.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.070+0000 7f140bfff700 1 --2- 172.21.15.40:0/219601605 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f13f4066710 0x7f13f4068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:34.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.070+0000 7f140bfff700 1 -- 172.21.15.40:0/219601605 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f140c0859a0 con 0x7f14140c5190 2024-09-13T18:14:34.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.070+0000 7f1412ffd700 1 --2- 172.21.15.40:0/219601605 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f13f4066710 0x7f13f4068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:34.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.074+0000 7f1412ffd700 1 --2- 172.21.15.40:0/219601605 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f13f4066710 0x7f13f4068bd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f141406b3e0 tx=0x7f13fc009270 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:34.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.082+0000 7f140bfff700 1 -- 172.21.15.40:0/219601605 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f140c052960 con 0x7f14140c5190 2024-09-13T18:14:34.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.346+0000 7f1418d3d700 1 -- 172.21.15.40:0/219601605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1414157050 con 0x7f14140c5190 2024-09-13T18:14:34.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.346+0000 7f140bfff700 1 -- 172.21.15.40:0/219601605 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f140c055f80 con 0x7f14140c5190 2024-09-13T18:14:34.358 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":192,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:14:34.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.350+0000 7f1418d3d700 1 -- 172.21.15.40:0/219601605 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f13f4066710 msgr2=0x7f13f4068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:34.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.350+0000 7f1418d3d700 1 --2- 172.21.15.40:0/219601605 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f13f4066710 0x7f13f4068bd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f141406b3e0 tx=0x7f13fc009270 comp rx=0 tx=0).stop 2024-09-13T18:14:34.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.350+0000 7f1418d3d700 1 -- 172.21.15.40:0/219601605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14140c5190 msgr2=0x7f1414151750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:34.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.350+0000 7f1418d3d700 1 --2- 172.21.15.40:0/219601605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14140c5190 0x7f1414151750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f140c015040 tx=0x7f140c004af0 comp rx=0 tx=0).stop 2024-09-13T18:14:34.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.350+0000 7f1418d3d700 1 -- 172.21.15.40:0/219601605 shutdown_connections 2024-09-13T18:14:34.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.350+0000 7f1418d3d700 1 --2- 172.21.15.40:0/219601605 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f13f4066710 0x7f13f4068bd0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:34.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.350+0000 7f1418d3d700 1 --2- 172.21.15.40:0/219601605 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f14140c7260 0x7f1414145bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:34.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.350+0000 7f1418d3d700 1 --2- 172.21.15.40:0/219601605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f14140c5570 0x7f1414156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:34.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.350+0000 7f1418d3d700 1 --2- 172.21.15.40:0/219601605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14140c5190 0x7f1414151750 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:34.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.350+0000 7f1418d3d700 1 -- 172.21.15.40:0/219601605 >> 172.21.15.40:0/219601605 conn(0x7f14140cb2e0 msgr2=0x7f14141575f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:34.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.350+0000 7f1418d3d700 1 -- 172.21.15.40:0/219601605 shutdown_connections 2024-09-13T18:14:34.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:34.350+0000 7f1418d3d700 1 -- 172.21.15.40:0/219601605 wait complete. 2024-09-13T18:14:37.375 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:14:37.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.486+0000 7f0ce358c700 1 Processor -- start 2024-09-13T18:14:37.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.486+0000 7f0ce358c700 1 -- start start 2024-09-13T18:14:37.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.486+0000 7f0ce358c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0cdc153880 0x7f0cdc0c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:37.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.486+0000 7f0ce358c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cdc15cf60 0x7f0cdc15d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:37.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.486+0000 7f0ce358c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cdc15d930 con 0x7f0cdc0cb2f0 2024-09-13T18:14:37.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.486+0000 7f0ce358c700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cdc159770 con 0x7f0cdc153880 2024-09-13T18:14:37.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.486+0000 7f0ce358c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc159a50 con 0x7f0cdc15cf60 2024-09-13T18:14:37.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.486+0000 7f0ce258a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0cdc153880 0x7f0cdc0c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:37.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.486+0000 7f0ce2d8b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cdc15cf60 0x7f0cdc15d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:37.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.486+0000 7f0ce258a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0cdc153880 0x7f0cdc0c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:51910/0 (socket says 172.21.15.40:51910) 2024-09-13T18:14:37.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.486+0000 7f0ce258a700 1 -- 172.21.15.40:0/3216125747 learned_addr learned my addr 172.21.15.40:0/3216125747 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:37.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.486+0000 7f0ce258a700 1 -- 172.21.15.40:0/3216125747 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cdc15cf60 msgr2=0x7f0cdc15d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:14:37.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.486+0000 7f0ce258a700 1 --2- 172.21.15.40:0/3216125747 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cdc15cf60 0x7f0cdc15d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:37.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.486+0000 7f0ce258a700 1 -- 172.21.15.40:0/3216125747 >> v1:172.21.15.43:6789/0 conn(0x7f0cdc0cb2f0 legacy=0x7f0cdc0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:37.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.486+0000 7f0ce258a700 1 -- 172.21.15.40:0/3216125747 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cdc159d30 con 0x7f0cdc153880 2024-09-13T18:14:37.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.486+0000 7f0ce258a700 1 --2- 172.21.15.40:0/3216125747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cdc153880 0x7f0cdc0c9ef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009910 tx=0x7f0cd8009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9c1b8f1d3a617af server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:37.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.490+0000 7f0ce1588700 1 -- 172.21.15.40:0/3216125747 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8011830 con 0x7f0cdc153880 2024-09-13T18:14:37.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.490+0000 7f0ce1588700 1 -- 172.21.15.40:0/3216125747 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd8011990 con 0x7f0cdc153880 2024-09-13T18:14:37.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.490+0000 7f0ce1588700 1 -- 172.21.15.40:0/3216125747 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8011d40 con 0x7f0cdc153880 2024-09-13T18:14:37.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.490+0000 7f0ce358c700 1 -- 172.21.15.40:0/3216125747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cdc153880 msgr2=0x7f0cdc0c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:37.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.490+0000 7f0ce358c700 1 --2- 172.21.15.40:0/3216125747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cdc153880 0x7f0cdc0c9ef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009910 tx=0x7f0cd8009c20 comp rx=0 tx=0).stop 2024-09-13T18:14:37.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.490+0000 7f0ce358c700 1 -- 172.21.15.40:0/3216125747 shutdown_connections 2024-09-13T18:14:37.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.490+0000 7f0ce358c700 1 --2- 172.21.15.40:0/3216125747 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cdc15cf60 0x7f0cdc15d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:37.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.490+0000 7f0ce358c700 1 --2- 172.21.15.40:0/3216125747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cdc153880 0x7f0cdc0c9ef0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:37.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.490+0000 7f0ce358c700 1 -- 172.21.15.40:0/3216125747 >> 172.21.15.40:0/3216125747 conn(0x7f0cdc06c5b0 msgr2=0x7f0cdc06c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:37.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.490+0000 7f0ce358c700 1 -- 172.21.15.40:0/3216125747 shutdown_connections 2024-09-13T18:14:37.500 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.490+0000 7f0ce358c700 1 -- 172.21.15.40:0/3216125747 wait complete. 2024-09-13T18:14:37.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.490+0000 7f0ce358c700 1 Processor -- start 2024-09-13T18:14:37.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.490+0000 7f0ce358c700 1 -- start start 2024-09-13T18:14:37.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce358c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cdc0cb2f0 0x7f0cdc155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:37.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce358c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cdc153880 0x7f0cdc157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:37.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce358c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cdc15cf60 0x7f0cdc149c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:37.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce358c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc1581f0 con 0x7f0cdc153880 2024-09-13T18:14:37.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce358c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc14a190 con 0x7f0cdc15cf60 2024-09-13T18:14:37.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce358c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc1f2b70 con 0x7f0cdc0cb2f0 2024-09-13T18:14:37.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce1d89700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cdc153880 msgr2=0x7f0cdc157cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:14:37.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce1d89700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cdc153880 0x7f0cdc157cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:14:37.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce2d8b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cdc15cf60 0x7f0cdc149c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:37.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce258a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cdc0cb2f0 0x7f0cdc155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:37.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce2d8b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cdc15cf60 0x7f0cdc149c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:51920/0 (socket says 172.21.15.40:51920) 2024-09-13T18:14:37.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce2d8b700 1 -- 172.21.15.40:0/4182432906 learned_addr learned my addr 172.21.15.40:0/4182432906 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:37.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce258a700 1 -- 172.21.15.40:0/4182432906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cdc15cf60 msgr2=0x7f0cdc149c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:37.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce258a700 1 --2- 172.21.15.40:0/4182432906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cdc15cf60 0x7f0cdc149c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:37.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce258a700 1 -- 172.21.15.40:0/4182432906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cdc153880 msgr2=0x7f0cdc157cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:14:37.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce258a700 1 --2- 172.21.15.40:0/4182432906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cdc153880 0x7f0cdc157cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:37.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce258a700 1 -- 172.21.15.40:0/4182432906 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd8009570 con 0x7f0cdc0cb2f0 2024-09-13T18:14:37.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce2d8b700 1 --2- 172.21.15.40:0/4182432906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cdc15cf60 0x7f0cdc149c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:14:37.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce258a700 1 --2- 172.21.15.40:0/4182432906 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cdc0cb2f0 0x7f0cdc155bd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8016000 tx=0x7f0cd8004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:37.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0cd2ffd700 1 -- 172.21.15.40:0/4182432906 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8004380 con 0x7f0cdc0cb2f0 2024-09-13T18:14:37.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce358c700 1 -- 172.21.15.40:0/4182432906 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cdc1f2890 con 0x7f0cdc0cb2f0 2024-09-13T18:14:37.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0cd2ffd700 1 -- 172.21.15.40:0/4182432906 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd801f070 con 0x7f0cdc0cb2f0 2024-09-13T18:14:37.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0cd2ffd700 1 -- 172.21.15.40:0/4182432906 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd80036c0 con 0x7f0cdc0cb2f0 2024-09-13T18:14:37.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce358c700 1 -- 172.21.15.40:0/4182432906 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cdc154280 con 0x7f0cdc0cb2f0 2024-09-13T18:14:37.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.494+0000 7f0ce358c700 1 -- 172.21.15.40:0/4182432906 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cdc0ca740 con 0x7f0cdc0cb2f0 2024-09-13T18:14:37.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.498+0000 7f0cd2ffd700 1 -- 172.21.15.40:0/4182432906 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0cd8021030 con 0x7f0cdc0cb2f0 2024-09-13T18:14:37.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.498+0000 7f0cd2ffd700 1 --2- 172.21.15.40:0/4182432906 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0cc0066710 0x7f0cc0068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:37.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.498+0000 7f0cd2ffd700 1 -- 172.21.15.40:0/4182432906 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0cd8085900 con 0x7f0cdc0cb2f0 2024-09-13T18:14:37.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.498+0000 7f0ce1d89700 1 --2- 172.21.15.40:0/4182432906 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0cc0066710 0x7f0cc0068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:37.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.498+0000 7f0ce1d89700 1 --2- 172.21.15.40:0/4182432906 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0cc0066710 0x7f0cc0068bd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc153070 tx=0x7f0ccc007a80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:37.518 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.510+0000 7f0cd2ffd700 1 -- 172.21.15.40:0/4182432906 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0cd8052830 con 0x7f0cdc0cb2f0 2024-09-13T18:14:37.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.798+0000 7f0ce358c700 1 -- 172.21.15.40:0/4182432906 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0cdc0cf910 con 0x7f0cdc0cb2f0 2024-09-13T18:14:37.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.798+0000 7f0cd2ffd700 1 -- 172.21.15.40:0/4182432906 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f0cd8055e50 con 0x7f0cdc0cb2f0 2024-09-13T18:14:37.809 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":192,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:14:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.802+0000 7f0ce358c700 1 -- 172.21.15.40:0/4182432906 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0cc0066710 msgr2=0x7f0cc0068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.802+0000 7f0ce358c700 1 --2- 172.21.15.40:0/4182432906 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0cc0066710 0x7f0cc0068bd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc153070 tx=0x7f0ccc007a80 comp rx=0 tx=0).stop 2024-09-13T18:14:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.802+0000 7f0ce358c700 1 -- 172.21.15.40:0/4182432906 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cdc0cb2f0 msgr2=0x7f0cdc155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.802+0000 7f0ce358c700 1 --2- 172.21.15.40:0/4182432906 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cdc0cb2f0 0x7f0cdc155bd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8016000 tx=0x7f0cd8004770 comp rx=0 tx=0).stop 2024-09-13T18:14:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.802+0000 7f0ce358c700 1 -- 172.21.15.40:0/4182432906 shutdown_connections 2024-09-13T18:14:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.802+0000 7f0ce358c700 1 --2- 172.21.15.40:0/4182432906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cdc15cf60 0x7f0cdc149c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.802+0000 7f0ce358c700 1 --2- 172.21.15.40:0/4182432906 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cdc153880 0x7f0cdc157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.802+0000 7f0ce358c700 1 --2- 172.21.15.40:0/4182432906 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0cc0066710 0x7f0cc0068bd0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.802+0000 7f0ce358c700 1 --2- 172.21.15.40:0/4182432906 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0cdc0cb2f0 0x7f0cdc155bd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.802+0000 7f0ce358c700 1 -- 172.21.15.40:0/4182432906 >> 172.21.15.40:0/4182432906 conn(0x7f0cdc06c5b0 msgr2=0x7f0cdc14fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.802+0000 7f0ce358c700 1 -- 172.21.15.40:0/4182432906 shutdown_connections 2024-09-13T18:14:37.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:37.802+0000 7f0ce358c700 1 -- 172.21.15.40:0/4182432906 wait complete. 2024-09-13T18:14:37.917 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:14:37.919 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:14:40.826 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:14:40.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:40.942+0000 7f8b7b59e700 1 Processor -- start 2024-09-13T18:14:40.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:40.942+0000 7f8b7b59e700 1 -- start start 2024-09-13T18:14:40.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:40.942+0000 7f8b7b59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8b7c0c7880 0x7f8b7c06f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:40.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:40.942+0000 7f8b7b59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7c0c8680 con 0x7f8b7c153990 2024-09-13T18:14:40.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:40.942+0000 7f8b7b59e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7c0c8960 con 0x7f8b7c06ee30 2024-09-13T18:14:40.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:40.942+0000 7f8b7b59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b7c0c40b0 con 0x7f8b7c0c7880 2024-09-13T18:14:40.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:40.942+0000 7f8b7a59c700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f8b7c153990 legacy=0x7f8b7c06b4a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:14:40.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:40.942+0000 7f8b7ad9d700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f8b7c0c7880 msgr2=0x7f8b7c06f5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:14:40.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:40.942+0000 7f8b7ad9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8b7c0c7880 0x7f8b7c06f5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:14:40.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:40.942+0000 7f8b73fff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8b7c06ee30 0x7f8b7c161d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:59088/0 (socket says 172.21.15.40:59088) 2024-09-13T18:14:40.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:40.942+0000 7f8b73fff700 1 -- 172.21.15.40:0/4218468950 learned_addr learned my addr 172.21.15.40:0/4218468950 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:14:41.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:41.146+0000 7f8b7a59c700 1 -- 172.21.15.40:0/4218468950 >> v1:172.21.15.40:6789/0 conn(0x7f8b7c153990 legacy=0x7f8b7c06b4a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:14:41.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:41.146+0000 7f8b7ad9d700 1 -- 172.21.15.40:0/4218468950 >> v2:172.21.15.40:3300/0 conn(0x7f8b7c0c7880 msgr2=0x7f8b7c06f5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:14:41.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:41.146+0000 7f8b7ad9d700 1 --2- 172.21.15.40:0/4218468950 >> v2:172.21.15.40:3300/0 conn(0x7f8b7c0c7880 0x7f8b7c06f5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:14:41.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:41.549+0000 7f8b7a59c700 1 -- 172.21.15.40:0/4218468950 >> v1:172.21.15.40:6789/0 conn(0x7f8b7c153990 legacy=0x7f8b7c06b4a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:14:41.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:41.549+0000 7f8b7ad9d700 1 -- 172.21.15.40:0/4218468950 >> v2:172.21.15.40:3300/0 conn(0x7f8b7c0c7880 msgr2=0x7f8b7c06f5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:14:41.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:41.549+0000 7f8b7ad9d700 1 --2- 172.21.15.40:0/4218468950 >> v2:172.21.15.40:3300/0 conn(0x7f8b7c0c7880 0x7f8b7c06f5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:14:42.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:42.353+0000 7f8b7a59c700 1 -- 172.21.15.40:0/4218468950 >> v1:172.21.15.40:6789/0 conn(0x7f8b7c153990 legacy=0x7f8b7c06b4a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:14:42.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:42.353+0000 7f8b7ad9d700 1 -- 172.21.15.40:0/4218468950 >> v2:172.21.15.40:3300/0 conn(0x7f8b7c0c7880 msgr2=0x7f8b7c06f5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:14:42.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:42.353+0000 7f8b7ad9d700 1 --2- 172.21.15.40:0/4218468950 >> v2:172.21.15.40:3300/0 conn(0x7f8b7c0c7880 0x7f8b7c06f5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:14:42.807 INFO:tasks.workunit.client.0.smithi040.stdout:fs setup took 1244 secs 2024-09-13T18:14:43.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:43.941+0000 7f8b78d99700 1 -- 172.21.15.40:0/4218468950 >> v2:172.21.15.40:3300/0 conn(0x7f8b7c0c7880 msgr2=0x7f8b7c06f5a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:14:43.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:43.941+0000 7f8b78d99700 1 --2- 172.21.15.40:0/4218468950 >> v2:172.21.15.40:3300/0 conn(0x7f8b7c0c7880 0x7f8b7c06f5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:43.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:43.941+0000 7f8b78d99700 1 -- 172.21.15.40:0/4218468950 >> v1:172.21.15.43:6789/0 conn(0x7f8b7c06ee30 legacy=0x7f8b7c161d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:43.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:43.941+0000 7f8b78d99700 1 -- 172.21.15.40:0/4218468950 >> v1:172.21.15.40:6789/0 conn(0x7f8b7c153990 legacy=0x7f8b7c06b4a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:14:43.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:43.941+0000 7f8b78d99700 1 --2- 172.21.15.40:0/4218468950 >> v2:172.21.15.43:3300/0 conn(0x7f8b5c003740 0x7f8b5c005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:43.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:43.941+0000 7f8b78d99700 1 --2- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b5c006240 0x7f8b5c0086e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:43.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:43.941+0000 7f8b78d99700 1 -- 172.21.15.40:0/4218468950 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b7c0c40b0 con 0x7f8b5c003740 2024-09-13T18:14:43.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:43.941+0000 7f8b78d99700 1 -- 172.21.15.40:0/4218468950 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c008e00 con 0x7f8b5c006240 2024-09-13T18:14:43.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:43.941+0000 7f8b7a59c700 1 --2- 172.21.15.40:0/4218468950 >> v2:172.21.15.43:3300/0 conn(0x7f8b5c003740 0x7f8b5c005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:43.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:43.941+0000 7f8b73fff700 1 --2- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b5c006240 0x7f8b5c0086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:43.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:43.941+0000 7f8b7a59c700 1 -- 172.21.15.40:0/4218468950 >> v2:172.21.15.43:3300/0 conn(0x7f8b5c003740 msgr2=0x7f8b5c005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:14:43.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:43.941+0000 7f8b7a59c700 1 -- 172.21.15.40:0/4218468950 >> v2:172.21.15.43:3300/0 conn(0x7f8b5c003740 msgr2=0x7f8b5c005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:14:43.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:43.941+0000 7f8b7a59c700 1 --2- 172.21.15.40:0/4218468950 >> v2:172.21.15.43:3300/0 conn(0x7f8b5c003740 0x7f8b5c005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:14:43.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:43.941+0000 7f8b7a59c700 1 --2- 172.21.15.40:0/4218468950 >> v2:172.21.15.43:3300/0 conn(0x7f8b5c003740 0x7f8b5c005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:14:43.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:43.941+0000 7f8b73fff700 1 -- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b5c006240 msgr2=0x7f8b5c0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:14:43.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:43.941+0000 7f8b73fff700 1 -- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b5c006240 msgr2=0x7f8b5c0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:14:43.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:43.941+0000 7f8b73fff700 1 --2- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b5c006240 0x7f8b5c0086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:14:43.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:43.941+0000 7f8b73fff700 1 --2- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b5c006240 0x7f8b5c0086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:14:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:44.145+0000 7f8b7a59c700 1 --2- 172.21.15.40:0/4218468950 >> v2:172.21.15.43:3300/0 conn(0x7f8b5c003740 0x7f8b5c005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:44.145+0000 7f8b73fff700 1 --2- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b5c006240 0x7f8b5c0086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:44.145+0000 7f8b7a59c700 1 -- 172.21.15.40:0/4218468950 >> v2:172.21.15.43:3300/0 conn(0x7f8b5c003740 msgr2=0x7f8b5c005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:14:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:44.145+0000 7f8b7a59c700 1 -- 172.21.15.40:0/4218468950 >> v2:172.21.15.43:3300/0 conn(0x7f8b5c003740 msgr2=0x7f8b5c005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:14:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:44.145+0000 7f8b7a59c700 1 --2- 172.21.15.40:0/4218468950 >> v2:172.21.15.43:3300/0 conn(0x7f8b5c003740 0x7f8b5c005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:14:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:44.145+0000 7f8b7a59c700 1 --2- 172.21.15.40:0/4218468950 >> v2:172.21.15.43:3300/0 conn(0x7f8b5c003740 0x7f8b5c005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:14:44.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:44.145+0000 7f8b73fff700 1 -- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b5c006240 msgr2=0x7f8b5c0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:14:44.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:44.145+0000 7f8b73fff700 1 -- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b5c006240 msgr2=0x7f8b5c0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:14:44.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:44.145+0000 7f8b73fff700 1 --2- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b5c006240 0x7f8b5c0086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:14:44.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:44.145+0000 7f8b73fff700 1 --2- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b5c006240 0x7f8b5c0086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:14:44.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:44.549+0000 7f8b7a59c700 1 --2- 172.21.15.40:0/4218468950 >> v2:172.21.15.43:3300/0 conn(0x7f8b5c003740 0x7f8b5c005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:44.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:44.549+0000 7f8b73fff700 1 --2- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b5c006240 0x7f8b5c0086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:44.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:44.549+0000 7f8b7a59c700 1 -- 172.21.15.40:0/4218468950 >> v2:172.21.15.43:3300/0 conn(0x7f8b5c003740 msgr2=0x7f8b5c005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:14:44.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:44.549+0000 7f8b7a59c700 1 -- 172.21.15.40:0/4218468950 >> v2:172.21.15.43:3300/0 conn(0x7f8b5c003740 msgr2=0x7f8b5c005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:14:44.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:44.549+0000 7f8b7a59c700 1 --2- 172.21.15.40:0/4218468950 >> v2:172.21.15.43:3300/0 conn(0x7f8b5c003740 0x7f8b5c005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:14:44.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:44.549+0000 7f8b7a59c700 1 --2- 172.21.15.40:0/4218468950 >> v2:172.21.15.43:3300/0 conn(0x7f8b5c003740 0x7f8b5c005bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:14:44.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:44.549+0000 7f8b73fff700 1 -- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b5c006240 msgr2=0x7f8b5c0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:14:44.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:44.549+0000 7f8b73fff700 1 -- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b5c006240 msgr2=0x7f8b5c0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:14:44.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:44.549+0000 7f8b73fff700 1 --2- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b5c006240 0x7f8b5c0086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:14:44.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:44.549+0000 7f8b73fff700 1 --2- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b5c006240 0x7f8b5c0086e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:14:44.601 INFO:tasks.workunit.client.0.smithi040.stdout:Syncing()...1 sec 2024-09-13T18:14:44.603 INFO:tasks.workunit.client.0.smithi040.stdout:Starting Actual Benchmark At: Fri Sep 13 18:14:44 2024 2024-09-13T18:14:44.603 INFO:tasks.workunit.client.0.smithi040.stdout: 2024-09-13T18:14:45.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.353+0000 7f8b7a59c700 1 --2- 172.21.15.40:0/4218468950 >> v2:172.21.15.43:3300/0 conn(0x7f8b5c003740 0x7f8b5c005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:45.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.353+0000 7f8b73fff700 1 --2- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b5c006240 0x7f8b5c0086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:45.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.353+0000 7f8b7a59c700 1 -- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b5c006240 msgr2=0x7f8b5c0086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:14:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.353+0000 7f8b7a59c700 1 --2- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b5c006240 0x7f8b5c0086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.353+0000 7f8b7a59c700 1 -- 172.21.15.40:0/4218468950 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c0c8c40 con 0x7f8b5c003740 2024-09-13T18:14:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.353+0000 7f8b73fff700 1 --2- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b5c006240 0x7f8b5c0086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:14:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.353+0000 7f8b7a59c700 1 --2- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b5c003740 0x7f8b5c005bc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c003a30 tx=0x7f8b6c00b6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=180ec89404ea111c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:45.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.353+0000 7f8b79d9b700 1 -- 172.21.15.40:0/4218468950 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c016610 con 0x7f8b5c003740 2024-09-13T18:14:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.353+0000 7f8b79d9b700 1 -- 172.21.15.40:0/4218468950 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6c011050 con 0x7f8b5c003740 2024-09-13T18:14:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.353+0000 7f8b79d9b700 1 -- 172.21.15.40:0/4218468950 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c016990 con 0x7f8b5c003740 2024-09-13T18:14:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.353+0000 7f8b7b59e700 1 -- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b5c003740 msgr2=0x7f8b5c005bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.353+0000 7f8b7b59e700 1 --2- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b5c003740 0x7f8b5c005bc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c003a30 tx=0x7f8b6c00b6d0 comp rx=0 tx=0).stop 2024-09-13T18:14:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.353+0000 7f8b7ad9d700 1 -- 172.21.15.40:0/4218468950 reap_dead start 2024-09-13T18:14:45.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.353+0000 7f8b7b59e700 1 -- 172.21.15.40:0/4218468950 shutdown_connections 2024-09-13T18:14:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.353+0000 7f8b7b59e700 1 --2- 172.21.15.40:0/4218468950 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b5c003740 0x7f8b5c005bc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.353+0000 7f8b7b59e700 1 -- 172.21.15.40:0/4218468950 >> 172.21.15.40:0/4218468950 conn(0x7f8b7c066090 msgr2=0x7f8b7c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.353+0000 7f8b7b59e700 1 -- 172.21.15.40:0/4218468950 shutdown_connections 2024-09-13T18:14:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.353+0000 7f8b7b59e700 1 -- 172.21.15.40:0/4218468950 wait complete. 2024-09-13T18:14:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b7b59e700 1 Processor -- start 2024-09-13T18:14:45.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b7b59e700 1 -- start start 2024-09-13T18:14:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b7b59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b5c003740 0x7f8b7c15c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b7b59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b7c14def0 0x7f8b7c0cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b7b59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b7c14c970 0x7f8b7c14ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b7b59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c15bd40 con 0x7f8b5c003740 2024-09-13T18:14:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b7b59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c14d340 con 0x7f8b7c14c970 2024-09-13T18:14:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b7b59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7c15ceb0 con 0x7f8b7c14def0 2024-09-13T18:14:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b7a59c700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b5c003740 msgr2=0x7f8b7c15c740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:14:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b7a59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b5c003740 0x7f8b7c15c740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:14:45.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b7ad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b7c14c970 0x7f8b7c14ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b73fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b7c14def0 0x7f8b7c0cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b7ad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b7c14c970 0x7f8b7c14ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:57474/0 (socket says 172.21.15.40:57474) 2024-09-13T18:14:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b7ad9d700 1 -- 172.21.15.40:0/192523928 learned_addr learned my addr 172.21.15.40:0/192523928 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b73fff700 1 -- 172.21.15.40:0/192523928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b7c14c970 msgr2=0x7f8b7c14ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b73fff700 1 --2- 172.21.15.40:0/192523928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b7c14c970 0x7f8b7c14ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b73fff700 1 -- 172.21.15.40:0/192523928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b5c003740 msgr2=0x7f8b7c15c740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:14:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b73fff700 1 --2- 172.21.15.40:0/192523928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b5c003740 0x7f8b7c15c740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b73fff700 1 -- 172.21.15.40:0/192523928 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6c009a20 con 0x7f8b7c14def0 2024-09-13T18:14:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b73fff700 1 --2- 172.21.15.40:0/192523928 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b7c14def0 0x7f8b7c0cc450 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8b64001260 tx=0x7f8b64001570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b72ffd700 1 -- 172.21.15.40:0/192523928 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6401c030 con 0x7f8b7c14def0 2024-09-13T18:14:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b72ffd700 1 -- 172.21.15.40:0/192523928 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b64004890 con 0x7f8b7c14def0 2024-09-13T18:14:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b72ffd700 1 -- 172.21.15.40:0/192523928 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b64004c00 con 0x7f8b7c14def0 2024-09-13T18:14:45.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b7b59e700 1 -- 172.21.15.40:0/192523928 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7c14e470 con 0x7f8b7c14def0 2024-09-13T18:14:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b7b59e700 1 -- 172.21.15.40:0/192523928 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7c159eb0 con 0x7f8b7c14def0 2024-09-13T18:14:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.357+0000 7f8b7b59e700 1 -- 172.21.15.40:0/192523928 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7c152c40 con 0x7f8b7c14def0 2024-09-13T18:14:45.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.361+0000 7f8b72ffd700 1 -- 172.21.15.40:0/192523928 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8b6401f070 con 0x7f8b7c14def0 2024-09-13T18:14:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.361+0000 7f8b72ffd700 1 --2- 172.21.15.40:0/192523928 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8b5c067610 0x7f8b5c069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.361+0000 7f8b72ffd700 1 -- 172.21.15.40:0/192523928 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8b6401d070 con 0x7f8b7c14def0 2024-09-13T18:14:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.361+0000 7f8b7a59c700 1 --2- 172.21.15.40:0/192523928 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8b5c067610 0x7f8b5c069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:45.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.361+0000 7f8b7a59c700 1 --2- 172.21.15.40:0/192523928 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8b5c067610 0x7f8b5c069ad0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00b710 tx=0x7f8b6c00d030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:45.376 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:45.365+0000 7f8b72ffd700 1 -- 172.21.15.40:0/192523928 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8b64051920 con 0x7f8b7c14def0 2024-09-13T18:14:46.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.005+0000 7f8b7b59e700 1 -- 172.21.15.40:0/192523928 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8b7c0c8680 con 0x7f8b7c14def0 2024-09-13T18:14:46.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.005+0000 7f8b72ffd700 1 -- 172.21.15.40:0/192523928 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f8b64054f40 con 0x7f8b7c14def0 2024-09-13T18:14:46.018 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":194,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:14:46.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.013+0000 7f8b7b59e700 1 -- 172.21.15.40:0/192523928 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8b5c067610 msgr2=0x7f8b5c069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:46.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.013+0000 7f8b7b59e700 1 --2- 172.21.15.40:0/192523928 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8b5c067610 0x7f8b5c069ad0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c00b710 tx=0x7f8b6c00d030 comp rx=0 tx=0).stop 2024-09-13T18:14:46.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.013+0000 7f8b7b59e700 1 -- 172.21.15.40:0/192523928 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b7c14def0 msgr2=0x7f8b7c0cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:46.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.013+0000 7f8b7b59e700 1 --2- 172.21.15.40:0/192523928 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b7c14def0 0x7f8b7c0cc450 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8b64001260 tx=0x7f8b64001570 comp rx=0 tx=0).stop 2024-09-13T18:14:46.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.013+0000 7f8b7b59e700 1 -- 172.21.15.40:0/192523928 shutdown_connections 2024-09-13T18:14:46.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.013+0000 7f8b7b59e700 1 --2- 172.21.15.40:0/192523928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b7c14c970 0x7f8b7c14ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:46.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.013+0000 7f8b7b59e700 1 --2- 172.21.15.40:0/192523928 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8b5c067610 0x7f8b5c069ad0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:46.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.013+0000 7f8b7b59e700 1 --2- 172.21.15.40:0/192523928 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8b7c14def0 0x7f8b7c0cc450 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:46.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.013+0000 7f8b7b59e700 1 --2- 172.21.15.40:0/192523928 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b5c003740 0x7f8b7c15c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:46.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.013+0000 7f8b7b59e700 1 -- 172.21.15.40:0/192523928 >> 172.21.15.40:0/192523928 conn(0x7f8b7c066090 msgr2=0x7f8b7c151d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:46.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.013+0000 7f8b7b59e700 1 -- 172.21.15.40:0/192523928 shutdown_connections 2024-09-13T18:14:46.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.013+0000 7f8b7b59e700 1 -- 172.21.15.40:0/192523928 wait complete. 2024-09-13T18:14:46.063 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T18:14:46.063 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-13T18:14:46.063 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:14:46.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.349+0000 7f604eb4b700 1 Processor -- start 2024-09-13T18:14:46.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.349+0000 7f604eb4b700 1 -- start start 2024-09-13T18:14:46.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.353+0000 7f604eb4b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f604814f8c0 0x7f60480c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:46.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.353+0000 7f604eb4b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60480724e0 0x7f604806d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:46.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.353+0000 7f604eb4b700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60480c6890 con 0x7f6048158bc0 2024-09-13T18:14:46.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.353+0000 7f604eb4b700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60480c7700 con 0x7f604814f8c0 2024-09-13T18:14:46.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.353+0000 7f604eb4b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60480c5f30 con 0x7f60480724e0 2024-09-13T18:14:46.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.353+0000 7f6047fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60480724e0 0x7f604806d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:46.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.353+0000 7f6047fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60480724e0 0x7f604806d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:56616/0 (socket says 172.21.15.40:56616) 2024-09-13T18:14:46.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.353+0000 7f6047fff700 1 -- 172.21.15.40:0/381994966 learned_addr learned my addr 172.21.15.40:0/381994966 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:46.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.353+0000 7f604d0e8700 1 --1- 172.21.15.40:0/381994966 >> v1:172.21.15.43:6789/0 conn(0x7f6048158bc0 0x7f604806a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:38834/0 (socket says 172.21.15.40:38834) 2024-09-13T18:14:46.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.353+0000 7f604c8e7700 1 --2- 172.21.15.40:0/381994966 >> v2:172.21.15.43:3300/0 conn(0x7f604814f8c0 0x7f60480c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:46.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.353+0000 7f6047fff700 1 -- 172.21.15.40:0/381994966 >> v2:172.21.15.43:3300/0 conn(0x7f604814f8c0 msgr2=0x7f60480c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:14:46.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.353+0000 7f6047fff700 1 --2- 172.21.15.40:0/381994966 >> v2:172.21.15.43:3300/0 conn(0x7f604814f8c0 0x7f60480c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:46.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.353+0000 7f6047fff700 1 -- 172.21.15.40:0/381994966 >> v1:172.21.15.43:6789/0 conn(0x7f6048158bc0 legacy=0x7f604806a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:46.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.353+0000 7f6047fff700 1 -- 172.21.15.40:0/381994966 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60480c1320 con 0x7f60480724e0 2024-09-13T18:14:46.373 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.361+0000 7f6047fff700 1 --2- 172.21.15.40:0/381994966 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60480724e0 0x7f604806d2d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6040010520 tx=0x7f6040010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ed0c7603e6271d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:46.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.365+0000 7f60477fe700 1 -- 172.21.15.40:0/381994966 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6040003c40 con 0x7f60480724e0 2024-09-13T18:14:46.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.365+0000 7f60477fe700 1 -- 172.21.15.40:0/381994966 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6040003da0 con 0x7f60480724e0 2024-09-13T18:14:46.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.365+0000 7f60477fe700 1 -- 172.21.15.40:0/381994966 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604000a520 con 0x7f60480724e0 2024-09-13T18:14:46.377 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.365+0000 7f604eb4b700 1 -- 172.21.15.40:0/381994966 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60480724e0 msgr2=0x7f604806d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:46.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.365+0000 7f604eb4b700 1 --2- 172.21.15.40:0/381994966 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60480724e0 0x7f604806d2d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6040010520 tx=0x7f6040010830 comp rx=0 tx=0).stop 2024-09-13T18:14:46.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.369+0000 7f604eb4b700 1 -- 172.21.15.40:0/381994966 shutdown_connections 2024-09-13T18:14:46.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.369+0000 7f604eb4b700 1 --2- 172.21.15.40:0/381994966 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f60480724e0 0x7f604806d2d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:46.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.369+0000 7f604eb4b700 1 --2- 172.21.15.40:0/381994966 >> v2:172.21.15.43:3300/0 conn(0x7f604814f8c0 0x7f60480c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:46.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.369+0000 7f604eb4b700 1 -- 172.21.15.40:0/381994966 >> 172.21.15.40:0/381994966 conn(0x7f6048066090 msgr2=0x7f60480664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:46.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.369+0000 7f604eb4b700 1 -- 172.21.15.40:0/381994966 shutdown_connections 2024-09-13T18:14:46.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.369+0000 7f604eb4b700 1 -- 172.21.15.40:0/381994966 wait complete. 2024-09-13T18:14:46.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.373+0000 7f604eb4b700 1 Processor -- start 2024-09-13T18:14:46.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.373+0000 7f604eb4b700 1 -- start start 2024-09-13T18:14:46.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.373+0000 7f604eb4b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60480724e0 0x7f60480c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:46.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.373+0000 7f604eb4b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f604814f8c0 0x7f6048153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:46.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.373+0000 7f604eb4b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6048158bc0 0x7f6048153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:46.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.373+0000 7f604d0e8700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6048158bc0 msgr2=0x7f6048153e60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:14:46.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.373+0000 7f604d0e8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6048158bc0 0x7f6048153e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:14:46.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.373+0000 7f604eb4b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6048155910 con 0x7f6048158bc0 2024-09-13T18:14:46.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.373+0000 7f604eb4b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60480715b0 con 0x7f60480724e0 2024-09-13T18:14:46.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.373+0000 7f604eb4b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60480712d0 con 0x7f604814f8c0 2024-09-13T18:14:46.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.377+0000 7f604c8e7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60480724e0 0x7f60480c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:46.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.377+0000 7f604c8e7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60480724e0 0x7f60480c9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:57478/0 (socket says 172.21.15.40:57478) 2024-09-13T18:14:46.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.377+0000 7f604c8e7700 1 -- 172.21.15.40:0/306819152 learned_addr learned my addr 172.21.15.40:0/306819152 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:46.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.377+0000 7f6047fff700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f604814f8c0 0x7f6048153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:46.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.377+0000 7f604c8e7700 1 -- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f604814f8c0 msgr2=0x7f6048153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:46.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.377+0000 7f604c8e7700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f604814f8c0 0x7f6048153920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:46.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.377+0000 7f604c8e7700 1 -- 172.21.15.40:0/306819152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6048158bc0 msgr2=0x7f6048153e60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:14:46.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.377+0000 7f604c8e7700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6048158bc0 0x7f6048153e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:46.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.377+0000 7f604c8e7700 1 -- 172.21.15.40:0/306819152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6040007790 con 0x7f60480724e0 2024-09-13T18:14:46.392 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.377+0000 7f604c8e7700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60480724e0 0x7f60480c9020 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f603c00e9d0 tx=0x7f603c00ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:46.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.381+0000 7f6047fff700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f604814f8c0 0x7f6048153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:14:46.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.385+0000 7f60457fa700 1 -- 172.21.15.40:0/306819152 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c017c60 con 0x7f60480724e0 2024-09-13T18:14:46.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.385+0000 7f60457fa700 1 -- 172.21.15.40:0/306819152 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603c017dc0 con 0x7f60480724e0 2024-09-13T18:14:46.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.385+0000 7f604eb4b700 1 -- 172.21.15.40:0/306819152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6048154e70 con 0x7f60480724e0 2024-09-13T18:14:46.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.385+0000 7f604eb4b700 1 -- 172.21.15.40:0/306819152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6048155430 con 0x7f60480724e0 2024-09-13T18:14:46.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.389+0000 7f602effd700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6020002a40 0x7f6020004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:46.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.389+0000 7f602effd700 1 -- 172.21.15.40:0/306819152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6020007f40 con 0x7f6020002a40 2024-09-13T18:14:46.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.389+0000 7f6047fff700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6020002a40 0x7f6020004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:46.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.389+0000 7f60457fa700 1 -- 172.21.15.40:0/306819152 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c00c840 con 0x7f60480724e0 2024-09-13T18:14:46.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.389+0000 7f60457fa700 1 -- 172.21.15.40:0/306819152 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f603c00c9e0 con 0x7f60480724e0 2024-09-13T18:14:46.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.389+0000 7f6047fff700 1 -- 172.21.15.40:0/306819152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6020008100 con 0x7f6020002a40 2024-09-13T18:14:46.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.389+0000 7f6047fff700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6020002a40 0x7f6020004f00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f60480c0580 tx=0x7f6040004b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:46.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.389+0000 7f60457fa700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f60340667b0 0x7f6034068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:46.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.389+0000 7f604d0e8700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f60340667b0 0x7f6034068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:46.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.389+0000 7f60457fa700 1 -- 172.21.15.40:0/306819152 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f603c08c9d0 con 0x7f60480724e0 2024-09-13T18:14:46.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.389+0000 7f60457fa700 1 -- 172.21.15.40:0/306819152 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6040010040 con 0x7f6020002a40 2024-09-13T18:14:46.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.389+0000 7f60457fa700 1 -- 172.21.15.40:0/306819152 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f604001e070 con 0x7f6020002a40 2024-09-13T18:14:46.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.389+0000 7f60457fa700 1 -- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6020002a40 msgr2=0x7f6020004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:46.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.389+0000 7f60457fa700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6020002a40 0x7f6020004f00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f60480c0580 tx=0x7f6040004b60 comp rx=0 tx=0).stop 2024-09-13T18:14:46.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.389+0000 7f60457fa700 1 -- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6020002a40 msgr2=0x7f6020004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:14:46.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.389+0000 7f60457fa700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6020002a40 0x7f6020004f00 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f60480c0580 tx=0x7f6040004b60 comp rx=0 tx=0).stop 2024-09-13T18:14:46.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.393+0000 7f604d0e8700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f60340667b0 0x7f6034068c70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6038001210 tx=0x7f6038007a90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:46.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.417+0000 7f602effd700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6020007020 0x7f6020007490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:46.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.417+0000 7f602effd700 1 -- 172.21.15.40:0/306819152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602000a400 con 0x7f6020007020 2024-09-13T18:14:46.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.417+0000 7f6047fff700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6020007020 0x7f6020007490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:46.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.417+0000 7f6047fff700 1 -- 172.21.15.40:0/306819152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f602000a5c0 con 0x7f6020007020 2024-09-13T18:14:46.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.421+0000 7f6047fff700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6020007020 0x7f6020007490 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6040000f80 tx=0x7f6040004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:46.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.421+0000 7f60457fa700 1 -- 172.21.15.40:0/306819152 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604002b8d0 con 0x7f6020007020 2024-09-13T18:14:46.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.421+0000 7f60457fa700 1 -- 172.21.15.40:0/306819152 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4607 (secure 0 0 0) 0x7f6040004030 con 0x7f6020007020 2024-09-13T18:14:46.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.421+0000 7f60457fa700 1 -- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6020007020 msgr2=0x7f6020007490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:46.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.421+0000 7f60457fa700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6020007020 0x7f6020007490 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6040000f80 tx=0x7f6040004af0 comp rx=0 tx=0).stop 2024-09-13T18:14:46.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.421+0000 7f60457fa700 1 -- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6020007020 msgr2=0x7f6020007490 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:14:46.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.421+0000 7f60457fa700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6020007020 0x7f6020007490 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:46.434 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:14:46.434 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:14:46.434 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:14:46.435 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:14:46.435 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 194, 2024-09-13T18:14:46.435 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:14:46.435 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:14:46.435 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:14:46.435 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:46.435 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:14:46.435 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:14:46.435 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:14:46.435 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:14:46.435 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:46.435 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:46.435 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:46.436 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:46.436 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:46.436 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:46.436 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:46.436 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:46.436 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:46.436 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:46.436 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:46.436 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:14:46.436 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:14:46.436 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:46.436 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:46.436 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:46.437 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:46.437 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:46.437 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:46.437 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:46.437 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:46.437 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:46.437 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:46.437 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:46.437 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.437 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:14:46.437 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:14:46.437 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:14:46.437 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:14:46.437 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:14:46.438 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:14:46.438 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:14:46.438 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:14:46.438 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:14:46.438 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:14:46.438 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:14:46.438 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:14:46.438 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:14:46.438 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:14:46.438 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:14:46.438 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:14:46.438 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:14:46.438 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:46.439 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:46.439 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:46.439 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:46.439 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:46.439 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:46.439 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:46.439 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:46.439 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:46.439 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:46.439 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:46.439 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:14:46.439 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.439 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:14:46.439 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.440 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:14:46.440 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:14:46.440 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:46.440 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:46.440 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.440 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:46.440 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:14:46.440 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:46.440 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.440 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.440 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:46.441 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:14:46.441 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:46.441 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:46.441 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:46.441 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.441 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:14:46.441 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:14:46.441 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:46.441 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:46.441 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:46.441 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.441 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.441 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:14:46.442 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:14:46.442 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:46.442 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:46.442 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.442 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:46.442 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:14:46.442 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:46.442 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.442 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.442 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:46.442 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:14:46.442 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:46.442 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:46.442 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:46.443 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.443 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:14:46.443 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:14:46.443 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:46.443 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:46.443 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:46.443 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.443 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.443 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:14:46.443 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:14:46.443 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:46.443 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:46.443 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.444 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:46.444 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:14:46.444 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:46.444 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.444 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.444 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:46.444 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:14:46.444 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:46.444 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:46.444 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:46.444 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.444 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:14:46.444 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:14:46.445 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:46.445 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:46.445 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:46.445 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:46.445 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:46.445 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.445 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:14:46.445 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:14:46.445 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.445 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:46.445 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:46.445 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:14:46.445 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:46.446 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:46.446 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:14:46.446 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.446 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:46.446 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:46.446 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:14:46.446 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:46.446 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:46.446 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:14:46.446 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.446 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:46.446 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:46.446 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:14:46.447 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:46.447 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:46.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.447 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:14:46.447 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:14:46.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.425+0000 7f602effd700 1 -- 172.21.15.40:0/306819152 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f60340667b0 msgr2=0x7f6034068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:46.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.425+0000 7f602effd700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f60340667b0 0x7f6034068c70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6038001210 tx=0x7f6038007a90 comp rx=0 tx=0).stop 2024-09-13T18:14:46.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.425+0000 7f604d0e8700 1 -- 172.21.15.40:0/306819152 reap_dead start 2024-09-13T18:14:46.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.425+0000 7f602effd700 1 -- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60480724e0 msgr2=0x7f60480c9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:46.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.425+0000 7f602effd700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60480724e0 0x7f60480c9020 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f603c00e9d0 tx=0x7f603c00ed90 comp rx=0 tx=0).stop 2024-09-13T18:14:46.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.425+0000 7f602effd700 1 -- 172.21.15.40:0/306819152 shutdown_connections 2024-09-13T18:14:46.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.425+0000 7f602effd700 1 --2- 172.21.15.40:0/306819152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60480724e0 0x7f60480c9020 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:46.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.425+0000 7f602effd700 1 -- 172.21.15.40:0/306819152 >> 172.21.15.40:0/306819152 conn(0x7f6048066090 msgr2=0x7f6048158390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:46.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.425+0000 7f602effd700 1 -- 172.21.15.40:0/306819152 shutdown_connections 2024-09-13T18:14:46.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.425+0000 7f602effd700 1 -- 172.21.15.40:0/306819152 wait complete. 2024-09-13T18:14:46.477 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:14:46.477 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T18:14:46.477 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:14:46.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.769+0000 7fede6e1c700 1 Processor -- start 2024-09-13T18:14:46.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.769+0000 7fede6e1c700 1 -- start start 2024-09-13T18:14:46.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.773+0000 7fede6e1c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fede00c7f60 0x7fede00c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:46.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.773+0000 7fede6e1c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede0167290 con 0x7fede00c8340 2024-09-13T18:14:46.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.773+0000 7fede6e1c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede0166680 con 0x7fede0163d00 2024-09-13T18:14:46.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.773+0000 7fede6e1c700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fede00c1630 con 0x7fede00c7f60 2024-09-13T18:14:46.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.773+0000 7fede5619700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7fede00c8340 legacy=0x7fede01635f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:14:46.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.773+0000 7fede5e1a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fede00c7f60 0x7fede00c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:46.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.773+0000 7fede5e1a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fede00c7f60 0x7fede00c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:57506/0 (socket says 172.21.15.40:57506) 2024-09-13T18:14:46.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.773+0000 7fede5e1a700 1 -- 172.21.15.40:0/1135812004 learned_addr learned my addr 172.21.15.40:0/1135812004 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:46.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.773+0000 7fede5e1a700 1 -- 172.21.15.40:0/1135812004 >> v1:172.21.15.43:6789/0 conn(0x7fede0163d00 legacy=0x7fede006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:46.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.773+0000 7fede5e1a700 1 -- 172.21.15.40:0/1135812004 >> v1:172.21.15.40:6789/0 conn(0x7fede00c8340 legacy=0x7fede01635f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:14:46.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.773+0000 7fede5e1a700 1 -- 172.21.15.40:0/1135812004 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede0166960 con 0x7fede00c7f60 2024-09-13T18:14:46.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.777+0000 7fede5e1a700 1 --2- 172.21.15.40:0/1135812004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fede00c7f60 0x7fede00c7250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7feddc008910 tx=0x7feddc00d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f0ec9c7ab881dbc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:46.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.777+0000 7fede4e18700 1 -- 172.21.15.40:0/1135812004 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc010070 con 0x7fede00c7f60 2024-09-13T18:14:46.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.777+0000 7fede4e18700 1 -- 172.21.15.40:0/1135812004 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feddc00d980 con 0x7fede00c7f60 2024-09-13T18:14:46.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.781+0000 7fede6e1c700 1 -- 172.21.15.40:0/1135812004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fede00c7f60 msgr2=0x7fede00c7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:46.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.781+0000 7fede6e1c700 1 --2- 172.21.15.40:0/1135812004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fede00c7f60 0x7fede00c7250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7feddc008910 tx=0x7feddc00d330 comp rx=0 tx=0).stop 2024-09-13T18:14:46.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.781+0000 7fede6e1c700 1 -- 172.21.15.40:0/1135812004 shutdown_connections 2024-09-13T18:14:46.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.781+0000 7fede6e1c700 1 --2- 172.21.15.40:0/1135812004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fede00c7f60 0x7fede00c7250 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7feddc008910 tx=0x7feddc00d330 comp rx=0 tx=0).stop 2024-09-13T18:14:46.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.781+0000 7fede6e1c700 1 -- 172.21.15.40:0/1135812004 >> 172.21.15.40:0/1135812004 conn(0x7fede0066090 msgr2=0x7fede00664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:46.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.785+0000 7fede6e1c700 1 -- 172.21.15.40:0/1135812004 shutdown_connections 2024-09-13T18:14:46.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.785+0000 7fede6e1c700 1 -- 172.21.15.40:0/1135812004 wait complete. 2024-09-13T18:14:46.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.785+0000 7fede6e1c700 1 Processor -- start 2024-09-13T18:14:46.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.785+0000 7fede6e1c700 1 -- start start 2024-09-13T18:14:46.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.785+0000 7fede6e1c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fede00c8340 0x7fede0070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:46.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.785+0000 7fede6e1c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede0163d00 0x7fede00c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:46.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.785+0000 7fede6e1c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fede01625e0 0x7fede01593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:46.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.785+0000 7fede6e1c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede0072020 con 0x7fede0163d00 2024-09-13T18:14:46.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.785+0000 7fede6e1c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede00cb540 con 0x7fede01625e0 2024-09-13T18:14:46.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.785+0000 7fede6e1c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede00cb820 con 0x7fede00c8340 2024-09-13T18:14:46.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.785+0000 7fede5619700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede0163d00 msgr2=0x7fede00c8f40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:14:46.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.785+0000 7fede5619700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede0163d00 0x7fede00c8f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:14:46.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.789+0000 7fede5e1a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fede00c8340 0x7fede0070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:46.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.789+0000 7fede661b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fede01625e0 0x7fede01593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:46.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.789+0000 7fede661b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fede01625e0 0x7fede01593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:57514/0 (socket says 172.21.15.40:57514) 2024-09-13T18:14:46.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.789+0000 7fede661b700 1 -- 172.21.15.40:0/3702682133 learned_addr learned my addr 172.21.15.40:0/3702682133 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:14:46.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.789+0000 7fede661b700 1 -- 172.21.15.40:0/3702682133 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fede00c8340 msgr2=0x7fede0070380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:46.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.789+0000 7fede661b700 1 --2- 172.21.15.40:0/3702682133 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fede00c8340 0x7fede0070380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:46.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.789+0000 7fede661b700 1 -- 172.21.15.40:0/3702682133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede0163d00 msgr2=0x7fede00c8f40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:14:46.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.789+0000 7fede661b700 1 --2- 172.21.15.40:0/3702682133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede0163d00 0x7fede00c8f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:46.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.789+0000 7fede661b700 1 -- 172.21.15.40:0/3702682133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feddc0085f0 con 0x7fede01625e0 2024-09-13T18:14:46.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.789+0000 7fede661b700 1 --2- 172.21.15.40:0/3702682133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fede01625e0 0x7fede01593c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fedd000c3c0 tx=0x7fedd000c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:46.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.789+0000 7fedd67fc700 1 -- 172.21.15.40:0/3702682133 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd00052d0 con 0x7fede01625e0 2024-09-13T18:14:46.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.793+0000 7fede6e1c700 1 -- 172.21.15.40:0/3702682133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede0072200 con 0x7fede01625e0 2024-09-13T18:14:46.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.793+0000 7fede6e1c700 1 -- 172.21.15.40:0/3702682133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede0162a60 con 0x7fede01625e0 2024-09-13T18:14:46.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.793+0000 7fedd67fc700 1 -- 172.21.15.40:0/3702682133 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd0005430 con 0x7fede01625e0 2024-09-13T18:14:46.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.793+0000 7fedd67fc700 1 -- 172.21.15.40:0/3702682133 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd00055a0 con 0x7fede01625e0 2024-09-13T18:14:46.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.793+0000 7fedd67fc700 1 -- 172.21.15.40:0/3702682133 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fedd00036a0 con 0x7fede01625e0 2024-09-13T18:14:46.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.793+0000 7fede6e1c700 1 --2- 172.21.15.40:0/3702682133 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fedc8002a40 0x7fedc8004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:46.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.793+0000 7fede6e1c700 1 -- 172.21.15.40:0/3702682133 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc8007f40 con 0x7fedc8002a40 2024-09-13T18:14:46.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.793+0000 7fedd67fc700 1 --2- 172.21.15.40:0/3702682133 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fedcc066930 0x7fedcc068df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:46.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.793+0000 7fede5619700 1 --2- 172.21.15.40:0/3702682133 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fedcc066930 0x7fedcc068df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:46.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.797+0000 7fede5619700 1 --2- 172.21.15.40:0/3702682133 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fedcc066930 0x7fedcc068df0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fede0164a90 tx=0x7fedd8009270 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:46.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.797+0000 7fedd67fc700 1 -- 172.21.15.40:0/3702682133 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fedd00837a0 con 0x7fede01625e0 2024-09-13T18:14:46.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.797+0000 7fede5e1a700 1 --2- 172.21.15.40:0/3702682133 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fedc8002a40 0x7fedc8004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:46.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.797+0000 7fede5e1a700 1 -- 172.21.15.40:0/3702682133 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fedc8008100 con 0x7fedc8002a40 2024-09-13T18:14:46.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.801+0000 7fede5e1a700 1 --2- 172.21.15.40:0/3702682133 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fedc8002a40 0x7fedc8004f00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feddc008910 tx=0x7feddc0047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:46.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.801+0000 7fedd67fc700 1 -- 172.21.15.40:0/3702682133 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc00dd30 con 0x7fedc8002a40 2024-09-13T18:14:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.801+0000 7fedd67fc700 1 -- 172.21.15.40:0/3702682133 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7feddc00de90 con 0x7fedc8002a40 2024-09-13T18:14:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.801+0000 7fedd67fc700 1 -- 172.21.15.40:0/3702682133 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fedc8002a40 msgr2=0x7fedc8004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.801+0000 7fedd67fc700 1 --2- 172.21.15.40:0/3702682133 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fedc8002a40 0x7fedc8004f00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feddc008910 tx=0x7feddc0047e0 comp rx=0 tx=0).stop 2024-09-13T18:14:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.801+0000 7fedd67fc700 1 -- 172.21.15.40:0/3702682133 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fedc8002a40 msgr2=0x7fedc8004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:14:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.801+0000 7fedd67fc700 1 --2- 172.21.15.40:0/3702682133 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fedc8002a40 0x7fedc8004f00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:46.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.841+0000 7fede6e1c700 1 --2- 172.21.15.40:0/3702682133 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fedc8006ee0 0x7fedc8007330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:14:46.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.841+0000 7fede6e1c700 1 -- 172.21.15.40:0/3702682133 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc800a4a0 con 0x7fedc8006ee0 2024-09-13T18:14:46.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.841+0000 7fede5e1a700 1 --2- 172.21.15.40:0/3702682133 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fedc8006ee0 0x7fedc8007330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:14:46.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.841+0000 7fede5e1a700 1 -- 172.21.15.40:0/3702682133 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fedc8007870 con 0x7fedc8006ee0 2024-09-13T18:14:46.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.841+0000 7fede5e1a700 1 --2- 172.21.15.40:0/3702682133 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fedc8006ee0 0x7fedc8007330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fedc80080d0 tx=0x7feddc015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:14:46.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.841+0000 7fedd67fc700 1 -- 172.21.15.40:0/3702682133 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc016e20 con 0x7fedc8006ee0 2024-09-13T18:14:46.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.841+0000 7fedd67fc700 1 -- 172.21.15.40:0/3702682133 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4937 (secure 0 0 0) 0x7feddc028610 con 0x7fedc8006ee0 2024-09-13T18:14:46.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.841+0000 7fedd67fc700 1 -- 172.21.15.40:0/3702682133 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fedc8006ee0 msgr2=0x7fedc8007330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:46.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.841+0000 7fedd67fc700 1 --2- 172.21.15.40:0/3702682133 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fedc8006ee0 0x7fedc8007330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fedc80080d0 tx=0x7feddc015000 comp rx=0 tx=0).stop 2024-09-13T18:14:46.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.841+0000 7fedd67fc700 1 -- 172.21.15.40:0/3702682133 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fedc8006ee0 msgr2=0x7fedc8007330 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:14:46.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.841+0000 7fedd67fc700 1 --2- 172.21.15.40:0/3702682133 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fedc8006ee0 0x7fedc8007330 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fedc80080d0 tx=0x7feddc015000 comp rx=0 tx=0).stop 2024-09-13T18:14:46.854 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:14:46.854 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:14:46.854 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:14:46.854 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:14:46.854 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 194, 2024-09-13T18:14:46.854 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:14:46.855 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:14:46.855 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:14:46.855 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:46.855 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:14:46.855 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:14:46.855 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:14:46.855 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:14:46.855 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:46.855 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:46.855 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:46.855 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:46.855 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:46.855 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:46.855 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:46.856 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:46.856 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:46.856 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:46.856 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:46.856 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:14:46.856 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:14:46.856 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:46.856 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:46.856 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:46.856 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:46.856 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:46.856 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:46.856 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:46.857 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:46.857 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:46.857 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:46.857 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:46.857 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.857 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:14:46.857 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:14:46.857 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:14:46.857 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:14:46.857 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:14:46.857 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:14:46.857 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:14:46.857 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:14:46.857 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:14:46.858 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:14:46.858 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:14:46.858 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:14:46.858 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:14:46.858 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:14:46.858 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:14:46.858 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:14:46.858 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:14:46.858 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:14:46.858 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:14:46.858 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:14:46.858 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:14:46.858 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:14:46.858 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:14:46.859 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:14:46.859 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:14:46.859 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:14:46.859 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:14:46.859 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:46.859 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:14:46.859 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.859 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:14:46.859 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.859 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:14:46.859 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:14:46.859 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:46.859 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:46.859 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.860 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:46.860 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:14:46.860 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:46.860 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.860 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.860 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:46.860 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:14:46.860 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:46.860 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:46.860 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:46.860 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.860 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:14:46.860 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:14:46.861 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:46.861 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:46.861 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:46.861 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.861 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.861 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:14:46.861 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:14:46.861 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:46.861 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:46.861 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.861 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:46.861 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:14:46.861 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:46.861 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.862 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.862 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:46.862 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:14:46.862 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:46.862 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:46.862 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:46.862 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.862 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:14:46.862 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:14:46.862 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:46.863 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:46.863 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:46.863 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.863 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:14:46.863 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:14:46.863 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:14:46.863 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:14:46.863 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.863 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:14:46.863 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:14:46.863 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:46.863 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.864 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.864 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:14:46.864 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:14:46.864 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:14:46.864 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:46.864 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:46.864 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.864 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:14:46.864 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:14:46.864 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:14:46.864 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:14:46.864 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:14:46.864 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:46.864 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:46.865 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.865 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:14:46.865 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:14:46.865 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.865 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:46.865 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:46.865 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:14:46.865 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:46.865 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:46.865 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:14:46.865 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.865 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:46.865 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:46.865 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:14:46.866 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:46.866 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:46.866 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:14:46.866 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.866 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:46.866 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:46.866 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:14:46.866 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:46.866 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:46.866 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:14:46.866 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.866 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:46.866 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:46.866 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:14:46.867 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:46.867 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:14:46.867 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:14:46.867 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:14:46.867 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:14:46.867 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:14:46.867 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:14:46.867 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:14:46.867 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:14:46.867 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:14:46.867 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:14:46.867 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:14:46.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.865+0000 7fede6e1c700 1 -- 172.21.15.40:0/3702682133 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fedcc066930 msgr2=0x7fedcc068df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:46.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.865+0000 7fede6e1c700 1 --2- 172.21.15.40:0/3702682133 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fedcc066930 0x7fedcc068df0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fede0164a90 tx=0x7fedd8009270 comp rx=0 tx=0).stop 2024-09-13T18:14:46.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.865+0000 7fede661b700 1 -- 172.21.15.40:0/3702682133 reap_dead start 2024-09-13T18:14:46.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.865+0000 7fede6e1c700 1 -- 172.21.15.40:0/3702682133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fede01625e0 msgr2=0x7fede01593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:14:46.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.865+0000 7fede6e1c700 1 --2- 172.21.15.40:0/3702682133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fede01625e0 0x7fede01593c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fedd000c3c0 tx=0x7fedd000c6d0 comp rx=0 tx=0).stop 2024-09-13T18:14:46.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.865+0000 7fede6e1c700 1 -- 172.21.15.40:0/3702682133 shutdown_connections 2024-09-13T18:14:46.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.865+0000 7fede6e1c700 1 --2- 172.21.15.40:0/3702682133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fede01625e0 0x7fede01593c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:14:46.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.865+0000 7fede6e1c700 1 -- 172.21.15.40:0/3702682133 >> 172.21.15.40:0/3702682133 conn(0x7fede0066090 msgr2=0x7fede00735e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:14:46.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.865+0000 7fede6e1c700 1 -- 172.21.15.40:0/3702682133 shutdown_connections 2024-09-13T18:14:46.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:14:46.865+0000 7fede6e1c700 1 -- 172.21.15.40:0/3702682133 wait complete. 2024-09-13T18:14:46.928 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:14:46.928 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:15:06.929 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T18:15:06.929 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-13T18:15:06.929 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T18:15:06.930 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T18:15:06.933 INFO:tasks.ceph.mon.a:Started 2024-09-13T18:15:06.933 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:15:06.933 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:15:07.966 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:15:07.971 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:15:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:07.993+0000 7f461658b700 1 Processor -- start 2024-09-13T18:15:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:07.993+0000 7f461658b700 1 -- start start 2024-09-13T18:15:08.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:07.997+0000 7f461658b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4610160390 0x7f46101669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:07.997+0000 7f461658b700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4610071930 con 0x7f4610167d10 2024-09-13T18:15:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:07.997+0000 7f461658b700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4610070a40 con 0x7f46100704e0 2024-09-13T18:15:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:07.997+0000 7f461658b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4610166f20 con 0x7f4610160390 2024-09-13T18:15:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:07.997+0000 7f460ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4610160390 0x7f46101669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:07.997+0000 7f460ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4610160390 0x7f46101669e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33384/0 (socket says 172.21.15.40:33384) 2024-09-13T18:15:08.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:07.997+0000 7f460ffff700 1 -- 172.21.15.40:0/1202453033 learned_addr learned my addr 172.21.15.40:0/1202453033 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:08.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.005+0000 7f460ffff700 1 -- 172.21.15.40:0/1202453033 >> v2:172.21.15.40:3300/0 conn(0x7f4610160390 msgr2=0x7f46101669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:15:08.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.005+0000 7f460ffff700 1 -- 172.21.15.40:0/1202453033 >> v2:172.21.15.40:3300/0 conn(0x7f4610160390 msgr2=0x7f46101669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:15:08.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.009+0000 7f460ffff700 1 --2- 172.21.15.40:0/1202453033 >> v2:172.21.15.40:3300/0 conn(0x7f4610160390 0x7f46101669e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.009+0000 7f460ffff700 1 --2- 172.21.15.40:0/1202453033 >> v2:172.21.15.40:3300/0 conn(0x7f4610160390 0x7f46101669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:15:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.005+0000 7f460effd700 1 -- 172.21.15.40:0/1202453033 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 867712553 0 0) 0x7f4610070a40 con 0x7f46100704e0 2024-09-13T18:15:08.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.009+0000 7f460effd700 1 -- 172.21.15.40:0/1202453033 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45fc0035d0 con 0x7f46100704e0 2024-09-13T18:15:08.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460effd700 1 -- 172.21.15.40:0/1202453033 <== mon.1 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 690724463 0 0) 0x7f45fc0035d0 con 0x7f46100704e0 2024-09-13T18:15:08.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460effd700 1 -- 172.21.15.40:0/1202453033 >> v2:172.21.15.40:3300/0 conn(0x7f4610160390 msgr2=0x7f46101669e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:15:08.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460effd700 1 --2- 172.21.15.40:0/1202453033 >> v2:172.21.15.40:3300/0 conn(0x7f4610160390 0x7f46101669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:08.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460effd700 1 -- 172.21.15.40:0/1202453033 >> v1:172.21.15.40:6789/0 conn(0x7f4610167d10 legacy=0x7f461006a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:08.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460effd700 1 -- 172.21.15.40:0/1202453033 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4610070d20 con 0x7f46100704e0 2024-09-13T18:15:08.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460effd700 1 -- 172.21.15.40:0/1202453033 <== mon.1 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f4600001670 con 0x7f46100704e0 2024-09-13T18:15:08.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460effd700 1 -- 172.21.15.40:0/1202453033 >> v1:172.21.15.43:6789/0 conn(0x7f46100704e0 legacy=0x7f4610167600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:08.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460effd700 1 --2- 172.21.15.40:0/1202453033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45fc003a20 0x7f45fc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:08.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460effd700 1 --2- 172.21.15.40:0/1202453033 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f45fc004870 0x7f45fc004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:08.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460ffff700 1 --2- 172.21.15.40:0/1202453033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45fc003a20 0x7f45fc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:08.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460effd700 1 --2- 172.21.15.40:0/1202453033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45fc005230 0x7f45fc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:08.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460effd700 1 -- 172.21.15.40:0/1202453033 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4610166f20 con 0x7f45fc003a20 2024-09-13T18:15:08.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460effd700 1 -- 172.21.15.40:0/1202453033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc00c160 con 0x7f45fc005230 2024-09-13T18:15:08.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460effd700 1 -- 172.21.15.40:0/1202453033 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc00c320 con 0x7f45fc004870 2024-09-13T18:15:08.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f4614b28700 1 --2- 172.21.15.40:0/1202453033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45fc005230 0x7f45fc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:08.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460ffff700 1 -- 172.21.15.40:0/1202453033 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f45fc004870 msgr2=0x7f45fc004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:15:08.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460ffff700 1 --2- 172.21.15.40:0/1202453033 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f45fc004870 0x7f45fc004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:08.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460ffff700 1 -- 172.21.15.40:0/1202453033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45fc005230 msgr2=0x7f45fc00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:15:08.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460ffff700 1 --2- 172.21.15.40:0/1202453033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45fc005230 0x7f45fc00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:08.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460ffff700 1 -- 172.21.15.40:0/1202453033 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4610070d20 con 0x7f45fc003a20 2024-09-13T18:15:08.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f4614b28700 1 --2- 172.21.15.40:0/1202453033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45fc005230 0x7f45fc00ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:15:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f4614b28700 1 -- 172.21.15.40:0/1202453033 reap_dead start 2024-09-13T18:15:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460ffff700 1 --2- 172.21.15.40:0/1202453033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45fc003a20 0x7f45fc003e40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009b60 tx=0x7f45fc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef4d5c65c2c18f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460effd700 1 -- 172.21.15.40:0/1202453033 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8007b80 con 0x7f45fc003a20 2024-09-13T18:15:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460effd700 1 -- 172.21.15.40:0/1202453033 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f800f040 con 0x7f45fc003a20 2024-09-13T18:15:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f460effd700 1 -- 172.21.15.40:0/1202453033 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f80133f0 con 0x7f45fc003a20 2024-09-13T18:15:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f461658b700 1 -- 172.21.15.40:0/1202453033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45fc003a20 msgr2=0x7f45fc003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f461658b700 1 --2- 172.21.15.40:0/1202453033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45fc003a20 0x7f45fc003e40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009b60 tx=0x7f45fc0058a0 comp rx=0 tx=0).stop 2024-09-13T18:15:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f461658b700 1 -- 172.21.15.40:0/1202453033 shutdown_connections 2024-09-13T18:15:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f461658b700 1 --2- 172.21.15.40:0/1202453033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45fc003a20 0x7f45fc003e40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f461658b700 1 -- 172.21.15.40:0/1202453033 >> 172.21.15.40:0/1202453033 conn(0x7f46100cabc0 msgr2=0x7f46100cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f461658b700 1 -- 172.21.15.40:0/1202453033 shutdown_connections 2024-09-13T18:15:08.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.013+0000 7f461658b700 1 -- 172.21.15.40:0/1202453033 wait complete. 2024-09-13T18:15:08.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.089+0000 7f461658b700 1 Processor -- start 2024-09-13T18:15:08.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.089+0000 7f461658b700 1 -- start start 2024-09-13T18:15:08.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.089+0000 7f461658b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f45fc003a20 0x7f46101f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:08.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.089+0000 7f461658b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46100c1760 0x7f46101f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:08.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.089+0000 7f461658b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4610200480 0x7f46102039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:08.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.089+0000 7f461658b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46101f34f0 con 0x7f4610200480 2024-09-13T18:15:08.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.089+0000 7f461658b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46101633f0 con 0x7f46100c1760 2024-09-13T18:15:08.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.089+0000 7f461658b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46101f07d0 con 0x7f45fc003a20 2024-09-13T18:15:08.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.089+0000 7f4614b28700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4610200480 0x7f46102039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:08.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.089+0000 7f4614b28700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4610200480 0x7f46102039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33404/0 (socket says 172.21.15.40:33404) 2024-09-13T18:15:08.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.089+0000 7f4614b28700 1 -- 172.21.15.40:0/1960973150 learned_addr learned my addr 172.21.15.40:0/1960973150 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:08.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.093+0000 7f460ffff700 1 --2- 172.21.15.40:0/1960973150 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f45fc003a20 0x7f46101f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:08.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.093+0000 7f460f7fe700 1 --2- 172.21.15.40:0/1960973150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46100c1760 0x7f46101f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:08.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.093+0000 7f4614b28700 1 -- 172.21.15.40:0/1960973150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4610200480 msgr2=0x7f46102039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:15:08.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.093+0000 7f4614b28700 1 -- 172.21.15.40:0/1960973150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4610200480 msgr2=0x7f46102039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:15:08.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.093+0000 7f4614b28700 1 --2- 172.21.15.40:0/1960973150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4610200480 0x7f46102039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:08.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.093+0000 7f4614b28700 1 --2- 172.21.15.40:0/1960973150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4610200480 0x7f46102039f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:15:08.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.093+0000 7f460f7fe700 1 -- 172.21.15.40:0/1960973150 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f45fc003a20 msgr2=0x7f46101f2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:08.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.093+0000 7f460f7fe700 1 --2- 172.21.15.40:0/1960973150 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f45fc003a20 0x7f46101f2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:08.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.093+0000 7f460f7fe700 1 -- 172.21.15.40:0/1960973150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4610200480 msgr2=0x7f46102039f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:15:08.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.093+0000 7f460f7fe700 1 --2- 172.21.15.40:0/1960973150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4610200480 0x7f46102039f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:08.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.093+0000 7f460f7fe700 1 -- 172.21.15.40:0/1960973150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4610161460 con 0x7f46100c1760 2024-09-13T18:15:08.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.093+0000 7f460f7fe700 1 --2- 172.21.15.40:0/1960973150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46100c1760 0x7f46101f2fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f460000c7f0 tx=0x7f460000cb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:08.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.093+0000 7f460cff9700 1 -- 172.21.15.40:0/1960973150 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600005500 con 0x7f46100c1760 2024-09-13T18:15:08.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.093+0000 7f460cff9700 1 -- 172.21.15.40:0/1960973150 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46000036a0 con 0x7f46100c1760 2024-09-13T18:15:08.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.093+0000 7f461658b700 1 -- 172.21.15.40:0/1960973150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46101f0c60 con 0x7f46100c1760 2024-09-13T18:15:08.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.093+0000 7f460cff9700 1 -- 172.21.15.40:0/1960973150 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600003a50 con 0x7f46100c1760 2024-09-13T18:15:08.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.093+0000 7f461658b700 1 -- 172.21.15.40:0/1960973150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46101efea0 con 0x7f46100c1760 2024-09-13T18:15:08.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.097+0000 7f461658b700 1 -- 172.21.15.40:0/1960973150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4610070a40 con 0x7f46100c1760 2024-09-13T18:15:08.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.097+0000 7f460cff9700 1 -- 172.21.15.40:0/1960973150 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4600003bb0 con 0x7f46100c1760 2024-09-13T18:15:08.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.097+0000 7f460cff9700 1 --2- 172.21.15.40:0/1960973150 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f45f0066800 0x7f45f0068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.097+0000 7f460ffff700 1 --2- 172.21.15.40:0/1960973150 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f45f0066800 0x7f45f0068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.097+0000 7f460ffff700 1 --2- 172.21.15.40:0/1960973150 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f45f0066800 0x7f45f0068cc0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009fd0 tx=0x7f45f800b000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:08.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.101+0000 7f460cff9700 1 -- 172.21.15.40:0/1960973150 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4600083c70 con 0x7f46100c1760 2024-09-13T18:15:08.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.113+0000 7f460cff9700 1 -- 172.21.15.40:0/1960973150 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4600050c00 con 0x7f46100c1760 2024-09-13T18:15:08.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.817+0000 7f461658b700 1 -- 172.21.15.40:0/1960973150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f461008b470 con 0x7f46100c1760 2024-09-13T18:15:08.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.817+0000 7f460cff9700 1 -- 172.21.15.40:0/1960973150 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f4600003e60 con 0x7f46100c1760 2024-09-13T18:15:08.827 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":194,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":24,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:15:08.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.829+0000 7f46067fc700 1 -- 172.21.15.40:0/1960973150 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f45f0066800 msgr2=0x7f45f0068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:08.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.829+0000 7f46067fc700 1 --2- 172.21.15.40:0/1960973150 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f45f0066800 0x7f45f0068cc0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009fd0 tx=0x7f45f800b000 comp rx=0 tx=0).stop 2024-09-13T18:15:08.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.829+0000 7f46067fc700 1 -- 172.21.15.40:0/1960973150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46100c1760 msgr2=0x7f46101f2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:08.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.829+0000 7f46067fc700 1 --2- 172.21.15.40:0/1960973150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46100c1760 0x7f46101f2fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f460000c7f0 tx=0x7f460000cb00 comp rx=0 tx=0).stop 2024-09-13T18:15:08.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.829+0000 7f46067fc700 1 -- 172.21.15.40:0/1960973150 shutdown_connections 2024-09-13T18:15:08.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.829+0000 7f46067fc700 1 --2- 172.21.15.40:0/1960973150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4610200480 0x7f46102039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:08.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.829+0000 7f46067fc700 1 --2- 172.21.15.40:0/1960973150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46100c1760 0x7f46101f2fb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:08.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.829+0000 7f46067fc700 1 --2- 172.21.15.40:0/1960973150 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f45f0066800 0x7f45f0068cc0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:08.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.829+0000 7f46067fc700 1 --2- 172.21.15.40:0/1960973150 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f45fc003a20 0x7f46101f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:08.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.829+0000 7f46067fc700 1 -- 172.21.15.40:0/1960973150 >> 172.21.15.40:0/1960973150 conn(0x7f46100cabc0 msgr2=0x7f4610163ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:08.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.829+0000 7f46067fc700 1 -- 172.21.15.40:0/1960973150 shutdown_connections 2024-09-13T18:15:08.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:08.829+0000 7f46067fc700 1 -- 172.21.15.40:0/1960973150 wait complete. 2024-09-13T18:15:11.855 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:15:12.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.149+0000 7f44d5112700 1 Processor -- start 2024-09-13T18:15:12.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.149+0000 7f44d5112700 1 -- start start 2024-09-13T18:15:12.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.149+0000 7f44d5112700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f44d015e1a0 0x7f44d006a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:12.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.149+0000 7f44d5112700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44d006a8d0 0x7f44d006ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:12.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.149+0000 7f44d5112700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d0071690 con 0x7f44d0154240 2024-09-13T18:15:12.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.149+0000 7f44d5112700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d00721e0 con 0x7f44d015e1a0 2024-09-13T18:15:12.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.149+0000 7f44d5112700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d0154620 con 0x7f44d006a8d0 2024-09-13T18:15:12.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.149+0000 7f44ced9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f44d0154240 0x7f44d015ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55978/0 (socket says 172.21.15.40:55978) 2024-09-13T18:15:12.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.149+0000 7f44ced9d700 1 -- 172.21.15.40:0/102600655 learned_addr learned my addr 172.21.15.40:0/102600655 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:15:12.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.149+0000 7f44ce59c700 1 --2- 172.21.15.40:0/102600655 >> v2:172.21.15.40:3300/0 conn(0x7f44d015e1a0 0x7f44d006a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:12.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.153+0000 7f44ce59c700 1 -- 172.21.15.40:0/102600655 >> v2:172.21.15.40:3300/0 conn(0x7f44d015e1a0 msgr2=0x7f44d006a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:15:12.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.153+0000 7f44ce59c700 1 -- 172.21.15.40:0/102600655 >> v2:172.21.15.40:3300/0 conn(0x7f44d015e1a0 msgr2=0x7f44d006a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:15:12.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.153+0000 7f44ce59c700 1 --2- 172.21.15.40:0/102600655 >> v2:172.21.15.40:3300/0 conn(0x7f44d015e1a0 0x7f44d006a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:12.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.153+0000 7f44ce59c700 1 --2- 172.21.15.40:0/102600655 >> v2:172.21.15.40:3300/0 conn(0x7f44d015e1a0 0x7f44d006a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:15:12.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.153+0000 7f44cf59e700 1 --2- 172.21.15.40:0/102600655 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44d006a8d0 0x7f44d006ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:12.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.153+0000 7f44cf59e700 1 -- 172.21.15.40:0/102600655 >> v2:172.21.15.40:3300/0 conn(0x7f44d015e1a0 msgr2=0x7f44d006a390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:15:12.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.153+0000 7f44cf59e700 1 --2- 172.21.15.40:0/102600655 >> v2:172.21.15.40:3300/0 conn(0x7f44d015e1a0 0x7f44d006a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:12.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.153+0000 7f44cf59e700 1 -- 172.21.15.40:0/102600655 >> v1:172.21.15.40:6789/0 conn(0x7f44d0154240 legacy=0x7f44d015ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:12.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.153+0000 7f44cf59e700 1 -- 172.21.15.40:0/102600655 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d0071b20 con 0x7f44d006a8d0 2024-09-13T18:15:12.161 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.153+0000 7f44cf59e700 1 --2- 172.21.15.40:0/102600655 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44d006a8d0 0x7f44d006ad20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009850 tx=0x7f44c0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40660d6ed660b618 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:12.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.157+0000 7f44cdd9b700 1 -- 172.21.15.40:0/102600655 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0011830 con 0x7f44d006a8d0 2024-09-13T18:15:12.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.157+0000 7f44cdd9b700 1 -- 172.21.15.40:0/102600655 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c0011990 con 0x7f44d006a8d0 2024-09-13T18:15:12.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.157+0000 7f44cdd9b700 1 -- 172.21.15.40:0/102600655 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0011d20 con 0x7f44d006a8d0 2024-09-13T18:15:12.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.157+0000 7f44d5112700 1 -- 172.21.15.40:0/102600655 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44d006a8d0 msgr2=0x7f44d006ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:12.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.157+0000 7f44d5112700 1 --2- 172.21.15.40:0/102600655 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44d006a8d0 0x7f44d006ad20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009850 tx=0x7f44c0009b60 comp rx=0 tx=0).stop 2024-09-13T18:15:12.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.161+0000 7f44d5112700 1 -- 172.21.15.40:0/102600655 shutdown_connections 2024-09-13T18:15:12.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.161+0000 7f44d5112700 1 --2- 172.21.15.40:0/102600655 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44d006a8d0 0x7f44d006ad20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:12.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.161+0000 7f44d5112700 1 --2- 172.21.15.40:0/102600655 >> v2:172.21.15.40:3300/0 conn(0x7f44d015e1a0 0x7f44d006a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:12.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.161+0000 7f44d5112700 1 -- 172.21.15.40:0/102600655 >> 172.21.15.40:0/102600655 conn(0x7f44d00cabc0 msgr2=0x7f44d00cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:12.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.165+0000 7f44d5112700 1 -- 172.21.15.40:0/102600655 shutdown_connections 2024-09-13T18:15:12.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.165+0000 7f44d5112700 1 -- 172.21.15.40:0/102600655 wait complete. 2024-09-13T18:15:12.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.165+0000 7f44d5112700 1 Processor -- start 2024-09-13T18:15:12.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.165+0000 7f44d5112700 1 -- start start 2024-09-13T18:15:12.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.165+0000 7f44d5112700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d006a8d0 0x7f44d0157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:12.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.165+0000 7f44d5112700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44d0154240 0x7f44d00c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:12.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.165+0000 7f44d5112700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44d015e1a0 0x7f44d00c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:12.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.165+0000 7f44d5112700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00c52b0 con 0x7f44d006a8d0 2024-09-13T18:15:12.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.165+0000 7f44d5112700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d0159830 con 0x7f44d0154240 2024-09-13T18:15:12.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.165+0000 7f44d5112700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d0159aa0 con 0x7f44d015e1a0 2024-09-13T18:15:12.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.169+0000 7f44ce59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44d0154240 0x7f44d00c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:12.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.169+0000 7f44ced9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d006a8d0 0x7f44d0157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:12.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.169+0000 7f44ce59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44d0154240 0x7f44d00c7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:36708/0 (socket says 172.21.15.40:36708) 2024-09-13T18:15:12.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.169+0000 7f44ce59c700 1 -- 172.21.15.40:0/3151401775 learned_addr learned my addr 172.21.15.40:0/3151401775 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:12.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.169+0000 7f44ced9d700 1 -- 172.21.15.40:0/3151401775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d006a8d0 msgr2=0x7f44d0157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:15:12.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.169+0000 7f44ced9d700 1 -- 172.21.15.40:0/3151401775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d006a8d0 msgr2=0x7f44d0157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:15:12.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.169+0000 7f44ced9d700 1 --2- 172.21.15.40:0/3151401775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d006a8d0 0x7f44d0157e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:12.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.169+0000 7f44ced9d700 1 --2- 172.21.15.40:0/3151401775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d006a8d0 0x7f44d0157e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:15:12.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.169+0000 7f44cf59e700 1 --2- 172.21.15.40:0/3151401775 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44d015e1a0 0x7f44d00c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:12.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.169+0000 7f44ce59c700 1 -- 172.21.15.40:0/3151401775 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44d015e1a0 msgr2=0x7f44d00c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:12.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.169+0000 7f44ce59c700 1 --2- 172.21.15.40:0/3151401775 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44d015e1a0 0x7f44d00c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:12.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.169+0000 7f44ce59c700 1 -- 172.21.15.40:0/3151401775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d006a8d0 msgr2=0x7f44d0157e00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:15:12.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.169+0000 7f44ce59c700 1 --2- 172.21.15.40:0/3151401775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d006a8d0 0x7f44d0157e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:12.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.169+0000 7f44ce59c700 1 -- 172.21.15.40:0/3151401775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c00094b0 con 0x7f44d0154240 2024-09-13T18:15:12.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.173+0000 7f44ce59c700 1 --2- 172.21.15.40:0/3151401775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44d0154240 0x7f44d00c7cd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f44c400e7e0 tx=0x7f44c400eba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:12.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.177+0000 7f44b77fe700 1 -- 172.21.15.40:0/3151401775 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c4013610 con 0x7f44d0154240 2024-09-13T18:15:12.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.177+0000 7f44b77fe700 1 -- 172.21.15.40:0/3151401775 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c4013770 con 0x7f44d0154240 2024-09-13T18:15:12.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.177+0000 7f44b77fe700 1 -- 172.21.15.40:0/3151401775 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c4013b20 con 0x7f44d0154240 2024-09-13T18:15:12.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.177+0000 7f44d5112700 1 -- 172.21.15.40:0/3151401775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d0159d80 con 0x7f44d0154240 2024-09-13T18:15:12.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.177+0000 7f44d5112700 1 -- 172.21.15.40:0/3151401775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d00c5500 con 0x7f44d0154240 2024-09-13T18:15:12.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.181+0000 7f44d5112700 1 -- 172.21.15.40:0/3151401775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b0005310 con 0x7f44d0154240 2024-09-13T18:15:12.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.181+0000 7f44b77fe700 1 -- 172.21.15.40:0/3151401775 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f44c4013c80 con 0x7f44d0154240 2024-09-13T18:15:12.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.181+0000 7f44b77fe700 1 --2- 172.21.15.40:0/3151401775 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f44bc066830 0x7f44bc068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:12.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.181+0000 7f44b77fe700 1 -- 172.21.15.40:0/3151401775 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f44c4084170 con 0x7f44d0154240 2024-09-13T18:15:12.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.185+0000 7f44ced9d700 1 --2- 172.21.15.40:0/3151401775 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f44bc066830 0x7f44bc068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:12.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.185+0000 7f44ced9d700 1 --2- 172.21.15.40:0/3151401775 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f44bc066830 0x7f44bc068cf0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f44c8005cb0 tx=0x7f44c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:12.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:12.197+0000 7f44b77fe700 1 -- 172.21.15.40:0/3151401775 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f44c4051190 con 0x7f44d0154240 2024-09-13T18:15:13.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:13.221+0000 7f44d5112700 1 -- 172.21.15.40:0/3151401775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f44b0005b40 con 0x7f44d0154240 2024-09-13T18:15:13.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:13.221+0000 7f44b77fe700 1 -- 172.21.15.40:0/3151401775 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f44c4021610 con 0x7f44d0154240 2024-09-13T18:15:13.232 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":194,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":28,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:15:13.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:13.233+0000 7f44b57fa700 1 -- 172.21.15.40:0/3151401775 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f44bc066830 msgr2=0x7f44bc068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:13.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:13.233+0000 7f44b57fa700 1 --2- 172.21.15.40:0/3151401775 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f44bc066830 0x7f44bc068cf0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f44c8005cb0 tx=0x7f44c8009040 comp rx=0 tx=0).stop 2024-09-13T18:15:13.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:13.233+0000 7f44b57fa700 1 -- 172.21.15.40:0/3151401775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44d0154240 msgr2=0x7f44d00c7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:13.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:13.233+0000 7f44b57fa700 1 --2- 172.21.15.40:0/3151401775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44d0154240 0x7f44d00c7cd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f44c400e7e0 tx=0x7f44c400eba0 comp rx=0 tx=0).stop 2024-09-13T18:15:13.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:13.233+0000 7f44b57fa700 1 -- 172.21.15.40:0/3151401775 shutdown_connections 2024-09-13T18:15:13.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:13.233+0000 7f44b57fa700 1 --2- 172.21.15.40:0/3151401775 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f44bc066830 0x7f44bc068cf0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:13.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:13.233+0000 7f44b57fa700 1 --2- 172.21.15.40:0/3151401775 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f44d015e1a0 0x7f44d00c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:13.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:13.233+0000 7f44b57fa700 1 --2- 172.21.15.40:0/3151401775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44d0154240 0x7f44d00c7cd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:13.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:13.233+0000 7f44b57fa700 1 --2- 172.21.15.40:0/3151401775 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44d006a8d0 0x7f44d0157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:13.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:13.233+0000 7f44b57fa700 1 -- 172.21.15.40:0/3151401775 >> 172.21.15.40:0/3151401775 conn(0x7f44d00cabc0 msgr2=0x7f44d00c4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:13.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:13.233+0000 7f44b57fa700 1 -- 172.21.15.40:0/3151401775 shutdown_connections 2024-09-13T18:15:13.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:13.233+0000 7f44b57fa700 1 -- 172.21.15.40:0/3151401775 wait complete. 2024-09-13T18:15:16.275 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:15:16.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:16.533+0000 7f3739c1e700 1 Processor -- start 2024-09-13T18:15:16.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:16.537+0000 7f3739c1e700 1 -- start start 2024-09-13T18:15:16.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:16.537+0000 7f3739c1e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 0x7f3734150d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:16.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:16.537+0000 7f3739c1e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37340c55c0 con 0x7f37340c7200 2024-09-13T18:15:16.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:16.537+0000 7f3739c1e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3734157710 con 0x7f3734071f70 2024-09-13T18:15:16.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:16.537+0000 7f3739c1e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f373406a390 con 0x7f3734072350 2024-09-13T18:15:16.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:16.537+0000 7f3733fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f37340c7200 0x7f3734161ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:55992/0 (socket says 172.21.15.40:55992) 2024-09-13T18:15:16.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:16.537+0000 7f3733fff700 1 -- 172.21.15.40:0/4100615866 learned_addr learned my addr 172.21.15.40:0/4100615866 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:15:16.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:16.537+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 0x7f3734150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:16.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:16.537+0000 7f3732ffd700 1 -- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 msgr2=0x7f3734150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:15:16.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:16.541+0000 7f3732ffd700 1 -- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 msgr2=0x7f3734150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:15:16.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:16.541+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 0x7f3734150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:16.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:16.541+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 0x7f3734150d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:15:16.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:16.745+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 0x7f3734150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:16.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:16.745+0000 7f3732ffd700 1 -- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 msgr2=0x7f3734150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:15:16.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:16.745+0000 7f3732ffd700 1 -- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 msgr2=0x7f3734150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:15:16.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:16.745+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 0x7f3734150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:16.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:16.745+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 0x7f3734150d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:15:17.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:17.153+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 0x7f3734150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:17.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:17.153+0000 7f3732ffd700 1 -- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 msgr2=0x7f3734150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:15:17.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:17.153+0000 7f3732ffd700 1 -- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 msgr2=0x7f3734150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:15:17.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:17.153+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 0x7f3734150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:17.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:17.153+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 0x7f3734150d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:15:17.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:17.957+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 0x7f3734150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:17.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:17.957+0000 7f3732ffd700 1 -- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 msgr2=0x7f3734150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:15:17.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:17.957+0000 7f3732ffd700 1 -- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 msgr2=0x7f3734150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:15:17.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:17.957+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 0x7f3734150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:17.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:17.957+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 0x7f3734150d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:15:17.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:17.985+0000 7f3733fff700 1 -- 172.21.15.40:0/4100615866 >> v1:172.21.15.40:6789/0 conn(0x7f37340c7200 legacy=0x7f3734161ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-13T18:15:17.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:17.985+0000 7f3733fff700 1 -- 172.21.15.40:0/4100615866 >> v1:172.21.15.40:6789/0 conn(0x7f37340c7200 legacy=0x7f3734161ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:15:17.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:17.985+0000 7f3733fff700 1 --1- 172.21.15.40:0/4100615866 >> v1:172.21.15.40:6789/0 conn(0x7f37340c7200 0x7f3734161ae0 :-1 s=OPENED pgs=27 cs=1 l=1).handle_message read tag failed 2024-09-13T18:15:17.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:17.985+0000 7f3733fff700 1 --1- 172.21.15.40:0/4100615866 >> v1:172.21.15.40:6789/0 conn(0x7f37340c7200 0x7f3734161ae0 :-1 s=OPENED pgs=27 cs=1 l=1).fault on lossy channel, failing 2024-09-13T18:15:19.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.537+0000 7f37317fa700 1 -- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 msgr2=0x7f3734150d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:15:19.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.537+0000 7f37317fa700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.43:3300/0 conn(0x7f3734072350 0x7f3734150d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:19.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.537+0000 7f37317fa700 1 -- 172.21.15.40:0/4100615866 >> v1:172.21.15.43:6789/0 conn(0x7f3734071f70 legacy=0x7f37340c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:19.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.537+0000 7f37317fa700 1 -- 172.21.15.40:0/4100615866 >> v1:172.21.15.40:6789/0 conn(0x7f37340c7200 legacy=0x7f3734161ae0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:15:19.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.537+0000 7f37317fa700 1 --2- 172.21.15.40:0/4100615866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37200036c0 0x7f3720005b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:19.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.537+0000 7f37317fa700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.40:3300/0 conn(0x7f37200061a0 0x7f3720008640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:19.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.537+0000 7f37317fa700 1 -- 172.21.15.40:0/4100615866 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3720008cd0 con 0x7f37200061a0 2024-09-13T18:15:19.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.537+0000 7f37317fa700 1 -- 172.21.15.40:0/4100615866 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3720008e40 con 0x7f37200036c0 2024-09-13T18:15:19.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.537+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.40:3300/0 conn(0x7f37200061a0 0x7f3720008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:19.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.537+0000 7f37337fe700 1 --2- 172.21.15.40:0/4100615866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37200036c0 0x7f3720005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:19.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.537+0000 7f3732ffd700 1 -- 172.21.15.40:0/4100615866 >> v2:172.21.15.40:3300/0 conn(0x7f37200061a0 msgr2=0x7f3720008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:15:19.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.537+0000 7f3732ffd700 1 -- 172.21.15.40:0/4100615866 >> v2:172.21.15.40:3300/0 conn(0x7f37200061a0 msgr2=0x7f3720008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:15:19.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.537+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.40:3300/0 conn(0x7f37200061a0 0x7f3720008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:19.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.537+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.40:3300/0 conn(0x7f37200061a0 0x7f3720008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:15:19.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.537+0000 7f37337fe700 1 -- 172.21.15.40:0/4100615866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37200036c0 msgr2=0x7f3720005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:15:19.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.537+0000 7f37337fe700 1 -- 172.21.15.40:0/4100615866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37200036c0 msgr2=0x7f3720005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:15:19.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.537+0000 7f37337fe700 1 --2- 172.21.15.40:0/4100615866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37200036c0 0x7f3720005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:19.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.537+0000 7f37337fe700 1 --2- 172.21.15.40:0/4100615866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37200036c0 0x7f3720005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:15:19.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.741+0000 7f37337fe700 1 --2- 172.21.15.40:0/4100615866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37200036c0 0x7f3720005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:19.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.741+0000 7f37337fe700 1 -- 172.21.15.40:0/4100615866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37200036c0 msgr2=0x7f3720005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:15:19.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.741+0000 7f37337fe700 1 -- 172.21.15.40:0/4100615866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37200036c0 msgr2=0x7f3720005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:15:19.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.741+0000 7f37337fe700 1 --2- 172.21.15.40:0/4100615866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37200036c0 0x7f3720005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:19.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.741+0000 7f37337fe700 1 --2- 172.21.15.40:0/4100615866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37200036c0 0x7f3720005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:15:19.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.741+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.40:3300/0 conn(0x7f37200061a0 0x7f3720008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:19.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.741+0000 7f3732ffd700 1 -- 172.21.15.40:0/4100615866 >> v2:172.21.15.40:3300/0 conn(0x7f37200061a0 msgr2=0x7f3720008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:15:19.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.741+0000 7f3732ffd700 1 -- 172.21.15.40:0/4100615866 >> v2:172.21.15.40:3300/0 conn(0x7f37200061a0 msgr2=0x7f3720008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:15:19.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.741+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.40:3300/0 conn(0x7f37200061a0 0x7f3720008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:19.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:19.741+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.40:3300/0 conn(0x7f37200061a0 0x7f3720008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:15:20.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.145+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.40:3300/0 conn(0x7f37200061a0 0x7f3720008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:20.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.145+0000 7f3732ffd700 1 -- 172.21.15.40:0/4100615866 >> v2:172.21.15.40:3300/0 conn(0x7f37200061a0 msgr2=0x7f3720008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:15:20.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.145+0000 7f37337fe700 1 --2- 172.21.15.40:0/4100615866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37200036c0 0x7f3720005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:20.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.145+0000 7f3732ffd700 1 -- 172.21.15.40:0/4100615866 >> v2:172.21.15.40:3300/0 conn(0x7f37200061a0 msgr2=0x7f3720008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:15:20.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.145+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.40:3300/0 conn(0x7f37200061a0 0x7f3720008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:20.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.145+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.40:3300/0 conn(0x7f37200061a0 0x7f3720008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:15:20.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.145+0000 7f37337fe700 1 -- 172.21.15.40:0/4100615866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37200036c0 msgr2=0x7f3720005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:15:20.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.145+0000 7f37337fe700 1 -- 172.21.15.40:0/4100615866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37200036c0 msgr2=0x7f3720005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:15:20.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.145+0000 7f37337fe700 1 --2- 172.21.15.40:0/4100615866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37200036c0 0x7f3720005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:20.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.145+0000 7f37337fe700 1 --2- 172.21.15.40:0/4100615866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37200036c0 0x7f3720005b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:15:20.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.949+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> v2:172.21.15.40:3300/0 conn(0x7f37200061a0 0x7f3720008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:20.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.949+0000 7f37337fe700 1 --2- 172.21.15.40:0/4100615866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37200036c0 0x7f3720005b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:20.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.949+0000 7f3732ffd700 1 -- 172.21.15.40:0/4100615866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37200036c0 msgr2=0x7f3720005b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:15:20.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.949+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37200036c0 0x7f3720005b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:20.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.949+0000 7f3732ffd700 1 -- 172.21.15.40:0/4100615866 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37341579d0 con 0x7f37200061a0 2024-09-13T18:15:20.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.949+0000 7f3732ffd700 1 --2- 172.21.15.40:0/4100615866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37200061a0 0x7f3720008640 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3724009100 tx=0x7f3724008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8a3b61fc59b5d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:20.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.949+0000 7f37327fc700 1 -- 172.21.15.40:0/4100615866 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724015040 con 0x7f37200061a0 2024-09-13T18:15:20.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.949+0000 7f37327fc700 1 -- 172.21.15.40:0/4100615866 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3724016040 con 0x7f37200061a0 2024-09-13T18:15:20.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.949+0000 7f37327fc700 1 -- 172.21.15.40:0/4100615866 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724004020 con 0x7f37200061a0 2024-09-13T18:15:20.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.949+0000 7f3739c1e700 1 -- 172.21.15.40:0/4100615866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37200061a0 msgr2=0x7f3720008640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:20.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.949+0000 7f3739c1e700 1 --2- 172.21.15.40:0/4100615866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37200061a0 0x7f3720008640 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3724009100 tx=0x7f3724008040 comp rx=0 tx=0).stop 2024-09-13T18:15:20.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.949+0000 7f3733fff700 1 -- 172.21.15.40:0/4100615866 reap_dead start 2024-09-13T18:15:20.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.949+0000 7f3739c1e700 1 -- 172.21.15.40:0/4100615866 shutdown_connections 2024-09-13T18:15:20.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.949+0000 7f3739c1e700 1 --2- 172.21.15.40:0/4100615866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37200061a0 0x7f3720008640 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:20.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.949+0000 7f3739c1e700 1 -- 172.21.15.40:0/4100615866 >> 172.21.15.40:0/4100615866 conn(0x7f37340cabc0 msgr2=0x7f37340cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:20.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.949+0000 7f3739c1e700 1 -- 172.21.15.40:0/4100615866 shutdown_connections 2024-09-13T18:15:20.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.957+0000 7f3739c1e700 1 -- 172.21.15.40:0/4100615866 wait complete. 2024-09-13T18:15:20.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.957+0000 7f3739c1e700 1 Processor -- start 2024-09-13T18:15:20.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.957+0000 7f3739c1e700 1 -- start start 2024-09-13T18:15:20.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.957+0000 7f3739c1e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37200061a0 0x7f3734135140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:20.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.957+0000 7f3739c1e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f373413e550 0x7f37341422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:20.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.957+0000 7f3739c1e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f373413d990 0x7f3734143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:20.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.957+0000 7f3733fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f373413d990 0x7f3734143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:20.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.957+0000 7f3733fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f373413d990 0x7f3734143a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60220/0 (socket says 172.21.15.40:60220) 2024-09-13T18:15:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.961+0000 7f3733fff700 1 -- 172.21.15.40:0/2347315789 learned_addr learned my addr 172.21.15.40:0/2347315789 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.961+0000 7f3739c1e700 1 -- 172.21.15.40:0/2347315789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373413d500 con 0x7f373413d990 2024-09-13T18:15:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.961+0000 7f3739c1e700 1 -- 172.21.15.40:0/2347315789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373413bb00 con 0x7f37200061a0 2024-09-13T18:15:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.961+0000 7f3739c1e700 1 -- 172.21.15.40:0/2347315789 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373413bde0 con 0x7f373413e550 2024-09-13T18:15:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.961+0000 7f3733fff700 1 -- 172.21.15.40:0/2347315789 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f373413e550 msgr2=0x7f37341422c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.961+0000 7f3733fff700 1 --2- 172.21.15.40:0/2347315789 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f373413e550 0x7f37341422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.961+0000 7f3733fff700 1 -- 172.21.15.40:0/2347315789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37200061a0 msgr2=0x7f3734135140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.961+0000 7f3733fff700 1 --2- 172.21.15.40:0/2347315789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37200061a0 0x7f3734135140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:20.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.961+0000 7f3733fff700 1 -- 172.21.15.40:0/2347315789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f372400e920 con 0x7f373413d990 2024-09-13T18:15:20.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.961+0000 7f3733fff700 1 --2- 172.21.15.40:0/2347315789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f373413d990 0x7f3734143a80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f373406fa80 tx=0x7f372c00cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:20.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.965+0000 7f371bfff700 1 -- 172.21.15.40:0/2347315789 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c0074e0 con 0x7f373413d990 2024-09-13T18:15:20.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.965+0000 7f371bfff700 1 -- 172.21.15.40:0/2347315789 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372c007640 con 0x7f373413d990 2024-09-13T18:15:20.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.965+0000 7f3739c1e700 1 -- 172.21.15.40:0/2347315789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373413ceb0 con 0x7f373413d990 2024-09-13T18:15:20.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.965+0000 7f371bfff700 1 -- 172.21.15.40:0/2347315789 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c0079b0 con 0x7f373413d990 2024-09-13T18:15:20.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.965+0000 7f3739c1e700 1 -- 172.21.15.40:0/2347315789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f373413ddf0 con 0x7f373413d990 2024-09-13T18:15:20.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.969+0000 7f371bfff700 1 -- 172.21.15.40:0/2347315789 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f372c01b3b0 con 0x7f373413d990 2024-09-13T18:15:20.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.973+0000 7f371bfff700 1 --2- 172.21.15.40:0/2347315789 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f37200676e0 0x7f3720069ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:20.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.973+0000 7f371bfff700 1 -- 172.21.15.40:0/2347315789 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f372c083d00 con 0x7f373413d990 2024-09-13T18:15:20.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.973+0000 7f37337fe700 1 --2- 172.21.15.40:0/2347315789 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f37200676e0 0x7f3720069ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:20.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.973+0000 7f37337fe700 1 --2- 172.21.15.40:0/2347315789 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f37200676e0 0x7f3720069ba0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3728007fd0 tx=0x7f3728000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:20.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.977+0000 7f3719ffb700 1 -- 172.21.15.40:0/2347315789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f373413cd10 con 0x7f373413d990 2024-09-13T18:15:20.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:20.985+0000 7f371bfff700 1 -- 172.21.15.40:0/2347315789 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f373413cd10 con 0x7f373413d990 2024-09-13T18:15:21.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:21.853+0000 7f3719ffb700 1 -- 172.21.15.40:0/2347315789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f373413cd10 con 0x7f373413d990 2024-09-13T18:15:21.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:21.853+0000 7f371bfff700 1 -- 172.21.15.40:0/2347315789 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f373413cd10 con 0x7f373413d990 2024-09-13T18:15:21.861 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":204,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:15:21.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:21.857+0000 7f3719ffb700 1 -- 172.21.15.40:0/2347315789 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f37200676e0 msgr2=0x7f3720069ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:21.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:21.857+0000 7f3719ffb700 1 --2- 172.21.15.40:0/2347315789 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f37200676e0 0x7f3720069ba0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3728007fd0 tx=0x7f3728000ed0 comp rx=0 tx=0).stop 2024-09-13T18:15:21.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:21.857+0000 7f3719ffb700 1 -- 172.21.15.40:0/2347315789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f373413d990 msgr2=0x7f3734143a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:21.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:21.857+0000 7f3719ffb700 1 --2- 172.21.15.40:0/2347315789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f373413d990 0x7f3734143a80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f373406fa80 tx=0x7f372c00cb20 comp rx=0 tx=0).stop 2024-09-13T18:15:21.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:21.857+0000 7f3719ffb700 1 -- 172.21.15.40:0/2347315789 shutdown_connections 2024-09-13T18:15:21.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:21.857+0000 7f3719ffb700 1 --2- 172.21.15.40:0/2347315789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f373413d990 0x7f3734143a80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:21.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:21.857+0000 7f3719ffb700 1 --2- 172.21.15.40:0/2347315789 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f37200676e0 0x7f3720069ba0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:21.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:21.857+0000 7f3719ffb700 1 --2- 172.21.15.40:0/2347315789 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f373413e550 0x7f37341422c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:21.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:21.857+0000 7f3719ffb700 1 --2- 172.21.15.40:0/2347315789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37200061a0 0x7f3734135140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:21.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:21.857+0000 7f3719ffb700 1 -- 172.21.15.40:0/2347315789 >> 172.21.15.40:0/2347315789 conn(0x7f37340cabc0 msgr2=0x7f3734150b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:21.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:21.857+0000 7f3719ffb700 1 -- 172.21.15.40:0/2347315789 shutdown_connections 2024-09-13T18:15:21.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:21.857+0000 7f3719ffb700 1 -- 172.21.15.40:0/2347315789 wait complete. 2024-09-13T18:15:21.902 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:15:21.902 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:15:21.902 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:15:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.217+0000 7fbdfbaa5700 1 Processor -- start 2024-09-13T18:15:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.217+0000 7fbdfbaa5700 1 -- start start 2024-09-13T18:15:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.217+0000 7fbdfbaa5700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbdf408b230 0x7fbdf4166d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.217+0000 7fbdfbaa5700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbdf406a390 0x7fbdf406a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.217+0000 7fbdfbaa5700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf41672b0 con 0x7fbdf4163a90 2024-09-13T18:15:22.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.217+0000 7fbdfbaa5700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdf406fee0 con 0x7fbdf406a390 2024-09-13T18:15:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.217+0000 7fbdfbaa5700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdf40c5f70 con 0x7fbdf408b230 2024-09-13T18:15:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.221+0000 7fbdf9040700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbdf408b230 0x7fbdf4166d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.221+0000 7fbdf9040700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbdf408b230 0x7fbdf4166d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:50708/0 (socket says 172.21.15.40:50708) 2024-09-13T18:15:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.221+0000 7fbdf9040700 1 -- 172.21.15.40:0/4087678326 learned_addr learned my addr 172.21.15.40:0/4087678326 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.221+0000 7fbdfa042700 1 --2- 172.21.15.40:0/4087678326 >> v2:172.21.15.40:3300/0 conn(0x7fbdf406a390 0x7fbdf406a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.229+0000 7fbdfa042700 1 -- 172.21.15.40:0/4087678326 >> v2:172.21.15.43:3300/0 conn(0x7fbdf408b230 msgr2=0x7fbdf4166d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:15:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.229+0000 7fbdfa042700 1 --2- 172.21.15.40:0/4087678326 >> v2:172.21.15.43:3300/0 conn(0x7fbdf408b230 0x7fbdf4166d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.229+0000 7fbdfa042700 1 -- 172.21.15.40:0/4087678326 >> v1:172.21.15.40:6789/0 conn(0x7fbdf4163a90 legacy=0x7fbdf40c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.229+0000 7fbdfa042700 1 -- 172.21.15.40:0/4087678326 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf406afb0 con 0x7fbdf406a390 2024-09-13T18:15:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.229+0000 7fbdfa042700 1 --2- 172.21.15.40:0/4087678326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf406a390 0x7fbdf406a7e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbdf406f4d0 tx=0x7fbdf0010e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcd2338b652ca1c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.229+0000 7fbdf883f700 1 -- 172.21.15.40:0/4087678326 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf00100d0 con 0x7fbdf406a390 2024-09-13T18:15:22.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.229+0000 7fbdf883f700 1 -- 172.21.15.40:0/4087678326 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf0010230 con 0x7fbdf406a390 2024-09-13T18:15:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.229+0000 7fbdfbaa5700 1 -- 172.21.15.40:0/4087678326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf406a390 msgr2=0x7fbdf406a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.229+0000 7fbdfbaa5700 1 --2- 172.21.15.40:0/4087678326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf406a390 0x7fbdf406a7e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbdf406f4d0 tx=0x7fbdf0010e70 comp rx=0 tx=0).stop 2024-09-13T18:15:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.229+0000 7fbdfbaa5700 1 -- 172.21.15.40:0/4087678326 shutdown_connections 2024-09-13T18:15:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.229+0000 7fbdfbaa5700 1 --2- 172.21.15.40:0/4087678326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf406a390 0x7fbdf406a7e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.229+0000 7fbdfbaa5700 1 --2- 172.21.15.40:0/4087678326 >> v2:172.21.15.43:3300/0 conn(0x7fbdf408b230 0x7fbdf4166d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.229+0000 7fbdfbaa5700 1 -- 172.21.15.40:0/4087678326 >> 172.21.15.40:0/4087678326 conn(0x7fbdf406d5b0 msgr2=0x7fbdf406d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.233+0000 7fbdfbaa5700 1 -- 172.21.15.40:0/4087678326 shutdown_connections 2024-09-13T18:15:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.233+0000 7fbdfbaa5700 1 -- 172.21.15.40:0/4087678326 wait complete. 2024-09-13T18:15:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.237+0000 7fbdfbaa5700 1 Processor -- start 2024-09-13T18:15:22.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.237+0000 7fbdfbaa5700 1 -- start start 2024-09-13T18:15:22.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.237+0000 7fbdfbaa5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf406a390 0x7fbdf4165800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:22.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.237+0000 7fbdfbaa5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdf408b230 0x7fbdf420ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:22.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.237+0000 7fbdfbaa5700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbdf420cfc0 0x7fbdf415f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:22.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.237+0000 7fbdfbaa5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf40c4080 con 0x7fbdf406a390 2024-09-13T18:15:22.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.237+0000 7fbdfbaa5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf40c39d0 con 0x7fbdf408b230 2024-09-13T18:15:22.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.237+0000 7fbdfbaa5700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf41706c0 con 0x7fbdf420cfc0 2024-09-13T18:15:22.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.241+0000 7fbdf9040700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdf408b230 0x7fbdf420ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.241+0000 7fbdf9040700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdf408b230 0x7fbdf420ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:50716/0 (socket says 172.21.15.40:50716) 2024-09-13T18:15:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.241+0000 7fbdf9040700 1 -- 172.21.15.40:0/456993619 learned_addr learned my addr 172.21.15.40:0/456993619 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.241+0000 7fbdf9841700 1 --2- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf406a390 0x7fbdf4165800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.241+0000 7fbdf9040700 1 -- 172.21.15.40:0/456993619 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbdf420cfc0 msgr2=0x7fbdf415f3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:15:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.241+0000 7fbdf9040700 1 --2- 172.21.15.40:0/456993619 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbdf420cfc0 0x7fbdf415f3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.241+0000 7fbdf9040700 1 -- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf406a390 msgr2=0x7fbdf4165800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.241+0000 7fbdf9040700 1 --2- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf406a390 0x7fbdf4165800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:22.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.241+0000 7fbdf9040700 1 -- 172.21.15.40:0/456993619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf00107f0 con 0x7fbdf408b230 2024-09-13T18:15:22.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.241+0000 7fbdf9040700 1 --2- 172.21.15.40:0/456993619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdf408b230 0x7fbdf420ca80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00dce0 tx=0x7fbdec00b0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:22.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.253+0000 7fbdea7fc700 1 -- 172.21.15.40:0/456993619 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec00bc10 con 0x7fbdf408b230 2024-09-13T18:15:22.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.253+0000 7fbdea7fc700 1 -- 172.21.15.40:0/456993619 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdec00bd70 con 0x7fbdf408b230 2024-09-13T18:15:22.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.253+0000 7fbdea7fc700 1 -- 172.21.15.40:0/456993619 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec012680 con 0x7fbdf408b230 2024-09-13T18:15:22.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.253+0000 7fbdfbaa5700 1 -- 172.21.15.40:0/456993619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf40c4530 con 0x7fbdf408b230 2024-09-13T18:15:22.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.253+0000 7fbdfbaa5700 1 -- 172.21.15.40:0/456993619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf40c3e20 con 0x7fbdf408b230 2024-09-13T18:15:22.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.257+0000 7fbdfbaa5700 1 --2- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf4167450 0x7fbdf415f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:22.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.257+0000 7fbdea7fc700 1 -- 172.21.15.40:0/456993619 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbdec010070 con 0x7fbdf408b230 2024-09-13T18:15:22.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.257+0000 7fbdfbaa5700 1 -- 172.21.15.40:0/456993619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf4067840 con 0x7fbdf4167450 2024-09-13T18:15:22.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.257+0000 7fbdea7fc700 1 --2- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbdd80667b0 0x7fbdd8068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:22.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.257+0000 7fbdfa042700 1 --2- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbdd80667b0 0x7fbdd8068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:22.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.257+0000 7fbdf9841700 1 --2- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf4167450 0x7fbdf415f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:22.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.257+0000 7fbdf9841700 1 -- 172.21.15.40:0/456993619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbdf40721d0 con 0x7fbdf4167450 2024-09-13T18:15:22.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.257+0000 7fbdfa042700 1 --2- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbdd80667b0 0x7fbdd8068c70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbdf420d430 tx=0x7fbdf000f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:22.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.257+0000 7fbdf9841700 1 --2- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf4167450 0x7fbdf415f8f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbdf406f500 tx=0x7fbde400fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:22.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.261+0000 7fbdea7fc700 1 -- 172.21.15.40:0/456993619 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbdec0848c0 con 0x7fbdf408b230 2024-09-13T18:15:22.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.261+0000 7fbdea7fc700 1 -- 172.21.15.40:0/456993619 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde4005320 con 0x7fbdf4167450 2024-09-13T18:15:22.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.265+0000 7fbdea7fc700 1 -- 172.21.15.40:0/456993619 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fbde4005480 con 0x7fbdf4167450 2024-09-13T18:15:22.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.265+0000 7fbdea7fc700 1 -- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf4167450 msgr2=0x7fbdf415f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:22.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.265+0000 7fbdea7fc700 1 --2- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf4167450 0x7fbdf415f8f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbdf406f500 tx=0x7fbde400fb00 comp rx=0 tx=0).stop 2024-09-13T18:15:22.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.265+0000 7fbdea7fc700 1 -- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf4167450 msgr2=0x7fbdf415f8f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:15:22.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.265+0000 7fbdea7fc700 1 --2- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf4167450 0x7fbdf415f8f0 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbdf406f500 tx=0x7fbde400fb00 comp rx=0 tx=0).stop 2024-09-13T18:15:22.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.285+0000 7fbdfbaa5700 1 --2- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf4210a00 0x7fbdf40c2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:22.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.285+0000 7fbdf9841700 1 --2- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf4210a00 0x7fbdf40c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:22.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.285+0000 7fbdfbaa5700 1 -- 172.21.15.40:0/456993619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf4167cb0 con 0x7fbdf4210a00 2024-09-13T18:15:22.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.289+0000 7fbdf9841700 1 -- 172.21.15.40:0/456993619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fbdf406b4e0 con 0x7fbdf4210a00 2024-09-13T18:15:22.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.289+0000 7fbdf9841700 1 --2- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf4210a00 0x7fbdf40c2090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbdf406f500 tx=0x7fbde4007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.289+0000 7fbdea7fc700 1 -- 172.21.15.40:0/456993619 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde4023070 con 0x7fbdf4210a00 2024-09-13T18:15:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.289+0000 7fbdea7fc700 1 -- 172.21.15.40:0/456993619 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4452 (secure 0 0 0) 0x7fbde401c300 con 0x7fbdf4210a00 2024-09-13T18:15:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.289+0000 7fbdea7fc700 1 -- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf4210a00 msgr2=0x7fbdf40c2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.289+0000 7fbdea7fc700 1 --2- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf4210a00 0x7fbdf40c2090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbdf406f500 tx=0x7fbde4007f90 comp rx=0 tx=0).stop 2024-09-13T18:15:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.289+0000 7fbdea7fc700 1 -- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf4210a00 msgr2=0x7fbdf40c2090 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:15:22.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.289+0000 7fbdea7fc700 1 --2- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbdf4210a00 0x7fbdf40c2090 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbdf406f500 tx=0x7fbde4007f90 comp rx=0 tx=0).stop 2024-09-13T18:15:22.297 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:15:22.297 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:15:22.297 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:15:22.297 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:15:22.297 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 204, 2024-09-13T18:15:22.297 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:15:22.297 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:15:22.297 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:15:22.297 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:15:22.297 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:22.298 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:15:22.298 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:15:22.298 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:15:22.298 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:15:22.298 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:22.298 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:22.298 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:22.298 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:22.298 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:22.298 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:22.298 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:22.298 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:22.298 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:22.299 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:22.299 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:22.299 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:15:22.299 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:15:22.299 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:22.299 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:22.299 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:22.299 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:22.299 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:22.299 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:22.299 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:22.299 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:22.300 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:22.300 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:22.300 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:22.300 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.300 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:15:22.300 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:15:22.300 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:15:22.300 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:15:22.300 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:15:22.301 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:15:22.301 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:15:22.301 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:15:22.301 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:15:22.301 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:15:22.302 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:15:22.302 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:15:22.302 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:15:22.302 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:15:22.302 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:15:22.302 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:15:22.302 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:15:22.302 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:22.303 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:22.303 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:22.303 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:22.303 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:22.303 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:22.303 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:22.303 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:22.303 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:22.303 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:22.303 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:22.303 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:15:22.304 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.304 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:15:22.304 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.304 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:15:22.304 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:15:22.304 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:22.304 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:22.304 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.304 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:22.304 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:15:22.305 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:22.305 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.305 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.305 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:22.305 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:15:22.305 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:22.305 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:22.305 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:22.305 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.305 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:15:22.306 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:15:22.306 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:22.306 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:22.306 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:22.306 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.306 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.307 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:15:22.307 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:15:22.307 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:22.307 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:22.307 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.307 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:22.307 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:15:22.307 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:22.307 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.307 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.307 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:22.308 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:15:22.308 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:22.308 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:22.308 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:22.308 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.308 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:15:22.308 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:15:22.308 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:22.308 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:22.308 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:22.308 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.308 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.309 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:15:22.309 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:15:22.309 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:22.309 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:22.309 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.309 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:22.309 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:15:22.309 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:22.309 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.310 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.310 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:22.310 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:15:22.310 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:22.310 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:22.310 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:22.310 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.310 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:15:22.310 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:15:22.310 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:22.310 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:22.310 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:22.310 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:22.311 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:22.311 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.311 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:15:22.311 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:15:22.311 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.311 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:22.311 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:22.311 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:15:22.311 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:22.311 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:22.312 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:15:22.312 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.312 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:22.312 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:22.312 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:15:22.312 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:22.312 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:22.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.312 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:15:22.312 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:15:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.293+0000 7fbdfbaa5700 1 -- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbdd80667b0 msgr2=0x7fbdd8068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.293+0000 7fbdfbaa5700 1 --2- 172.21.15.40:0/456993619 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbdd80667b0 0x7fbdd8068c70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbdf420d430 tx=0x7fbdf000f000 comp rx=0 tx=0).stop 2024-09-13T18:15:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.293+0000 7fbdfbaa5700 1 -- 172.21.15.40:0/456993619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdf408b230 msgr2=0x7fbdf420ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.293+0000 7fbdfbaa5700 1 --2- 172.21.15.40:0/456993619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdf408b230 0x7fbdf420ca80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00dce0 tx=0x7fbdec00b0d0 comp rx=0 tx=0).stop 2024-09-13T18:15:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.293+0000 7fbdfa042700 1 -- 172.21.15.40:0/456993619 reap_dead start 2024-09-13T18:15:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.293+0000 7fbdfa042700 1 -- 172.21.15.40:0/456993619 reap_dead start 2024-09-13T18:15:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.293+0000 7fbdfbaa5700 1 -- 172.21.15.40:0/456993619 shutdown_connections 2024-09-13T18:15:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.293+0000 7fbdfbaa5700 1 -- 172.21.15.40:0/456993619 >> 172.21.15.40:0/456993619 conn(0x7fbdf406d5b0 msgr2=0x7fbdf40ca130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.293+0000 7fbdfbaa5700 1 -- 172.21.15.40:0/456993619 shutdown_connections 2024-09-13T18:15:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.293+0000 7fbdfbaa5700 1 -- 172.21.15.40:0/456993619 wait complete. 2024-09-13T18:15:22.329 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:15:22.329 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:15:22.330 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:15:22.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.657+0000 7f20f2e24700 1 Processor -- start 2024-09-13T18:15:22.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.657+0000 7f20f2e24700 1 -- start start 2024-09-13T18:15:22.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.661+0000 7f20f2e24700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f20ec06b080 0x7f20ec0c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:22.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.661+0000 7f20f2e24700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f20ec0c61e0 0x7f20ec06d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:22.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.661+0000 7f20f2e24700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ec167290 con 0x7f20ec163c40 2024-09-13T18:15:22.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.661+0000 7f20f2e24700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ec0c6610 con 0x7f20ec06b080 2024-09-13T18:15:22.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.661+0000 7f20f2e24700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ec166680 con 0x7f20ec0c61e0 2024-09-13T18:15:22.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.661+0000 7f20f2623700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f20ec0c61e0 0x7f20ec06d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:22.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.661+0000 7f20f2623700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f20ec0c61e0 0x7f20ec06d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:50730/0 (socket says 172.21.15.40:50730) 2024-09-13T18:15:22.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.661+0000 7f20f2623700 1 -- 172.21.15.40:0/1440142052 learned_addr learned my addr 172.21.15.40:0/1440142052 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:22.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.661+0000 7f20f1621700 1 --2- 172.21.15.40:0/1440142052 >> v2:172.21.15.40:3300/0 conn(0x7f20ec06b080 0x7f20ec0c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:22.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.661+0000 7f20f1621700 1 -- 172.21.15.40:0/1440142052 >> v2:172.21.15.43:3300/0 conn(0x7f20ec0c61e0 msgr2=0x7f20ec06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:15:22.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.661+0000 7f20f1621700 1 --2- 172.21.15.40:0/1440142052 >> v2:172.21.15.43:3300/0 conn(0x7f20ec0c61e0 0x7f20ec06d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:22.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.661+0000 7f20f1621700 1 -- 172.21.15.40:0/1440142052 >> v1:172.21.15.40:6789/0 conn(0x7f20ec163c40 legacy=0x7f20ec0c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:15:22.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.661+0000 7f20f1621700 1 -- 172.21.15.40:0/1440142052 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec166960 con 0x7f20ec06b080 2024-09-13T18:15:22.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.661+0000 7f20f1621700 1 --2- 172.21.15.40:0/1440142052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20ec06b080 0x7f20ec0c7250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f20ec06cf30 tx=0x7f20e400f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd8c056d7167924 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:22.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.661+0000 7f20f0e20700 1 -- 172.21.15.40:0/1440142052 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e4003a40 con 0x7f20ec06b080 2024-09-13T18:15:22.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.661+0000 7f20f0e20700 1 -- 172.21.15.40:0/1440142052 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e4003ba0 con 0x7f20ec06b080 2024-09-13T18:15:22.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.661+0000 7f20f0e20700 1 -- 172.21.15.40:0/1440142052 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e400d0d0 con 0x7f20ec06b080 2024-09-13T18:15:22.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.665+0000 7f20f2e24700 1 -- 172.21.15.40:0/1440142052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20ec06b080 msgr2=0x7f20ec0c7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:22.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.665+0000 7f20f2e24700 1 --2- 172.21.15.40:0/1440142052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20ec06b080 0x7f20ec0c7250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f20ec06cf30 tx=0x7f20e400f650 comp rx=0 tx=0).stop 2024-09-13T18:15:22.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.665+0000 7f20f2e24700 1 -- 172.21.15.40:0/1440142052 shutdown_connections 2024-09-13T18:15:22.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.665+0000 7f20f2e24700 1 --2- 172.21.15.40:0/1440142052 >> v2:172.21.15.43:3300/0 conn(0x7f20ec0c61e0 0x7f20ec06d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:22.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.665+0000 7f20f2e24700 1 --2- 172.21.15.40:0/1440142052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20ec06b080 0x7f20ec0c7250 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:22.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.665+0000 7f20f2e24700 1 -- 172.21.15.40:0/1440142052 >> 172.21.15.40:0/1440142052 conn(0x7f20ec066090 msgr2=0x7f20ec0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:22.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.665+0000 7f20f2e24700 1 -- 172.21.15.40:0/1440142052 shutdown_connections 2024-09-13T18:15:22.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.665+0000 7f20f2e24700 1 -- 172.21.15.40:0/1440142052 wait complete. 2024-09-13T18:15:22.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.669+0000 7f20f2e24700 1 Processor -- start 2024-09-13T18:15:22.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.669+0000 7f20f2e24700 1 -- start start 2024-09-13T18:15:22.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.669+0000 7f20f2e24700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec06b080 0x7f20ec15aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:22.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.669+0000 7f20f2e24700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20ec0c61e0 0x7f20ec0c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:22.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.669+0000 7f20f2e24700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f20ec163c40 0x7f20ec0ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:22.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.669+0000 7f20f2e24700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec0c9030 con 0x7f20ec0c61e0 2024-09-13T18:15:22.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.669+0000 7f20f2e24700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec0ca2f0 con 0x7f20ec06b080 2024-09-13T18:15:22.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.669+0000 7f20f2e24700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec0c8d50 con 0x7f20ec163c40 2024-09-13T18:15:22.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.669+0000 7f20f1621700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20ec0c61e0 0x7f20ec0c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:22.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.669+0000 7f20f1621700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20ec0c61e0 0x7f20ec0c9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53034/0 (socket says 172.21.15.40:53034) 2024-09-13T18:15:22.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.669+0000 7f20f1621700 1 -- 172.21.15.40:0/4024376762 learned_addr learned my addr 172.21.15.40:0/4024376762 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:22.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.669+0000 7f20f1e22700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec06b080 0x7f20ec15aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:22.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.669+0000 7f20f1621700 1 -- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f20ec163c40 msgr2=0x7f20ec0ca610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:15:22.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.669+0000 7f20f2623700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f20ec163c40 0x7f20ec0ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:22.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.673+0000 7f20f1621700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f20ec163c40 0x7f20ec0ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:22.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.673+0000 7f20f1621700 1 -- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec06b080 msgr2=0x7f20ec15aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:22.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.673+0000 7f20f1621700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec06b080 0x7f20ec15aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:22.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.673+0000 7f20f1621700 1 -- 172.21.15.40:0/4024376762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e400f050 con 0x7f20ec0c61e0 2024-09-13T18:15:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.673+0000 7f20f1621700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20ec0c61e0 0x7f20ec0c9720 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f20e400f690 tx=0x7f20e4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.673+0000 7f20f1e22700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec06b080 0x7f20ec15aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:15:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.673+0000 7f20f2623700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f20ec163c40 0x7f20ec0ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:15:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.673+0000 7f20e27fc700 1 -- 172.21.15.40:0/4024376762 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e4018ce0 con 0x7f20ec0c61e0 2024-09-13T18:15:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.673+0000 7f20f2e24700 1 -- 172.21.15.40:0/4024376762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ec1582b0 con 0x7f20ec0c61e0 2024-09-13T18:15:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.673+0000 7f20f2e24700 1 -- 172.21.15.40:0/4024376762 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ec158870 con 0x7f20ec0c61e0 2024-09-13T18:15:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.673+0000 7f20e27fc700 1 -- 172.21.15.40:0/4024376762 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e4003e90 con 0x7f20ec0c61e0 2024-09-13T18:15:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.673+0000 7f20e27fc700 1 -- 172.21.15.40:0/4024376762 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e400b980 con 0x7f20ec0c61e0 2024-09-13T18:15:22.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.677+0000 7f20e27fc700 1 -- 172.21.15.40:0/4024376762 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f20e4009070 con 0x7f20ec0c61e0 2024-09-13T18:15:22.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.677+0000 7f20cffff700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec0c9c60 0x7f20ec0cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:22.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.677+0000 7f20cffff700 1 -- 172.21.15.40:0/4024376762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec067720 con 0x7f20ec0c9c60 2024-09-13T18:15:22.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.677+0000 7f20e27fc700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f20d0066800 0x7f20d0068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:22.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.677+0000 7f20f1e22700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec0c9c60 0x7f20ec0cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:22.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.677+0000 7f20f1e22700 1 -- 172.21.15.40:0/4024376762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f20ec166370 con 0x7f20ec0c9c60 2024-09-13T18:15:22.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.681+0000 7f20f2623700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f20d0066800 0x7f20d0068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:22.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.681+0000 7f20e27fc700 1 -- 172.21.15.40:0/4024376762 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f20e408f150 con 0x7f20ec0c61e0 2024-09-13T18:15:22.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.681+0000 7f20f2623700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f20d0066800 0x7f20d0068cc0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f20dc003950 tx=0x7f20dc00d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:22.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.685+0000 7f20f1e22700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec0c9c60 0x7f20ec0cab50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f20e800c720 tx=0x7f20e800cae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:22.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.685+0000 7f20e27fc700 1 -- 172.21.15.40:0/4024376762 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8011800 con 0x7f20ec0c9c60 2024-09-13T18:15:22.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.689+0000 7f20e27fc700 1 -- 172.21.15.40:0/4024376762 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f20e8011960 con 0x7f20ec0c9c60 2024-09-13T18:15:22.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.689+0000 7f20e27fc700 1 -- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec0c9c60 msgr2=0x7f20ec0cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:22.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.689+0000 7f20e27fc700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec0c9c60 0x7f20ec0cab50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f20e800c720 tx=0x7f20e800cae0 comp rx=0 tx=0).stop 2024-09-13T18:15:22.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.689+0000 7f20e27fc700 1 -- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec0c9c60 msgr2=0x7f20ec0cab50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:15:22.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.689+0000 7f20e27fc700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec0c9c60 0x7f20ec0cab50 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f20e800c720 tx=0x7f20e800cae0 comp rx=0 tx=0).stop 2024-09-13T18:15:22.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.713+0000 7f20cffff700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec157b90 0x7f20ec1fed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:22.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.713+0000 7f20cffff700 1 -- 172.21.15.40:0/4024376762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec166f20 con 0x7f20ec157b90 2024-09-13T18:15:22.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.717+0000 7f20f1e22700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec157b90 0x7f20ec1fed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:22.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.717+0000 7f20f1e22700 1 -- 172.21.15.40:0/4024376762 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f20ec0c83e0 con 0x7f20ec157b90 2024-09-13T18:15:22.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.717+0000 7f20f1e22700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec157b90 0x7f20ec1fed70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f20e800c720 tx=0x7f20e8004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:22.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.717+0000 7f20e27fc700 1 -- 172.21.15.40:0/4024376762 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8020070 con 0x7f20ec157b90 2024-09-13T18:15:22.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.721+0000 7f20e27fc700 1 -- 172.21.15.40:0/4024376762 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4616 (secure 0 0 0) 0x7f20e801bda0 con 0x7f20ec157b90 2024-09-13T18:15:22.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.721+0000 7f20e27fc700 1 -- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec157b90 msgr2=0x7f20ec1fed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:22.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.721+0000 7f20e27fc700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec157b90 0x7f20ec1fed70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f20e800c720 tx=0x7f20e8004760 comp rx=0 tx=0).stop 2024-09-13T18:15:22.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.721+0000 7f20e27fc700 1 -- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec157b90 msgr2=0x7f20ec1fed70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:15:22.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.721+0000 7f20e27fc700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20ec157b90 0x7f20ec1fed70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:22.728 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:15:22.728 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:15:22.728 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:15:22.728 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:15:22.728 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 204, 2024-09-13T18:15:22.728 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:15:22.728 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:15:22.729 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:15:22.729 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:15:22.729 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:22.729 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:15:22.729 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:15:22.729 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:15:22.730 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:15:22.730 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:22.730 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:22.730 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:22.730 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:22.730 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:22.730 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:22.730 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:22.730 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:22.730 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:22.731 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:22.731 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:22.731 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:15:22.731 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:15:22.731 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:22.731 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:22.731 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:22.731 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:22.731 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:22.731 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:22.731 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:22.731 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:22.731 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:22.732 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:22.732 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:22.732 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.732 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:15:22.732 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:15:22.732 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:15:22.732 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:15:22.732 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:15:22.732 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:15:22.732 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:15:22.732 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:15:22.732 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:15:22.732 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:15:22.732 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:15:22.733 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:15:22.733 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:15:22.733 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:15:22.733 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:15:22.733 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:15:22.733 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:15:22.733 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:22.733 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:22.733 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:22.733 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:22.733 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:22.733 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:22.733 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:22.734 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:22.734 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:22.734 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:22.734 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:22.734 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:15:22.734 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.734 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:15:22.734 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.734 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:15:22.734 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:15:22.734 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:22.734 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:22.734 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.734 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:22.735 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:15:22.735 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:22.735 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.735 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.735 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:22.735 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:15:22.735 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:22.735 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:22.736 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:22.736 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.736 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:15:22.736 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:15:22.736 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:22.736 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:22.736 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:22.736 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.736 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.736 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:15:22.736 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:15:22.736 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:22.736 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:22.737 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.737 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:22.737 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:15:22.737 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:22.737 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.737 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.737 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:22.737 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:15:22.737 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:22.737 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:22.737 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:22.737 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.737 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:15:22.737 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:15:22.738 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:22.738 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:22.738 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:22.738 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.738 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.738 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:15:22.738 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:15:22.738 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:22.738 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:22.738 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.738 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:22.738 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:15:22.738 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:22.738 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.739 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.739 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:22.739 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:15:22.739 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:22.739 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:22.739 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:22.739 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.739 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:15:22.739 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:15:22.739 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:22.739 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:22.739 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:22.739 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:22.739 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:22.740 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.740 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:15:22.740 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:15:22.740 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.740 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:22.740 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:22.740 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:15:22.740 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:22.740 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:22.740 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:15:22.740 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.740 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:22.740 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:22.741 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:15:22.741 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:22.741 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:22.741 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:15:22.741 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:22.741 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:22.741 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:22.741 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:15:22.741 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:22.741 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:22.742 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:22.742 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:15:22.742 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:15:22.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.729+0000 7f20cffff700 1 -- 172.21.15.40:0/4024376762 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f20d0066800 msgr2=0x7f20d0068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:22.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.729+0000 7f20cffff700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f20d0066800 0x7f20d0068cc0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f20dc003950 tx=0x7f20dc00d040 comp rx=0 tx=0).stop 2024-09-13T18:15:22.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.729+0000 7f20f2623700 1 -- 172.21.15.40:0/4024376762 reap_dead start 2024-09-13T18:15:22.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.729+0000 7f20cffff700 1 -- 172.21.15.40:0/4024376762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20ec0c61e0 msgr2=0x7f20ec0c9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:22.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.729+0000 7f20cffff700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20ec0c61e0 0x7f20ec0c9720 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f20e400f690 tx=0x7f20e4004af0 comp rx=0 tx=0).stop 2024-09-13T18:15:22.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.729+0000 7f20cffff700 1 -- 172.21.15.40:0/4024376762 shutdown_connections 2024-09-13T18:15:22.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.729+0000 7f20cffff700 1 --2- 172.21.15.40:0/4024376762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f20ec0c61e0 0x7f20ec0c9720 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:22.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.729+0000 7f20cffff700 1 -- 172.21.15.40:0/4024376762 >> 172.21.15.40:0/4024376762 conn(0x7f20ec066090 msgr2=0x7f20ec161c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:22.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.733+0000 7f20cffff700 1 -- 172.21.15.40:0/4024376762 shutdown_connections 2024-09-13T18:15:22.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:22.733+0000 7f20cffff700 1 -- 172.21.15.40:0/4024376762 wait complete. 2024-09-13T18:15:22.792 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:15:22.792 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:15:22.792 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:15:23.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.061+0000 7f6e90e6a700 1 Processor -- start 2024-09-13T18:15:23.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.061+0000 7f6e90e6a700 1 -- start start 2024-09-13T18:15:23.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.065+0000 7f6e90e6a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e8c0c7f60 0x7f6e8c0c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:23.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.065+0000 7f6e90e6a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6e8c163d00 0x7f6e8c0c6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:23.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.065+0000 7f6e90e6a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e8c167290 con 0x7f6e8c0c8340 2024-09-13T18:15:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.065+0000 7f6e90e6a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e8c0c1630 con 0x7f6e8c163d00 2024-09-13T18:15:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.065+0000 7f6e90e6a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c166680 con 0x7f6e8c0c7f60 2024-09-13T18:15:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.065+0000 7f6e89d9b700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6e8c0c8340 0x7f6e8c1635f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40646/0 (socket says 172.21.15.40:40646) 2024-09-13T18:15:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.065+0000 7f6e89d9b700 1 -- 172.21.15.40:0/3168541217 learned_addr learned my addr 172.21.15.40:0/3168541217 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:15:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.065+0000 7f6e8a59c700 1 --2- 172.21.15.40:0/3168541217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e8c0c7f60 0x7f6e8c0c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.065+0000 7f6e8ad9d700 1 --2- 172.21.15.40:0/3168541217 >> v2:172.21.15.40:3300/0 conn(0x7f6e8c163d00 0x7f6e8c0c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.065+0000 7f6e8a59c700 1 -- 172.21.15.40:0/3168541217 >> v2:172.21.15.40:3300/0 conn(0x7f6e8c163d00 msgr2=0x7f6e8c0c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:15:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.065+0000 7f6e8a59c700 1 --2- 172.21.15.40:0/3168541217 >> v2:172.21.15.40:3300/0 conn(0x7f6e8c163d00 0x7f6e8c0c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.065+0000 7f6e8a59c700 1 -- 172.21.15.40:0/3168541217 >> v1:172.21.15.40:6789/0 conn(0x7f6e8c0c8340 legacy=0x7f6e8c1635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.065+0000 7f6e8a59c700 1 -- 172.21.15.40:0/3168541217 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8c166960 con 0x7f6e8c0c7f60 2024-09-13T18:15:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.065+0000 7f6e8ad9d700 1 --2- 172.21.15.40:0/3168541217 >> v2:172.21.15.40:3300/0 conn(0x7f6e8c163d00 0x7f6e8c0c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:15:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.065+0000 7f6e8a59c700 1 --2- 172.21.15.40:0/3168541217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e8c0c7f60 0x7f6e8c0c7250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0088d0 tx=0x7f6e7c008be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4be6f16b1e4cde62 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:23.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.073+0000 7f6e8959a700 1 -- 172.21.15.40:0/3168541217 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c018c60 con 0x7f6e8c0c7f60 2024-09-13T18:15:23.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.073+0000 7f6e8959a700 1 -- 172.21.15.40:0/3168541217 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7c018dc0 con 0x7f6e8c0c7f60 2024-09-13T18:15:23.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.073+0000 7f6e90e6a700 1 -- 172.21.15.40:0/3168541217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e8c0c7f60 msgr2=0x7f6e8c0c7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:23.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.073+0000 7f6e90e6a700 1 --2- 172.21.15.40:0/3168541217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e8c0c7f60 0x7f6e8c0c7250 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0088d0 tx=0x7f6e7c008be0 comp rx=0 tx=0).stop 2024-09-13T18:15:23.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.081+0000 7f6e90e6a700 1 -- 172.21.15.40:0/3168541217 shutdown_connections 2024-09-13T18:15:23.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.081+0000 7f6e90e6a700 1 --2- 172.21.15.40:0/3168541217 >> v2:172.21.15.40:3300/0 conn(0x7f6e8c163d00 0x7f6e8c0c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:23.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.081+0000 7f6e90e6a700 1 --2- 172.21.15.40:0/3168541217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e8c0c7f60 0x7f6e8c0c7250 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:23.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.081+0000 7f6e90e6a700 1 -- 172.21.15.40:0/3168541217 >> 172.21.15.40:0/3168541217 conn(0x7f6e8c066090 msgr2=0x7f6e8c0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:23.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.081+0000 7f6e90e6a700 1 -- 172.21.15.40:0/3168541217 shutdown_connections 2024-09-13T18:15:23.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.081+0000 7f6e90e6a700 1 -- 172.21.15.40:0/3168541217 wait complete. 2024-09-13T18:15:23.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.085+0000 7f6e90e6a700 1 Processor -- start 2024-09-13T18:15:23.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.085+0000 7f6e90e6a700 1 -- start start 2024-09-13T18:15:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.085+0000 7f6e90e6a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e8c0c7f60 0x7f6e8c1fc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.085+0000 7f6e90e6a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e8c0c8340 0x7f6e8c08ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.085+0000 7f6e90e6a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e8c163d00 0x7f6e8c161500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.085+0000 7f6e90e6a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c2105e0 con 0x7f6e8c163d00 2024-09-13T18:15:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.085+0000 7f6e90e6a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c1fcde0 con 0x7f6e8c0c8340 2024-09-13T18:15:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.085+0000 7f6e90e6a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c210ad0 con 0x7f6e8c0c7f60 2024-09-13T18:15:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.085+0000 7f6e8a59c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e8c0c7f60 0x7f6e8c1fc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.085+0000 7f6e8ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e8c163d00 0x7f6e8c161500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.085+0000 7f6e8ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e8c163d00 0x7f6e8c161500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53058/0 (socket says 172.21.15.40:53058) 2024-09-13T18:15:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.085+0000 7f6e8ad9d700 1 -- 172.21.15.40:0/2393768060 learned_addr learned my addr 172.21.15.40:0/2393768060 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.085+0000 7f6e8ad9d700 1 -- 172.21.15.40:0/2393768060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e8c0c7f60 msgr2=0x7f6e8c1fc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.085+0000 7f6e8ad9d700 1 --2- 172.21.15.40:0/2393768060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e8c0c7f60 0x7f6e8c1fc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:23.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.085+0000 7f6e8ad9d700 1 -- 172.21.15.40:0/2393768060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e8c0c8340 msgr2=0x7f6e8c08ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:23.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.085+0000 7f6e8ad9d700 1 --2- 172.21.15.40:0/2393768060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e8c0c8340 0x7f6e8c08ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:23.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.085+0000 7f6e8ad9d700 1 -- 172.21.15.40:0/2393768060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8000b560 con 0x7f6e8c163d00 2024-09-13T18:15:23.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.085+0000 7f6e8ad9d700 1 --2- 172.21.15.40:0/2393768060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e8c163d00 0x7f6e8c161500 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000fb10 tx=0x7f6e8000fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:23.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.089+0000 7f6e7affd700 1 -- 172.21.15.40:0/2393768060 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8000dcb0 con 0x7f6e8c163d00 2024-09-13T18:15:23.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.089+0000 7f6e90e6a700 1 -- 172.21.15.40:0/2393768060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7c008530 con 0x7f6e8c163d00 2024-09-13T18:15:23.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.089+0000 7f6e90e6a700 1 -- 172.21.15.40:0/2393768060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e8c210810 con 0x7f6e8c163d00 2024-09-13T18:15:23.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.089+0000 7f6e7affd700 1 -- 172.21.15.40:0/2393768060 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8000de30 con 0x7f6e8c163d00 2024-09-13T18:15:23.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.089+0000 7f6e7affd700 1 -- 172.21.15.40:0/2393768060 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e80018680 con 0x7f6e8c163d00 2024-09-13T18:15:23.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.093+0000 7f6e90e6a700 1 --2- 172.21.15.40:0/2393768060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e70002a40 0x7f6e70004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:23.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.093+0000 7f6e90e6a700 1 -- 172.21.15.40:0/2393768060 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e70007f40 con 0x7f6e70002a40 2024-09-13T18:15:23.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.093+0000 7f6e8a59c700 1 --2- 172.21.15.40:0/2393768060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e70002a40 0x7f6e70004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:23.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.093+0000 7f6e8a59c700 1 -- 172.21.15.40:0/2393768060 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6e70008100 con 0x7f6e70002a40 2024-09-13T18:15:23.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.093+0000 7f6e8a59c700 1 --2- 172.21.15.40:0/2393768060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e70002a40 0x7f6e70004f00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0088d0 tx=0x7f6e7c004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:23.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.097+0000 7f6e7affd700 1 -- 172.21.15.40:0/2393768060 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6e80010440 con 0x7f6e8c163d00 2024-09-13T18:15:23.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.097+0000 7f6e7affd700 1 --2- 172.21.15.40:0/2393768060 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e6c066780 0x7f6e6c068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:23.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.097+0000 7f6e7affd700 1 -- 172.21.15.40:0/2393768060 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c003e60 con 0x7f6e70002a40 2024-09-13T18:15:23.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.097+0000 7f6e7affd700 1 -- 172.21.15.40:0/2393768060 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6e80084c00 con 0x7f6e8c163d00 2024-09-13T18:15:23.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.097+0000 7f6e89d9b700 1 --2- 172.21.15.40:0/2393768060 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e6c066780 0x7f6e6c068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:23.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.097+0000 7f6e7affd700 1 -- 172.21.15.40:0/2393768060 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f6e7c0180f0 con 0x7f6e70002a40 2024-09-13T18:15:23.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.097+0000 7f6e7affd700 1 -- 172.21.15.40:0/2393768060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e70002a40 msgr2=0x7f6e70004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:23.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.097+0000 7f6e7affd700 1 --2- 172.21.15.40:0/2393768060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e70002a40 0x7f6e70004f00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0088d0 tx=0x7f6e7c004770 comp rx=0 tx=0).stop 2024-09-13T18:15:23.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.097+0000 7f6e7affd700 1 -- 172.21.15.40:0/2393768060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e70002a40 msgr2=0x7f6e70004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:15:23.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.097+0000 7f6e7affd700 1 --2- 172.21.15.40:0/2393768060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e70002a40 0x7f6e70004f00 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0088d0 tx=0x7f6e7c004770 comp rx=0 tx=0).stop 2024-09-13T18:15:23.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.097+0000 7f6e89d9b700 1 --2- 172.21.15.40:0/2393768060 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e6c066780 0x7f6e6c068c40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6e840012e0 tx=0x7f6e84001860 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:23.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.113+0000 7f6e90e6a700 1 --2- 172.21.15.40:0/2393768060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e70006fb0 0x7f6e70007440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:23.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.113+0000 7f6e90e6a700 1 -- 172.21.15.40:0/2393768060 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7000a4a0 con 0x7f6e70006fb0 2024-09-13T18:15:23.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.113+0000 7f6e8a59c700 1 --2- 172.21.15.40:0/2393768060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e70006fb0 0x7f6e70007440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:23.119 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.113+0000 7f6e8a59c700 1 -- 172.21.15.40:0/2393768060 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f6e70007980 con 0x7f6e70006fb0 2024-09-13T18:15:23.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.113+0000 7f6e8a59c700 1 --2- 172.21.15.40:0/2393768060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e70006fb0 0x7f6e70007440 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6e700080d0 tx=0x7f6e7c00a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:23.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.113+0000 7f6e7affd700 1 -- 172.21.15.40:0/2393768060 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c00b4b0 con 0x7f6e70006fb0 2024-09-13T18:15:23.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.117+0000 7f6e7affd700 1 -- 172.21.15.40:0/2393768060 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7f6e7c00b610 con 0x7f6e70006fb0 2024-09-13T18:15:23.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.117+0000 7f6e7affd700 1 -- 172.21.15.40:0/2393768060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e70006fb0 msgr2=0x7f6e70007440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:23.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.117+0000 7f6e7affd700 1 --2- 172.21.15.40:0/2393768060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e70006fb0 0x7f6e70007440 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6e700080d0 tx=0x7f6e7c00a000 comp rx=0 tx=0).stop 2024-09-13T18:15:23.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.117+0000 7f6e7affd700 1 -- 172.21.15.40:0/2393768060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e70006fb0 msgr2=0x7f6e70007440 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:15:23.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.117+0000 7f6e7affd700 1 --2- 172.21.15.40:0/2393768060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e70006fb0 0x7f6e70007440 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6e700080d0 tx=0x7f6e7c00a000 comp rx=0 tx=0).stop 2024-09-13T18:15:23.123 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:15:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:15:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:15:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:15:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 204, 2024-09-13T18:15:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:15:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:15:23.123 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:15:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:15:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:15:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:15:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:15:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:15:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:23.124 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:15:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:15:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:23.125 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:15:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:15:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:15:23.126 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:15:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:15:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:15:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:15:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:15:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:15:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:15:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:15:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:15:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:15:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:15:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:15:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:15:23.128 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:15:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:15:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:23.129 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:15:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:15:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:15:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:15:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:23.130 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:15:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:15:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:15:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:23.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:15:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:15:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:15:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:15:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:23.132 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:15:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:15:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:15:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:15:23.133 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:23.134 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:23.134 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:23.134 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:23.134 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:15:23.134 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:23.134 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:23.134 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:23.134 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:23.134 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:15:23.134 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:23.134 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:23.134 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:15:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:15:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:15:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:15:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:15:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:15:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:15:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:15:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:15:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:15:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:15:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:15:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:15:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:15:23.138 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:15:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.133+0000 7f6e78ff9700 1 -- 172.21.15.40:0/2393768060 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e6c066780 msgr2=0x7f6e6c068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:23.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.133+0000 7f6e78ff9700 1 --2- 172.21.15.40:0/2393768060 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e6c066780 0x7f6e6c068c40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6e840012e0 tx=0x7f6e84001860 comp rx=0 tx=0).stop 2024-09-13T18:15:23.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.133+0000 7f6e8ad9d700 1 -- 172.21.15.40:0/2393768060 reap_dead start 2024-09-13T18:15:23.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.133+0000 7f6e78ff9700 1 -- 172.21.15.40:0/2393768060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e8c163d00 msgr2=0x7f6e8c161500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:23.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.133+0000 7f6e78ff9700 1 --2- 172.21.15.40:0/2393768060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e8c163d00 0x7f6e8c161500 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000fb10 tx=0x7f6e8000fe20 comp rx=0 tx=0).stop 2024-09-13T18:15:23.203 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.133+0000 7f6e78ff9700 1 -- 172.21.15.40:0/2393768060 shutdown_connections 2024-09-13T18:15:23.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.133+0000 7f6e78ff9700 1 --2- 172.21.15.40:0/2393768060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e8c163d00 0x7f6e8c161500 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:23.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.133+0000 7f6e78ff9700 1 -- 172.21.15.40:0/2393768060 >> 172.21.15.40:0/2393768060 conn(0x7f6e8c066090 msgr2=0x7f6e8c164220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:23.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.133+0000 7f6e78ff9700 1 -- 172.21.15.40:0/2393768060 shutdown_connections 2024-09-13T18:15:23.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.133+0000 7f6e78ff9700 1 -- 172.21.15.40:0/2393768060 wait complete. 2024-09-13T18:15:23.219 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:15:23.220 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:15:23.220 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:15:23.532 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.521+0000 7fba3635c700 1 Processor -- start 2024-09-13T18:15:23.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.521+0000 7fba3635c700 1 -- start start 2024-09-13T18:15:23.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.525+0000 7fba3635c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba30154240 0x7fba3015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:23.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.525+0000 7fba3635c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba3015f510 con 0x7fba3015ec60 2024-09-13T18:15:23.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.525+0000 7fba3635c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba30071690 con 0x7fba3015f0a0 2024-09-13T18:15:23.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.525+0000 7fba3635c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba300721e0 con 0x7fba30154240 2024-09-13T18:15:23.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.525+0000 7fba2f7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fba3015ec60 0x7fba3006a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:40652/0 (socket says 172.21.15.40:40652) 2024-09-13T18:15:23.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.525+0000 7fba2f7fe700 1 -- 172.21.15.40:0/198590737 learned_addr learned my addr 172.21.15.40:0/198590737 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:15:23.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.525+0000 7fba2ffff700 1 --2- 172.21.15.40:0/198590737 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba30154240 0x7fba3015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:23.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.525+0000 7fba2ffff700 1 -- 172.21.15.40:0/198590737 >> v1:172.21.15.43:6789/0 conn(0x7fba3015f0a0 legacy=0x7fba3006aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:23.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.525+0000 7fba2ffff700 1 -- 172.21.15.40:0/198590737 >> v1:172.21.15.40:6789/0 conn(0x7fba3015ec60 legacy=0x7fba3006a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:23.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.525+0000 7fba2ffff700 1 -- 172.21.15.40:0/198590737 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba30154620 con 0x7fba30154240 2024-09-13T18:15:23.535 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.529+0000 7fba2ffff700 1 --2- 172.21.15.40:0/198590737 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba30154240 0x7fba3015e1a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fba20009880 tx=0x7fba20009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7ce6bf842ee2910 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.533+0000 7fba2effd700 1 -- 172.21.15.40:0/198590737 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba20004230 con 0x7fba30154240 2024-09-13T18:15:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.533+0000 7fba2effd700 1 -- 172.21.15.40:0/198590737 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba20004390 con 0x7fba30154240 2024-09-13T18:15:23.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.533+0000 7fba2effd700 1 -- 172.21.15.40:0/198590737 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba200036c0 con 0x7fba30154240 2024-09-13T18:15:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.533+0000 7fba3635c700 1 -- 172.21.15.40:0/198590737 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba30154240 msgr2=0x7fba3015e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:23.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.533+0000 7fba3635c700 1 --2- 172.21.15.40:0/198590737 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba30154240 0x7fba3015e1a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fba20009880 tx=0x7fba20009b90 comp rx=0 tx=0).stop 2024-09-13T18:15:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.537+0000 7fba3635c700 1 -- 172.21.15.40:0/198590737 shutdown_connections 2024-09-13T18:15:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.537+0000 7fba3635c700 1 --2- 172.21.15.40:0/198590737 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba30154240 0x7fba3015e1a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:23.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.537+0000 7fba3635c700 1 -- 172.21.15.40:0/198590737 >> 172.21.15.40:0/198590737 conn(0x7fba300cabc0 msgr2=0x7fba300cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:23.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.537+0000 7fba3635c700 1 -- 172.21.15.40:0/198590737 shutdown_connections 2024-09-13T18:15:23.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.537+0000 7fba3635c700 1 -- 172.21.15.40:0/198590737 wait complete. 2024-09-13T18:15:23.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.541+0000 7fba3635c700 1 Processor -- start 2024-09-13T18:15:23.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.541+0000 7fba3635c700 1 -- start start 2024-09-13T18:15:23.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.545+0000 7fba3635c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba30154240 0x7fba3006a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:23.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.545+0000 7fba3635c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba3015ec60 0x7fba300c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:23.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.545+0000 7fba3635c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba3015f0a0 0x7fba301400e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:23.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.545+0000 7fba3635c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3006afc0 con 0x7fba30154240 2024-09-13T18:15:23.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.545+0000 7fba3635c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3013cb70 con 0x7fba3015ec60 2024-09-13T18:15:23.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.545+0000 7fba3635c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3013ce50 con 0x7fba3015f0a0 2024-09-13T18:15:23.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.545+0000 7fba2ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba30154240 0x7fba3006a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:23.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.545+0000 7fba2ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba30154240 0x7fba3006a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53070/0 (socket says 172.21.15.40:53070) 2024-09-13T18:15:23.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.545+0000 7fba2ffff700 1 -- 172.21.15.40:0/2276823978 learned_addr learned my addr 172.21.15.40:0/2276823978 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:23.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.545+0000 7fba2ffff700 1 -- 172.21.15.40:0/2276823978 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba3015f0a0 msgr2=0x7fba301400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:23.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.545+0000 7fba2ffff700 1 --2- 172.21.15.40:0/2276823978 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba3015f0a0 0x7fba301400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:23.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.545+0000 7fba2ffff700 1 -- 172.21.15.40:0/2276823978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba3015ec60 msgr2=0x7fba300c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:23.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.545+0000 7fba2ffff700 1 --2- 172.21.15.40:0/2276823978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba3015ec60 0x7fba300c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:23.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.545+0000 7fba2ffff700 1 -- 172.21.15.40:0/2276823978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba200094b0 con 0x7fba30154240 2024-09-13T18:15:23.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.549+0000 7fba2ffff700 1 --2- 172.21.15.40:0/2276823978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba30154240 0x7fba3006a930 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fba200090b0 tx=0x7fba20004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:23.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.553+0000 7fba2cff9700 1 -- 172.21.15.40:0/2276823978 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba20011830 con 0x7fba30154240 2024-09-13T18:15:23.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.553+0000 7fba3635c700 1 -- 172.21.15.40:0/2276823978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3013d130 con 0x7fba30154240 2024-09-13T18:15:23.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.553+0000 7fba3635c700 1 -- 172.21.15.40:0/2276823978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3013d6f0 con 0x7fba30154240 2024-09-13T18:15:23.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.553+0000 7fba2cff9700 1 -- 172.21.15.40:0/2276823978 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2001f070 con 0x7fba30154240 2024-09-13T18:15:23.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.553+0000 7fba2cff9700 1 -- 172.21.15.40:0/2276823978 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba20011bf0 con 0x7fba30154240 2024-09-13T18:15:23.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.553+0000 7fba2cff9700 1 -- 172.21.15.40:0/2276823978 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fba20011d50 con 0x7fba30154240 2024-09-13T18:15:23.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.553+0000 7fba2cff9700 1 --2- 172.21.15.40:0/2276823978 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fba1c066780 0x7fba1c068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:23.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.553+0000 7fba2cff9700 1 -- 172.21.15.40:0/2276823978 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fba20021070 con 0x7fba30154240 2024-09-13T18:15:23.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.553+0000 7fba2f7fe700 1 --2- 172.21.15.40:0/2276823978 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fba1c066780 0x7fba1c068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:23.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.557+0000 7fba2f7fe700 1 --2- 172.21.15.40:0/2276823978 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fba1c066780 0x7fba1c068c40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fba280012e0 tx=0x7fba280019f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:23.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.557+0000 7fba3635c700 1 -- 172.21.15.40:0/2276823978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba10005310 con 0x7fba30154240 2024-09-13T18:15:23.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:23.565+0000 7fba2cff9700 1 -- 172.21.15.40:0/2276823978 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fba20052f60 con 0x7fba30154240 2024-09-13T18:15:24.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:24.209+0000 7fba3635c700 1 -- 172.21.15.40:0/2276823978 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fba10005890 con 0x7fba30154240 2024-09-13T18:15:24.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:24.213+0000 7fba2cff9700 1 -- 172.21.15.40:0/2276823978 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fba20056580 con 0x7fba30154240 2024-09-13T18:15:24.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:24.217+0000 7fba167fc700 1 -- 172.21.15.40:0/2276823978 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fba1c066780 msgr2=0x7fba1c068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:24.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:24.217+0000 7fba167fc700 1 --2- 172.21.15.40:0/2276823978 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fba1c066780 0x7fba1c068c40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fba280012e0 tx=0x7fba280019f0 comp rx=0 tx=0).stop 2024-09-13T18:15:24.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:24.217+0000 7fba167fc700 1 -- 172.21.15.40:0/2276823978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba30154240 msgr2=0x7fba3006a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:24.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:24.217+0000 7fba167fc700 1 --2- 172.21.15.40:0/2276823978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba30154240 0x7fba3006a930 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fba200090b0 tx=0x7fba20004770 comp rx=0 tx=0).stop 2024-09-13T18:15:24.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:24.217+0000 7fba167fc700 1 -- 172.21.15.40:0/2276823978 shutdown_connections 2024-09-13T18:15:24.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:24.217+0000 7fba167fc700 1 --2- 172.21.15.40:0/2276823978 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fba1c066780 0x7fba1c068c40 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:24.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:24.217+0000 7fba167fc700 1 --2- 172.21.15.40:0/2276823978 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba3015f0a0 0x7fba301400e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:24.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:24.217+0000 7fba167fc700 1 --2- 172.21.15.40:0/2276823978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba3015ec60 0x7fba300c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:24.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:24.221+0000 7fba167fc700 1 --2- 172.21.15.40:0/2276823978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba30154240 0x7fba3006a930 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:24.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:24.221+0000 7fba167fc700 1 -- 172.21.15.40:0/2276823978 >> 172.21.15.40:0/2276823978 conn(0x7fba300cabc0 msgr2=0x7fba300c3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:24.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:24.221+0000 7fba167fc700 1 -- 172.21.15.40:0/2276823978 shutdown_connections 2024-09-13T18:15:24.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:24.221+0000 7fba167fc700 1 -- 172.21.15.40:0/2276823978 wait complete. 2024-09-13T18:15:24.255 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:15:34.257 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:15:34.257 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:15:34.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.481+0000 7f821544a700 1 Processor -- start 2024-09-13T18:15:34.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.481+0000 7f821544a700 1 -- start start 2024-09-13T18:15:34.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.481+0000 7f821544a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8210160390 0x7f82101669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:34.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.481+0000 7f821544a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8210071930 con 0x7f8210167d10 2024-09-13T18:15:34.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.481+0000 7f821544a700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8210070a40 con 0x7f82100704e0 2024-09-13T18:15:34.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.481+0000 7f821544a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8210166f20 con 0x7f8210160390 2024-09-13T18:15:34.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.481+0000 7f820ffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8210160390 0x7f82101669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:34.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.481+0000 7f820f7fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f82100704e0 0x7f8210167600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:46604/0 (socket says 172.21.15.40:46604) 2024-09-13T18:15:34.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.481+0000 7f820f7fe700 1 -- 172.21.15.40:0/80885826 learned_addr learned my addr 172.21.15.40:0/80885826 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:15:34.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.485+0000 7f820effd700 1 -- 172.21.15.40:0/80885826 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769200074 0 0) 0x7f8210070a40 con 0x7f82100704e0 2024-09-13T18:15:34.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.485+0000 7f820effd700 1 -- 172.21.15.40:0/80885826 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81f80035d0 con 0x7f82100704e0 2024-09-13T18:15:34.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.485+0000 7f820effd700 1 -- 172.21.15.40:0/80885826 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3423095180 0 0) 0x7f8210071930 con 0x7f8210167d10 2024-09-13T18:15:34.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.485+0000 7f820effd700 1 -- 172.21.15.40:0/80885826 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8210070a40 con 0x7f8210167d10 2024-09-13T18:15:34.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.485+0000 7f820ffff700 1 -- 172.21.15.40:0/80885826 >> v1:172.21.15.43:6789/0 conn(0x7f82100704e0 legacy=0x7f8210167600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:34.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.485+0000 7f820ffff700 1 -- 172.21.15.40:0/80885826 >> v1:172.21.15.40:6789/0 conn(0x7f8210167d10 legacy=0x7f821006a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:34.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.485+0000 7f820ffff700 1 -- 172.21.15.40:0/80885826 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8210070d20 con 0x7f8210160390 2024-09-13T18:15:34.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.485+0000 7f820ffff700 1 --2- 172.21.15.40:0/80885826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8210160390 0x7f82101669e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8200009850 tx=0x7f8200009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2decdd187b9c24fd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:34.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.485+0000 7f820effd700 1 -- 172.21.15.40:0/80885826 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820000f940 con 0x7f8210160390 2024-09-13T18:15:34.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.485+0000 7f820effd700 1 -- 172.21.15.40:0/80885826 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820000faa0 con 0x7f8210160390 2024-09-13T18:15:34.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.485+0000 7f820effd700 1 -- 172.21.15.40:0/80885826 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820000fe50 con 0x7f8210160390 2024-09-13T18:15:34.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.489+0000 7f821544a700 1 -- 172.21.15.40:0/80885826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8210160390 msgr2=0x7f82101669e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:34.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.489+0000 7f821544a700 1 --2- 172.21.15.40:0/80885826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8210160390 0x7f82101669e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8200009850 tx=0x7f8200009b60 comp rx=0 tx=0).stop 2024-09-13T18:15:34.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.489+0000 7f821544a700 1 -- 172.21.15.40:0/80885826 shutdown_connections 2024-09-13T18:15:34.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.489+0000 7f821544a700 1 --2- 172.21.15.40:0/80885826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8210160390 0x7f82101669e0 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8200009850 tx=0x7f8200009b60 comp rx=0 tx=0).stop 2024-09-13T18:15:34.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.489+0000 7f821544a700 1 -- 172.21.15.40:0/80885826 >> 172.21.15.40:0/80885826 conn(0x7f82100cabc0 msgr2=0x7f82100cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:34.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.493+0000 7f821544a700 1 -- 172.21.15.40:0/80885826 shutdown_connections 2024-09-13T18:15:34.499 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.493+0000 7f821544a700 1 -- 172.21.15.40:0/80885826 wait complete. 2024-09-13T18:15:34.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.497+0000 7f821544a700 1 Processor -- start 2024-09-13T18:15:34.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.497+0000 7f821544a700 1 -- start start 2024-09-13T18:15:34.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.497+0000 7f821544a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82100704e0 0x7f82100ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:34.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.497+0000 7f821544a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8210167d10 0x7f821015be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:34.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.497+0000 7f821544a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8210164490 0x7f8210208180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:34.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.497+0000 7f821544a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8210165fd0 con 0x7f82100704e0 2024-09-13T18:15:34.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.497+0000 7f821544a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82100c8f90 con 0x7f8210164490 2024-09-13T18:15:34.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.497+0000 7f821544a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82100c9270 con 0x7f8210167d10 2024-09-13T18:15:34.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.497+0000 7f820ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82100704e0 0x7f82100ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:34.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.497+0000 7f820ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82100704e0 0x7f82100ca370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55314/0 (socket says 172.21.15.40:55314) 2024-09-13T18:15:34.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.497+0000 7f820ffff700 1 -- 172.21.15.40:0/2275663031 learned_addr learned my addr 172.21.15.40:0/2275663031 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:34.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.497+0000 7f820f7fe700 1 --2- 172.21.15.40:0/2275663031 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8210167d10 0x7f821015be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:34.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.501+0000 7f820f7fe700 1 -- 172.21.15.40:0/2275663031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8210164490 msgr2=0x7f8210208180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:34.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.501+0000 7f820f7fe700 1 --2- 172.21.15.40:0/2275663031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8210164490 0x7f8210208180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:34.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.501+0000 7f820f7fe700 1 -- 172.21.15.40:0/2275663031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82100704e0 msgr2=0x7f82100ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:34.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.501+0000 7f820f7fe700 1 --2- 172.21.15.40:0/2275663031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82100704e0 0x7f82100ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:34.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.501+0000 7f820f7fe700 1 -- 172.21.15.40:0/2275663031 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82000094b0 con 0x7f8210167d10 2024-09-13T18:15:34.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.501+0000 7f820ffff700 1 --2- 172.21.15.40:0/2275663031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82100704e0 0x7f82100ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:15:34.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.501+0000 7f820f7fe700 1 --2- 172.21.15.40:0/2275663031 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8210167d10 0x7f821015be00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f820800c8d0 tx=0x7f820800cc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:34.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.501+0000 7f820cff9700 1 -- 172.21.15.40:0/2275663031 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820800aa90 con 0x7f8210167d10 2024-09-13T18:15:34.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.501+0000 7f820cff9700 1 -- 172.21.15.40:0/2275663031 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820800abf0 con 0x7f8210167d10 2024-09-13T18:15:34.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.501+0000 7f821544a700 1 -- 172.21.15.40:0/2275663031 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8210166240 con 0x7f8210167d10 2024-09-13T18:15:34.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.501+0000 7f821544a700 1 -- 172.21.15.40:0/2275663031 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82100ca8b0 con 0x7f8210167d10 2024-09-13T18:15:34.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.501+0000 7f820cff9700 1 -- 172.21.15.40:0/2275663031 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208011450 con 0x7f8210167d10 2024-09-13T18:15:34.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.505+0000 7f821544a700 1 -- 172.21.15.40:0/2275663031 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8210166760 con 0x7f8210167d10 2024-09-13T18:15:34.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.505+0000 7f820cff9700 1 -- 172.21.15.40:0/2275663031 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f82080115b0 con 0x7f8210167d10 2024-09-13T18:15:34.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.505+0000 7f820cff9700 1 --2- 172.21.15.40:0/2275663031 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f81fc066730 0x7f81fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:34.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.505+0000 7f820cff9700 1 -- 172.21.15.40:0/2275663031 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f820800d040 con 0x7f8210167d10 2024-09-13T18:15:34.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.505+0000 7f820ffff700 1 --2- 172.21.15.40:0/2275663031 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f81fc066730 0x7f81fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:34.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.517+0000 7f820cff9700 1 -- 172.21.15.40:0/2275663031 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8210166760 con 0x7f8210167d10 2024-09-13T18:15:34.526 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:34.517+0000 7f820ffff700 1 --2- 172.21.15.40:0/2275663031 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f81fc066730 0x7f81fc068bf0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8200019040 tx=0x7f8200008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:35.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.449+0000 7f821544a700 1 -- 172.21.15.40:0/2275663031 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f8210166760 con 0x7f8210167d10 2024-09-13T18:15:35.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.449+0000 7f820cff9700 1 -- 172.21.15.40:0/2275663031 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f8208015620 con 0x7f8210167d10 2024-09-13T18:15:35.457 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":204,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:15:35.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.453+0000 7f821544a700 1 -- 172.21.15.40:0/2275663031 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f81fc066730 msgr2=0x7f81fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:35.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.453+0000 7f821544a700 1 --2- 172.21.15.40:0/2275663031 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f81fc066730 0x7f81fc068bf0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8200019040 tx=0x7f8200008030 comp rx=0 tx=0).stop 2024-09-13T18:15:35.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.453+0000 7f821544a700 1 -- 172.21.15.40:0/2275663031 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8210167d10 msgr2=0x7f821015be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:35.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.453+0000 7f821544a700 1 --2- 172.21.15.40:0/2275663031 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8210167d10 0x7f821015be00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f820800c8d0 tx=0x7f820800cc90 comp rx=0 tx=0).stop 2024-09-13T18:15:35.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.453+0000 7f821544a700 1 -- 172.21.15.40:0/2275663031 shutdown_connections 2024-09-13T18:15:35.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.453+0000 7f821544a700 1 --2- 172.21.15.40:0/2275663031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8210164490 0x7f8210208180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:35.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.453+0000 7f821544a700 1 --2- 172.21.15.40:0/2275663031 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f81fc066730 0x7f81fc068bf0 secure :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8200019040 tx=0x7f8200008030 comp rx=0 tx=0).stop 2024-09-13T18:15:35.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.453+0000 7f821544a700 1 --2- 172.21.15.40:0/2275663031 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8210167d10 0x7f821015be00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:35.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.453+0000 7f821544a700 1 --2- 172.21.15.40:0/2275663031 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82100704e0 0x7f82100ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:35.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.453+0000 7f821544a700 1 -- 172.21.15.40:0/2275663031 >> 172.21.15.40:0/2275663031 conn(0x7f82100cabc0 msgr2=0x7f82101ee260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:35.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.457+0000 7f821544a700 1 -- 172.21.15.40:0/2275663031 shutdown_connections 2024-09-13T18:15:35.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.457+0000 7f821544a700 1 -- 172.21.15.40:0/2275663031 wait complete. 2024-09-13T18:15:35.483 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:15:35.483 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T18:15:35.483 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:15:35.483 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:15:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.893+0000 7ff273fff700 1 Processor -- start 2024-09-13T18:15:35.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.893+0000 7ff273fff700 1 -- start start 2024-09-13T18:15:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.897+0000 7ff273fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff27414fa40 0x7ff2740c1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.897+0000 7ff273fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff27406d6f0 0x7ff274155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.897+0000 7ff273fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2740c75e0 con 0x7ff27406d2d0 2024-09-13T18:15:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.897+0000 7ff273fff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2740c67c0 con 0x7ff27414fa40 2024-09-13T18:15:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.897+0000 7ff273fff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff27406db40 con 0x7ff27406d6f0 2024-09-13T18:15:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.897+0000 7ff2737fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff27406d6f0 0x7ff274155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:35.903 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.897+0000 7ff2737fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff27406d6f0 0x7ff274155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:52626/0 (socket says 172.21.15.40:52626) 2024-09-13T18:15:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.897+0000 7ff2737fe700 1 -- 172.21.15.40:0/2487609915 learned_addr learned my addr 172.21.15.40:0/2487609915 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:35.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.897+0000 7ff2737fe700 1 -- 172.21.15.40:0/2487609915 >> v2:172.21.15.40:3300/0 conn(0x7ff27414fa40 msgr2=0x7ff2740c1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:15:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.897+0000 7ff2737fe700 1 --2- 172.21.15.40:0/2487609915 >> v2:172.21.15.40:3300/0 conn(0x7ff27414fa40 0x7ff2740c1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.897+0000 7ff2737fe700 1 -- 172.21.15.40:0/2487609915 >> v1:172.21.15.43:6789/0 conn(0x7ff27406d2d0 legacy=0x7ff2740c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.897+0000 7ff2737fe700 1 -- 172.21.15.40:0/2487609915 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2740c69d0 con 0x7ff27406d6f0 2024-09-13T18:15:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.897+0000 7ff2737fe700 1 --2- 172.21.15.40:0/2487609915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff27406d6f0 0x7ff274155ca0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff27406b920 tx=0x7ff268009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfd1db5415e91a00 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.897+0000 7ff271ffb700 1 -- 172.21.15.40:0/2487609915 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff268004230 con 0x7ff27406d6f0 2024-09-13T18:15:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.897+0000 7ff271ffb700 1 -- 172.21.15.40:0/2487609915 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff268004390 con 0x7ff27406d6f0 2024-09-13T18:15:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.897+0000 7ff273fff700 1 -- 172.21.15.40:0/2487609915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff27406d6f0 msgr2=0x7ff274155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:35.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.897+0000 7ff273fff700 1 --2- 172.21.15.40:0/2487609915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff27406d6f0 0x7ff274155ca0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff27406b920 tx=0x7ff268009bf0 comp rx=0 tx=0).stop 2024-09-13T18:15:35.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.901+0000 7ff273fff700 1 -- 172.21.15.40:0/2487609915 shutdown_connections 2024-09-13T18:15:35.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.901+0000 7ff273fff700 1 --2- 172.21.15.40:0/2487609915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff27406d6f0 0x7ff274155ca0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:35.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.901+0000 7ff273fff700 1 --2- 172.21.15.40:0/2487609915 >> v2:172.21.15.40:3300/0 conn(0x7ff27414fa40 0x7ff2740c1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:35.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.901+0000 7ff273fff700 1 -- 172.21.15.40:0/2487609915 >> 172.21.15.40:0/2487609915 conn(0x7ff274066090 msgr2=0x7ff2740664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:35.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.901+0000 7ff273fff700 1 -- 172.21.15.40:0/2487609915 shutdown_connections 2024-09-13T18:15:35.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.901+0000 7ff273fff700 1 -- 172.21.15.40:0/2487609915 wait complete. 2024-09-13T18:15:35.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.901+0000 7ff273fff700 1 Processor -- start 2024-09-13T18:15:35.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.901+0000 7ff273fff700 1 -- start start 2024-09-13T18:15:35.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.901+0000 7ff273fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff27406d2d0 0x7ff274153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:35.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.901+0000 7ff273fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff27406d6f0 0x7ff274154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:35.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.901+0000 7ff273fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff27414fa40 0x7ff274149f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:35.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.901+0000 7ff273fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff274154e90 con 0x7ff27406d2d0 2024-09-13T18:15:35.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.901+0000 7ff273fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2741433b0 con 0x7ff27414fa40 2024-09-13T18:15:35.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.901+0000 7ff273fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff274144260 con 0x7ff27406d6f0 2024-09-13T18:15:35.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.905+0000 7ff2727fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff27406d6f0 0x7ff274154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:35.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.905+0000 7ff2727fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff27406d6f0 0x7ff274154260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:41492/0 (socket says 172.21.15.40:41492) 2024-09-13T18:15:35.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.905+0000 7ff2727fc700 1 -- 172.21.15.40:0/3882486320 learned_addr learned my addr 172.21.15.40:0/3882486320 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:35.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.905+0000 7ff272ffd700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff27406d2d0 0x7ff274153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:35.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.905+0000 7ff2737fe700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff27414fa40 0x7ff274149f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:35.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.905+0000 7ff2727fc700 1 -- 172.21.15.40:0/3882486320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff27414fa40 msgr2=0x7ff274149f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:35.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.905+0000 7ff2727fc700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff27414fa40 0x7ff274149f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:35.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.905+0000 7ff2727fc700 1 -- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff27406d2d0 msgr2=0x7ff274153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:35.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.905+0000 7ff2727fc700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff27406d2d0 0x7ff274153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:35.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.905+0000 7ff2727fc700 1 -- 172.21.15.40:0/3882486320 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff268009570 con 0x7ff27406d6f0 2024-09-13T18:15:35.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.905+0000 7ff2737fe700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff27414fa40 0x7ff274149f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:15:35.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.905+0000 7ff2727fc700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff27406d6f0 0x7ff274154260 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0020c0 tx=0x7ff26c00a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:35.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.905+0000 7ff2637fe700 1 -- 172.21.15.40:0/3882486320 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c003e50 con 0x7ff27406d6f0 2024-09-13T18:15:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.905+0000 7ff273fff700 1 -- 172.21.15.40:0/3882486320 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2741e8570 con 0x7ff27406d6f0 2024-09-13T18:15:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.905+0000 7ff273fff700 1 -- 172.21.15.40:0/3882486320 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2741e6d20 con 0x7ff27406d6f0 2024-09-13T18:15:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.905+0000 7ff2637fe700 1 -- 172.21.15.40:0/3882486320 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26c00a930 con 0x7ff27406d6f0 2024-09-13T18:15:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.905+0000 7ff2637fe700 1 -- 172.21.15.40:0/3882486320 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c00ace0 con 0x7ff27406d6f0 2024-09-13T18:15:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.905+0000 7ff2637fe700 1 -- 172.21.15.40:0/3882486320 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff26c005070 con 0x7ff27406d6f0 2024-09-13T18:15:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.909+0000 7ff2637fe700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff25c0aa9b0 0x7ff25c0ace70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.909+0000 7ff2637fe700 1 -- 172.21.15.40:0/3882486320 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff26c083f70 con 0x7ff27406d6f0 2024-09-13T18:15:35.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.909+0000 7ff272ffd700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff25c0aa9b0 0x7ff25c0ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:35.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.909+0000 7ff273fff700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff258002a40 0x7ff258004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:35.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.909+0000 7ff273fff700 1 -- 172.21.15.40:0/3882486320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff258007f40 con 0x7ff258002a40 2024-09-13T18:15:35.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.909+0000 7ff2737fe700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff258002a40 0x7ff258004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:35.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.909+0000 7ff2737fe700 1 -- 172.21.15.40:0/3882486320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff258008100 con 0x7ff258002a40 2024-09-13T18:15:35.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.909+0000 7ff272ffd700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff25c0aa9b0 0x7ff25c0ace70 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff264009990 tx=0x7ff264009110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:35.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.909+0000 7ff2737fe700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff258002a40 0x7ff258004f00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff268004700 tx=0x7ff268004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:35.921 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.909+0000 7ff2637fe700 1 -- 172.21.15.40:0/3882486320 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26800f2f0 con 0x7ff258002a40 2024-09-13T18:15:35.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.917+0000 7ff2637fe700 1 -- 172.21.15.40:0/3882486320 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff268004500 con 0x7ff258002a40 2024-09-13T18:15:35.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.917+0000 7ff2637fe700 1 -- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff258002a40 msgr2=0x7ff258004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:35.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.917+0000 7ff2637fe700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff258002a40 0x7ff258004f00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff268004700 tx=0x7ff268004770 comp rx=0 tx=0).stop 2024-09-13T18:15:35.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.917+0000 7ff2637fe700 1 -- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff258002a40 msgr2=0x7ff258004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:15:35.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.917+0000 7ff2637fe700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff258002a40 0x7ff258004f00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:35.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.933+0000 7ff2617fa700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff258001e20 0x7ff25800cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:35.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.933+0000 7ff2737fe700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff258001e20 0x7ff25800cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:35.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.933+0000 7ff2617fa700 1 -- 172.21.15.40:0/3882486320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2580023b0 con 0x7ff258001e20 2024-09-13T18:15:35.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.937+0000 7ff2737fe700 1 -- 172.21.15.40:0/3882486320 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff25800a2e0 con 0x7ff258001e20 2024-09-13T18:15:35.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.937+0000 7ff2737fe700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff258001e20 0x7ff25800cbf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff268009320 tx=0x7ff268013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:35.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.937+0000 7ff2637fe700 1 -- 172.21.15.40:0/3882486320 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff268022910 con 0x7ff258001e20 2024-09-13T18:15:35.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.937+0000 7ff2637fe700 1 -- 172.21.15.40:0/3882486320 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4453 (secure 0 0 0) 0x7ff268022a70 con 0x7ff258001e20 2024-09-13T18:15:35.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.937+0000 7ff2637fe700 1 -- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff258001e20 msgr2=0x7ff25800cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:35.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.941+0000 7ff2637fe700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff258001e20 0x7ff25800cbf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff268009320 tx=0x7ff268013000 comp rx=0 tx=0).stop 2024-09-13T18:15:35.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.941+0000 7ff2637fe700 1 -- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff258001e20 msgr2=0x7ff25800cbf0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:15:35.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.941+0000 7ff2637fe700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff258001e20 0x7ff25800cbf0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:35.949 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:15:35.949 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:15:35.949 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:15:35.949 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:15:35.949 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 204, 2024-09-13T18:15:35.949 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:15:35.949 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:15:35.949 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:15:35.949 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:15:35.950 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:35.950 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 15, 2024-09-13T18:15:35.950 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:15:35.950 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:15:35.950 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:15:35.950 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:35.950 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:35.950 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:35.950 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:35.950 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:35.950 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:35.951 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:35.951 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:35.951 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:35.951 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:35.951 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:35.951 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:15:35.951 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:15:35.951 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:35.951 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:35.951 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:35.951 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:35.951 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:35.952 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:35.952 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:35.952 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:35.952 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:35.952 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:35.952 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:35.952 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:35.952 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:15:35.952 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:15:35.952 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:15:35.952 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:15:35.952 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:15:35.953 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:15:35.953 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:15:35.953 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:15:35.953 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:15:35.953 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:15:35.953 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:15:35.953 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:15:35.953 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:15:35.953 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:15:35.953 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:15:35.953 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:15:35.953 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:15:35.953 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:35.954 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:35.954 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:35.954 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:35.954 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:35.954 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:35.954 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:35.954 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:35.954 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:35.954 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:35.954 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:35.954 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:15:35.954 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:35.955 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:15:35.955 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:35.955 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:15:35.955 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:15:35.955 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:35.955 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:35.955 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:35.955 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:35.955 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:15:35.955 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:35.955 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:35.955 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:35.955 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:35.956 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:15:35.956 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:35.956 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:35.956 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:35.956 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:35.956 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:15:35.956 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:15:35.956 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:35.956 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:35.956 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:35.956 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:35.956 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:35.957 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:15:35.957 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:15:35.957 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:35.957 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:35.957 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:35.957 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:35.957 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:15:35.957 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:35.957 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:35.957 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:35.957 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:35.957 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:15:35.957 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:35.958 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:35.958 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:35.958 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:35.958 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:15:35.958 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:15:35.958 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:35.958 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:35.958 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:35.958 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:35.958 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:35.958 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:15:35.959 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:15:35.959 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:35.959 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:35.959 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:35.959 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:35.959 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:15:35.959 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:35.959 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:35.959 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:35.959 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:35.960 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:15:35.960 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:35.960 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:35.960 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:35.960 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:35.960 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:15:35.960 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:15:35.960 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:35.960 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:35.960 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:35.960 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:35.960 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:35.961 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:35.961 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:15:35.961 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:15:35.961 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:35.961 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:35.961 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:35.961 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:15:35.961 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:35.961 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:35.961 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:15:35.961 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:35.961 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:35.961 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:35.961 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:15:35.962 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:35.962 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:35.962 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:35.962 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:15:35.962 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:15:35.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.961+0000 7ff273fff700 1 -- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff25c0aa9b0 msgr2=0x7ff25c0ace70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:35.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.961+0000 7ff273fff700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff25c0aa9b0 0x7ff25c0ace70 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff264009990 tx=0x7ff264009110 comp rx=0 tx=0).stop 2024-09-13T18:15:35.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.961+0000 7ff2737fe700 1 -- 172.21.15.40:0/3882486320 reap_dead start 2024-09-13T18:15:35.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.961+0000 7ff273fff700 1 -- 172.21.15.40:0/3882486320 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff27406d6f0 msgr2=0x7ff274154260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:35.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.961+0000 7ff273fff700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff27406d6f0 0x7ff274154260 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0020c0 tx=0x7ff26c00a340 comp rx=0 tx=0).stop 2024-09-13T18:15:35.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.961+0000 7ff273fff700 1 -- 172.21.15.40:0/3882486320 shutdown_connections 2024-09-13T18:15:35.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.961+0000 7ff273fff700 1 --2- 172.21.15.40:0/3882486320 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff27406d6f0 0x7ff274154260 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:35.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.961+0000 7ff273fff700 1 -- 172.21.15.40:0/3882486320 >> 172.21.15.40:0/3882486320 conn(0x7ff274066090 msgr2=0x7ff2740c13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:35.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.961+0000 7ff273fff700 1 -- 172.21.15.40:0/3882486320 shutdown_connections 2024-09-13T18:15:35.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:35.961+0000 7ff273fff700 1 -- 172.21.15.40:0/3882486320 wait complete. 2024-09-13T18:15:36.009 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:15:36.009 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:15:36.010 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:15:36.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.957+0000 7f614da3f700 1 Processor -- start 2024-09-13T18:15:36.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.957+0000 7f614da3f700 1 -- start start 2024-09-13T18:15:36.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.961+0000 7f614da3f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6148157e90 0x7f61480c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:36.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.961+0000 7f614da3f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f61480c57d0 0x7f614815e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:36.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.961+0000 7f614da3f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61481619c0 con 0x7f614815e6b0 2024-09-13T18:15:36.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.961+0000 7f614da3f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6148158270 con 0x7f61480c57d0 2024-09-13T18:15:36.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.961+0000 7f614da3f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f614815dd50 con 0x7f6148157e90 2024-09-13T18:15:36.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.961+0000 7f6147fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f61480c57d0 0x7f614815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:36.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.961+0000 7f6147fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f61480c57d0 0x7f614815e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55366/0 (socket says 172.21.15.40:55366) 2024-09-13T18:15:36.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.961+0000 7f6147fff700 1 -- 172.21.15.40:0/1196233137 learned_addr learned my addr 172.21.15.40:0/1196233137 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:36.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.961+0000 7f6147fff700 1 -- 172.21.15.40:0/1196233137 >> v2:172.21.15.43:3300/0 conn(0x7f6148157e90 msgr2=0x7f61480c71e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:15:36.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.961+0000 7f6147fff700 1 --2- 172.21.15.40:0/1196233137 >> v2:172.21.15.43:3300/0 conn(0x7f6148157e90 0x7f61480c71e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:36.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.961+0000 7f6147fff700 1 -- 172.21.15.40:0/1196233137 >> v1:172.21.15.43:6789/0 conn(0x7f614815e6b0 legacy=0x7f614806d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:15:36.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.961+0000 7f6147fff700 1 -- 172.21.15.40:0/1196233137 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6148161ed0 con 0x7f61480c57d0 2024-09-13T18:15:36.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.961+0000 7f6147fff700 1 --2- 172.21.15.40:0/1196233137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61480c57d0 0x7f614815e170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f61480cbe90 tx=0x7f6138009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1b422566bc18f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:36.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.961+0000 7f61477fe700 1 -- 172.21.15.40:0/1196233137 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613800b8a0 con 0x7f61480c57d0 2024-09-13T18:15:36.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.961+0000 7f61477fe700 1 -- 172.21.15.40:0/1196233137 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613800ba00 con 0x7f61480c57d0 2024-09-13T18:15:36.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.961+0000 7f61477fe700 1 -- 172.21.15.40:0/1196233137 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613800bd90 con 0x7f61480c57d0 2024-09-13T18:15:36.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.965+0000 7f614da3f700 1 -- 172.21.15.40:0/1196233137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61480c57d0 msgr2=0x7f614815e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:36.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.965+0000 7f614da3f700 1 --2- 172.21.15.40:0/1196233137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61480c57d0 0x7f614815e170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f61480cbe90 tx=0x7f6138009bf0 comp rx=0 tx=0).stop 2024-09-13T18:15:36.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.965+0000 7f614da3f700 1 -- 172.21.15.40:0/1196233137 shutdown_connections 2024-09-13T18:15:36.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.965+0000 7f614da3f700 1 --2- 172.21.15.40:0/1196233137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61480c57d0 0x7f614815e170 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:36.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.965+0000 7f614da3f700 1 --2- 172.21.15.40:0/1196233137 >> v2:172.21.15.43:3300/0 conn(0x7f6148157e90 0x7f61480c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:36.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.965+0000 7f614da3f700 1 -- 172.21.15.40:0/1196233137 >> 172.21.15.40:0/1196233137 conn(0x7f6148066090 msgr2=0x7f61480664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:36.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.969+0000 7f614da3f700 1 -- 172.21.15.40:0/1196233137 shutdown_connections 2024-09-13T18:15:36.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.969+0000 7f614da3f700 1 -- 172.21.15.40:0/1196233137 wait complete. 2024-09-13T18:15:36.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.969+0000 7f614da3f700 1 Processor -- start 2024-09-13T18:15:36.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.977+0000 7f614da3f700 1 -- start start 2024-09-13T18:15:36.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.977+0000 7f614da3f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61480c57d0 0x7f61480c4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:36.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.977+0000 7f614da3f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6148157e90 0x7f6148160d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:36.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.977+0000 7f614da3f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f614815e6b0 0x7f61480c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:36.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.977+0000 7f614da3f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61480cb120 con 0x7f614815e6b0 2024-09-13T18:15:36.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.977+0000 7f614da3f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61480c5160 con 0x7f6148157e90 2024-09-13T18:15:36.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.977+0000 7f614da3f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61480c4e80 con 0x7f61480c57d0 2024-09-13T18:15:36.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.981+0000 7f6147fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6148157e90 0x7f6148160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:36.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.981+0000 7f6147fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6148157e90 0x7f6148160d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:52638/0 (socket says 172.21.15.40:52638) 2024-09-13T18:15:36.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.981+0000 7f6147fff700 1 -- 172.21.15.40:0/1081858300 learned_addr learned my addr 172.21.15.40:0/1081858300 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:36.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.981+0000 7f6147fff700 1 -- 172.21.15.40:0/1081858300 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61480c57d0 msgr2=0x7f61480c4840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:15:36.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.981+0000 7f6147fff700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f61480c57d0 0x7f61480c4840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:36.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.981+0000 7f6147fff700 1 -- 172.21.15.40:0/1081858300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f614815e6b0 msgr2=0x7f61480c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:37.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.993+0000 7f614d23e700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f614815e6b0 0x7f61480c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:37.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.993+0000 7f6147fff700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f614815e6b0 0x7f61480c2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:37.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.993+0000 7f6147fff700 1 -- 172.21.15.40:0/1081858300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6138009570 con 0x7f6148157e90 2024-09-13T18:15:37.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.993+0000 7f614d23e700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f614815e6b0 0x7f61480c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:15:37.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.993+0000 7f6147fff700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6148157e90 0x7f6148160d00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f614806b430 tx=0x7f6138004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:37.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.993+0000 7f61457fa700 1 -- 172.21.15.40:0/1081858300 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61380038c0 con 0x7f6148157e90 2024-09-13T18:15:37.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.993+0000 7f61457fa700 1 -- 172.21.15.40:0/1081858300 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6138003a20 con 0x7f6148157e90 2024-09-13T18:15:37.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.993+0000 7f614da3f700 1 -- 172.21.15.40:0/1081858300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61480ca780 con 0x7f6148157e90 2024-09-13T18:15:37.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.997+0000 7f614da3f700 1 -- 172.21.15.40:0/1081858300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61480cad40 con 0x7f6148157e90 2024-09-13T18:15:37.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:36.997+0000 7f61457fa700 1 -- 172.21.15.40:0/1081858300 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138003db0 con 0x7f6148157e90 2024-09-13T18:15:37.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.001+0000 7f61457fa700 1 -- 172.21.15.40:0/1081858300 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6138024440 con 0x7f6148157e90 2024-09-13T18:15:37.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.001+0000 7f614da3f700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f614814e570 0x7f61480c28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:37.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.001+0000 7f614da3f700 1 -- 172.21.15.40:0/1081858300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61480caf50 con 0x7f614814e570 2024-09-13T18:15:37.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.001+0000 7f61457fa700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6134066800 0x7f6134068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:37.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.001+0000 7f614ca3d700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f614814e570 0x7f61480c28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:37.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.001+0000 7f61457fa700 1 -- 172.21.15.40:0/1081858300 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6138085700 con 0x7f6148157e90 2024-09-13T18:15:37.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.001+0000 7f614d23e700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6134066800 0x7f6134068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:37.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.005+0000 7f614ca3d700 1 -- 172.21.15.40:0/1081858300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f61480c62f0 con 0x7f614814e570 2024-09-13T18:15:37.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.005+0000 7f614ca3d700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f614814e570 0x7f61480c28c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f613c00b310 tx=0x7f613c00b6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:37.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.005+0000 7f614d23e700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6134066800 0x7f6134068cc0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f61400020f0 tx=0x7f6140009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:37.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.005+0000 7f61457fa700 1 -- 172.21.15.40:0/1081858300 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c011630 con 0x7f614814e570 2024-09-13T18:15:37.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.005+0000 7f61457fa700 1 -- 172.21.15.40:0/1081858300 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f613c011790 con 0x7f614814e570 2024-09-13T18:15:37.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.009+0000 7f61457fa700 1 -- 172.21.15.40:0/1081858300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f614814e570 msgr2=0x7f61480c28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:37.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.009+0000 7f61457fa700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f614814e570 0x7f61480c28c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f613c00b310 tx=0x7f613c00b6d0 comp rx=0 tx=0).stop 2024-09-13T18:15:37.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.009+0000 7f61457fa700 1 -- 172.21.15.40:0/1081858300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f614814e570 msgr2=0x7f61480c28c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:15:37.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.013+0000 7f61457fa700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f614814e570 0x7f61480c28c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:37.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.049+0000 7f614da3f700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61480c8bd0 0x7f6148159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:37.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.049+0000 7f614da3f700 1 -- 172.21.15.40:0/1081858300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6148067720 con 0x7f61480c8bd0 2024-09-13T18:15:37.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.049+0000 7f614ca3d700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61480c8bd0 0x7f6148159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:37.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.049+0000 7f614ca3d700 1 -- 172.21.15.40:0/1081858300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f61480c6010 con 0x7f61480c8bd0 2024-09-13T18:15:37.056 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.049+0000 7f614ca3d700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61480c8bd0 0x7f6148159400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f613c016000 tx=0x7f613c004820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:37.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.049+0000 7f61457fa700 1 -- 172.21.15.40:0/1081858300 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c025680 con 0x7f61480c8bd0 2024-09-13T18:15:37.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.049+0000 7f61457fa700 1 -- 172.21.15.40:0/1081858300 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4617 (secure 0 0 0) 0x7f613c011e90 con 0x7f61480c8bd0 2024-09-13T18:15:37.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.053+0000 7f61457fa700 1 -- 172.21.15.40:0/1081858300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61480c8bd0 msgr2=0x7f6148159400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:37.058 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.053+0000 7f61457fa700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61480c8bd0 0x7f6148159400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f613c016000 tx=0x7f613c004820 comp rx=0 tx=0).stop 2024-09-13T18:15:37.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.053+0000 7f61457fa700 1 -- 172.21.15.40:0/1081858300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61480c8bd0 msgr2=0x7f6148159400 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:15:37.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.053+0000 7f61457fa700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61480c8bd0 0x7f6148159400 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:37.059 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:15:37.059 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:15:37.060 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:15:37.060 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:15:37.060 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 204, 2024-09-13T18:15:37.060 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:15:37.060 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:15:37.060 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:15:37.060 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:15:37.060 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:37.060 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 16, 2024-09-13T18:15:37.060 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:15:37.060 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:15:37.060 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:15:37.060 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:37.061 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:37.061 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:37.061 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:37.061 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:37.061 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:37.061 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:37.061 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:37.061 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:37.061 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:37.061 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:37.061 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:15:37.061 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:15:37.061 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:37.061 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:37.062 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:37.062 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:37.062 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:37.062 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:37.062 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:37.062 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:37.062 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:37.062 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:37.062 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:37.062 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.062 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:15:37.062 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:15:37.062 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:15:37.063 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:15:37.063 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:15:37.063 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:15:37.063 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:15:37.063 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:15:37.063 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:15:37.063 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:15:37.063 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:15:37.063 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:15:37.063 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:15:37.063 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:15:37.063 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:15:37.063 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:15:37.064 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:15:37.064 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:37.064 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:37.064 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:37.064 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:37.064 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:37.064 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:37.064 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:37.064 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:37.064 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:37.064 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:37.064 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:37.064 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:15:37.064 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.065 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:15:37.065 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.065 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:15:37.065 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:15:37.065 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:37.065 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:37.065 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.065 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:37.065 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:15:37.065 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:37.065 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.065 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.065 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:37.066 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:15:37.066 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:37.066 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:37.066 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:37.066 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.066 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:15:37.066 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:15:37.066 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:37.066 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:37.066 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:37.066 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.066 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.067 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:15:37.067 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:15:37.067 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:37.067 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:37.067 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.067 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:37.067 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:15:37.067 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:37.067 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.067 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.067 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:37.067 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:15:37.067 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:37.067 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:37.068 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:37.068 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.068 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:15:37.068 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:15:37.068 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:37.068 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:37.068 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:37.068 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.068 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.068 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:15:37.068 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:15:37.068 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:37.068 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:37.069 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.069 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:37.069 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:15:37.069 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:37.069 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.069 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.069 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:37.069 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:15:37.069 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:37.069 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:37.069 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:37.069 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.069 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:15:37.069 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:15:37.070 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:37.070 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:37.070 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:37.070 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:37.070 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:37.070 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.070 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:15:37.070 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:15:37.070 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.070 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:37.070 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:37.070 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:15:37.070 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:37.071 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:37.071 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:15:37.071 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.071 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:37.071 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:37.071 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:15:37.071 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:37.071 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:37.071 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:15:37.071 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.071 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:37.071 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:37.071 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:15:37.071 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:37.072 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:37.072 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.072 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:15:37.072 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:15:37.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.069+0000 7f614da3f700 1 -- 172.21.15.40:0/1081858300 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6134066800 msgr2=0x7f6134068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:37.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.069+0000 7f614da3f700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6134066800 0x7f6134068cc0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f61400020f0 tx=0x7f6140009040 comp rx=0 tx=0).stop 2024-09-13T18:15:37.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.069+0000 7f614da3f700 1 -- 172.21.15.40:0/1081858300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6148157e90 msgr2=0x7f6148160d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:37.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.069+0000 7f614da3f700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6148157e90 0x7f6148160d00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f614806b430 tx=0x7f6138004770 comp rx=0 tx=0).stop 2024-09-13T18:15:37.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.069+0000 7f614d23e700 1 -- 172.21.15.40:0/1081858300 reap_dead start 2024-09-13T18:15:37.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.069+0000 7f614da3f700 1 -- 172.21.15.40:0/1081858300 shutdown_connections 2024-09-13T18:15:37.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.077+0000 7f614da3f700 1 --2- 172.21.15.40:0/1081858300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6148157e90 0x7f6148160d00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:37.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.077+0000 7f614da3f700 1 -- 172.21.15.40:0/1081858300 >> 172.21.15.40:0/1081858300 conn(0x7f6148066090 msgr2=0x7f614806d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:37.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.077+0000 7f614da3f700 1 -- 172.21.15.40:0/1081858300 shutdown_connections 2024-09-13T18:15:37.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.081+0000 7f614da3f700 1 -- 172.21.15.40:0/1081858300 wait complete. 2024-09-13T18:15:37.112 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:15:37.112 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:15:37.112 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:15:37.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.397+0000 7f9a65903700 1 Processor -- start 2024-09-13T18:15:37.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.397+0000 7f9a65903700 1 -- start start 2024-09-13T18:15:37.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.397+0000 7f9a65903700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9a60163d00 0x7f9a600c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:37.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.397+0000 7f9a65903700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a600c8340 0x7f9a600c6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:37.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.397+0000 7f9a65903700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a60167290 con 0x7f9a600c7f60 2024-09-13T18:15:37.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.397+0000 7f9a65903700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a600c1630 con 0x7f9a60163d00 2024-09-13T18:15:37.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.397+0000 7f9a65903700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a60166680 con 0x7f9a600c8340 2024-09-13T18:15:37.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.397+0000 7f9a5effd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9a600c7f60 0x7f9a601635f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:47622/0 (socket says 172.21.15.40:47622) 2024-09-13T18:15:37.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.397+0000 7f9a5effd700 1 -- 172.21.15.40:0/3455811937 learned_addr learned my addr 172.21.15.40:0/3455811937 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:15:37.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.397+0000 7f9a5e7fc700 1 --2- 172.21.15.40:0/3455811937 >> v2:172.21.15.43:3300/0 conn(0x7f9a60163d00 0x7f9a600c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:37.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.397+0000 7f9a5f7fe700 1 --2- 172.21.15.40:0/3455811937 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a600c8340 0x7f9a600c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:37.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.397+0000 7f9a5f7fe700 1 -- 172.21.15.40:0/3455811937 >> v2:172.21.15.43:3300/0 conn(0x7f9a60163d00 msgr2=0x7f9a600c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:15:37.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.397+0000 7f9a5f7fe700 1 --2- 172.21.15.40:0/3455811937 >> v2:172.21.15.43:3300/0 conn(0x7f9a60163d00 0x7f9a600c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:37.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.397+0000 7f9a5f7fe700 1 -- 172.21.15.40:0/3455811937 >> v1:172.21.15.40:6789/0 conn(0x7f9a600c7f60 legacy=0x7f9a601635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:37.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.397+0000 7f9a5f7fe700 1 -- 172.21.15.40:0/3455811937 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a60166960 con 0x7f9a600c8340 2024-09-13T18:15:37.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.397+0000 7f9a5f7fe700 1 --2- 172.21.15.40:0/3455811937 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a600c8340 0x7f9a600c6190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400b900 tx=0x7f9a5400bc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a65000d4038f7111 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:37.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.397+0000 7f9a5dffb700 1 -- 172.21.15.40:0/3455811937 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441767961 0 0) 0x7f9a60167290 con 0x7f9a600c7f60 2024-09-13T18:15:37.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.401+0000 7f9a5dffb700 1 -- 172.21.15.40:0/3455811937 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5400ca30 con 0x7f9a600c8340 2024-09-13T18:15:37.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.401+0000 7f9a5dffb700 1 -- 172.21.15.40:0/3455811937 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5400cb90 con 0x7f9a600c8340 2024-09-13T18:15:37.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.401+0000 7f9a65903700 1 -- 172.21.15.40:0/3455811937 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a600c8340 msgr2=0x7f9a600c6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:37.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.401+0000 7f9a65903700 1 --2- 172.21.15.40:0/3455811937 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a600c8340 0x7f9a600c6190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400b900 tx=0x7f9a5400bc10 comp rx=0 tx=0).stop 2024-09-13T18:15:37.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.401+0000 7f9a65903700 1 -- 172.21.15.40:0/3455811937 shutdown_connections 2024-09-13T18:15:37.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.401+0000 7f9a65903700 1 --2- 172.21.15.40:0/3455811937 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a600c8340 0x7f9a600c6190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:37.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.401+0000 7f9a65903700 1 --2- 172.21.15.40:0/3455811937 >> v2:172.21.15.43:3300/0 conn(0x7f9a60163d00 0x7f9a600c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:37.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.401+0000 7f9a65903700 1 -- 172.21.15.40:0/3455811937 >> 172.21.15.40:0/3455811937 conn(0x7f9a60066090 msgr2=0x7f9a600664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:37.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.401+0000 7f9a65903700 1 -- 172.21.15.40:0/3455811937 shutdown_connections 2024-09-13T18:15:37.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.401+0000 7f9a65903700 1 -- 172.21.15.40:0/3455811937 wait complete. 2024-09-13T18:15:37.410 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.401+0000 7f9a65903700 1 Processor -- start 2024-09-13T18:15:37.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.405+0000 7f9a65903700 1 -- start start 2024-09-13T18:15:37.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.405+0000 7f9a65903700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a600c7f60 0x7f9a600c4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:37.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.405+0000 7f9a65903700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a60163d00 0x7f9a600c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:37.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.405+0000 7f9a65903700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a600c92b0 0x7f9a600711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:37.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.405+0000 7f9a65903700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a601602b0 con 0x7f9a600c92b0 2024-09-13T18:15:37.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.405+0000 7f9a65903700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a600c9800 con 0x7f9a60163d00 2024-09-13T18:15:37.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.405+0000 7f9a65903700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a601fc1e0 con 0x7f9a600c7f60 2024-09-13T18:15:37.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.405+0000 7f9a5f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a600c92b0 0x7f9a600711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:37.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.405+0000 7f9a5f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a600c92b0 0x7f9a600711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55386/0 (socket says 172.21.15.40:55386) 2024-09-13T18:15:37.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.405+0000 7f9a5f7fe700 1 -- 172.21.15.40:0/1794246005 learned_addr learned my addr 172.21.15.40:0/1794246005 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:37.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.405+0000 7f9a5e7fc700 1 --2- 172.21.15.40:0/1794246005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a60163d00 0x7f9a600c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:37.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.405+0000 7f9a5f7fe700 1 -- 172.21.15.40:0/1794246005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a600c7f60 msgr2=0x7f9a600c4890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:15:37.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.405+0000 7f9a5f7fe700 1 --2- 172.21.15.40:0/1794246005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a600c7f60 0x7f9a600c4890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:37.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.405+0000 7f9a5f7fe700 1 -- 172.21.15.40:0/1794246005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a60163d00 msgr2=0x7f9a600c4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:37.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.405+0000 7f9a5f7fe700 1 --2- 172.21.15.40:0/1794246005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a60163d00 0x7f9a600c4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:37.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.405+0000 7f9a5f7fe700 1 -- 172.21.15.40:0/1794246005 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5400b560 con 0x7f9a600c92b0 2024-09-13T18:15:37.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.405+0000 7f9a5f7fe700 1 --2- 172.21.15.40:0/1794246005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a600c92b0 0x7f9a600711a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9a54013000 tx=0x7f9a54004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:37.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.413+0000 7f9a47fff700 1 -- 172.21.15.40:0/1794246005 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5401b040 con 0x7f9a600c92b0 2024-09-13T18:15:37.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.413+0000 7f9a65903700 1 -- 172.21.15.40:0/1794246005 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a60072440 con 0x7f9a600c92b0 2024-09-13T18:15:37.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.413+0000 7f9a65903700 1 -- 172.21.15.40:0/1794246005 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a600c9a50 con 0x7f9a600c92b0 2024-09-13T18:15:37.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.413+0000 7f9a47fff700 1 -- 172.21.15.40:0/1794246005 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a54022680 con 0x7f9a600c92b0 2024-09-13T18:15:37.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.413+0000 7f9a47fff700 1 -- 172.21.15.40:0/1794246005 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a54022a10 con 0x7f9a600c92b0 2024-09-13T18:15:37.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.417+0000 7f9a45ffb700 1 --2- 172.21.15.40:0/1794246005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a40002a40 0x7f9a40004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:37.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.417+0000 7f9a47fff700 1 -- 172.21.15.40:0/1794246005 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9a5400ccd0 con 0x7f9a600c92b0 2024-09-13T18:15:37.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.417+0000 7f9a45ffb700 1 -- 172.21.15.40:0/1794246005 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a40007f40 con 0x7f9a40002a40 2024-09-13T18:15:37.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.417+0000 7f9a5effd700 1 --2- 172.21.15.40:0/1794246005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a40002a40 0x7f9a40004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:37.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.421+0000 7f9a5effd700 1 -- 172.21.15.40:0/1794246005 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a40008100 con 0x7f9a40002a40 2024-09-13T18:15:37.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.421+0000 7f9a47fff700 1 --2- 172.21.15.40:0/1794246005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9a4c066800 0x7f9a4c068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:37.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.425+0000 7f9a5e7fc700 1 --2- 172.21.15.40:0/1794246005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9a4c066800 0x7f9a4c068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:37.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.425+0000 7f9a47fff700 1 -- 172.21.15.40:0/1794246005 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9a540869f0 con 0x7f9a600c92b0 2024-09-13T18:15:37.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.425+0000 7f9a5e7fc700 1 --2- 172.21.15.40:0/1794246005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9a4c066800 0x7f9a4c068cc0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9a601ee4a0 tx=0x7f9a58012040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:37.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.429+0000 7f9a5effd700 1 --2- 172.21.15.40:0/1794246005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a40002a40 0x7f9a40004f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9a50000c00 tx=0x7f9a5000c6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:37.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.429+0000 7f9a47fff700 1 -- 172.21.15.40:0/1794246005 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a50004aa0 con 0x7f9a40002a40 2024-09-13T18:15:37.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.429+0000 7f9a47fff700 1 -- 172.21.15.40:0/1794246005 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9a50004c00 con 0x7f9a40002a40 2024-09-13T18:15:37.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.429+0000 7f9a47fff700 1 -- 172.21.15.40:0/1794246005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a40002a40 msgr2=0x7f9a40004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:37.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.429+0000 7f9a47fff700 1 --2- 172.21.15.40:0/1794246005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a40002a40 0x7f9a40004f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9a50000c00 tx=0x7f9a5000c6f0 comp rx=0 tx=0).stop 2024-09-13T18:15:37.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.429+0000 7f9a47fff700 1 -- 172.21.15.40:0/1794246005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a40002a40 msgr2=0x7f9a40004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:15:37.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.429+0000 7f9a47fff700 1 --2- 172.21.15.40:0/1794246005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a40002a40 0x7f9a40004f00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:37.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.453+0000 7f9a45ffb700 1 --2- 172.21.15.40:0/1794246005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a40007020 0x7f9a40007490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:37.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.453+0000 7f9a45ffb700 1 -- 172.21.15.40:0/1794246005 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4000a400 con 0x7f9a40007020 2024-09-13T18:15:37.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.453+0000 7f9a5effd700 1 --2- 172.21.15.40:0/1794246005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a40007020 0x7f9a40007490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.465+0000 7f9a5effd700 1 -- 172.21.15.40:0/1794246005 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9a4000a5c0 con 0x7f9a40007020 2024-09-13T18:15:37.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.465+0000 7f9a5effd700 1 --2- 172.21.15.40:0/1794246005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a40007020 0x7f9a40007490 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9a50013000 tx=0x7f9a50000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:37.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.465+0000 7f9a47fff700 1 -- 172.21.15.40:0/1794246005 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a50020070 con 0x7f9a40007020 2024-09-13T18:15:37.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.465+0000 7f9a47fff700 1 -- 172.21.15.40:0/1794246005 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4949 (secure 0 0 0) 0x7f9a5001b320 con 0x7f9a40007020 2024-09-13T18:15:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.465+0000 7f9a47fff700 1 -- 172.21.15.40:0/1794246005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a40007020 msgr2=0x7f9a40007490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:37.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.465+0000 7f9a47fff700 1 --2- 172.21.15.40:0/1794246005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a40007020 0x7f9a40007490 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9a50013000 tx=0x7f9a50000ed0 comp rx=0 tx=0).stop 2024-09-13T18:15:37.473 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:15:37.473 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:15:37.473 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:15:37.473 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:15:37.473 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 204, 2024-09-13T18:15:37.473 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:15:37.473 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:15:37.473 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:15:37.474 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:15:37.474 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:37.474 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 16, 2024-09-13T18:15:37.474 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:15:37.474 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:15:37.474 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:15:37.474 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:37.474 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:37.474 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:37.474 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:37.474 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:37.474 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:37.474 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:37.474 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:37.475 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:37.475 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:37.475 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:37.475 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:15:37.475 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:15:37.475 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:37.475 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:37.475 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:37.475 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:37.475 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:37.475 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:37.475 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:37.475 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:37.475 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:37.476 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:37.476 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:37.476 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.476 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:15:37.476 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:15:37.476 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:15:37.476 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:15:37.476 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:15:37.476 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:15:37.476 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:15:37.476 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:15:37.476 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:15:37.476 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:15:37.477 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:15:37.477 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:15:37.477 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:15:37.477 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:15:37.477 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:15:37.477 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:15:37.477 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:15:37.477 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:37.477 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:37.477 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:37.477 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:37.477 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:37.477 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:37.477 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:37.477 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:37.478 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:37.478 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:37.478 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:37.478 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:15:37.478 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.478 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:15:37.478 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.478 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:15:37.478 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:15:37.478 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:37.478 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:37.478 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.478 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:37.480 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:15:37.480 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:37.480 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.480 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.480 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:37.480 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:15:37.480 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:37.480 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:37.480 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:37.480 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.481 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:15:37.481 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:15:37.481 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:37.481 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:37.481 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:37.481 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.481 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.481 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:15:37.481 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:15:37.481 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:37.481 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:37.481 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.481 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:37.482 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:15:37.482 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:37.482 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.482 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.482 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:37.482 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:15:37.482 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:37.482 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:37.482 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:37.482 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.482 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:15:37.482 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:15:37.482 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:37.483 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:37.483 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:37.483 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.483 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.483 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:15:37.483 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:15:37.483 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:37.483 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:37.483 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.483 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:37.483 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:15:37.483 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:37.483 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.484 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.484 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:37.484 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:15:37.484 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:37.484 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:37.484 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:37.484 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.484 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:15:37.484 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:15:37.484 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:37.484 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:37.484 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:37.484 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:37.485 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:37.485 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.485 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:15:37.485 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:15:37.485 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.485 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:37.485 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:37.485 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:15:37.485 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:37.485 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:37.485 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:15:37.486 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.486 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:37.486 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:37.486 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:15:37.486 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:37.486 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:37.486 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:15:37.486 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.486 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:37.486 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:37.486 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 7 2024-09-13T18:15:37.486 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:37.486 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:37.487 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:15:37.487 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.487 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:37.487 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:37.487 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:15:37.487 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:37.487 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:37.487 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:15:37.487 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:37.487 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:37.487 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:37.487 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:15:37.487 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:37.488 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:37.488 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:37.488 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:15:37.488 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:15:37.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.465+0000 7f9a47fff700 1 -- 172.21.15.40:0/1794246005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a40007020 msgr2=0x7f9a40007490 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:15:37.489 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.465+0000 7f9a47fff700 1 --2- 172.21.15.40:0/1794246005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9a40007020 0x7f9a40007490 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:37.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.469+0000 7f9a65903700 1 -- 172.21.15.40:0/1794246005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9a4c066800 msgr2=0x7f9a4c068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:37.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.469+0000 7f9a65903700 1 --2- 172.21.15.40:0/1794246005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9a4c066800 0x7f9a4c068cc0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9a601ee4a0 tx=0x7f9a58012040 comp rx=0 tx=0).stop 2024-09-13T18:15:37.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.469+0000 7f9a5f7fe700 1 -- 172.21.15.40:0/1794246005 reap_dead start 2024-09-13T18:15:37.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.469+0000 7f9a65903700 1 -- 172.21.15.40:0/1794246005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a600c92b0 msgr2=0x7f9a600711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:37.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.469+0000 7f9a65903700 1 --2- 172.21.15.40:0/1794246005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a600c92b0 0x7f9a600711a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9a54013000 tx=0x7f9a54004770 comp rx=0 tx=0).stop 2024-09-13T18:15:37.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.469+0000 7f9a65903700 1 -- 172.21.15.40:0/1794246005 shutdown_connections 2024-09-13T18:15:37.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.469+0000 7f9a65903700 1 --2- 172.21.15.40:0/1794246005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a600c92b0 0x7f9a600711a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:37.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.469+0000 7f9a65903700 1 -- 172.21.15.40:0/1794246005 >> 172.21.15.40:0/1794246005 conn(0x7f9a60066090 msgr2=0x7f9a60164930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:37.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.469+0000 7f9a65903700 1 -- 172.21.15.40:0/1794246005 shutdown_connections 2024-09-13T18:15:37.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:37.469+0000 7f9a65903700 1 -- 172.21.15.40:0/1794246005 wait complete. 2024-09-13T18:15:37.526 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:15:37.526 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-13T18:15:37.527 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T18:15:37.527 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-13T18:15:37.527 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-13T18:15:37.527 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T18:15:37.527 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:15:37.555 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T18:15:37.556 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T18:15:37.556 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:15:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.085+0000 7fdfddbe5700 1 Processor -- start 2024-09-13T18:15:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.085+0000 7fdfddbe5700 1 -- start start 2024-09-13T18:15:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfddbe5700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdfd8167d10 0x7fdfd81669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfddbe5700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdfd80704e0 0x7fdfd806a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfddbe5700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd8071930 con 0x7fdfd8160390 2024-09-13T18:15:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfddbe5700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfd8166f20 con 0x7fdfd80704e0 2024-09-13T18:15:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfddbe5700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfd8070a40 con 0x7fdfd8167d10 2024-09-13T18:15:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfdcbe3700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7fdfd8160390 legacy=0x7fdfd8167600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:15:38.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfd7fff700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7fdfd8167d10 msgr2=0x7fdfd81669e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:15:38.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfd7fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdfd8167d10 0x7fdfd81669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:15:38.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfdd3e4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdfd80704e0 0x7fdfd806a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:38.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfdd3e4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdfd80704e0 0x7fdfd806a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55388/0 (socket says 172.21.15.40:55388) 2024-09-13T18:15:38.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfdd3e4700 1 -- 172.21.15.40:0/4119355214 learned_addr learned my addr 172.21.15.40:0/4119355214 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:38.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfdd3e4700 1 -- 172.21.15.40:0/4119355214 >> v2:172.21.15.43:3300/0 conn(0x7fdfd8167d10 msgr2=0x7fdfd81669e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:15:38.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfdd3e4700 1 --2- 172.21.15.40:0/4119355214 >> v2:172.21.15.43:3300/0 conn(0x7fdfd8167d10 0x7fdfd81669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:38.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfdd3e4700 1 -- 172.21.15.40:0/4119355214 >> v1:172.21.15.43:6789/0 conn(0x7fdfd8160390 legacy=0x7fdfd8167600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:15:38.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfdd3e4700 1 -- 172.21.15.40:0/4119355214 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd8070d20 con 0x7fdfd80704e0 2024-09-13T18:15:38.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfdd3e4700 1 --2- 172.21.15.40:0/4119355214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfd80704e0 0x7fdfd806a390 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8072a80 tx=0x7fdfcc00d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b452eb2d8e5309cf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:38.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfd77fe700 1 -- 172.21.15.40:0/4119355214 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc010070 con 0x7fdfd80704e0 2024-09-13T18:15:38.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfd77fe700 1 -- 172.21.15.40:0/4119355214 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfcc008a30 con 0x7fdfd80704e0 2024-09-13T18:15:38.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfd77fe700 1 -- 172.21.15.40:0/4119355214 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc00dbc0 con 0x7fdfd80704e0 2024-09-13T18:15:38.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfddbe5700 1 -- 172.21.15.40:0/4119355214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfd80704e0 msgr2=0x7fdfd806a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:38.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.089+0000 7fdfddbe5700 1 --2- 172.21.15.40:0/4119355214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfd80704e0 0x7fdfd806a390 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8072a80 tx=0x7fdfcc00d330 comp rx=0 tx=0).stop 2024-09-13T18:15:38.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.093+0000 7fdfddbe5700 1 -- 172.21.15.40:0/4119355214 shutdown_connections 2024-09-13T18:15:38.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.093+0000 7fdfddbe5700 1 --2- 172.21.15.40:0/4119355214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfd80704e0 0x7fdfd806a390 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:38.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.093+0000 7fdfddbe5700 1 --2- 172.21.15.40:0/4119355214 >> v2:172.21.15.43:3300/0 conn(0x7fdfd8167d10 0x7fdfd81669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:38.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.093+0000 7fdfddbe5700 1 -- 172.21.15.40:0/4119355214 >> 172.21.15.40:0/4119355214 conn(0x7fdfd80cabc0 msgr2=0x7fdfd80cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:38.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.101+0000 7fdfddbe5700 1 -- 172.21.15.40:0/4119355214 shutdown_connections 2024-09-13T18:15:38.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.105+0000 7fdfddbe5700 1 -- 172.21.15.40:0/4119355214 wait complete. 2024-09-13T18:15:38.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.109+0000 7fdfddbe5700 1 Processor -- start 2024-09-13T18:15:38.118 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.109+0000 7fdfddbe5700 1 -- start start 2024-09-13T18:15:38.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.113+0000 7fdfddbe5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd80704e0 0x7fdfd80c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:38.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.113+0000 7fdfddbe5700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdfd8160390 0x7fdfd815e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.113+0000 7fdfddbe5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfd8167d10 0x7fdfd81efa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.113+0000 7fdfddbe5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd81fd300 con 0x7fdfd8167d10 2024-09-13T18:15:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.113+0000 7fdfddbe5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd81fcd60 con 0x7fdfd80704e0 2024-09-13T18:15:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.113+0000 7fdfddbe5700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd80c6c10 con 0x7fdfd8160390 2024-09-13T18:15:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.113+0000 7fdfd7fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdfd8160390 0x7fdfd815e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.113+0000 7fdfd7fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdfd8160390 0x7fdfd815e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:41530/0 (socket says 172.21.15.40:41530) 2024-09-13T18:15:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.113+0000 7fdfd7fff700 1 -- 172.21.15.40:0/4056571480 learned_addr learned my addr 172.21.15.40:0/4056571480 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.113+0000 7fdfdcbe3700 1 -- 172.21.15.40:0/4056571480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd80704e0 msgr2=0x7fdfd80c52b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:15:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.113+0000 7fdfdcbe3700 1 --2- 172.21.15.40:0/4056571480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd80704e0 0x7fdfd80c52b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:15:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.113+0000 7fdfd7fff700 1 -- 172.21.15.40:0/4056571480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd80704e0 msgr2=0x7fdfd80c52b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:15:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.113+0000 7fdfd7fff700 1 --2- 172.21.15.40:0/4056571480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd80704e0 0x7fdfd80c52b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.113+0000 7fdfd7fff700 1 -- 172.21.15.40:0/4056571480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfd8167d10 msgr2=0x7fdfd81efa50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:15:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.113+0000 7fdfd7fff700 1 --2- 172.21.15.40:0/4056571480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfd8167d10 0x7fdfd81efa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:38.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.113+0000 7fdfd7fff700 1 -- 172.21.15.40:0/4056571480 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc0085f0 con 0x7fdfd8160390 2024-09-13T18:15:38.127 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.117+0000 7fdfd7fff700 1 --2- 172.21.15.40:0/4056571480 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdfd8160390 0x7fdfd815e950 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000b9e0 tx=0x7fdfd000bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:38.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.133+0000 7fdfd57fa700 1 -- 172.21.15.40:0/4056571480 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd000ca30 con 0x7fdfd8160390 2024-09-13T18:15:38.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.133+0000 7fdfd57fa700 1 -- 172.21.15.40:0/4056571480 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd000cbb0 con 0x7fdfd8160390 2024-09-13T18:15:38.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.133+0000 7fdfd57fa700 1 -- 172.21.15.40:0/4056571480 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0015470 con 0x7fdfd8160390 2024-09-13T18:15:38.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.133+0000 7fdfddbe5700 1 -- 172.21.15.40:0/4056571480 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd81fd020 con 0x7fdfd8160390 2024-09-13T18:15:38.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.137+0000 7fdfddbe5700 1 -- 172.21.15.40:0/4056571480 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd81eef70 con 0x7fdfd8160390 2024-09-13T18:15:38.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.137+0000 7fdfddbe5700 1 -- 172.21.15.40:0/4056571480 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd8071580 con 0x7fdfd8160390 2024-09-13T18:15:38.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.137+0000 7fdfd57fa700 1 -- 172.21.15.40:0/4056571480 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdfd0012070 con 0x7fdfd8160390 2024-09-13T18:15:38.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.141+0000 7fdfd57fa700 1 --2- 172.21.15.40:0/4056571480 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdfc4066810 0x7fdfc4068cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:38.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.141+0000 7fdfdcbe3700 1 --2- 172.21.15.40:0/4056571480 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdfc4066810 0x7fdfc4068cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:38.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.141+0000 7fdfdcbe3700 1 --2- 172.21.15.40:0/4056571480 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdfc4066810 0x7fdfc4068cd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8006fd0 tx=0x7fdfc8006de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:38.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.141+0000 7fdfd57fa700 1 -- 172.21.15.40:0/4056571480 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdfd0084850 con 0x7fdfd8160390 2024-09-13T18:15:38.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.149+0000 7fdfd57fa700 1 -- 172.21.15.40:0/4056571480 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdfd0051870 con 0x7fdfd8160390 2024-09-13T18:15:38.356 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:15:38.360 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:15:38.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.985+0000 7fdfddbe5700 1 -- 172.21.15.40:0/4056571480 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdfd81653e0 con 0x7fdfd8160390 2024-09-13T18:15:38.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.985+0000 7fdfd57fa700 1 -- 172.21.15.40:0/4056571480 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7fdfd0054e90 con 0x7fdfd8160390 2024-09-13T18:15:38.992 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":204,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:15:38.997 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.989+0000 7fdfbeffd700 1 -- 172.21.15.40:0/4056571480 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdfc4066810 msgr2=0x7fdfc4068cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:38.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.989+0000 7fdfbeffd700 1 --2- 172.21.15.40:0/4056571480 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdfc4066810 0x7fdfc4068cd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8006fd0 tx=0x7fdfc8006de0 comp rx=0 tx=0).stop 2024-09-13T18:15:38.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.989+0000 7fdfbeffd700 1 -- 172.21.15.40:0/4056571480 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdfd8160390 msgr2=0x7fdfd815e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:38.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.989+0000 7fdfbeffd700 1 --2- 172.21.15.40:0/4056571480 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdfd8160390 0x7fdfd815e950 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000b9e0 tx=0x7fdfd000bda0 comp rx=0 tx=0).stop 2024-09-13T18:15:39.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.993+0000 7fdfbeffd700 1 -- 172.21.15.40:0/4056571480 shutdown_connections 2024-09-13T18:15:39.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.993+0000 7fdfbeffd700 1 --2- 172.21.15.40:0/4056571480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfd8167d10 0x7fdfd81efa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:39.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.993+0000 7fdfbeffd700 1 --2- 172.21.15.40:0/4056571480 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdfc4066810 0x7fdfc4068cd0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:39.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.993+0000 7fdfbeffd700 1 --2- 172.21.15.40:0/4056571480 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdfd8160390 0x7fdfd815e950 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:39.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.993+0000 7fdfbeffd700 1 --2- 172.21.15.40:0/4056571480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd80704e0 0x7fdfd80c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:39.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.993+0000 7fdfbeffd700 1 -- 172.21.15.40:0/4056571480 >> 172.21.15.40:0/4056571480 conn(0x7fdfd80cabc0 msgr2=0x7fdfd815f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:39.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.993+0000 7fdfbeffd700 1 -- 172.21.15.40:0/4056571480 shutdown_connections 2024-09-13T18:15:39.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:38.993+0000 7fdfbeffd700 1 -- 172.21.15.40:0/4056571480 wait complete. 2024-09-13T18:15:42.040 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:15:42.640 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.633+0000 7fbf5c8fe700 1 Processor -- start 2024-09-13T18:15:42.642 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.633+0000 7fbf5c8fe700 1 -- start start 2024-09-13T18:15:42.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.637+0000 7fbf5c8fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbf500ddf40 0x7fbf5004b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:42.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.637+0000 7fbf5c8fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf50045620 0x7fbf500d4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:42.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.637+0000 7fbf5c8fe700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf50006510 con 0x7fbf500de320 2024-09-13T18:15:42.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.637+0000 7fbf5c8fe700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5004bc40 con 0x7fbf500ddf40 2024-09-13T18:15:42.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.637+0000 7fbf5c8fe700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5004b2d0 con 0x7fbf50045620 2024-09-13T18:15:42.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.637+0000 7fbf56d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbf500ddf40 0x7fbf5004b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:42.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.637+0000 7fbf56d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbf500ddf40 0x7fbf5004b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49208/0 (socket says 172.21.15.40:49208) 2024-09-13T18:15:42.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.637+0000 7fbf56d9d700 1 -- 172.21.15.40:0/3861184960 learned_addr learned my addr 172.21.15.40:0/3861184960 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:42.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.637+0000 7fbf5659c700 1 -- 172.21.15.40:0/3861184960 >> v1:172.21.15.43:6789/0 conn(0x7fbf500de320 legacy=0x7fbf50044f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:15:42.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.637+0000 7fbf56d9d700 1 -- 172.21.15.40:0/3861184960 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf50045620 msgr2=0x7fbf500d4430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:15:42.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.637+0000 7fbf56d9d700 1 --2- 172.21.15.40:0/3861184960 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf50045620 0x7fbf500d4430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:42.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.637+0000 7fbf56d9d700 1 -- 172.21.15.40:0/3861184960 >> v1:172.21.15.43:6789/0 conn(0x7fbf500de320 legacy=0x7fbf50044f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:15:42.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.637+0000 7fbf56d9d700 1 -- 172.21.15.40:0/3861184960 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf50006760 con 0x7fbf500ddf40 2024-09-13T18:15:42.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.637+0000 7fbf56d9d700 1 --2- 172.21.15.40:0/3861184960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf500ddf40 0x7fbf5004b700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbf58063990 tx=0x7fbf58069e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c86827d93a89156 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:42.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.641+0000 7fbf55d9b700 1 -- 172.21.15.40:0/3861184960 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58068a10 con 0x7fbf500ddf40 2024-09-13T18:15:42.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.641+0000 7fbf55d9b700 1 -- 172.21.15.40:0/3861184960 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf58067dd0 con 0x7fbf500ddf40 2024-09-13T18:15:42.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.641+0000 7fbf55d9b700 1 -- 172.21.15.40:0/3861184960 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf580744e0 con 0x7fbf500ddf40 2024-09-13T18:15:42.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.641+0000 7fbf5c8fe700 1 -- 172.21.15.40:0/3861184960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf500ddf40 msgr2=0x7fbf5004b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:42.653 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.641+0000 7fbf5c8fe700 1 --2- 172.21.15.40:0/3861184960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf500ddf40 0x7fbf5004b700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbf58063990 tx=0x7fbf58069e70 comp rx=0 tx=0).stop 2024-09-13T18:15:42.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.653+0000 7fbf5c8fe700 1 -- 172.21.15.40:0/3861184960 shutdown_connections 2024-09-13T18:15:42.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.653+0000 7fbf5c8fe700 1 --2- 172.21.15.40:0/3861184960 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf50045620 0x7fbf500d4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:42.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.653+0000 7fbf5c8fe700 1 --2- 172.21.15.40:0/3861184960 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf500ddf40 0x7fbf5004b700 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:42.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.653+0000 7fbf5c8fe700 1 -- 172.21.15.40:0/3861184960 >> 172.21.15.40:0/3861184960 conn(0x7fbf5004fd00 msgr2=0x7fbf50050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:42.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.653+0000 7fbf5c8fe700 1 -- 172.21.15.40:0/3861184960 shutdown_connections 2024-09-13T18:15:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.653+0000 7fbf5c8fe700 1 -- 172.21.15.40:0/3861184960 wait complete. 2024-09-13T18:15:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.653+0000 7fbf5c8fe700 1 Processor -- start 2024-09-13T18:15:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.653+0000 7fbf5c8fe700 1 -- start start 2024-09-13T18:15:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf5c8fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf50045620 0x7fbf50192290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf5c8fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf500ddf40 0x7fbf500dcb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf5c8fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf500de320 0x7fbf500cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf5c8fe700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf500cceb0 con 0x7fbf500ddf40 2024-09-13T18:15:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf5c8fe700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf501927d0 con 0x7fbf50045620 2024-09-13T18:15:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf5c8fe700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf500cc090 con 0x7fbf500de320 2024-09-13T18:15:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf56d9d700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf50045620 msgr2=0x7fbf50192290 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:15:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf56d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf50045620 0x7fbf50192290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:15:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf5759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf500de320 0x7fbf500cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf5759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf500de320 0x7fbf500cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:55754/0 (socket says 172.21.15.40:55754) 2024-09-13T18:15:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf5759e700 1 -- 172.21.15.40:0/3148595919 learned_addr learned my addr 172.21.15.40:0/3148595919 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf5759e700 1 -- 172.21.15.40:0/3148595919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf50045620 msgr2=0x7fbf50192290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:15:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf5759e700 1 --2- 172.21.15.40:0/3148595919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf50045620 0x7fbf50192290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf5759e700 1 -- 172.21.15.40:0/3148595919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf500ddf40 msgr2=0x7fbf500dcb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:15:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf5659c700 1 --2- 172.21.15.40:0/3148595919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf500ddf40 0x7fbf500dcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf5759e700 1 --2- 172.21.15.40:0/3148595919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf500ddf40 0x7fbf500dcb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf5759e700 1 -- 172.21.15.40:0/3148595919 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5804f5c0 con 0x7fbf500de320 2024-09-13T18:15:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf5659c700 1 --2- 172.21.15.40:0/3148595919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf500ddf40 0x7fbf500dcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:15:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf5759e700 1 --2- 172.21.15.40:0/3148595919 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf500de320 0x7fbf500cb5b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800b410 tx=0x7fbf4800b7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf3f7fe700 1 -- 172.21.15.40:0/3148595919 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf48011830 con 0x7fbf500de320 2024-09-13T18:15:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf3f7fe700 1 -- 172.21.15.40:0/3148595919 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf480119b0 con 0x7fbf500de320 2024-09-13T18:15:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf3f7fe700 1 -- 172.21.15.40:0/3148595919 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf48011d60 con 0x7fbf500de320 2024-09-13T18:15:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf5c8fe700 1 -- 172.21.15.40:0/3148595919 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf500cc2e0 con 0x7fbf500de320 2024-09-13T18:15:42.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.657+0000 7fbf5c8fe700 1 -- 172.21.15.40:0/3148595919 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf500ccbd0 con 0x7fbf500de320 2024-09-13T18:15:42.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.665+0000 7fbf3f7fe700 1 -- 172.21.15.40:0/3148595919 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbf4801a410 con 0x7fbf500de320 2024-09-13T18:15:42.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.665+0000 7fbf3f7fe700 1 --2- 172.21.15.40:0/3148595919 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbf44066830 0x7fbf44068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:42.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.665+0000 7fbf3f7fe700 1 -- 172.21.15.40:0/3148595919 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbf48083f80 con 0x7fbf500de320 2024-09-13T18:15:42.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.665+0000 7fbf56d9d700 1 --2- 172.21.15.40:0/3148595919 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbf44066830 0x7fbf44068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:42.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.665+0000 7fbf5c8fe700 1 -- 172.21.15.40:0/3148595919 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf50050820 con 0x7fbf500de320 2024-09-13T18:15:42.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.669+0000 7fbf56d9d700 1 --2- 172.21.15.40:0/3148595919 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbf44066830 0x7fbf44068cf0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbf5004d880 tx=0x7fbf5804f750 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:42.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:42.685+0000 7fbf3f7fe700 1 -- 172.21.15.40:0/3148595919 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbf48050e80 con 0x7fbf500de320 2024-09-13T18:15:43.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:43.432+0000 7fbf5c8fe700 1 -- 172.21.15.40:0/3148595919 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fbf500474e0 con 0x7fbf500de320 2024-09-13T18:15:43.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:43.432+0000 7fbf3f7fe700 1 -- 172.21.15.40:0/3148595919 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7fbf48015030 con 0x7fbf500de320 2024-09-13T18:15:43.443 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":204,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:15:43.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:43.436+0000 7fbf3d7fa700 1 -- 172.21.15.40:0/3148595919 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbf44066830 msgr2=0x7fbf44068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:43.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:43.436+0000 7fbf3d7fa700 1 --2- 172.21.15.40:0/3148595919 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbf44066830 0x7fbf44068cf0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbf5004d880 tx=0x7fbf5804f750 comp rx=0 tx=0).stop 2024-09-13T18:15:43.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:43.436+0000 7fbf3d7fa700 1 -- 172.21.15.40:0/3148595919 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf500de320 msgr2=0x7fbf500cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:43.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:43.436+0000 7fbf3d7fa700 1 --2- 172.21.15.40:0/3148595919 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf500de320 0x7fbf500cb5b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800b410 tx=0x7fbf4800b7d0 comp rx=0 tx=0).stop 2024-09-13T18:15:43.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:43.436+0000 7fbf3d7fa700 1 -- 172.21.15.40:0/3148595919 shutdown_connections 2024-09-13T18:15:43.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:43.436+0000 7fbf3d7fa700 1 --2- 172.21.15.40:0/3148595919 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbf44066830 0x7fbf44068cf0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:43.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:43.436+0000 7fbf3d7fa700 1 --2- 172.21.15.40:0/3148595919 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf500de320 0x7fbf500cb5b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:43.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:43.436+0000 7fbf3d7fa700 1 --2- 172.21.15.40:0/3148595919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf500ddf40 0x7fbf500dcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:43.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:43.436+0000 7fbf3d7fa700 1 --2- 172.21.15.40:0/3148595919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf50045620 0x7fbf50192290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:43.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:43.436+0000 7fbf3d7fa700 1 -- 172.21.15.40:0/3148595919 >> 172.21.15.40:0/3148595919 conn(0x7fbf5004fd00 msgr2=0x7fbf5004ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:43.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:43.436+0000 7fbf3d7fa700 1 -- 172.21.15.40:0/3148595919 shutdown_connections 2024-09-13T18:15:43.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:43.436+0000 7fbf3d7fa700 1 -- 172.21.15.40:0/3148595919 wait complete. 2024-09-13T18:15:46.464 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:15:46.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:46.812+0000 7f6a8442b700 1 Processor -- start 2024-09-13T18:15:46.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:46.812+0000 7f6a8442b700 1 -- start start 2024-09-13T18:15:46.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:46.812+0000 7f6a8442b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6a7c167d10 0x7f6a7c1669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:46.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:46.812+0000 7f6a8442b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 0x7f6a7c06a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:46.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:46.812+0000 7f6a8442b700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c071930 con 0x7f6a7c160390 2024-09-13T18:15:46.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:46.812+0000 7f6a8442b700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a7c166f20 con 0x7f6a7c167d10 2024-09-13T18:15:46.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:46.812+0000 7f6a8442b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c070a40 con 0x7f6a7c0704e0 2024-09-13T18:15:46.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:46.812+0000 7f6a819c6700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7f6a7c167d10 msgr2=0x7f6a7c1669e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:15:46.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:46.812+0000 7f6a819c6700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6a7c167d10 0x7f6a7c1669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:15:46.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:46.812+0000 7f6a821c7700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6a7c160390 0x7f6a7c167600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:39718/0 (socket says 172.21.15.40:39718) 2024-09-13T18:15:46.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:46.812+0000 7f6a821c7700 1 -- 172.21.15.40:0/3640467449 learned_addr learned my addr 172.21.15.40:0/3640467449 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:15:46.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:46.812+0000 7f6a829c8700 1 --2- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 0x7f6a7c06a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:46.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:46.812+0000 7f6a829c8700 1 -- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 msgr2=0x7f6a7c06a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-09-13T18:15:46.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:46.812+0000 7f6a829c8700 1 -- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 msgr2=0x7f6a7c06a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:15:46.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:46.812+0000 7f6a829c8700 1 --2- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 0x7f6a7c06a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:46.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:46.812+0000 7f6a829c8700 1 --2- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 0x7f6a7c06a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:15:47.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:47.016+0000 7f6a819c6700 1 -- 172.21.15.40:0/3640467449 >> v2:172.21.15.43:3300/0 conn(0x7f6a7c167d10 msgr2=0x7f6a7c1669e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:15:47.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:47.016+0000 7f6a819c6700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.43:3300/0 conn(0x7f6a7c167d10 0x7f6a7c1669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:15:47.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:47.016+0000 7f6a829c8700 1 --2- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 0x7f6a7c06a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:47.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:47.016+0000 7f6a829c8700 1 -- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 msgr2=0x7f6a7c06a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:15:47.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:47.016+0000 7f6a829c8700 1 -- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 msgr2=0x7f6a7c06a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:15:47.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:47.016+0000 7f6a829c8700 1 --2- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 0x7f6a7c06a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:47.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:47.016+0000 7f6a829c8700 1 --2- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 0x7f6a7c06a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:15:47.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:47.420+0000 7f6a829c8700 1 --2- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 0x7f6a7c06a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:47.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:47.420+0000 7f6a819c6700 1 -- 172.21.15.40:0/3640467449 >> v2:172.21.15.43:3300/0 conn(0x7f6a7c167d10 msgr2=0x7f6a7c1669e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:15:47.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:47.420+0000 7f6a819c6700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.43:3300/0 conn(0x7f6a7c167d10 0x7f6a7c1669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:15:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:47.420+0000 7f6a829c8700 1 -- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 msgr2=0x7f6a7c06a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:15:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:47.420+0000 7f6a829c8700 1 -- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 msgr2=0x7f6a7c06a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:15:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:47.420+0000 7f6a829c8700 1 --2- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 0x7f6a7c06a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:47.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:47.420+0000 7f6a829c8700 1 --2- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 0x7f6a7c06a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:15:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:48.224+0000 7f6a819c6700 1 -- 172.21.15.40:0/3640467449 >> v2:172.21.15.43:3300/0 conn(0x7f6a7c167d10 msgr2=0x7f6a7c1669e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:15:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:48.224+0000 7f6a819c6700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.43:3300/0 conn(0x7f6a7c167d10 0x7f6a7c1669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:15:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:48.224+0000 7f6a829c8700 1 --2- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 0x7f6a7c06a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:48.224+0000 7f6a829c8700 1 -- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 msgr2=0x7f6a7c06a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-09-13T18:15:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:48.224+0000 7f6a829c8700 1 -- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 msgr2=0x7f6a7c06a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:15:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:48.224+0000 7f6a829c8700 1 --2- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 0x7f6a7c06a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:48.224+0000 7f6a829c8700 1 --2- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 0x7f6a7c06a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:15:49.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:49.812+0000 7f6a73fff700 1 -- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 msgr2=0x7f6a7c06a390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:15:49.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:49.812+0000 7f6a73fff700 1 --2- 172.21.15.40:0/3640467449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a7c0704e0 0x7f6a7c06a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:49.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:49.812+0000 7f6a73fff700 1 -- 172.21.15.40:0/3640467449 >> v2:172.21.15.43:3300/0 conn(0x7f6a7c167d10 msgr2=0x7f6a7c1669e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:15:49.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:49.812+0000 7f6a73fff700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.43:3300/0 conn(0x7f6a7c167d10 0x7f6a7c1669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:49.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:49.812+0000 7f6a73fff700 1 -- 172.21.15.40:0/3640467449 >> v1:172.21.15.40:6789/0 conn(0x7f6a7c160390 legacy=0x7f6a7c167600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:49.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:49.812+0000 7f6a73fff700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 0x7f6a60005be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:49.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:49.812+0000 7f6a73fff700 1 -- 172.21.15.40:0/3640467449 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c070d20 con 0x7f6a600062c0 2024-09-13T18:15:49.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:49.812+0000 7f6a73fff700 1 -- 172.21.15.40:0/3640467449 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a7c166f20 con 0x7f6a60003760 2024-09-13T18:15:49.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:49.812+0000 7f6a821c7700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 0x7f6a60005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:49.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:49.812+0000 7f6a821c7700 1 -- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 msgr2=0x7f6a60005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-09-13T18:15:49.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:49.812+0000 7f6a821c7700 1 -- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 msgr2=0x7f6a60005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:15:49.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:49.812+0000 7f6a821c7700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 0x7f6a60005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:49.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:49.812+0000 7f6a821c7700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 0x7f6a60005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:15:49.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:49.812+0000 7f6a819c6700 1 -- 172.21.15.40:0/3640467449 >> v1:172.21.15.43:6789/0 conn(0x7f6a600062c0 legacy=0x7f6a60008780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:15:50.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:50.016+0000 7f6a819c6700 1 -- 172.21.15.40:0/3640467449 >> v1:172.21.15.43:6789/0 conn(0x7f6a600062c0 legacy=0x7f6a60008780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:15:50.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:50.016+0000 7f6a821c7700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 0x7f6a60005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:50.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:50.016+0000 7f6a821c7700 1 -- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 msgr2=0x7f6a60005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:15:50.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:50.016+0000 7f6a821c7700 1 -- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 msgr2=0x7f6a60005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:15:50.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:50.016+0000 7f6a821c7700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 0x7f6a60005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:50.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:50.016+0000 7f6a821c7700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 0x7f6a60005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:15:50.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:50.420+0000 7f6a821c7700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 0x7f6a60005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:50.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:50.420+0000 7f6a819c6700 1 -- 172.21.15.40:0/3640467449 >> v1:172.21.15.43:6789/0 conn(0x7f6a600062c0 legacy=0x7f6a60008780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:15:50.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:50.420+0000 7f6a821c7700 1 -- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 msgr2=0x7f6a60005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-09-13T18:15:50.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:50.420+0000 7f6a821c7700 1 -- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 msgr2=0x7f6a60005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:15:50.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:50.420+0000 7f6a821c7700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 0x7f6a60005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:50.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:50.420+0000 7f6a821c7700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 0x7f6a60005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:15:51.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:51.224+0000 7f6a821c7700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 0x7f6a60005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:51.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:51.224+0000 7f6a819c6700 1 -- 172.21.15.40:0/3640467449 >> v1:172.21.15.43:6789/0 conn(0x7f6a600062c0 legacy=0x7f6a60008780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:15:51.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:51.224+0000 7f6a821c7700 1 -- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 msgr2=0x7f6a60005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-09-13T18:15:51.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:51.224+0000 7f6a821c7700 1 -- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 msgr2=0x7f6a60005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:15:51.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:51.224+0000 7f6a821c7700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 0x7f6a60005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:15:51.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:51.224+0000 7f6a821c7700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 0x7f6a60005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:15:52.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.812+0000 7f6a73fff700 1 -- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 msgr2=0x7f6a60005be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:15:52.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.812+0000 7f6a73fff700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60003760 0x7f6a60005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:52.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.812+0000 7f6a73fff700 1 -- 172.21.15.40:0/3640467449 >> v1:172.21.15.43:6789/0 conn(0x7f6a600062c0 legacy=0x7f6a60008780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:15:52.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.812+0000 7f6a73fff700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60009fe0 0x7f6a6000a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:52.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.812+0000 7f6a73fff700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.43:3300/0 conn(0x7f6a7c167d10 0x7f6a60012bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:52.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.812+0000 7f6a73fff700 1 -- 172.21.15.40:0/3640467449 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6000b060 con 0x7f6a6000a9b0 2024-09-13T18:15:52.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.812+0000 7f6a73fff700 1 -- 172.21.15.40:0/3640467449 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a7c166f20 con 0x7f6a60009fe0 2024-09-13T18:15:52.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.812+0000 7f6a73fff700 1 -- 172.21.15.40:0/3640467449 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a7c070a40 con 0x7f6a7c167d10 2024-09-13T18:15:52.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.812+0000 7f6a829c8700 1 -- 172.21.15.40:0/3640467449 reap_dead start 2024-09-13T18:15:52.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.812+0000 7f6a819c6700 1 -- 172.21.15.40:0/3640467449 >> v1:172.21.15.43:6789/0 conn(0x7f6a6000a9b0 legacy=0x7f6a6000f450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:15:52.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.812+0000 7f6a829c8700 1 -- 172.21.15.40:0/3640467449 >> v2:172.21.15.43:3300/0 conn(0x7f6a7c167d10 msgr2=0x7f6a60012bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:15:52.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.812+0000 7f6a829c8700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.43:3300/0 conn(0x7f6a7c167d10 0x7f6a60012bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:15:52.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.812+0000 7f6a821c7700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.40:3300/0 conn(0x7f6a60009fe0 0x7f6a6000a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:52.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.812+0000 7f6a821c7700 1 -- 172.21.15.40:0/3640467449 >> v2:172.21.15.43:3300/0 conn(0x7f6a7c167d10 msgr2=0x7f6a60012bc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:15:52.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.812+0000 7f6a821c7700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.43:3300/0 conn(0x7f6a7c167d10 0x7f6a60012bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:52.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.812+0000 7f6a821c7700 1 -- 172.21.15.40:0/3640467449 >> v1:172.21.15.43:6789/0 conn(0x7f6a6000a9b0 legacy=0x7f6a6000f450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:15:52.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.812+0000 7f6a821c7700 1 -- 172.21.15.40:0/3640467449 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c070d20 con 0x7f6a60009fe0 2024-09-13T18:15:52.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.816+0000 7f6a821c7700 1 --2- 172.21.15.40:0/3640467449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a60009fe0 0x7f6a6000a420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000af80 tx=0x7f6a78007330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b288b1440e3bcdfa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:52.823 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.816+0000 7f6a811c5700 1 -- 172.21.15.40:0/3640467449 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78017460 con 0x7f6a60009fe0 2024-09-13T18:15:52.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.816+0000 7f6a811c5700 1 -- 172.21.15.40:0/3640467449 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a780175c0 con 0x7f6a60009fe0 2024-09-13T18:15:52.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.816+0000 7f6a811c5700 1 -- 172.21.15.40:0/3640467449 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78017930 con 0x7f6a60009fe0 2024-09-13T18:15:52.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.816+0000 7f6a8442b700 1 -- 172.21.15.40:0/3640467449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a60009fe0 msgr2=0x7f6a6000a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:52.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.816+0000 7f6a8442b700 1 --2- 172.21.15.40:0/3640467449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a60009fe0 0x7f6a6000a420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000af80 tx=0x7f6a78007330 comp rx=0 tx=0).stop 2024-09-13T18:15:52.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.816+0000 7f6a8442b700 1 -- 172.21.15.40:0/3640467449 shutdown_connections 2024-09-13T18:15:52.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.816+0000 7f6a8442b700 1 --2- 172.21.15.40:0/3640467449 >> v2:172.21.15.43:3300/0 conn(0x7f6a7c167d10 0x7f6a60012bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:52.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.816+0000 7f6a8442b700 1 --2- 172.21.15.40:0/3640467449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a60009fe0 0x7f6a6000a420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:52.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.816+0000 7f6a8442b700 1 -- 172.21.15.40:0/3640467449 >> 172.21.15.40:0/3640467449 conn(0x7f6a7c0cabc0 msgr2=0x7f6a7c0cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:52.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.816+0000 7f6a8442b700 1 -- 172.21.15.40:0/3640467449 shutdown_connections 2024-09-13T18:15:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.816+0000 7f6a8442b700 1 -- 172.21.15.40:0/3640467449 wait complete. 2024-09-13T18:15:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a8442b700 1 Processor -- start 2024-09-13T18:15:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a8442b700 1 -- start start 2024-09-13T18:15:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a8442b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a60009fe0 0x7f6a7c0c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:52.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a8442b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6000a9b0 0x7f6a7c202b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:52.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a8442b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a7c167d10 0x7f6a7c20d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:52.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a8442b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0c5e30 con 0x7f6a60009fe0 2024-09-13T18:15:52.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a8442b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c20cbe0 con 0x7f6a7c167d10 2024-09-13T18:15:52.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a8442b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0c64d0 con 0x7f6a6000a9b0 2024-09-13T18:15:52.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a829c8700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a7c167d10 msgr2=0x7f6a7c20d3c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:15:52.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a829c8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a7c167d10 0x7f6a7c20d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:15:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a821c7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a60009fe0 0x7f6a7c0c58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a821c7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a60009fe0 0x7f6a7c0c58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44814/0 (socket says 172.21.15.40:44814) 2024-09-13T18:15:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a821c7700 1 -- 172.21.15.40:0/399694973 learned_addr learned my addr 172.21.15.40:0/399694973 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a819c6700 1 --2- 172.21.15.40:0/399694973 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6000a9b0 0x7f6a7c202b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a821c7700 1 -- 172.21.15.40:0/399694973 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6000a9b0 msgr2=0x7f6a7c202b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a821c7700 1 --2- 172.21.15.40:0/399694973 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6000a9b0 0x7f6a7c202b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a821c7700 1 -- 172.21.15.40:0/399694973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a7c167d10 msgr2=0x7f6a7c20d3c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:15:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a821c7700 1 --2- 172.21.15.40:0/399694973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a7c167d10 0x7f6a7c20d3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a821c7700 1 -- 172.21.15.40:0/399694973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7800f040 con 0x7f6a60009fe0 2024-09-13T18:15:52.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a821c7700 1 --2- 172.21.15.40:0/399694973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a60009fe0 0x7f6a7c0c58f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6a78007370 tx=0x7f6a78001050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:52.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.820+0000 7f6a72ffd700 1 -- 172.21.15.40:0/399694973 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78027070 con 0x7f6a60009fe0 2024-09-13T18:15:52.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.824+0000 7f6a8442b700 1 -- 172.21.15.40:0/399694973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7c210f30 con 0x7f6a60009fe0 2024-09-13T18:15:52.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.824+0000 7f6a8442b700 1 -- 172.21.15.40:0/399694973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c0c9b50 con 0x7f6a60009fe0 2024-09-13T18:15:52.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.824+0000 7f6a72ffd700 1 -- 172.21.15.40:0/399694973 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a780079e0 con 0x7f6a60009fe0 2024-09-13T18:15:52.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.824+0000 7f6a72ffd700 1 -- 172.21.15.40:0/399694973 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a78017460 con 0x7f6a60009fe0 2024-09-13T18:15:52.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.824+0000 7f6a72ffd700 1 -- 172.21.15.40:0/399694973 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6a780228d0 con 0x7f6a60009fe0 2024-09-13T18:15:52.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.824+0000 7f6a72ffd700 1 --2- 172.21.15.40:0/399694973 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6a68066860 0x7f6a68068d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:52.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.824+0000 7f6a819c6700 1 --2- 172.21.15.40:0/399694973 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6a68066860 0x7f6a68068d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:52.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.824+0000 7f6a819c6700 1 --2- 172.21.15.40:0/399694973 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6a68066860 0x7f6a68068d20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6a74003ef0 tx=0x7f6a740070b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:52.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.824+0000 7f6a72ffd700 1 -- 172.21.15.40:0/399694973 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6a780858e0 con 0x7f6a60009fe0 2024-09-13T18:15:52.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.828+0000 7f6a8442b700 1 -- 172.21.15.40:0/399694973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a60003210 con 0x7f6a60009fe0 2024-09-13T18:15:52.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:52.836+0000 7f6a72ffd700 1 -- 172.21.15.40:0/399694973 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6a780564b0 con 0x7f6a60009fe0 2024-09-13T18:15:53.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.548+0000 7f6a8442b700 1 -- 172.21.15.40:0/399694973 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f6a60005770 con 0x7f6a60009fe0 2024-09-13T18:15:53.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.548+0000 7f6a72ffd700 1 -- 172.21.15.40:0/399694973 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f6a7801b080 con 0x7f6a60009fe0 2024-09-13T18:15:53.556 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":208,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:15:53.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.552+0000 7f6a70ff9700 1 -- 172.21.15.40:0/399694973 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6a68066860 msgr2=0x7f6a68068d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:53.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.552+0000 7f6a70ff9700 1 --2- 172.21.15.40:0/399694973 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6a68066860 0x7f6a68068d20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6a74003ef0 tx=0x7f6a740070b0 comp rx=0 tx=0).stop 2024-09-13T18:15:53.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.552+0000 7f6a70ff9700 1 -- 172.21.15.40:0/399694973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a60009fe0 msgr2=0x7f6a7c0c58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:53.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.552+0000 7f6a70ff9700 1 --2- 172.21.15.40:0/399694973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a60009fe0 0x7f6a7c0c58f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6a78007370 tx=0x7f6a78001050 comp rx=0 tx=0).stop 2024-09-13T18:15:53.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.552+0000 7f6a70ff9700 1 -- 172.21.15.40:0/399694973 shutdown_connections 2024-09-13T18:15:53.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.552+0000 7f6a70ff9700 1 --2- 172.21.15.40:0/399694973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a7c167d10 0x7f6a7c20d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:53.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.552+0000 7f6a70ff9700 1 --2- 172.21.15.40:0/399694973 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6a68066860 0x7f6a68068d20 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:53.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.552+0000 7f6a70ff9700 1 --2- 172.21.15.40:0/399694973 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6a6000a9b0 0x7f6a7c202b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:53.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.552+0000 7f6a70ff9700 1 --2- 172.21.15.40:0/399694973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6a60009fe0 0x7f6a7c0c58f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:53.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.556+0000 7f6a70ff9700 1 -- 172.21.15.40:0/399694973 >> 172.21.15.40:0/399694973 conn(0x7f6a7c0cabc0 msgr2=0x7f6a7c0c89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:53.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.556+0000 7f6a70ff9700 1 -- 172.21.15.40:0/399694973 shutdown_connections 2024-09-13T18:15:53.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.556+0000 7f6a70ff9700 1 -- 172.21.15.40:0/399694973 wait complete. 2024-09-13T18:15:53.583 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T18:15:53.584 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-13T18:15:53.584 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:15:53.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.824+0000 7fb54e797700 1 Processor -- start 2024-09-13T18:15:53.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.824+0000 7fb54e797700 1 -- start start 2024-09-13T18:15:53.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.828+0000 7fb54e797700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5480c1360 0x7fb5480c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.828+0000 7fb54e797700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb5480c5f10 0x7fb548155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.828+0000 7fb54e797700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5480c75e0 con 0x7fb54814f9a0 2024-09-13T18:15:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.828+0000 7fb54e797700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5480c67c0 con 0x7fb5480c5f10 2024-09-13T18:15:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.828+0000 7fb54d795700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7fb54814f9a0 legacy=0x7fb54806d2d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:15:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.828+0000 7fb54cf94700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5480c1360 0x7fb5480c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.828+0000 7fb54cf94700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5480c1360 0x7fb5480c59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:49782/0 (socket says 172.21.15.40:49782) 2024-09-13T18:15:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.828+0000 7fb54cf94700 1 -- 172.21.15.40:0/195577239 learned_addr learned my addr 172.21.15.40:0/195577239 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.828+0000 7fb54e797700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480c1780 con 0x7fb5480c1360 2024-09-13T18:15:53.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.828+0000 7fb54cf94700 1 -- 172.21.15.40:0/195577239 >> v2:172.21.15.40:3300/0 conn(0x7fb5480c5f10 msgr2=0x7fb548155ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:15:53.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.828+0000 7fb54cf94700 1 --2- 172.21.15.40:0/195577239 >> v2:172.21.15.40:3300/0 conn(0x7fb5480c5f10 0x7fb548155ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:53.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.828+0000 7fb54cf94700 1 -- 172.21.15.40:0/195577239 >> v1:172.21.15.43:6789/0 conn(0x7fb54814f9a0 legacy=0x7fb54806d2d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:15:53.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.828+0000 7fb54cf94700 1 -- 172.21.15.40:0/195577239 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5480c69d0 con 0x7fb5480c1360 2024-09-13T18:15:53.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.832+0000 7fb54cf94700 1 --2- 172.21.15.40:0/195577239 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5480c1360 0x7fb5480c59d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb540009d40 tx=0x7fb54000c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c49254ffb3f1491 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:53.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.832+0000 7fb53ffff700 1 -- 172.21.15.40:0/195577239 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb540009e90 con 0x7fb5480c1360 2024-09-13T18:15:53.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.832+0000 7fb53ffff700 1 -- 172.21.15.40:0/195577239 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54000ca90 con 0x7fb5480c1360 2024-09-13T18:15:53.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.832+0000 7fb54e797700 1 -- 172.21.15.40:0/195577239 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5480c1360 msgr2=0x7fb5480c59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:53.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.832+0000 7fb54e797700 1 --2- 172.21.15.40:0/195577239 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5480c1360 0x7fb5480c59d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb540009d40 tx=0x7fb54000c440 comp rx=0 tx=0).stop 2024-09-13T18:15:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.832+0000 7fb54e797700 1 -- 172.21.15.40:0/195577239 shutdown_connections 2024-09-13T18:15:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.832+0000 7fb54e797700 1 --2- 172.21.15.40:0/195577239 >> v2:172.21.15.40:3300/0 conn(0x7fb5480c5f10 0x7fb548155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.832+0000 7fb54e797700 1 --2- 172.21.15.40:0/195577239 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5480c1360 0x7fb5480c59d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:53.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.832+0000 7fb54e797700 1 -- 172.21.15.40:0/195577239 >> 172.21.15.40:0/195577239 conn(0x7fb548066090 msgr2=0x7fb5480664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:53.844 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.836+0000 7fb54e797700 1 -- 172.21.15.40:0/195577239 shutdown_connections 2024-09-13T18:15:53.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.836+0000 7fb54e797700 1 -- 172.21.15.40:0/195577239 wait complete. 2024-09-13T18:15:53.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.840+0000 7fb54e797700 1 Processor -- start 2024-09-13T18:15:53.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.840+0000 7fb54e797700 1 -- start start 2024-09-13T18:15:53.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.840+0000 7fb54e797700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5480c1360 0x7fb5481f7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:53.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.840+0000 7fb54e797700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5480c5f10 0x7fb5481f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:53.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.840+0000 7fb54e797700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb54814f9a0 0x7fb5481f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:53.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.840+0000 7fb54e797700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb548142f50 con 0x7fb54814f9a0 2024-09-13T18:15:53.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.840+0000 7fb54e797700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb548139ee0 con 0x7fb5480c5f10 2024-09-13T18:15:53.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.840+0000 7fb54e797700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5481f7e70 con 0x7fb5480c1360 2024-09-13T18:15:53.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.840+0000 7fb54cf94700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5480c5f10 msgr2=0x7fb5481f41a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:15:53.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.840+0000 7fb54cf94700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5480c5f10 0x7fb5481f41a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:15:53.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.840+0000 7fb54d795700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5480c1360 0x7fb5481f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:53.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.840+0000 7fb54d795700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5480c1360 0x7fb5481f7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:49784/0 (socket says 172.21.15.40:49784) 2024-09-13T18:15:53.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.840+0000 7fb54d795700 1 -- 172.21.15.40:0/284663566 learned_addr learned my addr 172.21.15.40:0/284663566 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:53.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.840+0000 7fb54d795700 1 -- 172.21.15.40:0/284663566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5480c5f10 msgr2=0x7fb5481f41a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:15:53.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.840+0000 7fb54d795700 1 --2- 172.21.15.40:0/284663566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5480c5f10 0x7fb5481f41a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:53.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.840+0000 7fb54d795700 1 -- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb54814f9a0 msgr2=0x7fb5481f46e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:15:53.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.840+0000 7fb54d795700 1 --2- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb54814f9a0 0x7fb5481f46e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:53.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.840+0000 7fb54d795700 1 -- 172.21.15.40:0/284663566 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54000d040 con 0x7fb5480c1360 2024-09-13T18:15:53.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.840+0000 7fb54d795700 1 --2- 172.21.15.40:0/284663566 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5480c1360 0x7fb5481f7370 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb54400b570 tx=0x7fb54400b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:53.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.840+0000 7fb53dffb700 1 -- 172.21.15.40:0/284663566 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544003e20 con 0x7fb5480c1360 2024-09-13T18:15:53.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.844+0000 7fb54e797700 1 -- 172.21.15.40:0/284663566 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb548154e70 con 0x7fb5480c1360 2024-09-13T18:15:53.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.844+0000 7fb54e797700 1 -- 172.21.15.40:0/284663566 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb548155150 con 0x7fb5480c1360 2024-09-13T18:15:53.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.848+0000 7fb53dffb700 1 -- 172.21.15.40:0/284663566 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb544005070 con 0x7fb5480c1360 2024-09-13T18:15:53.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.852+0000 7fb53dffb700 1 -- 172.21.15.40:0/284663566 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544005420 con 0x7fb5480c1360 2024-09-13T18:15:53.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.852+0000 7fb54e797700 1 --2- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb528002a40 0x7fb528004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:53.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.852+0000 7fb54e797700 1 -- 172.21.15.40:0/284663566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb528007f40 con 0x7fb528002a40 2024-09-13T18:15:53.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.852+0000 7fb53dffb700 1 -- 172.21.15.40:0/284663566 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb5440055c0 con 0x7fb5480c1360 2024-09-13T18:15:53.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.852+0000 7fb54cf94700 1 --2- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb528002a40 0x7fb528004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:53.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.852+0000 7fb54cf94700 1 -- 172.21.15.40:0/284663566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb528008100 con 0x7fb528002a40 2024-09-13T18:15:53.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.852+0000 7fb53dffb700 1 --2- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb52c066710 0x7fb52c068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:53.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.852+0000 7fb54cf94700 1 --2- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb528002a40 0x7fb528004f00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb548156bf0 tx=0x7fb540004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:53.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.852+0000 7fb54df96700 1 --2- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb52c066710 0x7fb52c068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:53.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.852+0000 7fb53dffb700 1 -- 172.21.15.40:0/284663566 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb544084180 con 0x7fb5480c1360 2024-09-13T18:15:53.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.852+0000 7fb54df96700 1 --2- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb52c066710 0x7fb52c068bd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb5380096b0 tx=0x7fb538006c80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:53.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.852+0000 7fb53dffb700 1 -- 172.21.15.40:0/284663566 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb540004850 con 0x7fb528002a40 2024-09-13T18:15:53.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.852+0000 7fb53dffb700 1 -- 172.21.15.40:0/284663566 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb54000ce40 con 0x7fb528002a40 2024-09-13T18:15:53.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.852+0000 7fb53dffb700 1 -- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb528002a40 msgr2=0x7fb528004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:53.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.852+0000 7fb53dffb700 1 --2- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb528002a40 0x7fb528004f00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb548156bf0 tx=0x7fb540004af0 comp rx=0 tx=0).stop 2024-09-13T18:15:53.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.852+0000 7fb53dffb700 1 -- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb528002a40 msgr2=0x7fb528004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:15:53.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.852+0000 7fb53dffb700 1 --2- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb528002a40 0x7fb528004f00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:53.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.876+0000 7fb54e797700 1 --2- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb528006f30 0x7fb528007380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:53.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.876+0000 7fb54e797700 1 -- 172.21.15.40:0/284663566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52800a4a0 con 0x7fb528006f30 2024-09-13T18:15:53.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.876+0000 7fb54cf94700 1 --2- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb528006f30 0x7fb528007380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:53.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.876+0000 7fb54cf94700 1 -- 172.21.15.40:0/284663566 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb5280078c0 con 0x7fb528006f30 2024-09-13T18:15:53.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.876+0000 7fb54cf94700 1 --2- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb528006f30 0x7fb528007380 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb540009fd0 tx=0x7fb540009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:53.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.876+0000 7fb53dffb700 1 -- 172.21.15.40:0/284663566 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54001b710 con 0x7fb528006f30 2024-09-13T18:15:53.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.876+0000 7fb53dffb700 1 -- 172.21.15.40:0/284663566 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4441 (secure 0 0 0) 0x7fb54001b870 con 0x7fb528006f30 2024-09-13T18:15:53.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.876+0000 7fb53dffb700 1 -- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb528006f30 msgr2=0x7fb528007380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:53.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.876+0000 7fb53dffb700 1 --2- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb528006f30 0x7fb528007380 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb540009fd0 tx=0x7fb540009810 comp rx=0 tx=0).stop 2024-09-13T18:15:53.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.876+0000 7fb53dffb700 1 -- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb528006f30 msgr2=0x7fb528007380 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:15:53.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.876+0000 7fb53dffb700 1 --2- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb528006f30 0x7fb528007380 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb540009fd0 tx=0x7fb540009810 comp rx=0 tx=0).stop 2024-09-13T18:15:53.888 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:15:53.888 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:15:53.888 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:15:53.888 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:15:53.888 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 208, 2024-09-13T18:15:53.889 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:15:53.889 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:15:53.889 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:15:53.889 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:53.889 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:15:53.889 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:15:53.889 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:15:53.889 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:15:53.889 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:53.889 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:53.889 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:53.889 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:15:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:15:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:53.890 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:15:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:15:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:15:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:15:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:15:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:15:53.891 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:15:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:15:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:15:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:15:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:15:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:15:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:15:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:15:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:15:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:15:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:15:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:53.892 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:15:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:15:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:15:53.893 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:15:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:15:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:15:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:53.894 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:15:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:15:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:15:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:15:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:53.895 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:15:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:15:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:15:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:15:53.896 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:15:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:15:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:15:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:53.897 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:15:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:15:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:15:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:53.898 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:15:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:15:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:15:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:15:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:53.899 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:53.900 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:53.900 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:15:53.900 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:53.900 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:53.900 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:53.900 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:15:53.900 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:15:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.880+0000 7fb5337fe700 1 -- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb52c066710 msgr2=0x7fb52c068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.880+0000 7fb5337fe700 1 --2- 172.21.15.40:0/284663566 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb52c066710 0x7fb52c068bd0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb5380096b0 tx=0x7fb538006c80 comp rx=0 tx=0).stop 2024-09-13T18:15:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.880+0000 7fb5337fe700 1 -- 172.21.15.40:0/284663566 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5480c1360 msgr2=0x7fb5481f7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.880+0000 7fb5337fe700 1 --2- 172.21.15.40:0/284663566 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5480c1360 0x7fb5481f7370 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb54400b570 tx=0x7fb54400b930 comp rx=0 tx=0).stop 2024-09-13T18:15:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.880+0000 7fb54df96700 1 -- 172.21.15.40:0/284663566 reap_dead start 2024-09-13T18:15:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.880+0000 7fb5337fe700 1 -- 172.21.15.40:0/284663566 shutdown_connections 2024-09-13T18:15:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.880+0000 7fb5337fe700 1 --2- 172.21.15.40:0/284663566 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb5480c1360 0x7fb5481f7370 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.880+0000 7fb5337fe700 1 -- 172.21.15.40:0/284663566 >> 172.21.15.40:0/284663566 conn(0x7fb548066090 msgr2=0x7fb54806a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.880+0000 7fb5337fe700 1 -- 172.21.15.40:0/284663566 shutdown_connections 2024-09-13T18:15:53.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:53.880+0000 7fb5337fe700 1 -- 172.21.15.40:0/284663566 wait complete. 2024-09-13T18:15:53.911 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:15:53.911 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T18:15:53.912 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:15:54.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.212+0000 7fd85d402700 1 Processor -- start 2024-09-13T18:15:54.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.212+0000 7fd85d402700 1 -- start start 2024-09-13T18:15:54.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.212+0000 7fd85d402700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8500f6590 0x7fd850006eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:54.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.212+0000 7fd85d402700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd850044be0 0x7fd850100270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:54.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.212+0000 7fd85d402700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd850006a20 con 0x7fd85004ad40 2024-09-13T18:15:54.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.212+0000 7fd85d402700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8500073f0 con 0x7fd8500f6590 2024-09-13T18:15:54.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.212+0000 7fd85d402700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8501007b0 con 0x7fd850044be0 2024-09-13T18:15:54.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.212+0000 7fd8567fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd850044be0 0x7fd850100270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:54.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.212+0000 7fd8567fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd850044be0 0x7fd850100270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:49794/0 (socket says 172.21.15.40:49794) 2024-09-13T18:15:54.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.212+0000 7fd8567fc700 1 -- 172.21.15.40:0/7243706 learned_addr learned my addr 172.21.15.40:0/7243706 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:54.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.212+0000 7fd8567fc700 1 -- 172.21.15.40:0/7243706 >> v2:172.21.15.40:3300/0 conn(0x7fd8500f6590 msgr2=0x7fd850006eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:15:54.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.212+0000 7fd8567fc700 1 --2- 172.21.15.40:0/7243706 >> v2:172.21.15.40:3300/0 conn(0x7fd8500f6590 0x7fd850006eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:54.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.212+0000 7fd8567fc700 1 -- 172.21.15.40:0/7243706 >> v1:172.21.15.40:6789/0 conn(0x7fd85004ad40 legacy=0x7fd85004bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:15:54.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.212+0000 7fd8567fc700 1 -- 172.21.15.40:0/7243706 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd850005ba0 con 0x7fd850044be0 2024-09-13T18:15:54.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.212+0000 7fd8567fc700 1 --2- 172.21.15.40:0/7243706 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd850044be0 0x7fd850100270 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd858063990 tx=0x7fd858069e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb0faf86195d5d73 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:54.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.212+0000 7fd855ffb700 1 -- 172.21.15.40:0/7243706 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858067dd0 con 0x7fd850044be0 2024-09-13T18:15:54.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.212+0000 7fd855ffb700 1 -- 172.21.15.40:0/7243706 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd858067f30 con 0x7fd850044be0 2024-09-13T18:15:54.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.212+0000 7fd855ffb700 1 -- 172.21.15.40:0/7243706 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858074720 con 0x7fd850044be0 2024-09-13T18:15:54.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.216+0000 7fd85d402700 1 -- 172.21.15.40:0/7243706 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd850044be0 msgr2=0x7fd850100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:54.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.216+0000 7fd85d402700 1 --2- 172.21.15.40:0/7243706 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd850044be0 0x7fd850100270 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd858063990 tx=0x7fd858069e70 comp rx=0 tx=0).stop 2024-09-13T18:15:54.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.220+0000 7fd85d402700 1 -- 172.21.15.40:0/7243706 shutdown_connections 2024-09-13T18:15:54.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.220+0000 7fd85d402700 1 --2- 172.21.15.40:0/7243706 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd850044be0 0x7fd850100270 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:54.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.220+0000 7fd85d402700 1 --2- 172.21.15.40:0/7243706 >> v2:172.21.15.40:3300/0 conn(0x7fd8500f6590 0x7fd850006eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:54.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.220+0000 7fd85d402700 1 -- 172.21.15.40:0/7243706 >> 172.21.15.40:0/7243706 conn(0x7fd85004fd00 msgr2=0x7fd850050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:54.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.220+0000 7fd85d402700 1 -- 172.21.15.40:0/7243706 shutdown_connections 2024-09-13T18:15:54.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.220+0000 7fd85d402700 1 -- 172.21.15.40:0/7243706 wait complete. 2024-09-13T18:15:54.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.220+0000 7fd85d402700 1 Processor -- start 2024-09-13T18:15:54.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.224+0000 7fd85d402700 1 -- start start 2024-09-13T18:15:54.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.224+0000 7fd85d402700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd850044be0 0x7fd8501856d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:54.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.224+0000 7fd85d402700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd85004ad40 0x7fd8500f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:54.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.224+0000 7fd85d402700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8500f6590 0x7fd8500348a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:54.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.224+0000 7fd85d402700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd850035d20 con 0x7fd850044be0 2024-09-13T18:15:54.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.224+0000 7fd85d402700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd850186030 con 0x7fd8500f6590 2024-09-13T18:15:54.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.224+0000 7fd85d402700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85004fa60 con 0x7fd85004ad40 2024-09-13T18:15:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.224+0000 7fd8577fe700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8500f6590 msgr2=0x7fd8500348a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:15:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.224+0000 7fd8577fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8500f6590 0x7fd8500348a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:15:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.224+0000 7fd856ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd850044be0 0x7fd8501856d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.224+0000 7fd856ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd850044be0 0x7fd8501856d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44848/0 (socket says 172.21.15.40:44848) 2024-09-13T18:15:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.224+0000 7fd856ffd700 1 -- 172.21.15.40:0/60106974 learned_addr learned my addr 172.21.15.40:0/60106974 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:15:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.224+0000 7fd856ffd700 1 -- 172.21.15.40:0/60106974 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd85004ad40 msgr2=0x7fd8500f2070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:15:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.224+0000 7fd856ffd700 1 --2- 172.21.15.40:0/60106974 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd85004ad40 0x7fd8500f2070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.224+0000 7fd856ffd700 1 -- 172.21.15.40:0/60106974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8500f6590 msgr2=0x7fd8500348a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:15:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.224+0000 7fd856ffd700 1 --2- 172.21.15.40:0/60106974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8500f6590 0x7fd8500348a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.224+0000 7fd856ffd700 1 -- 172.21.15.40:0/60106974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85804f5c0 con 0x7fd850044be0 2024-09-13T18:15:54.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.228+0000 7fd856ffd700 1 --2- 172.21.15.40:0/60106974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd850044be0 0x7fd8501856d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd84800b550 tx=0x7fd84800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:54.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.228+0000 7fd83f7fe700 1 -- 172.21.15.40:0/60106974 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848011830 con 0x7fd850044be0 2024-09-13T18:15:54.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.228+0000 7fd85d402700 1 -- 172.21.15.40:0/60106974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85004f4e0 con 0x7fd850044be0 2024-09-13T18:15:54.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.228+0000 7fd85d402700 1 -- 172.21.15.40:0/60106974 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85004f780 con 0x7fd850044be0 2024-09-13T18:15:54.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.228+0000 7fd83f7fe700 1 -- 172.21.15.40:0/60106974 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd848011990 con 0x7fd850044be0 2024-09-13T18:15:54.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.228+0000 7fd83f7fe700 1 -- 172.21.15.40:0/60106974 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848011d40 con 0x7fd850044be0 2024-09-13T18:15:54.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.232+0000 7fd83f7fe700 1 -- 172.21.15.40:0/60106974 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd84801a400 con 0x7fd850044be0 2024-09-13T18:15:54.241 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.232+0000 7fd83f7fe700 1 --2- 172.21.15.40:0/60106974 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd844066830 0x7fd844068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:54.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.232+0000 7fd83f7fe700 1 -- 172.21.15.40:0/60106974 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd848084460 con 0x7fd850044be0 2024-09-13T18:15:54.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.232+0000 7fd8567fc700 1 --2- 172.21.15.40:0/60106974 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd844066830 0x7fd844068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:54.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.236+0000 7fd83d7fa700 1 --2- 172.21.15.40:0/60106974 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd838002a40 0x7fd838004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:54.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.236+0000 7fd83d7fa700 1 -- 172.21.15.40:0/60106974 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd838007f40 con 0x7fd838002a40 2024-09-13T18:15:54.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.236+0000 7fd8567fc700 1 --2- 172.21.15.40:0/60106974 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd844066830 0x7fd844068cf0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd8501008f0 tx=0x7fd85804f750 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:54.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.236+0000 7fd8577fe700 1 --2- 172.21.15.40:0/60106974 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd838002a40 0x7fd838004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:54.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.240+0000 7fd8577fe700 1 -- 172.21.15.40:0/60106974 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd838008100 con 0x7fd838002a40 2024-09-13T18:15:54.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.240+0000 7fd8577fe700 1 --2- 172.21.15.40:0/60106974 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd838002a40 0x7fd838004f00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd84c000fe0 tx=0x7fd84c00c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:54.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.240+0000 7fd83f7fe700 1 -- 172.21.15.40:0/60106974 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c0072d0 con 0x7fd838002a40 2024-09-13T18:15:54.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.240+0000 7fd83f7fe700 1 -- 172.21.15.40:0/60106974 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd84c007430 con 0x7fd838002a40 2024-09-13T18:15:54.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.240+0000 7fd83f7fe700 1 -- 172.21.15.40:0/60106974 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd838002a40 msgr2=0x7fd838004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:54.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.240+0000 7fd83f7fe700 1 --2- 172.21.15.40:0/60106974 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd838002a40 0x7fd838004f00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd84c000fe0 tx=0x7fd84c00c710 comp rx=0 tx=0).stop 2024-09-13T18:15:54.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.240+0000 7fd83f7fe700 1 -- 172.21.15.40:0/60106974 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd838002a40 msgr2=0x7fd838004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:15:54.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.240+0000 7fd83f7fe700 1 --2- 172.21.15.40:0/60106974 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd838002a40 0x7fd838004f00 secure :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd84c000fe0 tx=0x7fd84c00c710 comp rx=0 tx=0).stop 2024-09-13T18:15:54.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.256+0000 7fd85d402700 1 --2- 172.21.15.40:0/60106974 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd838001d30 0x7fd83800cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:15:54.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.256+0000 7fd85d402700 1 -- 172.21.15.40:0/60106974 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd838002340 con 0x7fd838001d30 2024-09-13T18:15:54.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.264+0000 7fd8577fe700 1 --2- 172.21.15.40:0/60106974 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd838001d30 0x7fd83800cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:15:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.268+0000 7fd8577fe700 1 -- 172.21.15.40:0/60106974 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd83800a2e0 con 0x7fd838001d30 2024-09-13T18:15:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.268+0000 7fd8577fe700 1 --2- 172.21.15.40:0/60106974 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd838001d30 0x7fd83800cbf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd84c013000 tx=0x7fd84c001050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:15:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.268+0000 7fd83f7fe700 1 -- 172.21.15.40:0/60106974 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c020070 con 0x7fd838001d30 2024-09-13T18:15:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.268+0000 7fd83f7fe700 1 -- 172.21.15.40:0/60106974 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4937 (secure 0 0 0) 0x7fd84c01b110 con 0x7fd838001d30 2024-09-13T18:15:54.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.268+0000 7fd83f7fe700 1 -- 172.21.15.40:0/60106974 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd838001d30 msgr2=0x7fd83800cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:54.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.268+0000 7fd83f7fe700 1 --2- 172.21.15.40:0/60106974 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd838001d30 0x7fd83800cbf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd84c013000 tx=0x7fd84c001050 comp rx=0 tx=0).stop 2024-09-13T18:15:54.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.268+0000 7fd83f7fe700 1 -- 172.21.15.40:0/60106974 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd838001d30 msgr2=0x7fd83800cbf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:15:54.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.268+0000 7fd83f7fe700 1 --2- 172.21.15.40:0/60106974 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd838001d30 0x7fd83800cbf0 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd84c013000 tx=0x7fd84c001050 comp rx=0 tx=0).stop 2024-09-13T18:15:54.288 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:15:54.288 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:15:54.288 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:15:54.288 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:15:54.288 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 208, 2024-09-13T18:15:54.288 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:15:54.288 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:15:54.288 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:15:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:15:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:15:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:15:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:15:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:54.289 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:15:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:15:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:54.290 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:15:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:15:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:15:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:15:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:15:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:15:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:15:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:15:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:15:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:15:54.291 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:15:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:15:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:15:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:15:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:15:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:15:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:15:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:15:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:15:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:15:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:15:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:15:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:15:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:15:54.292 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:15:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:15:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:15:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:15:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:15:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:15:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:15:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:54.293 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:15:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:15:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:15:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:15:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:54.294 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:15:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:15:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:15:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:54.295 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:15:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:15:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:15:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:15:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:15:54.296 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:15:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:15:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:15:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:15:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:15:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:15:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:15:54.297 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:15:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:15:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:15:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:15:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:15:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:15:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:15:54.298 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:15:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:15:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:15:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:54.299 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:15:54.300 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:54.300 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:54.300 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:54.300 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 8 2024-09-13T18:15:54.300 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:54.300 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:54.300 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:15:54.300 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:54.300 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:54.300 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:15:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:15:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:15:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:15:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:15:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:15:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:15:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:15:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:15:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:15:54.301 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:15:54.301 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:15:54.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.280+0000 7fd83d7fa700 1 -- 172.21.15.40:0/60106974 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd844066830 msgr2=0x7fd844068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.280+0000 7fd83d7fa700 1 --2- 172.21.15.40:0/60106974 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd844066830 0x7fd844068cf0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd8501008f0 tx=0x7fd85804f750 comp rx=0 tx=0).stop 2024-09-13T18:15:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.280+0000 7fd83d7fa700 1 -- 172.21.15.40:0/60106974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd850044be0 msgr2=0x7fd8501856d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:15:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.280+0000 7fd83d7fa700 1 --2- 172.21.15.40:0/60106974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd850044be0 0x7fd8501856d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd84800b550 tx=0x7fd84800b910 comp rx=0 tx=0).stop 2024-09-13T18:15:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.280+0000 7fd8577fe700 1 -- 172.21.15.40:0/60106974 reap_dead start 2024-09-13T18:15:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.280+0000 7fd83d7fa700 1 -- 172.21.15.40:0/60106974 shutdown_connections 2024-09-13T18:15:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.280+0000 7fd83d7fa700 1 --2- 172.21.15.40:0/60106974 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd850044be0 0x7fd8501856d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:15:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.280+0000 7fd83d7fa700 1 -- 172.21.15.40:0/60106974 >> 172.21.15.40:0/60106974 conn(0x7fd85004fd00 msgr2=0x7fd8500ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:15:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.280+0000 7fd83d7fa700 1 -- 172.21.15.40:0/60106974 shutdown_connections 2024-09-13T18:15:54.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:15:54.280+0000 7fd83d7fa700 1 -- 172.21.15.40:0/60106974 wait complete. 2024-09-13T18:15:54.313 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:15:54.314 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:16:08.760 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:16:08.763 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:16:14.315 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-13T18:16:14.315 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-13T18:16:14.315 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T18:16:14.316 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T18:16:14.318 INFO:tasks.ceph.mon.b:Started 2024-09-13T18:16:14.318 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:16:14.318 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:16:15.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.064+0000 7f6810e5f700 1 Processor -- start 2024-09-13T18:16:15.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.068+0000 7f6810e5f700 1 -- start start 2024-09-13T18:16:15.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.068+0000 7f6810e5f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f680c0c55c0 0x7f680c0c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:15.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.068+0000 7f6810e5f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680c155f30 con 0x7f680c06a390 2024-09-13T18:16:15.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.068+0000 7f6810e5f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680c0c7110 con 0x7f680c14f5b0 2024-09-13T18:16:15.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.068+0000 7f6810e5f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f680c1553f0 con 0x7f680c0c55c0 2024-09-13T18:16:15.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.068+0000 7f6809d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f680c0c55c0 0x7f680c0c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:15.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.068+0000 7f680ad9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f680c06a390 0x7f680c15fc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53408/0 (socket says 172.21.15.40:53408) 2024-09-13T18:16:15.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.068+0000 7f680ad9d700 1 -- 172.21.15.40:0/1745265800 learned_addr learned my addr 172.21.15.40:0/1745265800 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:16:15.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.068+0000 7f6809d9b700 1 -- 172.21.15.40:0/1745265800 >> v1:172.21.15.43:6789/0 conn(0x7f680c14f5b0 legacy=0x7f680c0c6960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:16:15.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.068+0000 7f6809d9b700 1 -- 172.21.15.40:0/1745265800 >> v1:172.21.15.40:6789/0 conn(0x7f680c06a390 legacy=0x7f680c15fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:15.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.068+0000 7f6809d9b700 1 -- 172.21.15.40:0/1745265800 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c155600 con 0x7f680c0c55c0 2024-09-13T18:16:15.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.072+0000 7f6809d9b700 1 --2- 172.21.15.40:0/1745265800 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f680c0c55c0 0x7f680c0c5a00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009d90 tx=0x7f67f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275d3959954c0c0c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:15.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.088+0000 7f680959a700 1 -- 172.21.15.40:0/1745265800 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f400e070 con 0x7f680c0c55c0 2024-09-13T18:16:15.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.088+0000 7f680959a700 1 -- 172.21.15.40:0/1745265800 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f400bad0 con 0x7f680c0c55c0 2024-09-13T18:16:15.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.092+0000 7f6810e5f700 1 -- 172.21.15.40:0/1745265800 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f680c0c55c0 msgr2=0x7f680c0c5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:15.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.092+0000 7f6810e5f700 1 --2- 172.21.15.40:0/1745265800 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f680c0c55c0 0x7f680c0c5a00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009d90 tx=0x7f67f400b330 comp rx=0 tx=0).stop 2024-09-13T18:16:15.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.092+0000 7f6810e5f700 1 -- 172.21.15.40:0/1745265800 shutdown_connections 2024-09-13T18:16:15.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.092+0000 7f6810e5f700 1 --2- 172.21.15.40:0/1745265800 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f680c0c55c0 0x7f680c0c5a00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:15.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.092+0000 7f6810e5f700 1 -- 172.21.15.40:0/1745265800 >> 172.21.15.40:0/1745265800 conn(0x7f680c0cabc0 msgr2=0x7f680c0cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:15.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.096+0000 7f6810e5f700 1 -- 172.21.15.40:0/1745265800 shutdown_connections 2024-09-13T18:16:15.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.096+0000 7f6810e5f700 1 -- 172.21.15.40:0/1745265800 wait complete. 2024-09-13T18:16:15.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.096+0000 7f6810e5f700 1 Processor -- start 2024-09-13T18:16:15.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.096+0000 7f6810e5f700 1 -- start start 2024-09-13T18:16:15.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f6810e5f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680c06a390 0x7f680c151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:15.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f6810e5f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f680c0c55c0 0x7f680c0bd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:15.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f6810e5f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f680c14f5b0 0x7f680c0ba600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:15.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f6810e5f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c1ffd60 con 0x7f680c0c55c0 2024-09-13T18:16:15.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f6810e5f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c1529c0 con 0x7f680c06a390 2024-09-13T18:16:15.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f6810e5f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680c152470 con 0x7f680c14f5b0 2024-09-13T18:16:15.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f6809d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f680c0c55c0 0x7f680c0bd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:15.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f6809d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f680c0c55c0 0x7f680c0bd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36178/0 (socket says 172.21.15.40:36178) 2024-09-13T18:16:15.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f6809d9b700 1 -- 172.21.15.40:0/3196029210 learned_addr learned my addr 172.21.15.40:0/3196029210 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:16:15.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f680a59c700 1 --2- 172.21.15.40:0/3196029210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680c06a390 0x7f680c151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:15.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f680ad9d700 1 --2- 172.21.15.40:0/3196029210 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f680c14f5b0 0x7f680c0ba600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:15.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f680a59c700 1 -- 172.21.15.40:0/3196029210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680c06a390 msgr2=0x7f680c151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-13T18:16:15.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f680a59c700 1 -- 172.21.15.40:0/3196029210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680c06a390 msgr2=0x7f680c151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:16:15.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f680a59c700 1 --2- 172.21.15.40:0/3196029210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680c06a390 0x7f680c151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:16:15.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f680a59c700 1 --2- 172.21.15.40:0/3196029210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680c06a390 0x7f680c151f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:16:15.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f6809d9b700 1 -- 172.21.15.40:0/3196029210 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f680c14f5b0 msgr2=0x7f680c0ba600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:15.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f6809d9b700 1 --2- 172.21.15.40:0/3196029210 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f680c14f5b0 0x7f680c0ba600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:15.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f6809d9b700 1 -- 172.21.15.40:0/3196029210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680c06a390 msgr2=0x7f680c151f30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:16:15.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f6809d9b700 1 --2- 172.21.15.40:0/3196029210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680c06a390 0x7f680c151f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:15.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f6809d9b700 1 -- 172.21.15.40:0/3196029210 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f4009a70 con 0x7f680c0c55c0 2024-09-13T18:16:15.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f6809d9b700 1 --2- 172.21.15.40:0/3196029210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f680c0c55c0 0x7f680c0bd160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f680c06f150 tx=0x7f67f4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:15.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f6802ffd700 1 -- 172.21.15.40:0/3196029210 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4012650 con 0x7f680c0c55c0 2024-09-13T18:16:15.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f6810e5f700 1 -- 172.21.15.40:0/3196029210 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680c2001f0 con 0x7f680c0c55c0 2024-09-13T18:16:15.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.100+0000 7f6810e5f700 1 -- 172.21.15.40:0/3196029210 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680c1526e0 con 0x7f680c0c55c0 2024-09-13T18:16:15.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.104+0000 7f6810e5f700 1 -- 172.21.15.40:0/3196029210 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67ec005310 con 0x7f680c0c55c0 2024-09-13T18:16:15.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.124+0000 7f6802ffd700 1 -- 172.21.15.40:0/3196029210 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f40127b0 con 0x7f680c0c55c0 2024-09-13T18:16:15.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.124+0000 7f6802ffd700 1 -- 172.21.15.40:0/3196029210 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4012b60 con 0x7f680c0c55c0 2024-09-13T18:16:15.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.124+0000 7f6802ffd700 1 -- 172.21.15.40:0/3196029210 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f67f4019070 con 0x7f680c0c55c0 2024-09-13T18:16:15.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.124+0000 7f6802ffd700 1 --2- 172.21.15.40:0/3196029210 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f67f8066a20 0x7f67f8068ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:15.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.124+0000 7f6802ffd700 1 -- 172.21.15.40:0/3196029210 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f67f4086980 con 0x7f680c0c55c0 2024-09-13T18:16:15.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.124+0000 7f680a59c700 1 --2- 172.21.15.40:0/3196029210 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f67f8066a20 0x7f67f8068ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:15.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.124+0000 7f6802ffd700 1 -- 172.21.15.40:0/3196029210 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f67f40b1e50 con 0x7f680c0c55c0 2024-09-13T18:16:15.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.128+0000 7f680a59c700 1 --2- 172.21.15.40:0/3196029210 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f67f8066a20 0x7f67f8068ee0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006fd0 tx=0x7f67fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:15.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.988+0000 7f6810e5f700 1 -- 172.21.15.40:0/3196029210 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f67ec005890 con 0x7f680c0c55c0 2024-09-13T18:16:15.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.988+0000 7f6802ffd700 1 -- 172.21.15.40:0/3196029210 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f67f4017610 con 0x7f680c0c55c0 2024-09-13T18:16:15.997 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":208,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:16:15.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.992+0000 7f6810e5f700 1 -- 172.21.15.40:0/3196029210 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f67f8066a20 msgr2=0x7f67f8068ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:16.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.992+0000 7f6810e5f700 1 --2- 172.21.15.40:0/3196029210 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f67f8066a20 0x7f67f8068ee0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006fd0 tx=0x7f67fc008040 comp rx=0 tx=0).stop 2024-09-13T18:16:16.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.992+0000 7f6810e5f700 1 -- 172.21.15.40:0/3196029210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f680c0c55c0 msgr2=0x7f680c0bd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:16.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.992+0000 7f6810e5f700 1 --2- 172.21.15.40:0/3196029210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f680c0c55c0 0x7f680c0bd160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f680c06f150 tx=0x7f67f4004af0 comp rx=0 tx=0).stop 2024-09-13T18:16:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.992+0000 7f6810e5f700 1 -- 172.21.15.40:0/3196029210 shutdown_connections 2024-09-13T18:16:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.992+0000 7f6810e5f700 1 --2- 172.21.15.40:0/3196029210 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f67f8066a20 0x7f67f8068ee0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.992+0000 7f6810e5f700 1 --2- 172.21.15.40:0/3196029210 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f680c14f5b0 0x7f680c0ba600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.992+0000 7f6810e5f700 1 --2- 172.21.15.40:0/3196029210 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f680c0c55c0 0x7f680c0bd160 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.992+0000 7f6810e5f700 1 --2- 172.21.15.40:0/3196029210 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f680c06a390 0x7f680c151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.992+0000 7f6810e5f700 1 -- 172.21.15.40:0/3196029210 >> 172.21.15.40:0/3196029210 conn(0x7f680c0cabc0 msgr2=0x7f680c0cafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.992+0000 7f6810e5f700 1 -- 172.21.15.40:0/3196029210 shutdown_connections 2024-09-13T18:16:16.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:15.992+0000 7f6810e5f700 1 -- 172.21.15.40:0/3196029210 wait complete. 2024-09-13T18:16:19.048 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:16:19.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.456+0000 7fc7bac15700 1 Processor -- start 2024-09-13T18:16:19.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.456+0000 7fc7bac15700 1 -- start start 2024-09-13T18:16:19.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.456+0000 7fc7bac15700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc7b4071b40 0x7fc7b4071f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:19.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.456+0000 7fc7bac15700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7b40c55c0 0x7fc7b40c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:19.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.456+0000 7fc7bac15700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b40cba20 con 0x7fc7b4070500 2024-09-13T18:16:19.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.456+0000 7fc7bac15700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7b40c6f80 con 0x7fc7b4071b40 2024-09-13T18:16:19.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.456+0000 7fc7bac15700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b40724c0 con 0x7fc7b40c55c0 2024-09-13T18:16:19.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.460+0000 7fc7b9c13700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc7b4070500 0x7fc7b41500c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53420/0 (socket says 172.21.15.40:53420) 2024-09-13T18:16:19.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.460+0000 7fc7b9c13700 1 -- 172.21.15.40:0/3473741042 learned_addr learned my addr 172.21.15.40:0/3473741042 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:16:19.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.460+0000 7fc7b9412700 1 --2- 172.21.15.40:0/3473741042 >> v2:172.21.15.40:3300/0 conn(0x7fc7b4071b40 0x7fc7b4071f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:19.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.460+0000 7fc7ba414700 1 --2- 172.21.15.40:0/3473741042 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7b40c55c0 0x7fc7b40c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:19.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.460+0000 7fc7b9412700 1 -- 172.21.15.40:0/3473741042 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7b40c55c0 msgr2=0x7fc7b40c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:16:19.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.460+0000 7fc7b9412700 1 --2- 172.21.15.40:0/3473741042 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7b40c55c0 0x7fc7b40c5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:19.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.460+0000 7fc7b9412700 1 -- 172.21.15.40:0/3473741042 >> v1:172.21.15.40:6789/0 conn(0x7fc7b4070500 legacy=0x7fc7b41500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:16:19.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.460+0000 7fc7b9412700 1 -- 172.21.15.40:0/3473741042 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b41560e0 con 0x7fc7b4071b40 2024-09-13T18:16:19.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.460+0000 7fc7b9412700 1 --2- 172.21.15.40:0/3473741042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7b4071b40 0x7fc7b4071f80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009850 tx=0x7fc7b0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f895c2247dca840 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:19.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.460+0000 7fc7b8c11700 1 -- 172.21.15.40:0/3473741042 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0004230 con 0x7fc7b4071b40 2024-09-13T18:16:19.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.460+0000 7fc7b8c11700 1 -- 172.21.15.40:0/3473741042 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b0004390 con 0x7fc7b4071b40 2024-09-13T18:16:19.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.460+0000 7fc7bac15700 1 -- 172.21.15.40:0/3473741042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7b4071b40 msgr2=0x7fc7b4071f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:19.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.460+0000 7fc7bac15700 1 --2- 172.21.15.40:0/3473741042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7b4071b40 0x7fc7b4071f80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009850 tx=0x7fc7b0009b60 comp rx=0 tx=0).stop 2024-09-13T18:16:19.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.472+0000 7fc7bac15700 1 -- 172.21.15.40:0/3473741042 shutdown_connections 2024-09-13T18:16:19.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.472+0000 7fc7bac15700 1 --2- 172.21.15.40:0/3473741042 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7b40c55c0 0x7fc7b40c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:19.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.472+0000 7fc7bac15700 1 --2- 172.21.15.40:0/3473741042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7b4071b40 0x7fc7b4071f80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:19.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.472+0000 7fc7bac15700 1 -- 172.21.15.40:0/3473741042 >> 172.21.15.40:0/3473741042 conn(0x7fc7b40cabc0 msgr2=0x7fc7b40cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:19.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.472+0000 7fc7bac15700 1 -- 172.21.15.40:0/3473741042 shutdown_connections 2024-09-13T18:16:19.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.472+0000 7fc7bac15700 1 -- 172.21.15.40:0/3473741042 wait complete. 2024-09-13T18:16:19.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.476+0000 7fc7bac15700 1 Processor -- start 2024-09-13T18:16:19.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.476+0000 7fc7bac15700 1 -- start start 2024-09-13T18:16:19.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.476+0000 7fc7bac15700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7b4070500 0x7fc7b413cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:19.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.476+0000 7fc7bac15700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7b4071b40 0x7fc7b414ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:19.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.476+0000 7fc7bac15700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7b40c55c0 0x7fc7b414f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:19.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.476+0000 7fc7bac15700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b413dd20 con 0x7fc7b4070500 2024-09-13T18:16:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.476+0000 7fc7bac15700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b414a360 con 0x7fc7b4071b40 2024-09-13T18:16:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.476+0000 7fc7bac15700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b41326a0 con 0x7fc7b40c55c0 2024-09-13T18:16:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.476+0000 7fc7b9412700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7b4071b40 0x7fc7b414ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.476+0000 7fc7b9412700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7b4071b40 0x7fc7b414ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:40314/0 (socket says 172.21.15.40:40314) 2024-09-13T18:16:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.476+0000 7fc7b9412700 1 -- 172.21.15.40:0/4189444047 learned_addr learned my addr 172.21.15.40:0/4189444047 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:16:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.476+0000 7fc7b9412700 1 -- 172.21.15.40:0/4189444047 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7b40c55c0 msgr2=0x7fc7b414f4b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:16:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.476+0000 7fc7b9412700 1 --2- 172.21.15.40:0/4189444047 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7b40c55c0 0x7fc7b414f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.476+0000 7fc7b9412700 1 -- 172.21.15.40:0/4189444047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7b4070500 msgr2=0x7fc7b413cd40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:16:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.476+0000 7fc7b9412700 1 --2- 172.21.15.40:0/4189444047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7b4070500 0x7fc7b413cd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.476+0000 7fc7b9412700 1 -- 172.21.15.40:0/4189444047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b00094b0 con 0x7fc7b4071b40 2024-09-13T18:16:19.493 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.476+0000 7fc7b9412700 1 --2- 172.21.15.40:0/4189444047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7b4071b40 0x7fc7b414ef70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00090b0 tx=0x7fc7b0004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:19.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.488+0000 7fc7aa7fc700 1 -- 172.21.15.40:0/4189444047 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0011a40 con 0x7fc7b4071b40 2024-09-13T18:16:19.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.488+0000 7fc7aa7fc700 1 -- 172.21.15.40:0/4189444047 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b001e070 con 0x7fc7b4071b40 2024-09-13T18:16:19.495 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.488+0000 7fc7bac15700 1 -- 172.21.15.40:0/4189444047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b413c5d0 con 0x7fc7b4071b40 2024-09-13T18:16:19.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.488+0000 7fc7bac15700 1 -- 172.21.15.40:0/4189444047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b414a570 con 0x7fc7b4071b40 2024-09-13T18:16:19.496 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.488+0000 7fc7aa7fc700 1 -- 172.21.15.40:0/4189444047 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0011de0 con 0x7fc7b4071b40 2024-09-13T18:16:19.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.488+0000 7fc7aa7fc700 1 -- 172.21.15.40:0/4189444047 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc7b001a430 con 0x7fc7b4071b40 2024-09-13T18:16:19.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.496+0000 7fc7aa7fc700 1 --2- 172.21.15.40:0/4189444047 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc798066800 0x7fc798068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:19.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.496+0000 7fc7aa7fc700 1 -- 172.21.15.40:0/4189444047 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc7b0085470 con 0x7fc7b4071b40 2024-09-13T18:16:19.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.496+0000 7fc7b9c13700 1 --2- 172.21.15.40:0/4189444047 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc798066800 0x7fc798068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:19.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.496+0000 7fc7b9c13700 1 --2- 172.21.15.40:0/4189444047 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc798066800 0x7fc798068cc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac005aa0 tx=0x7fc7ac004ff0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:19.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.504+0000 7fc7bac15700 1 -- 172.21.15.40:0/4189444047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b40c80c0 con 0x7fc7b4071b40 2024-09-13T18:16:19.522 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:19.512+0000 7fc7aa7fc700 1 -- 172.21.15.40:0/4189444047 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc7b0052400 con 0x7fc7b4071b40 2024-09-13T18:16:20.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.224+0000 7fc7bac15700 1 -- 172.21.15.40:0/4189444047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc7b40c80c0 con 0x7fc7b4071b40 2024-09-13T18:16:20.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.224+0000 7fc7aa7fc700 1 -- 172.21.15.40:0/4189444047 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fc7b0020020 con 0x7fc7b4071b40 2024-09-13T18:16:20.233 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":210,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:16:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.228+0000 7fc7bac15700 1 -- 172.21.15.40:0/4189444047 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc798066800 msgr2=0x7fc798068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.228+0000 7fc7bac15700 1 --2- 172.21.15.40:0/4189444047 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc798066800 0x7fc798068cc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac005aa0 tx=0x7fc7ac004ff0 comp rx=0 tx=0).stop 2024-09-13T18:16:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.228+0000 7fc7bac15700 1 -- 172.21.15.40:0/4189444047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7b4071b40 msgr2=0x7fc7b414ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.228+0000 7fc7bac15700 1 --2- 172.21.15.40:0/4189444047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7b4071b40 0x7fc7b414ef70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00090b0 tx=0x7fc7b0004770 comp rx=0 tx=0).stop 2024-09-13T18:16:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.228+0000 7fc7bac15700 1 -- 172.21.15.40:0/4189444047 shutdown_connections 2024-09-13T18:16:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.228+0000 7fc7bac15700 1 --2- 172.21.15.40:0/4189444047 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc798066800 0x7fc798068cc0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.228+0000 7fc7bac15700 1 --2- 172.21.15.40:0/4189444047 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7b40c55c0 0x7fc7b414f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.228+0000 7fc7bac15700 1 --2- 172.21.15.40:0/4189444047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7b4071b40 0x7fc7b414ef70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.228+0000 7fc7bac15700 1 --2- 172.21.15.40:0/4189444047 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7b4070500 0x7fc7b413cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.228+0000 7fc7bac15700 1 -- 172.21.15.40:0/4189444047 >> 172.21.15.40:0/4189444047 conn(0x7fc7b40cabc0 msgr2=0x7fc7b41500c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.228+0000 7fc7bac15700 1 -- 172.21.15.40:0/4189444047 shutdown_connections 2024-09-13T18:16:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.232+0000 7fc7bac15700 1 -- 172.21.15.40:0/4189444047 wait complete. 2024-09-13T18:16:20.277 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:16:20.277 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:16:20.277 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:16:20.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.544+0000 7f7981fea700 1 Processor -- start 2024-09-13T18:16:20.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.544+0000 7f7981fea700 1 -- start start 2024-09-13T18:16:20.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.544+0000 7f7981fea700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f797c14f8c0 0x7f797c0c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:20.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.544+0000 7f7981fea700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797c0c6890 con 0x7f797c158bc0 2024-09-13T18:16:20.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.544+0000 7f7981fea700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797c0c5f30 con 0x7f797c0724e0 2024-09-13T18:16:20.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.544+0000 7f7981fea700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797c0c7700 con 0x7f797c14f8c0 2024-09-13T18:16:20.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.544+0000 7f79817e9700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f797c158bc0 0x7f797c15ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53426/0 (socket says 172.21.15.40:53426) 2024-09-13T18:16:20.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.544+0000 7f79817e9700 1 -- 172.21.15.40:0/3244807694 learned_addr learned my addr 172.21.15.40:0/3244807694 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:16:20.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.544+0000 7f7980fe8700 1 --2- 172.21.15.40:0/3244807694 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f797c14f8c0 0x7f797c0c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:20.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.544+0000 7f7980fe8700 1 -- 172.21.15.40:0/3244807694 >> v1:172.21.15.43:6789/0 conn(0x7f797c0724e0 legacy=0x7f797c06d2d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:16:20.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.544+0000 7f7980fe8700 1 -- 172.21.15.40:0/3244807694 >> v1:172.21.15.40:6789/0 conn(0x7f797c158bc0 legacy=0x7f797c15ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:20.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.544+0000 7f797b7fe700 1 -- 172.21.15.40:0/3244807694 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1189977675 0 0) 0x7f797c0c6890 con 0x7f797c158bc0 2024-09-13T18:16:20.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.544+0000 7f7980fe8700 1 -- 172.21.15.40:0/3244807694 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c0c1320 con 0x7f797c14f8c0 2024-09-13T18:16:20.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.544+0000 7f7980fe8700 1 --2- 172.21.15.40:0/3244807694 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f797c14f8c0 0x7f797c0c71c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f797000bb30 tx=0x7f797000be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8e63a148247f18 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:20.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.556+0000 7f797b7fe700 1 -- 172.21.15.40:0/3244807694 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797000dce0 con 0x7f797c14f8c0 2024-09-13T18:16:20.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.556+0000 7f797b7fe700 1 -- 172.21.15.40:0/3244807694 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797000de40 con 0x7f797c14f8c0 2024-09-13T18:16:20.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.556+0000 7f797b7fe700 1 -- 172.21.15.40:0/3244807694 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970004270 con 0x7f797c14f8c0 2024-09-13T18:16:20.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.556+0000 7f7981fea700 1 -- 172.21.15.40:0/3244807694 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f797c14f8c0 msgr2=0x7f797c0c71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:20.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.556+0000 7f7981fea700 1 --2- 172.21.15.40:0/3244807694 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f797c14f8c0 0x7f797c0c71c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f797000bb30 tx=0x7f797000be40 comp rx=0 tx=0).stop 2024-09-13T18:16:20.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.556+0000 7f7981fea700 1 -- 172.21.15.40:0/3244807694 shutdown_connections 2024-09-13T18:16:20.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.556+0000 7f7981fea700 1 --2- 172.21.15.40:0/3244807694 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f797c14f8c0 0x7f797c0c71c0 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f797000bb30 tx=0x7f797000be40 comp rx=0 tx=0).stop 2024-09-13T18:16:20.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.556+0000 7f7981fea700 1 -- 172.21.15.40:0/3244807694 >> 172.21.15.40:0/3244807694 conn(0x7f797c066090 msgr2=0x7f797c0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:20.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.556+0000 7f7981fea700 1 -- 172.21.15.40:0/3244807694 shutdown_connections 2024-09-13T18:16:20.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.556+0000 7f7981fea700 1 -- 172.21.15.40:0/3244807694 wait complete. 2024-09-13T18:16:20.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.560+0000 7f7981fea700 1 Processor -- start 2024-09-13T18:16:20.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.560+0000 7f7981fea700 1 -- start start 2024-09-13T18:16:20.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f7981fea700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f797c0724e0 0x7f797c1fcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:20.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f7981fea700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f797c158bc0 0x7f797c1fd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:20.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f7981fea700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f797c154a40 0x7f797c153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:20.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f7981fea700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c14ca30 con 0x7f797c0724e0 2024-09-13T18:16:20.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f7981fea700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c1fc7c0 con 0x7f797c158bc0 2024-09-13T18:16:20.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f7981fea700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797c1fc490 con 0x7f797c154a40 2024-09-13T18:16:20.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f7980fe8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f797c0724e0 0x7f797c1fcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:20.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f7980fe8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f797c0724e0 0x7f797c1fcfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36196/0 (socket says 172.21.15.40:36196) 2024-09-13T18:16:20.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f7980fe8700 1 -- 172.21.15.40:0/3248944962 learned_addr learned my addr 172.21.15.40:0/3248944962 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:16:20.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f797bfff700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f797c158bc0 0x7f797c1fd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:20.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f79817e9700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f797c154a40 0x7f797c153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:20.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f79817e9700 1 -- 172.21.15.40:0/3248944962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f797c158bc0 msgr2=0x7f797c1fd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:20.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f79817e9700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f797c158bc0 0x7f797c1fd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:20.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f79817e9700 1 -- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f797c0724e0 msgr2=0x7f797c1fcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:20.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f79817e9700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f797c0724e0 0x7f797c1fcfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:20.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f79817e9700 1 -- 172.21.15.40:0/3248944962 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797000b760 con 0x7f797c154a40 2024-09-13T18:16:20.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f7980fe8700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f797c0724e0 0x7f797c1fcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:16:20.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f79817e9700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f797c154a40 0x7f797c153480 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f796c00b440 tx=0x7f796c00b750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:20.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f79797fa700 1 -- 172.21.15.40:0/3248944962 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c006070 con 0x7f797c154a40 2024-09-13T18:16:20.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f79797fa700 1 -- 172.21.15.40:0/3248944962 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796c0061d0 con 0x7f797c154a40 2024-09-13T18:16:20.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f79797fa700 1 -- 172.21.15.40:0/3248944962 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796c006580 con 0x7f797c154a40 2024-09-13T18:16:20.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f7981fea700 1 -- 172.21.15.40:0/3248944962 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797c158620 con 0x7f797c154a40 2024-09-13T18:16:20.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.564+0000 7f7981fea700 1 -- 172.21.15.40:0/3248944962 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797c070d20 con 0x7f797c154a40 2024-09-13T18:16:20.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.572+0000 7f79797fa700 1 -- 172.21.15.40:0/3248944962 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f796c0066e0 con 0x7f797c154a40 2024-09-13T18:16:20.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.572+0000 7f79797fa700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7968066800 0x7f7968068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:20.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.572+0000 7f7980fe8700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7968066800 0x7f7968068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:20.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.572+0000 7f7981fea700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795c002a40 0x7f795c004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:20.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.572+0000 7f79797fa700 1 -- 172.21.15.40:0/3248944962 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f796c083700 con 0x7f797c154a40 2024-09-13T18:16:20.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.572+0000 7f7981fea700 1 -- 172.21.15.40:0/3248944962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c007f40 con 0x7f795c002a40 2024-09-13T18:16:20.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.572+0000 7f7980fe8700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7968066800 0x7f7968068cc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f797000be80 tx=0x7f7970012000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:20.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.576+0000 7f797bfff700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795c002a40 0x7f795c004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:20.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.580+0000 7f797bfff700 1 -- 172.21.15.40:0/3248944962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f795c008100 con 0x7f795c002a40 2024-09-13T18:16:20.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.580+0000 7f797bfff700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795c002a40 0x7f795c004f00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f797c06db90 tx=0x7f797400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:20.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.584+0000 7f79797fa700 1 -- 172.21.15.40:0/3248944962 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797400b040 con 0x7f795c002a40 2024-09-13T18:16:20.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.584+0000 7f79797fa700 1 -- 172.21.15.40:0/3248944962 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f797400aa80 con 0x7f795c002a40 2024-09-13T18:16:20.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.584+0000 7f79797fa700 1 -- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795c002a40 msgr2=0x7f795c004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:20.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.596+0000 7f79797fa700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795c002a40 0x7f795c004f00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f797c06db90 tx=0x7f797400cde0 comp rx=0 tx=0).stop 2024-09-13T18:16:20.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.596+0000 7f79797fa700 1 -- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795c002a40 msgr2=0x7f795c004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:16:20.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.596+0000 7f79797fa700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795c002a40 0x7f795c004f00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:20.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.596+0000 7f7981fea700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795c006fc0 0x7f795c0073f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:20.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.596+0000 7f7981fea700 1 -- 172.21.15.40:0/3248944962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c00a4a0 con 0x7f795c006fc0 2024-09-13T18:16:20.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.600+0000 7f797bfff700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795c006fc0 0x7f795c0073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:20.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.600+0000 7f797bfff700 1 -- 172.21.15.40:0/3248944962 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f795c007930 con 0x7f795c006fc0 2024-09-13T18:16:20.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.600+0000 7f797bfff700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795c006fc0 0x7f795c0073f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f797c14ba60 tx=0x7f7974004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:20.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.600+0000 7f79797fa700 1 -- 172.21.15.40:0/3248944962 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974028070 con 0x7f795c006fc0 2024-09-13T18:16:20.609 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:16:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:16:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:16:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:16:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 210, 2024-09-13T18:16:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:16:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:16:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:16:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:16:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:16:20.609 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:16:20.610 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:16:20.610 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:16:20.610 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:20.610 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:20.610 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:20.610 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:20.610 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:20.610 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:20.610 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:20.610 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:20.610 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:20.610 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:20.610 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:20.611 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:16:20.611 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:16:20.611 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:20.611 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:20.611 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:20.611 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:20.611 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:20.611 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:20.611 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:20.611 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:20.611 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:20.611 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:20.611 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:20.612 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.612 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:16:20.612 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:16:20.612 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:16:20.612 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:16:20.612 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:16:20.612 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:16:20.612 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:16:20.612 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:16:20.613 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:16:20.613 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:16:20.613 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:16:20.613 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:16:20.613 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:16:20.613 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:16:20.613 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:16:20.613 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:16:20.613 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:16:20.613 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:20.613 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:20.613 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:20.613 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:20.613 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:20.614 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:20.614 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:20.614 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:20.614 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:20.614 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:20.614 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:20.614 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:16:20.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.614 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:16:20.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.614 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:16:20.614 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:16:20.614 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:20.614 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:20.615 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.615 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:20.615 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:16:20.615 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:20.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.615 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.615 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:20.615 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:16:20.615 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:20.615 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:20.615 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:20.615 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.615 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:16:20.616 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:16:20.616 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:20.616 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:20.616 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:20.616 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.616 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:16:20.616 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:16:20.616 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:20.616 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:20.616 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.616 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:20.616 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:16:20.617 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:20.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.617 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.617 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:20.617 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:16:20.617 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:20.617 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:20.617 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:20.617 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.617 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:16:20.617 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:16:20.617 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:20.617 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:20.617 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:20.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.618 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:16:20.618 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:16:20.618 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:20.618 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:20.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.618 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:20.618 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:16:20.618 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:20.618 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.618 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.618 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:20.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:16:20.619 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:20.619 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:20.619 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:20.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.619 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:16:20.619 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:16:20.619 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:20.619 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:20.619 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:20.619 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:20.619 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:20.619 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.619 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:16:20.620 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:16:20.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.620 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:20.620 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:20.620 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:16:20.620 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:20.620 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:20.620 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:16:20.620 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.620 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:20.620 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:20.620 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:16:20.620 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:20.620 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:20.621 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.621 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:16:20.621 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:16:20.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.600+0000 7f79797fa700 1 -- 172.21.15.40:0/3248944962 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4452 (secure 0 0 0) 0x7f7974020040 con 0x7f795c006fc0 2024-09-13T18:16:20.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.600+0000 7f79797fa700 1 -- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795c006fc0 msgr2=0x7f795c0073f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:20.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.600+0000 7f79797fa700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795c006fc0 0x7f795c0073f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f797c14ba60 tx=0x7f7974004ba0 comp rx=0 tx=0).stop 2024-09-13T18:16:20.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.600+0000 7f79797fa700 1 -- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795c006fc0 msgr2=0x7f795c0073f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:16:20.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.600+0000 7f79797fa700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f795c006fc0 0x7f795c0073f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:20.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.600+0000 7f7981fea700 1 -- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7968066800 msgr2=0x7f7968068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:20.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.600+0000 7f7981fea700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7968066800 0x7f7968068cc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f797000be80 tx=0x7f7970012000 comp rx=0 tx=0).stop 2024-09-13T18:16:20.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.600+0000 7f79817e9700 1 -- 172.21.15.40:0/3248944962 reap_dead start 2024-09-13T18:16:20.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.600+0000 7f7981fea700 1 -- 172.21.15.40:0/3248944962 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f797c154a40 msgr2=0x7f797c153480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:20.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.600+0000 7f7981fea700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f797c154a40 0x7f797c153480 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f796c00b440 tx=0x7f796c00b750 comp rx=0 tx=0).stop 2024-09-13T18:16:20.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.604+0000 7f7981fea700 1 -- 172.21.15.40:0/3248944962 shutdown_connections 2024-09-13T18:16:20.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.604+0000 7f7981fea700 1 --2- 172.21.15.40:0/3248944962 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f797c154a40 0x7f797c153480 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:20.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.604+0000 7f7981fea700 1 -- 172.21.15.40:0/3248944962 >> 172.21.15.40:0/3248944962 conn(0x7f797c066090 msgr2=0x7f797c070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:20.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.604+0000 7f7981fea700 1 -- 172.21.15.40:0/3248944962 shutdown_connections 2024-09-13T18:16:20.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.604+0000 7f7981fea700 1 -- 172.21.15.40:0/3248944962 wait complete. 2024-09-13T18:16:20.639 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:16:20.639 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:16:20.639 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:16:20.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.844+0000 7fef4e94f700 1 Processor -- start 2024-09-13T18:16:20.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.844+0000 7fef4e94f700 1 -- start start 2024-09-13T18:16:20.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.844+0000 7fef4e94f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fef48072270 0x7fef48159690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:20.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.844+0000 7fef4e94f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fef480c6390 0x7fef48163f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:20.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.844+0000 7fef4e94f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef48072650 con 0x7fef480c7320 2024-09-13T18:16:20.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.844+0000 7fef4e94f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef480c5f00 con 0x7fef480c6390 2024-09-13T18:16:20.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.844+0000 7fef4e94f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef480c7130 con 0x7fef48072270 2024-09-13T18:16:20.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.844+0000 7fef4ceec700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fef480c6390 0x7fef48163f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:20.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.844+0000 7fef4ceec700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fef480c6390 0x7fef48163f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:40344/0 (socket says 172.21.15.40:40344) 2024-09-13T18:16:20.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.844+0000 7fef4ceec700 1 -- 172.21.15.40:0/593209334 learned_addr learned my addr 172.21.15.40:0/593209334 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:16:20.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.852+0000 7fef47fff700 1 --2- 172.21.15.40:0/593209334 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fef48072270 0x7fef48159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:20.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.852+0000 7fef4ceec700 1 -- 172.21.15.40:0/593209334 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fef48072270 msgr2=0x7fef48159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:16:20.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.852+0000 7fef4ceec700 1 --2- 172.21.15.40:0/593209334 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fef48072270 0x7fef48159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:20.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.852+0000 7fef4ceec700 1 -- 172.21.15.40:0/593209334 >> v1:172.21.15.43:6789/0 conn(0x7fef480c7320 legacy=0x7fef4806d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:20.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.852+0000 7fef4ceec700 1 -- 172.21.15.40:0/593209334 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef480c8450 con 0x7fef480c6390 2024-09-13T18:16:20.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.852+0000 7fef4ceec700 1 --2- 172.21.15.40:0/593209334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef480c6390 0x7fef48163f30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fef38009910 tx=0x7fef38009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14d0daff6470635c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:20.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.852+0000 7fef46ffd700 1 -- 172.21.15.40:0/593209334 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef38004230 con 0x7fef480c6390 2024-09-13T18:16:20.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.852+0000 7fef46ffd700 1 -- 172.21.15.40:0/593209334 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef38004390 con 0x7fef480c6390 2024-09-13T18:16:20.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.852+0000 7fef46ffd700 1 -- 172.21.15.40:0/593209334 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef38009e60 con 0x7fef480c6390 2024-09-13T18:16:20.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.856+0000 7fef4e94f700 1 -- 172.21.15.40:0/593209334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef480c6390 msgr2=0x7fef48163f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:20.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.856+0000 7fef4e94f700 1 --2- 172.21.15.40:0/593209334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef480c6390 0x7fef48163f30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fef38009910 tx=0x7fef38009c20 comp rx=0 tx=0).stop 2024-09-13T18:16:20.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.856+0000 7fef4e94f700 1 -- 172.21.15.40:0/593209334 shutdown_connections 2024-09-13T18:16:20.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.856+0000 7fef4e94f700 1 --2- 172.21.15.40:0/593209334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef480c6390 0x7fef48163f30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:20.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.856+0000 7fef4e94f700 1 --2- 172.21.15.40:0/593209334 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fef48072270 0x7fef48159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:20.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.856+0000 7fef4e94f700 1 -- 172.21.15.40:0/593209334 >> 172.21.15.40:0/593209334 conn(0x7fef48066090 msgr2=0x7fef480664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:20.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.856+0000 7fef4e94f700 1 -- 172.21.15.40:0/593209334 shutdown_connections 2024-09-13T18:16:20.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.856+0000 7fef4e94f700 1 -- 172.21.15.40:0/593209334 wait complete. 2024-09-13T18:16:20.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.856+0000 7fef4e94f700 1 Processor -- start 2024-09-13T18:16:20.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.856+0000 7fef4e94f700 1 -- start start 2024-09-13T18:16:20.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef4e94f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fef48072270 0x7fef48150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:20.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef4e94f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef480c6390 0x7fef480cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:20.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef4e94f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef480c7320 0x7fef480c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:20.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef4e94f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef481513d0 con 0x7fef480c6390 2024-09-13T18:16:20.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef4e94f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef48151060 con 0x7fef480c7320 2024-09-13T18:16:20.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef4e94f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef481ef390 con 0x7fef48072270 2024-09-13T18:16:20.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef477fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef480c6390 0x7fef480cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:20.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef477fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef480c6390 0x7fef480cb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:36212/0 (socket says 172.21.15.40:36212) 2024-09-13T18:16:20.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef477fe700 1 -- 172.21.15.40:0/3510088942 learned_addr learned my addr 172.21.15.40:0/3510088942 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:16:20.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef47fff700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fef48072270 0x7fef48150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:20.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef4ceec700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef480c7320 0x7fef480c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:20.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef477fe700 1 -- 172.21.15.40:0/3510088942 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fef48072270 msgr2=0x7fef48150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:20.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef477fe700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fef48072270 0x7fef48150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:20.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef477fe700 1 -- 172.21.15.40:0/3510088942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef480c7320 msgr2=0x7fef480c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:20.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef477fe700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef480c7320 0x7fef480c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:20.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef477fe700 1 -- 172.21.15.40:0/3510088942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef38009570 con 0x7fef480c6390 2024-09-13T18:16:20.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef477fe700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef480c6390 0x7fef480cb3b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fef40002090 tx=0x7fef4000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:20.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef44ff9700 1 -- 172.21.15.40:0/3510088942 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef40003e50 con 0x7fef480c6390 2024-09-13T18:16:20.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef44ff9700 1 -- 172.21.15.40:0/3510088942 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4000a9f0 con 0x7fef480c6390 2024-09-13T18:16:20.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef4e94f700 1 -- 172.21.15.40:0/3510088942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef481ef650 con 0x7fef480c6390 2024-09-13T18:16:20.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.860+0000 7fef4e94f700 1 -- 172.21.15.40:0/3510088942 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef481efc10 con 0x7fef480c6390 2024-09-13T18:16:20.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.864+0000 7fef44ff9700 1 -- 172.21.15.40:0/3510088942 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4000ad80 con 0x7fef480c6390 2024-09-13T18:16:20.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.864+0000 7fef44ff9700 1 -- 172.21.15.40:0/3510088942 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fef40005070 con 0x7fef480c6390 2024-09-13T18:16:20.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.868+0000 7fef44ff9700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fef340667b0 0x7fef34068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:20.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.868+0000 7fef44ff9700 1 -- 172.21.15.40:0/3510088942 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fef400843c0 con 0x7fef480c6390 2024-09-13T18:16:20.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.868+0000 7fef47fff700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fef340667b0 0x7fef34068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:20.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.868+0000 7fef47fff700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fef340667b0 0x7fef34068c70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00ba30 tx=0x7fef3c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:20.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.872+0000 7fef4e94f700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef28002a40 0x7fef28004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:20.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.872+0000 7fef4e94f700 1 -- 172.21.15.40:0/3510088942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef28007f40 con 0x7fef28002a40 2024-09-13T18:16:20.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.876+0000 7fef4ceec700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef28002a40 0x7fef28004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:20.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.876+0000 7fef4ceec700 1 -- 172.21.15.40:0/3510088942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef28008100 con 0x7fef28002a40 2024-09-13T18:16:20.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.876+0000 7fef4ceec700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef28002a40 0x7fef28004f00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fef38004700 tx=0x7fef38004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:20.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.880+0000 7fef44ff9700 1 -- 172.21.15.40:0/3510088942 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3800bd00 con 0x7fef28002a40 2024-09-13T18:16:20.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.880+0000 7fef44ff9700 1 -- 172.21.15.40:0/3510088942 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fef3800be60 con 0x7fef28002a40 2024-09-13T18:16:20.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.880+0000 7fef44ff9700 1 -- 172.21.15.40:0/3510088942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef28002a40 msgr2=0x7fef28004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:20.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.880+0000 7fef44ff9700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef28002a40 0x7fef28004f00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fef38004700 tx=0x7fef38004770 comp rx=0 tx=0).stop 2024-09-13T18:16:20.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.880+0000 7fef44ff9700 1 -- 172.21.15.40:0/3510088942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef28002a40 msgr2=0x7fef28004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:16:20.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.880+0000 7fef44ff9700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef28002a40 0x7fef28004f00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:20.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.924+0000 7fef4e94f700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef28007020 0x7fef280074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:20.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.924+0000 7fef4e94f700 1 -- 172.21.15.40:0/3510088942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2800a400 con 0x7fef28007020 2024-09-13T18:16:20.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.924+0000 7fef4ceec700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef28007020 0x7fef280074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:20.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.924+0000 7fef4ceec700 1 -- 172.21.15.40:0/3510088942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fef2800a5c0 con 0x7fef28007020 2024-09-13T18:16:20.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.924+0000 7fef4ceec700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef28007020 0x7fef280074b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fef38009c60 tx=0x7fef38011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:20.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.924+0000 7fef44ff9700 1 -- 172.21.15.40:0/3510088942 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3802c8f0 con 0x7fef28007020 2024-09-13T18:16:20.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.928+0000 7fef44ff9700 1 -- 172.21.15.40:0/3510088942 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4450 (secure 0 0 0) 0x7fef38030070 con 0x7fef28007020 2024-09-13T18:16:20.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.928+0000 7fef44ff9700 1 -- 172.21.15.40:0/3510088942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef28007020 msgr2=0x7fef280074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:20.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.928+0000 7fef44ff9700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef28007020 0x7fef280074b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fef38009c60 tx=0x7fef38011000 comp rx=0 tx=0).stop 2024-09-13T18:16:20.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.928+0000 7fef44ff9700 1 -- 172.21.15.40:0/3510088942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef28007020 msgr2=0x7fef280074b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:16:20.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.928+0000 7fef44ff9700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef28007020 0x7fef280074b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:20.935 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:16:20.935 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:16:20.935 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:16:20.936 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:16:20.936 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 210, 2024-09-13T18:16:20.936 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:16:20.936 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:16:20.936 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:16:20.936 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:16:20.936 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:20.936 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:16:20.936 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:16:20.936 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:16:20.937 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:16:20.937 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:20.937 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:20.937 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:20.937 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:20.937 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:20.937 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:20.937 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:20.937 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:20.937 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:20.938 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:20.938 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:20.938 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:16:20.938 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:16:20.938 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:20.938 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:20.938 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:20.938 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:20.938 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:20.938 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:20.938 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:20.938 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:20.938 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:20.938 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:20.938 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:20.939 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.939 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:16:20.939 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:16:20.939 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:16:20.939 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:16:20.939 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:16:20.939 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:16:20.939 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:16:20.939 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:16:20.939 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:16:20.939 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:16:20.939 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:16:20.939 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:16:20.939 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:16:20.940 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:16:20.940 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:16:20.940 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:16:20.940 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:16:20.940 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:20.940 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:20.940 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:20.940 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:20.940 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:20.940 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:20.940 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:20.941 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:20.941 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:20.941 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:20.941 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:20.941 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:16:20.941 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.941 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:16:20.941 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.941 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:16:20.941 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:16:20.941 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:20.941 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:20.942 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.942 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:20.942 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:16:20.942 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:20.942 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.942 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.942 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:20.942 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:16:20.942 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:20.942 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:20.942 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:20.942 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.943 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:16:20.943 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:16:20.943 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:20.943 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:20.943 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:20.943 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.943 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.943 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:16:20.943 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:16:20.943 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:20.943 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:20.943 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.944 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:20.944 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:16:20.944 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:20.944 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.944 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.944 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:20.944 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:16:20.944 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:20.944 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:20.944 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:20.944 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.944 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:16:20.944 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:16:20.945 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:20.945 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:20.945 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:20.945 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.945 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.945 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:16:20.945 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:16:20.945 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:20.945 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:20.945 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.945 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:20.946 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:16:20.946 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:20.946 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.946 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.946 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:20.946 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:16:20.946 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:20.946 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:20.946 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:20.946 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.946 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:16:20.946 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:16:20.946 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:20.947 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:20.947 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:20.947 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:20.947 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:20.947 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.947 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:16:20.947 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:16:20.947 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.948 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:20.948 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:20.948 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:16:20.948 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:20.948 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:20.948 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:16:20.948 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:20.948 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:20.948 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:20.948 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:16:20.948 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:20.948 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:20.949 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:20.949 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:16:20.949 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:16:20.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.928+0000 7fef2e7fc700 1 -- 172.21.15.40:0/3510088942 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fef340667b0 msgr2=0x7fef34068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:20.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.928+0000 7fef2e7fc700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fef340667b0 0x7fef34068c70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00ba30 tx=0x7fef3c008040 comp rx=0 tx=0).stop 2024-09-13T18:16:20.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.928+0000 7fef4ceec700 1 -- 172.21.15.40:0/3510088942 reap_dead start 2024-09-13T18:16:20.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.928+0000 7fef2e7fc700 1 -- 172.21.15.40:0/3510088942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef480c6390 msgr2=0x7fef480cb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:20.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.928+0000 7fef2e7fc700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef480c6390 0x7fef480cb3b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fef40002090 tx=0x7fef4000a340 comp rx=0 tx=0).stop 2024-09-13T18:16:20.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.928+0000 7fef2e7fc700 1 -- 172.21.15.40:0/3510088942 shutdown_connections 2024-09-13T18:16:20.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.928+0000 7fef2e7fc700 1 --2- 172.21.15.40:0/3510088942 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fef480c6390 0x7fef480cb3b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:20.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.928+0000 7fef2e7fc700 1 -- 172.21.15.40:0/3510088942 >> 172.21.15.40:0/3510088942 conn(0x7fef48066090 msgr2=0x7fef4806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:20.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.928+0000 7fef2e7fc700 1 -- 172.21.15.40:0/3510088942 shutdown_connections 2024-09-13T18:16:20.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:20.928+0000 7fef2e7fc700 1 -- 172.21.15.40:0/3510088942 wait complete. 2024-09-13T18:16:20.978 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:16:20.978 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:16:20.979 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:16:21.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.308+0000 7ffac084e700 1 Processor -- start 2024-09-13T18:16:21.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.308+0000 7ffac084e700 1 -- start start 2024-09-13T18:16:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.312+0000 7ffac084e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffabc0c78d0 0x7ffabc0c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.312+0000 7ffac084e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffabc158bc0 0x7ffabc06d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.312+0000 7ffac084e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffabc14fd60 con 0x7ffabc0724e0 2024-09-13T18:16:21.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.312+0000 7ffac084e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffabc0c7700 con 0x7ffabc158bc0 2024-09-13T18:16:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.312+0000 7ffac084e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc0c6890 con 0x7ffabc0c78d0 2024-09-13T18:16:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.312+0000 7ffabad9d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffabc158bc0 0x7ffabc06d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.312+0000 7ffabad9d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffabc158bc0 0x7ffabc06d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:40370/0 (socket says 172.21.15.40:40370) 2024-09-13T18:16:21.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.312+0000 7ffabad9d700 1 -- 172.21.15.40:0/3729239506 learned_addr learned my addr 172.21.15.40:0/3729239506 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:16:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.312+0000 7ffabad9d700 1 -- 172.21.15.40:0/3729239506 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffabc0c78d0 msgr2=0x7ffabc0c71c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:16:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.312+0000 7ffabad9d700 1 --2- 172.21.15.40:0/3729239506 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffabc0c78d0 0x7ffabc0c71c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:21.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.312+0000 7ffabad9d700 1 -- 172.21.15.40:0/3729239506 >> v1:172.21.15.43:6789/0 conn(0x7ffabc0724e0 legacy=0x7ffabc14f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:16:21.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.316+0000 7ffabad9d700 1 -- 172.21.15.40:0/3729239506 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc0c1320 con 0x7ffabc158bc0 2024-09-13T18:16:21.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.320+0000 7ffabad9d700 1 --2- 172.21.15.40:0/3729239506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc158bc0 0x7ffabc06d2d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009e50 tx=0x7ffab400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae3febed94377b14 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.324+0000 7ffab959a700 1 -- 172.21.15.40:0/3729239506 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab400e070 con 0x7ffabc158bc0 2024-09-13T18:16:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.324+0000 7ffab959a700 1 -- 172.21.15.40:0/3729239506 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab4003a40 con 0x7ffabc158bc0 2024-09-13T18:16:21.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.324+0000 7ffab959a700 1 -- 172.21.15.40:0/3729239506 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4003df0 con 0x7ffabc158bc0 2024-09-13T18:16:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.324+0000 7ffac084e700 1 -- 172.21.15.40:0/3729239506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc158bc0 msgr2=0x7ffabc06d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:21.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.324+0000 7ffac084e700 1 --2- 172.21.15.40:0/3729239506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc158bc0 0x7ffabc06d2d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009e50 tx=0x7ffab400b330 comp rx=0 tx=0).stop 2024-09-13T18:16:21.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.324+0000 7ffac084e700 1 -- 172.21.15.40:0/3729239506 shutdown_connections 2024-09-13T18:16:21.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.324+0000 7ffac084e700 1 --2- 172.21.15.40:0/3729239506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc158bc0 0x7ffabc06d2d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:21.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.324+0000 7ffac084e700 1 --2- 172.21.15.40:0/3729239506 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffabc0c78d0 0x7ffabc0c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:21.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.324+0000 7ffac084e700 1 -- 172.21.15.40:0/3729239506 >> 172.21.15.40:0/3729239506 conn(0x7ffabc066090 msgr2=0x7ffabc0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:21.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.328+0000 7ffac084e700 1 -- 172.21.15.40:0/3729239506 shutdown_connections 2024-09-13T18:16:21.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.328+0000 7ffac084e700 1 -- 172.21.15.40:0/3729239506 wait complete. 2024-09-13T18:16:21.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.332+0000 7ffac084e700 1 Processor -- start 2024-09-13T18:16:21.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.332+0000 7ffac084e700 1 -- start start 2024-09-13T18:16:21.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.332+0000 7ffac084e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffabc0724e0 0x7ffabc0c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:21.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.332+0000 7ffac084e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc0c78d0 0x7ffabc0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:21.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.332+0000 7ffac084e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffabc158bc0 0x7ffabc1550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:21.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.332+0000 7ffac084e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc0cbb70 con 0x7ffabc158bc0 2024-09-13T18:16:21.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.332+0000 7ffac084e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc167180 con 0x7ffabc0c78d0 2024-09-13T18:16:21.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.332+0000 7ffac084e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc0c9240 con 0x7ffabc0724e0 2024-09-13T18:16:21.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.332+0000 7ffab9d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc0c78d0 0x7ffabc0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:21.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.332+0000 7ffab9d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc0c78d0 0x7ffabc0cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:40376/0 (socket says 172.21.15.40:40376) 2024-09-13T18:16:21.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.332+0000 7ffab9d9b700 1 -- 172.21.15.40:0/299566925 learned_addr learned my addr 172.21.15.40:0/299566925 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:16:21.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.332+0000 7ffaba59c700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffabc0724e0 0x7ffabc0c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:21.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.332+0000 7ffabad9d700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffabc158bc0 0x7ffabc1550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:21.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.332+0000 7ffab9d9b700 1 -- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffabc0724e0 msgr2=0x7ffabc0c21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:21.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.332+0000 7ffab9d9b700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffabc0724e0 0x7ffabc0c21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:21.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.332+0000 7ffab9d9b700 1 -- 172.21.15.40:0/299566925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffabc158bc0 msgr2=0x7ffabc1550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:21.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.332+0000 7ffab9d9b700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffabc158bc0 0x7ffabc1550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:21.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.332+0000 7ffab9d9b700 1 -- 172.21.15.40:0/299566925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab4009b30 con 0x7ffabc0c78d0 2024-09-13T18:16:21.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.332+0000 7ffab9d9b700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc0c78d0 0x7ffabc0cb630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffab000b450 tx=0x7ffab000b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:21.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.336+0000 7ffaaaffd700 1 -- 172.21.15.40:0/299566925 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab0004020 con 0x7ffabc0c78d0 2024-09-13T18:16:21.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.340+0000 7ffac084e700 1 -- 172.21.15.40:0/299566925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffabc0c9520 con 0x7ffabc0c78d0 2024-09-13T18:16:21.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.340+0000 7ffac084e700 1 -- 172.21.15.40:0/299566925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffabc0c8bd0 con 0x7ffabc0c78d0 2024-09-13T18:16:21.348 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.340+0000 7ffaaaffd700 1 -- 172.21.15.40:0/299566925 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab0004180 con 0x7ffabc0c78d0 2024-09-13T18:16:21.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.340+0000 7ffaaaffd700 1 -- 172.21.15.40:0/299566925 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab0004530 con 0x7ffabc0c78d0 2024-09-13T18:16:21.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.344+0000 7ffaaaffd700 1 -- 172.21.15.40:0/299566925 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffab00036a0 con 0x7ffabc0c78d0 2024-09-13T18:16:21.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.344+0000 7ffaaaffd700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffaa40669c0 0x7ffaa4068e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.344+0000 7ffaba59c700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffaa40669c0 0x7ffaa4068e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:21.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.344+0000 7ffaba59c700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffaa40669c0 0x7ffaa4068e80 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ffaac006fd0 tx=0x7ffaac009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:21.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.348+0000 7ffaaaffd700 1 -- 172.21.15.40:0/299566925 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ffab0083f20 con 0x7ffabc0c78d0 2024-09-13T18:16:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.352+0000 7ffac084e700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffaa0002a40 0x7ffaa0004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:21.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.352+0000 7ffac084e700 1 -- 172.21.15.40:0/299566925 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa0007f40 con 0x7ffaa0002a40 2024-09-13T18:16:21.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.352+0000 7ffabad9d700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffaa0002a40 0x7ffaa0004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:21.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.352+0000 7ffabad9d700 1 -- 172.21.15.40:0/299566925 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffaa0008100 con 0x7ffaa0002a40 2024-09-13T18:16:21.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.352+0000 7ffabad9d700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffaa0002a40 0x7ffaa0004f00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009e50 tx=0x7ffab4004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:21.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.356+0000 7ffaaaffd700 1 -- 172.21.15.40:0/299566925 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4004440 con 0x7ffaa0002a40 2024-09-13T18:16:21.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.356+0000 7ffaaaffd700 1 -- 172.21.15.40:0/299566925 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ffab40045a0 con 0x7ffaa0002a40 2024-09-13T18:16:21.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.364+0000 7ffaaaffd700 1 -- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffaa0002a40 msgr2=0x7ffaa0004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:21.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.364+0000 7ffaaaffd700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffaa0002a40 0x7ffaa0004f00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009e50 tx=0x7ffab4004af0 comp rx=0 tx=0).stop 2024-09-13T18:16:21.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.364+0000 7ffaaaffd700 1 -- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffaa0002a40 msgr2=0x7ffaa0004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:16:21.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.364+0000 7ffaaaffd700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffaa0002a40 0x7ffaa0004f00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:21.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.372+0000 7ffac084e700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffaa0007020 0x7ffaa0007490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:21.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.372+0000 7ffac084e700 1 -- 172.21.15.40:0/299566925 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa000a400 con 0x7ffaa0007020 2024-09-13T18:16:21.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.372+0000 7ffabad9d700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffaa0007020 0x7ffaa0007490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:21.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.372+0000 7ffabad9d700 1 -- 172.21.15.40:0/299566925 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ffaa000a5c0 con 0x7ffaa0007020 2024-09-13T18:16:21.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.372+0000 7ffabad9d700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffaa0007020 0x7ffaa0007490 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00080d0 tx=0x7ffab4011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:21.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.376+0000 7ffaaaffd700 1 -- 172.21.15.40:0/299566925 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab402c070 con 0x7ffaa0007020 2024-09-13T18:16:21.383 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.376+0000 7ffaaaffd700 1 -- 172.21.15.40:0/299566925 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7ffab40126f0 con 0x7ffaa0007020 2024-09-13T18:16:21.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.376+0000 7ffaaaffd700 1 -- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffaa0007020 msgr2=0x7ffaa0007490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:21.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.376+0000 7ffaaaffd700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffaa0007020 0x7ffaa0007490 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00080d0 tx=0x7ffab4011000 comp rx=0 tx=0).stop 2024-09-13T18:16:21.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.376+0000 7ffaaaffd700 1 -- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffaa0007020 msgr2=0x7ffaa0007490 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:16:21.384 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.376+0000 7ffaaaffd700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffaa0007020 0x7ffaa0007490 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:21.385 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:16:21.385 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:16:21.385 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:16:21.385 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:16:21.385 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 210, 2024-09-13T18:16:21.385 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:16:21.386 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:16:21.386 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:16:21.386 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:16:21.386 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:21.386 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 3, 2024-09-13T18:16:21.386 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:16:21.386 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:16:21.386 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:16:21.386 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:21.386 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:21.386 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:21.386 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:21.386 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:21.387 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:21.387 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:21.387 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:21.387 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:21.387 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:21.387 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:21.387 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:16:21.387 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:16:21.387 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:21.387 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:21.387 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:21.388 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:21.388 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:21.388 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:21.388 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:21.388 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:21.388 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:21.388 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:21.388 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:21.388 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:21.389 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:16:21.389 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:16:21.389 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:16:21.389 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:16:21.389 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:16:21.389 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:16:21.389 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:16:21.389 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:16:21.389 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:16:21.389 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:16:21.389 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:16:21.389 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:16:21.389 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:16:21.390 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:16:21.390 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:16:21.390 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:16:21.390 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:16:21.390 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:21.390 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:21.390 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:21.390 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:21.390 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:21.390 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:21.390 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:21.390 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:21.390 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:21.390 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:21.391 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:21.391 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:16:21.391 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:21.391 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:16:21.391 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:21.391 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:16:21.391 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:16:21.391 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:21.391 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:21.391 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:21.391 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:21.391 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:16:21.391 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:21.392 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:21.392 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:21.392 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:21.392 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:16:21.392 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:21.392 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:21.392 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:21.392 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:21.392 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:16:21.393 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:16:21.393 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:21.393 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:21.393 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:21.393 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:21.393 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:21.393 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:16:21.393 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:16:21.393 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:21.393 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:21.393 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:21.393 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:21.393 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:16:21.393 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:21.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:21.394 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:21.394 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:21.394 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:16:21.394 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:21.394 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:21.394 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:21.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:21.394 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:16:21.394 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:16:21.394 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:21.394 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:21.394 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:21.394 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:21.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:21.395 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:16:21.395 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:16:21.395 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:21.395 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:21.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:21.395 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:21.395 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:16:21.395 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:21.395 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:21.395 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:21.395 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:21.395 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:16:21.395 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:21.396 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:21.396 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:21.396 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:21.396 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:16:21.396 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:16:21.396 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:21.396 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:21.396 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:21.396 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:21.396 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:21.396 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:21.396 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:16:21.396 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:16:21.397 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:21.397 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:21.397 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:21.397 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:16:21.397 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:21.397 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:21.397 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:16:21.397 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:21.398 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:21.398 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:21.398 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:16:21.398 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:21.398 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:21.398 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:16:21.398 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:21.399 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:21.399 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:21.399 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 8 2024-09-13T18:16:21.399 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:21.399 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:21.399 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:16:21.399 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:21.399 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:21.399 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:21.399 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:16:21.399 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:21.399 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:21.399 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:16:21.399 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:21.400 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:21.400 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:21.400 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:16:21.400 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:21.400 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:21.400 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:21.400 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:16:21.400 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:16:21.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.380+0000 7ffac084e700 1 -- 172.21.15.40:0/299566925 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffaa40669c0 msgr2=0x7ffaa4068e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:21.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.380+0000 7ffac084e700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ffaa40669c0 0x7ffaa4068e80 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ffaac006fd0 tx=0x7ffaac009190 comp rx=0 tx=0).stop 2024-09-13T18:16:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.380+0000 7ffabad9d700 1 -- 172.21.15.40:0/299566925 reap_dead start 2024-09-13T18:16:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.380+0000 7ffac084e700 1 -- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc0c78d0 msgr2=0x7ffabc0cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.380+0000 7ffac084e700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc0c78d0 0x7ffabc0cb630 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffab000b450 tx=0x7ffab000b760 comp rx=0 tx=0).stop 2024-09-13T18:16:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.380+0000 7ffac084e700 1 -- 172.21.15.40:0/299566925 shutdown_connections 2024-09-13T18:16:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.380+0000 7ffac084e700 1 --2- 172.21.15.40:0/299566925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc0c78d0 0x7ffabc0cb630 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.380+0000 7ffac084e700 1 -- 172.21.15.40:0/299566925 >> 172.21.15.40:0/299566925 conn(0x7ffabc066090 msgr2=0x7ffabc1575f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.384+0000 7ffac084e700 1 -- 172.21.15.40:0/299566925 shutdown_connections 2024-09-13T18:16:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.384+0000 7ffac084e700 1 -- 172.21.15.40:0/299566925 wait complete. 2024-09-13T18:16:21.409 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:16:21.409 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:16:21.409 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:16:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.676+0000 7fcb40bc1700 1 Processor -- start 2024-09-13T18:16:21.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.676+0000 7fcb40bc1700 1 -- start start 2024-09-13T18:16:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.680+0000 7fcb40bc1700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcb3c071f70 0x7fcb3c0c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:21.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.680+0000 7fcb40bc1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcb3c072350 0x7fcb3c150d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:21.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.680+0000 7fcb40bc1700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb3c0c55c0 con 0x7fcb3c0c7200 2024-09-13T18:16:21.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.680+0000 7fcb40bc1700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb3c06a390 con 0x7fcb3c072350 2024-09-13T18:16:21.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.680+0000 7fcb40bc1700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb3c157710 con 0x7fcb3c071f70 2024-09-13T18:16:21.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.680+0000 7fcb3a59c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcb3c071f70 0x7fcb3c0c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:21.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.680+0000 7fcb3ad9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcb3c0c7200 0x7fcb3c15a700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52122/0 (socket says 172.21.15.40:52122) 2024-09-13T18:16:21.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.680+0000 7fcb3ad9d700 1 -- 172.21.15.40:0/419390757 learned_addr learned my addr 172.21.15.40:0/419390757 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:16:21.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.680+0000 7fcb39d9b700 1 --2- 172.21.15.40:0/419390757 >> v2:172.21.15.40:3300/0 conn(0x7fcb3c072350 0x7fcb3c150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.680+0000 7fcb39d9b700 1 -- 172.21.15.40:0/419390757 >> v2:172.21.15.43:3300/0 conn(0x7fcb3c071f70 msgr2=0x7fcb3c0c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:16:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.680+0000 7fcb39d9b700 1 --2- 172.21.15.40:0/419390757 >> v2:172.21.15.43:3300/0 conn(0x7fcb3c071f70 0x7fcb3c0c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.680+0000 7fcb39d9b700 1 -- 172.21.15.40:0/419390757 >> v1:172.21.15.40:6789/0 conn(0x7fcb3c0c7200 legacy=0x7fcb3c15a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.680+0000 7fcb39d9b700 1 -- 172.21.15.40:0/419390757 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3c1579d0 con 0x7fcb3c072350 2024-09-13T18:16:21.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.680+0000 7fcb39d9b700 1 --2- 172.21.15.40:0/419390757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb3c072350 0x7fcb3c150d40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0672a0 tx=0x7fcb3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a045cd288c2bb8cc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.680+0000 7fcb3959a700 1 -- 172.21.15.40:0/419390757 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3400e070 con 0x7fcb3c072350 2024-09-13T18:16:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.680+0000 7fcb3959a700 1 -- 172.21.15.40:0/419390757 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3400bb20 con 0x7fcb3c072350 2024-09-13T18:16:21.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.680+0000 7fcb3959a700 1 -- 172.21.15.40:0/419390757 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34012430 con 0x7fcb3c072350 2024-09-13T18:16:21.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.680+0000 7fcb40bc1700 1 -- 172.21.15.40:0/419390757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb3c072350 msgr2=0x7fcb3c150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:21.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.680+0000 7fcb40bc1700 1 --2- 172.21.15.40:0/419390757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb3c072350 0x7fcb3c150d40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0672a0 tx=0x7fcb3400b330 comp rx=0 tx=0).stop 2024-09-13T18:16:21.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.688+0000 7fcb40bc1700 1 -- 172.21.15.40:0/419390757 shutdown_connections 2024-09-13T18:16:21.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.688+0000 7fcb40bc1700 1 --2- 172.21.15.40:0/419390757 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb3c072350 0x7fcb3c150d40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:21.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.688+0000 7fcb40bc1700 1 --2- 172.21.15.40:0/419390757 >> v2:172.21.15.43:3300/0 conn(0x7fcb3c071f70 0x7fcb3c0c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:21.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.688+0000 7fcb40bc1700 1 -- 172.21.15.40:0/419390757 >> 172.21.15.40:0/419390757 conn(0x7fcb3c0cabc0 msgr2=0x7fcb3c0cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:21.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.688+0000 7fcb40bc1700 1 -- 172.21.15.40:0/419390757 shutdown_connections 2024-09-13T18:16:21.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.688+0000 7fcb40bc1700 1 -- 172.21.15.40:0/419390757 wait complete. 2024-09-13T18:16:21.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.688+0000 7fcb40bc1700 1 Processor -- start 2024-09-13T18:16:21.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.688+0000 7fcb40bc1700 1 -- start start 2024-09-13T18:16:21.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.688+0000 7fcb40bc1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb3c071f70 0x7fcb3c14d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:21.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.692+0000 7fcb40bc1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb3c072350 0x7fcb3c1e2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:21.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.692+0000 7fcb40bc1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb3c0c7200 0x7fcb3c1e28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:21.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.692+0000 7fcb40bc1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c1e3030 con 0x7fcb3c0c7200 2024-09-13T18:16:21.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.692+0000 7fcb40bc1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c1e4890 con 0x7fcb3c071f70 2024-09-13T18:16:21.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.692+0000 7fcb40bc1700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c1fc200 con 0x7fcb3c072350 2024-09-13T18:16:21.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.692+0000 7fcb3ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb3c0c7200 0x7fcb3c1e28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:21.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.692+0000 7fcb3ad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb3c0c7200 0x7fcb3c1e28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33628/0 (socket says 172.21.15.40:33628) 2024-09-13T18:16:21.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.692+0000 7fcb3ad9d700 1 -- 172.21.15.40:0/3773011402 learned_addr learned my addr 172.21.15.40:0/3773011402 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:16:21.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.692+0000 7fcb3a59c700 1 --2- 172.21.15.40:0/3773011402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb3c071f70 0x7fcb3c14d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:21.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.692+0000 7fcb3ad9d700 1 -- 172.21.15.40:0/3773011402 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb3c072350 msgr2=0x7fcb3c1e2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:21.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.692+0000 7fcb3ad9d700 1 --2- 172.21.15.40:0/3773011402 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb3c072350 0x7fcb3c1e2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:21.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.692+0000 7fcb3ad9d700 1 -- 172.21.15.40:0/3773011402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb3c071f70 msgr2=0x7fcb3c14d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:21.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.692+0000 7fcb3ad9d700 1 --2- 172.21.15.40:0/3773011402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb3c071f70 0x7fcb3c14d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:21.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.692+0000 7fcb3ad9d700 1 -- 172.21.15.40:0/3773011402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb34009a70 con 0x7fcb3c0c7200 2024-09-13T18:16:21.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.692+0000 7fcb3ad9d700 1 --2- 172.21.15.40:0/3773011402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb3c0c7200 0x7fcb3c1e28d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000c370 tx=0x7fcb3000c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:21.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.692+0000 7fcb2affd700 1 -- 172.21.15.40:0/3773011402 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30005d90 con 0x7fcb3c0c7200 2024-09-13T18:16:21.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.692+0000 7fcb2affd700 1 -- 172.21.15.40:0/3773011402 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3000ce30 con 0x7fcb3c0c7200 2024-09-13T18:16:21.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.692+0000 7fcb2affd700 1 -- 172.21.15.40:0/3773011402 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb300072d0 con 0x7fcb3c0c7200 2024-09-13T18:16:21.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.692+0000 7fcb40bc1700 1 -- 172.21.15.40:0/3773011402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3c1fc430 con 0x7fcb3c0c7200 2024-09-13T18:16:21.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.692+0000 7fcb40bc1700 1 -- 172.21.15.40:0/3773011402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3c1e42d0 con 0x7fcb3c0c7200 2024-09-13T18:16:21.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.696+0000 7fcb40bc1700 1 -- 172.21.15.40:0/3773011402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb3c154da0 con 0x7fcb3c0c7200 2024-09-13T18:16:21.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.696+0000 7fcb2affd700 1 -- 172.21.15.40:0/3773011402 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcb30007430 con 0x7fcb3c0c7200 2024-09-13T18:16:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.716+0000 7fcb2affd700 1 --2- 172.21.15.40:0/3773011402 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcb1c066730 0x7fcb1c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.716+0000 7fcb2affd700 1 -- 172.21.15.40:0/3773011402 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcb30083680 con 0x7fcb3c0c7200 2024-09-13T18:16:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.716+0000 7fcb3a59c700 1 --2- 172.21.15.40:0/3773011402 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcb1c066730 0x7fcb1c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.716+0000 7fcb2affd700 1 -- 172.21.15.40:0/3773011402 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcb30085bf0 con 0x7fcb3c0c7200 2024-09-13T18:16:21.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:21.716+0000 7fcb3a59c700 1 --2- 172.21.15.40:0/3773011402 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcb1c066730 0x7fcb1c068bf0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009bf0 tx=0x7fcb2c009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:22.652 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:22.644+0000 7fcb40bc1700 1 -- 172.21.15.40:0/3773011402 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fcb3c0cb6e0 con 0x7fcb3c0c7200 2024-09-13T18:16:22.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:22.648+0000 7fcb2affd700 1 -- 172.21.15.40:0/3773011402 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fcb300506d0 con 0x7fcb3c0c7200 2024-09-13T18:16:22.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:22.668+0000 7fcb40bc1700 1 -- 172.21.15.40:0/3773011402 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcb1c066730 msgr2=0x7fcb1c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:22.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:22.672+0000 7fcb40bc1700 1 --2- 172.21.15.40:0/3773011402 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcb1c066730 0x7fcb1c068bf0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009bf0 tx=0x7fcb2c009240 comp rx=0 tx=0).stop 2024-09-13T18:16:22.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:22.672+0000 7fcb40bc1700 1 -- 172.21.15.40:0/3773011402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb3c0c7200 msgr2=0x7fcb3c1e28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:22.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:22.672+0000 7fcb40bc1700 1 --2- 172.21.15.40:0/3773011402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb3c0c7200 0x7fcb3c1e28d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000c370 tx=0x7fcb3000c680 comp rx=0 tx=0).stop 2024-09-13T18:16:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:22.672+0000 7fcb40bc1700 1 -- 172.21.15.40:0/3773011402 shutdown_connections 2024-09-13T18:16:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:22.672+0000 7fcb40bc1700 1 --2- 172.21.15.40:0/3773011402 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb3c0c7200 0x7fcb3c1e28d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:22.672+0000 7fcb40bc1700 1 --2- 172.21.15.40:0/3773011402 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcb1c066730 0x7fcb1c068bf0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:22.672+0000 7fcb40bc1700 1 --2- 172.21.15.40:0/3773011402 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb3c072350 0x7fcb3c1e2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:22.672+0000 7fcb40bc1700 1 --2- 172.21.15.40:0/3773011402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb3c071f70 0x7fcb3c14d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:22.672+0000 7fcb40bc1700 1 -- 172.21.15.40:0/3773011402 >> 172.21.15.40:0/3773011402 conn(0x7fcb3c0cabc0 msgr2=0x7fcb3c0c5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:22.672+0000 7fcb40bc1700 1 -- 172.21.15.40:0/3773011402 shutdown_connections 2024-09-13T18:16:22.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:22.672+0000 7fcb40bc1700 1 -- 172.21.15.40:0/3773011402 wait complete. 2024-09-13T18:16:22.709 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:16:32.710 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:16:32.711 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:16:33.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.048+0000 7fe939d0b700 1 Processor -- start 2024-09-13T18:16:33.055 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.048+0000 7fe939d0b700 1 -- start start 2024-09-13T18:16:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.048+0000 7fe939d0b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe9340c55c0 0x7fe9340c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.048+0000 7fe939d0b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe93406a390 0x7fe93406a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.048+0000 7fe939d0b700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe934155f30 con 0x7fe93414f5b0 2024-09-13T18:16:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.048+0000 7fe939d0b700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9341553f0 con 0x7fe9340c55c0 2024-09-13T18:16:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.048+0000 7fe939d0b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9340c7110 con 0x7fe93406a390 2024-09-13T18:16:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.048+0000 7fe933fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe93406a390 0x7fe93406a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.048+0000 7fe933fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe93406a390 0x7fe93406a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:56972/0 (socket says 172.21.15.40:56972) 2024-09-13T18:16:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.048+0000 7fe933fff700 1 -- 172.21.15.40:0/523547780 learned_addr learned my addr 172.21.15.40:0/523547780 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:16:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.048+0000 7fe933fff700 1 -- 172.21.15.40:0/523547780 >> v2:172.21.15.43:3300/0 conn(0x7fe9340c55c0 msgr2=0x7fe9340c5a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:16:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.048+0000 7fe933fff700 1 --2- 172.21.15.40:0/523547780 >> v2:172.21.15.43:3300/0 conn(0x7fe9340c55c0 0x7fe9340c5a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.048+0000 7fe933fff700 1 -- 172.21.15.40:0/523547780 >> v1:172.21.15.40:6789/0 conn(0x7fe93414f5b0 legacy=0x7fe9340c6960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:16:33.063 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.048+0000 7fe933fff700 1 -- 172.21.15.40:0/523547780 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe934155600 con 0x7fe93406a390 2024-09-13T18:16:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.052+0000 7fe933fff700 1 --2- 172.21.15.40:0/523547780 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe93406a390 0x7fe93406a820 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009e50 tx=0x7fe92c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12610994f7f5a0e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.056+0000 7fe9327fc700 1 -- 172.21.15.40:0/523547780 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c00e070 con 0x7fe93406a390 2024-09-13T18:16:33.064 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.056+0000 7fe9327fc700 1 -- 172.21.15.40:0/523547780 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92c003a40 con 0x7fe93406a390 2024-09-13T18:16:33.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.056+0000 7fe939d0b700 1 -- 172.21.15.40:0/523547780 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe93406a390 msgr2=0x7fe93406a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:33.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.056+0000 7fe939d0b700 1 --2- 172.21.15.40:0/523547780 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe93406a390 0x7fe93406a820 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe92c009e50 tx=0x7fe92c00b330 comp rx=0 tx=0).stop 2024-09-13T18:16:33.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.056+0000 7fe939d0b700 1 -- 172.21.15.40:0/523547780 shutdown_connections 2024-09-13T18:16:33.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.056+0000 7fe939d0b700 1 --2- 172.21.15.40:0/523547780 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe93406a390 0x7fe93406a820 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:33.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.056+0000 7fe939d0b700 1 --2- 172.21.15.40:0/523547780 >> v2:172.21.15.43:3300/0 conn(0x7fe9340c55c0 0x7fe9340c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:33.067 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.056+0000 7fe939d0b700 1 -- 172.21.15.40:0/523547780 >> 172.21.15.40:0/523547780 conn(0x7fe9340cabc0 msgr2=0x7fe9340cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:33.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.060+0000 7fe939d0b700 1 -- 172.21.15.40:0/523547780 shutdown_connections 2024-09-13T18:16:33.068 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.060+0000 7fe939d0b700 1 -- 172.21.15.40:0/523547780 wait complete. 2024-09-13T18:16:33.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.060+0000 7fe939d0b700 1 Processor -- start 2024-09-13T18:16:33.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.064+0000 7fe939d0b700 1 -- start start 2024-09-13T18:16:33.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.064+0000 7fe939d0b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9340c55c0 0x7fe934160f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:33.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.064+0000 7fe939d0b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe93414f5b0 0x7fe9341524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:33.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.064+0000 7fe939d0b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe93415fc90 0x7fe9341f3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:33.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.064+0000 7fe939d0b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe934161d20 con 0x7fe9340c55c0 2024-09-13T18:16:33.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.064+0000 7fe939d0b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9340ca7a0 con 0x7fe93414f5b0 2024-09-13T18:16:33.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.064+0000 7fe939d0b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9341530f0 con 0x7fe93415fc90 2024-09-13T18:16:33.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.064+0000 7fe933fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe93415fc90 0x7fe9341f3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:33.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.064+0000 7fe933fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe93415fc90 0x7fe9341f3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:56982/0 (socket says 172.21.15.40:56982) 2024-09-13T18:16:33.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.064+0000 7fe933fff700 1 -- 172.21.15.40:0/3034222309 learned_addr learned my addr 172.21.15.40:0/3034222309 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:16:33.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.064+0000 7fe933fff700 1 -- 172.21.15.40:0/3034222309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe93414f5b0 msgr2=0x7fe9341524d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:16:33.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.064+0000 7fe933fff700 1 --2- 172.21.15.40:0/3034222309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe93414f5b0 0x7fe9341524d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:33.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.064+0000 7fe933fff700 1 -- 172.21.15.40:0/3034222309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9340c55c0 msgr2=0x7fe934160f40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:16:33.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.064+0000 7fe933fff700 1 --2- 172.21.15.40:0/3034222309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9340c55c0 0x7fe934160f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:33.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.064+0000 7fe933fff700 1 -- 172.21.15.40:0/3034222309 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c009b30 con 0x7fe93415fc90 2024-09-13T18:16:33.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.068+0000 7fe933fff700 1 --2- 172.21.15.40:0/3034222309 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe93415fc90 0x7fe9341f3350 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe92c004290 tx=0x7fe92c004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.072+0000 7fe91bfff700 1 -- 172.21.15.40:0/3034222309 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c0048e0 con 0x7fe93415fc90 2024-09-13T18:16:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.072+0000 7fe91bfff700 1 -- 172.21.15.40:0/3034222309 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92c003a40 con 0x7fe93415fc90 2024-09-13T18:16:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.072+0000 7fe91bfff700 1 -- 172.21.15.40:0/3034222309 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c012630 con 0x7fe93415fc90 2024-09-13T18:16:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.072+0000 7fe939d0b700 1 -- 172.21.15.40:0/3034222309 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9341533d0 con 0x7fe93415fc90 2024-09-13T18:16:33.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.072+0000 7fe939d0b700 1 -- 172.21.15.40:0/3034222309 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9340ca4c0 con 0x7fe93415fc90 2024-09-13T18:16:33.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.076+0000 7fe91bfff700 1 -- 172.21.15.40:0/3034222309 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe92c012790 con 0x7fe93415fc90 2024-09-13T18:16:33.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.076+0000 7fe91bfff700 1 --2- 172.21.15.40:0/3034222309 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe920066830 0x7fe920068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:33.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.076+0000 7fe91bfff700 1 -- 172.21.15.40:0/3034222309 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe92c085e70 con 0x7fe93415fc90 2024-09-13T18:16:33.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.076+0000 7fe939d0b700 1 -- 172.21.15.40:0/3034222309 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe934072830 con 0x7fe93415fc90 2024-09-13T18:16:33.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.080+0000 7fe9337fe700 1 --2- 172.21.15.40:0/3034222309 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe920066830 0x7fe920068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:33.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.080+0000 7fe9337fe700 1 --2- 172.21.15.40:0/3034222309 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe920066830 0x7fe920068cf0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe924009690 tx=0x7fe924006bf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:33.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.088+0000 7fe91bfff700 1 -- 172.21.15.40:0/3034222309 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe92c052e90 con 0x7fe93415fc90 2024-09-13T18:16:33.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.920+0000 7fe939d0b700 1 -- 172.21.15.40:0/3034222309 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe93406faf0 con 0x7fe93415fc90 2024-09-13T18:16:33.927 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.920+0000 7fe91bfff700 1 -- 172.21.15.40:0/3034222309 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7fe92c017070 con 0x7fe93415fc90 2024-09-13T18:16:33.930 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":210,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:16:33.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.924+0000 7fe939d0b700 1 -- 172.21.15.40:0/3034222309 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe920066830 msgr2=0x7fe920068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:33.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.924+0000 7fe939d0b700 1 --2- 172.21.15.40:0/3034222309 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe920066830 0x7fe920068cf0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe924009690 tx=0x7fe924006bf0 comp rx=0 tx=0).stop 2024-09-13T18:16:33.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.924+0000 7fe939d0b700 1 -- 172.21.15.40:0/3034222309 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe93415fc90 msgr2=0x7fe9341f3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:33.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.924+0000 7fe939d0b700 1 --2- 172.21.15.40:0/3034222309 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe93415fc90 0x7fe9341f3350 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe92c004290 tx=0x7fe92c004af0 comp rx=0 tx=0).stop 2024-09-13T18:16:33.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.924+0000 7fe939d0b700 1 -- 172.21.15.40:0/3034222309 shutdown_connections 2024-09-13T18:16:33.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.924+0000 7fe939d0b700 1 --2- 172.21.15.40:0/3034222309 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe920066830 0x7fe920068cf0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:33.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.924+0000 7fe939d0b700 1 --2- 172.21.15.40:0/3034222309 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe93415fc90 0x7fe9341f3350 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:33.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.924+0000 7fe939d0b700 1 --2- 172.21.15.40:0/3034222309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe93414f5b0 0x7fe9341524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:33.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.924+0000 7fe939d0b700 1 --2- 172.21.15.40:0/3034222309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9340c55c0 0x7fe934160f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:33.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.924+0000 7fe939d0b700 1 -- 172.21.15.40:0/3034222309 >> 172.21.15.40:0/3034222309 conn(0x7fe9340cabc0 msgr2=0x7fe934154120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:33.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.924+0000 7fe939d0b700 1 -- 172.21.15.40:0/3034222309 shutdown_connections 2024-09-13T18:16:33.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:33.924+0000 7fe939d0b700 1 -- 172.21.15.40:0/3034222309 wait complete. 2024-09-13T18:16:33.958 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:16:33.958 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T18:16:33.958 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:16:33.958 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:16:34.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.248+0000 7f8d3d238700 1 Processor -- start 2024-09-13T18:16:34.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.248+0000 7f8d3d238700 1 -- start start 2024-09-13T18:16:34.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.248+0000 7f8d3d238700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8d38153770 0x7f8d3806a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:34.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.248+0000 7f8d3d238700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8d3815d230 0x7f8d3806a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:34.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.248+0000 7f8d3d238700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d380729d0 con 0x7f8d3806afb0 2024-09-13T18:16:34.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.248+0000 7f8d3d238700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d3815a2c0 con 0x7f8d38153770 2024-09-13T18:16:34.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.248+0000 7f8d3d238700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d380c0970 con 0x7f8d3815d230 2024-09-13T18:16:34.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.252+0000 7f8d36d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8d38153770 0x7f8d3806a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:34.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.252+0000 7f8d36d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f8d38153770 0x7f8d3806a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59270/0 (socket says 172.21.15.40:59270) 2024-09-13T18:16:34.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.252+0000 7f8d36d9d700 1 -- 172.21.15.40:0/278381897 learned_addr learned my addr 172.21.15.40:0/278381897 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:16:34.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.252+0000 7f8d36d9d700 1 -- 172.21.15.40:0/278381897 >> v2:172.21.15.43:3300/0 conn(0x7f8d3815d230 msgr2=0x7f8d3806a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:16:34.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.252+0000 7f8d36d9d700 1 --2- 172.21.15.40:0/278381897 >> v2:172.21.15.43:3300/0 conn(0x7f8d3815d230 0x7f8d3806a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:34.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.252+0000 7f8d36d9d700 1 -- 172.21.15.40:0/278381897 >> v1:172.21.15.43:6789/0 conn(0x7f8d3806afb0 legacy=0x7f8d3815cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:34.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.252+0000 7f8d36d9d700 1 -- 172.21.15.40:0/278381897 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d380c6d00 con 0x7f8d38153770 2024-09-13T18:16:34.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.252+0000 7f8d36d9d700 1 --2- 172.21.15.40:0/278381897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d38153770 0x7f8d3806a390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8d3806d560 tx=0x7f8d2c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a75a7cbbae0f64c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:34.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.252+0000 7f8d35d9b700 1 -- 172.21.15.40:0/278381897 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c00b8a0 con 0x7f8d38153770 2024-09-13T18:16:34.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.252+0000 7f8d35d9b700 1 -- 172.21.15.40:0/278381897 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2c00ba00 con 0x7f8d38153770 2024-09-13T18:16:34.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.252+0000 7f8d35d9b700 1 -- 172.21.15.40:0/278381897 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c00bd90 con 0x7f8d38153770 2024-09-13T18:16:34.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.252+0000 7f8d3d238700 1 -- 172.21.15.40:0/278381897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d38153770 msgr2=0x7f8d3806a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:34.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.252+0000 7f8d3d238700 1 --2- 172.21.15.40:0/278381897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d38153770 0x7f8d3806a390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8d3806d560 tx=0x7f8d2c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:16:34.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.252+0000 7f8d3d238700 1 -- 172.21.15.40:0/278381897 shutdown_connections 2024-09-13T18:16:34.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.252+0000 7f8d3d238700 1 --2- 172.21.15.40:0/278381897 >> v2:172.21.15.43:3300/0 conn(0x7f8d3815d230 0x7f8d3806a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:34.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.252+0000 7f8d3d238700 1 --2- 172.21.15.40:0/278381897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d38153770 0x7f8d3806a390 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:34.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.252+0000 7f8d3d238700 1 -- 172.21.15.40:0/278381897 >> 172.21.15.40:0/278381897 conn(0x7f8d3806d5b0 msgr2=0x7f8d3806d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:34.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.256+0000 7f8d3d238700 1 -- 172.21.15.40:0/278381897 shutdown_connections 2024-09-13T18:16:34.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.260+0000 7f8d3d238700 1 -- 172.21.15.40:0/278381897 wait complete. 2024-09-13T18:16:34.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.260+0000 7f8d3d238700 1 Processor -- start 2024-09-13T18:16:34.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.260+0000 7f8d3d238700 1 -- start start 2024-09-13T18:16:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.268+0000 7f8d3d238700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d3806afb0 0x7f8d3815c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.268+0000 7f8d3d238700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d38153770 0x7f8d380af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.268+0000 7f8d3d238700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d3815d230 0x7f8d3815b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.268+0000 7f8d3d238700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d380afb00 con 0x7f8d3806afb0 2024-09-13T18:16:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.268+0000 7f8d3d238700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d381eeca0 con 0x7f8d3815d230 2024-09-13T18:16:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.268+0000 7f8d3d238700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d381f2c90 con 0x7f8d38153770 2024-09-13T18:16:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.268+0000 7f8d3759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d3815d230 0x7f8d3815b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.268+0000 7f8d3759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d3815d230 0x7f8d3815b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:47764/0 (socket says 172.21.15.40:47764) 2024-09-13T18:16:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.268+0000 7f8d3759e700 1 -- 172.21.15.40:0/376886938 learned_addr learned my addr 172.21.15.40:0/376886938 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:16:34.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.268+0000 7f8d3659c700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d38153770 0x7f8d380af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.268+0000 7f8d3659c700 1 -- 172.21.15.40:0/376886938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d3815d230 msgr2=0x7f8d3815b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.268+0000 7f8d3659c700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d3815d230 0x7f8d3815b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.268+0000 7f8d36d9d700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d3806afb0 0x7f8d3815c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.268+0000 7f8d3659c700 1 -- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d3806afb0 msgr2=0x7f8d3815c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.268+0000 7f8d3659c700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d3806afb0 0x7f8d3815c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.268+0000 7f8d3659c700 1 -- 172.21.15.40:0/376886938 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c009570 con 0x7f8d38153770 2024-09-13T18:16:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.268+0000 7f8d36d9d700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d3806afb0 0x7f8d3815c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:16:34.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.268+0000 7f8d3659c700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d38153770 0x7f8d380af5c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000cbe0 tx=0x7f8d3000cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:34.288 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.280+0000 7f8d277fe700 1 -- 172.21.15.40:0/376886938 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30003c20 con 0x7f8d38153770 2024-09-13T18:16:34.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.280+0000 7f8d3d238700 1 -- 172.21.15.40:0/376886938 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d381eef40 con 0x7f8d38153770 2024-09-13T18:16:34.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.280+0000 7f8d3d238700 1 -- 172.21.15.40:0/376886938 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d381ef500 con 0x7f8d38153770 2024-09-13T18:16:34.289 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.280+0000 7f8d277fe700 1 -- 172.21.15.40:0/376886938 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d30003d80 con 0x7f8d38153770 2024-09-13T18:16:34.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.280+0000 7f8d277fe700 1 -- 172.21.15.40:0/376886938 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3000ab50 con 0x7f8d38153770 2024-09-13T18:16:34.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.284+0000 7f8d277fe700 1 -- 172.21.15.40:0/376886938 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8d3000acb0 con 0x7f8d38153770 2024-09-13T18:16:34.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.288+0000 7f8d3d238700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d14002a40 0x7f8d14004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:34.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.292+0000 7f8d36d9d700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d14002a40 0x7f8d14004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:34.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.292+0000 7f8d3d238700 1 -- 172.21.15.40:0/376886938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d14007f40 con 0x7f8d14002a40 2024-09-13T18:16:34.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.292+0000 7f8d36d9d700 1 -- 172.21.15.40:0/376886938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d14008100 con 0x7f8d14002a40 2024-09-13T18:16:34.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.292+0000 7f8d36d9d700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d14002a40 0x7f8d14004f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8d3806d560 tx=0x7f8d2c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:34.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.296+0000 7f8d277fe700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8d1c066800 0x7f8d1c068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:34.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.296+0000 7f8d277fe700 1 -- 172.21.15.40:0/376886938 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8d30084440 con 0x7f8d38153770 2024-09-13T18:16:34.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.296+0000 7f8d277fe700 1 -- 172.21.15.40:0/376886938 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c003ad0 con 0x7f8d14002a40 2024-09-13T18:16:34.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.296+0000 7f8d277fe700 1 -- 172.21.15.40:0/376886938 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8d2c00b820 con 0x7f8d14002a40 2024-09-13T18:16:34.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.296+0000 7f8d277fe700 1 -- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d14002a40 msgr2=0x7f8d14004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:34.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.296+0000 7f8d277fe700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d14002a40 0x7f8d14004f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8d3806d560 tx=0x7f8d2c004770 comp rx=0 tx=0).stop 2024-09-13T18:16:34.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.296+0000 7f8d3759e700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8d1c066800 0x7f8d1c068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:34.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.296+0000 7f8d277fe700 1 -- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d14002a40 msgr2=0x7f8d14004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:16:34.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.296+0000 7f8d277fe700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d14002a40 0x7f8d14004f00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:34.306 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.300+0000 7f8d3759e700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8d1c066800 0x7f8d1c068cc0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8d280098b0 tx=0x7f8d28009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:34.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.316+0000 7f8d3d238700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d14007020 0x7f8d140074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:34.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.316+0000 7f8d3d238700 1 -- 172.21.15.40:0/376886938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1400a400 con 0x7f8d14007020 2024-09-13T18:16:34.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.316+0000 7f8d36d9d700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d14007020 0x7f8d140074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:34.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.316+0000 7f8d36d9d700 1 -- 172.21.15.40:0/376886938 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f8d1400a5c0 con 0x7f8d14007020 2024-09-13T18:16:34.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.316+0000 7f8d36d9d700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d14007020 0x7f8d140074b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009c30 tx=0x7f8d2c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:34.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.320+0000 7f8d277fe700 1 -- 172.21.15.40:0/376886938 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c035640 con 0x7f8d14007020 2024-09-13T18:16:34.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.320+0000 7f8d277fe700 1 -- 172.21.15.40:0/376886938 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4453 (secure 0 0 0) 0x7f8d2c0357a0 con 0x7f8d14007020 2024-09-13T18:16:34.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.320+0000 7f8d277fe700 1 -- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d14007020 msgr2=0x7f8d140074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:34.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.320+0000 7f8d277fe700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d14007020 0x7f8d140074b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009c30 tx=0x7f8d2c011000 comp rx=0 tx=0).stop 2024-09-13T18:16:34.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.320+0000 7f8d277fe700 1 -- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d14007020 msgr2=0x7f8d140074b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:16:34.327 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:16:34.327 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:16:34.327 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:16:34.327 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:16:34.327 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 210, 2024-09-13T18:16:34.327 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:16:34.327 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:16:34.328 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:16:34.328 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:16:34.328 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:34.328 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 16, 2024-09-13T18:16:34.328 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:16:34.328 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:16:34.328 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:16:34.328 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:34.328 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:34.328 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:34.328 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:34.328 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:34.328 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:16:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:16:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:34.330 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:34.330 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:34.330 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:34.330 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:34.330 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.330 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:16:34.330 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:16:34.330 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:16:34.330 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:16:34.330 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:16:34.330 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:16:34.330 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:16:34.330 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:16:34.330 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:16:34.331 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:16:34.331 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:16:34.331 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:16:34.331 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:16:34.332 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:16:34.332 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:16:34.332 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:16:34.332 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:16:34.332 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:34.332 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:34.332 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:34.332 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:34.332 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:34.332 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:34.332 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:34.332 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:34.332 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:34.333 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:34.333 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:34.333 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:16:34.333 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.333 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:16:34.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.333 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:16:34.333 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:16:34.333 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:34.333 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:34.333 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.333 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:34.333 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:16:34.333 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:34.334 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.334 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.334 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:34.334 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:16:34.334 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:34.334 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:34.334 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:34.334 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.334 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:16:34.334 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:16:34.334 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:34.334 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:34.334 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:34.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.335 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.335 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:16:34.335 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:16:34.335 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:34.335 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:34.335 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.335 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:34.335 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:16:34.335 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:34.335 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.335 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.336 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:34.336 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:16:34.336 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:34.336 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:34.336 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:34.336 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.336 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:16:34.337 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:16:34.337 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:34.337 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:34.337 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:34.337 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.337 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:16:34.337 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:16:34.337 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:34.337 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:34.337 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.337 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:34.338 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:16:34.338 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:34.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.338 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.338 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:34.338 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:16:34.338 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:34.338 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:34.338 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:34.338 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.339 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:16:34.339 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:16:34.339 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:34.339 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:34.339 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:34.339 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:34.339 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:34.339 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.339 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:16:34.339 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:16:34.339 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.339 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:34.339 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:34.339 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:16:34.340 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:34.340 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:34.340 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:16:34.340 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.340 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:34.340 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:34.340 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:16:34.340 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:34.340 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:34.340 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.340 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:16:34.340 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:16:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.320+0000 7f8d277fe700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8d14007020 0x7f8d140074b0 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009c30 tx=0x7f8d2c011000 comp rx=0 tx=0).stop 2024-09-13T18:16:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.324+0000 7f8d257fa700 1 -- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8d1c066800 msgr2=0x7f8d1c068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.324+0000 7f8d257fa700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8d1c066800 0x7f8d1c068cc0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8d280098b0 tx=0x7f8d28009190 comp rx=0 tx=0).stop 2024-09-13T18:16:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.328+0000 7f8d3759e700 1 -- 172.21.15.40:0/376886938 reap_dead start 2024-09-13T18:16:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.328+0000 7f8d257fa700 1 -- 172.21.15.40:0/376886938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d38153770 msgr2=0x7f8d380af5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.328+0000 7f8d257fa700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d38153770 0x7f8d380af5c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000cbe0 tx=0x7f8d3000cef0 comp rx=0 tx=0).stop 2024-09-13T18:16:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.328+0000 7f8d257fa700 1 -- 172.21.15.40:0/376886938 shutdown_connections 2024-09-13T18:16:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.328+0000 7f8d257fa700 1 --2- 172.21.15.40:0/376886938 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8d38153770 0x7f8d380af5c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.328+0000 7f8d257fa700 1 -- 172.21.15.40:0/376886938 >> 172.21.15.40:0/376886938 conn(0x7f8d3806d5b0 msgr2=0x7f8d3806c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.328+0000 7f8d257fa700 1 -- 172.21.15.40:0/376886938 shutdown_connections 2024-09-13T18:16:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.328+0000 7f8d257fa700 1 -- 172.21.15.40:0/376886938 wait complete. 2024-09-13T18:16:34.356 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:16:34.357 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:16:34.357 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:16:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.664+0000 7f568e98e700 1 Processor -- start 2024-09-13T18:16:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.664+0000 7f568e98e700 1 -- start start 2024-09-13T18:16:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.664+0000 7f568e98e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f56800dcd90 0x7f5680006eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.664+0000 7f568e98e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f56800e6a70 0x7f568004bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.664+0000 7f568e98e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5680006a20 con 0x7f5680044be0 2024-09-13T18:16:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.664+0000 7f568e98e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56800073f0 con 0x7f56800dcd90 2024-09-13T18:16:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.664+0000 7f568e98e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5680005ba0 con 0x7f56800e6a70 2024-09-13T18:16:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.668+0000 7f5687fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f56800dcd90 0x7f5680006eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.668+0000 7f5687fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f56800dcd90 0x7f5680006eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59310/0 (socket says 172.21.15.40:59310) 2024-09-13T18:16:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.668+0000 7f5687fff700 1 -- 172.21.15.40:0/633397072 learned_addr learned my addr 172.21.15.40:0/633397072 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:16:34.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.668+0000 7f5687fff700 1 -- 172.21.15.40:0/633397072 >> v2:172.21.15.43:3300/0 conn(0x7f56800e6a70 msgr2=0x7f568004bc80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:16:34.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.668+0000 7f5687fff700 1 --2- 172.21.15.40:0/633397072 >> v2:172.21.15.43:3300/0 conn(0x7f56800e6a70 0x7f568004bc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:34.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.668+0000 7f5687fff700 1 -- 172.21.15.40:0/633397072 >> v1:172.21.15.43:6789/0 conn(0x7f5680044be0 legacy=0x7f568004ad40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:16:34.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.668+0000 7f5687fff700 1 -- 172.21.15.40:0/633397072 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568004b450 con 0x7f56800dcd90 2024-09-13T18:16:34.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.668+0000 7f5687fff700 1 --2- 172.21.15.40:0/633397072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56800dcd90 0x7f5680006eb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5688063990 tx=0x7f5688069e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5694ee09bcdd4c4d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:34.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.668+0000 7f5686ffd700 1 -- 172.21.15.40:0/633397072 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5688067fd0 con 0x7f56800dcd90 2024-09-13T18:16:34.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.672+0000 7f5686ffd700 1 -- 172.21.15.40:0/633397072 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5688068a80 con 0x7f56800dcd90 2024-09-13T18:16:34.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.672+0000 7f568e98e700 1 -- 172.21.15.40:0/633397072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56800dcd90 msgr2=0x7f5680006eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:34.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.672+0000 7f568e98e700 1 --2- 172.21.15.40:0/633397072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56800dcd90 0x7f5680006eb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5688063990 tx=0x7f5688069e70 comp rx=0 tx=0).stop 2024-09-13T18:16:34.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.676+0000 7f568e98e700 1 -- 172.21.15.40:0/633397072 shutdown_connections 2024-09-13T18:16:34.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.676+0000 7f568e98e700 1 --2- 172.21.15.40:0/633397072 >> v2:172.21.15.43:3300/0 conn(0x7f56800e6a70 0x7f568004bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:34.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.676+0000 7f568e98e700 1 --2- 172.21.15.40:0/633397072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56800dcd90 0x7f5680006eb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:34.683 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.676+0000 7f568e98e700 1 -- 172.21.15.40:0/633397072 >> 172.21.15.40:0/633397072 conn(0x7f568004fd00 msgr2=0x7f5680050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:34.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.676+0000 7f568e98e700 1 -- 172.21.15.40:0/633397072 shutdown_connections 2024-09-13T18:16:34.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.676+0000 7f568e98e700 1 -- 172.21.15.40:0/633397072 wait complete. 2024-09-13T18:16:34.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.676+0000 7f568e98e700 1 Processor -- start 2024-09-13T18:16:34.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.676+0000 7f568e98e700 1 -- start start 2024-09-13T18:16:34.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.680+0000 7f568e98e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5680044be0 0x7f56800c4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:34.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.680+0000 7f568e98e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56800dcd90 0x7f56800ba870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:34.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.680+0000 7f568e98e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56800e6a70 0x7f5680034e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:34.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.680+0000 7f568e98e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56800d7720 con 0x7f56800dcd90 2024-09-13T18:16:34.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.680+0000 7f568e98e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568004e740 con 0x7f5680044be0 2024-09-13T18:16:34.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.680+0000 7f568e98e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56800d6f00 con 0x7f56800e6a70 2024-09-13T18:16:34.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.680+0000 7f568cf2b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56800e6a70 0x7f5680034e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:34.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.680+0000 7f5687fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5680044be0 0x7f56800c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:34.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.680+0000 7f568cf2b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56800e6a70 0x7f5680034e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:56992/0 (socket says 172.21.15.40:56992) 2024-09-13T18:16:34.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.680+0000 7f568cf2b700 1 -- 172.21.15.40:0/943801251 learned_addr learned my addr 172.21.15.40:0/943801251 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:16:34.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.680+0000 7f568cf2b700 1 -- 172.21.15.40:0/943801251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5680044be0 msgr2=0x7f56800c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:34.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.680+0000 7f568cf2b700 1 --2- 172.21.15.40:0/943801251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5680044be0 0x7f56800c4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:34.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.680+0000 7f568cf2b700 1 -- 172.21.15.40:0/943801251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56800dcd90 msgr2=0x7f56800ba870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:16:34.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.680+0000 7f568cf2b700 1 --2- 172.21.15.40:0/943801251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56800dcd90 0x7f56800ba870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:34.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.680+0000 7f568cf2b700 1 -- 172.21.15.40:0/943801251 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568804f600 con 0x7f56800e6a70 2024-09-13T18:16:34.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.680+0000 7f568cf2b700 1 --2- 172.21.15.40:0/943801251 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56800e6a70 0x7f5680034e00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f567c00b440 tx=0x7f567c00b750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:34.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.692+0000 7f5684ff9700 1 -- 172.21.15.40:0/943801251 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c011830 con 0x7f56800e6a70 2024-09-13T18:16:34.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.692+0000 7f568e98e700 1 -- 172.21.15.40:0/943801251 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56800d71a0 con 0x7f56800e6a70 2024-09-13T18:16:34.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.692+0000 7f568e98e700 1 -- 172.21.15.40:0/943801251 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568004e0f0 con 0x7f56800e6a70 2024-09-13T18:16:34.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.692+0000 7f5684ff9700 1 -- 172.21.15.40:0/943801251 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c0119b0 con 0x7f56800e6a70 2024-09-13T18:16:34.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.692+0000 7f5684ff9700 1 -- 172.21.15.40:0/943801251 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c011d60 con 0x7f56800e6a70 2024-09-13T18:16:34.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.696+0000 7f5684ff9700 1 -- 172.21.15.40:0/943801251 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f567c01a400 con 0x7f56800e6a70 2024-09-13T18:16:34.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.696+0000 7f5684ff9700 1 --2- 172.21.15.40:0/943801251 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f56780669e0 0x7f5678068ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:34.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.696+0000 7f5684ff9700 1 -- 172.21.15.40:0/943801251 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f567c084480 con 0x7f56800e6a70 2024-09-13T18:16:34.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.696+0000 7f568e98e700 1 --2- 172.21.15.40:0/943801251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5668002a40 0x7f5668004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:34.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.696+0000 7f568e98e700 1 -- 172.21.15.40:0/943801251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5668007f40 con 0x7f5668002a40 2024-09-13T18:16:34.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.696+0000 7f5687fff700 1 --2- 172.21.15.40:0/943801251 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f56780669e0 0x7f5678068ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:34.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.696+0000 7f56877fe700 1 --2- 172.21.15.40:0/943801251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5668002a40 0x7f5668004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:34.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.700+0000 7f56877fe700 1 -- 172.21.15.40:0/943801251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5668008100 con 0x7f5668002a40 2024-09-13T18:16:34.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.700+0000 7f56877fe700 1 --2- 172.21.15.40:0/943801251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5668002a40 0x7f5668004f00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f568004d190 tx=0x7f567400b6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:34.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.700+0000 7f5687fff700 1 --2- 172.21.15.40:0/943801251 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f56780669e0 0x7f5678068ea0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f568004d880 tx=0x7f568807c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:34.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.700+0000 7f5684ff9700 1 -- 172.21.15.40:0/943801251 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5674011620 con 0x7f5668002a40 2024-09-13T18:16:34.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.700+0000 7f5684ff9700 1 -- 172.21.15.40:0/943801251 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f5674011780 con 0x7f5668002a40 2024-09-13T18:16:34.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.700+0000 7f5684ff9700 1 -- 172.21.15.40:0/943801251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5668002a40 msgr2=0x7f5668004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:34.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.700+0000 7f5684ff9700 1 --2- 172.21.15.40:0/943801251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5668002a40 0x7f5668004f00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f568004d190 tx=0x7f567400b6a0 comp rx=0 tx=0).stop 2024-09-13T18:16:34.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.700+0000 7f5684ff9700 1 -- 172.21.15.40:0/943801251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5668002a40 msgr2=0x7f5668004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:16:34.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.700+0000 7f5684ff9700 1 --2- 172.21.15.40:0/943801251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5668002a40 0x7f5668004f00 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f568004d190 tx=0x7f567400b6a0 comp rx=0 tx=0).stop 2024-09-13T18:16:34.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.732+0000 7f568e98e700 1 --2- 172.21.15.40:0/943801251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5668006fe0 0x7f5668007470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:34.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.732+0000 7f568e98e700 1 -- 172.21.15.40:0/943801251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566800a400 con 0x7f5668006fe0 2024-09-13T18:16:34.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.732+0000 7f56877fe700 1 --2- 172.21.15.40:0/943801251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5668006fe0 0x7f5668007470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:34.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.732+0000 7f56877fe700 1 -- 172.21.15.40:0/943801251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f566800a5c0 con 0x7f5668006fe0 2024-09-13T18:16:34.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.736+0000 7f56877fe700 1 --2- 172.21.15.40:0/943801251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5668006fe0 0x7f5668007470 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5674009fd0 tx=0x7f5674004820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:34.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.736+0000 7f5684ff9700 1 -- 172.21.15.40:0/943801251 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567401c2d0 con 0x7f5668006fe0 2024-09-13T18:16:34.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.736+0000 7f5684ff9700 1 -- 172.21.15.40:0/943801251 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4451 (secure 0 0 0) 0x7f5674004020 con 0x7f5668006fe0 2024-09-13T18:16:34.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.736+0000 7f5684ff9700 1 -- 172.21.15.40:0/943801251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5668006fe0 msgr2=0x7f5668007470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:34.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.736+0000 7f5684ff9700 1 --2- 172.21.15.40:0/943801251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5668006fe0 0x7f5668007470 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5674009fd0 tx=0x7f5674004820 comp rx=0 tx=0).stop 2024-09-13T18:16:34.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.736+0000 7f5684ff9700 1 -- 172.21.15.40:0/943801251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5668006fe0 msgr2=0x7f5668007470 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:16:34.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.736+0000 7f5684ff9700 1 --2- 172.21.15.40:0/943801251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5668006fe0 0x7f5668007470 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5674009fd0 tx=0x7f5674004820 comp rx=0 tx=0).stop 2024-09-13T18:16:34.744 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:16:34.744 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:16:34.745 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:16:34.745 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:16:34.745 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 210, 2024-09-13T18:16:34.745 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:16:34.745 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:16:34.745 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:16:34.745 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:16:34.745 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:34.745 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 16, 2024-09-13T18:16:34.745 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:16:34.745 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:16:34.745 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:16:34.745 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:34.746 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:34.746 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:34.746 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:34.746 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:34.746 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:34.746 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:34.746 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:34.746 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:34.746 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:34.746 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:34.746 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:16:34.746 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:16:34.746 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:34.746 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:34.747 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:34.747 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:34.747 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:34.747 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:34.747 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:34.747 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:34.747 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:34.747 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:34.747 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:34.747 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.747 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:16:34.747 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:16:34.747 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:16:34.748 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:16:34.748 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:16:34.748 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:16:34.748 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:16:34.748 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:16:34.748 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:16:34.748 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:16:34.748 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:16:34.748 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:16:34.748 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:16:34.748 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:16:34.748 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:16:34.748 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:16:34.748 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:16:34.749 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:34.749 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:34.749 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:34.749 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:34.749 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:34.749 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:34.749 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:34.749 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:34.749 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:34.749 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:34.749 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:34.749 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:16:34.749 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.749 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:16:34.749 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.750 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:16:34.750 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:16:34.750 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:34.750 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:34.750 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.750 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:34.750 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:16:34.750 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:34.750 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.750 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.750 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:34.750 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:16:34.750 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:34.750 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:34.751 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:34.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.751 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:16:34.751 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:16:34.751 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:34.751 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:34.751 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:34.751 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.751 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.751 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:16:34.751 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:16:34.751 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:34.751 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:34.751 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.751 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:34.752 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:16:34.752 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:34.752 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.752 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.752 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:34.752 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:16:34.752 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:34.752 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:34.752 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:34.752 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.752 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:16:34.752 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:16:34.753 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:34.753 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:34.753 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:34.753 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.753 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.753 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:16:34.753 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:16:34.753 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:34.753 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:34.754 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.754 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:34.754 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:16:34.754 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:34.754 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.754 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.754 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:34.754 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:16:34.754 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:34.754 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:34.754 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:34.754 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.754 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:16:34.754 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:16:34.755 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:34.755 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:34.755 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:34.755 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:34.755 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:34.755 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.755 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:16:34.755 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:16:34.755 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.755 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:34.755 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:34.755 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:16:34.756 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:34.756 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:34.756 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:16:34.756 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:34.756 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:34.756 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:34.756 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:16:34.756 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:34.756 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:34.756 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:34.756 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:16:34.756 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:16:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.736+0000 7f568e98e700 1 -- 172.21.15.40:0/943801251 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f56780669e0 msgr2=0x7f5678068ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.736+0000 7f568e98e700 1 --2- 172.21.15.40:0/943801251 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f56780669e0 0x7f5678068ea0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f568004d880 tx=0x7f568807c040 comp rx=0 tx=0).stop 2024-09-13T18:16:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.736+0000 7f568cf2b700 1 -- 172.21.15.40:0/943801251 reap_dead start 2024-09-13T18:16:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.736+0000 7f568e98e700 1 -- 172.21.15.40:0/943801251 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56800e6a70 msgr2=0x7f5680034e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.736+0000 7f568e98e700 1 --2- 172.21.15.40:0/943801251 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56800e6a70 0x7f5680034e00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f567c00b440 tx=0x7f567c00b750 comp rx=0 tx=0).stop 2024-09-13T18:16:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.736+0000 7f568e98e700 1 -- 172.21.15.40:0/943801251 shutdown_connections 2024-09-13T18:16:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.740+0000 7f568e98e700 1 --2- 172.21.15.40:0/943801251 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56800e6a70 0x7f5680034e00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.740+0000 7f568e98e700 1 -- 172.21.15.40:0/943801251 >> 172.21.15.40:0/943801251 conn(0x7f568004fd00 msgr2=0x7f56800465d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.740+0000 7f568e98e700 1 -- 172.21.15.40:0/943801251 shutdown_connections 2024-09-13T18:16:34.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:34.740+0000 7f568e98e700 1 -- 172.21.15.40:0/943801251 wait complete. 2024-09-13T18:16:34.775 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:16:34.775 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:16:34.775 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:16:35.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.192+0000 7ff19ef6c700 1 Processor -- start 2024-09-13T18:16:35.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.192+0000 7ff19ef6c700 1 -- start start 2024-09-13T18:16:35.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19ef6c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1900dcd90 0x7ff190006eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:35.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19ef6c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff1900e6a70 0x7ff19004bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:35.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19ef6c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff190006a20 con 0x7ff190044be0 2024-09-13T18:16:35.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19ef6c700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1900073f0 con 0x7ff1900e6a70 2024-09-13T18:16:35.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19ef6c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff190005ba0 con 0x7ff1900dcd90 2024-09-13T18:16:35.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19d769700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff190044be0 0x7ff19004ad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:38302/0 (socket says 172.21.15.40:38302) 2024-09-13T18:16:35.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19d769700 1 -- 172.21.15.40:0/2849529523 learned_addr learned my addr 172.21.15.40:0/2849529523 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:16:35.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19cf68700 1 -- 172.21.15.40:0/2849529523 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3818649222 0 0) 0x7ff190006a20 con 0x7ff190044be0 2024-09-13T18:16:35.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19e76b700 1 --2- 172.21.15.40:0/2849529523 >> v2:172.21.15.40:3300/0 conn(0x7ff1900e6a70 0x7ff19004bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:35.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19cf68700 1 -- 172.21.15.40:0/2849529523 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1840035d0 con 0x7ff190044be0 2024-09-13T18:16:35.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19cf68700 1 -- 172.21.15.40:0/2849529523 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4204691900 0 0) 0x7ff1840035d0 con 0x7ff190044be0 2024-09-13T18:16:35.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19cf68700 1 -- 172.21.15.40:0/2849529523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1900dcd90 msgr2=0x7ff190006eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:16:35.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19cf68700 1 --2- 172.21.15.40:0/2849529523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1900dcd90 0x7ff190006eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:35.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19cf68700 1 -- 172.21.15.40:0/2849529523 >> v2:172.21.15.40:3300/0 conn(0x7ff1900e6a70 msgr2=0x7ff19004bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:16:35.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19cf68700 1 --2- 172.21.15.40:0/2849529523 >> v2:172.21.15.40:3300/0 conn(0x7ff1900e6a70 0x7ff19004bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:35.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19cf68700 1 -- 172.21.15.40:0/2849529523 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19004b450 con 0x7ff190044be0 2024-09-13T18:16:35.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19cf68700 1 -- 172.21.15.40:0/2849529523 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7ff194001670 con 0x7ff190044be0 2024-09-13T18:16:35.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19cf68700 1 -- 172.21.15.40:0/2849529523 >> v1:172.21.15.40:6789/0 conn(0x7ff190044be0 legacy=0x7ff19004ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:35.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19cf68700 1 --2- 172.21.15.40:0/2849529523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff184003aa0 0x7ff184003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:35.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19cf68700 1 --2- 172.21.15.40:0/2849529523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff184004870 0x7ff184004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:35.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19cf68700 1 --2- 172.21.15.40:0/2849529523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff184005300 0x7ff18400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:35.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19cf68700 1 -- 172.21.15.40:0/2849529523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900073f0 con 0x7ff184005300 2024-09-13T18:16:35.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19cf68700 1 -- 172.21.15.40:0/2849529523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff190005ba0 con 0x7ff184004870 2024-09-13T18:16:35.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19cf68700 1 -- 172.21.15.40:0/2849529523 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18400c290 con 0x7ff184003aa0 2024-09-13T18:16:35.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.196+0000 7ff19e76b700 1 --2- 172.21.15.40:0/2849529523 >> v2:172.21.15.40:3300/0 conn(0x7ff1900e6a70 0x7ff19004bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:16:35.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.200+0000 7ff19d769700 1 --2- 172.21.15.40:0/2849529523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff184004870 0x7ff184004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:35.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.200+0000 7ff19e76b700 1 --2- 172.21.15.40:0/2849529523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff184005300 0x7ff18400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:35.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.200+0000 7ff19d769700 1 -- 172.21.15.40:0/2849529523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff184003aa0 msgr2=0x7ff184003e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:16:35.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.200+0000 7ff19d769700 1 --2- 172.21.15.40:0/2849529523 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff184003aa0 0x7ff184003e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:35.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.200+0000 7ff19d769700 1 -- 172.21.15.40:0/2849529523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff184005300 msgr2=0x7ff18400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:16:35.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.200+0000 7ff19d769700 1 --2- 172.21.15.40:0/2849529523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff184005300 0x7ff18400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:35.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.200+0000 7ff19d769700 1 -- 172.21.15.40:0/2849529523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1840035d0 con 0x7ff184004870 2024-09-13T18:16:35.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.204+0000 7ff19e76b700 1 --2- 172.21.15.40:0/2849529523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff184005300 0x7ff18400ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:16:35.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.204+0000 7ff19d769700 1 --2- 172.21.15.40:0/2849529523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff184004870 0x7ff184004cf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff194004fe0 tx=0x7ff1840058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1807b280e2cdb6b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:35.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.204+0000 7ff19cf68700 1 -- 172.21.15.40:0/2849529523 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194004ad0 con 0x7ff184004870 2024-09-13T18:16:35.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.204+0000 7ff19cf68700 1 -- 172.21.15.40:0/2849529523 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff194005980 con 0x7ff184004870 2024-09-13T18:16:35.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.204+0000 7ff19ef6c700 1 -- 172.21.15.40:0/2849529523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff184004870 msgr2=0x7ff184004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:35.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.204+0000 7ff19ef6c700 1 --2- 172.21.15.40:0/2849529523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff184004870 0x7ff184004cf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff194004fe0 tx=0x7ff1840058a0 comp rx=0 tx=0).stop 2024-09-13T18:16:35.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.204+0000 7ff19e76b700 1 -- 172.21.15.40:0/2849529523 reap_dead start 2024-09-13T18:16:35.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.204+0000 7ff19ef6c700 1 -- 172.21.15.40:0/2849529523 shutdown_connections 2024-09-13T18:16:35.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.204+0000 7ff19ef6c700 1 -- 172.21.15.40:0/2849529523 >> 172.21.15.40:0/2849529523 conn(0x7ff19004fd00 msgr2=0x7ff190050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:35.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.204+0000 7ff19ef6c700 1 -- 172.21.15.40:0/2849529523 shutdown_connections 2024-09-13T18:16:35.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.204+0000 7ff19ef6c700 1 -- 172.21.15.40:0/2849529523 wait complete. 2024-09-13T18:16:35.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.216+0000 7ff19ef6c700 1 Processor -- start 2024-09-13T18:16:35.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.216+0000 7ff19ef6c700 1 -- start start 2024-09-13T18:16:35.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.216+0000 7ff19ef6c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff184004870 0x7ff190047000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:35.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.216+0000 7ff19ef6c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1900485b0 0x7ff190035390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:35.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.216+0000 7ff19ef6c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff19017ca60 0x7ff19017cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:35.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.216+0000 7ff19ef6c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff190178e30 con 0x7ff1900485b0 2024-09-13T18:16:35.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.216+0000 7ff19ef6c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900476b0 con 0x7ff184004870 2024-09-13T18:16:35.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.216+0000 7ff19ef6c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19016da30 con 0x7ff19017ca60 2024-09-13T18:16:35.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.216+0000 7ff19d769700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1900485b0 0x7ff190035390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:35.224 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.216+0000 7ff19d769700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1900485b0 0x7ff190035390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59344/0 (socket says 172.21.15.40:59344) 2024-09-13T18:16:35.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.216+0000 7ff19d769700 1 -- 172.21.15.40:0/4287654071 learned_addr learned my addr 172.21.15.40:0/4287654071 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:16:35.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.216+0000 7ff19d769700 1 -- 172.21.15.40:0/4287654071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff19017ca60 msgr2=0x7ff19017cf10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:16:35.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.216+0000 7ff19d769700 1 --2- 172.21.15.40:0/4287654071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff19017ca60 0x7ff19017cf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:35.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.216+0000 7ff19d769700 1 -- 172.21.15.40:0/4287654071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff184004870 msgr2=0x7ff190047000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:16:35.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.216+0000 7ff19d769700 1 --2- 172.21.15.40:0/4287654071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff184004870 0x7ff190047000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:35.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.216+0000 7ff19d769700 1 -- 172.21.15.40:0/4287654071 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19016dd10 con 0x7ff1900485b0 2024-09-13T18:16:35.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.216+0000 7ff19d769700 1 --2- 172.21.15.40:0/4287654071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1900485b0 0x7ff190035390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff18400c260 tx=0x7ff194001de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:35.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.216+0000 7ff18e7fc700 1 -- 172.21.15.40:0/4287654071 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194004ad0 con 0x7ff1900485b0 2024-09-13T18:16:35.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.216+0000 7ff18e7fc700 1 -- 172.21.15.40:0/4287654071 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff194005e90 con 0x7ff1900485b0 2024-09-13T18:16:35.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.216+0000 7ff18e7fc700 1 -- 172.21.15.40:0/4287654071 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194008520 con 0x7ff1900485b0 2024-09-13T18:16:35.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.220+0000 7ff19ef6c700 1 -- 172.21.15.40:0/4287654071 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19016dff0 con 0x7ff1900485b0 2024-09-13T18:16:35.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.220+0000 7ff19ef6c700 1 -- 172.21.15.40:0/4287654071 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff190045be0 con 0x7ff1900485b0 2024-09-13T18:16:35.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.220+0000 7ff18e7fc700 1 -- 172.21.15.40:0/4287654071 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff194008680 con 0x7ff1900485b0 2024-09-13T18:16:35.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.220+0000 7ff18e7fc700 1 --2- 172.21.15.40:0/4287654071 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff17c0667b0 0x7ff17c068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:35.229 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.220+0000 7ff18e7fc700 1 -- 172.21.15.40:0/4287654071 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff1940a6ab0 con 0x7ff1900485b0 2024-09-13T18:16:35.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.224+0000 7ff19df6a700 1 --2- 172.21.15.40:0/4287654071 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff17c0667b0 0x7ff17c068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:35.230 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.224+0000 7ff19df6a700 1 --2- 172.21.15.40:0/4287654071 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff17c0667b0 0x7ff17c068c70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff198063cf0 tx=0x7ff198065ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:35.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.224+0000 7ff17bfff700 1 --2- 172.21.15.40:0/4287654071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff19004b630 0x7ff190002970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:35.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.228+0000 7ff17bfff700 1 -- 172.21.15.40:0/4287654071 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19016dea0 con 0x7ff19004b630 2024-09-13T18:16:35.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.228+0000 7ff19e76b700 1 --2- 172.21.15.40:0/4287654071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff19004b630 0x7ff190002970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:35.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.228+0000 7ff19e76b700 1 -- 172.21.15.40:0/4287654071 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1900063d0 con 0x7ff19004b630 2024-09-13T18:16:35.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.228+0000 7ff19e76b700 1 --2- 172.21.15.40:0/4287654071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff19004b630 0x7ff190002970 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff19000d590 tx=0x7ff18800f7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:35.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.228+0000 7ff18e7fc700 1 -- 172.21.15.40:0/4287654071 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18800fe10 con 0x7ff19004b630 2024-09-13T18:16:35.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.228+0000 7ff18e7fc700 1 -- 172.21.15.40:0/4287654071 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff18800c030 con 0x7ff19004b630 2024-09-13T18:16:35.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.228+0000 7ff18e7fc700 1 -- 172.21.15.40:0/4287654071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff19004b630 msgr2=0x7ff190002970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:35.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.228+0000 7ff18e7fc700 1 --2- 172.21.15.40:0/4287654071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff19004b630 0x7ff190002970 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff19000d590 tx=0x7ff18800f7a0 comp rx=0 tx=0).stop 2024-09-13T18:16:35.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.228+0000 7ff18e7fc700 1 -- 172.21.15.40:0/4287654071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff19004b630 msgr2=0x7ff190002970 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:16:35.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.228+0000 7ff18e7fc700 1 --2- 172.21.15.40:0/4287654071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff19004b630 0x7ff190002970 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:35.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.244+0000 7ff17bfff700 1 --2- 172.21.15.40:0/4287654071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1900d24a0 0x7ff190002eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:35.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.244+0000 7ff17bfff700 1 -- 172.21.15.40:0/4287654071 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900084b0 con 0x7ff1900d24a0 2024-09-13T18:16:35.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.244+0000 7ff19e76b700 1 --2- 172.21.15.40:0/4287654071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1900d24a0 0x7ff190002eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:35.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.244+0000 7ff19e76b700 1 -- 172.21.15.40:0/4287654071 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff1900060f0 con 0x7ff1900d24a0 2024-09-13T18:16:35.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.248+0000 7ff19e76b700 1 --2- 172.21.15.40:0/4287654071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1900d24a0 0x7ff190002eb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff19017ce90 tx=0x7ff188004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:35.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.248+0000 7ff18e7fc700 1 -- 172.21.15.40:0/4287654071 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1880045c0 con 0x7ff1900d24a0 2024-09-13T18:16:35.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.252+0000 7ff18e7fc700 1 -- 172.21.15.40:0/4287654071 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4949 (secure 0 0 0) 0x7ff188005a40 con 0x7ff1900d24a0 2024-09-13T18:16:35.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.252+0000 7ff18e7fc700 1 -- 172.21.15.40:0/4287654071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1900d24a0 msgr2=0x7ff190002eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:35.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.252+0000 7ff18e7fc700 1 --2- 172.21.15.40:0/4287654071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1900d24a0 0x7ff190002eb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff19017ce90 tx=0x7ff188004770 comp rx=0 tx=0).stop 2024-09-13T18:16:35.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.252+0000 7ff18e7fc700 1 -- 172.21.15.40:0/4287654071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1900d24a0 msgr2=0x7ff190002eb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:16:35.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.252+0000 7ff18e7fc700 1 --2- 172.21.15.40:0/4287654071 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1900d24a0 0x7ff190002eb0 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff19017ce90 tx=0x7ff188004770 comp rx=0 tx=0).stop 2024-09-13T18:16:35.258 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:16:35.259 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:16:35.259 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:16:35.259 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:16:35.259 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 210, 2024-09-13T18:16:35.259 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:16:35.259 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:16:35.259 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:16:35.259 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:16:35.259 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:35.259 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 17, 2024-09-13T18:16:35.259 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:16:35.259 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:16:35.259 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:16:35.259 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:35.260 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:35.260 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:35.260 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:35.260 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:35.260 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:35.260 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:35.260 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:35.260 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:35.260 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:35.260 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:35.260 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:16:35.260 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:16:35.260 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:35.261 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:35.261 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:35.261 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:35.261 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:35.261 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:35.261 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:35.261 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:35.261 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:35.261 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:35.261 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:35.261 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:35.261 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:16:35.261 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:16:35.261 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:16:35.262 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:16:35.262 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:16:35.262 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:16:35.262 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:16:35.262 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:16:35.262 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:16:35.262 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:16:35.262 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:16:35.262 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:16:35.262 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:16:35.262 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:16:35.262 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:16:35.262 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:16:35.262 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:16:35.262 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:35.263 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:35.263 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:35.263 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:35.263 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:35.263 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:35.263 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:35.263 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:35.263 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:35.263 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:35.263 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:35.263 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:16:35.263 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:35.263 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:16:35.263 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:35.264 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:16:35.264 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:16:35.264 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:35.264 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:35.264 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:35.264 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:35.264 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:16:35.264 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:35.264 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:35.264 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:35.264 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:35.264 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:16:35.264 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:35.264 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:35.265 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:35.265 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:35.265 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:16:35.265 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:16:35.265 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:35.268 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:35.268 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:35.268 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:35.268 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:35.268 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:16:35.268 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:16:35.268 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:35.268 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:35.269 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:35.269 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:35.269 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:16:35.269 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:35.269 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:35.269 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:35.269 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:35.269 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:16:35.269 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:35.269 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:35.269 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:35.269 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:35.269 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:16:35.269 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:16:35.270 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:35.270 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:35.270 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:35.270 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:35.270 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:35.270 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:16:35.270 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:16:35.270 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:35.270 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:35.270 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:35.270 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:35.270 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:16:35.270 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:35.270 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:35.270 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:35.271 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:35.271 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:16:35.271 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:35.271 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:35.271 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:35.271 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:35.271 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:16:35.271 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:16:35.271 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:35.271 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:35.271 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:35.271 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:35.271 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:35.271 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:35.272 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:16:35.272 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:16:35.272 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:35.272 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:35.272 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:35.272 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:16:35.272 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:35.272 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:35.272 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:16:35.272 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:35.272 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:35.272 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:35.272 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:16:35.272 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:35.273 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:35.273 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:16:35.273 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:35.273 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:35.273 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:35.273 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 8 2024-09-13T18:16:35.273 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:35.273 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:35.273 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:16:35.273 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:35.273 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:35.273 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:35.273 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:16:35.273 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:35.273 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:35.274 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:16:35.274 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:35.274 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:35.274 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:35.274 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:16:35.274 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:35.274 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:35.274 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:35.274 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:16:35.274 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:16:35.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.260+0000 7ff17bfff700 1 -- 172.21.15.40:0/4287654071 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff17c0667b0 msgr2=0x7ff17c068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:35.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.260+0000 7ff17bfff700 1 --2- 172.21.15.40:0/4287654071 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff17c0667b0 0x7ff17c068c70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff198063cf0 tx=0x7ff198065ea0 comp rx=0 tx=0).stop 2024-09-13T18:16:35.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.260+0000 7ff19e76b700 1 -- 172.21.15.40:0/4287654071 reap_dead start 2024-09-13T18:16:35.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.260+0000 7ff17bfff700 1 -- 172.21.15.40:0/4287654071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1900485b0 msgr2=0x7ff190035390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:35.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.260+0000 7ff17bfff700 1 --2- 172.21.15.40:0/4287654071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1900485b0 0x7ff190035390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff18400c260 tx=0x7ff194001de0 comp rx=0 tx=0).stop 2024-09-13T18:16:35.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.264+0000 7ff17bfff700 1 -- 172.21.15.40:0/4287654071 shutdown_connections 2024-09-13T18:16:35.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.264+0000 7ff17bfff700 1 --2- 172.21.15.40:0/4287654071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1900485b0 0x7ff190035390 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:35.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.264+0000 7ff17bfff700 1 -- 172.21.15.40:0/4287654071 >> 172.21.15.40:0/4287654071 conn(0x7ff19004fd00 msgr2=0x7ff1900450c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:35.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.264+0000 7ff17bfff700 1 -- 172.21.15.40:0/4287654071 shutdown_connections 2024-09-13T18:16:35.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:35.264+0000 7ff17bfff700 1 -- 172.21.15.40:0/4287654071 wait complete. 2024-09-13T18:16:35.327 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:16:35.327 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-13T18:16:35.328 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['b'] 2024-09-13T18:16:35.328 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-13T18:16:35.328 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-13T18:16:35.328 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T18:16:35.328 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:16:35.382 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T18:16:35.382 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-13T18:16:35.383 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-13T18:16:35.383 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-13T18:16:39.311 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:16:39.313 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:16:50.384 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-13T18:16:50.384 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-13T18:16:50.384 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T18:16:50.384 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:16:52.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.140+0000 7f433d53c700 1 Processor -- start 2024-09-13T18:16:52.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.140+0000 7f433d53c700 1 -- start start 2024-09-13T18:16:52.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.144+0000 7f433d53c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4338160390 0x7f43381669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:52.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.144+0000 7f433d53c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 0x7f4338167600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:52.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.144+0000 7f433d53c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4338071930 con 0x7f4338167b40 2024-09-13T18:16:52.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.144+0000 7f433d53c700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4338166f20 con 0x7f43380704e0 2024-09-13T18:16:52.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.144+0000 7f433d53c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4338167fd0 con 0x7f4338160390 2024-09-13T18:16:52.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.144+0000 7f4336ffd700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4338160390 msgr2=0x7f43381669e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:16:52.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.144+0000 7f4336ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4338160390 0x7f43381669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:16:52.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.144+0000 7f43367fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 0x7f4338167600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:52.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.144+0000 7f43367fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 0x7f4338167600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55840/0 (socket says 172.21.15.40:55840) 2024-09-13T18:16:52.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.144+0000 7f43367fc700 1 -- 172.21.15.40:0/1701306612 learned_addr learned my addr 172.21.15.40:0/1701306612 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:16:52.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.144+0000 7f43367fc700 1 -- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 msgr2=0x7f4338167600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-09-13T18:16:52.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.144+0000 7f43367fc700 1 -- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 msgr2=0x7f4338167600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:16:52.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.144+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 0x7f4338167600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:16:52.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.144+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 0x7f4338167600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:16:52.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.348+0000 7f4336ffd700 1 -- 172.21.15.40:0/1701306612 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4338160390 msgr2=0x7f43381669e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:16:52.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.348+0000 7f4336ffd700 1 --2- 172.21.15.40:0/1701306612 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4338160390 0x7f43381669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:16:52.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.348+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 0x7f4338167600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:52.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.348+0000 7f43367fc700 1 -- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 msgr2=0x7f4338167600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:16:52.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.348+0000 7f43367fc700 1 -- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 msgr2=0x7f4338167600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:16:52.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.348+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 0x7f4338167600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:16:52.355 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.348+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 0x7f4338167600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:16:52.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.752+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 0x7f4338167600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:52.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.752+0000 7f43367fc700 1 -- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 msgr2=0x7f4338167600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:16:52.758 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.752+0000 7f43367fc700 1 -- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 msgr2=0x7f4338167600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:16:52.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.752+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 0x7f4338167600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:16:52.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.752+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 0x7f4338167600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:16:52.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.752+0000 7f4336ffd700 1 -- 172.21.15.40:0/1701306612 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4338160390 msgr2=0x7f43381669e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:16:52.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:52.752+0000 7f4336ffd700 1 --2- 172.21.15.40:0/1701306612 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4338160390 0x7f43381669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:16:53.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:53.556+0000 7f4336ffd700 1 -- 172.21.15.40:0/1701306612 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4338160390 msgr2=0x7f43381669e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:16:53.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:53.556+0000 7f4336ffd700 1 --2- 172.21.15.40:0/1701306612 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4338160390 0x7f43381669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:16:53.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:53.556+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 0x7f4338167600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:53.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:53.556+0000 7f43367fc700 1 -- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 msgr2=0x7f4338167600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:16:53.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:53.556+0000 7f43367fc700 1 -- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 msgr2=0x7f4338167600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:16:53.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:53.556+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 0x7f4338167600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:16:53.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:53.556+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 0x7f4338167600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:16:55.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.144+0000 7f4334ff9700 1 -- 172.21.15.40:0/1701306612 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4338160390 msgr2=0x7f43381669e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:16:55.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.144+0000 7f4334ff9700 1 --2- 172.21.15.40:0/1701306612 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4338160390 0x7f43381669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:55.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.144+0000 7f4334ff9700 1 -- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 msgr2=0x7f4338167600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:16:55.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.144+0000 7f4334ff9700 1 --2- 172.21.15.40:0/1701306612 >> v2:172.21.15.40:3300/0 conn(0x7f43380704e0 0x7f4338167600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:55.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.144+0000 7f4334ff9700 1 -- 172.21.15.40:0/1701306612 >> v1:172.21.15.43:6789/0 conn(0x7f4338167b40 legacy=0x7f433806a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:55.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.144+0000 7f4334ff9700 1 --2- 172.21.15.40:0/1701306612 >> v2:172.21.15.43:3300/0 conn(0x7f4324007480 0x7f4324009940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:55.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.144+0000 7f4334ff9700 1 -- 172.21.15.40:0/1701306612 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4338070a40 con 0x7f4324003740 2024-09-13T18:16:55.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.144+0000 7f4334ff9700 1 -- 172.21.15.40:0/1701306612 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4338166f20 con 0x7f4324007480 2024-09-13T18:16:55.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.144+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> v2:172.21.15.43:3300/0 conn(0x7f4324007480 0x7f4324009940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:55.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.144+0000 7f43367fc700 1 -- 172.21.15.40:0/1701306612 >> v2:172.21.15.43:3300/0 conn(0x7f4324007480 msgr2=0x7f4324009940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:16:55.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.144+0000 7f43367fc700 1 -- 172.21.15.40:0/1701306612 >> v2:172.21.15.43:3300/0 conn(0x7f4324007480 msgr2=0x7f4324009940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:16:55.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.144+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> v2:172.21.15.43:3300/0 conn(0x7f4324007480 0x7f4324009940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:16:55.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.144+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> v2:172.21.15.43:3300/0 conn(0x7f4324007480 0x7f4324009940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:16:55.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.352+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> v2:172.21.15.43:3300/0 conn(0x7f4324007480 0x7f4324009940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:55.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.352+0000 7f43367fc700 1 -- 172.21.15.40:0/1701306612 >> v2:172.21.15.43:3300/0 conn(0x7f4324007480 msgr2=0x7f4324009940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:16:55.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.352+0000 7f43367fc700 1 -- 172.21.15.40:0/1701306612 >> v2:172.21.15.43:3300/0 conn(0x7f4324007480 msgr2=0x7f4324009940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:16:55.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.352+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> v2:172.21.15.43:3300/0 conn(0x7f4324007480 0x7f4324009940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:16:55.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.352+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> v2:172.21.15.43:3300/0 conn(0x7f4324007480 0x7f4324009940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:16:55.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f4335ffb700 1 -- 172.21.15.40:0/1701306612 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 27312289 0 0) 0x7f4338070a40 con 0x7f4324003740 2024-09-13T18:16:55.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f4335ffb700 1 -- 172.21.15.40:0/1701306612 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43200035d0 con 0x7f4324003740 2024-09-13T18:16:55.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f4335ffb700 1 -- 172.21.15.40:0/1701306612 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1267577889 0 0) 0x7f43200035d0 con 0x7f4324003740 2024-09-13T18:16:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f4335ffb700 1 -- 172.21.15.40:0/1701306612 >> v2:172.21.15.43:3300/0 conn(0x7f4324007480 msgr2=0x7f4324009940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:16:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f4335ffb700 1 --2- 172.21.15.40:0/1701306612 >> v2:172.21.15.43:3300/0 conn(0x7f4324007480 0x7f4324009940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f4335ffb700 1 -- 172.21.15.40:0/1701306612 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432400b040 con 0x7f4324003740 2024-09-13T18:16:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f4335ffb700 1 -- 172.21.15.40:0/1701306612 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f43280043f0 con 0x7f4324003740 2024-09-13T18:16:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f4335ffb700 1 -- 172.21.15.40:0/1701306612 >> v1:172.21.15.40:6789/0 conn(0x7f4324003740 legacy=0x7f4324005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f4335ffb700 1 --2- 172.21.15.40:0/1701306612 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4338160390 0x7f4320003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f43377fe700 1 -- 172.21.15.40:0/1701306612 reap_dead start 2024-09-13T18:16:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f4335ffb700 1 --2- 172.21.15.40:0/1701306612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320004050 0x7f4320004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f4335ffb700 1 --2- 172.21.15.40:0/1701306612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4320004db0 0x7f4320005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f4335ffb700 1 -- 172.21.15.40:0/1701306612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4338166f20 con 0x7f4320004db0 2024-09-13T18:16:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f4335ffb700 1 -- 172.21.15.40:0/1701306612 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4320014400 con 0x7f4320004050 2024-09-13T18:16:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f4335ffb700 1 -- 172.21.15.40:0/1701306612 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43200145c0 con 0x7f4338160390 2024-09-13T18:16:55.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320004050 0x7f4320004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:55.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f43377fe700 1 --2- 172.21.15.40:0/1701306612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4320004db0 0x7f4320005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:55.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f4336ffd700 1 -- 172.21.15.40:0/1701306612 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4338160390 msgr2=0x7f4320003a20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:16:55.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f4336ffd700 1 --2- 172.21.15.40:0/1701306612 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4338160390 0x7f4320003a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:16:55.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f43367fc700 1 -- 172.21.15.40:0/1701306612 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4338160390 msgr2=0x7f4320003a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:16:55.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4338160390 0x7f4320003a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:55.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f43367fc700 1 -- 172.21.15.40:0/1701306612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4320004db0 msgr2=0x7f4320005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:16:55.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4320004db0 0x7f4320005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:55.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f43367fc700 1 -- 172.21.15.40:0/1701306612 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432400b040 con 0x7f4320004050 2024-09-13T18:16:55.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.416+0000 7f43367fc700 1 --2- 172.21.15.40:0/1701306612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320004050 0x7f4320004870 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f43300124e0 tx=0x7f43200058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ce8680c486ee59b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:55.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.436+0000 7f4335ffb700 1 -- 172.21.15.40:0/1701306612 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433000f420 con 0x7f4320004050 2024-09-13T18:16:55.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.436+0000 7f4335ffb700 1 -- 172.21.15.40:0/1701306612 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433000f5a0 con 0x7f4320004050 2024-09-13T18:16:55.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.436+0000 7f4335ffb700 1 -- 172.21.15.40:0/1701306612 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433000f950 con 0x7f4320004050 2024-09-13T18:16:55.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.436+0000 7f433d53c700 1 -- 172.21.15.40:0/1701306612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320004050 msgr2=0x7f4320004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:55.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.436+0000 7f433d53c700 1 --2- 172.21.15.40:0/1701306612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320004050 0x7f4320004870 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f43300124e0 tx=0x7f43200058a0 comp rx=0 tx=0).stop 2024-09-13T18:16:55.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.436+0000 7f433d53c700 1 -- 172.21.15.40:0/1701306612 shutdown_connections 2024-09-13T18:16:55.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.436+0000 7f433d53c700 1 --2- 172.21.15.40:0/1701306612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320004050 0x7f4320004870 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:55.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.436+0000 7f433d53c700 1 --2- 172.21.15.40:0/1701306612 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4338160390 0x7f4320003a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:55.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.436+0000 7f433d53c700 1 --2- 172.21.15.40:0/1701306612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4320004db0 0x7f4320005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:55.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.436+0000 7f433d53c700 1 -- 172.21.15.40:0/1701306612 >> 172.21.15.40:0/1701306612 conn(0x7f43380cabc0 msgr2=0x7f43380cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:55.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.436+0000 7f433d53c700 1 -- 172.21.15.40:0/1701306612 shutdown_connections 2024-09-13T18:16:55.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.436+0000 7f433d53c700 1 -- 172.21.15.40:0/1701306612 wait complete. 2024-09-13T18:16:55.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.440+0000 7f433d53c700 1 Processor -- start 2024-09-13T18:16:55.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.440+0000 7f433d53c700 1 -- start start 2024-09-13T18:16:55.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.440+0000 7f433d53c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320004050 0x7f4338163240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:55.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.440+0000 7f433d53c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4320004db0 0x7f43380c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:55.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.440+0000 7f433d53c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4338160390 0x7f43380c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:55.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.440+0000 7f433d53c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4338162830 con 0x7f4338160390 2024-09-13T18:16:55.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.440+0000 7f433d53c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4338161fd0 con 0x7f4320004050 2024-09-13T18:16:55.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.440+0000 7f433d53c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43380c6ac0 con 0x7f4320004db0 2024-09-13T18:16:55.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.440+0000 7f4336ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320004050 0x7f4338163240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:55.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.440+0000 7f4336ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320004050 0x7f4338163240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:55290/0 (socket says 172.21.15.40:55290) 2024-09-13T18:16:55.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.440+0000 7f4336ffd700 1 -- 172.21.15.40:0/813296191 learned_addr learned my addr 172.21.15.40:0/813296191 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:16:55.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.440+0000 7f43367fc700 1 -- 172.21.15.40:0/813296191 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4320004db0 msgr2=0x7f43380c7180 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:16:55.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.440+0000 7f43367fc700 1 --2- 172.21.15.40:0/813296191 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4320004db0 0x7f43380c7180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:16:55.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.440+0000 7f4336ffd700 1 -- 172.21.15.40:0/813296191 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4320004db0 msgr2=0x7f43380c7180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:16:55.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.440+0000 7f4336ffd700 1 --2- 172.21.15.40:0/813296191 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4320004db0 0x7f43380c7180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:55.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.444+0000 7f4336ffd700 1 -- 172.21.15.40:0/813296191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4338160390 msgr2=0x7f43380c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:55.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.444+0000 7f4336ffd700 1 --2- 172.21.15.40:0/813296191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4338160390 0x7f43380c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:55.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.444+0000 7f4336ffd700 1 -- 172.21.15.40:0/813296191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43380c5940 con 0x7f4320004050 2024-09-13T18:16:55.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.444+0000 7f4336ffd700 1 --2- 172.21.15.40:0/813296191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320004050 0x7f4338163240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4328003b70 tx=0x7f432800de40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:55.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.452+0000 7f431f7fe700 1 -- 172.21.15.40:0/813296191 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432800ba60 con 0x7f4320004050 2024-09-13T18:16:55.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.452+0000 7f433d53c700 1 -- 172.21.15.40:0/813296191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43380c53a0 con 0x7f4320004050 2024-09-13T18:16:55.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.452+0000 7f433d53c700 1 -- 172.21.15.40:0/813296191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43381621e0 con 0x7f4320004050 2024-09-13T18:16:55.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.452+0000 7f431f7fe700 1 -- 172.21.15.40:0/813296191 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432800bbc0 con 0x7f4320004050 2024-09-13T18:16:55.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.452+0000 7f431f7fe700 1 -- 172.21.15.40:0/813296191 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328013440 con 0x7f4320004050 2024-09-13T18:16:55.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.452+0000 7f431f7fe700 1 -- 172.21.15.40:0/813296191 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4328005020 con 0x7f4320004050 2024-09-13T18:16:55.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.452+0000 7f431f7fe700 1 --2- 172.21.15.40:0/813296191 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4324067380 0x7f4324069840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:55.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.452+0000 7f431f7fe700 1 -- 172.21.15.40:0/813296191 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f432800f040 con 0x7f4320004050 2024-09-13T18:16:55.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.456+0000 7f43367fc700 1 --2- 172.21.15.40:0/813296191 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4324067380 0x7f4324069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:55.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.456+0000 7f431d7fa700 1 -- 172.21.15.40:0/813296191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4338072870 con 0x7f4320004050 2024-09-13T18:16:55.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.456+0000 7f43367fc700 1 --2- 172.21.15.40:0/813296191 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4324067380 0x7f4324069840 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f43380c4e90 tx=0x7f4330017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:55.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:55.460+0000 7f431f7fe700 1 -- 172.21.15.40:0/813296191 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f43280512d0 con 0x7f4320004050 2024-09-13T18:16:56.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.388+0000 7f431d7fa700 1 -- 172.21.15.40:0/813296191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4338167600 con 0x7f4320004050 2024-09-13T18:16:56.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.388+0000 7f431f7fe700 1 -- 172.21.15.40:0/813296191 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f43280548f0 con 0x7f4320004050 2024-09-13T18:16:56.396 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":212,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:16:56.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.392+0000 7f433d53c700 1 -- 172.21.15.40:0/813296191 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4324067380 msgr2=0x7f4324069840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:56.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.392+0000 7f433d53c700 1 --2- 172.21.15.40:0/813296191 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4324067380 0x7f4324069840 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f43380c4e90 tx=0x7f4330017000 comp rx=0 tx=0).stop 2024-09-13T18:16:56.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.392+0000 7f433d53c700 1 -- 172.21.15.40:0/813296191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320004050 msgr2=0x7f4338163240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:56.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.392+0000 7f433d53c700 1 --2- 172.21.15.40:0/813296191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320004050 0x7f4338163240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4328003b70 tx=0x7f432800de40 comp rx=0 tx=0).stop 2024-09-13T18:16:56.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.392+0000 7f433d53c700 1 -- 172.21.15.40:0/813296191 shutdown_connections 2024-09-13T18:16:56.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.392+0000 7f433d53c700 1 --2- 172.21.15.40:0/813296191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4338160390 0x7f43380c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:56.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.392+0000 7f433d53c700 1 --2- 172.21.15.40:0/813296191 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4324067380 0x7f4324069840 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:56.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.392+0000 7f433d53c700 1 --2- 172.21.15.40:0/813296191 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4320004db0 0x7f43380c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:56.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.392+0000 7f433d53c700 1 --2- 172.21.15.40:0/813296191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4320004050 0x7f4338163240 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:56.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.392+0000 7f433d53c700 1 -- 172.21.15.40:0/813296191 >> 172.21.15.40:0/813296191 conn(0x7f43380cabc0 msgr2=0x7f43380ca1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:56.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.392+0000 7f433d53c700 1 -- 172.21.15.40:0/813296191 shutdown_connections 2024-09-13T18:16:56.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.392+0000 7f433d53c700 1 -- 172.21.15.40:0/813296191 wait complete. 2024-09-13T18:16:56.421 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T18:16:56.421 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-13T18:16:56.421 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:16:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.788+0000 7f3db9e05700 1 Processor -- start 2024-09-13T18:16:56.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.788+0000 7f3db9e05700 1 -- start start 2024-09-13T18:16:56.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.804+0000 7f3db9e05700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3db415e4e0 0x7f3db40cb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:56.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.804+0000 7f3db9e05700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db406d270 con 0x7f3db4161560 2024-09-13T18:16:56.812 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.804+0000 7f3db9e05700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db41610d0 con 0x7f3db4161980 2024-09-13T18:16:56.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.804+0000 7f3db9e05700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db40cb900 con 0x7f3db415e4e0 2024-09-13T18:16:56.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.804+0000 7f3db37fe700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3db415e4e0 msgr2=0x7f3db40cb200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:16:56.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.804+0000 7f3db37fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3db415e4e0 0x7f3db40cb200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:16:56.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.804+0000 7f3db2ffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3db4161560 0x7f3db40c9f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33548/0 (socket says 172.21.15.40:33548) 2024-09-13T18:16:56.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.804+0000 7f3db2ffd700 1 -- 172.21.15.40:0/2946764664 learned_addr learned my addr 172.21.15.40:0/2946764664 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:16:56.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.804+0000 7f3db27fc700 1 -- 172.21.15.40:0/2946764664 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3110256776 0 0) 0x7f3db406d270 con 0x7f3db4161560 2024-09-13T18:16:56.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.804+0000 7f3db27fc700 1 -- 172.21.15.40:0/2946764664 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d9c0035d0 con 0x7f3db4161560 2024-09-13T18:16:56.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.804+0000 7f3db27fc700 1 -- 172.21.15.40:0/2946764664 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4292607827 0 0) 0x7f3db41610d0 con 0x7f3db4161980 2024-09-13T18:16:56.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.804+0000 7f3db27fc700 1 -- 172.21.15.40:0/2946764664 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3db406d270 con 0x7f3db4161980 2024-09-13T18:16:56.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.808+0000 7f3db27fc700 1 -- 172.21.15.40:0/2946764664 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3160096127 0 0) 0x7f3d9c0035d0 con 0x7f3db4161560 2024-09-13T18:16:56.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.808+0000 7f3db27fc700 1 -- 172.21.15.40:0/2946764664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3db415e4e0 msgr2=0x7f3db40cb200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:16:56.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.808+0000 7f3db27fc700 1 --2- 172.21.15.40:0/2946764664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3db415e4e0 0x7f3db40cb200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:56.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.808+0000 7f3db27fc700 1 -- 172.21.15.40:0/2946764664 >> v1:172.21.15.43:6789/0 conn(0x7f3db4161980 legacy=0x7f3db4168240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:56.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.808+0000 7f3db27fc700 1 -- 172.21.15.40:0/2946764664 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db4161e30 con 0x7f3db4161560 2024-09-13T18:16:56.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.808+0000 7f3db27fc700 1 -- 172.21.15.40:0/2946764664 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f3da80015a0 con 0x7f3db4161560 2024-09-13T18:16:56.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.808+0000 7f3db27fc700 1 -- 172.21.15.40:0/2946764664 >> v1:172.21.15.40:6789/0 conn(0x7f3db4161560 legacy=0x7f3db40c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:56.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.808+0000 7f3db27fc700 1 --2- 172.21.15.40:0/2946764664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d9c004c70 0x7f3d9c005050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:56.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.808+0000 7f3db37fe700 1 --2- 172.21.15.40:0/2946764664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d9c004c70 0x7f3d9c005050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:56.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.808+0000 7f3db27fc700 1 --2- 172.21.15.40:0/2946764664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3db415e4e0 0x7f3d9c003810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:56.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.808+0000 7f3db27fc700 1 --2- 172.21.15.40:0/2946764664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d9c003e70 0x7f3d9c0042f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:56.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.808+0000 7f3db2ffd700 1 -- 172.21.15.40:0/2946764664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3db415e4e0 msgr2=0x7f3d9c003810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:16:56.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.808+0000 7f3db2ffd700 1 --2- 172.21.15.40:0/2946764664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3db415e4e0 0x7f3d9c003810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:16:56.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.808+0000 7f3db27fc700 1 -- 172.21.15.40:0/2946764664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dac001440 con 0x7f3d9c004c70 2024-09-13T18:16:56.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.808+0000 7f3db27fc700 1 -- 172.21.15.40:0/2946764664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db40cb900 con 0x7f3d9c003e70 2024-09-13T18:16:56.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.808+0000 7f3db3fff700 1 --2- 172.21.15.40:0/2946764664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d9c003e70 0x7f3d9c0042f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:56.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.808+0000 7f3db27fc700 1 -- 172.21.15.40:0/2946764664 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c00bd90 con 0x7f3db415e4e0 2024-09-13T18:16:56.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.808+0000 7f3db37fe700 1 -- 172.21.15.40:0/2946764664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3db415e4e0 msgr2=0x7f3d9c003810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:16:56.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.808+0000 7f3db37fe700 1 --2- 172.21.15.40:0/2946764664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3db415e4e0 0x7f3d9c003810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:56.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.812+0000 7f3db37fe700 1 -- 172.21.15.40:0/2946764664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d9c003e70 msgr2=0x7f3d9c0042f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:16:56.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.812+0000 7f3db37fe700 1 --2- 172.21.15.40:0/2946764664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d9c003e70 0x7f3d9c0042f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:56.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.812+0000 7f3db37fe700 1 -- 172.21.15.40:0/2946764664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c0035d0 con 0x7f3d9c004c70 2024-09-13T18:16:56.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.812+0000 7f3db3fff700 1 --2- 172.21.15.40:0/2946764664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d9c003e70 0x7f3d9c0042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:16:56.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.812+0000 7f3db37fe700 1 --2- 172.21.15.40:0/2946764664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d9c004c70 0x7f3d9c005050 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3da400b3e0 tx=0x7f3d9c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ab46ee56d09a52 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:56.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.812+0000 7f3db27fc700 1 -- 172.21.15.40:0/2946764664 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da40098d0 con 0x7f3d9c004c70 2024-09-13T18:16:56.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.812+0000 7f3db27fc700 1 -- 172.21.15.40:0/2946764664 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da4003a60 con 0x7f3d9c004c70 2024-09-13T18:16:56.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.812+0000 7f3db9e05700 1 -- 172.21.15.40:0/2946764664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d9c004c70 msgr2=0x7f3d9c005050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:56.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.812+0000 7f3db9e05700 1 --2- 172.21.15.40:0/2946764664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d9c004c70 0x7f3d9c005050 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3da400b3e0 tx=0x7f3d9c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:16:56.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.812+0000 7f3db3fff700 1 -- 172.21.15.40:0/2946764664 reap_dead start 2024-09-13T18:16:56.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.812+0000 7f3db9e05700 1 -- 172.21.15.40:0/2946764664 shutdown_connections 2024-09-13T18:16:56.820 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.812+0000 7f3db9e05700 1 -- 172.21.15.40:0/2946764664 >> 172.21.15.40:0/2946764664 conn(0x7f3db40cf3a0 msgr2=0x7f3db40cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:56.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.812+0000 7f3db9e05700 1 -- 172.21.15.40:0/2946764664 shutdown_connections 2024-09-13T18:16:56.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.812+0000 7f3db9e05700 1 -- 172.21.15.40:0/2946764664 wait complete. 2024-09-13T18:16:56.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.816+0000 7f3db9e05700 1 Processor -- start 2024-09-13T18:16:56.824 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.816+0000 7f3db9e05700 1 -- start start 2024-09-13T18:16:56.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.816+0000 7f3db9e05700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3db41f7070 0x7f3db41f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:56.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.816+0000 7f3db9e05700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3db4155c60 0x7f3db40cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:56.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.816+0000 7f3db9e05700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3db40cd6e0 0x7f3db4152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:56.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.816+0000 7f3db37fe700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3db41f7070 msgr2=0x7f3db41f7450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:16:56.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.816+0000 7f3db37fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3db41f7070 0x7f3db41f7450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:16:56.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.816+0000 7f3db9e05700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db4154b00 con 0x7f3db4155c60 2024-09-13T18:16:56.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.816+0000 7f3db9e05700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db41559a0 con 0x7f3db40cd6e0 2024-09-13T18:16:56.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.816+0000 7f3db3fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3db40cd6e0 0x7f3db4152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:56.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.816+0000 7f3db3fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3db40cd6e0 0x7f3db4152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:55316/0 (socket says 172.21.15.40:55316) 2024-09-13T18:16:56.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.816+0000 7f3db3fff700 1 -- 172.21.15.40:0/708275010 learned_addr learned my addr 172.21.15.40:0/708275010 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:16:56.825 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.816+0000 7f3db9e05700 1 -- 172.21.15.40:0/708275010 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db40cdb60 con 0x7f3db41f7070 2024-09-13T18:16:56.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.816+0000 7f3db3fff700 1 -- 172.21.15.40:0/708275010 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3db41f7070 msgr2=0x7f3db41f7450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:16:56.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.816+0000 7f3db3fff700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3db41f7070 0x7f3db41f7450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:56.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.816+0000 7f3db3fff700 1 -- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3db4155c60 msgr2=0x7f3db40cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:56.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.816+0000 7f3db2ffd700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3db4155c60 0x7f3db40cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:56.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.820+0000 7f3db3fff700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3db4155c60 0x7f3db40cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:56.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.820+0000 7f3db3fff700 1 -- 172.21.15.40:0/708275010 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db41f7ac0 con 0x7f3db40cd6e0 2024-09-13T18:16:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.820+0000 7f3db2ffd700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3db4155c60 0x7f3db40cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:16:56.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.820+0000 7f3db3fff700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3db40cd6e0 0x7f3db4152220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3db4161d60 tx=0x7f3dac00d430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:56.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.824+0000 7f3d9bfff700 1 -- 172.21.15.40:0/708275010 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac0017f0 con 0x7f3db40cd6e0 2024-09-13T18:16:56.830 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.824+0000 7f3db9e05700 1 -- 172.21.15.40:0/708275010 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db4156e70 con 0x7f3db40cd6e0 2024-09-13T18:16:56.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.824+0000 7f3db9e05700 1 -- 172.21.15.40:0/708275010 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db4157410 con 0x7f3db40cd6e0 2024-09-13T18:16:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.824+0000 7f3d9bfff700 1 -- 172.21.15.40:0/708275010 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dac003c30 con 0x7f3db40cd6e0 2024-09-13T18:16:56.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.824+0000 7f3d9bfff700 1 -- 172.21.15.40:0/708275010 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac00d990 con 0x7f3db40cd6e0 2024-09-13T18:16:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.836+0000 7f3d99ffb700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d94002a40 0x7f3d94004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:56.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.836+0000 7f3d99ffb700 1 -- 172.21.15.40:0/708275010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d94007f40 con 0x7f3d94002a40 2024-09-13T18:16:56.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.840+0000 7f3db37fe700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d94002a40 0x7f3d94004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:56.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.840+0000 7f3db37fe700 1 -- 172.21.15.40:0/708275010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d94008100 con 0x7f3d94002a40 2024-09-13T18:16:56.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.840+0000 7f3db37fe700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d94002a40 0x7f3d94004f00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3da4014000 tx=0x7f3da4005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:56.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.848+0000 7f3d9bfff700 1 -- 172.21.15.40:0/708275010 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3dac00daf0 con 0x7f3db40cd6e0 2024-09-13T18:16:56.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.856+0000 7f3d9bfff700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3d9c067920 0x7f3d9c069de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:56.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.856+0000 7f3d9bfff700 1 -- 172.21.15.40:0/708275010 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4019040 con 0x7f3d94002a40 2024-09-13T18:16:56.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.856+0000 7f3d9bfff700 1 -- 172.21.15.40:0/708275010 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3dac0845e0 con 0x7f3db40cd6e0 2024-09-13T18:16:56.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.856+0000 7f3db2ffd700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3d9c067920 0x7f3d9c069de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:56.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.856+0000 7f3d9bfff700 1 -- 172.21.15.40:0/708275010 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3da40050d0 con 0x7f3d94002a40 2024-09-13T18:16:56.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.856+0000 7f3d9bfff700 1 -- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d94002a40 msgr2=0x7f3d94004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:56.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.856+0000 7f3d9bfff700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d94002a40 0x7f3d94004f00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3da4014000 tx=0x7f3da4005ac0 comp rx=0 tx=0).stop 2024-09-13T18:16:56.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.856+0000 7f3d9bfff700 1 -- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d94002a40 msgr2=0x7f3d94004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:16:56.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.856+0000 7f3d9bfff700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d94002a40 0x7f3d94004f00 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3da4014000 tx=0x7f3da4005ac0 comp rx=0 tx=0).stop 2024-09-13T18:16:56.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.856+0000 7f3db2ffd700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3d9c067920 0x7f3d9c069de0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3da8000f80 tx=0x7f3da8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:56.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.868+0000 7f3d99ffb700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d94007040 0x7f3d940074d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:56.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.868+0000 7f3db37fe700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d94007040 0x7f3d940074d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:56.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.868+0000 7f3d99ffb700 1 -- 172.21.15.40:0/708275010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9400a400 con 0x7f3d94007040 2024-09-13T18:16:56.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.872+0000 7f3db37fe700 1 -- 172.21.15.40:0/708275010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3d9400a5c0 con 0x7f3d94007040 2024-09-13T18:16:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.872+0000 7f3db37fe700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d94007040 0x7f3d940074d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3d940080d0 tx=0x7f3da4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.872+0000 7f3d9bfff700 1 -- 172.21.15.40:0/708275010 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da402a2d0 con 0x7f3d94007040 2024-09-13T18:16:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.872+0000 7f3d9bfff700 1 -- 172.21.15.40:0/708275010 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4773 (secure 0 0 0) 0x7f3da40058d0 con 0x7f3d94007040 2024-09-13T18:16:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.872+0000 7f3d9bfff700 1 -- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d94007040 msgr2=0x7f3d940074d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:56.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.872+0000 7f3d9bfff700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d94007040 0x7f3d940074d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3d940080d0 tx=0x7f3da4010000 comp rx=0 tx=0).stop 2024-09-13T18:16:56.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.872+0000 7f3d9bfff700 1 -- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d94007040 msgr2=0x7f3d940074d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:16:56.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.872+0000 7f3d9bfff700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3d94007040 0x7f3d940074d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:56.880 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:16:56.880 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:16:56.880 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:16:56.880 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:16:56.880 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 212, 2024-09-13T18:16:56.880 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:16:56.881 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:16:56.881 INFO:teuthology.orchestra.run.smithi040.stdout: 1 2024-09-13T18:16:56.881 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:56.881 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:16:56.881 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:16:56.881 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:16:56.881 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:16:56.881 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:56.881 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:56.881 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:56.881 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:56.881 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:56.881 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:56.881 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:56.882 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:56.882 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:56.882 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:56.882 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:56.882 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:16:56.882 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:16:56.882 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:56.882 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:56.882 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:56.882 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:56.882 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:56.882 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:56.882 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:56.882 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:56.882 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:56.883 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:56.883 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:56.883 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:56.883 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:16:56.883 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:16:56.883 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:16:56.883 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:16:56.883 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:16:56.883 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:16:56.883 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:16:56.883 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:16:56.883 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:16:56.883 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:16:56.883 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:16:56.883 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:16:56.884 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:16:56.884 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:16:56.884 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:16:56.884 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:16:56.884 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:16:56.884 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:56.884 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:56.884 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:56.884 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:56.884 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:56.884 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:56.884 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:56.884 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:56.884 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:56.884 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:56.885 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:56.885 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:16:56.885 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:56.885 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:16:56.885 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:56.885 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:16:56.885 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:16:56.885 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:56.885 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:56.885 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:56.885 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:56.885 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:16:56.885 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:56.885 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:56.886 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:56.886 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:56.886 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:16:56.886 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:56.886 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:56.886 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:56.886 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:56.886 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:16:56.886 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:16:56.886 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:56.886 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:56.886 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:56.886 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:56.886 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:56.886 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:16:56.887 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:16:56.887 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:56.887 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:56.887 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:56.887 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:56.887 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:16:56.887 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:56.887 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:56.887 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:56.887 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:56.887 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:16:56.887 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:56.887 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:56.887 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:56.887 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:56.888 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:16:56.888 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:16:56.888 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:56.888 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:56.888 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:56.888 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:56.888 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:56.888 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:16:56.888 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:16:56.888 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:56.888 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:56.888 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:56.888 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:56.889 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:16:56.889 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:56.889 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:56.889 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:56.889 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:56.889 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:16:56.889 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:56.889 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:56.889 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:56.889 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:56.889 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:16:56.889 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:16:56.890 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:56.890 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:56.890 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:56.890 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:56.890 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:56.890 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:56.890 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:16:56.890 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:16:56.890 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:56.890 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:56.891 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:56.891 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:16:56.891 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:56.891 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:56.891 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:16:56.891 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:56.891 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:56.891 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:56.892 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:16:56.892 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:56.892 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:56.892 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:16:56.892 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:56.892 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:56.892 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:56.892 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:16:56.893 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:56.893 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:56.893 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:16:56.893 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:56.893 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:56.894 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:56.894 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:16:56.894 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:56.894 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:56.894 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:56.895 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:16:56.895 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:16:56.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.872+0000 7f3db9e05700 1 -- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3d9c067920 msgr2=0x7f3d9c069de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.872+0000 7f3db9e05700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3d9c067920 0x7f3d9c069de0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3da8000f80 tx=0x7f3da8009040 comp rx=0 tx=0).stop 2024-09-13T18:16:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.872+0000 7f3db3fff700 1 -- 172.21.15.40:0/708275010 reap_dead start 2024-09-13T18:16:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.872+0000 7f3db9e05700 1 -- 172.21.15.40:0/708275010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3db40cd6e0 msgr2=0x7f3db4152220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.872+0000 7f3db9e05700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3db40cd6e0 0x7f3db4152220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3db4161d60 tx=0x7f3dac00d430 comp rx=0 tx=0).stop 2024-09-13T18:16:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.876+0000 7f3db9e05700 1 -- 172.21.15.40:0/708275010 shutdown_connections 2024-09-13T18:16:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.876+0000 7f3db9e05700 1 --2- 172.21.15.40:0/708275010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3db40cd6e0 0x7f3db4152220 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:56.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.876+0000 7f3db9e05700 1 -- 172.21.15.40:0/708275010 >> 172.21.15.40:0/708275010 conn(0x7f3db40cf3a0 msgr2=0x7f3db40c4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:56.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.876+0000 7f3db9e05700 1 -- 172.21.15.40:0/708275010 shutdown_connections 2024-09-13T18:16:56.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:56.880+0000 7f3db9e05700 1 -- 172.21.15.40:0/708275010 wait complete. 2024-09-13T18:16:56.919 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:16:56.920 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-13T18:16:56.920 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:16:57.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.296+0000 7f9857fff700 1 Processor -- start 2024-09-13T18:16:57.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.296+0000 7f9857fff700 1 -- start start 2024-09-13T18:16:57.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.296+0000 7f9857fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9858163d00 0x7f98580c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:57.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.296+0000 7f9857fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98580c8340 0x7f98580c6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:57.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.296+0000 7f9857fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9858167290 con 0x7f98580c7f60 2024-09-13T18:16:57.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.296+0000 7f9857fff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98580c1630 con 0x7f98580c8340 2024-09-13T18:16:57.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.296+0000 7f9857fff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9858166680 con 0x7f9858163d00 2024-09-13T18:16:57.307 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.300+0000 7f98567fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9858163d00 0x7f98580c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:57.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.300+0000 7f98567fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9858163d00 0x7f98580c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:55330/0 (socket says 172.21.15.40:55330) 2024-09-13T18:16:57.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.300+0000 7f98567fc700 1 -- 172.21.15.40:0/1038895913 learned_addr learned my addr 172.21.15.40:0/1038895913 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:16:57.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.300+0000 7f98567fc700 1 -- 172.21.15.40:0/1038895913 >> v2:172.21.15.40:3300/0 conn(0x7f98580c8340 msgr2=0x7f98580c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:16:57.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.300+0000 7f98567fc700 1 --2- 172.21.15.40:0/1038895913 >> v2:172.21.15.40:3300/0 conn(0x7f98580c8340 0x7f98580c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:57.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.300+0000 7f98567fc700 1 -- 172.21.15.40:0/1038895913 >> v1:172.21.15.40:6789/0 conn(0x7f98580c7f60 legacy=0x7f98581635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:16:57.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.300+0000 7f98567fc700 1 -- 172.21.15.40:0/1038895913 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9858166960 con 0x7f9858163d00 2024-09-13T18:16:57.308 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.300+0000 7f98567fc700 1 --2- 172.21.15.40:0/1038895913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9858163d00 0x7f98580c7250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f98580c5470 tx=0x7f985000e680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e216825c5f3a7ca3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:57.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.300+0000 7f9855ffb700 1 -- 172.21.15.40:0/1038895913 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9850018650 con 0x7f9858163d00 2024-09-13T18:16:57.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.300+0000 7f9855ffb700 1 -- 172.21.15.40:0/1038895913 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98500187b0 con 0x7f9858163d00 2024-09-13T18:16:57.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.300+0000 7f9857fff700 1 -- 172.21.15.40:0/1038895913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9858163d00 msgr2=0x7f98580c7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:57.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.300+0000 7f9857fff700 1 --2- 172.21.15.40:0/1038895913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9858163d00 0x7f98580c7250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f98580c5470 tx=0x7f985000e680 comp rx=0 tx=0).stop 2024-09-13T18:16:57.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.300+0000 7f9857fff700 1 -- 172.21.15.40:0/1038895913 shutdown_connections 2024-09-13T18:16:57.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.300+0000 7f9857fff700 1 --2- 172.21.15.40:0/1038895913 >> v2:172.21.15.40:3300/0 conn(0x7f98580c8340 0x7f98580c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:57.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.300+0000 7f9857fff700 1 --2- 172.21.15.40:0/1038895913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9858163d00 0x7f98580c7250 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:57.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.300+0000 7f9857fff700 1 -- 172.21.15.40:0/1038895913 >> 172.21.15.40:0/1038895913 conn(0x7f9858066090 msgr2=0x7f98580664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:57.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.308+0000 7f9857fff700 1 -- 172.21.15.40:0/1038895913 shutdown_connections 2024-09-13T18:16:57.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.308+0000 7f9857fff700 1 -- 172.21.15.40:0/1038895913 wait complete. 2024-09-13T18:16:57.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.308+0000 7f9857fff700 1 Processor -- start 2024-09-13T18:16:57.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.308+0000 7f9857fff700 1 -- start start 2024-09-13T18:16:57.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.308+0000 7f9857fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f98580c7f60 0x7f98580cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:57.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.308+0000 7f9857fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98580c8340 0x7f98581eefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:57.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.308+0000 7f9857fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98581fc380 0x7f98581ef4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:57.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.308+0000 7f9857fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985808b9f0 con 0x7f98580c8340 2024-09-13T18:16:57.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.308+0000 7f9857fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98581ec6c0 con 0x7f98581fc380 2024-09-13T18:16:57.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.308+0000 7f9857fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98581fc870 con 0x7f98580c7f60 2024-09-13T18:16:57.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.308+0000 7f9856ffd700 1 -- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f98580c7f60 msgr2=0x7f98580cae50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3301/0 2024-09-13T18:16:57.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.308+0000 7f9856ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f98580c7f60 0x7f98580cae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:16:57.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.308+0000 7f98567fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98580c8340 0x7f98581eefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:57.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.308+0000 7f98567fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98580c8340 0x7f98581eefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55938/0 (socket says 172.21.15.40:55938) 2024-09-13T18:16:57.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.308+0000 7f98567fc700 1 -- 172.21.15.40:0/3862104687 learned_addr learned my addr 172.21.15.40:0/3862104687 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:16:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.312+0000 7f98567fc700 1 -- 172.21.15.40:0/3862104687 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f98580c7f60 msgr2=0x7f98580cae50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:16:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.312+0000 7f98567fc700 1 --2- 172.21.15.40:0/3862104687 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f98580c7f60 0x7f98580cae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.312+0000 7f98567fc700 1 -- 172.21.15.40:0/3862104687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98581fc380 msgr2=0x7f98581ef4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.312+0000 7f98567fc700 1 --2- 172.21.15.40:0/3862104687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98581fc380 0x7f98581ef4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.312+0000 7f98567fc700 1 -- 172.21.15.40:0/3862104687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9850008b20 con 0x7f98580c8340 2024-09-13T18:16:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.312+0000 7f98567fc700 1 --2- 172.21.15.40:0/3862104687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98580c8340 0x7f98581eefa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9850014000 tx=0x7f9850004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.312+0000 7f983f7fe700 1 -- 172.21.15.40:0/3862104687 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9850027070 con 0x7f98580c8340 2024-09-13T18:16:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.312+0000 7f9857fff700 1 -- 172.21.15.40:0/3862104687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985808b470 con 0x7f98580c8340 2024-09-13T18:16:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.312+0000 7f9857fff700 1 -- 172.21.15.40:0/3862104687 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98581ec8d0 con 0x7f98580c8340 2024-09-13T18:16:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.312+0000 7f983f7fe700 1 -- 172.21.15.40:0/3862104687 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98500089a0 con 0x7f98580c8340 2024-09-13T18:16:57.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.312+0000 7f983f7fe700 1 -- 172.21.15.40:0/3862104687 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985001c040 con 0x7f98580c8340 2024-09-13T18:16:57.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.316+0000 7f983f7fe700 1 -- 172.21.15.40:0/3862104687 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f985000fd30 con 0x7f98580c8340 2024-09-13T18:16:57.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.316+0000 7f983f7fe700 1 --2- 172.21.15.40:0/3862104687 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9844066780 0x7f9844068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:57.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.316+0000 7f983f7fe700 1 -- 172.21.15.40:0/3862104687 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9850088670 con 0x7f98580c8340 2024-09-13T18:16:57.324 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.316+0000 7f9856ffd700 1 --2- 172.21.15.40:0/3862104687 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9844066780 0x7f9844068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:57.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.316+0000 7f9857fff700 1 --2- 172.21.15.40:0/3862104687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9838002a40 0x7f9838004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:57.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.320+0000 7f9857fff700 1 -- 172.21.15.40:0/3862104687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9838007f40 con 0x7f9838002a40 2024-09-13T18:16:57.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.320+0000 7f98577fe700 1 --2- 172.21.15.40:0/3862104687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9838002a40 0x7f9838004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:57.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.320+0000 7f98577fe700 1 -- 172.21.15.40:0/3862104687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9838008100 con 0x7f9838002a40 2024-09-13T18:16:57.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.320+0000 7f98577fe700 1 --2- 172.21.15.40:0/3862104687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9838002a40 0x7f9838004f00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f984c00b310 tx=0x7f984c00b620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:57.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.320+0000 7f983f7fe700 1 -- 172.21.15.40:0/3862104687 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c011620 con 0x7f9838002a40 2024-09-13T18:16:57.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.320+0000 7f983f7fe700 1 -- 172.21.15.40:0/3862104687 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f984c011780 con 0x7f9838002a40 2024-09-13T18:16:57.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.320+0000 7f983f7fe700 1 -- 172.21.15.40:0/3862104687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9838002a40 msgr2=0x7f9838004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:57.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.320+0000 7f983f7fe700 1 --2- 172.21.15.40:0/3862104687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9838002a40 0x7f9838004f00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f984c00b310 tx=0x7f984c00b620 comp rx=0 tx=0).stop 2024-09-13T18:16:57.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.320+0000 7f983f7fe700 1 -- 172.21.15.40:0/3862104687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9838002a40 msgr2=0x7f9838004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:16:57.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.320+0000 7f983f7fe700 1 --2- 172.21.15.40:0/3862104687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9838002a40 0x7f9838004f00 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f984c00b310 tx=0x7f984c00b620 comp rx=0 tx=0).stop 2024-09-13T18:16:57.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.320+0000 7f9856ffd700 1 --2- 172.21.15.40:0/3862104687 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9844066780 0x7f9844068c40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9848007fd0 tx=0x7f9848007aa0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:57.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.336+0000 7f9857fff700 1 --2- 172.21.15.40:0/3862104687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9838006ea0 0x7f9838007310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:16:57.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.336+0000 7f9857fff700 1 -- 172.21.15.40:0/3862104687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983800a4a0 con 0x7f9838006ea0 2024-09-13T18:16:57.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.336+0000 7f98577fe700 1 --2- 172.21.15.40:0/3862104687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9838006ea0 0x7f9838007310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:16:57.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.336+0000 7f98577fe700 1 -- 172.21.15.40:0/3862104687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f9838007850 con 0x7f9838006ea0 2024-09-13T18:16:57.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.336+0000 7f98577fe700 1 --2- 172.21.15.40:0/3862104687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9838006ea0 0x7f9838007310 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f984c0092e0 tx=0x7f984c0047e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:16:57.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.336+0000 7f983f7fe700 1 -- 172.21.15.40:0/3862104687 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c01cd90 con 0x7f9838006ea0 2024-09-13T18:16:57.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.336+0000 7f983f7fe700 1 -- 172.21.15.40:0/3862104687 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4937 (secure 0 0 0) 0x7f984c01a420 con 0x7f9838006ea0 2024-09-13T18:16:57.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.336+0000 7f983f7fe700 1 -- 172.21.15.40:0/3862104687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9838006ea0 msgr2=0x7f9838007310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:57.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.336+0000 7f983f7fe700 1 --2- 172.21.15.40:0/3862104687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9838006ea0 0x7f9838007310 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f984c0092e0 tx=0x7f984c0047e0 comp rx=0 tx=0).stop 2024-09-13T18:16:57.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.336+0000 7f983f7fe700 1 -- 172.21.15.40:0/3862104687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9838006ea0 msgr2=0x7f9838007310 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:16:57.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.336+0000 7f983f7fe700 1 --2- 172.21.15.40:0/3862104687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9838006ea0 0x7f9838007310 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f984c0092e0 tx=0x7f984c0047e0 comp rx=0 tx=0).stop 2024-09-13T18:16:57.346 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:16:57.346 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:16:57.346 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:16:57.346 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:16:57.346 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 212, 2024-09-13T18:16:57.346 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:16:57.347 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:16:57.347 INFO:teuthology.orchestra.run.smithi040.stdout: 1 2024-09-13T18:16:57.347 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:57.347 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:16:57.347 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:16:57.347 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:16:57.347 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:16:57.347 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:57.347 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:57.347 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:57.347 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:57.347 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:57.347 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:57.347 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:57.347 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:57.348 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:57.348 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:57.348 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:57.348 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:16:57.348 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:16:57.348 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:57.348 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:57.348 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:57.348 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:57.348 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:57.348 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:57.348 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:57.349 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:57.349 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:57.349 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:57.349 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:57.349 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:57.349 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:16:57.349 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:16:57.349 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:16:57.349 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:16:57.349 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:16:57.349 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:16:57.349 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:16:57.349 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:16:57.350 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:16:57.350 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:16:57.350 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:16:57.350 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:16:57.350 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:16:57.350 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:16:57.350 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:16:57.350 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:16:57.350 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:16:57.350 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:16:57.350 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:16:57.350 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:16:57.350 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:16:57.350 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:16:57.351 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:16:57.351 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:16:57.351 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:16:57.351 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:16:57.351 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:16:57.351 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:57.351 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:16:57.351 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:57.351 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:16:57.351 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:57.351 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:16:57.351 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:16:57.351 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:57.351 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:57.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:57.352 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:57.352 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:16:57.352 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:57.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:57.352 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:57.352 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:57.352 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:16:57.352 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:57.352 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:57.352 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:57.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:57.352 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:16:57.352 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:16:57.352 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:57.353 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:57.353 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:57.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:57.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:57.353 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:16:57.353 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:16:57.353 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:57.353 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:57.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:57.353 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:57.353 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:16:57.353 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:57.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:57.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:57.354 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:57.354 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:16:57.354 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:57.354 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:57.354 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:57.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:57.354 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:16:57.354 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:16:57.354 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:57.354 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:57.354 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:57.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:57.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:57.355 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:16:57.355 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:16:57.355 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:16:57.355 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:16:57.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:57.355 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:16:57.355 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:16:57.355 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:57.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:57.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:57.355 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:16:57.355 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:16:57.355 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:16:57.355 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:57.356 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:57.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:57.356 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:16:57.356 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:16:57.356 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:16:57.356 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:16:57.356 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:16:57.356 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:57.356 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:57.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:57.356 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:16:57.356 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:16:57.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:57.357 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:57.357 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:57.357 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:16:57.357 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:57.357 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:57.357 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:16:57.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:57.357 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:57.357 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:57.357 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:16:57.357 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:57.357 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:57.357 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:16:57.358 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:57.358 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:57.358 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:57.358 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:16:57.358 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:57.358 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:57.358 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:16:57.358 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:57.358 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:57.358 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:57.358 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:16:57.358 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:57.358 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:16:57.358 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:16:57.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:16:57.359 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:16:57.359 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:16:57.359 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:16:57.359 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:16:57.359 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:16:57.359 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:16:57.359 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:16:57.359 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:16:57.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.348+0000 7f983d7fa700 1 -- 172.21.15.40:0/3862104687 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9844066780 msgr2=0x7f9844068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.348+0000 7f983d7fa700 1 --2- 172.21.15.40:0/3862104687 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9844066780 0x7f9844068c40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9848007fd0 tx=0x7f9848007aa0 comp rx=0 tx=0).stop 2024-09-13T18:16:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.348+0000 7f983d7fa700 1 -- 172.21.15.40:0/3862104687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98580c8340 msgr2=0x7f98581eefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:16:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.348+0000 7f983d7fa700 1 --2- 172.21.15.40:0/3862104687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98580c8340 0x7f98581eefa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9850014000 tx=0x7f9850004af0 comp rx=0 tx=0).stop 2024-09-13T18:16:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.348+0000 7f98577fe700 1 -- 172.21.15.40:0/3862104687 reap_dead start 2024-09-13T18:16:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.348+0000 7f983d7fa700 1 -- 172.21.15.40:0/3862104687 shutdown_connections 2024-09-13T18:16:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.348+0000 7f983d7fa700 1 --2- 172.21.15.40:0/3862104687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98580c8340 0x7f98581eefa0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:16:57.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.348+0000 7f983d7fa700 1 -- 172.21.15.40:0/3862104687 >> 172.21.15.40:0/3862104687 conn(0x7f9858066090 msgr2=0x7f98581633c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:16:57.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.356+0000 7f983d7fa700 1 -- 172.21.15.40:0/3862104687 shutdown_connections 2024-09-13T18:16:57.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:16:57.356+0000 7f983d7fa700 1 -- 172.21.15.40:0/3862104687 wait complete. 2024-09-13T18:16:57.384 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:16:57.384 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:17:09.744 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:17:09.747 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:17:17.385 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-13T18:17:17.386 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-13T18:17:17.386 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T18:17:17.386 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T18:17:17.388 INFO:tasks.ceph.mon.c:Started 2024-09-13T18:17:17.389 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-13T18:17:17.389 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-13T18:17:17.389 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-13T18:17:32.390 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-13T18:17:32.391 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-13T18:17:32.391 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:17:32.391 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:17:32.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.752+0000 7fdef1e28700 1 Processor -- start 2024-09-13T18:17:32.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.752+0000 7fdef1e28700 1 -- start start 2024-09-13T18:17:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.760+0000 7fdef1e28700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 0x7fdeec15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.760+0000 7fdef1e28700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 0x7fdeec15f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.760+0000 7fdef1e28700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeec071690 con 0x7fdeec06a390 2024-09-13T18:17:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.760+0000 7fdef1e28700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeec0721e0 con 0x7fdeec154240 2024-09-13T18:17:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.760+0000 7fdef1e28700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec15e6e0 con 0x7fdeec15ec60 2024-09-13T18:17:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.760+0000 7fdeebfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 0x7fdeec15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.760+0000 7fdeebfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 0x7fdeec15f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:46598/0 (socket says 172.21.15.40:46598) 2024-09-13T18:17:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.760+0000 7fdeebfff700 1 -- 172.21.15.40:0/3451452339 learned_addr learned my addr 172.21.15.40:0/3451452339 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:17:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.760+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 0x7fdeec15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.760+0000 7fdeebfff700 1 -- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 msgr2=0x7fdeec15f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:17:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.760+0000 7fdeebfff700 1 -- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 msgr2=0x7fdeec15f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:17:32.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.760+0000 7fdeebfff700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 0x7fdeec15f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:17:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.760+0000 7fdeebfff700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 0x7fdeec15f0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:17:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.760+0000 7fdef0e26700 1 -- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 msgr2=0x7fdeec15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:17:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.760+0000 7fdef0e26700 1 -- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 msgr2=0x7fdeec15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:17:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.760+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 0x7fdeec15e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:17:32.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.760+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 0x7fdeec15e1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:17:32.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.964+0000 7fdeebfff700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 0x7fdeec15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:32.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.964+0000 7fdeebfff700 1 -- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 msgr2=0x7fdeec15f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:17:32.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.964+0000 7fdeebfff700 1 -- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 msgr2=0x7fdeec15f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:17:32.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.964+0000 7fdeebfff700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 0x7fdeec15f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:17:32.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.964+0000 7fdeebfff700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 0x7fdeec15f0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:17:32.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.964+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 0x7fdeec15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:32.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.964+0000 7fdef0e26700 1 -- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 msgr2=0x7fdeec15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:17:32.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.964+0000 7fdef0e26700 1 -- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 msgr2=0x7fdeec15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:17:32.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.964+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 0x7fdeec15e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:17:32.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:32.964+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 0x7fdeec15e1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:17:33.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:33.368+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 0x7fdeec15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:33.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:33.368+0000 7fdeebfff700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 0x7fdeec15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:33.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:33.368+0000 7fdef0e26700 1 -- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 msgr2=0x7fdeec15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:17:33.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:33.368+0000 7fdef0e26700 1 -- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 msgr2=0x7fdeec15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:17:33.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:33.368+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 0x7fdeec15e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:17:33.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:33.368+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 0x7fdeec15e1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:17:33.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:33.368+0000 7fdeebfff700 1 -- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 msgr2=0x7fdeec15f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:17:33.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:33.368+0000 7fdeebfff700 1 -- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 msgr2=0x7fdeec15f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:17:33.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:33.368+0000 7fdeebfff700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 0x7fdeec15f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:17:33.375 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:33.368+0000 7fdeebfff700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 0x7fdeec15f0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:17:34.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:34.172+0000 7fdeebfff700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 0x7fdeec15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:34.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:34.172+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 0x7fdeec15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:34.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:34.172+0000 7fdeebfff700 1 -- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 msgr2=0x7fdeec15f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:17:34.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:34.172+0000 7fdeebfff700 1 -- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 msgr2=0x7fdeec15f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:17:34.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:34.172+0000 7fdeebfff700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 0x7fdeec15f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:17:34.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:34.172+0000 7fdeebfff700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 0x7fdeec15f0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:17:34.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:34.172+0000 7fdef0e26700 1 -- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 msgr2=0x7fdeec15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:17:34.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:34.172+0000 7fdef0e26700 1 -- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 msgr2=0x7fdeec15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:17:34.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:34.172+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 0x7fdeec15e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:17:34.181 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:34.172+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 0x7fdeec15e1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:17:35.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:35.764+0000 7fdeea7fc700 1 -- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 msgr2=0x7fdeec15f0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:17:35.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:35.764+0000 7fdeea7fc700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 0x7fdeec15f0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:35.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:35.764+0000 7fdeea7fc700 1 -- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 msgr2=0x7fdeec15e1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:17:35.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:35.764+0000 7fdeea7fc700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.43:3300/0 conn(0x7fdeec154240 0x7fdeec15e1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:35.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:35.764+0000 7fdeea7fc700 1 -- 172.21.15.40:0/3451452339 >> v1:172.21.15.43:6789/0 conn(0x7fdeec06a390 legacy=0x7fdeec06a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:35.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:35.764+0000 7fdeea7fc700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.40:3300/0 conn(0x7fded8003760 0x7fded8005be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:35.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:35.764+0000 7fdeea7fc700 1 -- 172.21.15.40:0/3451452339 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeec154620 con 0x7fded80062c0 2024-09-13T18:17:35.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:35.764+0000 7fdeea7fc700 1 -- 172.21.15.40:0/3451452339 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeec0721e0 con 0x7fded8003760 2024-09-13T18:17:35.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:35.768+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.40:3300/0 conn(0x7fded8003760 0x7fded8005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:35.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:35.768+0000 7fdef0e26700 1 -- 172.21.15.40:0/3451452339 >> v2:172.21.15.40:3300/0 conn(0x7fded8003760 msgr2=0x7fded8005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:17:35.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:35.768+0000 7fdef0e26700 1 -- 172.21.15.40:0/3451452339 >> v2:172.21.15.40:3300/0 conn(0x7fded8003760 msgr2=0x7fded8005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:17:35.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:35.768+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.40:3300/0 conn(0x7fded8003760 0x7fded8005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:17:35.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:35.768+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.40:3300/0 conn(0x7fded8003760 0x7fded8005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:17:35.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:35.972+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.40:3300/0 conn(0x7fded8003760 0x7fded8005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:35.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:35.972+0000 7fdef0e26700 1 -- 172.21.15.40:0/3451452339 >> v2:172.21.15.40:3300/0 conn(0x7fded8003760 msgr2=0x7fded8005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:17:35.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:35.972+0000 7fdef0e26700 1 -- 172.21.15.40:0/3451452339 >> v2:172.21.15.40:3300/0 conn(0x7fded8003760 msgr2=0x7fded8005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:17:35.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:35.972+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.40:3300/0 conn(0x7fded8003760 0x7fded8005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:17:35.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:35.972+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.40:3300/0 conn(0x7fded8003760 0x7fded8005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:17:36.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.376+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.40:3300/0 conn(0x7fded8003760 0x7fded8005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:36.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.376+0000 7fdef0e26700 1 -- 172.21.15.40:0/3451452339 >> v2:172.21.15.40:3300/0 conn(0x7fded8003760 msgr2=0x7fded8005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:17:36.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.376+0000 7fdef0e26700 1 -- 172.21.15.40:0/3451452339 >> v2:172.21.15.40:3300/0 conn(0x7fded8003760 msgr2=0x7fded8005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:17:36.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.376+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.40:3300/0 conn(0x7fded8003760 0x7fded8005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:17:36.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.376+0000 7fdef0e26700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.40:3300/0 conn(0x7fded8003760 0x7fded8005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:17:36.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.784+0000 7fdeeb7fe700 1 -- 172.21.15.40:0/3451452339 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1879802660 0 0) 0x7fdeec154620 con 0x7fded80062c0 2024-09-13T18:17:36.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.784+0000 7fdeeb7fe700 1 -- 172.21.15.40:0/3451452339 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fded40035d0 con 0x7fded80062c0 2024-09-13T18:17:36.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.784+0000 7fdeeb7fe700 1 -- 172.21.15.40:0/3451452339 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2536370794 0 0) 0x7fded40035d0 con 0x7fded80062c0 2024-09-13T18:17:36.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.788+0000 7fdeeb7fe700 1 -- 172.21.15.40:0/3451452339 >> v2:172.21.15.40:3300/0 conn(0x7fded8003760 msgr2=0x7fded8005be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:17:36.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.788+0000 7fdeeb7fe700 1 --2- 172.21.15.40:0/3451452339 >> v2:172.21.15.40:3300/0 conn(0x7fded8003760 0x7fded8005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:36.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.788+0000 7fdeeb7fe700 1 -- 172.21.15.40:0/3451452339 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded800b0c0 con 0x7fded80062c0 2024-09-13T18:17:36.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.788+0000 7fdeeb7fe700 1 -- 172.21.15.40:0/3451452339 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fdedc009590 con 0x7fded80062c0 2024-09-13T18:17:36.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.788+0000 7fdeeb7fe700 1 -- 172.21.15.40:0/3451452339 >> v1:172.21.15.40:6789/0 conn(0x7fded80062c0 legacy=0x7fded8008780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:36.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.788+0000 7fdef1627700 1 -- 172.21.15.40:0/3451452339 reap_dead start 2024-09-13T18:17:36.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.788+0000 7fdeeb7fe700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 0x7fded4003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.788+0000 7fdeeb7fe700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fded4004080 0x7fded4004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.788+0000 7fdeeb7fe700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded4004e00 0x7fded4005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.788+0000 7fdeeb7fe700 1 -- 172.21.15.40:0/3451452339 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec0721e0 con 0x7fded4004080 2024-09-13T18:17:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.788+0000 7fdeeb7fe700 1 -- 172.21.15.40:0/3451452339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded400bd10 con 0x7fded4004e00 2024-09-13T18:17:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.788+0000 7fdeeb7fe700 1 -- 172.21.15.40:0/3451452339 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded400bed0 con 0x7fdeec15ec60 2024-09-13T18:17:36.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.788+0000 7fdef1627700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded4004e00 0x7fded4005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:36.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.788+0000 7fdef1627700 1 -- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 msgr2=0x7fded4003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:17:36.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.788+0000 7fdef1627700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 0x7fded4003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:36.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.788+0000 7fdef1627700 1 -- 172.21.15.40:0/3451452339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fded4004080 msgr2=0x7fded4004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:17:36.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.788+0000 7fdef1627700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fded4004080 0x7fded4004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:36.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.788+0000 7fdef1627700 1 -- 172.21.15.40:0/3451452339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded800b0c0 con 0x7fded4004e00 2024-09-13T18:17:36.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.788+0000 7fdef1627700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded4004e00 0x7fded4005280 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdeec067320 tx=0x7fded40058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91d151dac3671b6c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:36.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.792+0000 7fdeeb7fe700 1 -- 172.21.15.40:0/3451452339 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee4001c90 con 0x7fded4004e00 2024-09-13T18:17:36.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.792+0000 7fdeeb7fe700 1 -- 172.21.15.40:0/3451452339 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee400cb60 con 0x7fded4004e00 2024-09-13T18:17:36.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.792+0000 7fdeeb7fe700 1 -- 172.21.15.40:0/3451452339 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee40070d0 con 0x7fded4004e00 2024-09-13T18:17:36.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.792+0000 7fdef1e28700 1 -- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded4004e00 msgr2=0x7fded4005280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:36.799 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.792+0000 7fdef1e28700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded4004e00 0x7fded4005280 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdeec067320 tx=0x7fded40058a0 comp rx=0 tx=0).stop 2024-09-13T18:17:36.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.792+0000 7fdef1e28700 1 -- 172.21.15.40:0/3451452339 shutdown_connections 2024-09-13T18:17:36.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.792+0000 7fdef1e28700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded4004e00 0x7fded4005280 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:36.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.792+0000 7fdef1e28700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fded4004080 0x7fded4004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:36.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.792+0000 7fdef1e28700 1 --2- 172.21.15.40:0/3451452339 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdeec15ec60 0x7fded4003a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:36.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.792+0000 7fdef1e28700 1 -- 172.21.15.40:0/3451452339 >> 172.21.15.40:0/3451452339 conn(0x7fdeec0cabc0 msgr2=0x7fdeec0cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:36.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.792+0000 7fdef1e28700 1 -- 172.21.15.40:0/3451452339 shutdown_connections 2024-09-13T18:17:36.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.792+0000 7fdef1e28700 1 -- 172.21.15.40:0/3451452339 wait complete. 2024-09-13T18:17:36.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.796+0000 7fdef1e28700 1 Processor -- start 2024-09-13T18:17:36.802 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.796+0000 7fdef1e28700 1 -- start start 2024-09-13T18:17:36.803 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.796+0000 7fdef1e28700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fded4004080 0x7fdeec1fe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.796+0000 7fdef1e28700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded4004e00 0x7fdeec1596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.796+0000 7fdef1e28700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdeec15ec60 0x7fdeec159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.796+0000 7fdef1627700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdeec15ec60 0x7fdeec159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.796+0000 7fdef1627700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdeec15ec60 0x7fdeec159c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46318/0 (socket says 172.21.15.40:46318) 2024-09-13T18:17:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.796+0000 7fdef1e28700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec159240 con 0x7fdeec15ec60 2024-09-13T18:17:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.796+0000 7fdef1e28700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec158f40 con 0x7fded4004e00 2024-09-13T18:17:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.796+0000 7fdef1e28700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec1ff390 con 0x7fded4004080 2024-09-13T18:17:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.796+0000 7fdef1627700 1 -- 172.21.15.40:0/680357639 learned_addr learned my addr 172.21.15.40:0/680357639 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:17:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.796+0000 7fdef1627700 1 -- 172.21.15.40:0/680357639 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fded4004080 msgr2=0x7fdeec1fe8b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:17:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.796+0000 7fdef1627700 1 --2- 172.21.15.40:0/680357639 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fded4004080 0x7fdeec1fe8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:36.805 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.796+0000 7fdef1627700 1 -- 172.21.15.40:0/680357639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded4004e00 msgr2=0x7fdeec1596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.796+0000 7fdef1627700 1 --2- 172.21.15.40:0/680357639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded4004e00 0x7fdeec1596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.796+0000 7fdef1627700 1 -- 172.21.15.40:0/680357639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeec0c8160 con 0x7fdeec15ec60 2024-09-13T18:17:36.806 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.796+0000 7fdef1627700 1 --2- 172.21.15.40:0/680357639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdeec15ec60 0x7fdeec159c10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fded80061a0 tx=0x7fdee40014d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:36.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.804+0000 7fdee97fa700 1 -- 172.21.15.40:0/680357639 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee4007a70 con 0x7fdeec15ec60 2024-09-13T18:17:36.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.804+0000 7fdef1e28700 1 -- 172.21.15.40:0/680357639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeec0c8440 con 0x7fdeec15ec60 2024-09-13T18:17:36.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.804+0000 7fdef1e28700 1 -- 172.21.15.40:0/680357639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeec154b20 con 0x7fdeec15ec60 2024-09-13T18:17:36.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.808+0000 7fdee97fa700 1 -- 172.21.15.40:0/680357639 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee400ccd0 con 0x7fdeec15ec60 2024-09-13T18:17:36.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.808+0000 7fdee97fa700 1 -- 172.21.15.40:0/680357639 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee4003a20 con 0x7fdeec15ec60 2024-09-13T18:17:36.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.808+0000 7fdee97fa700 1 -- 172.21.15.40:0/680357639 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdee4019030 con 0x7fdeec15ec60 2024-09-13T18:17:36.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.808+0000 7fdee97fa700 1 --2- 172.21.15.40:0/680357639 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fded80670b0 0x7fded8069570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:36.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.808+0000 7fdef0e26700 1 --2- 172.21.15.40:0/680357639 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fded80670b0 0x7fded8069570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:36.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.812+0000 7fdef0e26700 1 --2- 172.21.15.40:0/680357639 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fded80670b0 0x7fded8069570 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fdee0009100 tx=0x7fdee0018570 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:36.819 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.812+0000 7fdee97fa700 1 -- 172.21.15.40:0/680357639 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdee4085f30 con 0x7fdeec15ec60 2024-09-13T18:17:36.822 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.816+0000 7fdef1e28700 1 -- 172.21.15.40:0/680357639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdecc005310 con 0x7fdeec15ec60 2024-09-13T18:17:36.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:36.828+0000 7fdee97fa700 1 -- 172.21.15.40:0/680357639 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdee4052f50 con 0x7fdeec15ec60 2024-09-13T18:17:37.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.452+0000 7fdef1e28700 1 -- 172.21.15.40:0/680357639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdecc005b40 con 0x7fdeec15ec60 2024-09-13T18:17:37.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.460+0000 7fdee97fa700 1 -- 172.21.15.40:0/680357639 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7fdee402b080 con 0x7fdeec15ec60 2024-09-13T18:17:37.468 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":218,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:17:37.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.460+0000 7fded2ffd700 1 -- 172.21.15.40:0/680357639 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fded80670b0 msgr2=0x7fded8069570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:37.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.460+0000 7fded2ffd700 1 --2- 172.21.15.40:0/680357639 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fded80670b0 0x7fded8069570 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fdee0009100 tx=0x7fdee0018570 comp rx=0 tx=0).stop 2024-09-13T18:17:37.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.460+0000 7fded2ffd700 1 -- 172.21.15.40:0/680357639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdeec15ec60 msgr2=0x7fdeec159c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:37.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.460+0000 7fded2ffd700 1 --2- 172.21.15.40:0/680357639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdeec15ec60 0x7fdeec159c10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fded80061a0 tx=0x7fdee40014d0 comp rx=0 tx=0).stop 2024-09-13T18:17:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.464+0000 7fded2ffd700 1 -- 172.21.15.40:0/680357639 shutdown_connections 2024-09-13T18:17:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.464+0000 7fded2ffd700 1 --2- 172.21.15.40:0/680357639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdeec15ec60 0x7fdeec159c10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.464+0000 7fded2ffd700 1 --2- 172.21.15.40:0/680357639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fded4004e00 0x7fdeec1596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:37.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.464+0000 7fded2ffd700 1 --2- 172.21.15.40:0/680357639 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fded80670b0 0x7fded8069570 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:37.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.464+0000 7fded2ffd700 1 --2- 172.21.15.40:0/680357639 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fded4004080 0x7fdeec1fe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:37.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.464+0000 7fded2ffd700 1 -- 172.21.15.40:0/680357639 >> 172.21.15.40:0/680357639 conn(0x7fdeec0cabc0 msgr2=0x7fdeec1529f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:37.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.464+0000 7fded2ffd700 1 -- 172.21.15.40:0/680357639 shutdown_connections 2024-09-13T18:17:37.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.464+0000 7fded2ffd700 1 -- 172.21.15.40:0/680357639 wait complete. 2024-09-13T18:17:37.490 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:17:37.490 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:17:37.490 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:17:37.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.744+0000 7f18fd8da700 1 Processor -- start 2024-09-13T18:17:37.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.744+0000 7f18fd8da700 1 -- start start 2024-09-13T18:17:37.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.744+0000 7f18fd8da700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f80c7f60 0x7f18f80c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:37.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.744+0000 7f18fd8da700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f18f80c8340 0x7f18f80c6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:37.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.744+0000 7f18fd8da700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f8167290 con 0x7f18f806b080 2024-09-13T18:17:37.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.744+0000 7f18fd8da700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f80c1630 con 0x7f18f80c8340 2024-09-13T18:17:37.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.744+0000 7f18fd8da700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f8166680 con 0x7f18f80c7f60 2024-09-13T18:17:37.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.752+0000 7f18f67fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f18f80c8340 0x7f18f80c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:37.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.752+0000 7f18f6ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f80c7f60 0x7f18f80c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:37.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.752+0000 7f18f67fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f18f80c8340 0x7f18f80c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:47070/0 (socket says 172.21.15.40:47070) 2024-09-13T18:17:37.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.752+0000 7f18f67fc700 1 -- 172.21.15.40:0/2661694249 learned_addr learned my addr 172.21.15.40:0/2661694249 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:17:37.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.752+0000 7f18f67fc700 1 -- 172.21.15.40:0/2661694249 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f80c7f60 msgr2=0x7f18f80c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:17:37.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.752+0000 7f18f67fc700 1 --2- 172.21.15.40:0/2661694249 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f80c7f60 0x7f18f80c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:37.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.752+0000 7f18f67fc700 1 -- 172.21.15.40:0/2661694249 >> v1:172.21.15.40:6789/0 conn(0x7f18f806b080 legacy=0x7f18f81635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:17:37.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.752+0000 7f18f67fc700 1 -- 172.21.15.40:0/2661694249 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f8166960 con 0x7f18f80c8340 2024-09-13T18:17:37.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.752+0000 7f18f67fc700 1 --2- 172.21.15.40:0/2661694249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f80c8340 0x7f18f80c6190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f18f0008ec0 tx=0x7f18f0010880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45bf247f78ef197f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:37.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.752+0000 7f18f5ffb700 1 -- 172.21.15.40:0/2661694249 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0003c40 con 0x7f18f80c8340 2024-09-13T18:17:37.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.752+0000 7f18f5ffb700 1 -- 172.21.15.40:0/2661694249 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f0003da0 con 0x7f18f80c8340 2024-09-13T18:17:37.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.752+0000 7f18f5ffb700 1 -- 172.21.15.40:0/2661694249 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0008240 con 0x7f18f80c8340 2024-09-13T18:17:37.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.752+0000 7f18fd8da700 1 -- 172.21.15.40:0/2661694249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f80c8340 msgr2=0x7f18f80c6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:37.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.752+0000 7f18fd8da700 1 --2- 172.21.15.40:0/2661694249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f80c8340 0x7f18f80c6190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f18f0008ec0 tx=0x7f18f0010880 comp rx=0 tx=0).stop 2024-09-13T18:17:37.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.752+0000 7f18fd8da700 1 -- 172.21.15.40:0/2661694249 shutdown_connections 2024-09-13T18:17:37.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.752+0000 7f18fd8da700 1 --2- 172.21.15.40:0/2661694249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f80c8340 0x7f18f80c6190 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:37.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.752+0000 7f18fd8da700 1 --2- 172.21.15.40:0/2661694249 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f80c7f60 0x7f18f80c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:37.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.752+0000 7f18fd8da700 1 -- 172.21.15.40:0/2661694249 >> 172.21.15.40:0/2661694249 conn(0x7f18f8066090 msgr2=0x7f18f80664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:37.768 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.756+0000 7f18fd8da700 1 -- 172.21.15.40:0/2661694249 shutdown_connections 2024-09-13T18:17:37.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.756+0000 7f18fd8da700 1 -- 172.21.15.40:0/2661694249 wait complete. 2024-09-13T18:17:37.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.756+0000 7f18fd8da700 1 Processor -- start 2024-09-13T18:17:37.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.756+0000 7f18fd8da700 1 -- start start 2024-09-13T18:17:37.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.756+0000 7f18fd8da700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f806b080 0x7f18f815dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:37.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.756+0000 7f18fd8da700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f80c7f60 0x7f18f8159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:37.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.756+0000 7f18fd8da700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f80c8340 0x7f18f815a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:37.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.756+0000 7f18fd8da700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f8159b40 con 0x7f18f80c8340 2024-09-13T18:17:37.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.756+0000 7f18fd8da700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f8164f10 con 0x7f18f80c7f60 2024-09-13T18:17:37.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.756+0000 7f18fd8da700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f81fffc0 con 0x7f18f806b080 2024-09-13T18:17:37.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.756+0000 7f18f67fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f80c7f60 0x7f18f8159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:37.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.756+0000 7f18f6ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f806b080 0x7f18f815dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:37.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.756+0000 7f18f67fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f80c7f60 0x7f18f8159fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:47078/0 (socket says 172.21.15.40:47078) 2024-09-13T18:17:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.756+0000 7f18f67fc700 1 -- 172.21.15.40:0/2290678005 learned_addr learned my addr 172.21.15.40:0/2290678005 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:17:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.760+0000 7f18f77fe700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f80c8340 0x7f18f815a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.760+0000 7f18f67fc700 1 -- 172.21.15.40:0/2290678005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f806b080 msgr2=0x7f18f815dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.760+0000 7f18f67fc700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f18f806b080 0x7f18f815dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.760+0000 7f18f67fc700 1 -- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f80c8340 msgr2=0x7f18f815a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.760+0000 7f18f67fc700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f80c8340 0x7f18f815a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.760+0000 7f18f67fc700 1 -- 172.21.15.40:0/2290678005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f0008b20 con 0x7f18f80c7f60 2024-09-13T18:17:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.760+0000 7f18f77fe700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f80c8340 0x7f18f815a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:17:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.760+0000 7f18f67fc700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f80c7f60 0x7f18f8159fd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f18f00108c0 tx=0x7f18f0004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.760+0000 7f18dffff700 1 -- 172.21.15.40:0/2290678005 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f00225f0 con 0x7f18f80c7f60 2024-09-13T18:17:37.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.760+0000 7f18dffff700 1 -- 172.21.15.40:0/2290678005 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f001e040 con 0x7f18f80c7f60 2024-09-13T18:17:37.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.760+0000 7f18dffff700 1 -- 172.21.15.40:0/2290678005 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0022990 con 0x7f18f80c7f60 2024-09-13T18:17:37.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.760+0000 7f18fd8da700 1 -- 172.21.15.40:0/2290678005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f81ff440 con 0x7f18f80c7f60 2024-09-13T18:17:37.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.760+0000 7f18fd8da700 1 -- 172.21.15.40:0/2290678005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f81ffa00 con 0x7f18f80c7f60 2024-09-13T18:17:37.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.764+0000 7f18fd8da700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f815b880 0x7f18f81585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:37.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.764+0000 7f18fd8da700 1 -- 172.21.15.40:0/2290678005 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f815b380 con 0x7f18f815b880 2024-09-13T18:17:37.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.764+0000 7f18f6ffd700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f815b880 0x7f18f81585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:37.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.764+0000 7f18f6ffd700 1 -- 172.21.15.40:0/2290678005 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f18f8166370 con 0x7f18f815b880 2024-09-13T18:17:37.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.764+0000 7f18f6ffd700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f815b880 0x7f18f81585d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f18e8010e80 tx=0x7f18e800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:37.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.768+0000 7f18dffff700 1 -- 172.21.15.40:0/2290678005 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e8015070 con 0x7f18f815b880 2024-09-13T18:17:37.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.768+0000 7f18dffff700 1 -- 172.21.15.40:0/2290678005 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f18f0022af0 con 0x7f18f80c7f60 2024-09-13T18:17:37.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.768+0000 7f18dffff700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f18e4066820 0x7f18e4068ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:37.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.768+0000 7f18dffff700 1 -- 172.21.15.40:0/2290678005 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f18f0089e90 con 0x7f18f80c7f60 2024-09-13T18:17:37.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.768+0000 7f18dffff700 1 -- 172.21.15.40:0/2290678005 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f18e800fdf0 con 0x7f18f815b880 2024-09-13T18:17:37.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.768+0000 7f18dffff700 1 -- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f815b880 msgr2=0x7f18f81585d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:37.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.768+0000 7f18dffff700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f815b880 0x7f18f81585d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f18e8010e80 tx=0x7f18e800f710 comp rx=0 tx=0).stop 2024-09-13T18:17:37.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.768+0000 7f18dffff700 1 -- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f815b880 msgr2=0x7f18f81585d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:17:37.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.768+0000 7f18dffff700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f815b880 0x7f18f81585d0 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f18e8010e80 tx=0x7f18e800f710 comp rx=0 tx=0).stop 2024-09-13T18:17:37.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.768+0000 7f18f77fe700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f18e4066820 0x7f18e4068ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:37.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.768+0000 7f18f77fe700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f18e4066820 0x7f18e4068ce0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0017c0 tx=0x7f18ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:37.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.780+0000 7f18ddffb700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f815c1f0 0x7f18f8158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:37.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.784+0000 7f18f6ffd700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f815c1f0 0x7f18f8158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:37.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.784+0000 7f18ddffb700 1 -- 172.21.15.40:0/2290678005 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f8067720 con 0x7f18f815c1f0 2024-09-13T18:17:37.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.784+0000 7f18f6ffd700 1 -- 172.21.15.40:0/2290678005 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f18f8166f20 con 0x7f18f815c1f0 2024-09-13T18:17:37.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.784+0000 7f18f6ffd700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f815c1f0 0x7f18f8158b10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f18e8010e80 tx=0x7f18e8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:37.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.784+0000 7f18dffff700 1 -- 172.21.15.40:0/2290678005 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e8025070 con 0x7f18f815c1f0 2024-09-13T18:17:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.788+0000 7f18dffff700 1 -- 172.21.15.40:0/2290678005 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4784 (secure 0 0 0) 0x7f18e8003d50 con 0x7f18f815c1f0 2024-09-13T18:17:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.788+0000 7f18dffff700 1 -- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f815c1f0 msgr2=0x7f18f8158b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.788+0000 7f18dffff700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f815c1f0 0x7f18f8158b10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f18e8010e80 tx=0x7f18e8004770 comp rx=0 tx=0).stop 2024-09-13T18:17:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.788+0000 7f18dffff700 1 -- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f815c1f0 msgr2=0x7f18f8158b10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:17:37.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.788+0000 7f18dffff700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f18f815c1f0 0x7f18f8158b10 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f18e8010e80 tx=0x7f18e8004770 comp rx=0 tx=0).stop 2024-09-13T18:17:37.795 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:17:37.795 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:17:37.795 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:17:37.795 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:17:37.795 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 218, 2024-09-13T18:17:37.795 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:17:37.795 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:17:37.795 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:17:37.795 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:17:37.795 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:37.795 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:17:37.796 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:17:37.796 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:17:37.796 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:17:37.796 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:17:37.796 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:17:37.796 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:17:37.796 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:17:37.796 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:17:37.796 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:17:37.796 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:17:37.796 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:17:37.796 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:17:37.796 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:17:37.797 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:37.797 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:17:37.797 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:17:37.797 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:17:37.797 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:17:37.797 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:17:37.797 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:17:37.797 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:17:37.797 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:17:37.797 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:17:37.798 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:17:37.798 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:17:37.798 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:17:37.798 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:37.798 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:37.798 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:17:37.798 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:17:37.798 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:17:37.798 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:17:37.798 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:17:37.798 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:17:37.798 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:17:37.798 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:17:37.798 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:17:37.799 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:17:37.799 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:17:37.799 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:17:37.799 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:17:37.799 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:17:37.799 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:17:37.799 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:17:37.799 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:17:37.799 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:17:37.799 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:17:37.799 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:17:37.799 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:17:37.799 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:17:37.800 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:17:37.800 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:17:37.800 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:17:37.800 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:17:37.800 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:17:37.800 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:37.800 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:17:37.800 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:37.800 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:17:37.800 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:37.800 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:17:37.800 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:17:37.800 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:17:37.801 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:17:37.801 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:37.801 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:17:37.801 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:17:37.801 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:37.801 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:37.801 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:37.801 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:17:37.801 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:17:37.801 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:37.801 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:37.801 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:37.801 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:37.801 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:17:37.801 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:17:37.802 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:17:37.802 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:17:37.802 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:17:37.802 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:37.802 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:37.802 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:17:37.802 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:17:37.802 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:17:37.802 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:17:37.802 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:37.802 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:17:37.803 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:17:37.803 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:37.803 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:37.803 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:37.803 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:17:37.803 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:17:37.803 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:37.803 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:37.803 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:37.803 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:37.803 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:17:37.803 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:17:37.803 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:17:37.804 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:17:37.804 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:17:37.804 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:37.804 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:37.804 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:17:37.804 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:17:37.804 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:17:37.804 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:17:37.804 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:37.804 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:17:37.804 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:17:37.804 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:37.804 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:37.805 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:37.805 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:17:37.805 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:17:37.805 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:37.805 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:37.805 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:37.805 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:37.805 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:17:37.805 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:17:37.805 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:17:37.805 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:17:37.805 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:17:37.805 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:37.806 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:37.806 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:37.806 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:17:37.806 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:17:37.806 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:37.806 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:37.806 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:37.806 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:17:37.806 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:37.806 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:37.806 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:17:37.806 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:37.806 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:37.806 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:37.807 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:17:37.807 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:37.807 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:37.807 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:17:37.807 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:37.807 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:37.807 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:37.807 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:17:37.807 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:37.807 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:37.808 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:17:37.808 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:37.808 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:37.808 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:37.808 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:17:37.808 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:37.808 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:37.808 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:37.808 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:17:37.808 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:17:37.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.788+0000 7f18ddffb700 1 -- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f18e4066820 msgr2=0x7f18e4068ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:37.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.788+0000 7f18ddffb700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f18e4066820 0x7f18e4068ce0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0017c0 tx=0x7f18ec009040 comp rx=0 tx=0).stop 2024-09-13T18:17:37.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.788+0000 7f18ddffb700 1 -- 172.21.15.40:0/2290678005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f80c7f60 msgr2=0x7f18f8159fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.788+0000 7f18ddffb700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f80c7f60 0x7f18f8159fd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f18f00108c0 tx=0x7f18f0004af0 comp rx=0 tx=0).stop 2024-09-13T18:17:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.788+0000 7f18f77fe700 1 -- 172.21.15.40:0/2290678005 reap_dead start 2024-09-13T18:17:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.788+0000 7f18ddffb700 1 -- 172.21.15.40:0/2290678005 shutdown_connections 2024-09-13T18:17:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.788+0000 7f18ddffb700 1 --2- 172.21.15.40:0/2290678005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f80c7f60 0x7f18f8159fd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.788+0000 7f18ddffb700 1 -- 172.21.15.40:0/2290678005 >> 172.21.15.40:0/2290678005 conn(0x7f18f8066090 msgr2=0x7f18f80bfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.792+0000 7f18ddffb700 1 -- 172.21.15.40:0/2290678005 shutdown_connections 2024-09-13T18:17:37.811 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:37.792+0000 7f18ddffb700 1 -- 172.21.15.40:0/2290678005 wait complete. 2024-09-13T18:17:37.834 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:17:37.834 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:17:37.834 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:17:38.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.188+0000 7f957f00e700 1 Processor -- start 2024-09-13T18:17:38.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.188+0000 7f957f00e700 1 -- start start 2024-09-13T18:17:38.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.188+0000 7f957f00e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95780c7f60 0x7f95780c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:38.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.188+0000 7f957f00e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f95780c7250 0x7f95781577f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:38.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.188+0000 7f957f00e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f957806b400 con 0x7f95780c8340 2024-09-13T18:17:38.195 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.188+0000 7f957f00e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f957815e920 con 0x7f95780c7f60 2024-09-13T18:17:38.196 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.188+0000 7f957f00e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957815dda0 con 0x7f95780c7250 2024-09-13T18:17:38.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.188+0000 7f957e00c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95780c7f60 0x7f95780c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:38.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.188+0000 7f957e00c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f95780c7f60 0x7f95780c61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46358/0 (socket says 172.21.15.40:46358) 2024-09-13T18:17:38.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.188+0000 7f957e00c700 1 -- 172.21.15.40:0/3999384392 learned_addr learned my addr 172.21.15.40:0/3999384392 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:17:38.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.188+0000 7f957e00c700 1 -- 172.21.15.40:0/3999384392 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f95780c7250 msgr2=0x7f95781577f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:17:38.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.188+0000 7f957e00c700 1 --2- 172.21.15.40:0/3999384392 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f95780c7250 0x7f95781577f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.188+0000 7f957e00c700 1 -- 172.21.15.40:0/3999384392 >> v1:172.21.15.40:6789/0 conn(0x7f95780c8340 legacy=0x7f957806d920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:17:38.197 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.188+0000 7f957e00c700 1 -- 172.21.15.40:0/3999384392 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957815e080 con 0x7f95780c7f60 2024-09-13T18:17:38.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.192+0000 7f957e00c700 1 --2- 172.21.15.40:0/3999384392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780c7f60 0x7f95780c61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f957000b1a0 tx=0x7f957000b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2db64f5f8524da server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:38.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.196+0000 7f957d00a700 1 -- 172.21.15.40:0/3999384392 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957000e070 con 0x7f95780c7f60 2024-09-13T18:17:38.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.196+0000 7f957d00a700 1 -- 172.21.15.40:0/3999384392 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95700090d0 con 0x7f95780c7f60 2024-09-13T18:17:38.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.196+0000 7f957d00a700 1 -- 172.21.15.40:0/3999384392 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957000be00 con 0x7f95780c7f60 2024-09-13T18:17:38.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.200+0000 7f957f00e700 1 -- 172.21.15.40:0/3999384392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780c7f60 msgr2=0x7f95780c61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:38.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.200+0000 7f957f00e700 1 --2- 172.21.15.40:0/3999384392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780c7f60 0x7f95780c61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f957000b1a0 tx=0x7f957000b4b0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.200+0000 7f957f00e700 1 -- 172.21.15.40:0/3999384392 shutdown_connections 2024-09-13T18:17:38.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.200+0000 7f957f00e700 1 --2- 172.21.15.40:0/3999384392 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f95780c7250 0x7f95781577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.200+0000 7f957f00e700 1 --2- 172.21.15.40:0/3999384392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780c7f60 0x7f95780c61e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.200+0000 7f957f00e700 1 -- 172.21.15.40:0/3999384392 >> 172.21.15.40:0/3999384392 conn(0x7f9578066090 msgr2=0x7f95780664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:38.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.200+0000 7f957f00e700 1 -- 172.21.15.40:0/3999384392 shutdown_connections 2024-09-13T18:17:38.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.200+0000 7f957f00e700 1 -- 172.21.15.40:0/3999384392 wait complete. 2024-09-13T18:17:38.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.200+0000 7f957f00e700 1 Processor -- start 2024-09-13T18:17:38.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.200+0000 7f957f00e700 1 -- start start 2024-09-13T18:17:38.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.200+0000 7f957f00e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95780c7250 0x7f95780c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:38.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.200+0000 7f957f00e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780c7f60 0x7f9578071670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:38.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.200+0000 7f957f00e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f95780c8340 0x7f9578071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:38.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.200+0000 7f957f00e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9578144f20 con 0x7f95780c7f60 2024-09-13T18:17:38.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.204+0000 7f957f00e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9578144b50 con 0x7f95780c7250 2024-09-13T18:17:38.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.204+0000 7f957f00e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9578145170 con 0x7f95780c8340 2024-09-13T18:17:38.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.204+0000 7f957e00c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95780c7250 0x7f95780c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:38.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.204+0000 7f957e00c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95780c7250 0x7f95780c4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:47094/0 (socket says 172.21.15.40:47094) 2024-09-13T18:17:38.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.204+0000 7f957e00c700 1 -- 172.21.15.40:0/68039173 learned_addr learned my addr 172.21.15.40:0/68039173 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:17:38.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.204+0000 7f957e80d700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f95780c8340 0x7f9578071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:38.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.204+0000 7f957d80b700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780c7f60 0x7f9578071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:38.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.204+0000 7f957e00c700 1 -- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f95780c8340 msgr2=0x7f9578071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:38.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.204+0000 7f957e00c700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f95780c8340 0x7f9578071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.204+0000 7f957e00c700 1 -- 172.21.15.40:0/68039173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780c7f60 msgr2=0x7f9578071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:38.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.204+0000 7f957e00c700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780c7f60 0x7f9578071670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.204+0000 7f957e00c700 1 -- 172.21.15.40:0/68039173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9570009b30 con 0x7f95780c7250 2024-09-13T18:17:38.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.204+0000 7f957d80b700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f95780c7f60 0x7f9578071670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:17:38.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.204+0000 7f957e00c700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95780c7250 0x7f95780c4bb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f95780c88c0 tx=0x7f9570004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:38.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.204+0000 7f956effd700 1 -- 172.21.15.40:0/68039173 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570004580 con 0x7f95780c7250 2024-09-13T18:17:38.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.204+0000 7f956effd700 1 -- 172.21.15.40:0/68039173 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95700046e0 con 0x7f95780c7250 2024-09-13T18:17:38.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.204+0000 7f957f00e700 1 -- 172.21.15.40:0/68039173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9578153800 con 0x7f95780c7250 2024-09-13T18:17:38.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.204+0000 7f957f00e700 1 -- 172.21.15.40:0/68039173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9578157870 con 0x7f95780c7250 2024-09-13T18:17:38.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.208+0000 7f956cff9700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f957815f100 0x7f957815f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:38.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.208+0000 7f956cff9700 1 -- 172.21.15.40:0/68039173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9578067720 con 0x7f957815f100 2024-09-13T18:17:38.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.208+0000 7f957d80b700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f957815f100 0x7f957815f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:38.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.208+0000 7f957d80b700 1 -- 172.21.15.40:0/68039173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f957806d400 con 0x7f957815f100 2024-09-13T18:17:38.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.208+0000 7f957d80b700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f957815f100 0x7f957815f720 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f95780c6680 tx=0x7f956800ca80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:38.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.212+0000 7f956effd700 1 -- 172.21.15.40:0/68039173 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570012430 con 0x7f95780c7250 2024-09-13T18:17:38.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.212+0000 7f956effd700 1 -- 172.21.15.40:0/68039173 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9568011800 con 0x7f957815f100 2024-09-13T18:17:38.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.212+0000 7f956effd700 1 -- 172.21.15.40:0/68039173 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9570019070 con 0x7f95780c7250 2024-09-13T18:17:38.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.212+0000 7f956effd700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f955c066800 0x7f955c068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:38.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.212+0000 7f956effd700 1 -- 172.21.15.40:0/68039173 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f9568011960 con 0x7f957815f100 2024-09-13T18:17:38.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.212+0000 7f956effd700 1 -- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f957815f100 msgr2=0x7f957815f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:38.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.212+0000 7f956effd700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f957815f100 0x7f957815f720 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f95780c6680 tx=0x7f956800ca80 comp rx=0 tx=0).stop 2024-09-13T18:17:38.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.212+0000 7f957e80d700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f955c066800 0x7f955c068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:38.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.212+0000 7f956effd700 1 -- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f957815f100 msgr2=0x7f957815f720 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:17:38.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.212+0000 7f956effd700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f957815f100 0x7f957815f720 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f95780c6680 tx=0x7f956800ca80 comp rx=0 tx=0).stop 2024-09-13T18:17:38.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.212+0000 7f956effd700 1 -- 172.21.15.40:0/68039173 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9570085830 con 0x7f95780c7250 2024-09-13T18:17:38.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.216+0000 7f957e80d700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f955c066800 0x7f955c068cc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9578161c40 tx=0x7f9574008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:38.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.228+0000 7f956cff9700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95780c9ba0 0x7f95780c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:38.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.228+0000 7f956cff9700 1 -- 172.21.15.40:0/68039173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957815e640 con 0x7f95780c9ba0 2024-09-13T18:17:38.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.232+0000 7f957d80b700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95780c9ba0 0x7f95780c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:38.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.236+0000 7f957d80b700 1 -- 172.21.15.40:0/68039173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f95780c7680 con 0x7f95780c9ba0 2024-09-13T18:17:38.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.236+0000 7f957d80b700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95780c9ba0 0x7f95780c9fb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f95780c6680 tx=0x7f95680047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:38.250 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.244+0000 7f956effd700 1 -- 172.21.15.40:0/68039173 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956801f040 con 0x7f95780c9ba0 2024-09-13T18:17:38.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.244+0000 7f956effd700 1 -- 172.21.15.40:0/68039173 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7f9568004020 con 0x7f95780c9ba0 2024-09-13T18:17:38.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.244+0000 7f956effd700 1 -- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95780c9ba0 msgr2=0x7f95780c9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:38.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.244+0000 7f956effd700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95780c9ba0 0x7f95780c9fb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f95780c6680 tx=0x7f95680047b0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.244+0000 7f956effd700 1 -- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95780c9ba0 msgr2=0x7f95780c9fb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:17:38.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.244+0000 7f956effd700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95780c9ba0 0x7f95780c9fb0 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f95780c6680 tx=0x7f95680047b0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.252 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:17:38.252 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:17:38.252 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:17:38.252 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:17:38.252 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 218, 2024-09-13T18:17:38.252 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:17:38.252 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:17:38.252 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:17:38.252 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:17:38.252 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:38.252 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:17:38.253 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:17:38.253 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:17:38.253 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:17:38.253 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:17:38.253 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:17:38.253 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:17:38.253 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:17:38.253 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:17:38.253 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:17:38.253 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:17:38.253 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:17:38.254 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:17:38.254 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:17:38.254 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:38.254 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:17:38.254 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:17:38.254 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:17:38.254 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:17:38.254 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:17:38.254 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:17:38.254 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:17:38.254 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:17:38.255 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:17:38.255 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:17:38.255 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:17:38.255 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:17:38.255 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:38.255 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.255 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:17:38.255 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:17:38.255 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:17:38.255 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:17:38.255 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:17:38.255 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:17:38.255 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:17:38.256 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:17:38.256 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:17:38.256 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:17:38.256 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:17:38.256 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:17:38.256 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:17:38.256 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:17:38.256 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:17:38.256 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:17:38.256 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:17:38.256 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:17:38.256 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:17:38.256 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:17:38.257 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:17:38.257 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:17:38.257 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:17:38.257 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:17:38.257 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:17:38.257 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:17:38.257 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:17:38.257 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:38.257 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:17:38.257 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.257 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:17:38.257 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.258 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:17:38.258 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:17:38.258 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:17:38.258 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:17:38.258 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.258 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:17:38.258 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:17:38.258 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:38.258 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.259 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.259 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:17:38.259 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:17:38.259 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:38.259 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:38.259 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:38.259 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.259 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:17:38.259 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:17:38.259 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:17:38.259 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:17:38.259 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:17:38.259 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.259 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.260 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:17:38.260 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:17:38.260 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:17:38.260 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:17:38.260 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.260 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:17:38.260 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:17:38.260 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:38.260 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.260 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.260 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:17:38.260 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:17:38.260 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:38.261 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:38.261 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:38.261 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.261 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:17:38.261 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:17:38.261 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:17:38.261 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:17:38.261 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:17:38.261 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.261 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.261 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:17:38.261 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:17:38.261 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:17:38.261 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:17:38.262 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.262 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:17:38.262 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:17:38.262 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:38.262 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.262 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.262 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:17:38.262 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:17:38.262 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:38.262 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:38.262 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:38.262 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.262 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:17:38.262 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:17:38.263 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:17:38.263 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:17:38.263 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:17:38.263 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:38.263 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:38.263 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.263 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:17:38.263 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:17:38.263 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.263 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:38.263 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:38.263 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:17:38.263 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:38.263 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:38.264 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:17:38.264 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.264 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:38.264 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:38.264 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:17:38.264 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:38.264 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:38.264 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:17:38.264 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.264 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:38.264 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:38.265 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:17:38.265 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:38.265 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:38.265 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:17:38.265 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.265 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:38.265 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:38.265 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:17:38.265 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:38.265 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:38.265 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:17:38.265 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.265 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:38.266 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:38.266 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:17:38.266 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:38.266 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:38.266 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.266 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:17:38.266 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:17:38.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.252+0000 7f957f00e700 1 -- 172.21.15.40:0/68039173 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f955c066800 msgr2=0x7f955c068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:38.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.252+0000 7f957f00e700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f955c066800 0x7f955c068cc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9578161c40 tx=0x7f9574008040 comp rx=0 tx=0).stop 2024-09-13T18:17:38.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.252+0000 7f957f00e700 1 -- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95780c7250 msgr2=0x7f95780c4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:38.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.252+0000 7f957e80d700 1 -- 172.21.15.40:0/68039173 reap_dead start 2024-09-13T18:17:38.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.252+0000 7f957f00e700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95780c7250 0x7f95780c4bb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f95780c88c0 tx=0x7f9570004af0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.252+0000 7f957f00e700 1 -- 172.21.15.40:0/68039173 shutdown_connections 2024-09-13T18:17:38.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.252+0000 7f957f00e700 1 --2- 172.21.15.40:0/68039173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95780c7250 0x7f95780c4bb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.252+0000 7f957f00e700 1 -- 172.21.15.40:0/68039173 >> 172.21.15.40:0/68039173 conn(0x7f9578066090 msgr2=0x7f957815c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:38.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.256+0000 7f957f00e700 1 -- 172.21.15.40:0/68039173 shutdown_connections 2024-09-13T18:17:38.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.256+0000 7f957f00e700 1 -- 172.21.15.40:0/68039173 wait complete. 2024-09-13T18:17:38.317 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:17:38.317 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:17:38.317 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:17:38.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.824+0000 7fc3f242b700 1 Processor -- start 2024-09-13T18:17:38.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.828+0000 7fc3f242b700 1 -- start start 2024-09-13T18:17:38.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.828+0000 7fc3f242b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc3ec157e90 0x7fc3ec0c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:38.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.828+0000 7fc3f242b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3ec0c57d0 0x7fc3ec15e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:38.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.828+0000 7fc3f242b700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3ec1619c0 con 0x7fc3ec15e6b0 2024-09-13T18:17:38.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.828+0000 7fc3f242b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3ec158270 con 0x7fc3ec157e90 2024-09-13T18:17:38.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.828+0000 7fc3f242b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec15dd50 con 0x7fc3ec0c57d0 2024-09-13T18:17:38.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.828+0000 7fc3f09c8700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc3ec15e6b0 0x7fc3ec06d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:52084/0 (socket says 172.21.15.40:52084) 2024-09-13T18:17:38.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.828+0000 7fc3f09c8700 1 -- 172.21.15.40:0/2665192596 learned_addr learned my addr 172.21.15.40:0/2665192596 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:17:38.837 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.828+0000 7fc3ebfff700 1 --2- 172.21.15.40:0/2665192596 >> v2:172.21.15.40:3300/0 conn(0x7fc3ec157e90 0x7fc3ec0c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:38.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.828+0000 7fc3eb7fe700 1 --2- 172.21.15.40:0/2665192596 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3ec0c57d0 0x7fc3ec15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:38.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.832+0000 7fc3ebfff700 1 -- 172.21.15.40:0/2665192596 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3ec0c57d0 msgr2=0x7fc3ec15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:17:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.832+0000 7fc3ebfff700 1 --2- 172.21.15.40:0/2665192596 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3ec0c57d0 0x7fc3ec15e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.832+0000 7fc3ebfff700 1 -- 172.21.15.40:0/2665192596 >> v1:172.21.15.43:6789/0 conn(0x7fc3ec15e6b0 legacy=0x7fc3ec06d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.832+0000 7fc3ebfff700 1 -- 172.21.15.40:0/2665192596 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec161ed0 con 0x7fc3ec157e90 2024-09-13T18:17:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.832+0000 7fc3ebfff700 1 --2- 172.21.15.40:0/2665192596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3ec157e90 0x7fc3ec0c71e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009850 tx=0x7fc3e0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b767ce88a2b497ec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:38.848 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.840+0000 7fc3eaffd700 1 -- 172.21.15.40:0/2665192596 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0004230 con 0x7fc3ec157e90 2024-09-13T18:17:38.849 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.840+0000 7fc3eaffd700 1 -- 172.21.15.40:0/2665192596 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e0004390 con 0x7fc3ec157e90 2024-09-13T18:17:38.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.840+0000 7fc3f242b700 1 -- 172.21.15.40:0/2665192596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3ec157e90 msgr2=0x7fc3ec0c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:38.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.840+0000 7fc3f242b700 1 --2- 172.21.15.40:0/2665192596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3ec157e90 0x7fc3ec0c71e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009850 tx=0x7fc3e0009b60 comp rx=0 tx=0).stop 2024-09-13T18:17:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.848+0000 7fc3f242b700 1 -- 172.21.15.40:0/2665192596 shutdown_connections 2024-09-13T18:17:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.848+0000 7fc3f242b700 1 --2- 172.21.15.40:0/2665192596 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3ec0c57d0 0x7fc3ec15e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.848+0000 7fc3f242b700 1 --2- 172.21.15.40:0/2665192596 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3ec157e90 0x7fc3ec0c71e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.848+0000 7fc3f242b700 1 -- 172.21.15.40:0/2665192596 >> 172.21.15.40:0/2665192596 conn(0x7fc3ec066090 msgr2=0x7fc3ec0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.848+0000 7fc3f242b700 1 -- 172.21.15.40:0/2665192596 shutdown_connections 2024-09-13T18:17:38.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.848+0000 7fc3f242b700 1 -- 172.21.15.40:0/2665192596 wait complete. 2024-09-13T18:17:38.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.852+0000 7fc3f242b700 1 Processor -- start 2024-09-13T18:17:38.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.860+0000 7fc3f242b700 1 -- start start 2024-09-13T18:17:38.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.860+0000 7fc3f242b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3ec0c57d0 0x7fc3ec0c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:38.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.860+0000 7fc3f242b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3ec157e90 0x7fc3ec1533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:38.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.860+0000 7fc3f242b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3ec15e6b0 0x7fc3ec159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:38.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.860+0000 7fc3f242b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec15b5e0 con 0x7fc3ec0c57d0 2024-09-13T18:17:38.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.860+0000 7fc3f242b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec158570 con 0x7fc3ec157e90 2024-09-13T18:17:38.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.860+0000 7fc3f242b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec152cb0 con 0x7fc3ec15e6b0 2024-09-13T18:17:38.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.860+0000 7fc3f09c8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3ec15e6b0 0x7fc3ec159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:38.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.860+0000 7fc3f09c8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3ec15e6b0 0x7fc3ec159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:46674/0 (socket says 172.21.15.40:46674) 2024-09-13T18:17:38.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.860+0000 7fc3f09c8700 1 -- 172.21.15.40:0/4033655469 learned_addr learned my addr 172.21.15.40:0/4033655469 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:17:38.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.860+0000 7fc3f09c8700 1 -- 172.21.15.40:0/4033655469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3ec157e90 msgr2=0x7fc3ec1533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:38.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.860+0000 7fc3f09c8700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3ec157e90 0x7fc3ec1533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.860+0000 7fc3ebfff700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3ec0c57d0 0x7fc3ec0c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:38.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.860+0000 7fc3f09c8700 1 -- 172.21.15.40:0/4033655469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3ec0c57d0 msgr2=0x7fc3ec0c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:38.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.860+0000 7fc3f09c8700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3ec0c57d0 0x7fc3ec0c4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.860+0000 7fc3f09c8700 1 -- 172.21.15.40:0/4033655469 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e00094b0 con 0x7fc3ec15e6b0 2024-09-13T18:17:38.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.860+0000 7fc3ebfff700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3ec0c57d0 0x7fc3ec0c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:17:38.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.864+0000 7fc3f09c8700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3ec15e6b0 0x7fc3ec159e50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00c520 tx=0x7fc3dc00c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:38.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.868+0000 7fc3e8ff9700 1 -- 172.21.15.40:0/4033655469 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc004ca0 con 0x7fc3ec15e6b0 2024-09-13T18:17:38.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.868+0000 7fc3f242b700 1 -- 172.21.15.40:0/4033655469 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3ec15ad40 con 0x7fc3ec15e6b0 2024-09-13T18:17:38.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.868+0000 7fc3f242b700 1 -- 172.21.15.40:0/4033655469 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3ec15b300 con 0x7fc3ec15e6b0 2024-09-13T18:17:38.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.872+0000 7fc3e8ff9700 1 -- 172.21.15.40:0/4033655469 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc004e00 con 0x7fc3ec15e6b0 2024-09-13T18:17:38.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.872+0000 7fc3e8ff9700 1 -- 172.21.15.40:0/4033655469 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0072f0 con 0x7fc3ec15e6b0 2024-09-13T18:17:38.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.876+0000 7fc3e8ff9700 1 -- 172.21.15.40:0/4033655469 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc3dc007450 con 0x7fc3ec15e6b0 2024-09-13T18:17:38.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.876+0000 7fc3e8ff9700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc3d8066800 0x7fc3d8068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:38.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.876+0000 7fc3e8ff9700 1 -- 172.21.15.40:0/4033655469 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc3dc084250 con 0x7fc3ec15e6b0 2024-09-13T18:17:38.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.876+0000 7fc3f242b700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3cc002a40 0x7fc3cc004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:38.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.876+0000 7fc3f242b700 1 -- 172.21.15.40:0/4033655469 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc007f40 con 0x7fc3cc002a40 2024-09-13T18:17:38.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.876+0000 7fc3eb7fe700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3cc002a40 0x7fc3cc004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:38.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.876+0000 7fc3eb7fe700 1 -- 172.21.15.40:0/4033655469 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc3cc008100 con 0x7fc3cc002a40 2024-09-13T18:17:38.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.880+0000 7fc3eb7fe700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3cc002a40 0x7fc3cc004f00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0c6f60 tx=0x7fc3e400e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:38.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.880+0000 7fc3ebfff700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc3d8066800 0x7fc3d8068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:38.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.880+0000 7fc3e8ff9700 1 -- 172.21.15.40:0/4033655469 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e400ed60 con 0x7fc3cc002a40 2024-09-13T18:17:38.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.880+0000 7fc3e8ff9700 1 -- 172.21.15.40:0/4033655469 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc3e400eec0 con 0x7fc3cc002a40 2024-09-13T18:17:38.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.880+0000 7fc3e8ff9700 1 -- 172.21.15.40:0/4033655469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3cc002a40 msgr2=0x7fc3cc004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:38.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.880+0000 7fc3e8ff9700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3cc002a40 0x7fc3cc004f00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0c6f60 tx=0x7fc3e400e3c0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.884+0000 7fc3e8ff9700 1 -- 172.21.15.40:0/4033655469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3cc002a40 msgr2=0x7fc3cc004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:17:38.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.884+0000 7fc3e8ff9700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3cc002a40 0x7fc3cc004f00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.884+0000 7fc3ebfff700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc3d8066800 0x7fc3d8068cc0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009ba0 tx=0x7fc3e0010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:38.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.928+0000 7fc3f242b700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3cc007020 0x7fc3cc007490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:38.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.928+0000 7fc3f242b700 1 -- 172.21.15.40:0/4033655469 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc00a400 con 0x7fc3cc007020 2024-09-13T18:17:38.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.932+0000 7fc3eb7fe700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3cc007020 0x7fc3cc007490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:38.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.932+0000 7fc3eb7fe700 1 -- 172.21.15.40:0/4033655469 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc3cc00a5c0 con 0x7fc3cc007020 2024-09-13T18:17:38.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.936+0000 7fc3eb7fe700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3cc007020 0x7fc3cc007490 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4018000 tx=0x7fc3e4004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:38.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.936+0000 7fc3e8ff9700 1 -- 172.21.15.40:0/4033655469 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e401e7d0 con 0x7fc3cc007020 2024-09-13T18:17:38.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.940+0000 7fc3e8ff9700 1 -- 172.21.15.40:0/4033655469 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4450 (secure 0 0 0) 0x7fc3e4022070 con 0x7fc3cc007020 2024-09-13T18:17:38.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.940+0000 7fc3e8ff9700 1 -- 172.21.15.40:0/4033655469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3cc007020 msgr2=0x7fc3cc007490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:38.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.940+0000 7fc3e8ff9700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3cc007020 0x7fc3cc007490 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4018000 tx=0x7fc3e4004af0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.940+0000 7fc3e8ff9700 1 -- 172.21.15.40:0/4033655469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3cc007020 msgr2=0x7fc3cc007490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:17:38.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.940+0000 7fc3e8ff9700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3cc007020 0x7fc3cc007490 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4018000 tx=0x7fc3e4004af0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.947 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:17:38.947 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:17:38.947 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:17:38.947 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:17:38.947 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 218, 2024-09-13T18:17:38.947 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:17:38.947 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:17:38.947 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:17:38.947 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:17:38.947 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:38.947 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:17:38.948 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:17:38.948 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:17:38.948 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:17:38.948 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:17:38.948 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:17:38.948 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:17:38.948 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:17:38.948 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:17:38.948 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:17:38.948 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:17:38.948 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:17:38.948 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:17:38.948 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:17:38.949 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:38.949 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:17:38.949 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:17:38.949 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:17:38.949 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:17:38.949 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:17:38.949 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:17:38.949 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:17:38.949 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:17:38.949 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:17:38.949 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:17:38.949 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:17:38.949 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:17:38.949 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:38.950 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.950 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:17:38.950 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:17:38.950 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:17:38.950 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:17:38.950 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:17:38.950 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:17:38.950 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:17:38.950 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:17:38.950 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:17:38.950 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:17:38.950 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:17:38.950 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:17:38.950 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:17:38.950 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:17:38.951 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:17:38.951 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:17:38.951 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:17:38.951 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:17:38.951 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:17:38.951 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:17:38.951 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:17:38.951 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:17:38.951 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:17:38.951 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:17:38.951 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:17:38.951 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:17:38.951 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:17:38.951 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:38.952 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:17:38.952 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.952 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:17:38.952 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.952 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:17:38.952 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:17:38.952 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:17:38.952 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:17:38.952 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.952 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:17:38.952 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:17:38.952 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:38.952 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.952 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.953 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:17:38.953 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:17:38.953 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:38.953 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:38.953 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:38.953 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.953 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:17:38.953 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:17:38.953 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:17:38.953 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:17:38.953 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:17:38.953 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.953 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.953 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:17:38.954 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:17:38.954 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:17:38.954 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:17:38.954 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.954 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:17:38.954 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:17:38.954 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:38.954 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.954 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.954 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:17:38.954 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:17:38.954 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:38.954 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:38.955 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:38.955 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.955 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:17:38.955 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:17:38.955 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:17:38.955 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:17:38.955 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:17:38.955 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.955 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.955 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:17:38.955 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:17:38.955 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:17:38.955 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:17:38.955 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.956 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:17:38.956 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:17:38.956 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:38.956 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.956 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.956 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:17:38.956 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:17:38.956 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:38.956 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:38.956 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:38.956 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.956 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:17:38.956 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:17:38.956 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:17:38.957 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:17:38.957 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:17:38.957 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:38.957 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:38.957 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.957 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:17:38.957 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:17:38.957 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.957 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:38.957 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:38.957 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:17:38.957 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:38.957 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:38.957 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:17:38.958 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:38.958 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:38.958 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:38.958 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:17:38.958 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:38.958 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:38.958 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:38.958 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:17:38.958 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:17:38.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.940+0000 7fc3f242b700 1 -- 172.21.15.40:0/4033655469 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc3d8066800 msgr2=0x7fc3d8068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:38.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.940+0000 7fc3f242b700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc3d8066800 0x7fc3d8068cc0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009ba0 tx=0x7fc3e0010000 comp rx=0 tx=0).stop 2024-09-13T18:17:38.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.940+0000 7fc3f09c8700 1 -- 172.21.15.40:0/4033655469 reap_dead start 2024-09-13T18:17:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.940+0000 7fc3f242b700 1 -- 172.21.15.40:0/4033655469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3ec15e6b0 msgr2=0x7fc3ec159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.940+0000 7fc3f242b700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3ec15e6b0 0x7fc3ec159e50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00c520 tx=0x7fc3dc00c8e0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.940+0000 7fc3f242b700 1 -- 172.21.15.40:0/4033655469 shutdown_connections 2024-09-13T18:17:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.940+0000 7fc3f242b700 1 --2- 172.21.15.40:0/4033655469 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc3ec15e6b0 0x7fc3ec159e50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.940+0000 7fc3f242b700 1 -- 172.21.15.40:0/4033655469 >> 172.21.15.40:0/4033655469 conn(0x7fc3ec066090 msgr2=0x7fc3ec06d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.940+0000 7fc3f242b700 1 -- 172.21.15.40:0/4033655469 shutdown_connections 2024-09-13T18:17:38.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:38.940+0000 7fc3f242b700 1 -- 172.21.15.40:0/4033655469 wait complete. 2024-09-13T18:17:39.004 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:17:39.004 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:17:39.005 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:17:39.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.416+0000 7fba7b59e700 1 Processor -- start 2024-09-13T18:17:39.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.416+0000 7fba7b59e700 1 -- start start 2024-09-13T18:17:39.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.416+0000 7fba7b59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fba7c06aaa0 0x7fba7c169100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:39.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.416+0000 7fba7b59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba7c071690 con 0x7fba7c15e1a0 2024-09-13T18:17:39.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.416+0000 7fba7b59e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba7c154620 con 0x7fba7c154240 2024-09-13T18:17:39.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.416+0000 7fba7b59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7c0721e0 con 0x7fba7c06aaa0 2024-09-13T18:17:39.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.420+0000 7fba7ad9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fba7c06aaa0 0x7fba7c169100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:39.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.420+0000 7fba7ad9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fba7c06aaa0 0x7fba7c169100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46404/0 (socket says 172.21.15.40:46404) 2024-09-13T18:17:39.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.420+0000 7fba7ad9d700 1 -- 172.21.15.40:0/1279342630 learned_addr learned my addr 172.21.15.40:0/1279342630 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:17:39.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.420+0000 7fba7ad9d700 1 -- 172.21.15.40:0/1279342630 >> v1:172.21.15.43:6789/0 conn(0x7fba7c154240 legacy=0x7fba7c15ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:17:39.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.420+0000 7fba7ad9d700 1 -- 172.21.15.40:0/1279342630 >> v1:172.21.15.40:6789/0 conn(0x7fba7c15e1a0 legacy=0x7fba7c06a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:17:39.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.424+0000 7fba7ad9d700 1 -- 172.21.15.40:0/1279342630 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7c071b20 con 0x7fba7c06aaa0 2024-09-13T18:17:39.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.424+0000 7fba7ad9d700 1 --2- 172.21.15.40:0/1279342630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba7c06aaa0 0x7fba7c169100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fba74009e50 tx=0x7fba7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0c6520295640e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:39.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.424+0000 7fba7959a700 1 -- 172.21.15.40:0/1279342630 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7400e070 con 0x7fba7c06aaa0 2024-09-13T18:17:39.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.424+0000 7fba7959a700 1 -- 172.21.15.40:0/1279342630 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba74003a40 con 0x7fba7c06aaa0 2024-09-13T18:17:39.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.424+0000 7fba7959a700 1 -- 172.21.15.40:0/1279342630 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba74003df0 con 0x7fba7c06aaa0 2024-09-13T18:17:39.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.428+0000 7fba7b59e700 1 -- 172.21.15.40:0/1279342630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba7c06aaa0 msgr2=0x7fba7c169100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:39.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.428+0000 7fba7b59e700 1 --2- 172.21.15.40:0/1279342630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba7c06aaa0 0x7fba7c169100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fba74009e50 tx=0x7fba7400b330 comp rx=0 tx=0).stop 2024-09-13T18:17:39.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.428+0000 7fba7b59e700 1 -- 172.21.15.40:0/1279342630 shutdown_connections 2024-09-13T18:17:39.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.428+0000 7fba7b59e700 1 --2- 172.21.15.40:0/1279342630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba7c06aaa0 0x7fba7c169100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:39.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.428+0000 7fba7b59e700 1 -- 172.21.15.40:0/1279342630 >> 172.21.15.40:0/1279342630 conn(0x7fba7c0cabc0 msgr2=0x7fba7c0cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:39.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.444+0000 7fba7b59e700 1 -- 172.21.15.40:0/1279342630 shutdown_connections 2024-09-13T18:17:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.448+0000 7fba7b59e700 1 -- 172.21.15.40:0/1279342630 wait complete. 2024-09-13T18:17:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.452+0000 7fba7b59e700 1 Processor -- start 2024-09-13T18:17:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.452+0000 7fba7b59e700 1 -- start start 2024-09-13T18:17:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.452+0000 7fba7b59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba7c06aaa0 0x7fba7c1582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.452+0000 7fba7b59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba7c154240 0x7fba7c0c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.452+0000 7fba7b59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba7c15e1a0 0x7fba7c150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.452+0000 7fba7b59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c0c45b0 con 0x7fba7c154240 2024-09-13T18:17:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.452+0000 7fba7b59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c158db0 con 0x7fba7c15e1a0 2024-09-13T18:17:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.452+0000 7fba7b59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c159c10 con 0x7fba7c06aaa0 2024-09-13T18:17:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.452+0000 7fba79d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba7c154240 0x7fba7c0c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:39.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.452+0000 7fba79d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba7c154240 0x7fba7c0c52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46418/0 (socket says 172.21.15.40:46418) 2024-09-13T18:17:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.452+0000 7fba79d9b700 1 -- 172.21.15.40:0/2482387434 learned_addr learned my addr 172.21.15.40:0/2482387434 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:17:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.452+0000 7fba7ad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba7c15e1a0 0x7fba7c150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:39.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.452+0000 7fba7a59c700 1 --2- 172.21.15.40:0/2482387434 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba7c06aaa0 0x7fba7c1582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:39.463 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.456+0000 7fba7a59c700 1 -- 172.21.15.40:0/2482387434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba7c15e1a0 msgr2=0x7fba7c150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:39.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.456+0000 7fba7a59c700 1 --2- 172.21.15.40:0/2482387434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba7c15e1a0 0x7fba7c150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:39.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.456+0000 7fba7a59c700 1 -- 172.21.15.40:0/2482387434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba7c154240 msgr2=0x7fba7c0c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:39.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.456+0000 7fba7a59c700 1 --2- 172.21.15.40:0/2482387434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba7c154240 0x7fba7c0c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:39.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.464+0000 7fba7a59c700 1 -- 172.21.15.40:0/2482387434 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba74009b30 con 0x7fba7c06aaa0 2024-09-13T18:17:39.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.464+0000 7fba79d9b700 1 --2- 172.21.15.40:0/2482387434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba7c154240 0x7fba7c0c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:17:39.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.464+0000 7fba7ad9d700 1 --2- 172.21.15.40:0/2482387434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba7c15e1a0 0x7fba7c150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:17:39.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.464+0000 7fba7a59c700 1 --2- 172.21.15.40:0/2482387434 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba7c06aaa0 0x7fba7c1582b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00c400 tx=0x7fba6c00c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:39.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.472+0000 7fba6affd700 1 -- 172.21.15.40:0/2482387434 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0054e0 con 0x7fba7c06aaa0 2024-09-13T18:17:39.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.472+0000 7fba7b59e700 1 -- 172.21.15.40:0/2482387434 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7c159000 con 0x7fba7c06aaa0 2024-09-13T18:17:39.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.472+0000 7fba7b59e700 1 -- 172.21.15.40:0/2482387434 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7c1595c0 con 0x7fba7c06aaa0 2024-09-13T18:17:39.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.472+0000 7fba6affd700 1 -- 172.21.15.40:0/2482387434 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6c0036a0 con 0x7fba7c06aaa0 2024-09-13T18:17:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.472+0000 7fba6affd700 1 -- 172.21.15.40:0/2482387434 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c003a50 con 0x7fba7c06aaa0 2024-09-13T18:17:39.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.476+0000 7fba7b59e700 1 -- 172.21.15.40:0/2482387434 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba60005310 con 0x7fba7c06aaa0 2024-09-13T18:17:39.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.500+0000 7fba6affd700 1 -- 172.21.15.40:0/2482387434 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fba6c003bb0 con 0x7fba7c06aaa0 2024-09-13T18:17:39.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.500+0000 7fba6affd700 1 --2- 172.21.15.40:0/2482387434 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fba5c066780 0x7fba5c068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:39.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.500+0000 7fba6affd700 1 -- 172.21.15.40:0/2482387434 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fba6c083d40 con 0x7fba7c06aaa0 2024-09-13T18:17:39.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.500+0000 7fba6affd700 1 -- 172.21.15.40:0/2482387434 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fba6c0afdb0 con 0x7fba7c06aaa0 2024-09-13T18:17:39.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.500+0000 7fba79d9b700 1 --2- 172.21.15.40:0/2482387434 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fba5c066780 0x7fba5c068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:39.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:39.500+0000 7fba79d9b700 1 --2- 172.21.15.40:0/2482387434 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fba5c066780 0x7fba5c068c40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fba7c0cb3f0 tx=0x7fba70009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:40.203 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:17:40.210 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:17:40.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:40.360+0000 7fba7b59e700 1 -- 172.21.15.40:0/2482387434 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fba60005890 con 0x7fba7c06aaa0 2024-09-13T18:17:40.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:40.364+0000 7fba6affd700 1 -- 172.21.15.40:0/2482387434 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fba6c017080 con 0x7fba7c06aaa0 2024-09-13T18:17:40.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:40.388+0000 7fba68ff9700 1 -- 172.21.15.40:0/2482387434 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fba5c066780 msgr2=0x7fba5c068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:40.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:40.388+0000 7fba68ff9700 1 --2- 172.21.15.40:0/2482387434 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fba5c066780 0x7fba5c068c40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fba7c0cb3f0 tx=0x7fba70009040 comp rx=0 tx=0).stop 2024-09-13T18:17:40.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:40.388+0000 7fba68ff9700 1 -- 172.21.15.40:0/2482387434 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba7c06aaa0 msgr2=0x7fba7c1582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:40.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:40.388+0000 7fba68ff9700 1 --2- 172.21.15.40:0/2482387434 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba7c06aaa0 0x7fba7c1582b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00c400 tx=0x7fba6c00c710 comp rx=0 tx=0).stop 2024-09-13T18:17:40.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:40.388+0000 7fba68ff9700 1 -- 172.21.15.40:0/2482387434 shutdown_connections 2024-09-13T18:17:40.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:40.388+0000 7fba68ff9700 1 --2- 172.21.15.40:0/2482387434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba7c15e1a0 0x7fba7c150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:40.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:40.388+0000 7fba68ff9700 1 --2- 172.21.15.40:0/2482387434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba7c154240 0x7fba7c0c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:40.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:40.388+0000 7fba68ff9700 1 --2- 172.21.15.40:0/2482387434 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fba5c066780 0x7fba5c068c40 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:40.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:40.388+0000 7fba68ff9700 1 --2- 172.21.15.40:0/2482387434 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fba7c06aaa0 0x7fba7c1582b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:40.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:40.388+0000 7fba68ff9700 1 -- 172.21.15.40:0/2482387434 >> 172.21.15.40:0/2482387434 conn(0x7fba7c0cabc0 msgr2=0x7fba7c15d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:40.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:40.388+0000 7fba68ff9700 1 -- 172.21.15.40:0/2482387434 shutdown_connections 2024-09-13T18:17:40.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:40.388+0000 7fba68ff9700 1 -- 172.21.15.40:0/2482387434 wait complete. 2024-09-13T18:17:40.426 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:17:50.427 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:17:50.428 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:17:50.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.580+0000 7fc789e2d700 1 Processor -- start 2024-09-13T18:17:50.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.580+0000 7fc789e2d700 1 -- start start 2024-09-13T18:17:50.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.580+0000 7fc789e2d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc7841505c0 0x7fc7840c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:50.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.580+0000 7fc789e2d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7840c2a30 0x7fc784160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:50.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.580+0000 7fc789e2d700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7840c4bd0 con 0x7fc7840c5520 2024-09-13T18:17:50.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.580+0000 7fc789e2d700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7840c3670 con 0x7fc7841505c0 2024-09-13T18:17:50.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.580+0000 7fc789e2d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7840c7200 con 0x7fc7840c2a30 2024-09-13T18:17:50.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.580+0000 7fc7837fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc7841505c0 0x7fc7840c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:50.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.580+0000 7fc7837fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc7841505c0 0x7fc7840c4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59252/0 (socket says 172.21.15.40:59252) 2024-09-13T18:17:50.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.580+0000 7fc7837fe700 1 -- 172.21.15.40:0/2147521282 learned_addr learned my addr 172.21.15.40:0/2147521282 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:17:50.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.584+0000 7fc783fff700 1 --2- 172.21.15.40:0/2147521282 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7840c2a30 0x7fc784160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:50.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.584+0000 7fc7837fe700 1 -- 172.21.15.40:0/2147521282 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7840c2a30 msgr2=0x7fc784160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:17:50.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.584+0000 7fc7837fe700 1 --2- 172.21.15.40:0/2147521282 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7840c2a30 0x7fc784160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:50.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.584+0000 7fc7837fe700 1 -- 172.21.15.40:0/2147521282 >> v1:172.21.15.43:6789/0 conn(0x7fc7840c5520 legacy=0x7fc7840cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:50.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.584+0000 7fc7837fe700 1 -- 172.21.15.40:0/2147521282 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7840c74e0 con 0x7fc7841505c0 2024-09-13T18:17:50.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.584+0000 7fc783fff700 1 --2- 172.21.15.40:0/2147521282 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7840c2a30 0x7fc784160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:17:50.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.584+0000 7fc7837fe700 1 --2- 172.21.15.40:0/2147521282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7841505c0 0x7fc7840c4fe0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00d150 tx=0x7fc77c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c440dc61eda95f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:50.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.584+0000 7fc7827fc700 1 -- 172.21.15.40:0/2147521282 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c010070 con 0x7fc7841505c0 2024-09-13T18:17:50.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.584+0000 7fc7827fc700 1 -- 172.21.15.40:0/2147521282 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c00ddc0 con 0x7fc7841505c0 2024-09-13T18:17:50.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.584+0000 7fc7827fc700 1 -- 172.21.15.40:0/2147521282 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c00b310 con 0x7fc7841505c0 2024-09-13T18:17:50.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.584+0000 7fc789e2d700 1 -- 172.21.15.40:0/2147521282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7841505c0 msgr2=0x7fc7840c4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:50.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.584+0000 7fc789e2d700 1 --2- 172.21.15.40:0/2147521282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7841505c0 0x7fc7840c4fe0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00d150 tx=0x7fc77c00d460 comp rx=0 tx=0).stop 2024-09-13T18:17:50.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.584+0000 7fc789e2d700 1 -- 172.21.15.40:0/2147521282 shutdown_connections 2024-09-13T18:17:50.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.584+0000 7fc789e2d700 1 --2- 172.21.15.40:0/2147521282 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7840c2a30 0x7fc784160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:50.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.584+0000 7fc789e2d700 1 --2- 172.21.15.40:0/2147521282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7841505c0 0x7fc7840c4fe0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:50.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.584+0000 7fc789e2d700 1 -- 172.21.15.40:0/2147521282 >> 172.21.15.40:0/2147521282 conn(0x7fc78406a390 msgr2=0x7fc78406a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:50.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.584+0000 7fc789e2d700 1 -- 172.21.15.40:0/2147521282 shutdown_connections 2024-09-13T18:17:50.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.584+0000 7fc789e2d700 1 -- 172.21.15.40:0/2147521282 wait complete. 2024-09-13T18:17:50.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc789e2d700 1 Processor -- start 2024-09-13T18:17:50.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc789e2d700 1 -- start start 2024-09-13T18:17:50.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc789e2d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7840c2a30 0x7fc784146e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:50.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc789e2d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7840c5520 0x7fc784147350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:50.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc789e2d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7841505c0 0x7fc784144ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:50.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc789e2d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7841570d0 con 0x7fc7840c2a30 2024-09-13T18:17:50.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc7837fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7840c2a30 0x7fc784146e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:50.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc789e2d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc784146090 con 0x7fc7841505c0 2024-09-13T18:17:50.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc7837fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7840c2a30 0x7fc784146e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59266/0 (socket says 172.21.15.40:59266) 2024-09-13T18:17:50.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc7837fe700 1 -- 172.21.15.40:0/1348246721 learned_addr learned my addr 172.21.15.40:0/1348246721 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:17:50.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc789e2d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7841eb6a0 con 0x7fc7840c5520 2024-09-13T18:17:50.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc782ffd700 1 --2- 172.21.15.40:0/1348246721 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7840c5520 0x7fc784147350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:50.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc783fff700 1 --2- 172.21.15.40:0/1348246721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7841505c0 0x7fc784144ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:50.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc7837fe700 1 -- 172.21.15.40:0/1348246721 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7840c5520 msgr2=0x7fc784147350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:50.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc7837fe700 1 --2- 172.21.15.40:0/1348246721 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7840c5520 0x7fc784147350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:50.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc7837fe700 1 -- 172.21.15.40:0/1348246721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7841505c0 msgr2=0x7fc784144ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:50.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc7837fe700 1 --2- 172.21.15.40:0/1348246721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7841505c0 0x7fc784144ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:50.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc7837fe700 1 -- 172.21.15.40:0/1348246721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c00bd40 con 0x7fc7840c2a30 2024-09-13T18:17:50.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc783fff700 1 --2- 172.21.15.40:0/1348246721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7841505c0 0x7fc784144ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:17:50.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc7837fe700 1 --2- 172.21.15.40:0/1348246721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7840c2a30 0x7fc784146e10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc78406b2c0 tx=0x7fc77c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:50.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc76bfff700 1 -- 172.21.15.40:0/1348246721 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c004240 con 0x7fc7840c2a30 2024-09-13T18:17:50.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc76bfff700 1 -- 172.21.15.40:0/1348246721 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c0043a0 con 0x7fc7840c2a30 2024-09-13T18:17:50.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc789e2d700 1 -- 172.21.15.40:0/1348246721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc784146330 con 0x7fc7840c2a30 2024-09-13T18:17:50.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc76bfff700 1 -- 172.21.15.40:0/1348246721 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c004750 con 0x7fc7840c2a30 2024-09-13T18:17:50.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.588+0000 7fc789e2d700 1 -- 172.21.15.40:0/1348246721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7841468f0 con 0x7fc7840c2a30 2024-09-13T18:17:50.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.592+0000 7fc789e2d700 1 -- 172.21.15.40:0/1348246721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc784074db0 con 0x7fc7840c2a30 2024-09-13T18:17:50.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.592+0000 7fc76bfff700 1 -- 172.21.15.40:0/1348246721 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc77c028450 con 0x7fc7840c2a30 2024-09-13T18:17:50.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.592+0000 7fc76bfff700 1 --2- 172.21.15.40:0/1348246721 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc770066730 0x7fc770068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:50.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.592+0000 7fc782ffd700 1 --2- 172.21.15.40:0/1348246721 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc770066730 0x7fc770068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:50.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.592+0000 7fc76bfff700 1 -- 172.21.15.40:0/1348246721 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc77c088020 con 0x7fc7840c2a30 2024-09-13T18:17:50.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.596+0000 7fc782ffd700 1 --2- 172.21.15.40:0/1348246721 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc770066730 0x7fc770068bf0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc7840cbc60 tx=0x7fc778009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:50.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:50.608+0000 7fc76bfff700 1 -- 172.21.15.40:0/1348246721 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc77c054fd0 con 0x7fc7840c2a30 2024-09-13T18:17:51.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.436+0000 7fc789e2d700 1 -- 172.21.15.40:0/1348246721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fc7840cc6e0 con 0x7fc7840c2a30 2024-09-13T18:17:51.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.436+0000 7fc76bfff700 1 -- 172.21.15.40:0/1348246721 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7fc77c023620 con 0x7fc7840c2a30 2024-09-13T18:17:51.445 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":218,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:17:51.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.440+0000 7fc769ffb700 1 -- 172.21.15.40:0/1348246721 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc770066730 msgr2=0x7fc770068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:51.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.440+0000 7fc769ffb700 1 --2- 172.21.15.40:0/1348246721 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc770066730 0x7fc770068bf0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc7840cbc60 tx=0x7fc778009040 comp rx=0 tx=0).stop 2024-09-13T18:17:51.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.440+0000 7fc769ffb700 1 -- 172.21.15.40:0/1348246721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7840c2a30 msgr2=0x7fc784146e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:51.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.440+0000 7fc769ffb700 1 --2- 172.21.15.40:0/1348246721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7840c2a30 0x7fc784146e10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc78406b2c0 tx=0x7fc77c004af0 comp rx=0 tx=0).stop 2024-09-13T18:17:51.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.440+0000 7fc769ffb700 1 -- 172.21.15.40:0/1348246721 shutdown_connections 2024-09-13T18:17:51.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.440+0000 7fc769ffb700 1 --2- 172.21.15.40:0/1348246721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7841505c0 0x7fc784144ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:51.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.440+0000 7fc769ffb700 1 --2- 172.21.15.40:0/1348246721 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc770066730 0x7fc770068bf0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:51.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.440+0000 7fc769ffb700 1 --2- 172.21.15.40:0/1348246721 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7840c5520 0x7fc784147350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:51.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.440+0000 7fc769ffb700 1 --2- 172.21.15.40:0/1348246721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7840c2a30 0x7fc784146e10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:51.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.444+0000 7fc769ffb700 1 -- 172.21.15.40:0/1348246721 >> 172.21.15.40:0/1348246721 conn(0x7fc78406a390 msgr2=0x7fc7840c6170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:51.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.444+0000 7fc769ffb700 1 -- 172.21.15.40:0/1348246721 shutdown_connections 2024-09-13T18:17:51.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.444+0000 7fc769ffb700 1 -- 172.21.15.40:0/1348246721 wait complete. 2024-09-13T18:17:51.477 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:17:51.478 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T18:17:51.478 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:17:51.478 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:17:51.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.852+0000 7fe27d31a700 1 Processor -- start 2024-09-13T18:17:51.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.856+0000 7fe27d31a700 1 -- start start 2024-09-13T18:17:51.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.856+0000 7fe27d31a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe278157c60 0x7fe27806b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:51.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.856+0000 7fe27d31a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe27815e3c0 0x7fe27806da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:51.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.856+0000 7fe27d31a700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2780bfe80 con 0x7fe27806df80 2024-09-13T18:17:51.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.856+0000 7fe27d31a700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2780c5b80 con 0x7fe27815e3c0 2024-09-13T18:17:51.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.856+0000 7fe27d31a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2780c6480 con 0x7fe278157c60 2024-09-13T18:17:51.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.856+0000 7fe2767fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe27815e3c0 0x7fe27806da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:51.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.856+0000 7fe2767fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe27815e3c0 0x7fe27806da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:52244/0 (socket says 172.21.15.40:52244) 2024-09-13T18:17:51.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.856+0000 7fe2767fc700 1 -- 172.21.15.40:0/645943781 learned_addr learned my addr 172.21.15.40:0/645943781 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:17:51.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.856+0000 7fe276ffd700 1 --2- 172.21.15.40:0/645943781 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe278157c60 0x7fe27806b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:51.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.856+0000 7fe2767fc700 1 -- 172.21.15.40:0/645943781 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe278157c60 msgr2=0x7fe27806b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:17:51.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.856+0000 7fe2767fc700 1 --2- 172.21.15.40:0/645943781 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe278157c60 0x7fe27806b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:51.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.856+0000 7fe2767fc700 1 -- 172.21.15.40:0/645943781 >> v1:172.21.15.43:6789/0 conn(0x7fe27806df80 legacy=0x7fe2780c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:17:51.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.856+0000 7fe2767fc700 1 -- 172.21.15.40:0/645943781 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27806d2d0 con 0x7fe27815e3c0 2024-09-13T18:17:51.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.856+0000 7fe2767fc700 1 --2- 172.21.15.40:0/645943781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe27815e3c0 0x7fe27806da40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe270009e50 tx=0x7fe27000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7092110a48f30840 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:51.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.860+0000 7fe275ffb700 1 -- 172.21.15.40:0/645943781 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27000e070 con 0x7fe27815e3c0 2024-09-13T18:17:51.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.860+0000 7fe275ffb700 1 -- 172.21.15.40:0/645943781 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe270003a40 con 0x7fe27815e3c0 2024-09-13T18:17:51.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.860+0000 7fe27d31a700 1 -- 172.21.15.40:0/645943781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe27815e3c0 msgr2=0x7fe27806da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:51.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.860+0000 7fe27d31a700 1 --2- 172.21.15.40:0/645943781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe27815e3c0 0x7fe27806da40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe270009e50 tx=0x7fe27000b330 comp rx=0 tx=0).stop 2024-09-13T18:17:51.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.860+0000 7fe27d31a700 1 -- 172.21.15.40:0/645943781 shutdown_connections 2024-09-13T18:17:51.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.860+0000 7fe27d31a700 1 --2- 172.21.15.40:0/645943781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe27815e3c0 0x7fe27806da40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:51.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.860+0000 7fe27d31a700 1 --2- 172.21.15.40:0/645943781 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe278157c60 0x7fe27806b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:51.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.860+0000 7fe27d31a700 1 -- 172.21.15.40:0/645943781 >> 172.21.15.40:0/645943781 conn(0x7fe278066090 msgr2=0x7fe2780664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:51.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.860+0000 7fe27d31a700 1 -- 172.21.15.40:0/645943781 shutdown_connections 2024-09-13T18:17:51.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.860+0000 7fe27d31a700 1 -- 172.21.15.40:0/645943781 wait complete. 2024-09-13T18:17:51.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.864+0000 7fe27d31a700 1 Processor -- start 2024-09-13T18:17:51.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.864+0000 7fe27d31a700 1 -- start start 2024-09-13T18:17:51.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.864+0000 7fe27d31a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe27806df80 0x7fe278071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:51.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.864+0000 7fe27d31a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe278157c60 0x7fe27815ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:51.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.864+0000 7fe2767fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe278157c60 0x7fe27815ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:51.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.864+0000 7fe2767fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe278157c60 0x7fe27815ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55264/0 (socket says 172.21.15.40:55264) 2024-09-13T18:17:51.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.864+0000 7fe2767fc700 1 -- 172.21.15.40:0/2542424085 learned_addr learned my addr 172.21.15.40:0/2542424085 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:17:51.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.864+0000 7fe27d31a700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe27815e3c0 0x7fe27815f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:51.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.864+0000 7fe27d31a700 1 -- 172.21.15.40:0/2542424085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2780c2630 con 0x7fe278157c60 2024-09-13T18:17:51.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.864+0000 7fe27d31a700 1 -- 172.21.15.40:0/2542424085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2781321b0 con 0x7fe27815e3c0 2024-09-13T18:17:51.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.864+0000 7fe27d31a700 1 -- 172.21.15.40:0/2542424085 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe278160fc0 con 0x7fe27806df80 2024-09-13T18:17:51.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.868+0000 7fe2767fc700 1 -- 172.21.15.40:0/2542424085 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe27806df80 msgr2=0x7fe278071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:51.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.868+0000 7fe276ffd700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe27806df80 0x7fe278071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:51.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.868+0000 7fe2767fc700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe27806df80 0x7fe278071eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:51.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.868+0000 7fe2767fc700 1 -- 172.21.15.40:0/2542424085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe27815e3c0 msgr2=0x7fe27815f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:51.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.868+0000 7fe2767fc700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe27815e3c0 0x7fe27815f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:51.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.868+0000 7fe2767fc700 1 -- 172.21.15.40:0/2542424085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe270009b30 con 0x7fe278157c60 2024-09-13T18:17:51.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.868+0000 7fe2767fc700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe278157c60 0x7fe27815ed40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe27000b370 tx=0x7fe270004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:51.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.876+0000 7fe2677fe700 1 -- 172.21.15.40:0/2542424085 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe270004850 con 0x7fe278157c60 2024-09-13T18:17:51.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.876+0000 7fe27d31a700 1 -- 172.21.15.40:0/2542424085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe278131880 con 0x7fe278157c60 2024-09-13T18:17:51.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.876+0000 7fe27d31a700 1 -- 172.21.15.40:0/2542424085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe278131e40 con 0x7fe278157c60 2024-09-13T18:17:51.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.876+0000 7fe2677fe700 1 -- 172.21.15.40:0/2542424085 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe270012430 con 0x7fe278157c60 2024-09-13T18:17:51.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.876+0000 7fe2677fe700 1 -- 172.21.15.40:0/2542424085 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2700127e0 con 0x7fe278157c60 2024-09-13T18:17:51.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.884+0000 7fe27d31a700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe254002a40 0x7fe254004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:51.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.884+0000 7fe27d31a700 1 -- 172.21.15.40:0/2542424085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe254007f40 con 0x7fe254002a40 2024-09-13T18:17:51.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.884+0000 7fe276ffd700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe254002a40 0x7fe254004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:51.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.884+0000 7fe276ffd700 1 -- 172.21.15.40:0/2542424085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe254008100 con 0x7fe254002a40 2024-09-13T18:17:51.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.884+0000 7fe276ffd700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe254002a40 0x7fe254004f00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe2680107c0 tx=0x7fe268010b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:51.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.888+0000 7fe2677fe700 1 -- 172.21.15.40:0/2542424085 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe270012940 con 0x7fe278157c60 2024-09-13T18:17:51.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.888+0000 7fe2677fe700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe260066920 0x7fe260068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:51.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.888+0000 7fe2677fe700 1 -- 172.21.15.40:0/2542424085 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe268014800 con 0x7fe254002a40 2024-09-13T18:17:51.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.888+0000 7fe2677fe700 1 -- 172.21.15.40:0/2542424085 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe270085bf0 con 0x7fe278157c60 2024-09-13T18:17:51.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.888+0000 7fe2777fe700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe260066920 0x7fe260068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:51.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.888+0000 7fe2777fe700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe260066920 0x7fe260068de0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe26c003db0 tx=0x7fe26c007a60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:51.904 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.896+0000 7fe2677fe700 1 -- 172.21.15.40:0/2542424085 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe268014960 con 0x7fe254002a40 2024-09-13T18:17:51.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.908+0000 7fe2677fe700 1 -- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe254002a40 msgr2=0x7fe254004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:51.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.908+0000 7fe2677fe700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe254002a40 0x7fe254004f00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe2680107c0 tx=0x7fe268010b80 comp rx=0 tx=0).stop 2024-09-13T18:17:51.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.916+0000 7fe2677fe700 1 -- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe254002a40 msgr2=0x7fe254004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:17:51.924 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.916+0000 7fe2677fe700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe254002a40 0x7fe254004f00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:51.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.924+0000 7fe27d31a700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe254006fb0 0x7fe254007440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:51.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.928+0000 7fe27d31a700 1 -- 172.21.15.40:0/2542424085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25400a4a0 con 0x7fe254006fb0 2024-09-13T18:17:51.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.928+0000 7fe276ffd700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe254006fb0 0x7fe254007440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:51.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.928+0000 7fe276ffd700 1 -- 172.21.15.40:0/2542424085 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe254007980 con 0x7fe254006fb0 2024-09-13T18:17:51.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.928+0000 7fe276ffd700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe254006fb0 0x7fe254007440 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe2680107c0 tx=0x7fe268004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:51.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.928+0000 7fe2677fe700 1 -- 172.21.15.40:0/2542424085 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe268020050 con 0x7fe254006fb0 2024-09-13T18:17:51.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.928+0000 7fe2677fe700 1 -- 172.21.15.40:0/2542424085 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4785 (secure 0 0 0) 0x7fe268025070 con 0x7fe254006fb0 2024-09-13T18:17:51.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.928+0000 7fe2677fe700 1 -- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe254006fb0 msgr2=0x7fe254007440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:51.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.928+0000 7fe2677fe700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe254006fb0 0x7fe254007440 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe2680107c0 tx=0x7fe268004770 comp rx=0 tx=0).stop 2024-09-13T18:17:51.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.928+0000 7fe2677fe700 1 -- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe254006fb0 msgr2=0x7fe254007440 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:17:51.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.928+0000 7fe2677fe700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe254006fb0 0x7fe254007440 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:51.937 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:17:51.937 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:17:51.938 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:17:51.938 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:17:51.938 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 218, 2024-09-13T18:17:51.938 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:17:51.938 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:17:51.938 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:17:51.938 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:17:51.938 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:51.938 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 14, 2024-09-13T18:17:51.938 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:17:51.938 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:17:51.938 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:17:51.938 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:17:51.938 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:17:51.939 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:17:51.939 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:17:51.939 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:17:51.939 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:17:51.939 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:17:51.939 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:17:51.939 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:17:51.939 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:17:51.939 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:51.939 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:17:51.939 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:17:51.939 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:17:51.939 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:17:51.939 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:17:51.940 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:17:51.940 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:17:51.940 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:17:51.940 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:17:51.940 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:17:51.940 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:17:51.940 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:17:51.940 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:51.940 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:51.940 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:17:51.941 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:17:51.941 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:17:51.941 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:17:51.941 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:17:51.941 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:17:51.941 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:17:51.941 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:17:51.941 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:17:51.941 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:17:51.941 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:17:51.941 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:17:51.941 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:17:51.941 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:17:51.941 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:17:51.942 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:17:51.942 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:17:51.942 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:17:51.942 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:17:51.942 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:17:51.942 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:17:51.942 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:17:51.942 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:17:51.942 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:17:51.942 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:17:51.942 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:17:51.942 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:17:51.942 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:51.943 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:17:51.943 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:51.943 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:17:51.943 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:51.943 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:17:51.943 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:17:51.943 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:17:51.943 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:17:51.943 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:51.943 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:17:51.943 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:17:51.943 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:51.944 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:51.944 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:51.944 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:17:51.944 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:17:51.944 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:51.944 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:51.944 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:51.944 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:51.944 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:17:51.944 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:17:51.944 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:17:51.945 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:17:51.945 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:17:51.945 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:51.945 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:51.945 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:17:51.945 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:17:51.945 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:17:51.945 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:17:51.945 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:51.945 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:17:51.945 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:17:51.945 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:51.945 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:51.945 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:51.946 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:17:51.946 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:17:51.946 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:51.946 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:51.946 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:51.946 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:51.946 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:17:51.946 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:17:51.946 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:17:51.946 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:17:51.946 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:17:51.946 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:51.946 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:51.947 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:17:51.947 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:17:51.947 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:17:51.947 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:17:51.947 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:51.947 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:17:51.947 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:17:51.947 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:51.947 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:51.947 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:51.947 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:17:51.947 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:17:51.947 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:51.947 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:51.948 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:51.948 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:51.948 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:17:51.948 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:17:51.948 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:17:51.948 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:17:51.948 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:17:51.948 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:51.948 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:51.948 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:51.948 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:17:51.948 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:17:51.948 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:51.948 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:51.949 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:51.949 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:17:51.949 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:51.949 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:51.949 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:17:51.949 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:51.949 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:51.949 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:51.949 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:17:51.949 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:51.949 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:51.949 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:17:51.949 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:51.949 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:51.950 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:51.950 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:17:51.950 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:51.950 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:51.950 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:17:51.950 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:51.950 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:51.950 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:51.950 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:17:51.950 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:51.950 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:51.950 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:51.950 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:17:51.950 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:17:51.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.944+0000 7fe2657fa700 1 -- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe260066920 msgr2=0x7fe260068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:51.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.944+0000 7fe2657fa700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe260066920 0x7fe260068de0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe26c003db0 tx=0x7fe26c007a60 comp rx=0 tx=0).stop 2024-09-13T18:17:51.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.944+0000 7fe2777fe700 1 -- 172.21.15.40:0/2542424085 reap_dead start 2024-09-13T18:17:51.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.944+0000 7fe2657fa700 1 -- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe278157c60 msgr2=0x7fe27815ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:51.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.944+0000 7fe2657fa700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe278157c60 0x7fe27815ed40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe27000b370 tx=0x7fe270004af0 comp rx=0 tx=0).stop 2024-09-13T18:17:51.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.944+0000 7fe2657fa700 1 -- 172.21.15.40:0/2542424085 shutdown_connections 2024-09-13T18:17:51.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.944+0000 7fe2657fa700 1 --2- 172.21.15.40:0/2542424085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe278157c60 0x7fe27815ed40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:51.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.944+0000 7fe2657fa700 1 -- 172.21.15.40:0/2542424085 >> 172.21.15.40:0/2542424085 conn(0x7fe278066090 msgr2=0x7fe2780c7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:51.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.944+0000 7fe2657fa700 1 -- 172.21.15.40:0/2542424085 shutdown_connections 2024-09-13T18:17:51.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:51.944+0000 7fe2657fa700 1 -- 172.21.15.40:0/2542424085 wait complete. 2024-09-13T18:17:51.988 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:17:51.989 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:17:51.989 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:17:52.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.392+0000 7fe933926700 1 Processor -- start 2024-09-13T18:17:52.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.392+0000 7fe933926700 1 -- start start 2024-09-13T18:17:52.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.392+0000 7fe933926700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe92c0c7f60 0x7fe92c0c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:52.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.392+0000 7fe933926700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe92c0c8340 0x7fe92c0c6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:52.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.392+0000 7fe933926700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92c167290 con 0x7fe92c06b080 2024-09-13T18:17:52.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.392+0000 7fe933926700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe92c0c1630 con 0x7fe92c0c8340 2024-09-13T18:17:52.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.392+0000 7fe933926700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c166680 con 0x7fe92c0c7f60 2024-09-13T18:17:52.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.392+0000 7fe930ec1700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe92c0c8340 0x7fe92c0c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:52.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.392+0000 7fe930ec1700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe92c0c8340 0x7fe92c0c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:52258/0 (socket says 172.21.15.40:52258) 2024-09-13T18:17:52.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.392+0000 7fe930ec1700 1 -- 172.21.15.40:0/1975980848 learned_addr learned my addr 172.21.15.40:0/1975980848 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:17:52.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.396+0000 7fe930ec1700 1 -- 172.21.15.40:0/1975980848 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe92c0c7f60 msgr2=0x7fe92c0c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:17:52.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.396+0000 7fe930ec1700 1 --2- 172.21.15.40:0/1975980848 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe92c0c7f60 0x7fe92c0c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:52.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.396+0000 7fe930ec1700 1 -- 172.21.15.40:0/1975980848 >> v1:172.21.15.43:6789/0 conn(0x7fe92c06b080 legacy=0x7fe92c1635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:52.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.396+0000 7fe930ec1700 1 -- 172.21.15.40:0/1975980848 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c166960 con 0x7fe92c0c8340 2024-09-13T18:17:52.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.396+0000 7fe930ec1700 1 --2- 172.21.15.40:0/1975980848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe92c0c8340 0x7fe92c0c6190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe9240061e0 tx=0x7fe924007b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1022a19662b38e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:52.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.396+0000 7fe923fff700 1 -- 172.21.15.40:0/1975980848 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924011070 con 0x7fe92c0c8340 2024-09-13T18:17:52.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.396+0000 7fe923fff700 1 -- 172.21.15.40:0/1975980848 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92400cc70 con 0x7fe92c0c8340 2024-09-13T18:17:52.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.396+0000 7fe933926700 1 -- 172.21.15.40:0/1975980848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe92c0c8340 msgr2=0x7fe92c0c6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:52.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.396+0000 7fe933926700 1 --2- 172.21.15.40:0/1975980848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe92c0c8340 0x7fe92c0c6190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe9240061e0 tx=0x7fe924007b30 comp rx=0 tx=0).stop 2024-09-13T18:17:52.407 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.400+0000 7fe933926700 1 -- 172.21.15.40:0/1975980848 shutdown_connections 2024-09-13T18:17:52.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.400+0000 7fe933926700 1 --2- 172.21.15.40:0/1975980848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe92c0c8340 0x7fe92c0c6190 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:52.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.400+0000 7fe933926700 1 --2- 172.21.15.40:0/1975980848 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe92c0c7f60 0x7fe92c0c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:52.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.400+0000 7fe933926700 1 -- 172.21.15.40:0/1975980848 >> 172.21.15.40:0/1975980848 conn(0x7fe92c066090 msgr2=0x7fe92c0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:52.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.400+0000 7fe933926700 1 -- 172.21.15.40:0/1975980848 shutdown_connections 2024-09-13T18:17:52.408 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.400+0000 7fe933926700 1 -- 172.21.15.40:0/1975980848 wait complete. 2024-09-13T18:17:52.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.400+0000 7fe933926700 1 Processor -- start 2024-09-13T18:17:52.409 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.400+0000 7fe933926700 1 -- start start 2024-09-13T18:17:52.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.400+0000 7fe933926700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe92c06b080 0x7fe92c1fca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:52.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.400+0000 7fe933926700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe92c0c7f60 0x7fe92c0af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:52.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.400+0000 7fe933926700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe92c0c8340 0x7fe92c1ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:52.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.400+0000 7fe9316c2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe92c06b080 0x7fe92c1fca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:52.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.404+0000 7fe9316c2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe92c06b080 0x7fe92c1fca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55302/0 (socket says 172.21.15.40:55302) 2024-09-13T18:17:52.412 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.404+0000 7fe930ec1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe92c0c7f60 0x7fe92c0af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:52.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.404+0000 7fe930ec1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe92c0c7f60 0x7fe92c0af3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:36480/0 (socket says 172.21.15.40:36480) 2024-09-13T18:17:52.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.404+0000 7fe9316c2700 1 -- 172.21.15.40:0/3229401123 learned_addr learned my addr 172.21.15.40:0/3229401123 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:17:52.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.404+0000 7fe933926700 1 -- 172.21.15.40:0/3229401123 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0ae520 con 0x7fe92c06b080 2024-09-13T18:17:52.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.404+0000 7fe933926700 1 -- 172.21.15.40:0/3229401123 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c2003b0 con 0x7fe92c0c8340 2024-09-13T18:17:52.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.404+0000 7fe933926700 1 -- 172.21.15.40:0/3229401123 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0aaac0 con 0x7fe92c0c7f60 2024-09-13T18:17:52.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.404+0000 7fe930ec1700 1 -- 172.21.15.40:0/3229401123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe92c0c8340 msgr2=0x7fe92c1ffe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:52.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.404+0000 7fe930ec1700 1 --2- 172.21.15.40:0/3229401123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe92c0c8340 0x7fe92c1ffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:52.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.404+0000 7fe930ec1700 1 -- 172.21.15.40:0/3229401123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe92c06b080 msgr2=0x7fe92c1fca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:52.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.404+0000 7fe930ec1700 1 --2- 172.21.15.40:0/3229401123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe92c06b080 0x7fe92c1fca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:52.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.404+0000 7fe930ec1700 1 -- 172.21.15.40:0/3229401123 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe924007530 con 0x7fe92c0c7f60 2024-09-13T18:17:52.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.408+0000 7fe930ec1700 1 --2- 172.21.15.40:0/3229401123 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe92c0c7f60 0x7fe92c0af3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe92400b000 tx=0x7fe924004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:52.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.408+0000 7fe921ffb700 1 -- 172.21.15.40:0/3229401123 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92401d070 con 0x7fe92c0c7f60 2024-09-13T18:17:52.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.412+0000 7fe933926700 1 -- 172.21.15.40:0/3229401123 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92c0aada0 con 0x7fe92c0c7f60 2024-09-13T18:17:52.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.412+0000 7fe933926700 1 -- 172.21.15.40:0/3229401123 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92c1fc240 con 0x7fe92c0c7f60 2024-09-13T18:17:52.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.416+0000 7fe921ffb700 1 -- 172.21.15.40:0/3229401123 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92401e040 con 0x7fe92c0c7f60 2024-09-13T18:17:52.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.416+0000 7fe921ffb700 1 -- 172.21.15.40:0/3229401123 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924004270 con 0x7fe92c0c7f60 2024-09-13T18:17:52.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.416+0000 7fe921ffb700 1 -- 172.21.15.40:0/3229401123 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe924015070 con 0x7fe92c0c7f60 2024-09-13T18:17:52.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.416+0000 7fe9177fe700 1 --2- 172.21.15.40:0/3229401123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c002a40 0x7fe90c004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:52.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.416+0000 7fe9177fe700 1 -- 172.21.15.40:0/3229401123 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c007f40 con 0x7fe90c002a40 2024-09-13T18:17:52.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.416+0000 7fe9316c2700 1 --2- 172.21.15.40:0/3229401123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c002a40 0x7fe90c004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:52.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.420+0000 7fe921ffb700 1 --2- 172.21.15.40:0/3229401123 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe9100668b0 0x7fe910068d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:52.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.420+0000 7fe9316c2700 1 -- 172.21.15.40:0/3229401123 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe90c008100 con 0x7fe90c002a40 2024-09-13T18:17:52.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.420+0000 7fe921ffb700 1 -- 172.21.15.40:0/3229401123 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe92408a0c0 con 0x7fe92c0c7f60 2024-09-13T18:17:52.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.420+0000 7fe931ec3700 1 --2- 172.21.15.40:0/3229401123 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe9100668b0 0x7fe910068d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:52.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.420+0000 7fe931ec3700 1 --2- 172.21.15.40:0/3229401123 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe9100668b0 0x7fe910068d70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe91c000f60 tx=0x7fe91c007a80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:52.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.420+0000 7fe9316c2700 1 --2- 172.21.15.40:0/3229401123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c002a40 0x7fe90c004f00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe92800e7a0 tx=0x7fe92800eb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:52.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.420+0000 7fe921ffb700 1 -- 172.21.15.40:0/3229401123 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe928013680 con 0x7fe90c002a40 2024-09-13T18:17:52.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.424+0000 7fe921ffb700 1 -- 172.21.15.40:0/3229401123 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe9280137e0 con 0x7fe90c002a40 2024-09-13T18:17:52.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.424+0000 7fe921ffb700 1 -- 172.21.15.40:0/3229401123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c002a40 msgr2=0x7fe90c004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:52.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.424+0000 7fe921ffb700 1 --2- 172.21.15.40:0/3229401123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c002a40 0x7fe90c004f00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe92800e7a0 tx=0x7fe92800eb60 comp rx=0 tx=0).stop 2024-09-13T18:17:52.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.424+0000 7fe921ffb700 1 -- 172.21.15.40:0/3229401123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c002a40 msgr2=0x7fe90c004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:17:52.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.424+0000 7fe921ffb700 1 --2- 172.21.15.40:0/3229401123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c002a40 0x7fe90c004f00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:52.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.452+0000 7fe933926700 1 --2- 172.21.15.40:0/3229401123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c001e20 0x7fe90c00cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:52.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.452+0000 7fe933926700 1 -- 172.21.15.40:0/3229401123 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c0023b0 con 0x7fe90c001e20 2024-09-13T18:17:52.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.452+0000 7fe9316c2700 1 --2- 172.21.15.40:0/3229401123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c001e20 0x7fe90c00cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:52.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.452+0000 7fe9316c2700 1 -- 172.21.15.40:0/3229401123 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe90c00a2e0 con 0x7fe90c001e20 2024-09-13T18:17:52.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.452+0000 7fe9316c2700 1 --2- 172.21.15.40:0/3229401123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c001e20 0x7fe90c00cbf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe928019000 tx=0x7fe928004880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:52.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.452+0000 7fe921ffb700 1 -- 172.21.15.40:0/3229401123 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe928022070 con 0x7fe90c001e20 2024-09-13T18:17:52.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.452+0000 7fe921ffb700 1 -- 172.21.15.40:0/3229401123 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4949 (secure 0 0 0) 0x7fe928005b50 con 0x7fe90c001e20 2024-09-13T18:17:52.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.452+0000 7fe921ffb700 1 -- 172.21.15.40:0/3229401123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c001e20 msgr2=0x7fe90c00cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:52.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.452+0000 7fe921ffb700 1 --2- 172.21.15.40:0/3229401123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c001e20 0x7fe90c00cbf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe928019000 tx=0x7fe928004880 comp rx=0 tx=0).stop 2024-09-13T18:17:52.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.452+0000 7fe921ffb700 1 -- 172.21.15.40:0/3229401123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c001e20 msgr2=0x7fe90c00cbf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:17:52.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.452+0000 7fe921ffb700 1 --2- 172.21.15.40:0/3229401123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe90c001e20 0x7fe90c00cbf0 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe928019000 tx=0x7fe928004880 comp rx=0 tx=0).stop 2024-09-13T18:17:52.465 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:17:52.465 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:17:52.465 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:17:52.466 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:17:52.466 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 218, 2024-09-13T18:17:52.466 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:17:52.466 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:17:52.466 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:17:52.466 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:17:52.466 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:52.466 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 15, 2024-09-13T18:17:52.466 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:17:52.466 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:17:52.466 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:17:52.466 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:17:52.466 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:17:52.466 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:17:52.467 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:17:52.467 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:17:52.467 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:17:52.467 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:17:52.467 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:17:52.467 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:17:52.467 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:17:52.467 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:52.467 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:17:52.467 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:17:52.467 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:17:52.467 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:17:52.467 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:17:52.467 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:17:52.468 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:17:52.468 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:17:52.468 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:17:52.468 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:17:52.468 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:17:52.468 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:17:52.468 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:52.468 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.468 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:17:52.468 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:17:52.468 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:17:52.468 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:17:52.468 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:17:52.469 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:17:52.469 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:17:52.469 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:17:52.469 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:17:52.469 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:17:52.469 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:17:52.469 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:17:52.469 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:17:52.469 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:17:52.469 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:17:52.469 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:17:52.469 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:17:52.469 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:17:52.469 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:17:52.470 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:17:52.470 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:17:52.470 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:17:52.470 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:17:52.470 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:17:52.470 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:17:52.470 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:17:52.470 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:17:52.470 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:52.470 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:17:52.470 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.470 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:17:52.470 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.470 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:17:52.471 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:17:52.471 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:17:52.471 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:17:52.471 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.471 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:17:52.471 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:17:52.471 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:52.471 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.471 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.471 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:17:52.471 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:17:52.471 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:52.472 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:52.472 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:52.472 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.472 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:17:52.472 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:17:52.472 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:17:52.472 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:17:52.472 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:17:52.472 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.472 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.472 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:17:52.472 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:17:52.472 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:17:52.472 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:17:52.473 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.473 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:17:52.473 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:17:52.473 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:52.473 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.473 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.473 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:17:52.473 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:17:52.473 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:52.473 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:52.473 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:52.473 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.473 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:17:52.474 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:17:52.474 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:17:52.474 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:17:52.474 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:17:52.474 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.474 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.474 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:17:52.474 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:17:52.474 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:17:52.474 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:17:52.474 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.474 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:17:52.474 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:17:52.474 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:52.475 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.475 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.475 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:17:52.475 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:17:52.475 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:52.475 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:52.475 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:52.475 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.475 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:17:52.475 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:17:52.476 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:17:52.476 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:17:52.476 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:17:52.476 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:52.476 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:52.477 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.477 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:17:52.477 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:17:52.477 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.477 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:52.477 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:52.478 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:17:52.478 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:52.478 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:52.478 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:17:52.478 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.478 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:52.479 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:52.479 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:17:52.479 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:52.479 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:52.479 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:17:52.480 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.480 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:52.480 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:52.480 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:17:52.480 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:52.480 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:52.481 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:17:52.481 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.481 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:52.481 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:52.481 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:17:52.482 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:52.482 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:52.482 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:17:52.482 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.482 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:52.482 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:52.483 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:17:52.483 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:52.483 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:52.483 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.483 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:17:52.484 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:17:52.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.460+0000 7fe933926700 1 -- 172.21.15.40:0/3229401123 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe9100668b0 msgr2=0x7fe910068d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:52.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.460+0000 7fe933926700 1 --2- 172.21.15.40:0/3229401123 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe9100668b0 0x7fe910068d70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe91c000f60 tx=0x7fe91c007a80 comp rx=0 tx=0).stop 2024-09-13T18:17:52.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.460+0000 7fe931ec3700 1 -- 172.21.15.40:0/3229401123 reap_dead start 2024-09-13T18:17:52.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.460+0000 7fe933926700 1 -- 172.21.15.40:0/3229401123 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe92c0c7f60 msgr2=0x7fe92c0af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.460+0000 7fe933926700 1 --2- 172.21.15.40:0/3229401123 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe92c0c7f60 0x7fe92c0af3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe92400b000 tx=0x7fe924004af0 comp rx=0 tx=0).stop 2024-09-13T18:17:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.460+0000 7fe933926700 1 -- 172.21.15.40:0/3229401123 shutdown_connections 2024-09-13T18:17:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.460+0000 7fe933926700 1 --2- 172.21.15.40:0/3229401123 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe92c0c7f60 0x7fe92c0af3c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.460+0000 7fe933926700 1 -- 172.21.15.40:0/3229401123 >> 172.21.15.40:0/3229401123 conn(0x7fe92c066090 msgr2=0x7fe92c0c5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.460+0000 7fe933926700 1 -- 172.21.15.40:0/3229401123 shutdown_connections 2024-09-13T18:17:52.488 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.460+0000 7fe933926700 1 -- 172.21.15.40:0/3229401123 wait complete. 2024-09-13T18:17:52.489 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:17:52.490 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:17:52.490 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:17:52.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.832+0000 7f801965a700 1 Processor -- start 2024-09-13T18:17:52.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.832+0000 7f801965a700 1 -- start start 2024-09-13T18:17:52.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.832+0000 7f801965a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f800c0d48e0 0x7f800c005cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:52.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.832+0000 7f801965a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f800c0061f0 0x7f800c034690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:52.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.832+0000 7f801965a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f800c035040 con 0x7f800c034bd0 2024-09-13T18:17:52.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.832+0000 7f801965a700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f800c0067d0 con 0x7f800c0061f0 2024-09-13T18:17:52.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.832+0000 7f801965a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800c006ab0 con 0x7f800c0d48e0 2024-09-13T18:17:52.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.832+0000 7f80127fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f800c0d48e0 0x7f800c005cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:52.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.832+0000 7f80127fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f800c0d48e0 0x7f800c005cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:36490/0 (socket says 172.21.15.40:36490) 2024-09-13T18:17:52.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.832+0000 7f80127fc700 1 -- 172.21.15.40:0/144768328 learned_addr learned my addr 172.21.15.40:0/144768328 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:17:52.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.832+0000 7f80127fc700 1 -- 172.21.15.40:0/144768328 >> v2:172.21.15.43:3300/0 conn(0x7f800c0061f0 msgr2=0x7f800c034690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:17:52.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.832+0000 7f80127fc700 1 --2- 172.21.15.40:0/144768328 >> v2:172.21.15.43:3300/0 conn(0x7f800c0061f0 0x7f800c034690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:52.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.832+0000 7f80127fc700 1 -- 172.21.15.40:0/144768328 >> v1:172.21.15.40:6789/0 conn(0x7f800c034bd0 legacy=0x7f800c0da660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:17:52.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.832+0000 7f80127fc700 1 -- 172.21.15.40:0/144768328 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800c006d90 con 0x7f800c0d48e0 2024-09-13T18:17:52.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.832+0000 7f80127fc700 1 --2- 172.21.15.40:0/144768328 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f800c0d48e0 0x7f800c005cb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8004007a60 tx=0x7f8004010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22756df80d385855 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:52.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.836+0000 7f8013fff700 1 -- 172.21.15.40:0/144768328 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8004010270 con 0x7f800c0d48e0 2024-09-13T18:17:52.845 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.836+0000 7f8013fff700 1 -- 172.21.15.40:0/144768328 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8004019a60 con 0x7f800c0d48e0 2024-09-13T18:17:52.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.844+0000 7f801965a700 1 -- 172.21.15.40:0/144768328 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f800c0d48e0 msgr2=0x7f800c005cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:52.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.844+0000 7f801965a700 1 --2- 172.21.15.40:0/144768328 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f800c0d48e0 0x7f800c005cb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8004007a60 tx=0x7f8004010800 comp rx=0 tx=0).stop 2024-09-13T18:17:52.850 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.844+0000 7f801965a700 1 -- 172.21.15.40:0/144768328 shutdown_connections 2024-09-13T18:17:52.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.844+0000 7f801965a700 1 --2- 172.21.15.40:0/144768328 >> v2:172.21.15.43:3300/0 conn(0x7f800c0061f0 0x7f800c034690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:52.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.844+0000 7f801965a700 1 --2- 172.21.15.40:0/144768328 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f800c0d48e0 0x7f800c005cb0 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8004007a60 tx=0x7f8004010800 comp rx=0 tx=0).stop 2024-09-13T18:17:52.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.844+0000 7f801965a700 1 -- 172.21.15.40:0/144768328 >> 172.21.15.40:0/144768328 conn(0x7f800c04feb0 msgr2=0x7f800c0502c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:52.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.844+0000 7f801965a700 1 -- 172.21.15.40:0/144768328 shutdown_connections 2024-09-13T18:17:52.851 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.844+0000 7f801965a700 1 -- 172.21.15.40:0/144768328 wait complete. 2024-09-13T18:17:52.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.852+0000 7f801965a700 1 Processor -- start 2024-09-13T18:17:52.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.852+0000 7f801965a700 1 -- start start 2024-09-13T18:17:52.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.852+0000 7f801965a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f800c0061f0 0x7f800c048110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:52.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.852+0000 7f801965a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800c034bd0 0x7f800c048650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:52.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.852+0000 7f801965a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f800c0dc800 0x7f800c049610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:52.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.852+0000 7f801965a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c0ccfb0 con 0x7f800c034bd0 2024-09-13T18:17:52.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.852+0000 7f801965a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c0ce230 con 0x7f800c0061f0 2024-09-13T18:17:52.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.852+0000 7f801965a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800c047600 con 0x7f800c0dc800 2024-09-13T18:17:52.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.852+0000 7f8012ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f800c0dc800 0x7f800c049610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:52.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.852+0000 7f8012ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f800c0dc800 0x7f800c049610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:36506/0 (socket says 172.21.15.40:36506) 2024-09-13T18:17:52.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.852+0000 7f8012ffd700 1 -- 172.21.15.40:0/3411158202 learned_addr learned my addr 172.21.15.40:0/3411158202 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:17:52.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.852+0000 7f8012ffd700 1 -- 172.21.15.40:0/3411158202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f800c0061f0 msgr2=0x7f800c048110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:17:52.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.852+0000 7f8012ffd700 1 --2- 172.21.15.40:0/3411158202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f800c0061f0 0x7f800c048110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:52.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.852+0000 7f8012ffd700 1 -- 172.21.15.40:0/3411158202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800c034bd0 msgr2=0x7f800c048650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:17:52.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.852+0000 7f8012ffd700 1 --2- 172.21.15.40:0/3411158202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f800c034bd0 0x7f800c048650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:52.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.852+0000 7f8012ffd700 1 -- 172.21.15.40:0/3411158202 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8004007790 con 0x7f800c0dc800 2024-09-13T18:17:52.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.852+0000 7f8012ffd700 1 --2- 172.21.15.40:0/3411158202 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f800c0dc800 0x7f800c049610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f800800b440 tx=0x7f800800b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:52.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.856+0000 7f7ffb7fe700 1 -- 172.21.15.40:0/3411158202 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008011830 con 0x7f800c0dc800 2024-09-13T18:17:52.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.856+0000 7f801965a700 1 -- 172.21.15.40:0/3411158202 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800c0d4e40 con 0x7f800c0dc800 2024-09-13T18:17:52.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.856+0000 7f801965a700 1 -- 172.21.15.40:0/3411158202 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c1712d0 con 0x7f800c0dc800 2024-09-13T18:17:52.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.856+0000 7f7ffb7fe700 1 -- 172.21.15.40:0/3411158202 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8008011990 con 0x7f800c0dc800 2024-09-13T18:17:52.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.856+0000 7f7ffb7fe700 1 -- 172.21.15.40:0/3411158202 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008011d40 con 0x7f800c0dc800 2024-09-13T18:17:52.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.860+0000 7f801965a700 1 --2- 172.21.15.40:0/3411158202 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7ff4002a40 0x7f7ff4004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:52.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.860+0000 7f801965a700 1 -- 172.21.15.40:0/3411158202 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff4007f40 con 0x7f7ff4002a40 2024-09-13T18:17:52.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.860+0000 7f80127fc700 1 --2- 172.21.15.40:0/3411158202 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7ff4002a40 0x7f7ff4004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:52.870 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.864+0000 7f80127fc700 1 -- 172.21.15.40:0/3411158202 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7ff4008100 con 0x7f7ff4002a40 2024-09-13T18:17:52.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.864+0000 7f7ffb7fe700 1 -- 172.21.15.40:0/3411158202 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f800801a400 con 0x7f800c0dc800 2024-09-13T18:17:52.871 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.864+0000 7f80127fc700 1 --2- 172.21.15.40:0/3411158202 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7ff4002a40 0x7f7ff4004f00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f800c00d2f0 tx=0x7f8004004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:52.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.864+0000 7f7ffb7fe700 1 --2- 172.21.15.40:0/3411158202 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8000066800 0x7f8000068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:52.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.864+0000 7f7ffb7fe700 1 -- 172.21.15.40:0/3411158202 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8008084460 con 0x7f800c0dc800 2024-09-13T18:17:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.864+0000 7f7ffb7fe700 1 -- 172.21.15.40:0/3411158202 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8004016070 con 0x7f7ff4002a40 2024-09-13T18:17:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.864+0000 7f7ffb7fe700 1 -- 172.21.15.40:0/3411158202 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f8004007c40 con 0x7f7ff4002a40 2024-09-13T18:17:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.864+0000 7f7ffb7fe700 1 -- 172.21.15.40:0/3411158202 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7ff4002a40 msgr2=0x7f7ff4004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.864+0000 7f7ffb7fe700 1 --2- 172.21.15.40:0/3411158202 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7ff4002a40 0x7f7ff4004f00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f800c00d2f0 tx=0x7f8004004af0 comp rx=0 tx=0).stop 2024-09-13T18:17:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.864+0000 7f7ffb7fe700 1 -- 172.21.15.40:0/3411158202 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7ff4002a40 msgr2=0x7f7ff4004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:17:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.864+0000 7f7ffb7fe700 1 --2- 172.21.15.40:0/3411158202 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7ff4002a40 0x7f7ff4004f00 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f800c00d2f0 tx=0x7f8004004af0 comp rx=0 tx=0).stop 2024-09-13T18:17:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.864+0000 7f8011ffb700 1 --2- 172.21.15.40:0/3411158202 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8000066800 0x7f8000068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:52.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.868+0000 7f8011ffb700 1 --2- 172.21.15.40:0/3411158202 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8000066800 0x7f8000068cc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8014064010 tx=0x7f8014069e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:52.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.876+0000 7f7ff97fa700 1 --2- 172.21.15.40:0/3411158202 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7ff4001e30 0x7f7ff400cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:52.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.876+0000 7f7ff97fa700 1 -- 172.21.15.40:0/3411158202 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff40023e0 con 0x7f7ff4001e30 2024-09-13T18:17:52.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.876+0000 7f80127fc700 1 --2- 172.21.15.40:0/3411158202 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7ff4001e30 0x7f7ff400cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:52.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.884+0000 7f80127fc700 1 -- 172.21.15.40:0/3411158202 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7ff400a310 con 0x7f7ff4001e30 2024-09-13T18:17:52.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.884+0000 7f80127fc700 1 --2- 172.21.15.40:0/3411158202 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7ff4001e30 0x7f7ff400cbf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40080d0 tx=0x7f800400ace0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:52.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.884+0000 7f7ffb7fe700 1 -- 172.21.15.40:0/3411158202 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800402a8d0 con 0x7f7ff4001e30 2024-09-13T18:17:52.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.888+0000 7f7ffb7fe700 1 -- 172.21.15.40:0/3411158202 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4451 (secure 0 0 0) 0x7f800402e070 con 0x7f7ff4001e30 2024-09-13T18:17:52.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.888+0000 7f7ffb7fe700 1 -- 172.21.15.40:0/3411158202 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7ff4001e30 msgr2=0x7f7ff400cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:52.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.888+0000 7f7ffb7fe700 1 --2- 172.21.15.40:0/3411158202 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7ff4001e30 0x7f7ff400cbf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40080d0 tx=0x7f800400ace0 comp rx=0 tx=0).stop 2024-09-13T18:17:52.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.888+0000 7f7ffb7fe700 1 -- 172.21.15.40:0/3411158202 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7ff4001e30 msgr2=0x7f7ff400cbf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:17:52.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.888+0000 7f7ffb7fe700 1 --2- 172.21.15.40:0/3411158202 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7ff4001e30 0x7f7ff400cbf0 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40080d0 tx=0x7f800400ace0 comp rx=0 tx=0).stop 2024-09-13T18:17:52.895 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:17:52.895 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:17:52.895 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:17:52.895 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:17:52.895 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 218, 2024-09-13T18:17:52.895 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:17:52.895 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:17:52.895 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:17:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:17:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 15, 2024-09-13T18:17:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:17:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:17:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:17:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:17:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:17:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:17:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:17:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:17:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:17:52.896 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:17:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:17:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:17:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:17:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:17:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:17:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:17:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:17:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:17:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:17:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:17:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:17:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:17:52.897 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:17:52.898 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:17:52.898 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:17:52.898 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:52.898 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.898 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:17:52.898 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:17:52.898 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:17:52.898 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:17:52.898 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:17:52.898 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:17:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:17:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:17:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:17:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:17:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:17:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:17:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:17:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:17:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:17:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:17:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:17:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:17:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:17:52.899 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:17:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:17:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:17:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:17:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:17:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:17:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:17:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:17:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:17:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:17:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:17:52.900 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:17:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:17:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:17:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:17:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:17:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:17:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:17:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:52.901 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:17:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:17:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:17:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:17:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:17:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:17:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:17:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:17:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:17:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:17:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:17:52.902 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:17:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:17:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:17:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:17:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:17:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:17:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:17:52.903 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:17:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:17:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:17:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:17:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:17:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:17:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:17:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:17:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:17:52.904 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:17:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:17:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:17:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:17:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:17:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:17:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:17:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.905 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:17:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:17:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:17:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:17:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:17:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:17:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:17:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:17:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:17:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:17:52.906 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:17:52.906 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:17:52.922 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.916+0000 7f7ff97fa700 1 -- 172.21.15.40:0/3411158202 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8000066800 msgr2=0x7f8000068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:52.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.916+0000 7f7ff97fa700 1 --2- 172.21.15.40:0/3411158202 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8000066800 0x7f8000068cc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8014064010 tx=0x7f8014069e70 comp rx=0 tx=0).stop 2024-09-13T18:17:52.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.916+0000 7f8012ffd700 1 -- 172.21.15.40:0/3411158202 reap_dead start 2024-09-13T18:17:52.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.916+0000 7f7ff97fa700 1 -- 172.21.15.40:0/3411158202 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f800c0dc800 msgr2=0x7f800c049610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:52.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.916+0000 7f7ff97fa700 1 --2- 172.21.15.40:0/3411158202 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f800c0dc800 0x7f800c049610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f800800b440 tx=0x7f800800b800 comp rx=0 tx=0).stop 2024-09-13T18:17:52.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.916+0000 7f7ff97fa700 1 -- 172.21.15.40:0/3411158202 shutdown_connections 2024-09-13T18:17:52.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.916+0000 7f7ff97fa700 1 --2- 172.21.15.40:0/3411158202 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f800c0dc800 0x7f800c049610 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:52.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.916+0000 7f7ff97fa700 1 -- 172.21.15.40:0/3411158202 >> 172.21.15.40:0/3411158202 conn(0x7f800c04feb0 msgr2=0x7f800c0cf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:52.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.916+0000 7f7ff97fa700 1 -- 172.21.15.40:0/3411158202 shutdown_connections 2024-09-13T18:17:52.923 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:52.916+0000 7f7ff97fa700 1 -- 172.21.15.40:0/3411158202 wait complete. 2024-09-13T18:17:52.951 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:17:52.951 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-13T18:17:52.951 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T18:17:52.951 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-13T18:17:52.951 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T18:17:52.952 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T18:17:52.952 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:17:53.020 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T18:17:53.020 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T18:17:53.021 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:17:53.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.240+0000 7f35f72ed700 1 Processor -- start 2024-09-13T18:17:53.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.240+0000 7f35f72ed700 1 -- start start 2024-09-13T18:17:53.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.240+0000 7f35f72ed700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f35f006aaa0 0x7f35f0169100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:53.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.240+0000 7f35f72ed700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f0071690 con 0x7f35f015e1a0 2024-09-13T18:17:53.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.240+0000 7f35f72ed700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f0154620 con 0x7f35f0154240 2024-09-13T18:17:53.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.240+0000 7f35f72ed700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f00721e0 con 0x7f35f006aaa0 2024-09-13T18:17:53.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.240+0000 7f35f6aec700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f35f006aaa0 msgr2=0x7f35f0169100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:17:53.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.240+0000 7f35f6aec700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f35f006aaa0 0x7f35f0169100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:17:53.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.240+0000 7f35f5aea700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f35f015e1a0 legacy=0x7f35f006a390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:17:53.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.240+0000 7f35f62eb700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f35f0154240 0x7f35f015ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:39464/0 (socket says 172.21.15.40:39464) 2024-09-13T18:17:53.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.240+0000 7f35f62eb700 1 -- 172.21.15.40:0/3886034701 learned_addr learned my addr 172.21.15.40:0/3886034701 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:17:53.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.244+0000 7f35f52e9700 1 -- 172.21.15.40:0/3886034701 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3669731073 0 0) 0x7f35f0154620 con 0x7f35f0154240 2024-09-13T18:17:53.251 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.244+0000 7f35f52e9700 1 -- 172.21.15.40:0/3886034701 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35dc0035d0 con 0x7f35f0154240 2024-09-13T18:17:53.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.244+0000 7f35f52e9700 1 -- 172.21.15.40:0/3886034701 <== mon.1 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2326893397 0 0) 0x7f35dc0035d0 con 0x7f35f0154240 2024-09-13T18:17:53.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.244+0000 7f35f52e9700 1 -- 172.21.15.40:0/3886034701 >> v2:172.21.15.40:3300/0 conn(0x7f35f006aaa0 msgr2=0x7f35f0169100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:17:53.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.244+0000 7f35f52e9700 1 --2- 172.21.15.40:0/3886034701 >> v2:172.21.15.40:3300/0 conn(0x7f35f006aaa0 0x7f35f0169100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:53.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.244+0000 7f35f52e9700 1 -- 172.21.15.40:0/3886034701 >> v1:172.21.15.40:6789/0 conn(0x7f35f015e1a0 legacy=0x7f35f006a390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:17:53.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.244+0000 7f35f52e9700 1 -- 172.21.15.40:0/3886034701 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f0071b20 con 0x7f35f0154240 2024-09-13T18:17:53.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.244+0000 7f35f52e9700 1 -- 172.21.15.40:0/3886034701 <== mon.1 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f35e0001670 con 0x7f35f0154240 2024-09-13T18:17:53.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.244+0000 7f35f52e9700 1 -- 172.21.15.40:0/3886034701 >> v1:172.21.15.43:6789/0 conn(0x7f35f0154240 legacy=0x7f35f015ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:53.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.244+0000 7f35f52e9700 1 --2- 172.21.15.40:0/3886034701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35dc003a20 0x7f35dc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:53.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.244+0000 7f35f52e9700 1 --2- 172.21.15.40:0/3886034701 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35dc004870 0x7f35dc004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:53.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.244+0000 7f35f52e9700 1 --2- 172.21.15.40:0/3886034701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35dc005230 0x7f35dc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:53.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.244+0000 7f35f6aec700 1 -- 172.21.15.40:0/3886034701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35dc005230 msgr2=0x7f35dc00ba30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:17:53.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.244+0000 7f35f6aec700 1 --2- 172.21.15.40:0/3886034701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35dc005230 0x7f35dc00ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:17:53.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.244+0000 7f35f52e9700 1 -- 172.21.15.40:0/3886034701 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f00721e0 con 0x7f35dc005230 2024-09-13T18:17:53.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.244+0000 7f35f52e9700 1 -- 172.21.15.40:0/3886034701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc00c160 con 0x7f35dc003a20 2024-09-13T18:17:53.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.244+0000 7f35f52e9700 1 -- 172.21.15.40:0/3886034701 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc00c320 con 0x7f35dc004870 2024-09-13T18:17:53.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.244+0000 7f35f5aea700 1 --2- 172.21.15.40:0/3886034701 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35dc004870 0x7f35dc004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:53.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.248+0000 7f35f62eb700 1 --2- 172.21.15.40:0/3886034701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35dc003a20 0x7f35dc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:53.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.248+0000 7f35f5aea700 1 -- 172.21.15.40:0/3886034701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35dc003a20 msgr2=0x7f35dc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:17:53.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.248+0000 7f35f5aea700 1 --2- 172.21.15.40:0/3886034701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35dc003a20 0x7f35dc003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:53.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.248+0000 7f35f5aea700 1 -- 172.21.15.40:0/3886034701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35dc005230 msgr2=0x7f35dc00ba30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:17:53.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.248+0000 7f35f5aea700 1 --2- 172.21.15.40:0/3886034701 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35dc005230 0x7f35dc00ba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:53.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.248+0000 7f35f5aea700 1 -- 172.21.15.40:0/3886034701 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc0035d0 con 0x7f35dc004870 2024-09-13T18:17:53.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.248+0000 7f35f6aec700 1 -- 172.21.15.40:0/3886034701 reap_dead start 2024-09-13T18:17:53.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.248+0000 7f35f5aea700 1 --2- 172.21.15.40:0/3886034701 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35dc004870 0x7f35dc004cf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0098e0 tx=0x7f35dc0058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aec24312ec7cdcf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:53.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.248+0000 7f35f52e9700 1 -- 172.21.15.40:0/3886034701 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec003b90 con 0x7f35dc004870 2024-09-13T18:17:53.257 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.248+0000 7f35f52e9700 1 -- 172.21.15.40:0/3886034701 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35ec003cf0 con 0x7f35dc004870 2024-09-13T18:17:53.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.252+0000 7f35f72ed700 1 -- 172.21.15.40:0/3886034701 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35dc004870 msgr2=0x7f35dc004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:53.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.252+0000 7f35f72ed700 1 --2- 172.21.15.40:0/3886034701 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35dc004870 0x7f35dc004cf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f35ec0098e0 tx=0x7f35dc0058a0 comp rx=0 tx=0).stop 2024-09-13T18:17:53.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.252+0000 7f35f72ed700 1 -- 172.21.15.40:0/3886034701 shutdown_connections 2024-09-13T18:17:53.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.252+0000 7f35f72ed700 1 --2- 172.21.15.40:0/3886034701 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35dc004870 0x7f35dc004cf0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:53.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.252+0000 7f35f72ed700 1 -- 172.21.15.40:0/3886034701 >> 172.21.15.40:0/3886034701 conn(0x7f35f00cabc0 msgr2=0x7f35f00cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:53.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.252+0000 7f35f72ed700 1 -- 172.21.15.40:0/3886034701 shutdown_connections 2024-09-13T18:17:53.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.252+0000 7f35f72ed700 1 -- 172.21.15.40:0/3886034701 wait complete. 2024-09-13T18:17:53.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.252+0000 7f35f72ed700 1 Processor -- start 2024-09-13T18:17:53.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.252+0000 7f35f72ed700 1 -- start start 2024-09-13T18:17:53.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.252+0000 7f35f72ed700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f0149a10 0x7f35f00b8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:53.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.252+0000 7f35f72ed700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f014da00 0x7f35f014de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:53.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.252+0000 7f35f72ed700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f0150e60 0x7f35f01512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:53.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.252+0000 7f35f72ed700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f00b8d90 con 0x7f35f0149a10 2024-09-13T18:17:53.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.252+0000 7f35f72ed700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f0156200 con 0x7f35f014da00 2024-09-13T18:17:53.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.252+0000 7f35f72ed700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f00b9130 con 0x7f35f0150e60 2024-09-13T18:17:53.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.252+0000 7f35f62eb700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f0149a10 msgr2=0x7f35f00b8850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:17:53.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.252+0000 7f35f62eb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f0149a10 0x7f35f00b8850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:17:53.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.256+0000 7f35f6aec700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f0150e60 0x7f35f01512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:53.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.256+0000 7f35f5aea700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f014da00 0x7f35f014de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:53.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.256+0000 7f35f6aec700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f0150e60 0x7f35f01512e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:36604/0 (socket says 172.21.15.40:36604) 2024-09-13T18:17:53.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.256+0000 7f35f6aec700 1 -- 172.21.15.40:0/2192467752 learned_addr learned my addr 172.21.15.40:0/2192467752 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:17:53.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.256+0000 7f35f5aea700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f014da00 0x7f35f014de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:52368/0 (socket says 172.21.15.40:52368) 2024-09-13T18:17:53.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.256+0000 7f35f5aea700 1 -- 172.21.15.40:0/2192467752 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f0150e60 msgr2=0x7f35f01512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:53.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.256+0000 7f35f5aea700 1 --2- 172.21.15.40:0/2192467752 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f0150e60 0x7f35f01512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:53.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.256+0000 7f35f5aea700 1 -- 172.21.15.40:0/2192467752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f0149a10 msgr2=0x7f35f00b8850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:17:53.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.256+0000 7f35f5aea700 1 --2- 172.21.15.40:0/2192467752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f0149a10 0x7f35f00b8850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:53.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.256+0000 7f35f5aea700 1 -- 172.21.15.40:0/2192467752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f01579f0 con 0x7f35f014da00 2024-09-13T18:17:53.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.256+0000 7f35f5aea700 1 --2- 172.21.15.40:0/2192467752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f014da00 0x7f35f014de60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f35ec011000 tx=0x7f35ec0059f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:53.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.256+0000 7f35e6ffd700 1 -- 172.21.15.40:0/2192467752 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec009380 con 0x7f35f014da00 2024-09-13T18:17:53.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.256+0000 7f35e6ffd700 1 -- 172.21.15.40:0/2192467752 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35ec01d840 con 0x7f35f014da00 2024-09-13T18:17:53.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.256+0000 7f35e6ffd700 1 -- 172.21.15.40:0/2192467752 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec01dbf0 con 0x7f35f014da00 2024-09-13T18:17:53.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.256+0000 7f35f72ed700 1 -- 172.21.15.40:0/2192467752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f0155160 con 0x7f35f014da00 2024-09-13T18:17:53.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.256+0000 7f35f72ed700 1 -- 172.21.15.40:0/2192467752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f0155720 con 0x7f35f014da00 2024-09-13T18:17:53.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.260+0000 7f35f72ed700 1 -- 172.21.15.40:0/2192467752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f0071320 con 0x7f35f014da00 2024-09-13T18:17:53.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.260+0000 7f35e6ffd700 1 -- 172.21.15.40:0/2192467752 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f35ec01dd50 con 0x7f35f014da00 2024-09-13T18:17:53.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.260+0000 7f35e6ffd700 1 --2- 172.21.15.40:0/2192467752 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f35dc067e60 0x7f35dc06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:53.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.260+0000 7f35f62eb700 1 --2- 172.21.15.40:0/2192467752 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f35dc067e60 0x7f35dc06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:53.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.264+0000 7f35f62eb700 1 --2- 172.21.15.40:0/2192467752 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f35dc067e60 0x7f35dc06a320 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f35f004f940 tx=0x7f35e0001010 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:53.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.268+0000 7f35e6ffd700 1 -- 172.21.15.40:0/2192467752 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f35ec085150 con 0x7f35f014da00 2024-09-13T18:17:53.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:53.272+0000 7f35e6ffd700 1 -- 172.21.15.40:0/2192467752 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f35ec0520f0 con 0x7f35f014da00 2024-09-13T18:17:54.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:54.340+0000 7f35f72ed700 1 -- 172.21.15.40:0/2192467752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f35f006b370 con 0x7f35f014da00 2024-09-13T18:17:54.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:54.340+0000 7f35e6ffd700 1 -- 172.21.15.40:0/2192467752 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f35ec055710 con 0x7f35f014da00 2024-09-13T18:17:54.347 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":218,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:17:54.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:54.352+0000 7f35f72ed700 1 -- 172.21.15.40:0/2192467752 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f35dc067e60 msgr2=0x7f35dc06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:54.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:54.352+0000 7f35f72ed700 1 --2- 172.21.15.40:0/2192467752 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f35dc067e60 0x7f35dc06a320 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f35f004f940 tx=0x7f35e0001010 comp rx=0 tx=0).stop 2024-09-13T18:17:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:54.352+0000 7f35f72ed700 1 -- 172.21.15.40:0/2192467752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f014da00 msgr2=0x7f35f014de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:54.352+0000 7f35f72ed700 1 --2- 172.21.15.40:0/2192467752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f014da00 0x7f35f014de60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f35ec011000 tx=0x7f35ec0059f0 comp rx=0 tx=0).stop 2024-09-13T18:17:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:54.352+0000 7f35f72ed700 1 -- 172.21.15.40:0/2192467752 shutdown_connections 2024-09-13T18:17:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:54.352+0000 7f35f72ed700 1 --2- 172.21.15.40:0/2192467752 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f35dc067e60 0x7f35dc06a320 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:54.352+0000 7f35f72ed700 1 --2- 172.21.15.40:0/2192467752 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f35f0150e60 0x7f35f01512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:54.352+0000 7f35f72ed700 1 --2- 172.21.15.40:0/2192467752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f014da00 0x7f35f014de60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:54.352+0000 7f35f72ed700 1 --2- 172.21.15.40:0/2192467752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f0149a10 0x7f35f00b8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:54.352+0000 7f35f72ed700 1 -- 172.21.15.40:0/2192467752 >> 172.21.15.40:0/2192467752 conn(0x7f35f00cabc0 msgr2=0x7f35f00cafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:54.352+0000 7f35f72ed700 1 -- 172.21.15.40:0/2192467752 shutdown_connections 2024-09-13T18:17:54.361 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:54.352+0000 7f35f72ed700 1 -- 172.21.15.40:0/2192467752 wait complete. 2024-09-13T18:17:57.383 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:17:57.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.651+0000 7ff1dff75700 1 Processor -- start 2024-09-13T18:17:57.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.651+0000 7ff1dff75700 1 -- start start 2024-09-13T18:17:57.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.651+0000 7ff1dff75700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff1d8154240 0x7ff1d815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:57.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.651+0000 7ff1dff75700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d815ec60 0x7ff1d815f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:57.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.651+0000 7ff1dff75700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d8071690 con 0x7ff1d806a390 2024-09-13T18:17:57.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.651+0000 7ff1dff75700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d80721e0 con 0x7ff1d8154240 2024-09-13T18:17:57.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.651+0000 7ff1dff75700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d815e6e0 con 0x7ff1d815ec60 2024-09-13T18:17:57.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.651+0000 7ff1ddd11700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7ff1d8154240 msgr2=0x7ff1d815e1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:17:57.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.651+0000 7ff1ddd11700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff1d8154240 0x7ff1d815e1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:17:57.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.651+0000 7ff1dd510700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d815ec60 0x7ff1d815f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:57.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.651+0000 7ff1dd510700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d815ec60 0x7ff1d815f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:36620/0 (socket says 172.21.15.40:36620) 2024-09-13T18:17:57.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.651+0000 7ff1dd510700 1 -- 172.21.15.40:0/2607067542 learned_addr learned my addr 172.21.15.40:0/2607067542 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:17:57.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.651+0000 7ff1dd510700 1 -- 172.21.15.40:0/2607067542 >> v2:172.21.15.40:3300/0 conn(0x7ff1d8154240 msgr2=0x7ff1d815e1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:17:57.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.651+0000 7ff1dd510700 1 --2- 172.21.15.40:0/2607067542 >> v2:172.21.15.40:3300/0 conn(0x7ff1d8154240 0x7ff1d815e1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:57.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.651+0000 7ff1dd510700 1 -- 172.21.15.40:0/2607067542 >> v1:172.21.15.43:6789/0 conn(0x7ff1d806a390 legacy=0x7ff1d806a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:57.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.651+0000 7ff1dd510700 1 -- 172.21.15.40:0/2607067542 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d8154620 con 0x7ff1d815ec60 2024-09-13T18:17:57.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.651+0000 7ff1dd510700 1 --2- 172.21.15.40:0/2607067542 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d815ec60 0x7ff1d815f0a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009e50 tx=0x7ff1d000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f494c096d190bdaa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:57.665 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.659+0000 7ff1dcd0f700 1 -- 172.21.15.40:0/2607067542 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d000e070 con 0x7ff1d815ec60 2024-09-13T18:17:57.666 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.659+0000 7ff1dcd0f700 1 -- 172.21.15.40:0/2607067542 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d000b9c0 con 0x7ff1d815ec60 2024-09-13T18:17:57.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.659+0000 7ff1dff75700 1 -- 172.21.15.40:0/2607067542 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d815ec60 msgr2=0x7ff1d815f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:57.667 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.659+0000 7ff1dff75700 1 --2- 172.21.15.40:0/2607067542 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d815ec60 0x7ff1d815f0a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009e50 tx=0x7ff1d000b330 comp rx=0 tx=0).stop 2024-09-13T18:17:57.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.659+0000 7ff1dff75700 1 -- 172.21.15.40:0/2607067542 shutdown_connections 2024-09-13T18:17:57.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.659+0000 7ff1dff75700 1 --2- 172.21.15.40:0/2607067542 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d815ec60 0x7ff1d815f0a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:57.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.659+0000 7ff1dff75700 1 --2- 172.21.15.40:0/2607067542 >> v2:172.21.15.40:3300/0 conn(0x7ff1d8154240 0x7ff1d815e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:57.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.659+0000 7ff1dff75700 1 -- 172.21.15.40:0/2607067542 >> 172.21.15.40:0/2607067542 conn(0x7ff1d80cabc0 msgr2=0x7ff1d80cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:57.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.659+0000 7ff1dff75700 1 -- 172.21.15.40:0/2607067542 shutdown_connections 2024-09-13T18:17:57.668 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.659+0000 7ff1dff75700 1 -- 172.21.15.40:0/2607067542 wait complete. 2024-09-13T18:17:57.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.659+0000 7ff1dff75700 1 Processor -- start 2024-09-13T18:17:57.669 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.663+0000 7ff1dff75700 1 -- start start 2024-09-13T18:17:57.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.663+0000 7ff1dff75700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d806a390 0x7ff1d813e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:57.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.663+0000 7ff1dff75700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1d8154240 0x7ff1d8140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:57.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.663+0000 7ff1dff75700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d813c1f0 0x7ff1d813b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:57.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.663+0000 7ff1dff75700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d815cb30 con 0x7ff1d8154240 2024-09-13T18:17:57.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.663+0000 7ff1dff75700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d815c370 con 0x7ff1d806a390 2024-09-13T18:17:57.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.663+0000 7ff1dff75700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d815c040 con 0x7ff1d813c1f0 2024-09-13T18:17:57.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.663+0000 7ff1dd510700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1d8154240 msgr2=0x7ff1d8140ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:17:57.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.663+0000 7ff1dd510700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1d8154240 0x7ff1d8140ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:17:57.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.663+0000 7ff1ddd11700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d806a390 0x7ff1d813e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:57.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.663+0000 7ff1ddd11700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d806a390 0x7ff1d813e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:52382/0 (socket says 172.21.15.40:52382) 2024-09-13T18:17:57.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.663+0000 7ff1ddd11700 1 -- 172.21.15.40:0/3796506429 learned_addr learned my addr 172.21.15.40:0/3796506429 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:17:57.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.667+0000 7ff1de512700 1 --2- 172.21.15.40:0/3796506429 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d813c1f0 0x7ff1d813b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:57.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.667+0000 7ff1ddd11700 1 -- 172.21.15.40:0/3796506429 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d813c1f0 msgr2=0x7ff1d813b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:57.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.667+0000 7ff1ddd11700 1 --2- 172.21.15.40:0/3796506429 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d813c1f0 0x7ff1d813b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:57.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.667+0000 7ff1ddd11700 1 -- 172.21.15.40:0/3796506429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1d8154240 msgr2=0x7ff1d8140ad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:17:57.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.667+0000 7ff1ddd11700 1 --2- 172.21.15.40:0/3796506429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1d8154240 0x7ff1d8140ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:57.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.667+0000 7ff1ddd11700 1 -- 172.21.15.40:0/3796506429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d0009b30 con 0x7ff1d806a390 2024-09-13T18:17:57.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.667+0000 7ff1ddd11700 1 --2- 172.21.15.40:0/3796506429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d806a390 0x7ff1d813e5a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400b570 tx=0x7ff1d400b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:57.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.667+0000 7ff1ce7fc700 1 -- 172.21.15.40:0/3796506429 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4003e70 con 0x7ff1d806a390 2024-09-13T18:17:57.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.667+0000 7ff1dff75700 1 -- 172.21.15.40:0/3796506429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d815c870 con 0x7ff1d806a390 2024-09-13T18:17:57.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.667+0000 7ff1dff75700 1 -- 172.21.15.40:0/3796506429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d8142590 con 0x7ff1d806a390 2024-09-13T18:17:57.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.671+0000 7ff1ce7fc700 1 -- 172.21.15.40:0/3796506429 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d4005070 con 0x7ff1d806a390 2024-09-13T18:17:57.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.671+0000 7ff1ce7fc700 1 -- 172.21.15.40:0/3796506429 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4005440 con 0x7ff1d806a390 2024-09-13T18:17:57.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.679+0000 7ff1dff75700 1 -- 172.21.15.40:0/3796506429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c0005310 con 0x7ff1d806a390 2024-09-13T18:17:57.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.675+0000 7ff1ce7fc700 1 -- 172.21.15.40:0/3796506429 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff1d40055a0 con 0x7ff1d806a390 2024-09-13T18:17:57.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.683+0000 7ff1ce7fc700 1 --2- 172.21.15.40:0/3796506429 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff1bc066760 0x7ff1bc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:17:57.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.683+0000 7ff1ce7fc700 1 -- 172.21.15.40:0/3796506429 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff1d4083940 con 0x7ff1d806a390 2024-09-13T18:17:57.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.687+0000 7ff1dd510700 1 --2- 172.21.15.40:0/3796506429 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff1bc066760 0x7ff1bc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:17:57.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.691+0000 7ff1ce7fc700 1 -- 172.21.15.40:0/3796506429 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff1d4050960 con 0x7ff1d806a390 2024-09-13T18:17:57.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:57.691+0000 7ff1dd510700 1 --2- 172.21.15.40:0/3796506429 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff1bc066760 0x7ff1bc068c20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff1d815bf30 tx=0x7ff1d0009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:17:58.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:58.503+0000 7ff1dff75700 1 -- 172.21.15.40:0/3796506429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7ff1c0005b40 con 0x7ff1d806a390 2024-09-13T18:17:58.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:58.503+0000 7ff1ce7fc700 1 -- 172.21.15.40:0/3796506429 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7ff1d4053f80 con 0x7ff1d806a390 2024-09-13T18:17:58.520 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":218,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:17:58.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:58.559+0000 7ff1dff75700 1 -- 172.21.15.40:0/3796506429 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff1bc066760 msgr2=0x7ff1bc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:58.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:58.559+0000 7ff1dff75700 1 --2- 172.21.15.40:0/3796506429 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff1bc066760 0x7ff1bc068c20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff1d815bf30 tx=0x7ff1d0009000 comp rx=0 tx=0).stop 2024-09-13T18:17:58.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:58.559+0000 7ff1dff75700 1 -- 172.21.15.40:0/3796506429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d806a390 msgr2=0x7ff1d813e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:17:58.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:58.559+0000 7ff1dff75700 1 --2- 172.21.15.40:0/3796506429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d806a390 0x7ff1d813e5a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400b570 tx=0x7ff1d400b930 comp rx=0 tx=0).stop 2024-09-13T18:17:58.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:58.559+0000 7ff1dff75700 1 -- 172.21.15.40:0/3796506429 shutdown_connections 2024-09-13T18:17:58.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:58.559+0000 7ff1dff75700 1 --2- 172.21.15.40:0/3796506429 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff1bc066760 0x7ff1bc068c20 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:58.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:58.559+0000 7ff1dff75700 1 --2- 172.21.15.40:0/3796506429 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff1d813c1f0 0x7ff1d813b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:58.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:58.559+0000 7ff1dff75700 1 --2- 172.21.15.40:0/3796506429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1d8154240 0x7ff1d8140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:58.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:58.559+0000 7ff1dff75700 1 --2- 172.21.15.40:0/3796506429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d806a390 0x7ff1d813e5a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:17:58.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:58.559+0000 7ff1dff75700 1 -- 172.21.15.40:0/3796506429 >> 172.21.15.40:0/3796506429 conn(0x7ff1d80cabc0 msgr2=0x7ff1d80c4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:17:58.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:58.559+0000 7ff1dff75700 1 -- 172.21.15.40:0/3796506429 shutdown_connections 2024-09-13T18:17:58.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:17:58.559+0000 7ff1dff75700 1 -- 172.21.15.40:0/3796506429 wait complete. 2024-09-13T18:18:01.599 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:18:01.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.859+0000 7fefbf616700 1 Processor -- start 2024-09-13T18:18:01.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.859+0000 7fefbf616700 1 -- start start 2024-09-13T18:18:01.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.859+0000 7fefbf616700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefb8160390 0x7fefb81669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:01.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.859+0000 7fefbf616700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fefb80704e0 0x7fefb8167600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:01.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.859+0000 7fefbf616700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb8071930 con 0x7fefb8167b40 2024-09-13T18:18:01.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.859+0000 7fefbf616700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb8166f20 con 0x7fefb80704e0 2024-09-13T18:18:01.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.859+0000 7fefbf616700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb8167fd0 con 0x7fefb8160390 2024-09-13T18:18:01.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.859+0000 7fefbcbb1700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7fefb80704e0 msgr2=0x7fefb8167600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:18:01.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.859+0000 7fefbcbb1700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fefb80704e0 0x7fefb8167600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:18:01.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.863+0000 7fefbdbb3700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fefb8167b40 0x7fefb806a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:34770/0 (socket says 172.21.15.40:34770) 2024-09-13T18:18:01.872 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.863+0000 7fefbdbb3700 1 -- 172.21.15.40:0/2833199184 learned_addr learned my addr 172.21.15.40:0/2833199184 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:18:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.863+0000 7fefbd3b2700 1 --2- 172.21.15.40:0/2833199184 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefb8160390 0x7fefb81669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.863+0000 7fefbd3b2700 1 -- 172.21.15.40:0/2833199184 >> v2:172.21.15.40:3300/0 conn(0x7fefb80704e0 msgr2=0x7fefb8167600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:18:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.863+0000 7fefbd3b2700 1 --2- 172.21.15.40:0/2833199184 >> v2:172.21.15.40:3300/0 conn(0x7fefb80704e0 0x7fefb8167600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.863+0000 7fefbd3b2700 1 -- 172.21.15.40:0/2833199184 >> v1:172.21.15.43:6789/0 conn(0x7fefb8167b40 legacy=0x7fefb806a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.863+0000 7fefbd3b2700 1 -- 172.21.15.40:0/2833199184 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb8070a40 con 0x7fefb8160390 2024-09-13T18:18:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.863+0000 7fefbd3b2700 1 --2- 172.21.15.40:0/2833199184 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefb8160390 0x7fefb81669e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fefb40088b0 tx=0x7fefb4008bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9ddc148b40505f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.863+0000 7fefaffff700 1 -- 172.21.15.40:0/2833199184 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb40122c0 con 0x7fefb8160390 2024-09-13T18:18:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.863+0000 7fefaffff700 1 -- 172.21.15.40:0/2833199184 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb401ea20 con 0x7fefb8160390 2024-09-13T18:18:01.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.863+0000 7fefaffff700 1 -- 172.21.15.40:0/2833199184 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb401edb0 con 0x7fefb8160390 2024-09-13T18:18:01.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.863+0000 7fefbf616700 1 -- 172.21.15.40:0/2833199184 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefb8160390 msgr2=0x7fefb81669e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:01.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.863+0000 7fefbf616700 1 --2- 172.21.15.40:0/2833199184 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefb8160390 0x7fefb81669e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fefb40088b0 tx=0x7fefb4008bc0 comp rx=0 tx=0).stop 2024-09-13T18:18:01.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.863+0000 7fefbf616700 1 -- 172.21.15.40:0/2833199184 shutdown_connections 2024-09-13T18:18:01.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.863+0000 7fefbf616700 1 --2- 172.21.15.40:0/2833199184 >> v2:172.21.15.40:3300/0 conn(0x7fefb80704e0 0x7fefb8167600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:01.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.863+0000 7fefbf616700 1 --2- 172.21.15.40:0/2833199184 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefb8160390 0x7fefb81669e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:01.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.863+0000 7fefbf616700 1 -- 172.21.15.40:0/2833199184 >> 172.21.15.40:0/2833199184 conn(0x7fefb80cabc0 msgr2=0x7fefb80cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.863+0000 7fefbf616700 1 -- 172.21.15.40:0/2833199184 shutdown_connections 2024-09-13T18:18:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.863+0000 7fefbf616700 1 -- 172.21.15.40:0/2833199184 wait complete. 2024-09-13T18:18:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.867+0000 7fefbf616700 1 Processor -- start 2024-09-13T18:18:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.867+0000 7fefbf616700 1 -- start start 2024-09-13T18:18:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.867+0000 7fefbf616700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefb80704e0 0x7fefb8160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.867+0000 7fefbf616700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefb8160390 0x7fefb80c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.867+0000 7fefbf616700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefb8167b40 0x7fefb81ff870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.867+0000 7fefbf616700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb80c6170 con 0x7fefb80704e0 2024-09-13T18:18:01.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.867+0000 7fefbf616700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb8160dd0 con 0x7fefb8160390 2024-09-13T18:18:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.867+0000 7fefbf616700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb81eed40 con 0x7fefb8167b40 2024-09-13T18:18:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.867+0000 7fefbd3b2700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefb80704e0 msgr2=0x7fefb8160890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:18:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.867+0000 7fefbd3b2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefb80704e0 0x7fefb8160890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:18:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.867+0000 7fefbcbb1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefb8160390 0x7fefb80c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.867+0000 7fefbcbb1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefb8160390 0x7fefb80c7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:37066/0 (socket says 172.21.15.40:37066) 2024-09-13T18:18:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.867+0000 7fefbcbb1700 1 -- 172.21.15.40:0/4251562304 learned_addr learned my addr 172.21.15.40:0/4251562304 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.867+0000 7fefbdbb3700 1 --2- 172.21.15.40:0/4251562304 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefb8167b40 0x7fefb81ff870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.871+0000 7fefbcbb1700 1 -- 172.21.15.40:0/4251562304 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefb8167b40 msgr2=0x7fefb81ff870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.871+0000 7fefbcbb1700 1 --2- 172.21.15.40:0/4251562304 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefb8167b40 0x7fefb81ff870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.871+0000 7fefbcbb1700 1 -- 172.21.15.40:0/4251562304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefb80704e0 msgr2=0x7fefb8160890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:18:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.871+0000 7fefbcbb1700 1 --2- 172.21.15.40:0/4251562304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefb80704e0 0x7fefb8160890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.871+0000 7fefbcbb1700 1 -- 172.21.15.40:0/4251562304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb4008540 con 0x7fefb8160390 2024-09-13T18:18:01.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.871+0000 7fefbcbb1700 1 --2- 172.21.15.40:0/4251562304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefb8160390 0x7fefb80c7da0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fefb000b9e0 tx=0x7fefb000bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:01.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.871+0000 7fefadffb700 1 -- 172.21.15.40:0/4251562304 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb000ca30 con 0x7fefb8160390 2024-09-13T18:18:01.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.871+0000 7fefadffb700 1 -- 172.21.15.40:0/4251562304 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb000cbb0 con 0x7fefb8160390 2024-09-13T18:18:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.871+0000 7fefadffb700 1 -- 172.21.15.40:0/4251562304 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb0015470 con 0x7fefb8160390 2024-09-13T18:18:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.871+0000 7fefbf616700 1 -- 172.21.15.40:0/4251562304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb80c5560 con 0x7fefb8160390 2024-09-13T18:18:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.871+0000 7fefbf616700 1 -- 172.21.15.40:0/4251562304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb80c5b20 con 0x7fefb8160390 2024-09-13T18:18:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.871+0000 7fefbf616700 1 -- 172.21.15.40:0/4251562304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb80700b0 con 0x7fefb8160390 2024-09-13T18:18:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.871+0000 7fefadffb700 1 -- 172.21.15.40:0/4251562304 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fefb0012070 con 0x7fefb8160390 2024-09-13T18:18:01.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.875+0000 7fefadffb700 1 --2- 172.21.15.40:0/4251562304 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fef9c066730 0x7fef9c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:01.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.875+0000 7fefbd3b2700 1 --2- 172.21.15.40:0/4251562304 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fef9c066730 0x7fef9c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:01.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.875+0000 7fefbd3b2700 1 --2- 172.21.15.40:0/4251562304 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fef9c066730 0x7fef9c068bf0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fefb4000c00 tx=0x7fefb400d000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:01.883 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.875+0000 7fefadffb700 1 -- 172.21.15.40:0/4251562304 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fefb0084340 con 0x7fefb8160390 2024-09-13T18:18:01.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:01.879+0000 7fefadffb700 1 -- 172.21.15.40:0/4251562304 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fefb00512d0 con 0x7fefb8160390 2024-09-13T18:18:02.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:02.555+0000 7fefbf616700 1 -- 172.21.15.40:0/4251562304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fefb80c62e0 con 0x7fefb8160390 2024-09-13T18:18:02.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:02.555+0000 7fefadffb700 1 -- 172.21.15.40:0/4251562304 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7fefb00548f0 con 0x7fefb8160390 2024-09-13T18:18:02.565 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":218,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:18:02.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:02.571+0000 7fefbf616700 1 -- 172.21.15.40:0/4251562304 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fef9c066730 msgr2=0x7fef9c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:02.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:02.571+0000 7fefbf616700 1 --2- 172.21.15.40:0/4251562304 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fef9c066730 0x7fef9c068bf0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fefb4000c00 tx=0x7fefb400d000 comp rx=0 tx=0).stop 2024-09-13T18:18:02.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:02.571+0000 7fefbf616700 1 -- 172.21.15.40:0/4251562304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefb8160390 msgr2=0x7fefb80c7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:02.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:02.571+0000 7fefbf616700 1 --2- 172.21.15.40:0/4251562304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefb8160390 0x7fefb80c7da0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fefb000b9e0 tx=0x7fefb000bda0 comp rx=0 tx=0).stop 2024-09-13T18:18:02.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:02.571+0000 7fefbf616700 1 -- 172.21.15.40:0/4251562304 shutdown_connections 2024-09-13T18:18:02.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:02.571+0000 7fefbf616700 1 --2- 172.21.15.40:0/4251562304 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fef9c066730 0x7fef9c068bf0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:02.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:02.571+0000 7fefbf616700 1 --2- 172.21.15.40:0/4251562304 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefb8167b40 0x7fefb81ff870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:02.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:02.571+0000 7fefbf616700 1 --2- 172.21.15.40:0/4251562304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefb8160390 0x7fefb80c7da0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:02.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:02.571+0000 7fefbf616700 1 --2- 172.21.15.40:0/4251562304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefb80704e0 0x7fefb8160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:02.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:02.571+0000 7fefbf616700 1 -- 172.21.15.40:0/4251562304 >> 172.21.15.40:0/4251562304 conn(0x7fefb80cabc0 msgr2=0x7fefb81641e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:02.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:02.571+0000 7fefbf616700 1 -- 172.21.15.40:0/4251562304 shutdown_connections 2024-09-13T18:18:02.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:02.571+0000 7fefbf616700 1 -- 172.21.15.40:0/4251562304 wait complete. 2024-09-13T18:18:05.617 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:18:05.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:05.955+0000 7f36aa015700 1 Processor -- start 2024-09-13T18:18:05.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:05.955+0000 7f36aa015700 1 -- start start 2024-09-13T18:18:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:05.955+0000 7f36aa015700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 0x7f36a41669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:05.955+0000 7f36aa015700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 0x7f36a406a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:05.955+0000 7f36aa015700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a4071930 con 0x7f36a4160390 2024-09-13T18:18:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:05.955+0000 7f36aa015700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a4166f20 con 0x7f36a4167d10 2024-09-13T18:18:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:05.955+0000 7f36aa015700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a4070a40 con 0x7f36a40704e0 2024-09-13T18:18:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:05.955+0000 7f36a9013700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f36a4160390 legacy=0x7f36a4167600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:18:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:05.955+0000 7f36a8812700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 0x7f36a41669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:05.955+0000 7f36a8812700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 0x7f36a41669e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:37080/0 (socket says 172.21.15.40:37080) 2024-09-13T18:18:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:05.955+0000 7f36a8812700 1 -- 172.21.15.40:0/633630485 learned_addr learned my addr 172.21.15.40:0/633630485 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:05.955+0000 7f36a9814700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 0x7f36a406a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:05.959+0000 7f36a8812700 1 -- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 msgr2=0x7f36a41669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-09-13T18:18:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:05.959+0000 7f36a8812700 1 -- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 msgr2=0x7f36a41669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:18:05.968 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:05.959+0000 7f36a8812700 1 --2- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 0x7f36a41669e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:18:05.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:05.959+0000 7f36a8812700 1 --2- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 0x7f36a41669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:18:05.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:05.959+0000 7f36a9814700 1 -- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 msgr2=0x7f36a406a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:18:05.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:05.959+0000 7f36a9814700 1 -- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 msgr2=0x7f36a406a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:18:05.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:05.959+0000 7f36a9814700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 0x7f36a406a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:18:05.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:05.959+0000 7f36a9814700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 0x7f36a406a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:18:06.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.159+0000 7f36a9013700 1 -- 172.21.15.40:0/633630485 >> v1:172.21.15.40:6789/0 conn(0x7f36a4160390 legacy=0x7f36a4167600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:18:06.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.163+0000 7f36a9814700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 0x7f36a406a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:06.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.163+0000 7f36a9814700 1 -- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 msgr2=0x7f36a406a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-09-13T18:18:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.163+0000 7f36a9814700 1 -- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 msgr2=0x7f36a406a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:18:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.163+0000 7f36a9814700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 0x7f36a406a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:18:06.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.163+0000 7f36a9814700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 0x7f36a406a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:18:06.174 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.163+0000 7f36a8812700 1 --2- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 0x7f36a41669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.167+0000 7f36a8812700 1 -- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 msgr2=0x7f36a41669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-09-13T18:18:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.167+0000 7f36a8812700 1 -- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 msgr2=0x7f36a41669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:18:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.167+0000 7f36a8812700 1 --2- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 0x7f36a41669e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:18:06.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.167+0000 7f36a8812700 1 --2- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 0x7f36a41669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:18:06.570 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.563+0000 7f36a9013700 1 -- 172.21.15.40:0/633630485 >> v1:172.21.15.40:6789/0 conn(0x7f36a4160390 legacy=0x7f36a4167600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:18:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.567+0000 7f36a9814700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 0x7f36a406a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.567+0000 7f36a9814700 1 -- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 msgr2=0x7f36a406a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-09-13T18:18:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.567+0000 7f36a9814700 1 -- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 msgr2=0x7f36a406a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:18:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.567+0000 7f36a9814700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 0x7f36a406a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:18:06.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.567+0000 7f36a9814700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 0x7f36a406a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:18:06.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.571+0000 7f36a8812700 1 --2- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 0x7f36a41669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:06.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.571+0000 7f36a8812700 1 -- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 msgr2=0x7f36a41669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-09-13T18:18:06.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.571+0000 7f36a8812700 1 -- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 msgr2=0x7f36a41669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:18:06.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.571+0000 7f36a8812700 1 --2- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 0x7f36a41669e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:18:06.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:06.571+0000 7f36a8812700 1 --2- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 0x7f36a41669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:18:07.374 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:07.367+0000 7f36a9013700 1 -- 172.21.15.40:0/633630485 >> v1:172.21.15.40:6789/0 conn(0x7f36a4160390 legacy=0x7f36a4167600 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:18:07.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:07.371+0000 7f36a9814700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 0x7f36a406a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:07.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:07.371+0000 7f36a9814700 1 -- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 msgr2=0x7f36a406a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-09-13T18:18:07.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:07.371+0000 7f36a9814700 1 -- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 msgr2=0x7f36a406a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:18:07.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:07.371+0000 7f36a9814700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 0x7f36a406a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:18:07.378 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:07.371+0000 7f36a9814700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 0x7f36a406a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:18:07.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:07.375+0000 7f36a8812700 1 --2- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 0x7f36a41669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:07.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:07.375+0000 7f36a8812700 1 -- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 msgr2=0x7f36a41669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-09-13T18:18:07.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:07.375+0000 7f36a8812700 1 -- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 msgr2=0x7f36a41669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:18:07.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:07.375+0000 7f36a8812700 1 --2- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 0x7f36a41669e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:18:07.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:07.375+0000 7f36a8812700 1 --2- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 0x7f36a41669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:18:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.955+0000 7f369affd700 1 -- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 msgr2=0x7f36a406a390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:18:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.955+0000 7f369affd700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f36a40704e0 0x7f36a406a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.955+0000 7f369affd700 1 -- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 msgr2=0x7f36a41669e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:18:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.955+0000 7f369affd700 1 --2- 172.21.15.40:0/633630485 >> v2:172.21.15.43:3300/0 conn(0x7f36a4167d10 0x7f36a41669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.955+0000 7f369affd700 1 -- 172.21.15.40:0/633630485 >> v1:172.21.15.40:6789/0 conn(0x7f36a4160390 legacy=0x7f36a4167600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:18:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.955+0000 7f369affd700 1 --2- 172.21.15.40:0/633630485 >> v2:172.21.15.40:3300/0 conn(0x7f3688007460 0x7f3688009920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:08.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.955+0000 7f369affd700 1 -- 172.21.15.40:0/633630485 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a4070d20 con 0x7f3688003740 2024-09-13T18:18:08.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.955+0000 7f369affd700 1 -- 172.21.15.40:0/633630485 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a4166f20 con 0x7f3688007460 2024-09-13T18:18:08.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.955+0000 7f36a8812700 1 -- 172.21.15.40:0/633630485 >> v2:172.21.15.40:3300/0 conn(0x7f3688007460 msgr2=0x7f3688009920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:18:08.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.955+0000 7f36a8812700 1 --2- 172.21.15.40:0/633630485 >> v2:172.21.15.40:3300/0 conn(0x7f3688007460 0x7f3688009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:18:08.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.967+0000 7f369bfff700 1 -- 172.21.15.40:0/633630485 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924568251 0 0) 0x7f36a4070d20 con 0x7f3688003740 2024-09-13T18:18:08.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.967+0000 7f369bfff700 1 -- 172.21.15.40:0/633630485 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36900035d0 con 0x7f3688003740 2024-09-13T18:18:08.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.975+0000 7f369bfff700 1 -- 172.21.15.40:0/633630485 <== mon.1 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3752936311 0 0) 0x7f36900035d0 con 0x7f3688003740 2024-09-13T18:18:08.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.975+0000 7f369bfff700 1 -- 172.21.15.40:0/633630485 >> v2:172.21.15.40:3300/0 conn(0x7f3688007460 msgr2=0x7f3688009920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:18:08.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.975+0000 7f369bfff700 1 --2- 172.21.15.40:0/633630485 >> v2:172.21.15.40:3300/0 conn(0x7f3688007460 0x7f3688009920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:08.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.975+0000 7f369bfff700 1 -- 172.21.15.40:0/633630485 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a4071930 con 0x7f3688003740 2024-09-13T18:18:08.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.975+0000 7f369bfff700 1 -- 172.21.15.40:0/633630485 <== mon.1 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f36a00043f0 con 0x7f3688003740 2024-09-13T18:18:08.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.975+0000 7f369bfff700 1 -- 172.21.15.40:0/633630485 >> v1:172.21.15.43:6789/0 conn(0x7f3688003740 legacy=0x7f3688005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:08.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.975+0000 7f36a9814700 1 -- 172.21.15.40:0/633630485 reap_dead start 2024-09-13T18:18:08.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.975+0000 7f369bfff700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3690003a20 0x7f3690003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:08.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.975+0000 7f369bfff700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3690004870 0x7f3690004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:08.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.975+0000 7f369bfff700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3690005350 0x7f369000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:08.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.975+0000 7f369bfff700 1 -- 172.21.15.40:0/633630485 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a4166f20 con 0x7f3690003a20 2024-09-13T18:18:08.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.975+0000 7f369bfff700 1 -- 172.21.15.40:0/633630485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369000c250 con 0x7f3690004870 2024-09-13T18:18:08.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.975+0000 7f369bfff700 1 -- 172.21.15.40:0/633630485 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369000c410 con 0x7f3690005350 2024-09-13T18:18:08.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.975+0000 7f36a9013700 1 -- 172.21.15.40:0/633630485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3690003a20 msgr2=0x7f3690003e20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:18:08.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.975+0000 7f36a9013700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3690003a20 0x7f3690003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:18:08.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.979+0000 7f36a8812700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3690004870 0x7f3690004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:08.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.979+0000 7f36a9814700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3690005350 0x7f369000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:08.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.979+0000 7f36a8812700 1 -- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3690005350 msgr2=0x7f369000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:18:08.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.979+0000 7f36a8812700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3690005350 0x7f369000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:08.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.979+0000 7f36a8812700 1 -- 172.21.15.40:0/633630485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3690003a20 msgr2=0x7f3690003e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:18:08.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.979+0000 7f36a8812700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3690003a20 0x7f3690003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:08.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.979+0000 7f36a8812700 1 -- 172.21.15.40:0/633630485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a4071930 con 0x7f3690004870 2024-09-13T18:18:08.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.979+0000 7f36a8812700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3690004870 0x7f3690004cf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f369c007450 tx=0x7f36900058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afea15f722ea8008 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:08.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.979+0000 7f369bfff700 1 -- 172.21.15.40:0/633630485 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c011070 con 0x7f3690004870 2024-09-13T18:18:08.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.979+0000 7f369bfff700 1 -- 172.21.15.40:0/633630485 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369c012040 con 0x7f3690004870 2024-09-13T18:18:08.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.979+0000 7f369bfff700 1 -- 172.21.15.40:0/633630485 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c00ae90 con 0x7f3690004870 2024-09-13T18:18:08.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.979+0000 7f36aa015700 1 -- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3690004870 msgr2=0x7f3690004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:08.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.979+0000 7f36aa015700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3690004870 0x7f3690004cf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f369c007450 tx=0x7f36900058a0 comp rx=0 tx=0).stop 2024-09-13T18:18:08.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.979+0000 7f36aa015700 1 -- 172.21.15.40:0/633630485 shutdown_connections 2024-09-13T18:18:08.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.979+0000 7f36aa015700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3690005350 0x7f369000ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:08.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.979+0000 7f36aa015700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3690004870 0x7f3690004cf0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:08.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.979+0000 7f36aa015700 1 --2- 172.21.15.40:0/633630485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3690003a20 0x7f3690003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:08.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.979+0000 7f36aa015700 1 -- 172.21.15.40:0/633630485 >> 172.21.15.40:0/633630485 conn(0x7f36a40cabc0 msgr2=0x7f36a40cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:08.989 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.979+0000 7f36aa015700 1 -- 172.21.15.40:0/633630485 shutdown_connections 2024-09-13T18:18:08.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.979+0000 7f36aa015700 1 -- 172.21.15.40:0/633630485 wait complete. 2024-09-13T18:18:08.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.983+0000 7f36aa015700 1 Processor -- start 2024-09-13T18:18:08.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.983+0000 7f36aa015700 1 -- start start 2024-09-13T18:18:08.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.983+0000 7f36aa015700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3690003a20 0x7f36a415bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:08.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.983+0000 7f36aa015700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3690004870 0x7f36a415d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:08.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.983+0000 7f36aa015700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3690005350 0x7f36a41f0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:08.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.983+0000 7f36aa015700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a415e510 con 0x7f3690005350 2024-09-13T18:18:08.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.983+0000 7f36aa015700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a415df50 con 0x7f3690004870 2024-09-13T18:18:08.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.983+0000 7f36aa015700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a415e230 con 0x7f3690003a20 2024-09-13T18:18:08.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.983+0000 7f36a9814700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3690005350 msgr2=0x7f36a41f0130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:18:08.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.983+0000 7f36a9814700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3690005350 0x7f36a41f0130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:18:08.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.983+0000 7f36a9013700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3690003a20 0x7f36a415bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:08.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.983+0000 7f36a9013700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3690003a20 0x7f36a415bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:50838/0 (socket says 172.21.15.40:50838) 2024-09-13T18:18:08.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.983+0000 7f36a9013700 1 -- 172.21.15.40:0/413488664 learned_addr learned my addr 172.21.15.40:0/413488664 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:08.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.983+0000 7f36a9013700 1 -- 172.21.15.40:0/413488664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3690004870 msgr2=0x7f36a415d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:08.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.983+0000 7f36a9013700 1 --2- 172.21.15.40:0/413488664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3690004870 0x7f36a415d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:08.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.983+0000 7f36a9013700 1 -- 172.21.15.40:0/413488664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3690005350 msgr2=0x7f36a41f0130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:18:08.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.983+0000 7f36a9013700 1 --2- 172.21.15.40:0/413488664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3690005350 0x7f36a41f0130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:08.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.983+0000 7f36a9013700 1 -- 172.21.15.40:0/413488664 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a415b430 con 0x7f3690003a20 2024-09-13T18:18:08.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.983+0000 7f36a9013700 1 --2- 172.21.15.40:0/413488664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3690003a20 0x7f36a415bf10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f36a0003940 tx=0x7f36a000cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:08.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.991+0000 7f3699ffb700 1 -- 172.21.15.40:0/413488664 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a00074e0 con 0x7f3690003a20 2024-09-13T18:18:09.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.991+0000 7f36aa015700 1 -- 172.21.15.40:0/413488664 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a415b710 con 0x7f3690003a20 2024-09-13T18:18:09.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.991+0000 7f36aa015700 1 -- 172.21.15.40:0/413488664 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a40c29a0 con 0x7f3690003a20 2024-09-13T18:18:09.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.995+0000 7f3699ffb700 1 -- 172.21.15.40:0/413488664 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a0004630 con 0x7f3690003a20 2024-09-13T18:18:09.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.995+0000 7f3699ffb700 1 -- 172.21.15.40:0/413488664 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a00049e0 con 0x7f3690003a20 2024-09-13T18:18:09.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.995+0000 7f3699ffb700 1 -- 172.21.15.40:0/413488664 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f36a000a3d0 con 0x7f3690003a20 2024-09-13T18:18:09.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.995+0000 7f3699ffb700 1 --2- 172.21.15.40:0/413488664 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3688066f30 0x7f36880693f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:09.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.995+0000 7f3699ffb700 1 -- 172.21.15.40:0/413488664 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f36a0083d10 con 0x7f3690003a20 2024-09-13T18:18:09.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.995+0000 7f36a8812700 1 --2- 172.21.15.40:0/413488664 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3688066f30 0x7f36880693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:09.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.995+0000 7f36aa015700 1 -- 172.21.15.40:0/413488664 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3684005310 con 0x7f3690003a20 2024-09-13T18:18:09.005 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:08.995+0000 7f36a8812700 1 --2- 172.21.15.40:0/413488664 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3688066f30 0x7f36880693f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f36a406d0a0 tx=0x7f369c016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:09.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:09.007+0000 7f3699ffb700 1 -- 172.21.15.40:0/413488664 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f36a0050d50 con 0x7f3690003a20 2024-09-13T18:18:09.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:09.895+0000 7f36aa015700 1 -- 172.21.15.40:0/413488664 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3684005b40 con 0x7f3690003a20 2024-09-13T18:18:09.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:09.895+0000 7f3699ffb700 1 -- 172.21.15.40:0/413488664 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f36a0018030 con 0x7f3690003a20 2024-09-13T18:18:09.903 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":220,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:18:09.906 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:09.899+0000 7f36aa015700 1 -- 172.21.15.40:0/413488664 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3688066f30 msgr2=0x7f36880693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:09.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:09.899+0000 7f36aa015700 1 --2- 172.21.15.40:0/413488664 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3688066f30 0x7f36880693f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f36a406d0a0 tx=0x7f369c016000 comp rx=0 tx=0).stop 2024-09-13T18:18:09.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:09.899+0000 7f36aa015700 1 -- 172.21.15.40:0/413488664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3690003a20 msgr2=0x7f36a415bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:09.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:09.899+0000 7f36aa015700 1 --2- 172.21.15.40:0/413488664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3690003a20 0x7f36a415bf10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f36a0003940 tx=0x7f36a000cbf0 comp rx=0 tx=0).stop 2024-09-13T18:18:09.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:09.899+0000 7f36aa015700 1 -- 172.21.15.40:0/413488664 shutdown_connections 2024-09-13T18:18:09.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:09.899+0000 7f36aa015700 1 --2- 172.21.15.40:0/413488664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3690005350 0x7f36a41f0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:09.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:09.899+0000 7f36aa015700 1 --2- 172.21.15.40:0/413488664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3690004870 0x7f36a415d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:09.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:09.899+0000 7f36aa015700 1 --2- 172.21.15.40:0/413488664 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3688066f30 0x7f36880693f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:09.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:09.899+0000 7f36aa015700 1 --2- 172.21.15.40:0/413488664 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3690003a20 0x7f36a415bf10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:09.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:09.899+0000 7f36aa015700 1 -- 172.21.15.40:0/413488664 >> 172.21.15.40:0/413488664 conn(0x7f36a40cabc0 msgr2=0x7f36a40c34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:09.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:09.899+0000 7f36aa015700 1 -- 172.21.15.40:0/413488664 shutdown_connections 2024-09-13T18:18:09.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:09.899+0000 7f36aa015700 1 -- 172.21.15.40:0/413488664 wait complete. 2024-09-13T18:18:09.924 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T18:18:09.924 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-13T18:18:09.925 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:18:10.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.203+0000 7f94a15b9700 1 Processor -- start 2024-09-13T18:18:10.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.203+0000 7f94a15b9700 1 -- start start 2024-09-13T18:18:10.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.203+0000 7f94a15b9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f949c156c10 0x7f949c0714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:10.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.203+0000 7f94a15b9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f949c0c7010 0x7f949c160b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:10.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.203+0000 7f94a15b9700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f949c0c7570 con 0x7f949c071110 2024-09-13T18:18:10.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.203+0000 7f94a15b9700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f949c071a30 con 0x7f949c0c7010 2024-09-13T18:18:10.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.203+0000 7f94a15b9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949c06dde0 con 0x7f949c156c10 2024-09-13T18:18:10.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.203+0000 7f949affd700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f949c071110 legacy=0x7f949c156500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:18:10.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.203+0000 7f949b7fe700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f949c0c7010 msgr2=0x7f949c160b80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:18:10.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.203+0000 7f949b7fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f949c0c7010 0x7f949c160b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:18:10.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.203+0000 7f949a7fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f949c156c10 0x7f949c0714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:10.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.207+0000 7f949a7fc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f949c156c10 0x7f949c0714f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:50848/0 (socket says 172.21.15.40:50848) 2024-09-13T18:18:10.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.207+0000 7f949a7fc700 1 -- 172.21.15.40:0/1953922790 learned_addr learned my addr 172.21.15.40:0/1953922790 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:10.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.235+0000 7f949a7fc700 1 -- 172.21.15.40:0/1953922790 >> v2:172.21.15.40:3300/0 conn(0x7f949c0c7010 msgr2=0x7f949c160b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:18:10.242 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.235+0000 7f949a7fc700 1 --2- 172.21.15.40:0/1953922790 >> v2:172.21.15.40:3300/0 conn(0x7f949c0c7010 0x7f949c160b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:10.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.235+0000 7f949a7fc700 1 -- 172.21.15.40:0/1953922790 >> v1:172.21.15.40:6789/0 conn(0x7f949c071110 legacy=0x7f949c156500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:18:10.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.235+0000 7f949a7fc700 1 -- 172.21.15.40:0/1953922790 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c0bff90 con 0x7f949c156c10 2024-09-13T18:18:10.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.239+0000 7f949a7fc700 1 --2- 172.21.15.40:0/1953922790 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f949c156c10 0x7f949c0714f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f949400d150 tx=0x7f949400d460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89199038b632f201 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:10.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.251+0000 7f9499ffb700 1 -- 172.21.15.40:0/1953922790 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9494010070 con 0x7f949c156c10 2024-09-13T18:18:10.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.251+0000 7f9499ffb700 1 -- 172.21.15.40:0/1953922790 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949400dd80 con 0x7f949c156c10 2024-09-13T18:18:10.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.251+0000 7f94a15b9700 1 -- 172.21.15.40:0/1953922790 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f949c156c10 msgr2=0x7f949c0714f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:10.259 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.251+0000 7f94a15b9700 1 --2- 172.21.15.40:0/1953922790 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f949c156c10 0x7f949c0714f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f949400d150 tx=0x7f949400d460 comp rx=0 tx=0).stop 2024-09-13T18:18:10.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.255+0000 7f94a15b9700 1 -- 172.21.15.40:0/1953922790 shutdown_connections 2024-09-13T18:18:10.262 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.255+0000 7f94a15b9700 1 --2- 172.21.15.40:0/1953922790 >> v2:172.21.15.40:3300/0 conn(0x7f949c0c7010 0x7f949c160b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:10.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.255+0000 7f94a15b9700 1 --2- 172.21.15.40:0/1953922790 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f949c156c10 0x7f949c0714f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:10.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.255+0000 7f94a15b9700 1 -- 172.21.15.40:0/1953922790 >> 172.21.15.40:0/1953922790 conn(0x7f949c066090 msgr2=0x7f949c0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:10.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.263+0000 7f94a15b9700 1 -- 172.21.15.40:0/1953922790 shutdown_connections 2024-09-13T18:18:10.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.263+0000 7f94a15b9700 1 -- 172.21.15.40:0/1953922790 wait complete. 2024-09-13T18:18:10.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.263+0000 7f94a15b9700 1 Processor -- start 2024-09-13T18:18:10.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.263+0000 7f94a15b9700 1 -- start start 2024-09-13T18:18:10.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.267+0000 7f94a15b9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f949c071110 0x7f949c14bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:10.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.267+0000 7f94a15b9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f949c0c7010 0x7f949c14e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:10.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.267+0000 7f94a15b9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f949c156c10 0x7f949c14e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:10.274 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.267+0000 7f94a15b9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c145cd0 con 0x7f949c156c10 2024-09-13T18:18:10.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.267+0000 7f94a15b9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c14b800 con 0x7f949c0c7010 2024-09-13T18:18:10.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.267+0000 7f94a15b9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949c14ffa0 con 0x7f949c071110 2024-09-13T18:18:10.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.267+0000 7f949b7fe700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f949c156c10 msgr2=0x7f949c14e930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:18:10.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.267+0000 7f949b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f949c156c10 0x7f949c14e930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:18:10.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.267+0000 7f949a7fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f949c0c7010 0x7f949c14e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:10.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.267+0000 7f949a7fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f949c0c7010 0x7f949c14e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:37140/0 (socket says 172.21.15.40:37140) 2024-09-13T18:18:10.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.267+0000 7f949a7fc700 1 -- 172.21.15.40:0/2180452433 learned_addr learned my addr 172.21.15.40:0/2180452433 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:10.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.267+0000 7f949affd700 1 --2- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f949c071110 0x7f949c14bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:10.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.267+0000 7f949affd700 1 -- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f949c0c7010 msgr2=0x7f949c14e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:10.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.267+0000 7f949affd700 1 --2- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f949c0c7010 0x7f949c14e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:10.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.267+0000 7f949affd700 1 -- 172.21.15.40:0/2180452433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f949c156c10 msgr2=0x7f949c14e930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:18:10.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.267+0000 7f949affd700 1 --2- 172.21.15.40:0/2180452433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f949c156c10 0x7f949c14e930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:10.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.267+0000 7f949affd700 1 -- 172.21.15.40:0/2180452433 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949400bd40 con 0x7f949c071110 2024-09-13T18:18:10.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.267+0000 7f949affd700 1 --2- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f949c071110 0x7f949c14bc90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f949000b4e0 tx=0x7f949000b7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:10.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.271+0000 7f94837fe700 1 -- 172.21.15.40:0/2180452433 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490003e70 con 0x7f949c071110 2024-09-13T18:18:10.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.271+0000 7f94837fe700 1 -- 172.21.15.40:0/2180452433 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94900050f0 con 0x7f949c071110 2024-09-13T18:18:10.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.271+0000 7f94a15b9700 1 -- 172.21.15.40:0/2180452433 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949c1f40d0 con 0x7f949c071110 2024-09-13T18:18:10.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.271+0000 7f94a15b9700 1 -- 172.21.15.40:0/2180452433 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949c14aed0 con 0x7f949c071110 2024-09-13T18:18:10.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.271+0000 7f94837fe700 1 -- 172.21.15.40:0/2180452433 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94900054c0 con 0x7f949c071110 2024-09-13T18:18:10.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.283+0000 7f94837fe700 1 -- 172.21.15.40:0/2180452433 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9490005620 con 0x7f949c071110 2024-09-13T18:18:10.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.283+0000 7f94837fe700 1 --2- 172.21.15.40:0/2180452433 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9488066730 0x7f9488068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:10.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.283+0000 7f949a7fc700 1 --2- 172.21.15.40:0/2180452433 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9488066730 0x7f9488068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:10.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.283+0000 7f94837fe700 1 -- 172.21.15.40:0/2180452433 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9490083860 con 0x7f949c071110 2024-09-13T18:18:10.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.283+0000 7f949a7fc700 1 --2- 172.21.15.40:0/2180452433 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9488066730 0x7f9488068bf0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f949c1e9cc0 tx=0x7f949400b000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:10.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.291+0000 7f94a15b9700 1 --2- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c002a40 0x7f947c004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:10.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.291+0000 7f94a15b9700 1 -- 172.21.15.40:0/2180452433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c007f40 con 0x7f947c002a40 2024-09-13T18:18:10.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.291+0000 7f949b7fe700 1 --2- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c002a40 0x7f947c004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:10.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.291+0000 7f949b7fe700 1 -- 172.21.15.40:0/2180452433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f947c008100 con 0x7f947c002a40 2024-09-13T18:18:10.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.291+0000 7f949b7fe700 1 --2- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c002a40 0x7f947c004f00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f948c00b450 tx=0x7f948c00b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:10.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.291+0000 7f94837fe700 1 -- 172.21.15.40:0/2180452433 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c003e20 con 0x7f947c002a40 2024-09-13T18:18:10.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.291+0000 7f94837fe700 1 -- 172.21.15.40:0/2180452433 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f948c005070 con 0x7f947c002a40 2024-09-13T18:18:10.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.291+0000 7f94837fe700 1 -- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c002a40 msgr2=0x7f947c004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:10.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.291+0000 7f94837fe700 1 --2- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c002a40 0x7f947c004f00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f948c00b450 tx=0x7f948c00b810 comp rx=0 tx=0).stop 2024-09-13T18:18:10.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.291+0000 7f94837fe700 1 -- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c002a40 msgr2=0x7f947c004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:18:10.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.291+0000 7f94837fe700 1 --2- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c002a40 0x7f947c004f00 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f948c00b450 tx=0x7f948c00b810 comp rx=0 tx=0).stop 2024-09-13T18:18:10.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.331+0000 7f94817fa700 1 --2- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c001de0 0x7f947c00cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:10.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.331+0000 7f94817fa700 1 -- 172.21.15.40:0/2180452433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c002390 con 0x7f947c001de0 2024-09-13T18:18:10.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.331+0000 7f949b7fe700 1 --2- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c001de0 0x7f947c00cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:10.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.335+0000 7f949b7fe700 1 -- 172.21.15.40:0/2180452433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f947c00a2e0 con 0x7f947c001de0 2024-09-13T18:18:10.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.339+0000 7f949b7fe700 1 --2- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c001de0 0x7f947c00cbf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f948c009fd0 tx=0x7f948c00e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:10.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.339+0000 7f94837fe700 1 -- 172.21.15.40:0/2180452433 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c024660 con 0x7f947c001de0 2024-09-13T18:18:10.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.339+0000 7f94837fe700 1 -- 172.21.15.40:0/2180452433 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4939 (secure 0 0 0) 0x7f948c01be20 con 0x7f947c001de0 2024-09-13T18:18:10.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.339+0000 7f94837fe700 1 -- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c001de0 msgr2=0x7f947c00cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:10.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.339+0000 7f94837fe700 1 --2- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c001de0 0x7f947c00cbf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f948c009fd0 tx=0x7f948c00e000 comp rx=0 tx=0).stop 2024-09-13T18:18:10.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.339+0000 7f94837fe700 1 -- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c001de0 msgr2=0x7f947c00cbf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:18:10.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.339+0000 7f94837fe700 1 --2- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f947c001de0 0x7f947c00cbf0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f948c009fd0 tx=0x7f948c00e000 comp rx=0 tx=0).stop 2024-09-13T18:18:10.347 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:18:10.347 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:18:10.347 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:18:10.347 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:18:10.347 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 220, 2024-09-13T18:18:10.347 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:18:10.348 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:18:10.348 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:18:10.348 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:10.348 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:18:10.348 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:18:10.348 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:18:10.348 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:18:10.348 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:18:10.348 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:18:10.348 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:18:10.348 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:18:10.348 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:18:10.348 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:18:10.349 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:18:10.349 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:18:10.349 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:18:10.349 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:18:10.349 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:10.349 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:18:10.349 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:18:10.349 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:18:10.349 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:18:10.349 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:18:10.349 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:18:10.349 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:18:10.349 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:18:10.350 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:18:10.350 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:18:10.350 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:18:10.350 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:18:10.350 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:10.350 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.350 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:18:10.350 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:18:10.350 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:18:10.350 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:18:10.350 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:18:10.350 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:18:10.350 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:18:10.350 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:18:10.351 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:18:10.351 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:18:10.351 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:18:10.351 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:18:10.351 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:18:10.351 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:18:10.351 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:18:10.351 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:18:10.351 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:18:10.351 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:18:10.351 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:18:10.351 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:18:10.351 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:18:10.352 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:18:10.352 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:18:10.352 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:18:10.352 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:18:10.352 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:18:10.352 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:18:10.352 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:10.352 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:18:10.352 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.353 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:18:10.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.353 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:18:10.353 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:18:10.353 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:18:10.353 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:18:10.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.353 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:18:10.353 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:18:10.353 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:10.353 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.353 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.353 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:18:10.353 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:18:10.354 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:10.354 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:10.354 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:10.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.354 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:18:10.354 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:18:10.354 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:18:10.354 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:18:10.354 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:18:10.354 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.354 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.354 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:18:10.355 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:18:10.355 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:18:10.355 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:18:10.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.355 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:18:10.355 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:18:10.355 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:10.355 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.355 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.355 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:18:10.355 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:18:10.355 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:10.355 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:10.356 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:10.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.356 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:18:10.356 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:18:10.356 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:18:10.356 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:18:10.356 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:18:10.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.356 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.356 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:18:10.356 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:18:10.356 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:18:10.357 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:18:10.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.357 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:18:10.357 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:18:10.357 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:10.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.357 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.357 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:18:10.357 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:18:10.357 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:10.357 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:10.357 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:10.357 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.357 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:18:10.358 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:18:10.358 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:18:10.358 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:18:10.358 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:18:10.358 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:10.358 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:10.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.358 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:18:10.358 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:18:10.358 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.358 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:10.358 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:10.358 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:18:10.359 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:10.359 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:10.359 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:18:10.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.359 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:10.359 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:10.359 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:18:10.359 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:10.359 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:10.359 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:18:10.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.359 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:10.359 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:10.359 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:18:10.360 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:10.360 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:10.360 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:18:10.360 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.360 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:10.360 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:10.360 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:18:10.360 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:10.360 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:10.360 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:18:10.361 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.361 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:10.361 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:10.361 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:18:10.361 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:10.361 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:10.361 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.362 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:18:10.362 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:18:10.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.343+0000 7f94817fa700 1 -- 172.21.15.40:0/2180452433 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9488066730 msgr2=0x7f9488068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:10.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.343+0000 7f94817fa700 1 --2- 172.21.15.40:0/2180452433 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9488066730 0x7f9488068bf0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f949c1e9cc0 tx=0x7f949400b000 comp rx=0 tx=0).stop 2024-09-13T18:18:10.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.343+0000 7f94817fa700 1 -- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f949c071110 msgr2=0x7f949c14bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:10.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.343+0000 7f94817fa700 1 --2- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f949c071110 0x7f949c14bc90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f949000b4e0 tx=0x7f949000b7f0 comp rx=0 tx=0).stop 2024-09-13T18:18:10.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.343+0000 7f949b7fe700 1 -- 172.21.15.40:0/2180452433 reap_dead start 2024-09-13T18:18:10.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.343+0000 7f94817fa700 1 -- 172.21.15.40:0/2180452433 shutdown_connections 2024-09-13T18:18:10.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.343+0000 7f94817fa700 1 --2- 172.21.15.40:0/2180452433 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f949c071110 0x7f949c14bc90 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f949000b4e0 tx=0x7f949000b7f0 comp rx=0 tx=0).stop 2024-09-13T18:18:10.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.343+0000 7f94817fa700 1 -- 172.21.15.40:0/2180452433 >> 172.21.15.40:0/2180452433 conn(0x7f949c066090 msgr2=0x7f949c066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:10.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.343+0000 7f94817fa700 1 -- 172.21.15.40:0/2180452433 shutdown_connections 2024-09-13T18:18:10.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.343+0000 7f94817fa700 1 -- 172.21.15.40:0/2180452433 wait complete. 2024-09-13T18:18:10.407 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:18:10.407 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T18:18:10.407 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:18:10.678 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:18:10.680 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:18:10.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.703+0000 7ff60598f700 1 Processor -- start 2024-09-13T18:18:10.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.703+0000 7ff60598f700 1 -- start start 2024-09-13T18:18:10.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.703+0000 7ff60598f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff600163d00 0x7ff6000c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.703+0000 7ff60598f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff600167290 con 0x7ff6000c7f60 2024-09-13T18:18:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.703+0000 7ff60598f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff600166680 con 0x7ff6000c8340 2024-09-13T18:18:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.703+0000 7ff60598f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6000c1630 con 0x7ff600163d00 2024-09-13T18:18:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.703+0000 7ff5fffff700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7ff600163d00 msgr2=0x7ff6000c7250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:18:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.703+0000 7ff60498d700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7ff6000c7f60 legacy=0x7ff6001635f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:18:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.703+0000 7ff5fffff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff600163d00 0x7ff6000c7250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:18:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.703+0000 7ff60518e700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff6000c8340 0x7ff60006d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:34780/0 (socket says 172.21.15.40:34780) 2024-09-13T18:18:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.703+0000 7ff60518e700 1 -- 172.21.15.40:0/1714714887 learned_addr learned my addr 172.21.15.40:0/1714714887 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:18:10.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.703+0000 7ff5ff7fe700 1 -- 172.21.15.40:0/1714714887 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 623525081 0 0) 0x7ff600166680 con 0x7ff6000c8340 2024-09-13T18:18:10.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff5ff7fe700 1 -- 172.21.15.40:0/1714714887 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5f00035d0 con 0x7ff6000c8340 2024-09-13T18:18:10.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff5ff7fe700 1 -- 172.21.15.40:0/1714714887 <== mon.1 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1800495849 0 0) 0x7ff5f00035d0 con 0x7ff6000c8340 2024-09-13T18:18:10.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff5ff7fe700 1 -- 172.21.15.40:0/1714714887 >> v2:172.21.15.40:3300/0 conn(0x7ff600163d00 msgr2=0x7ff6000c7250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:18:10.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff5ff7fe700 1 --2- 172.21.15.40:0/1714714887 >> v2:172.21.15.40:3300/0 conn(0x7ff600163d00 0x7ff6000c7250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:10.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff5ff7fe700 1 -- 172.21.15.40:0/1714714887 >> v1:172.21.15.40:6789/0 conn(0x7ff6000c7f60 legacy=0x7ff6001635f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:18:10.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff5ff7fe700 1 -- 172.21.15.40:0/1714714887 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff600166960 con 0x7ff6000c8340 2024-09-13T18:18:10.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff5ff7fe700 1 -- 172.21.15.40:0/1714714887 <== mon.1 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7ff5f80019f0 con 0x7ff6000c8340 2024-09-13T18:18:10.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff5ff7fe700 1 -- 172.21.15.40:0/1714714887 >> v1:172.21.15.43:6789/0 conn(0x7ff6000c8340 legacy=0x7ff60006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:10.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff5ff7fe700 1 --2- 172.21.15.40:0/1714714887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5f0003bf0 0x7ff5f0003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:10.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff5ff7fe700 1 --2- 172.21.15.40:0/1714714887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5f0004730 0x7ff5f0004bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:10.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff5ff7fe700 1 --2- 172.21.15.40:0/1714714887 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5f000b610 0x7ff5f000daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:10.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff60498d700 1 -- 172.21.15.40:0/1714714887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5f0003bf0 msgr2=0x7ff5f0003fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:18:10.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff60498d700 1 --2- 172.21.15.40:0/1714714887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5f0003bf0 0x7ff5f0003fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:18:10.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff5ff7fe700 1 -- 172.21.15.40:0/1714714887 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6000c1630 con 0x7ff5f0003bf0 2024-09-13T18:18:10.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff5ff7fe700 1 -- 172.21.15.40:0/1714714887 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f000e370 con 0x7ff5f0004730 2024-09-13T18:18:10.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff5ff7fe700 1 -- 172.21.15.40:0/1714714887 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f000e530 con 0x7ff5f000b610 2024-09-13T18:18:10.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff60518e700 1 --2- 172.21.15.40:0/1714714887 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5f000b610 0x7ff5f000daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:10.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff5fffff700 1 --2- 172.21.15.40:0/1714714887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5f0004730 0x7ff5f0004bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:10.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff60518e700 1 -- 172.21.15.40:0/1714714887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5f0004730 msgr2=0x7ff5f0004bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:18:10.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff60518e700 1 --2- 172.21.15.40:0/1714714887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5f0004730 0x7ff5f0004bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:10.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff60518e700 1 -- 172.21.15.40:0/1714714887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5f0003bf0 msgr2=0x7ff5f0003fd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:18:10.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff60518e700 1 --2- 172.21.15.40:0/1714714887 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff5f0003bf0 0x7ff5f0003fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:10.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff60518e700 1 -- 172.21.15.40:0/1714714887 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f00035d0 con 0x7ff5f000b610 2024-09-13T18:18:10.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.739+0000 7ff60518e700 1 -- 172.21.15.40:0/1714714887 reap_dead start 2024-09-13T18:18:10.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.743+0000 7ff60518e700 1 --2- 172.21.15.40:0/1714714887 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5f000b610 0x7ff5f000daa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8005660 tx=0x7ff5f0003880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffdffcc7151d00c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:10.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.747+0000 7ff5ff7fe700 1 -- 172.21.15.40:0/1714714887 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f800dab0 con 0x7ff5f000b610 2024-09-13T18:18:10.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.747+0000 7ff5ff7fe700 1 -- 172.21.15.40:0/1714714887 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f8016070 con 0x7ff5f000b610 2024-09-13T18:18:10.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.747+0000 7ff5ff7fe700 1 -- 172.21.15.40:0/1714714887 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f800de50 con 0x7ff5f000b610 2024-09-13T18:18:10.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.747+0000 7ff60598f700 1 -- 172.21.15.40:0/1714714887 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5f000b610 msgr2=0x7ff5f000daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:10.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.747+0000 7ff60598f700 1 --2- 172.21.15.40:0/1714714887 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5f000b610 0x7ff5f000daa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8005660 tx=0x7ff5f0003880 comp rx=0 tx=0).stop 2024-09-13T18:18:10.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.755+0000 7ff60598f700 1 -- 172.21.15.40:0/1714714887 shutdown_connections 2024-09-13T18:18:10.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.755+0000 7ff60598f700 1 --2- 172.21.15.40:0/1714714887 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5f000b610 0x7ff5f000daa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:10.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.755+0000 7ff60598f700 1 -- 172.21.15.40:0/1714714887 >> 172.21.15.40:0/1714714887 conn(0x7ff600066090 msgr2=0x7ff6000664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:10.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.763+0000 7ff60598f700 1 -- 172.21.15.40:0/1714714887 shutdown_connections 2024-09-13T18:18:10.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.763+0000 7ff60598f700 1 -- 172.21.15.40:0/1714714887 wait complete. 2024-09-13T18:18:10.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.767+0000 7ff60598f700 1 Processor -- start 2024-09-13T18:18:10.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.767+0000 7ff60598f700 1 -- start start 2024-09-13T18:18:10.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.767+0000 7ff60598f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5f000b610 0x7ff600071330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:10.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.767+0000 7ff60598f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60015e370 0x7ff6001f8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:10.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.767+0000 7ff60598f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6001f8c80 0x7ff6000c2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:10.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.767+0000 7ff60598f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6000c21d0 con 0x7ff60015e370 2024-09-13T18:18:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.767+0000 7ff60598f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff600204b80 con 0x7ff6001f8c80 2024-09-13T18:18:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.767+0000 7ff60598f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff600071ee0 con 0x7ff5f000b610 2024-09-13T18:18:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.771+0000 7ff5fffff700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60015e370 msgr2=0x7ff6001f8740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:18:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.771+0000 7ff5fffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60015e370 0x7ff6001f8740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:18:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.771+0000 7ff60498d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5f000b610 0x7ff600071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.771+0000 7ff60498d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5f000b610 0x7ff600071330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:50874/0 (socket says 172.21.15.40:50874) 2024-09-13T18:18:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.771+0000 7ff60498d700 1 -- 172.21.15.40:0/757195112 learned_addr learned my addr 172.21.15.40:0/757195112 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.771+0000 7ff60498d700 1 -- 172.21.15.40:0/757195112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6001f8c80 msgr2=0x7ff6000c2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.771+0000 7ff60498d700 1 --2- 172.21.15.40:0/757195112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6001f8c80 0x7ff6000c2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.771+0000 7ff60498d700 1 -- 172.21.15.40:0/757195112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60015e370 msgr2=0x7ff6001f8740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:18:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.771+0000 7ff60498d700 1 --2- 172.21.15.40:0/757195112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff60015e370 0x7ff6001f8740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.771+0000 7ff60498d700 1 -- 172.21.15.40:0/757195112 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6000721c0 con 0x7ff5f000b610 2024-09-13T18:18:10.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.771+0000 7ff60498d700 1 --2- 172.21.15.40:0/757195112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5f000b610 0x7ff600071330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400b520 tx=0x7ff5f400b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:10.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.771+0000 7ff5fd7fa700 1 -- 172.21.15.40:0/757195112 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4003df0 con 0x7ff5f000b610 2024-09-13T18:18:10.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.771+0000 7ff60598f700 1 -- 172.21.15.40:0/757195112 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6000724a0 con 0x7ff5f000b610 2024-09-13T18:18:10.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.771+0000 7ff60598f700 1 -- 172.21.15.40:0/757195112 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff600071870 con 0x7ff5f000b610 2024-09-13T18:18:10.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.771+0000 7ff5fd7fa700 1 -- 172.21.15.40:0/757195112 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f40050f0 con 0x7ff5f000b610 2024-09-13T18:18:10.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.775+0000 7ff5fd7fa700 1 -- 172.21.15.40:0/757195112 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f40054a0 con 0x7ff5f000b610 2024-09-13T18:18:10.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.775+0000 7ff60598f700 1 --2- 172.21.15.40:0/757195112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5e0002a40 0x7ff5e0004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:10.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.775+0000 7ff60598f700 1 -- 172.21.15.40:0/757195112 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e0007f40 con 0x7ff5e0002a40 2024-09-13T18:18:10.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.775+0000 7ff5fd7fa700 1 -- 172.21.15.40:0/757195112 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff5f4005680 con 0x7ff5f000b610 2024-09-13T18:18:10.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.775+0000 7ff5fffff700 1 --2- 172.21.15.40:0/757195112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5e0002a40 0x7ff5e0004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:10.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.779+0000 7ff5fffff700 1 -- 172.21.15.40:0/757195112 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5e0008100 con 0x7ff5e0002a40 2024-09-13T18:18:10.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.779+0000 7ff5fffff700 1 --2- 172.21.15.40:0/757195112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5e0002a40 0x7ff5e0004f00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff6000c1bd0 tx=0x7ff5ec00dd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:10.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.783+0000 7ff5fd7fa700 1 --2- 172.21.15.40:0/757195112 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff5e4066730 0x7ff5e4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:10.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.783+0000 7ff5fd7fa700 1 -- 172.21.15.40:0/757195112 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff5f4083910 con 0x7ff5f000b610 2024-09-13T18:18:10.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.783+0000 7ff60518e700 1 --2- 172.21.15.40:0/757195112 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff5e4066730 0x7ff5e4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:10.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.783+0000 7ff5fd7fa700 1 -- 172.21.15.40:0/757195112 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec003e70 con 0x7ff5e0002a40 2024-09-13T18:18:10.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.783+0000 7ff5fd7fa700 1 -- 172.21.15.40:0/757195112 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff5ec00b7b0 con 0x7ff5e0002a40 2024-09-13T18:18:10.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.783+0000 7ff60518e700 1 --2- 172.21.15.40:0/757195112 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff5e4066730 0x7ff5e4068bf0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000e6c0 tx=0x7ff5f8007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:10.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.783+0000 7ff5fd7fa700 1 -- 172.21.15.40:0/757195112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5e0002a40 msgr2=0x7ff5e0004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:10.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.783+0000 7ff5fd7fa700 1 --2- 172.21.15.40:0/757195112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5e0002a40 0x7ff5e0004f00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff6000c1bd0 tx=0x7ff5ec00dd70 comp rx=0 tx=0).stop 2024-09-13T18:18:10.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.783+0000 7ff5fd7fa700 1 -- 172.21.15.40:0/757195112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5e0002a40 msgr2=0x7ff5e0004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:18:10.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.783+0000 7ff5fd7fa700 1 --2- 172.21.15.40:0/757195112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5e0002a40 0x7ff5e0004f00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:10.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.807+0000 7ff60598f700 1 --2- 172.21.15.40:0/757195112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5e0007020 0x7ff5e00074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:10.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.807+0000 7ff60598f700 1 -- 172.21.15.40:0/757195112 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e000a400 con 0x7ff5e0007020 2024-09-13T18:18:10.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.807+0000 7ff5fffff700 1 --2- 172.21.15.40:0/757195112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5e0007020 0x7ff5e00074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:10.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.807+0000 7ff5fffff700 1 -- 172.21.15.40:0/757195112 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff5e000a5c0 con 0x7ff5e0007020 2024-09-13T18:18:10.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.807+0000 7ff5fffff700 1 --2- 172.21.15.40:0/757195112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5e0007020 0x7ff5e00074b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec014000 tx=0x7ff5ec005b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:10.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.811+0000 7ff5fd7fa700 1 -- 172.21.15.40:0/757195112 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec01d880 con 0x7ff5e0007020 2024-09-13T18:18:10.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.811+0000 7ff5fd7fa700 1 -- 172.21.15.40:0/757195112 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4771 (secure 0 0 0) 0x7ff5ec021070 con 0x7ff5e0007020 2024-09-13T18:18:10.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.811+0000 7ff5fd7fa700 1 -- 172.21.15.40:0/757195112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5e0007020 msgr2=0x7ff5e00074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:10.821 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.811+0000 7ff5fd7fa700 1 --2- 172.21.15.40:0/757195112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5e0007020 0x7ff5e00074b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec014000 tx=0x7ff5ec005b30 comp rx=0 tx=0).stop 2024-09-13T18:18:10.822 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:18:10.822 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:18:10.822 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:18:10.823 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:18:10.823 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 220, 2024-09-13T18:18:10.823 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:18:10.823 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:18:10.823 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:18:10.824 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:10.824 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:18:10.824 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:18:10.824 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:18:10.824 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:18:10.825 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:18:10.825 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:18:10.825 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:18:10.825 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:18:10.825 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:18:10.825 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:18:10.826 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:18:10.826 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:18:10.826 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:18:10.826 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:18:10.826 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:10.827 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:18:10.827 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:18:10.827 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:18:10.827 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:18:10.827 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:18:10.828 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:18:10.828 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:18:10.828 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:18:10.828 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:18:10.828 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:18:10.828 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:18:10.829 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:18:10.829 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:10.829 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.829 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:18:10.829 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:18:10.830 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:18:10.830 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:18:10.830 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:18:10.830 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:18:10.830 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:18:10.830 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:18:10.831 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:18:10.831 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:18:10.831 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:18:10.831 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:18:10.831 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:18:10.831 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:18:10.831 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:18:10.831 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:18:10.832 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:18:10.832 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:18:10.832 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:18:10.832 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:18:10.832 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:18:10.832 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:18:10.832 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:18:10.832 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:18:10.832 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:18:10.832 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:18:10.832 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:18:10.833 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:10.833 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:18:10.833 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.833 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:18:10.833 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.833 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:18:10.833 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:18:10.834 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:18:10.834 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:18:10.834 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.834 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:18:10.834 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:18:10.834 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:10.834 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.834 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.834 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:18:10.834 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:18:10.835 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:10.835 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:10.835 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:10.835 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.835 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:18:10.835 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:18:10.835 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:18:10.835 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:18:10.835 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:18:10.835 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.835 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.835 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:18:10.835 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:18:10.836 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:18:10.836 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:18:10.836 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.836 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:18:10.836 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:18:10.836 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:10.836 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.836 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.836 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:18:10.836 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:18:10.836 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:10.836 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:10.837 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:10.837 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.837 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:18:10.837 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:18:10.837 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:18:10.837 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:18:10.837 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:18:10.837 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:18:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:18:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:18:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:18:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:18:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:18:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:18:10.838 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:18:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:18:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:18:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:18:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:18:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:18:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:18:10.839 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:18:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:18:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:18:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:18:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:18:10.840 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:18:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:18:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:18:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:10.841 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:18:10.842 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:18:10.842 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.811+0000 7ff5fd7fa700 1 -- 172.21.15.40:0/757195112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5e0007020 msgr2=0x7ff5e00074b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:18:10.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.811+0000 7ff5fd7fa700 1 --2- 172.21.15.40:0/757195112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5e0007020 0x7ff5e00074b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:10.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.815+0000 7ff5eaffd700 1 -- 172.21.15.40:0/757195112 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff5e4066730 msgr2=0x7ff5e4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:10.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.815+0000 7ff5eaffd700 1 --2- 172.21.15.40:0/757195112 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff5e4066730 0x7ff5e4068bf0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000e6c0 tx=0x7ff5f8007000 comp rx=0 tx=0).stop 2024-09-13T18:18:10.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.815+0000 7ff60518e700 1 -- 172.21.15.40:0/757195112 reap_dead start 2024-09-13T18:18:10.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.815+0000 7ff5eaffd700 1 -- 172.21.15.40:0/757195112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5f000b610 msgr2=0x7ff600071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:10.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.815+0000 7ff5eaffd700 1 --2- 172.21.15.40:0/757195112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5f000b610 0x7ff600071330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400b520 tx=0x7ff5f400b8e0 comp rx=0 tx=0).stop 2024-09-13T18:18:10.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.815+0000 7ff5eaffd700 1 -- 172.21.15.40:0/757195112 shutdown_connections 2024-09-13T18:18:10.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.815+0000 7ff5eaffd700 1 --2- 172.21.15.40:0/757195112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff5f000b610 0x7ff600071330 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:10.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.815+0000 7ff5eaffd700 1 -- 172.21.15.40:0/757195112 >> 172.21.15.40:0/757195112 conn(0x7ff600066090 msgr2=0x7ff600164bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:10.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.815+0000 7ff5eaffd700 1 -- 172.21.15.40:0/757195112 shutdown_connections 2024-09-13T18:18:10.843 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:10.819+0000 7ff5eaffd700 1 -- 172.21.15.40:0/757195112 wait complete. 2024-09-13T18:18:10.859 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:18:10.859 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:18:30.860 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T18:18:30.860 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-13T18:18:30.860 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T18:18:30.861 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T18:18:30.863 INFO:tasks.ceph.mon.a:Started 2024-09-13T18:18:30.863 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:18:30.863 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:18:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.219+0000 7f3a977ea700 1 Processor -- start 2024-09-13T18:18:31.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.219+0000 7f3a977ea700 1 -- start start 2024-09-13T18:18:31.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.219+0000 7f3a977ea700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9006fb20 0x7f3a9006d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:31.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.219+0000 7f3a977ea700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3a900bfe80 0x7f3a901502b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:31.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.219+0000 7f3a977ea700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a90071bf0 con 0x7f3a901507f0 2024-09-13T18:18:31.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.219+0000 7f3a977ea700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a90071330 con 0x7f3a900bfe80 2024-09-13T18:18:31.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.219+0000 7f3a977ea700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9015cd60 con 0x7f3a9006fb20 2024-09-13T18:18:31.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.223+0000 7f3a95586700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9006fb20 0x7f3a9006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:31.243 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.223+0000 7f3a95586700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9006fb20 0x7f3a9006d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:40598/0 (socket says 172.21.15.40:40598) 2024-09-13T18:18:31.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.223+0000 7f3a95586700 1 -- 172.21.15.40:0/163848644 learned_addr learned my addr 172.21.15.40:0/163848644 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:31.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.223+0000 7f3a95586700 1 -- 172.21.15.40:0/163848644 >> v2:172.21.15.40:3300/0 conn(0x7f3a900bfe80 msgr2=0x7f3a901502b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:18:31.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.227+0000 7f3a94d85700 1 -- 172.21.15.40:0/163848644 >> v2:172.21.15.40:3300/0 conn(0x7f3a900bfe80 msgr2=0x7f3a901502b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:18:31.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.227+0000 7f3a94d85700 1 --2- 172.21.15.40:0/163848644 >> v2:172.21.15.40:3300/0 conn(0x7f3a900bfe80 0x7f3a901502b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:18:31.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.227+0000 7f3a95586700 1 --2- 172.21.15.40:0/163848644 >> v2:172.21.15.40:3300/0 conn(0x7f3a900bfe80 0x7f3a901502b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:31.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.227+0000 7f3a95d87700 1 -- 172.21.15.40:0/163848644 >> v1:172.21.15.40:6789/0 conn(0x7f3a901507f0 legacy=0x7f3a900c04f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:18:31.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.235+0000 7f3a95586700 1 -- 172.21.15.40:0/163848644 >> v1:172.21.15.40:6789/0 conn(0x7f3a901507f0 legacy=0x7f3a900c04f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:18:31.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.235+0000 7f3a95586700 1 -- 172.21.15.40:0/163848644 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a900c73a0 con 0x7f3a9006fb20 2024-09-13T18:18:31.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.235+0000 7f3a95586700 1 --2- 172.21.15.40:0/163848644 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9006fb20 0x7f3a9006d570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009e80 tx=0x7f3a8800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b1b0d360d96cdd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:31.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.235+0000 7f3a87fff700 1 -- 172.21.15.40:0/163848644 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8800e070 con 0x7f3a9006fb20 2024-09-13T18:18:31.244 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.235+0000 7f3a87fff700 1 -- 172.21.15.40:0/163848644 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a88003a40 con 0x7f3a9006fb20 2024-09-13T18:18:31.245 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.235+0000 7f3a977ea700 1 -- 172.21.15.40:0/163848644 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9006fb20 msgr2=0x7f3a9006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:31.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.235+0000 7f3a977ea700 1 --2- 172.21.15.40:0/163848644 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9006fb20 0x7f3a9006d570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009e80 tx=0x7f3a8800b330 comp rx=0 tx=0).stop 2024-09-13T18:18:31.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.239+0000 7f3a977ea700 1 -- 172.21.15.40:0/163848644 shutdown_connections 2024-09-13T18:18:31.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.239+0000 7f3a977ea700 1 --2- 172.21.15.40:0/163848644 >> v2:172.21.15.40:3300/0 conn(0x7f3a900bfe80 0x7f3a901502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:31.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.239+0000 7f3a977ea700 1 --2- 172.21.15.40:0/163848644 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9006fb20 0x7f3a9006d570 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:31.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.239+0000 7f3a977ea700 1 -- 172.21.15.40:0/163848644 >> 172.21.15.40:0/163848644 conn(0x7f3a900cadd0 msgr2=0x7f3a900cb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:31.249 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.243+0000 7f3a977ea700 1 -- 172.21.15.40:0/163848644 shutdown_connections 2024-09-13T18:18:31.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.247+0000 7f3a977ea700 1 -- 172.21.15.40:0/163848644 wait complete. 2024-09-13T18:18:31.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.247+0000 7f3a977ea700 1 Processor -- start 2024-09-13T18:18:31.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.247+0000 7f3a977ea700 1 -- start start 2024-09-13T18:18:31.279 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.251+0000 7f3a977ea700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9006fb20 0x7f3a90147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:31.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.251+0000 7f3a977ea700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a900bfe80 0x7f3a900c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:31.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.251+0000 7f3a977ea700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a901507f0 0x7f3a901487a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:31.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.251+0000 7f3a977ea700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a901443d0 con 0x7f3a901507f0 2024-09-13T18:18:31.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.251+0000 7f3a977ea700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a90147d50 con 0x7f3a900bfe80 2024-09-13T18:18:31.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.251+0000 7f3a977ea700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9015c6c0 con 0x7f3a9006fb20 2024-09-13T18:18:31.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.251+0000 7f3a94d85700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a900bfe80 0x7f3a900c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:31.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.251+0000 7f3a94d85700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a900bfe80 0x7f3a900c9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:50882/0 (socket says 172.21.15.40:50882) 2024-09-13T18:18:31.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.251+0000 7f3a94d85700 1 -- 172.21.15.40:0/1327080714 learned_addr learned my addr 172.21.15.40:0/1327080714 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:31.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.251+0000 7f3a95586700 1 --2- 172.21.15.40:0/1327080714 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9006fb20 0x7f3a90147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:31.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.255+0000 7f3a95d87700 1 -- 172.21.15.40:0/1327080714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a901507f0 msgr2=0x7f3a901487a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:18:31.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.255+0000 7f3a95d87700 1 --2- 172.21.15.40:0/1327080714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a901507f0 0x7f3a901487a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:18:31.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.255+0000 7f3a95586700 1 -- 172.21.15.40:0/1327080714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a900bfe80 msgr2=0x7f3a900c9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:31.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.255+0000 7f3a95586700 1 --2- 172.21.15.40:0/1327080714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a900bfe80 0x7f3a900c9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:31.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.255+0000 7f3a95586700 1 -- 172.21.15.40:0/1327080714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a901507f0 msgr2=0x7f3a901487a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:18:31.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.255+0000 7f3a95586700 1 --2- 172.21.15.40:0/1327080714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a901507f0 0x7f3a901487a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:31.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.255+0000 7f3a95586700 1 -- 172.21.15.40:0/1327080714 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a88009b30 con 0x7f3a9006fb20 2024-09-13T18:18:31.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.259+0000 7f3a95586700 1 --2- 172.21.15.40:0/1327080714 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9006fb20 0x7f3a90147810 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009e50 tx=0x7f3a88004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:31.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.263+0000 7f3a85ffb700 1 -- 172.21.15.40:0/1327080714 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a880048a0 con 0x7f3a9006fb20 2024-09-13T18:18:31.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.263+0000 7f3a977ea700 1 -- 172.21.15.40:0/1327080714 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a90148310 con 0x7f3a9006fb20 2024-09-13T18:18:31.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.263+0000 7f3a977ea700 1 -- 172.21.15.40:0/1327080714 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9015c8d0 con 0x7f3a9006fb20 2024-09-13T18:18:31.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.263+0000 7f3a85ffb700 1 -- 172.21.15.40:0/1327080714 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a88012430 con 0x7f3a9006fb20 2024-09-13T18:18:31.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.263+0000 7f3a85ffb700 1 -- 172.21.15.40:0/1327080714 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88012820 con 0x7f3a9006fb20 2024-09-13T18:18:31.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.267+0000 7f3a85ffb700 1 -- 172.21.15.40:0/1327080714 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3a88012980 con 0x7f3a9006fb20 2024-09-13T18:18:31.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.267+0000 7f3a85ffb700 1 --2- 172.21.15.40:0/1327080714 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3a74066730 0x7f3a74068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:31.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.267+0000 7f3a85ffb700 1 -- 172.21.15.40:0/1327080714 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3a88085b70 con 0x7f3a9006fb20 2024-09-13T18:18:31.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.267+0000 7f3a94d85700 1 --2- 172.21.15.40:0/1327080714 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3a74066730 0x7f3a74068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:31.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.271+0000 7f3a94d85700 1 --2- 172.21.15.40:0/1327080714 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3a74066730 0x7f3a74068bf0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f3a80009500 tx=0x7f3a80009490 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:31.286 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.271+0000 7f3a977ea700 1 -- 172.21.15.40:0/1327080714 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a70005310 con 0x7f3a9006fb20 2024-09-13T18:18:31.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:31.291+0000 7f3a85ffb700 1 -- 172.21.15.40:0/1327080714 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3a88021080 con 0x7f3a9006fb20 2024-09-13T18:18:32.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:32.243+0000 7f3a977ea700 1 -- 172.21.15.40:0/1327080714 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f3a70005b40 con 0x7f3a9006fb20 2024-09-13T18:18:32.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:32.243+0000 7f3a85ffb700 1 -- 172.21.15.40:0/1327080714 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f3a88017070 con 0x7f3a9006fb20 2024-09-13T18:18:32.253 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":220,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":24,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:18:32.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:32.259+0000 7f3a977ea700 1 -- 172.21.15.40:0/1327080714 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3a74066730 msgr2=0x7f3a74068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:32.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:32.259+0000 7f3a977ea700 1 --2- 172.21.15.40:0/1327080714 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3a74066730 0x7f3a74068bf0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f3a80009500 tx=0x7f3a80009490 comp rx=0 tx=0).stop 2024-09-13T18:18:32.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:32.259+0000 7f3a977ea700 1 -- 172.21.15.40:0/1327080714 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9006fb20 msgr2=0x7f3a90147810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:32.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:32.259+0000 7f3a977ea700 1 --2- 172.21.15.40:0/1327080714 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9006fb20 0x7f3a90147810 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3a88009e50 tx=0x7f3a88004af0 comp rx=0 tx=0).stop 2024-09-13T18:18:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:32.271+0000 7f3a977ea700 1 -- 172.21.15.40:0/1327080714 shutdown_connections 2024-09-13T18:18:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:32.271+0000 7f3a977ea700 1 --2- 172.21.15.40:0/1327080714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3a901507f0 0x7f3a901487a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:32.271+0000 7f3a977ea700 1 --2- 172.21.15.40:0/1327080714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a900bfe80 0x7f3a900c9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:32.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:32.271+0000 7f3a977ea700 1 --2- 172.21.15.40:0/1327080714 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3a74066730 0x7f3a74068bf0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:32.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:32.271+0000 7f3a977ea700 1 --2- 172.21.15.40:0/1327080714 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3a9006fb20 0x7f3a90147810 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:32.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:32.271+0000 7f3a977ea700 1 -- 172.21.15.40:0/1327080714 >> 172.21.15.40:0/1327080714 conn(0x7f3a900cadd0 msgr2=0x7f3a90145d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:32.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:32.271+0000 7f3a977ea700 1 -- 172.21.15.40:0/1327080714 shutdown_connections 2024-09-13T18:18:32.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:32.271+0000 7f3a977ea700 1 -- 172.21.15.40:0/1327080714 wait complete. 2024-09-13T18:18:35.307 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:18:35.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.679+0000 7f26e67dc700 1 Processor -- start 2024-09-13T18:18:35.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.679+0000 7f26e67dc700 1 -- start start 2024-09-13T18:18:35.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.679+0000 7f26e67dc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f26e0154240 0x7f26e015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:35.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.679+0000 7f26e67dc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f26e015f0a0 0x7f26e006aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:35.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.679+0000 7f26e67dc700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e015f510 con 0x7f26e015ec60 2024-09-13T18:18:35.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.679+0000 7f26e67dc700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e00721e0 con 0x7f26e015f0a0 2024-09-13T18:18:35.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.679+0000 7f26e67dc700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e015e6e0 con 0x7f26e0154240 2024-09-13T18:18:35.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.679+0000 7f26e4fd9700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f26e015ec60 0x7f26e006a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:53224/0 (socket says 172.21.15.40:53224) 2024-09-13T18:18:35.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.679+0000 7f26e4fd9700 1 -- 172.21.15.40:0/596871636 learned_addr learned my addr 172.21.15.40:0/596871636 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:18:35.690 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.679+0000 7f26e57da700 1 --2- 172.21.15.40:0/596871636 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f26e0154240 0x7f26e015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:35.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.679+0000 7f26e5fdb700 1 --2- 172.21.15.40:0/596871636 >> v2:172.21.15.43:3300/0 conn(0x7f26e015f0a0 0x7f26e006aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:35.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.683+0000 7f26e57da700 1 -- 172.21.15.40:0/596871636 >> v2:172.21.15.43:3300/0 conn(0x7f26e015f0a0 msgr2=0x7f26e006aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:18:35.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.683+0000 7f26e57da700 1 --2- 172.21.15.40:0/596871636 >> v2:172.21.15.43:3300/0 conn(0x7f26e015f0a0 0x7f26e006aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:35.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.683+0000 7f26e57da700 1 -- 172.21.15.40:0/596871636 >> v1:172.21.15.43:6789/0 conn(0x7f26e015ec60 legacy=0x7f26e006a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:35.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.683+0000 7f26e57da700 1 -- 172.21.15.40:0/596871636 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e0071690 con 0x7f26e0154240 2024-09-13T18:18:35.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.683+0000 7f26e57da700 1 --2- 172.21.15.40:0/596871636 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f26e0154240 0x7f26e015e1a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009850 tx=0x7f26dc009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1529704e529a777 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:35.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.683+0000 7f26d7fff700 1 -- 172.21.15.40:0/596871636 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3046582591 0 0) 0x7f26d8001490 con 0x7f26e015ec60 2024-09-13T18:18:35.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.683+0000 7f26d7fff700 1 -- 172.21.15.40:0/596871636 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc011830 con 0x7f26e0154240 2024-09-13T18:18:35.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.683+0000 7f26d7fff700 1 -- 172.21.15.40:0/596871636 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26dc011990 con 0x7f26e0154240 2024-09-13T18:18:35.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.683+0000 7f26e67dc700 1 -- 172.21.15.40:0/596871636 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f26e0154240 msgr2=0x7f26e015e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:35.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.683+0000 7f26e67dc700 1 --2- 172.21.15.40:0/596871636 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f26e0154240 0x7f26e015e1a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009850 tx=0x7f26dc009b60 comp rx=0 tx=0).stop 2024-09-13T18:18:35.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.683+0000 7f26e67dc700 1 -- 172.21.15.40:0/596871636 shutdown_connections 2024-09-13T18:18:35.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.683+0000 7f26e67dc700 1 --2- 172.21.15.40:0/596871636 >> v2:172.21.15.43:3300/0 conn(0x7f26e015f0a0 0x7f26e006aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:35.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.683+0000 7f26e67dc700 1 --2- 172.21.15.40:0/596871636 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f26e0154240 0x7f26e015e1a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:35.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.683+0000 7f26e67dc700 1 -- 172.21.15.40:0/596871636 >> 172.21.15.40:0/596871636 conn(0x7f26e00cabc0 msgr2=0x7f26e00cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:35.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.683+0000 7f26e67dc700 1 -- 172.21.15.40:0/596871636 shutdown_connections 2024-09-13T18:18:35.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.683+0000 7f26e67dc700 1 -- 172.21.15.40:0/596871636 wait complete. 2024-09-13T18:18:35.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.683+0000 7f26e67dc700 1 Processor -- start 2024-09-13T18:18:35.693 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e67dc700 1 -- start start 2024-09-13T18:18:35.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e67dc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f26e0154240 0x7f26e014e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:35.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e67dc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e015ec60 0x7f26e0150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:35.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e57da700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f26e0154240 0x7f26e014e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:35.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e57da700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f26e0154240 0x7f26e014e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:52516/0 (socket says 172.21.15.40:52516) 2024-09-13T18:18:35.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e67dc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e015f0a0 0x7f26e01f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:35.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e57da700 1 -- 172.21.15.40:0/682139486 learned_addr learned my addr 172.21.15.40:0/682139486 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:35.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e4fd9700 1 --2- 172.21.15.40:0/682139486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e015ec60 0x7f26e0150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:35.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e67dc700 1 -- 172.21.15.40:0/682139486 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e00c6c40 con 0x7f26e015ec60 2024-09-13T18:18:35.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e67dc700 1 -- 172.21.15.40:0/682139486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e00c85d0 con 0x7f26e015f0a0 2024-09-13T18:18:35.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e67dc700 1 -- 172.21.15.40:0/682139486 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e014fd10 con 0x7f26e0154240 2024-09-13T18:18:35.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e5fdb700 1 --2- 172.21.15.40:0/682139486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e015f0a0 0x7f26e01f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:35.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e4fd9700 1 -- 172.21.15.40:0/682139486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e015ec60 msgr2=0x7f26e0150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:18:35.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e4fd9700 1 -- 172.21.15.40:0/682139486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e015ec60 msgr2=0x7f26e0150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:18:35.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e4fd9700 1 --2- 172.21.15.40:0/682139486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e015ec60 0x7f26e0150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:18:35.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e4fd9700 1 --2- 172.21.15.40:0/682139486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e015ec60 0x7f26e0150240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:18:35.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e57da700 1 -- 172.21.15.40:0/682139486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e015f0a0 msgr2=0x7f26e01f73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:35.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e57da700 1 --2- 172.21.15.40:0/682139486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e015f0a0 0x7f26e01f73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:35.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e57da700 1 -- 172.21.15.40:0/682139486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e015ec60 msgr2=0x7f26e0150240 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:18:35.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e57da700 1 --2- 172.21.15.40:0/682139486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e015ec60 0x7f26e0150240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:35.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e57da700 1 -- 172.21.15.40:0/682139486 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26dc0094b0 con 0x7f26e0154240 2024-09-13T18:18:35.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.687+0000 7f26e57da700 1 --2- 172.21.15.40:0/682139486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f26e0154240 0x7f26e014e7a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f26dc0090b0 tx=0x7f26dc004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:35.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.691+0000 7f26d5ffb700 1 -- 172.21.15.40:0/682139486 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc01abc0 con 0x7f26e0154240 2024-09-13T18:18:35.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.691+0000 7f26d5ffb700 1 -- 172.21.15.40:0/682139486 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26dc01ad40 con 0x7f26e0154240 2024-09-13T18:18:35.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.691+0000 7f26d5ffb700 1 -- 172.21.15.40:0/682139486 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc024660 con 0x7f26e0154240 2024-09-13T18:18:35.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.691+0000 7f26e67dc700 1 -- 172.21.15.40:0/682139486 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e014e320 con 0x7f26e0154240 2024-09-13T18:18:35.701 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.691+0000 7f26e67dc700 1 -- 172.21.15.40:0/682139486 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e01f6bc0 con 0x7f26e0154240 2024-09-13T18:18:35.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.695+0000 7f26d5ffb700 1 -- 172.21.15.40:0/682139486 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f26dc022070 con 0x7f26e0154240 2024-09-13T18:18:35.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.699+0000 7f26e67dc700 1 -- 172.21.15.40:0/682139486 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e0071040 con 0x7f26e0154240 2024-09-13T18:18:35.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.703+0000 7f26d5ffb700 1 --2- 172.21.15.40:0/682139486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f26c4066800 0x7f26c4068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:35.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.703+0000 7f26e4fd9700 1 --2- 172.21.15.40:0/682139486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f26c4066800 0x7f26c4068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:35.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.711+0000 7f26d5ffb700 1 -- 172.21.15.40:0/682139486 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f26dc0855c0 con 0x7f26e0154240 2024-09-13T18:18:35.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.711+0000 7f26e4fd9700 1 --2- 172.21.15.40:0/682139486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f26c4066800 0x7f26c4068cc0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f26e01ede60 tx=0x7f26d8001960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:35.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:35.711+0000 7f26d5ffb700 1 -- 172.21.15.40:0/682139486 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f26dc052670 con 0x7f26e0154240 2024-09-13T18:18:36.389 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:36.383+0000 7f26e67dc700 1 -- 172.21.15.40:0/682139486 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f26e00c4da0 con 0x7f26e0154240 2024-09-13T18:18:36.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:36.391+0000 7f26d5ffb700 1 -- 172.21.15.40:0/682139486 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f26dc015080 con 0x7f26e0154240 2024-09-13T18:18:36.399 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":220,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":28,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:18:36.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:36.395+0000 7f26c37fe700 1 -- 172.21.15.40:0/682139486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f26c4066800 msgr2=0x7f26c4068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:36.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:36.395+0000 7f26c37fe700 1 --2- 172.21.15.40:0/682139486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f26c4066800 0x7f26c4068cc0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f26e01ede60 tx=0x7f26d8001960 comp rx=0 tx=0).stop 2024-09-13T18:18:36.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:36.395+0000 7f26c37fe700 1 -- 172.21.15.40:0/682139486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f26e0154240 msgr2=0x7f26e014e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:36.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:36.395+0000 7f26c37fe700 1 --2- 172.21.15.40:0/682139486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f26e0154240 0x7f26e014e7a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f26dc0090b0 tx=0x7f26dc004770 comp rx=0 tx=0).stop 2024-09-13T18:18:36.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:36.395+0000 7f26c37fe700 1 -- 172.21.15.40:0/682139486 shutdown_connections 2024-09-13T18:18:36.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:36.395+0000 7f26c37fe700 1 --2- 172.21.15.40:0/682139486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26e015f0a0 0x7f26e01f73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:36.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:36.395+0000 7f26c37fe700 1 --2- 172.21.15.40:0/682139486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e015ec60 0x7f26e0150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:36.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:36.395+0000 7f26c37fe700 1 --2- 172.21.15.40:0/682139486 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f26c4066800 0x7f26c4068cc0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:36.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:36.395+0000 7f26c37fe700 1 --2- 172.21.15.40:0/682139486 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f26e0154240 0x7f26e014e7a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:36.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:36.395+0000 7f26c37fe700 1 -- 172.21.15.40:0/682139486 >> 172.21.15.40:0/682139486 conn(0x7f26e00cabc0 msgr2=0x7f26e015da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:36.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:36.395+0000 7f26c37fe700 1 -- 172.21.15.40:0/682139486 shutdown_connections 2024-09-13T18:18:36.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:36.395+0000 7f26c37fe700 1 -- 172.21.15.40:0/682139486 wait complete. 2024-09-13T18:18:39.447 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:18:39.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.947+0000 7f1f2759e700 1 Processor -- start 2024-09-13T18:18:39.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.947+0000 7f1f2759e700 1 -- start start 2024-09-13T18:18:39.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.951+0000 7f1f2759e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f28154240 0x7f1f2815e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:39.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.951+0000 7f1f2759e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f28071690 con 0x7f1f2815ec60 2024-09-13T18:18:39.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.951+0000 7f1f2759e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f2815f4e0 con 0x7f1f2815f0d0 2024-09-13T18:18:39.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.951+0000 7f1f2759e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f280721e0 con 0x7f1f28154240 2024-09-13T18:18:39.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.951+0000 7f1f26d9d700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1f2815f0d0 0x7f1f2806aaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:53230/0 (socket says 172.21.15.40:53230) 2024-09-13T18:18:39.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.951+0000 7f1f26d9d700 1 -- 172.21.15.40:0/2481896897 learned_addr learned my addr 172.21.15.40:0/2481896897 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:18:39.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.955+0000 7f1f2659c700 1 --2- 172.21.15.40:0/2481896897 >> v2:172.21.15.43:3300/0 conn(0x7f1f28154240 0x7f1f2815e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:39.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.955+0000 7f1f2659c700 1 -- 172.21.15.40:0/2481896897 >> v1:172.21.15.43:6789/0 conn(0x7f1f2815f0d0 legacy=0x7f1f2806aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:39.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.955+0000 7f1f2659c700 1 -- 172.21.15.40:0/2481896897 >> v1:172.21.15.40:6789/0 conn(0x7f1f2815ec60 legacy=0x7f1f2806a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:39.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.955+0000 7f1f2659c700 1 -- 172.21.15.40:0/2481896897 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f28154620 con 0x7f1f28154240 2024-09-13T18:18:39.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.955+0000 7f1f2659c700 1 --2- 172.21.15.40:0/2481896897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f28154240 0x7f1f2815e1a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0097d0 tx=0x7f1f1c009ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55eb5e30473d6a40 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:39.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.967+0000 7f1f2559a700 1 -- 172.21.15.40:0/2481896897 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c004230 con 0x7f1f28154240 2024-09-13T18:18:39.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.967+0000 7f1f2559a700 1 -- 172.21.15.40:0/2481896897 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1c004390 con 0x7f1f28154240 2024-09-13T18:18:39.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.967+0000 7f1f2559a700 1 -- 172.21.15.40:0/2481896897 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c0036c0 con 0x7f1f28154240 2024-09-13T18:18:39.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.971+0000 7f1f2759e700 1 -- 172.21.15.40:0/2481896897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f28154240 msgr2=0x7f1f2815e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:39.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.971+0000 7f1f2759e700 1 --2- 172.21.15.40:0/2481896897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f28154240 0x7f1f2815e1a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0097d0 tx=0x7f1f1c009ae0 comp rx=0 tx=0).stop 2024-09-13T18:18:39.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.975+0000 7f1f2759e700 1 -- 172.21.15.40:0/2481896897 shutdown_connections 2024-09-13T18:18:39.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.975+0000 7f1f2759e700 1 --2- 172.21.15.40:0/2481896897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f28154240 0x7f1f2815e1a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:39.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.975+0000 7f1f2759e700 1 -- 172.21.15.40:0/2481896897 >> 172.21.15.40:0/2481896897 conn(0x7f1f280cabc0 msgr2=0x7f1f280cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:39.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.975+0000 7f1f2759e700 1 -- 172.21.15.40:0/2481896897 shutdown_connections 2024-09-13T18:18:39.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.975+0000 7f1f2759e700 1 -- 172.21.15.40:0/2481896897 wait complete. 2024-09-13T18:18:39.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.975+0000 7f1f2759e700 1 Processor -- start 2024-09-13T18:18:39.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.975+0000 7f1f2759e700 1 -- start start 2024-09-13T18:18:39.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f2759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f28154240 0x7f1f2815a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:39.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f2759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2815ec60 0x7f1f28159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:39.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f2759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f2815f0d0 0x7f1f280c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:39.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f2759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f28141790 con 0x7f1f2815ec60 2024-09-13T18:18:39.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f2759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f280c8310 con 0x7f1f28154240 2024-09-13T18:18:39.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f2759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2815bf40 con 0x7f1f2815f0d0 2024-09-13T18:18:39.990 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f2659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f28154240 0x7f1f2815a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:39.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f25d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2815ec60 0x7f1f28159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:39.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f25d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2815ec60 0x7f1f28159840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47866/0 (socket says 172.21.15.40:47866) 2024-09-13T18:18:39.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f25d9b700 1 -- 172.21.15.40:0/2578869427 learned_addr learned my addr 172.21.15.40:0/2578869427 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:39.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f26d9d700 1 --2- 172.21.15.40:0/2578869427 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f2815f0d0 0x7f1f280c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:39.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f25d9b700 1 -- 172.21.15.40:0/2578869427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2815ec60 msgr2=0x7f1f28159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:18:39.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f25d9b700 1 -- 172.21.15.40:0/2578869427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2815ec60 msgr2=0x7f1f28159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:18:39.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f25d9b700 1 --2- 172.21.15.40:0/2578869427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2815ec60 0x7f1f28159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:18:39.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f25d9b700 1 --2- 172.21.15.40:0/2578869427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2815ec60 0x7f1f28159840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:18:39.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f26d9d700 1 -- 172.21.15.40:0/2578869427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f28154240 msgr2=0x7f1f2815a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:39.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f26d9d700 1 --2- 172.21.15.40:0/2578869427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f28154240 0x7f1f2815a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:39.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f26d9d700 1 -- 172.21.15.40:0/2578869427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2815ec60 msgr2=0x7f1f28159840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:18:39.991 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f26d9d700 1 --2- 172.21.15.40:0/2578869427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2815ec60 0x7f1f28159840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:39.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f26d9d700 1 -- 172.21.15.40:0/2578869427 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1c0094b0 con 0x7f1f2815f0d0 2024-09-13T18:18:39.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.979+0000 7f1f26d9d700 1 --2- 172.21.15.40:0/2578869427 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f2815f0d0 0x7f1f280c7480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800c3c0 tx=0x7f1f1800c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:39.992 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.983+0000 7f1f16ffd700 1 -- 172.21.15.40:0/2578869427 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18004ca0 con 0x7f1f2815f0d0 2024-09-13T18:18:39.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.983+0000 7f1f2759e700 1 -- 172.21.15.40:0/2578869427 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2815c220 con 0x7f1f2815f0d0 2024-09-13T18:18:39.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.983+0000 7f1f16ffd700 1 -- 172.21.15.40:0/2578869427 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f18004e00 con 0x7f1f2815f0d0 2024-09-13T18:18:39.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.983+0000 7f1f2759e700 1 -- 172.21.15.40:0/2578869427 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f28141c80 con 0x7f1f2815f0d0 2024-09-13T18:18:39.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.987+0000 7f1f16ffd700 1 -- 172.21.15.40:0/2578869427 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18007310 con 0x7f1f2815f0d0 2024-09-13T18:18:39.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.991+0000 7f1f2759e700 1 -- 172.21.15.40:0/2578869427 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f08005310 con 0x7f1f2815f0d0 2024-09-13T18:18:39.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.991+0000 7f1f16ffd700 1 -- 172.21.15.40:0/2578869427 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1f180074f0 con 0x7f1f2815f0d0 2024-09-13T18:18:39.998 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.991+0000 7f1f16ffd700 1 --2- 172.21.15.40:0/2578869427 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f10066a40 0x7f1f10068f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:39.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.991+0000 7f1f2659c700 1 --2- 172.21.15.40:0/2578869427 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f10066a40 0x7f1f10068f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:39.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.991+0000 7f1f2659c700 1 --2- 172.21.15.40:0/2578869427 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f10066a40 0x7f1f10068f00 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0097d0 tx=0x7f1f1c010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:39.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:39.991+0000 7f1f16ffd700 1 -- 172.21.15.40:0/2578869427 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1f18084100 con 0x7f1f2815f0d0 2024-09-13T18:18:40.009 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:40.003+0000 7f1f16ffd700 1 -- 172.21.15.40:0/2578869427 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1f18051120 con 0x7f1f2815f0d0 2024-09-13T18:18:41.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:40.995+0000 7f1f2759e700 1 -- 172.21.15.40:0/2578869427 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1f08005b40 con 0x7f1f2815f0d0 2024-09-13T18:18:41.004 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:40.995+0000 7f1f16ffd700 1 -- 172.21.15.40:0/2578869427 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f1f18054740 con 0x7f1f2815f0d0 2024-09-13T18:18:41.010 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":220,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":33,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:18:41.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:41.003+0000 7f1f14ff9700 1 -- 172.21.15.40:0/2578869427 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f10066a40 msgr2=0x7f1f10068f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:41.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:41.003+0000 7f1f14ff9700 1 --2- 172.21.15.40:0/2578869427 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f10066a40 0x7f1f10068f00 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0097d0 tx=0x7f1f1c010000 comp rx=0 tx=0).stop 2024-09-13T18:18:41.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:41.003+0000 7f1f14ff9700 1 -- 172.21.15.40:0/2578869427 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f2815f0d0 msgr2=0x7f1f280c7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:41.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:41.003+0000 7f1f14ff9700 1 --2- 172.21.15.40:0/2578869427 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f2815f0d0 0x7f1f280c7480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800c3c0 tx=0x7f1f1800c6d0 comp rx=0 tx=0).stop 2024-09-13T18:18:41.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:41.003+0000 7f1f14ff9700 1 -- 172.21.15.40:0/2578869427 shutdown_connections 2024-09-13T18:18:41.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:41.003+0000 7f1f14ff9700 1 --2- 172.21.15.40:0/2578869427 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1f10066a40 0x7f1f10068f00 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:41.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:41.003+0000 7f1f14ff9700 1 --2- 172.21.15.40:0/2578869427 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f2815f0d0 0x7f1f280c7480 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:41.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:41.003+0000 7f1f14ff9700 1 --2- 172.21.15.40:0/2578869427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2815ec60 0x7f1f28159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:41.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:41.003+0000 7f1f14ff9700 1 --2- 172.21.15.40:0/2578869427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f28154240 0x7f1f2815a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:41.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:41.003+0000 7f1f14ff9700 1 -- 172.21.15.40:0/2578869427 >> 172.21.15.40:0/2578869427 conn(0x7f1f280cabc0 msgr2=0x7f1f280c4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:41.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:41.003+0000 7f1f14ff9700 1 -- 172.21.15.40:0/2578869427 shutdown_connections 2024-09-13T18:18:41.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:41.007+0000 7f1f14ff9700 1 -- 172.21.15.40:0/2578869427 wait complete. 2024-09-13T18:18:41.262 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:18:41.265 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:18:41.706 INFO:tasks.ceph.osd.2.smithi040.stderr:2024-09-13T18:18:41.699+0000 7f5e74262700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 62551) UID: 0 2024-09-13T18:18:41.706 INFO:tasks.ceph.osd.0.smithi040.stderr:2024-09-13T18:18:41.699+0000 7f2510fcc700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 62551) UID: 0 2024-09-13T18:18:41.717 INFO:tasks.ceph.osd.3.smithi040.stderr:2024-09-13T18:18:41.699+0000 7fad5293d700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 62551) UID: 0 2024-09-13T18:18:41.726 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-13T18:18:41.703+0000 7f4b22046700 -1 Fail to open '/proc/62551/cmdline' error = (2) No such file or directory 2024-09-13T18:18:41.726 INFO:tasks.ceph.osd.1.smithi040.stderr:2024-09-13T18:18:41.703+0000 7f4b22046700 -1 received signal: Hangup from (PID: 62551) UID: 0 2024-09-13T18:18:44.048 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:18:44.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.455+0000 7f51cf662700 1 Processor -- start 2024-09-13T18:18:44.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.455+0000 7f51cf662700 1 -- start start 2024-09-13T18:18:44.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.455+0000 7f51cf662700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f51c8160390 0x7f51c81669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:44.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.455+0000 7f51cf662700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f51c80704e0 0x7f51c8167600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:44.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.455+0000 7f51cf662700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c8071930 con 0x7f51c8167b40 2024-09-13T18:18:44.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.455+0000 7f51cf662700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c8166f20 con 0x7f51c8160390 2024-09-13T18:18:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.455+0000 7f51cf662700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c8167fb0 con 0x7f51c80704e0 2024-09-13T18:18:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.455+0000 7f51ccbfd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f51c80704e0 0x7f51c8167600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.455+0000 7f51ccbfd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f51c80704e0 0x7f51c8167600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:48194/0 (socket says 172.21.15.40:48194) 2024-09-13T18:18:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.455+0000 7f51ccbfd700 1 -- 172.21.15.40:0/3807605571 learned_addr learned my addr 172.21.15.40:0/3807605571 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.455+0000 7f51cd3fe700 1 --2- 172.21.15.40:0/3807605571 >> v2:172.21.15.40:3300/0 conn(0x7f51c8160390 0x7f51c81669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.459+0000 7f51ccbfd700 1 -- 172.21.15.40:0/3807605571 >> v2:172.21.15.40:3300/0 conn(0x7f51c8160390 msgr2=0x7f51c81669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:18:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.459+0000 7f51ccbfd700 1 --2- 172.21.15.40:0/3807605571 >> v2:172.21.15.40:3300/0 conn(0x7f51c8160390 0x7f51c81669e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.459+0000 7f51ccbfd700 1 -- 172.21.15.40:0/3807605571 >> v1:172.21.15.40:6789/0 conn(0x7f51c8167b40 legacy=0x7f51c806a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.459+0000 7f51ccbfd700 1 -- 172.21.15.40:0/3807605571 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c8070a40 con 0x7f51c80704e0 2024-09-13T18:18:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.459+0000 7f51cd3fe700 1 --2- 172.21.15.40:0/3807605571 >> v2:172.21.15.40:3300/0 conn(0x7f51c8160390 0x7f51c81669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:18:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.459+0000 7f51ccbfd700 1 --2- 172.21.15.40:0/3807605571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51c80704e0 0x7f51c8167600 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f51c8072a80 tx=0x7f51c000b1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=709e26837b6e2846 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.459+0000 7f51bffff700 1 -- 172.21.15.40:0/3807605571 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c000bce0 con 0x7f51c80704e0 2024-09-13T18:18:44.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.459+0000 7f51bffff700 1 -- 172.21.15.40:0/3807605571 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c000be40 con 0x7f51c80704e0 2024-09-13T18:18:44.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.459+0000 7f51cf662700 1 -- 172.21.15.40:0/3807605571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51c80704e0 msgr2=0x7f51c8167600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:44.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.459+0000 7f51cf662700 1 --2- 172.21.15.40:0/3807605571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51c80704e0 0x7f51c8167600 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f51c8072a80 tx=0x7f51c000b1d0 comp rx=0 tx=0).stop 2024-09-13T18:18:44.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.479+0000 7f51cf662700 1 -- 172.21.15.40:0/3807605571 shutdown_connections 2024-09-13T18:18:44.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.479+0000 7f51cf662700 1 --2- 172.21.15.40:0/3807605571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51c80704e0 0x7f51c8167600 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:44.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.479+0000 7f51cf662700 1 --2- 172.21.15.40:0/3807605571 >> v2:172.21.15.40:3300/0 conn(0x7f51c8160390 0x7f51c81669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:44.487 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.479+0000 7f51cf662700 1 -- 172.21.15.40:0/3807605571 >> 172.21.15.40:0/3807605571 conn(0x7f51c80cabc0 msgr2=0x7f51c80cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:44.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.483+0000 7f51cf662700 1 -- 172.21.15.40:0/3807605571 shutdown_connections 2024-09-13T18:18:44.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.487+0000 7f51cf662700 1 -- 172.21.15.40:0/3807605571 wait complete. 2024-09-13T18:18:44.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.487+0000 7f51cf662700 1 Processor -- start 2024-09-13T18:18:44.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.491+0000 7f51cf662700 1 -- start start 2024-09-13T18:18:44.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.491+0000 7f51cf662700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c80704e0 0x7f51c8161f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:44.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.491+0000 7f51cf662700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51c8160390 0x7f51c81624c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:44.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.491+0000 7f51cf662700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51c8167b40 0x7f51c81597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:44.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.491+0000 7f51cf662700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c80c1c90 con 0x7f51c80704e0 2024-09-13T18:18:44.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.491+0000 7f51cf662700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c80c74f0 con 0x7f51c8160390 2024-09-13T18:18:44.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.491+0000 7f51cf662700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c8156da0 con 0x7f51c8167b40 2024-09-13T18:18:44.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.491+0000 7f51cd3fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c80704e0 0x7f51c8161f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:44.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.491+0000 7f51cd3fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c80704e0 0x7f51c8161f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53156/0 (socket says 172.21.15.40:53156) 2024-09-13T18:18:44.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.491+0000 7f51cd3fe700 1 -- 172.21.15.40:0/2148004418 learned_addr learned my addr 172.21.15.40:0/2148004418 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:44.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.491+0000 7f51cd3fe700 1 -- 172.21.15.40:0/2148004418 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51c8167b40 msgr2=0x7f51c81597e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:18:44.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.491+0000 7f51cd3fe700 1 --2- 172.21.15.40:0/2148004418 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51c8167b40 0x7f51c81597e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:44.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.491+0000 7f51cd3fe700 1 -- 172.21.15.40:0/2148004418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51c8160390 msgr2=0x7f51c81624c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:18:44.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.491+0000 7f51cd3fe700 1 --2- 172.21.15.40:0/2148004418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51c8160390 0x7f51c81624c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:44.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.491+0000 7f51cd3fe700 1 -- 172.21.15.40:0/2148004418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c400b560 con 0x7f51c80704e0 2024-09-13T18:18:44.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.491+0000 7f51cd3fe700 1 --2- 172.21.15.40:0/2148004418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c80704e0 0x7f51c8161f80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f51c400fde0 tx=0x7f51c400d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:44.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.495+0000 7f51bdffb700 1 -- 172.21.15.40:0/2148004418 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4018440 con 0x7f51c80704e0 2024-09-13T18:18:44.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.495+0000 7f51cf662700 1 -- 172.21.15.40:0/2148004418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c0008a70 con 0x7f51c80704e0 2024-09-13T18:18:44.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.495+0000 7f51cf662700 1 -- 172.21.15.40:0/2148004418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c8160890 con 0x7f51c80704e0 2024-09-13T18:18:44.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.495+0000 7f51bdffb700 1 -- 172.21.15.40:0/2148004418 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c40185a0 con 0x7f51c80704e0 2024-09-13T18:18:44.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.495+0000 7f51bdffb700 1 -- 172.21.15.40:0/2148004418 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4018950 con 0x7f51c80704e0 2024-09-13T18:18:44.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.499+0000 7f51bdffb700 1 -- 172.21.15.40:0/2148004418 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f51c4018ab0 con 0x7f51c80704e0 2024-09-13T18:18:44.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.499+0000 7f51bdffb700 1 --2- 172.21.15.40:0/2148004418 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f51b40669a0 0x7f51b4068e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:44.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.499+0000 7f51bdffb700 1 -- 172.21.15.40:0/2148004418 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f51c4084c60 con 0x7f51c80704e0 2024-09-13T18:18:44.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.499+0000 7f51ccbfd700 1 --2- 172.21.15.40:0/2148004418 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f51b40669a0 0x7f51b4068e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:44.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.503+0000 7f51cf662700 1 -- 172.21.15.40:0/2148004418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a4005310 con 0x7f51c80704e0 2024-09-13T18:18:44.520 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.511+0000 7f51bdffb700 1 -- 172.21.15.40:0/2148004418 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f51c4051d60 con 0x7f51c80704e0 2024-09-13T18:18:44.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:44.523+0000 7f51ccbfd700 1 --2- 172.21.15.40:0/2148004418 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f51b40669a0 0x7f51b4068e60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f51c8161f10 tx=0x7f51c001a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:45.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.343+0000 7f51cf662700 1 -- 172.21.15.40:0/2148004418 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f51a4005b40 con 0x7f51c80704e0 2024-09-13T18:18:45.360 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.351+0000 7f51bdffb700 1 -- 172.21.15.40:0/2148004418 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f51c401d070 con 0x7f51c80704e0 2024-09-13T18:18:45.360 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":224,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:18:45.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.371+0000 7f51cf662700 1 -- 172.21.15.40:0/2148004418 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f51b40669a0 msgr2=0x7f51b4068e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:45.379 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.371+0000 7f51cf662700 1 --2- 172.21.15.40:0/2148004418 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f51b40669a0 0x7f51b4068e60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f51c8161f10 tx=0x7f51c001a000 comp rx=0 tx=0).stop 2024-09-13T18:18:45.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.371+0000 7f51cf662700 1 -- 172.21.15.40:0/2148004418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c80704e0 msgr2=0x7f51c8161f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:45.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.371+0000 7f51cf662700 1 --2- 172.21.15.40:0/2148004418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c80704e0 0x7f51c8161f80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f51c400fde0 tx=0x7f51c400d630 comp rx=0 tx=0).stop 2024-09-13T18:18:45.380 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.371+0000 7f51cf662700 1 -- 172.21.15.40:0/2148004418 shutdown_connections 2024-09-13T18:18:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.371+0000 7f51cf662700 1 --2- 172.21.15.40:0/2148004418 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f51b40669a0 0x7f51b4068e60 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.371+0000 7f51cf662700 1 --2- 172.21.15.40:0/2148004418 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51c8167b40 0x7f51c81597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.371+0000 7f51cf662700 1 --2- 172.21.15.40:0/2148004418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51c8160390 0x7f51c81624c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.371+0000 7f51cf662700 1 --2- 172.21.15.40:0/2148004418 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51c80704e0 0x7f51c8161f80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.371+0000 7f51cf662700 1 -- 172.21.15.40:0/2148004418 >> 172.21.15.40:0/2148004418 conn(0x7f51c80cabc0 msgr2=0x7f51c80c4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.371+0000 7f51cf662700 1 -- 172.21.15.40:0/2148004418 shutdown_connections 2024-09-13T18:18:45.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.371+0000 7f51cf662700 1 -- 172.21.15.40:0/2148004418 wait complete. 2024-09-13T18:18:45.455 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:18:45.455 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:18:45.455 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:18:45.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.887+0000 7f2ac384f700 1 Processor -- start 2024-09-13T18:18:45.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.887+0000 7f2ac384f700 1 -- start start 2024-09-13T18:18:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.887+0000 7f2ac384f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2abc0c5d90 0x7f2abc166d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.887+0000 7f2ac384f700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2abc0c6170 0x7f2abc0c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.887+0000 7f2ac384f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2abc06ad40 con 0x7f2abc0c6de0 2024-09-13T18:18:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.887+0000 7f2ac384f700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2abc06fd30 con 0x7f2abc0c6170 2024-09-13T18:18:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.887+0000 7f2ac384f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc06a4a0 con 0x7f2abc0c5d90 2024-09-13T18:18:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.887+0000 7f2ac15eb700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2abc0c5d90 0x7f2abc166d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.887+0000 7f2ac15eb700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2abc0c5d90 0x7f2abc166d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:49028/0 (socket says 172.21.15.40:49028) 2024-09-13T18:18:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.887+0000 7f2ac15eb700 1 -- 172.21.15.40:0/4097564329 learned_addr learned my addr 172.21.15.40:0/4097564329 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.887+0000 7f2ac0dea700 1 --2- 172.21.15.40:0/4097564329 >> v2:172.21.15.40:3300/0 conn(0x7f2abc0c6170 0x7f2abc0c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.891+0000 7f2ac15eb700 1 -- 172.21.15.40:0/4097564329 >> v2:172.21.15.40:3300/0 conn(0x7f2abc0c6170 msgr2=0x7f2abc0c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:18:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.891+0000 7f2ac15eb700 1 --2- 172.21.15.40:0/4097564329 >> v2:172.21.15.40:3300/0 conn(0x7f2abc0c6170 0x7f2abc0c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.891+0000 7f2ac15eb700 1 -- 172.21.15.40:0/4097564329 >> v1:172.21.15.40:6789/0 conn(0x7f2abc0c6de0 legacy=0x7f2abc1633d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:18:45.900 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.891+0000 7f2ac15eb700 1 -- 172.21.15.40:0/4097564329 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2abc06a780 con 0x7f2abc0c5d90 2024-09-13T18:18:45.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.891+0000 7f2ac15eb700 1 --2- 172.21.15.40:0/4097564329 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2abc0c5d90 0x7f2abc166d00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800a9b0 tx=0x7f2ab800acc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b80351dae3db843f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:45.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.891+0000 7f2ab3fff700 1 -- 172.21.15.40:0/4097564329 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8019a60 con 0x7f2abc0c5d90 2024-09-13T18:18:45.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.891+0000 7f2ab3fff700 1 -- 172.21.15.40:0/4097564329 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab8019bc0 con 0x7f2abc0c5d90 2024-09-13T18:18:45.901 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.891+0000 7f2ac384f700 1 -- 172.21.15.40:0/4097564329 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2abc0c5d90 msgr2=0x7f2abc166d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:45.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.891+0000 7f2ac384f700 1 --2- 172.21.15.40:0/4097564329 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2abc0c5d90 0x7f2abc166d00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800a9b0 tx=0x7f2ab800acc0 comp rx=0 tx=0).stop 2024-09-13T18:18:45.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.891+0000 7f2ac384f700 1 -- 172.21.15.40:0/4097564329 shutdown_connections 2024-09-13T18:18:45.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.891+0000 7f2ac384f700 1 --2- 172.21.15.40:0/4097564329 >> v2:172.21.15.40:3300/0 conn(0x7f2abc0c6170 0x7f2abc0c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:45.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.891+0000 7f2ac384f700 1 --2- 172.21.15.40:0/4097564329 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2abc0c5d90 0x7f2abc166d00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:45.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.891+0000 7f2ac384f700 1 -- 172.21.15.40:0/4097564329 >> 172.21.15.40:0/4097564329 conn(0x7f2abc0cad00 msgr2=0x7f2abc0cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:45.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.895+0000 7f2ac384f700 1 -- 172.21.15.40:0/4097564329 shutdown_connections 2024-09-13T18:18:45.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.895+0000 7f2ac384f700 1 -- 172.21.15.40:0/4097564329 wait complete. 2024-09-13T18:18:45.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.899+0000 7f2ac384f700 1 Processor -- start 2024-09-13T18:18:45.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.899+0000 7f2ac384f700 1 -- start start 2024-09-13T18:18:45.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.899+0000 7f2ac384f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2abc0c6170 0x7f2abc0c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:45.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.899+0000 7f2ac384f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2abc0c6de0 0x7f2abc159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:45.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.899+0000 7f2ac384f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2abc160690 0x7f2abc1ff460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:45.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.899+0000 7f2ac384f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc0c03e0 con 0x7f2abc0c6170 2024-09-13T18:18:45.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.899+0000 7f2ac384f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc158680 con 0x7f2abc160690 2024-09-13T18:18:45.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.899+0000 7f2ac384f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc158960 con 0x7f2abc0c6de0 2024-09-13T18:18:45.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.899+0000 7f2ac0dea700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2abc0c6de0 0x7f2abc159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:45.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.899+0000 7f2ac0dea700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2abc0c6de0 0x7f2abc159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:49040/0 (socket says 172.21.15.40:49040) 2024-09-13T18:18:45.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.899+0000 7f2ac0dea700 1 -- 172.21.15.40:0/4260774115 learned_addr learned my addr 172.21.15.40:0/4260774115 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:45.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.903+0000 7f2ac1dec700 1 --2- 172.21.15.40:0/4260774115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2abc160690 0x7f2abc1ff460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:45.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.903+0000 7f2ac0dea700 1 -- 172.21.15.40:0/4260774115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2abc160690 msgr2=0x7f2abc1ff460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:45.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.903+0000 7f2ac0dea700 1 --2- 172.21.15.40:0/4260774115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2abc160690 0x7f2abc1ff460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:45.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.903+0000 7f2ac0dea700 1 -- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2abc0c6170 msgr2=0x7f2abc0c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:45.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.903+0000 7f2ac0dea700 1 --2- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2abc0c6170 0x7f2abc0c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:45.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.903+0000 7f2ac0dea700 1 -- 172.21.15.40:0/4260774115 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab800a610 con 0x7f2abc0c6de0 2024-09-13T18:18:45.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.903+0000 7f2ac0dea700 1 --2- 172.21.15.40:0/4260774115 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2abc0c6de0 0x7f2abc159470 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4007ec0 tx=0x7f2ab400e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:45.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.903+0000 7f2ab1ffb700 1 -- 172.21.15.40:0/4260774115 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab400eea0 con 0x7f2abc0c6de0 2024-09-13T18:18:45.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.903+0000 7f2ac384f700 1 -- 172.21.15.40:0/4260774115 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2abc158c40 con 0x7f2abc0c6de0 2024-09-13T18:18:45.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.903+0000 7f2ac384f700 1 -- 172.21.15.40:0/4260774115 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2abc1599b0 con 0x7f2abc0c6de0 2024-09-13T18:18:45.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.903+0000 7f2ab1ffb700 1 -- 172.21.15.40:0/4260774115 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab4003a40 con 0x7f2abc0c6de0 2024-09-13T18:18:45.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.903+0000 7f2ab1ffb700 1 -- 172.21.15.40:0/4260774115 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4003e10 con 0x7f2abc0c6de0 2024-09-13T18:18:45.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.907+0000 7f2ab1ffb700 1 -- 172.21.15.40:0/4260774115 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2ab4004030 con 0x7f2abc0c6de0 2024-09-13T18:18:45.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.907+0000 7f2ab1ffb700 1 --2- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2aa80669a0 0x7f2aa8068e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:45.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.907+0000 7f2ac15eb700 1 --2- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2aa80669a0 0x7f2aa8068e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:45.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.907+0000 7f2ab1ffb700 1 -- 172.21.15.40:0/4260774115 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2ab4084720 con 0x7f2abc0c6de0 2024-09-13T18:18:45.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.907+0000 7f2ac15eb700 1 --2- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2aa80669a0 0x7f2aa8068e60 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800ad00 tx=0x7f2ab800e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:45.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.927+0000 7f2ac384f700 1 --2- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a9c002a40 0x7f2a9c004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:45.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.927+0000 7f2ac1dec700 1 --2- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a9c002a40 0x7f2a9c004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:45.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.927+0000 7f2ac384f700 1 -- 172.21.15.40:0/4260774115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c007f40 con 0x7f2a9c002a40 2024-09-13T18:18:45.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.927+0000 7f2ac1dec700 1 -- 172.21.15.40:0/4260774115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2a9c008100 con 0x7f2a9c002a40 2024-09-13T18:18:45.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.927+0000 7f2ac1dec700 1 --2- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a9c002a40 0x7f2a9c004f00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0c89d0 tx=0x7f2aac00e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:45.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.931+0000 7f2ab1ffb700 1 -- 172.21.15.40:0/4260774115 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac0050d0 con 0x7f2a9c002a40 2024-09-13T18:18:45.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.931+0000 7f2ab1ffb700 1 -- 172.21.15.40:0/4260774115 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f2aac005230 con 0x7f2a9c002a40 2024-09-13T18:18:45.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.931+0000 7f2ab1ffb700 1 -- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a9c002a40 msgr2=0x7f2a9c004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:45.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.931+0000 7f2ab1ffb700 1 --2- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a9c002a40 0x7f2a9c004f00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0c89d0 tx=0x7f2aac00e700 comp rx=0 tx=0).stop 2024-09-13T18:18:45.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.931+0000 7f2ab1ffb700 1 -- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a9c002a40 msgr2=0x7f2a9c004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:18:45.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.931+0000 7f2ab1ffb700 1 --2- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a9c002a40 0x7f2a9c004f00 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0c89d0 tx=0x7f2aac00e700 comp rx=0 tx=0).stop 2024-09-13T18:18:45.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.955+0000 7f2aa77fe700 1 --2- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a9c001de0 0x7f2a9c00cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:45.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.955+0000 7f2ac1dec700 1 --2- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a9c001de0 0x7f2a9c00cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:45.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.955+0000 7f2aa77fe700 1 -- 172.21.15.40:0/4260774115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c002390 con 0x7f2a9c001de0 2024-09-13T18:18:45.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.955+0000 7f2ac1dec700 1 -- 172.21.15.40:0/4260774115 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f2a9c00a2e0 con 0x7f2a9c001de0 2024-09-13T18:18:45.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.959+0000 7f2ac1dec700 1 --2- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a9c001de0 0x7f2a9c00cbf0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0c89d0 tx=0x7f2aac000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:45.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.959+0000 7f2ab1ffb700 1 -- 172.21.15.40:0/4260774115 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac021070 con 0x7f2a9c001de0 2024-09-13T18:18:45.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.959+0000 7f2ab1ffb700 1 -- 172.21.15.40:0/4260774115 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4452 (secure 0 0 0) 0x7f2aac0036a0 con 0x7f2a9c001de0 2024-09-13T18:18:45.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.959+0000 7f2ab1ffb700 1 -- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a9c001de0 msgr2=0x7f2a9c00cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:45.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.959+0000 7f2ab1ffb700 1 --2- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a9c001de0 0x7f2a9c00cbf0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0c89d0 tx=0x7f2aac000ed0 comp rx=0 tx=0).stop 2024-09-13T18:18:45.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.959+0000 7f2ab1ffb700 1 -- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a9c001de0 msgr2=0x7f2a9c00cbf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:18:45.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.959+0000 7f2ab1ffb700 1 --2- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2a9c001de0 0x7f2a9c00cbf0 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0c89d0 tx=0x7f2aac000ed0 comp rx=0 tx=0).stop 2024-09-13T18:18:45.967 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:18:45.967 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:18:45.967 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:18:45.967 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:18:45.967 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 224, 2024-09-13T18:18:45.967 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:18:45.967 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:18:45.967 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:18:45.968 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:18:45.968 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:45.968 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:18:45.968 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:18:45.968 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:18:45.968 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:18:45.968 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:18:45.968 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:18:45.968 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:18:45.968 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:18:45.968 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:18:45.968 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:18:45.968 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:18:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:18:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:18:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:18:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:18:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:18:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:18:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:18:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:18:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:18:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:18:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:18:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:18:45.969 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:18:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:18:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:18:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:18:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:18:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:18:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:18:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:18:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:18:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:18:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:18:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:18:45.970 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:18:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:18:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:18:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:18:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:18:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:18:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:18:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:18:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:18:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:18:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:18:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:18:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:18:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:18:45.971 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:18:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:18:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:18:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:18:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:18:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:18:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:18:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:18:45.972 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:18:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:18:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:18:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:18:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:18:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:18:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:45.973 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:18:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:18:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:18:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:18:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:18:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:18:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:18:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:18:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:18:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:45.974 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:18:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:18:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:18:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:18:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:18:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:18:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:18:45.975 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:18:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:18:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:18:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:18:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:18:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:18:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:18:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:18:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:18:45.976 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:18:45.977 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:45.977 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:45.977 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:45.977 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:45.977 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:18:45.977 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:18:45.977 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:18:45.977 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:18:45.977 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:18:45.977 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:45.977 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:45.979 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:45.980 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:18:45.980 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:18:45.980 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:45.980 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:45.980 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:45.980 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:18:45.980 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:45.980 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:45.980 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:18:45.980 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:45.980 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:45.980 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:45.980 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:18:45.980 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:45.981 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:45.981 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:45.981 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:18:45.981 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:18:45.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.959+0000 7f2ac384f700 1 -- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2aa80669a0 msgr2=0x7f2aa8068e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:45.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.959+0000 7f2ac384f700 1 --2- 172.21.15.40:0/4260774115 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2aa80669a0 0x7f2aa8068e60 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800ad00 tx=0x7f2ab800e000 comp rx=0 tx=0).stop 2024-09-13T18:18:45.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.959+0000 7f2ac1dec700 1 -- 172.21.15.40:0/4260774115 reap_dead start 2024-09-13T18:18:45.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.959+0000 7f2ac384f700 1 -- 172.21.15.40:0/4260774115 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2abc0c6de0 msgr2=0x7f2abc159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:45.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.959+0000 7f2ac384f700 1 --2- 172.21.15.40:0/4260774115 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2abc0c6de0 0x7f2abc159470 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4007ec0 tx=0x7f2ab400e3c0 comp rx=0 tx=0).stop 2024-09-13T18:18:45.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.959+0000 7f2ac384f700 1 -- 172.21.15.40:0/4260774115 shutdown_connections 2024-09-13T18:18:45.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.959+0000 7f2ac384f700 1 --2- 172.21.15.40:0/4260774115 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2abc0c6de0 0x7f2abc159470 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:45.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.959+0000 7f2ac384f700 1 -- 172.21.15.40:0/4260774115 >> 172.21.15.40:0/4260774115 conn(0x7f2abc0cad00 msgr2=0x7f2abc1659a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:45.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.959+0000 7f2ac384f700 1 -- 172.21.15.40:0/4260774115 shutdown_connections 2024-09-13T18:18:45.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:45.959+0000 7f2ac384f700 1 -- 172.21.15.40:0/4260774115 wait complete. 2024-09-13T18:18:45.984 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:18:45.984 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:18:45.984 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:18:46.231 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.223+0000 7fcb7d3f7700 1 Processor -- start 2024-09-13T18:18:46.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.223+0000 7fcb7d3f7700 1 -- start start 2024-09-13T18:18:46.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.227+0000 7fcb7d3f7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb78151310 0x7fcb78157180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:46.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.227+0000 7fcb7d3f7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcb780c8ba0 0x7fcb7815b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:46.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.227+0000 7fcb7d3f7700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb780c6ed0 con 0x7fcb781516f0 2024-09-13T18:18:46.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.227+0000 7fcb7d3f7700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb7815adf0 con 0x7fcb780c8ba0 2024-09-13T18:18:46.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.227+0000 7fcb7d3f7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7815a770 con 0x7fcb78151310 2024-09-13T18:18:46.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.227+0000 7fcb767fc700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcb781516f0 0x7fcb7806f220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:37626/0 (socket says 172.21.15.40:37626) 2024-09-13T18:18:46.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.227+0000 7fcb767fc700 1 -- 172.21.15.40:0/1543854061 learned_addr learned my addr 172.21.15.40:0/1543854061 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:18:46.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.227+0000 7fcb76ffd700 1 --2- 172.21.15.40:0/1543854061 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb78151310 0x7fcb78157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:46.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.227+0000 7fcb77fff700 1 --2- 172.21.15.40:0/1543854061 >> v2:172.21.15.40:3300/0 conn(0x7fcb780c8ba0 0x7fcb7815b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:46.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.227+0000 7fcb75ffb700 1 -- 172.21.15.40:0/1543854061 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024231185 0 0) 0x7fcb780c6ed0 con 0x7fcb781516f0 2024-09-13T18:18:46.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.227+0000 7fcb75ffb700 1 -- 172.21.15.40:0/1543854061 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb600035d0 con 0x7fcb781516f0 2024-09-13T18:18:46.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.227+0000 7fcb76ffd700 1 -- 172.21.15.40:0/1543854061 >> v2:172.21.15.40:3300/0 conn(0x7fcb780c8ba0 msgr2=0x7fcb7815b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:18:46.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.227+0000 7fcb76ffd700 1 --2- 172.21.15.40:0/1543854061 >> v2:172.21.15.40:3300/0 conn(0x7fcb780c8ba0 0x7fcb7815b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:46.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.227+0000 7fcb76ffd700 1 -- 172.21.15.40:0/1543854061 >> v1:172.21.15.43:6789/0 conn(0x7fcb781516f0 legacy=0x7fcb7806f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:46.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.227+0000 7fcb76ffd700 1 -- 172.21.15.40:0/1543854061 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb780c7120 con 0x7fcb78151310 2024-09-13T18:18:46.236 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.227+0000 7fcb77fff700 1 --2- 172.21.15.40:0/1543854061 >> v2:172.21.15.40:3300/0 conn(0x7fcb780c8ba0 0x7fcb7815b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:18:46.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.227+0000 7fcb76ffd700 1 --2- 172.21.15.40:0/1543854061 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb78151310 0x7fcb78157180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009850 tx=0x7fcb6c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df3a88237a98fa3a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:46.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.227+0000 7fcb75ffb700 1 -- 172.21.15.40:0/1543854061 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c011830 con 0x7fcb78151310 2024-09-13T18:18:46.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.227+0000 7fcb75ffb700 1 -- 172.21.15.40:0/1543854061 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6c011990 con 0x7fcb78151310 2024-09-13T18:18:46.237 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.227+0000 7fcb75ffb700 1 -- 172.21.15.40:0/1543854061 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c011d40 con 0x7fcb78151310 2024-09-13T18:18:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.231+0000 7fcb7d3f7700 1 -- 172.21.15.40:0/1543854061 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb78151310 msgr2=0x7fcb78157180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.231+0000 7fcb7d3f7700 1 --2- 172.21.15.40:0/1543854061 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb78151310 0x7fcb78157180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009850 tx=0x7fcb6c009b60 comp rx=0 tx=0).stop 2024-09-13T18:18:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.231+0000 7fcb7d3f7700 1 -- 172.21.15.40:0/1543854061 shutdown_connections 2024-09-13T18:18:46.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.231+0000 7fcb7d3f7700 1 --2- 172.21.15.40:0/1543854061 >> v2:172.21.15.40:3300/0 conn(0x7fcb780c8ba0 0x7fcb7815b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:46.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.231+0000 7fcb7d3f7700 1 --2- 172.21.15.40:0/1543854061 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb78151310 0x7fcb78157180 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:46.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.231+0000 7fcb7d3f7700 1 -- 172.21.15.40:0/1543854061 >> 172.21.15.40:0/1543854061 conn(0x7fcb780cc4d0 msgr2=0x7fcb780cc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:46.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.231+0000 7fcb7d3f7700 1 -- 172.21.15.40:0/1543854061 shutdown_connections 2024-09-13T18:18:46.240 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.231+0000 7fcb7d3f7700 1 -- 172.21.15.40:0/1543854061 wait complete. 2024-09-13T18:18:46.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.243+0000 7fcb7d3f7700 1 Processor -- start 2024-09-13T18:18:46.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.243+0000 7fcb7d3f7700 1 -- start start 2024-09-13T18:18:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.247+0000 7fcb7d3f7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb780c8ba0 0x7fcb7806a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.247+0000 7fcb7d3f7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb78151310 0x7fcb78151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.247+0000 7fcb76ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb780c8ba0 0x7fcb7806a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.247+0000 7fcb76ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb780c8ba0 0x7fcb7806a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:49070/0 (socket says 172.21.15.40:49070) 2024-09-13T18:18:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.247+0000 7fcb767fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb78151310 0x7fcb78151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.247+0000 7fcb767fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb78151310 0x7fcb78151d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53206/0 (socket says 172.21.15.40:53206) 2024-09-13T18:18:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.247+0000 7fcb76ffd700 1 -- 172.21.15.40:0/3067487470 learned_addr learned my addr 172.21.15.40:0/3067487470 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.247+0000 7fcb7d3f7700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb781516f0 0x7fcb781eb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.247+0000 7fcb7d3f7700 1 -- 172.21.15.40:0/3067487470 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb781559f0 con 0x7fcb78151310 2024-09-13T18:18:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.247+0000 7fcb7d3f7700 1 -- 172.21.15.40:0/3067487470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb781562f0 con 0x7fcb781516f0 2024-09-13T18:18:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.247+0000 7fcb77fff700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb781516f0 0x7fcb781eb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:46.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.247+0000 7fcb7d3f7700 1 -- 172.21.15.40:0/3067487470 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7806a8d0 con 0x7fcb780c8ba0 2024-09-13T18:18:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.247+0000 7fcb767fc700 1 -- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb780c8ba0 msgr2=0x7fcb7806a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.247+0000 7fcb767fc700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcb780c8ba0 0x7fcb7806a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.247+0000 7fcb767fc700 1 -- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb781516f0 msgr2=0x7fcb781eb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.247+0000 7fcb767fc700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb781516f0 0x7fcb781eb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.247+0000 7fcb767fc700 1 -- 172.21.15.40:0/3067487470 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6c0094b0 con 0x7fcb78151310 2024-09-13T18:18:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.247+0000 7fcb767fc700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb78151310 0x7fcb78151d40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800c3e0 tx=0x7fcb6800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.247+0000 7fcb77fff700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb781516f0 0x7fcb781eb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:18:46.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.255+0000 7fcb5f7fe700 1 -- 172.21.15.40:0/3067487470 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68005d90 con 0x7fcb78151310 2024-09-13T18:18:46.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.255+0000 7fcb5f7fe700 1 -- 172.21.15.40:0/3067487470 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb680070d0 con 0x7fcb78151310 2024-09-13T18:18:46.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.255+0000 7fcb7d3f7700 1 -- 172.21.15.40:0/3067487470 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb7806ab40 con 0x7fcb78151310 2024-09-13T18:18:46.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.255+0000 7fcb7d3f7700 1 -- 172.21.15.40:0/3067487470 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb7806b100 con 0x7fcb78151310 2024-09-13T18:18:46.264 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.255+0000 7fcb5f7fe700 1 -- 172.21.15.40:0/3067487470 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68007460 con 0x7fcb78151310 2024-09-13T18:18:46.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.259+0000 7fcb7d3f7700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb781565b0 0x7fcb7806b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:46.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.259+0000 7fcb7d3f7700 1 -- 172.21.15.40:0/3067487470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb78067840 con 0x7fcb781565b0 2024-09-13T18:18:46.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.259+0000 7fcb5f7fe700 1 -- 172.21.15.40:0/3067487470 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcb680075c0 con 0x7fcb78151310 2024-09-13T18:18:46.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.259+0000 7fcb76ffd700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb781565b0 0x7fcb7806b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:46.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.259+0000 7fcb76ffd700 1 -- 172.21.15.40:0/3067487470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb7815aa10 con 0x7fcb781565b0 2024-09-13T18:18:46.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.259+0000 7fcb76ffd700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb781565b0 0x7fcb7806b4a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009ba0 tx=0x7fcb6c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:46.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.259+0000 7fcb5f7fe700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcb64066800 0x7fcb64068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:46.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.259+0000 7fcb77fff700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcb64066800 0x7fcb64068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:46.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.263+0000 7fcb77fff700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcb64066800 0x7fcb64068cc0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fcb7001a550 tx=0x7fcb7001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:46.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.263+0000 7fcb5f7fe700 1 -- 172.21.15.40:0/3067487470 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcb68083910 con 0x7fcb78151310 2024-09-13T18:18:46.270 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.263+0000 7fcb5f7fe700 1 -- 172.21.15.40:0/3067487470 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c004550 con 0x7fcb781565b0 2024-09-13T18:18:46.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.267+0000 7fcb5f7fe700 1 -- 172.21.15.40:0/3067487470 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fcb6c0036c0 con 0x7fcb781565b0 2024-09-13T18:18:46.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.267+0000 7fcb5f7fe700 1 -- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb781565b0 msgr2=0x7fcb7806b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:46.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.267+0000 7fcb5f7fe700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb781565b0 0x7fcb7806b4a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009ba0 tx=0x7fcb6c004770 comp rx=0 tx=0).stop 2024-09-13T18:18:46.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.267+0000 7fcb5f7fe700 1 -- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb781565b0 msgr2=0x7fcb7806b4a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:18:46.277 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.267+0000 7fcb5f7fe700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb781565b0 0x7fcb7806b4a0 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009ba0 tx=0x7fcb6c004770 comp rx=0 tx=0).stop 2024-09-13T18:18:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.283+0000 7fcb7d3f7700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7814b9c0 0x7fcb7814bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:46.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.287+0000 7fcb76ffd700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7814b9c0 0x7fcb7814bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:46.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.287+0000 7fcb7d3f7700 1 -- 172.21.15.40:0/3067487470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb780c76e0 con 0x7fcb7814b9c0 2024-09-13T18:18:46.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.291+0000 7fcb76ffd700 1 -- 172.21.15.40:0/3067487470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fcb7806f930 con 0x7fcb7814b9c0 2024-09-13T18:18:46.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.291+0000 7fcb76ffd700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7814b9c0 0x7fcb7814bda0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c023000 tx=0x7fcb6c011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:46.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.291+0000 7fcb5f7fe700 1 -- 172.21.15.40:0/3067487470 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c01abf0 con 0x7fcb7814b9c0 2024-09-13T18:18:46.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.295+0000 7fcb5f7fe700 1 -- 172.21.15.40:0/3067487470 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7fcb6c01ad50 con 0x7fcb7814b9c0 2024-09-13T18:18:46.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.295+0000 7fcb5f7fe700 1 -- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7814b9c0 msgr2=0x7fcb7814bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:46.305 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.295+0000 7fcb5f7fe700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7814b9c0 0x7fcb7814bda0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c023000 tx=0x7fcb6c011000 comp rx=0 tx=0).stop 2024-09-13T18:18:46.305 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:18:46.305 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:18:46.306 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:18:46.306 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:18:46.306 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 224, 2024-09-13T18:18:46.306 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:18:46.306 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:18:46.306 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:18:46.306 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:18:46.306 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:46.306 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:18:46.306 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:18:46.306 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:18:46.306 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:18:46.306 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:18:46.307 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:18:46.307 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:18:46.307 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:18:46.307 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:18:46.307 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:18:46.307 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:18:46.307 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:18:46.307 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:18:46.307 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:18:46.307 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:46.307 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:18:46.307 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:18:46.307 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:18:46.308 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:18:46.308 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:18:46.308 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:18:46.308 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:18:46.308 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:18:46.308 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:18:46.308 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:18:46.308 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:18:46.308 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:18:46.308 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:46.308 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.308 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:18:46.308 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:18:46.308 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:18:46.309 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:18:46.309 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:18:46.309 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:18:46.309 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:18:46.309 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:18:46.309 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:18:46.309 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:18:46.309 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:18:46.309 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:18:46.309 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:18:46.309 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:18:46.309 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:18:46.309 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:18:46.310 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:18:46.310 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:18:46.310 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:18:46.310 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:18:46.310 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:18:46.310 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:18:46.310 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:18:46.310 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:18:46.310 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:18:46.310 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:18:46.310 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:18:46.310 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:46.310 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:18:46.310 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.311 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:18:46.311 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.311 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:18:46.311 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:18:46.311 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:18:46.311 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:18:46.311 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.311 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:18:46.311 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:18:46.311 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:46.311 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.311 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.311 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:18:46.312 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:18:46.312 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:46.312 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:46.312 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:46.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.312 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:18:46.312 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:18:46.312 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:18:46.312 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:18:46.312 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:18:46.312 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.312 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.312 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:18:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:18:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:18:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:18:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:18:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:18:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:18:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:18:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:46.313 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:18:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:18:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:18:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:18:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:18:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:18:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:18:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:18:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:18:46.314 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:18:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:18:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:18:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:18:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:18:46.315 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:18:46.316 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:18:46.316 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:18:46.316 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:18:46.316 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:46.316 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:46.316 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.316 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:18:46.316 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:18:46.316 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.316 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:46.316 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:46.316 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:18:46.316 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:46.316 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:46.317 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:18:46.317 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.317 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:46.317 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:46.317 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:18:46.317 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:46.317 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:46.317 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:18:46.317 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.317 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:46.317 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:46.317 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:18:46.317 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:46.317 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:46.318 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:18:46.318 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.318 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:46.318 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:46.318 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:18:46.318 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:46.318 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:46.318 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:18:46.318 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.318 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:46.318 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:46.318 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:18:46.318 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:46.318 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:46.319 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.319 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:18:46.319 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:18:46.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.299+0000 7fcb5f7fe700 1 -- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7814b9c0 msgr2=0x7fcb7814bda0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:18:46.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.299+0000 7fcb5f7fe700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb7814b9c0 0x7fcb7814bda0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:46.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.319+0000 7fcb5d7fa700 1 -- 172.21.15.40:0/3067487470 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcb64066800 msgr2=0x7fcb64068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:46.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.319+0000 7fcb5d7fa700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcb64066800 0x7fcb64068cc0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fcb7001a550 tx=0x7fcb7001c040 comp rx=0 tx=0).stop 2024-09-13T18:18:46.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.319+0000 7fcb77fff700 1 -- 172.21.15.40:0/3067487470 reap_dead start 2024-09-13T18:18:46.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.319+0000 7fcb5d7fa700 1 -- 172.21.15.40:0/3067487470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb78151310 msgr2=0x7fcb78151d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:46.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.319+0000 7fcb5d7fa700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb78151310 0x7fcb78151d40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800c3e0 tx=0x7fcb6800c7a0 comp rx=0 tx=0).stop 2024-09-13T18:18:46.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.319+0000 7fcb5d7fa700 1 -- 172.21.15.40:0/3067487470 shutdown_connections 2024-09-13T18:18:46.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.319+0000 7fcb5d7fa700 1 --2- 172.21.15.40:0/3067487470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcb78151310 0x7fcb78151d40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:46.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.319+0000 7fcb5d7fa700 1 -- 172.21.15.40:0/3067487470 >> 172.21.15.40:0/3067487470 conn(0x7fcb780cc4d0 msgr2=0x7fcb7814d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:46.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.319+0000 7fcb5d7fa700 1 -- 172.21.15.40:0/3067487470 shutdown_connections 2024-09-13T18:18:46.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.319+0000 7fcb5d7fa700 1 -- 172.21.15.40:0/3067487470 wait complete. 2024-09-13T18:18:46.350 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:18:46.350 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:18:46.350 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:18:46.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.675+0000 7f456f721700 1 Processor -- start 2024-09-13T18:18:46.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.675+0000 7f456f721700 1 -- start start 2024-09-13T18:18:46.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.679+0000 7f456f721700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f456808b230 0x7f4568166d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:46.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.679+0000 7f456f721700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4568163a00 0x7f45680c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:46.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.679+0000 7f456f721700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45681672b0 con 0x7f456806a390 2024-09-13T18:18:46.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.679+0000 7f456f721700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f456806fee0 con 0x7f456808b230 2024-09-13T18:18:46.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.679+0000 7f456f721700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456806afb0 con 0x7f4568163a00 2024-09-13T18:18:46.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.679+0000 7f456dcbe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f456806a390 0x7f4568169830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:37638/0 (socket says 172.21.15.40:37638) 2024-09-13T18:18:46.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.679+0000 7f456dcbe700 1 -- 172.21.15.40:0/1401442302 learned_addr learned my addr 172.21.15.40:0/1401442302 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:18:46.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.679+0000 7f455ffff700 1 -- 172.21.15.40:0/1401442302 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3712649852 0 0) 0x7f4558001110 con 0x7f456806a390 2024-09-13T18:18:46.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.679+0000 7f455ffff700 1 -- 172.21.15.40:0/1401442302 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45540035d0 con 0x7f456806a390 2024-09-13T18:18:46.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.683+0000 7f456ccbc700 1 --2- 172.21.15.40:0/1401442302 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4568163a00 0x7f45680c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:46.691 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.683+0000 7f455ffff700 1 -- 172.21.15.40:0/1401442302 <== mon.1 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 594284873 0 0) 0x7f45540035d0 con 0x7f456806a390 2024-09-13T18:18:46.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.683+0000 7f455ffff700 1 -- 172.21.15.40:0/1401442302 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4568163a00 msgr2=0x7f45680c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:18:46.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.683+0000 7f455ffff700 1 --2- 172.21.15.40:0/1401442302 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4568163a00 0x7f45680c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:46.692 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.683+0000 7f455ffff700 1 -- 172.21.15.40:0/1401442302 >> v2:172.21.15.43:3300/0 conn(0x7f456808b230 msgr2=0x7f4568166d70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:18:46.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.691+0000 7f455ffff700 1 --2- 172.21.15.40:0/1401442302 >> v2:172.21.15.43:3300/0 conn(0x7f456808b230 0x7f4568166d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:46.698 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.691+0000 7f455ffff700 1 -- 172.21.15.40:0/1401442302 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45680c72d0 con 0x7f456806a390 2024-09-13T18:18:46.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.691+0000 7f455ffff700 1 -- 172.21.15.40:0/1401442302 <== mon.1 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f4558001800 con 0x7f456806a390 2024-09-13T18:18:46.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.691+0000 7f455ffff700 1 -- 172.21.15.40:0/1401442302 >> v1:172.21.15.43:6789/0 conn(0x7f456806a390 legacy=0x7f4568169830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:46.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.691+0000 7f455ffff700 1 --2- 172.21.15.40:0/1401442302 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4568163a00 0x7f4554003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:46.699 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.691+0000 7f455ffff700 1 --2- 172.21.15.40:0/1401442302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45540040e0 0x7f4554004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:46.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.695+0000 7f455ffff700 1 --2- 172.21.15.40:0/1401442302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4554004db0 0x7f4554005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:46.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.695+0000 7f455ffff700 1 -- 172.21.15.40:0/1401442302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456806fee0 con 0x7f4554004db0 2024-09-13T18:18:46.702 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.695+0000 7f455ffff700 1 -- 172.21.15.40:0/1401442302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456806afb0 con 0x7f45540040e0 2024-09-13T18:18:46.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.695+0000 7f455ffff700 1 -- 172.21.15.40:0/1401442302 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455400deb0 con 0x7f4568163a00 2024-09-13T18:18:46.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.695+0000 7f456dcbe700 1 --2- 172.21.15.40:0/1401442302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4554004db0 0x7f4554005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:46.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.691+0000 7f456ccbc700 1 --2- 172.21.15.40:0/1401442302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45540040e0 0x7f4554004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:46.703 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.691+0000 7f456d4bd700 1 --2- 172.21.15.40:0/1401442302 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4568163a00 0x7f4554003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:46.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.695+0000 7f456dcbe700 1 -- 172.21.15.40:0/1401442302 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4568163a00 msgr2=0x7f4554003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:18:46.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.695+0000 7f456dcbe700 1 --2- 172.21.15.40:0/1401442302 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4568163a00 0x7f4554003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:46.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.695+0000 7f456dcbe700 1 -- 172.21.15.40:0/1401442302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45540040e0 msgr2=0x7f4554004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:18:46.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.695+0000 7f456dcbe700 1 --2- 172.21.15.40:0/1401442302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45540040e0 0x7f4554004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:46.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.695+0000 7f456dcbe700 1 -- 172.21.15.40:0/1401442302 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45540035d0 con 0x7f4554004db0 2024-09-13T18:18:46.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.695+0000 7f456ccbc700 1 --2- 172.21.15.40:0/1401442302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45540040e0 0x7f4554004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:18:46.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.695+0000 7f456dcbe700 1 --2- 172.21.15.40:0/1401442302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4554004db0 0x7f4554005230 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4558005030 tx=0x7f45540058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae25d67333af9610 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:46.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.695+0000 7f455ffff700 1 -- 172.21.15.40:0/1401442302 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4558005b80 con 0x7f4554004db0 2024-09-13T18:18:46.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.695+0000 7f455ffff700 1 -- 172.21.15.40:0/1401442302 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4558005ce0 con 0x7f4554004db0 2024-09-13T18:18:46.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.695+0000 7f456f721700 1 -- 172.21.15.40:0/1401442302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4554004db0 msgr2=0x7f4554005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:46.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.695+0000 7f456f721700 1 --2- 172.21.15.40:0/1401442302 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4554004db0 0x7f4554005230 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4558005030 tx=0x7f45540058a0 comp rx=0 tx=0).stop 2024-09-13T18:18:46.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.703+0000 7f456dcbe700 1 -- 172.21.15.40:0/1401442302 reap_dead start 2024-09-13T18:18:46.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.703+0000 7f456f721700 1 -- 172.21.15.40:0/1401442302 shutdown_connections 2024-09-13T18:18:46.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.703+0000 7f456f721700 1 -- 172.21.15.40:0/1401442302 >> 172.21.15.40:0/1401442302 conn(0x7f456806d5b0 msgr2=0x7f456806d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:46.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.719+0000 7f456f721700 1 -- 172.21.15.40:0/1401442302 shutdown_connections 2024-09-13T18:18:46.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.719+0000 7f456f721700 1 -- 172.21.15.40:0/1401442302 wait complete. 2024-09-13T18:18:46.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.723+0000 7f456f721700 1 Processor -- start 2024-09-13T18:18:46.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.723+0000 7f456f721700 1 -- start start 2024-09-13T18:18:46.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.727+0000 7f456f721700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4568165be0 0x7f45681eee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:46.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.727+0000 7f456f721700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4568165fc0 0x7f45681fc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:46.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.727+0000 7f456f721700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45680c2890 0x7f45680c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:46.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.727+0000 7f456f721700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45681ec370 con 0x7f45680c2890 2024-09-13T18:18:46.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.727+0000 7f456f721700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45680c3cd0 con 0x7f4568165fc0 2024-09-13T18:18:46.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.727+0000 7f456f721700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45681fce60 con 0x7f4568165be0 2024-09-13T18:18:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.727+0000 7f456ccbc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4568165fc0 0x7f45681fc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.727+0000 7f456ccbc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4568165fc0 0x7f45681fc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:48240/0 (socket says 172.21.15.40:48240) 2024-09-13T18:18:46.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.731+0000 7f456ccbc700 1 -- 172.21.15.40:0/2925801416 learned_addr learned my addr 172.21.15.40:0/2925801416 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:46.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.731+0000 7f456ccbc700 1 -- 172.21.15.40:0/2925801416 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4568165be0 msgr2=0x7f45681eee20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:18:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.731+0000 7f456dcbe700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45680c2890 0x7f45680c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.731+0000 7f456ccbc700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4568165be0 0x7f45681eee20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.731+0000 7f456ccbc700 1 -- 172.21.15.40:0/2925801416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45680c2890 msgr2=0x7f45680c2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.731+0000 7f456ccbc700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45680c2890 0x7f45680c2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.731+0000 7f456ccbc700 1 -- 172.21.15.40:0/2925801416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45680c3f00 con 0x7f4568165fc0 2024-09-13T18:18:46.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.731+0000 7f456dcbe700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45680c2890 0x7f45680c2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:18:46.740 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.731+0000 7f456ccbc700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4568165fc0 0x7f45681fc920 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f45600046b0 tx=0x7f4560007330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:46.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.735+0000 7f455dffb700 1 -- 172.21.15.40:0/2925801416 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4560007db0 con 0x7f4568165fc0 2024-09-13T18:18:46.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.735+0000 7f456f721700 1 -- 172.21.15.40:0/2925801416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45680c41e0 con 0x7f4568165fc0 2024-09-13T18:18:46.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.735+0000 7f456f721700 1 -- 172.21.15.40:0/2925801416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45680c47a0 con 0x7f4568165fc0 2024-09-13T18:18:46.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.735+0000 7f455dffb700 1 -- 172.21.15.40:0/2925801416 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456000f050 con 0x7f4568165fc0 2024-09-13T18:18:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.735+0000 7f455dffb700 1 -- 172.21.15.40:0/2925801416 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4560014610 con 0x7f4568165fc0 2024-09-13T18:18:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.735+0000 7f456f721700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4548002a40 0x7f4548004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.735+0000 7f456f721700 1 -- 172.21.15.40:0/2925801416 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4548007f40 con 0x7f4548002a40 2024-09-13T18:18:46.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.735+0000 7f456d4bd700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4548002a40 0x7f4548004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:46.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.743+0000 7f456d4bd700 1 -- 172.21.15.40:0/2925801416 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4548008100 con 0x7f4548002a40 2024-09-13T18:18:46.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.747+0000 7f456d4bd700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4548002a40 0x7f4548004f00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f456400e7a0 tx=0x7f456400eb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:46.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.747+0000 7f455dffb700 1 -- 172.21.15.40:0/2925801416 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4560014770 con 0x7f4568165fc0 2024-09-13T18:18:46.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.747+0000 7f455dffb700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f454c066780 0x7f454c068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:46.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.747+0000 7f456dcbe700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f454c066780 0x7f454c068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:46.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.747+0000 7f456dcbe700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f454c066780 0x7f454c068c40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4558005030 tx=0x7f455800a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:46.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.747+0000 7f455dffb700 1 -- 172.21.15.40:0/2925801416 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4560084c00 con 0x7f4568165fc0 2024-09-13T18:18:46.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.759+0000 7f455dffb700 1 -- 172.21.15.40:0/2925801416 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45640137e0 con 0x7f4548002a40 2024-09-13T18:18:46.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.771+0000 7f455dffb700 1 -- 172.21.15.40:0/2925801416 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f4564013940 con 0x7f4548002a40 2024-09-13T18:18:46.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.779+0000 7f455dffb700 1 -- 172.21.15.40:0/2925801416 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4548002a40 msgr2=0x7f4548004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:46.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.779+0000 7f455dffb700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4548002a40 0x7f4548004f00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f456400e7a0 tx=0x7f456400eb60 comp rx=0 tx=0).stop 2024-09-13T18:18:46.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.779+0000 7f455dffb700 1 -- 172.21.15.40:0/2925801416 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4548002a40 msgr2=0x7f4548004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:18:46.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.779+0000 7f455dffb700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4548002a40 0x7f4548004f00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:46.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.819+0000 7f456f721700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4548007020 0x7f4548007490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:46.828 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.819+0000 7f456f721700 1 -- 172.21.15.40:0/2925801416 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454800a400 con 0x7f4548007020 2024-09-13T18:18:46.829 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.819+0000 7f456d4bd700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4548007020 0x7f4548007490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:46.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.823+0000 7f456d4bd700 1 -- 172.21.15.40:0/2925801416 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f454800a5c0 con 0x7f4548007020 2024-09-13T18:18:46.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.827+0000 7f456d4bd700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4548007020 0x7f4548007490 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f456400e7a0 tx=0x7f4564004820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:46.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.831+0000 7f455dffb700 1 -- 172.21.15.40:0/2925801416 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4564023070 con 0x7f4548007020 2024-09-13T18:18:46.838 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.831+0000 7f455dffb700 1 -- 172.21.15.40:0/2925801416 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4782 (secure 0 0 0) 0x7f4564006820 con 0x7f4548007020 2024-09-13T18:18:46.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.831+0000 7f455dffb700 1 -- 172.21.15.40:0/2925801416 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4548007020 msgr2=0x7f4548007490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:46.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.831+0000 7f455dffb700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4548007020 0x7f4548007490 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f456400e7a0 tx=0x7f4564004820 comp rx=0 tx=0).stop 2024-09-13T18:18:46.839 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:18:46.839 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:18:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:18:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:18:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 224, 2024-09-13T18:18:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:18:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:18:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:18:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:18:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:18:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:18:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:18:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:18:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:18:46.840 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:18:46.841 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:18:46.841 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:18:46.841 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:18:46.841 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:18:46.841 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:18:46.841 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:18:46.841 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:18:46.841 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:18:46.841 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:46.841 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:18:46.841 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:18:46.841 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:18:46.841 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:18:46.842 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:18:46.842 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:18:46.842 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:18:46.842 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:18:46.842 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:18:46.842 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:18:46.842 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:18:46.842 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:18:46.842 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:46.842 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.842 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:18:46.842 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:18:46.842 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:18:46.842 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:18:46.842 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:18:46.843 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:18:46.843 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:18:46.843 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:18:46.843 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:18:46.843 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:18:46.843 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:18:46.843 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:18:46.843 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:18:46.843 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:18:46.843 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:18:46.843 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:18:46.843 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:18:46.843 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:18:46.843 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:18:46.844 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:18:46.844 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:18:46.844 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:18:46.844 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:18:46.844 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:18:46.844 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:18:46.844 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:18:46.844 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:18:46.844 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:46.844 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:18:46.844 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.844 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:18:46.844 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.845 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:18:46.845 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:18:46.845 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:18:46.845 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:18:46.845 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.845 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:18:46.845 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:18:46.845 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:46.845 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.845 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.845 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:18:46.845 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:18:46.845 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:46.845 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:46.846 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:46.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.846 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:18:46.846 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:18:46.846 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:18:46.846 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:18:46.846 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:18:46.846 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.846 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.846 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:18:46.846 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:18:46.846 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:18:46.846 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:18:46.847 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.847 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:18:46.847 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:18:46.847 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:46.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.847 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.847 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:18:46.847 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:18:46.847 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:46.847 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:46.847 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:46.847 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.847 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:18:46.847 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:18:46.848 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:18:46.848 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:18:46.848 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:18:46.848 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.848 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.848 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:18:46.848 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:18:46.848 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:18:46.848 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:18:46.848 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.848 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:18:46.848 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:18:46.848 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:46.848 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.849 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.849 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:18:46.849 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:18:46.849 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:46.849 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:46.849 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:46.849 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.849 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:18:46.849 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:18:46.849 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:18:46.849 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:18:46.849 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:18:46.849 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:46.849 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:46.850 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.850 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:18:46.850 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:18:46.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.850 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:46.850 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:46.850 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:18:46.850 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:46.850 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:46.850 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:18:46.850 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.850 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:46.850 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:46.850 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:18:46.851 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:46.851 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:46.851 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:18:46.851 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.851 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:46.851 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:46.851 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:18:46.851 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:46.851 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:46.851 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:18:46.851 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:46.851 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:46.851 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:46.851 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:18:46.852 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:46.852 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:46.852 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:46.852 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:18:46.852 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:18:46.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.831+0000 7f455dffb700 1 -- 172.21.15.40:0/2925801416 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4548007020 msgr2=0x7f4548007490 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:18:46.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.831+0000 7f455dffb700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4548007020 0x7f4548007490 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:46.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.839+0000 7f45537fe700 1 -- 172.21.15.40:0/2925801416 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f454c066780 msgr2=0x7f454c068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:46.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.839+0000 7f45537fe700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f454c066780 0x7f454c068c40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4558005030 tx=0x7f455800a000 comp rx=0 tx=0).stop 2024-09-13T18:18:46.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.839+0000 7f45537fe700 1 -- 172.21.15.40:0/2925801416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4568165fc0 msgr2=0x7f45681fc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:46.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.839+0000 7f45537fe700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4568165fc0 0x7f45681fc920 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f45600046b0 tx=0x7f4560007330 comp rx=0 tx=0).stop 2024-09-13T18:18:46.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.839+0000 7f456dcbe700 1 -- 172.21.15.40:0/2925801416 reap_dead start 2024-09-13T18:18:46.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.839+0000 7f45537fe700 1 -- 172.21.15.40:0/2925801416 shutdown_connections 2024-09-13T18:18:46.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.839+0000 7f45537fe700 1 --2- 172.21.15.40:0/2925801416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4568165fc0 0x7f45681fc920 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:46.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.839+0000 7f45537fe700 1 -- 172.21.15.40:0/2925801416 >> 172.21.15.40:0/2925801416 conn(0x7f456806d5b0 msgr2=0x7f45680c0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:46.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.839+0000 7f45537fe700 1 -- 172.21.15.40:0/2925801416 shutdown_connections 2024-09-13T18:18:46.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:46.839+0000 7f45537fe700 1 -- 172.21.15.40:0/2925801416 wait complete. 2024-09-13T18:18:46.882 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:18:46.882 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:18:46.882 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:18:47.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.131+0000 7fbcd1812700 1 Processor -- start 2024-09-13T18:18:47.141 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.131+0000 7fbcd1812700 1 -- start start 2024-09-13T18:18:47.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.131+0000 7fbcd1812700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbccc154240 0x7fbccc15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:47.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.131+0000 7fbcd1812700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbccc15ec60 0x7fbccc15f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:47.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.131+0000 7fbcd1812700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbccc071690 con 0x7fbccc06a390 2024-09-13T18:18:47.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.131+0000 7fbcd1812700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbccc0721e0 con 0x7fbccc15ec60 2024-09-13T18:18:47.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.131+0000 7fbcd1812700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc15e6e0 con 0x7fbccc154240 2024-09-13T18:18:47.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.131+0000 7fbcd1011700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbccc06a390 0x7fbccc06a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:52358/0 (socket says 172.21.15.40:52358) 2024-09-13T18:18:47.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.131+0000 7fbcd1011700 1 -- 172.21.15.40:0/3012105096 learned_addr learned my addr 172.21.15.40:0/3012105096 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:18:47.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.131+0000 7fbcd0810700 1 --2- 172.21.15.40:0/3012105096 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbccc154240 0x7fbccc15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:47.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.135+0000 7fbccbfff700 1 --2- 172.21.15.40:0/3012105096 >> v2:172.21.15.43:3300/0 conn(0x7fbccc15ec60 0x7fbccc15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:47.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.135+0000 7fbcd0810700 1 -- 172.21.15.40:0/3012105096 >> v2:172.21.15.43:3300/0 conn(0x7fbccc15ec60 msgr2=0x7fbccc15f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:18:47.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.135+0000 7fbcd0810700 1 --2- 172.21.15.40:0/3012105096 >> v2:172.21.15.43:3300/0 conn(0x7fbccc15ec60 0x7fbccc15f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:47.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.135+0000 7fbcd0810700 1 -- 172.21.15.40:0/3012105096 >> v1:172.21.15.40:6789/0 conn(0x7fbccc06a390 legacy=0x7fbccc06a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:47.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.135+0000 7fbcd0810700 1 -- 172.21.15.40:0/3012105096 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc154620 con 0x7fbccc154240 2024-09-13T18:18:47.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.135+0000 7fbccb7fe700 1 -- 172.21.15.40:0/3012105096 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1545041232 0 0) 0x7fbccc071690 con 0x7fbccc06a390 2024-09-13T18:18:47.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.135+0000 7fbcd0810700 1 --2- 172.21.15.40:0/3012105096 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbccc154240 0x7fbccc15e1a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4009d90 tx=0x7fbcc400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f653a70002a714ce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:47.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.135+0000 7fbccb7fe700 1 -- 172.21.15.40:0/3012105096 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc400e070 con 0x7fbccc154240 2024-09-13T18:18:47.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.135+0000 7fbccb7fe700 1 -- 172.21.15.40:0/3012105096 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc400bbd0 con 0x7fbccc154240 2024-09-13T18:18:47.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.135+0000 7fbccb7fe700 1 -- 172.21.15.40:0/3012105096 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4012430 con 0x7fbccc154240 2024-09-13T18:18:47.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.135+0000 7fbcd1812700 1 -- 172.21.15.40:0/3012105096 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbccc154240 msgr2=0x7fbccc15e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:47.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.135+0000 7fbcd1812700 1 --2- 172.21.15.40:0/3012105096 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbccc154240 0x7fbccc15e1a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4009d90 tx=0x7fbcc400b330 comp rx=0 tx=0).stop 2024-09-13T18:18:47.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.135+0000 7fbcd1812700 1 -- 172.21.15.40:0/3012105096 shutdown_connections 2024-09-13T18:18:47.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.135+0000 7fbcd1812700 1 --2- 172.21.15.40:0/3012105096 >> v2:172.21.15.43:3300/0 conn(0x7fbccc15ec60 0x7fbccc15f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:47.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.135+0000 7fbcd1812700 1 --2- 172.21.15.40:0/3012105096 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbccc154240 0x7fbccc15e1a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:47.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.135+0000 7fbcd1812700 1 -- 172.21.15.40:0/3012105096 >> 172.21.15.40:0/3012105096 conn(0x7fbccc0cabc0 msgr2=0x7fbccc0cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:47.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.135+0000 7fbcd1812700 1 -- 172.21.15.40:0/3012105096 shutdown_connections 2024-09-13T18:18:47.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.135+0000 7fbcd1812700 1 -- 172.21.15.40:0/3012105096 wait complete. 2024-09-13T18:18:47.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.139+0000 7fbcd1812700 1 Processor -- start 2024-09-13T18:18:47.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.139+0000 7fbcd1812700 1 -- start start 2024-09-13T18:18:47.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.139+0000 7fbcd1812700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbccc06a390 0x7fbccc0c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:47.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.139+0000 7fbcd1812700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbccc154240 0x7fbccc1569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:47.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.139+0000 7fbcd1812700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbccc15ec60 0x7fbccc156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.139+0000 7fbcd1812700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc1563d0 con 0x7fbccc154240 2024-09-13T18:18:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.139+0000 7fbcd1812700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc15e2e0 con 0x7fbccc06a390 2024-09-13T18:18:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.139+0000 7fbcd1812700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc1f3e30 con 0x7fbccc15ec60 2024-09-13T18:18:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.139+0000 7fbcd0810700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbccc06a390 0x7fbccc0c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.139+0000 7fbcd0810700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbccc06a390 0x7fbccc0c55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:48270/0 (socket says 172.21.15.40:48270) 2024-09-13T18:18:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.139+0000 7fbcd0810700 1 -- 172.21.15.40:0/3696951322 learned_addr learned my addr 172.21.15.40:0/3696951322 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.139+0000 7fbcd1011700 1 --2- 172.21.15.40:0/3696951322 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbccc15ec60 0x7fbccc156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.139+0000 7fbcd1011700 1 -- 172.21.15.40:0/3696951322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbccc06a390 msgr2=0x7fbccc0c55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.139+0000 7fbcd1011700 1 --2- 172.21.15.40:0/3696951322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbccc06a390 0x7fbccc0c55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.139+0000 7fbcd1011700 1 -- 172.21.15.40:0/3696951322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbccc154240 msgr2=0x7fbccc1569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.139+0000 7fbcd1011700 1 --2- 172.21.15.40:0/3696951322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbccc154240 0x7fbccc1569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.139+0000 7fbcd1011700 1 -- 172.21.15.40:0/3696951322 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc4009a70 con 0x7fbccc15ec60 2024-09-13T18:18:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.139+0000 7fbcd1011700 1 --2- 172.21.15.40:0/3696951322 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbccc15ec60 0x7fbccc156f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00c480 tx=0x7fbcbc00c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:47.158 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.143+0000 7fbcc97fa700 1 -- 172.21.15.40:0/3696951322 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc004ca0 con 0x7fbccc15ec60 2024-09-13T18:18:47.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.143+0000 7fbcd1812700 1 -- 172.21.15.40:0/3696951322 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbccc1f4110 con 0x7fbccc15ec60 2024-09-13T18:18:47.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.143+0000 7fbcd1812700 1 -- 172.21.15.40:0/3696951322 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbccc0c5d50 con 0x7fbccc15ec60 2024-09-13T18:18:47.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.143+0000 7fbcc97fa700 1 -- 172.21.15.40:0/3696951322 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcbc004e20 con 0x7fbccc15ec60 2024-09-13T18:18:47.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.143+0000 7fbcc97fa700 1 -- 172.21.15.40:0/3696951322 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc007310 con 0x7fbccc15ec60 2024-09-13T18:18:47.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.143+0000 7fbcb2ffd700 1 -- 172.21.15.40:0/3696951322 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcac005310 con 0x7fbccc15ec60 2024-09-13T18:18:47.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.151+0000 7fbcc97fa700 1 -- 172.21.15.40:0/3696951322 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbcbc0074b0 con 0x7fbccc15ec60 2024-09-13T18:18:47.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.151+0000 7fbcc97fa700 1 --2- 172.21.15.40:0/3696951322 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbcb8066800 0x7fbcb8068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:47.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.151+0000 7fbcc97fa700 1 -- 172.21.15.40:0/3696951322 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbcbc084050 con 0x7fbccc15ec60 2024-09-13T18:18:47.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.151+0000 7fbcd0810700 1 --2- 172.21.15.40:0/3696951322 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbcb8066800 0x7fbcb8068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:47.160 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.151+0000 7fbcd0810700 1 --2- 172.21.15.40:0/3696951322 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbcb8066800 0x7fbcb8068cc0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400b370 tx=0x7fbcc4011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:47.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.167+0000 7fbcc97fa700 1 -- 172.21.15.40:0/3696951322 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbcbc051020 con 0x7fbccc15ec60 2024-09-13T18:18:47.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.951+0000 7fbcb2ffd700 1 -- 172.21.15.40:0/3696951322 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7fbcac005d30 con 0x7fbccc15ec60 2024-09-13T18:18:47.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.951+0000 7fbcc97fa700 1 -- 172.21.15.40:0/3696951322 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7fbcbc054600 con 0x7fbccc15ec60 2024-09-13T18:18:47.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.963+0000 7fbcb2ffd700 1 -- 172.21.15.40:0/3696951322 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbcb8066800 msgr2=0x7fbcb8068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:47.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.963+0000 7fbcb2ffd700 1 --2- 172.21.15.40:0/3696951322 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbcb8066800 0x7fbcb8068cc0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400b370 tx=0x7fbcc4011000 comp rx=0 tx=0).stop 2024-09-13T18:18:47.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.963+0000 7fbcb2ffd700 1 -- 172.21.15.40:0/3696951322 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbccc15ec60 msgr2=0x7fbccc156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:47.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.963+0000 7fbcb2ffd700 1 --2- 172.21.15.40:0/3696951322 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbccc15ec60 0x7fbccc156f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00c480 tx=0x7fbcbc00c790 comp rx=0 tx=0).stop 2024-09-13T18:18:47.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.963+0000 7fbcb2ffd700 1 -- 172.21.15.40:0/3696951322 shutdown_connections 2024-09-13T18:18:47.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.963+0000 7fbcb2ffd700 1 --2- 172.21.15.40:0/3696951322 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbcb8066800 0x7fbcb8068cc0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:47.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.963+0000 7fbcb2ffd700 1 --2- 172.21.15.40:0/3696951322 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbccc15ec60 0x7fbccc156f30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:47.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.963+0000 7fbcb2ffd700 1 --2- 172.21.15.40:0/3696951322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbccc154240 0x7fbccc1569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:47.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.963+0000 7fbcb2ffd700 1 --2- 172.21.15.40:0/3696951322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbccc06a390 0x7fbccc0c55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:47.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.963+0000 7fbcb2ffd700 1 -- 172.21.15.40:0/3696951322 >> 172.21.15.40:0/3696951322 conn(0x7fbccc0cabc0 msgr2=0x7fbccc150ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:47.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.963+0000 7fbcb2ffd700 1 -- 172.21.15.40:0/3696951322 shutdown_connections 2024-09-13T18:18:47.974 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:47.963+0000 7fbcb2ffd700 1 -- 172.21.15.40:0/3696951322 wait complete. 2024-09-13T18:18:47.995 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:18:57.996 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:18:57.997 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:18:58.523 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.507+0000 7f680f59e700 1 Processor -- start 2024-09-13T18:18:58.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.507+0000 7f680f59e700 1 -- start start 2024-09-13T18:18:58.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.511+0000 7f680f59e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6810154240 0x7f681015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:58.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.511+0000 7f680f59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6810071690 con 0x7f681015ec60 2024-09-13T18:18:58.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.511+0000 7f680f59e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f681015f4e0 con 0x7f681015f0d0 2024-09-13T18:18:58.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.511+0000 7f680f59e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68100721e0 con 0x7f6810154240 2024-09-13T18:18:58.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.511+0000 7f680e59c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6810154240 0x7f681015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:58.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.511+0000 7f680e59c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6810154240 0x7f681015e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:37198/0 (socket says 172.21.15.40:37198) 2024-09-13T18:18:58.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.511+0000 7f680e59c700 1 -- 172.21.15.40:0/770253151 learned_addr learned my addr 172.21.15.40:0/770253151 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:58.524 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.511+0000 7f680e59c700 1 -- 172.21.15.40:0/770253151 >> v1:172.21.15.43:6789/0 conn(0x7f681015f0d0 legacy=0x7f681006aaa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:18:58.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.515+0000 7f680e59c700 1 -- 172.21.15.40:0/770253151 >> v1:172.21.15.40:6789/0 conn(0x7f681015ec60 legacy=0x7f681006a390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:18:58.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.519+0000 7f680e59c700 1 -- 172.21.15.40:0/770253151 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6810154620 con 0x7f6810154240 2024-09-13T18:18:58.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.519+0000 7f680e59c700 1 --2- 172.21.15.40:0/770253151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6810154240 0x7f681015e1a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6800009910 tx=0x7f6800009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c10d5781a1e438 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:58.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.519+0000 7f680d59a700 1 -- 172.21.15.40:0/770253151 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6800004230 con 0x7f6810154240 2024-09-13T18:18:58.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.519+0000 7f680d59a700 1 -- 172.21.15.40:0/770253151 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6800004390 con 0x7f6810154240 2024-09-13T18:18:58.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.519+0000 7f680f59e700 1 -- 172.21.15.40:0/770253151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6810154240 msgr2=0x7f681015e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:58.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.519+0000 7f680f59e700 1 --2- 172.21.15.40:0/770253151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6810154240 0x7f681015e1a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6800009910 tx=0x7f6800009c20 comp rx=0 tx=0).stop 2024-09-13T18:18:58.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.523+0000 7f680f59e700 1 -- 172.21.15.40:0/770253151 shutdown_connections 2024-09-13T18:18:58.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.523+0000 7f680f59e700 1 --2- 172.21.15.40:0/770253151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6810154240 0x7f681015e1a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:58.533 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.523+0000 7f680f59e700 1 -- 172.21.15.40:0/770253151 >> 172.21.15.40:0/770253151 conn(0x7f68100cabc0 msgr2=0x7f68100cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:58.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.527+0000 7f680f59e700 1 -- 172.21.15.40:0/770253151 shutdown_connections 2024-09-13T18:18:58.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.531+0000 7f680f59e700 1 -- 172.21.15.40:0/770253151 wait complete. 2024-09-13T18:18:58.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.531+0000 7f680f59e700 1 Processor -- start 2024-09-13T18:18:58.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.531+0000 7f680f59e700 1 -- start start 2024-09-13T18:18:58.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.531+0000 7f680f59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6810154240 0x7f681016a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:58.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.531+0000 7f680f59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f681015ec60 0x7f68100c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:58.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.531+0000 7f680f59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f681015f0d0 0x7f6810154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:58.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.531+0000 7f680f59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681013bb60 con 0x7f681015f0d0 2024-09-13T18:18:58.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.531+0000 7f680f59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681013a950 con 0x7f681015ec60 2024-09-13T18:18:58.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.531+0000 7f680f59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6810139d60 con 0x7f6810154240 2024-09-13T18:18:58.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.531+0000 7f680ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f681015f0d0 0x7f6810154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.531+0000 7f680ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f681015f0d0 0x7f6810154800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53576/0 (socket says 172.21.15.40:53576) 2024-09-13T18:18:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.531+0000 7f680ed9d700 1 -- 172.21.15.40:0/1764982316 learned_addr learned my addr 172.21.15.40:0/1764982316 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.531+0000 7f680dd9b700 1 --2- 172.21.15.40:0/1764982316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f681015ec60 0x7f68100c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.531+0000 7f680ed9d700 1 -- 172.21.15.40:0/1764982316 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6810154240 msgr2=0x7f681016a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.531+0000 7f680ed9d700 1 --2- 172.21.15.40:0/1764982316 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6810154240 0x7f681016a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.531+0000 7f680ed9d700 1 -- 172.21.15.40:0/1764982316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f681015ec60 msgr2=0x7f68100c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.531+0000 7f680ed9d700 1 --2- 172.21.15.40:0/1764982316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f681015ec60 0x7f68100c7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.531+0000 7f680ed9d700 1 -- 172.21.15.40:0/1764982316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6800009570 con 0x7f681015f0d0 2024-09-13T18:18:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.535+0000 7f680ed9d700 1 --2- 172.21.15.40:0/1764982316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f681015f0d0 0x7f6810154800 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f680400b410 tx=0x7f680400b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.535+0000 7f67feffd700 1 -- 172.21.15.40:0/1764982316 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804011830 con 0x7f681015f0d0 2024-09-13T18:18:58.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.535+0000 7f680f59e700 1 -- 172.21.15.40:0/1764982316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68101380d0 con 0x7f681015f0d0 2024-09-13T18:18:58.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.535+0000 7f680f59e700 1 -- 172.21.15.40:0/1764982316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681013a300 con 0x7f681015f0d0 2024-09-13T18:18:58.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.535+0000 7f67feffd700 1 -- 172.21.15.40:0/1764982316 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6804011990 con 0x7f681015f0d0 2024-09-13T18:18:58.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.535+0000 7f67feffd700 1 -- 172.21.15.40:0/1764982316 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804011d40 con 0x7f681015f0d0 2024-09-13T18:18:58.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.535+0000 7f67feffd700 1 -- 172.21.15.40:0/1764982316 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f680401a400 con 0x7f681015f0d0 2024-09-13T18:18:58.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.539+0000 7f67feffd700 1 --2- 172.21.15.40:0/1764982316 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f67f0066880 0x7f67f0068d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:58.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.539+0000 7f67feffd700 1 -- 172.21.15.40:0/1764982316 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f68040844c0 con 0x7f681015f0d0 2024-09-13T18:18:58.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.539+0000 7f680e59c700 1 --2- 172.21.15.40:0/1764982316 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f67f0066880 0x7f67f0068d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:58.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.555+0000 7f680e59c700 1 --2- 172.21.15.40:0/1764982316 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f67f0066880 0x7f67f0068d40 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6800009910 tx=0x7f6800010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:58.568 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.555+0000 7f680f59e700 1 -- 172.21.15.40:0/1764982316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f4005310 con 0x7f681015f0d0 2024-09-13T18:18:58.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:58.567+0000 7f67feffd700 1 -- 172.21.15.40:0/1764982316 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6804059e60 con 0x7f681015f0d0 2024-09-13T18:18:59.225 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.215+0000 7f680f59e700 1 -- 172.21.15.40:0/1764982316 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f67f4005b40 con 0x7f681015f0d0 2024-09-13T18:18:59.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.219+0000 7f67feffd700 1 -- 172.21.15.40:0/1764982316 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f6804015070 con 0x7f681015f0d0 2024-09-13T18:18:59.226 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":224,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:18:59.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.239+0000 7f680f59e700 1 -- 172.21.15.40:0/1764982316 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f67f0066880 msgr2=0x7f67f0068d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:59.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.239+0000 7f680f59e700 1 --2- 172.21.15.40:0/1764982316 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f67f0066880 0x7f67f0068d40 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6800009910 tx=0x7f6800010000 comp rx=0 tx=0).stop 2024-09-13T18:18:59.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.239+0000 7f680f59e700 1 -- 172.21.15.40:0/1764982316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f681015f0d0 msgr2=0x7f6810154800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:59.246 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.239+0000 7f680f59e700 1 --2- 172.21.15.40:0/1764982316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f681015f0d0 0x7f6810154800 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f680400b410 tx=0x7f680400b720 comp rx=0 tx=0).stop 2024-09-13T18:18:59.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.239+0000 7f680f59e700 1 -- 172.21.15.40:0/1764982316 shutdown_connections 2024-09-13T18:18:59.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.239+0000 7f680f59e700 1 --2- 172.21.15.40:0/1764982316 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f681015f0d0 0x7f6810154800 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:59.247 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.239+0000 7f680f59e700 1 --2- 172.21.15.40:0/1764982316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f681015ec60 0x7f68100c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:59.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.239+0000 7f680f59e700 1 --2- 172.21.15.40:0/1764982316 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f67f0066880 0x7f67f0068d40 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:59.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.239+0000 7f680f59e700 1 --2- 172.21.15.40:0/1764982316 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6810154240 0x7f681016a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:59.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.239+0000 7f680f59e700 1 -- 172.21.15.40:0/1764982316 >> 172.21.15.40:0/1764982316 conn(0x7f68100cabc0 msgr2=0x7f68100c25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:59.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.239+0000 7f680f59e700 1 -- 172.21.15.40:0/1764982316 shutdown_connections 2024-09-13T18:18:59.248 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.239+0000 7f680f59e700 1 -- 172.21.15.40:0/1764982316 wait complete. 2024-09-13T18:18:59.267 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:18:59.267 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T18:18:59.267 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:18:59.268 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:18:59.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.619+0000 7fa7e3fff700 1 Processor -- start 2024-09-13T18:18:59.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.619+0000 7fa7e3fff700 1 -- start start 2024-09-13T18:18:59.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.623+0000 7fa7e3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7e40c7f60 0x7fa7e40c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:59.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.623+0000 7fa7e3fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa7e40c8340 0x7fa7e40c6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:59.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.623+0000 7fa7e3fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e4167290 con 0x7fa7e406b080 2024-09-13T18:18:59.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.623+0000 7fa7e3fff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e40c1630 con 0x7fa7e40c8340 2024-09-13T18:18:59.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.623+0000 7fa7e3fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e4166680 con 0x7fa7e40c7f60 2024-09-13T18:18:59.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.623+0000 7fa7e27fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa7e40c8340 0x7fa7e40c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:59.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.623+0000 7fa7e27fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa7e40c8340 0x7fa7e40c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53582/0 (socket says 172.21.15.40:53582) 2024-09-13T18:18:59.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.623+0000 7fa7e27fc700 1 -- 172.21.15.40:0/3820734232 learned_addr learned my addr 172.21.15.40:0/3820734232 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:59.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.623+0000 7fa7e2ffd700 1 --2- 172.21.15.40:0/3820734232 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7e40c7f60 0x7fa7e40c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:59.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.623+0000 7fa7e27fc700 1 -- 172.21.15.40:0/3820734232 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7e40c7f60 msgr2=0x7fa7e40c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:18:59.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.623+0000 7fa7e27fc700 1 --2- 172.21.15.40:0/3820734232 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7e40c7f60 0x7fa7e40c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:59.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.623+0000 7fa7e27fc700 1 -- 172.21.15.40:0/3820734232 >> v1:172.21.15.43:6789/0 conn(0x7fa7e406b080 legacy=0x7fa7e41635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:18:59.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.623+0000 7fa7e27fc700 1 -- 172.21.15.40:0/3820734232 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e4166960 con 0x7fa7e40c8340 2024-09-13T18:18:59.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.623+0000 7fa7e27fc700 1 --2- 172.21.15.40:0/3820734232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e40c8340 0x7fa7e40c6190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc008e10 tx=0x7fa7dc00b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaba931853368940 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:59.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.623+0000 7fa7e1ffb700 1 -- 172.21.15.40:0/3820734232 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc00be60 con 0x7fa7e40c8340 2024-09-13T18:18:59.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.623+0000 7fa7e1ffb700 1 -- 172.21.15.40:0/3820734232 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7dc003a40 con 0x7fa7e40c8340 2024-09-13T18:18:59.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.627+0000 7fa7e3fff700 1 -- 172.21.15.40:0/3820734232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e40c8340 msgr2=0x7fa7e40c6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:59.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.627+0000 7fa7e3fff700 1 --2- 172.21.15.40:0/3820734232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e40c8340 0x7fa7e40c6190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc008e10 tx=0x7fa7dc00b4b0 comp rx=0 tx=0).stop 2024-09-13T18:18:59.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.627+0000 7fa7e3fff700 1 -- 172.21.15.40:0/3820734232 shutdown_connections 2024-09-13T18:18:59.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.627+0000 7fa7e3fff700 1 --2- 172.21.15.40:0/3820734232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e40c8340 0x7fa7e40c6190 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:59.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.627+0000 7fa7e3fff700 1 --2- 172.21.15.40:0/3820734232 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7e40c7f60 0x7fa7e40c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:59.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.627+0000 7fa7e3fff700 1 -- 172.21.15.40:0/3820734232 >> 172.21.15.40:0/3820734232 conn(0x7fa7e4066090 msgr2=0x7fa7e40664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:59.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.635+0000 7fa7e3fff700 1 -- 172.21.15.40:0/3820734232 shutdown_connections 2024-09-13T18:18:59.645 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.635+0000 7fa7e3fff700 1 -- 172.21.15.40:0/3820734232 wait complete. 2024-09-13T18:18:59.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.635+0000 7fa7e3fff700 1 Processor -- start 2024-09-13T18:18:59.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.635+0000 7fa7e3fff700 1 -- start start 2024-09-13T18:18:59.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.635+0000 7fa7e3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7e406b080 0x7fa7e40c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:59.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.635+0000 7fa7e3fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e40c7f60 0x7fa7e40c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:59.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.635+0000 7fa7e3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e408b660 0x7fa7e40710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:59.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.635+0000 7fa7e3fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e40cbc50 con 0x7fa7e408b660 2024-09-13T18:18:59.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.635+0000 7fa7e3fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e40c9280 con 0x7fa7e40c7f60 2024-09-13T18:18:59.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.635+0000 7fa7e3fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e40cb1b0 con 0x7fa7e406b080 2024-09-13T18:18:59.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.635+0000 7fa7e37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e408b660 0x7fa7e40710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:59.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.639+0000 7fa7e37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e408b660 0x7fa7e40710f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53586/0 (socket says 172.21.15.40:53586) 2024-09-13T18:18:59.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.639+0000 7fa7e37fe700 1 -- 172.21.15.40:0/151278036 learned_addr learned my addr 172.21.15.40:0/151278036 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:18:59.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.639+0000 7fa7e27fc700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e40c7f60 0x7fa7e40c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:59.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.639+0000 7fa7e2ffd700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7e406b080 0x7fa7e40c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:59.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.639+0000 7fa7e37fe700 1 -- 172.21.15.40:0/151278036 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7e406b080 msgr2=0x7fa7e40c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:59.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.639+0000 7fa7e37fe700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7e406b080 0x7fa7e40c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:59.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.639+0000 7fa7e37fe700 1 -- 172.21.15.40:0/151278036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e40c7f60 msgr2=0x7fa7e40c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:59.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.639+0000 7fa7e37fe700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e40c7f60 0x7fa7e40c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:59.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.639+0000 7fa7e37fe700 1 -- 172.21.15.40:0/151278036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc008a70 con 0x7fa7e408b660 2024-09-13T18:18:59.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.639+0000 7fa7e27fc700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e40c7f60 0x7fa7e40c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:18:59.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.639+0000 7fa7e2ffd700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa7e406b080 0x7fa7e40c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:18:59.648 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.639+0000 7fa7e37fe700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e408b660 0x7fa7e40710f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800c470 tx=0x7fa7d800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:59.649 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.643+0000 7fa7cb7fe700 1 -- 172.21.15.40:0/151278036 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8011790 con 0x7fa7e408b660 2024-09-13T18:18:59.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.643+0000 7fa7cb7fe700 1 -- 172.21.15.40:0/151278036 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d80118f0 con 0x7fa7e408b660 2024-09-13T18:18:59.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.643+0000 7fa7cb7fe700 1 -- 172.21.15.40:0/151278036 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8011ca0 con 0x7fa7e408b660 2024-09-13T18:18:59.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.643+0000 7fa7e3fff700 1 -- 172.21.15.40:0/151278036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e4072570 con 0x7fa7e408b660 2024-09-13T18:18:59.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.643+0000 7fa7e3fff700 1 -- 172.21.15.40:0/151278036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e40cb690 con 0x7fa7e408b660 2024-09-13T18:18:59.656 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.647+0000 7fa7cb7fe700 1 -- 172.21.15.40:0/151278036 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa7d801c460 con 0x7fa7e408b660 2024-09-13T18:18:59.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.647+0000 7fa7cb7fe700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa7d0066800 0x7fa7d0068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:59.658 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.651+0000 7fa7cb7fe700 1 -- 172.21.15.40:0/151278036 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa7d8084420 con 0x7fa7e408b660 2024-09-13T18:18:59.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.651+0000 7fa7e3fff700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e40c8e70 0x7fa7e4071630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:59.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.651+0000 7fa7e3fff700 1 -- 172.21.15.40:0/151278036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e4067720 con 0x7fa7e40c8e70 2024-09-13T18:18:59.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.651+0000 7fa7e2ffd700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa7d0066800 0x7fa7d0068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:59.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.651+0000 7fa7e27fc700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e40c8e70 0x7fa7e4071630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:59.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.651+0000 7fa7e27fc700 1 -- 172.21.15.40:0/151278036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa7e408bc60 con 0x7fa7e40c8e70 2024-09-13T18:18:59.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.651+0000 7fa7e2ffd700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa7d0066800 0x7fa7d0068cc0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa7d400bb60 tx=0x7fa7d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:59.661 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.655+0000 7fa7e27fc700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e40c8e70 0x7fa7e4071630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc008e10 tx=0x7fa7dc004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:59.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.655+0000 7fa7cb7fe700 1 -- 172.21.15.40:0/151278036 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc018070 con 0x7fa7e40c8e70 2024-09-13T18:18:59.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.655+0000 7fa7cb7fe700 1 -- 172.21.15.40:0/151278036 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa7dc0088f0 con 0x7fa7e40c8e70 2024-09-13T18:18:59.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.655+0000 7fa7cb7fe700 1 -- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e40c8e70 msgr2=0x7fa7e4071630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:59.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.655+0000 7fa7cb7fe700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e40c8e70 0x7fa7e4071630 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc008e10 tx=0x7fa7dc004af0 comp rx=0 tx=0).stop 2024-09-13T18:18:59.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.655+0000 7fa7cb7fe700 1 -- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e40c8e70 msgr2=0x7fa7e4071630 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:18:59.662 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.655+0000 7fa7cb7fe700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e40c8e70 0x7fa7e4071630 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:59.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.671+0000 7fa7c9ffb700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e415b160 0x7fa7e4160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:18:59.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.671+0000 7fa7e27fc700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e415b160 0x7fa7e4160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:18:59.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.671+0000 7fa7c9ffb700 1 -- 172.21.15.40:0/151278036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e4166370 con 0x7fa7e415b160 2024-09-13T18:18:59.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.671+0000 7fa7e27fc700 1 -- 172.21.15.40:0/151278036 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa7e4166f20 con 0x7fa7e415b160 2024-09-13T18:18:59.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.671+0000 7fa7e27fc700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e415b160 0x7fa7e4160c10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc00b4f0 tx=0x7fa7dc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:18:59.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.671+0000 7fa7cb7fe700 1 -- 172.21.15.40:0/151278036 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc029320 con 0x7fa7e415b160 2024-09-13T18:18:59.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.671+0000 7fa7cb7fe700 1 -- 172.21.15.40:0/151278036 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4453 (secure 0 0 0) 0x7fa7dc011640 con 0x7fa7e415b160 2024-09-13T18:18:59.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.671+0000 7fa7cb7fe700 1 -- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e415b160 msgr2=0x7fa7e4160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:59.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.671+0000 7fa7cb7fe700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e415b160 0x7fa7e4160c10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc00b4f0 tx=0x7fa7dc01a000 comp rx=0 tx=0).stop 2024-09-13T18:18:59.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.671+0000 7fa7cb7fe700 1 -- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e415b160 msgr2=0x7fa7e4160c10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:18:59.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.671+0000 7fa7cb7fe700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e415b160 0x7fa7e4160c10 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc00b4f0 tx=0x7fa7dc01a000 comp rx=0 tx=0).stop 2024-09-13T18:18:59.682 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:18:59.682 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:18:59.682 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:18:59.682 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:18:59.682 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 224, 2024-09-13T18:18:59.682 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:18:59.682 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:18:59.682 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:18:59.682 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:18:59.682 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:59.683 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 15, 2024-09-13T18:18:59.683 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:18:59.683 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:18:59.683 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:18:59.683 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:18:59.683 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:18:59.683 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:18:59.683 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:18:59.683 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:18:59.683 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:18:59.683 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:18:59.683 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:18:59.683 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:18:59.683 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:18:59.684 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:59.684 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:18:59.684 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:18:59.684 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:18:59.684 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:18:59.685 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:18:59.685 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:18:59.685 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:18:59.685 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:18:59.685 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:18:59.685 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:18:59.685 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:18:59.685 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:18:59.685 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:59.685 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:59.685 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:18:59.685 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:18:59.686 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:18:59.686 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:18:59.686 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:18:59.686 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:18:59.686 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:18:59.686 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:18:59.686 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:18:59.686 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:18:59.686 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:18:59.686 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:18:59.686 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:18:59.686 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:18:59.686 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:18:59.686 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:18:59.686 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:18:59.687 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:18:59.687 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:18:59.687 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:18:59.687 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:18:59.687 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:18:59.687 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:18:59.687 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:18:59.687 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:18:59.687 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:18:59.687 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:18:59.687 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:59.687 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:18:59.687 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:59.688 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:18:59.688 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:59.688 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:18:59.688 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:18:59.688 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:18:59.688 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:18:59.688 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:59.688 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:18:59.688 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:18:59.688 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:59.688 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:59.688 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:59.688 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:18:59.689 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:18:59.689 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:59.689 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:59.689 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:59.689 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:59.689 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:18:59.689 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:18:59.689 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:18:59.689 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:18:59.689 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:18:59.689 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:59.689 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:59.689 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:18:59.689 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:18:59.690 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:18:59.690 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:18:59.690 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:59.690 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:18:59.690 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:18:59.690 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:59.690 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:59.690 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:59.690 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:18:59.690 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:18:59.690 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:59.690 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:59.690 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:59.690 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:59.690 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:18:59.691 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:18:59.691 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:18:59.691 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:18:59.691 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:18:59.691 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:59.691 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:59.691 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:18:59.691 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:18:59.691 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:18:59.691 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:18:59.691 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:59.691 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:18:59.692 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:18:59.692 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:59.692 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:59.692 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:59.692 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:18:59.692 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:18:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:18:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:18:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:18:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:18:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:18:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:18:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:18:59.693 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:18:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:18:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:18:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:18:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:18:59.694 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:18:59.695 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:18:59.695 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:18:59.695 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:18:59.695 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:18:59.695 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:18:59.695 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:18:59.695 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:18:59.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.675+0000 7fa7e3fff700 1 -- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa7d0066800 msgr2=0x7fa7d0068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:59.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.675+0000 7fa7e3fff700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa7d0066800 0x7fa7d0068cc0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa7d400bb60 tx=0x7fa7d4008040 comp rx=0 tx=0).stop 2024-09-13T18:18:59.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.675+0000 7fa7e37fe700 1 -- 172.21.15.40:0/151278036 reap_dead start 2024-09-13T18:18:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.675+0000 7fa7e3fff700 1 -- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e408b660 msgr2=0x7fa7e40710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:18:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.675+0000 7fa7e3fff700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e408b660 0x7fa7e40710f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800c470 tx=0x7fa7d800c830 comp rx=0 tx=0).stop 2024-09-13T18:18:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.675+0000 7fa7e3fff700 1 -- 172.21.15.40:0/151278036 shutdown_connections 2024-09-13T18:18:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.675+0000 7fa7e3fff700 1 --2- 172.21.15.40:0/151278036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7e408b660 0x7fa7e40710f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:18:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.675+0000 7fa7e3fff700 1 -- 172.21.15.40:0/151278036 >> 172.21.15.40:0/151278036 conn(0x7fa7e4066090 msgr2=0x7fa7e41633c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:18:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.675+0000 7fa7e3fff700 1 -- 172.21.15.40:0/151278036 shutdown_connections 2024-09-13T18:18:59.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:18:59.675+0000 7fa7e3fff700 1 -- 172.21.15.40:0/151278036 wait complete. 2024-09-13T18:18:59.705 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:18:59.705 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:18:59.706 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:19:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.059+0000 7fc7e34bb700 1 Processor -- start 2024-09-13T18:19:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.059+0000 7fc7e34bb700 1 -- start start 2024-09-13T18:19:00.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.059+0000 7fc7e34bb700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc7dc163d00 0x7fc7dc0c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:00.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.059+0000 7fc7e34bb700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7dc167290 con 0x7fc7dc0c8340 2024-09-13T18:19:00.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.059+0000 7fc7e34bb700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7dc166680 con 0x7fc7dc0c7f60 2024-09-13T18:19:00.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.059+0000 7fc7e34bb700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7dc0c1630 con 0x7fc7dc163d00 2024-09-13T18:19:00.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.059+0000 7fc7e1cb8700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc7dc163d00 0x7fc7dc0c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:00.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.059+0000 7fc7e1cb8700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc7dc163d00 0x7fc7dc0c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:37226/0 (socket says 172.21.15.40:37226) 2024-09-13T18:19:00.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.059+0000 7fc7e1cb8700 1 -- 172.21.15.40:0/2970281692 learned_addr learned my addr 172.21.15.40:0/2970281692 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:19:00.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e14b7700 1 -- 172.21.15.40:0/2970281692 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3012213858 0 0) 0x7fc7dc166680 con 0x7fc7dc0c7f60 2024-09-13T18:19:00.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e14b7700 1 -- 172.21.15.40:0/2970281692 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7c80035b0 con 0x7fc7dc0c7f60 2024-09-13T18:19:00.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e14b7700 1 -- 172.21.15.40:0/2970281692 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294643117 0 0) 0x7fc7dc167290 con 0x7fc7dc0c8340 2024-09-13T18:19:00.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e14b7700 1 -- 172.21.15.40:0/2970281692 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7dc166680 con 0x7fc7dc0c8340 2024-09-13T18:19:00.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e14b7700 1 -- 172.21.15.40:0/2970281692 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2998922455 0 0) 0x7fc7dc166680 con 0x7fc7dc0c8340 2024-09-13T18:19:00.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e14b7700 1 -- 172.21.15.40:0/2970281692 >> v2:172.21.15.43:3300/0 conn(0x7fc7dc163d00 msgr2=0x7fc7dc0c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:19:00.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e14b7700 1 --2- 172.21.15.40:0/2970281692 >> v2:172.21.15.43:3300/0 conn(0x7fc7dc163d00 0x7fc7dc0c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:00.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e14b7700 1 -- 172.21.15.40:0/2970281692 >> v1:172.21.15.43:6789/0 conn(0x7fc7dc0c7f60 legacy=0x7fc7dc1635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:00.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e14b7700 1 -- 172.21.15.40:0/2970281692 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7dc166960 con 0x7fc7dc0c8340 2024-09-13T18:19:00.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e14b7700 1 -- 172.21.15.40:0/2970281692 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fc7cc0015a0 con 0x7fc7dc0c8340 2024-09-13T18:19:00.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e14b7700 1 -- 172.21.15.40:0/2970281692 >> v1:172.21.15.40:6789/0 conn(0x7fc7dc0c8340 legacy=0x7fc7dc06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:00.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e14b7700 1 --2- 172.21.15.40:0/2970281692 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7c8003d80 0x7fc7c8004160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:00.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e14b7700 1 --2- 172.21.15.40:0/2970281692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c8004760 0x7fc7c8008eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:00.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e14b7700 1 --2- 172.21.15.40:0/2970281692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7c80093f0 0x7fc7c800b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:00.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e14b7700 1 -- 172.21.15.40:0/2970281692 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d8001230 con 0x7fc7c8004760 2024-09-13T18:19:00.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e14b7700 1 -- 172.21.15.40:0/2970281692 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc0c1630 con 0x7fc7c80093f0 2024-09-13T18:19:00.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e14b7700 1 -- 172.21.15.40:0/2970281692 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c800c0f0 con 0x7fc7c8003d80 2024-09-13T18:19:00.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e2cba700 1 --2- 172.21.15.40:0/2970281692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7c80093f0 0x7fc7c800b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:00.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e2cba700 1 -- 172.21.15.40:0/2970281692 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7c8003d80 msgr2=0x7fc7c8004160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:19:00.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e2cba700 1 --2- 172.21.15.40:0/2970281692 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7c8003d80 0x7fc7c8004160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:00.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e2cba700 1 -- 172.21.15.40:0/2970281692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c8004760 msgr2=0x7fc7c8008eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:19:00.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e2cba700 1 --2- 172.21.15.40:0/2970281692 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c8004760 0x7fc7c8008eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:00.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e2cba700 1 -- 172.21.15.40:0/2970281692 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7dc166680 con 0x7fc7c80093f0 2024-09-13T18:19:00.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e2cba700 1 -- 172.21.15.40:0/2970281692 reap_dead start 2024-09-13T18:19:00.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e2cba700 1 --2- 172.21.15.40:0/2970281692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7c80093f0 0x7fc7c800b8b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00c1a0 tx=0x7fc7c8004e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efa241d54c9b1028 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:00.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e14b7700 1 -- 172.21.15.40:0/2970281692 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc00ce80 con 0x7fc7c80093f0 2024-09-13T18:19:00.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.063+0000 7fc7e14b7700 1 -- 172.21.15.40:0/2970281692 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7cc034410 con 0x7fc7c80093f0 2024-09-13T18:19:00.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.067+0000 7fc7e34bb700 1 -- 172.21.15.40:0/2970281692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7c80093f0 msgr2=0x7fc7c800b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:00.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.067+0000 7fc7e34bb700 1 --2- 172.21.15.40:0/2970281692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7c80093f0 0x7fc7c800b8b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00c1a0 tx=0x7fc7c8004e00 comp rx=0 tx=0).stop 2024-09-13T18:19:00.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.067+0000 7fc7e34bb700 1 -- 172.21.15.40:0/2970281692 shutdown_connections 2024-09-13T18:19:00.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.067+0000 7fc7e34bb700 1 --2- 172.21.15.40:0/2970281692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7c80093f0 0x7fc7c800b8b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:00.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.067+0000 7fc7e34bb700 1 -- 172.21.15.40:0/2970281692 >> 172.21.15.40:0/2970281692 conn(0x7fc7dc066090 msgr2=0x7fc7dc0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:00.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.071+0000 7fc7e34bb700 1 -- 172.21.15.40:0/2970281692 shutdown_connections 2024-09-13T18:19:00.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.071+0000 7fc7e34bb700 1 -- 172.21.15.40:0/2970281692 wait complete. 2024-09-13T18:19:00.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.079+0000 7fc7e34bb700 1 Processor -- start 2024-09-13T18:19:00.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.079+0000 7fc7e34bb700 1 -- start start 2024-09-13T18:19:00.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.079+0000 7fc7e34bb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c80093f0 0x7fc7dc072310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:00.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.079+0000 7fc7e34bb700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7dc15e8c0 0x7fc7dc0711f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:00.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.079+0000 7fc7e34bb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7dc23fc60 0x7fc7dc2400e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:00.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.079+0000 7fc7e34bb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc15fe90 con 0x7fc7c80093f0 2024-09-13T18:19:00.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.079+0000 7fc7e34bb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc240620 con 0x7fc7dc23fc60 2024-09-13T18:19:00.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.079+0000 7fc7e34bb700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc15f280 con 0x7fc7dc15e8c0 2024-09-13T18:19:00.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.079+0000 7fc7e1cb8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7dc15e8c0 0x7fc7dc0711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:00.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.079+0000 7fc7e1cb8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7dc15e8c0 0x7fc7dc0711f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:51584/0 (socket says 172.21.15.40:51584) 2024-09-13T18:19:00.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.079+0000 7fc7e1cb8700 1 -- 172.21.15.40:0/3445489764 learned_addr learned my addr 172.21.15.40:0/3445489764 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:19:00.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.079+0000 7fc7e2cba700 1 --2- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7dc23fc60 0x7fc7dc2400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:00.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.079+0000 7fc7e1cb8700 1 -- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7dc23fc60 msgr2=0x7fc7dc2400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:00.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.079+0000 7fc7e1cb8700 1 --2- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7dc23fc60 0x7fc7dc2400e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:00.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.079+0000 7fc7e1cb8700 1 -- 172.21.15.40:0/3445489764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c80093f0 msgr2=0x7fc7dc072310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:19:00.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.079+0000 7fc7e1cb8700 1 --2- 172.21.15.40:0/3445489764 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc7c80093f0 0x7fc7dc072310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:00.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.079+0000 7fc7e1cb8700 1 -- 172.21.15.40:0/3445489764 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7dc15f560 con 0x7fc7dc15e8c0 2024-09-13T18:19:00.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.079+0000 7fc7e1cb8700 1 --2- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7dc15e8c0 0x7fc7dc0711f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc1640e0 tx=0x7fc7d400ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:00.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.083+0000 7fc7d2ffd700 1 -- 172.21.15.40:0/3445489764 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4003e50 con 0x7fc7dc15e8c0 2024-09-13T18:19:00.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.083+0000 7fc7d2ffd700 1 -- 172.21.15.40:0/3445489764 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d400f040 con 0x7fc7dc15e8c0 2024-09-13T18:19:00.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.083+0000 7fc7d2ffd700 1 -- 172.21.15.40:0/3445489764 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4009350 con 0x7fc7dc15e8c0 2024-09-13T18:19:00.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.083+0000 7fc7e34bb700 1 -- 172.21.15.40:0/3445489764 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7dc15f840 con 0x7fc7dc15e8c0 2024-09-13T18:19:00.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.083+0000 7fc7e34bb700 1 -- 172.21.15.40:0/3445489764 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7dc15ed20 con 0x7fc7dc15e8c0 2024-09-13T18:19:00.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.099+0000 7fc7e34bb700 1 --2- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7dc242e80 0x7fc7dc243260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:00.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.099+0000 7fc7e34bb700 1 -- 172.21.15.40:0/3445489764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc15f6f0 con 0x7fc7dc242e80 2024-09-13T18:19:00.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.099+0000 7fc7e24b9700 1 --2- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7dc242e80 0x7fc7dc243260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:00.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.103+0000 7fc7d2ffd700 1 -- 172.21.15.40:0/3445489764 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc7d40094b0 con 0x7fc7dc15e8c0 2024-09-13T18:19:00.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.103+0000 7fc7e24b9700 1 -- 172.21.15.40:0/3445489764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc7dc08bc60 con 0x7fc7dc242e80 2024-09-13T18:19:00.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.103+0000 7fc7e24b9700 1 --2- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7dc242e80 0x7fc7dc243260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800c6b0 tx=0x7fc7d800ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:00.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.103+0000 7fc7d2ffd700 1 --2- 172.21.15.40:0/3445489764 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc7c0066800 0x7fc7c0068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:00.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.103+0000 7fc7d2ffd700 1 -- 172.21.15.40:0/3445489764 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc7d4084640 con 0x7fc7dc15e8c0 2024-09-13T18:19:00.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.103+0000 7fc7e2cba700 1 --2- 172.21.15.40:0/3445489764 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc7c0066800 0x7fc7c0068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:00.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.103+0000 7fc7d2ffd700 1 -- 172.21.15.40:0/3445489764 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d80036a0 con 0x7fc7dc242e80 2024-09-13T18:19:00.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.103+0000 7fc7d2ffd700 1 -- 172.21.15.40:0/3445489764 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fc7d8003800 con 0x7fc7dc242e80 2024-09-13T18:19:00.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.103+0000 7fc7d2ffd700 1 -- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7dc242e80 msgr2=0x7fc7dc243260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:00.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.103+0000 7fc7d2ffd700 1 --2- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7dc242e80 0x7fc7dc243260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800c6b0 tx=0x7fc7d800ca70 comp rx=0 tx=0).stop 2024-09-13T18:19:00.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.103+0000 7fc7d2ffd700 1 -- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7dc242e80 msgr2=0x7fc7dc243260 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:19:00.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.103+0000 7fc7d2ffd700 1 --2- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7dc242e80 0x7fc7dc243260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:00.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.103+0000 7fc7e2cba700 1 --2- 172.21.15.40:0/3445489764 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc7c0066800 0x7fc7c0068cc0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800c0c0 tx=0x7fc7cc033000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:00.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.119+0000 7fc7e34bb700 1 --2- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7dc2437a0 0x7fc7dc15b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:00.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.119+0000 7fc7e34bb700 1 -- 172.21.15.40:0/3445489764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc067720 con 0x7fc7dc2437a0 2024-09-13T18:19:00.129 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.123+0000 7fc7e24b9700 1 --2- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7dc2437a0 0x7fc7dc15b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:00.133 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.127+0000 7fc7e24b9700 1 -- 172.21.15.40:0/3445489764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fc7dc166370 con 0x7fc7dc2437a0 2024-09-13T18:19:00.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.127+0000 7fc7e24b9700 1 --2- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7dc2437a0 0x7fc7dc15b770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8014000 tx=0x7fc7d8000fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:00.134 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.127+0000 7fc7d2ffd700 1 -- 172.21.15.40:0/3445489764 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d801a2a0 con 0x7fc7dc2437a0 2024-09-13T18:19:00.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.127+0000 7fc7d2ffd700 1 -- 172.21.15.40:0/3445489764 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4949 (secure 0 0 0) 0x7fc7d8021310 con 0x7fc7dc2437a0 2024-09-13T18:19:00.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.127+0000 7fc7d2ffd700 1 -- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7dc2437a0 msgr2=0x7fc7dc15b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:00.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.127+0000 7fc7d2ffd700 1 --2- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7dc2437a0 0x7fc7dc15b770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8014000 tx=0x7fc7d8000fe0 comp rx=0 tx=0).stop 2024-09-13T18:19:00.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.127+0000 7fc7d2ffd700 1 -- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7dc2437a0 msgr2=0x7fc7dc15b770 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:19:00.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.127+0000 7fc7d2ffd700 1 --2- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7dc2437a0 0x7fc7dc15b770 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8014000 tx=0x7fc7d8000fe0 comp rx=0 tx=0).stop 2024-09-13T18:19:00.142 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:19:00.142 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:19:00.142 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:19:00.142 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:19:00.142 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 224, 2024-09-13T18:19:00.142 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:19:00.142 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:19:00.142 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:19:00.142 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:19:00.142 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:00.142 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 15, 2024-09-13T18:19:00.142 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:19:00.143 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:19:00.143 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:19:00.143 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:19:00.143 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:19:00.143 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:19:00.143 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:19:00.143 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:19:00.143 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:19:00.143 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:19:00.143 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:19:00.143 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:19:00.143 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:19:00.144 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:00.144 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:19:00.144 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:19:00.144 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:19:00.144 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:19:00.144 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:19:00.144 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:19:00.144 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:19:00.144 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:19:00.144 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:19:00.144 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:19:00.144 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:19:00.144 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:19:00.145 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:00.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.145 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:19:00.145 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:19:00.145 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:19:00.145 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:19:00.145 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:19:00.145 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:19:00.145 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:19:00.145 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:19:00.145 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:19:00.145 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:19:00.145 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:19:00.145 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:19:00.146 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:19:00.146 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:19:00.146 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:19:00.146 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:19:00.146 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:19:00.146 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:19:00.146 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:19:00.146 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:19:00.146 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:19:00.146 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:19:00.146 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:19:00.146 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:19:00.146 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:19:00.146 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:19:00.147 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:19:00.147 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:00.147 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:19:00.147 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.147 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:19:00.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.147 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:19:00.147 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:19:00.147 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:19:00.147 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:19:00.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.147 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:19:00.147 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:19:00.147 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:00.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.148 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:19:00.148 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:19:00.148 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:00.148 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:00.148 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:00.148 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.148 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:19:00.148 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:19:00.148 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:19:00.148 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:19:00.148 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:19:00.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.149 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.149 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:19:00.149 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:19:00.149 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:19:00.149 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:19:00.149 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.149 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:19:00.149 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:19:00.149 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:00.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.149 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.149 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:19:00.150 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:19:00.150 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:00.150 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:00.150 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:00.150 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.150 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:19:00.150 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:19:00.150 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:19:00.150 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:19:00.150 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:19:00.150 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.150 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.150 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:19:00.150 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:19:00.151 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:19:00.151 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:19:00.151 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.151 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:19:00.151 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:19:00.151 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:00.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.151 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.151 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:19:00.151 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:19:00.151 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:00.151 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:00.151 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:00.151 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.152 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:19:00.152 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:19:00.152 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:19:00.152 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:19:00.152 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:19:00.152 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:00.152 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:00.152 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.152 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:19:00.152 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:19:00.152 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.152 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:19:00.152 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:19:00.153 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:19:00.153 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:00.153 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:00.153 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:19:00.153 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.153 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:19:00.153 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:19:00.153 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:19:00.153 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:00.153 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:00.153 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:19:00.153 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.153 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:19:00.154 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:19:00.154 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:19:00.154 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:00.154 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:00.154 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:19:00.154 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.154 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:19:00.154 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:19:00.154 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:19:00.154 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:00.154 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:00.154 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:19:00.154 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.154 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:19:00.155 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:19:00.155 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:19:00.155 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:00.155 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:00.155 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.155 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:19:00.155 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:19:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.155+0000 7fc7d0ff9700 1 -- 172.21.15.40:0/3445489764 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc7c0066800 msgr2=0x7fc7c0068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.155+0000 7fc7d0ff9700 1 --2- 172.21.15.40:0/3445489764 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc7c0066800 0x7fc7c0068cc0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800c0c0 tx=0x7fc7cc033000 comp rx=0 tx=0).stop 2024-09-13T18:19:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.155+0000 7fc7d0ff9700 1 -- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7dc15e8c0 msgr2=0x7fc7dc0711f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.155+0000 7fc7d0ff9700 1 --2- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7dc15e8c0 0x7fc7dc0711f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc1640e0 tx=0x7fc7d400ef60 comp rx=0 tx=0).stop 2024-09-13T18:19:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.155+0000 7fc7e2cba700 1 -- 172.21.15.40:0/3445489764 reap_dead start 2024-09-13T18:19:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.155+0000 7fc7d0ff9700 1 -- 172.21.15.40:0/3445489764 shutdown_connections 2024-09-13T18:19:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.155+0000 7fc7d0ff9700 1 --2- 172.21.15.40:0/3445489764 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc7dc15e8c0 0x7fc7dc0711f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.155+0000 7fc7d0ff9700 1 -- 172.21.15.40:0/3445489764 >> 172.21.15.40:0/3445489764 conn(0x7fc7dc066090 msgr2=0x7fc7dc0c6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.155+0000 7fc7d0ff9700 1 -- 172.21.15.40:0/3445489764 shutdown_connections 2024-09-13T18:19:00.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.155+0000 7fc7d0ff9700 1 -- 172.21.15.40:0/3445489764 wait complete. 2024-09-13T18:19:00.182 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:19:00.183 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:19:00.183 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:19:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.371+0000 7f3ad6572700 1 Processor -- start 2024-09-13T18:19:00.381 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.371+0000 7f3ad6572700 1 -- start start 2024-09-13T18:19:00.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.375+0000 7f3ad6572700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ad00c7f60 0x7f3ad00c17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:00.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.375+0000 7f3ad6572700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3ad00c8340 0x7f3ad00bff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:00.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.375+0000 7f3ad6572700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad00af830 con 0x7f3ad00b00b0 2024-09-13T18:19:00.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.375+0000 7f3ad6572700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad00c75c0 con 0x7f3ad00c8340 2024-09-13T18:19:00.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.375+0000 7f3ad6572700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad006b3c0 con 0x7f3ad00c7f60 2024-09-13T18:19:00.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.375+0000 7f3ad5570700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ad00c7f60 0x7f3ad00c17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:00.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.375+0000 7f3ad5570700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ad00c7f60 0x7f3ad00c17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:51586/0 (socket says 172.21.15.40:51586) 2024-09-13T18:19:00.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.375+0000 7f3ad5570700 1 -- 172.21.15.40:0/753754267 learned_addr learned my addr 172.21.15.40:0/753754267 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:19:00.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.375+0000 7f3ad5570700 1 -- 172.21.15.40:0/753754267 >> v2:172.21.15.43:3300/0 conn(0x7f3ad00c8340 msgr2=0x7f3ad00bff60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:19:00.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.375+0000 7f3ad5570700 1 --2- 172.21.15.40:0/753754267 >> v2:172.21.15.43:3300/0 conn(0x7f3ad00c8340 0x7f3ad00bff60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:00.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.375+0000 7f3ad5570700 1 -- 172.21.15.40:0/753754267 >> v1:172.21.15.43:6789/0 conn(0x7f3ad00b00b0 legacy=0x7f3ad006d2d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:19:00.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.375+0000 7f3ad5570700 1 -- 172.21.15.40:0/753754267 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad006b120 con 0x7f3ad00c7f60 2024-09-13T18:19:00.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.375+0000 7f3ad5570700 1 --2- 172.21.15.40:0/753754267 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ad00c7f60 0x7f3ad00c17e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000ffd0 tx=0x7f3ac0014850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84a521e85817d8aa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:00.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.375+0000 7f3ac7fff700 1 -- 172.21.15.40:0/753754267 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0005e50 con 0x7f3ad00c7f60 2024-09-13T18:19:00.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.375+0000 7f3ac7fff700 1 -- 172.21.15.40:0/753754267 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac000f0c0 con 0x7f3ad00c7f60 2024-09-13T18:19:00.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.375+0000 7f3ad6572700 1 -- 172.21.15.40:0/753754267 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ad00c7f60 msgr2=0x7f3ad00c17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:00.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.375+0000 7f3ad6572700 1 --2- 172.21.15.40:0/753754267 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ad00c7f60 0x7f3ad00c17e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000ffd0 tx=0x7f3ac0014850 comp rx=0 tx=0).stop 2024-09-13T18:19:00.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.379+0000 7f3ad6572700 1 -- 172.21.15.40:0/753754267 shutdown_connections 2024-09-13T18:19:00.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.379+0000 7f3ad6572700 1 --2- 172.21.15.40:0/753754267 >> v2:172.21.15.43:3300/0 conn(0x7f3ad00c8340 0x7f3ad00bff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:00.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.379+0000 7f3ad6572700 1 --2- 172.21.15.40:0/753754267 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ad00c7f60 0x7f3ad00c17e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:00.387 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.379+0000 7f3ad6572700 1 -- 172.21.15.40:0/753754267 >> 172.21.15.40:0/753754267 conn(0x7f3ad0066090 msgr2=0x7f3ad00664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:00.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.395+0000 7f3ad6572700 1 -- 172.21.15.40:0/753754267 shutdown_connections 2024-09-13T18:19:00.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.395+0000 7f3ad6572700 1 -- 172.21.15.40:0/753754267 wait complete. 2024-09-13T18:19:00.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.403+0000 7f3ad6572700 1 Processor -- start 2024-09-13T18:19:00.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.403+0000 7f3ad6572700 1 -- start start 2024-09-13T18:19:00.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.403+0000 7f3ad6572700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ad00b00b0 0x7f3ad00aaa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:00.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.403+0000 7f3ad6572700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ad00c7f60 0x7f3ad0070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:00.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.403+0000 7f3ad6572700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ad00c8340 0x7f3ad01fea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:00.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.403+0000 7f3ad5570700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ad00b00b0 0x7f3ad00aaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:00.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.403+0000 7f3ad5570700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ad00b00b0 0x7f3ad00aaa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:53610/0 (socket says 172.21.15.40:53610) 2024-09-13T18:19:00.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.403+0000 7f3ad5570700 1 -- 172.21.15.40:0/2183584530 learned_addr learned my addr 172.21.15.40:0/2183584530 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:19:00.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.403+0000 7f3ad4d6f700 1 --2- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ad00c7f60 0x7f3ad0070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:00.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.403+0000 7f3ad5d71700 1 --2- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ad00c8340 0x7f3ad01fea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:00.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.407+0000 7f3ad6572700 1 -- 172.21.15.40:0/2183584530 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad01fbe60 con 0x7f3ad00b00b0 2024-09-13T18:19:00.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.407+0000 7f3ad6572700 1 -- 172.21.15.40:0/2183584530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad007b300 con 0x7f3ad00c8340 2024-09-13T18:19:00.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.407+0000 7f3ad6572700 1 -- 172.21.15.40:0/2183584530 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad00aa0d0 con 0x7f3ad00c7f60 2024-09-13T18:19:00.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.407+0000 7f3ad5d71700 1 -- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ad00c7f60 msgr2=0x7f3ad0070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:00.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.407+0000 7f3ad5d71700 1 --2- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ad00c7f60 0x7f3ad0070700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:00.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.407+0000 7f3ad5d71700 1 -- 172.21.15.40:0/2183584530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ad00b00b0 msgr2=0x7f3ad00aaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:00.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.407+0000 7f3ad5d71700 1 --2- 172.21.15.40:0/2183584530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ad00b00b0 0x7f3ad00aaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:00.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.407+0000 7f3ad5d71700 1 -- 172.21.15.40:0/2183584530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac000fdb0 con 0x7f3ad00c8340 2024-09-13T18:19:00.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.407+0000 7f3ad5d71700 1 --2- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ad00c8340 0x7f3ad01fea00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800bdd0 tx=0x7f3ac8009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:00.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.407+0000 7f3ac5ffb700 1 -- 172.21.15.40:0/2183584530 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac800cc10 con 0x7f3ad00c8340 2024-09-13T18:19:00.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.407+0000 7f3ac5ffb700 1 -- 172.21.15.40:0/2183584530 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac800cd70 con 0x7f3ad00c8340 2024-09-13T18:19:00.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.407+0000 7f3ad6572700 1 -- 172.21.15.40:0/2183584530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad00ab0c0 con 0x7f3ad00c8340 2024-09-13T18:19:00.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.407+0000 7f3ac5ffb700 1 -- 172.21.15.40:0/2183584530 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8014680 con 0x7f3ad00c8340 2024-09-13T18:19:00.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.407+0000 7f3ad6572700 1 -- 172.21.15.40:0/2183584530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad01fb7a0 con 0x7f3ad00c8340 2024-09-13T18:19:00.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.411+0000 7f3ad6572700 1 --2- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ad01fc180 0x7f3ad016b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:00.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.411+0000 7f3ac5ffb700 1 -- 172.21.15.40:0/2183584530 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3ac801c360 con 0x7f3ad00c8340 2024-09-13T18:19:00.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.411+0000 7f3ad5570700 1 --2- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ad01fc180 0x7f3ad016b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:00.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.411+0000 7f3ac5ffb700 1 --2- 172.21.15.40:0/2183584530 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3ab40667b0 0x7f3ab4068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:00.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.411+0000 7f3ac5ffb700 1 -- 172.21.15.40:0/2183584530 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3ac8083dd0 con 0x7f3ad00c8340 2024-09-13T18:19:00.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.411+0000 7f3ad4d6f700 1 --2- 172.21.15.40:0/2183584530 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3ab40667b0 0x7f3ab4068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:00.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.411+0000 7f3ad6572700 1 -- 172.21.15.40:0/2183584530 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad0067720 con 0x7f3ad01fc180 2024-09-13T18:19:00.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.411+0000 7f3ad5570700 1 -- 172.21.15.40:0/2183584530 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3ad00bfe80 con 0x7f3ad01fc180 2024-09-13T18:19:00.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.411+0000 7f3ad4d6f700 1 --2- 172.21.15.40:0/2183584530 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3ab40667b0 0x7f3ab4068c70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0070f70 tx=0x7f3acc0090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:00.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.411+0000 7f3ad5570700 1 --2- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ad01fc180 0x7f3ad016b190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0000c00 tx=0x7f3ac0004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:00.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.415+0000 7f3ac5ffb700 1 -- 172.21.15.40:0/2183584530 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0030420 con 0x7f3ad01fc180 2024-09-13T18:19:00.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.415+0000 7f3ac5ffb700 1 -- 172.21.15.40:0/2183584530 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f3ac0030580 con 0x7f3ad01fc180 2024-09-13T18:19:00.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.415+0000 7f3ac5ffb700 1 -- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ad01fc180 msgr2=0x7f3ad016b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:00.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.415+0000 7f3ac5ffb700 1 --2- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ad01fc180 0x7f3ad016b190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0000c00 tx=0x7f3ac0004770 comp rx=0 tx=0).stop 2024-09-13T18:19:00.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.415+0000 7f3ac5ffb700 1 -- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ad01fc180 msgr2=0x7f3ad016b190 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:19:00.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.415+0000 7f3ac5ffb700 1 --2- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3ad01fc180 0x7f3ad016b190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:00.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.427+0000 7f3abb7fe700 1 --2- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3aa80028f0 0x7f3aa8004d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:00.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.427+0000 7f3abb7fe700 1 -- 172.21.15.40:0/2183584530 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa8007dd0 con 0x7f3aa80028f0 2024-09-13T18:19:00.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.427+0000 7f3ad5570700 1 --2- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3aa80028f0 0x7f3aa8004d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:00.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.427+0000 7f3ad5570700 1 -- 172.21.15.40:0/2183584530 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f3aa8007f90 con 0x7f3aa80028f0 2024-09-13T18:19:00.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.427+0000 7f3ad5570700 1 --2- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3aa80028f0 0x7f3aa8004d90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000fad0 tx=0x7f3ac0012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:00.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.431+0000 7f3ac5ffb700 1 -- 172.21.15.40:0/2183584530 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0030d90 con 0x7f3aa80028f0 2024-09-13T18:19:00.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.431+0000 7f3ac5ffb700 1 -- 172.21.15.40:0/2183584530 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4783 (secure 0 0 0) 0x7f3ac0039410 con 0x7f3aa80028f0 2024-09-13T18:19:00.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.431+0000 7f3ac5ffb700 1 -- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3aa80028f0 msgr2=0x7f3aa8004d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:00.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.431+0000 7f3ac5ffb700 1 --2- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3aa80028f0 0x7f3aa8004d90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000fad0 tx=0x7f3ac0012000 comp rx=0 tx=0).stop 2024-09-13T18:19:00.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.431+0000 7f3ac5ffb700 1 -- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3aa80028f0 msgr2=0x7f3aa8004d90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:19:00.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.431+0000 7f3ac5ffb700 1 --2- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3aa80028f0 0x7f3aa8004d90 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000fad0 tx=0x7f3ac0012000 comp rx=0 tx=0).stop 2024-09-13T18:19:00.439 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:19:00.439 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:19:00.439 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:19:00.439 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:19:00.439 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 224, 2024-09-13T18:19:00.439 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:19:00.439 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:19:00.439 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:19:00.439 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:19:00.440 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:00.440 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 16, 2024-09-13T18:19:00.440 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:19:00.440 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:19:00.440 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:19:00.440 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:19:00.440 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:19:00.440 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:19:00.440 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:19:00.440 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:19:00.440 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:19:00.440 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:19:00.440 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:19:00.441 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:19:00.441 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:19:00.441 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:00.441 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:19:00.441 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:19:00.441 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:19:00.441 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:19:00.441 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:19:00.441 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:19:00.441 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:19:00.441 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:19:00.441 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:19:00.441 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:19:00.441 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:19:00.442 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:19:00.442 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:00.442 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.442 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:19:00.442 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:19:00.442 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:19:00.442 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:19:00.442 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:19:00.442 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:19:00.442 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:19:00.442 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:19:00.442 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:19:00.442 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:19:00.442 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:19:00.442 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:19:00.442 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:19:00.443 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:19:00.443 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:19:00.443 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:19:00.443 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:19:00.443 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:19:00.443 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:19:00.443 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:19:00.443 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:19:00.443 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:19:00.443 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:19:00.443 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:19:00.443 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:19:00.443 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:19:00.443 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:19:00.444 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:00.444 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:19:00.444 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.444 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:19:00.444 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.444 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:19:00.444 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:19:00.444 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:19:00.444 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:19:00.444 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.444 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:19:00.444 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:19:00.444 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:00.444 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.445 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.445 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:19:00.445 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:19:00.445 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:00.445 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:00.445 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:00.445 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.445 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:19:00.445 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:19:00.445 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:19:00.446 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:19:00.446 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:19:00.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.446 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.446 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:19:00.446 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:19:00.446 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:19:00.446 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:19:00.446 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.446 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:19:00.446 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:19:00.446 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:00.446 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.447 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.447 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:19:00.447 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:19:00.447 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:00.447 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:00.447 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:00.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.447 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:19:00.447 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:19:00.447 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:19:00.447 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:19:00.447 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:19:00.447 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.447 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.448 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:19:00.448 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:19:00.448 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:19:00.448 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:19:00.448 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.448 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:19:00.448 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:19:00.448 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:00.448 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.448 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.448 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:19:00.448 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:19:00.448 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:00.449 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:00.449 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:00.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.449 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:19:00.449 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:19:00.449 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:19:00.449 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:19:00.449 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:19:00.449 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:00.449 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:00.449 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.450 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:19:00.450 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:19:00.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.450 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:19:00.450 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:19:00.450 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:19:00.450 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:00.450 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:00.450 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:19:00.450 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.450 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:19:00.450 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:19:00.450 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:19:00.450 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:00.451 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:00.451 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:19:00.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.451 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:19:00.451 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:19:00.451 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:19:00.451 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:00.451 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:00.451 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:19:00.451 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:00.451 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:19:00.451 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:19:00.451 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:19:00.451 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:00.452 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:00.452 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:00.452 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:19:00.452 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:19:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.431+0000 7f3abb7fe700 1 -- 172.21.15.40:0/2183584530 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3ab40667b0 msgr2=0x7f3ab4068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.431+0000 7f3abb7fe700 1 --2- 172.21.15.40:0/2183584530 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3ab40667b0 0x7f3ab4068c70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0070f70 tx=0x7f3acc0090e0 comp rx=0 tx=0).stop 2024-09-13T18:19:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.431+0000 7f3abb7fe700 1 -- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ad00c8340 msgr2=0x7f3ad01fea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.431+0000 7f3abb7fe700 1 --2- 172.21.15.40:0/2183584530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ad00c8340 0x7f3ad01fea00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800bdd0 tx=0x7f3ac8009ed0 comp rx=0 tx=0).stop 2024-09-13T18:19:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.431+0000 7f3ad5d71700 1 -- 172.21.15.40:0/2183584530 reap_dead start 2024-09-13T18:19:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.431+0000 7f3ad5d71700 1 -- 172.21.15.40:0/2183584530 reap_dead start 2024-09-13T18:19:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.431+0000 7f3abb7fe700 1 -- 172.21.15.40:0/2183584530 shutdown_connections 2024-09-13T18:19:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.431+0000 7f3abb7fe700 1 -- 172.21.15.40:0/2183584530 >> 172.21.15.40:0/2183584530 conn(0x7f3ad0066090 msgr2=0x7f3ad00ade10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.431+0000 7f3abb7fe700 1 -- 172.21.15.40:0/2183584530 shutdown_connections 2024-09-13T18:19:00.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:00.431+0000 7f3abb7fe700 1 -- 172.21.15.40:0/2183584530 wait complete. 2024-09-13T18:19:00.476 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:19:00.477 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-13T18:19:00.477 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['c'] 2024-09-13T18:19:00.477 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-13T18:19:00.477 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T18:19:00.477 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T18:19:00.477 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:19:00.542 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T18:19:00.543 INFO:tasks.mon_thrash:Sending STOP to mon c 2024-09-13T18:19:00.543 INFO:tasks.ceph.mon.c:Sent signal 19 2024-09-13T18:19:00.543 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-13T18:19:15.544 INFO:tasks.mon_thrash:Sending CONT to mon c 2024-09-13T18:19:15.545 INFO:tasks.ceph.mon.c:Sent signal 18 2024-09-13T18:19:15.545 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T18:19:15.546 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:19:16.252 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.243+0000 7f136619f700 1 Processor -- start 2024-09-13T18:19:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.243+0000 7f136619f700 1 -- start start 2024-09-13T18:19:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.247+0000 7f136619f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 0x7f135804b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.247+0000 7f136619f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1358006510 con 0x7f13580de320 2024-09-13T18:19:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.247+0000 7f136619f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f135804bc40 con 0x7f1358045620 2024-09-13T18:19:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.247+0000 7f136619f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f135804b2d0 con 0x7f13580ddf40 2024-09-13T18:19:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.247+0000 7f136499c700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7f13580de320 legacy=0x7f1358044f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:19:16.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.247+0000 7f136599e700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1358045620 0x7f13580d4430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:43694/0 (socket says 172.21.15.40:43694) 2024-09-13T18:19:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.247+0000 7f136599e700 1 -- 172.21.15.40:0/3295713240 learned_addr learned my addr 172.21.15.40:0/3295713240 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:19:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.251+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 0x7f135804b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.251+0000 7f136519d700 1 -- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 msgr2=0x7f135804b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:19:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.251+0000 7f136519d700 1 -- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 msgr2=0x7f135804b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:19:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.251+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 0x7f135804b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:19:16.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.251+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 0x7f135804b700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:19:16.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.455+0000 7f136499c700 1 -- 172.21.15.40:0/3295713240 >> v1:172.21.15.40:6789/0 conn(0x7f13580de320 legacy=0x7f1358044f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:19:16.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.455+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 0x7f135804b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:16.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.455+0000 7f136519d700 1 -- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 msgr2=0x7f135804b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:19:16.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.455+0000 7f136519d700 1 -- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 msgr2=0x7f135804b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:19:16.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.455+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 0x7f135804b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:19:16.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.455+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 0x7f135804b700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:19:16.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.859+0000 7f136499c700 1 -- 172.21.15.40:0/3295713240 >> v1:172.21.15.40:6789/0 conn(0x7f13580de320 legacy=0x7f1358044f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:19:16.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.859+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 0x7f135804b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:16.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.863+0000 7f136519d700 1 -- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 msgr2=0x7f135804b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:19:16.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.863+0000 7f136519d700 1 -- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 msgr2=0x7f135804b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:19:16.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.863+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 0x7f135804b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:19:16.869 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:16.863+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 0x7f135804b700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:19:17.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:17.663+0000 7f136499c700 1 -- 172.21.15.40:0/3295713240 >> v1:172.21.15.40:6789/0 conn(0x7f13580de320 legacy=0x7f1358044f10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:19:17.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:17.667+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 0x7f135804b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:17.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:17.671+0000 7f136519d700 1 -- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 msgr2=0x7f135804b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:19:17.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:17.671+0000 7f136519d700 1 -- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 msgr2=0x7f135804b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:19:17.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:17.671+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 0x7f135804b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:19:17.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:17.671+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 0x7f135804b700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:19:19.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.247+0000 7f1356ffd700 1 -- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 msgr2=0x7f135804b700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:19:19.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.247+0000 7f1356ffd700 1 --2- 172.21.15.40:0/3295713240 >> v2:172.21.15.43:3300/0 conn(0x7f13580ddf40 0x7f135804b700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:19.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.247+0000 7f1356ffd700 1 -- 172.21.15.40:0/3295713240 >> v1:172.21.15.43:6789/0 conn(0x7f1358045620 legacy=0x7f13580d4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:19.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.247+0000 7f1356ffd700 1 -- 172.21.15.40:0/3295713240 >> v1:172.21.15.40:6789/0 conn(0x7f13580de320 legacy=0x7f1358044f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:19:19.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.247+0000 7f1356ffd700 1 --2- 172.21.15.40:0/3295713240 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1344003740 0x7f1344005b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:19.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.247+0000 7f1356ffd700 1 --2- 172.21.15.40:0/3295713240 >> v2:172.21.15.40:3300/0 conn(0x7f1344006200 0x7f13440086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:19.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.247+0000 7f1356ffd700 1 -- 172.21.15.40:0/3295713240 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f135804b2d0 con 0x7f1344006200 2024-09-13T18:19:19.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.247+0000 7f1356ffd700 1 -- 172.21.15.40:0/3295713240 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1344008d50 con 0x7f1344003740 2024-09-13T18:19:19.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.247+0000 7f136499c700 1 -- 172.21.15.40:0/3295713240 >> v2:172.21.15.40:3300/0 conn(0x7f1344006200 msgr2=0x7f13440086a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:19:19.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.247+0000 7f136499c700 1 --2- 172.21.15.40:0/3295713240 >> v2:172.21.15.40:3300/0 conn(0x7f1344006200 0x7f13440086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:19:19.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.247+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1344003740 0x7f1344005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:19.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.247+0000 7f136519d700 1 -- 172.21.15.40:0/3295713240 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1344003740 msgr2=0x7f1344005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:19:19.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.247+0000 7f136519d700 1 -- 172.21.15.40:0/3295713240 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1344003740 msgr2=0x7f1344005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:19:19.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.247+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1344003740 0x7f1344005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:19:19.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.247+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1344003740 0x7f1344005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:19:19.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.451+0000 7f136499c700 1 -- 172.21.15.40:0/3295713240 >> v2:172.21.15.40:3300/0 conn(0x7f1344006200 msgr2=0x7f13440086a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:19:19.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.451+0000 7f136499c700 1 --2- 172.21.15.40:0/3295713240 >> v2:172.21.15.40:3300/0 conn(0x7f1344006200 0x7f13440086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:19:19.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.451+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1344003740 0x7f1344005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:19.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.451+0000 7f136519d700 1 -- 172.21.15.40:0/3295713240 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1344003740 msgr2=0x7f1344005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:19:19.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.451+0000 7f136519d700 1 -- 172.21.15.40:0/3295713240 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1344003740 msgr2=0x7f1344005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:19:19.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.451+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1344003740 0x7f1344005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:19:19.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.451+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1344003740 0x7f1344005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:19:19.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.859+0000 7f136499c700 1 -- 172.21.15.40:0/3295713240 >> v2:172.21.15.40:3300/0 conn(0x7f1344006200 msgr2=0x7f13440086a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:19:19.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.859+0000 7f136499c700 1 --2- 172.21.15.40:0/3295713240 >> v2:172.21.15.40:3300/0 conn(0x7f1344006200 0x7f13440086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:19:19.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.859+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1344003740 0x7f1344005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:19.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.859+0000 7f136519d700 1 -- 172.21.15.40:0/3295713240 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1344003740 msgr2=0x7f1344005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:19:19.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.859+0000 7f136519d700 1 -- 172.21.15.40:0/3295713240 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1344003740 msgr2=0x7f1344005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:19:19.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.859+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1344003740 0x7f1344005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:19:19.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:19.859+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1344003740 0x7f1344005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:19:20.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.663+0000 7f136499c700 1 -- 172.21.15.40:0/3295713240 >> v2:172.21.15.40:3300/0 conn(0x7f1344006200 msgr2=0x7f13440086a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:19:20.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.663+0000 7f136499c700 1 --2- 172.21.15.40:0/3295713240 >> v2:172.21.15.40:3300/0 conn(0x7f1344006200 0x7f13440086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:19:20.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.663+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1344003740 0x7f1344005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.663+0000 7f136519d700 1 -- 172.21.15.40:0/3295713240 >> v2:172.21.15.40:3300/0 conn(0x7f1344006200 msgr2=0x7f13440086a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:19:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.663+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> v2:172.21.15.40:3300/0 conn(0x7f1344006200 0x7f13440086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.663+0000 7f136519d700 1 -- 172.21.15.40:0/3295713240 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1358006760 con 0x7f1344003740 2024-09-13T18:19:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.663+0000 7f136519d700 1 --2- 172.21.15.40:0/3295713240 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1344003740 0x7f1344005b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f135c009100 tx=0x7f135c003b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdd1a9f228960e17 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.663+0000 7f1357fff700 1 -- 172.21.15.40:0/3295713240 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c01c070 con 0x7f1344003740 2024-09-13T18:19:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.663+0000 7f1357fff700 1 -- 172.21.15.40:0/3295713240 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135c0170d0 con 0x7f1344003740 2024-09-13T18:19:20.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.663+0000 7f1357fff700 1 -- 172.21.15.40:0/3295713240 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c017440 con 0x7f1344003740 2024-09-13T18:19:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.663+0000 7f136619f700 1 -- 172.21.15.40:0/3295713240 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1344003740 msgr2=0x7f1344005b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.663+0000 7f136619f700 1 --2- 172.21.15.40:0/3295713240 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1344003740 0x7f1344005b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f135c009100 tx=0x7f135c003b30 comp rx=0 tx=0).stop 2024-09-13T18:19:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.663+0000 7f136599e700 1 -- 172.21.15.40:0/3295713240 reap_dead start 2024-09-13T18:19:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.663+0000 7f136619f700 1 -- 172.21.15.40:0/3295713240 shutdown_connections 2024-09-13T18:19:20.672 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.663+0000 7f136619f700 1 -- 172.21.15.40:0/3295713240 >> 172.21.15.40:0/3295713240 conn(0x7f135804fd00 msgr2=0x7f1358050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.663+0000 7f136619f700 1 -- 172.21.15.40:0/3295713240 shutdown_connections 2024-09-13T18:19:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.663+0000 7f136619f700 1 -- 172.21.15.40:0/3295713240 wait complete. 2024-09-13T18:19:20.673 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.667+0000 7f136619f700 1 Processor -- start 2024-09-13T18:19:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.667+0000 7f136619f700 1 -- start start 2024-09-13T18:19:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.667+0000 7f136619f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1344003740 0x7f13580dd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.667+0000 7f136619f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358047870 0x7f13580dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:20.675 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.667+0000 7f136619f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f13580cbce0 0x7f13580cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.667+0000 7f136619f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13580476f0 con 0x7f1358047870 2024-09-13T18:19:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.667+0000 7f136619f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13580cd0d0 con 0x7f1344003740 2024-09-13T18:19:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.667+0000 7f136619f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13580ccb50 con 0x7f13580cbce0 2024-09-13T18:19:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.667+0000 7f136519d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1344003740 0x7f13580dd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.667+0000 7f136499c700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358047870 msgr2=0x7f13580dbf80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:19:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.667+0000 7f136519d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1344003740 0x7f13580dd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:42226/0 (socket says 172.21.15.40:42226) 2024-09-13T18:19:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.667+0000 7f136519d700 1 -- 172.21.15.40:0/4282431062 learned_addr learned my addr 172.21.15.40:0/4282431062 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:19:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.667+0000 7f136499c700 1 --2- 172.21.15.40:0/4282431062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358047870 0x7f13580dbf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:19:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.667+0000 7f136599e700 1 --2- 172.21.15.40:0/4282431062 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f13580cbce0 0x7f13580cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.667+0000 7f136519d700 1 -- 172.21.15.40:0/4282431062 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f13580cbce0 msgr2=0x7f13580cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.667+0000 7f136519d700 1 --2- 172.21.15.40:0/4282431062 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f13580cbce0 0x7f13580cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:20.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.667+0000 7f136519d700 1 -- 172.21.15.40:0/4282431062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358047870 msgr2=0x7f13580dbf80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:19:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.667+0000 7f136519d700 1 --2- 172.21.15.40:0/4282431062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358047870 0x7f13580dbf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.667+0000 7f136519d700 1 -- 172.21.15.40:0/4282431062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135c004560 con 0x7f1344003740 2024-09-13T18:19:20.677 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.667+0000 7f136519d700 1 --2- 172.21.15.40:0/4282431062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1344003740 0x7f13580dd5c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f135c003b70 tx=0x7f135c0047e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:20.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.675+0000 7f1355ffb700 1 -- 172.21.15.40:0/4282431062 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c014670 con 0x7f1344003740 2024-09-13T18:19:20.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.675+0000 7f1355ffb700 1 -- 172.21.15.40:0/4282431062 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135c0147d0 con 0x7f1344003740 2024-09-13T18:19:20.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.675+0000 7f136619f700 1 -- 172.21.15.40:0/4282431062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1358047cd0 con 0x7f1344003740 2024-09-13T18:19:20.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.675+0000 7f136619f700 1 -- 172.21.15.40:0/4282431062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13580ccdf0 con 0x7f1344003740 2024-09-13T18:19:20.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.679+0000 7f1355ffb700 1 -- 172.21.15.40:0/4282431062 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c014b80 con 0x7f1344003740 2024-09-13T18:19:20.687 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.679+0000 7f1355ffb700 1 -- 172.21.15.40:0/4282431062 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f135c010040 con 0x7f1344003740 2024-09-13T18:19:20.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.679+0000 7f1355ffb700 1 --2- 172.21.15.40:0/4282431062 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1344067580 0x7f1344069a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:20.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.679+0000 7f1355ffb700 1 -- 172.21.15.40:0/4282431062 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f135c086e30 con 0x7f1344003740 2024-09-13T18:19:20.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.679+0000 7f136499c700 1 --2- 172.21.15.40:0/4282431062 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1344067580 0x7f1344069a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:20.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.679+0000 7f136619f700 1 -- 172.21.15.40:0/4282431062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f135804dae0 con 0x7f1344003740 2024-09-13T18:19:20.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.679+0000 7f136499c700 1 --2- 172.21.15.40:0/4282431062 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1344067580 0x7f1344069a40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f136004ffd0 tx=0x7f1360068640 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:20.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:20.687+0000 7f1355ffb700 1 -- 172.21.15.40:0/4282431062 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f135c053f70 con 0x7f1344003740 2024-09-13T18:19:21.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:21.851+0000 7f136619f700 1 -- 172.21.15.40:0/4282431062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1358050820 con 0x7f1344003740 2024-09-13T18:19:21.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:21.851+0000 7f1355ffb700 1 -- 172.21.15.40:0/4282431062 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f135c057590 con 0x7f1344003740 2024-09-13T18:19:21.863 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":226,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:19:21.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:21.879+0000 7f134b7fe700 1 -- 172.21.15.40:0/4282431062 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1344067580 msgr2=0x7f1344069a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:21.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:21.879+0000 7f134b7fe700 1 --2- 172.21.15.40:0/4282431062 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1344067580 0x7f1344069a40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f136004ffd0 tx=0x7f1360068640 comp rx=0 tx=0).stop 2024-09-13T18:19:21.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:21.879+0000 7f134b7fe700 1 -- 172.21.15.40:0/4282431062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1344003740 msgr2=0x7f13580dd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:21.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:21.879+0000 7f134b7fe700 1 --2- 172.21.15.40:0/4282431062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1344003740 0x7f13580dd5c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f135c003b70 tx=0x7f135c0047e0 comp rx=0 tx=0).stop 2024-09-13T18:19:21.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:21.883+0000 7f134b7fe700 1 -- 172.21.15.40:0/4282431062 shutdown_connections 2024-09-13T18:19:21.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:21.883+0000 7f134b7fe700 1 --2- 172.21.15.40:0/4282431062 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1344067580 0x7f1344069a40 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:21.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:21.883+0000 7f134b7fe700 1 --2- 172.21.15.40:0/4282431062 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f13580cbce0 0x7f13580cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:21.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:21.883+0000 7f134b7fe700 1 --2- 172.21.15.40:0/4282431062 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1358047870 0x7f13580dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:21.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:21.883+0000 7f134b7fe700 1 --2- 172.21.15.40:0/4282431062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1344003740 0x7f13580dd5c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:21.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:21.883+0000 7f134b7fe700 1 -- 172.21.15.40:0/4282431062 >> 172.21.15.40:0/4282431062 conn(0x7f135804fd00 msgr2=0x7f135804ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:21.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:21.883+0000 7f134b7fe700 1 -- 172.21.15.40:0/4282431062 shutdown_connections 2024-09-13T18:19:21.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:21.887+0000 7f134b7fe700 1 -- 172.21.15.40:0/4282431062 wait complete. 2024-09-13T18:19:21.930 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T18:19:21.930 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-13T18:19:21.930 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:19:22.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.355+0000 7f0fad443700 1 Processor -- start 2024-09-13T18:19:22.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.355+0000 7f0fad443700 1 -- start start 2024-09-13T18:19:22.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.355+0000 7f0fad443700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0fa80c7320 0x7f0fa806d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:22.366 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.355+0000 7f0fad443700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0fa80c6390 0x7f0fa806d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:22.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.355+0000 7f0fad443700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fa8072650 con 0x7f0fa8072270 2024-09-13T18:19:22.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.355+0000 7f0fad443700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fa80c5f00 con 0x7f0fa80c6390 2024-09-13T18:19:22.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.355+0000 7f0fad443700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fa80c7130 con 0x7f0fa80c7320 2024-09-13T18:19:22.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.355+0000 7f0fa77fe700 1 -- >> v2:172.21.15.40:3300/0 conn(0x7f0fa80c6390 msgr2=0x7f0fa806d810 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:19:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.355+0000 7f0fa77fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0fa80c6390 0x7f0fa806d810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:19:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.355+0000 7f0fa67fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0fa80c7320 0x7f0fa806d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.355+0000 7f0fa67fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0fa80c7320 0x7f0fa806d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:53344/0 (socket says 172.21.15.40:53344) 2024-09-13T18:19:22.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.355+0000 7f0fa67fc700 1 -- 172.21.15.40:0/2643136271 learned_addr learned my addr 172.21.15.40:0/2643136271 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:19:22.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.355+0000 7f0fa67fc700 1 -- 172.21.15.40:0/2643136271 >> v2:172.21.15.40:3300/0 conn(0x7f0fa80c6390 msgr2=0x7f0fa806d810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:19:22.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.355+0000 7f0fa67fc700 1 --2- 172.21.15.40:0/2643136271 >> v2:172.21.15.40:3300/0 conn(0x7f0fa80c6390 0x7f0fa806d810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:22.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.355+0000 7f0fa67fc700 1 -- 172.21.15.40:0/2643136271 >> v1:172.21.15.43:6789/0 conn(0x7f0fa8072270 legacy=0x7f0fa8159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:19:22.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.355+0000 7f0fa67fc700 1 -- 172.21.15.40:0/2643136271 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa80c8450 con 0x7f0fa80c7320 2024-09-13T18:19:22.369 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.355+0000 7f0fa67fc700 1 --2- 172.21.15.40:0/2643136271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fa80c7320 0x7f0fa806d2d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0fa806b640 tx=0x7f0f98009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f779721aaea32e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:22.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.359+0000 7f0fa5ffb700 1 -- 172.21.15.40:0/2643136271 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98004230 con 0x7f0fa80c7320 2024-09-13T18:19:22.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.359+0000 7f0fa5ffb700 1 -- 172.21.15.40:0/2643136271 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f98004390 con 0x7f0fa80c7320 2024-09-13T18:19:22.370 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.359+0000 7f0fad443700 1 -- 172.21.15.40:0/2643136271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fa80c7320 msgr2=0x7f0fa806d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:22.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.359+0000 7f0fad443700 1 --2- 172.21.15.40:0/2643136271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fa80c7320 0x7f0fa806d2d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0fa806b640 tx=0x7f0f98009bf0 comp rx=0 tx=0).stop 2024-09-13T18:19:22.371 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.363+0000 7f0fad443700 1 -- 172.21.15.40:0/2643136271 shutdown_connections 2024-09-13T18:19:22.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.363+0000 7f0fad443700 1 --2- 172.21.15.40:0/2643136271 >> v2:172.21.15.40:3300/0 conn(0x7f0fa80c6390 0x7f0fa806d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:22.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.363+0000 7f0fad443700 1 --2- 172.21.15.40:0/2643136271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fa80c7320 0x7f0fa806d2d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:22.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.363+0000 7f0fad443700 1 -- 172.21.15.40:0/2643136271 >> 172.21.15.40:0/2643136271 conn(0x7f0fa8066090 msgr2=0x7f0fa80664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:22.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.403+0000 7f0fad443700 1 -- 172.21.15.40:0/2643136271 shutdown_connections 2024-09-13T18:19:22.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.403+0000 7f0fad443700 1 -- 172.21.15.40:0/2643136271 wait complete. 2024-09-13T18:19:22.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.403+0000 7f0fad443700 1 Processor -- start 2024-09-13T18:19:22.413 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.403+0000 7f0fad443700 1 -- start start 2024-09-13T18:19:22.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.407+0000 7f0fad443700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0fa8072270 0x7f0fa8152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:22.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.407+0000 7f0fad443700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa80c6390 0x7f0fa814cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:22.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.407+0000 7f0fad443700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fa80c7320 0x7f0fa8153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:22.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.407+0000 7f0fad443700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa8152da0 con 0x7f0fa80c6390 2024-09-13T18:19:22.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.407+0000 7f0fa67fc700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa80c6390 msgr2=0x7f0fa814cac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:19:22.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.407+0000 7f0fad443700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa8151ca0 con 0x7f0fa80c7320 2024-09-13T18:19:22.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.407+0000 7f0fad443700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa81f35e0 con 0x7f0fa8072270 2024-09-13T18:19:22.414 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.407+0000 7f0fa67fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa80c6390 0x7f0fa814cac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:19:22.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.411+0000 7f0fa6ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0fa8072270 0x7f0fa8152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:22.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.411+0000 7f0fa6ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0fa8072270 0x7f0fa8152860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:41266/0 (socket says 172.21.15.40:41266) 2024-09-13T18:19:22.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.411+0000 7f0fa77fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fa80c7320 0x7f0fa8153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:22.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.411+0000 7f0fa77fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fa80c7320 0x7f0fa8153100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:53360/0 (socket says 172.21.15.40:53360) 2024-09-13T18:19:22.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.411+0000 7f0fa77fe700 1 -- 172.21.15.40:0/2605035295 learned_addr learned my addr 172.21.15.40:0/2605035295 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:19:22.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.415+0000 7f0fa77fe700 1 -- 172.21.15.40:0/2605035295 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0fa8072270 msgr2=0x7f0fa8152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:22.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.415+0000 7f0fa77fe700 1 --2- 172.21.15.40:0/2605035295 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0fa8072270 0x7f0fa8152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:22.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.415+0000 7f0fa77fe700 1 -- 172.21.15.40:0/2605035295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa80c6390 msgr2=0x7f0fa814cac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:19:22.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.415+0000 7f0fa77fe700 1 --2- 172.21.15.40:0/2605035295 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fa80c6390 0x7f0fa814cac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:22.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.415+0000 7f0fa77fe700 1 -- 172.21.15.40:0/2605035295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f98009570 con 0x7f0fa80c7320 2024-09-13T18:19:22.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.415+0000 7f0fa77fe700 1 --2- 172.21.15.40:0/2605035295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fa80c7320 0x7f0fa8153100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00b480 tx=0x7f0f9c00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:22.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.419+0000 7f0f8f7fe700 1 -- 172.21.15.40:0/2605035295 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c003e70 con 0x7f0fa80c7320 2024-09-13T18:19:22.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.419+0000 7f0fad443700 1 -- 172.21.15.40:0/2605035295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa81f38c0 con 0x7f0fa80c7320 2024-09-13T18:19:22.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.419+0000 7f0fad443700 1 -- 172.21.15.40:0/2605035295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa80cb4d0 con 0x7f0fa80c7320 2024-09-13T18:19:22.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.419+0000 7f0f8f7fe700 1 -- 172.21.15.40:0/2605035295 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9c0060d0 con 0x7f0fa80c7320 2024-09-13T18:19:22.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.419+0000 7f0f8f7fe700 1 -- 172.21.15.40:0/2605035295 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c006460 con 0x7f0fa80c7320 2024-09-13T18:19:22.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.423+0000 7f0f8f7fe700 1 -- 172.21.15.40:0/2605035295 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0f9c0065c0 con 0x7f0fa80c7320 2024-09-13T18:19:22.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.427+0000 7f0f8f7fe700 1 --2- 172.21.15.40:0/2605035295 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0f94066800 0x7f0f94068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:22.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.427+0000 7f0f8f7fe700 1 -- 172.21.15.40:0/2605035295 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0f9c083890 con 0x7f0fa80c7320 2024-09-13T18:19:22.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.427+0000 7f0fa6ffd700 1 --2- 172.21.15.40:0/2605035295 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0f94066800 0x7f0f94068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:22.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.427+0000 7f0fad443700 1 --2- 172.21.15.40:0/2605035295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f88002a40 0x7f0f88004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:22.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.427+0000 7f0fad443700 1 -- 172.21.15.40:0/2605035295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f88007f40 con 0x7f0f88002a40 2024-09-13T18:19:22.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.427+0000 7f0fa67fc700 1 --2- 172.21.15.40:0/2605035295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f88002a40 0x7f0f88004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:22.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.427+0000 7f0fa67fc700 1 -- 172.21.15.40:0/2605035295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f88008100 con 0x7f0f88002a40 2024-09-13T18:19:22.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.431+0000 7f0fa67fc700 1 --2- 172.21.15.40:0/2605035295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f88002a40 0x7f0f88004f00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0f98004700 tx=0x7f0f98004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:22.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.431+0000 7f0f8f7fe700 1 -- 172.21.15.40:0/2605035295 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98003e60 con 0x7f0f88002a40 2024-09-13T18:19:22.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.431+0000 7f0f8f7fe700 1 -- 172.21.15.40:0/2605035295 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f0f9800b660 con 0x7f0f88002a40 2024-09-13T18:19:22.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.431+0000 7f0f8f7fe700 1 -- 172.21.15.40:0/2605035295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f88002a40 msgr2=0x7f0f88004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:22.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.431+0000 7f0f8f7fe700 1 --2- 172.21.15.40:0/2605035295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f88002a40 0x7f0f88004f00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0f98004700 tx=0x7f0f98004770 comp rx=0 tx=0).stop 2024-09-13T18:19:22.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.431+0000 7f0f8f7fe700 1 -- 172.21.15.40:0/2605035295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f88002a40 msgr2=0x7f0f88004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:19:22.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.431+0000 7f0f8f7fe700 1 --2- 172.21.15.40:0/2605035295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f88002a40 0x7f0f88004f00 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0f98004700 tx=0x7f0f98004770 comp rx=0 tx=0).stop 2024-09-13T18:19:22.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.435+0000 7f0fa6ffd700 1 --2- 172.21.15.40:0/2605035295 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0f94066800 0x7f0f94068cc0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0001ea0 tx=0x7f0fa0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.447+0000 7f0f8d7fa700 1 --2- 172.21.15.40:0/2605035295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f800028f0 0x7f0f80004d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.447+0000 7f0f8d7fa700 1 -- 172.21.15.40:0/2605035295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80007dd0 con 0x7f0f800028f0 2024-09-13T18:19:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.447+0000 7f0fa67fc700 1 --2- 172.21.15.40:0/2605035295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f800028f0 0x7f0f80004d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:22.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.447+0000 7f0fa67fc700 1 -- 172.21.15.40:0/2605035295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f0f80007f90 con 0x7f0f800028f0 2024-09-13T18:19:22.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.447+0000 7f0fa67fc700 1 --2- 172.21.15.40:0/2605035295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f800028f0 0x7f0f80004d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0f880080d0 tx=0x7f0f980092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:22.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.447+0000 7f0f8f7fe700 1 -- 172.21.15.40:0/2605035295 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9802a780 con 0x7f0f800028f0 2024-09-13T18:19:22.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.447+0000 7f0f8f7fe700 1 -- 172.21.15.40:0/2605035295 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4939 (secure 0 0 0) 0x7f0f9802a8e0 con 0x7f0f800028f0 2024-09-13T18:19:22.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.447+0000 7f0f8f7fe700 1 -- 172.21.15.40:0/2605035295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f800028f0 msgr2=0x7f0f80004d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:22.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.447+0000 7f0f8f7fe700 1 --2- 172.21.15.40:0/2605035295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f800028f0 0x7f0f80004d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0f880080d0 tx=0x7f0f980092e0 comp rx=0 tx=0).stop 2024-09-13T18:19:22.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.447+0000 7f0f8f7fe700 1 -- 172.21.15.40:0/2605035295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f800028f0 msgr2=0x7f0f80004d90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:19:22.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.447+0000 7f0f8f7fe700 1 --2- 172.21.15.40:0/2605035295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f800028f0 0x7f0f80004d90 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0f880080d0 tx=0x7f0f980092e0 comp rx=0 tx=0).stop 2024-09-13T18:19:22.476 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:19:22.476 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:19:22.476 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:19:22.476 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:19:22.476 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 226, 2024-09-13T18:19:22.476 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:19:22.476 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:19:22.476 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:19:22.476 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:22.477 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:19:22.477 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:19:22.477 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:19:22.477 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:19:22.477 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:19:22.477 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:19:22.477 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:19:22.477 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:19:22.477 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:19:22.477 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:19:22.477 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:19:22.477 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:19:22.478 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:19:22.478 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:19:22.478 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:22.478 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:19:22.478 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:19:22.478 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:19:22.478 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:19:22.478 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:19:22.478 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:19:22.478 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:19:22.478 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:19:22.478 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:19:22.478 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:19:22.478 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:19:22.479 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:19:22.479 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:22.479 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:22.479 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:19:22.479 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:19:22.479 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:19:22.479 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:19:22.479 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:19:22.479 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:19:22.479 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:19:22.479 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:19:22.479 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:19:22.479 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:19:22.479 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:19:22.479 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:19:22.480 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:19:22.480 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:19:22.480 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:19:22.480 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:19:22.480 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:19:22.480 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:19:22.480 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:19:22.480 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:19:22.480 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:19:22.480 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:19:22.480 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:19:22.480 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:19:22.480 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:19:22.481 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:19:22.481 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:19:22.481 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:22.481 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:19:22.481 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:22.481 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:19:22.481 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:22.481 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:19:22.481 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:19:22.481 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:19:22.481 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:19:22.481 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:22.481 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:19:22.481 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:19:22.482 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:22.482 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:22.482 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:22.482 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:19:22.482 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:19:22.482 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:22.482 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:22.482 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:22.482 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:22.482 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:19:22.482 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:19:22.482 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:19:22.482 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:19:22.482 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:19:22.483 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:22.483 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:22.483 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:19:22.483 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:19:22.483 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:19:22.483 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:19:22.483 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:22.483 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:19:22.483 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:19:22.483 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:22.483 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:22.483 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:22.483 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:19:22.483 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:19:22.484 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:22.484 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:22.484 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:22.484 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:22.484 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:19:22.484 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:19:22.484 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:19:22.484 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:19:22.484 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:19:22.484 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:22.484 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:22.484 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:19:22.484 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:19:22.484 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:19:22.485 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:19:22.485 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:22.485 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:19:22.485 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:19:22.485 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:22.485 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:22.485 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:22.485 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:19:22.485 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:19:22.485 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:22.485 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:22.485 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:22.485 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:22.485 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:19:22.486 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:19:22.486 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:19:22.486 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:19:22.486 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:19:22.486 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:22.486 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:22.486 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:22.486 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:19:22.486 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:19:22.486 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:22.486 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:19:22.486 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:19:22.486 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:19:22.486 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:22.487 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:22.487 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:19:22.487 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:22.487 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:19:22.487 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:19:22.487 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:19:22.487 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:22.487 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:22.487 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:19:22.487 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:22.487 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:19:22.487 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:19:22.487 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:19:22.487 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:22.488 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:22.488 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:19:22.488 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:22.488 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:19:22.488 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:19:22.488 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:19:22.488 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:22.488 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:22.488 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:19:22.488 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:22.488 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:19:22.488 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:19:22.488 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:19:22.489 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:22.489 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:22.489 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:22.489 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:19:22.489 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:19:22.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.471+0000 7f0f8d7fa700 1 -- 172.21.15.40:0/2605035295 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0f94066800 msgr2=0x7f0f94068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:22.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.471+0000 7f0f8d7fa700 1 --2- 172.21.15.40:0/2605035295 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0f94066800 0x7f0f94068cc0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0001ea0 tx=0x7f0fa0009040 comp rx=0 tx=0).stop 2024-09-13T18:19:22.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.471+0000 7f0fa77fe700 1 -- 172.21.15.40:0/2605035295 reap_dead start 2024-09-13T18:19:22.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.471+0000 7f0f8d7fa700 1 -- 172.21.15.40:0/2605035295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fa80c7320 msgr2=0x7f0fa8153100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:22.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.471+0000 7f0f8d7fa700 1 --2- 172.21.15.40:0/2605035295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fa80c7320 0x7f0fa8153100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00b480 tx=0x7f0f9c00b840 comp rx=0 tx=0).stop 2024-09-13T18:19:22.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.471+0000 7f0f8d7fa700 1 -- 172.21.15.40:0/2605035295 shutdown_connections 2024-09-13T18:19:22.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.471+0000 7f0f8d7fa700 1 --2- 172.21.15.40:0/2605035295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fa80c7320 0x7f0fa8153100 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:22.490 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.471+0000 7f0f8d7fa700 1 -- 172.21.15.40:0/2605035295 >> 172.21.15.40:0/2605035295 conn(0x7f0fa8066090 msgr2=0x7f0fa806d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:22.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.471+0000 7f0f8d7fa700 1 -- 172.21.15.40:0/2605035295 shutdown_connections 2024-09-13T18:19:22.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:22.471+0000 7f0f8d7fa700 1 -- 172.21.15.40:0/2605035295 wait complete. 2024-09-13T18:19:22.550 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:19:22.551 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T18:19:22.551 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:19:23.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.067+0000 7faaf6d3e700 1 Processor -- start 2024-09-13T18:19:23.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.067+0000 7faaf6d3e700 1 -- start start 2024-09-13T18:19:23.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.067+0000 7faaf6d3e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faaf00724e0 0x7faaf00c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.067+0000 7faaf6d3e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faaf0158bc0 0x7faaf006d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.067+0000 7faaf6d3e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaf014fd60 con 0x7faaf00c78d0 2024-09-13T18:19:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.067+0000 7faaf6d3e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaf00c7700 con 0x7faaf0158bc0 2024-09-13T18:19:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.067+0000 7faaf6d3e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf00c6890 con 0x7faaf00724e0 2024-09-13T18:19:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.067+0000 7faaf5d3c700 1 -- >> v1:172.21.15.40:6789/0 conn(0x7faaf00c78d0 legacy=0x7faaf014f650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.40:6789/0 2024-09-13T18:19:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.067+0000 7faaf553b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faaf00724e0 0x7faaf00c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.067+0000 7faaf553b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faaf00724e0 0x7faaf00c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:41270/0 (socket says 172.21.15.40:41270) 2024-09-13T18:19:23.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.067+0000 7faaf553b700 1 -- 172.21.15.40:0/3845719069 learned_addr learned my addr 172.21.15.40:0/3845719069 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:19:23.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.067+0000 7faaf653d700 1 --2- 172.21.15.40:0/3845719069 >> v2:172.21.15.43:3300/0 conn(0x7faaf0158bc0 0x7faaf006d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:23.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.067+0000 7faaf553b700 1 -- 172.21.15.40:0/3845719069 >> v2:172.21.15.43:3300/0 conn(0x7faaf0158bc0 msgr2=0x7faaf006d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:19:23.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.067+0000 7faaf553b700 1 --2- 172.21.15.40:0/3845719069 >> v2:172.21.15.43:3300/0 conn(0x7faaf0158bc0 0x7faaf006d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:23.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.067+0000 7faaf553b700 1 -- 172.21.15.40:0/3845719069 >> v1:172.21.15.40:6789/0 conn(0x7faaf00c78d0 legacy=0x7faaf014f650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:19:23.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.067+0000 7faaf553b700 1 -- 172.21.15.40:0/3845719069 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf00c1320 con 0x7faaf00724e0 2024-09-13T18:19:23.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.067+0000 7faaf553b700 1 --2- 172.21.15.40:0/3845719069 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faaf00724e0 0x7faaf00c71c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faae8009e50 tx=0x7faae800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64969a87a49db117 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:23.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.067+0000 7faaf4d3a700 1 -- 172.21.15.40:0/3845719069 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae800e070 con 0x7faaf00724e0 2024-09-13T18:19:23.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.067+0000 7faaf4d3a700 1 -- 172.21.15.40:0/3845719069 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae8003a40 con 0x7faaf00724e0 2024-09-13T18:19:23.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.067+0000 7faaf4d3a700 1 -- 172.21.15.40:0/3845719069 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae8003df0 con 0x7faaf00724e0 2024-09-13T18:19:23.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.071+0000 7faaf6d3e700 1 -- 172.21.15.40:0/3845719069 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faaf00724e0 msgr2=0x7faaf00c71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:23.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.071+0000 7faaf6d3e700 1 --2- 172.21.15.40:0/3845719069 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faaf00724e0 0x7faaf00c71c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faae8009e50 tx=0x7faae800b330 comp rx=0 tx=0).stop 2024-09-13T18:19:23.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.071+0000 7faaf6d3e700 1 -- 172.21.15.40:0/3845719069 shutdown_connections 2024-09-13T18:19:23.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.071+0000 7faaf6d3e700 1 --2- 172.21.15.40:0/3845719069 >> v2:172.21.15.43:3300/0 conn(0x7faaf0158bc0 0x7faaf006d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:23.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.071+0000 7faaf6d3e700 1 --2- 172.21.15.40:0/3845719069 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faaf00724e0 0x7faaf00c71c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:23.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.071+0000 7faaf6d3e700 1 -- 172.21.15.40:0/3845719069 >> 172.21.15.40:0/3845719069 conn(0x7faaf0066090 msgr2=0x7faaf00664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:23.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.075+0000 7faaf6d3e700 1 -- 172.21.15.40:0/3845719069 shutdown_connections 2024-09-13T18:19:23.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.079+0000 7faaf6d3e700 1 -- 172.21.15.40:0/3845719069 wait complete. 2024-09-13T18:19:23.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.079+0000 7faaf6d3e700 1 Processor -- start 2024-09-13T18:19:23.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.079+0000 7faaf6d3e700 1 -- start start 2024-09-13T18:19:23.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.079+0000 7faaf6d3e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf00724e0 0x7faaf0137b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:23.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.079+0000 7faaf6d3e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaf00c78d0 0x7faaf0157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:23.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.079+0000 7faaf6d3e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faaf0158bc0 0x7faaf0153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:23.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.079+0000 7faaf6d3e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf0152930 con 0x7faaf00724e0 2024-09-13T18:19:23.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.079+0000 7faaf6d3e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf01538e0 con 0x7faaf00c78d0 2024-09-13T18:19:23.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.079+0000 7faaf6d3e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf0155650 con 0x7faaf0158bc0 2024-09-13T18:19:23.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.079+0000 7faaf5d3c700 1 -- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf00724e0 msgr2=0x7faaf0137b00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.40:3300/0 2024-09-13T18:19:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.079+0000 7faaf5d3c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf00724e0 0x7faaf0137b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:19:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.079+0000 7faaf653d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faaf0158bc0 0x7faaf0153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.079+0000 7faaf653d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faaf0158bc0 0x7faaf0153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:41274/0 (socket says 172.21.15.40:41274) 2024-09-13T18:19:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.079+0000 7faaf653d700 1 -- 172.21.15.40:0/731268814 learned_addr learned my addr 172.21.15.40:0/731268814 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:19:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.083+0000 7faaf553b700 1 --2- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaf00c78d0 0x7faaf0157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.083+0000 7faaf653d700 1 -- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaf00c78d0 msgr2=0x7faaf0157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.083+0000 7faaf653d700 1 --2- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaf00c78d0 0x7faaf0157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.083+0000 7faaf653d700 1 -- 172.21.15.40:0/731268814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf00724e0 msgr2=0x7faaf0137b00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:19:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.083+0000 7faaf653d700 1 --2- 172.21.15.40:0/731268814 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faaf00724e0 0x7faaf0137b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.083+0000 7faaf653d700 1 -- 172.21.15.40:0/731268814 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae8009b30 con 0x7faaf0158bc0 2024-09-13T18:19:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.083+0000 7faaf653d700 1 --2- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faaf0158bc0 0x7faaf0153d70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faaec00eb90 tx=0x7faaec00eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.083+0000 7faae67fc700 1 -- 172.21.15.40:0/731268814 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec009740 con 0x7faaf0158bc0 2024-09-13T18:19:23.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.083+0000 7faae67fc700 1 -- 172.21.15.40:0/731268814 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaec00c850 con 0x7faaf0158bc0 2024-09-13T18:19:23.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.083+0000 7faaf6d3e700 1 -- 172.21.15.40:0/731268814 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf0152b80 con 0x7faaf0158bc0 2024-09-13T18:19:23.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.083+0000 7faaf6d3e700 1 -- 172.21.15.40:0/731268814 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf0153290 con 0x7faaf0158bc0 2024-09-13T18:19:23.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.083+0000 7faae67fc700 1 -- 172.21.15.40:0/731268814 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec00cbe0 con 0x7faaf0158bc0 2024-09-13T18:19:23.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.091+0000 7faadbfff700 1 --2- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad0002a40 0x7faad0004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:23.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.091+0000 7faadbfff700 1 -- 172.21.15.40:0/731268814 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad0007f40 con 0x7faad0002a40 2024-09-13T18:19:23.098 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.091+0000 7faaf5d3c700 1 --2- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad0002a40 0x7faad0004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:23.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.095+0000 7faae67fc700 1 -- 172.21.15.40:0/731268814 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faaec00cd40 con 0x7faaf0158bc0 2024-09-13T18:19:23.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.095+0000 7faaf5d3c700 1 -- 172.21.15.40:0/731268814 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faad0008100 con 0x7faad0002a40 2024-09-13T18:19:23.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.095+0000 7faaf5d3c700 1 --2- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad0002a40 0x7faad0004f00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faae000bcc0 tx=0x7faae0009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:23.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.095+0000 7faae67fc700 1 --2- 172.21.15.40:0/731268814 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faadc066870 0x7faadc068d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:23.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.095+0000 7faae67fc700 1 -- 172.21.15.40:0/731268814 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7faaec0845b0 con 0x7faaf0158bc0 2024-09-13T18:19:23.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.095+0000 7faae67fc700 1 -- 172.21.15.40:0/731268814 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae0003ab0 con 0x7faad0002a40 2024-09-13T18:19:23.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.095+0000 7faaf553b700 1 --2- 172.21.15.40:0/731268814 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faadc066870 0x7faadc068d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:23.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.095+0000 7faaf553b700 1 --2- 172.21.15.40:0/731268814 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faadc066870 0x7faadc068d30 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7faae8009e50 tx=0x7faae8009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:23.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.099+0000 7faae67fc700 1 -- 172.21.15.40:0/731268814 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7faae0003c10 con 0x7faad0002a40 2024-09-13T18:19:23.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.099+0000 7faae67fc700 1 -- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad0002a40 msgr2=0x7faad0004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:23.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.099+0000 7faae67fc700 1 --2- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad0002a40 0x7faad0004f00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faae000bcc0 tx=0x7faae0009f30 comp rx=0 tx=0).stop 2024-09-13T18:19:23.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.099+0000 7faae67fc700 1 -- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad0002a40 msgr2=0x7faad0004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:19:23.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.099+0000 7faae67fc700 1 --2- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad0002a40 0x7faad0004f00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.111+0000 7faadbfff700 1 --2- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad0007020 0x7faad0007490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:23.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.111+0000 7faadbfff700 1 -- 172.21.15.40:0/731268814 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad000a400 con 0x7faad0007020 2024-09-13T18:19:23.122 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.115+0000 7faaf5d3c700 1 --2- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad0007020 0x7faad0007490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:23.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.123+0000 7faaf5d3c700 1 -- 172.21.15.40:0/731268814 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7faad000a5c0 con 0x7faad0007020 2024-09-13T18:19:23.130 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.123+0000 7faaf5d3c700 1 --2- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad0007020 0x7faad0007490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faae0009fd0 tx=0x7faae0005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:23.131 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.123+0000 7faae67fc700 1 -- 172.21.15.40:0/731268814 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae001a990 con 0x7faad0007020 2024-09-13T18:19:23.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.123+0000 7faae67fc700 1 -- 172.21.15.40:0/731268814 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4771 (secure 0 0 0) 0x7faae001aaf0 con 0x7faad0007020 2024-09-13T18:19:23.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.123+0000 7faae67fc700 1 -- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad0007020 msgr2=0x7faad0007490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:23.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.123+0000 7faae67fc700 1 --2- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad0007020 0x7faad0007490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faae0009fd0 tx=0x7faae0005ad0 comp rx=0 tx=0).stop 2024-09-13T18:19:23.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.123+0000 7faae67fc700 1 -- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad0007020 msgr2=0x7faad0007490 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:19:23.132 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.123+0000 7faae67fc700 1 --2- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faad0007020 0x7faad0007490 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:23.135 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:19:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:19:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:19:23.135 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:19:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 226, 2024-09-13T18:19:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:19:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:19:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:19:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 2, 2024-09-13T18:19:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:19:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:19:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:19:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:19:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:19:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:19:23.136 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:19:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:19:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:19:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:19:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:19:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:19:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:19:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:19:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:19:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:19:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:19:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:19:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:19:23.137 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:19:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:19:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:19:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:19:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:19:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:19:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:19:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:19:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:19:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:19:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:19:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:19:23.138 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:19:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:19:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:19:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:19:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:19:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:19:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:19:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:19:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:19:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:19:23.139 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:19:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:19:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:19:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:19:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:19:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:19:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:19:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:19:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:19:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:19:23.140 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:19:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:19:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:19:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:19:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:19:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:19:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:19:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:19:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:19:23.141 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:19:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:19:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:19:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:19:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:19:23.142 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:19:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:19:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:19:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:19:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:19:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:19:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:19:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:19:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:23.143 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:19:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:19:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:19:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:19:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:19:23.144 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:19:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:19:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:19:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:19:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:19:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:19:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:19:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:19:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:23.145 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:19:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:19:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:19:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:19:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:19:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:19:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:19:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:23.146 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:19:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:19:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:19:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:19:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:19:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:19:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:19:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:19:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:19:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:23.147 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:19:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:19:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:19:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:19:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:19:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:19:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:19:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:19:23.148 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:23.149 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:23.149 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:23.149 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:19:23.149 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:19:23.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.139+0000 7faaf6d3e700 1 -- 172.21.15.40:0/731268814 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faadc066870 msgr2=0x7faadc068d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:23.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.139+0000 7faaf6d3e700 1 --2- 172.21.15.40:0/731268814 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7faadc066870 0x7faadc068d30 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7faae8009e50 tx=0x7faae8009000 comp rx=0 tx=0).stop 2024-09-13T18:19:23.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.139+0000 7faaf653d700 1 -- 172.21.15.40:0/731268814 reap_dead start 2024-09-13T18:19:23.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.139+0000 7faaf6d3e700 1 -- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faaf0158bc0 msgr2=0x7faaf0153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:23.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.139+0000 7faaf6d3e700 1 --2- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faaf0158bc0 0x7faaf0153d70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faaec00eb90 tx=0x7faaec00eea0 comp rx=0 tx=0).stop 2024-09-13T18:19:23.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.139+0000 7faaf6d3e700 1 -- 172.21.15.40:0/731268814 shutdown_connections 2024-09-13T18:19:23.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.139+0000 7faaf6d3e700 1 --2- 172.21.15.40:0/731268814 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7faaf0158bc0 0x7faaf0153d70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:23.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.139+0000 7faaf6d3e700 1 -- 172.21.15.40:0/731268814 >> 172.21.15.40:0/731268814 conn(0x7faaf0066090 msgr2=0x7faaf00c9570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:23.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.139+0000 7faaf6d3e700 1 -- 172.21.15.40:0/731268814 shutdown_connections 2024-09-13T18:19:23.151 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:23.139+0000 7faaf6d3e700 1 -- 172.21.15.40:0/731268814 wait complete. 2024-09-13T18:19:23.169 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:19:23.169 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:19:43.170 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-13T18:19:43.170 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-13T18:19:43.170 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T18:19:43.170 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T18:19:43.173 INFO:tasks.ceph.mon.a:Started 2024-09-13T18:19:43.173 INFO:tasks.mon_thrash:Sending STOP to mon c 2024-09-13T18:19:43.173 INFO:tasks.ceph.mon.c:Sent signal 19 2024-09-13T18:19:43.173 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-13T18:19:58.174 INFO:tasks.mon_thrash:Sending CONT to mon c 2024-09-13T18:19:58.175 INFO:tasks.ceph.mon.c:Sent signal 18 2024-09-13T18:19:58.175 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:19:58.175 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:19:58.695 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.682+0000 7f0933a03700 1 Processor -- start 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.682+0000 7f0933a03700 1 -- start start 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.686+0000 7f0933a03700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f092c154240 0x7f092c15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.686+0000 7f0933a03700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f092c15ec60 0x7f092c15f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.686+0000 7f0933a03700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f092c06a390 0x7f092c06a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.686+0000 7f0933a03700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f092c0721e0 con 0x7f092c06a390 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.686+0000 7f0933a03700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f092c15e6e0 con 0x7f092c15ec60 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.686+0000 7f0933a03700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092c071690 con 0x7f092c154240 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.686+0000 7f093179f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f092c154240 0x7f092c15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:58.696 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.686+0000 7f093179f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f092c154240 0x7f092c15e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:58900/0 (socket says 172.21.15.40:58900) 2024-09-13T18:19:58.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.686+0000 7f093179f700 1 -- 172.21.15.40:0/3207408984 learned_addr learned my addr 172.21.15.40:0/3207408984 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:19:58.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.686+0000 7f093179f700 1 -- 172.21.15.40:0/3207408984 >> v2:172.21.15.43:3300/0 conn(0x7f092c15ec60 msgr2=0x7f092c15f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:19:58.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.686+0000 7f093179f700 1 --2- 172.21.15.40:0/3207408984 >> v2:172.21.15.43:3300/0 conn(0x7f092c15ec60 0x7f092c15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.686+0000 7f093179f700 1 -- 172.21.15.40:0/3207408984 >> v2:172.21.15.40:3300/0 conn(0x7f092c06a390 msgr2=0x7f092c06a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:19:58.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.686+0000 7f093179f700 1 --2- 172.21.15.40:0/3207408984 >> v2:172.21.15.40:3300/0 conn(0x7f092c06a390 0x7f092c06a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.686+0000 7f093179f700 1 -- 172.21.15.40:0/3207408984 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c154620 con 0x7f092c154240 2024-09-13T18:19:58.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.686+0000 7f093179f700 1 --2- 172.21.15.40:0/3207408984 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f092c154240 0x7f092c15e1a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f092c067320 tx=0x7f092400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b70fc059fa0fe12 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:58.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.686+0000 7f0923fff700 1 -- 172.21.15.40:0/3207408984 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092400e070 con 0x7f092c154240 2024-09-13T18:19:58.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.686+0000 7f0923fff700 1 -- 172.21.15.40:0/3207408984 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092400b9c0 con 0x7f092c154240 2024-09-13T18:19:58.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.690+0000 7f0933a03700 1 -- 172.21.15.40:0/3207408984 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f092c154240 msgr2=0x7f092c15e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:58.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.690+0000 7f0933a03700 1 --2- 172.21.15.40:0/3207408984 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f092c154240 0x7f092c15e1a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f092c067320 tx=0x7f092400b330 comp rx=0 tx=0).stop 2024-09-13T18:19:58.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.690+0000 7f0933a03700 1 -- 172.21.15.40:0/3207408984 shutdown_connections 2024-09-13T18:19:58.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.690+0000 7f0933a03700 1 --2- 172.21.15.40:0/3207408984 >> v2:172.21.15.40:3300/0 conn(0x7f092c06a390 0x7f092c06a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.690+0000 7f0933a03700 1 --2- 172.21.15.40:0/3207408984 >> v2:172.21.15.43:3300/0 conn(0x7f092c15ec60 0x7f092c15f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.690+0000 7f0933a03700 1 --2- 172.21.15.40:0/3207408984 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f092c154240 0x7f092c15e1a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.690+0000 7f0933a03700 1 -- 172.21.15.40:0/3207408984 >> 172.21.15.40:0/3207408984 conn(0x7f092c0cabc0 msgr2=0x7f092c0cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:58.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.690+0000 7f0933a03700 1 -- 172.21.15.40:0/3207408984 shutdown_connections 2024-09-13T18:19:58.704 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.690+0000 7f0933a03700 1 -- 172.21.15.40:0/3207408984 wait complete. 2024-09-13T18:19:58.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.694+0000 7f0933a03700 1 Processor -- start 2024-09-13T18:19:58.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.694+0000 7f0933a03700 1 -- start start 2024-09-13T18:19:58.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.694+0000 7f0933a03700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f092c06a390 0x7f092c0c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:58.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.694+0000 7f0933a03700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f092c154240 0x7f092c0c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:58.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.694+0000 7f0933a03700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f092c15ec60 0x7f092c0c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:58.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.694+0000 7f0933a03700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c140f50 con 0x7f092c06a390 2024-09-13T18:19:58.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.694+0000 7f0933a03700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c141370 con 0x7f092c154240 2024-09-13T18:19:58.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.694+0000 7f0933a03700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092c13e880 con 0x7f092c15ec60 2024-09-13T18:19:58.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.694+0000 7f093179f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f092c06a390 0x7f092c0c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:58.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.694+0000 7f093179f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f092c06a390 0x7f092c0c52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39216/0 (socket says 172.21.15.40:39216) 2024-09-13T18:19:58.705 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.694+0000 7f093179f700 1 -- 172.21.15.40:0/3124386060 learned_addr learned my addr 172.21.15.40:0/3124386060 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:19:58.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.694+0000 7f0931fa0700 1 --2- 172.21.15.40:0/3124386060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f092c15ec60 0x7f092c0c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:58.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.694+0000 7f0930f9e700 1 --2- 172.21.15.40:0/3124386060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f092c154240 0x7f092c0c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:58.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.694+0000 7f093179f700 1 -- 172.21.15.40:0/3124386060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f092c15ec60 msgr2=0x7f092c0c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:58.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.694+0000 7f093179f700 1 --2- 172.21.15.40:0/3124386060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f092c15ec60 0x7f092c0c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.694+0000 7f093179f700 1 -- 172.21.15.40:0/3124386060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f092c154240 msgr2=0x7f092c0c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:58.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.694+0000 7f093179f700 1 --2- 172.21.15.40:0/3124386060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f092c154240 0x7f092c0c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:58.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.694+0000 7f093179f700 1 -- 172.21.15.40:0/3124386060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0924009b30 con 0x7f092c06a390 2024-09-13T18:19:58.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.694+0000 7f093179f700 1 --2- 172.21.15.40:0/3124386060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f092c06a390 0x7f092c0c52b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f092c0717d0 tx=0x7f0924004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:58.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.698+0000 7f0921ffb700 1 -- 172.21.15.40:0/3124386060 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092401c610 con 0x7f092c06a390 2024-09-13T18:19:58.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.698+0000 7f0921ffb700 1 -- 172.21.15.40:0/3124386060 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092400bed0 con 0x7f092c06a390 2024-09-13T18:19:58.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.698+0000 7f0921ffb700 1 -- 172.21.15.40:0/3124386060 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092401c9b0 con 0x7f092c06a390 2024-09-13T18:19:58.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.698+0000 7f0933a03700 1 -- 172.21.15.40:0/3124386060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092c1402d0 con 0x7f092c06a390 2024-09-13T18:19:58.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.698+0000 7f0933a03700 1 -- 172.21.15.40:0/3124386060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092c140890 con 0x7f092c06a390 2024-09-13T18:19:58.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.698+0000 7f0921ffb700 1 -- 172.21.15.40:0/3124386060 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f092401cb10 con 0x7f092c06a390 2024-09-13T18:19:58.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.698+0000 7f0921ffb700 1 --2- 172.21.15.40:0/3124386060 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0910066760 0x7f0910068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:58.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.698+0000 7f0930f9e700 1 --2- 172.21.15.40:0/3124386060 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0910066760 0x7f0910068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:58.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.702+0000 7f0933a03700 1 -- 172.21.15.40:0/3124386060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092c0c3b40 con 0x7f092c06a390 2024-09-13T18:19:58.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.702+0000 7f0930f9e700 1 --2- 172.21.15.40:0/3124386060 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0910066760 0x7f0910068c20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f091c006fd0 tx=0x7f091c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:58.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.702+0000 7f0921ffb700 1 -- 172.21.15.40:0/3124386060 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0924085d20 con 0x7f092c06a390 2024-09-13T18:19:58.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:58.710+0000 7f0921ffb700 1 -- 172.21.15.40:0/3124386060 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0924052cd0 con 0x7f092c06a390 2024-09-13T18:19:59.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.426+0000 7f0933a03700 1 -- 172.21.15.40:0/3124386060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f092c071040 con 0x7f092c06a390 2024-09-13T18:19:59.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.426+0000 7f0921ffb700 1 -- 172.21.15.40:0/3124386060 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f09240562f0 con 0x7f092c06a390 2024-09-13T18:19:59.439 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":234,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:19:59.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.434+0000 7f0933a03700 1 -- 172.21.15.40:0/3124386060 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0910066760 msgr2=0x7f0910068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.434+0000 7f0933a03700 1 --2- 172.21.15.40:0/3124386060 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0910066760 0x7f0910068c20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f091c006fd0 tx=0x7f091c008040 comp rx=0 tx=0).stop 2024-09-13T18:19:59.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.434+0000 7f0933a03700 1 -- 172.21.15.40:0/3124386060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f092c06a390 msgr2=0x7f092c0c52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.434+0000 7f0933a03700 1 --2- 172.21.15.40:0/3124386060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f092c06a390 0x7f092c0c52b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f092c0717d0 tx=0x7f0924004af0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.434+0000 7f0933a03700 1 -- 172.21.15.40:0/3124386060 shutdown_connections 2024-09-13T18:19:59.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.434+0000 7f0933a03700 1 --2- 172.21.15.40:0/3124386060 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0910066760 0x7f0910068c20 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.434+0000 7f0933a03700 1 --2- 172.21.15.40:0/3124386060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f092c15ec60 0x7f092c0c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.434+0000 7f0933a03700 1 --2- 172.21.15.40:0/3124386060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f092c154240 0x7f092c0c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.434+0000 7f0933a03700 1 --2- 172.21.15.40:0/3124386060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f092c06a390 0x7f092c0c52b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.434+0000 7f0933a03700 1 -- 172.21.15.40:0/3124386060 >> 172.21.15.40:0/3124386060 conn(0x7f092c0cabc0 msgr2=0x7f092c167880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:59.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.442+0000 7f0933a03700 1 -- 172.21.15.40:0/3124386060 shutdown_connections 2024-09-13T18:19:59.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.442+0000 7f0933a03700 1 -- 172.21.15.40:0/3124386060 wait complete. 2024-09-13T18:19:59.470 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:19:59.470 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:19:59.470 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:19:59.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.726+0000 7fa20f2c2700 1 Processor -- start 2024-09-13T18:19:59.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.726+0000 7fa20f2c2700 1 -- start start 2024-09-13T18:19:59.735 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.726+0000 7fa20f2c2700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa20814f8c0 0x7fa2080c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.726+0000 7fa20d05e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa20814f8c0 0x7fa2080c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.726+0000 7fa20d05e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa20814f8c0 0x7fa2080c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:43850/0 (socket says 172.21.15.40:43850) 2024-09-13T18:19:59.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.726+0000 7fa20d05e700 1 -- 172.21.15.40:0/1010089526 learned_addr learned my addr 172.21.15.40:0/1010089526 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:19:59.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.726+0000 7fa20f2c2700 1 --2- 172.21.15.40:0/1010089526 >> v2:172.21.15.40:3300/0 conn(0x7fa2080c78d0 0x7fa20815ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.726+0000 7fa20d85f700 1 --2- 172.21.15.40:0/1010089526 >> v2:172.21.15.40:3300/0 conn(0x7fa2080c78d0 0x7fa20815ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.726+0000 7fa20f2c2700 1 -- 172.21.15.40:0/1010089526 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2080c6890 con 0x7fa2080724e0 2024-09-13T18:19:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.730+0000 7fa20f2c2700 1 -- 172.21.15.40:0/1010089526 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2080c7700 con 0x7fa2080c78d0 2024-09-13T18:19:59.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.730+0000 7fa20f2c2700 1 -- 172.21.15.40:0/1010089526 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2080c5f30 con 0x7fa20814f8c0 2024-09-13T18:19:59.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.730+0000 7fa20d85f700 1 -- 172.21.15.40:0/1010089526 >> v2:172.21.15.43:3300/0 conn(0x7fa20814f8c0 msgr2=0x7fa2080c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:19:59.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.730+0000 7fa20d85f700 1 --2- 172.21.15.40:0/1010089526 >> v2:172.21.15.43:3300/0 conn(0x7fa20814f8c0 0x7fa2080c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.730+0000 7fa20d85f700 1 -- 172.21.15.40:0/1010089526 >> v1:172.21.15.40:6789/0 conn(0x7fa2080724e0 legacy=0x7fa20806d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.730+0000 7fa20d85f700 1 -- 172.21.15.40:0/1010089526 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2080c1320 con 0x7fa2080c78d0 2024-09-13T18:19:59.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.730+0000 7fa20d85f700 1 --2- 172.21.15.40:0/1010089526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2080c78d0 0x7fa20815ff50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa2080cbe70 tx=0x7fa20000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e74ec5202bb9cc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:59.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.730+0000 7fa1fffff700 1 -- 172.21.15.40:0/1010089526 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20000bcd0 con 0x7fa2080c78d0 2024-09-13T18:19:59.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.730+0000 7fa1fffff700 1 -- 172.21.15.40:0/1010089526 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20000be30 con 0x7fa2080c78d0 2024-09-13T18:19:59.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.730+0000 7fa20f2c2700 1 -- 172.21.15.40:0/1010089526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2080c78d0 msgr2=0x7fa20815ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.730+0000 7fa20f2c2700 1 --2- 172.21.15.40:0/1010089526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2080c78d0 0x7fa20815ff50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa2080cbe70 tx=0x7fa20000b330 comp rx=0 tx=0).stop 2024-09-13T18:19:59.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.730+0000 7fa20f2c2700 1 -- 172.21.15.40:0/1010089526 shutdown_connections 2024-09-13T18:19:59.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.730+0000 7fa20f2c2700 1 --2- 172.21.15.40:0/1010089526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2080c78d0 0x7fa20815ff50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.730+0000 7fa20f2c2700 1 --2- 172.21.15.40:0/1010089526 >> v2:172.21.15.43:3300/0 conn(0x7fa20814f8c0 0x7fa2080c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.730+0000 7fa20f2c2700 1 -- 172.21.15.40:0/1010089526 >> 172.21.15.40:0/1010089526 conn(0x7fa208066090 msgr2=0x7fa2080664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:59.741 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.730+0000 7fa20f2c2700 1 -- 172.21.15.40:0/1010089526 shutdown_connections 2024-09-13T18:19:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.730+0000 7fa20f2c2700 1 -- 172.21.15.40:0/1010089526 wait complete. 2024-09-13T18:19:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.734+0000 7fa20f2c2700 1 Processor -- start 2024-09-13T18:19:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.734+0000 7fa20f2c2700 1 -- start start 2024-09-13T18:19:59.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.734+0000 7fa20f2c2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2080724e0 0x7fa2081f1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.734+0000 7fa20f2c2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa20814f8c0 0x7fa20814d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.734+0000 7fa20f2c2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa208151660 0x7fa20814d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.734+0000 7fa20c85d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa20814f8c0 0x7fa20814d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.734+0000 7fa20c85d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa20814f8c0 0x7fa20814d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:58928/0 (socket says 172.21.15.40:58928) 2024-09-13T18:19:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.734+0000 7fa20d85f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa208151660 0x7fa20814d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.734+0000 7fa20d85f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa208151660 0x7fa20814d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39230/0 (socket says 172.21.15.40:39230) 2024-09-13T18:19:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.738+0000 7fa20c85d700 1 -- 172.21.15.40:0/2588633618 learned_addr learned my addr 172.21.15.40:0/2588633618 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:19:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.738+0000 7fa20f2c2700 1 -- 172.21.15.40:0/2588633618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2081fd3f0 con 0x7fa208151660 2024-09-13T18:19:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.738+0000 7fa20f2c2700 1 -- 172.21.15.40:0/2588633618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2080c2280 con 0x7fa2080724e0 2024-09-13T18:19:59.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.738+0000 7fa20f2c2700 1 -- 172.21.15.40:0/2588633618 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2081509b0 con 0x7fa20814f8c0 2024-09-13T18:19:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.738+0000 7fa20d05e700 1 --2- 172.21.15.40:0/2588633618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2080724e0 0x7fa2081f1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.738+0000 7fa20d85f700 1 -- 172.21.15.40:0/2588633618 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa20814f8c0 msgr2=0x7fa20814d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.738+0000 7fa20d85f700 1 --2- 172.21.15.40:0/2588633618 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa20814f8c0 0x7fa20814d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.738+0000 7fa20d85f700 1 -- 172.21.15.40:0/2588633618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2080724e0 msgr2=0x7fa2081f1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.738+0000 7fa20d85f700 1 --2- 172.21.15.40:0/2588633618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2080724e0 0x7fa2081f1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.738+0000 7fa20d85f700 1 -- 172.21.15.40:0/2588633618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa200009a70 con 0x7fa208151660 2024-09-13T18:19:59.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.738+0000 7fa20d85f700 1 --2- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa208151660 0x7fa20814d970 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa200017000 tx=0x7fa200004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:59.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.738+0000 7fa1fdffb700 1 -- 172.21.15.40:0/2588633618 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20000e030 con 0x7fa208151660 2024-09-13T18:19:59.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.738+0000 7fa20f2c2700 1 -- 172.21.15.40:0/2588633618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa208150c50 con 0x7fa208151660 2024-09-13T18:19:59.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.738+0000 7fa20f2c2700 1 -- 172.21.15.40:0/2588633618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2081fd010 con 0x7fa208151660 2024-09-13T18:19:59.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.738+0000 7fa1fdffb700 1 -- 172.21.15.40:0/2588633618 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa200004660 con 0x7fa208151660 2024-09-13T18:19:59.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.738+0000 7fa1fdffb700 1 -- 172.21.15.40:0/2588633618 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20001f070 con 0x7fa208151660 2024-09-13T18:19:59.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.738+0000 7fa20f2c2700 1 --2- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8002a40 0x7fa1e8004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.742+0000 7fa20d05e700 1 --2- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8002a40 0x7fa1e8004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.742+0000 7fa1fdffb700 1 -- 172.21.15.40:0/2588633618 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa200011460 con 0x7fa208151660 2024-09-13T18:19:59.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.742+0000 7fa1fdffb700 1 --2- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1ec066800 0x7fa1ec068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.742+0000 7fa20c85d700 1 --2- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1ec066800 0x7fa1ec068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.742+0000 7fa1fdffb700 1 -- 172.21.15.40:0/2588633618 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa200085bb0 con 0x7fa208151660 2024-09-13T18:19:59.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.742+0000 7fa20f2c2700 1 -- 172.21.15.40:0/2588633618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8007f40 con 0x7fa1e8002a40 2024-09-13T18:19:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.742+0000 7fa20d05e700 1 -- 172.21.15.40:0/2588633618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa1e8008100 con 0x7fa1e8002a40 2024-09-13T18:19:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.742+0000 7fa20d05e700 1 --2- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8002a40 0x7fa1e8004f00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa204010780 tx=0x7fa204010b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.742+0000 7fa20c85d700 1 --2- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1ec066800 0x7fa1ec068cc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa2081f1e90 tx=0x7fa1f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:59.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.742+0000 7fa1fdffb700 1 -- 172.21.15.40:0/2588633618 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20400f040 con 0x7fa1e8002a40 2024-09-13T18:19:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.742+0000 7fa1fdffb700 1 -- 172.21.15.40:0/2588633618 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fa204004230 con 0x7fa1e8002a40 2024-09-13T18:19:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.742+0000 7fa1fdffb700 1 -- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8002a40 msgr2=0x7fa1e8004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.742+0000 7fa1fdffb700 1 --2- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8002a40 0x7fa1e8004f00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa204010780 tx=0x7fa204010b40 comp rx=0 tx=0).stop 2024-09-13T18:19:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.742+0000 7fa1fdffb700 1 -- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8002a40 msgr2=0x7fa1e8004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:19:59.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.742+0000 7fa1fdffb700 1 --2- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8002a40 0x7fa1e8004f00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.754+0000 7fa20f2c2700 1 --2- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8006fd0 0x7fa1e8007440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.754+0000 7fa20d05e700 1 --2- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8006fd0 0x7fa1e8007440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.754+0000 7fa20f2c2700 1 -- 172.21.15.40:0/2588633618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e800a400 con 0x7fa1e8006fd0 2024-09-13T18:19:59.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.754+0000 7fa20d05e700 1 -- 172.21.15.40:0/2588633618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fa1e800a5c0 con 0x7fa1e8006fd0 2024-09-13T18:19:59.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.766+0000 7fa20d05e700 1 --2- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8006fd0 0x7fa1e8007440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa204010780 tx=0x7fa204004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:59.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.766+0000 7fa1fdffb700 1 -- 172.21.15.40:0/2588633618 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20402b070 con 0x7fa1e8006fd0 2024-09-13T18:19:59.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.766+0000 7fa1fdffb700 1 -- 172.21.15.40:0/2588633618 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4452 (secure 0 0 0) 0x7fa204023040 con 0x7fa1e8006fd0 2024-09-13T18:19:59.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.766+0000 7fa1fdffb700 1 -- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8006fd0 msgr2=0x7fa1e8007440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.766+0000 7fa1fdffb700 1 --2- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8006fd0 0x7fa1e8007440 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa204010780 tx=0x7fa204004770 comp rx=0 tx=0).stop 2024-09-13T18:19:59.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.766+0000 7fa1fdffb700 1 -- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8006fd0 msgr2=0x7fa1e8007440 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:19:59.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.766+0000 7fa1fdffb700 1 --2- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1e8006fd0 0x7fa1e8007440 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.778 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:19:59.778 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:19:59.778 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:19:59.778 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:19:59.778 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 234, 2024-09-13T18:19:59.778 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:19:59.778 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:19:59.778 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:19:59.778 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:19:59.778 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:59.778 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:19:59.778 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:19:59.778 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:19:59.779 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:19:59.779 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:19:59.779 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:19:59.779 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:19:59.779 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:19:59.779 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:19:59.779 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:19:59.779 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:19:59.779 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:19:59.779 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:19:59.779 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:19:59.779 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:59.779 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:19:59.779 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:19:59.780 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:19:59.780 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:19:59.780 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:19:59.780 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:19:59.780 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:19:59.780 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:19:59.780 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:19:59.780 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:19:59.780 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:19:59.780 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:19:59.780 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:59.780 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:59.780 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:19:59.781 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:19:59.781 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:19:59.781 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:19:59.781 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:19:59.781 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:19:59.781 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:19:59.781 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:19:59.781 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:19:59.781 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:19:59.781 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:19:59.781 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:19:59.781 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:19:59.781 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:19:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:19:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:19:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:19:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:19:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:19:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:19:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:19:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:19:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:19:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:19:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:19:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:19:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:19:59.782 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:19:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:19:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:19:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:19:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:19:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:19:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:19:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:19:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:59.783 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:19:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:19:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:19:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:19:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:19:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:19:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:19:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:59.784 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:19:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:19:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:19:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:19:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:19:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:19:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:19:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:19:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:59.785 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:19:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:19:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:19:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:19:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:19:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:19:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:19:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:19:59.786 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:19:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:19:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:19:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:19:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:19:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:19:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:19:59.787 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:19:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:19:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:19:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:19:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:19:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:19:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:19:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:19:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:19:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:59.788 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:19:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:19:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:19:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:19:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:19:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:19:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:19:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:19:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:19:59.789 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:19:59.789 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:19:59.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.774+0000 7fa1f37fe700 1 -- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1ec066800 msgr2=0x7fa1ec068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.774+0000 7fa1f37fe700 1 --2- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa1ec066800 0x7fa1ec068cc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa2081f1e90 tx=0x7fa1f8009040 comp rx=0 tx=0).stop 2024-09-13T18:19:59.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.774+0000 7fa20d85f700 1 -- 172.21.15.40:0/2588633618 reap_dead start 2024-09-13T18:19:59.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.774+0000 7fa1f37fe700 1 -- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa208151660 msgr2=0x7fa20814d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.774+0000 7fa1f37fe700 1 --2- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa208151660 0x7fa20814d970 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa200017000 tx=0x7fa200004af0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.774+0000 7fa1f37fe700 1 -- 172.21.15.40:0/2588633618 shutdown_connections 2024-09-13T18:19:59.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.774+0000 7fa1f37fe700 1 --2- 172.21.15.40:0/2588633618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa208151660 0x7fa20814d970 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.774+0000 7fa1f37fe700 1 -- 172.21.15.40:0/2588633618 >> 172.21.15.40:0/2588633618 conn(0x7fa208066090 msgr2=0x7fa208158760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:59.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.774+0000 7fa1f37fe700 1 -- 172.21.15.40:0/2588633618 shutdown_connections 2024-09-13T18:19:59.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.774+0000 7fa1f37fe700 1 -- 172.21.15.40:0/2588633618 wait complete. 2024-09-13T18:19:59.802 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:19:59.802 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:19:59.802 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:19:59.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.942+0000 7f77e1367700 1 Processor -- start 2024-09-13T18:19:59.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.942+0000 7f77e1367700 1 -- start start 2024-09-13T18:19:59.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.942+0000 7f77e1367700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f77dc0c7320 0x7f77dc06d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.942+0000 7f77e1367700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77dc072650 con 0x7f77dc072270 2024-09-13T18:19:59.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.942+0000 7f77e1367700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77dc0c5f00 con 0x7f77dc0c6390 2024-09-13T18:19:59.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.942+0000 7f77e1367700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77dc0c7130 con 0x7f77dc0c7320 2024-09-13T18:19:59.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.942+0000 7f77daffd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f77dc072270 0x7f77dc159690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:57296/0 (socket says 172.21.15.40:57296) 2024-09-13T18:19:59.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.942+0000 7f77daffd700 1 -- 172.21.15.40:0/2373583621 learned_addr learned my addr 172.21.15.40:0/2373583621 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:19:59.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77da7fc700 1 --2- 172.21.15.40:0/2373583621 >> v2:172.21.15.40:3300/0 conn(0x7f77dc0c7320 0x7f77dc06d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77d9ffb700 1 -- 172.21.15.40:0/2373583621 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2391215062 0 0) 0x7f77dc072650 con 0x7f77dc072270 2024-09-13T18:19:59.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77d9ffb700 1 -- 172.21.15.40:0/2373583621 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77c40035d0 con 0x7f77dc072270 2024-09-13T18:19:59.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77d9ffb700 1 -- 172.21.15.40:0/2373583621 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 45497493 0 0) 0x7f77c40035d0 con 0x7f77dc072270 2024-09-13T18:19:59.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77d9ffb700 1 -- 172.21.15.40:0/2373583621 >> v2:172.21.15.40:3300/0 conn(0x7f77dc0c7320 msgr2=0x7f77dc06d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:19:59.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77d9ffb700 1 --2- 172.21.15.40:0/2373583621 >> v2:172.21.15.40:3300/0 conn(0x7f77dc0c7320 0x7f77dc06d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77d9ffb700 1 -- 172.21.15.40:0/2373583621 >> v1:172.21.15.43:6789/0 conn(0x7f77dc0c6390 legacy=0x7f77dc163f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:19:59.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77d9ffb700 1 -- 172.21.15.40:0/2373583621 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc0c8450 con 0x7f77dc072270 2024-09-13T18:19:59.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77d9ffb700 1 -- 172.21.15.40:0/2373583621 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f77d40019f0 con 0x7f77dc072270 2024-09-13T18:19:59.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77da7fc700 1 --2- 172.21.15.40:0/2373583621 >> v2:172.21.15.40:3300/0 conn(0x7f77dc0c7320 0x7f77dc06d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:19:59.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77d9ffb700 1 -- 172.21.15.40:0/2373583621 >> v1:172.21.15.40:6789/0 conn(0x7f77dc072270 legacy=0x7f77dc159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77d9ffb700 1 --2- 172.21.15.40:0/2373583621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77c4003af0 0x7f77c4003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77d9ffb700 1 --2- 172.21.15.40:0/2373583621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77c4004870 0x7f77c4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77d9ffb700 1 --2- 172.21.15.40:0/2373583621 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f77c4005430 0x7f77c400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77d9ffb700 1 -- 172.21.15.40:0/2373583621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc0c7130 con 0x7f77c4004870 2024-09-13T18:19:59.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77da7fc700 1 --2- 172.21.15.40:0/2373583621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77c4004870 0x7f77c4004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77daffd700 1 --2- 172.21.15.40:0/2373583621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77c4003af0 0x7f77c4003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77db7fe700 1 --2- 172.21.15.40:0/2373583621 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f77c4005430 0x7f77c400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77d9ffb700 1 -- 172.21.15.40:0/2373583621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c400c2e0 con 0x7f77c4003af0 2024-09-13T18:19:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77d9ffb700 1 -- 172.21.15.40:0/2373583621 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c400c4a0 con 0x7f77c4005430 2024-09-13T18:19:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77daffd700 1 -- 172.21.15.40:0/2373583621 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f77c4005430 msgr2=0x7f77c400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:19:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77daffd700 1 --2- 172.21.15.40:0/2373583621 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f77c4005430 0x7f77c400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77daffd700 1 -- 172.21.15.40:0/2373583621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77c4004870 msgr2=0x7f77c4004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:19:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77daffd700 1 --2- 172.21.15.40:0/2373583621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77c4004870 0x7f77c4004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77daffd700 1 -- 172.21.15.40:0/2373583621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c40035d0 con 0x7f77c4003af0 2024-09-13T18:19:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77da7fc700 1 --2- 172.21.15.40:0/2373583621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77c4004870 0x7f77c4004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:19:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77db7fe700 1 -- 172.21.15.40:0/2373583621 reap_dead start 2024-09-13T18:19:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77daffd700 1 --2- 172.21.15.40:0/2373583621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77c4003af0 0x7f77c4003ed0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f77d4005960 tx=0x7f77c40058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcdb7aa0a7354893 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:59.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77d9ffb700 1 -- 172.21.15.40:0/2373583621 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d4005420 con 0x7f77c4003af0 2024-09-13T18:19:59.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77d9ffb700 1 -- 172.21.15.40:0/2373583621 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d400d910 con 0x7f77c4003af0 2024-09-13T18:19:59.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.946+0000 7f77d9ffb700 1 -- 172.21.15.40:0/2373583621 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d400dca0 con 0x7f77c4003af0 2024-09-13T18:19:59.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.950+0000 7f77e1367700 1 -- 172.21.15.40:0/2373583621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77c4003af0 msgr2=0x7f77c4003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.950+0000 7f77e1367700 1 --2- 172.21.15.40:0/2373583621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77c4003af0 0x7f77c4003ed0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f77d4005960 tx=0x7f77c40058a0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.950+0000 7f77e1367700 1 -- 172.21.15.40:0/2373583621 shutdown_connections 2024-09-13T18:19:59.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.950+0000 7f77e1367700 1 --2- 172.21.15.40:0/2373583621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77c4003af0 0x7f77c4003ed0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.950+0000 7f77e1367700 1 -- 172.21.15.40:0/2373583621 >> 172.21.15.40:0/2373583621 conn(0x7f77dc066090 msgr2=0x7f77dc0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:19:59.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.954+0000 7f77e1367700 1 -- 172.21.15.40:0/2373583621 shutdown_connections 2024-09-13T18:19:59.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.962+0000 7f77e1367700 1 -- 172.21.15.40:0/2373583621 wait complete. 2024-09-13T18:19:59.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.962+0000 7f77e1367700 1 Processor -- start 2024-09-13T18:19:59.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.962+0000 7f77e1367700 1 -- start start 2024-09-13T18:19:59.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.962+0000 7f77e1367700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77c4003af0 0x7f77dc06a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.962+0000 7f77e1367700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f77dc0ca010 0x7f77dc0ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.962+0000 7f77e1367700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77dc153b00 0x7f77dc153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.962+0000 7f77e1367700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc08bbe0 con 0x7f77dc153b00 2024-09-13T18:19:59.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.962+0000 7f77e1367700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc15be00 con 0x7f77c4003af0 2024-09-13T18:19:59.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.962+0000 7f77e1367700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc0c8e50 con 0x7f77dc0ca010 2024-09-13T18:19:59.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.962+0000 7f77daffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77c4003af0 0x7f77dc06a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.962+0000 7f77db7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77dc153b00 0x7f77dc153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.962+0000 7f77db7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77dc153b00 0x7f77dc153f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39274/0 (socket says 172.21.15.40:39274) 2024-09-13T18:19:59.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.962+0000 7f77db7fe700 1 -- 172.21.15.40:0/410916691 learned_addr learned my addr 172.21.15.40:0/410916691 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:19:59.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.966+0000 7f77da7fc700 1 --2- 172.21.15.40:0/410916691 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f77dc0ca010 0x7f77dc0ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.966+0000 7f77db7fe700 1 -- 172.21.15.40:0/410916691 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f77dc0ca010 msgr2=0x7f77dc0ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.966+0000 7f77db7fe700 1 --2- 172.21.15.40:0/410916691 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f77dc0ca010 0x7f77dc0ca470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.966+0000 7f77db7fe700 1 -- 172.21.15.40:0/410916691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77c4003af0 msgr2=0x7f77dc06a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.966+0000 7f77db7fe700 1 --2- 172.21.15.40:0/410916691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77c4003af0 0x7f77dc06a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:19:59.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.966+0000 7f77db7fe700 1 -- 172.21.15.40:0/410916691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc0ca9b0 con 0x7f77dc153b00 2024-09-13T18:19:59.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.966+0000 7f77db7fe700 1 --2- 172.21.15.40:0/410916691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77dc153b00 0x7f77dc153f80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f77cc007fd0 tx=0x7f77cc0055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:59.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.970+0000 7f77c37fe700 1 -- 172.21.15.40:0/410916691 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc019040 con 0x7f77dc153b00 2024-09-13T18:19:59.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.970+0000 7f77c37fe700 1 -- 172.21.15.40:0/410916691 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77cc012520 con 0x7f77dc153b00 2024-09-13T18:19:59.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.970+0000 7f77e1367700 1 -- 172.21.15.40:0/410916691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77dc0c9570 con 0x7f77dc153b00 2024-09-13T18:19:59.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.970+0000 7f77e1367700 1 -- 172.21.15.40:0/410916691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77dc0c9220 con 0x7f77dc153b00 2024-09-13T18:19:59.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.970+0000 7f77c37fe700 1 -- 172.21.15.40:0/410916691 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc006ac0 con 0x7f77dc153b00 2024-09-13T18:19:59.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.974+0000 7f77c37fe700 1 -- 172.21.15.40:0/410916691 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f77cc006c20 con 0x7f77dc153b00 2024-09-13T18:19:59.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.974+0000 7f77c37fe700 1 --2- 172.21.15.40:0/410916691 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f77c40685f0 0x7f77c406aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.974+0000 7f77daffd700 1 --2- 172.21.15.40:0/410916691 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f77c40685f0 0x7f77c406aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.974+0000 7f77daffd700 1 --2- 172.21.15.40:0/410916691 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f77c40685f0 0x7f77c406aab0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f77d4000f80 tx=0x7f77d400a070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:59.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.974+0000 7f77c37fe700 1 -- 172.21.15.40:0/410916691 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f77cc08bc40 con 0x7f77dc153b00 2024-09-13T18:19:59.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.974+0000 7f77e1367700 1 --2- 172.21.15.40:0/410916691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc002a40 0x7f77bc004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:19:59.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.974+0000 7f77e1367700 1 -- 172.21.15.40:0/410916691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc007f40 con 0x7f77bc002a40 2024-09-13T18:19:59.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.974+0000 7f77da7fc700 1 --2- 172.21.15.40:0/410916691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc002a40 0x7f77bc004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:19:59.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.974+0000 7f77da7fc700 1 -- 172.21.15.40:0/410916691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f77bc008100 con 0x7f77bc002a40 2024-09-13T18:19:59.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.974+0000 7f77da7fc700 1 --2- 172.21.15.40:0/410916691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc002a40 0x7f77bc004f00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0c1b30 tx=0x7f77d0011890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:19:59.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.974+0000 7f77c37fe700 1 -- 172.21.15.40:0/410916691 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d0015870 con 0x7f77bc002a40 2024-09-13T18:19:59.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.974+0000 7f77c37fe700 1 -- 172.21.15.40:0/410916691 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f77d000c040 con 0x7f77bc002a40 2024-09-13T18:19:59.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.978+0000 7f77c37fe700 1 -- 172.21.15.40:0/410916691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc002a40 msgr2=0x7f77bc004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:19:59.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.978+0000 7f77c37fe700 1 --2- 172.21.15.40:0/410916691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc002a40 0x7f77bc004f00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0c1b30 tx=0x7f77d0011890 comp rx=0 tx=0).stop 2024-09-13T18:19:59.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.978+0000 7f77c37fe700 1 -- 172.21.15.40:0/410916691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc002a40 msgr2=0x7f77bc004f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:19:59.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.978+0000 7f77c37fe700 1 --2- 172.21.15.40:0/410916691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc002a40 0x7f77bc004f00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.998+0000 7f77e1367700 1 --2- 172.21.15.40:0/410916691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc006fd0 0x7f77bc007460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:00.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.998+0000 7f77e1367700 1 -- 172.21.15.40:0/410916691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc00a400 con 0x7f77bc006fd0 2024-09-13T18:20:00.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.998+0000 7f77da7fc700 1 --2- 172.21.15.40:0/410916691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc006fd0 0x7f77bc007460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:00.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.998+0000 7f77da7fc700 1 -- 172.21.15.40:0/410916691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f77bc00a5c0 con 0x7f77bc006fd0 2024-09-13T18:20:00.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.998+0000 7f77da7fc700 1 --2- 172.21.15.40:0/410916691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc006fd0 0x7f77bc007460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f77d0006fa0 tx=0x7f77d0004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:00.007 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.998+0000 7f77c37fe700 1 -- 172.21.15.40:0/410916691 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d0023040 con 0x7f77bc006fd0 2024-09-13T18:20:00.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.998+0000 7f77c37fe700 1 -- 172.21.15.40:0/410916691 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4948 (secure 0 0 0) 0x7f77d00147f0 con 0x7f77bc006fd0 2024-09-13T18:20:00.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.998+0000 7f77c37fe700 1 -- 172.21.15.40:0/410916691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc006fd0 msgr2=0x7f77bc007460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:00.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.998+0000 7f77c37fe700 1 --2- 172.21.15.40:0/410916691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc006fd0 0x7f77bc007460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f77d0006fa0 tx=0x7f77d0004770 comp rx=0 tx=0).stop 2024-09-13T18:20:00.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.998+0000 7f77c37fe700 1 -- 172.21.15.40:0/410916691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc006fd0 msgr2=0x7f77bc007460 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:20:00.008 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:19:59.998+0000 7f77c37fe700 1 --2- 172.21.15.40:0/410916691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77bc006fd0 0x7f77bc007460 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f77d0006fa0 tx=0x7f77d0004770 comp rx=0 tx=0).stop 2024-09-13T18:20:00.008 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:20:00.009 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:20:00.009 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:20:00.009 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:20:00.009 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 234, 2024-09-13T18:20:00.009 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:20:00.009 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:20:00.009 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:20:00.009 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:20:00.009 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:00.009 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:20:00.009 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:20:00.009 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:20:00.010 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:20:00.010 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:00.010 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:00.010 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:00.010 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:00.010 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:00.010 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:00.010 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:00.010 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:00.010 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:00.010 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:00.010 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:00.010 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:20:00.011 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:20:00.011 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:00.011 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:00.011 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:00.011 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:00.011 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:00.011 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:00.011 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:00.011 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:00.011 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:00.011 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:00.011 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:00.011 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.012 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:20:00.012 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:20:00.012 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:20:00.012 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:20:00.012 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:20:00.012 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:20:00.012 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:20:00.012 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:20:00.012 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:20:00.012 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:20:00.012 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:20:00.012 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:20:00.012 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:20:00.012 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:20:00.013 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:20:00.013 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:20:00.013 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:20:00.013 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:00.013 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:00.013 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:00.013 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:00.013 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:00.013 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:00.013 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:00.013 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:00.013 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:00.013 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:00.013 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:00.014 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:20:00.014 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.014 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:20:00.014 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.014 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:20:00.014 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:20:00.014 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:00.014 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:00.014 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.014 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:00.014 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:20:00.014 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:00.014 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.014 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.014 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:00.015 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:20:00.015 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:00.015 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:00.015 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:00.015 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.015 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:20:00.015 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:20:00.015 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:00.015 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:00.015 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:00.015 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.015 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.015 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:20:00.015 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:20:00.016 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:00.016 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:00.016 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.016 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:00.016 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:20:00.016 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:00.016 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.016 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.016 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:00.016 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:20:00.016 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:00.017 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:00.017 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:00.017 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.017 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:20:00.017 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:20:00.017 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:00.017 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:00.017 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:00.017 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.017 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.017 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:20:00.017 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:20:00.017 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:00.018 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:00.018 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.018 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:00.018 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:20:00.018 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:00.018 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.018 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.018 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:00.018 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:20:00.018 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:00.019 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:00.019 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:00.019 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.019 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:20:00.019 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:20:00.019 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:00.019 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:00.019 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:00.019 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:00.019 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:00.019 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.020 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:20:00.020 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:20:00.020 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.020 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:00.020 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:00.020 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:20:00.020 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:00.020 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:00.020 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:20:00.020 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.020 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:00.020 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:00.020 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:20:00.020 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:00.021 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:00.021 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:20:00.021 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.021 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:00.021 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:00.021 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:20:00.021 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:00.021 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:00.021 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:20:00.021 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.021 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:00.021 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:00.022 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:20:00.022 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:00.022 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:00.022 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:20:00.022 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.022 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:00.022 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:00.022 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:20:00.022 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:00.022 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:00.022 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.022 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:20:00.022 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:20:00.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.010+0000 7f77c17fa700 1 -- 172.21.15.40:0/410916691 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f77c40685f0 msgr2=0x7f77c406aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:00.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.010+0000 7f77c17fa700 1 --2- 172.21.15.40:0/410916691 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f77c40685f0 0x7f77c406aab0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f77d4000f80 tx=0x7f77d400a070 comp rx=0 tx=0).stop 2024-09-13T18:20:00.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.010+0000 7f77db7fe700 1 -- 172.21.15.40:0/410916691 reap_dead start 2024-09-13T18:20:00.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.010+0000 7f77c17fa700 1 -- 172.21.15.40:0/410916691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77dc153b00 msgr2=0x7f77dc153f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:00.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.010+0000 7f77c17fa700 1 --2- 172.21.15.40:0/410916691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77dc153b00 0x7f77dc153f80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f77cc007fd0 tx=0x7f77cc0055e0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.010+0000 7f77c17fa700 1 -- 172.21.15.40:0/410916691 shutdown_connections 2024-09-13T18:20:00.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.010+0000 7f77c17fa700 1 --2- 172.21.15.40:0/410916691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77dc153b00 0x7f77dc153f80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.010+0000 7f77c17fa700 1 -- 172.21.15.40:0/410916691 >> 172.21.15.40:0/410916691 conn(0x7f77dc066090 msgr2=0x7f77dc06d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:00.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.010+0000 7f77c17fa700 1 -- 172.21.15.40:0/410916691 shutdown_connections 2024-09-13T18:20:00.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.010+0000 7f77c17fa700 1 -- 172.21.15.40:0/410916691 wait complete. 2024-09-13T18:20:00.040 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:20:00.040 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:20:00.040 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:20:00.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.266+0000 7fb6b96f9700 1 Processor -- start 2024-09-13T18:20:00.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.266+0000 7fb6b96f9700 1 -- start start 2024-09-13T18:20:00.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.266+0000 7fb6b96f9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb6b40c7f60 0x7fb6b40c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:00.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.266+0000 7fb6b96f9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb6b4163d00 0x7fb6b40c6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:00.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.266+0000 7fb6b96f9700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b4167290 con 0x7fb6b40c8340 2024-09-13T18:20:00.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.266+0000 7fb6b96f9700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b40c1630 con 0x7fb6b40c7f60 2024-09-13T18:20:00.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.266+0000 7fb6b96f9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b4166680 con 0x7fb6b4163d00 2024-09-13T18:20:00.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.266+0000 7fb6b2ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb6b40c7f60 0x7fb6b40c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:00.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.266+0000 7fb6b2ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb6b40c7f60 0x7fb6b40c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39290/0 (socket says 172.21.15.40:39290) 2024-09-13T18:20:00.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.266+0000 7fb6b2ffd700 1 -- 172.21.15.40:0/1869446459 learned_addr learned my addr 172.21.15.40:0/1869446459 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:00.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.266+0000 7fb6b37fe700 1 --2- 172.21.15.40:0/1869446459 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb6b4163d00 0x7fb6b40c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:00.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.270+0000 7fb6b2ffd700 1 -- 172.21.15.40:0/1869446459 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb6b4163d00 msgr2=0x7fb6b40c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:20:00.281 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.270+0000 7fb6b2ffd700 1 --2- 172.21.15.40:0/1869446459 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb6b4163d00 0x7fb6b40c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.270+0000 7fb6b2ffd700 1 -- 172.21.15.40:0/1869446459 >> v1:172.21.15.43:6789/0 conn(0x7fb6b40c8340 legacy=0x7fb6b41635f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:20:00.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.270+0000 7fb6b2ffd700 1 -- 172.21.15.40:0/1869446459 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b4166960 con 0x7fb6b40c7f60 2024-09-13T18:20:00.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.270+0000 7fb6b2ffd700 1 --2- 172.21.15.40:0/1869446459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b40c7f60 0x7fb6b40c7250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4008990 tx=0x7fb6a4008ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3de99618705749 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:00.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.270+0000 7fb6b1ffb700 1 -- 172.21.15.40:0/1869446459 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4018c60 con 0x7fb6b40c7f60 2024-09-13T18:20:00.282 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.270+0000 7fb6b1ffb700 1 -- 172.21.15.40:0/1869446459 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a4018dc0 con 0x7fb6b40c7f60 2024-09-13T18:20:00.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.270+0000 7fb6b96f9700 1 -- 172.21.15.40:0/1869446459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b40c7f60 msgr2=0x7fb6b40c7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:00.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.270+0000 7fb6b96f9700 1 --2- 172.21.15.40:0/1869446459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b40c7f60 0x7fb6b40c7250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4008990 tx=0x7fb6a4008ca0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.270+0000 7fb6b96f9700 1 -- 172.21.15.40:0/1869446459 shutdown_connections 2024-09-13T18:20:00.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.270+0000 7fb6b96f9700 1 --2- 172.21.15.40:0/1869446459 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb6b4163d00 0x7fb6b40c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.270+0000 7fb6b96f9700 1 --2- 172.21.15.40:0/1869446459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b40c7f60 0x7fb6b40c7250 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.270+0000 7fb6b96f9700 1 -- 172.21.15.40:0/1869446459 >> 172.21.15.40:0/1869446459 conn(0x7fb6b4066090 msgr2=0x7fb6b40664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:00.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.282+0000 7fb6b96f9700 1 -- 172.21.15.40:0/1869446459 shutdown_connections 2024-09-13T18:20:00.295 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.286+0000 7fb6b96f9700 1 -- 172.21.15.40:0/1869446459 wait complete. 2024-09-13T18:20:00.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.286+0000 7fb6b96f9700 1 Processor -- start 2024-09-13T18:20:00.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.286+0000 7fb6b96f9700 1 -- start start 2024-09-13T18:20:00.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.286+0000 7fb6b96f9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb6b40c8340 0x7fb6b41ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:00.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.286+0000 7fb6b96f9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b4163d00 0x7fb6b41fbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:00.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.286+0000 7fb6b96f9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b40caa20 0x7fb6b4162040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:00.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.286+0000 7fb6b96f9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b41ff890 con 0x7fb6b40caa20 2024-09-13T18:20:00.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.286+0000 7fb6b96f9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b40c10e0 con 0x7fb6b4163d00 2024-09-13T18:20:00.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.286+0000 7fb6b96f9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b40caf10 con 0x7fb6b40c8340 2024-09-13T18:20:00.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.286+0000 7fb6b2ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb6b40c8340 0x7fb6b41ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:00.301 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.286+0000 7fb6b37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b40caa20 0x7fb6b4162040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:00.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.286+0000 7fb6b37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b40caa20 0x7fb6b4162040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39300/0 (socket says 172.21.15.40:39300) 2024-09-13T18:20:00.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.286+0000 7fb6b37fe700 1 -- 172.21.15.40:0/1004696638 learned_addr learned my addr 172.21.15.40:0/1004696638 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:00.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.290+0000 7fb6b2ffd700 1 -- 172.21.15.40:0/1004696638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b4163d00 msgr2=0x7fb6b41fbc30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:20:00.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.290+0000 7fb6b2ffd700 1 --2- 172.21.15.40:0/1004696638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b4163d00 0x7fb6b41fbc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.290+0000 7fb6b2ffd700 1 -- 172.21.15.40:0/1004696638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b40caa20 msgr2=0x7fb6b4162040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:00.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.290+0000 7fb6b2ffd700 1 --2- 172.21.15.40:0/1004696638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b40caa20 0x7fb6b4162040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.290+0000 7fb6b2ffd700 1 -- 172.21.15.40:0/1004696638 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a40085f0 con 0x7fb6b40c8340 2024-09-13T18:20:00.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.290+0000 7fb6b2ffd700 1 --2- 172.21.15.40:0/1004696638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb6b40c8340 0x7fb6b41ffd20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6a401c000 tx=0x7fb6a4004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:00.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.290+0000 7fb69b7fe700 1 -- 172.21.15.40:0/1004696638 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4008470 con 0x7fb6b40c8340 2024-09-13T18:20:00.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.290+0000 7fb69b7fe700 1 -- 172.21.15.40:0/1004696638 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a4018a60 con 0x7fb6b40c8340 2024-09-13T18:20:00.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.290+0000 7fb69b7fe700 1 -- 172.21.15.40:0/1004696638 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a4018df0 con 0x7fb6b40c8340 2024-09-13T18:20:00.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.290+0000 7fb6b96f9700 1 -- 172.21.15.40:0/1004696638 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b40cb1f0 con 0x7fb6b40c8340 2024-09-13T18:20:00.302 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.290+0000 7fb6b96f9700 1 -- 172.21.15.40:0/1004696638 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b41ff520 con 0x7fb6b40c8340 2024-09-13T18:20:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.294+0000 7fb69b7fe700 1 -- 172.21.15.40:0/1004696638 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb6a4012040 con 0x7fb6b40c8340 2024-09-13T18:20:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.294+0000 7fb69b7fe700 1 --2- 172.21.15.40:0/1004696638 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb6a0088790 0x7fb6a008ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:00.303 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.294+0000 7fb69b7fe700 1 -- 172.21.15.40:0/1004696638 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb6a4084d60 con 0x7fb6b40c8340 2024-09-13T18:20:00.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.294+0000 7fb6997fa700 1 --2- 172.21.15.40:0/1004696638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb694002a40 0x7fb694004f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:00.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.294+0000 7fb6997fa700 1 -- 172.21.15.40:0/1004696638 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb694007f40 con 0x7fb694002a40 2024-09-13T18:20:00.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.294+0000 7fb6b27fc700 1 --2- 172.21.15.40:0/1004696638 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb6a0088790 0x7fb6a008ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:00.304 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.294+0000 7fb6b27fc700 1 --2- 172.21.15.40:0/1004696638 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb6a0088790 0x7fb6a008ac50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40ca9b0 tx=0x7fb6ac009250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:00.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.298+0000 7fb6b37fe700 1 --2- 172.21.15.40:0/1004696638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb694002a40 0x7fb694004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:00.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.298+0000 7fb6b37fe700 1 -- 172.21.15.40:0/1004696638 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb694008100 con 0x7fb694002a40 2024-09-13T18:20:00.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.298+0000 7fb6b37fe700 1 --2- 172.21.15.40:0/1004696638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb694002a40 0x7fb694004f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8010b70 tx=0x7fb6a8010f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:00.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.302+0000 7fb69b7fe700 1 -- 172.21.15.40:0/1004696638 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8015070 con 0x7fb694002a40 2024-09-13T18:20:00.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.302+0000 7fb69b7fe700 1 -- 172.21.15.40:0/1004696638 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb6a800fc80 con 0x7fb694002a40 2024-09-13T18:20:00.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.302+0000 7fb69b7fe700 1 -- 172.21.15.40:0/1004696638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb694002a40 msgr2=0x7fb694004f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:00.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.302+0000 7fb69b7fe700 1 --2- 172.21.15.40:0/1004696638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb694002a40 0x7fb694004f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8010b70 tx=0x7fb6a8010f30 comp rx=0 tx=0).stop 2024-09-13T18:20:00.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.302+0000 7fb69b7fe700 1 -- 172.21.15.40:0/1004696638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb694002a40 msgr2=0x7fb694004f00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:20:00.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.302+0000 7fb69b7fe700 1 --2- 172.21.15.40:0/1004696638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb694002a40 0x7fb694004f00 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8010b70 tx=0x7fb6a8010f30 comp rx=0 tx=0).stop 2024-09-13T18:20:00.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.330+0000 7fb6997fa700 1 --2- 172.21.15.40:0/1004696638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb694006fe0 0x7fb694007470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:00.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.334+0000 7fb6b37fe700 1 --2- 172.21.15.40:0/1004696638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb694006fe0 0x7fb694007470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:00.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.334+0000 7fb6997fa700 1 -- 172.21.15.40:0/1004696638 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69400a400 con 0x7fb694006fe0 2024-09-13T18:20:00.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.334+0000 7fb6b37fe700 1 -- 172.21.15.40:0/1004696638 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb69400a5c0 con 0x7fb694006fe0 2024-09-13T18:20:00.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.334+0000 7fb6b37fe700 1 --2- 172.21.15.40:0/1004696638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb694006fe0 0x7fb694007470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb6a801b000 tx=0x7fb6a8004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:00.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.338+0000 7fb69b7fe700 1 -- 172.21.15.40:0/1004696638 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8015070 con 0x7fb694006fe0 2024-09-13T18:20:00.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.338+0000 7fb69b7fe700 1 -- 172.21.15.40:0/1004696638 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4782 (secure 0 0 0) 0x7fb6a8021300 con 0x7fb694006fe0 2024-09-13T18:20:00.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.338+0000 7fb69b7fe700 1 -- 172.21.15.40:0/1004696638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb694006fe0 msgr2=0x7fb694007470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:00.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.338+0000 7fb69b7fe700 1 --2- 172.21.15.40:0/1004696638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb694006fe0 0x7fb694007470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb6a801b000 tx=0x7fb6a8004770 comp rx=0 tx=0).stop 2024-09-13T18:20:00.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.338+0000 7fb69b7fe700 1 -- 172.21.15.40:0/1004696638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb694006fe0 msgr2=0x7fb694007470 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:20:00.346 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.338+0000 7fb69b7fe700 1 --2- 172.21.15.40:0/1004696638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb694006fe0 0x7fb694007470 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb6a801b000 tx=0x7fb6a8004770 comp rx=0 tx=0).stop 2024-09-13T18:20:00.356 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:20:00.357 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:20:00.360 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:20:00.360 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:20:00.361 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 234, 2024-09-13T18:20:00.361 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:20:00.361 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:20:00.361 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:20:00.361 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:20:00.361 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:00.362 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:20:00.362 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:20:00.362 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:20:00.362 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:20:00.362 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:00.362 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:00.363 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:00.363 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:00.363 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:00.363 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:00.363 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:00.363 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:00.364 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:00.364 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:00.364 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:00.364 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:20:00.364 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:20:00.364 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:00.365 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:00.365 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:00.365 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:00.365 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:00.365 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:00.365 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:00.366 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:00.366 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:00.366 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:00.366 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:00.367 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.367 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:20:00.367 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:20:00.367 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:20:00.367 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:20:00.367 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:20:00.368 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:20:00.368 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:20:00.368 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:20:00.368 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:20:00.368 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:20:00.368 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:20:00.369 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:20:00.369 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:20:00.369 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:20:00.369 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:20:00.369 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:20:00.369 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:20:00.370 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:00.370 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:00.370 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:00.370 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:00.370 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:00.370 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:00.371 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:00.371 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:00.371 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:00.371 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:00.371 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:00.372 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:20:00.372 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.372 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:20:00.372 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.372 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:20:00.372 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:20:00.373 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:00.373 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:00.373 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.373 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:00.373 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:20:00.373 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:00.374 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.374 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.374 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:00.374 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:20:00.374 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:00.374 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:00.375 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:00.375 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.375 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:20:00.375 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:20:00.375 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:00.375 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:00.376 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:00.376 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.376 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.376 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:20:00.376 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:20:00.376 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:00.377 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:00.377 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.377 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:00.377 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:20:00.377 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:00.377 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.377 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.378 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:00.378 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:20:00.378 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:00.378 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:00.378 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:00.378 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.378 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:20:00.378 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:20:00.379 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:00.379 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:00.379 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:00.379 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.379 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.379 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:20:00.379 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:20:00.379 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:00.379 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:00.379 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.380 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:00.380 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:20:00.380 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:00.380 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.380 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.380 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:00.380 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:20:00.380 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:00.380 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:00.381 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:00.381 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.381 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:20:00.381 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:20:00.381 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:00.381 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:00.381 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:00.381 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:00.381 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:00.381 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.381 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:20:00.381 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:20:00.381 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.381 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:00.382 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:00.382 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:20:00.382 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:00.382 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:00.382 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:20:00.382 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.382 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:00.382 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:00.382 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:20:00.382 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:00.382 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:00.382 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:20:00.383 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.383 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:00.383 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:00.383 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:20:00.383 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:00.383 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:00.383 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:20:00.383 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:00.383 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:00.383 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:00.384 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:20:00.384 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:00.384 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:00.384 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:00.384 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:20:00.384 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:20:00.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.358+0000 7fb6b96f9700 1 -- 172.21.15.40:0/1004696638 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb6a0088790 msgr2=0x7fb6a008ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:00.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.358+0000 7fb6b96f9700 1 --2- 172.21.15.40:0/1004696638 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb6a0088790 0x7fb6a008ac50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40ca9b0 tx=0x7fb6ac009250 comp rx=0 tx=0).stop 2024-09-13T18:20:00.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.362+0000 7fb6b37fe700 1 -- 172.21.15.40:0/1004696638 reap_dead start 2024-09-13T18:20:00.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.362+0000 7fb6b96f9700 1 -- 172.21.15.40:0/1004696638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb6b40c8340 msgr2=0x7fb6b41ffd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:00.385 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.362+0000 7fb6b96f9700 1 --2- 172.21.15.40:0/1004696638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb6b40c8340 0x7fb6b41ffd20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6a401c000 tx=0x7fb6a4004770 comp rx=0 tx=0).stop 2024-09-13T18:20:00.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.362+0000 7fb6b96f9700 1 -- 172.21.15.40:0/1004696638 shutdown_connections 2024-09-13T18:20:00.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.362+0000 7fb6b96f9700 1 --2- 172.21.15.40:0/1004696638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb6b40c8340 0x7fb6b41ffd20 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb6a401c000 tx=0x7fb6a4004770 comp rx=0 tx=0).stop 2024-09-13T18:20:00.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.362+0000 7fb6b96f9700 1 -- 172.21.15.40:0/1004696638 >> 172.21.15.40:0/1004696638 conn(0x7fb6b4066090 msgr2=0x7fb6b41633c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:00.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.366+0000 7fb6b96f9700 1 -- 172.21.15.40:0/1004696638 shutdown_connections 2024-09-13T18:20:00.386 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.370+0000 7fb6b96f9700 1 -- 172.21.15.40:0/1004696638 wait complete. 2024-09-13T18:20:00.415 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:20:00.415 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:20:00.415 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:20:00.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.566+0000 7f70f759e700 1 Processor -- start 2024-09-13T18:20:00.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.566+0000 7f70f759e700 1 -- start start 2024-09-13T18:20:00.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70f81669e0 0x7f70f8160120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:00.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f759e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f70f8167600 0x7f70f8167a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:00.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f759e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f70f806a390 0x7f70f806a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:00.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f759e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f8071930 con 0x7f70f8167600 2024-09-13T18:20:00.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f759e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f8167f80 con 0x7f70f806a390 2024-09-13T18:20:00.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f8160660 con 0x7f70f81669e0 2024-09-13T18:20:00.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f5d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f70f8167600 0x7f70f8167a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:00.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f5d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f70f8167600 0x7f70f8167a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39314/0 (socket says 172.21.15.40:39314) 2024-09-13T18:20:00.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f5d9b700 1 -- 172.21.15.40:0/1541994573 learned_addr learned my addr 172.21.15.40:0/1541994573 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:00.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f659c700 1 --2- 172.21.15.40:0/1541994573 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70f81669e0 0x7f70f8160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:00.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f6d9d700 1 --2- 172.21.15.40:0/1541994573 >> v2:172.21.15.43:3300/0 conn(0x7f70f806a390 0x7f70f806a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:00.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f5d9b700 1 -- 172.21.15.40:0/1541994573 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70f81669e0 msgr2=0x7f70f8160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:20:00.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f5d9b700 1 --2- 172.21.15.40:0/1541994573 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70f81669e0 0x7f70f8160120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f5d9b700 1 -- 172.21.15.40:0/1541994573 >> v2:172.21.15.43:3300/0 conn(0x7f70f806a390 msgr2=0x7f70f806a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:20:00.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f5d9b700 1 --2- 172.21.15.40:0/1541994573 >> v2:172.21.15.43:3300/0 conn(0x7f70f806a390 0x7f70f806a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f5d9b700 1 -- 172.21.15.40:0/1541994573 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f8070d20 con 0x7f70f8167600 2024-09-13T18:20:00.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f5d9b700 1 --2- 172.21.15.40:0/1541994573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70f8167600 0x7f70f8167a40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f70f80c5260 tx=0x7f70f0009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6da536873f655d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:00.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f559a700 1 -- 172.21.15.40:0/1541994573 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0003c40 con 0x7f70f8167600 2024-09-13T18:20:00.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f559a700 1 -- 172.21.15.40:0/1541994573 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f000be90 con 0x7f70f8167600 2024-09-13T18:20:00.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f559a700 1 -- 172.21.15.40:0/1541994573 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0004030 con 0x7f70f8167600 2024-09-13T18:20:00.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f759e700 1 -- 172.21.15.40:0/1541994573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70f8167600 msgr2=0x7f70f8167a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:00.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.570+0000 7f70f759e700 1 --2- 172.21.15.40:0/1541994573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70f8167600 0x7f70f8167a40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f70f80c5260 tx=0x7f70f0009720 comp rx=0 tx=0).stop 2024-09-13T18:20:00.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.578+0000 7f70f759e700 1 -- 172.21.15.40:0/1541994573 shutdown_connections 2024-09-13T18:20:00.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.578+0000 7f70f759e700 1 --2- 172.21.15.40:0/1541994573 >> v2:172.21.15.43:3300/0 conn(0x7f70f806a390 0x7f70f806a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.578+0000 7f70f759e700 1 --2- 172.21.15.40:0/1541994573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70f8167600 0x7f70f8167a40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.578+0000 7f70f759e700 1 --2- 172.21.15.40:0/1541994573 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70f81669e0 0x7f70f8160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.578+0000 7f70f759e700 1 -- 172.21.15.40:0/1541994573 >> 172.21.15.40:0/1541994573 conn(0x7f70f80cabc0 msgr2=0x7f70f80cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:00.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.578+0000 7f70f759e700 1 -- 172.21.15.40:0/1541994573 shutdown_connections 2024-09-13T18:20:00.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.578+0000 7f70f759e700 1 -- 172.21.15.40:0/1541994573 wait complete. 2024-09-13T18:20:00.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.578+0000 7f70f759e700 1 Processor -- start 2024-09-13T18:20:00.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.578+0000 7f70f759e700 1 -- start start 2024-09-13T18:20:00.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70f759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70f806a390 0x7f70f8161b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:00.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70f759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f81669e0 0x7f70f80c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:00.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70f759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70f8167600 0x7f70f8157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:00.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70f759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f815b840 con 0x7f70f8167600 2024-09-13T18:20:00.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70f759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f81620c0 con 0x7f70f81669e0 2024-09-13T18:20:00.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70f759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f81fbcb0 con 0x7f70f806a390 2024-09-13T18:20:00.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70f6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70f8167600 0x7f70f8157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:00.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70f6d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70f8167600 0x7f70f8157320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39330/0 (socket says 172.21.15.40:39330) 2024-09-13T18:20:00.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70f6d9d700 1 -- 172.21.15.40:0/1810326873 learned_addr learned my addr 172.21.15.40:0/1810326873 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:00.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70f5d9b700 1 --2- 172.21.15.40:0/1810326873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f81669e0 0x7f70f80c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:00.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70f659c700 1 --2- 172.21.15.40:0/1810326873 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70f806a390 0x7f70f8161b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:00.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70f6d9d700 1 -- 172.21.15.40:0/1810326873 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70f806a390 msgr2=0x7f70f8161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:00.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70f6d9d700 1 --2- 172.21.15.40:0/1810326873 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70f806a390 0x7f70f8161b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70f6d9d700 1 -- 172.21.15.40:0/1810326873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f81669e0 msgr2=0x7f70f80c37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:00.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70f6d9d700 1 --2- 172.21.15.40:0/1810326873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f81669e0 0x7f70f80c37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:00.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70f6d9d700 1 -- 172.21.15.40:0/1810326873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f0009040 con 0x7f70f8167600 2024-09-13T18:20:00.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70f5d9b700 1 --2- 172.21.15.40:0/1810326873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f81669e0 0x7f70f80c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:00.593 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70f6d9d700 1 --2- 172.21.15.40:0/1810326873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70f8167600 0x7f70f8157320 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00fd60 tx=0x7f70ec015f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:00.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70e6ffd700 1 -- 172.21.15.40:0/1810326873 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec013ce0 con 0x7f70f8167600 2024-09-13T18:20:00.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70e6ffd700 1 -- 172.21.15.40:0/1810326873 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec017040 con 0x7f70f8167600 2024-09-13T18:20:00.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70e6ffd700 1 -- 172.21.15.40:0/1810326873 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec01d640 con 0x7f70f8167600 2024-09-13T18:20:00.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70f759e700 1 -- 172.21.15.40:0/1810326873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f81fbf70 con 0x7f70f8167600 2024-09-13T18:20:00.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.582+0000 7f70f759e700 1 -- 172.21.15.40:0/1810326873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f81fc530 con 0x7f70f8167600 2024-09-13T18:20:00.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.586+0000 7f70f759e700 1 -- 172.21.15.40:0/1810326873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f8072870 con 0x7f70f8167600 2024-09-13T18:20:00.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.594+0000 7f70e6ffd700 1 -- 172.21.15.40:0/1810326873 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f70ec027440 con 0x7f70f8167600 2024-09-13T18:20:00.602 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.594+0000 7f70e6ffd700 1 --2- 172.21.15.40:0/1810326873 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70d8066780 0x7f70d8068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:00.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.594+0000 7f70e6ffd700 1 -- 172.21.15.40:0/1810326873 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f70ec08c610 con 0x7f70f8167600 2024-09-13T18:20:00.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.594+0000 7f70e6ffd700 1 -- 172.21.15.40:0/1810326873 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f70ec0596a0 con 0x7f70f8167600 2024-09-13T18:20:00.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.594+0000 7f70f659c700 1 --2- 172.21.15.40:0/1810326873 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70d8066780 0x7f70d8068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:00.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:00.594+0000 7f70f659c700 1 --2- 172.21.15.40:0/1810326873 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70d8066780 0x7f70d8068c40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009c40 tx=0x7f70e8009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:01.018+0000 7f70f759e700 1 -- 172.21.15.40:0/1810326873 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f70f80704e0 con 0x7f70f8167600 2024-09-13T18:20:01.029 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:01.018+0000 7f70e6ffd700 1 -- 172.21.15.40:0/1810326873 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f70ec013e60 con 0x7f70f8167600 2024-09-13T18:20:01.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:01.034+0000 7f70e4ff9700 1 -- 172.21.15.40:0/1810326873 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70d8066780 msgr2=0x7f70d8068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:01.043 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:01.034+0000 7f70e4ff9700 1 --2- 172.21.15.40:0/1810326873 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70d8066780 0x7f70d8068c40 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009c40 tx=0x7f70e8009240 comp rx=0 tx=0).stop 2024-09-13T18:20:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:01.034+0000 7f70e4ff9700 1 -- 172.21.15.40:0/1810326873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70f8167600 msgr2=0x7f70f8157320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:01.034+0000 7f70e4ff9700 1 --2- 172.21.15.40:0/1810326873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70f8167600 0x7f70f8157320 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00fd60 tx=0x7f70ec015f90 comp rx=0 tx=0).stop 2024-09-13T18:20:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:01.034+0000 7f70e4ff9700 1 -- 172.21.15.40:0/1810326873 shutdown_connections 2024-09-13T18:20:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:01.034+0000 7f70e4ff9700 1 --2- 172.21.15.40:0/1810326873 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70f8167600 0x7f70f8157320 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:01.034+0000 7f70e4ff9700 1 --2- 172.21.15.40:0/1810326873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f81669e0 0x7f70f80c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:01.034+0000 7f70e4ff9700 1 --2- 172.21.15.40:0/1810326873 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f70d8066780 0x7f70d8068c40 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:01.034+0000 7f70e4ff9700 1 --2- 172.21.15.40:0/1810326873 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f70f806a390 0x7f70f8161b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:01.034+0000 7f70e4ff9700 1 -- 172.21.15.40:0/1810326873 >> 172.21.15.40:0/1810326873 conn(0x7f70f80cabc0 msgr2=0x7f70f806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:01.034+0000 7f70e4ff9700 1 -- 172.21.15.40:0/1810326873 shutdown_connections 2024-09-13T18:20:01.044 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:01.034+0000 7f70e4ff9700 1 -- 172.21.15.40:0/1810326873 wait complete. 2024-09-13T18:20:01.058 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:20:11.059 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:20:11.060 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:20:12.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.454+0000 7fe58e6b3700 1 Processor -- start 2024-09-13T18:20:12.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.454+0000 7fe58e6b3700 1 -- start start 2024-09-13T18:20:12.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe58e6b3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe58806aaa0 0x7fe588169100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:12.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe58e6b3700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe588071690 con 0x7fe588154240 2024-09-13T18:20:12.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe58e6b3700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe588154620 con 0x7fe58815e1a0 2024-09-13T18:20:12.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe58e6b3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5880721e0 con 0x7fe58806aaa0 2024-09-13T18:20:12.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe587fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe588154240 0x7fe58815ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58414/0 (socket says 172.21.15.40:58414) 2024-09-13T18:20:12.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe587fff700 1 -- 172.21.15.40:0/2101989265 learned_addr learned my addr 172.21.15.40:0/2101989265 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:20:12.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe58cc50700 1 --2- 172.21.15.40:0/2101989265 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe58806aaa0 0x7fe588169100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:12.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe586ffd700 1 -- 172.21.15.40:0/2101989265 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502791187 0 0) 0x7fe588154620 con 0x7fe58815e1a0 2024-09-13T18:20:12.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe586ffd700 1 -- 172.21.15.40:0/2101989265 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5700035d0 con 0x7fe58815e1a0 2024-09-13T18:20:12.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe586ffd700 1 -- 172.21.15.40:0/2101989265 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 212672123 0 0) 0x7fe588071690 con 0x7fe588154240 2024-09-13T18:20:12.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe586ffd700 1 -- 172.21.15.40:0/2101989265 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe588154620 con 0x7fe588154240 2024-09-13T18:20:12.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe58cc50700 1 -- 172.21.15.40:0/2101989265 >> v1:172.21.15.43:6789/0 conn(0x7fe58815e1a0 legacy=0x7fe58806a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:12.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe58cc50700 1 -- 172.21.15.40:0/2101989265 >> v1:172.21.15.40:6789/0 conn(0x7fe588154240 legacy=0x7fe58815ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:12.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe58cc50700 1 -- 172.21.15.40:0/2101989265 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe588071b20 con 0x7fe58806aaa0 2024-09-13T18:20:12.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe58cc50700 1 --2- 172.21.15.40:0/2101989265 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe58806aaa0 0x7fe588169100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe580009dc0 tx=0x7fe58000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26d95ed3765c4ec9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:12.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe586ffd700 1 -- 172.21.15.40:0/2101989265 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58000e070 con 0x7fe58806aaa0 2024-09-13T18:20:12.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe586ffd700 1 -- 172.21.15.40:0/2101989265 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58000bb20 con 0x7fe58806aaa0 2024-09-13T18:20:12.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe58e6b3700 1 -- 172.21.15.40:0/2101989265 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe58806aaa0 msgr2=0x7fe588169100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:12.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe58e6b3700 1 --2- 172.21.15.40:0/2101989265 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe58806aaa0 0x7fe588169100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe580009dc0 tx=0x7fe58000b330 comp rx=0 tx=0).stop 2024-09-13T18:20:12.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe58e6b3700 1 -- 172.21.15.40:0/2101989265 shutdown_connections 2024-09-13T18:20:12.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe58e6b3700 1 --2- 172.21.15.40:0/2101989265 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe58806aaa0 0x7fe588169100 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:12.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.458+0000 7fe58e6b3700 1 -- 172.21.15.40:0/2101989265 >> 172.21.15.40:0/2101989265 conn(0x7fe5880cabc0 msgr2=0x7fe5880cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:12.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.466+0000 7fe58e6b3700 1 -- 172.21.15.40:0/2101989265 shutdown_connections 2024-09-13T18:20:12.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.466+0000 7fe58e6b3700 1 -- 172.21.15.40:0/2101989265 wait complete. 2024-09-13T18:20:12.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe58e6b3700 1 Processor -- start 2024-09-13T18:20:12.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe58e6b3700 1 -- start start 2024-09-13T18:20:12.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe58e6b3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58806aaa0 0x7fe588154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:12.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe58e6b3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe588154240 0x7fe588154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:12.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe58e6b3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe58815e1a0 0x7fe5881425f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:12.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe58e6b3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58814c370 con 0x7fe58806aaa0 2024-09-13T18:20:12.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe58e6b3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5880b8280 con 0x7fe58815e1a0 2024-09-13T18:20:12.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe58e6b3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe588150970 con 0x7fe588154240 2024-09-13T18:20:12.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe587fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58806aaa0 0x7fe588154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:12.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe587fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58806aaa0 0x7fe588154800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54206/0 (socket says 172.21.15.40:54206) 2024-09-13T18:20:12.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe587fff700 1 -- 172.21.15.40:0/1163053150 learned_addr learned my addr 172.21.15.40:0/1163053150 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:12.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe5877fe700 1 --2- 172.21.15.40:0/1163053150 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe588154240 0x7fe588154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:12.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe58cc50700 1 --2- 172.21.15.40:0/1163053150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe58815e1a0 0x7fe5881425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:12.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe58cc50700 1 -- 172.21.15.40:0/1163053150 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe588154240 msgr2=0x7fe588154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:12.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe58cc50700 1 --2- 172.21.15.40:0/1163053150 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe588154240 0x7fe588154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:12.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe58cc50700 1 -- 172.21.15.40:0/1163053150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58806aaa0 msgr2=0x7fe588154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:12.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe58cc50700 1 --2- 172.21.15.40:0/1163053150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58806aaa0 0x7fe588154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:12.482 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe58cc50700 1 -- 172.21.15.40:0/1163053150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe580009a70 con 0x7fe58815e1a0 2024-09-13T18:20:12.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe587fff700 1 --2- 172.21.15.40:0/1163053150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58806aaa0 0x7fe588154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:20:12.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe58cc50700 1 --2- 172.21.15.40:0/1163053150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe58815e1a0 0x7fe5881425f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe580016000 tx=0x7fe580004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:12.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe584ff9700 1 -- 172.21.15.40:0/1163053150 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58001b810 con 0x7fe58815e1a0 2024-09-13T18:20:12.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe584ff9700 1 -- 172.21.15.40:0/1163053150 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe580020070 con 0x7fe58815e1a0 2024-09-13T18:20:12.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe58e6b3700 1 -- 172.21.15.40:0/1163053150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe588150c50 con 0x7fe58815e1a0 2024-09-13T18:20:12.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe584ff9700 1 -- 172.21.15.40:0/1163053150 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58001bbb0 con 0x7fe58815e1a0 2024-09-13T18:20:12.483 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.470+0000 7fe58e6b3700 1 -- 172.21.15.40:0/1163053150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe588151210 con 0x7fe58815e1a0 2024-09-13T18:20:12.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.474+0000 7fe58e6b3700 1 -- 172.21.15.40:0/1163053150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5880689c0 con 0x7fe58815e1a0 2024-09-13T18:20:12.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.474+0000 7fe584ff9700 1 -- 172.21.15.40:0/1163053150 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe58001bd10 con 0x7fe58815e1a0 2024-09-13T18:20:12.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.474+0000 7fe584ff9700 1 --2- 172.21.15.40:0/1163053150 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe574066780 0x7fe574068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:12.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.474+0000 7fe584ff9700 1 -- 172.21.15.40:0/1163053150 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe580022070 con 0x7fe58815e1a0 2024-09-13T18:20:12.486 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.478+0000 7fe587fff700 1 --2- 172.21.15.40:0/1163053150 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe574066780 0x7fe574068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:12.491 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.482+0000 7fe587fff700 1 --2- 172.21.15.40:0/1163053150 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe574066780 0x7fe574068c40 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0011e0 tx=0x7fe57c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:12.492 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.482+0000 7fe584ff9700 1 -- 172.21.15.40:0/1163053150 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe580053430 con 0x7fe58815e1a0 2024-09-13T18:20:12.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.886+0000 7fe58e6b3700 1 -- 172.21.15.40:0/1163053150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fe588071320 con 0x7fe58815e1a0 2024-09-13T18:20:12.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.886+0000 7fe584ff9700 1 -- 172.21.15.40:0/1163053150 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7fe580056a50 con 0x7fe58815e1a0 2024-09-13T18:20:12.894 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":234,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:20:12.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.886+0000 7fe58e6b3700 1 -- 172.21.15.40:0/1163053150 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe574066780 msgr2=0x7fe574068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:12.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.886+0000 7fe58e6b3700 1 --2- 172.21.15.40:0/1163053150 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe574066780 0x7fe574068c40 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0011e0 tx=0x7fe57c009040 comp rx=0 tx=0).stop 2024-09-13T18:20:12.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.886+0000 7fe58e6b3700 1 -- 172.21.15.40:0/1163053150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe58815e1a0 msgr2=0x7fe5881425f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:12.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.886+0000 7fe58e6b3700 1 --2- 172.21.15.40:0/1163053150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe58815e1a0 0x7fe5881425f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe580016000 tx=0x7fe580004af0 comp rx=0 tx=0).stop 2024-09-13T18:20:12.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.890+0000 7fe58e6b3700 1 -- 172.21.15.40:0/1163053150 shutdown_connections 2024-09-13T18:20:12.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.890+0000 7fe58e6b3700 1 --2- 172.21.15.40:0/1163053150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe58815e1a0 0x7fe5881425f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:12.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.890+0000 7fe58e6b3700 1 --2- 172.21.15.40:0/1163053150 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe574066780 0x7fe574068c40 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:12.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.890+0000 7fe58e6b3700 1 --2- 172.21.15.40:0/1163053150 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe588154240 0x7fe588154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:12.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.890+0000 7fe58e6b3700 1 --2- 172.21.15.40:0/1163053150 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe58806aaa0 0x7fe588154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:12.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.890+0000 7fe58e6b3700 1 -- 172.21.15.40:0/1163053150 >> 172.21.15.40:0/1163053150 conn(0x7fe5880cabc0 msgr2=0x7fe5880cb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:12.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.890+0000 7fe58e6b3700 1 -- 172.21.15.40:0/1163053150 shutdown_connections 2024-09-13T18:20:12.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:12.890+0000 7fe58e6b3700 1 -- 172.21.15.40:0/1163053150 wait complete. 2024-09-13T18:20:12.916 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:20:12.916 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-13T18:20:12.916 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:20:12.916 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:20:13.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.074+0000 7fce05df8700 1 Processor -- start 2024-09-13T18:20:13.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.074+0000 7fce05df8700 1 -- start start 2024-09-13T18:20:13.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.074+0000 7fce05df8700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fce0014f8c0 0x7fce000c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.074+0000 7fce05df8700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce000c6890 con 0x7fce00158bc0 2024-09-13T18:20:13.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.074+0000 7fce05df8700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce000c5f30 con 0x7fce000724e0 2024-09-13T18:20:13.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.074+0000 7fce05df8700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce000c7700 con 0x7fce0014f8c0 2024-09-13T18:20:13.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.074+0000 7fce055f7700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fce00158bc0 0x7fce0015ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58416/0 (socket says 172.21.15.40:58416) 2024-09-13T18:20:13.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.074+0000 7fce055f7700 1 -- 172.21.15.40:0/3543211172 learned_addr learned my addr 172.21.15.40:0/3543211172 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:20:13.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.074+0000 7fce04df6700 1 --2- 172.21.15.40:0/3543211172 >> v2:172.21.15.40:3300/0 conn(0x7fce0014f8c0 0x7fce000c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.078+0000 7fcdff7fe700 1 -- 172.21.15.40:0/3543211172 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960248844 0 0) 0x7fce000c6890 con 0x7fce00158bc0 2024-09-13T18:20:13.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.078+0000 7fcdff7fe700 1 -- 172.21.15.40:0/3543211172 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcde80035d0 con 0x7fce00158bc0 2024-09-13T18:20:13.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.078+0000 7fcdff7fe700 1 -- 172.21.15.40:0/3543211172 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2240657648 0 0) 0x7fcde80035d0 con 0x7fce00158bc0 2024-09-13T18:20:13.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.078+0000 7fce04df6700 1 -- 172.21.15.40:0/3543211172 >> v1:172.21.15.43:6789/0 conn(0x7fce000724e0 legacy=0x7fce0006d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.078+0000 7fce04df6700 1 -- 172.21.15.40:0/3543211172 >> v1:172.21.15.40:6789/0 conn(0x7fce00158bc0 legacy=0x7fce0015ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.078+0000 7fce04df6700 1 -- 172.21.15.40:0/3543211172 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce000c1320 con 0x7fce0014f8c0 2024-09-13T18:20:13.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.078+0000 7fce04df6700 1 --2- 172.21.15.40:0/3543211172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0014f8c0 0x7fce000c71c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009850 tx=0x7fcdf0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820f552b7106b56 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.078+0000 7fcdff7fe700 1 -- 172.21.15.40:0/3543211172 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4001650 con 0x7fce0014f8c0 2024-09-13T18:20:13.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.078+0000 7fcdff7fe700 1 -- 172.21.15.40:0/3543211172 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf0004030 con 0x7fce0014f8c0 2024-09-13T18:20:13.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.078+0000 7fcdff7fe700 1 -- 172.21.15.40:0/3543211172 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf00043a0 con 0x7fce0014f8c0 2024-09-13T18:20:13.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.078+0000 7fce05df8700 1 -- 172.21.15.40:0/3543211172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0014f8c0 msgr2=0x7fce000c71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.078+0000 7fce05df8700 1 --2- 172.21.15.40:0/3543211172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0014f8c0 0x7fce000c71c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009850 tx=0x7fcdf0009b60 comp rx=0 tx=0).stop 2024-09-13T18:20:13.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.078+0000 7fce05df8700 1 -- 172.21.15.40:0/3543211172 shutdown_connections 2024-09-13T18:20:13.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.078+0000 7fce05df8700 1 --2- 172.21.15.40:0/3543211172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0014f8c0 0x7fce000c71c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.078+0000 7fce05df8700 1 -- 172.21.15.40:0/3543211172 >> 172.21.15.40:0/3543211172 conn(0x7fce00066090 msgr2=0x7fce000664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:13.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.078+0000 7fce05df8700 1 -- 172.21.15.40:0/3543211172 shutdown_connections 2024-09-13T18:20:13.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.078+0000 7fce05df8700 1 -- 172.21.15.40:0/3543211172 wait complete. 2024-09-13T18:20:13.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.078+0000 7fce05df8700 1 Processor -- start 2024-09-13T18:20:13.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.078+0000 7fce05df8700 1 -- start start 2024-09-13T18:20:13.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fce05df8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce000724e0 0x7fce00151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fce05df8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0014f8c0 0x7fce00150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fce05df8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce00158bc0 0x7fce0014a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fce05df8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce002001d0 con 0x7fce0014f8c0 2024-09-13T18:20:13.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fce05df8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce00152e30 con 0x7fce00158bc0 2024-09-13T18:20:13.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fce05df8700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0014aba0 con 0x7fce000724e0 2024-09-13T18:20:13.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fce055f7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce00158bc0 0x7fce0014a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fce055f7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce00158bc0 0x7fce0014a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:33158/0 (socket says 172.21.15.40:33158) 2024-09-13T18:20:13.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fce055f7700 1 -- 172.21.15.40:0/2436140117 learned_addr learned my addr 172.21.15.40:0/2436140117 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:13.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fce055f7700 1 -- 172.21.15.40:0/2436140117 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce000724e0 msgr2=0x7fce00151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fce04df6700 1 --2- 172.21.15.40:0/2436140117 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce000724e0 0x7fce00151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fce055f7700 1 --2- 172.21.15.40:0/2436140117 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fce000724e0 0x7fce00151840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fce055f7700 1 -- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0014f8c0 msgr2=0x7fce00150a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:20:13.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fce055f7700 1 --2- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0014f8c0 0x7fce00150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fce055f7700 1 -- 172.21.15.40:0/2436140117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf00094b0 con 0x7fce00158bc0 2024-09-13T18:20:13.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fce055f7700 1 --2- 172.21.15.40:0/2436140117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce00158bc0 0x7fce0014a180 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400c570 tx=0x7fcdf400c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fcdfd7fa700 1 -- 172.21.15.40:0/2436140117 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4004ca0 con 0x7fce00158bc0 2024-09-13T18:20:13.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fcdfd7fa700 1 -- 172.21.15.40:0/2436140117 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf4004e20 con 0x7fce00158bc0 2024-09-13T18:20:13.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fce05df8700 1 -- 172.21.15.40:0/2436140117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce00155c80 con 0x7fce00158bc0 2024-09-13T18:20:13.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fcdfd7fa700 1 -- 172.21.15.40:0/2436140117 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4007310 con 0x7fce00158bc0 2024-09-13T18:20:13.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.086+0000 7fce05df8700 1 -- 172.21.15.40:0/2436140117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce00150fc0 con 0x7fce00158bc0 2024-09-13T18:20:13.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.090+0000 7fcdfd7fa700 1 -- 172.21.15.40:0/2436140117 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcdf4007470 con 0x7fce00158bc0 2024-09-13T18:20:13.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.094+0000 7fcdfd7fa700 1 --2- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcdec066730 0x7fcdec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.094+0000 7fce04df6700 1 --2- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcdec066730 0x7fcdec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.094+0000 7fcdfd7fa700 1 -- 172.21.15.40:0/2436140117 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcdf40842b0 con 0x7fce00158bc0 2024-09-13T18:20:13.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.094+0000 7fce04df6700 1 --2- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcdec066730 0x7fcdec068bf0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0000c00 tx=0x7fcdf0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.098+0000 7fce05df8700 1 --2- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0014a6c0 0x7fce00200320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.098+0000 7fce05df8700 1 -- 172.21.15.40:0/2436140117 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce00067720 con 0x7fce0014a6c0 2024-09-13T18:20:13.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.098+0000 7fcdfffff700 1 --2- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0014a6c0 0x7fce00200320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.098+0000 7fcdfffff700 1 -- 172.21.15.40:0/2436140117 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fce0014fca0 con 0x7fce0014a6c0 2024-09-13T18:20:13.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.098+0000 7fcdfffff700 1 --2- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0014a6c0 0x7fce00200320 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fce00157a60 tx=0x7fcdf800cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.098+0000 7fcdfd7fa700 1 -- 172.21.15.40:0/2436140117 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8003c20 con 0x7fce0014a6c0 2024-09-13T18:20:13.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.102+0000 7fcdfd7fa700 1 -- 172.21.15.40:0/2436140117 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fcdf8003d80 con 0x7fce0014a6c0 2024-09-13T18:20:13.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.102+0000 7fcdfd7fa700 1 -- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0014a6c0 msgr2=0x7fce00200320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.102+0000 7fcdfd7fa700 1 --2- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0014a6c0 0x7fce00200320 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fce00157a60 tx=0x7fcdf800cea0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.102+0000 7fcdfd7fa700 1 -- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0014a6c0 msgr2=0x7fce00200320 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:20:13.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.102+0000 7fcdfd7fa700 1 --2- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce0014a6c0 0x7fce00200320 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.114+0000 7fce05df8700 1 --2- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce00159240 0x7fce0014fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.114+0000 7fce05df8700 1 -- 172.21.15.40:0/2436140117 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce000c5c50 con 0x7fce00159240 2024-09-13T18:20:13.124 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.114+0000 7fcdfffff700 1 --2- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce00159240 0x7fce0014fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.114+0000 7fcdfffff700 1 -- 172.21.15.40:0/2436140117 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fce000c6550 con 0x7fce00159240 2024-09-13T18:20:13.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.114+0000 7fcdfffff700 1 --2- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce00159240 0x7fce0014fee0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80012e0 tx=0x7fcdf8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.125 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.114+0000 7fcdfd7fa700 1 -- 172.21.15.40:0/2436140117 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8020070 con 0x7fce00159240 2024-09-13T18:20:13.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.118+0000 7fcdfd7fa700 1 -- 172.21.15.40:0/2436140117 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4453 (secure 0 0 0) 0x7fcdf801bb30 con 0x7fce00159240 2024-09-13T18:20:13.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.118+0000 7fcdfd7fa700 1 -- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce00159240 msgr2=0x7fce0014fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.118+0000 7fcdfd7fa700 1 --2- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce00159240 0x7fce0014fee0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80012e0 tx=0x7fcdf8001250 comp rx=0 tx=0).stop 2024-09-13T18:20:13.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.118+0000 7fcdfd7fa700 1 -- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce00159240 msgr2=0x7fce0014fee0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:20:13.126 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.118+0000 7fcdfd7fa700 1 --2- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fce00159240 0x7fce0014fee0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.126 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:20:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:20:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:20:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:20:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 234, 2024-09-13T18:20:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:20:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:20:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:20:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:20:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 14, 2024-09-13T18:20:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:20:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:20:13.127 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:20:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:20:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:20:13.128 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:20:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:20:13.129 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:20:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:20:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:20:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:20:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:20:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:20:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:20:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:20:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:20:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:20:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:20:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:20:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:20:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:20:13.130 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:20:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:20:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:20:13.131 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:20:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:20:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:20:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:20:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:13.132 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:20:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:20:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:20:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:20:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:13.133 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:20:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:20:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:20:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:20:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:13.134 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:20:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:20:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.135 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:20:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:20:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:20:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:20:13.136 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:20:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:20:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:20:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:20:13.137 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:20:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.138 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:20:13.138 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:20:13.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.118+0000 7fce05df8700 1 -- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcdec066730 msgr2=0x7fcdec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.118+0000 7fce05df8700 1 --2- 172.21.15.40:0/2436140117 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcdec066730 0x7fcdec068bf0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0000c00 tx=0x7fcdf0008030 comp rx=0 tx=0).stop 2024-09-13T18:20:13.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.118+0000 7fce05df8700 1 -- 172.21.15.40:0/2436140117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce00158bc0 msgr2=0x7fce0014a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.118+0000 7fce05df8700 1 --2- 172.21.15.40:0/2436140117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce00158bc0 0x7fce0014a180 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400c570 tx=0x7fcdf400c930 comp rx=0 tx=0).stop 2024-09-13T18:20:13.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.118+0000 7fce055f7700 1 -- 172.21.15.40:0/2436140117 reap_dead start 2024-09-13T18:20:13.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.118+0000 7fce055f7700 1 -- 172.21.15.40:0/2436140117 reap_dead start 2024-09-13T18:20:13.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.118+0000 7fce05df8700 1 -- 172.21.15.40:0/2436140117 shutdown_connections 2024-09-13T18:20:13.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.118+0000 7fce05df8700 1 -- 172.21.15.40:0/2436140117 >> 172.21.15.40:0/2436140117 conn(0x7fce00066090 msgr2=0x7fce00156c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:13.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.118+0000 7fce05df8700 1 -- 172.21.15.40:0/2436140117 shutdown_connections 2024-09-13T18:20:13.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.118+0000 7fce05df8700 1 -- 172.21.15.40:0/2436140117 wait complete. 2024-09-13T18:20:13.146 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:20:13.147 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:20:13.147 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:20:13.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdba54da700 1 Processor -- start 2024-09-13T18:20:13.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdba54da700 1 -- start start 2024-09-13T18:20:13.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdba54da700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdba0157d10 0x7fdba00b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdba54da700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdba0161c60 0x7fdba00c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdba54da700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba00b1000 con 0x7fdba006afb0 2024-09-13T18:20:13.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdba54da700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba00729e0 con 0x7fdba0161c60 2024-09-13T18:20:13.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdba54da700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba00c6c20 con 0x7fdba0157d10 2024-09-13T18:20:13.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdb9f7fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdba006afb0 0x7fdba006a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:36566/0 (socket says 172.21.15.40:36566) 2024-09-13T18:20:13.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdb9f7fe700 1 -- 172.21.15.40:0/3427303640 learned_addr learned my addr 172.21.15.40:0/3427303640 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:20:13.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdb9dffb700 1 -- 172.21.15.40:0/3427303640 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037153510 0 0) 0x7fdba00b1000 con 0x7fdba006afb0 2024-09-13T18:20:13.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdb9dffb700 1 -- 172.21.15.40:0/3427303640 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb880035d0 con 0x7fdba006afb0 2024-09-13T18:20:13.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdb9dffb700 1 -- 172.21.15.40:0/3427303640 <== mon.1 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2748928102 0 0) 0x7fdb880035d0 con 0x7fdba006afb0 2024-09-13T18:20:13.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdb9dffb700 1 -- 172.21.15.40:0/3427303640 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdba0157d10 msgr2=0x7fdba00b0ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:20:13.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdb9dffb700 1 --2- 172.21.15.40:0/3427303640 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdba0157d10 0x7fdba00b0ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdb9dffb700 1 -- 172.21.15.40:0/3427303640 >> v2:172.21.15.40:3300/0 conn(0x7fdba0161c60 msgr2=0x7fdba00c5860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:20:13.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdb9dffb700 1 --2- 172.21.15.40:0/3427303640 >> v2:172.21.15.40:3300/0 conn(0x7fdba0161c60 0x7fdba00c5860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdb9dffb700 1 -- 172.21.15.40:0/3427303640 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba01617d0 con 0x7fdba006afb0 2024-09-13T18:20:13.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdb9dffb700 1 -- 172.21.15.40:0/3427303640 <== mon.1 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fdb90001670 con 0x7fdba006afb0 2024-09-13T18:20:13.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdb9dffb700 1 -- 172.21.15.40:0/3427303640 >> v1:172.21.15.43:6789/0 conn(0x7fdba006afb0 legacy=0x7fdba006a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdb9dffb700 1 --2- 172.21.15.40:0/3427303640 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdb88003a50 0x7fdb88003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdb9dffb700 1 --2- 172.21.15.40:0/3427303640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb88004870 0x7fdb88004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdb9dffb700 1 --2- 172.21.15.40:0/3427303640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb880052a0 0x7fdb8802da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdb9dffb700 1 -- 172.21.15.40:0/3427303640 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba00729e0 con 0x7fdb88004870 2024-09-13T18:20:13.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdb9dffb700 1 -- 172.21.15.40:0/3427303640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba00c6c20 con 0x7fdb880052a0 2024-09-13T18:20:13.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.302+0000 7fdb9dffb700 1 -- 172.21.15.40:0/3427303640 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8802e240 con 0x7fdb88003a50 2024-09-13T18:20:13.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.306+0000 7fdb9f7fe700 1 --2- 172.21.15.40:0/3427303640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb880052a0 0x7fdb8802da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.306+0000 7fdb9e7fc700 1 --2- 172.21.15.40:0/3427303640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb88004870 0x7fdb88004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.306+0000 7fdb9f7fe700 1 -- 172.21.15.40:0/3427303640 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdb88003a50 msgr2=0x7fdb88003e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:20:13.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.306+0000 7fdb9f7fe700 1 --2- 172.21.15.40:0/3427303640 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdb88003a50 0x7fdb88003e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.306+0000 7fdb9f7fe700 1 -- 172.21.15.40:0/3427303640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb88004870 msgr2=0x7fdb88004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:20:13.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.306+0000 7fdb9f7fe700 1 --2- 172.21.15.40:0/3427303640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb88004870 0x7fdb88004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.306+0000 7fdb9f7fe700 1 -- 172.21.15.40:0/3427303640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb880035d0 con 0x7fdb880052a0 2024-09-13T18:20:13.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.306+0000 7fdb9e7fc700 1 --2- 172.21.15.40:0/3427303640 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb88004870 0x7fdb88004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:20:13.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.306+0000 7fdb9f7fe700 1 --2- 172.21.15.40:0/3427303640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb880052a0 0x7fdb8802da40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000c2f0 tx=0x7fdb880058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65cfbac9baa2cb6c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.310+0000 7fdb9dffb700 1 -- 172.21.15.40:0/3427303640 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9000ce30 con 0x7fdb880052a0 2024-09-13T18:20:13.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.310+0000 7fdb9dffb700 1 -- 172.21.15.40:0/3427303640 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb90007bc0 con 0x7fdb880052a0 2024-09-13T18:20:13.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.310+0000 7fdb9dffb700 1 -- 172.21.15.40:0/3427303640 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb90005bc0 con 0x7fdb880052a0 2024-09-13T18:20:13.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.310+0000 7fdba54da700 1 -- 172.21.15.40:0/3427303640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb880052a0 msgr2=0x7fdb8802da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.310+0000 7fdba54da700 1 --2- 172.21.15.40:0/3427303640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb880052a0 0x7fdb8802da40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000c2f0 tx=0x7fdb880058a0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.321 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.310+0000 7fdb9f7fe700 1 -- 172.21.15.40:0/3427303640 reap_dead start 2024-09-13T18:20:13.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.314+0000 7fdba54da700 1 -- 172.21.15.40:0/3427303640 shutdown_connections 2024-09-13T18:20:13.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.314+0000 7fdba54da700 1 -- 172.21.15.40:0/3427303640 >> 172.21.15.40:0/3427303640 conn(0x7fdba006d5b0 msgr2=0x7fdba006d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:13.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.318+0000 7fdba54da700 1 -- 172.21.15.40:0/3427303640 shutdown_connections 2024-09-13T18:20:13.329 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.318+0000 7fdba54da700 1 -- 172.21.15.40:0/3427303640 wait complete. 2024-09-13T18:20:13.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.322+0000 7fdba54da700 1 Processor -- start 2024-09-13T18:20:13.330 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.322+0000 7fdba54da700 1 -- start start 2024-09-13T18:20:13.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.322+0000 7fdba54da700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb880052a0 0x7fdba00c3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.322+0000 7fdba54da700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdba00c2120 0x7fdba015df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.322+0000 7fdba54da700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdba015bb80 0x7fdba015c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.322+0000 7fdba54da700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba015da00 con 0x7fdb880052a0 2024-09-13T18:20:13.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.322+0000 7fdba54da700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba015c540 con 0x7fdba00c2120 2024-09-13T18:20:13.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.322+0000 7fdba54da700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba00ca100 con 0x7fdba015bb80 2024-09-13T18:20:13.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.322+0000 7fdb9f7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdba015bb80 0x7fdba015c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.322+0000 7fdb9f7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdba015bb80 0x7fdba015c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:34696/0 (socket says 172.21.15.40:34696) 2024-09-13T18:20:13.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.322+0000 7fdb9f7fe700 1 -- 172.21.15.40:0/2448107495 learned_addr learned my addr 172.21.15.40:0/2448107495 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:13.332 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.322+0000 7fdb9e7fc700 1 --2- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdba00c2120 0x7fdba015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.322+0000 7fdb9f7fe700 1 -- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdba00c2120 msgr2=0x7fdba015df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.322+0000 7fdb9f7fe700 1 --2- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdba00c2120 0x7fdba015df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.322+0000 7fdb9f7fe700 1 -- 172.21.15.40:0/2448107495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb880052a0 msgr2=0x7fdba00c3000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:20:13.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.322+0000 7fdb9f7fe700 1 --2- 172.21.15.40:0/2448107495 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdb880052a0 0x7fdba00c3000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.322+0000 7fdb9f7fe700 1 -- 172.21.15.40:0/2448107495 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba015d740 con 0x7fdba015bb80 2024-09-13T18:20:13.333 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.322+0000 7fdb9f7fe700 1 --2- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdba015bb80 0x7fdba015c000 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdb8802e210 tx=0x7fdba0070c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.326+0000 7fdb8f7fe700 1 -- 172.21.15.40:0/2448107495 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb90001660 con 0x7fdba015bb80 2024-09-13T18:20:13.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.326+0000 7fdb8f7fe700 1 -- 172.21.15.40:0/2448107495 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb900070d0 con 0x7fdba015bb80 2024-09-13T18:20:13.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.326+0000 7fdba54da700 1 -- 172.21.15.40:0/2448107495 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba01570c0 con 0x7fdba015bb80 2024-09-13T18:20:13.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.326+0000 7fdba54da700 1 -- 172.21.15.40:0/2448107495 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba0157680 con 0x7fdba015bb80 2024-09-13T18:20:13.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.326+0000 7fdb8f7fe700 1 -- 172.21.15.40:0/2448107495 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb90007480 con 0x7fdba015bb80 2024-09-13T18:20:13.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.326+0000 7fdba54da700 1 --2- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdba015b5c0 0x7fdba0149610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.336 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.326+0000 7fdba54da700 1 -- 172.21.15.40:0/2448107495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba015db70 con 0x7fdba015b5c0 2024-09-13T18:20:13.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.326+0000 7fdb9effd700 1 --2- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdba015b5c0 0x7fdba0149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.326+0000 7fdb8f7fe700 1 -- 172.21.15.40:0/2448107495 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdb900075e0 con 0x7fdba015bb80 2024-09-13T18:20:13.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.326+0000 7fdb9effd700 1 -- 172.21.15.40:0/2448107495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdba0070850 con 0x7fdba015b5c0 2024-09-13T18:20:13.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.326+0000 7fdb8f7fe700 1 --2- 172.21.15.40:0/2448107495 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdb840667b0 0x7fdb84068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.326+0000 7fdb8f7fe700 1 -- 172.21.15.40:0/2448107495 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdb90084980 con 0x7fdba015bb80 2024-09-13T18:20:13.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.326+0000 7fdb9effd700 1 --2- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdba015b5c0 0x7fdba0149610 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdba00686b0 tx=0x7fdb98009e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.330+0000 7fdb9e7fc700 1 --2- 172.21.15.40:0/2448107495 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdb840667b0 0x7fdb84068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.330+0000 7fdb8f7fe700 1 -- 172.21.15.40:0/2448107495 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9800cad0 con 0x7fdba015b5c0 2024-09-13T18:20:13.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.330+0000 7fdb8f7fe700 1 -- 172.21.15.40:0/2448107495 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fdb9800cc30 con 0x7fdba015b5c0 2024-09-13T18:20:13.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.330+0000 7fdb8f7fe700 1 -- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdba015b5c0 msgr2=0x7fdba0149610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.330+0000 7fdb8f7fe700 1 --2- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdba015b5c0 0x7fdba0149610 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdba00686b0 tx=0x7fdb98009e50 comp rx=0 tx=0).stop 2024-09-13T18:20:13.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.330+0000 7fdb8f7fe700 1 -- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdba015b5c0 msgr2=0x7fdba0149610 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:20:13.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.330+0000 7fdb8f7fe700 1 --2- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdba015b5c0 0x7fdba0149610 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdba00686b0 tx=0x7fdb98009e50 comp rx=0 tx=0).stop 2024-09-13T18:20:13.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.334+0000 7fdb9e7fc700 1 --2- 172.21.15.40:0/2448107495 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdb840667b0 0x7fdb84068c70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fdba0160c50 tx=0x7fdb94009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.349 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdba54da700 1 --2- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdba0149b50 0x7fdba01ee180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdba54da700 1 -- 172.21.15.40:0/2448107495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba0067840 con 0x7fdba0149b50 2024-09-13T18:20:13.350 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdb9effd700 1 --2- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdba0149b50 0x7fdba01ee180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdb9effd700 1 -- 172.21.15.40:0/2448107495 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fdba00b10b0 con 0x7fdba0149b50 2024-09-13T18:20:13.351 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdb9effd700 1 --2- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdba0149b50 0x7fdba01ee180 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdba00686b0 tx=0x7fdb98004ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdb8f7fe700 1 -- 172.21.15.40:0/2448107495 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9801cae0 con 0x7fdba0149b50 2024-09-13T18:20:13.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdb8f7fe700 1 -- 172.21.15.40:0/2448107495 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4949 (secure 0 0 0) 0x7fdb98020070 con 0x7fdba0149b50 2024-09-13T18:20:13.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdb8f7fe700 1 -- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdba0149b50 msgr2=0x7fdba01ee180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdb8f7fe700 1 --2- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdba0149b50 0x7fdba01ee180 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdba00686b0 tx=0x7fdb98004ae0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdb8f7fe700 1 -- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdba0149b50 msgr2=0x7fdba01ee180 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:20:13.352 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdb8f7fe700 1 --2- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdba0149b50 0x7fdba01ee180 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdba00686b0 tx=0x7fdb98004ae0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.353 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:20:13.353 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:20:13.353 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:20:13.353 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:20:13.353 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 234, 2024-09-13T18:20:13.353 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:20:13.353 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:20:13.353 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:20:13.353 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:20:13.353 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 14, 2024-09-13T18:20:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:20:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:20:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:20:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:13.354 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:13.355 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:20:13.355 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:20:13.355 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:13.355 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:13.355 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:13.355 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:13.355 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:13.355 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:13.355 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:13.355 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:13.355 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:13.356 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:13.356 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:13.356 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.356 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:20:13.356 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:20:13.356 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:20:13.356 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:20:13.356 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:20:13.356 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:20:13.356 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:20:13.356 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:20:13.356 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:20:13.357 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:20:13.357 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:20:13.357 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:20:13.357 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:20:13.357 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:20:13.357 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:20:13.357 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:20:13.357 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:20:13.357 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:13.357 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:13.357 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:13.358 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:13.358 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:13.358 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:13.358 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:13.358 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:13.358 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:13.358 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:13.358 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:13.358 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:20:13.358 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.358 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:20:13.358 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.358 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:20:13.358 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:20:13.359 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:13.359 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:13.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.359 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:13.359 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:20:13.359 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:13.359 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.359 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.359 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:13.359 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:20:13.359 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:13.359 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.359 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:13.360 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.360 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:20:13.360 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:20:13.360 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:13.360 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:13.360 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:13.360 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.360 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.360 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:20:13.360 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:20:13.360 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:13.360 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:13.360 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.360 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:13.361 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:20:13.361 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:13.361 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.361 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.361 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:13.361 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:20:13.361 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:13.361 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.361 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:13.361 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.361 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:20:13.361 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:20:13.361 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:13.362 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:13.362 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:13.362 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.362 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.362 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:20:13.362 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:20:13.362 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:13.362 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:13.362 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.362 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:13.362 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:20:13.362 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:13.362 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.362 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.363 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:13.363 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:20:13.363 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:13.363 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.363 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:13.363 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.363 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:20:13.363 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:20:13.363 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:13.363 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:13.363 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:13.363 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.363 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:13.364 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.364 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:20:13.364 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:20:13.364 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.364 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:13.364 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:13.364 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:20:13.364 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.364 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:13.364 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:20:13.364 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.364 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:13.364 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:13.364 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4 2024-09-13T18:20:13.365 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.365 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:13.365 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:20:13.365 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.365 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:13.365 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:13.365 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 6 2024-09-13T18:20:13.365 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.365 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:13.365 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:20:13.365 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.365 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:13.365 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:13.366 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:20:13.366 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.366 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:13.366 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:20:13.366 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.366 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:13.366 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:13.366 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:20:13.366 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.366 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:13.366 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.366 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:20:13.366 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:20:13.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdba54da700 1 -- 172.21.15.40:0/2448107495 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdb840667b0 msgr2=0x7fdb84068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdba54da700 1 --2- 172.21.15.40:0/2448107495 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdb840667b0 0x7fdb84068c70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fdba0160c50 tx=0x7fdb94009060 comp rx=0 tx=0).stop 2024-09-13T18:20:13.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdba54da700 1 -- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdba015bb80 msgr2=0x7fdba015c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdb9f7fe700 1 -- 172.21.15.40:0/2448107495 reap_dead start 2024-09-13T18:20:13.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdba54da700 1 --2- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdba015bb80 0x7fdba015c000 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdb8802e210 tx=0x7fdba0070c70 comp rx=0 tx=0).stop 2024-09-13T18:20:13.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdba54da700 1 -- 172.21.15.40:0/2448107495 shutdown_connections 2024-09-13T18:20:13.367 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdba54da700 1 --2- 172.21.15.40:0/2448107495 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdba015bb80 0x7fdba015c000 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdba54da700 1 -- 172.21.15.40:0/2448107495 >> 172.21.15.40:0/2448107495 conn(0x7fdba006d5b0 msgr2=0x7fdba006a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:13.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdba54da700 1 -- 172.21.15.40:0/2448107495 shutdown_connections 2024-09-13T18:20:13.368 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.342+0000 7fdba54da700 1 -- 172.21.15.40:0/2448107495 wait complete. 2024-09-13T18:20:13.369 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:20:13.369 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:20:13.369 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:20:13.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.518+0000 7fb308cf7700 1 Processor -- start 2024-09-13T18:20:13.527 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.518+0000 7fb308cf7700 1 -- start start 2024-09-13T18:20:13.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.518+0000 7fb308cf7700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb3040c3a70 0x7fb30415af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.528 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.518+0000 7fb308cf7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3040663b0 0x7fb30415b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.518+0000 7fb308cf7700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb30415aa80 con 0x7fb3040c3690 2024-09-13T18:20:13.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.518+0000 7fb308cf7700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb304157730 con 0x7fb3040663b0 2024-09-13T18:20:13.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.518+0000 7fb308cf7700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb304157a10 con 0x7fb3040c3a70 2024-09-13T18:20:13.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.518+0000 7fb302d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3040663b0 0x7fb30415b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.518+0000 7fb301d9b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb3040c3a70 0x7fb30415af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.518+0000 7fb302d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3040663b0 0x7fb30415b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54270/0 (socket says 172.21.15.40:54270) 2024-09-13T18:20:13.529 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.518+0000 7fb302d9d700 1 -- 172.21.15.40:0/2815764493 learned_addr learned my addr 172.21.15.40:0/2815764493 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:13.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.518+0000 7fb302d9d700 1 -- 172.21.15.40:0/2815764493 >> v2:172.21.15.43:3300/0 conn(0x7fb3040c3a70 msgr2=0x7fb30415af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:20:13.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.518+0000 7fb302d9d700 1 --2- 172.21.15.40:0/2815764493 >> v2:172.21.15.43:3300/0 conn(0x7fb3040c3a70 0x7fb30415af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.530 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.518+0000 7fb302d9d700 1 -- 172.21.15.40:0/2815764493 >> v1:172.21.15.40:6789/0 conn(0x7fb3040c3690 legacy=0x7fb3040c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:20:13.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.526+0000 7fb302d9d700 1 -- 172.21.15.40:0/2815764493 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb304157cf0 con 0x7fb3040663b0 2024-09-13T18:20:13.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.526+0000 7fb302d9d700 1 --2- 172.21.15.40:0/2815764493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3040663b0 0x7fb30415b450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb3040c5270 tx=0x7fb2ec009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f23bd42ca2f668e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.526+0000 7fb30159a700 1 -- 172.21.15.40:0/2815764493 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec00b8a0 con 0x7fb3040663b0 2024-09-13T18:20:13.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.526+0000 7fb30159a700 1 -- 172.21.15.40:0/2815764493 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ec00ba00 con 0x7fb3040663b0 2024-09-13T18:20:13.536 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.526+0000 7fb30159a700 1 -- 172.21.15.40:0/2815764493 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec00bd90 con 0x7fb3040663b0 2024-09-13T18:20:13.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.526+0000 7fb308cf7700 1 -- 172.21.15.40:0/2815764493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3040663b0 msgr2=0x7fb30415b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.526+0000 7fb308cf7700 1 --2- 172.21.15.40:0/2815764493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3040663b0 0x7fb30415b450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb3040c5270 tx=0x7fb2ec009bf0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.530+0000 7fb308cf7700 1 -- 172.21.15.40:0/2815764493 shutdown_connections 2024-09-13T18:20:13.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.530+0000 7fb308cf7700 1 --2- 172.21.15.40:0/2815764493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3040663b0 0x7fb30415b450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.530+0000 7fb308cf7700 1 --2- 172.21.15.40:0/2815764493 >> v2:172.21.15.43:3300/0 conn(0x7fb3040c3a70 0x7fb30415af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.537 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.530+0000 7fb308cf7700 1 -- 172.21.15.40:0/2815764493 >> 172.21.15.40:0/2815764493 conn(0x7fb30406a390 msgr2=0x7fb30406a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:13.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.530+0000 7fb308cf7700 1 -- 172.21.15.40:0/2815764493 shutdown_connections 2024-09-13T18:20:13.538 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.530+0000 7fb308cf7700 1 -- 172.21.15.40:0/2815764493 wait complete. 2024-09-13T18:20:13.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.530+0000 7fb308cf7700 1 Processor -- start 2024-09-13T18:20:13.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.530+0000 7fb308cf7700 1 -- start start 2024-09-13T18:20:13.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.530+0000 7fb308cf7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3040663b0 0x7fb304151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.530+0000 7fb308cf7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3040c3690 0x7fb304154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.530+0000 7fb308cf7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3040c3a70 0x7fb3041f03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.530+0000 7fb308cf7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb304148220 con 0x7fb3040c3690 2024-09-13T18:20:13.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.530+0000 7fb308cf7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30415a8c0 con 0x7fb3040c3a70 2024-09-13T18:20:13.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.530+0000 7fb308cf7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3041f1080 con 0x7fb3040663b0 2024-09-13T18:20:13.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.530+0000 7fb302d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3040c3a70 0x7fb3041f03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.541 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.530+0000 7fb302d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3040c3a70 0x7fb3041f03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:33234/0 (socket says 172.21.15.40:33234) 2024-09-13T18:20:13.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.530+0000 7fb302d9d700 1 -- 172.21.15.40:0/2745445506 learned_addr learned my addr 172.21.15.40:0/2745445506 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:13.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.530+0000 7fb30259c700 1 --2- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3040663b0 0x7fb304151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.534+0000 7fb302d9d700 1 -- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3040663b0 msgr2=0x7fb304151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.534+0000 7fb302d9d700 1 --2- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3040663b0 0x7fb304151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.534+0000 7fb302d9d700 1 -- 172.21.15.40:0/2745445506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3040c3690 msgr2=0x7fb304154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.534+0000 7fb302d9d700 1 --2- 172.21.15.40:0/2745445506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3040c3690 0x7fb304154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.534+0000 7fb302d9d700 1 -- 172.21.15.40:0/2745445506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec009570 con 0x7fb3040c3a70 2024-09-13T18:20:13.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.534+0000 7fb302d9d700 1 --2- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3040c3a70 0x7fb3041f03e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec013000 tx=0x7fb2ec004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.534+0000 7fb2faffd700 1 -- 172.21.15.40:0/2745445506 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec0038c0 con 0x7fb3040c3a70 2024-09-13T18:20:13.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.534+0000 7fb2faffd700 1 -- 172.21.15.40:0/2745445506 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ec003a20 con 0x7fb3040c3a70 2024-09-13T18:20:13.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.534+0000 7fb2faffd700 1 -- 172.21.15.40:0/2745445506 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec003dd0 con 0x7fb3040c3a70 2024-09-13T18:20:13.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.534+0000 7fb308cf7700 1 -- 172.21.15.40:0/2745445506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb30415a5e0 con 0x7fb3040c3a70 2024-09-13T18:20:13.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.534+0000 7fb308cf7700 1 -- 172.21.15.40:0/2745445506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb304152b70 con 0x7fb3040c3a70 2024-09-13T18:20:13.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.538+0000 7fb2faffd700 1 -- 172.21.15.40:0/2745445506 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb2ec011420 con 0x7fb3040c3a70 2024-09-13T18:20:13.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.538+0000 7fb2faffd700 1 --2- 172.21.15.40:0/2745445506 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb2e4066760 0x7fb2e4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.538+0000 7fb2faffd700 1 -- 172.21.15.40:0/2745445506 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb2ec085980 con 0x7fb3040c3a70 2024-09-13T18:20:13.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.538+0000 7fb30259c700 1 --2- 172.21.15.40:0/2745445506 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb2e4066760 0x7fb2e4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.538+0000 7fb308cf7700 1 --2- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3041f0920 0x7fb30414c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.538+0000 7fb308cf7700 1 -- 172.21.15.40:0/2745445506 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb304067720 con 0x7fb3041f0920 2024-09-13T18:20:13.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.538+0000 7fb301d9b700 1 --2- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3041f0920 0x7fb30414c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.538+0000 7fb30259c700 1 --2- 172.21.15.40:0/2745445506 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb2e4066760 0x7fb2e4068c20 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00f290 tx=0x7fb2fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.538+0000 7fb301d9b700 1 -- 172.21.15.40:0/2745445506 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb304157400 con 0x7fb3041f0920 2024-09-13T18:20:13.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.538+0000 7fb301d9b700 1 --2- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3041f0920 0x7fb30414c5f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb30415bc90 tx=0x7fb2f400ca80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.538+0000 7fb2faffd700 1 -- 172.21.15.40:0/2745445506 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4011800 con 0x7fb3041f0920 2024-09-13T18:20:13.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.538+0000 7fb2faffd700 1 -- 172.21.15.40:0/2745445506 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fb2f4011960 con 0x7fb3041f0920 2024-09-13T18:20:13.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.538+0000 7fb2faffd700 1 -- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3041f0920 msgr2=0x7fb30414c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.538+0000 7fb2faffd700 1 --2- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3041f0920 0x7fb30414c5f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb30415bc90 tx=0x7fb2f400ca80 comp rx=0 tx=0).stop 2024-09-13T18:20:13.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.538+0000 7fb2faffd700 1 -- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3041f0920 msgr2=0x7fb30414c5f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:20:13.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.538+0000 7fb2faffd700 1 --2- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3041f0920 0x7fb30414c5f0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb30415bc90 tx=0x7fb2f400ca80 comp rx=0 tx=0).stop 2024-09-13T18:20:13.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.554+0000 7fb308cf7700 1 --2- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3041531a0 0x7fb30414cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.554+0000 7fb308cf7700 1 -- 172.21.15.40:0/2745445506 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3041582b0 con 0x7fb3041531a0 2024-09-13T18:20:13.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.558+0000 7fb301d9b700 1 --2- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3041531a0 0x7fb30414cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.558+0000 7fb301d9b700 1 -- 172.21.15.40:0/2745445506 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fb3041509f0 con 0x7fb3041531a0 2024-09-13T18:20:13.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.558+0000 7fb301d9b700 1 --2- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3041531a0 0x7fb30414cb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4006fd0 tx=0x7fb2f400f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.566 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.558+0000 7fb2faffd700 1 -- 172.21.15.40:0/2745445506 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f401f040 con 0x7fb3041531a0 2024-09-13T18:20:13.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.558+0000 7fb2faffd700 1 -- 172.21.15.40:0/2745445506 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4783 (secure 0 0 0) 0x7fb2f40193b0 con 0x7fb3041531a0 2024-09-13T18:20:13.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.558+0000 7fb2faffd700 1 -- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3041531a0 msgr2=0x7fb30414cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.558+0000 7fb2faffd700 1 --2- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3041531a0 0x7fb30414cb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4006fd0 tx=0x7fb2f400f000 comp rx=0 tx=0).stop 2024-09-13T18:20:13.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.558+0000 7fb2faffd700 1 -- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3041531a0 msgr2=0x7fb30414cb30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:20:13.567 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.558+0000 7fb2faffd700 1 --2- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb3041531a0 0x7fb30414cb30 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4006fd0 tx=0x7fb2f400f000 comp rx=0 tx=0).stop 2024-09-13T18:20:13.567 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:20:13.568 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:20:13.568 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:20:13.568 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:20:13.568 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 234, 2024-09-13T18:20:13.568 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:20:13.568 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:20:13.568 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:20:13.568 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:20:13.568 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:13.568 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 14, 2024-09-13T18:20:13.568 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:20:13.568 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:20:13.569 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:20:13.569 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:13.569 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:13.569 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:13.569 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:13.569 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:13.569 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:13.569 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:13.569 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:13.569 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:13.569 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:13.569 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:13.569 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:20:13.569 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:20:13.570 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:13.570 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:13.570 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:13.570 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:13.570 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:13.570 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:13.570 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:13.570 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:13.570 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:13.570 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:13.570 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:13.570 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.570 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:20:13.570 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:20:13.571 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:20:13.571 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:20:13.571 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:20:13.571 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:20:13.571 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:20:13.571 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:20:13.571 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:20:13.571 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:20:13.571 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:20:13.571 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:20:13.571 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:20:13.571 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:20:13.571 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:20:13.571 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:20:13.572 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:20:13.572 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:13.572 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:13.572 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:13.572 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:13.572 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:13.572 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:13.572 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:13.572 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:13.572 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:13.572 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:13.572 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:13.572 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:20:13.572 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.573 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:20:13.573 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.573 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:20:13.573 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:20:13.573 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:13.573 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:13.573 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.573 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:13.573 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:20:13.573 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:13.573 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.573 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.573 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:13.573 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:20:13.574 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:13.574 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.574 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:13.574 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.574 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:20:13.574 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:20:13.574 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:13.574 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:13.574 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:13.574 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.574 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.574 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:20:13.574 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:20:13.574 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:13.575 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:13.575 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.575 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:13.575 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:20:13.575 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:13.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.575 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.575 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:13.575 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:20:13.575 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:13.575 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.575 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:13.575 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.575 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:20:13.576 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:20:13.576 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:13.576 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:13.576 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:13.576 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.576 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.576 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:20:13.576 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:20:13.576 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:13.576 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:13.576 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.576 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:13.576 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:20:13.577 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:13.577 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.577 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.577 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:13.577 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:20:13.577 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:13.577 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.577 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:13.577 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.577 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:20:13.577 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:20:13.577 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:13.577 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:13.577 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:13.577 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.578 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:13.578 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.578 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:20:13.578 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:20:13.578 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.578 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:13.578 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:13.578 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:20:13.578 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.578 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:13.578 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:20:13.578 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.578 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:13.579 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:13.579 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:20:13.579 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.579 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:13.579 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:20:13.579 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.579 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:13.579 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:13.579 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:20:13.579 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.579 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:13.579 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:20:13.579 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:13.579 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:13.579 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:13.580 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:20:13.580 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:13.580 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:13.580 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:13.580 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:20:13.580 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:20:13.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.574+0000 7fb2f8ff9700 1 -- 172.21.15.40:0/2745445506 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb2e4066760 msgr2=0x7fb2e4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.574+0000 7fb2f8ff9700 1 --2- 172.21.15.40:0/2745445506 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb2e4066760 0x7fb2e4068c20 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00f290 tx=0x7fb2fc009040 comp rx=0 tx=0).stop 2024-09-13T18:20:13.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.574+0000 7fb302d9d700 1 -- 172.21.15.40:0/2745445506 reap_dead start 2024-09-13T18:20:13.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.574+0000 7fb2f8ff9700 1 -- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3040c3a70 msgr2=0x7fb3041f03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.574+0000 7fb2f8ff9700 1 --2- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3040c3a70 0x7fb3041f03e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec013000 tx=0x7fb2ec004770 comp rx=0 tx=0).stop 2024-09-13T18:20:13.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.574+0000 7fb2f8ff9700 1 -- 172.21.15.40:0/2745445506 shutdown_connections 2024-09-13T18:20:13.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.574+0000 7fb2f8ff9700 1 --2- 172.21.15.40:0/2745445506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3040c3a70 0x7fb3041f03e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.574+0000 7fb2f8ff9700 1 -- 172.21.15.40:0/2745445506 >> 172.21.15.40:0/2745445506 conn(0x7fb30406a390 msgr2=0x7fb30406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:13.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.574+0000 7fb2f8ff9700 1 -- 172.21.15.40:0/2745445506 shutdown_connections 2024-09-13T18:20:13.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.574+0000 7fb2f8ff9700 1 -- 172.21.15.40:0/2745445506 wait complete. 2024-09-13T18:20:13.600 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:20:13.600 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-13T18:20:13.600 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-13T18:20:13.600 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-13T18:20:13.600 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-13T18:20:13.600 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T18:20:13.601 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:20:13.637 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T18:20:13.637 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-13T18:20:13.637 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:20:13.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.778+0000 7f240a16b700 1 Processor -- start 2024-09-13T18:20:13.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.778+0000 7f240a16b700 1 -- start start 2024-09-13T18:20:13.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.778+0000 7f240a16b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f24040cb6e0 0x7f24040c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.778+0000 7f240a16b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24040c55c0 0x7f24040c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.778+0000 7f240a16b700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2404155f30 con 0x7f240414f340 2024-09-13T18:20:13.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.778+0000 7f240a16b700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24041553f0 con 0x7f24040cb6e0 2024-09-13T18:20:13.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.778+0000 7f240a16b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24040c6ea0 con 0x7f24040c55c0 2024-09-13T18:20:13.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.778+0000 7f2409169700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f24040cb6e0 0x7f24040c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.778+0000 7f2408968700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24040c55c0 0x7f24040c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.778+0000 7f2408968700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24040c55c0 0x7f24040c5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:34808/0 (socket says 172.21.15.40:34808) 2024-09-13T18:20:13.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.778+0000 7f2408968700 1 -- 172.21.15.40:0/2829857244 learned_addr learned my addr 172.21.15.40:0/2829857244 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:13.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.778+0000 7f2408968700 1 -- 172.21.15.40:0/2829857244 >> v2:172.21.15.40:3300/0 conn(0x7f24040cb6e0 msgr2=0x7f24040c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:20:13.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.778+0000 7f2408968700 1 --2- 172.21.15.40:0/2829857244 >> v2:172.21.15.40:3300/0 conn(0x7f24040cb6e0 0x7f24040c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.778+0000 7f2408968700 1 -- 172.21.15.40:0/2829857244 >> v1:172.21.15.40:6789/0 conn(0x7f240414f340 legacy=0x7f240406a390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:20:13.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.778+0000 7f2408968700 1 -- 172.21.15.40:0/2829857244 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2404155600 con 0x7f24040c55c0 2024-09-13T18:20:13.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.778+0000 7f2409169700 1 --2- 172.21.15.40:0/2829857244 >> v2:172.21.15.40:3300/0 conn(0x7f24040cb6e0 0x7f24040c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:20:13.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.782+0000 7f2408968700 1 --2- 172.21.15.40:0/2829857244 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24040c55c0 0x7f24040c5a00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009910 tx=0x7f23f4009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e51b5fb03be47604 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.782+0000 7f23fbfff700 1 -- 172.21.15.40:0/2829857244 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4004230 con 0x7f24040c55c0 2024-09-13T18:20:13.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.786+0000 7f23fbfff700 1 -- 172.21.15.40:0/2829857244 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f4004390 con 0x7f24040c55c0 2024-09-13T18:20:13.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.786+0000 7f23fbfff700 1 -- 172.21.15.40:0/2829857244 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f40036c0 con 0x7f24040c55c0 2024-09-13T18:20:13.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.786+0000 7f240a16b700 1 -- 172.21.15.40:0/2829857244 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24040c55c0 msgr2=0x7f24040c5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.786+0000 7f240a16b700 1 --2- 172.21.15.40:0/2829857244 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24040c55c0 0x7f24040c5a00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009910 tx=0x7f23f4009c20 comp rx=0 tx=0).stop 2024-09-13T18:20:13.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.786+0000 7f240a16b700 1 -- 172.21.15.40:0/2829857244 shutdown_connections 2024-09-13T18:20:13.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.786+0000 7f240a16b700 1 --2- 172.21.15.40:0/2829857244 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f24040c55c0 0x7f24040c5a00 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009910 tx=0x7f23f4009c20 comp rx=0 tx=0).stop 2024-09-13T18:20:13.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.786+0000 7f240a16b700 1 --2- 172.21.15.40:0/2829857244 >> v2:172.21.15.40:3300/0 conn(0x7f24040cb6e0 0x7f24040c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.786+0000 7f240a16b700 1 -- 172.21.15.40:0/2829857244 >> 172.21.15.40:0/2829857244 conn(0x7f24040cabc0 msgr2=0x7f24040cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:13.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.798+0000 7f240a16b700 1 -- 172.21.15.40:0/2829857244 shutdown_connections 2024-09-13T18:20:13.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.798+0000 7f240a16b700 1 -- 172.21.15.40:0/2829857244 wait complete. 2024-09-13T18:20:13.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f240a16b700 1 Processor -- start 2024-09-13T18:20:13.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f240a16b700 1 -- start start 2024-09-13T18:20:13.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f240a16b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24040cb6e0 0x7f24041486e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.813 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f240a16b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f240414f340 0x7f24041eed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f240a16b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f240414b300 0x7f240413e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f240a16b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24041ee8b0 con 0x7f240414f340 2024-09-13T18:20:13.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f240a16b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24041439e0 con 0x7f24040cb6e0 2024-09-13T18:20:13.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f240a16b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24041ee580 con 0x7f240414b300 2024-09-13T18:20:13.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f2409169700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24040cb6e0 msgr2=0x7f24041486e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:20:13.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f2409169700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24040cb6e0 0x7f24041486e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:20:13.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f240996a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f240414b300 0x7f240413e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f240996a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f240414b300 0x7f240413e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:34822/0 (socket says 172.21.15.40:34822) 2024-09-13T18:20:13.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f240996a700 1 -- 172.21.15.40:0/707234020 learned_addr learned my addr 172.21.15.40:0/707234020 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:13.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f2408968700 1 --2- 172.21.15.40:0/707234020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f240414f340 0x7f24041eed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f240996a700 1 -- 172.21.15.40:0/707234020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24040cb6e0 msgr2=0x7f24041486e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:20:13.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f240996a700 1 --2- 172.21.15.40:0/707234020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24040cb6e0 0x7f24041486e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f240996a700 1 -- 172.21.15.40:0/707234020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f240414f340 msgr2=0x7f24041eed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f240996a700 1 --2- 172.21.15.40:0/707234020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f240414f340 0x7f24041eed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f240996a700 1 -- 172.21.15.40:0/707234020 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23fc009a70 con 0x7f240414b300 2024-09-13T18:20:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f2408968700 1 --2- 172.21.15.40:0/707234020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f240414f340 0x7f24041eed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:20:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f240996a700 1 --2- 172.21.15.40:0/707234020 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f240414b300 0x7f240413e4f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f240000b3e0 tx=0x7f240000b6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f23f9ffb700 1 -- 172.21.15.40:0/707234020 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240000be20 con 0x7f240414b300 2024-09-13T18:20:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f23f9ffb700 1 -- 172.21.15.40:0/707234020 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2400033640 con 0x7f240414b300 2024-09-13T18:20:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f23f9ffb700 1 -- 172.21.15.40:0/707234020 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24000339f0 con 0x7f240414b300 2024-09-13T18:20:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f240a16b700 1 -- 172.21.15.40:0/707234020 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f4009570 con 0x7f240414b300 2024-09-13T18:20:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.802+0000 7f240a16b700 1 -- 172.21.15.40:0/707234020 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24040ca370 con 0x7f240414b300 2024-09-13T18:20:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.806+0000 7f240a16b700 1 -- 172.21.15.40:0/707234020 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e4005310 con 0x7f240414b300 2024-09-13T18:20:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.806+0000 7f23f9ffb700 1 -- 172.21.15.40:0/707234020 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2400033b50 con 0x7f240414b300 2024-09-13T18:20:13.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.806+0000 7f23f9ffb700 1 --2- 172.21.15.40:0/707234020 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f23e8066760 0x7f23e8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:13.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.810+0000 7f23f9ffb700 1 -- 172.21.15.40:0/707234020 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f24000a5a70 con 0x7f240414b300 2024-09-13T18:20:13.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.810+0000 7f2409169700 1 --2- 172.21.15.40:0/707234020 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f23e8066760 0x7f23e8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:13.818 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.810+0000 7f2409169700 1 --2- 172.21.15.40:0/707234020 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f23e8066760 0x7f23e8068c20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009fd0 tx=0x7f23fc009750 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:13.826 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:13.818+0000 7f23f9ffb700 1 -- 172.21.15.40:0/707234020 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2400076590 con 0x7f240414b300 2024-09-13T18:20:14.232 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:14.222+0000 7f240a16b700 1 -- 172.21.15.40:0/707234020 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f23e4005180 con 0x7f240414b300 2024-09-13T18:20:14.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:14.222+0000 7f23f9ffb700 1 -- 172.21.15.40:0/707234020 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f2400076120 con 0x7f240414b300 2024-09-13T18:20:14.233 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":234,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:20:14.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:14.230+0000 7f240a16b700 1 -- 172.21.15.40:0/707234020 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f23e8066760 msgr2=0x7f23e8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:14.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:14.230+0000 7f240a16b700 1 --2- 172.21.15.40:0/707234020 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f23e8066760 0x7f23e8068c20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009fd0 tx=0x7f23fc009750 comp rx=0 tx=0).stop 2024-09-13T18:20:14.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:14.230+0000 7f240a16b700 1 -- 172.21.15.40:0/707234020 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f240414b300 msgr2=0x7f240413e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:14.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:14.230+0000 7f240a16b700 1 --2- 172.21.15.40:0/707234020 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f240414b300 0x7f240413e4f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f240000b3e0 tx=0x7f240000b6f0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.238 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:14.230+0000 7f240a16b700 1 -- 172.21.15.40:0/707234020 shutdown_connections 2024-09-13T18:20:14.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:14.230+0000 7f240a16b700 1 --2- 172.21.15.40:0/707234020 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f23e8066760 0x7f23e8068c20 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:14.230+0000 7f240a16b700 1 --2- 172.21.15.40:0/707234020 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f240414b300 0x7f240413e4f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:14.230+0000 7f240a16b700 1 --2- 172.21.15.40:0/707234020 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f240414f340 0x7f24041eed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:14.230+0000 7f240a16b700 1 --2- 172.21.15.40:0/707234020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24040cb6e0 0x7f24041486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:14.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:14.230+0000 7f240a16b700 1 -- 172.21.15.40:0/707234020 >> 172.21.15.40:0/707234020 conn(0x7f24040cabc0 msgr2=0x7f24040c2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:14.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:14.230+0000 7f240a16b700 1 -- 172.21.15.40:0/707234020 shutdown_connections 2024-09-13T18:20:14.239 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:14.230+0000 7f240a16b700 1 -- 172.21.15.40:0/707234020 wait complete. 2024-09-13T18:20:14.917 INFO:tasks.workunit.client.0.smithi040.stdout:Syncing()...15 sec 2024-09-13T18:20:14.917 INFO:tasks.workunit.client.0.smithi040.stdout:FFSB benchmark finished at: Fri Sep 13 18:20:14 2024 2024-09-13T18:20:14.917 INFO:tasks.workunit.client.0.smithi040.stdout: 2024-09-13T18:20:14.918 INFO:tasks.workunit.client.0.smithi040.stdout:Results: 2024-09-13T18:20:14.918 INFO:tasks.workunit.client.0.smithi040.stdout:Benchmark took 330.31 sec 2024-09-13T18:20:14.918 INFO:tasks.workunit.client.0.smithi040.stdout: 2024-09-13T18:20:14.918 INFO:tasks.workunit.client.0.smithi040.stdout:Total Results 2024-09-13T18:20:14.918 INFO:tasks.workunit.client.0.smithi040.stdout:=============== 2024-09-13T18:20:14.918 INFO:tasks.workunit.client.0.smithi040.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-09-13T18:20:14.918 INFO:tasks.workunit.client.0.smithi040.stdout: ======= ============ ========= ======= =========== ========== 2024-09-13T18:20:14.918 INFO:tasks.workunit.client.0.smithi040.stdout: write : 751360 2274.68 100.000% 100.000% 8.89MB/sec 2024-09-13T18:20:14.918 INFO:tasks.workunit.client.0.smithi040.stdout:- 2024-09-13T18:20:14.918 INFO:tasks.workunit.client.0.smithi040.stdout:2274.68 Transactions per Second 2024-09-13T18:20:14.918 INFO:tasks.workunit.client.0.smithi040.stdout: 2024-09-13T18:20:14.918 INFO:tasks.workunit.client.0.smithi040.stdout:Throughput Results 2024-09-13T18:20:14.919 INFO:tasks.workunit.client.0.smithi040.stdout:=================== 2024-09-13T18:20:14.919 INFO:tasks.workunit.client.0.smithi040.stdout:Write Throughput: 8.89MB/sec 2024-09-13T18:20:14.920 INFO:tasks.workunit.client.0.smithi040.stdout: 2024-09-13T18:20:14.920 INFO:tasks.workunit.client.0.smithi040.stdout:System Call Latency statistics in millisecs 2024-09-13T18:20:14.920 INFO:tasks.workunit.client.0.smithi040.stdout:===== 2024-09-13T18:20:14.920 INFO:tasks.workunit.client.0.smithi040.stdout: Min Avg Max Total Calls 2024-09-13T18:20:14.920 INFO:tasks.workunit.client.0.smithi040.stdout: ======== ======== ======== ============ 2024-09-13T18:20:14.920 INFO:tasks.workunit.client.0.smithi040.stdout:[ open] 0.486000 201.055158 7298.792969 587 2024-09-13T18:20:14.920 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[0] 5000.000000 - 10000.000000 : 8 2024-09-13T18:20:14.920 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[1] 2000.000000 - 5000.000000 : 12 2024-09-13T18:20:14.921 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[2] 1000.000000 - 2000.000000 : 5 2024-09-13T18:20:14.921 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[3] 500.000000 - 1000.000000 : 4 2024-09-13T18:20:14.921 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[4] 200.000000 - 500.000000 : 30 2024-09-13T18:20:14.921 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[5] 100.000000 - 200.000000 : 2 2024-09-13T18:20:14.921 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[6] 50.000000 - 100.000000 : 8 2024-09-13T18:20:14.921 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[7] 20.000000 - 50.000000 : 72 2024-09-13T18:20:14.921 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[8] 10.000000 - 20.000000 : 169 2024-09-13T18:20:14.921 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[9] 5.000000 - 10.000000 : 145 2024-09-13T18:20:14.921 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[10] 2.000000 - 5.000000 : 107 2024-09-13T18:20:14.921 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[11] 1.000000 - 2.000000 : 20 2024-09-13T18:20:14.921 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[12] 0.500000 - 1.000000 : 4 2024-09-13T18:20:14.921 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[13] 0.200000 - 0.500000 : 1 2024-09-13T18:20:14.921 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[14] 0.100000 - 0.200000 : 0 2024-09-13T18:20:14.922 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2024-09-13T18:20:14.922 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-09-13T18:20:14.922 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-13T18:20:14.922 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-13T18:20:14.922 INFO:tasks.workunit.client.0.smithi040.stdout: 2024-09-13T18:20:14.922 INFO:tasks.workunit.client.0.smithi040.stdout:[ write] 0.087000 13.023974 434.359985 751360 2024-09-13T18:20:14.922 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-13T18:20:14.922 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-13T18:20:14.922 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-13T18:20:14.922 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-13T18:20:14.922 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[4] 200.000000 - 500.000000 : 141 2024-09-13T18:20:14.922 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[5] 100.000000 - 200.000000 : 5249 2024-09-13T18:20:14.922 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[6] 50.000000 - 100.000000 : 32834 2024-09-13T18:20:14.923 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[7] 20.000000 - 50.000000 : 116877 2024-09-13T18:20:14.923 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[8] 10.000000 - 20.000000 : 134894 2024-09-13T18:20:14.923 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[9] 5.000000 - 10.000000 : 122078 2024-09-13T18:20:14.923 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[10] 2.000000 - 5.000000 : 98991 2024-09-13T18:20:14.923 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[11] 1.000000 - 2.000000 : 41688 2024-09-13T18:20:14.923 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[12] 0.500000 - 1.000000 : 36414 2024-09-13T18:20:14.923 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[13] 0.200000 - 0.500000 : 125551 2024-09-13T18:20:14.923 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[14] 0.100000 - 0.200000 : 36629 2024-09-13T18:20:14.923 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[15] 0.050000 - 0.100000 : 14 2024-09-13T18:20:14.923 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-09-13T18:20:14.923 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-13T18:20:14.923 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-13T18:20:14.923 INFO:tasks.workunit.client.0.smithi040.stdout: 2024-09-13T18:20:14.923 INFO:tasks.workunit.client.0.smithi040.stdout:[ lseek] 0.000000 0.001224 6.632000 751360 2024-09-13T18:20:14.924 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-13T18:20:14.924 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-13T18:20:14.924 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-13T18:20:14.924 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-13T18:20:14.924 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-13T18:20:14.924 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-09-13T18:20:14.924 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-09-13T18:20:14.924 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2024-09-13T18:20:14.924 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[8] 10.000000 - 20.000000 : 0 2024-09-13T18:20:14.924 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[9] 5.000000 - 10.000000 : 1 2024-09-13T18:20:14.924 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[10] 2.000000 - 5.000000 : 7 2024-09-13T18:20:14.924 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[11] 1.000000 - 2.000000 : 3 2024-09-13T18:20:14.924 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[12] 0.500000 - 1.000000 : 24 2024-09-13T18:20:14.924 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[13] 0.200000 - 0.500000 : 43 2024-09-13T18:20:14.925 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[14] 0.100000 - 0.200000 : 32 2024-09-13T18:20:14.925 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[15] 0.050000 - 0.100000 : 71 2024-09-13T18:20:14.925 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[16] 0.020000 - 0.050000 : 294 2024-09-13T18:20:14.925 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[17] 0.010000 - 0.020000 : 1075 2024-09-13T18:20:14.925 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[18] 0.000000 - 0.010000 : 749810 2024-09-13T18:20:14.925 INFO:tasks.workunit.client.0.smithi040.stdout: 2024-09-13T18:20:14.925 INFO:tasks.workunit.client.0.smithi040.stdout:[ close] 0.025000 5.292622 213.964005 587 2024-09-13T18:20:14.925 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-13T18:20:14.925 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-13T18:20:14.925 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-13T18:20:14.925 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-13T18:20:14.925 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[4] 200.000000 - 500.000000 : 1 2024-09-13T18:20:14.925 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[5] 100.000000 - 200.000000 : 1 2024-09-13T18:20:14.926 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[6] 50.000000 - 100.000000 : 19 2024-09-13T18:20:14.926 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[7] 20.000000 - 50.000000 : 20 2024-09-13T18:20:14.926 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[8] 10.000000 - 20.000000 : 16 2024-09-13T18:20:14.926 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[9] 5.000000 - 10.000000 : 30 2024-09-13T18:20:14.926 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[10] 2.000000 - 5.000000 : 63 2024-09-13T18:20:14.926 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[11] 1.000000 - 2.000000 : 47 2024-09-13T18:20:14.926 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[12] 0.500000 - 1.000000 : 50 2024-09-13T18:20:14.926 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[13] 0.200000 - 0.500000 : 63 2024-09-13T18:20:14.926 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[14] 0.100000 - 0.200000 : 82 2024-09-13T18:20:14.926 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[15] 0.050000 - 0.100000 : 117 2024-09-13T18:20:14.926 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[16] 0.020000 - 0.050000 : 78 2024-09-13T18:20:14.926 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-13T18:20:14.926 INFO:tasks.workunit.client.0.smithi040.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-13T18:20:14.926 INFO:tasks.workunit.client.0.smithi040.stdout: 2024-09-13T18:20:14.926 INFO:tasks.workunit.client.0.smithi040.stdout: 2024-09-13T18:20:14.927 INFO:tasks.workunit.client.0.smithi040.stdout: 2024-09-13T18:20:14.927 INFO:tasks.workunit.client.0.smithi040.stdout: 2024-09-13T18:20:14.927 INFO:tasks.workunit.client.0.smithi040.stdout:0.0% User Time 2024-09-13T18:20:14.927 INFO:tasks.workunit.client.0.smithi040.stdout:49.8% System Time 2024-09-13T18:20:14.927 INFO:tasks.workunit.client.0.smithi040.stdout:49.8% CPU Utilization 2024-09-13T18:20:17.257 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:20:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.386+0000 7f174e65a700 1 Processor -- start 2024-09-13T18:20:17.577 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.386+0000 7f174e65a700 1 -- start start 2024-09-13T18:20:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.386+0000 7f174e65a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1748160390 0x7f17481669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.386+0000 7f174e65a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1748071930 con 0x7f17480704e0 2024-09-13T18:20:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.386+0000 7f174e65a700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1748070a40 con 0x7f1748167d10 2024-09-13T18:20:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.386+0000 7f174e65a700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1748166f20 con 0x7f1748160390 2024-09-13T18:20:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.386+0000 7f174ce57700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f17480704e0 0x7f1748167600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58428/0 (socket says 172.21.15.40:58428) 2024-09-13T18:20:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.386+0000 7f174ce57700 1 -- 172.21.15.40:0/2791145840 learned_addr learned my addr 172.21.15.40:0/2791145840 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:20:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.386+0000 7f174d658700 1 -- 172.21.15.40:0/2791145840 >> v2:172.21.15.43:3300/0 conn(0x7f1748160390 msgr2=0x7f17481669e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:20:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.386+0000 7f174d658700 1 --2- 172.21.15.40:0/2791145840 >> v2:172.21.15.43:3300/0 conn(0x7f1748160390 0x7f17481669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:20:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.386+0000 7f174de59700 1 -- 172.21.15.40:0/2791145840 >> v1:172.21.15.43:6789/0 conn(0x7f1748167d10 legacy=0x7f174806a390 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:20:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.386+0000 7f173ffff700 1 -- 172.21.15.40:0/2791145840 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 159231149 0 0) 0x7f1748071930 con 0x7f17480704e0 2024-09-13T18:20:17.578 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.566+0000 7f173ffff700 1 -- 172.21.15.40:0/2791145840 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17340035d0 con 0x7f17480704e0 2024-09-13T18:20:17.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f173ffff700 1 -- 172.21.15.40:0/2791145840 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1178627359 0 0) 0x7f17340035d0 con 0x7f17480704e0 2024-09-13T18:20:17.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f173ffff700 1 -- 172.21.15.40:0/2791145840 >> v2:172.21.15.43:3300/0 conn(0x7f1748160390 msgr2=0x7f17481669e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:20:17.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f173ffff700 1 --2- 172.21.15.40:0/2791145840 >> v2:172.21.15.43:3300/0 conn(0x7f1748160390 0x7f17481669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f173ffff700 1 -- 172.21.15.40:0/2791145840 >> v1:172.21.15.43:6789/0 conn(0x7f1748167d10 legacy=0x7f174806a390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:20:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f173ffff700 1 -- 172.21.15.40:0/2791145840 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1748070d20 con 0x7f17480704e0 2024-09-13T18:20:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f173ffff700 1 -- 172.21.15.40:0/2791145840 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f17440015a0 con 0x7f17480704e0 2024-09-13T18:20:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f173ffff700 1 -- 172.21.15.40:0/2791145840 >> v1:172.21.15.40:6789/0 conn(0x7f17480704e0 legacy=0x7f1748167600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f173ffff700 1 --2- 172.21.15.40:0/2791145840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1734003c40 0x7f1734008500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f173ffff700 1 --2- 172.21.15.40:0/2791145840 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1734008a90 0x7f173400af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f173ffff700 1 --2- 172.21.15.40:0/2791145840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f173400b700 0x7f173400dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f173ffff700 1 -- 172.21.15.40:0/2791145840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1748166f20 con 0x7f173400b700 2024-09-13T18:20:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f173ffff700 1 -- 172.21.15.40:0/2791145840 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173400e490 con 0x7f1734003c40 2024-09-13T18:20:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f173ffff700 1 -- 172.21.15.40:0/2791145840 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173400e650 con 0x7f1734008a90 2024-09-13T18:20:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f174d658700 1 -- 172.21.15.40:0/2791145840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1734003c40 msgr2=0x7f1734008500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:20:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f174d658700 1 --2- 172.21.15.40:0/2791145840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1734003c40 0x7f1734008500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:20:17.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f174de59700 1 --2- 172.21.15.40:0/2791145840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f173400b700 0x7f173400dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:17.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f174ce57700 1 --2- 172.21.15.40:0/2791145840 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1734008a90 0x7f173400af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:17.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f174de59700 1 -- 172.21.15.40:0/2791145840 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1734008a90 msgr2=0x7f173400af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:20:17.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f174de59700 1 --2- 172.21.15.40:0/2791145840 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1734008a90 0x7f173400af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:17.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f174de59700 1 -- 172.21.15.40:0/2791145840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1734003c40 msgr2=0x7f1734008500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:20:17.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f174de59700 1 --2- 172.21.15.40:0/2791145840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1734003c40 0x7f1734008500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:17.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f174de59700 1 -- 172.21.15.40:0/2791145840 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17340035d0 con 0x7f173400b700 2024-09-13T18:20:17.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f174de59700 1 -- 172.21.15.40:0/2791145840 reap_dead start 2024-09-13T18:20:17.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f174de59700 1 --2- 172.21.15.40:0/2791145840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f173400b700 0x7f173400dbc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f17380094a0 tx=0x7f1734003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22de2501c1271dff server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:17.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f173ffff700 1 -- 172.21.15.40:0/2791145840 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738003b60 con 0x7f173400b700 2024-09-13T18:20:17.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f173ffff700 1 -- 172.21.15.40:0/2791145840 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1738003d00 con 0x7f173400b700 2024-09-13T18:20:17.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f173ffff700 1 -- 172.21.15.40:0/2791145840 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17380052b0 con 0x7f173400b700 2024-09-13T18:20:17.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f174e65a700 1 -- 172.21.15.40:0/2791145840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f173400b700 msgr2=0x7f173400dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:17.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f174e65a700 1 --2- 172.21.15.40:0/2791145840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f173400b700 0x7f173400dbc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f17380094a0 tx=0x7f1734003880 comp rx=0 tx=0).stop 2024-09-13T18:20:17.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f174e65a700 1 -- 172.21.15.40:0/2791145840 shutdown_connections 2024-09-13T18:20:17.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f174e65a700 1 --2- 172.21.15.40:0/2791145840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f173400b700 0x7f173400dbc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:17.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f174e65a700 1 -- 172.21.15.40:0/2791145840 >> 172.21.15.40:0/2791145840 conn(0x7f17480cabc0 msgr2=0x7f17480cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:17.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f174e65a700 1 -- 172.21.15.40:0/2791145840 shutdown_connections 2024-09-13T18:20:17.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.570+0000 7f174e65a700 1 -- 172.21.15.40:0/2791145840 wait complete. 2024-09-13T18:20:17.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174e65a700 1 Processor -- start 2024-09-13T18:20:17.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174e65a700 1 -- start start 2024-09-13T18:20:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174e65a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f173400b700 0x7f174808aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174e65a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17481f15e0 0x7f1748160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174e65a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17481f0e60 0x7f17481ef860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174e65a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174808b0e0 con 0x7f17481f0e60 2024-09-13T18:20:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174e65a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17480c2bf0 con 0x7f17481f15e0 2024-09-13T18:20:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174e65a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17481633b0 con 0x7f173400b700 2024-09-13T18:20:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174ce57700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17481f15e0 msgr2=0x7f1748160890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:20:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174ce57700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17481f15e0 0x7f1748160890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:20:17.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174d658700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f173400b700 0x7f174808aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174de59700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17481f0e60 0x7f17481ef860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174de59700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17481f0e60 0x7f17481ef860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54374/0 (socket says 172.21.15.40:54374) 2024-09-13T18:20:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174de59700 1 -- 172.21.15.40:0/4290804697 learned_addr learned my addr 172.21.15.40:0/4290804697 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174de59700 1 -- 172.21.15.40:0/4290804697 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f173400b700 msgr2=0x7f174808aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174de59700 1 --2- 172.21.15.40:0/4290804697 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f173400b700 0x7f174808aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174de59700 1 -- 172.21.15.40:0/4290804697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17481f15e0 msgr2=0x7f1748160890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:20:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174de59700 1 --2- 172.21.15.40:0/4290804697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17481f15e0 0x7f1748160890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174de59700 1 -- 172.21.15.40:0/4290804697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1748163690 con 0x7f17481f0e60 2024-09-13T18:20:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174de59700 1 --2- 172.21.15.40:0/4290804697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17481f0e60 0x7f17481ef860 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1748163b50 tx=0x7f17380059f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f173dffb700 1 -- 172.21.15.40:0/4290804697 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738011e90 con 0x7f17481f0e60 2024-09-13T18:20:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174e65a700 1 -- 172.21.15.40:0/4290804697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174820cab0 con 0x7f17481f0e60 2024-09-13T18:20:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f174e65a700 1 -- 172.21.15.40:0/4290804697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174820d070 con 0x7f17481f0e60 2024-09-13T18:20:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f173dffb700 1 -- 172.21.15.40:0/4290804697 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1738003940 con 0x7f17481f0e60 2024-09-13T18:20:17.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.574+0000 7f173dffb700 1 -- 172.21.15.40:0/4290804697 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738005070 con 0x7f17481f0e60 2024-09-13T18:20:17.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.578+0000 7f173dffb700 1 -- 172.21.15.40:0/4290804697 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f173801f020 con 0x7f17481f0e60 2024-09-13T18:20:17.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.578+0000 7f174e65a700 1 -- 172.21.15.40:0/4290804697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1728005310 con 0x7f17481f0e60 2024-09-13T18:20:17.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.578+0000 7f173dffb700 1 --2- 172.21.15.40:0/4290804697 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f172c0666e0 0x7f172c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:17.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.578+0000 7f173dffb700 1 -- 172.21.15.40:0/4290804697 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1738084830 con 0x7f17481f0e60 2024-09-13T18:20:17.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.578+0000 7f174d658700 1 --2- 172.21.15.40:0/4290804697 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f172c0666e0 0x7f172c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:17.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.578+0000 7f174d658700 1 --2- 172.21.15.40:0/4290804697 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f172c0666e0 0x7f172c068ba0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1740003d60 tx=0x7f1740009690 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:17.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.586+0000 7f173dffb700 1 -- 172.21.15.40:0/4290804697 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f17380517d0 con 0x7f17481f0e60 2024-09-13T18:20:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.926+0000 7f174e65a700 1 -- 172.21.15.40:0/4290804697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f1728005b40 con 0x7f17481f0e60 2024-09-13T18:20:17.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.930+0000 7f173dffb700 1 -- 172.21.15.40:0/4290804697 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7f1738027620 con 0x7f17481f0e60 2024-09-13T18:20:17.938 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":234,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:20:17.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.930+0000 7f174e65a700 1 -- 172.21.15.40:0/4290804697 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f172c0666e0 msgr2=0x7f172c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:17.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.930+0000 7f174e65a700 1 --2- 172.21.15.40:0/4290804697 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f172c0666e0 0x7f172c068ba0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1740003d60 tx=0x7f1740009690 comp rx=0 tx=0).stop 2024-09-13T18:20:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.930+0000 7f174e65a700 1 -- 172.21.15.40:0/4290804697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17481f0e60 msgr2=0x7f17481ef860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.930+0000 7f174e65a700 1 --2- 172.21.15.40:0/4290804697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17481f0e60 0x7f17481ef860 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1748163b50 tx=0x7f17380059f0 comp rx=0 tx=0).stop 2024-09-13T18:20:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.930+0000 7f174e65a700 1 -- 172.21.15.40:0/4290804697 shutdown_connections 2024-09-13T18:20:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.930+0000 7f174e65a700 1 --2- 172.21.15.40:0/4290804697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f17481f0e60 0x7f17481ef860 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.930+0000 7f174e65a700 1 --2- 172.21.15.40:0/4290804697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17481f15e0 0x7f1748160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:17.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.930+0000 7f174e65a700 1 --2- 172.21.15.40:0/4290804697 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f172c0666e0 0x7f172c068ba0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:17.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.930+0000 7f174e65a700 1 --2- 172.21.15.40:0/4290804697 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f173400b700 0x7f174808aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:17.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.930+0000 7f174e65a700 1 -- 172.21.15.40:0/4290804697 >> 172.21.15.40:0/4290804697 conn(0x7f17480cabc0 msgr2=0x7f17481ee520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:17.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.930+0000 7f174e65a700 1 -- 172.21.15.40:0/4290804697 shutdown_connections 2024-09-13T18:20:17.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:17.930+0000 7f174e65a700 1 -- 172.21.15.40:0/4290804697 wait complete. 2024-09-13T18:20:20.954 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:20:21.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.070+0000 7fdc7f3f3700 1 Processor -- start 2024-09-13T18:20:21.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.070+0000 7fdc7f3f3700 1 -- start start 2024-09-13T18:20:21.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.070+0000 7fdc7f3f3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdc780c55a0 0x7fdc7806ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:21.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.070+0000 7fdc7f3f3700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdc780c5980 0x7fdc78150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:21.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.070+0000 7fdc7f3f3700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc78071100 con 0x7fdc780b1d60 2024-09-13T18:20:21.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.070+0000 7fdc7f3f3700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7806f1a0 con 0x7fdc780c5980 2024-09-13T18:20:21.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.070+0000 7fdc7f3f3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc78070b80 con 0x7fdc780c55a0 2024-09-13T18:20:21.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.070+0000 7fdc7ebf2700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7fdc780c5980 msgr2=0x7fdc78150ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:20:21.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.070+0000 7fdc7dbf0700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdc780b1d60 0x7fdc7815aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58442/0 (socket says 172.21.15.40:58442) 2024-09-13T18:20:21.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.070+0000 7fdc7dbf0700 1 -- 172.21.15.40:0/4084359638 learned_addr learned my addr 172.21.15.40:0/4084359638 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:20:21.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.070+0000 7fdc7ebf2700 1 --2- 172.21.15.40:0/4084359638 >> v2:172.21.15.43:3300/0 conn(0x7fdc780c5980 0x7fdc78150ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:20:21.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.070+0000 7fdc7e3f1700 1 --2- 172.21.15.40:0/4084359638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdc780c55a0 0x7fdc7806ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:21.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.074+0000 7fdc7e3f1700 1 -- 172.21.15.40:0/4084359638 >> v2:172.21.15.43:3300/0 conn(0x7fdc780c5980 msgr2=0x7fdc78150ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:20:21.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.074+0000 7fdc7e3f1700 1 --2- 172.21.15.40:0/4084359638 >> v2:172.21.15.43:3300/0 conn(0x7fdc780c5980 0x7fdc78150ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:21.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.074+0000 7fdc7e3f1700 1 -- 172.21.15.40:0/4084359638 >> v1:172.21.15.40:6789/0 conn(0x7fdc780b1d60 legacy=0x7fdc7815aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:21.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.074+0000 7fdc7e3f1700 1 -- 172.21.15.40:0/4084359638 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc78070e20 con 0x7fdc780c55a0 2024-09-13T18:20:21.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.074+0000 7fdc7e3f1700 1 --2- 172.21.15.40:0/4084359638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdc780c55a0 0x7fdc7806ec60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009850 tx=0x7fdc74009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b55cf6f27f4040d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:21.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.074+0000 7fdc7d3ef700 1 -- 172.21.15.40:0/4084359638 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc74011830 con 0x7fdc780c55a0 2024-09-13T18:20:21.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.074+0000 7fdc7d3ef700 1 -- 172.21.15.40:0/4084359638 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc74011990 con 0x7fdc780c55a0 2024-09-13T18:20:21.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.074+0000 7fdc7d3ef700 1 -- 172.21.15.40:0/4084359638 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc74011d20 con 0x7fdc780c55a0 2024-09-13T18:20:21.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.074+0000 7fdc7f3f3700 1 -- 172.21.15.40:0/4084359638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdc780c55a0 msgr2=0x7fdc7806ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:21.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.074+0000 7fdc7f3f3700 1 --2- 172.21.15.40:0/4084359638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdc780c55a0 0x7fdc7806ec60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009850 tx=0x7fdc74009b60 comp rx=0 tx=0).stop 2024-09-13T18:20:21.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.074+0000 7fdc7f3f3700 1 -- 172.21.15.40:0/4084359638 shutdown_connections 2024-09-13T18:20:21.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.074+0000 7fdc7f3f3700 1 --2- 172.21.15.40:0/4084359638 >> v2:172.21.15.43:3300/0 conn(0x7fdc780c5980 0x7fdc78150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:21.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.074+0000 7fdc7f3f3700 1 --2- 172.21.15.40:0/4084359638 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdc780c55a0 0x7fdc7806ec60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.074+0000 7fdc7f3f3700 1 -- 172.21.15.40:0/4084359638 >> 172.21.15.40:0/4084359638 conn(0x7fdc780cbd80 msgr2=0x7fdc780cc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.074+0000 7fdc7f3f3700 1 -- 172.21.15.40:0/4084359638 shutdown_connections 2024-09-13T18:20:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.074+0000 7fdc7f3f3700 1 -- 172.21.15.40:0/4084359638 wait complete. 2024-09-13T18:20:21.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7f3f3700 1 Processor -- start 2024-09-13T18:20:21.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7f3f3700 1 -- start start 2024-09-13T18:20:21.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7f3f3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc780b1d60 0x7fdc781ed130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:21.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7f3f3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdc780c55a0 0x7fdc7806b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:21.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7f3f3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc780c5980 0x7fdc78156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:21.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7f3f3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc781ed670 con 0x7fdc780c5980 2024-09-13T18:20:21.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7f3f3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc781ee0f0 con 0x7fdc780b1d60 2024-09-13T18:20:21.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7f3f3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc78157c20 con 0x7fdc780c55a0 2024-09-13T18:20:21.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7ebf2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc780c5980 0x7fdc78156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:21.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7ebf2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc780c5980 0x7fdc78156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:54376/0 (socket says 172.21.15.40:54376) 2024-09-13T18:20:21.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7ebf2700 1 -- 172.21.15.40:0/1726487924 learned_addr learned my addr 172.21.15.40:0/1726487924 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:21.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7e3f1700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc780b1d60 msgr2=0x7fdc781ed130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:20:21.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7e3f1700 1 --2- 172.21.15.40:0/1726487924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc780b1d60 0x7fdc781ed130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:20:21.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7dbf0700 1 --2- 172.21.15.40:0/1726487924 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdc780c55a0 0x7fdc7806b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:21.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7ebf2700 1 -- 172.21.15.40:0/1726487924 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdc780c55a0 msgr2=0x7fdc7806b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:21.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7ebf2700 1 --2- 172.21.15.40:0/1726487924 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdc780c55a0 0x7fdc7806b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7ebf2700 1 -- 172.21.15.40:0/1726487924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc780b1d60 msgr2=0x7fdc781ed130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:20:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7ebf2700 1 --2- 172.21.15.40:0/1726487924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc780b1d60 0x7fdc781ed130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7ebf2700 1 -- 172.21.15.40:0/1726487924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc740094b0 con 0x7fdc780c5980 2024-09-13T18:20:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7dbf0700 1 --2- 172.21.15.40:0/1726487924 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdc780c55a0 0x7fdc7806b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:20:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.078+0000 7fdc7ebf2700 1 --2- 172.21.15.40:0/1726487924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc780c5980 0x7fdc78156300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000b990 tx=0x7fdc7000bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.082+0000 7fdc6effd700 1 -- 172.21.15.40:0/1726487924 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7000d8e0 con 0x7fdc780c5980 2024-09-13T18:20:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.082+0000 7fdc6effd700 1 -- 172.21.15.40:0/1726487924 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc70006e90 con 0x7fdc780c5980 2024-09-13T18:20:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.082+0000 7fdc7f3f3700 1 -- 172.21.15.40:0/1726487924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc781ed850 con 0x7fdc780c5980 2024-09-13T18:20:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.082+0000 7fdc6effd700 1 -- 172.21.15.40:0/1726487924 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7000dc60 con 0x7fdc780c5980 2024-09-13T18:20:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.082+0000 7fdc7f3f3700 1 -- 172.21.15.40:0/1726487924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc781ede10 con 0x7fdc780c5980 2024-09-13T18:20:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.082+0000 7fdc7f3f3700 1 -- 172.21.15.40:0/1726487924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc78071530 con 0x7fdc780c5980 2024-09-13T18:20:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.082+0000 7fdc6effd700 1 -- 172.21.15.40:0/1726487924 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdc7000c400 con 0x7fdc780c5980 2024-09-13T18:20:21.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.082+0000 7fdc6effd700 1 --2- 172.21.15.40:0/1726487924 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdc5c0666e0 0x7fdc5c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:21.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.082+0000 7fdc6effd700 1 -- 172.21.15.40:0/1726487924 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fdc70084470 con 0x7fdc780c5980 2024-09-13T18:20:21.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.082+0000 7fdc7e3f1700 1 --2- 172.21.15.40:0/1726487924 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdc5c0666e0 0x7fdc5c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:21.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.086+0000 7fdc7e3f1700 1 --2- 172.21.15.40:0/1726487924 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdc5c0666e0 0x7fdc5c068ba0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdc740090b0 tx=0x7fdc74008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:21.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.090+0000 7fdc6effd700 1 -- 172.21.15.40:0/1726487924 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdc70051420 con 0x7fdc780c5980 2024-09-13T18:20:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.322+0000 7fdc7f3f3700 1 -- 172.21.15.40:0/1726487924 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7fdc78154bc0 con 0x7fdc780c5980 2024-09-13T18:20:21.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.322+0000 7fdc6effd700 1 -- 172.21.15.40:0/1726487924 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7fdc70054a40 con 0x7fdc780c5980 2024-09-13T18:20:21.332 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":234,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:20:21.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.326+0000 7fdc7f3f3700 1 -- 172.21.15.40:0/1726487924 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdc5c0666e0 msgr2=0x7fdc5c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:21.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.326+0000 7fdc7f3f3700 1 --2- 172.21.15.40:0/1726487924 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdc5c0666e0 0x7fdc5c068ba0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdc740090b0 tx=0x7fdc74008030 comp rx=0 tx=0).stop 2024-09-13T18:20:21.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.326+0000 7fdc7f3f3700 1 -- 172.21.15.40:0/1726487924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc780c5980 msgr2=0x7fdc78156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:21.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.326+0000 7fdc7f3f3700 1 --2- 172.21.15.40:0/1726487924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc780c5980 0x7fdc78156300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000b990 tx=0x7fdc7000bca0 comp rx=0 tx=0).stop 2024-09-13T18:20:21.334 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.326+0000 7fdc7f3f3700 1 -- 172.21.15.40:0/1726487924 shutdown_connections 2024-09-13T18:20:21.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.326+0000 7fdc7f3f3700 1 --2- 172.21.15.40:0/1726487924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc780c5980 0x7fdc78156300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:21.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.326+0000 7fdc7f3f3700 1 --2- 172.21.15.40:0/1726487924 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fdc5c0666e0 0x7fdc5c068ba0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:21.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.326+0000 7fdc7f3f3700 1 --2- 172.21.15.40:0/1726487924 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdc780c55a0 0x7fdc7806b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:21.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.326+0000 7fdc7f3f3700 1 --2- 172.21.15.40:0/1726487924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc780b1d60 0x7fdc781ed130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:21.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.326+0000 7fdc7f3f3700 1 -- 172.21.15.40:0/1726487924 >> 172.21.15.40:0/1726487924 conn(0x7fdc780cbd80 msgr2=0x7fdc78150c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:21.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.326+0000 7fdc7f3f3700 1 -- 172.21.15.40:0/1726487924 shutdown_connections 2024-09-13T18:20:21.335 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:21.326+0000 7fdc7f3f3700 1 -- 172.21.15.40:0/1726487924 wait complete. 2024-09-13T18:20:22.564 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-13T18:20:22.564 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-13T18:20:22.601 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-09-13T18:20:22.602 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-13T18:20:22.874 DEBUG:teuthology.parallel:result is None 2024-09-13T18:20:22.875 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-13T18:20:23.379 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-13T18:20:23.379 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2024-09-13T18:20:23.388 INFO:tasks.mon_thrash:joining mon_thrasher 2024-09-13T18:20:24.347 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:20:24.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.458+0000 7f098705e700 1 Processor -- start 2024-09-13T18:20:24.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.458+0000 7f098705e700 1 -- start start 2024-09-13T18:20:24.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.458+0000 7f098705e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 0x7f09800c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:24.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.458+0000 7f098705e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f09800b1af0 0x7f098006e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:24.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.458+0000 7f098705e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09800c84b0 con 0x7f0980157c90 2024-09-13T18:20:24.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.458+0000 7f098705e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098015d720 con 0x7f09801578b0 2024-09-13T18:20:24.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.458+0000 7f098705e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09800717a0 con 0x7f09800b1af0 2024-09-13T18:20:24.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.458+0000 7f098605c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 0x7f09800c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:24.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.458+0000 7f098685d700 1 -- >> v2:172.21.15.43:3300/0 conn(0x7f09800b1af0 msgr2=0x7f098006e3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:20:24.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.458+0000 7f098605c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 0x7f09800c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34508/0 (socket says 172.21.15.40:34508) 2024-09-13T18:20:24.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.458+0000 7f098685d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f09800b1af0 0x7f098006e3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:20:24.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.458+0000 7f098605c700 1 -- 172.21.15.40:0/3262597844 learned_addr learned my addr 172.21.15.40:0/3262597844 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:24.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.458+0000 7f098605c700 1 -- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 msgr2=0x7f09800c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:20:24.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.458+0000 7f098605c700 1 -- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 msgr2=0x7f09800c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:20:24.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.458+0000 7f098605c700 1 --2- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 0x7f09800c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:20:24.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.458+0000 7f098605c700 1 --2- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 0x7f09800c7f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:20:24.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.662+0000 7f098685d700 1 -- 172.21.15.40:0/3262597844 >> v2:172.21.15.43:3300/0 conn(0x7f09800b1af0 msgr2=0x7f098006e3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:20:24.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.662+0000 7f098685d700 1 --2- 172.21.15.40:0/3262597844 >> v2:172.21.15.43:3300/0 conn(0x7f09800b1af0 0x7f098006e3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:20:24.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.662+0000 7f098605c700 1 --2- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 0x7f09800c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:24.670 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.662+0000 7f098605c700 1 -- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 msgr2=0x7f09800c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:20:24.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.662+0000 7f098605c700 1 -- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 msgr2=0x7f09800c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:20:24.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.662+0000 7f098605c700 1 --2- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 0x7f09800c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:20:24.671 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:24.662+0000 7f098605c700 1 --2- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 0x7f09800c7f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:20:25.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:25.066+0000 7f098605c700 1 --2- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 0x7f09800c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:25.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:25.066+0000 7f098685d700 1 -- 172.21.15.40:0/3262597844 >> v2:172.21.15.43:3300/0 conn(0x7f09800b1af0 msgr2=0x7f098006e3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:20:25.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:25.066+0000 7f098685d700 1 --2- 172.21.15.40:0/3262597844 >> v2:172.21.15.43:3300/0 conn(0x7f09800b1af0 0x7f098006e3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:20:25.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:25.066+0000 7f098605c700 1 -- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 msgr2=0x7f09800c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:20:25.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:25.066+0000 7f098605c700 1 -- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 msgr2=0x7f09800c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:20:25.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:25.066+0000 7f098605c700 1 --2- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 0x7f09800c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:20:25.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:25.066+0000 7f098605c700 1 --2- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 0x7f09800c7f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T18:20:25.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:25.870+0000 7f098685d700 1 -- 172.21.15.40:0/3262597844 >> v2:172.21.15.43:3300/0 conn(0x7f09800b1af0 msgr2=0x7f098006e3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:20:25.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:25.870+0000 7f098685d700 1 --2- 172.21.15.40:0/3262597844 >> v2:172.21.15.43:3300/0 conn(0x7f09800b1af0 0x7f098006e3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:20:25.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:25.870+0000 7f098605c700 1 --2- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 0x7f09800c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:25.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:25.870+0000 7f098605c700 1 -- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 msgr2=0x7f09800c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-13T18:20:25.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:25.870+0000 7f098605c700 1 -- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 msgr2=0x7f09800c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:20:25.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:25.870+0000 7f098605c700 1 --2- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 0x7f09800c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:20:25.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:25.870+0000 7f098605c700 1 --2- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 0x7f09800c7f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-13T18:20:27.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:27.462+0000 7f0973fff700 1 -- 172.21.15.40:0/3262597844 >> v2:172.21.15.43:3300/0 conn(0x7f09800b1af0 msgr2=0x7f098006e3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:20:27.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:27.462+0000 7f0973fff700 1 --2- 172.21.15.40:0/3262597844 >> v2:172.21.15.43:3300/0 conn(0x7f09800b1af0 0x7f098006e3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:27.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:27.462+0000 7f0973fff700 1 -- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 msgr2=0x7f09800c7f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:20:27.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:27.462+0000 7f0973fff700 1 --2- 172.21.15.40:0/3262597844 >> v2:172.21.15.40:3300/0 conn(0x7f09801578b0 0x7f09800c7f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:27.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:27.462+0000 7f0973fff700 1 -- 172.21.15.40:0/3262597844 >> v1:172.21.15.40:6789/0 conn(0x7f0980157c90 legacy=0x7f098015e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:27.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:27.462+0000 7f0973fff700 1 --2- 172.21.15.40:0/3262597844 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 0x7f09640098d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:27.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:27.462+0000 7f0973fff700 1 -- 172.21.15.40:0/3262597844 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f098015dc60 con 0x7f0964003730 2024-09-13T18:20:27.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:27.462+0000 7f0973fff700 1 -- 172.21.15.40:0/3262597844 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098015d720 con 0x7f0964007410 2024-09-13T18:20:27.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:27.462+0000 7f098605c700 1 -- 172.21.15.40:0/3262597844 >> v1:172.21.15.43:6789/0 conn(0x7f0964003730 legacy=0x7f0964005bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:20:27.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:27.466+0000 7f098585b700 1 --2- 172.21.15.40:0/3262597844 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 0x7f09640098d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:27.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:27.466+0000 7f098585b700 1 -- 172.21.15.40:0/3262597844 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 msgr2=0x7f09640098d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:20:27.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:27.466+0000 7f098585b700 1 -- 172.21.15.40:0/3262597844 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 msgr2=0x7f09640098d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:20:27.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:27.466+0000 7f098585b700 1 --2- 172.21.15.40:0/3262597844 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 0x7f09640098d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:20:27.474 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:27.466+0000 7f098585b700 1 --2- 172.21.15.40:0/3262597844 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 0x7f09640098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:20:27.674 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:27.666+0000 7f098605c700 1 -- 172.21.15.40:0/3262597844 >> v1:172.21.15.43:6789/0 conn(0x7f0964003730 legacy=0x7f0964005bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:20:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:27.670+0000 7f098585b700 1 --2- 172.21.15.40:0/3262597844 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 0x7f09640098d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:27.670+0000 7f098585b700 1 -- 172.21.15.40:0/3262597844 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 msgr2=0x7f09640098d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:20:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:27.670+0000 7f098585b700 1 -- 172.21.15.40:0/3262597844 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 msgr2=0x7f09640098d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:20:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:27.670+0000 7f098585b700 1 --2- 172.21.15.40:0/3262597844 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 0x7f09640098d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:20:27.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:27.670+0000 7f098585b700 1 --2- 172.21.15.40:0/3262597844 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 0x7f09640098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T18:20:28.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.070+0000 7f098605c700 1 -- 172.21.15.40:0/3262597844 >> v1:172.21.15.43:6789/0 conn(0x7f0964003730 legacy=0x7f0964005bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:20:28.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.074+0000 7f098585b700 1 --2- 172.21.15.40:0/3262597844 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 0x7f09640098d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.074+0000 7f098585b700 1 -- 172.21.15.40:0/3262597844 >> v1:172.21.15.43:6789/0 conn(0x7f0964003730 legacy=0x7f0964005bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:20:28.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.074+0000 7f098585b700 1 -- 172.21.15.40:0/3262597844 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096400b0f0 con 0x7f0964007410 2024-09-13T18:20:28.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.074+0000 7f098585b700 1 --2- 172.21.15.40:0/3262597844 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 0x7f09640098d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0974005350 tx=0x7f0974005660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=681576f892174057 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:28.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.074+0000 7f098505a700 1 -- 172.21.15.40:0/3262597844 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974001290 con 0x7f0964007410 2024-09-13T18:20:28.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.074+0000 7f098505a700 1 -- 172.21.15.40:0/3262597844 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09740013f0 con 0x7f0964007410 2024-09-13T18:20:28.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.074+0000 7f098505a700 1 -- 172.21.15.40:0/3262597844 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974001780 con 0x7f0964007410 2024-09-13T18:20:28.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.074+0000 7f098705e700 1 -- 172.21.15.40:0/3262597844 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 msgr2=0x7f09640098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.074+0000 7f098705e700 1 --2- 172.21.15.40:0/3262597844 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 0x7f09640098d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0974005350 tx=0x7f0974005660 comp rx=0 tx=0).stop 2024-09-13T18:20:28.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.074+0000 7f098685d700 1 -- 172.21.15.40:0/3262597844 reap_dead start 2024-09-13T18:20:28.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.074+0000 7f098705e700 1 -- 172.21.15.40:0/3262597844 shutdown_connections 2024-09-13T18:20:28.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.074+0000 7f098705e700 1 -- 172.21.15.40:0/3262597844 >> 172.21.15.40:0/3262597844 conn(0x7f09800cbd80 msgr2=0x7f09800cc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:28.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.074+0000 7f098705e700 1 -- 172.21.15.40:0/3262597844 shutdown_connections 2024-09-13T18:20:28.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.074+0000 7f098705e700 1 -- 172.21.15.40:0/3262597844 wait complete. 2024-09-13T18:20:28.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.078+0000 7f098705e700 1 Processor -- start 2024-09-13T18:20:28.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.078+0000 7f098705e700 1 -- start start 2024-09-13T18:20:28.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.078+0000 7f098705e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 0x7f098014d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.078+0000 7f098705e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f098015b360 0x7f0980150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.078+0000 7f098705e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09801f6ad0 0x7f09801f6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.078+0000 7f098705e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09801f7490 con 0x7f098015b360 2024-09-13T18:20:28.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.078+0000 7f098685d700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09801f6ad0 msgr2=0x7f09801f6f50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:20:28.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.078+0000 7f098705e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098014da40 con 0x7f09801f6ad0 2024-09-13T18:20:28.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.078+0000 7f098685d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09801f6ad0 0x7f09801f6f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:20:28.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.078+0000 7f098705e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098014bc90 con 0x7f0964007410 2024-09-13T18:20:28.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.078+0000 7f098585b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f098015b360 0x7f0980150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.078+0000 7f098605c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 0x7f098014d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.078+0000 7f098605c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 0x7f098014d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:38204/0 (socket says 172.21.15.40:38204) 2024-09-13T18:20:28.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.078+0000 7f098605c700 1 -- 172.21.15.40:0/3299560806 learned_addr learned my addr 172.21.15.40:0/3299560806 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:28.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.082+0000 7f098605c700 1 -- 172.21.15.40:0/3299560806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09801f6ad0 msgr2=0x7f09801f6f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:20:28.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.082+0000 7f098605c700 1 --2- 172.21.15.40:0/3299560806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09801f6ad0 0x7f09801f6f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.082+0000 7f098605c700 1 -- 172.21.15.40:0/3299560806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f098015b360 msgr2=0x7f0980150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.082+0000 7f098605c700 1 --2- 172.21.15.40:0/3299560806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f098015b360 0x7f0980150430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.082+0000 7f098605c700 1 -- 172.21.15.40:0/3299560806 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097400f040 con 0x7f0964007410 2024-09-13T18:20:28.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.082+0000 7f098585b700 1 --2- 172.21.15.40:0/3299560806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f098015b360 0x7f0980150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:20:28.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.082+0000 7f098605c700 1 --2- 172.21.15.40:0/3299560806 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 0x7f098014d400 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f097c010700 tx=0x7f097c010a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:28.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.082+0000 7f0972ffd700 1 -- 172.21.15.40:0/3299560806 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097c0168a0 con 0x7f0964007410 2024-09-13T18:20:28.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.082+0000 7f0972ffd700 1 -- 172.21.15.40:0/3299560806 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097c00c040 con 0x7f0964007410 2024-09-13T18:20:28.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.082+0000 7f098705e700 1 -- 172.21.15.40:0/3299560806 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f098014bf70 con 0x7f0964007410 2024-09-13T18:20:28.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.082+0000 7f0972ffd700 1 -- 172.21.15.40:0/3299560806 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097c016c60 con 0x7f0964007410 2024-09-13T18:20:28.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.082+0000 7f098705e700 1 -- 172.21.15.40:0/3299560806 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f098014c530 con 0x7f0964007410 2024-09-13T18:20:28.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.082+0000 7f098705e700 1 -- 172.21.15.40:0/3299560806 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f098006b730 con 0x7f0964007410 2024-09-13T18:20:28.093 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.082+0000 7f0972ffd700 1 -- 172.21.15.40:0/3299560806 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f097c017470 con 0x7f0964007410 2024-09-13T18:20:28.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.086+0000 7f0972ffd700 1 --2- 172.21.15.40:0/3299560806 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f09640675a0 0x7f0964069a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.086+0000 7f0972ffd700 1 -- 172.21.15.40:0/3299560806 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f097c085550 con 0x7f0964007410 2024-09-13T18:20:28.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.086+0000 7f098585b700 1 --2- 172.21.15.40:0/3299560806 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f09640675a0 0x7f0964069a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.086+0000 7f098585b700 1 --2- 172.21.15.40:0/3299560806 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f09640675a0 0x7f0964069a60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f09740056a0 tx=0x7f0974010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:28.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.094+0000 7f0972ffd700 1 -- 172.21.15.40:0/3299560806 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f097c052550 con 0x7f0964007410 2024-09-13T18:20:28.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.326+0000 7f098705e700 1 -- 172.21.15.40:0/3299560806 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0980070ae0 con 0x7f0964007410 2024-09-13T18:20:28.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.330+0000 7f0972ffd700 1 -- 172.21.15.40:0/3299560806 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1534 (secure 0 0 0) 0x7f097c055b70 con 0x7f0964007410 2024-09-13T18:20:28.338 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":238,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:20:28.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.330+0000 7f098705e700 1 -- 172.21.15.40:0/3299560806 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f09640675a0 msgr2=0x7f0964069a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.330+0000 7f098705e700 1 --2- 172.21.15.40:0/3299560806 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f09640675a0 0x7f0964069a60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f09740056a0 tx=0x7f0974010000 comp rx=0 tx=0).stop 2024-09-13T18:20:28.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.330+0000 7f098705e700 1 -- 172.21.15.40:0/3299560806 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 msgr2=0x7f098014d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.330+0000 7f098705e700 1 --2- 172.21.15.40:0/3299560806 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 0x7f098014d400 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f097c010700 tx=0x7f097c010a10 comp rx=0 tx=0).stop 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.330+0000 7f098705e700 1 -- 172.21.15.40:0/3299560806 shutdown_connections 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.330+0000 7f098705e700 1 --2- 172.21.15.40:0/3299560806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09801f6ad0 0x7f09801f6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.330+0000 7f098705e700 1 --2- 172.21.15.40:0/3299560806 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f098015b360 0x7f0980150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.330+0000 7f098705e700 1 --2- 172.21.15.40:0/3299560806 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f09640675a0 0x7f0964069a60 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.330+0000 7f098705e700 1 --2- 172.21.15.40:0/3299560806 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0964007410 0x7f098014d400 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.330+0000 7f098705e700 1 -- 172.21.15.40:0/3299560806 >> 172.21.15.40:0/3299560806 conn(0x7f09800cbd80 msgr2=0x7f09800c17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.330+0000 7f098705e700 1 -- 172.21.15.40:0/3299560806 shutdown_connections 2024-09-13T18:20:28.341 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.330+0000 7f098705e700 1 -- 172.21.15.40:0/3299560806 wait complete. 2024-09-13T18:20:28.352 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-13T18:20:28.352 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-13T18:20:28.353 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:20:28.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.526+0000 7fd69f19e700 1 Processor -- start 2024-09-13T18:20:28.534 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.526+0000 7fd69f19e700 1 -- start start 2024-09-13T18:20:28.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.530+0000 7fd69f19e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd6900361d0 0x7fd690007930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.530+0000 7fd69f19e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd690051160 0x7fd69004c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.530+0000 7fd69f19e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd690006890 con 0x7fd69004b330 2024-09-13T18:20:28.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.530+0000 7fd69f19e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd690006b70 con 0x7fd6900361d0 2024-09-13T18:20:28.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.530+0000 7fd69f19e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd690006e50 con 0x7fd690051160 2024-09-13T18:20:28.539 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.530+0000 7fd69e99d700 1 -- >> v1:172.21.15.43:6789/0 conn(0x7fd69004b330 legacy=0x7fd6900ddfc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.43:6789/0 2024-09-13T18:20:28.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.530+0000 7fd69d99b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd690051160 0x7fd69004c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.530+0000 7fd69d99b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd690051160 0x7fd69004c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:38216/0 (socket says 172.21.15.40:38216) 2024-09-13T18:20:28.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.530+0000 7fd69d99b700 1 -- 172.21.15.40:0/531574823 learned_addr learned my addr 172.21.15.40:0/531574823 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:28.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.530+0000 7fd69e19c700 1 -- 172.21.15.40:0/531574823 >> v2:172.21.15.43:3300/0 conn(0x7fd6900361d0 msgr2=0x7fd690007930 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:20:28.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.534+0000 7fd69d99b700 1 -- 172.21.15.40:0/531574823 >> v2:172.21.15.43:3300/0 conn(0x7fd6900361d0 msgr2=0x7fd690007930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:20:28.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.538+0000 7fd69e19c700 1 --2- 172.21.15.40:0/531574823 >> v2:172.21.15.43:3300/0 conn(0x7fd6900361d0 0x7fd690007930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:20:28.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.538+0000 7fd69d99b700 1 --2- 172.21.15.40:0/531574823 >> v2:172.21.15.43:3300/0 conn(0x7fd6900361d0 0x7fd690007930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.538+0000 7fd69d99b700 1 -- 172.21.15.40:0/531574823 >> v1:172.21.15.43:6789/0 conn(0x7fd69004b330 legacy=0x7fd6900ddfc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:20:28.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.538+0000 7fd69d99b700 1 -- 172.21.15.40:0/531574823 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd690007130 con 0x7fd690051160 2024-09-13T18:20:28.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.538+0000 7fd69d99b700 1 --2- 172.21.15.40:0/531574823 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd690051160 0x7fd69004c370 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd688006fd0 tx=0x7fd68800bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c990375e9008bc0f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:28.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.538+0000 7fd69d19a700 1 -- 172.21.15.40:0/531574823 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68800dc00 con 0x7fd690051160 2024-09-13T18:20:28.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.538+0000 7fd69d19a700 1 -- 172.21.15.40:0/531574823 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68800dd60 con 0x7fd690051160 2024-09-13T18:20:28.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.538+0000 7fd69d19a700 1 -- 172.21.15.40:0/531574823 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688017640 con 0x7fd690051160 2024-09-13T18:20:28.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.538+0000 7fd69f19e700 1 -- 172.21.15.40:0/531574823 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd690051160 msgr2=0x7fd69004c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.538+0000 7fd69f19e700 1 --2- 172.21.15.40:0/531574823 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd690051160 0x7fd69004c370 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd688006fd0 tx=0x7fd68800bea0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.542+0000 7fd69f19e700 1 -- 172.21.15.40:0/531574823 shutdown_connections 2024-09-13T18:20:28.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.542+0000 7fd69f19e700 1 --2- 172.21.15.40:0/531574823 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd690051160 0x7fd69004c370 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.542+0000 7fd69f19e700 1 --2- 172.21.15.40:0/531574823 >> v2:172.21.15.43:3300/0 conn(0x7fd6900361d0 0x7fd690007930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.542+0000 7fd69f19e700 1 -- 172.21.15.40:0/531574823 >> 172.21.15.40:0/531574823 conn(0x7fd6900505b0 msgr2=0x7fd6900509c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:28.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.542+0000 7fd69f19e700 1 -- 172.21.15.40:0/531574823 shutdown_connections 2024-09-13T18:20:28.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.542+0000 7fd69f19e700 1 -- 172.21.15.40:0/531574823 wait complete. 2024-09-13T18:20:28.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.542+0000 7fd69f19e700 1 Processor -- start 2024-09-13T18:20:28.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.542+0000 7fd69f19e700 1 -- start start 2024-09-13T18:20:28.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.542+0000 7fd69f19e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6900361d0 0x7fd690162ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.542+0000 7fd69f19e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69004b330 0x7fd69004a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.542+0000 7fd69f19e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd690051160 0x7fd6900478b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.542+0000 7fd69f19e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd690170330 con 0x7fd69004b330 2024-09-13T18:20:28.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.542+0000 7fd69f19e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd690163fd0 con 0x7fd6900361d0 2024-09-13T18:20:28.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.542+0000 7fd69f19e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd690170b50 con 0x7fd690051160 2024-09-13T18:20:28.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.546+0000 7fd69e19c700 1 -- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6900361d0 msgr2=0x7fd690162ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:20:28.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.546+0000 7fd69e19c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6900361d0 0x7fd690162ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:20:28.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.546+0000 7fd69d99b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69004b330 0x7fd69004a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.546+0000 7fd69d99b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69004b330 0x7fd69004a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34568/0 (socket says 172.21.15.40:34568) 2024-09-13T18:20:28.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.546+0000 7fd69d99b700 1 -- 172.21.15.40:0/1247287646 learned_addr learned my addr 172.21.15.40:0/1247287646 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:28.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.546+0000 7fd69e99d700 1 --2- 172.21.15.40:0/1247287646 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd690051160 0x7fd6900478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.546+0000 7fd69d99b700 1 -- 172.21.15.40:0/1247287646 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd690051160 msgr2=0x7fd6900478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.546+0000 7fd69d99b700 1 --2- 172.21.15.40:0/1247287646 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd690051160 0x7fd6900478b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.546+0000 7fd69d99b700 1 -- 172.21.15.40:0/1247287646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6900361d0 msgr2=0x7fd690162ad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:20:28.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.546+0000 7fd69d99b700 1 --2- 172.21.15.40:0/1247287646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6900361d0 0x7fd690162ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.546+0000 7fd69d99b700 1 -- 172.21.15.40:0/1247287646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68800b820 con 0x7fd69004b330 2024-09-13T18:20:28.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.546+0000 7fd69e99d700 1 --2- 172.21.15.40:0/1247287646 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd690051160 0x7fd6900478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:28.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.546+0000 7fd69d99b700 1 --2- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69004b330 0x7fd69004a7b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd690036710 tx=0x7fd688004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:28.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.550+0000 7fd68effd700 1 -- 172.21.15.40:0/1247287646 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688017e00 con 0x7fd69004b330 2024-09-13T18:20:28.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.550+0000 7fd68effd700 1 -- 172.21.15.40:0/1247287646 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd688022420 con 0x7fd69004b330 2024-09-13T18:20:28.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.550+0000 7fd69f19e700 1 -- 172.21.15.40:0/1247287646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd690047df0 con 0x7fd69004b330 2024-09-13T18:20:28.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.550+0000 7fd68effd700 1 -- 172.21.15.40:0/1247287646 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6880227d0 con 0x7fd69004b330 2024-09-13T18:20:28.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.550+0000 7fd69f19e700 1 -- 172.21.15.40:0/1247287646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd690163a10 con 0x7fd69004b330 2024-09-13T18:20:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.550+0000 7fd68effd700 1 -- 172.21.15.40:0/1247287646 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd68801e040 con 0x7fd69004b330 2024-09-13T18:20:28.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.550+0000 7fd68effd700 1 --2- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd67c066760 0x7fd67c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.550+0000 7fd68effd700 1 -- 172.21.15.40:0/1247287646 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd688087d60 con 0x7fd69004b330 2024-09-13T18:20:28.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.550+0000 7fd69e19c700 1 --2- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd67c066760 0x7fd67c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.562 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.554+0000 7fd69e19c700 1 --2- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd67c066760 0x7fd67c068c20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd694003a30 tx=0x7fd694009270 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:28.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.562+0000 7fd69f19e700 1 --2- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd690046500 0x7fd6900d7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.569 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.562+0000 7fd69f19e700 1 -- 172.21.15.40:0/1247287646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd690008d20 con 0x7fd690046500 2024-09-13T18:20:28.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.562+0000 7fd69e99d700 1 --2- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd690046500 0x7fd6900d7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.573 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.566+0000 7fd69e99d700 1 -- 172.21.15.40:0/1247287646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd690033410 con 0x7fd690046500 2024-09-13T18:20:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.566+0000 7fd69e99d700 1 --2- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd690046500 0x7fd6900d7700 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd69804ffd0 tx=0x7fd6980747c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.566+0000 7fd68effd700 1 -- 172.21.15.40:0/1247287646 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698072340 con 0x7fd690046500 2024-09-13T18:20:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.566+0000 7fd68effd700 1 -- 172.21.15.40:0/1247287646 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fd6980724a0 con 0x7fd690046500 2024-09-13T18:20:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.566+0000 7fd68effd700 1 -- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd690046500 msgr2=0x7fd6900d7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.574 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.566+0000 7fd68effd700 1 --2- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd690046500 0x7fd6900d7700 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd69804ffd0 tx=0x7fd6980747c0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.566+0000 7fd68effd700 1 -- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd690046500 msgr2=0x7fd6900d7700 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:20:28.575 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.566+0000 7fd68effd700 1 --2- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd690046500 0x7fd6900d7700 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.586+0000 7fd69f19e700 1 --2- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd690170720 0x7fd6900d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.586+0000 7fd69f19e700 1 -- 172.21.15.40:0/1247287646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd690170cc0 con 0x7fd690170720 2024-09-13T18:20:28.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.586+0000 7fd69e99d700 1 --2- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd690170720 0x7fd6900d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.586+0000 7fd69e99d700 1 -- 172.21.15.40:0/1247287646 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fd690163d50 con 0x7fd690170720 2024-09-13T18:20:28.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.586+0000 7fd69e99d700 1 --2- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd690170720 0x7fd6900d3e10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd69802ce60 tx=0x7fd698068470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:28.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.586+0000 7fd68effd700 1 -- 172.21.15.40:0/1247287646 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69809c070 con 0x7fd690170720 2024-09-13T18:20:28.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.586+0000 7fd68effd700 1 -- 172.21.15.40:0/1247287646 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4939 (secure 0 0 0) 0x7fd6980970d0 con 0x7fd690170720 2024-09-13T18:20:28.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.586+0000 7fd68effd700 1 -- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd690170720 msgr2=0x7fd6900d3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.586+0000 7fd68effd700 1 --2- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd690170720 0x7fd6900d3e10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd69802ce60 tx=0x7fd698068470 comp rx=0 tx=0).stop 2024-09-13T18:20:28.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.586+0000 7fd68effd700 1 -- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd690170720 msgr2=0x7fd6900d3e10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:20:28.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.590+0000 7fd68effd700 1 --2- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd690170720 0x7fd6900d3e10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.601 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:20:28.601 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:20:28.601 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:20:28.601 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:20:28.601 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 238, 2024-09-13T18:20:28.602 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:20:28.602 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:20:28.602 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:20:28.602 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:28.602 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 0, 2024-09-13T18:20:28.602 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:20:28.602 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:20:28.602 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:20:28.602 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:28.602 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:28.602 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:28.602 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:28.602 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:28.602 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:28.603 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:28.603 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:28.603 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:28.603 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:28.603 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:28.603 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:20:28.603 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:20:28.603 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:28.603 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:28.603 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:28.603 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:28.603 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:28.603 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:28.603 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:28.604 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:28.604 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:28.604 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:28.604 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:28.604 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:28.604 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:20:28.604 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:20:28.604 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:20:28.604 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:20:28.604 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:20:28.604 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:20:28.604 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:20:28.604 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:20:28.605 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:20:28.605 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:20:28.605 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:20:28.605 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:20:28.605 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:20:28.605 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:20:28.605 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:20:28.605 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:20:28.605 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:20:28.605 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:28.605 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:28.605 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:28.605 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:28.605 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:28.606 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:28.606 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:28.606 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:28.606 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:28.606 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:28.606 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:28.606 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:20:28.606 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:28.606 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:20:28.606 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:28.606 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:20:28.606 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:20:28.606 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:28.607 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:28.607 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:28.607 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:28.607 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:20:28.607 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:28.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:28.607 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:28.607 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:28.607 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:20:28.607 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:28.607 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:28.607 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:28.607 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:28.607 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:20:28.607 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:20:28.608 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:28.608 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:28.608 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:28.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:28.608 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:28.608 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:20:28.608 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:20:28.608 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:28.608 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:28.608 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:28.608 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:28.608 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:20:28.608 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:28.608 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:28.609 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:28.609 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:28.609 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:20:28.609 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:28.609 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:28.609 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:28.609 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:28.609 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:20:28.609 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:20:28.609 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:28.609 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:28.609 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:28.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:28.610 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:28.610 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:20:28.610 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:20:28.610 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:28.610 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:28.610 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:28.610 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:28.610 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:20:28.610 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:28.610 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:28.610 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:28.610 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:28.611 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:20:28.611 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:28.611 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:28.611 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:28.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:28.611 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:20:28.611 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:20:28.611 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:28.611 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:28.611 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:28.611 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:28.611 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:28.611 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:28.612 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:20:28.612 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:20:28.612 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:28.612 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:28.612 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:28.612 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:20:28.612 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:28.612 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:28.612 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:20:28.612 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:28.612 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:28.612 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:28.612 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:20:28.613 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:28.613 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:28.613 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:20:28.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:28.613 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:28.613 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:28.613 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:20:28.613 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:28.613 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:28.613 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:20:28.613 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:28.613 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:28.613 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:28.614 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:20:28.614 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:28.614 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:28.614 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:20:28.614 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:28.614 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:28.614 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:28.614 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:20:28.614 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:28.614 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:28.614 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:28.614 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:20:28.614 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:20:28.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.618+0000 7fd69f19e700 1 -- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd67c066760 msgr2=0x7fd67c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.618+0000 7fd69f19e700 1 --2- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fd67c066760 0x7fd67c068c20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd694003a30 tx=0x7fd694009270 comp rx=0 tx=0).stop 2024-09-13T18:20:28.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.618+0000 7fd69f19e700 1 -- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69004b330 msgr2=0x7fd69004a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.618+0000 7fd69f19e700 1 --2- 172.21.15.40:0/1247287646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd69004b330 0x7fd69004a7b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd690036710 tx=0x7fd688004770 comp rx=0 tx=0).stop 2024-09-13T18:20:28.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.618+0000 7fd69e99d700 1 -- 172.21.15.40:0/1247287646 reap_dead start 2024-09-13T18:20:28.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.618+0000 7fd69f19e700 1 -- 172.21.15.40:0/1247287646 shutdown_connections 2024-09-13T18:20:28.626 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.618+0000 7fd69f19e700 1 -- 172.21.15.40:0/1247287646 >> 172.21.15.40:0/1247287646 conn(0x7fd6900505b0 msgr2=0x7fd6900db240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:28.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.618+0000 7fd69f19e700 1 -- 172.21.15.40:0/1247287646 shutdown_connections 2024-09-13T18:20:28.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.622+0000 7fd69f19e700 1 -- 172.21.15.40:0/1247287646 wait complete. 2024-09-13T18:20:28.656 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:20:28.656 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-13T18:20:28.656 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:20:28.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.938+0000 7ff42f59e700 1 Processor -- start 2024-09-13T18:20:28.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.938+0000 7ff42f59e700 1 -- start start 2024-09-13T18:20:28.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.938+0000 7ff42f59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff4300724e0 0x7ff4300c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.938+0000 7ff42f59e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff430158bc0 0x7ff43015ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.938+0000 7ff42f59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4300c6890 con 0x7ff43014f8c0 2024-09-13T18:20:28.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.938+0000 7ff42f59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4300c7700 con 0x7ff4300724e0 2024-09-13T18:20:28.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.938+0000 7ff42f59e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4300c5f30 con 0x7ff430158bc0 2024-09-13T18:20:28.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.938+0000 7ff42e59c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff43014f8c0 0x7ff43006d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:42500/0 (socket says 172.21.15.40:42500) 2024-09-13T18:20:28.947 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.938+0000 7ff42e59c700 1 -- 172.21.15.40:0/1427503086 learned_addr learned my addr 172.21.15.40:0/1427503086 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:20:28.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.938+0000 7ff42ed9d700 1 -- 172.21.15.40:0/1427503086 >> v2:172.21.15.43:3300/0 conn(0x7ff430158bc0 msgr2=0x7ff43015ff50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:20:28.948 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.938+0000 7ff42ed9d700 1 --2- 172.21.15.40:0/1427503086 >> v2:172.21.15.43:3300/0 conn(0x7ff430158bc0 0x7ff43015ff50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:20:28.949 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.942+0000 7ff42d59a700 1 -- 172.21.15.40:0/1427503086 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2040431433 0 0) 0x7ff4300c6890 con 0x7ff43014f8c0 2024-09-13T18:20:28.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.942+0000 7ff42d59a700 1 -- 172.21.15.40:0/1427503086 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4180035d0 con 0x7ff43014f8c0 2024-09-13T18:20:28.950 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.942+0000 7ff42dd9b700 1 --2- 172.21.15.40:0/1427503086 >> v2:172.21.15.40:3300/0 conn(0x7ff4300724e0 0x7ff4300c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.942+0000 7ff42d59a700 1 -- 172.21.15.40:0/1427503086 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3151622188 0 0) 0x7ff4180035d0 con 0x7ff43014f8c0 2024-09-13T18:20:28.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.942+0000 7ff42d59a700 1 -- 172.21.15.40:0/1427503086 >> v2:172.21.15.43:3300/0 conn(0x7ff430158bc0 msgr2=0x7ff43015ff50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:20:28.951 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.942+0000 7ff42d59a700 1 --2- 172.21.15.40:0/1427503086 >> v2:172.21.15.43:3300/0 conn(0x7ff430158bc0 0x7ff43015ff50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.942+0000 7ff42d59a700 1 -- 172.21.15.40:0/1427503086 >> v2:172.21.15.40:3300/0 conn(0x7ff4300724e0 msgr2=0x7ff4300c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:20:28.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.942+0000 7ff42d59a700 1 --2- 172.21.15.40:0/1427503086 >> v2:172.21.15.40:3300/0 conn(0x7ff4300724e0 0x7ff4300c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.942+0000 7ff42d59a700 1 -- 172.21.15.40:0/1427503086 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4300c1320 con 0x7ff43014f8c0 2024-09-13T18:20:28.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.942+0000 7ff42d59a700 1 -- 172.21.15.40:0/1427503086 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7ff420004d60 con 0x7ff43014f8c0 2024-09-13T18:20:28.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.942+0000 7ff42dd9b700 1 --2- 172.21.15.40:0/1427503086 >> v2:172.21.15.40:3300/0 conn(0x7ff4300724e0 0x7ff4300c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:20:28.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.942+0000 7ff42d59a700 1 -- 172.21.15.40:0/1427503086 >> v1:172.21.15.40:6789/0 conn(0x7ff43014f8c0 legacy=0x7ff43006d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.942+0000 7ff42d59a700 1 --2- 172.21.15.40:0/1427503086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff418003a20 0x7ff418003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.942+0000 7ff42e59c700 1 -- 172.21.15.40:0/1427503086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff418003a20 msgr2=0x7ff418003e00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.43:3300/0 2024-09-13T18:20:28.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.942+0000 7ff42e59c700 1 --2- 172.21.15.40:0/1427503086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff418003a20 0x7ff418003e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:20:28.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.946+0000 7ff42d59a700 1 --2- 172.21.15.40:0/1427503086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff418004870 0x7ff418004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.946+0000 7ff42d59a700 1 --2- 172.21.15.40:0/1427503086 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff418005230 0x7ff41800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.946+0000 7ff42d59a700 1 -- 172.21.15.40:0/1427503086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4300c7700 con 0x7ff418004870 2024-09-13T18:20:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.946+0000 7ff42d59a700 1 -- 172.21.15.40:0/1427503086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4300c5f30 con 0x7ff418003a20 2024-09-13T18:20:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.946+0000 7ff42d59a700 1 -- 172.21.15.40:0/1427503086 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41800c100 con 0x7ff418005230 2024-09-13T18:20:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.946+0000 7ff42dd9b700 1 --2- 172.21.15.40:0/1427503086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff418004870 0x7ff418004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.946+0000 7ff42ed9d700 1 --2- 172.21.15.40:0/1427503086 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff418005230 0x7ff41800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.946+0000 7ff42dd9b700 1 -- 172.21.15.40:0/1427503086 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff418005230 msgr2=0x7ff41800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:20:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.946+0000 7ff42dd9b700 1 --2- 172.21.15.40:0/1427503086 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff418005230 0x7ff41800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.946+0000 7ff42dd9b700 1 -- 172.21.15.40:0/1427503086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff418003a20 msgr2=0x7ff418003e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:20:28.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.946+0000 7ff42dd9b700 1 --2- 172.21.15.40:0/1427503086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff418003a20 0x7ff418003e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.946+0000 7ff42dd9b700 1 -- 172.21.15.40:0/1427503086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4180035d0 con 0x7ff418004870 2024-09-13T18:20:28.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.946+0000 7ff42ed9d700 1 -- 172.21.15.40:0/1427503086 reap_dead start 2024-09-13T18:20:28.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.946+0000 7ff42dd9b700 1 --2- 172.21.15.40:0/1427503086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff418004870 0x7ff418004cf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff42400abc0 tx=0x7ff4180058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b5651e77303e80 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:28.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.946+0000 7ff42d59a700 1 -- 172.21.15.40:0/1427503086 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff424012070 con 0x7ff418004870 2024-09-13T18:20:28.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.946+0000 7ff42d59a700 1 -- 172.21.15.40:0/1427503086 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff424004030 con 0x7ff418004870 2024-09-13T18:20:28.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.946+0000 7ff42d59a700 1 -- 172.21.15.40:0/1427503086 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42400a880 con 0x7ff418004870 2024-09-13T18:20:28.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.950+0000 7ff42f59e700 1 -- 172.21.15.40:0/1427503086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff418004870 msgr2=0x7ff418004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.950+0000 7ff42f59e700 1 --2- 172.21.15.40:0/1427503086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff418004870 0x7ff418004cf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff42400abc0 tx=0x7ff4180058a0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.954+0000 7ff42f59e700 1 -- 172.21.15.40:0/1427503086 shutdown_connections 2024-09-13T18:20:28.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.954+0000 7ff42f59e700 1 --2- 172.21.15.40:0/1427503086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff418004870 0x7ff418004cf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.954+0000 7ff42f59e700 1 -- 172.21.15.40:0/1427503086 >> 172.21.15.40:0/1427503086 conn(0x7ff430066090 msgr2=0x7ff4300664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:28.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.966+0000 7ff42f59e700 1 -- 172.21.15.40:0/1427503086 shutdown_connections 2024-09-13T18:20:28.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.966+0000 7ff42f59e700 1 -- 172.21.15.40:0/1427503086 wait complete. 2024-09-13T18:20:28.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.966+0000 7ff42f59e700 1 Processor -- start 2024-09-13T18:20:28.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.970+0000 7ff42f59e700 1 -- start start 2024-09-13T18:20:28.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.970+0000 7ff42f59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff418004870 0x7ff43014d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.970+0000 7ff42f59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff430148c10 0x7ff4301480c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.970+0000 7ff42f59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff430151eb0 0x7ff430152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.970+0000 7ff42f59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff430151a20 con 0x7ff430151eb0 2024-09-13T18:20:28.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.970+0000 7ff42f59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff430152870 con 0x7ff430148c10 2024-09-13T18:20:28.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.970+0000 7ff42f59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4301eeb20 con 0x7ff418004870 2024-09-13T18:20:28.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.970+0000 7ff42ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff430151eb0 0x7ff430152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.970+0000 7ff42e59c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff418004870 0x7ff43014d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.970+0000 7ff42e59c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff418004870 0x7ff43014d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:38250/0 (socket says 172.21.15.40:38250) 2024-09-13T18:20:28.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.970+0000 7ff42ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff430151eb0 0x7ff430152330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34606/0 (socket says 172.21.15.40:34606) 2024-09-13T18:20:28.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.970+0000 7ff42e59c700 1 -- 172.21.15.40:0/2071501947 learned_addr learned my addr 172.21.15.40:0/2071501947 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:28.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.970+0000 7ff42ed9d700 1 -- 172.21.15.40:0/2071501947 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff418004870 msgr2=0x7ff43014d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.970+0000 7ff42ed9d700 1 --2- 172.21.15.40:0/2071501947 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff418004870 0x7ff43014d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.970+0000 7ff42ed9d700 1 -- 172.21.15.40:0/2071501947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff430148c10 msgr2=0x7ff4301480c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:20:28.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.970+0000 7ff42ed9d700 1 --2- 172.21.15.40:0/2071501947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff430148c10 0x7ff4301480c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.970+0000 7ff42ed9d700 1 -- 172.21.15.40:0/2071501947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4301510f0 con 0x7ff430151eb0 2024-09-13T18:20:28.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.970+0000 7ff42ed9d700 1 --2- 172.21.15.40:0/2071501947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff430151eb0 0x7ff430152330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff430070370 tx=0x7ff42800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:28.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.974+0000 7ff41effd700 1 -- 172.21.15.40:0/2071501947 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42800cbd0 con 0x7ff430151eb0 2024-09-13T18:20:28.981 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.974+0000 7ff41effd700 1 -- 172.21.15.40:0/2071501947 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42800cd50 con 0x7ff430151eb0 2024-09-13T18:20:28.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.974+0000 7ff42f59e700 1 -- 172.21.15.40:0/2071501947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4301513d0 con 0x7ff430151eb0 2024-09-13T18:20:28.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.974+0000 7ff41effd700 1 -- 172.21.15.40:0/2071501947 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff428017680 con 0x7ff430151eb0 2024-09-13T18:20:28.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.974+0000 7ff42f59e700 1 -- 172.21.15.40:0/2071501947 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4301ee7e0 con 0x7ff430151eb0 2024-09-13T18:20:28.982 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.974+0000 7ff41effd700 1 -- 172.21.15.40:0/2071501947 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff42800b3a0 con 0x7ff430151eb0 2024-09-13T18:20:28.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.974+0000 7ff41effd700 1 --2- 172.21.15.40:0/2071501947 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff410066780 0x7ff410068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.974+0000 7ff41effd700 1 -- 172.21.15.40:0/2071501947 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff428084a20 con 0x7ff430151eb0 2024-09-13T18:20:28.983 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.974+0000 7ff42e59c700 1 --2- 172.21.15.40:0/2071501947 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff410066780 0x7ff410068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.984 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.974+0000 7ff42e59c700 1 --2- 172.21.15.40:0/2071501947 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff410066780 0x7ff410068c40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff420005140 tx=0x7ff420004b20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:28.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.978+0000 7ff42f59e700 1 --2- 172.21.15.40:0/2071501947 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4301466c0 0x7ff430146aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.978+0000 7ff42f59e700 1 -- 172.21.15.40:0/2071501947 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff430151280 con 0x7ff4301466c0 2024-09-13T18:20:28.985 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.978+0000 7ff42dd9b700 1 --2- 172.21.15.40:0/2071501947 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4301466c0 0x7ff430146aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.978+0000 7ff42dd9b700 1 -- 172.21.15.40:0/2071501947 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff43006e4e0 con 0x7ff4301466c0 2024-09-13T18:20:28.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.978+0000 7ff42dd9b700 1 --2- 172.21.15.40:0/2071501947 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4301466c0 0x7ff430146aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff42400abc0 tx=0x7ff4240047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:28.986 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.978+0000 7ff41effd700 1 -- 172.21.15.40:0/2071501947 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42400db60 con 0x7ff4301466c0 2024-09-13T18:20:28.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.978+0000 7ff41effd700 1 -- 172.21.15.40:0/2071501947 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7ff4240036c0 con 0x7ff4301466c0 2024-09-13T18:20:28.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.978+0000 7ff41effd700 1 -- 172.21.15.40:0/2071501947 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4301466c0 msgr2=0x7ff430146aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:28.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.978+0000 7ff41effd700 1 --2- 172.21.15.40:0/2071501947 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4301466c0 0x7ff430146aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff42400abc0 tx=0x7ff4240047e0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.978+0000 7ff41effd700 1 -- 172.21.15.40:0/2071501947 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4301466c0 msgr2=0x7ff430146aa0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:20:28.987 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.978+0000 7ff41effd700 1 --2- 172.21.15.40:0/2071501947 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4301466c0 0x7ff430146aa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:28.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff42f59e700 1 --2- 172.21.15.40:0/2071501947 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4301ea860 0x7ff4301eac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:28.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff42f59e700 1 -- 172.21.15.40:0/2071501947 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff430067720 con 0x7ff4301ea860 2024-09-13T18:20:28.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff42dd9b700 1 --2- 172.21.15.40:0/2071501947 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4301ea860 0x7ff4301eac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:28.999 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff42dd9b700 1 -- 172.21.15.40:0/2071501947 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7ff43014e000 con 0x7ff4301ea860 2024-09-13T18:20:29.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff42dd9b700 1 --2- 172.21.15.40:0/2071501947 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4301ea860 0x7ff4301eac70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff43006b5d0 tx=0x7ff42400a2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:29.000 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff41effd700 1 -- 172.21.15.40:0/2071501947 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff424005c50 con 0x7ff4301ea860 2024-09-13T18:20:29.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff41effd700 1 -- 172.21.15.40:0/2071501947 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4771 (secure 0 0 0) 0x7ff424005db0 con 0x7ff4301ea860 2024-09-13T18:20:29.001 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff41effd700 1 -- 172.21.15.40:0/2071501947 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4301ea860 msgr2=0x7ff4301eac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:29.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff41effd700 1 --2- 172.21.15.40:0/2071501947 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4301ea860 0x7ff4301eac70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff43006b5d0 tx=0x7ff42400a2a0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff41effd700 1 -- 172.21.15.40:0/2071501947 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4301ea860 msgr2=0x7ff4301eac70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:20:29.002 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff41effd700 1 --2- 172.21.15.40:0/2071501947 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff4301ea860 0x7ff4301eac70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.002 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:20:29.002 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:20:29.002 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:20:29.002 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:20:29.002 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 238, 2024-09-13T18:20:29.002 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:20:29.003 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:20:29.003 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:20:29.003 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:29.003 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 1, 2024-09-13T18:20:29.003 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:20:29.003 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:20:29.003 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:20:29.003 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:29.003 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:29.003 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:29.003 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:29.003 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:29.003 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:29.003 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:29.004 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:29.004 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:29.004 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:29.004 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:29.004 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:20:29.004 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:20:29.004 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:29.004 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:29.004 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:29.004 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:29.004 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:29.004 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:29.004 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:29.005 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:29.005 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:29.005 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:29.005 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:29.005 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:29.005 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:20:29.005 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:20:29.005 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:20:29.005 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:20:29.005 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:20:29.005 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:20:29.005 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:20:29.005 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:20:29.005 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:20:29.006 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:20:29.006 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:20:29.006 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:20:29.006 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:20:29.006 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:20:29.006 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:20:29.006 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:20:29.006 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:20:29.006 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:29.006 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:29.006 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:29.006 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:29.006 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:29.006 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:29.007 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:29.007 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:29.007 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:29.007 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:29.007 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:29.007 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:20:29.007 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:29.007 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:20:29.007 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:29.007 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:20:29.007 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:20:29.007 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:29.007 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:29.007 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:29.008 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:29.008 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:20:29.008 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:29.008 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:29.008 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:29.008 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:29.008 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:20:29.008 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:29.008 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:29.008 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:29.008 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:29.008 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:20:29.008 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:20:29.008 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:29.009 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:29.009 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:29.009 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:29.009 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:29.009 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:20:29.009 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:20:29.009 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:29.009 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:29.009 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:29.009 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:29.009 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:20:29.009 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:29.009 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:29.010 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:29.010 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:29.010 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:20:29.010 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:29.010 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:29.010 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:29.010 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:29.010 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:20:29.010 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:20:29.010 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:29.010 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:29.010 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:29.010 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:29.010 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:29.011 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:20:29.011 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:20:29.011 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:29.011 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:29.011 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:29.011 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:29.011 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:20:29.011 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:29.011 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:29.011 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:29.011 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:29.011 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:20:29.011 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:29.011 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:29.012 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:29.012 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:29.012 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:20:29.012 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:20:29.012 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:29.012 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:29.012 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:29.012 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:29.012 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:29.012 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:29.012 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:20:29.012 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:20:29.012 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:29.012 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:29.013 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:29.013 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:20:29.013 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:29.013 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:29.013 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:20:29.013 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:29.013 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:29.013 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:29.013 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:20:29.013 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:29.013 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:29.013 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:20:29.013 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:29.013 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:29.014 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:29.014 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:20:29.014 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:29.014 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:29.014 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:20:29.014 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:29.014 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:29.014 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:29.014 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:20:29.014 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:29.014 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:29.014 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:29.014 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:20:29.014 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:20:29.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff42f59e700 1 -- 172.21.15.40:0/2071501947 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff410066780 msgr2=0x7ff410068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:29.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff42f59e700 1 --2- 172.21.15.40:0/2071501947 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff410066780 0x7ff410068c40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff420005140 tx=0x7ff420004b20 comp rx=0 tx=0).stop 2024-09-13T18:20:29.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff42f59e700 1 -- 172.21.15.40:0/2071501947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff430151eb0 msgr2=0x7ff430152330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:29.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff42f59e700 1 --2- 172.21.15.40:0/2071501947 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff430151eb0 0x7ff430152330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff430070370 tx=0x7ff42800ecf0 comp rx=0 tx=0).stop 2024-09-13T18:20:29.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff42ed9d700 1 -- 172.21.15.40:0/2071501947 reap_dead start 2024-09-13T18:20:29.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff42ed9d700 1 -- 172.21.15.40:0/2071501947 reap_dead start 2024-09-13T18:20:29.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff42f59e700 1 -- 172.21.15.40:0/2071501947 shutdown_connections 2024-09-13T18:20:29.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff42f59e700 1 -- 172.21.15.40:0/2071501947 >> 172.21.15.40:0/2071501947 conn(0x7ff430066090 msgr2=0x7ff4300cb540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:29.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff42f59e700 1 -- 172.21.15.40:0/2071501947 shutdown_connections 2024-09-13T18:20:29.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:28.990+0000 7ff42f59e700 1 -- 172.21.15.40:0/2071501947 wait complete. 2024-09-13T18:20:29.017 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-13T18:20:29.017 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-13T18:20:49.018 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-13T18:20:49.018 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-13T18:20:49.018 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T18:20:49.019 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T18:20:49.022 INFO:tasks.ceph.mon.b:Started 2024-09-13T18:20:49.023 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:20:49.023 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:20:50.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.153+0000 7f4e2deb6700 1 Processor -- start 2024-09-13T18:20:50.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.153+0000 7f4e2deb6700 1 -- start start 2024-09-13T18:20:50.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.153+0000 7f4e2deb6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e28070e40 0x7f4e280c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:50.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.153+0000 7f4e2deb6700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4e2806e450 0x7f4e281574d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:50.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.153+0000 7f4e2deb6700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e2806d720 con 0x7f4e28071220 2024-09-13T18:20:50.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.153+0000 7f4e2deb6700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e28071a20 con 0x7f4e2806e450 2024-09-13T18:20:50.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.153+0000 7f4e2deb6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e280c7400 con 0x7f4e28070e40 2024-09-13T18:20:50.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.153+0000 7f4e2ceb4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e28070e40 0x7f4e280c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:50.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.153+0000 7f4e2ceb4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e28070e40 0x7f4e280c6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:36178/0 (socket says 172.21.15.40:36178) 2024-09-13T18:20:50.165 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.153+0000 7f4e2ceb4700 1 -- 172.21.15.40:0/2205027889 learned_addr learned my addr 172.21.15.40:0/2205027889 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:50.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.153+0000 7f4e2ceb4700 1 -- 172.21.15.40:0/2205027889 >> v2:172.21.15.43:3300/0 conn(0x7f4e2806e450 msgr2=0x7f4e281574d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:20:50.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.153+0000 7f4e2ceb4700 1 --2- 172.21.15.40:0/2205027889 >> v2:172.21.15.43:3300/0 conn(0x7f4e2806e450 0x7f4e281574d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:50.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.153+0000 7f4e2ceb4700 1 -- 172.21.15.40:0/2205027889 >> v1:172.21.15.43:6789/0 conn(0x7f4e28071220 legacy=0x7f4e2815d790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:20:50.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.153+0000 7f4e2ceb4700 1 -- 172.21.15.40:0/2205027889 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2815e290 con 0x7f4e28070e40 2024-09-13T18:20:50.166 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.157+0000 7f4e2ceb4700 1 --2- 172.21.15.40:0/2205027889 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e28070e40 0x7f4e280c6ec0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009e50 tx=0x7f4e2000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e16942ca059e65be server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:50.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.157+0000 7f4e277fe700 1 -- 172.21.15.40:0/2205027889 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2000e070 con 0x7f4e28070e40 2024-09-13T18:20:50.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.157+0000 7f4e277fe700 1 -- 172.21.15.40:0/2205027889 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e20003a40 con 0x7f4e28070e40 2024-09-13T18:20:50.167 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.157+0000 7f4e277fe700 1 -- 172.21.15.40:0/2205027889 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e20003df0 con 0x7f4e28070e40 2024-09-13T18:20:50.168 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.157+0000 7f4e2deb6700 1 -- 172.21.15.40:0/2205027889 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e28070e40 msgr2=0x7f4e280c6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:50.169 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.157+0000 7f4e2deb6700 1 --2- 172.21.15.40:0/2205027889 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e28070e40 0x7f4e280c6ec0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009e50 tx=0x7f4e2000b330 comp rx=0 tx=0).stop 2024-09-13T18:20:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.161+0000 7f4e2deb6700 1 -- 172.21.15.40:0/2205027889 shutdown_connections 2024-09-13T18:20:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.161+0000 7f4e2deb6700 1 --2- 172.21.15.40:0/2205027889 >> v2:172.21.15.43:3300/0 conn(0x7f4e2806e450 0x7f4e281574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.161+0000 7f4e2deb6700 1 --2- 172.21.15.40:0/2205027889 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e28070e40 0x7f4e280c6ec0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.161+0000 7f4e2deb6700 1 -- 172.21.15.40:0/2205027889 >> 172.21.15.40:0/2205027889 conn(0x7f4e280cb4c0 msgr2=0x7f4e280cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.161+0000 7f4e2deb6700 1 -- 172.21.15.40:0/2205027889 shutdown_connections 2024-09-13T18:20:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.161+0000 7f4e2deb6700 1 -- 172.21.15.40:0/2205027889 wait complete. 2024-09-13T18:20:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.161+0000 7f4e2deb6700 1 Processor -- start 2024-09-13T18:20:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2deb6700 1 -- start start 2024-09-13T18:20:50.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2deb6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e2806e450 0x7f4e2815d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:50.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2deb6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e28070e40 0x7f4e2815cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:50.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2deb6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e28071220 0x7f4e281e3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:50.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2deb6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2815b040 con 0x7f4e2806e450 2024-09-13T18:20:50.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2deb6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e28158cf0 con 0x7f4e28071220 2024-09-13T18:20:50.177 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2deb6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2815ad60 con 0x7f4e28070e40 2024-09-13T18:20:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2ceb4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e2806e450 0x7f4e2815d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e27fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e28070e40 0x7f4e2815cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2d6b5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e28071220 0x7f4e281e3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2ceb4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e2806e450 0x7f4e2815d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:52824/0 (socket says 172.21.15.40:52824) 2024-09-13T18:20:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e27fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e28070e40 0x7f4e2815cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:36188/0 (socket says 172.21.15.40:36188) 2024-09-13T18:20:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2ceb4700 1 -- 172.21.15.40:0/2588191143 learned_addr learned my addr 172.21.15.40:0/2588191143 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2d6b5700 1 -- 172.21.15.40:0/2588191143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e28071220 msgr2=0x7f4e281e3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-13T18:20:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2d6b5700 1 -- 172.21.15.40:0/2588191143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e28071220 msgr2=0x7f4e281e3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:20:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2d6b5700 1 --2- 172.21.15.40:0/2588191143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e28071220 0x7f4e281e3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:20:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2d6b5700 1 --2- 172.21.15.40:0/2588191143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e28071220 0x7f4e281e3850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:20:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2ceb4700 1 -- 172.21.15.40:0/2588191143 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e28070e40 msgr2=0x7f4e2815cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2ceb4700 1 --2- 172.21.15.40:0/2588191143 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e28070e40 0x7f4e2815cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2ceb4700 1 -- 172.21.15.40:0/2588191143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e28071220 msgr2=0x7f4e281e3850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:20:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2ceb4700 1 --2- 172.21.15.40:0/2588191143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e28071220 0x7f4e281e3850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2ceb4700 1 -- 172.21.15.40:0/2588191143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e20009b30 con 0x7f4e2806e450 2024-09-13T18:20:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.165+0000 7f4e2ceb4700 1 --2- 172.21.15.40:0/2588191143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e2806e450 0x7f4e2815d570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000b370 tx=0x7f4e20004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.169+0000 7f4e257fa700 1 -- 172.21.15.40:0/2588191143 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e20004440 con 0x7f4e2806e450 2024-09-13T18:20:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.169+0000 7f4e257fa700 1 -- 172.21.15.40:0/2588191143 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e200045c0 con 0x7f4e2806e450 2024-09-13T18:20:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.169+0000 7f4e2deb6700 1 -- 172.21.15.40:0/2588191143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e28157f30 con 0x7f4e2806e450 2024-09-13T18:20:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.169+0000 7f4e2deb6700 1 -- 172.21.15.40:0/2588191143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e281584f0 con 0x7f4e2806e450 2024-09-13T18:20:50.180 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.169+0000 7f4e257fa700 1 -- 172.21.15.40:0/2588191143 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e20004970 con 0x7f4e2806e450 2024-09-13T18:20:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.169+0000 7f4e257fa700 1 -- 172.21.15.40:0/2588191143 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4e20012430 con 0x7f4e2806e450 2024-09-13T18:20:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.169+0000 7f4e257fa700 1 --2- 172.21.15.40:0/2588191143 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4e14066730 0x7f4e14068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.169+0000 7f4e257fa700 1 -- 172.21.15.40:0/2588191143 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4e20085c60 con 0x7f4e2806e450 2024-09-13T18:20:50.183 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.169+0000 7f4e27fff700 1 --2- 172.21.15.40:0/2588191143 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4e14066730 0x7f4e14068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:50.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.177+0000 7f4e0effd700 1 -- 172.21.15.40:0/2588191143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e00005310 con 0x7f4e2806e450 2024-09-13T18:20:50.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.177+0000 7f4e27fff700 1 --2- 172.21.15.40:0/2588191143 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4e14066730 0x7f4e14068bf0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4e281567a0 tx=0x7f4e18009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:50.194 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.185+0000 7f4e257fa700 1 -- 172.21.15.40:0/2588191143 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4e20056690 con 0x7f4e2806e450 2024-09-13T18:20:50.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.437+0000 7f4e0effd700 1 -- 172.21.15.40:0/2588191143 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f4e00005b40 con 0x7f4e2806e450 2024-09-13T18:20:50.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.437+0000 7f4e257fa700 1 -- 172.21.15.40:0/2588191143 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f4e20017070 con 0x7f4e2806e450 2024-09-13T18:20:50.447 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":238,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:20:50.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.441+0000 7f4e0effd700 1 -- 172.21.15.40:0/2588191143 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4e14066730 msgr2=0x7f4e14068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:50.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.441+0000 7f4e0effd700 1 --2- 172.21.15.40:0/2588191143 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4e14066730 0x7f4e14068bf0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4e281567a0 tx=0x7f4e18009190 comp rx=0 tx=0).stop 2024-09-13T18:20:50.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.441+0000 7f4e0effd700 1 -- 172.21.15.40:0/2588191143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e2806e450 msgr2=0x7f4e2815d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:50.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.441+0000 7f4e0effd700 1 --2- 172.21.15.40:0/2588191143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e2806e450 0x7f4e2815d570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000b370 tx=0x7f4e20004af0 comp rx=0 tx=0).stop 2024-09-13T18:20:50.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.441+0000 7f4e0effd700 1 -- 172.21.15.40:0/2588191143 shutdown_connections 2024-09-13T18:20:50.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.441+0000 7f4e0effd700 1 --2- 172.21.15.40:0/2588191143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e28071220 0x7f4e281e3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:50.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.441+0000 7f4e0effd700 1 --2- 172.21.15.40:0/2588191143 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4e14066730 0x7f4e14068bf0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:50.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.441+0000 7f4e0effd700 1 --2- 172.21.15.40:0/2588191143 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4e28070e40 0x7f4e2815cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:50.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.441+0000 7f4e0effd700 1 --2- 172.21.15.40:0/2588191143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4e2806e450 0x7f4e2815d570 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:50.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.441+0000 7f4e0effd700 1 -- 172.21.15.40:0/2588191143 >> 172.21.15.40:0/2588191143 conn(0x7f4e280cb4c0 msgr2=0x7f4e280ca2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:50.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.441+0000 7f4e0effd700 1 -- 172.21.15.40:0/2588191143 shutdown_connections 2024-09-13T18:20:50.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:50.441+0000 7f4e0effd700 1 -- 172.21.15.40:0/2588191143 wait complete. 2024-09-13T18:20:53.462 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:20:53.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.573+0000 7f5330d18700 1 Processor -- start 2024-09-13T18:20:53.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.573+0000 7f5330d18700 1 -- start start 2024-09-13T18:20:53.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.573+0000 7f5330d18700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f532c071f70 0x7f532c0c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:53.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.573+0000 7f5330d18700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f532c0c7200 0x7f532c161ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:53.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.573+0000 7f5330d18700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532c0c55c0 con 0x7f532c072350 2024-09-13T18:20:53.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.573+0000 7f5330d18700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532c06a390 con 0x7f532c071f70 2024-09-13T18:20:53.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.573+0000 7f5330d18700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532c157710 con 0x7f532c0c7200 2024-09-13T18:20:53.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.573+0000 7f532b7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f532c071f70 0x7f532c0c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:53.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.573+0000 7f532affd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f532c072350 0x7f532c150d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:54910/0 (socket says 172.21.15.40:54910) 2024-09-13T18:20:53.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.573+0000 7f532bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f532c0c7200 0x7f532c161ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:53.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.573+0000 7f532affd700 1 -- 172.21.15.40:0/2259086840 learned_addr learned my addr 172.21.15.40:0/2259086840 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:20:53.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.577+0000 7f532b7fe700 1 -- 172.21.15.40:0/2259086840 >> v2:172.21.15.43:3300/0 conn(0x7f532c071f70 msgr2=0x7f532c0c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T18:20:53.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.577+0000 7f532b7fe700 1 -- 172.21.15.40:0/2259086840 >> v2:172.21.15.43:3300/0 conn(0x7f532c071f70 msgr2=0x7f532c0c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:20:53.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.577+0000 7f532b7fe700 1 --2- 172.21.15.40:0/2259086840 >> v2:172.21.15.43:3300/0 conn(0x7f532c071f70 0x7f532c0c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:20:53.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.577+0000 7f532b7fe700 1 --2- 172.21.15.40:0/2259086840 >> v2:172.21.15.43:3300/0 conn(0x7f532c071f70 0x7f532c0c5d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:20:53.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.577+0000 7f532bfff700 1 -- 172.21.15.40:0/2259086840 >> v2:172.21.15.43:3300/0 conn(0x7f532c071f70 msgr2=0x7f532c0c5d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:20:53.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.577+0000 7f532bfff700 1 --2- 172.21.15.40:0/2259086840 >> v2:172.21.15.43:3300/0 conn(0x7f532c071f70 0x7f532c0c5d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:53.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.577+0000 7f532bfff700 1 -- 172.21.15.40:0/2259086840 >> v1:172.21.15.43:6789/0 conn(0x7f532c072350 legacy=0x7f532c150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:53.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.577+0000 7f532bfff700 1 -- 172.21.15.40:0/2259086840 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c1579d0 con 0x7f532c0c7200 2024-09-13T18:20:53.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.577+0000 7f532bfff700 1 --2- 172.21.15.40:0/2259086840 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f532c0c7200 0x7f532c161ae0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5324009d90 tx=0x7f532400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7316f4ae83d9c43d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:53.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.577+0000 7f532a7fc700 1 -- 172.21.15.40:0/2259086840 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532400e070 con 0x7f532c0c7200 2024-09-13T18:20:53.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.577+0000 7f532a7fc700 1 -- 172.21.15.40:0/2259086840 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532400bbd0 con 0x7f532c0c7200 2024-09-13T18:20:53.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.577+0000 7f532a7fc700 1 -- 172.21.15.40:0/2259086840 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5324012430 con 0x7f532c0c7200 2024-09-13T18:20:53.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.577+0000 7f5330d18700 1 -- 172.21.15.40:0/2259086840 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f532c0c7200 msgr2=0x7f532c161ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:53.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.577+0000 7f5330d18700 1 --2- 172.21.15.40:0/2259086840 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f532c0c7200 0x7f532c161ae0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5324009d90 tx=0x7f532400b330 comp rx=0 tx=0).stop 2024-09-13T18:20:53.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.577+0000 7f5330d18700 1 -- 172.21.15.40:0/2259086840 shutdown_connections 2024-09-13T18:20:53.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.577+0000 7f5330d18700 1 --2- 172.21.15.40:0/2259086840 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f532c0c7200 0x7f532c161ae0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:53.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.577+0000 7f5330d18700 1 --2- 172.21.15.40:0/2259086840 >> v2:172.21.15.43:3300/0 conn(0x7f532c071f70 0x7f532c0c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:53.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.577+0000 7f5330d18700 1 -- 172.21.15.40:0/2259086840 >> 172.21.15.40:0/2259086840 conn(0x7f532c0cabc0 msgr2=0x7f532c0cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:53.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.577+0000 7f5330d18700 1 -- 172.21.15.40:0/2259086840 shutdown_connections 2024-09-13T18:20:53.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.577+0000 7f5330d18700 1 -- 172.21.15.40:0/2259086840 wait complete. 2024-09-13T18:20:53.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.581+0000 7f5330d18700 1 Processor -- start 2024-09-13T18:20:53.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.581+0000 7f5330d18700 1 -- start start 2024-09-13T18:20:53.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.581+0000 7f5330d18700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f532c071f70 0x7f532c1ee820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:53.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.581+0000 7f5330d18700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f532c072350 0x7f532c1eed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:53.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.581+0000 7f5330d18700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f532c0c7200 0x7f532c1f02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:53.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.581+0000 7f5330d18700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c1ee400 con 0x7f532c0c7200 2024-09-13T18:20:53.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.581+0000 7f5330d18700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c158970 con 0x7f532c072350 2024-09-13T18:20:53.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.581+0000 7f5330d18700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532c14a3b0 con 0x7f532c071f70 2024-09-13T18:20:53.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.581+0000 7f532bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f532c0c7200 0x7f532c1f02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:53.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.581+0000 7f532b7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f532c071f70 0x7f532c1ee820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:53.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.581+0000 7f532bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f532c0c7200 0x7f532c1f02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58100/0 (socket says 172.21.15.40:58100) 2024-09-13T18:20:53.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.581+0000 7f532bfff700 1 -- 172.21.15.40:0/200894601 learned_addr learned my addr 172.21.15.40:0/200894601 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:53.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.581+0000 7f532affd700 1 --2- 172.21.15.40:0/200894601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f532c072350 0x7f532c1eed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:53.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.581+0000 7f532affd700 1 -- 172.21.15.40:0/200894601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f532c072350 msgr2=0x7f532c1eed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-13T18:20:53.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.581+0000 7f532affd700 1 -- 172.21.15.40:0/200894601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f532c072350 msgr2=0x7f532c1eed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-13T18:20:53.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.581+0000 7f532affd700 1 --2- 172.21.15.40:0/200894601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f532c072350 0x7f532c1eed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:20:53.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.581+0000 7f532affd700 1 --2- 172.21.15.40:0/200894601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f532c072350 0x7f532c1eed60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:20:53.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.581+0000 7f532bfff700 1 -- 172.21.15.40:0/200894601 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f532c071f70 msgr2=0x7f532c1ee820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:53.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.585+0000 7f532bfff700 1 --2- 172.21.15.40:0/200894601 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f532c071f70 0x7f532c1ee820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:53.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.585+0000 7f532bfff700 1 -- 172.21.15.40:0/200894601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f532c072350 msgr2=0x7f532c1eed60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:20:53.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.585+0000 7f532bfff700 1 --2- 172.21.15.40:0/200894601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f532c072350 0x7f532c1eed60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:53.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.585+0000 7f532bfff700 1 -- 172.21.15.40:0/200894601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5324009a70 con 0x7f532c0c7200 2024-09-13T18:20:53.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.585+0000 7f532bfff700 1 --2- 172.21.15.40:0/200894601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f532c0c7200 0x7f532c1f02f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5324014040 tx=0x7f5324004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:53.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.585+0000 7f5313fff700 1 -- 172.21.15.40:0/200894601 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5324012430 con 0x7f532c0c7200 2024-09-13T18:20:53.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.585+0000 7f5313fff700 1 -- 172.21.15.40:0/200894601 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5324020070 con 0x7f532c0c7200 2024-09-13T18:20:53.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.585+0000 7f5330d18700 1 -- 172.21.15.40:0/200894601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532c14a6f0 con 0x7f532c0c7200 2024-09-13T18:20:53.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.585+0000 7f5313fff700 1 -- 172.21.15.40:0/200894601 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5324004280 con 0x7f532c0c7200 2024-09-13T18:20:53.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.585+0000 7f5330d18700 1 -- 172.21.15.40:0/200894601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532c14a990 con 0x7f532c0c7200 2024-09-13T18:20:53.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.585+0000 7f5330d18700 1 -- 172.21.15.40:0/200894601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532c1ee540 con 0x7f532c0c7200 2024-09-13T18:20:53.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.585+0000 7f5313fff700 1 -- 172.21.15.40:0/200894601 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f53240043e0 con 0x7f532c0c7200 2024-09-13T18:20:53.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.589+0000 7f5313fff700 1 --2- 172.21.15.40:0/200894601 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5318066730 0x7f5318068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:53.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.589+0000 7f5313fff700 1 -- 172.21.15.40:0/200894601 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5324022070 con 0x7f532c0c7200 2024-09-13T18:20:53.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.589+0000 7f532b7fe700 1 --2- 172.21.15.40:0/200894601 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5318066730 0x7f5318068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:53.598 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.589+0000 7f532b7fe700 1 --2- 172.21.15.40:0/200894601 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5318066730 0x7f5318068bf0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5320009100 tx=0x7f5320008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:53.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.593+0000 7f5313fff700 1 -- 172.21.15.40:0/200894601 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f532c1ee540 con 0x7f532c0c7200 2024-09-13T18:20:53.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.853+0000 7f5330d18700 1 -- 172.21.15.40:0/200894601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f532c1ee540 con 0x7f532c0c7200 2024-09-13T18:20:53.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.857+0000 7f5313fff700 1 -- 172.21.15.40:0/200894601 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1535 (secure 0 0 0) 0x7f532c1ee540 con 0x7f532c0c7200 2024-09-13T18:20:53.869 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":238,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":26,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:20:53.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.865+0000 7f5330d18700 1 -- 172.21.15.40:0/200894601 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5318066730 msgr2=0x7f5318068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:53.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.865+0000 7f5330d18700 1 --2- 172.21.15.40:0/200894601 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5318066730 0x7f5318068bf0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5320009100 tx=0x7f5320008040 comp rx=0 tx=0).stop 2024-09-13T18:20:53.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.865+0000 7f5330d18700 1 -- 172.21.15.40:0/200894601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f532c0c7200 msgr2=0x7f532c1f02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:53.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.865+0000 7f5330d18700 1 --2- 172.21.15.40:0/200894601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f532c0c7200 0x7f532c1f02f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5324014040 tx=0x7f5324004af0 comp rx=0 tx=0).stop 2024-09-13T18:20:53.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.865+0000 7f5330d18700 1 -- 172.21.15.40:0/200894601 shutdown_connections 2024-09-13T18:20:53.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.865+0000 7f5330d18700 1 --2- 172.21.15.40:0/200894601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f532c0c7200 0x7f532c1f02f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:53.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.865+0000 7f5330d18700 1 --2- 172.21.15.40:0/200894601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f532c072350 0x7f532c1eed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:53.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.865+0000 7f5330d18700 1 --2- 172.21.15.40:0/200894601 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5318066730 0x7f5318068bf0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:53.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.865+0000 7f5330d18700 1 --2- 172.21.15.40:0/200894601 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f532c071f70 0x7f532c1ee820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:53.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.865+0000 7f5330d18700 1 -- 172.21.15.40:0/200894601 >> 172.21.15.40:0/200894601 conn(0x7f532c0cabc0 msgr2=0x7f532c0c76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:53.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.865+0000 7f5330d18700 1 -- 172.21.15.40:0/200894601 shutdown_connections 2024-09-13T18:20:53.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:53.865+0000 7f5330d18700 1 -- 172.21.15.40:0/200894601 wait complete. 2024-09-13T18:20:56.902 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:20:57.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.013+0000 7f0ac359e700 1 Processor -- start 2024-09-13T18:20:57.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.013+0000 7f0ac359e700 1 -- start start 2024-09-13T18:20:57.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.013+0000 7f0ac359e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0ac4151470 0x7f0ac40c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.013+0000 7f0ac359e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac40c4520 con 0x7f0ac40b1bd0 2024-09-13T18:20:57.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.013+0000 7f0ac359e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac40c4dc0 con 0x7f0ac4071bf0 2024-09-13T18:20:57.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.013+0000 7f0ac359e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac4157cf0 con 0x7f0ac4151470 2024-09-13T18:20:57.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.013+0000 7f0ac259c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0ac4151470 0x7f0ac40c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.013+0000 7f0ac259c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0ac4151470 0x7f0ac40c5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58102/0 (socket says 172.21.15.40:58102) 2024-09-13T18:20:57.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.013+0000 7f0ac259c700 1 -- 172.21.15.40:0/3808541590 learned_addr learned my addr 172.21.15.40:0/3808541590 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:57.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.013+0000 7f0ac259c700 1 -- 172.21.15.40:0/3808541590 >> v1:172.21.15.43:6789/0 conn(0x7f0ac4071bf0 legacy=0x7f0ac4161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.013+0000 7f0ac259c700 1 -- 172.21.15.40:0/3808541590 >> v1:172.21.15.40:6789/0 conn(0x7f0ac40b1bd0 legacy=0x7f0ac41571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.013+0000 7f0ac259c700 1 -- 172.21.15.40:0/3808541590 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac4157fd0 con 0x7f0ac4151470 2024-09-13T18:20:57.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.013+0000 7f0ac259c700 1 --2- 172.21.15.40:0/3808541590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac4151470 0x7f0ac40c5250 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009910 tx=0x7f0ab4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4f99a0c0a34d66 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:57.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.017+0000 7f0ac159a700 1 -- 172.21.15.40:0/3808541590 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400ba60 con 0x7f0ac4151470 2024-09-13T18:20:57.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.017+0000 7f0ac159a700 1 -- 172.21.15.40:0/3808541590 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab400bbc0 con 0x7f0ac4151470 2024-09-13T18:20:57.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.017+0000 7f0ac159a700 1 -- 172.21.15.40:0/3808541590 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4009e60 con 0x7f0ac4151470 2024-09-13T18:20:57.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.017+0000 7f0ac359e700 1 -- 172.21.15.40:0/3808541590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac4151470 msgr2=0x7f0ac40c5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.017+0000 7f0ac359e700 1 --2- 172.21.15.40:0/3808541590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac4151470 0x7f0ac40c5250 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009910 tx=0x7f0ab4009c20 comp rx=0 tx=0).stop 2024-09-13T18:20:57.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.017+0000 7f0ac359e700 1 -- 172.21.15.40:0/3808541590 shutdown_connections 2024-09-13T18:20:57.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.017+0000 7f0ac359e700 1 --2- 172.21.15.40:0/3808541590 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac4151470 0x7f0ac40c5250 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.017+0000 7f0ac359e700 1 -- 172.21.15.40:0/3808541590 >> 172.21.15.40:0/3808541590 conn(0x7f0ac406e4f0 msgr2=0x7f0ac406e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:57.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.017+0000 7f0ac359e700 1 -- 172.21.15.40:0/3808541590 shutdown_connections 2024-09-13T18:20:57.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.017+0000 7f0ac359e700 1 -- 172.21.15.40:0/3808541590 wait complete. 2024-09-13T18:20:57.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.021+0000 7f0ac359e700 1 Processor -- start 2024-09-13T18:20:57.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.021+0000 7f0ac359e700 1 -- start start 2024-09-13T18:20:57.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.021+0000 7f0ac359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac4071bf0 0x7f0ac40c11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.021+0000 7f0ac359e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ac40b1bd0 0x7f0ac40c1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.021+0000 7f0ac359e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ac4151470 0x7f0ac41efe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.021+0000 7f0ac359e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac414b370 con 0x7f0ac4071bf0 2024-09-13T18:20:57.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.021+0000 7f0ac359e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac41f2840 con 0x7f0ac40b1bd0 2024-09-13T18:20:57.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.021+0000 7f0ac359e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac406af10 con 0x7f0ac4151470 2024-09-13T18:20:57.034 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.021+0000 7f0ac259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac4071bf0 0x7f0ac40c11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.021+0000 7f0ac259c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac4071bf0 0x7f0ac40c11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58110/0 (socket says 172.21.15.40:58110) 2024-09-13T18:20:57.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.021+0000 7f0ac259c700 1 -- 172.21.15.40:0/2395408459 learned_addr learned my addr 172.21.15.40:0/2395408459 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:57.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.021+0000 7f0ac2d9d700 1 --2- 172.21.15.40:0/2395408459 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ac4151470 0x7f0ac41efe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.021+0000 7f0ac1d9b700 1 --2- 172.21.15.40:0/2395408459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ac40b1bd0 0x7f0ac40c1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.021+0000 7f0ac259c700 1 -- 172.21.15.40:0/2395408459 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ac4151470 msgr2=0x7f0ac41efe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.021+0000 7f0ac259c700 1 --2- 172.21.15.40:0/2395408459 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ac4151470 0x7f0ac41efe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.021+0000 7f0ac259c700 1 -- 172.21.15.40:0/2395408459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ac40b1bd0 msgr2=0x7f0ac40c1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.021+0000 7f0ac259c700 1 --2- 172.21.15.40:0/2395408459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ac40b1bd0 0x7f0ac40c1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.021+0000 7f0ac259c700 1 -- 172.21.15.40:0/2395408459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab4009570 con 0x7f0ac4071bf0 2024-09-13T18:20:57.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.021+0000 7f0ac1d9b700 1 --2- 172.21.15.40:0/2395408459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ac40b1bd0 0x7f0ac40c1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:20:57.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.025+0000 7f0ac259c700 1 --2- 172.21.15.40:0/2395408459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac4071bf0 0x7f0ac40c11b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4000c00 tx=0x7f0ab4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:57.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.025+0000 7f0ab2ffd700 1 -- 172.21.15.40:0/2395408459 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40036c0 con 0x7f0ac4071bf0 2024-09-13T18:20:57.035 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.025+0000 7f0ab2ffd700 1 -- 172.21.15.40:0/2395408459 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab401f070 con 0x7f0ac4071bf0 2024-09-13T18:20:57.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.025+0000 7f0ab2ffd700 1 -- 172.21.15.40:0/2395408459 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4003a80 con 0x7f0ac4071bf0 2024-09-13T18:20:57.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.025+0000 7f0ac359e700 1 -- 172.21.15.40:0/2395408459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac40c2620 con 0x7f0ac4071bf0 2024-09-13T18:20:57.036 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.025+0000 7f0ac359e700 1 -- 172.21.15.40:0/2395408459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac41ef600 con 0x7f0ac4071bf0 2024-09-13T18:20:57.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.025+0000 7f0ac359e700 1 -- 172.21.15.40:0/2395408459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac414b4f0 con 0x7f0ac4071bf0 2024-09-13T18:20:57.037 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.025+0000 7f0ab2ffd700 1 -- 172.21.15.40:0/2395408459 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0ab4003be0 con 0x7f0ac4071bf0 2024-09-13T18:20:57.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.025+0000 7f0ab2ffd700 1 --2- 172.21.15.40:0/2395408459 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0aa40667b0 0x7f0aa4068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.025+0000 7f0ac1d9b700 1 --2- 172.21.15.40:0/2395408459 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0aa40667b0 0x7f0aa4068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.025+0000 7f0ab2ffd700 1 -- 172.21.15.40:0/2395408459 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0ab4021070 con 0x7f0ac4071bf0 2024-09-13T18:20:57.038 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.029+0000 7f0ac1d9b700 1 --2- 172.21.15.40:0/2395408459 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0aa40667b0 0x7f0aa4068c70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40c7f20 tx=0x7f0ab8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:57.046 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.037+0000 7f0ab2ffd700 1 -- 172.21.15.40:0/2395408459 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0ab4052be0 con 0x7f0ac4071bf0 2024-09-13T18:20:57.283 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.273+0000 7f0ac359e700 1 -- 172.21.15.40:0/2395408459 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7f0ac4072020 con 0x7f0ac4071bf0 2024-09-13T18:20:57.284 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.273+0000 7f0ab2ffd700 1 -- 172.21.15.40:0/2395408459 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1540 (secure 0 0 0) 0x7f0ab4015080 con 0x7f0ac4071bf0 2024-09-13T18:20:57.284 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":240,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:20:57.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.277+0000 7f0ac359e700 1 -- 172.21.15.40:0/2395408459 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0aa40667b0 msgr2=0x7f0aa4068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.277+0000 7f0ac359e700 1 --2- 172.21.15.40:0/2395408459 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0aa40667b0 0x7f0aa4068c70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40c7f20 tx=0x7f0ab8009040 comp rx=0 tx=0).stop 2024-09-13T18:20:57.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.277+0000 7f0ac359e700 1 -- 172.21.15.40:0/2395408459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac4071bf0 msgr2=0x7f0ac40c11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.277+0000 7f0ac359e700 1 --2- 172.21.15.40:0/2395408459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac4071bf0 0x7f0ac40c11b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4000c00 tx=0x7f0ab4004770 comp rx=0 tx=0).stop 2024-09-13T18:20:57.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.277+0000 7f0ac359e700 1 -- 172.21.15.40:0/2395408459 shutdown_connections 2024-09-13T18:20:57.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.281+0000 7f0ac359e700 1 --2- 172.21.15.40:0/2395408459 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f0aa40667b0 0x7f0aa4068c70 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.281+0000 7f0ac359e700 1 --2- 172.21.15.40:0/2395408459 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ac4151470 0x7f0ac41efe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.281+0000 7f0ac359e700 1 --2- 172.21.15.40:0/2395408459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ac40b1bd0 0x7f0ac40c1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.281+0000 7f0ac359e700 1 --2- 172.21.15.40:0/2395408459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac4071bf0 0x7f0ac40c11b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.281+0000 7f0ac359e700 1 -- 172.21.15.40:0/2395408459 >> 172.21.15.40:0/2395408459 conn(0x7f0ac406e4f0 msgr2=0x7f0ac4072230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:57.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.281+0000 7f0ac359e700 1 -- 172.21.15.40:0/2395408459 shutdown_connections 2024-09-13T18:20:57.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.281+0000 7f0ac359e700 1 -- 172.21.15.40:0/2395408459 wait complete. 2024-09-13T18:20:57.305 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:20:57.305 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-13T18:20:57.305 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-13T18:20:57.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.417+0000 7fe024f1c700 1 Processor -- start 2024-09-13T18:20:57.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.417+0000 7fe024f1c700 1 -- start start 2024-09-13T18:20:57.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.417+0000 7fe024f1c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe018046740 0x7fe018046bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.430 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.417+0000 7fe024f1c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0180470f0 con 0x7fe0180daa00 2024-09-13T18:20:57.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.417+0000 7fe024f1c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0180daeb0 con 0x7fe0180d4f30 2024-09-13T18:20:57.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.417+0000 7fe024f1c700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0180dba30 con 0x7fe018046740 2024-09-13T18:20:57.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.417+0000 7fe01effd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe0180daa00 0x7fe01804c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49134/0 (socket says 172.21.15.40:49134) 2024-09-13T18:20:57.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.417+0000 7fe01effd700 1 -- 172.21.15.40:0/2327348807 learned_addr learned my addr 172.21.15.40:0/2327348807 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:20:57.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.417+0000 7fe01ffff700 1 --2- 172.21.15.40:0/2327348807 >> v2:172.21.15.43:3300/0 conn(0x7fe018046740 0x7fe018046bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.417+0000 7fe01e7fc700 1 -- 172.21.15.40:0/2327348807 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647533767 0 0) 0x7fe0180470f0 con 0x7fe0180daa00 2024-09-13T18:20:57.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.417+0000 7fe01e7fc700 1 -- 172.21.15.40:0/2327348807 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0100035d0 con 0x7fe0180daa00 2024-09-13T18:20:57.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.417+0000 7fe01e7fc700 1 -- 172.21.15.40:0/2327348807 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587230967 0 0) 0x7fe0180daeb0 con 0x7fe0180d4f30 2024-09-13T18:20:57.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.417+0000 7fe01e7fc700 1 -- 172.21.15.40:0/2327348807 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0180470f0 con 0x7fe0180d4f30 2024-09-13T18:20:57.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.417+0000 7fe01ffff700 1 -- 172.21.15.40:0/2327348807 >> v1:172.21.15.43:6789/0 conn(0x7fe0180d4f30 legacy=0x7fe01804b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.417+0000 7fe01ffff700 1 -- 172.21.15.40:0/2327348807 >> v1:172.21.15.40:6789/0 conn(0x7fe0180daa00 legacy=0x7fe01804c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.417+0000 7fe01ffff700 1 -- 172.21.15.40:0/2327348807 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0180db190 con 0x7fe018046740 2024-09-13T18:20:57.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.417+0000 7fe01ffff700 1 --2- 172.21.15.40:0/2327348807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe018046740 0x7fe018046bb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe020069620 tx=0x7fe020069e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=908eb5b0503c3658 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:57.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.421+0000 7fe01e7fc700 1 -- 172.21.15.40:0/2327348807 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0200689a0 con 0x7fe018046740 2024-09-13T18:20:57.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.421+0000 7fe01e7fc700 1 -- 172.21.15.40:0/2327348807 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe020067dd0 con 0x7fe018046740 2024-09-13T18:20:57.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.421+0000 7fe01e7fc700 1 -- 172.21.15.40:0/2327348807 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0200744e0 con 0x7fe018046740 2024-09-13T18:20:57.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.421+0000 7fe024f1c700 1 -- 172.21.15.40:0/2327348807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe018046740 msgr2=0x7fe018046bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.421+0000 7fe024f1c700 1 --2- 172.21.15.40:0/2327348807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe018046740 0x7fe018046bb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe020069620 tx=0x7fe020069e70 comp rx=0 tx=0).stop 2024-09-13T18:20:57.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.421+0000 7fe024f1c700 1 -- 172.21.15.40:0/2327348807 shutdown_connections 2024-09-13T18:20:57.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.421+0000 7fe024f1c700 1 --2- 172.21.15.40:0/2327348807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe018046740 0x7fe018046bb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.421+0000 7fe024f1c700 1 -- 172.21.15.40:0/2327348807 >> 172.21.15.40:0/2327348807 conn(0x7fe018050590 msgr2=0x7fe0180509a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:57.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.421+0000 7fe024f1c700 1 -- 172.21.15.40:0/2327348807 shutdown_connections 2024-09-13T18:20:57.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.421+0000 7fe024f1c700 1 -- 172.21.15.40:0/2327348807 wait complete. 2024-09-13T18:20:57.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe024f1c700 1 Processor -- start 2024-09-13T18:20:57.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe024f1c700 1 -- start start 2024-09-13T18:20:57.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe024f1c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe018046740 0x7fe018177b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe024f1c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe0180d4f30 0x7fe018173a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe024f1c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0180daa00 0x7fe0180c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe024f1c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe018035570 con 0x7fe018046740 2024-09-13T18:20:57.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe024f1c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe018173f50 con 0x7fe0180daa00 2024-09-13T18:20:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe024f1c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0180cb4b0 con 0x7fe0180d4f30 2024-09-13T18:20:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe01f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe018046740 0x7fe018177b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe01f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe018046740 0x7fe018177b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58122/0 (socket says 172.21.15.40:58122) 2024-09-13T18:20:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe01f7fe700 1 -- 172.21.15.40:0/4094489480 learned_addr learned my addr 172.21.15.40:0/4094489480 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:57.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe01ffff700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0180daa00 0x7fe0180c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe01f7fe700 1 -- 172.21.15.40:0/4094489480 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe0180d4f30 msgr2=0x7fe018173a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe01effd700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe0180d4f30 0x7fe018173a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe01f7fe700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe0180d4f30 0x7fe018173a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe01f7fe700 1 -- 172.21.15.40:0/4094489480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0180daa00 msgr2=0x7fe0180c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe01f7fe700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0180daa00 0x7fe0180c5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe01f7fe700 1 -- 172.21.15.40:0/4094489480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02004f5c0 con 0x7fe018046740 2024-09-13T18:20:57.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe01ffff700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0180daa00 0x7fe0180c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:20:57.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe01effd700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe0180d4f30 0x7fe018173a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:20:57.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe01f7fe700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe018046740 0x7fe018177b00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe01400c730 tx=0x7fe01400caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:57.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe00bfff700 1 -- 172.21.15.40:0/4094489480 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0140052d0 con 0x7fe018046740 2024-09-13T18:20:57.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe00bfff700 1 -- 172.21.15.40:0/4094489480 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe014005430 con 0x7fe018046740 2024-09-13T18:20:57.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe024f1c700 1 -- 172.21.15.40:0/4094489480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0180c7740 con 0x7fe018046740 2024-09-13T18:20:57.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe00bfff700 1 -- 172.21.15.40:0/4094489480 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0140055a0 con 0x7fe018046740 2024-09-13T18:20:57.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe024f1c700 1 -- 172.21.15.40:0/4094489480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe018177020 con 0x7fe018046740 2024-09-13T18:20:57.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe024f1c700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0181771b0 0x7fe0180d0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe024f1c700 1 -- 172.21.15.40:0/4094489480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe018008d40 con 0x7fe0181771b0 2024-09-13T18:20:57.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.425+0000 7fe01effd700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0181771b0 0x7fe0180d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.429+0000 7fe01effd700 1 -- 172.21.15.40:0/4094489480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe018046220 con 0x7fe0181771b0 2024-09-13T18:20:57.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.429+0000 7fe01effd700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0181771b0 0x7fe0180d0690 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe0180d26c0 tx=0x7fe00c00da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:57.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.429+0000 7fe00bfff700 1 -- 172.21.15.40:0/4094489480 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c00b070 con 0x7fe0181771b0 2024-09-13T18:20:57.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.429+0000 7fe00bfff700 1 -- 172.21.15.40:0/4094489480 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7fe00c005d90 con 0x7fe0181771b0 2024-09-13T18:20:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.429+0000 7fe00bfff700 1 -- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0181771b0 msgr2=0x7fe0180d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.429+0000 7fe00bfff700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0181771b0 0x7fe0180d0690 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe0180d26c0 tx=0x7fe00c00da80 comp rx=0 tx=0).stop 2024-09-13T18:20:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.429+0000 7fe00bfff700 1 -- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0181771b0 msgr2=0x7fe0180d0690 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:20:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.429+0000 7fe00bfff700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0181771b0 0x7fe0180d0690 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.429+0000 7fe00bfff700 1 -- 172.21.15.40:0/4094489480 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe0140038e0 con 0x7fe018046740 2024-09-13T18:20:57.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.429+0000 7fe00bfff700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe0040663e0 0x7fe0040688a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.429+0000 7fe00bfff700 1 -- 172.21.15.40:0/4094489480 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe014083b30 con 0x7fe018046740 2024-09-13T18:20:57.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.429+0000 7fe01effd700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe0040663e0 0x7fe0040688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.429+0000 7fe01effd700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe0040663e0 0x7fe0040688a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe00c007fd0 tx=0x7fe00c007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:57.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe024f1c700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0180d0bd0 0x7fe01804f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe024f1c700 1 -- 172.21.15.40:0/4094489480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0180db750 con 0x7fe0180d0bd0 2024-09-13T18:20:57.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe01ffff700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0180d0bd0 0x7fe01804f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe01ffff700 1 -- 172.21.15.40:0/4094489480 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7fe01804ba20 con 0x7fe0180d0bd0 2024-09-13T18:20:57.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe01ffff700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0180d0bd0 0x7fe01804f4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe02004f940 tx=0x7fe020068470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:57.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe00bfff700 1 -- 172.21.15.40:0/4094489480 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02009b070 con 0x7fe0180d0bd0 2024-09-13T18:20:57.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe00bfff700 1 -- 172.21.15.40:0/4094489480 <== mon.0 v2:172.21.15.40:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4950 (secure 0 0 0) 0x7fe020067730 con 0x7fe0180d0bd0 2024-09-13T18:20:57.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe00bfff700 1 -- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0180d0bd0 msgr2=0x7fe01804f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe00bfff700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0180d0bd0 0x7fe01804f4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe02004f940 tx=0x7fe020068470 comp rx=0 tx=0).stop 2024-09-13T18:20:57.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe00bfff700 1 -- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0180d0bd0 msgr2=0x7fe01804f4e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:20:57.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe00bfff700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0180d0bd0 0x7fe01804f4e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.455 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:20:57.455 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:20:57.456 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:20:57.456 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "leader", 2024-09-13T18:20:57.456 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 240, 2024-09-13T18:20:57.456 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:20:57.456 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:20:57.456 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:20:57.456 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:20:57.457 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:57.457 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 3, 2024-09-13T18:20:57.457 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:20:57.457 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:20:57.457 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:20:57.457 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:57.457 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:57.457 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:57.457 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:57.457 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:57.457 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:57.457 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:57.458 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:57.458 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:57.458 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:57.458 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:57.458 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:20:57.458 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:20:57.458 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:57.458 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:57.458 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:57.458 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:57.458 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:57.458 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:57.459 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:57.459 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:57.459 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:57.459 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:57.459 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:57.459 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.459 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:20:57.459 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:20:57.459 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:20:57.459 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:20:57.459 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:20:57.459 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:20:57.459 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:20:57.459 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:20:57.460 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:20:57.460 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:20:57.460 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:20:57.460 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:20:57.460 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:20:57.460 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:20:57.460 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:20:57.460 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:20:57.460 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:20:57.460 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:57.460 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:57.460 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:57.460 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:57.460 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:57.461 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:57.461 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:57.461 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:57.461 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:57.461 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:57.461 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:57.461 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:20:57.461 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.461 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:20:57.461 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.461 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:20:57.461 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:20:57.461 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:57.462 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:57.462 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.462 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:57.462 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:20:57.462 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:57.462 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.462 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.462 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:57.462 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:20:57.462 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:57.462 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.462 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:57.462 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.462 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:20:57.463 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:20:57.463 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:57.463 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:57.463 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:57.463 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.463 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.463 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:20:57.463 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:20:57.463 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:57.463 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:57.463 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.463 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:57.463 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:20:57.464 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:57.464 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.464 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.464 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:57.464 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:20:57.464 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:57.464 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.464 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:57.464 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.464 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:20:57.464 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:20:57.465 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:57.465 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:57.465 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:57.465 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.465 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.465 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:20:57.465 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:20:57.465 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:57.465 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:57.465 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.465 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:57.465 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:20:57.465 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:57.466 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.466 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.466 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:57.466 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:20:57.466 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:57.466 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.466 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:57.466 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.466 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:20:57.466 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:20:57.466 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:57.466 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:57.467 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:57.467 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.467 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:57.467 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.467 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:20:57.467 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:20:57.467 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.467 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:57.467 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:57.467 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:20:57.467 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.467 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:57.468 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:20:57.468 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.468 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:57.468 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:57.468 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:20:57.468 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.469 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:57.469 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:20:57.469 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.469 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:57.469 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:57.469 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:20:57.469 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.470 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:57.470 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:20:57.470 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.470 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:57.470 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:57.470 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:20:57.471 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.471 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:57.471 INFO:teuthology.orchestra.run.smithi040.stdout: "mgr": [ 2024-09-13T18:20:57.471 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.471 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:57.471 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:57.472 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 2 2024-09-13T18:20:57.472 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.472 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:57.472 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.472 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:20:57.472 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:20:57.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe024f1c700 1 -- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe0040663e0 msgr2=0x7fe0040688a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.478 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe024f1c700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe0040663e0 0x7fe0040688a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe00c007fd0 tx=0x7fe00c007000 comp rx=0 tx=0).stop 2024-09-13T18:20:57.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe024f1c700 1 -- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe018046740 msgr2=0x7fe018177b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe024f1c700 1 --2- 172.21.15.40:0/4094489480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe018046740 0x7fe018177b00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe01400c730 tx=0x7fe01400caf0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe01ffff700 1 -- 172.21.15.40:0/4094489480 reap_dead start 2024-09-13T18:20:57.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe01ffff700 1 -- 172.21.15.40:0/4094489480 reap_dead start 2024-09-13T18:20:57.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe024f1c700 1 -- 172.21.15.40:0/4094489480 shutdown_connections 2024-09-13T18:20:57.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe024f1c700 1 -- 172.21.15.40:0/4094489480 >> 172.21.15.40:0/4094489480 conn(0x7fe018050590 msgr2=0x7fe01804c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:57.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe024f1c700 1 -- 172.21.15.40:0/4094489480 shutdown_connections 2024-09-13T18:20:57.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.441+0000 7fe024f1c700 1 -- 172.21.15.40:0/4094489480 wait complete. 2024-09-13T18:20:57.482 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:20:57.482 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-13T18:20:57.483 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-13T18:20:57.700 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.689+0000 7f7b2f59e700 1 Processor -- start 2024-09-13T18:20:57.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.697+0000 7f7b2f59e700 1 -- start start 2024-09-13T18:20:57.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.697+0000 7f7b2f59e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7b30158bc0 0x7f7b300c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.697+0000 7f7b2f59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b300c6890 con 0x7f7b3014f8c0 2024-09-13T18:20:57.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.697+0000 7f7b2f59e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b300c5f30 con 0x7f7b300724e0 2024-09-13T18:20:57.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.697+0000 7f7b2f59e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b300c7700 con 0x7f7b30158bc0 2024-09-13T18:20:57.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.697+0000 7f7b2ed9d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7b30158bc0 0x7f7b300c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.697+0000 7f7b2ed9d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7b30158bc0 0x7f7b300c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:40960/0 (socket says 172.21.15.40:40960) 2024-09-13T18:20:57.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.697+0000 7f7b2ed9d700 1 -- 172.21.15.40:0/969493011 learned_addr learned my addr 172.21.15.40:0/969493011 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:57.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.697+0000 7f7b2ed9d700 1 -- 172.21.15.40:0/969493011 >> v1:172.21.15.43:6789/0 conn(0x7f7b300724e0 legacy=0x7f7b3015df40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:20:57.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.697+0000 7f7b2ed9d700 1 -- 172.21.15.40:0/969493011 >> v1:172.21.15.40:6789/0 conn(0x7f7b3014f8c0 legacy=0x7f7b3006d2d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:20:57.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.697+0000 7f7b2ed9d700 1 -- 172.21.15.40:0/969493011 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b300c1320 con 0x7f7b30158bc0 2024-09-13T18:20:57.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.697+0000 7f7b2ed9d700 1 --2- 172.21.15.40:0/969493011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b30158bc0 0x7f7b300c71c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009910 tx=0x7f7b20009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea45ede900f8c2cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:57.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.701+0000 7f7b2d59a700 1 -- 172.21.15.40:0/969493011 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20004230 con 0x7f7b30158bc0 2024-09-13T18:20:57.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.705+0000 7f7b2d59a700 1 -- 172.21.15.40:0/969493011 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b20004390 con 0x7f7b30158bc0 2024-09-13T18:20:57.715 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.705+0000 7f7b2d59a700 1 -- 172.21.15.40:0/969493011 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20009e60 con 0x7f7b30158bc0 2024-09-13T18:20:57.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.705+0000 7f7b2f59e700 1 -- 172.21.15.40:0/969493011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b30158bc0 msgr2=0x7f7b300c71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.705+0000 7f7b2f59e700 1 --2- 172.21.15.40:0/969493011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b30158bc0 0x7f7b300c71c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009910 tx=0x7f7b20009c20 comp rx=0 tx=0).stop 2024-09-13T18:20:57.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.705+0000 7f7b2f59e700 1 -- 172.21.15.40:0/969493011 shutdown_connections 2024-09-13T18:20:57.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.705+0000 7f7b2f59e700 1 --2- 172.21.15.40:0/969493011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b30158bc0 0x7f7b300c71c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.705+0000 7f7b2f59e700 1 -- 172.21.15.40:0/969493011 >> 172.21.15.40:0/969493011 conn(0x7f7b30066090 msgr2=0x7f7b300664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:57.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.705+0000 7f7b2f59e700 1 -- 172.21.15.40:0/969493011 shutdown_connections 2024-09-13T18:20:57.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.709+0000 7f7b2f59e700 1 -- 172.21.15.40:0/969493011 wait complete. 2024-09-13T18:20:57.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.709+0000 7f7b2f59e700 1 Processor -- start 2024-09-13T18:20:57.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.709+0000 7f7b2f59e700 1 -- start start 2024-09-13T18:20:57.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.713+0000 7f7b2f59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b300724e0 0x7f7b301520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.713+0000 7f7b2f59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b30158bc0 0x7f7b30150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.713+0000 7f7b2f59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b301fcd90 0x7f7b301fc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.713+0000 7f7b2f59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b30151920 con 0x7f7b301fcd90 2024-09-13T18:20:57.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.713+0000 7f7b2f59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b301fc860 con 0x7f7b300724e0 2024-09-13T18:20:57.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.713+0000 7f7b2f59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b301fd210 con 0x7f7b30158bc0 2024-09-13T18:20:57.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.713+0000 7f7b2ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b301fcd90 0x7f7b301fc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.713+0000 7f7b2ed9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b301fcd90 0x7f7b301fc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58160/0 (socket says 172.21.15.40:58160) 2024-09-13T18:20:57.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.713+0000 7f7b2ed9d700 1 -- 172.21.15.40:0/2929196435 learned_addr learned my addr 172.21.15.40:0/2929196435 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:57.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.713+0000 7f7b2ed9d700 1 -- 172.21.15.40:0/2929196435 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b30158bc0 msgr2=0x7f7b30150c10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:20:57.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.713+0000 7f7b2ed9d700 1 --2- 172.21.15.40:0/2929196435 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7b30158bc0 0x7f7b30150c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.713+0000 7f7b2ed9d700 1 -- 172.21.15.40:0/2929196435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b300724e0 msgr2=0x7f7b301520b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:20:57.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.713+0000 7f7b2ed9d700 1 --2- 172.21.15.40:0/2929196435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b300724e0 0x7f7b301520b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.713+0000 7f7b2ed9d700 1 -- 172.21.15.40:0/2929196435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b20009570 con 0x7f7b301fcd90 2024-09-13T18:20:57.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.713+0000 7f7b2ed9d700 1 --2- 172.21.15.40:0/2929196435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b301fcd90 0x7f7b301fc320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000b3c0 tx=0x7f7b20004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:57.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.717+0000 7f7b1effd700 1 -- 172.21.15.40:0/2929196435 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20004500 con 0x7f7b301fcd90 2024-09-13T18:20:57.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.717+0000 7f7b2f59e700 1 -- 172.21.15.40:0/2929196435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3014fee0 con 0x7f7b301fcd90 2024-09-13T18:20:57.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.717+0000 7f7b2f59e700 1 -- 172.21.15.40:0/2929196435 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b30151df0 con 0x7f7b301fcd90 2024-09-13T18:20:57.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.717+0000 7f7b1effd700 1 -- 172.21.15.40:0/2929196435 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b200036c0 con 0x7f7b301fcd90 2024-09-13T18:20:57.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.717+0000 7f7b1effd700 1 -- 172.21.15.40:0/2929196435 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20003a30 con 0x7f7b301fcd90 2024-09-13T18:20:57.730 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.721+0000 7f7b1effd700 1 -- 172.21.15.40:0/2929196435 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7b20003b90 con 0x7f7b301fcd90 2024-09-13T18:20:57.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.721+0000 7f7b1effd700 1 --2- 172.21.15.40:0/2929196435 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7b10066800 0x7f7b10068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.731 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.721+0000 7f7b1effd700 1 -- 172.21.15.40:0/2929196435 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7b200858d0 con 0x7f7b301fcd90 2024-09-13T18:20:57.733 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.725+0000 7f7b2e59c700 1 --2- 172.21.15.40:0/2929196435 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7b10066800 0x7f7b10068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.736 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.725+0000 7f7b2e59c700 1 --2- 172.21.15.40:0/2929196435 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7b10066800 0x7f7b10068cc0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7b3006b3f0 tx=0x7f7b24006c80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:57.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.729+0000 7f7b2f59e700 1 --2- 172.21.15.40:0/2929196435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b30156e10 0x7f7b30150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.729+0000 7f7b2f59e700 1 -- 172.21.15.40:0/2929196435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b30067720 con 0x7f7b30156e10 2024-09-13T18:20:57.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.741+0000 7f7b2dd9b700 1 --2- 172.21.15.40:0/2929196435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b30156e10 0x7f7b30150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.741+0000 7f7b2dd9b700 1 -- 172.21.15.40:0/2929196435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b3014fca0 con 0x7f7b30156e10 2024-09-13T18:20:57.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.745+0000 7f7b2dd9b700 1 --2- 172.21.15.40:0/2929196435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b30156e10 0x7f7b30150070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7b301f61a0 tx=0x7f7b28009ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.753+0000 7f7b1effd700 1 -- 172.21.15.40:0/2929196435 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2800cb90 con 0x7f7b30156e10 2024-09-13T18:20:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.753+0000 7f7b1effd700 1 -- 172.21.15.40:0/2929196435 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f7b2800ccf0 con 0x7f7b30156e10 2024-09-13T18:20:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.753+0000 7f7b1effd700 1 -- 172.21.15.40:0/2929196435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b30156e10 msgr2=0x7f7b30150070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.753+0000 7f7b1effd700 1 --2- 172.21.15.40:0/2929196435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b30156e10 0x7f7b30150070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7b301f61a0 tx=0x7f7b28009ef0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.753+0000 7f7b1effd700 1 -- 172.21.15.40:0/2929196435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b30156e10 msgr2=0x7f7b30150070 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:20:57.763 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.753+0000 7f7b1effd700 1 --2- 172.21.15.40:0/2929196435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b30156e10 0x7f7b30150070 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7b301f61a0 tx=0x7f7b28009ef0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.773+0000 7f7b2f59e700 1 --2- 172.21.15.40:0/2929196435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b301571f0 0x7f7b301505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.773+0000 7f7b2f59e700 1 -- 172.21.15.40:0/2929196435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300c5c50 con 0x7f7b301571f0 2024-09-13T18:20:57.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.773+0000 7f7b2dd9b700 1 --2- 172.21.15.40:0/2929196435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b301571f0 0x7f7b301505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.773+0000 7f7b2dd9b700 1 -- 172.21.15.40:0/2929196435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f7b300c6550 con 0x7f7b301571f0 2024-09-13T18:20:57.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.773+0000 7f7b2dd9b700 1 --2- 172.21.15.40:0/2929196435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b301571f0 0x7f7b301505b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7b301f61a0 tx=0x7f7b28004b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:57.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.777+0000 7f7b1effd700 1 -- 172.21.15.40:0/2929196435 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28014d40 con 0x7f7b301571f0 2024-09-13T18:20:57.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.777+0000 7f7b1effd700 1 -- 172.21.15.40:0/2929196435 <== mon.1 v2:172.21.15.43:3300/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4450 (secure 0 0 0) 0x7f7b28024430 con 0x7f7b301571f0 2024-09-13T18:20:57.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.777+0000 7f7b1effd700 1 -- 172.21.15.40:0/2929196435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b301571f0 msgr2=0x7f7b301505b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.777+0000 7f7b1effd700 1 --2- 172.21.15.40:0/2929196435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b301571f0 0x7f7b301505b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7b301f61a0 tx=0x7f7b28004b10 comp rx=0 tx=0).stop 2024-09-13T18:20:57.787 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:20:57.787 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:20:57.787 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:20:57.787 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:20:57.787 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 240, 2024-09-13T18:20:57.787 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:20:57.788 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:20:57.788 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:20:57.788 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:20:57.788 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:57.788 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 3, 2024-09-13T18:20:57.788 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:20:57.788 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:20:57.788 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:20:57.788 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:57.788 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:57.788 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:57.788 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:57.788 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:57.789 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:57.789 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:57.789 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:57.789 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:57.789 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:57.789 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:57.789 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:20:57.789 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:20:57.789 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:57.789 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:57.789 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:57.789 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:57.789 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:57.789 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:57.790 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:57.790 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:57.790 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:57.790 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:57.790 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:57.790 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.790 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:20:57.790 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:20:57.790 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:20:57.790 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:20:57.790 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:20:57.790 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:20:57.790 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:20:57.791 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:20:57.791 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:20:57.791 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:20:57.791 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:20:57.791 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:20:57.791 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:20:57.791 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:20:57.791 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:20:57.791 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:20:57.791 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:20:57.791 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:57.791 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:57.791 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:57.791 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:57.792 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:57.792 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:57.792 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:57.792 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:57.792 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:57.792 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:57.792 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:57.792 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:20:57.792 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.792 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:20:57.792 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.792 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:20:57.792 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:20:57.792 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:57.793 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:57.793 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.793 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:57.793 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:20:57.793 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:57.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.793 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.793 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:57.793 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:20:57.793 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:57.793 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.793 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:57.793 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.793 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:20:57.794 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:20:57.794 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:57.794 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:57.794 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:57.794 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.794 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.794 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:20:57.794 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:20:57.794 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:57.794 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:57.794 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.794 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:57.794 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:20:57.794 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:57.795 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.795 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.795 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:57.795 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:20:57.795 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:57.795 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.795 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:57.795 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.795 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:20:57.795 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:20:57.795 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:57.795 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:57.795 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:57.795 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.796 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.796 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:20:57.796 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:20:57.796 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:57.796 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:57.796 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.796 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:57.796 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:20:57.796 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:57.796 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.796 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.796 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:57.796 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:20:57.797 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:57.797 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.797 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:57.797 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.797 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:20:57.797 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:20:57.797 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:57.797 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:57.797 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:57.797 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.797 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:57.797 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.797 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:20:57.797 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:20:57.798 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.798 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:57.798 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:57.798 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:20:57.798 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.798 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:57.798 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:20:57.798 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.798 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:57.798 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:57.798 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:20:57.798 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.798 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:57.798 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.799 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:20:57.799 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:20:57.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.777+0000 7f7b1effd700 1 -- 172.21.15.40:0/2929196435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b301571f0 msgr2=0x7f7b301505b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:20:57.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.777+0000 7f7b1effd700 1 --2- 172.21.15.40:0/2929196435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b301571f0 0x7f7b301505b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.777+0000 7f7b2f59e700 1 -- 172.21.15.40:0/2929196435 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7b10066800 msgr2=0x7f7b10068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.777+0000 7f7b2f59e700 1 --2- 172.21.15.40:0/2929196435 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7b10066800 0x7f7b10068cc0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7b3006b3f0 tx=0x7f7b24006c80 comp rx=0 tx=0).stop 2024-09-13T18:20:57.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.777+0000 7f7b2f59e700 1 -- 172.21.15.40:0/2929196435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b301fcd90 msgr2=0x7f7b301fc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.777+0000 7f7b2f59e700 1 --2- 172.21.15.40:0/2929196435 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b301fcd90 0x7f7b301fc320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000b3c0 tx=0x7f7b20004770 comp rx=0 tx=0).stop 2024-09-13T18:20:57.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.777+0000 7f7b2ed9d700 1 -- 172.21.15.40:0/2929196435 reap_dead start 2024-09-13T18:20:57.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.777+0000 7f7b2ed9d700 1 -- 172.21.15.40:0/2929196435 reap_dead start 2024-09-13T18:20:57.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.777+0000 7f7b2f59e700 1 -- 172.21.15.40:0/2929196435 shutdown_connections 2024-09-13T18:20:57.800 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.777+0000 7f7b2f59e700 1 -- 172.21.15.40:0/2929196435 >> 172.21.15.40:0/2929196435 conn(0x7f7b30066090 msgr2=0x7f7b30070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:57.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.777+0000 7f7b2f59e700 1 -- 172.21.15.40:0/2929196435 shutdown_connections 2024-09-13T18:20:57.801 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.777+0000 7f7b2f59e700 1 -- 172.21.15.40:0/2929196435 wait complete. 2024-09-13T18:20:57.824 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:20:57.825 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-13T18:20:57.825 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-13T18:20:57.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4637fff700 1 Processor -- start 2024-09-13T18:20:57.930 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4637fff700 1 -- start start 2024-09-13T18:20:57.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4637fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46300d8920 0x7f4630049090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4637fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4630007480 0x7f463004efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4637fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4630054df0 con 0x7f46300e1c00 2024-09-13T18:20:57.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4637fff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4630050a10 con 0x7f4630007480 2024-09-13T18:20:57.931 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4637fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4630006ff0 con 0x7f46300d8920 2024-09-13T18:20:57.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4636ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46300d8920 0x7f4630049090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f46367fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4630007480 0x7f463004efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4636ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46300d8920 0x7f4630049090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:55578/0 (socket says 172.21.15.40:55578) 2024-09-13T18:20:57.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4636ffd700 1 -- 172.21.15.40:0/2311272910 learned_addr learned my addr 172.21.15.40:0/2311272910 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:57.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f46367fc700 1 -- 172.21.15.40:0/2311272910 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46300d8920 msgr2=0x7f4630049090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:20:57.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f46367fc700 1 --2- 172.21.15.40:0/2311272910 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46300d8920 0x7f4630049090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f46367fc700 1 -- 172.21.15.40:0/2311272910 >> v1:172.21.15.43:6789/0 conn(0x7f46300e1c00 legacy=0x7f46300e2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f46367fc700 1 -- 172.21.15.40:0/2311272910 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46300495d0 con 0x7f4630007480 2024-09-13T18:20:57.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f46367fc700 1 --2- 172.21.15.40:0/2311272910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4630007480 0x7f463004efb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4628009840 tx=0x7f4628009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d91f1523aad5955 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:57.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4635ffb700 1 -- 172.21.15.40:0/2311272910 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4628005e50 con 0x7f4630007480 2024-09-13T18:20:57.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4635ffb700 1 -- 172.21.15.40:0/2311272910 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4628006030 con 0x7f4630007480 2024-09-13T18:20:57.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4635ffb700 1 -- 172.21.15.40:0/2311272910 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46280063c0 con 0x7f4630007480 2024-09-13T18:20:57.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4637fff700 1 -- 172.21.15.40:0/2311272910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4630007480 msgr2=0x7f463004efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4637fff700 1 --2- 172.21.15.40:0/2311272910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4630007480 0x7f463004efb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4628009840 tx=0x7f4628009b50 comp rx=0 tx=0).stop 2024-09-13T18:20:57.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4637fff700 1 -- 172.21.15.40:0/2311272910 shutdown_connections 2024-09-13T18:20:57.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4637fff700 1 --2- 172.21.15.40:0/2311272910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4630007480 0x7f463004efb0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4637fff700 1 --2- 172.21.15.40:0/2311272910 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46300d8920 0x7f4630049090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4637fff700 1 -- 172.21.15.40:0/2311272910 >> 172.21.15.40:0/2311272910 conn(0x7f4630054110 msgr2=0x7f4630054520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:57.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4637fff700 1 -- 172.21.15.40:0/2311272910 shutdown_connections 2024-09-13T18:20:57.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.921+0000 7f4637fff700 1 -- 172.21.15.40:0/2311272910 wait complete. 2024-09-13T18:20:57.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.925+0000 7f4637fff700 1 Processor -- start 2024-09-13T18:20:57.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.925+0000 7f4637fff700 1 -- start start 2024-09-13T18:20:57.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.925+0000 7f4637fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4630007480 0x7f4630038330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.925+0000 7f4637fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46300d8920 0x7f4630177c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.925+0000 7f4637fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46300e1c00 0x7f4630052800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.925+0000 7f4637fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4630052d40 con 0x7f46300d8920 2024-09-13T18:20:57.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.925+0000 7f4637fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46300e07e0 con 0x7f4630007480 2024-09-13T18:20:57.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.925+0000 7f4637fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4630051ff0 con 0x7f46300e1c00 2024-09-13T18:20:57.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.925+0000 7f46367fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46300d8920 0x7f4630177c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.925+0000 7f4636ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4630007480 0x7f4630038330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.925+0000 7f46367fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46300d8920 0x7f4630177c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58164/0 (socket says 172.21.15.40:58164) 2024-09-13T18:20:57.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.925+0000 7f46367fc700 1 -- 172.21.15.40:0/3236520285 learned_addr learned my addr 172.21.15.40:0/3236520285 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:57.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.925+0000 7f46377fe700 1 --2- 172.21.15.40:0/3236520285 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46300e1c00 0x7f4630052800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.925+0000 7f46367fc700 1 -- 172.21.15.40:0/3236520285 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46300e1c00 msgr2=0x7f4630052800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.925+0000 7f46367fc700 1 --2- 172.21.15.40:0/3236520285 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f46300e1c00 0x7f4630052800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.925+0000 7f46367fc700 1 -- 172.21.15.40:0/3236520285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4630007480 msgr2=0x7f4630038330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.925+0000 7f46367fc700 1 --2- 172.21.15.40:0/3236520285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4630007480 0x7f4630038330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.925+0000 7f46367fc700 1 -- 172.21.15.40:0/3236520285 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4628009450 con 0x7f46300d8920 2024-09-13T18:20:57.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.929+0000 7f46367fc700 1 --2- 172.21.15.40:0/3236520285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46300d8920 0x7f4630177c40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4628000c00 tx=0x7f4628004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:57.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.929+0000 7f461f7fe700 1 -- 172.21.15.40:0/3236520285 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4628003ad0 con 0x7f46300d8920 2024-09-13T18:20:57.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.929+0000 7f461f7fe700 1 -- 172.21.15.40:0/3236520285 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4628003c30 con 0x7f46300d8920 2024-09-13T18:20:57.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.929+0000 7f461f7fe700 1 -- 172.21.15.40:0/3236520285 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46280061a0 con 0x7f46300d8920 2024-09-13T18:20:57.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.929+0000 7f4637fff700 1 -- 172.21.15.40:0/3236520285 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46301771a0 con 0x7f46300d8920 2024-09-13T18:20:57.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.929+0000 7f4637fff700 1 -- 172.21.15.40:0/3236520285 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4630053b80 con 0x7f46300d8920 2024-09-13T18:20:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.929+0000 7f4637fff700 1 --2- 172.21.15.40:0/3236520285 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4630052fb0 0x7f46300cf140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.929+0000 7f4637fff700 1 -- 172.21.15.40:0/3236520285 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46300084b0 con 0x7f4630052fb0 2024-09-13T18:20:57.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.929+0000 7f461f7fe700 1 -- 172.21.15.40:0/3236520285 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4628022070 con 0x7f46300d8920 2024-09-13T18:20:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.929+0000 7f4636ffd700 1 --2- 172.21.15.40:0/3236520285 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4630052fb0 0x7f46300cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.929+0000 7f461f7fe700 1 --2- 172.21.15.40:0/3236520285 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4624066730 0x7f4624068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.929+0000 7f4636ffd700 1 -- 172.21.15.40:0/3236520285 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f46300069e0 con 0x7f4630052fb0 2024-09-13T18:20:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.929+0000 7f461f7fe700 1 -- 172.21.15.40:0/3236520285 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4628085cd0 con 0x7f46300d8920 2024-09-13T18:20:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.929+0000 7f4636ffd700 1 --2- 172.21.15.40:0/3236520285 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4630052fb0 0x7f46300cf140 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f462c0119a0 tx=0x7f462c011d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.933+0000 7f461f7fe700 1 -- 172.21.15.40:0/3236520285 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c0043a0 con 0x7f4630052fb0 2024-09-13T18:20:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.933+0000 7f461f7fe700 1 -- 172.21.15.40:0/3236520285 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+9332 (secure 0 0 0) 0x7f462c004500 con 0x7f4630052fb0 2024-09-13T18:20:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.933+0000 7f461f7fe700 1 -- 172.21.15.40:0/3236520285 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4630052fb0 msgr2=0x7f46300cf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.933+0000 7f461f7fe700 1 --2- 172.21.15.40:0/3236520285 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4630052fb0 0x7f46300cf140 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f462c0119a0 tx=0x7f462c011d60 comp rx=0 tx=0).stop 2024-09-13T18:20:57.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.933+0000 7f461f7fe700 1 -- 172.21.15.40:0/3236520285 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4630052fb0 msgr2=0x7f46300cf140 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:20:57.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.933+0000 7f461f7fe700 1 --2- 172.21.15.40:0/3236520285 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4630052fb0 0x7f46300cf140 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.933+0000 7f46377fe700 1 --2- 172.21.15.40:0/3236520285 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4624066730 0x7f4624068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.933+0000 7f46377fe700 1 --2- 172.21.15.40:0/3236520285 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4624066730 0x7f4624068bf0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4638063cf0 tx=0x7f4638065ea0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:57.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.945+0000 7f4637fff700 1 --2- 172.21.15.40:0/3236520285 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4630177330 0x7f46300cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:57.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.945+0000 7f4637fff700 1 -- 172.21.15.40:0/3236520285 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46300501b0 con 0x7f4630177330 2024-09-13T18:20:57.954 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.945+0000 7f4636ffd700 1 --2- 172.21.15.40:0/3236520285 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4630177330 0x7f46300cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:57.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.945+0000 7f4636ffd700 1 -- 172.21.15.40:0/3236520285 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- command(tid 2: {"prefix": "mon_status"}) v1 -- 0x7f4630050450 con 0x7f4630177330 2024-09-13T18:20:57.955 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.945+0000 7f4636ffd700 1 --2- 172.21.15.40:0/3236520285 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4630177330 0x7f46300cb920 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f462c0119a0 tx=0x7f462c004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:57.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.945+0000 7f461f7fe700 1 -- 172.21.15.40:0/3236520285 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c01e650 con 0x7f4630177330 2024-09-13T18:20:57.956 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.945+0000 7f461f7fe700 1 -- 172.21.15.40:0/3236520285 <== mon.2 v2:172.21.15.43:3301/0 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+4782 (secure 0 0 0) 0x7f462c024040 con 0x7f4630177330 2024-09-13T18:20:57.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.945+0000 7f461f7fe700 1 -- 172.21.15.40:0/3236520285 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4630177330 msgr2=0x7f46300cb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.945+0000 7f461f7fe700 1 --2- 172.21.15.40:0/3236520285 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4630177330 0x7f46300cb920 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f462c0119a0 tx=0x7f462c004770 comp rx=0 tx=0).stop 2024-09-13T18:20:57.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.945+0000 7f461f7fe700 1 -- 172.21.15.40:0/3236520285 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4630177330 msgr2=0x7f46300cb920 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T18:20:57.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.945+0000 7f461f7fe700 1 --2- 172.21.15.40:0/3236520285 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4630177330 0x7f46300cb920 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.957 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:20:57.957 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:20:57.957 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:20:57.957 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "peon", 2024-09-13T18:20:57.958 INFO:teuthology.orchestra.run.smithi040.stdout: "election_epoch": 240, 2024-09-13T18:20:57.958 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum": [ 2024-09-13T18:20:57.958 INFO:teuthology.orchestra.run.smithi040.stdout: 0, 2024-09-13T18:20:57.958 INFO:teuthology.orchestra.run.smithi040.stdout: 1, 2024-09-13T18:20:57.958 INFO:teuthology.orchestra.run.smithi040.stdout: 2 2024-09-13T18:20:57.958 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:57.958 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_age": 3, 2024-09-13T18:20:57.958 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:20:57.958 INFO:teuthology.orchestra.run.smithi040.stdout: "required_con": "2449958758054445060", 2024-09-13T18:20:57.958 INFO:teuthology.orchestra.run.smithi040.stdout: "required_mon": [ 2024-09-13T18:20:57.958 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:57.958 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:57.959 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:57.959 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:57.959 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:57.959 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:57.959 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:57.959 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:57.959 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:57.959 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:57.959 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:57.959 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_con": "4540138322906710015", 2024-09-13T18:20:57.959 INFO:teuthology.orchestra.run.smithi040.stdout: "quorum_mon": [ 2024-09-13T18:20:57.960 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:57.960 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:57.960 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:57.960 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:57.960 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:57.960 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:57.960 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:57.960 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:57.960 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:57.960 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:57.961 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:57.961 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.961 INFO:teuthology.orchestra.run.smithi040.stdout: "outside_quorum": [], 2024-09-13T18:20:57.961 INFO:teuthology.orchestra.run.smithi040.stdout: "extra_probe_peers": [], 2024-09-13T18:20:57.961 INFO:teuthology.orchestra.run.smithi040.stdout: "sync_provider": [], 2024-09-13T18:20:57.961 INFO:teuthology.orchestra.run.smithi040.stdout: "monmap": { 2024-09-13T18:20:57.961 INFO:teuthology.orchestra.run.smithi040.stdout: "epoch": 1, 2024-09-13T18:20:57.961 INFO:teuthology.orchestra.run.smithi040.stdout: "fsid": "841d6eb0-79fe-4108-906c-e8798e475996", 2024-09-13T18:20:57.961 INFO:teuthology.orchestra.run.smithi040.stdout: "modified": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:20:57.961 INFO:teuthology.orchestra.run.smithi040.stdout: "created": "2024-09-13T17:49:34.123582Z", 2024-09-13T18:20:57.961 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release": 18, 2024-09-13T18:20:57.962 INFO:teuthology.orchestra.run.smithi040.stdout: "min_mon_release_name": "reef", 2024-09-13T18:20:57.962 INFO:teuthology.orchestra.run.smithi040.stdout: "election_strategy": 1, 2024-09-13T18:20:57.962 INFO:teuthology.orchestra.run.smithi040.stdout: "disallowed_leaders: ": "", 2024-09-13T18:20:57.962 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false, 2024-09-13T18:20:57.962 INFO:teuthology.orchestra.run.smithi040.stdout: "tiebreaker_mon": "", 2024-09-13T18:20:57.962 INFO:teuthology.orchestra.run.smithi040.stdout: "removed_ranks: ": "", 2024-09-13T18:20:57.962 INFO:teuthology.orchestra.run.smithi040.stdout: "features": { 2024-09-13T18:20:57.962 INFO:teuthology.orchestra.run.smithi040.stdout: "persistent": [ 2024-09-13T18:20:57.962 INFO:teuthology.orchestra.run.smithi040.stdout: "kraken", 2024-09-13T18:20:57.962 INFO:teuthology.orchestra.run.smithi040.stdout: "luminous", 2024-09-13T18:20:57.962 INFO:teuthology.orchestra.run.smithi040.stdout: "mimic", 2024-09-13T18:20:57.962 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap-prune", 2024-09-13T18:20:57.963 INFO:teuthology.orchestra.run.smithi040.stdout: "nautilus", 2024-09-13T18:20:57.963 INFO:teuthology.orchestra.run.smithi040.stdout: "octopus", 2024-09-13T18:20:57.963 INFO:teuthology.orchestra.run.smithi040.stdout: "pacific", 2024-09-13T18:20:57.963 INFO:teuthology.orchestra.run.smithi040.stdout: "elector-pinging", 2024-09-13T18:20:57.963 INFO:teuthology.orchestra.run.smithi040.stdout: "quincy", 2024-09-13T18:20:57.963 INFO:teuthology.orchestra.run.smithi040.stdout: "reef" 2024-09-13T18:20:57.963 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:57.963 INFO:teuthology.orchestra.run.smithi040.stdout: "optional": [] 2024-09-13T18:20:57.963 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.963 INFO:teuthology.orchestra.run.smithi040.stdout: "mons": [ 2024-09-13T18:20:57.963 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.963 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 0, 2024-09-13T18:20:57.963 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "a", 2024-09-13T18:20:57.964 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:57.964 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:57.964 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.964 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:57.964 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:3300", 2024-09-13T18:20:57.964 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:57.964 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.964 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.964 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:57.964 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789", 2024-09-13T18:20:57.964 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:57.964 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.965 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:57.965 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.965 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.40:6789/0", 2024-09-13T18:20:57.965 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.40:6789/0", 2024-09-13T18:20:57.965 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:57.965 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:57.965 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:57.965 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.965 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.965 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 1, 2024-09-13T18:20:57.965 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "b", 2024-09-13T18:20:57.966 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:57.966 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:57.966 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.966 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:57.966 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3300", 2024-09-13T18:20:57.966 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:57.966 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.966 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.967 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:57.967 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789", 2024-09-13T18:20:57.967 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:57.967 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.967 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:57.967 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.967 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6789/0", 2024-09-13T18:20:57.967 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6789/0", 2024-09-13T18:20:57.967 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:57.967 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:57.967 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:57.967 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.968 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.968 INFO:teuthology.orchestra.run.smithi040.stdout: "rank": 2, 2024-09-13T18:20:57.968 INFO:teuthology.orchestra.run.smithi040.stdout: "name": "c", 2024-09-13T18:20:57.968 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addrs": { 2024-09-13T18:20:57.968 INFO:teuthology.orchestra.run.smithi040.stdout: "addrvec": [ 2024-09-13T18:20:57.968 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.968 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v2", 2024-09-13T18:20:57.968 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:3301", 2024-09-13T18:20:57.968 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:57.969 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.969 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.969 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-09-13T18:20:57.969 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790", 2024-09-13T18:20:57.969 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 0 2024-09-13T18:20:57.969 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.969 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:57.969 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.969 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "172.21.15.43:6790/0", 2024-09-13T18:20:57.969 INFO:teuthology.orchestra.run.smithi040.stdout: "public_addr": "172.21.15.43:6790/0", 2024-09-13T18:20:57.969 INFO:teuthology.orchestra.run.smithi040.stdout: "priority": 0, 2024-09-13T18:20:57.969 INFO:teuthology.orchestra.run.smithi040.stdout: "weight": 0, 2024-09-13T18:20:57.970 INFO:teuthology.orchestra.run.smithi040.stdout: "crush_location": "{}" 2024-09-13T18:20:57.970 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.970 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:57.970 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.970 INFO:teuthology.orchestra.run.smithi040.stdout: "feature_map": { 2024-09-13T18:20:57.970 INFO:teuthology.orchestra.run.smithi040.stdout: "mon": [ 2024-09-13T18:20:57.970 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.970 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:57.970 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:57.970 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 1 2024-09-13T18:20:57.970 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.970 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:57.970 INFO:teuthology.orchestra.run.smithi040.stdout: "mds": [ 2024-09-13T18:20:57.971 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.971 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:57.971 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:57.971 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:20:57.971 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.971 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:57.971 INFO:teuthology.orchestra.run.smithi040.stdout: "osd": [ 2024-09-13T18:20:57.971 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.971 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:57.971 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:57.972 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 5 2024-09-13T18:20:57.972 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.972 INFO:teuthology.orchestra.run.smithi040.stdout: ], 2024-09-13T18:20:57.972 INFO:teuthology.orchestra.run.smithi040.stdout: "client": [ 2024-09-13T18:20:57.972 INFO:teuthology.orchestra.run.smithi040.stdout: { 2024-09-13T18:20:57.972 INFO:teuthology.orchestra.run.smithi040.stdout: "features": "0x3f01cfbffffdffff", 2024-09-13T18:20:57.973 INFO:teuthology.orchestra.run.smithi040.stdout: "release": "reef", 2024-09-13T18:20:57.973 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 3 2024-09-13T18:20:57.973 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-09-13T18:20:57.973 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-09-13T18:20:57.973 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-09-13T18:20:57.973 INFO:teuthology.orchestra.run.smithi040.stdout: "stretch_mode": false 2024-09-13T18:20:57.974 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:20:57.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.945+0000 7f4637fff700 1 -- 172.21.15.40:0/3236520285 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4624066730 msgr2=0x7f4624068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.945+0000 7f4637fff700 1 --2- 172.21.15.40:0/3236520285 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4624066730 0x7f4624068bf0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4638063cf0 tx=0x7f4638065ea0 comp rx=0 tx=0).stop 2024-09-13T18:20:57.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.945+0000 7f4637fff700 1 -- 172.21.15.40:0/3236520285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46300d8920 msgr2=0x7f4630177c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:57.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.945+0000 7f4637fff700 1 --2- 172.21.15.40:0/3236520285 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f46300d8920 0x7f4630177c40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4628000c00 tx=0x7f4628004770 comp rx=0 tx=0).stop 2024-09-13T18:20:57.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.945+0000 7f46377fe700 1 -- 172.21.15.40:0/3236520285 reap_dead start 2024-09-13T18:20:57.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.945+0000 7f46377fe700 1 -- 172.21.15.40:0/3236520285 reap_dead start 2024-09-13T18:20:57.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.949+0000 7f4637fff700 1 -- 172.21.15.40:0/3236520285 shutdown_connections 2024-09-13T18:20:57.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.949+0000 7f4637fff700 1 -- 172.21.15.40:0/3236520285 >> 172.21.15.40:0/3236520285 conn(0x7f4630054110 msgr2=0x7f463004a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:57.977 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.949+0000 7f4637fff700 1 -- 172.21.15.40:0/3236520285 shutdown_connections 2024-09-13T18:20:57.978 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:57.949+0000 7f4637fff700 1 -- 172.21.15.40:0/3236520285 wait complete. 2024-09-13T18:20:57.978 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-13T18:20:57.978 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-13T18:20:57.979 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-13T18:20:58.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.089+0000 7f106dc65700 1 Processor -- start 2024-09-13T18:20:58.099 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.089+0000 7f106dc65700 1 -- start start 2024-09-13T18:20:58.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.089+0000 7f106dc65700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f10681510d0 0x7f1068157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:58.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.089+0000 7f106dc65700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10681514b0 0x7f10680c2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:58.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.089+0000 7f106dc65700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f106815a8b0 con 0x7f10680c7400 2024-09-13T18:20:58.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.089+0000 7f106dc65700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10681581e0 con 0x7f10681510d0 2024-09-13T18:20:58.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.089+0000 7f106dc65700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1068066390 con 0x7f10681514b0 2024-09-13T18:20:58.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.089+0000 7f106d464700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f10680c7400 0x7f10680c4dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:49142/0 (socket says 172.21.15.40:49142) 2024-09-13T18:20:58.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.089+0000 7f106d464700 1 -- 172.21.15.40:0/2149359257 learned_addr learned my addr 172.21.15.40:0/2149359257 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:20:58.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.089+0000 7f106cc63700 1 --2- 172.21.15.40:0/2149359257 >> v2:172.21.15.43:3300/0 conn(0x7f10681510d0 0x7f1068157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:58.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.089+0000 7f1067fff700 1 --2- 172.21.15.40:0/2149359257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10681514b0 0x7f10680c2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:58.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.089+0000 7f10677fe700 1 -- 172.21.15.40:0/2149359257 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 854073277 0 0) 0x7f106815a8b0 con 0x7f10680c7400 2024-09-13T18:20:58.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.089+0000 7f10677fe700 1 -- 172.21.15.40:0/2149359257 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10500035d0 con 0x7f10680c7400 2024-09-13T18:20:58.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.089+0000 7f106cc63700 1 -- 172.21.15.40:0/2149359257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10681514b0 msgr2=0x7f10680c2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:20:58.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.089+0000 7f106cc63700 1 --2- 172.21.15.40:0/2149359257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10681514b0 0x7f10680c2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:58.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.089+0000 7f106cc63700 1 -- 172.21.15.40:0/2149359257 >> v1:172.21.15.40:6789/0 conn(0x7f10680c7400 legacy=0x7f10680c4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:58.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.093+0000 7f106cc63700 1 -- 172.21.15.40:0/2149359257 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106815ab20 con 0x7f10681510d0 2024-09-13T18:20:58.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.093+0000 7f106cc63700 1 --2- 172.21.15.40:0/2149359257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10681510d0 0x7f1068157ca0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1058009d90 tx=0x7f105800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=beb75c357532d8c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:58.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.093+0000 7f10677fe700 1 -- 172.21.15.40:0/2149359257 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105800e070 con 0x7f10681510d0 2024-09-13T18:20:58.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.093+0000 7f10677fe700 1 -- 172.21.15.40:0/2149359257 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105800bb20 con 0x7f10681510d0 2024-09-13T18:20:58.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.093+0000 7f10677fe700 1 -- 172.21.15.40:0/2149359257 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058012430 con 0x7f10681510d0 2024-09-13T18:20:58.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.093+0000 7f106dc65700 1 -- 172.21.15.40:0/2149359257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10681510d0 msgr2=0x7f1068157ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:58.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.093+0000 7f106dc65700 1 --2- 172.21.15.40:0/2149359257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10681510d0 0x7f1068157ca0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1058009d90 tx=0x7f105800b330 comp rx=0 tx=0).stop 2024-09-13T18:20:58.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.093+0000 7f106dc65700 1 -- 172.21.15.40:0/2149359257 shutdown_connections 2024-09-13T18:20:58.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.093+0000 7f106dc65700 1 --2- 172.21.15.40:0/2149359257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10681514b0 0x7f10680c2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:58.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.093+0000 7f106dc65700 1 --2- 172.21.15.40:0/2149359257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10681510d0 0x7f1068157ca0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:58.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.093+0000 7f106dc65700 1 -- 172.21.15.40:0/2149359257 >> 172.21.15.40:0/2149359257 conn(0x7f10680c7d70 msgr2=0x7f10680c8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:58.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.093+0000 7f106dc65700 1 -- 172.21.15.40:0/2149359257 shutdown_connections 2024-09-13T18:20:58.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.093+0000 7f106dc65700 1 -- 172.21.15.40:0/2149359257 wait complete. 2024-09-13T18:20:58.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.093+0000 7f106dc65700 1 Processor -- start 2024-09-13T18:20:58.106 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.093+0000 7f106dc65700 1 -- start start 2024-09-13T18:20:58.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f106dc65700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10680c7400 0x7f1068144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:58.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f106dc65700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10681510d0 0x7f106814b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:58.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f106dc65700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10681514b0 0x7f106814bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:58.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f106dc65700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10681eda50 con 0x7f10681510d0 2024-09-13T18:20:58.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f106dc65700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106814ad10 con 0x7f10681514b0 2024-09-13T18:20:58.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f106dc65700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10681ece40 con 0x7f10680c7400 2024-09-13T18:20:58.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f1067fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10681510d0 0x7f106814b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:58.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f1067fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10681510d0 0x7f106814b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58170/0 (socket says 172.21.15.40:58170) 2024-09-13T18:20:58.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f1067fff700 1 -- 172.21.15.40:0/2875023518 learned_addr learned my addr 172.21.15.40:0/2875023518 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:20:58.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f106cc63700 1 --2- 172.21.15.40:0/2875023518 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10680c7400 0x7f1068144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:58.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f106d464700 1 --2- 172.21.15.40:0/2875023518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10681514b0 0x7f106814bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:58.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f1067fff700 1 -- 172.21.15.40:0/2875023518 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10680c7400 msgr2=0x7f1068144900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:58.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f1067fff700 1 --2- 172.21.15.40:0/2875023518 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10680c7400 0x7f1068144900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:58.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f1067fff700 1 -- 172.21.15.40:0/2875023518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10681514b0 msgr2=0x7f106814bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:58.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f1067fff700 1 --2- 172.21.15.40:0/2875023518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10681514b0 0x7f106814bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:58.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f1067fff700 1 -- 172.21.15.40:0/2875023518 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1058009a70 con 0x7f10681510d0 2024-09-13T18:20:58.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f1067fff700 1 --2- 172.21.15.40:0/2875023518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10681510d0 0x7f106814b6a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f105c009fd0 tx=0x7f105c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:58.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f10657fa700 1 -- 172.21.15.40:0/2875023518 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c0095a0 con 0x7f10681510d0 2024-09-13T18:20:58.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f10657fa700 1 -- 172.21.15.40:0/2875023518 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105c013680 con 0x7f10681510d0 2024-09-13T18:20:58.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f106dc65700 1 -- 172.21.15.40:0/2875023518 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10681ed120 con 0x7f10681510d0 2024-09-13T18:20:58.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f10657fa700 1 -- 172.21.15.40:0/2875023518 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c013a30 con 0x7f10681510d0 2024-09-13T18:20:58.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f106dc65700 1 -- 172.21.15.40:0/2875023518 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10681ed6e0 con 0x7f10681510d0 2024-09-13T18:20:58.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.097+0000 7f106dc65700 1 -- 172.21.15.40:0/2875023518 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10681ed8a0 con 0x7f10681510d0 2024-09-13T18:20:58.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.101+0000 7f10657fa700 1 -- 172.21.15.40:0/2875023518 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f105c013b90 con 0x7f10681510d0 2024-09-13T18:20:58.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.101+0000 7f10657fa700 1 --2- 172.21.15.40:0/2875023518 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1054066760 0x7f1054068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:20:58.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.101+0000 7f10657fa700 1 -- 172.21.15.40:0/2875023518 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f105c084320 con 0x7f10681510d0 2024-09-13T18:20:58.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.101+0000 7f106cc63700 1 --2- 172.21.15.40:0/2875023518 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1054066760 0x7f1054068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:20:58.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.101+0000 7f106cc63700 1 --2- 172.21.15.40:0/2875023518 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1054066760 0x7f1054068c20 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1058014040 tx=0x7f1058011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:20:58.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.109+0000 7f10657fa700 1 -- 172.21.15.40:0/2875023518 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f105c051240 con 0x7f10681510d0 2024-09-13T18:20:58.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.441+0000 7f106dc65700 1 -- 172.21.15.40:0/2875023518 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) v1 -- 0x7f1068066670 con 0x7f10681510d0 2024-09-13T18:20:58.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.445+0000 7f10657fa700 1 -- 172.21.15.40:0/2875023518 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) v1 ==== 57+0+0 (secure 0 0 0) 0x7f105c054860 con 0x7f10681510d0 2024-09-13T18:20:58.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.449+0000 7f106dc65700 1 -- 172.21.15.40:0/2875023518 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1054066760 msgr2=0x7f1054068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:58.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.449+0000 7f106dc65700 1 --2- 172.21.15.40:0/2875023518 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1054066760 0x7f1054068c20 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1058014040 tx=0x7f1058011000 comp rx=0 tx=0).stop 2024-09-13T18:20:58.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.449+0000 7f106dc65700 1 -- 172.21.15.40:0/2875023518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10681510d0 msgr2=0x7f106814b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:20:58.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.449+0000 7f106dc65700 1 --2- 172.21.15.40:0/2875023518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10681510d0 0x7f106814b6a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f105c009fd0 tx=0x7f105c00eb20 comp rx=0 tx=0).stop 2024-09-13T18:20:58.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.449+0000 7f106dc65700 1 -- 172.21.15.40:0/2875023518 shutdown_connections 2024-09-13T18:20:58.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.449+0000 7f106dc65700 1 --2- 172.21.15.40:0/2875023518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10681514b0 0x7f106814bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:58.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.449+0000 7f106dc65700 1 --2- 172.21.15.40:0/2875023518 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10681510d0 0x7f106814b6a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:58.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.449+0000 7f106dc65700 1 --2- 172.21.15.40:0/2875023518 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1054066760 0x7f1054068c20 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:58.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.449+0000 7f106dc65700 1 --2- 172.21.15.40:0/2875023518 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10680c7400 0x7f1068144900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:20:58.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.449+0000 7f106dc65700 1 -- 172.21.15.40:0/2875023518 >> 172.21.15.40:0/2875023518 conn(0x7f10680c7d70 msgr2=0x7f10680c4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:20:58.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.449+0000 7f106dc65700 1 -- 172.21.15.40:0/2875023518 shutdown_connections 2024-09-13T18:20:58.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:20:58.449+0000 7f106dc65700 1 -- 172.21.15.40:0/2875023518 wait complete. 2024-09-13T18:20:58.473 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-13T18:21:08.474 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:21:08.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.593+0000 7f2f8f2ff700 1 Processor -- start 2024-09-13T18:21:08.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.593+0000 7f2f8f2ff700 1 -- start start 2024-09-13T18:21:08.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8f2ff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2f88159890 0x7f2f880722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:08.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8f2ff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f8806b870 con 0x7f2f880c78a0 2024-09-13T18:21:08.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8f2ff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f88071dc0 con 0x7f2f880702e0 2024-09-13T18:21:08.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8f2ff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f8806ed60 con 0x7f2f88159890 2024-09-13T18:21:08.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8e2fd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2f88159890 0x7f2f880722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:08.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8eafe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2f880c78a0 0x7f2f8814fd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50726/0 (socket says 172.21.15.40:50726) 2024-09-13T18:21:08.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8eafe700 1 -- 172.21.15.40:0/2617093594 learned_addr learned my addr 172.21.15.40:0/2617093594 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:08.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8e2fd700 1 -- 172.21.15.40:0/2617093594 >> v1:172.21.15.43:6789/0 conn(0x7f2f880702e0 legacy=0x7f2f88159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:08.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8e2fd700 1 -- 172.21.15.40:0/2617093594 >> v1:172.21.15.40:6789/0 conn(0x7f2f880c78a0 legacy=0x7f2f8814fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:08.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8d2fb700 1 -- 172.21.15.40:0/2617093594 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1471402464 0 0) 0x7f2f8806b870 con 0x7f2f880c78a0 2024-09-13T18:21:08.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8e2fd700 1 -- 172.21.15.40:0/2617093594 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f88070cd0 con 0x7f2f88159890 2024-09-13T18:21:08.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8e2fd700 1 --2- 172.21.15.40:0/2617093594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f88159890 0x7f2f880722c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009850 tx=0x7f2f84009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587e983b5c7fd512 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:08.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8d2fb700 1 -- 172.21.15.40:0/2617093594 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f84011830 con 0x7f2f88159890 2024-09-13T18:21:08.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8d2fb700 1 -- 172.21.15.40:0/2617093594 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f84011990 con 0x7f2f88159890 2024-09-13T18:21:08.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8d2fb700 1 -- 172.21.15.40:0/2617093594 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f84011d20 con 0x7f2f88159890 2024-09-13T18:21:08.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8f2ff700 1 -- 172.21.15.40:0/2617093594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f88159890 msgr2=0x7f2f880722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:08.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8f2ff700 1 --2- 172.21.15.40:0/2617093594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f88159890 0x7f2f880722c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009850 tx=0x7f2f84009b60 comp rx=0 tx=0).stop 2024-09-13T18:21:08.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8f2ff700 1 -- 172.21.15.40:0/2617093594 shutdown_connections 2024-09-13T18:21:08.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8f2ff700 1 --2- 172.21.15.40:0/2617093594 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f88159890 0x7f2f880722c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:08.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8f2ff700 1 -- 172.21.15.40:0/2617093594 >> 172.21.15.40:0/2617093594 conn(0x7f2f880c7d70 msgr2=0x7f2f880c8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:08.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8f2ff700 1 -- 172.21.15.40:0/2617093594 shutdown_connections 2024-09-13T18:21:08.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.597+0000 7f2f8f2ff700 1 -- 172.21.15.40:0/2617093594 wait complete. 2024-09-13T18:21:08.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f8f2ff700 1 Processor -- start 2024-09-13T18:21:08.607 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f8f2ff700 1 -- start start 2024-09-13T18:21:08.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f8f2ff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f880702e0 0x7f2f8806e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:08.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f8f2ff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f880c78a0 0x7f2f88142ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:08.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f8f2ff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f88159890 0x7f2f88148b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:08.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f8f2ff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8806df20 con 0x7f2f880702e0 2024-09-13T18:21:08.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f8f2ff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8806d9f0 con 0x7f2f88159890 2024-09-13T18:21:08.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f8f2ff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8806dc40 con 0x7f2f880c78a0 2024-09-13T18:21:08.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f8e2fd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f880702e0 0x7f2f8806e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:08.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f8e2fd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f880702e0 0x7f2f8806e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37338/0 (socket says 172.21.15.40:37338) 2024-09-13T18:21:08.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f8e2fd700 1 -- 172.21.15.40:0/3764268805 learned_addr learned my addr 172.21.15.40:0/3764268805 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:08.610 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f8eafe700 1 --2- 172.21.15.40:0/3764268805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f88159890 0x7f2f88148b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:08.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f8dafc700 1 --2- 172.21.15.40:0/3764268805 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f880c78a0 0x7f2f88142ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:08.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f8e2fd700 1 -- 172.21.15.40:0/3764268805 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f880c78a0 msgr2=0x7f2f88142ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:08.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f8e2fd700 1 --2- 172.21.15.40:0/3764268805 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f880c78a0 0x7f2f88142ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:08.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f8e2fd700 1 -- 172.21.15.40:0/3764268805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f88159890 msgr2=0x7f2f88148b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:08.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f8e2fd700 1 --2- 172.21.15.40:0/3764268805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f88159890 0x7f2f88148b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:08.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f8e2fd700 1 -- 172.21.15.40:0/3764268805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f840094b0 con 0x7f2f880702e0 2024-09-13T18:21:08.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f8eafe700 1 --2- 172.21.15.40:0/3764268805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f88159890 0x7f2f88148b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:08.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f8e2fd700 1 --2- 172.21.15.40:0/3764268805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f880702e0 0x7f2f8806e180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009fd0 tx=0x7f2f84004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:08.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.601+0000 7f2f7effd700 1 -- 172.21.15.40:0/3764268805 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8400bc10 con 0x7f2f880702e0 2024-09-13T18:21:08.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.605+0000 7f2f7effd700 1 -- 172.21.15.40:0/3764268805 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8401e070 con 0x7f2f880702e0 2024-09-13T18:21:08.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.605+0000 7f2f8f2ff700 1 -- 172.21.15.40:0/3764268805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f880cab80 con 0x7f2f880702e0 2024-09-13T18:21:08.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.605+0000 7f2f7effd700 1 -- 172.21.15.40:0/3764268805 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8401a420 con 0x7f2f880702e0 2024-09-13T18:21:08.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.605+0000 7f2f8f2ff700 1 -- 172.21.15.40:0/3764268805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f881423a0 con 0x7f2f880702e0 2024-09-13T18:21:08.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.605+0000 7f2f7effd700 1 -- 172.21.15.40:0/3764268805 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2f8401a580 con 0x7f2f880702e0 2024-09-13T18:21:08.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.605+0000 7f2f8f2ff700 1 -- 172.21.15.40:0/3764268805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f88142d20 con 0x7f2f880702e0 2024-09-13T18:21:08.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.605+0000 7f2f7effd700 1 --2- 172.21.15.40:0/3764268805 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f6c066730 0x7f2f6c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:08.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.605+0000 7f2f7effd700 1 -- 172.21.15.40:0/3764268805 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2f84085440 con 0x7f2f880702e0 2024-09-13T18:21:08.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.605+0000 7f2f8dafc700 1 --2- 172.21.15.40:0/3764268805 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f6c066730 0x7f2f6c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:08.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.605+0000 7f2f8dafc700 1 --2- 172.21.15.40:0/3764268805 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f6c066730 0x7f2f6c068bf0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2f880c74d0 tx=0x7f2f78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:08.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.613+0000 7f2f7effd700 1 -- 172.21.15.40:0/3764268805 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2f840523f0 con 0x7f2f880702e0 2024-09-13T18:21:08.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.825+0000 7f2f8f2ff700 1 -- 172.21.15.40:0/3764268805 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f880722c0 con 0x7f2f880702e0 2024-09-13T18:21:08.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.825+0000 7f2f7effd700 1 -- 172.21.15.40:0/3764268805 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6268 (secure 0 0 0) 0x7f2f84055a10 con 0x7f2f880702e0 2024-09-13T18:21:08.832 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T18:21:08.832 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6839/1861594206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1861594206},{"type":"v1","addr":"172.21.15.40:6839","nonce":1861594206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1548394691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1548394691},{"type":"v1","addr":"172.21.15.40:6835","nonce":1548394691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/486611242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":486611242},{"type":"v1","addr":"172.21.15.40:6837","nonce":486611242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:50:34.562623+0000","modified":"2024-09-13T17:51:12.236489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4288,"mds_1":4285,"mds_2":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"f","rank":2,"incarnation":14,"state":"up:active","state_seq":12,"addr":"172.21.15.43:6833/526975022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":526975022},{"type":"v1","addr":"172.21.15.43:6833","nonce":526975022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4285":{"gid":4285,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.43:6835/2586910944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2586910944},{"type":"v1","addr":"172.21.15.43:6835","nonce":2586910944}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/334160033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":334160033},{"type":"v1","addr":"172.21.15.43:6837","nonce":334160033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:21:08.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.829+0000 7f2f8f2ff700 1 -- 172.21.15.40:0/3764268805 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f6c066730 msgr2=0x7f2f6c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:08.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.829+0000 7f2f8f2ff700 1 --2- 172.21.15.40:0/3764268805 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f6c066730 0x7f2f6c068bf0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2f880c74d0 tx=0x7f2f78009040 comp rx=0 tx=0).stop 2024-09-13T18:21:08.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.829+0000 7f2f8f2ff700 1 -- 172.21.15.40:0/3764268805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f880702e0 msgr2=0x7f2f8806e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:08.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.829+0000 7f2f8f2ff700 1 --2- 172.21.15.40:0/3764268805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f880702e0 0x7f2f8806e180 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009fd0 tx=0x7f2f84004770 comp rx=0 tx=0).stop 2024-09-13T18:21:08.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.829+0000 7f2f8f2ff700 1 -- 172.21.15.40:0/3764268805 shutdown_connections 2024-09-13T18:21:08.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.829+0000 7f2f8f2ff700 1 --2- 172.21.15.40:0/3764268805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f88159890 0x7f2f88148b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:08.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.829+0000 7f2f8f2ff700 1 --2- 172.21.15.40:0/3764268805 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f6c066730 0x7f2f6c068bf0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:08.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.829+0000 7f2f8f2ff700 1 --2- 172.21.15.40:0/3764268805 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2f880c78a0 0x7f2f88142ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:08.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.829+0000 7f2f8f2ff700 1 --2- 172.21.15.40:0/3764268805 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f880702e0 0x7f2f8806e180 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:08.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.829+0000 7f2f8f2ff700 1 -- 172.21.15.40:0/3764268805 >> 172.21.15.40:0/3764268805 conn(0x7f2f880c7d70 msgr2=0x7f2f880c8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:08.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.829+0000 7f2f8f2ff700 1 -- 172.21.15.40:0/3764268805 shutdown_connections 2024-09-13T18:21:08.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.829+0000 7f2f8f2ff700 1 -- 172.21.15.40:0/3764268805 wait complete. 2024-09-13T18:21:08.836 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 15 2024-09-13T18:21:08.848 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-13T18:21:08.848 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-13T18:21:08.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf8f9d700 1 Processor -- start 2024-09-13T18:21:08.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf8f9d700 1 -- start start 2024-09-13T18:21:08.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf8f9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feaf4071f70 0x7feaf40c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:08.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf8f9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feaf4072350 0x7feaf4150d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:08.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf8f9d700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf40c55c0 con 0x7feaf40c7200 2024-09-13T18:21:08.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf8f9d700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf406a390 con 0x7feaf4071f70 2024-09-13T18:21:08.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf8f9d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf4157710 con 0x7feaf4072350 2024-09-13T18:21:08.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf37fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feaf4071f70 0x7feaf40c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:08.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf37fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7feaf4071f70 0x7feaf40c5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37348/0 (socket says 172.21.15.40:37348) 2024-09-13T18:21:08.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf37fe700 1 -- 172.21.15.40:0/346018840 learned_addr learned my addr 172.21.15.40:0/346018840 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:08.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf2ffd700 1 --2- 172.21.15.40:0/346018840 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feaf4072350 0x7feaf4150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:08.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf37fe700 1 -- 172.21.15.40:0/346018840 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feaf4072350 msgr2=0x7feaf4150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:08.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf37fe700 1 --2- 172.21.15.40:0/346018840 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feaf4072350 0x7feaf4150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:08.964 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf37fe700 1 -- 172.21.15.40:0/346018840 >> v1:172.21.15.40:6789/0 conn(0x7feaf40c7200 legacy=0x7feaf415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:08.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf37fe700 1 -- 172.21.15.40:0/346018840 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf41579d0 con 0x7feaf4071f70 2024-09-13T18:21:08.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf37fe700 1 --2- 172.21.15.40:0/346018840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf4071f70 0x7feaf40c5d30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feadc009910 tx=0x7feadc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f115f749edf3ad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:08.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf27fc700 1 -- 172.21.15.40:0/346018840 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc004230 con 0x7feaf4071f70 2024-09-13T18:21:08.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf27fc700 1 -- 172.21.15.40:0/346018840 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc004390 con 0x7feaf4071f70 2024-09-13T18:21:08.965 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf27fc700 1 -- 172.21.15.40:0/346018840 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc009e60 con 0x7feaf4071f70 2024-09-13T18:21:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf8f9d700 1 -- 172.21.15.40:0/346018840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf4071f70 msgr2=0x7feaf40c5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf8f9d700 1 --2- 172.21.15.40:0/346018840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf4071f70 0x7feaf40c5d30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feadc009910 tx=0x7feadc009c20 comp rx=0 tx=0).stop 2024-09-13T18:21:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf8f9d700 1 -- 172.21.15.40:0/346018840 shutdown_connections 2024-09-13T18:21:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf8f9d700 1 --2- 172.21.15.40:0/346018840 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feaf4072350 0x7feaf4150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf8f9d700 1 --2- 172.21.15.40:0/346018840 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf4071f70 0x7feaf40c5d30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf8f9d700 1 -- 172.21.15.40:0/346018840 >> 172.21.15.40:0/346018840 conn(0x7feaf40cabc0 msgr2=0x7feaf40cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf8f9d700 1 -- 172.21.15.40:0/346018840 shutdown_connections 2024-09-13T18:21:08.966 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.957+0000 7feaf8f9d700 1 -- 172.21.15.40:0/346018840 wait complete. 2024-09-13T18:21:08.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.961+0000 7feaf8f9d700 1 Processor -- start 2024-09-13T18:21:08.967 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.961+0000 7feaf8f9d700 1 -- start start 2024-09-13T18:21:08.969 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.961+0000 7feaf8f9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf4071f70 0x7feaf4143790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:08.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.961+0000 7feaf8f9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaf4072350 0x7feaf41eff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:08.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.961+0000 7feaf8f9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feaf40c7200 0x7feaf4153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:08.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.961+0000 7feaf8f9d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf4154830 con 0x7feaf4071f70 2024-09-13T18:21:08.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.961+0000 7feaf8f9d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf4152080 con 0x7feaf4072350 2024-09-13T18:21:08.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.961+0000 7feaf8f9d700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf4156de0 con 0x7feaf40c7200 2024-09-13T18:21:08.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.961+0000 7feaf37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf4071f70 0x7feaf4143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:08.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.961+0000 7feaf37fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf4071f70 0x7feaf4143790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37354/0 (socket says 172.21.15.40:37354) 2024-09-13T18:21:08.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.961+0000 7feaf37fe700 1 -- 172.21.15.40:0/1658057326 learned_addr learned my addr 172.21.15.40:0/1658057326 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:08.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.961+0000 7feaf2ffd700 1 --2- 172.21.15.40:0/1658057326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaf4072350 0x7feaf41eff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:08.970 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.961+0000 7feaf3fff700 1 --2- 172.21.15.40:0/1658057326 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feaf40c7200 0x7feaf4153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:08.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.961+0000 7feaf37fe700 1 -- 172.21.15.40:0/1658057326 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feaf40c7200 msgr2=0x7feaf4153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:08.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.961+0000 7feaf37fe700 1 --2- 172.21.15.40:0/1658057326 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feaf40c7200 0x7feaf4153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:08.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.961+0000 7feaf37fe700 1 -- 172.21.15.40:0/1658057326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaf4072350 msgr2=0x7feaf41eff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:08.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.961+0000 7feaf37fe700 1 --2- 172.21.15.40:0/1658057326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaf4072350 0x7feaf41eff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:08.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.961+0000 7feaf37fe700 1 -- 172.21.15.40:0/1658057326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feadc009570 con 0x7feaf4071f70 2024-09-13T18:21:08.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.961+0000 7feaf2ffd700 1 --2- 172.21.15.40:0/1658057326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaf4072350 0x7feaf41eff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:08.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.961+0000 7feaf37fe700 1 --2- 172.21.15.40:0/1658057326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf4071f70 0x7feaf4143790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feadc004000 tx=0x7feadc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:08.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.965+0000 7feaebfff700 1 -- 172.21.15.40:0/1658057326 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc00bdb0 con 0x7feaf4071f70 2024-09-13T18:21:08.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.965+0000 7feaebfff700 1 -- 172.21.15.40:0/1658057326 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc01f070 con 0x7feaf4071f70 2024-09-13T18:21:08.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.965+0000 7feaf8f9d700 1 -- 172.21.15.40:0/1658057326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf41570a0 con 0x7feaf4071f70 2024-09-13T18:21:08.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.965+0000 7feaebfff700 1 -- 172.21.15.40:0/1658057326 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc003900 con 0x7feaf4071f70 2024-09-13T18:21:08.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.965+0000 7feaf8f9d700 1 -- 172.21.15.40:0/1658057326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf4151a30 con 0x7feaf4071f70 2024-09-13T18:21:08.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.965+0000 7feaf8f9d700 1 -- 172.21.15.40:0/1658057326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf40cb6e0 con 0x7feaf4071f70 2024-09-13T18:21:08.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.965+0000 7feaebfff700 1 -- 172.21.15.40:0/1658057326 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feadc021030 con 0x7feaf4071f70 2024-09-13T18:21:08.972 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.965+0000 7feaebfff700 1 --2- 172.21.15.40:0/1658057326 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fead4066760 0x7fead4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:08.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.965+0000 7feaf2ffd700 1 --2- 172.21.15.40:0/1658057326 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fead4066760 0x7fead4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:08.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.965+0000 7feaebfff700 1 -- 172.21.15.40:0/1658057326 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7feadc085a40 con 0x7feaf4071f70 2024-09-13T18:21:08.973 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.965+0000 7feaf2ffd700 1 --2- 172.21.15.40:0/1658057326 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fead4066760 0x7fead4068c20 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7feaf41441b0 tx=0x7feae400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:08.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:08.973+0000 7feaebfff700 1 -- 172.21.15.40:0/1658057326 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7feadc087e60 con 0x7feaf4071f70 2024-09-13T18:21:09.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.213+0000 7feaf8f9d700 1 -- 172.21.15.40:0/1658057326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) v1 -- 0x7feaf40cb6e0 con 0x7feaf4071f70 2024-09-13T18:21:09.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.217+0000 7feaebfff700 1 -- 172.21.15.40:0/1658057326 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) v1 ==== 61+0+1541 (secure 0 0 0) 0x7feadc052a80 con 0x7feaf4071f70 2024-09-13T18:21:09.222 INFO:teuthology.orchestra.run.smithi040.stdout:{"election_epoch":240,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","modified":"2024-09-13T17:49:34.123582Z","created":"2024-09-13T17:49:34.123582Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-13T18:21:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.221+0000 7feaf8f9d700 1 -- 172.21.15.40:0/1658057326 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fead4066760 msgr2=0x7fead4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.221+0000 7feaf8f9d700 1 --2- 172.21.15.40:0/1658057326 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fead4066760 0x7fead4068c20 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7feaf41441b0 tx=0x7feae400b040 comp rx=0 tx=0).stop 2024-09-13T18:21:09.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.221+0000 7feaf8f9d700 1 -- 172.21.15.40:0/1658057326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf4071f70 msgr2=0x7feaf4143790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:09.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.221+0000 7feaf8f9d700 1 --2- 172.21.15.40:0/1658057326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf4071f70 0x7feaf4143790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feadc004000 tx=0x7feadc004770 comp rx=0 tx=0).stop 2024-09-13T18:21:09.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.221+0000 7feaf8f9d700 1 -- 172.21.15.40:0/1658057326 shutdown_connections 2024-09-13T18:21:09.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.221+0000 7feaf8f9d700 1 --2- 172.21.15.40:0/1658057326 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fead4066760 0x7fead4068c20 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.221+0000 7feaf8f9d700 1 --2- 172.21.15.40:0/1658057326 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7feaf40c7200 0x7feaf4153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.221+0000 7feaf8f9d700 1 --2- 172.21.15.40:0/1658057326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feaf4072350 0x7feaf41eff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.221+0000 7feaf8f9d700 1 --2- 172.21.15.40:0/1658057326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feaf4071f70 0x7feaf4143790 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.221+0000 7feaf8f9d700 1 -- 172.21.15.40:0/1658057326 >> 172.21.15.40:0/1658057326 conn(0x7feaf40cabc0 msgr2=0x7feaf414f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:09.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.221+0000 7feaf8f9d700 1 -- 172.21.15.40:0/1658057326 shutdown_connections 2024-09-13T18:21:09.228 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.221+0000 7feaf8f9d700 1 -- 172.21.15.40:0/1658057326 wait complete. 2024-09-13T18:21:09.242 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-13T18:21:09.242 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-13T18:21:09.251 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-13T18:21:09.251 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T18:21:09.251 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:21:09.276 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T18:21:09.276 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:21:09.284 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:21:09.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.441+0000 7f10fea56700 1 Processor -- start 2024-09-13T18:21:09.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.441+0000 7f10fea56700 1 -- start start 2024-09-13T18:21:09.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fea56700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10f815e1a0 0x7f10f806a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:09.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fea56700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f10f806a8d0 0x7f10f806ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:09.450 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fea56700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f8071690 con 0x7f10f8154240 2024-09-13T18:21:09.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fea56700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f80721e0 con 0x7f10f806a8d0 2024-09-13T18:21:09.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fea56700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f8154620 con 0x7f10f815e1a0 2024-09-13T18:21:09.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fda54700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f10f8154240 0x7f10f815ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50742/0 (socket says 172.21.15.40:50742) 2024-09-13T18:21:09.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fda54700 1 -- 172.21.15.40:0/3496367264 learned_addr learned my addr 172.21.15.40:0/3496367264 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:09.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fe255700 1 --2- 172.21.15.40:0/3496367264 >> v2:172.21.15.40:3300/0 conn(0x7f10f806a8d0 0x7f10f806ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:09.451 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fd253700 1 --2- 172.21.15.40:0/3496367264 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10f815e1a0 0x7f10f806a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fe255700 1 -- 172.21.15.40:0/3496367264 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10f815e1a0 msgr2=0x7f10f806a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fe255700 1 --2- 172.21.15.40:0/3496367264 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10f815e1a0 0x7f10f806a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fe255700 1 -- 172.21.15.40:0/3496367264 >> v1:172.21.15.40:6789/0 conn(0x7f10f8154240 legacy=0x7f10f815ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fe255700 1 -- 172.21.15.40:0/3496367264 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f8071b20 con 0x7f10f806a8d0 2024-09-13T18:21:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fe255700 1 --2- 172.21.15.40:0/3496367264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f806a8d0 0x7f10f806ad40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009850 tx=0x7f10e8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635942ffe517ca81 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fca52700 1 -- 172.21.15.40:0/3496367264 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8011830 con 0x7f10f806a8d0 2024-09-13T18:21:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fca52700 1 -- 172.21.15.40:0/3496367264 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e8011990 con 0x7f10f806a8d0 2024-09-13T18:21:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fca52700 1 -- 172.21.15.40:0/3496367264 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8011d40 con 0x7f10f806a8d0 2024-09-13T18:21:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fea56700 1 -- 172.21.15.40:0/3496367264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f806a8d0 msgr2=0x7f10f806ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fea56700 1 --2- 172.21.15.40:0/3496367264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f806a8d0 0x7f10f806ad40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009850 tx=0x7f10e8009b60 comp rx=0 tx=0).stop 2024-09-13T18:21:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fea56700 1 -- 172.21.15.40:0/3496367264 shutdown_connections 2024-09-13T18:21:09.453 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fea56700 1 --2- 172.21.15.40:0/3496367264 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f806a8d0 0x7f10f806ad40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fea56700 1 --2- 172.21.15.40:0/3496367264 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10f815e1a0 0x7f10f806a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fea56700 1 -- 172.21.15.40:0/3496367264 >> 172.21.15.40:0/3496367264 conn(0x7f10f80cabc0 msgr2=0x7f10f80cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fea56700 1 -- 172.21.15.40:0/3496367264 shutdown_connections 2024-09-13T18:21:09.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.445+0000 7f10fea56700 1 -- 172.21.15.40:0/3496367264 wait complete. 2024-09-13T18:21:09.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.449+0000 7f10fea56700 1 Processor -- start 2024-09-13T18:21:09.455 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.449+0000 7f10fea56700 1 -- start start 2024-09-13T18:21:09.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.449+0000 7f10fea56700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f806a8d0 0x7f10f80c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:09.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.449+0000 7f10fea56700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f8154240 0x7f10f81f38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:09.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.449+0000 7f10fea56700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10f815e1a0 0x7f10f81f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:09.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.449+0000 7f10fea56700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f81f3410 con 0x7f10f8154240 2024-09-13T18:21:09.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.449+0000 7f10fea56700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f81f67e0 con 0x7f10f806a8d0 2024-09-13T18:21:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.449+0000 7f10fea56700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f81f6a80 con 0x7f10f815e1a0 2024-09-13T18:21:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.449+0000 7f10fd253700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f8154240 0x7f10f81f38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.449+0000 7f10fda54700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f806a8d0 0x7f10f80c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.449+0000 7f10fd253700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f8154240 0x7f10f81f38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37358/0 (socket says 172.21.15.40:37358) 2024-09-13T18:21:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.449+0000 7f10fe255700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10f815e1a0 0x7f10f81f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.449+0000 7f10fd253700 1 -- 172.21.15.40:0/4160108855 learned_addr learned my addr 172.21.15.40:0/4160108855 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.449+0000 7f10fd253700 1 -- 172.21.15.40:0/4160108855 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10f815e1a0 msgr2=0x7f10f81f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.449+0000 7f10fd253700 1 --2- 172.21.15.40:0/4160108855 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10f815e1a0 0x7f10f81f3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.449+0000 7f10fd253700 1 -- 172.21.15.40:0/4160108855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f806a8d0 msgr2=0x7f10f80c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.449+0000 7f10fd253700 1 --2- 172.21.15.40:0/4160108855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f806a8d0 0x7f10f80c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.449+0000 7f10fd253700 1 -- 172.21.15.40:0/4160108855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e80094b0 con 0x7f10f8154240 2024-09-13T18:21:09.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.449+0000 7f10fda54700 1 --2- 172.21.15.40:0/4160108855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f806a8d0 0x7f10f80c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:21:09.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.453+0000 7f10fd253700 1 --2- 172.21.15.40:0/4160108855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f8154240 0x7f10f81f38a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00e840 tx=0x7f10ec00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:09.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.453+0000 7f10f67fc700 1 -- 172.21.15.40:0/4160108855 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec0095a0 con 0x7f10f8154240 2024-09-13T18:21:09.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.453+0000 7f10f67fc700 1 -- 172.21.15.40:0/4160108855 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ec013680 con 0x7f10f8154240 2024-09-13T18:21:09.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.453+0000 7f10fea56700 1 -- 172.21.15.40:0/4160108855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f81f6d60 con 0x7f10f8154240 2024-09-13T18:21:09.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.453+0000 7f10f67fc700 1 -- 172.21.15.40:0/4160108855 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec0139f0 con 0x7f10f8154240 2024-09-13T18:21:09.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.453+0000 7f10fea56700 1 -- 172.21.15.40:0/4160108855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f81f7320 con 0x7f10f8154240 2024-09-13T18:21:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.453+0000 7f10f67fc700 1 -- 172.21.15.40:0/4160108855 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f10ec013b50 con 0x7f10f8154240 2024-09-13T18:21:09.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.457+0000 7f10f67fc700 1 --2- 172.21.15.40:0/4160108855 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f10e4066760 0x7f10e4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:09.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.457+0000 7f10f67fc700 1 -- 172.21.15.40:0/4160108855 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f10ec0846e0 con 0x7f10f8154240 2024-09-13T18:21:09.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.457+0000 7f10fda54700 1 --2- 172.21.15.40:0/4160108855 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f10e4066760 0x7f10e4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:09.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.457+0000 7f10fea56700 1 -- 172.21.15.40:0/4160108855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f8071320 con 0x7f10f8154240 2024-09-13T18:21:09.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.457+0000 7f10fda54700 1 --2- 172.21.15.40:0/4160108855 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f10e4066760 0x7f10e4068c20 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f10f0005cb0 tx=0x7f10f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:09.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.465+0000 7f10f67fc700 1 -- 172.21.15.40:0/4160108855 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f10ec051670 con 0x7f10f8154240 2024-09-13T18:21:09.644 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.637+0000 7f10fea56700 1 -- 172.21.15.40:0/4160108855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10f806b370 con 0x7f10f8154240 2024-09-13T18:21:09.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.641+0000 7f10f67fc700 1 -- 172.21.15.40:0/4160108855 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18650 (secure 0 0 0) 0x7f10ec021070 con 0x7f10f8154240 2024-09-13T18:21:09.647 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T18:21:09.648 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":27,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:51:16.266176+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:50:32.740336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:50:33.552914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:50:36.623639+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:21:09.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.641+0000 7f10fea56700 1 -- 172.21.15.40:0/4160108855 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f10e4066760 msgr2=0x7f10e4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:09.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.641+0000 7f10fea56700 1 --2- 172.21.15.40:0/4160108855 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f10e4066760 0x7f10e4068c20 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f10f0005cb0 tx=0x7f10f0009040 comp rx=0 tx=0).stop 2024-09-13T18:21:09.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.641+0000 7f10fea56700 1 -- 172.21.15.40:0/4160108855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f8154240 msgr2=0x7f10f81f38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:09.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.641+0000 7f10fea56700 1 --2- 172.21.15.40:0/4160108855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f8154240 0x7f10f81f38a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00e840 tx=0x7f10ec00eb50 comp rx=0 tx=0).stop 2024-09-13T18:21:09.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.645+0000 7f10fea56700 1 -- 172.21.15.40:0/4160108855 shutdown_connections 2024-09-13T18:21:09.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.645+0000 7f10fea56700 1 --2- 172.21.15.40:0/4160108855 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f10e4066760 0x7f10e4068c20 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.650 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.645+0000 7f10fea56700 1 --2- 172.21.15.40:0/4160108855 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f10f815e1a0 0x7f10f81f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.645+0000 7f10fea56700 1 --2- 172.21.15.40:0/4160108855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f8154240 0x7f10f81f38a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.645+0000 7f10fea56700 1 --2- 172.21.15.40:0/4160108855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f806a8d0 0x7f10f80c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.645+0000 7f10fea56700 1 -- 172.21.15.40:0/4160108855 >> 172.21.15.40:0/4160108855 conn(0x7f10f80cabc0 msgr2=0x7f10f80cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:09.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.645+0000 7f10fea56700 1 -- 172.21.15.40:0/4160108855 shutdown_connections 2024-09-13T18:21:09.651 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.645+0000 7f10fea56700 1 -- 172.21.15.40:0/4160108855 wait complete. 2024-09-13T18:21:09.664 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:50:21.057171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:50:32.740336+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:50:33.552914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:50:36.623639+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:21:09.664 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:21:09.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.777+0000 7fbf62b07700 1 Processor -- start 2024-09-13T18:21:09.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.777+0000 7fbf62b07700 1 -- start start 2024-09-13T18:21:09.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.777+0000 7fbf62b07700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf5c150720 0x7fbf5c0c7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:09.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.777+0000 7fbf62b07700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbf5c0b1fb0 0x7fbf5c071f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.777+0000 7fbf62b07700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf5c06bef0 con 0x7fbf5c0c7e40 2024-09-13T18:21:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.777+0000 7fbf62b07700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5c06bad0 con 0x7fbf5c0b1fb0 2024-09-13T18:21:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.777+0000 7fbf62b07700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c06b4a0 con 0x7fbf5c150720 2024-09-13T18:21:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.777+0000 7fbf610a4700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbf5c0c7e40 0x7fbf5c1564c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50748/0 (socket says 172.21.15.40:50748) 2024-09-13T18:21:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.777+0000 7fbf610a4700 1 -- 172.21.15.40:0/3547198409 learned_addr learned my addr 172.21.15.40:0/3547198409 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.777+0000 7fbf5bfff700 1 --2- 172.21.15.40:0/3547198409 >> v2:172.21.15.43:3300/0 conn(0x7fbf5c0b1fb0 0x7fbf5c071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.777+0000 7fbf608a3700 1 --2- 172.21.15.40:0/3547198409 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf5c150720 0x7fbf5c0c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.781+0000 7fbf5b7fe700 1 -- 172.21.15.40:0/3547198409 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56778717 0 0) 0x7fbf5c06bef0 con 0x7fbf5c0c7e40 2024-09-13T18:21:09.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.781+0000 7fbf5b7fe700 1 -- 172.21.15.40:0/3547198409 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf3c0035d0 con 0x7fbf5c0c7e40 2024-09-13T18:21:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.781+0000 7fbf608a3700 1 -- 172.21.15.40:0/3547198409 >> v2:172.21.15.43:3300/0 conn(0x7fbf5c0b1fb0 msgr2=0x7fbf5c071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.781+0000 7fbf608a3700 1 --2- 172.21.15.40:0/3547198409 >> v2:172.21.15.43:3300/0 conn(0x7fbf5c0b1fb0 0x7fbf5c071f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.781+0000 7fbf608a3700 1 -- 172.21.15.40:0/3547198409 >> v1:172.21.15.40:6789/0 conn(0x7fbf5c0c7e40 legacy=0x7fbf5c1564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.781+0000 7fbf608a3700 1 -- 172.21.15.40:0/3547198409 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c0c6c10 con 0x7fbf5c150720 2024-09-13T18:21:09.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.781+0000 7fbf5bfff700 1 --2- 172.21.15.40:0/3547198409 >> v2:172.21.15.43:3300/0 conn(0x7fbf5c0b1fb0 0x7fbf5c071f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:21:09.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.781+0000 7fbf608a3700 1 --2- 172.21.15.40:0/3547198409 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf5c150720 0x7fbf5c0c7900 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009850 tx=0x7fbf4c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4e006426bf89d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:09.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.781+0000 7fbf5b7fe700 1 -- 172.21.15.40:0/3547198409 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c011830 con 0x7fbf5c150720 2024-09-13T18:21:09.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.781+0000 7fbf5b7fe700 1 -- 172.21.15.40:0/3547198409 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4c011990 con 0x7fbf5c150720 2024-09-13T18:21:09.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.781+0000 7fbf5b7fe700 1 -- 172.21.15.40:0/3547198409 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c011d00 con 0x7fbf5c150720 2024-09-13T18:21:09.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.781+0000 7fbf62b07700 1 -- 172.21.15.40:0/3547198409 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf5c150720 msgr2=0x7fbf5c0c7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:09.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.781+0000 7fbf62b07700 1 --2- 172.21.15.40:0/3547198409 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf5c150720 0x7fbf5c0c7900 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009850 tx=0x7fbf4c009b60 comp rx=0 tx=0).stop 2024-09-13T18:21:09.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.781+0000 7fbf62b07700 1 -- 172.21.15.40:0/3547198409 shutdown_connections 2024-09-13T18:21:09.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.781+0000 7fbf62b07700 1 --2- 172.21.15.40:0/3547198409 >> v2:172.21.15.43:3300/0 conn(0x7fbf5c0b1fb0 0x7fbf5c071f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.781+0000 7fbf62b07700 1 --2- 172.21.15.40:0/3547198409 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf5c150720 0x7fbf5c0c7900 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.781+0000 7fbf62b07700 1 -- 172.21.15.40:0/3547198409 >> 172.21.15.40:0/3547198409 conn(0x7fbf5c0c88e0 msgr2=0x7fbf5c0c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:09.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.781+0000 7fbf62b07700 1 -- 172.21.15.40:0/3547198409 shutdown_connections 2024-09-13T18:21:09.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.781+0000 7fbf62b07700 1 -- 172.21.15.40:0/3547198409 wait complete. 2024-09-13T18:21:09.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf62b07700 1 Processor -- start 2024-09-13T18:21:09.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf62b07700 1 -- start start 2024-09-13T18:21:09.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf62b07700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf5c0b1fb0 0x7fbf5c1ee160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:09.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf62b07700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf5c0c7e40 0x7fbf5c158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:09.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf62b07700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf5c150720 0x7fbf5c157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:09.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf62b07700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c153360 con 0x7fbf5c150720 2024-09-13T18:21:09.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf62b07700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c1eee00 con 0x7fbf5c0c7e40 2024-09-13T18:21:09.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf62b07700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c153780 con 0x7fbf5c0b1fb0 2024-09-13T18:21:09.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf610a4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf5c150720 0x7fbf5c157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:09.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf610a4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf5c150720 0x7fbf5c157c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37360/0 (socket says 172.21.15.40:37360) 2024-09-13T18:21:09.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf610a4700 1 -- 172.21.15.40:0/2980867128 learned_addr learned my addr 172.21.15.40:0/2980867128 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:09.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf5bfff700 1 --2- 172.21.15.40:0/2980867128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf5c0c7e40 0x7fbf5c158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:09.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf608a3700 1 --2- 172.21.15.40:0/2980867128 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf5c0b1fb0 0x7fbf5c1ee160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:09.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf610a4700 1 -- 172.21.15.40:0/2980867128 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf5c0b1fb0 msgr2=0x7fbf5c1ee160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:09.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf610a4700 1 --2- 172.21.15.40:0/2980867128 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf5c0b1fb0 0x7fbf5c1ee160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf610a4700 1 -- 172.21.15.40:0/2980867128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf5c0c7e40 msgr2=0x7fbf5c158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:09.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf610a4700 1 --2- 172.21.15.40:0/2980867128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf5c0c7e40 0x7fbf5c158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf610a4700 1 -- 172.21.15.40:0/2980867128 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf440094b0 con 0x7fbf5c150720 2024-09-13T18:21:09.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf610a4700 1 --2- 172.21.15.40:0/2980867128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf5c150720 0x7fbf5c157c10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400caa0 tx=0x7fbf5400cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:09.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf597fa700 1 -- 172.21.15.40:0/2980867128 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54007480 con 0x7fbf5c150720 2024-09-13T18:21:09.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf597fa700 1 -- 172.21.15.40:0/2980867128 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf540075e0 con 0x7fbf5c150720 2024-09-13T18:21:09.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf62b07700 1 -- 172.21.15.40:0/2980867128 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf4c0094b0 con 0x7fbf5c150720 2024-09-13T18:21:09.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf597fa700 1 -- 172.21.15.40:0/2980867128 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54007970 con 0x7fbf5c150720 2024-09-13T18:21:09.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.785+0000 7fbf62b07700 1 -- 172.21.15.40:0/2980867128 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5c1eeb20 con 0x7fbf5c150720 2024-09-13T18:21:09.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.789+0000 7fbf62b07700 1 -- 172.21.15.40:0/2980867128 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf5c1534a0 con 0x7fbf5c150720 2024-09-13T18:21:09.796 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.789+0000 7fbf597fa700 1 -- 172.21.15.40:0/2980867128 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbf5401b3b0 con 0x7fbf5c150720 2024-09-13T18:21:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.789+0000 7fbf597fa700 1 --2- 172.21.15.40:0/2980867128 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbf40066730 0x7fbf40068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.789+0000 7fbf597fa700 1 -- 172.21.15.40:0/2980867128 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbf54083ec0 con 0x7fbf5c150720 2024-09-13T18:21:09.797 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.789+0000 7fbf608a3700 1 --2- 172.21.15.40:0/2980867128 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbf40066730 0x7fbf40068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:09.798 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.793+0000 7fbf608a3700 1 --2- 172.21.15.40:0/2980867128 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbf40066730 0x7fbf40068bf0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0090b0 tx=0x7fbf4c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:09.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.797+0000 7fbf597fa700 1 -- 172.21.15.40:0/2980867128 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbf5c1534a0 con 0x7fbf5c150720 2024-09-13T18:21:09.975 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.969+0000 7fbf62b07700 1 -- 172.21.15.40:0/2980867128 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf5c1534a0 con 0x7fbf5c150720 2024-09-13T18:21:09.976 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.969+0000 7fbf597fa700 1 -- 172.21.15.40:0/2980867128 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf5c1534a0 con 0x7fbf5c150720 2024-09-13T18:21:09.976 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-09-13T18:21:09.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.973+0000 7fbf62b07700 1 -- 172.21.15.40:0/2980867128 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbf40066730 msgr2=0x7fbf40068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:09.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.973+0000 7fbf62b07700 1 --2- 172.21.15.40:0/2980867128 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbf40066730 0x7fbf40068bf0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0090b0 tx=0x7fbf4c008030 comp rx=0 tx=0).stop 2024-09-13T18:21:09.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.973+0000 7fbf62b07700 1 -- 172.21.15.40:0/2980867128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf5c150720 msgr2=0x7fbf5c157c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:09.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.973+0000 7fbf62b07700 1 --2- 172.21.15.40:0/2980867128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf5c150720 0x7fbf5c157c10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400caa0 tx=0x7fbf5400cad0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.973+0000 7fbf62b07700 1 -- 172.21.15.40:0/2980867128 shutdown_connections 2024-09-13T18:21:09.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.973+0000 7fbf62b07700 1 --2- 172.21.15.40:0/2980867128 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf5c150720 0x7fbf5c157c10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.973+0000 7fbf62b07700 1 --2- 172.21.15.40:0/2980867128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf5c0c7e40 0x7fbf5c158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.973+0000 7fbf62b07700 1 --2- 172.21.15.40:0/2980867128 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbf40066730 0x7fbf40068bf0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.973+0000 7fbf62b07700 1 --2- 172.21.15.40:0/2980867128 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbf5c0b1fb0 0x7fbf5c1ee160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:09.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.973+0000 7fbf62b07700 1 -- 172.21.15.40:0/2980867128 >> 172.21.15.40:0/2980867128 conn(0x7fbf5c0c88e0 msgr2=0x7fbf5c14ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:09.979 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.973+0000 7fbf62b07700 1 -- 172.21.15.40:0/2980867128 shutdown_connections 2024-09-13T18:21:09.980 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:09.973+0000 7fbf62b07700 1 -- 172.21.15.40:0/2980867128 wait complete. 2024-09-13T18:21:09.991 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:21:10.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.097+0000 7ff3c257c700 1 Processor -- start 2024-09-13T18:21:10.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.097+0000 7ff3c257c700 1 -- start start 2024-09-13T18:21:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c257c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff3bc1513a0 0x7ff3bc0c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c257c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff3bc0c2950 0x7ff3bc157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c257c700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3bc0c2d80 con 0x7ff3bc066090 2024-09-13T18:21:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c257c700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3bc0c6690 con 0x7ff3bc1513a0 2024-09-13T18:21:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c257c700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3bc0c4880 con 0x7ff3bc0c2950 2024-09-13T18:21:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c157a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff3bc1513a0 0x7ff3bc0c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c0d79700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff3bc066090 0x7ff3bc0c7490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50764/0 (socket says 172.21.15.40:50764) 2024-09-13T18:21:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c0d79700 1 -- 172.21.15.40:0/2379883608 learned_addr learned my addr 172.21.15.40:0/2379883608 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c1d7b700 1 --2- 172.21.15.40:0/2379883608 >> v2:172.21.15.43:3300/0 conn(0x7ff3bc0c2950 0x7ff3bc157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:10.107 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c157a700 1 -- 172.21.15.40:0/2379883608 >> v2:172.21.15.43:3300/0 conn(0x7ff3bc0c2950 msgr2=0x7ff3bc157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:10.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c157a700 1 --2- 172.21.15.40:0/2379883608 >> v2:172.21.15.43:3300/0 conn(0x7ff3bc0c2950 0x7ff3bc157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c157a700 1 -- 172.21.15.40:0/2379883608 >> v1:172.21.15.40:6789/0 conn(0x7ff3bc066090 legacy=0x7ff3bc0c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:10.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3bbfff700 1 -- 172.21.15.40:0/2379883608 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 528276000 0 0) 0x7ff3bc0c2d80 con 0x7ff3bc066090 2024-09-13T18:21:10.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c157a700 1 -- 172.21.15.40:0/2379883608 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3bc0c51b0 con 0x7ff3bc1513a0 2024-09-13T18:21:10.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c157a700 1 --2- 172.21.15.40:0/2379883608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3bc1513a0 0x7ff3bc0c6ab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0c9300 tx=0x7ff3b4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc934bbce25970d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:10.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3bbfff700 1 -- 172.21.15.40:0/2379883608 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4011830 con 0x7ff3bc1513a0 2024-09-13T18:21:10.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3bbfff700 1 -- 172.21.15.40:0/2379883608 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b4011990 con 0x7ff3bc1513a0 2024-09-13T18:21:10.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3bbfff700 1 -- 172.21.15.40:0/2379883608 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4011d20 con 0x7ff3bc1513a0 2024-09-13T18:21:10.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c257c700 1 -- 172.21.15.40:0/2379883608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3bc1513a0 msgr2=0x7ff3bc0c6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:10.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c257c700 1 --2- 172.21.15.40:0/2379883608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3bc1513a0 0x7ff3bc0c6ab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0c9300 tx=0x7ff3b4009b30 comp rx=0 tx=0).stop 2024-09-13T18:21:10.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c257c700 1 -- 172.21.15.40:0/2379883608 shutdown_connections 2024-09-13T18:21:10.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c257c700 1 --2- 172.21.15.40:0/2379883608 >> v2:172.21.15.43:3300/0 conn(0x7ff3bc0c2950 0x7ff3bc157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.109 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c257c700 1 --2- 172.21.15.40:0/2379883608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3bc1513a0 0x7ff3bc0c6ab0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c257c700 1 -- 172.21.15.40:0/2379883608 >> 172.21.15.40:0/2379883608 conn(0x7ff3bc0cb450 msgr2=0x7ff3bc0cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:10.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c257c700 1 -- 172.21.15.40:0/2379883608 shutdown_connections 2024-09-13T18:21:10.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.101+0000 7ff3c257c700 1 -- 172.21.15.40:0/2379883608 wait complete. 2024-09-13T18:21:10.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c257c700 1 Processor -- start 2024-09-13T18:21:10.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c257c700 1 -- start start 2024-09-13T18:21:10.111 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c257c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3bc066090 0x7ff3bc15b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c257c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3bc0c2950 0x7ff3bc1f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c257c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3bc1513a0 0x7ff3bc1ee320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c257c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc0cb180 con 0x7ff3bc0c2950 2024-09-13T18:21:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c257c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc1f3480 con 0x7ff3bc066090 2024-09-13T18:21:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c257c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3bc0cbbc0 con 0x7ff3bc1513a0 2024-09-13T18:21:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c0d79700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3bc0c2950 0x7ff3bc1f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c0d79700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3bc0c2950 0x7ff3bc1f27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37372/0 (socket says 172.21.15.40:37372) 2024-09-13T18:21:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c0d79700 1 -- 172.21.15.40:0/306450621 learned_addr learned my addr 172.21.15.40:0/306450621 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c157a700 1 --2- 172.21.15.40:0/306450621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3bc066090 0x7ff3bc15b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c1d7b700 1 --2- 172.21.15.40:0/306450621 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3bc1513a0 0x7ff3bc1ee320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c0d79700 1 -- 172.21.15.40:0/306450621 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3bc1513a0 msgr2=0x7ff3bc1ee320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c0d79700 1 --2- 172.21.15.40:0/306450621 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3bc1513a0 0x7ff3bc1ee320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c0d79700 1 -- 172.21.15.40:0/306450621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3bc066090 msgr2=0x7ff3bc15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:10.112 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c0d79700 1 --2- 172.21.15.40:0/306450621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3bc066090 0x7ff3bc15b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c0d79700 1 -- 172.21.15.40:0/306450621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b40094b0 con 0x7ff3bc0c2950 2024-09-13T18:21:10.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c157a700 1 --2- 172.21.15.40:0/306450621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3bc066090 0x7ff3bc15b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:21:10.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c0d79700 1 --2- 172.21.15.40:0/306450621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3bc0c2950 0x7ff3bc1f27e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000c4e0 tx=0x7ff3b000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:10.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3b9ffb700 1 -- 172.21.15.40:0/306450621 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0004ca0 con 0x7ff3bc0c2950 2024-09-13T18:21:10.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3b9ffb700 1 -- 172.21.15.40:0/306450621 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b0004e20 con 0x7ff3bc0c2950 2024-09-13T18:21:10.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c257c700 1 -- 172.21.15.40:0/306450621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3bc0caea0 con 0x7ff3bc0c2950 2024-09-13T18:21:10.113 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3b9ffb700 1 -- 172.21.15.40:0/306450621 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0007310 con 0x7ff3bc0c2950 2024-09-13T18:21:10.114 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.105+0000 7ff3c257c700 1 -- 172.21.15.40:0/306450621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3bc1edb20 con 0x7ff3bc0c2950 2024-09-13T18:21:10.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.109+0000 7ff3c257c700 1 -- 172.21.15.40:0/306450621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3bc072740 con 0x7ff3bc0c2950 2024-09-13T18:21:10.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.109+0000 7ff3b9ffb700 1 -- 172.21.15.40:0/306450621 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff3b0007470 con 0x7ff3bc0c2950 2024-09-13T18:21:10.115 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.109+0000 7ff3b9ffb700 1 --2- 172.21.15.40:0/306450621 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff3a8066730 0x7ff3a8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:10.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.109+0000 7ff3b9ffb700 1 -- 172.21.15.40:0/306450621 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff3b0084140 con 0x7ff3bc0c2950 2024-09-13T18:21:10.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.109+0000 7ff3c157a700 1 --2- 172.21.15.40:0/306450621 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff3a8066730 0x7ff3a8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:10.116 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.109+0000 7ff3c157a700 1 --2- 172.21.15.40:0/306450621 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff3a8066730 0x7ff3a8068bf0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40090b0 tx=0x7ff3b4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:10.121 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.117+0000 7ff3b9ffb700 1 -- 172.21.15.40:0/306450621 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff3b0051070 con 0x7ff3bc0c2950 2024-09-13T18:21:10.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.285+0000 7ff3c257c700 1 -- 172.21.15.40:0/306450621 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff3bc0c2630 con 0x7ff3bc0c2950 2024-09-13T18:21:10.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.285+0000 7ff3b9ffb700 1 -- 172.21.15.40:0/306450621 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff3b0007720 con 0x7ff3bc0c2950 2024-09-13T18:21:10.293 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-09-13T18:21:10.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.289+0000 7ff3c257c700 1 -- 172.21.15.40:0/306450621 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff3a8066730 msgr2=0x7ff3a8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:10.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.289+0000 7ff3c257c700 1 --2- 172.21.15.40:0/306450621 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff3a8066730 0x7ff3a8068bf0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40090b0 tx=0x7ff3b4008030 comp rx=0 tx=0).stop 2024-09-13T18:21:10.298 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.289+0000 7ff3c257c700 1 -- 172.21.15.40:0/306450621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3bc0c2950 msgr2=0x7ff3bc1f27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:10.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.289+0000 7ff3c257c700 1 --2- 172.21.15.40:0/306450621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3bc0c2950 0x7ff3bc1f27e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000c4e0 tx=0x7ff3b000c7f0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.289+0000 7ff3c257c700 1 -- 172.21.15.40:0/306450621 shutdown_connections 2024-09-13T18:21:10.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.289+0000 7ff3c257c700 1 --2- 172.21.15.40:0/306450621 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff3a8066730 0x7ff3a8068bf0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.289+0000 7ff3c257c700 1 --2- 172.21.15.40:0/306450621 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff3bc1513a0 0x7ff3bc1ee320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.299 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.289+0000 7ff3c257c700 1 --2- 172.21.15.40:0/306450621 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3bc0c2950 0x7ff3bc1f27e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.289+0000 7ff3c257c700 1 --2- 172.21.15.40:0/306450621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3bc066090 0x7ff3bc15b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.293+0000 7ff3c257c700 1 -- 172.21.15.40:0/306450621 >> 172.21.15.40:0/306450621 conn(0x7ff3bc0cb450 msgr2=0x7ff3bc0c9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:10.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.293+0000 7ff3c257c700 1 -- 172.21.15.40:0/306450621 shutdown_connections 2024-09-13T18:21:10.300 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.293+0000 7ff3c257c700 1 -- 172.21.15.40:0/306450621 wait complete. 2024-09-13T18:21:10.316 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:21:10.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.429+0000 7f636ebd1700 1 Processor -- start 2024-09-13T18:21:10.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.429+0000 7f636ebd1700 1 -- start start 2024-09-13T18:21:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636ebd1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f636815a260 0x7f63680c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636ebd1700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63680cbdc0 con 0x7f63680cf580 2024-09-13T18:21:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636ebd1700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6368153f80 con 0x7f63680cf1a0 2024-09-13T18:21:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636ebd1700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63680caed0 con 0x7f636815a260 2024-09-13T18:21:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636d3ce700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f63680cf580 0x7f63680cc250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50776/0 (socket says 172.21.15.40:50776) 2024-09-13T18:21:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636d3ce700 1 -- 172.21.15.40:0/3887785957 learned_addr learned my addr 172.21.15.40:0/3887785957 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:10.438 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636e3d0700 1 --2- 172.21.15.40:0/3887785957 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f636815a260 0x7f63680c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636cbcd700 1 -- 172.21.15.40:0/3887785957 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2399667085 0 0) 0x7f63680cbdc0 con 0x7f63680cf580 2024-09-13T18:21:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636cbcd700 1 -- 172.21.15.40:0/3887785957 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63540035d0 con 0x7f63680cf580 2024-09-13T18:21:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636cbcd700 1 -- 172.21.15.40:0/3887785957 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 798268343 0 0) 0x7f6368153f80 con 0x7f63680cf1a0 2024-09-13T18:21:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636cbcd700 1 -- 172.21.15.40:0/3887785957 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63680cbdc0 con 0x7f63680cf1a0 2024-09-13T18:21:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636cbcd700 1 -- 172.21.15.40:0/3887785957 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2458917150 0 0) 0x7f63540035d0 con 0x7f63680cf580 2024-09-13T18:21:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636e3d0700 1 -- 172.21.15.40:0/3887785957 >> v1:172.21.15.43:6789/0 conn(0x7f63680cf1a0 legacy=0x7f63680c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636e3d0700 1 -- 172.21.15.40:0/3887785957 >> v1:172.21.15.40:6789/0 conn(0x7f63680cf580 legacy=0x7f63680cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636e3d0700 1 -- 172.21.15.40:0/3887785957 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63680cb1b0 con 0x7f636815a260 2024-09-13T18:21:10.440 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636e3d0700 1 --2- 172.21.15.40:0/3887785957 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f636815a260 0x7f63680c8aa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6360009880 tx=0x7f6360009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd83971e9e615b8c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636cbcd700 1 -- 172.21.15.40:0/3887785957 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63640019f0 con 0x7f636815a260 2024-09-13T18:21:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636cbcd700 1 -- 172.21.15.40:0/3887785957 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6360004030 con 0x7f636815a260 2024-09-13T18:21:10.441 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636cbcd700 1 -- 172.21.15.40:0/3887785957 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63600043c0 con 0x7f636815a260 2024-09-13T18:21:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636ebd1700 1 -- 172.21.15.40:0/3887785957 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f636815a260 msgr2=0x7f63680c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636ebd1700 1 --2- 172.21.15.40:0/3887785957 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f636815a260 0x7f63680c8aa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6360009880 tx=0x7f6360009b90 comp rx=0 tx=0).stop 2024-09-13T18:21:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636ebd1700 1 -- 172.21.15.40:0/3887785957 shutdown_connections 2024-09-13T18:21:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636ebd1700 1 --2- 172.21.15.40:0/3887785957 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f636815a260 0x7f63680c8aa0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636ebd1700 1 -- 172.21.15.40:0/3887785957 >> 172.21.15.40:0/3887785957 conn(0x7f6368066090 msgr2=0x7f6368066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636ebd1700 1 -- 172.21.15.40:0/3887785957 shutdown_connections 2024-09-13T18:21:10.442 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.433+0000 7f636ebd1700 1 -- 172.21.15.40:0/3887785957 wait complete. 2024-09-13T18:21:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636ebd1700 1 Processor -- start 2024-09-13T18:21:10.443 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636ebd1700 1 -- start start 2024-09-13T18:21:10.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636ebd1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63680cf1a0 0x7f63681f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:10.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636ebd1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63680cf580 0x7f63680b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:10.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636ebd1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f636815a260 0x7f63680c3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:10.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636ebd1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63680c47e0 con 0x7f636815a260 2024-09-13T18:21:10.445 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636ebd1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63680c30f0 con 0x7f63680cf580 2024-09-13T18:21:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636ebd1700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63681f2d40 con 0x7f63680cf1a0 2024-09-13T18:21:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636e3d0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f636815a260 0x7f63680c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636e3d0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f636815a260 0x7f63680c3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37376/0 (socket says 172.21.15.40:37376) 2024-09-13T18:21:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636e3d0700 1 -- 172.21.15.40:0/3525930452 learned_addr learned my addr 172.21.15.40:0/3525930452 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636d3ce700 1 --2- 172.21.15.40:0/3525930452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63680cf580 0x7f63680b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636dbcf700 1 --2- 172.21.15.40:0/3525930452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63680cf1a0 0x7f63681f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636e3d0700 1 -- 172.21.15.40:0/3525930452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63680cf1a0 msgr2=0x7f63681f1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636e3d0700 1 --2- 172.21.15.40:0/3525930452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63680cf1a0 0x7f63681f1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636e3d0700 1 -- 172.21.15.40:0/3525930452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63680cf580 msgr2=0x7f63680b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:10.446 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636e3d0700 1 --2- 172.21.15.40:0/3525930452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63680cf580 0x7f63680b2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636e3d0700 1 -- 172.21.15.40:0/3525930452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63600094b0 con 0x7f636815a260 2024-09-13T18:21:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636dbcf700 1 --2- 172.21.15.40:0/3525930452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63680cf1a0 0x7f63681f1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636e3d0700 1 --2- 172.21.15.40:0/3525930452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f636815a260 0x7f63680c3580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f63600090b0 tx=0x7f6360004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f635e7fc700 1 -- 172.21.15.40:0/3525930452 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6360003c50 con 0x7f636815a260 2024-09-13T18:21:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f635e7fc700 1 -- 172.21.15.40:0/3525930452 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6360003db0 con 0x7f636815a260 2024-09-13T18:21:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636ebd1700 1 -- 172.21.15.40:0/3525930452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63681f24a0 con 0x7f636815a260 2024-09-13T18:21:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f635e7fc700 1 -- 172.21.15.40:0/3525930452 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6360011850 con 0x7f636815a260 2024-09-13T18:21:10.447 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.437+0000 7f636ebd1700 1 -- 172.21.15.40:0/3525930452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63681f2a60 con 0x7f636815a260 2024-09-13T18:21:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.441+0000 7f636ebd1700 1 -- 172.21.15.40:0/3525930452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6368067a40 con 0x7f636815a260 2024-09-13T18:21:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.441+0000 7f635e7fc700 1 -- 172.21.15.40:0/3525930452 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6360018050 con 0x7f636815a260 2024-09-13T18:21:10.448 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.441+0000 7f635e7fc700 1 --2- 172.21.15.40:0/3525930452 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f634c066730 0x7f634c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:10.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.441+0000 7f635e7fc700 1 -- 172.21.15.40:0/3525930452 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6360084ff0 con 0x7f636815a260 2024-09-13T18:21:10.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.441+0000 7f636dbcf700 1 --2- 172.21.15.40:0/3525930452 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f634c066730 0x7f634c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:10.449 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.445+0000 7f636dbcf700 1 --2- 172.21.15.40:0/3525930452 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f634c066730 0x7f634c068bf0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f63580011e0 tx=0x7f6358009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:10.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.449+0000 7f635e7fc700 1 -- 172.21.15.40:0/3525930452 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f63600b6db0 con 0x7f636815a260 2024-09-13T18:21:10.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.621+0000 7f636ebd1700 1 -- 172.21.15.40:0/3525930452 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f63680c32a0 con 0x7f636815a260 2024-09-13T18:21:10.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.621+0000 7f635e7fc700 1 -- 172.21.15.40:0/3525930452 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f63680c32a0 con 0x7f636815a260 2024-09-13T18:21:10.630 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-09-13T18:21:10.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.625+0000 7f636ebd1700 1 -- 172.21.15.40:0/3525930452 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f634c066730 msgr2=0x7f634c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:10.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.625+0000 7f636ebd1700 1 --2- 172.21.15.40:0/3525930452 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f634c066730 0x7f634c068bf0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f63580011e0 tx=0x7f6358009040 comp rx=0 tx=0).stop 2024-09-13T18:21:10.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.625+0000 7f636ebd1700 1 -- 172.21.15.40:0/3525930452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f636815a260 msgr2=0x7f63680c3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:10.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.625+0000 7f636ebd1700 1 --2- 172.21.15.40:0/3525930452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f636815a260 0x7f63680c3580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f63600090b0 tx=0x7f6360004770 comp rx=0 tx=0).stop 2024-09-13T18:21:10.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.625+0000 7f636ebd1700 1 -- 172.21.15.40:0/3525930452 shutdown_connections 2024-09-13T18:21:10.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.625+0000 7f636ebd1700 1 --2- 172.21.15.40:0/3525930452 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f636815a260 0x7f63680c3580 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.625+0000 7f636ebd1700 1 --2- 172.21.15.40:0/3525930452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63680cf580 0x7f63680b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.625+0000 7f636ebd1700 1 --2- 172.21.15.40:0/3525930452 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f634c066730 0x7f634c068bf0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.625+0000 7f636ebd1700 1 --2- 172.21.15.40:0/3525930452 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f63680cf1a0 0x7f63681f1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.625+0000 7f636ebd1700 1 -- 172.21.15.40:0/3525930452 >> 172.21.15.40:0/3525930452 conn(0x7f6368066090 msgr2=0x7f63680c7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:10.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.625+0000 7f636ebd1700 1 -- 172.21.15.40:0/3525930452 shutdown_connections 2024-09-13T18:21:10.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.625+0000 7f636ebd1700 1 -- 172.21.15.40:0/3525930452 wait complete. 2024-09-13T18:21:10.644 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:21:10.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.757+0000 7f9139064700 1 Processor -- start 2024-09-13T18:21:10.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.757+0000 7f9139064700 1 -- start start 2024-09-13T18:21:10.766 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9139064700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f913415a260 0x7f91340c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:10.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9139064700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91340cbdc0 con 0x7f91340cf1a0 2024-09-13T18:21:10.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9139064700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9134153f80 con 0x7f91340cf580 2024-09-13T18:21:10.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9139064700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91340caed0 con 0x7f913415a260 2024-09-13T18:21:10.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9133fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f91340cf1a0 0x7f91340c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50788/0 (socket says 172.21.15.40:50788) 2024-09-13T18:21:10.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9133fff700 1 -- 172.21.15.40:0/1551504894 learned_addr learned my addr 172.21.15.40:0/1551504894 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:10.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9138863700 1 --2- 172.21.15.40:0/1551504894 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f913415a260 0x7f91340c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:10.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9132ffd700 1 -- 172.21.15.40:0/1551504894 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3109618259 0 0) 0x7f91340cbdc0 con 0x7f91340cf1a0 2024-09-13T18:21:10.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9132ffd700 1 -- 172.21.15.40:0/1551504894 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f911c0035d0 con 0x7f91340cf1a0 2024-09-13T18:21:10.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9132ffd700 1 -- 172.21.15.40:0/1551504894 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3018156032 0 0) 0x7f9134153f80 con 0x7f91340cf580 2024-09-13T18:21:10.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9132ffd700 1 -- 172.21.15.40:0/1551504894 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91340cbdc0 con 0x7f91340cf580 2024-09-13T18:21:10.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9132ffd700 1 -- 172.21.15.40:0/1551504894 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1474575974 0 0) 0x7f911c0035d0 con 0x7f91340cf1a0 2024-09-13T18:21:10.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9138863700 1 -- 172.21.15.40:0/1551504894 >> v1:172.21.15.43:6789/0 conn(0x7f91340cf580 legacy=0x7f91340cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:10.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9138863700 1 -- 172.21.15.40:0/1551504894 >> v1:172.21.15.40:6789/0 conn(0x7f91340cf1a0 legacy=0x7f91340c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:10.769 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9138863700 1 -- 172.21.15.40:0/1551504894 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91340cb1b0 con 0x7f913415a260 2024-09-13T18:21:10.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9138863700 1 --2- 172.21.15.40:0/1551504894 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f913415a260 0x7f91340c8aa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f912c009dc0 tx=0x7f912c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cee55f47ae1285e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:10.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9132ffd700 1 -- 172.21.15.40:0/1551504894 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c00e070 con 0x7f913415a260 2024-09-13T18:21:10.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9132ffd700 1 -- 172.21.15.40:0/1551504894 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c003a40 con 0x7f913415a260 2024-09-13T18:21:10.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9132ffd700 1 -- 172.21.15.40:0/1551504894 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9124001670 con 0x7f913415a260 2024-09-13T18:21:10.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9139064700 1 -- 172.21.15.40:0/1551504894 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f913415a260 msgr2=0x7f91340c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:10.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9139064700 1 --2- 172.21.15.40:0/1551504894 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f913415a260 0x7f91340c8aa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f912c009dc0 tx=0x7f912c00b330 comp rx=0 tx=0).stop 2024-09-13T18:21:10.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9139064700 1 -- 172.21.15.40:0/1551504894 shutdown_connections 2024-09-13T18:21:10.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9139064700 1 --2- 172.21.15.40:0/1551504894 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f913415a260 0x7f91340c8aa0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9139064700 1 -- 172.21.15.40:0/1551504894 >> 172.21.15.40:0/1551504894 conn(0x7f9134066090 msgr2=0x7f9134066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:10.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9139064700 1 -- 172.21.15.40:0/1551504894 shutdown_connections 2024-09-13T18:21:10.770 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.761+0000 7f9139064700 1 -- 172.21.15.40:0/1551504894 wait complete. 2024-09-13T18:21:10.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.765+0000 7f9139064700 1 Processor -- start 2024-09-13T18:21:10.771 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.765+0000 7f9139064700 1 -- start start 2024-09-13T18:21:10.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.765+0000 7f9139064700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91340cf1a0 0x7f91340ab730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:10.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.765+0000 7f9139064700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91340cf580 0x7f91340c2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:10.772 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.765+0000 7f9139064700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f913415a260 0x7f91340c3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:10.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.765+0000 7f9139064700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91340abc70 con 0x7f91340cf580 2024-09-13T18:21:10.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.765+0000 7f9139064700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91340aa590 con 0x7f91340cf1a0 2024-09-13T18:21:10.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.765+0000 7f9139064700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91340aab80 con 0x7f913415a260 2024-09-13T18:21:10.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.765+0000 7f91337fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91340cf580 0x7f91340c2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:10.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.765+0000 7f91337fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91340cf580 0x7f91340c2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37384/0 (socket says 172.21.15.40:37384) 2024-09-13T18:21:10.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.765+0000 7f91337fe700 1 -- 172.21.15.40:0/110004661 learned_addr learned my addr 172.21.15.40:0/110004661 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:10.773 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.765+0000 7f9133fff700 1 --2- 172.21.15.40:0/110004661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91340cf1a0 0x7f91340ab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:10.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.765+0000 7f9138863700 1 --2- 172.21.15.40:0/110004661 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f913415a260 0x7f91340c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:10.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.765+0000 7f91337fe700 1 -- 172.21.15.40:0/110004661 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f913415a260 msgr2=0x7f91340c3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:10.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.765+0000 7f91337fe700 1 --2- 172.21.15.40:0/110004661 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f913415a260 0x7f91340c3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.765+0000 7f91337fe700 1 -- 172.21.15.40:0/110004661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91340cf1a0 msgr2=0x7f91340ab730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:10.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.765+0000 7f91337fe700 1 --2- 172.21.15.40:0/110004661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91340cf1a0 0x7f91340ab730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.765+0000 7f91337fe700 1 -- 172.21.15.40:0/110004661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c009a70 con 0x7f91340cf580 2024-09-13T18:21:10.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.765+0000 7f91337fe700 1 --2- 172.21.15.40:0/110004661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91340cf580 0x7f91340c2d10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f912800c500 tx=0x7f912800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:10.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.769+0000 7f9130ff9700 1 -- 172.21.15.40:0/110004661 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91280052d0 con 0x7f91340cf580 2024-09-13T18:21:10.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.769+0000 7f9130ff9700 1 -- 172.21.15.40:0/110004661 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9128005450 con 0x7f91340cf580 2024-09-13T18:21:10.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.769+0000 7f9139064700 1 -- 172.21.15.40:0/110004661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91340c42c0 con 0x7f91340cf580 2024-09-13T18:21:10.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.769+0000 7f9130ff9700 1 -- 172.21.15.40:0/110004661 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91280038e0 con 0x7f91340cf580 2024-09-13T18:21:10.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.769+0000 7f9139064700 1 -- 172.21.15.40:0/110004661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91340c47e0 con 0x7f91340cf580 2024-09-13T18:21:10.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.769+0000 7f9139064700 1 -- 172.21.15.40:0/110004661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9134067a40 con 0x7f91340cf580 2024-09-13T18:21:10.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.773+0000 7f9130ff9700 1 -- 172.21.15.40:0/110004661 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9128003a40 con 0x7f91340cf580 2024-09-13T18:21:10.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.773+0000 7f9130ff9700 1 --2- 172.21.15.40:0/110004661 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9120066730 0x7f9120068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:10.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.773+0000 7f9133fff700 1 --2- 172.21.15.40:0/110004661 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9120066730 0x7f9120068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:10.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.773+0000 7f9130ff9700 1 -- 172.21.15.40:0/110004661 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f91280837f0 con 0x7f91340cf580 2024-09-13T18:21:10.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.773+0000 7f9133fff700 1 --2- 172.21.15.40:0/110004661 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9120066730 0x7f9120068bf0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9124001380 tx=0x7f9124000f10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:10.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.777+0000 7f9130ff9700 1 -- 172.21.15.40:0/110004661 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9128050940 con 0x7f91340cf580 2024-09-13T18:21:10.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.945+0000 7f9139064700 1 -- 172.21.15.40:0/110004661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f91340c4260 con 0x7f91340cf580 2024-09-13T18:21:10.953 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.945+0000 7f9130ff9700 1 -- 172.21.15.40:0/110004661 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9128017020 con 0x7f91340cf580 2024-09-13T18:21:10.953 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-09-13T18:21:10.957 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.949+0000 7f9139064700 1 -- 172.21.15.40:0/110004661 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9120066730 msgr2=0x7f9120068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:10.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.949+0000 7f9139064700 1 --2- 172.21.15.40:0/110004661 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9120066730 0x7f9120068bf0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9124001380 tx=0x7f9124000f10 comp rx=0 tx=0).stop 2024-09-13T18:21:10.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.949+0000 7f9139064700 1 -- 172.21.15.40:0/110004661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91340cf580 msgr2=0x7f91340c2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:10.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.949+0000 7f9139064700 1 --2- 172.21.15.40:0/110004661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91340cf580 0x7f91340c2d10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f912800c500 tx=0x7f912800c810 comp rx=0 tx=0).stop 2024-09-13T18:21:10.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.949+0000 7f9139064700 1 -- 172.21.15.40:0/110004661 shutdown_connections 2024-09-13T18:21:10.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.949+0000 7f9139064700 1 --2- 172.21.15.40:0/110004661 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9120066730 0x7f9120068bf0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.949+0000 7f9139064700 1 --2- 172.21.15.40:0/110004661 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f913415a260 0x7f91340c3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.949+0000 7f9139064700 1 --2- 172.21.15.40:0/110004661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f91340cf580 0x7f91340c2d10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.949+0000 7f9139064700 1 --2- 172.21.15.40:0/110004661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91340cf1a0 0x7f91340ab730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:10.959 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.949+0000 7f9139064700 1 -- 172.21.15.40:0/110004661 >> 172.21.15.40:0/110004661 conn(0x7f9134066090 msgr2=0x7f91340c8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:10.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.949+0000 7f9139064700 1 -- 172.21.15.40:0/110004661 shutdown_connections 2024-09-13T18:21:10.960 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:10.949+0000 7f9139064700 1 -- 172.21.15.40:0/110004661 wait complete. 2024-09-13T18:21:10.974 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T18:21:11.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.081+0000 7f1d3bf4f700 1 Processor -- start 2024-09-13T18:21:11.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.081+0000 7f1d3bf4f700 1 -- start start 2024-09-13T18:21:11.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d3bf4f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1d3415cea0 0x7f1d341500d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d3bf4f700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d3415d370 con 0x7f1d340cc290 2024-09-13T18:21:11.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d3bf4f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d3415d650 con 0x7f1d340cc670 2024-09-13T18:21:11.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d3bf4f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d341598b0 con 0x7f1d3415cea0 2024-09-13T18:21:11.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d39ceb700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1d340cc290 0x7f1d34159fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50800/0 (socket says 172.21.15.40:50800) 2024-09-13T18:21:11.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d39ceb700 1 -- 172.21.15.40:0/4291130626 learned_addr learned my addr 172.21.15.40:0/4291130626 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:11.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d394ea700 1 --2- 172.21.15.40:0/4291130626 >> v2:172.21.15.43:3300/0 conn(0x7f1d3415cea0 0x7f1d341500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d38ce9700 1 -- 172.21.15.40:0/4291130626 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1750126420 0 0) 0x7f1d3415d370 con 0x7f1d340cc290 2024-09-13T18:21:11.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d38ce9700 1 -- 172.21.15.40:0/4291130626 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d200035d0 con 0x7f1d340cc290 2024-09-13T18:21:11.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d38ce9700 1 -- 172.21.15.40:0/4291130626 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4193456317 0 0) 0x7f1d200035d0 con 0x7f1d340cc290 2024-09-13T18:21:11.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d38ce9700 1 -- 172.21.15.40:0/4291130626 >> v2:172.21.15.43:3300/0 conn(0x7f1d3415cea0 msgr2=0x7f1d341500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:11.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d38ce9700 1 --2- 172.21.15.40:0/4291130626 >> v2:172.21.15.43:3300/0 conn(0x7f1d3415cea0 0x7f1d341500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d38ce9700 1 -- 172.21.15.40:0/4291130626 >> v1:172.21.15.43:6789/0 conn(0x7f1d340cc670 legacy=0x7f1d34164230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d38ce9700 1 -- 172.21.15.40:0/4291130626 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3415d930 con 0x7f1d340cc290 2024-09-13T18:21:11.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d38ce9700 1 -- 172.21.15.40:0/4291130626 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f1d30001670 con 0x7f1d340cc290 2024-09-13T18:21:11.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d38ce9700 1 -- 172.21.15.40:0/4291130626 >> v1:172.21.15.40:6789/0 conn(0x7f1d340cc290 legacy=0x7f1d34159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d38ce9700 1 --2- 172.21.15.40:0/4291130626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d20003a20 0x7f1d20003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d38ce9700 1 --2- 172.21.15.40:0/4291130626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d20004870 0x7f1d20004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d38ce9700 1 --2- 172.21.15.40:0/4291130626 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d20005230 0x7f1d2000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d38ce9700 1 -- 172.21.15.40:0/4291130626 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d341598b0 con 0x7f1d20004870 2024-09-13T18:21:11.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d38ce9700 1 -- 172.21.15.40:0/4291130626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2000c0e0 con 0x7f1d20003a20 2024-09-13T18:21:11.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d38ce9700 1 -- 172.21.15.40:0/4291130626 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2000c2a0 con 0x7f1d20005230 2024-09-13T18:21:11.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d394ea700 1 --2- 172.21.15.40:0/4291130626 >> v2:172.21.15.43:3300/0 conn(0x7f1d3415cea0 0x7f1d341500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:21:11.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d39ceb700 1 --2- 172.21.15.40:0/4291130626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d20003a20 0x7f1d20003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d394ea700 1 --2- 172.21.15.40:0/4291130626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d20004870 0x7f1d20004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d3a4ec700 1 --2- 172.21.15.40:0/4291130626 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d20005230 0x7f1d2000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d394ea700 1 -- 172.21.15.40:0/4291130626 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d20005230 msgr2=0x7f1d2000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:11.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d394ea700 1 --2- 172.21.15.40:0/4291130626 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d20005230 0x7f1d2000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d394ea700 1 -- 172.21.15.40:0/4291130626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d20003a20 msgr2=0x7f1d20003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:11.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d394ea700 1 --2- 172.21.15.40:0/4291130626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d20003a20 0x7f1d20003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d394ea700 1 -- 172.21.15.40:0/4291130626 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d200035d0 con 0x7f1d20004870 2024-09-13T18:21:11.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d3a4ec700 1 --2- 172.21.15.40:0/4291130626 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d20005230 0x7f1d2000ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:11.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d3a4ec700 1 -- 172.21.15.40:0/4291130626 reap_dead start 2024-09-13T18:21:11.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d394ea700 1 --2- 172.21.15.40:0/4291130626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d20004870 0x7f1d20004cf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009d20 tx=0x7f1d200058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49c8d9931b8ef44 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:11.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d38ce9700 1 -- 172.21.15.40:0/4291130626 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d24009e70 con 0x7f1d20004870 2024-09-13T18:21:11.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d38ce9700 1 -- 172.21.15.40:0/4291130626 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2400e9a0 con 0x7f1d20004870 2024-09-13T18:21:11.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.085+0000 7f1d38ce9700 1 -- 172.21.15.40:0/4291130626 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2400ed10 con 0x7f1d20004870 2024-09-13T18:21:11.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d3bf4f700 1 -- 172.21.15.40:0/4291130626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d20004870 msgr2=0x7f1d20004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d3bf4f700 1 --2- 172.21.15.40:0/4291130626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d20004870 0x7f1d20004cf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009d20 tx=0x7f1d200058a0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d3bf4f700 1 -- 172.21.15.40:0/4291130626 shutdown_connections 2024-09-13T18:21:11.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d3bf4f700 1 --2- 172.21.15.40:0/4291130626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d20004870 0x7f1d20004cf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d3bf4f700 1 -- 172.21.15.40:0/4291130626 >> 172.21.15.40:0/4291130626 conn(0x7f1d34066480 msgr2=0x7f1d34066890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:11.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d3bf4f700 1 -- 172.21.15.40:0/4291130626 shutdown_connections 2024-09-13T18:21:11.097 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d3bf4f700 1 -- 172.21.15.40:0/4291130626 wait complete. 2024-09-13T18:21:11.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d3bf4f700 1 Processor -- start 2024-09-13T18:21:11.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d3bf4f700 1 -- start start 2024-09-13T18:21:11.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d3bf4f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d20004870 0x7f1d341f2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d3bf4f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d341f2bd0 0x7f1d341515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d3bf4f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d3414ee80 0x7f1d34151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d3bf4f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d34152800 con 0x7f1d20004870 2024-09-13T18:21:11.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d3bf4f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d341f3370 con 0x7f1d341f2bd0 2024-09-13T18:21:11.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d3bf4f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d34151160 con 0x7f1d3414ee80 2024-09-13T18:21:11.100 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d39ceb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d20004870 0x7f1d341f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d39ceb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d20004870 0x7f1d341f2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:37402/0 (socket says 172.21.15.40:37402) 2024-09-13T18:21:11.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d39ceb700 1 -- 172.21.15.40:0/418463072 learned_addr learned my addr 172.21.15.40:0/418463072 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:11.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d394ea700 1 --2- 172.21.15.40:0/418463072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d341f2bd0 0x7f1d341515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d3a4ec700 1 --2- 172.21.15.40:0/418463072 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d3414ee80 0x7f1d34151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d39ceb700 1 -- 172.21.15.40:0/418463072 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d3414ee80 msgr2=0x7f1d34151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d39ceb700 1 --2- 172.21.15.40:0/418463072 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d3414ee80 0x7f1d34151b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d39ceb700 1 -- 172.21.15.40:0/418463072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d341f2bd0 msgr2=0x7f1d341515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d39ceb700 1 --2- 172.21.15.40:0/418463072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d341f2bd0 0x7f1d341515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.089+0000 7f1d39ceb700 1 -- 172.21.15.40:0/418463072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3414fbc0 con 0x7f1d20004870 2024-09-13T18:21:11.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.093+0000 7f1d39ceb700 1 --2- 172.21.15.40:0/418463072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d20004870 0x7f1d341f2690 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1d300072a0 tx=0x7f1d30007660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:11.101 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.093+0000 7f1d2a7fc700 1 -- 172.21.15.40:0/418463072 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30001490 con 0x7f1d20004870 2024-09-13T18:21:11.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.093+0000 7f1d2a7fc700 1 -- 172.21.15.40:0/418463072 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3000f850 con 0x7f1d20004870 2024-09-13T18:21:11.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.093+0000 7f1d3bf4f700 1 -- 172.21.15.40:0/418463072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d34153ab0 con 0x7f1d20004870 2024-09-13T18:21:11.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.093+0000 7f1d2a7fc700 1 -- 172.21.15.40:0/418463072 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3000fc00 con 0x7f1d20004870 2024-09-13T18:21:11.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.093+0000 7f1d3bf4f700 1 -- 172.21.15.40:0/418463072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d3414f8a0 con 0x7f1d20004870 2024-09-13T18:21:11.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.093+0000 7f1d3bf4f700 1 -- 172.21.15.40:0/418463072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d340c3fd0 con 0x7f1d20004870 2024-09-13T18:21:11.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.093+0000 7f1d2a7fc700 1 -- 172.21.15.40:0/418463072 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1d3000fd60 con 0x7f1d20004870 2024-09-13T18:21:11.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.093+0000 7f1d2a7fc700 1 --2- 172.21.15.40:0/418463072 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d18066730 0x7f1d18068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.093+0000 7f1d2a7fc700 1 -- 172.21.15.40:0/418463072 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1d30084570 con 0x7f1d20004870 2024-09-13T18:21:11.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.093+0000 7f1d394ea700 1 --2- 172.21.15.40:0/418463072 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d18066730 0x7f1d18068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.097+0000 7f1d394ea700 1 --2- 172.21.15.40:0/418463072 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d18066730 0x7f1d18068bf0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1d240090b0 tx=0x7f1d24008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:11.110 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.105+0000 7f1d2a7fc700 1 -- 172.21.15.40:0/418463072 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1d30051520 con 0x7f1d20004870 2024-09-13T18:21:11.292 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.285+0000 7f1d3bf4f700 1 -- 172.21.15.40:0/418463072 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1d340c5c70 con 0x7f1d20004870 2024-09-13T18:21:11.293 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.285+0000 7f1d2a7fc700 1 -- 172.21.15.40:0/418463072 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1d30054b40 con 0x7f1d20004870 2024-09-13T18:21:11.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.289+0000 7f1d3bf4f700 1 -- 172.21.15.40:0/418463072 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d18066730 msgr2=0x7f1d18068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.289+0000 7f1d3bf4f700 1 --2- 172.21.15.40:0/418463072 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d18066730 0x7f1d18068bf0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1d240090b0 tx=0x7f1d24008030 comp rx=0 tx=0).stop 2024-09-13T18:21:11.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.289+0000 7f1d3bf4f700 1 -- 172.21.15.40:0/418463072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d20004870 msgr2=0x7f1d341f2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.289+0000 7f1d3bf4f700 1 --2- 172.21.15.40:0/418463072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d20004870 0x7f1d341f2690 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1d300072a0 tx=0x7f1d30007660 comp rx=0 tx=0).stop 2024-09-13T18:21:11.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.289+0000 7f1d3bf4f700 1 -- 172.21.15.40:0/418463072 shutdown_connections 2024-09-13T18:21:11.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.289+0000 7f1d3bf4f700 1 --2- 172.21.15.40:0/418463072 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f1d18066730 0x7f1d18068bf0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.289+0000 7f1d3bf4f700 1 --2- 172.21.15.40:0/418463072 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1d3414ee80 0x7f1d34151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.289+0000 7f1d3bf4f700 1 --2- 172.21.15.40:0/418463072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d341f2bd0 0x7f1d341515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.289+0000 7f1d3bf4f700 1 --2- 172.21.15.40:0/418463072 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d20004870 0x7f1d341f2690 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.289+0000 7f1d3bf4f700 1 -- 172.21.15.40:0/418463072 >> 172.21.15.40:0/418463072 conn(0x7f1d34066480 msgr2=0x7f1d340b3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:11.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.289+0000 7f1d3bf4f700 1 -- 172.21.15.40:0/418463072 shutdown_connections 2024-09-13T18:21:11.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.289+0000 7f1d3bf4f700 1 -- 172.21.15.40:0/418463072 wait complete. 2024-09-13T18:21:11.297 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-09-13T18:21:11.307 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-09-13T18:21:11.308 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:21:11.313 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:21:11.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.417+0000 7f5dab59e700 1 Processor -- start 2024-09-13T18:21:11.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.417+0000 7f5dab59e700 1 -- start start 2024-09-13T18:21:11.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.417+0000 7f5dab59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5dac071e60 0x7f5dac0c0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.417+0000 7f5dab59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5dac072240 0x7f5dac0c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.417+0000 7f5dab59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dac0b2210 con 0x7f5dac0c28f0 2024-09-13T18:21:11.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.417+0000 7f5dab59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dac0c3820 con 0x7f5dac072240 2024-09-13T18:21:11.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.417+0000 7f5dab59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac0c1780 con 0x7f5dac071e60 2024-09-13T18:21:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.417+0000 7f5da9d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5dac072240 0x7f5dac0c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.417+0000 7f5da9d9b700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f5dac072240 0x7f5dac0c32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48434/0 (socket says 172.21.15.40:48434) 2024-09-13T18:21:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.417+0000 7f5da9d9b700 1 -- 172.21.15.40:0/3897967318 learned_addr learned my addr 172.21.15.40:0/3897967318 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.421+0000 7f5daa59c700 1 --2- 172.21.15.40:0/3897967318 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5dac071e60 0x7f5dac0c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.421+0000 7f5da9d9b700 1 -- 172.21.15.40:0/3897967318 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5dac071e60 msgr2=0x7f5dac0c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.421+0000 7f5da9d9b700 1 --2- 172.21.15.40:0/3897967318 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5dac071e60 0x7f5dac0c0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.421+0000 7f5da9d9b700 1 -- 172.21.15.40:0/3897967318 >> v1:172.21.15.40:6789/0 conn(0x7f5dac0c28f0 legacy=0x7f5dac1565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.421+0000 7f5da9d9b700 1 -- 172.21.15.40:0/3897967318 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dac0c1a60 con 0x7f5dac072240 2024-09-13T18:21:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.421+0000 7f5da9d9b700 1 --2- 172.21.15.40:0/3897967318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dac072240 0x7f5dac0c32e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009910 tx=0x7f5da0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1baa7ce6bd32adce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.421+0000 7f5da959a700 1 -- 172.21.15.40:0/3897967318 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da000ba60 con 0x7f5dac072240 2024-09-13T18:21:11.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.421+0000 7f5da959a700 1 -- 172.21.15.40:0/3897967318 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da000bbc0 con 0x7f5dac072240 2024-09-13T18:21:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.421+0000 7f5da959a700 1 -- 172.21.15.40:0/3897967318 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da0009e60 con 0x7f5dac072240 2024-09-13T18:21:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.421+0000 7f5dab59e700 1 -- 172.21.15.40:0/3897967318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dac072240 msgr2=0x7f5dac0c32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.421+0000 7f5dab59e700 1 --2- 172.21.15.40:0/3897967318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dac072240 0x7f5dac0c32e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009910 tx=0x7f5da0009c20 comp rx=0 tx=0).stop 2024-09-13T18:21:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.421+0000 7f5dab59e700 1 -- 172.21.15.40:0/3897967318 shutdown_connections 2024-09-13T18:21:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.421+0000 7f5dab59e700 1 --2- 172.21.15.40:0/3897967318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dac072240 0x7f5dac0c32e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.421+0000 7f5dab59e700 1 --2- 172.21.15.40:0/3897967318 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5dac071e60 0x7f5dac0c0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.421+0000 7f5dab59e700 1 -- 172.21.15.40:0/3897967318 >> 172.21.15.40:0/3897967318 conn(0x7f5dac06e4f0 msgr2=0x7f5dac06e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.421+0000 7f5dab59e700 1 -- 172.21.15.40:0/3897967318 shutdown_connections 2024-09-13T18:21:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.421+0000 7f5dab59e700 1 -- 172.21.15.40:0/3897967318 wait complete. 2024-09-13T18:21:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.421+0000 7f5dab59e700 1 Processor -- start 2024-09-13T18:21:11.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.421+0000 7f5dab59e700 1 -- start start 2024-09-13T18:21:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5dab59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dac071e60 0x7f5dac06bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5dab59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5dac072240 0x7f5dac1efba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5dab59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5dac0c28f0 0x7f5dac157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5dab59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac1549e0 con 0x7f5dac071e60 2024-09-13T18:21:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5dab59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac1ef710 con 0x7f5dac072240 2024-09-13T18:21:11.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5dab59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dac06c0b0 con 0x7f5dac0c28f0 2024-09-13T18:21:11.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5daa59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dac071e60 0x7f5dac06bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5daa59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dac071e60 0x7f5dac06bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48448/0 (socket says 172.21.15.40:48448) 2024-09-13T18:21:11.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5daa59c700 1 -- 172.21.15.40:0/1909155301 learned_addr learned my addr 172.21.15.40:0/1909155301 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:11.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5da9d9b700 1 --2- 172.21.15.40:0/1909155301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5dac072240 0x7f5dac1efba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5daad9d700 1 --2- 172.21.15.40:0/1909155301 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5dac0c28f0 0x7f5dac157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5daa59c700 1 -- 172.21.15.40:0/1909155301 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5dac0c28f0 msgr2=0x7f5dac157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5daa59c700 1 --2- 172.21.15.40:0/1909155301 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5dac0c28f0 0x7f5dac157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5daa59c700 1 -- 172.21.15.40:0/1909155301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5dac072240 msgr2=0x7f5dac1efba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5daa59c700 1 --2- 172.21.15.40:0/1909155301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5dac072240 0x7f5dac1efba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5daa59c700 1 -- 172.21.15.40:0/1909155301 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da0009570 con 0x7f5dac071e60 2024-09-13T18:21:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5daa59c700 1 --2- 172.21.15.40:0/1909155301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dac071e60 0x7f5dac06bb70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00d860 tx=0x7f5d9c00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5d9affd700 1 -- 172.21.15.40:0/1909155301 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c0125f0 con 0x7f5dac071e60 2024-09-13T18:21:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5d9affd700 1 -- 172.21.15.40:0/1909155301 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9c012750 con 0x7f5dac071e60 2024-09-13T18:21:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5dab59e700 1 -- 172.21.15.40:0/1909155301 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dac06d1c0 con 0x7f5dac071e60 2024-09-13T18:21:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5d9affd700 1 -- 172.21.15.40:0/1909155301 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c012ac0 con 0x7f5dac071e60 2024-09-13T18:21:11.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.425+0000 7f5dab59e700 1 -- 172.21.15.40:0/1909155301 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dac154700 con 0x7f5dac071e60 2024-09-13T18:21:11.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.429+0000 7f5dab59e700 1 -- 172.21.15.40:0/1909155301 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dac15a610 con 0x7f5dac071e60 2024-09-13T18:21:11.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.429+0000 7f5d9affd700 1 -- 172.21.15.40:0/1909155301 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5d9c012c20 con 0x7f5dac071e60 2024-09-13T18:21:11.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.429+0000 7f5d9affd700 1 --2- 172.21.15.40:0/1909155301 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5d8c066760 0x7f5d8c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.429+0000 7f5da9d9b700 1 --2- 172.21.15.40:0/1909155301 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5d8c066760 0x7f5d8c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.429+0000 7f5d9affd700 1 -- 172.21.15.40:0/1909155301 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5d9c084630 con 0x7f5dac071e60 2024-09-13T18:21:11.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.429+0000 7f5da9d9b700 1 --2- 172.21.15.40:0/1909155301 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5d8c066760 0x7f5d8c068c20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009910 tx=0x7f5da0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:11.444 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.437+0000 7f5d9affd700 1 -- 172.21.15.40:0/1909155301 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5d9c0515f0 con 0x7f5dac071e60 2024-09-13T18:21:11.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.613+0000 7f5dab59e700 1 -- 172.21.15.40:0/1909155301 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5dac0c2da0 con 0x7f5dac071e60 2024-09-13T18:21:11.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.613+0000 7f5d9affd700 1 -- 172.21.15.40:0/1909155301 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18650 (secure 0 0 0) 0x7f5d9c012ed0 con 0x7f5dac071e60 2024-09-13T18:21:11.620 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T18:21:11.620 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":27,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:51:16.266176+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:50:32.740336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:50:33.552914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:50:36.623639+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:21:11.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.617+0000 7f5dab59e700 1 -- 172.21.15.40:0/1909155301 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5d8c066760 msgr2=0x7f5d8c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.617+0000 7f5dab59e700 1 --2- 172.21.15.40:0/1909155301 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5d8c066760 0x7f5d8c068c20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5da0009910 tx=0x7f5da0008030 comp rx=0 tx=0).stop 2024-09-13T18:21:11.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.617+0000 7f5dab59e700 1 -- 172.21.15.40:0/1909155301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dac071e60 msgr2=0x7f5dac06bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.617+0000 7f5dab59e700 1 --2- 172.21.15.40:0/1909155301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dac071e60 0x7f5dac06bb70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00d860 tx=0x7f5d9c00db70 comp rx=0 tx=0).stop 2024-09-13T18:21:11.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.617+0000 7f5dab59e700 1 -- 172.21.15.40:0/1909155301 shutdown_connections 2024-09-13T18:21:11.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.617+0000 7f5dab59e700 1 --2- 172.21.15.40:0/1909155301 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5d8c066760 0x7f5d8c068c20 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.617+0000 7f5dab59e700 1 --2- 172.21.15.40:0/1909155301 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5dac0c28f0 0x7f5dac157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.617+0000 7f5dab59e700 1 --2- 172.21.15.40:0/1909155301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5dac072240 0x7f5dac1efba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.617+0000 7f5dab59e700 1 --2- 172.21.15.40:0/1909155301 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5dac071e60 0x7f5dac06bb70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.617+0000 7f5dab59e700 1 -- 172.21.15.40:0/1909155301 >> 172.21.15.40:0/1909155301 conn(0x7f5dac06e4f0 msgr2=0x7f5dac06e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:11.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.617+0000 7f5dab59e700 1 -- 172.21.15.40:0/1909155301 shutdown_connections 2024-09-13T18:21:11.623 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.617+0000 7f5dab59e700 1 -- 172.21.15.40:0/1909155301 wait complete. 2024-09-13T18:21:11.635 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:50:21.057171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:50:32.740336+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:50:33.552914+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-13T17:50:36.623639+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-13T18:21:11.635 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:21:11.748 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.741+0000 7ff71dee6700 1 Processor -- start 2024-09-13T18:21:11.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.741+0000 7ff71dee6700 1 -- start start 2024-09-13T18:21:11.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff71dee6700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff7180c6870 0x7ff71806cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff71dee6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff7180719a0 0x7ff71806d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff71dee6700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff718070c60 con 0x7ff718150840 2024-09-13T18:21:11.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff71dee6700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff718070f40 con 0x7ff7180c6870 2024-09-13T18:21:11.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff71dee6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7180c73d0 con 0x7ff7180719a0 2024-09-13T18:21:11.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff7177fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff718150840 0x7ff71806b4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53472/0 (socket says 172.21.15.40:53472) 2024-09-13T18:21:11.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff7177fe700 1 -- 172.21.15.40:0/1931421934 learned_addr learned my addr 172.21.15.40:0/1931421934 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff70ffff700 1 --2- 172.21.15.40:0/1931421934 >> v2:172.21.15.43:3300/0 conn(0x7ff7180c6870 0x7ff71806cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff717fff700 1 --2- 172.21.15.40:0/1931421934 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff7180719a0 0x7ff71806d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff716ffd700 1 -- 172.21.15.40:0/1931421934 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3235799161 0 0) 0x7ff718070c60 con 0x7ff718150840 2024-09-13T18:21:11.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff716ffd700 1 -- 172.21.15.40:0/1931421934 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7040035d0 con 0x7ff718150840 2024-09-13T18:21:11.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff70ffff700 1 -- 172.21.15.40:0/1931421934 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff7180719a0 msgr2=0x7ff71806d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:11.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff70ffff700 1 --2- 172.21.15.40:0/1931421934 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff7180719a0 0x7ff71806d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff70ffff700 1 -- 172.21.15.40:0/1931421934 >> v1:172.21.15.40:6789/0 conn(0x7ff718150840 legacy=0x7ff71806b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff70ffff700 1 -- 172.21.15.40:0/1931421934 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7180c76b0 con 0x7ff7180c6870 2024-09-13T18:21:11.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff717fff700 1 --2- 172.21.15.40:0/1931421934 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff7180719a0 0x7ff71806d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:21:11.752 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff70ffff700 1 --2- 172.21.15.40:0/1931421934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7180c6870 0x7ff71806cb70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff700009850 tx=0x7ff700009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=997e27e4ece4202f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:11.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff716ffd700 1 -- 172.21.15.40:0/1931421934 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff700011830 con 0x7ff7180c6870 2024-09-13T18:21:11.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff716ffd700 1 -- 172.21.15.40:0/1931421934 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff700011990 con 0x7ff7180c6870 2024-09-13T18:21:11.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff716ffd700 1 -- 172.21.15.40:0/1931421934 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff700011d40 con 0x7ff7180c6870 2024-09-13T18:21:11.753 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff71dee6700 1 -- 172.21.15.40:0/1931421934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7180c6870 msgr2=0x7ff71806cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff71dee6700 1 --2- 172.21.15.40:0/1931421934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7180c6870 0x7ff71806cb70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff700009850 tx=0x7ff700009b60 comp rx=0 tx=0).stop 2024-09-13T18:21:11.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff71dee6700 1 -- 172.21.15.40:0/1931421934 shutdown_connections 2024-09-13T18:21:11.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff71dee6700 1 --2- 172.21.15.40:0/1931421934 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff7180719a0 0x7ff71806d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff71dee6700 1 --2- 172.21.15.40:0/1931421934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7180c6870 0x7ff71806cb70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff71dee6700 1 -- 172.21.15.40:0/1931421934 >> 172.21.15.40:0/1931421934 conn(0x7ff7180c8970 msgr2=0x7ff7180c8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:11.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff71dee6700 1 -- 172.21.15.40:0/1931421934 shutdown_connections 2024-09-13T18:21:11.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.745+0000 7ff71dee6700 1 -- 172.21.15.40:0/1931421934 wait complete. 2024-09-13T18:21:11.754 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff71dee6700 1 Processor -- start 2024-09-13T18:21:11.755 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff71dee6700 1 -- start start 2024-09-13T18:21:11.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff71dee6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7180719a0 0x7ff718149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff71dee6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7180c6870 0x7ff7181f0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.756 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff71dee6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff718150840 0x7ff7181465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff71dee6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7181497b0 con 0x7ff7180c6870 2024-09-13T18:21:11.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff71dee6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff718148bd0 con 0x7ff7180719a0 2024-09-13T18:21:11.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff71dee6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7181f0ab0 con 0x7ff718150840 2024-09-13T18:21:11.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff70ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7180c6870 0x7ff7181f0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff7177fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7180719a0 0x7ff718149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff7177fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7180719a0 0x7ff718149060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:55116/0 (socket says 172.21.15.40:55116) 2024-09-13T18:21:11.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff70ffff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7180c6870 0x7ff7181f0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48458/0 (socket says 172.21.15.40:48458) 2024-09-13T18:21:11.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff7177fe700 1 -- 172.21.15.40:0/2579111263 learned_addr learned my addr 172.21.15.40:0/2579111263 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:11.757 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff717fff700 1 --2- 172.21.15.40:0/2579111263 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff718150840 0x7ff7181465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff70ffff700 1 -- 172.21.15.40:0/2579111263 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff718150840 msgr2=0x7ff7181465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff70ffff700 1 --2- 172.21.15.40:0/2579111263 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff718150840 0x7ff7181465e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff70ffff700 1 -- 172.21.15.40:0/2579111263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7180719a0 msgr2=0x7ff718149060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff70ffff700 1 --2- 172.21.15.40:0/2579111263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7180719a0 0x7ff718149060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff70ffff700 1 -- 172.21.15.40:0/2579111263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff710009a70 con 0x7ff7180c6870 2024-09-13T18:21:11.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff7177fe700 1 --2- 172.21.15.40:0/2579111263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7180719a0 0x7ff718149060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:21:11.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff70ffff700 1 --2- 172.21.15.40:0/2579111263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7180c6870 0x7ff7181f0570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff7180c99f0 tx=0x7ff700004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:11.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.749+0000 7ff714ff9700 1 -- 172.21.15.40:0/2579111263 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff700004510 con 0x7ff7180c6870 2024-09-13T18:21:11.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.753+0000 7ff714ff9700 1 -- 172.21.15.40:0/2579111263 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7000036c0 con 0x7ff7180c6870 2024-09-13T18:21:11.759 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.753+0000 7ff71dee6700 1 -- 172.21.15.40:0/2579111263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7000094b0 con 0x7ff7180c6870 2024-09-13T18:21:11.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.753+0000 7ff714ff9700 1 -- 172.21.15.40:0/2579111263 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff700003a70 con 0x7ff7180c6870 2024-09-13T18:21:11.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.753+0000 7ff71dee6700 1 -- 172.21.15.40:0/2579111263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7181570a0 con 0x7ff7180c6870 2024-09-13T18:21:11.760 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.753+0000 7ff71dee6700 1 -- 172.21.15.40:0/2579111263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff718148d60 con 0x7ff7180c6870 2024-09-13T18:21:11.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.753+0000 7ff714ff9700 1 -- 172.21.15.40:0/2579111263 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff700003bd0 con 0x7ff7180c6870 2024-09-13T18:21:11.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.753+0000 7ff714ff9700 1 --2- 172.21.15.40:0/2579111263 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff6f8066730 0x7ff6f8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:11.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.753+0000 7ff714ff9700 1 -- 172.21.15.40:0/2579111263 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ff700021030 con 0x7ff7180c6870 2024-09-13T18:21:11.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.753+0000 7ff7177fe700 1 --2- 172.21.15.40:0/2579111263 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff6f8066730 0x7ff6f8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:11.761 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.753+0000 7ff7177fe700 1 --2- 172.21.15.40:0/2579111263 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff6f8066730 0x7ff6f8068bf0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff7080011e0 tx=0x7ff708009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:11.767 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.761+0000 7ff714ff9700 1 -- 172.21.15.40:0/2579111263 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff700052910 con 0x7ff7180c6870 2024-09-13T18:21:11.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.929+0000 7ff71dee6700 1 -- 172.21.15.40:0/2579111263 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff718157350 con 0x7ff7180c6870 2024-09-13T18:21:11.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.929+0000 7ff714ff9700 1 -- 172.21.15.40:0/2579111263 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff700055f30 con 0x7ff7180c6870 2024-09-13T18:21:11.937 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2024-09-13T18:21:11.942 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.937+0000 7ff71dee6700 1 -- 172.21.15.40:0/2579111263 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff6f8066730 msgr2=0x7ff6f8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.937+0000 7ff71dee6700 1 --2- 172.21.15.40:0/2579111263 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff6f8066730 0x7ff6f8068bf0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff7080011e0 tx=0x7ff708009040 comp rx=0 tx=0).stop 2024-09-13T18:21:11.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.937+0000 7ff71dee6700 1 -- 172.21.15.40:0/2579111263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7180c6870 msgr2=0x7ff7181f0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:11.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.937+0000 7ff71dee6700 1 --2- 172.21.15.40:0/2579111263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7180c6870 0x7ff7181f0570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff7180c99f0 tx=0x7ff700004770 comp rx=0 tx=0).stop 2024-09-13T18:21:11.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.937+0000 7ff71dee6700 1 -- 172.21.15.40:0/2579111263 shutdown_connections 2024-09-13T18:21:11.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.937+0000 7ff71dee6700 1 --2- 172.21.15.40:0/2579111263 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7ff6f8066730 0x7ff6f8068bf0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.937+0000 7ff71dee6700 1 --2- 172.21.15.40:0/2579111263 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff718150840 0x7ff7181465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.937+0000 7ff71dee6700 1 --2- 172.21.15.40:0/2579111263 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7180c6870 0x7ff7181f0570 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.937+0000 7ff71dee6700 1 --2- 172.21.15.40:0/2579111263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7180719a0 0x7ff718149060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:11.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.937+0000 7ff71dee6700 1 -- 172.21.15.40:0/2579111263 >> 172.21.15.40:0/2579111263 conn(0x7ff7180c8970 msgr2=0x7ff71806c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:11.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.937+0000 7ff71dee6700 1 -- 172.21.15.40:0/2579111263 shutdown_connections 2024-09-13T18:21:11.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:11.937+0000 7ff71dee6700 1 -- 172.21.15.40:0/2579111263 wait complete. 2024-09-13T18:21:11.957 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:21:12.075 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.069+0000 7f3b56fa9700 1 Processor -- start 2024-09-13T18:21:12.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.069+0000 7f3b56fa9700 1 -- start start 2024-09-13T18:21:12.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.069+0000 7f3b56fa9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3b501503c0 0x7f3b500c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:12.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.069+0000 7f3b56fa9700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3b500c8500 0x7f3b500c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:12.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.069+0000 7f3b56fa9700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b500c8f80 con 0x7f3b500c71c0 2024-09-13T18:21:12.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.069+0000 7f3b56fa9700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b500c2a80 con 0x7f3b501503c0 2024-09-13T18:21:12.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.069+0000 7f3b56fa9700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b501507a0 con 0x7f3b500c8500 2024-09-13T18:21:12.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.069+0000 7f3b55fa7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3b501503c0 0x7f3b500c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:12.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.069+0000 7f3b55fa7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3b501503c0 0x7f3b500c79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48474/0 (socket says 172.21.15.40:48474) 2024-09-13T18:21:12.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.069+0000 7f3b55fa7700 1 -- 172.21.15.40:0/4165266745 learned_addr learned my addr 172.21.15.40:0/4165266745 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:12.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.069+0000 7f3b557a6700 1 --2- 172.21.15.40:0/4165266745 >> v2:172.21.15.43:3300/0 conn(0x7f3b500c8500 0x7f3b500c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:12.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.073+0000 7f3b55fa7700 1 -- 172.21.15.40:0/4165266745 >> v2:172.21.15.43:3300/0 conn(0x7f3b500c8500 msgr2=0x7f3b500c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:12.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.073+0000 7f3b55fa7700 1 --2- 172.21.15.40:0/4165266745 >> v2:172.21.15.43:3300/0 conn(0x7f3b500c8500 0x7f3b500c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.073+0000 7f3b55fa7700 1 -- 172.21.15.40:0/4165266745 >> v1:172.21.15.40:6789/0 conn(0x7f3b500c71c0 legacy=0x7f3b50159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:12.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.073+0000 7f3b55fa7700 1 -- 172.21.15.40:0/4165266745 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b500c8a40 con 0x7f3b501503c0 2024-09-13T18:21:12.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.073+0000 7f3b55fa7700 1 --2- 172.21.15.40:0/4165266745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b501503c0 0x7f3b500c79c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3b50072b70 tx=0x7f3b4c00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc647ad0dfe741d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:12.079 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.073+0000 7f3b54fa5700 1 -- 172.21.15.40:0/4165266745 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c00dd90 con 0x7f3b501503c0 2024-09-13T18:21:12.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.073+0000 7f3b54fa5700 1 -- 172.21.15.40:0/4165266745 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4c010470 con 0x7f3b501503c0 2024-09-13T18:21:12.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.073+0000 7f3b54fa5700 1 -- 172.21.15.40:0/4165266745 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c010800 con 0x7f3b501503c0 2024-09-13T18:21:12.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.073+0000 7f3b56fa9700 1 -- 172.21.15.40:0/4165266745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b501503c0 msgr2=0x7f3b500c79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:12.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.073+0000 7f3b56fa9700 1 --2- 172.21.15.40:0/4165266745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b501503c0 0x7f3b500c79c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3b50072b70 tx=0x7f3b4c00be90 comp rx=0 tx=0).stop 2024-09-13T18:21:12.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.073+0000 7f3b56fa9700 1 -- 172.21.15.40:0/4165266745 shutdown_connections 2024-09-13T18:21:12.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.073+0000 7f3b56fa9700 1 --2- 172.21.15.40:0/4165266745 >> v2:172.21.15.43:3300/0 conn(0x7f3b500c8500 0x7f3b500c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.073+0000 7f3b56fa9700 1 --2- 172.21.15.40:0/4165266745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b501503c0 0x7f3b500c79c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.073+0000 7f3b56fa9700 1 -- 172.21.15.40:0/4165266745 >> 172.21.15.40:0/4165266745 conn(0x7f3b5006e6d0 msgr2=0x7f3b5006eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:12.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.073+0000 7f3b56fa9700 1 -- 172.21.15.40:0/4165266745 shutdown_connections 2024-09-13T18:21:12.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.073+0000 7f3b56fa9700 1 -- 172.21.15.40:0/4165266745 wait complete. 2024-09-13T18:21:12.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.073+0000 7f3b56fa9700 1 Processor -- start 2024-09-13T18:21:12.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.073+0000 7f3b56fa9700 1 -- start start 2024-09-13T18:21:12.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b56fa9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b500c71c0 0x7f3b501ebaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:12.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b56fa9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b500c8500 0x7f3b501df700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:12.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b56fa9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b501503c0 0x7f3b500c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:12.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b56fa9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5015a610 con 0x7f3b500c8500 2024-09-13T18:21:12.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b56fa9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b500c50d0 con 0x7f3b501503c0 2024-09-13T18:21:12.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b56fa9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b501ec030 con 0x7f3b500c71c0 2024-09-13T18:21:12.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b557a6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b500c8500 0x7f3b501df700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:12.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b557a6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b500c8500 0x7f3b501df700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48482/0 (socket says 172.21.15.40:48482) 2024-09-13T18:21:12.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b557a6700 1 -- 172.21.15.40:0/1335358789 learned_addr learned my addr 172.21.15.40:0/1335358789 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:12.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b567a8700 1 --2- 172.21.15.40:0/1335358789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b501503c0 0x7f3b500c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:12.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b55fa7700 1 --2- 172.21.15.40:0/1335358789 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b500c71c0 0x7f3b501ebaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:12.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b557a6700 1 -- 172.21.15.40:0/1335358789 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b500c71c0 msgr2=0x7f3b501ebaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:12.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b557a6700 1 --2- 172.21.15.40:0/1335358789 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b500c71c0 0x7f3b501ebaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b557a6700 1 -- 172.21.15.40:0/1335358789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b501503c0 msgr2=0x7f3b500c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:12.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b557a6700 1 --2- 172.21.15.40:0/1335358789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b501503c0 0x7f3b500c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b557a6700 1 -- 172.21.15.40:0/1335358789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c00b810 con 0x7f3b500c8500 2024-09-13T18:21:12.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b557a6700 1 --2- 172.21.15.40:0/1335358789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b500c8500 0x7f3b501df700 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000e7c0 tx=0x7f3b4000ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:12.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b467fc700 1 -- 172.21.15.40:0/1335358789 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40013610 con 0x7f3b500c8500 2024-09-13T18:21:12.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b467fc700 1 -- 172.21.15.40:0/1335358789 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b40013770 con 0x7f3b500c8500 2024-09-13T18:21:12.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b56fa9700 1 -- 172.21.15.40:0/1335358789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b501eebe0 con 0x7f3b500c8500 2024-09-13T18:21:12.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b467fc700 1 -- 172.21.15.40:0/1335358789 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40013ae0 con 0x7f3b500c8500 2024-09-13T18:21:12.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b56fa9700 1 -- 172.21.15.40:0/1335358789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b501ef1a0 con 0x7f3b500c8500 2024-09-13T18:21:12.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.077+0000 7f3b56fa9700 1 -- 172.21.15.40:0/1335358789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b500c6c80 con 0x7f3b500c8500 2024-09-13T18:21:12.085 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.081+0000 7f3b467fc700 1 -- 172.21.15.40:0/1335358789 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3b40013c40 con 0x7f3b500c8500 2024-09-13T18:21:12.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.081+0000 7f3b467fc700 1 --2- 172.21.15.40:0/1335358789 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3b34066760 0x7f3b34068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:12.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.081+0000 7f3b55fa7700 1 --2- 172.21.15.40:0/1335358789 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3b34066760 0x7f3b34068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:12.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.081+0000 7f3b467fc700 1 -- 172.21.15.40:0/1335358789 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3b40084570 con 0x7f3b500c8500 2024-09-13T18:21:12.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.081+0000 7f3b55fa7700 1 --2- 172.21.15.40:0/1335358789 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3b34066760 0x7f3b34068c20 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00bfd0 tx=0x7f3b4c013000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:12.091 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.085+0000 7f3b467fc700 1 -- 172.21.15.40:0/1335358789 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3b400515b0 con 0x7f3b500c8500 2024-09-13T18:21:12.260 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.253+0000 7f3b56fa9700 1 -- 172.21.15.40:0/1335358789 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3b500c0fa0 con 0x7f3b500c8500 2024-09-13T18:21:12.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.253+0000 7f3b467fc700 1 -- 172.21.15.40:0/1335358789 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3b40054bd0 con 0x7f3b500c8500 2024-09-13T18:21:12.261 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-09-13T18:21:12.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.257+0000 7f3b56fa9700 1 -- 172.21.15.40:0/1335358789 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3b34066760 msgr2=0x7f3b34068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:12.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.257+0000 7f3b56fa9700 1 --2- 172.21.15.40:0/1335358789 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3b34066760 0x7f3b34068c20 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00bfd0 tx=0x7f3b4c013000 comp rx=0 tx=0).stop 2024-09-13T18:21:12.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.257+0000 7f3b56fa9700 1 -- 172.21.15.40:0/1335358789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b500c8500 msgr2=0x7f3b501df700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:12.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.257+0000 7f3b56fa9700 1 --2- 172.21.15.40:0/1335358789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b500c8500 0x7f3b501df700 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000e7c0 tx=0x7f3b4000ead0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.257+0000 7f3b56fa9700 1 -- 172.21.15.40:0/1335358789 shutdown_connections 2024-09-13T18:21:12.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.257+0000 7f3b56fa9700 1 --2- 172.21.15.40:0/1335358789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b501503c0 0x7f3b500c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.257+0000 7f3b56fa9700 1 --2- 172.21.15.40:0/1335358789 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b500c8500 0x7f3b501df700 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.257+0000 7f3b56fa9700 1 --2- 172.21.15.40:0/1335358789 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3b34066760 0x7f3b34068c20 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.257+0000 7f3b56fa9700 1 --2- 172.21.15.40:0/1335358789 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3b500c71c0 0x7f3b501ebaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.257+0000 7f3b56fa9700 1 -- 172.21.15.40:0/1335358789 >> 172.21.15.40:0/1335358789 conn(0x7f3b5006e6d0 msgr2=0x7f3b5006a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:12.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.257+0000 7f3b56fa9700 1 -- 172.21.15.40:0/1335358789 shutdown_connections 2024-09-13T18:21:12.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.257+0000 7f3b56fa9700 1 -- 172.21.15.40:0/1335358789 wait complete. 2024-09-13T18:21:12.275 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:21:12.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.385+0000 7fe19bfff700 1 Processor -- start 2024-09-13T18:21:12.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.385+0000 7fe19bfff700 1 -- start start 2024-09-13T18:21:12.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.385+0000 7fe19bfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe19c159630 0x7fe19c159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:12.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.385+0000 7fe19bfff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe19c0c8f80 con 0x7fe19c0c8500 2024-09-13T18:21:12.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.385+0000 7fe19bfff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe19c0c2a80 con 0x7fe19c1503c0 2024-09-13T18:21:12.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.385+0000 7fe19bfff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe19c0c80d0 con 0x7fe19c159630 2024-09-13T18:21:12.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.385+0000 7fe19a7fc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe19c0c8500 0x7fe19c0c6c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53502/0 (socket says 172.21.15.40:53502) 2024-09-13T18:21:12.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.385+0000 7fe19a7fc700 1 -- 172.21.15.40:0/2618009071 learned_addr learned my addr 172.21.15.40:0/2618009071 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:12.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.385+0000 7fe19b7fe700 1 --2- 172.21.15.40:0/2618009071 >> v2:172.21.15.43:3300/0 conn(0x7fe19c159630 0x7fe19c159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:12.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.385+0000 7fe199ffb700 1 -- 172.21.15.40:0/2618009071 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3402101904 0 0) 0x7fe19c0c8f80 con 0x7fe19c0c8500 2024-09-13T18:21:12.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.389+0000 7fe199ffb700 1 -- 172.21.15.40:0/2618009071 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1840035d0 con 0x7fe19c0c8500 2024-09-13T18:21:12.394 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.389+0000 7fe19b7fe700 1 -- 172.21.15.40:0/2618009071 >> v1:172.21.15.43:6789/0 conn(0x7fe19c1503c0 legacy=0x7fe19c0c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:12.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.389+0000 7fe19b7fe700 1 -- 172.21.15.40:0/2618009071 >> v1:172.21.15.40:6789/0 conn(0x7fe19c0c8500 legacy=0x7fe19c0c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:12.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.389+0000 7fe19b7fe700 1 -- 172.21.15.40:0/2618009071 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19c1507a0 con 0x7fe19c159630 2024-09-13T18:21:12.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.389+0000 7fe19b7fe700 1 --2- 172.21.15.40:0/2618009071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe19c159630 0x7fe19c159aa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe194009d90 tx=0x7fe19400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a982e88afcf028a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:12.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.389+0000 7fe199ffb700 1 -- 172.21.15.40:0/2618009071 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19400e070 con 0x7fe19c159630 2024-09-13T18:21:12.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.389+0000 7fe199ffb700 1 -- 172.21.15.40:0/2618009071 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19400bb20 con 0x7fe19c159630 2024-09-13T18:21:12.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.389+0000 7fe199ffb700 1 -- 172.21.15.40:0/2618009071 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19400be90 con 0x7fe19c159630 2024-09-13T18:21:12.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.389+0000 7fe19bfff700 1 -- 172.21.15.40:0/2618009071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe19c159630 msgr2=0x7fe19c159aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:12.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.389+0000 7fe19bfff700 1 --2- 172.21.15.40:0/2618009071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe19c159630 0x7fe19c159aa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe194009d90 tx=0x7fe19400b330 comp rx=0 tx=0).stop 2024-09-13T18:21:12.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.389+0000 7fe19bfff700 1 -- 172.21.15.40:0/2618009071 shutdown_connections 2024-09-13T18:21:12.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.389+0000 7fe19bfff700 1 --2- 172.21.15.40:0/2618009071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe19c159630 0x7fe19c159aa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.389+0000 7fe19bfff700 1 -- 172.21.15.40:0/2618009071 >> 172.21.15.40:0/2618009071 conn(0x7fe19c06e6d0 msgr2=0x7fe19c06eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:12.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.389+0000 7fe19bfff700 1 -- 172.21.15.40:0/2618009071 shutdown_connections 2024-09-13T18:21:12.396 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.389+0000 7fe19bfff700 1 -- 172.21.15.40:0/2618009071 wait complete. 2024-09-13T18:21:12.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.389+0000 7fe19bfff700 1 Processor -- start 2024-09-13T18:21:12.397 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe19bfff700 1 -- start start 2024-09-13T18:21:12.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe19bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe19c0c8500 0x7fe19c0c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:12.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe19bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe19c1503c0 0x7fe19c1f0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:12.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe19bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe19c159630 0x7fe19c147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:12.400 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe19bfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c142c20 con 0x7fe19c159630 2024-09-13T18:21:12.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe19bfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c149110 con 0x7fe19c1503c0 2024-09-13T18:21:12.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe19bfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c149c20 con 0x7fe19c0c8500 2024-09-13T18:21:12.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe19b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe19c159630 0x7fe19c147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:12.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe19b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe19c159630 0x7fe19c147320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48496/0 (socket says 172.21.15.40:48496) 2024-09-13T18:21:12.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe19b7fe700 1 -- 172.21.15.40:0/195987896 learned_addr learned my addr 172.21.15.40:0/195987896 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:12.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe19a7fc700 1 --2- 172.21.15.40:0/195987896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe19c1503c0 0x7fe19c1f0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:12.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe19affd700 1 --2- 172.21.15.40:0/195987896 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe19c0c8500 0x7fe19c0c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:12.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe19b7fe700 1 -- 172.21.15.40:0/195987896 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe19c0c8500 msgr2=0x7fe19c0c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:12.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe19b7fe700 1 --2- 172.21.15.40:0/195987896 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe19c0c8500 0x7fe19c0c5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe19b7fe700 1 -- 172.21.15.40:0/195987896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe19c1503c0 msgr2=0x7fe19c1f0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:12.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe19b7fe700 1 --2- 172.21.15.40:0/195987896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe19c1503c0 0x7fe19c1f0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe19b7fe700 1 -- 172.21.15.40:0/195987896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe194009a70 con 0x7fe19c159630 2024-09-13T18:21:12.401 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe19b7fe700 1 --2- 172.21.15.40:0/195987896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe19c159630 0x7fe19c147320 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe19c066a90 tx=0x7fe194004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:12.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe1837fe700 1 -- 172.21.15.40:0/195987896 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194004800 con 0x7fe19c159630 2024-09-13T18:21:12.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe1837fe700 1 -- 172.21.15.40:0/195987896 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe194020070 con 0x7fe19c159630 2024-09-13T18:21:12.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe19bfff700 1 -- 172.21.15.40:0/195987896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe19c149380 con 0x7fe19c159630 2024-09-13T18:21:12.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe1837fe700 1 -- 172.21.15.40:0/195987896 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1940276c0 con 0x7fe19c159630 2024-09-13T18:21:12.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.393+0000 7fe19bfff700 1 -- 172.21.15.40:0/195987896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe19c149940 con 0x7fe19c159630 2024-09-13T18:21:12.402 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.397+0000 7fe19bfff700 1 -- 172.21.15.40:0/195987896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe19c142d60 con 0x7fe19c159630 2024-09-13T18:21:12.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.397+0000 7fe1837fe700 1 -- 172.21.15.40:0/195987896 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe194012430 con 0x7fe19c159630 2024-09-13T18:21:12.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.397+0000 7fe1837fe700 1 --2- 172.21.15.40:0/195987896 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe188066760 0x7fe188068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:12.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.397+0000 7fe1837fe700 1 -- 172.21.15.40:0/195987896 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe194022070 con 0x7fe19c159630 2024-09-13T18:21:12.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.397+0000 7fe19affd700 1 --2- 172.21.15.40:0/195987896 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe188066760 0x7fe188068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:12.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.397+0000 7fe19affd700 1 --2- 172.21.15.40:0/195987896 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe188066760 0x7fe188068c20 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe18c000f60 tx=0x7fe18c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:12.411 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.405+0000 7fe1837fe700 1 -- 172.21.15.40:0/195987896 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe19c142d60 con 0x7fe19c159630 2024-09-13T18:21:12.579 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.573+0000 7fe19bfff700 1 -- 172.21.15.40:0/195987896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe19c142d60 con 0x7fe19c159630 2024-09-13T18:21:12.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.573+0000 7fe1837fe700 1 -- 172.21.15.40:0/195987896 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe19c142d60 con 0x7fe19c159630 2024-09-13T18:21:12.580 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2024-09-13T18:21:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.577+0000 7fe19bfff700 1 -- 172.21.15.40:0/195987896 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe188066760 msgr2=0x7fe188068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.577+0000 7fe19bfff700 1 --2- 172.21.15.40:0/195987896 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe188066760 0x7fe188068c20 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe18c000f60 tx=0x7fe18c009040 comp rx=0 tx=0).stop 2024-09-13T18:21:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.577+0000 7fe19bfff700 1 -- 172.21.15.40:0/195987896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe19c159630 msgr2=0x7fe19c147320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.577+0000 7fe19bfff700 1 --2- 172.21.15.40:0/195987896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe19c159630 0x7fe19c147320 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe19c066a90 tx=0x7fe194004af0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.577+0000 7fe19bfff700 1 -- 172.21.15.40:0/195987896 shutdown_connections 2024-09-13T18:21:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.577+0000 7fe19bfff700 1 --2- 172.21.15.40:0/195987896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe19c159630 0x7fe19c147320 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.577+0000 7fe19bfff700 1 --2- 172.21.15.40:0/195987896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe19c1503c0 0x7fe19c1f0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.577+0000 7fe19bfff700 1 --2- 172.21.15.40:0/195987896 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe188066760 0x7fe188068c20 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.577+0000 7fe19bfff700 1 --2- 172.21.15.40:0/195987896 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe19c0c8500 0x7fe19c0c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.577+0000 7fe19bfff700 1 -- 172.21.15.40:0/195987896 >> 172.21.15.40:0/195987896 conn(0x7fe19c06e6d0 msgr2=0x7fe19c0c79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:12.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.577+0000 7fe19bfff700 1 -- 172.21.15.40:0/195987896 shutdown_connections 2024-09-13T18:21:12.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.577+0000 7fe19bfff700 1 -- 172.21.15.40:0/195987896 wait complete. 2024-09-13T18:21:12.594 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-13T18:21:12.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.701+0000 7fad77fff700 1 Processor -- start 2024-09-13T18:21:12.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.701+0000 7fad77fff700 1 -- start start 2024-09-13T18:21:12.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.701+0000 7fad77fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad780c7d30 0x7fad780ca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:12.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.701+0000 7fad77fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fad780c8110 0x7fad78155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:12.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.701+0000 7fad77fff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad78155b20 con 0x7fad780cad40 2024-09-13T18:21:12.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.701+0000 7fad77fff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad780cab20 con 0x7fad780c8110 2024-09-13T18:21:12.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.701+0000 7fad77fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad78159e80 con 0x7fad780c7d30 2024-09-13T18:21:12.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.701+0000 7fad767fc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fad780cad40 0x7fad7814f7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53518/0 (socket says 172.21.15.40:53518) 2024-09-13T18:21:12.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.701+0000 7fad767fc700 1 -- 172.21.15.40:0/4036128940 learned_addr learned my addr 172.21.15.40:0/4036128940 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:12.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.701+0000 7fad777fe700 1 --2- 172.21.15.40:0/4036128940 >> v2:172.21.15.43:3300/0 conn(0x7fad780c8110 0x7fad78155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:12.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.701+0000 7fad76ffd700 1 --2- 172.21.15.40:0/4036128940 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad780c7d30 0x7fad780ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:12.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.701+0000 7fad75ffb700 1 -- 172.21.15.40:0/4036128940 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2478666581 0 0) 0x7fad78155b20 con 0x7fad780cad40 2024-09-13T18:21:12.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.701+0000 7fad75ffb700 1 -- 172.21.15.40:0/4036128940 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad600035d0 con 0x7fad780cad40 2024-09-13T18:21:12.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.701+0000 7fad777fe700 1 -- 172.21.15.40:0/4036128940 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad780c7d30 msgr2=0x7fad780ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:12.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.701+0000 7fad777fe700 1 --2- 172.21.15.40:0/4036128940 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad780c7d30 0x7fad780ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.701+0000 7fad777fe700 1 -- 172.21.15.40:0/4036128940 >> v1:172.21.15.40:6789/0 conn(0x7fad780cad40 legacy=0x7fad7814f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:12.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.701+0000 7fad777fe700 1 -- 172.21.15.40:0/4036128940 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad78159080 con 0x7fad780c8110 2024-09-13T18:21:12.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.705+0000 7fad777fe700 1 --2- 172.21.15.40:0/4036128940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad780c8110 0x7fad78155fb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fad70009d90 tx=0x7fad7000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d78035ae668b3d7b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:12.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.705+0000 7fad75ffb700 1 -- 172.21.15.40:0/4036128940 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7000e070 con 0x7fad780c8110 2024-09-13T18:21:12.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.705+0000 7fad75ffb700 1 -- 172.21.15.40:0/4036128940 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7000bb20 con 0x7fad780c8110 2024-09-13T18:21:12.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.705+0000 7fad75ffb700 1 -- 172.21.15.40:0/4036128940 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70012430 con 0x7fad780c8110 2024-09-13T18:21:12.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.705+0000 7fad77fff700 1 -- 172.21.15.40:0/4036128940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad780c8110 msgr2=0x7fad78155fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:12.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.705+0000 7fad77fff700 1 --2- 172.21.15.40:0/4036128940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad780c8110 0x7fad78155fb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fad70009d90 tx=0x7fad7000b330 comp rx=0 tx=0).stop 2024-09-13T18:21:12.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.705+0000 7fad77fff700 1 -- 172.21.15.40:0/4036128940 shutdown_connections 2024-09-13T18:21:12.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.705+0000 7fad77fff700 1 --2- 172.21.15.40:0/4036128940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad780c8110 0x7fad78155fb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.705+0000 7fad77fff700 1 --2- 172.21.15.40:0/4036128940 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad780c7d30 0x7fad780ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.705+0000 7fad77fff700 1 -- 172.21.15.40:0/4036128940 >> 172.21.15.40:0/4036128940 conn(0x7fad7806a390 msgr2=0x7fad7806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:12.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.705+0000 7fad77fff700 1 -- 172.21.15.40:0/4036128940 shutdown_connections 2024-09-13T18:21:12.712 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.705+0000 7fad77fff700 1 -- 172.21.15.40:0/4036128940 wait complete. 2024-09-13T18:21:12.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.705+0000 7fad77fff700 1 Processor -- start 2024-09-13T18:21:12.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.705+0000 7fad77fff700 1 -- start start 2024-09-13T18:21:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad77fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad780c7d30 0x7fad781514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad77fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad780c8110 0x7fad78152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad77fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad780cad40 0x7fad780bf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad77fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad780bfae0 con 0x7fad780c8110 2024-09-13T18:21:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad77fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad78151090 con 0x7fad780cad40 2024-09-13T18:21:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad77fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad781523b0 con 0x7fad780c7d30 2024-09-13T18:21:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad767fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad780c8110 0x7fad78152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad767fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad780c8110 0x7fad78152890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48500/0 (socket says 172.21.15.40:48500) 2024-09-13T18:21:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad767fc700 1 -- 172.21.15.40:0/454403977 learned_addr learned my addr 172.21.15.40:0/454403977 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad76ffd700 1 --2- 172.21.15.40:0/454403977 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad780c7d30 0x7fad781514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:12.716 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad777fe700 1 --2- 172.21.15.40:0/454403977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad780cad40 0x7fad780bf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:12.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad767fc700 1 -- 172.21.15.40:0/454403977 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad780c7d30 msgr2=0x7fad781514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:12.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad767fc700 1 --2- 172.21.15.40:0/454403977 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad780c7d30 0x7fad781514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad767fc700 1 -- 172.21.15.40:0/454403977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad780cad40 msgr2=0x7fad780bf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:12.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad767fc700 1 --2- 172.21.15.40:0/454403977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad780cad40 0x7fad780bf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad767fc700 1 -- 172.21.15.40:0/454403977 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad70009a70 con 0x7fad780c8110 2024-09-13T18:21:12.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad777fe700 1 --2- 172.21.15.40:0/454403977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad780cad40 0x7fad780bf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:12.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad767fc700 1 --2- 172.21.15.40:0/454403977 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad780c8110 0x7fad78152890 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fad6c005930 tx=0x7fad6c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:12.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad5f7fe700 1 -- 172.21.15.40:0/454403977 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c005d90 con 0x7fad780c8110 2024-09-13T18:21:12.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad5f7fe700 1 -- 172.21.15.40:0/454403977 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c0070d0 con 0x7fad780c8110 2024-09-13T18:21:12.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad77fff700 1 -- 172.21.15.40:0/454403977 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad780c93d0 con 0x7fad780c8110 2024-09-13T18:21:12.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad5f7fe700 1 -- 172.21.15.40:0/454403977 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c007480 con 0x7fad780c8110 2024-09-13T18:21:12.717 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad77fff700 1 -- 172.21.15.40:0/454403977 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad780c9690 con 0x7fad780c8110 2024-09-13T18:21:12.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.709+0000 7fad77fff700 1 -- 172.21.15.40:0/454403977 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad78152570 con 0x7fad780c8110 2024-09-13T18:21:12.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.713+0000 7fad5f7fe700 1 -- 172.21.15.40:0/454403977 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fad6c0075e0 con 0x7fad780c8110 2024-09-13T18:21:12.718 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.713+0000 7fad5f7fe700 1 --2- 172.21.15.40:0/454403977 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad64066760 0x7fad64068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.713+0000 7fad5f7fe700 1 -- 172.21.15.40:0/454403977 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fad6c083b00 con 0x7fad780c8110 2024-09-13T18:21:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.713+0000 7fad76ffd700 1 --2- 172.21.15.40:0/454403977 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad64066760 0x7fad64068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:12.719 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.713+0000 7fad76ffd700 1 --2- 172.21.15.40:0/454403977 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad64066760 0x7fad64068c20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fad6800ac50 tx=0x7fad6800a350 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:12.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.717+0000 7fad5f7fe700 1 -- 172.21.15.40:0/454403977 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fad6c085e60 con 0x7fad780c8110 2024-09-13T18:21:12.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.885+0000 7fad77fff700 1 -- 172.21.15.40:0/454403977 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fad780c8580 con 0x7fad780c8110 2024-09-13T18:21:12.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.885+0000 7fad5f7fe700 1 -- 172.21.15.40:0/454403977 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fad6c050980 con 0x7fad780c8110 2024-09-13T18:21:12.891 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2024-09-13T18:21:12.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.889+0000 7fad77fff700 1 -- 172.21.15.40:0/454403977 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad64066760 msgr2=0x7fad64068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:12.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.889+0000 7fad77fff700 1 --2- 172.21.15.40:0/454403977 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad64066760 0x7fad64068c20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fad6800ac50 tx=0x7fad6800a350 comp rx=0 tx=0).stop 2024-09-13T18:21:12.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.889+0000 7fad77fff700 1 -- 172.21.15.40:0/454403977 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad780c8110 msgr2=0x7fad78152890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:12.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.889+0000 7fad77fff700 1 --2- 172.21.15.40:0/454403977 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad780c8110 0x7fad78152890 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fad6c005930 tx=0x7fad6c00c9e0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.889+0000 7fad77fff700 1 -- 172.21.15.40:0/454403977 shutdown_connections 2024-09-13T18:21:12.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.889+0000 7fad77fff700 1 --2- 172.21.15.40:0/454403977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad780cad40 0x7fad780bf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.889+0000 7fad77fff700 1 --2- 172.21.15.40:0/454403977 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fad780c8110 0x7fad78152890 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.889+0000 7fad77fff700 1 --2- 172.21.15.40:0/454403977 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fad64066760 0x7fad64068c20 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.889+0000 7fad77fff700 1 --2- 172.21.15.40:0/454403977 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fad780c7d30 0x7fad781514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:12.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.889+0000 7fad77fff700 1 -- 172.21.15.40:0/454403977 >> 172.21.15.40:0/454403977 conn(0x7fad7806a390 msgr2=0x7fad780c73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:12.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.889+0000 7fad77fff700 1 -- 172.21.15.40:0/454403977 shutdown_connections 2024-09-13T18:21:12.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:12.889+0000 7fad77fff700 1 -- 172.21.15.40:0/454403977 wait complete. 2024-09-13T18:21:12.906 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T18:21:13.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.005+0000 7f9ea359e700 1 Processor -- start 2024-09-13T18:21:13.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.005+0000 7f9ea359e700 1 -- start start 2024-09-13T18:21:13.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea359e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ea41535c0 0x7f9ea40c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:13.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea359e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9ea415d6e0 0x7f9ea41593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:13.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea359e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9ea4070b50 0x7f9ea4159900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:13.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea359e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea415c9b0 con 0x7f9ea4070b50 2024-09-13T18:21:13.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea359e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea415d250 con 0x7f9ea415d6e0 2024-09-13T18:21:13.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea359e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea4159ee0 con 0x7f9ea41535c0 2024-09-13T18:21:13.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea2d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9ea4070b50 0x7f9ea4159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:13.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea1d9b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9ea415d6e0 0x7f9ea41593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:13.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea2d9d700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9ea4070b50 0x7f9ea4159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48504/0 (socket says 172.21.15.40:48504) 2024-09-13T18:21:13.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea2d9d700 1 -- 172.21.15.40:0/570468360 learned_addr learned my addr 172.21.15.40:0/570468360 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:13.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea259c700 1 --2- 172.21.15.40:0/570468360 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ea41535c0 0x7f9ea40c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:13.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea2d9d700 1 -- 172.21.15.40:0/570468360 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ea41535c0 msgr2=0x7f9ea40c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:13.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea2d9d700 1 --2- 172.21.15.40:0/570468360 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ea41535c0 0x7f9ea40c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:13.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea2d9d700 1 -- 172.21.15.40:0/570468360 >> v2:172.21.15.43:3300/0 conn(0x7f9ea415d6e0 msgr2=0x7f9ea41593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:13.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea2d9d700 1 --2- 172.21.15.40:0/570468360 >> v2:172.21.15.43:3300/0 conn(0x7f9ea415d6e0 0x7f9ea41593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:13.017 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea2d9d700 1 -- 172.21.15.40:0/570468360 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea415a1c0 con 0x7f9ea4070b50 2024-09-13T18:21:13.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea1d9b700 1 --2- 172.21.15.40:0/570468360 >> v2:172.21.15.43:3300/0 conn(0x7f9ea415d6e0 0x7f9ea41593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:21:13.018 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea2d9d700 1 --2- 172.21.15.40:0/570468360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea4070b50 0x7f9ea4159900 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ea406b260 tx=0x7f9e9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9decfcef6bf21944 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:13.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea159a700 1 -- 172.21.15.40:0/570468360 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00e070 con 0x7f9ea4070b50 2024-09-13T18:21:13.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea159a700 1 -- 172.21.15.40:0/570468360 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c00b910 con 0x7f9ea4070b50 2024-09-13T18:21:13.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea159a700 1 -- 172.21.15.40:0/570468360 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00bcc0 con 0x7f9ea4070b50 2024-09-13T18:21:13.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea359e700 1 -- 172.21.15.40:0/570468360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea4070b50 msgr2=0x7f9ea4159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:13.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea359e700 1 --2- 172.21.15.40:0/570468360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea4070b50 0x7f9ea4159900 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ea406b260 tx=0x7f9e9c00b330 comp rx=0 tx=0).stop 2024-09-13T18:21:13.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea359e700 1 -- 172.21.15.40:0/570468360 shutdown_connections 2024-09-13T18:21:13.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea359e700 1 --2- 172.21.15.40:0/570468360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea4070b50 0x7f9ea4159900 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:13.019 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea359e700 1 --2- 172.21.15.40:0/570468360 >> v2:172.21.15.43:3300/0 conn(0x7f9ea415d6e0 0x7f9ea41593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:13.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea359e700 1 --2- 172.21.15.40:0/570468360 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ea41535c0 0x7f9ea40c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:13.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea359e700 1 -- 172.21.15.40:0/570468360 >> 172.21.15.40:0/570468360 conn(0x7f9ea406cba0 msgr2=0x7f9ea406a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:13.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea359e700 1 -- 172.21.15.40:0/570468360 shutdown_connections 2024-09-13T18:21:13.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.009+0000 7f9ea359e700 1 -- 172.21.15.40:0/570468360 wait complete. 2024-09-13T18:21:13.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea359e700 1 Processor -- start 2024-09-13T18:21:13.020 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea359e700 1 -- start start 2024-09-13T18:21:13.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea359e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ea4070b50 0x7f9ea414eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:13.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea359e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ea41535c0 0x7f9ea41f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:13.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea415d6e0 0x7f9ea41f2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:13.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea359e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea415b2c0 con 0x7f9ea415d6e0 2024-09-13T18:21:13.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea359e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea41ef5a0 con 0x7f9ea41535c0 2024-09-13T18:21:13.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea359e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40c5700 con 0x7f9ea4070b50 2024-09-13T18:21:13.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea2d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea415d6e0 0x7f9ea41f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:13.022 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea2d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea415d6e0 0x7f9ea41f2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48516/0 (socket says 172.21.15.40:48516) 2024-09-13T18:21:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea259c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ea4070b50 0x7f9ea414eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea2d9d700 1 -- 172.21.15.40:0/2966528970 learned_addr learned my addr 172.21.15.40:0/2966528970 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea1d9b700 1 --2- 172.21.15.40:0/2966528970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ea41535c0 0x7f9ea41f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea2d9d700 1 -- 172.21.15.40:0/2966528970 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ea4070b50 msgr2=0x7f9ea414eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea2d9d700 1 --2- 172.21.15.40:0/2966528970 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ea4070b50 0x7f9ea414eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea2d9d700 1 -- 172.21.15.40:0/2966528970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ea41535c0 msgr2=0x7f9ea41f1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea2d9d700 1 --2- 172.21.15.40:0/2966528970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ea41535c0 0x7f9ea41f1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea2d9d700 1 -- 172.21.15.40:0/2966528970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e980094b0 con 0x7f9ea415d6e0 2024-09-13T18:21:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea2d9d700 1 --2- 172.21.15.40:0/2966528970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea415d6e0 0x7f9ea41f2420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9ea406ed60 tx=0x7f9e9c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9e92ffd700 1 -- 172.21.15.40:0/2966528970 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c004880 con 0x7f9ea415d6e0 2024-09-13T18:21:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9e92ffd700 1 -- 172.21.15.40:0/2966528970 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c0099b0 con 0x7f9ea415d6e0 2024-09-13T18:21:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea359e700 1 -- 172.21.15.40:0/2966528970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e9c009b30 con 0x7f9ea415d6e0 2024-09-13T18:21:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9e92ffd700 1 -- 172.21.15.40:0/2966528970 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c01b670 con 0x7f9ea415d6e0 2024-09-13T18:21:13.023 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.013+0000 7f9ea359e700 1 -- 172.21.15.40:0/2966528970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea41eec70 con 0x7f9ea415d6e0 2024-09-13T18:21:13.024 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.017+0000 7f9ea359e700 1 -- 172.21.15.40:0/2966528970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea40c6c70 con 0x7f9ea415d6e0 2024-09-13T18:21:13.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.017+0000 7f9e92ffd700 1 -- 172.21.15.40:0/2966528970 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9e9c01b7d0 con 0x7f9ea415d6e0 2024-09-13T18:21:13.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.017+0000 7f9e92ffd700 1 --2- 172.21.15.40:0/2966528970 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e84066730 0x7f9e84068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:13.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.017+0000 7f9e92ffd700 1 -- 172.21.15.40:0/2966528970 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f9e9c085870 con 0x7f9ea415d6e0 2024-09-13T18:21:13.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.017+0000 7f9ea259c700 1 --2- 172.21.15.40:0/2966528970 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e84066730 0x7f9e84068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:13.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.017+0000 7f9ea259c700 1 --2- 172.21.15.40:0/2966528970 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e84066730 0x7f9e84068bf0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009b70 tx=0x7f9e94009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:13.031 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.025+0000 7f9e92ffd700 1 -- 172.21.15.40:0/2966528970 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9e9c052820 con 0x7f9ea415d6e0 2024-09-13T18:21:13.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.205+0000 7f9ea359e700 1 -- 172.21.15.40:0/2966528970 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9ea41ef6e0 con 0x7f9ea415d6e0 2024-09-13T18:21:13.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.205+0000 7f9e92ffd700 1 -- 172.21.15.40:0/2966528970 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9e9c055e40 con 0x7f9ea415d6e0 2024-09-13T18:21:13.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.209+0000 7f9ea359e700 1 -- 172.21.15.40:0/2966528970 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e84066730 msgr2=0x7f9e84068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:13.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.209+0000 7f9ea359e700 1 --2- 172.21.15.40:0/2966528970 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e84066730 0x7f9e84068bf0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009b70 tx=0x7f9e94009240 comp rx=0 tx=0).stop 2024-09-13T18:21:13.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.209+0000 7f9ea359e700 1 -- 172.21.15.40:0/2966528970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea415d6e0 msgr2=0x7f9ea41f2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:13.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.209+0000 7f9ea359e700 1 --2- 172.21.15.40:0/2966528970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea415d6e0 0x7f9ea41f2420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9ea406ed60 tx=0x7f9e9c004af0 comp rx=0 tx=0).stop 2024-09-13T18:21:13.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.209+0000 7f9ea359e700 1 -- 172.21.15.40:0/2966528970 shutdown_connections 2024-09-13T18:21:13.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.209+0000 7f9ea359e700 1 --2- 172.21.15.40:0/2966528970 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ea415d6e0 0x7f9ea41f2420 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:13.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.209+0000 7f9ea359e700 1 --2- 172.21.15.40:0/2966528970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ea41535c0 0x7f9ea41f1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:13.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.209+0000 7f9ea359e700 1 --2- 172.21.15.40:0/2966528970 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9e84066730 0x7f9e84068bf0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:13.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.209+0000 7f9ea359e700 1 --2- 172.21.15.40:0/2966528970 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ea4070b50 0x7f9ea414eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:13.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.209+0000 7f9ea359e700 1 -- 172.21.15.40:0/2966528970 >> 172.21.15.40:0/2966528970 conn(0x7f9ea406cba0 msgr2=0x7f9ea4152050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:13.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.209+0000 7f9ea359e700 1 -- 172.21.15.40:0/2966528970 shutdown_connections 2024-09-13T18:21:13.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:13.209+0000 7f9ea359e700 1 -- 172.21.15.40:0/2966528970 wait complete. 2024-09-13T18:21:13.216 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2024-09-13T18:21:13.226 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi040.front.sepia.ceph.com... 2024-09-13T18:21:13.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:21:13.226 DEBUG:teuthology.orchestra.run.smithi040:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-13T18:21:13.247 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:21:13.261 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:ceph-fuse[35133]: fuse finished with error 0 and tester_r 0 2024-09-13T18:21:19.349 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2024-09-13T18:21:19.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:21:19.349 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T18:21:19.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:21:19.408 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-09-13T18:21:19.567 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T18:21:19.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:21:19.567 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T18:21:19.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:21:19.585 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-09-13T18:21:19.711 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T18:21:19.725 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T18:21:19.726 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:21:19.852 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.845+0000 7f3c682f9700 1 Processor -- start 2024-09-13T18:21:19.853 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.845+0000 7f3c682f9700 1 -- start start 2024-09-13T18:21:19.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.845+0000 7f3c682f9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c6006fb20 0x7f3c600c04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:19.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.845+0000 7f3c682f9700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3c6006d950 0x7f3c600c0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:19.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.845+0000 7f3c682f9700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c60071bf0 con 0x7f3c6006d570 2024-09-13T18:21:19.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.845+0000 7f3c682f9700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c60071330 con 0x7f3c6006d950 2024-09-13T18:21:19.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.845+0000 7f3c682f9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c600c73a0 con 0x7f3c6006fb20 2024-09-13T18:21:19.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c66095700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3c6006d570 0x7f3c601502b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53522/0 (socket says 172.21.15.40:53522) 2024-09-13T18:21:19.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c66095700 1 -- 172.21.15.40:0/4148280957 learned_addr learned my addr 172.21.15.40:0/4148280957 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:19.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c65894700 1 --2- 172.21.15.40:0/4148280957 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c6006fb20 0x7f3c600c04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:19.854 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c66896700 1 --2- 172.21.15.40:0/4148280957 >> v2:172.21.15.43:3300/0 conn(0x7f3c6006d950 0x7f3c600c0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:19.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c65093700 1 -- 172.21.15.40:0/4148280957 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3325487103 0 0) 0x7f3c60071bf0 con 0x7f3c6006d570 2024-09-13T18:21:19.855 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c65093700 1 -- 172.21.15.40:0/4148280957 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c4c0035d0 con 0x7f3c6006d570 2024-09-13T18:21:19.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c65894700 1 -- 172.21.15.40:0/4148280957 >> v2:172.21.15.43:3300/0 conn(0x7f3c6006d950 msgr2=0x7f3c600c0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:19.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c65894700 1 --2- 172.21.15.40:0/4148280957 >> v2:172.21.15.43:3300/0 conn(0x7f3c6006d950 0x7f3c600c0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:19.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c65894700 1 -- 172.21.15.40:0/4148280957 >> v1:172.21.15.40:6789/0 conn(0x7f3c6006d570 legacy=0x7f3c601502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:19.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c65894700 1 -- 172.21.15.40:0/4148280957 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c600715a0 con 0x7f3c6006fb20 2024-09-13T18:21:19.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c65894700 1 --2- 172.21.15.40:0/4148280957 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c6006fb20 0x7f3c600c04f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009dc0 tx=0x7f3c5c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edf30ccca7d21ed3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:19.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c65093700 1 -- 172.21.15.40:0/4148280957 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c00e070 con 0x7f3c6006fb20 2024-09-13T18:21:19.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c65093700 1 -- 172.21.15.40:0/4148280957 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5c00bae0 con 0x7f3c6006fb20 2024-09-13T18:21:19.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c65093700 1 -- 172.21.15.40:0/4148280957 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c00be90 con 0x7f3c6006fb20 2024-09-13T18:21:19.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c682f9700 1 -- 172.21.15.40:0/4148280957 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c6006fb20 msgr2=0x7f3c600c04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:19.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c682f9700 1 --2- 172.21.15.40:0/4148280957 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c6006fb20 0x7f3c600c04f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009dc0 tx=0x7f3c5c00b330 comp rx=0 tx=0).stop 2024-09-13T18:21:19.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c682f9700 1 -- 172.21.15.40:0/4148280957 shutdown_connections 2024-09-13T18:21:19.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c682f9700 1 --2- 172.21.15.40:0/4148280957 >> v2:172.21.15.43:3300/0 conn(0x7f3c6006d950 0x7f3c600c0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:19.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c682f9700 1 --2- 172.21.15.40:0/4148280957 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c6006fb20 0x7f3c600c04f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:19.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c682f9700 1 -- 172.21.15.40:0/4148280957 >> 172.21.15.40:0/4148280957 conn(0x7f3c600cadd0 msgr2=0x7f3c600cb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:19.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c682f9700 1 -- 172.21.15.40:0/4148280957 shutdown_connections 2024-09-13T18:21:19.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.849+0000 7f3c682f9700 1 -- 172.21.15.40:0/4148280957 wait complete. 2024-09-13T18:21:19.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.853+0000 7f3c682f9700 1 Processor -- start 2024-09-13T18:21:19.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.853+0000 7f3c682f9700 1 -- start start 2024-09-13T18:21:19.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.853+0000 7f3c682f9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c6006d570 0x7f3c601402a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:19.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.853+0000 7f3c682f9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c6006d950 0x7f3c6013a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:19.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.853+0000 7f3c682f9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c6006fb20 0x7f3c601374d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:19.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.853+0000 7f3c682f9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c601445d0 con 0x7f3c6006d950 2024-09-13T18:21:19.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.853+0000 7f3c682f9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c601440e0 con 0x7f3c6006d570 2024-09-13T18:21:19.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.853+0000 7f3c682f9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6013c9a0 con 0x7f3c6006fb20 2024-09-13T18:21:19.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.853+0000 7f3c65894700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c6006d950 0x7f3c6013a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:19.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.853+0000 7f3c65894700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c6006d950 0x7f3c6013a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48522/0 (socket says 172.21.15.40:48522) 2024-09-13T18:21:19.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.853+0000 7f3c65894700 1 -- 172.21.15.40:0/3726798846 learned_addr learned my addr 172.21.15.40:0/3726798846 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:19.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.853+0000 7f3c66095700 1 --2- 172.21.15.40:0/3726798846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c6006d570 0x7f3c601402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:19.862 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.853+0000 7f3c66896700 1 --2- 172.21.15.40:0/3726798846 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c6006fb20 0x7f3c601374d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.857+0000 7f3c65894700 1 -- 172.21.15.40:0/3726798846 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c6006fb20 msgr2=0x7f3c601374d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.857+0000 7f3c65894700 1 --2- 172.21.15.40:0/3726798846 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c6006fb20 0x7f3c601374d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.857+0000 7f3c65894700 1 -- 172.21.15.40:0/3726798846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c6006d570 msgr2=0x7f3c601402a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.857+0000 7f3c65894700 1 --2- 172.21.15.40:0/3726798846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c6006d570 0x7f3c601402a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.857+0000 7f3c65894700 1 -- 172.21.15.40:0/3726798846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c009a70 con 0x7f3c6006d950 2024-09-13T18:21:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.857+0000 7f3c66896700 1 --2- 172.21.15.40:0/3726798846 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c6006fb20 0x7f3c601374d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:21:19.863 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.857+0000 7f3c66095700 1 --2- 172.21.15.40:0/3726798846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c6006d570 0x7f3c601402a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:21:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.857+0000 7f3c65894700 1 --2- 172.21.15.40:0/3726798846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c6006d950 0x7f3c6013a790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c000f80 tx=0x7f3c5c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.857+0000 7f3c52ffd700 1 -- 172.21.15.40:0/3726798846 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c004550 con 0x7f3c6006d950 2024-09-13T18:21:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.857+0000 7f3c52ffd700 1 -- 172.21.15.40:0/3726798846 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5c020070 con 0x7f3c6006d950 2024-09-13T18:21:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.857+0000 7f3c682f9700 1 -- 172.21.15.40:0/3726798846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c60138fe0 con 0x7f3c6006d950 2024-09-13T18:21:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.857+0000 7f3c52ffd700 1 -- 172.21.15.40:0/3726798846 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c004910 con 0x7f3c6006d950 2024-09-13T18:21:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.857+0000 7f3c682f9700 1 -- 172.21.15.40:0/3726798846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c60139f90 con 0x7f3c6006d950 2024-09-13T18:21:19.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.857+0000 7f3c682f9700 1 -- 172.21.15.40:0/3726798846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c60139df0 con 0x7f3c6006d950 2024-09-13T18:21:19.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.861+0000 7f3c52ffd700 1 -- 172.21.15.40:0/3726798846 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3c5c012430 con 0x7f3c6006d950 2024-09-13T18:21:19.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.861+0000 7f3c52ffd700 1 --2- 172.21.15.40:0/3726798846 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3c44066760 0x7f3c44068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:19.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.861+0000 7f3c52ffd700 1 -- 172.21.15.40:0/3726798846 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3c5c022070 con 0x7f3c6006d950 2024-09-13T18:21:19.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.861+0000 7f3c66095700 1 --2- 172.21.15.40:0/3726798846 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3c44066760 0x7f3c44068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:19.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.861+0000 7f3c66095700 1 --2- 172.21.15.40:0/3726798846 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3c44066760 0x7f3c44068c20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3c58007fd0 tx=0x7f3c58007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:19.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:19.869+0000 7f3c52ffd700 1 -- 172.21.15.40:0/3726798846 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3c60139df0 con 0x7f3c6006d950 2024-09-13T18:21:20.042 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.037+0000 7f3c682f9700 1 -- 172.21.15.40:0/3726798846 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3c6013c610 con 0x7f3c44066760 2024-09-13T18:21:20.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.065+0000 7f3c52ffd700 1 -- 172.21.15.40:0/3726798846 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+369002 (secure 0 0 0) 0x7f3c6013c610 con 0x7f3c44066760 2024-09-13T18:21:20.077 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T18:21:20.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.069+0000 7f3c682f9700 1 -- 172.21.15.40:0/3726798846 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3c44066760 msgr2=0x7f3c44068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.069+0000 7f3c682f9700 1 --2- 172.21.15.40:0/3726798846 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3c44066760 0x7f3c44068c20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3c58007fd0 tx=0x7f3c58007de0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.069+0000 7f3c682f9700 1 -- 172.21.15.40:0/3726798846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c6006d950 msgr2=0x7f3c6013a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.069+0000 7f3c682f9700 1 --2- 172.21.15.40:0/3726798846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c6006d950 0x7f3c6013a790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c000f80 tx=0x7f3c5c004af0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.069+0000 7f3c682f9700 1 -- 172.21.15.40:0/3726798846 shutdown_connections 2024-09-13T18:21:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.069+0000 7f3c682f9700 1 --2- 172.21.15.40:0/3726798846 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3c44066760 0x7f3c44068c20 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.069+0000 7f3c682f9700 1 --2- 172.21.15.40:0/3726798846 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3c6006fb20 0x7f3c601374d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.069+0000 7f3c682f9700 1 --2- 172.21.15.40:0/3726798846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c6006d950 0x7f3c6013a790 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.069+0000 7f3c682f9700 1 --2- 172.21.15.40:0/3726798846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3c6006d570 0x7f3c601402a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.069+0000 7f3c682f9700 1 -- 172.21.15.40:0/3726798846 >> 172.21.15.40:0/3726798846 conn(0x7f3c600cadd0 msgr2=0x7f3c60145170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.069+0000 7f3c682f9700 1 -- 172.21.15.40:0/3726798846 shutdown_connections 2024-09-13T18:21:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.069+0000 7f3c682f9700 1 -- 172.21.15.40:0/3726798846 wait complete. 2024-09-13T18:21:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-13T18:21:20.092 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":949,"stamp":"2024-09-13T18:21:19.774698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":16532352,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":237,"num_read_kb":279,"num_write":755012,"num_write_kb":16125724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183058,"ondisk_log_size":183058,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10715160,"kb_used_data":34648,"kb_used_omap":12,"kb_used_meta":10680435,"kb_avail":744259560,"statfs":{"total":773094113280,"available":762121789440,"internally_reserved":0,"allocated":35479552,"data_stored":33992096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":10936766008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005254"},"pg_stats":[{"pgid":"2.1f","version":"27'268","reported_seq":299,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.261706+0000","last_change":"2024-09-13T17:50:34.212158+0000","last_active":"2024-09-13T18:21:13.261706+0000","last_peered":"2024-09-13T18:21:13.261706+0000","last_clean":"2024-09-13T18:21:13.261706+0000","last_became_active":"2024-09-13T17:50:34.211867+0000","last_became_peered":"2024-09-13T17:50:34.211867+0000","last_unstale":"2024-09-13T18:21:13.261706+0000","last_undegraded":"2024-09-13T18:21:13.261706+0000","last_fullsized":"2024-09-13T18:21:13.261706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:14.671095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3260943,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":267,"num_write_kb":3318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'10","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306110+0000","last_change":"2024-09-13T17:50:34.587396+0000","last_active":"2024-09-13T18:21:13.306110+0000","last_peered":"2024-09-13T18:21:13.306110+0000","last_clean":"2024-09-13T18:21:13.306110+0000","last_became_active":"2024-09-13T17:50:34.587108+0000","last_became_peered":"2024-09-13T17:50:34.587108+0000","last_unstale":"2024-09-13T18:21:13.306110+0000","last_undegraded":"2024-09-13T18:21:13.306110+0000","last_fullsized":"2024-09-13T18:21:13.306110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:39:16.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'5594","reported_seq":5620,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.876951+0000","last_change":"2024-09-13T17:50:38.615059+0000","last_active":"2024-09-13T18:21:02.876951+0000","last_peered":"2024-09-13T18:21:02.876951+0000","last_clean":"2024-09-13T18:21:02.876951+0000","last_became_active":"2024-09-13T17:50:38.614723+0000","last_became_peered":"2024-09-13T17:50:38.614723+0000","last_unstale":"2024-09-13T18:21:02.876951+0000","last_undegraded":"2024-09-13T18:21:02.876951+0000","last_fullsized":"2024-09-13T18:21:02.876951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5594,"log_dups_size":0,"ondisk_log_size":5594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:23:01.902181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23732,"num_write_kb":533374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'6","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.773351+0000","last_change":"2024-09-13T17:50:34.204760+0000","last_active":"2024-09-13T18:21:02.773351+0000","last_peered":"2024-09-13T18:21:02.773351+0000","last_clean":"2024-09-13T18:21:02.773351+0000","last_became_active":"2024-09-13T17:50:34.202930+0000","last_became_peered":"2024-09-13T17:50:34.202930+0000","last_unstale":"2024-09-13T18:21:02.773351+0000","last_undegraded":"2024-09-13T18:21:02.773351+0000","last_fullsized":"2024-09-13T18:21:02.773351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:59.784701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'20","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306487+0000","last_change":"2024-09-13T17:51:16.366352+0000","last_active":"2024-09-13T18:21:13.306487+0000","last_peered":"2024-09-13T18:21:13.306487+0000","last_clean":"2024-09-13T18:21:13.306487+0000","last_became_active":"2024-09-13T17:51:16.365492+0000","last_became_peered":"2024-09-13T17:51:16.365492+0000","last_unstale":"2024-09-13T18:21:13.306487+0000","last_undegraded":"2024-09-13T18:21:13.306487+0000","last_fullsized":"2024-09-13T18:21:13.306487+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:42.192182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'5615","reported_seq":5641,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.875980+0000","last_change":"2024-09-13T17:50:38.618751+0000","last_active":"2024-09-13T18:21:02.875980+0000","last_peered":"2024-09-13T18:21:02.875980+0000","last_clean":"2024-09-13T18:21:02.875980+0000","last_became_active":"2024-09-13T17:50:38.618423+0000","last_became_peered":"2024-09-13T17:50:38.618423+0000","last_unstale":"2024-09-13T18:21:02.875980+0000","last_undegraded":"2024-09-13T18:21:02.875980+0000","last_fullsized":"2024-09-13T18:21:02.875980+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5615,"log_dups_size":0,"ondisk_log_size":5615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:50:02.418785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23762,"num_write_kb":516686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'11","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.801229+0000","last_change":"2024-09-13T17:50:34.214972+0000","last_active":"2024-09-13T18:21:02.801229+0000","last_peered":"2024-09-13T18:21:02.801229+0000","last_clean":"2024-09-13T18:21:02.801229+0000","last_became_active":"2024-09-13T17:50:34.214652+0000","last_became_peered":"2024-09-13T17:50:34.214652+0000","last_unstale":"2024-09-13T18:21:02.801229+0000","last_undegraded":"2024-09-13T18:21:02.801229+0000","last_fullsized":"2024-09-13T18:21:02.801229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:57.176854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'18","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.292937+0000","last_change":"2024-09-13T17:51:16.281831+0000","last_active":"2024-09-13T18:21:13.292937+0000","last_peered":"2024-09-13T18:21:13.292937+0000","last_clean":"2024-09-13T18:21:13.292937+0000","last_became_active":"2024-09-13T17:51:16.281549+0000","last_became_peered":"2024-09-13T17:51:16.281549+0000","last_unstale":"2024-09-13T18:21:13.292937+0000","last_undegraded":"2024-09-13T18:21:13.292937+0000","last_fullsized":"2024-09-13T18:21:13.292937+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:43.894437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"27'5142","reported_seq":5168,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.803386+0000","last_change":"2024-09-13T17:50:38.616219+0000","last_active":"2024-09-13T18:21:02.803386+0000","last_peered":"2024-09-13T18:21:02.803386+0000","last_clean":"2024-09-13T18:21:02.803386+0000","last_became_active":"2024-09-13T17:50:38.614329+0000","last_became_peered":"2024-09-13T17:50:38.614329+0000","last_unstale":"2024-09-13T18:21:02.803386+0000","last_undegraded":"2024-09-13T18:21:02.803386+0000","last_fullsized":"2024-09-13T18:21:02.803386+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5142,"log_dups_size":0,"ondisk_log_size":5142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:25.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21334,"num_write_kb":478261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'8","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.159841+0000","last_change":"2024-09-13T17:50:34.216123+0000","last_active":"2024-09-13T18:20:25.159841+0000","last_peered":"2024-09-13T18:20:25.159841+0000","last_clean":"2024-09-13T18:20:25.159841+0000","last_became_active":"2024-09-13T17:50:34.215661+0000","last_became_peered":"2024-09-13T17:50:34.215661+0000","last_unstale":"2024-09-13T18:20:25.159841+0000","last_undegraded":"2024-09-13T18:20:25.159841+0000","last_fullsized":"2024-09-13T18:20:25.159841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:20.895037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'30","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.318958+0000","last_change":"2024-09-13T17:50:34.599673+0000","last_active":"2024-09-13T18:21:13.318958+0000","last_peered":"2024-09-13T18:21:13.318958+0000","last_clean":"2024-09-13T18:21:13.318958+0000","last_became_active":"2024-09-13T17:50:34.599409+0000","last_became_peered":"2024-09-13T17:50:34.599409+0000","last_unstale":"2024-09-13T18:21:13.318958+0000","last_undegraded":"2024-09-13T18:21:13.318958+0000","last_fullsized":"2024-09-13T18:21:13.318958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:03:31.394385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'4871","reported_seq":4897,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.743078+0000","last_change":"2024-09-13T17:50:38.616317+0000","last_active":"2024-09-13T18:21:02.743078+0000","last_peered":"2024-09-13T18:21:02.743078+0000","last_clean":"2024-09-13T18:21:02.743078+0000","last_became_active":"2024-09-13T17:50:38.614311+0000","last_became_peered":"2024-09-13T17:50:38.614311+0000","last_unstale":"2024-09-13T18:21:02.743078+0000","last_undegraded":"2024-09-13T18:21:02.743078+0000","last_fullsized":"2024-09-13T18:21:02.743078+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4871,"log_dups_size":0,"ondisk_log_size":4871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:29.221522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20106,"num_write_kb":432357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'14","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.313143+0000","last_change":"2024-09-13T17:50:34.599490+0000","last_active":"2024-09-13T18:21:13.313143+0000","last_peered":"2024-09-13T18:21:13.313143+0000","last_clean":"2024-09-13T18:21:13.313143+0000","last_became_active":"2024-09-13T17:50:34.599207+0000","last_became_peered":"2024-09-13T17:50:34.599207+0000","last_unstale":"2024-09-13T18:21:13.313143+0000","last_undegraded":"2024-09-13T18:21:13.313143+0000","last_fullsized":"2024-09-13T18:21:13.313143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:11.178161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'1092","reported_seq":1122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.671116+0000","last_change":"2024-09-13T17:50:34.215697+0000","last_active":"2024-09-13T18:21:02.671116+0000","last_peered":"2024-09-13T18:21:02.671116+0000","last_clean":"2024-09-13T18:21:02.671116+0000","last_became_active":"2024-09-13T17:50:34.215409+0000","last_became_peered":"2024-09-13T17:50:34.215409+0000","last_unstale":"2024-09-13T18:21:02.671116+0000","last_undegraded":"2024-09-13T18:21:02.671116+0000","last_fullsized":"2024-09-13T18:21:02.671116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:21.609567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1088,"num_write_kb":4339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'5394","reported_seq":5421,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.852704+0000","last_change":"2024-09-13T17:50:38.611817+0000","last_active":"2024-09-13T18:21:02.852704+0000","last_peered":"2024-09-13T18:21:02.852704+0000","last_clean":"2024-09-13T18:21:02.852704+0000","last_became_active":"2024-09-13T17:50:38.611263+0000","last_became_peered":"2024-09-13T17:50:38.611263+0000","last_unstale":"2024-09-13T18:21:02.852704+0000","last_undegraded":"2024-09-13T18:21:02.852704+0000","last_fullsized":"2024-09-13T18:21:02.852704+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5394,"log_dups_size":0,"ondisk_log_size":5394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:56:47.466672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22326,"num_write_kb":494524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'19","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.308364+0000","last_change":"2024-09-13T17:50:34.586307+0000","last_active":"2024-09-13T18:21:13.308364+0000","last_peered":"2024-09-13T18:21:13.308364+0000","last_clean":"2024-09-13T18:21:13.308364+0000","last_became_active":"2024-09-13T17:50:34.585831+0000","last_became_peered":"2024-09-13T17:50:34.585831+0000","last_unstale":"2024-09-13T18:21:13.308364+0000","last_undegraded":"2024-09-13T18:21:13.308364+0000","last_fullsized":"2024-09-13T18:21:13.308364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:40.912352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'9","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.773590+0000","last_change":"2024-09-13T17:50:34.211850+0000","last_active":"2024-09-13T18:21:02.773590+0000","last_peered":"2024-09-13T18:21:02.773590+0000","last_clean":"2024-09-13T18:21:02.773590+0000","last_became_active":"2024-09-13T17:50:34.211594+0000","last_became_peered":"2024-09-13T17:50:34.211594+0000","last_unstale":"2024-09-13T18:21:02.773590+0000","last_undegraded":"2024-09-13T18:21:02.773590+0000","last_fullsized":"2024-09-13T18:21:02.773590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:35:02.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'5504","reported_seq":5531,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.796881+0000","last_change":"2024-09-13T17:50:38.613576+0000","last_active":"2024-09-13T18:21:02.796881+0000","last_peered":"2024-09-13T18:21:02.796881+0000","last_clean":"2024-09-13T18:21:02.796881+0000","last_became_active":"2024-09-13T17:50:38.612577+0000","last_became_peered":"2024-09-13T17:50:38.612577+0000","last_unstale":"2024-09-13T18:21:02.796881+0000","last_undegraded":"2024-09-13T18:21:02.796881+0000","last_fullsized":"2024-09-13T18:21:02.796881+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5504,"log_dups_size":0,"ondisk_log_size":5504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:25.551060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23286,"num_write_kb":502495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'19","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306452+0000","last_change":"2024-09-13T17:50:34.587273+0000","last_active":"2024-09-13T18:21:13.306452+0000","last_peered":"2024-09-13T18:21:13.306452+0000","last_clean":"2024-09-13T18:21:13.306452+0000","last_became_active":"2024-09-13T17:50:34.587027+0000","last_became_peered":"2024-09-13T17:50:34.587027+0000","last_unstale":"2024-09-13T18:21:13.306452+0000","last_undegraded":"2024-09-13T18:21:13.306452+0000","last_fullsized":"2024-09-13T18:21:13.306452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:58.826737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'5","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.773643+0000","last_change":"2024-09-13T17:50:34.211845+0000","last_active":"2024-09-13T18:21:02.773643+0000","last_peered":"2024-09-13T18:21:02.773643+0000","last_clean":"2024-09-13T18:21:02.773643+0000","last_became_active":"2024-09-13T17:50:34.211568+0000","last_became_peered":"2024-09-13T17:50:34.211568+0000","last_unstale":"2024-09-13T18:21:02.773643+0000","last_undegraded":"2024-09-13T18:21:02.773643+0000","last_fullsized":"2024-09-13T18:21:02.773643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:56.348984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'6938","reported_seq":6965,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.647468+0000","last_change":"2024-09-13T17:50:38.610292+0000","last_active":"2024-09-13T18:21:02.647468+0000","last_peered":"2024-09-13T18:21:02.647468+0000","last_clean":"2024-09-13T18:21:02.647468+0000","last_became_active":"2024-09-13T17:50:38.609832+0000","last_became_peered":"2024-09-13T17:50:38.609832+0000","last_unstale":"2024-09-13T18:21:02.647468+0000","last_undegraded":"2024-09-13T18:21:02.647468+0000","last_fullsized":"2024-09-13T18:21:02.647468+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6938,"log_dups_size":0,"ondisk_log_size":6938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:13.783068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28644,"num_write_kb":622131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"27'12","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.931907+0000","last_change":"2024-09-13T17:50:34.593566+0000","last_active":"2024-09-13T18:21:02.931907+0000","last_peered":"2024-09-13T18:21:02.931907+0000","last_clean":"2024-09-13T18:21:02.931907+0000","last_became_active":"2024-09-13T17:50:34.588794+0000","last_became_peered":"2024-09-13T17:50:34.588794+0000","last_unstale":"2024-09-13T18:21:02.931907+0000","last_undegraded":"2024-09-13T18:21:02.931907+0000","last_fullsized":"2024-09-13T18:21:02.931907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:13.192453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'6","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.689388+0000","last_change":"2024-09-13T17:50:34.214638+0000","last_active":"2024-09-13T18:21:02.689388+0000","last_peered":"2024-09-13T18:21:02.689388+0000","last_clean":"2024-09-13T18:21:02.689388+0000","last_became_active":"2024-09-13T17:50:34.214292+0000","last_became_peered":"2024-09-13T17:50:34.214292+0000","last_unstale":"2024-09-13T18:21:02.689388+0000","last_undegraded":"2024-09-13T18:21:02.689388+0000","last_fullsized":"2024-09-13T18:21:02.689388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:57.377104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'4588","reported_seq":4615,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.886736+0000","last_change":"2024-09-13T17:50:38.613368+0000","last_active":"2024-09-13T18:21:02.886736+0000","last_peered":"2024-09-13T18:21:02.886736+0000","last_clean":"2024-09-13T18:21:02.886736+0000","last_became_active":"2024-09-13T17:50:38.613010+0000","last_became_peered":"2024-09-13T17:50:38.613010+0000","last_unstale":"2024-09-13T18:21:02.886736+0000","last_undegraded":"2024-09-13T18:21:02.886736+0000","last_fullsized":"2024-09-13T18:21:02.886736+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4588,"log_dups_size":0,"ondisk_log_size":4588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:42.706625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19354,"num_write_kb":454021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'26","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.302413+0000","last_change":"2024-09-13T17:50:34.582327+0000","last_active":"2024-09-13T18:21:13.302413+0000","last_peered":"2024-09-13T18:21:13.302413+0000","last_clean":"2024-09-13T18:21:13.302413+0000","last_became_active":"2024-09-13T17:50:34.582050+0000","last_became_peered":"2024-09-13T17:50:34.582050+0000","last_unstale":"2024-09-13T18:21:13.302413+0000","last_undegraded":"2024-09-13T18:21:13.302413+0000","last_fullsized":"2024-09-13T18:21:13.302413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:52.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'9","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.838047+0000","last_change":"2024-09-13T17:50:34.211512+0000","last_active":"2024-09-13T18:21:02.838047+0000","last_peered":"2024-09-13T18:21:02.838047+0000","last_clean":"2024-09-13T18:21:02.838047+0000","last_became_active":"2024-09-13T17:50:34.211230+0000","last_became_peered":"2024-09-13T17:50:34.211230+0000","last_unstale":"2024-09-13T18:21:02.838047+0000","last_undegraded":"2024-09-13T18:21:02.838047+0000","last_fullsized":"2024-09-13T18:21:02.838047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:40.707777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'6024","reported_seq":6052,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.702767+0000","last_change":"2024-09-13T17:50:38.608255+0000","last_active":"2024-09-13T18:21:02.702767+0000","last_peered":"2024-09-13T18:21:02.702767+0000","last_clean":"2024-09-13T18:21:02.702767+0000","last_became_active":"2024-09-13T17:50:38.606697+0000","last_became_peered":"2024-09-13T17:50:38.606697+0000","last_unstale":"2024-09-13T18:21:02.702767+0000","last_undegraded":"2024-09-13T18:21:02.702767+0000","last_fullsized":"2024-09-13T18:21:02.702767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6024,"log_dups_size":0,"ondisk_log_size":6024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:47.265007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25552,"num_write_kb":507487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'18","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.314914+0000","last_change":"2024-09-13T17:50:34.594476+0000","last_active":"2024-09-13T18:21:13.314914+0000","last_peered":"2024-09-13T18:21:13.314914+0000","last_clean":"2024-09-13T18:21:13.314914+0000","last_became_active":"2024-09-13T17:50:34.594215+0000","last_became_peered":"2024-09-13T17:50:34.594215+0000","last_unstale":"2024-09-13T18:21:13.314914+0000","last_undegraded":"2024-09-13T18:21:13.314914+0000","last_fullsized":"2024-09-13T18:21:13.314914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:02:27.528019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'284","reported_seq":315,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.685813+0000","last_change":"2024-09-13T17:50:34.204018+0000","last_active":"2024-09-13T18:21:02.685813+0000","last_peered":"2024-09-13T18:21:02.685813+0000","last_clean":"2024-09-13T18:21:02.685813+0000","last_became_active":"2024-09-13T17:50:34.203702+0000","last_became_peered":"2024-09-13T17:50:34.203702+0000","last_unstale":"2024-09-13T18:21:02.685813+0000","last_undegraded":"2024-09-13T18:21:02.685813+0000","last_fullsized":"2024-09-13T18:21:02.685813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:44.431226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":605,"num_write_kb":197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'6030","reported_seq":6069,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.817288+0000","last_change":"2024-09-13T17:51:16.365782+0000","last_active":"2024-09-13T18:21:02.817288+0000","last_peered":"2024-09-13T18:21:02.817288+0000","last_clean":"2024-09-13T18:21:02.817288+0000","last_became_active":"2024-09-13T17:51:16.364540+0000","last_became_peered":"2024-09-13T17:51:16.364540+0000","last_unstale":"2024-09-13T18:21:02.817288+0000","last_undegraded":"2024-09-13T18:21:02.817288+0000","last_fullsized":"2024-09-13T18:21:02.817288+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6030,"log_dups_size":0,"ondisk_log_size":6030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:18.753557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25617,"num_write_kb":548663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'18","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.290374+0000","last_change":"2024-09-13T17:50:34.586802+0000","last_active":"2024-09-13T18:21:13.290374+0000","last_peered":"2024-09-13T18:21:13.290374+0000","last_clean":"2024-09-13T18:21:13.290374+0000","last_became_active":"2024-09-13T17:50:34.586552+0000","last_became_peered":"2024-09-13T17:50:34.586552+0000","last_unstale":"2024-09-13T18:21:13.290374+0000","last_undegraded":"2024-09-13T18:21:13.290374+0000","last_fullsized":"2024-09-13T18:21:13.290374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:43.383975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'7","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.741537+0000","last_change":"2024-09-13T17:50:34.217083+0000","last_active":"2024-09-13T18:21:02.741537+0000","last_peered":"2024-09-13T18:21:02.741537+0000","last_clean":"2024-09-13T18:21:02.741537+0000","last_became_active":"2024-09-13T17:50:34.216758+0000","last_became_peered":"2024-09-13T17:50:34.216758+0000","last_unstale":"2024-09-13T18:21:02.741537+0000","last_undegraded":"2024-09-13T18:21:02.741537+0000","last_fullsized":"2024-09-13T18:21:02.741537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:36.473942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'4887","reported_seq":4915,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.617066+0000","last_change":"2024-09-13T17:50:38.614667+0000","last_active":"2024-09-13T18:21:02.617066+0000","last_peered":"2024-09-13T18:21:02.617066+0000","last_clean":"2024-09-13T18:21:02.617066+0000","last_became_active":"2024-09-13T17:50:38.614284+0000","last_became_peered":"2024-09-13T17:50:38.614284+0000","last_unstale":"2024-09-13T18:21:02.617066+0000","last_undegraded":"2024-09-13T18:21:02.617066+0000","last_fullsized":"2024-09-13T18:21:02.617066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4887,"log_dups_size":0,"ondisk_log_size":4887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:29.616794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20690,"num_write_kb":442963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'16","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.315360+0000","last_change":"2024-09-13T17:50:34.588603+0000","last_active":"2024-09-13T18:21:13.315360+0000","last_peered":"2024-09-13T18:21:13.315360+0000","last_clean":"2024-09-13T18:21:13.315360+0000","last_became_active":"2024-09-13T17:50:34.588317+0000","last_became_peered":"2024-09-13T17:50:34.588317+0000","last_unstale":"2024-09-13T18:21:13.315360+0000","last_undegraded":"2024-09-13T18:21:13.315360+0000","last_fullsized":"2024-09-13T18:21:13.315360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:03.743130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'3","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.727771+0000","last_change":"2024-09-13T17:50:34.204335+0000","last_active":"2024-09-13T18:21:02.727771+0000","last_peered":"2024-09-13T18:21:02.727771+0000","last_clean":"2024-09-13T18:21:02.727771+0000","last_became_active":"2024-09-13T17:50:34.204060+0000","last_became_peered":"2024-09-13T17:50:34.204060+0000","last_unstale":"2024-09-13T18:21:02.727771+0000","last_undegraded":"2024-09-13T18:21:02.727771+0000","last_fullsized":"2024-09-13T18:21:02.727771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:54:41.831344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'6031","reported_seq":6059,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.587876+0000","last_change":"2024-09-13T17:50:38.619590+0000","last_active":"2024-09-13T18:21:02.587876+0000","last_peered":"2024-09-13T18:21:02.587876+0000","last_clean":"2024-09-13T18:21:02.587876+0000","last_became_active":"2024-09-13T17:50:38.617257+0000","last_became_peered":"2024-09-13T17:50:38.617257+0000","last_unstale":"2024-09-13T18:21:02.587876+0000","last_undegraded":"2024-09-13T18:21:02.587876+0000","last_fullsized":"2024-09-13T18:21:02.587876+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6031,"log_dups_size":0,"ondisk_log_size":6031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:34.870291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25529,"num_write_kb":564991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'16","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310390+0000","last_change":"2024-09-13T17:50:34.588466+0000","last_active":"2024-09-13T18:21:13.310390+0000","last_peered":"2024-09-13T18:21:13.310390+0000","last_clean":"2024-09-13T18:21:13.310390+0000","last_became_active":"2024-09-13T17:50:34.588063+0000","last_became_peered":"2024-09-13T17:50:34.588063+0000","last_unstale":"2024-09-13T18:21:13.310390+0000","last_undegraded":"2024-09-13T18:21:13.310390+0000","last_fullsized":"2024-09-13T18:21:13.310390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:43:00.669203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'4","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.629940+0000","last_change":"2024-09-13T17:50:34.209239+0000","last_active":"2024-09-13T18:21:02.629940+0000","last_peered":"2024-09-13T18:21:02.629940+0000","last_clean":"2024-09-13T18:21:02.629940+0000","last_became_active":"2024-09-13T17:50:34.208196+0000","last_became_peered":"2024-09-13T17:50:34.208196+0000","last_unstale":"2024-09-13T18:21:02.629940+0000","last_undegraded":"2024-09-13T18:21:02.629940+0000","last_fullsized":"2024-09-13T18:21:02.629940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:39.253917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'5410","reported_seq":5438,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.883285+0000","last_change":"2024-09-13T17:50:38.611021+0000","last_active":"2024-09-13T18:21:02.883285+0000","last_peered":"2024-09-13T18:21:02.883285+0000","last_clean":"2024-09-13T18:21:02.883285+0000","last_became_active":"2024-09-13T17:50:38.609794+0000","last_became_peered":"2024-09-13T17:50:38.609794+0000","last_unstale":"2024-09-13T18:21:02.883285+0000","last_undegraded":"2024-09-13T18:21:02.883285+0000","last_fullsized":"2024-09-13T18:21:02.883285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5410,"log_dups_size":0,"ondisk_log_size":5410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:59.980236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22186,"num_write_kb":506224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'22","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.282060+0000","last_change":"2024-09-13T17:50:34.584290+0000","last_active":"2024-09-13T18:21:13.282060+0000","last_peered":"2024-09-13T18:21:13.282060+0000","last_clean":"2024-09-13T18:21:13.282060+0000","last_became_active":"2024-09-13T17:50:34.584023+0000","last_became_peered":"2024-09-13T17:50:34.584023+0000","last_unstale":"2024-09-13T18:21:13.282060+0000","last_undegraded":"2024-09-13T18:21:13.282060+0000","last_fullsized":"2024-09-13T18:21:13.282060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:11.304986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'127","reported_seq":279,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:31.126933+0000","last_change":"2024-09-13T17:50:34.198932+0000","last_active":"2024-09-13T18:20:31.126933+0000","last_peered":"2024-09-13T18:20:31.126933+0000","last_clean":"2024-09-13T18:20:31.126933+0000","last_became_active":"2024-09-13T17:50:34.198603+0000","last_became_peered":"2024-09-13T17:50:34.198603+0000","last_unstale":"2024-09-13T18:20:31.126933+0000","last_undegraded":"2024-09-13T18:20:31.126933+0000","last_fullsized":"2024-09-13T18:20:31.126933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:00.917126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":94032,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":179,"num_write":125,"num_write_kb":180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'5807","reported_seq":5835,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.679912+0000","last_change":"2024-09-13T17:50:38.611326+0000","last_active":"2024-09-13T18:21:02.679912+0000","last_peered":"2024-09-13T18:21:02.679912+0000","last_clean":"2024-09-13T18:21:02.679912+0000","last_became_active":"2024-09-13T17:50:38.610900+0000","last_became_peered":"2024-09-13T17:50:38.610900+0000","last_unstale":"2024-09-13T18:21:02.679912+0000","last_undegraded":"2024-09-13T18:21:02.679912+0000","last_fullsized":"2024-09-13T18:21:02.679912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5807,"log_dups_size":0,"ondisk_log_size":5807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:17:09.994736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24429,"num_write_kb":531605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'13","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310665+0000","last_change":"2024-09-13T17:50:34.594482+0000","last_active":"2024-09-13T18:21:13.310665+0000","last_peered":"2024-09-13T18:21:13.310665+0000","last_clean":"2024-09-13T18:21:13.310665+0000","last_became_active":"2024-09-13T17:50:34.594201+0000","last_became_peered":"2024-09-13T17:50:34.594201+0000","last_unstale":"2024-09-13T18:21:13.310665+0000","last_undegraded":"2024-09-13T18:21:13.310665+0000","last_fullsized":"2024-09-13T18:21:13.310665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:06.697507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'8","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.813884+0000","last_change":"2024-09-13T17:50:34.218376+0000","last_active":"2024-09-13T18:21:02.813884+0000","last_peered":"2024-09-13T18:21:02.813884+0000","last_clean":"2024-09-13T18:21:02.813884+0000","last_became_active":"2024-09-13T17:50:34.218069+0000","last_became_peered":"2024-09-13T17:50:34.218069+0000","last_unstale":"2024-09-13T18:21:02.813884+0000","last_undegraded":"2024-09-13T18:21:02.813884+0000","last_fullsized":"2024-09-13T18:21:02.813884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:53:54.288969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'4957","reported_seq":4985,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.963407+0000","last_change":"2024-09-13T17:50:38.613206+0000","last_active":"2024-09-13T18:21:02.963407+0000","last_peered":"2024-09-13T18:21:02.963407+0000","last_clean":"2024-09-13T18:21:02.963407+0000","last_became_active":"2024-09-13T17:50:38.612856+0000","last_became_peered":"2024-09-13T17:50:38.612856+0000","last_unstale":"2024-09-13T18:21:02.963407+0000","last_undegraded":"2024-09-13T18:21:02.963407+0000","last_fullsized":"2024-09-13T18:21:02.963407+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4957,"log_dups_size":0,"ondisk_log_size":4957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:31:20.190996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20520,"num_write_kb":434212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'16","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.316288+0000","last_change":"2024-09-13T17:50:34.582931+0000","last_active":"2024-09-13T18:21:13.316288+0000","last_peered":"2024-09-13T18:21:13.316288+0000","last_clean":"2024-09-13T18:21:13.316288+0000","last_became_active":"2024-09-13T17:50:34.582446+0000","last_became_peered":"2024-09-13T17:50:34.582446+0000","last_unstale":"2024-09-13T18:21:13.316288+0000","last_undegraded":"2024-09-13T18:21:13.316288+0000","last_fullsized":"2024-09-13T18:21:13.316288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:30.554785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'8","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.157427+0000","last_change":"2024-09-13T17:50:34.201237+0000","last_active":"2024-09-13T18:20:25.157427+0000","last_peered":"2024-09-13T18:20:25.157427+0000","last_clean":"2024-09-13T18:20:25.157427+0000","last_became_active":"2024-09-13T17:50:34.200873+0000","last_became_peered":"2024-09-13T17:50:34.200873+0000","last_unstale":"2024-09-13T18:20:25.157427+0000","last_undegraded":"2024-09-13T18:20:25.157427+0000","last_fullsized":"2024-09-13T18:20:25.157427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:02.711342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":185,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:18:31.266834+0000","last_change":"2024-09-13T17:50:22.095367+0000","last_active":"2024-09-13T18:18:31.266834+0000","last_peered":"2024-09-13T18:18:31.266834+0000","last_clean":"2024-09-13T18:18:31.266834+0000","last_became_active":"2024-09-13T17:50:22.093734+0000","last_became_peered":"2024-09-13T17:50:22.093734+0000","last_unstale":"2024-09-13T18:18:31.266834+0000","last_undegraded":"2024-09-13T18:18:31.266834+0000","last_fullsized":"2024-09-13T18:18:31.266834+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_clean_scrub_stamp":"2024-09-13T17:50:21.057812+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:23.531613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'5208","reported_seq":5236,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.886919+0000","last_change":"2024-09-13T17:50:38.618880+0000","last_active":"2024-09-13T18:21:02.886919+0000","last_peered":"2024-09-13T18:21:02.886919+0000","last_clean":"2024-09-13T18:21:02.886919+0000","last_became_active":"2024-09-13T17:50:38.618488+0000","last_became_peered":"2024-09-13T17:50:38.618488+0000","last_unstale":"2024-09-13T18:21:02.886919+0000","last_undegraded":"2024-09-13T18:21:02.886919+0000","last_fullsized":"2024-09-13T18:21:02.886919+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5208,"log_dups_size":0,"ondisk_log_size":5208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:03.431876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22093,"num_write_kb":481370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'22","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317778+0000","last_change":"2024-09-13T17:50:34.585881+0000","last_active":"2024-09-13T18:21:13.317778+0000","last_peered":"2024-09-13T18:21:13.317778+0000","last_clean":"2024-09-13T18:21:13.317778+0000","last_became_active":"2024-09-13T17:50:34.585166+0000","last_became_peered":"2024-09-13T17:50:34.585166+0000","last_unstale":"2024-09-13T18:21:13.317778+0000","last_undegraded":"2024-09-13T18:21:13.317778+0000","last_fullsized":"2024-09-13T18:21:13.317778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:06.654658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'5","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.654921+0000","last_change":"2024-09-13T17:50:34.210934+0000","last_active":"2024-09-13T18:21:02.654921+0000","last_peered":"2024-09-13T18:21:02.654921+0000","last_clean":"2024-09-13T18:21:02.654921+0000","last_became_active":"2024-09-13T17:50:34.210589+0000","last_became_peered":"2024-09-13T17:50:34.210589+0000","last_unstale":"2024-09-13T18:21:02.654921+0000","last_undegraded":"2024-09-13T18:21:02.654921+0000","last_fullsized":"2024-09-13T18:21:02.654921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:26.213295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'5302","reported_seq":5340,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.817379+0000","last_change":"2024-09-13T17:51:16.364911+0000","last_active":"2024-09-13T18:21:02.817379+0000","last_peered":"2024-09-13T18:21:02.817379+0000","last_clean":"2024-09-13T18:21:02.817379+0000","last_became_active":"2024-09-13T17:51:16.363974+0000","last_became_peered":"2024-09-13T17:51:16.363974+0000","last_unstale":"2024-09-13T18:21:02.817379+0000","last_undegraded":"2024-09-13T18:21:02.817379+0000","last_fullsized":"2024-09-13T18:21:02.817379+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5302,"log_dups_size":0,"ondisk_log_size":5302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:12.730197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22724,"num_write_kb":488088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'20","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.309412+0000","last_change":"2024-09-13T17:50:34.590169+0000","last_active":"2024-09-13T18:21:13.309412+0000","last_peered":"2024-09-13T18:21:13.309412+0000","last_clean":"2024-09-13T18:21:13.309412+0000","last_became_active":"2024-09-13T17:50:34.589434+0000","last_became_peered":"2024-09-13T17:50:34.589434+0000","last_unstale":"2024-09-13T18:21:13.309412+0000","last_undegraded":"2024-09-13T18:21:13.309412+0000","last_fullsized":"2024-09-13T18:21:13.309412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:12.545412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'4","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.155671+0000","last_change":"2024-09-13T17:50:34.204269+0000","last_active":"2024-09-13T18:20:25.155671+0000","last_peered":"2024-09-13T18:20:25.155671+0000","last_clean":"2024-09-13T18:20:25.155671+0000","last_became_active":"2024-09-13T17:50:34.203874+0000","last_became_peered":"2024-09-13T17:50:34.203874+0000","last_unstale":"2024-09-13T18:20:25.155671+0000","last_undegraded":"2024-09-13T18:20:25.155671+0000","last_fullsized":"2024-09-13T18:20:25.155671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:10.410267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'5348","reported_seq":5375,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.944893+0000","last_change":"2024-09-13T17:50:38.615989+0000","last_active":"2024-09-13T18:21:02.944893+0000","last_peered":"2024-09-13T18:21:02.944893+0000","last_clean":"2024-09-13T18:21:02.944893+0000","last_became_active":"2024-09-13T17:50:38.615530+0000","last_became_peered":"2024-09-13T17:50:38.615530+0000","last_unstale":"2024-09-13T18:21:02.944893+0000","last_undegraded":"2024-09-13T18:21:02.944893+0000","last_fullsized":"2024-09-13T18:21:02.944893+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5348,"log_dups_size":0,"ondisk_log_size":5348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:41.064092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22134,"num_write_kb":473293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'20","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.321742+0000","last_change":"2024-09-13T17:50:34.595120+0000","last_active":"2024-09-13T18:21:13.321742+0000","last_peered":"2024-09-13T18:21:13.321742+0000","last_clean":"2024-09-13T18:21:13.321742+0000","last_became_active":"2024-09-13T17:50:34.594842+0000","last_became_peered":"2024-09-13T17:50:34.594842+0000","last_unstale":"2024-09-13T18:21:13.321742+0000","last_undegraded":"2024-09-13T18:21:13.321742+0000","last_fullsized":"2024-09-13T18:21:13.321742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:15:38.982524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'9","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.683517+0000","last_change":"2024-09-13T17:50:34.199957+0000","last_active":"2024-09-13T18:21:02.683517+0000","last_peered":"2024-09-13T18:21:02.683517+0000","last_clean":"2024-09-13T18:21:02.683517+0000","last_became_active":"2024-09-13T17:50:34.199650+0000","last_became_peered":"2024-09-13T17:50:34.199650+0000","last_unstale":"2024-09-13T18:21:02.683517+0000","last_undegraded":"2024-09-13T18:21:02.683517+0000","last_fullsized":"2024-09-13T18:21:02.683517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:26.929549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'5161","reported_seq":5189,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.762344+0000","last_change":"2024-09-13T17:50:38.609731+0000","last_active":"2024-09-13T18:21:02.762344+0000","last_peered":"2024-09-13T18:21:02.762344+0000","last_clean":"2024-09-13T18:21:02.762344+0000","last_became_active":"2024-09-13T17:50:38.609354+0000","last_became_peered":"2024-09-13T17:50:38.609354+0000","last_unstale":"2024-09-13T18:21:02.762344+0000","last_undegraded":"2024-09-13T18:21:02.762344+0000","last_fullsized":"2024-09-13T18:21:02.762344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5161,"log_dups_size":0,"ondisk_log_size":5161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:14.142369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21929,"num_write_kb":464406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'10","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.313609+0000","last_change":"2024-09-13T17:50:34.595146+0000","last_active":"2024-09-13T18:21:13.313609+0000","last_peered":"2024-09-13T18:21:13.313609+0000","last_clean":"2024-09-13T18:21:13.313609+0000","last_became_active":"2024-09-13T17:50:34.594876+0000","last_became_peered":"2024-09-13T17:50:34.594876+0000","last_unstale":"2024-09-13T18:21:13.313609+0000","last_undegraded":"2024-09-13T18:21:13.313609+0000","last_fullsized":"2024-09-13T18:21:13.313609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:26:10.638438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'4","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:19:51.940271+0000","last_change":"2024-09-13T17:50:34.200640+0000","last_active":"2024-09-13T18:19:51.940271+0000","last_peered":"2024-09-13T18:19:51.940271+0000","last_clean":"2024-09-13T18:19:51.940271+0000","last_became_active":"2024-09-13T17:50:34.200197+0000","last_became_peered":"2024-09-13T17:50:34.200197+0000","last_unstale":"2024-09-13T18:19:51.940271+0000","last_undegraded":"2024-09-13T18:19:51.940271+0000","last_fullsized":"2024-09-13T18:19:51.940271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:37:53.039720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'5052","reported_seq":5090,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.778127+0000","last_change":"2024-09-13T17:51:16.281995+0000","last_active":"2024-09-13T18:21:02.778127+0000","last_peered":"2024-09-13T18:21:02.778127+0000","last_clean":"2024-09-13T18:21:02.778127+0000","last_became_active":"2024-09-13T17:51:16.281476+0000","last_became_peered":"2024-09-13T17:51:16.281476+0000","last_unstale":"2024-09-13T18:21:02.778127+0000","last_undegraded":"2024-09-13T18:21:02.778127+0000","last_fullsized":"2024-09-13T18:21:02.778127+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5052,"log_dups_size":0,"ondisk_log_size":5052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:21:56.254641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20895,"num_write_kb":464282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3,6],"acting":[4,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'18","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.301423+0000","last_change":"2024-09-13T17:50:34.585741+0000","last_active":"2024-09-13T18:21:13.301423+0000","last_peered":"2024-09-13T18:21:13.301423+0000","last_clean":"2024-09-13T18:21:13.301423+0000","last_became_active":"2024-09-13T17:50:34.585464+0000","last_became_peered":"2024-09-13T17:50:34.585464+0000","last_unstale":"2024-09-13T18:21:13.301423+0000","last_undegraded":"2024-09-13T18:21:13.301423+0000","last_fullsized":"2024-09-13T18:21:13.301423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:59:22.031459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'19","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.673198+0000","last_change":"2024-09-13T17:50:34.201272+0000","last_active":"2024-09-13T18:21:02.673198+0000","last_peered":"2024-09-13T18:21:02.673198+0000","last_clean":"2024-09-13T18:21:02.673198+0000","last_became_active":"2024-09-13T17:50:34.200936+0000","last_became_peered":"2024-09-13T17:50:34.200936+0000","last_unstale":"2024-09-13T18:21:02.673198+0000","last_undegraded":"2024-09-13T18:21:02.673198+0000","last_fullsized":"2024-09-13T18:21:02.673198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:52.094879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'6502","reported_seq":6528,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.900747+0000","last_change":"2024-09-13T17:50:38.615229+0000","last_active":"2024-09-13T18:21:02.900747+0000","last_peered":"2024-09-13T18:21:02.900747+0000","last_clean":"2024-09-13T18:21:02.900747+0000","last_became_active":"2024-09-13T17:50:38.614739+0000","last_became_peered":"2024-09-13T17:50:38.614739+0000","last_unstale":"2024-09-13T18:21:02.900747+0000","last_undegraded":"2024-09-13T18:21:02.900747+0000","last_fullsized":"2024-09-13T18:21:02.900747+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6502,"log_dups_size":0,"ondisk_log_size":6502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:18.619529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27696,"num_write_kb":544723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'20","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.287949+0000","last_change":"2024-09-13T17:50:34.588848+0000","last_active":"2024-09-13T18:21:13.287949+0000","last_peered":"2024-09-13T18:21:13.287949+0000","last_clean":"2024-09-13T18:21:13.287949+0000","last_became_active":"2024-09-13T17:50:34.588565+0000","last_became_peered":"2024-09-13T17:50:34.588565+0000","last_unstale":"2024-09-13T18:21:13.287949+0000","last_undegraded":"2024-09-13T18:21:13.287949+0000","last_fullsized":"2024-09-13T18:21:13.287949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:47.927865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'7","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.707815+0000","last_change":"2024-09-13T17:50:34.199008+0000","last_active":"2024-09-13T18:21:02.707815+0000","last_peered":"2024-09-13T18:21:02.707815+0000","last_clean":"2024-09-13T18:21:02.707815+0000","last_became_active":"2024-09-13T17:50:34.197839+0000","last_became_peered":"2024-09-13T17:50:34.197839+0000","last_unstale":"2024-09-13T18:21:02.707815+0000","last_undegraded":"2024-09-13T18:21:02.707815+0000","last_fullsized":"2024-09-13T18:21:02.707815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:51.430710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'6326","reported_seq":6352,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:01.698688+0000","last_change":"2024-09-13T17:50:38.611080+0000","last_active":"2024-09-13T18:21:01.698688+0000","last_peered":"2024-09-13T18:21:01.698688+0000","last_clean":"2024-09-13T18:21:01.698688+0000","last_became_active":"2024-09-13T17:50:38.610688+0000","last_became_peered":"2024-09-13T17:50:38.610688+0000","last_unstale":"2024-09-13T18:21:01.698688+0000","last_undegraded":"2024-09-13T18:21:01.698688+0000","last_fullsized":"2024-09-13T18:21:01.698688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6326,"log_dups_size":0,"ondisk_log_size":6326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:28.527617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27199,"num_write_kb":538670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'29","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317546+0000","last_change":"2024-09-13T17:50:34.594763+0000","last_active":"2024-09-13T18:21:13.317546+0000","last_peered":"2024-09-13T18:21:13.317546+0000","last_clean":"2024-09-13T18:21:13.317546+0000","last_became_active":"2024-09-13T17:50:34.594471+0000","last_became_peered":"2024-09-13T17:50:34.594471+0000","last_unstale":"2024-09-13T18:21:13.317546+0000","last_undegraded":"2024-09-13T18:21:13.317546+0000","last_fullsized":"2024-09-13T18:21:13.317546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:20:58.718271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'6","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.839570+0000","last_change":"2024-09-13T17:50:34.209432+0000","last_active":"2024-09-13T18:21:02.839570+0000","last_peered":"2024-09-13T18:21:02.839570+0000","last_clean":"2024-09-13T18:21:02.839570+0000","last_became_active":"2024-09-13T17:50:34.208817+0000","last_became_peered":"2024-09-13T17:50:34.208817+0000","last_unstale":"2024-09-13T18:21:02.839570+0000","last_undegraded":"2024-09-13T18:21:02.839570+0000","last_fullsized":"2024-09-13T18:21:02.839570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:49.587528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'5788","reported_seq":5815,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.947982+0000","last_change":"2024-09-13T17:50:38.609461+0000","last_active":"2024-09-13T18:21:02.947982+0000","last_peered":"2024-09-13T18:21:02.947982+0000","last_clean":"2024-09-13T18:21:02.947982+0000","last_became_active":"2024-09-13T17:50:38.609128+0000","last_became_peered":"2024-09-13T17:50:38.609128+0000","last_unstale":"2024-09-13T18:21:02.947982+0000","last_undegraded":"2024-09-13T18:21:02.947982+0000","last_fullsized":"2024-09-13T18:21:02.947982+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5788,"log_dups_size":0,"ondisk_log_size":5788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:02.286825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24140,"num_write_kb":538739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'24","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.305222+0000","last_change":"2024-09-13T17:50:34.592970+0000","last_active":"2024-09-13T18:21:13.305222+0000","last_peered":"2024-09-13T18:21:13.305222+0000","last_clean":"2024-09-13T18:21:13.305222+0000","last_became_active":"2024-09-13T17:50:34.592669+0000","last_became_peered":"2024-09-13T17:50:34.592669+0000","last_unstale":"2024-09-13T18:21:13.305222+0000","last_undegraded":"2024-09-13T18:21:13.305222+0000","last_fullsized":"2024-09-13T18:21:13.305222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:52.031261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'15","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.785525+0000","last_change":"2024-09-13T17:50:34.200798+0000","last_active":"2024-09-13T18:21:02.785525+0000","last_peered":"2024-09-13T18:21:02.785525+0000","last_clean":"2024-09-13T18:21:02.785525+0000","last_became_active":"2024-09-13T17:50:34.200529+0000","last_became_peered":"2024-09-13T17:50:34.200529+0000","last_unstale":"2024-09-13T18:21:02.785525+0000","last_undegraded":"2024-09-13T18:21:02.785525+0000","last_fullsized":"2024-09-13T18:21:02.785525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:26.334296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'5464","reported_seq":5490,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.598966+0000","last_change":"2024-09-13T17:50:38.606287+0000","last_active":"2024-09-13T18:21:02.598966+0000","last_peered":"2024-09-13T18:21:02.598966+0000","last_clean":"2024-09-13T18:21:02.598966+0000","last_became_active":"2024-09-13T17:50:38.605826+0000","last_became_peered":"2024-09-13T17:50:38.605826+0000","last_unstale":"2024-09-13T18:21:02.598966+0000","last_undegraded":"2024-09-13T18:21:02.598966+0000","last_fullsized":"2024-09-13T18:21:02.598966+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5464,"log_dups_size":0,"ondisk_log_size":5464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:18.709454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23044,"num_write_kb":472739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'18","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310869+0000","last_change":"2024-09-13T17:50:34.591579+0000","last_active":"2024-09-13T18:21:13.310869+0000","last_peered":"2024-09-13T18:21:13.310869+0000","last_clean":"2024-09-13T18:21:13.310869+0000","last_became_active":"2024-09-13T17:50:34.586165+0000","last_became_peered":"2024-09-13T17:50:34.586165+0000","last_unstale":"2024-09-13T18:21:13.310869+0000","last_undegraded":"2024-09-13T18:21:13.310869+0000","last_fullsized":"2024-09-13T18:21:13.310869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:56:19.205176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'417","reported_seq":447,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.705527+0000","last_change":"2024-09-13T17:50:34.203218+0000","last_active":"2024-09-13T18:21:02.705527+0000","last_peered":"2024-09-13T18:21:02.705527+0000","last_clean":"2024-09-13T18:21:02.705527+0000","last_became_active":"2024-09-13T17:50:34.202948+0000","last_became_peered":"2024-09-13T17:50:34.202948+0000","last_unstale":"2024-09-13T18:21:02.705527+0000","last_undegraded":"2024-09-13T18:21:02.705527+0000","last_fullsized":"2024-09-13T18:21:02.705527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:46.535569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":4324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'5887","reported_seq":5914,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.956090+0000","last_change":"2024-09-13T17:50:38.610597+0000","last_active":"2024-09-13T18:21:02.956090+0000","last_peered":"2024-09-13T18:21:02.956090+0000","last_clean":"2024-09-13T18:21:02.956090+0000","last_became_active":"2024-09-13T17:50:38.610055+0000","last_became_peered":"2024-09-13T17:50:38.610055+0000","last_unstale":"2024-09-13T18:21:02.956090+0000","last_undegraded":"2024-09-13T18:21:02.956090+0000","last_fullsized":"2024-09-13T18:21:02.956090+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5887,"log_dups_size":0,"ondisk_log_size":5887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:55.754936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24664,"num_write_kb":495799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'21","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317972+0000","last_change":"2024-09-13T17:50:34.593963+0000","last_active":"2024-09-13T18:21:13.317972+0000","last_peered":"2024-09-13T18:21:13.317972+0000","last_clean":"2024-09-13T18:21:13.317972+0000","last_became_active":"2024-09-13T17:50:34.593454+0000","last_became_peered":"2024-09-13T17:50:34.593454+0000","last_unstale":"2024-09-13T18:21:13.317972+0000","last_undegraded":"2024-09-13T18:21:13.317972+0000","last_fullsized":"2024-09-13T18:21:13.317972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:18.883521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'67","reported_seq":96,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.464353+0000","last_change":"2024-09-13T17:50:34.207936+0000","last_active":"2024-09-13T18:21:02.464353+0000","last_peered":"2024-09-13T18:21:02.464353+0000","last_clean":"2024-09-13T18:21:02.464353+0000","last_became_active":"2024-09-13T17:50:34.207605+0000","last_became_peered":"2024-09-13T17:50:34.207605+0000","last_unstale":"2024-09-13T18:21:02.464353+0000","last_undegraded":"2024-09-13T18:21:02.464353+0000","last_fullsized":"2024-09-13T18:21:02.464353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:05.494406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'5937","reported_seq":5964,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.965268+0000","last_change":"2024-09-13T17:50:38.616297+0000","last_active":"2024-09-13T18:21:02.965268+0000","last_peered":"2024-09-13T18:21:02.965268+0000","last_clean":"2024-09-13T18:21:02.965268+0000","last_became_active":"2024-09-13T17:50:38.614165+0000","last_became_peered":"2024-09-13T17:50:38.614165+0000","last_unstale":"2024-09-13T18:21:02.965268+0000","last_undegraded":"2024-09-13T18:21:02.965268+0000","last_fullsized":"2024-09-13T18:21:02.965268+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5937,"log_dups_size":0,"ondisk_log_size":5937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:11.357026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25111,"num_write_kb":509795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'25","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.311085+0000","last_change":"2024-09-13T17:50:34.583001+0000","last_active":"2024-09-13T18:21:13.311085+0000","last_peered":"2024-09-13T18:21:13.311085+0000","last_clean":"2024-09-13T18:21:13.311085+0000","last_became_active":"2024-09-13T17:50:34.582529+0000","last_became_peered":"2024-09-13T17:50:34.582529+0000","last_unstale":"2024-09-13T18:21:13.311085+0000","last_undegraded":"2024-09-13T18:21:13.311085+0000","last_fullsized":"2024-09-13T18:21:13.311085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:05.287181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'5","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:53.035777+0000","last_change":"2024-09-13T17:50:34.205138+0000","last_active":"2024-09-13T18:20:53.035777+0000","last_peered":"2024-09-13T18:20:53.035777+0000","last_clean":"2024-09-13T18:20:53.035777+0000","last_became_active":"2024-09-13T17:50:34.203360+0000","last_became_peered":"2024-09-13T17:50:34.203360+0000","last_unstale":"2024-09-13T18:20:53.035777+0000","last_undegraded":"2024-09-13T18:20:53.035777+0000","last_fullsized":"2024-09-13T18:20:53.035777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:35.756963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'5934","reported_seq":5961,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.811068+0000","last_change":"2024-09-13T17:50:38.611036+0000","last_active":"2024-09-13T18:21:02.811068+0000","last_peered":"2024-09-13T18:21:02.811068+0000","last_clean":"2024-09-13T18:21:02.811068+0000","last_became_active":"2024-09-13T17:50:38.610692+0000","last_became_peered":"2024-09-13T17:50:38.610692+0000","last_unstale":"2024-09-13T18:21:02.811068+0000","last_undegraded":"2024-09-13T18:21:02.811068+0000","last_fullsized":"2024-09-13T18:21:02.811068+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5934,"log_dups_size":0,"ondisk_log_size":5934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:10.147998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25100,"num_write_kb":497535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'23","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.311752+0000","last_change":"2024-09-13T17:51:16.365295+0000","last_active":"2024-09-13T18:21:13.311752+0000","last_peered":"2024-09-13T18:21:13.311752+0000","last_clean":"2024-09-13T18:21:13.311752+0000","last_became_active":"2024-09-13T17:51:16.364437+0000","last_became_peered":"2024-09-13T17:51:16.364437+0000","last_unstale":"2024-09-13T18:21:13.311752+0000","last_undegraded":"2024-09-13T18:21:13.311752+0000","last_fullsized":"2024-09-13T18:21:13.311752+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:23.208899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"27'10","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.645634+0000","last_change":"2024-09-13T17:50:34.204376+0000","last_active":"2024-09-13T18:21:02.645634+0000","last_peered":"2024-09-13T18:21:02.645634+0000","last_clean":"2024-09-13T18:21:02.645634+0000","last_became_active":"2024-09-13T17:50:34.204003+0000","last_became_peered":"2024-09-13T17:50:34.204003+0000","last_unstale":"2024-09-13T18:21:02.645634+0000","last_undegraded":"2024-09-13T18:21:02.645634+0000","last_fullsized":"2024-09-13T18:21:02.645634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:14.023990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'5363","reported_seq":5390,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.796875+0000","last_change":"2024-09-13T17:50:38.606144+0000","last_active":"2024-09-13T18:21:02.796875+0000","last_peered":"2024-09-13T18:21:02.796875+0000","last_clean":"2024-09-13T18:21:02.796875+0000","last_became_active":"2024-09-13T17:50:38.605646+0000","last_became_peered":"2024-09-13T17:50:38.605646+0000","last_unstale":"2024-09-13T18:21:02.796875+0000","last_undegraded":"2024-09-13T18:21:02.796875+0000","last_fullsized":"2024-09-13T18:21:02.796875+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5363,"log_dups_size":0,"ondisk_log_size":5363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:07.120075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22831,"num_write_kb":508903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'6067","reported_seq":6096,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.803568+0000","last_change":"2024-09-13T17:50:38.618855+0000","last_active":"2024-09-13T18:21:02.803568+0000","last_peered":"2024-09-13T18:21:02.803568+0000","last_clean":"2024-09-13T18:21:02.803568+0000","last_became_active":"2024-09-13T17:50:38.618449+0000","last_became_peered":"2024-09-13T17:50:38.618449+0000","last_unstale":"2024-09-13T18:21:02.803568+0000","last_undegraded":"2024-09-13T18:21:02.803568+0000","last_fullsized":"2024-09-13T18:21:02.803568+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6067,"log_dups_size":0,"ondisk_log_size":6067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:29:01.114087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25532,"num_write_kb":568825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'21","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.298787+0000","last_change":"2024-09-13T17:50:34.590079+0000","last_active":"2024-09-13T18:21:13.298787+0000","last_peered":"2024-09-13T18:21:13.298787+0000","last_clean":"2024-09-13T18:21:13.298787+0000","last_became_active":"2024-09-13T17:50:34.589396+0000","last_became_peered":"2024-09-13T17:50:34.589396+0000","last_unstale":"2024-09-13T18:21:13.298787+0000","last_undegraded":"2024-09-13T18:21:13.298787+0000","last_fullsized":"2024-09-13T18:21:13.298787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:03.308341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:17:11.375959+0000","last_change":"2024-09-13T17:50:34.206575+0000","last_active":"2024-09-13T18:17:11.375959+0000","last_peered":"2024-09-13T18:17:11.375959+0000","last_clean":"2024-09-13T18:17:11.375959+0000","last_became_active":"2024-09-13T17:50:34.206308+0000","last_became_peered":"2024-09-13T17:50:34.206308+0000","last_unstale":"2024-09-13T18:17:11.375959+0000","last_undegraded":"2024-09-13T18:17:11.375959+0000","last_fullsized":"2024-09-13T18:17:11.375959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:34.766538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'5881","reported_seq":5909,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.927672+0000","last_change":"2024-09-13T17:50:38.618131+0000","last_active":"2024-09-13T18:21:02.927672+0000","last_peered":"2024-09-13T18:21:02.927672+0000","last_clean":"2024-09-13T18:21:02.927672+0000","last_became_active":"2024-09-13T17:50:38.617611+0000","last_became_peered":"2024-09-13T17:50:38.617611+0000","last_unstale":"2024-09-13T18:21:02.927672+0000","last_undegraded":"2024-09-13T18:21:02.927672+0000","last_fullsized":"2024-09-13T18:21:02.927672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5881,"log_dups_size":0,"ondisk_log_size":5881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:30.746445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25019,"num_write_kb":530139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'27","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.307795+0000","last_change":"2024-09-13T17:50:34.582153+0000","last_active":"2024-09-13T18:21:13.307795+0000","last_peered":"2024-09-13T18:21:13.307795+0000","last_clean":"2024-09-13T18:21:13.307795+0000","last_became_active":"2024-09-13T17:50:34.581708+0000","last_became_peered":"2024-09-13T17:50:34.581708+0000","last_unstale":"2024-09-13T18:21:13.307795+0000","last_undegraded":"2024-09-13T18:21:13.307795+0000","last_fullsized":"2024-09-13T18:21:13.307795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:22.238810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'4","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:19:47.928146+0000","last_change":"2024-09-13T17:50:34.205489+0000","last_active":"2024-09-13T18:19:47.928146+0000","last_peered":"2024-09-13T18:19:47.928146+0000","last_clean":"2024-09-13T18:19:47.928146+0000","last_became_active":"2024-09-13T17:50:34.205202+0000","last_became_peered":"2024-09-13T17:50:34.205202+0000","last_unstale":"2024-09-13T18:19:47.928146+0000","last_undegraded":"2024-09-13T18:19:47.928146+0000","last_fullsized":"2024-09-13T18:19:47.928146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:54.450276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'6","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.825727+0000","last_change":"2024-09-13T17:50:34.202470+0000","last_active":"2024-09-13T18:21:02.825727+0000","last_peered":"2024-09-13T18:21:02.825727+0000","last_clean":"2024-09-13T18:21:02.825727+0000","last_became_active":"2024-09-13T17:50:34.202163+0000","last_became_peered":"2024-09-13T17:50:34.202163+0000","last_unstale":"2024-09-13T18:21:02.825727+0000","last_undegraded":"2024-09-13T18:21:02.825727+0000","last_fullsized":"2024-09-13T18:21:02.825727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:03.472932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'18","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312647+0000","last_change":"2024-09-13T17:50:34.590947+0000","last_active":"2024-09-13T18:21:13.312647+0000","last_peered":"2024-09-13T18:21:13.312647+0000","last_clean":"2024-09-13T18:21:13.312647+0000","last_became_active":"2024-09-13T17:50:34.590497+0000","last_became_peered":"2024-09-13T17:50:34.590497+0000","last_unstale":"2024-09-13T18:21:13.312647+0000","last_undegraded":"2024-09-13T18:21:13.312647+0000","last_fullsized":"2024-09-13T18:21:13.312647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:56:27.848092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'4651","reported_seq":4677,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.602408+0000","last_change":"2024-09-13T17:50:38.618518+0000","last_active":"2024-09-13T18:21:02.602408+0000","last_peered":"2024-09-13T18:21:02.602408+0000","last_clean":"2024-09-13T18:21:02.602408+0000","last_became_active":"2024-09-13T17:50:38.618108+0000","last_became_peered":"2024-09-13T17:50:38.618108+0000","last_unstale":"2024-09-13T18:21:02.602408+0000","last_undegraded":"2024-09-13T18:21:02.602408+0000","last_fullsized":"2024-09-13T18:21:02.602408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4651,"log_dups_size":0,"ondisk_log_size":4651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:42.260237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19488,"num_write_kb":413684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'690","reported_seq":718,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.791718+0000","last_change":"2024-09-13T17:50:34.218459+0000","last_active":"2024-09-13T18:21:02.791718+0000","last_peered":"2024-09-13T18:21:02.791718+0000","last_clean":"2024-09-13T18:21:02.791718+0000","last_became_active":"2024-09-13T17:50:34.218087+0000","last_became_peered":"2024-09-13T17:50:34.218087+0000","last_unstale":"2024-09-13T18:21:02.791718+0000","last_undegraded":"2024-09-13T18:21:02.791718+0000","last_fullsized":"2024-09-13T18:21:02.791718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:56.914544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195461,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'20","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.315199+0000","last_change":"2024-09-13T17:50:34.583845+0000","last_active":"2024-09-13T18:21:13.315199+0000","last_peered":"2024-09-13T18:21:13.315199+0000","last_clean":"2024-09-13T18:21:13.315199+0000","last_became_active":"2024-09-13T17:50:34.583283+0000","last_became_peered":"2024-09-13T17:50:34.583283+0000","last_unstale":"2024-09-13T18:21:13.315199+0000","last_undegraded":"2024-09-13T18:21:13.315199+0000","last_fullsized":"2024-09-13T18:21:13.315199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:55.630285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'5883","reported_seq":5909,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.716917+0000","last_change":"2024-09-13T17:50:38.611987+0000","last_active":"2024-09-13T18:21:02.716917+0000","last_peered":"2024-09-13T18:21:02.716917+0000","last_clean":"2024-09-13T18:21:02.716917+0000","last_became_active":"2024-09-13T17:50:38.611636+0000","last_became_peered":"2024-09-13T17:50:38.611636+0000","last_unstale":"2024-09-13T18:21:02.716917+0000","last_undegraded":"2024-09-13T18:21:02.716917+0000","last_fullsized":"2024-09-13T18:21:02.716917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5883,"log_dups_size":0,"ondisk_log_size":5883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:41.111562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24816,"num_write_kb":545450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'25","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.300796+0000","last_change":"2024-09-13T17:50:34.588927+0000","last_active":"2024-09-13T18:21:13.300796+0000","last_peered":"2024-09-13T18:21:13.300796+0000","last_clean":"2024-09-13T18:21:13.300796+0000","last_became_active":"2024-09-13T17:50:34.588651+0000","last_became_peered":"2024-09-13T17:50:34.588651+0000","last_unstale":"2024-09-13T18:21:13.300796+0000","last_undegraded":"2024-09-13T18:21:13.300796+0000","last_fullsized":"2024-09-13T18:21:13.300796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:49.429670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'24","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310371+0000","last_change":"2024-09-13T17:50:34.583795+0000","last_active":"2024-09-13T18:21:13.310371+0000","last_peered":"2024-09-13T18:21:13.310371+0000","last_clean":"2024-09-13T18:21:13.310371+0000","last_became_active":"2024-09-13T17:50:34.583493+0000","last_became_peered":"2024-09-13T17:50:34.583493+0000","last_unstale":"2024-09-13T18:21:13.310371+0000","last_undegraded":"2024-09-13T18:21:13.310371+0000","last_fullsized":"2024-09-13T18:21:13.310371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:30.463932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'25","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.314410+0000","last_change":"2024-09-13T17:50:34.587624+0000","last_active":"2024-09-13T18:21:13.314410+0000","last_peered":"2024-09-13T18:21:13.314410+0000","last_clean":"2024-09-13T18:21:13.314410+0000","last_became_active":"2024-09-13T17:50:34.587119+0000","last_became_peered":"2024-09-13T17:50:34.587119+0000","last_unstale":"2024-09-13T18:21:13.314410+0000","last_undegraded":"2024-09-13T18:21:13.314410+0000","last_fullsized":"2024-09-13T18:21:13.314410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:16.783899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'17","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.308557+0000","last_change":"2024-09-13T17:50:34.581923+0000","last_active":"2024-09-13T18:21:13.308557+0000","last_peered":"2024-09-13T18:21:13.308557+0000","last_clean":"2024-09-13T18:21:13.308557+0000","last_became_active":"2024-09-13T17:50:34.581628+0000","last_became_peered":"2024-09-13T17:50:34.581628+0000","last_unstale":"2024-09-13T18:21:13.308557+0000","last_undegraded":"2024-09-13T18:21:13.308557+0000","last_fullsized":"2024-09-13T18:21:13.308557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:28:55.936864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'14","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312666+0000","last_change":"2024-09-13T17:50:34.595037+0000","last_active":"2024-09-13T18:21:13.312666+0000","last_peered":"2024-09-13T18:21:13.312666+0000","last_clean":"2024-09-13T18:21:13.312666+0000","last_became_active":"2024-09-13T17:50:34.594857+0000","last_became_peered":"2024-09-13T17:50:34.594857+0000","last_unstale":"2024-09-13T18:21:13.312666+0000","last_undegraded":"2024-09-13T18:21:13.312666+0000","last_fullsized":"2024-09-13T18:21:13.312666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:11.420413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'15","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.298148+0000","last_change":"2024-09-13T17:50:34.589651+0000","last_active":"2024-09-13T18:21:13.298148+0000","last_peered":"2024-09-13T18:21:13.298148+0000","last_clean":"2024-09-13T18:21:13.298148+0000","last_became_active":"2024-09-13T17:50:34.589354+0000","last_became_peered":"2024-09-13T17:50:34.589354+0000","last_unstale":"2024-09-13T18:21:13.298148+0000","last_undegraded":"2024-09-13T18:21:13.298148+0000","last_fullsized":"2024-09-13T18:21:13.298148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:31.294048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'24","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317654+0000","last_change":"2024-09-13T17:50:34.597949+0000","last_active":"2024-09-13T18:21:13.317654+0000","last_peered":"2024-09-13T18:21:13.317654+0000","last_clean":"2024-09-13T18:21:13.317654+0000","last_became_active":"2024-09-13T17:50:34.597621+0000","last_became_peered":"2024-09-13T17:50:34.597621+0000","last_unstale":"2024-09-13T18:21:13.317654+0000","last_undegraded":"2024-09-13T18:21:13.317654+0000","last_fullsized":"2024-09-13T18:21:13.317654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:23.133462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'15","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312343+0000","last_change":"2024-09-13T17:50:34.589893+0000","last_active":"2024-09-13T18:21:13.312343+0000","last_peered":"2024-09-13T18:21:13.312343+0000","last_clean":"2024-09-13T18:21:13.312343+0000","last_became_active":"2024-09-13T17:50:34.589626+0000","last_became_peered":"2024-09-13T17:50:34.589626+0000","last_unstale":"2024-09-13T18:21:13.312343+0000","last_undegraded":"2024-09-13T18:21:13.312343+0000","last_fullsized":"2024-09-13T18:21:13.312343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:41.321025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'16","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.308839+0000","last_change":"2024-09-13T17:50:34.590046+0000","last_active":"2024-09-13T18:21:13.308839+0000","last_peered":"2024-09-13T18:21:13.308839+0000","last_clean":"2024-09-13T18:21:13.308839+0000","last_became_active":"2024-09-13T17:50:34.589757+0000","last_became_peered":"2024-09-13T17:50:34.589757+0000","last_unstale":"2024-09-13T18:21:13.308839+0000","last_undegraded":"2024-09-13T18:21:13.308839+0000","last_fullsized":"2024-09-13T18:21:13.308839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:28.239550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'23","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312608+0000","last_change":"2024-09-13T17:50:34.595518+0000","last_active":"2024-09-13T18:21:13.312608+0000","last_peered":"2024-09-13T18:21:13.312608+0000","last_clean":"2024-09-13T18:21:13.312608+0000","last_became_active":"2024-09-13T17:50:34.595242+0000","last_became_peered":"2024-09-13T17:50:34.595242+0000","last_unstale":"2024-09-13T18:21:13.312608+0000","last_undegraded":"2024-09-13T18:21:13.312608+0000","last_fullsized":"2024-09-13T18:21:13.312608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:26.149384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'21","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.315503+0000","last_change":"2024-09-13T17:50:34.593183+0000","last_active":"2024-09-13T18:21:13.315503+0000","last_peered":"2024-09-13T18:21:13.315503+0000","last_clean":"2024-09-13T18:21:13.315503+0000","last_became_active":"2024-09-13T17:50:34.592893+0000","last_became_peered":"2024-09-13T17:50:34.592893+0000","last_unstale":"2024-09-13T18:21:13.315503+0000","last_undegraded":"2024-09-13T18:21:13.315503+0000","last_fullsized":"2024-09-13T18:21:13.315503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:19.459505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'29","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.311869+0000","last_change":"2024-09-13T17:50:34.594679+0000","last_active":"2024-09-13T18:21:13.311869+0000","last_peered":"2024-09-13T18:21:13.311869+0000","last_clean":"2024-09-13T18:21:13.311869+0000","last_became_active":"2024-09-13T17:50:34.593750+0000","last_became_peered":"2024-09-13T17:50:34.593750+0000","last_unstale":"2024-09-13T18:21:13.311869+0000","last_undegraded":"2024-09-13T18:21:13.311869+0000","last_fullsized":"2024-09-13T18:21:13.311869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:02.135577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'19","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.313398+0000","last_change":"2024-09-13T17:50:34.598736+0000","last_active":"2024-09-13T18:21:13.313398+0000","last_peered":"2024-09-13T18:21:13.313398+0000","last_clean":"2024-09-13T18:21:13.313398+0000","last_became_active":"2024-09-13T17:50:34.598433+0000","last_became_peered":"2024-09-13T17:50:34.598433+0000","last_unstale":"2024-09-13T18:21:13.313398+0000","last_undegraded":"2024-09-13T18:21:13.313398+0000","last_fullsized":"2024-09-13T18:21:13.313398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:08.975139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'22","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.301045+0000","last_change":"2024-09-13T17:50:34.591795+0000","last_active":"2024-09-13T18:21:13.301045+0000","last_peered":"2024-09-13T18:21:13.301045+0000","last_clean":"2024-09-13T18:21:13.301045+0000","last_became_active":"2024-09-13T17:50:34.591286+0000","last_became_peered":"2024-09-13T17:50:34.591286+0000","last_unstale":"2024-09-13T18:21:13.301045+0000","last_undegraded":"2024-09-13T18:21:13.301045+0000","last_fullsized":"2024-09-13T18:21:13.301045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:31.580247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'28","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.300800+0000","last_change":"2024-09-13T17:50:34.587774+0000","last_active":"2024-09-13T18:21:13.300800+0000","last_peered":"2024-09-13T18:21:13.300800+0000","last_clean":"2024-09-13T18:21:13.300800+0000","last_became_active":"2024-09-13T17:50:34.587506+0000","last_became_peered":"2024-09-13T17:50:34.587506+0000","last_unstale":"2024-09-13T18:21:13.300800+0000","last_undegraded":"2024-09-13T18:21:13.300800+0000","last_fullsized":"2024-09-13T18:21:13.300800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:46.275528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'22","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.318554+0000","last_change":"2024-09-13T17:50:34.596100+0000","last_active":"2024-09-13T18:21:13.318554+0000","last_peered":"2024-09-13T18:21:13.318554+0000","last_clean":"2024-09-13T18:21:13.318554+0000","last_became_active":"2024-09-13T17:50:34.595798+0000","last_became_peered":"2024-09-13T17:50:34.595798+0000","last_unstale":"2024-09-13T18:21:13.318554+0000","last_undegraded":"2024-09-13T18:21:13.318554+0000","last_fullsized":"2024-09-13T18:21:13.318554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:49.543845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'21","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306399+0000","last_change":"2024-09-13T17:50:34.584818+0000","last_active":"2024-09-13T18:21:13.306399+0000","last_peered":"2024-09-13T18:21:13.306399+0000","last_clean":"2024-09-13T18:21:13.306399+0000","last_became_active":"2024-09-13T17:50:34.584516+0000","last_became_peered":"2024-09-13T17:50:34.584516+0000","last_unstale":"2024-09-13T18:21:13.306399+0000","last_undegraded":"2024-09-13T18:21:13.306399+0000","last_fullsized":"2024-09-13T18:21:13.306399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:52.402024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'23","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.292739+0000","last_change":"2024-09-13T17:50:34.598886+0000","last_active":"2024-09-13T18:21:13.292739+0000","last_peered":"2024-09-13T18:21:13.292739+0000","last_clean":"2024-09-13T18:21:13.292739+0000","last_became_active":"2024-09-13T17:50:34.598648+0000","last_became_peered":"2024-09-13T17:50:34.598648+0000","last_unstale":"2024-09-13T18:21:13.292739+0000","last_undegraded":"2024-09-13T18:21:13.292739+0000","last_fullsized":"2024-09-13T18:21:13.292739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:00:43.063223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'24","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.302458+0000","last_change":"2024-09-13T17:50:34.582649+0000","last_active":"2024-09-13T18:21:13.302458+0000","last_peered":"2024-09-13T18:21:13.302458+0000","last_clean":"2024-09-13T18:21:13.302458+0000","last_became_active":"2024-09-13T17:50:34.581699+0000","last_became_peered":"2024-09-13T17:50:34.581699+0000","last_unstale":"2024-09-13T18:21:13.302458+0000","last_undegraded":"2024-09-13T18:21:13.302458+0000","last_fullsized":"2024-09-13T18:21:13.302458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:18.959303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'23","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306620+0000","last_change":"2024-09-13T17:50:34.587484+0000","last_active":"2024-09-13T18:21:13.306620+0000","last_peered":"2024-09-13T18:21:13.306620+0000","last_clean":"2024-09-13T18:21:13.306620+0000","last_became_active":"2024-09-13T17:50:34.587217+0000","last_became_peered":"2024-09-13T17:50:34.587217+0000","last_unstale":"2024-09-13T18:21:13.306620+0000","last_undegraded":"2024-09-13T18:21:13.306620+0000","last_fullsized":"2024-09-13T18:21:13.306620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:24:19.172607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'18","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.315670+0000","last_change":"2024-09-13T17:50:34.592871+0000","last_active":"2024-09-13T18:21:13.315670+0000","last_peered":"2024-09-13T18:21:13.315670+0000","last_clean":"2024-09-13T18:21:13.315670+0000","last_became_active":"2024-09-13T17:50:34.588513+0000","last_became_peered":"2024-09-13T17:50:34.588513+0000","last_unstale":"2024-09-13T18:21:13.315670+0000","last_undegraded":"2024-09-13T18:21:13.315670+0000","last_fullsized":"2024-09-13T18:21:13.315670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:33.131420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'19","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.319025+0000","last_change":"2024-09-13T17:50:34.585713+0000","last_active":"2024-09-13T18:21:13.319025+0000","last_peered":"2024-09-13T18:21:13.319025+0000","last_clean":"2024-09-13T18:21:13.319025+0000","last_became_active":"2024-09-13T17:50:34.585438+0000","last_became_peered":"2024-09-13T17:50:34.585438+0000","last_unstale":"2024-09-13T18:21:13.319025+0000","last_undegraded":"2024-09-13T18:21:13.319025+0000","last_fullsized":"2024-09-13T18:21:13.319025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:56:49.905393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'30","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312596+0000","last_change":"2024-09-13T17:50:34.589628+0000","last_active":"2024-09-13T18:21:13.312596+0000","last_peered":"2024-09-13T18:21:13.312596+0000","last_clean":"2024-09-13T18:21:13.312596+0000","last_became_active":"2024-09-13T17:50:34.587665+0000","last_became_peered":"2024-09-13T17:50:34.587665+0000","last_unstale":"2024-09-13T18:21:13.312596+0000","last_undegraded":"2024-09-13T18:21:13.312596+0000","last_fullsized":"2024-09-13T18:21:13.312596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:15.518264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'27","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.321326+0000","last_change":"2024-09-13T17:50:34.591776+0000","last_active":"2024-09-13T18:21:13.321326+0000","last_peered":"2024-09-13T18:21:13.321326+0000","last_clean":"2024-09-13T18:21:13.321326+0000","last_became_active":"2024-09-13T17:50:34.591496+0000","last_became_peered":"2024-09-13T17:50:34.591496+0000","last_unstale":"2024-09-13T18:21:13.321326+0000","last_undegraded":"2024-09-13T18:21:13.321326+0000","last_fullsized":"2024-09-13T18:21:13.321326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:48.651999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'15","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310654+0000","last_change":"2024-09-13T17:51:16.282372+0000","last_active":"2024-09-13T18:21:13.310654+0000","last_peered":"2024-09-13T18:21:13.310654+0000","last_clean":"2024-09-13T18:21:13.310654+0000","last_became_active":"2024-09-13T17:51:16.282065+0000","last_became_peered":"2024-09-13T17:51:16.282065+0000","last_unstale":"2024-09-13T18:21:13.310654+0000","last_undegraded":"2024-09-13T18:21:13.310654+0000","last_fullsized":"2024-09-13T18:21:13.310654+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:49.291224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"27'14","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310971+0000","last_change":"2024-09-13T17:50:34.585531+0000","last_active":"2024-09-13T18:21:13.310971+0000","last_peered":"2024-09-13T18:21:13.310971+0000","last_clean":"2024-09-13T18:21:13.310971+0000","last_became_active":"2024-09-13T17:50:34.585226+0000","last_became_peered":"2024-09-13T17:50:34.585226+0000","last_unstale":"2024-09-13T18:21:13.310971+0000","last_undegraded":"2024-09-13T18:21:13.310971+0000","last_fullsized":"2024-09-13T18:21:13.310971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:17.105028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'17","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.314101+0000","last_change":"2024-09-13T17:50:34.586750+0000","last_active":"2024-09-13T18:21:13.314101+0000","last_peered":"2024-09-13T18:21:13.314101+0000","last_clean":"2024-09-13T18:21:13.314101+0000","last_became_active":"2024-09-13T17:50:34.584564+0000","last_became_peered":"2024-09-13T17:50:34.584564+0000","last_unstale":"2024-09-13T18:21:13.314101+0000","last_undegraded":"2024-09-13T18:21:13.314101+0000","last_fullsized":"2024-09-13T18:21:13.314101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:54.033257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'18","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.307217+0000","last_change":"2024-09-13T17:50:34.585938+0000","last_active":"2024-09-13T18:21:13.307217+0000","last_peered":"2024-09-13T18:21:13.307217+0000","last_clean":"2024-09-13T18:21:13.307217+0000","last_became_active":"2024-09-13T17:50:34.585584+0000","last_became_peered":"2024-09-13T17:50:34.585584+0000","last_unstale":"2024-09-13T18:21:13.307217+0000","last_undegraded":"2024-09-13T18:21:13.307217+0000","last_fullsized":"2024-09-13T18:21:13.307217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:32.623468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'15","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310571+0000","last_change":"2024-09-13T17:50:34.588762+0000","last_active":"2024-09-13T18:21:13.310571+0000","last_peered":"2024-09-13T18:21:13.310571+0000","last_clean":"2024-09-13T18:21:13.310571+0000","last_became_active":"2024-09-13T17:50:34.588489+0000","last_became_peered":"2024-09-13T17:50:34.588489+0000","last_unstale":"2024-09-13T18:21:13.310571+0000","last_undegraded":"2024-09-13T18:21:13.310571+0000","last_fullsized":"2024-09-13T18:21:13.310571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:38.266297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'18","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.302008+0000","last_change":"2024-09-13T17:50:34.583889+0000","last_active":"2024-09-13T18:21:13.302008+0000","last_peered":"2024-09-13T18:21:13.302008+0000","last_clean":"2024-09-13T18:21:13.302008+0000","last_became_active":"2024-09-13T17:50:34.583551+0000","last_became_peered":"2024-09-13T17:50:34.583551+0000","last_unstale":"2024-09-13T18:21:13.302008+0000","last_undegraded":"2024-09-13T18:21:13.302008+0000","last_fullsized":"2024-09-13T18:21:13.302008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:19.888626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'10","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.284491+0000","last_change":"2024-09-13T17:50:34.585617+0000","last_active":"2024-09-13T18:21:13.284491+0000","last_peered":"2024-09-13T18:21:13.284491+0000","last_clean":"2024-09-13T18:21:13.284491+0000","last_became_active":"2024-09-13T17:50:34.585340+0000","last_became_peered":"2024-09-13T17:50:34.585340+0000","last_unstale":"2024-09-13T18:21:13.284491+0000","last_undegraded":"2024-09-13T18:21:13.284491+0000","last_fullsized":"2024-09-13T18:21:13.284491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:18.331045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'17","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317071+0000","last_change":"2024-09-13T17:50:34.592365+0000","last_active":"2024-09-13T18:21:13.317071+0000","last_peered":"2024-09-13T18:21:13.317071+0000","last_clean":"2024-09-13T18:21:13.317071+0000","last_became_active":"2024-09-13T17:50:34.592046+0000","last_became_peered":"2024-09-13T17:50:34.592046+0000","last_unstale":"2024-09-13T18:21:13.317071+0000","last_undegraded":"2024-09-13T18:21:13.317071+0000","last_fullsized":"2024-09-13T18:21:13.317071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:54.283683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":751482,"num_write_kb":16106434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178546,"ondisk_log_size":178546,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1271,"ondisk_log_size":1271,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":15941984,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":121,"num_read_kb":179,"num_write":3371,"num_write_kb":16874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32030720,"data_stored":31883968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3124,"ondisk_log_size":3124,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509812,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1768076,"kb_used_data":8580,"kb_used_omap":1,"kb_used_meta":1759486,"kb_avail":92603764,"statfs":{"total":96636764160,"available":94826254336,"internally_reserved":0,"allocated":8785920,"data_stored":8600393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":1801714120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":2.246,"5min":5.9219999999999997,"15min":2.669},"min":{"1min":0.19,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":8.5329999999999995,"5min":41.494999999999997,"15min":50.511000000000003},"last":0.63},{"interface":"front","average":{"1min":2.6070000000000002,"5min":3.4350000000000001,"15min":1.6080000000000001},"min":{"1min":0.23100000000000001,"5min":0.193,"15min":0.193},"max":{"1min":18.760000000000002,"5min":18.760000000000002,"15min":18.760000000000002},"last":0.56999999999999995}]},{"osd":1,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":3.4969999999999999,"15min":1.8009999999999999},"min":{"1min":0.34000000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":6.25,"5min":19.626000000000001,"15min":19.626000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":3.4430000000000001,"5min":4.101,"15min":2.0379999999999998},"min":{"1min":0.29599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":22.126999999999999,"5min":22.126999999999999,"15min":22.126999999999999},"last":0.71999999999999997}]},{"osd":2,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.907,"15min":1.296},"min":{"1min":0.29399999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":7.4610000000000003,"5min":22.010000000000002,"15min":22.010000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.49,"5min":2.1800000000000002,"15min":1.4830000000000001},"min":{"1min":0.38400000000000001,"5min":0.309,"15min":0.309},"max":{"1min":7.4850000000000003,"5min":24.170999999999999,"15min":24.170999999999999},"last":0.74399999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":2.0409999999999999,"15min":1.466},"min":{"1min":0.33200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":5.593,"5min":24.359000000000002,"15min":24.359000000000002},"last":0.87},{"interface":"front","average":{"1min":1.8109999999999999,"5min":2.3410000000000002,"15min":1.4570000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":10.132999999999999,"5min":15.539999999999999,"15min":15.539999999999999},"last":1.0069999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":2.5840000000000001,"15min":1.6160000000000001},"min":{"1min":0.35699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":8.5739999999999998,"5min":49.838000000000001,"15min":49.838000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.96999999999999997,"5min":2.5369999999999999,"15min":1.605},"min":{"1min":0.43099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":4.3810000000000002,"5min":41.591999999999999,"15min":41.591999999999999},"last":0.81100000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":2.23,"5min":2.5579999999999998,"15min":1.7270000000000001},"min":{"1min":0.46100000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":8.6029999999999998,"5min":24.457000000000001,"15min":24.457000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.677,"5min":2.2919999999999998,"15min":1.6200000000000001},"min":{"1min":0.42399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":7.4290000000000003,"5min":24.387,"15min":24.387},"last":1.1319999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.869,"15min":1.3740000000000001},"min":{"1min":0.505,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":4.2750000000000004,"5min":14.138999999999999,"15min":14.138999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.155,"5min":1.97,"15min":1.4219999999999999},"min":{"1min":0.44900000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":4.4320000000000004,"5min":14.013,"15min":14.013},"last":0.99299999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509814,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1055244,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":1054846,"kb_avail":93316596,"statfs":{"total":96636764160,"available":95556194304,"internally_reserved":0,"allocated":397312,"data_stored":210684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1080162758},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":2.0819999999999999,"5min":2.0739999999999998,"15min":1.0980000000000001},"min":{"1min":0.17499999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":9.9280000000000008,"5min":17.971,"15min":17.971},"last":0.45900000000000002},{"interface":"front","average":{"1min":2.1469999999999998,"5min":2.1240000000000001,"15min":1.177},"min":{"1min":0.247,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":9.9700000000000006,"5min":18.001999999999999,"15min":18.001999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":2.694,"15min":1.677},"min":{"1min":0.26100000000000001,"5min":0.217,"15min":0.217},"max":{"1min":3.871,"5min":24.599,"15min":56.939999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.4710000000000001,"5min":2.9350000000000001,"15min":1.7649999999999999},"min":{"1min":0.23200000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":9.0169999999999995,"5min":24.75,"15min":57.006999999999998},"last":0.61899999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":3.5219999999999998,"5min":3.0790000000000002,"15min":1.5980000000000001},"min":{"1min":0.30499999999999999,"5min":0.218,"15min":0.218},"max":{"1min":16.111999999999998,"5min":16.707000000000001,"15min":16.707000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":3.0169999999999999,"5min":2.6349999999999998,"15min":1.5569999999999999},"min":{"1min":0.309,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":12.449999999999999,"5min":16.640999999999998,"15min":16.640999999999998},"last":0.69399999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.52,"15min":1.238},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":8.2260000000000009,"5min":8.2829999999999995,"15min":14.177},"last":1.073},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.284,"15min":1.159},"min":{"1min":0.33900000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":8.1980000000000004,"5min":8.1980000000000004,"15min":14.396000000000001},"last":1.0589999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.77600000000000002,"15min":1.0009999999999999},"min":{"1min":0.379,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.2250000000000001,"5min":4.1020000000000003,"15min":14.153},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.181,"5min":1.224,"15min":1.137},"min":{"1min":0.32800000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":7.476,"5min":7.476,"15min":14.375999999999999},"last":0.82099999999999995}]},{"osd":6,"last update":"Fri Sep 13 18:20:30 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.1519999999999999,"15min":1.1020000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":6.8730000000000002,"5min":8.3010000000000002,"15min":8.3010000000000002},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.125,"5min":1.196,"15min":1.2150000000000001},"min":{"1min":0.46700000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":6.9500000000000002,"5min":8.3320000000000007,"15min":14.074},"last":0.996}]},{"osd":7,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.163,"15min":1.0780000000000001},"min":{"1min":0.40400000000000003,"5min":0.37,"15min":0.37},"max":{"1min":4.1079999999999997,"5min":8.0410000000000004,"15min":8.0410000000000004},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.238,"15min":1.1930000000000001},"min":{"1min":0.37,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.9900000000000002,"5min":6.5300000000000002,"15min":14.307},"last":1.111}]}]},{"osd":1,"up_from":15,"seq":64424509813,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1738228,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":1737918,"kb_avail":92633612,"statfs":{"total":96636764160,"available":94856818688,"internally_reserved":0,"allocated":307200,"data_stored":116868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1779628486},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.9119999999999999,"5min":3.7890000000000001,"15min":1.891},"min":{"1min":0.23999999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":9.9879999999999995,"5min":21.949999999999999,"15min":21.949999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.4610000000000001,"5min":2.863,"15min":1.5860000000000001},"min":{"1min":0.219,"5min":0.193,"15min":0.193},"max":{"1min":7.8730000000000002,"5min":23.338999999999999,"15min":23.338999999999999},"last":0.58999999999999997}]},{"osd":2,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":2.5699999999999998,"15min":1.4950000000000001},"min":{"1min":0.311,"5min":0.22,"15min":0.22},"max":{"1min":6.6399999999999997,"5min":14.869999999999999,"15min":14.869999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.4390000000000001,"5min":2.383,"15min":1.4059999999999999},"min":{"1min":0.23599999999999999,"5min":0.223,"15min":0.223},"max":{"1min":6.5300000000000002,"5min":15.058,"15min":15.058},"last":0.83499999999999996}]},{"osd":3,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":2.323,"5min":6.8940000000000001,"15min":3.1539999999999999},"min":{"1min":0.314,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":8.2390000000000008,"5min":41.369999999999997,"15min":41.369999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":2.8769999999999998,"5min":6.2549999999999999,"15min":2.8900000000000001},"min":{"1min":0.27900000000000003,"5min":0.216,"15min":0.216},"max":{"1min":8.8360000000000003,"5min":36.439,"15min":40.325000000000003},"last":0.63300000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.978,"5min":3.9390000000000001,"15min":2.1070000000000002},"min":{"1min":0.30399999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":8.5980000000000008,"5min":103.44499999999999,"15min":103.44499999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":2.1850000000000001,"5min":3.3199999999999998,"15min":1.889},"min":{"1min":0.312,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":8.2789999999999999,"5min":40.953000000000003,"15min":40.953000000000003},"last":0.95899999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:20:20 2024","interfaces":[{"interface":"back","average":{"1min":1.9750000000000001,"5min":2.661,"15min":1.726},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":9.8970000000000002,"5min":18.954000000000001,"15min":18.954000000000001},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.431,"5min":2.0840000000000001,"15min":1.516},"min":{"1min":0.31,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":7.383,"5min":15.413,"15min":15.413},"last":1.0429999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":2.4089999999999998,"15min":1.569},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":7.4199999999999999,"5min":15.462999999999999,"15min":15.462999999999999},"last":1.244},{"interface":"front","average":{"1min":1.6719999999999999,"5min":2.5190000000000001,"15min":1.663},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":7.4429999999999996,"5min":15.49,"15min":15.49},"last":1.3009999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.764,"15min":1.375},"min":{"1min":0.33600000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":6.6159999999999997,"5min":15.093999999999999,"15min":15.093999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.494,"5min":1.8380000000000001,"15min":1.4590000000000001},"min":{"1min":0.41899999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":6.6920000000000002,"5min":15.119,"15min":15.119},"last":1.27}]}]},{"osd":0,"up_from":15,"seq":64424509815,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1247072,"kb_used_data":4056,"kb_used_omap":1,"kb_used_meta":1243006,"kb_avail":93124768,"statfs":{"total":96636764160,"available":95359762432,"internally_reserved":0,"allocated":4153344,"data_stored":3967313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1272838598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":2.8570000000000002,"5min":3.2130000000000001,"15min":1.7050000000000001},"min":{"1min":0.182,"5min":0.182,"15min":0.182},"max":{"1min":17.693000000000001,"5min":23.004000000000001,"15min":24.202000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":2.9489999999999998,"5min":3.2690000000000001,"15min":1.8380000000000001},"min":{"1min":0.32100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":17.736000000000001,"5min":23.039999999999999,"15min":24.260999999999999},"last":0.876}]},{"osd":2,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.897,"15min":1.1060000000000001},"min":{"1min":0.20399999999999999,"5min":0.186,"15min":0.186},"max":{"1min":6.4320000000000004,"5min":17.859000000000002,"15min":17.859000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.77,"5min":2.0920000000000001,"15min":1.3180000000000001},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":8.6199999999999992,"5min":19.678999999999998,"15min":19.678999999999998},"last":0.78900000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":2.2330000000000001,"5min":4.0970000000000004,"15min":1.9630000000000001},"min":{"1min":0.23699999999999999,"5min":0.219,"15min":0.219},"max":{"1min":11.378,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":3.6720000000000002,"5min":3.27,"15min":1.8129999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":13.736000000000001,"5min":15.856999999999999,"15min":15.856999999999999},"last":0.92400000000000004}]},{"osd":4,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.661,"15min":1.258},"min":{"1min":0.34699999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":8.2829999999999995,"5min":8.7010000000000005,"15min":8.7010000000000005},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.1719999999999999,"5min":2.657,"15min":1.548},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":6.3090000000000002,"5min":16.291,"15min":16.291},"last":1.044}]},{"osd":5,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":2.0960000000000001,"5min":1.974,"15min":1.3260000000000001},"min":{"1min":0.312,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":8.4800000000000004,"5min":15.18,"15min":15.18},"last":1.079},{"interface":"front","average":{"1min":1.7689999999999999,"5min":1.9850000000000001,"15min":1.4119999999999999},"min":{"1min":0.34699999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":8.5559999999999992,"5min":17.852,"15min":17.852},"last":1.135}]},{"osd":6,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":3.1669999999999998,"5min":2.3860000000000001,"15min":1.5600000000000001},"min":{"1min":0.39200000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":15.467000000000001,"5min":15.467000000000001,"15min":15.467000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.7290000000000001,"5min":1.823,"15min":1.4139999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":10.359,"5min":17.908000000000001,"15min":17.908000000000001},"last":1.278}]},{"osd":7,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":1.909,"5min":1.887,"15min":1.381},"min":{"1min":0.40100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":8.2550000000000008,"5min":18.986999999999998,"15min":18.986999999999998},"last":1.242},{"interface":"front","average":{"1min":1.952,"5min":1.9710000000000001,"15min":1.47},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.373},"max":{"1min":8.3279999999999994,"5min":19.021000000000001,"15min":19.021000000000001},"last":1.1699999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542517,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1207524,"kb_used_data":12252,"kb_used_omap":1,"kb_used_meta":1195262,"kb_avail":93164316,"statfs":{"total":96636764160,"available":95400259584,"internally_reserved":0,"allocated":12546048,"data_stored":12357112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":1223948747},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":2.0619999999999998,"15min":1.246},"min":{"1min":0.245,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":5.367,"5min":35.195,"15min":35.195},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.2529999999999999,"5min":2.206,"15min":1.391},"min":{"1min":0.34000000000000002,"5min":0.315,"15min":0.315},"max":{"1min":5.4269999999999996,"5min":35.253999999999998,"15min":35.253999999999998},"last":0.92900000000000005}]},{"osd":1,"last update":"Fri Sep 13 18:20:27 2024","interfaces":[{"interface":"back","average":{"1min":2.4940000000000002,"5min":2.6930000000000001,"15min":1.4730000000000001},"min":{"1min":0.27100000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":13.058999999999999,"5min":19.585000000000001,"15min":19.585000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":2.4060000000000001,"5min":2.6680000000000001,"15min":1.635},"min":{"1min":0.33400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":17.398,"5min":19.617000000000001,"15min":19.617000000000001},"last":0.74199999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.6100000000000001,"15min":1.143},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":7.1299999999999999,"5min":15.789999999999999,"15min":15.789999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.5569999999999999,"15min":1.161},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.9379999999999997,"5min":12.965,"15min":12.965},"last":1.1950000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":3.427,"15min":1.8700000000000001},"min":{"1min":0.36299999999999999,"5min":0.316,"15min":0.316},"max":{"1min":5.4370000000000003,"5min":40.948999999999998,"15min":40.948999999999998},"last":0.98199999999999998},{"interface":"front","average":{"1min":2.0739999999999998,"5min":3.198,"15min":1.6870000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":20.024999999999999,"5min":25.234000000000002,"15min":25.234000000000002},"last":1.115}]},{"osd":4,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1739999999999999,"15min":0.99399999999999999},"min":{"1min":0.31,"5min":0.252,"15min":0.252},"max":{"1min":4.5220000000000002,"5min":8.8670000000000009,"15min":8.8670000000000009},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.321,"15min":1.181},"min":{"1min":0.38500000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":4.383,"5min":10.279999999999999,"15min":10.279999999999999},"last":0.79900000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.123,"15min":0.98499999999999999},"min":{"1min":0.34300000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":4.4790000000000001,"5min":8.1699999999999999,"15min":8.1699999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.98999999999999999,"15min":1.0289999999999999},"min":{"1min":0.33100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":3.0489999999999999,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":1.004}]},{"osd":6,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.4750000000000001,"15min":1.2190000000000001},"min":{"1min":0.34599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":6.7960000000000003,"5min":8.2219999999999995,"15min":8.2219999999999995},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.508,"15min":1.2549999999999999},"min":{"1min":0.434,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":7.048,"5min":8.2469999999999999,"15min":8.2469999999999999},"last":1.27}]}]},{"osd":6,"up_from":14,"seq":60129542517,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1419436,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":1415038,"kb_avail":92952404,"statfs":{"total":96636764160,"available":95183261696,"internally_reserved":0,"allocated":4493312,"data_stored":4311530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1448999367},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":2.0329999999999999,"15min":1.21},"min":{"1min":0.23699999999999999,"5min":0.214,"15min":0.214},"max":{"1min":3.9470000000000001,"5min":12.808999999999999,"15min":12.808999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.091,"5min":1.7030000000000001,"15min":1.161},"min":{"1min":0.31,"5min":0.218,"15min":0.218},"max":{"1min":5.7670000000000003,"5min":12.308999999999999,"15min":12.308999999999999},"last":0.80500000000000005}]},{"osd":1,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":2.0470000000000002,"5min":2.27,"15min":1.371},"min":{"1min":0.34799999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":5.8380000000000001,"5min":14.199999999999999,"15min":14.199999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.621,"5min":2.339,"15min":1.413},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.7990000000000004,"5min":14.260999999999999,"15min":14.260999999999999},"last":1.109}]},{"osd":2,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.6839999999999999,"5min":1.821,"15min":1.2110000000000001},"min":{"1min":0.29999999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":6.0359999999999996,"5min":10.792999999999999,"15min":10.792999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.742,"5min":1.8759999999999999,"15min":1.371},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":6.0750000000000002,"5min":10.826000000000001,"15min":10.826000000000001},"last":1.252}]},{"osd":3,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":3.379,"5min":5.0670000000000002,"15min":2.4660000000000002},"min":{"1min":0.35799999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":10.324999999999999,"5min":84.867999999999995,"15min":84.867999999999995},"last":0.93600000000000005},{"interface":"front","average":{"1min":2.5449999999999999,"5min":3.5779999999999998,"15min":1.8979999999999999},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":10.231,"5min":84.343999999999994,"15min":84.343999999999994},"last":1.01}]},{"osd":4,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":2.0739999999999998,"5min":1.6990000000000001,"15min":1.2070000000000001},"min":{"1min":0.39200000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":9.5649999999999995,"5min":11.044,"15min":11.044},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.294,"15min":1.113},"min":{"1min":0.38400000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":9.4719999999999995,"5min":9.4719999999999995,"15min":9.4719999999999995},"last":0.97599999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2689999999999999,"15min":1.1559999999999999},"min":{"1min":0.376,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.0939999999999999,"5min":8.7880000000000003,"15min":8.7880000000000003},"last":1.151},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.149,"15min":1.1000000000000001},"min":{"1min":0.41499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.1709999999999998,"5min":7.569,"15min":7.569},"last":1.222}]},{"osd":7,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.1739999999999999,"15min":1.181},"min":{"1min":0.434,"5min":0.32900000000000001,"15min":0.29999999999999999},"max":{"1min":2.8029999999999999,"5min":7.1820000000000004,"15min":7.1820000000000004},"last":1.335},{"interface":"front","average":{"1min":1.133,"5min":1.294,"15min":1.256},"min":{"1min":0.53700000000000003,"5min":0.34499999999999997,"15min":0.33500000000000002},"max":{"1min":2.871,"5min":7.2039999999999997,"15min":7.2039999999999997},"last":1.379}]}]},{"osd":5,"up_from":14,"seq":60129542519,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":978544,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":978238,"kb_avail":93393296,"statfs":{"total":96636764160,"available":95634735104,"internally_reserved":0,"allocated":303104,"data_stored":117302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1001716166},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.889,"15min":1.161},"min":{"1min":0.27500000000000002,"5min":0.218,"15min":0.218},"max":{"1min":7.3650000000000002,"5min":10.148999999999999,"15min":10.148999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.9550000000000001,"15min":1.3220000000000001},"min":{"1min":0.34200000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":7.4029999999999996,"5min":10.185,"15min":10.185},"last":0.96999999999999997}]},{"osd":1,"last update":"Fri Sep 13 18:20:27 2024","interfaces":[{"interface":"back","average":{"1min":1.879,"5min":1.897,"15min":1.1699999999999999},"min":{"1min":0.29799999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":10.08,"5min":10.436999999999999,"15min":10.436999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":2.0099999999999998,"5min":2.3460000000000001,"15min":1.385},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":18.584,"5min":19.388000000000002,"15min":19.388000000000002},"last":1.2609999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.875,"5min":1.5329999999999999,"15min":1.0980000000000001},"min":{"1min":0.33900000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":10.183,"5min":12.667,"15min":12.667},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.032,"5min":1.6439999999999999,"15min":1.198},"min":{"1min":0.377,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":5.7469999999999999,"5min":34.905999999999999,"15min":34.905999999999999},"last":1.119}]},{"osd":3,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":3.2999999999999998,"5min":4.9039999999999999,"15min":2.286},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":19.831,"5min":28.295000000000002,"15min":28.295000000000002},"last":0.81399999999999995},{"interface":"front","average":{"1min":3.9260000000000002,"5min":3.1459999999999999,"15min":1.7210000000000001},"min":{"1min":0.39700000000000002,"5min":0.318,"15min":0.318},"max":{"1min":19.731000000000002,"5min":19.731000000000002,"15min":19.731000000000002},"last":1.0149999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.744,"5min":1.0760000000000001,"15min":0.96199999999999997},"min":{"1min":0.308,"5min":0.255,"15min":0.255},"max":{"1min":10.023999999999999,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":2.3650000000000002,"5min":1.571,"15min":1.22},"min":{"1min":0.39400000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":10.210000000000001,"5min":10.210000000000001,"15min":10.210000000000001},"last":1.075}]},{"osd":6,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.9530000000000001,"5min":1.373,"15min":1.131},"min":{"1min":0.437,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":6.6849999999999996,"5min":6.6849999999999996,"15min":6.6849999999999996},"last":1.179},{"interface":"front","average":{"1min":2.012,"5min":1.3200000000000001,"15min":1.2},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":10.151,"5min":10.151,"15min":10.151},"last":1.2130000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":1.2330000000000001,"15min":1.143},"min":{"1min":0.42599999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":6.2910000000000004,"5min":7.9329999999999998,"15min":7.9329999999999998},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.71,"5min":1.107,"15min":1.1519999999999999},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":5.976,"5min":5.976,"15min":5.976},"last":1.3620000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542517,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1301036,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":1296638,"kb_avail":93070804,"statfs":{"total":96636764160,"available":95304503296,"internally_reserved":0,"allocated":4493312,"data_stored":4310894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1327757766},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":2.1499999999999999,"15min":1.2829999999999999},"min":{"1min":0.26200000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":6.984,"5min":28.439,"15min":28.439},"last":0.92200000000000004},{"interface":"front","average":{"1min":2.1949999999999998,"5min":2.4079999999999999,"15min":1.4430000000000001},"min":{"1min":0.309,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":16.943000000000001,"5min":28.422000000000001,"15min":28.422000000000001},"last":0.748}]},{"osd":1,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":3.7789999999999999,"5min":5.5380000000000003,"15min":2.577},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":12.124000000000001,"5min":38.895000000000003,"15min":38.895000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":3.2069999999999999,"5min":4.0599999999999996,"15min":1.976},"min":{"1min":0.30199999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":12.026999999999999,"5min":32.670000000000002,"15min":32.670000000000002},"last":0.52600000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.72,"15min":1.1599999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":4.085,"5min":25.167999999999999,"15min":25.167999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":2.0150000000000001,"15min":1.321},"min":{"1min":0.373,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":4.1150000000000002,"5min":25.218,"15min":25.218},"last":1.1439999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":2.6419999999999999,"5min":3.0129999999999999,"15min":1.7250000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":11.441000000000001,"5min":43.945999999999998,"15min":43.945999999999998},"last":1.008},{"interface":"front","average":{"1min":3.2610000000000001,"5min":3.9329999999999998,"15min":2.052},"min":{"1min":0.39200000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":29.640000000000001,"5min":29.640000000000001,"15min":29.640000000000001},"last":0.81399999999999995}]},{"osd":5,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.2050000000000001,"15min":1.069},"min":{"1min":0.28199999999999997,"5min":0.255,"15min":0.255},"max":{"1min":12.282999999999999,"5min":12.282999999999999,"15min":12.282999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.3160000000000001,"15min":1.1739999999999999},"min":{"1min":0.36699999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":16.995999999999999,"5min":16.995999999999999,"15min":16.995999999999999},"last":0.98099999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":2.0430000000000001,"5min":2.0979999999999999,"15min":1.387},"min":{"1min":0.30499999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":12.42,"5min":23.010999999999999,"15min":23.010999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.504,"5min":1.23,"15min":1.181},"min":{"1min":0.40000000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":6.6040000000000001,"5min":7.9489999999999998,"15min":7.9489999999999998},"last":1.1919999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.448,"15min":1.27},"min":{"1min":0.30499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.0179999999999998,"5min":20.417000000000002,"15min":20.417000000000002},"last":1.236},{"interface":"front","average":{"1min":1.359,"5min":1.4019999999999999,"15min":1.284},"min":{"1min":0.41199999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":12.308999999999999,"5min":12.308999999999999,"15min":12.308999999999999},"last":1.0629999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3276800,"data_stored":3261021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":94760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8503296,"data_stored":8484469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4194970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4195606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":11669504,"data_stored":11650820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:21:20.099 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:21:20.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.201+0000 7fced359e700 1 Processor -- start 2024-09-13T18:21:20.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.201+0000 7fced359e700 1 -- start start 2024-09-13T18:21:20.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.201+0000 7fced359e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fced4151470 0x7fced40c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:20.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.201+0000 7fced359e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fced4071bf0 0x7fced4072080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:20.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.201+0000 7fced359e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fced40c4520 con 0x7fced40b1bd0 2024-09-13T18:21:20.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.201+0000 7fced359e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fced40c4dc0 con 0x7fced4071bf0 2024-09-13T18:21:20.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.201+0000 7fced359e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced4157cf0 con 0x7fced4151470 2024-09-13T18:21:20.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.201+0000 7fced2d9d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fced4071bf0 0x7fced4072080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:20.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.201+0000 7fced259c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fced4151470 0x7fced40c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:20.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.201+0000 7fced2d9d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fced4071bf0 0x7fced4072080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:55212/0 (socket says 172.21.15.40:55212) 2024-09-13T18:21:20.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.201+0000 7fced2d9d700 1 -- 172.21.15.40:0/1868967303 learned_addr learned my addr 172.21.15.40:0/1868967303 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:20.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.201+0000 7fced2d9d700 1 -- 172.21.15.40:0/1868967303 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fced4151470 msgr2=0x7fced40c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:20.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.201+0000 7fced2d9d700 1 --2- 172.21.15.40:0/1868967303 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fced4151470 0x7fced40c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.201+0000 7fced2d9d700 1 -- 172.21.15.40:0/1868967303 >> v1:172.21.15.43:6789/0 conn(0x7fced40b1bd0 legacy=0x7fced41571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:20.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.201+0000 7fced2d9d700 1 -- 172.21.15.40:0/1868967303 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced4157fd0 con 0x7fced4071bf0 2024-09-13T18:21:20.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.201+0000 7fced2d9d700 1 --2- 172.21.15.40:0/1868967303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fced4071bf0 0x7fced4072080 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009e50 tx=0x7fcecc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bb571ac7b9ed17f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:20.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced159a700 1 -- 172.21.15.40:0/1868967303 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc00e070 con 0x7fced4071bf0 2024-09-13T18:21:20.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced159a700 1 -- 172.21.15.40:0/1868967303 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc00b910 con 0x7fced4071bf0 2024-09-13T18:21:20.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced159a700 1 -- 172.21.15.40:0/1868967303 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc00bcc0 con 0x7fced4071bf0 2024-09-13T18:21:20.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced359e700 1 -- 172.21.15.40:0/1868967303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fced4071bf0 msgr2=0x7fced4072080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:20.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced359e700 1 --2- 172.21.15.40:0/1868967303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fced4071bf0 0x7fced4072080 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009e50 tx=0x7fcecc00b330 comp rx=0 tx=0).stop 2024-09-13T18:21:20.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced359e700 1 -- 172.21.15.40:0/1868967303 shutdown_connections 2024-09-13T18:21:20.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced359e700 1 --2- 172.21.15.40:0/1868967303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fced4071bf0 0x7fced4072080 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced359e700 1 --2- 172.21.15.40:0/1868967303 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fced4151470 0x7fced40c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced359e700 1 -- 172.21.15.40:0/1868967303 >> 172.21.15.40:0/1868967303 conn(0x7fced406e4f0 msgr2=0x7fced406e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:20.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced359e700 1 -- 172.21.15.40:0/1868967303 shutdown_connections 2024-09-13T18:21:20.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced359e700 1 -- 172.21.15.40:0/1868967303 wait complete. 2024-09-13T18:21:20.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced359e700 1 Processor -- start 2024-09-13T18:21:20.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced359e700 1 -- start start 2024-09-13T18:21:20.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced359e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fced4071bf0 0x7fced40c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:20.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced359e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fced40b1bd0 0x7fced41e2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:20.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced359e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fced4151470 0x7fced41ef100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:20.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced359e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced41e3340 con 0x7fced4151470 2024-09-13T18:21:20.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced359e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced41f3040 con 0x7fced40b1bd0 2024-09-13T18:21:20.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced359e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced40c1680 con 0x7fced4071bf0 2024-09-13T18:21:20.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced2d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fced4151470 0x7fced41ef100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:20.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced2d9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fced4151470 0x7fced41ef100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48538/0 (socket says 172.21.15.40:48538) 2024-09-13T18:21:20.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced2d9d700 1 -- 172.21.15.40:0/1811700415 learned_addr learned my addr 172.21.15.40:0/1811700415 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:20.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced1d9b700 1 --2- 172.21.15.40:0/1811700415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fced40b1bd0 0x7fced41e2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:20.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced259c700 1 --2- 172.21.15.40:0/1811700415 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fced4071bf0 0x7fced40c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:20.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced2d9d700 1 -- 172.21.15.40:0/1811700415 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fced4071bf0 msgr2=0x7fced40c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:20.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced2d9d700 1 --2- 172.21.15.40:0/1811700415 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fced4071bf0 0x7fced40c28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced2d9d700 1 -- 172.21.15.40:0/1811700415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fced40b1bd0 msgr2=0x7fced41e2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:20.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced2d9d700 1 --2- 172.21.15.40:0/1811700415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fced40b1bd0 0x7fced41e2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced2d9d700 1 -- 172.21.15.40:0/1811700415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc009b30 con 0x7fced4151470 2024-09-13T18:21:20.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced259c700 1 --2- 172.21.15.40:0/1811700415 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fced4071bf0 0x7fced40c28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:20.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.205+0000 7fced2d9d700 1 --2- 172.21.15.40:0/1811700415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fced4151470 0x7fced41ef100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fced40c9900 tx=0x7fcecc004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:20.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.209+0000 7fcec2ffd700 1 -- 172.21.15.40:0/1811700415 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc0048b0 con 0x7fced4151470 2024-09-13T18:21:20.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.209+0000 7fcec2ffd700 1 -- 172.21.15.40:0/1811700415 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc0092e0 con 0x7fced4151470 2024-09-13T18:21:20.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.209+0000 7fced359e700 1 -- 172.21.15.40:0/1811700415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced41f34d0 con 0x7fced4151470 2024-09-13T18:21:20.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.209+0000 7fcec2ffd700 1 -- 172.21.15.40:0/1811700415 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc01b650 con 0x7fced4151470 2024-09-13T18:21:20.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.209+0000 7fced359e700 1 -- 172.21.15.40:0/1811700415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced41f29f0 con 0x7fced4151470 2024-09-13T18:21:20.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.209+0000 7fced359e700 1 -- 172.21.15.40:0/1811700415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced4072080 con 0x7fced4151470 2024-09-13T18:21:20.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.209+0000 7fcec2ffd700 1 -- 172.21.15.40:0/1811700415 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcecc01b7b0 con 0x7fced4151470 2024-09-13T18:21:20.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.209+0000 7fcec2ffd700 1 --2- 172.21.15.40:0/1811700415 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fceb4066760 0x7fceb4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:20.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.209+0000 7fcec2ffd700 1 -- 172.21.15.40:0/1811700415 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcecc085d60 con 0x7fced4151470 2024-09-13T18:21:20.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.209+0000 7fced259c700 1 --2- 172.21.15.40:0/1811700415 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fceb4066760 0x7fceb4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:20.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.213+0000 7fced259c700 1 --2- 172.21.15.40:0/1811700415 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fceb4066760 0x7fceb4068c20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009c40 tx=0x7fcec4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:20.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.217+0000 7fcec2ffd700 1 -- 172.21.15.40:0/1811700415 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcecc052da0 con 0x7fced4151470 2024-09-13T18:21:20.388 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.381+0000 7fced359e700 1 -- 172.21.15.40:0/1811700415 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fced4151850 con 0x7fceb4066760 2024-09-13T18:21:20.406 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.401+0000 7fcec2ffd700 1 -- 172.21.15.40:0/1811700415 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+369002 (secure 0 0 0) 0x7fcec4004030 con 0x7fceb4066760 2024-09-13T18:21:20.412 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T18:21:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.405+0000 7fced359e700 1 -- 172.21.15.40:0/1811700415 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fceb4066760 msgr2=0x7fceb4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.405+0000 7fced359e700 1 --2- 172.21.15.40:0/1811700415 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fceb4066760 0x7fceb4068c20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009c40 tx=0x7fcec4008040 comp rx=0 tx=0).stop 2024-09-13T18:21:20.415 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.405+0000 7fced359e700 1 -- 172.21.15.40:0/1811700415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fced4151470 msgr2=0x7fced41ef100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.405+0000 7fced359e700 1 --2- 172.21.15.40:0/1811700415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fced4151470 0x7fced41ef100 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fced40c9900 tx=0x7fcecc004af0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.405+0000 7fced359e700 1 -- 172.21.15.40:0/1811700415 shutdown_connections 2024-09-13T18:21:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.405+0000 7fced359e700 1 --2- 172.21.15.40:0/1811700415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fced4151470 0x7fced41ef100 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.405+0000 7fced359e700 1 --2- 172.21.15.40:0/1811700415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fced40b1bd0 0x7fced41e2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.405+0000 7fced359e700 1 --2- 172.21.15.40:0/1811700415 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fceb4066760 0x7fceb4068c20 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.405+0000 7fced359e700 1 --2- 172.21.15.40:0/1811700415 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fced4071bf0 0x7fced40c28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.405+0000 7fced359e700 1 -- 172.21.15.40:0/1811700415 >> 172.21.15.40:0/1811700415 conn(0x7fced406e4f0 msgr2=0x7fced41503b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.405+0000 7fced359e700 1 -- 172.21.15.40:0/1811700415 shutdown_connections 2024-09-13T18:21:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.405+0000 7fced359e700 1 -- 172.21.15.40:0/1811700415 wait complete. 2024-09-13T18:21:20.416 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-13T18:21:20.428 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":949,"stamp":"2024-09-13T18:21:19.774698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":16532352,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":237,"num_read_kb":279,"num_write":755012,"num_write_kb":16125724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183058,"ondisk_log_size":183058,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10715160,"kb_used_data":34648,"kb_used_omap":12,"kb_used_meta":10680435,"kb_avail":744259560,"statfs":{"total":773094113280,"available":762121789440,"internally_reserved":0,"allocated":35479552,"data_stored":33992096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":10936766008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005254"},"pg_stats":[{"pgid":"2.1f","version":"27'268","reported_seq":299,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.261706+0000","last_change":"2024-09-13T17:50:34.212158+0000","last_active":"2024-09-13T18:21:13.261706+0000","last_peered":"2024-09-13T18:21:13.261706+0000","last_clean":"2024-09-13T18:21:13.261706+0000","last_became_active":"2024-09-13T17:50:34.211867+0000","last_became_peered":"2024-09-13T17:50:34.211867+0000","last_unstale":"2024-09-13T18:21:13.261706+0000","last_undegraded":"2024-09-13T18:21:13.261706+0000","last_fullsized":"2024-09-13T18:21:13.261706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:14.671095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3260943,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":267,"num_write_kb":3318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'10","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306110+0000","last_change":"2024-09-13T17:50:34.587396+0000","last_active":"2024-09-13T18:21:13.306110+0000","last_peered":"2024-09-13T18:21:13.306110+0000","last_clean":"2024-09-13T18:21:13.306110+0000","last_became_active":"2024-09-13T17:50:34.587108+0000","last_became_peered":"2024-09-13T17:50:34.587108+0000","last_unstale":"2024-09-13T18:21:13.306110+0000","last_undegraded":"2024-09-13T18:21:13.306110+0000","last_fullsized":"2024-09-13T18:21:13.306110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:39:16.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'5594","reported_seq":5620,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.876951+0000","last_change":"2024-09-13T17:50:38.615059+0000","last_active":"2024-09-13T18:21:02.876951+0000","last_peered":"2024-09-13T18:21:02.876951+0000","last_clean":"2024-09-13T18:21:02.876951+0000","last_became_active":"2024-09-13T17:50:38.614723+0000","last_became_peered":"2024-09-13T17:50:38.614723+0000","last_unstale":"2024-09-13T18:21:02.876951+0000","last_undegraded":"2024-09-13T18:21:02.876951+0000","last_fullsized":"2024-09-13T18:21:02.876951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5594,"log_dups_size":0,"ondisk_log_size":5594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:23:01.902181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23732,"num_write_kb":533374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'6","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.773351+0000","last_change":"2024-09-13T17:50:34.204760+0000","last_active":"2024-09-13T18:21:02.773351+0000","last_peered":"2024-09-13T18:21:02.773351+0000","last_clean":"2024-09-13T18:21:02.773351+0000","last_became_active":"2024-09-13T17:50:34.202930+0000","last_became_peered":"2024-09-13T17:50:34.202930+0000","last_unstale":"2024-09-13T18:21:02.773351+0000","last_undegraded":"2024-09-13T18:21:02.773351+0000","last_fullsized":"2024-09-13T18:21:02.773351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:59.784701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'20","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306487+0000","last_change":"2024-09-13T17:51:16.366352+0000","last_active":"2024-09-13T18:21:13.306487+0000","last_peered":"2024-09-13T18:21:13.306487+0000","last_clean":"2024-09-13T18:21:13.306487+0000","last_became_active":"2024-09-13T17:51:16.365492+0000","last_became_peered":"2024-09-13T17:51:16.365492+0000","last_unstale":"2024-09-13T18:21:13.306487+0000","last_undegraded":"2024-09-13T18:21:13.306487+0000","last_fullsized":"2024-09-13T18:21:13.306487+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:42.192182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'5615","reported_seq":5641,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.875980+0000","last_change":"2024-09-13T17:50:38.618751+0000","last_active":"2024-09-13T18:21:02.875980+0000","last_peered":"2024-09-13T18:21:02.875980+0000","last_clean":"2024-09-13T18:21:02.875980+0000","last_became_active":"2024-09-13T17:50:38.618423+0000","last_became_peered":"2024-09-13T17:50:38.618423+0000","last_unstale":"2024-09-13T18:21:02.875980+0000","last_undegraded":"2024-09-13T18:21:02.875980+0000","last_fullsized":"2024-09-13T18:21:02.875980+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5615,"log_dups_size":0,"ondisk_log_size":5615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:50:02.418785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23762,"num_write_kb":516686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'11","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.801229+0000","last_change":"2024-09-13T17:50:34.214972+0000","last_active":"2024-09-13T18:21:02.801229+0000","last_peered":"2024-09-13T18:21:02.801229+0000","last_clean":"2024-09-13T18:21:02.801229+0000","last_became_active":"2024-09-13T17:50:34.214652+0000","last_became_peered":"2024-09-13T17:50:34.214652+0000","last_unstale":"2024-09-13T18:21:02.801229+0000","last_undegraded":"2024-09-13T18:21:02.801229+0000","last_fullsized":"2024-09-13T18:21:02.801229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:57.176854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'18","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.292937+0000","last_change":"2024-09-13T17:51:16.281831+0000","last_active":"2024-09-13T18:21:13.292937+0000","last_peered":"2024-09-13T18:21:13.292937+0000","last_clean":"2024-09-13T18:21:13.292937+0000","last_became_active":"2024-09-13T17:51:16.281549+0000","last_became_peered":"2024-09-13T17:51:16.281549+0000","last_unstale":"2024-09-13T18:21:13.292937+0000","last_undegraded":"2024-09-13T18:21:13.292937+0000","last_fullsized":"2024-09-13T18:21:13.292937+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:43.894437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"27'5142","reported_seq":5168,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.803386+0000","last_change":"2024-09-13T17:50:38.616219+0000","last_active":"2024-09-13T18:21:02.803386+0000","last_peered":"2024-09-13T18:21:02.803386+0000","last_clean":"2024-09-13T18:21:02.803386+0000","last_became_active":"2024-09-13T17:50:38.614329+0000","last_became_peered":"2024-09-13T17:50:38.614329+0000","last_unstale":"2024-09-13T18:21:02.803386+0000","last_undegraded":"2024-09-13T18:21:02.803386+0000","last_fullsized":"2024-09-13T18:21:02.803386+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5142,"log_dups_size":0,"ondisk_log_size":5142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:25.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21334,"num_write_kb":478261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'8","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.159841+0000","last_change":"2024-09-13T17:50:34.216123+0000","last_active":"2024-09-13T18:20:25.159841+0000","last_peered":"2024-09-13T18:20:25.159841+0000","last_clean":"2024-09-13T18:20:25.159841+0000","last_became_active":"2024-09-13T17:50:34.215661+0000","last_became_peered":"2024-09-13T17:50:34.215661+0000","last_unstale":"2024-09-13T18:20:25.159841+0000","last_undegraded":"2024-09-13T18:20:25.159841+0000","last_fullsized":"2024-09-13T18:20:25.159841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:20.895037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'30","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.318958+0000","last_change":"2024-09-13T17:50:34.599673+0000","last_active":"2024-09-13T18:21:13.318958+0000","last_peered":"2024-09-13T18:21:13.318958+0000","last_clean":"2024-09-13T18:21:13.318958+0000","last_became_active":"2024-09-13T17:50:34.599409+0000","last_became_peered":"2024-09-13T17:50:34.599409+0000","last_unstale":"2024-09-13T18:21:13.318958+0000","last_undegraded":"2024-09-13T18:21:13.318958+0000","last_fullsized":"2024-09-13T18:21:13.318958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:03:31.394385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'4871","reported_seq":4897,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.743078+0000","last_change":"2024-09-13T17:50:38.616317+0000","last_active":"2024-09-13T18:21:02.743078+0000","last_peered":"2024-09-13T18:21:02.743078+0000","last_clean":"2024-09-13T18:21:02.743078+0000","last_became_active":"2024-09-13T17:50:38.614311+0000","last_became_peered":"2024-09-13T17:50:38.614311+0000","last_unstale":"2024-09-13T18:21:02.743078+0000","last_undegraded":"2024-09-13T18:21:02.743078+0000","last_fullsized":"2024-09-13T18:21:02.743078+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4871,"log_dups_size":0,"ondisk_log_size":4871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:29.221522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20106,"num_write_kb":432357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'14","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.313143+0000","last_change":"2024-09-13T17:50:34.599490+0000","last_active":"2024-09-13T18:21:13.313143+0000","last_peered":"2024-09-13T18:21:13.313143+0000","last_clean":"2024-09-13T18:21:13.313143+0000","last_became_active":"2024-09-13T17:50:34.599207+0000","last_became_peered":"2024-09-13T17:50:34.599207+0000","last_unstale":"2024-09-13T18:21:13.313143+0000","last_undegraded":"2024-09-13T18:21:13.313143+0000","last_fullsized":"2024-09-13T18:21:13.313143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:11.178161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'1092","reported_seq":1122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.671116+0000","last_change":"2024-09-13T17:50:34.215697+0000","last_active":"2024-09-13T18:21:02.671116+0000","last_peered":"2024-09-13T18:21:02.671116+0000","last_clean":"2024-09-13T18:21:02.671116+0000","last_became_active":"2024-09-13T17:50:34.215409+0000","last_became_peered":"2024-09-13T17:50:34.215409+0000","last_unstale":"2024-09-13T18:21:02.671116+0000","last_undegraded":"2024-09-13T18:21:02.671116+0000","last_fullsized":"2024-09-13T18:21:02.671116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:21.609567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1088,"num_write_kb":4339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'5394","reported_seq":5421,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.852704+0000","last_change":"2024-09-13T17:50:38.611817+0000","last_active":"2024-09-13T18:21:02.852704+0000","last_peered":"2024-09-13T18:21:02.852704+0000","last_clean":"2024-09-13T18:21:02.852704+0000","last_became_active":"2024-09-13T17:50:38.611263+0000","last_became_peered":"2024-09-13T17:50:38.611263+0000","last_unstale":"2024-09-13T18:21:02.852704+0000","last_undegraded":"2024-09-13T18:21:02.852704+0000","last_fullsized":"2024-09-13T18:21:02.852704+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5394,"log_dups_size":0,"ondisk_log_size":5394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:56:47.466672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22326,"num_write_kb":494524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'19","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.308364+0000","last_change":"2024-09-13T17:50:34.586307+0000","last_active":"2024-09-13T18:21:13.308364+0000","last_peered":"2024-09-13T18:21:13.308364+0000","last_clean":"2024-09-13T18:21:13.308364+0000","last_became_active":"2024-09-13T17:50:34.585831+0000","last_became_peered":"2024-09-13T17:50:34.585831+0000","last_unstale":"2024-09-13T18:21:13.308364+0000","last_undegraded":"2024-09-13T18:21:13.308364+0000","last_fullsized":"2024-09-13T18:21:13.308364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:40.912352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'9","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.773590+0000","last_change":"2024-09-13T17:50:34.211850+0000","last_active":"2024-09-13T18:21:02.773590+0000","last_peered":"2024-09-13T18:21:02.773590+0000","last_clean":"2024-09-13T18:21:02.773590+0000","last_became_active":"2024-09-13T17:50:34.211594+0000","last_became_peered":"2024-09-13T17:50:34.211594+0000","last_unstale":"2024-09-13T18:21:02.773590+0000","last_undegraded":"2024-09-13T18:21:02.773590+0000","last_fullsized":"2024-09-13T18:21:02.773590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:35:02.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'5504","reported_seq":5531,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.796881+0000","last_change":"2024-09-13T17:50:38.613576+0000","last_active":"2024-09-13T18:21:02.796881+0000","last_peered":"2024-09-13T18:21:02.796881+0000","last_clean":"2024-09-13T18:21:02.796881+0000","last_became_active":"2024-09-13T17:50:38.612577+0000","last_became_peered":"2024-09-13T17:50:38.612577+0000","last_unstale":"2024-09-13T18:21:02.796881+0000","last_undegraded":"2024-09-13T18:21:02.796881+0000","last_fullsized":"2024-09-13T18:21:02.796881+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5504,"log_dups_size":0,"ondisk_log_size":5504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:25.551060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23286,"num_write_kb":502495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'19","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306452+0000","last_change":"2024-09-13T17:50:34.587273+0000","last_active":"2024-09-13T18:21:13.306452+0000","last_peered":"2024-09-13T18:21:13.306452+0000","last_clean":"2024-09-13T18:21:13.306452+0000","last_became_active":"2024-09-13T17:50:34.587027+0000","last_became_peered":"2024-09-13T17:50:34.587027+0000","last_unstale":"2024-09-13T18:21:13.306452+0000","last_undegraded":"2024-09-13T18:21:13.306452+0000","last_fullsized":"2024-09-13T18:21:13.306452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:58.826737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'5","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.773643+0000","last_change":"2024-09-13T17:50:34.211845+0000","last_active":"2024-09-13T18:21:02.773643+0000","last_peered":"2024-09-13T18:21:02.773643+0000","last_clean":"2024-09-13T18:21:02.773643+0000","last_became_active":"2024-09-13T17:50:34.211568+0000","last_became_peered":"2024-09-13T17:50:34.211568+0000","last_unstale":"2024-09-13T18:21:02.773643+0000","last_undegraded":"2024-09-13T18:21:02.773643+0000","last_fullsized":"2024-09-13T18:21:02.773643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:56.348984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'6938","reported_seq":6965,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.647468+0000","last_change":"2024-09-13T17:50:38.610292+0000","last_active":"2024-09-13T18:21:02.647468+0000","last_peered":"2024-09-13T18:21:02.647468+0000","last_clean":"2024-09-13T18:21:02.647468+0000","last_became_active":"2024-09-13T17:50:38.609832+0000","last_became_peered":"2024-09-13T17:50:38.609832+0000","last_unstale":"2024-09-13T18:21:02.647468+0000","last_undegraded":"2024-09-13T18:21:02.647468+0000","last_fullsized":"2024-09-13T18:21:02.647468+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6938,"log_dups_size":0,"ondisk_log_size":6938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:13.783068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28644,"num_write_kb":622131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"27'12","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.931907+0000","last_change":"2024-09-13T17:50:34.593566+0000","last_active":"2024-09-13T18:21:02.931907+0000","last_peered":"2024-09-13T18:21:02.931907+0000","last_clean":"2024-09-13T18:21:02.931907+0000","last_became_active":"2024-09-13T17:50:34.588794+0000","last_became_peered":"2024-09-13T17:50:34.588794+0000","last_unstale":"2024-09-13T18:21:02.931907+0000","last_undegraded":"2024-09-13T18:21:02.931907+0000","last_fullsized":"2024-09-13T18:21:02.931907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:13.192453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'6","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.689388+0000","last_change":"2024-09-13T17:50:34.214638+0000","last_active":"2024-09-13T18:21:02.689388+0000","last_peered":"2024-09-13T18:21:02.689388+0000","last_clean":"2024-09-13T18:21:02.689388+0000","last_became_active":"2024-09-13T17:50:34.214292+0000","last_became_peered":"2024-09-13T17:50:34.214292+0000","last_unstale":"2024-09-13T18:21:02.689388+0000","last_undegraded":"2024-09-13T18:21:02.689388+0000","last_fullsized":"2024-09-13T18:21:02.689388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:57.377104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'4588","reported_seq":4615,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.886736+0000","last_change":"2024-09-13T17:50:38.613368+0000","last_active":"2024-09-13T18:21:02.886736+0000","last_peered":"2024-09-13T18:21:02.886736+0000","last_clean":"2024-09-13T18:21:02.886736+0000","last_became_active":"2024-09-13T17:50:38.613010+0000","last_became_peered":"2024-09-13T17:50:38.613010+0000","last_unstale":"2024-09-13T18:21:02.886736+0000","last_undegraded":"2024-09-13T18:21:02.886736+0000","last_fullsized":"2024-09-13T18:21:02.886736+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4588,"log_dups_size":0,"ondisk_log_size":4588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:42.706625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19354,"num_write_kb":454021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'26","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.302413+0000","last_change":"2024-09-13T17:50:34.582327+0000","last_active":"2024-09-13T18:21:13.302413+0000","last_peered":"2024-09-13T18:21:13.302413+0000","last_clean":"2024-09-13T18:21:13.302413+0000","last_became_active":"2024-09-13T17:50:34.582050+0000","last_became_peered":"2024-09-13T17:50:34.582050+0000","last_unstale":"2024-09-13T18:21:13.302413+0000","last_undegraded":"2024-09-13T18:21:13.302413+0000","last_fullsized":"2024-09-13T18:21:13.302413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:52.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'9","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.838047+0000","last_change":"2024-09-13T17:50:34.211512+0000","last_active":"2024-09-13T18:21:02.838047+0000","last_peered":"2024-09-13T18:21:02.838047+0000","last_clean":"2024-09-13T18:21:02.838047+0000","last_became_active":"2024-09-13T17:50:34.211230+0000","last_became_peered":"2024-09-13T17:50:34.211230+0000","last_unstale":"2024-09-13T18:21:02.838047+0000","last_undegraded":"2024-09-13T18:21:02.838047+0000","last_fullsized":"2024-09-13T18:21:02.838047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:40.707777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'6024","reported_seq":6052,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.702767+0000","last_change":"2024-09-13T17:50:38.608255+0000","last_active":"2024-09-13T18:21:02.702767+0000","last_peered":"2024-09-13T18:21:02.702767+0000","last_clean":"2024-09-13T18:21:02.702767+0000","last_became_active":"2024-09-13T17:50:38.606697+0000","last_became_peered":"2024-09-13T17:50:38.606697+0000","last_unstale":"2024-09-13T18:21:02.702767+0000","last_undegraded":"2024-09-13T18:21:02.702767+0000","last_fullsized":"2024-09-13T18:21:02.702767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6024,"log_dups_size":0,"ondisk_log_size":6024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:47.265007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25552,"num_write_kb":507487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'18","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.314914+0000","last_change":"2024-09-13T17:50:34.594476+0000","last_active":"2024-09-13T18:21:13.314914+0000","last_peered":"2024-09-13T18:21:13.314914+0000","last_clean":"2024-09-13T18:21:13.314914+0000","last_became_active":"2024-09-13T17:50:34.594215+0000","last_became_peered":"2024-09-13T17:50:34.594215+0000","last_unstale":"2024-09-13T18:21:13.314914+0000","last_undegraded":"2024-09-13T18:21:13.314914+0000","last_fullsized":"2024-09-13T18:21:13.314914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:02:27.528019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'284","reported_seq":315,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.685813+0000","last_change":"2024-09-13T17:50:34.204018+0000","last_active":"2024-09-13T18:21:02.685813+0000","last_peered":"2024-09-13T18:21:02.685813+0000","last_clean":"2024-09-13T18:21:02.685813+0000","last_became_active":"2024-09-13T17:50:34.203702+0000","last_became_peered":"2024-09-13T17:50:34.203702+0000","last_unstale":"2024-09-13T18:21:02.685813+0000","last_undegraded":"2024-09-13T18:21:02.685813+0000","last_fullsized":"2024-09-13T18:21:02.685813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:44.431226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":605,"num_write_kb":197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'6030","reported_seq":6069,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.817288+0000","last_change":"2024-09-13T17:51:16.365782+0000","last_active":"2024-09-13T18:21:02.817288+0000","last_peered":"2024-09-13T18:21:02.817288+0000","last_clean":"2024-09-13T18:21:02.817288+0000","last_became_active":"2024-09-13T17:51:16.364540+0000","last_became_peered":"2024-09-13T17:51:16.364540+0000","last_unstale":"2024-09-13T18:21:02.817288+0000","last_undegraded":"2024-09-13T18:21:02.817288+0000","last_fullsized":"2024-09-13T18:21:02.817288+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6030,"log_dups_size":0,"ondisk_log_size":6030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:18.753557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25617,"num_write_kb":548663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'18","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.290374+0000","last_change":"2024-09-13T17:50:34.586802+0000","last_active":"2024-09-13T18:21:13.290374+0000","last_peered":"2024-09-13T18:21:13.290374+0000","last_clean":"2024-09-13T18:21:13.290374+0000","last_became_active":"2024-09-13T17:50:34.586552+0000","last_became_peered":"2024-09-13T17:50:34.586552+0000","last_unstale":"2024-09-13T18:21:13.290374+0000","last_undegraded":"2024-09-13T18:21:13.290374+0000","last_fullsized":"2024-09-13T18:21:13.290374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:43.383975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'7","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.741537+0000","last_change":"2024-09-13T17:50:34.217083+0000","last_active":"2024-09-13T18:21:02.741537+0000","last_peered":"2024-09-13T18:21:02.741537+0000","last_clean":"2024-09-13T18:21:02.741537+0000","last_became_active":"2024-09-13T17:50:34.216758+0000","last_became_peered":"2024-09-13T17:50:34.216758+0000","last_unstale":"2024-09-13T18:21:02.741537+0000","last_undegraded":"2024-09-13T18:21:02.741537+0000","last_fullsized":"2024-09-13T18:21:02.741537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:36.473942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'4887","reported_seq":4915,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.617066+0000","last_change":"2024-09-13T17:50:38.614667+0000","last_active":"2024-09-13T18:21:02.617066+0000","last_peered":"2024-09-13T18:21:02.617066+0000","last_clean":"2024-09-13T18:21:02.617066+0000","last_became_active":"2024-09-13T17:50:38.614284+0000","last_became_peered":"2024-09-13T17:50:38.614284+0000","last_unstale":"2024-09-13T18:21:02.617066+0000","last_undegraded":"2024-09-13T18:21:02.617066+0000","last_fullsized":"2024-09-13T18:21:02.617066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4887,"log_dups_size":0,"ondisk_log_size":4887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:29.616794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20690,"num_write_kb":442963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'16","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.315360+0000","last_change":"2024-09-13T17:50:34.588603+0000","last_active":"2024-09-13T18:21:13.315360+0000","last_peered":"2024-09-13T18:21:13.315360+0000","last_clean":"2024-09-13T18:21:13.315360+0000","last_became_active":"2024-09-13T17:50:34.588317+0000","last_became_peered":"2024-09-13T17:50:34.588317+0000","last_unstale":"2024-09-13T18:21:13.315360+0000","last_undegraded":"2024-09-13T18:21:13.315360+0000","last_fullsized":"2024-09-13T18:21:13.315360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:03.743130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'3","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.727771+0000","last_change":"2024-09-13T17:50:34.204335+0000","last_active":"2024-09-13T18:21:02.727771+0000","last_peered":"2024-09-13T18:21:02.727771+0000","last_clean":"2024-09-13T18:21:02.727771+0000","last_became_active":"2024-09-13T17:50:34.204060+0000","last_became_peered":"2024-09-13T17:50:34.204060+0000","last_unstale":"2024-09-13T18:21:02.727771+0000","last_undegraded":"2024-09-13T18:21:02.727771+0000","last_fullsized":"2024-09-13T18:21:02.727771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:54:41.831344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'6031","reported_seq":6059,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.587876+0000","last_change":"2024-09-13T17:50:38.619590+0000","last_active":"2024-09-13T18:21:02.587876+0000","last_peered":"2024-09-13T18:21:02.587876+0000","last_clean":"2024-09-13T18:21:02.587876+0000","last_became_active":"2024-09-13T17:50:38.617257+0000","last_became_peered":"2024-09-13T17:50:38.617257+0000","last_unstale":"2024-09-13T18:21:02.587876+0000","last_undegraded":"2024-09-13T18:21:02.587876+0000","last_fullsized":"2024-09-13T18:21:02.587876+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6031,"log_dups_size":0,"ondisk_log_size":6031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:34.870291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25529,"num_write_kb":564991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'16","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310390+0000","last_change":"2024-09-13T17:50:34.588466+0000","last_active":"2024-09-13T18:21:13.310390+0000","last_peered":"2024-09-13T18:21:13.310390+0000","last_clean":"2024-09-13T18:21:13.310390+0000","last_became_active":"2024-09-13T17:50:34.588063+0000","last_became_peered":"2024-09-13T17:50:34.588063+0000","last_unstale":"2024-09-13T18:21:13.310390+0000","last_undegraded":"2024-09-13T18:21:13.310390+0000","last_fullsized":"2024-09-13T18:21:13.310390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:43:00.669203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'4","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.629940+0000","last_change":"2024-09-13T17:50:34.209239+0000","last_active":"2024-09-13T18:21:02.629940+0000","last_peered":"2024-09-13T18:21:02.629940+0000","last_clean":"2024-09-13T18:21:02.629940+0000","last_became_active":"2024-09-13T17:50:34.208196+0000","last_became_peered":"2024-09-13T17:50:34.208196+0000","last_unstale":"2024-09-13T18:21:02.629940+0000","last_undegraded":"2024-09-13T18:21:02.629940+0000","last_fullsized":"2024-09-13T18:21:02.629940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:39.253917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'5410","reported_seq":5438,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.883285+0000","last_change":"2024-09-13T17:50:38.611021+0000","last_active":"2024-09-13T18:21:02.883285+0000","last_peered":"2024-09-13T18:21:02.883285+0000","last_clean":"2024-09-13T18:21:02.883285+0000","last_became_active":"2024-09-13T17:50:38.609794+0000","last_became_peered":"2024-09-13T17:50:38.609794+0000","last_unstale":"2024-09-13T18:21:02.883285+0000","last_undegraded":"2024-09-13T18:21:02.883285+0000","last_fullsized":"2024-09-13T18:21:02.883285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5410,"log_dups_size":0,"ondisk_log_size":5410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:59.980236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22186,"num_write_kb":506224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'22","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.282060+0000","last_change":"2024-09-13T17:50:34.584290+0000","last_active":"2024-09-13T18:21:13.282060+0000","last_peered":"2024-09-13T18:21:13.282060+0000","last_clean":"2024-09-13T18:21:13.282060+0000","last_became_active":"2024-09-13T17:50:34.584023+0000","last_became_peered":"2024-09-13T17:50:34.584023+0000","last_unstale":"2024-09-13T18:21:13.282060+0000","last_undegraded":"2024-09-13T18:21:13.282060+0000","last_fullsized":"2024-09-13T18:21:13.282060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:11.304986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'127","reported_seq":279,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:31.126933+0000","last_change":"2024-09-13T17:50:34.198932+0000","last_active":"2024-09-13T18:20:31.126933+0000","last_peered":"2024-09-13T18:20:31.126933+0000","last_clean":"2024-09-13T18:20:31.126933+0000","last_became_active":"2024-09-13T17:50:34.198603+0000","last_became_peered":"2024-09-13T17:50:34.198603+0000","last_unstale":"2024-09-13T18:20:31.126933+0000","last_undegraded":"2024-09-13T18:20:31.126933+0000","last_fullsized":"2024-09-13T18:20:31.126933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:00.917126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":94032,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":179,"num_write":125,"num_write_kb":180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'5807","reported_seq":5835,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.679912+0000","last_change":"2024-09-13T17:50:38.611326+0000","last_active":"2024-09-13T18:21:02.679912+0000","last_peered":"2024-09-13T18:21:02.679912+0000","last_clean":"2024-09-13T18:21:02.679912+0000","last_became_active":"2024-09-13T17:50:38.610900+0000","last_became_peered":"2024-09-13T17:50:38.610900+0000","last_unstale":"2024-09-13T18:21:02.679912+0000","last_undegraded":"2024-09-13T18:21:02.679912+0000","last_fullsized":"2024-09-13T18:21:02.679912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5807,"log_dups_size":0,"ondisk_log_size":5807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:17:09.994736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24429,"num_write_kb":531605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'13","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310665+0000","last_change":"2024-09-13T17:50:34.594482+0000","last_active":"2024-09-13T18:21:13.310665+0000","last_peered":"2024-09-13T18:21:13.310665+0000","last_clean":"2024-09-13T18:21:13.310665+0000","last_became_active":"2024-09-13T17:50:34.594201+0000","last_became_peered":"2024-09-13T17:50:34.594201+0000","last_unstale":"2024-09-13T18:21:13.310665+0000","last_undegraded":"2024-09-13T18:21:13.310665+0000","last_fullsized":"2024-09-13T18:21:13.310665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:06.697507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'8","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.813884+0000","last_change":"2024-09-13T17:50:34.218376+0000","last_active":"2024-09-13T18:21:02.813884+0000","last_peered":"2024-09-13T18:21:02.813884+0000","last_clean":"2024-09-13T18:21:02.813884+0000","last_became_active":"2024-09-13T17:50:34.218069+0000","last_became_peered":"2024-09-13T17:50:34.218069+0000","last_unstale":"2024-09-13T18:21:02.813884+0000","last_undegraded":"2024-09-13T18:21:02.813884+0000","last_fullsized":"2024-09-13T18:21:02.813884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:53:54.288969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'4957","reported_seq":4985,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.963407+0000","last_change":"2024-09-13T17:50:38.613206+0000","last_active":"2024-09-13T18:21:02.963407+0000","last_peered":"2024-09-13T18:21:02.963407+0000","last_clean":"2024-09-13T18:21:02.963407+0000","last_became_active":"2024-09-13T17:50:38.612856+0000","last_became_peered":"2024-09-13T17:50:38.612856+0000","last_unstale":"2024-09-13T18:21:02.963407+0000","last_undegraded":"2024-09-13T18:21:02.963407+0000","last_fullsized":"2024-09-13T18:21:02.963407+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4957,"log_dups_size":0,"ondisk_log_size":4957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:31:20.190996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20520,"num_write_kb":434212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'16","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.316288+0000","last_change":"2024-09-13T17:50:34.582931+0000","last_active":"2024-09-13T18:21:13.316288+0000","last_peered":"2024-09-13T18:21:13.316288+0000","last_clean":"2024-09-13T18:21:13.316288+0000","last_became_active":"2024-09-13T17:50:34.582446+0000","last_became_peered":"2024-09-13T17:50:34.582446+0000","last_unstale":"2024-09-13T18:21:13.316288+0000","last_undegraded":"2024-09-13T18:21:13.316288+0000","last_fullsized":"2024-09-13T18:21:13.316288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:30.554785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'8","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.157427+0000","last_change":"2024-09-13T17:50:34.201237+0000","last_active":"2024-09-13T18:20:25.157427+0000","last_peered":"2024-09-13T18:20:25.157427+0000","last_clean":"2024-09-13T18:20:25.157427+0000","last_became_active":"2024-09-13T17:50:34.200873+0000","last_became_peered":"2024-09-13T17:50:34.200873+0000","last_unstale":"2024-09-13T18:20:25.157427+0000","last_undegraded":"2024-09-13T18:20:25.157427+0000","last_fullsized":"2024-09-13T18:20:25.157427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:02.711342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":185,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:18:31.266834+0000","last_change":"2024-09-13T17:50:22.095367+0000","last_active":"2024-09-13T18:18:31.266834+0000","last_peered":"2024-09-13T18:18:31.266834+0000","last_clean":"2024-09-13T18:18:31.266834+0000","last_became_active":"2024-09-13T17:50:22.093734+0000","last_became_peered":"2024-09-13T17:50:22.093734+0000","last_unstale":"2024-09-13T18:18:31.266834+0000","last_undegraded":"2024-09-13T18:18:31.266834+0000","last_fullsized":"2024-09-13T18:18:31.266834+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_clean_scrub_stamp":"2024-09-13T17:50:21.057812+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:23.531613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'5208","reported_seq":5236,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.886919+0000","last_change":"2024-09-13T17:50:38.618880+0000","last_active":"2024-09-13T18:21:02.886919+0000","last_peered":"2024-09-13T18:21:02.886919+0000","last_clean":"2024-09-13T18:21:02.886919+0000","last_became_active":"2024-09-13T17:50:38.618488+0000","last_became_peered":"2024-09-13T17:50:38.618488+0000","last_unstale":"2024-09-13T18:21:02.886919+0000","last_undegraded":"2024-09-13T18:21:02.886919+0000","last_fullsized":"2024-09-13T18:21:02.886919+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5208,"log_dups_size":0,"ondisk_log_size":5208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:03.431876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22093,"num_write_kb":481370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'22","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317778+0000","last_change":"2024-09-13T17:50:34.585881+0000","last_active":"2024-09-13T18:21:13.317778+0000","last_peered":"2024-09-13T18:21:13.317778+0000","last_clean":"2024-09-13T18:21:13.317778+0000","last_became_active":"2024-09-13T17:50:34.585166+0000","last_became_peered":"2024-09-13T17:50:34.585166+0000","last_unstale":"2024-09-13T18:21:13.317778+0000","last_undegraded":"2024-09-13T18:21:13.317778+0000","last_fullsized":"2024-09-13T18:21:13.317778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:06.654658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'5","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.654921+0000","last_change":"2024-09-13T17:50:34.210934+0000","last_active":"2024-09-13T18:21:02.654921+0000","last_peered":"2024-09-13T18:21:02.654921+0000","last_clean":"2024-09-13T18:21:02.654921+0000","last_became_active":"2024-09-13T17:50:34.210589+0000","last_became_peered":"2024-09-13T17:50:34.210589+0000","last_unstale":"2024-09-13T18:21:02.654921+0000","last_undegraded":"2024-09-13T18:21:02.654921+0000","last_fullsized":"2024-09-13T18:21:02.654921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:26.213295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'5302","reported_seq":5340,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.817379+0000","last_change":"2024-09-13T17:51:16.364911+0000","last_active":"2024-09-13T18:21:02.817379+0000","last_peered":"2024-09-13T18:21:02.817379+0000","last_clean":"2024-09-13T18:21:02.817379+0000","last_became_active":"2024-09-13T17:51:16.363974+0000","last_became_peered":"2024-09-13T17:51:16.363974+0000","last_unstale":"2024-09-13T18:21:02.817379+0000","last_undegraded":"2024-09-13T18:21:02.817379+0000","last_fullsized":"2024-09-13T18:21:02.817379+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5302,"log_dups_size":0,"ondisk_log_size":5302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:12.730197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22724,"num_write_kb":488088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'20","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.309412+0000","last_change":"2024-09-13T17:50:34.590169+0000","last_active":"2024-09-13T18:21:13.309412+0000","last_peered":"2024-09-13T18:21:13.309412+0000","last_clean":"2024-09-13T18:21:13.309412+0000","last_became_active":"2024-09-13T17:50:34.589434+0000","last_became_peered":"2024-09-13T17:50:34.589434+0000","last_unstale":"2024-09-13T18:21:13.309412+0000","last_undegraded":"2024-09-13T18:21:13.309412+0000","last_fullsized":"2024-09-13T18:21:13.309412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:12.545412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'4","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.155671+0000","last_change":"2024-09-13T17:50:34.204269+0000","last_active":"2024-09-13T18:20:25.155671+0000","last_peered":"2024-09-13T18:20:25.155671+0000","last_clean":"2024-09-13T18:20:25.155671+0000","last_became_active":"2024-09-13T17:50:34.203874+0000","last_became_peered":"2024-09-13T17:50:34.203874+0000","last_unstale":"2024-09-13T18:20:25.155671+0000","last_undegraded":"2024-09-13T18:20:25.155671+0000","last_fullsized":"2024-09-13T18:20:25.155671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:10.410267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'5348","reported_seq":5375,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.944893+0000","last_change":"2024-09-13T17:50:38.615989+0000","last_active":"2024-09-13T18:21:02.944893+0000","last_peered":"2024-09-13T18:21:02.944893+0000","last_clean":"2024-09-13T18:21:02.944893+0000","last_became_active":"2024-09-13T17:50:38.615530+0000","last_became_peered":"2024-09-13T17:50:38.615530+0000","last_unstale":"2024-09-13T18:21:02.944893+0000","last_undegraded":"2024-09-13T18:21:02.944893+0000","last_fullsized":"2024-09-13T18:21:02.944893+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5348,"log_dups_size":0,"ondisk_log_size":5348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:41.064092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22134,"num_write_kb":473293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'20","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.321742+0000","last_change":"2024-09-13T17:50:34.595120+0000","last_active":"2024-09-13T18:21:13.321742+0000","last_peered":"2024-09-13T18:21:13.321742+0000","last_clean":"2024-09-13T18:21:13.321742+0000","last_became_active":"2024-09-13T17:50:34.594842+0000","last_became_peered":"2024-09-13T17:50:34.594842+0000","last_unstale":"2024-09-13T18:21:13.321742+0000","last_undegraded":"2024-09-13T18:21:13.321742+0000","last_fullsized":"2024-09-13T18:21:13.321742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:15:38.982524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'9","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.683517+0000","last_change":"2024-09-13T17:50:34.199957+0000","last_active":"2024-09-13T18:21:02.683517+0000","last_peered":"2024-09-13T18:21:02.683517+0000","last_clean":"2024-09-13T18:21:02.683517+0000","last_became_active":"2024-09-13T17:50:34.199650+0000","last_became_peered":"2024-09-13T17:50:34.199650+0000","last_unstale":"2024-09-13T18:21:02.683517+0000","last_undegraded":"2024-09-13T18:21:02.683517+0000","last_fullsized":"2024-09-13T18:21:02.683517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:26.929549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'5161","reported_seq":5189,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.762344+0000","last_change":"2024-09-13T17:50:38.609731+0000","last_active":"2024-09-13T18:21:02.762344+0000","last_peered":"2024-09-13T18:21:02.762344+0000","last_clean":"2024-09-13T18:21:02.762344+0000","last_became_active":"2024-09-13T17:50:38.609354+0000","last_became_peered":"2024-09-13T17:50:38.609354+0000","last_unstale":"2024-09-13T18:21:02.762344+0000","last_undegraded":"2024-09-13T18:21:02.762344+0000","last_fullsized":"2024-09-13T18:21:02.762344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5161,"log_dups_size":0,"ondisk_log_size":5161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:14.142369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21929,"num_write_kb":464406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'10","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.313609+0000","last_change":"2024-09-13T17:50:34.595146+0000","last_active":"2024-09-13T18:21:13.313609+0000","last_peered":"2024-09-13T18:21:13.313609+0000","last_clean":"2024-09-13T18:21:13.313609+0000","last_became_active":"2024-09-13T17:50:34.594876+0000","last_became_peered":"2024-09-13T17:50:34.594876+0000","last_unstale":"2024-09-13T18:21:13.313609+0000","last_undegraded":"2024-09-13T18:21:13.313609+0000","last_fullsized":"2024-09-13T18:21:13.313609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:26:10.638438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'4","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:19:51.940271+0000","last_change":"2024-09-13T17:50:34.200640+0000","last_active":"2024-09-13T18:19:51.940271+0000","last_peered":"2024-09-13T18:19:51.940271+0000","last_clean":"2024-09-13T18:19:51.940271+0000","last_became_active":"2024-09-13T17:50:34.200197+0000","last_became_peered":"2024-09-13T17:50:34.200197+0000","last_unstale":"2024-09-13T18:19:51.940271+0000","last_undegraded":"2024-09-13T18:19:51.940271+0000","last_fullsized":"2024-09-13T18:19:51.940271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:37:53.039720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'5052","reported_seq":5090,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.778127+0000","last_change":"2024-09-13T17:51:16.281995+0000","last_active":"2024-09-13T18:21:02.778127+0000","last_peered":"2024-09-13T18:21:02.778127+0000","last_clean":"2024-09-13T18:21:02.778127+0000","last_became_active":"2024-09-13T17:51:16.281476+0000","last_became_peered":"2024-09-13T17:51:16.281476+0000","last_unstale":"2024-09-13T18:21:02.778127+0000","last_undegraded":"2024-09-13T18:21:02.778127+0000","last_fullsized":"2024-09-13T18:21:02.778127+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5052,"log_dups_size":0,"ondisk_log_size":5052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:21:56.254641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20895,"num_write_kb":464282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3,6],"acting":[4,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'18","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.301423+0000","last_change":"2024-09-13T17:50:34.585741+0000","last_active":"2024-09-13T18:21:13.301423+0000","last_peered":"2024-09-13T18:21:13.301423+0000","last_clean":"2024-09-13T18:21:13.301423+0000","last_became_active":"2024-09-13T17:50:34.585464+0000","last_became_peered":"2024-09-13T17:50:34.585464+0000","last_unstale":"2024-09-13T18:21:13.301423+0000","last_undegraded":"2024-09-13T18:21:13.301423+0000","last_fullsized":"2024-09-13T18:21:13.301423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:59:22.031459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'19","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.673198+0000","last_change":"2024-09-13T17:50:34.201272+0000","last_active":"2024-09-13T18:21:02.673198+0000","last_peered":"2024-09-13T18:21:02.673198+0000","last_clean":"2024-09-13T18:21:02.673198+0000","last_became_active":"2024-09-13T17:50:34.200936+0000","last_became_peered":"2024-09-13T17:50:34.200936+0000","last_unstale":"2024-09-13T18:21:02.673198+0000","last_undegraded":"2024-09-13T18:21:02.673198+0000","last_fullsized":"2024-09-13T18:21:02.673198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:52.094879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'6502","reported_seq":6528,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.900747+0000","last_change":"2024-09-13T17:50:38.615229+0000","last_active":"2024-09-13T18:21:02.900747+0000","last_peered":"2024-09-13T18:21:02.900747+0000","last_clean":"2024-09-13T18:21:02.900747+0000","last_became_active":"2024-09-13T17:50:38.614739+0000","last_became_peered":"2024-09-13T17:50:38.614739+0000","last_unstale":"2024-09-13T18:21:02.900747+0000","last_undegraded":"2024-09-13T18:21:02.900747+0000","last_fullsized":"2024-09-13T18:21:02.900747+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6502,"log_dups_size":0,"ondisk_log_size":6502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:18.619529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27696,"num_write_kb":544723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'20","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.287949+0000","last_change":"2024-09-13T17:50:34.588848+0000","last_active":"2024-09-13T18:21:13.287949+0000","last_peered":"2024-09-13T18:21:13.287949+0000","last_clean":"2024-09-13T18:21:13.287949+0000","last_became_active":"2024-09-13T17:50:34.588565+0000","last_became_peered":"2024-09-13T17:50:34.588565+0000","last_unstale":"2024-09-13T18:21:13.287949+0000","last_undegraded":"2024-09-13T18:21:13.287949+0000","last_fullsized":"2024-09-13T18:21:13.287949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:47.927865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'7","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.707815+0000","last_change":"2024-09-13T17:50:34.199008+0000","last_active":"2024-09-13T18:21:02.707815+0000","last_peered":"2024-09-13T18:21:02.707815+0000","last_clean":"2024-09-13T18:21:02.707815+0000","last_became_active":"2024-09-13T17:50:34.197839+0000","last_became_peered":"2024-09-13T17:50:34.197839+0000","last_unstale":"2024-09-13T18:21:02.707815+0000","last_undegraded":"2024-09-13T18:21:02.707815+0000","last_fullsized":"2024-09-13T18:21:02.707815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:51.430710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'6326","reported_seq":6352,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:01.698688+0000","last_change":"2024-09-13T17:50:38.611080+0000","last_active":"2024-09-13T18:21:01.698688+0000","last_peered":"2024-09-13T18:21:01.698688+0000","last_clean":"2024-09-13T18:21:01.698688+0000","last_became_active":"2024-09-13T17:50:38.610688+0000","last_became_peered":"2024-09-13T17:50:38.610688+0000","last_unstale":"2024-09-13T18:21:01.698688+0000","last_undegraded":"2024-09-13T18:21:01.698688+0000","last_fullsized":"2024-09-13T18:21:01.698688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6326,"log_dups_size":0,"ondisk_log_size":6326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:28.527617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27199,"num_write_kb":538670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'29","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317546+0000","last_change":"2024-09-13T17:50:34.594763+0000","last_active":"2024-09-13T18:21:13.317546+0000","last_peered":"2024-09-13T18:21:13.317546+0000","last_clean":"2024-09-13T18:21:13.317546+0000","last_became_active":"2024-09-13T17:50:34.594471+0000","last_became_peered":"2024-09-13T17:50:34.594471+0000","last_unstale":"2024-09-13T18:21:13.317546+0000","last_undegraded":"2024-09-13T18:21:13.317546+0000","last_fullsized":"2024-09-13T18:21:13.317546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:20:58.718271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'6","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.839570+0000","last_change":"2024-09-13T17:50:34.209432+0000","last_active":"2024-09-13T18:21:02.839570+0000","last_peered":"2024-09-13T18:21:02.839570+0000","last_clean":"2024-09-13T18:21:02.839570+0000","last_became_active":"2024-09-13T17:50:34.208817+0000","last_became_peered":"2024-09-13T17:50:34.208817+0000","last_unstale":"2024-09-13T18:21:02.839570+0000","last_undegraded":"2024-09-13T18:21:02.839570+0000","last_fullsized":"2024-09-13T18:21:02.839570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:49.587528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'5788","reported_seq":5815,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.947982+0000","last_change":"2024-09-13T17:50:38.609461+0000","last_active":"2024-09-13T18:21:02.947982+0000","last_peered":"2024-09-13T18:21:02.947982+0000","last_clean":"2024-09-13T18:21:02.947982+0000","last_became_active":"2024-09-13T17:50:38.609128+0000","last_became_peered":"2024-09-13T17:50:38.609128+0000","last_unstale":"2024-09-13T18:21:02.947982+0000","last_undegraded":"2024-09-13T18:21:02.947982+0000","last_fullsized":"2024-09-13T18:21:02.947982+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5788,"log_dups_size":0,"ondisk_log_size":5788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:02.286825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24140,"num_write_kb":538739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'24","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.305222+0000","last_change":"2024-09-13T17:50:34.592970+0000","last_active":"2024-09-13T18:21:13.305222+0000","last_peered":"2024-09-13T18:21:13.305222+0000","last_clean":"2024-09-13T18:21:13.305222+0000","last_became_active":"2024-09-13T17:50:34.592669+0000","last_became_peered":"2024-09-13T17:50:34.592669+0000","last_unstale":"2024-09-13T18:21:13.305222+0000","last_undegraded":"2024-09-13T18:21:13.305222+0000","last_fullsized":"2024-09-13T18:21:13.305222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:52.031261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'15","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.785525+0000","last_change":"2024-09-13T17:50:34.200798+0000","last_active":"2024-09-13T18:21:02.785525+0000","last_peered":"2024-09-13T18:21:02.785525+0000","last_clean":"2024-09-13T18:21:02.785525+0000","last_became_active":"2024-09-13T17:50:34.200529+0000","last_became_peered":"2024-09-13T17:50:34.200529+0000","last_unstale":"2024-09-13T18:21:02.785525+0000","last_undegraded":"2024-09-13T18:21:02.785525+0000","last_fullsized":"2024-09-13T18:21:02.785525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:26.334296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'5464","reported_seq":5490,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.598966+0000","last_change":"2024-09-13T17:50:38.606287+0000","last_active":"2024-09-13T18:21:02.598966+0000","last_peered":"2024-09-13T18:21:02.598966+0000","last_clean":"2024-09-13T18:21:02.598966+0000","last_became_active":"2024-09-13T17:50:38.605826+0000","last_became_peered":"2024-09-13T17:50:38.605826+0000","last_unstale":"2024-09-13T18:21:02.598966+0000","last_undegraded":"2024-09-13T18:21:02.598966+0000","last_fullsized":"2024-09-13T18:21:02.598966+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5464,"log_dups_size":0,"ondisk_log_size":5464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:18.709454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23044,"num_write_kb":472739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'18","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310869+0000","last_change":"2024-09-13T17:50:34.591579+0000","last_active":"2024-09-13T18:21:13.310869+0000","last_peered":"2024-09-13T18:21:13.310869+0000","last_clean":"2024-09-13T18:21:13.310869+0000","last_became_active":"2024-09-13T17:50:34.586165+0000","last_became_peered":"2024-09-13T17:50:34.586165+0000","last_unstale":"2024-09-13T18:21:13.310869+0000","last_undegraded":"2024-09-13T18:21:13.310869+0000","last_fullsized":"2024-09-13T18:21:13.310869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:56:19.205176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'417","reported_seq":447,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.705527+0000","last_change":"2024-09-13T17:50:34.203218+0000","last_active":"2024-09-13T18:21:02.705527+0000","last_peered":"2024-09-13T18:21:02.705527+0000","last_clean":"2024-09-13T18:21:02.705527+0000","last_became_active":"2024-09-13T17:50:34.202948+0000","last_became_peered":"2024-09-13T17:50:34.202948+0000","last_unstale":"2024-09-13T18:21:02.705527+0000","last_undegraded":"2024-09-13T18:21:02.705527+0000","last_fullsized":"2024-09-13T18:21:02.705527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:46.535569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":4324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'5887","reported_seq":5914,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.956090+0000","last_change":"2024-09-13T17:50:38.610597+0000","last_active":"2024-09-13T18:21:02.956090+0000","last_peered":"2024-09-13T18:21:02.956090+0000","last_clean":"2024-09-13T18:21:02.956090+0000","last_became_active":"2024-09-13T17:50:38.610055+0000","last_became_peered":"2024-09-13T17:50:38.610055+0000","last_unstale":"2024-09-13T18:21:02.956090+0000","last_undegraded":"2024-09-13T18:21:02.956090+0000","last_fullsized":"2024-09-13T18:21:02.956090+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5887,"log_dups_size":0,"ondisk_log_size":5887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:55.754936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24664,"num_write_kb":495799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'21","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317972+0000","last_change":"2024-09-13T17:50:34.593963+0000","last_active":"2024-09-13T18:21:13.317972+0000","last_peered":"2024-09-13T18:21:13.317972+0000","last_clean":"2024-09-13T18:21:13.317972+0000","last_became_active":"2024-09-13T17:50:34.593454+0000","last_became_peered":"2024-09-13T17:50:34.593454+0000","last_unstale":"2024-09-13T18:21:13.317972+0000","last_undegraded":"2024-09-13T18:21:13.317972+0000","last_fullsized":"2024-09-13T18:21:13.317972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:18.883521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'67","reported_seq":96,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.464353+0000","last_change":"2024-09-13T17:50:34.207936+0000","last_active":"2024-09-13T18:21:02.464353+0000","last_peered":"2024-09-13T18:21:02.464353+0000","last_clean":"2024-09-13T18:21:02.464353+0000","last_became_active":"2024-09-13T17:50:34.207605+0000","last_became_peered":"2024-09-13T17:50:34.207605+0000","last_unstale":"2024-09-13T18:21:02.464353+0000","last_undegraded":"2024-09-13T18:21:02.464353+0000","last_fullsized":"2024-09-13T18:21:02.464353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:05.494406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'5937","reported_seq":5964,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.965268+0000","last_change":"2024-09-13T17:50:38.616297+0000","last_active":"2024-09-13T18:21:02.965268+0000","last_peered":"2024-09-13T18:21:02.965268+0000","last_clean":"2024-09-13T18:21:02.965268+0000","last_became_active":"2024-09-13T17:50:38.614165+0000","last_became_peered":"2024-09-13T17:50:38.614165+0000","last_unstale":"2024-09-13T18:21:02.965268+0000","last_undegraded":"2024-09-13T18:21:02.965268+0000","last_fullsized":"2024-09-13T18:21:02.965268+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5937,"log_dups_size":0,"ondisk_log_size":5937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:11.357026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25111,"num_write_kb":509795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'25","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.311085+0000","last_change":"2024-09-13T17:50:34.583001+0000","last_active":"2024-09-13T18:21:13.311085+0000","last_peered":"2024-09-13T18:21:13.311085+0000","last_clean":"2024-09-13T18:21:13.311085+0000","last_became_active":"2024-09-13T17:50:34.582529+0000","last_became_peered":"2024-09-13T17:50:34.582529+0000","last_unstale":"2024-09-13T18:21:13.311085+0000","last_undegraded":"2024-09-13T18:21:13.311085+0000","last_fullsized":"2024-09-13T18:21:13.311085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:05.287181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'5","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:53.035777+0000","last_change":"2024-09-13T17:50:34.205138+0000","last_active":"2024-09-13T18:20:53.035777+0000","last_peered":"2024-09-13T18:20:53.035777+0000","last_clean":"2024-09-13T18:20:53.035777+0000","last_became_active":"2024-09-13T17:50:34.203360+0000","last_became_peered":"2024-09-13T17:50:34.203360+0000","last_unstale":"2024-09-13T18:20:53.035777+0000","last_undegraded":"2024-09-13T18:20:53.035777+0000","last_fullsized":"2024-09-13T18:20:53.035777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:35.756963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'5934","reported_seq":5961,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.811068+0000","last_change":"2024-09-13T17:50:38.611036+0000","last_active":"2024-09-13T18:21:02.811068+0000","last_peered":"2024-09-13T18:21:02.811068+0000","last_clean":"2024-09-13T18:21:02.811068+0000","last_became_active":"2024-09-13T17:50:38.610692+0000","last_became_peered":"2024-09-13T17:50:38.610692+0000","last_unstale":"2024-09-13T18:21:02.811068+0000","last_undegraded":"2024-09-13T18:21:02.811068+0000","last_fullsized":"2024-09-13T18:21:02.811068+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5934,"log_dups_size":0,"ondisk_log_size":5934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:10.147998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25100,"num_write_kb":497535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'23","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.311752+0000","last_change":"2024-09-13T17:51:16.365295+0000","last_active":"2024-09-13T18:21:13.311752+0000","last_peered":"2024-09-13T18:21:13.311752+0000","last_clean":"2024-09-13T18:21:13.311752+0000","last_became_active":"2024-09-13T17:51:16.364437+0000","last_became_peered":"2024-09-13T17:51:16.364437+0000","last_unstale":"2024-09-13T18:21:13.311752+0000","last_undegraded":"2024-09-13T18:21:13.311752+0000","last_fullsized":"2024-09-13T18:21:13.311752+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:23.208899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"27'10","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.645634+0000","last_change":"2024-09-13T17:50:34.204376+0000","last_active":"2024-09-13T18:21:02.645634+0000","last_peered":"2024-09-13T18:21:02.645634+0000","last_clean":"2024-09-13T18:21:02.645634+0000","last_became_active":"2024-09-13T17:50:34.204003+0000","last_became_peered":"2024-09-13T17:50:34.204003+0000","last_unstale":"2024-09-13T18:21:02.645634+0000","last_undegraded":"2024-09-13T18:21:02.645634+0000","last_fullsized":"2024-09-13T18:21:02.645634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:14.023990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'5363","reported_seq":5390,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.796875+0000","last_change":"2024-09-13T17:50:38.606144+0000","last_active":"2024-09-13T18:21:02.796875+0000","last_peered":"2024-09-13T18:21:02.796875+0000","last_clean":"2024-09-13T18:21:02.796875+0000","last_became_active":"2024-09-13T17:50:38.605646+0000","last_became_peered":"2024-09-13T17:50:38.605646+0000","last_unstale":"2024-09-13T18:21:02.796875+0000","last_undegraded":"2024-09-13T18:21:02.796875+0000","last_fullsized":"2024-09-13T18:21:02.796875+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5363,"log_dups_size":0,"ondisk_log_size":5363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:07.120075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22831,"num_write_kb":508903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'6067","reported_seq":6096,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.803568+0000","last_change":"2024-09-13T17:50:38.618855+0000","last_active":"2024-09-13T18:21:02.803568+0000","last_peered":"2024-09-13T18:21:02.803568+0000","last_clean":"2024-09-13T18:21:02.803568+0000","last_became_active":"2024-09-13T17:50:38.618449+0000","last_became_peered":"2024-09-13T17:50:38.618449+0000","last_unstale":"2024-09-13T18:21:02.803568+0000","last_undegraded":"2024-09-13T18:21:02.803568+0000","last_fullsized":"2024-09-13T18:21:02.803568+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6067,"log_dups_size":0,"ondisk_log_size":6067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:29:01.114087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25532,"num_write_kb":568825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'21","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.298787+0000","last_change":"2024-09-13T17:50:34.590079+0000","last_active":"2024-09-13T18:21:13.298787+0000","last_peered":"2024-09-13T18:21:13.298787+0000","last_clean":"2024-09-13T18:21:13.298787+0000","last_became_active":"2024-09-13T17:50:34.589396+0000","last_became_peered":"2024-09-13T17:50:34.589396+0000","last_unstale":"2024-09-13T18:21:13.298787+0000","last_undegraded":"2024-09-13T18:21:13.298787+0000","last_fullsized":"2024-09-13T18:21:13.298787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:03.308341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:17:11.375959+0000","last_change":"2024-09-13T17:50:34.206575+0000","last_active":"2024-09-13T18:17:11.375959+0000","last_peered":"2024-09-13T18:17:11.375959+0000","last_clean":"2024-09-13T18:17:11.375959+0000","last_became_active":"2024-09-13T17:50:34.206308+0000","last_became_peered":"2024-09-13T17:50:34.206308+0000","last_unstale":"2024-09-13T18:17:11.375959+0000","last_undegraded":"2024-09-13T18:17:11.375959+0000","last_fullsized":"2024-09-13T18:17:11.375959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:34.766538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'5881","reported_seq":5909,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.927672+0000","last_change":"2024-09-13T17:50:38.618131+0000","last_active":"2024-09-13T18:21:02.927672+0000","last_peered":"2024-09-13T18:21:02.927672+0000","last_clean":"2024-09-13T18:21:02.927672+0000","last_became_active":"2024-09-13T17:50:38.617611+0000","last_became_peered":"2024-09-13T17:50:38.617611+0000","last_unstale":"2024-09-13T18:21:02.927672+0000","last_undegraded":"2024-09-13T18:21:02.927672+0000","last_fullsized":"2024-09-13T18:21:02.927672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5881,"log_dups_size":0,"ondisk_log_size":5881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:30.746445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25019,"num_write_kb":530139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'27","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.307795+0000","last_change":"2024-09-13T17:50:34.582153+0000","last_active":"2024-09-13T18:21:13.307795+0000","last_peered":"2024-09-13T18:21:13.307795+0000","last_clean":"2024-09-13T18:21:13.307795+0000","last_became_active":"2024-09-13T17:50:34.581708+0000","last_became_peered":"2024-09-13T17:50:34.581708+0000","last_unstale":"2024-09-13T18:21:13.307795+0000","last_undegraded":"2024-09-13T18:21:13.307795+0000","last_fullsized":"2024-09-13T18:21:13.307795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:22.238810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'4","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:19:47.928146+0000","last_change":"2024-09-13T17:50:34.205489+0000","last_active":"2024-09-13T18:19:47.928146+0000","last_peered":"2024-09-13T18:19:47.928146+0000","last_clean":"2024-09-13T18:19:47.928146+0000","last_became_active":"2024-09-13T17:50:34.205202+0000","last_became_peered":"2024-09-13T17:50:34.205202+0000","last_unstale":"2024-09-13T18:19:47.928146+0000","last_undegraded":"2024-09-13T18:19:47.928146+0000","last_fullsized":"2024-09-13T18:19:47.928146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:54.450276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'6","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.825727+0000","last_change":"2024-09-13T17:50:34.202470+0000","last_active":"2024-09-13T18:21:02.825727+0000","last_peered":"2024-09-13T18:21:02.825727+0000","last_clean":"2024-09-13T18:21:02.825727+0000","last_became_active":"2024-09-13T17:50:34.202163+0000","last_became_peered":"2024-09-13T17:50:34.202163+0000","last_unstale":"2024-09-13T18:21:02.825727+0000","last_undegraded":"2024-09-13T18:21:02.825727+0000","last_fullsized":"2024-09-13T18:21:02.825727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:03.472932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'18","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312647+0000","last_change":"2024-09-13T17:50:34.590947+0000","last_active":"2024-09-13T18:21:13.312647+0000","last_peered":"2024-09-13T18:21:13.312647+0000","last_clean":"2024-09-13T18:21:13.312647+0000","last_became_active":"2024-09-13T17:50:34.590497+0000","last_became_peered":"2024-09-13T17:50:34.590497+0000","last_unstale":"2024-09-13T18:21:13.312647+0000","last_undegraded":"2024-09-13T18:21:13.312647+0000","last_fullsized":"2024-09-13T18:21:13.312647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:56:27.848092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'4651","reported_seq":4677,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.602408+0000","last_change":"2024-09-13T17:50:38.618518+0000","last_active":"2024-09-13T18:21:02.602408+0000","last_peered":"2024-09-13T18:21:02.602408+0000","last_clean":"2024-09-13T18:21:02.602408+0000","last_became_active":"2024-09-13T17:50:38.618108+0000","last_became_peered":"2024-09-13T17:50:38.618108+0000","last_unstale":"2024-09-13T18:21:02.602408+0000","last_undegraded":"2024-09-13T18:21:02.602408+0000","last_fullsized":"2024-09-13T18:21:02.602408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4651,"log_dups_size":0,"ondisk_log_size":4651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:42.260237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19488,"num_write_kb":413684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'690","reported_seq":718,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.791718+0000","last_change":"2024-09-13T17:50:34.218459+0000","last_active":"2024-09-13T18:21:02.791718+0000","last_peered":"2024-09-13T18:21:02.791718+0000","last_clean":"2024-09-13T18:21:02.791718+0000","last_became_active":"2024-09-13T17:50:34.218087+0000","last_became_peered":"2024-09-13T17:50:34.218087+0000","last_unstale":"2024-09-13T18:21:02.791718+0000","last_undegraded":"2024-09-13T18:21:02.791718+0000","last_fullsized":"2024-09-13T18:21:02.791718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:56.914544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195461,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'20","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.315199+0000","last_change":"2024-09-13T17:50:34.583845+0000","last_active":"2024-09-13T18:21:13.315199+0000","last_peered":"2024-09-13T18:21:13.315199+0000","last_clean":"2024-09-13T18:21:13.315199+0000","last_became_active":"2024-09-13T17:50:34.583283+0000","last_became_peered":"2024-09-13T17:50:34.583283+0000","last_unstale":"2024-09-13T18:21:13.315199+0000","last_undegraded":"2024-09-13T18:21:13.315199+0000","last_fullsized":"2024-09-13T18:21:13.315199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:55.630285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'5883","reported_seq":5909,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.716917+0000","last_change":"2024-09-13T17:50:38.611987+0000","last_active":"2024-09-13T18:21:02.716917+0000","last_peered":"2024-09-13T18:21:02.716917+0000","last_clean":"2024-09-13T18:21:02.716917+0000","last_became_active":"2024-09-13T17:50:38.611636+0000","last_became_peered":"2024-09-13T17:50:38.611636+0000","last_unstale":"2024-09-13T18:21:02.716917+0000","last_undegraded":"2024-09-13T18:21:02.716917+0000","last_fullsized":"2024-09-13T18:21:02.716917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5883,"log_dups_size":0,"ondisk_log_size":5883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:41.111562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24816,"num_write_kb":545450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'25","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.300796+0000","last_change":"2024-09-13T17:50:34.588927+0000","last_active":"2024-09-13T18:21:13.300796+0000","last_peered":"2024-09-13T18:21:13.300796+0000","last_clean":"2024-09-13T18:21:13.300796+0000","last_became_active":"2024-09-13T17:50:34.588651+0000","last_became_peered":"2024-09-13T17:50:34.588651+0000","last_unstale":"2024-09-13T18:21:13.300796+0000","last_undegraded":"2024-09-13T18:21:13.300796+0000","last_fullsized":"2024-09-13T18:21:13.300796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:49.429670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'24","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310371+0000","last_change":"2024-09-13T17:50:34.583795+0000","last_active":"2024-09-13T18:21:13.310371+0000","last_peered":"2024-09-13T18:21:13.310371+0000","last_clean":"2024-09-13T18:21:13.310371+0000","last_became_active":"2024-09-13T17:50:34.583493+0000","last_became_peered":"2024-09-13T17:50:34.583493+0000","last_unstale":"2024-09-13T18:21:13.310371+0000","last_undegraded":"2024-09-13T18:21:13.310371+0000","last_fullsized":"2024-09-13T18:21:13.310371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:30.463932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'25","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.314410+0000","last_change":"2024-09-13T17:50:34.587624+0000","last_active":"2024-09-13T18:21:13.314410+0000","last_peered":"2024-09-13T18:21:13.314410+0000","last_clean":"2024-09-13T18:21:13.314410+0000","last_became_active":"2024-09-13T17:50:34.587119+0000","last_became_peered":"2024-09-13T17:50:34.587119+0000","last_unstale":"2024-09-13T18:21:13.314410+0000","last_undegraded":"2024-09-13T18:21:13.314410+0000","last_fullsized":"2024-09-13T18:21:13.314410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:16.783899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'17","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.308557+0000","last_change":"2024-09-13T17:50:34.581923+0000","last_active":"2024-09-13T18:21:13.308557+0000","last_peered":"2024-09-13T18:21:13.308557+0000","last_clean":"2024-09-13T18:21:13.308557+0000","last_became_active":"2024-09-13T17:50:34.581628+0000","last_became_peered":"2024-09-13T17:50:34.581628+0000","last_unstale":"2024-09-13T18:21:13.308557+0000","last_undegraded":"2024-09-13T18:21:13.308557+0000","last_fullsized":"2024-09-13T18:21:13.308557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:28:55.936864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'14","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312666+0000","last_change":"2024-09-13T17:50:34.595037+0000","last_active":"2024-09-13T18:21:13.312666+0000","last_peered":"2024-09-13T18:21:13.312666+0000","last_clean":"2024-09-13T18:21:13.312666+0000","last_became_active":"2024-09-13T17:50:34.594857+0000","last_became_peered":"2024-09-13T17:50:34.594857+0000","last_unstale":"2024-09-13T18:21:13.312666+0000","last_undegraded":"2024-09-13T18:21:13.312666+0000","last_fullsized":"2024-09-13T18:21:13.312666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:11.420413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'15","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.298148+0000","last_change":"2024-09-13T17:50:34.589651+0000","last_active":"2024-09-13T18:21:13.298148+0000","last_peered":"2024-09-13T18:21:13.298148+0000","last_clean":"2024-09-13T18:21:13.298148+0000","last_became_active":"2024-09-13T17:50:34.589354+0000","last_became_peered":"2024-09-13T17:50:34.589354+0000","last_unstale":"2024-09-13T18:21:13.298148+0000","last_undegraded":"2024-09-13T18:21:13.298148+0000","last_fullsized":"2024-09-13T18:21:13.298148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:31.294048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'24","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317654+0000","last_change":"2024-09-13T17:50:34.597949+0000","last_active":"2024-09-13T18:21:13.317654+0000","last_peered":"2024-09-13T18:21:13.317654+0000","last_clean":"2024-09-13T18:21:13.317654+0000","last_became_active":"2024-09-13T17:50:34.597621+0000","last_became_peered":"2024-09-13T17:50:34.597621+0000","last_unstale":"2024-09-13T18:21:13.317654+0000","last_undegraded":"2024-09-13T18:21:13.317654+0000","last_fullsized":"2024-09-13T18:21:13.317654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:23.133462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'15","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312343+0000","last_change":"2024-09-13T17:50:34.589893+0000","last_active":"2024-09-13T18:21:13.312343+0000","last_peered":"2024-09-13T18:21:13.312343+0000","last_clean":"2024-09-13T18:21:13.312343+0000","last_became_active":"2024-09-13T17:50:34.589626+0000","last_became_peered":"2024-09-13T17:50:34.589626+0000","last_unstale":"2024-09-13T18:21:13.312343+0000","last_undegraded":"2024-09-13T18:21:13.312343+0000","last_fullsized":"2024-09-13T18:21:13.312343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:41.321025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'16","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.308839+0000","last_change":"2024-09-13T17:50:34.590046+0000","last_active":"2024-09-13T18:21:13.308839+0000","last_peered":"2024-09-13T18:21:13.308839+0000","last_clean":"2024-09-13T18:21:13.308839+0000","last_became_active":"2024-09-13T17:50:34.589757+0000","last_became_peered":"2024-09-13T17:50:34.589757+0000","last_unstale":"2024-09-13T18:21:13.308839+0000","last_undegraded":"2024-09-13T18:21:13.308839+0000","last_fullsized":"2024-09-13T18:21:13.308839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:28.239550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'23","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312608+0000","last_change":"2024-09-13T17:50:34.595518+0000","last_active":"2024-09-13T18:21:13.312608+0000","last_peered":"2024-09-13T18:21:13.312608+0000","last_clean":"2024-09-13T18:21:13.312608+0000","last_became_active":"2024-09-13T17:50:34.595242+0000","last_became_peered":"2024-09-13T17:50:34.595242+0000","last_unstale":"2024-09-13T18:21:13.312608+0000","last_undegraded":"2024-09-13T18:21:13.312608+0000","last_fullsized":"2024-09-13T18:21:13.312608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:26.149384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'21","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.315503+0000","last_change":"2024-09-13T17:50:34.593183+0000","last_active":"2024-09-13T18:21:13.315503+0000","last_peered":"2024-09-13T18:21:13.315503+0000","last_clean":"2024-09-13T18:21:13.315503+0000","last_became_active":"2024-09-13T17:50:34.592893+0000","last_became_peered":"2024-09-13T17:50:34.592893+0000","last_unstale":"2024-09-13T18:21:13.315503+0000","last_undegraded":"2024-09-13T18:21:13.315503+0000","last_fullsized":"2024-09-13T18:21:13.315503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:19.459505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'29","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.311869+0000","last_change":"2024-09-13T17:50:34.594679+0000","last_active":"2024-09-13T18:21:13.311869+0000","last_peered":"2024-09-13T18:21:13.311869+0000","last_clean":"2024-09-13T18:21:13.311869+0000","last_became_active":"2024-09-13T17:50:34.593750+0000","last_became_peered":"2024-09-13T17:50:34.593750+0000","last_unstale":"2024-09-13T18:21:13.311869+0000","last_undegraded":"2024-09-13T18:21:13.311869+0000","last_fullsized":"2024-09-13T18:21:13.311869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:02.135577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'19","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.313398+0000","last_change":"2024-09-13T17:50:34.598736+0000","last_active":"2024-09-13T18:21:13.313398+0000","last_peered":"2024-09-13T18:21:13.313398+0000","last_clean":"2024-09-13T18:21:13.313398+0000","last_became_active":"2024-09-13T17:50:34.598433+0000","last_became_peered":"2024-09-13T17:50:34.598433+0000","last_unstale":"2024-09-13T18:21:13.313398+0000","last_undegraded":"2024-09-13T18:21:13.313398+0000","last_fullsized":"2024-09-13T18:21:13.313398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:08.975139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'22","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.301045+0000","last_change":"2024-09-13T17:50:34.591795+0000","last_active":"2024-09-13T18:21:13.301045+0000","last_peered":"2024-09-13T18:21:13.301045+0000","last_clean":"2024-09-13T18:21:13.301045+0000","last_became_active":"2024-09-13T17:50:34.591286+0000","last_became_peered":"2024-09-13T17:50:34.591286+0000","last_unstale":"2024-09-13T18:21:13.301045+0000","last_undegraded":"2024-09-13T18:21:13.301045+0000","last_fullsized":"2024-09-13T18:21:13.301045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:31.580247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'28","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.300800+0000","last_change":"2024-09-13T17:50:34.587774+0000","last_active":"2024-09-13T18:21:13.300800+0000","last_peered":"2024-09-13T18:21:13.300800+0000","last_clean":"2024-09-13T18:21:13.300800+0000","last_became_active":"2024-09-13T17:50:34.587506+0000","last_became_peered":"2024-09-13T17:50:34.587506+0000","last_unstale":"2024-09-13T18:21:13.300800+0000","last_undegraded":"2024-09-13T18:21:13.300800+0000","last_fullsized":"2024-09-13T18:21:13.300800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:46.275528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'22","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.318554+0000","last_change":"2024-09-13T17:50:34.596100+0000","last_active":"2024-09-13T18:21:13.318554+0000","last_peered":"2024-09-13T18:21:13.318554+0000","last_clean":"2024-09-13T18:21:13.318554+0000","last_became_active":"2024-09-13T17:50:34.595798+0000","last_became_peered":"2024-09-13T17:50:34.595798+0000","last_unstale":"2024-09-13T18:21:13.318554+0000","last_undegraded":"2024-09-13T18:21:13.318554+0000","last_fullsized":"2024-09-13T18:21:13.318554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:49.543845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'21","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306399+0000","last_change":"2024-09-13T17:50:34.584818+0000","last_active":"2024-09-13T18:21:13.306399+0000","last_peered":"2024-09-13T18:21:13.306399+0000","last_clean":"2024-09-13T18:21:13.306399+0000","last_became_active":"2024-09-13T17:50:34.584516+0000","last_became_peered":"2024-09-13T17:50:34.584516+0000","last_unstale":"2024-09-13T18:21:13.306399+0000","last_undegraded":"2024-09-13T18:21:13.306399+0000","last_fullsized":"2024-09-13T18:21:13.306399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:52.402024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'23","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.292739+0000","last_change":"2024-09-13T17:50:34.598886+0000","last_active":"2024-09-13T18:21:13.292739+0000","last_peered":"2024-09-13T18:21:13.292739+0000","last_clean":"2024-09-13T18:21:13.292739+0000","last_became_active":"2024-09-13T17:50:34.598648+0000","last_became_peered":"2024-09-13T17:50:34.598648+0000","last_unstale":"2024-09-13T18:21:13.292739+0000","last_undegraded":"2024-09-13T18:21:13.292739+0000","last_fullsized":"2024-09-13T18:21:13.292739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:00:43.063223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'24","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.302458+0000","last_change":"2024-09-13T17:50:34.582649+0000","last_active":"2024-09-13T18:21:13.302458+0000","last_peered":"2024-09-13T18:21:13.302458+0000","last_clean":"2024-09-13T18:21:13.302458+0000","last_became_active":"2024-09-13T17:50:34.581699+0000","last_became_peered":"2024-09-13T17:50:34.581699+0000","last_unstale":"2024-09-13T18:21:13.302458+0000","last_undegraded":"2024-09-13T18:21:13.302458+0000","last_fullsized":"2024-09-13T18:21:13.302458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:18.959303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'23","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306620+0000","last_change":"2024-09-13T17:50:34.587484+0000","last_active":"2024-09-13T18:21:13.306620+0000","last_peered":"2024-09-13T18:21:13.306620+0000","last_clean":"2024-09-13T18:21:13.306620+0000","last_became_active":"2024-09-13T17:50:34.587217+0000","last_became_peered":"2024-09-13T17:50:34.587217+0000","last_unstale":"2024-09-13T18:21:13.306620+0000","last_undegraded":"2024-09-13T18:21:13.306620+0000","last_fullsized":"2024-09-13T18:21:13.306620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:24:19.172607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'18","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.315670+0000","last_change":"2024-09-13T17:50:34.592871+0000","last_active":"2024-09-13T18:21:13.315670+0000","last_peered":"2024-09-13T18:21:13.315670+0000","last_clean":"2024-09-13T18:21:13.315670+0000","last_became_active":"2024-09-13T17:50:34.588513+0000","last_became_peered":"2024-09-13T17:50:34.588513+0000","last_unstale":"2024-09-13T18:21:13.315670+0000","last_undegraded":"2024-09-13T18:21:13.315670+0000","last_fullsized":"2024-09-13T18:21:13.315670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:33.131420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'19","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.319025+0000","last_change":"2024-09-13T17:50:34.585713+0000","last_active":"2024-09-13T18:21:13.319025+0000","last_peered":"2024-09-13T18:21:13.319025+0000","last_clean":"2024-09-13T18:21:13.319025+0000","last_became_active":"2024-09-13T17:50:34.585438+0000","last_became_peered":"2024-09-13T17:50:34.585438+0000","last_unstale":"2024-09-13T18:21:13.319025+0000","last_undegraded":"2024-09-13T18:21:13.319025+0000","last_fullsized":"2024-09-13T18:21:13.319025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:56:49.905393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'30","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312596+0000","last_change":"2024-09-13T17:50:34.589628+0000","last_active":"2024-09-13T18:21:13.312596+0000","last_peered":"2024-09-13T18:21:13.312596+0000","last_clean":"2024-09-13T18:21:13.312596+0000","last_became_active":"2024-09-13T17:50:34.587665+0000","last_became_peered":"2024-09-13T17:50:34.587665+0000","last_unstale":"2024-09-13T18:21:13.312596+0000","last_undegraded":"2024-09-13T18:21:13.312596+0000","last_fullsized":"2024-09-13T18:21:13.312596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:15.518264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'27","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.321326+0000","last_change":"2024-09-13T17:50:34.591776+0000","last_active":"2024-09-13T18:21:13.321326+0000","last_peered":"2024-09-13T18:21:13.321326+0000","last_clean":"2024-09-13T18:21:13.321326+0000","last_became_active":"2024-09-13T17:50:34.591496+0000","last_became_peered":"2024-09-13T17:50:34.591496+0000","last_unstale":"2024-09-13T18:21:13.321326+0000","last_undegraded":"2024-09-13T18:21:13.321326+0000","last_fullsized":"2024-09-13T18:21:13.321326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:48.651999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'15","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310654+0000","last_change":"2024-09-13T17:51:16.282372+0000","last_active":"2024-09-13T18:21:13.310654+0000","last_peered":"2024-09-13T18:21:13.310654+0000","last_clean":"2024-09-13T18:21:13.310654+0000","last_became_active":"2024-09-13T17:51:16.282065+0000","last_became_peered":"2024-09-13T17:51:16.282065+0000","last_unstale":"2024-09-13T18:21:13.310654+0000","last_undegraded":"2024-09-13T18:21:13.310654+0000","last_fullsized":"2024-09-13T18:21:13.310654+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:49.291224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"27'14","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310971+0000","last_change":"2024-09-13T17:50:34.585531+0000","last_active":"2024-09-13T18:21:13.310971+0000","last_peered":"2024-09-13T18:21:13.310971+0000","last_clean":"2024-09-13T18:21:13.310971+0000","last_became_active":"2024-09-13T17:50:34.585226+0000","last_became_peered":"2024-09-13T17:50:34.585226+0000","last_unstale":"2024-09-13T18:21:13.310971+0000","last_undegraded":"2024-09-13T18:21:13.310971+0000","last_fullsized":"2024-09-13T18:21:13.310971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:17.105028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'17","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.314101+0000","last_change":"2024-09-13T17:50:34.586750+0000","last_active":"2024-09-13T18:21:13.314101+0000","last_peered":"2024-09-13T18:21:13.314101+0000","last_clean":"2024-09-13T18:21:13.314101+0000","last_became_active":"2024-09-13T17:50:34.584564+0000","last_became_peered":"2024-09-13T17:50:34.584564+0000","last_unstale":"2024-09-13T18:21:13.314101+0000","last_undegraded":"2024-09-13T18:21:13.314101+0000","last_fullsized":"2024-09-13T18:21:13.314101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:54.033257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'18","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.307217+0000","last_change":"2024-09-13T17:50:34.585938+0000","last_active":"2024-09-13T18:21:13.307217+0000","last_peered":"2024-09-13T18:21:13.307217+0000","last_clean":"2024-09-13T18:21:13.307217+0000","last_became_active":"2024-09-13T17:50:34.585584+0000","last_became_peered":"2024-09-13T17:50:34.585584+0000","last_unstale":"2024-09-13T18:21:13.307217+0000","last_undegraded":"2024-09-13T18:21:13.307217+0000","last_fullsized":"2024-09-13T18:21:13.307217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:32.623468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'15","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310571+0000","last_change":"2024-09-13T17:50:34.588762+0000","last_active":"2024-09-13T18:21:13.310571+0000","last_peered":"2024-09-13T18:21:13.310571+0000","last_clean":"2024-09-13T18:21:13.310571+0000","last_became_active":"2024-09-13T17:50:34.588489+0000","last_became_peered":"2024-09-13T17:50:34.588489+0000","last_unstale":"2024-09-13T18:21:13.310571+0000","last_undegraded":"2024-09-13T18:21:13.310571+0000","last_fullsized":"2024-09-13T18:21:13.310571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:38.266297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'18","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.302008+0000","last_change":"2024-09-13T17:50:34.583889+0000","last_active":"2024-09-13T18:21:13.302008+0000","last_peered":"2024-09-13T18:21:13.302008+0000","last_clean":"2024-09-13T18:21:13.302008+0000","last_became_active":"2024-09-13T17:50:34.583551+0000","last_became_peered":"2024-09-13T17:50:34.583551+0000","last_unstale":"2024-09-13T18:21:13.302008+0000","last_undegraded":"2024-09-13T18:21:13.302008+0000","last_fullsized":"2024-09-13T18:21:13.302008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:19.888626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'10","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.284491+0000","last_change":"2024-09-13T17:50:34.585617+0000","last_active":"2024-09-13T18:21:13.284491+0000","last_peered":"2024-09-13T18:21:13.284491+0000","last_clean":"2024-09-13T18:21:13.284491+0000","last_became_active":"2024-09-13T17:50:34.585340+0000","last_became_peered":"2024-09-13T17:50:34.585340+0000","last_unstale":"2024-09-13T18:21:13.284491+0000","last_undegraded":"2024-09-13T18:21:13.284491+0000","last_fullsized":"2024-09-13T18:21:13.284491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:18.331045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'17","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317071+0000","last_change":"2024-09-13T17:50:34.592365+0000","last_active":"2024-09-13T18:21:13.317071+0000","last_peered":"2024-09-13T18:21:13.317071+0000","last_clean":"2024-09-13T18:21:13.317071+0000","last_became_active":"2024-09-13T17:50:34.592046+0000","last_became_peered":"2024-09-13T17:50:34.592046+0000","last_unstale":"2024-09-13T18:21:13.317071+0000","last_undegraded":"2024-09-13T18:21:13.317071+0000","last_fullsized":"2024-09-13T18:21:13.317071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:54.283683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":751482,"num_write_kb":16106434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178546,"ondisk_log_size":178546,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1271,"ondisk_log_size":1271,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":15941984,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":121,"num_read_kb":179,"num_write":3371,"num_write_kb":16874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32030720,"data_stored":31883968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3124,"ondisk_log_size":3124,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509812,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1768076,"kb_used_data":8580,"kb_used_omap":1,"kb_used_meta":1759486,"kb_avail":92603764,"statfs":{"total":96636764160,"available":94826254336,"internally_reserved":0,"allocated":8785920,"data_stored":8600393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":1801714120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":2.246,"5min":5.9219999999999997,"15min":2.669},"min":{"1min":0.19,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":8.5329999999999995,"5min":41.494999999999997,"15min":50.511000000000003},"last":0.63},{"interface":"front","average":{"1min":2.6070000000000002,"5min":3.4350000000000001,"15min":1.6080000000000001},"min":{"1min":0.23100000000000001,"5min":0.193,"15min":0.193},"max":{"1min":18.760000000000002,"5min":18.760000000000002,"15min":18.760000000000002},"last":0.56999999999999995}]},{"osd":1,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":3.4969999999999999,"15min":1.8009999999999999},"min":{"1min":0.34000000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":6.25,"5min":19.626000000000001,"15min":19.626000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":3.4430000000000001,"5min":4.101,"15min":2.0379999999999998},"min":{"1min":0.29599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":22.126999999999999,"5min":22.126999999999999,"15min":22.126999999999999},"last":0.71999999999999997}]},{"osd":2,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.907,"15min":1.296},"min":{"1min":0.29399999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":7.4610000000000003,"5min":22.010000000000002,"15min":22.010000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.49,"5min":2.1800000000000002,"15min":1.4830000000000001},"min":{"1min":0.38400000000000001,"5min":0.309,"15min":0.309},"max":{"1min":7.4850000000000003,"5min":24.170999999999999,"15min":24.170999999999999},"last":0.74399999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":2.0409999999999999,"15min":1.466},"min":{"1min":0.33200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":5.593,"5min":24.359000000000002,"15min":24.359000000000002},"last":0.87},{"interface":"front","average":{"1min":1.8109999999999999,"5min":2.3410000000000002,"15min":1.4570000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":10.132999999999999,"5min":15.539999999999999,"15min":15.539999999999999},"last":1.0069999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":2.5840000000000001,"15min":1.6160000000000001},"min":{"1min":0.35699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":8.5739999999999998,"5min":49.838000000000001,"15min":49.838000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.96999999999999997,"5min":2.5369999999999999,"15min":1.605},"min":{"1min":0.43099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":4.3810000000000002,"5min":41.591999999999999,"15min":41.591999999999999},"last":0.81100000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":2.23,"5min":2.5579999999999998,"15min":1.7270000000000001},"min":{"1min":0.46100000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":8.6029999999999998,"5min":24.457000000000001,"15min":24.457000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.677,"5min":2.2919999999999998,"15min":1.6200000000000001},"min":{"1min":0.42399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":7.4290000000000003,"5min":24.387,"15min":24.387},"last":1.1319999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.869,"15min":1.3740000000000001},"min":{"1min":0.505,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":4.2750000000000004,"5min":14.138999999999999,"15min":14.138999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.155,"5min":1.97,"15min":1.4219999999999999},"min":{"1min":0.44900000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":4.4320000000000004,"5min":14.013,"15min":14.013},"last":0.99299999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509814,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1055244,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":1054846,"kb_avail":93316596,"statfs":{"total":96636764160,"available":95556194304,"internally_reserved":0,"allocated":397312,"data_stored":210684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1080162758},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":2.0819999999999999,"5min":2.0739999999999998,"15min":1.0980000000000001},"min":{"1min":0.17499999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":9.9280000000000008,"5min":17.971,"15min":17.971},"last":0.45900000000000002},{"interface":"front","average":{"1min":2.1469999999999998,"5min":2.1240000000000001,"15min":1.177},"min":{"1min":0.247,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":9.9700000000000006,"5min":18.001999999999999,"15min":18.001999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":2.694,"15min":1.677},"min":{"1min":0.26100000000000001,"5min":0.217,"15min":0.217},"max":{"1min":3.871,"5min":24.599,"15min":56.939999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.4710000000000001,"5min":2.9350000000000001,"15min":1.7649999999999999},"min":{"1min":0.23200000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":9.0169999999999995,"5min":24.75,"15min":57.006999999999998},"last":0.61899999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":3.5219999999999998,"5min":3.0790000000000002,"15min":1.5980000000000001},"min":{"1min":0.30499999999999999,"5min":0.218,"15min":0.218},"max":{"1min":16.111999999999998,"5min":16.707000000000001,"15min":16.707000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":3.0169999999999999,"5min":2.6349999999999998,"15min":1.5569999999999999},"min":{"1min":0.309,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":12.449999999999999,"5min":16.640999999999998,"15min":16.640999999999998},"last":0.69399999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.52,"15min":1.238},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":8.2260000000000009,"5min":8.2829999999999995,"15min":14.177},"last":1.073},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.284,"15min":1.159},"min":{"1min":0.33900000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":8.1980000000000004,"5min":8.1980000000000004,"15min":14.396000000000001},"last":1.0589999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.77600000000000002,"15min":1.0009999999999999},"min":{"1min":0.379,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.2250000000000001,"5min":4.1020000000000003,"15min":14.153},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.181,"5min":1.224,"15min":1.137},"min":{"1min":0.32800000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":7.476,"5min":7.476,"15min":14.375999999999999},"last":0.82099999999999995}]},{"osd":6,"last update":"Fri Sep 13 18:20:30 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.1519999999999999,"15min":1.1020000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":6.8730000000000002,"5min":8.3010000000000002,"15min":8.3010000000000002},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.125,"5min":1.196,"15min":1.2150000000000001},"min":{"1min":0.46700000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":6.9500000000000002,"5min":8.3320000000000007,"15min":14.074},"last":0.996}]},{"osd":7,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.163,"15min":1.0780000000000001},"min":{"1min":0.40400000000000003,"5min":0.37,"15min":0.37},"max":{"1min":4.1079999999999997,"5min":8.0410000000000004,"15min":8.0410000000000004},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.238,"15min":1.1930000000000001},"min":{"1min":0.37,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.9900000000000002,"5min":6.5300000000000002,"15min":14.307},"last":1.111}]}]},{"osd":1,"up_from":15,"seq":64424509813,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1738228,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":1737918,"kb_avail":92633612,"statfs":{"total":96636764160,"available":94856818688,"internally_reserved":0,"allocated":307200,"data_stored":116868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1779628486},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.9119999999999999,"5min":3.7890000000000001,"15min":1.891},"min":{"1min":0.23999999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":9.9879999999999995,"5min":21.949999999999999,"15min":21.949999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.4610000000000001,"5min":2.863,"15min":1.5860000000000001},"min":{"1min":0.219,"5min":0.193,"15min":0.193},"max":{"1min":7.8730000000000002,"5min":23.338999999999999,"15min":23.338999999999999},"last":0.58999999999999997}]},{"osd":2,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":2.5699999999999998,"15min":1.4950000000000001},"min":{"1min":0.311,"5min":0.22,"15min":0.22},"max":{"1min":6.6399999999999997,"5min":14.869999999999999,"15min":14.869999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.4390000000000001,"5min":2.383,"15min":1.4059999999999999},"min":{"1min":0.23599999999999999,"5min":0.223,"15min":0.223},"max":{"1min":6.5300000000000002,"5min":15.058,"15min":15.058},"last":0.83499999999999996}]},{"osd":3,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":2.323,"5min":6.8940000000000001,"15min":3.1539999999999999},"min":{"1min":0.314,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":8.2390000000000008,"5min":41.369999999999997,"15min":41.369999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":2.8769999999999998,"5min":6.2549999999999999,"15min":2.8900000000000001},"min":{"1min":0.27900000000000003,"5min":0.216,"15min":0.216},"max":{"1min":8.8360000000000003,"5min":36.439,"15min":40.325000000000003},"last":0.63300000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.978,"5min":3.9390000000000001,"15min":2.1070000000000002},"min":{"1min":0.30399999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":8.5980000000000008,"5min":103.44499999999999,"15min":103.44499999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":2.1850000000000001,"5min":3.3199999999999998,"15min":1.889},"min":{"1min":0.312,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":8.2789999999999999,"5min":40.953000000000003,"15min":40.953000000000003},"last":0.95899999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:20:20 2024","interfaces":[{"interface":"back","average":{"1min":1.9750000000000001,"5min":2.661,"15min":1.726},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":9.8970000000000002,"5min":18.954000000000001,"15min":18.954000000000001},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.431,"5min":2.0840000000000001,"15min":1.516},"min":{"1min":0.31,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":7.383,"5min":15.413,"15min":15.413},"last":1.0429999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":2.4089999999999998,"15min":1.569},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":7.4199999999999999,"5min":15.462999999999999,"15min":15.462999999999999},"last":1.244},{"interface":"front","average":{"1min":1.6719999999999999,"5min":2.5190000000000001,"15min":1.663},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":7.4429999999999996,"5min":15.49,"15min":15.49},"last":1.3009999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.764,"15min":1.375},"min":{"1min":0.33600000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":6.6159999999999997,"5min":15.093999999999999,"15min":15.093999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.494,"5min":1.8380000000000001,"15min":1.4590000000000001},"min":{"1min":0.41899999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":6.6920000000000002,"5min":15.119,"15min":15.119},"last":1.27}]}]},{"osd":0,"up_from":15,"seq":64424509815,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1247072,"kb_used_data":4056,"kb_used_omap":1,"kb_used_meta":1243006,"kb_avail":93124768,"statfs":{"total":96636764160,"available":95359762432,"internally_reserved":0,"allocated":4153344,"data_stored":3967313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1272838598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":2.8570000000000002,"5min":3.2130000000000001,"15min":1.7050000000000001},"min":{"1min":0.182,"5min":0.182,"15min":0.182},"max":{"1min":17.693000000000001,"5min":23.004000000000001,"15min":24.202000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":2.9489999999999998,"5min":3.2690000000000001,"15min":1.8380000000000001},"min":{"1min":0.32100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":17.736000000000001,"5min":23.039999999999999,"15min":24.260999999999999},"last":0.876}]},{"osd":2,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.897,"15min":1.1060000000000001},"min":{"1min":0.20399999999999999,"5min":0.186,"15min":0.186},"max":{"1min":6.4320000000000004,"5min":17.859000000000002,"15min":17.859000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.77,"5min":2.0920000000000001,"15min":1.3180000000000001},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":8.6199999999999992,"5min":19.678999999999998,"15min":19.678999999999998},"last":0.78900000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":2.2330000000000001,"5min":4.0970000000000004,"15min":1.9630000000000001},"min":{"1min":0.23699999999999999,"5min":0.219,"15min":0.219},"max":{"1min":11.378,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":3.6720000000000002,"5min":3.27,"15min":1.8129999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":13.736000000000001,"5min":15.856999999999999,"15min":15.856999999999999},"last":0.92400000000000004}]},{"osd":4,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.661,"15min":1.258},"min":{"1min":0.34699999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":8.2829999999999995,"5min":8.7010000000000005,"15min":8.7010000000000005},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.1719999999999999,"5min":2.657,"15min":1.548},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":6.3090000000000002,"5min":16.291,"15min":16.291},"last":1.044}]},{"osd":5,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":2.0960000000000001,"5min":1.974,"15min":1.3260000000000001},"min":{"1min":0.312,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":8.4800000000000004,"5min":15.18,"15min":15.18},"last":1.079},{"interface":"front","average":{"1min":1.7689999999999999,"5min":1.9850000000000001,"15min":1.4119999999999999},"min":{"1min":0.34699999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":8.5559999999999992,"5min":17.852,"15min":17.852},"last":1.135}]},{"osd":6,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":3.1669999999999998,"5min":2.3860000000000001,"15min":1.5600000000000001},"min":{"1min":0.39200000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":15.467000000000001,"5min":15.467000000000001,"15min":15.467000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.7290000000000001,"5min":1.823,"15min":1.4139999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":10.359,"5min":17.908000000000001,"15min":17.908000000000001},"last":1.278}]},{"osd":7,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":1.909,"5min":1.887,"15min":1.381},"min":{"1min":0.40100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":8.2550000000000008,"5min":18.986999999999998,"15min":18.986999999999998},"last":1.242},{"interface":"front","average":{"1min":1.952,"5min":1.9710000000000001,"15min":1.47},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.373},"max":{"1min":8.3279999999999994,"5min":19.021000000000001,"15min":19.021000000000001},"last":1.1699999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542517,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1207524,"kb_used_data":12252,"kb_used_omap":1,"kb_used_meta":1195262,"kb_avail":93164316,"statfs":{"total":96636764160,"available":95400259584,"internally_reserved":0,"allocated":12546048,"data_stored":12357112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":1223948747},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":2.0619999999999998,"15min":1.246},"min":{"1min":0.245,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":5.367,"5min":35.195,"15min":35.195},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.2529999999999999,"5min":2.206,"15min":1.391},"min":{"1min":0.34000000000000002,"5min":0.315,"15min":0.315},"max":{"1min":5.4269999999999996,"5min":35.253999999999998,"15min":35.253999999999998},"last":0.92900000000000005}]},{"osd":1,"last update":"Fri Sep 13 18:20:27 2024","interfaces":[{"interface":"back","average":{"1min":2.4940000000000002,"5min":2.6930000000000001,"15min":1.4730000000000001},"min":{"1min":0.27100000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":13.058999999999999,"5min":19.585000000000001,"15min":19.585000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":2.4060000000000001,"5min":2.6680000000000001,"15min":1.635},"min":{"1min":0.33400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":17.398,"5min":19.617000000000001,"15min":19.617000000000001},"last":0.74199999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.6100000000000001,"15min":1.143},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":7.1299999999999999,"5min":15.789999999999999,"15min":15.789999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.5569999999999999,"15min":1.161},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.9379999999999997,"5min":12.965,"15min":12.965},"last":1.1950000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":3.427,"15min":1.8700000000000001},"min":{"1min":0.36299999999999999,"5min":0.316,"15min":0.316},"max":{"1min":5.4370000000000003,"5min":40.948999999999998,"15min":40.948999999999998},"last":0.98199999999999998},{"interface":"front","average":{"1min":2.0739999999999998,"5min":3.198,"15min":1.6870000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":20.024999999999999,"5min":25.234000000000002,"15min":25.234000000000002},"last":1.115}]},{"osd":4,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1739999999999999,"15min":0.99399999999999999},"min":{"1min":0.31,"5min":0.252,"15min":0.252},"max":{"1min":4.5220000000000002,"5min":8.8670000000000009,"15min":8.8670000000000009},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.321,"15min":1.181},"min":{"1min":0.38500000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":4.383,"5min":10.279999999999999,"15min":10.279999999999999},"last":0.79900000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.123,"15min":0.98499999999999999},"min":{"1min":0.34300000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":4.4790000000000001,"5min":8.1699999999999999,"15min":8.1699999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.98999999999999999,"15min":1.0289999999999999},"min":{"1min":0.33100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":3.0489999999999999,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":1.004}]},{"osd":6,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.4750000000000001,"15min":1.2190000000000001},"min":{"1min":0.34599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":6.7960000000000003,"5min":8.2219999999999995,"15min":8.2219999999999995},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.508,"15min":1.2549999999999999},"min":{"1min":0.434,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":7.048,"5min":8.2469999999999999,"15min":8.2469999999999999},"last":1.27}]}]},{"osd":6,"up_from":14,"seq":60129542517,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1419436,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":1415038,"kb_avail":92952404,"statfs":{"total":96636764160,"available":95183261696,"internally_reserved":0,"allocated":4493312,"data_stored":4311530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1448999367},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":2.0329999999999999,"15min":1.21},"min":{"1min":0.23699999999999999,"5min":0.214,"15min":0.214},"max":{"1min":3.9470000000000001,"5min":12.808999999999999,"15min":12.808999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.091,"5min":1.7030000000000001,"15min":1.161},"min":{"1min":0.31,"5min":0.218,"15min":0.218},"max":{"1min":5.7670000000000003,"5min":12.308999999999999,"15min":12.308999999999999},"last":0.80500000000000005}]},{"osd":1,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":2.0470000000000002,"5min":2.27,"15min":1.371},"min":{"1min":0.34799999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":5.8380000000000001,"5min":14.199999999999999,"15min":14.199999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.621,"5min":2.339,"15min":1.413},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.7990000000000004,"5min":14.260999999999999,"15min":14.260999999999999},"last":1.109}]},{"osd":2,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.6839999999999999,"5min":1.821,"15min":1.2110000000000001},"min":{"1min":0.29999999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":6.0359999999999996,"5min":10.792999999999999,"15min":10.792999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.742,"5min":1.8759999999999999,"15min":1.371},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":6.0750000000000002,"5min":10.826000000000001,"15min":10.826000000000001},"last":1.252}]},{"osd":3,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":3.379,"5min":5.0670000000000002,"15min":2.4660000000000002},"min":{"1min":0.35799999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":10.324999999999999,"5min":84.867999999999995,"15min":84.867999999999995},"last":0.93600000000000005},{"interface":"front","average":{"1min":2.5449999999999999,"5min":3.5779999999999998,"15min":1.8979999999999999},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":10.231,"5min":84.343999999999994,"15min":84.343999999999994},"last":1.01}]},{"osd":4,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":2.0739999999999998,"5min":1.6990000000000001,"15min":1.2070000000000001},"min":{"1min":0.39200000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":9.5649999999999995,"5min":11.044,"15min":11.044},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.294,"15min":1.113},"min":{"1min":0.38400000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":9.4719999999999995,"5min":9.4719999999999995,"15min":9.4719999999999995},"last":0.97599999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2689999999999999,"15min":1.1559999999999999},"min":{"1min":0.376,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.0939999999999999,"5min":8.7880000000000003,"15min":8.7880000000000003},"last":1.151},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.149,"15min":1.1000000000000001},"min":{"1min":0.41499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.1709999999999998,"5min":7.569,"15min":7.569},"last":1.222}]},{"osd":7,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.1739999999999999,"15min":1.181},"min":{"1min":0.434,"5min":0.32900000000000001,"15min":0.29999999999999999},"max":{"1min":2.8029999999999999,"5min":7.1820000000000004,"15min":7.1820000000000004},"last":1.335},{"interface":"front","average":{"1min":1.133,"5min":1.294,"15min":1.256},"min":{"1min":0.53700000000000003,"5min":0.34499999999999997,"15min":0.33500000000000002},"max":{"1min":2.871,"5min":7.2039999999999997,"15min":7.2039999999999997},"last":1.379}]}]},{"osd":5,"up_from":14,"seq":60129542519,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":978544,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":978238,"kb_avail":93393296,"statfs":{"total":96636764160,"available":95634735104,"internally_reserved":0,"allocated":303104,"data_stored":117302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1001716166},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.889,"15min":1.161},"min":{"1min":0.27500000000000002,"5min":0.218,"15min":0.218},"max":{"1min":7.3650000000000002,"5min":10.148999999999999,"15min":10.148999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.9550000000000001,"15min":1.3220000000000001},"min":{"1min":0.34200000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":7.4029999999999996,"5min":10.185,"15min":10.185},"last":0.96999999999999997}]},{"osd":1,"last update":"Fri Sep 13 18:20:27 2024","interfaces":[{"interface":"back","average":{"1min":1.879,"5min":1.897,"15min":1.1699999999999999},"min":{"1min":0.29799999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":10.08,"5min":10.436999999999999,"15min":10.436999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":2.0099999999999998,"5min":2.3460000000000001,"15min":1.385},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":18.584,"5min":19.388000000000002,"15min":19.388000000000002},"last":1.2609999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.875,"5min":1.5329999999999999,"15min":1.0980000000000001},"min":{"1min":0.33900000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":10.183,"5min":12.667,"15min":12.667},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.032,"5min":1.6439999999999999,"15min":1.198},"min":{"1min":0.377,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":5.7469999999999999,"5min":34.905999999999999,"15min":34.905999999999999},"last":1.119}]},{"osd":3,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":3.2999999999999998,"5min":4.9039999999999999,"15min":2.286},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":19.831,"5min":28.295000000000002,"15min":28.295000000000002},"last":0.81399999999999995},{"interface":"front","average":{"1min":3.9260000000000002,"5min":3.1459999999999999,"15min":1.7210000000000001},"min":{"1min":0.39700000000000002,"5min":0.318,"15min":0.318},"max":{"1min":19.731000000000002,"5min":19.731000000000002,"15min":19.731000000000002},"last":1.0149999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.744,"5min":1.0760000000000001,"15min":0.96199999999999997},"min":{"1min":0.308,"5min":0.255,"15min":0.255},"max":{"1min":10.023999999999999,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":2.3650000000000002,"5min":1.571,"15min":1.22},"min":{"1min":0.39400000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":10.210000000000001,"5min":10.210000000000001,"15min":10.210000000000001},"last":1.075}]},{"osd":6,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.9530000000000001,"5min":1.373,"15min":1.131},"min":{"1min":0.437,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":6.6849999999999996,"5min":6.6849999999999996,"15min":6.6849999999999996},"last":1.179},{"interface":"front","average":{"1min":2.012,"5min":1.3200000000000001,"15min":1.2},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":10.151,"5min":10.151,"15min":10.151},"last":1.2130000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":1.2330000000000001,"15min":1.143},"min":{"1min":0.42599999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":6.2910000000000004,"5min":7.9329999999999998,"15min":7.9329999999999998},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.71,"5min":1.107,"15min":1.1519999999999999},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":5.976,"5min":5.976,"15min":5.976},"last":1.3620000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542517,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1301036,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":1296638,"kb_avail":93070804,"statfs":{"total":96636764160,"available":95304503296,"internally_reserved":0,"allocated":4493312,"data_stored":4310894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1327757766},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":2.1499999999999999,"15min":1.2829999999999999},"min":{"1min":0.26200000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":6.984,"5min":28.439,"15min":28.439},"last":0.92200000000000004},{"interface":"front","average":{"1min":2.1949999999999998,"5min":2.4079999999999999,"15min":1.4430000000000001},"min":{"1min":0.309,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":16.943000000000001,"5min":28.422000000000001,"15min":28.422000000000001},"last":0.748}]},{"osd":1,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":3.7789999999999999,"5min":5.5380000000000003,"15min":2.577},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":12.124000000000001,"5min":38.895000000000003,"15min":38.895000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":3.2069999999999999,"5min":4.0599999999999996,"15min":1.976},"min":{"1min":0.30199999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":12.026999999999999,"5min":32.670000000000002,"15min":32.670000000000002},"last":0.52600000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.72,"15min":1.1599999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":4.085,"5min":25.167999999999999,"15min":25.167999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":2.0150000000000001,"15min":1.321},"min":{"1min":0.373,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":4.1150000000000002,"5min":25.218,"15min":25.218},"last":1.1439999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":2.6419999999999999,"5min":3.0129999999999999,"15min":1.7250000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":11.441000000000001,"5min":43.945999999999998,"15min":43.945999999999998},"last":1.008},{"interface":"front","average":{"1min":3.2610000000000001,"5min":3.9329999999999998,"15min":2.052},"min":{"1min":0.39200000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":29.640000000000001,"5min":29.640000000000001,"15min":29.640000000000001},"last":0.81399999999999995}]},{"osd":5,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.2050000000000001,"15min":1.069},"min":{"1min":0.28199999999999997,"5min":0.255,"15min":0.255},"max":{"1min":12.282999999999999,"5min":12.282999999999999,"15min":12.282999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.3160000000000001,"15min":1.1739999999999999},"min":{"1min":0.36699999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":16.995999999999999,"5min":16.995999999999999,"15min":16.995999999999999},"last":0.98099999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":2.0430000000000001,"5min":2.0979999999999999,"15min":1.387},"min":{"1min":0.30499999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":12.42,"5min":23.010999999999999,"15min":23.010999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.504,"5min":1.23,"15min":1.181},"min":{"1min":0.40000000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":6.6040000000000001,"5min":7.9489999999999998,"15min":7.9489999999999998},"last":1.1919999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.448,"15min":1.27},"min":{"1min":0.30499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.0179999999999998,"5min":20.417000000000002,"15min":20.417000000000002},"last":1.236},{"interface":"front","average":{"1min":1.359,"5min":1.4019999999999999,"15min":1.284},"min":{"1min":0.41199999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":12.308999999999999,"5min":12.308999999999999,"15min":12.308999999999999},"last":1.0629999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3276800,"data_stored":3261021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":94760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8503296,"data_stored":8484469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4194970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4195606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":11669504,"data_stored":11650820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:21:20.435 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T18:21:20.436 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:21:20.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f9791365700 1 Processor -- start 2024-09-13T18:21:20.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f9791365700 1 -- start start 2024-09-13T18:21:20.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f9791365700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f978c0b2d00 0x7f978c15acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:20.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f9791365700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f978c070120 0x7f978c15b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:20.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f9791365700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f978c15a840 con 0x7f978c1512c0 2024-09-13T18:21:20.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f9791365700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f978c1516a0 con 0x7f978c070120 2024-09-13T18:21:20.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f9791365700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f978c1580a0 con 0x7f978c0b2d00 2024-09-13T18:21:20.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f9790b64700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f978c070120 0x7f978c15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:20.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f978bfff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f978c1512c0 0x7f978c0c78f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53530/0 (socket says 172.21.15.40:53530) 2024-09-13T18:21:20.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f978bfff700 1 -- 172.21.15.40:0/1724159517 learned_addr learned my addr 172.21.15.40:0/1724159517 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:20.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f978b7fe700 1 --2- 172.21.15.40:0/1724159517 >> v2:172.21.15.43:3300/0 conn(0x7f978c0b2d00 0x7f978c15acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:20.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f978affd700 1 -- 172.21.15.40:0/1724159517 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3117602666 0 0) 0x7f978c15a840 con 0x7f978c1512c0 2024-09-13T18:21:20.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f978affd700 1 -- 172.21.15.40:0/1724159517 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97740035d0 con 0x7f978c1512c0 2024-09-13T18:21:20.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f9790b64700 1 -- 172.21.15.40:0/1724159517 >> v2:172.21.15.43:3300/0 conn(0x7f978c0b2d00 msgr2=0x7f978c15acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:20.547 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f9790b64700 1 --2- 172.21.15.40:0/1724159517 >> v2:172.21.15.43:3300/0 conn(0x7f978c0b2d00 0x7f978c15acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f9790b64700 1 -- 172.21.15.40:0/1724159517 >> v1:172.21.15.40:6789/0 conn(0x7f978c1512c0 legacy=0x7f978c0c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:20.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f9790b64700 1 -- 172.21.15.40:0/1724159517 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978c157520 con 0x7f978c070120 2024-09-13T18:21:20.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f9790b64700 1 --2- 172.21.15.40:0/1724159517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f978c070120 0x7f978c15b210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f978c074420 tx=0x7f978400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6eaeaf53626a659 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:20.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f978affd700 1 -- 172.21.15.40:0/1724159517 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f978400e070 con 0x7f978c070120 2024-09-13T18:21:20.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f978affd700 1 -- 172.21.15.40:0/1724159517 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978400bb20 con 0x7f978c070120 2024-09-13T18:21:20.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.537+0000 7f978affd700 1 -- 172.21.15.40:0/1724159517 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9784012430 con 0x7f978c070120 2024-09-13T18:21:20.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f9791365700 1 -- 172.21.15.40:0/1724159517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f978c070120 msgr2=0x7f978c15b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:20.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f9791365700 1 --2- 172.21.15.40:0/1724159517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f978c070120 0x7f978c15b210 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f978c074420 tx=0x7f978400b330 comp rx=0 tx=0).stop 2024-09-13T18:21:20.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f9791365700 1 -- 172.21.15.40:0/1724159517 shutdown_connections 2024-09-13T18:21:20.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f9791365700 1 --2- 172.21.15.40:0/1724159517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f978c070120 0x7f978c15b210 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f9791365700 1 --2- 172.21.15.40:0/1724159517 >> v2:172.21.15.43:3300/0 conn(0x7f978c0b2d00 0x7f978c15acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f9791365700 1 -- 172.21.15.40:0/1724159517 >> 172.21.15.40:0/1724159517 conn(0x7f978c0ccef0 msgr2=0x7f978c0cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:20.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f9791365700 1 -- 172.21.15.40:0/1724159517 shutdown_connections 2024-09-13T18:21:20.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f9791365700 1 -- 172.21.15.40:0/1724159517 wait complete. 2024-09-13T18:21:20.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f9791365700 1 Processor -- start 2024-09-13T18:21:20.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f9791365700 1 -- start start 2024-09-13T18:21:20.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f9791365700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f978c070120 0x7f978c151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:20.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f9791365700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f978c0b2d00 0x7f978c14c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:20.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f9791365700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f978c1512c0 0x7f978c14d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:20.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f9791365700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c14c1a0 con 0x7f978c0b2d00 2024-09-13T18:21:20.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f9791365700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c152490 con 0x7f978c1512c0 2024-09-13T18:21:20.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f9791365700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978c152060 con 0x7f978c070120 2024-09-13T18:21:20.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f978b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f978c0b2d00 0x7f978c14c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:20.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f978b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f978c0b2d00 0x7f978c14c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48560/0 (socket says 172.21.15.40:48560) 2024-09-13T18:21:20.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f978b7fe700 1 -- 172.21.15.40:0/3598601777 learned_addr learned my addr 172.21.15.40:0/3598601777 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:20.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f9790b64700 1 --2- 172.21.15.40:0/3598601777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f978c1512c0 0x7f978c14d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:20.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f978bfff700 1 --2- 172.21.15.40:0/3598601777 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f978c070120 0x7f978c151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:20.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f978b7fe700 1 -- 172.21.15.40:0/3598601777 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f978c070120 msgr2=0x7f978c151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:20.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f978b7fe700 1 --2- 172.21.15.40:0/3598601777 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f978c070120 0x7f978c151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f978b7fe700 1 -- 172.21.15.40:0/3598601777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f978c1512c0 msgr2=0x7f978c14d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:20.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f978b7fe700 1 --2- 172.21.15.40:0/3598601777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f978c1512c0 0x7f978c14d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f978b7fe700 1 -- 172.21.15.40:0/3598601777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9784009a70 con 0x7f978c0b2d00 2024-09-13T18:21:20.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f978b7fe700 1 --2- 172.21.15.40:0/3598601777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f978c0b2d00 0x7f978c14c630 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f977c00e840 tx=0x7f977c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:20.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.541+0000 7f9788ff9700 1 -- 172.21.15.40:0/3598601777 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c0095a0 con 0x7f978c0b2d00 2024-09-13T18:21:20.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.545+0000 7f9788ff9700 1 -- 172.21.15.40:0/3598601777 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977c013680 con 0x7f978c0b2d00 2024-09-13T18:21:20.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.545+0000 7f9791365700 1 -- 172.21.15.40:0/3598601777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f978c0c4f90 con 0x7f978c0b2d00 2024-09-13T18:21:20.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.545+0000 7f9788ff9700 1 -- 172.21.15.40:0/3598601777 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c0139f0 con 0x7f978c0b2d00 2024-09-13T18:21:20.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.545+0000 7f9791365700 1 -- 172.21.15.40:0/3598601777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f978c0c5550 con 0x7f978c0b2d00 2024-09-13T18:21:20.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.545+0000 7f9791365700 1 -- 172.21.15.40:0/3598601777 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f978c14bf70 con 0x7f978c0b2d00 2024-09-13T18:21:20.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.545+0000 7f9788ff9700 1 -- 172.21.15.40:0/3598601777 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f977c013b50 con 0x7f978c0b2d00 2024-09-13T18:21:20.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.545+0000 7f9788ff9700 1 --2- 172.21.15.40:0/3598601777 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9778066730 0x7f9778068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:20.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.545+0000 7f9788ff9700 1 -- 172.21.15.40:0/3598601777 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f977c084340 con 0x7f978c0b2d00 2024-09-13T18:21:20.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.545+0000 7f978bfff700 1 --2- 172.21.15.40:0/3598601777 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9778066730 0x7f9778068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:20.555 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.545+0000 7f978bfff700 1 --2- 172.21.15.40:0/3598601777 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9778066730 0x7f9778068bf0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f9780007fd0 tx=0x7f9780009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:20.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.553+0000 7f9788ff9700 1 -- 172.21.15.40:0/3598601777 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f978c14bf70 con 0x7f978c0b2d00 2024-09-13T18:21:20.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.717+0000 7f9791365700 1 -- 172.21.15.40:0/3598601777 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f978c157dc0 con 0x7f9778066730 2024-09-13T18:21:20.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.729+0000 7f9788ff9700 1 -- 172.21.15.40:0/3598601777 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+369002 (secure 0 0 0) 0x7f978000f690 con 0x7f9778066730 2024-09-13T18:21:20.740 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T18:21:20.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.733+0000 7f9791365700 1 -- 172.21.15.40:0/3598601777 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9778066730 msgr2=0x7f9778068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:20.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.733+0000 7f9791365700 1 --2- 172.21.15.40:0/3598601777 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9778066730 0x7f9778068bf0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f9780007fd0 tx=0x7f9780009040 comp rx=0 tx=0).stop 2024-09-13T18:21:20.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.733+0000 7f9791365700 1 -- 172.21.15.40:0/3598601777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f978c0b2d00 msgr2=0x7f978c14c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:20.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.733+0000 7f9791365700 1 --2- 172.21.15.40:0/3598601777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f978c0b2d00 0x7f978c14c630 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f977c00e840 tx=0x7f977c00eb50 comp rx=0 tx=0).stop 2024-09-13T18:21:20.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.733+0000 7f9791365700 1 -- 172.21.15.40:0/3598601777 shutdown_connections 2024-09-13T18:21:20.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.733+0000 7f9791365700 1 --2- 172.21.15.40:0/3598601777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f978c1512c0 0x7f978c14d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.733+0000 7f9791365700 1 --2- 172.21.15.40:0/3598601777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f978c0b2d00 0x7f978c14c630 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.733+0000 7f9791365700 1 --2- 172.21.15.40:0/3598601777 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9778066730 0x7f9778068bf0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.733+0000 7f9791365700 1 --2- 172.21.15.40:0/3598601777 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f978c070120 0x7f978c151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.733+0000 7f9791365700 1 -- 172.21.15.40:0/3598601777 >> 172.21.15.40:0/3598601777 conn(0x7f978c0ccef0 msgr2=0x7f978c155750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:20.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.733+0000 7f9791365700 1 -- 172.21.15.40:0/3598601777 shutdown_connections 2024-09-13T18:21:20.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.733+0000 7f9791365700 1 -- 172.21.15.40:0/3598601777 wait complete. 2024-09-13T18:21:20.744 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-13T18:21:20.754 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":949,"stamp":"2024-09-13T18:21:19.774698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":16532352,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":237,"num_read_kb":279,"num_write":755012,"num_write_kb":16125724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183058,"ondisk_log_size":183058,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10715160,"kb_used_data":34648,"kb_used_omap":12,"kb_used_meta":10680435,"kb_avail":744259560,"statfs":{"total":773094113280,"available":762121789440,"internally_reserved":0,"allocated":35479552,"data_stored":33992096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":10936766008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005254"},"pg_stats":[{"pgid":"2.1f","version":"27'268","reported_seq":299,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.261706+0000","last_change":"2024-09-13T17:50:34.212158+0000","last_active":"2024-09-13T18:21:13.261706+0000","last_peered":"2024-09-13T18:21:13.261706+0000","last_clean":"2024-09-13T18:21:13.261706+0000","last_became_active":"2024-09-13T17:50:34.211867+0000","last_became_peered":"2024-09-13T17:50:34.211867+0000","last_unstale":"2024-09-13T18:21:13.261706+0000","last_undegraded":"2024-09-13T18:21:13.261706+0000","last_fullsized":"2024-09-13T18:21:13.261706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:14.671095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3260943,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":267,"num_write_kb":3318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'10","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306110+0000","last_change":"2024-09-13T17:50:34.587396+0000","last_active":"2024-09-13T18:21:13.306110+0000","last_peered":"2024-09-13T18:21:13.306110+0000","last_clean":"2024-09-13T18:21:13.306110+0000","last_became_active":"2024-09-13T17:50:34.587108+0000","last_became_peered":"2024-09-13T17:50:34.587108+0000","last_unstale":"2024-09-13T18:21:13.306110+0000","last_undegraded":"2024-09-13T18:21:13.306110+0000","last_fullsized":"2024-09-13T18:21:13.306110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:39:16.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'5594","reported_seq":5620,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.876951+0000","last_change":"2024-09-13T17:50:38.615059+0000","last_active":"2024-09-13T18:21:02.876951+0000","last_peered":"2024-09-13T18:21:02.876951+0000","last_clean":"2024-09-13T18:21:02.876951+0000","last_became_active":"2024-09-13T17:50:38.614723+0000","last_became_peered":"2024-09-13T17:50:38.614723+0000","last_unstale":"2024-09-13T18:21:02.876951+0000","last_undegraded":"2024-09-13T18:21:02.876951+0000","last_fullsized":"2024-09-13T18:21:02.876951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5594,"log_dups_size":0,"ondisk_log_size":5594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:23:01.902181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23732,"num_write_kb":533374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'6","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.773351+0000","last_change":"2024-09-13T17:50:34.204760+0000","last_active":"2024-09-13T18:21:02.773351+0000","last_peered":"2024-09-13T18:21:02.773351+0000","last_clean":"2024-09-13T18:21:02.773351+0000","last_became_active":"2024-09-13T17:50:34.202930+0000","last_became_peered":"2024-09-13T17:50:34.202930+0000","last_unstale":"2024-09-13T18:21:02.773351+0000","last_undegraded":"2024-09-13T18:21:02.773351+0000","last_fullsized":"2024-09-13T18:21:02.773351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:59.784701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'20","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306487+0000","last_change":"2024-09-13T17:51:16.366352+0000","last_active":"2024-09-13T18:21:13.306487+0000","last_peered":"2024-09-13T18:21:13.306487+0000","last_clean":"2024-09-13T18:21:13.306487+0000","last_became_active":"2024-09-13T17:51:16.365492+0000","last_became_peered":"2024-09-13T17:51:16.365492+0000","last_unstale":"2024-09-13T18:21:13.306487+0000","last_undegraded":"2024-09-13T18:21:13.306487+0000","last_fullsized":"2024-09-13T18:21:13.306487+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:42.192182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'5615","reported_seq":5641,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.875980+0000","last_change":"2024-09-13T17:50:38.618751+0000","last_active":"2024-09-13T18:21:02.875980+0000","last_peered":"2024-09-13T18:21:02.875980+0000","last_clean":"2024-09-13T18:21:02.875980+0000","last_became_active":"2024-09-13T17:50:38.618423+0000","last_became_peered":"2024-09-13T17:50:38.618423+0000","last_unstale":"2024-09-13T18:21:02.875980+0000","last_undegraded":"2024-09-13T18:21:02.875980+0000","last_fullsized":"2024-09-13T18:21:02.875980+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5615,"log_dups_size":0,"ondisk_log_size":5615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:50:02.418785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23762,"num_write_kb":516686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'11","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.801229+0000","last_change":"2024-09-13T17:50:34.214972+0000","last_active":"2024-09-13T18:21:02.801229+0000","last_peered":"2024-09-13T18:21:02.801229+0000","last_clean":"2024-09-13T18:21:02.801229+0000","last_became_active":"2024-09-13T17:50:34.214652+0000","last_became_peered":"2024-09-13T17:50:34.214652+0000","last_unstale":"2024-09-13T18:21:02.801229+0000","last_undegraded":"2024-09-13T18:21:02.801229+0000","last_fullsized":"2024-09-13T18:21:02.801229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:57.176854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'18","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.292937+0000","last_change":"2024-09-13T17:51:16.281831+0000","last_active":"2024-09-13T18:21:13.292937+0000","last_peered":"2024-09-13T18:21:13.292937+0000","last_clean":"2024-09-13T18:21:13.292937+0000","last_became_active":"2024-09-13T17:51:16.281549+0000","last_became_peered":"2024-09-13T17:51:16.281549+0000","last_unstale":"2024-09-13T18:21:13.292937+0000","last_undegraded":"2024-09-13T18:21:13.292937+0000","last_fullsized":"2024-09-13T18:21:13.292937+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:43.894437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"27'5142","reported_seq":5168,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.803386+0000","last_change":"2024-09-13T17:50:38.616219+0000","last_active":"2024-09-13T18:21:02.803386+0000","last_peered":"2024-09-13T18:21:02.803386+0000","last_clean":"2024-09-13T18:21:02.803386+0000","last_became_active":"2024-09-13T17:50:38.614329+0000","last_became_peered":"2024-09-13T17:50:38.614329+0000","last_unstale":"2024-09-13T18:21:02.803386+0000","last_undegraded":"2024-09-13T18:21:02.803386+0000","last_fullsized":"2024-09-13T18:21:02.803386+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5142,"log_dups_size":0,"ondisk_log_size":5142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:25.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21334,"num_write_kb":478261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'8","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.159841+0000","last_change":"2024-09-13T17:50:34.216123+0000","last_active":"2024-09-13T18:20:25.159841+0000","last_peered":"2024-09-13T18:20:25.159841+0000","last_clean":"2024-09-13T18:20:25.159841+0000","last_became_active":"2024-09-13T17:50:34.215661+0000","last_became_peered":"2024-09-13T17:50:34.215661+0000","last_unstale":"2024-09-13T18:20:25.159841+0000","last_undegraded":"2024-09-13T18:20:25.159841+0000","last_fullsized":"2024-09-13T18:20:25.159841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:20.895037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'30","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.318958+0000","last_change":"2024-09-13T17:50:34.599673+0000","last_active":"2024-09-13T18:21:13.318958+0000","last_peered":"2024-09-13T18:21:13.318958+0000","last_clean":"2024-09-13T18:21:13.318958+0000","last_became_active":"2024-09-13T17:50:34.599409+0000","last_became_peered":"2024-09-13T17:50:34.599409+0000","last_unstale":"2024-09-13T18:21:13.318958+0000","last_undegraded":"2024-09-13T18:21:13.318958+0000","last_fullsized":"2024-09-13T18:21:13.318958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:03:31.394385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'4871","reported_seq":4897,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.743078+0000","last_change":"2024-09-13T17:50:38.616317+0000","last_active":"2024-09-13T18:21:02.743078+0000","last_peered":"2024-09-13T18:21:02.743078+0000","last_clean":"2024-09-13T18:21:02.743078+0000","last_became_active":"2024-09-13T17:50:38.614311+0000","last_became_peered":"2024-09-13T17:50:38.614311+0000","last_unstale":"2024-09-13T18:21:02.743078+0000","last_undegraded":"2024-09-13T18:21:02.743078+0000","last_fullsized":"2024-09-13T18:21:02.743078+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4871,"log_dups_size":0,"ondisk_log_size":4871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:29.221522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20106,"num_write_kb":432357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'14","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.313143+0000","last_change":"2024-09-13T17:50:34.599490+0000","last_active":"2024-09-13T18:21:13.313143+0000","last_peered":"2024-09-13T18:21:13.313143+0000","last_clean":"2024-09-13T18:21:13.313143+0000","last_became_active":"2024-09-13T17:50:34.599207+0000","last_became_peered":"2024-09-13T17:50:34.599207+0000","last_unstale":"2024-09-13T18:21:13.313143+0000","last_undegraded":"2024-09-13T18:21:13.313143+0000","last_fullsized":"2024-09-13T18:21:13.313143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:11.178161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'1092","reported_seq":1122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.671116+0000","last_change":"2024-09-13T17:50:34.215697+0000","last_active":"2024-09-13T18:21:02.671116+0000","last_peered":"2024-09-13T18:21:02.671116+0000","last_clean":"2024-09-13T18:21:02.671116+0000","last_became_active":"2024-09-13T17:50:34.215409+0000","last_became_peered":"2024-09-13T17:50:34.215409+0000","last_unstale":"2024-09-13T18:21:02.671116+0000","last_undegraded":"2024-09-13T18:21:02.671116+0000","last_fullsized":"2024-09-13T18:21:02.671116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:21.609567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1088,"num_write_kb":4339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'5394","reported_seq":5421,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.852704+0000","last_change":"2024-09-13T17:50:38.611817+0000","last_active":"2024-09-13T18:21:02.852704+0000","last_peered":"2024-09-13T18:21:02.852704+0000","last_clean":"2024-09-13T18:21:02.852704+0000","last_became_active":"2024-09-13T17:50:38.611263+0000","last_became_peered":"2024-09-13T17:50:38.611263+0000","last_unstale":"2024-09-13T18:21:02.852704+0000","last_undegraded":"2024-09-13T18:21:02.852704+0000","last_fullsized":"2024-09-13T18:21:02.852704+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5394,"log_dups_size":0,"ondisk_log_size":5394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:56:47.466672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22326,"num_write_kb":494524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'19","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.308364+0000","last_change":"2024-09-13T17:50:34.586307+0000","last_active":"2024-09-13T18:21:13.308364+0000","last_peered":"2024-09-13T18:21:13.308364+0000","last_clean":"2024-09-13T18:21:13.308364+0000","last_became_active":"2024-09-13T17:50:34.585831+0000","last_became_peered":"2024-09-13T17:50:34.585831+0000","last_unstale":"2024-09-13T18:21:13.308364+0000","last_undegraded":"2024-09-13T18:21:13.308364+0000","last_fullsized":"2024-09-13T18:21:13.308364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:40.912352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'9","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.773590+0000","last_change":"2024-09-13T17:50:34.211850+0000","last_active":"2024-09-13T18:21:02.773590+0000","last_peered":"2024-09-13T18:21:02.773590+0000","last_clean":"2024-09-13T18:21:02.773590+0000","last_became_active":"2024-09-13T17:50:34.211594+0000","last_became_peered":"2024-09-13T17:50:34.211594+0000","last_unstale":"2024-09-13T18:21:02.773590+0000","last_undegraded":"2024-09-13T18:21:02.773590+0000","last_fullsized":"2024-09-13T18:21:02.773590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:35:02.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'5504","reported_seq":5531,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.796881+0000","last_change":"2024-09-13T17:50:38.613576+0000","last_active":"2024-09-13T18:21:02.796881+0000","last_peered":"2024-09-13T18:21:02.796881+0000","last_clean":"2024-09-13T18:21:02.796881+0000","last_became_active":"2024-09-13T17:50:38.612577+0000","last_became_peered":"2024-09-13T17:50:38.612577+0000","last_unstale":"2024-09-13T18:21:02.796881+0000","last_undegraded":"2024-09-13T18:21:02.796881+0000","last_fullsized":"2024-09-13T18:21:02.796881+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5504,"log_dups_size":0,"ondisk_log_size":5504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:25.551060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23286,"num_write_kb":502495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'19","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306452+0000","last_change":"2024-09-13T17:50:34.587273+0000","last_active":"2024-09-13T18:21:13.306452+0000","last_peered":"2024-09-13T18:21:13.306452+0000","last_clean":"2024-09-13T18:21:13.306452+0000","last_became_active":"2024-09-13T17:50:34.587027+0000","last_became_peered":"2024-09-13T17:50:34.587027+0000","last_unstale":"2024-09-13T18:21:13.306452+0000","last_undegraded":"2024-09-13T18:21:13.306452+0000","last_fullsized":"2024-09-13T18:21:13.306452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:58.826737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'5","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.773643+0000","last_change":"2024-09-13T17:50:34.211845+0000","last_active":"2024-09-13T18:21:02.773643+0000","last_peered":"2024-09-13T18:21:02.773643+0000","last_clean":"2024-09-13T18:21:02.773643+0000","last_became_active":"2024-09-13T17:50:34.211568+0000","last_became_peered":"2024-09-13T17:50:34.211568+0000","last_unstale":"2024-09-13T18:21:02.773643+0000","last_undegraded":"2024-09-13T18:21:02.773643+0000","last_fullsized":"2024-09-13T18:21:02.773643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:56.348984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'6938","reported_seq":6965,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.647468+0000","last_change":"2024-09-13T17:50:38.610292+0000","last_active":"2024-09-13T18:21:02.647468+0000","last_peered":"2024-09-13T18:21:02.647468+0000","last_clean":"2024-09-13T18:21:02.647468+0000","last_became_active":"2024-09-13T17:50:38.609832+0000","last_became_peered":"2024-09-13T17:50:38.609832+0000","last_unstale":"2024-09-13T18:21:02.647468+0000","last_undegraded":"2024-09-13T18:21:02.647468+0000","last_fullsized":"2024-09-13T18:21:02.647468+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6938,"log_dups_size":0,"ondisk_log_size":6938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:13.783068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28644,"num_write_kb":622131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"27'12","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.931907+0000","last_change":"2024-09-13T17:50:34.593566+0000","last_active":"2024-09-13T18:21:02.931907+0000","last_peered":"2024-09-13T18:21:02.931907+0000","last_clean":"2024-09-13T18:21:02.931907+0000","last_became_active":"2024-09-13T17:50:34.588794+0000","last_became_peered":"2024-09-13T17:50:34.588794+0000","last_unstale":"2024-09-13T18:21:02.931907+0000","last_undegraded":"2024-09-13T18:21:02.931907+0000","last_fullsized":"2024-09-13T18:21:02.931907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:13.192453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'6","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.689388+0000","last_change":"2024-09-13T17:50:34.214638+0000","last_active":"2024-09-13T18:21:02.689388+0000","last_peered":"2024-09-13T18:21:02.689388+0000","last_clean":"2024-09-13T18:21:02.689388+0000","last_became_active":"2024-09-13T17:50:34.214292+0000","last_became_peered":"2024-09-13T17:50:34.214292+0000","last_unstale":"2024-09-13T18:21:02.689388+0000","last_undegraded":"2024-09-13T18:21:02.689388+0000","last_fullsized":"2024-09-13T18:21:02.689388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:57.377104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'4588","reported_seq":4615,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.886736+0000","last_change":"2024-09-13T17:50:38.613368+0000","last_active":"2024-09-13T18:21:02.886736+0000","last_peered":"2024-09-13T18:21:02.886736+0000","last_clean":"2024-09-13T18:21:02.886736+0000","last_became_active":"2024-09-13T17:50:38.613010+0000","last_became_peered":"2024-09-13T17:50:38.613010+0000","last_unstale":"2024-09-13T18:21:02.886736+0000","last_undegraded":"2024-09-13T18:21:02.886736+0000","last_fullsized":"2024-09-13T18:21:02.886736+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4588,"log_dups_size":0,"ondisk_log_size":4588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:42.706625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19354,"num_write_kb":454021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'26","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.302413+0000","last_change":"2024-09-13T17:50:34.582327+0000","last_active":"2024-09-13T18:21:13.302413+0000","last_peered":"2024-09-13T18:21:13.302413+0000","last_clean":"2024-09-13T18:21:13.302413+0000","last_became_active":"2024-09-13T17:50:34.582050+0000","last_became_peered":"2024-09-13T17:50:34.582050+0000","last_unstale":"2024-09-13T18:21:13.302413+0000","last_undegraded":"2024-09-13T18:21:13.302413+0000","last_fullsized":"2024-09-13T18:21:13.302413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:52.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'9","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.838047+0000","last_change":"2024-09-13T17:50:34.211512+0000","last_active":"2024-09-13T18:21:02.838047+0000","last_peered":"2024-09-13T18:21:02.838047+0000","last_clean":"2024-09-13T18:21:02.838047+0000","last_became_active":"2024-09-13T17:50:34.211230+0000","last_became_peered":"2024-09-13T17:50:34.211230+0000","last_unstale":"2024-09-13T18:21:02.838047+0000","last_undegraded":"2024-09-13T18:21:02.838047+0000","last_fullsized":"2024-09-13T18:21:02.838047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:40.707777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'6024","reported_seq":6052,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.702767+0000","last_change":"2024-09-13T17:50:38.608255+0000","last_active":"2024-09-13T18:21:02.702767+0000","last_peered":"2024-09-13T18:21:02.702767+0000","last_clean":"2024-09-13T18:21:02.702767+0000","last_became_active":"2024-09-13T17:50:38.606697+0000","last_became_peered":"2024-09-13T17:50:38.606697+0000","last_unstale":"2024-09-13T18:21:02.702767+0000","last_undegraded":"2024-09-13T18:21:02.702767+0000","last_fullsized":"2024-09-13T18:21:02.702767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6024,"log_dups_size":0,"ondisk_log_size":6024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:47.265007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25552,"num_write_kb":507487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'18","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.314914+0000","last_change":"2024-09-13T17:50:34.594476+0000","last_active":"2024-09-13T18:21:13.314914+0000","last_peered":"2024-09-13T18:21:13.314914+0000","last_clean":"2024-09-13T18:21:13.314914+0000","last_became_active":"2024-09-13T17:50:34.594215+0000","last_became_peered":"2024-09-13T17:50:34.594215+0000","last_unstale":"2024-09-13T18:21:13.314914+0000","last_undegraded":"2024-09-13T18:21:13.314914+0000","last_fullsized":"2024-09-13T18:21:13.314914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:02:27.528019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'284","reported_seq":315,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.685813+0000","last_change":"2024-09-13T17:50:34.204018+0000","last_active":"2024-09-13T18:21:02.685813+0000","last_peered":"2024-09-13T18:21:02.685813+0000","last_clean":"2024-09-13T18:21:02.685813+0000","last_became_active":"2024-09-13T17:50:34.203702+0000","last_became_peered":"2024-09-13T17:50:34.203702+0000","last_unstale":"2024-09-13T18:21:02.685813+0000","last_undegraded":"2024-09-13T18:21:02.685813+0000","last_fullsized":"2024-09-13T18:21:02.685813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:44.431226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":605,"num_write_kb":197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'6030","reported_seq":6069,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.817288+0000","last_change":"2024-09-13T17:51:16.365782+0000","last_active":"2024-09-13T18:21:02.817288+0000","last_peered":"2024-09-13T18:21:02.817288+0000","last_clean":"2024-09-13T18:21:02.817288+0000","last_became_active":"2024-09-13T17:51:16.364540+0000","last_became_peered":"2024-09-13T17:51:16.364540+0000","last_unstale":"2024-09-13T18:21:02.817288+0000","last_undegraded":"2024-09-13T18:21:02.817288+0000","last_fullsized":"2024-09-13T18:21:02.817288+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6030,"log_dups_size":0,"ondisk_log_size":6030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:18.753557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25617,"num_write_kb":548663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'18","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.290374+0000","last_change":"2024-09-13T17:50:34.586802+0000","last_active":"2024-09-13T18:21:13.290374+0000","last_peered":"2024-09-13T18:21:13.290374+0000","last_clean":"2024-09-13T18:21:13.290374+0000","last_became_active":"2024-09-13T17:50:34.586552+0000","last_became_peered":"2024-09-13T17:50:34.586552+0000","last_unstale":"2024-09-13T18:21:13.290374+0000","last_undegraded":"2024-09-13T18:21:13.290374+0000","last_fullsized":"2024-09-13T18:21:13.290374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:43.383975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'7","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.741537+0000","last_change":"2024-09-13T17:50:34.217083+0000","last_active":"2024-09-13T18:21:02.741537+0000","last_peered":"2024-09-13T18:21:02.741537+0000","last_clean":"2024-09-13T18:21:02.741537+0000","last_became_active":"2024-09-13T17:50:34.216758+0000","last_became_peered":"2024-09-13T17:50:34.216758+0000","last_unstale":"2024-09-13T18:21:02.741537+0000","last_undegraded":"2024-09-13T18:21:02.741537+0000","last_fullsized":"2024-09-13T18:21:02.741537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:36.473942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'4887","reported_seq":4915,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.617066+0000","last_change":"2024-09-13T17:50:38.614667+0000","last_active":"2024-09-13T18:21:02.617066+0000","last_peered":"2024-09-13T18:21:02.617066+0000","last_clean":"2024-09-13T18:21:02.617066+0000","last_became_active":"2024-09-13T17:50:38.614284+0000","last_became_peered":"2024-09-13T17:50:38.614284+0000","last_unstale":"2024-09-13T18:21:02.617066+0000","last_undegraded":"2024-09-13T18:21:02.617066+0000","last_fullsized":"2024-09-13T18:21:02.617066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4887,"log_dups_size":0,"ondisk_log_size":4887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:29.616794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20690,"num_write_kb":442963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'16","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.315360+0000","last_change":"2024-09-13T17:50:34.588603+0000","last_active":"2024-09-13T18:21:13.315360+0000","last_peered":"2024-09-13T18:21:13.315360+0000","last_clean":"2024-09-13T18:21:13.315360+0000","last_became_active":"2024-09-13T17:50:34.588317+0000","last_became_peered":"2024-09-13T17:50:34.588317+0000","last_unstale":"2024-09-13T18:21:13.315360+0000","last_undegraded":"2024-09-13T18:21:13.315360+0000","last_fullsized":"2024-09-13T18:21:13.315360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:03.743130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'3","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.727771+0000","last_change":"2024-09-13T17:50:34.204335+0000","last_active":"2024-09-13T18:21:02.727771+0000","last_peered":"2024-09-13T18:21:02.727771+0000","last_clean":"2024-09-13T18:21:02.727771+0000","last_became_active":"2024-09-13T17:50:34.204060+0000","last_became_peered":"2024-09-13T17:50:34.204060+0000","last_unstale":"2024-09-13T18:21:02.727771+0000","last_undegraded":"2024-09-13T18:21:02.727771+0000","last_fullsized":"2024-09-13T18:21:02.727771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:54:41.831344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'6031","reported_seq":6059,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.587876+0000","last_change":"2024-09-13T17:50:38.619590+0000","last_active":"2024-09-13T18:21:02.587876+0000","last_peered":"2024-09-13T18:21:02.587876+0000","last_clean":"2024-09-13T18:21:02.587876+0000","last_became_active":"2024-09-13T17:50:38.617257+0000","last_became_peered":"2024-09-13T17:50:38.617257+0000","last_unstale":"2024-09-13T18:21:02.587876+0000","last_undegraded":"2024-09-13T18:21:02.587876+0000","last_fullsized":"2024-09-13T18:21:02.587876+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6031,"log_dups_size":0,"ondisk_log_size":6031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:34.870291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25529,"num_write_kb":564991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'16","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310390+0000","last_change":"2024-09-13T17:50:34.588466+0000","last_active":"2024-09-13T18:21:13.310390+0000","last_peered":"2024-09-13T18:21:13.310390+0000","last_clean":"2024-09-13T18:21:13.310390+0000","last_became_active":"2024-09-13T17:50:34.588063+0000","last_became_peered":"2024-09-13T17:50:34.588063+0000","last_unstale":"2024-09-13T18:21:13.310390+0000","last_undegraded":"2024-09-13T18:21:13.310390+0000","last_fullsized":"2024-09-13T18:21:13.310390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:43:00.669203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'4","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.629940+0000","last_change":"2024-09-13T17:50:34.209239+0000","last_active":"2024-09-13T18:21:02.629940+0000","last_peered":"2024-09-13T18:21:02.629940+0000","last_clean":"2024-09-13T18:21:02.629940+0000","last_became_active":"2024-09-13T17:50:34.208196+0000","last_became_peered":"2024-09-13T17:50:34.208196+0000","last_unstale":"2024-09-13T18:21:02.629940+0000","last_undegraded":"2024-09-13T18:21:02.629940+0000","last_fullsized":"2024-09-13T18:21:02.629940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:39.253917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'5410","reported_seq":5438,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.883285+0000","last_change":"2024-09-13T17:50:38.611021+0000","last_active":"2024-09-13T18:21:02.883285+0000","last_peered":"2024-09-13T18:21:02.883285+0000","last_clean":"2024-09-13T18:21:02.883285+0000","last_became_active":"2024-09-13T17:50:38.609794+0000","last_became_peered":"2024-09-13T17:50:38.609794+0000","last_unstale":"2024-09-13T18:21:02.883285+0000","last_undegraded":"2024-09-13T18:21:02.883285+0000","last_fullsized":"2024-09-13T18:21:02.883285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5410,"log_dups_size":0,"ondisk_log_size":5410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:59.980236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22186,"num_write_kb":506224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'22","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.282060+0000","last_change":"2024-09-13T17:50:34.584290+0000","last_active":"2024-09-13T18:21:13.282060+0000","last_peered":"2024-09-13T18:21:13.282060+0000","last_clean":"2024-09-13T18:21:13.282060+0000","last_became_active":"2024-09-13T17:50:34.584023+0000","last_became_peered":"2024-09-13T17:50:34.584023+0000","last_unstale":"2024-09-13T18:21:13.282060+0000","last_undegraded":"2024-09-13T18:21:13.282060+0000","last_fullsized":"2024-09-13T18:21:13.282060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:11.304986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'127","reported_seq":279,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:31.126933+0000","last_change":"2024-09-13T17:50:34.198932+0000","last_active":"2024-09-13T18:20:31.126933+0000","last_peered":"2024-09-13T18:20:31.126933+0000","last_clean":"2024-09-13T18:20:31.126933+0000","last_became_active":"2024-09-13T17:50:34.198603+0000","last_became_peered":"2024-09-13T17:50:34.198603+0000","last_unstale":"2024-09-13T18:20:31.126933+0000","last_undegraded":"2024-09-13T18:20:31.126933+0000","last_fullsized":"2024-09-13T18:20:31.126933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:00.917126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":94032,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":179,"num_write":125,"num_write_kb":180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'5807","reported_seq":5835,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.679912+0000","last_change":"2024-09-13T17:50:38.611326+0000","last_active":"2024-09-13T18:21:02.679912+0000","last_peered":"2024-09-13T18:21:02.679912+0000","last_clean":"2024-09-13T18:21:02.679912+0000","last_became_active":"2024-09-13T17:50:38.610900+0000","last_became_peered":"2024-09-13T17:50:38.610900+0000","last_unstale":"2024-09-13T18:21:02.679912+0000","last_undegraded":"2024-09-13T18:21:02.679912+0000","last_fullsized":"2024-09-13T18:21:02.679912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5807,"log_dups_size":0,"ondisk_log_size":5807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:17:09.994736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24429,"num_write_kb":531605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'13","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310665+0000","last_change":"2024-09-13T17:50:34.594482+0000","last_active":"2024-09-13T18:21:13.310665+0000","last_peered":"2024-09-13T18:21:13.310665+0000","last_clean":"2024-09-13T18:21:13.310665+0000","last_became_active":"2024-09-13T17:50:34.594201+0000","last_became_peered":"2024-09-13T17:50:34.594201+0000","last_unstale":"2024-09-13T18:21:13.310665+0000","last_undegraded":"2024-09-13T18:21:13.310665+0000","last_fullsized":"2024-09-13T18:21:13.310665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:06.697507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'8","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.813884+0000","last_change":"2024-09-13T17:50:34.218376+0000","last_active":"2024-09-13T18:21:02.813884+0000","last_peered":"2024-09-13T18:21:02.813884+0000","last_clean":"2024-09-13T18:21:02.813884+0000","last_became_active":"2024-09-13T17:50:34.218069+0000","last_became_peered":"2024-09-13T17:50:34.218069+0000","last_unstale":"2024-09-13T18:21:02.813884+0000","last_undegraded":"2024-09-13T18:21:02.813884+0000","last_fullsized":"2024-09-13T18:21:02.813884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:53:54.288969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'4957","reported_seq":4985,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.963407+0000","last_change":"2024-09-13T17:50:38.613206+0000","last_active":"2024-09-13T18:21:02.963407+0000","last_peered":"2024-09-13T18:21:02.963407+0000","last_clean":"2024-09-13T18:21:02.963407+0000","last_became_active":"2024-09-13T17:50:38.612856+0000","last_became_peered":"2024-09-13T17:50:38.612856+0000","last_unstale":"2024-09-13T18:21:02.963407+0000","last_undegraded":"2024-09-13T18:21:02.963407+0000","last_fullsized":"2024-09-13T18:21:02.963407+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4957,"log_dups_size":0,"ondisk_log_size":4957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:31:20.190996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20520,"num_write_kb":434212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'16","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.316288+0000","last_change":"2024-09-13T17:50:34.582931+0000","last_active":"2024-09-13T18:21:13.316288+0000","last_peered":"2024-09-13T18:21:13.316288+0000","last_clean":"2024-09-13T18:21:13.316288+0000","last_became_active":"2024-09-13T17:50:34.582446+0000","last_became_peered":"2024-09-13T17:50:34.582446+0000","last_unstale":"2024-09-13T18:21:13.316288+0000","last_undegraded":"2024-09-13T18:21:13.316288+0000","last_fullsized":"2024-09-13T18:21:13.316288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:30.554785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'8","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.157427+0000","last_change":"2024-09-13T17:50:34.201237+0000","last_active":"2024-09-13T18:20:25.157427+0000","last_peered":"2024-09-13T18:20:25.157427+0000","last_clean":"2024-09-13T18:20:25.157427+0000","last_became_active":"2024-09-13T17:50:34.200873+0000","last_became_peered":"2024-09-13T17:50:34.200873+0000","last_unstale":"2024-09-13T18:20:25.157427+0000","last_undegraded":"2024-09-13T18:20:25.157427+0000","last_fullsized":"2024-09-13T18:20:25.157427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:02.711342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":185,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:18:31.266834+0000","last_change":"2024-09-13T17:50:22.095367+0000","last_active":"2024-09-13T18:18:31.266834+0000","last_peered":"2024-09-13T18:18:31.266834+0000","last_clean":"2024-09-13T18:18:31.266834+0000","last_became_active":"2024-09-13T17:50:22.093734+0000","last_became_peered":"2024-09-13T17:50:22.093734+0000","last_unstale":"2024-09-13T18:18:31.266834+0000","last_undegraded":"2024-09-13T18:18:31.266834+0000","last_fullsized":"2024-09-13T18:18:31.266834+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_clean_scrub_stamp":"2024-09-13T17:50:21.057812+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:23.531613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'5208","reported_seq":5236,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.886919+0000","last_change":"2024-09-13T17:50:38.618880+0000","last_active":"2024-09-13T18:21:02.886919+0000","last_peered":"2024-09-13T18:21:02.886919+0000","last_clean":"2024-09-13T18:21:02.886919+0000","last_became_active":"2024-09-13T17:50:38.618488+0000","last_became_peered":"2024-09-13T17:50:38.618488+0000","last_unstale":"2024-09-13T18:21:02.886919+0000","last_undegraded":"2024-09-13T18:21:02.886919+0000","last_fullsized":"2024-09-13T18:21:02.886919+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5208,"log_dups_size":0,"ondisk_log_size":5208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:03.431876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22093,"num_write_kb":481370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'22","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317778+0000","last_change":"2024-09-13T17:50:34.585881+0000","last_active":"2024-09-13T18:21:13.317778+0000","last_peered":"2024-09-13T18:21:13.317778+0000","last_clean":"2024-09-13T18:21:13.317778+0000","last_became_active":"2024-09-13T17:50:34.585166+0000","last_became_peered":"2024-09-13T17:50:34.585166+0000","last_unstale":"2024-09-13T18:21:13.317778+0000","last_undegraded":"2024-09-13T18:21:13.317778+0000","last_fullsized":"2024-09-13T18:21:13.317778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:06.654658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'5","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.654921+0000","last_change":"2024-09-13T17:50:34.210934+0000","last_active":"2024-09-13T18:21:02.654921+0000","last_peered":"2024-09-13T18:21:02.654921+0000","last_clean":"2024-09-13T18:21:02.654921+0000","last_became_active":"2024-09-13T17:50:34.210589+0000","last_became_peered":"2024-09-13T17:50:34.210589+0000","last_unstale":"2024-09-13T18:21:02.654921+0000","last_undegraded":"2024-09-13T18:21:02.654921+0000","last_fullsized":"2024-09-13T18:21:02.654921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:26.213295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'5302","reported_seq":5340,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.817379+0000","last_change":"2024-09-13T17:51:16.364911+0000","last_active":"2024-09-13T18:21:02.817379+0000","last_peered":"2024-09-13T18:21:02.817379+0000","last_clean":"2024-09-13T18:21:02.817379+0000","last_became_active":"2024-09-13T17:51:16.363974+0000","last_became_peered":"2024-09-13T17:51:16.363974+0000","last_unstale":"2024-09-13T18:21:02.817379+0000","last_undegraded":"2024-09-13T18:21:02.817379+0000","last_fullsized":"2024-09-13T18:21:02.817379+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5302,"log_dups_size":0,"ondisk_log_size":5302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:12.730197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22724,"num_write_kb":488088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'20","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.309412+0000","last_change":"2024-09-13T17:50:34.590169+0000","last_active":"2024-09-13T18:21:13.309412+0000","last_peered":"2024-09-13T18:21:13.309412+0000","last_clean":"2024-09-13T18:21:13.309412+0000","last_became_active":"2024-09-13T17:50:34.589434+0000","last_became_peered":"2024-09-13T17:50:34.589434+0000","last_unstale":"2024-09-13T18:21:13.309412+0000","last_undegraded":"2024-09-13T18:21:13.309412+0000","last_fullsized":"2024-09-13T18:21:13.309412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:12.545412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'4","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.155671+0000","last_change":"2024-09-13T17:50:34.204269+0000","last_active":"2024-09-13T18:20:25.155671+0000","last_peered":"2024-09-13T18:20:25.155671+0000","last_clean":"2024-09-13T18:20:25.155671+0000","last_became_active":"2024-09-13T17:50:34.203874+0000","last_became_peered":"2024-09-13T17:50:34.203874+0000","last_unstale":"2024-09-13T18:20:25.155671+0000","last_undegraded":"2024-09-13T18:20:25.155671+0000","last_fullsized":"2024-09-13T18:20:25.155671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:10.410267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'5348","reported_seq":5375,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.944893+0000","last_change":"2024-09-13T17:50:38.615989+0000","last_active":"2024-09-13T18:21:02.944893+0000","last_peered":"2024-09-13T18:21:02.944893+0000","last_clean":"2024-09-13T18:21:02.944893+0000","last_became_active":"2024-09-13T17:50:38.615530+0000","last_became_peered":"2024-09-13T17:50:38.615530+0000","last_unstale":"2024-09-13T18:21:02.944893+0000","last_undegraded":"2024-09-13T18:21:02.944893+0000","last_fullsized":"2024-09-13T18:21:02.944893+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5348,"log_dups_size":0,"ondisk_log_size":5348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:41.064092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22134,"num_write_kb":473293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'20","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.321742+0000","last_change":"2024-09-13T17:50:34.595120+0000","last_active":"2024-09-13T18:21:13.321742+0000","last_peered":"2024-09-13T18:21:13.321742+0000","last_clean":"2024-09-13T18:21:13.321742+0000","last_became_active":"2024-09-13T17:50:34.594842+0000","last_became_peered":"2024-09-13T17:50:34.594842+0000","last_unstale":"2024-09-13T18:21:13.321742+0000","last_undegraded":"2024-09-13T18:21:13.321742+0000","last_fullsized":"2024-09-13T18:21:13.321742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:15:38.982524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'9","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.683517+0000","last_change":"2024-09-13T17:50:34.199957+0000","last_active":"2024-09-13T18:21:02.683517+0000","last_peered":"2024-09-13T18:21:02.683517+0000","last_clean":"2024-09-13T18:21:02.683517+0000","last_became_active":"2024-09-13T17:50:34.199650+0000","last_became_peered":"2024-09-13T17:50:34.199650+0000","last_unstale":"2024-09-13T18:21:02.683517+0000","last_undegraded":"2024-09-13T18:21:02.683517+0000","last_fullsized":"2024-09-13T18:21:02.683517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:26.929549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'5161","reported_seq":5189,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.762344+0000","last_change":"2024-09-13T17:50:38.609731+0000","last_active":"2024-09-13T18:21:02.762344+0000","last_peered":"2024-09-13T18:21:02.762344+0000","last_clean":"2024-09-13T18:21:02.762344+0000","last_became_active":"2024-09-13T17:50:38.609354+0000","last_became_peered":"2024-09-13T17:50:38.609354+0000","last_unstale":"2024-09-13T18:21:02.762344+0000","last_undegraded":"2024-09-13T18:21:02.762344+0000","last_fullsized":"2024-09-13T18:21:02.762344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5161,"log_dups_size":0,"ondisk_log_size":5161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:14.142369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21929,"num_write_kb":464406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'10","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.313609+0000","last_change":"2024-09-13T17:50:34.595146+0000","last_active":"2024-09-13T18:21:13.313609+0000","last_peered":"2024-09-13T18:21:13.313609+0000","last_clean":"2024-09-13T18:21:13.313609+0000","last_became_active":"2024-09-13T17:50:34.594876+0000","last_became_peered":"2024-09-13T17:50:34.594876+0000","last_unstale":"2024-09-13T18:21:13.313609+0000","last_undegraded":"2024-09-13T18:21:13.313609+0000","last_fullsized":"2024-09-13T18:21:13.313609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:26:10.638438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'4","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:19:51.940271+0000","last_change":"2024-09-13T17:50:34.200640+0000","last_active":"2024-09-13T18:19:51.940271+0000","last_peered":"2024-09-13T18:19:51.940271+0000","last_clean":"2024-09-13T18:19:51.940271+0000","last_became_active":"2024-09-13T17:50:34.200197+0000","last_became_peered":"2024-09-13T17:50:34.200197+0000","last_unstale":"2024-09-13T18:19:51.940271+0000","last_undegraded":"2024-09-13T18:19:51.940271+0000","last_fullsized":"2024-09-13T18:19:51.940271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:37:53.039720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'5052","reported_seq":5090,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.778127+0000","last_change":"2024-09-13T17:51:16.281995+0000","last_active":"2024-09-13T18:21:02.778127+0000","last_peered":"2024-09-13T18:21:02.778127+0000","last_clean":"2024-09-13T18:21:02.778127+0000","last_became_active":"2024-09-13T17:51:16.281476+0000","last_became_peered":"2024-09-13T17:51:16.281476+0000","last_unstale":"2024-09-13T18:21:02.778127+0000","last_undegraded":"2024-09-13T18:21:02.778127+0000","last_fullsized":"2024-09-13T18:21:02.778127+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5052,"log_dups_size":0,"ondisk_log_size":5052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:21:56.254641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20895,"num_write_kb":464282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3,6],"acting":[4,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'18","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.301423+0000","last_change":"2024-09-13T17:50:34.585741+0000","last_active":"2024-09-13T18:21:13.301423+0000","last_peered":"2024-09-13T18:21:13.301423+0000","last_clean":"2024-09-13T18:21:13.301423+0000","last_became_active":"2024-09-13T17:50:34.585464+0000","last_became_peered":"2024-09-13T17:50:34.585464+0000","last_unstale":"2024-09-13T18:21:13.301423+0000","last_undegraded":"2024-09-13T18:21:13.301423+0000","last_fullsized":"2024-09-13T18:21:13.301423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:59:22.031459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'19","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.673198+0000","last_change":"2024-09-13T17:50:34.201272+0000","last_active":"2024-09-13T18:21:02.673198+0000","last_peered":"2024-09-13T18:21:02.673198+0000","last_clean":"2024-09-13T18:21:02.673198+0000","last_became_active":"2024-09-13T17:50:34.200936+0000","last_became_peered":"2024-09-13T17:50:34.200936+0000","last_unstale":"2024-09-13T18:21:02.673198+0000","last_undegraded":"2024-09-13T18:21:02.673198+0000","last_fullsized":"2024-09-13T18:21:02.673198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:52.094879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'6502","reported_seq":6528,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.900747+0000","last_change":"2024-09-13T17:50:38.615229+0000","last_active":"2024-09-13T18:21:02.900747+0000","last_peered":"2024-09-13T18:21:02.900747+0000","last_clean":"2024-09-13T18:21:02.900747+0000","last_became_active":"2024-09-13T17:50:38.614739+0000","last_became_peered":"2024-09-13T17:50:38.614739+0000","last_unstale":"2024-09-13T18:21:02.900747+0000","last_undegraded":"2024-09-13T18:21:02.900747+0000","last_fullsized":"2024-09-13T18:21:02.900747+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6502,"log_dups_size":0,"ondisk_log_size":6502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:18.619529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27696,"num_write_kb":544723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'20","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.287949+0000","last_change":"2024-09-13T17:50:34.588848+0000","last_active":"2024-09-13T18:21:13.287949+0000","last_peered":"2024-09-13T18:21:13.287949+0000","last_clean":"2024-09-13T18:21:13.287949+0000","last_became_active":"2024-09-13T17:50:34.588565+0000","last_became_peered":"2024-09-13T17:50:34.588565+0000","last_unstale":"2024-09-13T18:21:13.287949+0000","last_undegraded":"2024-09-13T18:21:13.287949+0000","last_fullsized":"2024-09-13T18:21:13.287949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:47.927865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'7","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.707815+0000","last_change":"2024-09-13T17:50:34.199008+0000","last_active":"2024-09-13T18:21:02.707815+0000","last_peered":"2024-09-13T18:21:02.707815+0000","last_clean":"2024-09-13T18:21:02.707815+0000","last_became_active":"2024-09-13T17:50:34.197839+0000","last_became_peered":"2024-09-13T17:50:34.197839+0000","last_unstale":"2024-09-13T18:21:02.707815+0000","last_undegraded":"2024-09-13T18:21:02.707815+0000","last_fullsized":"2024-09-13T18:21:02.707815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:51.430710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'6326","reported_seq":6352,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:01.698688+0000","last_change":"2024-09-13T17:50:38.611080+0000","last_active":"2024-09-13T18:21:01.698688+0000","last_peered":"2024-09-13T18:21:01.698688+0000","last_clean":"2024-09-13T18:21:01.698688+0000","last_became_active":"2024-09-13T17:50:38.610688+0000","last_became_peered":"2024-09-13T17:50:38.610688+0000","last_unstale":"2024-09-13T18:21:01.698688+0000","last_undegraded":"2024-09-13T18:21:01.698688+0000","last_fullsized":"2024-09-13T18:21:01.698688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6326,"log_dups_size":0,"ondisk_log_size":6326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:28.527617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27199,"num_write_kb":538670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'29","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317546+0000","last_change":"2024-09-13T17:50:34.594763+0000","last_active":"2024-09-13T18:21:13.317546+0000","last_peered":"2024-09-13T18:21:13.317546+0000","last_clean":"2024-09-13T18:21:13.317546+0000","last_became_active":"2024-09-13T17:50:34.594471+0000","last_became_peered":"2024-09-13T17:50:34.594471+0000","last_unstale":"2024-09-13T18:21:13.317546+0000","last_undegraded":"2024-09-13T18:21:13.317546+0000","last_fullsized":"2024-09-13T18:21:13.317546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:20:58.718271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'6","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.839570+0000","last_change":"2024-09-13T17:50:34.209432+0000","last_active":"2024-09-13T18:21:02.839570+0000","last_peered":"2024-09-13T18:21:02.839570+0000","last_clean":"2024-09-13T18:21:02.839570+0000","last_became_active":"2024-09-13T17:50:34.208817+0000","last_became_peered":"2024-09-13T17:50:34.208817+0000","last_unstale":"2024-09-13T18:21:02.839570+0000","last_undegraded":"2024-09-13T18:21:02.839570+0000","last_fullsized":"2024-09-13T18:21:02.839570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:49.587528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'5788","reported_seq":5815,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.947982+0000","last_change":"2024-09-13T17:50:38.609461+0000","last_active":"2024-09-13T18:21:02.947982+0000","last_peered":"2024-09-13T18:21:02.947982+0000","last_clean":"2024-09-13T18:21:02.947982+0000","last_became_active":"2024-09-13T17:50:38.609128+0000","last_became_peered":"2024-09-13T17:50:38.609128+0000","last_unstale":"2024-09-13T18:21:02.947982+0000","last_undegraded":"2024-09-13T18:21:02.947982+0000","last_fullsized":"2024-09-13T18:21:02.947982+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5788,"log_dups_size":0,"ondisk_log_size":5788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:02.286825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24140,"num_write_kb":538739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'24","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.305222+0000","last_change":"2024-09-13T17:50:34.592970+0000","last_active":"2024-09-13T18:21:13.305222+0000","last_peered":"2024-09-13T18:21:13.305222+0000","last_clean":"2024-09-13T18:21:13.305222+0000","last_became_active":"2024-09-13T17:50:34.592669+0000","last_became_peered":"2024-09-13T17:50:34.592669+0000","last_unstale":"2024-09-13T18:21:13.305222+0000","last_undegraded":"2024-09-13T18:21:13.305222+0000","last_fullsized":"2024-09-13T18:21:13.305222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:52.031261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'15","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.785525+0000","last_change":"2024-09-13T17:50:34.200798+0000","last_active":"2024-09-13T18:21:02.785525+0000","last_peered":"2024-09-13T18:21:02.785525+0000","last_clean":"2024-09-13T18:21:02.785525+0000","last_became_active":"2024-09-13T17:50:34.200529+0000","last_became_peered":"2024-09-13T17:50:34.200529+0000","last_unstale":"2024-09-13T18:21:02.785525+0000","last_undegraded":"2024-09-13T18:21:02.785525+0000","last_fullsized":"2024-09-13T18:21:02.785525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:26.334296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'5464","reported_seq":5490,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.598966+0000","last_change":"2024-09-13T17:50:38.606287+0000","last_active":"2024-09-13T18:21:02.598966+0000","last_peered":"2024-09-13T18:21:02.598966+0000","last_clean":"2024-09-13T18:21:02.598966+0000","last_became_active":"2024-09-13T17:50:38.605826+0000","last_became_peered":"2024-09-13T17:50:38.605826+0000","last_unstale":"2024-09-13T18:21:02.598966+0000","last_undegraded":"2024-09-13T18:21:02.598966+0000","last_fullsized":"2024-09-13T18:21:02.598966+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5464,"log_dups_size":0,"ondisk_log_size":5464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:18.709454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23044,"num_write_kb":472739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'18","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310869+0000","last_change":"2024-09-13T17:50:34.591579+0000","last_active":"2024-09-13T18:21:13.310869+0000","last_peered":"2024-09-13T18:21:13.310869+0000","last_clean":"2024-09-13T18:21:13.310869+0000","last_became_active":"2024-09-13T17:50:34.586165+0000","last_became_peered":"2024-09-13T17:50:34.586165+0000","last_unstale":"2024-09-13T18:21:13.310869+0000","last_undegraded":"2024-09-13T18:21:13.310869+0000","last_fullsized":"2024-09-13T18:21:13.310869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:56:19.205176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'417","reported_seq":447,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.705527+0000","last_change":"2024-09-13T17:50:34.203218+0000","last_active":"2024-09-13T18:21:02.705527+0000","last_peered":"2024-09-13T18:21:02.705527+0000","last_clean":"2024-09-13T18:21:02.705527+0000","last_became_active":"2024-09-13T17:50:34.202948+0000","last_became_peered":"2024-09-13T17:50:34.202948+0000","last_unstale":"2024-09-13T18:21:02.705527+0000","last_undegraded":"2024-09-13T18:21:02.705527+0000","last_fullsized":"2024-09-13T18:21:02.705527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:46.535569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":4324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'5887","reported_seq":5914,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.956090+0000","last_change":"2024-09-13T17:50:38.610597+0000","last_active":"2024-09-13T18:21:02.956090+0000","last_peered":"2024-09-13T18:21:02.956090+0000","last_clean":"2024-09-13T18:21:02.956090+0000","last_became_active":"2024-09-13T17:50:38.610055+0000","last_became_peered":"2024-09-13T17:50:38.610055+0000","last_unstale":"2024-09-13T18:21:02.956090+0000","last_undegraded":"2024-09-13T18:21:02.956090+0000","last_fullsized":"2024-09-13T18:21:02.956090+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5887,"log_dups_size":0,"ondisk_log_size":5887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:55.754936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24664,"num_write_kb":495799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'21","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317972+0000","last_change":"2024-09-13T17:50:34.593963+0000","last_active":"2024-09-13T18:21:13.317972+0000","last_peered":"2024-09-13T18:21:13.317972+0000","last_clean":"2024-09-13T18:21:13.317972+0000","last_became_active":"2024-09-13T17:50:34.593454+0000","last_became_peered":"2024-09-13T17:50:34.593454+0000","last_unstale":"2024-09-13T18:21:13.317972+0000","last_undegraded":"2024-09-13T18:21:13.317972+0000","last_fullsized":"2024-09-13T18:21:13.317972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:18.883521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'67","reported_seq":96,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.464353+0000","last_change":"2024-09-13T17:50:34.207936+0000","last_active":"2024-09-13T18:21:02.464353+0000","last_peered":"2024-09-13T18:21:02.464353+0000","last_clean":"2024-09-13T18:21:02.464353+0000","last_became_active":"2024-09-13T17:50:34.207605+0000","last_became_peered":"2024-09-13T17:50:34.207605+0000","last_unstale":"2024-09-13T18:21:02.464353+0000","last_undegraded":"2024-09-13T18:21:02.464353+0000","last_fullsized":"2024-09-13T18:21:02.464353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:05.494406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'5937","reported_seq":5964,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.965268+0000","last_change":"2024-09-13T17:50:38.616297+0000","last_active":"2024-09-13T18:21:02.965268+0000","last_peered":"2024-09-13T18:21:02.965268+0000","last_clean":"2024-09-13T18:21:02.965268+0000","last_became_active":"2024-09-13T17:50:38.614165+0000","last_became_peered":"2024-09-13T17:50:38.614165+0000","last_unstale":"2024-09-13T18:21:02.965268+0000","last_undegraded":"2024-09-13T18:21:02.965268+0000","last_fullsized":"2024-09-13T18:21:02.965268+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5937,"log_dups_size":0,"ondisk_log_size":5937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:11.357026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25111,"num_write_kb":509795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'25","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.311085+0000","last_change":"2024-09-13T17:50:34.583001+0000","last_active":"2024-09-13T18:21:13.311085+0000","last_peered":"2024-09-13T18:21:13.311085+0000","last_clean":"2024-09-13T18:21:13.311085+0000","last_became_active":"2024-09-13T17:50:34.582529+0000","last_became_peered":"2024-09-13T17:50:34.582529+0000","last_unstale":"2024-09-13T18:21:13.311085+0000","last_undegraded":"2024-09-13T18:21:13.311085+0000","last_fullsized":"2024-09-13T18:21:13.311085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:05.287181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'5","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:53.035777+0000","last_change":"2024-09-13T17:50:34.205138+0000","last_active":"2024-09-13T18:20:53.035777+0000","last_peered":"2024-09-13T18:20:53.035777+0000","last_clean":"2024-09-13T18:20:53.035777+0000","last_became_active":"2024-09-13T17:50:34.203360+0000","last_became_peered":"2024-09-13T17:50:34.203360+0000","last_unstale":"2024-09-13T18:20:53.035777+0000","last_undegraded":"2024-09-13T18:20:53.035777+0000","last_fullsized":"2024-09-13T18:20:53.035777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:35.756963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'5934","reported_seq":5961,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.811068+0000","last_change":"2024-09-13T17:50:38.611036+0000","last_active":"2024-09-13T18:21:02.811068+0000","last_peered":"2024-09-13T18:21:02.811068+0000","last_clean":"2024-09-13T18:21:02.811068+0000","last_became_active":"2024-09-13T17:50:38.610692+0000","last_became_peered":"2024-09-13T17:50:38.610692+0000","last_unstale":"2024-09-13T18:21:02.811068+0000","last_undegraded":"2024-09-13T18:21:02.811068+0000","last_fullsized":"2024-09-13T18:21:02.811068+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5934,"log_dups_size":0,"ondisk_log_size":5934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:10.147998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25100,"num_write_kb":497535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'23","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.311752+0000","last_change":"2024-09-13T17:51:16.365295+0000","last_active":"2024-09-13T18:21:13.311752+0000","last_peered":"2024-09-13T18:21:13.311752+0000","last_clean":"2024-09-13T18:21:13.311752+0000","last_became_active":"2024-09-13T17:51:16.364437+0000","last_became_peered":"2024-09-13T17:51:16.364437+0000","last_unstale":"2024-09-13T18:21:13.311752+0000","last_undegraded":"2024-09-13T18:21:13.311752+0000","last_fullsized":"2024-09-13T18:21:13.311752+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:23.208899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"27'10","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.645634+0000","last_change":"2024-09-13T17:50:34.204376+0000","last_active":"2024-09-13T18:21:02.645634+0000","last_peered":"2024-09-13T18:21:02.645634+0000","last_clean":"2024-09-13T18:21:02.645634+0000","last_became_active":"2024-09-13T17:50:34.204003+0000","last_became_peered":"2024-09-13T17:50:34.204003+0000","last_unstale":"2024-09-13T18:21:02.645634+0000","last_undegraded":"2024-09-13T18:21:02.645634+0000","last_fullsized":"2024-09-13T18:21:02.645634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:14.023990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'5363","reported_seq":5390,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.796875+0000","last_change":"2024-09-13T17:50:38.606144+0000","last_active":"2024-09-13T18:21:02.796875+0000","last_peered":"2024-09-13T18:21:02.796875+0000","last_clean":"2024-09-13T18:21:02.796875+0000","last_became_active":"2024-09-13T17:50:38.605646+0000","last_became_peered":"2024-09-13T17:50:38.605646+0000","last_unstale":"2024-09-13T18:21:02.796875+0000","last_undegraded":"2024-09-13T18:21:02.796875+0000","last_fullsized":"2024-09-13T18:21:02.796875+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5363,"log_dups_size":0,"ondisk_log_size":5363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:07.120075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22831,"num_write_kb":508903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'6067","reported_seq":6096,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.803568+0000","last_change":"2024-09-13T17:50:38.618855+0000","last_active":"2024-09-13T18:21:02.803568+0000","last_peered":"2024-09-13T18:21:02.803568+0000","last_clean":"2024-09-13T18:21:02.803568+0000","last_became_active":"2024-09-13T17:50:38.618449+0000","last_became_peered":"2024-09-13T17:50:38.618449+0000","last_unstale":"2024-09-13T18:21:02.803568+0000","last_undegraded":"2024-09-13T18:21:02.803568+0000","last_fullsized":"2024-09-13T18:21:02.803568+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6067,"log_dups_size":0,"ondisk_log_size":6067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:29:01.114087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25532,"num_write_kb":568825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'21","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.298787+0000","last_change":"2024-09-13T17:50:34.590079+0000","last_active":"2024-09-13T18:21:13.298787+0000","last_peered":"2024-09-13T18:21:13.298787+0000","last_clean":"2024-09-13T18:21:13.298787+0000","last_became_active":"2024-09-13T17:50:34.589396+0000","last_became_peered":"2024-09-13T17:50:34.589396+0000","last_unstale":"2024-09-13T18:21:13.298787+0000","last_undegraded":"2024-09-13T18:21:13.298787+0000","last_fullsized":"2024-09-13T18:21:13.298787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:03.308341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:17:11.375959+0000","last_change":"2024-09-13T17:50:34.206575+0000","last_active":"2024-09-13T18:17:11.375959+0000","last_peered":"2024-09-13T18:17:11.375959+0000","last_clean":"2024-09-13T18:17:11.375959+0000","last_became_active":"2024-09-13T17:50:34.206308+0000","last_became_peered":"2024-09-13T17:50:34.206308+0000","last_unstale":"2024-09-13T18:17:11.375959+0000","last_undegraded":"2024-09-13T18:17:11.375959+0000","last_fullsized":"2024-09-13T18:17:11.375959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:34.766538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'5881","reported_seq":5909,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.927672+0000","last_change":"2024-09-13T17:50:38.618131+0000","last_active":"2024-09-13T18:21:02.927672+0000","last_peered":"2024-09-13T18:21:02.927672+0000","last_clean":"2024-09-13T18:21:02.927672+0000","last_became_active":"2024-09-13T17:50:38.617611+0000","last_became_peered":"2024-09-13T17:50:38.617611+0000","last_unstale":"2024-09-13T18:21:02.927672+0000","last_undegraded":"2024-09-13T18:21:02.927672+0000","last_fullsized":"2024-09-13T18:21:02.927672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5881,"log_dups_size":0,"ondisk_log_size":5881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:30.746445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25019,"num_write_kb":530139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'27","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.307795+0000","last_change":"2024-09-13T17:50:34.582153+0000","last_active":"2024-09-13T18:21:13.307795+0000","last_peered":"2024-09-13T18:21:13.307795+0000","last_clean":"2024-09-13T18:21:13.307795+0000","last_became_active":"2024-09-13T17:50:34.581708+0000","last_became_peered":"2024-09-13T17:50:34.581708+0000","last_unstale":"2024-09-13T18:21:13.307795+0000","last_undegraded":"2024-09-13T18:21:13.307795+0000","last_fullsized":"2024-09-13T18:21:13.307795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:22.238810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'4","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:19:47.928146+0000","last_change":"2024-09-13T17:50:34.205489+0000","last_active":"2024-09-13T18:19:47.928146+0000","last_peered":"2024-09-13T18:19:47.928146+0000","last_clean":"2024-09-13T18:19:47.928146+0000","last_became_active":"2024-09-13T17:50:34.205202+0000","last_became_peered":"2024-09-13T17:50:34.205202+0000","last_unstale":"2024-09-13T18:19:47.928146+0000","last_undegraded":"2024-09-13T18:19:47.928146+0000","last_fullsized":"2024-09-13T18:19:47.928146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:54.450276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'6","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.825727+0000","last_change":"2024-09-13T17:50:34.202470+0000","last_active":"2024-09-13T18:21:02.825727+0000","last_peered":"2024-09-13T18:21:02.825727+0000","last_clean":"2024-09-13T18:21:02.825727+0000","last_became_active":"2024-09-13T17:50:34.202163+0000","last_became_peered":"2024-09-13T17:50:34.202163+0000","last_unstale":"2024-09-13T18:21:02.825727+0000","last_undegraded":"2024-09-13T18:21:02.825727+0000","last_fullsized":"2024-09-13T18:21:02.825727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:03.472932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'18","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312647+0000","last_change":"2024-09-13T17:50:34.590947+0000","last_active":"2024-09-13T18:21:13.312647+0000","last_peered":"2024-09-13T18:21:13.312647+0000","last_clean":"2024-09-13T18:21:13.312647+0000","last_became_active":"2024-09-13T17:50:34.590497+0000","last_became_peered":"2024-09-13T17:50:34.590497+0000","last_unstale":"2024-09-13T18:21:13.312647+0000","last_undegraded":"2024-09-13T18:21:13.312647+0000","last_fullsized":"2024-09-13T18:21:13.312647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:56:27.848092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'4651","reported_seq":4677,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.602408+0000","last_change":"2024-09-13T17:50:38.618518+0000","last_active":"2024-09-13T18:21:02.602408+0000","last_peered":"2024-09-13T18:21:02.602408+0000","last_clean":"2024-09-13T18:21:02.602408+0000","last_became_active":"2024-09-13T17:50:38.618108+0000","last_became_peered":"2024-09-13T17:50:38.618108+0000","last_unstale":"2024-09-13T18:21:02.602408+0000","last_undegraded":"2024-09-13T18:21:02.602408+0000","last_fullsized":"2024-09-13T18:21:02.602408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4651,"log_dups_size":0,"ondisk_log_size":4651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:42.260237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19488,"num_write_kb":413684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'690","reported_seq":718,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.791718+0000","last_change":"2024-09-13T17:50:34.218459+0000","last_active":"2024-09-13T18:21:02.791718+0000","last_peered":"2024-09-13T18:21:02.791718+0000","last_clean":"2024-09-13T18:21:02.791718+0000","last_became_active":"2024-09-13T17:50:34.218087+0000","last_became_peered":"2024-09-13T17:50:34.218087+0000","last_unstale":"2024-09-13T18:21:02.791718+0000","last_undegraded":"2024-09-13T18:21:02.791718+0000","last_fullsized":"2024-09-13T18:21:02.791718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:56.914544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195461,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'20","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.315199+0000","last_change":"2024-09-13T17:50:34.583845+0000","last_active":"2024-09-13T18:21:13.315199+0000","last_peered":"2024-09-13T18:21:13.315199+0000","last_clean":"2024-09-13T18:21:13.315199+0000","last_became_active":"2024-09-13T17:50:34.583283+0000","last_became_peered":"2024-09-13T17:50:34.583283+0000","last_unstale":"2024-09-13T18:21:13.315199+0000","last_undegraded":"2024-09-13T18:21:13.315199+0000","last_fullsized":"2024-09-13T18:21:13.315199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:55.630285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'5883","reported_seq":5909,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.716917+0000","last_change":"2024-09-13T17:50:38.611987+0000","last_active":"2024-09-13T18:21:02.716917+0000","last_peered":"2024-09-13T18:21:02.716917+0000","last_clean":"2024-09-13T18:21:02.716917+0000","last_became_active":"2024-09-13T17:50:38.611636+0000","last_became_peered":"2024-09-13T17:50:38.611636+0000","last_unstale":"2024-09-13T18:21:02.716917+0000","last_undegraded":"2024-09-13T18:21:02.716917+0000","last_fullsized":"2024-09-13T18:21:02.716917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5883,"log_dups_size":0,"ondisk_log_size":5883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:41.111562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24816,"num_write_kb":545450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'25","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.300796+0000","last_change":"2024-09-13T17:50:34.588927+0000","last_active":"2024-09-13T18:21:13.300796+0000","last_peered":"2024-09-13T18:21:13.300796+0000","last_clean":"2024-09-13T18:21:13.300796+0000","last_became_active":"2024-09-13T17:50:34.588651+0000","last_became_peered":"2024-09-13T17:50:34.588651+0000","last_unstale":"2024-09-13T18:21:13.300796+0000","last_undegraded":"2024-09-13T18:21:13.300796+0000","last_fullsized":"2024-09-13T18:21:13.300796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:49.429670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'24","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310371+0000","last_change":"2024-09-13T17:50:34.583795+0000","last_active":"2024-09-13T18:21:13.310371+0000","last_peered":"2024-09-13T18:21:13.310371+0000","last_clean":"2024-09-13T18:21:13.310371+0000","last_became_active":"2024-09-13T17:50:34.583493+0000","last_became_peered":"2024-09-13T17:50:34.583493+0000","last_unstale":"2024-09-13T18:21:13.310371+0000","last_undegraded":"2024-09-13T18:21:13.310371+0000","last_fullsized":"2024-09-13T18:21:13.310371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:30.463932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'25","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.314410+0000","last_change":"2024-09-13T17:50:34.587624+0000","last_active":"2024-09-13T18:21:13.314410+0000","last_peered":"2024-09-13T18:21:13.314410+0000","last_clean":"2024-09-13T18:21:13.314410+0000","last_became_active":"2024-09-13T17:50:34.587119+0000","last_became_peered":"2024-09-13T17:50:34.587119+0000","last_unstale":"2024-09-13T18:21:13.314410+0000","last_undegraded":"2024-09-13T18:21:13.314410+0000","last_fullsized":"2024-09-13T18:21:13.314410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:16.783899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'17","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.308557+0000","last_change":"2024-09-13T17:50:34.581923+0000","last_active":"2024-09-13T18:21:13.308557+0000","last_peered":"2024-09-13T18:21:13.308557+0000","last_clean":"2024-09-13T18:21:13.308557+0000","last_became_active":"2024-09-13T17:50:34.581628+0000","last_became_peered":"2024-09-13T17:50:34.581628+0000","last_unstale":"2024-09-13T18:21:13.308557+0000","last_undegraded":"2024-09-13T18:21:13.308557+0000","last_fullsized":"2024-09-13T18:21:13.308557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:28:55.936864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'14","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312666+0000","last_change":"2024-09-13T17:50:34.595037+0000","last_active":"2024-09-13T18:21:13.312666+0000","last_peered":"2024-09-13T18:21:13.312666+0000","last_clean":"2024-09-13T18:21:13.312666+0000","last_became_active":"2024-09-13T17:50:34.594857+0000","last_became_peered":"2024-09-13T17:50:34.594857+0000","last_unstale":"2024-09-13T18:21:13.312666+0000","last_undegraded":"2024-09-13T18:21:13.312666+0000","last_fullsized":"2024-09-13T18:21:13.312666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:11.420413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'15","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.298148+0000","last_change":"2024-09-13T17:50:34.589651+0000","last_active":"2024-09-13T18:21:13.298148+0000","last_peered":"2024-09-13T18:21:13.298148+0000","last_clean":"2024-09-13T18:21:13.298148+0000","last_became_active":"2024-09-13T17:50:34.589354+0000","last_became_peered":"2024-09-13T17:50:34.589354+0000","last_unstale":"2024-09-13T18:21:13.298148+0000","last_undegraded":"2024-09-13T18:21:13.298148+0000","last_fullsized":"2024-09-13T18:21:13.298148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:31.294048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'24","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317654+0000","last_change":"2024-09-13T17:50:34.597949+0000","last_active":"2024-09-13T18:21:13.317654+0000","last_peered":"2024-09-13T18:21:13.317654+0000","last_clean":"2024-09-13T18:21:13.317654+0000","last_became_active":"2024-09-13T17:50:34.597621+0000","last_became_peered":"2024-09-13T17:50:34.597621+0000","last_unstale":"2024-09-13T18:21:13.317654+0000","last_undegraded":"2024-09-13T18:21:13.317654+0000","last_fullsized":"2024-09-13T18:21:13.317654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:23.133462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'15","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312343+0000","last_change":"2024-09-13T17:50:34.589893+0000","last_active":"2024-09-13T18:21:13.312343+0000","last_peered":"2024-09-13T18:21:13.312343+0000","last_clean":"2024-09-13T18:21:13.312343+0000","last_became_active":"2024-09-13T17:50:34.589626+0000","last_became_peered":"2024-09-13T17:50:34.589626+0000","last_unstale":"2024-09-13T18:21:13.312343+0000","last_undegraded":"2024-09-13T18:21:13.312343+0000","last_fullsized":"2024-09-13T18:21:13.312343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:41.321025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'16","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.308839+0000","last_change":"2024-09-13T17:50:34.590046+0000","last_active":"2024-09-13T18:21:13.308839+0000","last_peered":"2024-09-13T18:21:13.308839+0000","last_clean":"2024-09-13T18:21:13.308839+0000","last_became_active":"2024-09-13T17:50:34.589757+0000","last_became_peered":"2024-09-13T17:50:34.589757+0000","last_unstale":"2024-09-13T18:21:13.308839+0000","last_undegraded":"2024-09-13T18:21:13.308839+0000","last_fullsized":"2024-09-13T18:21:13.308839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:28.239550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'23","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312608+0000","last_change":"2024-09-13T17:50:34.595518+0000","last_active":"2024-09-13T18:21:13.312608+0000","last_peered":"2024-09-13T18:21:13.312608+0000","last_clean":"2024-09-13T18:21:13.312608+0000","last_became_active":"2024-09-13T17:50:34.595242+0000","last_became_peered":"2024-09-13T17:50:34.595242+0000","last_unstale":"2024-09-13T18:21:13.312608+0000","last_undegraded":"2024-09-13T18:21:13.312608+0000","last_fullsized":"2024-09-13T18:21:13.312608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:26.149384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'21","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.315503+0000","last_change":"2024-09-13T17:50:34.593183+0000","last_active":"2024-09-13T18:21:13.315503+0000","last_peered":"2024-09-13T18:21:13.315503+0000","last_clean":"2024-09-13T18:21:13.315503+0000","last_became_active":"2024-09-13T17:50:34.592893+0000","last_became_peered":"2024-09-13T17:50:34.592893+0000","last_unstale":"2024-09-13T18:21:13.315503+0000","last_undegraded":"2024-09-13T18:21:13.315503+0000","last_fullsized":"2024-09-13T18:21:13.315503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:19.459505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'29","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.311869+0000","last_change":"2024-09-13T17:50:34.594679+0000","last_active":"2024-09-13T18:21:13.311869+0000","last_peered":"2024-09-13T18:21:13.311869+0000","last_clean":"2024-09-13T18:21:13.311869+0000","last_became_active":"2024-09-13T17:50:34.593750+0000","last_became_peered":"2024-09-13T17:50:34.593750+0000","last_unstale":"2024-09-13T18:21:13.311869+0000","last_undegraded":"2024-09-13T18:21:13.311869+0000","last_fullsized":"2024-09-13T18:21:13.311869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:02.135577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'19","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.313398+0000","last_change":"2024-09-13T17:50:34.598736+0000","last_active":"2024-09-13T18:21:13.313398+0000","last_peered":"2024-09-13T18:21:13.313398+0000","last_clean":"2024-09-13T18:21:13.313398+0000","last_became_active":"2024-09-13T17:50:34.598433+0000","last_became_peered":"2024-09-13T17:50:34.598433+0000","last_unstale":"2024-09-13T18:21:13.313398+0000","last_undegraded":"2024-09-13T18:21:13.313398+0000","last_fullsized":"2024-09-13T18:21:13.313398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:08.975139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'22","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.301045+0000","last_change":"2024-09-13T17:50:34.591795+0000","last_active":"2024-09-13T18:21:13.301045+0000","last_peered":"2024-09-13T18:21:13.301045+0000","last_clean":"2024-09-13T18:21:13.301045+0000","last_became_active":"2024-09-13T17:50:34.591286+0000","last_became_peered":"2024-09-13T17:50:34.591286+0000","last_unstale":"2024-09-13T18:21:13.301045+0000","last_undegraded":"2024-09-13T18:21:13.301045+0000","last_fullsized":"2024-09-13T18:21:13.301045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:31.580247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'28","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.300800+0000","last_change":"2024-09-13T17:50:34.587774+0000","last_active":"2024-09-13T18:21:13.300800+0000","last_peered":"2024-09-13T18:21:13.300800+0000","last_clean":"2024-09-13T18:21:13.300800+0000","last_became_active":"2024-09-13T17:50:34.587506+0000","last_became_peered":"2024-09-13T17:50:34.587506+0000","last_unstale":"2024-09-13T18:21:13.300800+0000","last_undegraded":"2024-09-13T18:21:13.300800+0000","last_fullsized":"2024-09-13T18:21:13.300800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:46.275528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'22","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.318554+0000","last_change":"2024-09-13T17:50:34.596100+0000","last_active":"2024-09-13T18:21:13.318554+0000","last_peered":"2024-09-13T18:21:13.318554+0000","last_clean":"2024-09-13T18:21:13.318554+0000","last_became_active":"2024-09-13T17:50:34.595798+0000","last_became_peered":"2024-09-13T17:50:34.595798+0000","last_unstale":"2024-09-13T18:21:13.318554+0000","last_undegraded":"2024-09-13T18:21:13.318554+0000","last_fullsized":"2024-09-13T18:21:13.318554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:49.543845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'21","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306399+0000","last_change":"2024-09-13T17:50:34.584818+0000","last_active":"2024-09-13T18:21:13.306399+0000","last_peered":"2024-09-13T18:21:13.306399+0000","last_clean":"2024-09-13T18:21:13.306399+0000","last_became_active":"2024-09-13T17:50:34.584516+0000","last_became_peered":"2024-09-13T17:50:34.584516+0000","last_unstale":"2024-09-13T18:21:13.306399+0000","last_undegraded":"2024-09-13T18:21:13.306399+0000","last_fullsized":"2024-09-13T18:21:13.306399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:52.402024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'23","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.292739+0000","last_change":"2024-09-13T17:50:34.598886+0000","last_active":"2024-09-13T18:21:13.292739+0000","last_peered":"2024-09-13T18:21:13.292739+0000","last_clean":"2024-09-13T18:21:13.292739+0000","last_became_active":"2024-09-13T17:50:34.598648+0000","last_became_peered":"2024-09-13T17:50:34.598648+0000","last_unstale":"2024-09-13T18:21:13.292739+0000","last_undegraded":"2024-09-13T18:21:13.292739+0000","last_fullsized":"2024-09-13T18:21:13.292739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:00:43.063223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'24","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.302458+0000","last_change":"2024-09-13T17:50:34.582649+0000","last_active":"2024-09-13T18:21:13.302458+0000","last_peered":"2024-09-13T18:21:13.302458+0000","last_clean":"2024-09-13T18:21:13.302458+0000","last_became_active":"2024-09-13T17:50:34.581699+0000","last_became_peered":"2024-09-13T17:50:34.581699+0000","last_unstale":"2024-09-13T18:21:13.302458+0000","last_undegraded":"2024-09-13T18:21:13.302458+0000","last_fullsized":"2024-09-13T18:21:13.302458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:18.959303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'23","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306620+0000","last_change":"2024-09-13T17:50:34.587484+0000","last_active":"2024-09-13T18:21:13.306620+0000","last_peered":"2024-09-13T18:21:13.306620+0000","last_clean":"2024-09-13T18:21:13.306620+0000","last_became_active":"2024-09-13T17:50:34.587217+0000","last_became_peered":"2024-09-13T17:50:34.587217+0000","last_unstale":"2024-09-13T18:21:13.306620+0000","last_undegraded":"2024-09-13T18:21:13.306620+0000","last_fullsized":"2024-09-13T18:21:13.306620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:24:19.172607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'18","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.315670+0000","last_change":"2024-09-13T17:50:34.592871+0000","last_active":"2024-09-13T18:21:13.315670+0000","last_peered":"2024-09-13T18:21:13.315670+0000","last_clean":"2024-09-13T18:21:13.315670+0000","last_became_active":"2024-09-13T17:50:34.588513+0000","last_became_peered":"2024-09-13T17:50:34.588513+0000","last_unstale":"2024-09-13T18:21:13.315670+0000","last_undegraded":"2024-09-13T18:21:13.315670+0000","last_fullsized":"2024-09-13T18:21:13.315670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:33.131420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'19","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.319025+0000","last_change":"2024-09-13T17:50:34.585713+0000","last_active":"2024-09-13T18:21:13.319025+0000","last_peered":"2024-09-13T18:21:13.319025+0000","last_clean":"2024-09-13T18:21:13.319025+0000","last_became_active":"2024-09-13T17:50:34.585438+0000","last_became_peered":"2024-09-13T17:50:34.585438+0000","last_unstale":"2024-09-13T18:21:13.319025+0000","last_undegraded":"2024-09-13T18:21:13.319025+0000","last_fullsized":"2024-09-13T18:21:13.319025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:56:49.905393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'30","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312596+0000","last_change":"2024-09-13T17:50:34.589628+0000","last_active":"2024-09-13T18:21:13.312596+0000","last_peered":"2024-09-13T18:21:13.312596+0000","last_clean":"2024-09-13T18:21:13.312596+0000","last_became_active":"2024-09-13T17:50:34.587665+0000","last_became_peered":"2024-09-13T17:50:34.587665+0000","last_unstale":"2024-09-13T18:21:13.312596+0000","last_undegraded":"2024-09-13T18:21:13.312596+0000","last_fullsized":"2024-09-13T18:21:13.312596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:15.518264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'27","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.321326+0000","last_change":"2024-09-13T17:50:34.591776+0000","last_active":"2024-09-13T18:21:13.321326+0000","last_peered":"2024-09-13T18:21:13.321326+0000","last_clean":"2024-09-13T18:21:13.321326+0000","last_became_active":"2024-09-13T17:50:34.591496+0000","last_became_peered":"2024-09-13T17:50:34.591496+0000","last_unstale":"2024-09-13T18:21:13.321326+0000","last_undegraded":"2024-09-13T18:21:13.321326+0000","last_fullsized":"2024-09-13T18:21:13.321326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:48.651999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'15","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310654+0000","last_change":"2024-09-13T17:51:16.282372+0000","last_active":"2024-09-13T18:21:13.310654+0000","last_peered":"2024-09-13T18:21:13.310654+0000","last_clean":"2024-09-13T18:21:13.310654+0000","last_became_active":"2024-09-13T17:51:16.282065+0000","last_became_peered":"2024-09-13T17:51:16.282065+0000","last_unstale":"2024-09-13T18:21:13.310654+0000","last_undegraded":"2024-09-13T18:21:13.310654+0000","last_fullsized":"2024-09-13T18:21:13.310654+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:49.291224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"27'14","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310971+0000","last_change":"2024-09-13T17:50:34.585531+0000","last_active":"2024-09-13T18:21:13.310971+0000","last_peered":"2024-09-13T18:21:13.310971+0000","last_clean":"2024-09-13T18:21:13.310971+0000","last_became_active":"2024-09-13T17:50:34.585226+0000","last_became_peered":"2024-09-13T17:50:34.585226+0000","last_unstale":"2024-09-13T18:21:13.310971+0000","last_undegraded":"2024-09-13T18:21:13.310971+0000","last_fullsized":"2024-09-13T18:21:13.310971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:17.105028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'17","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.314101+0000","last_change":"2024-09-13T17:50:34.586750+0000","last_active":"2024-09-13T18:21:13.314101+0000","last_peered":"2024-09-13T18:21:13.314101+0000","last_clean":"2024-09-13T18:21:13.314101+0000","last_became_active":"2024-09-13T17:50:34.584564+0000","last_became_peered":"2024-09-13T17:50:34.584564+0000","last_unstale":"2024-09-13T18:21:13.314101+0000","last_undegraded":"2024-09-13T18:21:13.314101+0000","last_fullsized":"2024-09-13T18:21:13.314101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:54.033257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'18","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.307217+0000","last_change":"2024-09-13T17:50:34.585938+0000","last_active":"2024-09-13T18:21:13.307217+0000","last_peered":"2024-09-13T18:21:13.307217+0000","last_clean":"2024-09-13T18:21:13.307217+0000","last_became_active":"2024-09-13T17:50:34.585584+0000","last_became_peered":"2024-09-13T17:50:34.585584+0000","last_unstale":"2024-09-13T18:21:13.307217+0000","last_undegraded":"2024-09-13T18:21:13.307217+0000","last_fullsized":"2024-09-13T18:21:13.307217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:32.623468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'15","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310571+0000","last_change":"2024-09-13T17:50:34.588762+0000","last_active":"2024-09-13T18:21:13.310571+0000","last_peered":"2024-09-13T18:21:13.310571+0000","last_clean":"2024-09-13T18:21:13.310571+0000","last_became_active":"2024-09-13T17:50:34.588489+0000","last_became_peered":"2024-09-13T17:50:34.588489+0000","last_unstale":"2024-09-13T18:21:13.310571+0000","last_undegraded":"2024-09-13T18:21:13.310571+0000","last_fullsized":"2024-09-13T18:21:13.310571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:38.266297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'18","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.302008+0000","last_change":"2024-09-13T17:50:34.583889+0000","last_active":"2024-09-13T18:21:13.302008+0000","last_peered":"2024-09-13T18:21:13.302008+0000","last_clean":"2024-09-13T18:21:13.302008+0000","last_became_active":"2024-09-13T17:50:34.583551+0000","last_became_peered":"2024-09-13T17:50:34.583551+0000","last_unstale":"2024-09-13T18:21:13.302008+0000","last_undegraded":"2024-09-13T18:21:13.302008+0000","last_fullsized":"2024-09-13T18:21:13.302008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:19.888626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'10","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.284491+0000","last_change":"2024-09-13T17:50:34.585617+0000","last_active":"2024-09-13T18:21:13.284491+0000","last_peered":"2024-09-13T18:21:13.284491+0000","last_clean":"2024-09-13T18:21:13.284491+0000","last_became_active":"2024-09-13T17:50:34.585340+0000","last_became_peered":"2024-09-13T17:50:34.585340+0000","last_unstale":"2024-09-13T18:21:13.284491+0000","last_undegraded":"2024-09-13T18:21:13.284491+0000","last_fullsized":"2024-09-13T18:21:13.284491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:18.331045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'17","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317071+0000","last_change":"2024-09-13T17:50:34.592365+0000","last_active":"2024-09-13T18:21:13.317071+0000","last_peered":"2024-09-13T18:21:13.317071+0000","last_clean":"2024-09-13T18:21:13.317071+0000","last_became_active":"2024-09-13T17:50:34.592046+0000","last_became_peered":"2024-09-13T17:50:34.592046+0000","last_unstale":"2024-09-13T18:21:13.317071+0000","last_undegraded":"2024-09-13T18:21:13.317071+0000","last_fullsized":"2024-09-13T18:21:13.317071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:54.283683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":751482,"num_write_kb":16106434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178546,"ondisk_log_size":178546,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1271,"ondisk_log_size":1271,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":15941984,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":121,"num_read_kb":179,"num_write":3371,"num_write_kb":16874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32030720,"data_stored":31883968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3124,"ondisk_log_size":3124,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509812,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1768076,"kb_used_data":8580,"kb_used_omap":1,"kb_used_meta":1759486,"kb_avail":92603764,"statfs":{"total":96636764160,"available":94826254336,"internally_reserved":0,"allocated":8785920,"data_stored":8600393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":1801714120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":2.246,"5min":5.9219999999999997,"15min":2.669},"min":{"1min":0.19,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":8.5329999999999995,"5min":41.494999999999997,"15min":50.511000000000003},"last":0.63},{"interface":"front","average":{"1min":2.6070000000000002,"5min":3.4350000000000001,"15min":1.6080000000000001},"min":{"1min":0.23100000000000001,"5min":0.193,"15min":0.193},"max":{"1min":18.760000000000002,"5min":18.760000000000002,"15min":18.760000000000002},"last":0.56999999999999995}]},{"osd":1,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":3.4969999999999999,"15min":1.8009999999999999},"min":{"1min":0.34000000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":6.25,"5min":19.626000000000001,"15min":19.626000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":3.4430000000000001,"5min":4.101,"15min":2.0379999999999998},"min":{"1min":0.29599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":22.126999999999999,"5min":22.126999999999999,"15min":22.126999999999999},"last":0.71999999999999997}]},{"osd":2,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.907,"15min":1.296},"min":{"1min":0.29399999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":7.4610000000000003,"5min":22.010000000000002,"15min":22.010000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.49,"5min":2.1800000000000002,"15min":1.4830000000000001},"min":{"1min":0.38400000000000001,"5min":0.309,"15min":0.309},"max":{"1min":7.4850000000000003,"5min":24.170999999999999,"15min":24.170999999999999},"last":0.74399999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":2.0409999999999999,"15min":1.466},"min":{"1min":0.33200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":5.593,"5min":24.359000000000002,"15min":24.359000000000002},"last":0.87},{"interface":"front","average":{"1min":1.8109999999999999,"5min":2.3410000000000002,"15min":1.4570000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":10.132999999999999,"5min":15.539999999999999,"15min":15.539999999999999},"last":1.0069999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":2.5840000000000001,"15min":1.6160000000000001},"min":{"1min":0.35699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":8.5739999999999998,"5min":49.838000000000001,"15min":49.838000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.96999999999999997,"5min":2.5369999999999999,"15min":1.605},"min":{"1min":0.43099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":4.3810000000000002,"5min":41.591999999999999,"15min":41.591999999999999},"last":0.81100000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":2.23,"5min":2.5579999999999998,"15min":1.7270000000000001},"min":{"1min":0.46100000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":8.6029999999999998,"5min":24.457000000000001,"15min":24.457000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.677,"5min":2.2919999999999998,"15min":1.6200000000000001},"min":{"1min":0.42399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":7.4290000000000003,"5min":24.387,"15min":24.387},"last":1.1319999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.869,"15min":1.3740000000000001},"min":{"1min":0.505,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":4.2750000000000004,"5min":14.138999999999999,"15min":14.138999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.155,"5min":1.97,"15min":1.4219999999999999},"min":{"1min":0.44900000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":4.4320000000000004,"5min":14.013,"15min":14.013},"last":0.99299999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509814,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1055244,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":1054846,"kb_avail":93316596,"statfs":{"total":96636764160,"available":95556194304,"internally_reserved":0,"allocated":397312,"data_stored":210684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1080162758},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":2.0819999999999999,"5min":2.0739999999999998,"15min":1.0980000000000001},"min":{"1min":0.17499999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":9.9280000000000008,"5min":17.971,"15min":17.971},"last":0.45900000000000002},{"interface":"front","average":{"1min":2.1469999999999998,"5min":2.1240000000000001,"15min":1.177},"min":{"1min":0.247,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":9.9700000000000006,"5min":18.001999999999999,"15min":18.001999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":2.694,"15min":1.677},"min":{"1min":0.26100000000000001,"5min":0.217,"15min":0.217},"max":{"1min":3.871,"5min":24.599,"15min":56.939999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.4710000000000001,"5min":2.9350000000000001,"15min":1.7649999999999999},"min":{"1min":0.23200000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":9.0169999999999995,"5min":24.75,"15min":57.006999999999998},"last":0.61899999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":3.5219999999999998,"5min":3.0790000000000002,"15min":1.5980000000000001},"min":{"1min":0.30499999999999999,"5min":0.218,"15min":0.218},"max":{"1min":16.111999999999998,"5min":16.707000000000001,"15min":16.707000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":3.0169999999999999,"5min":2.6349999999999998,"15min":1.5569999999999999},"min":{"1min":0.309,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":12.449999999999999,"5min":16.640999999999998,"15min":16.640999999999998},"last":0.69399999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.52,"15min":1.238},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":8.2260000000000009,"5min":8.2829999999999995,"15min":14.177},"last":1.073},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.284,"15min":1.159},"min":{"1min":0.33900000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":8.1980000000000004,"5min":8.1980000000000004,"15min":14.396000000000001},"last":1.0589999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.77600000000000002,"15min":1.0009999999999999},"min":{"1min":0.379,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.2250000000000001,"5min":4.1020000000000003,"15min":14.153},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.181,"5min":1.224,"15min":1.137},"min":{"1min":0.32800000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":7.476,"5min":7.476,"15min":14.375999999999999},"last":0.82099999999999995}]},{"osd":6,"last update":"Fri Sep 13 18:20:30 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.1519999999999999,"15min":1.1020000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":6.8730000000000002,"5min":8.3010000000000002,"15min":8.3010000000000002},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.125,"5min":1.196,"15min":1.2150000000000001},"min":{"1min":0.46700000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":6.9500000000000002,"5min":8.3320000000000007,"15min":14.074},"last":0.996}]},{"osd":7,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.163,"15min":1.0780000000000001},"min":{"1min":0.40400000000000003,"5min":0.37,"15min":0.37},"max":{"1min":4.1079999999999997,"5min":8.0410000000000004,"15min":8.0410000000000004},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.238,"15min":1.1930000000000001},"min":{"1min":0.37,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.9900000000000002,"5min":6.5300000000000002,"15min":14.307},"last":1.111}]}]},{"osd":1,"up_from":15,"seq":64424509813,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1738228,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":1737918,"kb_avail":92633612,"statfs":{"total":96636764160,"available":94856818688,"internally_reserved":0,"allocated":307200,"data_stored":116868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1779628486},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.9119999999999999,"5min":3.7890000000000001,"15min":1.891},"min":{"1min":0.23999999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":9.9879999999999995,"5min":21.949999999999999,"15min":21.949999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.4610000000000001,"5min":2.863,"15min":1.5860000000000001},"min":{"1min":0.219,"5min":0.193,"15min":0.193},"max":{"1min":7.8730000000000002,"5min":23.338999999999999,"15min":23.338999999999999},"last":0.58999999999999997}]},{"osd":2,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":2.5699999999999998,"15min":1.4950000000000001},"min":{"1min":0.311,"5min":0.22,"15min":0.22},"max":{"1min":6.6399999999999997,"5min":14.869999999999999,"15min":14.869999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.4390000000000001,"5min":2.383,"15min":1.4059999999999999},"min":{"1min":0.23599999999999999,"5min":0.223,"15min":0.223},"max":{"1min":6.5300000000000002,"5min":15.058,"15min":15.058},"last":0.83499999999999996}]},{"osd":3,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":2.323,"5min":6.8940000000000001,"15min":3.1539999999999999},"min":{"1min":0.314,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":8.2390000000000008,"5min":41.369999999999997,"15min":41.369999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":2.8769999999999998,"5min":6.2549999999999999,"15min":2.8900000000000001},"min":{"1min":0.27900000000000003,"5min":0.216,"15min":0.216},"max":{"1min":8.8360000000000003,"5min":36.439,"15min":40.325000000000003},"last":0.63300000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.978,"5min":3.9390000000000001,"15min":2.1070000000000002},"min":{"1min":0.30399999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":8.5980000000000008,"5min":103.44499999999999,"15min":103.44499999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":2.1850000000000001,"5min":3.3199999999999998,"15min":1.889},"min":{"1min":0.312,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":8.2789999999999999,"5min":40.953000000000003,"15min":40.953000000000003},"last":0.95899999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:20:20 2024","interfaces":[{"interface":"back","average":{"1min":1.9750000000000001,"5min":2.661,"15min":1.726},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":9.8970000000000002,"5min":18.954000000000001,"15min":18.954000000000001},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.431,"5min":2.0840000000000001,"15min":1.516},"min":{"1min":0.31,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":7.383,"5min":15.413,"15min":15.413},"last":1.0429999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":2.4089999999999998,"15min":1.569},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":7.4199999999999999,"5min":15.462999999999999,"15min":15.462999999999999},"last":1.244},{"interface":"front","average":{"1min":1.6719999999999999,"5min":2.5190000000000001,"15min":1.663},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":7.4429999999999996,"5min":15.49,"15min":15.49},"last":1.3009999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4470000000000001,"5min":1.764,"15min":1.375},"min":{"1min":0.33600000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":6.6159999999999997,"5min":15.093999999999999,"15min":15.093999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.494,"5min":1.8380000000000001,"15min":1.4590000000000001},"min":{"1min":0.41899999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":6.6920000000000002,"5min":15.119,"15min":15.119},"last":1.27}]}]},{"osd":0,"up_from":15,"seq":64424509815,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1247072,"kb_used_data":4056,"kb_used_omap":1,"kb_used_meta":1243006,"kb_avail":93124768,"statfs":{"total":96636764160,"available":95359762432,"internally_reserved":0,"allocated":4153344,"data_stored":3967313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1272838598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":2.8570000000000002,"5min":3.2130000000000001,"15min":1.7050000000000001},"min":{"1min":0.182,"5min":0.182,"15min":0.182},"max":{"1min":17.693000000000001,"5min":23.004000000000001,"15min":24.202000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":2.9489999999999998,"5min":3.2690000000000001,"15min":1.8380000000000001},"min":{"1min":0.32100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":17.736000000000001,"5min":23.039999999999999,"15min":24.260999999999999},"last":0.876}]},{"osd":2,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.897,"15min":1.1060000000000001},"min":{"1min":0.20399999999999999,"5min":0.186,"15min":0.186},"max":{"1min":6.4320000000000004,"5min":17.859000000000002,"15min":17.859000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.77,"5min":2.0920000000000001,"15min":1.3180000000000001},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":8.6199999999999992,"5min":19.678999999999998,"15min":19.678999999999998},"last":0.78900000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":2.2330000000000001,"5min":4.0970000000000004,"15min":1.9630000000000001},"min":{"1min":0.23699999999999999,"5min":0.219,"15min":0.219},"max":{"1min":11.378,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":3.6720000000000002,"5min":3.27,"15min":1.8129999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":13.736000000000001,"5min":15.856999999999999,"15min":15.856999999999999},"last":0.92400000000000004}]},{"osd":4,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.661,"15min":1.258},"min":{"1min":0.34699999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":8.2829999999999995,"5min":8.7010000000000005,"15min":8.7010000000000005},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.1719999999999999,"5min":2.657,"15min":1.548},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":6.3090000000000002,"5min":16.291,"15min":16.291},"last":1.044}]},{"osd":5,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":2.0960000000000001,"5min":1.974,"15min":1.3260000000000001},"min":{"1min":0.312,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":8.4800000000000004,"5min":15.18,"15min":15.18},"last":1.079},{"interface":"front","average":{"1min":1.7689999999999999,"5min":1.9850000000000001,"15min":1.4119999999999999},"min":{"1min":0.34699999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":8.5559999999999992,"5min":17.852,"15min":17.852},"last":1.135}]},{"osd":6,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":3.1669999999999998,"5min":2.3860000000000001,"15min":1.5600000000000001},"min":{"1min":0.39200000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":15.467000000000001,"5min":15.467000000000001,"15min":15.467000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.7290000000000001,"5min":1.823,"15min":1.4139999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":10.359,"5min":17.908000000000001,"15min":17.908000000000001},"last":1.278}]},{"osd":7,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":1.909,"5min":1.887,"15min":1.381},"min":{"1min":0.40100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":8.2550000000000008,"5min":18.986999999999998,"15min":18.986999999999998},"last":1.242},{"interface":"front","average":{"1min":1.952,"5min":1.9710000000000001,"15min":1.47},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.373},"max":{"1min":8.3279999999999994,"5min":19.021000000000001,"15min":19.021000000000001},"last":1.1699999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542517,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1207524,"kb_used_data":12252,"kb_used_omap":1,"kb_used_meta":1195262,"kb_avail":93164316,"statfs":{"total":96636764160,"available":95400259584,"internally_reserved":0,"allocated":12546048,"data_stored":12357112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":1223948747},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":2.0619999999999998,"15min":1.246},"min":{"1min":0.245,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":5.367,"5min":35.195,"15min":35.195},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.2529999999999999,"5min":2.206,"15min":1.391},"min":{"1min":0.34000000000000002,"5min":0.315,"15min":0.315},"max":{"1min":5.4269999999999996,"5min":35.253999999999998,"15min":35.253999999999998},"last":0.92900000000000005}]},{"osd":1,"last update":"Fri Sep 13 18:20:27 2024","interfaces":[{"interface":"back","average":{"1min":2.4940000000000002,"5min":2.6930000000000001,"15min":1.4730000000000001},"min":{"1min":0.27100000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":13.058999999999999,"5min":19.585000000000001,"15min":19.585000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":2.4060000000000001,"5min":2.6680000000000001,"15min":1.635},"min":{"1min":0.33400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":17.398,"5min":19.617000000000001,"15min":19.617000000000001},"last":0.74199999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":1.6100000000000001,"15min":1.143},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":7.1299999999999999,"5min":15.789999999999999,"15min":15.789999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.5569999999999999,"15min":1.161},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.9379999999999997,"5min":12.965,"15min":12.965},"last":1.1950000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":3.427,"15min":1.8700000000000001},"min":{"1min":0.36299999999999999,"5min":0.316,"15min":0.316},"max":{"1min":5.4370000000000003,"5min":40.948999999999998,"15min":40.948999999999998},"last":0.98199999999999998},{"interface":"front","average":{"1min":2.0739999999999998,"5min":3.198,"15min":1.6870000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":20.024999999999999,"5min":25.234000000000002,"15min":25.234000000000002},"last":1.115}]},{"osd":4,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1739999999999999,"15min":0.99399999999999999},"min":{"1min":0.31,"5min":0.252,"15min":0.252},"max":{"1min":4.5220000000000002,"5min":8.8670000000000009,"15min":8.8670000000000009},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.321,"15min":1.181},"min":{"1min":0.38500000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":4.383,"5min":10.279999999999999,"15min":10.279999999999999},"last":0.79900000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.123,"15min":0.98499999999999999},"min":{"1min":0.34300000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":4.4790000000000001,"5min":8.1699999999999999,"15min":8.1699999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.98999999999999999,"15min":1.0289999999999999},"min":{"1min":0.33100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":3.0489999999999999,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":1.004}]},{"osd":6,"last update":"Fri Sep 13 18:20:19 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.4750000000000001,"15min":1.2190000000000001},"min":{"1min":0.34599999999999997,"5min":0.309,"15min":0.309},"max":{"1min":6.7960000000000003,"5min":8.2219999999999995,"15min":8.2219999999999995},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.508,"15min":1.2549999999999999},"min":{"1min":0.434,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":7.048,"5min":8.2469999999999999,"15min":8.2469999999999999},"last":1.27}]}]},{"osd":6,"up_from":14,"seq":60129542517,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1419436,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":1415038,"kb_avail":92952404,"statfs":{"total":96636764160,"available":95183261696,"internally_reserved":0,"allocated":4493312,"data_stored":4311530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1448999367},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":2.0329999999999999,"15min":1.21},"min":{"1min":0.23699999999999999,"5min":0.214,"15min":0.214},"max":{"1min":3.9470000000000001,"5min":12.808999999999999,"15min":12.808999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.091,"5min":1.7030000000000001,"15min":1.161},"min":{"1min":0.31,"5min":0.218,"15min":0.218},"max":{"1min":5.7670000000000003,"5min":12.308999999999999,"15min":12.308999999999999},"last":0.80500000000000005}]},{"osd":1,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":2.0470000000000002,"5min":2.27,"15min":1.371},"min":{"1min":0.34799999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":5.8380000000000001,"5min":14.199999999999999,"15min":14.199999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.621,"5min":2.339,"15min":1.413},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.7990000000000004,"5min":14.260999999999999,"15min":14.260999999999999},"last":1.109}]},{"osd":2,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.6839999999999999,"5min":1.821,"15min":1.2110000000000001},"min":{"1min":0.29999999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":6.0359999999999996,"5min":10.792999999999999,"15min":10.792999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.742,"5min":1.8759999999999999,"15min":1.371},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":6.0750000000000002,"5min":10.826000000000001,"15min":10.826000000000001},"last":1.252}]},{"osd":3,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":3.379,"5min":5.0670000000000002,"15min":2.4660000000000002},"min":{"1min":0.35799999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":10.324999999999999,"5min":84.867999999999995,"15min":84.867999999999995},"last":0.93600000000000005},{"interface":"front","average":{"1min":2.5449999999999999,"5min":3.5779999999999998,"15min":1.8979999999999999},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":10.231,"5min":84.343999999999994,"15min":84.343999999999994},"last":1.01}]},{"osd":4,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":2.0739999999999998,"5min":1.6990000000000001,"15min":1.2070000000000001},"min":{"1min":0.39200000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":9.5649999999999995,"5min":11.044,"15min":11.044},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.294,"15min":1.113},"min":{"1min":0.38400000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":9.4719999999999995,"5min":9.4719999999999995,"15min":9.4719999999999995},"last":0.97599999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2689999999999999,"15min":1.1559999999999999},"min":{"1min":0.376,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.0939999999999999,"5min":8.7880000000000003,"15min":8.7880000000000003},"last":1.151},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.149,"15min":1.1000000000000001},"min":{"1min":0.41499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.1709999999999998,"5min":7.569,"15min":7.569},"last":1.222}]},{"osd":7,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.1739999999999999,"15min":1.181},"min":{"1min":0.434,"5min":0.32900000000000001,"15min":0.29999999999999999},"max":{"1min":2.8029999999999999,"5min":7.1820000000000004,"15min":7.1820000000000004},"last":1.335},{"interface":"front","average":{"1min":1.133,"5min":1.294,"15min":1.256},"min":{"1min":0.53700000000000003,"5min":0.34499999999999997,"15min":0.33500000000000002},"max":{"1min":2.871,"5min":7.2039999999999997,"15min":7.2039999999999997},"last":1.379}]}]},{"osd":5,"up_from":14,"seq":60129542519,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":978544,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":978238,"kb_avail":93393296,"statfs":{"total":96636764160,"available":95634735104,"internally_reserved":0,"allocated":303104,"data_stored":117302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1001716166},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.889,"15min":1.161},"min":{"1min":0.27500000000000002,"5min":0.218,"15min":0.218},"max":{"1min":7.3650000000000002,"5min":10.148999999999999,"15min":10.148999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.9550000000000001,"15min":1.3220000000000001},"min":{"1min":0.34200000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":7.4029999999999996,"5min":10.185,"15min":10.185},"last":0.96999999999999997}]},{"osd":1,"last update":"Fri Sep 13 18:20:27 2024","interfaces":[{"interface":"back","average":{"1min":1.879,"5min":1.897,"15min":1.1699999999999999},"min":{"1min":0.29799999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":10.08,"5min":10.436999999999999,"15min":10.436999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":2.0099999999999998,"5min":2.3460000000000001,"15min":1.385},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":18.584,"5min":19.388000000000002,"15min":19.388000000000002},"last":1.2609999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.875,"5min":1.5329999999999999,"15min":1.0980000000000001},"min":{"1min":0.33900000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":10.183,"5min":12.667,"15min":12.667},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.032,"5min":1.6439999999999999,"15min":1.198},"min":{"1min":0.377,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":5.7469999999999999,"5min":34.905999999999999,"15min":34.905999999999999},"last":1.119}]},{"osd":3,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":3.2999999999999998,"5min":4.9039999999999999,"15min":2.286},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":19.831,"5min":28.295000000000002,"15min":28.295000000000002},"last":0.81399999999999995},{"interface":"front","average":{"1min":3.9260000000000002,"5min":3.1459999999999999,"15min":1.7210000000000001},"min":{"1min":0.39700000000000002,"5min":0.318,"15min":0.318},"max":{"1min":19.731000000000002,"5min":19.731000000000002,"15min":19.731000000000002},"last":1.0149999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.744,"5min":1.0760000000000001,"15min":0.96199999999999997},"min":{"1min":0.308,"5min":0.255,"15min":0.255},"max":{"1min":10.023999999999999,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":2.3650000000000002,"5min":1.571,"15min":1.22},"min":{"1min":0.39400000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":10.210000000000001,"5min":10.210000000000001,"15min":10.210000000000001},"last":1.075}]},{"osd":6,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.9530000000000001,"5min":1.373,"15min":1.131},"min":{"1min":0.437,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":6.6849999999999996,"5min":6.6849999999999996,"15min":6.6849999999999996},"last":1.179},{"interface":"front","average":{"1min":2.012,"5min":1.3200000000000001,"15min":1.2},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":10.151,"5min":10.151,"15min":10.151},"last":1.2130000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":1.2330000000000001,"15min":1.143},"min":{"1min":0.42599999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":6.2910000000000004,"5min":7.9329999999999998,"15min":7.9329999999999998},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.71,"5min":1.107,"15min":1.1519999999999999},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":5.976,"5min":5.976,"15min":5.976},"last":1.3620000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542517,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1301036,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":1296638,"kb_avail":93070804,"statfs":{"total":96636764160,"available":95304503296,"internally_reserved":0,"allocated":4493312,"data_stored":4310894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1327757766},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":2.1499999999999999,"15min":1.2829999999999999},"min":{"1min":0.26200000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":6.984,"5min":28.439,"15min":28.439},"last":0.92200000000000004},{"interface":"front","average":{"1min":2.1949999999999998,"5min":2.4079999999999999,"15min":1.4430000000000001},"min":{"1min":0.309,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":16.943000000000001,"5min":28.422000000000001,"15min":28.422000000000001},"last":0.748}]},{"osd":1,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":3.7789999999999999,"5min":5.5380000000000003,"15min":2.577},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":12.124000000000001,"5min":38.895000000000003,"15min":38.895000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":3.2069999999999999,"5min":4.0599999999999996,"15min":1.976},"min":{"1min":0.30199999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":12.026999999999999,"5min":32.670000000000002,"15min":32.670000000000002},"last":0.52600000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.72,"15min":1.1599999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":4.085,"5min":25.167999999999999,"15min":25.167999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":2.0150000000000001,"15min":1.321},"min":{"1min":0.373,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":4.1150000000000002,"5min":25.218,"15min":25.218},"last":1.1439999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":2.6419999999999999,"5min":3.0129999999999999,"15min":1.7250000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":11.441000000000001,"5min":43.945999999999998,"15min":43.945999999999998},"last":1.008},{"interface":"front","average":{"1min":3.2610000000000001,"5min":3.9329999999999998,"15min":2.052},"min":{"1min":0.39200000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":29.640000000000001,"5min":29.640000000000001,"15min":29.640000000000001},"last":0.81399999999999995}]},{"osd":5,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.2050000000000001,"15min":1.069},"min":{"1min":0.28199999999999997,"5min":0.255,"15min":0.255},"max":{"1min":12.282999999999999,"5min":12.282999999999999,"15min":12.282999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.3160000000000001,"15min":1.1739999999999999},"min":{"1min":0.36699999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":16.995999999999999,"5min":16.995999999999999,"15min":16.995999999999999},"last":0.98099999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":2.0430000000000001,"5min":2.0979999999999999,"15min":1.387},"min":{"1min":0.30499999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":12.42,"5min":23.010999999999999,"15min":23.010999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.504,"5min":1.23,"15min":1.181},"min":{"1min":0.40000000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":6.6040000000000001,"5min":7.9489999999999998,"15min":7.9489999999999998},"last":1.1919999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:20:13 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.448,"15min":1.27},"min":{"1min":0.30499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":4.0179999999999998,"5min":20.417000000000002,"15min":20.417000000000002},"last":1.236},{"interface":"front","average":{"1min":1.359,"5min":1.4019999999999999,"15min":1.284},"min":{"1min":0.41199999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":12.308999999999999,"5min":12.308999999999999,"15min":12.308999999999999},"last":1.0629999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3276800,"data_stored":3261021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":94760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8503296,"data_stored":8484469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4194970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4195606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":11669504,"data_stored":11650820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:21:20.761 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:21:20.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.873+0000 7fe66889a700 1 Processor -- start 2024-09-13T18:21:20.882 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.873+0000 7fe66889a700 1 -- start start 2024-09-13T18:21:20.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.877+0000 7fe66889a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe65c04f370 0x7fe65c04eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:20.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.877+0000 7fe66889a700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe65c007410 0x7fe65c04f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:20.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.877+0000 7fe66889a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe65c0df5d0 con 0x7fe65c049210 2024-09-13T18:21:20.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.877+0000 7fe66889a700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe65c036fe0 con 0x7fe65c007410 2024-09-13T18:21:20.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.877+0000 7fe66889a700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe65c04fc90 con 0x7fe65c04f370 2024-09-13T18:21:20.886 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.877+0000 7fe662ffd700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe65c007410 0x7fe65c04f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:20.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.877+0000 7fe663fff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe65c049210 0x7fe65c0e1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:53532/0 (socket says 172.21.15.40:53532) 2024-09-13T18:21:20.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.877+0000 7fe663fff700 1 -- 172.21.15.40:0/1777028412 learned_addr learned my addr 172.21.15.40:0/1777028412 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:20.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.877+0000 7fe6637fe700 1 --2- 172.21.15.40:0/1777028412 >> v2:172.21.15.43:3300/0 conn(0x7fe65c04f370 0x7fe65c04eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:20.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.877+0000 7fe662ffd700 1 -- 172.21.15.40:0/1777028412 >> v2:172.21.15.43:3300/0 conn(0x7fe65c04f370 msgr2=0x7fe65c04eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:20.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.877+0000 7fe662ffd700 1 --2- 172.21.15.40:0/1777028412 >> v2:172.21.15.43:3300/0 conn(0x7fe65c04f370 0x7fe65c04eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.877+0000 7fe662ffd700 1 -- 172.21.15.40:0/1777028412 >> v1:172.21.15.40:6789/0 conn(0x7fe65c049210 legacy=0x7fe65c0e1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:20.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.877+0000 7fe662ffd700 1 -- 172.21.15.40:0/1777028412 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65c0d8d60 con 0x7fe65c007410 2024-09-13T18:21:20.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.877+0000 7fe662ffd700 1 --2- 172.21.15.40:0/1777028412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe65c007410 0x7fe65c04f750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00ee80 tx=0x7fe650009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7b3e3d02799279 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:20.887 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.877+0000 7fe6627fc700 1 -- 172.21.15.40:0/1777028412 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650011830 con 0x7fe65c007410 2024-09-13T18:21:20.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.877+0000 7fe6627fc700 1 -- 172.21.15.40:0/1777028412 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe650011990 con 0x7fe65c007410 2024-09-13T18:21:20.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.877+0000 7fe6627fc700 1 -- 172.21.15.40:0/1777028412 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650011d20 con 0x7fe65c007410 2024-09-13T18:21:20.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe66889a700 1 -- 172.21.15.40:0/1777028412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe65c007410 msgr2=0x7fe65c04f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:20.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe66889a700 1 --2- 172.21.15.40:0/1777028412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe65c007410 0x7fe65c04f750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00ee80 tx=0x7fe650009b30 comp rx=0 tx=0).stop 2024-09-13T18:21:20.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe66889a700 1 -- 172.21.15.40:0/1777028412 shutdown_connections 2024-09-13T18:21:20.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe66889a700 1 --2- 172.21.15.40:0/1777028412 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe65c007410 0x7fe65c04f750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe66889a700 1 --2- 172.21.15.40:0/1777028412 >> v2:172.21.15.43:3300/0 conn(0x7fe65c04f370 0x7fe65c04eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe66889a700 1 -- 172.21.15.40:0/1777028412 >> 172.21.15.40:0/1777028412 conn(0x7fe65c00e940 msgr2=0x7fe65c00fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:20.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe66889a700 1 -- 172.21.15.40:0/1777028412 shutdown_connections 2024-09-13T18:21:20.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe66889a700 1 -- 172.21.15.40:0/1777028412 wait complete. 2024-09-13T18:21:20.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe66889a700 1 Processor -- start 2024-09-13T18:21:20.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe66889a700 1 -- start start 2024-09-13T18:21:20.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe66889a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe65c007410 0x7fe65c0dfa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:20.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe66889a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe65c049210 0x7fe65c0e0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:20.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe66889a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe65c04f370 0x7fe65c0d9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:20.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe66889a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c0ddc00 con 0x7fe65c007410 2024-09-13T18:21:20.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe66889a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c0de460 con 0x7fe65c049210 2024-09-13T18:21:20.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe66889a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c0ddea0 con 0x7fe65c04f370 2024-09-13T18:21:20.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe6637fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe65c007410 0x7fe65c0dfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:20.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe6637fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe65c007410 0x7fe65c0dfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48584/0 (socket says 172.21.15.40:48584) 2024-09-13T18:21:20.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe6637fe700 1 -- 172.21.15.40:0/1152128610 learned_addr learned my addr 172.21.15.40:0/1152128610 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe663fff700 1 --2- 172.21.15.40:0/1152128610 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe65c04f370 0x7fe65c0d9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe662ffd700 1 --2- 172.21.15.40:0/1152128610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe65c049210 0x7fe65c0e0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe6637fe700 1 -- 172.21.15.40:0/1152128610 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe65c04f370 msgr2=0x7fe65c0d9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe6637fe700 1 --2- 172.21.15.40:0/1152128610 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe65c04f370 0x7fe65c0d9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe6637fe700 1 -- 172.21.15.40:0/1152128610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe65c049210 msgr2=0x7fe65c0e0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe6637fe700 1 --2- 172.21.15.40:0/1152128610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe65c049210 0x7fe65c0e0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe6637fe700 1 -- 172.21.15.40:0/1152128610 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6500094b0 con 0x7fe65c007410 2024-09-13T18:21:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.881+0000 7fe662ffd700 1 --2- 172.21.15.40:0/1152128610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe65c049210 0x7fe65c0e0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.885+0000 7fe6637fe700 1 --2- 172.21.15.40:0/1152128610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe65c007410 0x7fe65c0dfa80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe65800e9d0 tx=0x7fe65800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.885+0000 7fe64ffff700 1 -- 172.21.15.40:0/1152128610 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658009720 con 0x7fe65c007410 2024-09-13T18:21:20.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.885+0000 7fe64ffff700 1 -- 172.21.15.40:0/1152128610 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65800c850 con 0x7fe65c007410 2024-09-13T18:21:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.885+0000 7fe66889a700 1 -- 172.21.15.40:0/1152128610 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65c0de180 con 0x7fe65c007410 2024-09-13T18:21:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.885+0000 7fe64ffff700 1 -- 172.21.15.40:0/1152128610 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65800cbc0 con 0x7fe65c007410 2024-09-13T18:21:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.885+0000 7fe66889a700 1 -- 172.21.15.40:0/1152128610 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65c0e09f0 con 0x7fe65c007410 2024-09-13T18:21:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.885+0000 7fe66889a700 1 -- 172.21.15.40:0/1152128610 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe65c04f150 con 0x7fe65c007410 2024-09-13T18:21:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.885+0000 7fe64ffff700 1 -- 172.21.15.40:0/1152128610 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe65800cd20 con 0x7fe65c007410 2024-09-13T18:21:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.889+0000 7fe64ffff700 1 --2- 172.21.15.40:0/1152128610 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe648066760 0x7fe648068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.889+0000 7fe64ffff700 1 -- 172.21.15.40:0/1152128610 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe658084580 con 0x7fe65c007410 2024-09-13T18:21:20.898 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.889+0000 7fe662ffd700 1 --2- 172.21.15.40:0/1152128610 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe648066760 0x7fe648068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.889+0000 7fe662ffd700 1 --2- 172.21.15.40:0/1152128610 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe648066760 0x7fe648068c20 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe650000c00 tx=0x7fe650008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:20.899 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:20.893+0000 7fe64ffff700 1 -- 172.21.15.40:0/1152128610 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe6580515c0 con 0x7fe65c007410 2024-09-13T18:21:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:21.085+0000 7fe66889a700 1 -- 172.21.15.40:0/1152128610 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe65c0497f0 con 0x7fe65c007410 2024-09-13T18:21:21.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:21.089+0000 7fe64ffff700 1 -- 172.21.15.40:0/1152128610 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18650 (secure 0 0 0) 0x7fe658054be0 con 0x7fe65c007410 2024-09-13T18:21:21.187 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T18:21:21.187 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":27,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:51:16.266176+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:50:32.740336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:50:33.552914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:50:36.623639+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:21:21.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:21.093+0000 7fe66889a700 1 -- 172.21.15.40:0/1152128610 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe648066760 msgr2=0x7fe648068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:21.093+0000 7fe66889a700 1 --2- 172.21.15.40:0/1152128610 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe648066760 0x7fe648068c20 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe650000c00 tx=0x7fe650008030 comp rx=0 tx=0).stop 2024-09-13T18:21:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:21.093+0000 7fe66889a700 1 -- 172.21.15.40:0/1152128610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe65c007410 msgr2=0x7fe65c0dfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:21.093+0000 7fe66889a700 1 --2- 172.21.15.40:0/1152128610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe65c007410 0x7fe65c0dfa80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe65800e9d0 tx=0x7fe65800ed90 comp rx=0 tx=0).stop 2024-09-13T18:21:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:21.093+0000 7fe66889a700 1 -- 172.21.15.40:0/1152128610 shutdown_connections 2024-09-13T18:21:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:21.093+0000 7fe66889a700 1 --2- 172.21.15.40:0/1152128610 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fe648066760 0x7fe648068c20 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:21.093+0000 7fe66889a700 1 --2- 172.21.15.40:0/1152128610 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe65c04f370 0x7fe65c0d9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:21.093+0000 7fe66889a700 1 --2- 172.21.15.40:0/1152128610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe65c049210 0x7fe65c0e0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:21.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:21.093+0000 7fe66889a700 1 --2- 172.21.15.40:0/1152128610 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe65c007410 0x7fe65c0dfa80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:21.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:21.093+0000 7fe66889a700 1 -- 172.21.15.40:0/1152128610 >> 172.21.15.40:0/1152128610 conn(0x7fe65c00e940 msgr2=0x7fe65c0d7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:21.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:21.093+0000 7fe66889a700 1 -- 172.21.15.40:0/1152128610 shutdown_connections 2024-09-13T18:21:21.190 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:21.093+0000 7fe66889a700 1 -- 172.21.15.40:0/1152128610 wait complete. 2024-09-13T18:21:22.191 INFO:tasks.ceph:Scrubbing osd.0 2024-09-13T18:21:22.192 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:21:22.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f0519ebc700 1 Processor -- start 2024-09-13T18:21:22.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f0519ebc700 1 -- start start 2024-09-13T18:21:22.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f0519ebc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f05140cbfb0 0x7f05140c4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f0519ebc700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051415a2c0 con 0x7f05140c3e90 2024-09-13T18:21:22.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f0519ebc700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05140ca310 con 0x7f05140c9e00 2024-09-13T18:21:22.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f0519ebc700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05140cfd90 con 0x7f05140cbfb0 2024-09-13T18:21:22.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f05196bb700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f05140cbfb0 0x7f05140c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f0518eba700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f05140c3e90 0x7f05140cb8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51408/0 (socket says 172.21.15.40:51408) 2024-09-13T18:21:22.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f0518eba700 1 -- 172.21.15.40:0/1933088825 learned_addr learned my addr 172.21.15.40:0/1933088825 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:22.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f05196bb700 1 -- 172.21.15.40:0/1933088825 >> v1:172.21.15.43:6789/0 conn(0x7f05140c9e00 legacy=0x7f0514161cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f05196bb700 1 -- 172.21.15.40:0/1933088825 >> v1:172.21.15.40:6789/0 conn(0x7f05140c3e90 legacy=0x7f05140cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f05196bb700 1 -- 172.21.15.40:0/1933088825 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05141596b0 con 0x7f05140cbfb0 2024-09-13T18:21:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f05137fe700 1 -- 172.21.15.40:0/1933088825 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2500800189 0 0) 0x7f051415a2c0 con 0x7f05140c3e90 2024-09-13T18:21:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f05196bb700 1 --2- 172.21.15.40:0/1933088825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05140cbfb0 0x7f05140c4270 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f050c009d90 tx=0x7f050c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9f2627208ac226 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f05137fe700 1 -- 172.21.15.40:0/1933088825 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c00e070 con 0x7f05140cbfb0 2024-09-13T18:21:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f05137fe700 1 -- 172.21.15.40:0/1933088825 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c00bbd0 con 0x7f05140cbfb0 2024-09-13T18:21:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f05137fe700 1 -- 172.21.15.40:0/1933088825 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c012430 con 0x7f05140cbfb0 2024-09-13T18:21:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f0519ebc700 1 -- 172.21.15.40:0/1933088825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05140cbfb0 msgr2=0x7f05140c4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f0519ebc700 1 --2- 172.21.15.40:0/1933088825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05140cbfb0 0x7f05140c4270 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f050c009d90 tx=0x7f050c00b330 comp rx=0 tx=0).stop 2024-09-13T18:21:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f0519ebc700 1 -- 172.21.15.40:0/1933088825 shutdown_connections 2024-09-13T18:21:22.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f0519ebc700 1 --2- 172.21.15.40:0/1933088825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05140cbfb0 0x7f05140c4270 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f0519ebc700 1 -- 172.21.15.40:0/1933088825 >> 172.21.15.40:0/1933088825 conn(0x7f0514066270 msgr2=0x7f0514066680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f0519ebc700 1 -- 172.21.15.40:0/1933088825 shutdown_connections 2024-09-13T18:21:22.314 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.305+0000 7f0519ebc700 1 -- 172.21.15.40:0/1933088825 wait complete. 2024-09-13T18:21:22.315 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f0519ebc700 1 Processor -- start 2024-09-13T18:21:22.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f0519ebc700 1 -- start start 2024-09-13T18:21:22.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f0519ebc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05140c3e90 0x7f051413dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f0519ebc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05140c9e00 0x7f05141e4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f0519ebc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05140cbfb0 0x7f05141f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f0519ebc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051413e0e0 con 0x7f05140cbfb0 2024-09-13T18:21:22.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f0519ebc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05141e4100 con 0x7f05140c9e00 2024-09-13T18:21:22.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f0519ebc700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05140b32d0 con 0x7f05140c3e90 2024-09-13T18:21:22.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f05196bb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05140cbfb0 0x7f05141f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f05196bb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05140cbfb0 0x7f05141f27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46928/0 (socket says 172.21.15.40:46928) 2024-09-13T18:21:22.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f05196bb700 1 -- 172.21.15.40:0/384243322 learned_addr learned my addr 172.21.15.40:0/384243322 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:22.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f0518eba700 1 --2- 172.21.15.40:0/384243322 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05140c3e90 0x7f051413dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.316 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f0513fff700 1 --2- 172.21.15.40:0/384243322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05140c9e00 0x7f05141e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f05196bb700 1 -- 172.21.15.40:0/384243322 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05140c3e90 msgr2=0x7f051413dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f05196bb700 1 --2- 172.21.15.40:0/384243322 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05140c3e90 0x7f051413dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f05196bb700 1 -- 172.21.15.40:0/384243322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05140c9e00 msgr2=0x7f05141e4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f05196bb700 1 --2- 172.21.15.40:0/384243322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05140c9e00 0x7f05141e4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f05196bb700 1 -- 172.21.15.40:0/384243322 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050c009a70 con 0x7f05140cbfb0 2024-09-13T18:21:22.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f05196bb700 1 --2- 172.21.15.40:0/384243322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05140cbfb0 0x7f05141f27f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f050c014040 tx=0x7f050c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:22.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f05117fa700 1 -- 172.21.15.40:0/384243322 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c01b810 con 0x7f05140cbfb0 2024-09-13T18:21:22.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f05117fa700 1 -- 172.21.15.40:0/384243322 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c020070 con 0x7f05140cbfb0 2024-09-13T18:21:22.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f0519ebc700 1 -- 172.21.15.40:0/384243322 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05141e4ad0 con 0x7f05140cbfb0 2024-09-13T18:21:22.317 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f05117fa700 1 -- 172.21.15.40:0/384243322 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c01bbb0 con 0x7f05140cbfb0 2024-09-13T18:21:22.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f0519ebc700 1 -- 172.21.15.40:0/384243322 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05140b3540 con 0x7f05140cbfb0 2024-09-13T18:21:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f0519ebc700 1 -- 172.21.15.40:0/384243322 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0514159f50 con 0x7f05140cbfb0 2024-09-13T18:21:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.309+0000 7f05117fa700 1 -- 172.21.15.40:0/384243322 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f050c01bd10 con 0x7f05140cbfb0 2024-09-13T18:21:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.313+0000 7f05117fa700 1 --2- 172.21.15.40:0/384243322 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f04f8066760 0x7f04f8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.313+0000 7f0518eba700 1 --2- 172.21.15.40:0/384243322 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f04f8066760 0x7f04f8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.313+0000 7f05117fa700 1 -- 172.21.15.40:0/384243322 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f050c022070 con 0x7f05140cbfb0 2024-09-13T18:21:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.313+0000 7f05117fa700 1 --2- 172.21.15.40:0/384243322 >> [v2:172.21.15.40:6816/3493711342,v1:172.21.15.40:6817/3493711342] conn(0x7f04f8070040 0x7f04f80724e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.313+0000 7f05117fa700 1 -- 172.21.15.40:0/384243322 --> [v2:172.21.15.40:6816/3493711342,v1:172.21.15.40:6817/3493711342] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f04f8072bb0 con 0x7f04f8070040 2024-09-13T18:21:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.313+0000 7f05117fa700 1 -- 172.21.15.40:0/384243322 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f050c088850 con 0x7f05140cbfb0 2024-09-13T18:21:22.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.313+0000 7f0513fff700 1 --2- 172.21.15.40:0/384243322 >> [v2:172.21.15.40:6816/3493711342,v1:172.21.15.40:6817/3493711342] conn(0x7f04f8070040 0x7f04f80724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.313+0000 7f0518eba700 1 --2- 172.21.15.40:0/384243322 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f04f8066760 0x7f04f8068c20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0508000f60 tx=0x7f0508007c40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:22.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.313+0000 7f0513fff700 1 --2- 172.21.15.40:0/384243322 >> [v2:172.21.15.40:6816/3493711342,v1:172.21.15.40:6817/3493711342] conn(0x7f04f8070040 0x7f04f80724e0 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:22.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.313+0000 7f05117fa700 1 -- 172.21.15.40:0/384243322 <== osd.0 v2:172.21.15.40:6816/3493711342 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f04f8072bb0 con 0x7f04f8070040 2024-09-13T18:21:22.337 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.333+0000 7f0519ebc700 1 -- 172.21.15.40:0/384243322 --> [v2:172.21.15.40:6816/3493711342,v1:172.21.15.40:6817/3493711342] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0514159c70 con 0x7f04f8070040 2024-09-13T18:21:22.338 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.333+0000 7f05117fa700 1 -- 172.21.15.40:0/384243322 <== osd.0 v2:172.21.15.40:6816/3493711342 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f05000050e0 con 0x7f04f8070040 2024-09-13T18:21:22.338 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:21:22.338 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:21:22.338 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:21:22.339 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.333+0000 7f0519ebc700 1 -- 172.21.15.40:0/384243322 >> [v2:172.21.15.40:6816/3493711342,v1:172.21.15.40:6817/3493711342] conn(0x7f04f8070040 msgr2=0x7f04f80724e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.333+0000 7f0519ebc700 1 --2- 172.21.15.40:0/384243322 >> [v2:172.21.15.40:6816/3493711342,v1:172.21.15.40:6817/3493711342] conn(0x7f04f8070040 0x7f04f80724e0 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.333+0000 7f0519ebc700 1 -- 172.21.15.40:0/384243322 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f04f8066760 msgr2=0x7f04f8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.333+0000 7f0519ebc700 1 --2- 172.21.15.40:0/384243322 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f04f8066760 0x7f04f8068c20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0508000f60 tx=0x7f0508007c40 comp rx=0 tx=0).stop 2024-09-13T18:21:22.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.333+0000 7f0519ebc700 1 -- 172.21.15.40:0/384243322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05140cbfb0 msgr2=0x7f05141f27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.333+0000 7f0519ebc700 1 --2- 172.21.15.40:0/384243322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05140cbfb0 0x7f05141f27f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f050c014040 tx=0x7f050c004af0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.333+0000 7f05196bb700 1 -- 172.21.15.40:0/384243322 reap_dead start 2024-09-13T18:21:22.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.333+0000 7f0519ebc700 1 -- 172.21.15.40:0/384243322 shutdown_connections 2024-09-13T18:21:22.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.333+0000 7f0519ebc700 1 -- 172.21.15.40:0/384243322 >> 172.21.15.40:0/384243322 conn(0x7f0514066270 msgr2=0x7f0514157650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:22.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.333+0000 7f0519ebc700 1 -- 172.21.15.40:0/384243322 shutdown_connections 2024-09-13T18:21:22.340 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.333+0000 7f0519ebc700 1 -- 172.21.15.40:0/384243322 wait complete. 2024-09-13T18:21:22.353 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-13T18:21:22.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78af59e700 1 Processor -- start 2024-09-13T18:21:22.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78af59e700 1 -- start start 2024-09-13T18:21:22.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78af59e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f78b0156b80 0x7f78b006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78af59e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f78b00b21a0 0x7f78b01595b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78af59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b015d8a0 con 0x7f78b006d070 2024-09-13T18:21:22.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78af59e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b006bde0 con 0x7f78b0156b80 2024-09-13T18:21:22.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78af59e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b006d4a0 con 0x7f78b00b21a0 2024-09-13T18:21:22.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78ae59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f78b0156b80 0x7f78b006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78ae59c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f78b0156b80 0x7f78b006e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46942/0 (socket says 172.21.15.40:46942) 2024-09-13T18:21:22.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78ae59c700 1 -- 172.21.15.40:0/1716340734 learned_addr learned my addr 172.21.15.40:0/1716340734 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:22.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78aed9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f78b006d070 0x7f78b00cd350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51414/0 (socket says 172.21.15.40:51414) 2024-09-13T18:21:22.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78add9b700 1 --2- 172.21.15.40:0/1716340734 >> v2:172.21.15.43:3300/0 conn(0x7f78b00b21a0 0x7f78b01595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78ae59c700 1 -- 172.21.15.40:0/1716340734 >> v2:172.21.15.43:3300/0 conn(0x7f78b00b21a0 msgr2=0x7f78b01595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:22.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78ae59c700 1 --2- 172.21.15.40:0/1716340734 >> v2:172.21.15.43:3300/0 conn(0x7f78b00b21a0 0x7f78b01595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78ae59c700 1 -- 172.21.15.40:0/1716340734 >> v1:172.21.15.40:6789/0 conn(0x7f78b006d070 legacy=0x7f78b00cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78ae59c700 1 -- 172.21.15.40:0/1716340734 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b006e1d0 con 0x7f78b0156b80 2024-09-13T18:21:22.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78ae59c700 1 --2- 172.21.15.40:0/1716340734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b0156b80 0x7f78b006e9a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f78b00c3de0 tx=0x7f78a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc046d461d3577ee server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:22.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78ad59a700 1 -- 172.21.15.40:0/1716340734 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a800e070 con 0x7f78b0156b80 2024-09-13T18:21:22.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78ad59a700 1 -- 172.21.15.40:0/1716340734 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a800b9c0 con 0x7f78b0156b80 2024-09-13T18:21:22.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78ad59a700 1 -- 172.21.15.40:0/1716340734 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a800bd70 con 0x7f78b0156b80 2024-09-13T18:21:22.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78af59e700 1 -- 172.21.15.40:0/1716340734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b0156b80 msgr2=0x7f78b006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78af59e700 1 --2- 172.21.15.40:0/1716340734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b0156b80 0x7f78b006e9a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f78b00c3de0 tx=0x7f78a800b330 comp rx=0 tx=0).stop 2024-09-13T18:21:22.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78af59e700 1 -- 172.21.15.40:0/1716340734 shutdown_connections 2024-09-13T18:21:22.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78af59e700 1 --2- 172.21.15.40:0/1716340734 >> v2:172.21.15.43:3300/0 conn(0x7f78b00b21a0 0x7f78b01595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78af59e700 1 --2- 172.21.15.40:0/1716340734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b0156b80 0x7f78b006e9a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78af59e700 1 -- 172.21.15.40:0/1716340734 >> 172.21.15.40:0/1716340734 conn(0x7f78b0072070 msgr2=0x7f78b0072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:22.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78af59e700 1 -- 172.21.15.40:0/1716340734 shutdown_connections 2024-09-13T18:21:22.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.457+0000 7f78af59e700 1 -- 172.21.15.40:0/1716340734 wait complete. 2024-09-13T18:21:22.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78af59e700 1 Processor -- start 2024-09-13T18:21:22.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78af59e700 1 -- start start 2024-09-13T18:21:22.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78af59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78b006d070 0x7f78b014fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78af59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b00b21a0 0x7f78b01ef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78af59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f78b0156b80 0x7f78b014d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78af59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b015c3a0 con 0x7f78b00b21a0 2024-09-13T18:21:22.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78af59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b00adcb0 con 0x7f78b006d070 2024-09-13T18:21:22.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78af59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b0150fc0 con 0x7f78b0156b80 2024-09-13T18:21:22.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78add9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b00b21a0 0x7f78b01ef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78add9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b00b21a0 0x7f78b01ef0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46946/0 (socket says 172.21.15.40:46946) 2024-09-13T18:21:22.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78add9b700 1 -- 172.21.15.40:0/3864008522 learned_addr learned my addr 172.21.15.40:0/3864008522 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:22.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78ae59c700 1 --2- 172.21.15.40:0/3864008522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78b006d070 0x7f78b014fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78add9b700 1 -- 172.21.15.40:0/3864008522 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f78b0156b80 msgr2=0x7f78b014d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78add9b700 1 --2- 172.21.15.40:0/3864008522 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f78b0156b80 0x7f78b014d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78add9b700 1 -- 172.21.15.40:0/3864008522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78b006d070 msgr2=0x7f78b014fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78add9b700 1 --2- 172.21.15.40:0/3864008522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78b006d070 0x7f78b014fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78add9b700 1 -- 172.21.15.40:0/3864008522 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a8009b30 con 0x7f78b00b21a0 2024-09-13T18:21:22.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78ae59c700 1 --2- 172.21.15.40:0/3864008522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78b006d070 0x7f78b014fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:22.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78add9b700 1 --2- 172.21.15.40:0/3864008522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b00b21a0 0x7f78b01ef0c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f78a000e810 tx=0x7f78a000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:22.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78a6ffd700 1 -- 172.21.15.40:0/3864008522 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0013610 con 0x7f78b00b21a0 2024-09-13T18:21:22.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78af59e700 1 -- 172.21.15.40:0/3864008522 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b01512a0 con 0x7f78b00b21a0 2024-09-13T18:21:22.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78af59e700 1 -- 172.21.15.40:0/3864008522 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b015c5f0 con 0x7f78b00b21a0 2024-09-13T18:21:22.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78a6ffd700 1 -- 172.21.15.40:0/3864008522 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a0013770 con 0x7f78b00b21a0 2024-09-13T18:21:22.470 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.461+0000 7f78a6ffd700 1 -- 172.21.15.40:0/3864008522 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0013b20 con 0x7f78b00b21a0 2024-09-13T18:21:22.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.465+0000 7f78af59e700 1 -- 172.21.15.40:0/3864008522 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7894005310 con 0x7f78b00b21a0 2024-09-13T18:21:22.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.465+0000 7f78a6ffd700 1 -- 172.21.15.40:0/3864008522 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f78a0013c80 con 0x7f78b00b21a0 2024-09-13T18:21:22.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.465+0000 7f78a6ffd700 1 --2- 172.21.15.40:0/3864008522 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7890066730 0x7f7890068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.465+0000 7f78a6ffd700 1 -- 172.21.15.40:0/3864008522 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f78a0084640 con 0x7f78b00b21a0 2024-09-13T18:21:22.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.465+0000 7f78ae59c700 1 --2- 172.21.15.40:0/3864008522 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7890066730 0x7f7890068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.472 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.465+0000 7f78ae59c700 1 --2- 172.21.15.40:0/3864008522 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7890066730 0x7f7890068bf0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f78a800b370 tx=0x7f78a8009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:22.476 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.469+0000 7f78a6ffd700 1 -- 172.21.15.40:0/3864008522 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f78a0051600 con 0x7f78b00b21a0 2024-09-13T18:21:22.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.637+0000 7f78af59e700 1 -- 172.21.15.40:0/3864008522 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f7894000bc0 con 0x7f7890066730 2024-09-13T18:21:22.643 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.637+0000 7f78a6ffd700 1 -- 172.21.15.40:0/3864008522 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7894000bc0 con 0x7f7890066730 2024-09-13T18:21:22.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.641+0000 7f78af59e700 1 -- 172.21.15.40:0/3864008522 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7890066730 msgr2=0x7f7890068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.641+0000 7f78af59e700 1 --2- 172.21.15.40:0/3864008522 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7890066730 0x7f7890068bf0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f78a800b370 tx=0x7f78a8009000 comp rx=0 tx=0).stop 2024-09-13T18:21:22.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.641+0000 7f78af59e700 1 -- 172.21.15.40:0/3864008522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b00b21a0 msgr2=0x7f78b01ef0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.641+0000 7f78af59e700 1 --2- 172.21.15.40:0/3864008522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b00b21a0 0x7f78b01ef0c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f78a000e810 tx=0x7f78a000ebd0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.641+0000 7f78af59e700 1 -- 172.21.15.40:0/3864008522 shutdown_connections 2024-09-13T18:21:22.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.641+0000 7f78af59e700 1 --2- 172.21.15.40:0/3864008522 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f7890066730 0x7f7890068bf0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.641+0000 7f78af59e700 1 --2- 172.21.15.40:0/3864008522 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f78b0156b80 0x7f78b014d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.641+0000 7f78af59e700 1 --2- 172.21.15.40:0/3864008522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78b00b21a0 0x7f78b01ef0c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.641+0000 7f78af59e700 1 --2- 172.21.15.40:0/3864008522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78b006d070 0x7f78b014fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.641+0000 7f78af59e700 1 -- 172.21.15.40:0/3864008522 >> 172.21.15.40:0/3864008522 conn(0x7f78b0072070 msgr2=0x7f78b00cbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:22.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.641+0000 7f78af59e700 1 -- 172.21.15.40:0/3864008522 shutdown_connections 2024-09-13T18:21:22.647 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.641+0000 7f78af59e700 1 -- 172.21.15.40:0/3864008522 wait complete. 2024-09-13T18:21:22.648 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 0 to deep-scrub 2024-09-13T18:21:22.661 INFO:tasks.ceph:Scrubbing osd.1 2024-09-13T18:21:22.661 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:21:22.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.769+0000 7f4d7f59e700 1 Processor -- start 2024-09-13T18:21:22.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.769+0000 7f4d7f59e700 1 -- start start 2024-09-13T18:21:22.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7f59e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4d800cb670 0x7f4d80157330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7f59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4d800c5890 0x7f4d80157870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7f59e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d800c9d40 con 0x7f4d800c54b0 2024-09-13T18:21:22.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7f59e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d800ca300 con 0x7f4d800cb670 2024-09-13T18:21:22.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7f59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800722d0 con 0x7f4d800c5890 2024-09-13T18:21:22.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7e59c700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f4d800c54b0 0x7f4d80151100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51424/0 (socket says 172.21.15.40:51424) 2024-09-13T18:21:22.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7e59c700 1 -- 172.21.15.40:0/2809289354 learned_addr learned my addr 172.21.15.40:0/2809289354 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:22.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7dd9b700 1 --2- 172.21.15.40:0/2809289354 >> v2:172.21.15.43:3300/0 conn(0x7f4d800cb670 0x7f4d80157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7ed9d700 1 --2- 172.21.15.40:0/2809289354 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4d800c5890 0x7f4d80157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7d59a700 1 -- 172.21.15.40:0/2809289354 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901822692 0 0) 0x7f4d800c9d40 con 0x7f4d800c54b0 2024-09-13T18:21:22.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7d59a700 1 -- 172.21.15.40:0/2809289354 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d680035d0 con 0x7f4d800c54b0 2024-09-13T18:21:22.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7d59a700 1 -- 172.21.15.40:0/2809289354 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1381083644 0 0) 0x7f4d680035d0 con 0x7f4d800c54b0 2024-09-13T18:21:22.781 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7d59a700 1 -- 172.21.15.40:0/2809289354 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4d800c5890 msgr2=0x7f4d80157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:22.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7d59a700 1 --2- 172.21.15.40:0/2809289354 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4d800c5890 0x7f4d80157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7d59a700 1 -- 172.21.15.40:0/2809289354 >> v2:172.21.15.43:3300/0 conn(0x7f4d800cb670 msgr2=0x7f4d80157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:22.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7d59a700 1 --2- 172.21.15.40:0/2809289354 >> v2:172.21.15.43:3300/0 conn(0x7f4d800cb670 0x7f4d80157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7d59a700 1 -- 172.21.15.40:0/2809289354 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d80072540 con 0x7f4d800c54b0 2024-09-13T18:21:22.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7d59a700 1 -- 172.21.15.40:0/2809289354 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f4d700019f0 con 0x7f4d800c54b0 2024-09-13T18:21:22.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7d59a700 1 -- 172.21.15.40:0/2809289354 >> v1:172.21.15.40:6789/0 conn(0x7f4d800c54b0 legacy=0x7f4d80151100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7d59a700 1 --2- 172.21.15.40:0/2809289354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d68003a20 0x7f4d68003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7d59a700 1 --2- 172.21.15.40:0/2809289354 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4d800c5890 0x7f4d68004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7d59a700 1 --2- 172.21.15.40:0/2809289354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d68004db0 0x7f4d68005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7d59a700 1 -- 172.21.15.40:0/2809289354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800ca300 con 0x7f4d68003a20 2024-09-13T18:21:22.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7d59a700 1 -- 172.21.15.40:0/2809289354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800722d0 con 0x7f4d68004db0 2024-09-13T18:21:22.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7d59a700 1 -- 172.21.15.40:0/2809289354 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6800ba30 con 0x7f4d800c5890 2024-09-13T18:21:22.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7e59c700 1 --2- 172.21.15.40:0/2809289354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d68003a20 0x7f4d68003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7dd9b700 1 --2- 172.21.15.40:0/2809289354 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4d800c5890 0x7f4d68004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7e59c700 1 -- 172.21.15.40:0/2809289354 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4d800c5890 msgr2=0x7f4d68004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:22.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7e59c700 1 --2- 172.21.15.40:0/2809289354 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4d800c5890 0x7f4d68004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7ed9d700 1 --2- 172.21.15.40:0/2809289354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d68004db0 0x7f4d68005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7e59c700 1 -- 172.21.15.40:0/2809289354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d68004db0 msgr2=0x7f4d68005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:22.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7e59c700 1 --2- 172.21.15.40:0/2809289354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d68004db0 0x7f4d68005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7e59c700 1 -- 172.21.15.40:0/2809289354 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d680035d0 con 0x7f4d68003a20 2024-09-13T18:21:22.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7dd9b700 1 --2- 172.21.15.40:0/2809289354 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4d800c5890 0x7f4d68004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:22.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7ed9d700 1 --2- 172.21.15.40:0/2809289354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d68004db0 0x7f4d68005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:22.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7e59c700 1 --2- 172.21.15.40:0/2809289354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d68003a20 0x7f4d68003e40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000a860 tx=0x7f4d680058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d5c463734a69c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:22.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7d59a700 1 -- 172.21.15.40:0/2809289354 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d700072d0 con 0x7f4d68003a20 2024-09-13T18:21:22.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7d59a700 1 -- 172.21.15.40:0/2809289354 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d70007430 con 0x7f4d68003a20 2024-09-13T18:21:22.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7d59a700 1 -- 172.21.15.40:0/2809289354 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d700077e0 con 0x7f4d68003a20 2024-09-13T18:21:22.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7f59e700 1 -- 172.21.15.40:0/2809289354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d68003a20 msgr2=0x7f4d68003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7f59e700 1 --2- 172.21.15.40:0/2809289354 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d68003a20 0x7f4d68003e40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000a860 tx=0x7f4d680058a0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7ed9d700 1 -- 172.21.15.40:0/2809289354 reap_dead start 2024-09-13T18:21:22.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7f59e700 1 -- 172.21.15.40:0/2809289354 shutdown_connections 2024-09-13T18:21:22.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7f59e700 1 -- 172.21.15.40:0/2809289354 >> 172.21.15.40:0/2809289354 conn(0x7f4d8006e620 msgr2=0x7f4d8006ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:22.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7f59e700 1 -- 172.21.15.40:0/2809289354 shutdown_connections 2024-09-13T18:21:22.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.773+0000 7f4d7f59e700 1 -- 172.21.15.40:0/2809289354 wait complete. 2024-09-13T18:21:22.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7f59e700 1 Processor -- start 2024-09-13T18:21:22.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7f59e700 1 -- start start 2024-09-13T18:21:22.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7f59e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d801568c0 0x7f4d800c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7f59e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4d80156ca0 0x7f4d800c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7f59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d80155c90 0x7f4d801ede00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.786 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7f59e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800c5090 con 0x7f4d801568c0 2024-09-13T18:21:22.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7f59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8006c0c0 con 0x7f4d80155c90 2024-09-13T18:21:22.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7f59e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d801eabc0 con 0x7f4d80156ca0 2024-09-13T18:21:22.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d801568c0 0x7f4d800c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7e59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d801568c0 0x7f4d800c34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46976/0 (socket says 172.21.15.40:46976) 2024-09-13T18:21:22.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7e59c700 1 -- 172.21.15.40:0/2960438508 learned_addr learned my addr 172.21.15.40:0/2960438508 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:22.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7ed9d700 1 --2- 172.21.15.40:0/2960438508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d80155c90 0x7f4d801ede00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7dd9b700 1 --2- 172.21.15.40:0/2960438508 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4d80156ca0 0x7f4d800c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7e59c700 1 -- 172.21.15.40:0/2960438508 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4d80156ca0 msgr2=0x7f4d800c3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7e59c700 1 --2- 172.21.15.40:0/2960438508 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4d80156ca0 0x7f4d800c3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7e59c700 1 -- 172.21.15.40:0/2960438508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d80155c90 msgr2=0x7f4d801ede00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7e59c700 1 --2- 172.21.15.40:0/2960438508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d80155c90 0x7f4d801ede00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7e59c700 1 -- 172.21.15.40:0/2960438508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8014e660 con 0x7f4d801568c0 2024-09-13T18:21:22.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7e59c700 1 --2- 172.21.15.40:0/2960438508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d801568c0 0x7f4d800c34f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4d70011000 tx=0x7f4d70001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:22.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d6effd700 1 -- 172.21.15.40:0/2960438508 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d70012600 con 0x7f4d801568c0 2024-09-13T18:21:22.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d6effd700 1 -- 172.21.15.40:0/2960438508 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d70012760 con 0x7f4d801568c0 2024-09-13T18:21:22.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7f59e700 1 -- 172.21.15.40:0/2960438508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d80154f30 con 0x7f4d801568c0 2024-09-13T18:21:22.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d6effd700 1 -- 172.21.15.40:0/2960438508 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d70012af0 con 0x7f4d801568c0 2024-09-13T18:21:22.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.777+0000 7f4d7f59e700 1 -- 172.21.15.40:0/2960438508 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d800c8e70 con 0x7f4d801568c0 2024-09-13T18:21:22.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.781+0000 7f4d6effd700 1 -- 172.21.15.40:0/2960438508 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4d7001a070 con 0x7f4d801568c0 2024-09-13T18:21:22.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.781+0000 7f4d6effd700 1 --2- 172.21.15.40:0/2960438508 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4d60066760 0x7f4d60068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.781+0000 7f4d6effd700 1 -- 172.21.15.40:0/2960438508 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4d700861a0 con 0x7f4d801568c0 2024-09-13T18:21:22.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.781+0000 7f4d7dd9b700 1 --2- 172.21.15.40:0/2960438508 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4d60066760 0x7f4d60068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.781+0000 7f4d7dd9b700 1 --2- 172.21.15.40:0/2960438508 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4d60066760 0x7f4d60068c20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4d80074ec0 tx=0x7f4d74008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:22.789 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.785+0000 7f4d7f59e700 1 --2- 172.21.15.40:0/2960438508 >> [v2:172.21.15.40:6808/2576246869,v1:172.21.15.40:6809/2576246869] conn(0x7f4d801ee340 0x7f4d801ee720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.785+0000 7f4d7f59e700 1 -- 172.21.15.40:0/2960438508 --> [v2:172.21.15.40:6808/2576246869,v1:172.21.15.40:6809/2576246869] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4d800c9f90 con 0x7f4d801ee340 2024-09-13T18:21:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.785+0000 7f4d7ed9d700 1 --2- 172.21.15.40:0/2960438508 >> [v2:172.21.15.40:6808/2576246869,v1:172.21.15.40:6809/2576246869] conn(0x7f4d801ee340 0x7f4d801ee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.785+0000 7f4d7ed9d700 1 --2- 172.21.15.40:0/2960438508 >> [v2:172.21.15.40:6808/2576246869,v1:172.21.15.40:6809/2576246869] conn(0x7f4d801ee340 0x7f4d801ee720 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:22.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.785+0000 7f4d6effd700 1 -- 172.21.15.40:0/2960438508 <== osd.1 v2:172.21.15.40:6808/2576246869 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f4d780099b0 con 0x7f4d801ee340 2024-09-13T18:21:22.807 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.801+0000 7f4d7f59e700 1 -- 172.21.15.40:0/2960438508 --> [v2:172.21.15.40:6808/2576246869,v1:172.21.15.40:6809/2576246869] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4d801eec60 con 0x7f4d801ee340 2024-09-13T18:21:22.808 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.801+0000 7f4d6effd700 1 -- 172.21.15.40:0/2960438508 <== osd.1 v2:172.21.15.40:6808/2576246869 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4d801eec60 con 0x7f4d801ee340 2024-09-13T18:21:22.808 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:21:22.808 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:21:22.808 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:21:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.801+0000 7f4d7f59e700 1 -- 172.21.15.40:0/2960438508 >> [v2:172.21.15.40:6808/2576246869,v1:172.21.15.40:6809/2576246869] conn(0x7f4d801ee340 msgr2=0x7f4d801ee720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.801+0000 7f4d7f59e700 1 --2- 172.21.15.40:0/2960438508 >> [v2:172.21.15.40:6808/2576246869,v1:172.21.15.40:6809/2576246869] conn(0x7f4d801ee340 0x7f4d801ee720 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.801+0000 7f4d7f59e700 1 -- 172.21.15.40:0/2960438508 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4d60066760 msgr2=0x7f4d60068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.801+0000 7f4d7f59e700 1 --2- 172.21.15.40:0/2960438508 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f4d60066760 0x7f4d60068c20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4d80074ec0 tx=0x7f4d74008040 comp rx=0 tx=0).stop 2024-09-13T18:21:22.809 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.801+0000 7f4d7f59e700 1 -- 172.21.15.40:0/2960438508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d801568c0 msgr2=0x7f4d800c34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.801+0000 7f4d7f59e700 1 --2- 172.21.15.40:0/2960438508 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d801568c0 0x7f4d800c34f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4d70011000 tx=0x7f4d70001d80 comp rx=0 tx=0).stop 2024-09-13T18:21:22.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.801+0000 7f4d7ed9d700 1 -- 172.21.15.40:0/2960438508 reap_dead start 2024-09-13T18:21:22.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.801+0000 7f4d7f59e700 1 -- 172.21.15.40:0/2960438508 shutdown_connections 2024-09-13T18:21:22.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.801+0000 7f4d7f59e700 1 -- 172.21.15.40:0/2960438508 >> 172.21.15.40:0/2960438508 conn(0x7f4d8006e620 msgr2=0x7f4d800c85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:22.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.801+0000 7f4d7f59e700 1 -- 172.21.15.40:0/2960438508 shutdown_connections 2024-09-13T18:21:22.810 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.801+0000 7f4d7f59e700 1 -- 172.21.15.40:0/2960438508 wait complete. 2024-09-13T18:21:22.825 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-13T18:21:22.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.929+0000 7f2fadcec700 1 Processor -- start 2024-09-13T18:21:22.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.929+0000 7f2fadcec700 1 -- start start 2024-09-13T18:21:22.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.929+0000 7f2fadcec700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2fa8162d50 0x7f2fa8165140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.929+0000 7f2fadcec700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa8159e50 con 0x7f2fa8153720 2024-09-13T18:21:22.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.929+0000 7f2fadcec700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa815d390 con 0x7f2fa80cf9b0 2024-09-13T18:21:22.937 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.929+0000 7f2fadcec700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa80c5ea0 con 0x7f2fa8162d50 2024-09-13T18:21:22.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.929+0000 7f2faccea700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2fa8153720 0x7f2fa80ca510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51430/0 (socket says 172.21.15.40:51430) 2024-09-13T18:21:22.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.929+0000 7f2faccea700 1 -- 172.21.15.40:0/3642564752 learned_addr learned my addr 172.21.15.40:0/3642564752 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:22.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.929+0000 7f2fad4eb700 1 --2- 172.21.15.40:0/3642564752 >> v2:172.21.15.40:3300/0 conn(0x7f2fa8162d50 0x7f2fa8165140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.929+0000 7f2fa77fe700 1 -- 172.21.15.40:0/3642564752 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176281266 0 0) 0x7f2fa8159e50 con 0x7f2fa8153720 2024-09-13T18:21:22.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.929+0000 7f2fa77fe700 1 -- 172.21.15.40:0/3642564752 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f980035d0 con 0x7f2fa8153720 2024-09-13T18:21:22.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.929+0000 7f2fad4eb700 1 -- 172.21.15.40:0/3642564752 >> v1:172.21.15.43:6789/0 conn(0x7f2fa80cf9b0 legacy=0x7f2fa806c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.929+0000 7f2fad4eb700 1 -- 172.21.15.40:0/3642564752 >> v1:172.21.15.40:6789/0 conn(0x7f2fa8153720 legacy=0x7f2fa80ca510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.929+0000 7f2fad4eb700 1 -- 172.21.15.40:0/3642564752 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa80cac20 con 0x7f2fa8162d50 2024-09-13T18:21:22.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.929+0000 7f2fad4eb700 1 --2- 172.21.15.40:0/3642564752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fa8162d50 0x7f2fa8165140 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009d90 tx=0x7f2fa000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102186fb51c40966 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:22.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.929+0000 7f2fa77fe700 1 -- 172.21.15.40:0/3642564752 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa000e070 con 0x7f2fa8162d50 2024-09-13T18:21:22.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.929+0000 7f2fa77fe700 1 -- 172.21.15.40:0/3642564752 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa000bb20 con 0x7f2fa8162d50 2024-09-13T18:21:22.938 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.929+0000 7f2fa77fe700 1 -- 172.21.15.40:0/3642564752 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0012430 con 0x7f2fa8162d50 2024-09-13T18:21:22.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.933+0000 7f2fadcec700 1 -- 172.21.15.40:0/3642564752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fa8162d50 msgr2=0x7f2fa8165140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.933+0000 7f2fadcec700 1 --2- 172.21.15.40:0/3642564752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fa8162d50 0x7f2fa8165140 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009d90 tx=0x7f2fa000b330 comp rx=0 tx=0).stop 2024-09-13T18:21:22.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.933+0000 7f2fadcec700 1 -- 172.21.15.40:0/3642564752 shutdown_connections 2024-09-13T18:21:22.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.933+0000 7f2fadcec700 1 --2- 172.21.15.40:0/3642564752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fa8162d50 0x7f2fa8165140 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.933+0000 7f2fadcec700 1 -- 172.21.15.40:0/3642564752 >> 172.21.15.40:0/3642564752 conn(0x7f2fa80666c0 msgr2=0x7f2fa80cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:22.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.933+0000 7f2fadcec700 1 -- 172.21.15.40:0/3642564752 shutdown_connections 2024-09-13T18:21:22.939 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.933+0000 7f2fadcec700 1 -- 172.21.15.40:0/3642564752 wait complete. 2024-09-13T18:21:22.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.933+0000 7f2fadcec700 1 Processor -- start 2024-09-13T18:21:22.940 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.933+0000 7f2fadcec700 1 -- start start 2024-09-13T18:21:22.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.933+0000 7f2fadcec700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fa80cf9b0 0x7f2fa8153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.933+0000 7f2fadcec700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fa8153720 0x7f2fa80c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.933+0000 7f2fadcec700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fa8162d50 0x7f2fa81f2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.933+0000 7f2fadcec700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa81f34a0 con 0x7f2fa80cf9b0 2024-09-13T18:21:22.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.933+0000 7f2fadcec700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa81544b0 con 0x7f2fa8153720 2024-09-13T18:21:22.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.933+0000 7f2fadcec700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa80c6760 con 0x7f2fa8162d50 2024-09-13T18:21:22.943 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.933+0000 7f2faccea700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fa80cf9b0 0x7f2fa8153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.933+0000 7f2faccea700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fa80cf9b0 0x7f2fa8153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47004/0 (socket says 172.21.15.40:47004) 2024-09-13T18:21:22.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.933+0000 7f2faccea700 1 -- 172.21.15.40:0/2161203336 learned_addr learned my addr 172.21.15.40:0/2161203336 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:22.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.937+0000 7f2fa7fff700 1 --2- 172.21.15.40:0/2161203336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fa8153720 0x7f2fa80c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.937+0000 7f2faccea700 1 -- 172.21.15.40:0/2161203336 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fa8162d50 msgr2=0x7f2fa81f2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.937+0000 7f2fad4eb700 1 --2- 172.21.15.40:0/2161203336 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fa8162d50 0x7f2fa81f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.937+0000 7f2faccea700 1 --2- 172.21.15.40:0/2161203336 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fa8162d50 0x7f2fa81f2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.937+0000 7f2faccea700 1 -- 172.21.15.40:0/2161203336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fa8153720 msgr2=0x7f2fa80c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:22.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.937+0000 7f2faccea700 1 --2- 172.21.15.40:0/2161203336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fa8153720 0x7f2fa80c73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:22.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.937+0000 7f2faccea700 1 -- 172.21.15.40:0/2161203336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa0009a70 con 0x7f2fa80cf9b0 2024-09-13T18:21:22.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.937+0000 7f2fa7fff700 1 --2- 172.21.15.40:0/2161203336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fa8153720 0x7f2fa80c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:22.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.937+0000 7f2faccea700 1 --2- 172.21.15.40:0/2161203336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fa80cf9b0 0x7f2fa8153dd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00c700 tx=0x7f2f9c00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:22.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.937+0000 7f2fa57fa700 1 -- 172.21.15.40:0/2161203336 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c005d90 con 0x7f2fa80cf9b0 2024-09-13T18:21:22.944 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.937+0000 7f2fa57fa700 1 -- 172.21.15.40:0/2161203336 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9c0070d0 con 0x7f2fa80cf9b0 2024-09-13T18:21:22.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.937+0000 7f2fadcec700 1 -- 172.21.15.40:0/2161203336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa81546e0 con 0x7f2fa80cf9b0 2024-09-13T18:21:22.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.937+0000 7f2fa57fa700 1 -- 172.21.15.40:0/2161203336 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c007480 con 0x7f2fa80cf9b0 2024-09-13T18:21:22.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.937+0000 7f2fadcec700 1 -- 172.21.15.40:0/2161203336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa80c63f0 con 0x7f2fa80cf9b0 2024-09-13T18:21:22.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.937+0000 7f2fadcec700 1 -- 172.21.15.40:0/2161203336 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa8067a40 con 0x7f2fa80cf9b0 2024-09-13T18:21:22.945 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.937+0000 7f2fa57fa700 1 -- 172.21.15.40:0/2161203336 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2f9c0075e0 con 0x7f2fa80cf9b0 2024-09-13T18:21:22.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.937+0000 7f2fa57fa700 1 --2- 172.21.15.40:0/2161203336 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f8c066730 0x7f2f8c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:22.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.937+0000 7f2fa57fa700 1 -- 172.21.15.40:0/2161203336 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2f9c083af0 con 0x7f2fa80cf9b0 2024-09-13T18:21:22.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.937+0000 7f2fa7fff700 1 --2- 172.21.15.40:0/2161203336 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f8c066730 0x7f2f8c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:22.946 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.941+0000 7f2fa7fff700 1 --2- 172.21.15.40:0/2161203336 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f8c066730 0x7f2f8c068bf0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2fa81f00e0 tx=0x7f2f94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:22.952 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:22.945+0000 7f2fa57fa700 1 -- 172.21.15.40:0/2161203336 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2f9c050ab0 con 0x7f2fa80cf9b0 2024-09-13T18:21:23.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.129+0000 7f2fadcec700 1 -- 172.21.15.40:0/2161203336 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fa8153b00 con 0x7f2f8c066730 2024-09-13T18:21:23.135 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.129+0000 7f2fa57fa700 1 -- 172.21.15.40:0/2161203336 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2f940050d0 con 0x7f2f8c066730 2024-09-13T18:21:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.133+0000 7f2fadcec700 1 -- 172.21.15.40:0/2161203336 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f8c066730 msgr2=0x7f2f8c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.138 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.133+0000 7f2fadcec700 1 --2- 172.21.15.40:0/2161203336 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f8c066730 0x7f2f8c068bf0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2fa81f00e0 tx=0x7f2f94009040 comp rx=0 tx=0).stop 2024-09-13T18:21:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.133+0000 7f2fadcec700 1 -- 172.21.15.40:0/2161203336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fa80cf9b0 msgr2=0x7f2fa8153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.133+0000 7f2fadcec700 1 --2- 172.21.15.40:0/2161203336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fa80cf9b0 0x7f2fa8153dd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00c700 tx=0x7f2f9c00cac0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.133+0000 7f2fadcec700 1 -- 172.21.15.40:0/2161203336 shutdown_connections 2024-09-13T18:21:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.133+0000 7f2fadcec700 1 --2- 172.21.15.40:0/2161203336 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2f8c066730 0x7f2f8c068bf0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.133+0000 7f2fadcec700 1 --2- 172.21.15.40:0/2161203336 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2fa8162d50 0x7f2fa81f2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.133+0000 7f2fadcec700 1 --2- 172.21.15.40:0/2161203336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fa8153720 0x7f2fa80c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.133+0000 7f2fadcec700 1 --2- 172.21.15.40:0/2161203336 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2fa80cf9b0 0x7f2fa8153dd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.133+0000 7f2fadcec700 1 -- 172.21.15.40:0/2161203336 >> 172.21.15.40:0/2161203336 conn(0x7f2fa80666c0 msgr2=0x7f2fa80c4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.133+0000 7f2fadcec700 1 -- 172.21.15.40:0/2161203336 shutdown_connections 2024-09-13T18:21:23.139 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.133+0000 7f2fadcec700 1 -- 172.21.15.40:0/2161203336 wait complete. 2024-09-13T18:21:23.140 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 1 to deep-scrub 2024-09-13T18:21:23.149 INFO:tasks.ceph:Scrubbing osd.2 2024-09-13T18:21:23.150 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:21:23.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.253+0000 7fec44c79700 1 Processor -- start 2024-09-13T18:21:23.261 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec44c79700 1 -- start start 2024-09-13T18:21:23.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec44c79700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fec400c2980 0x7fec400c9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec44c79700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec400cad20 con 0x7fec40153900 2024-09-13T18:21:23.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec44c79700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec400b2250 con 0x7fec400cbb00 2024-09-13T18:21:23.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec44c79700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec40070a30 con 0x7fec400c2980 2024-09-13T18:21:23.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3f7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fec40153900 0x7fec4015d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51442/0 (socket says 172.21.15.40:51442) 2024-09-13T18:21:23.265 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3f7fe700 1 -- 172.21.15.40:0/429653248 learned_addr learned my addr 172.21.15.40:0/429653248 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:23.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3effd700 1 --2- 172.21.15.40:0/429653248 >> v2:172.21.15.43:3300/0 conn(0x7fec400c2980 0x7fec400c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3e7fc700 1 -- 172.21.15.40:0/429653248 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 86199485 0 0) 0x7fec400cad20 con 0x7fec40153900 2024-09-13T18:21:23.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3e7fc700 1 -- 172.21.15.40:0/429653248 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec280035d0 con 0x7fec40153900 2024-09-13T18:21:23.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3e7fc700 1 -- 172.21.15.40:0/429653248 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3925408421 0 0) 0x7fec280035d0 con 0x7fec40153900 2024-09-13T18:21:23.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3e7fc700 1 -- 172.21.15.40:0/429653248 >> v2:172.21.15.43:3300/0 conn(0x7fec400c2980 msgr2=0x7fec400c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:23.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3e7fc700 1 --2- 172.21.15.40:0/429653248 >> v2:172.21.15.43:3300/0 conn(0x7fec400c2980 0x7fec400c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3e7fc700 1 -- 172.21.15.40:0/429653248 >> v1:172.21.15.43:6789/0 conn(0x7fec400cbb00 legacy=0x7fec40163e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3e7fc700 1 -- 172.21.15.40:0/429653248 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec40153ce0 con 0x7fec40153900 2024-09-13T18:21:23.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3e7fc700 1 -- 172.21.15.40:0/429653248 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fec30001670 con 0x7fec40153900 2024-09-13T18:21:23.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3e7fc700 1 -- 172.21.15.40:0/429653248 >> v1:172.21.15.40:6789/0 conn(0x7fec40153900 legacy=0x7fec4015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3e7fc700 1 --2- 172.21.15.40:0/429653248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec28003a70 0x7fec28003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3e7fc700 1 --2- 172.21.15.40:0/429653248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec28004870 0x7fec28004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.266 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3e7fc700 1 --2- 172.21.15.40:0/429653248 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec28005230 0x7fec2800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3e7fc700 1 -- 172.21.15.40:0/429653248 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2800c100 con 0x7fec28003a70 2024-09-13T18:21:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3e7fc700 1 -- 172.21.15.40:0/429653248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2800c270 con 0x7fec28004870 2024-09-13T18:21:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3e7fc700 1 -- 172.21.15.40:0/429653248 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2800c430 con 0x7fec28005230 2024-09-13T18:21:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3f7fe700 1 --2- 172.21.15.40:0/429653248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec28003a70 0x7fec28003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3effd700 1 --2- 172.21.15.40:0/429653248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec28004870 0x7fec28004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3ffff700 1 --2- 172.21.15.40:0/429653248 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec28005230 0x7fec2800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3f7fe700 1 -- 172.21.15.40:0/429653248 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec28005230 msgr2=0x7fec2800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3f7fe700 1 --2- 172.21.15.40:0/429653248 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec28005230 0x7fec2800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3f7fe700 1 -- 172.21.15.40:0/429653248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec28004870 msgr2=0x7fec28004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3f7fe700 1 --2- 172.21.15.40:0/429653248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec28004870 0x7fec28004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3f7fe700 1 -- 172.21.15.40:0/429653248 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec280035d0 con 0x7fec28003a70 2024-09-13T18:21:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3ffff700 1 -- 172.21.15.40:0/429653248 reap_dead start 2024-09-13T18:21:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3f7fe700 1 --2- 172.21.15.40:0/429653248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec28003a70 0x7fec28003e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fec3000c210 tx=0x7fec280058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc0e8461ca04a89 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:23.267 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3e7fc700 1 -- 172.21.15.40:0/429653248 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec30007b90 con 0x7fec28003a70 2024-09-13T18:21:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3e7fc700 1 -- 172.21.15.40:0/429653248 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3000cd50 con 0x7fec28003a70 2024-09-13T18:21:23.268 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.257+0000 7fec3e7fc700 1 -- 172.21.15.40:0/429653248 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec30005bc0 con 0x7fec28003a70 2024-09-13T18:21:23.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.261+0000 7fec44c79700 1 -- 172.21.15.40:0/429653248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec28003a70 msgr2=0x7fec28003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.261+0000 7fec44c79700 1 --2- 172.21.15.40:0/429653248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec28003a70 0x7fec28003e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fec3000c210 tx=0x7fec280058a0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.261+0000 7fec44c79700 1 -- 172.21.15.40:0/429653248 shutdown_connections 2024-09-13T18:21:23.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.261+0000 7fec44c79700 1 --2- 172.21.15.40:0/429653248 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec28003a70 0x7fec28003e90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.261+0000 7fec44c79700 1 -- 172.21.15.40:0/429653248 >> 172.21.15.40:0/429653248 conn(0x7fec4006e620 msgr2=0x7fec4006ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:23.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.261+0000 7fec44c79700 1 -- 172.21.15.40:0/429653248 shutdown_connections 2024-09-13T18:21:23.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.261+0000 7fec44c79700 1 -- 172.21.15.40:0/429653248 wait complete. 2024-09-13T18:21:23.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.261+0000 7fec44c79700 1 Processor -- start 2024-09-13T18:21:23.269 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.261+0000 7fec44c79700 1 -- start start 2024-09-13T18:21:23.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec44c79700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec28003a70 0x7fec401f2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec44c79700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec400c37c0 0x7fec40156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec44c79700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec401ee770 0x7fec400713f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec44c79700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec40156e30 con 0x7fec28003a70 2024-09-13T18:21:23.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec44c79700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec40152c60 con 0x7fec401ee770 2024-09-13T18:21:23.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec44c79700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec40156870 con 0x7fec400c37c0 2024-09-13T18:21:23.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec3f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec28003a70 0x7fec401f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec3f7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec28003a70 0x7fec401f2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47018/0 (socket says 172.21.15.40:47018) 2024-09-13T18:21:23.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec3f7fe700 1 -- 172.21.15.40:0/2333744822 learned_addr learned my addr 172.21.15.40:0/2333744822 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:23.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec3effd700 1 --2- 172.21.15.40:0/2333744822 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec400c37c0 0x7fec40156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec3ffff700 1 --2- 172.21.15.40:0/2333744822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec401ee770 0x7fec400713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec3f7fe700 1 -- 172.21.15.40:0/2333744822 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec400c37c0 msgr2=0x7fec40156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.271 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec3f7fe700 1 --2- 172.21.15.40:0/2333744822 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec400c37c0 0x7fec40156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec3f7fe700 1 -- 172.21.15.40:0/2333744822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec401ee770 msgr2=0x7fec400713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec3f7fe700 1 --2- 172.21.15.40:0/2333744822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec401ee770 0x7fec400713f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec3f7fe700 1 -- 172.21.15.40:0/2333744822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec40156b50 con 0x7fec28003a70 2024-09-13T18:21:23.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec3ffff700 1 --2- 172.21.15.40:0/2333744822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec401ee770 0x7fec400713f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:23.272 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec3f7fe700 1 --2- 172.21.15.40:0/2333744822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec28003a70 0x7fec401f2860 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fec30000c00 tx=0x7fec30001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec27fff700 1 -- 172.21.15.40:0/2333744822 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec30004e90 con 0x7fec28003a70 2024-09-13T18:21:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec27fff700 1 -- 172.21.15.40:0/2333744822 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3000cec0 con 0x7fec28003a70 2024-09-13T18:21:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec44c79700 1 -- 172.21.15.40:0/2333744822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec401f1d80 con 0x7fec28003a70 2024-09-13T18:21:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec27fff700 1 -- 172.21.15.40:0/2333744822 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300072f0 con 0x7fec28003a70 2024-09-13T18:21:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec44c79700 1 -- 172.21.15.40:0/2333744822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec401f2340 con 0x7fec28003a70 2024-09-13T18:21:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.265+0000 7fec44c79700 1 -- 172.21.15.40:0/2333744822 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fec400cb250 con 0x7fec28003a70 2024-09-13T18:21:23.273 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.269+0000 7fec27fff700 1 -- 172.21.15.40:0/2333744822 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fec30007450 con 0x7fec28003a70 2024-09-13T18:21:23.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.269+0000 7fec27fff700 1 --2- 172.21.15.40:0/2333744822 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fec2c066760 0x7fec2c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.269+0000 7fec27fff700 1 -- 172.21.15.40:0/2333744822 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fec300854b0 con 0x7fec28003a70 2024-09-13T18:21:23.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.269+0000 7fec3effd700 1 --2- 172.21.15.40:0/2333744822 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fec2c066760 0x7fec2c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.269+0000 7fec27fff700 1 --2- 172.21.15.40:0/2333744822 >> [v2:172.21.15.40:6800/44856195,v1:172.21.15.40:6801/44856195] conn(0x7fec2c06fed0 0x7fec2c072370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.269+0000 7fec27fff700 1 -- 172.21.15.40:0/2333744822 --> [v2:172.21.15.40:6800/44856195,v1:172.21.15.40:6801/44856195] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec2c072a40 con 0x7fec2c06fed0 2024-09-13T18:21:23.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.269+0000 7fec27fff700 1 -- 172.21.15.40:0/2333744822 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec30087b70 con 0x7fec28003a70 2024-09-13T18:21:23.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.269+0000 7fec3ffff700 1 --2- 172.21.15.40:0/2333744822 >> [v2:172.21.15.40:6800/44856195,v1:172.21.15.40:6801/44856195] conn(0x7fec2c06fed0 0x7fec2c072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.269+0000 7fec3effd700 1 --2- 172.21.15.40:0/2333744822 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fec2c066760 0x7fec2c068c20 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fec40071c80 tx=0x7fec34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:23.275 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.269+0000 7fec3ffff700 1 --2- 172.21.15.40:0/2333744822 >> [v2:172.21.15.40:6800/44856195,v1:172.21.15.40:6801/44856195] conn(0x7fec2c06fed0 0x7fec2c072370 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:23.276 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.269+0000 7fec27fff700 1 -- 172.21.15.40:0/2333744822 <== osd.2 v2:172.21.15.40:6800/44856195 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fec2c072a40 con 0x7fec2c06fed0 2024-09-13T18:21:23.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.289+0000 7fec44c79700 1 -- 172.21.15.40:0/2333744822 --> [v2:172.21.15.40:6800/44856195,v1:172.21.15.40:6801/44856195] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fec400caf70 con 0x7fec2c06fed0 2024-09-13T18:21:23.294 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.289+0000 7fec27fff700 1 -- 172.21.15.40:0/2333744822 <== osd.2 v2:172.21.15.40:6800/44856195 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fec38001be0 con 0x7fec2c06fed0 2024-09-13T18:21:23.294 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:21:23.295 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:21:23.295 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:21:23.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.289+0000 7fec44c79700 1 -- 172.21.15.40:0/2333744822 >> [v2:172.21.15.40:6800/44856195,v1:172.21.15.40:6801/44856195] conn(0x7fec2c06fed0 msgr2=0x7fec2c072370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.289+0000 7fec44c79700 1 --2- 172.21.15.40:0/2333744822 >> [v2:172.21.15.40:6800/44856195,v1:172.21.15.40:6801/44856195] conn(0x7fec2c06fed0 0x7fec2c072370 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.289+0000 7fec44c79700 1 -- 172.21.15.40:0/2333744822 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fec2c066760 msgr2=0x7fec2c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.289+0000 7fec44c79700 1 --2- 172.21.15.40:0/2333744822 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fec2c066760 0x7fec2c068c20 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fec40071c80 tx=0x7fec34008040 comp rx=0 tx=0).stop 2024-09-13T18:21:23.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.289+0000 7fec44c79700 1 -- 172.21.15.40:0/2333744822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec28003a70 msgr2=0x7fec401f2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.289+0000 7fec44c79700 1 --2- 172.21.15.40:0/2333744822 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fec28003a70 0x7fec401f2860 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fec30000c00 tx=0x7fec30001930 comp rx=0 tx=0).stop 2024-09-13T18:21:23.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.289+0000 7fec3ffff700 1 -- 172.21.15.40:0/2333744822 reap_dead start 2024-09-13T18:21:23.296 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.289+0000 7fec44c79700 1 -- 172.21.15.40:0/2333744822 shutdown_connections 2024-09-13T18:21:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.289+0000 7fec44c79700 1 -- 172.21.15.40:0/2333744822 >> 172.21.15.40:0/2333744822 conn(0x7fec4006e620 msgr2=0x7fec40151b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.289+0000 7fec44c79700 1 -- 172.21.15.40:0/2333744822 shutdown_connections 2024-09-13T18:21:23.297 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.289+0000 7fec44c79700 1 -- 172.21.15.40:0/2333744822 wait complete. 2024-09-13T18:21:23.311 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-13T18:21:23.417 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4bbfff700 1 Processor -- start 2024-09-13T18:21:23.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4bbfff700 1 -- start start 2024-09-13T18:21:23.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4bbfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb4bc14fdd0 0x7fb4bc0c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4bbfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb4bc0c72e0 0x7fb4bc0c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4bbfff700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4bc159eb0 con 0x7fb4bc0c5f10 2024-09-13T18:21:23.419 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4bbfff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4bc0c76e0 con 0x7fb4bc14fdd0 2024-09-13T18:21:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4bbfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc158fc0 con 0x7fb4bc0c72e0 2024-09-13T18:21:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4bb7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb4bc0c5f10 0x7fb4bc0c0a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51456/0 (socket says 172.21.15.40:51456) 2024-09-13T18:21:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4bb7fe700 1 -- 172.21.15.40:0/2438512658 learned_addr learned my addr 172.21.15.40:0/2438512658 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4b27fc700 1 --2- 172.21.15.40:0/2438512658 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb4bc0c72e0 0x7fb4bc0c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4baffd700 1 --2- 172.21.15.40:0/2438512658 >> v2:172.21.15.43:3300/0 conn(0x7fb4bc14fdd0 0x7fb4bc0c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4ba7fc700 1 -- 172.21.15.40:0/2438512658 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3557683269 0 0) 0x7fb4bc159eb0 con 0x7fb4bc0c5f10 2024-09-13T18:21:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4ba7fc700 1 -- 172.21.15.40:0/2438512658 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4a80035d0 con 0x7fb4bc0c5f10 2024-09-13T18:21:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4b27fc700 1 -- 172.21.15.40:0/2438512658 >> v2:172.21.15.43:3300/0 conn(0x7fb4bc14fdd0 msgr2=0x7fb4bc0c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4b27fc700 1 --2- 172.21.15.40:0/2438512658 >> v2:172.21.15.43:3300/0 conn(0x7fb4bc14fdd0 0x7fb4bc0c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4b27fc700 1 -- 172.21.15.40:0/2438512658 >> v1:172.21.15.40:6789/0 conn(0x7fb4bc0c5f10 legacy=0x7fb4bc0c0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4b27fc700 1 -- 172.21.15.40:0/2438512658 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc1592a0 con 0x7fb4bc0c72e0 2024-09-13T18:21:23.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4ba7fc700 1 -- 172.21.15.40:0/2438512658 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3922933852 0 0) 0x7fb4a80035d0 con 0x7fb4bc0c5f10 2024-09-13T18:21:23.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4b27fc700 1 --2- 172.21.15.40:0/2438512658 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb4bc0c72e0 0x7fb4bc0c59d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400a9f0 tx=0x7fb4a400ad00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6710f03375d17b5a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:23.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4ba7fc700 1 -- 172.21.15.40:0/2438512658 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a400cb70 con 0x7fb4bc0c72e0 2024-09-13T18:21:23.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4ba7fc700 1 -- 172.21.15.40:0/2438512658 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a400ae90 con 0x7fb4bc0c72e0 2024-09-13T18:21:23.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4ba7fc700 1 -- 172.21.15.40:0/2438512658 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4018400 con 0x7fb4bc0c72e0 2024-09-13T18:21:23.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4bbfff700 1 -- 172.21.15.40:0/2438512658 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb4bc0c72e0 msgr2=0x7fb4bc0c59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4bbfff700 1 --2- 172.21.15.40:0/2438512658 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb4bc0c72e0 0x7fb4bc0c59d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400a9f0 tx=0x7fb4a400ad00 comp rx=0 tx=0).stop 2024-09-13T18:21:23.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4bbfff700 1 -- 172.21.15.40:0/2438512658 shutdown_connections 2024-09-13T18:21:23.422 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4bbfff700 1 --2- 172.21.15.40:0/2438512658 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb4bc0c72e0 0x7fb4bc0c59d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4bbfff700 1 --2- 172.21.15.40:0/2438512658 >> v2:172.21.15.43:3300/0 conn(0x7fb4bc14fdd0 0x7fb4bc0c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4bbfff700 1 -- 172.21.15.40:0/2438512658 >> 172.21.15.40:0/2438512658 conn(0x7fb4bc066090 msgr2=0x7fb4bc0664a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:23.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4bbfff700 1 -- 172.21.15.40:0/2438512658 shutdown_connections 2024-09-13T18:21:23.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.413+0000 7fb4bbfff700 1 -- 172.21.15.40:0/2438512658 wait complete. 2024-09-13T18:21:23.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4bbfff700 1 Processor -- start 2024-09-13T18:21:23.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4bbfff700 1 -- start start 2024-09-13T18:21:23.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4bbfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb4bc0c5f10 0x7fb4bc14c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4bbfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4bc0c72e0 0x7fb4bc0cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4bbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4bc14fdd0 0x7fb4bc0cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4bbfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc152220 con 0x7fb4bc14fdd0 2024-09-13T18:21:23.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4bbfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc1502a0 con 0x7fb4bc0c72e0 2024-09-13T18:21:23.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4bbfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc1eaaf0 con 0x7fb4bc0c5f10 2024-09-13T18:21:23.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4bb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4bc14fdd0 0x7fb4bc0cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4bb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4bc14fdd0 0x7fb4bc0cb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47024/0 (socket says 172.21.15.40:47024) 2024-09-13T18:21:23.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4bb7fe700 1 -- 172.21.15.40:0/2343172972 learned_addr learned my addr 172.21.15.40:0/2343172972 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:23.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4baffd700 1 --2- 172.21.15.40:0/2343172972 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb4bc0c5f10 0x7fb4bc14c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4b27fc700 1 --2- 172.21.15.40:0/2343172972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4bc0c72e0 0x7fb4bc0cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4bb7fe700 1 -- 172.21.15.40:0/2343172972 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb4bc0c5f10 msgr2=0x7fb4bc14c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4bb7fe700 1 --2- 172.21.15.40:0/2343172972 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb4bc0c5f10 0x7fb4bc14c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4bb7fe700 1 -- 172.21.15.40:0/2343172972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4bc0c72e0 msgr2=0x7fb4bc0cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4bb7fe700 1 --2- 172.21.15.40:0/2343172972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4bc0c72e0 0x7fb4bc0cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4bb7fe700 1 -- 172.21.15.40:0/2343172972 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a400a650 con 0x7fb4bc14fdd0 2024-09-13T18:21:23.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4bb7fe700 1 --2- 172.21.15.40:0/2343172972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4bc14fdd0 0x7fb4bc0cb6f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400edb0 tx=0x7fb4b400ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:23.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4b8ff9700 1 -- 172.21.15.40:0/2343172972 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400cbf0 con 0x7fb4bc14fdd0 2024-09-13T18:21:23.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.417+0000 7fb4b8ff9700 1 -- 172.21.15.40:0/2343172972 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b400cd70 con 0x7fb4bc14fdd0 2024-09-13T18:21:23.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.421+0000 7fb4bbfff700 1 -- 172.21.15.40:0/2343172972 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4bc1eadb0 con 0x7fb4bc14fdd0 2024-09-13T18:21:23.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.421+0000 7fb4b8ff9700 1 -- 172.21.15.40:0/2343172972 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b40116b0 con 0x7fb4bc14fdd0 2024-09-13T18:21:23.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.421+0000 7fb4bbfff700 1 -- 172.21.15.40:0/2343172972 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4bc1eb370 con 0x7fb4bc14fdd0 2024-09-13T18:21:23.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.421+0000 7fb4b8ff9700 1 -- 172.21.15.40:0/2343172972 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb4b4011810 con 0x7fb4bc14fdd0 2024-09-13T18:21:23.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.421+0000 7fb4bbfff700 1 -- 172.21.15.40:0/2343172972 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4bc1504a0 con 0x7fb4bc14fdd0 2024-09-13T18:21:23.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.421+0000 7fb4b8ff9700 1 --2- 172.21.15.40:0/2343172972 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb49c066730 0x7fb49c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.421+0000 7fb4baffd700 1 --2- 172.21.15.40:0/2343172972 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb49c066730 0x7fb49c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.421+0000 7fb4b8ff9700 1 -- 172.21.15.40:0/2343172972 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb4b4086e10 con 0x7fb4bc14fdd0 2024-09-13T18:21:23.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.421+0000 7fb4baffd700 1 --2- 172.21.15.40:0/2343172972 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb49c066730 0x7fb49c068bf0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac009b50 tx=0x7fb4ac009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:23.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.429+0000 7fb4b8ff9700 1 -- 172.21.15.40:0/2343172972 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb4bc1504a0 con 0x7fb4bc14fdd0 2024-09-13T18:21:23.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.597+0000 7fb4bbfff700 1 -- 172.21.15.40:0/2343172972 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4bc159b40 con 0x7fb49c066730 2024-09-13T18:21:23.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.597+0000 7fb4b8ff9700 1 -- 172.21.15.40:0/2343172972 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb4ac004020 con 0x7fb49c066730 2024-09-13T18:21:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.601+0000 7fb4bbfff700 1 -- 172.21.15.40:0/2343172972 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb49c066730 msgr2=0x7fb49c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.601+0000 7fb4bbfff700 1 --2- 172.21.15.40:0/2343172972 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb49c066730 0x7fb49c068bf0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac009b50 tx=0x7fb4ac009240 comp rx=0 tx=0).stop 2024-09-13T18:21:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.601+0000 7fb4bbfff700 1 -- 172.21.15.40:0/2343172972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4bc14fdd0 msgr2=0x7fb4bc0cb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.601+0000 7fb4bbfff700 1 --2- 172.21.15.40:0/2343172972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4bc14fdd0 0x7fb4bc0cb6f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400edb0 tx=0x7fb4b400ede0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.601+0000 7fb4bbfff700 1 -- 172.21.15.40:0/2343172972 shutdown_connections 2024-09-13T18:21:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.601+0000 7fb4bbfff700 1 --2- 172.21.15.40:0/2343172972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb4bc14fdd0 0x7fb4bc0cb6f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.601+0000 7fb4bbfff700 1 --2- 172.21.15.40:0/2343172972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4bc0c72e0 0x7fb4bc0cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.608 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.601+0000 7fb4bbfff700 1 --2- 172.21.15.40:0/2343172972 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fb49c066730 0x7fb49c068bf0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.601+0000 7fb4bbfff700 1 --2- 172.21.15.40:0/2343172972 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fb4bc0c5f10 0x7fb4bc14c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.601+0000 7fb4bbfff700 1 -- 172.21.15.40:0/2343172972 >> 172.21.15.40:0/2343172972 conn(0x7fb4bc066090 msgr2=0x7fb4bc145ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:23.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.601+0000 7fb4bbfff700 1 -- 172.21.15.40:0/2343172972 shutdown_connections 2024-09-13T18:21:23.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.601+0000 7fb4bbfff700 1 -- 172.21.15.40:0/2343172972 wait complete. 2024-09-13T18:21:23.609 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 2 to deep-scrub 2024-09-13T18:21:23.619 INFO:tasks.ceph:Scrubbing osd.3 2024-09-13T18:21:23.620 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:21:23.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e75979700 1 Processor -- start 2024-09-13T18:21:23.734 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e75979700 1 -- start start 2024-09-13T18:21:23.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e75979700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e68051a10 0x7f6e680dbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e75979700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6e68045d00 0x7f6e6804d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e75979700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e68007830 con 0x7f6e6804cc20 2024-09-13T18:21:23.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e75979700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e6804d540 con 0x7f6e68045d00 2024-09-13T18:21:23.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e75979700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e68007a80 con 0x7f6e68051a10 2024-09-13T18:21:23.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e74977700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6e6804cc20 0x7f6e68008b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51472/0 (socket says 172.21.15.40:51472) 2024-09-13T18:21:23.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e74977700 1 -- 172.21.15.40:0/3340984261 learned_addr learned my addr 172.21.15.40:0/3340984261 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:23.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e75178700 1 --2- 172.21.15.40:0/3340984261 >> v2:172.21.15.43:3300/0 conn(0x7f6e68045d00 0x7f6e6804d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6ffff700 1 --2- 172.21.15.40:0/3340984261 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e68051a10 0x7f6e680dbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.737 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6f7fe700 1 -- 172.21.15.40:0/3340984261 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 578728222 0 0) 0x7f6e68007830 con 0x7f6e6804cc20 2024-09-13T18:21:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6f7fe700 1 -- 172.21.15.40:0/3340984261 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e600035d0 con 0x7f6e6804cc20 2024-09-13T18:21:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6f7fe700 1 -- 172.21.15.40:0/3340984261 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3587544541 0 0) 0x7f6e600035d0 con 0x7f6e6804cc20 2024-09-13T18:21:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6f7fe700 1 -- 172.21.15.40:0/3340984261 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e68051a10 msgr2=0x7f6e680dbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6f7fe700 1 --2- 172.21.15.40:0/3340984261 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e68051a10 0x7f6e680dbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6f7fe700 1 -- 172.21.15.40:0/3340984261 >> v2:172.21.15.43:3300/0 conn(0x7f6e68045d00 msgr2=0x7f6e6804d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6f7fe700 1 --2- 172.21.15.40:0/3340984261 >> v2:172.21.15.43:3300/0 conn(0x7f6e68045d00 0x7f6e6804d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6f7fe700 1 -- 172.21.15.40:0/3340984261 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e68007d60 con 0x7f6e6804cc20 2024-09-13T18:21:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6f7fe700 1 -- 172.21.15.40:0/3340984261 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7f6e64001670 con 0x7f6e6804cc20 2024-09-13T18:21:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6f7fe700 1 -- 172.21.15.40:0/3340984261 >> v1:172.21.15.40:6789/0 conn(0x7f6e6804cc20 legacy=0x7f6e68008b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6f7fe700 1 --2- 172.21.15.40:0/3340984261 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e68051a10 0x7f6e60003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6f7fe700 1 --2- 172.21.15.40:0/3340984261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e60003fa0 0x7f6e60004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6f7fe700 1 --2- 172.21.15.40:0/3340984261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e60004db0 0x7f6e60005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.738 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6f7fe700 1 -- 172.21.15.40:0/3340984261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6804d540 con 0x7f6e60004db0 2024-09-13T18:21:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6f7fe700 1 -- 172.21.15.40:0/3340984261 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e68007a80 con 0x7f6e60003fa0 2024-09-13T18:21:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6f7fe700 1 -- 172.21.15.40:0/3340984261 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6000bab0 con 0x7f6e68051a10 2024-09-13T18:21:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e75178700 1 --2- 172.21.15.40:0/3340984261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e60004db0 0x7f6e60005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e75178700 1 -- 172.21.15.40:0/3340984261 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e68051a10 msgr2=0x7f6e60003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e75178700 1 --2- 172.21.15.40:0/3340984261 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e68051a10 0x7f6e60003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6ffff700 1 --2- 172.21.15.40:0/3340984261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e60003fa0 0x7f6e60004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e75178700 1 -- 172.21.15.40:0/3340984261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e60003fa0 msgr2=0x7f6e60004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e75178700 1 --2- 172.21.15.40:0/3340984261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e60003fa0 0x7f6e60004870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e75178700 1 -- 172.21.15.40:0/3340984261 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e600035d0 con 0x7f6e60004db0 2024-09-13T18:21:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e75178700 1 --2- 172.21.15.40:0/3340984261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e60004db0 0x7f6e60005230 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6e7002ce60 tx=0x7f6e600058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f109dfa7ba64205 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6f7fe700 1 -- 172.21.15.40:0/3340984261 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e70090070 con 0x7f6e60004db0 2024-09-13T18:21:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6f7fe700 1 -- 172.21.15.40:0/3340984261 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e70068920 con 0x7f6e60004db0 2024-09-13T18:21:23.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e6f7fe700 1 -- 172.21.15.40:0/3340984261 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e70068a90 con 0x7f6e60004db0 2024-09-13T18:21:23.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.729+0000 7f6e75979700 1 -- 172.21.15.40:0/3340984261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e60004db0 msgr2=0x7f6e60005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75979700 1 --2- 172.21.15.40:0/3340984261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e60004db0 0x7f6e60005230 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6e7002ce60 tx=0x7f6e600058a0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75178700 1 -- 172.21.15.40:0/3340984261 reap_dead start 2024-09-13T18:21:23.742 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75979700 1 -- 172.21.15.40:0/3340984261 shutdown_connections 2024-09-13T18:21:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75979700 1 -- 172.21.15.40:0/3340984261 >> 172.21.15.40:0/3340984261 conn(0x7f6e68050e60 msgr2=0x7f6e68051270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75979700 1 -- 172.21.15.40:0/3340984261 shutdown_connections 2024-09-13T18:21:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75979700 1 -- 172.21.15.40:0/3340984261 wait complete. 2024-09-13T18:21:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75979700 1 Processor -- start 2024-09-13T18:21:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75979700 1 -- start start 2024-09-13T18:21:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75979700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e680da190 0x7f6e680dd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75979700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e680da570 0x7f6e6804a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75979700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e680d8620 0x7f6e6804aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75979700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6804afc0 con 0x7f6e680d8620 2024-09-13T18:21:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75979700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e68170cc0 con 0x7f6e680da190 2024-09-13T18:21:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75979700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e68170f80 con 0x7f6e680da570 2024-09-13T18:21:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75178700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e680d8620 0x7f6e6804aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75178700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e680d8620 0x7f6e6804aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47052/0 (socket says 172.21.15.40:47052) 2024-09-13T18:21:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75178700 1 -- 172.21.15.40:0/3950164724 learned_addr learned my addr 172.21.15.40:0/3950164724 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e74977700 1 --2- 172.21.15.40:0/3950164724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e680da190 0x7f6e680dd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e6ffff700 1 --2- 172.21.15.40:0/3950164724 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e680da570 0x7f6e6804a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75178700 1 -- 172.21.15.40:0/3950164724 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e680da570 msgr2=0x7f6e6804a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75178700 1 --2- 172.21.15.40:0/3950164724 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6e680da570 0x7f6e6804a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75178700 1 -- 172.21.15.40:0/3950164724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e680da190 msgr2=0x7f6e680dd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75178700 1 --2- 172.21.15.40:0/3950164724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e680da190 0x7f6e680dd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75178700 1 -- 172.21.15.40:0/3950164724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e680dc810 con 0x7f6e680d8620 2024-09-13T18:21:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e74977700 1 --2- 172.21.15.40:0/3950164724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e680da190 0x7f6e680dd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75178700 1 --2- 172.21.15.40:0/3950164724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e680d8620 0x7f6e6804aa80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6e7004f790 tx=0x7f6e70069e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e6d7fa700 1 -- 172.21.15.40:0/3950164724 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e70067930 con 0x7f6e680d8620 2024-09-13T18:21:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e6d7fa700 1 -- 172.21.15.40:0/3950164724 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e70072150 con 0x7f6e680d8620 2024-09-13T18:21:23.744 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75979700 1 -- 172.21.15.40:0/3950164724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e680dcaf0 con 0x7f6e680d8620 2024-09-13T18:21:23.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e6d7fa700 1 -- 172.21.15.40:0/3950164724 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e700724c0 con 0x7f6e680d8620 2024-09-13T18:21:23.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.733+0000 7f6e75979700 1 -- 172.21.15.40:0/3950164724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e680dd0b0 con 0x7f6e680d8620 2024-09-13T18:21:23.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.737+0000 7f6e75979700 1 -- 172.21.15.40:0/3950164724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6e68008600 con 0x7f6e680d8620 2024-09-13T18:21:23.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.737+0000 7f6e6d7fa700 1 -- 172.21.15.40:0/3950164724 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6e7009b070 con 0x7f6e680d8620 2024-09-13T18:21:23.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.737+0000 7f6e6d7fa700 1 --2- 172.21.15.40:0/3950164724 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e54066760 0x7f6e54068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.737+0000 7f6e6d7fa700 1 -- 172.21.15.40:0/3950164724 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6e700ffbb0 con 0x7f6e680d8620 2024-09-13T18:21:23.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.737+0000 7f6e74977700 1 --2- 172.21.15.40:0/3950164724 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e54066760 0x7f6e54068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.737+0000 7f6e6d7fa700 1 --2- 172.21.15.40:0/3950164724 >> [v2:172.21.15.40:6824/1926907414,v1:172.21.15.40:6825/1926907414] conn(0x7f6e54070040 0x7f6e540724e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.737+0000 7f6e6d7fa700 1 -- 172.21.15.40:0/3950164724 --> [v2:172.21.15.40:6824/1926907414,v1:172.21.15.40:6825/1926907414] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6e54072bb0 con 0x7f6e54070040 2024-09-13T18:21:23.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.737+0000 7f6e6d7fa700 1 -- 172.21.15.40:0/3950164724 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e700fff20 con 0x7f6e680d8620 2024-09-13T18:21:23.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.737+0000 7f6e6ffff700 1 --2- 172.21.15.40:0/3950164724 >> [v2:172.21.15.40:6824/1926907414,v1:172.21.15.40:6825/1926907414] conn(0x7f6e54070040 0x7f6e540724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.737+0000 7f6e74977700 1 --2- 172.21.15.40:0/3950164724 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e54066760 0x7f6e54068c20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6e640017a0 tx=0x7f6e64001610 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:23.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.737+0000 7f6e6ffff700 1 --2- 172.21.15.40:0/3950164724 >> [v2:172.21.15.40:6824/1926907414,v1:172.21.15.40:6825/1926907414] conn(0x7f6e54070040 0x7f6e540724e0 crc :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:23.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.737+0000 7f6e6d7fa700 1 -- 172.21.15.40:0/3950164724 <== osd.3 v2:172.21.15.40:6824/1926907414 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f6e54072bb0 con 0x7f6e54070040 2024-09-13T18:21:23.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.757+0000 7f6e75979700 1 -- 172.21.15.40:0/3950164724 --> [v2:172.21.15.40:6824/1926907414,v1:172.21.15.40:6825/1926907414] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6e68008320 con 0x7f6e54070040 2024-09-13T18:21:23.762 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.757+0000 7f6e6d7fa700 1 -- 172.21.15.40:0/3950164724 <== osd.3 v2:172.21.15.40:6824/1926907414 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6e5c009b50 con 0x7f6e54070040 2024-09-13T18:21:23.763 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:21:23.763 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:21:23.763 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:21:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.757+0000 7f6e75979700 1 -- 172.21.15.40:0/3950164724 >> [v2:172.21.15.40:6824/1926907414,v1:172.21.15.40:6825/1926907414] conn(0x7f6e54070040 msgr2=0x7f6e540724e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.757+0000 7f6e75979700 1 --2- 172.21.15.40:0/3950164724 >> [v2:172.21.15.40:6824/1926907414,v1:172.21.15.40:6825/1926907414] conn(0x7f6e54070040 0x7f6e540724e0 crc :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.757+0000 7f6e75979700 1 -- 172.21.15.40:0/3950164724 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e54066760 msgr2=0x7f6e54068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.757+0000 7f6e75979700 1 --2- 172.21.15.40:0/3950164724 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6e54066760 0x7f6e54068c20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6e640017a0 tx=0x7f6e64001610 comp rx=0 tx=0).stop 2024-09-13T18:21:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.757+0000 7f6e75979700 1 -- 172.21.15.40:0/3950164724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e680d8620 msgr2=0x7f6e6804aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.757+0000 7f6e75979700 1 --2- 172.21.15.40:0/3950164724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6e680d8620 0x7f6e6804aa80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6e7004f790 tx=0x7f6e70069e70 comp rx=0 tx=0).stop 2024-09-13T18:21:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.757+0000 7f6e75178700 1 -- 172.21.15.40:0/3950164724 reap_dead start 2024-09-13T18:21:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.757+0000 7f6e75979700 1 -- 172.21.15.40:0/3950164724 shutdown_connections 2024-09-13T18:21:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.757+0000 7f6e75979700 1 -- 172.21.15.40:0/3950164724 >> 172.21.15.40:0/3950164724 conn(0x7f6e68050e60 msgr2=0x7f6e680d46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:23.764 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.757+0000 7f6e75979700 1 -- 172.21.15.40:0/3950164724 shutdown_connections 2024-09-13T18:21:23.765 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.757+0000 7f6e75979700 1 -- 172.21.15.40:0/3950164724 wait complete. 2024-09-13T18:21:23.774 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-13T18:21:23.888 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.881+0000 7effe72e6700 1 Processor -- start 2024-09-13T18:21:23.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.881+0000 7effe72e6700 1 -- start start 2024-09-13T18:21:23.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.881+0000 7effe72e6700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7effe00cf1a0 0x7effe015a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe72e6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7effe00cc250 0x7effe00c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe72e6700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe015a7a0 con 0x7effe00cf580 2024-09-13T18:21:23.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe72e6700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe00cbdc0 con 0x7effe00cf1a0 2024-09-13T18:21:23.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe72e6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe0153f80 con 0x7effe00cc250 2024-09-13T18:21:23.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe62e4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7effe00cf1a0 0x7effe015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe62e4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7effe00cf1a0 0x7effe015a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47066/0 (socket says 172.21.15.40:47066) 2024-09-13T18:21:23.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe62e4700 1 -- 172.21.15.40:0/13616385 learned_addr learned my addr 172.21.15.40:0/13616385 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:23.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe62e4700 1 -- 172.21.15.40:0/13616385 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7effe00cc250 msgr2=0x7effe00c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:23.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe62e4700 1 --2- 172.21.15.40:0/13616385 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7effe00cc250 0x7effe00c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe62e4700 1 -- 172.21.15.40:0/13616385 >> v1:172.21.15.43:6789/0 conn(0x7effe00cf580 legacy=0x7effe00c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:23.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe62e4700 1 -- 172.21.15.40:0/13616385 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe00caed0 con 0x7effe00cf1a0 2024-09-13T18:21:23.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe62e4700 1 --2- 172.21.15.40:0/13616385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effe00cf1a0 0x7effe015a260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7effdc009910 tx=0x7effdc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6484a7b9f02e591e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:23.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe52e2700 1 -- 172.21.15.40:0/13616385 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc004230 con 0x7effe00cf1a0 2024-09-13T18:21:23.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe52e2700 1 -- 172.21.15.40:0/13616385 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effdc004390 con 0x7effe00cf1a0 2024-09-13T18:21:23.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe52e2700 1 -- 172.21.15.40:0/13616385 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc009e60 con 0x7effe00cf1a0 2024-09-13T18:21:23.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe72e6700 1 -- 172.21.15.40:0/13616385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effe00cf1a0 msgr2=0x7effe015a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe72e6700 1 --2- 172.21.15.40:0/13616385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effe00cf1a0 0x7effe015a260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7effdc009910 tx=0x7effdc009c20 comp rx=0 tx=0).stop 2024-09-13T18:21:23.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe72e6700 1 -- 172.21.15.40:0/13616385 shutdown_connections 2024-09-13T18:21:23.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe72e6700 1 --2- 172.21.15.40:0/13616385 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7effe00cc250 0x7effe00c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.892 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe72e6700 1 --2- 172.21.15.40:0/13616385 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effe00cf1a0 0x7effe015a260 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe72e6700 1 -- 172.21.15.40:0/13616385 >> 172.21.15.40:0/13616385 conn(0x7effe0066090 msgr2=0x7effe0066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:23.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe72e6700 1 -- 172.21.15.40:0/13616385 shutdown_connections 2024-09-13T18:21:23.893 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe72e6700 1 -- 172.21.15.40:0/13616385 wait complete. 2024-09-13T18:21:23.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe72e6700 1 Processor -- start 2024-09-13T18:21:23.894 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe72e6700 1 -- start start 2024-09-13T18:21:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe72e6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe00cc250 0x7effe01e5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe72e6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effe00cf1a0 0x7effe015abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe72e6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7effe00cf580 0x7effe01f25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe72e6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe01e6740 con 0x7effe00cf1a0 2024-09-13T18:21:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe72e6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe01598e0 con 0x7effe00cc250 2024-09-13T18:21:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe72e6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe01e6460 con 0x7effe00cf580 2024-09-13T18:21:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.885+0000 7effe5ae3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effe00cf1a0 0x7effe015abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effe5ae3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effe00cf1a0 0x7effe015abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47082/0 (socket says 172.21.15.40:47082) 2024-09-13T18:21:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effe5ae3700 1 -- 172.21.15.40:0/996101648 learned_addr learned my addr 172.21.15.40:0/996101648 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effe6ae5700 1 --2- 172.21.15.40:0/996101648 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7effe00cf580 0x7effe01f25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effe62e4700 1 --2- 172.21.15.40:0/996101648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe00cc250 0x7effe01e5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effe5ae3700 1 -- 172.21.15.40:0/996101648 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7effe00cf580 msgr2=0x7effe01f25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.895 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effe5ae3700 1 --2- 172.21.15.40:0/996101648 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7effe00cf580 0x7effe01f25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effe5ae3700 1 -- 172.21.15.40:0/996101648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe00cc250 msgr2=0x7effe01e5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effe5ae3700 1 --2- 172.21.15.40:0/996101648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe00cc250 0x7effe01e5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effe5ae3700 1 -- 172.21.15.40:0/996101648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effdc009570 con 0x7effe00cf1a0 2024-09-13T18:21:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effe6ae5700 1 --2- 172.21.15.40:0/996101648 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7effe00cf580 0x7effe01f25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:21:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effe5ae3700 1 --2- 172.21.15.40:0/996101648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effe00cf1a0 0x7effe015abb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7effd000b450 tx=0x7effd000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effd6ffd700 1 -- 172.21.15.40:0/996101648 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0004020 con 0x7effe00cf1a0 2024-09-13T18:21:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effd6ffd700 1 -- 172.21.15.40:0/996101648 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd0004180 con 0x7effe00cf1a0 2024-09-13T18:21:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effe72e6700 1 -- 172.21.15.40:0/996101648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe0159b80 con 0x7effe00cf1a0 2024-09-13T18:21:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effd6ffd700 1 -- 172.21.15.40:0/996101648 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0004530 con 0x7effe00cf1a0 2024-09-13T18:21:23.896 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effe72e6700 1 -- 172.21.15.40:0/996101648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe015a140 con 0x7effe00cf1a0 2024-09-13T18:21:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effd6ffd700 1 -- 172.21.15.40:0/996101648 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7effd00036a0 con 0x7effe00cf1a0 2024-09-13T18:21:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effe72e6700 1 -- 172.21.15.40:0/996101648 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe0067a40 con 0x7effe00cf1a0 2024-09-13T18:21:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effd6ffd700 1 --2- 172.21.15.40:0/996101648 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7effc4066730 0x7effc4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effd6ffd700 1 -- 172.21.15.40:0/996101648 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7effd0084110 con 0x7effe00cf1a0 2024-09-13T18:21:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effe62e4700 1 --2- 172.21.15.40:0/996101648 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7effc4066730 0x7effc4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:23.897 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.889+0000 7effe62e4700 1 --2- 172.21.15.40:0/996101648 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7effc4066730 0x7effc4068bf0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7effdc009910 tx=0x7effdc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:23.902 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:23.897+0000 7effd6ffd700 1 -- 172.21.15.40:0/996101648 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7effd0051030 con 0x7effe00cf1a0 2024-09-13T18:21:24.073 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.069+0000 7effe72e6700 1 -- 172.21.15.40:0/996101648 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7effe00b2f00 con 0x7effc4066730 2024-09-13T18:21:24.074 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.069+0000 7effd6ffd700 1 -- 172.21.15.40:0/996101648 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7effe00b2f00 con 0x7effc4066730 2024-09-13T18:21:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.069+0000 7effe72e6700 1 -- 172.21.15.40:0/996101648 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7effc4066730 msgr2=0x7effc4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.069+0000 7effe72e6700 1 --2- 172.21.15.40:0/996101648 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7effc4066730 0x7effc4068bf0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7effdc009910 tx=0x7effdc008030 comp rx=0 tx=0).stop 2024-09-13T18:21:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.069+0000 7effe72e6700 1 -- 172.21.15.40:0/996101648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effe00cf1a0 msgr2=0x7effe015abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.069+0000 7effe72e6700 1 --2- 172.21.15.40:0/996101648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effe00cf1a0 0x7effe015abb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7effd000b450 tx=0x7effd000b810 comp rx=0 tx=0).stop 2024-09-13T18:21:24.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.069+0000 7effe72e6700 1 -- 172.21.15.40:0/996101648 shutdown_connections 2024-09-13T18:21:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.069+0000 7effe72e6700 1 --2- 172.21.15.40:0/996101648 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7effc4066730 0x7effc4068bf0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.069+0000 7effe72e6700 1 --2- 172.21.15.40:0/996101648 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7effe00cf580 0x7effe01f25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.069+0000 7effe72e6700 1 --2- 172.21.15.40:0/996101648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7effe00cf1a0 0x7effe015abb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.069+0000 7effe72e6700 1 --2- 172.21.15.40:0/996101648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe00cc250 0x7effe01e5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.069+0000 7effe72e6700 1 -- 172.21.15.40:0/996101648 >> 172.21.15.40:0/996101648 conn(0x7effe0066090 msgr2=0x7effe00c7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.069+0000 7effe72e6700 1 -- 172.21.15.40:0/996101648 shutdown_connections 2024-09-13T18:21:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.069+0000 7effe72e6700 1 -- 172.21.15.40:0/996101648 wait complete. 2024-09-13T18:21:24.078 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 3 to deep-scrub 2024-09-13T18:21:24.089 INFO:tasks.ceph:Scrubbing osd.4 2024-09-13T18:21:24.089 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:21:24.199 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.193+0000 7fefda6e6700 1 Processor -- start 2024-09-13T18:21:24.200 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.193+0000 7fefda6e6700 1 -- start start 2024-09-13T18:21:24.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.193+0000 7fefda6e6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefd40c3140 0x7fefd415b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.193+0000 7fefda6e6700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd4066090 con 0x7fefd41515a0 2024-09-13T18:21:24.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.193+0000 7fefda6e6700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd40c5200 con 0x7fefd40c75d0 2024-09-13T18:21:24.201 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.193+0000 7fefda6e6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd40c5850 con 0x7fefd40c3140 2024-09-13T18:21:24.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.193+0000 7fefd96e4700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fefd41515a0 0x7fefd40c2a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51482/0 (socket says 172.21.15.40:51482) 2024-09-13T18:21:24.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.193+0000 7fefd96e4700 1 -- 172.21.15.40:0/965979010 learned_addr learned my addr 172.21.15.40:0/965979010 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:24.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefd8ee3700 1 --2- 172.21.15.40:0/965979010 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefd40c3140 0x7fefd415b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefcbfff700 1 -- 172.21.15.40:0/965979010 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2197810898 0 0) 0x7fefd4066090 con 0x7fefd41515a0 2024-09-13T18:21:24.202 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefcbfff700 1 -- 172.21.15.40:0/965979010 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefc00035d0 con 0x7fefd41515a0 2024-09-13T18:21:24.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefcbfff700 1 -- 172.21.15.40:0/965979010 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3997562774 0 0) 0x7fefc00035d0 con 0x7fefd41515a0 2024-09-13T18:21:24.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefcbfff700 1 -- 172.21.15.40:0/965979010 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefd40c3140 msgr2=0x7fefd415b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:24.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefcbfff700 1 --2- 172.21.15.40:0/965979010 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefd40c3140 0x7fefd415b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefcbfff700 1 -- 172.21.15.40:0/965979010 >> v1:172.21.15.43:6789/0 conn(0x7fefd40c75d0 legacy=0x7fefd406d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefcbfff700 1 -- 172.21.15.40:0/965979010 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd40662c0 con 0x7fefd41515a0 2024-09-13T18:21:24.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefcbfff700 1 -- 172.21.15.40:0/965979010 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fefd0001670 con 0x7fefd41515a0 2024-09-13T18:21:24.204 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefcbfff700 1 -- 172.21.15.40:0/965979010 >> v1:172.21.15.40:6789/0 conn(0x7fefd41515a0 legacy=0x7fefd40c2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefcbfff700 1 --2- 172.21.15.40:0/965979010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc0003a50 0x7fefc0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefcbfff700 1 --2- 172.21.15.40:0/965979010 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefd40c3140 0x7fefc0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefcbfff700 1 --2- 172.21.15.40:0/965979010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefc0004db0 0x7fefc0005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefcbfff700 1 -- 172.21.15.40:0/965979010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd40c5850 con 0x7fefc0003a50 2024-09-13T18:21:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefcbfff700 1 -- 172.21.15.40:0/965979010 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc000bae0 con 0x7fefc0004db0 2024-09-13T18:21:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefcbfff700 1 -- 172.21.15.40:0/965979010 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc000bca0 con 0x7fefd40c3140 2024-09-13T18:21:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefd96e4700 1 --2- 172.21.15.40:0/965979010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc0003a50 0x7fefc0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefd8ee3700 1 --2- 172.21.15.40:0/965979010 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefd40c3140 0x7fefc0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefd9ee5700 1 --2- 172.21.15.40:0/965979010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefc0004db0 0x7fefc0005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefd96e4700 1 -- 172.21.15.40:0/965979010 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefd40c3140 msgr2=0x7fefc0004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefd96e4700 1 --2- 172.21.15.40:0/965979010 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefd40c3140 0x7fefc0004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefd96e4700 1 -- 172.21.15.40:0/965979010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefc0004db0 msgr2=0x7fefc0005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefd96e4700 1 --2- 172.21.15.40:0/965979010 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefc0004db0 0x7fefc0005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefd96e4700 1 -- 172.21.15.40:0/965979010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc00035d0 con 0x7fefc0003a50 2024-09-13T18:21:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefd96e4700 1 --2- 172.21.15.40:0/965979010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc0003a50 0x7fefc0003e30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fefd000c2f0 tx=0x7fefc00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a981ff11851c6a9f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefcbfff700 1 -- 172.21.15.40:0/965979010 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd000ce30 con 0x7fefc0003a50 2024-09-13T18:21:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefcbfff700 1 -- 172.21.15.40:0/965979010 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd0005980 con 0x7fefc0003a50 2024-09-13T18:21:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefcbfff700 1 -- 172.21.15.40:0/965979010 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd0005d10 con 0x7fefc0003a50 2024-09-13T18:21:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefda6e6700 1 -- 172.21.15.40:0/965979010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc0003a50 msgr2=0x7fefc0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefda6e6700 1 --2- 172.21.15.40:0/965979010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc0003a50 0x7fefc0003e30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fefd000c2f0 tx=0x7fefc00058a0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefd9ee5700 1 -- 172.21.15.40:0/965979010 reap_dead start 2024-09-13T18:21:24.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefda6e6700 1 -- 172.21.15.40:0/965979010 shutdown_connections 2024-09-13T18:21:24.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefda6e6700 1 -- 172.21.15.40:0/965979010 >> 172.21.15.40:0/965979010 conn(0x7fefd40cb450 msgr2=0x7fefd40cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:24.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefda6e6700 1 -- 172.21.15.40:0/965979010 shutdown_connections 2024-09-13T18:21:24.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.197+0000 7fefda6e6700 1 -- 172.21.15.40:0/965979010 wait complete. 2024-09-13T18:21:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefda6e6700 1 Processor -- start 2024-09-13T18:21:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefda6e6700 1 -- start start 2024-09-13T18:21:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefda6e6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc0003a50 0x7fefd4155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefda6e6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefd415a9e0 0x7fefd4151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefda6e6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefd4157970 0x7fefd4157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefda6e6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd4155f40 con 0x7fefc0003a50 2024-09-13T18:21:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefda6e6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd414a5d0 con 0x7fefd4157970 2024-09-13T18:21:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefda6e6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd41eb990 con 0x7fefd415a9e0 2024-09-13T18:21:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefd96e4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc0003a50 0x7fefd4155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefd96e4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc0003a50 0x7fefd4155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47094/0 (socket says 172.21.15.40:47094) 2024-09-13T18:21:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefd96e4700 1 -- 172.21.15.40:0/2757887257 learned_addr learned my addr 172.21.15.40:0/2757887257 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefd8ee3700 1 --2- 172.21.15.40:0/2757887257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefd415a9e0 0x7fefd4151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefd9ee5700 1 --2- 172.21.15.40:0/2757887257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefd4157970 0x7fefd4157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefd96e4700 1 -- 172.21.15.40:0/2757887257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefd415a9e0 msgr2=0x7fefd4151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefd96e4700 1 --2- 172.21.15.40:0/2757887257 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fefd415a9e0 0x7fefd4151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefd96e4700 1 -- 172.21.15.40:0/2757887257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefd4157970 msgr2=0x7fefd4157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefd96e4700 1 --2- 172.21.15.40:0/2757887257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefd4157970 0x7fefd4157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefd96e4700 1 -- 172.21.15.40:0/2757887257 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd41ebc30 con 0x7fefc0003a50 2024-09-13T18:21:24.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefd9ee5700 1 --2- 172.21.15.40:0/2757887257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefd4157970 0x7fefd4157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefd96e4700 1 --2- 172.21.15.40:0/2757887257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc0003a50 0x7fefd4155a00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fefd0001930 tx=0x7fefd0001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefc9ffb700 1 -- 172.21.15.40:0/2757887257 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd0001060 con 0x7fefc0003a50 2024-09-13T18:21:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefc9ffb700 1 -- 172.21.15.40:0/2757887257 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd0004aa0 con 0x7fefc0003a50 2024-09-13T18:21:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefda6e6700 1 -- 172.21.15.40:0/2757887257 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd4149ca0 con 0x7fefc0003a50 2024-09-13T18:21:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefc9ffb700 1 -- 172.21.15.40:0/2757887257 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd0004e50 con 0x7fefc0003a50 2024-09-13T18:21:24.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.201+0000 7fefda6e6700 1 -- 172.21.15.40:0/2757887257 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd414a260 con 0x7fefc0003a50 2024-09-13T18:21:24.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.205+0000 7fefda6e6700 1 -- 172.21.15.40:0/2757887257 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fefd4151980 con 0x7fefc0003a50 2024-09-13T18:21:24.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.205+0000 7fefc9ffb700 1 -- 172.21.15.40:0/2757887257 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fefd00070d0 con 0x7fefc0003a50 2024-09-13T18:21:24.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.205+0000 7fefc9ffb700 1 --2- 172.21.15.40:0/2757887257 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fefb8066760 0x7fefb8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.205+0000 7fefc9ffb700 1 -- 172.21.15.40:0/2757887257 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fefd00859d0 con 0x7fefc0003a50 2024-09-13T18:21:24.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.205+0000 7fefd8ee3700 1 --2- 172.21.15.40:0/2757887257 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fefb8066760 0x7fefb8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.205+0000 7fefc9ffb700 1 --2- 172.21.15.40:0/2757887257 >> [v2:172.21.15.43:6824/3105831381,v1:172.21.15.43:6825/3105831381] conn(0x7fefb8070040 0x7fefb80724e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.205+0000 7fefc9ffb700 1 -- 172.21.15.40:0/2757887257 --> [v2:172.21.15.43:6824/3105831381,v1:172.21.15.43:6825/3105831381] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fefb8072bb0 con 0x7fefb8070040 2024-09-13T18:21:24.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.205+0000 7fefc9ffb700 1 -- 172.21.15.40:0/2757887257 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefd008a2c0 con 0x7fefc0003a50 2024-09-13T18:21:24.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.205+0000 7fefd9ee5700 1 --2- 172.21.15.40:0/2757887257 >> [v2:172.21.15.43:6824/3105831381,v1:172.21.15.43:6825/3105831381] conn(0x7fefb8070040 0x7fefb80724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.205+0000 7fefd8ee3700 1 --2- 172.21.15.40:0/2757887257 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fefb8066760 0x7fefb8068c20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fefd4154f00 tx=0x7fefc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:24.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.205+0000 7fefd9ee5700 1 --2- 172.21.15.40:0/2757887257 >> [v2:172.21.15.43:6824/3105831381,v1:172.21.15.43:6825/3105831381] conn(0x7fefb8070040 0x7fefb80724e0 crc :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:24.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.209+0000 7fefc9ffb700 1 -- 172.21.15.40:0/2757887257 <== osd.4 v2:172.21.15.43:6824/3105831381 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fefb8072bb0 con 0x7fefb8070040 2024-09-13T18:21:24.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.225+0000 7fefda6e6700 1 -- 172.21.15.40:0/2757887257 --> [v2:172.21.15.43:6824/3105831381,v1:172.21.15.43:6825/3105831381] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fefd40c5470 con 0x7fefb8070040 2024-09-13T18:21:24.233 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.225+0000 7fefc9ffb700 1 -- 172.21.15.40:0/2757887257 <== osd.4 v2:172.21.15.43:6824/3105831381 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fefcc001db0 con 0x7fefb8070040 2024-09-13T18:21:24.233 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:21:24.234 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:21:24.234 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:21:24.234 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.229+0000 7fefda6e6700 1 -- 172.21.15.40:0/2757887257 >> [v2:172.21.15.43:6824/3105831381,v1:172.21.15.43:6825/3105831381] conn(0x7fefb8070040 msgr2=0x7fefb80724e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.229+0000 7fefda6e6700 1 --2- 172.21.15.40:0/2757887257 >> [v2:172.21.15.43:6824/3105831381,v1:172.21.15.43:6825/3105831381] conn(0x7fefb8070040 0x7fefb80724e0 crc :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.229+0000 7fefda6e6700 1 -- 172.21.15.40:0/2757887257 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fefb8066760 msgr2=0x7fefb8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.229+0000 7fefda6e6700 1 --2- 172.21.15.40:0/2757887257 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fefb8066760 0x7fefb8068c20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fefd4154f00 tx=0x7fefc4008040 comp rx=0 tx=0).stop 2024-09-13T18:21:24.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.229+0000 7fefda6e6700 1 -- 172.21.15.40:0/2757887257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc0003a50 msgr2=0x7fefd4155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.229+0000 7fefda6e6700 1 --2- 172.21.15.40:0/2757887257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fefc0003a50 0x7fefd4155a00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fefd0001930 tx=0x7fefd0001a00 comp rx=0 tx=0).stop 2024-09-13T18:21:24.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.229+0000 7fefd9ee5700 1 -- 172.21.15.40:0/2757887257 reap_dead start 2024-09-13T18:21:24.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.229+0000 7fefda6e6700 1 -- 172.21.15.40:0/2757887257 shutdown_connections 2024-09-13T18:21:24.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.229+0000 7fefda6e6700 1 -- 172.21.15.40:0/2757887257 >> 172.21.15.40:0/2757887257 conn(0x7fefd40cb450 msgr2=0x7fefd414df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:24.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.229+0000 7fefda6e6700 1 -- 172.21.15.40:0/2757887257 shutdown_connections 2024-09-13T18:21:24.235 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.229+0000 7fefda6e6700 1 -- 172.21.15.40:0/2757887257 wait complete. 2024-09-13T18:21:24.249 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-13T18:21:24.353 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df6fac700 1 Processor -- start 2024-09-13T18:21:24.354 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df6fac700 1 -- start start 2024-09-13T18:21:24.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df6fac700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6df015ce10 0x7f6df00c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df6fac700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6df006d540 0x7f6df0070fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df6fac700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df006d970 con 0x7f6df015d1f0 2024-09-13T18:21:24.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df6fac700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df0071860 con 0x7f6df006d540 2024-09-13T18:21:24.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df6fac700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df00c5e50 con 0x7f6df015ce10 2024-09-13T18:21:24.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df57a9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6df006d540 0x7f6df0070fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df57a9700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6df006d540 0x7f6df0070fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47108/0 (socket says 172.21.15.40:47108) 2024-09-13T18:21:24.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df57a9700 1 -- 172.21.15.40:0/3502872294 learned_addr learned my addr 172.21.15.40:0/3502872294 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:24.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df5faa700 1 --2- 172.21.15.40:0/3502872294 >> v2:172.21.15.43:3300/0 conn(0x7f6df015ce10 0x7f6df00c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df57a9700 1 -- 172.21.15.40:0/3502872294 >> v2:172.21.15.43:3300/0 conn(0x7f6df015ce10 msgr2=0x7f6df00c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:24.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df57a9700 1 --2- 172.21.15.40:0/3502872294 >> v2:172.21.15.43:3300/0 conn(0x7f6df015ce10 0x7f6df00c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.356 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df57a9700 1 -- 172.21.15.40:0/3502872294 >> v1:172.21.15.43:6789/0 conn(0x7f6df015d1f0 legacy=0x7f6df01533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df57a9700 1 -- 172.21.15.40:0/3502872294 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df00c6130 con 0x7f6df006d540 2024-09-13T18:21:24.357 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df57a9700 1 --2- 172.21.15.40:0/3502872294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6df006d540 0x7f6df0070fd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6df006db00 tx=0x7f6de0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8920b312df18d4b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:24.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df4fa8700 1 -- 172.21.15.40:0/3502872294 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de0004230 con 0x7f6df006d540 2024-09-13T18:21:24.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df4fa8700 1 -- 172.21.15.40:0/3502872294 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de0004390 con 0x7f6df006d540 2024-09-13T18:21:24.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df4fa8700 1 -- 172.21.15.40:0/3502872294 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de00036c0 con 0x7f6df006d540 2024-09-13T18:21:24.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df6fac700 1 -- 172.21.15.40:0/3502872294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6df006d540 msgr2=0x7f6df0070fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df6fac700 1 --2- 172.21.15.40:0/3502872294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6df006d540 0x7f6df0070fd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6df006db00 tx=0x7f6de0009bf0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df6fac700 1 -- 172.21.15.40:0/3502872294 shutdown_connections 2024-09-13T18:21:24.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df6fac700 1 --2- 172.21.15.40:0/3502872294 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6df006d540 0x7f6df0070fd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df6fac700 1 --2- 172.21.15.40:0/3502872294 >> v2:172.21.15.43:3300/0 conn(0x7f6df015ce10 0x7f6df00c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df6fac700 1 -- 172.21.15.40:0/3502872294 >> 172.21.15.40:0/3502872294 conn(0x7f6df00cb450 msgr2=0x7f6df00cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:24.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df6fac700 1 -- 172.21.15.40:0/3502872294 shutdown_connections 2024-09-13T18:21:24.358 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.349+0000 7f6df6fac700 1 -- 172.21.15.40:0/3502872294 wait complete. 2024-09-13T18:21:24.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df6fac700 1 Processor -- start 2024-09-13T18:21:24.359 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df6fac700 1 -- start start 2024-09-13T18:21:24.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df6fac700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6df006d540 0x7f6df0146de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df6fac700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df015ce10 0x7f6df01f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df6fac700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6df015d1f0 0x7f6df0145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df6fac700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df01477e0 con 0x7f6df015d1f0 2024-09-13T18:21:24.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df6fac700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df01f3300 con 0x7f6df015ce10 2024-09-13T18:21:24.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df6fac700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df015a790 con 0x7f6df006d540 2024-09-13T18:21:24.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df67ab700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6df015d1f0 0x7f6df0145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df67ab700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6df015d1f0 0x7f6df0145ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47120/0 (socket says 172.21.15.40:47120) 2024-09-13T18:21:24.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df67ab700 1 -- 172.21.15.40:0/57311843 learned_addr learned my addr 172.21.15.40:0/57311843 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:24.362 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df5faa700 1 --2- 172.21.15.40:0/57311843 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6df006d540 0x7f6df0146de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df57a9700 1 --2- 172.21.15.40:0/57311843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df015ce10 0x7f6df01f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df67ab700 1 -- 172.21.15.40:0/57311843 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6df006d540 msgr2=0x7f6df0146de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df67ab700 1 --2- 172.21.15.40:0/57311843 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6df006d540 0x7f6df0146de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df67ab700 1 -- 172.21.15.40:0/57311843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df015ce10 msgr2=0x7f6df01f28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df67ab700 1 --2- 172.21.15.40:0/57311843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df015ce10 0x7f6df01f28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df67ab700 1 -- 172.21.15.40:0/57311843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de0009570 con 0x7f6df015d1f0 2024-09-13T18:21:24.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df5faa700 1 --2- 172.21.15.40:0/57311843 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6df006d540 0x7f6df0146de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:24.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df57a9700 1 --2- 172.21.15.40:0/57311843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df015ce10 0x7f6df01f28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:24.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6df67ab700 1 --2- 172.21.15.40:0/57311843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6df015d1f0 0x7f6df0145ed0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6de800cc50 tx=0x7f6de800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:24.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.353+0000 7f6de67fc700 1 -- 172.21.15.40:0/57311843 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de800ad80 con 0x7f6df015d1f0 2024-09-13T18:21:24.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.357+0000 7f6de67fc700 1 -- 172.21.15.40:0/57311843 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de800ce50 con 0x7f6df015d1f0 2024-09-13T18:21:24.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.357+0000 7f6df6fac700 1 -- 172.21.15.40:0/57311843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df015aa70 con 0x7f6df015d1f0 2024-09-13T18:21:24.363 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.357+0000 7f6de67fc700 1 -- 172.21.15.40:0/57311843 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de8003c60 con 0x7f6df015d1f0 2024-09-13T18:21:24.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.357+0000 7f6df6fac700 1 -- 172.21.15.40:0/57311843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df00c4d90 con 0x7f6df015d1f0 2024-09-13T18:21:24.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.357+0000 7f6df6fac700 1 -- 172.21.15.40:0/57311843 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df0066570 con 0x7f6df015d1f0 2024-09-13T18:21:24.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.357+0000 7f6de67fc700 1 -- 172.21.15.40:0/57311843 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6de80050d0 con 0x7f6df015d1f0 2024-09-13T18:21:24.364 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.357+0000 7f6de67fc700 1 --2- 172.21.15.40:0/57311843 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6dd4066730 0x7f6dd4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.357+0000 7f6de67fc700 1 -- 172.21.15.40:0/57311843 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6de8084590 con 0x7f6df015d1f0 2024-09-13T18:21:24.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.357+0000 7f6df5faa700 1 --2- 172.21.15.40:0/57311843 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6dd4066730 0x7f6dd4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.365 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.357+0000 7f6df5faa700 1 --2- 172.21.15.40:0/57311843 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6dd4066730 0x7f6dd4068bf0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6dec009c40 tx=0x7f6dec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:24.372 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.365+0000 7f6de67fc700 1 -- 172.21.15.40:0/57311843 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6de80515d0 con 0x7f6df015d1f0 2024-09-13T18:21:24.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.537+0000 7f6df6fac700 1 -- 172.21.15.40:0/57311843 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6df006c000 con 0x7f6dd4066730 2024-09-13T18:21:24.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.537+0000 7f6de67fc700 1 -- 172.21.15.40:0/57311843 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6df006c000 con 0x7f6dd4066730 2024-09-13T18:21:24.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.541+0000 7f6df6fac700 1 -- 172.21.15.40:0/57311843 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6dd4066730 msgr2=0x7f6dd4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.541+0000 7f6df6fac700 1 --2- 172.21.15.40:0/57311843 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6dd4066730 0x7f6dd4068bf0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6dec009c40 tx=0x7f6dec008040 comp rx=0 tx=0).stop 2024-09-13T18:21:24.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.541+0000 7f6df6fac700 1 -- 172.21.15.40:0/57311843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6df015d1f0 msgr2=0x7f6df0145ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.541+0000 7f6df6fac700 1 --2- 172.21.15.40:0/57311843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6df015d1f0 0x7f6df0145ed0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6de800cc50 tx=0x7f6de800a340 comp rx=0 tx=0).stop 2024-09-13T18:21:24.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.541+0000 7f6df6fac700 1 -- 172.21.15.40:0/57311843 shutdown_connections 2024-09-13T18:21:24.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.541+0000 7f6df6fac700 1 --2- 172.21.15.40:0/57311843 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6df015d1f0 0x7f6df0145ed0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.541+0000 7f6df6fac700 1 --2- 172.21.15.40:0/57311843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6df015ce10 0x7f6df01f28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.541+0000 7f6df6fac700 1 --2- 172.21.15.40:0/57311843 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f6dd4066730 0x7f6dd4068bf0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.541+0000 7f6df6fac700 1 --2- 172.21.15.40:0/57311843 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6df006d540 0x7f6df0146de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.541+0000 7f6df6fac700 1 -- 172.21.15.40:0/57311843 >> 172.21.15.40:0/57311843 conn(0x7f6df00cb450 msgr2=0x7f6df0151cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:24.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.541+0000 7f6df6fac700 1 -- 172.21.15.40:0/57311843 shutdown_connections 2024-09-13T18:21:24.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.541+0000 7f6df6fac700 1 -- 172.21.15.40:0/57311843 wait complete. 2024-09-13T18:21:24.553 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 4 to deep-scrub 2024-09-13T18:21:24.561 INFO:tasks.ceph:Scrubbing osd.5 2024-09-13T18:21:24.561 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:21:24.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.669+0000 7f37a66ce700 1 Processor -- start 2024-09-13T18:21:24.676 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.669+0000 7f37a66ce700 1 -- start start 2024-09-13T18:21:24.678 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.669+0000 7f37a66ce700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37a00c3f80 0x7f37a0158950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.669+0000 7f37a66ce700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37a00b3060 0x7f37a0158e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.669+0000 7f37a66ce700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a00c44d0 con 0x7f37a00c3ba0 2024-09-13T18:21:24.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.669+0000 7f37a66ce700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a00c29f0 con 0x7f37a00c3f80 2024-09-13T18:21:24.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.669+0000 7f37a66ce700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a00c4cc0 con 0x7f37a00b3060 2024-09-13T18:21:24.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f37a4ecb700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f37a00c3f80 0x7f37a0158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f37a56cc700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f37a00c3ba0 0x7f37a00c8be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51496/0 (socket says 172.21.15.40:51496) 2024-09-13T18:21:24.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f37a56cc700 1 -- 172.21.15.40:0/3301075372 learned_addr learned my addr 172.21.15.40:0/3301075372 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:24.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f37a5ecd700 1 --2- 172.21.15.40:0/3301075372 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37a00b3060 0x7f37a0158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f37a4ecb700 1 -- 172.21.15.40:0/3301075372 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37a00b3060 msgr2=0x7f37a0158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:24.679 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f37a4ecb700 1 --2- 172.21.15.40:0/3301075372 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37a00b3060 0x7f37a0158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f37a4ecb700 1 -- 172.21.15.40:0/3301075372 >> v1:172.21.15.40:6789/0 conn(0x7f37a00c3ba0 legacy=0x7f37a00c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f37a4ecb700 1 -- 172.21.15.40:0/3301075372 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a00c2e80 con 0x7f37a00c3f80 2024-09-13T18:21:24.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f37a4ecb700 1 --2- 172.21.15.40:0/3301075372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a00c3f80 0x7f37a0158950 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3790009850 tx=0x7f3790009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d3a2e8518d7b81 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:24.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f3797fff700 1 -- 172.21.15.40:0/3301075372 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3790011830 con 0x7f37a00c3f80 2024-09-13T18:21:24.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f3797fff700 1 -- 172.21.15.40:0/3301075372 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3790011990 con 0x7f37a00c3f80 2024-09-13T18:21:24.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f3797fff700 1 -- 172.21.15.40:0/3301075372 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3790011d20 con 0x7f37a00c3f80 2024-09-13T18:21:24.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f37a66ce700 1 -- 172.21.15.40:0/3301075372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a00c3f80 msgr2=0x7f37a0158950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f37a66ce700 1 --2- 172.21.15.40:0/3301075372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a00c3f80 0x7f37a0158950 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3790009850 tx=0x7f3790009b60 comp rx=0 tx=0).stop 2024-09-13T18:21:24.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f37a66ce700 1 -- 172.21.15.40:0/3301075372 shutdown_connections 2024-09-13T18:21:24.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f37a66ce700 1 --2- 172.21.15.40:0/3301075372 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37a00b3060 0x7f37a0158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f37a66ce700 1 --2- 172.21.15.40:0/3301075372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a00c3f80 0x7f37a0158950 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.681 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f37a66ce700 1 -- 172.21.15.40:0/3301075372 >> 172.21.15.40:0/3301075372 conn(0x7f37a006f810 msgr2=0x7f37a006fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:24.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f37a66ce700 1 -- 172.21.15.40:0/3301075372 shutdown_connections 2024-09-13T18:21:24.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f37a66ce700 1 -- 172.21.15.40:0/3301075372 wait complete. 2024-09-13T18:21:24.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f37a66ce700 1 Processor -- start 2024-09-13T18:21:24.682 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.673+0000 7f37a66ce700 1 -- start start 2024-09-13T18:21:24.684 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a66ce700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37a00b3060 0x7f37a0153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a66ce700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37a00c3ba0 0x7f37a014f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a66ce700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a00c3f80 0x7f37a01e1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a66ce700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a0154100 con 0x7f37a00c3f80 2024-09-13T18:21:24.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a66ce700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a01e06a0 con 0x7f37a00b3060 2024-09-13T18:21:24.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a66ce700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a01e1680 con 0x7f37a00c3ba0 2024-09-13T18:21:24.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a5ecd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a00c3f80 0x7f37a01e1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a5ecd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a00c3f80 0x7f37a01e1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47128/0 (socket says 172.21.15.40:47128) 2024-09-13T18:21:24.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a5ecd700 1 -- 172.21.15.40:0/2207028827 learned_addr learned my addr 172.21.15.40:0/2207028827 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:24.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a56cc700 1 --2- 172.21.15.40:0/2207028827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37a00b3060 0x7f37a0153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a4ecb700 1 --2- 172.21.15.40:0/2207028827 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37a00c3ba0 0x7f37a014f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a5ecd700 1 -- 172.21.15.40:0/2207028827 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37a00c3ba0 msgr2=0x7f37a014f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a5ecd700 1 --2- 172.21.15.40:0/2207028827 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37a00c3ba0 0x7f37a014f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.685 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a5ecd700 1 -- 172.21.15.40:0/2207028827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37a00b3060 msgr2=0x7f37a0153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a5ecd700 1 --2- 172.21.15.40:0/2207028827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37a00b3060 0x7f37a0153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a5ecd700 1 -- 172.21.15.40:0/2207028827 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37900094b0 con 0x7f37a00c3f80 2024-09-13T18:21:24.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a56cc700 1 --2- 172.21.15.40:0/2207028827 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37a00b3060 0x7f37a0153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:24.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a5ecd700 1 --2- 172.21.15.40:0/2207028827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a00c3f80 0x7f37a01e1b10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3798007e40 tx=0x7f379800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:24.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f3795ffb700 1 -- 172.21.15.40:0/2207028827 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379800eea0 con 0x7f37a00c3f80 2024-09-13T18:21:24.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f3795ffb700 1 -- 172.21.15.40:0/2207028827 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3798014420 con 0x7f37a00c3f80 2024-09-13T18:21:24.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a66ce700 1 -- 172.21.15.40:0/2207028827 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a0154c50 con 0x7f37a00c3f80 2024-09-13T18:21:24.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f3795ffb700 1 -- 172.21.15.40:0/2207028827 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37980147b0 con 0x7f37a00c3f80 2024-09-13T18:21:24.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a66ce700 1 -- 172.21.15.40:0/2207028827 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a01e00e0 con 0x7f37a00c3f80 2024-09-13T18:21:24.686 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.677+0000 7f37a66ce700 1 -- 172.21.15.40:0/2207028827 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f37a00c23a0 con 0x7f37a00c3f80 2024-09-13T18:21:24.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.681+0000 7f3795ffb700 1 -- 172.21.15.40:0/2207028827 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3798014910 con 0x7f37a00c3f80 2024-09-13T18:21:24.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.681+0000 7f3795ffb700 1 --2- 172.21.15.40:0/2207028827 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3784066730 0x7f3784068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.681+0000 7f3795ffb700 1 -- 172.21.15.40:0/2207028827 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3798084ef0 con 0x7f37a00c3f80 2024-09-13T18:21:24.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.681+0000 7f37a56cc700 1 --2- 172.21.15.40:0/2207028827 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3784066730 0x7f3784068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.681+0000 7f3795ffb700 1 --2- 172.21.15.40:0/2207028827 >> [v2:172.21.15.43:6800/858063952,v1:172.21.15.43:6801/858063952] conn(0x7f3784070010 0x7f37840724b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.681+0000 7f3795ffb700 1 -- 172.21.15.40:0/2207028827 --> [v2:172.21.15.43:6800/858063952,v1:172.21.15.43:6801/858063952] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3784072b80 con 0x7f3784070010 2024-09-13T18:21:24.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.681+0000 7f3795ffb700 1 -- 172.21.15.40:0/2207028827 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37980874d0 con 0x7f37a00c3f80 2024-09-13T18:21:24.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.681+0000 7f37a56cc700 1 --2- 172.21.15.40:0/2207028827 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3784066730 0x7f3784068bf0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f379c007fd0 tx=0x7f379c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:24.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.681+0000 7f37a4ecb700 1 --2- 172.21.15.40:0/2207028827 >> [v2:172.21.15.43:6800/858063952,v1:172.21.15.43:6801/858063952] conn(0x7f3784070010 0x7f37840724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.688 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.681+0000 7f37a4ecb700 1 --2- 172.21.15.40:0/2207028827 >> [v2:172.21.15.43:6800/858063952,v1:172.21.15.43:6801/858063952] conn(0x7f3784070010 0x7f37840724b0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:24.689 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.685+0000 7f3795ffb700 1 -- 172.21.15.40:0/2207028827 <== osd.5 v2:172.21.15.43:6800/858063952 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f3784072b80 con 0x7f3784070010 2024-09-13T18:21:24.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.701+0000 7f37a66ce700 1 -- 172.21.15.40:0/2207028827 --> [v2:172.21.15.43:6800/858063952,v1:172.21.15.43:6801/858063952] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f37a00c49e0 con 0x7f3784070010 2024-09-13T18:21:24.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.701+0000 7f3795ffb700 1 -- 172.21.15.40:0/2207028827 <== osd.5 v2:172.21.15.43:6800/858063952 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f3790011b80 con 0x7f3784070010 2024-09-13T18:21:24.707 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:21:24.707 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:21:24.707 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:21:24.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.701+0000 7f37a66ce700 1 -- 172.21.15.40:0/2207028827 >> [v2:172.21.15.43:6800/858063952,v1:172.21.15.43:6801/858063952] conn(0x7f3784070010 msgr2=0x7f37840724b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.701+0000 7f37a66ce700 1 --2- 172.21.15.40:0/2207028827 >> [v2:172.21.15.43:6800/858063952,v1:172.21.15.43:6801/858063952] conn(0x7f3784070010 0x7f37840724b0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.701+0000 7f37a66ce700 1 -- 172.21.15.40:0/2207028827 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3784066730 msgr2=0x7f3784068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.701+0000 7f37a66ce700 1 --2- 172.21.15.40:0/2207028827 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f3784066730 0x7f3784068bf0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f379c007fd0 tx=0x7f379c009040 comp rx=0 tx=0).stop 2024-09-13T18:21:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.701+0000 7f37a66ce700 1 -- 172.21.15.40:0/2207028827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a00c3f80 msgr2=0x7f37a01e1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.701+0000 7f37a66ce700 1 --2- 172.21.15.40:0/2207028827 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37a00c3f80 0x7f37a01e1b10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3798007e40 tx=0x7f379800e3c0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.701+0000 7f37a5ecd700 1 -- 172.21.15.40:0/2207028827 reap_dead start 2024-09-13T18:21:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.701+0000 7f37a66ce700 1 -- 172.21.15.40:0/2207028827 shutdown_connections 2024-09-13T18:21:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.701+0000 7f37a66ce700 1 -- 172.21.15.40:0/2207028827 >> 172.21.15.40:0/2207028827 conn(0x7f37a006f810 msgr2=0x7f37a00c6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.701+0000 7f37a66ce700 1 -- 172.21.15.40:0/2207028827 shutdown_connections 2024-09-13T18:21:24.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.701+0000 7f37a66ce700 1 -- 172.21.15.40:0/2207028827 wait complete. 2024-09-13T18:21:24.717 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-13T18:21:24.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf6e77b700 1 Processor -- start 2024-09-13T18:21:24.827 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf6e77b700 1 -- start start 2024-09-13T18:21:24.831 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf6e77b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcf680cf1a0 0x7fcf6815a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf6e77b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf680cf580 0x7fcf680c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf6e77b700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf6815a7a0 con 0x7fcf680cc250 2024-09-13T18:21:24.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf6e77b700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf680cbdc0 con 0x7fcf680cf1a0 2024-09-13T18:21:24.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf6e77b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf68153f80 con 0x7fcf680cf580 2024-09-13T18:21:24.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf6df7a700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fcf680cc250 0x7fcf680c8aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51504/0 (socket says 172.21.15.40:51504) 2024-09-13T18:21:24.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf6df7a700 1 -- 172.21.15.40:0/1332672089 learned_addr learned my addr 172.21.15.40:0/1332672089 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:24.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf6d779700 1 --2- 172.21.15.40:0/1332672089 >> v2:172.21.15.43:3300/0 conn(0x7fcf680cf1a0 0x7fcf6815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf6cf78700 1 --2- 172.21.15.40:0/1332672089 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf680cf580 0x7fcf680c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.832 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf5ffff700 1 -- 172.21.15.40:0/1332672089 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129247081 0 0) 0x7fcf60001490 con 0x7fcf680cc250 2024-09-13T18:21:24.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf5ffff700 1 -- 172.21.15.40:0/1332672089 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf540035d0 con 0x7fcf680cc250 2024-09-13T18:21:24.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf5ffff700 1 -- 172.21.15.40:0/1332672089 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3532474533 0 0) 0x7fcf540035d0 con 0x7fcf680cc250 2024-09-13T18:21:24.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf5ffff700 1 -- 172.21.15.40:0/1332672089 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf680cf580 msgr2=0x7fcf680c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:24.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf5ffff700 1 --2- 172.21.15.40:0/1332672089 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf680cf580 0x7fcf680c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf5ffff700 1 -- 172.21.15.40:0/1332672089 >> v2:172.21.15.43:3300/0 conn(0x7fcf680cf1a0 msgr2=0x7fcf6815a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:24.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf5ffff700 1 --2- 172.21.15.40:0/1332672089 >> v2:172.21.15.43:3300/0 conn(0x7fcf680cf1a0 0x7fcf6815a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf5ffff700 1 -- 172.21.15.40:0/1332672089 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf680caed0 con 0x7fcf680cc250 2024-09-13T18:21:24.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf5ffff700 1 -- 172.21.15.40:0/1332672089 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fcf60001b80 con 0x7fcf680cc250 2024-09-13T18:21:24.833 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf5ffff700 1 -- 172.21.15.40:0/1332672089 >> v1:172.21.15.40:6789/0 conn(0x7fcf680cc250 legacy=0x7fcf680c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf5ffff700 1 --2- 172.21.15.40:0/1332672089 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf680cf580 0x7fcf54003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf5ffff700 1 --2- 172.21.15.40:0/1332672089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf54003fa0 0x7fcf54004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf5ffff700 1 --2- 172.21.15.40:0/1332672089 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf54004db0 0x7fcf54005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf5ffff700 1 -- 172.21.15.40:0/1332672089 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf680cbdc0 con 0x7fcf54004db0 2024-09-13T18:21:24.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf5ffff700 1 -- 172.21.15.40:0/1332672089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf68153f80 con 0x7fcf54003fa0 2024-09-13T18:21:24.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf5ffff700 1 -- 172.21.15.40:0/1332672089 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5400ba60 con 0x7fcf680cf580 2024-09-13T18:21:24.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf6df7a700 1 --2- 172.21.15.40:0/1332672089 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf54004db0 0x7fcf54005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf6cf78700 1 --2- 172.21.15.40:0/1332672089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf54003fa0 0x7fcf54004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.821+0000 7fcf6d779700 1 --2- 172.21.15.40:0/1332672089 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf680cf580 0x7fcf54003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.834 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6df7a700 1 -- 172.21.15.40:0/1332672089 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf680cf580 msgr2=0x7fcf54003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:24.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6df7a700 1 --2- 172.21.15.40:0/1332672089 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf680cf580 0x7fcf54003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6df7a700 1 -- 172.21.15.40:0/1332672089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf54003fa0 msgr2=0x7fcf54004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:24.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6df7a700 1 --2- 172.21.15.40:0/1332672089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf54003fa0 0x7fcf54004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6df7a700 1 -- 172.21.15.40:0/1332672089 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf540035d0 con 0x7fcf54004db0 2024-09-13T18:21:24.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6df7a700 1 --2- 172.21.15.40:0/1332672089 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf54004db0 0x7fcf54005230 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000a830 tx=0x7fcf540058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3961075a3cd02dd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:24.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf5ffff700 1 -- 172.21.15.40:0/1332672089 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf600072d0 con 0x7fcf54004db0 2024-09-13T18:21:24.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf5ffff700 1 -- 172.21.15.40:0/1332672089 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf60007450 con 0x7fcf54004db0 2024-09-13T18:21:24.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf5ffff700 1 -- 172.21.15.40:0/1332672089 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf60007800 con 0x7fcf54004db0 2024-09-13T18:21:24.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6e77b700 1 -- 172.21.15.40:0/1332672089 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf54004db0 msgr2=0x7fcf54005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6e77b700 1 --2- 172.21.15.40:0/1332672089 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf54004db0 0x7fcf54005230 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000a830 tx=0x7fcf540058a0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6df7a700 1 -- 172.21.15.40:0/1332672089 reap_dead start 2024-09-13T18:21:24.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6e77b700 1 -- 172.21.15.40:0/1332672089 shutdown_connections 2024-09-13T18:21:24.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6e77b700 1 -- 172.21.15.40:0/1332672089 >> 172.21.15.40:0/1332672089 conn(0x7fcf68066090 msgr2=0x7fcf68066470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:24.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6e77b700 1 -- 172.21.15.40:0/1332672089 shutdown_connections 2024-09-13T18:21:24.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6e77b700 1 -- 172.21.15.40:0/1332672089 wait complete. 2024-09-13T18:21:24.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6e77b700 1 Processor -- start 2024-09-13T18:21:24.836 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6e77b700 1 -- start start 2024-09-13T18:21:24.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6e77b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf54004db0 0x7fcf68159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6e77b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf681591f0 0x7fcf68159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6e77b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6814e270 0x7fcf6814e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6e77b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf681530e0 con 0x7fcf6814e270 2024-09-13T18:21:24.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6e77b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf68156cb0 con 0x7fcf681591f0 2024-09-13T18:21:24.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6e77b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf681ef5e0 con 0x7fcf54004db0 2024-09-13T18:21:24.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6df7a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6814e270 0x7fcf6814e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6df7a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6814e270 0x7fcf6814e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47136/0 (socket says 172.21.15.40:47136) 2024-09-13T18:21:24.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6df7a700 1 -- 172.21.15.40:0/3507098016 learned_addr learned my addr 172.21.15.40:0/3507098016 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:24.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6d779700 1 --2- 172.21.15.40:0/3507098016 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf54004db0 0x7fcf68159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.839 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6cf78700 1 --2- 172.21.15.40:0/3507098016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf681591f0 0x7fcf68159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6df7a700 1 -- 172.21.15.40:0/3507098016 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf54004db0 msgr2=0x7fcf68159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6df7a700 1 --2- 172.21.15.40:0/3507098016 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf54004db0 0x7fcf68159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6df7a700 1 -- 172.21.15.40:0/3507098016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf681591f0 msgr2=0x7fcf68159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:24.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6df7a700 1 --2- 172.21.15.40:0/3507098016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf681591f0 0x7fcf68159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:24.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6df7a700 1 -- 172.21.15.40:0/3507098016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf681efad0 con 0x7fcf6814e270 2024-09-13T18:21:24.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.825+0000 7fcf6df7a700 1 --2- 172.21.15.40:0/3507098016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6814e270 0x7fcf6814e6f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcf540047a0 tx=0x7fcf60001f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:24.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.829+0000 7fcf5dffb700 1 -- 172.21.15.40:0/3507098016 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf60024070 con 0x7fcf6814e270 2024-09-13T18:21:24.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.829+0000 7fcf5dffb700 1 -- 172.21.15.40:0/3507098016 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf60001490 con 0x7fcf6814e270 2024-09-13T18:21:24.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.829+0000 7fcf6e77b700 1 -- 172.21.15.40:0/3507098016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf68156ea0 con 0x7fcf6814e270 2024-09-13T18:21:24.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.829+0000 7fcf5dffb700 1 -- 172.21.15.40:0/3507098016 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6001f430 con 0x7fcf6814e270 2024-09-13T18:21:24.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.829+0000 7fcf6e77b700 1 -- 172.21.15.40:0/3507098016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf68152d00 con 0x7fcf6814e270 2024-09-13T18:21:24.840 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.829+0000 7fcf6e77b700 1 -- 172.21.15.40:0/3507098016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf68067a40 con 0x7fcf6814e270 2024-09-13T18:21:24.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.829+0000 7fcf5dffb700 1 -- 172.21.15.40:0/3507098016 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcf6001f590 con 0x7fcf6814e270 2024-09-13T18:21:24.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.829+0000 7fcf5dffb700 1 --2- 172.21.15.40:0/3507098016 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcf4c066760 0x7fcf4c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:24.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.829+0000 7fcf5dffb700 1 -- 172.21.15.40:0/3507098016 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcf60086360 con 0x7fcf6814e270 2024-09-13T18:21:24.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.829+0000 7fcf6d779700 1 --2- 172.21.15.40:0/3507098016 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcf4c066760 0x7fcf4c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:24.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.833+0000 7fcf6d779700 1 --2- 172.21.15.40:0/3507098016 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcf4c066760 0x7fcf4c068c20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009bf0 tx=0x7fcf64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:24.841 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:24.833+0000 7fcf5dffb700 1 -- 172.21.15.40:0/3507098016 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcf600533a0 con 0x7fcf6814e270 2024-09-13T18:21:25.010 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.005+0000 7fcf6e77b700 1 -- 172.21.15.40:0/3507098016 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf6814ec30 con 0x7fcf4c066760 2024-09-13T18:21:25.011 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.005+0000 7fcf5dffb700 1 -- 172.21.15.40:0/3507098016 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcf6814ec30 con 0x7fcf4c066760 2024-09-13T18:21:25.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.005+0000 7fcf6e77b700 1 -- 172.21.15.40:0/3507098016 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcf4c066760 msgr2=0x7fcf4c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.005+0000 7fcf6e77b700 1 --2- 172.21.15.40:0/3507098016 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcf4c066760 0x7fcf4c068c20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009bf0 tx=0x7fcf64008040 comp rx=0 tx=0).stop 2024-09-13T18:21:25.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.005+0000 7fcf6e77b700 1 -- 172.21.15.40:0/3507098016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6814e270 msgr2=0x7fcf6814e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.005+0000 7fcf6e77b700 1 --2- 172.21.15.40:0/3507098016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6814e270 0x7fcf6814e6f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcf540047a0 tx=0x7fcf60001f10 comp rx=0 tx=0).stop 2024-09-13T18:21:25.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.005+0000 7fcf6e77b700 1 -- 172.21.15.40:0/3507098016 shutdown_connections 2024-09-13T18:21:25.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.005+0000 7fcf6e77b700 1 --2- 172.21.15.40:0/3507098016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcf6814e270 0x7fcf6814e6f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.014 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.005+0000 7fcf6e77b700 1 --2- 172.21.15.40:0/3507098016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf681591f0 0x7fcf68159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.005+0000 7fcf6e77b700 1 --2- 172.21.15.40:0/3507098016 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fcf4c066760 0x7fcf4c068c20 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.009+0000 7fcf6e77b700 1 --2- 172.21.15.40:0/3507098016 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcf54004db0 0x7fcf68159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.009+0000 7fcf6e77b700 1 -- 172.21.15.40:0/3507098016 >> 172.21.15.40:0/3507098016 conn(0x7fcf68066090 msgr2=0x7fcf680c8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:25.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.009+0000 7fcf6e77b700 1 -- 172.21.15.40:0/3507098016 shutdown_connections 2024-09-13T18:21:25.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.009+0000 7fcf6e77b700 1 -- 172.21.15.40:0/3507098016 wait complete. 2024-09-13T18:21:25.015 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 5 to deep-scrub 2024-09-13T18:21:25.025 INFO:tasks.ceph:Scrubbing osd.6 2024-09-13T18:21:25.025 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:21:25.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.137+0000 7f8495f7a700 1 Processor -- start 2024-09-13T18:21:25.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.137+0000 7f8495f7a700 1 -- start start 2024-09-13T18:21:25.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.137+0000 7f8495f7a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8490153900 0x7f84900c9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.137+0000 7f8495f7a700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84900cad20 con 0x7f84900cbb00 2024-09-13T18:21:25.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.137+0000 7f8495f7a700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84900b2250 con 0x7f84900c2980 2024-09-13T18:21:25.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.137+0000 7f8495f7a700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8490070a30 con 0x7f8490153900 2024-09-13T18:21:25.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.137+0000 7f8495779700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f84900cbb00 0x7f8490163e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51512/0 (socket says 172.21.15.40:51512) 2024-09-13T18:21:25.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.137+0000 7f8495779700 1 -- 172.21.15.40:0/2451453517 learned_addr learned my addr 172.21.15.40:0/2451453517 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:25.146 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.141+0000 7f8494f78700 1 --2- 172.21.15.40:0/2451453517 >> v2:172.21.15.43:3300/0 conn(0x7f8490153900 0x7f84900c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.141+0000 7f848f7fe700 1 -- 172.21.15.40:0/2451453517 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383528849 0 0) 0x7f84900cad20 con 0x7f84900cbb00 2024-09-13T18:21:25.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.141+0000 7f848f7fe700 1 -- 172.21.15.40:0/2451453517 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84800035d0 con 0x7f84900cbb00 2024-09-13T18:21:25.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.141+0000 7f8494f78700 1 -- 172.21.15.40:0/2451453517 >> v1:172.21.15.43:6789/0 conn(0x7f84900c2980 legacy=0x7f849015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.141+0000 7f8494f78700 1 -- 172.21.15.40:0/2451453517 >> v1:172.21.15.40:6789/0 conn(0x7f84900cbb00 legacy=0x7f8490163e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.141+0000 7f8494f78700 1 -- 172.21.15.40:0/2451453517 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8490153ce0 con 0x7f8490153900 2024-09-13T18:21:25.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.141+0000 7f8494f78700 1 --2- 172.21.15.40:0/2451453517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8490153900 0x7f84900c9950 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8484009850 tx=0x7f8484009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40a991e2f6ac5be5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:25.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.141+0000 7f848f7fe700 1 -- 172.21.15.40:0/2451453517 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8484011830 con 0x7f8490153900 2024-09-13T18:21:25.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.141+0000 7f848f7fe700 1 -- 172.21.15.40:0/2451453517 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8484011990 con 0x7f8490153900 2024-09-13T18:21:25.147 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.141+0000 7f848f7fe700 1 -- 172.21.15.40:0/2451453517 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8484011d00 con 0x7f8490153900 2024-09-13T18:21:25.148 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.141+0000 7f8495f7a700 1 -- 172.21.15.40:0/2451453517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8490153900 msgr2=0x7f84900c9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.141+0000 7f8495f7a700 1 --2- 172.21.15.40:0/2451453517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8490153900 0x7f84900c9950 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8484009850 tx=0x7f8484009b60 comp rx=0 tx=0).stop 2024-09-13T18:21:25.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.141+0000 7f8495f7a700 1 -- 172.21.15.40:0/2451453517 shutdown_connections 2024-09-13T18:21:25.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.141+0000 7f8495f7a700 1 --2- 172.21.15.40:0/2451453517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8490153900 0x7f84900c9950 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.141+0000 7f8495f7a700 1 -- 172.21.15.40:0/2451453517 >> 172.21.15.40:0/2451453517 conn(0x7f849006e620 msgr2=0x7f849006ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:25.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.141+0000 7f8495f7a700 1 -- 172.21.15.40:0/2451453517 shutdown_connections 2024-09-13T18:21:25.149 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.141+0000 7f8495f7a700 1 -- 172.21.15.40:0/2451453517 wait complete. 2024-09-13T18:21:25.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f8495f7a700 1 Processor -- start 2024-09-13T18:21:25.150 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f8495f7a700 1 -- start start 2024-09-13T18:21:25.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f8495f7a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f84900c2980 0x7f8490155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f8495f7a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84900cbb00 0x7f8490156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f8495f7a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8490153900 0x7f84900c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f8495f7a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84900c1f50 con 0x7f8490153900 2024-09-13T18:21:25.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f8495f7a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84900c3a40 con 0x7f84900cbb00 2024-09-13T18:21:25.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f8495f7a700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84900c3f70 con 0x7f84900c2980 2024-09-13T18:21:25.153 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f8495779700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8490153900 0x7f84900c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f8495779700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8490153900 0x7f84900c5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47138/0 (socket says 172.21.15.40:47138) 2024-09-13T18:21:25.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f848ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84900cbb00 0x7f8490156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f8495779700 1 -- 172.21.15.40:0/1221411048 learned_addr learned my addr 172.21.15.40:0/1221411048 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:25.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f8494f78700 1 --2- 172.21.15.40:0/1221411048 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f84900c2980 0x7f8490155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f8495779700 1 -- 172.21.15.40:0/1221411048 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f84900c2980 msgr2=0x7f8490155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f8495779700 1 --2- 172.21.15.40:0/1221411048 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f84900c2980 0x7f8490155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f8495779700 1 -- 172.21.15.40:0/1221411048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84900cbb00 msgr2=0x7f8490156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f8495779700 1 --2- 172.21.15.40:0/1221411048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84900cbb00 0x7f8490156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f8495779700 1 -- 172.21.15.40:0/1221411048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84840094b0 con 0x7f8490153900 2024-09-13T18:21:25.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f8495779700 1 --2- 172.21.15.40:0/1221411048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8490153900 0x7f84900c5490 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f848800cb70 tx=0x7f848800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:25.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f848d7fa700 1 -- 172.21.15.40:0/1221411048 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488007480 con 0x7f8490153900 2024-09-13T18:21:25.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f848d7fa700 1 -- 172.21.15.40:0/1221411048 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84880075e0 con 0x7f8490153900 2024-09-13T18:21:25.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f8495f7a700 1 -- 172.21.15.40:0/1221411048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84900c3c90 con 0x7f8490153900 2024-09-13T18:21:25.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f848d7fa700 1 -- 172.21.15.40:0/1221411048 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488007950 con 0x7f8490153900 2024-09-13T18:21:25.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.145+0000 7f8495f7a700 1 -- 172.21.15.40:0/1221411048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84900c1be0 con 0x7f8490153900 2024-09-13T18:21:25.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.149+0000 7f8495f7a700 1 -- 172.21.15.40:0/1221411048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f84900cb250 con 0x7f8490153900 2024-09-13T18:21:25.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.149+0000 7f848d7fa700 1 -- 172.21.15.40:0/1221411048 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f848801b3b0 con 0x7f8490153900 2024-09-13T18:21:25.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.149+0000 7f848d7fa700 1 --2- 172.21.15.40:0/1221411048 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8474066760 0x7f8474068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.149+0000 7f848d7fa700 1 -- 172.21.15.40:0/1221411048 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8488083dc0 con 0x7f8490153900 2024-09-13T18:21:25.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.149+0000 7f8494f78700 1 --2- 172.21.15.40:0/1221411048 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8474066760 0x7f8474068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.149+0000 7f848d7fa700 1 --2- 172.21.15.40:0/1221411048 >> [v2:172.21.15.43:6808/480073165,v1:172.21.15.43:6809/480073165] conn(0x7f8474070040 0x7f84740724e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.149+0000 7f848d7fa700 1 -- 172.21.15.40:0/1221411048 --> [v2:172.21.15.43:6808/480073165,v1:172.21.15.43:6809/480073165] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8474072bb0 con 0x7f8474070040 2024-09-13T18:21:25.156 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.149+0000 7f848d7fa700 1 -- 172.21.15.40:0/1221411048 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84880863a0 con 0x7f8490153900 2024-09-13T18:21:25.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.149+0000 7f848ffff700 1 --2- 172.21.15.40:0/1221411048 >> [v2:172.21.15.43:6808/480073165,v1:172.21.15.43:6809/480073165] conn(0x7f8474070040 0x7f84740724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.149+0000 7f8494f78700 1 --2- 172.21.15.40:0/1221411048 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8474066760 0x7f8474068c20 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f84840090b0 tx=0x7f848401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:25.157 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.149+0000 7f848ffff700 1 --2- 172.21.15.40:0/1221411048 >> [v2:172.21.15.43:6808/480073165,v1:172.21.15.43:6809/480073165] conn(0x7f8474070040 0x7f84740724e0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:25.159 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.153+0000 7f848d7fa700 1 -- 172.21.15.40:0/1221411048 <== osd.6 v2:172.21.15.43:6808/480073165 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f8474072bb0 con 0x7f8474070040 2024-09-13T18:21:25.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.177+0000 7f8495f7a700 1 -- 172.21.15.40:0/1221411048 --> [v2:172.21.15.43:6808/480073165,v1:172.21.15.43:6809/480073165] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f84900caf70 con 0x7f8474070040 2024-09-13T18:21:25.184 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.177+0000 7f848d7fa700 1 -- 172.21.15.40:0/1221411048 <== osd.6 v2:172.21.15.43:6808/480073165 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f847c001600 con 0x7f8474070040 2024-09-13T18:21:25.185 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:21:25.185 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:21:25.185 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:21:25.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.177+0000 7f8495f7a700 1 -- 172.21.15.40:0/1221411048 >> [v2:172.21.15.43:6808/480073165,v1:172.21.15.43:6809/480073165] conn(0x7f8474070040 msgr2=0x7f84740724e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.177+0000 7f8495f7a700 1 --2- 172.21.15.40:0/1221411048 >> [v2:172.21.15.43:6808/480073165,v1:172.21.15.43:6809/480073165] conn(0x7f8474070040 0x7f84740724e0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.181+0000 7f8495f7a700 1 -- 172.21.15.40:0/1221411048 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8474066760 msgr2=0x7f8474068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.181+0000 7f8495f7a700 1 --2- 172.21.15.40:0/1221411048 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f8474066760 0x7f8474068c20 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f84840090b0 tx=0x7f848401c040 comp rx=0 tx=0).stop 2024-09-13T18:21:25.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.181+0000 7f8495f7a700 1 -- 172.21.15.40:0/1221411048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8490153900 msgr2=0x7f84900c5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.181+0000 7f8495f7a700 1 --2- 172.21.15.40:0/1221411048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8490153900 0x7f84900c5490 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f848800cb70 tx=0x7f848800cba0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.181+0000 7f8495779700 1 -- 172.21.15.40:0/1221411048 reap_dead start 2024-09-13T18:21:25.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.181+0000 7f8495f7a700 1 -- 172.21.15.40:0/1221411048 shutdown_connections 2024-09-13T18:21:25.186 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.181+0000 7f8495f7a700 1 -- 172.21.15.40:0/1221411048 >> 172.21.15.40:0/1221411048 conn(0x7f849006e620 msgr2=0x7f8490159220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:25.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.181+0000 7f8495f7a700 1 -- 172.21.15.40:0/1221411048 shutdown_connections 2024-09-13T18:21:25.187 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.181+0000 7f8495f7a700 1 -- 172.21.15.40:0/1221411048 wait complete. 2024-09-13T18:21:25.205 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-13T18:21:25.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27b5153700 1 Processor -- start 2024-09-13T18:21:25.318 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27b5153700 1 -- start start 2024-09-13T18:21:25.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27b5153700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27b00cba70 0x7f27b00c3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27b5153700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b01561c0 con 0x7f27b014fab0 2024-09-13T18:21:25.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27b5153700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b01564a0 con 0x7f27b014fe90 2024-09-13T18:21:25.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27b5153700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b0156780 con 0x7f27b00cba70 2024-09-13T18:21:25.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27affff700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f27b014fab0 0x7f27b0155910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:51518/0 (socket says 172.21.15.40:51518) 2024-09-13T18:21:25.319 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27affff700 1 -- 172.21.15.40:0/3307599000 learned_addr learned my addr 172.21.15.40:0/3307599000 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:25.320 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27b4952700 1 --2- 172.21.15.40:0/3307599000 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27b00cba70 0x7f27b00c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27aeffd700 1 -- 172.21.15.40:0/3307599000 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443810975 0 0) 0x7f27b01561c0 con 0x7f27b014fab0 2024-09-13T18:21:25.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27aeffd700 1 -- 172.21.15.40:0/3307599000 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27980035d0 con 0x7f27b014fab0 2024-09-13T18:21:25.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27aeffd700 1 -- 172.21.15.40:0/3307599000 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3358026943 0 0) 0x7f27b01564a0 con 0x7f27b014fe90 2024-09-13T18:21:25.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27aeffd700 1 -- 172.21.15.40:0/3307599000 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27b01561c0 con 0x7f27b014fe90 2024-09-13T18:21:25.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27aeffd700 1 -- 172.21.15.40:0/3307599000 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4103103566 0 0) 0x7f27980035d0 con 0x7f27b014fab0 2024-09-13T18:21:25.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27b4952700 1 -- 172.21.15.40:0/3307599000 >> v1:172.21.15.43:6789/0 conn(0x7f27b014fe90 legacy=0x7f27b00cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27b4952700 1 -- 172.21.15.40:0/3307599000 >> v1:172.21.15.40:6789/0 conn(0x7f27b014fab0 legacy=0x7f27b0155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27b4952700 1 -- 172.21.15.40:0/3307599000 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b0158ec0 con 0x7f27b00cba70 2024-09-13T18:21:25.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27b4952700 1 --2- 172.21.15.40:0/3307599000 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27b00cba70 0x7f27b00c3650 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f27a000aa20 tx=0x7f27a000ad30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92a295f115bc0c9e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:25.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27aeffd700 1 -- 172.21.15.40:0/3307599000 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a80019f0 con 0x7f27b00cba70 2024-09-13T18:21:25.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27aeffd700 1 -- 172.21.15.40:0/3307599000 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a000aec0 con 0x7f27b00cba70 2024-09-13T18:21:25.322 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.313+0000 7f27aeffd700 1 -- 172.21.15.40:0/3307599000 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a0004230 con 0x7f27b00cba70 2024-09-13T18:21:25.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b5153700 1 -- 172.21.15.40:0/3307599000 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27b00cba70 msgr2=0x7f27b00c3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b5153700 1 --2- 172.21.15.40:0/3307599000 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27b00cba70 0x7f27b00c3650 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f27a000aa20 tx=0x7f27a000ad30 comp rx=0 tx=0).stop 2024-09-13T18:21:25.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b5153700 1 -- 172.21.15.40:0/3307599000 shutdown_connections 2024-09-13T18:21:25.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b5153700 1 --2- 172.21.15.40:0/3307599000 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27b00cba70 0x7f27b00c3650 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b5153700 1 -- 172.21.15.40:0/3307599000 >> 172.21.15.40:0/3307599000 conn(0x7f27b006a390 msgr2=0x7f27b006a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:25.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b5153700 1 -- 172.21.15.40:0/3307599000 shutdown_connections 2024-09-13T18:21:25.323 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b5153700 1 -- 172.21.15.40:0/3307599000 wait complete. 2024-09-13T18:21:25.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b5153700 1 Processor -- start 2024-09-13T18:21:25.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b5153700 1 -- start start 2024-09-13T18:21:25.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b5153700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27b014fab0 0x7f27b01581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b5153700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27b0154d00 0x7f27b0156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b5153700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27b01e3bd0 0x7f27b00b8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b5153700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b00b9210 con 0x7f27b01e3bd0 2024-09-13T18:21:25.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b5153700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b01571e0 con 0x7f27b0154d00 2024-09-13T18:21:25.325 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b5153700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b0158b80 con 0x7f27b014fab0 2024-09-13T18:21:25.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b4952700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27b01e3bd0 0x7f27b00b8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b4952700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27b01e3bd0 0x7f27b00b8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47152/0 (socket says 172.21.15.40:47152) 2024-09-13T18:21:25.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b4952700 1 -- 172.21.15.40:0/1797044156 learned_addr learned my addr 172.21.15.40:0/1797044156 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:25.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27affff700 1 --2- 172.21.15.40:0/1797044156 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27b014fab0 0x7f27b01581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27af7fe700 1 --2- 172.21.15.40:0/1797044156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27b0154d00 0x7f27b0156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b4952700 1 -- 172.21.15.40:0/1797044156 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27b014fab0 msgr2=0x7f27b01581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b4952700 1 --2- 172.21.15.40:0/1797044156 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27b014fab0 0x7f27b01581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b4952700 1 -- 172.21.15.40:0/1797044156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27b0154d00 msgr2=0x7f27b0156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b4952700 1 --2- 172.21.15.40:0/1797044156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27b0154d00 0x7f27b0156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b4952700 1 -- 172.21.15.40:0/1797044156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a000a650 con 0x7f27b01e3bd0 2024-09-13T18:21:25.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b4952700 1 --2- 172.21.15.40:0/1797044156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27b01e3bd0 0x7f27b00b8160 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f27a0004000 tx=0x7f27a00047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:25.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27acff9700 1 -- 172.21.15.40:0/1797044156 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a001e070 con 0x7f27b01e3bd0 2024-09-13T18:21:25.326 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27acff9700 1 -- 172.21.15.40:0/1797044156 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a00045f0 con 0x7f27b01e3bd0 2024-09-13T18:21:25.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b5153700 1 -- 172.21.15.40:0/1797044156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b00c1ea0 con 0x7f27b01e3bd0 2024-09-13T18:21:25.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27acff9700 1 -- 172.21.15.40:0/1797044156 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a00038e0 con 0x7f27b01e3bd0 2024-09-13T18:21:25.327 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.317+0000 7f27b5153700 1 -- 172.21.15.40:0/1797044156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b0154500 con 0x7f27b01e3bd0 2024-09-13T18:21:25.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.321+0000 7f27b5153700 1 -- 172.21.15.40:0/1797044156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b00c2e70 con 0x7f27b01e3bd0 2024-09-13T18:21:25.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.321+0000 7f27acff9700 1 -- 172.21.15.40:0/1797044156 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f27a0003a40 con 0x7f27b01e3bd0 2024-09-13T18:21:25.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.321+0000 7f27acff9700 1 --2- 172.21.15.40:0/1797044156 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f279c066760 0x7f279c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.321+0000 7f27acff9700 1 -- 172.21.15.40:0/1797044156 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f27a0085670 con 0x7f27b01e3bd0 2024-09-13T18:21:25.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.321+0000 7f27affff700 1 --2- 172.21.15.40:0/1797044156 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f279c066760 0x7f279c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.328 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.321+0000 7f27affff700 1 --2- 172.21.15.40:0/1797044156 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f279c066760 0x7f279c068c20 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f27a8002230 tx=0x7f27a8002100 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:25.331 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.325+0000 7f27acff9700 1 -- 172.21.15.40:0/1797044156 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f27a00526b0 con 0x7f27b01e3bd0 2024-09-13T18:21:25.497 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.489+0000 7f27b5153700 1 -- 172.21.15.40:0/1797044156 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f27b0159480 con 0x7f279c066760 2024-09-13T18:21:25.498 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.493+0000 7f27acff9700 1 -- 172.21.15.40:0/1797044156 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f27a8001970 con 0x7f279c066760 2024-09-13T18:21:25.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.493+0000 7f27b5153700 1 -- 172.21.15.40:0/1797044156 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f279c066760 msgr2=0x7f279c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.493+0000 7f27b5153700 1 --2- 172.21.15.40:0/1797044156 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f279c066760 0x7f279c068c20 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f27a8002230 tx=0x7f27a8002100 comp rx=0 tx=0).stop 2024-09-13T18:21:25.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.493+0000 7f27b5153700 1 -- 172.21.15.40:0/1797044156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27b01e3bd0 msgr2=0x7f27b00b8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.493+0000 7f27b5153700 1 --2- 172.21.15.40:0/1797044156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27b01e3bd0 0x7f27b00b8160 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f27a0004000 tx=0x7f27a00047e0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.493+0000 7f27b5153700 1 -- 172.21.15.40:0/1797044156 shutdown_connections 2024-09-13T18:21:25.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.493+0000 7f27b5153700 1 --2- 172.21.15.40:0/1797044156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27b01e3bd0 0x7f27b00b8160 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.493+0000 7f27b5153700 1 --2- 172.21.15.40:0/1797044156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27b0154d00 0x7f27b0156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.493+0000 7f27b5153700 1 --2- 172.21.15.40:0/1797044156 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f279c066760 0x7f279c068c20 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.493+0000 7f27b5153700 1 --2- 172.21.15.40:0/1797044156 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27b014fab0 0x7f27b01581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.493+0000 7f27b5153700 1 -- 172.21.15.40:0/1797044156 >> 172.21.15.40:0/1797044156 conn(0x7f27b006a390 msgr2=0x7f27b00cb360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:25.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.493+0000 7f27b5153700 1 -- 172.21.15.40:0/1797044156 shutdown_connections 2024-09-13T18:21:25.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.493+0000 7f27b5153700 1 -- 172.21.15.40:0/1797044156 wait complete. 2024-09-13T18:21:25.503 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 6 to deep-scrub 2024-09-13T18:21:25.512 INFO:tasks.ceph:Scrubbing osd.7 2024-09-13T18:21:25.512 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:21:25.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.617+0000 7f53cbfff700 1 Processor -- start 2024-09-13T18:21:25.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.617+0000 7f53cbfff700 1 -- start start 2024-09-13T18:21:25.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53cbfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f53cc153900 0x7f53cc0c9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53cbfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f53cc0c2980 0x7f53cc15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53cbfff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53cc0cad20 con 0x7f53cc15d720 2024-09-13T18:21:25.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53cbfff700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53cc070a30 con 0x7f53cc0c2980 2024-09-13T18:21:25.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53cbfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc071e50 con 0x7f53cc153900 2024-09-13T18:21:25.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53caffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f53cc153900 0x7f53cc0c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53ca7fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f53cc0c2980 0x7f53cc15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53caffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f53cc153900 0x7f53cc0c9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:57902/0 (socket says 172.21.15.40:57902) 2024-09-13T18:21:25.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53caffd700 1 -- 172.21.15.40:0/3892916836 learned_addr learned my addr 172.21.15.40:0/3892916836 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:25.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53caffd700 1 -- 172.21.15.40:0/3892916836 >> v2:172.21.15.43:3300/0 conn(0x7f53cc0c2980 msgr2=0x7f53cc15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:25.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53caffd700 1 --2- 172.21.15.40:0/3892916836 >> v2:172.21.15.43:3300/0 conn(0x7f53cc0c2980 0x7f53cc15d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53caffd700 1 -- 172.21.15.40:0/3892916836 >> v1:172.21.15.43:6789/0 conn(0x7f53cc15d720 legacy=0x7f53cc0cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53caffd700 1 -- 172.21.15.40:0/3892916836 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc0b2250 con 0x7f53cc153900 2024-09-13T18:21:25.629 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53caffd700 1 --2- 172.21.15.40:0/3892916836 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f53cc153900 0x7f53cc0c9950 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009910 tx=0x7f53bc009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=431d82ae80685d62 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:25.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53c9ffb700 1 -- 172.21.15.40:0/3892916836 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc011830 con 0x7f53cc153900 2024-09-13T18:21:25.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53c9ffb700 1 -- 172.21.15.40:0/3892916836 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc011990 con 0x7f53cc153900 2024-09-13T18:21:25.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53c9ffb700 1 -- 172.21.15.40:0/3892916836 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc011d40 con 0x7f53cc153900 2024-09-13T18:21:25.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53cbfff700 1 -- 172.21.15.40:0/3892916836 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f53cc153900 msgr2=0x7f53cc0c9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53cbfff700 1 --2- 172.21.15.40:0/3892916836 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f53cc153900 0x7f53cc0c9950 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009910 tx=0x7f53bc009c20 comp rx=0 tx=0).stop 2024-09-13T18:21:25.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53cbfff700 1 -- 172.21.15.40:0/3892916836 shutdown_connections 2024-09-13T18:21:25.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53cbfff700 1 --2- 172.21.15.40:0/3892916836 >> v2:172.21.15.43:3300/0 conn(0x7f53cc0c2980 0x7f53cc15d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53cbfff700 1 --2- 172.21.15.40:0/3892916836 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f53cc153900 0x7f53cc0c9950 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53cbfff700 1 -- 172.21.15.40:0/3892916836 >> 172.21.15.40:0/3892916836 conn(0x7f53cc06e620 msgr2=0x7f53cc06ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:25.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53cbfff700 1 -- 172.21.15.40:0/3892916836 shutdown_connections 2024-09-13T18:21:25.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.621+0000 7f53cbfff700 1 -- 172.21.15.40:0/3892916836 wait complete. 2024-09-13T18:21:25.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53cbfff700 1 Processor -- start 2024-09-13T18:21:25.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53cbfff700 1 -- start start 2024-09-13T18:21:25.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53cbfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53cc0c2980 0x7f53cc0c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53cbfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f53cc153900 0x7f53cc1f2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53cbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53cc15d720 0x7f53cc0c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53cbfff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc159610 con 0x7f53cc15d720 2024-09-13T18:21:25.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53cbfff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc153f20 con 0x7f53cc0c2980 2024-09-13T18:21:25.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53cbfff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc1f3000 con 0x7f53cc153900 2024-09-13T18:21:25.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53cb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53cc15d720 0x7f53cc0c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53cb7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53cc15d720 0x7f53cc0c4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47154/0 (socket says 172.21.15.40:47154) 2024-09-13T18:21:25.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53cb7fe700 1 -- 172.21.15.40:0/213474377 learned_addr learned my addr 172.21.15.40:0/213474377 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:25.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53ca7fc700 1 --2- 172.21.15.40:0/213474377 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f53cc153900 0x7f53cc1f2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53caffd700 1 --2- 172.21.15.40:0/213474377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53cc0c2980 0x7f53cc0c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53cb7fe700 1 -- 172.21.15.40:0/213474377 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f53cc153900 msgr2=0x7f53cc1f2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53cb7fe700 1 --2- 172.21.15.40:0/213474377 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f53cc153900 0x7f53cc1f2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53cb7fe700 1 -- 172.21.15.40:0/213474377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53cc0c2980 msgr2=0x7f53cc0c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53cb7fe700 1 --2- 172.21.15.40:0/213474377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53cc0c2980 0x7f53cc0c39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53cb7fe700 1 -- 172.21.15.40:0/213474377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53bc009570 con 0x7f53cc15d720 2024-09-13T18:21:25.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53cb7fe700 1 --2- 172.21.15.40:0/213474377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53cc15d720 0x7f53cc0c4900 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f53c400cb50 tx=0x7f53c400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:25.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53b3fff700 1 -- 172.21.15.40:0/213474377 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4003c20 con 0x7f53cc15d720 2024-09-13T18:21:25.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53cbfff700 1 -- 172.21.15.40:0/213474377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53cc154170 con 0x7f53cc15d720 2024-09-13T18:21:25.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53cbfff700 1 -- 172.21.15.40:0/213474377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53cc1525a0 con 0x7f53cc15d720 2024-09-13T18:21:25.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53b3fff700 1 -- 172.21.15.40:0/213474377 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c4003d80 con 0x7f53cc15d720 2024-09-13T18:21:25.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53b3fff700 1 -- 172.21.15.40:0/213474377 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c400a9e0 con 0x7f53cc15d720 2024-09-13T18:21:25.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.625+0000 7f53cbfff700 1 -- 172.21.15.40:0/213474377 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f53cc0caf70 con 0x7f53cc15d720 2024-09-13T18:21:25.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.629+0000 7f53b3fff700 1 -- 172.21.15.40:0/213474377 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f53c400ab40 con 0x7f53cc15d720 2024-09-13T18:21:25.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.629+0000 7f53b3fff700 1 --2- 172.21.15.40:0/213474377 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f53b8066730 0x7f53b8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.629+0000 7f53caffd700 1 --2- 172.21.15.40:0/213474377 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f53b8066730 0x7f53b8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.629+0000 7f53b3fff700 1 -- 172.21.15.40:0/213474377 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f53c4084f90 con 0x7f53cc15d720 2024-09-13T18:21:25.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.629+0000 7f53b3fff700 1 --2- 172.21.15.40:0/213474377 >> [v2:172.21.15.43:6816/2080568799,v1:172.21.15.43:6817/2080568799] conn(0x7f53b806ff90 0x7f53b8072430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.629+0000 7f53b3fff700 1 -- 172.21.15.40:0/213474377 --> [v2:172.21.15.43:6816/2080568799,v1:172.21.15.43:6817/2080568799] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f53b8072b00 con 0x7f53b806ff90 2024-09-13T18:21:25.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.629+0000 7f53b3fff700 1 -- 172.21.15.40:0/213474377 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53c40853e0 con 0x7f53cc15d720 2024-09-13T18:21:25.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.629+0000 7f53ca7fc700 1 --2- 172.21.15.40:0/213474377 >> [v2:172.21.15.43:6816/2080568799,v1:172.21.15.43:6817/2080568799] conn(0x7f53b806ff90 0x7f53b8072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.629+0000 7f53caffd700 1 --2- 172.21.15.40:0/213474377 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f53b8066730 0x7f53b8068bf0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009c60 tx=0x7f53bc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:25.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.629+0000 7f53ca7fc700 1 --2- 172.21.15.40:0/213474377 >> [v2:172.21.15.43:6816/2080568799,v1:172.21.15.43:6817/2080568799] conn(0x7f53b806ff90 0x7f53b8072430 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:25.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.633+0000 7f53b3fff700 1 -- 172.21.15.40:0/213474377 <== osd.7 v2:172.21.15.43:6816/2080568799 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f53b8072b00 con 0x7f53b806ff90 2024-09-13T18:21:25.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.649+0000 7f53cbfff700 1 -- 172.21.15.40:0/213474377 --> [v2:172.21.15.43:6816/2080568799,v1:172.21.15.43:6817/2080568799] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f53cc15a390 con 0x7f53b806ff90 2024-09-13T18:21:25.657 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.653+0000 7f53b3fff700 1 -- 172.21.15.40:0/213474377 <== osd.7 v2:172.21.15.43:6816/2080568799 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f53c0009d60 con 0x7f53b806ff90 2024-09-13T18:21:25.658 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-09-13T18:21:25.658 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:21:25.658 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-09-13T18:21:25.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.653+0000 7f53cbfff700 1 -- 172.21.15.40:0/213474377 >> [v2:172.21.15.43:6816/2080568799,v1:172.21.15.43:6817/2080568799] conn(0x7f53b806ff90 msgr2=0x7f53b8072430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.653+0000 7f53cbfff700 1 --2- 172.21.15.40:0/213474377 >> [v2:172.21.15.43:6816/2080568799,v1:172.21.15.43:6817/2080568799] conn(0x7f53b806ff90 0x7f53b8072430 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.653+0000 7f53cbfff700 1 -- 172.21.15.40:0/213474377 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f53b8066730 msgr2=0x7f53b8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.653+0000 7f53cbfff700 1 --2- 172.21.15.40:0/213474377 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f53b8066730 0x7f53b8068bf0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009c60 tx=0x7f53bc008030 comp rx=0 tx=0).stop 2024-09-13T18:21:25.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.653+0000 7f53cbfff700 1 -- 172.21.15.40:0/213474377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53cc15d720 msgr2=0x7f53cc0c4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.653+0000 7f53cbfff700 1 --2- 172.21.15.40:0/213474377 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53cc15d720 0x7f53cc0c4900 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f53c400cb50 tx=0x7f53c400ce60 comp rx=0 tx=0).stop 2024-09-13T18:21:25.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.653+0000 7f53cb7fe700 1 -- 172.21.15.40:0/213474377 reap_dead start 2024-09-13T18:21:25.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.653+0000 7f53cbfff700 1 -- 172.21.15.40:0/213474377 shutdown_connections 2024-09-13T18:21:25.659 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.653+0000 7f53cbfff700 1 -- 172.21.15.40:0/213474377 >> 172.21.15.40:0/213474377 conn(0x7f53cc06e620 msgr2=0x7f53cc06a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:25.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.653+0000 7f53cbfff700 1 -- 172.21.15.40:0/213474377 shutdown_connections 2024-09-13T18:21:25.660 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.653+0000 7f53cbfff700 1 -- 172.21.15.40:0/213474377 wait complete. 2024-09-13T18:21:25.669 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-13T18:21:25.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b3fff700 1 Processor -- start 2024-09-13T18:21:25.775 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b3fff700 1 -- start start 2024-09-13T18:21:25.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96b4153730 0x7f96b4159f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b3fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f96b415cfe0 0x7f96b415d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b3fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b415a450 con 0x7f96b40c7fb0 2024-09-13T18:21:25.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b3fff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b415d9d0 con 0x7f96b415cfe0 2024-09-13T18:21:25.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b3fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b4070ae0 con 0x7f96b4153730 2024-09-13T18:21:25.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b37fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f96b415cfe0 0x7f96b415d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b27fc700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f96b40c7fb0 0x7f96b40c8630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.40:60074/0 (socket says 172.21.15.40:60074) 2024-09-13T18:21:25.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b27fc700 1 -- 172.21.15.40:0/3312635309 learned_addr learned my addr 172.21.15.40:0/3312635309 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:21:25.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b37fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f96b415cfe0 0x7f96b415d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47160/0 (socket says 172.21.15.40:47160) 2024-09-13T18:21:25.777 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b2ffd700 1 --2- 172.21.15.40:0/3312635309 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96b4153730 0x7f96b4159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b1ffb700 1 -- 172.21.15.40:0/3312635309 <== mon.1 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2815216340 0 0) 0x7f96a8001110 con 0x7f96b40c7fb0 2024-09-13T18:21:25.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b1ffb700 1 -- 172.21.15.40:0/3312635309 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f969c0035d0 con 0x7f96b40c7fb0 2024-09-13T18:21:25.778 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b37fe700 1 -- 172.21.15.40:0/3312635309 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96b4153730 msgr2=0x7f96b4159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:25.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b37fe700 1 --2- 172.21.15.40:0/3312635309 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96b4153730 0x7f96b4159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b37fe700 1 -- 172.21.15.40:0/3312635309 >> v1:172.21.15.43:6789/0 conn(0x7f96b40c7fb0 legacy=0x7f96b40c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b37fe700 1 -- 172.21.15.40:0/3312635309 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b4071010 con 0x7f96b415cfe0 2024-09-13T18:21:25.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b37fe700 1 --2- 172.21.15.40:0/3312635309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96b415cfe0 0x7f96b415d490 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f96ac009d90 tx=0x7f96ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b990edf0cf6170 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:25.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b1ffb700 1 -- 172.21.15.40:0/3312635309 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac00e070 con 0x7f96b415cfe0 2024-09-13T18:21:25.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b1ffb700 1 -- 172.21.15.40:0/3312635309 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ac00bb20 con 0x7f96b415cfe0 2024-09-13T18:21:25.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.769+0000 7f96b1ffb700 1 -- 172.21.15.40:0/3312635309 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac012430 con 0x7f96b415cfe0 2024-09-13T18:21:25.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.773+0000 7f96b3fff700 1 -- 172.21.15.40:0/3312635309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96b415cfe0 msgr2=0x7f96b415d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.773+0000 7f96b3fff700 1 --2- 172.21.15.40:0/3312635309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96b415cfe0 0x7f96b415d490 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f96ac009d90 tx=0x7f96ac00b330 comp rx=0 tx=0).stop 2024-09-13T18:21:25.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.773+0000 7f96b3fff700 1 -- 172.21.15.40:0/3312635309 shutdown_connections 2024-09-13T18:21:25.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.773+0000 7f96b3fff700 1 --2- 172.21.15.40:0/3312635309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96b415cfe0 0x7f96b415d490 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.773+0000 7f96b3fff700 1 --2- 172.21.15.40:0/3312635309 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96b4153730 0x7f96b4159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.773+0000 7f96b3fff700 1 -- 172.21.15.40:0/3312635309 >> 172.21.15.40:0/3312635309 conn(0x7f96b406e6d0 msgr2=0x7f96b406eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:25.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.773+0000 7f96b3fff700 1 -- 172.21.15.40:0/3312635309 shutdown_connections 2024-09-13T18:21:25.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.773+0000 7f96b3fff700 1 -- 172.21.15.40:0/3312635309 wait complete. 2024-09-13T18:21:25.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.773+0000 7f96b3fff700 1 Processor -- start 2024-09-13T18:21:25.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.773+0000 7f96b3fff700 1 -- start start 2024-09-13T18:21:25.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.773+0000 7f96b3fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96b40c7fb0 0x7f96b415b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.773+0000 7f96b3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96b4153730 0x7f96b40c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.773+0000 7f96b3fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96b415cfe0 0x7f96b414aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.773+0000 7f96b3fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b41f2ca0 con 0x7f96b4153730 2024-09-13T18:21:25.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.773+0000 7f96b3fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b415c7b0 con 0x7f96b40c7fb0 2024-09-13T18:21:25.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.773+0000 7f96b3fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b415c4d0 con 0x7f96b415cfe0 2024-09-13T18:21:25.782 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.773+0000 7f96b27fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96b4153730 0x7f96b40c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.773+0000 7f96b27fc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96b4153730 0x7f96b40c99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47166/0 (socket says 172.21.15.40:47166) 2024-09-13T18:21:25.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.773+0000 7f96b27fc700 1 -- 172.21.15.40:0/940504477 learned_addr learned my addr 172.21.15.40:0/940504477 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:25.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.777+0000 7f96b37fe700 1 --2- 172.21.15.40:0/940504477 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96b415cfe0 0x7f96b414aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.777+0000 7f96b2ffd700 1 --2- 172.21.15.40:0/940504477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96b40c7fb0 0x7f96b415b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.777+0000 7f96b27fc700 1 -- 172.21.15.40:0/940504477 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96b415cfe0 msgr2=0x7f96b414aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.777+0000 7f96b27fc700 1 --2- 172.21.15.40:0/940504477 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96b415cfe0 0x7f96b414aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.777+0000 7f96b27fc700 1 -- 172.21.15.40:0/940504477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96b40c7fb0 msgr2=0x7f96b415b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.777+0000 7f96b27fc700 1 --2- 172.21.15.40:0/940504477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96b40c7fb0 0x7f96b415b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.777+0000 7f96b27fc700 1 -- 172.21.15.40:0/940504477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ac009a70 con 0x7f96b4153730 2024-09-13T18:21:25.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.777+0000 7f96b2ffd700 1 --2- 172.21.15.40:0/940504477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96b40c7fb0 0x7f96b415b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:25.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.777+0000 7f96b27fc700 1 --2- 172.21.15.40:0/940504477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96b4153730 0x7f96b40c99a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f96a800c730 tx=0x7f96a800caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:25.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.777+0000 7f969b7fe700 1 -- 172.21.15.40:0/940504477 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8005d90 con 0x7f96b4153730 2024-09-13T18:21:25.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.777+0000 7f969b7fe700 1 -- 172.21.15.40:0/940504477 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a80070d0 con 0x7f96b4153730 2024-09-13T18:21:25.783 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.777+0000 7f96b3fff700 1 -- 172.21.15.40:0/940504477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b41f2300 con 0x7f96b4153730 2024-09-13T18:21:25.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.777+0000 7f969b7fe700 1 -- 172.21.15.40:0/940504477 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8007460 con 0x7f96b4153730 2024-09-13T18:21:25.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.777+0000 7f96b3fff700 1 -- 172.21.15.40:0/940504477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b41f28c0 con 0x7f96b4153730 2024-09-13T18:21:25.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.777+0000 7f96b3fff700 1 -- 172.21.15.40:0/940504477 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b41f2de0 con 0x7f96b4153730 2024-09-13T18:21:25.784 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.777+0000 7f969b7fe700 1 -- 172.21.15.40:0/940504477 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f96a80075c0 con 0x7f96b4153730 2024-09-13T18:21:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.777+0000 7f969b7fe700 1 --2- 172.21.15.40:0/940504477 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f96a0066730 0x7f96a0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.777+0000 7f969b7fe700 1 -- 172.21.15.40:0/940504477 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f96a8083890 con 0x7f96b4153730 2024-09-13T18:21:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.777+0000 7f96b2ffd700 1 --2- 172.21.15.40:0/940504477 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f96a0066730 0x7f96a0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:25.785 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.781+0000 7f96b2ffd700 1 --2- 172.21.15.40:0/940504477 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f96a0066730 0x7f96a0068bf0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009c40 tx=0x7f96a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:25.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.781+0000 7f969b7fe700 1 -- 172.21.15.40:0/940504477 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f96b41f2de0 con 0x7f96b4153730 2024-09-13T18:21:25.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.953+0000 7f96b3fff700 1 -- 172.21.15.40:0/940504477 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f96b40675e0 con 0x7f96a0066730 2024-09-13T18:21:25.958 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.953+0000 7f969b7fe700 1 -- 172.21.15.40:0/940504477 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f96b40675e0 con 0x7f96a0066730 2024-09-13T18:21:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.953+0000 7f96b3fff700 1 -- 172.21.15.40:0/940504477 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f96a0066730 msgr2=0x7f96a0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.961 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.953+0000 7f96b3fff700 1 --2- 172.21.15.40:0/940504477 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f96a0066730 0x7f96a0068bf0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009c40 tx=0x7f96a4008040 comp rx=0 tx=0).stop 2024-09-13T18:21:25.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.953+0000 7f96b3fff700 1 -- 172.21.15.40:0/940504477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96b4153730 msgr2=0x7f96b40c99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:25.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.953+0000 7f96b3fff700 1 --2- 172.21.15.40:0/940504477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96b4153730 0x7f96b40c99a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f96a800c730 tx=0x7f96a800caf0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.953+0000 7f96b3fff700 1 -- 172.21.15.40:0/940504477 shutdown_connections 2024-09-13T18:21:25.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.953+0000 7f96b3fff700 1 --2- 172.21.15.40:0/940504477 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f96a0066730 0x7f96a0068bf0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.953+0000 7f96b3fff700 1 --2- 172.21.15.40:0/940504477 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f96b415cfe0 0x7f96b414aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.962 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.953+0000 7f96b3fff700 1 --2- 172.21.15.40:0/940504477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f96b4153730 0x7f96b40c99a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.953+0000 7f96b3fff700 1 --2- 172.21.15.40:0/940504477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96b40c7fb0 0x7f96b415b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:25.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.953+0000 7f96b3fff700 1 -- 172.21.15.40:0/940504477 >> 172.21.15.40:0/940504477 conn(0x7f96b406e6d0 msgr2=0x7f96b406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:25.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.953+0000 7f96b3fff700 1 -- 172.21.15.40:0/940504477 shutdown_connections 2024-09-13T18:21:25.963 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:25.953+0000 7f96b3fff700 1 -- 172.21.15.40:0/940504477 wait complete. 2024-09-13T18:21:25.964 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 7 to deep-scrub 2024-09-13T18:21:25.972 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:21:26.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa016ddc700 1 Processor -- start 2024-09-13T18:21:26.078 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa016ddc700 1 -- start start 2024-09-13T18:21:26.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa016ddc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa010150250 0x7fa0100c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa016ddc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa010070370 0x7fa0100c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa016ddc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa010159670 0x7fa010159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa016ddc700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0100710a0 con 0x7fa010150250 2024-09-13T18:21:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa016ddc700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa010072900 con 0x7fa010159670 2024-09-13T18:21:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa016ddc700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa010071d80 con 0x7fa010070370 2024-09-13T18:21:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa015dda700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa010150250 0x7fa0100c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa015dda700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa010150250 0x7fa0100c4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47174/0 (socket says 172.21.15.40:47174) 2024-09-13T18:21:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa015dda700 1 -- 172.21.15.40:0/3368453639 learned_addr learned my addr 172.21.15.40:0/3368453639 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa0155d9700 1 --2- 172.21.15.40:0/3368453639 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa010070370 0x7fa0100c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa0165db700 1 --2- 172.21.15.40:0/3368453639 >> v2:172.21.15.43:3300/0 conn(0x7fa010159670 0x7fa010159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa015dda700 1 -- 172.21.15.40:0/3368453639 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa010070370 msgr2=0x7fa0100c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:26.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa015dda700 1 --2- 172.21.15.40:0/3368453639 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa010070370 0x7fa0100c4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa015dda700 1 -- 172.21.15.40:0/3368453639 >> v2:172.21.15.43:3300/0 conn(0x7fa010159670 msgr2=0x7fa010159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa015dda700 1 --2- 172.21.15.40:0/3368453639 >> v2:172.21.15.43:3300/0 conn(0x7fa010159670 0x7fa010159ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa015dda700 1 -- 172.21.15.40:0/3368453639 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa010072060 con 0x7fa010150250 2024-09-13T18:21:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa0155d9700 1 --2- 172.21.15.40:0/3368453639 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa010070370 0x7fa0100c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:21:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa015dda700 1 --2- 172.21.15.40:0/3368453639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa010150250 0x7fa0100c4580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa0100c8ef0 tx=0x7fa00c00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01caf6e1613c83a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa014dd8700 1 -- 172.21.15.40:0/3368453639 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c00dd30 con 0x7fa010150250 2024-09-13T18:21:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa014dd8700 1 -- 172.21.15.40:0/3368453639 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c00de90 con 0x7fa010150250 2024-09-13T18:21:26.082 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa014dd8700 1 -- 172.21.15.40:0/3368453639 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c010690 con 0x7fa010150250 2024-09-13T18:21:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa016ddc700 1 -- 172.21.15.40:0/3368453639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa010150250 msgr2=0x7fa0100c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa016ddc700 1 --2- 172.21.15.40:0/3368453639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa010150250 0x7fa0100c4580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa0100c8ef0 tx=0x7fa00c00be90 comp rx=0 tx=0).stop 2024-09-13T18:21:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa016ddc700 1 -- 172.21.15.40:0/3368453639 shutdown_connections 2024-09-13T18:21:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa016ddc700 1 --2- 172.21.15.40:0/3368453639 >> v2:172.21.15.43:3300/0 conn(0x7fa010159670 0x7fa010159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:26.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa016ddc700 1 --2- 172.21.15.40:0/3368453639 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa010070370 0x7fa0100c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:26.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa016ddc700 1 --2- 172.21.15.40:0/3368453639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa010150250 0x7fa0100c4580 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:26.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa016ddc700 1 -- 172.21.15.40:0/3368453639 >> 172.21.15.40:0/3368453639 conn(0x7fa0100cb4c0 msgr2=0x7fa0100cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:26.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa016ddc700 1 -- 172.21.15.40:0/3368453639 shutdown_connections 2024-09-13T18:21:26.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.073+0000 7fa016ddc700 1 -- 172.21.15.40:0/3368453639 wait complete. 2024-09-13T18:21:26.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.077+0000 7fa016ddc700 1 Processor -- start 2024-09-13T18:21:26.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.077+0000 7fa016ddc700 1 -- start start 2024-09-13T18:21:26.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.077+0000 7fa016ddc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa010070370 0x7fa010151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.077+0000 7fa016ddc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa010150250 0x7fa0100c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.077+0000 7fa016ddc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa010159670 0x7fa0101eec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.077+0000 7fa016ddc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0101ee840 con 0x7fa010150250 2024-09-13T18:21:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.077+0000 7fa016ddc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa010152940 con 0x7fa010159670 2024-09-13T18:21:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.077+0000 7fa016ddc700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0101ee4a0 con 0x7fa010070370 2024-09-13T18:21:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.077+0000 7fa0155d9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa010150250 0x7fa0100c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.077+0000 7fa0155d9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa010150250 0x7fa0100c4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:47190/0 (socket says 172.21.15.40:47190) 2024-09-13T18:21:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.077+0000 7fa0155d9700 1 -- 172.21.15.40:0/1994606824 learned_addr learned my addr 172.21.15.40:0/1994606824 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.077+0000 7fa015dda700 1 --2- 172.21.15.40:0/1994606824 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa010070370 0x7fa010151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.077+0000 7fa0165db700 1 --2- 172.21.15.40:0/1994606824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa010159670 0x7fa0101eec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.077+0000 7fa0155d9700 1 -- 172.21.15.40:0/1994606824 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa010070370 msgr2=0x7fa010151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:26.087 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.077+0000 7fa0155d9700 1 --2- 172.21.15.40:0/1994606824 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa010070370 0x7fa010151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.077+0000 7fa0155d9700 1 -- 172.21.15.40:0/1994606824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa010159670 msgr2=0x7fa0101eec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.077+0000 7fa0155d9700 1 --2- 172.21.15.40:0/1994606824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa010159670 0x7fa0101eec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.077+0000 7fa0155d9700 1 -- 172.21.15.40:0/1994606824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c00b810 con 0x7fa010150250 2024-09-13T18:21:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.077+0000 7fa0155d9700 1 --2- 172.21.15.40:0/1994606824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa010150250 0x7fa0100c4cb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa00000e840 tx=0x7fa00000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.077+0000 7fa0067fc700 1 -- 172.21.15.40:0/1994606824 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa000013610 con 0x7fa010150250 2024-09-13T18:21:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.081+0000 7fa0067fc700 1 -- 172.21.15.40:0/1994606824 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa000013770 con 0x7fa010150250 2024-09-13T18:21:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.081+0000 7fa016ddc700 1 -- 172.21.15.40:0/1994606824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0101530b0 con 0x7fa010150250 2024-09-13T18:21:26.088 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.081+0000 7fa0067fc700 1 -- 172.21.15.40:0/1994606824 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa000013b20 con 0x7fa010150250 2024-09-13T18:21:26.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.081+0000 7fa016ddc700 1 -- 172.21.15.40:0/1994606824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa010153670 con 0x7fa010150250 2024-09-13T18:21:26.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.081+0000 7fa016ddc700 1 -- 172.21.15.40:0/1994606824 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0101eeab0 con 0x7fa010150250 2024-09-13T18:21:26.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.081+0000 7fa0067fc700 1 -- 172.21.15.40:0/1994606824 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa000013c80 con 0x7fa010150250 2024-09-13T18:21:26.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.081+0000 7fa0067fc700 1 --2- 172.21.15.40:0/1994606824 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9ff4066730 0x7f9ff4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.081+0000 7fa015dda700 1 --2- 172.21.15.40:0/1994606824 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9ff4066730 0x7f9ff4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.081+0000 7fa0067fc700 1 -- 172.21.15.40:0/1994606824 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa000084130 con 0x7fa010150250 2024-09-13T18:21:26.090 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.081+0000 7fa015dda700 1 --2- 172.21.15.40:0/1994606824 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9ff4066730 0x7f9ff4068bf0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa00c000c00 tx=0x7fa00c013000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:26.096 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.089+0000 7fa0067fc700 1 -- 172.21.15.40:0/1994606824 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa000051040 con 0x7fa010150250 2024-09-13T18:21:26.263 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.257+0000 7fa016ddc700 1 -- 172.21.15.40:0/1994606824 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0100c4060 con 0x7f9ff4066730 2024-09-13T18:21:26.280 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.273+0000 7fa0067fc700 1 -- 172.21.15.40:0/1994606824 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+368879 (secure 0 0 0) 0x7fa00c020040 con 0x7f9ff4066730 2024-09-13T18:21:26.287 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T18:21:26.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.277+0000 7fa016ddc700 1 -- 172.21.15.40:0/1994606824 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9ff4066730 msgr2=0x7f9ff4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:26.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.277+0000 7fa016ddc700 1 --2- 172.21.15.40:0/1994606824 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9ff4066730 0x7f9ff4068bf0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa00c000c00 tx=0x7fa00c013000 comp rx=0 tx=0).stop 2024-09-13T18:21:26.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.277+0000 7fa016ddc700 1 -- 172.21.15.40:0/1994606824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa010150250 msgr2=0x7fa0100c4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:26.290 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.277+0000 7fa016ddc700 1 --2- 172.21.15.40:0/1994606824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa010150250 0x7fa0100c4cb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa00000e840 tx=0x7fa00000eb50 comp rx=0 tx=0).stop 2024-09-13T18:21:26.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.281+0000 7fa016ddc700 1 -- 172.21.15.40:0/1994606824 shutdown_connections 2024-09-13T18:21:26.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.281+0000 7fa016ddc700 1 --2- 172.21.15.40:0/1994606824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa010159670 0x7fa0101eec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:26.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.281+0000 7fa016ddc700 1 --2- 172.21.15.40:0/1994606824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa010150250 0x7fa0100c4cb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:26.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.281+0000 7fa016ddc700 1 --2- 172.21.15.40:0/1994606824 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f9ff4066730 0x7f9ff4068bf0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:26.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.281+0000 7fa016ddc700 1 --2- 172.21.15.40:0/1994606824 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa010070370 0x7fa010151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:26.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.281+0000 7fa016ddc700 1 -- 172.21.15.40:0/1994606824 >> 172.21.15.40:0/1994606824 conn(0x7fa0100cb4c0 msgr2=0x7fa0100c28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:26.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.281+0000 7fa016ddc700 1 -- 172.21.15.40:0/1994606824 shutdown_connections 2024-09-13T18:21:26.291 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:26.281+0000 7fa016ddc700 1 -- 172.21.15.40:0/1994606824 wait complete. 2024-09-13T18:21:26.291 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-13T18:21:26.298 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":952,"stamp":"2024-09-13T18:21:25.777273+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":16532352,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":237,"num_read_kb":279,"num_write":755012,"num_write_kb":16125724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183058,"ondisk_log_size":183058,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10715160,"kb_used_data":34648,"kb_used_omap":12,"kb_used_meta":10680435,"kb_avail":744259560,"statfs":{"total":773094113280,"available":762121789440,"internally_reserved":0,"allocated":35479552,"data_stored":33992096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":10936766008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005242"},"pg_stats":[{"pgid":"2.1f","version":"27'268","reported_seq":299,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.261706+0000","last_change":"2024-09-13T17:50:34.212158+0000","last_active":"2024-09-13T18:21:13.261706+0000","last_peered":"2024-09-13T18:21:13.261706+0000","last_clean":"2024-09-13T18:21:13.261706+0000","last_became_active":"2024-09-13T17:50:34.211867+0000","last_became_peered":"2024-09-13T17:50:34.211867+0000","last_unstale":"2024-09-13T18:21:13.261706+0000","last_undegraded":"2024-09-13T18:21:13.261706+0000","last_fullsized":"2024-09-13T18:21:13.261706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:14.671095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3260943,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":267,"num_write_kb":3318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'10","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306110+0000","last_change":"2024-09-13T17:50:34.587396+0000","last_active":"2024-09-13T18:21:13.306110+0000","last_peered":"2024-09-13T18:21:13.306110+0000","last_clean":"2024-09-13T18:21:13.306110+0000","last_became_active":"2024-09-13T17:50:34.587108+0000","last_became_peered":"2024-09-13T17:50:34.587108+0000","last_unstale":"2024-09-13T18:21:13.306110+0000","last_undegraded":"2024-09-13T18:21:13.306110+0000","last_fullsized":"2024-09-13T18:21:13.306110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:39:16.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'5594","reported_seq":5620,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.876951+0000","last_change":"2024-09-13T17:50:38.615059+0000","last_active":"2024-09-13T18:21:02.876951+0000","last_peered":"2024-09-13T18:21:02.876951+0000","last_clean":"2024-09-13T18:21:02.876951+0000","last_became_active":"2024-09-13T17:50:38.614723+0000","last_became_peered":"2024-09-13T17:50:38.614723+0000","last_unstale":"2024-09-13T18:21:02.876951+0000","last_undegraded":"2024-09-13T18:21:02.876951+0000","last_fullsized":"2024-09-13T18:21:02.876951+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5594,"log_dups_size":0,"ondisk_log_size":5594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:23:01.902181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23732,"num_write_kb":533374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'6","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.773351+0000","last_change":"2024-09-13T17:50:34.204760+0000","last_active":"2024-09-13T18:21:02.773351+0000","last_peered":"2024-09-13T18:21:02.773351+0000","last_clean":"2024-09-13T18:21:02.773351+0000","last_became_active":"2024-09-13T17:50:34.202930+0000","last_became_peered":"2024-09-13T17:50:34.202930+0000","last_unstale":"2024-09-13T18:21:02.773351+0000","last_undegraded":"2024-09-13T18:21:02.773351+0000","last_fullsized":"2024-09-13T18:21:02.773351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:39:59.784701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'20","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306487+0000","last_change":"2024-09-13T17:51:16.366352+0000","last_active":"2024-09-13T18:21:13.306487+0000","last_peered":"2024-09-13T18:21:13.306487+0000","last_clean":"2024-09-13T18:21:13.306487+0000","last_became_active":"2024-09-13T17:51:16.365492+0000","last_became_peered":"2024-09-13T17:51:16.365492+0000","last_unstale":"2024-09-13T18:21:13.306487+0000","last_undegraded":"2024-09-13T18:21:13.306487+0000","last_fullsized":"2024-09-13T18:21:13.306487+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:42.192182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'5615","reported_seq":5641,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.875980+0000","last_change":"2024-09-13T17:50:38.618751+0000","last_active":"2024-09-13T18:21:02.875980+0000","last_peered":"2024-09-13T18:21:02.875980+0000","last_clean":"2024-09-13T18:21:02.875980+0000","last_became_active":"2024-09-13T17:50:38.618423+0000","last_became_peered":"2024-09-13T17:50:38.618423+0000","last_unstale":"2024-09-13T18:21:02.875980+0000","last_undegraded":"2024-09-13T18:21:02.875980+0000","last_fullsized":"2024-09-13T18:21:02.875980+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5615,"log_dups_size":0,"ondisk_log_size":5615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:50:02.418785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23762,"num_write_kb":516686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'11","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.801229+0000","last_change":"2024-09-13T17:50:34.214972+0000","last_active":"2024-09-13T18:21:02.801229+0000","last_peered":"2024-09-13T18:21:02.801229+0000","last_clean":"2024-09-13T18:21:02.801229+0000","last_became_active":"2024-09-13T17:50:34.214652+0000","last_became_peered":"2024-09-13T17:50:34.214652+0000","last_unstale":"2024-09-13T18:21:02.801229+0000","last_undegraded":"2024-09-13T18:21:02.801229+0000","last_fullsized":"2024-09-13T18:21:02.801229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:57.176854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'18","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.292937+0000","last_change":"2024-09-13T17:51:16.281831+0000","last_active":"2024-09-13T18:21:13.292937+0000","last_peered":"2024-09-13T18:21:13.292937+0000","last_clean":"2024-09-13T18:21:13.292937+0000","last_became_active":"2024-09-13T17:51:16.281549+0000","last_became_peered":"2024-09-13T17:51:16.281549+0000","last_unstale":"2024-09-13T18:21:13.292937+0000","last_undegraded":"2024-09-13T18:21:13.292937+0000","last_fullsized":"2024-09-13T18:21:13.292937+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:43.894437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"27'5142","reported_seq":5168,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.803386+0000","last_change":"2024-09-13T17:50:38.616219+0000","last_active":"2024-09-13T18:21:02.803386+0000","last_peered":"2024-09-13T18:21:02.803386+0000","last_clean":"2024-09-13T18:21:02.803386+0000","last_became_active":"2024-09-13T17:50:38.614329+0000","last_became_peered":"2024-09-13T17:50:38.614329+0000","last_unstale":"2024-09-13T18:21:02.803386+0000","last_undegraded":"2024-09-13T18:21:02.803386+0000","last_fullsized":"2024-09-13T18:21:02.803386+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5142,"log_dups_size":0,"ondisk_log_size":5142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:31:25.231536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21334,"num_write_kb":478261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'8","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.159841+0000","last_change":"2024-09-13T17:50:34.216123+0000","last_active":"2024-09-13T18:20:25.159841+0000","last_peered":"2024-09-13T18:20:25.159841+0000","last_clean":"2024-09-13T18:20:25.159841+0000","last_became_active":"2024-09-13T17:50:34.215661+0000","last_became_peered":"2024-09-13T17:50:34.215661+0000","last_unstale":"2024-09-13T18:20:25.159841+0000","last_undegraded":"2024-09-13T18:20:25.159841+0000","last_fullsized":"2024-09-13T18:20:25.159841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:20.895037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'30","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.318958+0000","last_change":"2024-09-13T17:50:34.599673+0000","last_active":"2024-09-13T18:21:13.318958+0000","last_peered":"2024-09-13T18:21:13.318958+0000","last_clean":"2024-09-13T18:21:13.318958+0000","last_became_active":"2024-09-13T17:50:34.599409+0000","last_became_peered":"2024-09-13T17:50:34.599409+0000","last_unstale":"2024-09-13T18:21:13.318958+0000","last_undegraded":"2024-09-13T18:21:13.318958+0000","last_fullsized":"2024-09-13T18:21:13.318958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:03:31.394385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'4871","reported_seq":4897,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.743078+0000","last_change":"2024-09-13T17:50:38.616317+0000","last_active":"2024-09-13T18:21:02.743078+0000","last_peered":"2024-09-13T18:21:02.743078+0000","last_clean":"2024-09-13T18:21:02.743078+0000","last_became_active":"2024-09-13T17:50:38.614311+0000","last_became_peered":"2024-09-13T17:50:38.614311+0000","last_unstale":"2024-09-13T18:21:02.743078+0000","last_undegraded":"2024-09-13T18:21:02.743078+0000","last_fullsized":"2024-09-13T18:21:02.743078+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4871,"log_dups_size":0,"ondisk_log_size":4871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:29.221522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20106,"num_write_kb":432357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'14","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.313143+0000","last_change":"2024-09-13T17:50:34.599490+0000","last_active":"2024-09-13T18:21:13.313143+0000","last_peered":"2024-09-13T18:21:13.313143+0000","last_clean":"2024-09-13T18:21:13.313143+0000","last_became_active":"2024-09-13T17:50:34.599207+0000","last_became_peered":"2024-09-13T17:50:34.599207+0000","last_unstale":"2024-09-13T18:21:13.313143+0000","last_undegraded":"2024-09-13T18:21:13.313143+0000","last_fullsized":"2024-09-13T18:21:13.313143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:11.178161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'1092","reported_seq":1122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.671116+0000","last_change":"2024-09-13T17:50:34.215697+0000","last_active":"2024-09-13T18:21:02.671116+0000","last_peered":"2024-09-13T18:21:02.671116+0000","last_clean":"2024-09-13T18:21:02.671116+0000","last_became_active":"2024-09-13T17:50:34.215409+0000","last_became_peered":"2024-09-13T17:50:34.215409+0000","last_unstale":"2024-09-13T18:21:02.671116+0000","last_undegraded":"2024-09-13T18:21:02.671116+0000","last_fullsized":"2024-09-13T18:21:02.671116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:34:21.609567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1088,"num_write_kb":4339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'5394","reported_seq":5421,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.852704+0000","last_change":"2024-09-13T17:50:38.611817+0000","last_active":"2024-09-13T18:21:02.852704+0000","last_peered":"2024-09-13T18:21:02.852704+0000","last_clean":"2024-09-13T18:21:02.852704+0000","last_became_active":"2024-09-13T17:50:38.611263+0000","last_became_peered":"2024-09-13T17:50:38.611263+0000","last_unstale":"2024-09-13T18:21:02.852704+0000","last_undegraded":"2024-09-13T18:21:02.852704+0000","last_fullsized":"2024-09-13T18:21:02.852704+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5394,"log_dups_size":0,"ondisk_log_size":5394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:56:47.466672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22326,"num_write_kb":494524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'19","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.308364+0000","last_change":"2024-09-13T17:50:34.586307+0000","last_active":"2024-09-13T18:21:13.308364+0000","last_peered":"2024-09-13T18:21:13.308364+0000","last_clean":"2024-09-13T18:21:13.308364+0000","last_became_active":"2024-09-13T17:50:34.585831+0000","last_became_peered":"2024-09-13T17:50:34.585831+0000","last_unstale":"2024-09-13T18:21:13.308364+0000","last_undegraded":"2024-09-13T18:21:13.308364+0000","last_fullsized":"2024-09-13T18:21:13.308364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:40.912352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'9","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.773590+0000","last_change":"2024-09-13T17:50:34.211850+0000","last_active":"2024-09-13T18:21:02.773590+0000","last_peered":"2024-09-13T18:21:02.773590+0000","last_clean":"2024-09-13T18:21:02.773590+0000","last_became_active":"2024-09-13T17:50:34.211594+0000","last_became_peered":"2024-09-13T17:50:34.211594+0000","last_unstale":"2024-09-13T18:21:02.773590+0000","last_undegraded":"2024-09-13T18:21:02.773590+0000","last_fullsized":"2024-09-13T18:21:02.773590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:35:02.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'5504","reported_seq":5531,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.796881+0000","last_change":"2024-09-13T17:50:38.613576+0000","last_active":"2024-09-13T18:21:02.796881+0000","last_peered":"2024-09-13T18:21:02.796881+0000","last_clean":"2024-09-13T18:21:02.796881+0000","last_became_active":"2024-09-13T17:50:38.612577+0000","last_became_peered":"2024-09-13T17:50:38.612577+0000","last_unstale":"2024-09-13T18:21:02.796881+0000","last_undegraded":"2024-09-13T18:21:02.796881+0000","last_fullsized":"2024-09-13T18:21:02.796881+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5504,"log_dups_size":0,"ondisk_log_size":5504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:25.551060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23286,"num_write_kb":502495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'19","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306452+0000","last_change":"2024-09-13T17:50:34.587273+0000","last_active":"2024-09-13T18:21:13.306452+0000","last_peered":"2024-09-13T18:21:13.306452+0000","last_clean":"2024-09-13T18:21:13.306452+0000","last_became_active":"2024-09-13T17:50:34.587027+0000","last_became_peered":"2024-09-13T17:50:34.587027+0000","last_unstale":"2024-09-13T18:21:13.306452+0000","last_undegraded":"2024-09-13T18:21:13.306452+0000","last_fullsized":"2024-09-13T18:21:13.306452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:31:58.826737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'5","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.773643+0000","last_change":"2024-09-13T17:50:34.211845+0000","last_active":"2024-09-13T18:21:02.773643+0000","last_peered":"2024-09-13T18:21:02.773643+0000","last_clean":"2024-09-13T18:21:02.773643+0000","last_became_active":"2024-09-13T17:50:34.211568+0000","last_became_peered":"2024-09-13T17:50:34.211568+0000","last_unstale":"2024-09-13T18:21:02.773643+0000","last_undegraded":"2024-09-13T18:21:02.773643+0000","last_fullsized":"2024-09-13T18:21:02.773643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:56.348984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'6938","reported_seq":6965,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.647468+0000","last_change":"2024-09-13T17:50:38.610292+0000","last_active":"2024-09-13T18:21:02.647468+0000","last_peered":"2024-09-13T18:21:02.647468+0000","last_clean":"2024-09-13T18:21:02.647468+0000","last_became_active":"2024-09-13T17:50:38.609832+0000","last_became_peered":"2024-09-13T17:50:38.609832+0000","last_unstale":"2024-09-13T18:21:02.647468+0000","last_undegraded":"2024-09-13T18:21:02.647468+0000","last_fullsized":"2024-09-13T18:21:02.647468+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6938,"log_dups_size":0,"ondisk_log_size":6938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:13.783068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28644,"num_write_kb":622131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"27'12","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.931907+0000","last_change":"2024-09-13T17:50:34.593566+0000","last_active":"2024-09-13T18:21:02.931907+0000","last_peered":"2024-09-13T18:21:02.931907+0000","last_clean":"2024-09-13T18:21:02.931907+0000","last_became_active":"2024-09-13T17:50:34.588794+0000","last_became_peered":"2024-09-13T17:50:34.588794+0000","last_unstale":"2024-09-13T18:21:02.931907+0000","last_undegraded":"2024-09-13T18:21:02.931907+0000","last_fullsized":"2024-09-13T18:21:02.931907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:41:13.192453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'6","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.689388+0000","last_change":"2024-09-13T17:50:34.214638+0000","last_active":"2024-09-13T18:21:02.689388+0000","last_peered":"2024-09-13T18:21:02.689388+0000","last_clean":"2024-09-13T18:21:02.689388+0000","last_became_active":"2024-09-13T17:50:34.214292+0000","last_became_peered":"2024-09-13T17:50:34.214292+0000","last_unstale":"2024-09-13T18:21:02.689388+0000","last_undegraded":"2024-09-13T18:21:02.689388+0000","last_fullsized":"2024-09-13T18:21:02.689388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:57.377104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'4588","reported_seq":4615,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.886736+0000","last_change":"2024-09-13T17:50:38.613368+0000","last_active":"2024-09-13T18:21:02.886736+0000","last_peered":"2024-09-13T18:21:02.886736+0000","last_clean":"2024-09-13T18:21:02.886736+0000","last_became_active":"2024-09-13T17:50:38.613010+0000","last_became_peered":"2024-09-13T17:50:38.613010+0000","last_unstale":"2024-09-13T18:21:02.886736+0000","last_undegraded":"2024-09-13T18:21:02.886736+0000","last_fullsized":"2024-09-13T18:21:02.886736+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4588,"log_dups_size":0,"ondisk_log_size":4588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:42.706625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19354,"num_write_kb":454021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'26","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.302413+0000","last_change":"2024-09-13T17:50:34.582327+0000","last_active":"2024-09-13T18:21:13.302413+0000","last_peered":"2024-09-13T18:21:13.302413+0000","last_clean":"2024-09-13T18:21:13.302413+0000","last_became_active":"2024-09-13T17:50:34.582050+0000","last_became_peered":"2024-09-13T17:50:34.582050+0000","last_unstale":"2024-09-13T18:21:13.302413+0000","last_undegraded":"2024-09-13T18:21:13.302413+0000","last_fullsized":"2024-09-13T18:21:13.302413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:52.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'9","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.838047+0000","last_change":"2024-09-13T17:50:34.211512+0000","last_active":"2024-09-13T18:21:02.838047+0000","last_peered":"2024-09-13T18:21:02.838047+0000","last_clean":"2024-09-13T18:21:02.838047+0000","last_became_active":"2024-09-13T17:50:34.211230+0000","last_became_peered":"2024-09-13T17:50:34.211230+0000","last_unstale":"2024-09-13T18:21:02.838047+0000","last_undegraded":"2024-09-13T18:21:02.838047+0000","last_fullsized":"2024-09-13T18:21:02.838047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:40.707777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'6024","reported_seq":6052,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.702767+0000","last_change":"2024-09-13T17:50:38.608255+0000","last_active":"2024-09-13T18:21:02.702767+0000","last_peered":"2024-09-13T18:21:02.702767+0000","last_clean":"2024-09-13T18:21:02.702767+0000","last_became_active":"2024-09-13T17:50:38.606697+0000","last_became_peered":"2024-09-13T17:50:38.606697+0000","last_unstale":"2024-09-13T18:21:02.702767+0000","last_undegraded":"2024-09-13T18:21:02.702767+0000","last_fullsized":"2024-09-13T18:21:02.702767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6024,"log_dups_size":0,"ondisk_log_size":6024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:47.265007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25552,"num_write_kb":507487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'18","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.314914+0000","last_change":"2024-09-13T17:50:34.594476+0000","last_active":"2024-09-13T18:21:13.314914+0000","last_peered":"2024-09-13T18:21:13.314914+0000","last_clean":"2024-09-13T18:21:13.314914+0000","last_became_active":"2024-09-13T17:50:34.594215+0000","last_became_peered":"2024-09-13T17:50:34.594215+0000","last_unstale":"2024-09-13T18:21:13.314914+0000","last_undegraded":"2024-09-13T18:21:13.314914+0000","last_fullsized":"2024-09-13T18:21:13.314914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:02:27.528019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'284","reported_seq":315,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.685813+0000","last_change":"2024-09-13T17:50:34.204018+0000","last_active":"2024-09-13T18:21:02.685813+0000","last_peered":"2024-09-13T18:21:02.685813+0000","last_clean":"2024-09-13T18:21:02.685813+0000","last_became_active":"2024-09-13T17:50:34.203702+0000","last_became_peered":"2024-09-13T17:50:34.203702+0000","last_unstale":"2024-09-13T18:21:02.685813+0000","last_undegraded":"2024-09-13T18:21:02.685813+0000","last_fullsized":"2024-09-13T18:21:02.685813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:44.431226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":605,"num_write_kb":197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'6030","reported_seq":6069,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.817288+0000","last_change":"2024-09-13T17:51:16.365782+0000","last_active":"2024-09-13T18:21:02.817288+0000","last_peered":"2024-09-13T18:21:02.817288+0000","last_clean":"2024-09-13T18:21:02.817288+0000","last_became_active":"2024-09-13T17:51:16.364540+0000","last_became_peered":"2024-09-13T17:51:16.364540+0000","last_unstale":"2024-09-13T18:21:02.817288+0000","last_undegraded":"2024-09-13T18:21:02.817288+0000","last_fullsized":"2024-09-13T18:21:02.817288+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6030,"log_dups_size":0,"ondisk_log_size":6030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:07:18.753557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25617,"num_write_kb":548663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'18","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.290374+0000","last_change":"2024-09-13T17:50:34.586802+0000","last_active":"2024-09-13T18:21:13.290374+0000","last_peered":"2024-09-13T18:21:13.290374+0000","last_clean":"2024-09-13T18:21:13.290374+0000","last_became_active":"2024-09-13T17:50:34.586552+0000","last_became_peered":"2024-09-13T17:50:34.586552+0000","last_unstale":"2024-09-13T18:21:13.290374+0000","last_undegraded":"2024-09-13T18:21:13.290374+0000","last_fullsized":"2024-09-13T18:21:13.290374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:43.383975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'7","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.741537+0000","last_change":"2024-09-13T17:50:34.217083+0000","last_active":"2024-09-13T18:21:02.741537+0000","last_peered":"2024-09-13T18:21:02.741537+0000","last_clean":"2024-09-13T18:21:02.741537+0000","last_became_active":"2024-09-13T17:50:34.216758+0000","last_became_peered":"2024-09-13T17:50:34.216758+0000","last_unstale":"2024-09-13T18:21:02.741537+0000","last_undegraded":"2024-09-13T18:21:02.741537+0000","last_fullsized":"2024-09-13T18:21:02.741537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:36.473942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'4887","reported_seq":4915,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.617066+0000","last_change":"2024-09-13T17:50:38.614667+0000","last_active":"2024-09-13T18:21:02.617066+0000","last_peered":"2024-09-13T18:21:02.617066+0000","last_clean":"2024-09-13T18:21:02.617066+0000","last_became_active":"2024-09-13T17:50:38.614284+0000","last_became_peered":"2024-09-13T17:50:38.614284+0000","last_unstale":"2024-09-13T18:21:02.617066+0000","last_undegraded":"2024-09-13T18:21:02.617066+0000","last_fullsized":"2024-09-13T18:21:02.617066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4887,"log_dups_size":0,"ondisk_log_size":4887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:29.616794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20690,"num_write_kb":442963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'16","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.315360+0000","last_change":"2024-09-13T17:50:34.588603+0000","last_active":"2024-09-13T18:21:13.315360+0000","last_peered":"2024-09-13T18:21:13.315360+0000","last_clean":"2024-09-13T18:21:13.315360+0000","last_became_active":"2024-09-13T17:50:34.588317+0000","last_became_peered":"2024-09-13T17:50:34.588317+0000","last_unstale":"2024-09-13T18:21:13.315360+0000","last_undegraded":"2024-09-13T18:21:13.315360+0000","last_fullsized":"2024-09-13T18:21:13.315360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:03.743130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'3","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.727771+0000","last_change":"2024-09-13T17:50:34.204335+0000","last_active":"2024-09-13T18:21:02.727771+0000","last_peered":"2024-09-13T18:21:02.727771+0000","last_clean":"2024-09-13T18:21:02.727771+0000","last_became_active":"2024-09-13T17:50:34.204060+0000","last_became_peered":"2024-09-13T17:50:34.204060+0000","last_unstale":"2024-09-13T18:21:02.727771+0000","last_undegraded":"2024-09-13T18:21:02.727771+0000","last_fullsized":"2024-09-13T18:21:02.727771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:54:41.831344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'6031","reported_seq":6059,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.587876+0000","last_change":"2024-09-13T17:50:38.619590+0000","last_active":"2024-09-13T18:21:02.587876+0000","last_peered":"2024-09-13T18:21:02.587876+0000","last_clean":"2024-09-13T18:21:02.587876+0000","last_became_active":"2024-09-13T17:50:38.617257+0000","last_became_peered":"2024-09-13T17:50:38.617257+0000","last_unstale":"2024-09-13T18:21:02.587876+0000","last_undegraded":"2024-09-13T18:21:02.587876+0000","last_fullsized":"2024-09-13T18:21:02.587876+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6031,"log_dups_size":0,"ondisk_log_size":6031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:13:34.870291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25529,"num_write_kb":564991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'16","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310390+0000","last_change":"2024-09-13T17:50:34.588466+0000","last_active":"2024-09-13T18:21:13.310390+0000","last_peered":"2024-09-13T18:21:13.310390+0000","last_clean":"2024-09-13T18:21:13.310390+0000","last_became_active":"2024-09-13T17:50:34.588063+0000","last_became_peered":"2024-09-13T17:50:34.588063+0000","last_unstale":"2024-09-13T18:21:13.310390+0000","last_undegraded":"2024-09-13T18:21:13.310390+0000","last_fullsized":"2024-09-13T18:21:13.310390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:43:00.669203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'4","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.629940+0000","last_change":"2024-09-13T17:50:34.209239+0000","last_active":"2024-09-13T18:21:02.629940+0000","last_peered":"2024-09-13T18:21:02.629940+0000","last_clean":"2024-09-13T18:21:02.629940+0000","last_became_active":"2024-09-13T17:50:34.208196+0000","last_became_peered":"2024-09-13T17:50:34.208196+0000","last_unstale":"2024-09-13T18:21:02.629940+0000","last_undegraded":"2024-09-13T18:21:02.629940+0000","last_fullsized":"2024-09-13T18:21:02.629940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:46:39.253917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'5410","reported_seq":5438,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.883285+0000","last_change":"2024-09-13T17:50:38.611021+0000","last_active":"2024-09-13T18:21:02.883285+0000","last_peered":"2024-09-13T18:21:02.883285+0000","last_clean":"2024-09-13T18:21:02.883285+0000","last_became_active":"2024-09-13T17:50:38.609794+0000","last_became_peered":"2024-09-13T17:50:38.609794+0000","last_unstale":"2024-09-13T18:21:02.883285+0000","last_undegraded":"2024-09-13T18:21:02.883285+0000","last_fullsized":"2024-09-13T18:21:02.883285+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5410,"log_dups_size":0,"ondisk_log_size":5410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:59.980236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22186,"num_write_kb":506224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'22","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.282060+0000","last_change":"2024-09-13T17:50:34.584290+0000","last_active":"2024-09-13T18:21:13.282060+0000","last_peered":"2024-09-13T18:21:13.282060+0000","last_clean":"2024-09-13T18:21:13.282060+0000","last_became_active":"2024-09-13T17:50:34.584023+0000","last_became_peered":"2024-09-13T17:50:34.584023+0000","last_unstale":"2024-09-13T18:21:13.282060+0000","last_undegraded":"2024-09-13T18:21:13.282060+0000","last_fullsized":"2024-09-13T18:21:13.282060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:11.304986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'127","reported_seq":279,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:31.126933+0000","last_change":"2024-09-13T17:50:34.198932+0000","last_active":"2024-09-13T18:20:31.126933+0000","last_peered":"2024-09-13T18:20:31.126933+0000","last_clean":"2024-09-13T18:20:31.126933+0000","last_became_active":"2024-09-13T17:50:34.198603+0000","last_became_peered":"2024-09-13T17:50:34.198603+0000","last_unstale":"2024-09-13T18:20:31.126933+0000","last_undegraded":"2024-09-13T18:20:31.126933+0000","last_fullsized":"2024-09-13T18:20:31.126933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:00.917126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":94032,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":179,"num_write":125,"num_write_kb":180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'5807","reported_seq":5835,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.679912+0000","last_change":"2024-09-13T17:50:38.611326+0000","last_active":"2024-09-13T18:21:02.679912+0000","last_peered":"2024-09-13T18:21:02.679912+0000","last_clean":"2024-09-13T18:21:02.679912+0000","last_became_active":"2024-09-13T17:50:38.610900+0000","last_became_peered":"2024-09-13T17:50:38.610900+0000","last_unstale":"2024-09-13T18:21:02.679912+0000","last_undegraded":"2024-09-13T18:21:02.679912+0000","last_fullsized":"2024-09-13T18:21:02.679912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5807,"log_dups_size":0,"ondisk_log_size":5807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:17:09.994736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24429,"num_write_kb":531605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'13","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310665+0000","last_change":"2024-09-13T17:50:34.594482+0000","last_active":"2024-09-13T18:21:13.310665+0000","last_peered":"2024-09-13T18:21:13.310665+0000","last_clean":"2024-09-13T18:21:13.310665+0000","last_became_active":"2024-09-13T17:50:34.594201+0000","last_became_peered":"2024-09-13T17:50:34.594201+0000","last_unstale":"2024-09-13T18:21:13.310665+0000","last_undegraded":"2024-09-13T18:21:13.310665+0000","last_fullsized":"2024-09-13T18:21:13.310665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:06.697507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'8","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.813884+0000","last_change":"2024-09-13T17:50:34.218376+0000","last_active":"2024-09-13T18:21:02.813884+0000","last_peered":"2024-09-13T18:21:02.813884+0000","last_clean":"2024-09-13T18:21:02.813884+0000","last_became_active":"2024-09-13T17:50:34.218069+0000","last_became_peered":"2024-09-13T17:50:34.218069+0000","last_unstale":"2024-09-13T18:21:02.813884+0000","last_undegraded":"2024-09-13T18:21:02.813884+0000","last_fullsized":"2024-09-13T18:21:02.813884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:53:54.288969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'4957","reported_seq":4985,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.963407+0000","last_change":"2024-09-13T17:50:38.613206+0000","last_active":"2024-09-13T18:21:02.963407+0000","last_peered":"2024-09-13T18:21:02.963407+0000","last_clean":"2024-09-13T18:21:02.963407+0000","last_became_active":"2024-09-13T17:50:38.612856+0000","last_became_peered":"2024-09-13T17:50:38.612856+0000","last_unstale":"2024-09-13T18:21:02.963407+0000","last_undegraded":"2024-09-13T18:21:02.963407+0000","last_fullsized":"2024-09-13T18:21:02.963407+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4957,"log_dups_size":0,"ondisk_log_size":4957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:31:20.190996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20520,"num_write_kb":434212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'16","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.316288+0000","last_change":"2024-09-13T17:50:34.582931+0000","last_active":"2024-09-13T18:21:13.316288+0000","last_peered":"2024-09-13T18:21:13.316288+0000","last_clean":"2024-09-13T18:21:13.316288+0000","last_became_active":"2024-09-13T17:50:34.582446+0000","last_became_peered":"2024-09-13T17:50:34.582446+0000","last_unstale":"2024-09-13T18:21:13.316288+0000","last_undegraded":"2024-09-13T18:21:13.316288+0000","last_fullsized":"2024-09-13T18:21:13.316288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:25:30.554785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'8","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.157427+0000","last_change":"2024-09-13T17:50:34.201237+0000","last_active":"2024-09-13T18:20:25.157427+0000","last_peered":"2024-09-13T18:20:25.157427+0000","last_clean":"2024-09-13T18:20:25.157427+0000","last_became_active":"2024-09-13T17:50:34.200873+0000","last_became_peered":"2024-09-13T17:50:34.200873+0000","last_unstale":"2024-09-13T18:20:25.157427+0000","last_undegraded":"2024-09-13T18:20:25.157427+0000","last_fullsized":"2024-09-13T18:20:25.157427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:02.711342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":185,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:18:31.266834+0000","last_change":"2024-09-13T17:50:22.095367+0000","last_active":"2024-09-13T18:18:31.266834+0000","last_peered":"2024-09-13T18:18:31.266834+0000","last_clean":"2024-09-13T18:18:31.266834+0000","last_became_active":"2024-09-13T17:50:22.093734+0000","last_became_peered":"2024-09-13T17:50:22.093734+0000","last_unstale":"2024-09-13T18:18:31.266834+0000","last_undegraded":"2024-09-13T18:18:31.266834+0000","last_fullsized":"2024-09-13T18:18:31.266834+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_clean_scrub_stamp":"2024-09-13T17:50:21.057812+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:23.531613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'5208","reported_seq":5236,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.886919+0000","last_change":"2024-09-13T17:50:38.618880+0000","last_active":"2024-09-13T18:21:02.886919+0000","last_peered":"2024-09-13T18:21:02.886919+0000","last_clean":"2024-09-13T18:21:02.886919+0000","last_became_active":"2024-09-13T17:50:38.618488+0000","last_became_peered":"2024-09-13T17:50:38.618488+0000","last_unstale":"2024-09-13T18:21:02.886919+0000","last_undegraded":"2024-09-13T18:21:02.886919+0000","last_fullsized":"2024-09-13T18:21:02.886919+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5208,"log_dups_size":0,"ondisk_log_size":5208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:18:03.431876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22093,"num_write_kb":481370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'22","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317778+0000","last_change":"2024-09-13T17:50:34.585881+0000","last_active":"2024-09-13T18:21:13.317778+0000","last_peered":"2024-09-13T18:21:13.317778+0000","last_clean":"2024-09-13T18:21:13.317778+0000","last_became_active":"2024-09-13T17:50:34.585166+0000","last_became_peered":"2024-09-13T17:50:34.585166+0000","last_unstale":"2024-09-13T18:21:13.317778+0000","last_undegraded":"2024-09-13T18:21:13.317778+0000","last_fullsized":"2024-09-13T18:21:13.317778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:06.654658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'5","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.654921+0000","last_change":"2024-09-13T17:50:34.210934+0000","last_active":"2024-09-13T18:21:02.654921+0000","last_peered":"2024-09-13T18:21:02.654921+0000","last_clean":"2024-09-13T18:21:02.654921+0000","last_became_active":"2024-09-13T17:50:34.210589+0000","last_became_peered":"2024-09-13T17:50:34.210589+0000","last_unstale":"2024-09-13T18:21:02.654921+0000","last_undegraded":"2024-09-13T18:21:02.654921+0000","last_fullsized":"2024-09-13T18:21:02.654921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:46:26.213295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'5302","reported_seq":5340,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.817379+0000","last_change":"2024-09-13T17:51:16.364911+0000","last_active":"2024-09-13T18:21:02.817379+0000","last_peered":"2024-09-13T18:21:02.817379+0000","last_clean":"2024-09-13T18:21:02.817379+0000","last_became_active":"2024-09-13T17:51:16.363974+0000","last_became_peered":"2024-09-13T17:51:16.363974+0000","last_unstale":"2024-09-13T18:21:02.817379+0000","last_undegraded":"2024-09-13T18:21:02.817379+0000","last_fullsized":"2024-09-13T18:21:02.817379+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5302,"log_dups_size":0,"ondisk_log_size":5302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:12.730197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22724,"num_write_kb":488088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'20","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.309412+0000","last_change":"2024-09-13T17:50:34.590169+0000","last_active":"2024-09-13T18:21:13.309412+0000","last_peered":"2024-09-13T18:21:13.309412+0000","last_clean":"2024-09-13T18:21:13.309412+0000","last_became_active":"2024-09-13T17:50:34.589434+0000","last_became_peered":"2024-09-13T17:50:34.589434+0000","last_unstale":"2024-09-13T18:21:13.309412+0000","last_undegraded":"2024-09-13T18:21:13.309412+0000","last_fullsized":"2024-09-13T18:21:13.309412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:12.545412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'4","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:25.155671+0000","last_change":"2024-09-13T17:50:34.204269+0000","last_active":"2024-09-13T18:20:25.155671+0000","last_peered":"2024-09-13T18:20:25.155671+0000","last_clean":"2024-09-13T18:20:25.155671+0000","last_became_active":"2024-09-13T17:50:34.203874+0000","last_became_peered":"2024-09-13T17:50:34.203874+0000","last_unstale":"2024-09-13T18:20:25.155671+0000","last_undegraded":"2024-09-13T18:20:25.155671+0000","last_fullsized":"2024-09-13T18:20:25.155671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:10.410267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'5348","reported_seq":5375,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.944893+0000","last_change":"2024-09-13T17:50:38.615989+0000","last_active":"2024-09-13T18:21:02.944893+0000","last_peered":"2024-09-13T18:21:02.944893+0000","last_clean":"2024-09-13T18:21:02.944893+0000","last_became_active":"2024-09-13T17:50:38.615530+0000","last_became_peered":"2024-09-13T17:50:38.615530+0000","last_unstale":"2024-09-13T18:21:02.944893+0000","last_undegraded":"2024-09-13T18:21:02.944893+0000","last_fullsized":"2024-09-13T18:21:02.944893+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5348,"log_dups_size":0,"ondisk_log_size":5348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:41.064092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22134,"num_write_kb":473293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'20","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.321742+0000","last_change":"2024-09-13T17:50:34.595120+0000","last_active":"2024-09-13T18:21:13.321742+0000","last_peered":"2024-09-13T18:21:13.321742+0000","last_clean":"2024-09-13T18:21:13.321742+0000","last_became_active":"2024-09-13T17:50:34.594842+0000","last_became_peered":"2024-09-13T17:50:34.594842+0000","last_unstale":"2024-09-13T18:21:13.321742+0000","last_undegraded":"2024-09-13T18:21:13.321742+0000","last_fullsized":"2024-09-13T18:21:13.321742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:15:38.982524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'9","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.683517+0000","last_change":"2024-09-13T17:50:34.199957+0000","last_active":"2024-09-13T18:21:02.683517+0000","last_peered":"2024-09-13T18:21:02.683517+0000","last_clean":"2024-09-13T18:21:02.683517+0000","last_became_active":"2024-09-13T17:50:34.199650+0000","last_became_peered":"2024-09-13T17:50:34.199650+0000","last_unstale":"2024-09-13T18:21:02.683517+0000","last_undegraded":"2024-09-13T18:21:02.683517+0000","last_fullsized":"2024-09-13T18:21:02.683517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:26.929549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'5161","reported_seq":5189,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.762344+0000","last_change":"2024-09-13T17:50:38.609731+0000","last_active":"2024-09-13T18:21:02.762344+0000","last_peered":"2024-09-13T18:21:02.762344+0000","last_clean":"2024-09-13T18:21:02.762344+0000","last_became_active":"2024-09-13T17:50:38.609354+0000","last_became_peered":"2024-09-13T17:50:38.609354+0000","last_unstale":"2024-09-13T18:21:02.762344+0000","last_undegraded":"2024-09-13T18:21:02.762344+0000","last_fullsized":"2024-09-13T18:21:02.762344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5161,"log_dups_size":0,"ondisk_log_size":5161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:14.142369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21929,"num_write_kb":464406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'10","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.313609+0000","last_change":"2024-09-13T17:50:34.595146+0000","last_active":"2024-09-13T18:21:13.313609+0000","last_peered":"2024-09-13T18:21:13.313609+0000","last_clean":"2024-09-13T18:21:13.313609+0000","last_became_active":"2024-09-13T17:50:34.594876+0000","last_became_peered":"2024-09-13T17:50:34.594876+0000","last_unstale":"2024-09-13T18:21:13.313609+0000","last_undegraded":"2024-09-13T18:21:13.313609+0000","last_fullsized":"2024-09-13T18:21:13.313609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:26:10.638438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'4","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:19:51.940271+0000","last_change":"2024-09-13T17:50:34.200640+0000","last_active":"2024-09-13T18:19:51.940271+0000","last_peered":"2024-09-13T18:19:51.940271+0000","last_clean":"2024-09-13T18:19:51.940271+0000","last_became_active":"2024-09-13T17:50:34.200197+0000","last_became_peered":"2024-09-13T17:50:34.200197+0000","last_unstale":"2024-09-13T18:19:51.940271+0000","last_undegraded":"2024-09-13T18:19:51.940271+0000","last_fullsized":"2024-09-13T18:19:51.940271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:37:53.039720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'5052","reported_seq":5090,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.778127+0000","last_change":"2024-09-13T17:51:16.281995+0000","last_active":"2024-09-13T18:21:02.778127+0000","last_peered":"2024-09-13T18:21:02.778127+0000","last_clean":"2024-09-13T18:21:02.778127+0000","last_became_active":"2024-09-13T17:51:16.281476+0000","last_became_peered":"2024-09-13T17:51:16.281476+0000","last_unstale":"2024-09-13T18:21:02.778127+0000","last_undegraded":"2024-09-13T18:21:02.778127+0000","last_fullsized":"2024-09-13T18:21:02.778127+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5052,"log_dups_size":0,"ondisk_log_size":5052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:21:56.254641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20895,"num_write_kb":464282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3,6],"acting":[4,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'18","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.301423+0000","last_change":"2024-09-13T17:50:34.585741+0000","last_active":"2024-09-13T18:21:13.301423+0000","last_peered":"2024-09-13T18:21:13.301423+0000","last_clean":"2024-09-13T18:21:13.301423+0000","last_became_active":"2024-09-13T17:50:34.585464+0000","last_became_peered":"2024-09-13T17:50:34.585464+0000","last_unstale":"2024-09-13T18:21:13.301423+0000","last_undegraded":"2024-09-13T18:21:13.301423+0000","last_fullsized":"2024-09-13T18:21:13.301423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:59:22.031459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'19","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.673198+0000","last_change":"2024-09-13T17:50:34.201272+0000","last_active":"2024-09-13T18:21:02.673198+0000","last_peered":"2024-09-13T18:21:02.673198+0000","last_clean":"2024-09-13T18:21:02.673198+0000","last_became_active":"2024-09-13T17:50:34.200936+0000","last_became_peered":"2024-09-13T17:50:34.200936+0000","last_unstale":"2024-09-13T18:21:02.673198+0000","last_undegraded":"2024-09-13T18:21:02.673198+0000","last_fullsized":"2024-09-13T18:21:02.673198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:44:52.094879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'6502","reported_seq":6528,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.900747+0000","last_change":"2024-09-13T17:50:38.615229+0000","last_active":"2024-09-13T18:21:02.900747+0000","last_peered":"2024-09-13T18:21:02.900747+0000","last_clean":"2024-09-13T18:21:02.900747+0000","last_became_active":"2024-09-13T17:50:38.614739+0000","last_became_peered":"2024-09-13T17:50:38.614739+0000","last_unstale":"2024-09-13T18:21:02.900747+0000","last_undegraded":"2024-09-13T18:21:02.900747+0000","last_fullsized":"2024-09-13T18:21:02.900747+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6502,"log_dups_size":0,"ondisk_log_size":6502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:18.619529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27696,"num_write_kb":544723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'20","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.287949+0000","last_change":"2024-09-13T17:50:34.588848+0000","last_active":"2024-09-13T18:21:13.287949+0000","last_peered":"2024-09-13T18:21:13.287949+0000","last_clean":"2024-09-13T18:21:13.287949+0000","last_became_active":"2024-09-13T17:50:34.588565+0000","last_became_peered":"2024-09-13T17:50:34.588565+0000","last_unstale":"2024-09-13T18:21:13.287949+0000","last_undegraded":"2024-09-13T18:21:13.287949+0000","last_fullsized":"2024-09-13T18:21:13.287949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:47.927865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'7","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.707815+0000","last_change":"2024-09-13T17:50:34.199008+0000","last_active":"2024-09-13T18:21:02.707815+0000","last_peered":"2024-09-13T18:21:02.707815+0000","last_clean":"2024-09-13T18:21:02.707815+0000","last_became_active":"2024-09-13T17:50:34.197839+0000","last_became_peered":"2024-09-13T17:50:34.197839+0000","last_unstale":"2024-09-13T18:21:02.707815+0000","last_undegraded":"2024-09-13T18:21:02.707815+0000","last_fullsized":"2024-09-13T18:21:02.707815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:11:51.430710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'6326","reported_seq":6352,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:01.698688+0000","last_change":"2024-09-13T17:50:38.611080+0000","last_active":"2024-09-13T18:21:01.698688+0000","last_peered":"2024-09-13T18:21:01.698688+0000","last_clean":"2024-09-13T18:21:01.698688+0000","last_became_active":"2024-09-13T17:50:38.610688+0000","last_became_peered":"2024-09-13T17:50:38.610688+0000","last_unstale":"2024-09-13T18:21:01.698688+0000","last_undegraded":"2024-09-13T18:21:01.698688+0000","last_fullsized":"2024-09-13T18:21:01.698688+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6326,"log_dups_size":0,"ondisk_log_size":6326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:49:28.527617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27199,"num_write_kb":538670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'29","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317546+0000","last_change":"2024-09-13T17:50:34.594763+0000","last_active":"2024-09-13T18:21:13.317546+0000","last_peered":"2024-09-13T18:21:13.317546+0000","last_clean":"2024-09-13T18:21:13.317546+0000","last_became_active":"2024-09-13T17:50:34.594471+0000","last_became_peered":"2024-09-13T17:50:34.594471+0000","last_unstale":"2024-09-13T18:21:13.317546+0000","last_undegraded":"2024-09-13T18:21:13.317546+0000","last_fullsized":"2024-09-13T18:21:13.317546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:20:58.718271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'6","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.839570+0000","last_change":"2024-09-13T17:50:34.209432+0000","last_active":"2024-09-13T18:21:02.839570+0000","last_peered":"2024-09-13T18:21:02.839570+0000","last_clean":"2024-09-13T18:21:02.839570+0000","last_became_active":"2024-09-13T17:50:34.208817+0000","last_became_peered":"2024-09-13T17:50:34.208817+0000","last_unstale":"2024-09-13T18:21:02.839570+0000","last_undegraded":"2024-09-13T18:21:02.839570+0000","last_fullsized":"2024-09-13T18:21:02.839570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:49.587528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'5788","reported_seq":5815,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.947982+0000","last_change":"2024-09-13T17:50:38.609461+0000","last_active":"2024-09-13T18:21:02.947982+0000","last_peered":"2024-09-13T18:21:02.947982+0000","last_clean":"2024-09-13T18:21:02.947982+0000","last_became_active":"2024-09-13T17:50:38.609128+0000","last_became_peered":"2024-09-13T17:50:38.609128+0000","last_unstale":"2024-09-13T18:21:02.947982+0000","last_undegraded":"2024-09-13T18:21:02.947982+0000","last_fullsized":"2024-09-13T18:21:02.947982+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5788,"log_dups_size":0,"ondisk_log_size":5788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:02.286825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24140,"num_write_kb":538739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'24","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.305222+0000","last_change":"2024-09-13T17:50:34.592970+0000","last_active":"2024-09-13T18:21:13.305222+0000","last_peered":"2024-09-13T18:21:13.305222+0000","last_clean":"2024-09-13T18:21:13.305222+0000","last_became_active":"2024-09-13T17:50:34.592669+0000","last_became_peered":"2024-09-13T17:50:34.592669+0000","last_unstale":"2024-09-13T18:21:13.305222+0000","last_undegraded":"2024-09-13T18:21:13.305222+0000","last_fullsized":"2024-09-13T18:21:13.305222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:52.031261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'15","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.785525+0000","last_change":"2024-09-13T17:50:34.200798+0000","last_active":"2024-09-13T18:21:02.785525+0000","last_peered":"2024-09-13T18:21:02.785525+0000","last_clean":"2024-09-13T18:21:02.785525+0000","last_became_active":"2024-09-13T17:50:34.200529+0000","last_became_peered":"2024-09-13T17:50:34.200529+0000","last_unstale":"2024-09-13T18:21:02.785525+0000","last_undegraded":"2024-09-13T18:21:02.785525+0000","last_fullsized":"2024-09-13T18:21:02.785525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:26.334296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'5464","reported_seq":5490,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.598966+0000","last_change":"2024-09-13T17:50:38.606287+0000","last_active":"2024-09-13T18:21:02.598966+0000","last_peered":"2024-09-13T18:21:02.598966+0000","last_clean":"2024-09-13T18:21:02.598966+0000","last_became_active":"2024-09-13T17:50:38.605826+0000","last_became_peered":"2024-09-13T17:50:38.605826+0000","last_unstale":"2024-09-13T18:21:02.598966+0000","last_undegraded":"2024-09-13T18:21:02.598966+0000","last_fullsized":"2024-09-13T18:21:02.598966+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5464,"log_dups_size":0,"ondisk_log_size":5464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:51:18.709454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23044,"num_write_kb":472739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'18","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310869+0000","last_change":"2024-09-13T17:50:34.591579+0000","last_active":"2024-09-13T18:21:13.310869+0000","last_peered":"2024-09-13T18:21:13.310869+0000","last_clean":"2024-09-13T18:21:13.310869+0000","last_became_active":"2024-09-13T17:50:34.586165+0000","last_became_peered":"2024-09-13T17:50:34.586165+0000","last_unstale":"2024-09-13T18:21:13.310869+0000","last_undegraded":"2024-09-13T18:21:13.310869+0000","last_fullsized":"2024-09-13T18:21:13.310869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:56:19.205176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'417","reported_seq":447,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.705527+0000","last_change":"2024-09-13T17:50:34.203218+0000","last_active":"2024-09-13T18:21:02.705527+0000","last_peered":"2024-09-13T18:21:02.705527+0000","last_clean":"2024-09-13T18:21:02.705527+0000","last_became_active":"2024-09-13T17:50:34.202948+0000","last_became_peered":"2024-09-13T17:50:34.202948+0000","last_unstale":"2024-09-13T18:21:02.705527+0000","last_undegraded":"2024-09-13T18:21:02.705527+0000","last_fullsized":"2024-09-13T18:21:02.705527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:56:46.535569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":4324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'5887","reported_seq":5914,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.956090+0000","last_change":"2024-09-13T17:50:38.610597+0000","last_active":"2024-09-13T18:21:02.956090+0000","last_peered":"2024-09-13T18:21:02.956090+0000","last_clean":"2024-09-13T18:21:02.956090+0000","last_became_active":"2024-09-13T17:50:38.610055+0000","last_became_peered":"2024-09-13T17:50:38.610055+0000","last_unstale":"2024-09-13T18:21:02.956090+0000","last_undegraded":"2024-09-13T18:21:02.956090+0000","last_fullsized":"2024-09-13T18:21:02.956090+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5887,"log_dups_size":0,"ondisk_log_size":5887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:39:55.754936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24664,"num_write_kb":495799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'21","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317972+0000","last_change":"2024-09-13T17:50:34.593963+0000","last_active":"2024-09-13T18:21:13.317972+0000","last_peered":"2024-09-13T18:21:13.317972+0000","last_clean":"2024-09-13T18:21:13.317972+0000","last_became_active":"2024-09-13T17:50:34.593454+0000","last_became_peered":"2024-09-13T17:50:34.593454+0000","last_unstale":"2024-09-13T18:21:13.317972+0000","last_undegraded":"2024-09-13T18:21:13.317972+0000","last_fullsized":"2024-09-13T18:21:13.317972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:18.883521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'67","reported_seq":96,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.464353+0000","last_change":"2024-09-13T17:50:34.207936+0000","last_active":"2024-09-13T18:21:02.464353+0000","last_peered":"2024-09-13T18:21:02.464353+0000","last_clean":"2024-09-13T18:21:02.464353+0000","last_became_active":"2024-09-13T17:50:34.207605+0000","last_became_peered":"2024-09-13T17:50:34.207605+0000","last_unstale":"2024-09-13T18:21:02.464353+0000","last_undegraded":"2024-09-13T18:21:02.464353+0000","last_fullsized":"2024-09-13T18:21:02.464353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:05.494406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'5937","reported_seq":5964,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.965268+0000","last_change":"2024-09-13T17:50:38.616297+0000","last_active":"2024-09-13T18:21:02.965268+0000","last_peered":"2024-09-13T18:21:02.965268+0000","last_clean":"2024-09-13T18:21:02.965268+0000","last_became_active":"2024-09-13T17:50:38.614165+0000","last_became_peered":"2024-09-13T17:50:38.614165+0000","last_unstale":"2024-09-13T18:21:02.965268+0000","last_undegraded":"2024-09-13T18:21:02.965268+0000","last_fullsized":"2024-09-13T18:21:02.965268+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5937,"log_dups_size":0,"ondisk_log_size":5937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:26:11.357026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25111,"num_write_kb":509795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'25","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.311085+0000","last_change":"2024-09-13T17:50:34.583001+0000","last_active":"2024-09-13T18:21:13.311085+0000","last_peered":"2024-09-13T18:21:13.311085+0000","last_clean":"2024-09-13T18:21:13.311085+0000","last_became_active":"2024-09-13T17:50:34.582529+0000","last_became_peered":"2024-09-13T17:50:34.582529+0000","last_unstale":"2024-09-13T18:21:13.311085+0000","last_undegraded":"2024-09-13T18:21:13.311085+0000","last_fullsized":"2024-09-13T18:21:13.311085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:05.287181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'5","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:53.035777+0000","last_change":"2024-09-13T17:50:34.205138+0000","last_active":"2024-09-13T18:20:53.035777+0000","last_peered":"2024-09-13T18:20:53.035777+0000","last_clean":"2024-09-13T18:20:53.035777+0000","last_became_active":"2024-09-13T17:50:34.203360+0000","last_became_peered":"2024-09-13T17:50:34.203360+0000","last_unstale":"2024-09-13T18:20:53.035777+0000","last_undegraded":"2024-09-13T18:20:53.035777+0000","last_fullsized":"2024-09-13T18:20:53.035777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:35.756963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'5934","reported_seq":5961,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.811068+0000","last_change":"2024-09-13T17:50:38.611036+0000","last_active":"2024-09-13T18:21:02.811068+0000","last_peered":"2024-09-13T18:21:02.811068+0000","last_clean":"2024-09-13T18:21:02.811068+0000","last_became_active":"2024-09-13T17:50:38.610692+0000","last_became_peered":"2024-09-13T17:50:38.610692+0000","last_unstale":"2024-09-13T18:21:02.811068+0000","last_undegraded":"2024-09-13T18:21:02.811068+0000","last_fullsized":"2024-09-13T18:21:02.811068+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5934,"log_dups_size":0,"ondisk_log_size":5934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:21:10.147998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25100,"num_write_kb":497535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'23","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.311752+0000","last_change":"2024-09-13T17:51:16.365295+0000","last_active":"2024-09-13T18:21:13.311752+0000","last_peered":"2024-09-13T18:21:13.311752+0000","last_clean":"2024-09-13T18:21:13.311752+0000","last_became_active":"2024-09-13T17:51:16.364437+0000","last_became_peered":"2024-09-13T17:51:16.364437+0000","last_unstale":"2024-09-13T18:21:13.311752+0000","last_undegraded":"2024-09-13T18:21:13.311752+0000","last_fullsized":"2024-09-13T18:21:13.311752+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:23.208899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"27'10","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.645634+0000","last_change":"2024-09-13T17:50:34.204376+0000","last_active":"2024-09-13T18:21:02.645634+0000","last_peered":"2024-09-13T18:21:02.645634+0000","last_clean":"2024-09-13T18:21:02.645634+0000","last_became_active":"2024-09-13T17:50:34.204003+0000","last_became_peered":"2024-09-13T17:50:34.204003+0000","last_unstale":"2024-09-13T18:21:02.645634+0000","last_undegraded":"2024-09-13T18:21:02.645634+0000","last_fullsized":"2024-09-13T18:21:02.645634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:44:14.023990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'5363","reported_seq":5390,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.796875+0000","last_change":"2024-09-13T17:50:38.606144+0000","last_active":"2024-09-13T18:21:02.796875+0000","last_peered":"2024-09-13T18:21:02.796875+0000","last_clean":"2024-09-13T18:21:02.796875+0000","last_became_active":"2024-09-13T17:50:38.605646+0000","last_became_peered":"2024-09-13T17:50:38.605646+0000","last_unstale":"2024-09-13T18:21:02.796875+0000","last_undegraded":"2024-09-13T18:21:02.796875+0000","last_fullsized":"2024-09-13T18:21:02.796875+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5363,"log_dups_size":0,"ondisk_log_size":5363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:51:07.120075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22831,"num_write_kb":508903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'6067","reported_seq":6096,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.803568+0000","last_change":"2024-09-13T17:50:38.618855+0000","last_active":"2024-09-13T18:21:02.803568+0000","last_peered":"2024-09-13T18:21:02.803568+0000","last_clean":"2024-09-13T18:21:02.803568+0000","last_became_active":"2024-09-13T17:50:38.618449+0000","last_became_peered":"2024-09-13T17:50:38.618449+0000","last_unstale":"2024-09-13T18:21:02.803568+0000","last_undegraded":"2024-09-13T18:21:02.803568+0000","last_fullsized":"2024-09-13T18:21:02.803568+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6067,"log_dups_size":0,"ondisk_log_size":6067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:29:01.114087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25532,"num_write_kb":568825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'21","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.298787+0000","last_change":"2024-09-13T17:50:34.590079+0000","last_active":"2024-09-13T18:21:13.298787+0000","last_peered":"2024-09-13T18:21:13.298787+0000","last_clean":"2024-09-13T18:21:13.298787+0000","last_became_active":"2024-09-13T17:50:34.589396+0000","last_became_peered":"2024-09-13T17:50:34.589396+0000","last_unstale":"2024-09-13T18:21:13.298787+0000","last_undegraded":"2024-09-13T18:21:13.298787+0000","last_fullsized":"2024-09-13T18:21:13.298787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:03.308341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:17:11.375959+0000","last_change":"2024-09-13T17:50:34.206575+0000","last_active":"2024-09-13T18:17:11.375959+0000","last_peered":"2024-09-13T18:17:11.375959+0000","last_clean":"2024-09-13T18:17:11.375959+0000","last_became_active":"2024-09-13T17:50:34.206308+0000","last_became_peered":"2024-09-13T17:50:34.206308+0000","last_unstale":"2024-09-13T18:17:11.375959+0000","last_undegraded":"2024-09-13T18:17:11.375959+0000","last_fullsized":"2024-09-13T18:17:11.375959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:37:34.766538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'5881","reported_seq":5909,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.927672+0000","last_change":"2024-09-13T17:50:38.618131+0000","last_active":"2024-09-13T18:21:02.927672+0000","last_peered":"2024-09-13T18:21:02.927672+0000","last_clean":"2024-09-13T18:21:02.927672+0000","last_became_active":"2024-09-13T17:50:38.617611+0000","last_became_peered":"2024-09-13T17:50:38.617611+0000","last_unstale":"2024-09-13T18:21:02.927672+0000","last_undegraded":"2024-09-13T18:21:02.927672+0000","last_fullsized":"2024-09-13T18:21:02.927672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5881,"log_dups_size":0,"ondisk_log_size":5881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:03:30.746445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25019,"num_write_kb":530139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'27","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.307795+0000","last_change":"2024-09-13T17:50:34.582153+0000","last_active":"2024-09-13T18:21:13.307795+0000","last_peered":"2024-09-13T18:21:13.307795+0000","last_clean":"2024-09-13T18:21:13.307795+0000","last_became_active":"2024-09-13T17:50:34.581708+0000","last_became_peered":"2024-09-13T17:50:34.581708+0000","last_unstale":"2024-09-13T18:21:13.307795+0000","last_undegraded":"2024-09-13T18:21:13.307795+0000","last_fullsized":"2024-09-13T18:21:13.307795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:22.238810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'4","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:19:47.928146+0000","last_change":"2024-09-13T17:50:34.205489+0000","last_active":"2024-09-13T18:19:47.928146+0000","last_peered":"2024-09-13T18:19:47.928146+0000","last_clean":"2024-09-13T18:19:47.928146+0000","last_became_active":"2024-09-13T17:50:34.205202+0000","last_became_peered":"2024-09-13T17:50:34.205202+0000","last_unstale":"2024-09-13T18:19:47.928146+0000","last_undegraded":"2024-09-13T18:19:47.928146+0000","last_fullsized":"2024-09-13T18:19:47.928146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:54.450276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'6","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.825727+0000","last_change":"2024-09-13T17:50:34.202470+0000","last_active":"2024-09-13T18:21:02.825727+0000","last_peered":"2024-09-13T18:21:02.825727+0000","last_clean":"2024-09-13T18:21:02.825727+0000","last_became_active":"2024-09-13T17:50:34.202163+0000","last_became_peered":"2024-09-13T17:50:34.202163+0000","last_unstale":"2024-09-13T18:21:02.825727+0000","last_undegraded":"2024-09-13T18:21:02.825727+0000","last_fullsized":"2024-09-13T18:21:02.825727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:03.472932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'18","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312647+0000","last_change":"2024-09-13T17:50:34.590947+0000","last_active":"2024-09-13T18:21:13.312647+0000","last_peered":"2024-09-13T18:21:13.312647+0000","last_clean":"2024-09-13T18:21:13.312647+0000","last_became_active":"2024-09-13T17:50:34.590497+0000","last_became_peered":"2024-09-13T17:50:34.590497+0000","last_unstale":"2024-09-13T18:21:13.312647+0000","last_undegraded":"2024-09-13T18:21:13.312647+0000","last_fullsized":"2024-09-13T18:21:13.312647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:56:27.848092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'4651","reported_seq":4677,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.602408+0000","last_change":"2024-09-13T17:50:38.618518+0000","last_active":"2024-09-13T18:21:02.602408+0000","last_peered":"2024-09-13T18:21:02.602408+0000","last_clean":"2024-09-13T18:21:02.602408+0000","last_became_active":"2024-09-13T17:50:38.618108+0000","last_became_peered":"2024-09-13T17:50:38.618108+0000","last_unstale":"2024-09-13T18:21:02.602408+0000","last_undegraded":"2024-09-13T18:21:02.602408+0000","last_fullsized":"2024-09-13T18:21:02.602408+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4651,"log_dups_size":0,"ondisk_log_size":4651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:42.260237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19488,"num_write_kb":413684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'690","reported_seq":718,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.791718+0000","last_change":"2024-09-13T17:50:34.218459+0000","last_active":"2024-09-13T18:21:02.791718+0000","last_peered":"2024-09-13T18:21:02.791718+0000","last_clean":"2024-09-13T18:21:02.791718+0000","last_became_active":"2024-09-13T17:50:34.218087+0000","last_became_peered":"2024-09-13T17:50:34.218087+0000","last_unstale":"2024-09-13T18:21:02.791718+0000","last_undegraded":"2024-09-13T18:21:02.791718+0000","last_fullsized":"2024-09-13T18:21:02.791718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:56.914544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195461,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'20","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.315199+0000","last_change":"2024-09-13T17:50:34.583845+0000","last_active":"2024-09-13T18:21:13.315199+0000","last_peered":"2024-09-13T18:21:13.315199+0000","last_clean":"2024-09-13T18:21:13.315199+0000","last_became_active":"2024-09-13T17:50:34.583283+0000","last_became_peered":"2024-09-13T17:50:34.583283+0000","last_unstale":"2024-09-13T18:21:13.315199+0000","last_undegraded":"2024-09-13T18:21:13.315199+0000","last_fullsized":"2024-09-13T18:21:13.315199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:55.630285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'5883","reported_seq":5909,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.716917+0000","last_change":"2024-09-13T17:50:38.611987+0000","last_active":"2024-09-13T18:21:02.716917+0000","last_peered":"2024-09-13T18:21:02.716917+0000","last_clean":"2024-09-13T18:21:02.716917+0000","last_became_active":"2024-09-13T17:50:38.611636+0000","last_became_peered":"2024-09-13T17:50:38.611636+0000","last_unstale":"2024-09-13T18:21:02.716917+0000","last_undegraded":"2024-09-13T18:21:02.716917+0000","last_fullsized":"2024-09-13T18:21:02.716917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5883,"log_dups_size":0,"ondisk_log_size":5883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:41.111562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24816,"num_write_kb":545450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'25","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.300796+0000","last_change":"2024-09-13T17:50:34.588927+0000","last_active":"2024-09-13T18:21:13.300796+0000","last_peered":"2024-09-13T18:21:13.300796+0000","last_clean":"2024-09-13T18:21:13.300796+0000","last_became_active":"2024-09-13T17:50:34.588651+0000","last_became_peered":"2024-09-13T17:50:34.588651+0000","last_unstale":"2024-09-13T18:21:13.300796+0000","last_undegraded":"2024-09-13T18:21:13.300796+0000","last_fullsized":"2024-09-13T18:21:13.300796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:49.429670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'24","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310371+0000","last_change":"2024-09-13T17:50:34.583795+0000","last_active":"2024-09-13T18:21:13.310371+0000","last_peered":"2024-09-13T18:21:13.310371+0000","last_clean":"2024-09-13T18:21:13.310371+0000","last_became_active":"2024-09-13T17:50:34.583493+0000","last_became_peered":"2024-09-13T17:50:34.583493+0000","last_unstale":"2024-09-13T18:21:13.310371+0000","last_undegraded":"2024-09-13T18:21:13.310371+0000","last_fullsized":"2024-09-13T18:21:13.310371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:30.463932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'25","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.314410+0000","last_change":"2024-09-13T17:50:34.587624+0000","last_active":"2024-09-13T18:21:13.314410+0000","last_peered":"2024-09-13T18:21:13.314410+0000","last_clean":"2024-09-13T18:21:13.314410+0000","last_became_active":"2024-09-13T17:50:34.587119+0000","last_became_peered":"2024-09-13T17:50:34.587119+0000","last_unstale":"2024-09-13T18:21:13.314410+0000","last_undegraded":"2024-09-13T18:21:13.314410+0000","last_fullsized":"2024-09-13T18:21:13.314410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:37:16.783899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'17","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.308557+0000","last_change":"2024-09-13T17:50:34.581923+0000","last_active":"2024-09-13T18:21:13.308557+0000","last_peered":"2024-09-13T18:21:13.308557+0000","last_clean":"2024-09-13T18:21:13.308557+0000","last_became_active":"2024-09-13T17:50:34.581628+0000","last_became_peered":"2024-09-13T17:50:34.581628+0000","last_unstale":"2024-09-13T18:21:13.308557+0000","last_undegraded":"2024-09-13T18:21:13.308557+0000","last_fullsized":"2024-09-13T18:21:13.308557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:28:55.936864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'14","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312666+0000","last_change":"2024-09-13T17:50:34.595037+0000","last_active":"2024-09-13T18:21:13.312666+0000","last_peered":"2024-09-13T18:21:13.312666+0000","last_clean":"2024-09-13T18:21:13.312666+0000","last_became_active":"2024-09-13T17:50:34.594857+0000","last_became_peered":"2024-09-13T17:50:34.594857+0000","last_unstale":"2024-09-13T18:21:13.312666+0000","last_undegraded":"2024-09-13T18:21:13.312666+0000","last_fullsized":"2024-09-13T18:21:13.312666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:42:11.420413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'15","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.298148+0000","last_change":"2024-09-13T17:50:34.589651+0000","last_active":"2024-09-13T18:21:13.298148+0000","last_peered":"2024-09-13T18:21:13.298148+0000","last_clean":"2024-09-13T18:21:13.298148+0000","last_became_active":"2024-09-13T17:50:34.589354+0000","last_became_peered":"2024-09-13T17:50:34.589354+0000","last_unstale":"2024-09-13T18:21:13.298148+0000","last_undegraded":"2024-09-13T18:21:13.298148+0000","last_fullsized":"2024-09-13T18:21:13.298148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:38:31.294048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'24","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317654+0000","last_change":"2024-09-13T17:50:34.597949+0000","last_active":"2024-09-13T18:21:13.317654+0000","last_peered":"2024-09-13T18:21:13.317654+0000","last_clean":"2024-09-13T18:21:13.317654+0000","last_became_active":"2024-09-13T17:50:34.597621+0000","last_became_peered":"2024-09-13T17:50:34.597621+0000","last_unstale":"2024-09-13T18:21:13.317654+0000","last_undegraded":"2024-09-13T18:21:13.317654+0000","last_fullsized":"2024-09-13T18:21:13.317654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:23.133462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'15","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312343+0000","last_change":"2024-09-13T17:50:34.589893+0000","last_active":"2024-09-13T18:21:13.312343+0000","last_peered":"2024-09-13T18:21:13.312343+0000","last_clean":"2024-09-13T18:21:13.312343+0000","last_became_active":"2024-09-13T17:50:34.589626+0000","last_became_peered":"2024-09-13T17:50:34.589626+0000","last_unstale":"2024-09-13T18:21:13.312343+0000","last_undegraded":"2024-09-13T18:21:13.312343+0000","last_fullsized":"2024-09-13T18:21:13.312343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:41.321025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'16","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.308839+0000","last_change":"2024-09-13T17:50:34.590046+0000","last_active":"2024-09-13T18:21:13.308839+0000","last_peered":"2024-09-13T18:21:13.308839+0000","last_clean":"2024-09-13T18:21:13.308839+0000","last_became_active":"2024-09-13T17:50:34.589757+0000","last_became_peered":"2024-09-13T17:50:34.589757+0000","last_unstale":"2024-09-13T18:21:13.308839+0000","last_undegraded":"2024-09-13T18:21:13.308839+0000","last_fullsized":"2024-09-13T18:21:13.308839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:28.239550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'23","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312608+0000","last_change":"2024-09-13T17:50:34.595518+0000","last_active":"2024-09-13T18:21:13.312608+0000","last_peered":"2024-09-13T18:21:13.312608+0000","last_clean":"2024-09-13T18:21:13.312608+0000","last_became_active":"2024-09-13T17:50:34.595242+0000","last_became_peered":"2024-09-13T17:50:34.595242+0000","last_unstale":"2024-09-13T18:21:13.312608+0000","last_undegraded":"2024-09-13T18:21:13.312608+0000","last_fullsized":"2024-09-13T18:21:13.312608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:26.149384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'21","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.315503+0000","last_change":"2024-09-13T17:50:34.593183+0000","last_active":"2024-09-13T18:21:13.315503+0000","last_peered":"2024-09-13T18:21:13.315503+0000","last_clean":"2024-09-13T18:21:13.315503+0000","last_became_active":"2024-09-13T17:50:34.592893+0000","last_became_peered":"2024-09-13T17:50:34.592893+0000","last_unstale":"2024-09-13T18:21:13.315503+0000","last_undegraded":"2024-09-13T18:21:13.315503+0000","last_fullsized":"2024-09-13T18:21:13.315503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:19.459505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'29","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.311869+0000","last_change":"2024-09-13T17:50:34.594679+0000","last_active":"2024-09-13T18:21:13.311869+0000","last_peered":"2024-09-13T18:21:13.311869+0000","last_clean":"2024-09-13T18:21:13.311869+0000","last_became_active":"2024-09-13T17:50:34.593750+0000","last_became_peered":"2024-09-13T17:50:34.593750+0000","last_unstale":"2024-09-13T18:21:13.311869+0000","last_undegraded":"2024-09-13T18:21:13.311869+0000","last_fullsized":"2024-09-13T18:21:13.311869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:15:02.135577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'19","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.313398+0000","last_change":"2024-09-13T17:50:34.598736+0000","last_active":"2024-09-13T18:21:13.313398+0000","last_peered":"2024-09-13T18:21:13.313398+0000","last_clean":"2024-09-13T18:21:13.313398+0000","last_became_active":"2024-09-13T17:50:34.598433+0000","last_became_peered":"2024-09-13T17:50:34.598433+0000","last_unstale":"2024-09-13T18:21:13.313398+0000","last_undegraded":"2024-09-13T18:21:13.313398+0000","last_fullsized":"2024-09-13T18:21:13.313398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:04:08.975139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'22","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.301045+0000","last_change":"2024-09-13T17:50:34.591795+0000","last_active":"2024-09-13T18:21:13.301045+0000","last_peered":"2024-09-13T18:21:13.301045+0000","last_clean":"2024-09-13T18:21:13.301045+0000","last_became_active":"2024-09-13T17:50:34.591286+0000","last_became_peered":"2024-09-13T17:50:34.591286+0000","last_unstale":"2024-09-13T18:21:13.301045+0000","last_undegraded":"2024-09-13T18:21:13.301045+0000","last_fullsized":"2024-09-13T18:21:13.301045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:31.580247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'28","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.300800+0000","last_change":"2024-09-13T17:50:34.587774+0000","last_active":"2024-09-13T18:21:13.300800+0000","last_peered":"2024-09-13T18:21:13.300800+0000","last_clean":"2024-09-13T18:21:13.300800+0000","last_became_active":"2024-09-13T17:50:34.587506+0000","last_became_peered":"2024-09-13T17:50:34.587506+0000","last_unstale":"2024-09-13T18:21:13.300800+0000","last_undegraded":"2024-09-13T18:21:13.300800+0000","last_fullsized":"2024-09-13T18:21:13.300800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:46.275528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'22","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.318554+0000","last_change":"2024-09-13T17:50:34.596100+0000","last_active":"2024-09-13T18:21:13.318554+0000","last_peered":"2024-09-13T18:21:13.318554+0000","last_clean":"2024-09-13T18:21:13.318554+0000","last_became_active":"2024-09-13T17:50:34.595798+0000","last_became_peered":"2024-09-13T17:50:34.595798+0000","last_unstale":"2024-09-13T18:21:13.318554+0000","last_undegraded":"2024-09-13T18:21:13.318554+0000","last_fullsized":"2024-09-13T18:21:13.318554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:04:49.543845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'21","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306399+0000","last_change":"2024-09-13T17:50:34.584818+0000","last_active":"2024-09-13T18:21:13.306399+0000","last_peered":"2024-09-13T18:21:13.306399+0000","last_clean":"2024-09-13T18:21:13.306399+0000","last_became_active":"2024-09-13T17:50:34.584516+0000","last_became_peered":"2024-09-13T17:50:34.584516+0000","last_unstale":"2024-09-13T18:21:13.306399+0000","last_undegraded":"2024-09-13T18:21:13.306399+0000","last_fullsized":"2024-09-13T18:21:13.306399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:52.402024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'23","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.292739+0000","last_change":"2024-09-13T17:50:34.598886+0000","last_active":"2024-09-13T18:21:13.292739+0000","last_peered":"2024-09-13T18:21:13.292739+0000","last_clean":"2024-09-13T18:21:13.292739+0000","last_became_active":"2024-09-13T17:50:34.598648+0000","last_became_peered":"2024-09-13T17:50:34.598648+0000","last_unstale":"2024-09-13T18:21:13.292739+0000","last_undegraded":"2024-09-13T18:21:13.292739+0000","last_fullsized":"2024-09-13T18:21:13.292739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:00:43.063223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'24","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.302458+0000","last_change":"2024-09-13T17:50:34.582649+0000","last_active":"2024-09-13T18:21:13.302458+0000","last_peered":"2024-09-13T18:21:13.302458+0000","last_clean":"2024-09-13T18:21:13.302458+0000","last_became_active":"2024-09-13T17:50:34.581699+0000","last_became_peered":"2024-09-13T17:50:34.581699+0000","last_unstale":"2024-09-13T18:21:13.302458+0000","last_undegraded":"2024-09-13T18:21:13.302458+0000","last_fullsized":"2024-09-13T18:21:13.302458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:18.959303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'23","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306620+0000","last_change":"2024-09-13T17:50:34.587484+0000","last_active":"2024-09-13T18:21:13.306620+0000","last_peered":"2024-09-13T18:21:13.306620+0000","last_clean":"2024-09-13T18:21:13.306620+0000","last_became_active":"2024-09-13T17:50:34.587217+0000","last_became_peered":"2024-09-13T17:50:34.587217+0000","last_unstale":"2024-09-13T18:21:13.306620+0000","last_undegraded":"2024-09-13T18:21:13.306620+0000","last_fullsized":"2024-09-13T18:21:13.306620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:24:19.172607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'18","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.315670+0000","last_change":"2024-09-13T17:50:34.592871+0000","last_active":"2024-09-13T18:21:13.315670+0000","last_peered":"2024-09-13T18:21:13.315670+0000","last_clean":"2024-09-13T18:21:13.315670+0000","last_became_active":"2024-09-13T17:50:34.588513+0000","last_became_peered":"2024-09-13T17:50:34.588513+0000","last_unstale":"2024-09-13T18:21:13.315670+0000","last_undegraded":"2024-09-13T18:21:13.315670+0000","last_fullsized":"2024-09-13T18:21:13.315670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:33.131420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'19","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.319025+0000","last_change":"2024-09-13T17:50:34.585713+0000","last_active":"2024-09-13T18:21:13.319025+0000","last_peered":"2024-09-13T18:21:13.319025+0000","last_clean":"2024-09-13T18:21:13.319025+0000","last_became_active":"2024-09-13T17:50:34.585438+0000","last_became_peered":"2024-09-13T17:50:34.585438+0000","last_unstale":"2024-09-13T18:21:13.319025+0000","last_undegraded":"2024-09-13T18:21:13.319025+0000","last_fullsized":"2024-09-13T18:21:13.319025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:56:49.905393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'30","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312596+0000","last_change":"2024-09-13T17:50:34.589628+0000","last_active":"2024-09-13T18:21:13.312596+0000","last_peered":"2024-09-13T18:21:13.312596+0000","last_clean":"2024-09-13T18:21:13.312596+0000","last_became_active":"2024-09-13T17:50:34.587665+0000","last_became_peered":"2024-09-13T17:50:34.587665+0000","last_unstale":"2024-09-13T18:21:13.312596+0000","last_undegraded":"2024-09-13T18:21:13.312596+0000","last_fullsized":"2024-09-13T18:21:13.312596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:15.518264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'27","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.321326+0000","last_change":"2024-09-13T17:50:34.591776+0000","last_active":"2024-09-13T18:21:13.321326+0000","last_peered":"2024-09-13T18:21:13.321326+0000","last_clean":"2024-09-13T18:21:13.321326+0000","last_became_active":"2024-09-13T17:50:34.591496+0000","last_became_peered":"2024-09-13T17:50:34.591496+0000","last_unstale":"2024-09-13T18:21:13.321326+0000","last_undegraded":"2024-09-13T18:21:13.321326+0000","last_fullsized":"2024-09-13T18:21:13.321326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:48.651999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'15","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310654+0000","last_change":"2024-09-13T17:51:16.282372+0000","last_active":"2024-09-13T18:21:13.310654+0000","last_peered":"2024-09-13T18:21:13.310654+0000","last_clean":"2024-09-13T18:21:13.310654+0000","last_became_active":"2024-09-13T17:51:16.282065+0000","last_became_peered":"2024-09-13T17:51:16.282065+0000","last_unstale":"2024-09-13T18:21:13.310654+0000","last_undegraded":"2024-09-13T18:21:13.310654+0000","last_fullsized":"2024-09-13T18:21:13.310654+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:49.291224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"27'14","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310971+0000","last_change":"2024-09-13T17:50:34.585531+0000","last_active":"2024-09-13T18:21:13.310971+0000","last_peered":"2024-09-13T18:21:13.310971+0000","last_clean":"2024-09-13T18:21:13.310971+0000","last_became_active":"2024-09-13T17:50:34.585226+0000","last_became_peered":"2024-09-13T17:50:34.585226+0000","last_unstale":"2024-09-13T18:21:13.310971+0000","last_undegraded":"2024-09-13T18:21:13.310971+0000","last_fullsized":"2024-09-13T18:21:13.310971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:17.105028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'17","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.314101+0000","last_change":"2024-09-13T17:50:34.586750+0000","last_active":"2024-09-13T18:21:13.314101+0000","last_peered":"2024-09-13T18:21:13.314101+0000","last_clean":"2024-09-13T18:21:13.314101+0000","last_became_active":"2024-09-13T17:50:34.584564+0000","last_became_peered":"2024-09-13T17:50:34.584564+0000","last_unstale":"2024-09-13T18:21:13.314101+0000","last_undegraded":"2024-09-13T18:21:13.314101+0000","last_fullsized":"2024-09-13T18:21:13.314101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:54.033257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'18","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.307217+0000","last_change":"2024-09-13T17:50:34.585938+0000","last_active":"2024-09-13T18:21:13.307217+0000","last_peered":"2024-09-13T18:21:13.307217+0000","last_clean":"2024-09-13T18:21:13.307217+0000","last_became_active":"2024-09-13T17:50:34.585584+0000","last_became_peered":"2024-09-13T17:50:34.585584+0000","last_unstale":"2024-09-13T18:21:13.307217+0000","last_undegraded":"2024-09-13T18:21:13.307217+0000","last_fullsized":"2024-09-13T18:21:13.307217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:32.623468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'15","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310571+0000","last_change":"2024-09-13T17:50:34.588762+0000","last_active":"2024-09-13T18:21:13.310571+0000","last_peered":"2024-09-13T18:21:13.310571+0000","last_clean":"2024-09-13T18:21:13.310571+0000","last_became_active":"2024-09-13T17:50:34.588489+0000","last_became_peered":"2024-09-13T17:50:34.588489+0000","last_unstale":"2024-09-13T18:21:13.310571+0000","last_undegraded":"2024-09-13T18:21:13.310571+0000","last_fullsized":"2024-09-13T18:21:13.310571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:38.266297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'18","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.302008+0000","last_change":"2024-09-13T17:50:34.583889+0000","last_active":"2024-09-13T18:21:13.302008+0000","last_peered":"2024-09-13T18:21:13.302008+0000","last_clean":"2024-09-13T18:21:13.302008+0000","last_became_active":"2024-09-13T17:50:34.583551+0000","last_became_peered":"2024-09-13T17:50:34.583551+0000","last_unstale":"2024-09-13T18:21:13.302008+0000","last_undegraded":"2024-09-13T18:21:13.302008+0000","last_fullsized":"2024-09-13T18:21:13.302008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:25:19.888626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'10","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.284491+0000","last_change":"2024-09-13T17:50:34.585617+0000","last_active":"2024-09-13T18:21:13.284491+0000","last_peered":"2024-09-13T18:21:13.284491+0000","last_clean":"2024-09-13T18:21:13.284491+0000","last_became_active":"2024-09-13T17:50:34.585340+0000","last_became_peered":"2024-09-13T17:50:34.585340+0000","last_unstale":"2024-09-13T18:21:13.284491+0000","last_undegraded":"2024-09-13T18:21:13.284491+0000","last_fullsized":"2024-09-13T18:21:13.284491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:18.331045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'17","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317071+0000","last_change":"2024-09-13T17:50:34.592365+0000","last_active":"2024-09-13T18:21:13.317071+0000","last_peered":"2024-09-13T18:21:13.317071+0000","last_clean":"2024-09-13T18:21:13.317071+0000","last_became_active":"2024-09-13T17:50:34.592046+0000","last_became_peered":"2024-09-13T17:50:34.592046+0000","last_unstale":"2024-09-13T18:21:13.317071+0000","last_undegraded":"2024-09-13T18:21:13.317071+0000","last_fullsized":"2024-09-13T18:21:13.317071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:54.283683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":751482,"num_write_kb":16106434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178546,"ondisk_log_size":178546,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1271,"ondisk_log_size":1271,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":15941984,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":121,"num_read_kb":179,"num_write":3371,"num_write_kb":16874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32030720,"data_stored":31883968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3124,"ondisk_log_size":3124,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509813,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1768076,"kb_used_data":8580,"kb_used_omap":1,"kb_used_meta":1759486,"kb_avail":92603764,"statfs":{"total":96636764160,"available":94826254336,"internally_reserved":0,"allocated":8785920,"data_stored":8600393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":1801714120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":2.246,"5min":5.9219999999999997,"15min":2.669},"min":{"1min":0.19,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":8.5329999999999995,"5min":41.494999999999997,"15min":50.511000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":2.6070000000000002,"5min":3.4350000000000001,"15min":1.6080000000000001},"min":{"1min":0.23100000000000001,"5min":0.193,"15min":0.193},"max":{"1min":18.760000000000002,"5min":18.760000000000002,"15min":18.760000000000002},"last":0.59299999999999997}]},{"osd":1,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":3.4969999999999999,"15min":1.8009999999999999},"min":{"1min":0.34000000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":6.25,"5min":19.626000000000001,"15min":19.626000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":3.4430000000000001,"5min":4.101,"15min":2.0379999999999998},"min":{"1min":0.29599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":22.126999999999999,"5min":22.126999999999999,"15min":22.126999999999999},"last":0.70599999999999996}]},{"osd":2,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.907,"15min":1.296},"min":{"1min":0.29399999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":7.4610000000000003,"5min":22.010000000000002,"15min":22.010000000000002},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.49,"5min":2.1800000000000002,"15min":1.4830000000000001},"min":{"1min":0.38400000000000001,"5min":0.309,"15min":0.309},"max":{"1min":7.4850000000000003,"5min":24.170999999999999,"15min":24.170999999999999},"last":1.0620000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":2.0409999999999999,"15min":1.466},"min":{"1min":0.33200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":5.593,"5min":24.359000000000002,"15min":24.359000000000002},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.8109999999999999,"5min":2.3410000000000002,"15min":1.4570000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":10.132999999999999,"5min":15.539999999999999,"15min":15.539999999999999},"last":1.028}]},{"osd":5,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":2.5840000000000001,"15min":1.6160000000000001},"min":{"1min":0.35699999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":8.5739999999999998,"5min":49.838000000000001,"15min":49.838000000000001},"last":1.008},{"interface":"front","average":{"1min":0.96999999999999997,"5min":2.5369999999999999,"15min":1.605},"min":{"1min":0.43099999999999999,"5min":0.375,"15min":0.375},"max":{"1min":4.3810000000000002,"5min":41.591999999999999,"15min":41.591999999999999},"last":1.1240000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":2.23,"5min":2.5579999999999998,"15min":1.7270000000000001},"min":{"1min":0.46100000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":8.6029999999999998,"5min":24.457000000000001,"15min":24.457000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.677,"5min":2.2919999999999998,"15min":1.6200000000000001},"min":{"1min":0.42399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":7.4290000000000003,"5min":24.387,"15min":24.387},"last":1.2050000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:20:23 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.869,"15min":1.3740000000000001},"min":{"1min":0.505,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":4.2750000000000004,"5min":14.138999999999999,"15min":14.138999999999999},"last":1.244},{"interface":"front","average":{"1min":1.155,"5min":1.97,"15min":1.4219999999999999},"min":{"1min":0.44900000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":4.4320000000000004,"5min":14.013,"15min":14.013},"last":1.2769999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509815,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1055244,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":1054846,"kb_avail":93316596,"statfs":{"total":96636764160,"available":95556194304,"internally_reserved":0,"allocated":397312,"data_stored":210684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1080162758},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":2.0819999999999999,"5min":2.0739999999999998,"15min":1.0980000000000001},"min":{"1min":0.17499999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":9.9280000000000008,"5min":17.971,"15min":17.971},"last":0.46000000000000002},{"interface":"front","average":{"1min":2.1469999999999998,"5min":2.1240000000000001,"15min":1.177},"min":{"1min":0.247,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":9.9700000000000006,"5min":18.001999999999999,"15min":18.001999999999999},"last":0.56399999999999995}]},{"osd":1,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":2.694,"15min":1.677},"min":{"1min":0.26100000000000001,"5min":0.217,"15min":0.217},"max":{"1min":3.871,"5min":24.599,"15min":56.939999999999998},"last":0.39800000000000002},{"interface":"front","average":{"1min":1.4710000000000001,"5min":2.9350000000000001,"15min":1.7649999999999999},"min":{"1min":0.23200000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":9.0169999999999995,"5min":24.75,"15min":57.006999999999998},"last":0.60999999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":3.5219999999999998,"5min":3.0790000000000002,"15min":1.5980000000000001},"min":{"1min":0.30499999999999999,"5min":0.218,"15min":0.218},"max":{"1min":16.111999999999998,"5min":16.707000000000001,"15min":16.707000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":3.0169999999999999,"5min":2.6349999999999998,"15min":1.5569999999999999},"min":{"1min":0.309,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":12.449999999999999,"5min":16.640999999999998,"15min":16.640999999999998},"last":0.497}]},{"osd":4,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.52,"15min":1.238},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":8.2260000000000009,"5min":8.2829999999999995,"15min":14.177},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.284,"15min":1.159},"min":{"1min":0.33900000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":8.1980000000000004,"5min":8.1980000000000004,"15min":14.396000000000001},"last":0.81599999999999995}]},{"osd":5,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.77600000000000002,"15min":1.0009999999999999},"min":{"1min":0.379,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.2250000000000001,"5min":4.1020000000000003,"15min":14.153},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.181,"5min":1.224,"15min":1.137},"min":{"1min":0.32800000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":7.476,"5min":7.476,"15min":14.375999999999999},"last":0.74099999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:20:30 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.1519999999999999,"15min":1.1020000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":6.8730000000000002,"5min":8.3010000000000002,"15min":8.3010000000000002},"last":0.64900000000000002},{"interface":"front","average":{"1min":1.125,"5min":1.196,"15min":1.2150000000000001},"min":{"1min":0.46700000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":6.9500000000000002,"5min":8.3320000000000007,"15min":14.074},"last":0.85699999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:20:21 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.163,"15min":1.0780000000000001},"min":{"1min":0.40400000000000003,"5min":0.37,"15min":0.37},"max":{"1min":4.1079999999999997,"5min":8.0410000000000004,"15min":8.0410000000000004},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.238,"15min":1.1930000000000001},"min":{"1min":0.37,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.9900000000000002,"5min":6.5300000000000002,"15min":14.307},"last":0.96499999999999997}]}]},{"osd":1,"up_from":15,"seq":64424509814,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1738228,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":1737918,"kb_avail":92633612,"statfs":{"total":96636764160,"available":94856818688,"internally_reserved":0,"allocated":307200,"data_stored":116868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1779628486},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":2.5430000000000001,"15min":1.907},"min":{"1min":0.42299999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":4.25,"5min":21.949999999999999,"15min":21.949999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":2.3759999999999999,"15min":1.593},"min":{"1min":0.27600000000000002,"5min":0.193,"15min":0.193},"max":{"1min":2.7349999999999999,"5min":23.338999999999999,"15min":23.338999999999999},"last":0.75700000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.8540000000000001,"15min":1.5149999999999999},"min":{"1min":0.38,"5min":0.22,"15min":0.22},"max":{"1min":5.5129999999999999,"5min":11.423999999999999,"15min":14.869999999999999},"last":1.024},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.962,"15min":1.4319999999999999},"min":{"1min":0.47399999999999998,"5min":0.223,"15min":0.223},"max":{"1min":5.4809999999999999,"5min":12.439,"15min":15.058},"last":0.97099999999999997}]},{"osd":3,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":4.3810000000000002,"15min":3.1800000000000002},"min":{"1min":0.44400000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":4.5839999999999996,"5min":27.838000000000001,"15min":41.369999999999997},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.0860000000000001,"5min":4.327,"15min":2.911},"min":{"1min":0.251,"5min":0.216,"15min":0.216},"max":{"1min":4.9299999999999997,"5min":30.077000000000002,"15min":40.325000000000003},"last":0.81999999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":2.4660000000000002,"15min":2.1480000000000001},"min":{"1min":0.51500000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":7.2089999999999996,"5min":29.702000000000002,"15min":103.44499999999999},"last":1.073},{"interface":"front","average":{"1min":1.2010000000000001,"5min":2.4689999999999999,"15min":1.905},"min":{"1min":0.52700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.75,"5min":29.425999999999998,"15min":40.953000000000003},"last":1.083}]},{"osd":5,"last update":"Fri Sep 13 18:20:20 2024","interfaces":[{"interface":"back","average":{"1min":1.9750000000000001,"5min":2.661,"15min":1.726},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":9.8970000000000002,"5min":18.954000000000001,"15min":18.954000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.431,"5min":2.0840000000000001,"15min":1.516},"min":{"1min":0.31,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":7.383,"5min":15.413,"15min":15.413},"last":0.92600000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.994,"5min":2.044,"15min":1.633},"min":{"1min":0.748,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":6.4889999999999999,"5min":13.67,"15min":15.462999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":2.0569999999999999,"5min":2.1419999999999999,"15min":1.7270000000000001},"min":{"1min":0.85699999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":6.5220000000000002,"5min":13.696,"15min":15.49},"last":1.107}]},{"osd":7,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.667,"15min":1.415},"min":{"1min":0.64500000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":7.9669999999999996,"5min":12.476000000000001,"15min":15.093999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.7569999999999999,"5min":1.7330000000000001,"15min":1.5},"min":{"1min":0.80200000000000005,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":7.9969999999999999,"5min":12.539999999999999,"15min":15.119},"last":1.3169999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509816,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1247072,"kb_used_data":4056,"kb_used_omap":1,"kb_used_meta":1243006,"kb_avail":93124768,"statfs":{"total":96636764160,"available":95359762432,"internally_reserved":0,"allocated":4153344,"data_stored":3967313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1272838598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":2.8570000000000002,"5min":3.2130000000000001,"15min":1.7050000000000001},"min":{"1min":0.182,"5min":0.182,"15min":0.182},"max":{"1min":17.693000000000001,"5min":23.004000000000001,"15min":24.202000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":2.9489999999999998,"5min":3.2690000000000001,"15min":1.8380000000000001},"min":{"1min":0.32100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":17.736000000000001,"5min":23.039999999999999,"15min":24.260999999999999},"last":0.753}]},{"osd":2,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":1.897,"15min":1.1060000000000001},"min":{"1min":0.20399999999999999,"5min":0.186,"15min":0.186},"max":{"1min":6.4320000000000004,"5min":17.859000000000002,"15min":17.859000000000002},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.77,"5min":2.0920000000000001,"15min":1.3180000000000001},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":8.6199999999999992,"5min":19.678999999999998,"15min":19.678999999999998},"last":0.89500000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":2.2330000000000001,"5min":4.0970000000000004,"15min":1.9630000000000001},"min":{"1min":0.23699999999999999,"5min":0.219,"15min":0.219},"max":{"1min":11.378,"5min":18.056999999999999,"15min":18.056999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":3.6720000000000002,"5min":3.27,"15min":1.8129999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":13.736000000000001,"5min":15.856999999999999,"15min":15.856999999999999},"last":1.23}]},{"osd":4,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.661,"15min":1.258},"min":{"1min":0.34699999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":8.2829999999999995,"5min":8.7010000000000005,"15min":8.7010000000000005},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.1719999999999999,"5min":2.657,"15min":1.548},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":6.3090000000000002,"5min":16.291,"15min":16.291},"last":1.137}]},{"osd":5,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":2.0960000000000001,"5min":1.974,"15min":1.3260000000000001},"min":{"1min":0.312,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":8.4800000000000004,"5min":15.18,"15min":15.18},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.7689999999999999,"5min":1.9850000000000001,"15min":1.4119999999999999},"min":{"1min":0.34699999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":8.5559999999999992,"5min":17.852,"15min":17.852},"last":1.321}]},{"osd":6,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":3.1669999999999998,"5min":2.3860000000000001,"15min":1.5600000000000001},"min":{"1min":0.39200000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":15.467000000000001,"5min":15.467000000000001,"15min":15.467000000000001},"last":1.276},{"interface":"front","average":{"1min":1.7290000000000001,"5min":1.823,"15min":1.4139999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":10.359,"5min":17.908000000000001,"15min":17.908000000000001},"last":1.0760000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:20:28 2024","interfaces":[{"interface":"back","average":{"1min":1.909,"5min":1.887,"15min":1.381},"min":{"1min":0.40100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":8.2550000000000008,"5min":18.986999999999998,"15min":18.986999999999998},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.952,"5min":1.9710000000000001,"15min":1.47},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.373},"max":{"1min":8.3279999999999994,"5min":19.021000000000001,"15min":19.021000000000001},"last":1.107}]}]},{"osd":7,"up_from":14,"seq":60129542518,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1207524,"kb_used_data":12252,"kb_used_omap":1,"kb_used_meta":1195262,"kb_avail":93164316,"statfs":{"total":96636764160,"available":95400259584,"internally_reserved":0,"allocated":12546048,"data_stored":12357112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":1223948747},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":1.9470000000000001,"15min":1.2410000000000001},"min":{"1min":0.26500000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":2.875,"5min":35.195,"15min":35.195},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.133,"5min":2.1320000000000001,"15min":1.411},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":5.0919999999999996,"5min":35.253999999999998,"15min":35.253999999999998},"last":0.66000000000000003}]},{"osd":1,"last update":"Fri Sep 13 18:20:27 2024","interfaces":[{"interface":"back","average":{"1min":2.4940000000000002,"5min":2.6930000000000001,"15min":1.4730000000000001},"min":{"1min":0.27100000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":13.058999999999999,"5min":19.585000000000001,"15min":19.585000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":2.4060000000000001,"5min":2.6680000000000001,"15min":1.635},"min":{"1min":0.33400000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":17.398,"5min":19.617000000000001,"15min":19.617000000000001},"last":0.93200000000000005}]},{"osd":2,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.597,"15min":1.1699999999999999},"min":{"1min":0.51200000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.5449999999999999,"5min":15.789999999999999,"15min":15.789999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":1.274,"5min":1.5680000000000001,"15min":1.1799999999999999},"min":{"1min":0.38700000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.5259999999999998,"5min":12.965,"15min":12.965},"last":1.107}]},{"osd":3,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":2.8980000000000001,"15min":1.9119999999999999},"min":{"1min":0.41099999999999998,"5min":0.316,"15min":0.316},"max":{"1min":7.8449999999999998,"5min":40.948999999999998,"15min":40.948999999999998},"last":0.85799999999999998},{"interface":"front","average":{"1min":2.0779999999999998,"5min":3.0139999999999998,"15min":1.768},"min":{"1min":0.39300000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":7.8810000000000002,"5min":25.234000000000002,"15min":25.234000000000002},"last":1.022}]},{"osd":4,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":1.165,"15min":0.98999999999999999},"min":{"1min":0.33700000000000002,"5min":0.28799999999999998,"15min":0.252},"max":{"1min":3.0609999999999999,"5min":8.8670000000000009,"15min":8.8670000000000009},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.3300000000000001,"15min":1.1859999999999999},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":6.069,"5min":10.279999999999999,"15min":10.279999999999999},"last":0.88300000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":1.0620000000000001,"15min":0.98199999999999998},"min":{"1min":0.374,"5min":0.311,"15min":0.27000000000000002},"max":{"1min":1.5289999999999999,"5min":8.1699999999999999,"15min":8.1699999999999999},"last":1.081},{"interface":"front","average":{"1min":0.878,"5min":0.96299999999999997,"15min":1.02},"min":{"1min":0.39800000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.9490000000000001,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":0.83599999999999997}]},{"osd":6,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.4790000000000001,"15min":1.236},"min":{"1min":0.41599999999999998,"5min":0.34599999999999997,"15min":0.309},"max":{"1min":3.597,"5min":8.2219999999999995,"15min":8.2219999999999995},"last":1.171},{"interface":"front","average":{"1min":1.419,"5min":1.548,"15min":1.2789999999999999},"min":{"1min":0.48499999999999999,"5min":0.434,"15min":0.34499999999999997},"max":{"1min":3.7210000000000001,"5min":8.2469999999999999,"15min":8.2469999999999999},"last":1.2190000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542518,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1419436,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":1415038,"kb_avail":92952404,"statfs":{"total":96636764160,"available":95183261696,"internally_reserved":0,"allocated":4493312,"data_stored":4311530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1448999367},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":2.0329999999999999,"15min":1.21},"min":{"1min":0.23699999999999999,"5min":0.214,"15min":0.214},"max":{"1min":3.9470000000000001,"5min":12.808999999999999,"15min":12.808999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.091,"5min":1.7030000000000001,"15min":1.161},"min":{"1min":0.31,"5min":0.218,"15min":0.218},"max":{"1min":5.7670000000000003,"5min":12.308999999999999,"15min":12.308999999999999},"last":0.80500000000000005}]},{"osd":1,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":2.0470000000000002,"5min":2.27,"15min":1.371},"min":{"1min":0.34799999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":5.8380000000000001,"5min":14.199999999999999,"15min":14.199999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.621,"5min":2.339,"15min":1.413},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.7990000000000004,"5min":14.260999999999999,"15min":14.260999999999999},"last":1.109}]},{"osd":2,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":1.6839999999999999,"5min":1.821,"15min":1.2110000000000001},"min":{"1min":0.29999999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":6.0359999999999996,"5min":10.792999999999999,"15min":10.792999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.742,"5min":1.8759999999999999,"15min":1.371},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":6.0750000000000002,"5min":10.826000000000001,"15min":10.826000000000001},"last":1.252}]},{"osd":3,"last update":"Fri Sep 13 18:20:16 2024","interfaces":[{"interface":"back","average":{"1min":3.379,"5min":5.0670000000000002,"15min":2.4660000000000002},"min":{"1min":0.35799999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":10.324999999999999,"5min":84.867999999999995,"15min":84.867999999999995},"last":0.93600000000000005},{"interface":"front","average":{"1min":2.5449999999999999,"5min":3.5779999999999998,"15min":1.8979999999999999},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":10.231,"5min":84.343999999999994,"15min":84.343999999999994},"last":1.01}]},{"osd":4,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":2.0739999999999998,"5min":1.6990000000000001,"15min":1.2070000000000001},"min":{"1min":0.39200000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":9.5649999999999995,"5min":11.044,"15min":11.044},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.294,"15min":1.113},"min":{"1min":0.38400000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":9.4719999999999995,"5min":9.4719999999999995,"15min":9.4719999999999995},"last":0.97599999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2689999999999999,"15min":1.1559999999999999},"min":{"1min":0.376,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.0939999999999999,"5min":8.7880000000000003,"15min":8.7880000000000003},"last":1.151},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.149,"15min":1.1000000000000001},"min":{"1min":0.41499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.1709999999999998,"5min":7.569,"15min":7.569},"last":1.222}]},{"osd":7,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.1739999999999999,"15min":1.181},"min":{"1min":0.434,"5min":0.32900000000000001,"15min":0.29999999999999999},"max":{"1min":2.8029999999999999,"5min":7.1820000000000004,"15min":7.1820000000000004},"last":1.335},{"interface":"front","average":{"1min":1.133,"5min":1.294,"15min":1.256},"min":{"1min":0.53700000000000003,"5min":0.34499999999999997,"15min":0.33500000000000002},"max":{"1min":2.871,"5min":7.2039999999999997,"15min":7.2039999999999997},"last":1.379}]}]},{"osd":5,"up_from":14,"seq":60129542520,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":978544,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":978238,"kb_avail":93393296,"statfs":{"total":96636764160,"available":95634735104,"internally_reserved":0,"allocated":303104,"data_stored":117302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1001716166},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.889,"15min":1.161},"min":{"1min":0.27500000000000002,"5min":0.218,"15min":0.218},"max":{"1min":7.3650000000000002,"5min":10.148999999999999,"15min":10.148999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.389,"5min":1.9550000000000001,"15min":1.3220000000000001},"min":{"1min":0.34200000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":7.4029999999999996,"5min":10.185,"15min":10.185},"last":0.74299999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:20:27 2024","interfaces":[{"interface":"back","average":{"1min":1.879,"5min":1.897,"15min":1.1699999999999999},"min":{"1min":0.29799999999999999,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":10.08,"5min":10.436999999999999,"15min":10.436999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":2.0099999999999998,"5min":2.3460000000000001,"15min":1.385},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":18.584,"5min":19.388000000000002,"15min":19.388000000000002},"last":0.81999999999999995}]},{"osd":2,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.875,"5min":1.5329999999999999,"15min":1.0980000000000001},"min":{"1min":0.33900000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":10.183,"5min":12.667,"15min":12.667},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.032,"5min":1.6439999999999999,"15min":1.198},"min":{"1min":0.377,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":5.7469999999999999,"5min":34.905999999999999,"15min":34.905999999999999},"last":0.996}]},{"osd":3,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":3.2999999999999998,"5min":4.9039999999999999,"15min":2.286},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":19.831,"5min":28.295000000000002,"15min":28.295000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":3.9260000000000002,"5min":3.1459999999999999,"15min":1.7210000000000001},"min":{"1min":0.39700000000000002,"5min":0.318,"15min":0.318},"max":{"1min":19.731000000000002,"5min":19.731000000000002,"15min":19.731000000000002},"last":1.2589999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.744,"5min":1.0760000000000001,"15min":0.96199999999999997},"min":{"1min":0.308,"5min":0.255,"15min":0.255},"max":{"1min":10.023999999999999,"5min":10.023999999999999,"15min":10.023999999999999},"last":1.03},{"interface":"front","average":{"1min":2.3650000000000002,"5min":1.571,"15min":1.22},"min":{"1min":0.39400000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":10.210000000000001,"5min":10.210000000000001,"15min":10.210000000000001},"last":1.103}]},{"osd":6,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.9530000000000001,"5min":1.373,"15min":1.131},"min":{"1min":0.437,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":6.6849999999999996,"5min":6.6849999999999996,"15min":6.6849999999999996},"last":1.3420000000000001},{"interface":"front","average":{"1min":2.012,"5min":1.3200000000000001,"15min":1.2},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":10.151,"5min":10.151,"15min":10.151},"last":1.1839999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:20:22 2024","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":1.2330000000000001,"15min":1.143},"min":{"1min":0.42599999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":6.2910000000000004,"5min":7.9329999999999998,"15min":7.9329999999999998},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.71,"5min":1.107,"15min":1.1519999999999999},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":5.976,"5min":5.976,"15min":5.976},"last":1.1519999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542518,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1301036,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":1296638,"kb_avail":93070804,"statfs":{"total":96636764160,"available":95304503296,"internally_reserved":0,"allocated":4493312,"data_stored":4310894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1327757766},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.5980000000000001,"15min":1.3160000000000001},"min":{"1min":0.25800000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":5.7759999999999998,"5min":14.318,"15min":28.439},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.022,"5min":1.772,"15min":1.4510000000000001},"min":{"1min":0.30499999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":3.0760000000000001,"5min":16.943000000000001,"15min":28.422000000000001},"last":0.70799999999999996}]},{"osd":1,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":1.738,"5min":4.3470000000000004,"15min":2.6389999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":21.359000000000002,"5min":30.277999999999999,"15min":38.895000000000003},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.75,"5min":3.6030000000000002,"15min":2.0270000000000001},"min":{"1min":0.28199999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":21.388999999999999,"5min":32.670000000000002,"15min":32.670000000000002},"last":0.81200000000000006}]},{"osd":2,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.7090000000000001,"15min":1.1870000000000001},"min":{"1min":0.32200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.7229999999999999,"5min":25.167999999999999,"15min":25.167999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.345,"5min":2.0099999999999998,"15min":1.3480000000000001},"min":{"1min":0.41899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":5.75,"5min":25.218,"15min":25.218},"last":1.002}]},{"osd":3,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":2.3799999999999999,"5min":2.8439999999999999,"15min":1.827},"min":{"1min":0.36299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":16.344999999999999,"5min":43.945999999999998,"15min":43.945999999999998},"last":0.89700000000000002},{"interface":"front","average":{"1min":2.4100000000000001,"5min":3.3250000000000002,"15min":2.1429999999999998},"min":{"1min":0.48899999999999999,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":15.901,"5min":29.640000000000001,"15min":29.640000000000001},"last":1.226}]},{"osd":5,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":1.218,"15min":1.0569999999999999},"min":{"1min":0.34799999999999998,"5min":0.255,"15min":0.255},"max":{"1min":1.266,"5min":12.282999999999999,"15min":12.282999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.425,"15min":1.1870000000000001},"min":{"1min":0.53100000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.7000000000000002,"5min":16.995999999999999,"15min":16.995999999999999},"last":1.145}]},{"osd":6,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.9530000000000001,"15min":1.401},"min":{"1min":0.441,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":9.3650000000000002,"5min":23.010999999999999,"15min":23.010999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.302,"5min":1.3720000000000001,"15min":1.194},"min":{"1min":0.42199999999999999,"5min":0.33800000000000002,"15min":0.33600000000000002},"max":{"1min":5.6369999999999996,"5min":7.9489999999999998,"15min":7.9489999999999998},"last":1.198}]},{"osd":7,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":1.5569999999999999,"5min":1.639,"15min":1.2949999999999999},"min":{"1min":0.45000000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":11.298,"5min":20.417000000000002,"15min":20.417000000000002},"last":1.26},{"interface":"front","average":{"1min":1.02,"5min":1.456,"15min":1.272},"min":{"1min":0.36399999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.9410000000000001,"5min":12.308999999999999,"15min":12.308999999999999},"last":1.2869999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3276800,"data_stored":3261021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":94760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8503296,"data_stored":8484469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4194970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4195606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":11669504,"data_stored":11650820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:21:26.365 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.365 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.366 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.366 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.366 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.366 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.366 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.366 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.366 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.366 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.367 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.367 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.367 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.367 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.367 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.367 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.367 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.367 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.367 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.367 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.368 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.368 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.368 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.368 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.368 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.368 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.368 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.368 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.368 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.369 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.369 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.369 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.369 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.369 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.369 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.369 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.369 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.369 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.370 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.370 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.370 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.370 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.370 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.370 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.370 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.370 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.370 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.371 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T17:50:21.057812+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.371 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.371 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.371 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.371 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.371 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.371 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.371 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.371 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.371 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.372 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.372 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.372 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.372 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.372 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.372 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.373 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.373 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.373 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.373 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.373 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.373 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.373 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.373 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.374 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.374 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.374 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.374 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.374 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.374 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.374 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.375 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.375 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.375 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.375 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.375 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.375 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.375 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.375 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.375 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.376 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.376 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.376 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.376 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.376 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.376 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.376 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.376 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.376 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.377 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.377 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.377 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.377 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.377 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.377 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.377 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.377 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.377 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.378 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.378 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.378 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.378 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.378 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.378 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.378 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.378 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.378 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.378 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.379 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.379 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.379 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.379 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.379 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.379 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.379 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.379 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.379 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.380 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.380 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.380 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.380 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.380 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:26.380 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T18:21:46.381 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:21:46.501 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.489+0000 7fbc831e1700 1 Processor -- start 2024-09-13T18:21:46.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.489+0000 7fbc831e1700 1 -- start start 2024-09-13T18:21:46.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.489+0000 7fbc831e1700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc7c1503d0 0x7fbc7c156940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:46.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.489+0000 7fbc831e1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbc7c06b4a0 0x7fbc7c071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:46.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.489+0000 7fbc831e1700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc7c15a520 con 0x7fbc7c072130 2024-09-13T18:21:46.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.489+0000 7fbc831e1700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc7c1564b0 con 0x7fbc7c1503d0 2024-09-13T18:21:46.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.489+0000 7fbc831e1700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c15a000 con 0x7fbc7c06b4a0 2024-09-13T18:21:46.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.489+0000 7fbc821df700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc7c1503d0 0x7fbc7c156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:46.503 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.489+0000 7fbc821df700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc7c1503d0 0x7fbc7c156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:38586/0 (socket says 172.21.15.40:38586) 2024-09-13T18:21:46.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.489+0000 7fbc821df700 1 -- 172.21.15.40:0/3251352582 learned_addr learned my addr 172.21.15.40:0/3251352582 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:46.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.489+0000 7fbc819de700 1 --2- 172.21.15.40:0/3251352582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbc7c06b4a0 0x7fbc7c071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:46.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.489+0000 7fbc821df700 1 -- 172.21.15.40:0/3251352582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbc7c06b4a0 msgr2=0x7fbc7c071bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:21:46.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.489+0000 7fbc821df700 1 --2- 172.21.15.40:0/3251352582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbc7c06b4a0 0x7fbc7c071bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:46.504 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.489+0000 7fbc821df700 1 -- 172.21.15.40:0/3251352582 >> v1:172.21.15.43:6789/0 conn(0x7fbc7c072130 legacy=0x7fbc7c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:46.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.489+0000 7fbc821df700 1 -- 172.21.15.40:0/3251352582 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7c15a280 con 0x7fbc7c1503d0 2024-09-13T18:21:46.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.493+0000 7fbc821df700 1 --2- 172.21.15.40:0/3251352582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc7c1503d0 0x7fbc7c156940 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009910 tx=0x7fbc78009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84669e4548a9624c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:46.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.493+0000 7fbc811dd700 1 -- 172.21.15.40:0/3251352582 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7800f8a0 con 0x7fbc7c1503d0 2024-09-13T18:21:46.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.493+0000 7fbc811dd700 1 -- 172.21.15.40:0/3251352582 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7800fa00 con 0x7fbc7c1503d0 2024-09-13T18:21:46.505 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.493+0000 7fbc811dd700 1 -- 172.21.15.40:0/3251352582 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7800fdb0 con 0x7fbc7c1503d0 2024-09-13T18:21:46.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.493+0000 7fbc831e1700 1 -- 172.21.15.40:0/3251352582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc7c1503d0 msgr2=0x7fbc7c156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:46.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.493+0000 7fbc831e1700 1 --2- 172.21.15.40:0/3251352582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc7c1503d0 0x7fbc7c156940 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009910 tx=0x7fbc78009c20 comp rx=0 tx=0).stop 2024-09-13T18:21:46.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.493+0000 7fbc831e1700 1 -- 172.21.15.40:0/3251352582 shutdown_connections 2024-09-13T18:21:46.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.493+0000 7fbc831e1700 1 --2- 172.21.15.40:0/3251352582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbc7c06b4a0 0x7fbc7c071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:46.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.493+0000 7fbc831e1700 1 --2- 172.21.15.40:0/3251352582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc7c1503d0 0x7fbc7c156940 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:46.506 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.493+0000 7fbc831e1700 1 -- 172.21.15.40:0/3251352582 >> 172.21.15.40:0/3251352582 conn(0x7fbc7c06e4f0 msgr2=0x7fbc7c06e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:46.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.493+0000 7fbc831e1700 1 -- 172.21.15.40:0/3251352582 shutdown_connections 2024-09-13T18:21:46.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.493+0000 7fbc831e1700 1 -- 172.21.15.40:0/3251352582 wait complete. 2024-09-13T18:21:46.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.493+0000 7fbc831e1700 1 Processor -- start 2024-09-13T18:21:46.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.493+0000 7fbc831e1700 1 -- start start 2024-09-13T18:21:46.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc831e1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc7c06b4a0 0x7fbc7c14e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:46.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc831e1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbc7c072130 0x7fbc7c153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:46.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc831e1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc7c1503d0 0x7fbc7c1ef730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:46.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc831e1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c158190 con 0x7fbc7c06b4a0 2024-09-13T18:21:46.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc831e1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c14edd0 con 0x7fbc7c1503d0 2024-09-13T18:21:46.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc831e1700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7c14eb30 con 0x7fbc7c072130 2024-09-13T18:21:46.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc821df700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc7c06b4a0 0x7fbc7c14e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:46.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc821df700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc7c06b4a0 0x7fbc7c14e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57990/0 (socket says 172.21.15.40:57990) 2024-09-13T18:21:46.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc821df700 1 -- 172.21.15.40:0/433959278 learned_addr learned my addr 172.21.15.40:0/433959278 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:21:46.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc829e0700 1 --2- 172.21.15.40:0/433959278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc7c1503d0 0x7fbc7c1ef730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:46.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc819de700 1 --2- 172.21.15.40:0/433959278 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbc7c072130 0x7fbc7c153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:46.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc821df700 1 -- 172.21.15.40:0/433959278 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbc7c072130 msgr2=0x7fbc7c153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:46.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc821df700 1 --2- 172.21.15.40:0/433959278 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbc7c072130 0x7fbc7c153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:46.514 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc821df700 1 -- 172.21.15.40:0/433959278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc7c1503d0 msgr2=0x7fbc7c1ef730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:46.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc821df700 1 --2- 172.21.15.40:0/433959278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc7c1503d0 0x7fbc7c1ef730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:46.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc821df700 1 -- 172.21.15.40:0/433959278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc78009570 con 0x7fbc7c06b4a0 2024-09-13T18:21:46.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc821df700 1 --2- 172.21.15.40:0/433959278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc7c06b4a0 0x7fbc7c14e5f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbc78000c00 tx=0x7fbc78004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:46.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc72ffd700 1 -- 172.21.15.40:0/433959278 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78003ad0 con 0x7fbc7c06b4a0 2024-09-13T18:21:46.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc72ffd700 1 -- 172.21.15.40:0/433959278 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc78003c30 con 0x7fbc7c06b4a0 2024-09-13T18:21:46.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc831e1700 1 -- 172.21.15.40:0/433959278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc7c14fb00 con 0x7fbc7c06b4a0 2024-09-13T18:21:46.515 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc72ffd700 1 -- 172.21.15.40:0/433959278 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc780045a0 con 0x7fbc7c06b4a0 2024-09-13T18:21:46.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc831e1700 1 -- 172.21.15.40:0/433959278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc7c1ef250 con 0x7fbc7c06b4a0 2024-09-13T18:21:46.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc831e1700 1 -- 172.21.15.40:0/433959278 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc7c1ef0b0 con 0x7fbc7c06b4a0 2024-09-13T18:21:46.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.497+0000 7fbc72ffd700 1 -- 172.21.15.40:0/433959278 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbc78019400 con 0x7fbc7c06b4a0 2024-09-13T18:21:46.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.501+0000 7fbc72ffd700 1 --2- 172.21.15.40:0/433959278 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbc60066730 0x7fbc60068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:21:46.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.501+0000 7fbc819de700 1 --2- 172.21.15.40:0/433959278 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbc60066730 0x7fbc60068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:21:46.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.501+0000 7fbc72ffd700 1 -- 172.21.15.40:0/433959278 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbc780851d0 con 0x7fbc7c06b4a0 2024-09-13T18:21:46.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.501+0000 7fbc819de700 1 --2- 172.21.15.40:0/433959278 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbc60066730 0x7fbc60068bf0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c06f420 tx=0x7fbc6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:21:46.517 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.505+0000 7fbc72ffd700 1 -- 172.21.15.40:0/433959278 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbc7c1ef0b0 con 0x7fbc7c06b4a0 2024-09-13T18:21:46.680 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.673+0000 7fbc831e1700 1 -- 172.21.15.40:0/433959278 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbc7c0b2130 con 0x7fbc60066730 2024-09-13T18:21:46.697 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.689+0000 7fbc72ffd700 1 -- 172.21.15.40:0/433959278 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+366542 (secure 0 0 0) 0x7fbc6c011780 con 0x7fbc60066730 2024-09-13T18:21:46.705 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T18:21:46.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.697+0000 7fbc831e1700 1 -- 172.21.15.40:0/433959278 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbc60066730 msgr2=0x7fbc60068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:46.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.697+0000 7fbc831e1700 1 --2- 172.21.15.40:0/433959278 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbc60066730 0x7fbc60068bf0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c06f420 tx=0x7fbc6c008040 comp rx=0 tx=0).stop 2024-09-13T18:21:46.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.697+0000 7fbc831e1700 1 -- 172.21.15.40:0/433959278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc7c06b4a0 msgr2=0x7fbc7c14e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:21:46.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.697+0000 7fbc831e1700 1 --2- 172.21.15.40:0/433959278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc7c06b4a0 0x7fbc7c14e5f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbc78000c00 tx=0x7fbc78004770 comp rx=0 tx=0).stop 2024-09-13T18:21:46.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.697+0000 7fbc831e1700 1 -- 172.21.15.40:0/433959278 shutdown_connections 2024-09-13T18:21:46.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.697+0000 7fbc831e1700 1 --2- 172.21.15.40:0/433959278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc7c1503d0 0x7fbc7c1ef730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:46.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.697+0000 7fbc831e1700 1 --2- 172.21.15.40:0/433959278 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbc60066730 0x7fbc60068bf0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:46.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.697+0000 7fbc831e1700 1 --2- 172.21.15.40:0/433959278 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fbc7c072130 0x7fbc7c153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:46.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.697+0000 7fbc831e1700 1 --2- 172.21.15.40:0/433959278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbc7c06b4a0 0x7fbc7c14e5f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:21:46.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.697+0000 7fbc831e1700 1 -- 172.21.15.40:0/433959278 >> 172.21.15.40:0/433959278 conn(0x7fbc7c06e4f0 msgr2=0x7fbc7c070bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:21:46.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.697+0000 7fbc831e1700 1 -- 172.21.15.40:0/433959278 shutdown_connections 2024-09-13T18:21:46.710 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:21:46.697+0000 7fbc831e1700 1 -- 172.21.15.40:0/433959278 wait complete. 2024-09-13T18:21:46.710 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-13T18:21:46.718 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":962,"stamp":"2024-09-13T18:21:45.782816+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":16532352,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":237,"num_read_kb":279,"num_write":755012,"num_write_kb":16125724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10000,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183102,"ondisk_log_size":183102,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10715160,"kb_used_data":34648,"kb_used_omap":12,"kb_used_meta":10680435,"kb_avail":744259560,"statfs":{"total":773094113280,"available":762121789440,"internally_reserved":0,"allocated":35479552,"data_stored":33992096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":10936766008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003419"},"pg_stats":[{"pgid":"2.1f","version":"27'272","reported_seq":315,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:30.246911+0000","last_change":"2024-09-13T18:21:30.247079+0000","last_active":"2024-09-13T18:21:30.246911+0000","last_peered":"2024-09-13T18:21:30.246911+0000","last_clean":"2024-09-13T18:21:30.246911+0000","last_became_active":"2024-09-13T17:50:34.211867+0000","last_became_peered":"2024-09-13T17:50:34.211867+0000","last_unstale":"2024-09-13T18:21:30.246911+0000","last_undegraded":"2024-09-13T18:21:30.246911+0000","last_fullsized":"2024-09-13T18:21:30.246911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'272","last_scrub_stamp":"2024-09-13T18:21:30.246880+0000","last_deep_scrub":"27'272","last_deep_scrub_stamp":"2024-09-13T18:21:30.246880+0000","last_clean_scrub_stamp":"2024-09-13T18:21:30.246880+0000","objects_scrubbed":5,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:57:30.406058+0000","scrub_duration":0.02016366,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3260943,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":267,"num_write_kb":3318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'10","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306167+0000","last_change":"2024-09-13T17:50:34.587396+0000","last_active":"2024-09-13T18:21:13.306167+0000","last_peered":"2024-09-13T18:21:13.306167+0000","last_clean":"2024-09-13T18:21:13.306167+0000","last_became_active":"2024-09-13T17:50:34.587108+0000","last_became_peered":"2024-09-13T17:50:34.587108+0000","last_unstale":"2024-09-13T18:21:13.306167+0000","last_undegraded":"2024-09-13T18:21:13.306167+0000","last_fullsized":"2024-09-13T18:21:13.306167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'5594","reported_seq":5621,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.921467+0000","last_change":"2024-09-13T17:50:38.615059+0000","last_active":"2024-09-13T18:21:02.921467+0000","last_peered":"2024-09-13T18:21:02.921467+0000","last_clean":"2024-09-13T18:21:02.921467+0000","last_became_active":"2024-09-13T17:50:38.614723+0000","last_became_peered":"2024-09-13T17:50:38.614723+0000","last_unstale":"2024-09-13T18:21:02.921467+0000","last_undegraded":"2024-09-13T18:21:02.921467+0000","last_fullsized":"2024-09-13T18:21:02.921467+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5594,"log_dups_size":0,"ondisk_log_size":5594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23732,"num_write_kb":533374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'6","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:36.210186+0000","last_change":"2024-09-13T18:21:36.210368+0000","last_active":"2024-09-13T18:21:36.210186+0000","last_peered":"2024-09-13T18:21:36.210186+0000","last_clean":"2024-09-13T18:21:36.210186+0000","last_became_active":"2024-09-13T17:50:34.202930+0000","last_became_peered":"2024-09-13T17:50:34.202930+0000","last_unstale":"2024-09-13T18:21:36.210186+0000","last_undegraded":"2024-09-13T18:21:36.210186+0000","last_fullsized":"2024-09-13T18:21:36.210186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-09-13T18:21:36.210139+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-09-13T18:21:36.210139+0000","last_clean_scrub_stamp":"2024-09-13T18:21:36.210139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:56.166821+0000","scrub_duration":0.0028089339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'20","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306569+0000","last_change":"2024-09-13T17:51:16.366352+0000","last_active":"2024-09-13T18:21:13.306569+0000","last_peered":"2024-09-13T18:21:13.306569+0000","last_clean":"2024-09-13T18:21:13.306569+0000","last_became_active":"2024-09-13T17:51:16.365492+0000","last_became_peered":"2024-09-13T17:51:16.365492+0000","last_unstale":"2024-09-13T18:21:13.306569+0000","last_undegraded":"2024-09-13T18:21:13.306569+0000","last_fullsized":"2024-09-13T18:21:13.306569+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'5615","reported_seq":5642,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.910391+0000","last_change":"2024-09-13T17:50:38.618751+0000","last_active":"2024-09-13T18:21:02.910391+0000","last_peered":"2024-09-13T18:21:02.910391+0000","last_clean":"2024-09-13T18:21:02.910391+0000","last_became_active":"2024-09-13T17:50:38.618423+0000","last_became_peered":"2024-09-13T17:50:38.618423+0000","last_unstale":"2024-09-13T18:21:02.910391+0000","last_undegraded":"2024-09-13T18:21:02.910391+0000","last_fullsized":"2024-09-13T18:21:02.910391+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5615,"log_dups_size":0,"ondisk_log_size":5615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23762,"num_write_kb":516686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'11","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.822268+0000","last_change":"2024-09-13T17:50:34.214972+0000","last_active":"2024-09-13T18:21:02.822268+0000","last_peered":"2024-09-13T18:21:02.822268+0000","last_clean":"2024-09-13T18:21:02.822268+0000","last_became_active":"2024-09-13T17:50:34.214652+0000","last_became_peered":"2024-09-13T17:50:34.214652+0000","last_unstale":"2024-09-13T18:21:02.822268+0000","last_undegraded":"2024-09-13T18:21:02.822268+0000","last_fullsized":"2024-09-13T18:21:02.822268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'18","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:17.096672+0000","last_change":"2024-09-13T17:51:16.281831+0000","last_active":"2024-09-13T18:21:17.096672+0000","last_peered":"2024-09-13T18:21:17.096672+0000","last_clean":"2024-09-13T18:21:17.096672+0000","last_became_active":"2024-09-13T17:51:16.281549+0000","last_became_peered":"2024-09-13T17:51:16.281549+0000","last_unstale":"2024-09-13T18:21:17.096672+0000","last_undegraded":"2024-09-13T18:21:17.096672+0000","last_fullsized":"2024-09-13T18:21:17.096672+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"27'5142","reported_seq":5169,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.914084+0000","last_change":"2024-09-13T17:50:38.616219+0000","last_active":"2024-09-13T18:21:02.914084+0000","last_peered":"2024-09-13T18:21:02.914084+0000","last_clean":"2024-09-13T18:21:02.914084+0000","last_became_active":"2024-09-13T17:50:38.614329+0000","last_became_peered":"2024-09-13T17:50:38.614329+0000","last_unstale":"2024-09-13T18:21:02.914084+0000","last_undegraded":"2024-09-13T18:21:02.914084+0000","last_fullsized":"2024-09-13T18:21:02.914084+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5142,"log_dups_size":0,"ondisk_log_size":5142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21334,"num_write_kb":478261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'8","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:36.022769+0000","last_change":"2024-09-13T17:50:34.216123+0000","last_active":"2024-09-13T18:20:36.022769+0000","last_peered":"2024-09-13T18:20:36.022769+0000","last_clean":"2024-09-13T18:20:36.022769+0000","last_became_active":"2024-09-13T17:50:34.215661+0000","last_became_peered":"2024-09-13T17:50:34.215661+0000","last_unstale":"2024-09-13T18:20:36.022769+0000","last_undegraded":"2024-09-13T18:20:36.022769+0000","last_fullsized":"2024-09-13T18:20:36.022769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'30","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.319051+0000","last_change":"2024-09-13T17:50:34.599673+0000","last_active":"2024-09-13T18:21:13.319051+0000","last_peered":"2024-09-13T18:21:13.319051+0000","last_clean":"2024-09-13T18:21:13.319051+0000","last_became_active":"2024-09-13T17:50:34.599409+0000","last_became_peered":"2024-09-13T17:50:34.599409+0000","last_unstale":"2024-09-13T18:21:13.319051+0000","last_undegraded":"2024-09-13T18:21:13.319051+0000","last_fullsized":"2024-09-13T18:21:13.319051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'4871","reported_seq":4898,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.910129+0000","last_change":"2024-09-13T17:50:38.616317+0000","last_active":"2024-09-13T18:21:02.910129+0000","last_peered":"2024-09-13T18:21:02.910129+0000","last_clean":"2024-09-13T18:21:02.910129+0000","last_became_active":"2024-09-13T17:50:38.614311+0000","last_became_peered":"2024-09-13T17:50:38.614311+0000","last_unstale":"2024-09-13T18:21:02.910129+0000","last_undegraded":"2024-09-13T18:21:02.910129+0000","last_fullsized":"2024-09-13T18:21:02.910129+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4871,"log_dups_size":0,"ondisk_log_size":4871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20106,"num_write_kb":432357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'14","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.313254+0000","last_change":"2024-09-13T17:50:34.599490+0000","last_active":"2024-09-13T18:21:13.313254+0000","last_peered":"2024-09-13T18:21:13.313254+0000","last_clean":"2024-09-13T18:21:13.313254+0000","last_became_active":"2024-09-13T17:50:34.599207+0000","last_became_peered":"2024-09-13T17:50:34.599207+0000","last_unstale":"2024-09-13T18:21:13.313254+0000","last_undegraded":"2024-09-13T18:21:13.313254+0000","last_fullsized":"2024-09-13T18:21:13.313254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'1092","reported_seq":1123,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.834040+0000","last_change":"2024-09-13T17:50:34.215697+0000","last_active":"2024-09-13T18:21:02.834040+0000","last_peered":"2024-09-13T18:21:02.834040+0000","last_clean":"2024-09-13T18:21:02.834040+0000","last_became_active":"2024-09-13T17:50:34.215409+0000","last_became_peered":"2024-09-13T17:50:34.215409+0000","last_unstale":"2024-09-13T18:21:02.834040+0000","last_undegraded":"2024-09-13T18:21:02.834040+0000","last_fullsized":"2024-09-13T18:21:02.834040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":1092,"log_dups_size":0,"ondisk_log_size":1092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1088,"num_write_kb":4339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'5394","reported_seq":5422,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.899942+0000","last_change":"2024-09-13T17:50:38.611817+0000","last_active":"2024-09-13T18:21:02.899942+0000","last_peered":"2024-09-13T18:21:02.899942+0000","last_clean":"2024-09-13T18:21:02.899942+0000","last_became_active":"2024-09-13T17:50:38.611263+0000","last_became_peered":"2024-09-13T17:50:38.611263+0000","last_unstale":"2024-09-13T18:21:02.899942+0000","last_undegraded":"2024-09-13T18:21:02.899942+0000","last_fullsized":"2024-09-13T18:21:02.899942+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5394,"log_dups_size":0,"ondisk_log_size":5394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22326,"num_write_kb":494524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'19","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.308493+0000","last_change":"2024-09-13T17:50:34.586307+0000","last_active":"2024-09-13T18:21:13.308493+0000","last_peered":"2024-09-13T18:21:13.308493+0000","last_clean":"2024-09-13T18:21:13.308493+0000","last_became_active":"2024-09-13T17:50:34.585831+0000","last_became_peered":"2024-09-13T17:50:34.585831+0000","last_unstale":"2024-09-13T18:21:13.308493+0000","last_undegraded":"2024-09-13T18:21:13.308493+0000","last_fullsized":"2024-09-13T18:21:13.308493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'10","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:34.350232+0000","last_change":"2024-09-13T18:21:34.350402+0000","last_active":"2024-09-13T18:21:34.350232+0000","last_peered":"2024-09-13T18:21:34.350232+0000","last_clean":"2024-09-13T18:21:34.350232+0000","last_became_active":"2024-09-13T17:50:34.211594+0000","last_became_peered":"2024-09-13T17:50:34.211594+0000","last_unstale":"2024-09-13T18:21:34.350232+0000","last_undegraded":"2024-09-13T18:21:34.350232+0000","last_fullsized":"2024-09-13T18:21:34.350232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-13T18:21:34.350185+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-13T18:21:34.350185+0000","last_clean_scrub_stamp":"2024-09-13T18:21:34.350185+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:03.648370+0000","scrub_duration":0.003876764,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'5504","reported_seq":5532,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.899928+0000","last_change":"2024-09-13T17:50:38.613576+0000","last_active":"2024-09-13T18:21:02.899928+0000","last_peered":"2024-09-13T18:21:02.899928+0000","last_clean":"2024-09-13T18:21:02.899928+0000","last_became_active":"2024-09-13T17:50:38.612577+0000","last_became_peered":"2024-09-13T17:50:38.612577+0000","last_unstale":"2024-09-13T18:21:02.899928+0000","last_undegraded":"2024-09-13T18:21:02.899928+0000","last_fullsized":"2024-09-13T18:21:02.899928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5504,"log_dups_size":0,"ondisk_log_size":5504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23286,"num_write_kb":502495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'19","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.306542+0000","last_change":"2024-09-13T17:50:34.587273+0000","last_active":"2024-09-13T18:21:13.306542+0000","last_peered":"2024-09-13T18:21:13.306542+0000","last_clean":"2024-09-13T18:21:13.306542+0000","last_became_active":"2024-09-13T17:50:34.587027+0000","last_became_peered":"2024-09-13T17:50:34.587027+0000","last_unstale":"2024-09-13T18:21:13.306542+0000","last_undegraded":"2024-09-13T18:21:13.306542+0000","last_fullsized":"2024-09-13T18:21:13.306542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'7","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:32.340544+0000","last_change":"2024-09-13T18:21:32.340739+0000","last_active":"2024-09-13T18:21:32.340544+0000","last_peered":"2024-09-13T18:21:32.340544+0000","last_clean":"2024-09-13T18:21:32.340544+0000","last_became_active":"2024-09-13T17:50:34.211568+0000","last_became_peered":"2024-09-13T17:50:34.211568+0000","last_unstale":"2024-09-13T18:21:32.340544+0000","last_undegraded":"2024-09-13T18:21:32.340544+0000","last_fullsized":"2024-09-13T18:21:32.340544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-13T18:21:32.340496+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-13T18:21:32.340496+0000","last_clean_scrub_stamp":"2024-09-13T18:21:32.340496+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:14:08.632192+0000","scrub_duration":0.0064919820000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'6938","reported_seq":6966,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.785985+0000","last_change":"2024-09-13T17:50:38.610292+0000","last_active":"2024-09-13T18:21:02.785985+0000","last_peered":"2024-09-13T18:21:02.785985+0000","last_clean":"2024-09-13T18:21:02.785985+0000","last_became_active":"2024-09-13T17:50:38.609832+0000","last_became_peered":"2024-09-13T17:50:38.609832+0000","last_unstale":"2024-09-13T18:21:02.785985+0000","last_undegraded":"2024-09-13T18:21:02.785985+0000","last_fullsized":"2024-09-13T18:21:02.785985+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6938,"log_dups_size":0,"ondisk_log_size":6938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28644,"num_write_kb":622131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"27'12","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:33.269238+0000","last_change":"2024-09-13T18:21:33.269565+0000","last_active":"2024-09-13T18:21:33.269238+0000","last_peered":"2024-09-13T18:21:33.269238+0000","last_clean":"2024-09-13T18:21:33.269238+0000","last_became_active":"2024-09-13T17:50:34.588794+0000","last_became_peered":"2024-09-13T17:50:34.588794+0000","last_unstale":"2024-09-13T18:21:33.269238+0000","last_undegraded":"2024-09-13T18:21:33.269238+0000","last_fullsized":"2024-09-13T18:21:33.269238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'12","last_scrub_stamp":"2024-09-13T18:21:33.269111+0000","last_deep_scrub":"27'12","last_deep_scrub_stamp":"2024-09-13T18:21:33.269111+0000","last_clean_scrub_stamp":"2024-09-13T18:21:33.269111+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:41.084595+0000","scrub_duration":0.003356562,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'6","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.729017+0000","last_change":"2024-09-13T17:50:34.214638+0000","last_active":"2024-09-13T18:21:02.729017+0000","last_peered":"2024-09-13T18:21:02.729017+0000","last_clean":"2024-09-13T18:21:02.729017+0000","last_became_active":"2024-09-13T17:50:34.214292+0000","last_became_peered":"2024-09-13T17:50:34.214292+0000","last_unstale":"2024-09-13T18:21:02.729017+0000","last_undegraded":"2024-09-13T18:21:02.729017+0000","last_fullsized":"2024-09-13T18:21:02.729017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'4588","reported_seq":4616,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.944693+0000","last_change":"2024-09-13T17:50:38.613368+0000","last_active":"2024-09-13T18:21:02.944693+0000","last_peered":"2024-09-13T18:21:02.944693+0000","last_clean":"2024-09-13T18:21:02.944693+0000","last_became_active":"2024-09-13T17:50:38.613010+0000","last_became_peered":"2024-09-13T17:50:38.613010+0000","last_unstale":"2024-09-13T18:21:02.944693+0000","last_undegraded":"2024-09-13T18:21:02.944693+0000","last_fullsized":"2024-09-13T18:21:02.944693+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4588,"log_dups_size":0,"ondisk_log_size":4588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19354,"num_write_kb":454021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'26","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.302587+0000","last_change":"2024-09-13T17:50:34.582327+0000","last_active":"2024-09-13T18:21:13.302587+0000","last_peered":"2024-09-13T18:21:13.302587+0000","last_clean":"2024-09-13T18:21:13.302587+0000","last_became_active":"2024-09-13T17:50:34.582050+0000","last_became_peered":"2024-09-13T17:50:34.582050+0000","last_unstale":"2024-09-13T18:21:13.302587+0000","last_undegraded":"2024-09-13T18:21:13.302587+0000","last_fullsized":"2024-09-13T18:21:13.302587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'11","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:32.305631+0000","last_change":"2024-09-13T18:21:32.306235+0000","last_active":"2024-09-13T18:21:32.305631+0000","last_peered":"2024-09-13T18:21:32.305631+0000","last_clean":"2024-09-13T18:21:32.305631+0000","last_became_active":"2024-09-13T17:50:34.211230+0000","last_became_peered":"2024-09-13T17:50:34.211230+0000","last_unstale":"2024-09-13T18:21:32.305631+0000","last_undegraded":"2024-09-13T18:21:32.305631+0000","last_fullsized":"2024-09-13T18:21:32.305631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-09-13T18:21:32.305579+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-09-13T18:21:32.305579+0000","last_clean_scrub_stamp":"2024-09-13T18:21:32.305579+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:08.984532+0000","scrub_duration":0.0066305000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'6024","reported_seq":6053,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.959977+0000","last_change":"2024-09-13T17:50:38.608255+0000","last_active":"2024-09-13T18:21:02.959977+0000","last_peered":"2024-09-13T18:21:02.959977+0000","last_clean":"2024-09-13T18:21:02.959977+0000","last_became_active":"2024-09-13T17:50:38.606697+0000","last_became_peered":"2024-09-13T17:50:38.606697+0000","last_unstale":"2024-09-13T18:21:02.959977+0000","last_undegraded":"2024-09-13T18:21:02.959977+0000","last_fullsized":"2024-09-13T18:21:02.959977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6024,"log_dups_size":0,"ondisk_log_size":6024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25552,"num_write_kb":507487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'18","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317154+0000","last_change":"2024-09-13T17:50:34.594476+0000","last_active":"2024-09-13T18:21:13.317154+0000","last_peered":"2024-09-13T18:21:13.317154+0000","last_clean":"2024-09-13T18:21:13.317154+0000","last_became_active":"2024-09-13T17:50:34.594215+0000","last_became_peered":"2024-09-13T17:50:34.594215+0000","last_unstale":"2024-09-13T18:21:13.317154+0000","last_undegraded":"2024-09-13T18:21:13.317154+0000","last_fullsized":"2024-09-13T18:21:13.317154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'286","reported_seq":329,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:36.413084+0000","last_change":"2024-09-13T18:21:36.413202+0000","last_active":"2024-09-13T18:21:36.413084+0000","last_peered":"2024-09-13T18:21:36.413084+0000","last_clean":"2024-09-13T18:21:36.413084+0000","last_became_active":"2024-09-13T17:50:34.203702+0000","last_became_peered":"2024-09-13T17:50:34.203702+0000","last_unstale":"2024-09-13T18:21:36.413084+0000","last_undegraded":"2024-09-13T18:21:36.413084+0000","last_fullsized":"2024-09-13T18:21:36.413084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'286","last_scrub_stamp":"2024-09-13T18:21:36.413050+0000","last_deep_scrub":"27'286","last_deep_scrub_stamp":"2024-09-13T18:21:36.413050+0000","last_clean_scrub_stamp":"2024-09-13T18:21:36.413050+0000","objects_scrubbed":2,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:44.535556+0000","scrub_duration":0.0068211640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":605,"num_write_kb":197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'6030","reported_seq":6070,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.838464+0000","last_change":"2024-09-13T17:51:16.365782+0000","last_active":"2024-09-13T18:21:02.838464+0000","last_peered":"2024-09-13T18:21:02.838464+0000","last_clean":"2024-09-13T18:21:02.838464+0000","last_became_active":"2024-09-13T17:51:16.364540+0000","last_became_peered":"2024-09-13T17:51:16.364540+0000","last_unstale":"2024-09-13T18:21:02.838464+0000","last_undegraded":"2024-09-13T18:21:02.838464+0000","last_fullsized":"2024-09-13T18:21:02.838464+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6030,"log_dups_size":0,"ondisk_log_size":6030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25617,"num_write_kb":548663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'18","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.290477+0000","last_change":"2024-09-13T17:50:34.586802+0000","last_active":"2024-09-13T18:21:13.290477+0000","last_peered":"2024-09-13T18:21:13.290477+0000","last_clean":"2024-09-13T18:21:13.290477+0000","last_became_active":"2024-09-13T17:50:34.586552+0000","last_became_peered":"2024-09-13T17:50:34.586552+0000","last_unstale":"2024-09-13T18:21:13.290477+0000","last_undegraded":"2024-09-13T18:21:13.290477+0000","last_fullsized":"2024-09-13T18:21:13.290477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'7","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.854148+0000","last_change":"2024-09-13T17:50:34.217083+0000","last_active":"2024-09-13T18:21:02.854148+0000","last_peered":"2024-09-13T18:21:02.854148+0000","last_clean":"2024-09-13T18:21:02.854148+0000","last_became_active":"2024-09-13T17:50:34.216758+0000","last_became_peered":"2024-09-13T17:50:34.216758+0000","last_unstale":"2024-09-13T18:21:02.854148+0000","last_undegraded":"2024-09-13T18:21:02.854148+0000","last_fullsized":"2024-09-13T18:21:02.854148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'4887","reported_seq":4916,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.697493+0000","last_change":"2024-09-13T17:50:38.614667+0000","last_active":"2024-09-13T18:21:02.697493+0000","last_peered":"2024-09-13T18:21:02.697493+0000","last_clean":"2024-09-13T18:21:02.697493+0000","last_became_active":"2024-09-13T17:50:38.614284+0000","last_became_peered":"2024-09-13T17:50:38.614284+0000","last_unstale":"2024-09-13T18:21:02.697493+0000","last_undegraded":"2024-09-13T18:21:02.697493+0000","last_fullsized":"2024-09-13T18:21:02.697493+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4887,"log_dups_size":0,"ondisk_log_size":4887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20690,"num_write_kb":442963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'16","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.315402+0000","last_change":"2024-09-13T17:50:34.588603+0000","last_active":"2024-09-13T18:21:13.315402+0000","last_peered":"2024-09-13T18:21:13.315402+0000","last_clean":"2024-09-13T18:21:13.315402+0000","last_became_active":"2024-09-13T17:50:34.588317+0000","last_became_peered":"2024-09-13T17:50:34.588317+0000","last_unstale":"2024-09-13T18:21:13.315402+0000","last_undegraded":"2024-09-13T18:21:13.315402+0000","last_fullsized":"2024-09-13T18:21:13.315402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'3","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.791579+0000","last_change":"2024-09-13T17:50:34.204335+0000","last_active":"2024-09-13T18:21:02.791579+0000","last_peered":"2024-09-13T18:21:02.791579+0000","last_clean":"2024-09-13T18:21:02.791579+0000","last_became_active":"2024-09-13T17:50:34.204060+0000","last_became_peered":"2024-09-13T17:50:34.204060+0000","last_unstale":"2024-09-13T18:21:02.791579+0000","last_undegraded":"2024-09-13T18:21:02.791579+0000","last_fullsized":"2024-09-13T18:21:02.791579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'6031","reported_seq":6060,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.728305+0000","last_change":"2024-09-13T17:50:38.619590+0000","last_active":"2024-09-13T18:21:02.728305+0000","last_peered":"2024-09-13T18:21:02.728305+0000","last_clean":"2024-09-13T18:21:02.728305+0000","last_became_active":"2024-09-13T17:50:38.617257+0000","last_became_peered":"2024-09-13T17:50:38.617257+0000","last_unstale":"2024-09-13T18:21:02.728305+0000","last_undegraded":"2024-09-13T18:21:02.728305+0000","last_fullsized":"2024-09-13T18:21:02.728305+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6031,"log_dups_size":0,"ondisk_log_size":6031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25529,"num_write_kb":564991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'16","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310453+0000","last_change":"2024-09-13T17:50:34.588466+0000","last_active":"2024-09-13T18:21:13.310453+0000","last_peered":"2024-09-13T18:21:13.310453+0000","last_clean":"2024-09-13T18:21:13.310453+0000","last_became_active":"2024-09-13T17:50:34.588063+0000","last_became_peered":"2024-09-13T17:50:34.588063+0000","last_unstale":"2024-09-13T18:21:13.310453+0000","last_undegraded":"2024-09-13T18:21:13.310453+0000","last_fullsized":"2024-09-13T18:21:13.310453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'4","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:35.892960+0000","last_change":"2024-09-13T18:21:35.893165+0000","last_active":"2024-09-13T18:21:35.892960+0000","last_peered":"2024-09-13T18:21:35.892960+0000","last_clean":"2024-09-13T18:21:35.892960+0000","last_became_active":"2024-09-13T17:50:34.208196+0000","last_became_peered":"2024-09-13T17:50:34.208196+0000","last_unstale":"2024-09-13T18:21:35.892960+0000","last_undegraded":"2024-09-13T18:21:35.892960+0000","last_fullsized":"2024-09-13T18:21:35.892960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-09-13T18:21:35.892908+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-09-13T18:21:35.892908+0000","last_clean_scrub_stamp":"2024-09-13T18:21:35.892908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:50.582547+0000","scrub_duration":0.0034542710000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'5410","reported_seq":5439,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.910724+0000","last_change":"2024-09-13T17:50:38.611021+0000","last_active":"2024-09-13T18:21:02.910724+0000","last_peered":"2024-09-13T18:21:02.910724+0000","last_clean":"2024-09-13T18:21:02.910724+0000","last_became_active":"2024-09-13T17:50:38.609794+0000","last_became_peered":"2024-09-13T17:50:38.609794+0000","last_unstale":"2024-09-13T18:21:02.910724+0000","last_undegraded":"2024-09-13T18:21:02.910724+0000","last_fullsized":"2024-09-13T18:21:02.910724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5410,"log_dups_size":0,"ondisk_log_size":5410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22186,"num_write_kb":506224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'22","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.290730+0000","last_change":"2024-09-13T17:50:34.584290+0000","last_active":"2024-09-13T18:21:13.290730+0000","last_peered":"2024-09-13T18:21:13.290730+0000","last_clean":"2024-09-13T18:21:13.290730+0000","last_became_active":"2024-09-13T17:50:34.584023+0000","last_became_peered":"2024-09-13T17:50:34.584023+0000","last_unstale":"2024-09-13T18:21:13.290730+0000","last_undegraded":"2024-09-13T18:21:13.290730+0000","last_fullsized":"2024-09-13T18:21:13.290730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'127","reported_seq":280,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.830874+0000","last_change":"2024-09-13T17:50:34.198932+0000","last_active":"2024-09-13T18:21:02.830874+0000","last_peered":"2024-09-13T18:21:02.830874+0000","last_clean":"2024-09-13T18:21:02.830874+0000","last_became_active":"2024-09-13T17:50:34.198603+0000","last_became_peered":"2024-09-13T17:50:34.198603+0000","last_unstale":"2024-09-13T18:21:02.830874+0000","last_undegraded":"2024-09-13T18:21:02.830874+0000","last_fullsized":"2024-09-13T18:21:02.830874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":94032,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":179,"num_write":125,"num_write_kb":180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'5807","reported_seq":5836,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.850332+0000","last_change":"2024-09-13T17:50:38.611326+0000","last_active":"2024-09-13T18:21:02.850332+0000","last_peered":"2024-09-13T18:21:02.850332+0000","last_clean":"2024-09-13T18:21:02.850332+0000","last_became_active":"2024-09-13T17:50:38.610900+0000","last_became_peered":"2024-09-13T17:50:38.610900+0000","last_unstale":"2024-09-13T18:21:02.850332+0000","last_undegraded":"2024-09-13T18:21:02.850332+0000","last_fullsized":"2024-09-13T18:21:02.850332+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5807,"log_dups_size":0,"ondisk_log_size":5807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24429,"num_write_kb":531605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'13","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:32.283757+0000","last_change":"2024-09-13T18:21:32.284016+0000","last_active":"2024-09-13T18:21:32.283757+0000","last_peered":"2024-09-13T18:21:32.283757+0000","last_clean":"2024-09-13T18:21:32.283757+0000","last_became_active":"2024-09-13T17:50:34.594201+0000","last_became_peered":"2024-09-13T17:50:34.594201+0000","last_unstale":"2024-09-13T18:21:32.283757+0000","last_undegraded":"2024-09-13T18:21:32.283757+0000","last_fullsized":"2024-09-13T18:21:32.283757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'13","last_scrub_stamp":"2024-09-13T18:21:32.283704+0000","last_deep_scrub":"27'13","last_deep_scrub_stamp":"2024-09-13T18:21:32.283704+0000","last_clean_scrub_stamp":"2024-09-13T18:21:32.283704+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:06.348602+0000","scrub_duration":0.0033353039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'8","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.833043+0000","last_change":"2024-09-13T17:50:34.218376+0000","last_active":"2024-09-13T18:21:02.833043+0000","last_peered":"2024-09-13T18:21:02.833043+0000","last_clean":"2024-09-13T18:21:02.833043+0000","last_became_active":"2024-09-13T17:50:34.218069+0000","last_became_peered":"2024-09-13T17:50:34.218069+0000","last_unstale":"2024-09-13T18:21:02.833043+0000","last_undegraded":"2024-09-13T18:21:02.833043+0000","last_fullsized":"2024-09-13T18:21:02.833043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'4957","reported_seq":4986,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.964133+0000","last_change":"2024-09-13T17:50:38.613206+0000","last_active":"2024-09-13T18:21:02.964133+0000","last_peered":"2024-09-13T18:21:02.964133+0000","last_clean":"2024-09-13T18:21:02.964133+0000","last_became_active":"2024-09-13T17:50:38.612856+0000","last_became_peered":"2024-09-13T17:50:38.612856+0000","last_unstale":"2024-09-13T18:21:02.964133+0000","last_undegraded":"2024-09-13T18:21:02.964133+0000","last_fullsized":"2024-09-13T18:21:02.964133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4957,"log_dups_size":0,"ondisk_log_size":4957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20520,"num_write_kb":434212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'16","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.316439+0000","last_change":"2024-09-13T17:50:34.582931+0000","last_active":"2024-09-13T18:21:13.316439+0000","last_peered":"2024-09-13T18:21:13.316439+0000","last_clean":"2024-09-13T18:21:13.316439+0000","last_became_active":"2024-09-13T17:50:34.582446+0000","last_became_peered":"2024-09-13T17:50:34.582446+0000","last_unstale":"2024-09-13T18:21:13.316439+0000","last_undegraded":"2024-09-13T18:21:13.316439+0000","last_fullsized":"2024-09-13T18:21:13.316439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'11","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.797367+0000","last_change":"2024-09-13T18:21:28.797573+0000","last_active":"2024-09-13T18:21:28.797367+0000","last_peered":"2024-09-13T18:21:28.797367+0000","last_clean":"2024-09-13T18:21:28.797367+0000","last_became_active":"2024-09-13T17:50:34.200873+0000","last_became_peered":"2024-09-13T17:50:34.200873+0000","last_unstale":"2024-09-13T18:21:28.797367+0000","last_undegraded":"2024-09-13T18:21:28.797367+0000","last_fullsized":"2024-09-13T18:21:28.797367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-09-13T18:21:28.797316+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-09-13T18:21:28.797316+0000","last_clean_scrub_stamp":"2024-09-13T18:21:28.797316+0000","objects_scrubbed":3,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:51:50.633990+0000","scrub_duration":0.0090957169999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":186,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:20:36.022631+0000","last_change":"2024-09-13T17:50:22.095367+0000","last_active":"2024-09-13T18:20:36.022631+0000","last_peered":"2024-09-13T18:20:36.022631+0000","last_clean":"2024-09-13T18:20:36.022631+0000","last_became_active":"2024-09-13T17:50:22.093734+0000","last_became_peered":"2024-09-13T17:50:22.093734+0000","last_unstale":"2024-09-13T18:20:36.022631+0000","last_undegraded":"2024-09-13T18:20:36.022631+0000","last_fullsized":"2024-09-13T18:20:36.022631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:21.057812+0000","last_clean_scrub_stamp":"2024-09-13T17:50:21.057812+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'5208","reported_seq":5237,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.913470+0000","last_change":"2024-09-13T17:50:38.618880+0000","last_active":"2024-09-13T18:21:02.913470+0000","last_peered":"2024-09-13T18:21:02.913470+0000","last_clean":"2024-09-13T18:21:02.913470+0000","last_became_active":"2024-09-13T17:50:38.618488+0000","last_became_peered":"2024-09-13T17:50:38.618488+0000","last_unstale":"2024-09-13T18:21:02.913470+0000","last_undegraded":"2024-09-13T18:21:02.913470+0000","last_fullsized":"2024-09-13T18:21:02.913470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5208,"log_dups_size":0,"ondisk_log_size":5208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22093,"num_write_kb":481370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'22","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.318438+0000","last_change":"2024-09-13T17:50:34.585881+0000","last_active":"2024-09-13T18:21:13.318438+0000","last_peered":"2024-09-13T18:21:13.318438+0000","last_clean":"2024-09-13T18:21:13.318438+0000","last_became_active":"2024-09-13T17:50:34.585166+0000","last_became_peered":"2024-09-13T17:50:34.585166+0000","last_unstale":"2024-09-13T18:21:13.318438+0000","last_undegraded":"2024-09-13T18:21:13.318438+0000","last_fullsized":"2024-09-13T18:21:13.318438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'7","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:34.775170+0000","last_change":"2024-09-13T18:21:34.775276+0000","last_active":"2024-09-13T18:21:34.775170+0000","last_peered":"2024-09-13T18:21:34.775170+0000","last_clean":"2024-09-13T18:21:34.775170+0000","last_became_active":"2024-09-13T17:50:34.210589+0000","last_became_peered":"2024-09-13T17:50:34.210589+0000","last_unstale":"2024-09-13T18:21:34.775170+0000","last_undegraded":"2024-09-13T18:21:34.775170+0000","last_fullsized":"2024-09-13T18:21:34.775170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-13T18:21:34.775137+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-13T18:21:34.775137+0000","last_clean_scrub_stamp":"2024-09-13T18:21:34.775137+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:40.007622+0000","scrub_duration":0.0064282710000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'5302","reported_seq":5341,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:06.106943+0000","last_change":"2024-09-13T17:51:16.364911+0000","last_active":"2024-09-13T18:21:06.106943+0000","last_peered":"2024-09-13T18:21:06.106943+0000","last_clean":"2024-09-13T18:21:06.106943+0000","last_became_active":"2024-09-13T17:51:16.363974+0000","last_became_peered":"2024-09-13T17:51:16.363974+0000","last_unstale":"2024-09-13T18:21:06.106943+0000","last_undegraded":"2024-09-13T18:21:06.106943+0000","last_fullsized":"2024-09-13T18:21:06.106943+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5302,"log_dups_size":0,"ondisk_log_size":5302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22724,"num_write_kb":488088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'20","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310467+0000","last_change":"2024-09-13T17:50:34.590169+0000","last_active":"2024-09-13T18:21:13.310467+0000","last_peered":"2024-09-13T18:21:13.310467+0000","last_clean":"2024-09-13T18:21:13.310467+0000","last_became_active":"2024-09-13T17:50:34.589434+0000","last_became_peered":"2024-09-13T17:50:34.589434+0000","last_unstale":"2024-09-13T18:21:13.310467+0000","last_undegraded":"2024-09-13T18:21:13.310467+0000","last_fullsized":"2024-09-13T18:21:13.310467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'5","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:33.383817+0000","last_change":"2024-09-13T18:21:33.384001+0000","last_active":"2024-09-13T18:21:33.383817+0000","last_peered":"2024-09-13T18:21:33.383817+0000","last_clean":"2024-09-13T18:21:33.383817+0000","last_became_active":"2024-09-13T17:50:34.203874+0000","last_became_peered":"2024-09-13T17:50:34.203874+0000","last_unstale":"2024-09-13T18:21:33.383817+0000","last_undegraded":"2024-09-13T18:21:33.383817+0000","last_fullsized":"2024-09-13T18:21:33.383817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-09-13T18:21:33.383770+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-09-13T18:21:33.383770+0000","last_clean_scrub_stamp":"2024-09-13T18:21:33.383770+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:07.542936+0000","scrub_duration":0.0056205279999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'5348","reported_seq":5376,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.945036+0000","last_change":"2024-09-13T17:50:38.615989+0000","last_active":"2024-09-13T18:21:02.945036+0000","last_peered":"2024-09-13T18:21:02.945036+0000","last_clean":"2024-09-13T18:21:02.945036+0000","last_became_active":"2024-09-13T17:50:38.615530+0000","last_became_peered":"2024-09-13T17:50:38.615530+0000","last_unstale":"2024-09-13T18:21:02.945036+0000","last_undegraded":"2024-09-13T18:21:02.945036+0000","last_fullsized":"2024-09-13T18:21:02.945036+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5348,"log_dups_size":0,"ondisk_log_size":5348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22134,"num_write_kb":473293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'20","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.322123+0000","last_change":"2024-09-13T17:50:34.595120+0000","last_active":"2024-09-13T18:21:13.322123+0000","last_peered":"2024-09-13T18:21:13.322123+0000","last_clean":"2024-09-13T18:21:13.322123+0000","last_became_active":"2024-09-13T17:50:34.594842+0000","last_became_peered":"2024-09-13T17:50:34.594842+0000","last_unstale":"2024-09-13T18:21:13.322123+0000","last_undegraded":"2024-09-13T18:21:13.322123+0000","last_fullsized":"2024-09-13T18:21:13.322123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'9","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.861186+0000","last_change":"2024-09-13T17:50:34.199957+0000","last_active":"2024-09-13T18:21:02.861186+0000","last_peered":"2024-09-13T18:21:02.861186+0000","last_clean":"2024-09-13T18:21:02.861186+0000","last_became_active":"2024-09-13T17:50:34.199650+0000","last_became_peered":"2024-09-13T17:50:34.199650+0000","last_unstale":"2024-09-13T18:21:02.861186+0000","last_undegraded":"2024-09-13T18:21:02.861186+0000","last_fullsized":"2024-09-13T18:21:02.861186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'5161","reported_seq":5190,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.766124+0000","last_change":"2024-09-13T17:50:38.609731+0000","last_active":"2024-09-13T18:21:02.766124+0000","last_peered":"2024-09-13T18:21:02.766124+0000","last_clean":"2024-09-13T18:21:02.766124+0000","last_became_active":"2024-09-13T17:50:38.609354+0000","last_became_peered":"2024-09-13T17:50:38.609354+0000","last_unstale":"2024-09-13T18:21:02.766124+0000","last_undegraded":"2024-09-13T18:21:02.766124+0000","last_fullsized":"2024-09-13T18:21:02.766124+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5161,"log_dups_size":0,"ondisk_log_size":5161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21929,"num_write_kb":464406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'10","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:26.347034+0000","last_change":"2024-09-13T18:21:26.347263+0000","last_active":"2024-09-13T18:21:26.347034+0000","last_peered":"2024-09-13T18:21:26.347034+0000","last_clean":"2024-09-13T18:21:26.347034+0000","last_became_active":"2024-09-13T17:50:34.594876+0000","last_became_peered":"2024-09-13T17:50:34.594876+0000","last_unstale":"2024-09-13T18:21:26.347034+0000","last_undegraded":"2024-09-13T18:21:26.347034+0000","last_fullsized":"2024-09-13T18:21:26.347034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-13T18:21:26.346976+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-13T18:21:26.346976+0000","last_clean_scrub_stamp":"2024-09-13T18:21:26.346976+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:25.039233+0000","scrub_duration":0.003420939,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'6","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.264168+0000","last_change":"2024-09-13T18:21:28.264384+0000","last_active":"2024-09-13T18:21:28.264168+0000","last_peered":"2024-09-13T18:21:28.264168+0000","last_clean":"2024-09-13T18:21:28.264168+0000","last_became_active":"2024-09-13T17:50:34.200197+0000","last_became_peered":"2024-09-13T17:50:34.200197+0000","last_unstale":"2024-09-13T18:21:28.264168+0000","last_undegraded":"2024-09-13T18:21:28.264168+0000","last_fullsized":"2024-09-13T18:21:28.264168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-09-13T18:21:28.264112+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-09-13T18:21:28.264112+0000","last_clean_scrub_stamp":"2024-09-13T18:21:28.264112+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:43:40.034203+0000","scrub_duration":0.0069816829999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'5052","reported_seq":5091,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.937231+0000","last_change":"2024-09-13T17:51:16.281995+0000","last_active":"2024-09-13T18:21:02.937231+0000","last_peered":"2024-09-13T18:21:02.937231+0000","last_clean":"2024-09-13T18:21:02.937231+0000","last_became_active":"2024-09-13T17:51:16.281476+0000","last_became_peered":"2024-09-13T17:51:16.281476+0000","last_unstale":"2024-09-13T18:21:02.937231+0000","last_undegraded":"2024-09-13T18:21:02.937231+0000","last_fullsized":"2024-09-13T18:21:02.937231+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5052,"log_dups_size":0,"ondisk_log_size":5052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20895,"num_write_kb":464282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3,6],"acting":[4,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'18","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.301541+0000","last_change":"2024-09-13T17:50:34.585741+0000","last_active":"2024-09-13T18:21:13.301541+0000","last_peered":"2024-09-13T18:21:13.301541+0000","last_clean":"2024-09-13T18:21:13.301541+0000","last_became_active":"2024-09-13T17:50:34.585464+0000","last_became_peered":"2024-09-13T17:50:34.585464+0000","last_unstale":"2024-09-13T18:21:13.301541+0000","last_undegraded":"2024-09-13T18:21:13.301541+0000","last_fullsized":"2024-09-13T18:21:13.301541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'27","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:36.784439+0000","last_change":"2024-09-13T18:21:36.784589+0000","last_active":"2024-09-13T18:21:36.784439+0000","last_peered":"2024-09-13T18:21:36.784439+0000","last_clean":"2024-09-13T18:21:36.784439+0000","last_became_active":"2024-09-13T17:50:34.200936+0000","last_became_peered":"2024-09-13T17:50:34.200936+0000","last_unstale":"2024-09-13T18:21:36.784439+0000","last_undegraded":"2024-09-13T18:21:36.784439+0000","last_fullsized":"2024-09-13T18:21:36.784439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'27","last_scrub_stamp":"2024-09-13T18:21:36.784405+0000","last_deep_scrub":"27'27","last_deep_scrub_stamp":"2024-09-13T18:21:36.784405+0000","last_clean_scrub_stamp":"2024-09-13T18:21:36.784405+0000","objects_scrubbed":8,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:11.675692+0000","scrub_duration":0.012535223,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'6502","reported_seq":6529,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.953302+0000","last_change":"2024-09-13T17:50:38.615229+0000","last_active":"2024-09-13T18:21:02.953302+0000","last_peered":"2024-09-13T18:21:02.953302+0000","last_clean":"2024-09-13T18:21:02.953302+0000","last_became_active":"2024-09-13T17:50:38.614739+0000","last_became_peered":"2024-09-13T17:50:38.614739+0000","last_unstale":"2024-09-13T18:21:02.953302+0000","last_undegraded":"2024-09-13T18:21:02.953302+0000","last_fullsized":"2024-09-13T18:21:02.953302+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6502,"log_dups_size":0,"ondisk_log_size":6502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27696,"num_write_kb":544723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'20","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:34.335658+0000","last_change":"2024-09-13T18:21:34.335970+0000","last_active":"2024-09-13T18:21:34.335658+0000","last_peered":"2024-09-13T18:21:34.335658+0000","last_clean":"2024-09-13T18:21:34.335658+0000","last_became_active":"2024-09-13T17:50:34.588565+0000","last_became_peered":"2024-09-13T17:50:34.588565+0000","last_unstale":"2024-09-13T18:21:34.335658+0000","last_undegraded":"2024-09-13T18:21:34.335658+0000","last_fullsized":"2024-09-13T18:21:34.335658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-13T18:21:34.335555+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-13T18:21:34.335555+0000","last_clean_scrub_stamp":"2024-09-13T18:21:34.335555+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:28.955486+0000","scrub_duration":0.0039531770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'9","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:26.414088+0000","last_change":"2024-09-13T18:21:26.414323+0000","last_active":"2024-09-13T18:21:26.414088+0000","last_peered":"2024-09-13T18:21:26.414088+0000","last_clean":"2024-09-13T18:21:26.414088+0000","last_became_active":"2024-09-13T17:50:34.197839+0000","last_became_peered":"2024-09-13T17:50:34.197839+0000","last_unstale":"2024-09-13T18:21:26.414088+0000","last_undegraded":"2024-09-13T18:21:26.414088+0000","last_fullsized":"2024-09-13T18:21:26.414088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-09-13T18:21:26.414037+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-09-13T18:21:26.414037+0000","last_clean_scrub_stamp":"2024-09-13T18:21:26.414037+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:22.483872+0000","scrub_duration":0.0068422659999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'6326","reported_seq":6353,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.544581+0000","last_change":"2024-09-13T17:50:38.611080+0000","last_active":"2024-09-13T18:21:02.544581+0000","last_peered":"2024-09-13T18:21:02.544581+0000","last_clean":"2024-09-13T18:21:02.544581+0000","last_became_active":"2024-09-13T17:50:38.610688+0000","last_became_peered":"2024-09-13T17:50:38.610688+0000","last_unstale":"2024-09-13T18:21:02.544581+0000","last_undegraded":"2024-09-13T18:21:02.544581+0000","last_fullsized":"2024-09-13T18:21:02.544581+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6326,"log_dups_size":0,"ondisk_log_size":6326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27199,"num_write_kb":538670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'29","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317672+0000","last_change":"2024-09-13T17:50:34.594763+0000","last_active":"2024-09-13T18:21:13.317672+0000","last_peered":"2024-09-13T18:21:13.317672+0000","last_clean":"2024-09-13T18:21:13.317672+0000","last_became_active":"2024-09-13T17:50:34.594471+0000","last_became_peered":"2024-09-13T17:50:34.594471+0000","last_unstale":"2024-09-13T18:21:13.317672+0000","last_undegraded":"2024-09-13T18:21:13.317672+0000","last_fullsized":"2024-09-13T18:21:13.317672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'7","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:37.911671+0000","last_change":"2024-09-13T18:21:37.911888+0000","last_active":"2024-09-13T18:21:37.911671+0000","last_peered":"2024-09-13T18:21:37.911671+0000","last_clean":"2024-09-13T18:21:37.911671+0000","last_became_active":"2024-09-13T17:50:34.208817+0000","last_became_peered":"2024-09-13T17:50:34.208817+0000","last_unstale":"2024-09-13T18:21:37.911671+0000","last_undegraded":"2024-09-13T18:21:37.911671+0000","last_fullsized":"2024-09-13T18:21:37.911671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-13T18:21:37.911585+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-13T18:21:37.911585+0000","last_clean_scrub_stamp":"2024-09-13T18:21:37.911585+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:43:22.483981+0000","scrub_duration":0.0050921259999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'5788","reported_seq":5816,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:17.228222+0000","last_change":"2024-09-13T17:50:38.609461+0000","last_active":"2024-09-13T18:21:17.228222+0000","last_peered":"2024-09-13T18:21:17.228222+0000","last_clean":"2024-09-13T18:21:17.228222+0000","last_became_active":"2024-09-13T17:50:38.609128+0000","last_became_peered":"2024-09-13T17:50:38.609128+0000","last_unstale":"2024-09-13T18:21:17.228222+0000","last_undegraded":"2024-09-13T18:21:17.228222+0000","last_fullsized":"2024-09-13T18:21:17.228222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5788,"log_dups_size":0,"ondisk_log_size":5788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24140,"num_write_kb":538739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'24","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.311423+0000","last_change":"2024-09-13T17:50:34.592970+0000","last_active":"2024-09-13T18:21:13.311423+0000","last_peered":"2024-09-13T18:21:13.311423+0000","last_clean":"2024-09-13T18:21:13.311423+0000","last_became_active":"2024-09-13T17:50:34.592669+0000","last_became_peered":"2024-09-13T17:50:34.592669+0000","last_unstale":"2024-09-13T18:21:13.311423+0000","last_undegraded":"2024-09-13T18:21:13.311423+0000","last_fullsized":"2024-09-13T18:21:13.311423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'19","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.241499+0000","last_change":"2024-09-13T18:21:28.241791+0000","last_active":"2024-09-13T18:21:28.241499+0000","last_peered":"2024-09-13T18:21:28.241499+0000","last_clean":"2024-09-13T18:21:28.241499+0000","last_became_active":"2024-09-13T17:50:34.200529+0000","last_became_peered":"2024-09-13T17:50:34.200529+0000","last_unstale":"2024-09-13T18:21:28.241499+0000","last_undegraded":"2024-09-13T18:21:28.241499+0000","last_fullsized":"2024-09-13T18:21:28.241499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-13T18:21:28.241430+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-13T18:21:28.241430+0000","last_clean_scrub_stamp":"2024-09-13T18:21:28.241430+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:08:39.927576+0000","scrub_duration":0.00889181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'5464","reported_seq":5491,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.614309+0000","last_change":"2024-09-13T17:50:38.606287+0000","last_active":"2024-09-13T18:21:02.614309+0000","last_peered":"2024-09-13T18:21:02.614309+0000","last_clean":"2024-09-13T18:21:02.614309+0000","last_became_active":"2024-09-13T17:50:38.605826+0000","last_became_peered":"2024-09-13T17:50:38.605826+0000","last_unstale":"2024-09-13T18:21:02.614309+0000","last_undegraded":"2024-09-13T18:21:02.614309+0000","last_fullsized":"2024-09-13T18:21:02.614309+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5464,"log_dups_size":0,"ondisk_log_size":5464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23044,"num_write_kb":472739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'18","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310963+0000","last_change":"2024-09-13T17:50:34.591579+0000","last_active":"2024-09-13T18:21:13.310963+0000","last_peered":"2024-09-13T18:21:13.310963+0000","last_clean":"2024-09-13T18:21:13.310963+0000","last_became_active":"2024-09-13T17:50:34.586165+0000","last_became_peered":"2024-09-13T17:50:34.586165+0000","last_unstale":"2024-09-13T18:21:13.310963+0000","last_undegraded":"2024-09-13T18:21:13.310963+0000","last_fullsized":"2024-09-13T18:21:13.310963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'421","reported_seq":463,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:31.308661+0000","last_change":"2024-09-13T18:21:31.308816+0000","last_active":"2024-09-13T18:21:31.308661+0000","last_peered":"2024-09-13T18:21:31.308661+0000","last_clean":"2024-09-13T18:21:31.308661+0000","last_became_active":"2024-09-13T17:50:34.202948+0000","last_became_peered":"2024-09-13T17:50:34.202948+0000","last_unstale":"2024-09-13T18:21:31.308661+0000","last_undegraded":"2024-09-13T18:21:31.308661+0000","last_fullsized":"2024-09-13T18:21:31.308661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'421","last_scrub_stamp":"2024-09-13T18:21:31.308629+0000","last_deep_scrub":"27'421","last_deep_scrub_stamp":"2024-09-13T18:21:31.308629+0000","last_clean_scrub_stamp":"2024-09-13T18:21:31.308629+0000","objects_scrubbed":5,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:55.983699+0000","scrub_duration":0.024451271,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":4324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'5887","reported_seq":5915,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:07.256416+0000","last_change":"2024-09-13T17:50:38.610597+0000","last_active":"2024-09-13T18:21:07.256416+0000","last_peered":"2024-09-13T18:21:07.256416+0000","last_clean":"2024-09-13T18:21:07.256416+0000","last_became_active":"2024-09-13T17:50:38.610055+0000","last_became_peered":"2024-09-13T17:50:38.610055+0000","last_unstale":"2024-09-13T18:21:07.256416+0000","last_undegraded":"2024-09-13T18:21:07.256416+0000","last_fullsized":"2024-09-13T18:21:07.256416+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5887,"log_dups_size":0,"ondisk_log_size":5887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24664,"num_write_kb":495799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'21","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:39.354348+0000","last_change":"2024-09-13T18:21:39.354570+0000","last_active":"2024-09-13T18:21:39.354348+0000","last_peered":"2024-09-13T18:21:39.354348+0000","last_clean":"2024-09-13T18:21:39.354348+0000","last_became_active":"2024-09-13T17:50:34.593454+0000","last_became_peered":"2024-09-13T17:50:34.593454+0000","last_unstale":"2024-09-13T18:21:39.354348+0000","last_undegraded":"2024-09-13T18:21:39.354348+0000","last_fullsized":"2024-09-13T18:21:39.354348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-09-13T18:21:39.354296+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-09-13T18:21:39.354296+0000","last_clean_scrub_stamp":"2024-09-13T18:21:39.354296+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:10:26.576365+0000","scrub_duration":0.003405356,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'67","reported_seq":97,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:03.750240+0000","last_change":"2024-09-13T17:50:34.207936+0000","last_active":"2024-09-13T18:21:03.750240+0000","last_peered":"2024-09-13T18:21:03.750240+0000","last_clean":"2024-09-13T18:21:03.750240+0000","last_became_active":"2024-09-13T17:50:34.207605+0000","last_became_peered":"2024-09-13T17:50:34.207605+0000","last_unstale":"2024-09-13T18:21:03.750240+0000","last_undegraded":"2024-09-13T18:21:03.750240+0000","last_fullsized":"2024-09-13T18:21:03.750240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:33.165496+0000","last_clean_scrub_stamp":"2024-09-13T17:50:33.165496+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'5937","reported_seq":5965,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.965445+0000","last_change":"2024-09-13T17:50:38.616297+0000","last_active":"2024-09-13T18:21:02.965445+0000","last_peered":"2024-09-13T18:21:02.965445+0000","last_clean":"2024-09-13T18:21:02.965445+0000","last_became_active":"2024-09-13T17:50:38.614165+0000","last_became_peered":"2024-09-13T17:50:38.614165+0000","last_unstale":"2024-09-13T18:21:02.965445+0000","last_undegraded":"2024-09-13T18:21:02.965445+0000","last_fullsized":"2024-09-13T18:21:02.965445+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5937,"log_dups_size":0,"ondisk_log_size":5937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25111,"num_write_kb":509795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'25","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.311206+0000","last_change":"2024-09-13T17:50:34.583001+0000","last_active":"2024-09-13T18:21:13.311206+0000","last_peered":"2024-09-13T18:21:13.311206+0000","last_clean":"2024-09-13T18:21:13.311206+0000","last_became_active":"2024-09-13T17:50:34.582529+0000","last_became_peered":"2024-09-13T17:50:34.582529+0000","last_unstale":"2024-09-13T18:21:13.311206+0000","last_undegraded":"2024-09-13T18:21:13.311206+0000","last_fullsized":"2024-09-13T18:21:13.311206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'6","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:31.812777+0000","last_change":"2024-09-13T18:21:31.812971+0000","last_active":"2024-09-13T18:21:31.812777+0000","last_peered":"2024-09-13T18:21:31.812777+0000","last_clean":"2024-09-13T18:21:31.812777+0000","last_became_active":"2024-09-13T17:50:34.203360+0000","last_became_peered":"2024-09-13T17:50:34.203360+0000","last_unstale":"2024-09-13T18:21:31.812777+0000","last_undegraded":"2024-09-13T18:21:31.812777+0000","last_fullsized":"2024-09-13T18:21:31.812777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-09-13T18:21:31.812723+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-09-13T18:21:31.812723+0000","last_clean_scrub_stamp":"2024-09-13T18:21:31.812723+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:43:43.954185+0000","scrub_duration":0.0059683940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'5934","reported_seq":5962,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.811199+0000","last_change":"2024-09-13T17:50:38.611036+0000","last_active":"2024-09-13T18:21:02.811199+0000","last_peered":"2024-09-13T18:21:02.811199+0000","last_clean":"2024-09-13T18:21:02.811199+0000","last_became_active":"2024-09-13T17:50:38.610692+0000","last_became_peered":"2024-09-13T17:50:38.610692+0000","last_unstale":"2024-09-13T18:21:02.811199+0000","last_undegraded":"2024-09-13T18:21:02.811199+0000","last_fullsized":"2024-09-13T18:21:02.811199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5934,"log_dups_size":0,"ondisk_log_size":5934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25100,"num_write_kb":497535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'23","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.311843+0000","last_change":"2024-09-13T17:51:16.365295+0000","last_active":"2024-09-13T18:21:13.311843+0000","last_peered":"2024-09-13T18:21:13.311843+0000","last_clean":"2024-09-13T18:21:13.311843+0000","last_became_active":"2024-09-13T17:51:16.364437+0000","last_became_peered":"2024-09-13T17:51:16.364437+0000","last_unstale":"2024-09-13T18:21:13.311843+0000","last_undegraded":"2024-09-13T18:21:13.311843+0000","last_fullsized":"2024-09-13T18:21:13.311843+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"27'11","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:33.306992+0000","last_change":"2024-09-13T18:21:33.307232+0000","last_active":"2024-09-13T18:21:33.306992+0000","last_peered":"2024-09-13T18:21:33.306992+0000","last_clean":"2024-09-13T18:21:33.306992+0000","last_became_active":"2024-09-13T17:50:34.204003+0000","last_became_peered":"2024-09-13T17:50:34.204003+0000","last_unstale":"2024-09-13T18:21:33.306992+0000","last_undegraded":"2024-09-13T18:21:33.306992+0000","last_fullsized":"2024-09-13T18:21:33.306992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-09-13T18:21:33.306942+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-09-13T18:21:33.306942+0000","last_clean_scrub_stamp":"2024-09-13T18:21:33.306942+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:13.945111+0000","scrub_duration":0.0066162260000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'5363","reported_seq":5391,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.822018+0000","last_change":"2024-09-13T17:50:38.606144+0000","last_active":"2024-09-13T18:21:02.822018+0000","last_peered":"2024-09-13T18:21:02.822018+0000","last_clean":"2024-09-13T18:21:02.822018+0000","last_became_active":"2024-09-13T17:50:38.605646+0000","last_became_peered":"2024-09-13T17:50:38.605646+0000","last_unstale":"2024-09-13T18:21:02.822018+0000","last_undegraded":"2024-09-13T18:21:02.822018+0000","last_fullsized":"2024-09-13T18:21:02.822018+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5363,"log_dups_size":0,"ondisk_log_size":5363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22831,"num_write_kb":508903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'6067","reported_seq":6097,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.920872+0000","last_change":"2024-09-13T17:50:38.618855+0000","last_active":"2024-09-13T18:21:02.920872+0000","last_peered":"2024-09-13T18:21:02.920872+0000","last_clean":"2024-09-13T18:21:02.920872+0000","last_became_active":"2024-09-13T17:50:38.618449+0000","last_became_peered":"2024-09-13T17:50:38.618449+0000","last_unstale":"2024-09-13T18:21:02.920872+0000","last_undegraded":"2024-09-13T18:21:02.920872+0000","last_fullsized":"2024-09-13T18:21:02.920872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6067,"log_dups_size":0,"ondisk_log_size":6067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25532,"num_write_kb":568825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'21","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.307712+0000","last_change":"2024-09-13T17:50:34.590079+0000","last_active":"2024-09-13T18:21:13.307712+0000","last_peered":"2024-09-13T18:21:13.307712+0000","last_clean":"2024-09-13T18:21:13.307712+0000","last_became_active":"2024-09-13T17:50:34.589396+0000","last_became_peered":"2024-09-13T17:50:34.589396+0000","last_unstale":"2024-09-13T18:21:13.307712+0000","last_undegraded":"2024-09-13T18:21:13.307712+0000","last_fullsized":"2024-09-13T18:21:13.307712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:32.850585+0000","last_change":"2024-09-13T18:21:32.850895+0000","last_active":"2024-09-13T18:21:32.850585+0000","last_peered":"2024-09-13T18:21:32.850585+0000","last_clean":"2024-09-13T18:21:32.850585+0000","last_became_active":"2024-09-13T17:50:34.206308+0000","last_became_peered":"2024-09-13T17:50:34.206308+0000","last_unstale":"2024-09-13T18:21:32.850585+0000","last_undegraded":"2024-09-13T18:21:32.850585+0000","last_fullsized":"2024-09-13T18:21:32.850585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-09-13T18:21:32.850533+0000","last_deep_scrub":"25'1","last_deep_scrub_stamp":"2024-09-13T18:21:32.850533+0000","last_clean_scrub_stamp":"2024-09-13T18:21:32.850533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:59:21.637907+0000","scrub_duration":0.0034298850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'5881","reported_seq":5910,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.928908+0000","last_change":"2024-09-13T17:50:38.618131+0000","last_active":"2024-09-13T18:21:02.928908+0000","last_peered":"2024-09-13T18:21:02.928908+0000","last_clean":"2024-09-13T18:21:02.928908+0000","last_became_active":"2024-09-13T17:50:38.617611+0000","last_became_peered":"2024-09-13T17:50:38.617611+0000","last_unstale":"2024-09-13T18:21:02.928908+0000","last_undegraded":"2024-09-13T18:21:02.928908+0000","last_fullsized":"2024-09-13T18:21:02.928908+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5881,"log_dups_size":0,"ondisk_log_size":5881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25019,"num_write_kb":530139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'27","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:34.190044+0000","last_change":"2024-09-13T18:21:34.190222+0000","last_active":"2024-09-13T18:21:34.190044+0000","last_peered":"2024-09-13T18:21:34.190044+0000","last_clean":"2024-09-13T18:21:34.190044+0000","last_became_active":"2024-09-13T17:50:34.581708+0000","last_became_peered":"2024-09-13T17:50:34.581708+0000","last_unstale":"2024-09-13T18:21:34.190044+0000","last_undegraded":"2024-09-13T18:21:34.190044+0000","last_fullsized":"2024-09-13T18:21:34.190044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'27","last_scrub_stamp":"2024-09-13T18:21:34.189993+0000","last_deep_scrub":"27'27","last_deep_scrub_stamp":"2024-09-13T18:21:34.189993+0000","last_clean_scrub_stamp":"2024-09-13T18:21:34.189993+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:32:35.944129+0000","scrub_duration":0.0029610209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'5","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:42.211549+0000","last_change":"2024-09-13T18:21:42.211743+0000","last_active":"2024-09-13T18:21:42.211549+0000","last_peered":"2024-09-13T18:21:42.211549+0000","last_clean":"2024-09-13T18:21:42.211549+0000","last_became_active":"2024-09-13T17:50:34.205202+0000","last_became_peered":"2024-09-13T17:50:34.205202+0000","last_unstale":"2024-09-13T18:21:42.211549+0000","last_undegraded":"2024-09-13T18:21:42.211549+0000","last_fullsized":"2024-09-13T18:21:42.211549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-09-13T18:21:42.211500+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-09-13T18:21:42.211500+0000","last_clean_scrub_stamp":"2024-09-13T18:21:42.211500+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:47:56.465375+0000","scrub_duration":0.0062275539999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'7","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:30.324177+0000","last_change":"2024-09-13T18:21:30.324321+0000","last_active":"2024-09-13T18:21:30.324177+0000","last_peered":"2024-09-13T18:21:30.324177+0000","last_clean":"2024-09-13T18:21:30.324177+0000","last_became_active":"2024-09-13T17:50:34.202163+0000","last_became_peered":"2024-09-13T17:50:34.202163+0000","last_unstale":"2024-09-13T18:21:30.324177+0000","last_undegraded":"2024-09-13T18:21:30.324177+0000","last_fullsized":"2024-09-13T18:21:30.324177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-13T18:21:30.324141+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-13T18:21:30.324141+0000","last_clean_scrub_stamp":"2024-09-13T18:21:30.324141+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:27.051230+0000","scrub_duration":0.00672063,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'18","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312775+0000","last_change":"2024-09-13T17:50:34.590947+0000","last_active":"2024-09-13T18:21:13.312775+0000","last_peered":"2024-09-13T18:21:13.312775+0000","last_clean":"2024-09-13T18:21:13.312775+0000","last_became_active":"2024-09-13T17:50:34.590497+0000","last_became_peered":"2024-09-13T17:50:34.590497+0000","last_unstale":"2024-09-13T18:21:13.312775+0000","last_undegraded":"2024-09-13T18:21:13.312775+0000","last_fullsized":"2024-09-13T18:21:13.312775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'4651","reported_seq":4678,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.868570+0000","last_change":"2024-09-13T17:50:38.618518+0000","last_active":"2024-09-13T18:21:02.868570+0000","last_peered":"2024-09-13T18:21:02.868570+0000","last_clean":"2024-09-13T18:21:02.868570+0000","last_became_active":"2024-09-13T17:50:38.618108+0000","last_became_peered":"2024-09-13T17:50:38.618108+0000","last_unstale":"2024-09-13T18:21:02.868570+0000","last_undegraded":"2024-09-13T18:21:02.868570+0000","last_fullsized":"2024-09-13T18:21:02.868570+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4651,"log_dups_size":0,"ondisk_log_size":4651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19488,"num_write_kb":413684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'692","reported_seq":732,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:38.185928+0000","last_change":"2024-09-13T18:21:38.186111+0000","last_active":"2024-09-13T18:21:38.185928+0000","last_peered":"2024-09-13T18:21:38.185928+0000","last_clean":"2024-09-13T18:21:38.185928+0000","last_became_active":"2024-09-13T17:50:34.218087+0000","last_became_peered":"2024-09-13T17:50:34.218087+0000","last_unstale":"2024-09-13T18:21:38.185928+0000","last_undegraded":"2024-09-13T18:21:38.185928+0000","last_fullsized":"2024-09-13T18:21:38.185928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'692","last_scrub_stamp":"2024-09-13T18:21:38.185889+0000","last_deep_scrub":"27'692","last_deep_scrub_stamp":"2024-09-13T18:21:38.185889+0000","last_clean_scrub_stamp":"2024-09-13T18:21:38.185889+0000","objects_scrubbed":3,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:16:47.594855+0000","scrub_duration":0.021444375000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195461,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'20","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:40.309829+0000","last_change":"2024-09-13T18:21:40.310070+0000","last_active":"2024-09-13T18:21:40.309829+0000","last_peered":"2024-09-13T18:21:40.309829+0000","last_clean":"2024-09-13T18:21:40.309829+0000","last_became_active":"2024-09-13T17:50:34.583283+0000","last_became_peered":"2024-09-13T17:50:34.583283+0000","last_unstale":"2024-09-13T18:21:40.309829+0000","last_undegraded":"2024-09-13T18:21:40.309829+0000","last_fullsized":"2024-09-13T18:21:40.309829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-13T18:21:40.309774+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-13T18:21:40.309774+0000","last_clean_scrub_stamp":"2024-09-13T18:21:40.309774+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:45:15.638497+0000","scrub_duration":0.0033293300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'5883","reported_seq":5910,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.843115+0000","last_change":"2024-09-13T17:50:38.611987+0000","last_active":"2024-09-13T18:21:02.843115+0000","last_peered":"2024-09-13T18:21:02.843115+0000","last_clean":"2024-09-13T18:21:02.843115+0000","last_became_active":"2024-09-13T17:50:38.611636+0000","last_became_peered":"2024-09-13T17:50:38.611636+0000","last_unstale":"2024-09-13T18:21:02.843115+0000","last_undegraded":"2024-09-13T18:21:02.843115+0000","last_fullsized":"2024-09-13T18:21:02.843115+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5883,"log_dups_size":0,"ondisk_log_size":5883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24816,"num_write_kb":545450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'25","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.302465+0000","last_change":"2024-09-13T17:50:34.588927+0000","last_active":"2024-09-13T18:21:13.302465+0000","last_peered":"2024-09-13T18:21:13.302465+0000","last_clean":"2024-09-13T18:21:13.302465+0000","last_became_active":"2024-09-13T17:50:34.588651+0000","last_became_peered":"2024-09-13T17:50:34.588651+0000","last_unstale":"2024-09-13T18:21:13.302465+0000","last_undegraded":"2024-09-13T18:21:13.302465+0000","last_fullsized":"2024-09-13T18:21:13.302465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'24","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310507+0000","last_change":"2024-09-13T17:50:34.583795+0000","last_active":"2024-09-13T18:21:13.310507+0000","last_peered":"2024-09-13T18:21:13.310507+0000","last_clean":"2024-09-13T18:21:13.310507+0000","last_became_active":"2024-09-13T17:50:34.583493+0000","last_became_peered":"2024-09-13T17:50:34.583493+0000","last_unstale":"2024-09-13T18:21:13.310507+0000","last_undegraded":"2024-09-13T18:21:13.310507+0000","last_fullsized":"2024-09-13T18:21:13.310507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'25","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.318636+0000","last_change":"2024-09-13T17:50:34.587624+0000","last_active":"2024-09-13T18:21:13.318636+0000","last_peered":"2024-09-13T18:21:13.318636+0000","last_clean":"2024-09-13T18:21:13.318636+0000","last_became_active":"2024-09-13T17:50:34.587119+0000","last_became_peered":"2024-09-13T17:50:34.587119+0000","last_unstale":"2024-09-13T18:21:13.318636+0000","last_undegraded":"2024-09-13T18:21:13.318636+0000","last_fullsized":"2024-09-13T18:21:13.318636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'17","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.308749+0000","last_change":"2024-09-13T17:50:34.581923+0000","last_active":"2024-09-13T18:21:13.308749+0000","last_peered":"2024-09-13T18:21:13.308749+0000","last_clean":"2024-09-13T18:21:13.308749+0000","last_became_active":"2024-09-13T17:50:34.581628+0000","last_became_peered":"2024-09-13T17:50:34.581628+0000","last_unstale":"2024-09-13T18:21:13.308749+0000","last_undegraded":"2024-09-13T18:21:13.308749+0000","last_fullsized":"2024-09-13T18:21:13.308749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'14","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312805+0000","last_change":"2024-09-13T17:50:34.595037+0000","last_active":"2024-09-13T18:21:13.312805+0000","last_peered":"2024-09-13T18:21:13.312805+0000","last_clean":"2024-09-13T18:21:13.312805+0000","last_became_active":"2024-09-13T17:50:34.594857+0000","last_became_peered":"2024-09-13T17:50:34.594857+0000","last_unstale":"2024-09-13T18:21:13.312805+0000","last_undegraded":"2024-09-13T18:21:13.312805+0000","last_fullsized":"2024-09-13T18:21:13.312805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'15","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.298383+0000","last_change":"2024-09-13T17:50:34.589651+0000","last_active":"2024-09-13T18:21:13.298383+0000","last_peered":"2024-09-13T18:21:13.298383+0000","last_clean":"2024-09-13T18:21:13.298383+0000","last_became_active":"2024-09-13T17:50:34.589354+0000","last_became_peered":"2024-09-13T17:50:34.589354+0000","last_unstale":"2024-09-13T18:21:13.298383+0000","last_undegraded":"2024-09-13T18:21:13.298383+0000","last_fullsized":"2024-09-13T18:21:13.298383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'24","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:38.403817+0000","last_change":"2024-09-13T18:21:38.404032+0000","last_active":"2024-09-13T18:21:38.403817+0000","last_peered":"2024-09-13T18:21:38.403817+0000","last_clean":"2024-09-13T18:21:38.403817+0000","last_became_active":"2024-09-13T17:50:34.597621+0000","last_became_peered":"2024-09-13T17:50:34.597621+0000","last_unstale":"2024-09-13T18:21:38.403817+0000","last_undegraded":"2024-09-13T18:21:38.403817+0000","last_fullsized":"2024-09-13T18:21:38.403817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-09-13T18:21:38.403767+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-09-13T18:21:38.403767+0000","last_clean_scrub_stamp":"2024-09-13T18:21:38.403767+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:33:01.593056+0000","scrub_duration":0.0041273439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'15","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:32.155724+0000","last_change":"2024-09-13T18:21:32.155926+0000","last_active":"2024-09-13T18:21:32.155724+0000","last_peered":"2024-09-13T18:21:32.155724+0000","last_clean":"2024-09-13T18:21:32.155724+0000","last_became_active":"2024-09-13T17:50:34.589626+0000","last_became_peered":"2024-09-13T17:50:34.589626+0000","last_unstale":"2024-09-13T18:21:32.155724+0000","last_undegraded":"2024-09-13T18:21:32.155724+0000","last_fullsized":"2024-09-13T18:21:32.155724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-09-13T18:21:32.155676+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-09-13T18:21:32.155676+0000","last_clean_scrub_stamp":"2024-09-13T18:21:32.155676+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:03:35.333502+0000","scrub_duration":0.0028998560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'16","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.308935+0000","last_change":"2024-09-13T17:50:34.590046+0000","last_active":"2024-09-13T18:21:13.308935+0000","last_peered":"2024-09-13T18:21:13.308935+0000","last_clean":"2024-09-13T18:21:13.308935+0000","last_became_active":"2024-09-13T17:50:34.589757+0000","last_became_peered":"2024-09-13T17:50:34.589757+0000","last_unstale":"2024-09-13T18:21:13.308935+0000","last_undegraded":"2024-09-13T18:21:13.308935+0000","last_fullsized":"2024-09-13T18:21:13.308935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'23","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.130076+0000","last_change":"2024-09-13T18:21:27.130283+0000","last_active":"2024-09-13T18:21:27.130076+0000","last_peered":"2024-09-13T18:21:27.130076+0000","last_clean":"2024-09-13T18:21:27.130076+0000","last_became_active":"2024-09-13T17:50:34.595242+0000","last_became_peered":"2024-09-13T17:50:34.595242+0000","last_unstale":"2024-09-13T18:21:27.130076+0000","last_undegraded":"2024-09-13T18:21:27.130076+0000","last_fullsized":"2024-09-13T18:21:27.130076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-13T18:21:27.130025+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-13T18:21:27.130025+0000","last_clean_scrub_stamp":"2024-09-13T18:21:27.130025+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:06.896321+0000","scrub_duration":0.003821638,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'21","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.318696+0000","last_change":"2024-09-13T17:50:34.593183+0000","last_active":"2024-09-13T18:21:13.318696+0000","last_peered":"2024-09-13T18:21:13.318696+0000","last_clean":"2024-09-13T18:21:13.318696+0000","last_became_active":"2024-09-13T17:50:34.592893+0000","last_became_peered":"2024-09-13T17:50:34.592893+0000","last_unstale":"2024-09-13T18:21:13.318696+0000","last_undegraded":"2024-09-13T18:21:13.318696+0000","last_fullsized":"2024-09-13T18:21:13.318696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'29","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.311937+0000","last_change":"2024-09-13T17:50:34.594679+0000","last_active":"2024-09-13T18:21:13.311937+0000","last_peered":"2024-09-13T18:21:13.311937+0000","last_clean":"2024-09-13T18:21:13.311937+0000","last_became_active":"2024-09-13T17:50:34.593750+0000","last_became_peered":"2024-09-13T17:50:34.593750+0000","last_unstale":"2024-09-13T18:21:13.311937+0000","last_undegraded":"2024-09-13T18:21:13.311937+0000","last_fullsized":"2024-09-13T18:21:13.311937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'19","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:37.375112+0000","last_change":"2024-09-13T18:21:37.375333+0000","last_active":"2024-09-13T18:21:37.375112+0000","last_peered":"2024-09-13T18:21:37.375112+0000","last_clean":"2024-09-13T18:21:37.375112+0000","last_became_active":"2024-09-13T17:50:34.598433+0000","last_became_peered":"2024-09-13T17:50:34.598433+0000","last_unstale":"2024-09-13T18:21:37.375112+0000","last_undegraded":"2024-09-13T18:21:37.375112+0000","last_fullsized":"2024-09-13T18:21:37.375112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-13T18:21:37.375056+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-13T18:21:37.375056+0000","last_clean_scrub_stamp":"2024-09-13T18:21:37.375056+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:02.999621+0000","scrub_duration":0.003734036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'22","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:36.325407+0000","last_change":"2024-09-13T18:21:36.325615+0000","last_active":"2024-09-13T18:21:36.325407+0000","last_peered":"2024-09-13T18:21:36.325407+0000","last_clean":"2024-09-13T18:21:36.325407+0000","last_became_active":"2024-09-13T17:50:34.591286+0000","last_became_peered":"2024-09-13T17:50:34.591286+0000","last_unstale":"2024-09-13T18:21:36.325407+0000","last_undegraded":"2024-09-13T18:21:36.325407+0000","last_fullsized":"2024-09-13T18:21:36.325407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-09-13T18:21:36.325355+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-09-13T18:21:36.325355+0000","last_clean_scrub_stamp":"2024-09-13T18:21:36.325355+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:12:08.676171+0000","scrub_duration":0.0029670930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'28","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.300986+0000","last_change":"2024-09-13T17:50:34.587774+0000","last_active":"2024-09-13T18:21:13.300986+0000","last_peered":"2024-09-13T18:21:13.300986+0000","last_clean":"2024-09-13T18:21:13.300986+0000","last_became_active":"2024-09-13T17:50:34.587506+0000","last_became_peered":"2024-09-13T17:50:34.587506+0000","last_unstale":"2024-09-13T18:21:13.300986+0000","last_undegraded":"2024-09-13T18:21:13.300986+0000","last_fullsized":"2024-09-13T18:21:13.300986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'22","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.319197+0000","last_change":"2024-09-13T17:50:34.596100+0000","last_active":"2024-09-13T18:21:13.319197+0000","last_peered":"2024-09-13T18:21:13.319197+0000","last_clean":"2024-09-13T18:21:13.319197+0000","last_became_active":"2024-09-13T17:50:34.595798+0000","last_became_peered":"2024-09-13T17:50:34.595798+0000","last_unstale":"2024-09-13T18:21:13.319197+0000","last_undegraded":"2024-09-13T18:21:13.319197+0000","last_fullsized":"2024-09-13T18:21:13.319197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'21","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:25.753731+0000","last_change":"2024-09-13T18:21:25.753960+0000","last_active":"2024-09-13T18:21:25.753731+0000","last_peered":"2024-09-13T18:21:25.753731+0000","last_clean":"2024-09-13T18:21:25.753731+0000","last_became_active":"2024-09-13T17:50:34.584516+0000","last_became_peered":"2024-09-13T17:50:34.584516+0000","last_unstale":"2024-09-13T18:21:25.753731+0000","last_undegraded":"2024-09-13T18:21:25.753731+0000","last_fullsized":"2024-09-13T18:21:25.753731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-09-13T18:21:25.753680+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-09-13T18:21:25.753680+0000","last_clean_scrub_stamp":"2024-09-13T18:21:25.753680+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:59.108188+0000","scrub_duration":0.0032900099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'23","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.293046+0000","last_change":"2024-09-13T17:50:34.598886+0000","last_active":"2024-09-13T18:21:13.293046+0000","last_peered":"2024-09-13T18:21:13.293046+0000","last_clean":"2024-09-13T18:21:13.293046+0000","last_became_active":"2024-09-13T17:50:34.598648+0000","last_became_peered":"2024-09-13T17:50:34.598648+0000","last_unstale":"2024-09-13T18:21:13.293046+0000","last_undegraded":"2024-09-13T18:21:13.293046+0000","last_fullsized":"2024-09-13T18:21:13.293046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'24","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:23.200378+0000","last_change":"2024-09-13T18:21:23.200494+0000","last_active":"2024-09-13T18:21:23.200378+0000","last_peered":"2024-09-13T18:21:23.200378+0000","last_clean":"2024-09-13T18:21:23.200378+0000","last_became_active":"2024-09-13T17:50:34.581699+0000","last_became_peered":"2024-09-13T17:50:34.581699+0000","last_unstale":"2024-09-13T18:21:23.200378+0000","last_undegraded":"2024-09-13T18:21:23.200378+0000","last_fullsized":"2024-09-13T18:21:23.200378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-09-13T18:21:23.200343+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-09-13T18:21:23.200343+0000","last_clean_scrub_stamp":"2024-09-13T18:21:23.200343+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:37:50.717684+0000","scrub_duration":0.002562838,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'23","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:25.110456+0000","last_change":"2024-09-13T18:21:25.110698+0000","last_active":"2024-09-13T18:21:25.110456+0000","last_peered":"2024-09-13T18:21:25.110456+0000","last_clean":"2024-09-13T18:21:25.110456+0000","last_became_active":"2024-09-13T17:50:34.587217+0000","last_became_peered":"2024-09-13T17:50:34.587217+0000","last_unstale":"2024-09-13T18:21:25.110456+0000","last_undegraded":"2024-09-13T18:21:25.110456+0000","last_fullsized":"2024-09-13T18:21:25.110456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-13T18:21:25.110401+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-13T18:21:25.110401+0000","last_clean_scrub_stamp":"2024-09-13T18:21:25.110401+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:08.271399+0000","scrub_duration":0.0034179560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'18","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.319416+0000","last_change":"2024-09-13T17:50:34.592871+0000","last_active":"2024-09-13T18:21:13.319416+0000","last_peered":"2024-09-13T18:21:13.319416+0000","last_clean":"2024-09-13T18:21:13.319416+0000","last_became_active":"2024-09-13T17:50:34.588513+0000","last_became_peered":"2024-09-13T17:50:34.588513+0000","last_unstale":"2024-09-13T18:21:13.319416+0000","last_undegraded":"2024-09-13T18:21:13.319416+0000","last_fullsized":"2024-09-13T18:21:13.319416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'19","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:40.418309+0000","last_change":"2024-09-13T18:21:40.418512+0000","last_active":"2024-09-13T18:21:40.418309+0000","last_peered":"2024-09-13T18:21:40.418309+0000","last_clean":"2024-09-13T18:21:40.418309+0000","last_became_active":"2024-09-13T17:50:34.585438+0000","last_became_peered":"2024-09-13T17:50:34.585438+0000","last_unstale":"2024-09-13T18:21:40.418309+0000","last_undegraded":"2024-09-13T18:21:40.418309+0000","last_fullsized":"2024-09-13T18:21:40.418309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-13T18:21:40.418258+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-13T18:21:40.418258+0000","last_clean_scrub_stamp":"2024-09-13T18:21:40.418258+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:52.939370+0000","scrub_duration":0.0033203519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'30","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.312747+0000","last_change":"2024-09-13T17:50:34.589628+0000","last_active":"2024-09-13T18:21:13.312747+0000","last_peered":"2024-09-13T18:21:13.312747+0000","last_clean":"2024-09-13T18:21:13.312747+0000","last_became_active":"2024-09-13T17:50:34.587665+0000","last_became_peered":"2024-09-13T17:50:34.587665+0000","last_unstale":"2024-09-13T18:21:13.312747+0000","last_undegraded":"2024-09-13T18:21:13.312747+0000","last_fullsized":"2024-09-13T18:21:13.312747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'27","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.322483+0000","last_change":"2024-09-13T17:50:34.591776+0000","last_active":"2024-09-13T18:21:13.322483+0000","last_peered":"2024-09-13T18:21:13.322483+0000","last_clean":"2024-09-13T18:21:13.322483+0000","last_became_active":"2024-09-13T17:50:34.591496+0000","last_became_peered":"2024-09-13T17:50:34.591496+0000","last_unstale":"2024-09-13T18:21:13.322483+0000","last_undegraded":"2024-09-13T18:21:13.322483+0000","last_fullsized":"2024-09-13T18:21:13.322483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'15","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:17.096582+0000","last_change":"2024-09-13T17:51:16.282372+0000","last_active":"2024-09-13T18:21:17.096582+0000","last_peered":"2024-09-13T18:21:17.096582+0000","last_clean":"2024-09-13T18:21:17.096582+0000","last_became_active":"2024-09-13T17:51:16.282065+0000","last_became_peered":"2024-09-13T17:51:16.282065+0000","last_unstale":"2024-09-13T18:21:17.096582+0000","last_undegraded":"2024-09-13T18:21:17.096582+0000","last_fullsized":"2024-09-13T18:21:17.096582+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"27'14","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.311285+0000","last_change":"2024-09-13T17:50:34.585531+0000","last_active":"2024-09-13T18:21:13.311285+0000","last_peered":"2024-09-13T18:21:13.311285+0000","last_clean":"2024-09-13T18:21:13.311285+0000","last_became_active":"2024-09-13T17:50:34.585226+0000","last_became_peered":"2024-09-13T17:50:34.585226+0000","last_unstale":"2024-09-13T18:21:13.311285+0000","last_undegraded":"2024-09-13T18:21:13.311285+0000","last_fullsized":"2024-09-13T18:21:13.311285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'17","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.318576+0000","last_change":"2024-09-13T17:50:34.586750+0000","last_active":"2024-09-13T18:21:13.318576+0000","last_peered":"2024-09-13T18:21:13.318576+0000","last_clean":"2024-09-13T18:21:13.318576+0000","last_became_active":"2024-09-13T17:50:34.584564+0000","last_became_peered":"2024-09-13T17:50:34.584564+0000","last_unstale":"2024-09-13T18:21:13.318576+0000","last_undegraded":"2024-09-13T18:21:13.318576+0000","last_fullsized":"2024-09-13T18:21:13.318576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'18","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.313624+0000","last_change":"2024-09-13T17:50:34.585938+0000","last_active":"2024-09-13T18:21:13.313624+0000","last_peered":"2024-09-13T18:21:13.313624+0000","last_clean":"2024-09-13T18:21:13.313624+0000","last_became_active":"2024-09-13T17:50:34.585584+0000","last_became_peered":"2024-09-13T17:50:34.585584+0000","last_unstale":"2024-09-13T18:21:13.313624+0000","last_undegraded":"2024-09-13T18:21:13.313624+0000","last_fullsized":"2024-09-13T18:21:13.313624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'15","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310679+0000","last_change":"2024-09-13T17:50:34.588762+0000","last_active":"2024-09-13T18:21:13.310679+0000","last_peered":"2024-09-13T18:21:13.310679+0000","last_clean":"2024-09-13T18:21:13.310679+0000","last_became_active":"2024-09-13T17:50:34.588489+0000","last_became_peered":"2024-09-13T17:50:34.588489+0000","last_unstale":"2024-09-13T18:21:13.310679+0000","last_undegraded":"2024-09-13T18:21:13.310679+0000","last_fullsized":"2024-09-13T18:21:13.310679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'18","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.302115+0000","last_change":"2024-09-13T17:50:34.583889+0000","last_active":"2024-09-13T18:21:13.302115+0000","last_peered":"2024-09-13T18:21:13.302115+0000","last_clean":"2024-09-13T18:21:13.302115+0000","last_became_active":"2024-09-13T17:50:34.583551+0000","last_became_peered":"2024-09-13T17:50:34.583551+0000","last_unstale":"2024-09-13T18:21:13.302115+0000","last_undegraded":"2024-09-13T18:21:13.302115+0000","last_fullsized":"2024-09-13T18:21:13.302115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'10","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:29.337850+0000","last_change":"2024-09-13T18:21:29.338126+0000","last_active":"2024-09-13T18:21:29.337850+0000","last_peered":"2024-09-13T18:21:29.337850+0000","last_clean":"2024-09-13T18:21:29.337850+0000","last_became_active":"2024-09-13T17:50:34.585340+0000","last_became_peered":"2024-09-13T17:50:34.585340+0000","last_unstale":"2024-09-13T18:21:29.337850+0000","last_undegraded":"2024-09-13T18:21:29.337850+0000","last_fullsized":"2024-09-13T18:21:29.337850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-13T18:21:29.337797+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-13T18:21:29.337797+0000","last_clean_scrub_stamp":"2024-09-13T18:21:29.337797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:52.759391+0000","scrub_duration":0.003218725,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'17","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317213+0000","last_change":"2024-09-13T17:50:34.592365+0000","last_active":"2024-09-13T18:21:13.317213+0000","last_peered":"2024-09-13T18:21:13.317213+0000","last_clean":"2024-09-13T18:21:13.317213+0000","last_became_active":"2024-09-13T17:50:34.592046+0000","last_became_peered":"2024-09-13T17:50:34.592046+0000","last_unstale":"2024-09-13T18:21:13.317213+0000","last_undegraded":"2024-09-13T18:21:13.317213+0000","last_fullsized":"2024-09-13T18:21:13.317213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":751482,"num_write_kb":16106434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178546,"ondisk_log_size":178546,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1271,"ondisk_log_size":1271,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":15941984,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":121,"num_read_kb":179,"num_write":3371,"num_write_kb":16874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10000,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32030720,"data_stored":31883968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3168,"ondisk_log_size":3168,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509817,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1768076,"kb_used_data":8580,"kb_used_omap":1,"kb_used_meta":1759486,"kb_avail":92603764,"statfs":{"total":96636764160,"available":94826254336,"internally_reserved":0,"allocated":8785920,"data_stored":8600393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":1801714120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":2.6030000000000002,"5min":4.7919999999999998,"15min":2.8069999999999999},"min":{"1min":0.28799999999999998,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":24.829000000000001,"5min":41.494999999999997,"15min":50.511000000000003},"last":0.42799999999999999},{"interface":"front","average":{"1min":1.708,"5min":3.0110000000000001,"15min":1.677},"min":{"1min":0.27300000000000002,"5min":0.193,"15min":0.193},"max":{"1min":9.9600000000000009,"5min":18.760000000000002,"15min":18.760000000000002},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":2.8199999999999998,"15min":1.841},"min":{"1min":0.52700000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":10.379,"5min":19.626000000000001,"15min":19.626000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.2629999999999999,"5min":3.0880000000000001,"15min":2.081},"min":{"1min":0.312,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.891,"5min":22.126999999999999,"15min":22.126999999999999},"last":0.63200000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":1.3400000000000001,"15min":1.2869999999999999},"min":{"1min":0.246,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.048,"5min":13.039999999999999,"15min":22.010000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.81999999999999995,"5min":1.4830000000000001,"15min":1.472},"min":{"1min":0.32300000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.202,"5min":13.061999999999999,"15min":24.170999999999999},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.3049999999999999,"15min":1.4570000000000001},"min":{"1min":0.377,"5min":0.307,"15min":0.307},"max":{"1min":1.6839999999999999,"5min":12.936,"15min":24.359000000000002},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.278,"5min":1.9670000000000001,"15min":1.4810000000000001},"min":{"1min":0.35099999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":3.7530000000000001,"5min":15.539999999999999,"15min":15.539999999999999},"last":0.70499999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.5700000000000001,"15min":1.6279999999999999},"min":{"1min":0.35299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":5.5229999999999997,"5min":12.984,"15min":49.838000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.5449999999999999,"15min":1.5980000000000001},"min":{"1min":0.42499999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.4730000000000001,"5min":16.334,"15min":41.591999999999999},"last":0.83499999999999996}]},{"osd":6,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":1.7010000000000001,"5min":1.875,"15min":1.764},"min":{"1min":0.46400000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":10.337999999999999,"5min":12.907999999999999,"15min":24.457000000000001},"last":0.879},{"interface":"front","average":{"1min":1.7050000000000001,"5min":1.7450000000000001,"15min":1.6579999999999999},"min":{"1min":0.45000000000000001,"5min":0.42399999999999999,"15min":0.40999999999999998},"max":{"1min":10.279,"5min":13.015000000000001,"15min":24.387},"last":1.0980000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.651,"15min":1.397},"min":{"1min":0.65400000000000003,"5min":0.34200000000000003,"15min":0.32300000000000001},"max":{"1min":4.649,"5min":14.138999999999999,"15min":14.138999999999999},"last":1.034},{"interface":"front","average":{"1min":1.323,"5min":1.847,"15min":1.4410000000000001},"min":{"1min":0.47699999999999998,"5min":0.377,"15min":0.35299999999999998},"max":{"1min":4.2270000000000003,"5min":14.013,"15min":14.013},"last":1.1439999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509819,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1055244,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":1054846,"kb_avail":93316596,"statfs":{"total":96636764160,"available":95556194304,"internally_reserved":0,"allocated":397312,"data_stored":210684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1080162758},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":1.7190000000000001,"15min":1.091},"min":{"1min":0.35199999999999998,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.68400000000000005,"5min":17.971,"15min":17.971},"last":0.5},{"interface":"front","average":{"1min":0.69299999999999995,"5min":1.873,"15min":1.179},"min":{"1min":0.40600000000000003,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.042,"5min":18.001999999999999,"15min":18.001999999999999},"last":0.89500000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":2.2759999999999998,"15min":1.6870000000000001},"min":{"1min":0.38,"5min":0.22600000000000001,"15min":0.217},"max":{"1min":1.609,"5min":24.599,"15min":56.939999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":2.5219999999999998,"15min":1.768},"min":{"1min":0.443,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.5369999999999999,"5min":24.75,"15min":57.006999999999998},"last":0.48599999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":2.79,"15min":1.599},"min":{"1min":0.40899999999999997,"5min":0.218,"15min":0.218},"max":{"1min":1.714,"5min":16.707000000000001,"15min":16.707000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":2.4510000000000001,"15min":1.554},"min":{"1min":0.27600000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.135,"5min":16.640999999999998,"15min":16.640999999999998},"last":0.57599999999999996}]},{"osd":4,"last update":"Fri Sep 13 18:21:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.4370000000000001,"15min":1.236},"min":{"1min":0.67800000000000005,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.6000000000000001,"5min":8.2829999999999995,"15min":14.177},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.056,"5min":1.258,"15min":1.1639999999999999},"min":{"1min":0.55600000000000005,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.617,"5min":8.1980000000000004,"15min":14.396000000000001},"last":0.95699999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:21:22 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":0.73599999999999999,"15min":1.0049999999999999},"min":{"1min":0.433,"5min":0.34200000000000003,"15min":0.34100000000000003},"max":{"1min":3.637,"5min":3.6840000000000002,"15min":14.153},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":1.1299999999999999,"15min":1.131},"min":{"1min":0.52100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.4450000000000001,"5min":7.476,"15min":14.375999999999999},"last":0.99399999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":0.97899999999999998,"15min":1.1000000000000001},"min":{"1min":0.505,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6080000000000001,"5min":6.8730000000000002,"15min":8.3010000000000002},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.151,"5min":1.038,"15min":1.2110000000000001},"min":{"1min":0.63700000000000001,"5min":0.40400000000000003,"15min":0.39200000000000002},"max":{"1min":1.6519999999999999,"5min":6.9500000000000002,"15min":14.074},"last":1.123}]},{"osd":7,"last update":"Fri Sep 13 18:21:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0700000000000001,"15min":1.079},"min":{"1min":0.47999999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.5569999999999999,"5min":8.0410000000000004,"15min":8.0410000000000004},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.1599999999999999,"15min":1.1859999999999999},"min":{"1min":0.56899999999999995,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.3799999999999999,"5min":6.5300000000000002,"15min":14.307},"last":0.92400000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509818,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1738228,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":1737918,"kb_avail":92633612,"statfs":{"total":96636764160,"available":94856818688,"internally_reserved":0,"allocated":307200,"data_stored":116868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1779628486},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":2.5430000000000001,"15min":1.907},"min":{"1min":0.42299999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":4.25,"5min":21.949999999999999,"15min":21.949999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":2.3759999999999999,"15min":1.593},"min":{"1min":0.27600000000000002,"5min":0.193,"15min":0.193},"max":{"1min":2.7349999999999999,"5min":23.338999999999999,"15min":23.338999999999999},"last":0.77100000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.8540000000000001,"15min":1.5149999999999999},"min":{"1min":0.38,"5min":0.22,"15min":0.22},"max":{"1min":5.5129999999999999,"5min":11.423999999999999,"15min":14.869999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.962,"15min":1.4319999999999999},"min":{"1min":0.47399999999999998,"5min":0.223,"15min":0.223},"max":{"1min":5.4809999999999999,"5min":12.439,"15min":15.058},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":4.3810000000000002,"15min":3.1800000000000002},"min":{"1min":0.44400000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":4.5839999999999996,"5min":27.838000000000001,"15min":41.369999999999997},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.0860000000000001,"5min":4.327,"15min":2.911},"min":{"1min":0.251,"5min":0.216,"15min":0.216},"max":{"1min":4.9299999999999997,"5min":30.077000000000002,"15min":40.325000000000003},"last":0.70099999999999996}]},{"osd":4,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":2.4660000000000002,"15min":2.1480000000000001},"min":{"1min":0.51500000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":7.2089999999999996,"5min":29.702000000000002,"15min":103.44499999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.2010000000000001,"5min":2.4689999999999999,"15min":1.905},"min":{"1min":0.52700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.75,"5min":29.425999999999998,"15min":40.953000000000003},"last":0.95899999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:21:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.964,"15min":1.744},"min":{"1min":0.60899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":4.548,"5min":15.169,"15min":18.954000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.6919999999999999,"15min":1.5269999999999999},"min":{"1min":0.36699999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.2279999999999998,"5min":13.630000000000001,"15min":15.413},"last":0.85499999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.994,"5min":2.044,"15min":1.633},"min":{"1min":0.748,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":6.4889999999999999,"5min":13.67,"15min":15.462999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":2.0569999999999999,"5min":2.1419999999999999,"15min":1.7270000000000001},"min":{"1min":0.85699999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":6.5220000000000002,"5min":13.696,"15min":15.49},"last":0.91900000000000004}]},{"osd":7,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.667,"15min":1.415},"min":{"1min":0.64500000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":7.9669999999999996,"5min":12.476000000000001,"15min":15.093999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.7569999999999999,"5min":1.7330000000000001,"15min":1.5},"min":{"1min":0.80200000000000005,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":7.9969999999999999,"5min":12.539999999999999,"15min":15.119},"last":1.087}]}]},{"osd":0,"up_from":15,"seq":64424509820,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1247072,"kb_used_data":4056,"kb_used_omap":1,"kb_used_meta":1243006,"kb_avail":93124768,"statfs":{"total":96636764160,"available":95359762432,"internally_reserved":0,"allocated":4153344,"data_stored":3967313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1272838598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":2.8849999999999998,"15min":1.7210000000000001},"min":{"1min":0.19800000000000001,"5min":0.182,"15min":0.182},"max":{"1min":7.0830000000000002,"5min":23.004000000000001,"15min":24.202000000000002},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.0700000000000001,"5min":2.9630000000000001,"15min":1.8640000000000001},"min":{"1min":0.375,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":7.1139999999999999,"5min":23.039999999999999,"15min":24.260999999999999},"last":1.075}]},{"osd":2,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":1.6359999999999999,"15min":1.1200000000000001},"min":{"1min":0.254,"5min":0.20399999999999999,"15min":0.186},"max":{"1min":6.4770000000000003,"5min":11.026999999999999,"15min":17.859000000000002},"last":0.63300000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.851,"15min":1.343},"min":{"1min":0.26600000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":7.4100000000000001,"5min":11.702999999999999,"15min":19.678999999999998},"last":0.73899999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":3.306,"15min":1.95},"min":{"1min":0.28100000000000003,"5min":0.219,"15min":0.219},"max":{"1min":1.0509999999999999,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.073,"5min":2.7130000000000001,"15min":1.8240000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":7.2210000000000001,"5min":15.856999999999999,"15min":15.856999999999999},"last":0.89100000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.6499999999999999,"15min":1.2729999999999999},"min":{"1min":0.32700000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":7.1959999999999997,"5min":8.7010000000000005,"15min":8.7010000000000005},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":2.2109999999999999,"15min":1.5409999999999999},"min":{"1min":0.34399999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.256,"5min":10.965999999999999,"15min":16.291},"last":0.91800000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.7629999999999999,"15min":1.327},"min":{"1min":0.33000000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":3.1339999999999999,"5min":15.18,"15min":15.18},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.8069999999999999,"15min":1.427},"min":{"1min":0.39100000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":7.1669999999999998,"5min":17.852,"15min":17.852},"last":1.1759999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":2.2869999999999999,"15min":1.577},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":7.2480000000000002,"5min":15.467000000000001,"15min":15.467000000000001},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.8109999999999999,"15min":1.4330000000000001},"min":{"1min":0.44900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":7.1429999999999998,"5min":17.908000000000001,"15min":17.908000000000001},"last":1.476}]},{"osd":7,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.897,"15min":1.3979999999999999},"min":{"1min":0.40000000000000002,"5min":0.33500000000000002,"15min":0.32700000000000001},"max":{"1min":5.5789999999999997,"5min":18.986999999999998,"15min":18.986999999999998},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.9770000000000001,"15min":1.4830000000000001},"min":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.373},"max":{"1min":5.6159999999999997,"5min":19.021000000000001,"15min":19.021000000000001},"last":0.96099999999999997}]}]},{"osd":7,"up_from":14,"seq":60129542522,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1207524,"kb_used_data":12252,"kb_used_omap":1,"kb_used_meta":1195262,"kb_avail":93164316,"statfs":{"total":96636764160,"available":95400259584,"internally_reserved":0,"allocated":12546048,"data_stored":12357112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":1223948747},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":1.9470000000000001,"15min":1.2410000000000001},"min":{"1min":0.26500000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":2.875,"5min":35.195,"15min":35.195},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.133,"5min":2.1320000000000001,"15min":1.411},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":5.0919999999999996,"5min":35.253999999999998,"15min":35.253999999999998},"last":1.1379999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:21:30 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":2.5270000000000001,"15min":1.4930000000000001},"min":{"1min":0.317,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":4.8070000000000004,"5min":19.585000000000001,"15min":19.585000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.98899999999999999,"5min":2.52,"15min":1.6359999999999999},"min":{"1min":0.41699999999999998,"5min":0.29599999999999999,"15min":0.28000000000000003},"max":{"1min":4.8220000000000001,"5min":19.617000000000001,"15min":19.617000000000001},"last":1.111}]},{"osd":2,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.597,"15min":1.1699999999999999},"min":{"1min":0.51200000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.5449999999999999,"5min":15.789999999999999,"15min":15.789999999999999},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.5680000000000001,"15min":1.1799999999999999},"min":{"1min":0.38700000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.5259999999999998,"5min":12.965,"15min":12.965},"last":1.353}]},{"osd":3,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":2.8980000000000001,"15min":1.9119999999999999},"min":{"1min":0.41099999999999998,"5min":0.316,"15min":0.316},"max":{"1min":7.8449999999999998,"5min":40.948999999999998,"15min":40.948999999999998},"last":0.754},{"interface":"front","average":{"1min":2.0779999999999998,"5min":3.0139999999999998,"15min":1.768},"min":{"1min":0.39300000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":7.8810000000000002,"5min":25.234000000000002,"15min":25.234000000000002},"last":1.022}]},{"osd":4,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":1.165,"15min":0.98999999999999999},"min":{"1min":0.33700000000000002,"5min":0.28799999999999998,"15min":0.252},"max":{"1min":3.0609999999999999,"5min":8.8670000000000009,"15min":8.8670000000000009},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.3300000000000001,"15min":1.1859999999999999},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":6.069,"5min":10.279999999999999,"15min":10.279999999999999},"last":0.94299999999999995}]},{"osd":5,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":1.0620000000000001,"15min":0.98199999999999998},"min":{"1min":0.374,"5min":0.311,"15min":0.27000000000000002},"max":{"1min":1.5289999999999999,"5min":8.1699999999999999,"15min":8.1699999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.878,"5min":0.96299999999999997,"15min":1.02},"min":{"1min":0.39800000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.9490000000000001,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":1.252}]},{"osd":6,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.4790000000000001,"15min":1.236},"min":{"1min":0.41599999999999998,"5min":0.34599999999999997,"15min":0.309},"max":{"1min":3.597,"5min":8.2219999999999995,"15min":8.2219999999999995},"last":1.29},{"interface":"front","average":{"1min":1.419,"5min":1.548,"15min":1.2789999999999999},"min":{"1min":0.48499999999999999,"5min":0.434,"15min":0.34499999999999997},"max":{"1min":3.7210000000000001,"5min":8.2469999999999999,"15min":8.2469999999999999},"last":1.403}]}]},{"osd":6,"up_from":14,"seq":60129542522,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1419436,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":1415038,"kb_avail":92952404,"statfs":{"total":96636764160,"available":95183261696,"internally_reserved":0,"allocated":4493312,"data_stored":4311530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1448999367},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":2.0750000000000002,"5min":2.0699999999999998,"15min":1.296},"min":{"1min":0.28000000000000003,"5min":0.214,"15min":0.214},"max":{"1min":11.666,"5min":12.808999999999999,"15min":12.808999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.6639999999999999,"15min":1.179},"min":{"1min":0.28899999999999998,"5min":0.218,"15min":0.218},"max":{"1min":7.8700000000000001,"5min":12.308999999999999,"15min":12.308999999999999},"last":0.79000000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":2.6840000000000002,"5min":2.323,"15min":1.4930000000000001},"min":{"1min":0.52300000000000002,"5min":0.313,"15min":0.28699999999999998},"max":{"1min":20.530999999999999,"5min":20.530999999999999,"15min":20.530999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":2.2959999999999998,"5min":2.2149999999999999,"15min":1.514},"min":{"1min":0.377,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":20.501999999999999,"5min":20.501999999999999,"15min":20.501999999999999},"last":1.016}]},{"osd":2,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.7350000000000001,"15min":1.2270000000000001},"min":{"1min":0.47699999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":3.6800000000000002,"5min":10.792999999999999,"15min":10.792999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.8140000000000001,"15min":1.381},"min":{"1min":0.51800000000000002,"5min":0.317,"15min":0.317},"max":{"1min":3.738,"5min":10.826000000000001,"15min":10.826000000000001},"last":0.86199999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":2.4940000000000002,"5min":3.8300000000000001,"15min":2.5750000000000002},"min":{"1min":0.36599999999999999,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":16.495999999999999,"5min":32.25,"15min":84.867999999999995},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.4199999999999999,"5min":2.5009999999999999,"15min":1.927},"min":{"1min":0.36699999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":8.6780000000000008,"5min":14.064,"15min":84.343999999999994},"last":1.04}]},{"osd":4,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":2.0739999999999998,"5min":1.6990000000000001,"15min":1.2070000000000001},"min":{"1min":0.39200000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":9.5649999999999995,"5min":11.044,"15min":11.044},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.294,"15min":1.113},"min":{"1min":0.38400000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":9.4719999999999995,"5min":9.4719999999999995,"15min":9.4719999999999995},"last":0.93300000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2689999999999999,"15min":1.1559999999999999},"min":{"1min":0.376,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.0939999999999999,"5min":8.7880000000000003,"15min":8.7880000000000003},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.149,"15min":1.1000000000000001},"min":{"1min":0.41499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.1709999999999998,"5min":7.569,"15min":7.569},"last":1.0860000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.1739999999999999,"15min":1.181},"min":{"1min":0.434,"5min":0.32900000000000001,"15min":0.29999999999999999},"max":{"1min":2.8029999999999999,"5min":7.1820000000000004,"15min":7.1820000000000004},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.133,"5min":1.294,"15min":1.256},"min":{"1min":0.53700000000000003,"5min":0.34499999999999997,"15min":0.33500000000000002},"max":{"1min":2.871,"5min":7.2039999999999997,"15min":7.2039999999999997},"last":1.2629999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542524,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":978544,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":978238,"kb_avail":93393296,"statfs":{"total":96636764160,"available":95634735104,"internally_reserved":0,"allocated":303104,"data_stored":117302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1001716166},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.595,"15min":1.1819999999999999},"min":{"1min":0.316,"5min":0.218,"15min":0.218},"max":{"1min":4.0919999999999996,"5min":10.051,"15min":10.148999999999999},"last":0.878},{"interface":"front","average":{"1min":1.038,"5min":1.633,"15min":1.3340000000000001},"min":{"1min":0.40500000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.9550000000000001,"5min":10.098000000000001,"15min":10.185},"last":1.3089999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.647,"15min":1.1879999999999999},"min":{"1min":0.28899999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":4.7119999999999997,"5min":10.08,"15min":10.436999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.032,"5min":1.843,"15min":1.387},"min":{"1min":0.44500000000000001,"5min":0.253,"15min":0.253},"max":{"1min":2.4369999999999998,"5min":19.388000000000002,"15min":19.388000000000002},"last":1.1000000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.5249999999999999,"15min":1.129},"min":{"1min":0.443,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":5.1660000000000004,"5min":12.667,"15min":12.667},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.6639999999999999,"15min":1.2},"min":{"1min":0.441,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.8900000000000001,"5min":34.905999999999999,"15min":34.905999999999999},"last":1.214}]},{"osd":3,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":3.6779999999999999,"15min":2.3220000000000001},"min":{"1min":0.374,"5min":0.314,"15min":0.314},"max":{"1min":9.077,"5min":25.149999999999999,"15min":28.295000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.242,"5min":2.9180000000000001,"15min":1.7410000000000001},"min":{"1min":0.436,"5min":0.38500000000000001,"15min":0.318},"max":{"1min":5.3689999999999998,"5min":19.731000000000002,"15min":19.731000000000002},"last":0.94299999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0960000000000001,"15min":0.97399999999999998},"min":{"1min":0.33000000000000002,"5min":0.28599999999999998,"15min":0.255},"max":{"1min":4.8140000000000001,"5min":10.023999999999999,"15min":10.023999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.599,"15min":1.2290000000000001},"min":{"1min":0.38700000000000001,"5min":0.32500000000000001,"15min":0.28999999999999998},"max":{"1min":4.7919999999999998,"5min":10.210000000000001,"15min":10.210000000000001},"last":1.3500000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.4039999999999999,"15min":1.1339999999999999},"min":{"1min":0.36799999999999999,"5min":0.33900000000000002,"15min":0.25800000000000001},"max":{"1min":1.5649999999999999,"5min":6.6849999999999996,"15min":6.6849999999999996},"last":1.018},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3779999999999999,"15min":1.2150000000000001},"min":{"1min":0.51800000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":5.0819999999999999,"5min":10.151,"15min":10.151},"last":1.4650000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.258,"15min":1.1459999999999999},"min":{"1min":0.46899999999999997,"5min":0.32900000000000001,"15min":0.32000000000000001},"max":{"1min":2.581,"5min":7.9329999999999998,"15min":7.9329999999999998},"last":1.452},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.196,"15min":1.151},"min":{"1min":0.53800000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.5,"5min":5.976,"15min":5.976},"last":1.399}]}]},{"osd":4,"up_from":14,"seq":60129542522,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1301036,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":1296638,"kb_avail":93070804,"statfs":{"total":96636764160,"available":95304503296,"internally_reserved":0,"allocated":4493312,"data_stored":4310894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1327757766},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.5980000000000001,"15min":1.3160000000000001},"min":{"1min":0.25800000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":5.7759999999999998,"5min":14.318,"15min":28.439},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.022,"5min":1.772,"15min":1.4510000000000001},"min":{"1min":0.30499999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":3.0760000000000001,"5min":16.943000000000001,"15min":28.422000000000001},"last":1.2769999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":1.738,"5min":4.3470000000000004,"15min":2.6389999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":21.359000000000002,"5min":30.277999999999999,"15min":38.895000000000003},"last":1.016},{"interface":"front","average":{"1min":1.75,"5min":3.6030000000000002,"15min":2.0270000000000001},"min":{"1min":0.28199999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":21.388999999999999,"5min":32.670000000000002,"15min":32.670000000000002},"last":1.111}]},{"osd":2,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.7090000000000001,"15min":1.1870000000000001},"min":{"1min":0.32200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.7229999999999999,"5min":25.167999999999999,"15min":25.167999999999999},"last":0.439},{"interface":"front","average":{"1min":1.345,"5min":2.0099999999999998,"15min":1.3480000000000001},"min":{"1min":0.41899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":5.75,"5min":25.218,"15min":25.218},"last":0.624}]},{"osd":3,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":2.3799999999999999,"5min":2.8439999999999999,"15min":1.827},"min":{"1min":0.36299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":16.344999999999999,"5min":43.945999999999998,"15min":43.945999999999998},"last":0.871},{"interface":"front","average":{"1min":2.4100000000000001,"5min":3.3250000000000002,"15min":2.1429999999999998},"min":{"1min":0.48899999999999999,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":15.901,"5min":29.640000000000001,"15min":29.640000000000001},"last":0.92500000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":1.218,"15min":1.0569999999999999},"min":{"1min":0.34799999999999998,"5min":0.255,"15min":0.255},"max":{"1min":1.266,"5min":12.282999999999999,"15min":12.282999999999999},"last":1.419},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.425,"15min":1.1870000000000001},"min":{"1min":0.53100000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.7000000000000002,"5min":16.995999999999999,"15min":16.995999999999999},"last":1.2090000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.9530000000000001,"15min":1.401},"min":{"1min":0.441,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":9.3650000000000002,"5min":23.010999999999999,"15min":23.010999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.302,"5min":1.3720000000000001,"15min":1.194},"min":{"1min":0.42199999999999999,"5min":0.33800000000000002,"15min":0.33600000000000002},"max":{"1min":5.6369999999999996,"5min":7.9489999999999998,"15min":7.9489999999999998},"last":1.0820000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":1.5569999999999999,"5min":1.639,"15min":1.2949999999999999},"min":{"1min":0.45000000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":11.298,"5min":20.417000000000002,"15min":20.417000000000002},"last":1.139},{"interface":"front","average":{"1min":1.02,"5min":1.456,"15min":1.272},"min":{"1min":0.36399999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.9410000000000001,"5min":12.308999999999999,"15min":12.308999999999999},"last":1.349}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3276800,"data_stored":3261021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":94760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8503296,"data_stored":8484469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4194970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4195606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":11669504,"data_stored":11650820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:21:46.727 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.727 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.727 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.728 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.728 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.728 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.728 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.728 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.728 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.729 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.729 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.729 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.729 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.729 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.729 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.729 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.730 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.730 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.730 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.730 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.730 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.730 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.730 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.730 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.731 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.731 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.731 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.731 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.731 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.731 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.731 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.731 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.731 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.732 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.732 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.732 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.732 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.732 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T17:50:21.057812+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.732 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.732 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.732 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.733 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.733 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.733 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.733 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.733 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.733 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.733 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.733 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.734 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.734 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.734 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.734 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.734 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.734 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.734 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.734 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-09-13T17:50:33.165496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.735 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.735 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.735 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.735 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.735 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.736 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.736 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.736 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.736 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.736 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.736 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.737 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.737 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.737 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.738 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.738 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.738 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.738 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.739 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.739 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.739 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.740 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.740 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.740 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.741 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.741 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.741 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.742 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.742 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.742 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.742 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.743 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.743 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:21:46.743 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T18:22:06.744 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:22:06.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.848+0000 7fc0099f9700 1 Processor -- start 2024-09-13T18:22:06.856 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.848+0000 7fc0099f9700 1 -- start start 2024-09-13T18:22:06.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.848+0000 7fc0099f9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc004072240 0x7fc0040c0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:06.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.848+0000 7fc0099f9700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0040b2210 con 0x7fc004071e60 2024-09-13T18:22:06.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.848+0000 7fc0099f9700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0040c1780 con 0x7fc0040c28f0 2024-09-13T18:22:06.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.848+0000 7fc0099f9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040c1a60 con 0x7fc004072240 2024-09-13T18:22:06.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.848+0000 7fc0089f7700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc004071e60 0x7fc0040c32e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58142/0 (socket says 172.21.15.40:58142) 2024-09-13T18:22:06.857 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.848+0000 7fc0089f7700 1 -- 172.21.15.40:0/3525201760 learned_addr learned my addr 172.21.15.40:0/3525201760 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:22:06.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.848+0000 7fc003fff700 1 --2- 172.21.15.40:0/3525201760 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc004072240 0x7fc0040c0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:06.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.848+0000 7fc0037fe700 1 -- 172.21.15.40:0/3525201760 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455108658 0 0) 0x7fc0040b2210 con 0x7fc004071e60 2024-09-13T18:22:06.858 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.848+0000 7fc0037fe700 1 -- 172.21.15.40:0/3525201760 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbff40035d0 con 0x7fc004071e60 2024-09-13T18:22:06.859 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0037fe700 1 -- 172.21.15.40:0/3525201760 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 668893569 0 0) 0x7fbff40035d0 con 0x7fc004071e60 2024-09-13T18:22:06.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0037fe700 1 -- 172.21.15.40:0/3525201760 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc004072240 msgr2=0x7fc0040c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:06.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0037fe700 1 --2- 172.21.15.40:0/3525201760 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc004072240 0x7fc0040c0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:06.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0037fe700 1 -- 172.21.15.40:0/3525201760 >> v1:172.21.15.43:6789/0 conn(0x7fc0040c28f0 legacy=0x7fc0041565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:06.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0037fe700 1 -- 172.21.15.40:0/3525201760 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0040c5d70 con 0x7fc004071e60 2024-09-13T18:22:06.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0037fe700 1 -- 172.21.15.40:0/3525201760 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4028387976 0 0) 0x7fbff8001670 con 0x7fc004071e60 2024-09-13T18:22:06.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0037fe700 1 -- 172.21.15.40:0/3525201760 >> v1:172.21.15.40:6789/0 conn(0x7fc004071e60 legacy=0x7fc0040c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:06.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0037fe700 1 --2- 172.21.15.40:0/3525201760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbff4003a50 0x7fbff4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:06.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0037fe700 1 --2- 172.21.15.40:0/3525201760 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc004072240 0x7fbff4004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:06.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0037fe700 1 --2- 172.21.15.40:0/3525201760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbff4004db0 0x7fbff4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:06.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0037fe700 1 -- 172.21.15.40:0/3525201760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040c1a60 con 0x7fbff4004db0 2024-09-13T18:22:06.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0037fe700 1 -- 172.21.15.40:0/3525201760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff400ba30 con 0x7fbff4003a50 2024-09-13T18:22:06.860 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0037fe700 1 -- 172.21.15.40:0/3525201760 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff400bbf0 con 0x7fc004072240 2024-09-13T18:22:06.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0091f8700 1 --2- 172.21.15.40:0/3525201760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbff4004db0 0x7fbff4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:06.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0089f7700 1 --2- 172.21.15.40:0/3525201760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbff4003a50 0x7fbff4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:06.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc003fff700 1 --2- 172.21.15.40:0/3525201760 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc004072240 0x7fbff4004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:06.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0091f8700 1 -- 172.21.15.40:0/3525201760 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc004072240 msgr2=0x7fbff4004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:06.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0091f8700 1 --2- 172.21.15.40:0/3525201760 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc004072240 0x7fbff4004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:06.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0091f8700 1 -- 172.21.15.40:0/3525201760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbff4003a50 msgr2=0x7fbff4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:06.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0091f8700 1 --2- 172.21.15.40:0/3525201760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbff4003a50 0x7fbff4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:06.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0091f8700 1 -- 172.21.15.40:0/3525201760 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff40035d0 con 0x7fbff4004db0 2024-09-13T18:22:06.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0091f8700 1 --2- 172.21.15.40:0/3525201760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbff4004db0 0x7fbff4005230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00c290 tx=0x7fbff40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2113168bd40e8c0c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:06.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0037fe700 1 -- 172.21.15.40:0/3525201760 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc001e70 con 0x7fbff4004db0 2024-09-13T18:22:06.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0037fe700 1 -- 172.21.15.40:0/3525201760 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc00ce50 con 0x7fbff4004db0 2024-09-13T18:22:06.861 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0037fe700 1 -- 172.21.15.40:0/3525201760 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc012660 con 0x7fbff4004db0 2024-09-13T18:22:06.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0099f9700 1 -- 172.21.15.40:0/3525201760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbff4004db0 msgr2=0x7fbff4005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:06.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0099f9700 1 --2- 172.21.15.40:0/3525201760 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbff4004db0 0x7fbff4005230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00c290 tx=0x7fbff40058a0 comp rx=0 tx=0).stop 2024-09-13T18:22:06.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0091f8700 1 -- 172.21.15.40:0/3525201760 reap_dead start 2024-09-13T18:22:06.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0099f9700 1 -- 172.21.15.40:0/3525201760 shutdown_connections 2024-09-13T18:22:06.864 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0099f9700 1 -- 172.21.15.40:0/3525201760 >> 172.21.15.40:0/3525201760 conn(0x7fc00406e4f0 msgr2=0x7fc00406e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:06.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0099f9700 1 -- 172.21.15.40:0/3525201760 shutdown_connections 2024-09-13T18:22:06.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.852+0000 7fc0099f9700 1 -- 172.21.15.40:0/3525201760 wait complete. 2024-09-13T18:22:06.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0099f9700 1 Processor -- start 2024-09-13T18:22:06.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0099f9700 1 -- start start 2024-09-13T18:22:06.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0099f9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbff4004db0 0x7fc004154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:06.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0099f9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0040aee40 0x7fc0041f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:06.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0099f9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0041f1fd0 0x7fc004150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:06.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0099f9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040aded0 con 0x7fbff4004db0 2024-09-13T18:22:06.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0099f9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0041ef400 con 0x7fc0041f1fd0 2024-09-13T18:22:06.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0099f9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040adbf0 con 0x7fc0040aee40 2024-09-13T18:22:06.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0089f7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbff4004db0 0x7fc004154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:06.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0089f7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbff4004db0 0x7fc004154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39122/0 (socket says 172.21.15.40:39122) 2024-09-13T18:22:06.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0089f7700 1 -- 172.21.15.40:0/2338231060 learned_addr learned my addr 172.21.15.40:0/2338231060 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:22:06.865 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0091f8700 1 --2- 172.21.15.40:0/2338231060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0041f1fd0 0x7fc004150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:06.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc003fff700 1 --2- 172.21.15.40:0/2338231060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0040aee40 0x7fc0041f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:06.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0089f7700 1 -- 172.21.15.40:0/2338231060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0040aee40 msgr2=0x7fc0041f1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:06.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0089f7700 1 --2- 172.21.15.40:0/2338231060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0040aee40 0x7fc0041f1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:06.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0089f7700 1 -- 172.21.15.40:0/2338231060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0041f1fd0 msgr2=0x7fc004150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:06.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0089f7700 1 --2- 172.21.15.40:0/2338231060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0041f1fd0 0x7fc004150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:06.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0089f7700 1 -- 172.21.15.40:0/2338231060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0041ee640 con 0x7fbff4004db0 2024-09-13T18:22:06.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0089f7700 1 --2- 172.21.15.40:0/2338231060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbff4004db0 0x7fc004154c80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbff80017a0 tx=0x7fbff8004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:06.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0017fa700 1 -- 172.21.15.40:0/2338231060 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff800f7d0 con 0x7fbff4004db0 2024-09-13T18:22:06.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0017fa700 1 -- 172.21.15.40:0/2338231060 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff800f950 con 0x7fbff4004db0 2024-09-13T18:22:06.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0099f9700 1 -- 172.21.15.40:0/2338231060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0041ee920 con 0x7fbff4004db0 2024-09-13T18:22:06.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0017fa700 1 -- 172.21.15.40:0/2338231060 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff800fd00 con 0x7fbff4004db0 2024-09-13T18:22:06.866 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0099f9700 1 -- 172.21.15.40:0/2338231060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0041eeee0 con 0x7fbff4004db0 2024-09-13T18:22:06.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0099f9700 1 -- 172.21.15.40:0/2338231060 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0040c1780 con 0x7fbff4004db0 2024-09-13T18:22:06.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.856+0000 7fc0017fa700 1 -- 172.21.15.40:0/2338231060 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbff801d460 con 0x7fbff4004db0 2024-09-13T18:22:06.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.860+0000 7fc0017fa700 1 --2- 172.21.15.40:0/2338231060 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbfe8066730 0x7fbfe8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:06.867 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.860+0000 7fc003fff700 1 --2- 172.21.15.40:0/2338231060 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbfe8066730 0x7fbfe8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:06.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.860+0000 7fc0017fa700 1 -- 172.21.15.40:0/2338231060 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbff80846a0 con 0x7fbff4004db0 2024-09-13T18:22:06.868 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.860+0000 7fc003fff700 1 --2- 172.21.15.40:0/2338231060 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbfe8066730 0x7fbfe8068bf0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc0040c8530 tx=0x7fbff0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:06.874 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:06.868+0000 7fc0017fa700 1 -- 172.21.15.40:0/2338231060 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbff80515c0 con 0x7fbff4004db0 2024-09-13T18:22:07.041 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:07.032+0000 7fc0099f9700 1 -- 172.21.15.40:0/2338231060 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0040c2da0 con 0x7fbfe8066730 2024-09-13T18:22:07.059 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:07.052+0000 7fc0017fa700 1 -- 172.21.15.40:0/2338231060 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+370357 (secure 0 0 0) 0x7fbff00137a0 con 0x7fbfe8066730 2024-09-13T18:22:07.060 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T18:22:07.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:07.056+0000 7fc0099f9700 1 -- 172.21.15.40:0/2338231060 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbfe8066730 msgr2=0x7fbfe8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:07.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:07.056+0000 7fc0099f9700 1 --2- 172.21.15.40:0/2338231060 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbfe8066730 0x7fbfe8068bf0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc0040c8530 tx=0x7fbff0008040 comp rx=0 tx=0).stop 2024-09-13T18:22:07.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:07.056+0000 7fc0099f9700 1 -- 172.21.15.40:0/2338231060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbff4004db0 msgr2=0x7fc004154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:07.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:07.056+0000 7fc0099f9700 1 --2- 172.21.15.40:0/2338231060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbff4004db0 0x7fc004154c80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbff80017a0 tx=0x7fbff8004e60 comp rx=0 tx=0).stop 2024-09-13T18:22:07.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:07.056+0000 7fc0099f9700 1 -- 172.21.15.40:0/2338231060 shutdown_connections 2024-09-13T18:22:07.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:07.056+0000 7fc0099f9700 1 --2- 172.21.15.40:0/2338231060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0041f1fd0 0x7fc004150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:07.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:07.056+0000 7fc0099f9700 1 --2- 172.21.15.40:0/2338231060 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fbfe8066730 0x7fbfe8068bf0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:07.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:07.056+0000 7fc0099f9700 1 --2- 172.21.15.40:0/2338231060 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc0040aee40 0x7fc0041f1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:07.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:07.056+0000 7fc0099f9700 1 --2- 172.21.15.40:0/2338231060 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbff4004db0 0x7fc004154c80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:07.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:07.056+0000 7fc0099f9700 1 -- 172.21.15.40:0/2338231060 >> 172.21.15.40:0/2338231060 conn(0x7fc00406e4f0 msgr2=0x7fc0040cad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:07.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:07.056+0000 7fc0099f9700 1 -- 172.21.15.40:0/2338231060 shutdown_connections 2024-09-13T18:22:07.072 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:07.056+0000 7fc0099f9700 1 -- 172.21.15.40:0/2338231060 wait complete. 2024-09-13T18:22:07.073 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-13T18:22:07.079 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":972,"stamp":"2024-09-13T18:22:05.788342+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":16532352,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":237,"num_read_kb":279,"num_write":755012,"num_write_kb":16125724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15000,"num_omap_keys":30,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183120,"ondisk_log_size":183120,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10723352,"kb_used_data":34648,"kb_used_omap":12,"kb_used_meta":10688627,"kb_avail":744251368,"statfs":{"total":773094113280,"available":762113400832,"internally_reserved":0,"allocated":35479552,"data_stored":33992096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":10945154616},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003431"},"pg_stats":[{"pgid":"2.1f","version":"27'272","reported_seq":316,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:30.247079+0000","last_change":"2024-09-13T18:21:30.247079+0000","last_active":"2024-09-13T18:21:30.247079+0000","last_peered":"2024-09-13T18:21:30.247079+0000","last_clean":"2024-09-13T18:21:30.247079+0000","last_became_active":"2024-09-13T17:50:34.211867+0000","last_became_peered":"2024-09-13T17:50:34.211867+0000","last_unstale":"2024-09-13T18:21:30.247079+0000","last_undegraded":"2024-09-13T18:21:30.247079+0000","last_fullsized":"2024-09-13T18:21:30.247079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'272","last_scrub_stamp":"2024-09-13T18:21:30.246880+0000","last_deep_scrub":"27'272","last_deep_scrub_stamp":"2024-09-13T18:21:30.246880+0000","last_clean_scrub_stamp":"2024-09-13T18:21:30.246880+0000","objects_scrubbed":5,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:57:30.406058+0000","scrub_duration":0.02016366,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3260943,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":267,"num_write_kb":3318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'10","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:51.192502+0000","last_change":"2024-09-13T18:21:51.192810+0000","last_active":"2024-09-13T18:21:51.192502+0000","last_peered":"2024-09-13T18:21:51.192502+0000","last_clean":"2024-09-13T18:21:51.192502+0000","last_became_active":"2024-09-13T17:50:34.587108+0000","last_became_peered":"2024-09-13T17:50:34.587108+0000","last_unstale":"2024-09-13T18:21:51.192502+0000","last_undegraded":"2024-09-13T18:21:51.192502+0000","last_fullsized":"2024-09-13T18:21:51.192502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-13T18:21:51.192437+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-13T18:21:51.192437+0000","last_clean_scrub_stamp":"2024-09-13T18:21:51.192437+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:50.505781+0000","scrub_duration":0.0029908370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'5594","reported_seq":5621,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.921467+0000","last_change":"2024-09-13T17:50:38.615059+0000","last_active":"2024-09-13T18:21:02.921467+0000","last_peered":"2024-09-13T18:21:02.921467+0000","last_clean":"2024-09-13T18:21:02.921467+0000","last_became_active":"2024-09-13T17:50:38.614723+0000","last_became_peered":"2024-09-13T17:50:38.614723+0000","last_unstale":"2024-09-13T18:21:02.921467+0000","last_undegraded":"2024-09-13T18:21:02.921467+0000","last_fullsized":"2024-09-13T18:21:02.921467+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5594,"log_dups_size":0,"ondisk_log_size":5594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23732,"num_write_kb":533374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'6","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:36.210186+0000","last_change":"2024-09-13T18:21:36.210368+0000","last_active":"2024-09-13T18:21:36.210186+0000","last_peered":"2024-09-13T18:21:36.210186+0000","last_clean":"2024-09-13T18:21:36.210186+0000","last_became_active":"2024-09-13T17:50:34.202930+0000","last_became_peered":"2024-09-13T17:50:34.202930+0000","last_unstale":"2024-09-13T18:21:36.210186+0000","last_undegraded":"2024-09-13T18:21:36.210186+0000","last_fullsized":"2024-09-13T18:21:36.210186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-09-13T18:21:36.210139+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-09-13T18:21:36.210139+0000","last_clean_scrub_stamp":"2024-09-13T18:21:36.210139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:56.166821+0000","scrub_duration":0.0028089339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'20","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:22.640193+0000","last_change":"2024-09-13T17:51:16.366352+0000","last_active":"2024-09-13T18:21:22.640193+0000","last_peered":"2024-09-13T18:21:22.640193+0000","last_clean":"2024-09-13T18:21:22.640193+0000","last_became_active":"2024-09-13T17:51:16.365492+0000","last_became_peered":"2024-09-13T17:51:16.365492+0000","last_unstale":"2024-09-13T18:21:22.640193+0000","last_undegraded":"2024-09-13T18:21:22.640193+0000","last_fullsized":"2024-09-13T18:21:22.640193+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'5615","reported_seq":5653,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:48.296560+0000","last_change":"2024-09-13T18:21:48.296837+0000","last_active":"2024-09-13T18:21:48.296560+0000","last_peered":"2024-09-13T18:21:48.296560+0000","last_clean":"2024-09-13T18:21:48.296560+0000","last_became_active":"2024-09-13T17:50:38.618423+0000","last_became_peered":"2024-09-13T17:50:38.618423+0000","last_unstale":"2024-09-13T18:21:48.296560+0000","last_undegraded":"2024-09-13T18:21:48.296560+0000","last_fullsized":"2024-09-13T18:21:48.296560+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5615","last_scrub_stamp":"2024-09-13T18:21:48.296488+0000","last_deep_scrub":"27'5615","last_deep_scrub_stamp":"2024-09-13T18:21:48.296488+0000","last_clean_scrub_stamp":"2024-09-13T18:21:48.296488+0000","objects_scrubbed":0,"log_size":5615,"log_dups_size":0,"ondisk_log_size":5615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:01.155282+0000","scrub_duration":0.034902473000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23762,"num_write_kb":516686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'13","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:00.532888+0000","last_change":"2024-09-13T18:22:00.533193+0000","last_active":"2024-09-13T18:22:00.532888+0000","last_peered":"2024-09-13T18:22:00.532888+0000","last_clean":"2024-09-13T18:22:00.532888+0000","last_became_active":"2024-09-13T17:50:34.214652+0000","last_became_peered":"2024-09-13T17:50:34.214652+0000","last_unstale":"2024-09-13T18:22:00.532888+0000","last_undegraded":"2024-09-13T18:22:00.532888+0000","last_fullsized":"2024-09-13T18:22:00.532888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'13","last_scrub_stamp":"2024-09-13T18:22:00.532833+0000","last_deep_scrub":"27'13","last_deep_scrub_stamp":"2024-09-13T18:22:00.532833+0000","last_clean_scrub_stamp":"2024-09-13T18:22:00.532833+0000","objects_scrubbed":2,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:05.246113+0000","scrub_duration":0.0070529670000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'18","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:01.707673+0000","last_change":"2024-09-13T18:22:01.707968+0000","last_active":"2024-09-13T18:22:01.707673+0000","last_peered":"2024-09-13T18:22:01.707673+0000","last_clean":"2024-09-13T18:22:01.707673+0000","last_became_active":"2024-09-13T17:51:16.281549+0000","last_became_peered":"2024-09-13T17:51:16.281549+0000","last_unstale":"2024-09-13T18:22:01.707673+0000","last_undegraded":"2024-09-13T18:22:01.707673+0000","last_fullsized":"2024-09-13T18:22:01.707673+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:22:01.707624+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:22:01.707624+0000","last_clean_scrub_stamp":"2024-09-13T18:22:01.707624+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:30.522086+0000","scrub_duration":0.0032846300000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"27'5142","reported_seq":5180,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:53.315998+0000","last_change":"2024-09-13T18:21:53.316137+0000","last_active":"2024-09-13T18:21:53.315998+0000","last_peered":"2024-09-13T18:21:53.315998+0000","last_clean":"2024-09-13T18:21:53.315998+0000","last_became_active":"2024-09-13T17:50:38.614329+0000","last_became_peered":"2024-09-13T17:50:38.614329+0000","last_unstale":"2024-09-13T18:21:53.315998+0000","last_undegraded":"2024-09-13T18:21:53.315998+0000","last_fullsized":"2024-09-13T18:21:53.315998+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5142","last_scrub_stamp":"2024-09-13T18:21:53.315976+0000","last_deep_scrub":"27'5142","last_deep_scrub_stamp":"2024-09-13T18:21:53.315976+0000","last_clean_scrub_stamp":"2024-09-13T18:21:53.315976+0000","objects_scrubbed":0,"log_size":5142,"log_dups_size":0,"ondisk_log_size":5142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:24.336476+0000","scrub_duration":0.034529740000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21334,"num_write_kb":478261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'9","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:52.469363+0000","last_change":"2024-09-13T18:21:52.469614+0000","last_active":"2024-09-13T18:21:52.469363+0000","last_peered":"2024-09-13T18:21:52.469363+0000","last_clean":"2024-09-13T18:21:52.469363+0000","last_became_active":"2024-09-13T17:50:34.215661+0000","last_became_peered":"2024-09-13T17:50:34.215661+0000","last_unstale":"2024-09-13T18:21:52.469363+0000","last_undegraded":"2024-09-13T18:21:52.469363+0000","last_fullsized":"2024-09-13T18:21:52.469363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-09-13T18:21:52.469312+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-09-13T18:21:52.469312+0000","last_clean_scrub_stamp":"2024-09-13T18:21:52.469312+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:38:08.307554+0000","scrub_duration":0.0060588509999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'30","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:57.810967+0000","last_change":"2024-09-13T18:21:57.811166+0000","last_active":"2024-09-13T18:21:57.810967+0000","last_peered":"2024-09-13T18:21:57.810967+0000","last_clean":"2024-09-13T18:21:57.810967+0000","last_became_active":"2024-09-13T17:50:34.599409+0000","last_became_peered":"2024-09-13T17:50:34.599409+0000","last_unstale":"2024-09-13T18:21:57.810967+0000","last_undegraded":"2024-09-13T18:21:57.810967+0000","last_fullsized":"2024-09-13T18:21:57.810967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'30","last_scrub_stamp":"2024-09-13T18:21:57.810914+0000","last_deep_scrub":"27'30","last_deep_scrub_stamp":"2024-09-13T18:21:57.810914+0000","last_clean_scrub_stamp":"2024-09-13T18:21:57.810914+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:17:02.009800+0000","scrub_duration":0.003036955,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'4871","reported_seq":4909,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:49.336009+0000","last_change":"2024-09-13T18:21:49.336395+0000","last_active":"2024-09-13T18:21:49.336009+0000","last_peered":"2024-09-13T18:21:49.336009+0000","last_clean":"2024-09-13T18:21:49.336009+0000","last_became_active":"2024-09-13T17:50:38.614311+0000","last_became_peered":"2024-09-13T17:50:38.614311+0000","last_unstale":"2024-09-13T18:21:49.336009+0000","last_undegraded":"2024-09-13T18:21:49.336009+0000","last_fullsized":"2024-09-13T18:21:49.336009+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4871","last_scrub_stamp":"2024-09-13T18:21:49.335944+0000","last_deep_scrub":"27'4871","last_deep_scrub_stamp":"2024-09-13T18:21:49.335944+0000","last_clean_scrub_stamp":"2024-09-13T18:21:49.335944+0000","objects_scrubbed":0,"log_size":4871,"log_dups_size":0,"ondisk_log_size":4871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:12.664992+0000","scrub_duration":0.039294008999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20106,"num_write_kb":432357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'14","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:43.433109+0000","last_change":"2024-09-13T18:21:43.433365+0000","last_active":"2024-09-13T18:21:43.433109+0000","last_peered":"2024-09-13T18:21:43.433109+0000","last_clean":"2024-09-13T18:21:43.433109+0000","last_became_active":"2024-09-13T17:50:34.599207+0000","last_became_peered":"2024-09-13T17:50:34.599207+0000","last_unstale":"2024-09-13T18:21:43.433109+0000","last_undegraded":"2024-09-13T18:21:43.433109+0000","last_fullsized":"2024-09-13T18:21:43.433109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-09-13T18:21:43.433055+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-09-13T18:21:43.433055+0000","last_clean_scrub_stamp":"2024-09-13T18:21:43.433055+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:28.085068+0000","scrub_duration":0.003161358,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'1095","reported_seq":1137,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:56.570106+0000","last_change":"2024-09-13T18:21:56.570301+0000","last_active":"2024-09-13T18:21:56.570106+0000","last_peered":"2024-09-13T18:21:56.570106+0000","last_clean":"2024-09-13T18:21:56.570106+0000","last_became_active":"2024-09-13T17:50:34.215409+0000","last_became_peered":"2024-09-13T17:50:34.215409+0000","last_unstale":"2024-09-13T18:21:56.570106+0000","last_undegraded":"2024-09-13T18:21:56.570106+0000","last_fullsized":"2024-09-13T18:21:56.570106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'1095","last_scrub_stamp":"2024-09-13T18:21:56.570057+0000","last_deep_scrub":"27'1095","last_deep_scrub_stamp":"2024-09-13T18:21:56.570057+0000","last_clean_scrub_stamp":"2024-09-13T18:21:56.570057+0000","objects_scrubbed":3,"log_size":1095,"log_dups_size":0,"ondisk_log_size":1095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:35:11.168274+0000","scrub_duration":0.023788684000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1088,"num_write_kb":4339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'5394","reported_seq":5433,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:57.350615+0000","last_change":"2024-09-13T18:21:57.350995+0000","last_active":"2024-09-13T18:21:57.350615+0000","last_peered":"2024-09-13T18:21:57.350615+0000","last_clean":"2024-09-13T18:21:57.350615+0000","last_became_active":"2024-09-13T17:50:38.611263+0000","last_became_peered":"2024-09-13T17:50:38.611263+0000","last_unstale":"2024-09-13T18:21:57.350615+0000","last_undegraded":"2024-09-13T18:21:57.350615+0000","last_fullsized":"2024-09-13T18:21:57.350615+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5394","last_scrub_stamp":"2024-09-13T18:21:57.350563+0000","last_deep_scrub":"27'5394","last_deep_scrub_stamp":"2024-09-13T18:21:57.350563+0000","last_clean_scrub_stamp":"2024-09-13T18:21:57.350563+0000","objects_scrubbed":0,"log_size":5394,"log_dups_size":0,"ondisk_log_size":5394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:17.207282+0000","scrub_duration":0.032566444,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22326,"num_write_kb":494524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'19","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:52.230080+0000","last_change":"2024-09-13T18:21:52.230080+0000","last_active":"2024-09-13T18:21:52.230080+0000","last_peered":"2024-09-13T18:21:52.230080+0000","last_clean":"2024-09-13T18:21:52.230080+0000","last_became_active":"2024-09-13T17:50:34.585831+0000","last_became_peered":"2024-09-13T17:50:34.585831+0000","last_unstale":"2024-09-13T18:21:52.230080+0000","last_undegraded":"2024-09-13T18:21:52.230080+0000","last_fullsized":"2024-09-13T18:21:52.230080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-13T18:21:52.229769+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-13T18:21:52.229769+0000","last_clean_scrub_stamp":"2024-09-13T18:21:52.229769+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:09.681178+0000","scrub_duration":0.0031441960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'10","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:34.350232+0000","last_change":"2024-09-13T18:21:34.350402+0000","last_active":"2024-09-13T18:21:34.350232+0000","last_peered":"2024-09-13T18:21:34.350232+0000","last_clean":"2024-09-13T18:21:34.350232+0000","last_became_active":"2024-09-13T17:50:34.211594+0000","last_became_peered":"2024-09-13T17:50:34.211594+0000","last_unstale":"2024-09-13T18:21:34.350232+0000","last_undegraded":"2024-09-13T18:21:34.350232+0000","last_fullsized":"2024-09-13T18:21:34.350232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-13T18:21:34.350185+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-13T18:21:34.350185+0000","last_clean_scrub_stamp":"2024-09-13T18:21:34.350185+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:03.648370+0000","scrub_duration":0.003876764,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'5504","reported_seq":5543,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:56.359229+0000","last_change":"2024-09-13T18:21:56.359423+0000","last_active":"2024-09-13T18:21:56.359229+0000","last_peered":"2024-09-13T18:21:56.359229+0000","last_clean":"2024-09-13T18:21:56.359229+0000","last_became_active":"2024-09-13T17:50:38.612577+0000","last_became_peered":"2024-09-13T17:50:38.612577+0000","last_unstale":"2024-09-13T18:21:56.359229+0000","last_undegraded":"2024-09-13T18:21:56.359229+0000","last_fullsized":"2024-09-13T18:21:56.359229+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5504","last_scrub_stamp":"2024-09-13T18:21:56.359197+0000","last_deep_scrub":"27'5504","last_deep_scrub_stamp":"2024-09-13T18:21:56.359197+0000","last_clean_scrub_stamp":"2024-09-13T18:21:56.359197+0000","objects_scrubbed":0,"log_size":5504,"log_dups_size":0,"ondisk_log_size":5504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:28.613391+0000","scrub_duration":0.033662368999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23286,"num_write_kb":502495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'19","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:48.195751+0000","last_change":"2024-09-13T18:21:48.195940+0000","last_active":"2024-09-13T18:21:48.195751+0000","last_peered":"2024-09-13T18:21:48.195751+0000","last_clean":"2024-09-13T18:21:48.195751+0000","last_became_active":"2024-09-13T17:50:34.587027+0000","last_became_peered":"2024-09-13T17:50:34.587027+0000","last_unstale":"2024-09-13T18:21:48.195751+0000","last_undegraded":"2024-09-13T18:21:48.195751+0000","last_fullsized":"2024-09-13T18:21:48.195751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-13T18:21:48.195703+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-13T18:21:48.195703+0000","last_clean_scrub_stamp":"2024-09-13T18:21:48.195703+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:55.755022+0000","scrub_duration":0.0031326050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'7","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:32.340544+0000","last_change":"2024-09-13T18:21:32.340739+0000","last_active":"2024-09-13T18:21:32.340544+0000","last_peered":"2024-09-13T18:21:32.340544+0000","last_clean":"2024-09-13T18:21:32.340544+0000","last_became_active":"2024-09-13T17:50:34.211568+0000","last_became_peered":"2024-09-13T17:50:34.211568+0000","last_unstale":"2024-09-13T18:21:32.340544+0000","last_undegraded":"2024-09-13T18:21:32.340544+0000","last_fullsized":"2024-09-13T18:21:32.340544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-13T18:21:32.340496+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-13T18:21:32.340496+0000","last_clean_scrub_stamp":"2024-09-13T18:21:32.340496+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:14:08.632192+0000","scrub_duration":0.0064919820000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'6938","reported_seq":6977,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:01.462764+0000","last_change":"2024-09-13T18:22:01.462926+0000","last_active":"2024-09-13T18:22:01.462764+0000","last_peered":"2024-09-13T18:22:01.462764+0000","last_clean":"2024-09-13T18:22:01.462764+0000","last_became_active":"2024-09-13T17:50:38.609832+0000","last_became_peered":"2024-09-13T17:50:38.609832+0000","last_unstale":"2024-09-13T18:22:01.462764+0000","last_undegraded":"2024-09-13T18:22:01.462764+0000","last_fullsized":"2024-09-13T18:22:01.462764+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6938","last_scrub_stamp":"2024-09-13T18:22:01.462726+0000","last_deep_scrub":"27'6938","last_deep_scrub_stamp":"2024-09-13T18:22:01.462726+0000","last_clean_scrub_stamp":"2024-09-13T18:22:01.462726+0000","objects_scrubbed":0,"log_size":6938,"log_dups_size":0,"ondisk_log_size":6938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:07.849758+0000","scrub_duration":0.053454132000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28644,"num_write_kb":622131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"27'12","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:33.269238+0000","last_change":"2024-09-13T18:21:33.269565+0000","last_active":"2024-09-13T18:21:33.269238+0000","last_peered":"2024-09-13T18:21:33.269238+0000","last_clean":"2024-09-13T18:21:33.269238+0000","last_became_active":"2024-09-13T17:50:34.588794+0000","last_became_peered":"2024-09-13T17:50:34.588794+0000","last_unstale":"2024-09-13T18:21:33.269238+0000","last_undegraded":"2024-09-13T18:21:33.269238+0000","last_fullsized":"2024-09-13T18:21:33.269238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'12","last_scrub_stamp":"2024-09-13T18:21:33.269111+0000","last_deep_scrub":"27'12","last_deep_scrub_stamp":"2024-09-13T18:21:33.269111+0000","last_clean_scrub_stamp":"2024-09-13T18:21:33.269111+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:41.084595+0000","scrub_duration":0.003356562,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'7","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:59.510367+0000","last_change":"2024-09-13T18:21:59.510653+0000","last_active":"2024-09-13T18:21:59.510367+0000","last_peered":"2024-09-13T18:21:59.510367+0000","last_clean":"2024-09-13T18:21:59.510367+0000","last_became_active":"2024-09-13T17:50:34.214292+0000","last_became_peered":"2024-09-13T17:50:34.214292+0000","last_unstale":"2024-09-13T18:21:59.510367+0000","last_undegraded":"2024-09-13T18:21:59.510367+0000","last_fullsized":"2024-09-13T18:21:59.510367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-13T18:21:59.510312+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-13T18:21:59.510312+0000","last_clean_scrub_stamp":"2024-09-13T18:21:59.510312+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:39.341902+0000","scrub_duration":0.0065891600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'4588","reported_seq":4627,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:59.338181+0000","last_change":"2024-09-13T18:21:59.338480+0000","last_active":"2024-09-13T18:21:59.338181+0000","last_peered":"2024-09-13T18:21:59.338181+0000","last_clean":"2024-09-13T18:21:59.338181+0000","last_became_active":"2024-09-13T17:50:38.613010+0000","last_became_peered":"2024-09-13T17:50:38.613010+0000","last_unstale":"2024-09-13T18:21:59.338181+0000","last_undegraded":"2024-09-13T18:21:59.338181+0000","last_fullsized":"2024-09-13T18:21:59.338181+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4588","last_scrub_stamp":"2024-09-13T18:21:59.338108+0000","last_deep_scrub":"27'4588","last_deep_scrub_stamp":"2024-09-13T18:21:59.338108+0000","last_clean_scrub_stamp":"2024-09-13T18:21:59.338108+0000","objects_scrubbed":0,"log_size":4588,"log_dups_size":0,"ondisk_log_size":4588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:34.989321+0000","scrub_duration":0.027635435999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19354,"num_write_kb":454021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'26","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:50.314073+0000","last_change":"2024-09-13T18:21:50.314318+0000","last_active":"2024-09-13T18:21:50.314073+0000","last_peered":"2024-09-13T18:21:50.314073+0000","last_clean":"2024-09-13T18:21:50.314073+0000","last_became_active":"2024-09-13T17:50:34.582050+0000","last_became_peered":"2024-09-13T17:50:34.582050+0000","last_unstale":"2024-09-13T18:21:50.314073+0000","last_undegraded":"2024-09-13T18:21:50.314073+0000","last_fullsized":"2024-09-13T18:21:50.314073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'26","last_scrub_stamp":"2024-09-13T18:21:50.314024+0000","last_deep_scrub":"27'26","last_deep_scrub_stamp":"2024-09-13T18:21:50.314024+0000","last_clean_scrub_stamp":"2024-09-13T18:21:50.314024+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:48:00.837821+0000","scrub_duration":0.0030625299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'11","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:32.305631+0000","last_change":"2024-09-13T18:21:32.306235+0000","last_active":"2024-09-13T18:21:32.305631+0000","last_peered":"2024-09-13T18:21:32.305631+0000","last_clean":"2024-09-13T18:21:32.305631+0000","last_became_active":"2024-09-13T17:50:34.211230+0000","last_became_peered":"2024-09-13T17:50:34.211230+0000","last_unstale":"2024-09-13T18:21:32.305631+0000","last_undegraded":"2024-09-13T18:21:32.305631+0000","last_fullsized":"2024-09-13T18:21:32.305631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-09-13T18:21:32.305579+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-09-13T18:21:32.305579+0000","last_clean_scrub_stamp":"2024-09-13T18:21:32.305579+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:08.984532+0000","scrub_duration":0.0066305000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'6024","reported_seq":6065,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:54.336876+0000","last_change":"2024-09-13T18:21:54.336876+0000","last_active":"2024-09-13T18:21:54.336876+0000","last_peered":"2024-09-13T18:21:54.336876+0000","last_clean":"2024-09-13T18:21:54.336876+0000","last_became_active":"2024-09-13T17:50:38.606697+0000","last_became_peered":"2024-09-13T17:50:38.606697+0000","last_unstale":"2024-09-13T18:21:54.336876+0000","last_undegraded":"2024-09-13T18:21:54.336876+0000","last_fullsized":"2024-09-13T18:21:54.336876+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6024","last_scrub_stamp":"2024-09-13T18:21:54.336672+0000","last_deep_scrub":"27'6024","last_deep_scrub_stamp":"2024-09-13T18:21:54.336672+0000","last_clean_scrub_stamp":"2024-09-13T18:21:54.336672+0000","objects_scrubbed":0,"log_size":6024,"log_dups_size":0,"ondisk_log_size":6024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:56.337299+0000","scrub_duration":0.040808092999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25552,"num_write_kb":507487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'18","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:45.222534+0000","last_change":"2024-09-13T18:21:45.222779+0000","last_active":"2024-09-13T18:21:45.222534+0000","last_peered":"2024-09-13T18:21:45.222534+0000","last_clean":"2024-09-13T18:21:45.222534+0000","last_became_active":"2024-09-13T17:50:34.594215+0000","last_became_peered":"2024-09-13T17:50:34.594215+0000","last_unstale":"2024-09-13T18:21:45.222534+0000","last_undegraded":"2024-09-13T18:21:45.222534+0000","last_fullsized":"2024-09-13T18:21:45.222534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:21:45.222475+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:21:45.222475+0000","last_clean_scrub_stamp":"2024-09-13T18:21:45.222475+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:21.803809+0000","scrub_duration":0.0034298839999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'286","reported_seq":329,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:36.413084+0000","last_change":"2024-09-13T18:21:36.413202+0000","last_active":"2024-09-13T18:21:36.413084+0000","last_peered":"2024-09-13T18:21:36.413084+0000","last_clean":"2024-09-13T18:21:36.413084+0000","last_became_active":"2024-09-13T17:50:34.203702+0000","last_became_peered":"2024-09-13T17:50:34.203702+0000","last_unstale":"2024-09-13T18:21:36.413084+0000","last_undegraded":"2024-09-13T18:21:36.413084+0000","last_fullsized":"2024-09-13T18:21:36.413084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'286","last_scrub_stamp":"2024-09-13T18:21:36.413050+0000","last_deep_scrub":"27'286","last_deep_scrub_stamp":"2024-09-13T18:21:36.413050+0000","last_clean_scrub_stamp":"2024-09-13T18:21:36.413050+0000","objects_scrubbed":2,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:44.535556+0000","scrub_duration":0.0068211640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":605,"num_write_kb":197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'6030","reported_seq":6070,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.838464+0000","last_change":"2024-09-13T17:51:16.365782+0000","last_active":"2024-09-13T18:21:02.838464+0000","last_peered":"2024-09-13T18:21:02.838464+0000","last_clean":"2024-09-13T18:21:02.838464+0000","last_became_active":"2024-09-13T17:51:16.364540+0000","last_became_peered":"2024-09-13T17:51:16.364540+0000","last_unstale":"2024-09-13T18:21:02.838464+0000","last_undegraded":"2024-09-13T18:21:02.838464+0000","last_fullsized":"2024-09-13T18:21:02.838464+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6030,"log_dups_size":0,"ondisk_log_size":6030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25617,"num_write_kb":548663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'18","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:50.405887+0000","last_change":"2024-09-13T18:21:50.405887+0000","last_active":"2024-09-13T18:21:50.405887+0000","last_peered":"2024-09-13T18:21:50.405887+0000","last_clean":"2024-09-13T18:21:50.405887+0000","last_became_active":"2024-09-13T17:50:34.586552+0000","last_became_peered":"2024-09-13T17:50:34.586552+0000","last_unstale":"2024-09-13T18:21:50.405887+0000","last_undegraded":"2024-09-13T18:21:50.405887+0000","last_fullsized":"2024-09-13T18:21:50.405887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:21:50.405568+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:21:50.405568+0000","last_clean_scrub_stamp":"2024-09-13T18:21:50.405568+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:09.337911+0000","scrub_duration":0.0019550739999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'8","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:57.533976+0000","last_change":"2024-09-13T18:21:57.533976+0000","last_active":"2024-09-13T18:21:57.533976+0000","last_peered":"2024-09-13T18:21:57.533976+0000","last_clean":"2024-09-13T18:21:57.533976+0000","last_became_active":"2024-09-13T17:50:34.216758+0000","last_became_peered":"2024-09-13T17:50:34.216758+0000","last_unstale":"2024-09-13T18:21:57.533976+0000","last_undegraded":"2024-09-13T18:21:57.533976+0000","last_fullsized":"2024-09-13T18:21:57.533976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-09-13T18:21:57.533649+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-09-13T18:21:57.533649+0000","last_clean_scrub_stamp":"2024-09-13T18:21:57.533649+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:14.399317+0000","scrub_duration":0.0047334860000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'4887","reported_seq":4917,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:22.640175+0000","last_change":"2024-09-13T17:50:38.614667+0000","last_active":"2024-09-13T18:21:22.640175+0000","last_peered":"2024-09-13T18:21:22.640175+0000","last_clean":"2024-09-13T18:21:22.640175+0000","last_became_active":"2024-09-13T17:50:38.614284+0000","last_became_peered":"2024-09-13T17:50:38.614284+0000","last_unstale":"2024-09-13T18:21:22.640175+0000","last_undegraded":"2024-09-13T18:21:22.640175+0000","last_fullsized":"2024-09-13T18:21:22.640175+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4887,"log_dups_size":0,"ondisk_log_size":4887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20690,"num_write_kb":442963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'16","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:53.757522+0000","last_change":"2024-09-13T18:21:53.757522+0000","last_active":"2024-09-13T18:21:53.757522+0000","last_peered":"2024-09-13T18:21:53.757522+0000","last_clean":"2024-09-13T18:21:53.757522+0000","last_became_active":"2024-09-13T17:50:34.588317+0000","last_became_peered":"2024-09-13T17:50:34.588317+0000","last_unstale":"2024-09-13T18:21:53.757522+0000","last_undegraded":"2024-09-13T18:21:53.757522+0000","last_fullsized":"2024-09-13T18:21:53.757522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-09-13T18:21:53.757275+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-09-13T18:21:53.757275+0000","last_clean_scrub_stamp":"2024-09-13T18:21:53.757275+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:03:27.571838+0000","scrub_duration":0.003575366,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'3","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:49.455036+0000","last_change":"2024-09-13T18:21:49.455036+0000","last_active":"2024-09-13T18:21:49.455036+0000","last_peered":"2024-09-13T18:21:49.455036+0000","last_clean":"2024-09-13T18:21:49.455036+0000","last_became_active":"2024-09-13T17:50:34.204060+0000","last_became_peered":"2024-09-13T17:50:34.204060+0000","last_unstale":"2024-09-13T18:21:49.455036+0000","last_undegraded":"2024-09-13T18:21:49.455036+0000","last_fullsized":"2024-09-13T18:21:49.455036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-09-13T18:21:49.454804+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-09-13T18:21:49.454804+0000","last_clean_scrub_stamp":"2024-09-13T18:21:49.454804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:26.292901+0000","scrub_duration":0.0030762760000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'6031","reported_seq":6061,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:23.132543+0000","last_change":"2024-09-13T17:50:38.619590+0000","last_active":"2024-09-13T18:21:23.132543+0000","last_peered":"2024-09-13T18:21:23.132543+0000","last_clean":"2024-09-13T18:21:23.132543+0000","last_became_active":"2024-09-13T17:50:38.617257+0000","last_became_peered":"2024-09-13T17:50:38.617257+0000","last_unstale":"2024-09-13T18:21:23.132543+0000","last_undegraded":"2024-09-13T18:21:23.132543+0000","last_fullsized":"2024-09-13T18:21:23.132543+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6031,"log_dups_size":0,"ondisk_log_size":6031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25529,"num_write_kb":564991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'16","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:45.268600+0000","last_change":"2024-09-13T18:21:45.268600+0000","last_active":"2024-09-13T18:21:45.268600+0000","last_peered":"2024-09-13T18:21:45.268600+0000","last_clean":"2024-09-13T18:21:45.268600+0000","last_became_active":"2024-09-13T17:50:34.588063+0000","last_became_peered":"2024-09-13T17:50:34.588063+0000","last_unstale":"2024-09-13T18:21:45.268600+0000","last_undegraded":"2024-09-13T18:21:45.268600+0000","last_fullsized":"2024-09-13T18:21:45.268600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-09-13T18:21:45.268335+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-09-13T18:21:45.268335+0000","last_clean_scrub_stamp":"2024-09-13T18:21:45.268335+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:13:11.956726+0000","scrub_duration":0.0030026689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'4","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:35.893165+0000","last_change":"2024-09-13T18:21:35.893165+0000","last_active":"2024-09-13T18:21:35.893165+0000","last_peered":"2024-09-13T18:21:35.893165+0000","last_clean":"2024-09-13T18:21:35.893165+0000","last_became_active":"2024-09-13T17:50:34.208196+0000","last_became_peered":"2024-09-13T17:50:34.208196+0000","last_unstale":"2024-09-13T18:21:35.893165+0000","last_undegraded":"2024-09-13T18:21:35.893165+0000","last_fullsized":"2024-09-13T18:21:35.893165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-09-13T18:21:35.892908+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-09-13T18:21:35.892908+0000","last_clean_scrub_stamp":"2024-09-13T18:21:35.892908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:50.582547+0000","scrub_duration":0.0034542710000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'5410","reported_seq":5451,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:00.439430+0000","last_change":"2024-09-13T18:22:00.439430+0000","last_active":"2024-09-13T18:22:00.439430+0000","last_peered":"2024-09-13T18:22:00.439430+0000","last_clean":"2024-09-13T18:22:00.439430+0000","last_became_active":"2024-09-13T17:50:38.609794+0000","last_became_peered":"2024-09-13T17:50:38.609794+0000","last_unstale":"2024-09-13T18:22:00.439430+0000","last_undegraded":"2024-09-13T18:22:00.439430+0000","last_fullsized":"2024-09-13T18:22:00.439430+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5410","last_scrub_stamp":"2024-09-13T18:22:00.439240+0000","last_deep_scrub":"27'5410","last_deep_scrub_stamp":"2024-09-13T18:22:00.439240+0000","last_clean_scrub_stamp":"2024-09-13T18:22:00.439240+0000","objects_scrubbed":0,"log_size":5410,"log_dups_size":0,"ondisk_log_size":5410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:57:58.327181+0000","scrub_duration":0.041838701999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22186,"num_write_kb":506224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'22","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:52.457138+0000","last_change":"2024-09-13T18:21:52.457337+0000","last_active":"2024-09-13T18:21:52.457138+0000","last_peered":"2024-09-13T18:21:52.457138+0000","last_clean":"2024-09-13T18:21:52.457138+0000","last_became_active":"2024-09-13T17:50:34.584023+0000","last_became_peered":"2024-09-13T17:50:34.584023+0000","last_unstale":"2024-09-13T18:21:52.457138+0000","last_undegraded":"2024-09-13T18:21:52.457138+0000","last_fullsized":"2024-09-13T18:21:52.457138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-09-13T18:21:52.457074+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-09-13T18:21:52.457074+0000","last_clean_scrub_stamp":"2024-09-13T18:21:52.457074+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:06:10.138005+0000","scrub_duration":0.0030016969999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'130","reported_seq":295,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:43.721726+0000","last_change":"2024-09-13T18:21:43.721726+0000","last_active":"2024-09-13T18:21:43.721726+0000","last_peered":"2024-09-13T18:21:43.721726+0000","last_clean":"2024-09-13T18:21:43.721726+0000","last_became_active":"2024-09-13T17:50:34.198603+0000","last_became_peered":"2024-09-13T17:50:34.198603+0000","last_unstale":"2024-09-13T18:21:43.721726+0000","last_undegraded":"2024-09-13T18:21:43.721726+0000","last_fullsized":"2024-09-13T18:21:43.721726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'130","last_scrub_stamp":"2024-09-13T18:21:43.721576+0000","last_deep_scrub":"27'130","last_deep_scrub_stamp":"2024-09-13T18:21:43.721576+0000","last_clean_scrub_stamp":"2024-09-13T18:21:43.721576+0000","objects_scrubbed":3,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:31.923344+0000","scrub_duration":0.0076490960000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":94032,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":179,"num_write":125,"num_write_kb":180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'5807","reported_seq":5847,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.539082+0000","last_change":"2024-09-13T18:21:55.539241+0000","last_active":"2024-09-13T18:21:55.539082+0000","last_peered":"2024-09-13T18:21:55.539082+0000","last_clean":"2024-09-13T18:21:55.539082+0000","last_became_active":"2024-09-13T17:50:38.610900+0000","last_became_peered":"2024-09-13T17:50:38.610900+0000","last_unstale":"2024-09-13T18:21:55.539082+0000","last_undegraded":"2024-09-13T18:21:55.539082+0000","last_fullsized":"2024-09-13T18:21:55.539082+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5807","last_scrub_stamp":"2024-09-13T18:21:55.539055+0000","last_deep_scrub":"27'5807","last_deep_scrub_stamp":"2024-09-13T18:21:55.539055+0000","last_clean_scrub_stamp":"2024-09-13T18:21:55.539055+0000","objects_scrubbed":0,"log_size":5807,"log_dups_size":0,"ondisk_log_size":5807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:19.172462+0000","scrub_duration":0.050879777000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24429,"num_write_kb":531605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'13","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:32.284016+0000","last_change":"2024-09-13T18:21:32.284016+0000","last_active":"2024-09-13T18:21:32.284016+0000","last_peered":"2024-09-13T18:21:32.284016+0000","last_clean":"2024-09-13T18:21:32.284016+0000","last_became_active":"2024-09-13T17:50:34.594201+0000","last_became_peered":"2024-09-13T17:50:34.594201+0000","last_unstale":"2024-09-13T18:21:32.284016+0000","last_undegraded":"2024-09-13T18:21:32.284016+0000","last_fullsized":"2024-09-13T18:21:32.284016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'13","last_scrub_stamp":"2024-09-13T18:21:32.283704+0000","last_deep_scrub":"27'13","last_deep_scrub_stamp":"2024-09-13T18:21:32.283704+0000","last_clean_scrub_stamp":"2024-09-13T18:21:32.283704+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:06.348602+0000","scrub_duration":0.0033353039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'9","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:45.449467+0000","last_change":"2024-09-13T18:21:45.449664+0000","last_active":"2024-09-13T18:21:45.449467+0000","last_peered":"2024-09-13T18:21:45.449467+0000","last_clean":"2024-09-13T18:21:45.449467+0000","last_became_active":"2024-09-13T17:50:34.218069+0000","last_became_peered":"2024-09-13T17:50:34.218069+0000","last_unstale":"2024-09-13T18:21:45.449467+0000","last_undegraded":"2024-09-13T18:21:45.449467+0000","last_fullsized":"2024-09-13T18:21:45.449467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-09-13T18:21:45.449415+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-09-13T18:21:45.449415+0000","last_clean_scrub_stamp":"2024-09-13T18:21:45.449415+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:45.223728+0000","scrub_duration":0.0064944169999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'4957","reported_seq":4986,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.964133+0000","last_change":"2024-09-13T17:50:38.613206+0000","last_active":"2024-09-13T18:21:02.964133+0000","last_peered":"2024-09-13T18:21:02.964133+0000","last_clean":"2024-09-13T18:21:02.964133+0000","last_became_active":"2024-09-13T17:50:38.612856+0000","last_became_peered":"2024-09-13T17:50:38.612856+0000","last_unstale":"2024-09-13T18:21:02.964133+0000","last_undegraded":"2024-09-13T18:21:02.964133+0000","last_fullsized":"2024-09-13T18:21:02.964133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":4957,"log_dups_size":0,"ondisk_log_size":4957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20520,"num_write_kb":434212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'16","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:43.170888+0000","last_change":"2024-09-13T18:21:43.170888+0000","last_active":"2024-09-13T18:21:43.170888+0000","last_peered":"2024-09-13T18:21:43.170888+0000","last_clean":"2024-09-13T18:21:43.170888+0000","last_became_active":"2024-09-13T17:50:34.582446+0000","last_became_peered":"2024-09-13T17:50:34.582446+0000","last_unstale":"2024-09-13T18:21:43.170888+0000","last_undegraded":"2024-09-13T18:21:43.170888+0000","last_fullsized":"2024-09-13T18:21:43.170888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-09-13T18:21:43.170750+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-09-13T18:21:43.170750+0000","last_clean_scrub_stamp":"2024-09-13T18:21:43.170750+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:39.625283+0000","scrub_duration":0.0027720919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'11","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.797573+0000","last_change":"2024-09-13T18:21:28.797573+0000","last_active":"2024-09-13T18:21:28.797573+0000","last_peered":"2024-09-13T18:21:28.797573+0000","last_clean":"2024-09-13T18:21:28.797573+0000","last_became_active":"2024-09-13T17:50:34.200873+0000","last_became_peered":"2024-09-13T17:50:34.200873+0000","last_unstale":"2024-09-13T18:21:28.797573+0000","last_undegraded":"2024-09-13T18:21:28.797573+0000","last_fullsized":"2024-09-13T18:21:28.797573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-09-13T18:21:28.797316+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-09-13T18:21:28.797316+0000","last_clean_scrub_stamp":"2024-09-13T18:21:28.797316+0000","objects_scrubbed":3,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:51:50.633990+0000","scrub_duration":0.0090957169999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"27'119","reported_seq":199,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:41.366780+0000","last_change":"2024-09-13T18:21:41.367018+0000","last_active":"2024-09-13T18:21:41.366780+0000","last_peered":"2024-09-13T18:21:41.366780+0000","last_clean":"2024-09-13T18:21:41.366780+0000","last_became_active":"2024-09-13T17:50:22.093734+0000","last_became_peered":"2024-09-13T17:50:22.093734+0000","last_unstale":"2024-09-13T18:21:41.366780+0000","last_undegraded":"2024-09-13T18:21:41.366780+0000","last_fullsized":"2024-09-13T18:21:41.366780+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'119","last_scrub_stamp":"2024-09-13T18:21:41.366718+0000","last_deep_scrub":"27'119","last_deep_scrub_stamp":"2024-09-13T18:21:41.366718+0000","last_clean_scrub_stamp":"2024-09-13T18:21:41.366718+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:37:54.724540+0000","scrub_duration":0.0086725589999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'5208","reported_seq":5249,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:52.415372+0000","last_change":"2024-09-13T18:21:52.415372+0000","last_active":"2024-09-13T18:21:52.415372+0000","last_peered":"2024-09-13T18:21:52.415372+0000","last_clean":"2024-09-13T18:21:52.415372+0000","last_became_active":"2024-09-13T17:50:38.618488+0000","last_became_peered":"2024-09-13T17:50:38.618488+0000","last_unstale":"2024-09-13T18:21:52.415372+0000","last_undegraded":"2024-09-13T18:21:52.415372+0000","last_fullsized":"2024-09-13T18:21:52.415372+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5208","last_scrub_stamp":"2024-09-13T18:21:52.415203+0000","last_deep_scrub":"27'5208","last_deep_scrub_stamp":"2024-09-13T18:21:52.415203+0000","last_clean_scrub_stamp":"2024-09-13T18:21:52.415203+0000","objects_scrubbed":0,"log_size":5208,"log_dups_size":0,"ondisk_log_size":5208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:21.608219+0000","scrub_duration":0.035964871000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22093,"num_write_kb":481370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'22","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.318438+0000","last_change":"2024-09-13T17:50:34.585881+0000","last_active":"2024-09-13T18:21:13.318438+0000","last_peered":"2024-09-13T18:21:13.318438+0000","last_clean":"2024-09-13T18:21:13.318438+0000","last_became_active":"2024-09-13T17:50:34.585166+0000","last_became_peered":"2024-09-13T17:50:34.585166+0000","last_unstale":"2024-09-13T18:21:13.318438+0000","last_undegraded":"2024-09-13T18:21:13.318438+0000","last_fullsized":"2024-09-13T18:21:13.318438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'7","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:34.775170+0000","last_change":"2024-09-13T18:21:34.775276+0000","last_active":"2024-09-13T18:21:34.775170+0000","last_peered":"2024-09-13T18:21:34.775170+0000","last_clean":"2024-09-13T18:21:34.775170+0000","last_became_active":"2024-09-13T17:50:34.210589+0000","last_became_peered":"2024-09-13T17:50:34.210589+0000","last_unstale":"2024-09-13T18:21:34.775170+0000","last_undegraded":"2024-09-13T18:21:34.775170+0000","last_fullsized":"2024-09-13T18:21:34.775170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-13T18:21:34.775137+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-13T18:21:34.775137+0000","last_clean_scrub_stamp":"2024-09-13T18:21:34.775137+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:40.007622+0000","scrub_duration":0.0064282710000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'5302","reported_seq":5352,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:50.384839+0000","last_change":"2024-09-13T18:21:50.385188+0000","last_active":"2024-09-13T18:21:50.384839+0000","last_peered":"2024-09-13T18:21:50.384839+0000","last_clean":"2024-09-13T18:21:50.384839+0000","last_became_active":"2024-09-13T17:51:16.363974+0000","last_became_peered":"2024-09-13T17:51:16.363974+0000","last_unstale":"2024-09-13T18:21:50.384839+0000","last_undegraded":"2024-09-13T18:21:50.384839+0000","last_fullsized":"2024-09-13T18:21:50.384839+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5302","last_scrub_stamp":"2024-09-13T18:21:50.384756+0000","last_deep_scrub":"27'5302","last_deep_scrub_stamp":"2024-09-13T18:21:50.384756+0000","last_clean_scrub_stamp":"2024-09-13T18:21:50.384756+0000","objects_scrubbed":0,"log_size":5302,"log_dups_size":0,"ondisk_log_size":5302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:19.818627+0000","scrub_duration":0.048673297999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22724,"num_write_kb":488088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'20","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310467+0000","last_change":"2024-09-13T17:50:34.590169+0000","last_active":"2024-09-13T18:21:13.310467+0000","last_peered":"2024-09-13T18:21:13.310467+0000","last_clean":"2024-09-13T18:21:13.310467+0000","last_became_active":"2024-09-13T17:50:34.589434+0000","last_became_peered":"2024-09-13T17:50:34.589434+0000","last_unstale":"2024-09-13T18:21:13.310467+0000","last_undegraded":"2024-09-13T18:21:13.310467+0000","last_fullsized":"2024-09-13T18:21:13.310467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'5","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:33.383817+0000","last_change":"2024-09-13T18:21:33.384001+0000","last_active":"2024-09-13T18:21:33.383817+0000","last_peered":"2024-09-13T18:21:33.383817+0000","last_clean":"2024-09-13T18:21:33.383817+0000","last_became_active":"2024-09-13T17:50:34.203874+0000","last_became_peered":"2024-09-13T17:50:34.203874+0000","last_unstale":"2024-09-13T18:21:33.383817+0000","last_undegraded":"2024-09-13T18:21:33.383817+0000","last_fullsized":"2024-09-13T18:21:33.383817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-09-13T18:21:33.383770+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-09-13T18:21:33.383770+0000","last_clean_scrub_stamp":"2024-09-13T18:21:33.383770+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:07.542936+0000","scrub_duration":0.0056205279999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'5348","reported_seq":5376,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.945036+0000","last_change":"2024-09-13T17:50:38.615989+0000","last_active":"2024-09-13T18:21:02.945036+0000","last_peered":"2024-09-13T18:21:02.945036+0000","last_clean":"2024-09-13T18:21:02.945036+0000","last_became_active":"2024-09-13T17:50:38.615530+0000","last_became_peered":"2024-09-13T17:50:38.615530+0000","last_unstale":"2024-09-13T18:21:02.945036+0000","last_undegraded":"2024-09-13T18:21:02.945036+0000","last_fullsized":"2024-09-13T18:21:02.945036+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5348,"log_dups_size":0,"ondisk_log_size":5348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22134,"num_write_kb":473293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'20","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.322123+0000","last_change":"2024-09-13T17:50:34.595120+0000","last_active":"2024-09-13T18:21:13.322123+0000","last_peered":"2024-09-13T18:21:13.322123+0000","last_clean":"2024-09-13T18:21:13.322123+0000","last_became_active":"2024-09-13T17:50:34.594842+0000","last_became_peered":"2024-09-13T17:50:34.594842+0000","last_unstale":"2024-09-13T18:21:13.322123+0000","last_undegraded":"2024-09-13T18:21:13.322123+0000","last_fullsized":"2024-09-13T18:21:13.322123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'10","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:42.696086+0000","last_change":"2024-09-13T18:21:42.696280+0000","last_active":"2024-09-13T18:21:42.696086+0000","last_peered":"2024-09-13T18:21:42.696086+0000","last_clean":"2024-09-13T18:21:42.696086+0000","last_became_active":"2024-09-13T17:50:34.199650+0000","last_became_peered":"2024-09-13T17:50:34.199650+0000","last_unstale":"2024-09-13T18:21:42.696086+0000","last_undegraded":"2024-09-13T18:21:42.696086+0000","last_fullsized":"2024-09-13T18:21:42.696086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-13T18:21:42.696034+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-13T18:21:42.696034+0000","last_clean_scrub_stamp":"2024-09-13T18:21:42.696034+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:47:19.246483+0000","scrub_duration":0.0045639139999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'5161","reported_seq":5190,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.766124+0000","last_change":"2024-09-13T17:50:38.609731+0000","last_active":"2024-09-13T18:21:02.766124+0000","last_peered":"2024-09-13T18:21:02.766124+0000","last_clean":"2024-09-13T18:21:02.766124+0000","last_became_active":"2024-09-13T17:50:38.609354+0000","last_became_peered":"2024-09-13T17:50:38.609354+0000","last_unstale":"2024-09-13T18:21:02.766124+0000","last_undegraded":"2024-09-13T18:21:02.766124+0000","last_fullsized":"2024-09-13T18:21:02.766124+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5161,"log_dups_size":0,"ondisk_log_size":5161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21929,"num_write_kb":464406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'10","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:26.347034+0000","last_change":"2024-09-13T18:21:26.347263+0000","last_active":"2024-09-13T18:21:26.347034+0000","last_peered":"2024-09-13T18:21:26.347034+0000","last_clean":"2024-09-13T18:21:26.347034+0000","last_became_active":"2024-09-13T17:50:34.594876+0000","last_became_peered":"2024-09-13T17:50:34.594876+0000","last_unstale":"2024-09-13T18:21:26.347034+0000","last_undegraded":"2024-09-13T18:21:26.347034+0000","last_fullsized":"2024-09-13T18:21:26.347034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-13T18:21:26.346976+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-13T18:21:26.346976+0000","last_clean_scrub_stamp":"2024-09-13T18:21:26.346976+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:25.039233+0000","scrub_duration":0.003420939,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'6","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.264168+0000","last_change":"2024-09-13T18:21:28.264384+0000","last_active":"2024-09-13T18:21:28.264168+0000","last_peered":"2024-09-13T18:21:28.264168+0000","last_clean":"2024-09-13T18:21:28.264168+0000","last_became_active":"2024-09-13T17:50:34.200197+0000","last_became_peered":"2024-09-13T17:50:34.200197+0000","last_unstale":"2024-09-13T18:21:28.264168+0000","last_undegraded":"2024-09-13T18:21:28.264168+0000","last_fullsized":"2024-09-13T18:21:28.264168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-09-13T18:21:28.264112+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-09-13T18:21:28.264112+0000","last_clean_scrub_stamp":"2024-09-13T18:21:28.264112+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:43:40.034203+0000","scrub_duration":0.0069816829999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'5052","reported_seq":5102,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.356895+0000","last_change":"2024-09-13T18:21:55.357111+0000","last_active":"2024-09-13T18:21:55.356895+0000","last_peered":"2024-09-13T18:21:55.356895+0000","last_clean":"2024-09-13T18:21:55.356895+0000","last_became_active":"2024-09-13T17:51:16.281476+0000","last_became_peered":"2024-09-13T17:51:16.281476+0000","last_unstale":"2024-09-13T18:21:55.356895+0000","last_undegraded":"2024-09-13T18:21:55.356895+0000","last_fullsized":"2024-09-13T18:21:55.356895+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5052","last_scrub_stamp":"2024-09-13T18:21:55.356852+0000","last_deep_scrub":"27'5052","last_deep_scrub_stamp":"2024-09-13T18:21:55.356852+0000","last_clean_scrub_stamp":"2024-09-13T18:21:55.356852+0000","objects_scrubbed":0,"log_size":5052,"log_dups_size":0,"ondisk_log_size":5052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:26:46.603249+0000","scrub_duration":0.044667539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20895,"num_write_kb":464282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3,6],"acting":[4,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'18","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.301541+0000","last_change":"2024-09-13T17:50:34.585741+0000","last_active":"2024-09-13T18:21:13.301541+0000","last_peered":"2024-09-13T18:21:13.301541+0000","last_clean":"2024-09-13T18:21:13.301541+0000","last_became_active":"2024-09-13T17:50:34.585464+0000","last_became_peered":"2024-09-13T17:50:34.585464+0000","last_unstale":"2024-09-13T18:21:13.301541+0000","last_undegraded":"2024-09-13T18:21:13.301541+0000","last_fullsized":"2024-09-13T18:21:13.301541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'27","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:36.784439+0000","last_change":"2024-09-13T18:21:36.784589+0000","last_active":"2024-09-13T18:21:36.784439+0000","last_peered":"2024-09-13T18:21:36.784439+0000","last_clean":"2024-09-13T18:21:36.784439+0000","last_became_active":"2024-09-13T17:50:34.200936+0000","last_became_peered":"2024-09-13T17:50:34.200936+0000","last_unstale":"2024-09-13T18:21:36.784439+0000","last_undegraded":"2024-09-13T18:21:36.784439+0000","last_fullsized":"2024-09-13T18:21:36.784439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'27","last_scrub_stamp":"2024-09-13T18:21:36.784405+0000","last_deep_scrub":"27'27","last_deep_scrub_stamp":"2024-09-13T18:21:36.784405+0000","last_clean_scrub_stamp":"2024-09-13T18:21:36.784405+0000","objects_scrubbed":8,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:11.675692+0000","scrub_duration":0.012535223,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'6502","reported_seq":6529,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.953302+0000","last_change":"2024-09-13T17:50:38.615229+0000","last_active":"2024-09-13T18:21:02.953302+0000","last_peered":"2024-09-13T18:21:02.953302+0000","last_clean":"2024-09-13T18:21:02.953302+0000","last_became_active":"2024-09-13T17:50:38.614739+0000","last_became_peered":"2024-09-13T17:50:38.614739+0000","last_unstale":"2024-09-13T18:21:02.953302+0000","last_undegraded":"2024-09-13T18:21:02.953302+0000","last_fullsized":"2024-09-13T18:21:02.953302+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6502,"log_dups_size":0,"ondisk_log_size":6502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27696,"num_write_kb":544723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'20","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:34.335658+0000","last_change":"2024-09-13T18:21:34.335970+0000","last_active":"2024-09-13T18:21:34.335658+0000","last_peered":"2024-09-13T18:21:34.335658+0000","last_clean":"2024-09-13T18:21:34.335658+0000","last_became_active":"2024-09-13T17:50:34.588565+0000","last_became_peered":"2024-09-13T17:50:34.588565+0000","last_unstale":"2024-09-13T18:21:34.335658+0000","last_undegraded":"2024-09-13T18:21:34.335658+0000","last_fullsized":"2024-09-13T18:21:34.335658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-13T18:21:34.335555+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-13T18:21:34.335555+0000","last_clean_scrub_stamp":"2024-09-13T18:21:34.335555+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:28.955486+0000","scrub_duration":0.0039531770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'9","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:26.414088+0000","last_change":"2024-09-13T18:21:26.414323+0000","last_active":"2024-09-13T18:21:26.414088+0000","last_peered":"2024-09-13T18:21:26.414088+0000","last_clean":"2024-09-13T18:21:26.414088+0000","last_became_active":"2024-09-13T17:50:34.197839+0000","last_became_peered":"2024-09-13T17:50:34.197839+0000","last_unstale":"2024-09-13T18:21:26.414088+0000","last_undegraded":"2024-09-13T18:21:26.414088+0000","last_fullsized":"2024-09-13T18:21:26.414088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-09-13T18:21:26.414037+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-09-13T18:21:26.414037+0000","last_clean_scrub_stamp":"2024-09-13T18:21:26.414037+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:22.483872+0000","scrub_duration":0.0068422659999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'6326","reported_seq":6353,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.544581+0000","last_change":"2024-09-13T17:50:38.611080+0000","last_active":"2024-09-13T18:21:02.544581+0000","last_peered":"2024-09-13T18:21:02.544581+0000","last_clean":"2024-09-13T18:21:02.544581+0000","last_became_active":"2024-09-13T17:50:38.610688+0000","last_became_peered":"2024-09-13T17:50:38.610688+0000","last_unstale":"2024-09-13T18:21:02.544581+0000","last_undegraded":"2024-09-13T18:21:02.544581+0000","last_fullsized":"2024-09-13T18:21:02.544581+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6326,"log_dups_size":0,"ondisk_log_size":6326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27199,"num_write_kb":538670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'29","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.317672+0000","last_change":"2024-09-13T17:50:34.594763+0000","last_active":"2024-09-13T18:21:13.317672+0000","last_peered":"2024-09-13T18:21:13.317672+0000","last_clean":"2024-09-13T18:21:13.317672+0000","last_became_active":"2024-09-13T17:50:34.594471+0000","last_became_peered":"2024-09-13T17:50:34.594471+0000","last_unstale":"2024-09-13T18:21:13.317672+0000","last_undegraded":"2024-09-13T18:21:13.317672+0000","last_fullsized":"2024-09-13T18:21:13.317672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'7","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:37.911671+0000","last_change":"2024-09-13T18:21:37.911888+0000","last_active":"2024-09-13T18:21:37.911671+0000","last_peered":"2024-09-13T18:21:37.911671+0000","last_clean":"2024-09-13T18:21:37.911671+0000","last_became_active":"2024-09-13T17:50:34.208817+0000","last_became_peered":"2024-09-13T17:50:34.208817+0000","last_unstale":"2024-09-13T18:21:37.911671+0000","last_undegraded":"2024-09-13T18:21:37.911671+0000","last_fullsized":"2024-09-13T18:21:37.911671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-13T18:21:37.911585+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-13T18:21:37.911585+0000","last_clean_scrub_stamp":"2024-09-13T18:21:37.911585+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:43:22.483981+0000","scrub_duration":0.0050921259999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'5788","reported_seq":5827,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:57.298804+0000","last_change":"2024-09-13T18:21:57.298969+0000","last_active":"2024-09-13T18:21:57.298804+0000","last_peered":"2024-09-13T18:21:57.298804+0000","last_clean":"2024-09-13T18:21:57.298804+0000","last_became_active":"2024-09-13T17:50:38.609128+0000","last_became_peered":"2024-09-13T17:50:38.609128+0000","last_unstale":"2024-09-13T18:21:57.298804+0000","last_undegraded":"2024-09-13T18:21:57.298804+0000","last_fullsized":"2024-09-13T18:21:57.298804+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5788","last_scrub_stamp":"2024-09-13T18:21:57.298769+0000","last_deep_scrub":"27'5788","last_deep_scrub_stamp":"2024-09-13T18:21:57.298769+0000","last_clean_scrub_stamp":"2024-09-13T18:21:57.298769+0000","objects_scrubbed":0,"log_size":5788,"log_dups_size":0,"ondisk_log_size":5788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:11:22.245296+0000","scrub_duration":0.044309794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24140,"num_write_kb":538739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'24","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:49.332934+0000","last_change":"2024-09-13T18:21:49.333129+0000","last_active":"2024-09-13T18:21:49.332934+0000","last_peered":"2024-09-13T18:21:49.332934+0000","last_clean":"2024-09-13T18:21:49.332934+0000","last_became_active":"2024-09-13T17:50:34.592669+0000","last_became_peered":"2024-09-13T17:50:34.592669+0000","last_unstale":"2024-09-13T18:21:49.332934+0000","last_undegraded":"2024-09-13T18:21:49.332934+0000","last_fullsized":"2024-09-13T18:21:49.332934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-09-13T18:21:49.332882+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-09-13T18:21:49.332882+0000","last_clean_scrub_stamp":"2024-09-13T18:21:49.332882+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:50.817551+0000","scrub_duration":0.0025807830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'19","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.241499+0000","last_change":"2024-09-13T18:21:28.241791+0000","last_active":"2024-09-13T18:21:28.241499+0000","last_peered":"2024-09-13T18:21:28.241499+0000","last_clean":"2024-09-13T18:21:28.241499+0000","last_became_active":"2024-09-13T17:50:34.200529+0000","last_became_peered":"2024-09-13T17:50:34.200529+0000","last_unstale":"2024-09-13T18:21:28.241499+0000","last_undegraded":"2024-09-13T18:21:28.241499+0000","last_fullsized":"2024-09-13T18:21:28.241499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-13T18:21:28.241430+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-13T18:21:28.241430+0000","last_clean_scrub_stamp":"2024-09-13T18:21:28.241430+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:08:39.927576+0000","scrub_duration":0.00889181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'5464","reported_seq":5491,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.614309+0000","last_change":"2024-09-13T17:50:38.606287+0000","last_active":"2024-09-13T18:21:02.614309+0000","last_peered":"2024-09-13T18:21:02.614309+0000","last_clean":"2024-09-13T18:21:02.614309+0000","last_became_active":"2024-09-13T17:50:38.605826+0000","last_became_peered":"2024-09-13T17:50:38.605826+0000","last_unstale":"2024-09-13T18:21:02.614309+0000","last_undegraded":"2024-09-13T18:21:02.614309+0000","last_fullsized":"2024-09-13T18:21:02.614309+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5464,"log_dups_size":0,"ondisk_log_size":5464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23044,"num_write_kb":472739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'18","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:25.955022+0000","last_change":"2024-09-13T17:50:34.591579+0000","last_active":"2024-09-13T18:21:25.955022+0000","last_peered":"2024-09-13T18:21:25.955022+0000","last_clean":"2024-09-13T18:21:25.955022+0000","last_became_active":"2024-09-13T17:50:34.586165+0000","last_became_peered":"2024-09-13T17:50:34.586165+0000","last_unstale":"2024-09-13T18:21:25.955022+0000","last_undegraded":"2024-09-13T18:21:25.955022+0000","last_fullsized":"2024-09-13T18:21:25.955022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'421","reported_seq":463,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:31.308661+0000","last_change":"2024-09-13T18:21:31.308816+0000","last_active":"2024-09-13T18:21:31.308661+0000","last_peered":"2024-09-13T18:21:31.308661+0000","last_clean":"2024-09-13T18:21:31.308661+0000","last_became_active":"2024-09-13T17:50:34.202948+0000","last_became_peered":"2024-09-13T17:50:34.202948+0000","last_unstale":"2024-09-13T18:21:31.308661+0000","last_undegraded":"2024-09-13T18:21:31.308661+0000","last_fullsized":"2024-09-13T18:21:31.308661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'421","last_scrub_stamp":"2024-09-13T18:21:31.308629+0000","last_deep_scrub":"27'421","last_deep_scrub_stamp":"2024-09-13T18:21:31.308629+0000","last_clean_scrub_stamp":"2024-09-13T18:21:31.308629+0000","objects_scrubbed":5,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:55.983699+0000","scrub_duration":0.024451271,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":4324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'5887","reported_seq":5926,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:57.523458+0000","last_change":"2024-09-13T18:21:57.523677+0000","last_active":"2024-09-13T18:21:57.523458+0000","last_peered":"2024-09-13T18:21:57.523458+0000","last_clean":"2024-09-13T18:21:57.523458+0000","last_became_active":"2024-09-13T17:50:38.610055+0000","last_became_peered":"2024-09-13T17:50:38.610055+0000","last_unstale":"2024-09-13T18:21:57.523458+0000","last_undegraded":"2024-09-13T18:21:57.523458+0000","last_fullsized":"2024-09-13T18:21:57.523458+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5887","last_scrub_stamp":"2024-09-13T18:21:57.523426+0000","last_deep_scrub":"27'5887","last_deep_scrub_stamp":"2024-09-13T18:21:57.523426+0000","last_clean_scrub_stamp":"2024-09-13T18:21:57.523426+0000","objects_scrubbed":0,"log_size":5887,"log_dups_size":0,"ondisk_log_size":5887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:10:56.767652+0000","scrub_duration":0.047711444999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24664,"num_write_kb":495799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'21","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:39.354348+0000","last_change":"2024-09-13T18:21:39.354570+0000","last_active":"2024-09-13T18:21:39.354348+0000","last_peered":"2024-09-13T18:21:39.354348+0000","last_clean":"2024-09-13T18:21:39.354348+0000","last_became_active":"2024-09-13T17:50:34.593454+0000","last_became_peered":"2024-09-13T17:50:34.593454+0000","last_unstale":"2024-09-13T18:21:39.354348+0000","last_undegraded":"2024-09-13T18:21:39.354348+0000","last_fullsized":"2024-09-13T18:21:39.354348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-09-13T18:21:39.354296+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-09-13T18:21:39.354296+0000","last_clean_scrub_stamp":"2024-09-13T18:21:39.354296+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:10:26.576365+0000","scrub_duration":0.003405356,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'70","reported_seq":112,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:45.487099+0000","last_change":"2024-09-13T18:21:45.487099+0000","last_active":"2024-09-13T18:21:45.487099+0000","last_peered":"2024-09-13T18:21:45.487099+0000","last_clean":"2024-09-13T18:21:45.487099+0000","last_became_active":"2024-09-13T17:50:34.207605+0000","last_became_peered":"2024-09-13T17:50:34.207605+0000","last_unstale":"2024-09-13T18:21:45.487099+0000","last_undegraded":"2024-09-13T18:21:45.487099+0000","last_fullsized":"2024-09-13T18:21:45.487099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'70","last_scrub_stamp":"2024-09-13T18:21:45.486787+0000","last_deep_scrub":"27'70","last_deep_scrub_stamp":"2024-09-13T18:21:45.486787+0000","last_clean_scrub_stamp":"2024-09-13T18:21:45.486787+0000","objects_scrubbed":3,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:25:53.510155+0000","scrub_duration":0.0066096169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'5937","reported_seq":5976,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:52.308856+0000","last_change":"2024-09-13T18:21:52.308991+0000","last_active":"2024-09-13T18:21:52.308856+0000","last_peered":"2024-09-13T18:21:52.308856+0000","last_clean":"2024-09-13T18:21:52.308856+0000","last_became_active":"2024-09-13T17:50:38.614165+0000","last_became_peered":"2024-09-13T17:50:38.614165+0000","last_unstale":"2024-09-13T18:21:52.308856+0000","last_undegraded":"2024-09-13T18:21:52.308856+0000","last_fullsized":"2024-09-13T18:21:52.308856+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5937","last_scrub_stamp":"2024-09-13T18:21:52.308833+0000","last_deep_scrub":"27'5937","last_deep_scrub_stamp":"2024-09-13T18:21:52.308833+0000","last_clean_scrub_stamp":"2024-09-13T18:21:52.308833+0000","objects_scrubbed":0,"log_size":5937,"log_dups_size":0,"ondisk_log_size":5937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:21:00.114058+0000","scrub_duration":0.040795066999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25111,"num_write_kb":509795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'25","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:51.296070+0000","last_change":"2024-09-13T18:21:51.296070+0000","last_active":"2024-09-13T18:21:51.296070+0000","last_peered":"2024-09-13T18:21:51.296070+0000","last_clean":"2024-09-13T18:21:51.296070+0000","last_became_active":"2024-09-13T17:50:34.582529+0000","last_became_peered":"2024-09-13T17:50:34.582529+0000","last_unstale":"2024-09-13T18:21:51.296070+0000","last_undegraded":"2024-09-13T18:21:51.296070+0000","last_fullsized":"2024-09-13T18:21:51.296070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'25","last_scrub_stamp":"2024-09-13T18:21:51.295749+0000","last_deep_scrub":"27'25","last_deep_scrub_stamp":"2024-09-13T18:21:51.295749+0000","last_clean_scrub_stamp":"2024-09-13T18:21:51.295749+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:20:57.681873+0000","scrub_duration":0.0032653209999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'6","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:31.812971+0000","last_change":"2024-09-13T18:21:31.812971+0000","last_active":"2024-09-13T18:21:31.812971+0000","last_peered":"2024-09-13T18:21:31.812971+0000","last_clean":"2024-09-13T18:21:31.812971+0000","last_became_active":"2024-09-13T17:50:34.203360+0000","last_became_peered":"2024-09-13T17:50:34.203360+0000","last_unstale":"2024-09-13T18:21:31.812971+0000","last_undegraded":"2024-09-13T18:21:31.812971+0000","last_fullsized":"2024-09-13T18:21:31.812971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-09-13T18:21:31.812723+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-09-13T18:21:31.812723+0000","last_clean_scrub_stamp":"2024-09-13T18:21:31.812723+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:43:43.954185+0000","scrub_duration":0.0059683940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'5934","reported_seq":5973,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.246781+0000","last_change":"2024-09-13T18:21:55.246917+0000","last_active":"2024-09-13T18:21:55.246781+0000","last_peered":"2024-09-13T18:21:55.246781+0000","last_clean":"2024-09-13T18:21:55.246781+0000","last_became_active":"2024-09-13T17:50:38.610692+0000","last_became_peered":"2024-09-13T17:50:38.610692+0000","last_unstale":"2024-09-13T18:21:55.246781+0000","last_undegraded":"2024-09-13T18:21:55.246781+0000","last_fullsized":"2024-09-13T18:21:55.246781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5934","last_scrub_stamp":"2024-09-13T18:21:55.246755+0000","last_deep_scrub":"27'5934","last_deep_scrub_stamp":"2024-09-13T18:21:55.246755+0000","last_clean_scrub_stamp":"2024-09-13T18:21:55.246755+0000","objects_scrubbed":0,"log_size":5934,"log_dups_size":0,"ondisk_log_size":5934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:01:01.221236+0000","scrub_duration":0.038730878000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25100,"num_write_kb":497535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'23","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.311843+0000","last_change":"2024-09-13T17:51:16.365295+0000","last_active":"2024-09-13T18:21:13.311843+0000","last_peered":"2024-09-13T18:21:13.311843+0000","last_clean":"2024-09-13T18:21:13.311843+0000","last_became_active":"2024-09-13T17:51:16.364437+0000","last_became_peered":"2024-09-13T17:51:16.364437+0000","last_unstale":"2024-09-13T18:21:13.311843+0000","last_undegraded":"2024-09-13T18:21:13.311843+0000","last_fullsized":"2024-09-13T18:21:13.311843+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"27'11","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:33.307232+0000","last_change":"2024-09-13T18:21:33.307232+0000","last_active":"2024-09-13T18:21:33.307232+0000","last_peered":"2024-09-13T18:21:33.307232+0000","last_clean":"2024-09-13T18:21:33.307232+0000","last_became_active":"2024-09-13T17:50:34.204003+0000","last_became_peered":"2024-09-13T17:50:34.204003+0000","last_unstale":"2024-09-13T18:21:33.307232+0000","last_undegraded":"2024-09-13T18:21:33.307232+0000","last_fullsized":"2024-09-13T18:21:33.307232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-09-13T18:21:33.306942+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-09-13T18:21:33.306942+0000","last_clean_scrub_stamp":"2024-09-13T18:21:33.306942+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:13.945111+0000","scrub_duration":0.0066162260000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'5363","reported_seq":5391,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.822018+0000","last_change":"2024-09-13T17:50:38.606144+0000","last_active":"2024-09-13T18:21:02.822018+0000","last_peered":"2024-09-13T18:21:02.822018+0000","last_clean":"2024-09-13T18:21:02.822018+0000","last_became_active":"2024-09-13T17:50:38.605646+0000","last_became_peered":"2024-09-13T17:50:38.605646+0000","last_unstale":"2024-09-13T18:21:02.822018+0000","last_undegraded":"2024-09-13T18:21:02.822018+0000","last_fullsized":"2024-09-13T18:21:02.822018+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5363,"log_dups_size":0,"ondisk_log_size":5363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22831,"num_write_kb":508903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'6067","reported_seq":6097,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.920872+0000","last_change":"2024-09-13T17:50:38.618855+0000","last_active":"2024-09-13T18:21:02.920872+0000","last_peered":"2024-09-13T18:21:02.920872+0000","last_clean":"2024-09-13T18:21:02.920872+0000","last_became_active":"2024-09-13T17:50:38.618449+0000","last_became_peered":"2024-09-13T17:50:38.618449+0000","last_unstale":"2024-09-13T18:21:02.920872+0000","last_undegraded":"2024-09-13T18:21:02.920872+0000","last_fullsized":"2024-09-13T18:21:02.920872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6067,"log_dups_size":0,"ondisk_log_size":6067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25532,"num_write_kb":568825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'21","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:54.500409+0000","last_change":"2024-09-13T18:21:54.500706+0000","last_active":"2024-09-13T18:21:54.500409+0000","last_peered":"2024-09-13T18:21:54.500409+0000","last_clean":"2024-09-13T18:21:54.500409+0000","last_became_active":"2024-09-13T17:50:34.589396+0000","last_became_peered":"2024-09-13T17:50:34.589396+0000","last_unstale":"2024-09-13T18:21:54.500409+0000","last_undegraded":"2024-09-13T18:21:54.500409+0000","last_fullsized":"2024-09-13T18:21:54.500409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-09-13T18:21:54.500280+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-09-13T18:21:54.500280+0000","last_clean_scrub_stamp":"2024-09-13T18:21:54.500280+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:24.989093+0000","scrub_duration":0.0028702549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:32.850895+0000","last_change":"2024-09-13T18:21:32.850895+0000","last_active":"2024-09-13T18:21:32.850895+0000","last_peered":"2024-09-13T18:21:32.850895+0000","last_clean":"2024-09-13T18:21:32.850895+0000","last_became_active":"2024-09-13T17:50:34.206308+0000","last_became_peered":"2024-09-13T17:50:34.206308+0000","last_unstale":"2024-09-13T18:21:32.850895+0000","last_undegraded":"2024-09-13T18:21:32.850895+0000","last_fullsized":"2024-09-13T18:21:32.850895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-09-13T18:21:32.850533+0000","last_deep_scrub":"25'1","last_deep_scrub_stamp":"2024-09-13T18:21:32.850533+0000","last_clean_scrub_stamp":"2024-09-13T18:21:32.850533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:59:21.637907+0000","scrub_duration":0.0034298850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'5881","reported_seq":5922,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:53.439711+0000","last_change":"2024-09-13T18:21:53.439711+0000","last_active":"2024-09-13T18:21:53.439711+0000","last_peered":"2024-09-13T18:21:53.439711+0000","last_clean":"2024-09-13T18:21:53.439711+0000","last_became_active":"2024-09-13T17:50:38.617611+0000","last_became_peered":"2024-09-13T17:50:38.617611+0000","last_unstale":"2024-09-13T18:21:53.439711+0000","last_undegraded":"2024-09-13T18:21:53.439711+0000","last_fullsized":"2024-09-13T18:21:53.439711+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5881","last_scrub_stamp":"2024-09-13T18:21:53.439266+0000","last_deep_scrub":"27'5881","last_deep_scrub_stamp":"2024-09-13T18:21:53.439266+0000","last_clean_scrub_stamp":"2024-09-13T18:21:53.439266+0000","objects_scrubbed":0,"log_size":5881,"log_dups_size":0,"ondisk_log_size":5881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:46:36.260829+0000","scrub_duration":0.043273643000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25019,"num_write_kb":530139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'27","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:34.190222+0000","last_change":"2024-09-13T18:21:34.190222+0000","last_active":"2024-09-13T18:21:34.190222+0000","last_peered":"2024-09-13T18:21:34.190222+0000","last_clean":"2024-09-13T18:21:34.190222+0000","last_became_active":"2024-09-13T17:50:34.581708+0000","last_became_peered":"2024-09-13T17:50:34.581708+0000","last_unstale":"2024-09-13T18:21:34.190222+0000","last_undegraded":"2024-09-13T18:21:34.190222+0000","last_fullsized":"2024-09-13T18:21:34.190222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'27","last_scrub_stamp":"2024-09-13T18:21:34.189993+0000","last_deep_scrub":"27'27","last_deep_scrub_stamp":"2024-09-13T18:21:34.189993+0000","last_clean_scrub_stamp":"2024-09-13T18:21:34.189993+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:32:35.944129+0000","scrub_duration":0.0029610209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'5","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:42.211549+0000","last_change":"2024-09-13T18:21:42.211743+0000","last_active":"2024-09-13T18:21:42.211549+0000","last_peered":"2024-09-13T18:21:42.211549+0000","last_clean":"2024-09-13T18:21:42.211549+0000","last_became_active":"2024-09-13T17:50:34.205202+0000","last_became_peered":"2024-09-13T17:50:34.205202+0000","last_unstale":"2024-09-13T18:21:42.211549+0000","last_undegraded":"2024-09-13T18:21:42.211549+0000","last_fullsized":"2024-09-13T18:21:42.211549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-09-13T18:21:42.211500+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-09-13T18:21:42.211500+0000","last_clean_scrub_stamp":"2024-09-13T18:21:42.211500+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:47:56.465375+0000","scrub_duration":0.0062275539999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'7","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:30.324177+0000","last_change":"2024-09-13T18:21:30.324321+0000","last_active":"2024-09-13T18:21:30.324177+0000","last_peered":"2024-09-13T18:21:30.324177+0000","last_clean":"2024-09-13T18:21:30.324177+0000","last_became_active":"2024-09-13T17:50:34.202163+0000","last_became_peered":"2024-09-13T17:50:34.202163+0000","last_unstale":"2024-09-13T18:21:30.324177+0000","last_undegraded":"2024-09-13T18:21:30.324177+0000","last_fullsized":"2024-09-13T18:21:30.324177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-13T18:21:30.324141+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-13T18:21:30.324141+0000","last_clean_scrub_stamp":"2024-09-13T18:21:30.324141+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:27.051230+0000","scrub_duration":0.00672063,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'18","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:45.443346+0000","last_change":"2024-09-13T18:21:45.443452+0000","last_active":"2024-09-13T18:21:45.443346+0000","last_peered":"2024-09-13T18:21:45.443346+0000","last_clean":"2024-09-13T18:21:45.443346+0000","last_became_active":"2024-09-13T17:50:34.590497+0000","last_became_peered":"2024-09-13T17:50:34.590497+0000","last_unstale":"2024-09-13T18:21:45.443346+0000","last_undegraded":"2024-09-13T18:21:45.443346+0000","last_fullsized":"2024-09-13T18:21:45.443346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:21:45.443314+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:21:45.443314+0000","last_clean_scrub_stamp":"2024-09-13T18:21:45.443314+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:29.220122+0000","scrub_duration":0.0029326640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'4651","reported_seq":4689,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:52.595124+0000","last_change":"2024-09-13T18:21:52.595243+0000","last_active":"2024-09-13T18:21:52.595124+0000","last_peered":"2024-09-13T18:21:52.595124+0000","last_clean":"2024-09-13T18:21:52.595124+0000","last_became_active":"2024-09-13T17:50:38.618108+0000","last_became_peered":"2024-09-13T17:50:38.618108+0000","last_unstale":"2024-09-13T18:21:52.595124+0000","last_undegraded":"2024-09-13T18:21:52.595124+0000","last_fullsized":"2024-09-13T18:21:52.595124+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4651","last_scrub_stamp":"2024-09-13T18:21:52.595101+0000","last_deep_scrub":"27'4651","last_deep_scrub_stamp":"2024-09-13T18:21:52.595101+0000","last_clean_scrub_stamp":"2024-09-13T18:21:52.595101+0000","objects_scrubbed":0,"log_size":4651,"log_dups_size":0,"ondisk_log_size":4651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:15:47.184237+0000","scrub_duration":0.047927588,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19488,"num_write_kb":413684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'692","reported_seq":732,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:38.185928+0000","last_change":"2024-09-13T18:21:38.186111+0000","last_active":"2024-09-13T18:21:38.185928+0000","last_peered":"2024-09-13T18:21:38.185928+0000","last_clean":"2024-09-13T18:21:38.185928+0000","last_became_active":"2024-09-13T17:50:34.218087+0000","last_became_peered":"2024-09-13T17:50:34.218087+0000","last_unstale":"2024-09-13T18:21:38.185928+0000","last_undegraded":"2024-09-13T18:21:38.185928+0000","last_fullsized":"2024-09-13T18:21:38.185928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'692","last_scrub_stamp":"2024-09-13T18:21:38.185889+0000","last_deep_scrub":"27'692","last_deep_scrub_stamp":"2024-09-13T18:21:38.185889+0000","last_clean_scrub_stamp":"2024-09-13T18:21:38.185889+0000","objects_scrubbed":3,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:16:47.594855+0000","scrub_duration":0.021444375000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195461,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'20","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:40.309829+0000","last_change":"2024-09-13T18:21:40.310070+0000","last_active":"2024-09-13T18:21:40.309829+0000","last_peered":"2024-09-13T18:21:40.309829+0000","last_clean":"2024-09-13T18:21:40.309829+0000","last_became_active":"2024-09-13T17:50:34.583283+0000","last_became_peered":"2024-09-13T17:50:34.583283+0000","last_unstale":"2024-09-13T18:21:40.309829+0000","last_undegraded":"2024-09-13T18:21:40.309829+0000","last_fullsized":"2024-09-13T18:21:40.309829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-13T18:21:40.309774+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-13T18:21:40.309774+0000","last_clean_scrub_stamp":"2024-09-13T18:21:40.309774+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:45:15.638497+0000","scrub_duration":0.0033293300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'5883","reported_seq":5910,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.843115+0000","last_change":"2024-09-13T17:50:38.611987+0000","last_active":"2024-09-13T18:21:02.843115+0000","last_peered":"2024-09-13T18:21:02.843115+0000","last_clean":"2024-09-13T18:21:02.843115+0000","last_became_active":"2024-09-13T17:50:38.611636+0000","last_became_peered":"2024-09-13T17:50:38.611636+0000","last_unstale":"2024-09-13T18:21:02.843115+0000","last_undegraded":"2024-09-13T18:21:02.843115+0000","last_fullsized":"2024-09-13T18:21:02.843115+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":5883,"log_dups_size":0,"ondisk_log_size":5883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24816,"num_write_kb":545450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'25","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:46.374486+0000","last_change":"2024-09-13T18:21:46.374486+0000","last_active":"2024-09-13T18:21:46.374486+0000","last_peered":"2024-09-13T18:21:46.374486+0000","last_clean":"2024-09-13T18:21:46.374486+0000","last_became_active":"2024-09-13T17:50:34.588651+0000","last_became_peered":"2024-09-13T17:50:34.588651+0000","last_unstale":"2024-09-13T18:21:46.374486+0000","last_undegraded":"2024-09-13T18:21:46.374486+0000","last_fullsized":"2024-09-13T18:21:46.374486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'25","last_scrub_stamp":"2024-09-13T18:21:46.374221+0000","last_deep_scrub":"27'25","last_deep_scrub_stamp":"2024-09-13T18:21:46.374221+0000","last_clean_scrub_stamp":"2024-09-13T18:21:46.374221+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:58.490650+0000","scrub_duration":0.003300449,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'24","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:45.720155+0000","last_change":"2024-09-13T18:21:45.720155+0000","last_active":"2024-09-13T18:21:45.720155+0000","last_peered":"2024-09-13T18:21:45.720155+0000","last_clean":"2024-09-13T18:21:45.720155+0000","last_became_active":"2024-09-13T17:50:34.583493+0000","last_became_peered":"2024-09-13T17:50:34.583493+0000","last_unstale":"2024-09-13T18:21:45.720155+0000","last_undegraded":"2024-09-13T18:21:45.720155+0000","last_fullsized":"2024-09-13T18:21:45.720155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-09-13T18:21:45.719919+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-09-13T18:21:45.719919+0000","last_clean_scrub_stamp":"2024-09-13T18:21:45.719919+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:15.218921+0000","scrub_duration":0.0030666199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'25","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:50.830134+0000","last_change":"2024-09-13T18:21:50.830134+0000","last_active":"2024-09-13T18:21:50.830134+0000","last_peered":"2024-09-13T18:21:50.830134+0000","last_clean":"2024-09-13T18:21:50.830134+0000","last_became_active":"2024-09-13T17:50:34.587119+0000","last_became_peered":"2024-09-13T17:50:34.587119+0000","last_unstale":"2024-09-13T18:21:50.830134+0000","last_undegraded":"2024-09-13T18:21:50.830134+0000","last_fullsized":"2024-09-13T18:21:50.830134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'25","last_scrub_stamp":"2024-09-13T18:21:50.829746+0000","last_deep_scrub":"27'25","last_deep_scrub_stamp":"2024-09-13T18:21:50.829746+0000","last_clean_scrub_stamp":"2024-09-13T18:21:50.829746+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:38:15.448330+0000","scrub_duration":0.0031738930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'17","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:57.236877+0000","last_change":"2024-09-13T18:21:57.236877+0000","last_active":"2024-09-13T18:21:57.236877+0000","last_peered":"2024-09-13T18:21:57.236877+0000","last_clean":"2024-09-13T18:21:57.236877+0000","last_became_active":"2024-09-13T17:50:34.581628+0000","last_became_peered":"2024-09-13T17:50:34.581628+0000","last_unstale":"2024-09-13T18:21:57.236877+0000","last_undegraded":"2024-09-13T18:21:57.236877+0000","last_fullsized":"2024-09-13T18:21:57.236877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'17","last_scrub_stamp":"2024-09-13T18:21:57.236610+0000","last_deep_scrub":"27'17","last_deep_scrub_stamp":"2024-09-13T18:21:57.236610+0000","last_clean_scrub_stamp":"2024-09-13T18:21:57.236610+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:24.087638+0000","scrub_duration":0.0024405049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'14","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:53.191673+0000","last_change":"2024-09-13T18:21:53.191956+0000","last_active":"2024-09-13T18:21:53.191673+0000","last_peered":"2024-09-13T18:21:53.191673+0000","last_clean":"2024-09-13T18:21:53.191673+0000","last_became_active":"2024-09-13T17:50:34.594857+0000","last_became_peered":"2024-09-13T17:50:34.594857+0000","last_unstale":"2024-09-13T18:21:53.191673+0000","last_undegraded":"2024-09-13T18:21:53.191673+0000","last_fullsized":"2024-09-13T18:21:53.191673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-09-13T18:21:53.191615+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-09-13T18:21:53.191615+0000","last_clean_scrub_stamp":"2024-09-13T18:21:53.191615+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:45.001540+0000","scrub_duration":0.0042100100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'15","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:53.457198+0000","last_change":"2024-09-13T18:21:53.457331+0000","last_active":"2024-09-13T18:21:53.457198+0000","last_peered":"2024-09-13T18:21:53.457198+0000","last_clean":"2024-09-13T18:21:53.457198+0000","last_became_active":"2024-09-13T17:50:34.589354+0000","last_became_peered":"2024-09-13T17:50:34.589354+0000","last_unstale":"2024-09-13T18:21:53.457198+0000","last_undegraded":"2024-09-13T18:21:53.457198+0000","last_fullsized":"2024-09-13T18:21:53.457198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-09-13T18:21:53.457163+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-09-13T18:21:53.457163+0000","last_clean_scrub_stamp":"2024-09-13T18:21:53.457163+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:15:40.638651+0000","scrub_duration":0.0022509019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'24","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:38.403817+0000","last_change":"2024-09-13T18:21:38.404032+0000","last_active":"2024-09-13T18:21:38.403817+0000","last_peered":"2024-09-13T18:21:38.403817+0000","last_clean":"2024-09-13T18:21:38.403817+0000","last_became_active":"2024-09-13T17:50:34.597621+0000","last_became_peered":"2024-09-13T17:50:34.597621+0000","last_unstale":"2024-09-13T18:21:38.403817+0000","last_undegraded":"2024-09-13T18:21:38.403817+0000","last_fullsized":"2024-09-13T18:21:38.403817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-09-13T18:21:38.403767+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-09-13T18:21:38.403767+0000","last_clean_scrub_stamp":"2024-09-13T18:21:38.403767+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:33:01.593056+0000","scrub_duration":0.0041273439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'15","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:32.155724+0000","last_change":"2024-09-13T18:21:32.155926+0000","last_active":"2024-09-13T18:21:32.155724+0000","last_peered":"2024-09-13T18:21:32.155724+0000","last_clean":"2024-09-13T18:21:32.155724+0000","last_became_active":"2024-09-13T17:50:34.589626+0000","last_became_peered":"2024-09-13T17:50:34.589626+0000","last_unstale":"2024-09-13T18:21:32.155724+0000","last_undegraded":"2024-09-13T18:21:32.155724+0000","last_fullsized":"2024-09-13T18:21:32.155724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-09-13T18:21:32.155676+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-09-13T18:21:32.155676+0000","last_clean_scrub_stamp":"2024-09-13T18:21:32.155676+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:03:35.333502+0000","scrub_duration":0.0028998560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'16","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:44.880892+0000","last_change":"2024-09-13T18:21:44.881098+0000","last_active":"2024-09-13T18:21:44.880892+0000","last_peered":"2024-09-13T18:21:44.880892+0000","last_clean":"2024-09-13T18:21:44.880892+0000","last_became_active":"2024-09-13T17:50:34.589757+0000","last_became_peered":"2024-09-13T17:50:34.589757+0000","last_unstale":"2024-09-13T18:21:44.880892+0000","last_undegraded":"2024-09-13T18:21:44.880892+0000","last_fullsized":"2024-09-13T18:21:44.880892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-09-13T18:21:44.880841+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-09-13T18:21:44.880841+0000","last_clean_scrub_stamp":"2024-09-13T18:21:44.880841+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:12:15.859928+0000","scrub_duration":0.0034118379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'23","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.130283+0000","last_change":"2024-09-13T18:21:27.130283+0000","last_active":"2024-09-13T18:21:27.130283+0000","last_peered":"2024-09-13T18:21:27.130283+0000","last_clean":"2024-09-13T18:21:27.130283+0000","last_became_active":"2024-09-13T17:50:34.595242+0000","last_became_peered":"2024-09-13T17:50:34.595242+0000","last_unstale":"2024-09-13T18:21:27.130283+0000","last_undegraded":"2024-09-13T18:21:27.130283+0000","last_fullsized":"2024-09-13T18:21:27.130283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-13T18:21:27.130025+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-13T18:21:27.130025+0000","last_clean_scrub_stamp":"2024-09-13T18:21:27.130025+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:06.896321+0000","scrub_duration":0.003821638,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'21","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:54.798183+0000","last_change":"2024-09-13T18:21:54.798385+0000","last_active":"2024-09-13T18:21:54.798183+0000","last_peered":"2024-09-13T18:21:54.798183+0000","last_clean":"2024-09-13T18:21:54.798183+0000","last_became_active":"2024-09-13T17:50:34.592893+0000","last_became_peered":"2024-09-13T17:50:34.592893+0000","last_unstale":"2024-09-13T18:21:54.798183+0000","last_undegraded":"2024-09-13T18:21:54.798183+0000","last_fullsized":"2024-09-13T18:21:54.798183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-09-13T18:21:54.798131+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-09-13T18:21:54.798131+0000","last_clean_scrub_stamp":"2024-09-13T18:21:54.798131+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:27:46.038569+0000","scrub_duration":0.0031872860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'29","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.311937+0000","last_change":"2024-09-13T17:50:34.594679+0000","last_active":"2024-09-13T18:21:13.311937+0000","last_peered":"2024-09-13T18:21:13.311937+0000","last_clean":"2024-09-13T18:21:13.311937+0000","last_became_active":"2024-09-13T17:50:34.593750+0000","last_became_peered":"2024-09-13T17:50:34.593750+0000","last_unstale":"2024-09-13T18:21:13.311937+0000","last_undegraded":"2024-09-13T18:21:13.311937+0000","last_fullsized":"2024-09-13T18:21:13.311937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'19","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:37.375112+0000","last_change":"2024-09-13T18:21:37.375333+0000","last_active":"2024-09-13T18:21:37.375112+0000","last_peered":"2024-09-13T18:21:37.375112+0000","last_clean":"2024-09-13T18:21:37.375112+0000","last_became_active":"2024-09-13T17:50:34.598433+0000","last_became_peered":"2024-09-13T17:50:34.598433+0000","last_unstale":"2024-09-13T18:21:37.375112+0000","last_undegraded":"2024-09-13T18:21:37.375112+0000","last_fullsized":"2024-09-13T18:21:37.375112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-13T18:21:37.375056+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-13T18:21:37.375056+0000","last_clean_scrub_stamp":"2024-09-13T18:21:37.375056+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:02.999621+0000","scrub_duration":0.003734036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'22","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:36.325407+0000","last_change":"2024-09-13T18:21:36.325615+0000","last_active":"2024-09-13T18:21:36.325407+0000","last_peered":"2024-09-13T18:21:36.325407+0000","last_clean":"2024-09-13T18:21:36.325407+0000","last_became_active":"2024-09-13T17:50:34.591286+0000","last_became_peered":"2024-09-13T17:50:34.591286+0000","last_unstale":"2024-09-13T18:21:36.325407+0000","last_undegraded":"2024-09-13T18:21:36.325407+0000","last_fullsized":"2024-09-13T18:21:36.325407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-09-13T18:21:36.325355+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-09-13T18:21:36.325355+0000","last_clean_scrub_stamp":"2024-09-13T18:21:36.325355+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:12:08.676171+0000","scrub_duration":0.0029670930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'28","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:44.767452+0000","last_change":"2024-09-13T18:21:44.767671+0000","last_active":"2024-09-13T18:21:44.767452+0000","last_peered":"2024-09-13T18:21:44.767452+0000","last_clean":"2024-09-13T18:21:44.767452+0000","last_became_active":"2024-09-13T17:50:34.587506+0000","last_became_peered":"2024-09-13T17:50:34.587506+0000","last_unstale":"2024-09-13T18:21:44.767452+0000","last_undegraded":"2024-09-13T18:21:44.767452+0000","last_fullsized":"2024-09-13T18:21:44.767452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'28","last_scrub_stamp":"2024-09-13T18:21:44.767402+0000","last_deep_scrub":"27'28","last_deep_scrub_stamp":"2024-09-13T18:21:44.767402+0000","last_clean_scrub_stamp":"2024-09-13T18:21:44.767402+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:28:00.460114+0000","scrub_duration":0.003174128,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'22","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:47.811825+0000","last_change":"2024-09-13T18:21:47.812043+0000","last_active":"2024-09-13T18:21:47.811825+0000","last_peered":"2024-09-13T18:21:47.811825+0000","last_clean":"2024-09-13T18:21:47.811825+0000","last_became_active":"2024-09-13T17:50:34.595798+0000","last_became_peered":"2024-09-13T17:50:34.595798+0000","last_unstale":"2024-09-13T18:21:47.811825+0000","last_undegraded":"2024-09-13T18:21:47.811825+0000","last_fullsized":"2024-09-13T18:21:47.811825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-09-13T18:21:47.811769+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-09-13T18:21:47.811769+0000","last_clean_scrub_stamp":"2024-09-13T18:21:47.811769+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:56.620401+0000","scrub_duration":0.0030899819999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'21","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:25.753731+0000","last_change":"2024-09-13T18:21:25.753960+0000","last_active":"2024-09-13T18:21:25.753731+0000","last_peered":"2024-09-13T18:21:25.753731+0000","last_clean":"2024-09-13T18:21:25.753731+0000","last_became_active":"2024-09-13T17:50:34.584516+0000","last_became_peered":"2024-09-13T17:50:34.584516+0000","last_unstale":"2024-09-13T18:21:25.753731+0000","last_undegraded":"2024-09-13T18:21:25.753731+0000","last_fullsized":"2024-09-13T18:21:25.753731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-09-13T18:21:25.753680+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-09-13T18:21:25.753680+0000","last_clean_scrub_stamp":"2024-09-13T18:21:25.753680+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:59.108188+0000","scrub_duration":0.0032900099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'23","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:51.445915+0000","last_change":"2024-09-13T18:21:51.446115+0000","last_active":"2024-09-13T18:21:51.445915+0000","last_peered":"2024-09-13T18:21:51.445915+0000","last_clean":"2024-09-13T18:21:51.445915+0000","last_became_active":"2024-09-13T17:50:34.598648+0000","last_became_peered":"2024-09-13T17:50:34.598648+0000","last_unstale":"2024-09-13T18:21:51.445915+0000","last_undegraded":"2024-09-13T18:21:51.445915+0000","last_fullsized":"2024-09-13T18:21:51.445915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-13T18:21:51.445846+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-13T18:21:51.445846+0000","last_clean_scrub_stamp":"2024-09-13T18:21:51.445846+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:09.890274+0000","scrub_duration":0.003395239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'24","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:23.200378+0000","last_change":"2024-09-13T18:21:23.200494+0000","last_active":"2024-09-13T18:21:23.200378+0000","last_peered":"2024-09-13T18:21:23.200378+0000","last_clean":"2024-09-13T18:21:23.200378+0000","last_became_active":"2024-09-13T17:50:34.581699+0000","last_became_peered":"2024-09-13T17:50:34.581699+0000","last_unstale":"2024-09-13T18:21:23.200378+0000","last_undegraded":"2024-09-13T18:21:23.200378+0000","last_fullsized":"2024-09-13T18:21:23.200378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-09-13T18:21:23.200343+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-09-13T18:21:23.200343+0000","last_clean_scrub_stamp":"2024-09-13T18:21:23.200343+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:37:50.717684+0000","scrub_duration":0.002562838,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'23","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:25.110698+0000","last_change":"2024-09-13T18:21:25.110698+0000","last_active":"2024-09-13T18:21:25.110698+0000","last_peered":"2024-09-13T18:21:25.110698+0000","last_clean":"2024-09-13T18:21:25.110698+0000","last_became_active":"2024-09-13T17:50:34.587217+0000","last_became_peered":"2024-09-13T17:50:34.587217+0000","last_unstale":"2024-09-13T18:21:25.110698+0000","last_undegraded":"2024-09-13T18:21:25.110698+0000","last_fullsized":"2024-09-13T18:21:25.110698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-13T18:21:25.110401+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-13T18:21:25.110401+0000","last_clean_scrub_stamp":"2024-09-13T18:21:25.110401+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:08.271399+0000","scrub_duration":0.0034179560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'18","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:47.310243+0000","last_change":"2024-09-13T18:21:47.310243+0000","last_active":"2024-09-13T18:21:47.310243+0000","last_peered":"2024-09-13T18:21:47.310243+0000","last_clean":"2024-09-13T18:21:47.310243+0000","last_became_active":"2024-09-13T17:50:34.588513+0000","last_became_peered":"2024-09-13T17:50:34.588513+0000","last_unstale":"2024-09-13T18:21:47.310243+0000","last_undegraded":"2024-09-13T18:21:47.310243+0000","last_fullsized":"2024-09-13T18:21:47.310243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:21:47.309947+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:21:47.309947+0000","last_clean_scrub_stamp":"2024-09-13T18:21:47.309947+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:52:22.085054+0000","scrub_duration":0.003219857,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'19","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:40.418512+0000","last_change":"2024-09-13T18:21:40.418512+0000","last_active":"2024-09-13T18:21:40.418512+0000","last_peered":"2024-09-13T18:21:40.418512+0000","last_clean":"2024-09-13T18:21:40.418512+0000","last_became_active":"2024-09-13T17:50:34.585438+0000","last_became_peered":"2024-09-13T17:50:34.585438+0000","last_unstale":"2024-09-13T18:21:40.418512+0000","last_undegraded":"2024-09-13T18:21:40.418512+0000","last_fullsized":"2024-09-13T18:21:40.418512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-13T18:21:40.418258+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-13T18:21:40.418258+0000","last_clean_scrub_stamp":"2024-09-13T18:21:40.418258+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:52.939370+0000","scrub_duration":0.0033203519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'30","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:48.381967+0000","last_change":"2024-09-13T18:21:48.381967+0000","last_active":"2024-09-13T18:21:48.381967+0000","last_peered":"2024-09-13T18:21:48.381967+0000","last_clean":"2024-09-13T18:21:48.381967+0000","last_became_active":"2024-09-13T17:50:34.587665+0000","last_became_peered":"2024-09-13T17:50:34.587665+0000","last_unstale":"2024-09-13T18:21:48.381967+0000","last_undegraded":"2024-09-13T18:21:48.381967+0000","last_fullsized":"2024-09-13T18:21:48.381967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'30","last_scrub_stamp":"2024-09-13T18:21:48.381520+0000","last_deep_scrub":"27'30","last_deep_scrub_stamp":"2024-09-13T18:21:48.381520+0000","last_clean_scrub_stamp":"2024-09-13T18:21:48.381520+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:15:59.624808+0000","scrub_duration":0.003678331,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'27","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.322483+0000","last_change":"2024-09-13T17:50:34.591776+0000","last_active":"2024-09-13T18:21:13.322483+0000","last_peered":"2024-09-13T18:21:13.322483+0000","last_clean":"2024-09-13T18:21:13.322483+0000","last_became_active":"2024-09-13T17:50:34.591496+0000","last_became_peered":"2024-09-13T17:50:34.591496+0000","last_unstale":"2024-09-13T18:21:13.322483+0000","last_undegraded":"2024-09-13T18:21:13.322483+0000","last_fullsized":"2024-09-13T18:21:13.322483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'15","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:59.687264+0000","last_change":"2024-09-13T18:21:59.687484+0000","last_active":"2024-09-13T18:21:59.687264+0000","last_peered":"2024-09-13T18:21:59.687264+0000","last_clean":"2024-09-13T18:21:59.687264+0000","last_became_active":"2024-09-13T17:51:16.282065+0000","last_became_peered":"2024-09-13T17:51:16.282065+0000","last_unstale":"2024-09-13T18:21:59.687264+0000","last_undegraded":"2024-09-13T18:21:59.687264+0000","last_fullsized":"2024-09-13T18:21:59.687264+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-09-13T18:21:59.687218+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-09-13T18:21:59.687218+0000","last_clean_scrub_stamp":"2024-09-13T18:21:59.687218+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:33:21.535758+0000","scrub_duration":0.003695977,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"27'14","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.311285+0000","last_change":"2024-09-13T17:50:34.585531+0000","last_active":"2024-09-13T18:21:13.311285+0000","last_peered":"2024-09-13T18:21:13.311285+0000","last_clean":"2024-09-13T18:21:13.311285+0000","last_became_active":"2024-09-13T17:50:34.585226+0000","last_became_peered":"2024-09-13T17:50:34.585226+0000","last_unstale":"2024-09-13T18:21:13.311285+0000","last_undegraded":"2024-09-13T18:21:13.311285+0000","last_fullsized":"2024-09-13T18:21:13.311285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'17","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:59.828261+0000","last_change":"2024-09-13T18:21:59.828532+0000","last_active":"2024-09-13T18:21:59.828261+0000","last_peered":"2024-09-13T18:21:59.828261+0000","last_clean":"2024-09-13T18:21:59.828261+0000","last_became_active":"2024-09-13T17:50:34.584564+0000","last_became_peered":"2024-09-13T17:50:34.584564+0000","last_unstale":"2024-09-13T18:21:59.828261+0000","last_undegraded":"2024-09-13T18:21:59.828261+0000","last_fullsized":"2024-09-13T18:21:59.828261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'17","last_scrub_stamp":"2024-09-13T18:21:59.828209+0000","last_deep_scrub":"27'17","last_deep_scrub_stamp":"2024-09-13T18:21:59.828209+0000","last_clean_scrub_stamp":"2024-09-13T18:21:59.828209+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:49:08.538007+0000","scrub_duration":0.0035508250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'18","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.313624+0000","last_change":"2024-09-13T17:50:34.585938+0000","last_active":"2024-09-13T18:21:13.313624+0000","last_peered":"2024-09-13T18:21:13.313624+0000","last_clean":"2024-09-13T18:21:13.313624+0000","last_became_active":"2024-09-13T17:50:34.585584+0000","last_became_peered":"2024-09-13T17:50:34.585584+0000","last_unstale":"2024-09-13T18:21:13.313624+0000","last_undegraded":"2024-09-13T18:21:13.313624+0000","last_fullsized":"2024-09-13T18:21:13.313624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'15","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.310679+0000","last_change":"2024-09-13T17:50:34.588762+0000","last_active":"2024-09-13T18:21:13.310679+0000","last_peered":"2024-09-13T18:21:13.310679+0000","last_clean":"2024-09-13T18:21:13.310679+0000","last_became_active":"2024-09-13T17:50:34.588489+0000","last_became_peered":"2024-09-13T17:50:34.588489+0000","last_unstale":"2024-09-13T18:21:13.310679+0000","last_undegraded":"2024-09-13T18:21:13.310679+0000","last_fullsized":"2024-09-13T18:21:13.310679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'18","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:22.639978+0000","last_change":"2024-09-13T17:50:34.583889+0000","last_active":"2024-09-13T18:21:22.639978+0000","last_peered":"2024-09-13T18:21:22.639978+0000","last_clean":"2024-09-13T18:21:22.639978+0000","last_became_active":"2024-09-13T17:50:34.583551+0000","last_became_peered":"2024-09-13T17:50:34.583551+0000","last_unstale":"2024-09-13T18:21:22.639978+0000","last_undegraded":"2024-09-13T18:21:22.639978+0000","last_fullsized":"2024-09-13T18:21:22.639978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'10","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:29.338126+0000","last_change":"2024-09-13T18:21:29.338126+0000","last_active":"2024-09-13T18:21:29.338126+0000","last_peered":"2024-09-13T18:21:29.338126+0000","last_clean":"2024-09-13T18:21:29.338126+0000","last_became_active":"2024-09-13T17:50:34.585340+0000","last_became_peered":"2024-09-13T17:50:34.585340+0000","last_unstale":"2024-09-13T18:21:29.338126+0000","last_undegraded":"2024-09-13T18:21:29.338126+0000","last_fullsized":"2024-09-13T18:21:29.338126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-13T18:21:29.337797+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-13T18:21:29.337797+0000","last_clean_scrub_stamp":"2024-09-13T18:21:29.337797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:52.759391+0000","scrub_duration":0.003218725,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'17","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:25.955172+0000","last_change":"2024-09-13T17:50:34.592365+0000","last_active":"2024-09-13T18:21:25.955172+0000","last_peered":"2024-09-13T18:21:25.955172+0000","last_clean":"2024-09-13T18:21:25.955172+0000","last_became_active":"2024-09-13T17:50:34.592046+0000","last_became_peered":"2024-09-13T17:50:34.592046+0000","last_unstale":"2024-09-13T18:21:25.955172+0000","last_undegraded":"2024-09-13T18:21:25.955172+0000","last_fullsized":"2024-09-13T18:21:25.955172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":751482,"num_write_kb":16106434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178546,"ondisk_log_size":178546,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1271,"ondisk_log_size":1271,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":15941984,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":121,"num_read_kb":179,"num_write":3371,"num_write_kb":16874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15000,"num_omap_keys":30,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32030720,"data_stored":31883968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3184,"ondisk_log_size":3184,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509821,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1772172,"kb_used_data":8580,"kb_used_omap":1,"kb_used_meta":1763582,"kb_avail":92599668,"statfs":{"total":96636764160,"available":94822060032,"internally_reserved":0,"allocated":8785920,"data_stored":8600393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":1805908424},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":2.6030000000000002,"5min":4.7919999999999998,"15min":2.8069999999999999},"min":{"1min":0.28799999999999998,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":24.829000000000001,"5min":41.494999999999997,"15min":50.511000000000003},"last":0.54000000000000004},{"interface":"front","average":{"1min":1.708,"5min":3.0110000000000001,"15min":1.677},"min":{"1min":0.27300000000000002,"5min":0.193,"15min":0.193},"max":{"1min":9.9600000000000009,"5min":18.760000000000002,"15min":18.760000000000002},"last":0.36299999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":2.8199999999999998,"15min":1.841},"min":{"1min":0.52700000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":10.379,"5min":19.626000000000001,"15min":19.626000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":1.2629999999999999,"5min":3.0880000000000001,"15min":2.081},"min":{"1min":0.312,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.891,"5min":22.126999999999999,"15min":22.126999999999999},"last":0.61399999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":1.3400000000000001,"15min":1.2869999999999999},"min":{"1min":0.246,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.048,"5min":13.039999999999999,"15min":22.010000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":1.4830000000000001,"15min":1.472},"min":{"1min":0.32300000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.202,"5min":13.061999999999999,"15min":24.170999999999999},"last":0.49399999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.3049999999999999,"15min":1.4570000000000001},"min":{"1min":0.377,"5min":0.307,"15min":0.307},"max":{"1min":1.6839999999999999,"5min":12.936,"15min":24.359000000000002},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.278,"5min":1.9670000000000001,"15min":1.4810000000000001},"min":{"1min":0.35099999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":3.7530000000000001,"5min":15.539999999999999,"15min":15.539999999999999},"last":0.68100000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.5700000000000001,"15min":1.6279999999999999},"min":{"1min":0.35299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":5.5229999999999997,"5min":12.984,"15min":49.838000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.5449999999999999,"15min":1.5980000000000001},"min":{"1min":0.42499999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.4730000000000001,"5min":16.334,"15min":41.591999999999999},"last":0.66400000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":1.7010000000000001,"5min":1.875,"15min":1.764},"min":{"1min":0.46400000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":10.337999999999999,"5min":12.907999999999999,"15min":24.457000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.7050000000000001,"5min":1.7450000000000001,"15min":1.6579999999999999},"min":{"1min":0.45000000000000001,"5min":0.42399999999999999,"15min":0.40999999999999998},"max":{"1min":10.279,"5min":13.015000000000001,"15min":24.387},"last":0.59799999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.651,"15min":1.397},"min":{"1min":0.65400000000000003,"5min":0.34200000000000003,"15min":0.32300000000000001},"max":{"1min":4.649,"5min":14.138999999999999,"15min":14.138999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.323,"5min":1.847,"15min":1.4410000000000001},"min":{"1min":0.47699999999999998,"5min":0.377,"15min":0.35299999999999998},"max":{"1min":4.2270000000000003,"5min":14.013,"15min":14.013},"last":0.73899999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509823,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1055244,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":1054846,"kb_avail":93316596,"statfs":{"total":96636764160,"available":95556194304,"internally_reserved":0,"allocated":397312,"data_stored":210684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1080162758},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":1.7190000000000001,"15min":1.091},"min":{"1min":0.35199999999999998,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.68400000000000005,"5min":17.971,"15min":17.971},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":1.873,"15min":1.179},"min":{"1min":0.40600000000000003,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.042,"5min":18.001999999999999,"15min":18.001999999999999},"last":1.0089999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":2.2759999999999998,"15min":1.6870000000000001},"min":{"1min":0.38,"5min":0.22600000000000001,"15min":0.217},"max":{"1min":1.609,"5min":24.599,"15min":56.939999999999998},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":2.5219999999999998,"15min":1.768},"min":{"1min":0.443,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.5369999999999999,"5min":24.75,"15min":57.006999999999998},"last":0.66500000000000004}]},{"osd":3,"last update":"Fri Sep 13 18:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":2.79,"15min":1.599},"min":{"1min":0.40899999999999997,"5min":0.218,"15min":0.218},"max":{"1min":1.714,"5min":16.707000000000001,"15min":16.707000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":2.4510000000000001,"15min":1.554},"min":{"1min":0.27600000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.135,"5min":16.640999999999998,"15min":16.640999999999998},"last":0.84899999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:21:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.4370000000000001,"15min":1.236},"min":{"1min":0.67800000000000005,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.6000000000000001,"5min":8.2829999999999995,"15min":14.177},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.258,"15min":1.1639999999999999},"min":{"1min":0.55600000000000005,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.617,"5min":8.1980000000000004,"15min":14.396000000000001},"last":0.70399999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:21:22 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":0.73599999999999999,"15min":1.0049999999999999},"min":{"1min":0.433,"5min":0.34200000000000003,"15min":0.34100000000000003},"max":{"1min":3.637,"5min":3.6840000000000002,"15min":14.153},"last":1.052},{"interface":"front","average":{"1min":0.86699999999999999,"5min":1.1299999999999999,"15min":1.131},"min":{"1min":0.52100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.4450000000000001,"5min":7.476,"15min":14.375999999999999},"last":0.77800000000000002}]},{"osd":6,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":0.97899999999999998,"15min":1.1000000000000001},"min":{"1min":0.505,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6080000000000001,"5min":6.8730000000000002,"15min":8.3010000000000002},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.038,"15min":1.2110000000000001},"min":{"1min":0.63700000000000001,"5min":0.40400000000000003,"15min":0.39200000000000002},"max":{"1min":1.6519999999999999,"5min":6.9500000000000002,"15min":14.074},"last":1.2929999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:21:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0700000000000001,"15min":1.079},"min":{"1min":0.47999999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.5569999999999999,"5min":8.0410000000000004,"15min":8.0410000000000004},"last":1.216},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.1599999999999999,"15min":1.1859999999999999},"min":{"1min":0.56899999999999995,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.3799999999999999,"5min":6.5300000000000002,"15min":14.307},"last":0.97299999999999998}]}]},{"osd":1,"up_from":15,"seq":64424509822,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1738228,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":1737918,"kb_avail":92633612,"statfs":{"total":96636764160,"available":94856818688,"internally_reserved":0,"allocated":307200,"data_stored":116868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1779628486},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":2.5430000000000001,"15min":1.907},"min":{"1min":0.42299999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":4.25,"5min":21.949999999999999,"15min":21.949999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":2.3759999999999999,"15min":1.593},"min":{"1min":0.27600000000000002,"5min":0.193,"15min":0.193},"max":{"1min":2.7349999999999999,"5min":23.338999999999999,"15min":23.338999999999999},"last":0.61399999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.8540000000000001,"15min":1.5149999999999999},"min":{"1min":0.38,"5min":0.22,"15min":0.22},"max":{"1min":5.5129999999999999,"5min":11.423999999999999,"15min":14.869999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.962,"15min":1.4319999999999999},"min":{"1min":0.47399999999999998,"5min":0.223,"15min":0.223},"max":{"1min":5.4809999999999999,"5min":12.439,"15min":15.058},"last":0.64600000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":4.3810000000000002,"15min":3.1800000000000002},"min":{"1min":0.44400000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":4.5839999999999996,"5min":27.838000000000001,"15min":41.369999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":1.0860000000000001,"5min":4.327,"15min":2.911},"min":{"1min":0.251,"5min":0.216,"15min":0.216},"max":{"1min":4.9299999999999997,"5min":30.077000000000002,"15min":40.325000000000003},"last":0.498}]},{"osd":4,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":2.4660000000000002,"15min":2.1480000000000001},"min":{"1min":0.51500000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":7.2089999999999996,"5min":29.702000000000002,"15min":103.44499999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.2010000000000001,"5min":2.4689999999999999,"15min":1.905},"min":{"1min":0.52700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.75,"5min":29.425999999999998,"15min":40.953000000000003},"last":0.81200000000000006}]},{"osd":5,"last update":"Fri Sep 13 18:21:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.964,"15min":1.744},"min":{"1min":0.60899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":4.548,"5min":15.169,"15min":18.954000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.6919999999999999,"15min":1.5269999999999999},"min":{"1min":0.36699999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.2279999999999998,"5min":13.630000000000001,"15min":15.413},"last":0.78500000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.994,"5min":2.044,"15min":1.633},"min":{"1min":0.748,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":6.4889999999999999,"5min":13.67,"15min":15.462999999999999},"last":0.997},{"interface":"front","average":{"1min":2.0569999999999999,"5min":2.1419999999999999,"15min":1.7270000000000001},"min":{"1min":0.85699999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":6.5220000000000002,"5min":13.696,"15min":15.49},"last":1.0469999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.667,"15min":1.415},"min":{"1min":0.64500000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":7.9669999999999996,"5min":12.476000000000001,"15min":15.093999999999999},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.7569999999999999,"5min":1.7330000000000001,"15min":1.5},"min":{"1min":0.80200000000000005,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":7.9969999999999999,"5min":12.539999999999999,"15min":15.119},"last":1.2410000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509824,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1247072,"kb_used_data":4056,"kb_used_omap":1,"kb_used_meta":1243006,"kb_avail":93124768,"statfs":{"total":96636764160,"available":95359762432,"internally_reserved":0,"allocated":4153344,"data_stored":3967313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1272838598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":2.8849999999999998,"15min":1.7210000000000001},"min":{"1min":0.19800000000000001,"5min":0.182,"15min":0.182},"max":{"1min":7.0830000000000002,"5min":23.004000000000001,"15min":24.202000000000002},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.0700000000000001,"5min":2.9630000000000001,"15min":1.8640000000000001},"min":{"1min":0.375,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":7.1139999999999999,"5min":23.039999999999999,"15min":24.260999999999999},"last":0.92800000000000005}]},{"osd":2,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":1.6359999999999999,"15min":1.1200000000000001},"min":{"1min":0.254,"5min":0.20399999999999999,"15min":0.186},"max":{"1min":6.4770000000000003,"5min":11.026999999999999,"15min":17.859000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.851,"15min":1.343},"min":{"1min":0.26600000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":7.4100000000000001,"5min":11.702999999999999,"15min":19.678999999999998},"last":0.97899999999999998}]},{"osd":3,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":3.306,"15min":1.95},"min":{"1min":0.28100000000000003,"5min":0.219,"15min":0.219},"max":{"1min":1.0509999999999999,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.073,"5min":2.7130000000000001,"15min":1.8240000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":7.2210000000000001,"5min":15.856999999999999,"15min":15.856999999999999},"last":0.747}]},{"osd":4,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.6499999999999999,"15min":1.2729999999999999},"min":{"1min":0.32700000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":7.1959999999999997,"5min":8.7010000000000005,"15min":8.7010000000000005},"last":1.111},{"interface":"front","average":{"1min":0.86899999999999999,"5min":2.2109999999999999,"15min":1.5409999999999999},"min":{"1min":0.34399999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.256,"5min":10.965999999999999,"15min":16.291},"last":1.2829999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.7629999999999999,"15min":1.327},"min":{"1min":0.33000000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":3.1339999999999999,"5min":15.18,"15min":15.18},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.8069999999999999,"15min":1.427},"min":{"1min":0.39100000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":7.1669999999999998,"5min":17.852,"15min":17.852},"last":1.1559999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":2.2869999999999999,"15min":1.577},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":7.2480000000000002,"5min":15.467000000000001,"15min":15.467000000000001},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.8109999999999999,"15min":1.4330000000000001},"min":{"1min":0.44900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":7.1429999999999998,"5min":17.908000000000001,"15min":17.908000000000001},"last":1.353}]},{"osd":7,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.897,"15min":1.3979999999999999},"min":{"1min":0.40000000000000002,"5min":0.33500000000000002,"15min":0.32700000000000001},"max":{"1min":5.5789999999999997,"5min":18.986999999999998,"15min":18.986999999999998},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.9770000000000001,"15min":1.4830000000000001},"min":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.373},"max":{"1min":5.6159999999999997,"5min":19.021000000000001,"15min":19.021000000000001},"last":1.3200000000000001}]}]},{"osd":7,"up_from":14,"seq":60129542526,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1211620,"kb_used_data":12252,"kb_used_omap":1,"kb_used_meta":1199358,"kb_avail":93160220,"statfs":{"total":96636764160,"available":95396065280,"internally_reserved":0,"allocated":12546048,"data_stored":12357112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":1228143051},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":1.9470000000000001,"15min":1.2410000000000001},"min":{"1min":0.26500000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":2.875,"5min":35.195,"15min":35.195},"last":0.748},{"interface":"front","average":{"1min":1.133,"5min":2.1320000000000001,"15min":1.411},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":5.0919999999999996,"5min":35.253999999999998,"15min":35.253999999999998},"last":1.0720000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:21:30 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":2.5270000000000001,"15min":1.4930000000000001},"min":{"1min":0.317,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":4.8070000000000004,"5min":19.585000000000001,"15min":19.585000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.98899999999999999,"5min":2.52,"15min":1.6359999999999999},"min":{"1min":0.41699999999999998,"5min":0.29599999999999999,"15min":0.28000000000000003},"max":{"1min":4.8220000000000001,"5min":19.617000000000001,"15min":19.617000000000001},"last":0.68400000000000005}]},{"osd":2,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.597,"15min":1.1699999999999999},"min":{"1min":0.51200000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.5449999999999999,"5min":15.789999999999999,"15min":15.789999999999999},"last":1.034},{"interface":"front","average":{"1min":1.274,"5min":1.5680000000000001,"15min":1.1799999999999999},"min":{"1min":0.38700000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.5259999999999998,"5min":12.965,"15min":12.965},"last":1.198}]},{"osd":3,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":2.8980000000000001,"15min":1.9119999999999999},"min":{"1min":0.41099999999999998,"5min":0.316,"15min":0.316},"max":{"1min":7.8449999999999998,"5min":40.948999999999998,"15min":40.948999999999998},"last":1.4219999999999999},{"interface":"front","average":{"1min":2.0779999999999998,"5min":3.0139999999999998,"15min":1.768},"min":{"1min":0.39300000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":7.8810000000000002,"5min":25.234000000000002,"15min":25.234000000000002},"last":1.4930000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":1.165,"15min":0.98999999999999999},"min":{"1min":0.33700000000000002,"5min":0.28799999999999998,"15min":0.252},"max":{"1min":3.0609999999999999,"5min":8.8670000000000009,"15min":8.8670000000000009},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.3300000000000001,"15min":1.1859999999999999},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":6.069,"5min":10.279999999999999,"15min":10.279999999999999},"last":0.98099999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":1.0620000000000001,"15min":0.98199999999999998},"min":{"1min":0.374,"5min":0.311,"15min":0.27000000000000002},"max":{"1min":1.5289999999999999,"5min":8.1699999999999999,"15min":8.1699999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.96299999999999997,"15min":1.02},"min":{"1min":0.39800000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.9490000000000001,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":1.1739999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.4790000000000001,"15min":1.236},"min":{"1min":0.41599999999999998,"5min":0.34599999999999997,"15min":0.309},"max":{"1min":3.597,"5min":8.2219999999999995,"15min":8.2219999999999995},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.419,"5min":1.548,"15min":1.2789999999999999},"min":{"1min":0.48499999999999999,"5min":0.434,"15min":0.34499999999999997},"max":{"1min":3.7210000000000001,"5min":8.2469999999999999,"15min":8.2469999999999999},"last":1.343}]}]},{"osd":6,"up_from":14,"seq":60129542526,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1419436,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":1415038,"kb_avail":92952404,"statfs":{"total":96636764160,"available":95183261696,"internally_reserved":0,"allocated":4493312,"data_stored":4311530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1448999367},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":2.0750000000000002,"5min":2.0699999999999998,"15min":1.296},"min":{"1min":0.28000000000000003,"5min":0.214,"15min":0.214},"max":{"1min":11.666,"5min":12.808999999999999,"15min":12.808999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.6639999999999999,"15min":1.179},"min":{"1min":0.28899999999999998,"5min":0.218,"15min":0.218},"max":{"1min":7.8700000000000001,"5min":12.308999999999999,"15min":12.308999999999999},"last":0.71999999999999997}]},{"osd":1,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":2.6840000000000002,"5min":2.323,"15min":1.4930000000000001},"min":{"1min":0.52300000000000002,"5min":0.313,"15min":0.28699999999999998},"max":{"1min":20.530999999999999,"5min":20.530999999999999,"15min":20.530999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":2.2959999999999998,"5min":2.2149999999999999,"15min":1.514},"min":{"1min":0.377,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":20.501999999999999,"5min":20.501999999999999,"15min":20.501999999999999},"last":0.5}]},{"osd":2,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.7350000000000001,"15min":1.2270000000000001},"min":{"1min":0.47699999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":3.6800000000000002,"5min":10.792999999999999,"15min":10.792999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.8140000000000001,"15min":1.381},"min":{"1min":0.51800000000000002,"5min":0.317,"15min":0.317},"max":{"1min":3.738,"5min":10.826000000000001,"15min":10.826000000000001},"last":1.155}]},{"osd":3,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":2.4940000000000002,"5min":3.8300000000000001,"15min":2.5750000000000002},"min":{"1min":0.36599999999999999,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":16.495999999999999,"5min":32.25,"15min":84.867999999999995},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.4199999999999999,"5min":2.5009999999999999,"15min":1.927},"min":{"1min":0.36699999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":8.6780000000000008,"5min":14.064,"15min":84.343999999999994},"last":0.89800000000000002}]},{"osd":4,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":2.0739999999999998,"5min":1.6990000000000001,"15min":1.2070000000000001},"min":{"1min":0.39200000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":9.5649999999999995,"5min":11.044,"15min":11.044},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.294,"15min":1.113},"min":{"1min":0.38400000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":9.4719999999999995,"5min":9.4719999999999995,"15min":9.4719999999999995},"last":0.78300000000000003}]},{"osd":5,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2689999999999999,"15min":1.1559999999999999},"min":{"1min":0.376,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.0939999999999999,"5min":8.7880000000000003,"15min":8.7880000000000003},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.149,"15min":1.1000000000000001},"min":{"1min":0.41499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.1709999999999998,"5min":7.569,"15min":7.569},"last":0.96399999999999997}]},{"osd":7,"last update":"Fri Sep 13 18:21:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.1739999999999999,"15min":1.181},"min":{"1min":0.434,"5min":0.32900000000000001,"15min":0.29999999999999999},"max":{"1min":2.8029999999999999,"5min":7.1820000000000004,"15min":7.1820000000000004},"last":1.202},{"interface":"front","average":{"1min":1.133,"5min":1.294,"15min":1.256},"min":{"1min":0.53700000000000003,"5min":0.34499999999999997,"15min":0.33500000000000002},"max":{"1min":2.871,"5min":7.2039999999999997,"15min":7.2039999999999997},"last":1.008}]}]},{"osd":5,"up_from":14,"seq":60129542528,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":978544,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":978238,"kb_avail":93393296,"statfs":{"total":96636764160,"available":95634735104,"internally_reserved":0,"allocated":303104,"data_stored":117302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1001716166},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.595,"15min":1.1819999999999999},"min":{"1min":0.316,"5min":0.218,"15min":0.218},"max":{"1min":4.0919999999999996,"5min":10.051,"15min":10.148999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.038,"5min":1.633,"15min":1.3340000000000001},"min":{"1min":0.40500000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.9550000000000001,"5min":10.098000000000001,"15min":10.185},"last":1.091}]},{"osd":1,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.647,"15min":1.1879999999999999},"min":{"1min":0.28899999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":4.7119999999999997,"5min":10.08,"15min":10.436999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.032,"5min":1.843,"15min":1.387},"min":{"1min":0.44500000000000001,"5min":0.253,"15min":0.253},"max":{"1min":2.4369999999999998,"5min":19.388000000000002,"15min":19.388000000000002},"last":0.84899999999999998}]},{"osd":2,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.5249999999999999,"15min":1.129},"min":{"1min":0.443,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":5.1660000000000004,"5min":12.667,"15min":12.667},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.6639999999999999,"15min":1.2},"min":{"1min":0.441,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.8900000000000001,"5min":34.905999999999999,"15min":34.905999999999999},"last":1.145}]},{"osd":3,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":3.6779999999999999,"15min":2.3220000000000001},"min":{"1min":0.374,"5min":0.314,"15min":0.314},"max":{"1min":9.077,"5min":25.149999999999999,"15min":28.295000000000002},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.242,"5min":2.9180000000000001,"15min":1.7410000000000001},"min":{"1min":0.436,"5min":0.38500000000000001,"15min":0.318},"max":{"1min":5.3689999999999998,"5min":19.731000000000002,"15min":19.731000000000002},"last":1.0289999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0960000000000001,"15min":0.97399999999999998},"min":{"1min":0.33000000000000002,"5min":0.28599999999999998,"15min":0.255},"max":{"1min":4.8140000000000001,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.599,"15min":1.2290000000000001},"min":{"1min":0.38700000000000001,"5min":0.32500000000000001,"15min":0.28999999999999998},"max":{"1min":4.7919999999999998,"5min":10.210000000000001,"15min":10.210000000000001},"last":0.89300000000000002}]},{"osd":6,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.4039999999999999,"15min":1.1339999999999999},"min":{"1min":0.36799999999999999,"5min":0.33900000000000002,"15min":0.25800000000000001},"max":{"1min":1.5649999999999999,"5min":6.6849999999999996,"15min":6.6849999999999996},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3779999999999999,"15min":1.2150000000000001},"min":{"1min":0.51800000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":5.0819999999999999,"5min":10.151,"15min":10.151},"last":0.98999999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.258,"15min":1.1459999999999999},"min":{"1min":0.46899999999999997,"5min":0.32900000000000001,"15min":0.32000000000000001},"max":{"1min":2.581,"5min":7.9329999999999998,"15min":7.9329999999999998},"last":1.194},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.196,"15min":1.151},"min":{"1min":0.53800000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.5,"5min":5.976,"15min":5.976},"last":1.2789999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542526,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1301036,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":1296638,"kb_avail":93070804,"statfs":{"total":96636764160,"available":95304503296,"internally_reserved":0,"allocated":4493312,"data_stored":4310894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1327757766},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.5980000000000001,"15min":1.3160000000000001},"min":{"1min":0.25800000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":5.7759999999999998,"5min":14.318,"15min":28.439},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.022,"5min":1.772,"15min":1.4510000000000001},"min":{"1min":0.30499999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":3.0760000000000001,"5min":16.943000000000001,"15min":28.422000000000001},"last":0.626}]},{"osd":1,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":1.738,"5min":4.3470000000000004,"15min":2.6389999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":21.359000000000002,"5min":30.277999999999999,"15min":38.895000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.75,"5min":3.6030000000000002,"15min":2.0270000000000001},"min":{"1min":0.28199999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":21.388999999999999,"5min":32.670000000000002,"15min":32.670000000000002},"last":1.02}]},{"osd":2,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.7090000000000001,"15min":1.1870000000000001},"min":{"1min":0.32200000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":5.7229999999999999,"5min":25.167999999999999,"15min":25.167999999999999},"last":1.04},{"interface":"front","average":{"1min":1.345,"5min":2.0099999999999998,"15min":1.3480000000000001},"min":{"1min":0.41899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":5.75,"5min":25.218,"15min":25.218},"last":0.78800000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":2.3799999999999999,"5min":2.8439999999999999,"15min":1.827},"min":{"1min":0.36299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":16.344999999999999,"5min":43.945999999999998,"15min":43.945999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":2.4100000000000001,"5min":3.3250000000000002,"15min":2.1429999999999998},"min":{"1min":0.48899999999999999,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":15.901,"5min":29.640000000000001,"15min":29.640000000000001},"last":0.96799999999999997}]},{"osd":5,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":1.218,"15min":1.0569999999999999},"min":{"1min":0.34799999999999998,"5min":0.255,"15min":0.255},"max":{"1min":1.266,"5min":12.282999999999999,"15min":12.282999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.425,"15min":1.1870000000000001},"min":{"1min":0.53100000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":6.7000000000000002,"5min":16.995999999999999,"15min":16.995999999999999},"last":0.92000000000000004}]},{"osd":6,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.9530000000000001,"15min":1.401},"min":{"1min":0.441,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":9.3650000000000002,"5min":23.010999999999999,"15min":23.010999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.302,"5min":1.3720000000000001,"15min":1.194},"min":{"1min":0.42199999999999999,"5min":0.33800000000000002,"15min":0.33600000000000002},"max":{"1min":5.6369999999999996,"5min":7.9489999999999998,"15min":7.9489999999999998},"last":1.117}]},{"osd":7,"last update":"Fri Sep 13 18:21:16 2024","interfaces":[{"interface":"back","average":{"1min":1.5569999999999999,"5min":1.639,"15min":1.2949999999999999},"min":{"1min":0.45000000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":11.298,"5min":20.417000000000002,"15min":20.417000000000002},"last":1.208},{"interface":"front","average":{"1min":1.02,"5min":1.456,"15min":1.272},"min":{"1min":0.36399999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.9410000000000001,"5min":12.308999999999999,"15min":12.308999999999999},"last":1.0720000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3276800,"data_stored":3261021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":94760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8503296,"data_stored":8484469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4194970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4195606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":11669504,"data_stored":11650820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:22:07.086 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.086 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.086 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.086 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.086 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.087 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.087 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.087 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.087 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.087 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.087 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.087 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.088 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.088 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.088 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.088 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.088 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.088 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.088 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.089 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.089 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.089 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.089 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.089 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.090 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.090 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.090 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.090 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:07.090 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T18:22:27.091 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:22:27.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.196+0000 7f263ebc0700 1 Processor -- start 2024-09-13T18:22:27.205 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.196+0000 7f263ebc0700 1 -- start start 2024-09-13T18:22:27.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.196+0000 7f263ebc0700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f26380c3eb0 0x7f26380cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:27.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.196+0000 7f263ebc0700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26380cc060 con 0x7f26380cc4f0 2024-09-13T18:22:27.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.196+0000 7f263ebc0700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2638153c20 con 0x7f2638153840 2024-09-13T18:22:27.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.196+0000 7f263ebc0700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26380b2840 con 0x7f26380c3eb0 2024-09-13T18:22:27.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.200+0000 7f263e3bf700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f26380c3eb0 0x7f26380cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:27.206 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.200+0000 7f263d3bd700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f26380cc4f0 0x7f26380c9e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60498/0 (socket says 172.21.15.40:60498) 2024-09-13T18:22:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.200+0000 7f263d3bd700 1 -- 172.21.15.40:0/230752598 learned_addr learned my addr 172.21.15.40:0/230752598 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:22:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.200+0000 7f263e3bf700 1 -- 172.21.15.40:0/230752598 >> v1:172.21.15.43:6789/0 conn(0x7f2638153840 legacy=0x7f263815d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.200+0000 7f263e3bf700 1 -- 172.21.15.40:0/230752598 >> v1:172.21.15.40:6789/0 conn(0x7f26380cc4f0 legacy=0x7f26380c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.200+0000 7f263e3bf700 1 -- 172.21.15.40:0/230752598 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26380cb1e0 con 0x7f26380c3eb0 2024-09-13T18:22:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.200+0000 7f263e3bf700 1 --2- 172.21.15.40:0/230752598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26380c3eb0 0x7f26380cf260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2630009d90 tx=0x7f263000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9c3b5737874ccb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.200+0000 7f263cbbc700 1 -- 172.21.15.40:0/230752598 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263000e070 con 0x7f26380c3eb0 2024-09-13T18:22:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.200+0000 7f263cbbc700 1 -- 172.21.15.40:0/230752598 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263000bbd0 con 0x7f26380c3eb0 2024-09-13T18:22:27.207 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.200+0000 7f263cbbc700 1 -- 172.21.15.40:0/230752598 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2630012430 con 0x7f26380c3eb0 2024-09-13T18:22:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.200+0000 7f263ebc0700 1 -- 172.21.15.40:0/230752598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26380c3eb0 msgr2=0x7f26380cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.200+0000 7f263ebc0700 1 --2- 172.21.15.40:0/230752598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26380c3eb0 0x7f26380cf260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2630009d90 tx=0x7f263000b330 comp rx=0 tx=0).stop 2024-09-13T18:22:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.200+0000 7f263ebc0700 1 -- 172.21.15.40:0/230752598 shutdown_connections 2024-09-13T18:22:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.200+0000 7f263ebc0700 1 --2- 172.21.15.40:0/230752598 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26380c3eb0 0x7f26380cf260 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.200+0000 7f263ebc0700 1 -- 172.21.15.40:0/230752598 >> 172.21.15.40:0/230752598 conn(0x7f263806c800 msgr2=0x7f263806cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.200+0000 7f263ebc0700 1 -- 172.21.15.40:0/230752598 shutdown_connections 2024-09-13T18:22:27.208 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.200+0000 7f263ebc0700 1 -- 172.21.15.40:0/230752598 wait complete. 2024-09-13T18:22:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.200+0000 7f263ebc0700 1 Processor -- start 2024-09-13T18:22:27.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.200+0000 7f263ebc0700 1 -- start start 2024-09-13T18:22:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263ebc0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26380c3eb0 0x7f26381efd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263ebc0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26380cc4f0 0x7f2638145d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263ebc0700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2638153840 0x7f26380ce120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:27.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263ebc0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26380cd800 con 0x7f26380c3eb0 2024-09-13T18:22:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263ebc0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26380ccd10 con 0x7f26380cc4f0 2024-09-13T18:22:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263ebc0700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26380cdc90 con 0x7f2638153840 2024-09-13T18:22:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263dbbe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26380c3eb0 0x7f26381efd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263dbbe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26380c3eb0 0x7f26381efd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55198/0 (socket says 172.21.15.40:55198) 2024-09-13T18:22:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263dbbe700 1 -- 172.21.15.40:0/2965899054 learned_addr learned my addr 172.21.15.40:0/2965899054 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:22:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263e3bf700 1 --2- 172.21.15.40:0/2965899054 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2638153840 0x7f26380ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263d3bd700 1 --2- 172.21.15.40:0/2965899054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26380cc4f0 0x7f2638145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263dbbe700 1 -- 172.21.15.40:0/2965899054 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2638153840 msgr2=0x7f26380ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263dbbe700 1 --2- 172.21.15.40:0/2965899054 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2638153840 0x7f26380ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263dbbe700 1 -- 172.21.15.40:0/2965899054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26380cc4f0 msgr2=0x7f2638145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263dbbe700 1 --2- 172.21.15.40:0/2965899054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26380cc4f0 0x7f2638145d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263dbbe700 1 -- 172.21.15.40:0/2965899054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2630009a70 con 0x7f26380c3eb0 2024-09-13T18:22:27.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263d3bd700 1 --2- 172.21.15.40:0/2965899054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26380cc4f0 0x7f2638145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:27.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263dbbe700 1 --2- 172.21.15.40:0/2965899054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26380c3eb0 0x7f26381efd60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f263400c500 tx=0x7f263400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:27.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f262e7fc700 1 -- 172.21.15.40:0/2965899054 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26340052d0 con 0x7f26380c3eb0 2024-09-13T18:22:27.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263ebc0700 1 -- 172.21.15.40:0/2965899054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26381466f0 con 0x7f26380c3eb0 2024-09-13T18:22:27.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f262e7fc700 1 -- 172.21.15.40:0/2965899054 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2634005430 con 0x7f26380c3eb0 2024-09-13T18:22:27.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f262e7fc700 1 -- 172.21.15.40:0/2965899054 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26340055a0 con 0x7f26380c3eb0 2024-09-13T18:22:27.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263ebc0700 1 -- 172.21.15.40:0/2965899054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26380cd240 con 0x7f26380c3eb0 2024-09-13T18:22:27.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f263ebc0700 1 -- 172.21.15.40:0/2965899054 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26380b2140 con 0x7f26380c3eb0 2024-09-13T18:22:27.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.204+0000 7f262e7fc700 1 -- 172.21.15.40:0/2965899054 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f26340038e0 con 0x7f26380c3eb0 2024-09-13T18:22:27.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.208+0000 7f262e7fc700 1 --2- 172.21.15.40:0/2965899054 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f261c066730 0x7f261c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:27.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.208+0000 7f263d3bd700 1 --2- 172.21.15.40:0/2965899054 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f261c066730 0x7f261c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:27.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.208+0000 7f262e7fc700 1 -- 172.21.15.40:0/2965899054 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2634083b40 con 0x7f26380c3eb0 2024-09-13T18:22:27.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.208+0000 7f263d3bd700 1 --2- 172.21.15.40:0/2965899054 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f261c066730 0x7f261c068bf0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f26381ecf10 tx=0x7f2628009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:27.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.212+0000 7f262e7fc700 1 -- 172.21.15.40:0/2965899054 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f26340509e0 con 0x7f26380c3eb0 2024-09-13T18:22:27.382 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.376+0000 7f263ebc0700 1 -- 172.21.15.40:0/2965899054 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f26380cba10 con 0x7f261c066730 2024-09-13T18:22:27.395 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.388+0000 7f262e7fc700 1 -- 172.21.15.40:0/2965899054 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+371476 (secure 0 0 0) 0x7f2628001c70 con 0x7f261c066730 2024-09-13T18:22:27.400 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T18:22:27.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.392+0000 7f263ebc0700 1 -- 172.21.15.40:0/2965899054 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f261c066730 msgr2=0x7f261c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:27.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.392+0000 7f263ebc0700 1 --2- 172.21.15.40:0/2965899054 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f261c066730 0x7f261c068bf0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f26381ecf10 tx=0x7f2628009040 comp rx=0 tx=0).stop 2024-09-13T18:22:27.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.392+0000 7f263ebc0700 1 -- 172.21.15.40:0/2965899054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26380c3eb0 msgr2=0x7f26381efd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:27.403 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.392+0000 7f263ebc0700 1 --2- 172.21.15.40:0/2965899054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26380c3eb0 0x7f26381efd60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f263400c500 tx=0x7f263400c8c0 comp rx=0 tx=0).stop 2024-09-13T18:22:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.392+0000 7f263ebc0700 1 -- 172.21.15.40:0/2965899054 shutdown_connections 2024-09-13T18:22:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.392+0000 7f263ebc0700 1 --2- 172.21.15.40:0/2965899054 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f261c066730 0x7f261c068bf0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.392+0000 7f263ebc0700 1 --2- 172.21.15.40:0/2965899054 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2638153840 0x7f26380ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.392+0000 7f263ebc0700 1 --2- 172.21.15.40:0/2965899054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26380cc4f0 0x7f2638145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.392+0000 7f263ebc0700 1 --2- 172.21.15.40:0/2965899054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26380c3eb0 0x7f26381efd60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.392+0000 7f263ebc0700 1 -- 172.21.15.40:0/2965899054 >> 172.21.15.40:0/2965899054 conn(0x7f263806c800 msgr2=0x7f263814f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.392+0000 7f263ebc0700 1 -- 172.21.15.40:0/2965899054 shutdown_connections 2024-09-13T18:22:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:27.392+0000 7f263ebc0700 1 -- 172.21.15.40:0/2965899054 wait complete. 2024-09-13T18:22:27.404 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-13T18:22:27.413 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":982,"stamp":"2024-09-13T18:22:25.795669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":16532352,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":237,"num_read_kb":279,"num_write":755012,"num_write_kb":16125724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15000,"num_omap_keys":30,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183120,"ondisk_log_size":183120,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10723352,"kb_used_data":34648,"kb_used_omap":12,"kb_used_meta":10688627,"kb_avail":744251368,"statfs":{"total":773094113280,"available":762113400832,"internally_reserved":0,"allocated":35479552,"data_stored":33992096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":10945154616},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004895"},"pg_stats":[{"pgid":"2.1f","version":"27'272","reported_seq":316,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:30.247079+0000","last_change":"2024-09-13T18:21:30.247079+0000","last_active":"2024-09-13T18:21:30.247079+0000","last_peered":"2024-09-13T18:21:30.247079+0000","last_clean":"2024-09-13T18:21:30.247079+0000","last_became_active":"2024-09-13T17:50:34.211867+0000","last_became_peered":"2024-09-13T17:50:34.211867+0000","last_unstale":"2024-09-13T18:21:30.247079+0000","last_undegraded":"2024-09-13T18:21:30.247079+0000","last_fullsized":"2024-09-13T18:21:30.247079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'272","last_scrub_stamp":"2024-09-13T18:21:30.246880+0000","last_deep_scrub":"27'272","last_deep_scrub_stamp":"2024-09-13T18:21:30.246880+0000","last_clean_scrub_stamp":"2024-09-13T18:21:30.246880+0000","objects_scrubbed":5,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:57:30.406058+0000","scrub_duration":0.02016366,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3260943,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":267,"num_write_kb":3318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'10","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:51.192502+0000","last_change":"2024-09-13T18:21:51.192810+0000","last_active":"2024-09-13T18:21:51.192502+0000","last_peered":"2024-09-13T18:21:51.192502+0000","last_clean":"2024-09-13T18:21:51.192502+0000","last_became_active":"2024-09-13T17:50:34.587108+0000","last_became_peered":"2024-09-13T17:50:34.587108+0000","last_unstale":"2024-09-13T18:21:51.192502+0000","last_undegraded":"2024-09-13T18:21:51.192502+0000","last_fullsized":"2024-09-13T18:21:51.192502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-13T18:21:51.192437+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-13T18:21:51.192437+0000","last_clean_scrub_stamp":"2024-09-13T18:21:51.192437+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:50.505781+0000","scrub_duration":0.0029908370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'5594","reported_seq":5633,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:16.243447+0000","last_change":"2024-09-13T18:22:16.243447+0000","last_active":"2024-09-13T18:22:16.243447+0000","last_peered":"2024-09-13T18:22:16.243447+0000","last_clean":"2024-09-13T18:22:16.243447+0000","last_became_active":"2024-09-13T17:50:38.614723+0000","last_became_peered":"2024-09-13T17:50:38.614723+0000","last_unstale":"2024-09-13T18:22:16.243447+0000","last_undegraded":"2024-09-13T18:22:16.243447+0000","last_fullsized":"2024-09-13T18:22:16.243447+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5594","last_scrub_stamp":"2024-09-13T18:22:16.243294+0000","last_deep_scrub":"27'5594","last_deep_scrub_stamp":"2024-09-13T18:22:16.243294+0000","last_clean_scrub_stamp":"2024-09-13T18:22:16.243294+0000","objects_scrubbed":0,"log_size":5594,"log_dups_size":0,"ondisk_log_size":5594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:17.525529+0000","scrub_duration":0.045218341000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23732,"num_write_kb":533374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'6","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:36.210186+0000","last_change":"2024-09-13T18:21:36.210368+0000","last_active":"2024-09-13T18:21:36.210186+0000","last_peered":"2024-09-13T18:21:36.210186+0000","last_clean":"2024-09-13T18:21:36.210186+0000","last_became_active":"2024-09-13T17:50:34.202930+0000","last_became_peered":"2024-09-13T17:50:34.202930+0000","last_unstale":"2024-09-13T18:21:36.210186+0000","last_undegraded":"2024-09-13T18:21:36.210186+0000","last_fullsized":"2024-09-13T18:21:36.210186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-09-13T18:21:36.210139+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-09-13T18:21:36.210139+0000","last_clean_scrub_stamp":"2024-09-13T18:21:36.210139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:56.166821+0000","scrub_duration":0.0028089339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'20","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:06.315683+0000","last_change":"2024-09-13T18:22:06.315936+0000","last_active":"2024-09-13T18:22:06.315683+0000","last_peered":"2024-09-13T18:22:06.315683+0000","last_clean":"2024-09-13T18:22:06.315683+0000","last_became_active":"2024-09-13T17:51:16.365492+0000","last_became_peered":"2024-09-13T17:51:16.365492+0000","last_unstale":"2024-09-13T18:22:06.315683+0000","last_undegraded":"2024-09-13T18:22:06.315683+0000","last_fullsized":"2024-09-13T18:22:06.315683+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-13T18:22:06.315628+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-13T18:22:06.315628+0000","last_clean_scrub_stamp":"2024-09-13T18:22:06.315628+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:31.992466+0000","scrub_duration":0.002894284,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'5615","reported_seq":5654,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:48.296837+0000","last_change":"2024-09-13T18:21:48.296837+0000","last_active":"2024-09-13T18:21:48.296837+0000","last_peered":"2024-09-13T18:21:48.296837+0000","last_clean":"2024-09-13T18:21:48.296837+0000","last_became_active":"2024-09-13T17:50:38.618423+0000","last_became_peered":"2024-09-13T17:50:38.618423+0000","last_unstale":"2024-09-13T18:21:48.296837+0000","last_undegraded":"2024-09-13T18:21:48.296837+0000","last_fullsized":"2024-09-13T18:21:48.296837+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5615","last_scrub_stamp":"2024-09-13T18:21:48.296488+0000","last_deep_scrub":"27'5615","last_deep_scrub_stamp":"2024-09-13T18:21:48.296488+0000","last_clean_scrub_stamp":"2024-09-13T18:21:48.296488+0000","objects_scrubbed":0,"log_size":5615,"log_dups_size":0,"ondisk_log_size":5615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:01.155282+0000","scrub_duration":0.034902473000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23762,"num_write_kb":516686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'13","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:00.532888+0000","last_change":"2024-09-13T18:22:00.533193+0000","last_active":"2024-09-13T18:22:00.532888+0000","last_peered":"2024-09-13T18:22:00.532888+0000","last_clean":"2024-09-13T18:22:00.532888+0000","last_became_active":"2024-09-13T17:50:34.214652+0000","last_became_peered":"2024-09-13T17:50:34.214652+0000","last_unstale":"2024-09-13T18:22:00.532888+0000","last_undegraded":"2024-09-13T18:22:00.532888+0000","last_fullsized":"2024-09-13T18:22:00.532888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'13","last_scrub_stamp":"2024-09-13T18:22:00.532833+0000","last_deep_scrub":"27'13","last_deep_scrub_stamp":"2024-09-13T18:22:00.532833+0000","last_clean_scrub_stamp":"2024-09-13T18:22:00.532833+0000","objects_scrubbed":2,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:05.246113+0000","scrub_duration":0.0070529670000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'18","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:01.707673+0000","last_change":"2024-09-13T18:22:01.707968+0000","last_active":"2024-09-13T18:22:01.707673+0000","last_peered":"2024-09-13T18:22:01.707673+0000","last_clean":"2024-09-13T18:22:01.707673+0000","last_became_active":"2024-09-13T17:51:16.281549+0000","last_became_peered":"2024-09-13T17:51:16.281549+0000","last_unstale":"2024-09-13T18:22:01.707673+0000","last_undegraded":"2024-09-13T18:22:01.707673+0000","last_fullsized":"2024-09-13T18:22:01.707673+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:22:01.707624+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:22:01.707624+0000","last_clean_scrub_stamp":"2024-09-13T18:22:01.707624+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:30.522086+0000","scrub_duration":0.0032846300000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"27'5142","reported_seq":5180,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:53.315998+0000","last_change":"2024-09-13T18:21:53.316137+0000","last_active":"2024-09-13T18:21:53.315998+0000","last_peered":"2024-09-13T18:21:53.315998+0000","last_clean":"2024-09-13T18:21:53.315998+0000","last_became_active":"2024-09-13T17:50:38.614329+0000","last_became_peered":"2024-09-13T17:50:38.614329+0000","last_unstale":"2024-09-13T18:21:53.315998+0000","last_undegraded":"2024-09-13T18:21:53.315998+0000","last_fullsized":"2024-09-13T18:21:53.315998+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5142","last_scrub_stamp":"2024-09-13T18:21:53.315976+0000","last_deep_scrub":"27'5142","last_deep_scrub_stamp":"2024-09-13T18:21:53.315976+0000","last_clean_scrub_stamp":"2024-09-13T18:21:53.315976+0000","objects_scrubbed":0,"log_size":5142,"log_dups_size":0,"ondisk_log_size":5142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:24.336476+0000","scrub_duration":0.034529740000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21334,"num_write_kb":478261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'9","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:52.469363+0000","last_change":"2024-09-13T18:21:52.469614+0000","last_active":"2024-09-13T18:21:52.469363+0000","last_peered":"2024-09-13T18:21:52.469363+0000","last_clean":"2024-09-13T18:21:52.469363+0000","last_became_active":"2024-09-13T17:50:34.215661+0000","last_became_peered":"2024-09-13T17:50:34.215661+0000","last_unstale":"2024-09-13T18:21:52.469363+0000","last_undegraded":"2024-09-13T18:21:52.469363+0000","last_fullsized":"2024-09-13T18:21:52.469363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-09-13T18:21:52.469312+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-09-13T18:21:52.469312+0000","last_clean_scrub_stamp":"2024-09-13T18:21:52.469312+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:38:08.307554+0000","scrub_duration":0.0060588509999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'30","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:57.810967+0000","last_change":"2024-09-13T18:21:57.811166+0000","last_active":"2024-09-13T18:21:57.810967+0000","last_peered":"2024-09-13T18:21:57.810967+0000","last_clean":"2024-09-13T18:21:57.810967+0000","last_became_active":"2024-09-13T17:50:34.599409+0000","last_became_peered":"2024-09-13T17:50:34.599409+0000","last_unstale":"2024-09-13T18:21:57.810967+0000","last_undegraded":"2024-09-13T18:21:57.810967+0000","last_fullsized":"2024-09-13T18:21:57.810967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'30","last_scrub_stamp":"2024-09-13T18:21:57.810914+0000","last_deep_scrub":"27'30","last_deep_scrub_stamp":"2024-09-13T18:21:57.810914+0000","last_clean_scrub_stamp":"2024-09-13T18:21:57.810914+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:17:02.009800+0000","scrub_duration":0.003036955,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'4871","reported_seq":4909,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:49.336009+0000","last_change":"2024-09-13T18:21:49.336395+0000","last_active":"2024-09-13T18:21:49.336009+0000","last_peered":"2024-09-13T18:21:49.336009+0000","last_clean":"2024-09-13T18:21:49.336009+0000","last_became_active":"2024-09-13T17:50:38.614311+0000","last_became_peered":"2024-09-13T17:50:38.614311+0000","last_unstale":"2024-09-13T18:21:49.336009+0000","last_undegraded":"2024-09-13T18:21:49.336009+0000","last_fullsized":"2024-09-13T18:21:49.336009+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4871","last_scrub_stamp":"2024-09-13T18:21:49.335944+0000","last_deep_scrub":"27'4871","last_deep_scrub_stamp":"2024-09-13T18:21:49.335944+0000","last_clean_scrub_stamp":"2024-09-13T18:21:49.335944+0000","objects_scrubbed":0,"log_size":4871,"log_dups_size":0,"ondisk_log_size":4871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:12.664992+0000","scrub_duration":0.039294008999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20106,"num_write_kb":432357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'14","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:43.433109+0000","last_change":"2024-09-13T18:21:43.433365+0000","last_active":"2024-09-13T18:21:43.433109+0000","last_peered":"2024-09-13T18:21:43.433109+0000","last_clean":"2024-09-13T18:21:43.433109+0000","last_became_active":"2024-09-13T17:50:34.599207+0000","last_became_peered":"2024-09-13T17:50:34.599207+0000","last_unstale":"2024-09-13T18:21:43.433109+0000","last_undegraded":"2024-09-13T18:21:43.433109+0000","last_fullsized":"2024-09-13T18:21:43.433109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-09-13T18:21:43.433055+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-09-13T18:21:43.433055+0000","last_clean_scrub_stamp":"2024-09-13T18:21:43.433055+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:28.085068+0000","scrub_duration":0.003161358,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'1095","reported_seq":1138,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:56.570301+0000","last_change":"2024-09-13T18:21:56.570301+0000","last_active":"2024-09-13T18:21:56.570301+0000","last_peered":"2024-09-13T18:21:56.570301+0000","last_clean":"2024-09-13T18:21:56.570301+0000","last_became_active":"2024-09-13T17:50:34.215409+0000","last_became_peered":"2024-09-13T17:50:34.215409+0000","last_unstale":"2024-09-13T18:21:56.570301+0000","last_undegraded":"2024-09-13T18:21:56.570301+0000","last_fullsized":"2024-09-13T18:21:56.570301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'1095","last_scrub_stamp":"2024-09-13T18:21:56.570057+0000","last_deep_scrub":"27'1095","last_deep_scrub_stamp":"2024-09-13T18:21:56.570057+0000","last_clean_scrub_stamp":"2024-09-13T18:21:56.570057+0000","objects_scrubbed":3,"log_size":1095,"log_dups_size":0,"ondisk_log_size":1095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:35:11.168274+0000","scrub_duration":0.023788684000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1088,"num_write_kb":4339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'5394","reported_seq":5434,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:57.350995+0000","last_change":"2024-09-13T18:21:57.350995+0000","last_active":"2024-09-13T18:21:57.350995+0000","last_peered":"2024-09-13T18:21:57.350995+0000","last_clean":"2024-09-13T18:21:57.350995+0000","last_became_active":"2024-09-13T17:50:38.611263+0000","last_became_peered":"2024-09-13T17:50:38.611263+0000","last_unstale":"2024-09-13T18:21:57.350995+0000","last_undegraded":"2024-09-13T18:21:57.350995+0000","last_fullsized":"2024-09-13T18:21:57.350995+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5394","last_scrub_stamp":"2024-09-13T18:21:57.350563+0000","last_deep_scrub":"27'5394","last_deep_scrub_stamp":"2024-09-13T18:21:57.350563+0000","last_clean_scrub_stamp":"2024-09-13T18:21:57.350563+0000","objects_scrubbed":0,"log_size":5394,"log_dups_size":0,"ondisk_log_size":5394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:17.207282+0000","scrub_duration":0.032566444,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22326,"num_write_kb":494524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'19","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:52.230080+0000","last_change":"2024-09-13T18:21:52.230080+0000","last_active":"2024-09-13T18:21:52.230080+0000","last_peered":"2024-09-13T18:21:52.230080+0000","last_clean":"2024-09-13T18:21:52.230080+0000","last_became_active":"2024-09-13T17:50:34.585831+0000","last_became_peered":"2024-09-13T17:50:34.585831+0000","last_unstale":"2024-09-13T18:21:52.230080+0000","last_undegraded":"2024-09-13T18:21:52.230080+0000","last_fullsized":"2024-09-13T18:21:52.230080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-13T18:21:52.229769+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-13T18:21:52.229769+0000","last_clean_scrub_stamp":"2024-09-13T18:21:52.229769+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:09.681178+0000","scrub_duration":0.0031441960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'10","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:34.350232+0000","last_change":"2024-09-13T18:21:34.350402+0000","last_active":"2024-09-13T18:21:34.350232+0000","last_peered":"2024-09-13T18:21:34.350232+0000","last_clean":"2024-09-13T18:21:34.350232+0000","last_became_active":"2024-09-13T17:50:34.211594+0000","last_became_peered":"2024-09-13T17:50:34.211594+0000","last_unstale":"2024-09-13T18:21:34.350232+0000","last_undegraded":"2024-09-13T18:21:34.350232+0000","last_fullsized":"2024-09-13T18:21:34.350232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-13T18:21:34.350185+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-13T18:21:34.350185+0000","last_clean_scrub_stamp":"2024-09-13T18:21:34.350185+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:03.648370+0000","scrub_duration":0.003876764,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'5504","reported_seq":5544,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:56.359423+0000","last_change":"2024-09-13T18:21:56.359423+0000","last_active":"2024-09-13T18:21:56.359423+0000","last_peered":"2024-09-13T18:21:56.359423+0000","last_clean":"2024-09-13T18:21:56.359423+0000","last_became_active":"2024-09-13T17:50:38.612577+0000","last_became_peered":"2024-09-13T17:50:38.612577+0000","last_unstale":"2024-09-13T18:21:56.359423+0000","last_undegraded":"2024-09-13T18:21:56.359423+0000","last_fullsized":"2024-09-13T18:21:56.359423+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5504","last_scrub_stamp":"2024-09-13T18:21:56.359197+0000","last_deep_scrub":"27'5504","last_deep_scrub_stamp":"2024-09-13T18:21:56.359197+0000","last_clean_scrub_stamp":"2024-09-13T18:21:56.359197+0000","objects_scrubbed":0,"log_size":5504,"log_dups_size":0,"ondisk_log_size":5504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:28.613391+0000","scrub_duration":0.033662368999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23286,"num_write_kb":502495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'19","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:48.195751+0000","last_change":"2024-09-13T18:21:48.195940+0000","last_active":"2024-09-13T18:21:48.195751+0000","last_peered":"2024-09-13T18:21:48.195751+0000","last_clean":"2024-09-13T18:21:48.195751+0000","last_became_active":"2024-09-13T17:50:34.587027+0000","last_became_peered":"2024-09-13T17:50:34.587027+0000","last_unstale":"2024-09-13T18:21:48.195751+0000","last_undegraded":"2024-09-13T18:21:48.195751+0000","last_fullsized":"2024-09-13T18:21:48.195751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-13T18:21:48.195703+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-13T18:21:48.195703+0000","last_clean_scrub_stamp":"2024-09-13T18:21:48.195703+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:55.755022+0000","scrub_duration":0.0031326050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'7","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:32.340544+0000","last_change":"2024-09-13T18:21:32.340739+0000","last_active":"2024-09-13T18:21:32.340544+0000","last_peered":"2024-09-13T18:21:32.340544+0000","last_clean":"2024-09-13T18:21:32.340544+0000","last_became_active":"2024-09-13T17:50:34.211568+0000","last_became_peered":"2024-09-13T17:50:34.211568+0000","last_unstale":"2024-09-13T18:21:32.340544+0000","last_undegraded":"2024-09-13T18:21:32.340544+0000","last_fullsized":"2024-09-13T18:21:32.340544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-13T18:21:32.340496+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-13T18:21:32.340496+0000","last_clean_scrub_stamp":"2024-09-13T18:21:32.340496+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:14:08.632192+0000","scrub_duration":0.0064919820000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'6938","reported_seq":6978,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:01.462926+0000","last_change":"2024-09-13T18:22:01.462926+0000","last_active":"2024-09-13T18:22:01.462926+0000","last_peered":"2024-09-13T18:22:01.462926+0000","last_clean":"2024-09-13T18:22:01.462926+0000","last_became_active":"2024-09-13T17:50:38.609832+0000","last_became_peered":"2024-09-13T17:50:38.609832+0000","last_unstale":"2024-09-13T18:22:01.462926+0000","last_undegraded":"2024-09-13T18:22:01.462926+0000","last_fullsized":"2024-09-13T18:22:01.462926+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6938","last_scrub_stamp":"2024-09-13T18:22:01.462726+0000","last_deep_scrub":"27'6938","last_deep_scrub_stamp":"2024-09-13T18:22:01.462726+0000","last_clean_scrub_stamp":"2024-09-13T18:22:01.462726+0000","objects_scrubbed":0,"log_size":6938,"log_dups_size":0,"ondisk_log_size":6938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:07.849758+0000","scrub_duration":0.053454132000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28644,"num_write_kb":622131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"27'12","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:33.269238+0000","last_change":"2024-09-13T18:21:33.269565+0000","last_active":"2024-09-13T18:21:33.269238+0000","last_peered":"2024-09-13T18:21:33.269238+0000","last_clean":"2024-09-13T18:21:33.269238+0000","last_became_active":"2024-09-13T17:50:34.588794+0000","last_became_peered":"2024-09-13T17:50:34.588794+0000","last_unstale":"2024-09-13T18:21:33.269238+0000","last_undegraded":"2024-09-13T18:21:33.269238+0000","last_fullsized":"2024-09-13T18:21:33.269238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'12","last_scrub_stamp":"2024-09-13T18:21:33.269111+0000","last_deep_scrub":"27'12","last_deep_scrub_stamp":"2024-09-13T18:21:33.269111+0000","last_clean_scrub_stamp":"2024-09-13T18:21:33.269111+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:41.084595+0000","scrub_duration":0.003356562,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'7","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:59.510367+0000","last_change":"2024-09-13T18:21:59.510653+0000","last_active":"2024-09-13T18:21:59.510367+0000","last_peered":"2024-09-13T18:21:59.510367+0000","last_clean":"2024-09-13T18:21:59.510367+0000","last_became_active":"2024-09-13T17:50:34.214292+0000","last_became_peered":"2024-09-13T17:50:34.214292+0000","last_unstale":"2024-09-13T18:21:59.510367+0000","last_undegraded":"2024-09-13T18:21:59.510367+0000","last_fullsized":"2024-09-13T18:21:59.510367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-13T18:21:59.510312+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-13T18:21:59.510312+0000","last_clean_scrub_stamp":"2024-09-13T18:21:59.510312+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:39.341902+0000","scrub_duration":0.0065891600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'4588","reported_seq":4628,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:59.338480+0000","last_change":"2024-09-13T18:21:59.338480+0000","last_active":"2024-09-13T18:21:59.338480+0000","last_peered":"2024-09-13T18:21:59.338480+0000","last_clean":"2024-09-13T18:21:59.338480+0000","last_became_active":"2024-09-13T17:50:38.613010+0000","last_became_peered":"2024-09-13T17:50:38.613010+0000","last_unstale":"2024-09-13T18:21:59.338480+0000","last_undegraded":"2024-09-13T18:21:59.338480+0000","last_fullsized":"2024-09-13T18:21:59.338480+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4588","last_scrub_stamp":"2024-09-13T18:21:59.338108+0000","last_deep_scrub":"27'4588","last_deep_scrub_stamp":"2024-09-13T18:21:59.338108+0000","last_clean_scrub_stamp":"2024-09-13T18:21:59.338108+0000","objects_scrubbed":0,"log_size":4588,"log_dups_size":0,"ondisk_log_size":4588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:34.989321+0000","scrub_duration":0.027635435999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19354,"num_write_kb":454021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'26","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:50.314073+0000","last_change":"2024-09-13T18:21:50.314318+0000","last_active":"2024-09-13T18:21:50.314073+0000","last_peered":"2024-09-13T18:21:50.314073+0000","last_clean":"2024-09-13T18:21:50.314073+0000","last_became_active":"2024-09-13T17:50:34.582050+0000","last_became_peered":"2024-09-13T17:50:34.582050+0000","last_unstale":"2024-09-13T18:21:50.314073+0000","last_undegraded":"2024-09-13T18:21:50.314073+0000","last_fullsized":"2024-09-13T18:21:50.314073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'26","last_scrub_stamp":"2024-09-13T18:21:50.314024+0000","last_deep_scrub":"27'26","last_deep_scrub_stamp":"2024-09-13T18:21:50.314024+0000","last_clean_scrub_stamp":"2024-09-13T18:21:50.314024+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:48:00.837821+0000","scrub_duration":0.0030625299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'11","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:32.305631+0000","last_change":"2024-09-13T18:21:32.306235+0000","last_active":"2024-09-13T18:21:32.305631+0000","last_peered":"2024-09-13T18:21:32.305631+0000","last_clean":"2024-09-13T18:21:32.305631+0000","last_became_active":"2024-09-13T17:50:34.211230+0000","last_became_peered":"2024-09-13T17:50:34.211230+0000","last_unstale":"2024-09-13T18:21:32.305631+0000","last_undegraded":"2024-09-13T18:21:32.305631+0000","last_fullsized":"2024-09-13T18:21:32.305631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-09-13T18:21:32.305579+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-09-13T18:21:32.305579+0000","last_clean_scrub_stamp":"2024-09-13T18:21:32.305579+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:08.984532+0000","scrub_duration":0.0066305000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'6024","reported_seq":6065,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:54.336876+0000","last_change":"2024-09-13T18:21:54.336876+0000","last_active":"2024-09-13T18:21:54.336876+0000","last_peered":"2024-09-13T18:21:54.336876+0000","last_clean":"2024-09-13T18:21:54.336876+0000","last_became_active":"2024-09-13T17:50:38.606697+0000","last_became_peered":"2024-09-13T17:50:38.606697+0000","last_unstale":"2024-09-13T18:21:54.336876+0000","last_undegraded":"2024-09-13T18:21:54.336876+0000","last_fullsized":"2024-09-13T18:21:54.336876+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6024","last_scrub_stamp":"2024-09-13T18:21:54.336672+0000","last_deep_scrub":"27'6024","last_deep_scrub_stamp":"2024-09-13T18:21:54.336672+0000","last_clean_scrub_stamp":"2024-09-13T18:21:54.336672+0000","objects_scrubbed":0,"log_size":6024,"log_dups_size":0,"ondisk_log_size":6024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:56.337299+0000","scrub_duration":0.040808092999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25552,"num_write_kb":507487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'18","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:45.222534+0000","last_change":"2024-09-13T18:21:45.222779+0000","last_active":"2024-09-13T18:21:45.222534+0000","last_peered":"2024-09-13T18:21:45.222534+0000","last_clean":"2024-09-13T18:21:45.222534+0000","last_became_active":"2024-09-13T17:50:34.594215+0000","last_became_peered":"2024-09-13T17:50:34.594215+0000","last_unstale":"2024-09-13T18:21:45.222534+0000","last_undegraded":"2024-09-13T18:21:45.222534+0000","last_fullsized":"2024-09-13T18:21:45.222534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:21:45.222475+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:21:45.222475+0000","last_clean_scrub_stamp":"2024-09-13T18:21:45.222475+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:21.803809+0000","scrub_duration":0.0034298839999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'286","reported_seq":329,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:36.413084+0000","last_change":"2024-09-13T18:21:36.413202+0000","last_active":"2024-09-13T18:21:36.413084+0000","last_peered":"2024-09-13T18:21:36.413084+0000","last_clean":"2024-09-13T18:21:36.413084+0000","last_became_active":"2024-09-13T17:50:34.203702+0000","last_became_peered":"2024-09-13T17:50:34.203702+0000","last_unstale":"2024-09-13T18:21:36.413084+0000","last_undegraded":"2024-09-13T18:21:36.413084+0000","last_fullsized":"2024-09-13T18:21:36.413084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'286","last_scrub_stamp":"2024-09-13T18:21:36.413050+0000","last_deep_scrub":"27'286","last_deep_scrub_stamp":"2024-09-13T18:21:36.413050+0000","last_clean_scrub_stamp":"2024-09-13T18:21:36.413050+0000","objects_scrubbed":2,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:44.535556+0000","scrub_duration":0.0068211640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":605,"num_write_kb":197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'6030","reported_seq":6081,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:13.299511+0000","last_change":"2024-09-13T18:22:13.299635+0000","last_active":"2024-09-13T18:22:13.299511+0000","last_peered":"2024-09-13T18:22:13.299511+0000","last_clean":"2024-09-13T18:22:13.299511+0000","last_became_active":"2024-09-13T17:51:16.364540+0000","last_became_peered":"2024-09-13T17:51:16.364540+0000","last_unstale":"2024-09-13T18:22:13.299511+0000","last_undegraded":"2024-09-13T18:22:13.299511+0000","last_fullsized":"2024-09-13T18:22:13.299511+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6030","last_scrub_stamp":"2024-09-13T18:22:13.299492+0000","last_deep_scrub":"27'6030","last_deep_scrub_stamp":"2024-09-13T18:22:13.299492+0000","last_clean_scrub_stamp":"2024-09-13T18:22:13.299492+0000","objects_scrubbed":0,"log_size":6030,"log_dups_size":0,"ondisk_log_size":6030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:59:05.583204+0000","scrub_duration":0.045983233999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25617,"num_write_kb":548663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'18","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:50.405887+0000","last_change":"2024-09-13T18:21:50.405887+0000","last_active":"2024-09-13T18:21:50.405887+0000","last_peered":"2024-09-13T18:21:50.405887+0000","last_clean":"2024-09-13T18:21:50.405887+0000","last_became_active":"2024-09-13T17:50:34.586552+0000","last_became_peered":"2024-09-13T17:50:34.586552+0000","last_unstale":"2024-09-13T18:21:50.405887+0000","last_undegraded":"2024-09-13T18:21:50.405887+0000","last_fullsized":"2024-09-13T18:21:50.405887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:21:50.405568+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:21:50.405568+0000","last_clean_scrub_stamp":"2024-09-13T18:21:50.405568+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:09.337911+0000","scrub_duration":0.0019550739999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'8","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:57.533976+0000","last_change":"2024-09-13T18:21:57.533976+0000","last_active":"2024-09-13T18:21:57.533976+0000","last_peered":"2024-09-13T18:21:57.533976+0000","last_clean":"2024-09-13T18:21:57.533976+0000","last_became_active":"2024-09-13T17:50:34.216758+0000","last_became_peered":"2024-09-13T17:50:34.216758+0000","last_unstale":"2024-09-13T18:21:57.533976+0000","last_undegraded":"2024-09-13T18:21:57.533976+0000","last_fullsized":"2024-09-13T18:21:57.533976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-09-13T18:21:57.533649+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-09-13T18:21:57.533649+0000","last_clean_scrub_stamp":"2024-09-13T18:21:57.533649+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:14.399317+0000","scrub_duration":0.0047334860000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'4887","reported_seq":4928,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:10.474473+0000","last_change":"2024-09-13T18:22:10.474699+0000","last_active":"2024-09-13T18:22:10.474473+0000","last_peered":"2024-09-13T18:22:10.474473+0000","last_clean":"2024-09-13T18:22:10.474473+0000","last_became_active":"2024-09-13T17:50:38.614284+0000","last_became_peered":"2024-09-13T17:50:38.614284+0000","last_unstale":"2024-09-13T18:22:10.474473+0000","last_undegraded":"2024-09-13T18:22:10.474473+0000","last_fullsized":"2024-09-13T18:22:10.474473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4887","last_scrub_stamp":"2024-09-13T18:22:10.474424+0000","last_deep_scrub":"27'4887","last_deep_scrub_stamp":"2024-09-13T18:22:10.474424+0000","last_clean_scrub_stamp":"2024-09-13T18:22:10.474424+0000","objects_scrubbed":0,"log_size":4887,"log_dups_size":0,"ondisk_log_size":4887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:48.155529+0000","scrub_duration":0.038393351999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20690,"num_write_kb":442963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'16","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:53.757522+0000","last_change":"2024-09-13T18:21:53.757522+0000","last_active":"2024-09-13T18:21:53.757522+0000","last_peered":"2024-09-13T18:21:53.757522+0000","last_clean":"2024-09-13T18:21:53.757522+0000","last_became_active":"2024-09-13T17:50:34.588317+0000","last_became_peered":"2024-09-13T17:50:34.588317+0000","last_unstale":"2024-09-13T18:21:53.757522+0000","last_undegraded":"2024-09-13T18:21:53.757522+0000","last_fullsized":"2024-09-13T18:21:53.757522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-09-13T18:21:53.757275+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-09-13T18:21:53.757275+0000","last_clean_scrub_stamp":"2024-09-13T18:21:53.757275+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:03:27.571838+0000","scrub_duration":0.003575366,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'3","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:49.455036+0000","last_change":"2024-09-13T18:21:49.455036+0000","last_active":"2024-09-13T18:21:49.455036+0000","last_peered":"2024-09-13T18:21:49.455036+0000","last_clean":"2024-09-13T18:21:49.455036+0000","last_became_active":"2024-09-13T17:50:34.204060+0000","last_became_peered":"2024-09-13T17:50:34.204060+0000","last_unstale":"2024-09-13T18:21:49.455036+0000","last_undegraded":"2024-09-13T18:21:49.455036+0000","last_fullsized":"2024-09-13T18:21:49.455036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-09-13T18:21:49.454804+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-09-13T18:21:49.454804+0000","last_clean_scrub_stamp":"2024-09-13T18:21:49.454804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:26.292901+0000","scrub_duration":0.0030762760000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'6031","reported_seq":6072,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:04.425037+0000","last_change":"2024-09-13T18:22:04.425153+0000","last_active":"2024-09-13T18:22:04.425037+0000","last_peered":"2024-09-13T18:22:04.425037+0000","last_clean":"2024-09-13T18:22:04.425037+0000","last_became_active":"2024-09-13T17:50:38.617257+0000","last_became_peered":"2024-09-13T17:50:38.617257+0000","last_unstale":"2024-09-13T18:22:04.425037+0000","last_undegraded":"2024-09-13T18:22:04.425037+0000","last_fullsized":"2024-09-13T18:22:04.425037+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6031","last_scrub_stamp":"2024-09-13T18:22:04.425017+0000","last_deep_scrub":"27'6031","last_deep_scrub_stamp":"2024-09-13T18:22:04.425017+0000","last_clean_scrub_stamp":"2024-09-13T18:22:04.425017+0000","objects_scrubbed":0,"log_size":6031,"log_dups_size":0,"ondisk_log_size":6031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:32.439206+0000","scrub_duration":0.045605909,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25529,"num_write_kb":564991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'16","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:45.268600+0000","last_change":"2024-09-13T18:21:45.268600+0000","last_active":"2024-09-13T18:21:45.268600+0000","last_peered":"2024-09-13T18:21:45.268600+0000","last_clean":"2024-09-13T18:21:45.268600+0000","last_became_active":"2024-09-13T17:50:34.588063+0000","last_became_peered":"2024-09-13T17:50:34.588063+0000","last_unstale":"2024-09-13T18:21:45.268600+0000","last_undegraded":"2024-09-13T18:21:45.268600+0000","last_fullsized":"2024-09-13T18:21:45.268600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-09-13T18:21:45.268335+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-09-13T18:21:45.268335+0000","last_clean_scrub_stamp":"2024-09-13T18:21:45.268335+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:13:11.956726+0000","scrub_duration":0.0030026689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'4","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:35.893165+0000","last_change":"2024-09-13T18:21:35.893165+0000","last_active":"2024-09-13T18:21:35.893165+0000","last_peered":"2024-09-13T18:21:35.893165+0000","last_clean":"2024-09-13T18:21:35.893165+0000","last_became_active":"2024-09-13T17:50:34.208196+0000","last_became_peered":"2024-09-13T17:50:34.208196+0000","last_unstale":"2024-09-13T18:21:35.893165+0000","last_undegraded":"2024-09-13T18:21:35.893165+0000","last_fullsized":"2024-09-13T18:21:35.893165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-09-13T18:21:35.892908+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-09-13T18:21:35.892908+0000","last_clean_scrub_stamp":"2024-09-13T18:21:35.892908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:50.582547+0000","scrub_duration":0.0034542710000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'5410","reported_seq":5451,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:00.439430+0000","last_change":"2024-09-13T18:22:00.439430+0000","last_active":"2024-09-13T18:22:00.439430+0000","last_peered":"2024-09-13T18:22:00.439430+0000","last_clean":"2024-09-13T18:22:00.439430+0000","last_became_active":"2024-09-13T17:50:38.609794+0000","last_became_peered":"2024-09-13T17:50:38.609794+0000","last_unstale":"2024-09-13T18:22:00.439430+0000","last_undegraded":"2024-09-13T18:22:00.439430+0000","last_fullsized":"2024-09-13T18:22:00.439430+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5410","last_scrub_stamp":"2024-09-13T18:22:00.439240+0000","last_deep_scrub":"27'5410","last_deep_scrub_stamp":"2024-09-13T18:22:00.439240+0000","last_clean_scrub_stamp":"2024-09-13T18:22:00.439240+0000","objects_scrubbed":0,"log_size":5410,"log_dups_size":0,"ondisk_log_size":5410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:57:58.327181+0000","scrub_duration":0.041838701999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22186,"num_write_kb":506224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'22","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:52.457138+0000","last_change":"2024-09-13T18:21:52.457337+0000","last_active":"2024-09-13T18:21:52.457138+0000","last_peered":"2024-09-13T18:21:52.457138+0000","last_clean":"2024-09-13T18:21:52.457138+0000","last_became_active":"2024-09-13T17:50:34.584023+0000","last_became_peered":"2024-09-13T17:50:34.584023+0000","last_unstale":"2024-09-13T18:21:52.457138+0000","last_undegraded":"2024-09-13T18:21:52.457138+0000","last_fullsized":"2024-09-13T18:21:52.457138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-09-13T18:21:52.457074+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-09-13T18:21:52.457074+0000","last_clean_scrub_stamp":"2024-09-13T18:21:52.457074+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:06:10.138005+0000","scrub_duration":0.0030016969999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'130","reported_seq":295,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:43.721726+0000","last_change":"2024-09-13T18:21:43.721726+0000","last_active":"2024-09-13T18:21:43.721726+0000","last_peered":"2024-09-13T18:21:43.721726+0000","last_clean":"2024-09-13T18:21:43.721726+0000","last_became_active":"2024-09-13T17:50:34.198603+0000","last_became_peered":"2024-09-13T17:50:34.198603+0000","last_unstale":"2024-09-13T18:21:43.721726+0000","last_undegraded":"2024-09-13T18:21:43.721726+0000","last_fullsized":"2024-09-13T18:21:43.721726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'130","last_scrub_stamp":"2024-09-13T18:21:43.721576+0000","last_deep_scrub":"27'130","last_deep_scrub_stamp":"2024-09-13T18:21:43.721576+0000","last_clean_scrub_stamp":"2024-09-13T18:21:43.721576+0000","objects_scrubbed":3,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:31.923344+0000","scrub_duration":0.0076490960000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":94032,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":179,"num_write":125,"num_write_kb":180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'5807","reported_seq":5847,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.539082+0000","last_change":"2024-09-13T18:21:55.539241+0000","last_active":"2024-09-13T18:21:55.539082+0000","last_peered":"2024-09-13T18:21:55.539082+0000","last_clean":"2024-09-13T18:21:55.539082+0000","last_became_active":"2024-09-13T17:50:38.610900+0000","last_became_peered":"2024-09-13T17:50:38.610900+0000","last_unstale":"2024-09-13T18:21:55.539082+0000","last_undegraded":"2024-09-13T18:21:55.539082+0000","last_fullsized":"2024-09-13T18:21:55.539082+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5807","last_scrub_stamp":"2024-09-13T18:21:55.539055+0000","last_deep_scrub":"27'5807","last_deep_scrub_stamp":"2024-09-13T18:21:55.539055+0000","last_clean_scrub_stamp":"2024-09-13T18:21:55.539055+0000","objects_scrubbed":0,"log_size":5807,"log_dups_size":0,"ondisk_log_size":5807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:19.172462+0000","scrub_duration":0.050879777000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24429,"num_write_kb":531605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'13","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:32.284016+0000","last_change":"2024-09-13T18:21:32.284016+0000","last_active":"2024-09-13T18:21:32.284016+0000","last_peered":"2024-09-13T18:21:32.284016+0000","last_clean":"2024-09-13T18:21:32.284016+0000","last_became_active":"2024-09-13T17:50:34.594201+0000","last_became_peered":"2024-09-13T17:50:34.594201+0000","last_unstale":"2024-09-13T18:21:32.284016+0000","last_undegraded":"2024-09-13T18:21:32.284016+0000","last_fullsized":"2024-09-13T18:21:32.284016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'13","last_scrub_stamp":"2024-09-13T18:21:32.283704+0000","last_deep_scrub":"27'13","last_deep_scrub_stamp":"2024-09-13T18:21:32.283704+0000","last_clean_scrub_stamp":"2024-09-13T18:21:32.283704+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:06.348602+0000","scrub_duration":0.0033353039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'9","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:45.449467+0000","last_change":"2024-09-13T18:21:45.449664+0000","last_active":"2024-09-13T18:21:45.449467+0000","last_peered":"2024-09-13T18:21:45.449467+0000","last_clean":"2024-09-13T18:21:45.449467+0000","last_became_active":"2024-09-13T17:50:34.218069+0000","last_became_peered":"2024-09-13T17:50:34.218069+0000","last_unstale":"2024-09-13T18:21:45.449467+0000","last_undegraded":"2024-09-13T18:21:45.449467+0000","last_fullsized":"2024-09-13T18:21:45.449467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-09-13T18:21:45.449415+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-09-13T18:21:45.449415+0000","last_clean_scrub_stamp":"2024-09-13T18:21:45.449415+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:45.223728+0000","scrub_duration":0.0064944169999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'4957","reported_seq":4997,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:08.402578+0000","last_change":"2024-09-13T18:22:08.402774+0000","last_active":"2024-09-13T18:22:08.402578+0000","last_peered":"2024-09-13T18:22:08.402578+0000","last_clean":"2024-09-13T18:22:08.402578+0000","last_became_active":"2024-09-13T17:50:38.612856+0000","last_became_peered":"2024-09-13T17:50:38.612856+0000","last_unstale":"2024-09-13T18:22:08.402578+0000","last_undegraded":"2024-09-13T18:22:08.402578+0000","last_fullsized":"2024-09-13T18:22:08.402578+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4957","last_scrub_stamp":"2024-09-13T18:22:08.402540+0000","last_deep_scrub":"27'4957","last_deep_scrub_stamp":"2024-09-13T18:22:08.402540+0000","last_clean_scrub_stamp":"2024-09-13T18:22:08.402540+0000","objects_scrubbed":0,"log_size":4957,"log_dups_size":0,"ondisk_log_size":4957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:09.900500+0000","scrub_duration":0.033427548000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20520,"num_write_kb":434212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'16","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:43.170888+0000","last_change":"2024-09-13T18:21:43.170888+0000","last_active":"2024-09-13T18:21:43.170888+0000","last_peered":"2024-09-13T18:21:43.170888+0000","last_clean":"2024-09-13T18:21:43.170888+0000","last_became_active":"2024-09-13T17:50:34.582446+0000","last_became_peered":"2024-09-13T17:50:34.582446+0000","last_unstale":"2024-09-13T18:21:43.170888+0000","last_undegraded":"2024-09-13T18:21:43.170888+0000","last_fullsized":"2024-09-13T18:21:43.170888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-09-13T18:21:43.170750+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-09-13T18:21:43.170750+0000","last_clean_scrub_stamp":"2024-09-13T18:21:43.170750+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:39.625283+0000","scrub_duration":0.0027720919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'11","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.797573+0000","last_change":"2024-09-13T18:21:28.797573+0000","last_active":"2024-09-13T18:21:28.797573+0000","last_peered":"2024-09-13T18:21:28.797573+0000","last_clean":"2024-09-13T18:21:28.797573+0000","last_became_active":"2024-09-13T17:50:34.200873+0000","last_became_peered":"2024-09-13T17:50:34.200873+0000","last_unstale":"2024-09-13T18:21:28.797573+0000","last_undegraded":"2024-09-13T18:21:28.797573+0000","last_fullsized":"2024-09-13T18:21:28.797573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-09-13T18:21:28.797316+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-09-13T18:21:28.797316+0000","last_clean_scrub_stamp":"2024-09-13T18:21:28.797316+0000","objects_scrubbed":3,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:51:50.633990+0000","scrub_duration":0.0090957169999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"27'119","reported_seq":199,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:41.366780+0000","last_change":"2024-09-13T18:21:41.367018+0000","last_active":"2024-09-13T18:21:41.366780+0000","last_peered":"2024-09-13T18:21:41.366780+0000","last_clean":"2024-09-13T18:21:41.366780+0000","last_became_active":"2024-09-13T17:50:22.093734+0000","last_became_peered":"2024-09-13T17:50:22.093734+0000","last_unstale":"2024-09-13T18:21:41.366780+0000","last_undegraded":"2024-09-13T18:21:41.366780+0000","last_fullsized":"2024-09-13T18:21:41.366780+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'119","last_scrub_stamp":"2024-09-13T18:21:41.366718+0000","last_deep_scrub":"27'119","last_deep_scrub_stamp":"2024-09-13T18:21:41.366718+0000","last_clean_scrub_stamp":"2024-09-13T18:21:41.366718+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:37:54.724540+0000","scrub_duration":0.0086725589999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'5208","reported_seq":5249,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:52.415372+0000","last_change":"2024-09-13T18:21:52.415372+0000","last_active":"2024-09-13T18:21:52.415372+0000","last_peered":"2024-09-13T18:21:52.415372+0000","last_clean":"2024-09-13T18:21:52.415372+0000","last_became_active":"2024-09-13T17:50:38.618488+0000","last_became_peered":"2024-09-13T17:50:38.618488+0000","last_unstale":"2024-09-13T18:21:52.415372+0000","last_undegraded":"2024-09-13T18:21:52.415372+0000","last_fullsized":"2024-09-13T18:21:52.415372+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5208","last_scrub_stamp":"2024-09-13T18:21:52.415203+0000","last_deep_scrub":"27'5208","last_deep_scrub_stamp":"2024-09-13T18:21:52.415203+0000","last_clean_scrub_stamp":"2024-09-13T18:21:52.415203+0000","objects_scrubbed":0,"log_size":5208,"log_dups_size":0,"ondisk_log_size":5208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:21.608219+0000","scrub_duration":0.035964871000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22093,"num_write_kb":481370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'22","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:25.954970+0000","last_change":"2024-09-13T17:50:34.585881+0000","last_active":"2024-09-13T18:21:25.954970+0000","last_peered":"2024-09-13T18:21:25.954970+0000","last_clean":"2024-09-13T18:21:25.954970+0000","last_became_active":"2024-09-13T17:50:34.585166+0000","last_became_peered":"2024-09-13T17:50:34.585166+0000","last_unstale":"2024-09-13T18:21:25.954970+0000","last_undegraded":"2024-09-13T18:21:25.954970+0000","last_fullsized":"2024-09-13T18:21:25.954970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'7","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:34.775276+0000","last_change":"2024-09-13T18:21:34.775276+0000","last_active":"2024-09-13T18:21:34.775276+0000","last_peered":"2024-09-13T18:21:34.775276+0000","last_clean":"2024-09-13T18:21:34.775276+0000","last_became_active":"2024-09-13T17:50:34.210589+0000","last_became_peered":"2024-09-13T17:50:34.210589+0000","last_unstale":"2024-09-13T18:21:34.775276+0000","last_undegraded":"2024-09-13T18:21:34.775276+0000","last_fullsized":"2024-09-13T18:21:34.775276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-13T18:21:34.775137+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-13T18:21:34.775137+0000","last_clean_scrub_stamp":"2024-09-13T18:21:34.775137+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:40.007622+0000","scrub_duration":0.0064282710000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'5302","reported_seq":5352,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:50.384839+0000","last_change":"2024-09-13T18:21:50.385188+0000","last_active":"2024-09-13T18:21:50.384839+0000","last_peered":"2024-09-13T18:21:50.384839+0000","last_clean":"2024-09-13T18:21:50.384839+0000","last_became_active":"2024-09-13T17:51:16.363974+0000","last_became_peered":"2024-09-13T17:51:16.363974+0000","last_unstale":"2024-09-13T18:21:50.384839+0000","last_undegraded":"2024-09-13T18:21:50.384839+0000","last_fullsized":"2024-09-13T18:21:50.384839+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5302","last_scrub_stamp":"2024-09-13T18:21:50.384756+0000","last_deep_scrub":"27'5302","last_deep_scrub_stamp":"2024-09-13T18:21:50.384756+0000","last_clean_scrub_stamp":"2024-09-13T18:21:50.384756+0000","objects_scrubbed":0,"log_size":5302,"log_dups_size":0,"ondisk_log_size":5302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:19.818627+0000","scrub_duration":0.048673297999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22724,"num_write_kb":488088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'20","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:02.792555+0000","last_change":"2024-09-13T18:22:02.792555+0000","last_active":"2024-09-13T18:22:02.792555+0000","last_peered":"2024-09-13T18:22:02.792555+0000","last_clean":"2024-09-13T18:22:02.792555+0000","last_became_active":"2024-09-13T17:50:34.589434+0000","last_became_peered":"2024-09-13T17:50:34.589434+0000","last_unstale":"2024-09-13T18:22:02.792555+0000","last_undegraded":"2024-09-13T18:22:02.792555+0000","last_fullsized":"2024-09-13T18:22:02.792555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-13T18:22:02.792302+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-13T18:22:02.792302+0000","last_clean_scrub_stamp":"2024-09-13T18:22:02.792302+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:46:31.470983+0000","scrub_duration":0.0033502279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'5","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:33.384001+0000","last_change":"2024-09-13T18:21:33.384001+0000","last_active":"2024-09-13T18:21:33.384001+0000","last_peered":"2024-09-13T18:21:33.384001+0000","last_clean":"2024-09-13T18:21:33.384001+0000","last_became_active":"2024-09-13T17:50:34.203874+0000","last_became_peered":"2024-09-13T17:50:34.203874+0000","last_unstale":"2024-09-13T18:21:33.384001+0000","last_undegraded":"2024-09-13T18:21:33.384001+0000","last_fullsized":"2024-09-13T18:21:33.384001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-09-13T18:21:33.383770+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-09-13T18:21:33.383770+0000","last_clean_scrub_stamp":"2024-09-13T18:21:33.383770+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:07.542936+0000","scrub_duration":0.0056205279999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'5348","reported_seq":5388,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:16.534723+0000","last_change":"2024-09-13T18:22:16.534840+0000","last_active":"2024-09-13T18:22:16.534723+0000","last_peered":"2024-09-13T18:22:16.534723+0000","last_clean":"2024-09-13T18:22:16.534723+0000","last_became_active":"2024-09-13T17:50:38.615530+0000","last_became_peered":"2024-09-13T17:50:38.615530+0000","last_unstale":"2024-09-13T18:22:16.534723+0000","last_undegraded":"2024-09-13T18:22:16.534723+0000","last_fullsized":"2024-09-13T18:22:16.534723+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5348","last_scrub_stamp":"2024-09-13T18:22:16.534702+0000","last_deep_scrub":"27'5348","last_deep_scrub_stamp":"2024-09-13T18:22:16.534702+0000","last_clean_scrub_stamp":"2024-09-13T18:22:16.534702+0000","objects_scrubbed":0,"log_size":5348,"log_dups_size":0,"ondisk_log_size":5348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:50:38.231077+0000","scrub_duration":0.032334993999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22134,"num_write_kb":473293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'20","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:17.369117+0000","last_change":"2024-09-13T18:22:17.369341+0000","last_active":"2024-09-13T18:22:17.369117+0000","last_peered":"2024-09-13T18:22:17.369117+0000","last_clean":"2024-09-13T18:22:17.369117+0000","last_became_active":"2024-09-13T17:50:34.594842+0000","last_became_peered":"2024-09-13T17:50:34.594842+0000","last_unstale":"2024-09-13T18:22:17.369117+0000","last_undegraded":"2024-09-13T18:22:17.369117+0000","last_fullsized":"2024-09-13T18:22:17.369117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-13T18:22:17.369066+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-13T18:22:17.369066+0000","last_clean_scrub_stamp":"2024-09-13T18:22:17.369066+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:18.496921+0000","scrub_duration":0.003511508,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'10","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:42.696280+0000","last_change":"2024-09-13T18:21:42.696280+0000","last_active":"2024-09-13T18:21:42.696280+0000","last_peered":"2024-09-13T18:21:42.696280+0000","last_clean":"2024-09-13T18:21:42.696280+0000","last_became_active":"2024-09-13T17:50:34.199650+0000","last_became_peered":"2024-09-13T17:50:34.199650+0000","last_unstale":"2024-09-13T18:21:42.696280+0000","last_undegraded":"2024-09-13T18:21:42.696280+0000","last_fullsized":"2024-09-13T18:21:42.696280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-13T18:21:42.696034+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-13T18:21:42.696034+0000","last_clean_scrub_stamp":"2024-09-13T18:21:42.696034+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:47:19.246483+0000","scrub_duration":0.0045639139999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'5161","reported_seq":5201,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:05.843990+0000","last_change":"2024-09-13T18:22:05.844240+0000","last_active":"2024-09-13T18:22:05.843990+0000","last_peered":"2024-09-13T18:22:05.843990+0000","last_clean":"2024-09-13T18:22:05.843990+0000","last_became_active":"2024-09-13T17:50:38.609354+0000","last_became_peered":"2024-09-13T17:50:38.609354+0000","last_unstale":"2024-09-13T18:22:05.843990+0000","last_undegraded":"2024-09-13T18:22:05.843990+0000","last_fullsized":"2024-09-13T18:22:05.843990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5161","last_scrub_stamp":"2024-09-13T18:22:05.843925+0000","last_deep_scrub":"27'5161","last_deep_scrub_stamp":"2024-09-13T18:22:05.843925+0000","last_clean_scrub_stamp":"2024-09-13T18:22:05.843925+0000","objects_scrubbed":0,"log_size":5161,"log_dups_size":0,"ondisk_log_size":5161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:32.045019+0000","scrub_duration":0.039900685999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21929,"num_write_kb":464406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'10","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:26.347263+0000","last_change":"2024-09-13T18:21:26.347263+0000","last_active":"2024-09-13T18:21:26.347263+0000","last_peered":"2024-09-13T18:21:26.347263+0000","last_clean":"2024-09-13T18:21:26.347263+0000","last_became_active":"2024-09-13T17:50:34.594876+0000","last_became_peered":"2024-09-13T17:50:34.594876+0000","last_unstale":"2024-09-13T18:21:26.347263+0000","last_undegraded":"2024-09-13T18:21:26.347263+0000","last_fullsized":"2024-09-13T18:21:26.347263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-13T18:21:26.346976+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-13T18:21:26.346976+0000","last_clean_scrub_stamp":"2024-09-13T18:21:26.346976+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:25.039233+0000","scrub_duration":0.003420939,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'6","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.264384+0000","last_change":"2024-09-13T18:21:28.264384+0000","last_active":"2024-09-13T18:21:28.264384+0000","last_peered":"2024-09-13T18:21:28.264384+0000","last_clean":"2024-09-13T18:21:28.264384+0000","last_became_active":"2024-09-13T17:50:34.200197+0000","last_became_peered":"2024-09-13T17:50:34.200197+0000","last_unstale":"2024-09-13T18:21:28.264384+0000","last_undegraded":"2024-09-13T18:21:28.264384+0000","last_fullsized":"2024-09-13T18:21:28.264384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-09-13T18:21:28.264112+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-09-13T18:21:28.264112+0000","last_clean_scrub_stamp":"2024-09-13T18:21:28.264112+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:43:40.034203+0000","scrub_duration":0.0069816829999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'5052","reported_seq":5102,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.356895+0000","last_change":"2024-09-13T18:21:55.357111+0000","last_active":"2024-09-13T18:21:55.356895+0000","last_peered":"2024-09-13T18:21:55.356895+0000","last_clean":"2024-09-13T18:21:55.356895+0000","last_became_active":"2024-09-13T17:51:16.281476+0000","last_became_peered":"2024-09-13T17:51:16.281476+0000","last_unstale":"2024-09-13T18:21:55.356895+0000","last_undegraded":"2024-09-13T18:21:55.356895+0000","last_fullsized":"2024-09-13T18:21:55.356895+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5052","last_scrub_stamp":"2024-09-13T18:21:55.356852+0000","last_deep_scrub":"27'5052","last_deep_scrub_stamp":"2024-09-13T18:21:55.356852+0000","last_clean_scrub_stamp":"2024-09-13T18:21:55.356852+0000","objects_scrubbed":0,"log_size":5052,"log_dups_size":0,"ondisk_log_size":5052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:26:46.603249+0000","scrub_duration":0.044667539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20895,"num_write_kb":464282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3,6],"acting":[4,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'18","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:03.563631+0000","last_change":"2024-09-13T18:22:03.563867+0000","last_active":"2024-09-13T18:22:03.563631+0000","last_peered":"2024-09-13T18:22:03.563631+0000","last_clean":"2024-09-13T18:22:03.563631+0000","last_became_active":"2024-09-13T17:50:34.585464+0000","last_became_peered":"2024-09-13T17:50:34.585464+0000","last_unstale":"2024-09-13T18:22:03.563631+0000","last_undegraded":"2024-09-13T18:22:03.563631+0000","last_fullsized":"2024-09-13T18:22:03.563631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:22:03.563575+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:22:03.563575+0000","last_clean_scrub_stamp":"2024-09-13T18:22:03.563575+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:37.401878+0000","scrub_duration":0.0038625619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'27","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:36.784439+0000","last_change":"2024-09-13T18:21:36.784589+0000","last_active":"2024-09-13T18:21:36.784439+0000","last_peered":"2024-09-13T18:21:36.784439+0000","last_clean":"2024-09-13T18:21:36.784439+0000","last_became_active":"2024-09-13T17:50:34.200936+0000","last_became_peered":"2024-09-13T17:50:34.200936+0000","last_unstale":"2024-09-13T18:21:36.784439+0000","last_undegraded":"2024-09-13T18:21:36.784439+0000","last_fullsized":"2024-09-13T18:21:36.784439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'27","last_scrub_stamp":"2024-09-13T18:21:36.784405+0000","last_deep_scrub":"27'27","last_deep_scrub_stamp":"2024-09-13T18:21:36.784405+0000","last_clean_scrub_stamp":"2024-09-13T18:21:36.784405+0000","objects_scrubbed":8,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:11.675692+0000","scrub_duration":0.012535223,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'6502","reported_seq":6530,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:22.640357+0000","last_change":"2024-09-13T17:50:38.615229+0000","last_active":"2024-09-13T18:21:22.640357+0000","last_peered":"2024-09-13T18:21:22.640357+0000","last_clean":"2024-09-13T18:21:22.640357+0000","last_became_active":"2024-09-13T17:50:38.614739+0000","last_became_peered":"2024-09-13T17:50:38.614739+0000","last_unstale":"2024-09-13T18:21:22.640357+0000","last_undegraded":"2024-09-13T18:21:22.640357+0000","last_fullsized":"2024-09-13T18:21:22.640357+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6502,"log_dups_size":0,"ondisk_log_size":6502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27696,"num_write_kb":544723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'20","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:34.335658+0000","last_change":"2024-09-13T18:21:34.335970+0000","last_active":"2024-09-13T18:21:34.335658+0000","last_peered":"2024-09-13T18:21:34.335658+0000","last_clean":"2024-09-13T18:21:34.335658+0000","last_became_active":"2024-09-13T17:50:34.588565+0000","last_became_peered":"2024-09-13T17:50:34.588565+0000","last_unstale":"2024-09-13T18:21:34.335658+0000","last_undegraded":"2024-09-13T18:21:34.335658+0000","last_fullsized":"2024-09-13T18:21:34.335658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-13T18:21:34.335555+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-13T18:21:34.335555+0000","last_clean_scrub_stamp":"2024-09-13T18:21:34.335555+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:28.955486+0000","scrub_duration":0.0039531770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'9","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:26.414088+0000","last_change":"2024-09-13T18:21:26.414323+0000","last_active":"2024-09-13T18:21:26.414088+0000","last_peered":"2024-09-13T18:21:26.414088+0000","last_clean":"2024-09-13T18:21:26.414088+0000","last_became_active":"2024-09-13T17:50:34.197839+0000","last_became_peered":"2024-09-13T17:50:34.197839+0000","last_unstale":"2024-09-13T18:21:26.414088+0000","last_undegraded":"2024-09-13T18:21:26.414088+0000","last_fullsized":"2024-09-13T18:21:26.414088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-09-13T18:21:26.414037+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-09-13T18:21:26.414037+0000","last_clean_scrub_stamp":"2024-09-13T18:21:26.414037+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:22.483872+0000","scrub_duration":0.0068422659999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'6326","reported_seq":6365,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.238971+0000","last_change":"2024-09-13T18:22:07.239099+0000","last_active":"2024-09-13T18:22:07.238971+0000","last_peered":"2024-09-13T18:22:07.238971+0000","last_clean":"2024-09-13T18:22:07.238971+0000","last_became_active":"2024-09-13T17:50:38.610688+0000","last_became_peered":"2024-09-13T17:50:38.610688+0000","last_unstale":"2024-09-13T18:22:07.238971+0000","last_undegraded":"2024-09-13T18:22:07.238971+0000","last_fullsized":"2024-09-13T18:22:07.238971+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6326","last_scrub_stamp":"2024-09-13T18:22:07.238951+0000","last_deep_scrub":"27'6326","last_deep_scrub_stamp":"2024-09-13T18:22:07.238951+0000","last_clean_scrub_stamp":"2024-09-13T18:22:07.238951+0000","objects_scrubbed":0,"log_size":6326,"log_dups_size":0,"ondisk_log_size":6326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:54.185569+0000","scrub_duration":0.043673352999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27199,"num_write_kb":538670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'29","reported_seq":70,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.460377+0000","last_change":"2024-09-13T18:22:07.460610+0000","last_active":"2024-09-13T18:22:07.460377+0000","last_peered":"2024-09-13T18:22:07.460377+0000","last_clean":"2024-09-13T18:22:07.460377+0000","last_became_active":"2024-09-13T17:50:34.594471+0000","last_became_peered":"2024-09-13T17:50:34.594471+0000","last_unstale":"2024-09-13T18:22:07.460377+0000","last_undegraded":"2024-09-13T18:22:07.460377+0000","last_fullsized":"2024-09-13T18:22:07.460377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'29","last_scrub_stamp":"2024-09-13T18:22:07.460318+0000","last_deep_scrub":"27'29","last_deep_scrub_stamp":"2024-09-13T18:22:07.460318+0000","last_clean_scrub_stamp":"2024-09-13T18:22:07.460318+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:13.560724+0000","scrub_duration":0.0031528630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'7","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:37.911671+0000","last_change":"2024-09-13T18:21:37.911888+0000","last_active":"2024-09-13T18:21:37.911671+0000","last_peered":"2024-09-13T18:21:37.911671+0000","last_clean":"2024-09-13T18:21:37.911671+0000","last_became_active":"2024-09-13T17:50:34.208817+0000","last_became_peered":"2024-09-13T17:50:34.208817+0000","last_unstale":"2024-09-13T18:21:37.911671+0000","last_undegraded":"2024-09-13T18:21:37.911671+0000","last_fullsized":"2024-09-13T18:21:37.911671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-13T18:21:37.911585+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-13T18:21:37.911585+0000","last_clean_scrub_stamp":"2024-09-13T18:21:37.911585+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:43:22.483981+0000","scrub_duration":0.0050921259999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'5788","reported_seq":5827,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:57.298804+0000","last_change":"2024-09-13T18:21:57.298969+0000","last_active":"2024-09-13T18:21:57.298804+0000","last_peered":"2024-09-13T18:21:57.298804+0000","last_clean":"2024-09-13T18:21:57.298804+0000","last_became_active":"2024-09-13T17:50:38.609128+0000","last_became_peered":"2024-09-13T17:50:38.609128+0000","last_unstale":"2024-09-13T18:21:57.298804+0000","last_undegraded":"2024-09-13T18:21:57.298804+0000","last_fullsized":"2024-09-13T18:21:57.298804+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5788","last_scrub_stamp":"2024-09-13T18:21:57.298769+0000","last_deep_scrub":"27'5788","last_deep_scrub_stamp":"2024-09-13T18:21:57.298769+0000","last_clean_scrub_stamp":"2024-09-13T18:21:57.298769+0000","objects_scrubbed":0,"log_size":5788,"log_dups_size":0,"ondisk_log_size":5788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:11:22.245296+0000","scrub_duration":0.044309794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24140,"num_write_kb":538739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'24","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:49.332934+0000","last_change":"2024-09-13T18:21:49.333129+0000","last_active":"2024-09-13T18:21:49.332934+0000","last_peered":"2024-09-13T18:21:49.332934+0000","last_clean":"2024-09-13T18:21:49.332934+0000","last_became_active":"2024-09-13T17:50:34.592669+0000","last_became_peered":"2024-09-13T17:50:34.592669+0000","last_unstale":"2024-09-13T18:21:49.332934+0000","last_undegraded":"2024-09-13T18:21:49.332934+0000","last_fullsized":"2024-09-13T18:21:49.332934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-09-13T18:21:49.332882+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-09-13T18:21:49.332882+0000","last_clean_scrub_stamp":"2024-09-13T18:21:49.332882+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:50.817551+0000","scrub_duration":0.0025807830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'19","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.241499+0000","last_change":"2024-09-13T18:21:28.241791+0000","last_active":"2024-09-13T18:21:28.241499+0000","last_peered":"2024-09-13T18:21:28.241499+0000","last_clean":"2024-09-13T18:21:28.241499+0000","last_became_active":"2024-09-13T17:50:34.200529+0000","last_became_peered":"2024-09-13T17:50:34.200529+0000","last_unstale":"2024-09-13T18:21:28.241499+0000","last_undegraded":"2024-09-13T18:21:28.241499+0000","last_fullsized":"2024-09-13T18:21:28.241499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-13T18:21:28.241430+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-13T18:21:28.241430+0000","last_clean_scrub_stamp":"2024-09-13T18:21:28.241430+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:08:39.927576+0000","scrub_duration":0.00889181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'5464","reported_seq":5503,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:06.898678+0000","last_change":"2024-09-13T18:22:06.898922+0000","last_active":"2024-09-13T18:22:06.898678+0000","last_peered":"2024-09-13T18:22:06.898678+0000","last_clean":"2024-09-13T18:22:06.898678+0000","last_became_active":"2024-09-13T17:50:38.605826+0000","last_became_peered":"2024-09-13T17:50:38.605826+0000","last_unstale":"2024-09-13T18:22:06.898678+0000","last_undegraded":"2024-09-13T18:22:06.898678+0000","last_fullsized":"2024-09-13T18:22:06.898678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5464","last_scrub_stamp":"2024-09-13T18:22:06.898615+0000","last_deep_scrub":"27'5464","last_deep_scrub_stamp":"2024-09-13T18:22:06.898615+0000","last_clean_scrub_stamp":"2024-09-13T18:22:06.898615+0000","objects_scrubbed":0,"log_size":5464,"log_dups_size":0,"ondisk_log_size":5464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:41.161935+0000","scrub_duration":0.051658339999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23044,"num_write_kb":472739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'18","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:25.955022+0000","last_change":"2024-09-13T17:50:34.591579+0000","last_active":"2024-09-13T18:21:25.955022+0000","last_peered":"2024-09-13T18:21:25.955022+0000","last_clean":"2024-09-13T18:21:25.955022+0000","last_became_active":"2024-09-13T17:50:34.586165+0000","last_became_peered":"2024-09-13T17:50:34.586165+0000","last_unstale":"2024-09-13T18:21:25.955022+0000","last_undegraded":"2024-09-13T18:21:25.955022+0000","last_fullsized":"2024-09-13T18:21:25.955022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'421","reported_seq":463,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:31.308661+0000","last_change":"2024-09-13T18:21:31.308816+0000","last_active":"2024-09-13T18:21:31.308661+0000","last_peered":"2024-09-13T18:21:31.308661+0000","last_clean":"2024-09-13T18:21:31.308661+0000","last_became_active":"2024-09-13T17:50:34.202948+0000","last_became_peered":"2024-09-13T17:50:34.202948+0000","last_unstale":"2024-09-13T18:21:31.308661+0000","last_undegraded":"2024-09-13T18:21:31.308661+0000","last_fullsized":"2024-09-13T18:21:31.308661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'421","last_scrub_stamp":"2024-09-13T18:21:31.308629+0000","last_deep_scrub":"27'421","last_deep_scrub_stamp":"2024-09-13T18:21:31.308629+0000","last_clean_scrub_stamp":"2024-09-13T18:21:31.308629+0000","objects_scrubbed":5,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:55.983699+0000","scrub_duration":0.024451271,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":4324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'5887","reported_seq":5926,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:57.523458+0000","last_change":"2024-09-13T18:21:57.523677+0000","last_active":"2024-09-13T18:21:57.523458+0000","last_peered":"2024-09-13T18:21:57.523458+0000","last_clean":"2024-09-13T18:21:57.523458+0000","last_became_active":"2024-09-13T17:50:38.610055+0000","last_became_peered":"2024-09-13T17:50:38.610055+0000","last_unstale":"2024-09-13T18:21:57.523458+0000","last_undegraded":"2024-09-13T18:21:57.523458+0000","last_fullsized":"2024-09-13T18:21:57.523458+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5887","last_scrub_stamp":"2024-09-13T18:21:57.523426+0000","last_deep_scrub":"27'5887","last_deep_scrub_stamp":"2024-09-13T18:21:57.523426+0000","last_clean_scrub_stamp":"2024-09-13T18:21:57.523426+0000","objects_scrubbed":0,"log_size":5887,"log_dups_size":0,"ondisk_log_size":5887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:10:56.767652+0000","scrub_duration":0.047711444999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24664,"num_write_kb":495799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'21","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:39.354348+0000","last_change":"2024-09-13T18:21:39.354570+0000","last_active":"2024-09-13T18:21:39.354348+0000","last_peered":"2024-09-13T18:21:39.354348+0000","last_clean":"2024-09-13T18:21:39.354348+0000","last_became_active":"2024-09-13T17:50:34.593454+0000","last_became_peered":"2024-09-13T17:50:34.593454+0000","last_unstale":"2024-09-13T18:21:39.354348+0000","last_undegraded":"2024-09-13T18:21:39.354348+0000","last_fullsized":"2024-09-13T18:21:39.354348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-09-13T18:21:39.354296+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-09-13T18:21:39.354296+0000","last_clean_scrub_stamp":"2024-09-13T18:21:39.354296+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:10:26.576365+0000","scrub_duration":0.003405356,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'70","reported_seq":112,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:45.487099+0000","last_change":"2024-09-13T18:21:45.487099+0000","last_active":"2024-09-13T18:21:45.487099+0000","last_peered":"2024-09-13T18:21:45.487099+0000","last_clean":"2024-09-13T18:21:45.487099+0000","last_became_active":"2024-09-13T17:50:34.207605+0000","last_became_peered":"2024-09-13T17:50:34.207605+0000","last_unstale":"2024-09-13T18:21:45.487099+0000","last_undegraded":"2024-09-13T18:21:45.487099+0000","last_fullsized":"2024-09-13T18:21:45.487099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'70","last_scrub_stamp":"2024-09-13T18:21:45.486787+0000","last_deep_scrub":"27'70","last_deep_scrub_stamp":"2024-09-13T18:21:45.486787+0000","last_clean_scrub_stamp":"2024-09-13T18:21:45.486787+0000","objects_scrubbed":3,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:25:53.510155+0000","scrub_duration":0.0066096169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'5937","reported_seq":5976,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:52.308856+0000","last_change":"2024-09-13T18:21:52.308991+0000","last_active":"2024-09-13T18:21:52.308856+0000","last_peered":"2024-09-13T18:21:52.308856+0000","last_clean":"2024-09-13T18:21:52.308856+0000","last_became_active":"2024-09-13T17:50:38.614165+0000","last_became_peered":"2024-09-13T17:50:38.614165+0000","last_unstale":"2024-09-13T18:21:52.308856+0000","last_undegraded":"2024-09-13T18:21:52.308856+0000","last_fullsized":"2024-09-13T18:21:52.308856+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5937","last_scrub_stamp":"2024-09-13T18:21:52.308833+0000","last_deep_scrub":"27'5937","last_deep_scrub_stamp":"2024-09-13T18:21:52.308833+0000","last_clean_scrub_stamp":"2024-09-13T18:21:52.308833+0000","objects_scrubbed":0,"log_size":5937,"log_dups_size":0,"ondisk_log_size":5937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:21:00.114058+0000","scrub_duration":0.040795066999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25111,"num_write_kb":509795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'25","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:51.296070+0000","last_change":"2024-09-13T18:21:51.296070+0000","last_active":"2024-09-13T18:21:51.296070+0000","last_peered":"2024-09-13T18:21:51.296070+0000","last_clean":"2024-09-13T18:21:51.296070+0000","last_became_active":"2024-09-13T17:50:34.582529+0000","last_became_peered":"2024-09-13T17:50:34.582529+0000","last_unstale":"2024-09-13T18:21:51.296070+0000","last_undegraded":"2024-09-13T18:21:51.296070+0000","last_fullsized":"2024-09-13T18:21:51.296070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'25","last_scrub_stamp":"2024-09-13T18:21:51.295749+0000","last_deep_scrub":"27'25","last_deep_scrub_stamp":"2024-09-13T18:21:51.295749+0000","last_clean_scrub_stamp":"2024-09-13T18:21:51.295749+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:20:57.681873+0000","scrub_duration":0.0032653209999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'6","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:31.812971+0000","last_change":"2024-09-13T18:21:31.812971+0000","last_active":"2024-09-13T18:21:31.812971+0000","last_peered":"2024-09-13T18:21:31.812971+0000","last_clean":"2024-09-13T18:21:31.812971+0000","last_became_active":"2024-09-13T17:50:34.203360+0000","last_became_peered":"2024-09-13T17:50:34.203360+0000","last_unstale":"2024-09-13T18:21:31.812971+0000","last_undegraded":"2024-09-13T18:21:31.812971+0000","last_fullsized":"2024-09-13T18:21:31.812971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-09-13T18:21:31.812723+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-09-13T18:21:31.812723+0000","last_clean_scrub_stamp":"2024-09-13T18:21:31.812723+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:43:43.954185+0000","scrub_duration":0.0059683940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'5934","reported_seq":5973,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.246781+0000","last_change":"2024-09-13T18:21:55.246917+0000","last_active":"2024-09-13T18:21:55.246781+0000","last_peered":"2024-09-13T18:21:55.246781+0000","last_clean":"2024-09-13T18:21:55.246781+0000","last_became_active":"2024-09-13T17:50:38.610692+0000","last_became_peered":"2024-09-13T17:50:38.610692+0000","last_unstale":"2024-09-13T18:21:55.246781+0000","last_undegraded":"2024-09-13T18:21:55.246781+0000","last_fullsized":"2024-09-13T18:21:55.246781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5934","last_scrub_stamp":"2024-09-13T18:21:55.246755+0000","last_deep_scrub":"27'5934","last_deep_scrub_stamp":"2024-09-13T18:21:55.246755+0000","last_clean_scrub_stamp":"2024-09-13T18:21:55.246755+0000","objects_scrubbed":0,"log_size":5934,"log_dups_size":0,"ondisk_log_size":5934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:01:01.221236+0000","scrub_duration":0.038730878000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25100,"num_write_kb":497535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'23","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:03.699694+0000","last_change":"2024-09-13T18:22:03.699946+0000","last_active":"2024-09-13T18:22:03.699694+0000","last_peered":"2024-09-13T18:22:03.699694+0000","last_clean":"2024-09-13T18:22:03.699694+0000","last_became_active":"2024-09-13T17:51:16.364437+0000","last_became_peered":"2024-09-13T17:51:16.364437+0000","last_unstale":"2024-09-13T18:22:03.699694+0000","last_undegraded":"2024-09-13T18:22:03.699694+0000","last_fullsized":"2024-09-13T18:22:03.699694+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-13T18:22:03.699637+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-13T18:22:03.699637+0000","last_clean_scrub_stamp":"2024-09-13T18:22:03.699637+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:41.983071+0000","scrub_duration":0.0035155130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"27'11","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:33.307232+0000","last_change":"2024-09-13T18:21:33.307232+0000","last_active":"2024-09-13T18:21:33.307232+0000","last_peered":"2024-09-13T18:21:33.307232+0000","last_clean":"2024-09-13T18:21:33.307232+0000","last_became_active":"2024-09-13T17:50:34.204003+0000","last_became_peered":"2024-09-13T17:50:34.204003+0000","last_unstale":"2024-09-13T18:21:33.307232+0000","last_undegraded":"2024-09-13T18:21:33.307232+0000","last_fullsized":"2024-09-13T18:21:33.307232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-09-13T18:21:33.306942+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-09-13T18:21:33.306942+0000","last_clean_scrub_stamp":"2024-09-13T18:21:33.306942+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:13.945111+0000","scrub_duration":0.0066162260000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'5363","reported_seq":5402,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:12.293263+0000","last_change":"2024-09-13T18:22:12.293476+0000","last_active":"2024-09-13T18:22:12.293263+0000","last_peered":"2024-09-13T18:22:12.293263+0000","last_clean":"2024-09-13T18:22:12.293263+0000","last_became_active":"2024-09-13T17:50:38.605646+0000","last_became_peered":"2024-09-13T17:50:38.605646+0000","last_unstale":"2024-09-13T18:22:12.293263+0000","last_undegraded":"2024-09-13T18:22:12.293263+0000","last_fullsized":"2024-09-13T18:22:12.293263+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5363","last_scrub_stamp":"2024-09-13T18:22:12.293220+0000","last_deep_scrub":"27'5363","last_deep_scrub_stamp":"2024-09-13T18:22:12.293220+0000","last_clean_scrub_stamp":"2024-09-13T18:22:12.293220+0000","objects_scrubbed":0,"log_size":5363,"log_dups_size":0,"ondisk_log_size":5363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:05:13.174210+0000","scrub_duration":0.048844970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22831,"num_write_kb":508903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'6067","reported_seq":6097,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:02.920872+0000","last_change":"2024-09-13T17:50:38.618855+0000","last_active":"2024-09-13T18:21:02.920872+0000","last_peered":"2024-09-13T18:21:02.920872+0000","last_clean":"2024-09-13T18:21:02.920872+0000","last_became_active":"2024-09-13T17:50:38.618449+0000","last_became_peered":"2024-09-13T17:50:38.618449+0000","last_unstale":"2024-09-13T18:21:02.920872+0000","last_undegraded":"2024-09-13T18:21:02.920872+0000","last_fullsized":"2024-09-13T18:21:02.920872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:37.577645+0000","last_clean_scrub_stamp":"2024-09-13T17:50:37.577645+0000","objects_scrubbed":0,"log_size":6067,"log_dups_size":0,"ondisk_log_size":6067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25532,"num_write_kb":568825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'21","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:54.500706+0000","last_change":"2024-09-13T18:21:54.500706+0000","last_active":"2024-09-13T18:21:54.500706+0000","last_peered":"2024-09-13T18:21:54.500706+0000","last_clean":"2024-09-13T18:21:54.500706+0000","last_became_active":"2024-09-13T17:50:34.589396+0000","last_became_peered":"2024-09-13T17:50:34.589396+0000","last_unstale":"2024-09-13T18:21:54.500706+0000","last_undegraded":"2024-09-13T18:21:54.500706+0000","last_fullsized":"2024-09-13T18:21:54.500706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-09-13T18:21:54.500280+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-09-13T18:21:54.500280+0000","last_clean_scrub_stamp":"2024-09-13T18:21:54.500280+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:24.989093+0000","scrub_duration":0.0028702549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:32.850895+0000","last_change":"2024-09-13T18:21:32.850895+0000","last_active":"2024-09-13T18:21:32.850895+0000","last_peered":"2024-09-13T18:21:32.850895+0000","last_clean":"2024-09-13T18:21:32.850895+0000","last_became_active":"2024-09-13T17:50:34.206308+0000","last_became_peered":"2024-09-13T17:50:34.206308+0000","last_unstale":"2024-09-13T18:21:32.850895+0000","last_undegraded":"2024-09-13T18:21:32.850895+0000","last_fullsized":"2024-09-13T18:21:32.850895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-09-13T18:21:32.850533+0000","last_deep_scrub":"25'1","last_deep_scrub_stamp":"2024-09-13T18:21:32.850533+0000","last_clean_scrub_stamp":"2024-09-13T18:21:32.850533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:59:21.637907+0000","scrub_duration":0.0034298850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'5881","reported_seq":5922,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:53.439711+0000","last_change":"2024-09-13T18:21:53.439711+0000","last_active":"2024-09-13T18:21:53.439711+0000","last_peered":"2024-09-13T18:21:53.439711+0000","last_clean":"2024-09-13T18:21:53.439711+0000","last_became_active":"2024-09-13T17:50:38.617611+0000","last_became_peered":"2024-09-13T17:50:38.617611+0000","last_unstale":"2024-09-13T18:21:53.439711+0000","last_undegraded":"2024-09-13T18:21:53.439711+0000","last_fullsized":"2024-09-13T18:21:53.439711+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5881","last_scrub_stamp":"2024-09-13T18:21:53.439266+0000","last_deep_scrub":"27'5881","last_deep_scrub_stamp":"2024-09-13T18:21:53.439266+0000","last_clean_scrub_stamp":"2024-09-13T18:21:53.439266+0000","objects_scrubbed":0,"log_size":5881,"log_dups_size":0,"ondisk_log_size":5881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:46:36.260829+0000","scrub_duration":0.043273643000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25019,"num_write_kb":530139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'27","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:34.190222+0000","last_change":"2024-09-13T18:21:34.190222+0000","last_active":"2024-09-13T18:21:34.190222+0000","last_peered":"2024-09-13T18:21:34.190222+0000","last_clean":"2024-09-13T18:21:34.190222+0000","last_became_active":"2024-09-13T17:50:34.581708+0000","last_became_peered":"2024-09-13T17:50:34.581708+0000","last_unstale":"2024-09-13T18:21:34.190222+0000","last_undegraded":"2024-09-13T18:21:34.190222+0000","last_fullsized":"2024-09-13T18:21:34.190222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'27","last_scrub_stamp":"2024-09-13T18:21:34.189993+0000","last_deep_scrub":"27'27","last_deep_scrub_stamp":"2024-09-13T18:21:34.189993+0000","last_clean_scrub_stamp":"2024-09-13T18:21:34.189993+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:32:35.944129+0000","scrub_duration":0.0029610209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'5","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:42.211743+0000","last_change":"2024-09-13T18:21:42.211743+0000","last_active":"2024-09-13T18:21:42.211743+0000","last_peered":"2024-09-13T18:21:42.211743+0000","last_clean":"2024-09-13T18:21:42.211743+0000","last_became_active":"2024-09-13T17:50:34.205202+0000","last_became_peered":"2024-09-13T17:50:34.205202+0000","last_unstale":"2024-09-13T18:21:42.211743+0000","last_undegraded":"2024-09-13T18:21:42.211743+0000","last_fullsized":"2024-09-13T18:21:42.211743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-09-13T18:21:42.211500+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-09-13T18:21:42.211500+0000","last_clean_scrub_stamp":"2024-09-13T18:21:42.211500+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:47:56.465375+0000","scrub_duration":0.0062275539999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'7","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:30.324177+0000","last_change":"2024-09-13T18:21:30.324321+0000","last_active":"2024-09-13T18:21:30.324177+0000","last_peered":"2024-09-13T18:21:30.324177+0000","last_clean":"2024-09-13T18:21:30.324177+0000","last_became_active":"2024-09-13T17:50:34.202163+0000","last_became_peered":"2024-09-13T17:50:34.202163+0000","last_unstale":"2024-09-13T18:21:30.324177+0000","last_undegraded":"2024-09-13T18:21:30.324177+0000","last_fullsized":"2024-09-13T18:21:30.324177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-13T18:21:30.324141+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-13T18:21:30.324141+0000","last_clean_scrub_stamp":"2024-09-13T18:21:30.324141+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:27.051230+0000","scrub_duration":0.00672063,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'18","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:45.443346+0000","last_change":"2024-09-13T18:21:45.443452+0000","last_active":"2024-09-13T18:21:45.443346+0000","last_peered":"2024-09-13T18:21:45.443346+0000","last_clean":"2024-09-13T18:21:45.443346+0000","last_became_active":"2024-09-13T17:50:34.590497+0000","last_became_peered":"2024-09-13T17:50:34.590497+0000","last_unstale":"2024-09-13T18:21:45.443346+0000","last_undegraded":"2024-09-13T18:21:45.443346+0000","last_fullsized":"2024-09-13T18:21:45.443346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:21:45.443314+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:21:45.443314+0000","last_clean_scrub_stamp":"2024-09-13T18:21:45.443314+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:29.220122+0000","scrub_duration":0.0029326640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'4651","reported_seq":4689,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:52.595124+0000","last_change":"2024-09-13T18:21:52.595243+0000","last_active":"2024-09-13T18:21:52.595124+0000","last_peered":"2024-09-13T18:21:52.595124+0000","last_clean":"2024-09-13T18:21:52.595124+0000","last_became_active":"2024-09-13T17:50:38.618108+0000","last_became_peered":"2024-09-13T17:50:38.618108+0000","last_unstale":"2024-09-13T18:21:52.595124+0000","last_undegraded":"2024-09-13T18:21:52.595124+0000","last_fullsized":"2024-09-13T18:21:52.595124+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4651","last_scrub_stamp":"2024-09-13T18:21:52.595101+0000","last_deep_scrub":"27'4651","last_deep_scrub_stamp":"2024-09-13T18:21:52.595101+0000","last_clean_scrub_stamp":"2024-09-13T18:21:52.595101+0000","objects_scrubbed":0,"log_size":4651,"log_dups_size":0,"ondisk_log_size":4651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:15:47.184237+0000","scrub_duration":0.047927588,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19488,"num_write_kb":413684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'692","reported_seq":732,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:38.185928+0000","last_change":"2024-09-13T18:21:38.186111+0000","last_active":"2024-09-13T18:21:38.185928+0000","last_peered":"2024-09-13T18:21:38.185928+0000","last_clean":"2024-09-13T18:21:38.185928+0000","last_became_active":"2024-09-13T17:50:34.218087+0000","last_became_peered":"2024-09-13T17:50:34.218087+0000","last_unstale":"2024-09-13T18:21:38.185928+0000","last_undegraded":"2024-09-13T18:21:38.185928+0000","last_fullsized":"2024-09-13T18:21:38.185928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'692","last_scrub_stamp":"2024-09-13T18:21:38.185889+0000","last_deep_scrub":"27'692","last_deep_scrub_stamp":"2024-09-13T18:21:38.185889+0000","last_clean_scrub_stamp":"2024-09-13T18:21:38.185889+0000","objects_scrubbed":3,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:16:47.594855+0000","scrub_duration":0.021444375000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195461,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'20","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:40.309829+0000","last_change":"2024-09-13T18:21:40.310070+0000","last_active":"2024-09-13T18:21:40.309829+0000","last_peered":"2024-09-13T18:21:40.309829+0000","last_clean":"2024-09-13T18:21:40.309829+0000","last_became_active":"2024-09-13T17:50:34.583283+0000","last_became_peered":"2024-09-13T17:50:34.583283+0000","last_unstale":"2024-09-13T18:21:40.309829+0000","last_undegraded":"2024-09-13T18:21:40.309829+0000","last_fullsized":"2024-09-13T18:21:40.309829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-13T18:21:40.309774+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-13T18:21:40.309774+0000","last_clean_scrub_stamp":"2024-09-13T18:21:40.309774+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:45:15.638497+0000","scrub_duration":0.0033293300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'5883","reported_seq":5921,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:09.261029+0000","last_change":"2024-09-13T18:22:09.261212+0000","last_active":"2024-09-13T18:22:09.261029+0000","last_peered":"2024-09-13T18:22:09.261029+0000","last_clean":"2024-09-13T18:22:09.261029+0000","last_became_active":"2024-09-13T17:50:38.611636+0000","last_became_peered":"2024-09-13T17:50:38.611636+0000","last_unstale":"2024-09-13T18:22:09.261029+0000","last_undegraded":"2024-09-13T18:22:09.261029+0000","last_fullsized":"2024-09-13T18:22:09.261029+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5883","last_scrub_stamp":"2024-09-13T18:22:09.260981+0000","last_deep_scrub":"27'5883","last_deep_scrub_stamp":"2024-09-13T18:22:09.260981+0000","last_clean_scrub_stamp":"2024-09-13T18:22:09.260981+0000","objects_scrubbed":0,"log_size":5883,"log_dups_size":0,"ondisk_log_size":5883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:12:31.012514+0000","scrub_duration":0.049256573999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24816,"num_write_kb":545450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'25","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:46.374486+0000","last_change":"2024-09-13T18:21:46.374486+0000","last_active":"2024-09-13T18:21:46.374486+0000","last_peered":"2024-09-13T18:21:46.374486+0000","last_clean":"2024-09-13T18:21:46.374486+0000","last_became_active":"2024-09-13T17:50:34.588651+0000","last_became_peered":"2024-09-13T17:50:34.588651+0000","last_unstale":"2024-09-13T18:21:46.374486+0000","last_undegraded":"2024-09-13T18:21:46.374486+0000","last_fullsized":"2024-09-13T18:21:46.374486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'25","last_scrub_stamp":"2024-09-13T18:21:46.374221+0000","last_deep_scrub":"27'25","last_deep_scrub_stamp":"2024-09-13T18:21:46.374221+0000","last_clean_scrub_stamp":"2024-09-13T18:21:46.374221+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:58.490650+0000","scrub_duration":0.003300449,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'24","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:45.720155+0000","last_change":"2024-09-13T18:21:45.720155+0000","last_active":"2024-09-13T18:21:45.720155+0000","last_peered":"2024-09-13T18:21:45.720155+0000","last_clean":"2024-09-13T18:21:45.720155+0000","last_became_active":"2024-09-13T17:50:34.583493+0000","last_became_peered":"2024-09-13T17:50:34.583493+0000","last_unstale":"2024-09-13T18:21:45.720155+0000","last_undegraded":"2024-09-13T18:21:45.720155+0000","last_fullsized":"2024-09-13T18:21:45.720155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-09-13T18:21:45.719919+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-09-13T18:21:45.719919+0000","last_clean_scrub_stamp":"2024-09-13T18:21:45.719919+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:15.218921+0000","scrub_duration":0.0030666199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'25","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:50.830134+0000","last_change":"2024-09-13T18:21:50.830134+0000","last_active":"2024-09-13T18:21:50.830134+0000","last_peered":"2024-09-13T18:21:50.830134+0000","last_clean":"2024-09-13T18:21:50.830134+0000","last_became_active":"2024-09-13T17:50:34.587119+0000","last_became_peered":"2024-09-13T17:50:34.587119+0000","last_unstale":"2024-09-13T18:21:50.830134+0000","last_undegraded":"2024-09-13T18:21:50.830134+0000","last_fullsized":"2024-09-13T18:21:50.830134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'25","last_scrub_stamp":"2024-09-13T18:21:50.829746+0000","last_deep_scrub":"27'25","last_deep_scrub_stamp":"2024-09-13T18:21:50.829746+0000","last_clean_scrub_stamp":"2024-09-13T18:21:50.829746+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:38:15.448330+0000","scrub_duration":0.0031738930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'17","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:57.236877+0000","last_change":"2024-09-13T18:21:57.236877+0000","last_active":"2024-09-13T18:21:57.236877+0000","last_peered":"2024-09-13T18:21:57.236877+0000","last_clean":"2024-09-13T18:21:57.236877+0000","last_became_active":"2024-09-13T17:50:34.581628+0000","last_became_peered":"2024-09-13T17:50:34.581628+0000","last_unstale":"2024-09-13T18:21:57.236877+0000","last_undegraded":"2024-09-13T18:21:57.236877+0000","last_fullsized":"2024-09-13T18:21:57.236877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'17","last_scrub_stamp":"2024-09-13T18:21:57.236610+0000","last_deep_scrub":"27'17","last_deep_scrub_stamp":"2024-09-13T18:21:57.236610+0000","last_clean_scrub_stamp":"2024-09-13T18:21:57.236610+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:24.087638+0000","scrub_duration":0.0024405049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'14","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:53.191673+0000","last_change":"2024-09-13T18:21:53.191956+0000","last_active":"2024-09-13T18:21:53.191673+0000","last_peered":"2024-09-13T18:21:53.191673+0000","last_clean":"2024-09-13T18:21:53.191673+0000","last_became_active":"2024-09-13T17:50:34.594857+0000","last_became_peered":"2024-09-13T17:50:34.594857+0000","last_unstale":"2024-09-13T18:21:53.191673+0000","last_undegraded":"2024-09-13T18:21:53.191673+0000","last_fullsized":"2024-09-13T18:21:53.191673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-09-13T18:21:53.191615+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-09-13T18:21:53.191615+0000","last_clean_scrub_stamp":"2024-09-13T18:21:53.191615+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:45.001540+0000","scrub_duration":0.0042100100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'15","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:53.457198+0000","last_change":"2024-09-13T18:21:53.457331+0000","last_active":"2024-09-13T18:21:53.457198+0000","last_peered":"2024-09-13T18:21:53.457198+0000","last_clean":"2024-09-13T18:21:53.457198+0000","last_became_active":"2024-09-13T17:50:34.589354+0000","last_became_peered":"2024-09-13T17:50:34.589354+0000","last_unstale":"2024-09-13T18:21:53.457198+0000","last_undegraded":"2024-09-13T18:21:53.457198+0000","last_fullsized":"2024-09-13T18:21:53.457198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-09-13T18:21:53.457163+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-09-13T18:21:53.457163+0000","last_clean_scrub_stamp":"2024-09-13T18:21:53.457163+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:15:40.638651+0000","scrub_duration":0.0022509019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'24","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:38.403817+0000","last_change":"2024-09-13T18:21:38.404032+0000","last_active":"2024-09-13T18:21:38.403817+0000","last_peered":"2024-09-13T18:21:38.403817+0000","last_clean":"2024-09-13T18:21:38.403817+0000","last_became_active":"2024-09-13T17:50:34.597621+0000","last_became_peered":"2024-09-13T17:50:34.597621+0000","last_unstale":"2024-09-13T18:21:38.403817+0000","last_undegraded":"2024-09-13T18:21:38.403817+0000","last_fullsized":"2024-09-13T18:21:38.403817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-09-13T18:21:38.403767+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-09-13T18:21:38.403767+0000","last_clean_scrub_stamp":"2024-09-13T18:21:38.403767+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:33:01.593056+0000","scrub_duration":0.0041273439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'15","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:32.155724+0000","last_change":"2024-09-13T18:21:32.155926+0000","last_active":"2024-09-13T18:21:32.155724+0000","last_peered":"2024-09-13T18:21:32.155724+0000","last_clean":"2024-09-13T18:21:32.155724+0000","last_became_active":"2024-09-13T17:50:34.589626+0000","last_became_peered":"2024-09-13T17:50:34.589626+0000","last_unstale":"2024-09-13T18:21:32.155724+0000","last_undegraded":"2024-09-13T18:21:32.155724+0000","last_fullsized":"2024-09-13T18:21:32.155724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-09-13T18:21:32.155676+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-09-13T18:21:32.155676+0000","last_clean_scrub_stamp":"2024-09-13T18:21:32.155676+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:03:35.333502+0000","scrub_duration":0.0028998560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'16","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:44.880892+0000","last_change":"2024-09-13T18:21:44.881098+0000","last_active":"2024-09-13T18:21:44.880892+0000","last_peered":"2024-09-13T18:21:44.880892+0000","last_clean":"2024-09-13T18:21:44.880892+0000","last_became_active":"2024-09-13T17:50:34.589757+0000","last_became_peered":"2024-09-13T17:50:34.589757+0000","last_unstale":"2024-09-13T18:21:44.880892+0000","last_undegraded":"2024-09-13T18:21:44.880892+0000","last_fullsized":"2024-09-13T18:21:44.880892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-09-13T18:21:44.880841+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-09-13T18:21:44.880841+0000","last_clean_scrub_stamp":"2024-09-13T18:21:44.880841+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:12:15.859928+0000","scrub_duration":0.0034118379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'23","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.130283+0000","last_change":"2024-09-13T18:21:27.130283+0000","last_active":"2024-09-13T18:21:27.130283+0000","last_peered":"2024-09-13T18:21:27.130283+0000","last_clean":"2024-09-13T18:21:27.130283+0000","last_became_active":"2024-09-13T17:50:34.595242+0000","last_became_peered":"2024-09-13T17:50:34.595242+0000","last_unstale":"2024-09-13T18:21:27.130283+0000","last_undegraded":"2024-09-13T18:21:27.130283+0000","last_fullsized":"2024-09-13T18:21:27.130283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-13T18:21:27.130025+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-13T18:21:27.130025+0000","last_clean_scrub_stamp":"2024-09-13T18:21:27.130025+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:06.896321+0000","scrub_duration":0.003821638,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'21","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:54.798385+0000","last_change":"2024-09-13T18:21:54.798385+0000","last_active":"2024-09-13T18:21:54.798385+0000","last_peered":"2024-09-13T18:21:54.798385+0000","last_clean":"2024-09-13T18:21:54.798385+0000","last_became_active":"2024-09-13T17:50:34.592893+0000","last_became_peered":"2024-09-13T17:50:34.592893+0000","last_unstale":"2024-09-13T18:21:54.798385+0000","last_undegraded":"2024-09-13T18:21:54.798385+0000","last_fullsized":"2024-09-13T18:21:54.798385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-09-13T18:21:54.798131+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-09-13T18:21:54.798131+0000","last_clean_scrub_stamp":"2024-09-13T18:21:54.798131+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:27:46.038569+0000","scrub_duration":0.0031872860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'29","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:02.275861+0000","last_change":"2024-09-13T18:22:02.275861+0000","last_active":"2024-09-13T18:22:02.275861+0000","last_peered":"2024-09-13T18:22:02.275861+0000","last_clean":"2024-09-13T18:22:02.275861+0000","last_became_active":"2024-09-13T17:50:34.593750+0000","last_became_peered":"2024-09-13T17:50:34.593750+0000","last_unstale":"2024-09-13T18:22:02.275861+0000","last_undegraded":"2024-09-13T18:22:02.275861+0000","last_fullsized":"2024-09-13T18:22:02.275861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'29","last_scrub_stamp":"2024-09-13T18:22:02.275708+0000","last_deep_scrub":"27'29","last_deep_scrub_stamp":"2024-09-13T18:22:02.275708+0000","last_clean_scrub_stamp":"2024-09-13T18:22:02.275708+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:04.409943+0000","scrub_duration":0.0028268960000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'19","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:37.375333+0000","last_change":"2024-09-13T18:21:37.375333+0000","last_active":"2024-09-13T18:21:37.375333+0000","last_peered":"2024-09-13T18:21:37.375333+0000","last_clean":"2024-09-13T18:21:37.375333+0000","last_became_active":"2024-09-13T17:50:34.598433+0000","last_became_peered":"2024-09-13T17:50:34.598433+0000","last_unstale":"2024-09-13T18:21:37.375333+0000","last_undegraded":"2024-09-13T18:21:37.375333+0000","last_fullsized":"2024-09-13T18:21:37.375333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-13T18:21:37.375056+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-13T18:21:37.375056+0000","last_clean_scrub_stamp":"2024-09-13T18:21:37.375056+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:02.999621+0000","scrub_duration":0.003734036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'22","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:36.325615+0000","last_change":"2024-09-13T18:21:36.325615+0000","last_active":"2024-09-13T18:21:36.325615+0000","last_peered":"2024-09-13T18:21:36.325615+0000","last_clean":"2024-09-13T18:21:36.325615+0000","last_became_active":"2024-09-13T17:50:34.591286+0000","last_became_peered":"2024-09-13T17:50:34.591286+0000","last_unstale":"2024-09-13T18:21:36.325615+0000","last_undegraded":"2024-09-13T18:21:36.325615+0000","last_fullsized":"2024-09-13T18:21:36.325615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-09-13T18:21:36.325355+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-09-13T18:21:36.325355+0000","last_clean_scrub_stamp":"2024-09-13T18:21:36.325355+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:12:08.676171+0000","scrub_duration":0.0029670930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'28","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:44.767452+0000","last_change":"2024-09-13T18:21:44.767671+0000","last_active":"2024-09-13T18:21:44.767452+0000","last_peered":"2024-09-13T18:21:44.767452+0000","last_clean":"2024-09-13T18:21:44.767452+0000","last_became_active":"2024-09-13T17:50:34.587506+0000","last_became_peered":"2024-09-13T17:50:34.587506+0000","last_unstale":"2024-09-13T18:21:44.767452+0000","last_undegraded":"2024-09-13T18:21:44.767452+0000","last_fullsized":"2024-09-13T18:21:44.767452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'28","last_scrub_stamp":"2024-09-13T18:21:44.767402+0000","last_deep_scrub":"27'28","last_deep_scrub_stamp":"2024-09-13T18:21:44.767402+0000","last_clean_scrub_stamp":"2024-09-13T18:21:44.767402+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:28:00.460114+0000","scrub_duration":0.003174128,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'22","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:47.811825+0000","last_change":"2024-09-13T18:21:47.812043+0000","last_active":"2024-09-13T18:21:47.811825+0000","last_peered":"2024-09-13T18:21:47.811825+0000","last_clean":"2024-09-13T18:21:47.811825+0000","last_became_active":"2024-09-13T17:50:34.595798+0000","last_became_peered":"2024-09-13T17:50:34.595798+0000","last_unstale":"2024-09-13T18:21:47.811825+0000","last_undegraded":"2024-09-13T18:21:47.811825+0000","last_fullsized":"2024-09-13T18:21:47.811825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-09-13T18:21:47.811769+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-09-13T18:21:47.811769+0000","last_clean_scrub_stamp":"2024-09-13T18:21:47.811769+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:56.620401+0000","scrub_duration":0.0030899819999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'21","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:25.753731+0000","last_change":"2024-09-13T18:21:25.753960+0000","last_active":"2024-09-13T18:21:25.753731+0000","last_peered":"2024-09-13T18:21:25.753731+0000","last_clean":"2024-09-13T18:21:25.753731+0000","last_became_active":"2024-09-13T17:50:34.584516+0000","last_became_peered":"2024-09-13T17:50:34.584516+0000","last_unstale":"2024-09-13T18:21:25.753731+0000","last_undegraded":"2024-09-13T18:21:25.753731+0000","last_fullsized":"2024-09-13T18:21:25.753731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-09-13T18:21:25.753680+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-09-13T18:21:25.753680+0000","last_clean_scrub_stamp":"2024-09-13T18:21:25.753680+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:59.108188+0000","scrub_duration":0.0032900099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'23","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:51.445915+0000","last_change":"2024-09-13T18:21:51.446115+0000","last_active":"2024-09-13T18:21:51.445915+0000","last_peered":"2024-09-13T18:21:51.445915+0000","last_clean":"2024-09-13T18:21:51.445915+0000","last_became_active":"2024-09-13T17:50:34.598648+0000","last_became_peered":"2024-09-13T17:50:34.598648+0000","last_unstale":"2024-09-13T18:21:51.445915+0000","last_undegraded":"2024-09-13T18:21:51.445915+0000","last_fullsized":"2024-09-13T18:21:51.445915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-13T18:21:51.445846+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-13T18:21:51.445846+0000","last_clean_scrub_stamp":"2024-09-13T18:21:51.445846+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:09.890274+0000","scrub_duration":0.003395239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'24","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:23.200378+0000","last_change":"2024-09-13T18:21:23.200494+0000","last_active":"2024-09-13T18:21:23.200378+0000","last_peered":"2024-09-13T18:21:23.200378+0000","last_clean":"2024-09-13T18:21:23.200378+0000","last_became_active":"2024-09-13T17:50:34.581699+0000","last_became_peered":"2024-09-13T17:50:34.581699+0000","last_unstale":"2024-09-13T18:21:23.200378+0000","last_undegraded":"2024-09-13T18:21:23.200378+0000","last_fullsized":"2024-09-13T18:21:23.200378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-09-13T18:21:23.200343+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-09-13T18:21:23.200343+0000","last_clean_scrub_stamp":"2024-09-13T18:21:23.200343+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:37:50.717684+0000","scrub_duration":0.002562838,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'23","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:25.110698+0000","last_change":"2024-09-13T18:21:25.110698+0000","last_active":"2024-09-13T18:21:25.110698+0000","last_peered":"2024-09-13T18:21:25.110698+0000","last_clean":"2024-09-13T18:21:25.110698+0000","last_became_active":"2024-09-13T17:50:34.587217+0000","last_became_peered":"2024-09-13T17:50:34.587217+0000","last_unstale":"2024-09-13T18:21:25.110698+0000","last_undegraded":"2024-09-13T18:21:25.110698+0000","last_fullsized":"2024-09-13T18:21:25.110698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-13T18:21:25.110401+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-13T18:21:25.110401+0000","last_clean_scrub_stamp":"2024-09-13T18:21:25.110401+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:08.271399+0000","scrub_duration":0.0034179560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'18","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:47.310243+0000","last_change":"2024-09-13T18:21:47.310243+0000","last_active":"2024-09-13T18:21:47.310243+0000","last_peered":"2024-09-13T18:21:47.310243+0000","last_clean":"2024-09-13T18:21:47.310243+0000","last_became_active":"2024-09-13T17:50:34.588513+0000","last_became_peered":"2024-09-13T17:50:34.588513+0000","last_unstale":"2024-09-13T18:21:47.310243+0000","last_undegraded":"2024-09-13T18:21:47.310243+0000","last_fullsized":"2024-09-13T18:21:47.310243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:21:47.309947+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:21:47.309947+0000","last_clean_scrub_stamp":"2024-09-13T18:21:47.309947+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:52:22.085054+0000","scrub_duration":0.003219857,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'19","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:40.418512+0000","last_change":"2024-09-13T18:21:40.418512+0000","last_active":"2024-09-13T18:21:40.418512+0000","last_peered":"2024-09-13T18:21:40.418512+0000","last_clean":"2024-09-13T18:21:40.418512+0000","last_became_active":"2024-09-13T17:50:34.585438+0000","last_became_peered":"2024-09-13T17:50:34.585438+0000","last_unstale":"2024-09-13T18:21:40.418512+0000","last_undegraded":"2024-09-13T18:21:40.418512+0000","last_fullsized":"2024-09-13T18:21:40.418512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-13T18:21:40.418258+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-13T18:21:40.418258+0000","last_clean_scrub_stamp":"2024-09-13T18:21:40.418258+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:52.939370+0000","scrub_duration":0.0033203519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'30","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:48.381967+0000","last_change":"2024-09-13T18:21:48.381967+0000","last_active":"2024-09-13T18:21:48.381967+0000","last_peered":"2024-09-13T18:21:48.381967+0000","last_clean":"2024-09-13T18:21:48.381967+0000","last_became_active":"2024-09-13T17:50:34.587665+0000","last_became_peered":"2024-09-13T17:50:34.587665+0000","last_unstale":"2024-09-13T18:21:48.381967+0000","last_undegraded":"2024-09-13T18:21:48.381967+0000","last_fullsized":"2024-09-13T18:21:48.381967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'30","last_scrub_stamp":"2024-09-13T18:21:48.381520+0000","last_deep_scrub":"27'30","last_deep_scrub_stamp":"2024-09-13T18:21:48.381520+0000","last_clean_scrub_stamp":"2024-09-13T18:21:48.381520+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:15:59.624808+0000","scrub_duration":0.003678331,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'27","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:25.955095+0000","last_change":"2024-09-13T17:50:34.591776+0000","last_active":"2024-09-13T18:21:25.955095+0000","last_peered":"2024-09-13T18:21:25.955095+0000","last_clean":"2024-09-13T18:21:25.955095+0000","last_became_active":"2024-09-13T17:50:34.591496+0000","last_became_peered":"2024-09-13T17:50:34.591496+0000","last_unstale":"2024-09-13T18:21:25.955095+0000","last_undegraded":"2024-09-13T18:21:25.955095+0000","last_fullsized":"2024-09-13T18:21:25.955095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'15","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:59.687264+0000","last_change":"2024-09-13T18:21:59.687484+0000","last_active":"2024-09-13T18:21:59.687264+0000","last_peered":"2024-09-13T18:21:59.687264+0000","last_clean":"2024-09-13T18:21:59.687264+0000","last_became_active":"2024-09-13T17:51:16.282065+0000","last_became_peered":"2024-09-13T17:51:16.282065+0000","last_unstale":"2024-09-13T18:21:59.687264+0000","last_undegraded":"2024-09-13T18:21:59.687264+0000","last_fullsized":"2024-09-13T18:21:59.687264+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-09-13T18:21:59.687218+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-09-13T18:21:59.687218+0000","last_clean_scrub_stamp":"2024-09-13T18:21:59.687218+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:33:21.535758+0000","scrub_duration":0.003695977,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"27'14","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.371845+0000","last_change":"2024-09-13T18:22:18.372074+0000","last_active":"2024-09-13T18:22:18.371845+0000","last_peered":"2024-09-13T18:22:18.371845+0000","last_clean":"2024-09-13T18:22:18.371845+0000","last_became_active":"2024-09-13T17:50:34.585226+0000","last_became_peered":"2024-09-13T17:50:34.585226+0000","last_unstale":"2024-09-13T18:22:18.371845+0000","last_undegraded":"2024-09-13T18:22:18.371845+0000","last_fullsized":"2024-09-13T18:22:18.371845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-09-13T18:22:18.371790+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-09-13T18:22:18.371790+0000","last_clean_scrub_stamp":"2024-09-13T18:22:18.371790+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:59.331860+0000","scrub_duration":0.003510441,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'17","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:59.828261+0000","last_change":"2024-09-13T18:21:59.828532+0000","last_active":"2024-09-13T18:21:59.828261+0000","last_peered":"2024-09-13T18:21:59.828261+0000","last_clean":"2024-09-13T18:21:59.828261+0000","last_became_active":"2024-09-13T17:50:34.584564+0000","last_became_peered":"2024-09-13T17:50:34.584564+0000","last_unstale":"2024-09-13T18:21:59.828261+0000","last_undegraded":"2024-09-13T18:21:59.828261+0000","last_fullsized":"2024-09-13T18:21:59.828261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'17","last_scrub_stamp":"2024-09-13T18:21:59.828209+0000","last_deep_scrub":"27'17","last_deep_scrub_stamp":"2024-09-13T18:21:59.828209+0000","last_clean_scrub_stamp":"2024-09-13T18:21:59.828209+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:49:08.538007+0000","scrub_duration":0.0035508250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'18","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:13.313624+0000","last_change":"2024-09-13T17:50:34.585938+0000","last_active":"2024-09-13T18:21:13.313624+0000","last_peered":"2024-09-13T18:21:13.313624+0000","last_clean":"2024-09-13T18:21:13.313624+0000","last_became_active":"2024-09-13T17:50:34.585584+0000","last_became_peered":"2024-09-13T17:50:34.585584+0000","last_unstale":"2024-09-13T18:21:13.313624+0000","last_undegraded":"2024-09-13T18:21:13.313624+0000","last_fullsized":"2024-09-13T18:21:13.313624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'15","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:03.809363+0000","last_change":"2024-09-13T18:22:03.809580+0000","last_active":"2024-09-13T18:22:03.809363+0000","last_peered":"2024-09-13T18:22:03.809363+0000","last_clean":"2024-09-13T18:22:03.809363+0000","last_became_active":"2024-09-13T17:50:34.588489+0000","last_became_peered":"2024-09-13T17:50:34.588489+0000","last_unstale":"2024-09-13T18:22:03.809363+0000","last_undegraded":"2024-09-13T18:22:03.809363+0000","last_fullsized":"2024-09-13T18:22:03.809363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-09-13T18:22:03.809309+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-09-13T18:22:03.809309+0000","last_clean_scrub_stamp":"2024-09-13T18:22:03.809309+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:57:27.506887+0000","scrub_duration":0.0032327089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'18","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:05.304224+0000","last_change":"2024-09-13T18:22:05.304413+0000","last_active":"2024-09-13T18:22:05.304224+0000","last_peered":"2024-09-13T18:22:05.304224+0000","last_clean":"2024-09-13T18:22:05.304224+0000","last_became_active":"2024-09-13T17:50:34.583551+0000","last_became_peered":"2024-09-13T17:50:34.583551+0000","last_unstale":"2024-09-13T18:22:05.304224+0000","last_undegraded":"2024-09-13T18:22:05.304224+0000","last_fullsized":"2024-09-13T18:22:05.304224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:22:05.304174+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:22:05.304174+0000","last_clean_scrub_stamp":"2024-09-13T18:22:05.304174+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:14:04.897154+0000","scrub_duration":0.0024008039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'10","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:29.338126+0000","last_change":"2024-09-13T18:21:29.338126+0000","last_active":"2024-09-13T18:21:29.338126+0000","last_peered":"2024-09-13T18:21:29.338126+0000","last_clean":"2024-09-13T18:21:29.338126+0000","last_became_active":"2024-09-13T17:50:34.585340+0000","last_became_peered":"2024-09-13T17:50:34.585340+0000","last_unstale":"2024-09-13T18:21:29.338126+0000","last_undegraded":"2024-09-13T18:21:29.338126+0000","last_fullsized":"2024-09-13T18:21:29.338126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-13T18:21:29.337797+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-13T18:21:29.337797+0000","last_clean_scrub_stamp":"2024-09-13T18:21:29.337797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:52.759391+0000","scrub_duration":0.003218725,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'17","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:25.955172+0000","last_change":"2024-09-13T17:50:34.592365+0000","last_active":"2024-09-13T18:21:25.955172+0000","last_peered":"2024-09-13T18:21:25.955172+0000","last_clean":"2024-09-13T18:21:25.955172+0000","last_became_active":"2024-09-13T17:50:34.592046+0000","last_became_peered":"2024-09-13T17:50:34.592046+0000","last_unstale":"2024-09-13T18:21:25.955172+0000","last_undegraded":"2024-09-13T18:21:25.955172+0000","last_fullsized":"2024-09-13T18:21:25.955172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:50:34.174247+0000","last_clean_scrub_stamp":"2024-09-13T17:50:34.174247+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":751482,"num_write_kb":16106434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178546,"ondisk_log_size":178546,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1271,"ondisk_log_size":1271,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":15941984,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":121,"num_read_kb":179,"num_write":3371,"num_write_kb":16874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15000,"num_omap_keys":30,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32030720,"data_stored":31883968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3184,"ondisk_log_size":3184,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509825,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1772172,"kb_used_data":8580,"kb_used_omap":1,"kb_used_meta":1763582,"kb_avail":92599668,"statfs":{"total":96636764160,"available":94822060032,"internally_reserved":0,"allocated":8785920,"data_stored":8600393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":1805908424},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":2.6030000000000002,"5min":4.7919999999999998,"15min":2.8069999999999999},"min":{"1min":0.28799999999999998,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":24.829000000000001,"5min":41.494999999999997,"15min":50.511000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":1.708,"5min":3.0110000000000001,"15min":1.677},"min":{"1min":0.27300000000000002,"5min":0.193,"15min":0.193},"max":{"1min":9.9600000000000009,"5min":18.760000000000002,"15min":18.760000000000002},"last":0.69299999999999995}]},{"osd":1,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":2.8199999999999998,"15min":1.841},"min":{"1min":0.52700000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":10.379,"5min":19.626000000000001,"15min":19.626000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":3.0880000000000001,"15min":2.081},"min":{"1min":0.312,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":3.891,"5min":22.126999999999999,"15min":22.126999999999999},"last":0.65000000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":1.3400000000000001,"15min":1.2869999999999999},"min":{"1min":0.246,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.048,"5min":13.039999999999999,"15min":22.010000000000002},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":1.4830000000000001,"15min":1.472},"min":{"1min":0.32300000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.202,"5min":13.061999999999999,"15min":24.170999999999999},"last":0.94499999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.3049999999999999,"15min":1.4570000000000001},"min":{"1min":0.377,"5min":0.307,"15min":0.307},"max":{"1min":1.6839999999999999,"5min":12.936,"15min":24.359000000000002},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.9670000000000001,"15min":1.4810000000000001},"min":{"1min":0.35099999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":3.7530000000000001,"5min":15.539999999999999,"15min":15.539999999999999},"last":0.74199999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.5700000000000001,"15min":1.6279999999999999},"min":{"1min":0.35299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":5.5229999999999997,"5min":12.984,"15min":49.838000000000001},"last":1.171},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.5449999999999999,"15min":1.5980000000000001},"min":{"1min":0.42499999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.4730000000000001,"5min":16.334,"15min":41.591999999999999},"last":1.282}]},{"osd":6,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":1.7010000000000001,"5min":1.875,"15min":1.764},"min":{"1min":0.46400000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":10.337999999999999,"5min":12.907999999999999,"15min":24.457000000000001},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.7050000000000001,"5min":1.7450000000000001,"15min":1.6579999999999999},"min":{"1min":0.45000000000000001,"5min":0.42399999999999999,"15min":0.40999999999999998},"max":{"1min":10.279,"5min":13.015000000000001,"15min":24.387},"last":1.208}]},{"osd":7,"last update":"Fri Sep 13 18:21:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.651,"15min":1.397},"min":{"1min":0.65400000000000003,"5min":0.34200000000000003,"15min":0.32300000000000001},"max":{"1min":4.649,"5min":14.138999999999999,"15min":14.138999999999999},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.847,"15min":1.4410000000000001},"min":{"1min":0.47699999999999998,"5min":0.377,"15min":0.35299999999999998},"max":{"1min":4.2270000000000003,"5min":14.013,"15min":14.013},"last":1.365}]}]},{"osd":2,"up_from":15,"seq":64424509827,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1055244,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":1054846,"kb_avail":93316596,"statfs":{"total":96636764160,"available":95556194304,"internally_reserved":0,"allocated":397312,"data_stored":210684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1080162758},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":1.7190000000000001,"15min":1.091},"min":{"1min":0.35199999999999998,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.68400000000000005,"5min":17.971,"15min":17.971},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":1.873,"15min":1.179},"min":{"1min":0.40600000000000003,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.042,"5min":18.001999999999999,"15min":18.001999999999999},"last":0.69599999999999995}]},{"osd":1,"last update":"Fri Sep 13 18:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":2.2759999999999998,"15min":1.6870000000000001},"min":{"1min":0.38,"5min":0.22600000000000001,"15min":0.217},"max":{"1min":1.609,"5min":24.599,"15min":56.939999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":2.5219999999999998,"15min":1.768},"min":{"1min":0.443,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":1.5369999999999999,"5min":24.75,"15min":57.006999999999998},"last":0.54300000000000004}]},{"osd":3,"last update":"Fri Sep 13 18:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":2.79,"15min":1.599},"min":{"1min":0.40899999999999997,"5min":0.218,"15min":0.218},"max":{"1min":1.714,"5min":16.707000000000001,"15min":16.707000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":2.4510000000000001,"15min":1.554},"min":{"1min":0.27600000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.135,"5min":16.640999999999998,"15min":16.640999999999998},"last":0.72299999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:21:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.4370000000000001,"15min":1.236},"min":{"1min":0.67800000000000005,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.6000000000000001,"5min":8.2829999999999995,"15min":14.177},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.056,"5min":1.258,"15min":1.1639999999999999},"min":{"1min":0.55600000000000005,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":3.617,"5min":8.1980000000000004,"15min":14.396000000000001},"last":0.91400000000000003}]},{"osd":5,"last update":"Fri Sep 13 18:21:22 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":0.73599999999999999,"15min":1.0049999999999999},"min":{"1min":0.433,"5min":0.34200000000000003,"15min":0.34100000000000003},"max":{"1min":3.637,"5min":3.6840000000000002,"15min":14.153},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":1.1299999999999999,"15min":1.131},"min":{"1min":0.52100000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.4450000000000001,"5min":7.476,"15min":14.375999999999999},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:21:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":0.97899999999999998,"15min":1.1000000000000001},"min":{"1min":0.505,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.6080000000000001,"5min":6.8730000000000002,"15min":8.3010000000000002},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.151,"5min":1.038,"15min":1.2110000000000001},"min":{"1min":0.63700000000000001,"5min":0.40400000000000003,"15min":0.39200000000000002},"max":{"1min":1.6519999999999999,"5min":6.9500000000000002,"15min":14.074},"last":0.94799999999999995}]},{"osd":7,"last update":"Fri Sep 13 18:21:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0700000000000001,"15min":1.079},"min":{"1min":0.47999999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.5569999999999999,"5min":8.0410000000000004,"15min":8.0410000000000004},"last":0.749},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.1599999999999999,"15min":1.1859999999999999},"min":{"1min":0.56899999999999995,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.3799999999999999,"5min":6.5300000000000002,"15min":14.307},"last":1.0569999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509826,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1738228,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":1737918,"kb_avail":92633612,"statfs":{"total":96636764160,"available":94856818688,"internally_reserved":0,"allocated":307200,"data_stored":116868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1779628486},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":2.5430000000000001,"15min":1.907},"min":{"1min":0.42299999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":4.25,"5min":21.949999999999999,"15min":21.949999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.79300000000000004,"5min":2.3759999999999999,"15min":1.593},"min":{"1min":0.27600000000000002,"5min":0.193,"15min":0.193},"max":{"1min":2.7349999999999999,"5min":23.338999999999999,"15min":23.338999999999999},"last":0.76700000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.8540000000000001,"15min":1.5149999999999999},"min":{"1min":0.38,"5min":0.22,"15min":0.22},"max":{"1min":5.5129999999999999,"5min":11.423999999999999,"15min":14.869999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.962,"15min":1.4319999999999999},"min":{"1min":0.47399999999999998,"5min":0.223,"15min":0.223},"max":{"1min":5.4809999999999999,"5min":12.439,"15min":15.058},"last":0.73499999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":4.3810000000000002,"15min":3.1800000000000002},"min":{"1min":0.44400000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":4.5839999999999996,"5min":27.838000000000001,"15min":41.369999999999997},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.0860000000000001,"5min":4.327,"15min":2.911},"min":{"1min":0.251,"5min":0.216,"15min":0.216},"max":{"1min":4.9299999999999997,"5min":30.077000000000002,"15min":40.325000000000003},"last":1.0569999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":2.4660000000000002,"15min":2.1480000000000001},"min":{"1min":0.51500000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":7.2089999999999996,"5min":29.702000000000002,"15min":103.44499999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.2010000000000001,"5min":2.4689999999999999,"15min":1.905},"min":{"1min":0.52700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.75,"5min":29.425999999999998,"15min":40.953000000000003},"last":1.0429999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:21:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.964,"15min":1.744},"min":{"1min":0.60899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":4.548,"5min":15.169,"15min":18.954000000000001},"last":1.194},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.6919999999999999,"15min":1.5269999999999999},"min":{"1min":0.36699999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":6.2279999999999998,"5min":13.630000000000001,"15min":15.413},"last":1.141}]},{"osd":6,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.994,"5min":2.044,"15min":1.633},"min":{"1min":0.748,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":6.4889999999999999,"5min":13.67,"15min":15.462999999999999},"last":1.2210000000000001},{"interface":"front","average":{"1min":2.0569999999999999,"5min":2.1419999999999999,"15min":1.7270000000000001},"min":{"1min":0.85699999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":6.5220000000000002,"5min":13.696,"15min":15.49},"last":1.278}]},{"osd":7,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.667,"15min":1.415},"min":{"1min":0.64500000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":7.9669999999999996,"5min":12.476000000000001,"15min":15.093999999999999},"last":1.181},{"interface":"front","average":{"1min":1.7569999999999999,"5min":1.7330000000000001,"15min":1.5},"min":{"1min":0.80200000000000005,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":7.9969999999999999,"5min":12.539999999999999,"15min":15.119},"last":1.1539999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509828,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1247072,"kb_used_data":4056,"kb_used_omap":1,"kb_used_meta":1243006,"kb_avail":93124768,"statfs":{"total":96636764160,"available":95359762432,"internally_reserved":0,"allocated":4153344,"data_stored":3967313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1272838598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":2.8849999999999998,"15min":1.7210000000000001},"min":{"1min":0.19800000000000001,"5min":0.182,"15min":0.182},"max":{"1min":7.0830000000000002,"5min":23.004000000000001,"15min":24.202000000000002},"last":0.68100000000000005},{"interface":"front","average":{"1min":1.0700000000000001,"5min":2.9630000000000001,"15min":1.8640000000000001},"min":{"1min":0.375,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":7.1139999999999999,"5min":23.039999999999999,"15min":24.260999999999999},"last":1.097}]},{"osd":2,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":1.6359999999999999,"15min":1.1200000000000001},"min":{"1min":0.254,"5min":0.20399999999999999,"15min":0.186},"max":{"1min":6.4770000000000003,"5min":11.026999999999999,"15min":17.859000000000002},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.851,"15min":1.343},"min":{"1min":0.26600000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":7.4100000000000001,"5min":11.702999999999999,"15min":19.678999999999998},"last":1.0209999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":3.306,"15min":1.95},"min":{"1min":0.28100000000000003,"5min":0.219,"15min":0.219},"max":{"1min":1.0509999999999999,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.073,"5min":2.7130000000000001,"15min":1.8240000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":7.2210000000000001,"5min":15.856999999999999,"15min":15.856999999999999},"last":0.78800000000000003}]},{"osd":4,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.6499999999999999,"15min":1.2729999999999999},"min":{"1min":0.32700000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":7.1959999999999997,"5min":8.7010000000000005,"15min":8.7010000000000005},"last":1.415},{"interface":"front","average":{"1min":0.86899999999999999,"5min":2.2109999999999999,"15min":1.5409999999999999},"min":{"1min":0.34399999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.256,"5min":10.965999999999999,"15min":16.291},"last":1.244}]},{"osd":5,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.7629999999999999,"15min":1.327},"min":{"1min":0.33000000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":3.1339999999999999,"5min":15.18,"15min":15.18},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.8069999999999999,"15min":1.427},"min":{"1min":0.39100000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":7.1669999999999998,"5min":17.852,"15min":17.852},"last":1.1459999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":2.2869999999999999,"15min":1.577},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":7.2480000000000002,"5min":15.467000000000001,"15min":15.467000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.8109999999999999,"15min":1.4330000000000001},"min":{"1min":0.44900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":7.1429999999999998,"5min":17.908000000000001,"15min":17.908000000000001},"last":1.3859999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:21:29 2024","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.897,"15min":1.3979999999999999},"min":{"1min":0.40000000000000002,"5min":0.33500000000000002,"15min":0.32700000000000001},"max":{"1min":5.5789999999999997,"5min":18.986999999999998,"15min":18.986999999999998},"last":1.048},{"interface":"front","average":{"1min":1.421,"5min":1.9770000000000001,"15min":1.4830000000000001},"min":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.373},"max":{"1min":5.6159999999999997,"5min":19.021000000000001,"15min":19.021000000000001},"last":1.2789999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542530,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1211620,"kb_used_data":12252,"kb_used_omap":1,"kb_used_meta":1199358,"kb_avail":93160220,"statfs":{"total":96636764160,"available":95396065280,"internally_reserved":0,"allocated":12546048,"data_stored":12357112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":1228143051},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":1.9470000000000001,"15min":1.2410000000000001},"min":{"1min":0.26500000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":2.875,"5min":35.195,"15min":35.195},"last":0.59299999999999997},{"interface":"front","average":{"1min":1.133,"5min":2.1320000000000001,"15min":1.411},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":5.0919999999999996,"5min":35.253999999999998,"15min":35.253999999999998},"last":0.68500000000000005}]},{"osd":1,"last update":"Fri Sep 13 18:21:30 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":2.5270000000000001,"15min":1.4930000000000001},"min":{"1min":0.317,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":4.8070000000000004,"5min":19.585000000000001,"15min":19.585000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.98899999999999999,"5min":2.52,"15min":1.6359999999999999},"min":{"1min":0.41699999999999998,"5min":0.29599999999999999,"15min":0.28000000000000003},"max":{"1min":4.8220000000000001,"5min":19.617000000000001,"15min":19.617000000000001},"last":0.91100000000000003}]},{"osd":2,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.597,"15min":1.1699999999999999},"min":{"1min":0.51200000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.5449999999999999,"5min":15.789999999999999,"15min":15.789999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.274,"5min":1.5680000000000001,"15min":1.1799999999999999},"min":{"1min":0.38700000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.5259999999999998,"5min":12.965,"15min":12.965},"last":0.99099999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":2.8980000000000001,"15min":1.9119999999999999},"min":{"1min":0.41099999999999998,"5min":0.316,"15min":0.316},"max":{"1min":7.8449999999999998,"5min":40.948999999999998,"15min":40.948999999999998},"last":1.0800000000000001},{"interface":"front","average":{"1min":2.0779999999999998,"5min":3.0139999999999998,"15min":1.768},"min":{"1min":0.39300000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":7.8810000000000002,"5min":25.234000000000002,"15min":25.234000000000002},"last":1.1319999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":1.165,"15min":0.98999999999999999},"min":{"1min":0.33700000000000002,"5min":0.28799999999999998,"15min":0.252},"max":{"1min":3.0609999999999999,"5min":8.8670000000000009,"15min":8.8670000000000009},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.3300000000000001,"15min":1.1859999999999999},"min":{"1min":0.34499999999999997,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":6.069,"5min":10.279999999999999,"15min":10.279999999999999},"last":0.95499999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":1.0620000000000001,"15min":0.98199999999999998},"min":{"1min":0.374,"5min":0.311,"15min":0.27000000000000002},"max":{"1min":1.5289999999999999,"5min":8.1699999999999999,"15min":8.1699999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.96299999999999997,"15min":1.02},"min":{"1min":0.39800000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.9490000000000001,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":0.84599999999999997}]},{"osd":6,"last update":"Fri Sep 13 18:21:20 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.4790000000000001,"15min":1.236},"min":{"1min":0.41599999999999998,"5min":0.34599999999999997,"15min":0.309},"max":{"1min":3.597,"5min":8.2219999999999995,"15min":8.2219999999999995},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.419,"5min":1.548,"15min":1.2789999999999999},"min":{"1min":0.48499999999999999,"5min":0.434,"15min":0.34499999999999997},"max":{"1min":3.7210000000000001,"5min":8.2469999999999999,"15min":8.2469999999999999},"last":1.2290000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542530,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1419436,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":1415038,"kb_avail":92952404,"statfs":{"total":96636764160,"available":95183261696,"internally_reserved":0,"allocated":4493312,"data_stored":4311530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1448999367},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":2.0750000000000002,"5min":2.0699999999999998,"15min":1.296},"min":{"1min":0.28000000000000003,"5min":0.214,"15min":0.214},"max":{"1min":11.666,"5min":12.808999999999999,"15min":12.808999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.6639999999999999,"15min":1.179},"min":{"1min":0.28899999999999998,"5min":0.218,"15min":0.218},"max":{"1min":7.8700000000000001,"5min":12.308999999999999,"15min":12.308999999999999},"last":1.006}]},{"osd":1,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":2.6840000000000002,"5min":2.323,"15min":1.4930000000000001},"min":{"1min":0.52300000000000002,"5min":0.313,"15min":0.28699999999999998},"max":{"1min":20.530999999999999,"5min":20.530999999999999,"15min":20.530999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":2.2959999999999998,"5min":2.2149999999999999,"15min":1.514},"min":{"1min":0.377,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":20.501999999999999,"5min":20.501999999999999,"15min":20.501999999999999},"last":0.747}]},{"osd":2,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.7350000000000001,"15min":1.2270000000000001},"min":{"1min":0.47699999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":3.6800000000000002,"5min":10.792999999999999,"15min":10.792999999999999},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.8140000000000001,"15min":1.381},"min":{"1min":0.51800000000000002,"5min":0.317,"15min":0.317},"max":{"1min":3.738,"5min":10.826000000000001,"15min":10.826000000000001},"last":1.3340000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:21:21 2024","interfaces":[{"interface":"back","average":{"1min":2.4940000000000002,"5min":3.8300000000000001,"15min":2.5750000000000002},"min":{"1min":0.36599999999999999,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":16.495999999999999,"5min":32.25,"15min":84.867999999999995},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.4199999999999999,"5min":2.5009999999999999,"15min":1.927},"min":{"1min":0.36699999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":8.6780000000000008,"5min":14.064,"15min":84.343999999999994},"last":1.1000000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.651,"15min":1.212},"min":{"1min":0.70199999999999996,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.1990000000000001,"5min":11.044,"15min":11.044},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.2589999999999999,"15min":1.117},"min":{"1min":0.78300000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.181,"5min":9.4719999999999995,"15min":9.4719999999999995},"last":1.181}]},{"osd":5,"last update":"Fri Sep 13 18:22:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.3400000000000001,"15min":1.1599999999999999},"min":{"1min":0.74199999999999999,"5min":0.318,"15min":0.29199999999999998},"max":{"1min":1.3979999999999999,"5min":8.7880000000000003,"15min":8.7880000000000003},"last":1.415},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1659999999999999,"15min":1.1080000000000001},"min":{"1min":0.92000000000000004,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.284,"5min":7.569,"15min":7.569},"last":1.306}]},{"osd":7,"last update":"Fri Sep 13 18:22:12 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3009999999999999,"15min":1.1890000000000001},"min":{"1min":0.91700000000000004,"5min":0.36799999999999999,"15min":0.29999999999999999},"max":{"1min":1.5840000000000001,"5min":7.1820000000000004,"15min":7.1820000000000004},"last":1.486},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3799999999999999,"15min":1.2629999999999999},"min":{"1min":1.008,"5min":0.40500000000000003,"15min":0.33500000000000002},"max":{"1min":1.635,"5min":7.2039999999999997,"15min":7.2039999999999997},"last":1.363}]}]},{"osd":5,"up_from":14,"seq":60129542532,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":978544,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":978238,"kb_avail":93393296,"statfs":{"total":96636764160,"available":95634735104,"internally_reserved":0,"allocated":303104,"data_stored":117302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1001716166},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.595,"15min":1.1819999999999999},"min":{"1min":0.316,"5min":0.218,"15min":0.218},"max":{"1min":4.0919999999999996,"5min":10.051,"15min":10.148999999999999},"last":1.091},{"interface":"front","average":{"1min":1.038,"5min":1.633,"15min":1.3340000000000001},"min":{"1min":0.40500000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.9550000000000001,"5min":10.098000000000001,"15min":10.185},"last":0.96299999999999997}]},{"osd":1,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.647,"15min":1.1879999999999999},"min":{"1min":0.28899999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":4.7119999999999997,"5min":10.08,"15min":10.436999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.032,"5min":1.843,"15min":1.387},"min":{"1min":0.44500000000000001,"5min":0.253,"15min":0.253},"max":{"1min":2.4369999999999998,"5min":19.388000000000002,"15min":19.388000000000002},"last":0.87}]},{"osd":2,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.5249999999999999,"15min":1.129},"min":{"1min":0.443,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":5.1660000000000004,"5min":12.667,"15min":12.667},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.6639999999999999,"15min":1.2},"min":{"1min":0.441,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.8900000000000001,"5min":34.905999999999999,"15min":34.905999999999999},"last":1.2989999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":3.6779999999999999,"15min":2.3220000000000001},"min":{"1min":0.374,"5min":0.314,"15min":0.314},"max":{"1min":9.077,"5min":25.149999999999999,"15min":28.295000000000002},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.242,"5min":2.9180000000000001,"15min":1.7410000000000001},"min":{"1min":0.436,"5min":0.38500000000000001,"15min":0.318},"max":{"1min":5.3689999999999998,"5min":19.731000000000002,"15min":19.731000000000002},"last":1.0469999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.0960000000000001,"15min":0.97399999999999998},"min":{"1min":0.33000000000000002,"5min":0.28599999999999998,"15min":0.255},"max":{"1min":4.8140000000000001,"5min":10.023999999999999,"15min":10.023999999999999},"last":1.196},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.599,"15min":1.2290000000000001},"min":{"1min":0.38700000000000001,"5min":0.32500000000000001,"15min":0.28999999999999998},"max":{"1min":4.7919999999999998,"5min":10.210000000000001,"15min":10.210000000000001},"last":1.2609999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":1.4039999999999999,"15min":1.1339999999999999},"min":{"1min":0.36799999999999999,"5min":0.33900000000000002,"15min":0.25800000000000001},"max":{"1min":1.5649999999999999,"5min":6.6849999999999996,"15min":6.6849999999999996},"last":1.139},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3779999999999999,"15min":1.2150000000000001},"min":{"1min":0.51800000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":5.0819999999999999,"5min":10.151,"15min":10.151},"last":1.339}]},{"osd":7,"last update":"Fri Sep 13 18:21:27 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.258,"15min":1.1459999999999999},"min":{"1min":0.46899999999999997,"5min":0.32900000000000001,"15min":0.32000000000000001},"max":{"1min":2.581,"5min":7.9329999999999998,"15min":7.9329999999999998},"last":1.462},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.196,"15min":1.151},"min":{"1min":0.53800000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":2.5,"5min":5.976,"15min":5.976},"last":1.5149999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542530,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1301036,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":1296638,"kb_avail":93070804,"statfs":{"total":96636764160,"available":95304503296,"internally_reserved":0,"allocated":4493312,"data_stored":4310894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1327757766},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.4610000000000001,"15min":1.3220000000000001},"min":{"1min":0.69199999999999995,"5min":0.21299999999999999,"15min":0.20999999999999999},"max":{"1min":1.272,"5min":14.318,"15min":28.439},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.575,"15min":1.4470000000000001},"min":{"1min":0.626,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.2769999999999999,"5min":16.943000000000001,"15min":28.422000000000001},"last":1.127}]},{"osd":1,"last update":"Fri Sep 13 18:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":3.7469999999999999,"15min":2.649},"min":{"1min":0.69999999999999996,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.2609999999999999,"5min":30.277999999999999,"15min":38.895000000000003},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.004,"5min":3.4129999999999998,"15min":2.028},"min":{"1min":0.80900000000000005,"5min":0.28199999999999997,"15min":0.27000000000000002},"max":{"1min":1.3049999999999999,"5min":32.670000000000002,"15min":32.670000000000002},"last":0.82099999999999995}]},{"osd":2,"last update":"Fri Sep 13 18:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":1.619,"15min":1.1870000000000001},"min":{"1min":0.439,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.2070000000000001,"5min":25.167999999999999,"15min":25.167999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.742,"15min":1.3520000000000001},"min":{"1min":0.624,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.1990000000000001,"5min":25.218,"15min":25.218},"last":0.75800000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":2.6259999999999999,"15min":1.8340000000000001},"min":{"1min":0.67500000000000004,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.304,"5min":43.945999999999998,"15min":43.945999999999998},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.038,"5min":2.927,"15min":2.145},"min":{"1min":0.68500000000000005,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":1.385,"5min":29.640000000000001,"15min":29.640000000000001},"last":0.96999999999999997}]},{"osd":5,"last update":"Fri Sep 13 18:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.1639999999999999,"15min":1.0600000000000001},"min":{"1min":0.78100000000000003,"5min":0.255,"15min":0.255},"max":{"1min":1.425,"5min":12.282999999999999,"15min":12.282999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.3839999999999999,"15min":1.1919999999999999},"min":{"1min":0.76100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.7210000000000001,"5min":16.995999999999999,"15min":16.995999999999999},"last":1.23}]},{"osd":6,"last update":"Fri Sep 13 18:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.4990000000000001,"15min":1.4079999999999999},"min":{"1min":0.80600000000000005,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.4930000000000001,"5min":12.42,"15min":23.010999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.256,"5min":1.3500000000000001,"15min":1.2070000000000001},"min":{"1min":0.93000000000000005,"5min":0.33800000000000002,"15min":0.33600000000000002},"max":{"1min":1.556,"5min":7.9489999999999998,"15min":7.9489999999999998},"last":1.2569999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.319,"15min":1.3069999999999999},"min":{"1min":0.98599999999999999,"5min":0.30499999999999999,"15min":0.28799999999999998},"max":{"1min":1.5960000000000001,"5min":11.298,"15min":20.417000000000002},"last":1.29},{"interface":"front","average":{"1min":1.321,"5min":1.351,"15min":1.276},"min":{"1min":1.0720000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.623,"5min":12.308999999999999,"15min":12.308999999999999},"last":1.3200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3276800,"data_stored":3261021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":94760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8503296,"data_stored":8484469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4194970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4195606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":11669504,"data_stored":11650820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:22:27.421 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:27.421 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:27.421 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:27.422 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-13T17:50:37.577645+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=37, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:27.422 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:27.422 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:27.423 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-09-13T17:50:34.174247+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=50, tm_sec=34, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=21, tm_sec=21, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:22:27.423 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T18:22:47.424 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:22:47.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.532+0000 7f5921403700 1 Processor -- start 2024-09-13T18:22:47.540 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.532+0000 7f5921403700 1 -- start start 2024-09-13T18:22:47.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.532+0000 7f5921403700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f591c1510d0 0x7f591c157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:47.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.532+0000 7f5921403700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f591c0c7400 0x7f591c0c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:47.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.532+0000 7f5921403700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591c15a8b0 con 0x7f591c1514b0 2024-09-13T18:22:47.542 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.532+0000 7f5921403700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591c0c2e80 con 0x7f591c0c7400 2024-09-13T18:22:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.532+0000 7f5921403700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591c1581e0 con 0x7f591c1510d0 2024-09-13T18:22:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.532+0000 7f5920c02700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f591c0c7400 0x7f591c0c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.532+0000 7f591b7fe700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f591c1514b0 0x7f591c0c2770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33380/0 (socket says 172.21.15.40:33380) 2024-09-13T18:22:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.532+0000 7f591b7fe700 1 -- 172.21.15.40:0/1489408978 learned_addr learned my addr 172.21.15.40:0/1489408978 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:22:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.536+0000 7f591bfff700 1 --2- 172.21.15.40:0/1489408978 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f591c1510d0 0x7f591c157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.536+0000 7f5920c02700 1 -- 172.21.15.40:0/1489408978 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f591c1510d0 msgr2=0x7f591c157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.536+0000 7f5920c02700 1 --2- 172.21.15.40:0/1489408978 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f591c1510d0 0x7f591c157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.536+0000 7f5920c02700 1 -- 172.21.15.40:0/1489408978 >> v1:172.21.15.40:6789/0 conn(0x7f591c1514b0 legacy=0x7f591c0c2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.536+0000 7f5920c02700 1 -- 172.21.15.40:0/1489408978 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c066390 con 0x7f591c0c7400 2024-09-13T18:22:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.536+0000 7f591affd700 1 -- 172.21.15.40:0/1489408978 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 622086297 0 0) 0x7f591c15a8b0 con 0x7f591c1514b0 2024-09-13T18:22:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.536+0000 7f591bfff700 1 --2- 172.21.15.40:0/1489408978 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f591c1510d0 0x7f591c157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.536+0000 7f5920c02700 1 --2- 172.21.15.40:0/1489408978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f591c0c7400 0x7f591c0c4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5914009d90 tx=0x7f591400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4dac9739c5812c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:47.543 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.536+0000 7f591affd700 1 -- 172.21.15.40:0/1489408978 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591400e070 con 0x7f591c0c7400 2024-09-13T18:22:47.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.536+0000 7f591affd700 1 -- 172.21.15.40:0/1489408978 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591400bbd0 con 0x7f591c0c7400 2024-09-13T18:22:47.544 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.536+0000 7f591affd700 1 -- 172.21.15.40:0/1489408978 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914012430 con 0x7f591c0c7400 2024-09-13T18:22:47.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.536+0000 7f5921403700 1 -- 172.21.15.40:0/1489408978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f591c0c7400 msgr2=0x7f591c0c4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:47.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.536+0000 7f5921403700 1 --2- 172.21.15.40:0/1489408978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f591c0c7400 0x7f591c0c4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5914009d90 tx=0x7f591400b330 comp rx=0 tx=0).stop 2024-09-13T18:22:47.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.536+0000 7f5921403700 1 -- 172.21.15.40:0/1489408978 shutdown_connections 2024-09-13T18:22:47.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.536+0000 7f5921403700 1 --2- 172.21.15.40:0/1489408978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f591c0c7400 0x7f591c0c4dd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:47.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.536+0000 7f5921403700 1 --2- 172.21.15.40:0/1489408978 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f591c1510d0 0x7f591c157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:47.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.536+0000 7f5921403700 1 -- 172.21.15.40:0/1489408978 >> 172.21.15.40:0/1489408978 conn(0x7f591c0c7d70 msgr2=0x7f591c0c8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:47.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.536+0000 7f5921403700 1 -- 172.21.15.40:0/1489408978 shutdown_connections 2024-09-13T18:22:47.545 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.536+0000 7f5921403700 1 -- 172.21.15.40:0/1489408978 wait complete. 2024-09-13T18:22:47.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f5921403700 1 Processor -- start 2024-09-13T18:22:47.546 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f5921403700 1 -- start start 2024-09-13T18:22:47.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f5921403700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f591c0c7400 0x7f591c14d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:47.548 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f5921403700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f591c1510d0 0x7f591c1599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:47.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f5921403700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f591c1514b0 0x7f591c159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:47.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f5921403700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c1ecc70 con 0x7f591c1510d0 2024-09-13T18:22:47.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f5921403700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c1eda90 con 0x7f591c0c7400 2024-09-13T18:22:47.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f5921403700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591c154c70 con 0x7f591c1514b0 2024-09-13T18:22:47.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f591b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f591c1510d0 0x7f591c1599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:47.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f591b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f591c1510d0 0x7f591c1599d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48722/0 (socket says 172.21.15.40:48722) 2024-09-13T18:22:47.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f591b7fe700 1 -- 172.21.15.40:0/4083262474 learned_addr learned my addr 172.21.15.40:0/4083262474 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:22:47.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f591bfff700 1 --2- 172.21.15.40:0/4083262474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f591c0c7400 0x7f591c14d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:47.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f5920c02700 1 --2- 172.21.15.40:0/4083262474 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f591c1514b0 0x7f591c159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:47.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f591b7fe700 1 -- 172.21.15.40:0/4083262474 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f591c1514b0 msgr2=0x7f591c159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:47.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f591b7fe700 1 --2- 172.21.15.40:0/4083262474 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f591c1514b0 0x7f591c159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:47.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f591b7fe700 1 -- 172.21.15.40:0/4083262474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f591c0c7400 msgr2=0x7f591c14d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:47.549 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f591b7fe700 1 --2- 172.21.15.40:0/4083262474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f591c0c7400 0x7f591c14d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:47.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f591b7fe700 1 -- 172.21.15.40:0/4083262474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5914009a70 con 0x7f591c1510d0 2024-09-13T18:22:47.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f591bfff700 1 --2- 172.21.15.40:0/4083262474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f591c0c7400 0x7f591c14d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:47.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f591b7fe700 1 --2- 172.21.15.40:0/4083262474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f591c1510d0 0x7f591c1599d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f591000c4e0 tx=0x7f591000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:47.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f5918ff9700 1 -- 172.21.15.40:0/4083262474 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5910004ca0 con 0x7f591c1510d0 2024-09-13T18:22:47.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f5918ff9700 1 -- 172.21.15.40:0/4083262474 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5910004e00 con 0x7f591c1510d0 2024-09-13T18:22:47.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f5921403700 1 -- 172.21.15.40:0/4083262474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591c14e2d0 con 0x7f591c1510d0 2024-09-13T18:22:47.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f5918ff9700 1 -- 172.21.15.40:0/4083262474 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5910007310 con 0x7f591c1510d0 2024-09-13T18:22:47.550 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f5921403700 1 -- 172.21.15.40:0/4083262474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591c1ed160 con 0x7f591c1510d0 2024-09-13T18:22:47.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.540+0000 7f5921403700 1 -- 172.21.15.40:0/4083262474 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591c1edbd0 con 0x7f591c1510d0 2024-09-13T18:22:47.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.544+0000 7f5918ff9700 1 -- 172.21.15.40:0/4083262474 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5910007470 con 0x7f591c1510d0 2024-09-13T18:22:47.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.544+0000 7f5918ff9700 1 --2- 172.21.15.40:0/4083262474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5908066730 0x7f5908068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:47.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.544+0000 7f591bfff700 1 --2- 172.21.15.40:0/4083262474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5908066730 0x7f5908068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:47.551 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.544+0000 7f5918ff9700 1 -- 172.21.15.40:0/4083262474 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5910084320 con 0x7f591c1510d0 2024-09-13T18:22:47.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.544+0000 7f591bfff700 1 --2- 172.21.15.40:0/4083262474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5908066730 0x7f5908068bf0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f590c009a10 tx=0x7f590c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:47.559 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.552+0000 7f5918ff9700 1 -- 172.21.15.40:0/4083262474 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f591c1edbd0 con 0x7f591c1510d0 2024-09-13T18:22:47.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.716+0000 7f5921403700 1 -- 172.21.15.40:0/4083262474 --> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f591c15b0e0 con 0x7f5908066730 2024-09-13T18:22:47.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.732+0000 7f5918ff9700 1 -- 172.21.15.40:0/4083262474 <== mgr.4101 v2:172.21.15.40:6832/3394537609 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+371868 (secure 0 0 0) 0x7f590c0096a0 con 0x7f5908066730 2024-09-13T18:22:47.745 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T18:22:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.736+0000 7f5921403700 1 -- 172.21.15.40:0/4083262474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5908066730 msgr2=0x7f5908068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:47.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.736+0000 7f5921403700 1 --2- 172.21.15.40:0/4083262474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5908066730 0x7f5908068bf0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f590c009a10 tx=0x7f590c008040 comp rx=0 tx=0).stop 2024-09-13T18:22:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.736+0000 7f5921403700 1 -- 172.21.15.40:0/4083262474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f591c1510d0 msgr2=0x7f591c1599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.736+0000 7f5921403700 1 --2- 172.21.15.40:0/4083262474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f591c1510d0 0x7f591c1599d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f591000c4e0 tx=0x7f591000c8a0 comp rx=0 tx=0).stop 2024-09-13T18:22:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.736+0000 7f5921403700 1 -- 172.21.15.40:0/4083262474 shutdown_connections 2024-09-13T18:22:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.736+0000 7f5921403700 1 --2- 172.21.15.40:0/4083262474 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f5908066730 0x7f5908068bf0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.736+0000 7f5921403700 1 --2- 172.21.15.40:0/4083262474 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f591c1514b0 0x7f591c159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.736+0000 7f5921403700 1 --2- 172.21.15.40:0/4083262474 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f591c1510d0 0x7f591c1599d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.736+0000 7f5921403700 1 --2- 172.21.15.40:0/4083262474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f591c0c7400 0x7f591c14d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:47.750 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.736+0000 7f5921403700 1 -- 172.21.15.40:0/4083262474 >> 172.21.15.40:0/4083262474 conn(0x7f591c0c7d70 msgr2=0x7f591c06df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.736+0000 7f5921403700 1 -- 172.21.15.40:0/4083262474 shutdown_connections 2024-09-13T18:22:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.736+0000 7f5921403700 1 -- 172.21.15.40:0/4083262474 wait complete. 2024-09-13T18:22:47.751 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2024-09-13T18:22:47.757 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":992,"stamp":"2024-09-13T18:22:45.804296+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":16532352,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":237,"num_read_kb":279,"num_write":755012,"num_write_kb":16125724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15000,"num_omap_keys":30,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183120,"ondisk_log_size":183120,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10723352,"kb_used_data":34648,"kb_used_omap":12,"kb_used_meta":10688627,"kb_avail":744251368,"statfs":{"total":773094113280,"available":762113400832,"internally_reserved":0,"allocated":35479552,"data_stored":33992096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":10945154616},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005120"},"pg_stats":[{"pgid":"2.1f","version":"27'272","reported_seq":316,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:30.247079+0000","last_change":"2024-09-13T18:21:30.247079+0000","last_active":"2024-09-13T18:21:30.247079+0000","last_peered":"2024-09-13T18:21:30.247079+0000","last_clean":"2024-09-13T18:21:30.247079+0000","last_became_active":"2024-09-13T17:50:34.211867+0000","last_became_peered":"2024-09-13T17:50:34.211867+0000","last_unstale":"2024-09-13T18:21:30.247079+0000","last_undegraded":"2024-09-13T18:21:30.247079+0000","last_fullsized":"2024-09-13T18:21:30.247079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'272","last_scrub_stamp":"2024-09-13T18:21:30.246880+0000","last_deep_scrub":"27'272","last_deep_scrub_stamp":"2024-09-13T18:21:30.246880+0000","last_clean_scrub_stamp":"2024-09-13T18:21:30.246880+0000","objects_scrubbed":5,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:57:30.406058+0000","scrub_duration":0.02016366,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3260943,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":267,"num_write_kb":3318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'10","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:51.192502+0000","last_change":"2024-09-13T18:21:51.192810+0000","last_active":"2024-09-13T18:21:51.192502+0000","last_peered":"2024-09-13T18:21:51.192502+0000","last_clean":"2024-09-13T18:21:51.192502+0000","last_became_active":"2024-09-13T17:50:34.587108+0000","last_became_peered":"2024-09-13T17:50:34.587108+0000","last_unstale":"2024-09-13T18:21:51.192502+0000","last_undegraded":"2024-09-13T18:21:51.192502+0000","last_fullsized":"2024-09-13T18:21:51.192502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-13T18:21:51.192437+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-13T18:21:51.192437+0000","last_clean_scrub_stamp":"2024-09-13T18:21:51.192437+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:08:50.505781+0000","scrub_duration":0.0029908370000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'5594","reported_seq":5633,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:16.243447+0000","last_change":"2024-09-13T18:22:16.243447+0000","last_active":"2024-09-13T18:22:16.243447+0000","last_peered":"2024-09-13T18:22:16.243447+0000","last_clean":"2024-09-13T18:22:16.243447+0000","last_became_active":"2024-09-13T17:50:38.614723+0000","last_became_peered":"2024-09-13T17:50:38.614723+0000","last_unstale":"2024-09-13T18:22:16.243447+0000","last_undegraded":"2024-09-13T18:22:16.243447+0000","last_fullsized":"2024-09-13T18:22:16.243447+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5594","last_scrub_stamp":"2024-09-13T18:22:16.243294+0000","last_deep_scrub":"27'5594","last_deep_scrub_stamp":"2024-09-13T18:22:16.243294+0000","last_clean_scrub_stamp":"2024-09-13T18:22:16.243294+0000","objects_scrubbed":0,"log_size":5594,"log_dups_size":0,"ondisk_log_size":5594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:40:17.525529+0000","scrub_duration":0.045218341000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23732,"num_write_kb":533374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'6","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:36.210186+0000","last_change":"2024-09-13T18:21:36.210368+0000","last_active":"2024-09-13T18:21:36.210186+0000","last_peered":"2024-09-13T18:21:36.210186+0000","last_clean":"2024-09-13T18:21:36.210186+0000","last_became_active":"2024-09-13T17:50:34.202930+0000","last_became_peered":"2024-09-13T17:50:34.202930+0000","last_unstale":"2024-09-13T18:21:36.210186+0000","last_undegraded":"2024-09-13T18:21:36.210186+0000","last_fullsized":"2024-09-13T18:21:36.210186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-09-13T18:21:36.210139+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-09-13T18:21:36.210139+0000","last_clean_scrub_stamp":"2024-09-13T18:21:36.210139+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:56.166821+0000","scrub_duration":0.0028089339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'20","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:06.315683+0000","last_change":"2024-09-13T18:22:06.315936+0000","last_active":"2024-09-13T18:22:06.315683+0000","last_peered":"2024-09-13T18:22:06.315683+0000","last_clean":"2024-09-13T18:22:06.315683+0000","last_became_active":"2024-09-13T17:51:16.365492+0000","last_became_peered":"2024-09-13T17:51:16.365492+0000","last_unstale":"2024-09-13T18:22:06.315683+0000","last_undegraded":"2024-09-13T18:22:06.315683+0000","last_fullsized":"2024-09-13T18:22:06.315683+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-13T18:22:06.315628+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-13T18:22:06.315628+0000","last_clean_scrub_stamp":"2024-09-13T18:22:06.315628+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:31.992466+0000","scrub_duration":0.002894284,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'5615","reported_seq":5654,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:48.296837+0000","last_change":"2024-09-13T18:21:48.296837+0000","last_active":"2024-09-13T18:21:48.296837+0000","last_peered":"2024-09-13T18:21:48.296837+0000","last_clean":"2024-09-13T18:21:48.296837+0000","last_became_active":"2024-09-13T17:50:38.618423+0000","last_became_peered":"2024-09-13T17:50:38.618423+0000","last_unstale":"2024-09-13T18:21:48.296837+0000","last_undegraded":"2024-09-13T18:21:48.296837+0000","last_fullsized":"2024-09-13T18:21:48.296837+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5615","last_scrub_stamp":"2024-09-13T18:21:48.296488+0000","last_deep_scrub":"27'5615","last_deep_scrub_stamp":"2024-09-13T18:21:48.296488+0000","last_clean_scrub_stamp":"2024-09-13T18:21:48.296488+0000","objects_scrubbed":0,"log_size":5615,"log_dups_size":0,"ondisk_log_size":5615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:52:01.155282+0000","scrub_duration":0.034902473000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23762,"num_write_kb":516686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'13","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:00.532888+0000","last_change":"2024-09-13T18:22:00.533193+0000","last_active":"2024-09-13T18:22:00.532888+0000","last_peered":"2024-09-13T18:22:00.532888+0000","last_clean":"2024-09-13T18:22:00.532888+0000","last_became_active":"2024-09-13T17:50:34.214652+0000","last_became_peered":"2024-09-13T17:50:34.214652+0000","last_unstale":"2024-09-13T18:22:00.532888+0000","last_undegraded":"2024-09-13T18:22:00.532888+0000","last_fullsized":"2024-09-13T18:22:00.532888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'13","last_scrub_stamp":"2024-09-13T18:22:00.532833+0000","last_deep_scrub":"27'13","last_deep_scrub_stamp":"2024-09-13T18:22:00.532833+0000","last_clean_scrub_stamp":"2024-09-13T18:22:00.532833+0000","objects_scrubbed":2,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:54:05.246113+0000","scrub_duration":0.0070529670000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'18","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:01.707673+0000","last_change":"2024-09-13T18:22:01.707968+0000","last_active":"2024-09-13T18:22:01.707673+0000","last_peered":"2024-09-13T18:22:01.707673+0000","last_clean":"2024-09-13T18:22:01.707673+0000","last_became_active":"2024-09-13T17:51:16.281549+0000","last_became_peered":"2024-09-13T17:51:16.281549+0000","last_unstale":"2024-09-13T18:22:01.707673+0000","last_undegraded":"2024-09-13T18:22:01.707673+0000","last_fullsized":"2024-09-13T18:22:01.707673+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:22:01.707624+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:22:01.707624+0000","last_clean_scrub_stamp":"2024-09-13T18:22:01.707624+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:30.522086+0000","scrub_duration":0.0032846300000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"27'5142","reported_seq":5180,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:53.315998+0000","last_change":"2024-09-13T18:21:53.316137+0000","last_active":"2024-09-13T18:21:53.315998+0000","last_peered":"2024-09-13T18:21:53.315998+0000","last_clean":"2024-09-13T18:21:53.315998+0000","last_became_active":"2024-09-13T17:50:38.614329+0000","last_became_peered":"2024-09-13T17:50:38.614329+0000","last_unstale":"2024-09-13T18:21:53.315998+0000","last_undegraded":"2024-09-13T18:21:53.315998+0000","last_fullsized":"2024-09-13T18:21:53.315998+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5142","last_scrub_stamp":"2024-09-13T18:21:53.315976+0000","last_deep_scrub":"27'5142","last_deep_scrub_stamp":"2024-09-13T18:21:53.315976+0000","last_clean_scrub_stamp":"2024-09-13T18:21:53.315976+0000","objects_scrubbed":0,"log_size":5142,"log_dups_size":0,"ondisk_log_size":5142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:24.336476+0000","scrub_duration":0.034529740000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21334,"num_write_kb":478261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'9","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:52.469363+0000","last_change":"2024-09-13T18:21:52.469614+0000","last_active":"2024-09-13T18:21:52.469363+0000","last_peered":"2024-09-13T18:21:52.469363+0000","last_clean":"2024-09-13T18:21:52.469363+0000","last_became_active":"2024-09-13T17:50:34.215661+0000","last_became_peered":"2024-09-13T17:50:34.215661+0000","last_unstale":"2024-09-13T18:21:52.469363+0000","last_undegraded":"2024-09-13T18:21:52.469363+0000","last_fullsized":"2024-09-13T18:21:52.469363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-09-13T18:21:52.469312+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-09-13T18:21:52.469312+0000","last_clean_scrub_stamp":"2024-09-13T18:21:52.469312+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:38:08.307554+0000","scrub_duration":0.0060588509999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'30","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:57.810967+0000","last_change":"2024-09-13T18:21:57.811166+0000","last_active":"2024-09-13T18:21:57.810967+0000","last_peered":"2024-09-13T18:21:57.810967+0000","last_clean":"2024-09-13T18:21:57.810967+0000","last_became_active":"2024-09-13T17:50:34.599409+0000","last_became_peered":"2024-09-13T17:50:34.599409+0000","last_unstale":"2024-09-13T18:21:57.810967+0000","last_undegraded":"2024-09-13T18:21:57.810967+0000","last_fullsized":"2024-09-13T18:21:57.810967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'30","last_scrub_stamp":"2024-09-13T18:21:57.810914+0000","last_deep_scrub":"27'30","last_deep_scrub_stamp":"2024-09-13T18:21:57.810914+0000","last_clean_scrub_stamp":"2024-09-13T18:21:57.810914+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:17:02.009800+0000","scrub_duration":0.003036955,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'4871","reported_seq":4909,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:49.336009+0000","last_change":"2024-09-13T18:21:49.336395+0000","last_active":"2024-09-13T18:21:49.336009+0000","last_peered":"2024-09-13T18:21:49.336009+0000","last_clean":"2024-09-13T18:21:49.336009+0000","last_became_active":"2024-09-13T17:50:38.614311+0000","last_became_peered":"2024-09-13T17:50:38.614311+0000","last_unstale":"2024-09-13T18:21:49.336009+0000","last_undegraded":"2024-09-13T18:21:49.336009+0000","last_fullsized":"2024-09-13T18:21:49.336009+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4871","last_scrub_stamp":"2024-09-13T18:21:49.335944+0000","last_deep_scrub":"27'4871","last_deep_scrub_stamp":"2024-09-13T18:21:49.335944+0000","last_clean_scrub_stamp":"2024-09-13T18:21:49.335944+0000","objects_scrubbed":0,"log_size":4871,"log_dups_size":0,"ondisk_log_size":4871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:12.664992+0000","scrub_duration":0.039294008999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20106,"num_write_kb":432357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'14","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:43.433109+0000","last_change":"2024-09-13T18:21:43.433365+0000","last_active":"2024-09-13T18:21:43.433109+0000","last_peered":"2024-09-13T18:21:43.433109+0000","last_clean":"2024-09-13T18:21:43.433109+0000","last_became_active":"2024-09-13T17:50:34.599207+0000","last_became_peered":"2024-09-13T17:50:34.599207+0000","last_unstale":"2024-09-13T18:21:43.433109+0000","last_undegraded":"2024-09-13T18:21:43.433109+0000","last_fullsized":"2024-09-13T18:21:43.433109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-09-13T18:21:43.433055+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-09-13T18:21:43.433055+0000","last_clean_scrub_stamp":"2024-09-13T18:21:43.433055+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:59:28.085068+0000","scrub_duration":0.003161358,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'1095","reported_seq":1138,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:56.570301+0000","last_change":"2024-09-13T18:21:56.570301+0000","last_active":"2024-09-13T18:21:56.570301+0000","last_peered":"2024-09-13T18:21:56.570301+0000","last_clean":"2024-09-13T18:21:56.570301+0000","last_became_active":"2024-09-13T17:50:34.215409+0000","last_became_peered":"2024-09-13T17:50:34.215409+0000","last_unstale":"2024-09-13T18:21:56.570301+0000","last_undegraded":"2024-09-13T18:21:56.570301+0000","last_fullsized":"2024-09-13T18:21:56.570301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'1095","last_scrub_stamp":"2024-09-13T18:21:56.570057+0000","last_deep_scrub":"27'1095","last_deep_scrub_stamp":"2024-09-13T18:21:56.570057+0000","last_clean_scrub_stamp":"2024-09-13T18:21:56.570057+0000","objects_scrubbed":3,"log_size":1095,"log_dups_size":0,"ondisk_log_size":1095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:35:11.168274+0000","scrub_duration":0.023788684000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1088,"num_write_kb":4339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'5394","reported_seq":5434,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:57.350995+0000","last_change":"2024-09-13T18:21:57.350995+0000","last_active":"2024-09-13T18:21:57.350995+0000","last_peered":"2024-09-13T18:21:57.350995+0000","last_clean":"2024-09-13T18:21:57.350995+0000","last_became_active":"2024-09-13T17:50:38.611263+0000","last_became_peered":"2024-09-13T17:50:38.611263+0000","last_unstale":"2024-09-13T18:21:57.350995+0000","last_undegraded":"2024-09-13T18:21:57.350995+0000","last_fullsized":"2024-09-13T18:21:57.350995+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5394","last_scrub_stamp":"2024-09-13T18:21:57.350563+0000","last_deep_scrub":"27'5394","last_deep_scrub_stamp":"2024-09-13T18:21:57.350563+0000","last_clean_scrub_stamp":"2024-09-13T18:21:57.350563+0000","objects_scrubbed":0,"log_size":5394,"log_dups_size":0,"ondisk_log_size":5394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:17.207282+0000","scrub_duration":0.032566444,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22326,"num_write_kb":494524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'19","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:52.230080+0000","last_change":"2024-09-13T18:21:52.230080+0000","last_active":"2024-09-13T18:21:52.230080+0000","last_peered":"2024-09-13T18:21:52.230080+0000","last_clean":"2024-09-13T18:21:52.230080+0000","last_became_active":"2024-09-13T17:50:34.585831+0000","last_became_peered":"2024-09-13T17:50:34.585831+0000","last_unstale":"2024-09-13T18:21:52.230080+0000","last_undegraded":"2024-09-13T18:21:52.230080+0000","last_fullsized":"2024-09-13T18:21:52.230080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-13T18:21:52.229769+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-13T18:21:52.229769+0000","last_clean_scrub_stamp":"2024-09-13T18:21:52.229769+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:23:09.681178+0000","scrub_duration":0.0031441960000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'10","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:34.350232+0000","last_change":"2024-09-13T18:21:34.350402+0000","last_active":"2024-09-13T18:21:34.350232+0000","last_peered":"2024-09-13T18:21:34.350232+0000","last_clean":"2024-09-13T18:21:34.350232+0000","last_became_active":"2024-09-13T17:50:34.211594+0000","last_became_peered":"2024-09-13T17:50:34.211594+0000","last_unstale":"2024-09-13T18:21:34.350232+0000","last_undegraded":"2024-09-13T18:21:34.350232+0000","last_fullsized":"2024-09-13T18:21:34.350232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-13T18:21:34.350185+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-13T18:21:34.350185+0000","last_clean_scrub_stamp":"2024-09-13T18:21:34.350185+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:34:03.648370+0000","scrub_duration":0.003876764,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'5504","reported_seq":5544,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:56.359423+0000","last_change":"2024-09-13T18:21:56.359423+0000","last_active":"2024-09-13T18:21:56.359423+0000","last_peered":"2024-09-13T18:21:56.359423+0000","last_clean":"2024-09-13T18:21:56.359423+0000","last_became_active":"2024-09-13T17:50:38.612577+0000","last_became_peered":"2024-09-13T17:50:38.612577+0000","last_unstale":"2024-09-13T18:21:56.359423+0000","last_undegraded":"2024-09-13T18:21:56.359423+0000","last_fullsized":"2024-09-13T18:21:56.359423+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5504","last_scrub_stamp":"2024-09-13T18:21:56.359197+0000","last_deep_scrub":"27'5504","last_deep_scrub_stamp":"2024-09-13T18:21:56.359197+0000","last_clean_scrub_stamp":"2024-09-13T18:21:56.359197+0000","objects_scrubbed":0,"log_size":5504,"log_dups_size":0,"ondisk_log_size":5504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:28.613391+0000","scrub_duration":0.033662368999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23286,"num_write_kb":502495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'19","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:48.195751+0000","last_change":"2024-09-13T18:21:48.195940+0000","last_active":"2024-09-13T18:21:48.195751+0000","last_peered":"2024-09-13T18:21:48.195751+0000","last_clean":"2024-09-13T18:21:48.195751+0000","last_became_active":"2024-09-13T17:50:34.587027+0000","last_became_peered":"2024-09-13T17:50:34.587027+0000","last_unstale":"2024-09-13T18:21:48.195751+0000","last_undegraded":"2024-09-13T18:21:48.195751+0000","last_fullsized":"2024-09-13T18:21:48.195751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-13T18:21:48.195703+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-13T18:21:48.195703+0000","last_clean_scrub_stamp":"2024-09-13T18:21:48.195703+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:55.755022+0000","scrub_duration":0.0031326050000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'7","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:32.340544+0000","last_change":"2024-09-13T18:21:32.340739+0000","last_active":"2024-09-13T18:21:32.340544+0000","last_peered":"2024-09-13T18:21:32.340544+0000","last_clean":"2024-09-13T18:21:32.340544+0000","last_became_active":"2024-09-13T17:50:34.211568+0000","last_became_peered":"2024-09-13T17:50:34.211568+0000","last_unstale":"2024-09-13T18:21:32.340544+0000","last_undegraded":"2024-09-13T18:21:32.340544+0000","last_fullsized":"2024-09-13T18:21:32.340544+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-13T18:21:32.340496+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-13T18:21:32.340496+0000","last_clean_scrub_stamp":"2024-09-13T18:21:32.340496+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:14:08.632192+0000","scrub_duration":0.0064919820000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'6938","reported_seq":6978,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:01.462926+0000","last_change":"2024-09-13T18:22:01.462926+0000","last_active":"2024-09-13T18:22:01.462926+0000","last_peered":"2024-09-13T18:22:01.462926+0000","last_clean":"2024-09-13T18:22:01.462926+0000","last_became_active":"2024-09-13T17:50:38.609832+0000","last_became_peered":"2024-09-13T17:50:38.609832+0000","last_unstale":"2024-09-13T18:22:01.462926+0000","last_undegraded":"2024-09-13T18:22:01.462926+0000","last_fullsized":"2024-09-13T18:22:01.462926+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6938","last_scrub_stamp":"2024-09-13T18:22:01.462726+0000","last_deep_scrub":"27'6938","last_deep_scrub_stamp":"2024-09-13T18:22:01.462726+0000","last_clean_scrub_stamp":"2024-09-13T18:22:01.462726+0000","objects_scrubbed":0,"log_size":6938,"log_dups_size":0,"ondisk_log_size":6938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:07.849758+0000","scrub_duration":0.053454132000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28644,"num_write_kb":622131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"27'12","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:33.269238+0000","last_change":"2024-09-13T18:21:33.269565+0000","last_active":"2024-09-13T18:21:33.269238+0000","last_peered":"2024-09-13T18:21:33.269238+0000","last_clean":"2024-09-13T18:21:33.269238+0000","last_became_active":"2024-09-13T17:50:34.588794+0000","last_became_peered":"2024-09-13T17:50:34.588794+0000","last_unstale":"2024-09-13T18:21:33.269238+0000","last_undegraded":"2024-09-13T18:21:33.269238+0000","last_fullsized":"2024-09-13T18:21:33.269238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'12","last_scrub_stamp":"2024-09-13T18:21:33.269111+0000","last_deep_scrub":"27'12","last_deep_scrub_stamp":"2024-09-13T18:21:33.269111+0000","last_clean_scrub_stamp":"2024-09-13T18:21:33.269111+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:41.084595+0000","scrub_duration":0.003356562,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'7","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:59.510367+0000","last_change":"2024-09-13T18:21:59.510653+0000","last_active":"2024-09-13T18:21:59.510367+0000","last_peered":"2024-09-13T18:21:59.510367+0000","last_clean":"2024-09-13T18:21:59.510367+0000","last_became_active":"2024-09-13T17:50:34.214292+0000","last_became_peered":"2024-09-13T17:50:34.214292+0000","last_unstale":"2024-09-13T18:21:59.510367+0000","last_undegraded":"2024-09-13T18:21:59.510367+0000","last_fullsized":"2024-09-13T18:21:59.510367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-13T18:21:59.510312+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-13T18:21:59.510312+0000","last_clean_scrub_stamp":"2024-09-13T18:21:59.510312+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:39.341902+0000","scrub_duration":0.0065891600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'4588","reported_seq":4628,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:59.338480+0000","last_change":"2024-09-13T18:21:59.338480+0000","last_active":"2024-09-13T18:21:59.338480+0000","last_peered":"2024-09-13T18:21:59.338480+0000","last_clean":"2024-09-13T18:21:59.338480+0000","last_became_active":"2024-09-13T17:50:38.613010+0000","last_became_peered":"2024-09-13T17:50:38.613010+0000","last_unstale":"2024-09-13T18:21:59.338480+0000","last_undegraded":"2024-09-13T18:21:59.338480+0000","last_fullsized":"2024-09-13T18:21:59.338480+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4588","last_scrub_stamp":"2024-09-13T18:21:59.338108+0000","last_deep_scrub":"27'4588","last_deep_scrub_stamp":"2024-09-13T18:21:59.338108+0000","last_clean_scrub_stamp":"2024-09-13T18:21:59.338108+0000","objects_scrubbed":0,"log_size":4588,"log_dups_size":0,"ondisk_log_size":4588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:34.989321+0000","scrub_duration":0.027635435999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19354,"num_write_kb":454021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'26","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:50.314073+0000","last_change":"2024-09-13T18:21:50.314318+0000","last_active":"2024-09-13T18:21:50.314073+0000","last_peered":"2024-09-13T18:21:50.314073+0000","last_clean":"2024-09-13T18:21:50.314073+0000","last_became_active":"2024-09-13T17:50:34.582050+0000","last_became_peered":"2024-09-13T17:50:34.582050+0000","last_unstale":"2024-09-13T18:21:50.314073+0000","last_undegraded":"2024-09-13T18:21:50.314073+0000","last_fullsized":"2024-09-13T18:21:50.314073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'26","last_scrub_stamp":"2024-09-13T18:21:50.314024+0000","last_deep_scrub":"27'26","last_deep_scrub_stamp":"2024-09-13T18:21:50.314024+0000","last_clean_scrub_stamp":"2024-09-13T18:21:50.314024+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:48:00.837821+0000","scrub_duration":0.0030625299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'11","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:32.305631+0000","last_change":"2024-09-13T18:21:32.306235+0000","last_active":"2024-09-13T18:21:32.305631+0000","last_peered":"2024-09-13T18:21:32.305631+0000","last_clean":"2024-09-13T18:21:32.305631+0000","last_became_active":"2024-09-13T17:50:34.211230+0000","last_became_peered":"2024-09-13T17:50:34.211230+0000","last_unstale":"2024-09-13T18:21:32.305631+0000","last_undegraded":"2024-09-13T18:21:32.305631+0000","last_fullsized":"2024-09-13T18:21:32.305631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-09-13T18:21:32.305579+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-09-13T18:21:32.305579+0000","last_clean_scrub_stamp":"2024-09-13T18:21:32.305579+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:38:08.984532+0000","scrub_duration":0.0066305000000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'6024","reported_seq":6065,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:54.336876+0000","last_change":"2024-09-13T18:21:54.336876+0000","last_active":"2024-09-13T18:21:54.336876+0000","last_peered":"2024-09-13T18:21:54.336876+0000","last_clean":"2024-09-13T18:21:54.336876+0000","last_became_active":"2024-09-13T17:50:38.606697+0000","last_became_peered":"2024-09-13T17:50:38.606697+0000","last_unstale":"2024-09-13T18:21:54.336876+0000","last_undegraded":"2024-09-13T18:21:54.336876+0000","last_fullsized":"2024-09-13T18:21:54.336876+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6024","last_scrub_stamp":"2024-09-13T18:21:54.336672+0000","last_deep_scrub":"27'6024","last_deep_scrub_stamp":"2024-09-13T18:21:54.336672+0000","last_clean_scrub_stamp":"2024-09-13T18:21:54.336672+0000","objects_scrubbed":0,"log_size":6024,"log_dups_size":0,"ondisk_log_size":6024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:56.337299+0000","scrub_duration":0.040808092999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25552,"num_write_kb":507487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'18","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:45.222534+0000","last_change":"2024-09-13T18:21:45.222779+0000","last_active":"2024-09-13T18:21:45.222534+0000","last_peered":"2024-09-13T18:21:45.222534+0000","last_clean":"2024-09-13T18:21:45.222534+0000","last_became_active":"2024-09-13T17:50:34.594215+0000","last_became_peered":"2024-09-13T17:50:34.594215+0000","last_unstale":"2024-09-13T18:21:45.222534+0000","last_undegraded":"2024-09-13T18:21:45.222534+0000","last_fullsized":"2024-09-13T18:21:45.222534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:21:45.222475+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:21:45.222475+0000","last_clean_scrub_stamp":"2024-09-13T18:21:45.222475+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:21.803809+0000","scrub_duration":0.0034298839999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'286","reported_seq":329,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:36.413084+0000","last_change":"2024-09-13T18:21:36.413202+0000","last_active":"2024-09-13T18:21:36.413084+0000","last_peered":"2024-09-13T18:21:36.413084+0000","last_clean":"2024-09-13T18:21:36.413084+0000","last_became_active":"2024-09-13T17:50:34.203702+0000","last_became_peered":"2024-09-13T17:50:34.203702+0000","last_unstale":"2024-09-13T18:21:36.413084+0000","last_undegraded":"2024-09-13T18:21:36.413084+0000","last_fullsized":"2024-09-13T18:21:36.413084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'286","last_scrub_stamp":"2024-09-13T18:21:36.413050+0000","last_deep_scrub":"27'286","last_deep_scrub_stamp":"2024-09-13T18:21:36.413050+0000","last_clean_scrub_stamp":"2024-09-13T18:21:36.413050+0000","objects_scrubbed":2,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:44.535556+0000","scrub_duration":0.0068211640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":605,"num_write_kb":197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'6030","reported_seq":6081,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:13.299511+0000","last_change":"2024-09-13T18:22:13.299635+0000","last_active":"2024-09-13T18:22:13.299511+0000","last_peered":"2024-09-13T18:22:13.299511+0000","last_clean":"2024-09-13T18:22:13.299511+0000","last_became_active":"2024-09-13T17:51:16.364540+0000","last_became_peered":"2024-09-13T17:51:16.364540+0000","last_unstale":"2024-09-13T18:22:13.299511+0000","last_undegraded":"2024-09-13T18:22:13.299511+0000","last_fullsized":"2024-09-13T18:22:13.299511+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6030","last_scrub_stamp":"2024-09-13T18:22:13.299492+0000","last_deep_scrub":"27'6030","last_deep_scrub_stamp":"2024-09-13T18:22:13.299492+0000","last_clean_scrub_stamp":"2024-09-13T18:22:13.299492+0000","objects_scrubbed":0,"log_size":6030,"log_dups_size":0,"ondisk_log_size":6030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:59:05.583204+0000","scrub_duration":0.045983233999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25617,"num_write_kb":548663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,2],"acting":[3,7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'18","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:50.405887+0000","last_change":"2024-09-13T18:21:50.405887+0000","last_active":"2024-09-13T18:21:50.405887+0000","last_peered":"2024-09-13T18:21:50.405887+0000","last_clean":"2024-09-13T18:21:50.405887+0000","last_became_active":"2024-09-13T17:50:34.586552+0000","last_became_peered":"2024-09-13T17:50:34.586552+0000","last_unstale":"2024-09-13T18:21:50.405887+0000","last_undegraded":"2024-09-13T18:21:50.405887+0000","last_fullsized":"2024-09-13T18:21:50.405887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:21:50.405568+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:21:50.405568+0000","last_clean_scrub_stamp":"2024-09-13T18:21:50.405568+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:09.337911+0000","scrub_duration":0.0019550739999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'8","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:57.533976+0000","last_change":"2024-09-13T18:21:57.533976+0000","last_active":"2024-09-13T18:21:57.533976+0000","last_peered":"2024-09-13T18:21:57.533976+0000","last_clean":"2024-09-13T18:21:57.533976+0000","last_became_active":"2024-09-13T17:50:34.216758+0000","last_became_peered":"2024-09-13T17:50:34.216758+0000","last_unstale":"2024-09-13T18:21:57.533976+0000","last_undegraded":"2024-09-13T18:21:57.533976+0000","last_fullsized":"2024-09-13T18:21:57.533976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-09-13T18:21:57.533649+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-09-13T18:21:57.533649+0000","last_clean_scrub_stamp":"2024-09-13T18:21:57.533649+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:14.399317+0000","scrub_duration":0.0047334860000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'4887","reported_seq":4928,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:10.474473+0000","last_change":"2024-09-13T18:22:10.474699+0000","last_active":"2024-09-13T18:22:10.474473+0000","last_peered":"2024-09-13T18:22:10.474473+0000","last_clean":"2024-09-13T18:22:10.474473+0000","last_became_active":"2024-09-13T17:50:38.614284+0000","last_became_peered":"2024-09-13T17:50:38.614284+0000","last_unstale":"2024-09-13T18:22:10.474473+0000","last_undegraded":"2024-09-13T18:22:10.474473+0000","last_fullsized":"2024-09-13T18:22:10.474473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4887","last_scrub_stamp":"2024-09-13T18:22:10.474424+0000","last_deep_scrub":"27'4887","last_deep_scrub_stamp":"2024-09-13T18:22:10.474424+0000","last_clean_scrub_stamp":"2024-09-13T18:22:10.474424+0000","objects_scrubbed":0,"log_size":4887,"log_dups_size":0,"ondisk_log_size":4887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:48.155529+0000","scrub_duration":0.038393351999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20690,"num_write_kb":442963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'16","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:53.757522+0000","last_change":"2024-09-13T18:21:53.757522+0000","last_active":"2024-09-13T18:21:53.757522+0000","last_peered":"2024-09-13T18:21:53.757522+0000","last_clean":"2024-09-13T18:21:53.757522+0000","last_became_active":"2024-09-13T17:50:34.588317+0000","last_became_peered":"2024-09-13T17:50:34.588317+0000","last_unstale":"2024-09-13T18:21:53.757522+0000","last_undegraded":"2024-09-13T18:21:53.757522+0000","last_fullsized":"2024-09-13T18:21:53.757522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-09-13T18:21:53.757275+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-09-13T18:21:53.757275+0000","last_clean_scrub_stamp":"2024-09-13T18:21:53.757275+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:03:27.571838+0000","scrub_duration":0.003575366,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'3","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:49.455036+0000","last_change":"2024-09-13T18:21:49.455036+0000","last_active":"2024-09-13T18:21:49.455036+0000","last_peered":"2024-09-13T18:21:49.455036+0000","last_clean":"2024-09-13T18:21:49.455036+0000","last_became_active":"2024-09-13T17:50:34.204060+0000","last_became_peered":"2024-09-13T17:50:34.204060+0000","last_unstale":"2024-09-13T18:21:49.455036+0000","last_undegraded":"2024-09-13T18:21:49.455036+0000","last_fullsized":"2024-09-13T18:21:49.455036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-09-13T18:21:49.454804+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-09-13T18:21:49.454804+0000","last_clean_scrub_stamp":"2024-09-13T18:21:49.454804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:26.292901+0000","scrub_duration":0.0030762760000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'6031","reported_seq":6072,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:04.425037+0000","last_change":"2024-09-13T18:22:04.425153+0000","last_active":"2024-09-13T18:22:04.425037+0000","last_peered":"2024-09-13T18:22:04.425037+0000","last_clean":"2024-09-13T18:22:04.425037+0000","last_became_active":"2024-09-13T17:50:38.617257+0000","last_became_peered":"2024-09-13T17:50:38.617257+0000","last_unstale":"2024-09-13T18:22:04.425037+0000","last_undegraded":"2024-09-13T18:22:04.425037+0000","last_fullsized":"2024-09-13T18:22:04.425037+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6031","last_scrub_stamp":"2024-09-13T18:22:04.425017+0000","last_deep_scrub":"27'6031","last_deep_scrub_stamp":"2024-09-13T18:22:04.425017+0000","last_clean_scrub_stamp":"2024-09-13T18:22:04.425017+0000","objects_scrubbed":0,"log_size":6031,"log_dups_size":0,"ondisk_log_size":6031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:25:32.439206+0000","scrub_duration":0.045605909,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25529,"num_write_kb":564991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'16","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:45.268600+0000","last_change":"2024-09-13T18:21:45.268600+0000","last_active":"2024-09-13T18:21:45.268600+0000","last_peered":"2024-09-13T18:21:45.268600+0000","last_clean":"2024-09-13T18:21:45.268600+0000","last_became_active":"2024-09-13T17:50:34.588063+0000","last_became_peered":"2024-09-13T17:50:34.588063+0000","last_unstale":"2024-09-13T18:21:45.268600+0000","last_undegraded":"2024-09-13T18:21:45.268600+0000","last_fullsized":"2024-09-13T18:21:45.268600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-09-13T18:21:45.268335+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-09-13T18:21:45.268335+0000","last_clean_scrub_stamp":"2024-09-13T18:21:45.268335+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:13:11.956726+0000","scrub_duration":0.0030026689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'4","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:35.893165+0000","last_change":"2024-09-13T18:21:35.893165+0000","last_active":"2024-09-13T18:21:35.893165+0000","last_peered":"2024-09-13T18:21:35.893165+0000","last_clean":"2024-09-13T18:21:35.893165+0000","last_became_active":"2024-09-13T17:50:34.208196+0000","last_became_peered":"2024-09-13T17:50:34.208196+0000","last_unstale":"2024-09-13T18:21:35.893165+0000","last_undegraded":"2024-09-13T18:21:35.893165+0000","last_fullsized":"2024-09-13T18:21:35.893165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-09-13T18:21:35.892908+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-09-13T18:21:35.892908+0000","last_clean_scrub_stamp":"2024-09-13T18:21:35.892908+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:39:50.582547+0000","scrub_duration":0.0034542710000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'5410","reported_seq":5451,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:00.439430+0000","last_change":"2024-09-13T18:22:00.439430+0000","last_active":"2024-09-13T18:22:00.439430+0000","last_peered":"2024-09-13T18:22:00.439430+0000","last_clean":"2024-09-13T18:22:00.439430+0000","last_became_active":"2024-09-13T17:50:38.609794+0000","last_became_peered":"2024-09-13T17:50:38.609794+0000","last_unstale":"2024-09-13T18:22:00.439430+0000","last_undegraded":"2024-09-13T18:22:00.439430+0000","last_fullsized":"2024-09-13T18:22:00.439430+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5410","last_scrub_stamp":"2024-09-13T18:22:00.439240+0000","last_deep_scrub":"27'5410","last_deep_scrub_stamp":"2024-09-13T18:22:00.439240+0000","last_clean_scrub_stamp":"2024-09-13T18:22:00.439240+0000","objects_scrubbed":0,"log_size":5410,"log_dups_size":0,"ondisk_log_size":5410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:57:58.327181+0000","scrub_duration":0.041838701999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22186,"num_write_kb":506224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'22","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:52.457337+0000","last_change":"2024-09-13T18:21:52.457337+0000","last_active":"2024-09-13T18:21:52.457337+0000","last_peered":"2024-09-13T18:21:52.457337+0000","last_clean":"2024-09-13T18:21:52.457337+0000","last_became_active":"2024-09-13T17:50:34.584023+0000","last_became_peered":"2024-09-13T17:50:34.584023+0000","last_unstale":"2024-09-13T18:21:52.457337+0000","last_undegraded":"2024-09-13T18:21:52.457337+0000","last_fullsized":"2024-09-13T18:21:52.457337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-09-13T18:21:52.457074+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-09-13T18:21:52.457074+0000","last_clean_scrub_stamp":"2024-09-13T18:21:52.457074+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:06:10.138005+0000","scrub_duration":0.0030016969999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'130","reported_seq":295,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:43.721726+0000","last_change":"2024-09-13T18:21:43.721726+0000","last_active":"2024-09-13T18:21:43.721726+0000","last_peered":"2024-09-13T18:21:43.721726+0000","last_clean":"2024-09-13T18:21:43.721726+0000","last_became_active":"2024-09-13T17:50:34.198603+0000","last_became_peered":"2024-09-13T17:50:34.198603+0000","last_unstale":"2024-09-13T18:21:43.721726+0000","last_undegraded":"2024-09-13T18:21:43.721726+0000","last_fullsized":"2024-09-13T18:21:43.721726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'130","last_scrub_stamp":"2024-09-13T18:21:43.721576+0000","last_deep_scrub":"27'130","last_deep_scrub_stamp":"2024-09-13T18:21:43.721576+0000","last_clean_scrub_stamp":"2024-09-13T18:21:43.721576+0000","objects_scrubbed":3,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:31.923344+0000","scrub_duration":0.0076490960000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":94032,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":179,"num_write":125,"num_write_kb":180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'5807","reported_seq":5847,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.539082+0000","last_change":"2024-09-13T18:21:55.539241+0000","last_active":"2024-09-13T18:21:55.539082+0000","last_peered":"2024-09-13T18:21:55.539082+0000","last_clean":"2024-09-13T18:21:55.539082+0000","last_became_active":"2024-09-13T17:50:38.610900+0000","last_became_peered":"2024-09-13T17:50:38.610900+0000","last_unstale":"2024-09-13T18:21:55.539082+0000","last_undegraded":"2024-09-13T18:21:55.539082+0000","last_fullsized":"2024-09-13T18:21:55.539082+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5807","last_scrub_stamp":"2024-09-13T18:21:55.539055+0000","last_deep_scrub":"27'5807","last_deep_scrub_stamp":"2024-09-13T18:21:55.539055+0000","last_clean_scrub_stamp":"2024-09-13T18:21:55.539055+0000","objects_scrubbed":0,"log_size":5807,"log_dups_size":0,"ondisk_log_size":5807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:19.172462+0000","scrub_duration":0.050879777000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24429,"num_write_kb":531605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'13","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:32.284016+0000","last_change":"2024-09-13T18:21:32.284016+0000","last_active":"2024-09-13T18:21:32.284016+0000","last_peered":"2024-09-13T18:21:32.284016+0000","last_clean":"2024-09-13T18:21:32.284016+0000","last_became_active":"2024-09-13T17:50:34.594201+0000","last_became_peered":"2024-09-13T17:50:34.594201+0000","last_unstale":"2024-09-13T18:21:32.284016+0000","last_undegraded":"2024-09-13T18:21:32.284016+0000","last_fullsized":"2024-09-13T18:21:32.284016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'13","last_scrub_stamp":"2024-09-13T18:21:32.283704+0000","last_deep_scrub":"27'13","last_deep_scrub_stamp":"2024-09-13T18:21:32.283704+0000","last_clean_scrub_stamp":"2024-09-13T18:21:32.283704+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:06.348602+0000","scrub_duration":0.0033353039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'9","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:45.449467+0000","last_change":"2024-09-13T18:21:45.449664+0000","last_active":"2024-09-13T18:21:45.449467+0000","last_peered":"2024-09-13T18:21:45.449467+0000","last_clean":"2024-09-13T18:21:45.449467+0000","last_became_active":"2024-09-13T17:50:34.218069+0000","last_became_peered":"2024-09-13T17:50:34.218069+0000","last_unstale":"2024-09-13T18:21:45.449467+0000","last_undegraded":"2024-09-13T18:21:45.449467+0000","last_fullsized":"2024-09-13T18:21:45.449467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-09-13T18:21:45.449415+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-09-13T18:21:45.449415+0000","last_clean_scrub_stamp":"2024-09-13T18:21:45.449415+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:45.223728+0000","scrub_duration":0.0064944169999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'4957","reported_seq":4997,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:08.402578+0000","last_change":"2024-09-13T18:22:08.402774+0000","last_active":"2024-09-13T18:22:08.402578+0000","last_peered":"2024-09-13T18:22:08.402578+0000","last_clean":"2024-09-13T18:22:08.402578+0000","last_became_active":"2024-09-13T17:50:38.612856+0000","last_became_peered":"2024-09-13T17:50:38.612856+0000","last_unstale":"2024-09-13T18:22:08.402578+0000","last_undegraded":"2024-09-13T18:22:08.402578+0000","last_fullsized":"2024-09-13T18:22:08.402578+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4957","last_scrub_stamp":"2024-09-13T18:22:08.402540+0000","last_deep_scrub":"27'4957","last_deep_scrub_stamp":"2024-09-13T18:22:08.402540+0000","last_clean_scrub_stamp":"2024-09-13T18:22:08.402540+0000","objects_scrubbed":0,"log_size":4957,"log_dups_size":0,"ondisk_log_size":4957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:09.900500+0000","scrub_duration":0.033427548000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20520,"num_write_kb":434212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'16","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:43.170888+0000","last_change":"2024-09-13T18:21:43.170888+0000","last_active":"2024-09-13T18:21:43.170888+0000","last_peered":"2024-09-13T18:21:43.170888+0000","last_clean":"2024-09-13T18:21:43.170888+0000","last_became_active":"2024-09-13T17:50:34.582446+0000","last_became_peered":"2024-09-13T17:50:34.582446+0000","last_unstale":"2024-09-13T18:21:43.170888+0000","last_undegraded":"2024-09-13T18:21:43.170888+0000","last_fullsized":"2024-09-13T18:21:43.170888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-09-13T18:21:43.170750+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-09-13T18:21:43.170750+0000","last_clean_scrub_stamp":"2024-09-13T18:21:43.170750+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:39.625283+0000","scrub_duration":0.0027720919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'11","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.797573+0000","last_change":"2024-09-13T18:21:28.797573+0000","last_active":"2024-09-13T18:21:28.797573+0000","last_peered":"2024-09-13T18:21:28.797573+0000","last_clean":"2024-09-13T18:21:28.797573+0000","last_became_active":"2024-09-13T17:50:34.200873+0000","last_became_peered":"2024-09-13T17:50:34.200873+0000","last_unstale":"2024-09-13T18:21:28.797573+0000","last_undegraded":"2024-09-13T18:21:28.797573+0000","last_fullsized":"2024-09-13T18:21:28.797573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-09-13T18:21:28.797316+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-09-13T18:21:28.797316+0000","last_clean_scrub_stamp":"2024-09-13T18:21:28.797316+0000","objects_scrubbed":3,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:51:50.633990+0000","scrub_duration":0.0090957169999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"27'119","reported_seq":200,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:41.367018+0000","last_change":"2024-09-13T18:21:41.367018+0000","last_active":"2024-09-13T18:21:41.367018+0000","last_peered":"2024-09-13T18:21:41.367018+0000","last_clean":"2024-09-13T18:21:41.367018+0000","last_became_active":"2024-09-13T17:50:22.093734+0000","last_became_peered":"2024-09-13T17:50:22.093734+0000","last_unstale":"2024-09-13T18:21:41.367018+0000","last_undegraded":"2024-09-13T18:21:41.367018+0000","last_fullsized":"2024-09-13T18:21:41.367018+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'119","last_scrub_stamp":"2024-09-13T18:21:41.366718+0000","last_deep_scrub":"27'119","last_deep_scrub_stamp":"2024-09-13T18:21:41.366718+0000","last_clean_scrub_stamp":"2024-09-13T18:21:41.366718+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:37:54.724540+0000","scrub_duration":0.0086725589999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'5208","reported_seq":5249,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:52.415372+0000","last_change":"2024-09-13T18:21:52.415372+0000","last_active":"2024-09-13T18:21:52.415372+0000","last_peered":"2024-09-13T18:21:52.415372+0000","last_clean":"2024-09-13T18:21:52.415372+0000","last_became_active":"2024-09-13T17:50:38.618488+0000","last_became_peered":"2024-09-13T17:50:38.618488+0000","last_unstale":"2024-09-13T18:21:52.415372+0000","last_undegraded":"2024-09-13T18:21:52.415372+0000","last_fullsized":"2024-09-13T18:21:52.415372+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5208","last_scrub_stamp":"2024-09-13T18:21:52.415203+0000","last_deep_scrub":"27'5208","last_deep_scrub_stamp":"2024-09-13T18:21:52.415203+0000","last_clean_scrub_stamp":"2024-09-13T18:21:52.415203+0000","objects_scrubbed":0,"log_size":5208,"log_dups_size":0,"ondisk_log_size":5208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:22:21.608219+0000","scrub_duration":0.035964871000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22093,"num_write_kb":481370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'22","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:21.402552+0000","last_change":"2024-09-13T18:22:21.402763+0000","last_active":"2024-09-13T18:22:21.402552+0000","last_peered":"2024-09-13T18:22:21.402552+0000","last_clean":"2024-09-13T18:22:21.402552+0000","last_became_active":"2024-09-13T17:50:34.585166+0000","last_became_peered":"2024-09-13T17:50:34.585166+0000","last_unstale":"2024-09-13T18:22:21.402552+0000","last_undegraded":"2024-09-13T18:22:21.402552+0000","last_fullsized":"2024-09-13T18:22:21.402552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-09-13T18:22:21.402490+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-09-13T18:22:21.402490+0000","last_clean_scrub_stamp":"2024-09-13T18:22:21.402490+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:34:40.735064+0000","scrub_duration":0.0032475910000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'7","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:34.775276+0000","last_change":"2024-09-13T18:21:34.775276+0000","last_active":"2024-09-13T18:21:34.775276+0000","last_peered":"2024-09-13T18:21:34.775276+0000","last_clean":"2024-09-13T18:21:34.775276+0000","last_became_active":"2024-09-13T17:50:34.210589+0000","last_became_peered":"2024-09-13T17:50:34.210589+0000","last_unstale":"2024-09-13T18:21:34.775276+0000","last_undegraded":"2024-09-13T18:21:34.775276+0000","last_fullsized":"2024-09-13T18:21:34.775276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-13T18:21:34.775137+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-13T18:21:34.775137+0000","last_clean_scrub_stamp":"2024-09-13T18:21:34.775137+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:03:40.007622+0000","scrub_duration":0.0064282710000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'5302","reported_seq":5352,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:50.384839+0000","last_change":"2024-09-13T18:21:50.385188+0000","last_active":"2024-09-13T18:21:50.384839+0000","last_peered":"2024-09-13T18:21:50.384839+0000","last_clean":"2024-09-13T18:21:50.384839+0000","last_became_active":"2024-09-13T17:51:16.363974+0000","last_became_peered":"2024-09-13T17:51:16.363974+0000","last_unstale":"2024-09-13T18:21:50.384839+0000","last_undegraded":"2024-09-13T18:21:50.384839+0000","last_fullsized":"2024-09-13T18:21:50.384839+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5302","last_scrub_stamp":"2024-09-13T18:21:50.384756+0000","last_deep_scrub":"27'5302","last_deep_scrub_stamp":"2024-09-13T18:21:50.384756+0000","last_clean_scrub_stamp":"2024-09-13T18:21:50.384756+0000","objects_scrubbed":0,"log_size":5302,"log_dups_size":0,"ondisk_log_size":5302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:19.818627+0000","scrub_duration":0.048673297999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22724,"num_write_kb":488088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'20","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:02.792555+0000","last_change":"2024-09-13T18:22:02.792555+0000","last_active":"2024-09-13T18:22:02.792555+0000","last_peered":"2024-09-13T18:22:02.792555+0000","last_clean":"2024-09-13T18:22:02.792555+0000","last_became_active":"2024-09-13T17:50:34.589434+0000","last_became_peered":"2024-09-13T17:50:34.589434+0000","last_unstale":"2024-09-13T18:22:02.792555+0000","last_undegraded":"2024-09-13T18:22:02.792555+0000","last_fullsized":"2024-09-13T18:22:02.792555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-13T18:22:02.792302+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-13T18:22:02.792302+0000","last_clean_scrub_stamp":"2024-09-13T18:22:02.792302+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:46:31.470983+0000","scrub_duration":0.0033502279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'5","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:33.384001+0000","last_change":"2024-09-13T18:21:33.384001+0000","last_active":"2024-09-13T18:21:33.384001+0000","last_peered":"2024-09-13T18:21:33.384001+0000","last_clean":"2024-09-13T18:21:33.384001+0000","last_became_active":"2024-09-13T17:50:34.203874+0000","last_became_peered":"2024-09-13T17:50:34.203874+0000","last_unstale":"2024-09-13T18:21:33.384001+0000","last_undegraded":"2024-09-13T18:21:33.384001+0000","last_fullsized":"2024-09-13T18:21:33.384001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-09-13T18:21:33.383770+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-09-13T18:21:33.383770+0000","last_clean_scrub_stamp":"2024-09-13T18:21:33.383770+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:07.542936+0000","scrub_duration":0.0056205279999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'5348","reported_seq":5388,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:16.534723+0000","last_change":"2024-09-13T18:22:16.534840+0000","last_active":"2024-09-13T18:22:16.534723+0000","last_peered":"2024-09-13T18:22:16.534723+0000","last_clean":"2024-09-13T18:22:16.534723+0000","last_became_active":"2024-09-13T17:50:38.615530+0000","last_became_peered":"2024-09-13T17:50:38.615530+0000","last_unstale":"2024-09-13T18:22:16.534723+0000","last_undegraded":"2024-09-13T18:22:16.534723+0000","last_fullsized":"2024-09-13T18:22:16.534723+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5348","last_scrub_stamp":"2024-09-13T18:22:16.534702+0000","last_deep_scrub":"27'5348","last_deep_scrub_stamp":"2024-09-13T18:22:16.534702+0000","last_clean_scrub_stamp":"2024-09-13T18:22:16.534702+0000","objects_scrubbed":0,"log_size":5348,"log_dups_size":0,"ondisk_log_size":5348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:50:38.231077+0000","scrub_duration":0.032334993999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22134,"num_write_kb":473293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'20","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:17.369117+0000","last_change":"2024-09-13T18:22:17.369341+0000","last_active":"2024-09-13T18:22:17.369117+0000","last_peered":"2024-09-13T18:22:17.369117+0000","last_clean":"2024-09-13T18:22:17.369117+0000","last_became_active":"2024-09-13T17:50:34.594842+0000","last_became_peered":"2024-09-13T17:50:34.594842+0000","last_unstale":"2024-09-13T18:22:17.369117+0000","last_undegraded":"2024-09-13T18:22:17.369117+0000","last_fullsized":"2024-09-13T18:22:17.369117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-13T18:22:17.369066+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-13T18:22:17.369066+0000","last_clean_scrub_stamp":"2024-09-13T18:22:17.369066+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:18.496921+0000","scrub_duration":0.003511508,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'10","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:42.696280+0000","last_change":"2024-09-13T18:21:42.696280+0000","last_active":"2024-09-13T18:21:42.696280+0000","last_peered":"2024-09-13T18:21:42.696280+0000","last_clean":"2024-09-13T18:21:42.696280+0000","last_became_active":"2024-09-13T17:50:34.199650+0000","last_became_peered":"2024-09-13T17:50:34.199650+0000","last_unstale":"2024-09-13T18:21:42.696280+0000","last_undegraded":"2024-09-13T18:21:42.696280+0000","last_fullsized":"2024-09-13T18:21:42.696280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-13T18:21:42.696034+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-13T18:21:42.696034+0000","last_clean_scrub_stamp":"2024-09-13T18:21:42.696034+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:47:19.246483+0000","scrub_duration":0.0045639139999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'5161","reported_seq":5201,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:05.843990+0000","last_change":"2024-09-13T18:22:05.844240+0000","last_active":"2024-09-13T18:22:05.843990+0000","last_peered":"2024-09-13T18:22:05.843990+0000","last_clean":"2024-09-13T18:22:05.843990+0000","last_became_active":"2024-09-13T17:50:38.609354+0000","last_became_peered":"2024-09-13T17:50:38.609354+0000","last_unstale":"2024-09-13T18:22:05.843990+0000","last_undegraded":"2024-09-13T18:22:05.843990+0000","last_fullsized":"2024-09-13T18:22:05.843990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5161","last_scrub_stamp":"2024-09-13T18:22:05.843925+0000","last_deep_scrub":"27'5161","last_deep_scrub_stamp":"2024-09-13T18:22:05.843925+0000","last_clean_scrub_stamp":"2024-09-13T18:22:05.843925+0000","objects_scrubbed":0,"log_size":5161,"log_dups_size":0,"ondisk_log_size":5161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:32.045019+0000","scrub_duration":0.039900685999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21929,"num_write_kb":464406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'10","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:26.347263+0000","last_change":"2024-09-13T18:21:26.347263+0000","last_active":"2024-09-13T18:21:26.347263+0000","last_peered":"2024-09-13T18:21:26.347263+0000","last_clean":"2024-09-13T18:21:26.347263+0000","last_became_active":"2024-09-13T17:50:34.594876+0000","last_became_peered":"2024-09-13T17:50:34.594876+0000","last_unstale":"2024-09-13T18:21:26.347263+0000","last_undegraded":"2024-09-13T18:21:26.347263+0000","last_fullsized":"2024-09-13T18:21:26.347263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-13T18:21:26.346976+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-13T18:21:26.346976+0000","last_clean_scrub_stamp":"2024-09-13T18:21:26.346976+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:25.039233+0000","scrub_duration":0.003420939,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'6","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.264384+0000","last_change":"2024-09-13T18:21:28.264384+0000","last_active":"2024-09-13T18:21:28.264384+0000","last_peered":"2024-09-13T18:21:28.264384+0000","last_clean":"2024-09-13T18:21:28.264384+0000","last_became_active":"2024-09-13T17:50:34.200197+0000","last_became_peered":"2024-09-13T17:50:34.200197+0000","last_unstale":"2024-09-13T18:21:28.264384+0000","last_undegraded":"2024-09-13T18:21:28.264384+0000","last_fullsized":"2024-09-13T18:21:28.264384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-09-13T18:21:28.264112+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-09-13T18:21:28.264112+0000","last_clean_scrub_stamp":"2024-09-13T18:21:28.264112+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:43:40.034203+0000","scrub_duration":0.0069816829999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'5052","reported_seq":5102,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.356895+0000","last_change":"2024-09-13T18:21:55.357111+0000","last_active":"2024-09-13T18:21:55.356895+0000","last_peered":"2024-09-13T18:21:55.356895+0000","last_clean":"2024-09-13T18:21:55.356895+0000","last_became_active":"2024-09-13T17:51:16.281476+0000","last_became_peered":"2024-09-13T17:51:16.281476+0000","last_unstale":"2024-09-13T18:21:55.356895+0000","last_undegraded":"2024-09-13T18:21:55.356895+0000","last_fullsized":"2024-09-13T18:21:55.356895+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5052","last_scrub_stamp":"2024-09-13T18:21:55.356852+0000","last_deep_scrub":"27'5052","last_deep_scrub_stamp":"2024-09-13T18:21:55.356852+0000","last_clean_scrub_stamp":"2024-09-13T18:21:55.356852+0000","objects_scrubbed":0,"log_size":5052,"log_dups_size":0,"ondisk_log_size":5052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:26:46.603249+0000","scrub_duration":0.044667539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20895,"num_write_kb":464282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3,6],"acting":[4,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'18","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:03.563631+0000","last_change":"2024-09-13T18:22:03.563867+0000","last_active":"2024-09-13T18:22:03.563631+0000","last_peered":"2024-09-13T18:22:03.563631+0000","last_clean":"2024-09-13T18:22:03.563631+0000","last_became_active":"2024-09-13T17:50:34.585464+0000","last_became_peered":"2024-09-13T17:50:34.585464+0000","last_unstale":"2024-09-13T18:22:03.563631+0000","last_undegraded":"2024-09-13T18:22:03.563631+0000","last_fullsized":"2024-09-13T18:22:03.563631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:22:03.563575+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:22:03.563575+0000","last_clean_scrub_stamp":"2024-09-13T18:22:03.563575+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:37.401878+0000","scrub_duration":0.0038625619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'27","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:36.784439+0000","last_change":"2024-09-13T18:21:36.784589+0000","last_active":"2024-09-13T18:21:36.784439+0000","last_peered":"2024-09-13T18:21:36.784439+0000","last_clean":"2024-09-13T18:21:36.784439+0000","last_became_active":"2024-09-13T17:50:34.200936+0000","last_became_peered":"2024-09-13T17:50:34.200936+0000","last_unstale":"2024-09-13T18:21:36.784439+0000","last_undegraded":"2024-09-13T18:21:36.784439+0000","last_fullsized":"2024-09-13T18:21:36.784439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'27","last_scrub_stamp":"2024-09-13T18:21:36.784405+0000","last_deep_scrub":"27'27","last_deep_scrub_stamp":"2024-09-13T18:21:36.784405+0000","last_clean_scrub_stamp":"2024-09-13T18:21:36.784405+0000","objects_scrubbed":8,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:11.675692+0000","scrub_duration":0.012535223,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'6502","reported_seq":6541,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:22.477025+0000","last_change":"2024-09-13T18:22:22.477141+0000","last_active":"2024-09-13T18:22:22.477025+0000","last_peered":"2024-09-13T18:22:22.477025+0000","last_clean":"2024-09-13T18:22:22.477025+0000","last_became_active":"2024-09-13T17:50:38.614739+0000","last_became_peered":"2024-09-13T17:50:38.614739+0000","last_unstale":"2024-09-13T18:22:22.477025+0000","last_undegraded":"2024-09-13T18:22:22.477025+0000","last_fullsized":"2024-09-13T18:22:22.477025+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6502","last_scrub_stamp":"2024-09-13T18:22:22.477005+0000","last_deep_scrub":"27'6502","last_deep_scrub_stamp":"2024-09-13T18:22:22.477005+0000","last_clean_scrub_stamp":"2024-09-13T18:22:22.477005+0000","objects_scrubbed":0,"log_size":6502,"log_dups_size":0,"ondisk_log_size":6502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:37.801904+0000","scrub_duration":0.045513719000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27696,"num_write_kb":544723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'20","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:34.335658+0000","last_change":"2024-09-13T18:21:34.335970+0000","last_active":"2024-09-13T18:21:34.335658+0000","last_peered":"2024-09-13T18:21:34.335658+0000","last_clean":"2024-09-13T18:21:34.335658+0000","last_became_active":"2024-09-13T17:50:34.588565+0000","last_became_peered":"2024-09-13T17:50:34.588565+0000","last_unstale":"2024-09-13T18:21:34.335658+0000","last_undegraded":"2024-09-13T18:21:34.335658+0000","last_fullsized":"2024-09-13T18:21:34.335658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-13T18:21:34.335555+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-13T18:21:34.335555+0000","last_clean_scrub_stamp":"2024-09-13T18:21:34.335555+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:28.955486+0000","scrub_duration":0.0039531770000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'9","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:26.414088+0000","last_change":"2024-09-13T18:21:26.414323+0000","last_active":"2024-09-13T18:21:26.414088+0000","last_peered":"2024-09-13T18:21:26.414088+0000","last_clean":"2024-09-13T18:21:26.414088+0000","last_became_active":"2024-09-13T17:50:34.197839+0000","last_became_peered":"2024-09-13T17:50:34.197839+0000","last_unstale":"2024-09-13T18:21:26.414088+0000","last_undegraded":"2024-09-13T18:21:26.414088+0000","last_fullsized":"2024-09-13T18:21:26.414088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-09-13T18:21:26.414037+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-09-13T18:21:26.414037+0000","last_clean_scrub_stamp":"2024-09-13T18:21:26.414037+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:43:22.483872+0000","scrub_duration":0.0068422659999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'6326","reported_seq":6365,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.238971+0000","last_change":"2024-09-13T18:22:07.239099+0000","last_active":"2024-09-13T18:22:07.238971+0000","last_peered":"2024-09-13T18:22:07.238971+0000","last_clean":"2024-09-13T18:22:07.238971+0000","last_became_active":"2024-09-13T17:50:38.610688+0000","last_became_peered":"2024-09-13T17:50:38.610688+0000","last_unstale":"2024-09-13T18:22:07.238971+0000","last_undegraded":"2024-09-13T18:22:07.238971+0000","last_fullsized":"2024-09-13T18:22:07.238971+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6326","last_scrub_stamp":"2024-09-13T18:22:07.238951+0000","last_deep_scrub":"27'6326","last_deep_scrub_stamp":"2024-09-13T18:22:07.238951+0000","last_clean_scrub_stamp":"2024-09-13T18:22:07.238951+0000","objects_scrubbed":0,"log_size":6326,"log_dups_size":0,"ondisk_log_size":6326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:54.185569+0000","scrub_duration":0.043673352999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27199,"num_write_kb":538670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'29","reported_seq":70,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:07.460377+0000","last_change":"2024-09-13T18:22:07.460610+0000","last_active":"2024-09-13T18:22:07.460377+0000","last_peered":"2024-09-13T18:22:07.460377+0000","last_clean":"2024-09-13T18:22:07.460377+0000","last_became_active":"2024-09-13T17:50:34.594471+0000","last_became_peered":"2024-09-13T17:50:34.594471+0000","last_unstale":"2024-09-13T18:22:07.460377+0000","last_undegraded":"2024-09-13T18:22:07.460377+0000","last_fullsized":"2024-09-13T18:22:07.460377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'29","last_scrub_stamp":"2024-09-13T18:22:07.460318+0000","last_deep_scrub":"27'29","last_deep_scrub_stamp":"2024-09-13T18:22:07.460318+0000","last_clean_scrub_stamp":"2024-09-13T18:22:07.460318+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:23:13.560724+0000","scrub_duration":0.0031528630000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'7","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:37.911671+0000","last_change":"2024-09-13T18:21:37.911888+0000","last_active":"2024-09-13T18:21:37.911671+0000","last_peered":"2024-09-13T18:21:37.911671+0000","last_clean":"2024-09-13T18:21:37.911671+0000","last_became_active":"2024-09-13T17:50:34.208817+0000","last_became_peered":"2024-09-13T17:50:34.208817+0000","last_unstale":"2024-09-13T18:21:37.911671+0000","last_undegraded":"2024-09-13T18:21:37.911671+0000","last_fullsized":"2024-09-13T18:21:37.911671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-13T18:21:37.911585+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-13T18:21:37.911585+0000","last_clean_scrub_stamp":"2024-09-13T18:21:37.911585+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:43:22.483981+0000","scrub_duration":0.0050921259999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'5788","reported_seq":5827,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:57.298804+0000","last_change":"2024-09-13T18:21:57.298969+0000","last_active":"2024-09-13T18:21:57.298804+0000","last_peered":"2024-09-13T18:21:57.298804+0000","last_clean":"2024-09-13T18:21:57.298804+0000","last_became_active":"2024-09-13T17:50:38.609128+0000","last_became_peered":"2024-09-13T17:50:38.609128+0000","last_unstale":"2024-09-13T18:21:57.298804+0000","last_undegraded":"2024-09-13T18:21:57.298804+0000","last_fullsized":"2024-09-13T18:21:57.298804+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5788","last_scrub_stamp":"2024-09-13T18:21:57.298769+0000","last_deep_scrub":"27'5788","last_deep_scrub_stamp":"2024-09-13T18:21:57.298769+0000","last_clean_scrub_stamp":"2024-09-13T18:21:57.298769+0000","objects_scrubbed":0,"log_size":5788,"log_dups_size":0,"ondisk_log_size":5788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:11:22.245296+0000","scrub_duration":0.044309794,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24140,"num_write_kb":538739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'24","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:49.332934+0000","last_change":"2024-09-13T18:21:49.333129+0000","last_active":"2024-09-13T18:21:49.332934+0000","last_peered":"2024-09-13T18:21:49.332934+0000","last_clean":"2024-09-13T18:21:49.332934+0000","last_became_active":"2024-09-13T17:50:34.592669+0000","last_became_peered":"2024-09-13T17:50:34.592669+0000","last_unstale":"2024-09-13T18:21:49.332934+0000","last_undegraded":"2024-09-13T18:21:49.332934+0000","last_fullsized":"2024-09-13T18:21:49.332934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-09-13T18:21:49.332882+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-09-13T18:21:49.332882+0000","last_clean_scrub_stamp":"2024-09-13T18:21:49.332882+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:50.817551+0000","scrub_duration":0.0025807830000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'19","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:28.241499+0000","last_change":"2024-09-13T18:21:28.241791+0000","last_active":"2024-09-13T18:21:28.241499+0000","last_peered":"2024-09-13T18:21:28.241499+0000","last_clean":"2024-09-13T18:21:28.241499+0000","last_became_active":"2024-09-13T17:50:34.200529+0000","last_became_peered":"2024-09-13T17:50:34.200529+0000","last_unstale":"2024-09-13T18:21:28.241499+0000","last_undegraded":"2024-09-13T18:21:28.241499+0000","last_fullsized":"2024-09-13T18:21:28.241499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-13T18:21:28.241430+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-13T18:21:28.241430+0000","last_clean_scrub_stamp":"2024-09-13T18:21:28.241430+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:08:39.927576+0000","scrub_duration":0.00889181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'5464","reported_seq":5503,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:06.898678+0000","last_change":"2024-09-13T18:22:06.898922+0000","last_active":"2024-09-13T18:22:06.898678+0000","last_peered":"2024-09-13T18:22:06.898678+0000","last_clean":"2024-09-13T18:22:06.898678+0000","last_became_active":"2024-09-13T17:50:38.605826+0000","last_became_peered":"2024-09-13T17:50:38.605826+0000","last_unstale":"2024-09-13T18:22:06.898678+0000","last_undegraded":"2024-09-13T18:22:06.898678+0000","last_fullsized":"2024-09-13T18:22:06.898678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5464","last_scrub_stamp":"2024-09-13T18:22:06.898615+0000","last_deep_scrub":"27'5464","last_deep_scrub_stamp":"2024-09-13T18:22:06.898615+0000","last_clean_scrub_stamp":"2024-09-13T18:22:06.898615+0000","objects_scrubbed":0,"log_size":5464,"log_dups_size":0,"ondisk_log_size":5464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:41.161935+0000","scrub_duration":0.051658339999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23044,"num_write_kb":472739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'18","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:27.562535+0000","last_change":"2024-09-13T18:22:27.562782+0000","last_active":"2024-09-13T18:22:27.562535+0000","last_peered":"2024-09-13T18:22:27.562535+0000","last_clean":"2024-09-13T18:22:27.562535+0000","last_became_active":"2024-09-13T17:50:34.586165+0000","last_became_peered":"2024-09-13T17:50:34.586165+0000","last_unstale":"2024-09-13T18:22:27.562535+0000","last_undegraded":"2024-09-13T18:22:27.562535+0000","last_fullsized":"2024-09-13T18:22:27.562535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:22:27.562482+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:22:27.562482+0000","last_clean_scrub_stamp":"2024-09-13T18:22:27.562482+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:57.419109+0000","scrub_duration":0.0033652489999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'421","reported_seq":463,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:31.308661+0000","last_change":"2024-09-13T18:21:31.308816+0000","last_active":"2024-09-13T18:21:31.308661+0000","last_peered":"2024-09-13T18:21:31.308661+0000","last_clean":"2024-09-13T18:21:31.308661+0000","last_became_active":"2024-09-13T17:50:34.202948+0000","last_became_peered":"2024-09-13T17:50:34.202948+0000","last_unstale":"2024-09-13T18:21:31.308661+0000","last_undegraded":"2024-09-13T18:21:31.308661+0000","last_fullsized":"2024-09-13T18:21:31.308661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'421","last_scrub_stamp":"2024-09-13T18:21:31.308629+0000","last_deep_scrub":"27'421","last_deep_scrub_stamp":"2024-09-13T18:21:31.308629+0000","last_clean_scrub_stamp":"2024-09-13T18:21:31.308629+0000","objects_scrubbed":5,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:00:55.983699+0000","scrub_duration":0.024451271,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":416,"num_write_kb":4324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'5887","reported_seq":5926,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:57.523458+0000","last_change":"2024-09-13T18:21:57.523677+0000","last_active":"2024-09-13T18:21:57.523458+0000","last_peered":"2024-09-13T18:21:57.523458+0000","last_clean":"2024-09-13T18:21:57.523458+0000","last_became_active":"2024-09-13T17:50:38.610055+0000","last_became_peered":"2024-09-13T17:50:38.610055+0000","last_unstale":"2024-09-13T18:21:57.523458+0000","last_undegraded":"2024-09-13T18:21:57.523458+0000","last_fullsized":"2024-09-13T18:21:57.523458+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5887","last_scrub_stamp":"2024-09-13T18:21:57.523426+0000","last_deep_scrub":"27'5887","last_deep_scrub_stamp":"2024-09-13T18:21:57.523426+0000","last_clean_scrub_stamp":"2024-09-13T18:21:57.523426+0000","objects_scrubbed":0,"log_size":5887,"log_dups_size":0,"ondisk_log_size":5887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:10:56.767652+0000","scrub_duration":0.047711444999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24664,"num_write_kb":495799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'21","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:39.354348+0000","last_change":"2024-09-13T18:21:39.354570+0000","last_active":"2024-09-13T18:21:39.354348+0000","last_peered":"2024-09-13T18:21:39.354348+0000","last_clean":"2024-09-13T18:21:39.354348+0000","last_became_active":"2024-09-13T17:50:34.593454+0000","last_became_peered":"2024-09-13T17:50:34.593454+0000","last_unstale":"2024-09-13T18:21:39.354348+0000","last_undegraded":"2024-09-13T18:21:39.354348+0000","last_fullsized":"2024-09-13T18:21:39.354348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-09-13T18:21:39.354296+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-09-13T18:21:39.354296+0000","last_clean_scrub_stamp":"2024-09-13T18:21:39.354296+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:10:26.576365+0000","scrub_duration":0.003405356,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'70","reported_seq":112,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:45.487099+0000","last_change":"2024-09-13T18:21:45.487099+0000","last_active":"2024-09-13T18:21:45.487099+0000","last_peered":"2024-09-13T18:21:45.487099+0000","last_clean":"2024-09-13T18:21:45.487099+0000","last_became_active":"2024-09-13T17:50:34.207605+0000","last_became_peered":"2024-09-13T17:50:34.207605+0000","last_unstale":"2024-09-13T18:21:45.487099+0000","last_undegraded":"2024-09-13T18:21:45.487099+0000","last_fullsized":"2024-09-13T18:21:45.487099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'70","last_scrub_stamp":"2024-09-13T18:21:45.486787+0000","last_deep_scrub":"27'70","last_deep_scrub_stamp":"2024-09-13T18:21:45.486787+0000","last_clean_scrub_stamp":"2024-09-13T18:21:45.486787+0000","objects_scrubbed":3,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:25:53.510155+0000","scrub_duration":0.0066096169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'5937","reported_seq":5976,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:52.308856+0000","last_change":"2024-09-13T18:21:52.308991+0000","last_active":"2024-09-13T18:21:52.308856+0000","last_peered":"2024-09-13T18:21:52.308856+0000","last_clean":"2024-09-13T18:21:52.308856+0000","last_became_active":"2024-09-13T17:50:38.614165+0000","last_became_peered":"2024-09-13T17:50:38.614165+0000","last_unstale":"2024-09-13T18:21:52.308856+0000","last_undegraded":"2024-09-13T18:21:52.308856+0000","last_fullsized":"2024-09-13T18:21:52.308856+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5937","last_scrub_stamp":"2024-09-13T18:21:52.308833+0000","last_deep_scrub":"27'5937","last_deep_scrub_stamp":"2024-09-13T18:21:52.308833+0000","last_clean_scrub_stamp":"2024-09-13T18:21:52.308833+0000","objects_scrubbed":0,"log_size":5937,"log_dups_size":0,"ondisk_log_size":5937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:21:00.114058+0000","scrub_duration":0.040795066999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25111,"num_write_kb":509795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'25","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:51.296070+0000","last_change":"2024-09-13T18:21:51.296070+0000","last_active":"2024-09-13T18:21:51.296070+0000","last_peered":"2024-09-13T18:21:51.296070+0000","last_clean":"2024-09-13T18:21:51.296070+0000","last_became_active":"2024-09-13T17:50:34.582529+0000","last_became_peered":"2024-09-13T17:50:34.582529+0000","last_unstale":"2024-09-13T18:21:51.296070+0000","last_undegraded":"2024-09-13T18:21:51.296070+0000","last_fullsized":"2024-09-13T18:21:51.296070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'25","last_scrub_stamp":"2024-09-13T18:21:51.295749+0000","last_deep_scrub":"27'25","last_deep_scrub_stamp":"2024-09-13T18:21:51.295749+0000","last_clean_scrub_stamp":"2024-09-13T18:21:51.295749+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:20:57.681873+0000","scrub_duration":0.0032653209999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'6","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:31.812971+0000","last_change":"2024-09-13T18:21:31.812971+0000","last_active":"2024-09-13T18:21:31.812971+0000","last_peered":"2024-09-13T18:21:31.812971+0000","last_clean":"2024-09-13T18:21:31.812971+0000","last_became_active":"2024-09-13T17:50:34.203360+0000","last_became_peered":"2024-09-13T17:50:34.203360+0000","last_unstale":"2024-09-13T18:21:31.812971+0000","last_undegraded":"2024-09-13T18:21:31.812971+0000","last_fullsized":"2024-09-13T18:21:31.812971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-09-13T18:21:31.812723+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-09-13T18:21:31.812723+0000","last_clean_scrub_stamp":"2024-09-13T18:21:31.812723+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:43:43.954185+0000","scrub_duration":0.0059683940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'5934","reported_seq":5973,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:55.246781+0000","last_change":"2024-09-13T18:21:55.246917+0000","last_active":"2024-09-13T18:21:55.246781+0000","last_peered":"2024-09-13T18:21:55.246781+0000","last_clean":"2024-09-13T18:21:55.246781+0000","last_became_active":"2024-09-13T17:50:38.610692+0000","last_became_peered":"2024-09-13T17:50:38.610692+0000","last_unstale":"2024-09-13T18:21:55.246781+0000","last_undegraded":"2024-09-13T18:21:55.246781+0000","last_fullsized":"2024-09-13T18:21:55.246781+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5934","last_scrub_stamp":"2024-09-13T18:21:55.246755+0000","last_deep_scrub":"27'5934","last_deep_scrub_stamp":"2024-09-13T18:21:55.246755+0000","last_clean_scrub_stamp":"2024-09-13T18:21:55.246755+0000","objects_scrubbed":0,"log_size":5934,"log_dups_size":0,"ondisk_log_size":5934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:01:01.221236+0000","scrub_duration":0.038730878000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25100,"num_write_kb":497535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'23","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:03.699946+0000","last_change":"2024-09-13T18:22:03.699946+0000","last_active":"2024-09-13T18:22:03.699946+0000","last_peered":"2024-09-13T18:22:03.699946+0000","last_clean":"2024-09-13T18:22:03.699946+0000","last_became_active":"2024-09-13T17:51:16.364437+0000","last_became_peered":"2024-09-13T17:51:16.364437+0000","last_unstale":"2024-09-13T18:22:03.699946+0000","last_undegraded":"2024-09-13T18:22:03.699946+0000","last_fullsized":"2024-09-13T18:22:03.699946+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-13T18:22:03.699637+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-13T18:22:03.699637+0000","last_clean_scrub_stamp":"2024-09-13T18:22:03.699637+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:31:41.983071+0000","scrub_duration":0.0035155130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"27'11","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:33.307232+0000","last_change":"2024-09-13T18:21:33.307232+0000","last_active":"2024-09-13T18:21:33.307232+0000","last_peered":"2024-09-13T18:21:33.307232+0000","last_clean":"2024-09-13T18:21:33.307232+0000","last_became_active":"2024-09-13T17:50:34.204003+0000","last_became_peered":"2024-09-13T17:50:34.204003+0000","last_unstale":"2024-09-13T18:21:33.307232+0000","last_undegraded":"2024-09-13T18:21:33.307232+0000","last_fullsized":"2024-09-13T18:21:33.307232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-09-13T18:21:33.306942+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-09-13T18:21:33.306942+0000","last_clean_scrub_stamp":"2024-09-13T18:21:33.306942+0000","objects_scrubbed":2,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:13.945111+0000","scrub_duration":0.0066162260000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'5363","reported_seq":5402,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:12.293263+0000","last_change":"2024-09-13T18:22:12.293476+0000","last_active":"2024-09-13T18:22:12.293263+0000","last_peered":"2024-09-13T18:22:12.293263+0000","last_clean":"2024-09-13T18:22:12.293263+0000","last_became_active":"2024-09-13T17:50:38.605646+0000","last_became_peered":"2024-09-13T17:50:38.605646+0000","last_unstale":"2024-09-13T18:22:12.293263+0000","last_undegraded":"2024-09-13T18:22:12.293263+0000","last_fullsized":"2024-09-13T18:22:12.293263+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5363","last_scrub_stamp":"2024-09-13T18:22:12.293220+0000","last_deep_scrub":"27'5363","last_deep_scrub_stamp":"2024-09-13T18:22:12.293220+0000","last_clean_scrub_stamp":"2024-09-13T18:22:12.293220+0000","objects_scrubbed":0,"log_size":5363,"log_dups_size":0,"ondisk_log_size":5363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:05:13.174210+0000","scrub_duration":0.048844970000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22831,"num_write_kb":508903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'6067","reported_seq":6108,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:23.487620+0000","last_change":"2024-09-13T18:22:23.487764+0000","last_active":"2024-09-13T18:22:23.487620+0000","last_peered":"2024-09-13T18:22:23.487620+0000","last_clean":"2024-09-13T18:22:23.487620+0000","last_became_active":"2024-09-13T17:50:38.618449+0000","last_became_peered":"2024-09-13T17:50:38.618449+0000","last_unstale":"2024-09-13T18:22:23.487620+0000","last_undegraded":"2024-09-13T18:22:23.487620+0000","last_fullsized":"2024-09-13T18:22:23.487620+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6067","last_scrub_stamp":"2024-09-13T18:22:23.487599+0000","last_deep_scrub":"27'6067","last_deep_scrub_stamp":"2024-09-13T18:22:23.487599+0000","last_clean_scrub_stamp":"2024-09-13T18:22:23.487599+0000","objects_scrubbed":0,"log_size":6067,"log_dups_size":0,"ondisk_log_size":6067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:54:20.385420+0000","scrub_duration":0.038402832999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25532,"num_write_kb":568825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'21","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:54.500706+0000","last_change":"2024-09-13T18:21:54.500706+0000","last_active":"2024-09-13T18:21:54.500706+0000","last_peered":"2024-09-13T18:21:54.500706+0000","last_clean":"2024-09-13T18:21:54.500706+0000","last_became_active":"2024-09-13T17:50:34.589396+0000","last_became_peered":"2024-09-13T17:50:34.589396+0000","last_unstale":"2024-09-13T18:21:54.500706+0000","last_undegraded":"2024-09-13T18:21:54.500706+0000","last_fullsized":"2024-09-13T18:21:54.500706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-09-13T18:21:54.500280+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-09-13T18:21:54.500280+0000","last_clean_scrub_stamp":"2024-09-13T18:21:54.500280+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:24.989093+0000","scrub_duration":0.0028702549999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"25'1","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:32.850895+0000","last_change":"2024-09-13T18:21:32.850895+0000","last_active":"2024-09-13T18:21:32.850895+0000","last_peered":"2024-09-13T18:21:32.850895+0000","last_clean":"2024-09-13T18:21:32.850895+0000","last_became_active":"2024-09-13T17:50:34.206308+0000","last_became_peered":"2024-09-13T17:50:34.206308+0000","last_unstale":"2024-09-13T18:21:32.850895+0000","last_undegraded":"2024-09-13T18:21:32.850895+0000","last_fullsized":"2024-09-13T18:21:32.850895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-09-13T18:21:32.850533+0000","last_deep_scrub":"25'1","last_deep_scrub_stamp":"2024-09-13T18:21:32.850533+0000","last_clean_scrub_stamp":"2024-09-13T18:21:32.850533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:59:21.637907+0000","scrub_duration":0.0034298850000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'5881","reported_seq":5922,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:53.439711+0000","last_change":"2024-09-13T18:21:53.439711+0000","last_active":"2024-09-13T18:21:53.439711+0000","last_peered":"2024-09-13T18:21:53.439711+0000","last_clean":"2024-09-13T18:21:53.439711+0000","last_became_active":"2024-09-13T17:50:38.617611+0000","last_became_peered":"2024-09-13T17:50:38.617611+0000","last_unstale":"2024-09-13T18:21:53.439711+0000","last_undegraded":"2024-09-13T18:21:53.439711+0000","last_fullsized":"2024-09-13T18:21:53.439711+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5881","last_scrub_stamp":"2024-09-13T18:21:53.439266+0000","last_deep_scrub":"27'5881","last_deep_scrub_stamp":"2024-09-13T18:21:53.439266+0000","last_clean_scrub_stamp":"2024-09-13T18:21:53.439266+0000","objects_scrubbed":0,"log_size":5881,"log_dups_size":0,"ondisk_log_size":5881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:46:36.260829+0000","scrub_duration":0.043273643000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25019,"num_write_kb":530139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'27","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:34.190222+0000","last_change":"2024-09-13T18:21:34.190222+0000","last_active":"2024-09-13T18:21:34.190222+0000","last_peered":"2024-09-13T18:21:34.190222+0000","last_clean":"2024-09-13T18:21:34.190222+0000","last_became_active":"2024-09-13T17:50:34.581708+0000","last_became_peered":"2024-09-13T17:50:34.581708+0000","last_unstale":"2024-09-13T18:21:34.190222+0000","last_undegraded":"2024-09-13T18:21:34.190222+0000","last_fullsized":"2024-09-13T18:21:34.190222+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'27","last_scrub_stamp":"2024-09-13T18:21:34.189993+0000","last_deep_scrub":"27'27","last_deep_scrub_stamp":"2024-09-13T18:21:34.189993+0000","last_clean_scrub_stamp":"2024-09-13T18:21:34.189993+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:32:35.944129+0000","scrub_duration":0.0029610209999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'5","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:42.211743+0000","last_change":"2024-09-13T18:21:42.211743+0000","last_active":"2024-09-13T18:21:42.211743+0000","last_peered":"2024-09-13T18:21:42.211743+0000","last_clean":"2024-09-13T18:21:42.211743+0000","last_became_active":"2024-09-13T17:50:34.205202+0000","last_became_peered":"2024-09-13T17:50:34.205202+0000","last_unstale":"2024-09-13T18:21:42.211743+0000","last_undegraded":"2024-09-13T18:21:42.211743+0000","last_fullsized":"2024-09-13T18:21:42.211743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-09-13T18:21:42.211500+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-09-13T18:21:42.211500+0000","last_clean_scrub_stamp":"2024-09-13T18:21:42.211500+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:47:56.465375+0000","scrub_duration":0.0062275539999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'7","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:30.324177+0000","last_change":"2024-09-13T18:21:30.324321+0000","last_active":"2024-09-13T18:21:30.324177+0000","last_peered":"2024-09-13T18:21:30.324177+0000","last_clean":"2024-09-13T18:21:30.324177+0000","last_became_active":"2024-09-13T17:50:34.202163+0000","last_became_peered":"2024-09-13T17:50:34.202163+0000","last_unstale":"2024-09-13T18:21:30.324177+0000","last_undegraded":"2024-09-13T18:21:30.324177+0000","last_fullsized":"2024-09-13T18:21:30.324177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-13T18:21:30.324141+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-13T18:21:30.324141+0000","last_clean_scrub_stamp":"2024-09-13T18:21:30.324141+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:27.051230+0000","scrub_duration":0.00672063,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'18","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:45.443346+0000","last_change":"2024-09-13T18:21:45.443452+0000","last_active":"2024-09-13T18:21:45.443346+0000","last_peered":"2024-09-13T18:21:45.443346+0000","last_clean":"2024-09-13T18:21:45.443346+0000","last_became_active":"2024-09-13T17:50:34.590497+0000","last_became_peered":"2024-09-13T17:50:34.590497+0000","last_unstale":"2024-09-13T18:21:45.443346+0000","last_undegraded":"2024-09-13T18:21:45.443346+0000","last_fullsized":"2024-09-13T18:21:45.443346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:21:45.443314+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:21:45.443314+0000","last_clean_scrub_stamp":"2024-09-13T18:21:45.443314+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:29.220122+0000","scrub_duration":0.0029326640000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'4651","reported_seq":4689,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:52.595124+0000","last_change":"2024-09-13T18:21:52.595243+0000","last_active":"2024-09-13T18:21:52.595124+0000","last_peered":"2024-09-13T18:21:52.595124+0000","last_clean":"2024-09-13T18:21:52.595124+0000","last_became_active":"2024-09-13T17:50:38.618108+0000","last_became_peered":"2024-09-13T17:50:38.618108+0000","last_unstale":"2024-09-13T18:21:52.595124+0000","last_undegraded":"2024-09-13T18:21:52.595124+0000","last_fullsized":"2024-09-13T18:21:52.595124+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4651","last_scrub_stamp":"2024-09-13T18:21:52.595101+0000","last_deep_scrub":"27'4651","last_deep_scrub_stamp":"2024-09-13T18:21:52.595101+0000","last_clean_scrub_stamp":"2024-09-13T18:21:52.595101+0000","objects_scrubbed":0,"log_size":4651,"log_dups_size":0,"ondisk_log_size":4651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:15:47.184237+0000","scrub_duration":0.047927588,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19488,"num_write_kb":413684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'692","reported_seq":732,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:38.185928+0000","last_change":"2024-09-13T18:21:38.186111+0000","last_active":"2024-09-13T18:21:38.185928+0000","last_peered":"2024-09-13T18:21:38.185928+0000","last_clean":"2024-09-13T18:21:38.185928+0000","last_became_active":"2024-09-13T17:50:34.218087+0000","last_became_peered":"2024-09-13T17:50:34.218087+0000","last_unstale":"2024-09-13T18:21:38.185928+0000","last_undegraded":"2024-09-13T18:21:38.185928+0000","last_fullsized":"2024-09-13T18:21:38.185928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'692","last_scrub_stamp":"2024-09-13T18:21:38.185889+0000","last_deep_scrub":"27'692","last_deep_scrub_stamp":"2024-09-13T18:21:38.185889+0000","last_clean_scrub_stamp":"2024-09-13T18:21:38.185889+0000","objects_scrubbed":3,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:16:47.594855+0000","scrub_duration":0.021444375000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195461,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":4320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'20","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:40.309829+0000","last_change":"2024-09-13T18:21:40.310070+0000","last_active":"2024-09-13T18:21:40.309829+0000","last_peered":"2024-09-13T18:21:40.309829+0000","last_clean":"2024-09-13T18:21:40.309829+0000","last_became_active":"2024-09-13T17:50:34.583283+0000","last_became_peered":"2024-09-13T17:50:34.583283+0000","last_unstale":"2024-09-13T18:21:40.309829+0000","last_undegraded":"2024-09-13T18:21:40.309829+0000","last_fullsized":"2024-09-13T18:21:40.309829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-13T18:21:40.309774+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-13T18:21:40.309774+0000","last_clean_scrub_stamp":"2024-09-13T18:21:40.309774+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:45:15.638497+0000","scrub_duration":0.0033293300000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'5883","reported_seq":5921,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:09.261029+0000","last_change":"2024-09-13T18:22:09.261212+0000","last_active":"2024-09-13T18:22:09.261029+0000","last_peered":"2024-09-13T18:22:09.261029+0000","last_clean":"2024-09-13T18:22:09.261029+0000","last_became_active":"2024-09-13T17:50:38.611636+0000","last_became_peered":"2024-09-13T17:50:38.611636+0000","last_unstale":"2024-09-13T18:22:09.261029+0000","last_undegraded":"2024-09-13T18:22:09.261029+0000","last_fullsized":"2024-09-13T18:22:09.261029+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5883","last_scrub_stamp":"2024-09-13T18:22:09.260981+0000","last_deep_scrub":"27'5883","last_deep_scrub_stamp":"2024-09-13T18:22:09.260981+0000","last_clean_scrub_stamp":"2024-09-13T18:22:09.260981+0000","objects_scrubbed":0,"log_size":5883,"log_dups_size":0,"ondisk_log_size":5883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:12:31.012514+0000","scrub_duration":0.049256573999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24816,"num_write_kb":545450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'25","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:46.374486+0000","last_change":"2024-09-13T18:21:46.374486+0000","last_active":"2024-09-13T18:21:46.374486+0000","last_peered":"2024-09-13T18:21:46.374486+0000","last_clean":"2024-09-13T18:21:46.374486+0000","last_became_active":"2024-09-13T17:50:34.588651+0000","last_became_peered":"2024-09-13T17:50:34.588651+0000","last_unstale":"2024-09-13T18:21:46.374486+0000","last_undegraded":"2024-09-13T18:21:46.374486+0000","last_fullsized":"2024-09-13T18:21:46.374486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'25","last_scrub_stamp":"2024-09-13T18:21:46.374221+0000","last_deep_scrub":"27'25","last_deep_scrub_stamp":"2024-09-13T18:21:46.374221+0000","last_clean_scrub_stamp":"2024-09-13T18:21:46.374221+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:17:58.490650+0000","scrub_duration":0.003300449,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'24","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:45.720155+0000","last_change":"2024-09-13T18:21:45.720155+0000","last_active":"2024-09-13T18:21:45.720155+0000","last_peered":"2024-09-13T18:21:45.720155+0000","last_clean":"2024-09-13T18:21:45.720155+0000","last_became_active":"2024-09-13T17:50:34.583493+0000","last_became_peered":"2024-09-13T17:50:34.583493+0000","last_unstale":"2024-09-13T18:21:45.720155+0000","last_undegraded":"2024-09-13T18:21:45.720155+0000","last_fullsized":"2024-09-13T18:21:45.720155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-09-13T18:21:45.719919+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-09-13T18:21:45.719919+0000","last_clean_scrub_stamp":"2024-09-13T18:21:45.719919+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:15.218921+0000","scrub_duration":0.0030666199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'25","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:50.830134+0000","last_change":"2024-09-13T18:21:50.830134+0000","last_active":"2024-09-13T18:21:50.830134+0000","last_peered":"2024-09-13T18:21:50.830134+0000","last_clean":"2024-09-13T18:21:50.830134+0000","last_became_active":"2024-09-13T17:50:34.587119+0000","last_became_peered":"2024-09-13T17:50:34.587119+0000","last_unstale":"2024-09-13T18:21:50.830134+0000","last_undegraded":"2024-09-13T18:21:50.830134+0000","last_fullsized":"2024-09-13T18:21:50.830134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'25","last_scrub_stamp":"2024-09-13T18:21:50.829746+0000","last_deep_scrub":"27'25","last_deep_scrub_stamp":"2024-09-13T18:21:50.829746+0000","last_clean_scrub_stamp":"2024-09-13T18:21:50.829746+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:38:15.448330+0000","scrub_duration":0.0031738930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'17","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:57.236877+0000","last_change":"2024-09-13T18:21:57.236877+0000","last_active":"2024-09-13T18:21:57.236877+0000","last_peered":"2024-09-13T18:21:57.236877+0000","last_clean":"2024-09-13T18:21:57.236877+0000","last_became_active":"2024-09-13T17:50:34.581628+0000","last_became_peered":"2024-09-13T17:50:34.581628+0000","last_unstale":"2024-09-13T18:21:57.236877+0000","last_undegraded":"2024-09-13T18:21:57.236877+0000","last_fullsized":"2024-09-13T18:21:57.236877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'17","last_scrub_stamp":"2024-09-13T18:21:57.236610+0000","last_deep_scrub":"27'17","last_deep_scrub_stamp":"2024-09-13T18:21:57.236610+0000","last_clean_scrub_stamp":"2024-09-13T18:21:57.236610+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:24:24.087638+0000","scrub_duration":0.0024405049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'14","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:53.191673+0000","last_change":"2024-09-13T18:21:53.191956+0000","last_active":"2024-09-13T18:21:53.191673+0000","last_peered":"2024-09-13T18:21:53.191673+0000","last_clean":"2024-09-13T18:21:53.191673+0000","last_became_active":"2024-09-13T17:50:34.594857+0000","last_became_peered":"2024-09-13T17:50:34.594857+0000","last_unstale":"2024-09-13T18:21:53.191673+0000","last_undegraded":"2024-09-13T18:21:53.191673+0000","last_fullsized":"2024-09-13T18:21:53.191673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-09-13T18:21:53.191615+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-09-13T18:21:53.191615+0000","last_clean_scrub_stamp":"2024-09-13T18:21:53.191615+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:35:45.001540+0000","scrub_duration":0.0042100100000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'15","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:53.457198+0000","last_change":"2024-09-13T18:21:53.457331+0000","last_active":"2024-09-13T18:21:53.457198+0000","last_peered":"2024-09-13T18:21:53.457198+0000","last_clean":"2024-09-13T18:21:53.457198+0000","last_became_active":"2024-09-13T17:50:34.589354+0000","last_became_peered":"2024-09-13T17:50:34.589354+0000","last_unstale":"2024-09-13T18:21:53.457198+0000","last_undegraded":"2024-09-13T18:21:53.457198+0000","last_fullsized":"2024-09-13T18:21:53.457198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-09-13T18:21:53.457163+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-09-13T18:21:53.457163+0000","last_clean_scrub_stamp":"2024-09-13T18:21:53.457163+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:15:40.638651+0000","scrub_duration":0.0022509019999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'24","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:38.403817+0000","last_change":"2024-09-13T18:21:38.404032+0000","last_active":"2024-09-13T18:21:38.403817+0000","last_peered":"2024-09-13T18:21:38.403817+0000","last_clean":"2024-09-13T18:21:38.403817+0000","last_became_active":"2024-09-13T17:50:34.597621+0000","last_became_peered":"2024-09-13T17:50:34.597621+0000","last_unstale":"2024-09-13T18:21:38.403817+0000","last_undegraded":"2024-09-13T18:21:38.403817+0000","last_fullsized":"2024-09-13T18:21:38.403817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-09-13T18:21:38.403767+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-09-13T18:21:38.403767+0000","last_clean_scrub_stamp":"2024-09-13T18:21:38.403767+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:33:01.593056+0000","scrub_duration":0.0041273439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'15","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:32.155724+0000","last_change":"2024-09-13T18:21:32.155926+0000","last_active":"2024-09-13T18:21:32.155724+0000","last_peered":"2024-09-13T18:21:32.155724+0000","last_clean":"2024-09-13T18:21:32.155724+0000","last_became_active":"2024-09-13T17:50:34.589626+0000","last_became_peered":"2024-09-13T17:50:34.589626+0000","last_unstale":"2024-09-13T18:21:32.155724+0000","last_undegraded":"2024-09-13T18:21:32.155724+0000","last_fullsized":"2024-09-13T18:21:32.155724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-09-13T18:21:32.155676+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-09-13T18:21:32.155676+0000","last_clean_scrub_stamp":"2024-09-13T18:21:32.155676+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:03:35.333502+0000","scrub_duration":0.0028998560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'16","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:44.880892+0000","last_change":"2024-09-13T18:21:44.881098+0000","last_active":"2024-09-13T18:21:44.880892+0000","last_peered":"2024-09-13T18:21:44.880892+0000","last_clean":"2024-09-13T18:21:44.880892+0000","last_became_active":"2024-09-13T17:50:34.589757+0000","last_became_peered":"2024-09-13T17:50:34.589757+0000","last_unstale":"2024-09-13T18:21:44.880892+0000","last_undegraded":"2024-09-13T18:21:44.880892+0000","last_fullsized":"2024-09-13T18:21:44.880892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-09-13T18:21:44.880841+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-09-13T18:21:44.880841+0000","last_clean_scrub_stamp":"2024-09-13T18:21:44.880841+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:12:15.859928+0000","scrub_duration":0.0034118379999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"27'23","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:27.130283+0000","last_change":"2024-09-13T18:21:27.130283+0000","last_active":"2024-09-13T18:21:27.130283+0000","last_peered":"2024-09-13T18:21:27.130283+0000","last_clean":"2024-09-13T18:21:27.130283+0000","last_became_active":"2024-09-13T17:50:34.595242+0000","last_became_peered":"2024-09-13T17:50:34.595242+0000","last_unstale":"2024-09-13T18:21:27.130283+0000","last_undegraded":"2024-09-13T18:21:27.130283+0000","last_fullsized":"2024-09-13T18:21:27.130283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-13T18:21:27.130025+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-13T18:21:27.130025+0000","last_clean_scrub_stamp":"2024-09-13T18:21:27.130025+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:52:06.896321+0000","scrub_duration":0.003821638,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'21","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:54.798385+0000","last_change":"2024-09-13T18:21:54.798385+0000","last_active":"2024-09-13T18:21:54.798385+0000","last_peered":"2024-09-13T18:21:54.798385+0000","last_clean":"2024-09-13T18:21:54.798385+0000","last_became_active":"2024-09-13T17:50:34.592893+0000","last_became_peered":"2024-09-13T17:50:34.592893+0000","last_unstale":"2024-09-13T18:21:54.798385+0000","last_undegraded":"2024-09-13T18:21:54.798385+0000","last_fullsized":"2024-09-13T18:21:54.798385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-09-13T18:21:54.798131+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-09-13T18:21:54.798131+0000","last_clean_scrub_stamp":"2024-09-13T18:21:54.798131+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:27:46.038569+0000","scrub_duration":0.0031872860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'29","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:02.275861+0000","last_change":"2024-09-13T18:22:02.275861+0000","last_active":"2024-09-13T18:22:02.275861+0000","last_peered":"2024-09-13T18:22:02.275861+0000","last_clean":"2024-09-13T18:22:02.275861+0000","last_became_active":"2024-09-13T17:50:34.593750+0000","last_became_peered":"2024-09-13T17:50:34.593750+0000","last_unstale":"2024-09-13T18:22:02.275861+0000","last_undegraded":"2024-09-13T18:22:02.275861+0000","last_fullsized":"2024-09-13T18:22:02.275861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'29","last_scrub_stamp":"2024-09-13T18:22:02.275708+0000","last_deep_scrub":"27'29","last_deep_scrub_stamp":"2024-09-13T18:22:02.275708+0000","last_clean_scrub_stamp":"2024-09-13T18:22:02.275708+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:29:04.409943+0000","scrub_duration":0.0028268960000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'19","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:37.375333+0000","last_change":"2024-09-13T18:21:37.375333+0000","last_active":"2024-09-13T18:21:37.375333+0000","last_peered":"2024-09-13T18:21:37.375333+0000","last_clean":"2024-09-13T18:21:37.375333+0000","last_became_active":"2024-09-13T17:50:34.598433+0000","last_became_peered":"2024-09-13T17:50:34.598433+0000","last_unstale":"2024-09-13T18:21:37.375333+0000","last_undegraded":"2024-09-13T18:21:37.375333+0000","last_fullsized":"2024-09-13T18:21:37.375333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-13T18:21:37.375056+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-13T18:21:37.375056+0000","last_clean_scrub_stamp":"2024-09-13T18:21:37.375056+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:38:02.999621+0000","scrub_duration":0.003734036,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'22","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:36.325615+0000","last_change":"2024-09-13T18:21:36.325615+0000","last_active":"2024-09-13T18:21:36.325615+0000","last_peered":"2024-09-13T18:21:36.325615+0000","last_clean":"2024-09-13T18:21:36.325615+0000","last_became_active":"2024-09-13T17:50:34.591286+0000","last_became_peered":"2024-09-13T17:50:34.591286+0000","last_unstale":"2024-09-13T18:21:36.325615+0000","last_undegraded":"2024-09-13T18:21:36.325615+0000","last_fullsized":"2024-09-13T18:21:36.325615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-09-13T18:21:36.325355+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-09-13T18:21:36.325355+0000","last_clean_scrub_stamp":"2024-09-13T18:21:36.325355+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:12:08.676171+0000","scrub_duration":0.0029670930000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'28","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:44.767452+0000","last_change":"2024-09-13T18:21:44.767671+0000","last_active":"2024-09-13T18:21:44.767452+0000","last_peered":"2024-09-13T18:21:44.767452+0000","last_clean":"2024-09-13T18:21:44.767452+0000","last_became_active":"2024-09-13T17:50:34.587506+0000","last_became_peered":"2024-09-13T17:50:34.587506+0000","last_unstale":"2024-09-13T18:21:44.767452+0000","last_undegraded":"2024-09-13T18:21:44.767452+0000","last_fullsized":"2024-09-13T18:21:44.767452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'28","last_scrub_stamp":"2024-09-13T18:21:44.767402+0000","last_deep_scrub":"27'28","last_deep_scrub_stamp":"2024-09-13T18:21:44.767402+0000","last_clean_scrub_stamp":"2024-09-13T18:21:44.767402+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:28:00.460114+0000","scrub_duration":0.003174128,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'22","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:47.811825+0000","last_change":"2024-09-13T18:21:47.812043+0000","last_active":"2024-09-13T18:21:47.811825+0000","last_peered":"2024-09-13T18:21:47.811825+0000","last_clean":"2024-09-13T18:21:47.811825+0000","last_became_active":"2024-09-13T17:50:34.595798+0000","last_became_peered":"2024-09-13T17:50:34.595798+0000","last_unstale":"2024-09-13T18:21:47.811825+0000","last_undegraded":"2024-09-13T18:21:47.811825+0000","last_fullsized":"2024-09-13T18:21:47.811825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-09-13T18:21:47.811769+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-09-13T18:21:47.811769+0000","last_clean_scrub_stamp":"2024-09-13T18:21:47.811769+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:56.620401+0000","scrub_duration":0.0030899819999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'21","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:25.753731+0000","last_change":"2024-09-13T18:21:25.753960+0000","last_active":"2024-09-13T18:21:25.753731+0000","last_peered":"2024-09-13T18:21:25.753731+0000","last_clean":"2024-09-13T18:21:25.753731+0000","last_became_active":"2024-09-13T17:50:34.584516+0000","last_became_peered":"2024-09-13T17:50:34.584516+0000","last_unstale":"2024-09-13T18:21:25.753731+0000","last_undegraded":"2024-09-13T18:21:25.753731+0000","last_fullsized":"2024-09-13T18:21:25.753731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-09-13T18:21:25.753680+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-09-13T18:21:25.753680+0000","last_clean_scrub_stamp":"2024-09-13T18:21:25.753680+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:59.108188+0000","scrub_duration":0.0032900099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'23","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:51.445915+0000","last_change":"2024-09-13T18:21:51.446115+0000","last_active":"2024-09-13T18:21:51.445915+0000","last_peered":"2024-09-13T18:21:51.445915+0000","last_clean":"2024-09-13T18:21:51.445915+0000","last_became_active":"2024-09-13T17:50:34.598648+0000","last_became_peered":"2024-09-13T17:50:34.598648+0000","last_unstale":"2024-09-13T18:21:51.445915+0000","last_undegraded":"2024-09-13T18:21:51.445915+0000","last_fullsized":"2024-09-13T18:21:51.445915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-13T18:21:51.445846+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-13T18:21:51.445846+0000","last_clean_scrub_stamp":"2024-09-13T18:21:51.445846+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:36:09.890274+0000","scrub_duration":0.003395239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'24","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:23.200378+0000","last_change":"2024-09-13T18:21:23.200494+0000","last_active":"2024-09-13T18:21:23.200378+0000","last_peered":"2024-09-13T18:21:23.200378+0000","last_clean":"2024-09-13T18:21:23.200378+0000","last_became_active":"2024-09-13T17:50:34.581699+0000","last_became_peered":"2024-09-13T17:50:34.581699+0000","last_unstale":"2024-09-13T18:21:23.200378+0000","last_undegraded":"2024-09-13T18:21:23.200378+0000","last_fullsized":"2024-09-13T18:21:23.200378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-09-13T18:21:23.200343+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-09-13T18:21:23.200343+0000","last_clean_scrub_stamp":"2024-09-13T18:21:23.200343+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:37:50.717684+0000","scrub_duration":0.002562838,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'23","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:25.110698+0000","last_change":"2024-09-13T18:21:25.110698+0000","last_active":"2024-09-13T18:21:25.110698+0000","last_peered":"2024-09-13T18:21:25.110698+0000","last_clean":"2024-09-13T18:21:25.110698+0000","last_became_active":"2024-09-13T17:50:34.587217+0000","last_became_peered":"2024-09-13T17:50:34.587217+0000","last_unstale":"2024-09-13T18:21:25.110698+0000","last_undegraded":"2024-09-13T18:21:25.110698+0000","last_fullsized":"2024-09-13T18:21:25.110698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-13T18:21:25.110401+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-13T18:21:25.110401+0000","last_clean_scrub_stamp":"2024-09-13T18:21:25.110401+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:08.271399+0000","scrub_duration":0.0034179560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'18","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:47.310243+0000","last_change":"2024-09-13T18:21:47.310243+0000","last_active":"2024-09-13T18:21:47.310243+0000","last_peered":"2024-09-13T18:21:47.310243+0000","last_clean":"2024-09-13T18:21:47.310243+0000","last_became_active":"2024-09-13T17:50:34.588513+0000","last_became_peered":"2024-09-13T17:50:34.588513+0000","last_unstale":"2024-09-13T18:21:47.310243+0000","last_undegraded":"2024-09-13T18:21:47.310243+0000","last_fullsized":"2024-09-13T18:21:47.310243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:21:47.309947+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:21:47.309947+0000","last_clean_scrub_stamp":"2024-09-13T18:21:47.309947+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:52:22.085054+0000","scrub_duration":0.003219857,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'19","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:40.418512+0000","last_change":"2024-09-13T18:21:40.418512+0000","last_active":"2024-09-13T18:21:40.418512+0000","last_peered":"2024-09-13T18:21:40.418512+0000","last_clean":"2024-09-13T18:21:40.418512+0000","last_became_active":"2024-09-13T17:50:34.585438+0000","last_became_peered":"2024-09-13T17:50:34.585438+0000","last_unstale":"2024-09-13T18:21:40.418512+0000","last_undegraded":"2024-09-13T18:21:40.418512+0000","last_fullsized":"2024-09-13T18:21:40.418512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-13T18:21:40.418258+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-13T18:21:40.418258+0000","last_clean_scrub_stamp":"2024-09-13T18:21:40.418258+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:07:52.939370+0000","scrub_duration":0.0033203519999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'30","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:48.381967+0000","last_change":"2024-09-13T18:21:48.381967+0000","last_active":"2024-09-13T18:21:48.381967+0000","last_peered":"2024-09-13T18:21:48.381967+0000","last_clean":"2024-09-13T18:21:48.381967+0000","last_became_active":"2024-09-13T17:50:34.587665+0000","last_became_peered":"2024-09-13T17:50:34.587665+0000","last_unstale":"2024-09-13T18:21:48.381967+0000","last_undegraded":"2024-09-13T18:21:48.381967+0000","last_fullsized":"2024-09-13T18:21:48.381967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'30","last_scrub_stamp":"2024-09-13T18:21:48.381520+0000","last_deep_scrub":"27'30","last_deep_scrub_stamp":"2024-09-13T18:21:48.381520+0000","last_clean_scrub_stamp":"2024-09-13T18:21:48.381520+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:15:59.624808+0000","scrub_duration":0.003678331,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'27","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:26.597934+0000","last_change":"2024-09-13T18:22:26.598175+0000","last_active":"2024-09-13T18:22:26.597934+0000","last_peered":"2024-09-13T18:22:26.597934+0000","last_clean":"2024-09-13T18:22:26.597934+0000","last_became_active":"2024-09-13T17:50:34.591496+0000","last_became_peered":"2024-09-13T17:50:34.591496+0000","last_unstale":"2024-09-13T18:22:26.597934+0000","last_undegraded":"2024-09-13T18:22:26.597934+0000","last_fullsized":"2024-09-13T18:22:26.597934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'27","last_scrub_stamp":"2024-09-13T18:22:26.597839+0000","last_deep_scrub":"27'27","last_deep_scrub_stamp":"2024-09-13T18:22:26.597839+0000","last_clean_scrub_stamp":"2024-09-13T18:22:26.597839+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:08:24.249154+0000","scrub_duration":0.003380386,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'15","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:59.687264+0000","last_change":"2024-09-13T18:21:59.687484+0000","last_active":"2024-09-13T18:21:59.687264+0000","last_peered":"2024-09-13T18:21:59.687264+0000","last_clean":"2024-09-13T18:21:59.687264+0000","last_became_active":"2024-09-13T17:51:16.282065+0000","last_became_peered":"2024-09-13T17:51:16.282065+0000","last_unstale":"2024-09-13T18:21:59.687264+0000","last_undegraded":"2024-09-13T18:21:59.687264+0000","last_fullsized":"2024-09-13T18:21:59.687264+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-09-13T18:21:59.687218+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-09-13T18:21:59.687218+0000","last_clean_scrub_stamp":"2024-09-13T18:21:59.687218+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:33:21.535758+0000","scrub_duration":0.003695977,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"27'14","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:18.371845+0000","last_change":"2024-09-13T18:22:18.372074+0000","last_active":"2024-09-13T18:22:18.371845+0000","last_peered":"2024-09-13T18:22:18.371845+0000","last_clean":"2024-09-13T18:22:18.371845+0000","last_became_active":"2024-09-13T17:50:34.585226+0000","last_became_peered":"2024-09-13T17:50:34.585226+0000","last_unstale":"2024-09-13T18:22:18.371845+0000","last_undegraded":"2024-09-13T18:22:18.371845+0000","last_fullsized":"2024-09-13T18:22:18.371845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-09-13T18:22:18.371790+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-09-13T18:22:18.371790+0000","last_clean_scrub_stamp":"2024-09-13T18:22:18.371790+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:16:59.331860+0000","scrub_duration":0.003510441,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'17","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:59.828261+0000","last_change":"2024-09-13T18:21:59.828532+0000","last_active":"2024-09-13T18:21:59.828261+0000","last_peered":"2024-09-13T18:21:59.828261+0000","last_clean":"2024-09-13T18:21:59.828261+0000","last_became_active":"2024-09-13T17:50:34.584564+0000","last_became_peered":"2024-09-13T17:50:34.584564+0000","last_unstale":"2024-09-13T18:21:59.828261+0000","last_undegraded":"2024-09-13T18:21:59.828261+0000","last_fullsized":"2024-09-13T18:21:59.828261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'17","last_scrub_stamp":"2024-09-13T18:21:59.828209+0000","last_deep_scrub":"27'17","last_deep_scrub_stamp":"2024-09-13T18:21:59.828209+0000","last_clean_scrub_stamp":"2024-09-13T18:21:59.828209+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:49:08.538007+0000","scrub_duration":0.0035508250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'18","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:23.486183+0000","last_change":"2024-09-13T18:22:23.486294+0000","last_active":"2024-09-13T18:22:23.486183+0000","last_peered":"2024-09-13T18:22:23.486183+0000","last_clean":"2024-09-13T18:22:23.486183+0000","last_became_active":"2024-09-13T17:50:34.585584+0000","last_became_peered":"2024-09-13T17:50:34.585584+0000","last_unstale":"2024-09-13T18:22:23.486183+0000","last_undegraded":"2024-09-13T18:22:23.486183+0000","last_fullsized":"2024-09-13T18:22:23.486183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:22:23.486161+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:22:23.486161+0000","last_clean_scrub_stamp":"2024-09-13T18:22:23.486161+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:51:24.612150+0000","scrub_duration":0.0024568749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'15","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:03.809580+0000","last_change":"2024-09-13T18:22:03.809580+0000","last_active":"2024-09-13T18:22:03.809580+0000","last_peered":"2024-09-13T18:22:03.809580+0000","last_clean":"2024-09-13T18:22:03.809580+0000","last_became_active":"2024-09-13T17:50:34.588489+0000","last_became_peered":"2024-09-13T17:50:34.588489+0000","last_unstale":"2024-09-13T18:22:03.809580+0000","last_undegraded":"2024-09-13T18:22:03.809580+0000","last_fullsized":"2024-09-13T18:22:03.809580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-09-13T18:22:03.809309+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-09-13T18:22:03.809309+0000","last_clean_scrub_stamp":"2024-09-13T18:22:03.809309+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:57:27.506887+0000","scrub_duration":0.0032327089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'18","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:05.304224+0000","last_change":"2024-09-13T18:22:05.304413+0000","last_active":"2024-09-13T18:22:05.304224+0000","last_peered":"2024-09-13T18:22:05.304224+0000","last_clean":"2024-09-13T18:22:05.304224+0000","last_became_active":"2024-09-13T17:50:34.583551+0000","last_became_peered":"2024-09-13T17:50:34.583551+0000","last_unstale":"2024-09-13T18:22:05.304224+0000","last_undegraded":"2024-09-13T18:22:05.304224+0000","last_fullsized":"2024-09-13T18:22:05.304224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-13T18:22:05.304174+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-13T18:22:05.304174+0000","last_clean_scrub_stamp":"2024-09-13T18:22:05.304174+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:14:04.897154+0000","scrub_duration":0.0024008039999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'10","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:21:29.338126+0000","last_change":"2024-09-13T18:21:29.338126+0000","last_active":"2024-09-13T18:21:29.338126+0000","last_peered":"2024-09-13T18:21:29.338126+0000","last_clean":"2024-09-13T18:21:29.338126+0000","last_became_active":"2024-09-13T17:50:34.585340+0000","last_became_peered":"2024-09-13T17:50:34.585340+0000","last_unstale":"2024-09-13T18:21:29.338126+0000","last_undegraded":"2024-09-13T18:21:29.338126+0000","last_fullsized":"2024-09-13T18:21:29.338126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-13T18:21:29.337797+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-13T18:21:29.337797+0000","last_clean_scrub_stamp":"2024-09-13T18:21:29.337797+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:39:52.759391+0000","scrub_duration":0.003218725,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'17","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T18:22:31.517791+0000","last_change":"2024-09-13T18:22:31.518012+0000","last_active":"2024-09-13T18:22:31.517791+0000","last_peered":"2024-09-13T18:22:31.517791+0000","last_clean":"2024-09-13T18:22:31.517791+0000","last_became_active":"2024-09-13T17:50:34.592046+0000","last_became_peered":"2024-09-13T17:50:34.592046+0000","last_unstale":"2024-09-13T18:22:31.517791+0000","last_undegraded":"2024-09-13T18:22:31.517791+0000","last_fullsized":"2024-09-13T18:22:31.517791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'17","last_scrub_stamp":"2024-09-13T18:22:31.517731+0000","last_deep_scrub":"27'17","last_deep_scrub_stamp":"2024-09-13T18:22:31.517731+0000","last_clean_scrub_stamp":"2024-09-13T18:22:31.517731+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:06.141615+0000","scrub_duration":0.003141535,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":751482,"num_write_kb":16106434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178546,"ondisk_log_size":178546,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1271,"ondisk_log_size":1271,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":15941984,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":121,"num_read_kb":179,"num_write":3371,"num_write_kb":16874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15000,"num_omap_keys":30,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32030720,"data_stored":31883968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3184,"ondisk_log_size":3184,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509829,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1772172,"kb_used_data":8580,"kb_used_omap":1,"kb_used_meta":1763582,"kb_avail":92599668,"statfs":{"total":96636764160,"available":94822060032,"internally_reserved":0,"allocated":8785920,"data_stored":8600393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":1805908424},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:28 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":4.0810000000000004,"15min":2.8069999999999999},"min":{"1min":0.42799999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.84799999999999998,"5min":41.494999999999997,"15min":50.511000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":2.6440000000000001,"15min":1.671},"min":{"1min":0.36299999999999999,"5min":0.193,"15min":0.193},"max":{"1min":0.95799999999999996,"5min":18.760000000000002,"15min":18.760000000000002},"last":0.66500000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:22:28 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":2.2919999999999998,"15min":1.839},"min":{"1min":0.40899999999999997,"5min":0.32000000000000001,"15min":0.23599999999999999},"max":{"1min":0.79000000000000004,"5min":19.626000000000001,"15min":19.626000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":2.5259999999999998,"15min":2.069},"min":{"1min":0.36599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.83699999999999997,"5min":22.126999999999999,"15min":22.126999999999999},"last":0.76600000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:22:28 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":1.071,"15min":1.2749999999999999},"min":{"1min":0.45800000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.0369999999999999,"5min":7.4610000000000003,"15min":22.010000000000002},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.85799999999999998,"5min":1.2070000000000001,"15min":1.46},"min":{"1min":0.49399999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.115,"5min":7.6550000000000002,"15min":24.170999999999999},"last":1.0489999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:22:28 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.0389999999999999,"15min":1.4510000000000001},"min":{"1min":0.66600000000000004,"5min":0.307,"15min":0.307},"max":{"1min":1.4299999999999999,"5min":7.7229999999999999,"15min":24.359000000000002},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":1.669,"15min":1.4710000000000001},"min":{"1min":0.625,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.365,"5min":15.539999999999999,"15min":15.539999999999999},"last":0.89900000000000002}]},{"osd":5,"last update":"Fri Sep 13 18:22:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.292,"15min":1.6259999999999999},"min":{"1min":0.69199999999999995,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.597,"5min":8.5739999999999998,"15min":49.838000000000001},"last":1.004},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.3049999999999999,"15min":1.5940000000000001},"min":{"1min":0.66400000000000003,"5min":0.375,"15min":0.375},"max":{"1min":1.48,"5min":16.334,"15min":41.591999999999999},"last":1.1519999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:22:28 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.643,"15min":1.7549999999999999},"min":{"1min":0.745,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.458,"5min":12.279,"15min":24.457000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.544,"15min":1.6519999999999999},"min":{"1min":0.59799999999999998,"5min":0.42399999999999999,"15min":0.40999999999999998},"max":{"1min":1.7010000000000001,"5min":10.279,"15min":24.387},"last":1.1930000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:22:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.403,"15min":1.4019999999999999},"min":{"1min":0.85299999999999998,"5min":0.34200000000000003,"15min":0.32300000000000001},"max":{"1min":1.5569999999999999,"5min":14.138999999999999,"15min":14.138999999999999},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.661,"15min":1.4430000000000001},"min":{"1min":0.73899999999999999,"5min":0.377,"15min":0.35299999999999998},"max":{"1min":1.593,"5min":14.013,"15min":14.013},"last":1.2310000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509831,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1055244,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":1054846,"kb_avail":93316596,"statfs":{"total":96636764160,"available":95556194304,"internally_reserved":0,"allocated":397312,"data_stored":210684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1080162758},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":1.4279999999999999,"15min":1.0980000000000001},"min":{"1min":0.41099999999999998,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.876,"5min":17.971,"15min":17.971},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":1.5349999999999999,"15min":1.1890000000000001},"min":{"1min":0.63600000000000001,"5min":0.247,"15min":0.22800000000000001},"max":{"1min":1.0089999999999999,"5min":18.001999999999999,"15min":18.001999999999999},"last":1.0609999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":1.905,"15min":1.6879999999999999},"min":{"1min":0.443,"5min":0.23699999999999999,"15min":0.217},"max":{"1min":0.94699999999999995,"5min":24.599,"15min":56.939999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.63,"5min":2.0619999999999998,"15min":1.7589999999999999},"min":{"1min":0.47999999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.81699999999999995,"5min":24.75,"15min":57.006999999999998},"last":0.63100000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":2.3220000000000001,"15min":1.597},"min":{"1min":0.442,"5min":0.218,"15min":0.218},"max":{"1min":0.80400000000000005,"5min":16.707000000000001,"15min":16.707000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":2.0289999999999999,"15min":1.5549999999999999},"min":{"1min":0.57599999999999996,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0660000000000001,"5min":16.640999999999998,"15min":16.640999999999998},"last":0.54100000000000004}]},{"osd":4,"last update":"Fri Sep 13 18:22:23 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":1.3879999999999999,"15min":1.23},"min":{"1min":0.71599999999999997,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.1220000000000001,"5min":8.2829999999999995,"15min":14.177},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.161,"15min":1.159},"min":{"1min":0.70399999999999996,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.2330000000000001,"5min":8.1980000000000004,"15min":14.396000000000001},"last":0.79500000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:22:23 2024","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.83999999999999997,"15min":1.0109999999999999},"min":{"1min":0.84199999999999997,"5min":0.34200000000000003,"15min":0.34100000000000003},"max":{"1min":1.327,"5min":3.6840000000000002,"15min":14.153},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.95599999999999996,"5min":1.0409999999999999,"15min":1.1259999999999999},"min":{"1min":0.752,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.2509999999999999,"5min":7.476,"15min":14.375999999999999},"last":0.76400000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":0.97799999999999998,"15min":1.1080000000000001},"min":{"1min":0.89300000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5880000000000001,"5min":6.8730000000000002,"15min":8.3010000000000002},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.038,"15min":1.22},"min":{"1min":0.94799999999999995,"5min":0.40400000000000003,"15min":0.39200000000000002},"max":{"1min":1.653,"5min":6.9500000000000002,"15min":14.074},"last":1.3640000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:22:23 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.03,"15min":1.089},"min":{"1min":0.749,"5min":0.37,"15min":0.37},"max":{"1min":1.2729999999999999,"5min":5.5270000000000001,"15min":8.0410000000000004},"last":1.286},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.0529999999999999,"15min":1.1830000000000001},"min":{"1min":0.84299999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.3169999999999999,"5min":4.1740000000000004,"15min":14.307},"last":0.81899999999999995}]}]},{"osd":1,"up_from":15,"seq":64424509830,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1738228,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":1737918,"kb_avail":92633612,"statfs":{"total":96636764160,"available":94856818688,"internally_reserved":0,"allocated":307200,"data_stored":116868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1779628486},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":1.8220000000000001,"15min":1.9059999999999999},"min":{"1min":0.44,"5min":0.188,"15min":0.16900000000000001},"max":{"1min":0.85999999999999999,"5min":21.949999999999999,"15min":21.949999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":2.0449999999999999,"15min":1.595},"min":{"1min":0.54700000000000004,"5min":0.193,"15min":0.193},"max":{"1min":0.98699999999999999,"5min":23.338999999999999,"15min":23.338999999999999},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":1.5569999999999999,"15min":1.504},"min":{"1min":0.44,"5min":0.22,"15min":0.22},"max":{"1min":1.107,"5min":11.409000000000001,"15min":14.869999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":1.6419999999999999,"15min":1.4350000000000001},"min":{"1min":0.48999999999999999,"5min":0.223,"15min":0.223},"max":{"1min":1.0549999999999999,"5min":12.439,"15min":15.058},"last":0.68400000000000005}]},{"osd":3,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":3.3140000000000001,"15min":3.1619999999999999},"min":{"1min":0.441,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.173,"5min":27.838000000000001,"15min":41.369999999999997},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.751,"5min":3.262,"15min":2.8959999999999999},"min":{"1min":0.376,"5min":0.251,"15min":0.216},"max":{"1min":1.161,"5min":30.077000000000002,"15min":40.325000000000003},"last":0.89900000000000002}]},{"osd":4,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":2.113,"15min":2.1230000000000002},"min":{"1min":0.627,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.325,"5min":29.702000000000002,"15min":103.44499999999999},"last":0.749},{"interface":"front","average":{"1min":0.98699999999999999,"5min":2.1699999999999999,"15min":1.901},"min":{"1min":0.80300000000000005,"5min":0.30199999999999999,"15min":0.27800000000000002},"max":{"1min":1.173,"5min":29.425999999999998,"15min":40.953000000000003},"last":0.76400000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.6739999999999999,"15min":1.7330000000000001},"min":{"1min":0.85599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.448,"5min":14.521000000000001,"15min":18.954000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.4359999999999999,"15min":1.5009999999999999},"min":{"1min":0.78500000000000003,"5min":0.31,"15min":0.29899999999999999},"max":{"1min":1.202,"5min":13.630000000000001,"15min":15.413},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.8300000000000001,"15min":1.6140000000000001},"min":{"1min":0.79300000000000004,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.276,"5min":13.67,"15min":15.462999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.103,"5min":1.8939999999999999,"15min":1.708},"min":{"1min":0.90600000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.4339999999999999,"5min":13.696,"15min":15.49},"last":1.01}]},{"osd":7,"last update":"Fri Sep 13 18:22:25 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.71,"15min":1.411},"min":{"1min":0.84499999999999997,"5min":0.33600000000000002,"15min":0.30499999999999999},"max":{"1min":1.3009999999999999,"5min":12.476000000000001,"15min":15.093999999999999},"last":1.135},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.7809999999999999,"15min":1.494},"min":{"1min":0.92400000000000004,"5min":0.41899999999999998,"15min":0.35799999999999998},"max":{"1min":1.413,"5min":12.539999999999999,"15min":15.119},"last":1.1839999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509832,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1247072,"kb_used_data":4056,"kb_used_omap":1,"kb_used_meta":1243006,"kb_avail":93124768,"statfs":{"total":96636764160,"available":95359762432,"internally_reserved":0,"allocated":4153344,"data_stored":3967313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1272838598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":2.427,"15min":1.728},"min":{"1min":0.33200000000000002,"5min":0.182,"15min":0.182},"max":{"1min":0.97899999999999998,"5min":23.004000000000001,"15min":24.202000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":2.5409999999999999,"15min":1.871},"min":{"1min":0.59699999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.107,"5min":23.039999999999999,"15min":24.260999999999999},"last":0.53700000000000003}]},{"osd":2,"last update":"Fri Sep 13 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":1.355,"15min":1.1240000000000001},"min":{"1min":0.46899999999999997,"5min":0.20399999999999999,"15min":0.186},"max":{"1min":0.88300000000000001,"5min":11.026999999999999,"15min":17.859000000000002},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.80800000000000005,"5min":1.6859999999999999,"15min":1.345},"min":{"1min":0.55600000000000005,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.0209999999999999,"5min":11.702999999999999,"15min":19.678999999999998},"last":0.748}]},{"osd":3,"last update":"Fri Sep 13 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":2.484,"15min":1.95},"min":{"1min":0.53200000000000003,"5min":0.219,"15min":0.219},"max":{"1min":1.115,"5min":18.056999999999999,"15min":18.056999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":2.3410000000000002,"15min":1.8149999999999999},"min":{"1min":0.51100000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.1619999999999999,"5min":15.856999999999999,"15min":15.856999999999999},"last":0.627}]},{"osd":4,"last update":"Fri Sep 13 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.5840000000000001,"15min":1.276},"min":{"1min":0.73499999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.415,"5min":8.7010000000000005,"15min":8.7010000000000005},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.637,"15min":1.5289999999999999},"min":{"1min":0.66200000000000003,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.403,"5min":10.965999999999999,"15min":16.291},"last":0.84799999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.704,"15min":1.3200000000000001},"min":{"1min":0.69299999999999995,"5min":0.312,"15min":0.29799999999999999},"max":{"1min":1.415,"5min":15.18,"15min":15.18},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.702,"15min":1.431},"min":{"1min":0.81699999999999995,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.3160000000000001,"5min":17.852,"15min":17.852},"last":0.91100000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":2.137,"15min":1.5860000000000001},"min":{"1min":0.76000000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.4319999999999999,"5min":15.467000000000001,"15min":15.467000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.776,"15min":1.4419999999999999},"min":{"1min":0.84099999999999997,"5min":0.378,"15min":0.378},"max":{"1min":1.498,"5min":17.908000000000001,"15min":17.908000000000001},"last":0.82999999999999996}]},{"osd":7,"last update":"Fri Sep 13 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.7030000000000001,"15min":1.3919999999999999},"min":{"1min":0.79300000000000004,"5min":0.33500000000000002,"15min":0.32700000000000001},"max":{"1min":1.4730000000000001,"5min":18.986999999999998,"15min":18.986999999999998},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.173,"5min":1.756,"15min":1.4790000000000001},"min":{"1min":0.88900000000000001,"5min":0.39500000000000002,"15min":0.373},"max":{"1min":1.552,"5min":19.021000000000001,"15min":19.021000000000001},"last":0.98499999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542534,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1211620,"kb_used_data":12252,"kb_used_omap":1,"kb_used_meta":1199358,"kb_avail":93160220,"statfs":{"total":96636764160,"available":95396065280,"internally_reserved":0,"allocated":12546048,"data_stored":12357112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":1228143051},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:22 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":1.5129999999999999,"15min":1.2529999999999999},"min":{"1min":0.52500000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":4.2140000000000004,"5min":11.31,"15min":35.195},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.7,"15min":1.4159999999999999},"min":{"1min":0.54600000000000004,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.3579999999999997,"5min":11.343,"15min":35.253999999999998},"last":0.93600000000000005}]},{"osd":1,"last update":"Fri Sep 13 18:22:33 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":2.1339999999999999,"15min":1.4990000000000001},"min":{"1min":0.61699999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.032,"5min":15.289,"15min":19.585000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.93999999999999995,"5min":2.202,"15min":1.6319999999999999},"min":{"1min":0.68400000000000005,"5min":0.33400000000000002,"15min":0.28000000000000003},"max":{"1min":1.2769999999999999,"5min":17.398,"15min":19.617000000000001},"last":0.97199999999999998}]},{"osd":2,"last update":"Fri Sep 13 18:22:22 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":1.4399999999999999,"15min":1.1659999999999999},"min":{"1min":0.69199999999999995,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.276,"5min":8.8119999999999994,"15min":15.789999999999999},"last":0.75},{"interface":"front","average":{"1min":1.121,"5min":1.472,"15min":1.1919999999999999},"min":{"1min":0.88800000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.373,"5min":8.0779999999999994,"15min":12.965},"last":1.131}]},{"osd":3,"last update":"Fri Sep 13 18:22:22 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":2.548,"15min":1.915},"min":{"1min":0.66900000000000004,"5min":0.35499999999999998,"15min":0.316},"max":{"1min":1.4650000000000001,"5min":40.948999999999998,"15min":40.948999999999998},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":2.9260000000000002,"15min":1.784},"min":{"1min":0.95899999999999996,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.4930000000000001,"5min":25.234000000000002,"15min":25.234000000000002},"last":1.212}]},{"osd":4,"last update":"Fri Sep 13 18:22:22 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":1.129,"15min":0.98799999999999999},"min":{"1min":0.65300000000000002,"5min":0.28799999999999998,"15min":0.252},"max":{"1min":1.2470000000000001,"5min":8.8670000000000009,"15min":8.8670000000000009},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.3109999999999999,"15min":1.1879999999999999},"min":{"1min":0.78300000000000003,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":4.2830000000000004,"5min":10.279999999999999,"15min":10.279999999999999},"last":1.0169999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:22:22 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1,"15min":0.97999999999999998},"min":{"1min":0.754,"5min":0.311,"15min":0.27000000000000002},"max":{"1min":1.1899999999999999,"5min":8.1699999999999999,"15min":8.1699999999999999},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.028,"15min":1.0309999999999999},"min":{"1min":0.69499999999999995,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":4.125,"5min":6.3220000000000001,"15min":6.3220000000000001},"last":1.107}]},{"osd":6,"last update":"Fri Sep 13 18:22:22 2024","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.512,"15min":1.2390000000000001},"min":{"1min":0.755,"5min":0.34599999999999997,"15min":0.309},"max":{"1min":1.502,"5min":8.2219999999999995,"15min":8.2219999999999995},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.327,"5min":1.5920000000000001,"15min":1.2909999999999999},"min":{"1min":1.161,"5min":0.434,"15min":0.34499999999999997},"max":{"1min":1.5369999999999999,"5min":8.2469999999999999,"15min":8.2469999999999999},"last":1.3340000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542534,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1419436,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":1415038,"kb_avail":92952404,"statfs":{"total":96636764160,"available":95183261696,"internally_reserved":0,"allocated":4493312,"data_stored":4311530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1448999367},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":1.74,"15min":1.3},"min":{"1min":0.59499999999999997,"5min":0.214,"15min":0.214},"max":{"1min":1.1060000000000001,"5min":11.666,"15min":12.808999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":1.4410000000000001,"15min":1.1830000000000001},"min":{"1min":0.63600000000000001,"5min":0.218,"15min":0.218},"max":{"1min":1.1659999999999999,"5min":7.8700000000000001,"15min":12.308999999999999},"last":0.91800000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:22:21 2024","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":1.9430000000000001,"15min":1.4950000000000001},"min":{"1min":0.52800000000000002,"5min":0.313,"15min":0.28699999999999998},"max":{"1min":1.206,"5min":20.530999999999999,"15min":20.530999999999999},"last":1.173},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1.7849999999999999,"15min":1.5189999999999999},"min":{"1min":0.5,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.083,"5min":20.501999999999999,"15min":20.501999999999999},"last":0.77300000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:22:21 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.4350000000000001,"15min":1.24},"min":{"1min":0.54700000000000004,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.415,"5min":9.1839999999999993,"15min":10.792999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":1.151,"5min":1.5389999999999999,"15min":1.3879999999999999},"min":{"1min":0.86199999999999999,"5min":0.317,"15min":0.317},"max":{"1min":1.3600000000000001,"5min":9.2219999999999995,"15min":10.826000000000001},"last":0.83199999999999996}]},{"osd":3,"last update":"Fri Sep 13 18:22:21 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":3.339,"15min":2.5790000000000002},"min":{"1min":0.71099999999999997,"5min":0.35799999999999998,"15min":0.33000000000000002},"max":{"1min":1.0720000000000001,"5min":32.25,"15min":84.867999999999995},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.036,"5min":2.3879999999999999,"15min":1.929},"min":{"1min":0.626,"5min":0.34599999999999997,"15min":0.33000000000000002},"max":{"1min":1.3380000000000001,"5min":14.064,"15min":84.343999999999994},"last":1.0660000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:22:12 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":1.651,"15min":1.212},"min":{"1min":0.70199999999999996,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.1990000000000001,"5min":11.044,"15min":11.044},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.2589999999999999,"15min":1.117},"min":{"1min":0.78300000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.181,"5min":9.4719999999999995,"15min":9.4719999999999995},"last":1.0309999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:22:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.3400000000000001,"15min":1.1599999999999999},"min":{"1min":0.74199999999999999,"5min":0.318,"15min":0.29199999999999998},"max":{"1min":1.3979999999999999,"5min":8.7880000000000003,"15min":8.7880000000000003},"last":1.236},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1659999999999999,"15min":1.1080000000000001},"min":{"1min":0.92000000000000004,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.284,"5min":7.569,"15min":7.569},"last":1.127}]},{"osd":7,"last update":"Fri Sep 13 18:22:12 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3009999999999999,"15min":1.1890000000000001},"min":{"1min":0.91700000000000004,"5min":0.36799999999999999,"15min":0.29999999999999999},"max":{"1min":1.5840000000000001,"5min":7.1820000000000004,"15min":7.1820000000000004},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3799999999999999,"15min":1.2629999999999999},"min":{"1min":1.008,"5min":0.40500000000000003,"15min":0.33500000000000002},"max":{"1min":1.635,"5min":7.2039999999999997,"15min":7.2039999999999997},"last":1.1100000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542536,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":978544,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":978238,"kb_avail":93393296,"statfs":{"total":96636764160,"available":95634735104,"internally_reserved":0,"allocated":303104,"data_stored":117302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1001716166},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":1.5760000000000001,"15min":1.1919999999999999},"min":{"1min":0.59699999999999998,"5min":0.23300000000000001,"15min":0.218},"max":{"1min":1.4690000000000001,"5min":10.051,"15min":10.148999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.6140000000000001,"15min":1.3340000000000001},"min":{"1min":0.67300000000000004,"5min":0.29799999999999999,"15min":0.27900000000000003},"max":{"1min":1.3089999999999999,"5min":10.098000000000001,"15min":10.185},"last":1.1399999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":1.474,"15min":1.1879999999999999},"min":{"1min":0.60399999999999998,"5min":0.27800000000000002,"15min":0.25700000000000001},"max":{"1min":1.234,"5min":10.08,"15min":10.436999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.3700000000000001,"15min":1.381},"min":{"1min":0.45900000000000002,"5min":0.253,"15min":0.253},"max":{"1min":1.26,"5min":18.584,"15min":19.388000000000002},"last":1.0049999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":1.3620000000000001,"15min":1.131},"min":{"1min":0.499,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.2110000000000001,"5min":10.183,"15min":12.667},"last":0.623},{"interface":"front","average":{"1min":1.056,"5min":1.3899999999999999,"15min":1.2},"min":{"1min":0.58299999999999996,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.462,"5min":34.905999999999999,"15min":34.905999999999999},"last":0.89800000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":2.9780000000000002,"15min":2.3119999999999998},"min":{"1min":0.5,"5min":0.314,"15min":0.314},"max":{"1min":0.93300000000000005,"5min":25.149999999999999,"15min":28.295000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.93300000000000005,"5min":2.5609999999999999,"15min":1.7310000000000001},"min":{"1min":0.61099999999999999,"5min":0.39700000000000002,"15min":0.318},"max":{"1min":1.0469999999999999,"5min":19.731000000000002,"15min":19.731000000000002},"last":0.96899999999999997}]},{"osd":4,"last update":"Fri Sep 13 18:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.1259999999999999,"15min":0.97999999999999998},"min":{"1min":0.628,"5min":0.28599999999999998,"15min":0.255},"max":{"1min":1.4770000000000001,"5min":10.023999999999999,"15min":10.023999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.5860000000000001,"15min":1.234},"min":{"1min":0.84099999999999997,"5min":0.32500000000000001,"15min":0.28999999999999998},"max":{"1min":1.4359999999999999,"5min":10.210000000000001,"15min":10.210000000000001},"last":0.93100000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:22:31 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.349,"15min":1.1279999999999999},"min":{"1min":0.73199999999999998,"5min":0.33900000000000002,"15min":0.25800000000000001},"max":{"1min":1.401,"5min":6.6849999999999996,"15min":6.6849999999999996},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.4179999999999999,"15min":1.218},"min":{"1min":0.71499999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.6160000000000001,"5min":10.151,"15min":10.151},"last":1.0409999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:22:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.3200000000000001,"15min":1.1459999999999999},"min":{"1min":0.70099999999999996,"5min":0.32900000000000001,"15min":0.32000000000000001},"max":{"1min":1.571,"5min":7.9329999999999998,"15min":7.9329999999999998},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.3,"15min":1.151},"min":{"1min":0.86499999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.6579999999999999,"5min":5.976,"15min":5.976},"last":1.1850000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542534,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1301036,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":1296638,"kb_avail":93070804,"statfs":{"total":96636764160,"available":95304503296,"internally_reserved":0,"allocated":4493312,"data_stored":4310894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1327757766},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.4610000000000001,"15min":1.3220000000000001},"min":{"1min":0.69199999999999995,"5min":0.21299999999999999,"15min":0.20999999999999999},"max":{"1min":1.272,"5min":14.318,"15min":28.439},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.575,"15min":1.4470000000000001},"min":{"1min":0.626,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.2769999999999999,"5min":16.943000000000001,"15min":28.422000000000001},"last":0.67000000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":3.7469999999999999,"15min":2.649},"min":{"1min":0.69999999999999996,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.2609999999999999,"5min":30.277999999999999,"15min":38.895000000000003},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.004,"5min":3.4129999999999998,"15min":2.028},"min":{"1min":0.80900000000000005,"5min":0.28199999999999997,"15min":0.27000000000000002},"max":{"1min":1.3049999999999999,"5min":32.670000000000002,"15min":32.670000000000002},"last":0.93899999999999995}]},{"osd":2,"last update":"Fri Sep 13 18:22:19 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":1.619,"15min":1.1870000000000001},"min":{"1min":0.439,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.2070000000000001,"5min":25.167999999999999,"15min":25.167999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.742,"15min":1.3520000000000001},"min":{"1min":0.624,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":1.1990000000000001,"5min":25.218,"15min":25.218},"last":0.73699999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":2.6259999999999999,"15min":1.8340000000000001},"min":{"1min":0.67500000000000004,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.304,"5min":43.945999999999998,"15min":43.945999999999998},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.038,"5min":2.927,"15min":2.145},"min":{"1min":0.68500000000000005,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":1.385,"5min":29.640000000000001,"15min":29.640000000000001},"last":0.98199999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.1639999999999999,"15min":1.0600000000000001},"min":{"1min":0.78100000000000003,"5min":0.255,"15min":0.255},"max":{"1min":1.425,"5min":12.282999999999999,"15min":12.282999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.3839999999999999,"15min":1.1919999999999999},"min":{"1min":0.76100000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.7210000000000001,"5min":16.995999999999999,"15min":16.995999999999999},"last":1.016}]},{"osd":6,"last update":"Fri Sep 13 18:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.4990000000000001,"15min":1.4079999999999999},"min":{"1min":0.80600000000000005,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.4930000000000001,"5min":12.42,"15min":23.010999999999999},"last":1.038},{"interface":"front","average":{"1min":1.256,"5min":1.3500000000000001,"15min":1.2070000000000001},"min":{"1min":0.93000000000000005,"5min":0.33800000000000002,"15min":0.33600000000000002},"max":{"1min":1.556,"5min":7.9489999999999998,"15min":7.9489999999999998},"last":1.165}]},{"osd":7,"last update":"Fri Sep 13 18:22:19 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.319,"15min":1.3069999999999999},"min":{"1min":0.98599999999999999,"5min":0.30499999999999999,"15min":0.28799999999999998},"max":{"1min":1.5960000000000001,"5min":11.298,"15min":20.417000000000002},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.351,"15min":1.276},"min":{"1min":1.0720000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.623,"5min":12.308999999999999,"15min":12.308999999999999},"last":1.24}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3276800,"data_stored":3261021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":94760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8503296,"data_stored":8484469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4194970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4210688,"data_stored":4195606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":11669504,"data_stored":11650820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:22:47.767 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T18:22:47.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.868+0000 7f744e7c2700 1 Processor -- start 2024-09-13T18:22:47.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.868+0000 7f744e7c2700 1 -- start start 2024-09-13T18:22:47.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.868+0000 7f744e7c2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7448151580 0x7f74480c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:47.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.868+0000 7f744e7c2700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f74480b1fb0 0x7f74480c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:47.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.868+0000 7f744e7c2700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7448070ae0 con 0x7f7448071260 2024-09-13T18:22:47.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.868+0000 7f744e7c2700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7448070d30 con 0x7f74480b1fb0 2024-09-13T18:22:47.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.868+0000 7f744e7c2700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74480c29c0 con 0x7f7448151580 2024-09-13T18:22:47.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.868+0000 7f744cfbf700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f74480b1fb0 0x7f74480c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:47.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.872+0000 7f744cfbf700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f74480b1fb0 0x7f74480c57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48728/0 (socket says 172.21.15.40:48728) 2024-09-13T18:22:47.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.872+0000 7f744cfbf700 1 -- 172.21.15.40:0/2577661834 learned_addr learned my addr 172.21.15.40:0/2577661834 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:22:47.878 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.872+0000 7f744d7c0700 1 --2- 172.21.15.40:0/2577661834 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7448151580 0x7f74480c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:47.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.872+0000 7f744cfbf700 1 -- 172.21.15.40:0/2577661834 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7448151580 msgr2=0x7f74480c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:47.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.872+0000 7f744cfbf700 1 --2- 172.21.15.40:0/2577661834 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7448151580 0x7f74480c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:47.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.872+0000 7f744cfbf700 1 -- 172.21.15.40:0/2577661834 >> v1:172.21.15.43:6789/0 conn(0x7f7448071260 legacy=0x7f7448157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:47.879 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.872+0000 7f744cfbf700 1 -- 172.21.15.40:0/2577661834 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74480c2ca0 con 0x7f74480b1fb0 2024-09-13T18:22:47.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.872+0000 7f744cfbf700 1 --2- 172.21.15.40:0/2577661834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74480b1fb0 0x7f74480c57f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7438009910 tx=0x7f7438009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395dd7e41a9b13b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:47.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.872+0000 7f743ffff700 1 -- 172.21.15.40:0/2577661834 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743800ba60 con 0x7f74480b1fb0 2024-09-13T18:22:47.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.872+0000 7f743ffff700 1 -- 172.21.15.40:0/2577661834 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743800bbc0 con 0x7f74480b1fb0 2024-09-13T18:22:47.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.872+0000 7f743ffff700 1 -- 172.21.15.40:0/2577661834 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7438009e60 con 0x7f74480b1fb0 2024-09-13T18:22:47.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.872+0000 7f744e7c2700 1 -- 172.21.15.40:0/2577661834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74480b1fb0 msgr2=0x7f74480c57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:47.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.872+0000 7f744e7c2700 1 --2- 172.21.15.40:0/2577661834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74480b1fb0 0x7f74480c57f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7438009910 tx=0x7f7438009c20 comp rx=0 tx=0).stop 2024-09-13T18:22:47.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.872+0000 7f744e7c2700 1 -- 172.21.15.40:0/2577661834 shutdown_connections 2024-09-13T18:22:47.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.872+0000 7f744e7c2700 1 --2- 172.21.15.40:0/2577661834 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74480b1fb0 0x7f74480c57f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:47.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.872+0000 7f744e7c2700 1 --2- 172.21.15.40:0/2577661834 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7448151580 0x7f74480c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:47.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.872+0000 7f744e7c2700 1 -- 172.21.15.40:0/2577661834 >> 172.21.15.40:0/2577661834 conn(0x7f744806e6d0 msgr2=0x7f744806eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:47.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.872+0000 7f744e7c2700 1 -- 172.21.15.40:0/2577661834 shutdown_connections 2024-09-13T18:22:47.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.872+0000 7f744e7c2700 1 -- 172.21.15.40:0/2577661834 wait complete. 2024-09-13T18:22:47.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744e7c2700 1 Processor -- start 2024-09-13T18:22:47.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744e7c2700 1 -- start start 2024-09-13T18:22:47.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744e7c2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7448071260 0x7f7448159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:47.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744e7c2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74480b1fb0 0x7f74481efe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:47.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744e7c2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7448151580 0x7f74481f0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:47.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744e7c2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74481ed550 con 0x7f7448071260 2024-09-13T18:22:47.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744e7c2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74481ec6c0 con 0x7f74480b1fb0 2024-09-13T18:22:47.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744e7c2700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f744814a330 con 0x7f7448151580 2024-09-13T18:22:47.884 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744d7c0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7448071260 0x7f7448159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:47.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744d7c0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7448071260 0x7f7448159360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48730/0 (socket says 172.21.15.40:48730) 2024-09-13T18:22:47.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744d7c0700 1 -- 172.21.15.40:0/2170967992 learned_addr learned my addr 172.21.15.40:0/2170967992 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:22:47.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744cfbf700 1 --2- 172.21.15.40:0/2170967992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74480b1fb0 0x7f74481efe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:47.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744dfc1700 1 --2- 172.21.15.40:0/2170967992 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7448151580 0x7f74481f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:47.885 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744d7c0700 1 -- 172.21.15.40:0/2170967992 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7448151580 msgr2=0x7f74481f0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:47.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744d7c0700 1 --2- 172.21.15.40:0/2170967992 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7448151580 0x7f74481f0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:47.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744d7c0700 1 -- 172.21.15.40:0/2170967992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74480b1fb0 msgr2=0x7f74481efe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:47.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744d7c0700 1 --2- 172.21.15.40:0/2170967992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74480b1fb0 0x7f74481efe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:47.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744d7c0700 1 -- 172.21.15.40:0/2170967992 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7438009570 con 0x7f7448071260 2024-09-13T18:22:47.889 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744d7c0700 1 --2- 172.21.15.40:0/2170967992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7448071260 0x7f7448159360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f744400d800 tx=0x7f744400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:47.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f743dffb700 1 -- 172.21.15.40:0/2170967992 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74440125f0 con 0x7f7448071260 2024-09-13T18:22:47.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f743dffb700 1 -- 172.21.15.40:0/2170967992 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7444012750 con 0x7f7448071260 2024-09-13T18:22:47.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f743dffb700 1 -- 172.21.15.40:0/2170967992 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7444012ac0 con 0x7f7448071260 2024-09-13T18:22:47.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744e7c2700 1 -- 172.21.15.40:0/2170967992 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74481ec940 con 0x7f7448071260 2024-09-13T18:22:47.890 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744e7c2700 1 -- 172.21.15.40:0/2170967992 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74481ecf00 con 0x7f7448071260 2024-09-13T18:22:47.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f744e7c2700 1 -- 172.21.15.40:0/2170967992 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7448150a60 con 0x7f7448071260 2024-09-13T18:22:47.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.876+0000 7f743dffb700 1 -- 172.21.15.40:0/2170967992 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7444012c20 con 0x7f7448071260 2024-09-13T18:22:47.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.880+0000 7f743dffb700 1 --2- 172.21.15.40:0/2170967992 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f742c066730 0x7f742c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:47.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.880+0000 7f743dffb700 1 -- 172.21.15.40:0/2170967992 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f74440849a0 con 0x7f7448071260 2024-09-13T18:22:47.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.880+0000 7f744cfbf700 1 --2- 172.21.15.40:0/2170967992 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f742c066730 0x7f742c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:47.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.880+0000 7f744cfbf700 1 --2- 172.21.15.40:0/2170967992 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f742c066730 0x7f742c068bf0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7438009910 tx=0x7f7438008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:47.891 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:47.884+0000 7f743dffb700 1 -- 172.21.15.40:0/2170967992 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f74440518c0 con 0x7f7448071260 2024-09-13T18:22:48.062 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.056+0000 7f744e7c2700 1 -- 172.21.15.40:0/2170967992 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f7448072670 con 0x7f7448071260 2024-09-13T18:22:48.070 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.064+0000 7f743dffb700 1 -- 172.21.15.40:0/2170967992 <== mon.0 v2:172.21.15.40:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f74440518c0 con 0x7f7448071260 2024-09-13T18:22:48.071 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.064+0000 7f743dffb700 1 -- 172.21.15.40:0/2170967992 <== mon.0 v2:172.21.15.40:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) v1 ==== 123+0+0 (secure 0 0 0) 0x7f7444012ed0 con 0x7f7448071260 2024-09-13T18:22:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.068+0000 7f744e7c2700 1 -- 172.21.15.40:0/2170967992 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f742c066730 msgr2=0x7f742c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.068+0000 7f744e7c2700 1 --2- 172.21.15.40:0/2170967992 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f742c066730 0x7f742c068bf0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7438009910 tx=0x7f7438008030 comp rx=0 tx=0).stop 2024-09-13T18:22:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.068+0000 7f744e7c2700 1 -- 172.21.15.40:0/2170967992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7448071260 msgr2=0x7f7448159360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.068+0000 7f744e7c2700 1 --2- 172.21.15.40:0/2170967992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7448071260 0x7f7448159360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f744400d800 tx=0x7f744400dbc0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.068+0000 7f744e7c2700 1 -- 172.21.15.40:0/2170967992 shutdown_connections 2024-09-13T18:22:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.068+0000 7f744e7c2700 1 --2- 172.21.15.40:0/2170967992 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f742c066730 0x7f742c068bf0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.068+0000 7f744e7c2700 1 --2- 172.21.15.40:0/2170967992 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7448151580 0x7f74481f0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.068+0000 7f744e7c2700 1 --2- 172.21.15.40:0/2170967992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74480b1fb0 0x7f74481efe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.068+0000 7f744e7c2700 1 --2- 172.21.15.40:0/2170967992 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7448071260 0x7f7448159360 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.068+0000 7f744e7c2700 1 -- 172.21.15.40:0/2170967992 >> 172.21.15.40:0/2170967992 conn(0x7f744806e6d0 msgr2=0x7f74480c52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.068+0000 7f744e7c2700 1 -- 172.21.15.40:0/2170967992 shutdown_connections 2024-09-13T18:22:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.068+0000 7f744e7c2700 1 -- 172.21.15.40:0/2170967992 wait complete. 2024-09-13T18:22:48.094 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T18:22:48.094 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:22:48.209 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.200+0000 7fa6f1bc7700 1 Processor -- start 2024-09-13T18:22:48.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f1bc7700 1 -- start start 2024-09-13T18:22:48.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f1bc7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa6ec1512e0 0x7fa6ec157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:48.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f1bc7700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa6ec0c87c0 0x7fa6ec06e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:48.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f1bc7700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6ec0c7c80 con 0x7fa6ec0b2c00 2024-09-13T18:22:48.210 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f1bc7700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ec06eeb0 con 0x7fa6ec0c87c0 2024-09-13T18:22:48.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f1bc7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec06e4e0 con 0x7fa6ec1512e0 2024-09-13T18:22:48.211 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f13c6700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa6ec0c87c0 0x7fa6ec06e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:48.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f0bc5700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa6ec1512e0 0x7fa6ec157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:48.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f13c6700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa6ec0c87c0 0x7fa6ec06e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:51316/0 (socket says 172.21.15.40:51316) 2024-09-13T18:22:48.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f13c6700 1 -- 172.21.15.40:0/2417470120 learned_addr learned my addr 172.21.15.40:0/2417470120 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:22:48.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f13c6700 1 -- 172.21.15.40:0/2417470120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa6ec1512e0 msgr2=0x7fa6ec157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:48.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f13c6700 1 --2- 172.21.15.40:0/2417470120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa6ec1512e0 0x7fa6ec157ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f13c6700 1 -- 172.21.15.40:0/2417470120 >> v1:172.21.15.43:6789/0 conn(0x7fa6ec0b2c00 legacy=0x7fa6ec0c1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:48.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f13c6700 1 -- 172.21.15.40:0/2417470120 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec06deb0 con 0x7fa6ec0c87c0 2024-09-13T18:22:48.212 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f13c6700 1 --2- 172.21.15.40:0/2417470120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6ec0c87c0 0x7fa6ec06e970 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4009e80 tx=0x7fa6e400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f700579c9305b59 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:48.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6eb7fe700 1 -- 172.21.15.40:0/2417470120 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400e070 con 0x7fa6ec0c87c0 2024-09-13T18:22:48.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6eb7fe700 1 -- 172.21.15.40:0/2417470120 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fa6e400b8d0 con 0x7fa6ec0c87c0 2024-09-13T18:22:48.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6eb7fe700 1 -- 172.21.15.40:0/2417470120 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400bca0 con 0x7fa6ec0c87c0 2024-09-13T18:22:48.213 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f1bc7700 1 -- 172.21.15.40:0/2417470120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6ec0c87c0 msgr2=0x7fa6ec06e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f1bc7700 1 --2- 172.21.15.40:0/2417470120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6ec0c87c0 0x7fa6ec06e970 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4009e80 tx=0x7fa6e400b330 comp rx=0 tx=0).stop 2024-09-13T18:22:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f1bc7700 1 -- 172.21.15.40:0/2417470120 shutdown_connections 2024-09-13T18:22:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f1bc7700 1 --2- 172.21.15.40:0/2417470120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6ec0c87c0 0x7fa6ec06e970 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f1bc7700 1 --2- 172.21.15.40:0/2417470120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa6ec1512e0 0x7fa6ec157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f1bc7700 1 -- 172.21.15.40:0/2417470120 >> 172.21.15.40:0/2417470120 conn(0x7fa6ec0ccef0 msgr2=0x7fa6ec0cd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f1bc7700 1 -- 172.21.15.40:0/2417470120 shutdown_connections 2024-09-13T18:22:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.204+0000 7fa6f1bc7700 1 -- 172.21.15.40:0/2417470120 wait complete. 2024-09-13T18:22:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6f1bc7700 1 Processor -- start 2024-09-13T18:22:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6f1bc7700 1 -- start start 2024-09-13T18:22:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6f1bc7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6ec0b2c00 0x7fa6ec1e3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6f1bc7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6ec0c87c0 0x7fa6ec14c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6f1bc7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa6ec1512e0 0x7fa6ec14cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6f1bc7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec14bac0 con 0x7fa6ec0c87c0 2024-09-13T18:22:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6f1bc7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec1e0d70 con 0x7fa6ec0b2c00 2024-09-13T18:22:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6f1bc7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec150a10 con 0x7fa6ec1512e0 2024-09-13T18:22:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6ebfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6ec0c87c0 0x7fa6ec14c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6ebfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6ec0c87c0 0x7fa6ec14c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48738/0 (socket says 172.21.15.40:48738) 2024-09-13T18:22:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6ebfff700 1 -- 172.21.15.40:0/1401554513 learned_addr learned my addr 172.21.15.40:0/1401554513 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:22:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6f0bc5700 1 --2- 172.21.15.40:0/1401554513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6ec0b2c00 0x7fa6ec1e3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6f13c6700 1 --2- 172.21.15.40:0/1401554513 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa6ec1512e0 0x7fa6ec14cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6ebfff700 1 -- 172.21.15.40:0/1401554513 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa6ec1512e0 msgr2=0x7fa6ec14cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6ebfff700 1 --2- 172.21.15.40:0/1401554513 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa6ec1512e0 0x7fa6ec14cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6ebfff700 1 -- 172.21.15.40:0/1401554513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6ec0b2c00 msgr2=0x7fa6ec1e3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6ebfff700 1 --2- 172.21.15.40:0/1401554513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6ec0b2c00 0x7fa6ec1e3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6ebfff700 1 -- 172.21.15.40:0/1401554513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e4009b30 con 0x7fa6ec0c87c0 2024-09-13T18:22:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6f13c6700 1 --2- 172.21.15.40:0/1401554513 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa6ec1512e0 0x7fa6ec14cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6ebfff700 1 --2- 172.21.15.40:0/1401554513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6ec0c87c0 0x7fa6ec14c680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800c4d0 tx=0x7fa6d800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6e97fa700 1 -- 172.21.15.40:0/1401554513 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d80052d0 con 0x7fa6ec0c87c0 2024-09-13T18:22:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6e97fa700 1 -- 172.21.15.40:0/1401554513 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fa6d8005430 con 0x7fa6ec0c87c0 2024-09-13T18:22:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6f1bc7700 1 -- 172.21.15.40:0/1401554513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ec14b720 con 0x7fa6ec0c87c0 2024-09-13T18:22:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6e97fa700 1 -- 172.21.15.40:0/1401554513 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d80055a0 con 0x7fa6ec0c87c0 2024-09-13T18:22:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6f1bc7700 1 -- 172.21.15.40:0/1401554513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ec14fea0 con 0x7fa6ec0c87c0 2024-09-13T18:22:48.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.208+0000 7fa6f1bc7700 1 -- 172.21.15.40:0/1401554513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ec14bc00 con 0x7fa6ec0c87c0 2024-09-13T18:22:48.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.212+0000 7fa6e97fa700 1 -- 172.21.15.40:0/1401554513 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa6d80038e0 con 0x7fa6ec0c87c0 2024-09-13T18:22:48.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.212+0000 7fa6e97fa700 1 --2- 172.21.15.40:0/1401554513 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa6d0066730 0x7fa6d0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:48.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.212+0000 7fa6e97fa700 1 -- 172.21.15.40:0/1401554513 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa6d8083910 con 0x7fa6ec0c87c0 2024-09-13T18:22:48.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.212+0000 7fa6f0bc5700 1 --2- 172.21.15.40:0/1401554513 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa6d0066730 0x7fa6d0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:48.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.212+0000 7fa6f0bc5700 1 --2- 172.21.15.40:0/1401554513 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa6d0066730 0x7fa6d0068bf0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009b50 tx=0x7fa6e0009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:48.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.220+0000 7fa6e97fa700 1 -- 172.21.15.40:0/1401554513 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa6ec14bc00 con 0x7fa6ec0c87c0 2024-09-13T18:22:48.432 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.424+0000 7fa6f1bc7700 1 -- 172.21.15.40:0/1401554513 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6ec14bc00 con 0x7fa6ec0c87c0 2024-09-13T18:22:48.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.424+0000 7fa6e97fa700 1 -- 172.21.15.40:0/1401554513 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6268 (secure 0 0 0) 0x7fa6ec14bc00 con 0x7fa6ec0c87c0 2024-09-13T18:22:48.433 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T18:22:48.433 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6839/1861594206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1861594206},{"type":"v1","addr":"172.21.15.40:6839","nonce":1861594206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1548394691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1548394691},{"type":"v1","addr":"172.21.15.40:6835","nonce":1548394691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/486611242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":486611242},{"type":"v1","addr":"172.21.15.40:6837","nonce":486611242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:50:34.562623+0000","modified":"2024-09-13T17:51:12.236489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4288,"mds_1":4285,"mds_2":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"f","rank":2,"incarnation":14,"state":"up:active","state_seq":12,"addr":"172.21.15.43:6833/526975022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":526975022},{"type":"v1","addr":"172.21.15.43:6833","nonce":526975022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4285":{"gid":4285,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.43:6835/2586910944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2586910944},{"type":"v1","addr":"172.21.15.43:6835","nonce":2586910944}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/334160033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":334160033},{"type":"v1","addr":"172.21.15.43:6837","nonce":334160033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:22:48.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.428+0000 7fa6f1bc7700 1 -- 172.21.15.40:0/1401554513 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa6d0066730 msgr2=0x7fa6d0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:48.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.428+0000 7fa6f1bc7700 1 --2- 172.21.15.40:0/1401554513 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa6d0066730 0x7fa6d0068bf0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009b50 tx=0x7fa6e0009240 comp rx=0 tx=0).stop 2024-09-13T18:22:48.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.428+0000 7fa6f1bc7700 1 -- 172.21.15.40:0/1401554513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6ec0c87c0 msgr2=0x7fa6ec14c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:48.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.428+0000 7fa6f1bc7700 1 --2- 172.21.15.40:0/1401554513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6ec0c87c0 0x7fa6ec14c680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800c4d0 tx=0x7fa6d800c7e0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.428+0000 7fa6f1bc7700 1 -- 172.21.15.40:0/1401554513 shutdown_connections 2024-09-13T18:22:48.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.428+0000 7fa6f1bc7700 1 --2- 172.21.15.40:0/1401554513 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fa6d0066730 0x7fa6d0068bf0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.428+0000 7fa6f1bc7700 1 --2- 172.21.15.40:0/1401554513 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa6ec1512e0 0x7fa6ec14cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.428+0000 7fa6f1bc7700 1 --2- 172.21.15.40:0/1401554513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa6ec0c87c0 0x7fa6ec14c680 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.428+0000 7fa6f1bc7700 1 --2- 172.21.15.40:0/1401554513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6ec0b2c00 0x7fa6ec1e3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.428+0000 7fa6f1bc7700 1 -- 172.21.15.40:0/1401554513 >> 172.21.15.40:0/1401554513 conn(0x7fa6ec0ccef0 msgr2=0x7fa6ec0ca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:48.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.428+0000 7fa6f1bc7700 1 -- 172.21.15.40:0/1401554513 shutdown_connections 2024-09-13T18:22:48.437 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.428+0000 7fa6f1bc7700 1 -- 172.21.15.40:0/1401554513 wait complete. 2024-09-13T18:22:48.437 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 15 2024-09-13T18:22:48.448 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:22:48.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.544+0000 7fc998a07700 1 Processor -- start 2024-09-13T18:22:48.552 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.544+0000 7fc998a07700 1 -- start start 2024-09-13T18:22:48.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.544+0000 7fc998a07700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc9941510d0 0x7fc994157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:48.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.544+0000 7fc998a07700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9940c7400 0x7fc9940c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:48.553 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.544+0000 7fc998a07700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99415a8b0 con 0x7fc9941514b0 2024-09-13T18:22:48.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.544+0000 7fc998a07700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9940c2e80 con 0x7fc9941510d0 2024-09-13T18:22:48.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.544+0000 7fc998a07700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9941581e0 con 0x7fc9940c7400 2024-09-13T18:22:48.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.544+0000 7fc993fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9940c7400 0x7fc9940c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:48.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.544+0000 7fc9937fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc9941510d0 0x7fc994157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:48.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.544+0000 7fc993fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9940c7400 0x7fc9940c4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.40:36128/0 (socket says 172.21.15.40:36128) 2024-09-13T18:22:48.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.544+0000 7fc9937fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc9941510d0 0x7fc994157ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.40:51344/0 (socket says 172.21.15.40:51344) 2024-09-13T18:22:48.554 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.544+0000 7fc993fff700 1 -- 172.21.15.40:0/4254408205 learned_addr learned my addr 172.21.15.40:0/4254408205 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:22:48.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.548+0000 7fc993fff700 1 -- 172.21.15.40:0/4254408205 >> v2:172.21.15.43:3300/0 conn(0x7fc9941510d0 msgr2=0x7fc994157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:48.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.548+0000 7fc993fff700 1 --2- 172.21.15.40:0/4254408205 >> v2:172.21.15.43:3300/0 conn(0x7fc9941510d0 0x7fc994157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.548+0000 7fc993fff700 1 -- 172.21.15.40:0/4254408205 >> v1:172.21.15.43:6789/0 conn(0x7fc9941514b0 legacy=0x7fc9940c2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:48.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.548+0000 7fc993fff700 1 -- 172.21.15.40:0/4254408205 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc994066390 con 0x7fc9940c7400 2024-09-13T18:22:48.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.548+0000 7fc993fff700 1 --2- 172.21.15.40:0/4254408205 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9940c7400 0x7fc9940c4dd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009e50 tx=0x7fc98c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a61f11e3eb258f7f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:48.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.548+0000 7fc9927fc700 1 -- 172.21.15.40:0/4254408205 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c00e070 con 0x7fc9940c7400 2024-09-13T18:22:48.556 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.548+0000 7fc9927fc700 1 -- 172.21.15.40:0/4254408205 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc98c00b910 con 0x7fc9940c7400 2024-09-13T18:22:48.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.548+0000 7fc9927fc700 1 -- 172.21.15.40:0/4254408205 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c00bce0 con 0x7fc9940c7400 2024-09-13T18:22:48.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.548+0000 7fc998a07700 1 -- 172.21.15.40:0/4254408205 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9940c7400 msgr2=0x7fc9940c4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:48.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.548+0000 7fc998a07700 1 --2- 172.21.15.40:0/4254408205 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9940c7400 0x7fc9940c4dd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009e50 tx=0x7fc98c00b330 comp rx=0 tx=0).stop 2024-09-13T18:22:48.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.548+0000 7fc998a07700 1 -- 172.21.15.40:0/4254408205 shutdown_connections 2024-09-13T18:22:48.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.548+0000 7fc998a07700 1 --2- 172.21.15.40:0/4254408205 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9940c7400 0x7fc9940c4dd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.548+0000 7fc998a07700 1 --2- 172.21.15.40:0/4254408205 >> v2:172.21.15.43:3300/0 conn(0x7fc9941510d0 0x7fc994157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.548+0000 7fc998a07700 1 -- 172.21.15.40:0/4254408205 >> 172.21.15.40:0/4254408205 conn(0x7fc9940c7d70 msgr2=0x7fc9940c8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:48.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.548+0000 7fc998a07700 1 -- 172.21.15.40:0/4254408205 shutdown_connections 2024-09-13T18:22:48.557 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.548+0000 7fc998a07700 1 -- 172.21.15.40:0/4254408205 wait complete. 2024-09-13T18:22:48.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc998a07700 1 Processor -- start 2024-09-13T18:22:48.558 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc998a07700 1 -- start start 2024-09-13T18:22:48.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc998a07700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9940c7400 0x7fc99414cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:48.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc998a07700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9941510d0 0x7fc99414fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:48.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc998a07700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9941514b0 0x7fc994150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:48.560 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc998a07700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994154b10 con 0x7fc9941514b0 2024-09-13T18:22:48.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc998a07700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9940713c0 con 0x7fc9940c7400 2024-09-13T18:22:48.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc998a07700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc994151ee0 con 0x7fc9941510d0 2024-09-13T18:22:48.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc993fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9941514b0 0x7fc994150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:48.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc993fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9941514b0 0x7fc994150270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48746/0 (socket says 172.21.15.40:48746) 2024-09-13T18:22:48.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc993fff700 1 -- 172.21.15.40:0/1921296722 learned_addr learned my addr 172.21.15.40:0/1921296722 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:22:48.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc9937fe700 1 --2- 172.21.15.40:0/1921296722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9940c7400 0x7fc99414cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:48.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc992ffd700 1 --2- 172.21.15.40:0/1921296722 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9941510d0 0x7fc99414fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:48.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc993fff700 1 -- 172.21.15.40:0/1921296722 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9941510d0 msgr2=0x7fc99414fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:48.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc993fff700 1 --2- 172.21.15.40:0/1921296722 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9941510d0 0x7fc99414fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc993fff700 1 -- 172.21.15.40:0/1921296722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9940c7400 msgr2=0x7fc99414cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:48.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc993fff700 1 --2- 172.21.15.40:0/1921296722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9940c7400 0x7fc99414cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc993fff700 1 -- 172.21.15.40:0/1921296722 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98c009b30 con 0x7fc9941514b0 2024-09-13T18:22:48.561 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc993fff700 1 --2- 172.21.15.40:0/1921296722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9941514b0 0x7fc994150270 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc994158320 tx=0x7fc98c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:48.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc97bfff700 1 -- 172.21.15.40:0/1921296722 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c004250 con 0x7fc9941514b0 2024-09-13T18:22:48.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc97bfff700 1 -- 172.21.15.40:0/1921296722 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc98c0043b0 con 0x7fc9941514b0 2024-09-13T18:22:48.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc998a07700 1 -- 172.21.15.40:0/1921296722 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99414f100 con 0x7fc9941514b0 2024-09-13T18:22:48.563 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc97bfff700 1 -- 172.21.15.40:0/1921296722 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c004780 con 0x7fc9941514b0 2024-09-13T18:22:48.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.552+0000 7fc998a07700 1 -- 172.21.15.40:0/1921296722 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc994070e00 con 0x7fc9941514b0 2024-09-13T18:22:48.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.556+0000 7fc998a07700 1 -- 172.21.15.40:0/1921296722 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc994152050 con 0x7fc9941514b0 2024-09-13T18:22:48.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.556+0000 7fc97bfff700 1 -- 172.21.15.40:0/1921296722 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc98c01c440 con 0x7fc9941514b0 2024-09-13T18:22:48.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.556+0000 7fc97bfff700 1 --2- 172.21.15.40:0/1921296722 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc980066730 0x7fc980068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:48.564 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.556+0000 7fc97bfff700 1 -- 172.21.15.40:0/1921296722 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc98c085f20 con 0x7fc9941514b0 2024-09-13T18:22:48.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.556+0000 7fc9937fe700 1 --2- 172.21.15.40:0/1921296722 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc980066730 0x7fc980068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:48.565 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.556+0000 7fc9937fe700 1 --2- 172.21.15.40:0/1921296722 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc980066730 0x7fc980068bf0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc984009bf0 tx=0x7fc984009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:48.571 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.564+0000 7fc97bfff700 1 -- 172.21.15.40:0/1921296722 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc98c053000 con 0x7fc9941514b0 2024-09-13T18:22:48.774 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.768+0000 7fc998a07700 1 -- 172.21.15.40:0/1921296722 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc994066090 con 0x7fc9941514b0 2024-09-13T18:22:48.776 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.768+0000 7fc97bfff700 1 -- 172.21.15.40:0/1921296722 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+6268 (secure 0 0 0) 0x7fc98c020020 con 0x7fc9941514b0 2024-09-13T18:22:48.776 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T18:22:48.776 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6839/1861594206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6838","nonce":1861594206},{"type":"v1","addr":"172.21.15.40:6839","nonce":1861594206}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/1548394691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":1548394691},{"type":"v1","addr":"172.21.15.40:6835","nonce":1548394691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6837/486611242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6836","nonce":486611242},{"type":"v1","addr":"172.21.15.40:6837","nonce":486611242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:50:34.562623+0000","modified":"2024-09-13T17:51:12.236489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4288,"mds_1":4285,"mds_2":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"f","rank":2,"incarnation":14,"state":"up:active","state_seq":12,"addr":"172.21.15.43:6833/526975022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":526975022},{"type":"v1","addr":"172.21.15.43:6833","nonce":526975022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4285":{"gid":4285,"name":"d","rank":1,"incarnation":10,"state":"up:active","state_seq":11,"addr":"172.21.15.43:6835/2586910944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2586910944},{"type":"v1","addr":"172.21.15.43:6835","nonce":2586910944}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/334160033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":334160033},{"type":"v1","addr":"172.21.15.43:6837","nonce":334160033}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:22:48.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.772+0000 7fc998a07700 1 -- 172.21.15.40:0/1921296722 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc980066730 msgr2=0x7fc980068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:48.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.772+0000 7fc998a07700 1 --2- 172.21.15.40:0/1921296722 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc980066730 0x7fc980068bf0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc984009bf0 tx=0x7fc984009240 comp rx=0 tx=0).stop 2024-09-13T18:22:48.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.772+0000 7fc998a07700 1 -- 172.21.15.40:0/1921296722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9941514b0 msgr2=0x7fc994150270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:48.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.772+0000 7fc998a07700 1 --2- 172.21.15.40:0/1921296722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9941514b0 0x7fc994150270 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc994158320 tx=0x7fc98c004af0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.779 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.772+0000 7fc998a07700 1 -- 172.21.15.40:0/1921296722 shutdown_connections 2024-09-13T18:22:48.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.772+0000 7fc998a07700 1 --2- 172.21.15.40:0/1921296722 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9941514b0 0x7fc994150270 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.772+0000 7fc998a07700 1 --2- 172.21.15.40:0/1921296722 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fc980066730 0x7fc980068bf0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.772+0000 7fc998a07700 1 --2- 172.21.15.40:0/1921296722 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc9941510d0 0x7fc99414fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.772+0000 7fc998a07700 1 --2- 172.21.15.40:0/1921296722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9940c7400 0x7fc99414cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.772+0000 7fc998a07700 1 -- 172.21.15.40:0/1921296722 >> 172.21.15.40:0/1921296722 conn(0x7fc9940c7d70 msgr2=0x7fc9940c9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:48.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.772+0000 7fc998a07700 1 -- 172.21.15.40:0/1921296722 shutdown_connections 2024-09-13T18:22:48.780 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.772+0000 7fc998a07700 1 -- 172.21.15.40:0/1921296722 wait complete. 2024-09-13T18:22:48.780 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 15 2024-09-13T18:22:48.797 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:22:48.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.896+0000 7f00cd0a4700 1 Processor -- start 2024-09-13T18:22:48.905 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.896+0000 7f00cd0a4700 1 -- start start 2024-09-13T18:22:48.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00cd0a4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00c815cbc0 0x7f00c806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:48.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00cd0a4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f00c815cfa0 0x7f00c815d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:48.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00cd0a4700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c806eb10 con 0x7f00c8153120 2024-09-13T18:22:48.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00cd0a4700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c80c7bb0 con 0x7f00c815cfa0 2024-09-13T18:22:48.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00cd0a4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c806b4c0 con 0x7f00c815cbc0 2024-09-13T18:22:48.907 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00c77fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f00c815cfa0 0x7f00c815d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:48.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00c77fe700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f00c815cfa0 0x7f00c815d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48754/0 (socket says 172.21.15.40:48754) 2024-09-13T18:22:48.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00c77fe700 1 -- 172.21.15.40:0/1037979802 learned_addr learned my addr 172.21.15.40:0/1037979802 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:22:48.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00cc8a3700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f00c8153120 0x7f00c806f9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33396/0 (socket says 172.21.15.40:33396) 2024-09-13T18:22:48.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00c7fff700 1 --2- 172.21.15.40:0/1037979802 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00c815cbc0 0x7f00c806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:48.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00c77fe700 1 -- 172.21.15.40:0/1037979802 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00c815cbc0 msgr2=0x7f00c806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:48.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00c77fe700 1 --2- 172.21.15.40:0/1037979802 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00c815cbc0 0x7f00c806e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00c77fe700 1 -- 172.21.15.40:0/1037979802 >> v1:172.21.15.40:6789/0 conn(0x7f00c8153120 legacy=0x7f00c806f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:48.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00c77fe700 1 -- 172.21.15.40:0/1037979802 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c806b7a0 con 0x7f00c815cfa0 2024-09-13T18:22:48.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00c7fff700 1 --2- 172.21.15.40:0/1037979802 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00c815cbc0 0x7f00c806e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:48.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00c77fe700 1 --2- 172.21.15.40:0/1037979802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c815cfa0 0x7f00c815d3a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009910 tx=0x7f00bc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89ff53838bc9cff server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:48.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00c6ffd700 1 -- 172.21.15.40:0/1037979802 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc00ba60 con 0x7f00c815cfa0 2024-09-13T18:22:48.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00c6ffd700 1 -- 172.21.15.40:0/1037979802 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f00bc00bbc0 con 0x7f00c815cfa0 2024-09-13T18:22:48.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00c6ffd700 1 -- 172.21.15.40:0/1037979802 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc009e60 con 0x7f00c815cfa0 2024-09-13T18:22:48.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00cd0a4700 1 -- 172.21.15.40:0/1037979802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c815cfa0 msgr2=0x7f00c815d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:48.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00cd0a4700 1 --2- 172.21.15.40:0/1037979802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c815cfa0 0x7f00c815d3a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009910 tx=0x7f00bc009c20 comp rx=0 tx=0).stop 2024-09-13T18:22:48.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00cd0a4700 1 -- 172.21.15.40:0/1037979802 shutdown_connections 2024-09-13T18:22:48.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00cd0a4700 1 --2- 172.21.15.40:0/1037979802 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c815cfa0 0x7f00c815d3a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00cd0a4700 1 --2- 172.21.15.40:0/1037979802 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00c815cbc0 0x7f00c806e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00cd0a4700 1 -- 172.21.15.40:0/1037979802 >> 172.21.15.40:0/1037979802 conn(0x7f00c80ccef0 msgr2=0x7f00c80cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:48.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00cd0a4700 1 -- 172.21.15.40:0/1037979802 shutdown_connections 2024-09-13T18:22:48.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00cd0a4700 1 -- 172.21.15.40:0/1037979802 wait complete. 2024-09-13T18:22:48.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00cd0a4700 1 Processor -- start 2024-09-13T18:22:48.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.900+0000 7f00cd0a4700 1 -- start start 2024-09-13T18:22:48.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00cd0a4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00c8153120 0x7f00c815bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:48.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00cd0a4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00c815cbc0 0x7f00c815c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:48.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00cd0a4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c815cfa0 0x7f00c814acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:48.912 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00cd0a4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c815c740 con 0x7f00c815cfa0 2024-09-13T18:22:48.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00cd0a4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c8159740 con 0x7f00c8153120 2024-09-13T18:22:48.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00cd0a4700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c8147590 con 0x7f00c815cbc0 2024-09-13T18:22:48.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00cc8a3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c815cfa0 0x7f00c814acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:48.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00cc8a3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c815cfa0 0x7f00c814acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48760/0 (socket says 172.21.15.40:48760) 2024-09-13T18:22:48.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00cc8a3700 1 -- 172.21.15.40:0/2980851112 learned_addr learned my addr 172.21.15.40:0/2980851112 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:22:48.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00c7fff700 1 --2- 172.21.15.40:0/2980851112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00c8153120 0x7f00c815bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:48.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00c77fe700 1 --2- 172.21.15.40:0/2980851112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00c815cbc0 0x7f00c815c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:48.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00cc8a3700 1 -- 172.21.15.40:0/2980851112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00c815cbc0 msgr2=0x7f00c815c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:48.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00cc8a3700 1 --2- 172.21.15.40:0/2980851112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00c815cbc0 0x7f00c815c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00cc8a3700 1 -- 172.21.15.40:0/2980851112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00c8153120 msgr2=0x7f00c815bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:48.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00cc8a3700 1 --2- 172.21.15.40:0/2980851112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00c8153120 0x7f00c815bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:48.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00cc8a3700 1 -- 172.21.15.40:0/2980851112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc009570 con 0x7f00c815cfa0 2024-09-13T18:22:48.913 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00c7fff700 1 --2- 172.21.15.40:0/2980851112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00c8153120 0x7f00c815bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:22:48.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00cc8a3700 1 --2- 172.21.15.40:0/2980851112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c815cfa0 0x7f00c814acb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f00c000cbd0 tx=0x7f00c000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:48.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00c4ff9700 1 -- 172.21.15.40:0/2980851112 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c000abb0 con 0x7f00c815cfa0 2024-09-13T18:22:48.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00c4ff9700 1 -- 172.21.15.40:0/2980851112 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f00c000ad10 con 0x7f00c815cfa0 2024-09-13T18:22:48.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00cd0a4700 1 -- 172.21.15.40:0/2980851112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c8157560 con 0x7f00c815cfa0 2024-09-13T18:22:48.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00c4ff9700 1 -- 172.21.15.40:0/2980851112 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c0007270 con 0x7f00c815cfa0 2024-09-13T18:22:48.914 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00cd0a4700 1 -- 172.21.15.40:0/2980851112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c8159460 con 0x7f00c815cfa0 2024-09-13T18:22:48.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00cd0a4700 1 -- 172.21.15.40:0/2980851112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c8157a60 con 0x7f00c815cfa0 2024-09-13T18:22:48.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00c4ff9700 1 -- 172.21.15.40:0/2980851112 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f00c00073d0 con 0x7f00c815cfa0 2024-09-13T18:22:48.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00c4ff9700 1 --2- 172.21.15.40:0/2980851112 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f00b4066760 0x7f00b4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:48.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00c4ff9700 1 -- 172.21.15.40:0/2980851112 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f00c00842d0 con 0x7f00c815cfa0 2024-09-13T18:22:48.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.904+0000 7f00c7fff700 1 --2- 172.21.15.40:0/2980851112 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f00b4066760 0x7f00b4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:48.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.908+0000 7f00c7fff700 1 --2- 172.21.15.40:0/2980851112 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f00b4066760 0x7f00b4068c20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f00b8009a20 tx=0x7f00b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:48.918 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:48.912+0000 7f00c4ff9700 1 -- 172.21.15.40:0/2980851112 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f00c8157a60 con 0x7f00c815cfa0 2024-09-13T18:22:49.086 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.080+0000 7f00cd0a4700 1 -- 172.21.15.40:0/2980851112 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00c8157a60 con 0x7f00c815cfa0 2024-09-13T18:22:49.089 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.080+0000 7f00c4ff9700 1 -- 172.21.15.40:0/2980851112 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18650 (secure 0 0 0) 0x7f00c8157a60 con 0x7f00c815cfa0 2024-09-13T18:22:49.090 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-09-13T18:22:49.090 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":27,"fsid":"841d6eb0-79fe-4108-906c-e8798e475996","created":"2024-09-13T17:50:09.558780+0000","modified":"2024-09-13T17:51:16.266176+0000","last_up_change":"2024-09-13T17:50:21.057812+0000","last_in_change":"2024-09-13T17:50:12.518018+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:50:21.057171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:50:32.740336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:50:33.552914+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-13T17:50:36.623639+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89379989-479d-4bd7-b339-d984d636856c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6817","nonce":3493711342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6819","nonce":3493711342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6823","nonce":3493711342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3493711342},{"type":"v1","addr":"172.21.15.40:6821","nonce":3493711342}]},"public_addr":"172.21.15.40:6817/3493711342","cluster_addr":"172.21.15.40:6819/3493711342","heartbeat_back_addr":"172.21.15.40:6823/3493711342","heartbeat_front_addr":"172.21.15.40:6821/3493711342","state":["exists","up"]},{"osd":1,"uuid":"bb549a81-885b-4473-9158-943c07955975","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6809","nonce":2576246869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6811","nonce":2576246869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6815","nonce":2576246869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":2576246869},{"type":"v1","addr":"172.21.15.40:6813","nonce":2576246869}]},"public_addr":"172.21.15.40:6809/2576246869","cluster_addr":"172.21.15.40:6811/2576246869","heartbeat_back_addr":"172.21.15.40:6815/2576246869","heartbeat_front_addr":"172.21.15.40:6813/2576246869","state":["exists","up"]},{"osd":2,"uuid":"d9af7ab2-8b2b-4bbf-a896-9adfbf9fb268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6801","nonce":44856195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6803","nonce":44856195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6807","nonce":44856195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":44856195},{"type":"v1","addr":"172.21.15.40:6805","nonce":44856195}]},"public_addr":"172.21.15.40:6801/44856195","cluster_addr":"172.21.15.40:6803/44856195","heartbeat_back_addr":"172.21.15.40:6807/44856195","heartbeat_front_addr":"172.21.15.40:6805/44856195","state":["exists","up"]},{"osd":3,"uuid":"df6ebccc-cd93-4154-9559-d2724c12fcae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6825","nonce":1926907414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6827","nonce":1926907414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6830","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6831","nonce":1926907414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":1926907414},{"type":"v1","addr":"172.21.15.40:6829","nonce":1926907414}]},"public_addr":"172.21.15.40:6825/1926907414","cluster_addr":"172.21.15.40:6827/1926907414","heartbeat_back_addr":"172.21.15.40:6831/1926907414","heartbeat_front_addr":"172.21.15.40:6829/1926907414","state":["exists","up"]},{"osd":4,"uuid":"7780b8ba-bfbc-4d79-a70a-36373693a8de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6825","nonce":3105831381}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6827","nonce":3105831381}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6831","nonce":3105831381}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3105831381},{"type":"v1","addr":"172.21.15.43:6829","nonce":3105831381}]},"public_addr":"172.21.15.43:6825/3105831381","cluster_addr":"172.21.15.43:6827/3105831381","heartbeat_back_addr":"172.21.15.43:6831/3105831381","heartbeat_front_addr":"172.21.15.43:6829/3105831381","state":["exists","up"]},{"osd":5,"uuid":"b46e4b51-7b2e-42f0-ae9f-1ae8f49ed33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6801","nonce":858063952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6803","nonce":858063952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6807","nonce":858063952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":858063952},{"type":"v1","addr":"172.21.15.43:6805","nonce":858063952}]},"public_addr":"172.21.15.43:6801/858063952","cluster_addr":"172.21.15.43:6803/858063952","heartbeat_back_addr":"172.21.15.43:6807/858063952","heartbeat_front_addr":"172.21.15.43:6805/858063952","state":["exists","up"]},{"osd":6,"uuid":"a1c0a15b-f29b-477c-a0fa-63832c91da48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6809","nonce":480073165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6811","nonce":480073165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6815","nonce":480073165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":480073165},{"type":"v1","addr":"172.21.15.43:6813","nonce":480073165}]},"public_addr":"172.21.15.43:6809/480073165","cluster_addr":"172.21.15.43:6811/480073165","heartbeat_back_addr":"172.21.15.43:6815/480073165","heartbeat_front_addr":"172.21.15.43:6813/480073165","state":["exists","up"]},{"osd":7,"uuid":"708c7296-d725-48c1-b96e-ecf8975d02fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6817","nonce":2080568799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6819","nonce":2080568799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6823","nonce":2080568799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2080568799},{"type":"v1","addr":"172.21.15.43:6821","nonce":2080568799}]},"public_addr":"172.21.15.43:6817/2080568799","cluster_addr":"172.21.15.43:6819/2080568799","heartbeat_back_addr":"172.21.15.43:6823/2080568799","heartbeat_front_addr":"172.21.15.43:6821/2080568799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.540290+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.812938+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.576555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.920744+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:18.050996+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.759602+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.389282+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:50:17.345033+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:22:49.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.084+0000 7f00cd0a4700 1 -- 172.21.15.40:0/2980851112 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f00b4066760 msgr2=0x7f00b4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:49.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.084+0000 7f00cd0a4700 1 --2- 172.21.15.40:0/2980851112 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f00b4066760 0x7f00b4068c20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f00b8009a20 tx=0x7f00b8008040 comp rx=0 tx=0).stop 2024-09-13T18:22:49.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.084+0000 7f00cd0a4700 1 -- 172.21.15.40:0/2980851112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c815cfa0 msgr2=0x7f00c814acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:49.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.084+0000 7f00cd0a4700 1 --2- 172.21.15.40:0/2980851112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c815cfa0 0x7f00c814acb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f00c000cbd0 tx=0x7f00c000cf90 comp rx=0 tx=0).stop 2024-09-13T18:22:49.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.084+0000 7f00cd0a4700 1 -- 172.21.15.40:0/2980851112 shutdown_connections 2024-09-13T18:22:49.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.084+0000 7f00cd0a4700 1 --2- 172.21.15.40:0/2980851112 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00c815cfa0 0x7f00c814acb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:49.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.084+0000 7f00cd0a4700 1 --2- 172.21.15.40:0/2980851112 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f00b4066760 0x7f00b4068c20 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:49.094 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.084+0000 7f00cd0a4700 1 --2- 172.21.15.40:0/2980851112 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f00c815cbc0 0x7f00c815c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:49.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.084+0000 7f00cd0a4700 1 --2- 172.21.15.40:0/2980851112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00c8153120 0x7f00c815bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:49.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.084+0000 7f00cd0a4700 1 -- 172.21.15.40:0/2980851112 >> 172.21.15.40:0/2980851112 conn(0x7f00c80ccef0 msgr2=0x7f00c80cd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:49.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.088+0000 7f00cd0a4700 1 -- 172.21.15.40:0/2980851112 shutdown_connections 2024-09-13T18:22:49.095 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.088+0000 7f00cd0a4700 1 -- 172.21.15.40:0/2980851112 wait complete. 2024-09-13T18:22:49.109 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T18:22:49.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.204+0000 7f164e403700 1 Processor -- start 2024-09-13T18:22:49.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.204+0000 7f164e403700 1 -- start start 2024-09-13T18:22:49.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.204+0000 7f164e403700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f16400d5640 0x7f1640045700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:49.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.204+0000 7f164e403700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f16400464b0 0x7f1640045c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:49.215 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.204+0000 7f164e403700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16400d5a20 con 0x7f1640050820 2024-09-13T18:22:49.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.204+0000 7f164e403700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16400dbe40 con 0x7f16400464b0 2024-09-13T18:22:49.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.204+0000 7f164e403700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16400dc120 con 0x7f16400d5640 2024-09-13T18:22:49.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.204+0000 7f164cc00700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1640050820 0x7f1640034b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:33412/0 (socket says 172.21.15.40:33412) 2024-09-13T18:22:49.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.204+0000 7f164cc00700 1 -- 172.21.15.40:0/440804050 learned_addr learned my addr 172.21.15.40:0/440804050 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:22:49.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.204+0000 7f164dc02700 1 --2- 172.21.15.40:0/440804050 >> v2:172.21.15.40:3300/0 conn(0x7f16400464b0 0x7f1640045c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:49.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.204+0000 7f164d401700 1 --2- 172.21.15.40:0/440804050 >> v2:172.21.15.43:3300/0 conn(0x7f16400d5640 0x7f1640045700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:49.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.204+0000 7f163ffff700 1 -- 172.21.15.40:0/440804050 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2513699490 0 0) 0x7f16400d5a20 con 0x7f1640050820 2024-09-13T18:22:49.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.204+0000 7f163ffff700 1 -- 172.21.15.40:0/440804050 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16340035d0 con 0x7f1640050820 2024-09-13T18:22:49.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.204+0000 7f164dc02700 1 -- 172.21.15.40:0/440804050 >> v2:172.21.15.43:3300/0 conn(0x7f16400d5640 msgr2=0x7f1640045700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:49.216 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.204+0000 7f164dc02700 1 --2- 172.21.15.40:0/440804050 >> v2:172.21.15.43:3300/0 conn(0x7f16400d5640 0x7f1640045700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:49.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.204+0000 7f164dc02700 1 -- 172.21.15.40:0/440804050 >> v1:172.21.15.40:6789/0 conn(0x7f1640050820 legacy=0x7f1640034b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:49.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.204+0000 7f164dc02700 1 -- 172.21.15.40:0/440804050 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1640005b00 con 0x7f16400464b0 2024-09-13T18:22:49.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164dc02700 1 --2- 172.21.15.40:0/440804050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16400464b0 0x7f1640045c40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1648069620 tx=0x7f1648069e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fb80f2adb2150b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:49.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f163ffff700 1 -- 172.21.15.40:0/440804050 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1648067dd0 con 0x7f16400464b0 2024-09-13T18:22:49.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f163ffff700 1 -- 172.21.15.40:0/440804050 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1648067f30 con 0x7f16400464b0 2024-09-13T18:22:49.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f163ffff700 1 -- 172.21.15.40:0/440804050 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16480746e0 con 0x7f16400464b0 2024-09-13T18:22:49.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164e403700 1 -- 172.21.15.40:0/440804050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16400464b0 msgr2=0x7f1640045c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:49.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164e403700 1 --2- 172.21.15.40:0/440804050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16400464b0 0x7f1640045c40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1648069620 tx=0x7f1648069e70 comp rx=0 tx=0).stop 2024-09-13T18:22:49.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164e403700 1 -- 172.21.15.40:0/440804050 shutdown_connections 2024-09-13T18:22:49.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164e403700 1 --2- 172.21.15.40:0/440804050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16400464b0 0x7f1640045c40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:49.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164e403700 1 --2- 172.21.15.40:0/440804050 >> v2:172.21.15.43:3300/0 conn(0x7f16400d5640 0x7f1640045700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:49.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164e403700 1 -- 172.21.15.40:0/440804050 >> 172.21.15.40:0/440804050 conn(0x7f164004fd00 msgr2=0x7f1640050110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:49.217 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164e403700 1 -- 172.21.15.40:0/440804050 shutdown_connections 2024-09-13T18:22:49.218 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164e403700 1 -- 172.21.15.40:0/440804050 wait complete. 2024-09-13T18:22:49.219 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164e403700 1 Processor -- start 2024-09-13T18:22:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164e403700 1 -- start start 2024-09-13T18:22:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164e403700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16400464b0 0x7f16400e4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164e403700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1640050820 0x7f16400d4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164e403700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16400d5640 0x7f16400e8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164e403700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16400e4bb0 con 0x7f16400464b0 2024-09-13T18:22:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164e403700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16400ccd10 con 0x7f16400d5640 2024-09-13T18:22:49.220 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164e403700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16400e7600 con 0x7f1640050820 2024-09-13T18:22:49.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164d401700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16400464b0 0x7f16400e4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:49.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164d401700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16400464b0 0x7f16400e4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48778/0 (socket says 172.21.15.40:48778) 2024-09-13T18:22:49.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164d401700 1 -- 172.21.15.40:0/1388672509 learned_addr learned my addr 172.21.15.40:0/1388672509 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:22:49.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164cc00700 1 --2- 172.21.15.40:0/1388672509 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1640050820 0x7f16400d4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:49.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164dc02700 1 --2- 172.21.15.40:0/1388672509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16400d5640 0x7f16400e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:49.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164d401700 1 -- 172.21.15.40:0/1388672509 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1640050820 msgr2=0x7f16400d4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:49.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164d401700 1 --2- 172.21.15.40:0/1388672509 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1640050820 0x7f16400d4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:49.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164d401700 1 -- 172.21.15.40:0/1388672509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16400d5640 msgr2=0x7f16400e8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:49.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164d401700 1 --2- 172.21.15.40:0/1388672509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16400d5640 0x7f16400e8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:49.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.208+0000 7f164d401700 1 -- 172.21.15.40:0/1388672509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164804f5c0 con 0x7f16400464b0 2024-09-13T18:22:49.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.212+0000 7f164d401700 1 --2- 172.21.15.40:0/1388672509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16400464b0 0x7f16400e4670 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f164400e840 tx=0x7f164400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:49.221 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.212+0000 7f163dffb700 1 -- 172.21.15.40:0/1388672509 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16440095a0 con 0x7f16400464b0 2024-09-13T18:22:49.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.212+0000 7f163dffb700 1 -- 172.21.15.40:0/1388672509 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f164400c690 con 0x7f16400464b0 2024-09-13T18:22:49.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.212+0000 7f164e403700 1 -- 172.21.15.40:0/1388672509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16400cd260 con 0x7f16400464b0 2024-09-13T18:22:49.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.212+0000 7f163dffb700 1 -- 172.21.15.40:0/1388672509 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164400ca20 con 0x7f16400464b0 2024-09-13T18:22:49.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.212+0000 7f164e403700 1 -- 172.21.15.40:0/1388672509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16400e6870 con 0x7f16400464b0 2024-09-13T18:22:49.222 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.212+0000 7f164e403700 1 -- 172.21.15.40:0/1388672509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16400355d0 con 0x7f16400464b0 2024-09-13T18:22:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.212+0000 7f163dffb700 1 -- 172.21.15.40:0/1388672509 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f164400cb80 con 0x7f16400464b0 2024-09-13T18:22:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.212+0000 7f163dffb700 1 --2- 172.21.15.40:0/1388672509 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f162c066730 0x7f162c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.216+0000 7f164cc00700 1 --2- 172.21.15.40:0/1388672509 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f162c066730 0x7f162c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.216+0000 7f163dffb700 1 -- 172.21.15.40:0/1388672509 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1644084630 con 0x7f16400464b0 2024-09-13T18:22:49.223 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.216+0000 7f164cc00700 1 --2- 172.21.15.40:0/1388672509 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f162c066730 0x7f162c068bf0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f164004c420 tx=0x7f1638009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:49.226 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.220+0000 7f163dffb700 1 -- 172.21.15.40:0/1388672509 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f16440514a0 con 0x7f16400464b0 2024-09-13T18:22:49.431 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.424+0000 7f164e403700 1 -- 172.21.15.40:0/1388672509 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f16400337f0 con 0x7f16400464b0 2024-09-13T18:22:49.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.456+0000 7f163dffb700 1 -- 172.21.15.40:0/1388672509 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v16) v1 ==== 161+0+0 (secure 0 0 0) 0x7f16440533d0 con 0x7f16400464b0 2024-09-13T18:22:49.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.460+0000 7f164e403700 1 -- 172.21.15.40:0/1388672509 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f162c066730 msgr2=0x7f162c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:49.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.460+0000 7f164e403700 1 --2- 172.21.15.40:0/1388672509 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f162c066730 0x7f162c068bf0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f164004c420 tx=0x7f1638009040 comp rx=0 tx=0).stop 2024-09-13T18:22:49.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.460+0000 7f164e403700 1 -- 172.21.15.40:0/1388672509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16400464b0 msgr2=0x7f16400e4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:49.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.460+0000 7f164e403700 1 --2- 172.21.15.40:0/1388672509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16400464b0 0x7f16400e4670 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f164400e840 tx=0x7f164400ec00 comp rx=0 tx=0).stop 2024-09-13T18:22:49.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.460+0000 7f164e403700 1 -- 172.21.15.40:0/1388672509 shutdown_connections 2024-09-13T18:22:49.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.460+0000 7f164e403700 1 --2- 172.21.15.40:0/1388672509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16400d5640 0x7f16400e8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:49.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.460+0000 7f164e403700 1 --2- 172.21.15.40:0/1388672509 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f162c066730 0x7f162c068bf0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:49.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.460+0000 7f164e403700 1 --2- 172.21.15.40:0/1388672509 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1640050820 0x7f16400d4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:49.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.460+0000 7f164e403700 1 --2- 172.21.15.40:0/1388672509 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f16400464b0 0x7f16400e4670 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:49.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.460+0000 7f164e403700 1 -- 172.21.15.40:0/1388672509 >> 172.21.15.40:0/1388672509 conn(0x7f164004fd00 msgr2=0x7f16400500e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:49.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.460+0000 7f164e403700 1 -- 172.21.15.40:0/1388672509 shutdown_connections 2024-09-13T18:22:49.469 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.460+0000 7f164e403700 1 -- 172.21.15.40:0/1388672509 wait complete. 2024-09-13T18:22:49.469 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T18:22:49.483 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T18:22:49.535 INFO:tasks.ceph.mds.b.smithi043.stdout:starting mds.b at 2024-09-13T18:22:49.535 INFO:tasks.ceph.mds.d.smithi043.stdout:starting mds.d at 2024-09-13T18:22:49.535 INFO:tasks.ceph.mds.f.smithi043.stdout:starting mds.f at 2024-09-13T18:22:49.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.584+0000 7fda57fff700 1 Processor -- start 2024-09-13T18:22:49.591 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.584+0000 7fda57fff700 1 -- start start 2024-09-13T18:22:49.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.584+0000 7fda57fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fda580c8500 0x7fda580c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:49.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.584+0000 7fda57fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda580c71c0 0x7fda58159630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:49.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.584+0000 7fda57fff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda580c8f80 con 0x7fda581503c0 2024-09-13T18:22:49.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.584+0000 7fda57fff700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda580c2a80 con 0x7fda580c8500 2024-09-13T18:22:49.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.584+0000 7fda57fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda580c80d0 con 0x7fda580c71c0 2024-09-13T18:22:49.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.584+0000 7fda567fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fda580c8500 0x7fda580c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:49.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.584+0000 7fda567fc700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fda580c8500 0x7fda580c6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48792/0 (socket says 172.21.15.40:48792) 2024-09-13T18:22:49.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.584+0000 7fda567fc700 1 -- 172.21.15.40:0/3300947782 learned_addr learned my addr 172.21.15.40:0/3300947782 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:22:49.594 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.584+0000 7fda577fe700 1 --2- 172.21.15.40:0/3300947782 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda580c71c0 0x7fda58159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:49.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.584+0000 7fda567fc700 1 -- 172.21.15.40:0/3300947782 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda580c71c0 msgr2=0x7fda58159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:49.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.584+0000 7fda567fc700 1 --2- 172.21.15.40:0/3300947782 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda580c71c0 0x7fda58159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:49.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.584+0000 7fda567fc700 1 -- 172.21.15.40:0/3300947782 >> v1:172.21.15.43:6789/0 conn(0x7fda581503c0 legacy=0x7fda580c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:49.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.584+0000 7fda567fc700 1 -- 172.21.15.40:0/3300947782 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda581507a0 con 0x7fda580c8500 2024-09-13T18:22:49.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.584+0000 7fda567fc700 1 --2- 172.21.15.40:0/3300947782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda580c8500 0x7fda580c6c80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fda48009910 tx=0x7fda48009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49553d39d0f970f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:49.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.588+0000 7fda55ffb700 1 -- 172.21.15.40:0/3300947782 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4800ba60 con 0x7fda580c8500 2024-09-13T18:22:49.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.588+0000 7fda55ffb700 1 -- 172.21.15.40:0/3300947782 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fda4800bbc0 con 0x7fda580c8500 2024-09-13T18:22:49.595 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.588+0000 7fda55ffb700 1 -- 172.21.15.40:0/3300947782 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda48009e60 con 0x7fda580c8500 2024-09-13T18:22:49.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.588+0000 7fda57fff700 1 -- 172.21.15.40:0/3300947782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda580c8500 msgr2=0x7fda580c6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:49.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.588+0000 7fda57fff700 1 --2- 172.21.15.40:0/3300947782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda580c8500 0x7fda580c6c80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fda48009910 tx=0x7fda48009c20 comp rx=0 tx=0).stop 2024-09-13T18:22:49.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.588+0000 7fda57fff700 1 -- 172.21.15.40:0/3300947782 shutdown_connections 2024-09-13T18:22:49.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.588+0000 7fda57fff700 1 --2- 172.21.15.40:0/3300947782 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda580c71c0 0x7fda58159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:49.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.588+0000 7fda57fff700 1 --2- 172.21.15.40:0/3300947782 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda580c8500 0x7fda580c6c80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:49.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.588+0000 7fda57fff700 1 -- 172.21.15.40:0/3300947782 >> 172.21.15.40:0/3300947782 conn(0x7fda5806e6d0 msgr2=0x7fda5806eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:49.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.588+0000 7fda57fff700 1 -- 172.21.15.40:0/3300947782 shutdown_connections 2024-09-13T18:22:49.596 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.588+0000 7fda57fff700 1 -- 172.21.15.40:0/3300947782 wait complete. 2024-09-13T18:22:49.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.588+0000 7fda57fff700 1 Processor -- start 2024-09-13T18:22:49.597 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.588+0000 7fda57fff700 1 -- start start 2024-09-13T18:22:49.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda57fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda580c71c0 0x7fda581ee680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:49.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda57fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda580c8500 0x7fda58152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:49.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda57fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda581503c0 0x7fda58143250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:49.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda57fff700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda581f0f70 con 0x7fda581503c0 2024-09-13T18:22:49.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda57fff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda58157330 con 0x7fda580c71c0 2024-09-13T18:22:49.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda57fff700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda581524e0 con 0x7fda580c8500 2024-09-13T18:22:49.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda577fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda581503c0 0x7fda58143250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:49.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda56ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda580c71c0 0x7fda581ee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:49.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda577fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda581503c0 0x7fda58143250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48796/0 (socket says 172.21.15.40:48796) 2024-09-13T18:22:49.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda577fe700 1 -- 172.21.15.40:0/167188156 learned_addr learned my addr 172.21.15.40:0/167188156 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:22:49.599 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda567fc700 1 --2- 172.21.15.40:0/167188156 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda580c8500 0x7fda58152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:49.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda577fe700 1 -- 172.21.15.40:0/167188156 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda580c8500 msgr2=0x7fda58152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:49.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda577fe700 1 --2- 172.21.15.40:0/167188156 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda580c8500 0x7fda58152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:49.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda577fe700 1 -- 172.21.15.40:0/167188156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda580c71c0 msgr2=0x7fda581ee680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:49.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda577fe700 1 --2- 172.21.15.40:0/167188156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda580c71c0 0x7fda581ee680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:49.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda577fe700 1 -- 172.21.15.40:0/167188156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda48009570 con 0x7fda581503c0 2024-09-13T18:22:49.600 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda56ffd700 1 --2- 172.21.15.40:0/167188156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda580c71c0 0x7fda581ee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:22:49.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda577fe700 1 --2- 172.21.15.40:0/167188156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda581503c0 0x7fda58143250 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fda50007e40 tx=0x7fda5000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:49.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda3f7fe700 1 -- 172.21.15.40:0/167188156 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5000ecd0 con 0x7fda581503c0 2024-09-13T18:22:49.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda3f7fe700 1 -- 172.21.15.40:0/167188156 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fda5000ee50 con 0x7fda581503c0 2024-09-13T18:22:49.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda57fff700 1 -- 172.21.15.40:0/167188156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda581eebc0 con 0x7fda581503c0 2024-09-13T18:22:49.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda3f7fe700 1 -- 172.21.15.40:0/167188156 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda50014680 con 0x7fda581503c0 2024-09-13T18:22:49.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda57fff700 1 -- 172.21.15.40:0/167188156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda58150f10 con 0x7fda581503c0 2024-09-13T18:22:49.601 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.592+0000 7fda57fff700 1 -- 172.21.15.40:0/167188156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda581f10b0 con 0x7fda581503c0 2024-09-13T18:22:49.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.596+0000 7fda3f7fe700 1 -- 172.21.15.40:0/167188156 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fda500147e0 con 0x7fda581503c0 2024-09-13T18:22:49.603 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.596+0000 7fda3f7fe700 1 --2- 172.21.15.40:0/167188156 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fda44066730 0x7fda44068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:49.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.596+0000 7fda3f7fe700 1 -- 172.21.15.40:0/167188156 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6921+0+0 (secure 0 0 0) 0x7fda50085910 con 0x7fda581503c0 2024-09-13T18:22:49.604 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.596+0000 7fda56ffd700 1 --2- 172.21.15.40:0/167188156 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fda44066730 0x7fda44068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:49.605 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.596+0000 7fda56ffd700 1 --2- 172.21.15.40:0/167188156 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fda44066730 0x7fda44068bf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fda4c000f60 tx=0x7fda4c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:49.606 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.600+0000 7fda3f7fe700 1 -- 172.21.15.40:0/167188156 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fda500528a0 con 0x7fda581503c0 2024-09-13T18:22:49.817 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:49.808+0000 7fda57fff700 1 -- 172.21.15.40:0/167188156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fda581f10b0 con 0x7fda581503c0 2024-09-13T18:22:50.473 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.464+0000 7fda3f7fe700 1 -- 172.21.15.40:0/167188156 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fda581f10b0 con 0x7fda581503c0 2024-09-13T18:22:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.472+0000 7fda57fff700 1 -- 172.21.15.40:0/167188156 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fda44066730 msgr2=0x7fda44068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.472+0000 7fda57fff700 1 --2- 172.21.15.40:0/167188156 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fda44066730 0x7fda44068bf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fda4c000f60 tx=0x7fda4c009040 comp rx=0 tx=0).stop 2024-09-13T18:22:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.472+0000 7fda57fff700 1 -- 172.21.15.40:0/167188156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda581503c0 msgr2=0x7fda58143250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.472+0000 7fda57fff700 1 --2- 172.21.15.40:0/167188156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda581503c0 0x7fda58143250 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fda50007e40 tx=0x7fda5000e3c0 comp rx=0 tx=0).stop 2024-09-13T18:22:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.472+0000 7fda57fff700 1 -- 172.21.15.40:0/167188156 shutdown_connections 2024-09-13T18:22:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.472+0000 7fda57fff700 1 --2- 172.21.15.40:0/167188156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda581503c0 0x7fda58143250 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.472+0000 7fda57fff700 1 --2- 172.21.15.40:0/167188156 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7fda44066730 0x7fda44068bf0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.472+0000 7fda57fff700 1 --2- 172.21.15.40:0/167188156 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fda580c8500 0x7fda58152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.472+0000 7fda57fff700 1 --2- 172.21.15.40:0/167188156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda580c71c0 0x7fda581ee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.472+0000 7fda57fff700 1 -- 172.21.15.40:0/167188156 >> 172.21.15.40:0/167188156 conn(0x7fda5806e6d0 msgr2=0x7fda58145680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:50.479 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.472+0000 7fda57fff700 1 -- 172.21.15.40:0/167188156 shutdown_connections 2024-09-13T18:22:50.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.472+0000 7fda57fff700 1 -- 172.21.15.40:0/167188156 wait complete. 2024-09-13T18:22:50.492 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T18:22:50.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.600+0000 7f3009aa3700 1 Processor -- start 2024-09-13T18:22:50.609 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.600+0000 7f3009aa3700 1 -- start start 2024-09-13T18:22:50.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.600+0000 7f3009aa3700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3004071360 0x7f300414ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:50.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.600+0000 7f3009aa3700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3004070350 con 0x7f3004070f80 2024-09-13T18:22:50.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.604+0000 7f3009aa3700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30040705f0 con 0x7f3004066420 2024-09-13T18:22:50.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.604+0000 7f3009aa3700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3004156c90 con 0x7f3004071360 2024-09-13T18:22:50.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.604+0000 7f3003fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3004071360 0x7f300414ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:50.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.604+0000 7f3003fff700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3004071360 0x7f300414ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48812/0 (socket says 172.21.15.40:48812) 2024-09-13T18:22:50.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.604+0000 7f3003fff700 1 -- 172.21.15.40:0/446686424 learned_addr learned my addr 172.21.15.40:0/446686424 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:22:50.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.604+0000 7f3003fff700 1 -- 172.21.15.40:0/446686424 >> v1:172.21.15.43:6789/0 conn(0x7f3004066420 legacy=0x7f3004160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:50.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.604+0000 7f3003fff700 1 -- 172.21.15.40:0/446686424 >> v1:172.21.15.40:6789/0 conn(0x7f3004070f80 legacy=0x7f3004071fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:50.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.604+0000 7f3003fff700 1 -- 172.21.15.40:0/446686424 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3004156f70 con 0x7f3004071360 2024-09-13T18:22:50.611 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.604+0000 7f3003fff700 1 --2- 172.21.15.40:0/446686424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3004071360 0x7f300414ff90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000bbb0 tx=0x7f2ff000bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44b6e5017448f6f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.604+0000 7f30037fe700 1 -- 172.21.15.40:0/446686424 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff000dd90 con 0x7f3004071360 2024-09-13T18:22:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.604+0000 7f30037fe700 1 -- 172.21.15.40:0/446686424 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2ff0010470 con 0x7f3004071360 2024-09-13T18:22:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.604+0000 7f30037fe700 1 -- 172.21.15.40:0/446686424 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0010840 con 0x7f3004071360 2024-09-13T18:22:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.604+0000 7f3009aa3700 1 -- 172.21.15.40:0/446686424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3004071360 msgr2=0x7f300414ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.604+0000 7f3009aa3700 1 --2- 172.21.15.40:0/446686424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3004071360 0x7f300414ff90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000bbb0 tx=0x7f2ff000bec0 comp rx=0 tx=0).stop 2024-09-13T18:22:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.604+0000 7f3009aa3700 1 -- 172.21.15.40:0/446686424 shutdown_connections 2024-09-13T18:22:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.604+0000 7f3009aa3700 1 --2- 172.21.15.40:0/446686424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3004071360 0x7f300414ff90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.604+0000 7f3009aa3700 1 -- 172.21.15.40:0/446686424 >> 172.21.15.40:0/446686424 conn(0x7f30040cb450 msgr2=0x7f30040cb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.604+0000 7f3009aa3700 1 -- 172.21.15.40:0/446686424 shutdown_connections 2024-09-13T18:22:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.604+0000 7f3009aa3700 1 -- 172.21.15.40:0/446686424 wait complete. 2024-09-13T18:22:50.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f3009aa3700 1 Processor -- start 2024-09-13T18:22:50.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f3009aa3700 1 -- start start 2024-09-13T18:22:50.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f3009aa3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3004066420 0x7f3004148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:50.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f3009aa3700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3004070f80 0x7f30041df180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:50.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f3009aa3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3004071360 0x7f3004154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:50.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f3009aa3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30041eb8e0 con 0x7f3004066420 2024-09-13T18:22:50.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f3009aa3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30041486a0 con 0x7f3004071360 2024-09-13T18:22:50.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f3009aa3700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3004155540 con 0x7f3004070f80 2024-09-13T18:22:50.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f3008aa1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3004066420 0x7f3004148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:50.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f3008aa1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3004066420 0x7f3004148160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48828/0 (socket says 172.21.15.40:48828) 2024-09-13T18:22:50.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f3008aa1700 1 -- 172.21.15.40:0/3754215272 learned_addr learned my addr 172.21.15.40:0/3754215272 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:22:50.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f30092a2700 1 --2- 172.21.15.40:0/3754215272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3004071360 0x7f3004154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:50.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f3003fff700 1 --2- 172.21.15.40:0/3754215272 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3004070f80 0x7f30041df180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:50.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f3008aa1700 1 -- 172.21.15.40:0/3754215272 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3004070f80 msgr2=0x7f30041df180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:50.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f3008aa1700 1 --2- 172.21.15.40:0/3754215272 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3004070f80 0x7f30041df180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:50.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f3008aa1700 1 -- 172.21.15.40:0/3754215272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3004071360 msgr2=0x7f3004154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:50.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f3008aa1700 1 --2- 172.21.15.40:0/3754215272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3004071360 0x7f3004154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:50.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f3008aa1700 1 -- 172.21.15.40:0/3754215272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff000b810 con 0x7f3004066420 2024-09-13T18:22:50.618 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f30092a2700 1 --2- 172.21.15.40:0/3754215272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3004071360 0x7f3004154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:22:50.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f3008aa1700 1 --2- 172.21.15.40:0/3754215272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3004066420 0x7f3004148160 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800c530 tx=0x7f2ff800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:50.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f30017fa700 1 -- 172.21.15.40:0/3754215272 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff80052d0 con 0x7f3004066420 2024-09-13T18:22:50.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f30017fa700 1 -- 172.21.15.40:0/3754215272 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2ff8005430 con 0x7f3004066420 2024-09-13T18:22:50.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f3009aa3700 1 -- 172.21.15.40:0/3754215272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30041eb600 con 0x7f3004066420 2024-09-13T18:22:50.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f30017fa700 1 -- 172.21.15.40:0/3754215272 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff80055a0 con 0x7f3004066420 2024-09-13T18:22:50.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.608+0000 7f3009aa3700 1 -- 172.21.15.40:0/3754215272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30041dec60 con 0x7f3004066420 2024-09-13T18:22:50.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.612+0000 7f3009aa3700 1 -- 172.21.15.40:0/3754215272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30041de7e0 con 0x7f3004066420 2024-09-13T18:22:50.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.612+0000 7f30017fa700 1 -- 172.21.15.40:0/3754215272 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2ff8003900 con 0x7f3004066420 2024-09-13T18:22:50.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.612+0000 7f30017fa700 1 --2- 172.21.15.40:0/3754215272 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2fe8066730 0x7f2fe8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:50.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.612+0000 7f3003fff700 1 --2- 172.21.15.40:0/3754215272 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2fe8066730 0x7f2fe8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:50.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.612+0000 7f30017fa700 1 -- 172.21.15.40:0/3754215272 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f2ff8083ce0 con 0x7f3004066420 2024-09-13T18:22:50.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.612+0000 7f3003fff700 1 --2- 172.21.15.40:0/3754215272 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2fe8066730 0x7f2fe8068bf0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3004071770 tx=0x7f2ff0013000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:50.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.620+0000 7f30017fa700 1 -- 172.21.15.40:0/3754215272 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f30041de7e0 con 0x7f3004066420 2024-09-13T18:22:50.816 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:50.808+0000 7f3009aa3700 1 -- 172.21.15.40:0/3754215272 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f30041de7e0 con 0x7f3004066420 2024-09-13T18:22:51.475 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.468+0000 7f30017fa700 1 -- 172.21.15.40:0/3754215272 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7f30041de7e0 con 0x7f3004066420 2024-09-13T18:22:51.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.472+0000 7f3009aa3700 1 -- 172.21.15.40:0/3754215272 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2fe8066730 msgr2=0x7f2fe8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:51.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.472+0000 7f3009aa3700 1 --2- 172.21.15.40:0/3754215272 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2fe8066730 0x7f2fe8068bf0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3004071770 tx=0x7f2ff0013000 comp rx=0 tx=0).stop 2024-09-13T18:22:51.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.472+0000 7f3009aa3700 1 -- 172.21.15.40:0/3754215272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3004066420 msgr2=0x7f3004148160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:51.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.472+0000 7f3009aa3700 1 --2- 172.21.15.40:0/3754215272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3004066420 0x7f3004148160 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800c530 tx=0x7f2ff800c8f0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.472+0000 7f3009aa3700 1 -- 172.21.15.40:0/3754215272 shutdown_connections 2024-09-13T18:22:51.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.472+0000 7f3009aa3700 1 --2- 172.21.15.40:0/3754215272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3004071360 0x7f3004154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.472+0000 7f3009aa3700 1 --2- 172.21.15.40:0/3754215272 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f2fe8066730 0x7f2fe8068bf0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.472+0000 7f3009aa3700 1 --2- 172.21.15.40:0/3754215272 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3004070f80 0x7f30041df180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.472+0000 7f3009aa3700 1 --2- 172.21.15.40:0/3754215272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3004066420 0x7f3004148160 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.480 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.472+0000 7f3009aa3700 1 -- 172.21.15.40:0/3754215272 >> 172.21.15.40:0/3754215272 conn(0x7f30040cb450 msgr2=0x7f30040cb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:51.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.472+0000 7f3009aa3700 1 -- 172.21.15.40:0/3754215272 shutdown_connections 2024-09-13T18:22:51.481 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.472+0000 7f3009aa3700 1 -- 172.21.15.40:0/3754215272 wait complete. 2024-09-13T18:22:51.481 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2024-09-13T18:22:51.494 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T18:22:51.612 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.604+0000 7f490759e700 1 Processor -- start 2024-09-13T18:22:51.613 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.604+0000 7f490759e700 1 -- start start 2024-09-13T18:22:51.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.604+0000 7f490759e700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4908150590 0x7f49081597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.604+0000 7f490759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49080c2020 0x7f49080c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.604+0000 7f490759e700 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49080c1b90 con 0x7f49080c67e0 2024-09-13T18:22:51.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.604+0000 7f490759e700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49080c82d0 con 0x7f4908150590 2024-09-13T18:22:51.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.604+0000 7f490759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4908159cf0 con 0x7f49080c2020 2024-09-13T18:22:51.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.604+0000 7f490659c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4908150590 0x7f49081597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.604+0000 7f4906d9d700 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f49080c67e0 0x7f49080c7150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:59990/0 (socket says 172.21.15.40:59990) 2024-09-13T18:22:51.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.604+0000 7f490659c700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4908150590 0x7f49081597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39530/0 (socket says 172.21.15.40:39530) 2024-09-13T18:22:51.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f4906d9d700 1 -- 172.21.15.40:0/1793730310 learned_addr learned my addr 172.21.15.40:0/1793730310 (peer_addr_for_me v1:172.21.15.40:0/0) 2024-09-13T18:22:51.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f4905d9b700 1 --2- 172.21.15.40:0/1793730310 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49080c2020 0x7f49080c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490659c700 1 -- 172.21.15.40:0/1793730310 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49080c2020 msgr2=0x7f49080c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:51.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490659c700 1 --2- 172.21.15.40:0/1793730310 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49080c2020 0x7f49080c9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490659c700 1 -- 172.21.15.40:0/1793730310 >> v1:172.21.15.40:6789/0 conn(0x7f49080c67e0 legacy=0x7f49080c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:51.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490659c700 1 -- 172.21.15.40:0/1793730310 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490806e510 con 0x7f4908150590 2024-09-13T18:22:51.615 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490659c700 1 --2- 172.21.15.40:0/1793730310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4908150590 0x7f49081597b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009850 tx=0x7f48f8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960876a7332c685c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:51.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490559a700 1 -- 172.21.15.40:0/1793730310 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8011830 con 0x7f4908150590 2024-09-13T18:22:51.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490559a700 1 -- 172.21.15.40:0/1793730310 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f48f8011990 con 0x7f4908150590 2024-09-13T18:22:51.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490559a700 1 -- 172.21.15.40:0/1793730310 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8011d40 con 0x7f4908150590 2024-09-13T18:22:51.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490759e700 1 -- 172.21.15.40:0/1793730310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4908150590 msgr2=0x7f49081597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:51.616 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490759e700 1 --2- 172.21.15.40:0/1793730310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4908150590 0x7f49081597b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009850 tx=0x7f48f8009b60 comp rx=0 tx=0).stop 2024-09-13T18:22:51.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490759e700 1 -- 172.21.15.40:0/1793730310 shutdown_connections 2024-09-13T18:22:51.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490759e700 1 --2- 172.21.15.40:0/1793730310 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49080c2020 0x7f49080c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490759e700 1 --2- 172.21.15.40:0/1793730310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4908150590 0x7f49081597b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490759e700 1 -- 172.21.15.40:0/1793730310 >> 172.21.15.40:0/1793730310 conn(0x7f49080c88e0 msgr2=0x7f49080c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:51.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490759e700 1 -- 172.21.15.40:0/1793730310 shutdown_connections 2024-09-13T18:22:51.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490759e700 1 -- 172.21.15.40:0/1793730310 wait complete. 2024-09-13T18:22:51.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490759e700 1 Processor -- start 2024-09-13T18:22:51.617 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490759e700 1 -- start start 2024-09-13T18:22:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490759e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49080c2020 0x7f49080b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490759e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49080c67e0 0x7f490814bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4908150590 0x7f4908157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490759e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4908157540 con 0x7f49080c67e0 2024-09-13T18:22:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49081585e0 con 0x7f4908150590 2024-09-13T18:22:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490759e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49081588c0 con 0x7f49080c2020 2024-09-13T18:22:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f4905d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49080c67e0 0x7f490814bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f4905d9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49080c67e0 0x7f490814bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39544/0 (socket says 172.21.15.40:39544) 2024-09-13T18:22:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f4905d9b700 1 -- 172.21.15.40:0/2807483092 learned_addr learned my addr 172.21.15.40:0/2807483092 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:22:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f490659c700 1 --2- 172.21.15.40:0/2807483092 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49080c2020 0x7f49080b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.608+0000 7f4906d9d700 1 --2- 172.21.15.40:0/2807483092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4908150590 0x7f4908157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.619 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.612+0000 7f4905d9b700 1 -- 172.21.15.40:0/2807483092 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49080c2020 msgr2=0x7f49080b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.612+0000 7f4905d9b700 1 --2- 172.21.15.40:0/2807483092 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49080c2020 0x7f49080b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.612+0000 7f4905d9b700 1 -- 172.21.15.40:0/2807483092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4908150590 msgr2=0x7f4908157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.612+0000 7f4905d9b700 1 --2- 172.21.15.40:0/2807483092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4908150590 0x7f4908157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.612+0000 7f4905d9b700 1 -- 172.21.15.40:0/2807483092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f80094b0 con 0x7f49080c67e0 2024-09-13T18:22:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.612+0000 7f4905d9b700 1 --2- 172.21.15.40:0/2807483092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49080c67e0 0x7f490814bac0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00da20 tx=0x7f48fc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.612+0000 7f48f6ffd700 1 -- 172.21.15.40:0/2807483092 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc004220 con 0x7f49080c67e0 2024-09-13T18:22:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.612+0000 7f48f6ffd700 1 -- 172.21.15.40:0/2807483092 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f48fc004380 con 0x7f49080c67e0 2024-09-13T18:22:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.612+0000 7f48f6ffd700 1 -- 172.21.15.40:0/2807483092 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc00a890 con 0x7f49080c67e0 2024-09-13T18:22:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.612+0000 7f490759e700 1 -- 172.21.15.40:0/2807483092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4908158ba0 con 0x7f49080c67e0 2024-09-13T18:22:51.620 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.612+0000 7f490759e700 1 -- 172.21.15.40:0/2807483092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490814ce10 con 0x7f49080c67e0 2024-09-13T18:22:51.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.612+0000 7f490759e700 1 -- 172.21.15.40:0/2807483092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49080c1820 con 0x7f49080c67e0 2024-09-13T18:22:51.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.612+0000 7f48f6ffd700 1 -- 172.21.15.40:0/2807483092 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f48fc00a9f0 con 0x7f49080c67e0 2024-09-13T18:22:51.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.612+0000 7f48f6ffd700 1 --2- 172.21.15.40:0/2807483092 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f48e8066730 0x7f48e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:51.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.612+0000 7f48f6ffd700 1 -- 172.21.15.40:0/2807483092 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f48fc084600 con 0x7f49080c67e0 2024-09-13T18:22:51.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.612+0000 7f490659c700 1 --2- 172.21.15.40:0/2807483092 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f48e8066730 0x7f48e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:51.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.612+0000 7f490659c700 1 --2- 172.21.15.40:0/2807483092 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f48e8066730 0x7f48e8068bf0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f48f8014040 tx=0x7f48f8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:51.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.616+0000 7f48f6ffd700 1 -- 172.21.15.40:0/2807483092 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f48fc0515d0 con 0x7f49080c67e0 2024-09-13T18:22:51.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:51.788+0000 7f490759e700 1 -- 172.21.15.40:0/2807483092 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f49080ca3a0 con 0x7f49080c67e0 2024-09-13T18:22:52.485 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.476+0000 7f48f6ffd700 1 -- 172.21.15.40:0/2807483092 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7f48fc054bf0 con 0x7f49080c67e0 2024-09-13T18:22:52.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.500+0000 7f490759e700 1 -- 172.21.15.40:0/2807483092 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f48e8066730 msgr2=0x7f48e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:52.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.500+0000 7f490759e700 1 --2- 172.21.15.40:0/2807483092 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f48e8066730 0x7f48e8068bf0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f48f8014040 tx=0x7f48f8008030 comp rx=0 tx=0).stop 2024-09-13T18:22:52.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.500+0000 7f490759e700 1 -- 172.21.15.40:0/2807483092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49080c67e0 msgr2=0x7f490814bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:52.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.500+0000 7f490759e700 1 --2- 172.21.15.40:0/2807483092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49080c67e0 0x7f490814bac0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00da20 tx=0x7f48fc00dde0 comp rx=0 tx=0).stop 2024-09-13T18:22:52.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.500+0000 7f490759e700 1 -- 172.21.15.40:0/2807483092 shutdown_connections 2024-09-13T18:22:52.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.500+0000 7f490759e700 1 --2- 172.21.15.40:0/2807483092 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4908150590 0x7f4908157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:52.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.500+0000 7f490759e700 1 --2- 172.21.15.40:0/2807483092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49080c67e0 0x7f490814bac0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:52.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.500+0000 7f490759e700 1 --2- 172.21.15.40:0/2807483092 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f48e8066730 0x7f48e8068bf0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:52.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.500+0000 7f490759e700 1 --2- 172.21.15.40:0/2807483092 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f49080c2020 0x7f49080b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:52.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.500+0000 7f490759e700 1 -- 172.21.15.40:0/2807483092 >> 172.21.15.40:0/2807483092 conn(0x7f49080c88e0 msgr2=0x7f49080c24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:52.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.500+0000 7f490759e700 1 -- 172.21.15.40:0/2807483092 shutdown_connections 2024-09-13T18:22:52.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.500+0000 7f490759e700 1 -- 172.21.15.40:0/2807483092 wait complete. 2024-09-13T18:22:52.509 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2024-09-13T18:22:52.520 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-13T18:22:52.627 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.620+0000 7f69d2aa7700 1 Processor -- start 2024-09-13T18:22:52.628 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.620+0000 7f69d2aa7700 1 -- start start 2024-09-13T18:22:52.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.620+0000 7f69d2aa7700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f69cc151490 0x7f69cc157b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:52.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.620+0000 7f69d2aa7700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f69cc0b1d60 0x7f69cc06efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:52.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.620+0000 7f69d2aa7700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69cc151870 con 0x7f69cc0c67e0 2024-09-13T18:22:52.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.620+0000 7f69d2aa7700 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69cc0c7530 con 0x7f69cc0b1d60 2024-09-13T18:22:52.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.620+0000 7f69d2aa7700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69cc157170 con 0x7f69cc151490 2024-09-13T18:22:52.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.620+0000 7f69d12a4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f69cc0b1d60 0x7f69cc06efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:52.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.620+0000 7f69d12a4700 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f69cc0b1d60 0x7f69cc06efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39554/0 (socket says 172.21.15.40:39554) 2024-09-13T18:22:52.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.620+0000 7f69d12a4700 1 -- 172.21.15.40:0/1221712202 learned_addr learned my addr 172.21.15.40:0/1221712202 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:22:52.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.620+0000 7f69d1aa5700 1 --2- 172.21.15.40:0/1221712202 >> v2:172.21.15.43:3300/0 conn(0x7f69cc151490 0x7f69cc157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:52.630 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.620+0000 7f69d12a4700 1 -- 172.21.15.40:0/1221712202 >> v2:172.21.15.43:3300/0 conn(0x7f69cc151490 msgr2=0x7f69cc157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:52.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.620+0000 7f69d12a4700 1 --2- 172.21.15.40:0/1221712202 >> v2:172.21.15.43:3300/0 conn(0x7f69cc151490 0x7f69cc157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:52.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.620+0000 7f69d12a4700 1 -- 172.21.15.40:0/1221712202 >> v1:172.21.15.43:6789/0 conn(0x7f69cc0c67e0 legacy=0x7f69cc06a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:22:52.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.620+0000 7f69d12a4700 1 -- 172.21.15.40:0/1221712202 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69cc0c6cb0 con 0x7f69cc0b1d60 2024-09-13T18:22:52.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.624+0000 7f69d12a4700 1 --2- 172.21.15.40:0/1221712202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69cc0b1d60 0x7f69cc06efc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f69cc06ddf0 tx=0x7f69bc009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41bf3635da411cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:52.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.624+0000 7f69d0aa3700 1 -- 172.21.15.40:0/1221712202 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc004230 con 0x7f69cc0b1d60 2024-09-13T18:22:52.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.624+0000 7f69d0aa3700 1 -- 172.21.15.40:0/1221712202 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f69bc004390 con 0x7f69cc0b1d60 2024-09-13T18:22:52.631 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.624+0000 7f69d0aa3700 1 -- 172.21.15.40:0/1221712202 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc0036c0 con 0x7f69cc0b1d60 2024-09-13T18:22:52.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.624+0000 7f69d2aa7700 1 -- 172.21.15.40:0/1221712202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69cc0b1d60 msgr2=0x7f69cc06efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:52.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.624+0000 7f69d2aa7700 1 --2- 172.21.15.40:0/1221712202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69cc0b1d60 0x7f69cc06efc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f69cc06ddf0 tx=0x7f69bc009bf0 comp rx=0 tx=0).stop 2024-09-13T18:22:52.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.624+0000 7f69d2aa7700 1 -- 172.21.15.40:0/1221712202 shutdown_connections 2024-09-13T18:22:52.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.624+0000 7f69d2aa7700 1 --2- 172.21.15.40:0/1221712202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69cc0b1d60 0x7f69cc06efc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:52.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.624+0000 7f69d2aa7700 1 --2- 172.21.15.40:0/1221712202 >> v2:172.21.15.43:3300/0 conn(0x7f69cc151490 0x7f69cc157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:52.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.624+0000 7f69d2aa7700 1 -- 172.21.15.40:0/1221712202 >> 172.21.15.40:0/1221712202 conn(0x7f69cc06e4a0 msgr2=0x7f69cc06e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:52.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.624+0000 7f69d2aa7700 1 -- 172.21.15.40:0/1221712202 shutdown_connections 2024-09-13T18:22:52.632 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.624+0000 7f69d2aa7700 1 -- 172.21.15.40:0/1221712202 wait complete. 2024-09-13T18:22:52.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.624+0000 7f69d2aa7700 1 Processor -- start 2024-09-13T18:22:52.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.624+0000 7f69d2aa7700 1 -- start start 2024-09-13T18:22:52.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69d2aa7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69cc0b1d60 0x7f69cc153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:52.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69d2aa7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69cc0c67e0 0x7f69cc1e2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:52.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69d2aa7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69cc151490 0x7f69cc1e07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:52.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69d2aa7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc154f10 con 0x7f69cc0c67e0 2024-09-13T18:22:52.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69d2aa7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc1550f0 con 0x7f69cc0b1d60 2024-09-13T18:22:52.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69d2aa7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc155670 con 0x7f69cc151490 2024-09-13T18:22:52.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69d12a4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69cc0c67e0 0x7f69cc1e2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:52.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69d12a4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69cc0c67e0 0x7f69cc1e2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39562/0 (socket says 172.21.15.40:39562) 2024-09-13T18:22:52.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69d12a4700 1 -- 172.21.15.40:0/1676689694 learned_addr learned my addr 172.21.15.40:0/1676689694 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-09-13T18:22:52.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69d1aa5700 1 --2- 172.21.15.40:0/1676689694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69cc0b1d60 0x7f69cc153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:52.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69d22a6700 1 --2- 172.21.15.40:0/1676689694 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69cc151490 0x7f69cc1e07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:52.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69d12a4700 1 -- 172.21.15.40:0/1676689694 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69cc151490 msgr2=0x7f69cc1e07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:52.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69d12a4700 1 --2- 172.21.15.40:0/1676689694 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69cc151490 0x7f69cc1e07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:52.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69d12a4700 1 -- 172.21.15.40:0/1676689694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69cc0b1d60 msgr2=0x7f69cc153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:52.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69d12a4700 1 --2- 172.21.15.40:0/1676689694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69cc0b1d60 0x7f69cc153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:52.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69d12a4700 1 -- 172.21.15.40:0/1676689694 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69bc009570 con 0x7f69cc0c67e0 2024-09-13T18:22:52.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69d12a4700 1 --2- 172.21.15.40:0/1676689694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69cc0c67e0 0x7f69cc1e2780 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f69bc004000 tx=0x7f69bc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69c27fc700 1 -- 172.21.15.40:0/1676689694 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc003ca0 con 0x7f69cc0c67e0 2024-09-13T18:22:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69c27fc700 1 -- 172.21.15.40:0/1676689694 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f69bc01f070 con 0x7f69cc0c67e0 2024-09-13T18:22:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69d2aa7700 1 -- 172.21.15.40:0/1676689694 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69cc155390 con 0x7f69cc0c67e0 2024-09-13T18:22:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69c27fc700 1 -- 172.21.15.40:0/1676689694 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc00b8c0 con 0x7f69cc0c67e0 2024-09-13T18:22:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.628+0000 7f69d2aa7700 1 -- 172.21.15.40:0/1676689694 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69cc1e36c0 con 0x7f69cc0c67e0 2024-09-13T18:22:52.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.632+0000 7f69d2aa7700 1 -- 172.21.15.40:0/1676689694 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69cc072770 con 0x7f69cc0c67e0 2024-09-13T18:22:52.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.632+0000 7f69c27fc700 1 -- 172.21.15.40:0/1676689694 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f69bc021030 con 0x7f69cc0c67e0 2024-09-13T18:22:52.638 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.632+0000 7f69c27fc700 1 --2- 172.21.15.40:0/1676689694 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f69b0066780 0x7f69b0068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:22:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.632+0000 7f69d1aa5700 1 --2- 172.21.15.40:0/1676689694 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f69b0066780 0x7f69b0068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:22:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.632+0000 7f69c27fc700 1 -- 172.21.15.40:0/1676689694 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5953+0+0 (secure 0 0 0) 0x7f69bc0856b0 con 0x7f69cc0c67e0 2024-09-13T18:22:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.632+0000 7f69d1aa5700 1 --2- 172.21.15.40:0/1676689694 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f69b0066780 0x7f69b0068c40 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f69c8007a60 tx=0x7f69c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:22:52.646 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.640+0000 7f69c27fc700 1 -- 172.21.15.40:0/1676689694 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f69bc052a90 con 0x7f69cc0c67e0 2024-09-13T18:22:52.814 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:52.804+0000 7f69d2aa7700 1 -- 172.21.15.40:0/1676689694 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f69cc072770 con 0x7f69cc0c67e0 2024-09-13T18:22:53.494 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:53.488+0000 7f69c27fc700 1 -- 172.21.15.40:0/1676689694 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) v1 ==== 178+0+0 (secure 0 0 0) 0x7f69bc0560b0 con 0x7f69cc0c67e0 2024-09-13T18:22:53.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:53.492+0000 7f69d2aa7700 1 -- 172.21.15.40:0/1676689694 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f69b0066780 msgr2=0x7f69b0068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:53.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:53.492+0000 7f69d2aa7700 1 --2- 172.21.15.40:0/1676689694 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f69b0066780 0x7f69b0068c40 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f69c8007a60 tx=0x7f69c8008040 comp rx=0 tx=0).stop 2024-09-13T18:22:53.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:53.492+0000 7f69d2aa7700 1 -- 172.21.15.40:0/1676689694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69cc0c67e0 msgr2=0x7f69cc1e2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:22:53.502 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:53.492+0000 7f69d2aa7700 1 --2- 172.21.15.40:0/1676689694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69cc0c67e0 0x7f69cc1e2780 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f69bc004000 tx=0x7f69bc004770 comp rx=0 tx=0).stop 2024-09-13T18:22:53.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:53.500+0000 7f69d2aa7700 1 -- 172.21.15.40:0/1676689694 shutdown_connections 2024-09-13T18:22:53.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:53.500+0000 7f69d2aa7700 1 --2- 172.21.15.40:0/1676689694 >> [v2:172.21.15.40:6832/3394537609,v1:172.21.15.40:6833/3394537609] conn(0x7f69b0066780 0x7f69b0068c40 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:53.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:53.500+0000 7f69d2aa7700 1 --2- 172.21.15.40:0/1676689694 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f69cc151490 0x7f69cc1e07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:53.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:53.500+0000 7f69d2aa7700 1 --2- 172.21.15.40:0/1676689694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69cc0c67e0 0x7f69cc1e2780 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:53.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:53.500+0000 7f69d2aa7700 1 --2- 172.21.15.40:0/1676689694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69cc0b1d60 0x7f69cc153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:22:53.507 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:53.500+0000 7f69d2aa7700 1 -- 172.21.15.40:0/1676689694 >> 172.21.15.40:0/1676689694 conn(0x7f69cc06e4a0 msgr2=0x7f69cc06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:22:53.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:53.500+0000 7f69d2aa7700 1 -- 172.21.15.40:0/1676689694 shutdown_connections 2024-09-13T18:22:53.509 INFO:teuthology.orchestra.run.smithi040.stderr:2024-09-13T18:22:53.500+0000 7f69d2aa7700 1 -- 172.21.15.40:0/1676689694 wait complete. 2024-09-13T18:22:53.514 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data_ec' removed 2024-09-13T18:22:53.528 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T18:22:53.529 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T18:22:53.529 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:22:53.540 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T18:22:53.541 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T18:22:53.541 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:22:53.553 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T18:22:53.553 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-09-13T18:22:53.553 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:22:53.572 INFO:tasks.ceph.mds.e:Stopped 2024-09-13T18:22:53.573 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T18:22:53.573 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:22:53.591 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T18:22:53.591 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T18:22:53.591 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:22:53.602 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T18:22:53.602 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-09-13T18:22:53.603 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:22:53.615 INFO:tasks.ceph.mds.f:Stopped 2024-09-13T18:22:53.615 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T18:22:53.616 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T18:22:53.616 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:22:59.510 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-13T18:22:59.718 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T18:22:59.718 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T18:22:59.718 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:23:05.619 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-13T18:23:05.619 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-13T18:23:05.821 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T18:23:05.821 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T18:23:05.821 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:23:11.628 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-13T18:23:11.629 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-13T18:23:11.923 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T18:23:11.924 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T18:23:11.924 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:23:17.536 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-13T18:23:17.537 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-13T18:23:18.026 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T18:23:18.026 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-13T18:23:18.027 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:23:23.344 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-09-13T18:23:23.344 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-13T18:23:24.128 INFO:tasks.ceph.osd.4:Stopped 2024-09-13T18:23:24.128 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-13T18:23:24.128 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:23:29.051 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-13T18:23:29.051 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-09-13T18:23:30.230 INFO:tasks.ceph.osd.5:Stopped 2024-09-13T18:23:30.231 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-13T18:23:30.231 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:23:34.656 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-13T18:23:34.656 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-13T18:23:36.333 INFO:tasks.ceph.osd.6:Stopped 2024-09-13T18:23:36.334 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-13T18:23:36.334 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:23:40.161 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-13T18:23:40.161 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-13T18:23:41.798 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:23:41.801 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:23:41.828 INFO:teuthology.orchestra.run.smithi040.stderr:ceph-osd: no process found 2024-09-13T18:23:41.842 INFO:teuthology.orchestra.run.smithi043.stderr:ceph-osd: no process found 2024-09-13T18:23:42.435 INFO:tasks.ceph.osd.7:Stopped 2024-09-13T18:23:42.436 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T18:23:42.436 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-13T18:23:42.436 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:23:42.468 INFO:tasks.ceph.mgr.y:Stopped 2024-09-13T18:23:42.468 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T18:23:42.468 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:23:42.500 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T18:23:42.500 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T18:23:42.500 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T18:23:42.501 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:23:42.541 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T18:23:42.541 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T18:23:42.541 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:23:42.580 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T18:23:42.581 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T18:23:42.581 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:23:42.613 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T18:23:42.613 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T18:23:42.613 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-13T18:23:42.763 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi040.front.sepia.ceph.com 2024-09-13T18:23:42.763 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T18:23:43.147 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi040.front.sepia.ceph.com 2024-09-13T18:23:43.148 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T18:23:43.432 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi040.front.sepia.ceph.com 2024-09-13T18:23:43.432 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T18:23:43.706 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi040.front.sepia.ceph.com 2024-09-13T18:23:43.706 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-13T18:23:43.976 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-13T18:23:43.977 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-13T18:23:44.475 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-13T18:23:44.475 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-13T18:23:44.741 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-13T18:23:44.741 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-13T18:23:44.980 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi043.front.sepia.ceph.com 2024-09-13T18:23:44.980 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-13T18:23:45.162 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-13T18:23:45.228 INFO:tasks.ceph:Archiving mon data... 2024-09-13T18:23:45.229 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903931/data/mon.a.tgz 2024-09-13T18:23:45.230 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2024-09-13T18:23:45.264 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.uPcuCGCLkI 2024-09-13T18:23:45.264 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.uPcuCGCLkI 2024-09-13T18:23:45.432 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.uPcuCGCLkI 2024-09-13T18:23:45.546 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.uPcuCGCLkI is 477KB 2024-09-13T18:23:45.612 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.uPcuCGCLkI 2024-09-13T18:23:45.619 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903931/data/mon.b.tgz 2024-09-13T18:23:45.619 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-09-13T18:23:45.665 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.IAEeFF3xS5 2024-09-13T18:23:45.666 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.IAEeFF3xS5 2024-09-13T18:23:45.800 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.IAEeFF3xS5 2024-09-13T18:23:45.862 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.IAEeFF3xS5 is 482KB 2024-09-13T18:23:45.932 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.IAEeFF3xS5 2024-09-13T18:23:45.955 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903931/data/mon.c.tgz 2024-09-13T18:23:45.956 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-09-13T18:23:45.962 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.OqtHucgSQ7 2024-09-13T18:23:45.963 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.OqtHucgSQ7 2024-09-13T18:23:46.066 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.OqtHucgSQ7 2024-09-13T18:23:46.086 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.OqtHucgSQ7 is 482KB 2024-09-13T18:23:46.152 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.OqtHucgSQ7 2024-09-13T18:23:46.158 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T18:23:46.158 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T18:23:46.160 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T18:23:46.185 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-13T18:23:46.186 DEBUG:teuthology.orchestra.run.smithi040:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T18:23:46.227 DEBUG:teuthology.orchestra.run.smithi043:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T18:23:46.263 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T18:23:46.321 INFO:teuthology.orchestra.run.smithi043.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T18:23:46.327 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T18:23:46.329 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903931/remote/smithi040/crash 2024-09-13T18:23:46.329 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T18:23:46.347 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903931/remote/smithi043/crash 2024-09-13T18:23:46.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T18:23:46.364 INFO:tasks.ceph:Shutting down logrotate 2024-09-13T18:26:49.741 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-13T18:26:49.771 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-13T18:26:49.796 INFO:tasks.ceph:Compressing logs... 2024-09-13T18:26:49.796 DEBUG:teuthology.orchestra.run.smithi040:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T18:26:49.814 DEBUG:teuthology.orchestra.run.smithi043:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T18:34:22.604 INFO:tasks.ceph:Archiving logs... 2024-09-13T18:34:22.605 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903931/remote/smithi040/log 2024-09-13T18:34:22.606 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T18:35:45.658 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903931/remote/smithi043/log 2024-09-13T18:35:45.659 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T18:36:46.616 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T18:36:46.650 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T18:36:46.650 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T18:36:46.653 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T18:36:46.752 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T18:36:46.761 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T18:36:46.761 DEBUG:teuthology.orchestra.run.smithi040:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T18:36:46.763 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T18:36:46.821 INFO:teuthology.orchestra.run.smithi040.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T18:36:46.821 INFO:teuthology.orchestra.run.smithi040.stdout:============================================================================== 2024-09-13T18:36:46.821 INFO:teuthology.orchestra.run.smithi040.stdout:*hv01.front.sepi 67.205.162.81 3 u 17 128 377 0.103 3.807 0.512 2024-09-13T18:36:46.821 INFO:teuthology.orchestra.run.smithi040.stdout:+hv02.front.sepi 63.231.80.2 3 u 78 128 377 0.120 0.426 0.852 2024-09-13T18:36:46.821 INFO:teuthology.orchestra.run.smithi040.stdout:+hv03.front.sepi 74.6.168.72 3 u 134 128 377 0.091 4.333 0.639 2024-09-13T18:36:46.821 INFO:teuthology.orchestra.run.smithi040.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2024-09-13T18:36:46.846 INFO:teuthology.orchestra.run.smithi043.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T18:36:46.846 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================== 2024-09-13T18:36:46.846 INFO:teuthology.orchestra.run.smithi043.stdout:*hv01.front.sepi 67.205.162.81 3 u 13 64 377 0.093 3.330 0.636 2024-09-13T18:36:46.846 INFO:teuthology.orchestra.run.smithi043.stdout:+hv02.front.sepi 63.231.80.2 3 u 13 64 377 0.103 -1.351 0.518 2024-09-13T18:36:46.846 INFO:teuthology.orchestra.run.smithi043.stdout:+hv03.front.sepi 74.6.168.72 3 u 15 64 377 0.104 2.876 0.402 2024-09-13T18:36:46.846 INFO:teuthology.orchestra.run.smithi043.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2024-09-13T18:36:46.847 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T18:36:46.857 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T18:36:46.857 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T18:36:46.867 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T18:36:46.879 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T18:36:46.894 INFO:teuthology.task.internal:Duration was 3380.529355 seconds 2024-09-13T18:36:46.894 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T18:36:46.904 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T18:36:46.904 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T18:36:46.907 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T18:36:47.074 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T18:36:47.075 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi040.front.sepia.ceph.com 2024-09-13T18:36:47.075 DEBUG:teuthology.orchestra.run.smithi040:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T18:36:47.090 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2024-09-13T18:36:47.090 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T18:36:47.106 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T18:36:47.106 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T18:36:47.109 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T18:36:47.188 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T18:36:47.188 DEBUG:teuthology.orchestra.run.smithi040:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T18:36:47.191 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T18:36:47.446 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T18:36:47.457 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T18:36:47.458 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T18:36:47.475 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T18:36:47.500 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T18:36:47.511 DEBUG:teuthology.orchestra.run.smithi040:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T18:36:47.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T18:36:47.537 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = core 2024-09-13T18:36:47.571 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2024-09-13T18:36:47.617 DEBUG:teuthology.orchestra.run.smithi040:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T18:36:47.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:36:47.624 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T18:36:47.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:36:47.631 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T18:36:47.645 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T18:36:47.646 DEBUG:teuthology.misc:Transferring archived files from smithi040:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903931/remote/smithi040 2024-09-13T18:36:47.646 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T18:36:47.674 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903931/remote/smithi043 2024-09-13T18:36:47.674 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T18:36:47.698 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T18:36:47.699 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T18:36:47.715 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T18:36:47.747 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T18:36:47.758 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T18:36:47.758 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T18:36:47.770 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T18:36:47.770 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T18:36:47.772 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T18:36:47.778 INFO:teuthology.orchestra.run.smithi040.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 18:36 /home/ubuntu/cephtest 2024-09-13T18:36:47.780 INFO:teuthology.orchestra.run.smithi043.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 18:36 /home/ubuntu/cephtest 2024-09-13T18:36:47.794 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T18:36:47.806 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T18:36:47.833 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} duration: 3380.529354572296 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-13T18:36:47.833 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T18:36:47.946 INFO:teuthology.run:pass